2024-02-17T12:28:26.744 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-17T12:28:26.744 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --verbose --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187 -- /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187/orig.config.yaml 2024-02-17T12:28:26.801 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-17T12:28:26.893 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187 branch: main description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: yweinste@redhat.com first_in_suite: false job_id: '7563187' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 6822c7b54e949006be7a0802650d01723d834d1d subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 6822c7b54e949006be7a0802650d01723d834d1d kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 6822c7b54e949006be7a0802650d01723d834d1d owner: scheduled_yuriw@teuthology priority: 51 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2794 sha1: 6822c7b54e949006be7a0802650d01723d834d1d sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 6822c7b54e949006be7a0802650d01723d834d1d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi018.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLObfZd/saF9BH2sXCdZad6yo1Qc46W/b4Kp+fhPkbWbn7NvGrcngqhUDdBXoehoNgFLsAotJRGuzcsZvsROwTo= smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE4Pik3gsxWbyQqQD788uEvHLv4KWovWW+R9FZ+VPWNTAqiu1BIcBFv6sAJeLMEEw9RNDh8rxoWubGCgVrPjfHE= smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYHYO1lPAwXnyp9sfjwl3DRRzg3MVYz/LkR3WLLC0DQtZKNq+tc8V8bJI97n4Xsjj4miJ9KNYZwEAtf8PnfP7I= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-16_20:13:25 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-17T12:28:26.893 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa; will attempt to use it 2024-02-17T12:28:26.894 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks 2024-02-17T12:28:26.894 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-17T12:28:26.896 INFO:teuthology.task.internal:Checking packages... 2024-02-17T12:28:26.919 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6822c7b54e949006be7a0802650d01723d834d1d' 2024-02-17T12:28:26.919 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-17T12:28:26.919 INFO:teuthology.packaging:ref: None 2024-02-17T12:28:26.919 INFO:teuthology.packaging:tag: None 2024-02-17T12:28:26.919 INFO:teuthology.packaging:branch: main 2024-02-17T12:28:26.919 INFO:teuthology.packaging:sha1: 6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:28:26.919 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-02-17T12:28:27.310 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1543.g0ed764e6 2024-02-17T12:28:27.311 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-17T12:28:27.318 INFO:teuthology.task.internal:no buildpackages task found 2024-02-17T12:28:27.318 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-17T12:28:27.362 INFO:teuthology.task.internal:Saving configuration 2024-02-17T12:28:27.383 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-17T12:28:27.389 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-17T12:28:27.414 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi018.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 12:23:24.230014', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLObfZd/saF9BH2sXCdZad6yo1Qc46W/b4Kp+fhPkbWbn7NvGrcngqhUDdBXoehoNgFLsAotJRGuzcsZvsROwTo='} 2024-02-17T12:28:27.434 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 12:23:24.228797', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE4Pik3gsxWbyQqQD788uEvHLv4KWovWW+R9FZ+VPWNTAqiu1BIcBFv6sAJeLMEEw9RNDh8rxoWubGCgVrPjfHE='} 2024-02-17T12:28:27.456 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 12:23:24.231077', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYHYO1lPAwXnyp9sfjwl3DRRzg3MVYz/LkR3WLLC0DQtZKNq+tc8V8bJI97n4Xsjj4miJ9KNYZwEAtf8PnfP7I='} 2024-02-17T12:28:27.456 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-17T12:28:27.462 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-02-17T12:28:27.463 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-02-17T12:28:27.463 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-02-17T12:28:27.463 INFO:teuthology.run_tasks:Running task console_log... 2024-02-17T12:28:27.583 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc84cb2f160>, signals=[15]) 2024-02-17T12:28:27.583 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-17T12:28:27.589 INFO:teuthology.task.internal:Opening connections... 2024-02-17T12:28:27.589 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi018.front.sepia.ceph.com 2024-02-17T12:28:27.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T12:28:27.695 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-02-17T12:28:27.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T12:28:27.783 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2024-02-17T12:28:27.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T12:28:27.871 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-17T12:28:27.879 DEBUG:teuthology.orchestra.run.smithi018:> uname -m 2024-02-17T12:28:27.898 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2024-02-17T12:28:27.898 DEBUG:teuthology.orchestra.run.smithi018:> cat /etc/os-release 2024-02-17T12:28:27.956 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Stream" 2024-02-17T12:28:27.956 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="9" 2024-02-17T12:28:27.956 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos" 2024-02-17T12:28:27.956 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora" 2024-02-17T12:28:27.956 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="9" 2024-02-17T12:28:27.956 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el9" 2024-02-17T12:28:27.956 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-17T12:28:27.956 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31" 2024-02-17T12:28:27.956 INFO:teuthology.orchestra.run.smithi018.stdout:LOGO="fedora-logo-icon" 2024-02-17T12:28:27.956 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-17T12:28:27.956 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://centos.org/" 2024-02-17T12:28:27.957 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-17T12:28:27.957 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-17T12:28:27.957 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-17T12:28:27.957 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2024-02-17T12:28:27.978 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-02-17T12:28:27.996 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-02-17T12:28:27.996 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-02-17T12:28:28.054 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2024-02-17T12:28:28.054 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2024-02-17T12:28:28.054 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2024-02-17T12:28:28.055 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2024-02-17T12:28:28.055 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2024-02-17T12:28:28.055 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2024-02-17T12:28:28.055 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-17T12:28:28.055 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-02-17T12:28:28.055 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2024-02-17T12:28:28.055 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-17T12:28:28.055 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2024-02-17T12:28:28.056 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-17T12:28:28.056 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-17T12:28:28.056 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-17T12:28:28.056 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-02-17T12:28:28.087 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2024-02-17T12:28:28.104 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2024-02-17T12:28:28.104 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-02-17T12:28:28.161 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2024-02-17T12:28:28.161 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9" 2024-02-17T12:28:28.161 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2024-02-17T12:28:28.162 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2024-02-17T12:28:28.162 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9" 2024-02-17T12:28:28.162 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9" 2024-02-17T12:28:28.162 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-17T12:28:28.162 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2024-02-17T12:28:28.162 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon" 2024-02-17T12:28:28.162 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-17T12:28:28.162 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2024-02-17T12:28:28.162 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-17T12:28:28.162 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-17T12:28:28.162 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-17T12:28:28.163 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-02-17T12:28:28.186 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-17T12:28:28.197 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-17T12:28:28.220 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-17T12:28:28.220 DEBUG:teuthology.orchestra.run.smithi018:> test '!' -e /home/ubuntu/cephtest 2024-02-17T12:28:28.223 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-02-17T12:28:28.225 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2024-02-17T12:28:28.241 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-17T12:28:28.248 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-17T12:28:28.248 DEBUG:teuthology.orchestra.run.smithi018:> test -z $(ls -A /var/lib/ceph) 2024-02-17T12:28:28.280 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-02-17T12:28:28.282 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2024-02-17T12:28:28.382 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-17T12:28:28.455 INFO:teuthology.run_tasks:Running task kernel... 2024-02-17T12:28:28.470 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-02-17T12:28:28.470 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-02-17T12:28:28.470 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-02-17T12:28:28.470 DEBUG:teuthology.orchestra.run.smithi018:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-17T12:28:28.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:28:28.487 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2024-02-17T12:28:28.545 INFO:teuthology.orchestra.run.smithi018.stdout:5.14.0-361.el9.x86_64 2024-02-17T12:28:28.545 INFO:teuthology.task.kernel:Running kernel on smithi018: 5.14.0-361.el9.x86_64 2024-02-17T12:28:28.545 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum install -y kernel 2024-02-17T12:28:31.034 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-9 - AppStream 7.6 MB/s | 6.7 MB 00:00 2024-02-17T12:28:32.500 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-02-17T12:28:33.374 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-02-17T12:28:35.259 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-02-17T12:28:40.293 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.1 MB 00:01 2024-02-17T12:28:42.003 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2024-02-17T12:28:44.103 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:01 2024-02-17T12:28:49.270 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-02-17T12:28:51.660 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-17T12:28:51.660 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-17T12:28:51.753 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-02-17T12:28:51.764 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-02-17T12:28:51.764 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-02-17T12:28:51.764 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-02-17T12:28:51.764 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-02-17T12:28:51.764 INFO:teuthology.orchestra.run.smithi018.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-17T12:28:51.765 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-02-17T12:28:51.765 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-17T12:28:51.765 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-17T12:28:51.765 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-17T12:28:51.765 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:28:51.765 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-02-17T12:28:51.765 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-02-17T12:28:51.765 INFO:teuthology.orchestra.run.smithi018.stdout:Install 4 Packages 2024-02-17T12:28:51.765 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:28:51.766 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 100 M 2024-02-17T12:28:51.766 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 123 M 2024-02-17T12:28:51.766 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-02-17T12:28:52.662 INFO:teuthology.orchestra.run.smithi018.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 6.8 MB/s | 6.1 MB 00:00 2024-02-17T12:28:53.404 INFO:teuthology.orchestra.run.smithi018.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 13 MB/s | 21 MB 00:01 2024-02-17T12:28:54.637 INFO:teuthology.orchestra.run.smithi018.stdout:(3/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 14 MB/s | 39 MB 00:02 2024-02-17T12:28:54.996 INFO:teuthology.orchestra.run.smithi018.stdout:(4/4): kernel-modules-core-5.14.0-421.el9.x86_6 14 MB/s | 33 MB 00:02 2024-02-17T12:28:55.000 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-02-17T12:28:55.000 INFO:teuthology.orchestra.run.smithi018.stdout:Total 31 MB/s | 100 MB 00:03 2024-02-17T12:28:55.000 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-02-17T12:28:55.458 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-02-17T12:28:55.458 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-02-17T12:28:56.534 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-02-17T12:28:56.535 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-02-17T12:28:59.108 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-02-17T12:28:59.813 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-17T12:28:59.947 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-17T12:29:02.843 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-17T12:29:03.026 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-17T12:29:08.097 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-17T12:29:08.206 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-17T12:29:13.292 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-17T12:29:43.979 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-17T12:29:43.980 INFO:teuthology.orchestra.run.smithi018.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-17T12:29:43.980 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:29:44.051 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-17T12:29:44.596 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-17T12:29:44.596 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-17T12:29:44.596 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-17T12:29:44.596 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-17T12:29:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-17T12:29:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:29:45.028 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-02-17T12:29:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-17T12:29:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-17T12:29:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-17T12:29:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-17T12:29:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:29:45.028 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-02-17T12:29:45.513 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q kernel | sort -rV | head -n 1 2024-02-17T12:29:45.647 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-17T12:29:45.648 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-17T12:29:45.648 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-17T12:29:45.648 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-17T12:29:45.648 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-17T12:29:45.648 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-02-17T12:29:45.648 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi018.front.sepia.ceph.com, path=None, version=distro) 2024-02-17T12:29:45.648 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q kernel | sort -rV | head -n 1 2024-02-17T12:29:45.761 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-17T12:29:45.761 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-17T12:29:45.761 DEBUG:teuthology.orchestra.run.smithi018:> sudo rpm -qi grub2-tools 2024-02-17T12:29:45.808 INFO:teuthology.orchestra.run.smithi018.stdout:Name : grub2-tools 2024-02-17T12:29:45.808 INFO:teuthology.orchestra.run.smithi018.stdout:Epoch : 1 2024-02-17T12:29:45.808 INFO:teuthology.orchestra.run.smithi018.stdout:Version : 2.06 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Release : 68.el9 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Architecture: x86_64 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Group : Unspecified 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Size : 8264254 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:License : GPLv3+ 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Packager : builder@centos.org 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Vendor : CentOS 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Summary : Support tools for GRUB. 2024-02-17T12:29:45.809 INFO:teuthology.orchestra.run.smithi018.stdout:Description : 2024-02-17T12:29:45.810 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:29:45.810 INFO:teuthology.orchestra.run.smithi018.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-17T12:29:45.810 INFO:teuthology.orchestra.run.smithi018.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-17T12:29:45.810 INFO:teuthology.orchestra.run.smithi018.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-17T12:29:45.810 INFO:teuthology.orchestra.run.smithi018.stdout:hardware devices. 2024-02-17T12:29:45.810 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:29:45.810 INFO:teuthology.orchestra.run.smithi018.stdout:This subpackage provides tools for support of all platforms. 2024-02-17T12:29:45.811 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-17T12:29:45.811 INFO:teuthology.task.kernel:Updating grub on smithi018 to boot 5.14.0-421.el9.x86_64 2024-02-17T12:29:45.811 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-17T12:29:46.387 INFO:teuthology.orchestra.run.smithi018.stderr:Generating grub configuration file ... 2024-02-17T12:29:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-17T12:29:47.898 INFO:teuthology.orchestra.run.smithi018.stderr:done 2024-02-17T12:29:47.900 DEBUG:teuthology.orchestra.run.smithi018:> mktemp 2024-02-17T12:29:47.921 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.dX9L9lDYlt 2024-02-17T12:29:47.921 DEBUG:teuthology.orchestra.run.smithi018:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dX9L9lDYlt 2024-02-17T12:29:47.994 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.dX9L9lDYlt 2024-02-17T12:29:48.266 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.dX9L9lDYlt is 6KB 2024-02-17T12:29:48.315 DEBUG:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.dX9L9lDYlt 2024-02-17T12:29:48.329 DEBUG:teuthology.orchestra.run.smithi018:> sudo /bin/ls /boot/loader/entries || true 2024-02-17T12:29:48.401 INFO:teuthology.orchestra.run.smithi018.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-17T12:29:48.401 INFO:teuthology.orchestra.run.smithi018.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-17T12:29:48.401 INFO:teuthology.orchestra.run.smithi018.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-17T12:29:48.401 INFO:teuthology.orchestra.run.smithi018.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-17T12:29:48.402 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-17T12:29:48.565 DEBUG:teuthology.orchestra.run.smithi018:> sudo shutdown -r now 2024-02-17T12:30:18.567 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-17T12:30:18.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-02-17T12:30:18.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T12:31:18.571 DEBUG:teuthology.orchestra.remote:timed out 2024-02-17T12:31:27.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-02-17T12:31:27.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T12:31:28.101 DEBUG:teuthology.orchestra.run.smithi018:> true 2024-02-17T12:31:28.559 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-02-17T12:31:28.559 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-421.el9.x86_64"... 2024-02-17T12:31:28.559 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2024-02-17T12:31:28.573 INFO:teuthology.orchestra.run.smithi018.stdout:5.14.0-421.el9.x86_64 2024-02-17T12:31:28.573 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-17T12:31:28.573 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-17T12:31:28.574 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-17T12:31:29.575 DEBUG:teuthology.parallel:result is None 2024-02-17T12:31:29.575 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-17T12:31:29.585 INFO:teuthology.task.internal:Creating test directory... 2024-02-17T12:31:29.585 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-17T12:31:29.588 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-17T12:31:29.590 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-17T12:31:29.608 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-17T12:31:29.618 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-17T12:31:29.654 INFO:teuthology.task.internal:Creating archive directory... 2024-02-17T12:31:29.655 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-17T12:31:29.657 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-17T12:31:29.659 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-17T12:31:29.782 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-17T12:31:29.790 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-17T12:31:29.790 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-17T12:31:29.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:31:29.807 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-17T12:31:29.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:31:29.824 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-17T12:31:29.826 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-17T12:31:29.850 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-17T12:31:29.887 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-17T12:31:29.904 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-17T12:31:29.911 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-17T12:31:29.927 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-17T12:31:29.932 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-17T12:31:29.935 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-17T12:31:29.937 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-17T12:31:29.948 INFO:teuthology.task.internal:Configuring sudo... 2024-02-17T12:31:29.948 DEBUG:teuthology.orchestra.run.smithi018:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-17T12:31:29.951 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-17T12:31:29.955 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-17T12:31:29.985 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-17T12:31:29.996 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-17T12:31:29.996 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-17T12:31:30.019 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-17T12:31:30.024 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-17T12:31:30.041 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-17T12:31:30.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-17T12:31:30.166 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-17T12:31:30.247 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-17T12:31:30.289 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:31:30.289 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-17T12:31:30.363 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-17T12:31:30.409 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-17T12:31:30.465 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-17T12:31:30.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-17T12:31:30.593 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-02-17T12:31:30.593 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-17T12:31:30.661 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-17T12:31:30.709 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-17T12:31:30.766 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-17T12:31:30.809 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-17T12:31:30.853 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-02-17T12:31:30.853 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-17T12:31:30.922 DEBUG:teuthology.orchestra.run.smithi018:> sudo service rsyslog restart 2024-02-17T12:31:30.926 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-02-17T12:31:30.930 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2024-02-17T12:31:30.970 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-17T12:31:30.975 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-17T12:31:31.004 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-17T12:31:31.378 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-17T12:31:31.405 INFO:teuthology.task.internal:Starting timer... 2024-02-17T12:31:31.405 INFO:teuthology.run_tasks:Running task pcp... 2024-02-17T12:31:31.444 INFO:teuthology.run_tasks:Running task selinux... 2024-02-17T12:31:31.476 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-17T12:31:31.476 DEBUG:teuthology.orchestra.run.smithi018:> sudo service auditd rotate 2024-02-17T12:31:31.528 INFO:teuthology.orchestra.run.smithi018.stdout:Rotating logs: 2024-02-17T12:31:31.530 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2024-02-17T12:31:31.586 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2024-02-17T12:31:31.588 DEBUG:teuthology.orchestra.run.smithi146:> sudo service auditd rotate 2024-02-17T12:31:31.643 INFO:teuthology.orchestra.run.smithi146.stdout:Rotating logs: 2024-02-17T12:31:31.648 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-17T12:31:31.648 DEBUG:teuthology.orchestra.run.smithi018:> /usr/sbin/getenforce 2024-02-17T12:31:31.673 INFO:teuthology.orchestra.run.smithi018.stdout:Permissive 2024-02-17T12:31:31.673 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2024-02-17T12:31:31.691 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2024-02-17T12:31:31.692 DEBUG:teuthology.orchestra.run.smithi146:> /usr/sbin/getenforce 2024-02-17T12:31:31.720 INFO:teuthology.orchestra.run.smithi146.stdout:Permissive 2024-02-17T12:31:31.721 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi018.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi146.front.sepia.ceph.com': 'permissive'} 2024-02-17T12:31:31.721 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-17T12:31:31.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:31:31.748 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-17T12:31:31.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:31:31.775 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-17T12:31:31.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:31:31.803 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-17T12:31:31.803 DEBUG:teuthology.orchestra.run.smithi018:> sudo /usr/sbin/setenforce permissive 2024-02-17T12:31:31.841 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2024-02-17T12:31:31.886 DEBUG:teuthology.orchestra.run.smithi146:> sudo /usr/sbin/setenforce permissive 2024-02-17T12:31:31.940 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-17T12:31:31.953 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-17T12:31:31.959 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-17T12:31:32.120 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-17T12:31:32.135 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-17T12:31:32.137 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi018.front.sepia.ceph.com,smithi119.front.sepia.ceph.com,smithi146.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-17T12:36:24.513 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi018.front.sepia.ceph.com'), Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi146.front.sepia.ceph.com')] 2024-02-17T12:36:24.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-02-17T12:36:24.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T12:36:24.615 DEBUG:teuthology.orchestra.run.smithi018:> true 2024-02-17T12:36:24.691 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-02-17T12:36:24.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-02-17T12:36:24.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T12:36:24.783 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-02-17T12:36:24.850 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-02-17T12:36:24.851 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-02-17T12:36:24.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T12:36:24.944 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-02-17T12:36:25.011 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-02-17T12:36:25.012 INFO:teuthology.run_tasks:Running task clock... 2024-02-17T12:36:25.023 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-17T12:36:25.023 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-17T12:36:25.023 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-17T12:36:25.026 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-17T12:36:25.026 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-17T12:36:25.028 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-17T12:36:25.028 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-17T12:36:25.060 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-17T12:36:25.060 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-17T12:36:25.073 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-17T12:36:25.074 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-17T12:36:25.084 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-17T12:36:25.102 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-17T12:36:25.102 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2024-02-17T12:36:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: ntpd: command not found 2024-02-17T12:36:25.116 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2024-02-17T12:36:25.119 INFO:teuthology.orchestra.run.smithi018.stdout:506 Cannot talk to daemon 2024-02-17T12:36:25.131 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: ntpd: command not found 2024-02-17T12:36:25.132 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-17T12:36:25.136 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-17T12:36:25.145 INFO:teuthology.orchestra.run.smithi146.stdout:506 Cannot talk to daemon 2024-02-17T12:36:25.149 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-17T12:36:25.153 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-17T12:36:25.161 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-17T12:36:25.177 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-17T12:36:25.193 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-02-17T12:36:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-17T12:36:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-02-17T12:36:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:36:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:36:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:36:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:36:25.210 INFO:teuthology.orchestra.run.smithi018.stderr:bash: line 1: ntpq: command not found 2024-02-17T12:36:25.213 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-17T12:36:25.213 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2024-02-17T12:36:25.213 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:36:25.213 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:36:25.214 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:36:25.214 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:36:25.221 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-02-17T12:36:25.225 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-17T12:36:25.225 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-02-17T12:36:25.225 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:36:25.225 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:36:25.226 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:36:25.226 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:36:25.227 INFO:teuthology.run_tasks:Running task install... 2024-02-17T12:36:25.235 DEBUG:teuthology.task.install:project ceph 2024-02-17T12:36:25.235 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d'}} 2024-02-17T12:36:25.235 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d'} 2024-02-17T12:36:25.235 INFO:teuthology.task.install:Using flavor: default 2024-02-17T12:36:25.244 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-17T12:36:25.244 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-02-17T12:36:25.245 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d', 'tag': None, 'wait_for_package': False} 2024-02-17T12:36:25.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:36:25.249 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d', 'tag': None, 'wait_for_package': False} 2024-02-17T12:36:25.249 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:36:25.250 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d', 'tag': None, 'wait_for_package': False} 2024-02-17T12:36:25.251 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:36:25.649 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/ 2024-02-17T12:36:25.649 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1515.g6822c7b5 2024-02-17T12:36:25.651 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/ 2024-02-17T12:36:25.651 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1515.g6822c7b5 2024-02-17T12:36:25.661 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/ 2024-02-17T12:36:25.661 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1515.g6822c7b5 2024-02-17T12:36:25.767 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-17T12:36:25.767 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-02-17T12:36:25.767 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-17T12:36:25.775 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-17T12:36:25.776 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-02-17T12:36:25.776 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-17T12:36:25.796 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-17T12:36:25.796 DEBUG:teuthology.orchestra.run.smithi146:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6822c7b54e949006be7a0802650d01723d834d1d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-17T12:36:25.805 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-17T12:36:25.805 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6822c7b54e949006be7a0802650d01723d834d1d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-17T12:36:25.865 DEBUG:teuthology.orchestra.run.smithi146:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-17T12:36:25.872 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-17T12:36:25.946 DEBUG:teuthology.orchestra.run.smithi146:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-17T12:36:25.952 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-17T12:36:25.959 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-17T12:36:25.959 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:36:25.959 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-17T12:36:25.988 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-17T12:36:25.989 DEBUG:teuthology.orchestra.run.smithi018:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6822c7b54e949006be7a0802650d01723d834d1d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-17T12:36:26.013 INFO:teuthology.orchestra.run.smithi146.stdout:check_obsoletes = 1 2024-02-17T12:36:26.014 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum clean all 2024-02-17T12:36:26.018 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2024-02-17T12:36:26.020 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2024-02-17T12:36:26.059 DEBUG:teuthology.orchestra.run.smithi018:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-17T12:36:26.140 DEBUG:teuthology.orchestra.run.smithi018:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-17T12:36:26.206 INFO:teuthology.orchestra.run.smithi018.stdout:check_obsoletes = 1 2024-02-17T12:36:26.208 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum clean all 2024-02-17T12:36:26.247 INFO:teuthology.orchestra.run.smithi146.stdout:74 files removed 2024-02-17T12:36:26.255 INFO:teuthology.orchestra.run.smithi119.stdout:74 files removed 2024-02-17T12:36:26.274 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-17T12:36:26.281 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-17T12:36:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:74 files removed 2024-02-17T12:36:26.487 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-17T12:36:27.076 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 162 kB/s | 82 kB 00:00 2024-02-17T12:36:27.292 INFO:teuthology.orchestra.run.smithi146.stdout:ceph packages for x86_64 113 kB/s | 82 kB 00:00 2024-02-17T12:36:27.403 INFO:teuthology.orchestra.run.smithi018.stdout:ceph packages for x86_64 138 kB/s | 82 kB 00:00 2024-02-17T12:36:27.810 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 14 kB/s | 10 kB 00:00 2024-02-17T12:36:28.009 INFO:teuthology.orchestra.run.smithi146.stdout:ceph noarch packages 15 kB/s | 10 kB 00:00 2024-02-17T12:36:28.079 INFO:teuthology.orchestra.run.smithi018.stdout:ceph noarch packages 16 kB/s | 10 kB 00:00 2024-02-17T12:36:28.502 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-02-17T12:36:28.579 INFO:teuthology.orchestra.run.smithi018.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-02-17T12:36:28.726 INFO:teuthology.orchestra.run.smithi146.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-02-17T12:36:29.716 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-9 - AppStream 6.1 MB/s | 6.7 MB 00:01 2024-02-17T12:36:29.738 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - AppStream 5.6 MB/s | 6.7 MB 00:01 2024-02-17T12:36:29.812 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS-9 - AppStream 6.4 MB/s | 6.7 MB 00:01 2024-02-17T12:36:31.745 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS-9 - BaseOS 2.5 MB/s | 2.0 MB 00:00 2024-02-17T12:36:31.974 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-9 - BaseOS 1.8 MB/s | 2.0 MB 00:01 2024-02-17T12:36:32.631 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - BaseOS 1.1 MB/s | 2.0 MB 00:01 2024-02-17T12:36:32.711 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-02-17T12:36:34.287 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.0 MB 00:01 2024-02-17T12:36:37.838 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 19 MB 00:02 2024-02-17T12:36:40.529 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - BaseOS 995 kB/s | 8.0 MB 00:08 2024-02-17T12:36:41.880 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-02-17T12:36:42.346 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-02-17T12:36:43.472 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2024-02-17T12:36:45.602 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 1.6 MB/s | 19 MB 00:11 2024-02-17T12:36:48.439 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.1 MB 00:02 2024-02-17T12:36:50.007 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 15 kB 00:00 2024-02-17T12:36:50.877 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.1 MB 00:01 2024-02-17T12:36:52.320 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 15 kB 00:00 2024-02-17T12:36:54.171 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-17T12:36:56.277 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 3.3 MB/s | 20 MB 00:06 2024-02-17T12:36:59.012 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-02-17T12:37:01.307 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-02-17T12:37:01.588 INFO:teuthology.orchestra.run.smithi146.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-17T12:37:01.588 INFO:teuthology.orchestra.run.smithi146.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-17T12:37:01.601 INFO:teuthology.orchestra.run.smithi146.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-17T12:37:01.601 INFO:teuthology.orchestra.run.smithi146.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-17T12:37:01.602 INFO:teuthology.orchestra.run.smithi146.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-17T12:37:01.603 INFO:teuthology.orchestra.run.smithi146.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-17T12:37:01.605 INFO:teuthology.orchestra.run.smithi146.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-17T12:37:01.608 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-17T12:37:01.609 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-17T12:37:01.690 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-02-17T12:37:01.719 INFO:teuthology.orchestra.run.smithi146.stdout:============================================================================================ 2024-02-17T12:37:01.719 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-02-17T12:37:01.719 INFO:teuthology.orchestra.run.smithi146.stdout:============================================================================================ 2024-02-17T12:37:01.719 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-02-17T12:37:01.719 INFO:teuthology.orchestra.run.smithi146.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-17T12:37:01.719 INFO:teuthology.orchestra.run.smithi146.stdout: ceph x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 6.6 k 2024-02-17T12:37:01.719 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 5.3 M 2024-02-17T12:37:01.719 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 885 k 2024-02-17T12:37:01.720 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 146 k 2024-02-17T12:37:01.720 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 1.5 M 2024-02-17T12:37:01.720 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 138 k 2024-02-17T12:37:01.720 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 1.8 M 2024-02-17T12:37:01.720 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 7.4 M 2024-02-17T12:37:01.720 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 52 k 2024-02-17T12:37:01.720 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 9.8 M 2024-02-17T12:37:01.720 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 47 M 2024-02-17T12:37:01.720 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 264 k 2024-02-17T12:37:01.720 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 748 k 2024-02-17T12:37:01.720 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 217 k 2024-02-17T12:37:01.721 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 27 k 2024-02-17T12:37:01.721 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 18 k 2024-02-17T12:37:01.721 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-17T12:37:01.721 INFO:teuthology.orchestra.run.smithi146.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-17T12:37:01.721 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-17T12:37:01.721 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 31 k 2024-02-17T12:37:01.721 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 743 k 2024-02-17T12:37:01.721 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 127 k 2024-02-17T12:37:01.721 INFO:teuthology.orchestra.run.smithi146.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-17T12:37:01.721 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-17T12:37:01.721 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-17T12:37:01.722 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-17T12:37:01.722 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-17T12:37:01.722 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-17T12:37:01.722 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 162 k 2024-02-17T12:37:01.722 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 321 k 2024-02-17T12:37:01.722 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 298 k 2024-02-17T12:37:01.722 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 99 k 2024-02-17T12:37:01.722 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 85 k 2024-02-17T12:37:01.722 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.1 M 2024-02-17T12:37:01.722 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 171 k 2024-02-17T12:37:01.722 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-17T12:37:01.723 INFO:teuthology.orchestra.run.smithi146.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-17T12:37:01.724 INFO:teuthology.orchestra.run.smithi146.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-17T12:37:01.724 INFO:teuthology.orchestra.run.smithi146.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-17T12:37:01.724 INFO:teuthology.orchestra.run.smithi146.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-17T12:37:01.724 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.4 M 2024-02-17T12:37:01.724 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.1 M 2024-02-17T12:37:01.724 INFO:teuthology.orchestra.run.smithi146.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-17T12:37:01.724 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-17T12:37:01.724 INFO:teuthology.orchestra.run.smithi146.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-17T12:37:01.724 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-17T12:37:01.724 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-17T12:37:01.725 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-17T12:37:01.725 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-17T12:37:01.725 INFO:teuthology.orchestra.run.smithi146.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-17T12:37:01.725 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-17T12:37:01.725 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-02-17T12:37:01.725 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-17T12:37:01.725 INFO:teuthology.orchestra.run.smithi146.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-17T12:37:01.725 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-17T12:37:01.725 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 20 M 2024-02-17T12:37:01.725 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 31 k 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 2.1 M 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 247 k 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 4.7 M 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 17 M 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 15 k 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 25 k 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-17T12:37:01.726 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 163 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 499 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 5.1 M 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-17T12:37:01.727 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 45 k 2024-02-17T12:37:01.728 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 131 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-17T12:37:01.729 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-17T12:37:01.730 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-17T12:37:01.731 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-17T12:37:01.731 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-17T12:37:01.731 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-17T12:37:01.731 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-17T12:37:01.731 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-17T12:37:01.731 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-17T12:37:01.731 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-17T12:37:01.731 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-17T12:37:01.731 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-17T12:37:01.731 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-17T12:37:01.731 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-17T12:37:01.732 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-17T12:37:01.732 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-17T12:37:01.732 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-17T12:37:01.732 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-17T12:37:01.732 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-17T12:37:01.732 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-17T12:37:01.732 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-17T12:37:01.732 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-17T12:37:01.732 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-17T12:37:01.733 INFO:teuthology.orchestra.run.smithi146.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-17T12:37:01.734 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-17T12:37:01.734 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-17T12:37:01.734 INFO:teuthology.orchestra.run.smithi146.stdout:Installing weak dependencies: 2024-02-17T12:37:01.734 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-17T12:37:01.734 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:01.734 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-02-17T12:37:01.734 INFO:teuthology.orchestra.run.smithi146.stdout:============================================================================================ 2024-02-17T12:37:01.734 INFO:teuthology.orchestra.run.smithi146.stdout:Install 150 Packages 2024-02-17T12:37:01.734 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 26 Packages 2024-02-17T12:37:01.734 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:01.735 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 223 M 2024-02-17T12:37:01.735 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-02-17T12:37:02.405 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 20 MB 00:18 2024-02-17T12:37:02.463 INFO:teuthology.orchestra.run.smithi146.stdout:(1/176): ceph-19.0.0-1515.g6822c7b5.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-02-17T12:37:02.955 INFO:teuthology.orchestra.run.smithi146.stdout:(2/176): ceph-fuse-19.0.0-1515.g6822c7b5.el9.x8 1.8 MB/s | 885 kB 00:00 2024-02-17T12:37:03.072 INFO:teuthology.orchestra.run.smithi146.stdout:(3/176): ceph-immutable-object-cache-19.0.0-151 1.2 MB/s | 146 kB 00:00 2024-02-17T12:37:03.889 INFO:teuthology.orchestra.run.smithi146.stdout:(4/176): ceph-common-19.0.0-1515.g6822c7b5.el9. 12 MB/s | 20 MB 00:01 2024-02-17T12:37:03.939 INFO:teuthology.orchestra.run.smithi146.stdout:(5/176): ceph-mds-19.0.0-1515.g6822c7b5.el9.x86 2.4 MB/s | 2.1 MB 00:00 2024-02-17T12:37:03.974 INFO:teuthology.orchestra.run.smithi018.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-17T12:37:03.976 INFO:teuthology.orchestra.run.smithi018.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-17T12:37:03.990 INFO:teuthology.orchestra.run.smithi018.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-17T12:37:03.992 INFO:teuthology.orchestra.run.smithi018.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-17T12:37:03.993 INFO:teuthology.orchestra.run.smithi018.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-17T12:37:03.994 INFO:teuthology.orchestra.run.smithi018.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-17T12:37:03.996 INFO:teuthology.orchestra.run.smithi018.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-17T12:37:03.999 INFO:teuthology.orchestra.run.smithi018.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-17T12:37:04.000 INFO:teuthology.orchestra.run.smithi018.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-17T12:37:04.006 INFO:teuthology.orchestra.run.smithi146.stdout:(6/176): ceph-mgr-19.0.0-1515.g6822c7b5.el9.x86 13 MB/s | 1.5 MB 00:00 2024-02-17T12:37:04.085 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-02-17T12:37:04.106 INFO:teuthology.orchestra.run.smithi018.stdout:============================================================================================ 2024-02-17T12:37:04.106 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-02-17T12:37:04.106 INFO:teuthology.orchestra.run.smithi018.stdout:============================================================================================ 2024-02-17T12:37:04.106 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-02-17T12:37:04.106 INFO:teuthology.orchestra.run.smithi018.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ceph x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 6.6 k 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 5.3 M 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 885 k 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 146 k 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 1.5 M 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 138 k 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 1.8 M 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 7.4 M 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 52 k 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 9.8 M 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 47 M 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-volume noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 264 k 2024-02-17T12:37:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 748 k 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-mirror x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 217 k 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-shell x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 27 k 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-top noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 18 k 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 31 k 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 743 k 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 127 k 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-17T12:37:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 162 k 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 321 k 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 298 k 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 99 k 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 85 k 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.1 M 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 171 k 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrading: 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-17T12:37:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-17T12:37:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: librados2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.4 M 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.1 M 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-17T12:37:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 20 M 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 31 k 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 2.1 M 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 247 k 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 4.7 M 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 17 M 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 15 k 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 25 k 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-17T12:37:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 163 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 499 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-17T12:37:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 5.1 M 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-17T12:37:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 45 k 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 131 k 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-17T12:37:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-17T12:37:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-17T12:37:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-17T12:37:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-17T12:37:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-17T12:37:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-17T12:37:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-17T12:37:04.120 INFO:teuthology.orchestra.run.smithi018.stdout:Installing weak dependencies: 2024-02-17T12:37:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-17T12:37:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:37:04.120 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-02-17T12:37:04.120 INFO:teuthology.orchestra.run.smithi018.stdout:============================================================================================ 2024-02-17T12:37:04.120 INFO:teuthology.orchestra.run.smithi018.stdout:Install 150 Packages 2024-02-17T12:37:04.120 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrade 26 Packages 2024-02-17T12:37:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:37:04.121 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 223 M 2024-02-17T12:37:04.121 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-02-17T12:37:04.273 INFO:teuthology.orchestra.run.smithi146.stdout:(7/176): ceph-base-19.0.0-1515.g6822c7b5.el9.x8 2.6 MB/s | 5.3 MB 00:02 2024-02-17T12:37:04.874 INFO:teuthology.orchestra.run.smithi018.stdout:(1/176): ceph-19.0.0-1515.g6822c7b5.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-02-17T12:37:04.915 INFO:teuthology.orchestra.run.smithi146.stdout:(8/176): ceph-osd-19.0.0-1515.g6822c7b5.el9.x86 18 MB/s | 17 MB 00:00 2024-02-17T12:37:04.965 INFO:teuthology.orchestra.run.smithi146.stdout:(9/176): ceph-selinux-19.0.0-1515.g6822c7b5.el9 504 kB/s | 25 kB 00:00 2024-02-17T12:37:05.282 INFO:teuthology.orchestra.run.smithi146.stdout:(10/176): ceph-mon-19.0.0-1515.g6822c7b5.el9.x8 3.5 MB/s | 4.7 MB 00:01 2024-02-17T12:37:05.415 INFO:teuthology.orchestra.run.smithi146.stdout:(11/176): cephfs-mirror-19.0.0-1515.g6822c7b5.e 1.6 MB/s | 217 kB 00:00 2024-02-17T12:37:05.416 INFO:teuthology.orchestra.run.smithi018.stdout:(2/176): ceph-fuse-19.0.0-1515.g6822c7b5.el9.x8 1.6 MB/s | 885 kB 00:00 2024-02-17T12:37:05.532 INFO:teuthology.orchestra.run.smithi146.stdout:(12/176): cephfs-shell-19.0.0-1515.g6822c7b5.el 232 kB/s | 27 kB 00:00 2024-02-17T12:37:05.558 INFO:teuthology.orchestra.run.smithi018.stdout:(3/176): ceph-immutable-object-cache-19.0.0-151 1.0 MB/s | 146 kB 00:00 2024-02-17T12:37:05.674 INFO:teuthology.orchestra.run.smithi146.stdout:(13/176): libcephfs-devel-19.0.0-1515.g6822c7b5 220 kB/s | 31 kB 00:00 2024-02-17T12:37:06.016 INFO:teuthology.orchestra.run.smithi146.stdout:(14/176): libcephfs2-19.0.0-1515.g6822c7b5.el9. 2.1 MB/s | 743 kB 00:00 2024-02-17T12:37:06.125 INFO:teuthology.orchestra.run.smithi146.stdout:(15/176): libcephsqlite-19.0.0-1515.g6822c7b5.e 1.5 MB/s | 163 kB 00:00 2024-02-17T12:37:06.250 INFO:teuthology.orchestra.run.smithi146.stdout:(16/176): librados-devel-19.0.0-1515.g6822c7b5. 1.0 MB/s | 127 kB 00:00 2024-02-17T12:37:06.567 INFO:teuthology.orchestra.run.smithi146.stdout:(17/176): libradosstriper1-19.0.0-1515.g6822c7b 1.5 MB/s | 499 kB 00:00 2024-02-17T12:37:06.583 INFO:teuthology.orchestra.run.smithi018.stdout:(4/176): ceph-mds-19.0.0-1515.g6822c7b5.el9.x86 2.1 MB/s | 2.1 MB 00:01 2024-02-17T12:37:06.917 INFO:teuthology.orchestra.run.smithi018.stdout:(5/176): ceph-base-19.0.0-1515.g6822c7b5.el9.x8 2.4 MB/s | 5.3 MB 00:02 2024-02-17T12:37:07.254 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-02-17T12:37:07.334 INFO:teuthology.orchestra.run.smithi018.stdout:(6/176): ceph-mgr-19.0.0-1515.g6822c7b5.el9.x86 2.0 MB/s | 1.5 MB 00:00 2024-02-17T12:37:07.359 INFO:teuthology.orchestra.run.smithi146.stdout:(18/176): ceph-radosgw-19.0.0-1515.g6822c7b5.el 3.2 MB/s | 9.8 MB 00:03 2024-02-17T12:37:07.476 INFO:teuthology.orchestra.run.smithi146.stdout:(19/176): python3-ceph-argparse-19.0.0-1515.g68 387 kB/s | 45 kB 00:00 2024-02-17T12:37:07.593 INFO:teuthology.orchestra.run.smithi146.stdout:(20/176): python3-ceph-common-19.0.0-1515.g6822 1.1 MB/s | 131 kB 00:00 2024-02-17T12:37:07.702 INFO:teuthology.orchestra.run.smithi146.stdout:(21/176): python3-cephfs-19.0.0-1515.g6822c7b5. 1.5 MB/s | 162 kB 00:00 2024-02-17T12:37:08.126 INFO:teuthology.orchestra.run.smithi018.stdout:(7/176): ceph-mon-19.0.0-1515.g6822c7b5.el9.x86 3.9 MB/s | 4.7 MB 00:01 2024-02-17T12:37:08.269 INFO:teuthology.orchestra.run.smithi146.stdout:(22/176): ceph-test-19.0.0-1515.g6822c7b5.el9.x 14 MB/s | 47 MB 00:03 2024-02-17T12:37:08.302 INFO:teuthology.orchestra.run.smithi146.stdout:(23/176): python3-rados-19.0.0-1515.g6822c7b5.e 534 kB/s | 321 kB 00:00 2024-02-17T12:37:08.327 INFO:teuthology.orchestra.run.smithi146.stdout:(24/176): python3-rbd-19.0.0-1515.g6822c7b5.el9 5.0 MB/s | 298 kB 00:00 2024-02-17T12:37:08.403 INFO:teuthology.orchestra.run.smithi146.stdout:(25/176): librgw2-19.0.0-1515.g6822c7b5.el9.x86 2.8 MB/s | 5.1 MB 00:01 2024-02-17T12:37:08.428 INFO:teuthology.orchestra.run.smithi146.stdout:(26/176): rbd-fuse-19.0.0-1515.g6822c7b5.el9.x8 851 kB/s | 85 kB 00:00 2024-02-17T12:37:08.444 INFO:teuthology.orchestra.run.smithi146.stdout:(27/176): python3-rgw-19.0.0-1515.g6822c7b5.el9 699 kB/s | 99 kB 00:00 2024-02-17T12:37:08.495 INFO:teuthology.orchestra.run.smithi146.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1515.g 628 kB/s | 31 kB 00:00 2024-02-17T12:37:08.520 INFO:teuthology.orchestra.run.smithi146.stdout:(29/176): rbd-nbd-19.0.0-1515.g6822c7b5.el9.x86 1.8 MB/s | 171 kB 00:00 2024-02-17T12:37:08.537 INFO:teuthology.orchestra.run.smithi146.stdout:(30/176): ceph-mgr-cephadm-19.0.0-1515.g6822c7b 3.2 MB/s | 138 kB 00:00 2024-02-17T12:37:09.187 INFO:teuthology.orchestra.run.smithi146.stdout:(31/176): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-02-17T12:37:09.246 INFO:teuthology.orchestra.run.smithi146.stdout:(32/176): rbd-mirror-19.0.0-1515.g6822c7b5.el9. 3.7 MB/s | 3.1 MB 00:00 2024-02-17T12:37:09.288 INFO:teuthology.orchestra.run.smithi146.stdout:(33/176): ceph-mgr-dashboard-19.0.0-1515.g6822c 2.3 MB/s | 1.8 MB 00:00 2024-02-17T12:37:09.313 INFO:teuthology.orchestra.run.smithi146.stdout:(34/176): ceph-mgr-modules-core-19.0.0-1515.g68 1.9 MB/s | 247 kB 00:00 2024-02-17T12:37:09.330 INFO:teuthology.orchestra.run.smithi146.stdout:(35/176): ceph-mgr-rook-19.0.0-1515.g6822c7b5.e 624 kB/s | 52 kB 00:00 2024-02-17T12:37:09.355 INFO:teuthology.orchestra.run.smithi146.stdout:(36/176): ceph-prometheus-alerts-19.0.0-1515.g6 228 kB/s | 15 kB 00:00 2024-02-17T12:37:09.381 INFO:teuthology.orchestra.run.smithi146.stdout:(37/176): ceph-volume-19.0.0-1515.g6822c7b5.el9 3.8 MB/s | 264 kB 00:00 2024-02-17T12:37:09.447 INFO:teuthology.orchestra.run.smithi146.stdout:(38/176): cephfs-top-19.0.0-1515.g6822c7b5.el9. 199 kB/s | 18 kB 00:00 2024-02-17T12:37:09.498 INFO:teuthology.orchestra.run.smithi146.stdout:(39/176): cephadm-19.0.0-1515.g6822c7b5.el9.noa 4.4 MB/s | 748 kB 00:00 2024-02-17T12:37:09.690 INFO:teuthology.orchestra.run.smithi146.stdout:(40/176): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 662 kB 00:00 2024-02-17T12:37:09.715 INFO:teuthology.orchestra.run.smithi146.stdout:(41/176): boost-program-options-1.75.0-8.el9.x8 555 kB/s | 106 kB 00:00 2024-02-17T12:37:09.740 INFO:teuthology.orchestra.run.smithi146.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 412 kB/s | 20 kB 00:00 2024-02-17T12:37:09.774 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-17T12:37:09.774 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-17T12:37:09.775 INFO:teuthology.orchestra.run.smithi018.stdout:(8/176): ceph-common-19.0.0-1515.g6822c7b5.el9. 3.9 MB/s | 20 MB 00:05 2024-02-17T12:37:09.782 INFO:teuthology.orchestra.run.smithi146.stdout:(43/176): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 920 kB 00:00 2024-02-17T12:37:09.786 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-17T12:37:09.787 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-17T12:37:09.788 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-17T12:37:09.789 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-17T12:37:09.791 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-17T12:37:09.793 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-17T12:37:09.794 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-17T12:37:09.807 INFO:teuthology.orchestra.run.smithi146.stdout:(44/176): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-02-17T12:37:09.824 INFO:teuthology.orchestra.run.smithi146.stdout:(45/176): flexiblas-3.0.4-8.el9.x86_64.rpm 380 kB/s | 32 kB 00:00 2024-02-17T12:37:09.875 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-02-17T12:37:09.901 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2024-02-17T12:37:09.901 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-02-17T12:37:09.901 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2024-02-17T12:37:09.901 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-02-17T12:37:09.901 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 6.6 k 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 5.3 M 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 885 k 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 146 k 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 1.5 M 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 138 k 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 1.8 M 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 7.4 M 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 52 k 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 9.8 M 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 47 M 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 264 k 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 748 k 2024-02-17T12:37:09.902 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 217 k 2024-02-17T12:37:09.903 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 27 k 2024-02-17T12:37:09.903 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 18 k 2024-02-17T12:37:09.903 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-17T12:37:09.903 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-17T12:37:09.903 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-17T12:37:09.903 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 31 k 2024-02-17T12:37:09.903 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 743 k 2024-02-17T12:37:09.903 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 127 k 2024-02-17T12:37:09.903 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-17T12:37:09.903 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-17T12:37:09.904 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-17T12:37:09.904 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-17T12:37:09.904 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-17T12:37:09.904 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-17T12:37:09.904 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 162 k 2024-02-17T12:37:09.904 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 321 k 2024-02-17T12:37:09.904 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 298 k 2024-02-17T12:37:09.904 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 99 k 2024-02-17T12:37:09.904 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 85 k 2024-02-17T12:37:09.905 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.1 M 2024-02-17T12:37:09.905 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 171 k 2024-02-17T12:37:09.905 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-17T12:37:09.905 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-02-17T12:37:09.905 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-17T12:37:09.905 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-17T12:37:09.905 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-17T12:37:09.905 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-17T12:37:09.905 INFO:teuthology.orchestra.run.smithi119.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-17T12:37:09.905 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.4 M 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.1 M 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-17T12:37:09.906 INFO:teuthology.orchestra.run.smithi119.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 20 M 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 31 k 2024-02-17T12:37:09.907 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 2.1 M 2024-02-17T12:37:09.908 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 247 k 2024-02-17T12:37:09.908 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 4.7 M 2024-02-17T12:37:09.908 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 17 M 2024-02-17T12:37:09.908 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 15 k 2024-02-17T12:37:09.908 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 25 k 2024-02-17T12:37:09.908 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-17T12:37:09.908 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-17T12:37:09.908 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-17T12:37:09.908 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-17T12:37:09.908 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-17T12:37:09.908 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-17T12:37:09.908 INFO:teuthology.orchestra.run.smithi119.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 163 k 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 499 k 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 5.1 M 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-17T12:37:09.909 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-17T12:37:09.910 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 45 k 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 131 k 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-17T12:37:09.911 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-17T12:37:09.912 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-17T12:37:09.913 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-17T12:37:09.914 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-17T12:37:09.914 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-17T12:37:09.914 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-17T12:37:09.914 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-17T12:37:09.914 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-17T12:37:09.914 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-17T12:37:09.914 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-17T12:37:09.914 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-17T12:37:09.914 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-17T12:37:09.915 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-17T12:37:09.916 INFO:teuthology.orchestra.run.smithi119.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-17T12:37:09.916 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-17T12:37:09.916 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-17T12:37:09.916 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2024-02-17T12:37:09.916 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-17T12:37:09.916 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:37:09.916 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-02-17T12:37:09.916 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2024-02-17T12:37:09.916 INFO:teuthology.orchestra.run.smithi119.stdout:Install 150 Packages 2024-02-17T12:37:09.916 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 26 Packages 2024-02-17T12:37:09.916 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:37:09.917 INFO:teuthology.orchestra.run.smithi018.stdout:(9/176): ceph-radosgw-19.0.0-1515.g6822c7b5.el9 5.5 MB/s | 9.8 MB 00:01 2024-02-17T12:37:09.918 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 223 M 2024-02-17T12:37:09.918 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-02-17T12:37:09.935 INFO:teuthology.orchestra.run.smithi018.stdout:(10/176): ceph-selinux-19.0.0-1515.g6822c7b5.el 159 kB/s | 25 kB 00:00 2024-02-17T12:37:09.941 INFO:teuthology.orchestra.run.smithi146.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 147 kB/s | 17 kB 00:00 2024-02-17T12:37:09.958 INFO:teuthology.orchestra.run.smithi146.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 75 kB/s | 10 kB 00:00 2024-02-17T12:37:10.001 INFO:teuthology.orchestra.run.smithi018.stdout:(11/176): cephfs-mirror-19.0.0-1515.g6822c7b5.e 3.2 MB/s | 217 kB 00:00 2024-02-17T12:37:10.010 INFO:teuthology.orchestra.run.smithi146.stdout:(48/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-02-17T12:37:10.033 INFO:teuthology.orchestra.run.smithi146.stdout:(49/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2024-02-17T12:37:10.046 INFO:teuthology.orchestra.run.smithi018.stdout:(12/176): cephfs-shell-19.0.0-1515.g6822c7b5.el 647 kB/s | 27 kB 00:00 2024-02-17T12:37:10.067 INFO:teuthology.orchestra.run.smithi146.stdout:(50/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 408 kB/s | 44 kB 00:00 2024-02-17T12:37:10.093 INFO:teuthology.orchestra.run.smithi146.stdout:(51/176): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-02-17T12:37:10.094 INFO:teuthology.orchestra.run.smithi018.stdout:(13/176): libcephfs-devel-19.0.0-1515.g6822c7b5 627 kB/s | 31 kB 00:00 2024-02-17T12:37:10.127 INFO:teuthology.orchestra.run.smithi146.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 425 kB/s | 24 kB 00:00 2024-02-17T12:37:10.152 INFO:teuthology.orchestra.run.smithi146.stdout:(53/176): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-02-17T12:37:10.177 INFO:teuthology.orchestra.run.smithi146.stdout:(54/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 553 kB/s | 46 kB 00:00 2024-02-17T12:37:10.202 INFO:teuthology.orchestra.run.smithi146.stdout:(55/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-02-17T12:37:10.205 INFO:teuthology.orchestra.run.smithi018.stdout:(14/176): libcephfs2-19.0.0-1515.g6822c7b5.el9. 6.7 MB/s | 743 kB 00:00 2024-02-17T12:37:10.227 INFO:teuthology.orchestra.run.smithi146.stdout:(56/176): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-02-17T12:37:10.252 INFO:teuthology.orchestra.run.smithi146.stdout:(57/176): openblas-0.3.21-2.el9.x86_64.rpm 456 kB/s | 34 kB 00:00 2024-02-17T12:37:10.261 INFO:teuthology.orchestra.run.smithi018.stdout:(15/176): libcephsqlite-19.0.0-1515.g6822c7b5.e 2.7 MB/s | 163 kB 00:00 2024-02-17T12:37:10.311 INFO:teuthology.orchestra.run.smithi018.stdout:(16/176): librados-devel-19.0.0-1515.g6822c7b5. 2.5 MB/s | 127 kB 00:00 2024-02-17T12:37:10.369 INFO:teuthology.orchestra.run.smithi146.stdout:(58/176): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-02-17T12:37:10.403 INFO:teuthology.orchestra.run.smithi018.stdout:(17/176): libradosstriper1-19.0.0-1515.g6822c7b 5.3 MB/s | 499 kB 00:00 2024-02-17T12:37:10.428 INFO:teuthology.orchestra.run.smithi146.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 8.8 MB/s | 1.5 MB 00:00 2024-02-17T12:37:10.487 INFO:teuthology.orchestra.run.smithi146.stdout:(60/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-02-17T12:37:10.512 INFO:teuthology.orchestra.run.smithi146.stdout:(61/176): postgresql-contrib-13.11-1.el9.x86_64 5.7 MB/s | 811 kB 00:00 2024-02-17T12:37:10.537 INFO:teuthology.orchestra.run.smithi146.stdout:(62/176): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2024-02-17T12:37:10.579 INFO:teuthology.orchestra.run.smithi146.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-02-17T12:37:10.729 INFO:teuthology.orchestra.run.smithi146.stdout:(64/176): postgresql-server-13.11-1.el9.x86_64. 26 MB/s | 5.7 MB 00:00 2024-02-17T12:37:10.737 INFO:teuthology.orchestra.run.smithi018.stdout:(18/176): ceph-osd-19.0.0-1515.g6822c7b5.el9.x8 4.9 MB/s | 17 MB 00:03 2024-02-17T12:37:10.754 INFO:teuthology.orchestra.run.smithi146.stdout:(65/176): python3-attrs-20.3.0-7.el9.noarch.rpm 359 kB/s | 78 kB 00:00 2024-02-17T12:37:10.756 INFO:teuthology.orchestra.run.smithi119.stdout:(1/176): ceph-19.0.0-1515.g6822c7b5.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-02-17T12:37:10.787 INFO:teuthology.orchestra.run.smithi018.stdout:(19/176): python3-ceph-argparse-19.0.0-1515.g68 909 kB/s | 45 kB 00:00 2024-02-17T12:37:10.840 INFO:teuthology.orchestra.run.smithi018.stdout:(20/176): python3-ceph-common-19.0.0-1515.g6822 2.6 MB/s | 131 kB 00:00 2024-02-17T12:37:10.863 INFO:teuthology.orchestra.run.smithi146.stdout:(66/176): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-02-17T12:37:10.887 INFO:teuthology.orchestra.run.smithi018.stdout:(21/176): python3-cephfs-19.0.0-1515.g6822c7b5. 3.2 MB/s | 162 kB 00:00 2024-02-17T12:37:10.888 INFO:teuthology.orchestra.run.smithi146.stdout:(67/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.4 MB/s | 227 kB 00:00 2024-02-17T12:37:10.905 INFO:teuthology.orchestra.run.smithi146.stdout:(68/176): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-02-17T12:37:10.972 INFO:teuthology.orchestra.run.smithi018.stdout:(22/176): python3-rados-19.0.0-1515.g6822c7b5.e 3.8 MB/s | 321 kB 00:00 2024-02-17T12:37:10.972 INFO:teuthology.orchestra.run.smithi146.stdout:(69/176): python3-markupsafe-1.1.1-12.el9.x86_6 476 kB/s | 32 kB 00:00 2024-02-17T12:37:10.997 INFO:teuthology.orchestra.run.smithi146.stdout:(70/176): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 144 kB 00:00 2024-02-17T12:37:11.006 INFO:teuthology.orchestra.run.smithi119.stdout:(2/176): ceph-fuse-19.0.0-1515.g6822c7b5.el9.x8 3.5 MB/s | 885 kB 00:00 2024-02-17T12:37:11.013 INFO:teuthology.orchestra.run.smithi146.stdout:(71/176): python3-jsonpointer-2.0-4.el9.noarch. 120 kB/s | 18 kB 00:00 2024-02-17T12:37:11.055 INFO:teuthology.orchestra.run.smithi018.stdout:(23/176): python3-rbd-19.0.0-1515.g6822c7b5.el9 3.5 MB/s | 298 kB 00:00 2024-02-17T12:37:11.073 INFO:teuthology.orchestra.run.smithi119.stdout:(3/176): ceph-immutable-object-cache-19.0.0-151 2.1 MB/s | 146 kB 00:00 2024-02-17T12:37:11.105 INFO:teuthology.orchestra.run.smithi018.stdout:(24/176): python3-rgw-19.0.0-1515.g6822c7b5.el9 1.9 MB/s | 99 kB 00:00 2024-02-17T12:37:11.172 INFO:teuthology.orchestra.run.smithi018.stdout:(25/176): librgw2-19.0.0-1515.g6822c7b5.el9.x86 6.7 MB/s | 5.1 MB 00:00 2024-02-17T12:37:11.189 INFO:teuthology.orchestra.run.smithi018.stdout:(26/176): rbd-fuse-19.0.0-1515.g6822c7b5.el9.x8 1.0 MB/s | 85 kB 00:00 2024-02-17T12:37:11.230 INFO:teuthology.orchestra.run.smithi018.stdout:(27/176): rbd-nbd-19.0.0-1515.g6822c7b5.el9.x86 4.0 MB/s | 171 kB 00:00 2024-02-17T12:37:11.281 INFO:teuthology.orchestra.run.smithi018.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1515.g 629 kB/s | 31 kB 00:00 2024-02-17T12:37:11.314 INFO:teuthology.orchestra.run.smithi146.stdout:(72/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 4.8 MB 00:00 2024-02-17T12:37:11.339 INFO:teuthology.orchestra.run.smithi146.stdout:(73/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 636 kB/s | 217 kB 00:00 2024-02-17T12:37:11.340 INFO:teuthology.orchestra.run.smithi018.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1515.g6822c7b 2.3 MB/s | 138 kB 00:00 2024-02-17T12:37:11.365 INFO:teuthology.orchestra.run.smithi146.stdout:(74/176): python3-oauthlib-3.1.1-5.el9.noarch.r 498 kB/s | 174 kB 00:00 2024-02-17T12:37:11.398 INFO:teuthology.orchestra.run.smithi119.stdout:(4/176): ceph-mds-19.0.0-1515.g6822c7b5.el9.x86 6.5 MB/s | 2.1 MB 00:00 2024-02-17T12:37:11.498 INFO:teuthology.orchestra.run.smithi146.stdout:(75/176): python3-packaging-20.9-5.el9.noarch.r 375 kB/s | 69 kB 00:00 2024-02-17T12:37:11.590 INFO:teuthology.orchestra.run.smithi146.stdout:(76/176): python3-prettytable-0.7.2-27.el9.noar 183 kB/s | 41 kB 00:00 2024-02-17T12:37:11.623 INFO:teuthology.orchestra.run.smithi119.stdout:(5/176): ceph-mgr-19.0.0-1515.g6822c7b5.el9.x86 6.6 MB/s | 1.5 MB 00:00 2024-02-17T12:37:11.631 INFO:teuthology.orchestra.run.smithi018.stdout:(30/176): rbd-mirror-19.0.0-1515.g6822c7b5.el9. 6.7 MB/s | 3.1 MB 00:00 2024-02-17T12:37:11.673 INFO:teuthology.orchestra.run.smithi018.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1515.g6822c 5.3 MB/s | 1.8 MB 00:00 2024-02-17T12:37:11.765 INFO:teuthology.orchestra.run.smithi018.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1515.g68 2.6 MB/s | 247 kB 00:00 2024-02-17T12:37:11.815 INFO:teuthology.orchestra.run.smithi146.stdout:(77/176): python3-protobuf-3.14.0-13.el9.noarch 528 kB/s | 238 kB 00:00 2024-02-17T12:37:11.824 INFO:teuthology.orchestra.run.smithi018.stdout:(33/176): ceph-mgr-rook-19.0.0-1515.g6822c7b5.e 897 kB/s | 52 kB 00:00 2024-02-17T12:37:11.866 INFO:teuthology.orchestra.run.smithi146.stdout:(78/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 360 kB/s | 132 kB 00:00 2024-02-17T12:37:11.882 INFO:teuthology.orchestra.run.smithi018.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1515.g6 263 kB/s | 15 kB 00:00 2024-02-17T12:37:11.890 INFO:teuthology.orchestra.run.smithi146.stdout:(79/176): python3-pyasn1-modules-0.4.8-6.el9.no 701 kB/s | 211 kB 00:00 2024-02-17T12:37:11.966 INFO:teuthology.orchestra.run.smithi146.stdout:(80/176): python3-pytz-2021.1-5.el9.noarch.rpm 314 kB/s | 47 kB 00:00 2024-02-17T12:37:11.983 INFO:teuthology.orchestra.run.smithi018.stdout:(35/176): ceph-volume-19.0.0-1515.g6822c7b5.el9 2.6 MB/s | 264 kB 00:00 2024-02-17T12:37:11.983 INFO:teuthology.orchestra.run.smithi146.stdout:(81/176): python3-requests-oauthlib-1.3.0-12.el 466 kB/s | 43 kB 00:00 2024-02-17T12:37:12.049 INFO:teuthology.orchestra.run.smithi119.stdout:(6/176): ceph-mon-19.0.0-1515.g6822c7b5.el9.x86 11 MB/s | 4.7 MB 00:00 2024-02-17T12:37:12.133 INFO:teuthology.orchestra.run.smithi018.stdout:(36/176): cephadm-19.0.0-1515.g6822c7b5.el9.noa 4.9 MB/s | 748 kB 00:00 2024-02-17T12:37:12.200 INFO:teuthology.orchestra.run.smithi018.stdout:(37/176): cephfs-top-19.0.0-1515.g6822c7b5.el9. 274 kB/s | 18 kB 00:00 2024-02-17T12:37:12.352 INFO:teuthology.orchestra.run.smithi018.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 662 kB 00:00 2024-02-17T12:37:12.417 INFO:teuthology.orchestra.run.smithi018.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 920 kB 00:00 2024-02-17T12:37:12.459 INFO:teuthology.orchestra.run.smithi018.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2024-02-17T12:37:12.493 INFO:teuthology.orchestra.run.smithi018.stdout:(41/176): elfutils-libelf-devel-0.190-2.el9.x86 609 kB/s | 20 kB 00:00 2024-02-17T12:37:12.543 INFO:teuthology.orchestra.run.smithi018.stdout:(42/176): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-02-17T12:37:12.566 INFO:teuthology.orchestra.run.smithi146.stdout:(82/176): python3-toml-0.10.2-6.el9.noarch.rpm 64 kB/s | 39 kB 00:00 2024-02-17T12:37:12.577 INFO:teuthology.orchestra.run.smithi018.stdout:(43/176): flexiblas-3.0.4-8.el9.x86_64.rpm 945 kB/s | 32 kB 00:00 2024-02-17T12:37:12.583 INFO:teuthology.orchestra.run.smithi146.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 68 kB/s | 41 kB 00:00 2024-02-17T12:37:12.702 INFO:teuthology.orchestra.run.smithi018.stdout:(44/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-02-17T12:37:12.736 INFO:teuthology.orchestra.run.smithi018.stdout:(45/176): flexiblas-openblas-openmp-3.0.4-8.el9 516 kB/s | 17 kB 00:00 2024-02-17T12:37:12.758 INFO:teuthology.orchestra.run.smithi146.stdout:(84/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-02-17T12:37:12.769 INFO:teuthology.orchestra.run.smithi018.stdout:(46/176): libaio-devel-0.3.111-13.el9.x86_64.rp 301 kB/s | 10 kB 00:00 2024-02-17T12:37:12.792 INFO:teuthology.orchestra.run.smithi146.stdout:(85/176): qatlib-23.11.0-1.el9.x86_64.rpm 956 kB/s | 215 kB 00:00 2024-02-17T12:37:12.811 INFO:teuthology.orchestra.run.smithi018.stdout:(47/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-02-17T12:37:12.817 INFO:teuthology.orchestra.run.smithi146.stdout:(86/176): qatlib-service-23.11.0-1.el9.x86_64.r 146 kB/s | 34 kB 00:00 2024-02-17T12:37:12.843 INFO:teuthology.orchestra.run.smithi146.stdout:(87/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 522 kB/s | 43 kB 00:00 2024-02-17T12:37:12.845 INFO:teuthology.orchestra.run.smithi018.stdout:(48/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-02-17T12:37:12.876 INFO:teuthology.orchestra.run.smithi146.stdout:(88/176): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-02-17T12:37:12.895 INFO:teuthology.orchestra.run.smithi018.stdout:(49/176): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-02-17T12:37:12.901 INFO:teuthology.orchestra.run.smithi146.stdout:(89/176): uuid-1.6.2-55.el9.x86_64.rpm 677 kB/s | 56 kB 00:00 2024-02-17T12:37:12.926 INFO:teuthology.orchestra.run.smithi146.stdout:(90/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 87 kB 00:00 2024-02-17T12:37:12.937 INFO:teuthology.orchestra.run.smithi018.stdout:(50/176): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 578 kB 00:00 2024-02-17T12:37:12.952 INFO:teuthology.orchestra.run.smithi146.stdout:(91/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-02-17T12:37:12.970 INFO:teuthology.orchestra.run.smithi018.stdout:(51/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 731 kB/s | 24 kB 00:00 2024-02-17T12:37:12.977 INFO:teuthology.orchestra.run.smithi146.stdout:(92/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-02-17T12:37:12.994 INFO:teuthology.orchestra.run.smithi146.stdout:(93/176): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2024-02-17T12:37:13.004 INFO:teuthology.orchestra.run.smithi018.stdout:(52/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-02-17T12:37:13.011 INFO:teuthology.orchestra.run.smithi146.stdout:(94/176): jq-1.6-16.el9.x86_64.rpm 5.5 MB/s | 186 kB 00:00 2024-02-17T12:37:13.046 INFO:teuthology.orchestra.run.smithi018.stdout:(53/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-02-17T12:37:13.053 INFO:teuthology.orchestra.run.smithi146.stdout:(95/176): libgfortran-11.4.1-3.el9.x86_64.rpm 10 MB/s | 802 kB 00:00 2024-02-17T12:37:13.078 INFO:teuthology.orchestra.run.smithi146.stdout:(96/176): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 194 kB 00:00 2024-02-17T12:37:13.103 INFO:teuthology.orchestra.run.smithi146.stdout:(97/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 217 kB 00:00 2024-02-17T12:37:13.128 INFO:teuthology.orchestra.run.smithi146.stdout:(98/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-02-17T12:37:13.130 INFO:teuthology.orchestra.run.smithi018.stdout:(54/176): ceph-mgr-diskprediction-local-19.0.0- 4.9 MB/s | 7.4 MB 00:01 2024-02-17T12:37:13.140 INFO:teuthology.orchestra.run.smithi119.stdout:(7/176): ceph-osd-19.0.0-1515.g6822c7b5.el9.x86 15 MB/s | 17 MB 00:01 2024-02-17T12:37:13.155 INFO:teuthology.orchestra.run.smithi018.stdout:(55/176): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2024-02-17T12:37:13.161 INFO:teuthology.orchestra.run.smithi146.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 1.7 MB/s | 103 kB 00:00 2024-02-17T12:37:13.181 INFO:teuthology.orchestra.run.smithi018.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-02-17T12:37:13.204 INFO:teuthology.orchestra.run.smithi146.stdout:(100/176): python3-cryptography-36.0.1-4.el9.x8 11 MB/s | 1.1 MB 00:00 2024-02-17T12:37:13.229 INFO:teuthology.orchestra.run.smithi146.stdout:(101/176): python3-pycparser-2.20-6.el9.noarch. 1.2 MB/s | 124 kB 00:00 2024-02-17T12:37:13.254 INFO:teuthology.orchestra.run.smithi146.stdout:(102/176): python3-requests-2.25.1-8.el9.noarch 1.2 MB/s | 113 kB 00:00 2024-02-17T12:37:13.288 INFO:teuthology.orchestra.run.smithi146.stdout:(103/176): python3-urllib3-1.26.5-5.el9.noarch. 3.1 MB/s | 187 kB 00:00 2024-02-17T12:37:13.289 INFO:teuthology.orchestra.run.smithi018.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-02-17T12:37:13.347 INFO:teuthology.orchestra.run.smithi146.stdout:(104/176): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-02-17T12:37:13.347 INFO:teuthology.orchestra.run.smithi018.stdout:(58/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2024-02-17T12:37:13.381 INFO:teuthology.orchestra.run.smithi018.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 17 MB/s | 1.5 MB 00:00 2024-02-17T12:37:13.414 INFO:teuthology.orchestra.run.smithi146.stdout:(105/176): python3-devel-3.9.17-2.el9.x86_64.rp 1.3 MB/s | 245 kB 00:00 2024-02-17T12:37:13.415 INFO:teuthology.orchestra.run.smithi018.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-02-17T12:37:13.440 INFO:teuthology.orchestra.run.smithi018.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-02-17T12:37:13.472 INFO:teuthology.orchestra.run.smithi146.stdout:(106/176): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-02-17T12:37:13.539 INFO:teuthology.orchestra.run.smithi146.stdout:(107/176): abseil-cpp-20211102.0-3.el9.x86_64.r 2.8 MB/s | 548 kB 00:00 2024-02-17T12:37:13.556 INFO:teuthology.orchestra.run.smithi146.stdout:(108/176): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-02-17T12:37:13.565 INFO:teuthology.orchestra.run.smithi018.stdout:(62/176): postgresql-server-13.11-1.el9.x86_64. 38 MB/s | 5.7 MB 00:00 2024-02-17T12:37:13.589 INFO:teuthology.orchestra.run.smithi146.stdout:(109/176): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-02-17T12:37:13.599 INFO:teuthology.orchestra.run.smithi018.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 6.5 MB/s | 1.0 MB 00:00 2024-02-17T12:37:13.615 INFO:teuthology.orchestra.run.smithi146.stdout:(110/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-02-17T12:37:13.624 INFO:teuthology.orchestra.run.smithi018.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-02-17T12:37:13.640 INFO:teuthology.orchestra.run.smithi146.stdout:(111/176): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-02-17T12:37:13.657 INFO:teuthology.orchestra.run.smithi119.stdout:(8/176): ceph-radosgw-19.0.0-1515.g6822c7b5.el9 19 MB/s | 9.8 MB 00:00 2024-02-17T12:37:13.699 INFO:teuthology.orchestra.run.smithi119.stdout:(9/176): ceph-selinux-19.0.0-1515.g6822c7b5.el9 610 kB/s | 25 kB 00:00 2024-02-17T12:37:13.757 INFO:teuthology.orchestra.run.smithi146.stdout:(112/176): libarrow-9.0.0-10.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-02-17T12:37:13.782 INFO:teuthology.orchestra.run.smithi146.stdout:(113/176): libunwind-1.6.2-1.el9.x86_64.rpm 403 kB/s | 67 kB 00:00 2024-02-17T12:37:13.783 INFO:teuthology.orchestra.run.smithi018.stdout:(65/176): python3-babel-2.9.1-2.el9.noarch.rpm 31 MB/s | 5.8 MB 00:00 2024-02-17T12:37:13.808 INFO:teuthology.orchestra.run.smithi146.stdout:(114/176): luarocks-3.9.2-1.el9.noarch.rpm 903 kB/s | 151 kB 00:00 2024-02-17T12:37:13.808 INFO:teuthology.orchestra.run.smithi018.stdout:(66/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.2 MB/s | 227 kB 00:00 2024-02-17T12:37:13.833 INFO:teuthology.orchestra.run.smithi018.stdout:(67/176): python3-jsonpatch-1.21-16.el9.noarch. 490 kB/s | 24 kB 00:00 2024-02-17T12:37:13.849 INFO:teuthology.orchestra.run.smithi146.stdout:(115/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-02-17T12:37:13.859 INFO:teuthology.orchestra.run.smithi018.stdout:(68/176): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-02-17T12:37:13.875 INFO:teuthology.orchestra.run.smithi146.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 652 kB/s | 43 kB 00:00 2024-02-17T12:37:13.884 INFO:teuthology.orchestra.run.smithi018.stdout:(69/176): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-02-17T12:37:13.900 INFO:teuthology.orchestra.run.smithi146.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-02-17T12:37:13.909 INFO:teuthology.orchestra.run.smithi018.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 631 kB/s | 32 kB 00:00 2024-02-17T12:37:13.925 INFO:teuthology.orchestra.run.smithi146.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-02-17T12:37:13.951 INFO:teuthology.orchestra.run.smithi146.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-02-17T12:37:13.984 INFO:teuthology.orchestra.run.smithi146.stdout:(120/176): python3-asyncssh-2.13.2-4.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-02-17T12:37:14.009 INFO:teuthology.orchestra.run.smithi146.stdout:(121/176): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-02-17T12:37:14.026 INFO:teuthology.orchestra.run.smithi018.stdout:(71/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-02-17T12:37:14.034 INFO:teuthology.orchestra.run.smithi146.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 704 kB/s | 35 kB 00:00 2024-02-17T12:37:14.052 INFO:teuthology.orchestra.run.smithi018.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-02-17T12:37:14.060 INFO:teuthology.orchestra.run.smithi146.stdout:(123/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-02-17T12:37:14.077 INFO:teuthology.orchestra.run.smithi018.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-02-17T12:37:14.085 INFO:teuthology.orchestra.run.smithi146.stdout:(124/176): python3-google-auth-2.27.0-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-02-17T12:37:14.102 INFO:teuthology.orchestra.run.smithi018.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-02-17T12:37:14.110 INFO:teuthology.orchestra.run.smithi146.stdout:(125/176): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-02-17T12:37:14.119 INFO:teuthology.orchestra.run.smithi018.stdout:(75/176): python3-prettytable-0.7.2-27.el9.noar 987 kB/s | 41 kB 00:00 2024-02-17T12:37:14.135 INFO:teuthology.orchestra.run.smithi146.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-02-17T12:37:14.144 INFO:teuthology.orchestra.run.smithi018.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-02-17T12:37:14.161 INFO:teuthology.orchestra.run.smithi146.stdout:(127/176): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-02-17T12:37:14.169 INFO:teuthology.orchestra.run.smithi018.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-02-17T12:37:14.186 INFO:teuthology.orchestra.run.smithi146.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-02-17T12:37:14.195 INFO:teuthology.orchestra.run.smithi018.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-02-17T12:37:14.211 INFO:teuthology.orchestra.run.smithi146.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-02-17T12:37:14.220 INFO:teuthology.orchestra.run.smithi018.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 945 kB/s | 47 kB 00:00 2024-02-17T12:37:14.237 INFO:teuthology.orchestra.run.smithi146.stdout:(130/176): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-02-17T12:37:14.245 INFO:teuthology.orchestra.run.smithi018.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 852 kB/s | 43 kB 00:00 2024-02-17T12:37:14.279 INFO:teuthology.orchestra.run.smithi146.stdout:(131/176): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-02-17T12:37:14.296 INFO:teuthology.orchestra.run.smithi146.stdout:(132/176): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-02-17T12:37:14.338 INFO:teuthology.orchestra.run.smithi146.stdout:(133/176): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-02-17T12:37:14.355 INFO:teuthology.orchestra.run.smithi146.stdout:(134/176): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-02-17T12:37:14.380 INFO:teuthology.orchestra.run.smithi146.stdout:(135/176): python3-more-itertools-8.12.0-2.el9. 783 kB/s | 79 kB 00:00 2024-02-17T12:37:14.413 INFO:teuthology.orchestra.run.smithi146.stdout:(136/176): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-02-17T12:37:14.439 INFO:teuthology.orchestra.run.smithi146.stdout:(137/176): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-02-17T12:37:14.464 INFO:teuthology.orchestra.run.smithi146.stdout:(138/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-02-17T12:37:14.487 INFO:teuthology.orchestra.run.smithi018.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 160 kB/s | 39 kB 00:00 2024-02-17T12:37:14.489 INFO:teuthology.orchestra.run.smithi146.stdout:(139/176): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-02-17T12:37:14.514 INFO:teuthology.orchestra.run.smithi146.stdout:(140/176): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-02-17T12:37:14.540 INFO:teuthology.orchestra.run.smithi146.stdout:(141/176): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-02-17T12:37:14.565 INFO:teuthology.orchestra.run.smithi146.stdout:(142/176): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-02-17T12:37:14.590 INFO:teuthology.orchestra.run.smithi146.stdout:(143/176): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-02-17T12:37:14.615 INFO:teuthology.orchestra.run.smithi146.stdout:(144/176): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-02-17T12:37:14.641 INFO:teuthology.orchestra.run.smithi146.stdout:(145/176): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-02-17T12:37:14.663 INFO:teuthology.orchestra.run.smithi018.stdout:(82/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-02-17T12:37:14.666 INFO:teuthology.orchestra.run.smithi146.stdout:(146/176): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-02-17T12:37:14.688 INFO:teuthology.orchestra.run.smithi018.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 207 kB/s | 41 kB 00:00 2024-02-17T12:37:14.691 INFO:teuthology.orchestra.run.smithi146.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-02-17T12:37:14.713 INFO:teuthology.orchestra.run.smithi018.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 215 kB 00:00 2024-02-17T12:37:14.716 INFO:teuthology.orchestra.run.smithi146.stdout:(148/176): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-02-17T12:37:14.739 INFO:teuthology.orchestra.run.smithi018.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 676 kB/s | 34 kB 00:00 2024-02-17T12:37:14.741 INFO:teuthology.orchestra.run.smithi146.stdout:(149/176): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-02-17T12:37:14.764 INFO:teuthology.orchestra.run.smithi018.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 868 kB/s | 43 kB 00:00 2024-02-17T12:37:14.784 INFO:teuthology.orchestra.run.smithi146.stdout:(150/176): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-02-17T12:37:14.789 INFO:teuthology.orchestra.run.smithi018.stdout:(87/176): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-02-17T12:37:14.814 INFO:teuthology.orchestra.run.smithi018.stdout:(88/176): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-02-17T12:37:14.817 INFO:teuthology.orchestra.run.smithi146.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-02-17T12:37:14.857 INFO:teuthology.orchestra.run.smithi018.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 87 kB 00:00 2024-02-17T12:37:14.881 INFO:teuthology.orchestra.run.smithi018.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 940 kB/s | 63 kB 00:00 2024-02-17T12:37:14.907 INFO:teuthology.orchestra.run.smithi018.stdout:(91/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 902 kB/s | 44 kB 00:00 2024-02-17T12:37:14.926 INFO:teuthology.orchestra.run.smithi146.stdout:(152/176): NetworkManager-1.45.91-1.el9.x86_64. 21 MB/s | 2.3 MB 00:00 2024-02-17T12:37:14.932 INFO:teuthology.orchestra.run.smithi018.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-02-17T12:37:14.957 INFO:teuthology.orchestra.run.smithi018.stdout:(93/176): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-02-17T12:37:14.991 INFO:teuthology.orchestra.run.smithi018.stdout:(94/176): libgfortran-11.4.1-3.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-02-17T12:37:15.016 INFO:teuthology.orchestra.run.smithi018.stdout:(95/176): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 194 kB 00:00 2024-02-17T12:37:15.034 INFO:teuthology.orchestra.run.smithi146.stdout:(153/176): NetworkManager-libnm-1.45.91-1.el9.x 17 MB/s | 1.8 MB 00:00 2024-02-17T12:37:15.041 INFO:teuthology.orchestra.run.smithi018.stdout:(96/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-02-17T12:37:15.067 INFO:teuthology.orchestra.run.smithi018.stdout:(97/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-17T12:37:15.067 INFO:teuthology.orchestra.run.smithi146.stdout:(154/176): NetworkManager-team-1.45.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-02-17T12:37:15.109 INFO:teuthology.orchestra.run.smithi018.stdout:(98/176): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-02-17T12:37:15.109 INFO:teuthology.orchestra.run.smithi146.stdout:(155/176): NetworkManager-tui-1.45.91-1.el9.x86 5.8 MB/s | 246 kB 00:00 2024-02-17T12:37:15.135 INFO:teuthology.orchestra.run.smithi018.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-02-17T12:37:15.151 INFO:teuthology.orchestra.run.smithi146.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 69 kB 00:00 2024-02-17T12:37:15.159 INFO:teuthology.orchestra.run.smithi018.stdout:(100/176): python3-pycparser-2.20-6.el9.noarch. 2.4 MB/s | 124 kB 00:00 2024-02-17T12:37:15.185 INFO:teuthology.orchestra.run.smithi018.stdout:(101/176): python3-requests-2.25.1-8.el9.noarch 2.2 MB/s | 113 kB 00:00 2024-02-17T12:37:15.185 INFO:teuthology.orchestra.run.smithi146.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.0 MB/s | 35 kB 00:00 2024-02-17T12:37:15.210 INFO:teuthology.orchestra.run.smithi018.stdout:(102/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-02-17T12:37:15.227 INFO:teuthology.orchestra.run.smithi146.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 191 kB 00:00 2024-02-17T12:37:15.269 INFO:teuthology.orchestra.run.smithi146.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 254 kB 00:00 2024-02-17T12:37:15.294 INFO:teuthology.orchestra.run.smithi018.stdout:(103/176): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-02-17T12:37:15.336 INFO:teuthology.orchestra.run.smithi146.stdout:(160/176): iproute-6.7.0-2.el9.x86_64.rpm 12 MB/s | 802 kB 00:00 2024-02-17T12:37:15.352 INFO:teuthology.orchestra.run.smithi018.stdout:(104/176): python3-devel-3.9.17-2.el9.x86_64.rp 1.4 MB/s | 245 kB 00:00 2024-02-17T12:37:15.386 INFO:teuthology.orchestra.run.smithi146.stdout:(161/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 8.5 MB/s | 433 kB 00:00 2024-02-17T12:37:15.419 INFO:teuthology.orchestra.run.smithi146.stdout:(162/176): libacl-2.3.1-4.el9.x86_64.rpm 664 kB/s | 22 kB 00:00 2024-02-17T12:37:15.453 INFO:teuthology.orchestra.run.smithi018.stdout:(105/176): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-02-17T12:37:15.461 INFO:teuthology.orchestra.run.smithi146.stdout:(163/176): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 107 kB 00:00 2024-02-17T12:37:15.495 INFO:teuthology.orchestra.run.smithi146.stdout:(164/176): libfdisk-2.37.4-18.el9.x86_64.rpm 4.5 MB/s | 154 kB 00:00 2024-02-17T12:37:15.537 INFO:teuthology.orchestra.run.smithi146.stdout:(165/176): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 135 kB 00:00 2024-02-17T12:37:15.570 INFO:teuthology.orchestra.run.smithi146.stdout:(166/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.9 MB/s | 63 kB 00:00 2024-02-17T12:37:15.604 INFO:teuthology.orchestra.run.smithi146.stdout:(167/176): libuuid-2.37.4-18.el9.x86_64.rpm 831 kB/s | 28 kB 00:00 2024-02-17T12:37:15.688 INFO:teuthology.orchestra.run.smithi146.stdout:(168/176): openssl-3.0.7-25.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2024-02-17T12:37:15.813 INFO:teuthology.orchestra.run.smithi146.stdout:(169/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 17 MB/s | 2.1 MB 00:00 2024-02-17T12:37:15.972 INFO:teuthology.orchestra.run.smithi146.stdout:(170/176): util-linux-2.37.4-18.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-02-17T12:37:16.019 INFO:teuthology.orchestra.run.smithi018.stdout:(106/176): gperftools-libs-2.9.1-2.el9.x86_64.r 573 kB/s | 309 kB 00:00 2024-02-17T12:37:16.047 INFO:teuthology.orchestra.run.smithi146.stdout:(171/176): util-linux-core-2.37.4-18.el9.x86_64 5.7 MB/s | 438 kB 00:00 2024-02-17T12:37:16.095 INFO:teuthology.orchestra.run.smithi018.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 260 kB/s | 19 kB 00:00 2024-02-17T12:37:16.097 INFO:teuthology.orchestra.run.smithi146.stdout:(172/176): util-linux-user-2.37.4-18.el9.x86_64 602 kB/s | 30 kB 00:00 2024-02-17T12:37:16.120 INFO:teuthology.orchestra.run.smithi018.stdout:(108/176): abseil-cpp-20211102.0-3.el9.x86_64.r 714 kB/s | 548 kB 00:00 2024-02-17T12:37:16.173 INFO:teuthology.orchestra.run.smithi146.stdout:(173/176): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-02-17T12:37:16.195 INFO:teuthology.orchestra.run.smithi018.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-02-17T12:37:16.279 INFO:teuthology.orchestra.run.smithi018.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 584 kB/s | 49 kB 00:00 2024-02-17T12:37:16.298 INFO:teuthology.orchestra.run.smithi146.stdout:(174/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.5 MB/s | 1.1 MB 00:00 2024-02-17T12:37:16.348 INFO:teuthology.orchestra.run.smithi146.stdout:(175/176): librados2-19.0.0-1515.g6822c7b5.el9. 2.1 MB/s | 3.4 MB 00:01 2024-02-17T12:37:16.406 INFO:teuthology.orchestra.run.smithi146.stdout:(176/176): librbd1-19.0.0-1515.g6822c7b5.el9.x8 1.9 MB/s | 3.1 MB 00:01 2024-02-17T12:37:16.411 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-02-17T12:37:16.411 INFO:teuthology.orchestra.run.smithi146.stdout:Total 15 MB/s | 223 MB 00:14 2024-02-17T12:37:16.421 INFO:teuthology.orchestra.run.smithi018.stdout:(111/176): libunwind-1.6.2-1.el9.x86_64.rpm 475 kB/s | 67 kB 00:00 2024-02-17T12:37:16.461 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-02-17T12:37:16.629 INFO:teuthology.orchestra.run.smithi018.stdout:(112/176): luarocks-3.9.2-1.el9.noarch.rpm 726 kB/s | 151 kB 00:00 2024-02-17T12:37:16.738 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-02-17T12:37:16.739 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-02-17T12:37:16.882 INFO:teuthology.orchestra.run.smithi119.stdout:(10/176): ceph-test-19.0.0-1515.g6822c7b5.el9.x 15 MB/s | 47 MB 00:03 2024-02-17T12:37:16.996 INFO:teuthology.orchestra.run.smithi018.stdout:(113/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.2 MB/s | 838 kB 00:00 2024-02-17T12:37:17.007 INFO:teuthology.orchestra.run.smithi119.stdout:(11/176): cephfs-mirror-19.0.0-1515.g6822c7b5.e 1.7 MB/s | 217 kB 00:00 2024-02-17T12:37:17.083 INFO:teuthology.orchestra.run.smithi119.stdout:(12/176): cephfs-shell-19.0.0-1515.g6822c7b5.el 361 kB/s | 27 kB 00:00 2024-02-17T12:37:17.158 INFO:teuthology.orchestra.run.smithi119.stdout:(13/176): libcephfs-devel-19.0.0-1515.g6822c7b5 416 kB/s | 31 kB 00:00 2024-02-17T12:37:17.317 INFO:teuthology.orchestra.run.smithi119.stdout:(14/176): libcephfs2-19.0.0-1515.g6822c7b5.el9. 4.6 MB/s | 743 kB 00:00 2024-02-17T12:37:17.400 INFO:teuthology.orchestra.run.smithi119.stdout:(15/176): libcephsqlite-19.0.0-1515.g6822c7b5.e 1.9 MB/s | 163 kB 00:00 2024-02-17T12:37:17.484 INFO:teuthology.orchestra.run.smithi119.stdout:(16/176): librados-devel-19.0.0-1515.g6822c7b5. 1.5 MB/s | 127 kB 00:00 2024-02-17T12:37:17.576 INFO:teuthology.orchestra.run.smithi119.stdout:(17/176): libradosstriper1-19.0.0-1515.g6822c7b 5.3 MB/s | 499 kB 00:00 2024-02-17T12:37:17.588 INFO:teuthology.orchestra.run.smithi018.stdout:(114/176): python3-asyncssh-2.13.2-4.el9.noarch 926 kB/s | 548 kB 00:00 2024-02-17T12:37:17.988 INFO:teuthology.orchestra.run.smithi018.stdout:(115/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 109 kB/s | 43 kB 00:00 2024-02-17T12:37:18.089 INFO:teuthology.orchestra.run.smithi018.stdout:(116/176): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-02-17T12:37:18.189 INFO:teuthology.orchestra.run.smithi018.stdout:(117/176): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-02-17T12:37:18.209 INFO:teuthology.orchestra.run.smithi119.stdout:(18/176): librgw2-19.0.0-1515.g6822c7b5.el9.x86 8.1 MB/s | 5.1 MB 00:00 2024-02-17T12:37:18.268 INFO:teuthology.orchestra.run.smithi119.stdout:(19/176): python3-ceph-argparse-19.0.0-1515.g68 779 kB/s | 45 kB 00:00 2024-02-17T12:37:18.343 INFO:teuthology.orchestra.run.smithi119.stdout:(20/176): python3-ceph-common-19.0.0-1515.g6822 1.7 MB/s | 131 kB 00:00 2024-02-17T12:37:18.415 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-02-17T12:37:18.416 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-02-17T12:37:18.418 INFO:teuthology.orchestra.run.smithi119.stdout:(21/176): python3-cephfs-19.0.0-1515.g6822c7b5. 2.1 MB/s | 162 kB 00:00 2024-02-17T12:37:18.510 INFO:teuthology.orchestra.run.smithi119.stdout:(22/176): python3-rados-19.0.0-1515.g6822c7b5.e 3.4 MB/s | 321 kB 00:00 2024-02-17T12:37:18.597 INFO:teuthology.orchestra.run.smithi018.stdout:(118/176): python3-cheroot-8.6.0-4.el9.noarch.r 421 kB/s | 172 kB 00:00 2024-02-17T12:37:18.602 INFO:teuthology.orchestra.run.smithi119.stdout:(23/176): python3-rbd-19.0.0-1515.g6822c7b5.el9 3.2 MB/s | 298 kB 00:00 2024-02-17T12:37:18.678 INFO:teuthology.orchestra.run.smithi119.stdout:(24/176): python3-rgw-19.0.0-1515.g6822c7b5.el9 1.3 MB/s | 99 kB 00:00 2024-02-17T12:37:18.753 INFO:teuthology.orchestra.run.smithi119.stdout:(25/176): rbd-fuse-19.0.0-1515.g6822c7b5.el9.x8 1.1 MB/s | 85 kB 00:00 2024-02-17T12:37:19.153 INFO:teuthology.orchestra.run.smithi119.stdout:(26/176): rbd-mirror-19.0.0-1515.g6822c7b5.el9. 7.7 MB/s | 3.1 MB 00:00 2024-02-17T12:37:19.237 INFO:teuthology.orchestra.run.smithi119.stdout:(27/176): rbd-nbd-19.0.0-1515.g6822c7b5.el9.x86 2.0 MB/s | 171 kB 00:00 2024-02-17T12:37:19.296 INFO:teuthology.orchestra.run.smithi119.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1515.g 537 kB/s | 31 kB 00:00 2024-02-17T12:37:19.354 INFO:teuthology.orchestra.run.smithi119.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1515.g6822c7b 2.3 MB/s | 138 kB 00:00 2024-02-17T12:37:19.588 INFO:teuthology.orchestra.run.smithi119.stdout:(30/176): ceph-mgr-dashboard-19.0.0-1515.g6822c 7.6 MB/s | 1.8 MB 00:00 2024-02-17T12:37:20.455 INFO:teuthology.orchestra.run.smithi119.stdout:(31/176): ceph-mgr-diskprediction-local-19.0.0- 8.5 MB/s | 7.4 MB 00:00 2024-02-17T12:37:20.522 INFO:teuthology.orchestra.run.smithi119.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1515.g68 3.6 MB/s | 247 kB 00:00 2024-02-17T12:37:20.580 INFO:teuthology.orchestra.run.smithi119.stdout:(33/176): ceph-mgr-rook-19.0.0-1515.g6822c7b5.e 897 kB/s | 52 kB 00:00 2024-02-17T12:37:20.639 INFO:teuthology.orchestra.run.smithi119.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1515.g6 262 kB/s | 15 kB 00:00 2024-02-17T12:37:20.706 INFO:teuthology.orchestra.run.smithi119.stdout:(35/176): ceph-volume-19.0.0-1515.g6822c7b5.el9 3.9 MB/s | 264 kB 00:00 2024-02-17T12:37:20.831 INFO:teuthology.orchestra.run.smithi119.stdout:(36/176): cephadm-19.0.0-1515.g6822c7b5.el9.noa 5.9 MB/s | 748 kB 00:00 2024-02-17T12:37:20.890 INFO:teuthology.orchestra.run.smithi119.stdout:(37/176): cephfs-top-19.0.0-1515.g6822c7b5.el9. 313 kB/s | 18 kB 00:00 2024-02-17T12:37:21.372 INFO:teuthology.orchestra.run.smithi018.stdout:(119/176): ceph-test-19.0.0-1515.g6822c7b5.el9. 4.1 MB/s | 47 MB 00:11 2024-02-17T12:37:21.548 INFO:teuthology.orchestra.run.smithi018.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 121 kB/s | 358 kB 00:02 2024-02-17T12:37:21.864 INFO:teuthology.orchestra.run.smithi018.stdout:(121/176): python3-colorama-0.4.4-12.el9.noarch 111 kB/s | 35 kB 00:00 2024-02-17T12:37:22.215 INFO:teuthology.orchestra.run.smithi119.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 500 kB/s | 662 kB 00:01 2024-02-17T12:37:23.482 INFO:teuthology.orchestra.run.smithi119.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 727 kB/s | 920 kB 00:01 2024-02-17T12:37:23.632 INFO:teuthology.orchestra.run.smithi119.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 705 kB/s | 106 kB 00:00 2024-02-17T12:37:23.724 INFO:teuthology.orchestra.run.smithi119.stdout:(41/176): elfutils-libelf-devel-0.190-2.el9.x86 223 kB/s | 20 kB 00:00 2024-02-17T12:37:23.990 INFO:teuthology.orchestra.run.smithi018.stdout:(122/176): python3-google-auth-2.27.0-1.el9.noa 95 kB/s | 203 kB 00:02 2024-02-17T12:37:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:(42/176): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 307 kB 00:00 2024-02-17T12:37:24.141 INFO:teuthology.orchestra.run.smithi119.stdout:(43/176): flexiblas-3.0.4-8.el9.x86_64.rpm 237 kB/s | 32 kB 00:00 2024-02-17T12:37:24.706 INFO:teuthology.orchestra.run.smithi018.stdout:(123/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 91 kB/s | 303 kB 00:03 2024-02-17T12:37:25.343 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-02-17T12:37:25.452 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-17T12:37:25.486 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-17T12:37:25.517 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-17T12:37:25.539 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-17T12:37:25.587 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-17T12:37:25.625 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-17T12:37:25.678 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-17T12:37:25.720 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-17T12:37:25.733 INFO:teuthology.orchestra.run.smithi119.stdout:(44/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.9 MB/s | 3.0 MB 00:01 2024-02-17T12:37:25.825 INFO:teuthology.orchestra.run.smithi119.stdout:(45/176): flexiblas-openblas-openmp-3.0.4-8.el9 187 kB/s | 17 kB 00:00 2024-02-17T12:37:25.838 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-17T12:37:25.916 INFO:teuthology.orchestra.run.smithi119.stdout:(46/176): libaio-devel-0.3.111-13.el9.x86_64.rp 110 kB/s | 10 kB 00:00 2024-02-17T12:37:25.932 INFO:teuthology.orchestra.run.smithi018.stdout:(124/176): python3-grpcio-tools-1.46.7-10.el9.x 118 kB/s | 144 kB 00:01 2024-02-17T12:37:25.991 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-17T12:37:26.033 INFO:teuthology.orchestra.run.smithi119.stdout:(47/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-02-17T12:37:26.090 INFO:teuthology.orchestra.run.smithi018.stdout:(125/176): python3-jaraco-8.2.1-3.el9.noarch.rp 68 kB/s | 11 kB 00:00 2024-02-17T12:37:26.100 INFO:teuthology.orchestra.run.smithi119.stdout:(48/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 661 kB/s | 44 kB 00:00 2024-02-17T12:37:26.199 INFO:teuthology.orchestra.run.smithi018.stdout:(126/176): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-02-17T12:37:26.316 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-17T12:37:26.332 INFO:teuthology.orchestra.run.smithi018.stdout:(127/176): python3-jaraco-collections-3.0.0-8.e 174 kB/s | 23 kB 00:00 2024-02-17T12:37:26.359 INFO:teuthology.orchestra.run.smithi119.stdout:(49/176): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 661 kB 00:00 2024-02-17T12:37:26.372 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T12:37:26.474 INFO:teuthology.orchestra.run.smithi018.stdout:(128/176): python3-jaraco-functools-3.5.0-2.el9 137 kB/s | 19 kB 00:00 2024-02-17T12:37:26.493 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T12:37:26.515 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T12:37:26.605 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T12:37:26.618 INFO:teuthology.orchestra.run.smithi119.stdout:(50/176): libtool-2.4.6-45.el9.x86_64.rpm 2.2 MB/s | 578 kB 00:00 2024-02-17T12:37:26.633 INFO:teuthology.orchestra.run.smithi018.stdout:(129/176): python3-jaraco-text-3.2.0-6.el9.noar 124 kB/s | 20 kB 00:00 2024-02-17T12:37:26.635 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 13/202 2024-02-17T12:37:26.701 INFO:teuthology.orchestra.run.smithi119.stdout:(51/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 293 kB/s | 24 kB 00:00 2024-02-17T12:37:26.783 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-17T12:37:26.793 INFO:teuthology.orchestra.run.smithi119.stdout:(52/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 506 kB/s | 46 kB 00:00 2024-02-17T12:37:26.832 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-17T12:37:26.852 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T12:37:26.935 INFO:teuthology.orchestra.run.smithi119.stdout:(53/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 282 kB 00:00 2024-02-17T12:37:26.954 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T12:37:26.978 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 17/202 2024-02-17T12:37:27.042 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 17/202 2024-02-17T12:37:27.052 INFO:teuthology.orchestra.run.smithi119.stdout:(54/176): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2024-02-17T12:37:27.082 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-17T12:37:27.105 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-17T12:37:27.121 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-17T12:37:27.144 INFO:teuthology.orchestra.run.smithi119.stdout:(55/176): openblas-0.3.21-2.el9.x86_64.rpm 371 kB/s | 34 kB 00:00 2024-02-17T12:37:27.305 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-17T12:37:27.344 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-17T12:37:27.375 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-17T12:37:27.480 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-17T12:37:27.577 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-17T12:37:27.637 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-17T12:37:27.713 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T12:37:27.788 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T12:37:28.013 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T12:37:28.061 INFO:teuthology.orchestra.run.smithi119.stdout:(56/176): ceph-base-19.0.0-1515.g6822c7b5.el9.x 313 kB/s | 5.3 MB 00:17 2024-02-17T12:37:28.119 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-17T12:37:28.230 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-17T12:37:28.286 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-17T12:37:28.354 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-17T12:37:28.407 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-17T12:37:28.438 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-17T12:37:28.474 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-17T12:37:28.514 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-17T12:37:28.578 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-17T12:37:28.615 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-17T12:37:28.659 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-17T12:37:28.691 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-17T12:37:28.719 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-17T12:37:28.787 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-17T12:37:28.824 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9. 39/202 2024-02-17T12:37:28.867 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el 40/202 2024-02-17T12:37:28.896 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_6 41/202 2024-02-17T12:37:28.920 INFO:teuthology.orchestra.run.smithi119.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 2.7 MB/s | 4.7 MB 00:01 2024-02-17T12:37:28.965 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-17T12:37:29.010 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-17T12:37:29.083 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-17T12:37:29.226 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-17T12:37:29.279 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-17T12:37:29.553 INFO:teuthology.orchestra.run.smithi119.stdout:(58/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 2.0 MB/s | 3.0 MB 00:01 2024-02-17T12:37:29.591 INFO:teuthology.orchestra.run.smithi018.stdout:(130/176): libarrow-9.0.0-10.el9.x86_64.rpm 335 kB/s | 4.4 MB 00:13 2024-02-17T12:37:29.804 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-17T12:37:29.845 INFO:teuthology.orchestra.run.smithi119.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 1.6 MB/s | 1.5 MB 00:00 2024-02-17T12:37:29.899 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-17T12:37:29.996 INFO:teuthology.orchestra.run.smithi119.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 1.8 MB/s | 811 kB 00:00 2024-02-17T12:37:30.000 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-17T12:37:30.044 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-17T12:37:30.053 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-17T12:37:30.071 INFO:teuthology.orchestra.run.smithi119.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 595 kB/s | 134 kB 00:00 2024-02-17T12:37:30.091 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-17T12:37:30.129 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_6 52/202 2024-02-17T12:37:30.160 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-17T12:37:30.249 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-17T12:37:30.311 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-17T12:37:30.546 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-17T12:37:30.706 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-17T12:37:30.755 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-17T12:37:30.799 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-17T12:37:30.825 INFO:teuthology.orchestra.run.smithi018.stdout:(131/176): python3-logutils-0.3.5-21.el9.noarch 38 kB/s | 46 kB 00:01 2024-02-17T12:37:30.938 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-17T12:37:30.962 INFO:teuthology.orchestra.run.smithi119.stdout:(62/176): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-02-17T12:37:30.993 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-17T12:37:31.048 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-17T12:37:31.112 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-17T12:37:31.144 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-17T12:37:31.180 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-17T12:37:31.227 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-17T12:37:31.263 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-17T12:37:31.306 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-17T12:37:31.348 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-17T12:37:31.513 INFO:teuthology.orchestra.run.smithi119.stdout:(63/176): python3-attrs-20.3.0-7.el9.noarch.rpm 141 kB/s | 78 kB 00:00 2024-02-17T12:37:31.558 INFO:teuthology.orchestra.run.smithi018.stdout:(132/176): python3-more-itertools-8.12.0-2.el9. 108 kB/s | 79 kB 00:00 2024-02-17T12:37:31.653 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-17T12:37:31.696 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-17T12:37:31.725 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-17T12:37:31.767 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-17T12:37:31.860 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-17T12:37:31.909 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-17T12:37:31.946 INFO:teuthology.orchestra.run.smithi119.stdout:(64/176): postgresql-server-13.11-1.el9.x86_64. 2.9 MB/s | 5.7 MB 00:01 2024-02-17T12:37:31.948 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-17T12:37:31.974 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-17T12:37:31.999 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-17T12:37:32.071 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-17T12:37:32.198 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-17T12:37:32.200 INFO:teuthology.orchestra.run.smithi018.stdout:(133/176): python3-natsort-7.1.1-5.el9.noarch.r 90 kB/s | 58 kB 00:00 2024-02-17T12:37:32.297 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-17T12:37:32.347 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-17T12:37:32.380 INFO:teuthology.orchestra.run.smithi119.stdout:(65/176): python3-jinja2-2.11.3-5.el9.noarch.rp 524 kB/s | 227 kB 00:00 2024-02-17T12:37:32.396 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-17T12:37:32.459 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-17T12:37:32.643 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-17T12:37:32.684 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-17T12:37:32.700 INFO:teuthology.orchestra.run.smithi018.stdout:(134/176): python3-kubernetes-26.1.0-2.el9.noar 172 kB/s | 1.0 MB 00:06 2024-02-17T12:37:32.812 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-17T12:37:32.863 INFO:teuthology.orchestra.run.smithi119.stdout:(66/176): python3-jsonpatch-1.21-16.el9.noarch. 50 kB/s | 24 kB 00:00 2024-02-17T12:37:32.865 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-17T12:37:32.916 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-17T12:37:32.947 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-17T12:37:32.982 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-17T12:37:33.013 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-17T12:37:33.041 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-17T12:37:33.075 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-17T12:37:33.115 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-17T12:37:33.146 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-17T12:37:33.176 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-17T12:37:33.253 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-17T12:37:33.286 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-17T12:37:33.319 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-17T12:37:33.339 INFO:teuthology.orchestra.run.smithi119.stdout:(67/176): python3-jsonpointer-2.0-4.el9.noarch. 38 kB/s | 18 kB 00:00 2024-02-17T12:37:33.361 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-17T12:37:33.375 INFO:teuthology.orchestra.run.smithi018.stdout:(135/176): python3-portend-3.1.0-2.el9.noarch.r 24 kB/s | 16 kB 00:00 2024-02-17T12:37:33.395 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-17T12:37:33.447 INFO:teuthology.orchestra.run.smithi119.stdout:(68/176): python3-babel-2.9.1-2.el9.noarch.rpm 3.0 MB/s | 5.8 MB 00:01 2024-02-17T12:37:33.483 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-17T12:37:33.523 INFO:teuthology.orchestra.run.smithi119.stdout:(69/176): python3-mako-1.1.4-6.el9.noarch.rpm 788 kB/s | 144 kB 00:00 2024-02-17T12:37:33.574 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-17T12:37:33.618 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-17T12:37:33.649 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-17T12:37:33.665 INFO:teuthology.orchestra.run.smithi119.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 145 kB/s | 32 kB 00:00 2024-02-17T12:37:33.696 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-17T12:37:33.793 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-17T12:37:33.842 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-17T12:37:34.290 INFO:teuthology.orchestra.run.smithi119.stdout:(71/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 348 kB/s | 217 kB 00:00 2024-02-17T12:37:34.568 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-17T12:37:34.584 INFO:teuthology.orchestra.run.smithi018.stdout:(136/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 74 kB/s | 90 kB 00:01 2024-02-17T12:37:34.614 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-17T12:37:34.648 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-17T12:37:34.696 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-17T12:37:34.698 INFO:teuthology.orchestra.run.smithi119.stdout:(72/176): python3-oauthlib-3.1.1-5.el9.noarch.r 428 kB/s | 174 kB 00:00 2024-02-17T12:37:34.784 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-17T12:37:34.815 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-17T12:37:34.900 INFO:teuthology.orchestra.run.smithi018.stdout:(137/176): python3-pyperclip-1.8.2-2.el9.noarch 89 kB/s | 28 kB 00:00 2024-02-17T12:37:34.993 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-17T12:37:35.022 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-17T12:37:35.105 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-17T12:37:35.126 INFO:teuthology.orchestra.run.smithi018.stdout:(138/176): python3-pecan-1.4.2-3.el9.noarch.rpm 93 kB/s | 272 kB 00:02 2024-02-17T12:37:35.151 INFO:teuthology.orchestra.run.smithi018.stdout:(139/176): python3-repoze-lru-0.7-16.el9.noarch 123 kB/s | 31 kB 00:00 2024-02-17T12:37:35.273 INFO:teuthology.orchestra.run.smithi119.stdout:(73/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.7 MB/s | 4.8 MB 00:01 2024-02-17T12:37:35.298 INFO:teuthology.orchestra.run.smithi119.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 114 kB/s | 69 kB 00:00 2024-02-17T12:37:35.474 INFO:teuthology.orchestra.run.smithi119.stdout:(75/176): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 238 kB 00:00 2024-02-17T12:37:35.532 INFO:teuthology.orchestra.run.smithi119.stdout:(76/176): python3-prettytable-0.7.2-27.el9.noar 176 kB/s | 41 kB 00:00 2024-02-17T12:37:35.691 INFO:teuthology.orchestra.run.smithi119.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 609 kB/s | 132 kB 00:00 2024-02-17T12:37:35.787 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-17T12:37:35.793 INFO:teuthology.orchestra.run.smithi018.stdout:(140/176): python3-rsa-4.9-2.el9.noarch.rpm 92 kB/s | 59 kB 00:00 2024-02-17T12:37:35.825 INFO:teuthology.orchestra.run.smithi119.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 721 kB/s | 211 kB 00:00 2024-02-17T12:37:35.851 INFO:teuthology.orchestra.run.smithi119.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 298 kB/s | 47 kB 00:00 2024-02-17T12:37:35.877 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-17T12:37:35.917 INFO:teuthology.orchestra.run.smithi119.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 466 kB/s | 43 kB 00:00 2024-02-17T12:37:36.328 INFO:teuthology.orchestra.run.smithi018.stdout:(141/176): python3-tempora-5.0.0-2.el9.noarch.r 67 kB/s | 36 kB 00:00 2024-02-17T12:37:37.084 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-17T12:37:37.158 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-17T12:37:37.168 INFO:teuthology.orchestra.run.smithi018.stdout:(142/176): python3-routes-2.5.1-5.el9.noarch.rp 92 kB/s | 188 kB 00:02 2024-02-17T12:37:37.185 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-17T12:37:37.275 INFO:teuthology.orchestra.run.smithi119.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 28 kB/s | 39 kB 00:01 2024-02-17T12:37:37.493 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-17T12:37:37.523 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 125/202 2024-02-17T12:37:37.612 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 125/202 2024-02-17T12:37:37.644 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 126/202 2024-02-17T12:37:37.680 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-17T12:37:37.685 INFO:teuthology.orchestra.run.smithi018.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 38 kB/s | 51 kB 00:01 2024-02-17T12:37:37.717 INFO:teuthology.orchestra.run.smithi119.stdout:(82/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 93 kB/s | 41 kB 00:00 2024-02-17T12:37:38.143 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-17T12:37:38.151 INFO:teuthology.orchestra.run.smithi119.stdout:(83/176): qatlib-23.11.0-1.el9.x86_64.rpm 498 kB/s | 215 kB 00:00 2024-02-17T12:37:38.358 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 129/202 2024-02-17T12:37:38.385 INFO:teuthology.orchestra.run.smithi018.stdout:(144/176): python3-websocket-client-1.2.3-2.el9 128 kB/s | 90 kB 00:00 2024-02-17T12:37:38.484 INFO:teuthology.orchestra.run.smithi119.stdout:(84/176): qatlib-service-23.11.0-1.el9.x86_64.r 101 kB/s | 34 kB 00:00 2024-02-17T12:37:38.555 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 129/202 2024-02-17T12:37:38.586 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 130/202 2024-02-17T12:37:38.818 INFO:teuthology.orchestra.run.smithi119.stdout:(85/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 130 kB/s | 43 kB 00:00 2024-02-17T12:37:38.919 INFO:teuthology.orchestra.run.smithi018.stdout:(145/176): python3-webob-1.8.7-6.el9.noarch.rpm 131 kB/s | 230 kB 00:01 2024-02-17T12:37:39.061 INFO:teuthology.orchestra.run.smithi018.stdout:(146/176): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-02-17T12:37:39.152 INFO:teuthology.orchestra.run.smithi119.stdout:(86/176): socat-1.7.4.1-5.el9.x86_64.rpm 901 kB/s | 300 kB 00:00 2024-02-17T12:37:39.178 INFO:teuthology.orchestra.run.smithi018.stdout:(147/176): python3-grpcio-1.46.7-10.el9.x86_64. 137 kB/s | 2.0 MB 00:15 2024-02-17T12:37:39.394 INFO:teuthology.orchestra.run.smithi119.stdout:(87/176): uuid-1.6.2-55.el9.x86_64.rpm 233 kB/s | 56 kB 00:00 2024-02-17T12:37:39.620 INFO:teuthology.orchestra.run.smithi018.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 343 kB/s | 192 kB 00:00 2024-02-17T12:37:39.686 INFO:teuthology.orchestra.run.smithi119.stdout:(88/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.1 MB/s | 16 MB 00:03 2024-02-17T12:37:39.711 INFO:teuthology.orchestra.run.smithi119.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 275 kB/s | 87 kB 00:00 2024-02-17T12:37:39.711 INFO:teuthology.orchestra.run.smithi018.stdout:(149/176): python3-werkzeug-2.0.3-3.el9.1.noarc 322 kB/s | 427 kB 00:01 2024-02-17T12:37:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 945 kB/s | 63 kB 00:00 2024-02-17T12:37:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:(91/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 659 kB/s | 44 kB 00:00 2024-02-17T12:37:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 79 kB 00:00 2024-02-17T12:37:39.854 INFO:teuthology.orchestra.run.smithi119.stdout:(93/176): jq-1.6-16.el9.x86_64.rpm 2.4 MB/s | 186 kB 00:00 2024-02-17T12:37:39.971 INFO:teuthology.orchestra.run.smithi119.stdout:(94/176): libgfortran-11.4.1-3.el9.x86_64.rpm 5.2 MB/s | 802 kB 00:00 2024-02-17T12:37:39.997 INFO:teuthology.orchestra.run.smithi119.stdout:(95/176): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 194 kB 00:00 2024-02-17T12:37:40.038 INFO:teuthology.orchestra.run.smithi119.stdout:(96/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-02-17T12:37:40.072 INFO:teuthology.orchestra.run.smithi119.stdout:(97/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-02-17T12:37:40.095 INFO:teuthology.orchestra.run.smithi018.stdout:(150/176): librados2-19.0.0-1515.g6822c7b5.el9. 7.1 MB/s | 3.4 MB 00:00 2024-02-17T12:37:40.153 INFO:teuthology.orchestra.run.smithi018.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 68 kB 00:00 2024-02-17T12:37:40.222 INFO:teuthology.orchestra.run.smithi119.stdout:(98/176): python3-cryptography-36.0.1-4.el9.x86 6.2 MB/s | 1.1 MB 00:00 2024-02-17T12:37:40.237 INFO:teuthology.orchestra.run.smithi018.stdout:(152/176): librbd1-19.0.0-1515.g6822c7b5.el9.x8 6.0 MB/s | 3.1 MB 00:00 2024-02-17T12:37:40.247 INFO:teuthology.orchestra.run.smithi119.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 587 kB/s | 103 kB 00:00 2024-02-17T12:37:40.282 INFO:teuthology.orchestra.run.smithi119.stdout:(100/176): python3-pycparser-2.20-6.el9.noarch. 2.1 MB/s | 124 kB 00:00 2024-02-17T12:37:40.296 INFO:teuthology.orchestra.run.smithi018.stdout:(153/176): NetworkManager-1.45.91-1.el9.x86_64. 16 MB/s | 2.3 MB 00:00 2024-02-17T12:37:40.323 INFO:teuthology.orchestra.run.smithi119.stdout:(101/176): python3-requests-2.25.1-8.el9.noarch 1.5 MB/s | 113 kB 00:00 2024-02-17T12:37:40.356 INFO:teuthology.orchestra.run.smithi119.stdout:(102/176): python3-urllib3-1.26.5-5.el9.noarch. 2.4 MB/s | 187 kB 00:00 2024-02-17T12:37:40.371 INFO:teuthology.orchestra.run.smithi018.stdout:(154/176): NetworkManager-libnm-1.45.91-1.el9.x 14 MB/s | 1.8 MB 00:00 2024-02-17T12:37:40.396 INFO:teuthology.orchestra.run.smithi018.stdout:(155/176): NetworkManager-team-1.45.91-1.el9.x8 423 kB/s | 41 kB 00:00 2024-02-17T12:37:40.430 INFO:teuthology.orchestra.run.smithi018.stdout:(156/176): NetworkManager-tui-1.45.91-1.el9.x86 4.1 MB/s | 246 kB 00:00 2024-02-17T12:37:40.463 INFO:teuthology.orchestra.run.smithi018.stdout:(157/176): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 69 kB 00:00 2024-02-17T12:37:40.489 INFO:teuthology.orchestra.run.smithi018.stdout:(158/176): elfutils-debuginfod-client-0.190-2.e 603 kB/s | 35 kB 00:00 2024-02-17T12:37:40.514 INFO:teuthology.orchestra.run.smithi018.stdout:(159/176): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-02-17T12:37:40.548 INFO:teuthology.orchestra.run.smithi018.stdout:(160/176): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 254 kB 00:00 2024-02-17T12:37:40.598 INFO:teuthology.orchestra.run.smithi018.stdout:(161/176): iproute-6.7.0-2.el9.x86_64.rpm 9.4 MB/s | 802 kB 00:00 2024-02-17T12:37:40.623 INFO:teuthology.orchestra.run.smithi018.stdout:(162/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 5.7 MB/s | 433 kB 00:00 2024-02-17T12:37:40.657 INFO:teuthology.orchestra.run.smithi018.stdout:(163/176): libacl-2.3.1-4.el9.x86_64.rpm 379 kB/s | 22 kB 00:00 2024-02-17T12:37:40.682 INFO:teuthology.orchestra.run.smithi119.stdout:(103/176): lua-devel-5.4.4-4.el9.x86_64.rpm 69 kB/s | 22 kB 00:00 2024-02-17T12:37:40.710 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T12:37:40.740 INFO:teuthology.orchestra.run.smithi119.stdout:(104/176): python3-devel-3.9.17-2.el9.x86_64.rp 589 kB/s | 245 kB 00:00 2024-02-17T12:37:40.748 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T12:37:40.807 INFO:teuthology.orchestra.run.smithi018.stdout:(164/176): libblkid-2.37.4-18.el9.x86_64.rpm 585 kB/s | 107 kB 00:00 2024-02-17T12:37:40.832 INFO:teuthology.orchestra.run.smithi018.stdout:(165/176): libfdisk-2.37.4-18.el9.x86_64.rpm 878 kB/s | 154 kB 00:00 2024-02-17T12:37:40.874 INFO:teuthology.orchestra.run.smithi018.stdout:(166/176): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 135 kB 00:00 2024-02-17T12:37:40.891 INFO:teuthology.orchestra.run.smithi018.stdout:(167/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.0 MB/s | 63 kB 00:00 2024-02-17T12:37:40.916 INFO:teuthology.orchestra.run.smithi018.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 661 kB/s | 28 kB 00:00 2024-02-17T12:37:40.992 INFO:teuthology.orchestra.run.smithi018.stdout:(169/176): openssl-3.0.7-25.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-02-17T12:37:41.092 INFO:teuthology.orchestra.run.smithi018.stdout:(170/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 12 MB/s | 2.1 MB 00:00 2024-02-17T12:37:41.134 INFO:teuthology.orchestra.run.smithi018.stdout:(171/176): thrift-0.15.0-2.el9.x86_64.rpm 828 kB/s | 1.6 MB 00:01 2024-02-17T12:37:41.176 INFO:teuthology.orchestra.run.smithi018.stdout:(172/176): util-linux-2.37.4-18.el9.x86_64.rpm 12 MB/s | 2.2 MB 00:00 2024-02-17T12:37:41.201 INFO:teuthology.orchestra.run.smithi018.stdout:(173/176): util-linux-core-2.37.4-18.el9.x86_64 4.0 MB/s | 438 kB 00:00 2024-02-17T12:37:41.226 INFO:teuthology.orchestra.run.smithi018.stdout:(174/176): util-linux-user-2.37.4-18.el9.x86_64 334 kB/s | 30 kB 00:00 2024-02-17T12:37:41.232 INFO:teuthology.orchestra.run.smithi119.stdout:(105/176): protobuf-compiler-3.14.0-13.el9.x86_ 1.5 MB/s | 863 kB 00:00 2024-02-17T12:37:41.268 INFO:teuthology.orchestra.run.smithi018.stdout:(175/176): xfsdump-3.1.12-4.el9.x86_64.rpm 3.4 MB/s | 321 kB 00:00 2024-02-17T12:37:41.286 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T12:37:41.327 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 132/202 2024-02-17T12:37:41.327 INFO:teuthology.orchestra.run.smithi018.stdout:(176/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 11 MB/s | 1.1 MB 00:00 2024-02-17T12:37:41.334 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-02-17T12:37:41.335 INFO:teuthology.orchestra.run.smithi018.stdout:Total 6.0 MB/s | 223 MB 00:37 2024-02-17T12:37:41.383 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-02-17T12:37:41.390 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 132/202 2024-02-17T12:37:41.390 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-17T12:37:41.391 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-17T12:37:41.391 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:41.407 INFO:teuthology.orchestra.run.smithi119.stdout:(106/176): abseil-cpp-20211102.0-3.el9.x86_64.r 822 kB/s | 548 kB 00:00 2024-02-17T12:37:41.419 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 133/202 2024-02-17T12:37:41.541 INFO:teuthology.orchestra.run.smithi119.stdout:(107/176): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-02-17T12:37:41.558 INFO:teuthology.orchestra.run.smithi119.stdout:(108/176): grpc-data-1.46.7-10.el9.noarch.rpm 129 kB/s | 19 kB 00:00 2024-02-17T12:37:41.709 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-02-17T12:37:41.709 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-02-17T12:37:41.809 INFO:teuthology.orchestra.run.smithi119.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 104 kB/s | 26 kB 00:00 2024-02-17T12:37:42.034 INFO:teuthology.orchestra.run.smithi119.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 216 kB/s | 49 kB 00:00 2024-02-17T12:37:42.317 INFO:teuthology.orchestra.run.smithi119.stdout:(111/176): libunwind-1.6.2-1.el9.x86_64.rpm 238 kB/s | 67 kB 00:00 2024-02-17T12:37:42.684 INFO:teuthology.orchestra.run.smithi119.stdout:(112/176): luarocks-3.9.2-1.el9.noarch.rpm 412 kB/s | 151 kB 00:00 2024-02-17T12:37:42.868 INFO:teuthology.orchestra.run.smithi119.stdout:(113/176): libarrow-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-02-17T12:37:42.985 INFO:teuthology.orchestra.run.smithi119.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-02-17T12:37:43.127 INFO:teuthology.orchestra.run.smithi119.stdout:(115/176): python3-asyncssh-2.13.2-4.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-02-17T12:37:43.152 INFO:teuthology.orchestra.run.smithi119.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 261 kB/s | 43 kB 00:00 2024-02-17T12:37:43.244 INFO:teuthology.orchestra.run.smithi119.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-02-17T12:37:43.311 INFO:teuthology.orchestra.run.smithi119.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 94 kB/s | 15 kB 00:00 2024-02-17T12:37:43.361 INFO:teuthology.orchestra.run.smithi119.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-02-17T12:37:43.436 INFO:teuthology.orchestra.run.smithi119.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-02-17T12:37:43.503 INFO:teuthology.orchestra.run.smithi119.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-02-17T12:37:43.546 INFO:teuthology.orchestra.run.smithi119.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-02-17T12:37:43.612 INFO:teuthology.orchestra.run.smithi119.stdout:(123/176): python3-google-auth-2.27.0-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-02-17T12:37:43.716 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-02-17T12:37:43.716 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-02-17T12:37:43.863 INFO:teuthology.orchestra.run.smithi119.stdout:(124/176): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-02-17T12:37:43.888 INFO:teuthology.orchestra.run.smithi119.stdout:(125/176): python3-grpcio-tools-1.46.7-10.el9.x 525 kB/s | 144 kB 00:00 2024-02-17T12:37:43.955 INFO:teuthology.orchestra.run.smithi119.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-02-17T12:37:43.972 INFO:teuthology.orchestra.run.smithi119.stdout:(127/176): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-02-17T12:37:44.039 INFO:teuthology.orchestra.run.smithi119.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-02-17T12:37:44.056 INFO:teuthology.orchestra.run.smithi119.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-02-17T12:37:44.115 INFO:teuthology.orchestra.run.smithi119.stdout:(130/176): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-02-17T12:37:44.223 INFO:teuthology.orchestra.run.smithi119.stdout:(131/176): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-02-17T12:37:44.240 INFO:teuthology.orchestra.run.smithi119.stdout:(132/176): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-02-17T12:37:44.332 INFO:teuthology.orchestra.run.smithi119.stdout:(133/176): python3-more-itertools-8.12.0-2.el9. 727 kB/s | 79 kB 00:00 2024-02-17T12:37:44.353 INFO:teuthology.orchestra.run.smithi119.stdout:(134/176): python3-natsort-7.1.1-5.el9.noarch.r 513 kB/s | 58 kB 00:00 2024-02-17T12:37:44.436 INFO:teuthology.orchestra.run.smithi119.stdout:(135/176): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-02-17T12:37:44.461 INFO:teuthology.orchestra.run.smithi119.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-02-17T12:37:44.545 INFO:teuthology.orchestra.run.smithi119.stdout:(137/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 829 kB/s | 90 kB 00:00 2024-02-17T12:37:44.570 INFO:teuthology.orchestra.run.smithi119.stdout:(138/176): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-02-17T12:37:44.646 INFO:teuthology.orchestra.run.smithi119.stdout:(139/176): python3-repoze-lru-0.7-16.el9.noarch 308 kB/s | 31 kB 00:00 2024-02-17T12:37:44.704 INFO:teuthology.orchestra.run.smithi119.stdout:(140/176): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-02-17T12:37:44.738 INFO:teuthology.orchestra.run.smithi119.stdout:(141/176): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-02-17T12:37:44.805 INFO:teuthology.orchestra.run.smithi119.stdout:(142/176): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-02-17T12:37:44.830 INFO:teuthology.orchestra.run.smithi119.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 556 kB/s | 51 kB 00:00 2024-02-17T12:37:44.906 INFO:teuthology.orchestra.run.smithi119.stdout:(144/176): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-02-17T12:37:44.931 INFO:teuthology.orchestra.run.smithi119.stdout:(145/176): python3-websocket-client-1.2.3-2.el9 895 kB/s | 90 kB 00:00 2024-02-17T12:37:45.014 INFO:teuthology.orchestra.run.smithi119.stdout:(146/176): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-02-17T12:37:45.040 INFO:teuthology.orchestra.run.smithi119.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-02-17T12:37:45.098 INFO:teuthology.orchestra.run.smithi119.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-02-17T12:37:45.257 INFO:teuthology.orchestra.run.smithi119.stdout:(149/176): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-02-17T12:37:45.424 INFO:teuthology.orchestra.run.smithi119.stdout:(150/176): librados2-19.0.0-1515.g6822c7b5.el9. 10 MB/s | 3.4 MB 00:00 2024-02-17T12:37:45.483 INFO:teuthology.orchestra.run.smithi119.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 68 kB 00:00 2024-02-17T12:37:45.883 INFO:teuthology.orchestra.run.smithi119.stdout:(152/176): NetworkManager-1.45.91-1.el9.x86_64. 5.6 MB/s | 2.3 MB 00:00 2024-02-17T12:37:46.283 INFO:teuthology.orchestra.run.smithi119.stdout:(153/176): NetworkManager-libnm-1.45.91-1.el9.x 4.5 MB/s | 1.8 MB 00:00 2024-02-17T12:37:46.317 INFO:teuthology.orchestra.run.smithi119.stdout:(154/176): NetworkManager-team-1.45.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-02-17T12:37:46.408 INFO:teuthology.orchestra.run.smithi119.stdout:(155/176): NetworkManager-tui-1.45.91-1.el9.x86 2.6 MB/s | 246 kB 00:00 2024-02-17T12:37:46.459 INFO:teuthology.orchestra.run.smithi119.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 69 kB 00:00 2024-02-17T12:37:46.501 INFO:teuthology.orchestra.run.smithi119.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 849 kB/s | 35 kB 00:00 2024-02-17T12:37:46.576 INFO:teuthology.orchestra.run.smithi119.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 191 kB 00:00 2024-02-17T12:37:46.660 INFO:teuthology.orchestra.run.smithi119.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 3.0 MB/s | 254 kB 00:00 2024-02-17T12:37:46.826 INFO:teuthology.orchestra.run.smithi119.stdout:(160/176): iproute-6.7.0-2.el9.x86_64.rpm 4.7 MB/s | 802 kB 00:00 2024-02-17T12:37:46.935 INFO:teuthology.orchestra.run.smithi119.stdout:(161/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 3.9 MB/s | 433 kB 00:00 2024-02-17T12:37:46.977 INFO:teuthology.orchestra.run.smithi119.stdout:(162/176): libacl-2.3.1-4.el9.x86_64.rpm 527 kB/s | 22 kB 00:00 2024-02-17T12:37:47.036 INFO:teuthology.orchestra.run.smithi119.stdout:(163/176): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 107 kB 00:00 2024-02-17T12:37:47.103 INFO:teuthology.orchestra.run.smithi119.stdout:(164/176): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 154 kB 00:00 2024-02-17T12:37:47.170 INFO:teuthology.orchestra.run.smithi119.stdout:(165/176): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 135 kB 00:00 2024-02-17T12:37:47.220 INFO:teuthology.orchestra.run.smithi119.stdout:(166/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 63 kB 00:00 2024-02-17T12:37:47.262 INFO:teuthology.orchestra.run.smithi119.stdout:(167/176): libuuid-2.37.4-18.el9.x86_64.rpm 661 kB/s | 28 kB 00:00 2024-02-17T12:37:47.587 INFO:teuthology.orchestra.run.smithi119.stdout:(168/176): openssl-3.0.7-25.el9.x86_64.rpm 3.6 MB/s | 1.2 MB 00:00 2024-02-17T12:37:47.838 INFO:teuthology.orchestra.run.smithi119.stdout:(169/176): ceph-common-19.0.0-1515.g6822c7b5.el 547 kB/s | 20 MB 00:37 2024-02-17T12:37:48.354 INFO:teuthology.orchestra.run.smithi119.stdout:(170/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 2.8 MB/s | 2.1 MB 00:00 2024-02-17T12:37:48.763 INFO:teuthology.orchestra.run.smithi119.stdout:(171/176): util-linux-core-2.37.4-18.el9.x86_64 1.0 MB/s | 438 kB 00:00 2024-02-17T12:37:48.805 INFO:teuthology.orchestra.run.smithi119.stdout:(172/176): util-linux-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 2.2 MB 00:00 2024-02-17T12:37:48.830 INFO:teuthology.orchestra.run.smithi119.stdout:(173/176): util-linux-user-2.37.4-18.el9.x86_64 450 kB/s | 30 kB 00:00 2024-02-17T12:37:48.906 INFO:teuthology.orchestra.run.smithi119.stdout:(174/176): xfsdump-3.1.12-4.el9.x86_64.rpm 3.1 MB/s | 321 kB 00:00 2024-02-17T12:37:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:(175/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.1 MB/s | 1.1 MB 00:00 2024-02-17T12:37:49.139 INFO:teuthology.orchestra.run.smithi119.stdout:(176/176): librbd1-19.0.0-1515.g6822c7b5.el9.x8 825 kB/s | 3.1 MB 00:03 2024-02-17T12:37:49.144 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-02-17T12:37:49.145 INFO:teuthology.orchestra.run.smithi119.stdout:Total 5.7 MB/s | 223 MB 00:39 2024-02-17T12:37:49.192 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-02-17T12:37:49.468 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-02-17T12:37:49.468 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-02-17T12:37:50.910 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-02-17T12:37:51.029 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-17T12:37:51.066 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-17T12:37:51.120 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-17T12:37:51.128 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-02-17T12:37:51.129 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-02-17T12:37:51.143 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-17T12:37:51.208 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-17T12:37:51.246 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-17T12:37:51.302 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-17T12:37:51.342 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-17T12:37:51.368 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-17T12:37:51.421 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-17T12:37:51.771 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-17T12:37:51.828 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T12:37:51.934 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T12:37:51.962 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T12:37:52.037 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T12:37:52.072 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 13/202 2024-02-17T12:37:52.247 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-17T12:37:52.430 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-17T12:37:52.457 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T12:37:52.562 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T12:37:52.591 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 17/202 2024-02-17T12:37:52.656 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 17/202 2024-02-17T12:37:52.687 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-17T12:37:52.709 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-17T12:37:52.724 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-17T12:37:52.918 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-17T12:37:52.949 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-17T12:37:52.989 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-17T12:37:53.131 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 133/202 2024-02-17T12:37:53.131 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /sys 2024-02-17T12:37:53.131 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /proc 2024-02-17T12:37:53.131 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /mnt 2024-02-17T12:37:53.131 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /var/tmp 2024-02-17T12:37:53.131 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /home 2024-02-17T12:37:53.131 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /root 2024-02-17T12:37:53.132 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /tmp 2024-02-17T12:37:53.132 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:53.174 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-17T12:37:53.296 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 134/202 2024-02-17T12:37:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-17T12:37:53.312 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 134/202 2024-02-17T12:37:53.312 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:37:53.312 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-17T12:37:53.312 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:53.359 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-17T12:37:53.442 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T12:37:53.485 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T12:37:53.675 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T12:37:53.785 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-17T12:37:53.882 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-17T12:37:53.916 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-17T12:37:53.987 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-17T12:37:54.047 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-17T12:37:54.077 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-17T12:37:54.112 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-17T12:37:54.153 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-17T12:37:54.219 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-17T12:37:54.252 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-17T12:37:54.297 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-17T12:37:54.330 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-17T12:37:54.366 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-17T12:37:54.438 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-17T12:37:54.479 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9. 39/202 2024-02-17T12:37:54.523 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el 40/202 2024-02-17T12:37:54.551 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_6 41/202 2024-02-17T12:37:54.623 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-17T12:37:54.791 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-17T12:37:54.851 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 135/202 2024-02-17T12:37:54.867 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-17T12:37:54.877 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 135/202 2024-02-17T12:37:54.877 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:37:54.877 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-17T12:37:54.877 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-17T12:37:54.877 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-17T12:37:54.878 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:55.003 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-17T12:37:55.060 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-17T12:37:55.076 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 136/202 2024-02-17T12:37:55.099 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 136/202 2024-02-17T12:37:55.099 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:37:55.100 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-17T12:37:55.100 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-17T12:37:55.100 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-17T12:37:55.100 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:55.535 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 137/202 2024-02-17T12:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 137/202 2024-02-17T12:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-17T12:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-17T12:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-17T12:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:55.626 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-17T12:37:55.654 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-17T12:37:55.707 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-17T12:37:55.790 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-17T12:37:55.841 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-17T12:37:55.849 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-17T12:37:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-17T12:37:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_6 52/202 2024-02-17T12:37:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-17T12:37:56.037 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 139/202 2024-02-17T12:37:56.064 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-17T12:37:56.068 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 139/202 2024-02-17T12:37:56.091 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.e 140/202 2024-02-17T12:37:56.144 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5. 141/202 2024-02-17T12:37:56.151 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-17T12:37:56.166 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 142/202 2024-02-17T12:37:56.395 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 142/202 2024-02-17T12:37:56.426 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 143/202 2024-02-17T12:37:56.444 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-17T12:37:56.615 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-17T12:37:56.678 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-17T12:37:56.731 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-17T12:37:56.878 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-17T12:37:56.933 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-17T12:37:56.990 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-17T12:37:57.064 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-17T12:37:57.100 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-17T12:37:57.138 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-17T12:37:57.185 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-17T12:37:57.220 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-17T12:37:57.264 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-17T12:37:57.307 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-17T12:37:57.384 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 143/202 2024-02-17T12:37:57.409 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 144/202 2024-02-17T12:37:57.455 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-17T12:37:57.510 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 144/202 2024-02-17T12:37:57.511 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-17T12:37:57.541 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-17T12:37:57.584 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-17T12:37:57.673 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el 145/202 2024-02-17T12:37:57.681 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-17T12:37:57.703 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 146/202 2024-02-17T12:37:57.716 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-17T12:37:57.726 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 146/202 2024-02-17T12:37:57.726 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:37:57.726 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-17T12:37:57.726 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-17T12:37:57.726 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-17T12:37:57.727 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:57.757 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-17T12:37:57.770 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 147/202 2024-02-17T12:37:57.781 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 147/202 2024-02-17T12:37:57.782 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-17T12:37:57.806 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-17T12:37:57.826 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 148/202 2024-02-17T12:37:57.895 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-17T12:37:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-02-17T12:37:58.036 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-17T12:37:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-17T12:37:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-17T12:37:58.153 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-17T12:37:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-17T12:37:58.206 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-17T12:37:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-17T12:37:58.265 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-17T12:37:58.269 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-17T12:37:58.316 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-17T12:37:58.330 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-17T12:37:58.370 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-17T12:37:58.403 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-17T12:37:58.429 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-17T12:37:58.471 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-17T12:37:58.535 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-17T12:37:58.578 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-17T12:37:58.679 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-17T12:37:58.714 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 150/202 2024-02-17T12:37:58.719 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-17T12:37:58.737 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 150/202 2024-02-17T12:37:58.737 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:37:58.737 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-17T12:37:58.737 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-17T12:37:58.737 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-17T12:37:58.737 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:58.759 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-17T12:37:58.781 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1515.g6822c 151/202 2024-02-17T12:37:58.798 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-17T12:37:58.799 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-17T12:37:58.803 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1515.g6822c 151/202 2024-02-17T12:37:58.803 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:37:58.803 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-17T12:37:58.803 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:58.831 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-17T12:37:58.848 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 152/202 2024-02-17T12:37:58.867 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-17T12:37:58.871 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 152/202 2024-02-17T12:37:58.871 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:37:58.871 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-17T12:37:58.871 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-17T12:37:58.871 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-17T12:37:58.871 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:58.872 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T12:37:58.897 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-17T12:37:58.926 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-17T12:37:58.960 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-17T12:37:58.993 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T12:37:59.000 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-17T12:37:59.023 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T12:37:59.031 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-17T12:37:59.061 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-17T12:37:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T12:37:59.140 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-17T12:37:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 13/202 2024-02-17T12:37:59.180 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-17T12:37:59.182 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T12:37:59.205 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T12:37:59.206 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:37:59.206 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-17T12:37:59.206 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-17T12:37:59.206 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-17T12:37:59.206 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:37:59.213 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-17T12:37:59.255 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-17T12:37:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-17T12:37:59.291 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-17T12:37:59.332 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-17T12:37:59.360 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T12:37:59.381 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-17T12:37:59.446 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T12:37:59.469 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 17/202 2024-02-17T12:37:59.471 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-17T12:37:59.520 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-17T12:37:59.533 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 17/202 2024-02-17T12:37:59.551 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-17T12:37:59.573 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-17T12:37:59.596 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-17T12:37:59.600 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-17T12:37:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-17T12:37:59.710 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-17T12:37:59.762 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-17T12:37:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-17T12:37:59.826 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-17T12:37:59.867 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-17T12:37:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-17T12:38:00.045 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-17T12:38:00.113 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-17T12:38:00.174 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T12:38:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T12:38:00.457 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T12:38:00.530 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-17T12:38:00.562 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-17T12:38:00.583 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-17T12:38:00.611 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-17T12:38:00.655 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-17T12:38:00.660 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-17T12:38:00.685 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-17T12:38:00.751 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-17T12:38:00.753 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-17T12:38:00.786 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-17T12:38:00.807 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-17T12:38:00.838 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-17T12:38:00.874 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-17T12:38:00.906 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-17T12:38:00.969 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-17T12:38:00.970 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-17T12:38:01.001 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-17T12:38:01.004 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-17T12:38:01.048 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-17T12:38:01.080 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-17T12:38:01.081 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-17T12:38:01.134 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-17T12:38:01.203 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-17T12:38:01.238 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9. 39/202 2024-02-17T12:38:01.282 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el 40/202 2024-02-17T12:38:01.311 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_6 41/202 2024-02-17T12:38:01.379 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-17T12:38:01.433 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-17T12:38:01.506 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-17T12:38:01.640 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-17T12:38:01.694 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-17T12:38:01.745 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-17T12:38:01.860 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-17T12:38:02.212 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-17T12:38:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-17T12:38:02.364 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-17T12:38:02.410 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-17T12:38:02.417 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-17T12:38:02.456 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-17T12:38:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_6 52/202 2024-02-17T12:38:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-17T12:38:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-17T12:38:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-17T12:38:02.919 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-17T12:38:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-17T12:38:03.128 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-17T12:38:03.138 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-17T12:38:03.173 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-17T12:38:03.222 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-17T12:38:03.248 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-17T12:38:03.296 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-17T12:38:03.342 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-17T12:38:03.397 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-17T12:38:03.469 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-17T12:38:03.501 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-17T12:38:03.537 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-17T12:38:03.572 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-17T12:38:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-17T12:38:03.595 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 125/202 2024-02-17T12:38:03.603 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 154/202 2024-02-17T12:38:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-17T12:38:03.644 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 155/202 2024-02-17T12:38:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-17T12:38:03.669 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 125/202 2024-02-17T12:38:03.674 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 156/202 2024-02-17T12:38:03.697 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-17T12:38:03.699 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 126/202 2024-02-17T12:38:03.701 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-17T12:38:03.736 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-17T12:38:03.830 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-17T12:38:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-17T12:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 158/202 2024-02-17T12:38:03.907 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-17T12:38:03.949 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-17T12:38:04.009 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-17T12:38:04.041 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-17T12:38:04.050 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 160/202 2024-02-17T12:38:04.082 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-17T12:38:04.088 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-17T12:38:04.122 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-17T12:38:04.140 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-17T12:38:04.148 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-17T12:38:04.173 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-17T12:38:04.213 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-17T12:38:04.216 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-17T12:38:04.243 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-17T12:38:04.248 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 129/202 2024-02-17T12:38:04.259 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-17T12:38:04.294 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-17T12:38:04.302 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 129/202 2024-02-17T12:38:04.325 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 130/202 2024-02-17T12:38:04.333 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_ 166/202 2024-02-17T12:38:04.363 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-17T12:38:04.365 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-17T12:38:04.418 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-17T12:38:04.462 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-17T12:38:04.469 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-17T12:38:04.505 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-17T12:38:04.512 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-17T12:38:04.553 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-17T12:38:04.608 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-17T12:38:04.790 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-17T12:38:04.843 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-17T12:38:04.969 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-17T12:38:05.007 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-17T12:38:05.049 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-17T12:38:05.093 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-17T12:38:05.124 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-17T12:38:05.148 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T12:38:05.155 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-17T12:38:05.183 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-17T12:38:05.193 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T12:38:05.212 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T12:38:05.217 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-17T12:38:05.256 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-17T12:38:05.288 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-17T12:38:05.318 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-17T12:38:05.394 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-17T12:38:05.428 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-17T12:38:05.462 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-17T12:38:05.503 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-17T12:38:05.537 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-17T12:38:05.625 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-17T12:38:05.707 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-17T12:38:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-17T12:38:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-17T12:38:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-17T12:38:05.926 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-17T12:38:05.976 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-17T12:38:06.034 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 172/202 2024-02-17T12:38:06.104 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-17T12:38:06.144 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-17T12:38:06.224 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-17T12:38:06.266 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-17T12:38:06.290 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-17T12:38:06.316 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-17T12:38:06.343 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-17T12:38:06.343 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-17T12:38:06.427 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-17T12:38:06.455 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-17T12:38:06.485 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-17T12:38:06.485 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-17T12:38:06.525 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T12:38:06.564 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T12:38:06.570 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-17T12:38:06.596 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-17T12:38:06.630 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-17T12:38:06.655 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-17T12:38:06.682 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-17T12:38:06.699 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-17T12:38:06.722 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-17T12:38:06.748 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-17T12:38:06.755 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-17T12:38:06.783 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-17T12:38:06.797 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-17T12:38:06.829 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-17T12:38:06.829 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-17T12:38:06.854 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-17T12:38:06.879 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-17T12:38:06.905 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-17T12:38:06.908 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-17T12:38:06.930 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-17T12:38:06.936 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T12:38:06.936 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T12:38:06.950 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-17T12:38:07.117 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T12:38:07.117 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-17T12:38:07.127 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-17T12:38:07.156 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-17T12:38:07.157 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T12:38:07.193 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 132/202 2024-02-17T12:38:07.207 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-17T12:38:07.224 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-17T12:38:07.232 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-17T12:38:07.257 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-17T12:38:07.274 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 132/202 2024-02-17T12:38:07.274 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-17T12:38:07.274 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-17T12:38:07.274 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:07.282 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-17T12:38:07.310 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 133/202 2024-02-17T12:38:07.315 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-17T12:38:07.380 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-17T12:38:07.854 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-17T12:38:07.969 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-17T12:38:09.159 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-17T12:38:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-17T12:38:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-17T12:38:09.551 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-17T12:38:09.583 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 125/202 2024-02-17T12:38:09.664 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 125/202 2024-02-17T12:38:09.696 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 126/202 2024-02-17T12:38:09.732 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-17T12:38:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-17T12:38:10.219 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 129/202 2024-02-17T12:38:10.290 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 129/202 2024-02-17T12:38:10.320 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 130/202 2024-02-17T12:38:11.997 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-17T12:38:11.997 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 1/202 2024-02-17T12:38:11.997 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 2/202 2024-02-17T12:38:11.997 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 3/202 2024-02-17T12:38:11.997 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 4/202 2024-02-17T12:38:11.997 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1515.g6822c 5/202 2024-02-17T12:38:11.997 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 6/202 2024-02-17T12:38:11.997 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 7/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 8/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 9/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 10/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 13/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 14/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_ 15/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 17/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_6 18/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 19/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 20/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el 21/202 2024-02-17T12:38:11.998 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9. 22/202 2024-02-17T12:38:11.999 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_6 23/202 2024-02-17T12:38:11.999 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 24/202 2024-02-17T12:38:11.999 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 25/202 2024-02-17T12:38:12.016 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 26/202 2024-02-17T12:38:12.016 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 27/202 2024-02-17T12:38:12.016 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 28/202 2024-02-17T12:38:12.016 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 29/202 2024-02-17T12:38:12.016 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5. 30/202 2024-02-17T12:38:12.016 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 31/202 2024-02-17T12:38:12.016 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 32/202 2024-02-17T12:38:12.016 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 33/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el 34/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 35/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.e 36/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 37/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 38/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 39/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-17T12:38:12.017 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-17T12:38:12.018 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-17T12:38:12.018 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-17T12:38:12.018 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-17T12:38:12.018 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-17T12:38:12.018 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-17T12:38:12.018 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-17T12:38:12.018 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-17T12:38:12.018 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-17T12:38:12.018 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-17T12:38:12.018 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-17T12:38:12.019 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-17T12:38:12.020 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-17T12:38:12.020 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-17T12:38:12.020 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-17T12:38:12.020 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-17T12:38:12.020 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-17T12:38:12.020 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-17T12:38:12.020 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-17T12:38:12.020 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-17T12:38:12.021 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-17T12:38:12.022 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-17T12:38:12.023 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-17T12:38:12.024 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-17T12:38:12.025 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 151/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-17T12:38:12.026 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-17T12:38:12.027 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-17T12:38:12.028 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-17T12:38:12.029 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-17T12:38:12.463 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T12:38:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T12:38:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T12:38:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 132/202 2024-02-17T12:38:13.151 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 132/202 2024-02-17T12:38:13.151 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-17T12:38:13.152 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-17T12:38:13.152 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 133/202 2024-02-17T12:38:15.046 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-17T12:38:15.046 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:38:15.046 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2024-02-17T12:38:15.046 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-17T12:38:15.046 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-17T12:38:15.046 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-17T12:38:15.046 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-17T12:38:15.046 INFO:teuthology.orchestra.run.smithi146.stdout: acl-2.3.1-4.el9.x86_64 2024-02-17T12:38:15.046 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-17T12:38:15.047 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-17T12:38:15.048 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-17T12:38:15.048 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-17T12:38:15.048 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-17T12:38:15.048 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-17T12:38:15.048 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-17T12:38:15.048 INFO:teuthology.orchestra.run.smithi146.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-17T12:38:15.048 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-17T12:38:15.048 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-02-17T12:38:15.048 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-17T12:38:15.048 INFO:teuthology.orchestra.run.smithi146.stdout: automake-1.16.2-8.el9.noarch 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: bison-3.7.4-5.el9.x86_64 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:15.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:15.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:15.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:15.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:15.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.051 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.051 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.051 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:15.051 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:15.051 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.051 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.051 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:15.051 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-17T12:38:15.051 INFO:teuthology.orchestra.run.smithi146.stdout: flex-2.6.4-9.el9.x86_64 2024-02-17T12:38:15.051 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-17T12:38:15.051 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-17T12:38:15.051 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: jq-1.6-16.el9.x86_64 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-17T12:38:15.052 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: lua-5.4.4-4.el9.x86_64 2024-02-17T12:38:15.053 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-13.11-1.el9.x86_64 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-17T12:38:15.054 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-17T12:38:15.055 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-17T12:38:15.056 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-17T12:38:15.056 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-17T12:38:15.056 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-17T12:38:15.056 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-17T12:38:15.056 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-17T12:38:15.056 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-17T12:38:15.056 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-17T12:38:15.056 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-17T12:38:15.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-17T12:38:15.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-17T12:38:15.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-17T12:38:15.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-17T12:38:15.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-17T12:38:15.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-17T12:38:15.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-17T12:38:15.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-17T12:38:15.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-17T12:38:15.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply-3.11-14.el9.noarch 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-17T12:38:15.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-17T12:38:15.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-17T12:38:15.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-17T12:38:15.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-17T12:38:15.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-17T12:38:15.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-17T12:38:15.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-17T12:38:15.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-17T12:38:15.060 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-17T12:38:15.060 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-17T12:38:15.060 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-17T12:38:15.060 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.061 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.061 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:15.061 INFO:teuthology.orchestra.run.smithi146.stdout: re2-1:20211101-3.el9.x86_64 2024-02-17T12:38:15.061 INFO:teuthology.orchestra.run.smithi146.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-17T12:38:15.061 INFO:teuthology.orchestra.run.smithi146.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-17T12:38:15.061 INFO:teuthology.orchestra.run.smithi146.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-17T12:38:15.061 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-17T12:38:15.061 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-17T12:38:15.061 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-17T12:38:15.062 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:38:15.062 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-02-17T12:38:15.632 DEBUG:teuthology.parallel:result is None 2024-02-17T12:38:19.381 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 133/202 2024-02-17T12:38:19.381 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /sys 2024-02-17T12:38:19.382 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /proc 2024-02-17T12:38:19.382 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /mnt 2024-02-17T12:38:19.382 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /var/tmp 2024-02-17T12:38:19.382 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /home 2024-02-17T12:38:19.382 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /root 2024-02-17T12:38:19.382 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /tmp 2024-02-17T12:38:19.382 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:19.548 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 134/202 2024-02-17T12:38:19.563 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 134/202 2024-02-17T12:38:19.563 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:19.563 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-17T12:38:19.563 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:21.074 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 135/202 2024-02-17T12:38:21.099 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 135/202 2024-02-17T12:38:21.099 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:21.099 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-17T12:38:21.099 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-17T12:38:21.099 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-17T12:38:21.099 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:21.316 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 136/202 2024-02-17T12:38:21.338 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 136/202 2024-02-17T12:38:21.338 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:21.338 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-17T12:38:21.339 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-17T12:38:21.339 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-17T12:38:21.339 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:21.808 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 137/202 2024-02-17T12:38:21.830 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 137/202 2024-02-17T12:38:21.830 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:21.830 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-17T12:38:21.831 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-17T12:38:21.831 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-17T12:38:21.831 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:21.927 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-17T12:38:22.363 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 139/202 2024-02-17T12:38:22.392 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 139/202 2024-02-17T12:38:22.424 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.e 140/202 2024-02-17T12:38:22.480 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5. 141/202 2024-02-17T12:38:22.507 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 142/202 2024-02-17T12:38:22.744 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 142/202 2024-02-17T12:38:22.774 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 143/202 2024-02-17T12:38:23.786 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 143/202 2024-02-17T12:38:23.816 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 144/202 2024-02-17T12:38:23.919 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 144/202 2024-02-17T12:38:24.076 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el 145/202 2024-02-17T12:38:24.101 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 146/202 2024-02-17T12:38:24.124 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 146/202 2024-02-17T12:38:24.124 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:24.124 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-17T12:38:24.124 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-17T12:38:24.124 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-17T12:38:24.124 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:24.169 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 147/202 2024-02-17T12:38:24.179 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 147/202 2024-02-17T12:38:24.223 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 148/202 2024-02-17T12:38:24.824 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 133/202 2024-02-17T12:38:24.825 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2024-02-17T12:38:24.825 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2024-02-17T12:38:24.825 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2024-02-17T12:38:24.825 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2024-02-17T12:38:24.825 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2024-02-17T12:38:24.825 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2024-02-17T12:38:24.825 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2024-02-17T12:38:24.825 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:24.994 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 134/202 2024-02-17T12:38:25.008 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 134/202 2024-02-17T12:38:25.009 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:25.009 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-17T12:38:25.009 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:25.116 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-17T12:38:25.154 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 150/202 2024-02-17T12:38:25.176 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 150/202 2024-02-17T12:38:25.177 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:25.177 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-17T12:38:25.177 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-17T12:38:25.177 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-17T12:38:25.177 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:25.222 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1515.g6822c 151/202 2024-02-17T12:38:25.243 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1515.g6822c 151/202 2024-02-17T12:38:25.243 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:25.243 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-17T12:38:25.243 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:25.289 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 152/202 2024-02-17T12:38:25.310 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 152/202 2024-02-17T12:38:25.311 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:25.311 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-17T12:38:25.312 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-17T12:38:25.312 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-17T12:38:25.312 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:25.631 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T12:38:25.653 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T12:38:25.653 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:25.653 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-17T12:38:25.653 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-17T12:38:25.653 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-17T12:38:25.654 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:26.455 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 135/202 2024-02-17T12:38:26.480 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 135/202 2024-02-17T12:38:26.480 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:26.480 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-17T12:38:26.481 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-17T12:38:26.481 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-17T12:38:26.481 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:26.681 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 136/202 2024-02-17T12:38:26.702 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 136/202 2024-02-17T12:38:26.702 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:26.702 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-17T12:38:26.703 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-17T12:38:26.703 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-17T12:38:26.703 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 137/202 2024-02-17T12:38:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 137/202 2024-02-17T12:38:27.162 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:27.162 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-17T12:38:27.162 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-17T12:38:27.162 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-17T12:38:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:27.250 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-17T12:38:27.734 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 139/202 2024-02-17T12:38:27.757 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 139/202 2024-02-17T12:38:27.788 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.e 140/202 2024-02-17T12:38:27.847 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5. 141/202 2024-02-17T12:38:27.872 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 142/202 2024-02-17T12:38:28.098 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 142/202 2024-02-17T12:38:28.130 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 143/202 2024-02-17T12:38:29.085 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 143/202 2024-02-17T12:38:29.116 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 144/202 2024-02-17T12:38:29.213 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 144/202 2024-02-17T12:38:29.376 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el 145/202 2024-02-17T12:38:29.407 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 146/202 2024-02-17T12:38:29.430 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 146/202 2024-02-17T12:38:29.430 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:29.430 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-17T12:38:29.430 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-17T12:38:29.430 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-17T12:38:29.430 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:29.475 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 147/202 2024-02-17T12:38:29.485 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 147/202 2024-02-17T12:38:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 148/202 2024-02-17T12:38:30.311 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 154/202 2024-02-17T12:38:30.352 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 155/202 2024-02-17T12:38:30.382 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 156/202 2024-02-17T12:38:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-17T12:38:30.409 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-17T12:38:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 150/202 2024-02-17T12:38:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 150/202 2024-02-17T12:38:30.441 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:30.441 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-17T12:38:30.441 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-17T12:38:30.442 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-17T12:38:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:30.514 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1515.g6822c 151/202 2024-02-17T12:38:30.535 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1515.g6822c 151/202 2024-02-17T12:38:30.535 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:30.535 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-17T12:38:30.535 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:30.586 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 152/202 2024-02-17T12:38:30.608 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 158/202 2024-02-17T12:38:30.609 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 152/202 2024-02-17T12:38:30.609 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:30.609 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-17T12:38:30.609 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-17T12:38:30.609 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-17T12:38:30.609 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:30.729 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-17T12:38:30.774 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 160/202 2024-02-17T12:38:30.844 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-17T12:38:30.899 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-17T12:38:30.921 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T12:38:30.942 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T12:38:30.942 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T12:38:30.943 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-17T12:38:30.943 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-17T12:38:30.943 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-17T12:38:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:30.986 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-17T12:38:31.026 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-17T12:38:31.061 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-17T12:38:31.091 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_ 166/202 2024-02-17T12:38:31.121 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-17T12:38:31.208 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-17T12:38:31.261 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-17T12:38:31.303 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-17T12:38:31.902 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T12:38:31.934 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T12:38:31.954 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T12:38:32.828 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 172/202 2024-02-17T12:38:32.905 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-17T12:38:32.949 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-17T12:38:33.032 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-17T12:38:33.060 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-17T12:38:33.085 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-17T12:38:33.111 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-17T12:38:33.137 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-17T12:38:33.137 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-17T12:38:33.239 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-17T12:38:33.267 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-17T12:38:33.305 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-17T12:38:33.305 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-17T12:38:33.381 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-17T12:38:33.415 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-17T12:38:33.450 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-17T12:38:33.475 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-17T12:38:33.502 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-17T12:38:33.534 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-17T12:38:33.571 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-17T12:38:33.616 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-17T12:38:33.648 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-17T12:38:33.672 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-17T12:38:33.698 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-17T12:38:33.715 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-17T12:38:33.741 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-17T12:38:33.747 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T12:38:33.747 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T12:38:33.935 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T12:38:33.936 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-17T12:38:34.010 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-17T12:38:34.034 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-17T12:38:34.060 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-17T12:38:34.085 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-17T12:38:34.118 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-17T12:38:34.197 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-17T12:38:35.367 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 154/202 2024-02-17T12:38:35.407 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 155/202 2024-02-17T12:38:35.438 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 156/202 2024-02-17T12:38:35.465 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-17T12:38:35.644 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 158/202 2024-02-17T12:38:35.755 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-17T12:38:35.796 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 160/202 2024-02-17T12:38:35.835 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-17T12:38:35.896 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-17T12:38:35.970 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-17T12:38:36.006 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-17T12:38:36.050 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-17T12:38:36.063 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_ 166/202 2024-02-17T12:38:36.093 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-17T12:38:36.157 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-17T12:38:36.207 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-17T12:38:36.250 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-17T12:38:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T12:38:36.902 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T12:38:36.921 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T12:38:37.732 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 172/202 2024-02-17T12:38:37.801 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-17T12:38:37.841 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-17T12:38:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-17T12:38:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-17T12:38:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-17T12:38:38.013 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-17T12:38:38.040 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-17T12:38:38.040 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-17T12:38:38.124 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-17T12:38:38.152 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-17T12:38:38.191 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-17T12:38:38.191 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-17T12:38:38.276 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-17T12:38:38.302 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-17T12:38:38.336 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-17T12:38:38.360 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-17T12:38:38.388 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-17T12:38:38.428 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-17T12:38:38.461 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-17T12:38:38.500 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-17T12:38:38.533 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-17T12:38:38.558 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-17T12:38:38.583 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-17T12:38:38.609 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-17T12:38:38.634 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-17T12:38:38.640 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T12:38:38.641 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T12:38:38.820 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T12:38:38.821 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-17T12:38:38.895 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-17T12:38:38.920 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-17T12:38:38.945 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-17T12:38:38.984 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-17T12:38:39.011 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-17T12:38:39.011 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 1/202 2024-02-17T12:38:39.011 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 2/202 2024-02-17T12:38:39.011 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 3/202 2024-02-17T12:38:39.011 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 4/202 2024-02-17T12:38:39.011 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1515.g6822c 5/202 2024-02-17T12:38:39.012 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 6/202 2024-02-17T12:38:39.012 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 7/202 2024-02-17T12:38:39.012 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 8/202 2024-02-17T12:38:39.012 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 9/202 2024-02-17T12:38:39.012 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 10/202 2024-02-17T12:38:39.012 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T12:38:39.013 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T12:38:39.013 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 13/202 2024-02-17T12:38:39.013 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 14/202 2024-02-17T12:38:39.013 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_ 15/202 2024-02-17T12:38:39.013 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T12:38:39.013 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 17/202 2024-02-17T12:38:39.014 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_6 18/202 2024-02-17T12:38:39.014 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 19/202 2024-02-17T12:38:39.014 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 20/202 2024-02-17T12:38:39.014 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el 21/202 2024-02-17T12:38:39.014 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9. 22/202 2024-02-17T12:38:39.014 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_6 23/202 2024-02-17T12:38:39.015 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 24/202 2024-02-17T12:38:39.015 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 25/202 2024-02-17T12:38:39.015 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 26/202 2024-02-17T12:38:39.015 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 27/202 2024-02-17T12:38:39.015 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 28/202 2024-02-17T12:38:39.015 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 29/202 2024-02-17T12:38:39.015 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5. 30/202 2024-02-17T12:38:39.016 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 31/202 2024-02-17T12:38:39.016 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 32/202 2024-02-17T12:38:39.016 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 33/202 2024-02-17T12:38:39.016 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el 34/202 2024-02-17T12:38:39.016 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 35/202 2024-02-17T12:38:39.016 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.e 36/202 2024-02-17T12:38:39.017 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 37/202 2024-02-17T12:38:39.017 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 38/202 2024-02-17T12:38:39.017 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 39/202 2024-02-17T12:38:39.017 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-17T12:38:39.017 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-17T12:38:39.017 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-17T12:38:39.017 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-17T12:38:39.018 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-17T12:38:39.018 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-17T12:38:39.018 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-17T12:38:39.018 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-17T12:38:39.018 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-17T12:38:39.019 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-17T12:38:39.019 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-17T12:38:39.019 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-17T12:38:39.019 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-17T12:38:39.019 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-17T12:38:39.019 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-17T12:38:39.020 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-17T12:38:39.020 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-17T12:38:39.020 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-17T12:38:39.020 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-17T12:38:39.020 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-17T12:38:39.020 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-17T12:38:39.021 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-17T12:38:39.021 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-17T12:38:39.021 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-17T12:38:39.021 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-17T12:38:39.021 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-17T12:38:39.021 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-17T12:38:39.022 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-17T12:38:39.022 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-17T12:38:39.022 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-17T12:38:39.022 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-17T12:38:39.022 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-17T12:38:39.022 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-17T12:38:39.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-17T12:38:39.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-17T12:38:39.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-17T12:38:39.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-17T12:38:39.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-17T12:38:39.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-17T12:38:39.024 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-17T12:38:39.024 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-17T12:38:39.024 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-17T12:38:39.024 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-17T12:38:39.024 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-17T12:38:39.024 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-17T12:38:39.025 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-17T12:38:39.025 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-17T12:38:39.025 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-17T12:38:39.025 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-17T12:38:39.025 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-17T12:38:39.025 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-17T12:38:39.026 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-17T12:38:39.026 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-17T12:38:39.026 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-17T12:38:39.026 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-17T12:38:39.026 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-17T12:38:39.026 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-17T12:38:39.027 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-17T12:38:39.027 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-17T12:38:39.027 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-17T12:38:39.027 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-17T12:38:39.027 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-17T12:38:39.027 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-17T12:38:39.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-17T12:38:39.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-17T12:38:39.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-17T12:38:39.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-17T12:38:39.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-17T12:38:39.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-17T12:38:39.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-17T12:38:39.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-17T12:38:39.028 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-17T12:38:39.029 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-17T12:38:39.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-17T12:38:39.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-17T12:38:39.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-17T12:38:39.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-17T12:38:39.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-17T12:38:39.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-17T12:38:39.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-17T12:38:39.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-17T12:38:39.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-17T12:38:39.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-17T12:38:39.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-17T12:38:39.030 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-17T12:38:39.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-17T12:38:39.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-17T12:38:39.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-17T12:38:39.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-17T12:38:39.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-17T12:38:39.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-17T12:38:39.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-17T12:38:39.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-17T12:38:39.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-17T12:38:39.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-17T12:38:39.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-17T12:38:39.031 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 151/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-17T12:38:39.032 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-17T12:38:39.033 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-17T12:38:39.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-17T12:38:39.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-17T12:38:39.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-17T12:38:39.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-17T12:38:39.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-17T12:38:39.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-17T12:38:39.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-17T12:38:39.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-17T12:38:39.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-17T12:38:39.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-17T12:38:39.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-17T12:38:39.034 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-17T12:38:39.035 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-17T12:38:39.036 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-17T12:38:39.076 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-17T12:38:41.892 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-17T12:38:41.892 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:41.892 INFO:teuthology.orchestra.run.smithi018.stdout:Upgraded: 2024-02-17T12:38:41.892 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-17T12:38:41.892 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: acl-2.3.1-4.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-17T12:38:41.893 INFO:teuthology.orchestra.run.smithi018.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-17T12:38:41.894 INFO:teuthology.orchestra.run.smithi018.stdout: automake-1.16.2-8.el9.noarch 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: bison-3.7.4-5.el9.x86_64 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:41.895 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:41.896 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: flex-2.6.4-9.el9.x86_64 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: jq-1.6-16.el9.x86_64 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.897 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-17T12:38:41.898 INFO:teuthology.orchestra.run.smithi018.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: lua-5.4.4-4.el9.x86_64 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-13.11-1.el9.x86_64 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-17T12:38:41.899 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-17T12:38:41.900 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-17T12:38:41.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-17T12:38:41.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ply-3.11-14.el9.noarch 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-17T12:38:41.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-17T12:38:41.904 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-17T12:38:41.905 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-17T12:38:41.905 INFO:teuthology.orchestra.run.smithi018.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-17T12:38:41.905 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.905 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.905 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:41.905 INFO:teuthology.orchestra.run.smithi018.stdout: re2-1:20211101-3.el9.x86_64 2024-02-17T12:38:41.905 INFO:teuthology.orchestra.run.smithi018.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-17T12:38:41.905 INFO:teuthology.orchestra.run.smithi018.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-17T12:38:41.905 INFO:teuthology.orchestra.run.smithi018.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-17T12:38:41.905 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-17T12:38:41.905 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-17T12:38:41.906 INFO:teuthology.orchestra.run.smithi018.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-17T12:38:41.906 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:38:41.906 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-02-17T12:38:42.524 DEBUG:teuthology.parallel:result is None 2024-02-17T12:38:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-17T12:38:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 1/202 2024-02-17T12:38:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 2/202 2024-02-17T12:38:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 3/202 2024-02-17T12:38:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 4/202 2024-02-17T12:38:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1515.g6822c 5/202 2024-02-17T12:38:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 6/202 2024-02-17T12:38:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 7/202 2024-02-17T12:38:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 8/202 2024-02-17T12:38:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 9/202 2024-02-17T12:38:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 10/202 2024-02-17T12:38:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T12:38:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 13/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 14/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_ 15/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 17/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_6 18/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 19/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 20/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el 21/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9. 22/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_6 23/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 24/202 2024-02-17T12:38:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 25/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 26/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 27/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 28/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 29/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5. 30/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 31/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 32/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 33/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el 34/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 35/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.e 36/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 37/202 2024-02-17T12:38:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 38/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 39/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-17T12:38:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-17T12:38:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-17T12:38:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-17T12:38:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-17T12:38:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-17T12:38:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-17T12:38:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-17T12:38:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 151/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-17T12:38:43.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-17T12:38:43.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-17T12:38:43.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-17T12:38:43.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-17T12:38:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-17T12:38:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:46.921 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-02-17T12:38:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-17T12:38:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-17T12:38:46.921 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: acl-2.3.1-4.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-17T12:38:46.922 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-17T12:38:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-17T12:38:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-17T12:38:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-17T12:38:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-17T12:38:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-17T12:38:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-17T12:38:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-17T12:38:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-17T12:38:46.923 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-02-17T12:38:46.923 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-17T12:38:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.2-8.el9.noarch 2024-02-17T12:38:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.7.4-5.el9.x86_64 2024-02-17T12:38:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-17T12:38:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.924 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:46.925 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.4-9.el9.x86_64 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: jq-1.6-16.el9.x86_64 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-17T12:38:46.926 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-17T12:38:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-17T12:38:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-17T12:38:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-17T12:38:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-17T12:38:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-17T12:38:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-17T12:38:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.927 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: lua-5.4.4-4.el9.x86_64 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-17T12:38:46.928 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-17T12:38:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-17T12:38:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-13.11-1.el9.x86_64 2024-02-17T12:38:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-17T12:38:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-17T12:38:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-17T12:38:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-17T12:38:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-17T12:38:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-17T12:38:46.929 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-17T12:38:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-17T12:38:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-17T12:38:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-17T12:38:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-17T12:38:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-17T12:38:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-17T12:38:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-17T12:38:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-17T12:38:46.931 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-17T12:38:46.932 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-17T12:38:46.933 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.934 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-17T12:38:46.935 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-17T12:38:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-17T12:38:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T12:38:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-3.el9.x86_64 2024-02-17T12:38:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-17T12:38:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-17T12:38:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-17T12:38:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-17T12:38:46.936 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-17T12:38:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-17T12:38:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-02-17T12:38:46.937 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-02-17T12:38:47.543 DEBUG:teuthology.parallel:result is None 2024-02-17T12:38:47.543 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:38:47.689 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-17T12:38:47.715 INFO:teuthology.orchestra.run.smithi018.stdout:19.0.0-1515.g6822c7b5.el9 2024-02-17T12:38:47.716 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1515.g6822c7b5.el9 2024-02-17T12:38:47.716 INFO:teuthology.task.install:The correct ceph version 19.0.0-1515.g6822c7b5 is installed. 2024-02-17T12:38:47.718 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:38:47.935 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-17T12:38:47.957 INFO:teuthology.orchestra.run.smithi119.stdout:19.0.0-1515.g6822c7b5.el9 2024-02-17T12:38:47.957 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1515.g6822c7b5.el9 2024-02-17T12:38:47.958 INFO:teuthology.task.install:The correct ceph version 19.0.0-1515.g6822c7b5 is installed. 2024-02-17T12:38:47.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:38:48.185 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-17T12:38:48.211 INFO:teuthology.orchestra.run.smithi146.stdout:19.0.0-1515.g6822c7b5.el9 2024-02-17T12:38:48.211 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1515.g6822c7b5.el9 2024-02-17T12:38:48.211 INFO:teuthology.task.install:The correct ceph version 19.0.0-1515.g6822c7b5 is installed. 2024-02-17T12:38:48.213 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-17T12:38:48.213 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:38:48.213 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-17T12:38:48.245 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-02-17T12:38:48.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-17T12:38:48.277 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-02-17T12:38:48.278 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-17T12:38:48.311 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-17T12:38:48.312 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:38:48.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/daemon-helper 2024-02-17T12:38:48.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-17T12:38:48.409 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-02-17T12:38:48.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-02-17T12:38:48.442 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-17T12:38:48.515 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-02-17T12:38:48.515 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2024-02-17T12:38:48.543 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-17T12:38:48.610 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-17T12:38:48.610 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:38:48.610 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-17T12:38:48.639 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-17T12:38:48.708 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-02-17T12:38:48.708 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-17T12:38:48.740 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-17T12:38:48.811 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-02-17T12:38:48.811 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-17T12:38:48.838 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-17T12:38:48.905 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-17T12:38:48.905 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:38:48.905 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/stdin-killer 2024-02-17T12:38:48.934 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-17T12:38:49.002 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-02-17T12:38:49.003 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-02-17T12:38:49.034 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-17T12:38:49.105 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-02-17T12:38:49.105 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2024-02-17T12:38:49.132 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-17T12:38:49.199 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-17T12:38:49.298 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-02-17T12:38:49.298 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:38:49.299 INFO:tasks.cephadm:Cluster fsid is 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:38:49.299 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-17T12:38:49.299 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.18', 'mon.b': '172.21.15.119', 'mon.c': '172.21.15.146'} 2024-02-17T12:38:49.299 INFO:tasks.cephadm:First mon is mon.a on smithi018 2024-02-17T12:38:49.299 INFO:tasks.cephadm:First mgr is x 2024-02-17T12:38:49.299 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-17T12:38:49.299 DEBUG:teuthology.orchestra.run.smithi018:> sudo hostname $(hostname -s) 2024-02-17T12:38:49.331 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname $(hostname -s) 2024-02-17T12:38:49.362 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname $(hostname -s) 2024-02-17T12:38:49.389 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-17T12:38:49.389 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:38:49.677 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76836/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1515-g6822c7b5', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1515.g6822c7b5'}, 'url': 'https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-16 19:04:48.315261', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'main', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-02-17T12:38:49.920 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref main, sha1 6822c7b54e949006be7a0802650d01723d834d1d from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:38:49.922 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/x86_64/flavors/default/cephadm 2024-02-17T12:38:49.922 DEBUG:teuthology.orchestra.run.smithi018:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-17T12:38:50.362 INFO:teuthology.orchestra.run.smithi018.stdout:-rw-r--r--. 1 ubuntu ubuntu 766938 Feb 17 12:38 /home/ubuntu/cephtest/cephadm 2024-02-17T12:38:50.363 DEBUG:teuthology.orchestra.run.smithi119:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-17T12:38:51.443 INFO:teuthology.orchestra.run.smithi119.stdout:-rw-r--r--. 1 ubuntu ubuntu 766938 Feb 17 12:38 /home/ubuntu/cephtest/cephadm 2024-02-17T12:38:51.443 DEBUG:teuthology.orchestra.run.smithi146:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-17T12:38:51.924 INFO:teuthology.orchestra.run.smithi146.stdout:-rw-r--r--. 1 ubuntu ubuntu 766938 Feb 17 12:38 /home/ubuntu/cephtest/cephadm 2024-02-17T12:38:51.924 DEBUG:teuthology.orchestra.run.smithi018:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-17T12:38:51.943 DEBUG:teuthology.orchestra.run.smithi119:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-17T12:38:51.961 DEBUG:teuthology.orchestra.run.smithi146:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-17T12:38:51.997 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d on all hosts... 2024-02-17T12:38:51.997 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d pull 2024-02-17T12:38:51.999 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d pull 2024-02-17T12:38:52.004 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d pull 2024-02-17T12:38:52.811 INFO:teuthology.orchestra.run.smithi146.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d... 2024-02-17T12:38:52.850 INFO:teuthology.orchestra.run.smithi119.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d... 2024-02-17T12:38:53.044 INFO:teuthology.orchestra.run.smithi018.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d... 2024-02-17T12:39:38.929 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-02-17T12:39:38.930 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev)", 2024-02-17T12:39:38.930 INFO:teuthology.orchestra.run.smithi146.stdout: "image_id": "32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd", 2024-02-17T12:39:38.930 INFO:teuthology.orchestra.run.smithi146.stdout: "repo_digests": [ 2024-02-17T12:39:38.930 INFO:teuthology.orchestra.run.smithi146.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d" 2024-02-17T12:39:38.930 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-02-17T12:39:38.930 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-02-17T12:39:39.165 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:39:39.165 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_version": "ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev)", 2024-02-17T12:39:39.165 INFO:teuthology.orchestra.run.smithi018.stdout: "image_id": "32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd", 2024-02-17T12:39:39.165 INFO:teuthology.orchestra.run.smithi018.stdout: "repo_digests": [ 2024-02-17T12:39:39.165 INFO:teuthology.orchestra.run.smithi018.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d" 2024-02-17T12:39:39.165 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-02-17T12:39:39.165 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:39:39.866 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-02-17T12:39:39.866 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev)", 2024-02-17T12:39:39.866 INFO:teuthology.orchestra.run.smithi119.stdout: "image_id": "32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd", 2024-02-17T12:39:39.866 INFO:teuthology.orchestra.run.smithi119.stdout: "repo_digests": [ 2024-02-17T12:39:39.866 INFO:teuthology.orchestra.run.smithi119.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d" 2024-02-17T12:39:39.867 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-02-17T12:39:39.867 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-02-17T12:39:39.891 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /etc/ceph 2024-02-17T12:39:39.926 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph 2024-02-17T12:39:39.956 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph 2024-02-17T12:39:39.989 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 777 /etc/ceph 2024-02-17T12:39:40.020 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /etc/ceph 2024-02-17T12:39:40.047 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /etc/ceph 2024-02-17T12:39:40.075 INFO:tasks.cephadm:Writing seed config... 2024-02-17T12:39:40.076 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-17T12:39:40.076 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-17T12:39:40.076 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-17T12:39:40.076 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-17T12:39:40.076 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-17T12:39:40.076 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-02-17T12:39:40.076 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-17T12:39:40.076 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-17T12:39:40.076 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-17T12:39:40.077 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-17T12:39:40.078 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-17T12:39:40.079 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-17T12:39:40.079 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-17T12:39:40.079 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-17T12:39:40.079 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-17T12:39:40.079 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-17T12:39:40.079 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-17T12:39:40.079 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-17T12:39:40.079 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-17T12:39:40.079 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-02-17T12:39:40.080 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:39:40.080 DEBUG:teuthology.orchestra.run.smithi018:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-17T12:39:40.096 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7fca71fe-cd91-11ee-95ba-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-17T12:39:40.097 DEBUG:teuthology.orchestra.run.smithi018:mon.a> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.a.service 2024-02-17T12:39:40.139 DEBUG:teuthology.orchestra.run.smithi018:mgr.x> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.x.service 2024-02-17T12:39:40.181 INFO:tasks.cephadm:Bootstrapping... 2024-02-17T12:39:40.182 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d -v bootstrap --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.18 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-17T12:39:40.382 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-02-17T12:39:40.382 INFO:teuthology.orchestra.run.smithi018.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d', '-v', 'bootstrap', '--fsid', '7fca71fe-cd91-11ee-95ba-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.18', '--skip-admin-label'] 2024-02-17T12:39:40.407 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-17T12:39:40.408 INFO:teuthology.orchestra.run.smithi018.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-17T12:39:40.408 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying podman|docker is present... 2024-02-17T12:39:40.431 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-17T12:39:40.431 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying lvm2 is present... 2024-02-17T12:39:40.431 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying time synchronization is in place... 2024-02-17T12:39:40.438 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-17T12:39:40.438 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-17T12:39:40.444 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-17T12:39:40.444 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-02-17T12:39:40.450 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout enabled 2024-02-17T12:39:40.456 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout active 2024-02-17T12:39:40.456 INFO:teuthology.orchestra.run.smithi018.stdout:Unit chronyd.service is enabled and running 2024-02-17T12:39:40.456 INFO:teuthology.orchestra.run.smithi018.stdout:Repeating the final host check... 2024-02-17T12:39:40.479 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-17T12:39:40.479 INFO:teuthology.orchestra.run.smithi018.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-17T12:39:40.479 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl is present 2024-02-17T12:39:40.479 INFO:teuthology.orchestra.run.smithi018.stdout:lvcreate is present 2024-02-17T12:39:40.485 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-17T12:39:40.485 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-17T12:39:40.491 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-17T12:39:40.491 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-02-17T12:39:40.497 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout enabled 2024-02-17T12:39:40.503 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout active 2024-02-17T12:39:40.504 INFO:teuthology.orchestra.run.smithi018.stdout:Unit chronyd.service is enabled and running 2024-02-17T12:39:40.504 INFO:teuthology.orchestra.run.smithi018.stdout:Host looks OK 2024-02-17T12:39:40.504 INFO:teuthology.orchestra.run.smithi018.stdout:Cluster fsid: 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:39:40.504 INFO:teuthology.orchestra.run.smithi018.stdout:Acquiring lock 140199277811312 on /run/cephadm/7fca71fe-cd91-11ee-95ba-87774f69a715.lock 2024-02-17T12:39:40.504 INFO:teuthology.orchestra.run.smithi018.stdout:Lock 140199277811312 acquired on /run/cephadm/7fca71fe-cd91-11ee-95ba-87774f69a715.lock 2024-02-17T12:39:40.504 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying IP 172.21.15.18 port 3300 ... 2024-02-17T12:39:40.504 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying IP 172.21.15.18 port 6789 ... 2024-02-17T12:39:40.504 INFO:teuthology.orchestra.run.smithi018.stdout:Base mon IP(s) is [172.21.15.18:3300, 172.21.15.18:6789], mon addrv is [v2:172.21.15.18:3300,v1:172.21.15.18:6789] 2024-02-17T12:39:40.507 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.18 metric 100 2024-02-17T12:39:40.508 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.18 metric 100 2024-02-17T12:39:40.510 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-17T12:39:40.510 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-17T12:39:40.510 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-17T12:39:40.513 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-17T12:39:40.513 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-17T12:39:40.513 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-17T12:39:40.513 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-17T12:39:40.513 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1362/64 scope link noprefixroute 2024-02-17T12:39:40.513 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-17T12:39:40.514 INFO:teuthology.orchestra.run.smithi018.stdout:Mon IP `172.21.15.18` is in CIDR network `172.21.0.0/20` 2024-02-17T12:39:40.514 INFO:teuthology.orchestra.run.smithi018.stdout:Mon IP `172.21.15.18` is in CIDR network `172.21.0.0/20` 2024-02-17T12:39:40.514 INFO:teuthology.orchestra.run.smithi018.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-17T12:39:40.514 INFO:teuthology.orchestra.run.smithi018.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-17T12:39:40.515 INFO:teuthology.orchestra.run.smithi018.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d... 2024-02-17T12:39:41.051 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd 2024-02-17T12:39:41.051 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d... 2024-02-17T12:39:41.051 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Getting image source signatures 2024-02-17T12:39:41.051 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Copying blob sha256:f0670f204a477bfeca6489528d255a0313ae7bf2a9ae6454558a9981c6019cc7 2024-02-17T12:39:41.051 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-17T12:39:41.051 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Copying config sha256:32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd 2024-02-17T12:39:41.051 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-17T12:39:41.939 INFO:teuthology.orchestra.run.smithi018.stdout:ceph: stdout ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:39:41.939 INFO:teuthology.orchestra.run.smithi018.stdout:Ceph version: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:39:41.939 INFO:teuthology.orchestra.run.smithi018.stdout:Extracting ceph user uid/gid from container image... 2024-02-17T12:39:42.787 INFO:teuthology.orchestra.run.smithi018.stdout:stat: stdout 167 167 2024-02-17T12:39:42.787 INFO:teuthology.orchestra.run.smithi018.stdout:Creating initial keys... 2024-02-17T12:39:43.826 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph-authtool: stdout AQAPqdBlfZkLCBAAEUjxEoMXBpaCiMfwqK7vFw== 2024-02-17T12:39:44.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph-authtool: stdout AQAQqdBlCMFtCBAAvkRp8oawV4GH1y/LNntjJw== 2024-02-17T12:39:45.630 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph-authtool: stdout AQARqdBlUZBBBhAAXrebbQcGdNN2MNJGeBdkrg== 2024-02-17T12:39:45.631 INFO:teuthology.orchestra.run.smithi018.stdout:Creating initial monmap... 2024-02-17T12:39:46.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-17T12:39:46.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-17T12:39:46.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:39:46.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-17T12:39:46.495 INFO:teuthology.orchestra.run.smithi018.stdout:monmaptool for a [v2:172.21.15.18:3300,v1:172.21.15.18:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-17T12:39:46.495 INFO:teuthology.orchestra.run.smithi018.stdout:setting min_mon_release = quincy 2024-02-17T12:39:46.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: set fsid to 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:39:46.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-17T12:39:46.495 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:39:46.495 INFO:teuthology.orchestra.run.smithi018.stdout:Creating mon... 2024-02-17T12:39:47.452 INFO:teuthology.orchestra.run.smithi018.stdout:create mon.a on 2024-02-17T12:39:47.654 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-17T12:39:47.808 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-17T12:39:47.988 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7fca71fe-cd91-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-7fca71fe-cd91-11ee-95ba-87774f69a715.target. 2024-02-17T12:39:47.988 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-7fca71fe-cd91-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-7fca71fe-cd91-11ee-95ba-87774f69a715.target. 2024-02-17T12:39:48.253 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.a 2024-02-17T12:39:48.253 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.a.service: Unit ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.a.service not loaded. 2024-02-17T12:39:48.411 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7fca71fe-cd91-11ee-95ba-87774f69a715.target.wants/ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.a.service → /etc/systemd/system/ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@.service. 2024-02-17T12:39:48.928 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-17T12:39:48.928 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-02-17T12:39:48.935 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-17T12:39:48.935 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-02-17T12:39:48.935 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-02-17T12:39:48.935 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to enable service . firewalld.service is not available 2024-02-17T12:39:48.935 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mon to start... 2024-02-17T12:39:48.935 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mon... 2024-02-17T12:39:49.986 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:49 smithi018 ceph-mon[32253]: from='client.? 172.21.15.18:0/1194969416' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-17T12:39:49.994 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout cluster: 2024-02-17T12:39:49.994 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout id: 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:39:49.994 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout services: 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.549948s) 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout data: 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout pgs: 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:39:49.995 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.351+0000 7f6e137db700 1 Processor -- start 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.351+0000 7f6e137db700 1 -- start start 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.351+0000 7f6e137db700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 0x7f6e0c107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.351+0000 7f6e137db700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c1080c0 con 0x7f6e0c1076d0 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.352+0000 7f6e11577700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 0x7f6e0c107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.352+0000 7f6e11577700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 0x7f6e0c107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58814/0 (socket says 172.21.15.18:58814) 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.352+0000 7f6e11577700 1 -- 172.21.15.18:0/4164129114 learned_addr learned my addr 172.21.15.18:0/4164129114 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.352+0000 7f6e11577700 1 -- 172.21.15.18:0/4164129114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c108880 con 0x7f6e0c1076d0 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.352+0000 7f6e11577700 1 --2- 172.21.15.18:0/4164129114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 0x7f6e0c107af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6e080098a0 tx=0x7f6e08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e418c834a13373 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.352+0000 7f6e03fff700 1 -- 172.21.15.18:0/4164129114 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e08004160 con 0x7f6e0c1076d0 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.352+0000 7f6e03fff700 1 -- 172.21.15.18:0/4164129114 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e080042c0 con 0x7f6e0c1076d0 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.352+0000 7f6e03fff700 1 -- 172.21.15.18:0/4164129114 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e080045a0 con 0x7f6e0c1076d0 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.353+0000 7f6e137db700 1 -- 172.21.15.18:0/4164129114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 msgr2=0x7f6e0c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:39:49.996 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.353+0000 7f6e137db700 1 --2- 172.21.15.18:0/4164129114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 0x7f6e0c107af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6e080098a0 tx=0x7f6e08009bb0 comp rx=0 tx=0).stop 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.353+0000 7f6e137db700 1 -- 172.21.15.18:0/4164129114 shutdown_connections 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.353+0000 7f6e137db700 1 --2- 172.21.15.18:0/4164129114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 0x7f6e0c107af0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.353+0000 7f6e137db700 1 -- 172.21.15.18:0/4164129114 >> 172.21.15.18:0/4164129114 conn(0x7f6e0c102d50 msgr2=0x7f6e0c105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.353+0000 7f6e137db700 1 -- 172.21.15.18:0/4164129114 shutdown_connections 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.353+0000 7f6e137db700 1 -- 172.21.15.18:0/4164129114 wait complete. 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.353+0000 7f6e137db700 1 Processor -- start 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e137db700 1 -- start start 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e137db700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 0x7f6e0c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e137db700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c19b110 con 0x7f6e0c1076d0 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e11577700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 0x7f6e0c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e11577700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 0x7f6e0c19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58826/0 (socket says 172.21.15.18:58826) 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e11577700 1 -- 172.21.15.18:0/1194969416 learned_addr learned my addr 172.21.15.18:0/1194969416 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:39:49.997 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e11577700 1 -- 172.21.15.18:0/1194969416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e08009550 con 0x7f6e0c1076d0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e11577700 1 --2- 172.21.15.18:0/1194969416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 0x7f6e0c19abd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6e08004000 tx=0x7f6e08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e027fc700 1 -- 172.21.15.18:0/1194969416 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e08011630 con 0x7f6e0c1076d0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e027fc700 1 -- 172.21.15.18:0/1194969416 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e08011790 con 0x7f6e0c1076d0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e137db700 1 -- 172.21.15.18:0/1194969416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0c19b310 con 0x7f6e0c1076d0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e027fc700 1 -- 172.21.15.18:0/1194969416 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e08011a30 con 0x7f6e0c1076d0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.354+0000 7f6e137db700 1 -- 172.21.15.18:0/1194969416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c19b7b0 con 0x7f6e0c1076d0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.355+0000 7f6e027fc700 1 -- 172.21.15.18:0/1194969416 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6e08021070 con 0x7f6e0c1076d0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.355+0000 7f6e027fc700 1 -- 172.21.15.18:0/1194969416 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6e0801b810 con 0x7f6e0c1076d0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.355+0000 7f6e137db700 1 -- 172.21.15.18:0/1194969416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0c1948a0 con 0x7f6e0c1076d0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.358+0000 7f6e027fc700 1 -- 172.21.15.18:0/1194969416 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6e0801ba70 con 0x7f6e0c1076d0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.431+0000 7f6e137db700 1 -- 172.21.15.18:0/1194969416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f6e0c108e60 con 0x7f6e0c1076d0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.431+0000 7f6e027fc700 1 -- 172.21.15.18:0/1194969416 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f6e0801bc10 con 0x7f6e0c1076d0 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.432+0000 7f6e137db700 1 -- 172.21.15.18:0/1194969416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 msgr2=0x7f6e0c19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:39:49.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.432+0000 7f6e137db700 1 --2- 172.21.15.18:0/1194969416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 0x7f6e0c19abd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6e08004000 tx=0x7f6e08004750 comp rx=0 tx=0).stop 2024-02-17T12:39:49.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.432+0000 7f6e137db700 1 -- 172.21.15.18:0/1194969416 shutdown_connections 2024-02-17T12:39:49.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.432+0000 7f6e137db700 1 --2- 172.21.15.18:0/1194969416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e0c1076d0 0x7f6e0c19abd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:39:49.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.432+0000 7f6e137db700 1 -- 172.21.15.18:0/1194969416 >> 172.21.15.18:0/1194969416 conn(0x7f6e0c102d50 msgr2=0x7f6e0c1919e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:39:49.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.432+0000 7f6e137db700 1 -- 172.21.15.18:0/1194969416 shutdown_connections 2024-02-17T12:39:49.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:49.432+0000 7f6e137db700 1 -- 172.21.15.18:0/1194969416 wait complete. 2024-02-17T12:39:49.999 INFO:teuthology.orchestra.run.smithi018.stdout:mon is available 2024-02-17T12:39:49.999 INFO:teuthology.orchestra.run.smithi018.stdout:Assimilating anything we can from ceph.conf... 2024-02-17T12:39:51.058 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:50 smithi018 ceph-mon[32253]: from='client.? 172.21.15.18:0/369177488' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-17T12:39:51.058 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:50 smithi018 ceph-mon[32253]: from='client.? 172.21.15.18:0/369177488' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-17T12:39:51.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:39:51.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [global] 2024-02-17T12:39:51.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout fsid = 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:39:51.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.18:3300,v1:172.21.15.18:6789] 2024-02-17T12:39:51.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-17T12:39:51.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-17T12:39:51.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-17T12:39:51.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [osd] 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.419+0000 7f07794a5700 1 Processor -- start 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.419+0000 7f07794a5700 1 -- start start 2024-02-17T12:39:51.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.419+0000 7f07794a5700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 0x7f0774106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:39:51.073 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.419+0000 7f07794a5700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0774107030 con 0x7f0774106640 2024-02-17T12:39:51.073 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.419+0000 7f0772ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 0x7f0774106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:39:51.073 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.419+0000 7f0772ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 0x7f0774106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43112/0 (socket says 172.21.15.18:43112) 2024-02-17T12:39:51.073 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.419+0000 7f0772ffd700 1 -- 172.21.15.18:0/1628352646 learned_addr learned my addr 172.21.15.18:0/1628352646 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:39:51.073 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.419+0000 7f0772ffd700 1 -- 172.21.15.18:0/1628352646 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0774107840 con 0x7f0774106640 2024-02-17T12:39:51.073 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.420+0000 7f0772ffd700 1 --2- 172.21.15.18:0/1628352646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 0x7f0774106a60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f07640098a0 tx=0x7f0764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f6af2354ee856a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:39:51.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.420+0000 7f0771ffb700 1 -- 172.21.15.18:0/1628352646 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0764004160 con 0x7f0774106640 2024-02-17T12:39:51.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.420+0000 7f0771ffb700 1 -- 172.21.15.18:0/1628352646 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07640042c0 con 0x7f0774106640 2024-02-17T12:39:51.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.420+0000 7f0771ffb700 1 -- 172.21.15.18:0/1628352646 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07640045a0 con 0x7f0774106640 2024-02-17T12:39:51.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.420+0000 7f07794a5700 1 -- 172.21.15.18:0/1628352646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 msgr2=0x7f0774106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:39:51.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.420+0000 7f07794a5700 1 --2- 172.21.15.18:0/1628352646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 0x7f0774106a60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f07640098a0 tx=0x7f0764009bb0 comp rx=0 tx=0).stop 2024-02-17T12:39:51.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.420+0000 7f07794a5700 1 -- 172.21.15.18:0/1628352646 shutdown_connections 2024-02-17T12:39:51.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.420+0000 7f07794a5700 1 --2- 172.21.15.18:0/1628352646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 0x7f0774106a60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:39:51.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.420+0000 7f07794a5700 1 -- 172.21.15.18:0/1628352646 >> 172.21.15.18:0/1628352646 conn(0x7f0774101c00 msgr2=0x7f0774104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:39:51.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.420+0000 7f07794a5700 1 -- 172.21.15.18:0/1628352646 shutdown_connections 2024-02-17T12:39:51.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.420+0000 7f07794a5700 1 -- 172.21.15.18:0/1628352646 wait complete. 2024-02-17T12:39:51.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.421+0000 7f07794a5700 1 Processor -- start 2024-02-17T12:39:51.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.421+0000 7f07794a5700 1 -- start start 2024-02-17T12:39:51.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.421+0000 7f07794a5700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 0x7f077419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:39:51.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.421+0000 7f07794a5700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077419b180 con 0x7f0774106640 2024-02-17T12:39:51.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.421+0000 7f0772ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 0x7f077419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:39:51.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.421+0000 7f0772ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 0x7f077419ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43124/0 (socket says 172.21.15.18:43124) 2024-02-17T12:39:51.076 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.421+0000 7f0772ffd700 1 -- 172.21.15.18:0/369177488 learned_addr learned my addr 172.21.15.18:0/369177488 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:39:51.076 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.422+0000 7f0772ffd700 1 -- 172.21.15.18:0/369177488 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0764009550 con 0x7f0774106640 2024-02-17T12:39:51.076 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.422+0000 7f0772ffd700 1 --2- 172.21.15.18:0/369177488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 0x7f077419ac40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0764004000 tx=0x7f0764004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:39:51.076 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.422+0000 7f075bfff700 1 -- 172.21.15.18:0/369177488 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0764011630 con 0x7f0774106640 2024-02-17T12:39:51.076 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.422+0000 7f075bfff700 1 -- 172.21.15.18:0/369177488 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0764011790 con 0x7f0774106640 2024-02-17T12:39:51.076 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.422+0000 7f075bfff700 1 -- 172.21.15.18:0/369177488 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0764011a30 con 0x7f0774106640 2024-02-17T12:39:51.076 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.422+0000 7f07794a5700 1 -- 172.21.15.18:0/369177488 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077419b380 con 0x7f0774106640 2024-02-17T12:39:51.076 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.422+0000 7f07794a5700 1 -- 172.21.15.18:0/369177488 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077419b820 con 0x7f0774106640 2024-02-17T12:39:51.076 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.422+0000 7f075bfff700 1 -- 172.21.15.18:0/369177488 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0764021070 con 0x7f0774106640 2024-02-17T12:39:51.076 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.422+0000 7f075bfff700 1 -- 172.21.15.18:0/369177488 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f076401b820 con 0x7f0774106640 2024-02-17T12:39:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.423+0000 7f07794a5700 1 -- 172.21.15.18:0/369177488 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0774194850 con 0x7f0774106640 2024-02-17T12:39:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.425+0000 7f075bfff700 1 -- 172.21.15.18:0/369177488 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f076401b9e0 con 0x7f0774106640 2024-02-17T12:39:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.495+0000 7f07794a5700 1 -- 172.21.15.18:0/369177488 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0774107bb0 con 0x7f0774106640 2024-02-17T12:39:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.498+0000 7f075bfff700 1 -- 172.21.15.18:0/369177488 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f076401bc30 con 0x7f0774106640 2024-02-17T12:39:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.498+0000 7f075bfff700 1 -- 172.21.15.18:0/369177488 <== mon.0 v2:172.21.15.18:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0764035a50 con 0x7f0774106640 2024-02-17T12:39:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.499+0000 7f07794a5700 1 -- 172.21.15.18:0/369177488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 msgr2=0x7f077419ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:39:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.499+0000 7f07794a5700 1 --2- 172.21.15.18:0/369177488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 0x7f077419ac40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0764004000 tx=0x7f0764004750 comp rx=0 tx=0).stop 2024-02-17T12:39:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.499+0000 7f07794a5700 1 -- 172.21.15.18:0/369177488 shutdown_connections 2024-02-17T12:39:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.499+0000 7f07794a5700 1 --2- 172.21.15.18:0/369177488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0774106640 0x7f077419ac40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:39:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.499+0000 7f07794a5700 1 -- 172.21.15.18:0/369177488 >> 172.21.15.18:0/369177488 conn(0x7f0774101c00 msgr2=0x7f0774191990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:39:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.499+0000 7f07794a5700 1 -- 172.21.15.18:0/369177488 shutdown_connections 2024-02-17T12:39:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:50.499+0000 7f07794a5700 1 -- 172.21.15.18:0/369177488 wait complete. 2024-02-17T12:39:51.078 INFO:teuthology.orchestra.run.smithi018.stdout:Generating new minimal ceph.conf... 2024-02-17T12:39:52.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.461+0000 7f9af48ee700 1 Processor -- start 2024-02-17T12:39:52.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.461+0000 7f9af48ee700 1 -- start start 2024-02-17T12:39:52.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.461+0000 7f9af48ee700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 0x7f9af01069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:39:52.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.461+0000 7f9af48ee700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af0106fc0 con 0x7f9af01065d0 2024-02-17T12:39:52.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.461+0000 7f9aee59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 0x7f9af01069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:39:52.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.461+0000 7f9aee59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 0x7f9af01069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43132/0 (socket says 172.21.15.18:43132) 2024-02-17T12:39:52.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.461+0000 7f9aee59c700 1 -- 172.21.15.18:0/3943882320 learned_addr learned my addr 172.21.15.18:0/3943882320 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.462+0000 7f9aee59c700 1 -- 172.21.15.18:0/3943882320 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af01077d0 con 0x7f9af01065d0 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.462+0000 7f9aee59c700 1 --2- 172.21.15.18:0/3943882320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 0x7f9af01069f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00098a0 tx=0x7f9ae0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608ddebc01fe41a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.462+0000 7f9aed59a700 1 -- 172.21.15.18:0/3943882320 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ae0004160 con 0x7f9af01065d0 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.462+0000 7f9aed59a700 1 -- 172.21.15.18:0/3943882320 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ae00042c0 con 0x7f9af01065d0 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.462+0000 7f9aed59a700 1 -- 172.21.15.18:0/3943882320 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ae0011ac0 con 0x7f9af01065d0 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.462+0000 7f9af48ee700 1 -- 172.21.15.18:0/3943882320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 msgr2=0x7f9af01069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.462+0000 7f9af48ee700 1 --2- 172.21.15.18:0/3943882320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 0x7f9af01069f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00098a0 tx=0x7f9ae0009bb0 comp rx=0 tx=0).stop 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.463+0000 7f9af48ee700 1 -- 172.21.15.18:0/3943882320 shutdown_connections 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.463+0000 7f9af48ee700 1 --2- 172.21.15.18:0/3943882320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 0x7f9af01069f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.463+0000 7f9af48ee700 1 -- 172.21.15.18:0/3943882320 >> 172.21.15.18:0/3943882320 conn(0x7f9af0101b70 msgr2=0x7f9af0103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.463+0000 7f9af48ee700 1 -- 172.21.15.18:0/3943882320 shutdown_connections 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.463+0000 7f9af48ee700 1 -- 172.21.15.18:0/3943882320 wait complete. 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.463+0000 7f9af48ee700 1 Processor -- start 2024-02-17T12:39:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.463+0000 7f9af48ee700 1 -- start start 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.463+0000 7f9af48ee700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 0x7f9af0191740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.464+0000 7f9af48ee700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af0191c80 con 0x7f9af01065d0 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.464+0000 7f9aee59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 0x7f9af0191740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.464+0000 7f9aee59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 0x7f9af0191740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43136/0 (socket says 172.21.15.18:43136) 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.464+0000 7f9aee59c700 1 -- 172.21.15.18:0/172816277 learned_addr learned my addr 172.21.15.18:0/172816277 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.464+0000 7f9aee59c700 1 -- 172.21.15.18:0/172816277 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae0009550 con 0x7f9af01065d0 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.464+0000 7f9aee59c700 1 --2- 172.21.15.18:0/172816277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 0x7f9af0191740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009fb0 tx=0x7f9ae0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.464+0000 7f9adf7fe700 1 -- 172.21.15.18:0/172816277 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ae00038b0 con 0x7f9af01065d0 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.464+0000 7f9adf7fe700 1 -- 172.21.15.18:0/172816277 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ae0003a10 con 0x7f9af01065d0 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.464+0000 7f9af48ee700 1 -- 172.21.15.18:0/172816277 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af018fe70 con 0x7f9af01065d0 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.464+0000 7f9adf7fe700 1 -- 172.21.15.18:0/172816277 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ae00195e0 con 0x7f9af01065d0 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.464+0000 7f9af48ee700 1 -- 172.21.15.18:0/172816277 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af0190310 con 0x7f9af01065d0 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.464+0000 7f9adf7fe700 1 -- 172.21.15.18:0/172816277 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9ae0019c00 con 0x7f9af01065d0 2024-02-17T12:39:52.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.465+0000 7f9adf7fe700 1 -- 172.21.15.18:0/172816277 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9ae0024990 con 0x7f9af01065d0 2024-02-17T12:39:52.057 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.465+0000 7f9af48ee700 1 -- 172.21.15.18:0/172816277 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af0190d20 con 0x7f9af01065d0 2024-02-17T12:39:52.057 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.468+0000 7f9adf7fe700 1 -- 172.21.15.18:0/172816277 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9ae0003b80 con 0x7f9af01065d0 2024-02-17T12:39:52.057 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.538+0000 7f9af48ee700 1 -- 172.21.15.18:0/172816277 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9af010aa00 con 0x7f9af01065d0 2024-02-17T12:39:52.057 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.538+0000 7f9adf7fe700 1 -- 172.21.15.18:0/172816277 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f9ae001aac0 con 0x7f9af01065d0 2024-02-17T12:39:52.057 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.539+0000 7f9af48ee700 1 -- 172.21.15.18:0/172816277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 msgr2=0x7f9af0191740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:39:52.057 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.539+0000 7f9af48ee700 1 --2- 172.21.15.18:0/172816277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 0x7f9af0191740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009fb0 tx=0x7f9ae0011b20 comp rx=0 tx=0).stop 2024-02-17T12:39:52.057 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.539+0000 7f9af48ee700 1 -- 172.21.15.18:0/172816277 shutdown_connections 2024-02-17T12:39:52.057 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.539+0000 7f9af48ee700 1 --2- 172.21.15.18:0/172816277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9af01065d0 0x7f9af0191740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:39:52.057 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.539+0000 7f9af48ee700 1 -- 172.21.15.18:0/172816277 >> 172.21.15.18:0/172816277 conn(0x7f9af0101b70 msgr2=0x7f9af0102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:39:52.057 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.539+0000 7f9af48ee700 1 -- 172.21.15.18:0/172816277 shutdown_connections 2024-02-17T12:39:52.057 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:51.539+0000 7f9af48ee700 1 -- 172.21.15.18:0/172816277 wait complete. 2024-02-17T12:39:52.057 INFO:teuthology.orchestra.run.smithi018.stdout:Restarting the monitor... 2024-02-17T12:39:52.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:51 smithi018 ceph-mon[32253]: from='client.? 172.21.15.18:0/172816277' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:39:52.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:52 smithi018 systemd[1]: Stopping Ceph mon.a for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:39:52.585 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a[32249]: 2024-02-17T12:39:52.304+0000 7f8d1b993700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:39:52.585 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a[32249]: 2024-02-17T12:39:52.304+0000 7f8d1b993700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-17T12:39:52.585 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:52 smithi018 podman[32500]: 2024-02-17 12:39:52.387282171 +0000 UTC m=+0.167938376 container died 46917ba772767064d4e4a2c0af0096b2cfe917fa696fd37d33cab86f4946c13d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=main-6822c7b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T12:39:52.902 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:52 smithi018 podman[32500]: 2024-02-17 12:39:52.688058285 +0000 UTC m=+0.468714488 container cleanup 46917ba772767064d4e4a2c0af0096b2cfe917fa696fd37d33cab86f4946c13d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-17T12:39:52.902 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:52 smithi018 bash[32500]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a 2024-02-17T12:39:53.236 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:52 smithi018 podman[32514]: 2024-02-17 12:39:52.905816989 +0000 UTC m=+0.516167909 container remove 46917ba772767064d4e4a2c0af0096b2cfe917fa696fd37d33cab86f4946c13d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-17T12:39:53.236 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-17T12:39:53.236 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 systemd[1]: Stopped Ceph mon.a for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:39:53.237 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 systemd[1]: Starting Ceph mon.a for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:39:53.237 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 podman[32606]: 2024-02-17 12:39:53.240126105 +0000 UTC m=+0.111638084 container create 0786d0b495a8fef71e03a9f2f818b75f54ace37ef0feafe2838aae4f0f7e9261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-17T12:39:53.497 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 podman[32606]: 2024-02-17 12:39:53.14529507 +0000 UTC m=+0.016807007 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:39:53.497 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 podman[32606]: 2024-02-17 12:39:53.425506924 +0000 UTC m=+0.297018864 container init 0786d0b495a8fef71e03a9f2f818b75f54ace37ef0feafe2838aae4f0f7e9261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T12:39:53.497 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 podman[32606]: 2024-02-17 12:39:53.428279944 +0000 UTC m=+0.299791885 container start 0786d0b495a8fef71e03a9f2f818b75f54ace37ef0feafe2838aae4f0f7e9261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-17T12:39:53.497 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: set uid:gid to 167:167 (ceph:ceph) 2024-02-17T12:39:53.497 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev), process ceph-mon, pid 2 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: pidfile_write: ignore empty --pid-file 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: load: jerasure load: lrc 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: RocksDB version: 7.9.2 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Git sha 0 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Compile date 2024-02-16 17:52:22 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: DB SUMMARY 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: DB Session ID: 596KSANCKLYM71MNRZLD 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: CURRENT file: CURRENT 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: IDENTITY file: IDENTITY 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100532 ; 2024-02-17T12:39:53.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.error_if_exists: 0 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.create_if_missing: 0 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.paranoid_checks: 1 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.env: 0x557620b2bae0 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.fs: PosixFileSystem 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.info_log: 0x5576224178c0 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_file_opening_threads: 16 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.statistics: (nil) 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.use_fsync: 0 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_log_file_size: 0 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-17T12:39:53.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-17T12:39:53.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.keep_log_file_num: 1000 2024-02-17T12:39:53.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.recycle_log_file_num: 0 2024-02-17T12:39:53.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.allow_fallocate: 1 2024-02-17T12:39:53.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.allow_mmap_reads: 0 2024-02-17T12:39:53.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.allow_mmap_writes: 0 2024-02-17T12:39:53.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.use_direct_reads: 0 2024-02-17T12:39:53.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-17T12:39:53.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.create_missing_column_families: 0 2024-02-17T12:39:53.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.db_log_dir: 2024-02-17T12:39:53.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.wal_dir: 2024-02-17T12:39:53.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-17T12:39:53.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-17T12:39:53.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-17T12:39:53.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-17T12:39:53.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-17T12:39:53.502 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-17T12:39:53.502 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.advise_random_on_open: 1 2024-02-17T12:39:53.502 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.db_write_buffer_size: 0 2024-02-17T12:39:53.502 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.write_buffer_manager: 0x5576223eb400 2024-02-17T12:39:53.502 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-17T12:39:53.502 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-17T12:39:53.502 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-17T12:39:53.502 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.rate_limiter: (nil) 2024-02-17T12:39:53.503 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-17T12:39:53.503 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.wal_recovery_mode: 2 2024-02-17T12:39:53.503 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.enable_thread_tracking: 0 2024-02-17T12:39:53.503 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.enable_pipelined_write: 0 2024-02-17T12:39:53.503 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.unordered_write: 0 2024-02-17T12:39:53.503 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-17T12:39:53.503 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-17T12:39:53.503 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-17T12:39:53.503 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-17T12:39:53.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.row_cache: None 2024-02-17T12:39:53.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.wal_filter: None 2024-02-17T12:39:53.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-17T12:39:53.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.allow_ingest_behind: 0 2024-02-17T12:39:53.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.two_write_queues: 0 2024-02-17T12:39:53.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.manual_wal_flush: 0 2024-02-17T12:39:53.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.wal_compression: 0 2024-02-17T12:39:53.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.atomic_flush: 0 2024-02-17T12:39:53.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-17T12:39:53.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-17T12:39:53.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-17T12:39:53.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.log_readahead_size: 0 2024-02-17T12:39:53.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-17T12:39:53.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.best_efforts_recovery: 0 2024-02-17T12:39:53.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-17T12:39:53.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-17T12:39:53.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.allow_data_in_errors: 0 2024-02-17T12:39:53.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.db_host_id: __hostname__ 2024-02-17T12:39:53.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-17T12:39:53.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_background_jobs: 2 2024-02-17T12:39:53.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_background_compactions: -1 2024-02-17T12:39:53.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_subcompactions: 1 2024-02-17T12:39:53.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-17T12:39:53.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-17T12:39:53.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-17T12:39:53.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_total_wal_size: 0 2024-02-17T12:39:53.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-17T12:39:53.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-17T12:39:53.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-17T12:39:53.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-17T12:39:53.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_open_files: -1 2024-02-17T12:39:53.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bytes_per_sync: 0 2024-02-17T12:39:53.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-17T12:39:53.508 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-17T12:39:53.508 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_readahead_size: 0 2024-02-17T12:39:53.508 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_background_flushes: -1 2024-02-17T12:39:53.508 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Compression algorithms supported: 2024-02-17T12:39:53.508 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-17T12:39:53.508 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: kZSTD supported: 0 2024-02-17T12:39:53.508 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: kXpressCompression supported: 0 2024-02-17T12:39:53.508 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: kLZ4HCCompression supported: 1 2024-02-17T12:39:53.508 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: kZlibCompression supported: 1 2024-02-17T12:39:53.508 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: kSnappyCompression supported: 1 2024-02-17T12:39:53.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: kLZ4Compression supported: 1 2024-02-17T12:39:53.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: kBZip2Compression supported: 0 2024-02-17T12:39:53.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-17T12:39:53.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-17T12:39:53.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-17T12:39:53.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-17T12:39:53.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-17T12:39:53.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.merge_operator: 2024-02-17T12:39:53.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_filter: None 2024-02-17T12:39:53.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_filter_factory: None 2024-02-17T12:39:53.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.sst_partitioner_factory: None 2024-02-17T12:39:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-17T12:39:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-17T12:39:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5576224179c0) 2024-02-17T12:39:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout: cache_index_and_filter_blocks: 1 2024-02-17T12:39:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-17T12:39:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-17T12:39:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout: pin_top_level_index_and_filter: 1 2024-02-17T12:39:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout: index_type: 0 2024-02-17T12:39:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout: data_block_index_type: 0 2024-02-17T12:39:53.511 INFO:journalctl@ceph.mon.a.smithi018.stdout: index_shortening: 1 2024-02-17T12:39:53.511 INFO:journalctl@ceph.mon.a.smithi018.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-17T12:39:53.511 INFO:journalctl@ceph.mon.a.smithi018.stdout: checksum: 4 2024-02-17T12:39:53.511 INFO:journalctl@ceph.mon.a.smithi018.stdout: no_block_cache: 0 2024-02-17T12:39:53.511 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_cache: 0x55762241b090 2024-02-17T12:39:53.511 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_cache_name: BinnedLRUCache 2024-02-17T12:39:53.511 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_cache_options: 2024-02-17T12:39:53.511 INFO:journalctl@ceph.mon.a.smithi018.stdout: capacity : 536870912 2024-02-17T12:39:53.512 INFO:journalctl@ceph.mon.a.smithi018.stdout: num_shard_bits : 4 2024-02-17T12:39:53.512 INFO:journalctl@ceph.mon.a.smithi018.stdout: strict_capacity_limit : 0 2024-02-17T12:39:53.512 INFO:journalctl@ceph.mon.a.smithi018.stdout: high_pri_pool_ratio: 0.000 2024-02-17T12:39:53.512 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_cache_compressed: (nil) 2024-02-17T12:39:53.512 INFO:journalctl@ceph.mon.a.smithi018.stdout: persistent_cache: (nil) 2024-02-17T12:39:53.512 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_size: 4096 2024-02-17T12:39:53.512 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_size_deviation: 10 2024-02-17T12:39:53.512 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_restart_interval: 16 2024-02-17T12:39:53.512 INFO:journalctl@ceph.mon.a.smithi018.stdout: index_block_restart_interval: 1 2024-02-17T12:39:53.513 INFO:journalctl@ceph.mon.a.smithi018.stdout: metadata_block_size: 4096 2024-02-17T12:39:53.513 INFO:journalctl@ceph.mon.a.smithi018.stdout: partition_filters: 0 2024-02-17T12:39:53.513 INFO:journalctl@ceph.mon.a.smithi018.stdout: use_delta_encoding: 1 2024-02-17T12:39:53.513 INFO:journalctl@ceph.mon.a.smithi018.stdout: filter_policy: bloomfilter 2024-02-17T12:39:53.513 INFO:journalctl@ceph.mon.a.smithi018.stdout: whole_key_filtering: 1 2024-02-17T12:39:53.513 INFO:journalctl@ceph.mon.a.smithi018.stdout: verify_compression: 0 2024-02-17T12:39:53.513 INFO:journalctl@ceph.mon.a.smithi018.stdout: read_amp_bytes_per_bit: 0 2024-02-17T12:39:53.513 INFO:journalctl@ceph.mon.a.smithi018.stdout: format_version: 5 2024-02-17T12:39:53.513 INFO:journalctl@ceph.mon.a.smithi018.stdout: enable_index_compression: 1 2024-02-17T12:39:53.513 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_align: 0 2024-02-17T12:39:53.513 INFO:journalctl@ceph.mon.a.smithi018.stdout: max_auto_readahead_size: 262144 2024-02-17T12:39:53.513 INFO:journalctl@ceph.mon.a.smithi018.stdout: prepopulate_block_cache: 0 2024-02-17T12:39:53.514 INFO:journalctl@ceph.mon.a.smithi018.stdout: initial_auto_readahead_size: 8192 2024-02-17T12:39:53.514 INFO:journalctl@ceph.mon.a.smithi018.stdout: num_file_reads_for_auto_readahead: 2 2024-02-17T12:39:53.514 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.write_buffer_size: 33554432 2024-02-17T12:39:53.514 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_write_buffer_number: 2 2024-02-17T12:39:53.514 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compression: NoCompression 2024-02-17T12:39:53.514 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bottommost_compression: Disabled 2024-02-17T12:39:53.514 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.prefix_extractor: nullptr 2024-02-17T12:39:53.514 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-17T12:39:53.514 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.num_levels: 7 2024-02-17T12:39:53.514 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-17T12:39:53.514 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-17T12:39:53.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-17T12:39:53.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-17T12:39:53.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-17T12:39:53.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-17T12:39:53.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-17T12:39:53.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-17T12:39:53.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compression_opts.level: 32767 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compression_opts.strategy: 0 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compression_opts.enabled: false 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-17T12:39:53.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.target_file_size_base: 67108864 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-17T12:39:53.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.arena_block_size: 1048576 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.disable_auto_compactions: 0 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-17T12:39:53.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-17T12:39:53.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-17T12:39:53.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-17T12:39:53.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.table_properties_collectors: 2024-02-17T12:39:53.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.inplace_update_support: 0 2024-02-17T12:39:53.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-17T12:39:53.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-17T12:39:53.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-17T12:39:53.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-17T12:39:53.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.bloom_locality: 0 2024-02-17T12:39:53.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.max_successive_merges: 0 2024-02-17T12:39:53.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-17T12:39:53.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.paranoid_file_checks: 0 2024-02-17T12:39:53.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.force_consistency_checks: 1 2024-02-17T12:39:53.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.report_bg_io_stats: 0 2024-02-17T12:39:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.ttl: 2592000 2024-02-17T12:39:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-17T12:39:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-17T12:39:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-17T12:39:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.enable_blob_files: false 2024-02-17T12:39:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.min_blob_size: 0 2024-02-17T12:39:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.blob_file_size: 268435456 2024-02-17T12:39:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-17T12:39:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-17T12:39:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-17T12:39:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-17T12:39:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.blob_file_starting_level: 0 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c6583f23-aa85-4dc9-b774-6e300e745646 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708173593456391, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708173593457325, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95593, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 93583, "index_size": 292, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13989, "raw_average_key_size": 50, "raw_value_size": 86024, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708173593, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c6583f23-aa85-4dc9-b774-6e300e745646", "db_session_id": "596KSANCKLYM71MNRZLD", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708173593457409, "job": 1, "event": "recovery_finished"} 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55762250c000 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: DB pointer 0x5576224f6000 2024-02-17T12:39:53.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** DB Stats ** 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** Compaction Stats [default] ** 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-17T12:39:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout: L0 2/0 95.19 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: Sum 2/0 95.19 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** Compaction Stats [default] ** 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 156.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-17T12:39:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-17T12:39:53.525 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-17T12:39:53.525 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-17T12:39:53.525 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative compaction: 0.00 GB write, 31.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-17T12:39:53.525 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval compaction: 0.00 GB write, 31.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-17T12:39:53.525 INFO:journalctl@ceph.mon.a.smithi018.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-17T12:39:53.525 INFO:journalctl@ceph.mon.a.smithi018.stdout: Block cache BinnedLRUCache@0x55762241b090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-17T12:39:53.525 INFO:journalctl@ceph.mon.a.smithi018.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-02-17T12:39:53.525 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-02-17T12:39:53.525 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-17T12:39:53.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: starting mon.a rank 0 at public addrs [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] at bind addrs [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:39:53.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: mon.a@-1(???) e1 preinit fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:39:53.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: mon.a@-1(???).mds e1 new map 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: mon.a@-1(???).mds e1 print_map 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout: e1 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout: legacy client fscid: -1 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout: No filesystems configured 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 ceph-mon[32625]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 bash[32606]: 0786d0b495a8fef71e03a9f2f818b75f54ace37ef0feafe2838aae4f0f7e9261 2024-02-17T12:39:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:53 smithi018 systemd[1]: Started Ceph mon.a for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:39:53.527 INFO:teuthology.orchestra.run.smithi018.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-17T12:39:54.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.965+0000 7f232b063700 1 Processor -- start 2024-02-17T12:39:54.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.965+0000 7f232b063700 1 -- start start 2024-02-17T12:39:54.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.965+0000 7f232b063700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 0x7f2324107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:39:54.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.965+0000 7f232b063700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2324108150 con 0x7f2324107760 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.966+0000 7f2328dff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 0x7f2324107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.966+0000 7f2328dff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 0x7f2324107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43148/0 (socket says 172.21.15.18:43148) 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.966+0000 7f2328dff700 1 -- 172.21.15.18:0/1227729165 learned_addr learned my addr 172.21.15.18:0/1227729165 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.966+0000 7f2328dff700 1 -- 172.21.15.18:0/1227729165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23241088c0 con 0x7f2324107760 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.966+0000 7f2328dff700 1 --2- 172.21.15.18:0/1227729165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 0x7f2324107b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f23180098a0 tx=0x7f2318009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f244fadb5623cf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.966+0000 7f23237fe700 1 -- 172.21.15.18:0/1227729165 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2318004160 con 0x7f2324107760 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.966+0000 7f23237fe700 1 -- 172.21.15.18:0/1227729165 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23180042c0 con 0x7f2324107760 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.966+0000 7f23237fe700 1 -- 172.21.15.18:0/1227729165 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2318011ac0 con 0x7f2324107760 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.967+0000 7f232b063700 1 -- 172.21.15.18:0/1227729165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 msgr2=0x7f2324107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.967+0000 7f232b063700 1 --2- 172.21.15.18:0/1227729165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 0x7f2324107b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f23180098a0 tx=0x7f2318009bb0 comp rx=0 tx=0).stop 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.967+0000 7f232b063700 1 -- 172.21.15.18:0/1227729165 shutdown_connections 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.967+0000 7f232b063700 1 --2- 172.21.15.18:0/1227729165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 0x7f2324107b80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.967+0000 7f232b063700 1 -- 172.21.15.18:0/1227729165 >> 172.21.15.18:0/1227729165 conn(0x7f2324102de0 msgr2=0x7f2324105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:39:54.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.967+0000 7f232b063700 1 -- 172.21.15.18:0/1227729165 shutdown_connections 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.967+0000 7f232b063700 1 -- 172.21.15.18:0/1227729165 wait complete. 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.968+0000 7f232b063700 1 Processor -- start 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.968+0000 7f232b063700 1 -- start start 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.968+0000 7f232b063700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 0x7f232419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.968+0000 7f232b063700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232419b9c0 con 0x7f2324107760 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.968+0000 7f2328dff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 0x7f232419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.968+0000 7f2328dff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 0x7f232419b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43158/0 (socket says 172.21.15.18:43158) 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.968+0000 7f2328dff700 1 -- 172.21.15.18:0/3719465075 learned_addr learned my addr 172.21.15.18:0/3719465075 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.968+0000 7f2328dff700 1 -- 172.21.15.18:0/3719465075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2318009550 con 0x7f2324107760 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.969+0000 7f2328dff700 1 --2- 172.21.15.18:0/3719465075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 0x7f232419b480 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2318011780 tx=0x7f2318011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.969+0000 7f2321ffb700 1 -- 172.21.15.18:0/3719465075 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2318003af0 con 0x7f2324107760 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.969+0000 7f2321ffb700 1 -- 172.21.15.18:0/3719465075 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2318003c50 con 0x7f2324107760 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.969+0000 7f232b063700 1 -- 172.21.15.18:0/3719465075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232419bbc0 con 0x7f2324107760 2024-02-17T12:39:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.969+0000 7f2321ffb700 1 -- 172.21.15.18:0/3719465075 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23180197e0 con 0x7f2324107760 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.969+0000 7f232b063700 1 -- 172.21.15.18:0/3719465075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232419c060 con 0x7f2324107760 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.969+0000 7f2321ffb700 1 -- 172.21.15.18:0/3719465075 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2318025460 con 0x7f2324107760 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.969+0000 7f2321ffb700 1 -- 172.21.15.18:0/3719465075 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f231800bb20 con 0x7f2324107760 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.970+0000 7f232b063700 1 -- 172.21.15.18:0/3719465075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23240632a0 con 0x7f2324107760 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:53.972+0000 7f2321ffb700 1 -- 172.21.15.18:0/3719465075 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f231802a060 con 0x7f2324107760 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:54.042+0000 7f232b063700 1 -- 172.21.15.18:0/3719465075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f232410baa0 con 0x7f2324107760 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:54.043+0000 7f2321ffb700 1 -- 172.21.15.18:0/3719465075 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f2318019d60 con 0x7f2324107760 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:54.044+0000 7f232b063700 1 -- 172.21.15.18:0/3719465075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 msgr2=0x7f232419b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:54.044+0000 7f232b063700 1 --2- 172.21.15.18:0/3719465075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 0x7f232419b480 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2318011780 tx=0x7f2318011b40 comp rx=0 tx=0).stop 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:54.044+0000 7f232b063700 1 -- 172.21.15.18:0/3719465075 shutdown_connections 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:54.044+0000 7f232b063700 1 --2- 172.21.15.18:0/3719465075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2324107760 0x7f232419b480 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:54.044+0000 7f232b063700 1 -- 172.21.15.18:0/3719465075 >> 172.21.15.18:0/3719465075 conn(0x7f2324102de0 msgr2=0x7f2324103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:54.044+0000 7f232b063700 1 -- 172.21.15.18:0/3719465075 shutdown_connections 2024-02-17T12:39:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:54.044+0000 7f232b063700 1 -- 172.21.15.18:0/3719465075 wait complete. 2024-02-17T12:39:54.590 INFO:teuthology.orchestra.run.smithi018.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-17T12:39:54.603 INFO:teuthology.orchestra.run.smithi018.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-17T12:39:54.603 INFO:teuthology.orchestra.run.smithi018.stdout:Creating mgr... 2024-02-17T12:39:54.603 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-17T12:39:54.603 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-17T12:39:54.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:54 smithi018 ceph-mon[32625]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-17T12:39:54.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:54 smithi018 ceph-mon[32625]: monmap e1: 1 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T12:39:54.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:54 smithi018 ceph-mon[32625]: fsmap 2024-02-17T12:39:54.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:54 smithi018 ceph-mon[32625]: osdmap e1: 0 total, 0 up, 0 in 2024-02-17T12:39:54.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:54 smithi018 ceph-mon[32625]: mgrmap e1: no daemons active 2024-02-17T12:39:54.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:54 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3719465075' entity='client.admin' 2024-02-17T12:39:54.829 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.x 2024-02-17T12:39:54.829 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.x.service: Unit ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.x.service not loaded. 2024-02-17T12:39:54.974 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7fca71fe-cd91-11ee-95ba-87774f69a715.target.wants/ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@.service. 2024-02-17T12:39:55.197 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:54 smithi018 systemd[1]: Starting Ceph mgr.x for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:39:55.454 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-17T12:39:55.455 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-02-17T12:39:55.461 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-17T12:39:55.462 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-02-17T12:39:55.462 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-02-17T12:39:55.462 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to enable service . firewalld.service is not available 2024-02-17T12:39:55.470 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-17T12:39:55.470 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-02-17T12:39:55.477 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-17T12:39:55.477 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-02-17T12:39:55.477 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-02-17T12:39:55.477 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-17T12:39:55.477 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr to start... 2024-02-17T12:39:55.477 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr... 2024-02-17T12:39:55.529 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:55 smithi018 podman[32837]: 2024-02-17 12:39:55.200772219 +0000 UTC m=+0.106921285 container create 186b5dfab411674446b6d0f42f4bd36de0d597c7491e6336e502c65bfadbf521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-17T12:39:55.530 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:55 smithi018 podman[32837]: 2024-02-17 12:39:55.111328167 +0000 UTC m=+0.017477236 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:39:55.530 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:55 smithi018 podman[32837]: 2024-02-17 12:39:55.386355066 +0000 UTC m=+0.292504133 container init 186b5dfab411674446b6d0f42f4bd36de0d597c7491e6336e502c65bfadbf521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T12:39:55.530 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:55 smithi018 podman[32837]: 2024-02-17 12:39:55.388746977 +0000 UTC m=+0.294896041 container start 186b5dfab411674446b6d0f42f4bd36de0d597c7491e6336e502c65bfadbf521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-17T12:39:55.530 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:55 smithi018 bash[32837]: 186b5dfab411674446b6d0f42f4bd36de0d597c7491e6336e502c65bfadbf521 2024-02-17T12:39:55.530 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:55 smithi018 systemd[1]: Started Ceph mgr.x for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:39:56.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:56 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1428203446' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-17T12:39:56.279 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:55 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:55.953+0000 7f6e8616a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T12:39:56.279 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:56 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:56.057+0000 7f6e8616a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T12:39:56.583 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:39:56.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "a" 2024-02-17T12:39:56.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-02-17T12:39:56.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-17T12:39:56.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-17T12:39:56.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:39:56.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-17T12:39:56.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-17T12:39:56.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:56.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-17T12:39:56.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:39:56.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-17T12:39:56.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-17T12:39:56.587 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-17T12:39:56.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-17T12:39:56.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-17T12:39:56.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-17T12:39:56.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:56.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-17T12:39:56.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:56.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-02-17T12:39:48.883321+0000", 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:56.597 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.907+0000 7f4d42f4a700 1 Processor -- start 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.907+0000 7f4d42f4a700 1 -- start start 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.907+0000 7f4d42f4a700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 0x7f4d3c107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.907+0000 7f4d42f4a700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c108070 con 0x7f4d3c107680 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.907+0000 7f4d40ce6700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 0x7f4d3c107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.907+0000 7f4d40ce6700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 0x7f4d3c107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43180/0 (socket says 172.21.15.18:43180) 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.907+0000 7f4d40ce6700 1 -- 172.21.15.18:0/1697211439 learned_addr learned my addr 172.21.15.18:0/1697211439 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.908+0000 7f4d40ce6700 1 -- 172.21.15.18:0/1697211439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c108830 con 0x7f4d3c107680 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.908+0000 7f4d40ce6700 1 --2- 172.21.15.18:0/1697211439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 0x7f4d3c107aa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4d300098a0 tx=0x7f4d30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fa713c825a1cf7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.908+0000 7f4d3b7fe700 1 -- 172.21.15.18:0/1697211439 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d30004160 con 0x7f4d3c107680 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.908+0000 7f4d3b7fe700 1 -- 172.21.15.18:0/1697211439 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d300042c0 con 0x7f4d3c107680 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.908+0000 7f4d3b7fe700 1 -- 172.21.15.18:0/1697211439 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d30011ac0 con 0x7f4d3c107680 2024-02-17T12:39:56.598 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.908+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1697211439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 msgr2=0x7f4d3c107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.908+0000 7f4d42f4a700 1 --2- 172.21.15.18:0/1697211439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 0x7f4d3c107aa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4d300098a0 tx=0x7f4d30009bb0 comp rx=0 tx=0).stop 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.909+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1697211439 shutdown_connections 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.909+0000 7f4d42f4a700 1 --2- 172.21.15.18:0/1697211439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 0x7f4d3c107aa0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.909+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1697211439 >> 172.21.15.18:0/1697211439 conn(0x7f4d3c102c80 msgr2=0x7f4d3c1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.909+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1697211439 shutdown_connections 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.909+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1697211439 wait complete. 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.909+0000 7f4d42f4a700 1 Processor -- start 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.909+0000 7f4d42f4a700 1 -- start start 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.909+0000 7f4d42f4a700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 0x7f4d3c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.910+0000 7f4d42f4a700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c19b930 con 0x7f4d3c107680 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.910+0000 7f4d40ce6700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 0x7f4d3c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.910+0000 7f4d40ce6700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 0x7f4d3c19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43192/0 (socket says 172.21.15.18:43192) 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.910+0000 7f4d40ce6700 1 -- 172.21.15.18:0/1428203446 learned_addr learned my addr 172.21.15.18:0/1428203446 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.910+0000 7f4d40ce6700 1 -- 172.21.15.18:0/1428203446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d30009550 con 0x7f4d3c107680 2024-02-17T12:39:56.599 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.910+0000 7f4d40ce6700 1 --2- 172.21.15.18:0/1428203446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 0x7f4d3c19b3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4d30000c00 tx=0x7f4d30011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.910+0000 7f4d39ffb700 1 -- 172.21.15.18:0/1428203446 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d30003aa0 con 0x7f4d3c107680 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.910+0000 7f4d39ffb700 1 -- 172.21.15.18:0/1428203446 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d30003c00 con 0x7f4d3c107680 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.910+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1428203446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c19bb30 con 0x7f4d3c107680 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.910+0000 7f4d39ffb700 1 -- 172.21.15.18:0/1428203446 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d300248a0 con 0x7f4d3c107680 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.910+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1428203446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c19bfd0 con 0x7f4d3c107680 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.911+0000 7f4d39ffb700 1 -- 172.21.15.18:0/1428203446 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4d3002d3f0 con 0x7f4d3c107680 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.911+0000 7f4d39ffb700 1 -- 172.21.15.18:0/1428203446 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4d30019780 con 0x7f4d3c107680 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.911+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1428203446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3c10ba60 con 0x7f4d3c107680 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.914+0000 7f4d39ffb700 1 -- 172.21.15.18:0/1428203446 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4d3001ac60 con 0x7f4d3c107680 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.987+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1428203446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4d3c10bc70 con 0x7f4d3c107680 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.988+0000 7f4d39ffb700 1 -- 172.21.15.18:0/1428203446 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f4d30035090 con 0x7f4d3c107680 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.989+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1428203446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 msgr2=0x7f4d3c19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:39:56.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.989+0000 7f4d42f4a700 1 --2- 172.21.15.18:0/1428203446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 0x7f4d3c19b3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4d30000c00 tx=0x7f4d30011b40 comp rx=0 tx=0).stop 2024-02-17T12:39:56.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.989+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1428203446 shutdown_connections 2024-02-17T12:39:56.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.989+0000 7f4d42f4a700 1 --2- 172.21.15.18:0/1428203446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d3c107680 0x7f4d3c19b3f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:39:56.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.989+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1428203446 >> 172.21.15.18:0/1428203446 conn(0x7f4d3c102c80 msgr2=0x7f4d3c103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:39:56.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.989+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1428203446 shutdown_connections 2024-02-17T12:39:56.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:55.989+0000 7f4d42f4a700 1 -- 172.21.15.18:0/1428203446 wait complete. 2024-02-17T12:39:56.601 INFO:teuthology.orchestra.run.smithi018.stdout:mgr not available, waiting (1/15)... 2024-02-17T12:39:56.955 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:56 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:56.722+0000 7f6e8616a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T12:39:56.955 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:56 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:56.816+0000 7f6e8616a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T12:39:57.279 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:56 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:56.958+0000 7f6e8616a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T12:39:57.279 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:57 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:57.055+0000 7f6e8616a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T12:39:57.279 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:57 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:57.140+0000 7f6e8616a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T12:39:57.697 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:57 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:57.405+0000 7f6e8616a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T12:39:57.697 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:57 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:57.517+0000 7f6e8616a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T12:39:57.697 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:57 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:57.604+0000 7f6e8616a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T12:39:57.698 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:57 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:57.700+0000 7f6e8616a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T12:39:58.029 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:57 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:57.877+0000 7f6e8616a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T12:39:59.123 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:39:59 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/130792441' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-17T12:39:59.124 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:58 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:58.879+0000 7f6e8616a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T12:39:59.124 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:59 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:59.017+0000 7f6e8616a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T12:39:59.633 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:39:59.633 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-02-17T12:39:59.633 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:39:59.633 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-02-17T12:39:59.633 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-17T12:39:59.633 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-17T12:39:59.633 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-17T12:39:59.633 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "a" 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:59.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-17T12:39:59.635 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-17T12:39:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-02-17T12:39:48.883321+0000", 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.969+0000 7fbb1c4c7700 1 Processor -- start 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.969+0000 7fbb1c4c7700 1 -- start start 2024-02-17T12:39:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.969+0000 7fbb1c4c7700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 0x7fbb14107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.969+0000 7fbb1c4c7700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb14108080 con 0x7fbb14107690 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.970+0000 7fbb1a263700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 0x7fbb14107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.970+0000 7fbb1a263700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 0x7fbb14107ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43200/0 (socket says 172.21.15.18:43200) 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.970+0000 7fbb1a263700 1 -- 172.21.15.18:0/3842735195 learned_addr learned my addr 172.21.15.18:0/3842735195 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.970+0000 7fbb1a263700 1 -- 172.21.15.18:0/3842735195 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb14108840 con 0x7fbb14107690 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.970+0000 7fbb1a263700 1 --2- 172.21.15.18:0/3842735195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 0x7fbb14107ab0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb040098a0 tx=0x7fbb04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b66eac975e64eb0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.970+0000 7fbb19261700 1 -- 172.21.15.18:0/3842735195 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb04004160 con 0x7fbb14107690 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.970+0000 7fbb19261700 1 -- 172.21.15.18:0/3842735195 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb040042c0 con 0x7fbb14107690 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.970+0000 7fbb19261700 1 -- 172.21.15.18:0/3842735195 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb04011ac0 con 0x7fbb14107690 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.971+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/3842735195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 msgr2=0x7fbb14107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.971+0000 7fbb1c4c7700 1 --2- 172.21.15.18:0/3842735195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 0x7fbb14107ab0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb040098a0 tx=0x7fbb04009bb0 comp rx=0 tx=0).stop 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.971+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/3842735195 shutdown_connections 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.971+0000 7fbb1c4c7700 1 --2- 172.21.15.18:0/3842735195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 0x7fbb14107ab0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:39:59.638 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.971+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/3842735195 >> 172.21.15.18:0/3842735195 conn(0x7fbb14102cb0 msgr2=0x7fbb141050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.971+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/3842735195 shutdown_connections 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.971+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/3842735195 wait complete. 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.971+0000 7fbb1c4c7700 1 Processor -- start 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.972+0000 7fbb1c4c7700 1 -- start start 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.972+0000 7fbb1c4c7700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 0x7fbb1419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.972+0000 7fbb1c4c7700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1419b960 con 0x7fbb14107690 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.972+0000 7fbb1a263700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 0x7fbb1419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.972+0000 7fbb1a263700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 0x7fbb1419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43216/0 (socket says 172.21.15.18:43216) 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.972+0000 7fbb1a263700 1 -- 172.21.15.18:0/130792441 learned_addr learned my addr 172.21.15.18:0/130792441 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.972+0000 7fbb1a263700 1 -- 172.21.15.18:0/130792441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb04009550 con 0x7fbb14107690 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.972+0000 7fbb1a263700 1 --2- 172.21.15.18:0/130792441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 0x7fbb1419b420 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbb04003f70 tx=0x7fbb04011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.972+0000 7fbb0b7fe700 1 -- 172.21.15.18:0/130792441 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb040039f0 con 0x7fbb14107690 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.972+0000 7fbb0b7fe700 1 -- 172.21.15.18:0/130792441 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb04003b50 con 0x7fbb14107690 2024-02-17T12:39:59.639 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.972+0000 7fbb0b7fe700 1 -- 172.21.15.18:0/130792441 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb0401a690 con 0x7fbb14107690 2024-02-17T12:39:59.640 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.972+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/130792441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1419bb60 con 0x7fbb14107690 2024-02-17T12:39:59.640 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.973+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/130792441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1419c000 con 0x7fbb14107690 2024-02-17T12:39:59.640 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.973+0000 7fbb0b7fe700 1 -- 172.21.15.18:0/130792441 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbb0401ac90 con 0x7fbb14107690 2024-02-17T12:39:59.640 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.973+0000 7fbb0b7fe700 1 -- 172.21.15.18:0/130792441 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbb040199b0 con 0x7fbb14107690 2024-02-17T12:39:59.640 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.973+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/130792441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1410ba70 con 0x7fbb14107690 2024-02-17T12:39:59.640 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:58.976+0000 7fbb0b7fe700 1 -- 172.21.15.18:0/130792441 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbb040249f0 con 0x7fbb14107690 2024-02-17T12:39:59.640 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:59.049+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/130792441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbb1410bc80 con 0x7fbb14107690 2024-02-17T12:39:59.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:59.050+0000 7fbb0b7fe700 1 -- 172.21.15.18:0/130792441 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fbb04019c10 con 0x7fbb14107690 2024-02-17T12:39:59.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:59.050+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/130792441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 msgr2=0x7fbb1419b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:39:59.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:59.050+0000 7fbb1c4c7700 1 --2- 172.21.15.18:0/130792441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 0x7fbb1419b420 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbb04003f70 tx=0x7fbb04011b40 comp rx=0 tx=0).stop 2024-02-17T12:39:59.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:59.051+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/130792441 shutdown_connections 2024-02-17T12:39:59.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:59.051+0000 7fbb1c4c7700 1 --2- 172.21.15.18:0/130792441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb14107690 0x7fbb1419b420 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:39:59.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:59.051+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/130792441 >> 172.21.15.18:0/130792441 conn(0x7fbb14102cb0 msgr2=0x7fbb14103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:39:59.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:59.051+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/130792441 shutdown_connections 2024-02-17T12:39:59.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:39:59.051+0000 7fbb1c4c7700 1 -- 172.21.15.18:0/130792441 wait complete. 2024-02-17T12:39:59.641 INFO:teuthology.orchestra.run.smithi018.stdout:mgr not available, waiting (2/15)... 2024-02-17T12:39:59.668 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:59 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:59.379+0000 7f6e8616a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T12:39:59.668 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:59 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:59.471+0000 7f6e8616a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T12:39:59.669 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:59 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:59.671+0000 7f6e8616a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:40:00.029 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:39:59 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:39:59.862+0000 7f6e8616a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T12:40:00.412 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:00.154+0000 7f6e8616a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:40:00.412 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:00.415+0000 7f6e8616a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T12:40:00.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:00.658+0000 7f6e8616a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T12:40:01.752 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:01 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:01.487+0000 7f6e8616a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T12:40:02.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:02 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/109236703' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-17T12:40:02.408 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:02 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:02.084+0000 7f6e8616a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T12:40:02.408 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:02 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:02.188+0000 7f6e8616a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T12:40:02.408 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:02 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:02.410+0000 7f6e8616a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T12:40:02.723 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:40:02.723 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-02-17T12:40:02.723 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:40:02.723 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-02-17T12:40:02.723 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-17T12:40:02.723 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "a" 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:40:02.724 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-17T12:40:02.725 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-17T12:40:02.725 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:02.725 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-17T12:40:02.725 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:40:02.725 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-17T12:40:02.725 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-17T12:40:02.725 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-17T12:40:02.734 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-17T12:40:02.734 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-17T12:40:02.734 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-17T12:40:02.734 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:02.734 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-17T12:40:02.734 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-17T12:40:02.735 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-17T12:40:02.735 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-17T12:40:02.735 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-17T12:40:02.735 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-17T12:40:02.735 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-17T12:40:02.735 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-17T12:40:02.735 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-17T12:40:02.735 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:02.735 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-17T12:40:02.735 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:40:02.735 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-17T12:40:02.735 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:40:02.736 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-02-17T12:39:48.883321+0000", 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.067+0000 7fc3c8fe8700 1 Processor -- start 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.067+0000 7fc3c8fe8700 1 -- start start 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.067+0000 7fc3c8fe8700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 0x7fc3c4107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.067+0000 7fc3c8fe8700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c4108070 con 0x7fc3c4107680 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.067+0000 7fc3c259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 0x7fc3c4107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.067+0000 7fc3c259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 0x7fc3c4107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40820/0 (socket says 172.21.15.18:40820) 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.067+0000 7fc3c259c700 1 -- 172.21.15.18:0/2382543526 learned_addr learned my addr 172.21.15.18:0/2382543526 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.068+0000 7fc3c259c700 1 -- 172.21.15.18:0/2382543526 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c4108830 con 0x7fc3c4107680 2024-02-17T12:40:02.737 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.068+0000 7fc3c259c700 1 --2- 172.21.15.18:0/2382543526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 0x7fc3c4107aa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40098a0 tx=0x7fc3b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b671ab6ae7bc8d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.068+0000 7fc3c159a700 1 -- 172.21.15.18:0/2382543526 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3b4004160 con 0x7fc3c4107680 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.068+0000 7fc3c159a700 1 -- 172.21.15.18:0/2382543526 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3b40042c0 con 0x7fc3c4107680 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.068+0000 7fc3c159a700 1 -- 172.21.15.18:0/2382543526 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3b4011ac0 con 0x7fc3c4107680 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.069+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/2382543526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 msgr2=0x7fc3c4107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.069+0000 7fc3c8fe8700 1 --2- 172.21.15.18:0/2382543526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 0x7fc3c4107aa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40098a0 tx=0x7fc3b4009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.069+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/2382543526 shutdown_connections 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.069+0000 7fc3c8fe8700 1 --2- 172.21.15.18:0/2382543526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 0x7fc3c4107aa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.069+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/2382543526 >> 172.21.15.18:0/2382543526 conn(0x7fc3c4102c80 msgr2=0x7fc3c41050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.069+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/2382543526 shutdown_connections 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.069+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/2382543526 wait complete. 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.070+0000 7fc3c8fe8700 1 Processor -- start 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.070+0000 7fc3c8fe8700 1 -- start start 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.070+0000 7fc3c8fe8700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 0x7fc3c419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:02.738 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.070+0000 7fc3c8fe8700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c419b930 con 0x7fc3c4107680 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.070+0000 7fc3c259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 0x7fc3c419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.070+0000 7fc3c259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 0x7fc3c419b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40828/0 (socket says 172.21.15.18:40828) 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.070+0000 7fc3c259c700 1 -- 172.21.15.18:0/109236703 learned_addr learned my addr 172.21.15.18:0/109236703 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.070+0000 7fc3c259c700 1 -- 172.21.15.18:0/109236703 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b4009550 con 0x7fc3c4107680 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.070+0000 7fc3c259c700 1 --2- 172.21.15.18:0/109236703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 0x7fc3c419b3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4000c00 tx=0x7fc3b4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.070+0000 7fc3b37fe700 1 -- 172.21.15.18:0/109236703 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3b4003aa0 con 0x7fc3c4107680 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.070+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/109236703 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c419bb30 con 0x7fc3c4107680 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.071+0000 7fc3b37fe700 1 -- 172.21.15.18:0/109236703 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3b4003c00 con 0x7fc3c4107680 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.071+0000 7fc3b37fe700 1 -- 172.21.15.18:0/109236703 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3b40248a0 con 0x7fc3c4107680 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.071+0000 7fc3b37fe700 1 -- 172.21.15.18:0/109236703 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc3b40193c0 con 0x7fc3c4107680 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.071+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/109236703 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c419bfd0 con 0x7fc3c4107680 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.071+0000 7fc3b37fe700 1 -- 172.21.15.18:0/109236703 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc3b4024e40 con 0x7fc3c4107680 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.072+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/109236703 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c410ba60 con 0x7fc3c4107680 2024-02-17T12:40:02.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.074+0000 7fc3b37fe700 1 -- 172.21.15.18:0/109236703 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc3b400b3e0 con 0x7fc3c4107680 2024-02-17T12:40:02.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.149+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/109236703 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc3c410bc70 con 0x7fc3c4107680 2024-02-17T12:40:02.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.150+0000 7fc3b37fe700 1 -- 172.21.15.18:0/109236703 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fc3b401ac60 con 0x7fc3c4107680 2024-02-17T12:40:02.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.151+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/109236703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 msgr2=0x7fc3c419b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:02.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.151+0000 7fc3c8fe8700 1 --2- 172.21.15.18:0/109236703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 0x7fc3c419b3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4000c00 tx=0x7fc3b4011b40 comp rx=0 tx=0).stop 2024-02-17T12:40:02.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.151+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/109236703 shutdown_connections 2024-02-17T12:40:02.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.151+0000 7fc3c8fe8700 1 --2- 172.21.15.18:0/109236703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3c4107680 0x7fc3c419b3f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:02.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.151+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/109236703 >> 172.21.15.18:0/109236703 conn(0x7fc3c4102c80 msgr2=0x7fc3c4103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:02.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.151+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/109236703 shutdown_connections 2024-02-17T12:40:02.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:02.151+0000 7fc3c8fe8700 1 -- 172.21.15.18:0/109236703 wait complete. 2024-02-17T12:40:02.740 INFO:teuthology.orchestra.run.smithi018.stdout:mgr not available, waiting (3/15)... 2024-02-17T12:40:03.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: Activating manager daemon x 2024-02-17T12:40:03.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: mgrmap e2: x(active, starting, since 0.00312838s) 2024-02-17T12:40:03.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: from='mgr.14100 172.21.15.18:0/2268195390' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-17T12:40:03.530 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: from='mgr.14100 172.21.15.18:0/2268195390' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-17T12:40:03.530 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: from='mgr.14100 172.21.15.18:0/2268195390' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-17T12:40:03.530 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: from='mgr.14100 172.21.15.18:0/2268195390' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:40:03.530 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: from='mgr.14100 172.21.15.18:0/2268195390' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-17T12:40:03.530 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: Manager daemon x is now available 2024-02-17T12:40:03.530 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: from='mgr.14100 172.21.15.18:0/2268195390' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-17T12:40:03.530 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: from='mgr.14100 172.21.15.18:0/2268195390' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-17T12:40:03.530 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: from='mgr.14100 172.21.15.18:0/2268195390' entity='mgr.x' 2024-02-17T12:40:03.530 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: from='mgr.14100 172.21.15.18:0/2268195390' entity='mgr.x' 2024-02-17T12:40:03.530 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:03 smithi018 ceph-mon[32625]: from='mgr.14100 172.21.15.18:0/2268195390' entity='mgr.x' 2024-02-17T12:40:04.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:04 smithi018 ceph-mon[32625]: mgrmap e3: x(active, since 1.00702s) 2024-02-17T12:40:06.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:06 smithi018 ceph-mon[32625]: mgrmap e4: x(active, since 3s) 2024-02-17T12:40:08.301 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:08 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3312616164' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-17T12:40:13.728 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:40:13.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-02-17T12:40:13.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:40:13.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-02-17T12:40:13.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-17T12:40:13.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-17T12:40:13.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-17T12:40:13.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:13.739 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "a" 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-17T12:40:13.740 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-17T12:40:13.741 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-17T12:40:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-02-17T12:39:48.883321+0000", 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.782+0000 7f891a1e6700 1 Processor -- start 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.782+0000 7f891a1e6700 1 -- start start 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.782+0000 7f891a1e6700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 0x7f8914107b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:13.743 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.782+0000 7f891a1e6700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8914108120 con 0x7f8914107730 2024-02-17T12:40:13.744 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.782+0000 7f89137fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 0x7f8914107b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:13.744 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.782+0000 7f89137fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 0x7f8914107b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40908/0 (socket says 172.21.15.18:40908) 2024-02-17T12:40:13.744 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.782+0000 7f89137fe700 1 -- 172.21.15.18:0/4030111344 learned_addr learned my addr 172.21.15.18:0/4030111344 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:13.744 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.783+0000 7f89137fe700 1 -- 172.21.15.18:0/4030111344 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89141088e0 con 0x7f8914107730 2024-02-17T12:40:13.744 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.783+0000 7f89137fe700 1 --2- 172.21.15.18:0/4030111344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 0x7f8914107b50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f89040098a0 tx=0x7f8904009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75614551b498c05a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:13.744 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.783+0000 7f89127fc700 1 -- 172.21.15.18:0/4030111344 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8904004160 con 0x7f8914107730 2024-02-17T12:40:13.744 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.783+0000 7f89127fc700 1 -- 172.21.15.18:0/4030111344 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89040042c0 con 0x7f8914107730 2024-02-17T12:40:13.744 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.783+0000 7f89127fc700 1 -- 172.21.15.18:0/4030111344 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8904011ac0 con 0x7f8914107730 2024-02-17T12:40:13.744 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.784+0000 7f891a1e6700 1 -- 172.21.15.18:0/4030111344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 msgr2=0x7f8914107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:13.744 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.784+0000 7f891a1e6700 1 --2- 172.21.15.18:0/4030111344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 0x7f8914107b50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f89040098a0 tx=0x7f8904009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:13.744 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.784+0000 7f891a1e6700 1 -- 172.21.15.18:0/4030111344 shutdown_connections 2024-02-17T12:40:13.744 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.784+0000 7f891a1e6700 1 --2- 172.21.15.18:0/4030111344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 0x7f8914107b50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.784+0000 7f891a1e6700 1 -- 172.21.15.18:0/4030111344 >> 172.21.15.18:0/4030111344 conn(0x7f8914102db0 msgr2=0x7f89141051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.784+0000 7f891a1e6700 1 -- 172.21.15.18:0/4030111344 shutdown_connections 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.784+0000 7f891a1e6700 1 -- 172.21.15.18:0/4030111344 wait complete. 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.784+0000 7f891a1e6700 1 Processor -- start 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.784+0000 7f891a1e6700 1 -- start start 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.785+0000 7f891a1e6700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 0x7f891407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.785+0000 7f891a1e6700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891407d0b0 con 0x7f8914107730 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.785+0000 7f89137fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 0x7f891407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.785+0000 7f89137fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 0x7f891407cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40914/0 (socket says 172.21.15.18:40914) 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.785+0000 7f89137fe700 1 -- 172.21.15.18:0/3312616164 learned_addr learned my addr 172.21.15.18:0/3312616164 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.785+0000 7f89137fe700 1 -- 172.21.15.18:0/3312616164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8904009550 con 0x7f8914107730 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.785+0000 7f89137fe700 1 --2- 172.21.15.18:0/3312616164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 0x7f891407cb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8904011780 tx=0x7f8904011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.785+0000 7f8910ff9700 1 -- 172.21.15.18:0/3312616164 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8904003af0 con 0x7f8914107730 2024-02-17T12:40:13.745 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.785+0000 7f8910ff9700 1 -- 172.21.15.18:0/3312616164 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8904003c50 con 0x7f8914107730 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.785+0000 7f891a1e6700 1 -- 172.21.15.18:0/3312616164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891407b2a0 con 0x7f8914107730 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.785+0000 7f8910ff9700 1 -- 172.21.15.18:0/3312616164 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89040197e0 con 0x7f8914107730 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.785+0000 7f891a1e6700 1 -- 172.21.15.18:0/3312616164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891407b740 con 0x7f8914107730 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.786+0000 7f8910ff9700 1 -- 172.21.15.18:0/3312616164 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f8904019a20 con 0x7f8914107730 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.786+0000 7f891a1e6700 1 -- 172.21.15.18:0/3312616164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8914068d50 con 0x7f8914107730 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.786+0000 7f8910ff9700 1 --2- 172.21.15.18:0/3312616164 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f88fc038e90 0x7f88fc03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.786+0000 7f8910ff9700 1 -- 172.21.15.18:0/3312616164 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f890404d3e0 con 0x7f8914107730 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.786+0000 7f8912ffd700 1 --2- 172.21.15.18:0/3312616164 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f88fc038e90 0x7f88fc03b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.787+0000 7f8912ffd700 1 --2- 172.21.15.18:0/3312616164 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f88fc038e90 0x7f88fc03b350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8908006fd0 tx=0x7f8908006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:07.792+0000 7f8910ff9700 1 -- 172.21.15.18:0/3312616164 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f89040188b0 con 0x7f8914107730 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:08.077+0000 7f891a1e6700 1 -- 172.21.15.18:0/3312616164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f891407bd10 con 0x7f8914107730 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:08.078+0000 7f8910ff9700 1 -- 172.21.15.18:0/3312616164 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f89040186d0 con 0x7f8914107730 2024-02-17T12:40:13.746 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:08.080+0000 7f891a1e6700 1 -- 172.21.15.18:0/3312616164 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f88fc038e90 msgr2=0x7f88fc03b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:13.747 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:08.080+0000 7f891a1e6700 1 --2- 172.21.15.18:0/3312616164 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f88fc038e90 0x7f88fc03b350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8908006fd0 tx=0x7f8908006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:13.747 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:08.080+0000 7f891a1e6700 1 -- 172.21.15.18:0/3312616164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 msgr2=0x7f891407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:13.747 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:08.080+0000 7f891a1e6700 1 --2- 172.21.15.18:0/3312616164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 0x7f891407cb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8904011780 tx=0x7f8904011b40 comp rx=0 tx=0).stop 2024-02-17T12:40:13.747 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:08.080+0000 7f891a1e6700 1 -- 172.21.15.18:0/3312616164 shutdown_connections 2024-02-17T12:40:13.747 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:08.080+0000 7f891a1e6700 1 --2- 172.21.15.18:0/3312616164 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f88fc038e90 0x7f88fc03b350 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:13.747 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:08.080+0000 7f891a1e6700 1 --2- 172.21.15.18:0/3312616164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8914107730 0x7f891407cb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:13.747 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:08.080+0000 7f891a1e6700 1 -- 172.21.15.18:0/3312616164 >> 172.21.15.18:0/3312616164 conn(0x7f8914102db0 msgr2=0x7f8914103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:13.747 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:08.080+0000 7f891a1e6700 1 -- 172.21.15.18:0/3312616164 shutdown_connections 2024-02-17T12:40:13.747 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:08.080+0000 7f891a1e6700 1 -- 172.21.15.18:0/3312616164 wait complete. 2024-02-17T12:40:13.747 INFO:teuthology.orchestra.run.smithi018.stdout:mgr is available 2024-02-17T12:40:14.755 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:14 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1439966457' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-17T12:40:15.033 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:40:15.033 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [global] 2024-02-17T12:40:15.033 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout fsid = 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:40:15.033 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-17T12:40:15.033 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-17T12:40:15.033 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-17T12:40:15.033 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-17T12:40:15.033 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [osd] 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.199+0000 7f981d856700 1 Processor -- start 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.199+0000 7f981d856700 1 -- start start 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.200+0000 7f981d856700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 0x7f9818107b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.200+0000 7f981d856700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818108120 con 0x7f9818107730 2024-02-17T12:40:15.034 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.200+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 0x7f9818107b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.200+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 0x7f9818107b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59192/0 (socket says 172.21.15.18:59192) 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.200+0000 7f9816ffd700 1 -- 172.21.15.18:0/1554310384 learned_addr learned my addr 172.21.15.18:0/1554310384 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.200+0000 7f9816ffd700 1 -- 172.21.15.18:0/1554310384 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98181088e0 con 0x7f9818107730 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.201+0000 7f9816ffd700 1 --2- 172.21.15.18:0/1554310384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 0x7f9818107b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98080098a0 tx=0x7f9808009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9237c932d207d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.201+0000 7f9815ffb700 1 -- 172.21.15.18:0/1554310384 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9808004160 con 0x7f9818107730 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.201+0000 7f9815ffb700 1 -- 172.21.15.18:0/1554310384 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98080042c0 con 0x7f9818107730 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.201+0000 7f9815ffb700 1 -- 172.21.15.18:0/1554310384 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9808011ac0 con 0x7f9818107730 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.201+0000 7f981d856700 1 -- 172.21.15.18:0/1554310384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 msgr2=0x7f9818107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.201+0000 7f981d856700 1 --2- 172.21.15.18:0/1554310384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 0x7f9818107b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98080098a0 tx=0x7f9808009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.201+0000 7f981d856700 1 -- 172.21.15.18:0/1554310384 shutdown_connections 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.201+0000 7f981d856700 1 --2- 172.21.15.18:0/1554310384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 0x7f9818107b50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.201+0000 7f981d856700 1 -- 172.21.15.18:0/1554310384 >> 172.21.15.18:0/1554310384 conn(0x7f9818102db0 msgr2=0x7f98181051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:15.035 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.201+0000 7f981d856700 1 -- 172.21.15.18:0/1554310384 shutdown_connections 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.201+0000 7f981d856700 1 -- 172.21.15.18:0/1554310384 wait complete. 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.202+0000 7f981d856700 1 Processor -- start 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.202+0000 7f981d856700 1 -- start start 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.202+0000 7f981d856700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 0x7f981819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.202+0000 7f981d856700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981819b990 con 0x7f9818107730 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.202+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 0x7f981819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.202+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 0x7f981819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59198/0 (socket says 172.21.15.18:59198) 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.202+0000 7f9816ffd700 1 -- 172.21.15.18:0/1439966457 learned_addr learned my addr 172.21.15.18:0/1439966457 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.203+0000 7f9816ffd700 1 -- 172.21.15.18:0/1439966457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9808009550 con 0x7f9818107730 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.203+0000 7f9816ffd700 1 --2- 172.21.15.18:0/1439966457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 0x7f981819b450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9808011780 tx=0x7f9808011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.203+0000 7f981c854700 1 -- 172.21.15.18:0/1439966457 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9808003af0 con 0x7f9818107730 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.203+0000 7f981d856700 1 -- 172.21.15.18:0/1439966457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981819bb90 con 0x7f9818107730 2024-02-17T12:40:15.036 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.203+0000 7f981c854700 1 -- 172.21.15.18:0/1439966457 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9808003c50 con 0x7f9818107730 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.203+0000 7f981c854700 1 -- 172.21.15.18:0/1439966457 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98080197e0 con 0x7f9818107730 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.203+0000 7f981d856700 1 -- 172.21.15.18:0/1439966457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981819c030 con 0x7f9818107730 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.204+0000 7f981c854700 1 -- 172.21.15.18:0/1439966457 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f9808019a20 con 0x7f9818107730 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.204+0000 7f981c854700 1 --2- 172.21.15.18:0/1439966457 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f9804038e90 0x7f980403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.204+0000 7f981d856700 1 -- 172.21.15.18:0/1439966457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981810bb10 con 0x7f9818107730 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.204+0000 7f981c854700 1 -- 172.21.15.18:0/1439966457 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f980804d3f0 con 0x7f9818107730 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.204+0000 7f98167fc700 1 --2- 172.21.15.18:0/1439966457 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f9804038e90 0x7f980403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.205+0000 7f98167fc700 1 --2- 172.21.15.18:0/1439966457 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f9804038e90 0x7f980403b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9800006fd0 tx=0x7f9800006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.209+0000 7f981c854700 1 -- 172.21.15.18:0/1439966457 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9808018e00 con 0x7f9818107730 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.411+0000 7f981d856700 1 -- 172.21.15.18:0/1439966457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f981819e970 con 0x7f9818107730 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.412+0000 7f981c854700 1 -- 172.21.15.18:0/1439966457 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f9808034020 con 0x7f9818107730 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.414+0000 7f981d856700 1 -- 172.21.15.18:0/1439966457 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f9804038e90 msgr2=0x7f980403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:15.037 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.414+0000 7f981d856700 1 --2- 172.21.15.18:0/1439966457 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f9804038e90 0x7f980403b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9800006fd0 tx=0x7f9800006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:15.038 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.414+0000 7f981d856700 1 -- 172.21.15.18:0/1439966457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 msgr2=0x7f981819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:15.038 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.414+0000 7f981d856700 1 --2- 172.21.15.18:0/1439966457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 0x7f981819b450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9808011780 tx=0x7f9808011b40 comp rx=0 tx=0).stop 2024-02-17T12:40:15.038 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.414+0000 7f981d856700 1 -- 172.21.15.18:0/1439966457 shutdown_connections 2024-02-17T12:40:15.038 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.414+0000 7f981d856700 1 --2- 172.21.15.18:0/1439966457 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f9804038e90 0x7f980403b350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:15.038 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.415+0000 7f981d856700 1 --2- 172.21.15.18:0/1439966457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9818107730 0x7f981819b450 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:15.038 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.415+0000 7f981d856700 1 -- 172.21.15.18:0/1439966457 >> 172.21.15.18:0/1439966457 conn(0x7f9818102db0 msgr2=0x7f9818103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:15.038 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.415+0000 7f981d856700 1 -- 172.21.15.18:0/1439966457 shutdown_connections 2024-02-17T12:40:15.038 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:14.415+0000 7f981d856700 1 -- 172.21.15.18:0/1439966457 wait complete. 2024-02-17T12:40:15.038 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling cephadm module... 2024-02-17T12:40:16.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:15 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3171159499' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-17T12:40:16.029 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:15 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: ignoring --setuser ceph since I am not root 2024-02-17T12:40:16.029 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:15 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: ignoring --setgroup ceph since I am not root 2024-02-17T12:40:16.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.487+0000 7f94e906b700 1 Processor -- start 2024-02-17T12:40:16.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.487+0000 7f94e906b700 1 -- start start 2024-02-17T12:40:16.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.487+0000 7f94e906b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 0x7f94e4107ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:16.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.487+0000 7f94e906b700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e4108090 con 0x7f94e41076a0 2024-02-17T12:40:16.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.487+0000 7f94e2d9d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 0x7f94e4107ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:16.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.487+0000 7f94e2d9d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 0x7f94e4107ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59206/0 (socket says 172.21.15.18:59206) 2024-02-17T12:40:16.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.487+0000 7f94e2d9d700 1 -- 172.21.15.18:0/3021001197 learned_addr learned my addr 172.21.15.18:0/3021001197 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:16.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.488+0000 7f94e2d9d700 1 -- 172.21.15.18:0/3021001197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e41088a0 con 0x7f94e41076a0 2024-02-17T12:40:16.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.488+0000 7f94e2d9d700 1 --2- 172.21.15.18:0/3021001197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 0x7f94e4107ac0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0098a0 tx=0x7f94dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaa265f86d4c2c4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:16.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.488+0000 7f94e1d9b700 1 -- 172.21.15.18:0/3021001197 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94dc004160 con 0x7f94e41076a0 2024-02-17T12:40:16.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.488+0000 7f94e1d9b700 1 -- 172.21.15.18:0/3021001197 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94dc0042c0 con 0x7f94e41076a0 2024-02-17T12:40:16.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.488+0000 7f94e1d9b700 1 -- 172.21.15.18:0/3021001197 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94dc011ac0 con 0x7f94e41076a0 2024-02-17T12:40:16.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.488+0000 7f94e906b700 1 -- 172.21.15.18:0/3021001197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 msgr2=0x7f94e4107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.488+0000 7f94e906b700 1 --2- 172.21.15.18:0/3021001197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 0x7f94e4107ac0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0098a0 tx=0x7f94dc009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.489+0000 7f94e906b700 1 -- 172.21.15.18:0/3021001197 shutdown_connections 2024-02-17T12:40:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.489+0000 7f94e906b700 1 --2- 172.21.15.18:0/3021001197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 0x7f94e4107ac0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.489+0000 7f94e906b700 1 -- 172.21.15.18:0/3021001197 >> 172.21.15.18:0/3021001197 conn(0x7f94e4102c80 msgr2=0x7f94e41050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.489+0000 7f94e906b700 1 -- 172.21.15.18:0/3021001197 shutdown_connections 2024-02-17T12:40:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.489+0000 7f94e906b700 1 -- 172.21.15.18:0/3021001197 wait complete. 2024-02-17T12:40:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.489+0000 7f94e906b700 1 Processor -- start 2024-02-17T12:40:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.489+0000 7f94e906b700 1 -- start start 2024-02-17T12:40:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.490+0000 7f94e906b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 0x7f94e419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.490+0000 7f94e906b700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e419b9e0 con 0x7f94e41076a0 2024-02-17T12:40:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.490+0000 7f94e2d9d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 0x7f94e419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:16.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.490+0000 7f94e2d9d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 0x7f94e419b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59218/0 (socket says 172.21.15.18:59218) 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.490+0000 7f94e2d9d700 1 -- 172.21.15.18:0/3171159499 learned_addr learned my addr 172.21.15.18:0/3171159499 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.490+0000 7f94e2d9d700 1 -- 172.21.15.18:0/3171159499 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc009550 con 0x7f94e41076a0 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.490+0000 7f94e2d9d700 1 --2- 172.21.15.18:0/3171159499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 0x7f94e419b4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f94dc000c00 tx=0x7f94dc011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.490+0000 7f94dbfff700 1 -- 172.21.15.18:0/3171159499 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94dc003aa0 con 0x7f94e41076a0 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.490+0000 7f94dbfff700 1 -- 172.21.15.18:0/3171159499 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94dc003c00 con 0x7f94e41076a0 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.490+0000 7f94e906b700 1 -- 172.21.15.18:0/3171159499 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e419bbe0 con 0x7f94e41076a0 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.490+0000 7f94dbfff700 1 -- 172.21.15.18:0/3171159499 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94dc0248a0 con 0x7f94e41076a0 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.490+0000 7f94e906b700 1 -- 172.21.15.18:0/3171159499 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e419c080 con 0x7f94e41076a0 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.491+0000 7f94dbfff700 1 -- 172.21.15.18:0/3171159499 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f94dc02d3f0 con 0x7f94e41076a0 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.491+0000 7f94dbfff700 1 --2- 172.21.15.18:0/3171159499 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f94d0038ee0 0x7f94d003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.491+0000 7f94e906b700 1 -- 172.21.15.18:0/3171159499 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e410bad0 con 0x7f94e41076a0 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.491+0000 7f94dbfff700 1 -- 172.21.15.18:0/3171159499 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f94dc04ce30 con 0x7f94e41076a0 2024-02-17T12:40:16.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.492+0000 7f94e259c700 1 --2- 172.21.15.18:0/3171159499 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f94d0038ee0 0x7f94d003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:16.416 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.492+0000 7f94e259c700 1 --2- 172.21.15.18:0/3171159499 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f94d0038ee0 0x7f94d003b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f94cc006fd0 tx=0x7f94cc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:16.416 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.496+0000 7f94dbfff700 1 -- 172.21.15.18:0/3171159499 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94dc010560 con 0x7f94e41076a0 2024-02-17T12:40:16.416 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.756+0000 7f94e906b700 1 -- 172.21.15.18:0/3171159499 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f94e419ed90 con 0x7f94e41076a0 2024-02-17T12:40:16.416 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.807+0000 7f94dbfff700 1 -- 172.21.15.18:0/3171159499 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f94dc00b3e0 con 0x7f94e41076a0 2024-02-17T12:40:16.416 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.808+0000 7f94dbfff700 1 -- 172.21.15.18:0/3171159499 <== mon.0 v2:172.21.15.18:3300/0 8 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f94dc027d10 con 0x7f94e41076a0 2024-02-17T12:40:16.416 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.810+0000 7f94e906b700 1 -- 172.21.15.18:0/3171159499 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f94d0038ee0 msgr2=0x7f94d003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:16.416 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.810+0000 7f94e906b700 1 --2- 172.21.15.18:0/3171159499 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f94d0038ee0 0x7f94d003b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f94cc006fd0 tx=0x7f94cc006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:16.416 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.810+0000 7f94e906b700 1 -- 172.21.15.18:0/3171159499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 msgr2=0x7f94e419b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:16.416 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.810+0000 7f94e906b700 1 --2- 172.21.15.18:0/3171159499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 0x7f94e419b4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f94dc000c00 tx=0x7f94dc011b40 comp rx=0 tx=0).stop 2024-02-17T12:40:16.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.810+0000 7f94e906b700 1 -- 172.21.15.18:0/3171159499 shutdown_connections 2024-02-17T12:40:16.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.810+0000 7f94e906b700 1 --2- 172.21.15.18:0/3171159499 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f94d0038ee0 0x7f94d003b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:16.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.810+0000 7f94e906b700 1 --2- 172.21.15.18:0/3171159499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e41076a0 0x7f94e419b4a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:16.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.810+0000 7f94e906b700 1 -- 172.21.15.18:0/3171159499 >> 172.21.15.18:0/3171159499 conn(0x7f94e4102c80 msgr2=0x7f94e4103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:16.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.810+0000 7f94e906b700 1 -- 172.21.15.18:0/3171159499 shutdown_connections 2024-02-17T12:40:16.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:15.810+0000 7f94e906b700 1 -- 172.21.15.18:0/3171159499 wait complete. 2024-02-17T12:40:16.480 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:16 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:16.383+0000 7fcdc0d90200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T12:40:16.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:16 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:16.482+0000 7fcdc0d90200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T12:40:17.136 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:16 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3171159499' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-17T12:40:17.136 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:16 smithi018 ceph-mon[32625]: mgrmap e5: x(active, since 13s) 2024-02-17T12:40:17.477 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:17.140+0000 7fcdc0d90200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T12:40:17.477 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:17.235+0000 7fcdc0d90200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T12:40:17.477 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:17.382+0000 7fcdc0d90200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T12:40:17.478 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:17.480+0000 7fcdc0d90200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T12:40:17.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:17.568+0000 7fcdc0d90200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T12:40:18.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:17 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1392738525' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-17T12:40:18.138 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:17.843+0000 7fcdc0d90200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T12:40:18.138 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:17.956+0000 7fcdc0d90200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T12:40:18.138 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:18 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:18.043+0000 7fcdc0d90200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T12:40:18.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-02-17T12:40:18.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-17T12:40:18.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-17T12:40:18.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-17T12:40:18.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-17T12:40:18.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-02-17T12:40:18.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.345+0000 7fdd1df67700 1 Processor -- start 2024-02-17T12:40:18.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.345+0000 7fdd1df67700 1 -- start start 2024-02-17T12:40:18.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.345+0000 7fdd1df67700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 0x7fdd18107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:18.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.345+0000 7fdd1df67700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd18108080 con 0x7fdd18107690 2024-02-17T12:40:18.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.345+0000 7fdd177fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 0x7fdd18107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:18.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.345+0000 7fdd177fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 0x7fdd18107ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59236/0 (socket says 172.21.15.18:59236) 2024-02-17T12:40:18.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.345+0000 7fdd177fe700 1 -- 172.21.15.18:0/3303399084 learned_addr learned my addr 172.21.15.18:0/3303399084 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:18.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.345+0000 7fdd177fe700 1 -- 172.21.15.18:0/3303399084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd18108840 con 0x7fdd18107690 2024-02-17T12:40:18.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.346+0000 7fdd177fe700 1 --2- 172.21.15.18:0/3303399084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 0x7fdd18107ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd080098a0 tx=0x7fdd08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781c025f0d510fe8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:18.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.346+0000 7fdd167fc700 1 -- 172.21.15.18:0/3303399084 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd08004160 con 0x7fdd18107690 2024-02-17T12:40:18.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.346+0000 7fdd167fc700 1 -- 172.21.15.18:0/3303399084 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd080042c0 con 0x7fdd18107690 2024-02-17T12:40:18.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.346+0000 7fdd167fc700 1 -- 172.21.15.18:0/3303399084 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd08011ac0 con 0x7fdd18107690 2024-02-17T12:40:18.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.347+0000 7fdd1df67700 1 -- 172.21.15.18:0/3303399084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 msgr2=0x7fdd18107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:18.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.347+0000 7fdd1df67700 1 --2- 172.21.15.18:0/3303399084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 0x7fdd18107ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd080098a0 tx=0x7fdd08009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:18.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.347+0000 7fdd1df67700 1 -- 172.21.15.18:0/3303399084 shutdown_connections 2024-02-17T12:40:18.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.347+0000 7fdd1df67700 1 --2- 172.21.15.18:0/3303399084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 0x7fdd18107ab0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:18.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.347+0000 7fdd1df67700 1 -- 172.21.15.18:0/3303399084 >> 172.21.15.18:0/3303399084 conn(0x7fdd18102cb0 msgr2=0x7fdd181050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:18.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.347+0000 7fdd1df67700 1 -- 172.21.15.18:0/3303399084 shutdown_connections 2024-02-17T12:40:18.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.347+0000 7fdd1df67700 1 -- 172.21.15.18:0/3303399084 wait complete. 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.347+0000 7fdd1df67700 1 Processor -- start 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.348+0000 7fdd1df67700 1 -- start start 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.348+0000 7fdd1df67700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 0x7fdd1819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.348+0000 7fdd1df67700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1819db60 con 0x7fdd18107690 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.348+0000 7fdd177fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 0x7fdd1819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.348+0000 7fdd177fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 0x7fdd1819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59238/0 (socket says 172.21.15.18:59238) 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.348+0000 7fdd177fe700 1 -- 172.21.15.18:0/1392738525 learned_addr learned my addr 172.21.15.18:0/1392738525 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.348+0000 7fdd177fe700 1 -- 172.21.15.18:0/1392738525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd08009550 con 0x7fdd18107690 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.348+0000 7fdd177fe700 1 --2- 172.21.15.18:0/1392738525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 0x7fdd1819d620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd08000c00 tx=0x7fdd08011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.348+0000 7fdd14ff9700 1 -- 172.21.15.18:0/1392738525 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd08003aa0 con 0x7fdd18107690 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.349+0000 7fdd1df67700 1 -- 172.21.15.18:0/1392738525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1819dd60 con 0x7fdd18107690 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.349+0000 7fdd14ff9700 1 -- 172.21.15.18:0/1392738525 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd08003c00 con 0x7fdd18107690 2024-02-17T12:40:18.248 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.349+0000 7fdd14ff9700 1 -- 172.21.15.18:0/1392738525 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd080248a0 con 0x7fdd18107690 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.349+0000 7fdd1df67700 1 -- 172.21.15.18:0/1392738525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1819e200 con 0x7fdd18107690 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.349+0000 7fdd14ff9700 1 -- 172.21.15.18:0/1392738525 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fdd0802d3f0 con 0x7fdd18107690 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.350+0000 7fdd14ff9700 1 --2- 172.21.15.18:0/1392738525 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7fdd00038ee0 0x7fdd0003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.350+0000 7fdd14ff9700 1 -- 172.21.15.18:0/1392738525 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdd0804de20 con 0x7fdd18107690 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.350+0000 7fdd16ffd700 1 -- 172.21.15.18:0/1392738525 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7fdd00038ee0 msgr2=0x7fdd0003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1543388552 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.350+0000 7fdd16ffd700 1 --2- 172.21.15.18:0/1392738525 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7fdd00038ee0 0x7fdd0003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.350+0000 7fdd1df67700 1 -- 172.21.15.18:0/1392738525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1810ba70 con 0x7fdd18107690 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.355+0000 7fdd14ff9700 1 -- 172.21.15.18:0/1392738525 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd08010740 con 0x7fdd18107690 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.551+0000 7fdd16ffd700 1 -- 172.21.15.18:0/1392738525 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7fdd00038ee0 msgr2=0x7fdd0003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1543388552 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.551+0000 7fdd16ffd700 1 --2- 172.21.15.18:0/1392738525 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7fdd00038ee0 0x7fdd0003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.616+0000 7fdd1df67700 1 -- 172.21.15.18:0/1392738525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fdd181a0ca0 con 0x7fdd18107690 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.617+0000 7fdd14ff9700 1 -- 172.21.15.18:0/1392738525 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fdd0800b3e0 con 0x7fdd18107690 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.619+0000 7fdd1df67700 1 -- 172.21.15.18:0/1392738525 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7fdd00038ee0 msgr2=0x7fdd0003b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:40:18.249 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.619+0000 7fdd1df67700 1 --2- 172.21.15.18:0/1392738525 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7fdd00038ee0 0x7fdd0003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:18.250 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.619+0000 7fdd1df67700 1 -- 172.21.15.18:0/1392738525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 msgr2=0x7fdd1819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:18.250 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.619+0000 7fdd1df67700 1 --2- 172.21.15.18:0/1392738525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 0x7fdd1819d620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd08000c00 tx=0x7fdd08011b40 comp rx=0 tx=0).stop 2024-02-17T12:40:18.250 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.619+0000 7fdd1df67700 1 -- 172.21.15.18:0/1392738525 shutdown_connections 2024-02-17T12:40:18.250 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.619+0000 7fdd1df67700 1 --2- 172.21.15.18:0/1392738525 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7fdd00038ee0 0x7fdd0003b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:18.250 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.619+0000 7fdd1df67700 1 --2- 172.21.15.18:0/1392738525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd18107690 0x7fdd1819d620 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:18.250 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.619+0000 7fdd1df67700 1 -- 172.21.15.18:0/1392738525 >> 172.21.15.18:0/1392738525 conn(0x7fdd18102cb0 msgr2=0x7fdd18103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:18.250 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.619+0000 7fdd1df67700 1 -- 172.21.15.18:0/1392738525 shutdown_connections 2024-02-17T12:40:18.250 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:17.619+0000 7fdd1df67700 1 -- 172.21.15.18:0/1392738525 wait complete. 2024-02-17T12:40:18.250 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for the mgr to restart... 2024-02-17T12:40:18.250 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr epoch 5... 2024-02-17T12:40:18.469 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:18 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:18.141+0000 7fcdc0d90200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T12:40:18.470 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:18 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:18.322+0000 7fcdc0d90200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T12:40:19.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:19 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:19.322+0000 7fcdc0d90200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T12:40:19.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:19 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:19.456+0000 7fcdc0d90200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T12:40:20.081 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:19 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:19.793+0000 7fcdc0d90200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T12:40:20.081 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:19 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:19.885+0000 7fcdc0d90200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T12:40:20.081 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:20 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:20.084+0000 7fcdc0d90200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:40:20.528 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:20 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:20.273+0000 7fcdc0d90200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T12:40:20.821 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:20 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:20.563+0000 7fcdc0d90200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:40:20.821 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:20 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:20.824+0000 7fcdc0d90200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T12:40:21.279 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:21 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:21.066+0000 7fcdc0d90200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T12:40:22.029 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:21 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:21.889+0000 7fcdc0d90200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T12:40:22.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:22.470+0000 7fcdc0d90200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T12:40:22.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:22.571+0000 7fcdc0d90200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T12:40:23.279 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:22.792+0000 7fcdc0d90200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T12:40:23.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: Active manager daemon x restarted 2024-02-17T12:40:23.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: Activating manager daemon x 2024-02-17T12:40:23.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: osdmap e2: 0 total, 0 up, 0 in 2024-02-17T12:40:23.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: mgrmap e6: x(active, starting, since 0.00311962s) 2024-02-17T12:40:23.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:40:23.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-17T12:40:23.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-17T12:40:23.280 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-17T12:40:23.280 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-17T12:40:23.280 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: Manager daemon x is now available 2024-02-17T12:40:23.280 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:23.280 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:23.280 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:40:23.280 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:22 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:40:24.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:23 smithi018 ceph-mon[32625]: Found migration_current of "None". Setting to last migration. 2024-02-17T12:40:24.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:23 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:40:24.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:23 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-17T12:40:24.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:23 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-17T12:40:24.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:23 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:24.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:23 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:24.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:23 smithi018 ceph-mon[32625]: mgrmap e7: x(active, since 1.00514s) 2024-02-17T12:40:24.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-02-17T12:40:24.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-17T12:40:24.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-17T12:40:24.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-02-17T12:40:24.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.696+0000 7f1f877ac700 1 Processor -- start 2024-02-17T12:40:24.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.696+0000 7f1f877ac700 1 -- start start 2024-02-17T12:40:24.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.697+0000 7f1f877ac700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 0x7f1f800731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:24.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.697+0000 7f1f877ac700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f800737b0 con 0x7f1f80072dc0 2024-02-17T12:40:24.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.697+0000 7f1f85548700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 0x7f1f800731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:24.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.697+0000 7f1f85548700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 0x7f1f800731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59244/0 (socket says 172.21.15.18:59244) 2024-02-17T12:40:24.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.697+0000 7f1f85548700 1 -- 172.21.15.18:0/3795739424 learned_addr learned my addr 172.21.15.18:0/3795739424 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:24.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.698+0000 7f1f85548700 1 -- 172.21.15.18:0/3795739424 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8007a0d0 con 0x7f1f80072dc0 2024-02-17T12:40:24.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.698+0000 7f1f85548700 1 --2- 172.21.15.18:0/3795739424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 0x7f1f800731e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800aa40 tx=0x7f1f7800ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7da0a81986b0de4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:24.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.699+0000 7f1f7ffff700 1 -- 172.21.15.18:0/3795739424 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f78010e50 con 0x7f1f80072dc0 2024-02-17T12:40:24.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.699+0000 7f1f7ffff700 1 -- 172.21.15.18:0/3795739424 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f780044e0 con 0x7f1f80072dc0 2024-02-17T12:40:24.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.699+0000 7f1f877ac700 1 -- 172.21.15.18:0/3795739424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 msgr2=0x7f1f800731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:24.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.700+0000 7f1f877ac700 1 --2- 172.21.15.18:0/3795739424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 0x7f1f800731e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800aa40 tx=0x7f1f7800ad50 comp rx=0 tx=0).stop 2024-02-17T12:40:24.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.700+0000 7f1f877ac700 1 -- 172.21.15.18:0/3795739424 shutdown_connections 2024-02-17T12:40:24.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.700+0000 7f1f877ac700 1 --2- 172.21.15.18:0/3795739424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 0x7f1f800731e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.700+0000 7f1f877ac700 1 -- 172.21.15.18:0/3795739424 >> 172.21.15.18:0/3795739424 conn(0x7f1f8006e4d0 msgr2=0x7f1f80070910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.701+0000 7f1f877ac700 1 -- 172.21.15.18:0/3795739424 shutdown_connections 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.701+0000 7f1f877ac700 1 -- 172.21.15.18:0/3795739424 wait complete. 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.701+0000 7f1f877ac700 1 Processor -- start 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.702+0000 7f1f877ac700 1 -- start start 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.702+0000 7f1f877ac700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 0x7f1f80087e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.702+0000 7f1f877ac700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f800883c0 con 0x7f1f80072dc0 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.702+0000 7f1f85548700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 0x7f1f80087e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.702+0000 7f1f85548700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 0x7f1f80087e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59254/0 (socket says 172.21.15.18:59254) 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.702+0000 7f1f85548700 1 -- 172.21.15.18:0/2422901218 learned_addr learned my addr 172.21.15.18:0/2422901218 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.703+0000 7f1f85548700 1 -- 172.21.15.18:0/2422901218 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7800a6f0 con 0x7f1f80072dc0 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.703+0000 7f1f85548700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 0x7f1f80087e80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800a5a0 tx=0x7f1f78003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.703+0000 7f1f7e7fc700 1 -- 172.21.15.18:0/2422901218 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f78003c50 con 0x7f1f80072dc0 2024-02-17T12:40:24.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.704+0000 7f1f877ac700 1 -- 172.21.15.18:0/2422901218 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f800885c0 con 0x7f1f80072dc0 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.704+0000 7f1f877ac700 1 -- 172.21.15.18:0/2422901218 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f80088a60 con 0x7f1f80072dc0 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.704+0000 7f1f7e7fc700 1 -- 172.21.15.18:0/2422901218 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f78018db0 con 0x7f1f80072dc0 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.705+0000 7f1f7e7fc700 1 -- 172.21.15.18:0/2422901218 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f780203e0 con 0x7f1f80072dc0 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.706+0000 7f1f7e7fc700 1 -- 172.21.15.18:0/2422901218 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f1f78021480 con 0x7f1f80072dc0 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.706+0000 7f1f7e7fc700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 0x7f1f6803b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.706+0000 7f1f7e7fc700 1 -- 172.21.15.18:0/2422901218 --> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f78018db0 con 0x7f1f68038c60 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.706+0000 7f1f7e7fc700 1 -- 172.21.15.18:0/2422901218 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1f78010070 con 0x7f1f80072dc0 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.707+0000 7f1f84d47700 1 -- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 msgr2=0x7f1f6803b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1543388552 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.707+0000 7f1f84d47700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 0x7f1f6803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.908+0000 7f1f84d47700 1 -- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 msgr2=0x7f1f6803b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1543388552 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:18.908+0000 7f1f84d47700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 0x7f1f6803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:19.309+0000 7f1f84d47700 1 -- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 msgr2=0x7f1f6803b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1543388552 2024-02-17T12:40:24.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:19.309+0000 7f1f84d47700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 0x7f1f6803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:20.110+0000 7f1f84d47700 1 -- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 msgr2=0x7f1f6803b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1543388552 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:20.110+0000 7f1f84d47700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 0x7f1f6803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:21.711+0000 7f1f84d47700 1 -- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 msgr2=0x7f1f6803b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1543388552 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:21.711+0000 7f1f84d47700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 0x7f1f6803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:22.806+0000 7f1f7e7fc700 1 -- 172.21.15.18:0/2422901218 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mgrmap(e 6) v1 ==== 45699+0+0 (secure 0 0 0) 0x7f1f7800ad50 con 0x7f1f80072dc0 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:22.807+0000 7f1f7e7fc700 1 -- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 msgr2=0x7f1f6803b120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:22.807+0000 7f1f7e7fc700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 0x7f1f6803b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.807+0000 7f1f7e7fc700 1 -- 172.21.15.18:0/2422901218 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f1f78027c90 con 0x7f1f80072dc0 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.808+0000 7f1f7e7fc700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1f6803c4d0 0x7f1f6803e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.808+0000 7f1f7e7fc700 1 -- 172.21.15.18:0/2422901218 --> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f78018db0 con 0x7f1f6803c4d0 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.808+0000 7f1f84d47700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1f6803c4d0 0x7f1f6803e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.809+0000 7f1f84d47700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1f6803c4d0 0x7f1f6803e8c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1f70003d90 tx=0x7f1f70007310 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.809+0000 7f1f7e7fc700 1 -- 172.21.15.18:0/2422901218 <== mgr.14120 v2:172.21.15.18:6800/1086757713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f1f78018db0 con 0x7f1f6803c4d0 2024-02-17T12:40:24.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.816+0000 7f1f877ac700 1 -- 172.21.15.18:0/2422901218 --> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f1f6c002630 con 0x7f1f6803c4d0 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.816+0000 7f1f7e7fc700 1 -- 172.21.15.18:0/2422901218 <== mgr.14120 v2:172.21.15.18:6800/1086757713 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f1f6c002630 con 0x7f1f6803c4d0 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.817+0000 7f1f877ac700 1 -- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1f6803c4d0 msgr2=0x7f1f6803e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.817+0000 7f1f877ac700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1f6803c4d0 0x7f1f6803e8c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1f70003d90 tx=0x7f1f70007310 comp rx=0 tx=0).stop 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.817+0000 7f1f877ac700 1 -- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 msgr2=0x7f1f80087e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.817+0000 7f1f877ac700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 0x7f1f80087e80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800a5a0 tx=0x7f1f78003980 comp rx=0 tx=0).stop 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.817+0000 7f1f877ac700 1 -- 172.21.15.18:0/2422901218 shutdown_connections 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.817+0000 7f1f877ac700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f80072dc0 0x7f1f80087e80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.817+0000 7f1f877ac700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1f6803c4d0 0x7f1f6803e8c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.817+0000 7f1f877ac700 1 --2- 172.21.15.18:0/2422901218 >> [v2:172.21.15.18:6800/1543388552,v1:172.21.15.18:6801/1543388552] conn(0x7f1f68038c60 0x7f1f6803b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.817+0000 7f1f877ac700 1 -- 172.21.15.18:0/2422901218 >> 172.21.15.18:0/2422901218 conn(0x7f1f8006e4d0 msgr2=0x7f1f8006ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.817+0000 7f1f877ac700 1 -- 172.21.15.18:0/2422901218 shutdown_connections 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:23.817+0000 7f1f877ac700 1 -- 172.21.15.18:0/2422901218 wait complete. 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:mgr epoch 5 is available 2024-02-17T12:40:24.382 INFO:teuthology.orchestra.run.smithi018.stdout:Setting orchestrator backend to cephadm... 2024-02-17T12:40:24.999 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:24 smithi018 ceph-mon[32625]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-17T12:40:24.999 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:24 smithi018 ceph-mon[32625]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-17T12:40:24.999 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:24 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:40:25.548 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.773+0000 7f3352c6b700 1 Processor -- start 2024-02-17T12:40:25.548 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.773+0000 7f3352c6b700 1 -- start start 2024-02-17T12:40:25.548 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.773+0000 7f3352c6b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 0x7f334c1069c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:25.548 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.773+0000 7f3352c6b700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c106f90 con 0x7f334c1065a0 2024-02-17T12:40:25.548 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.773+0000 7f3350a07700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 0x7f334c1069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:25.548 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.773+0000 7f3350a07700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 0x7f334c1069c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44568/0 (socket says 172.21.15.18:44568) 2024-02-17T12:40:25.548 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.773+0000 7f3350a07700 1 -- 172.21.15.18:0/2028053269 learned_addr learned my addr 172.21.15.18:0/2028053269 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:25.548 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.774+0000 7f3350a07700 1 -- 172.21.15.18:0/2028053269 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c1077a0 con 0x7f334c1065a0 2024-02-17T12:40:25.548 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.774+0000 7f3350a07700 1 --2- 172.21.15.18:0/2028053269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 0x7f334c1069c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33400098a0 tx=0x7f3340009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29bf72ab0939a4e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.774+0000 7f334b7fe700 1 -- 172.21.15.18:0/2028053269 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3340004160 con 0x7f334c1065a0 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.774+0000 7f334b7fe700 1 -- 172.21.15.18:0/2028053269 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33400042c0 con 0x7f334c1065a0 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.774+0000 7f334b7fe700 1 -- 172.21.15.18:0/2028053269 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3340011ac0 con 0x7f334c1065a0 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.774+0000 7f3352c6b700 1 -- 172.21.15.18:0/2028053269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 msgr2=0x7f334c1069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.774+0000 7f3352c6b700 1 --2- 172.21.15.18:0/2028053269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 0x7f334c1069c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33400098a0 tx=0x7f3340009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.774+0000 7f3352c6b700 1 -- 172.21.15.18:0/2028053269 shutdown_connections 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.774+0000 7f3352c6b700 1 --2- 172.21.15.18:0/2028053269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 0x7f334c1069c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.774+0000 7f3352c6b700 1 -- 172.21.15.18:0/2028053269 >> 172.21.15.18:0/2028053269 conn(0x7f334c101ba0 msgr2=0x7f334c103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.775+0000 7f3352c6b700 1 -- 172.21.15.18:0/2028053269 shutdown_connections 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.775+0000 7f3352c6b700 1 -- 172.21.15.18:0/2028053269 wait complete. 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.775+0000 7f3352c6b700 1 Processor -- start 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.775+0000 7f3352c6b700 1 -- start start 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.775+0000 7f3352c6b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 0x7f334c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:25.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.775+0000 7f3352c6b700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c19b9d0 con 0x7f334c1065a0 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.775+0000 7f3350a07700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 0x7f334c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.775+0000 7f3350a07700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 0x7f334c19b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44574/0 (socket says 172.21.15.18:44574) 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.775+0000 7f3350a07700 1 -- 172.21.15.18:0/515878720 learned_addr learned my addr 172.21.15.18:0/515878720 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.776+0000 7f3350a07700 1 -- 172.21.15.18:0/515878720 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3340009550 con 0x7f334c1065a0 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.776+0000 7f3350a07700 1 --2- 172.21.15.18:0/515878720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 0x7f334c19b490 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3340000c00 tx=0x7f3340011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.776+0000 7f3349ffb700 1 -- 172.21.15.18:0/515878720 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3340003aa0 con 0x7f334c1065a0 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.776+0000 7f3352c6b700 1 -- 172.21.15.18:0/515878720 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c19bbd0 con 0x7f334c1065a0 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.776+0000 7f3349ffb700 1 -- 172.21.15.18:0/515878720 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3340003c00 con 0x7f334c1065a0 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.776+0000 7f3349ffb700 1 -- 172.21.15.18:0/515878720 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33400248a0 con 0x7f334c1065a0 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.776+0000 7f3352c6b700 1 -- 172.21.15.18:0/515878720 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c19c070 con 0x7f334c1065a0 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.777+0000 7f3349ffb700 1 -- 172.21.15.18:0/515878720 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f334002d3f0 con 0x7f334c1065a0 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.777+0000 7f3352c6b700 1 -- 172.21.15.18:0/515878720 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c195640 con 0x7f334c1065a0 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.777+0000 7f3349ffb700 1 --2- 172.21.15.18:0/515878720 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f333c038dc0 0x7f333c03b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:25.550 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.777+0000 7f3349ffb700 1 -- 172.21.15.18:0/515878720 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f334004d130 con 0x7f334c1065a0 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.777+0000 7f334bfff700 1 --2- 172.21.15.18:0/515878720 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f333c038dc0 0x7f333c03b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.778+0000 7f334bfff700 1 --2- 172.21.15.18:0/515878720 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f333c038dc0 0x7f333c03b280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3338006fd0 tx=0x7f3338006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.782+0000 7f3349ffb700 1 -- 172.21.15.18:0/515878720 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3340010740 con 0x7f334c1065a0 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:24.860+0000 7f3349ffb700 1 -- 172.21.15.18:0/515878720 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f3340019410 con 0x7f334c1065a0 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.002+0000 7f3352c6b700 1 -- 172.21.15.18:0/515878720 --> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f334c02da30 con 0x7f333c038dc0 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.007+0000 7f3349ffb700 1 -- 172.21.15.18:0/515878720 <== mgr.14120 v2:172.21.15.18:6800/1086757713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f334c02da30 con 0x7f333c038dc0 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.009+0000 7f3352c6b700 1 -- 172.21.15.18:0/515878720 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f333c038dc0 msgr2=0x7f333c03b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.009+0000 7f3352c6b700 1 --2- 172.21.15.18:0/515878720 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f333c038dc0 0x7f333c03b280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3338006fd0 tx=0x7f3338006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.009+0000 7f3352c6b700 1 -- 172.21.15.18:0/515878720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 msgr2=0x7f334c19b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.009+0000 7f3352c6b700 1 --2- 172.21.15.18:0/515878720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 0x7f334c19b490 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3340000c00 tx=0x7f3340011b40 comp rx=0 tx=0).stop 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.009+0000 7f3352c6b700 1 -- 172.21.15.18:0/515878720 shutdown_connections 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.009+0000 7f3352c6b700 1 --2- 172.21.15.18:0/515878720 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f333c038dc0 0x7f333c03b280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.009+0000 7f3352c6b700 1 --2- 172.21.15.18:0/515878720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c1065a0 0x7f334c19b490 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:25.551 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.009+0000 7f3352c6b700 1 -- 172.21.15.18:0/515878720 >> 172.21.15.18:0/515878720 conn(0x7f334c101ba0 msgr2=0x7f334c102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:25.552 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.010+0000 7f3352c6b700 1 -- 172.21.15.18:0/515878720 shutdown_connections 2024-02-17T12:40:25.552 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.010+0000 7f3352c6b700 1 -- 172.21.15.18:0/515878720 wait complete. 2024-02-17T12:40:25.942 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:25 smithi018 ceph-mon[32625]: [17/Feb/2024:12:40:24] ENGINE Bus STARTING 2024-02-17T12:40:25.942 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:25 smithi018 ceph-mon[32625]: [17/Feb/2024:12:40:24] ENGINE Serving on http://172.21.15.18:8765 2024-02-17T12:40:25.942 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:25 smithi018 ceph-mon[32625]: [17/Feb/2024:12:40:24] ENGINE Serving on https://172.21.15.18:7150 2024-02-17T12:40:25.942 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:25 smithi018 ceph-mon[32625]: [17/Feb/2024:12:40:24] ENGINE Bus STARTED 2024-02-17T12:40:25.942 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:25 smithi018 ceph-mon[32625]: mgrmap e8: x(active, since 2s) 2024-02-17T12:40:25.943 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:25 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:25.943 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:25 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:40:26.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-17T12:40:26.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.979+0000 7f02b09cb700 1 Processor -- start 2024-02-17T12:40:26.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.979+0000 7f02b09cb700 1 -- start start 2024-02-17T12:40:26.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.979+0000 7f02b09cb700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 0x7f02ac107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:26.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.979+0000 7f02b09cb700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac1080c0 con 0x7f02ac1076d0 2024-02-17T12:40:26.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.979+0000 7f02aa59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 0x7f02ac107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:26.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.979+0000 7f02aa59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 0x7f02ac107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44586/0 (socket says 172.21.15.18:44586) 2024-02-17T12:40:26.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.979+0000 7f02aa59c700 1 -- 172.21.15.18:0/3197098869 learned_addr learned my addr 172.21.15.18:0/3197098869 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:26.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.979+0000 7f02aa59c700 1 -- 172.21.15.18:0/3197098869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac108880 con 0x7f02ac1076d0 2024-02-17T12:40:26.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.980+0000 7f02aa59c700 1 --2- 172.21.15.18:0/3197098869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 0x7f02ac107af0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f029c0098a0 tx=0x7f029c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b321b40606f35e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:26.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.980+0000 7f02a959a700 1 -- 172.21.15.18:0/3197098869 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f029c004160 con 0x7f02ac1076d0 2024-02-17T12:40:26.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.980+0000 7f02a959a700 1 -- 172.21.15.18:0/3197098869 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f029c0042c0 con 0x7f02ac1076d0 2024-02-17T12:40:26.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.980+0000 7f02a959a700 1 -- 172.21.15.18:0/3197098869 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f029c011ac0 con 0x7f02ac1076d0 2024-02-17T12:40:26.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.980+0000 7f02b09cb700 1 -- 172.21.15.18:0/3197098869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 msgr2=0x7f02ac107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:26.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.980+0000 7f02b09cb700 1 --2- 172.21.15.18:0/3197098869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 0x7f02ac107af0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f029c0098a0 tx=0x7f029c009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:26.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.980+0000 7f02b09cb700 1 -- 172.21.15.18:0/3197098869 shutdown_connections 2024-02-17T12:40:26.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.980+0000 7f02b09cb700 1 --2- 172.21.15.18:0/3197098869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 0x7f02ac107af0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:26.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.980+0000 7f02b09cb700 1 -- 172.21.15.18:0/3197098869 >> 172.21.15.18:0/3197098869 conn(0x7f02ac102d50 msgr2=0x7f02ac105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:26.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.981+0000 7f02b09cb700 1 -- 172.21.15.18:0/3197098869 shutdown_connections 2024-02-17T12:40:26.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.981+0000 7f02b09cb700 1 -- 172.21.15.18:0/3197098869 wait complete. 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.981+0000 7f02b09cb700 1 Processor -- start 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.981+0000 7f02b09cb700 1 -- start start 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.982+0000 7f02b09cb700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 0x7f02ac19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.982+0000 7f02b09cb700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac19b9e0 con 0x7f02ac1076d0 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.982+0000 7f02aa59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 0x7f02ac19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.982+0000 7f02aa59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 0x7f02ac19b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44594/0 (socket says 172.21.15.18:44594) 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.982+0000 7f02aa59c700 1 -- 172.21.15.18:0/3577796878 learned_addr learned my addr 172.21.15.18:0/3577796878 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.982+0000 7f02aa59c700 1 -- 172.21.15.18:0/3577796878 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c009550 con 0x7f02ac1076d0 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.982+0000 7f02aa59c700 1 --2- 172.21.15.18:0/3577796878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 0x7f02ac19b4a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f029c009fb0 tx=0x7f029c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.982+0000 7f029b7fe700 1 -- 172.21.15.18:0/3577796878 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f029c0038b0 con 0x7f02ac1076d0 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.982+0000 7f029b7fe700 1 -- 172.21.15.18:0/3577796878 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f029c003a10 con 0x7f02ac1076d0 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.982+0000 7f02b09cb700 1 -- 172.21.15.18:0/3577796878 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ac19bbe0 con 0x7f02ac1076d0 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.982+0000 7f029b7fe700 1 -- 172.21.15.18:0/3577796878 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f029c0195e0 con 0x7f02ac1076d0 2024-02-17T12:40:26.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.982+0000 7f02b09cb700 1 -- 172.21.15.18:0/3577796878 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac19c080 con 0x7f02ac1076d0 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.983+0000 7f029b7fe700 1 -- 172.21.15.18:0/3577796878 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f029c019900 con 0x7f02ac1076d0 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.983+0000 7f029b7fe700 1 --2- 172.21.15.18:0/3577796878 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f0294038e90 0x7f029403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.983+0000 7f029b7fe700 1 -- 172.21.15.18:0/3577796878 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f029c04d130 con 0x7f02ac1076d0 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.983+0000 7f02a9d9b700 1 --2- 172.21.15.18:0/3577796878 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f0294038e90 0x7f029403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.984+0000 7f02b09cb700 1 -- 172.21.15.18:0/3577796878 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac10bab0 con 0x7f02ac1076d0 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.984+0000 7f02a9d9b700 1 --2- 172.21.15.18:0/3577796878 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f0294038e90 0x7f029403b350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f02a0006fd0 tx=0x7f02a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:25.989+0000 7f029b7fe700 1 -- 172.21.15.18:0/3577796878 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f029c02b340 con 0x7f02ac1076d0 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:26.194+0000 7f02b09cb700 1 -- 172.21.15.18:0/3577796878 --> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f02ac103f80 con 0x7f0294038e90 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:26.195+0000 7f029b7fe700 1 -- 172.21.15.18:0/3577796878 <== mgr.14120 v2:172.21.15.18:6800/1086757713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f02ac103f80 con 0x7f0294038e90 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:26.197+0000 7f02b09cb700 1 -- 172.21.15.18:0/3577796878 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f0294038e90 msgr2=0x7f029403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:26.197+0000 7f02b09cb700 1 --2- 172.21.15.18:0/3577796878 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f0294038e90 0x7f029403b350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f02a0006fd0 tx=0x7f02a0006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:26.197+0000 7f02b09cb700 1 -- 172.21.15.18:0/3577796878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 msgr2=0x7f02ac19b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:26.197+0000 7f02b09cb700 1 --2- 172.21.15.18:0/3577796878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 0x7f02ac19b4a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f029c009fb0 tx=0x7f029c011b20 comp rx=0 tx=0).stop 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:26.197+0000 7f02b09cb700 1 -- 172.21.15.18:0/3577796878 shutdown_connections 2024-02-17T12:40:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:26.197+0000 7f02b09cb700 1 --2- 172.21.15.18:0/3577796878 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f0294038e90 0x7f029403b350 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:26.197+0000 7f02b09cb700 1 --2- 172.21.15.18:0/3577796878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02ac1076d0 0x7f02ac19b4a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:26.197+0000 7f02b09cb700 1 -- 172.21.15.18:0/3577796878 >> 172.21.15.18:0/3577796878 conn(0x7f02ac102d50 msgr2=0x7f02ac103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:26.197+0000 7f02b09cb700 1 -- 172.21.15.18:0/3577796878 shutdown_connections 2024-02-17T12:40:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:26.197+0000 7f02b09cb700 1 -- 172.21.15.18:0/3577796878 wait complete. 2024-02-17T12:40:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:Generating ssh key... 2024-02-17T12:40:27.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:26 smithi018 ceph-mon[32625]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:40:27.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: Generating public/private rsa key pair. 2024-02-17T12:40:27.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: Your identification has been saved in /tmp/tmps109snrt/key. 2024-02-17T12:40:27.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: Your public key has been saved in /tmp/tmps109snrt/key.pub. 2024-02-17T12:40:27.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: The key fingerprint is: 2024-02-17T12:40:27.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: SHA256:203Y+qbvDhNqLIsB9i2omcJOex2yJjOCtkvcqRlpxY4 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:40:27.780 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: The key's randomart image is: 2024-02-17T12:40:27.780 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: +---[RSA 3072]----+ 2024-02-17T12:40:27.780 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: | | 2024-02-17T12:40:27.780 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: | | 2024-02-17T12:40:27.780 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: | | 2024-02-17T12:40:27.780 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: | . o | 2024-02-17T12:40:27.780 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: | oo S o o | 2024-02-17T12:40:27.780 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: |..*=.o . + = | 2024-02-17T12:40:27.780 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: |oE.+B + = = . | 2024-02-17T12:40:27.780 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: |BX=+ = + +. | 2024-02-17T12:40:27.780 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: |OBO . . .*= | 2024-02-17T12:40:27.780 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: +----[SHA256]-----+ 2024-02-17T12:40:28.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-mon[32625]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:40:28.236 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:28.236 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:27 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:28.243 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.227+0000 7f63f7c94700 1 Processor -- start 2024-02-17T12:40:28.244 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.227+0000 7f63f7c94700 1 -- start start 2024-02-17T12:40:28.244 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.227+0000 7f63f7c94700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 0x7f63f010a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:28.244 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.227+0000 7f63f7c94700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f007aeb0 con 0x7f63f0108120 2024-02-17T12:40:28.244 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.227+0000 7f63f5a30700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 0x7f63f010a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:28.244 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.227+0000 7f63f5a30700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 0x7f63f010a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44596/0 (socket says 172.21.15.18:44596) 2024-02-17T12:40:28.244 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.227+0000 7f63f5a30700 1 -- 172.21.15.18:0/2680994501 learned_addr learned my addr 172.21.15.18:0/2680994501 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:28.244 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.228+0000 7f63f5a30700 1 -- 172.21.15.18:0/2680994501 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f010aa50 con 0x7f63f0108120 2024-02-17T12:40:28.244 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.228+0000 7f63f5a30700 1 --2- 172.21.15.18:0/2680994501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 0x7f63f010a510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0098a0 tx=0x7f63ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7b5a7e24fca6a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:28.244 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.228+0000 7f63f4a2e700 1 -- 172.21.15.18:0/2680994501 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63ec004160 con 0x7f63f0108120 2024-02-17T12:40:28.244 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.228+0000 7f63f4a2e700 1 -- 172.21.15.18:0/2680994501 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63ec0042c0 con 0x7f63f0108120 2024-02-17T12:40:28.244 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.228+0000 7f63f4a2e700 1 -- 172.21.15.18:0/2680994501 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63ec011ac0 con 0x7f63f0108120 2024-02-17T12:40:28.244 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.228+0000 7f63f7c94700 1 -- 172.21.15.18:0/2680994501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 msgr2=0x7f63f010a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.228+0000 7f63f7c94700 1 --2- 172.21.15.18:0/2680994501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 0x7f63f010a510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0098a0 tx=0x7f63ec009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.229+0000 7f63f7c94700 1 -- 172.21.15.18:0/2680994501 shutdown_connections 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.229+0000 7f63f7c94700 1 --2- 172.21.15.18:0/2680994501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 0x7f63f010a510 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.229+0000 7f63f7c94700 1 -- 172.21.15.18:0/2680994501 >> 172.21.15.18:0/2680994501 conn(0x7f63f0101bb0 msgr2=0x7f63f0103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.229+0000 7f63f7c94700 1 -- 172.21.15.18:0/2680994501 shutdown_connections 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.229+0000 7f63f7c94700 1 -- 172.21.15.18:0/2680994501 wait complete. 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.229+0000 7f63f7c94700 1 Processor -- start 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.229+0000 7f63f7c94700 1 -- start start 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.229+0000 7f63f7c94700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 0x7f63f019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.230+0000 7f63f7c94700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f019b9d0 con 0x7f63f0108120 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.230+0000 7f63f5a30700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 0x7f63f019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.230+0000 7f63f5a30700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 0x7f63f019b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44608/0 (socket says 172.21.15.18:44608) 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.230+0000 7f63f5a30700 1 -- 172.21.15.18:0/765032089 learned_addr learned my addr 172.21.15.18:0/765032089 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:28.245 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.230+0000 7f63f5a30700 1 -- 172.21.15.18:0/765032089 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec009550 con 0x7f63f0108120 2024-02-17T12:40:28.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.230+0000 7f63f5a30700 1 --2- 172.21.15.18:0/765032089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 0x7f63f019b490 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f63ec011a70 tx=0x7f63ec011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:28.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.230+0000 7f63e6ffd700 1 -- 172.21.15.18:0/765032089 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63ec0039c0 con 0x7f63f0108120 2024-02-17T12:40:28.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.230+0000 7f63e6ffd700 1 -- 172.21.15.18:0/765032089 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63ec003b20 con 0x7f63f0108120 2024-02-17T12:40:28.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.230+0000 7f63e6ffd700 1 -- 172.21.15.18:0/765032089 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63ec024610 con 0x7f63f0108120 2024-02-17T12:40:28.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.230+0000 7f63f7c94700 1 -- 172.21.15.18:0/765032089 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f019bbd0 con 0x7f63f0108120 2024-02-17T12:40:28.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.230+0000 7f63f7c94700 1 -- 172.21.15.18:0/765032089 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f019c070 con 0x7f63f0108120 2024-02-17T12:40:28.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.231+0000 7f63e6ffd700 1 -- 172.21.15.18:0/765032089 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f63ec02d3f0 con 0x7f63f0108120 2024-02-17T12:40:28.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.231+0000 7f63f7c94700 1 -- 172.21.15.18:0/765032089 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f010b8b0 con 0x7f63f0108120 2024-02-17T12:40:28.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.231+0000 7f63e6ffd700 1 --2- 172.21.15.18:0/765032089 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f63dc038f30 0x7f63dc03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:28.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.231+0000 7f63e6ffd700 1 -- 172.21.15.18:0/765032089 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f63ec04cec0 con 0x7f63f0108120 2024-02-17T12:40:28.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.231+0000 7f63f522f700 1 --2- 172.21.15.18:0/765032089 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f63dc038f30 0x7f63dc03b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:28.246 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.232+0000 7f63f522f700 1 --2- 172.21.15.18:0/765032089 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f63dc038f30 0x7f63dc03b3f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f63e0006fd0 tx=0x7f63e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.236+0000 7f63e6ffd700 1 -- 172.21.15.18:0/765032089 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f63ec018710 con 0x7f63f0108120 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.439+0000 7f63f7c94700 1 -- 172.21.15.18:0/765032089 --> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f63f0102db0 con 0x7f63dc038f30 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.649+0000 7f63e6ffd700 1 -- 172.21.15.18:0/765032089 <== mgr.14120 v2:172.21.15.18:6800/1086757713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f63f0102db0 con 0x7f63dc038f30 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.651+0000 7f63f7c94700 1 -- 172.21.15.18:0/765032089 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f63dc038f30 msgr2=0x7f63dc03b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.651+0000 7f63f7c94700 1 --2- 172.21.15.18:0/765032089 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f63dc038f30 0x7f63dc03b3f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f63e0006fd0 tx=0x7f63e0006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.651+0000 7f63f7c94700 1 -- 172.21.15.18:0/765032089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 msgr2=0x7f63f019b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.651+0000 7f63f7c94700 1 --2- 172.21.15.18:0/765032089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 0x7f63f019b490 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f63ec011a70 tx=0x7f63ec011b50 comp rx=0 tx=0).stop 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.651+0000 7f63f7c94700 1 -- 172.21.15.18:0/765032089 shutdown_connections 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.651+0000 7f63f7c94700 1 --2- 172.21.15.18:0/765032089 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f63dc038f30 0x7f63dc03b3f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.651+0000 7f63f7c94700 1 --2- 172.21.15.18:0/765032089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63f0108120 0x7f63f019b490 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.651+0000 7f63f7c94700 1 -- 172.21.15.18:0/765032089 >> 172.21.15.18:0/765032089 conn(0x7f63f0101bb0 msgr2=0x7f63f01026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.651+0000 7f63f7c94700 1 -- 172.21.15.18:0/765032089 shutdown_connections 2024-02-17T12:40:28.247 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:27.652+0000 7f63f7c94700 1 -- 172.21.15.18:0/765032089 wait complete. 2024-02-17T12:40:29.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:28 smithi018 ceph-mon[32625]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:40:29.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:28 smithi018 ceph-mon[32625]: Generating ssh key... 2024-02-17T12:40:29.367 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6wbvMwhwV/V6fyysXGUhkixH+DgA8ZH4x+AxNE4N3m5tcifNSvoxgqYHIZA/hFe0hRMgetzSNNniPNWfcaqEtJL3pOjufnPTeGjbUP1j2WWkI6gd1U0YnYvCSxfd6Pz3vls6upQ0ImodTn5Uste0J8d0WarWTw0oqIq5RSULiFh8Dnlbe4ejUUDeMFMy91rX1qdnHkvgfoubbm6g4803wyxzb05u9MQzhLHVay7rv2eu6GE2fQXOjAq4gXUa2LViMLvdYg/ZvtqgPZFQbRcy5c33bneTFEImpCbyO4/MEiDAJ+VjK9WaA/AiPbcAxjJC6g/2DaTCxv04MkWUTw52t+Uc5+lvYtthiKY3Y6dCR0aYXZW2+qf1S5V4h/7IbkxtBkAki9c52nMCvPLVeXe13FNvDAVshgAF5yV5ktQCTVY6tjjr9gkZPiDa76ObbpyMr0HNPqIr6zSz6+Q5SYOSoFDMSdwXrPa6l4rV/lVaje2gPcABFp8ttVvZ8OpTmoK8= ceph-7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:40:29.367 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.640+0000 7f4f9c2ed700 1 Processor -- start 2024-02-17T12:40:29.367 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.640+0000 7f4f9c2ed700 1 -- start start 2024-02-17T12:40:29.367 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.640+0000 7f4f9c2ed700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 0x7f4f9410a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:29.368 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.640+0000 7f4f9c2ed700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f94075640 con 0x7f4f941080e0 2024-02-17T12:40:29.368 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.641+0000 7f4f9a089700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 0x7f4f9410a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:29.368 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.641+0000 7f4f9a089700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 0x7f4f9410a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44622/0 (socket says 172.21.15.18:44622) 2024-02-17T12:40:29.368 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.641+0000 7f4f9a089700 1 -- 172.21.15.18:0/908145993 learned_addr learned my addr 172.21.15.18:0/908145993 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:29.368 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.641+0000 7f4f9a089700 1 -- 172.21.15.18:0/908145993 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9410aa10 con 0x7f4f941080e0 2024-02-17T12:40:29.368 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.641+0000 7f4f9a089700 1 --2- 172.21.15.18:0/908145993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 0x7f4f9410a4d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009e80 tx=0x7f4f8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c88ea975aa4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:29.368 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.641+0000 7f4f99087700 1 -- 172.21.15.18:0/908145993 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f8c00e070 con 0x7f4f941080e0 2024-02-17T12:40:29.368 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.641+0000 7f4f99087700 1 -- 172.21.15.18:0/908145993 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f8c00b8c0 con 0x7f4f941080e0 2024-02-17T12:40:29.368 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.641+0000 7f4f99087700 1 -- 172.21.15.18:0/908145993 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f8c004550 con 0x7f4f941080e0 2024-02-17T12:40:29.368 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.642+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/908145993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 msgr2=0x7f4f9410a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:29.368 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.642+0000 7f4f9c2ed700 1 --2- 172.21.15.18:0/908145993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 0x7f4f9410a4d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009e80 tx=0x7f4f8c00b330 comp rx=0 tx=0).stop 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.642+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/908145993 shutdown_connections 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.642+0000 7f4f9c2ed700 1 --2- 172.21.15.18:0/908145993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 0x7f4f9410a4d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.642+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/908145993 >> 172.21.15.18:0/908145993 conn(0x7f4f94101b90 msgr2=0x7f4f94103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.642+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/908145993 shutdown_connections 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.642+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/908145993 wait complete. 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.643+0000 7f4f9c2ed700 1 Processor -- start 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.643+0000 7f4f9c2ed700 1 -- start start 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.643+0000 7f4f9c2ed700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 0x7f4f9419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.643+0000 7f4f9c2ed700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9419b880 con 0x7f4f941080e0 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.643+0000 7f4f9a089700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 0x7f4f9419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.643+0000 7f4f9a089700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 0x7f4f9419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44630/0 (socket says 172.21.15.18:44630) 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.643+0000 7f4f9a089700 1 -- 172.21.15.18:0/2902659456 learned_addr learned my addr 172.21.15.18:0/2902659456 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.643+0000 7f4f9a089700 1 -- 172.21.15.18:0/2902659456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c009b30 con 0x7f4f941080e0 2024-02-17T12:40:29.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.643+0000 7f4f9a089700 1 --2- 172.21.15.18:0/2902659456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 0x7f4f9419b340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0042e0 tx=0x7f4f8c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.643+0000 7f4f937fe700 1 -- 172.21.15.18:0/2902659456 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f8c00e040 con 0x7f4f941080e0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.643+0000 7f4f937fe700 1 -- 172.21.15.18:0/2902659456 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f8c00bd70 con 0x7f4f941080e0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.643+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/2902659456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9419ba80 con 0x7f4f941080e0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.644+0000 7f4f937fe700 1 -- 172.21.15.18:0/2902659456 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f8c01c7f0 con 0x7f4f941080e0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.644+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/2902659456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9419bf00 con 0x7f4f941080e0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.644+0000 7f4f937fe700 1 -- 172.21.15.18:0/2902659456 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f4f8c01ca60 con 0x7f4f941080e0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.645+0000 7f4f937fe700 1 --2- 172.21.15.18:0/2902659456 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f4f7c038ee0 0x7f4f7c03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.645+0000 7f4f937fe700 1 -- 172.21.15.18:0/2902659456 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4f8c04cf80 con 0x7f4f941080e0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.645+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/2902659456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9410b870 con 0x7f4f941080e0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.645+0000 7f4f99888700 1 --2- 172.21.15.18:0/2902659456 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f4f7c038ee0 0x7f4f7c03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.645+0000 7f4f99888700 1 --2- 172.21.15.18:0/2902659456 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f4f7c038ee0 0x7f4f7c03b3a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4f88006fd0 tx=0x7f4f88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.650+0000 7f4f937fe700 1 -- 172.21.15.18:0/2902659456 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f8c017070 con 0x7f4f941080e0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.854+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/2902659456 --> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f94102ce0 con 0x7f4f7c038ee0 2024-02-17T12:40:29.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.855+0000 7f4f937fe700 1 -- 172.21.15.18:0/2902659456 <== mgr.14120 v2:172.21.15.18:6800/1086757713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f4f94102ce0 con 0x7f4f7c038ee0 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.857+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/2902659456 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f4f7c038ee0 msgr2=0x7f4f7c03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.857+0000 7f4f9c2ed700 1 --2- 172.21.15.18:0/2902659456 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f4f7c038ee0 0x7f4f7c03b3a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4f88006fd0 tx=0x7f4f88006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.857+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/2902659456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 msgr2=0x7f4f9419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.857+0000 7f4f9c2ed700 1 --2- 172.21.15.18:0/2902659456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 0x7f4f9419b340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0042e0 tx=0x7f4f8c003d00 comp rx=0 tx=0).stop 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.857+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/2902659456 shutdown_connections 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.857+0000 7f4f9c2ed700 1 --2- 172.21.15.18:0/2902659456 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f4f7c038ee0 0x7f4f7c03b3a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.857+0000 7f4f9c2ed700 1 --2- 172.21.15.18:0/2902659456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f941080e0 0x7f4f9419b340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.857+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/2902659456 >> 172.21.15.18:0/2902659456 conn(0x7f4f94101b90 msgr2=0x7f4f941025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.857+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/2902659456 shutdown_connections 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:28.857+0000 7f4f9c2ed700 1 -- 172.21.15.18:0/2902659456 wait complete. 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:Adding key to root@localhost authorized_keys... 2024-02-17T12:40:29.371 INFO:teuthology.orchestra.run.smithi018.stdout:Adding host smithi018... 2024-02-17T12:40:29.977 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:29 smithi018 ceph-mon[32625]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:40:31.171 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:30 smithi018 ceph-mon[32625]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi018", "addr": "172.21.15.18", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:40:31.171 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:30 smithi018 ceph-mon[32625]: Deploying cephadm binary to smithi018 2024-02-17T12:40:34.974 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Added host 'smithi018' with addr '172.21.15.18' 2024-02-17T12:40:34.974 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.765+0000 7f82dd5cb700 1 Processor -- start 2024-02-17T12:40:34.974 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.765+0000 7f82dd5cb700 1 -- start start 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.765+0000 7f82dd5cb700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 0x7f82d810a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.765+0000 7f82dd5cb700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d807adf0 con 0x7f82d8108280 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.765+0000 7f82d6ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 0x7f82d810a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.765+0000 7f82d6ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 0x7f82d810a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39742/0 (socket says 172.21.15.18:39742) 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.765+0000 7f82d6ffd700 1 -- 172.21.15.18:0/749728993 learned_addr learned my addr 172.21.15.18:0/749728993 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.765+0000 7f82d6ffd700 1 -- 172.21.15.18:0/749728993 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d807af30 con 0x7f82d8108280 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.766+0000 7f82d6ffd700 1 --2- 172.21.15.18:0/749728993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 0x7f82d810a670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f82c80098a0 tx=0x7f82c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe19c97aa596e4b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.766+0000 7f82d5ffb700 1 -- 172.21.15.18:0/749728993 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82c8004160 con 0x7f82d8108280 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.766+0000 7f82d5ffb700 1 -- 172.21.15.18:0/749728993 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82c80042c0 con 0x7f82d8108280 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.766+0000 7f82d5ffb700 1 -- 172.21.15.18:0/749728993 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82c8011ac0 con 0x7f82d8108280 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.766+0000 7f82dd5cb700 1 -- 172.21.15.18:0/749728993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 msgr2=0x7f82d810a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.766+0000 7f82dd5cb700 1 --2- 172.21.15.18:0/749728993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 0x7f82d810a670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f82c80098a0 tx=0x7f82c8009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.766+0000 7f82dd5cb700 1 -- 172.21.15.18:0/749728993 shutdown_connections 2024-02-17T12:40:34.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.766+0000 7f82dd5cb700 1 --2- 172.21.15.18:0/749728993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 0x7f82d810a670 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.766+0000 7f82dd5cb700 1 -- 172.21.15.18:0/749728993 >> 172.21.15.18:0/749728993 conn(0x7f82d8101c90 msgr2=0x7f82d81040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.766+0000 7f82dd5cb700 1 -- 172.21.15.18:0/749728993 shutdown_connections 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.766+0000 7f82dd5cb700 1 -- 172.21.15.18:0/749728993 wait complete. 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.767+0000 7f82dd5cb700 1 Processor -- start 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.767+0000 7f82dd5cb700 1 -- start start 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.767+0000 7f82dd5cb700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 0x7f82d819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.767+0000 7f82dd5cb700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d819dc70 con 0x7f82d8108280 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.767+0000 7f82d6ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 0x7f82d819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.767+0000 7f82d6ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 0x7f82d819d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39744/0 (socket says 172.21.15.18:39744) 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.767+0000 7f82d6ffd700 1 -- 172.21.15.18:0/1629230550 learned_addr learned my addr 172.21.15.18:0/1629230550 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.768+0000 7f82d6ffd700 1 -- 172.21.15.18:0/1629230550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c8009550 con 0x7f82d8108280 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.768+0000 7f82d6ffd700 1 --2- 172.21.15.18:0/1629230550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 0x7f82d819d730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009fb0 tx=0x7f82c8011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.768+0000 7f82bffff700 1 -- 172.21.15.18:0/1629230550 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82c80038b0 con 0x7f82d8108280 2024-02-17T12:40:34.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.768+0000 7f82bffff700 1 -- 172.21.15.18:0/1629230550 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82c8003a10 con 0x7f82d8108280 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.768+0000 7f82bffff700 1 -- 172.21.15.18:0/1629230550 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82c80195e0 con 0x7f82d8108280 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.768+0000 7f82dd5cb700 1 -- 172.21.15.18:0/1629230550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d819de70 con 0x7f82d8108280 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.768+0000 7f82dd5cb700 1 -- 172.21.15.18:0/1629230550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d819e310 con 0x7f82d8108280 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.769+0000 7f82bffff700 1 -- 172.21.15.18:0/1629230550 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f82c8019850 con 0x7f82d8108280 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.769+0000 7f82dd5cb700 1 -- 172.21.15.18:0/1629230550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d810b940 con 0x7f82d8108280 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.769+0000 7f82bffff700 1 --2- 172.21.15.18:0/1629230550 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f82c0038ee0 0x7f82c003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.769+0000 7f82bffff700 1 -- 172.21.15.18:0/1629230550 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f82c804d320 con 0x7f82d8108280 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.769+0000 7f82d67fc700 1 --2- 172.21.15.18:0/1629230550 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f82c0038ee0 0x7f82c003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.770+0000 7f82d67fc700 1 --2- 172.21.15.18:0/1629230550 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f82c0038ee0 0x7f82c003b3a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f82cc006fd0 tx=0x7f82cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.774+0000 7f82bffff700 1 -- 172.21.15.18:0/1629230550 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f82c8018530 con 0x7f82d8108280 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:29.980+0000 7f82dd5cb700 1 -- 172.21.15.18:0/1629230550 --> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi018", "addr": "172.21.15.18", "target": ["mon-mgr", ""]}) v1 -- 0x7f82d8102e60 con 0x7f82c0038ee0 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:34.419+0000 7f82bffff700 1 -- 172.21.15.18:0/1629230550 <== mgr.14120 v2:172.21.15.18:6800/1086757713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f82d8102e60 con 0x7f82c0038ee0 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:34.422+0000 7f82dd5cb700 1 -- 172.21.15.18:0/1629230550 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f82c0038ee0 msgr2=0x7f82c003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:34.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:34.422+0000 7f82dd5cb700 1 --2- 172.21.15.18:0/1629230550 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f82c0038ee0 0x7f82c003b3a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f82cc006fd0 tx=0x7f82cc006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:34.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:34.422+0000 7f82dd5cb700 1 -- 172.21.15.18:0/1629230550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 msgr2=0x7f82d819d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:34.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:34.422+0000 7f82dd5cb700 1 --2- 172.21.15.18:0/1629230550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 0x7f82d819d730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009fb0 tx=0x7f82c8011b20 comp rx=0 tx=0).stop 2024-02-17T12:40:34.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:34.422+0000 7f82dd5cb700 1 -- 172.21.15.18:0/1629230550 shutdown_connections 2024-02-17T12:40:34.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:34.422+0000 7f82dd5cb700 1 --2- 172.21.15.18:0/1629230550 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f82c0038ee0 0x7f82c003b3a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:34.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:34.422+0000 7f82dd5cb700 1 --2- 172.21.15.18:0/1629230550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d8108280 0x7f82d819d730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:34.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:34.422+0000 7f82dd5cb700 1 -- 172.21.15.18:0/1629230550 >> 172.21.15.18:0/1629230550 conn(0x7f82d8101c90 msgr2=0x7f82d8102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:34.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:34.422+0000 7f82dd5cb700 1 -- 172.21.15.18:0/1629230550 shutdown_connections 2024-02-17T12:40:34.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:34.422+0000 7f82dd5cb700 1 -- 172.21.15.18:0/1629230550 wait complete. 2024-02-17T12:40:34.978 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying unmanaged mon service... 2024-02-17T12:40:35.685 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:35 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:35.685 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:35 smithi018 ceph-mon[32625]: Added host smithi018 2024-02-17T12:40:35.685 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:35 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:40:36.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-17T12:40:36.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.459+0000 7fa6af8d6700 1 Processor -- start 2024-02-17T12:40:36.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.459+0000 7fa6af8d6700 1 -- start start 2024-02-17T12:40:36.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.459+0000 7fa6af8d6700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 0x7fa6a8105dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:36.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.459+0000 7fa6af8d6700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a8106390 con 0x7fa6a81059a0 2024-02-17T12:40:36.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.459+0000 7fa6ad672700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 0x7fa6a8105dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:36.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.459+0000 7fa6ad672700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 0x7fa6a8105dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39750/0 (socket says 172.21.15.18:39750) 2024-02-17T12:40:36.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.459+0000 7fa6ad672700 1 -- 172.21.15.18:0/3996573906 learned_addr learned my addr 172.21.15.18:0/3996573906 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:36.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.459+0000 7fa6ad672700 1 -- 172.21.15.18:0/3996573906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a8108930 con 0x7fa6a81059a0 2024-02-17T12:40:36.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.459+0000 7fa6ad672700 1 --2- 172.21.15.18:0/3996573906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 0x7fa6a8105dc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40098a0 tx=0x7fa6a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c187e101796de44a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:36.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.460+0000 7fa69ffff700 1 -- 172.21.15.18:0/3996573906 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6a4004160 con 0x7fa6a81059a0 2024-02-17T12:40:36.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.460+0000 7fa69ffff700 1 -- 172.21.15.18:0/3996573906 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6a40042c0 con 0x7fa6a81059a0 2024-02-17T12:40:36.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.460+0000 7fa69ffff700 1 -- 172.21.15.18:0/3996573906 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6a4011ac0 con 0x7fa6a81059a0 2024-02-17T12:40:36.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.460+0000 7fa6af8d6700 1 -- 172.21.15.18:0/3996573906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 msgr2=0x7fa6a8105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:36.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.460+0000 7fa6af8d6700 1 --2- 172.21.15.18:0/3996573906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 0x7fa6a8105dc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40098a0 tx=0x7fa6a4009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:36.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.460+0000 7fa6af8d6700 1 -- 172.21.15.18:0/3996573906 shutdown_connections 2024-02-17T12:40:36.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.460+0000 7fa6af8d6700 1 --2- 172.21.15.18:0/3996573906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 0x7fa6a8105dc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:36.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.460+0000 7fa6af8d6700 1 -- 172.21.15.18:0/3996573906 >> 172.21.15.18:0/3996573906 conn(0x7fa6a81012d0 msgr2=0x7fa6a8103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:36.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.461+0000 7fa6af8d6700 1 -- 172.21.15.18:0/3996573906 shutdown_connections 2024-02-17T12:40:36.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.461+0000 7fa6af8d6700 1 -- 172.21.15.18:0/3996573906 wait complete. 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.461+0000 7fa6af8d6700 1 Processor -- start 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.461+0000 7fa6af8d6700 1 -- start start 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.461+0000 7fa6af8d6700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 0x7fa6a819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.461+0000 7fa6af8d6700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a819b9e0 con 0x7fa6a81059a0 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.461+0000 7fa6ad672700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 0x7fa6a819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.461+0000 7fa6ad672700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 0x7fa6a819b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39764/0 (socket says 172.21.15.18:39764) 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.461+0000 7fa6ad672700 1 -- 172.21.15.18:0/1049919103 learned_addr learned my addr 172.21.15.18:0/1049919103 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.461+0000 7fa6ad672700 1 -- 172.21.15.18:0/1049919103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a4009550 con 0x7fa6a81059a0 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.462+0000 7fa6ad672700 1 --2- 172.21.15.18:0/1049919103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 0x7fa6a819b4a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4000c00 tx=0x7fa6a4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.462+0000 7fa69e7fc700 1 -- 172.21.15.18:0/1049919103 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6a4003aa0 con 0x7fa6a81059a0 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.462+0000 7fa69e7fc700 1 -- 172.21.15.18:0/1049919103 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6a4003c00 con 0x7fa6a81059a0 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.462+0000 7fa6af8d6700 1 -- 172.21.15.18:0/1049919103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a819bbe0 con 0x7fa6a81059a0 2024-02-17T12:40:36.345 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.462+0000 7fa69e7fc700 1 -- 172.21.15.18:0/1049919103 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6a40248a0 con 0x7fa6a81059a0 2024-02-17T12:40:36.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.462+0000 7fa6af8d6700 1 -- 172.21.15.18:0/1049919103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a819c080 con 0x7fa6a81059a0 2024-02-17T12:40:36.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.463+0000 7fa69e7fc700 1 -- 172.21.15.18:0/1049919103 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fa6a401a430 con 0x7fa6a81059a0 2024-02-17T12:40:36.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.463+0000 7fa6af8d6700 1 -- 172.21.15.18:0/1049919103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a810bb60 con 0x7fa6a81059a0 2024-02-17T12:40:36.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.463+0000 7fa69e7fc700 1 --2- 172.21.15.18:0/1049919103 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7fa694038f30 0x7fa69403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:36.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.463+0000 7fa69e7fc700 1 -- 172.21.15.18:0/1049919103 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa6a404d060 con 0x7fa6a81059a0 2024-02-17T12:40:36.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.463+0000 7fa6ace71700 1 --2- 172.21.15.18:0/1049919103 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7fa694038f30 0x7fa69403b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:36.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.463+0000 7fa6ace71700 1 --2- 172.21.15.18:0/1049919103 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7fa694038f30 0x7fa69403b3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa698006fd0 tx=0x7fa698006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:36.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.468+0000 7fa69e7fc700 1 -- 172.21.15.18:0/1049919103 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa6a4018710 con 0x7fa6a81059a0 2024-02-17T12:40:36.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.688+0000 7fa6af8d6700 1 -- 172.21.15.18:0/1049919103 --> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa6a81024a0 con 0x7fa694038f30 2024-02-17T12:40:36.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.693+0000 7fa69e7fc700 1 -- 172.21.15.18:0/1049919103 <== mgr.14120 v2:172.21.15.18:6800/1086757713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa6a81024a0 con 0x7fa694038f30 2024-02-17T12:40:36.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.695+0000 7fa6af8d6700 1 -- 172.21.15.18:0/1049919103 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7fa694038f30 msgr2=0x7fa69403b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:36.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.695+0000 7fa6af8d6700 1 --2- 172.21.15.18:0/1049919103 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7fa694038f30 0x7fa69403b3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa698006fd0 tx=0x7fa698006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:36.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.695+0000 7fa6af8d6700 1 -- 172.21.15.18:0/1049919103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 msgr2=0x7fa6a819b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:36.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.695+0000 7fa6af8d6700 1 --2- 172.21.15.18:0/1049919103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 0x7fa6a819b4a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4000c00 tx=0x7fa6a4011b40 comp rx=0 tx=0).stop 2024-02-17T12:40:36.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.695+0000 7fa6af8d6700 1 -- 172.21.15.18:0/1049919103 shutdown_connections 2024-02-17T12:40:36.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.695+0000 7fa6af8d6700 1 --2- 172.21.15.18:0/1049919103 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7fa694038f30 0x7fa69403b3f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:36.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.695+0000 7fa6af8d6700 1 --2- 172.21.15.18:0/1049919103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6a81059a0 0x7fa6a819b4a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:36.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.695+0000 7fa6af8d6700 1 -- 172.21.15.18:0/1049919103 >> 172.21.15.18:0/1049919103 conn(0x7fa6a81012d0 msgr2=0x7fa6a8101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:36.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.695+0000 7fa6af8d6700 1 -- 172.21.15.18:0/1049919103 shutdown_connections 2024-02-17T12:40:36.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:35.695+0000 7fa6af8d6700 1 -- 172.21.15.18:0/1049919103 wait complete. 2024-02-17T12:40:36.347 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying unmanaged mgr service... 2024-02-17T12:40:36.919 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:36 smithi018 ceph-mon[32625]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:40:36.920 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:36 smithi018 ceph-mon[32625]: Saving service mon spec with placement count:5 2024-02-17T12:40:36.920 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:36 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:38.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:38 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:38.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:38 smithi018 ceph-mon[32625]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:40:38.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:38 smithi018 ceph-mon[32625]: Saving service mgr spec with placement count:2 2024-02-17T12:40:38.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:38 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:38.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:38 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:38.387 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-17T12:40:38.387 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.491+0000 7efef659d700 1 Processor -- start 2024-02-17T12:40:38.387 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.491+0000 7efef659d700 1 -- start start 2024-02-17T12:40:38.387 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.491+0000 7efef659d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 0x7efef010a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:38.387 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.491+0000 7efef659d700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0075640 con 0x7efef0107f30 2024-02-17T12:40:38.387 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.491+0000 7efef559b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 0x7efef010a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:38.387 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.491+0000 7efef559b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 0x7efef010a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39778/0 (socket says 172.21.15.18:39778) 2024-02-17T12:40:38.387 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.491+0000 7efef559b700 1 -- 172.21.15.18:0/419745579 learned_addr learned my addr 172.21.15.18:0/419745579 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:38.387 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.492+0000 7efef559b700 1 -- 172.21.15.18:0/419745579 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef010a860 con 0x7efef0107f30 2024-02-17T12:40:38.387 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.492+0000 7efef559b700 1 --2- 172.21.15.18:0/419745579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 0x7efef010a320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efeec0098a0 tx=0x7efeec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8420f925110846c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:38.387 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.492+0000 7efee7fff700 1 -- 172.21.15.18:0/419745579 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efeec004160 con 0x7efef0107f30 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.492+0000 7efee7fff700 1 -- 172.21.15.18:0/419745579 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efeec0042c0 con 0x7efef0107f30 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.492+0000 7efee7fff700 1 -- 172.21.15.18:0/419745579 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efeec011a80 con 0x7efef0107f30 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.493+0000 7efef659d700 1 -- 172.21.15.18:0/419745579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 msgr2=0x7efef010a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.493+0000 7efef659d700 1 --2- 172.21.15.18:0/419745579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 0x7efef010a320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efeec0098a0 tx=0x7efeec009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.493+0000 7efef659d700 1 -- 172.21.15.18:0/419745579 shutdown_connections 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.493+0000 7efef659d700 1 --2- 172.21.15.18:0/419745579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 0x7efef010a320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.493+0000 7efef659d700 1 -- 172.21.15.18:0/419745579 >> 172.21.15.18:0/419745579 conn(0x7efef0101980 msgr2=0x7efef0103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.493+0000 7efef659d700 1 -- 172.21.15.18:0/419745579 shutdown_connections 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.493+0000 7efef659d700 1 -- 172.21.15.18:0/419745579 wait complete. 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.493+0000 7efef659d700 1 Processor -- start 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.493+0000 7efef659d700 1 -- start start 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.494+0000 7efef659d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 0x7efef01077b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:38.388 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.494+0000 7efef659d700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0107cf0 con 0x7efef0107f30 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.494+0000 7efef559b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 0x7efef01077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.494+0000 7efef559b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 0x7efef01077b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39788/0 (socket says 172.21.15.18:39788) 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.494+0000 7efef559b700 1 -- 172.21.15.18:0/1368734324 learned_addr learned my addr 172.21.15.18:0/1368734324 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.494+0000 7efef559b700 1 -- 172.21.15.18:0/1368734324 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec009550 con 0x7efef0107f30 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.494+0000 7efef559b700 1 --2- 172.21.15.18:0/1368734324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 0x7efef01077b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efeec003f20 tx=0x7efeec011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.494+0000 7efee67fc700 1 -- 172.21.15.18:0/1368734324 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efeec0039f0 con 0x7efef0107f30 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.494+0000 7efee67fc700 1 -- 172.21.15.18:0/1368734324 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efeec003b50 con 0x7efef0107f30 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.494+0000 7efef659d700 1 -- 172.21.15.18:0/1368734324 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef0105ee0 con 0x7efef0107f30 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.494+0000 7efee67fc700 1 -- 172.21.15.18:0/1368734324 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efeec01a6d0 con 0x7efef0107f30 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.494+0000 7efef659d700 1 -- 172.21.15.18:0/1368734324 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef0106380 con 0x7efef0107f30 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.495+0000 7efee67fc700 1 -- 172.21.15.18:0/1368734324 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7efeec01a940 con 0x7efef0107f30 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.495+0000 7efef659d700 1 -- 172.21.15.18:0/1368734324 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef010b700 con 0x7efef0107f30 2024-02-17T12:40:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.495+0000 7efee67fc700 1 --2- 172.21.15.18:0/1368734324 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7efedc038ee0 0x7efedc03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.495+0000 7efee67fc700 1 -- 172.21.15.18:0/1368734324 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efeec04d2b0 con 0x7efef0107f30 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.496+0000 7efef4d9a700 1 --2- 172.21.15.18:0/1368734324 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7efedc038ee0 0x7efedc03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.496+0000 7efef4d9a700 1 --2- 172.21.15.18:0/1368734324 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7efedc038ee0 0x7efedc03b3a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efee0006fd0 tx=0x7efee0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.500+0000 7efee67fc700 1 -- 172.21.15.18:0/1368734324 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efeec01abf0 con 0x7efef0107f30 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.718+0000 7efef659d700 1 -- 172.21.15.18:0/1368734324 --> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7efef0106990 con 0x7efedc038ee0 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.723+0000 7efee67fc700 1 -- 172.21.15.18:0/1368734324 <== mgr.14120 v2:172.21.15.18:6800/1086757713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efef0106990 con 0x7efedc038ee0 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.725+0000 7efef659d700 1 -- 172.21.15.18:0/1368734324 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7efedc038ee0 msgr2=0x7efedc03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.725+0000 7efef659d700 1 --2- 172.21.15.18:0/1368734324 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7efedc038ee0 0x7efedc03b3a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efee0006fd0 tx=0x7efee0006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.725+0000 7efef659d700 1 -- 172.21.15.18:0/1368734324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 msgr2=0x7efef01077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.725+0000 7efef659d700 1 --2- 172.21.15.18:0/1368734324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 0x7efef01077b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efeec003f20 tx=0x7efeec011b00 comp rx=0 tx=0).stop 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.725+0000 7efef659d700 1 -- 172.21.15.18:0/1368734324 shutdown_connections 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.725+0000 7efef659d700 1 --2- 172.21.15.18:0/1368734324 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7efedc038ee0 0x7efedc03b3a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.725+0000 7efef659d700 1 --2- 172.21.15.18:0/1368734324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efef0107f30 0x7efef01077b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:38.390 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.725+0000 7efef659d700 1 -- 172.21.15.18:0/1368734324 >> 172.21.15.18:0/1368734324 conn(0x7efef0101980 msgr2=0x7efef0102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:38.391 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.726+0000 7efef659d700 1 -- 172.21.15.18:0/1368734324 shutdown_connections 2024-02-17T12:40:38.391 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:37.726+0000 7efef659d700 1 -- 172.21.15.18:0/1368734324 wait complete. 2024-02-17T12:40:40.306 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:40 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1244155150' entity='client.admin' 2024-02-17T12:40:40.307 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:40 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:40.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.815+0000 7f11acb89700 1 Processor -- start 2024-02-17T12:40:40.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.815+0000 7f11acb89700 1 -- start start 2024-02-17T12:40:40.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.816+0000 7f11acb89700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 0x7f11a810a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:40.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.816+0000 7f11acb89700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a807ae70 con 0x7f11a8108150 2024-02-17T12:40:40.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.816+0000 7f11a659c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 0x7f11a810a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:40.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.816+0000 7f11a659c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 0x7f11a810a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39804/0 (socket says 172.21.15.18:39804) 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.816+0000 7f11a659c700 1 -- 172.21.15.18:0/2518709166 learned_addr learned my addr 172.21.15.18:0/2518709166 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.816+0000 7f11a659c700 1 -- 172.21.15.18:0/2518709166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a810aa80 con 0x7f11a8108150 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.817+0000 7f11a659c700 1 --2- 172.21.15.18:0/2518709166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 0x7f11a810a540 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f119c0098a0 tx=0x7f119c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6df492e9f01c24e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.817+0000 7f11a559a700 1 -- 172.21.15.18:0/2518709166 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f119c004160 con 0x7f11a8108150 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.817+0000 7f11a559a700 1 -- 172.21.15.18:0/2518709166 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f119c0042c0 con 0x7f11a8108150 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.817+0000 7f11a559a700 1 -- 172.21.15.18:0/2518709166 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f119c011a80 con 0x7f11a8108150 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.817+0000 7f11acb89700 1 -- 172.21.15.18:0/2518709166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 msgr2=0x7f11a810a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.817+0000 7f11acb89700 1 --2- 172.21.15.18:0/2518709166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 0x7f11a810a540 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f119c0098a0 tx=0x7f119c009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.818+0000 7f11acb89700 1 -- 172.21.15.18:0/2518709166 shutdown_connections 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.818+0000 7f11acb89700 1 --2- 172.21.15.18:0/2518709166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 0x7f11a810a540 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.818+0000 7f11acb89700 1 -- 172.21.15.18:0/2518709166 >> 172.21.15.18:0/2518709166 conn(0x7f11a8101b80 msgr2=0x7f11a8103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.818+0000 7f11acb89700 1 -- 172.21.15.18:0/2518709166 shutdown_connections 2024-02-17T12:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.818+0000 7f11acb89700 1 -- 172.21.15.18:0/2518709166 wait complete. 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.819+0000 7f11acb89700 1 Processor -- start 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.819+0000 7f11acb89700 1 -- start start 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.819+0000 7f11acb89700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 0x7f11a819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.819+0000 7f11acb89700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a819b960 con 0x7f11a8108150 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.819+0000 7f11a659c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 0x7f11a819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.819+0000 7f11a659c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 0x7f11a819b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39816/0 (socket says 172.21.15.18:39816) 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.819+0000 7f11a659c700 1 -- 172.21.15.18:0/1244155150 learned_addr learned my addr 172.21.15.18:0/1244155150 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.819+0000 7f11a659c700 1 -- 172.21.15.18:0/1244155150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c009550 con 0x7f11a8108150 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.820+0000 7f11a659c700 1 --2- 172.21.15.18:0/1244155150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 0x7f11a819b420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f119c009130 tx=0x7f119c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.820+0000 7f11977fe700 1 -- 172.21.15.18:0/1244155150 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f119c003aa0 con 0x7f11a8108150 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.820+0000 7f11977fe700 1 -- 172.21.15.18:0/1244155150 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f119c003c00 con 0x7f11a8108150 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.820+0000 7f11acb89700 1 -- 172.21.15.18:0/1244155150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a819bb60 con 0x7f11a8108150 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.820+0000 7f11977fe700 1 -- 172.21.15.18:0/1244155150 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f119c0248a0 con 0x7f11a8108150 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.820+0000 7f11acb89700 1 -- 172.21.15.18:0/1244155150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a819c000 con 0x7f11a8108150 2024-02-17T12:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.821+0000 7f11977fe700 1 -- 172.21.15.18:0/1244155150 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f119c024a40 con 0x7f11a8108150 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.821+0000 7f11977fe700 1 --2- 172.21.15.18:0/1244155150 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1190038ee0 0x7f119003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.821+0000 7f11977fe700 1 -- 172.21.15.18:0/1244155150 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f119c04e120 con 0x7f11a8108150 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.821+0000 7f11acb89700 1 -- 172.21.15.18:0/1244155150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a810b8c0 con 0x7f11a8108150 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.821+0000 7f11a5d9b700 1 --2- 172.21.15.18:0/1244155150 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1190038ee0 0x7f119003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.822+0000 7f11a5d9b700 1 --2- 172.21.15.18:0/1244155150 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1190038ee0 0x7f119003b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1198006fd0 tx=0x7f1198006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:38.827+0000 7f11977fe700 1 -- 172.21.15.18:0/1244155150 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f119c024cf0 con 0x7f11a8108150 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:39.047+0000 7f11acb89700 1 -- 172.21.15.18:0/1244155150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f11a819eaf0 con 0x7f11a8108150 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:39.050+0000 7f11977fe700 1 -- 172.21.15.18:0/1244155150 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f119c04b020 con 0x7f11a8108150 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:39.052+0000 7f11acb89700 1 -- 172.21.15.18:0/1244155150 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1190038ee0 msgr2=0x7f119003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:39.052+0000 7f11acb89700 1 --2- 172.21.15.18:0/1244155150 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1190038ee0 0x7f119003b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1198006fd0 tx=0x7f1198006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:39.052+0000 7f11acb89700 1 -- 172.21.15.18:0/1244155150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 msgr2=0x7f11a819b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:39.052+0000 7f11acb89700 1 --2- 172.21.15.18:0/1244155150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 0x7f11a819b420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f119c009130 tx=0x7f119c011b00 comp rx=0 tx=0).stop 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:39.052+0000 7f11acb89700 1 -- 172.21.15.18:0/1244155150 shutdown_connections 2024-02-17T12:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:39.052+0000 7f11acb89700 1 --2- 172.21.15.18:0/1244155150 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1190038ee0 0x7f119003b3a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:40.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:39.052+0000 7f11acb89700 1 --2- 172.21.15.18:0/1244155150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11a8108150 0x7f11a819b420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:40.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:39.052+0000 7f11acb89700 1 -- 172.21.15.18:0/1244155150 >> 172.21.15.18:0/1244155150 conn(0x7f11a8101b80 msgr2=0x7f11a8102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:40.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:39.052+0000 7f11acb89700 1 -- 172.21.15.18:0/1244155150 shutdown_connections 2024-02-17T12:40:40.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:39.052+0000 7f11acb89700 1 -- 172.21.15.18:0/1244155150 wait complete. 2024-02-17T12:40:41.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.790+0000 7ffa90940700 1 Processor -- start 2024-02-17T12:40:41.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.790+0000 7ffa90940700 1 -- start start 2024-02-17T12:40:41.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.791+0000 7ffa90940700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 0x7ffa8c107b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:41.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.791+0000 7ffa90940700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c1080e0 con 0x7ffa8c1076f0 2024-02-17T12:40:41.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.791+0000 7ffa8a59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 0x7ffa8c107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:41.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.791+0000 7ffa8a59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 0x7ffa8c107b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53196/0 (socket says 172.21.15.18:53196) 2024-02-17T12:40:41.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.791+0000 7ffa8a59c700 1 -- 172.21.15.18:0/1471319197 learned_addr learned my addr 172.21.15.18:0/1471319197 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:41.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.791+0000 7ffa8a59c700 1 -- 172.21.15.18:0/1471319197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c1088f0 con 0x7ffa8c1076f0 2024-02-17T12:40:41.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.791+0000 7ffa8a59c700 1 --2- 172.21.15.18:0/1471319197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 0x7ffa8c107b10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0098a0 tx=0x7ffa7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc06efa926cf2ea server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.792+0000 7ffa8959a700 1 -- 172.21.15.18:0/1471319197 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa7c004160 con 0x7ffa8c1076f0 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.792+0000 7ffa8959a700 1 -- 172.21.15.18:0/1471319197 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c0042c0 con 0x7ffa8c1076f0 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.792+0000 7ffa8959a700 1 -- 172.21.15.18:0/1471319197 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa7c011a80 con 0x7ffa8c1076f0 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.792+0000 7ffa90940700 1 -- 172.21.15.18:0/1471319197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 msgr2=0x7ffa8c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.792+0000 7ffa90940700 1 --2- 172.21.15.18:0/1471319197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 0x7ffa8c107b10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0098a0 tx=0x7ffa7c009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.792+0000 7ffa90940700 1 -- 172.21.15.18:0/1471319197 shutdown_connections 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.792+0000 7ffa90940700 1 --2- 172.21.15.18:0/1471319197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 0x7ffa8c107b10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.792+0000 7ffa90940700 1 -- 172.21.15.18:0/1471319197 >> 172.21.15.18:0/1471319197 conn(0x7ffa8c102d90 msgr2=0x7ffa8c1051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.792+0000 7ffa90940700 1 -- 172.21.15.18:0/1471319197 shutdown_connections 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.792+0000 7ffa90940700 1 -- 172.21.15.18:0/1471319197 wait complete. 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.793+0000 7ffa90940700 1 Processor -- start 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.793+0000 7ffa90940700 1 -- start start 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.793+0000 7ffa90940700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 0x7ffa8c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:41.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.793+0000 7ffa90940700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c19b9a0 con 0x7ffa8c1076f0 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.793+0000 7ffa8a59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 0x7ffa8c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.793+0000 7ffa8a59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 0x7ffa8c19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53202/0 (socket says 172.21.15.18:53202) 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.793+0000 7ffa8a59c700 1 -- 172.21.15.18:0/2839935573 learned_addr learned my addr 172.21.15.18:0/2839935573 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.794+0000 7ffa8a59c700 1 -- 172.21.15.18:0/2839935573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c009550 con 0x7ffa8c1076f0 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.794+0000 7ffa8a59c700 1 --2- 172.21.15.18:0/2839935573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 0x7ffa8c19b460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c000c00 tx=0x7ffa7c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.794+0000 7ffa7b7fe700 1 -- 172.21.15.18:0/2839935573 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa7c003aa0 con 0x7ffa8c1076f0 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.794+0000 7ffa90940700 1 -- 172.21.15.18:0/2839935573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8c19bba0 con 0x7ffa8c1076f0 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.794+0000 7ffa7b7fe700 1 -- 172.21.15.18:0/2839935573 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c003c00 con 0x7ffa8c1076f0 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.794+0000 7ffa7b7fe700 1 -- 172.21.15.18:0/2839935573 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa7c024930 con 0x7ffa8c1076f0 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.794+0000 7ffa90940700 1 -- 172.21.15.18:0/2839935573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c19c040 con 0x7ffa8c1076f0 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.795+0000 7ffa7b7fe700 1 -- 172.21.15.18:0/2839935573 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7ffa7c02d3f0 con 0x7ffa8c1076f0 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.795+0000 7ffa7b7fe700 1 --2- 172.21.15.18:0/2839935573 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7ffa74038ee0 0x7ffa7403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:41.496 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.795+0000 7ffa7b7fe700 1 -- 172.21.15.18:0/2839935573 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffa7c04d410 con 0x7ffa8c1076f0 2024-02-17T12:40:41.497 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.795+0000 7ffa89d9b700 1 --2- 172.21.15.18:0/2839935573 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7ffa74038ee0 0x7ffa7403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:41.497 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.795+0000 7ffa90940700 1 -- 172.21.15.18:0/2839935573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c10bb20 con 0x7ffa8c1076f0 2024-02-17T12:40:41.497 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.796+0000 7ffa89d9b700 1 --2- 172.21.15.18:0/2839935573 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7ffa74038ee0 0x7ffa7403b3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffa80006fd0 tx=0x7ffa80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:41.497 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:40.800+0000 7ffa7b7fe700 1 -- 172.21.15.18:0/2839935573 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa7c010970 con 0x7ffa8c1076f0 2024-02-17T12:40:41.497 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.006+0000 7ffa90940700 1 -- 172.21.15.18:0/2839935573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7ffa8c19eaf0 con 0x7ffa8c1076f0 2024-02-17T12:40:41.497 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.008+0000 7ffa7b7fe700 1 -- 172.21.15.18:0/2839935573 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7ffa7c010380 con 0x7ffa8c1076f0 2024-02-17T12:40:41.497 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.011+0000 7ffa90940700 1 -- 172.21.15.18:0/2839935573 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7ffa74038ee0 msgr2=0x7ffa7403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:41.497 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.011+0000 7ffa90940700 1 --2- 172.21.15.18:0/2839935573 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7ffa74038ee0 0x7ffa7403b3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffa80006fd0 tx=0x7ffa80006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:41.497 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.011+0000 7ffa90940700 1 -- 172.21.15.18:0/2839935573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 msgr2=0x7ffa8c19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:41.497 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.011+0000 7ffa90940700 1 --2- 172.21.15.18:0/2839935573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 0x7ffa8c19b460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c000c00 tx=0x7ffa7c011b00 comp rx=0 tx=0).stop 2024-02-17T12:40:41.498 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.011+0000 7ffa90940700 1 -- 172.21.15.18:0/2839935573 shutdown_connections 2024-02-17T12:40:41.498 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.011+0000 7ffa90940700 1 --2- 172.21.15.18:0/2839935573 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7ffa74038ee0 0x7ffa7403b3a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:41.498 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.011+0000 7ffa90940700 1 --2- 172.21.15.18:0/2839935573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa8c1076f0 0x7ffa8c19b460 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:41.498 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.011+0000 7ffa90940700 1 -- 172.21.15.18:0/2839935573 >> 172.21.15.18:0/2839935573 conn(0x7ffa8c102d90 msgr2=0x7ffa8c103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:41.498 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.011+0000 7ffa90940700 1 -- 172.21.15.18:0/2839935573 shutdown_connections 2024-02-17T12:40:41.498 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.011+0000 7ffa90940700 1 -- 172.21.15.18:0/2839935573 wait complete. 2024-02-17T12:40:41.498 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling the dashboard module... 2024-02-17T12:40:42.133 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:42 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2839935573' entity='client.admin' 2024-02-17T12:40:42.133 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:42 smithi018 ceph-mon[32625]: from='mgr.14120 172.21.15.18:0/2682730273' entity='mgr.x' 2024-02-17T12:40:43.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:43 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2773626545' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-17T12:40:43.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:43 smithi018 ceph-mon[32625]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:40:43.271 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:43 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: ignoring --setuser ceph since I am not root 2024-02-17T12:40:43.271 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:43 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: ignoring --setgroup ceph since I am not root 2024-02-17T12:40:43.932 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:43 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:43.600+0000 7efeb02e5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T12:40:43.932 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:43 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:43.697+0000 7efeb02e5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T12:40:43.945 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.864+0000 7f7cf0215700 1 Processor -- start 2024-02-17T12:40:43.945 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.864+0000 7f7cf0215700 1 -- start start 2024-02-17T12:40:43.945 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.865+0000 7f7cf0215700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 0x7f7ce8107f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:43.946 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.865+0000 7f7cf0215700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce8108530 con 0x7f7ce8107b40 2024-02-17T12:40:43.946 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.865+0000 7f7cedfb1700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 0x7f7ce8107f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:43.946 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.865+0000 7f7cedfb1700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 0x7f7ce8107f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53216/0 (socket says 172.21.15.18:53216) 2024-02-17T12:40:43.946 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.865+0000 7f7cedfb1700 1 -- 172.21.15.18:0/2108776389 learned_addr learned my addr 172.21.15.18:0/2108776389 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:43.946 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.865+0000 7f7cedfb1700 1 -- 172.21.15.18:0/2108776389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce81067a0 con 0x7f7ce8107b40 2024-02-17T12:40:43.946 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.865+0000 7f7cedfb1700 1 --2- 172.21.15.18:0/2108776389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 0x7f7ce8107f60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7cd80098a0 tx=0x7f7cd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4f038c9daa99a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:43.946 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.865+0000 7f7cecfaf700 1 -- 172.21.15.18:0/2108776389 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd8004160 con 0x7f7ce8107b40 2024-02-17T12:40:43.947 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.865+0000 7f7cecfaf700 1 -- 172.21.15.18:0/2108776389 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cd80042c0 con 0x7f7ce8107b40 2024-02-17T12:40:43.947 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.865+0000 7f7cecfaf700 1 -- 172.21.15.18:0/2108776389 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd8011a80 con 0x7f7ce8107b40 2024-02-17T12:40:43.947 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.866+0000 7f7cf0215700 1 -- 172.21.15.18:0/2108776389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 msgr2=0x7f7ce8107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:43.947 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.866+0000 7f7cf0215700 1 --2- 172.21.15.18:0/2108776389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 0x7f7ce8107f60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7cd80098a0 tx=0x7f7cd8009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:43.947 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.866+0000 7f7cf0215700 1 -- 172.21.15.18:0/2108776389 shutdown_connections 2024-02-17T12:40:43.947 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.866+0000 7f7cf0215700 1 --2- 172.21.15.18:0/2108776389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 0x7f7ce8107f60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:43.948 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.866+0000 7f7cf0215700 1 -- 172.21.15.18:0/2108776389 >> 172.21.15.18:0/2108776389 conn(0x7f7ce8101370 msgr2=0x7f7ce8103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:43.948 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.866+0000 7f7cf0215700 1 -- 172.21.15.18:0/2108776389 shutdown_connections 2024-02-17T12:40:43.948 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.866+0000 7f7cf0215700 1 -- 172.21.15.18:0/2108776389 wait complete. 2024-02-17T12:40:43.948 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.867+0000 7f7cf0215700 1 Processor -- start 2024-02-17T12:40:43.948 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.867+0000 7f7cf0215700 1 -- start start 2024-02-17T12:40:43.949 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.867+0000 7f7cf0215700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 0x7f7ce819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:43.949 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.867+0000 7f7cf0215700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce819b940 con 0x7f7ce8107b40 2024-02-17T12:40:43.949 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.867+0000 7f7cedfb1700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 0x7f7ce819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:43.949 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.867+0000 7f7cedfb1700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 0x7f7ce819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53218/0 (socket says 172.21.15.18:53218) 2024-02-17T12:40:43.949 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.867+0000 7f7cedfb1700 1 -- 172.21.15.18:0/2773626545 learned_addr learned my addr 172.21.15.18:0/2773626545 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:43.949 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.868+0000 7f7cedfb1700 1 -- 172.21.15.18:0/2773626545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd8009550 con 0x7f7ce8107b40 2024-02-17T12:40:43.950 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.868+0000 7f7cedfb1700 1 --2- 172.21.15.18:0/2773626545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 0x7f7ce819b400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8011740 tx=0x7f7cd8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:43.950 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.868+0000 7f7cdeffd700 1 -- 172.21.15.18:0/2773626545 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd8003b20 con 0x7f7ce8107b40 2024-02-17T12:40:43.950 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.868+0000 7f7cf0215700 1 -- 172.21.15.18:0/2773626545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce819bb40 con 0x7f7ce8107b40 2024-02-17T12:40:43.950 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.868+0000 7f7cdeffd700 1 -- 172.21.15.18:0/2773626545 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cd801a910 con 0x7f7ce8107b40 2024-02-17T12:40:43.950 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.868+0000 7f7cdeffd700 1 -- 172.21.15.18:0/2773626545 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd80248c0 con 0x7f7ce8107b40 2024-02-17T12:40:43.950 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.868+0000 7f7cf0215700 1 -- 172.21.15.18:0/2773626545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce819bfe0 con 0x7f7ce8107b40 2024-02-17T12:40:43.951 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.869+0000 7f7cdeffd700 1 -- 172.21.15.18:0/2773626545 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f7cd802d3f0 con 0x7f7ce8107b40 2024-02-17T12:40:43.951 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.869+0000 7f7cdeffd700 1 --2- 172.21.15.18:0/2773626545 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f7cd4038ee0 0x7f7cd403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:43.951 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.869+0000 7f7cdeffd700 1 -- 172.21.15.18:0/2773626545 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7cd804d4f0 con 0x7f7ce8107b40 2024-02-17T12:40:43.951 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.869+0000 7f7cf0215700 1 -- 172.21.15.18:0/2773626545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce8195600 con 0x7f7ce8107b40 2024-02-17T12:40:43.951 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.869+0000 7f7ced7b0700 1 --2- 172.21.15.18:0/2773626545 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f7cd4038ee0 0x7f7cd403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:43.951 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.870+0000 7f7ced7b0700 1 --2- 172.21.15.18:0/2773626545 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f7cd4038ee0 0x7f7cd403b3a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4006fd0 tx=0x7f7ce4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:43.952 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:41.874+0000 7f7cdeffd700 1 -- 172.21.15.18:0/2773626545 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7cd802d6f0 con 0x7f7ce8107b40 2024-02-17T12:40:43.952 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:42.136+0000 7f7cf0215700 1 -- 172.21.15.18:0/2773626545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f7ce810abd0 con 0x7f7ce8107b40 2024-02-17T12:40:43.952 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:43.011+0000 7f7cdeffd700 1 -- 172.21.15.18:0/2773626545 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f7cd8010380 con 0x7f7ce8107b40 2024-02-17T12:40:43.952 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:43.013+0000 7f7cdeffd700 1 -- 172.21.15.18:0/2773626545 <== mon.0 v2:172.21.15.18:3300/0 8 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f7cd802d8d0 con 0x7f7ce8107b40 2024-02-17T12:40:43.952 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:43.014+0000 7f7cf0215700 1 -- 172.21.15.18:0/2773626545 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f7cd4038ee0 msgr2=0x7f7cd403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:43.952 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:43.015+0000 7f7cf0215700 1 --2- 172.21.15.18:0/2773626545 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f7cd4038ee0 0x7f7cd403b3a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4006fd0 tx=0x7f7ce4006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:43.953 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:43.015+0000 7f7cf0215700 1 -- 172.21.15.18:0/2773626545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 msgr2=0x7f7ce819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:43.953 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:43.015+0000 7f7cf0215700 1 --2- 172.21.15.18:0/2773626545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 0x7f7ce819b400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8011740 tx=0x7f7cd8011b00 comp rx=0 tx=0).stop 2024-02-17T12:40:43.953 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:43.015+0000 7f7cf0215700 1 -- 172.21.15.18:0/2773626545 shutdown_connections 2024-02-17T12:40:43.953 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:43.015+0000 7f7cf0215700 1 --2- 172.21.15.18:0/2773626545 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f7cd4038ee0 0x7f7cd403b3a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:43.953 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:43.015+0000 7f7cf0215700 1 --2- 172.21.15.18:0/2773626545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce8107b40 0x7f7ce819b400 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:43.953 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:43.015+0000 7f7cf0215700 1 -- 172.21.15.18:0/2773626545 >> 172.21.15.18:0/2773626545 conn(0x7f7ce8101370 msgr2=0x7f7ce8102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:43.953 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:43.015+0000 7f7cf0215700 1 -- 172.21.15.18:0/2773626545 shutdown_connections 2024-02-17T12:40:43.954 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:43.015+0000 7f7cf0215700 1 -- 172.21.15.18:0/2773626545 wait complete. 2024-02-17T12:40:44.260 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:44 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2773626545' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-17T12:40:44.261 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:44 smithi018 ceph-mon[32625]: mgrmap e9: x(active, since 20s) 2024-02-17T12:40:44.528 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:44 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:44.367+0000 7efeb02e5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T12:40:44.529 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:44 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:44.467+0000 7efeb02e5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T12:40:45.008 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:44 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:44.611+0000 7efeb02e5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T12:40:45.009 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:44 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:44.720+0000 7efeb02e5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T12:40:45.009 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:44 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:44.806+0000 7efeb02e5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T12:40:45.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.365+0000 7f1a38f47700 1 Processor -- start 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.365+0000 7f1a38f47700 1 -- start start 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.366+0000 7f1a38f47700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 0x7f1a34107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.366+0000 7f1a38f47700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a34108070 con 0x7f1a34107680 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.366+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 0x7f1a34107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.366+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 0x7f1a34107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53242/0 (socket says 172.21.15.18:53242) 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.366+0000 7f1a3259c700 1 -- 172.21.15.18:0/991934946 learned_addr learned my addr 172.21.15.18:0/991934946 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.366+0000 7f1a3259c700 1 -- 172.21.15.18:0/991934946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a34108830 con 0x7f1a34107680 2024-02-17T12:40:45.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.366+0000 7f1a3259c700 1 --2- 172.21.15.18:0/991934946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 0x7f1a34107aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009e80 tx=0x7f1a2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c94251611cd9af9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.366+0000 7f1a3159a700 1 -- 172.21.15.18:0/991934946 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a2400e070 con 0x7f1a34107680 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.366+0000 7f1a3159a700 1 -- 172.21.15.18:0/991934946 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a2400b8c0 con 0x7f1a34107680 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.366+0000 7f1a3159a700 1 -- 172.21.15.18:0/991934946 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a240044a0 con 0x7f1a34107680 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.367+0000 7f1a38f47700 1 -- 172.21.15.18:0/991934946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 msgr2=0x7f1a34107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.367+0000 7f1a38f47700 1 --2- 172.21.15.18:0/991934946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 0x7f1a34107aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009e80 tx=0x7f1a2400b330 comp rx=0 tx=0).stop 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.367+0000 7f1a38f47700 1 -- 172.21.15.18:0/991934946 shutdown_connections 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.367+0000 7f1a38f47700 1 --2- 172.21.15.18:0/991934946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 0x7f1a34107aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.367+0000 7f1a38f47700 1 -- 172.21.15.18:0/991934946 >> 172.21.15.18:0/991934946 conn(0x7f1a34102c80 msgr2=0x7f1a341050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.367+0000 7f1a38f47700 1 -- 172.21.15.18:0/991934946 shutdown_connections 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.367+0000 7f1a38f47700 1 -- 172.21.15.18:0/991934946 wait complete. 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.368+0000 7f1a38f47700 1 Processor -- start 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.368+0000 7f1a38f47700 1 -- start start 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.368+0000 7f1a38f47700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 0x7f1a3419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:45.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.368+0000 7f1a38f47700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3419b8a0 con 0x7f1a34107680 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.368+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 0x7f1a3419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.369+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 0x7f1a3419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53250/0 (socket says 172.21.15.18:53250) 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.369+0000 7f1a3259c700 1 -- 172.21.15.18:0/2008350229 learned_addr learned my addr 172.21.15.18:0/2008350229 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.369+0000 7f1a3259c700 1 -- 172.21.15.18:0/2008350229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24009b30 con 0x7f1a34107680 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.369+0000 7f1a3259c700 1 --2- 172.21.15.18:0/2008350229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 0x7f1a3419b360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1a24004070 tx=0x7f1a2400bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.369+0000 7f1a2b7fe700 1 -- 172.21.15.18:0/2008350229 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a2400e040 con 0x7f1a34107680 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.369+0000 7f1a2b7fe700 1 -- 172.21.15.18:0/2008350229 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a24003d60 con 0x7f1a34107680 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.369+0000 7f1a38f47700 1 -- 172.21.15.18:0/2008350229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3419baa0 con 0x7f1a34107680 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.369+0000 7f1a2b7fe700 1 -- 172.21.15.18:0/2008350229 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a2401c780 con 0x7f1a34107680 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.369+0000 7f1a38f47700 1 -- 172.21.15.18:0/2008350229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3419bfa0 con 0x7f1a34107680 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.370+0000 7f1a2b7fe700 1 -- 172.21.15.18:0/2008350229 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f1a2401ca00 con 0x7f1a34107680 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.370+0000 7f1a38f47700 1 -- 172.21.15.18:0/2008350229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3410ba60 con 0x7f1a34107680 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.370+0000 7f1a2b7fe700 1 --2- 172.21.15.18:0/2008350229 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1a20038ee0 0x7f1a2003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:45.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.370+0000 7f1a2b7fe700 1 -- 172.21.15.18:0/2008350229 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1a2404d0e0 con 0x7f1a34107680 2024-02-17T12:40:45.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.370+0000 7f1a31d9b700 1 -- 172.21.15.18:0/2008350229 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1a20038ee0 msgr2=0x7f1a2003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1086757713 2024-02-17T12:40:45.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.370+0000 7f1a31d9b700 1 --2- 172.21.15.18:0/2008350229 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1a20038ee0 0x7f1a2003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T12:40:45.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.375+0000 7f1a2b7fe700 1 -- 172.21.15.18:0/2008350229 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a24011390 con 0x7f1a34107680 2024-02-17T12:40:45.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.572+0000 7f1a31d9b700 1 -- 172.21.15.18:0/2008350229 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1a20038ee0 msgr2=0x7f1a2003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1086757713 2024-02-17T12:40:45.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.572+0000 7f1a31d9b700 1 --2- 172.21.15.18:0/2008350229 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1a20038ee0 0x7f1a2003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-17T12:40:45.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.635+0000 7f1a38f47700 1 -- 172.21.15.18:0/2008350229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f1a3419ecd0 con 0x7f1a34107680 2024-02-17T12:40:45.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.636+0000 7f1a2b7fe700 1 -- 172.21.15.18:0/2008350229 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f1a24017070 con 0x7f1a34107680 2024-02-17T12:40:45.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.638+0000 7f1a38f47700 1 -- 172.21.15.18:0/2008350229 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1a20038ee0 msgr2=0x7f1a2003b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:40:45.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.638+0000 7f1a38f47700 1 --2- 172.21.15.18:0/2008350229 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1a20038ee0 0x7f1a2003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:45.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.638+0000 7f1a38f47700 1 -- 172.21.15.18:0/2008350229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 msgr2=0x7f1a3419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:45.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.638+0000 7f1a38f47700 1 --2- 172.21.15.18:0/2008350229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 0x7f1a3419b360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1a24004070 tx=0x7f1a2400bd10 comp rx=0 tx=0).stop 2024-02-17T12:40:45.185 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.638+0000 7f1a38f47700 1 -- 172.21.15.18:0/2008350229 shutdown_connections 2024-02-17T12:40:45.185 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.638+0000 7f1a38f47700 1 --2- 172.21.15.18:0/2008350229 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1a20038ee0 0x7f1a2003b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:45.185 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.638+0000 7f1a38f47700 1 --2- 172.21.15.18:0/2008350229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a34107680 0x7f1a3419b360 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:45.185 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.638+0000 7f1a38f47700 1 -- 172.21.15.18:0/2008350229 >> 172.21.15.18:0/2008350229 conn(0x7f1a34102c80 msgr2=0x7f1a341036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:45.185 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.638+0000 7f1a38f47700 1 -- 172.21.15.18:0/2008350229 shutdown_connections 2024-02-17T12:40:45.185 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:44.638+0000 7f1a38f47700 1 -- 172.21.15.18:0/2008350229 wait complete. 2024-02-17T12:40:45.185 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for the mgr to restart... 2024-02-17T12:40:45.185 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr epoch 9... 2024-02-17T12:40:45.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:45 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2008350229' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-17T12:40:45.279 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:45 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:45.077+0000 7efeb02e5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T12:40:45.279 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:45 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:45.192+0000 7efeb02e5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T12:40:45.560 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:45 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:45.283+0000 7efeb02e5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T12:40:45.561 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:45 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:45.381+0000 7efeb02e5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T12:40:45.561 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:45 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:45.564+0000 7efeb02e5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T12:40:47.016 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:46 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:46.559+0000 7efeb02e5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T12:40:47.016 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:46 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:46.694+0000 7efeb02e5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T12:40:47.278 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:47.033+0000 7efeb02e5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T12:40:47.278 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:47.126+0000 7efeb02e5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T12:40:47.778 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:47.326+0000 7efeb02e5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:40:47.779 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:47.517+0000 7efeb02e5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T12:40:48.074 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:47.815+0000 7efeb02e5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:40:48.528 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:48.078+0000 7efeb02e5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T12:40:48.529 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:48.322+0000 7efeb02e5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T12:40:49.528 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:49.158+0000 7efeb02e5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T12:40:50.028 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:49.772+0000 7efeb02e5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T12:40:50.028 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:49.874+0000 7efeb02e5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T12:40:50.372 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:40:50.100+0000 7efeb02e5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T12:40:50.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-mon[32625]: Active manager daemon x restarted 2024-02-17T12:40:50.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-mon[32625]: Activating manager daemon x 2024-02-17T12:40:50.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-mon[32625]: osdmap e3: 0 total, 0 up, 0 in 2024-02-17T12:40:50.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-mon[32625]: mgrmap e10: x(active, starting, since 0.00316776s) 2024-02-17T12:40:50.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:40:50.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-17T12:40:50.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-17T12:40:50.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-17T12:40:50.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-17T12:40:50.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-mon[32625]: Manager daemon x is now available 2024-02-17T12:40:50.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:40:50.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:50 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:40:51.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-17T12:40:51.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-17T12:40:51.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:51 smithi018 ceph-mon[32625]: mgrmap e11: x(active, since 1.00594s) 2024-02-17T12:40:51.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-02-17T12:40:51.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-17T12:40:51.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-17T12:40:51.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-02-17T12:40:51.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.612+0000 7f1abe75d700 1 Processor -- start 2024-02-17T12:40:51.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.612+0000 7f1abe75d700 1 -- start start 2024-02-17T12:40:51.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.612+0000 7f1abe75d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 0x7f1ab8107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:51.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.612+0000 7f1abe75d700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab8108110 con 0x7f1ab8107720 2024-02-17T12:40:51.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.612+0000 7f1ab7fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 0x7f1ab8107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:51.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.612+0000 7f1ab7fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 0x7f1ab8107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53258/0 (socket says 172.21.15.18:53258) 2024-02-17T12:40:51.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.612+0000 7f1ab7fff700 1 -- 172.21.15.18:0/3316383314 learned_addr learned my addr 172.21.15.18:0/3316383314 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:51.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.613+0000 7f1ab7fff700 1 -- 172.21.15.18:0/3316383314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab8108920 con 0x7f1ab8107720 2024-02-17T12:40:51.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.613+0000 7f1ab7fff700 1 --2- 172.21.15.18:0/3316383314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 0x7f1ab8107b40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80098a0 tx=0x7f1aa8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734aba98911a48a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:51.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.613+0000 7f1ab6ffd700 1 -- 172.21.15.18:0/3316383314 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1aa8004160 con 0x7f1ab8107720 2024-02-17T12:40:51.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.613+0000 7f1ab6ffd700 1 -- 172.21.15.18:0/3316383314 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1aa80042c0 con 0x7f1ab8107720 2024-02-17T12:40:51.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.613+0000 7f1ab6ffd700 1 -- 172.21.15.18:0/3316383314 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1aa8011a80 con 0x7f1ab8107720 2024-02-17T12:40:51.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.614+0000 7f1abe75d700 1 -- 172.21.15.18:0/3316383314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 msgr2=0x7f1ab8107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:51.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.614+0000 7f1abe75d700 1 --2- 172.21.15.18:0/3316383314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 0x7f1ab8107b40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80098a0 tx=0x7f1aa8009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:51.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.614+0000 7f1abe75d700 1 -- 172.21.15.18:0/3316383314 shutdown_connections 2024-02-17T12:40:51.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.614+0000 7f1abe75d700 1 --2- 172.21.15.18:0/3316383314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 0x7f1ab8107b40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:51.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.614+0000 7f1abe75d700 1 -- 172.21.15.18:0/3316383314 >> 172.21.15.18:0/3316383314 conn(0x7f1ab8102d60 msgr2=0x7f1ab8105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.614+0000 7f1abe75d700 1 -- 172.21.15.18:0/3316383314 shutdown_connections 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.614+0000 7f1abe75d700 1 -- 172.21.15.18:0/3316383314 wait complete. 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.614+0000 7f1abe75d700 1 Processor -- start 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.614+0000 7f1abe75d700 1 -- start start 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.615+0000 7f1abe75d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 0x7f1ab819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.615+0000 7f1abe75d700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab819b990 con 0x7f1ab8107720 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.615+0000 7f1ab7fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 0x7f1ab819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.615+0000 7f1ab7fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 0x7f1ab819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53266/0 (socket says 172.21.15.18:53266) 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.615+0000 7f1ab7fff700 1 -- 172.21.15.18:0/2594084645 learned_addr learned my addr 172.21.15.18:0/2594084645 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.615+0000 7f1ab7fff700 1 -- 172.21.15.18:0/2594084645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa8009550 con 0x7f1ab8107720 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.615+0000 7f1ab7fff700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 0x7f1ab819b450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8003f20 tx=0x7f1aa8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.615+0000 7f1ab57fa700 1 -- 172.21.15.18:0/2594084645 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1aa80039f0 con 0x7f1ab8107720 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.615+0000 7f1ab57fa700 1 -- 172.21.15.18:0/2594084645 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1aa8003b50 con 0x7f1ab8107720 2024-02-17T12:40:51.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.615+0000 7f1ab57fa700 1 -- 172.21.15.18:0/2594084645 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1aa801a690 con 0x7f1ab8107720 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.615+0000 7f1abe75d700 1 -- 172.21.15.18:0/2594084645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab819bb90 con 0x7f1ab8107720 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.616+0000 7f1abe75d700 1 -- 172.21.15.18:0/2594084645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab819c030 con 0x7f1ab8107720 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.616+0000 7f1ab57fa700 1 -- 172.21.15.18:0/2594084645 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f1aa801a900 con 0x7f1ab8107720 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.617+0000 7f1ab57fa700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 0x7f1aa403b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.617+0000 7f1ab57fa700 1 -- 172.21.15.18:0/2594084645 --> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa403bb10 con 0x7f1aa4038f40 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.617+0000 7f1ab57fa700 1 -- 172.21.15.18:0/2594084645 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1aa804d3a0 con 0x7f1ab8107720 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.617+0000 7f1ab77fe700 1 -- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 msgr2=0x7f1aa403b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1086757713 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.617+0000 7f1ab77fe700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 0x7f1aa403b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.818+0000 7f1ab77fe700 1 -- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 msgr2=0x7f1aa403b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1086757713 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:45.818+0000 7f1ab77fe700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 0x7f1aa403b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:46.219+0000 7f1ab77fe700 1 -- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 msgr2=0x7f1aa403b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1086757713 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:46.219+0000 7f1ab77fe700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 0x7f1aa403b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:47.020+0000 7f1ab77fe700 1 -- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 msgr2=0x7f1aa403b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1086757713 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:47.020+0000 7f1ab77fe700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 0x7f1aa403b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-17T12:40:51.910 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:48.621+0000 7f1ab77fe700 1 -- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 msgr2=0x7f1aa403b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1086757713 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:48.621+0000 7f1ab77fe700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 0x7f1aa403b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:50.115+0000 7f1ab57fa700 1 -- 172.21.15.18:0/2594084645 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mgrmap(e 10) v1 ==== 45712+0+0 (secure 0 0 0) 0x7f1aa802da00 con 0x7f1ab8107720 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:50.115+0000 7f1ab57fa700 1 -- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 msgr2=0x7f1aa403b400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:50.115+0000 7f1ab57fa700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 0x7f1aa403b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.117+0000 7f1ab57fa700 1 -- 172.21.15.18:0/2594084645 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7f1aa8029cf0 con 0x7f1ab8107720 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.117+0000 7f1ab57fa700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1aa403c9f0 0x7f1aa403ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.117+0000 7f1ab57fa700 1 -- 172.21.15.18:0/2594084645 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa403bb10 con 0x7f1aa403c9f0 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.117+0000 7f1ab77fe700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1aa403c9f0 0x7f1aa403ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.118+0000 7f1ab77fe700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1aa403c9f0 0x7f1aa403ede0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0003a10 tx=0x7f1aa0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.119+0000 7f1ab57fa700 1 -- 172.21.15.18:0/2594084645 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f1aa403bb10 con 0x7f1aa403c9f0 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.130+0000 7f1abe75d700 1 -- 172.21.15.18:0/2594084645 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f1ab810bb20 con 0x7f1aa403c9f0 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.130+0000 7f1ab57fa700 1 -- 172.21.15.18:0/2594084645 <== mgr.14152 v2:172.21.15.18:6800/3026633658 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f1ab810bb20 con 0x7f1aa403c9f0 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.131+0000 7f1abe75d700 1 -- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1aa403c9f0 msgr2=0x7f1aa403ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:51.911 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.131+0000 7f1abe75d700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1aa403c9f0 0x7f1aa403ede0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0003a10 tx=0x7f1aa0009270 comp rx=0 tx=0).stop 2024-02-17T12:40:51.912 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.131+0000 7f1abe75d700 1 -- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 msgr2=0x7f1ab819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:51.912 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.131+0000 7f1abe75d700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 0x7f1ab819b450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8003f20 tx=0x7f1aa8011b00 comp rx=0 tx=0).stop 2024-02-17T12:40:51.912 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.131+0000 7f1abe75d700 1 -- 172.21.15.18:0/2594084645 shutdown_connections 2024-02-17T12:40:51.912 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.131+0000 7f1abe75d700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab8107720 0x7f1ab819b450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:51.912 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.131+0000 7f1abe75d700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1aa403c9f0 0x7f1aa403ede0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:51.912 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.131+0000 7f1abe75d700 1 --2- 172.21.15.18:0/2594084645 >> [v2:172.21.15.18:6800/1086757713,v1:172.21.15.18:6801/1086757713] conn(0x7f1aa4038f40 0x7f1aa403b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:51.912 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.131+0000 7f1abe75d700 1 -- 172.21.15.18:0/2594084645 >> 172.21.15.18:0/2594084645 conn(0x7f1ab8102d60 msgr2=0x7f1ab81037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:51.912 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.131+0000 7f1abe75d700 1 -- 172.21.15.18:0/2594084645 shutdown_connections 2024-02-17T12:40:51.912 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:51.131+0000 7f1abe75d700 1 -- 172.21.15.18:0/2594084645 wait complete. 2024-02-17T12:40:51.912 INFO:teuthology.orchestra.run.smithi018.stdout:mgr epoch 9 is available 2024-02-17T12:40:51.912 INFO:teuthology.orchestra.run.smithi018.stdout:Generating a dashboard self-signed certificate... 2024-02-17T12:40:52.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:52 smithi018 ceph-mon[32625]: [17/Feb/2024:12:40:51] ENGINE Bus STARTING 2024-02-17T12:40:52.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:52 smithi018 ceph-mon[32625]: [17/Feb/2024:12:40:51] ENGINE Serving on https://172.21.15.18:7150 2024-02-17T12:40:52.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:52 smithi018 ceph-mon[32625]: [17/Feb/2024:12:40:51] ENGINE Serving on http://172.21.15.18:8765 2024-02-17T12:40:52.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:52 smithi018 ceph-mon[32625]: [17/Feb/2024:12:40:51] ENGINE Bus STARTED 2024-02-17T12:40:53.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-17T12:40:53.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.338+0000 7f40b02c3700 1 Processor -- start 2024-02-17T12:40:53.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.338+0000 7f40b02c3700 1 -- start start 2024-02-17T12:40:53.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.338+0000 7f40b02c3700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 0x7f40a8105da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:53.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.338+0000 7f40b02c3700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a8106370 con 0x7f40a8105980 2024-02-17T12:40:53.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.338+0000 7f40ae05f700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 0x7f40a8105da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:53.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.338+0000 7f40ae05f700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 0x7f40a8105da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60948/0 (socket says 172.21.15.18:60948) 2024-02-17T12:40:53.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.338+0000 7f40ae05f700 1 -- 172.21.15.18:0/4070083571 learned_addr learned my addr 172.21.15.18:0/4070083571 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:53.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.339+0000 7f40ae05f700 1 -- 172.21.15.18:0/4070083571 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a81064b0 con 0x7f40a8105980 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.339+0000 7f40ae05f700 1 --2- 172.21.15.18:0/4070083571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 0x7f40a8105da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f40a40098a0 tx=0x7f40a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e23e70c55a982f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.339+0000 7f40ad05d700 1 -- 172.21.15.18:0/4070083571 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40a4004160 con 0x7f40a8105980 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.339+0000 7f40ad05d700 1 -- 172.21.15.18:0/4070083571 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40a40042c0 con 0x7f40a8105980 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.339+0000 7f40ad05d700 1 -- 172.21.15.18:0/4070083571 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40a4011a80 con 0x7f40a8105980 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.339+0000 7f40b02c3700 1 -- 172.21.15.18:0/4070083571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 msgr2=0x7f40a8105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.339+0000 7f40b02c3700 1 --2- 172.21.15.18:0/4070083571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 0x7f40a8105da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f40a40098a0 tx=0x7f40a4009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.340+0000 7f40b02c3700 1 -- 172.21.15.18:0/4070083571 shutdown_connections 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.340+0000 7f40b02c3700 1 --2- 172.21.15.18:0/4070083571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 0x7f40a8105da0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.340+0000 7f40b02c3700 1 -- 172.21.15.18:0/4070083571 >> 172.21.15.18:0/4070083571 conn(0x7f40a8101430 msgr2=0x7f40a8103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.340+0000 7f40b02c3700 1 -- 172.21.15.18:0/4070083571 shutdown_connections 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.340+0000 7f40b02c3700 1 -- 172.21.15.18:0/4070083571 wait complete. 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.340+0000 7f40b02c3700 1 Processor -- start 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.340+0000 7f40b02c3700 1 -- start start 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.340+0000 7f40b02c3700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 0x7f40a8108270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:53.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.340+0000 7f40b02c3700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a81087b0 con 0x7f40a8105980 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.341+0000 7f40ae05f700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 0x7f40a8108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.341+0000 7f40ae05f700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 0x7f40a8108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60956/0 (socket says 172.21.15.18:60956) 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.341+0000 7f40ae05f700 1 -- 172.21.15.18:0/639707235 learned_addr learned my addr 172.21.15.18:0/639707235 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.341+0000 7f40ae05f700 1 -- 172.21.15.18:0/639707235 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4009550 con 0x7f40a8105980 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.341+0000 7f40ae05f700 1 --2- 172.21.15.18:0/639707235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 0x7f40a8108270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f40a4003f20 tx=0x7f40a4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.341+0000 7f409f7fe700 1 -- 172.21.15.18:0/639707235 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40a40039f0 con 0x7f40a8105980 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.341+0000 7f409f7fe700 1 -- 172.21.15.18:0/639707235 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40a4003b50 con 0x7f40a8105980 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.341+0000 7f40b02c3700 1 -- 172.21.15.18:0/639707235 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a81069a0 con 0x7f40a8105980 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.341+0000 7f409f7fe700 1 -- 172.21.15.18:0/639707235 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40a401a690 con 0x7f40a8105980 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.341+0000 7f40b02c3700 1 -- 172.21.15.18:0/639707235 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a8106e40 con 0x7f40a8105980 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.342+0000 7f409f7fe700 1 -- 172.21.15.18:0/639707235 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f40a401a900 con 0x7f40a8105980 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.342+0000 7f409f7fe700 1 --2- 172.21.15.18:0/639707235 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4094038f40 0x7f409403b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.342+0000 7f409f7fe700 1 -- 172.21.15.18:0/639707235 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f40a404d2f0 con 0x7f40a8105980 2024-02-17T12:40:53.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.342+0000 7f40b02c3700 1 -- 172.21.15.18:0/639707235 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a8068d50 con 0x7f40a8105980 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.342+0000 7f40ad85e700 1 --2- 172.21.15.18:0/639707235 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4094038f40 0x7f409403b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.343+0000 7f40ad85e700 1 --2- 172.21.15.18:0/639707235 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4094038f40 0x7f409403b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4098006fd0 tx=0x7f4098006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.347+0000 7f409f7fe700 1 -- 172.21.15.18:0/639707235 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f40a4022330 con 0x7f40a8105980 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.564+0000 7f40b02c3700 1 -- 172.21.15.18:0/639707235 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f40a8107300 con 0x7f4094038f40 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.675+0000 7f409f7fe700 1 -- 172.21.15.18:0/639707235 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f40a8107300 con 0x7f4094038f40 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.677+0000 7f40b02c3700 1 -- 172.21.15.18:0/639707235 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4094038f40 msgr2=0x7f409403b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.677+0000 7f40b02c3700 1 --2- 172.21.15.18:0/639707235 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4094038f40 0x7f409403b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4098006fd0 tx=0x7f4098006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.677+0000 7f40b02c3700 1 -- 172.21.15.18:0/639707235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 msgr2=0x7f40a8108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.677+0000 7f40b02c3700 1 --2- 172.21.15.18:0/639707235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 0x7f40a8108270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f40a4003f20 tx=0x7f40a4011b00 comp rx=0 tx=0).stop 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.677+0000 7f40b02c3700 1 -- 172.21.15.18:0/639707235 shutdown_connections 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.677+0000 7f40b02c3700 1 --2- 172.21.15.18:0/639707235 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4094038f40 0x7f409403b400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.677+0000 7f40b02c3700 1 --2- 172.21.15.18:0/639707235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40a8105980 0x7f40a8108270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.677+0000 7f40b02c3700 1 -- 172.21.15.18:0/639707235 >> 172.21.15.18:0/639707235 conn(0x7f40a8101430 msgr2=0x7f40a8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:53.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.677+0000 7f40b02c3700 1 -- 172.21.15.18:0/639707235 shutdown_connections 2024-02-17T12:40:53.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:52.677+0000 7f40b02c3700 1 -- 172.21.15.18:0/639707235 wait complete. 2024-02-17T12:40:53.181 INFO:teuthology.orchestra.run.smithi018.stdout:Creating initial admin user... 2024-02-17T12:40:53.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:53 smithi018 ceph-mon[32625]: mgrmap e12: x(active, since 2s) 2024-02-17T12:40:53.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:40:53.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:53 smithi018 ceph-mon[32625]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:40:53.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:40:53.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:40:54.701 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$u/14.0trmuSKTIl4JWTZaug4yTPz9B892AOogD9CJ3aLqx1XHTQmy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708173654, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-17T12:40:54.701 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.658+0000 7ff7e724f700 1 Processor -- start 2024-02-17T12:40:54.701 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.658+0000 7ff7e724f700 1 -- start start 2024-02-17T12:40:54.701 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.658+0000 7ff7e724f700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 0x7ff7e010a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:54.701 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.658+0000 7ff7e724f700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e0075560 con 0x7ff7e01081c0 2024-02-17T12:40:54.701 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.658+0000 7ff7e4feb700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 0x7ff7e010a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:54.701 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.658+0000 7ff7e4feb700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 0x7ff7e010a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60966/0 (socket says 172.21.15.18:60966) 2024-02-17T12:40:54.701 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.658+0000 7ff7e4feb700 1 -- 172.21.15.18:0/907511632 learned_addr learned my addr 172.21.15.18:0/907511632 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:54.701 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.659+0000 7ff7e4feb700 1 -- 172.21.15.18:0/907511632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e00756a0 con 0x7ff7e01081c0 2024-02-17T12:40:54.701 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.659+0000 7ff7e4feb700 1 --2- 172.21.15.18:0/907511632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 0x7ff7e010a5b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40098a0 tx=0x7ff7d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d69a93f7228c774 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:54.701 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.659+0000 7ff7df7fe700 1 -- 172.21.15.18:0/907511632 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7d4004160 con 0x7ff7e01081c0 2024-02-17T12:40:54.701 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.659+0000 7ff7df7fe700 1 -- 172.21.15.18:0/907511632 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7d40042c0 con 0x7ff7e01081c0 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.659+0000 7ff7df7fe700 1 -- 172.21.15.18:0/907511632 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7d4011a80 con 0x7ff7e01081c0 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.660+0000 7ff7e724f700 1 -- 172.21.15.18:0/907511632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 msgr2=0x7ff7e010a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.660+0000 7ff7e724f700 1 --2- 172.21.15.18:0/907511632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 0x7ff7e010a5b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40098a0 tx=0x7ff7d4009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.660+0000 7ff7e724f700 1 -- 172.21.15.18:0/907511632 shutdown_connections 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.660+0000 7ff7e724f700 1 --2- 172.21.15.18:0/907511632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 0x7ff7e010a5b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.660+0000 7ff7e724f700 1 -- 172.21.15.18:0/907511632 >> 172.21.15.18:0/907511632 conn(0x7ff7e0101b90 msgr2=0x7ff7e0103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.660+0000 7ff7e724f700 1 -- 172.21.15.18:0/907511632 shutdown_connections 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.660+0000 7ff7e724f700 1 -- 172.21.15.18:0/907511632 wait complete. 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.660+0000 7ff7e724f700 1 Processor -- start 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.660+0000 7ff7e724f700 1 -- start start 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.661+0000 7ff7e724f700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 0x7ff7e0107a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.661+0000 7ff7e724f700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e0107f80 con 0x7ff7e01081c0 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.661+0000 7ff7e4feb700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 0x7ff7e0107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:54.702 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.661+0000 7ff7e4feb700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 0x7ff7e0107a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60968/0 (socket says 172.21.15.18:60968) 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.661+0000 7ff7e4feb700 1 -- 172.21.15.18:0/676615873 learned_addr learned my addr 172.21.15.18:0/676615873 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.661+0000 7ff7e4feb700 1 -- 172.21.15.18:0/676615873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4009550 con 0x7ff7e01081c0 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.661+0000 7ff7e4feb700 1 --2- 172.21.15.18:0/676615873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 0x7ff7e0107a40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4011740 tx=0x7ff7d4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.661+0000 7ff7ddffb700 1 -- 172.21.15.18:0/676615873 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7d4003b20 con 0x7ff7e01081c0 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.661+0000 7ff7ddffb700 1 -- 172.21.15.18:0/676615873 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7d401a910 con 0x7ff7e01081c0 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.661+0000 7ff7ddffb700 1 -- 172.21.15.18:0/676615873 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7d4024900 con 0x7ff7e01081c0 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.661+0000 7ff7e724f700 1 -- 172.21.15.18:0/676615873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e0106170 con 0x7ff7e01081c0 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.662+0000 7ff7e724f700 1 -- 172.21.15.18:0/676615873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e0106610 con 0x7ff7e01081c0 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.662+0000 7ff7ddffb700 1 -- 172.21.15.18:0/676615873 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7ff7d4024a60 con 0x7ff7e01081c0 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.662+0000 7ff7e724f700 1 -- 172.21.15.18:0/676615873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e010b8c0 con 0x7ff7e01081c0 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.663+0000 7ff7ddffb700 1 --2- 172.21.15.18:0/676615873 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff7d0038f40 0x7ff7d003b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.663+0000 7ff7ddffb700 1 -- 172.21.15.18:0/676615873 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff7d404d440 con 0x7ff7e01081c0 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.663+0000 7ff7dffff700 1 --2- 172.21.15.18:0/676615873 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff7d0038f40 0x7ff7d003b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:54.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.663+0000 7ff7dffff700 1 --2- 172.21.15.18:0/676615873 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff7d0038f40 0x7ff7d003b400 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc006fd0 tx=0x7ff7cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.668+0000 7ff7ddffb700 1 -- 172.21.15.18:0/676615873 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7d4018a40 con 0x7ff7e01081c0 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:53.890+0000 7ff7e724f700 1 -- 172.21.15.18:0/676615873 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff7e00630c0 con 0x7ff7d0038f40 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:54.136+0000 7ff7ddffb700 1 -- 172.21.15.18:0/676615873 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7ff7e00630c0 con 0x7ff7d0038f40 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:54.139+0000 7ff7e724f700 1 -- 172.21.15.18:0/676615873 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff7d0038f40 msgr2=0x7ff7d003b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:54.139+0000 7ff7e724f700 1 --2- 172.21.15.18:0/676615873 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff7d0038f40 0x7ff7d003b400 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc006fd0 tx=0x7ff7cc006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:54.139+0000 7ff7e724f700 1 -- 172.21.15.18:0/676615873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 msgr2=0x7ff7e0107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:54.139+0000 7ff7e724f700 1 --2- 172.21.15.18:0/676615873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 0x7ff7e0107a40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4011740 tx=0x7ff7d4011a50 comp rx=0 tx=0).stop 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:54.139+0000 7ff7e724f700 1 -- 172.21.15.18:0/676615873 shutdown_connections 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:54.139+0000 7ff7e724f700 1 --2- 172.21.15.18:0/676615873 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff7d0038f40 0x7ff7d003b400 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:54.139+0000 7ff7e724f700 1 --2- 172.21.15.18:0/676615873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e01081c0 0x7ff7e0107a40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:54.139+0000 7ff7e724f700 1 -- 172.21.15.18:0/676615873 >> 172.21.15.18:0/676615873 conn(0x7ff7e0101b90 msgr2=0x7ff7e0102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:54.139+0000 7ff7e724f700 1 -- 172.21.15.18:0/676615873 shutdown_connections 2024-02-17T12:40:54.704 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:54.139+0000 7ff7e724f700 1 -- 172.21.15.18:0/676615873 wait complete. 2024-02-17T12:40:54.705 INFO:teuthology.orchestra.run.smithi018.stdout:Fetching dashboard port number... 2024-02-17T12:40:54.962 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:54 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:40:54.962 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:54 smithi018 ceph-mon[32625]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:40:54.962 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:54 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:40:55.879 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 8443 2024-02-17T12:40:55.879 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.068+0000 7f861e59f700 1 Processor -- start 2024-02-17T12:40:55.879 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.068+0000 7f861e59f700 1 -- start start 2024-02-17T12:40:55.879 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.068+0000 7f861e59f700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 0x7f861810a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:55.879 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.068+0000 7f861e59f700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8618075560 con 0x7f86181081d0 2024-02-17T12:40:55.879 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.068+0000 7f8617fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 0x7f861810a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:55.879 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.068+0000 7f8617fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 0x7f861810a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60970/0 (socket says 172.21.15.18:60970) 2024-02-17T12:40:55.880 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.068+0000 7f8617fff700 1 -- 172.21.15.18:0/2657078391 learned_addr learned my addr 172.21.15.18:0/2657078391 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:55.880 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.069+0000 7f8617fff700 1 -- 172.21.15.18:0/2657078391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180756a0 con 0x7f86181081d0 2024-02-17T12:40:55.880 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.069+0000 7f8617fff700 1 --2- 172.21.15.18:0/2657078391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 0x7f861810a5c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86080098a0 tx=0x7f8608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e305a9490547051b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:55.880 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.069+0000 7f8616ffd700 1 -- 172.21.15.18:0/2657078391 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8608004160 con 0x7f86181081d0 2024-02-17T12:40:55.880 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.069+0000 7f8616ffd700 1 -- 172.21.15.18:0/2657078391 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86080042c0 con 0x7f86181081d0 2024-02-17T12:40:55.880 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.069+0000 7f8616ffd700 1 -- 172.21.15.18:0/2657078391 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8608011a80 con 0x7f86181081d0 2024-02-17T12:40:55.880 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.069+0000 7f861e59f700 1 -- 172.21.15.18:0/2657078391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 msgr2=0x7f861810a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:55.880 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.069+0000 7f861e59f700 1 --2- 172.21.15.18:0/2657078391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 0x7f861810a5c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86080098a0 tx=0x7f8608009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:55.880 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.069+0000 7f861e59f700 1 -- 172.21.15.18:0/2657078391 shutdown_connections 2024-02-17T12:40:55.880 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.069+0000 7f861e59f700 1 --2- 172.21.15.18:0/2657078391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 0x7f861810a5c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:55.880 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.069+0000 7f861e59f700 1 -- 172.21.15.18:0/2657078391 >> 172.21.15.18:0/2657078391 conn(0x7f8618101ba0 msgr2=0x7f8618103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:55.880 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.070+0000 7f861e59f700 1 -- 172.21.15.18:0/2657078391 shutdown_connections 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.070+0000 7f861e59f700 1 -- 172.21.15.18:0/2657078391 wait complete. 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.070+0000 7f861e59f700 1 Processor -- start 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.070+0000 7f861e59f700 1 -- start start 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.070+0000 7f861e59f700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 0x7f8618107a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.070+0000 7f861e59f700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8618107f90 con 0x7f86181081d0 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.071+0000 7f8617fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 0x7f8618107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.071+0000 7f8617fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 0x7f8618107a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60982/0 (socket says 172.21.15.18:60982) 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.071+0000 7f8617fff700 1 -- 172.21.15.18:0/992565896 learned_addr learned my addr 172.21.15.18:0/992565896 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.071+0000 7f8617fff700 1 -- 172.21.15.18:0/992565896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8608009550 con 0x7f86181081d0 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.071+0000 7f8617fff700 1 --2- 172.21.15.18:0/992565896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 0x7f8618107a50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8608003f20 tx=0x7f8608011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.071+0000 7f86157fa700 1 -- 172.21.15.18:0/992565896 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86080039f0 con 0x7f86181081d0 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.071+0000 7f86157fa700 1 -- 172.21.15.18:0/992565896 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8608003b50 con 0x7f86181081d0 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.071+0000 7f86157fa700 1 -- 172.21.15.18:0/992565896 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f860801a6d0 con 0x7f86181081d0 2024-02-17T12:40:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.071+0000 7f861e59f700 1 -- 172.21.15.18:0/992565896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8618106180 con 0x7f86181081d0 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.071+0000 7f861e59f700 1 -- 172.21.15.18:0/992565896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8618106620 con 0x7f86181081d0 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.072+0000 7f86157fa700 1 -- 172.21.15.18:0/992565896 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f860801a940 con 0x7f86181081d0 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.072+0000 7f861e59f700 1 -- 172.21.15.18:0/992565896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861810b8d0 con 0x7f86181081d0 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.072+0000 7f86157fa700 1 --2- 172.21.15.18:0/992565896 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8600038f90 0x7f860003b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.072+0000 7f86157fa700 1 -- 172.21.15.18:0/992565896 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f860804d590 con 0x7f86181081d0 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.072+0000 7f86177fe700 1 --2- 172.21.15.18:0/992565896 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8600038f90 0x7f860003b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.073+0000 7f86177fe700 1 --2- 172.21.15.18:0/992565896 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8600038f90 0x7f860003b450 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f860c006fd0 tx=0x7f860c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.077+0000 7f86157fa700 1 -- 172.21.15.18:0/992565896 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8608018530 con 0x7f86181081d0 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.298+0000 7f861e59f700 1 -- 172.21.15.18:0/992565896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f861810bae0 con 0x7f86181081d0 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.299+0000 7f86157fa700 1 -- 172.21.15.18:0/992565896 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f8608018350 con 0x7f86181081d0 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.301+0000 7f861e59f700 1 -- 172.21.15.18:0/992565896 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8600038f90 msgr2=0x7f860003b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.301+0000 7f861e59f700 1 --2- 172.21.15.18:0/992565896 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8600038f90 0x7f860003b450 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f860c006fd0 tx=0x7f860c006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:55.882 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.301+0000 7f861e59f700 1 -- 172.21.15.18:0/992565896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 msgr2=0x7f8618107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:55.883 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.301+0000 7f861e59f700 1 --2- 172.21.15.18:0/992565896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 0x7f8618107a50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8608003f20 tx=0x7f8608011b00 comp rx=0 tx=0).stop 2024-02-17T12:40:55.883 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.301+0000 7f861e59f700 1 -- 172.21.15.18:0/992565896 shutdown_connections 2024-02-17T12:40:55.883 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.301+0000 7f861e59f700 1 --2- 172.21.15.18:0/992565896 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8600038f90 0x7f860003b450 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:55.883 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.301+0000 7f861e59f700 1 --2- 172.21.15.18:0/992565896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86181081d0 0x7f8618107a50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:55.883 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.301+0000 7f861e59f700 1 -- 172.21.15.18:0/992565896 >> 172.21.15.18:0/992565896 conn(0x7f8618101ba0 msgr2=0x7f8618102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:55.883 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.301+0000 7f861e59f700 1 -- 172.21.15.18:0/992565896 shutdown_connections 2024-02-17T12:40:55.883 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:55.301+0000 7f861e59f700 1 -- 172.21.15.18:0/992565896 wait complete. 2024-02-17T12:40:55.885 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-17T12:40:55.885 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-02-17T12:40:55.892 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-17T12:40:55.892 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-02-17T12:40:55.892 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-02-17T12:40:55.892 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-17T12:40:55.893 INFO:teuthology.orchestra.run.smithi018.stdout:Ceph Dashboard is now available at: 2024-02-17T12:40:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:40:55.894 INFO:teuthology.orchestra.run.smithi018.stdout: URL: https://smithi018.front.sepia.ceph.com:8443/ 2024-02-17T12:40:55.894 INFO:teuthology.orchestra.run.smithi018.stdout: User: admin 2024-02-17T12:40:55.894 INFO:teuthology.orchestra.run.smithi018.stdout: Password: nbq0ccxjgk 2024-02-17T12:40:55.894 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:40:55.894 INFO:teuthology.orchestra.run.smithi018.stdout:Saving cluster configuration to /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config directory 2024-02-17T12:40:55.894 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling autotune for osd_memory_target 2024-02-17T12:40:56.012 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:55 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/992565896' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-17T12:40:57.897 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:57 smithi018 ceph-mon[32625]: mgrmap e13: x(active, since 6s) 2024-02-17T12:40:57.905 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.103+0000 7f7cd9356700 1 Processor -- start 2024-02-17T12:40:57.905 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.103+0000 7f7cd9356700 1 -- start start 2024-02-17T12:40:57.905 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.104+0000 7f7cd9356700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 0x7f7cd407b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.104+0000 7f7cd9356700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd407b8f0 con 0x7f7cd407ce70 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.104+0000 7f7cd2ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 0x7f7cd407b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.104+0000 7f7cd2ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 0x7f7cd407b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60994/0 (socket says 172.21.15.18:60994) 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.104+0000 7f7cd2ffd700 1 -- 172.21.15.18:0/718784602 learned_addr learned my addr 172.21.15.18:0/718784602 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.104+0000 7f7cd2ffd700 1 -- 172.21.15.18:0/718784602 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd407ba30 con 0x7f7cd407ce70 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.104+0000 7f7cd2ffd700 1 --2- 172.21.15.18:0/718784602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 0x7f7cd407b320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40098a0 tx=0x7f7cc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee857c58413ac03c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.104+0000 7f7cd1ffb700 1 -- 172.21.15.18:0/718784602 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc4004160 con 0x7f7cd407ce70 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.104+0000 7f7cd1ffb700 1 -- 172.21.15.18:0/718784602 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cc40042c0 con 0x7f7cd407ce70 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.104+0000 7f7cd1ffb700 1 -- 172.21.15.18:0/718784602 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc4011a80 con 0x7f7cd407ce70 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.105+0000 7f7cd9356700 1 -- 172.21.15.18:0/718784602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 msgr2=0x7f7cd407b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.105+0000 7f7cd9356700 1 --2- 172.21.15.18:0/718784602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 0x7f7cd407b320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40098a0 tx=0x7f7cc4009bb0 comp rx=0 tx=0).stop 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.105+0000 7f7cd9356700 1 -- 172.21.15.18:0/718784602 shutdown_connections 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.105+0000 7f7cd9356700 1 --2- 172.21.15.18:0/718784602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 0x7f7cd407b320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:57.906 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.105+0000 7f7cd9356700 1 -- 172.21.15.18:0/718784602 >> 172.21.15.18:0/718784602 conn(0x7f7cd4102d80 msgr2=0x7f7cd41051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.105+0000 7f7cd9356700 1 -- 172.21.15.18:0/718784602 shutdown_connections 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.105+0000 7f7cd9356700 1 -- 172.21.15.18:0/718784602 wait complete. 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.106+0000 7f7cd9356700 1 Processor -- start 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.106+0000 7f7cd9356700 1 -- start start 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.106+0000 7f7cd9356700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 0x7f7cd419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.106+0000 7f7cd9356700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd419db40 con 0x7f7cd407ce70 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.106+0000 7f7cd2ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 0x7f7cd419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.106+0000 7f7cd2ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 0x7f7cd419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32772/0 (socket says 172.21.15.18:32772) 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.106+0000 7f7cd2ffd700 1 -- 172.21.15.18:0/439298309 learned_addr learned my addr 172.21.15.18:0/439298309 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.106+0000 7f7cd2ffd700 1 -- 172.21.15.18:0/439298309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc4009550 con 0x7f7cd407ce70 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.107+0000 7f7cd2ffd700 1 --2- 172.21.15.18:0/439298309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 0x7f7cd419d600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009130 tx=0x7f7cc4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.107+0000 7f7cbbfff700 1 -- 172.21.15.18:0/439298309 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc4003aa0 con 0x7f7cd407ce70 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.107+0000 7f7cbbfff700 1 -- 172.21.15.18:0/439298309 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cc4003c00 con 0x7f7cd407ce70 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.107+0000 7f7cd9356700 1 -- 172.21.15.18:0/439298309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd419dd40 con 0x7f7cd407ce70 2024-02-17T12:40:57.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.107+0000 7f7cbbfff700 1 -- 172.21.15.18:0/439298309 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc40248a0 con 0x7f7cd407ce70 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.107+0000 7f7cd9356700 1 -- 172.21.15.18:0/439298309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd419e1e0 con 0x7f7cd407ce70 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.108+0000 7f7cbbfff700 1 -- 172.21.15.18:0/439298309 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f7cc401ac10 con 0x7f7cd407ce70 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.108+0000 7f7cd9356700 1 -- 172.21.15.18:0/439298309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd410dac0 con 0x7f7cd407ce70 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.108+0000 7f7cbbfff700 1 --2- 172.21.15.18:0/439298309 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f7cbc038fd0 0x7f7cbc03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.108+0000 7f7cbbfff700 1 -- 172.21.15.18:0/439298309 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7cc404d370 con 0x7f7cd407ce70 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.108+0000 7f7cd27fc700 1 --2- 172.21.15.18:0/439298309 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f7cbc038fd0 0x7f7cbc03b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.109+0000 7f7cd27fc700 1 --2- 172.21.15.18:0/439298309 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f7cbc038fd0 0x7f7cbc03b490 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8006fd0 tx=0x7f7cc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.113+0000 7f7cbbfff700 1 -- 172.21.15.18:0/439298309 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7cc4018760 con 0x7f7cd407ce70 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.313+0000 7f7cd9356700 1 -- 172.21.15.18:0/439298309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f7cd41a0ac0 con 0x7f7cd407ce70 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.313+0000 7f7cbbfff700 1 -- 172.21.15.18:0/439298309 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f7cc400b3e0 con 0x7f7cd407ce70 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.315+0000 7f7cd9356700 1 -- 172.21.15.18:0/439298309 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f7cbc038fd0 msgr2=0x7f7cbc03b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.315+0000 7f7cd9356700 1 --2- 172.21.15.18:0/439298309 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f7cbc038fd0 0x7f7cbc03b490 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8006fd0 tx=0x7f7cc8006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:57.908 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.315+0000 7f7cd9356700 1 -- 172.21.15.18:0/439298309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 msgr2=0x7f7cd419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:57.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.315+0000 7f7cd9356700 1 --2- 172.21.15.18:0/439298309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 0x7f7cd419d600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009130 tx=0x7f7cc4011b00 comp rx=0 tx=0).stop 2024-02-17T12:40:57.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.315+0000 7f7cd9356700 1 -- 172.21.15.18:0/439298309 shutdown_connections 2024-02-17T12:40:57.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.315+0000 7f7cd9356700 1 --2- 172.21.15.18:0/439298309 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f7cbc038fd0 0x7f7cbc03b490 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:57.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.315+0000 7f7cd9356700 1 --2- 172.21.15.18:0/439298309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cd407ce70 0x7f7cd419d600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:57.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.315+0000 7f7cd9356700 1 -- 172.21.15.18:0/439298309 >> 172.21.15.18:0/439298309 conn(0x7f7cd4102d80 msgr2=0x7f7cd4103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:57.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.316+0000 7f7cd9356700 1 -- 172.21.15.18:0/439298309 shutdown_connections 2024-02-17T12:40:57.909 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:57.316+0000 7f7cd9356700 1 -- 172.21.15.18:0/439298309 wait complete. 2024-02-17T12:40:59.147 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.354+0000 7f9a6f3ad700 1 Processor -- start 2024-02-17T12:40:59.147 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.354+0000 7f9a6f3ad700 1 -- start start 2024-02-17T12:40:59.147 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.354+0000 7f9a6f3ad700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 0x7f9a68107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.354+0000 7f9a6f3ad700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a68108150 con 0x7f9a68107760 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.355+0000 7f9a6d149700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 0x7f9a68107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.355+0000 7f9a6d149700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 0x7f9a68107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32776/0 (socket says 172.21.15.18:32776) 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.355+0000 7f9a6d149700 1 -- 172.21.15.18:0/2584038535 learned_addr learned my addr 172.21.15.18:0/2584038535 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.355+0000 7f9a6d149700 1 -- 172.21.15.18:0/2584038535 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a681088c0 con 0x7f9a68107760 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.355+0000 7f9a6d149700 1 --2- 172.21.15.18:0/2584038535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 0x7f9a68107b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009e80 tx=0x7f9a6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abbd6921919bd65 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.355+0000 7f9a67fff700 1 -- 172.21.15.18:0/2584038535 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a6000e070 con 0x7f9a68107760 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.355+0000 7f9a67fff700 1 -- 172.21.15.18:0/2584038535 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a6000b8c0 con 0x7f9a68107760 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.355+0000 7f9a67fff700 1 -- 172.21.15.18:0/2584038535 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a600044a0 con 0x7f9a68107760 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.356+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/2584038535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 msgr2=0x7f9a68107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.356+0000 7f9a6f3ad700 1 --2- 172.21.15.18:0/2584038535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 0x7f9a68107b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009e80 tx=0x7f9a6000b330 comp rx=0 tx=0).stop 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.356+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/2584038535 shutdown_connections 2024-02-17T12:40:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.356+0000 7f9a6f3ad700 1 --2- 172.21.15.18:0/2584038535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 0x7f9a68107b80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.356+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/2584038535 >> 172.21.15.18:0/2584038535 conn(0x7f9a68102de0 msgr2=0x7f9a68105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.356+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/2584038535 shutdown_connections 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.356+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/2584038535 wait complete. 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.357+0000 7f9a6f3ad700 1 Processor -- start 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.357+0000 7f9a6f3ad700 1 -- start start 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.357+0000 7f9a6f3ad700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 0x7f9a6819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.357+0000 7f9a6f3ad700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6819b8d0 con 0x7f9a68107760 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.357+0000 7f9a6d149700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 0x7f9a6819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.357+0000 7f9a6d149700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 0x7f9a6819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32790/0 (socket says 172.21.15.18:32790) 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.357+0000 7f9a6d149700 1 -- 172.21.15.18:0/569796345 learned_addr learned my addr 172.21.15.18:0/569796345 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.357+0000 7f9a6d149700 1 -- 172.21.15.18:0/569796345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a60009b30 con 0x7f9a68107760 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.357+0000 7f9a6d149700 1 --2- 172.21.15.18:0/569796345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 0x7f9a6819b390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009fd0 tx=0x7f9a6000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.357+0000 7f9a667fc700 1 -- 172.21.15.18:0/569796345 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a6000e040 con 0x7f9a68107760 2024-02-17T12:40:59.149 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.358+0000 7f9a667fc700 1 -- 172.21.15.18:0/569796345 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a60003e20 con 0x7f9a68107760 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.358+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/569796345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6819bad0 con 0x7f9a68107760 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.358+0000 7f9a667fc700 1 -- 172.21.15.18:0/569796345 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a6001c6f0 con 0x7f9a68107760 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.358+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/569796345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6819bfd0 con 0x7f9a68107760 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.358+0000 7f9a667fc700 1 -- 172.21.15.18:0/569796345 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f9a6001c960 con 0x7f9a68107760 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.359+0000 7f9a667fc700 1 --2- 172.21.15.18:0/569796345 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9a50038fd0 0x7f9a5003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.359+0000 7f9a667fc700 1 -- 172.21.15.18:0/569796345 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9a6004d140 con 0x7f9a68107760 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.359+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/569796345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6810baf0 con 0x7f9a68107760 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.359+0000 7f9a6c948700 1 --2- 172.21.15.18:0/569796345 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9a50038fd0 0x7f9a5003b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.359+0000 7f9a6c948700 1 --2- 172.21.15.18:0/569796345 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9a50038fd0 0x7f9a5003b490 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c006fd0 tx=0x7f9a5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.364+0000 7f9a667fc700 1 -- 172.21.15.18:0/569796345 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a60017070 con 0x7f9a68107760 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.678+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/569796345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f9a6819ea90 con 0x7f9a68107760 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.680+0000 7f9a667fc700 1 -- 172.21.15.18:0/569796345 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f9a60020030 con 0x7f9a68107760 2024-02-17T12:40:59.150 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.682+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/569796345 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9a50038fd0 msgr2=0x7f9a5003b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.682+0000 7f9a6f3ad700 1 --2- 172.21.15.18:0/569796345 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9a50038fd0 0x7f9a5003b490 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c006fd0 tx=0x7f9a5c006e40 comp rx=0 tx=0).stop 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.682+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/569796345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 msgr2=0x7f9a6819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.682+0000 7f9a6f3ad700 1 --2- 172.21.15.18:0/569796345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 0x7f9a6819b390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009fd0 tx=0x7f9a6000bcf0 comp rx=0 tx=0).stop 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.682+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/569796345 shutdown_connections 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.682+0000 7f9a6f3ad700 1 --2- 172.21.15.18:0/569796345 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9a50038fd0 0x7f9a5003b490 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.682+0000 7f9a6f3ad700 1 --2- 172.21.15.18:0/569796345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a68107760 0x7f9a6819b390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.682+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/569796345 >> 172.21.15.18:0/569796345 conn(0x7f9a68102de0 msgr2=0x7f9a68103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.682+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/569796345 shutdown_connections 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-02-17T12:40:58.682+0000 7f9a6f3ad700 1 -- 172.21.15.18:0/569796345 wait complete. 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-17T12:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout:Or, if you are only running a single cluster on this host: 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout: ceph telemetry on 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout:For more information see: 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:40:59.152 INFO:teuthology.orchestra.run.smithi018.stdout:Bootstrap complete. 2024-02-17T12:40:59.183 INFO:tasks.cephadm:Fetching config... 2024-02-17T12:40:59.183 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:40:59.184 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-17T12:40:59.199 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-17T12:40:59.199 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:40:59.199 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-17T12:40:59.254 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-17T12:40:59.254 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:40:59.254 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-17T12:40:59.321 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-17T12:40:59.321 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:40:59.322 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-17T12:40:59.376 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-17T12:40:59.377 DEBUG:teuthology.orchestra.run.smithi018:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6wbvMwhwV/V6fyysXGUhkixH+DgA8ZH4x+AxNE4N3m5tcifNSvoxgqYHIZA/hFe0hRMgetzSNNniPNWfcaqEtJL3pOjufnPTeGjbUP1j2WWkI6gd1U0YnYvCSxfd6Pz3vls6upQ0ImodTn5Uste0J8d0WarWTw0oqIq5RSULiFh8Dnlbe4ejUUDeMFMy91rX1qdnHkvgfoubbm6g4803wyxzb05u9MQzhLHVay7rv2eu6GE2fQXOjAq4gXUa2LViMLvdYg/ZvtqgPZFQbRcy5c33bneTFEImpCbyO4/MEiDAJ+VjK9WaA/AiPbcAxjJC6g/2DaTCxv04MkWUTw52t+Uc5+lvYtthiKY3Y6dCR0aYXZW2+qf1S5V4h/7IbkxtBkAki9c52nMCvPLVeXe13FNvDAVshgAF5yV5ktQCTVY6tjjr9gkZPiDa76ObbpyMr0HNPqIr6zSz6+Q5SYOSoFDMSdwXrPa6l4rV/lVaje2gPcABFp8ttVvZ8OpTmoK8= ceph-7fca71fe-cd91-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-17T12:40:59.455 INFO:teuthology.orchestra.run.smithi018.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6wbvMwhwV/V6fyysXGUhkixH+DgA8ZH4x+AxNE4N3m5tcifNSvoxgqYHIZA/hFe0hRMgetzSNNniPNWfcaqEtJL3pOjufnPTeGjbUP1j2WWkI6gd1U0YnYvCSxfd6Pz3vls6upQ0ImodTn5Uste0J8d0WarWTw0oqIq5RSULiFh8Dnlbe4ejUUDeMFMy91rX1qdnHkvgfoubbm6g4803wyxzb05u9MQzhLHVay7rv2eu6GE2fQXOjAq4gXUa2LViMLvdYg/ZvtqgPZFQbRcy5c33bneTFEImpCbyO4/MEiDAJ+VjK9WaA/AiPbcAxjJC6g/2DaTCxv04MkWUTw52t+Uc5+lvYtthiKY3Y6dCR0aYXZW2+qf1S5V4h/7IbkxtBkAki9c52nMCvPLVeXe13FNvDAVshgAF5yV5ktQCTVY6tjjr9gkZPiDa76ObbpyMr0HNPqIr6zSz6+Q5SYOSoFDMSdwXrPa6l4rV/lVaje2gPcABFp8ttVvZ8OpTmoK8= ceph-7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:40:59.469 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6wbvMwhwV/V6fyysXGUhkixH+DgA8ZH4x+AxNE4N3m5tcifNSvoxgqYHIZA/hFe0hRMgetzSNNniPNWfcaqEtJL3pOjufnPTeGjbUP1j2WWkI6gd1U0YnYvCSxfd6Pz3vls6upQ0ImodTn5Uste0J8d0WarWTw0oqIq5RSULiFh8Dnlbe4ejUUDeMFMy91rX1qdnHkvgfoubbm6g4803wyxzb05u9MQzhLHVay7rv2eu6GE2fQXOjAq4gXUa2LViMLvdYg/ZvtqgPZFQbRcy5c33bneTFEImpCbyO4/MEiDAJ+VjK9WaA/AiPbcAxjJC6g/2DaTCxv04MkWUTw52t+Uc5+lvYtthiKY3Y6dCR0aYXZW2+qf1S5V4h/7IbkxtBkAki9c52nMCvPLVeXe13FNvDAVshgAF5yV5ktQCTVY6tjjr9gkZPiDa76ObbpyMr0HNPqIr6zSz6+Q5SYOSoFDMSdwXrPa6l4rV/lVaje2gPcABFp8ttVvZ8OpTmoK8= ceph-7fca71fe-cd91-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-17T12:40:59.517 INFO:teuthology.orchestra.run.smithi119.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6wbvMwhwV/V6fyysXGUhkixH+DgA8ZH4x+AxNE4N3m5tcifNSvoxgqYHIZA/hFe0hRMgetzSNNniPNWfcaqEtJL3pOjufnPTeGjbUP1j2WWkI6gd1U0YnYvCSxfd6Pz3vls6upQ0ImodTn5Uste0J8d0WarWTw0oqIq5RSULiFh8Dnlbe4ejUUDeMFMy91rX1qdnHkvgfoubbm6g4803wyxzb05u9MQzhLHVay7rv2eu6GE2fQXOjAq4gXUa2LViMLvdYg/ZvtqgPZFQbRcy5c33bneTFEImpCbyO4/MEiDAJ+VjK9WaA/AiPbcAxjJC6g/2DaTCxv04MkWUTw52t+Uc5+lvYtthiKY3Y6dCR0aYXZW2+qf1S5V4h/7IbkxtBkAki9c52nMCvPLVeXe13FNvDAVshgAF5yV5ktQCTVY6tjjr9gkZPiDa76ObbpyMr0HNPqIr6zSz6+Q5SYOSoFDMSdwXrPa6l4rV/lVaje2gPcABFp8ttVvZ8OpTmoK8= ceph-7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:40:59.532 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6wbvMwhwV/V6fyysXGUhkixH+DgA8ZH4x+AxNE4N3m5tcifNSvoxgqYHIZA/hFe0hRMgetzSNNniPNWfcaqEtJL3pOjufnPTeGjbUP1j2WWkI6gd1U0YnYvCSxfd6Pz3vls6upQ0ImodTn5Uste0J8d0WarWTw0oqIq5RSULiFh8Dnlbe4ejUUDeMFMy91rX1qdnHkvgfoubbm6g4803wyxzb05u9MQzhLHVay7rv2eu6GE2fQXOjAq4gXUa2LViMLvdYg/ZvtqgPZFQbRcy5c33bneTFEImpCbyO4/MEiDAJ+VjK9WaA/AiPbcAxjJC6g/2DaTCxv04MkWUTw52t+Uc5+lvYtthiKY3Y6dCR0aYXZW2+qf1S5V4h/7IbkxtBkAki9c52nMCvPLVeXe13FNvDAVshgAF5yV5ktQCTVY6tjjr9gkZPiDa76ObbpyMr0HNPqIr6zSz6+Q5SYOSoFDMSdwXrPa6l4rV/lVaje2gPcABFp8ttVvZ8OpTmoK8= ceph-7fca71fe-cd91-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-17T12:40:59.578 INFO:teuthology.orchestra.run.smithi146.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6wbvMwhwV/V6fyysXGUhkixH+DgA8ZH4x+AxNE4N3m5tcifNSvoxgqYHIZA/hFe0hRMgetzSNNniPNWfcaqEtJL3pOjufnPTeGjbUP1j2WWkI6gd1U0YnYvCSxfd6Pz3vls6upQ0ImodTn5Uste0J8d0WarWTw0oqIq5RSULiFh8Dnlbe4ejUUDeMFMy91rX1qdnHkvgfoubbm6g4803wyxzb05u9MQzhLHVay7rv2eu6GE2fQXOjAq4gXUa2LViMLvdYg/ZvtqgPZFQbRcy5c33bneTFEImpCbyO4/MEiDAJ+VjK9WaA/AiPbcAxjJC6g/2DaTCxv04MkWUTw52t+Uc5+lvYtthiKY3Y6dCR0aYXZW2+qf1S5V4h/7IbkxtBkAki9c52nMCvPLVeXe13FNvDAVshgAF5yV5ktQCTVY6tjjr9gkZPiDa76ObbpyMr0HNPqIr6zSz6+Q5SYOSoFDMSdwXrPa6l4rV/lVaje2gPcABFp8ttVvZ8OpTmoK8= ceph-7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:40:59.593 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-17T12:40:59.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:40:59 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/569796345' entity='client.admin' 2024-02-17T12:40:59.835 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:41:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.353+0000 7f9e43507700 1 -- 172.21.15.18:0/4026946667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c1021a0 msgr2=0x7f9e3c104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.353+0000 7f9e43507700 1 --2- 172.21.15.18:0/4026946667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c1021a0 0x7f9e3c104590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009910 tx=0x7f9e38009c20 comp rx=0 tx=0).stop 2024-02-17T12:41:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.353+0000 7f9e43507700 1 -- 172.21.15.18:0/4026946667 shutdown_connections 2024-02-17T12:41:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.353+0000 7f9e43507700 1 --2- 172.21.15.18:0/4026946667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c1021a0 0x7f9e3c104590 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:01.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.353+0000 7f9e43507700 1 -- 172.21.15.18:0/4026946667 >> 172.21.15.18:0/4026946667 conn(0x7f9e3c0fbb70 msgr2=0x7f9e3c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:01.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.353+0000 7f9e43507700 1 -- 172.21.15.18:0/4026946667 shutdown_connections 2024-02-17T12:41:01.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.353+0000 7f9e43507700 1 -- 172.21.15.18:0/4026946667 wait complete. 2024-02-17T12:41:01.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.354+0000 7f9e43507700 1 Processor -- start 2024-02-17T12:41:01.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.354+0000 7f9e43507700 1 -- start start 2024-02-17T12:41:01.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.354+0000 7f9e43507700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c1021a0 0x7f9e3c101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:01.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.354+0000 7f9e43507700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c101f60 con 0x7f9e3c1021a0 2024-02-17T12:41:01.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.354+0000 7f9e412a3700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c1021a0 0x7f9e3c101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:01.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.354+0000 7f9e412a3700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c1021a0 0x7f9e3c101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58722/0 (socket says 172.21.15.18:58722) 2024-02-17T12:41:01.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.354+0000 7f9e412a3700 1 -- 172.21.15.18:0/1580349548 learned_addr learned my addr 172.21.15.18:0/1580349548 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:41:01.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.354+0000 7f9e412a3700 1 -- 172.21.15.18:0/1580349548 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e380095f0 con 0x7f9e3c1021a0 2024-02-17T12:41:01.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.355+0000 7f9e412a3700 1 --2- 172.21.15.18:0/1580349548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c1021a0 0x7f9e3c101a20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9e38005950 tx=0x7f9e38004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.355+0000 7f9e327fc700 1 -- 172.21.15.18:0/1580349548 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e3801d070 con 0x7f9e3c1021a0 2024-02-17T12:41:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.355+0000 7f9e43507700 1 -- 172.21.15.18:0/1580349548 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c100150 con 0x7f9e3c1021a0 2024-02-17T12:41:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.355+0000 7f9e327fc700 1 -- 172.21.15.18:0/1580349548 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e3800bc30 con 0x7f9e3c1021a0 2024-02-17T12:41:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.355+0000 7f9e327fc700 1 -- 172.21.15.18:0/1580349548 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e38003ac0 con 0x7f9e3c1021a0 2024-02-17T12:41:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.355+0000 7f9e43507700 1 -- 172.21.15.18:0/1580349548 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c1005f0 con 0x7f9e3c1021a0 2024-02-17T12:41:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.356+0000 7f9e327fc700 1 -- 172.21.15.18:0/1580349548 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f9e3800b6f0 con 0x7f9e3c1021a0 2024-02-17T12:41:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.356+0000 7f9e43507700 1 -- 172.21.15.18:0/1580349548 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3c107e40 con 0x7f9e3c1021a0 2024-02-17T12:41:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.356+0000 7f9e327fc700 1 --2- 172.21.15.18:0/1580349548 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9e28038f80 0x7f9e2803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.356+0000 7f9e327fc700 1 -- 172.21.15.18:0/1580349548 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9e3804ceb0 con 0x7f9e3c1021a0 2024-02-17T12:41:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.356+0000 7f9e40aa2700 1 --2- 172.21.15.18:0/1580349548 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9e28038f80 0x7f9e2803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.356+0000 7f9e40aa2700 1 --2- 172.21.15.18:0/1580349548 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9e28038f80 0x7f9e2803b440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c006fd0 tx=0x7f9e2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:01.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.361+0000 7f9e327fc700 1 -- 172.21.15.18:0/1580349548 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9e3802ace0 con 0x7f9e3c1021a0 2024-02-17T12:41:01.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.564+0000 7f9e43507700 1 -- 172.21.15.18:0/1580349548 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f9e3c0632a0 con 0x7f9e3c1021a0 2024-02-17T12:41:01.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.566+0000 7f9e327fc700 1 -- 172.21.15.18:0/1580349548 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9e38026070 con 0x7f9e3c1021a0 2024-02-17T12:41:01.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.568+0000 7f9e43507700 1 -- 172.21.15.18:0/1580349548 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9e28038f80 msgr2=0x7f9e2803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:01.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.568+0000 7f9e43507700 1 --2- 172.21.15.18:0/1580349548 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9e28038f80 0x7f9e2803b440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c006fd0 tx=0x7f9e2c006e40 comp rx=0 tx=0).stop 2024-02-17T12:41:01.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.568+0000 7f9e43507700 1 -- 172.21.15.18:0/1580349548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c1021a0 msgr2=0x7f9e3c101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:01.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.568+0000 7f9e43507700 1 --2- 172.21.15.18:0/1580349548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c1021a0 0x7f9e3c101a20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9e38005950 tx=0x7f9e38004ff0 comp rx=0 tx=0).stop 2024-02-17T12:41:01.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.568+0000 7f9e43507700 1 -- 172.21.15.18:0/1580349548 shutdown_connections 2024-02-17T12:41:01.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.568+0000 7f9e43507700 1 --2- 172.21.15.18:0/1580349548 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9e28038f80 0x7f9e2803b440 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:01.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.568+0000 7f9e43507700 1 --2- 172.21.15.18:0/1580349548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c1021a0 0x7f9e3c101a20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:01.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.568+0000 7f9e43507700 1 -- 172.21.15.18:0/1580349548 >> 172.21.15.18:0/1580349548 conn(0x7f9e3c0fbb70 msgr2=0x7f9e3c0fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:01.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.568+0000 7f9e43507700 1 -- 172.21.15.18:0/1580349548 shutdown_connections 2024-02-17T12:41:01.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:01.568+0000 7f9e43507700 1 -- 172.21.15.18:0/1580349548 wait complete. 2024-02-17T12:41:02.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:02 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1580349548' entity='client.admin' 2024-02-17T12:41:02.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:02.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:02.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi018", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:41:02.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:02.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:02.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:02.897 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-17T12:41:02.897 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-17T12:41:03.403 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:41:05.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.155+0000 7f32f2876700 1 -- 172.21.15.18:0/3487406721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ec1016c0 msgr2=0x7f32ec101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:05.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.155+0000 7f32f2876700 1 --2- 172.21.15.18:0/3487406721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ec1016c0 0x7f32ec101ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009910 tx=0x7f32dc009c20 comp rx=0 tx=0).stop 2024-02-17T12:41:05.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.155+0000 7f32f2876700 1 -- 172.21.15.18:0/3487406721 shutdown_connections 2024-02-17T12:41:05.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.155+0000 7f32f2876700 1 --2- 172.21.15.18:0/3487406721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ec1016c0 0x7f32ec101ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:05.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.155+0000 7f32f2876700 1 -- 172.21.15.18:0/3487406721 >> 172.21.15.18:0/3487406721 conn(0x7f32ec0fcc60 msgr2=0x7f32ec0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:05.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.155+0000 7f32f2876700 1 -- 172.21.15.18:0/3487406721 shutdown_connections 2024-02-17T12:41:05.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.155+0000 7f32f2876700 1 -- 172.21.15.18:0/3487406721 wait complete. 2024-02-17T12:41:05.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.156+0000 7f32f2876700 1 Processor -- start 2024-02-17T12:41:05.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.156+0000 7f32f2876700 1 -- start start 2024-02-17T12:41:05.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.156+0000 7f32f2876700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ec1016c0 0x7f32ec196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:05.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.156+0000 7f32f2876700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec1970f0 con 0x7f32ec1016c0 2024-02-17T12:41:05.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.156+0000 7f32ebfff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ec1016c0 0x7f32ec196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:05.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.156+0000 7f32ebfff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ec1016c0 0x7f32ec196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58734/0 (socket says 172.21.15.18:58734) 2024-02-17T12:41:05.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.156+0000 7f32ebfff700 1 -- 172.21.15.18:0/2595725976 learned_addr learned my addr 172.21.15.18:0/2595725976 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:41:05.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.156+0000 7f32ebfff700 1 -- 172.21.15.18:0/2595725976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc0095f0 con 0x7f32ec1016c0 2024-02-17T12:41:05.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.156+0000 7f32ebfff700 1 --2- 172.21.15.18:0/2595725976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ec1016c0 0x7f32ec196bb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f32dc004f40 tx=0x7f32dc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:05.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.156+0000 7f32e9ffb700 1 -- 172.21.15.18:0/2595725976 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32dc01d070 con 0x7f32ec1016c0 2024-02-17T12:41:05.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.156+0000 7f32e9ffb700 1 -- 172.21.15.18:0/2595725976 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32dc0044a0 con 0x7f32ec1016c0 2024-02-17T12:41:05.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.156+0000 7f32f2876700 1 -- 172.21.15.18:0/2595725976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ec1972f0 con 0x7f32ec1016c0 2024-02-17T12:41:05.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.157+0000 7f32e9ffb700 1 -- 172.21.15.18:0/2595725976 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32dc003a40 con 0x7f32ec1016c0 2024-02-17T12:41:05.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.157+0000 7f32f2876700 1 -- 172.21.15.18:0/2595725976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ec197790 con 0x7f32ec1016c0 2024-02-17T12:41:05.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.158+0000 7f32e9ffb700 1 -- 172.21.15.18:0/2595725976 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f32dc003cb0 con 0x7f32ec1016c0 2024-02-17T12:41:05.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.158+0000 7f32f2876700 1 -- 172.21.15.18:0/2595725976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ec190520 con 0x7f32ec1016c0 2024-02-17T12:41:05.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.158+0000 7f32e9ffb700 1 --2- 172.21.15.18:0/2595725976 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f32d8038fe0 0x7f32d803b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:05.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.158+0000 7f32e9ffb700 1 -- 172.21.15.18:0/2595725976 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f32dc04ce10 con 0x7f32ec1016c0 2024-02-17T12:41:05.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.158+0000 7f32e3fff700 1 --2- 172.21.15.18:0/2595725976 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f32d8038fe0 0x7f32d803b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:05.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.158+0000 7f32e3fff700 1 --2- 172.21.15.18:0/2595725976 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f32d8038fe0 0x7f32d803b4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f32d4006fd0 tx=0x7f32d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:05.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.163+0000 7f32e9ffb700 1 -- 172.21.15.18:0/2595725976 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32dc029360 con 0x7f32ec1016c0 2024-02-17T12:41:05.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.369+0000 7f32f2876700 1 -- 172.21.15.18:0/2595725976 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f32ec0620b0 con 0x7f32d8038fe0 2024-02-17T12:41:05.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.372+0000 7f32e9ffb700 1 -- 172.21.15.18:0/2595725976 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f32ec0620b0 con 0x7f32d8038fe0 2024-02-17T12:41:05.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.374+0000 7f32f2876700 1 -- 172.21.15.18:0/2595725976 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f32d8038fe0 msgr2=0x7f32d803b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:05.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.374+0000 7f32f2876700 1 --2- 172.21.15.18:0/2595725976 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f32d8038fe0 0x7f32d803b4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f32d4006fd0 tx=0x7f32d4006e40 comp rx=0 tx=0).stop 2024-02-17T12:41:05.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.374+0000 7f32f2876700 1 -- 172.21.15.18:0/2595725976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ec1016c0 msgr2=0x7f32ec196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:05.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.374+0000 7f32f2876700 1 --2- 172.21.15.18:0/2595725976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ec1016c0 0x7f32ec196bb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f32dc004f40 tx=0x7f32dc005020 comp rx=0 tx=0).stop 2024-02-17T12:41:05.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.374+0000 7f32f2876700 1 -- 172.21.15.18:0/2595725976 shutdown_connections 2024-02-17T12:41:05.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.374+0000 7f32f2876700 1 --2- 172.21.15.18:0/2595725976 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f32d8038fe0 0x7f32d803b4a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:05.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.375+0000 7f32f2876700 1 --2- 172.21.15.18:0/2595725976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ec1016c0 0x7f32ec196bb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:05.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.375+0000 7f32f2876700 1 -- 172.21.15.18:0/2595725976 >> 172.21.15.18:0/2595725976 conn(0x7f32ec0fcc60 msgr2=0x7f32ec0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:05.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.375+0000 7f32f2876700 1 -- 172.21.15.18:0/2595725976 shutdown_connections 2024-02-17T12:41:05.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:05.375+0000 7f32f2876700 1 -- 172.21.15.18:0/2595725976 wait complete. 2024-02-17T12:41:06.084 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi119 2024-02-17T12:41:06.084 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-02-17T12:41:06.084 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/etc/ceph/ceph.conf 2024-02-17T12:41:06.102 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-02-17T12:41:06.102 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:41:06.160 INFO:tasks.cephadm:Adding host smithi119 to orchestrator... 2024-02-17T12:41:06.161 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch host add smithi119 2024-02-17T12:41:06.402 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:41:06.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:06 smithi018 ceph-mon[32625]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:41:06.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:06 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:06.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:06 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:06 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:06 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:06 smithi018 ceph-mon[32625]: Updating smithi018:/etc/ceph/ceph.conf 2024-02-17T12:41:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:06 smithi018 ceph-mon[32625]: Updating smithi018:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:06 smithi018 ceph-mon[32625]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:41:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:06 smithi018 ceph-mon[32625]: Updating smithi018:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:41:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:06 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:06 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:06 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:07.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.808+0000 7fd10a859700 1 -- 172.21.15.18:0/3578319806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd104106610 msgr2=0x7fd1041069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:07.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.808+0000 7fd10a859700 1 --2- 172.21.15.18:0/3578319806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd104106610 0x7fd1041069f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009910 tx=0x7fd0f4009c20 comp rx=0 tx=0).stop 2024-02-17T12:41:07.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.808+0000 7fd10a859700 1 -- 172.21.15.18:0/3578319806 shutdown_connections 2024-02-17T12:41:07.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.808+0000 7fd10a859700 1 --2- 172.21.15.18:0/3578319806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd104106610 0x7fd1041069f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:07.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.808+0000 7fd10a859700 1 -- 172.21.15.18:0/3578319806 >> 172.21.15.18:0/3578319806 conn(0x7fd1040771b0 msgr2=0x7fd1040fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:07.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.808+0000 7fd10a859700 1 -- 172.21.15.18:0/3578319806 shutdown_connections 2024-02-17T12:41:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.808+0000 7fd10a859700 1 -- 172.21.15.18:0/3578319806 wait complete. 2024-02-17T12:41:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.809+0000 7fd10a859700 1 Processor -- start 2024-02-17T12:41:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.809+0000 7fd10a859700 1 -- start start 2024-02-17T12:41:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.809+0000 7fd10a859700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd104106610 0x7fd104199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.809+0000 7fd10a859700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd104199590 con 0x7fd104106610 2024-02-17T12:41:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.809+0000 7fd103fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd104106610 0x7fd104199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.809+0000 7fd103fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd104106610 0x7fd104199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58758/0 (socket says 172.21.15.18:58758) 2024-02-17T12:41:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.809+0000 7fd103fff700 1 -- 172.21.15.18:0/3522434837 learned_addr learned my addr 172.21.15.18:0/3522434837 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:41:07.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.810+0000 7fd103fff700 1 -- 172.21.15.18:0/3522434837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f40095f0 con 0x7fd104106610 2024-02-17T12:41:07.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.810+0000 7fd103fff700 1 --2- 172.21.15.18:0/3522434837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd104106610 0x7fd104199050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4004f40 tx=0x7fd0f4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:07.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.810+0000 7fd1017fa700 1 -- 172.21.15.18:0/3522434837 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0f401d070 con 0x7fd104106610 2024-02-17T12:41:07.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.810+0000 7fd1017fa700 1 -- 172.21.15.18:0/3522434837 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0f400bc30 con 0x7fd104106610 2024-02-17T12:41:07.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.810+0000 7fd10a859700 1 -- 172.21.15.18:0/3522434837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1041997f0 con 0x7fd104106610 2024-02-17T12:41:07.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.810+0000 7fd1017fa700 1 -- 172.21.15.18:0/3522434837 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0f4003970 con 0x7fd104106610 2024-02-17T12:41:07.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.810+0000 7fd10a859700 1 -- 172.21.15.18:0/3522434837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd104199cf0 con 0x7fd104106610 2024-02-17T12:41:07.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.811+0000 7fd1017fa700 1 -- 172.21.15.18:0/3522434837 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fd0f4003be0 con 0x7fd104106610 2024-02-17T12:41:07.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.811+0000 7fd1017fa700 1 --2- 172.21.15.18:0/3522434837 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd0ec038fe0 0x7fd0ec03b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:07.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.811+0000 7fd1017fa700 1 -- 172.21.15.18:0/3522434837 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd0f404cbe0 con 0x7fd104106610 2024-02-17T12:41:07.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.811+0000 7fd1037fe700 1 --2- 172.21.15.18:0/3522434837 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd0ec038fe0 0x7fd0ec03b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:07.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.811+0000 7fd10a859700 1 -- 172.21.15.18:0/3522434837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd104101c80 con 0x7fd104106610 2024-02-17T12:41:07.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.811+0000 7fd1037fe700 1 --2- 172.21.15.18:0/3522434837 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd0ec038fe0 0x7fd0ec03b4a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8006fd0 tx=0x7fd0f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:07.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:07.816+0000 7fd1017fa700 1 -- 172.21.15.18:0/3522434837 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd0f4026020 con 0x7fd104106610 2024-02-17T12:41:08.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:08.023+0000 7fd10a859700 1 -- 172.21.15.18:0/3522434837 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi119", "target": ["mon-mgr", ""]}) v1 -- 0x7fd10419a410 con 0x7fd0ec038fe0 2024-02-17T12:41:08.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:08 smithi018 ceph-mon[32625]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi119", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:41:09.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:09 smithi018 ceph-mon[32625]: Deploying cephadm binary to smithi119 2024-02-17T12:41:11.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:11 smithi018 ceph-mon[32625]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:12.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:12.232+0000 7fd1017fa700 1 -- 172.21.15.18:0/3522434837 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd10419a410 con 0x7fd0ec038fe0 2024-02-17T12:41:12.229 INFO:teuthology.orchestra.run.smithi018.stdout:Added host 'smithi119' with addr '172.21.15.119' 2024-02-17T12:41:12.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:12.235+0000 7fd10a859700 1 -- 172.21.15.18:0/3522434837 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd0ec038fe0 msgr2=0x7fd0ec03b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:12.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:12.235+0000 7fd10a859700 1 --2- 172.21.15.18:0/3522434837 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd0ec038fe0 0x7fd0ec03b4a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8006fd0 tx=0x7fd0f8006e40 comp rx=0 tx=0).stop 2024-02-17T12:41:12.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:12.235+0000 7fd10a859700 1 -- 172.21.15.18:0/3522434837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd104106610 msgr2=0x7fd104199050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:12.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:12.235+0000 7fd10a859700 1 --2- 172.21.15.18:0/3522434837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd104106610 0x7fd104199050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4004f40 tx=0x7fd0f4005020 comp rx=0 tx=0).stop 2024-02-17T12:41:12.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:12.235+0000 7fd10a859700 1 -- 172.21.15.18:0/3522434837 shutdown_connections 2024-02-17T12:41:12.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:12.235+0000 7fd10a859700 1 --2- 172.21.15.18:0/3522434837 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd0ec038fe0 0x7fd0ec03b4a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:12.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:12.235+0000 7fd10a859700 1 --2- 172.21.15.18:0/3522434837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd104106610 0x7fd104199050 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:12.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:12.235+0000 7fd10a859700 1 -- 172.21.15.18:0/3522434837 >> 172.21.15.18:0/3522434837 conn(0x7fd1040771b0 msgr2=0x7fd1040fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:12.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:12.236+0000 7fd10a859700 1 -- 172.21.15.18:0/3522434837 shutdown_connections 2024-02-17T12:41:12.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:12.236+0000 7fd10a859700 1 -- 172.21.15.18:0/3522434837 wait complete. 2024-02-17T12:41:12.750 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-17T12:41:12.991 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:41:13.421 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:13 smithi018 ceph-mon[32625]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:13.421 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:13.421 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:13 smithi018 ceph-mon[32625]: Added host smithi119 2024-02-17T12:41:13.421 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:13.421 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:13.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:14.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.395+0000 7f4698fb4700 1 -- 172.21.15.18:0/2844390648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4694106620 msgr2=0x7f4694106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:14.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.395+0000 7f4698fb4700 1 --2- 172.21.15.18:0/2844390648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4694106620 0x7f4694106a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4684009910 tx=0x7f4684009c20 comp rx=0 tx=0).stop 2024-02-17T12:41:14.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.395+0000 7f4698fb4700 1 -- 172.21.15.18:0/2844390648 shutdown_connections 2024-02-17T12:41:14.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.395+0000 7f4698fb4700 1 --2- 172.21.15.18:0/2844390648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4694106620 0x7f4694106a00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:14.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.395+0000 7f4698fb4700 1 -- 172.21.15.18:0/2844390648 >> 172.21.15.18:0/2844390648 conn(0x7f46940fc060 msgr2=0x7f46940fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:14.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.395+0000 7f4698fb4700 1 -- 172.21.15.18:0/2844390648 shutdown_connections 2024-02-17T12:41:14.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.395+0000 7f4698fb4700 1 -- 172.21.15.18:0/2844390648 wait complete. 2024-02-17T12:41:14.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.396+0000 7f4698fb4700 1 Processor -- start 2024-02-17T12:41:14.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.396+0000 7f4698fb4700 1 -- start start 2024-02-17T12:41:14.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.396+0000 7f4698fb4700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4694106620 0x7f4694197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:14.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.396+0000 7f4698fb4700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469406aaa0 con 0x7f4694106620 2024-02-17T12:41:14.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.396+0000 7f469259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4694106620 0x7f4694197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:14.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.396+0000 7f469259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4694106620 0x7f4694197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50456/0 (socket says 172.21.15.18:50456) 2024-02-17T12:41:14.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.396+0000 7f469259c700 1 -- 172.21.15.18:0/1012587099 learned_addr learned my addr 172.21.15.18:0/1012587099 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:41:14.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.396+0000 7f469259c700 1 -- 172.21.15.18:0/1012587099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46840095f0 con 0x7f4694106620 2024-02-17T12:41:14.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.396+0000 7f469259c700 1 --2- 172.21.15.18:0/1012587099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4694106620 0x7f4694197ce0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4684004f40 tx=0x7f4684005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:14.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.397+0000 7f46837fe700 1 -- 172.21.15.18:0/1012587099 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f468401d070 con 0x7f4694106620 2024-02-17T12:41:14.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.397+0000 7f4698fb4700 1 -- 172.21.15.18:0/1012587099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46941982e0 con 0x7f4694106620 2024-02-17T12:41:14.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.397+0000 7f46837fe700 1 -- 172.21.15.18:0/1012587099 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f468400bc30 con 0x7f4694106620 2024-02-17T12:41:14.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.397+0000 7f46837fe700 1 -- 172.21.15.18:0/1012587099 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4684003970 con 0x7f4694106620 2024-02-17T12:41:14.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.397+0000 7f4698fb4700 1 -- 172.21.15.18:0/1012587099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4694198760 con 0x7f4694106620 2024-02-17T12:41:14.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.397+0000 7f46837fe700 1 -- 172.21.15.18:0/1012587099 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f4684003c30 con 0x7f4694106620 2024-02-17T12:41:14.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.397+0000 7f4698fb4700 1 -- 172.21.15.18:0/1012587099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4694101c90 con 0x7f4694106620 2024-02-17T12:41:14.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.398+0000 7f46837fe700 1 --2- 172.21.15.18:0/1012587099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f467c038f80 0x7f467c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:14.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.398+0000 7f46837fe700 1 -- 172.21.15.18:0/1012587099 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f468404cdb0 con 0x7f4694106620 2024-02-17T12:41:14.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.398+0000 7f4691d9b700 1 --2- 172.21.15.18:0/1012587099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f467c038f80 0x7f467c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:14.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.398+0000 7f4691d9b700 1 --2- 172.21.15.18:0/1012587099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f467c038f80 0x7f467c03b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4688006fd0 tx=0x7f4688006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:14.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.403+0000 7f46837fe700 1 -- 172.21.15.18:0/1012587099 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4684029910 con 0x7f4694106620 2024-02-17T12:41:14.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.615+0000 7f4698fb4700 1 -- 172.21.15.18:0/1012587099 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4694198b80 con 0x7f467c038f80 2024-02-17T12:41:14.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.616+0000 7f46837fe700 1 -- 172.21.15.18:0/1012587099 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f4694198b80 con 0x7f467c038f80 2024-02-17T12:41:14.612 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:41:14.612 INFO:teuthology.orchestra.run.smithi018.stdout:[{"addr": "172.21.15.18", "hostname": "smithi018", "labels": [], "status": ""}, {"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}] 2024-02-17T12:41:14.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.618+0000 7f4698fb4700 1 -- 172.21.15.18:0/1012587099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f467c038f80 msgr2=0x7f467c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:14.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.618+0000 7f4698fb4700 1 --2- 172.21.15.18:0/1012587099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f467c038f80 0x7f467c03b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4688006fd0 tx=0x7f4688006e40 comp rx=0 tx=0).stop 2024-02-17T12:41:14.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.618+0000 7f4698fb4700 1 -- 172.21.15.18:0/1012587099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4694106620 msgr2=0x7f4694197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:14.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.618+0000 7f4698fb4700 1 --2- 172.21.15.18:0/1012587099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4694106620 0x7f4694197ce0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4684004f40 tx=0x7f4684005020 comp rx=0 tx=0).stop 2024-02-17T12:41:14.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.618+0000 7f4698fb4700 1 -- 172.21.15.18:0/1012587099 shutdown_connections 2024-02-17T12:41:14.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.618+0000 7f4698fb4700 1 --2- 172.21.15.18:0/1012587099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f467c038f80 0x7f467c03b440 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:14.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.618+0000 7f4698fb4700 1 --2- 172.21.15.18:0/1012587099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4694106620 0x7f4694197ce0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:14.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.618+0000 7f4698fb4700 1 -- 172.21.15.18:0/1012587099 >> 172.21.15.18:0/1012587099 conn(0x7f46940fc060 msgr2=0x7f46940fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:14.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.618+0000 7f4698fb4700 1 -- 172.21.15.18:0/1012587099 shutdown_connections 2024-02-17T12:41:14.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:14.618+0000 7f4698fb4700 1 -- 172.21.15.18:0/1012587099 wait complete. 2024-02-17T12:41:15.202 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi146 2024-02-17T12:41:15.203 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-02-17T12:41:15.203 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/etc/ceph/ceph.conf 2024-02-17T12:41:15.221 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-02-17T12:41:15.221 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:41:15.278 INFO:tasks.cephadm:Adding host smithi146 to orchestrator... 2024-02-17T12:41:15.279 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch host add smithi146 2024-02-17T12:41:15.522 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:41:15.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:15 smithi018 ceph-mon[32625]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:15.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:15.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:15 smithi018 ceph-mon[32625]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:41:16.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.941+0000 7f428ec86700 1 -- 172.21.15.18:0/1674486337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42881076d0 msgr2=0x7f4288107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:16.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.941+0000 7f428ec86700 1 --2- 172.21.15.18:0/1674486337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42881076d0 0x7f4288107ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4278009910 tx=0x7f4278009c20 comp rx=0 tx=0).stop 2024-02-17T12:41:16.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.941+0000 7f428ec86700 1 -- 172.21.15.18:0/1674486337 shutdown_connections 2024-02-17T12:41:16.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.941+0000 7f428ec86700 1 --2- 172.21.15.18:0/1674486337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42881076d0 0x7f4288107ab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:16.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.941+0000 7f428ec86700 1 -- 172.21.15.18:0/1674486337 >> 172.21.15.18:0/1674486337 conn(0x7f42880fb7c0 msgr2=0x7f42880fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:16.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.941+0000 7f428ec86700 1 -- 172.21.15.18:0/1674486337 shutdown_connections 2024-02-17T12:41:16.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.941+0000 7f428ec86700 1 -- 172.21.15.18:0/1674486337 wait complete. 2024-02-17T12:41:16.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.941+0000 7f428ec86700 1 Processor -- start 2024-02-17T12:41:16.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.941+0000 7f428ec86700 1 -- start start 2024-02-17T12:41:16.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.942+0000 7f428ec86700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42881076d0 0x7f4288196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:16.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.942+0000 7f428ec86700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4288197480 con 0x7f42881076d0 2024-02-17T12:41:16.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.942+0000 7f428ca22700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42881076d0 0x7f4288196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:16.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.942+0000 7f428ca22700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42881076d0 0x7f4288196f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50488/0 (socket says 172.21.15.18:50488) 2024-02-17T12:41:16.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.942+0000 7f428ca22700 1 -- 172.21.15.18:0/4060523791 learned_addr learned my addr 172.21.15.18:0/4060523791 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:41:16.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.942+0000 7f428ca22700 1 -- 172.21.15.18:0/4060523791 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42780095f0 con 0x7f42881076d0 2024-02-17T12:41:16.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.942+0000 7f428ca22700 1 --2- 172.21.15.18:0/4060523791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42881076d0 0x7f4288196f40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4278006010 tx=0x7f4278004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:16.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.942+0000 7f4285ffb700 1 -- 172.21.15.18:0/4060523791 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f427801d070 con 0x7f42881076d0 2024-02-17T12:41:16.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.942+0000 7f4285ffb700 1 -- 172.21.15.18:0/4060523791 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f427800bbd0 con 0x7f42881076d0 2024-02-17T12:41:16.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.942+0000 7f428ec86700 1 -- 172.21.15.18:0/4060523791 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42881976e0 con 0x7f42881076d0 2024-02-17T12:41:16.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.942+0000 7f4285ffb700 1 -- 172.21.15.18:0/4060523791 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4278003880 con 0x7f42881076d0 2024-02-17T12:41:16.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.943+0000 7f428ec86700 1 -- 172.21.15.18:0/4060523791 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4288197be0 con 0x7f42881076d0 2024-02-17T12:41:16.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.943+0000 7f4285ffb700 1 -- 172.21.15.18:0/4060523791 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f4278003a70 con 0x7f42881076d0 2024-02-17T12:41:16.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.943+0000 7f4285ffb700 1 --2- 172.21.15.18:0/4060523791 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4270038fe0 0x7f427003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:16.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.944+0000 7f4285ffb700 1 -- 172.21.15.18:0/4060523791 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f427804cba0 con 0x7f42881076d0 2024-02-17T12:41:16.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.944+0000 7f428ec86700 1 -- 172.21.15.18:0/4060523791 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4288102dc0 con 0x7f42881076d0 2024-02-17T12:41:16.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.944+0000 7f4287fff700 1 --2- 172.21.15.18:0/4060523791 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4270038fe0 0x7f427003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:16.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.944+0000 7f4287fff700 1 --2- 172.21.15.18:0/4060523791 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4270038fe0 0x7f427003b4a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f427c006fd0 tx=0x7f427c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:16.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:16.949+0000 7f4285ffb700 1 -- 172.21.15.18:0/4060523791 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4278026020 con 0x7f42881076d0 2024-02-17T12:41:17.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:17.164+0000 7f428ec86700 1 -- 172.21.15.18:0/4060523791 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}) v1 -- 0x7f42881982c0 con 0x7f4270038fe0 2024-02-17T12:41:17.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:17 smithi018 ceph-mon[32625]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:18.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:41:18.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:18.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:18.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:18.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:18.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:41:18.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:18.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:18.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: Updating smithi119:/etc/ceph/ceph.conf 2024-02-17T12:41:18.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:18.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: Deploying cephadm binary to smithi146 2024-02-17T12:41:18.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:41:18.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:18.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:18.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:19.671 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:19 smithi018 ceph-mon[32625]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:19.671 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:19 smithi018 ceph-mon[32625]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:41:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:21.453+0000 7f4285ffb700 1 -- 172.21.15.18:0/4060523791 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f42881982c0 con 0x7f4270038fe0 2024-02-17T12:41:21.450 INFO:teuthology.orchestra.run.smithi018.stdout:Added host 'smithi146' with addr '172.21.15.146' 2024-02-17T12:41:21.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:21.456+0000 7f428ec86700 1 -- 172.21.15.18:0/4060523791 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4270038fe0 msgr2=0x7f427003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:21.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:21.456+0000 7f428ec86700 1 --2- 172.21.15.18:0/4060523791 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4270038fe0 0x7f427003b4a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f427c006fd0 tx=0x7f427c006e40 comp rx=0 tx=0).stop 2024-02-17T12:41:21.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:21.456+0000 7f428ec86700 1 -- 172.21.15.18:0/4060523791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42881076d0 msgr2=0x7f4288196f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:21.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:21.456+0000 7f428ec86700 1 --2- 172.21.15.18:0/4060523791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42881076d0 0x7f4288196f40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4278006010 tx=0x7f4278004ff0 comp rx=0 tx=0).stop 2024-02-17T12:41:21.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:21.457+0000 7f428ec86700 1 -- 172.21.15.18:0/4060523791 shutdown_connections 2024-02-17T12:41:21.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:21.457+0000 7f428ec86700 1 --2- 172.21.15.18:0/4060523791 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4270038fe0 0x7f427003b4a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:21.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:21.457+0000 7f428ec86700 1 --2- 172.21.15.18:0/4060523791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42881076d0 0x7f4288196f40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:21.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:21.457+0000 7f428ec86700 1 -- 172.21.15.18:0/4060523791 >> 172.21.15.18:0/4060523791 conn(0x7f42880fb7c0 msgr2=0x7f42880fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:21.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:21.457+0000 7f428ec86700 1 -- 172.21.15.18:0/4060523791 shutdown_connections 2024-02-17T12:41:21.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:21.457+0000 7f428ec86700 1 -- 172.21.15.18:0/4060523791 wait complete. 2024-02-17T12:41:21.751 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:21 smithi018 ceph-mon[32625]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:22.036 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-17T12:41:22.278 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:41:22.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:22 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:22.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:22 smithi018 ceph-mon[32625]: Added host smithi146 2024-02-17T12:41:22.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:22 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:22.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:22 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:23.720 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:23 smithi018 ceph-mon[32625]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:23.720 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:23 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:23.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.757+0000 7fc2f69b1700 1 -- 172.21.15.18:0/2108617128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2f0107650 msgr2=0x7fc2f0107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:23.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.757+0000 7fc2f69b1700 1 --2- 172.21.15.18:0/2108617128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2f0107650 0x7fc2f0107a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009910 tx=0x7fc2e0009c20 comp rx=0 tx=0).stop 2024-02-17T12:41:23.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.757+0000 7fc2f69b1700 1 -- 172.21.15.18:0/2108617128 shutdown_connections 2024-02-17T12:41:23.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.757+0000 7fc2f69b1700 1 --2- 172.21.15.18:0/2108617128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2f0107650 0x7fc2f0107a30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:23.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.757+0000 7fc2f69b1700 1 -- 172.21.15.18:0/2108617128 >> 172.21.15.18:0/2108617128 conn(0x7fc2f00fd180 msgr2=0x7fc2f00ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:23.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.757+0000 7fc2f69b1700 1 -- 172.21.15.18:0/2108617128 shutdown_connections 2024-02-17T12:41:23.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.757+0000 7fc2f69b1700 1 -- 172.21.15.18:0/2108617128 wait complete. 2024-02-17T12:41:23.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.758+0000 7fc2f69b1700 1 Processor -- start 2024-02-17T12:41:23.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.758+0000 7fc2f69b1700 1 -- start start 2024-02-17T12:41:23.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.758+0000 7fc2f69b1700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2f0107650 0x7fc2f0196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:23.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.758+0000 7fc2f69b1700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f01974c0 con 0x7fc2f0107650 2024-02-17T12:41:23.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.759+0000 7fc2effff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2f0107650 0x7fc2f0196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:23.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.759+0000 7fc2effff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2f0107650 0x7fc2f0196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54270/0 (socket says 172.21.15.18:54270) 2024-02-17T12:41:23.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.759+0000 7fc2effff700 1 -- 172.21.15.18:0/3886797474 learned_addr learned my addr 172.21.15.18:0/3886797474 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:41:23.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.759+0000 7fc2effff700 1 -- 172.21.15.18:0/3886797474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e00095f0 con 0x7fc2f0107650 2024-02-17T12:41:23.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.759+0000 7fc2effff700 1 --2- 172.21.15.18:0/3886797474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2f0107650 0x7fc2f0196f80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0006010 tx=0x7fc2e0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:23.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.759+0000 7fc2ed7fa700 1 -- 172.21.15.18:0/3886797474 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2e001d070 con 0x7fc2f0107650 2024-02-17T12:41:23.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.759+0000 7fc2f69b1700 1 -- 172.21.15.18:0/3886797474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f0197720 con 0x7fc2f0107650 2024-02-17T12:41:23.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.759+0000 7fc2ed7fa700 1 -- 172.21.15.18:0/3886797474 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2e000bbd0 con 0x7fc2f0107650 2024-02-17T12:41:23.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.759+0000 7fc2ed7fa700 1 -- 172.21.15.18:0/3886797474 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2e0003880 con 0x7fc2f0107650 2024-02-17T12:41:23.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.759+0000 7fc2f69b1700 1 -- 172.21.15.18:0/3886797474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f0197ba0 con 0x7fc2f0107650 2024-02-17T12:41:23.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.760+0000 7fc2ed7fa700 1 -- 172.21.15.18:0/3886797474 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fc2e0003ad0 con 0x7fc2f0107650 2024-02-17T12:41:23.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.760+0000 7fc2ed7fa700 1 --2- 172.21.15.18:0/3886797474 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc2d8038fe0 0x7fc2d803b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:23.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.760+0000 7fc2ed7fa700 1 -- 172.21.15.18:0/3886797474 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc2e004cd00 con 0x7fc2f0107650 2024-02-17T12:41:23.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.760+0000 7fc2f69b1700 1 -- 172.21.15.18:0/3886797474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f0102d90 con 0x7fc2f0107650 2024-02-17T12:41:23.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.760+0000 7fc2ef7fe700 1 --2- 172.21.15.18:0/3886797474 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc2d8038fe0 0x7fc2d803b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:23.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.761+0000 7fc2ef7fe700 1 --2- 172.21.15.18:0/3886797474 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc2d8038fe0 0x7fc2d803b4a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4006fd0 tx=0x7fc2e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:23.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.765+0000 7fc2ed7fa700 1 -- 172.21.15.18:0/3886797474 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc2e0026080 con 0x7fc2f0107650 2024-02-17T12:41:23.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.970+0000 7fc2f69b1700 1 -- 172.21.15.18:0/3886797474 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2f010bb00 con 0x7fc2d8038fe0 2024-02-17T12:41:23.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.972+0000 7fc2ed7fa700 1 -- 172.21.15.18:0/3886797474 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fc2f010bb00 con 0x7fc2d8038fe0 2024-02-17T12:41:23.969 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:41:23.969 INFO:teuthology.orchestra.run.smithi018.stdout:[{"addr": "172.21.15.18", "hostname": "smithi018", "labels": [], "status": ""}, {"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}, {"addr": "172.21.15.146", "hostname": "smithi146", "labels": [], "status": ""}] 2024-02-17T12:41:23.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.973+0000 7fc2f69b1700 1 -- 172.21.15.18:0/3886797474 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc2d8038fe0 msgr2=0x7fc2d803b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:23.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.973+0000 7fc2f69b1700 1 --2- 172.21.15.18:0/3886797474 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc2d8038fe0 0x7fc2d803b4a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4006fd0 tx=0x7fc2e4006e40 comp rx=0 tx=0).stop 2024-02-17T12:41:23.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.973+0000 7fc2f69b1700 1 -- 172.21.15.18:0/3886797474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2f0107650 msgr2=0x7fc2f0196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:23.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.973+0000 7fc2f69b1700 1 --2- 172.21.15.18:0/3886797474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2f0107650 0x7fc2f0196f80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0006010 tx=0x7fc2e0004ff0 comp rx=0 tx=0).stop 2024-02-17T12:41:23.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.974+0000 7fc2f69b1700 1 -- 172.21.15.18:0/3886797474 shutdown_connections 2024-02-17T12:41:23.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.974+0000 7fc2f69b1700 1 --2- 172.21.15.18:0/3886797474 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc2d8038fe0 0x7fc2d803b4a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:23.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.974+0000 7fc2f69b1700 1 --2- 172.21.15.18:0/3886797474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2f0107650 0x7fc2f0196f80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:23.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.974+0000 7fc2f69b1700 1 -- 172.21.15.18:0/3886797474 >> 172.21.15.18:0/3886797474 conn(0x7fc2f00fd180 msgr2=0x7fc2f0109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:23.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.974+0000 7fc2f69b1700 1 -- 172.21.15.18:0/3886797474 shutdown_connections 2024-02-17T12:41:23.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:23.974+0000 7fc2f69b1700 1 -- 172.21.15.18:0/3886797474 wait complete. 2024-02-17T12:41:24.559 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-17T12:41:24.560 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd crush tunables default 2024-02-17T12:41:24.799 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:41:25.227 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:25 smithi018 ceph-mon[32625]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:41:25.227 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:26.079 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:26 smithi018 ceph-mon[32625]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.316+0000 7fdd996d3700 1 -- 172.21.15.18:0/1903027538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd94106620 msgr2=0x7fdd94106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.316+0000 7fdd996d3700 1 --2- 172.21.15.18:0/1903027538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd94106620 0x7fdd94106a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009910 tx=0x7fdd7c009c20 comp rx=0 tx=0).stop 2024-02-17T12:41:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.316+0000 7fdd996d3700 1 -- 172.21.15.18:0/1903027538 shutdown_connections 2024-02-17T12:41:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.316+0000 7fdd996d3700 1 --2- 172.21.15.18:0/1903027538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd94106620 0x7fdd94106a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.316+0000 7fdd996d3700 1 -- 172.21.15.18:0/1903027538 >> 172.21.15.18:0/1903027538 conn(0x7fdd940fc060 msgr2=0x7fdd940fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.316+0000 7fdd996d3700 1 -- 172.21.15.18:0/1903027538 shutdown_connections 2024-02-17T12:41:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.316+0000 7fdd996d3700 1 -- 172.21.15.18:0/1903027538 wait complete. 2024-02-17T12:41:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.317+0000 7fdd996d3700 1 Processor -- start 2024-02-17T12:41:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.317+0000 7fdd996d3700 1 -- start start 2024-02-17T12:41:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.317+0000 7fdd996d3700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd94106620 0x7fdd94197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:26.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.317+0000 7fdd996d3700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9406aaa0 con 0x7fdd94106620 2024-02-17T12:41:26.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.317+0000 7fdd92ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd94106620 0x7fdd94197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:26.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.317+0000 7fdd92ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd94106620 0x7fdd94197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54302/0 (socket says 172.21.15.18:54302) 2024-02-17T12:41:26.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.317+0000 7fdd92ffd700 1 -- 172.21.15.18:0/517340665 learned_addr learned my addr 172.21.15.18:0/517340665 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:41:26.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.317+0000 7fdd92ffd700 1 -- 172.21.15.18:0/517340665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c0095f0 con 0x7fdd94106620 2024-02-17T12:41:26.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.317+0000 7fdd92ffd700 1 --2- 172.21.15.18:0/517340665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd94106620 0x7fdd94197ce0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c004f40 tx=0x7fdd7c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:26.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.318+0000 7fdd8bfff700 1 -- 172.21.15.18:0/517340665 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd7c01d070 con 0x7fdd94106620 2024-02-17T12:41:26.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.318+0000 7fdd8bfff700 1 -- 172.21.15.18:0/517340665 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd7c00bc30 con 0x7fdd94106620 2024-02-17T12:41:26.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.318+0000 7fdd996d3700 1 -- 172.21.15.18:0/517340665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd941982e0 con 0x7fdd94106620 2024-02-17T12:41:26.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.318+0000 7fdd8bfff700 1 -- 172.21.15.18:0/517340665 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd7c003970 con 0x7fdd94106620 2024-02-17T12:41:26.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.318+0000 7fdd996d3700 1 -- 172.21.15.18:0/517340665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd94198760 con 0x7fdd94106620 2024-02-17T12:41:26.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.319+0000 7fdd8bfff700 1 -- 172.21.15.18:0/517340665 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fdd7c003be0 con 0x7fdd94106620 2024-02-17T12:41:26.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.319+0000 7fdd996d3700 1 -- 172.21.15.18:0/517340665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd94101c90 con 0x7fdd94106620 2024-02-17T12:41:26.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.319+0000 7fdd8bfff700 1 --2- 172.21.15.18:0/517340665 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdd80038fe0 0x7fdd8003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:26.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.319+0000 7fdd8bfff700 1 -- 172.21.15.18:0/517340665 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdd7c04ce90 con 0x7fdd94106620 2024-02-17T12:41:26.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.319+0000 7fdd927fc700 1 --2- 172.21.15.18:0/517340665 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdd80038fe0 0x7fdd8003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:26.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.319+0000 7fdd927fc700 1 --2- 172.21.15.18:0/517340665 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdd80038fe0 0x7fdd8003b4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdd84006fd0 tx=0x7fdd84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:26.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.324+0000 7fdd8bfff700 1 -- 172.21.15.18:0/517340665 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd7c029950 con 0x7fdd94106620 2024-02-17T12:41:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:26.530+0000 7fdd996d3700 1 -- 172.21.15.18:0/517340665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fdd94198ae0 con 0x7fdd94106620 2024-02-17T12:41:27.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:27.032+0000 7fdd8bfff700 1 -- 172.21.15.18:0/517340665 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdd7c026070 con 0x7fdd94106620 2024-02-17T12:41:27.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:27.035+0000 7fdd996d3700 1 -- 172.21.15.18:0/517340665 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdd80038fe0 msgr2=0x7fdd8003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:27.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:27.035+0000 7fdd996d3700 1 --2- 172.21.15.18:0/517340665 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdd80038fe0 0x7fdd8003b4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdd84006fd0 tx=0x7fdd84006e40 comp rx=0 tx=0).stop 2024-02-17T12:41:27.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:27.035+0000 7fdd996d3700 1 -- 172.21.15.18:0/517340665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd94106620 msgr2=0x7fdd94197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:27.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:27.035+0000 7fdd996d3700 1 --2- 172.21.15.18:0/517340665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd94106620 0x7fdd94197ce0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c004f40 tx=0x7fdd7c005020 comp rx=0 tx=0).stop 2024-02-17T12:41:27.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:27.035+0000 7fdd996d3700 1 -- 172.21.15.18:0/517340665 shutdown_connections 2024-02-17T12:41:27.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:27.035+0000 7fdd996d3700 1 --2- 172.21.15.18:0/517340665 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdd80038fe0 0x7fdd8003b4a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:27.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:27.035+0000 7fdd996d3700 1 --2- 172.21.15.18:0/517340665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd94106620 0x7fdd94197ce0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:27.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:27.035+0000 7fdd996d3700 1 -- 172.21.15.18:0/517340665 >> 172.21.15.18:0/517340665 conn(0x7fdd940fc060 msgr2=0x7fdd940fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:27.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:27.036+0000 7fdd996d3700 1 -- 172.21.15.18:0/517340665 shutdown_connections 2024-02-17T12:41:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:27.036+0000 7fdd996d3700 1 -- 172.21.15.18:0/517340665 wait complete. 2024-02-17T12:41:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:adjusted tunables profile to default 2024-02-17T12:41:27.319 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:27 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/517340665' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-17T12:41:27.640 INFO:tasks.cephadm:Adding mon.a on smithi018 2024-02-17T12:41:27.640 INFO:tasks.cephadm:Adding mon.b on smithi119 2024-02-17T12:41:27.640 INFO:tasks.cephadm:Adding mon.c on smithi146 2024-02-17T12:41:27.640 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch apply mon '3;smithi018:172.21.15.18=a;smithi119:172.21.15.119=b;smithi146:172.21.15.146=c' 2024-02-17T12:41:27.876 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:28.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:28.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/517340665' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-17T12:41:28.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T12:41:28.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:28.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:28.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:28.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:28.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi146", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:41:28.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:28.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:28.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:28.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:28.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:28 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:29.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.143+0000 7fc70cc10700 1 -- 172.21.15.146:0/3189807584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7081028a0 msgr2=0x7fc708102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:29.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.143+0000 7fc70cc10700 1 --2- 172.21.15.146:0/3189807584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7081028a0 0x7fc708102c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009910 tx=0x7fc6fc009c20 comp rx=0 tx=0).stop 2024-02-17T12:41:29.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.143+0000 7fc70cc10700 1 -- 172.21.15.146:0/3189807584 shutdown_connections 2024-02-17T12:41:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.143+0000 7fc70cc10700 1 --2- 172.21.15.146:0/3189807584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7081028a0 0x7fc708102c80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.143+0000 7fc70cc10700 1 -- 172.21.15.146:0/3189807584 >> 172.21.15.146:0/3189807584 conn(0x7fc7080fc0b0 msgr2=0x7fc7080fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.143+0000 7fc70cc10700 1 -- 172.21.15.146:0/3189807584 shutdown_connections 2024-02-17T12:41:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.143+0000 7fc70cc10700 1 -- 172.21.15.146:0/3189807584 wait complete. 2024-02-17T12:41:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.144+0000 7fc70cc10700 1 Processor -- start 2024-02-17T12:41:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.144+0000 7fc70cc10700 1 -- start start 2024-02-17T12:41:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.144+0000 7fc70cc10700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7081028a0 0x7fc708196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:29.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.144+0000 7fc70cc10700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc708197520 con 0x7fc7081028a0 2024-02-17T12:41:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.144+0000 7fc70659c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7081028a0 0x7fc708196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.144+0000 7fc70659c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7081028a0 0x7fc708196fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.146:49252/0 (socket says 172.21.15.146:49252) 2024-02-17T12:41:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.144+0000 7fc70659c700 1 -- 172.21.15.146:0/1167428596 learned_addr learned my addr 172.21.15.146:0/1167428596 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-02-17T12:41:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.144+0000 7fc70659c700 1 -- 172.21.15.146:0/1167428596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc0095f0 con 0x7fc7081028a0 2024-02-17T12:41:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.145+0000 7fc70659c700 1 --2- 172.21.15.146:0/1167428596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7081028a0 0x7fc708196fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc005850 tx=0x7fc6fc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.145+0000 7fc6f77fe700 1 -- 172.21.15.146:0/1167428596 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6fc01d070 con 0x7fc7081028a0 2024-02-17T12:41:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.145+0000 7fc6f77fe700 1 -- 172.21.15.146:0/1167428596 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6fc00bc30 con 0x7fc7081028a0 2024-02-17T12:41:29.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.145+0000 7fc70cc10700 1 -- 172.21.15.146:0/1167428596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc708197720 con 0x7fc7081028a0 2024-02-17T12:41:29.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.145+0000 7fc6f77fe700 1 -- 172.21.15.146:0/1167428596 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6fc003a20 con 0x7fc7081028a0 2024-02-17T12:41:29.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.145+0000 7fc70cc10700 1 -- 172.21.15.146:0/1167428596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc708197b40 con 0x7fc7081028a0 2024-02-17T12:41:29.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.146+0000 7fc70cc10700 1 -- 172.21.15.146:0/1167428596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7080632a0 con 0x7fc7081028a0 2024-02-17T12:41:29.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.146+0000 7fc6f77fe700 1 -- 172.21.15.146:0/1167428596 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fc6fc004390 con 0x7fc7081028a0 2024-02-17T12:41:29.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.146+0000 7fc6f77fe700 1 --2- 172.21.15.146:0/1167428596 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc6f0038f80 0x7fc6f003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:29.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.146+0000 7fc6f77fe700 1 -- 172.21.15.146:0/1167428596 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc6fc04ce40 con 0x7fc7081028a0 2024-02-17T12:41:29.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.146+0000 7fc705d9b700 1 --2- 172.21.15.146:0/1167428596 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc6f0038f80 0x7fc6f003b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:29.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.147+0000 7fc705d9b700 1 --2- 172.21.15.146:0/1167428596 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc6f0038f80 0x7fc6f003b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8006fd0 tx=0x7fc6f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:29.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.151+0000 7fc6f77fe700 1 -- 172.21.15.146:0/1167428596 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc6fc02ac90 con 0x7fc7081028a0 2024-02-17T12:41:29.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.355+0000 7fc70cc10700 1 -- 172.21.15.146:0/1167428596 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi018:172.21.15.18=a;smithi119:172.21.15.119=b;smithi146:172.21.15.146=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fc70810bbf0 con 0x7fc6f0038f80 2024-02-17T12:41:29.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.362+0000 7fc6f77fe700 1 -- 172.21.15.146:0/1167428596 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc70810bbf0 con 0x7fc6f0038f80 2024-02-17T12:41:29.357 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled mon update... 2024-02-17T12:41:29.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.364+0000 7fc70cc10700 1 -- 172.21.15.146:0/1167428596 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc6f0038f80 msgr2=0x7fc6f003b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:29.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.364+0000 7fc70cc10700 1 --2- 172.21.15.146:0/1167428596 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc6f0038f80 0x7fc6f003b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8006fd0 tx=0x7fc6f8006e40 comp rx=0 tx=0).stop 2024-02-17T12:41:29.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.364+0000 7fc70cc10700 1 -- 172.21.15.146:0/1167428596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7081028a0 msgr2=0x7fc708196fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:29.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.364+0000 7fc70cc10700 1 --2- 172.21.15.146:0/1167428596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7081028a0 0x7fc708196fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc005850 tx=0x7fc6fc004ff0 comp rx=0 tx=0).stop 2024-02-17T12:41:29.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.364+0000 7fc70cc10700 1 -- 172.21.15.146:0/1167428596 shutdown_connections 2024-02-17T12:41:29.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.364+0000 7fc70cc10700 1 --2- 172.21.15.146:0/1167428596 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc6f0038f80 0x7fc6f003b440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:29.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.364+0000 7fc70cc10700 1 --2- 172.21.15.146:0/1167428596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7081028a0 0x7fc708196fe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:29.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.364+0000 7fc70cc10700 1 -- 172.21.15.146:0/1167428596 >> 172.21.15.146:0/1167428596 conn(0x7fc7080fc0b0 msgr2=0x7fc708103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:29.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.364+0000 7fc70cc10700 1 -- 172.21.15.146:0/1167428596 shutdown_connections 2024-02-17T12:41:29.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:29.364+0000 7fc70cc10700 1 -- 172.21.15.146:0/1167428596 wait complete. 2024-02-17T12:41:29.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:29 smithi018 ceph-mon[32625]: Updating smithi146:/etc/ceph/ceph.conf 2024-02-17T12:41:29.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:29 smithi018 ceph-mon[32625]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:29.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:29 smithi018 ceph-mon[32625]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:41:29.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:29 smithi018 ceph-mon[32625]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:41:29.944 DEBUG:teuthology.orchestra.run.smithi119:mon.b> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.b.service 2024-02-17T12:41:29.947 DEBUG:teuthology.orchestra.run.smithi146:mon.c> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.c.service 2024-02-17T12:41:29.949 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-17T12:41:29.950 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-17T12:41:30.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:30 smithi018 ceph-mon[32625]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:30.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:30.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:30.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:30.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:30.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:30.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:30.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:30.666 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:31.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:31 smithi018 ceph-mon[32625]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi018:172.21.15.18=a;smithi119:172.21.15.119=b;smithi146:172.21.15.146=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:41:31.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:31 smithi018 ceph-mon[32625]: Saving service mon spec with placement smithi018:172.21.15.18=a;smithi119:172.21.15.119=b;smithi146:172.21.15.146=c;count:3 2024-02-17T12:41:31.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:31 smithi018 ceph-mon[32625]: Deploying daemon mon.c on smithi146 2024-02-17T12:41:32.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:32 smithi018 ceph-mon[32625]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:33.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.394+0000 7fa754abb700 1 -- 172.21.15.146:0/2531735200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7501027f0 msgr2=0x7fa750102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:33.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.394+0000 7fa754abb700 1 --2- 172.21.15.146:0/2531735200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7501027f0 0x7fa750102bd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa744009910 tx=0x7fa744009c20 comp rx=0 tx=0).stop 2024-02-17T12:41:33.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.394+0000 7fa754abb700 1 -- 172.21.15.146:0/2531735200 shutdown_connections 2024-02-17T12:41:33.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.394+0000 7fa754abb700 1 --2- 172.21.15.146:0/2531735200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7501027f0 0x7fa750102bd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:33.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.394+0000 7fa754abb700 1 -- 172.21.15.146:0/2531735200 >> 172.21.15.146:0/2531735200 conn(0x7fa7500fc020 msgr2=0x7fa7500fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:33.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.394+0000 7fa754abb700 1 -- 172.21.15.146:0/2531735200 shutdown_connections 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.394+0000 7fa754abb700 1 -- 172.21.15.146:0/2531735200 wait complete. 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.395+0000 7fa754abb700 1 Processor -- start 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.395+0000 7fa754abb700 1 -- start start 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.395+0000 7fa754abb700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7501027f0 0x7fa750196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.395+0000 7fa754abb700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa750197530 con 0x7fa7501027f0 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.396+0000 7fa74e59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7501027f0 0x7fa750196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.396+0000 7fa74e59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7501027f0 0x7fa750196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.146:49274/0 (socket says 172.21.15.146:49274) 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.396+0000 7fa74e59c700 1 -- 172.21.15.146:0/3227036814 learned_addr learned my addr 172.21.15.146:0/3227036814 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.396+0000 7fa74e59c700 1 -- 172.21.15.146:0/3227036814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7440095f0 con 0x7fa7501027f0 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.396+0000 7fa74e59c700 1 --2- 172.21.15.146:0/3227036814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7501027f0 0x7fa750196ff0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa744005850 tx=0x7fa744004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.396+0000 7fa73f7fe700 1 -- 172.21.15.146:0/3227036814 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa74401d070 con 0x7fa7501027f0 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.396+0000 7fa754abb700 1 -- 172.21.15.146:0/3227036814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa750197730 con 0x7fa7501027f0 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.396+0000 7fa754abb700 1 -- 172.21.15.146:0/3227036814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa750197b50 con 0x7fa7501027f0 2024-02-17T12:41:33.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.396+0000 7fa73f7fe700 1 -- 172.21.15.146:0/3227036814 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa74400bc30 con 0x7fa7501027f0 2024-02-17T12:41:33.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.396+0000 7fa73f7fe700 1 -- 172.21.15.146:0/3227036814 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa744003a20 con 0x7fa7501027f0 2024-02-17T12:41:33.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.397+0000 7fa754abb700 1 -- 172.21.15.146:0/3227036814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7500632a0 con 0x7fa7501027f0 2024-02-17T12:41:33.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.397+0000 7fa73f7fe700 1 -- 172.21.15.146:0/3227036814 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fa744004390 con 0x7fa7501027f0 2024-02-17T12:41:33.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.398+0000 7fa73f7fe700 1 --2- 172.21.15.146:0/3227036814 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa738038f80 0x7fa73803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:33.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.398+0000 7fa73f7fe700 1 -- 172.21.15.146:0/3227036814 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa74404cf10 con 0x7fa7501027f0 2024-02-17T12:41:33.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.398+0000 7fa74dd9b700 1 --2- 172.21.15.146:0/3227036814 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa738038f80 0x7fa73803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:33.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.398+0000 7fa74dd9b700 1 --2- 172.21.15.146:0/3227036814 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa738038f80 0x7fa73803b440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa740006fd0 tx=0x7fa740006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:33.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.403+0000 7fa73f7fe700 1 -- 172.21.15.146:0/3227036814 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa74402aca0 con 0x7fa7501027f0 2024-02-17T12:41:33.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.699+0000 7fa754abb700 1 -- 172.21.15.146:0/3227036814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa75010bad0 con 0x7fa7501027f0 2024-02-17T12:41:33.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.699+0000 7fa73f7fe700 1 -- 172.21.15.146:0/3227036814 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7fa744026070 con 0x7fa7501027f0 2024-02-17T12:41:33.695 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:41:33.695 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":1,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","modified":"2024-02-17T12:39:45.933431Z","created":"2024-02-17T12:39:45.933431Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-17T12:41:33.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.701+0000 7fa754abb700 1 -- 172.21.15.146:0/3227036814 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa738038f80 msgr2=0x7fa73803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:33.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.701+0000 7fa754abb700 1 --2- 172.21.15.146:0/3227036814 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa738038f80 0x7fa73803b440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa740006fd0 tx=0x7fa740006e40 comp rx=0 tx=0).stop 2024-02-17T12:41:33.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.701+0000 7fa754abb700 1 -- 172.21.15.146:0/3227036814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7501027f0 msgr2=0x7fa750196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:33.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.701+0000 7fa754abb700 1 --2- 172.21.15.146:0/3227036814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7501027f0 0x7fa750196ff0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa744005850 tx=0x7fa744004ff0 comp rx=0 tx=0).stop 2024-02-17T12:41:33.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.701+0000 7fa754abb700 1 -- 172.21.15.146:0/3227036814 shutdown_connections 2024-02-17T12:41:33.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.701+0000 7fa754abb700 1 --2- 172.21.15.146:0/3227036814 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa738038f80 0x7fa73803b440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:33.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.701+0000 7fa754abb700 1 --2- 172.21.15.146:0/3227036814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7501027f0 0x7fa750196ff0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:33.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.701+0000 7fa754abb700 1 -- 172.21.15.146:0/3227036814 >> 172.21.15.146:0/3227036814 conn(0x7fa7500fc020 msgr2=0x7fa7501035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:33.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.702+0000 7fa754abb700 1 -- 172.21.15.146:0/3227036814 shutdown_connections 2024-02-17T12:41:33.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:33.702+0000 7fa754abb700 1 -- 172.21.15.146:0/3227036814 wait complete. 2024-02-17T12:41:33.698 INFO:teuthology.orchestra.run.smithi146.stderr:dumped monmap epoch 1 2024-02-17T12:41:33.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:33 smithi146 systemd[1]: Starting Ceph mon.c for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:41:34.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:34 smithi018 ceph-mon[32625]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:34.532 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:34 smithi018 ceph-mon[32625]: from='client.? 172.21.15.146:0/3227036814' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T12:41:34.601 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 podman[36435]: 2024-02-17 12:41:34.39623233 +0000 UTC m=+0.095563617 container create a924da2e36e4e94fd58fb612193f6b35d6c3fb2a54118be7a9d531088f8bccae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-17T12:41:34.602 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 podman[36435]: 2024-02-17 12:41:34.31881184 +0000 UTC m=+0.018143103 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:41:34.602 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 podman[36435]: 2024-02-17 12:41:34.606824009 +0000 UTC m=+0.306155251 container init a924da2e36e4e94fd58fb612193f6b35d6c3fb2a54118be7a9d531088f8bccae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-17T12:41:34.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 podman[36435]: 2024-02-17 12:41:34.609690871 +0000 UTC m=+0.309022111 container start a924da2e36e4e94fd58fb612193f6b35d6c3fb2a54118be7a9d531088f8bccae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T12:41:34.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: set uid:gid to 167:167 (ceph:ceph) 2024-02-17T12:41:34.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev), process ceph-mon, pid 2 2024-02-17T12:41:34.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: pidfile_write: ignore empty --pid-file 2024-02-17T12:41:34.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: load: jerasure load: lrc 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: RocksDB version: 7.9.2 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Git sha 0 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Compile date 2024-02-16 17:52:22 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: DB SUMMARY 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: DB Session ID: C8FD2MVDLF3OEAFOCAIM 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: CURRENT file: CURRENT 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: IDENTITY file: IDENTITY 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.error_if_exists: 0 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.create_if_missing: 0 2024-02-17T12:41:34.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.paranoid_checks: 1 2024-02-17T12:41:34.957 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-17T12:41:34.957 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-17T12:41:34.957 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-17T12:41:34.957 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.env: 0x56463319aae0 2024-02-17T12:41:34.957 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.fs: PosixFileSystem 2024-02-17T12:41:34.957 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.info_log: 0x5646356458c0 2024-02-17T12:41:34.957 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_file_opening_threads: 16 2024-02-17T12:41:34.957 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.statistics: (nil) 2024-02-17T12:41:34.957 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.use_fsync: 0 2024-02-17T12:41:34.957 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_log_file_size: 0 2024-02-17T12:41:34.957 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-17T12:41:34.957 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.keep_log_file_num: 1000 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.recycle_log_file_num: 0 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.allow_fallocate: 1 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.allow_mmap_reads: 0 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.allow_mmap_writes: 0 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.use_direct_reads: 0 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.create_missing_column_families: 0 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.db_log_dir: 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.wal_dir: 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-17T12:41:34.958 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-17T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-17T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-17T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.advise_random_on_open: 1 2024-02-17T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.db_write_buffer_size: 0 2024-02-17T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.write_buffer_manager: 0x564635619400 2024-02-17T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-17T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-17T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-17T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.rate_limiter: (nil) 2024-02-17T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-17T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.wal_recovery_mode: 2 2024-02-17T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.enable_thread_tracking: 0 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.enable_pipelined_write: 0 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.unordered_write: 0 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.row_cache: None 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.wal_filter: None 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.allow_ingest_behind: 0 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.two_write_queues: 0 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.manual_wal_flush: 0 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.wal_compression: 0 2024-02-17T12:41:34.960 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.atomic_flush: 0 2024-02-17T12:41:34.961 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-17T12:41:34.961 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-17T12:41:34.961 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-17T12:41:34.962 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.log_readahead_size: 0 2024-02-17T12:41:34.962 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-17T12:41:34.962 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.best_efforts_recovery: 0 2024-02-17T12:41:34.962 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.allow_data_in_errors: 0 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.db_host_id: __hostname__ 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_background_jobs: 2 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_background_compactions: -1 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_subcompactions: 1 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_total_wal_size: 0 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-17T12:41:34.963 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_open_files: -1 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bytes_per_sync: 0 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_readahead_size: 0 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_background_flushes: -1 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Compression algorithms supported: 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: kZSTD supported: 0 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: kXpressCompression supported: 0 2024-02-17T12:41:34.964 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: kLZ4HCCompression supported: 1 2024-02-17T12:41:34.965 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: kZlibCompression supported: 1 2024-02-17T12:41:34.965 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: kSnappyCompression supported: 1 2024-02-17T12:41:34.965 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: kLZ4Compression supported: 1 2024-02-17T12:41:34.965 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: kBZip2Compression supported: 0 2024-02-17T12:41:34.965 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-17T12:41:34.965 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-17T12:41:34.965 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-17T12:41:34.965 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-17T12:41:34.965 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-17T12:41:34.965 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.merge_operator: 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_filter: None 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_filter_factory: None 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.sst_partitioner_factory: None 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5646356459c0) 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout: cache_index_and_filter_blocks: 1 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout: pin_top_level_index_and_filter: 1 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout: index_type: 0 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout: data_block_index_type: 0 2024-02-17T12:41:34.966 INFO:journalctl@ceph.mon.c.smithi146.stdout: index_shortening: 1 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: checksum: 4 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: no_block_cache: 0 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_cache: 0x564635649090 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_cache_name: BinnedLRUCache 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_cache_options: 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: capacity : 536870912 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: num_shard_bits : 4 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: strict_capacity_limit : 0 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: high_pri_pool_ratio: 0.000 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_cache_compressed: (nil) 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: persistent_cache: (nil) 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_size: 4096 2024-02-17T12:41:34.967 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_size_deviation: 10 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_restart_interval: 16 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: index_block_restart_interval: 1 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: metadata_block_size: 4096 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: partition_filters: 0 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: use_delta_encoding: 1 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: filter_policy: bloomfilter 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: whole_key_filtering: 1 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: verify_compression: 0 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: read_amp_bytes_per_bit: 0 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: format_version: 5 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: enable_index_compression: 1 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: block_align: 0 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: max_auto_readahead_size: 262144 2024-02-17T12:41:34.968 INFO:journalctl@ceph.mon.c.smithi146.stdout: prepopulate_block_cache: 0 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout: initial_auto_readahead_size: 8192 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout: num_file_reads_for_auto_readahead: 2 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.write_buffer_size: 33554432 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_write_buffer_number: 2 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compression: NoCompression 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bottommost_compression: Disabled 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.prefix_extractor: nullptr 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.num_levels: 7 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-17T12:41:34.969 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compression_opts.level: 32767 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compression_opts.strategy: 0 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-17T12:41:34.970 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compression_opts.enabled: false 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.target_file_size_base: 67108864 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-17T12:41:34.971 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.arena_block_size: 1048576 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.disable_auto_compactions: 0 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-17T12:41:34.972 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.table_properties_collectors: 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.inplace_update_support: 0 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.bloom_locality: 0 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.max_successive_merges: 0 2024-02-17T12:41:34.973 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.paranoid_file_checks: 0 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.force_consistency_checks: 1 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.report_bg_io_stats: 0 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.ttl: 2592000 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.enable_blob_files: false 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.min_blob_size: 0 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.blob_file_size: 268435456 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-17T12:41:34.974 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.blob_file_starting_level: 0 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e2d26eba-0d97-447a-9c86-360eaee881a1 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708173694639294, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708173694639726, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708173694, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e2d26eba-0d97-447a-9c86-360eaee881a1", "db_session_id": "C8FD2MVDLF3OEAFOCAIM", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708173694639807, "job": 1, "event": "recovery_finished"} 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-17T12:41:34.975 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56463573a000 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: DB pointer 0x564635724000 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout: ** DB Stats ** 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout: 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout: ** Compaction Stats [default] ** 2024-02-17T12:41:34.976 INFO:journalctl@ceph.mon.c.smithi146.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: ** Compaction Stats [default] ** 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-17T12:41:34.977 INFO:journalctl@ceph.mon.c.smithi146.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout: Block cache BinnedLRUCache@0x564635649090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 7e-06 secs_since: 0 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout: 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: using public_addr v2:172.21.15.146:0/0 -> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: starting mon.c rank -1 at public addrs [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] at bind addrs [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:41:34.978 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(???) e0 preinit fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).mds e1 new map 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout: e1 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout: legacy client fscid: -1 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout: 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout: No filesystems configured 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-17T12:41:34.979 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: Added host smithi146 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:41:34.980 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/517340665' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/517340665' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi146", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.981 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: Updating smithi146:/etc/ceph/ceph.conf 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi018:172.21.15.18=a;smithi119:172.21.15.119=b;smithi146:172.21.15.146=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: Saving service mon spec with placement smithi018:172.21.15.18=a;smithi119:172.21.15.119=b;smithi146:172.21.15.146=c;count:3 2024-02-17T12:41:34.982 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: Deploying daemon mon.c on smithi146 2024-02-17T12:41:34.983 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:34.983 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:34.983 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: from='client.? 172.21.15.146:0/3227036814' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T12:41:34.983 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 ceph-mon[36454]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-17T12:41:34.983 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 bash[36435]: a924da2e36e4e94fd58fb612193f6b35d6c3fb2a54118be7a9d531088f8bccae 2024-02-17T12:41:34.983 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:34 smithi146 systemd[1]: Started Ceph mon.c for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:41:35.277 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-17T12:41:35.278 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-17T12:41:35.513 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:41:40.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:40.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:41:40.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: mon.a calling monitor election 2024-02-17T12:41:40.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:40.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:40.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:40.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:40.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: mon.c calling monitor election 2024-02-17T12:41:40.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:40.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:40.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:40.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:40.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-17T12:41:40.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: monmap e2: 2 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],c=[v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T12:41:40.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: fsmap 2024-02-17T12:41:40.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T12:41:40.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: mgrmap e13: x(active, since 49s) 2024-02-17T12:41:40.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: overall HEALTH_OK 2024-02-17T12:41:40.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:40.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:40.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:40.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:40.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:39 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:40.044 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:40.044 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:41:40.044 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: mon.a calling monitor election 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: mon.c calling monitor election 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: monmap e2: 2 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],c=[v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: fsmap 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T12:41:40.045 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: mgrmap e13: x(active, since 49s) 2024-02-17T12:41:40.046 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: overall HEALTH_OK 2024-02-17T12:41:40.046 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:40.046 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:40.046 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:40.046 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:40.046 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:39 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.047+0000 7fca57933700 1 -- 172.21.15.146:0/1768536851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca3c0057d0 msgr2=0x7fca3c007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.047+0000 7fca57933700 1 --2- 172.21.15.146:0/1768536851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca3c0057d0 0x7fca3c007c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca50100570 tx=0x7fca4000feb0 comp rx=0 tx=0).stop 2024-02-17T12:41:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.048+0000 7fca57933700 1 -- 172.21.15.146:0/1768536851 shutdown_connections 2024-02-17T12:41:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.048+0000 7fca57933700 1 --2- 172.21.15.146:0/1768536851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca3c0057d0 0x7fca3c007c60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.048+0000 7fca57933700 1 -- 172.21.15.146:0/1768536851 >> 172.21.15.146:0/1768536851 conn(0x7fca500fc110 msgr2=0x7fca500fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.048+0000 7fca57933700 1 -- 172.21.15.146:0/1768536851 shutdown_connections 2024-02-17T12:41:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.048+0000 7fca57933700 1 -- 172.21.15.146:0/1768536851 wait complete. 2024-02-17T12:41:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.048+0000 7fca57933700 1 Processor -- start 2024-02-17T12:41:40.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca57933700 1 -- start start 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca57933700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca3c0057d0 0x7fca50199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca57933700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca501997a0 0x7fca5019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca57933700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca50199d30 con 0x7fca501997a0 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca57933700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca50199ea0 con 0x7fca3c0057d0 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca556cf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca3c0057d0 0x7fca50199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca556cf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca3c0057d0 0x7fca50199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59900/0 (socket says 172.21.15.146:59900) 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca556cf700 1 -- 172.21.15.146:0/3296752393 learned_addr learned my addr 172.21.15.146:0/3296752393 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca54ece700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca501997a0 0x7fca5019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca556cf700 1 -- 172.21.15.146:0/3296752393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca3c0057d0 msgr2=0x7fca50199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca556cf700 1 -- 172.21.15.146:0/3296752393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca3c0057d0 msgr2=0x7fca50199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca556cf700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca3c0057d0 0x7fca50199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca556cf700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca3c0057d0 0x7fca50199260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T12:41:40.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca54ece700 1 -- 172.21.15.146:0/3296752393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca3c0057d0 msgr2=0x7fca50199260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:41:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca54ece700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca3c0057d0 0x7fca50199260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.049+0000 7fca54ece700 1 -- 172.21.15.146:0/3296752393 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4000f900 con 0x7fca501997a0 2024-02-17T12:41:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.050+0000 7fca54ece700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca501997a0 0x7fca5019db30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00b4d0 tx=0x7fca4c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.050+0000 7fca467fc700 1 -- 172.21.15.146:0/3296752393 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fca4c011630 con 0x7fca501997a0 2024-02-17T12:41:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.050+0000 7fca467fc700 1 -- 172.21.15.146:0/3296752393 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca4c011c70 con 0x7fca501997a0 2024-02-17T12:41:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.050+0000 7fca467fc700 1 -- 172.21.15.146:0/3296752393 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fca4c010de0 con 0x7fca501997a0 2024-02-17T12:41:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.050+0000 7fca57933700 1 -- 172.21.15.146:0/3296752393 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5019e0d0 con 0x7fca501997a0 2024-02-17T12:41:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.050+0000 7fca57933700 1 -- 172.21.15.146:0/3296752393 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5019e620 con 0x7fca501997a0 2024-02-17T12:41:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.051+0000 7fca467fc700 1 -- 172.21.15.146:0/3296752393 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fca4c011790 con 0x7fca501997a0 2024-02-17T12:41:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.051+0000 7fca57933700 1 -- 172.21.15.146:0/3296752393 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5018edd0 con 0x7fca501997a0 2024-02-17T12:41:40.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.051+0000 7fca467fc700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fca34038d70 0x7fca3403b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:40.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.051+0000 7fca467fc700 1 -- 172.21.15.146:0/3296752393 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fca4c04b7a0 con 0x7fca501997a0 2024-02-17T12:41:40.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.052+0000 7fca556cf700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fca34038d70 0x7fca3403b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:40.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.052+0000 7fca556cf700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fca34038d70 0x7fca3403b230 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca40005b40 tx=0x7fca40005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:40.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.057+0000 7fca467fc700 1 -- 172.21.15.146:0/3296752393 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca4c01a920 con 0x7fca501997a0 2024-02-17T12:41:40.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.328+0000 7fca57933700 1 -- 172.21.15.146:0/3296752393 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fca5019a5c0 con 0x7fca501997a0 2024-02-17T12:41:40.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.329+0000 7fca467fc700 1 -- 172.21.15.146:0/3296752393 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fca4c0187b0 con 0x7fca501997a0 2024-02-17T12:41:40.324 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:41:40.324 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":2,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","modified":"2024-02-17T12:41:34.662218Z","created":"2024-02-17T12:39:45.933431Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:3300","nonce":0},{"type":"v1","addr":"172.21.15.146:6789","nonce":0}]},"addr":"172.21.15.146:6789/0","public_addr":"172.21.15.146:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-17T12:41:40.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.331+0000 7fca57933700 1 -- 172.21.15.146:0/3296752393 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fca34038d70 msgr2=0x7fca3403b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:40.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.331+0000 7fca57933700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fca34038d70 0x7fca3403b230 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca40005b40 tx=0x7fca40005a90 comp rx=0 tx=0).stop 2024-02-17T12:41:40.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.331+0000 7fca57933700 1 -- 172.21.15.146:0/3296752393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca501997a0 msgr2=0x7fca5019db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:40.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.331+0000 7fca57933700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca501997a0 0x7fca5019db30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00b4d0 tx=0x7fca4c00b890 comp rx=0 tx=0).stop 2024-02-17T12:41:40.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.332+0000 7fca57933700 1 -- 172.21.15.146:0/3296752393 shutdown_connections 2024-02-17T12:41:40.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.332+0000 7fca57933700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fca34038d70 0x7fca3403b230 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:40.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.332+0000 7fca57933700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca3c0057d0 0x7fca50199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:40.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.332+0000 7fca57933700 1 --2- 172.21.15.146:0/3296752393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca501997a0 0x7fca5019db30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:40.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.332+0000 7fca57933700 1 -- 172.21.15.146:0/3296752393 >> 172.21.15.146:0/3296752393 conn(0x7fca500fc110 msgr2=0x7fca500fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:40.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.332+0000 7fca57933700 1 -- 172.21.15.146:0/3296752393 shutdown_connections 2024-02-17T12:41:40.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:40.332+0000 7fca57933700 1 -- 172.21.15.146:0/3296752393 wait complete. 2024-02-17T12:41:40.328 INFO:teuthology.orchestra.run.smithi146.stderr:dumped monmap epoch 2 2024-02-17T12:41:40.921 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:40 smithi146 ceph-mon[36454]: Deploying daemon mon.b on smithi119 2024-02-17T12:41:40.921 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:40 smithi146 ceph-mon[36454]: from='client.? 172.21.15.146:0/3296752393' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T12:41:40.921 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:41.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:40 smithi018 ceph-mon[32625]: Deploying daemon mon.b on smithi119 2024-02-17T12:41:41.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:40 smithi018 ceph-mon[32625]: from='client.? 172.21.15.146:0/3296752393' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T12:41:41.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:41.947 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-17T12:41:41.947 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-17T12:41:42.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:41 smithi018 ceph-mon[32625]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:42.183 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:41:42.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:41 smithi146 ceph-mon[36454]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 podman[36169]: 2024-02-17 12:41:43.542818709 +0000 UTC m=+0.364688118 container init 42ee1f6ae41230c3c762c51bd55e827a0753e767db96901b5fb37977b9174d60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-b, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-17T12:41:43.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 podman[36169]: 2024-02-17 12:41:43.546273864 +0000 UTC m=+0.368143265 container start 42ee1f6ae41230c3c762c51bd55e827a0753e767db96901b5fb37977b9174d60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-02-17T12:41:43.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.593+0000 7f0eff5ff700 1 -- 172.21.15.146:0/2253236964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ef8106670 msgr2=0x7f0ef8106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:43.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.593+0000 7f0eff5ff700 1 --2- 172.21.15.146:0/2253236964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ef8106670 0x7f0ef8106a50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009910 tx=0x7f0ef4009c20 comp rx=0 tx=0).stop 2024-02-17T12:41:43.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.593+0000 7f0eff5ff700 1 -- 172.21.15.146:0/2253236964 shutdown_connections 2024-02-17T12:41:43.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.593+0000 7f0eff5ff700 1 --2- 172.21.15.146:0/2253236964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ef8106670 0x7f0ef8106a50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:43.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.593+0000 7f0eff5ff700 1 -- 172.21.15.146:0/2253236964 >> 172.21.15.146:0/2253236964 conn(0x7f0ef80fc180 msgr2=0x7f0ef80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:43.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.593+0000 7f0eff5ff700 1 -- 172.21.15.146:0/2253236964 shutdown_connections 2024-02-17T12:41:43.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.593+0000 7f0eff5ff700 1 -- 172.21.15.146:0/2253236964 wait complete. 2024-02-17T12:41:43.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.593+0000 7f0eff5ff700 1 Processor -- start 2024-02-17T12:41:43.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0eff5ff700 1 -- start start 2024-02-17T12:41:43.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0eff5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8106670 0x7f0ef81980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:43.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0eff5ff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ef81985e0 0x7f0ef8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:43.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0eff5ff700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef806ae00 con 0x7f0ef81985e0 2024-02-17T12:41:43.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0eff5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef81a1110 con 0x7f0ef8106670 2024-02-17T12:41:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0efd39b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8106670 0x7f0ef81980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0efd39b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8106670 0x7f0ef81980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59902/0 (socket says 172.21.15.146:59902) 2024-02-17T12:41:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0efd39b700 1 -- 172.21.15.146:0/3551439408 learned_addr learned my addr 172.21.15.146:0/3551439408 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-02-17T12:41:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0efcb9a700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ef81985e0 0x7f0ef8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0efd39b700 1 -- 172.21.15.146:0/3551439408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8106670 msgr2=0x7f0ef81980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-17T12:41:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0efd39b700 1 -- 172.21.15.146:0/3551439408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8106670 msgr2=0x7f0ef81980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-17T12:41:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0efd39b700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8106670 0x7f0ef81980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-17T12:41:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0efd39b700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8106670 0x7f0ef81980a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T12:41:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0efcb9a700 1 -- 172.21.15.146:0/3551439408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8106670 msgr2=0x7f0ef81980a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:41:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0efcb9a700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8106670 0x7f0ef81980a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:43.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0efcb9a700 1 -- 172.21.15.146:0/3551439408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef40095f0 con 0x7f0ef81985e0 2024-02-17T12:41:43.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.594+0000 7f0efcb9a700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ef81985e0 0x7f0ef8198a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800b4d0 tx=0x7f0ee800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:43.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.595+0000 7f0eee7fc700 1 -- 172.21.15.146:0/3551439408 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0ee8011630 con 0x7f0ef81985e0 2024-02-17T12:41:43.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.595+0000 7f0eff5ff700 1 -- 172.21.15.146:0/3551439408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef81a13a0 con 0x7f0ef81985e0 2024-02-17T12:41:43.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.595+0000 7f0eee7fc700 1 -- 172.21.15.146:0/3551439408 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ee8011c70 con 0x7f0ef81985e0 2024-02-17T12:41:43.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.595+0000 7f0eee7fc700 1 -- 172.21.15.146:0/3551439408 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0ee8010de0 con 0x7f0ef81985e0 2024-02-17T12:41:43.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.595+0000 7f0eff5ff700 1 -- 172.21.15.146:0/3551439408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef81a18f0 con 0x7f0ef81985e0 2024-02-17T12:41:43.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.596+0000 7f0eff5ff700 1 -- 172.21.15.146:0/3551439408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef80632a0 con 0x7f0ef81985e0 2024-02-17T12:41:43.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.596+0000 7f0eee7fc700 1 -- 172.21.15.146:0/3551439408 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f0ee80104a0 con 0x7f0ef81985e0 2024-02-17T12:41:43.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.596+0000 7f0eee7fc700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0ee4038d20 0x7f0ee403b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:43.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.596+0000 7f0eee7fc700 1 -- 172.21.15.146:0/3551439408 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0ee804b6e0 con 0x7f0ef81985e0 2024-02-17T12:41:43.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.596+0000 7f0efd39b700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0ee4038d20 0x7f0ee403b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:43.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.597+0000 7f0efd39b700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0ee4038d20 0x7f0ee403b1e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4000c00 tx=0x7f0ef400b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:43.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.602+0000 7f0eee7fc700 1 -- 172.21.15.146:0/3551439408 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ee8011de0 con 0x7f0ef81985e0 2024-02-17T12:41:43.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.602+0000 7f0eee7fc700 1 -- 172.21.15.146:0/3551439408 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8011de0 con 0x7f0ef81985e0 2024-02-17T12:41:43.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: set uid:gid to 167:167 (ceph:ceph) 2024-02-17T12:41:43.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev), process ceph-mon, pid 2 2024-02-17T12:41:43.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pidfile_write: ignore empty --pid-file 2024-02-17T12:41:43.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: load: jerasure load: lrc 2024-02-17T12:41:43.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: RocksDB version: 7.9.2 2024-02-17T12:41:43.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Git sha 0 2024-02-17T12:41:43.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Compile date 2024-02-16 17:52:22 2024-02-17T12:41:43.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: DB SUMMARY 2024-02-17T12:41:43.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: DB Session ID: TCLP2HLIQEGQRUYGN2UT 2024-02-17T12:41:43.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: CURRENT file: CURRENT 2024-02-17T12:41:43.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: IDENTITY file: IDENTITY 2024-02-17T12:41:43.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.error_if_exists: 0 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.create_if_missing: 0 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.paranoid_checks: 1 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.env: 0x5649f6353ae0 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.fs: PosixFileSystem 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.info_log: 0x5649f6f918c0 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_file_opening_threads: 16 2024-02-17T12:41:43.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.statistics: (nil) 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.use_fsync: 0 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_log_file_size: 0 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.keep_log_file_num: 1000 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.recycle_log_file_num: 0 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.allow_fallocate: 1 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.allow_mmap_reads: 0 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.allow_mmap_writes: 0 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.use_direct_reads: 0 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.create_missing_column_families: 0 2024-02-17T12:41:43.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.db_log_dir: 2024-02-17T12:41:43.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.wal_dir: 2024-02-17T12:41:43.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-17T12:41:43.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-17T12:41:43.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-17T12:41:43.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-17T12:41:43.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-17T12:41:43.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-17T12:41:43.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.advise_random_on_open: 1 2024-02-17T12:41:43.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.db_write_buffer_size: 0 2024-02-17T12:41:43.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.write_buffer_manager: 0x5649f6f65400 2024-02-17T12:41:43.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-17T12:41:43.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-17T12:41:43.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-17T12:41:43.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.rate_limiter: (nil) 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.wal_recovery_mode: 2 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.enable_thread_tracking: 0 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.enable_pipelined_write: 0 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.unordered_write: 0 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.row_cache: None 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.wal_filter: None 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-17T12:41:43.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.allow_ingest_behind: 0 2024-02-17T12:41:43.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.two_write_queues: 0 2024-02-17T12:41:43.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.manual_wal_flush: 0 2024-02-17T12:41:43.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.wal_compression: 0 2024-02-17T12:41:43.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.atomic_flush: 0 2024-02-17T12:41:43.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-17T12:41:43.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-17T12:41:43.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-17T12:41:43.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.log_readahead_size: 0 2024-02-17T12:41:43.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-17T12:41:43.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.best_efforts_recovery: 0 2024-02-17T12:41:43.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-17T12:41:43.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-17T12:41:43.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.allow_data_in_errors: 0 2024-02-17T12:41:43.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.db_host_id: __hostname__ 2024-02-17T12:41:43.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-17T12:41:43.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_background_jobs: 2 2024-02-17T12:41:43.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_background_compactions: -1 2024-02-17T12:41:43.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_subcompactions: 1 2024-02-17T12:41:43.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-17T12:41:43.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-17T12:41:43.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-17T12:41:43.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_total_wal_size: 0 2024-02-17T12:41:43.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-17T12:41:43.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_open_files: -1 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bytes_per_sync: 0 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_readahead_size: 0 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_background_flushes: -1 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Compression algorithms supported: 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: kZSTD supported: 0 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: kXpressCompression supported: 0 2024-02-17T12:41:43.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: kLZ4HCCompression supported: 1 2024-02-17T12:41:43.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: kZlibCompression supported: 1 2024-02-17T12:41:43.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: kSnappyCompression supported: 1 2024-02-17T12:41:43.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: kLZ4Compression supported: 1 2024-02-17T12:41:43.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: kBZip2Compression supported: 0 2024-02-17T12:41:43.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-17T12:41:43.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-17T12:41:43.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-17T12:41:43.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-17T12:41:43.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-17T12:41:43.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.merge_operator: 2024-02-17T12:41:43.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_filter: None 2024-02-17T12:41:43.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_filter_factory: None 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.sst_partitioner_factory: None 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5649f6f919c0) 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout: cache_index_and_filter_blocks: 1 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout: pin_top_level_index_and_filter: 1 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout: index_type: 0 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout: data_block_index_type: 0 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout: index_shortening: 1 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-17T12:41:43.859 INFO:journalctl@ceph.mon.b.smithi119.stdout: checksum: 4 2024-02-17T12:41:43.860 INFO:journalctl@ceph.mon.b.smithi119.stdout: no_block_cache: 0 2024-02-17T12:41:43.860 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache: 0x5649f6f95090 2024-02-17T12:41:43.860 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache_name: BinnedLRUCache 2024-02-17T12:41:43.860 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache_options: 2024-02-17T12:41:43.860 INFO:journalctl@ceph.mon.b.smithi119.stdout: capacity : 536870912 2024-02-17T12:41:43.860 INFO:journalctl@ceph.mon.b.smithi119.stdout: num_shard_bits : 4 2024-02-17T12:41:43.860 INFO:journalctl@ceph.mon.b.smithi119.stdout: strict_capacity_limit : 0 2024-02-17T12:41:43.860 INFO:journalctl@ceph.mon.b.smithi119.stdout: high_pri_pool_ratio: 0.000 2024-02-17T12:41:43.860 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache_compressed: (nil) 2024-02-17T12:41:43.860 INFO:journalctl@ceph.mon.b.smithi119.stdout: persistent_cache: (nil) 2024-02-17T12:41:43.860 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_size: 4096 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_size_deviation: 10 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_restart_interval: 16 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: index_block_restart_interval: 1 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: metadata_block_size: 4096 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: partition_filters: 0 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: use_delta_encoding: 1 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: filter_policy: bloomfilter 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: whole_key_filtering: 1 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: verify_compression: 0 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: read_amp_bytes_per_bit: 0 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: format_version: 5 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: enable_index_compression: 1 2024-02-17T12:41:43.861 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_align: 0 2024-02-17T12:41:43.862 INFO:journalctl@ceph.mon.b.smithi119.stdout: max_auto_readahead_size: 262144 2024-02-17T12:41:43.862 INFO:journalctl@ceph.mon.b.smithi119.stdout: prepopulate_block_cache: 0 2024-02-17T12:41:43.862 INFO:journalctl@ceph.mon.b.smithi119.stdout: initial_auto_readahead_size: 8192 2024-02-17T12:41:43.862 INFO:journalctl@ceph.mon.b.smithi119.stdout: num_file_reads_for_auto_readahead: 2 2024-02-17T12:41:43.862 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.write_buffer_size: 33554432 2024-02-17T12:41:43.862 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_write_buffer_number: 2 2024-02-17T12:41:43.862 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compression: NoCompression 2024-02-17T12:41:43.862 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bottommost_compression: Disabled 2024-02-17T12:41:43.862 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.prefix_extractor: nullptr 2024-02-17T12:41:43.862 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-17T12:41:43.862 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.num_levels: 7 2024-02-17T12:41:43.862 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-17T12:41:43.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compression_opts.level: 32767 2024-02-17T12:41:43.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compression_opts.strategy: 0 2024-02-17T12:41:43.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-17T12:41:43.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-17T12:41:43.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-17T12:41:43.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-17T12:41:43.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compression_opts.enabled: false 2024-02-17T12:41:43.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-17T12:41:43.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-17T12:41:43.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-17T12:41:43.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-17T12:41:43.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.target_file_size_base: 67108864 2024-02-17T12:41:43.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-17T12:41:43.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-17T12:41:43.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-17T12:41:43.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-17T12:41:43.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-17T12:41:43.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-17T12:41:43.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-17T12:41:43.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-17T12:41:43.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-17T12:41:43.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-17T12:41:43.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-17T12:41:43.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-17T12:41:43.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-17T12:41:43.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-17T12:41:43.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.arena_block_size: 1048576 2024-02-17T12:41:43.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-17T12:41:43.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-17T12:41:43.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.disable_auto_compactions: 0 2024-02-17T12:41:43.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-17T12:41:43.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-17T12:41:43.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-17T12:41:43.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-17T12:41:43.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-17T12:41:43.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-17T12:41:43.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-17T12:41:43.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-17T12:41:43.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-17T12:41:43.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-17T12:41:43.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.table_properties_collectors: 2024-02-17T12:41:43.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.inplace_update_support: 0 2024-02-17T12:41:43.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-17T12:41:43.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-17T12:41:43.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-17T12:41:43.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-17T12:41:43.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.bloom_locality: 0 2024-02-17T12:41:43.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.max_successive_merges: 0 2024-02-17T12:41:43.868 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-17T12:41:43.868 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.paranoid_file_checks: 0 2024-02-17T12:41:43.868 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.force_consistency_checks: 1 2024-02-17T12:41:43.868 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.report_bg_io_stats: 0 2024-02-17T12:41:43.868 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.ttl: 2592000 2024-02-17T12:41:43.868 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-17T12:41:43.868 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-17T12:41:43.868 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-17T12:41:43.868 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.enable_blob_files: false 2024-02-17T12:41:43.868 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.min_blob_size: 0 2024-02-17T12:41:43.868 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.blob_file_size: 268435456 2024-02-17T12:41:43.868 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-17T12:41:43.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-17T12:41:43.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-17T12:41:43.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-17T12:41:43.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-17T12:41:43.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.blob_file_starting_level: 0 2024-02-17T12:41:43.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-17T12:41:43.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-17T12:41:43.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-17T12:41:43.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 868145c8-11f4-4f90-addd-3785b18176e0 2024-02-17T12:41:43.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708173703575377, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-17T12:41:43.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-17T12:41:43.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708173703575788, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708173703, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "868145c8-11f4-4f90-addd-3785b18176e0", "db_session_id": "TCLP2HLIQEGQRUYGN2UT", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-17T12:41:43.870 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708173703575865, "job": 1, "event": "recovery_finished"} 2024-02-17T12:41:43.870 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-17T12:41:43.870 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-17T12:41:43.870 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5649f7086000 2024-02-17T12:41:43.870 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: DB pointer 0x5649f7070000 2024-02-17T12:41:43.870 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-17T12:41:43.870 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-17T12:41:43.870 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** DB Stats ** 2024-02-17T12:41:43.870 INFO:journalctl@ceph.mon.b.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-17T12:41:43.870 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-17T12:41:43.870 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-17T12:41:43.870 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-17T12:41:43.871 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-17T12:41:43.871 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-17T12:41:43.871 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-17T12:41:43.871 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-02-17T12:41:43.871 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** Compaction Stats [default] ** 2024-02-17T12:41:43.871 INFO:journalctl@ceph.mon.b.smithi119.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-17T12:41:43.871 INFO:journalctl@ceph.mon.b.smithi119.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-17T12:41:43.871 INFO:journalctl@ceph.mon.b.smithi119.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-17T12:41:43.871 INFO:journalctl@ceph.mon.b.smithi119.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-17T12:41:43.871 INFO:journalctl@ceph.mon.b.smithi119.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-17T12:41:43.871 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-02-17T12:41:43.871 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** Compaction Stats [default] ** 2024-02-17T12:41:43.872 INFO:journalctl@ceph.mon.b.smithi119.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-17T12:41:43.872 INFO:journalctl@ceph.mon.b.smithi119.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-17T12:41:43.872 INFO:journalctl@ceph.mon.b.smithi119.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-17T12:41:43.872 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-02-17T12:41:43.872 INFO:journalctl@ceph.mon.b.smithi119.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-17T12:41:43.872 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-02-17T12:41:43.872 INFO:journalctl@ceph.mon.b.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-17T12:41:43.872 INFO:journalctl@ceph.mon.b.smithi119.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-17T12:41:43.872 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-17T12:41:43.872 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-17T12:41:43.872 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-17T12:41:43.872 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-17T12:41:43.873 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-17T12:41:43.873 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-17T12:41:43.873 INFO:journalctl@ceph.mon.b.smithi119.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-17T12:41:43.873 INFO:journalctl@ceph.mon.b.smithi119.stdout: Block cache BinnedLRUCache@0x5649f6f95090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-17T12:41:43.873 INFO:journalctl@ceph.mon.b.smithi119.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-17T12:41:43.873 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-02-17T12:41:43.873 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-17T12:41:43.873 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-17T12:41:43.873 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: using public_addr v2:172.21.15.119:0/0 -> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-02-17T12:41:43.873 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: starting mon.b rank -1 at public addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] at bind addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:41:43.873 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(???) e0 preinit fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:41:43.873 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).mds e1 new map 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout: e1 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout: legacy client fscid: -1 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout: No filesystems configured 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-17T12:41:43.874 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: Added host smithi146 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:41:43.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/517340665' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-17T12:41:43.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/517340665' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-17T12:41:43.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T12:41:43.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi146", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: Updating smithi146:/etc/ceph/ceph.conf 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:43.877 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi018:172.21.15.18=a;smithi119:172.21.15.119=b;smithi146:172.21.15.146=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: Saving service mon spec with placement smithi018:172.21.15.18=a;smithi119:172.21.15.119=b;smithi146:172.21.15.146=c;count:3 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: Deploying daemon mon.c on smithi146 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='client.? 172.21.15.146:0/3227036814' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:41:43.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.a calling monitor election 2024-02-17T12:41:43.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:43.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:43.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:43.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.c calling monitor election 2024-02-17T12:41:43.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:43.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:43.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:43.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-17T12:41:43.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: monmap e2: 2 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],c=[v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T12:41:43.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: fsmap 2024-02-17T12:41:43.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T12:41:43.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mgrmap e13: x(active, since 49s) 2024-02-17T12:41:43.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: overall HEALTH_OK 2024-02-17T12:41:43.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:43.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:43.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:43.881 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: Deploying daemon mon.b on smithi119 2024-02-17T12:41:43.881 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='client.? 172.21.15.146:0/3296752393' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T12:41:43.881 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:43.881 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:43.881 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 ceph-mon[36188]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-17T12:41:43.881 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 bash[36169]: 42ee1f6ae41230c3c762c51bd55e827a0753e767db96901b5fb37977b9174d60 2024-02-17T12:41:43.881 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:43 smithi119 systemd[1]: Started Ceph mon.b for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:41:43.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:43.874+0000 7f0eff5ff700 1 -- 172.21.15.146:0/3551439408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0ef810abf0 con 0x7f0ef81985e0 2024-02-17T12:41:48.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:48.637+0000 7f0eee7fc700 1 -- 172.21.15.146:0/3551439408 <== mon.0 v2:172.21.15.18:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1285 (secure 0 0 0) 0x7f0ee800d1c0 con 0x7f0ef81985e0 2024-02-17T12:41:48.634 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-02-17T12:41:48.635 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":3,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","modified":"2024-02-17T12:41:43.602277Z","created":"2024-02-17T12:39:45.933431Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:3300","nonce":0},{"type":"v1","addr":"172.21.15.146:6789","nonce":0}]},"addr":"172.21.15.146:6789/0","public_addr":"172.21.15.146:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-17T12:41:48.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:48.640+0000 7f0eff5ff700 1 -- 172.21.15.146:0/3551439408 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0ee4038d20 msgr2=0x7f0ee403b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:48.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:48.640+0000 7f0eff5ff700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0ee4038d20 0x7f0ee403b1e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4000c00 tx=0x7f0ef400b580 comp rx=0 tx=0).stop 2024-02-17T12:41:48.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:48.640+0000 7f0eff5ff700 1 -- 172.21.15.146:0/3551439408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ef81985e0 msgr2=0x7f0ef8198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:48.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:48.640+0000 7f0eff5ff700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ef81985e0 0x7f0ef8198a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800b4d0 tx=0x7f0ee800b890 comp rx=0 tx=0).stop 2024-02-17T12:41:48.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:48.640+0000 7f0eff5ff700 1 -- 172.21.15.146:0/3551439408 shutdown_connections 2024-02-17T12:41:48.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:48.640+0000 7f0eff5ff700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0ee4038d20 0x7f0ee403b1e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:48.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:48.640+0000 7f0eff5ff700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ef8106670 0x7f0ef81980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:48.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:48.640+0000 7f0eff5ff700 1 --2- 172.21.15.146:0/3551439408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ef81985e0 0x7f0ef8198a60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:48.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:48.640+0000 7f0eff5ff700 1 -- 172.21.15.146:0/3551439408 >> 172.21.15.146:0/3551439408 conn(0x7f0ef80fc180 msgr2=0x7f0ef8102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:48.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:48.641+0000 7f0eff5ff700 1 -- 172.21.15.146:0/3551439408 shutdown_connections 2024-02-17T12:41:48.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:48.641+0000 7f0eff5ff700 1 -- 172.21.15.146:0/3551439408 wait complete. 2024-02-17T12:41:48.638 INFO:teuthology.orchestra.run.smithi146.stderr:dumped monmap epoch 3 2024-02-17T12:41:48.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:41:48.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: mon.a calling monitor election 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: mon.c calling monitor election 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='client.? 172.21.15.146:0/3551439408' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: mon.b calling monitor election 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-17T12:41:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: monmap e3: 3 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],b=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T12:41:48.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: fsmap 2024-02-17T12:41:48.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T12:41:48.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: mgrmap e13: x(active, since 58s) 2024-02-17T12:41:48.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: overall HEALTH_OK 2024-02-17T12:41:48.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:48.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:48.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:48.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:49.032 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:41:49.033 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.033 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:49.033 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: mon.a calling monitor election 2024-02-17T12:41:49.033 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: mon.c calling monitor election 2024-02-17T12:41:49.033 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='client.? 172.21.15.146:0/3551439408' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T12:41:49.033 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:49.033 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.033 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.033 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: mon.b calling monitor election 2024-02-17T12:41:49.033 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:49.033 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.033 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-17T12:41:49.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: monmap e3: 3 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],b=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T12:41:49.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: fsmap 2024-02-17T12:41:49.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T12:41:49.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: mgrmap e13: x(active, since 58s) 2024-02-17T12:41:49.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: overall HEALTH_OK 2024-02-17T12:41:49.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:49.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:49.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:49.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:41:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:41:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: mon.a calling monitor election 2024-02-17T12:41:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: mon.c calling monitor election 2024-02-17T12:41:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='client.? 172.21.15.146:0/3551439408' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: mon.b calling monitor election 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: monmap e3: 3 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],b=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: fsmap 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: mgrmap e13: x(active, since 58s) 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: overall HEALTH_OK 2024-02-17T12:41:49.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:49.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:49.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:49.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:49.236 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-17T12:41:49.237 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph config generate-minimal-conf 2024-02-17T12:41:49.488 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:41:49.879 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:49 smithi018 ceph-mon[32625]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:49.879 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:49 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:49.879 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:49 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:49.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:49 smithi146 ceph-mon[36454]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:49.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:49 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:49.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:49 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:50.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:49 smithi119 ceph-mon[36188]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:50.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:49 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:50.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:49 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:41:51.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.180+0000 7f886b928700 1 -- 172.21.15.18:0/3542357580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88641076d0 msgr2=0x7f8864107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:51.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.180+0000 7f886b928700 1 --2- 172.21.15.18:0/3542357580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88641076d0 0x7f8864107ab0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8854009910 tx=0x7f8854009c20 comp rx=0 tx=0).stop 2024-02-17T12:41:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.180+0000 7f886b928700 1 -- 172.21.15.18:0/3542357580 shutdown_connections 2024-02-17T12:41:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.180+0000 7f886b928700 1 --2- 172.21.15.18:0/3542357580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88641076d0 0x7f8864107ab0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.180+0000 7f886b928700 1 -- 172.21.15.18:0/3542357580 >> 172.21.15.18:0/3542357580 conn(0x7f88640fb7c0 msgr2=0x7f88640fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.181+0000 7f886b928700 1 -- 172.21.15.18:0/3542357580 shutdown_connections 2024-02-17T12:41:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.181+0000 7f886b928700 1 -- 172.21.15.18:0/3542357580 wait complete. 2024-02-17T12:41:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.181+0000 7f886b928700 1 Processor -- start 2024-02-17T12:41:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.181+0000 7f886b928700 1 -- start start 2024-02-17T12:41:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.181+0000 7f886b928700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88641076d0 0x7f8864100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.181+0000 7f886b928700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88641011c0 0x7f8864101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.181+0000 7f886b928700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88641a3370 0x7f88641a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.181+0000 7f886b928700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88641a5d60 con 0x7f88641a3370 2024-02-17T12:41:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.181+0000 7f886b928700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88641a5ed0 con 0x7f88641076d0 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.181+0000 7f886b928700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88641a6090 con 0x7f88641011c0 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.182+0000 7f8869ec5700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88641a3370 0x7f88641a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.182+0000 7f8869ec5700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88641a3370 0x7f88641a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44024/0 (socket says 172.21.15.18:44024) 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.182+0000 7f8869ec5700 1 -- 172.21.15.18:0/1119001364 learned_addr learned my addr 172.21.15.18:0/1119001364 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.182+0000 7f8868ec3700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88641011c0 0x7f8864101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.182+0000 7f88696c4700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88641076d0 0x7f8864100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.182+0000 7f8869ec5700 1 -- 172.21.15.18:0/1119001364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88641011c0 msgr2=0x7f8864101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.182+0000 7f8869ec5700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88641011c0 0x7f8864101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.182+0000 7f8869ec5700 1 -- 172.21.15.18:0/1119001364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88641076d0 msgr2=0x7f8864100c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.182+0000 7f8869ec5700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88641076d0 0x7f8864100c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.182+0000 7f8869ec5700 1 -- 172.21.15.18:0/1119001364 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88540095f0 con 0x7f88641a3370 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.182+0000 7f8869ec5700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88641a3370 0x7f88641a5760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f885c00ba50 tx=0x7f885c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.183+0000 7f885a7fc700 1 -- 172.21.15.18:0/1119001364 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c00c930 con 0x7f88641a3370 2024-02-17T12:41:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.183+0000 7f885a7fc700 1 -- 172.21.15.18:0/1119001364 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f885c014440 con 0x7f88641a3370 2024-02-17T12:41:51.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.183+0000 7f885a7fc700 1 -- 172.21.15.18:0/1119001364 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c007590 con 0x7f88641a3370 2024-02-17T12:41:51.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.183+0000 7f886b928700 1 -- 172.21.15.18:0/1119001364 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88641a6370 con 0x7f88641a3370 2024-02-17T12:41:51.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.183+0000 7f886b928700 1 -- 172.21.15.18:0/1119001364 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88641a6820 con 0x7f88641a3370 2024-02-17T12:41:51.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.184+0000 7f885a7fc700 1 -- 172.21.15.18:0/1119001364 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f885c0077b0 con 0x7f88641a3370 2024-02-17T12:41:51.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.184+0000 7f886b928700 1 -- 172.21.15.18:0/1119001364 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88641021b0 con 0x7f88641a3370 2024-02-17T12:41:51.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.184+0000 7f885a7fc700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8850039460 0x7f885003b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:51.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.184+0000 7f885a7fc700 1 -- 172.21.15.18:0/1119001364 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f885c04b250 con 0x7f88641a3370 2024-02-17T12:41:51.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.184+0000 7f88696c4700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8850039460 0x7f885003b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:51.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.184+0000 7f88696c4700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8850039460 0x7f885003b920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8854006010 tx=0x7f885400b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:51.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.189+0000 7f885a7fc700 1 -- 172.21.15.18:0/1119001364 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f885c01b8b0 con 0x7f88641a3370 2024-02-17T12:41:51.248 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.248 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.248 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.248 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: Updating smithi018:/etc/ceph/ceph.conf 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: Updating smithi119:/etc/ceph/ceph.conf 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: Updating smithi146:/etc/ceph/ceph.conf 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.249 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.250 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.250 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.250 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.250 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:51.250 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T12:41:51.250 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:51.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:51.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: Updating smithi018:/etc/ceph/ceph.conf 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: Updating smithi119:/etc/ceph/ceph.conf 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: Updating smithi146:/etc/ceph/ceph.conf 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.349 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.349 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:51.349 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T12:41:51.349 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:51.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.405+0000 7f886b928700 1 -- 172.21.15.18:0/1119001364 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f886410bb00 con 0x7f88641a3370 2024-02-17T12:41:51.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.405+0000 7f885a7fc700 1 -- 172.21.15.18:0/1119001364 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f885c01b2c0 con 0x7f88641a3370 2024-02-17T12:41:51.401 INFO:teuthology.orchestra.run.smithi018.stdout:# minimal ceph.conf for 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:41:51.402 INFO:teuthology.orchestra.run.smithi018.stdout:[global] 2024-02-17T12:41:51.402 INFO:teuthology.orchestra.run.smithi018.stdout: fsid = 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:41:51.402 INFO:teuthology.orchestra.run.smithi018.stdout: mon_host = [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] 2024-02-17T12:41:51.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.407+0000 7f886b928700 1 -- 172.21.15.18:0/1119001364 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8850039460 msgr2=0x7f885003b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:51.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.407+0000 7f886b928700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8850039460 0x7f885003b920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8854006010 tx=0x7f885400b560 comp rx=0 tx=0).stop 2024-02-17T12:41:51.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.407+0000 7f886b928700 1 -- 172.21.15.18:0/1119001364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88641a3370 msgr2=0x7f88641a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:51.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.407+0000 7f886b928700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88641a3370 0x7f88641a5760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f885c00ba50 tx=0x7f885c00bd60 comp rx=0 tx=0).stop 2024-02-17T12:41:51.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.407+0000 7f886b928700 1 -- 172.21.15.18:0/1119001364 shutdown_connections 2024-02-17T12:41:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.407+0000 7f886b928700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88641a3370 0x7f88641a5760 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.407+0000 7f886b928700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88641076d0 0x7f8864100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.407+0000 7f886b928700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8850039460 0x7f885003b920 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.407+0000 7f886b928700 1 --2- 172.21.15.18:0/1119001364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88641011c0 0x7f8864101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.407+0000 7f886b928700 1 -- 172.21.15.18:0/1119001364 >> 172.21.15.18:0/1119001364 conn(0x7f88640fb7c0 msgr2=0x7f88640fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.408+0000 7f886b928700 1 -- 172.21.15.18:0/1119001364 shutdown_connections 2024-02-17T12:41:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:41:51.408+0000 7f886b928700 1 -- 172.21.15.18:0/1119001364 wait complete. 2024-02-17T12:41:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: Updating smithi018:/etc/ceph/ceph.conf 2024-02-17T12:41:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: Updating smithi119:/etc/ceph/ceph.conf 2024-02-17T12:41:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: Updating smithi146:/etc/ceph/ceph.conf 2024-02-17T12:41:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:51.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:51.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T12:41:51.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:52.048 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-17T12:41:52.049 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:41:52.049 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.conf 2024-02-17T12:41:52.078 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:41:52.079 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:41:52.146 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-02-17T12:41:52.147 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.conf 2024-02-17T12:41:52.181 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-02-17T12:41:52.181 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:41:52.251 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-02-17T12:41:52.251 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.conf 2024-02-17T12:41:52.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:52 smithi018 ceph-mon[32625]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:52.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:52 smithi018 ceph-mon[32625]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:52.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:52 smithi018 ceph-mon[32625]: Updating smithi018:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:52.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:52 smithi018 ceph-mon[32625]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:52.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:52 smithi018 ceph-mon[32625]: Reconfiguring mon.a (unknown last config time)... 2024-02-17T12:41:52.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:52 smithi018 ceph-mon[32625]: Reconfiguring daemon mon.a on smithi018 2024-02-17T12:41:52.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:52 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1119001364' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:52.285 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-02-17T12:41:52.285 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:41:52.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:52 smithi119 ceph-mon[36188]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:52.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:52 smithi119 ceph-mon[36188]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:52.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:52 smithi119 ceph-mon[36188]: Updating smithi018:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:52.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:52 smithi119 ceph-mon[36188]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:52.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:52 smithi119 ceph-mon[36188]: Reconfiguring mon.a (unknown last config time)... 2024-02-17T12:41:52.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:52 smithi119 ceph-mon[36188]: Reconfiguring daemon mon.a on smithi018 2024-02-17T12:41:52.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:52 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1119001364' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:52.354 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:52 smithi146 ceph-mon[36454]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:52.354 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:52 smithi146 ceph-mon[36454]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:52.354 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:52 smithi146 ceph-mon[36454]: Updating smithi018:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:52.354 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:52 smithi146 ceph-mon[36454]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:41:52.354 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:52 smithi146 ceph-mon[36454]: Reconfiguring mon.a (unknown last config time)... 2024-02-17T12:41:52.354 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:52 smithi146 ceph-mon[36454]: Reconfiguring daemon mon.a on smithi018 2024-02-17T12:41:52.354 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:52 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1119001364' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:52.358 INFO:tasks.cephadm:Adding mgr.x on smithi018 2024-02-17T12:41:52.358 INFO:tasks.cephadm:Adding mgr.y on smithi119 2024-02-17T12:41:52.358 INFO:tasks.cephadm:Adding mgr.z on smithi146 2024-02-17T12:41:52.358 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch apply mgr '3;smithi018=x;smithi119=y;smithi146=z' 2024-02-17T12:41:52.632 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:41:53.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:53 smithi146 ceph-mon[36454]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:53.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:53 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:53.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:53 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:53.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:53 smithi146 ceph-mon[36454]: Reconfiguring mon.b (monmap changed)... 2024-02-17T12:41:53.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:53 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:53.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:53 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T12:41:53.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:53 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:53.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:53 smithi146 ceph-mon[36454]: Reconfiguring daemon mon.b on smithi119 2024-02-17T12:41:54.013 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:53 smithi119 ceph-mon[36188]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:54.013 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:53 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:54.013 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:53 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:54.014 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:53 smithi119 ceph-mon[36188]: Reconfiguring mon.b (monmap changed)... 2024-02-17T12:41:54.014 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:53 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:54.014 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:53 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T12:41:54.014 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:53 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:54.014 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:53 smithi119 ceph-mon[36188]: Reconfiguring daemon mon.b on smithi119 2024-02-17T12:41:54.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:53 smithi018 ceph-mon[32625]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:54.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:54.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:54.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:53 smithi018 ceph-mon[32625]: Reconfiguring mon.b (monmap changed)... 2024-02-17T12:41:54.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:54.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T12:41:54.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:54.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:53 smithi018 ceph-mon[32625]: Reconfiguring daemon mon.b on smithi119 2024-02-17T12:41:54.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.212+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/135615246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0d80ffa60 msgr2=0x7ff0d80ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:54.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.212+0000 7ff0dd5a7700 1 --2- 172.21.15.146:0/135615246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0d80ffa60 0x7ff0d80ffe80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009910 tx=0x7ff0cc009c20 comp rx=0 tx=0).stop 2024-02-17T12:41:54.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.212+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/135615246 shutdown_connections 2024-02-17T12:41:54.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.212+0000 7ff0dd5a7700 1 --2- 172.21.15.146:0/135615246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0d80ffa60 0x7ff0d80ffe80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:54.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.212+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/135615246 >> 172.21.15.146:0/135615246 conn(0x7ff0d80fb3c0 msgr2=0x7ff0d80fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:54.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.212+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/135615246 shutdown_connections 2024-02-17T12:41:54.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.212+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/135615246 wait complete. 2024-02-17T12:41:54.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.213+0000 7ff0dd5a7700 1 Processor -- start 2024-02-17T12:41:54.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.213+0000 7ff0dd5a7700 1 -- start start 2024-02-17T12:41:54.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.213+0000 7ff0dd5a7700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0d80ffa60 0x7ff0d8196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:54.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.213+0000 7ff0dd5a7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d8197360 0x7ff0d819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.213+0000 7ff0dd5a7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0d81977e0 0x7ff0d819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.213+0000 7ff0dd5a7700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d819f100 con 0x7ff0d80ffa60 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.213+0000 7ff0dd5a7700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d819f270 con 0x7ff0d8197360 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.213+0000 7ff0dd5a7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d819f430 con 0x7ff0d81977e0 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.213+0000 7ff0d77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0d81977e0 0x7ff0d819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.213+0000 7ff0d77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0d81977e0 0x7ff0d819e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33578/0 (socket says 172.21.15.146:33578) 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.213+0000 7ff0d77fe700 1 -- 172.21.15.146:0/1772789633 learned_addr learned my addr 172.21.15.146:0/1772789633 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.213+0000 7ff0d67fc700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d8197360 0x7ff0d819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.214+0000 7ff0d6ffd700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0d80ffa60 0x7ff0d8196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.214+0000 7ff0d77fe700 1 -- 172.21.15.146:0/1772789633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d8197360 msgr2=0x7ff0d819c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.214+0000 7ff0d77fe700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d8197360 0x7ff0d819c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.214+0000 7ff0d77fe700 1 -- 172.21.15.146:0/1772789633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0d80ffa60 msgr2=0x7ff0d8196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.214+0000 7ff0d77fe700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0d80ffa60 0x7ff0d8196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:54.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.214+0000 7ff0d77fe700 1 -- 172.21.15.146:0/1772789633 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc0095f0 con 0x7ff0d81977e0 2024-02-17T12:41:54.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.214+0000 7ff0d77fe700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0d81977e0 0x7ff0d819e960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000ba50 tx=0x7ff0d000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:54.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.214+0000 7ff0bffff700 1 -- 172.21.15.146:0/1772789633 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d000c930 con 0x7ff0d81977e0 2024-02-17T12:41:54.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.214+0000 7ff0bffff700 1 -- 172.21.15.146:0/1772789633 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0d0014440 con 0x7ff0d81977e0 2024-02-17T12:41:54.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.214+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/1772789633 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d819f710 con 0x7ff0d81977e0 2024-02-17T12:41:54.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.214+0000 7ff0bffff700 1 -- 172.21.15.146:0/1772789633 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0007590 con 0x7ff0d81977e0 2024-02-17T12:41:54.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.214+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/1772789633 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d819fc90 con 0x7ff0d81977e0 2024-02-17T12:41:54.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.215+0000 7ff0bffff700 1 -- 172.21.15.146:0/1772789633 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7ff0d00077b0 con 0x7ff0d81977e0 2024-02-17T12:41:54.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.215+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/1772789633 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d81905c0 con 0x7ff0d81977e0 2024-02-17T12:41:54.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.215+0000 7ff0bffff700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff0c0039410 0x7ff0c003b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:41:54.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.215+0000 7ff0bffff700 1 -- 172.21.15.146:0/1772789633 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff0d004b2c0 con 0x7ff0d81977e0 2024-02-17T12:41:54.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.216+0000 7ff0d6ffd700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff0c0039410 0x7ff0c003b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:41:54.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.216+0000 7ff0d6ffd700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff0c0039410 0x7ff0c003b8d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009fd0 tx=0x7ff0cc005300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:41:54.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.221+0000 7ff0bffff700 1 -- 172.21.15.146:0/1772789633 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff0d001b8b0 con 0x7ff0d81977e0 2024-02-17T12:41:54.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.427+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/1772789633 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi018=x;smithi119=y;smithi146=z", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0d802d870 con 0x7ff0c0039410 2024-02-17T12:41:54.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.433+0000 7ff0bffff700 1 -- 172.21.15.146:0/1772789633 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff0d802d870 con 0x7ff0c0039410 2024-02-17T12:41:54.430 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled mgr update... 2024-02-17T12:41:54.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.436+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/1772789633 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff0c0039410 msgr2=0x7ff0c003b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:54.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.436+0000 7ff0dd5a7700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff0c0039410 0x7ff0c003b8d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009fd0 tx=0x7ff0cc005300 comp rx=0 tx=0).stop 2024-02-17T12:41:54.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.436+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/1772789633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0d81977e0 msgr2=0x7ff0d819e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:41:54.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.436+0000 7ff0dd5a7700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0d81977e0 0x7ff0d819e960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000ba50 tx=0x7ff0d000bd60 comp rx=0 tx=0).stop 2024-02-17T12:41:54.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.436+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/1772789633 shutdown_connections 2024-02-17T12:41:54.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.436+0000 7ff0dd5a7700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff0c0039410 0x7ff0c003b8d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:54.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.436+0000 7ff0dd5a7700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0d81977e0 0x7ff0d819e960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:54.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.436+0000 7ff0dd5a7700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0d80ffa60 0x7ff0d8196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:54.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.436+0000 7ff0dd5a7700 1 --2- 172.21.15.146:0/1772789633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d8197360 0x7ff0d819c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:41:54.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.436+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/1772789633 >> 172.21.15.146:0/1772789633 conn(0x7ff0d80fb3c0 msgr2=0x7ff0d80fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:41:54.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.436+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/1772789633 shutdown_connections 2024-02-17T12:41:54.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:41:54.437+0000 7ff0dd5a7700 1 -- 172.21.15.146:0/1772789633 wait complete. 2024-02-17T12:41:55.005 DEBUG:teuthology.orchestra.run.smithi119:mgr.y> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.y.service 2024-02-17T12:41:55.008 DEBUG:teuthology.orchestra.run.smithi146:mgr.z> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.z.service 2024-02-17T12:41:55.010 INFO:tasks.cephadm:Deploying OSDs... 2024-02-17T12:41:55.010 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:41:55.011 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/scratch_devs of=/dev/stdout 2024-02-17T12:41:55.029 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-17T12:41:55.029 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_1 2024-02-17T12:41:55.087 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-17T12:41:55.087 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T12:41:55.087 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-17T12:41:55.087 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:41:55.087 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T12:41:55.087 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-02-17 12:41:02.869363007 +0000 2024-02-17T12:41:55.087 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-02-17 12:35:28.042916625 +0000 2024-02-17T12:41:55.087 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-02-17 12:35:28.042916625 +0000 2024-02-17T12:41:55.087 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-02-17 12:35:28.042916625 +0000 2024-02-17T12:41:55.088 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-17T12:41:55.156 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-02-17T12:41:55.157 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-02-17T12:41:55.157 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000117638 s, 4.4 MB/s 2024-02-17T12:41:55.158 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-17T12:41:55.215 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_2 2024-02-17T12:41:55.271 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-17T12:41:55.271 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T12:41:55.271 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-17T12:41:55.271 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:41:55.271 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T12:41:55.271 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-02-17 12:41:02.903362607 +0000 2024-02-17T12:41:55.271 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-02-17 12:35:28.361913010 +0000 2024-02-17T12:41:55.271 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-02-17 12:35:28.361913010 +0000 2024-02-17T12:41:55.271 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-02-17 12:35:28.361913010 +0000 2024-02-17T12:41:55.271 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-17T12:41:55.332 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:55 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:55.332 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:55 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:55.332 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:55 smithi146 ceph-mon[36454]: Reconfiguring mon.c (monmap changed)... 2024-02-17T12:41:55.332 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:55 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:55.332 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:55 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T12:41:55.332 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:55 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:55.332 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:55 smithi146 ceph-mon[36454]: Reconfiguring daemon mon.c on smithi146 2024-02-17T12:41:55.332 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:55 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:55.336 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:55 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:55.336 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:55 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:55.336 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:55 smithi018 ceph-mon[32625]: Reconfiguring mon.c (monmap changed)... 2024-02-17T12:41:55.336 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:55 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:55.336 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:55 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T12:41:55.336 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:55 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:55.336 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:55 smithi018 ceph-mon[32625]: Reconfiguring daemon mon.c on smithi146 2024-02-17T12:41:55.336 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:55 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:55.338 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-02-17T12:41:55.338 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-02-17T12:41:55.338 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000103519 s, 4.9 MB/s 2024-02-17T12:41:55.339 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-17T12:41:55.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:55 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:55.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:55 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:55.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:55 smithi119 ceph-mon[36188]: Reconfiguring mon.c (monmap changed)... 2024-02-17T12:41:55.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:55 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T12:41:55.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:55 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T12:41:55.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:55 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:55.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:55 smithi119 ceph-mon[36188]: Reconfiguring daemon mon.c on smithi146 2024-02-17T12:41:55.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:55 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:55.396 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_3 2024-02-17T12:41:55.451 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-17T12:41:55.451 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T12:41:55.451 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-17T12:41:55.452 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:41:55.452 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T12:41:55.452 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-02-17 12:41:02.936362220 +0000 2024-02-17T12:41:55.452 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-02-17 12:35:28.686909327 +0000 2024-02-17T12:41:55.452 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-02-17 12:35:28.686909327 +0000 2024-02-17T12:41:55.452 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-02-17 12:35:28.686909327 +0000 2024-02-17T12:41:55.452 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-17T12:41:55.517 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-02-17T12:41:55.517 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-02-17T12:41:55.517 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 9.3357e-05 s, 5.5 MB/s 2024-02-17T12:41:55.519 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-17T12:41:55.577 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_4 2024-02-17T12:41:55.632 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-17T12:41:55.632 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T12:41:55.632 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-17T12:41:55.633 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:41:55.633 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T12:41:55.633 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-02-17 12:41:02.967361855 +0000 2024-02-17T12:41:55.633 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-02-17 12:35:29.011905644 +0000 2024-02-17T12:41:55.633 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-02-17 12:35:29.011905644 +0000 2024-02-17T12:41:55.633 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-02-17 12:35:29.011905644 +0000 2024-02-17T12:41:55.633 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-17T12:41:55.698 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-02-17T12:41:55.698 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-02-17T12:41:55.698 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000100578 s, 5.1 MB/s 2024-02-17T12:41:55.699 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-17T12:41:55.755 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-02-17T12:41:55.755 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-02-17T12:41:55.774 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-17T12:41:55.774 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-02-17T12:41:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-17T12:41:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T12:41:55.831 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-02-17T12:41:55.831 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:41:55.832 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T12:41:55.832 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-02-17 12:41:42.997733777 +0000 2024-02-17T12:41:55.832 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-02-17 12:35:54.344441033 +0000 2024-02-17T12:41:55.832 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-02-17 12:35:54.344441033 +0000 2024-02-17T12:41:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-02-17 12:35:54.344441033 +0000 2024-02-17T12:41:55.832 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-17T12:41:55.900 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-02-17T12:41:55.900 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-02-17T12:41:55.900 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000273442 s, 1.9 MB/s 2024-02-17T12:41:55.901 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-17T12:41:55.958 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-02-17T12:41:56.015 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-17T12:41:56.016 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T12:41:56.016 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-02-17T12:41:56.016 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:41:56.016 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T12:41:56.016 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-02-17 12:41:42.997733777 +0000 2024-02-17T12:41:56.016 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-02-17 12:35:54.693432105 +0000 2024-02-17T12:41:56.016 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-02-17 12:35:54.693432105 +0000 2024-02-17T12:41:56.016 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-02-17 12:35:54.693432105 +0000 2024-02-17T12:41:56.016 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-17T12:41:56.083 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-02-17T12:41:56.083 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-02-17T12:41:56.083 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000208177 s, 2.5 MB/s 2024-02-17T12:41:56.084 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-17T12:41:56.140 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-02-17T12:41:56.195 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-17T12:41:56.195 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T12:41:56.195 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-02-17T12:41:56.196 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:41:56.196 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T12:41:56.196 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-02-17 12:41:42.998733752 +0000 2024-02-17T12:41:56.196 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-02-17 12:35:55.038423279 +0000 2024-02-17T12:41:56.196 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-02-17 12:35:55.038423279 +0000 2024-02-17T12:41:56.196 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-02-17 12:35:55.038423279 +0000 2024-02-17T12:41:56.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-17T12:41:56.259 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:56 smithi119 ceph-mon[36188]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:56.259 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:56 smithi119 ceph-mon[36188]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi018=x;smithi119=y;smithi146=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:41:56.259 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:56 smithi119 ceph-mon[36188]: Saving service mgr spec with placement smithi018=x;smithi119=y;smithi146=z;count:3 2024-02-17T12:41:56.259 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:56 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:56.259 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:56 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:56.259 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:56 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:56.261 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-02-17T12:41:56.261 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-02-17T12:41:56.261 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000253628 s, 2.0 MB/s 2024-02-17T12:41:56.263 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-17T12:41:56.319 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-02-17T12:41:56.374 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-17T12:41:56.374 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T12:41:56.374 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-02-17T12:41:56.374 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:41:56.374 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T12:41:56.374 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-02-17 12:41:42.998733752 +0000 2024-02-17T12:41:56.374 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-02-17 12:35:55.388414326 +0000 2024-02-17T12:41:56.374 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-02-17 12:35:55.388414326 +0000 2024-02-17T12:41:56.374 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-02-17 12:35:55.388414326 +0000 2024-02-17T12:41:56.374 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-17T12:41:56.439 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-02-17T12:41:56.439 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-02-17T12:41:56.439 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000169691 s, 3.0 MB/s 2024-02-17T12:41:56.440 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-17T12:41:56.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:56 smithi146 ceph-mon[36454]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:56.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:56 smithi146 ceph-mon[36454]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi018=x;smithi119=y;smithi146=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:41:56.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:56 smithi146 ceph-mon[36454]: Saving service mgr spec with placement smithi018=x;smithi119=y;smithi146=z;count:3 2024-02-17T12:41:56.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:56 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:56.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:56 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:56.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:56 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:56.495 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-02-17T12:41:56.495 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2024-02-17T12:41:56.512 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-17T12:41:56.513 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2024-02-17T12:41:56.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:56 smithi018 ceph-mon[32625]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:56.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:56 smithi018 ceph-mon[32625]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi018=x;smithi119=y;smithi146=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:41:56.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:56 smithi018 ceph-mon[32625]: Saving service mgr spec with placement smithi018=x;smithi119=y;smithi146=z;count:3 2024-02-17T12:41:56.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:56 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:56.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:56 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:56.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:56 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:41:56.569 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-17T12:41:56.569 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T12:41:56.569 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-02-17T12:41:56.569 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:41:56.569 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T12:41:56.569 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-02-17 12:41:33.767201884 +0000 2024-02-17T12:41:56.569 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-02-17 12:35:59.725609293 +0000 2024-02-17T12:41:56.569 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-02-17 12:35:59.725609293 +0000 2024-02-17T12:41:56.570 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-02-17 12:35:59.725609293 +0000 2024-02-17T12:41:56.570 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-17T12:41:56.637 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-02-17T12:41:56.637 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-02-17T12:41:56.637 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000271226 s, 1.9 MB/s 2024-02-17T12:41:56.638 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-17T12:41:56.693 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2024-02-17T12:41:56.749 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-17T12:41:56.749 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T12:41:56.749 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-02-17T12:41:56.749 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:41:56.749 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T12:41:56.749 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-02-17 12:41:33.768201868 +0000 2024-02-17T12:41:56.749 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-02-17 12:36:00.048603934 +0000 2024-02-17T12:41:56.749 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-02-17 12:36:00.048603934 +0000 2024-02-17T12:41:56.750 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-02-17 12:36:00.048603934 +0000 2024-02-17T12:41:56.750 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-17T12:41:56.815 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-02-17T12:41:56.815 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-02-17T12:41:56.815 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.00019516 s, 2.6 MB/s 2024-02-17T12:41:56.816 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-17T12:41:56.871 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2024-02-17T12:41:56.925 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-17T12:41:56.926 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T12:41:56.926 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-02-17T12:41:56.926 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:41:56.926 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T12:41:56.926 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-02-17 12:41:33.768201868 +0000 2024-02-17T12:41:56.926 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-02-17 12:36:00.397598143 +0000 2024-02-17T12:41:56.926 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-02-17 12:36:00.397598143 +0000 2024-02-17T12:41:56.926 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-02-17 12:36:00.397598143 +0000 2024-02-17T12:41:56.926 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-17T12:41:56.991 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-02-17T12:41:56.992 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-02-17T12:41:56.992 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000205671 s, 2.5 MB/s 2024-02-17T12:41:56.993 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-17T12:41:57.048 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2024-02-17T12:41:57.103 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-17T12:41:57.103 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T12:41:57.103 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-02-17T12:41:57.103 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:41:57.104 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T12:41:57.104 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-02-17 12:41:33.768201868 +0000 2024-02-17T12:41:57.104 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-02-17 12:36:00.737592502 +0000 2024-02-17T12:41:57.104 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-02-17 12:36:00.737592502 +0000 2024-02-17T12:41:57.104 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-02-17 12:36:00.737592502 +0000 2024-02-17T12:41:57.104 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-17T12:41:57.169 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-02-17T12:41:57.169 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-02-17T12:41:57.170 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000190329 s, 2.7 MB/s 2024-02-17T12:41:57.171 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-17T12:41:57.227 INFO:tasks.cephadm:Deploying osd.0 on smithi018 with /dev/vg_nvme/lv_4... 2024-02-17T12:41:57.227 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-17T12:41:57.241 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:57.241 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:57.241 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:57.241 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T12:41:57.241 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-17T12:41:57.241 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T12:41:57.241 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:57.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:57.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:57.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:57.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T12:41:57.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-17T12:41:57.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T12:41:57.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:57.470 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:41:57.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:57.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:41:57.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:41:57.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T12:41:57.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-17T12:41:57.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T12:41:57.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:41:58.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:58 smithi018 ceph-mon[32625]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:58.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:58 smithi018 ceph-mon[32625]: Deploying daemon mgr.z on smithi146 2024-02-17T12:41:58.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:58 smithi119 ceph-mon[36188]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:58.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:58 smithi119 ceph-mon[36188]: Deploying daemon mgr.z on smithi146 2024-02-17T12:41:58.407 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:58 smithi146 ceph-mon[36454]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:58.407 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:58 smithi146 ceph-mon[36454]: Deploying daemon mgr.z on smithi146 2024-02-17T12:41:58.704 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:41:58 smithi146 systemd[1]: Starting Ceph mgr.z for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:41:59.128 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:41:58 smithi146 podman[38290]: 2024-02-17 12:41:58.856348852 +0000 UTC m=+0.098000453 container create f16d38e893343102c0ca121ac5ae56c169c47dac856a3ae350757f9a40f859b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T12:41:59.128 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:41:58 smithi146 podman[38290]: 2024-02-17 12:41:58.776303269 +0000 UTC m=+0.017954834 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:41:59.128 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:41:59 smithi146 podman[38290]: 2024-02-17 12:41:59.050123022 +0000 UTC m=+0.291774570 container init f16d38e893343102c0ca121ac5ae56c169c47dac856a3ae350757f9a40f859b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b) 2024-02-17T12:41:59.128 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:41:59 smithi146 podman[38290]: 2024-02-17 12:41:59.052619071 +0000 UTC m=+0.294270635 container start f16d38e893343102c0ca121ac5ae56c169c47dac856a3ae350757f9a40f859b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T12:41:59.129 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:41:59 smithi146 ceph-mgr[38310]: -- 172.21.15.146:0/445185208 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559ef5fe01a0 con 0x559ef5fb7000 2024-02-17T12:41:59.129 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:41:59 smithi146 bash[38290]: f16d38e893343102c0ca121ac5ae56c169c47dac856a3ae350757f9a40f859b1 2024-02-17T12:41:59.356 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-17T12:41:59.356 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-17T12:41:59.357 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-02-17T12:41:59.357 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-02-17T12:41:59.357 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00858969 s, 1.2 GB/s 2024-02-17T12:41:59.357 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-02-17T12:41:59.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:41:59 smithi146 ceph-mon[36454]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:59.454 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:41:59 smithi146 systemd[1]: Started Ceph mgr.z for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:41:59.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:41:59 smithi018 ceph-mon[32625]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:59.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:41:59 smithi119 ceph-mon[36188]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:41:59.897 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi018:vg_nvme/lv_4 2024-02-17T12:41:59.954 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:41:59 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:41:59.602+0000 7ffa6c1df200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T12:41:59.954 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:41:59 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:41:59.695+0000 7ffa6c1df200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T12:42:00.136 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:42:00.454 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:00.341+0000 7ffa6c1df200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T12:42:00.455 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:00.437+0000 7ffa6c1df200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T12:42:00.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:00.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:00.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:00.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T12:42:00.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-17T12:42:00.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T12:42:00.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:00.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-mon[36454]: Deploying daemon mgr.y on smithi119 2024-02-17T12:42:00.464 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:00.464 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:00.464 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:00.464 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T12:42:00.464 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-17T12:42:00.464 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T12:42:00.464 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:00.464 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:00 smithi018 ceph-mon[32625]: Deploying daemon mgr.y on smithi119 2024-02-17T12:42:00.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:00.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:00.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:00.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T12:42:00.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-17T12:42:00.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T12:42:00.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:00.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:00 smithi119 ceph-mon[36188]: Deploying daemon mgr.y on smithi119 2024-02-17T12:42:00.769 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:00.586+0000 7ffa6c1df200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T12:42:00.769 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:00.687+0000 7ffa6c1df200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T12:42:01.054 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:00 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:00.772+0000 7ffa6c1df200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T12:42:01.054 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:01.058+0000 7ffa6c1df200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T12:42:01.253 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:01 smithi119 systemd[1]: Starting Ceph mgr.y for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:42:01.352 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:01 smithi146 ceph-mon[36454]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:01.352 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:01.171+0000 7ffa6c1df200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T12:42:01.352 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:01.259+0000 7ffa6c1df200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T12:42:01.352 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:01.355+0000 7ffa6c1df200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T12:42:01.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:01 smithi018 ceph-mon[32625]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:01.565 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:01 smithi119 ceph-mon[36188]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:01.565 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:01 smithi119 podman[37553]: 2024-02-17 12:42:01.368082139 +0000 UTC m=+0.112186882 container create 68953ec3dd37828f08a33aab1711b2a0c38f33a38388b14ce7abb84bb373bad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b) 2024-02-17T12:42:01.565 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:01 smithi119 podman[37553]: 2024-02-17 12:42:01.274101293 +0000 UTC m=+0.018206058 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:42:01.565 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:01 smithi119 podman[37553]: 2024-02-17 12:42:01.570316366 +0000 UTC m=+0.314421118 container init 68953ec3dd37828f08a33aab1711b2a0c38f33a38388b14ce7abb84bb373bad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-17T12:42:01.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.588+0000 7fd123bd5700 1 -- 172.21.15.18:0/3343458941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd11c100ba0 msgr2=0x7fd11c100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:01.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.588+0000 7fd123bd5700 1 --2- 172.21.15.18:0/3343458941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd11c100ba0 0x7fd11c100fc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd1180099c0 tx=0x7fd118009cd0 comp rx=0 tx=0).stop 2024-02-17T12:42:01.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.588+0000 7fd123bd5700 1 -- 172.21.15.18:0/3343458941 shutdown_connections 2024-02-17T12:42:01.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.588+0000 7fd123bd5700 1 --2- 172.21.15.18:0/3343458941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd11c102740 0x7fd11c108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:01.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.588+0000 7fd123bd5700 1 --2- 172.21.15.18:0/3343458941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd11c100ba0 0x7fd11c100fc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:01.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.588+0000 7fd123bd5700 1 --2- 172.21.15.18:0/3343458941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd11c101da0 0x7fd11c102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:01.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.588+0000 7fd123bd5700 1 -- 172.21.15.18:0/3343458941 >> 172.21.15.18:0/3343458941 conn(0x7fd11c0fc160 msgr2=0x7fd11c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:42:01.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.588+0000 7fd123bd5700 1 -- 172.21.15.18:0/3343458941 shutdown_connections 2024-02-17T12:42:01.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.588+0000 7fd123bd5700 1 -- 172.21.15.18:0/3343458941 wait complete. 2024-02-17T12:42:01.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.588+0000 7fd123bd5700 1 Processor -- start 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd123bd5700 1 -- start start 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd123bd5700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd11c100ba0 0x7fd11c104ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd123bd5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd11c101da0 0x7fd11c103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd123bd5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd11c102740 0x7fd11c1036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd123bd5700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c103dd0 con 0x7fd11c100ba0 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd123bd5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c103f40 con 0x7fd11c102740 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd123bd5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c1a1d60 con 0x7fd11c101da0 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd121971700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd11c100ba0 0x7fd11c104ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd121971700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd11c100ba0 0x7fd11c104ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41850/0 (socket says 172.21.15.18:41850) 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd121971700 1 -- 172.21.15.18:0/3922459755 learned_addr learned my addr 172.21.15.18:0/3922459755 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd122172700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd11c102740 0x7fd11c1036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd121170700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd11c101da0 0x7fd11c103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:42:01.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd121971700 1 -- 172.21.15.18:0/3922459755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd11c101da0 msgr2=0x7fd11c103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd121971700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd11c101da0 0x7fd11c103180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd121971700 1 -- 172.21.15.18:0/3922459755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd11c102740 msgr2=0x7fd11c1036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd121971700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd11c102740 0x7fd11c1036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd121971700 1 -- 172.21.15.18:0/3922459755 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1180095f0 con 0x7fd11c100ba0 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd122172700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd11c102740 0x7fd11c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd121971700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd11c100ba0 0x7fd11c104ac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd1180036c0 tx=0x7fd11800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.589+0000 7fd112ffd700 1 -- 172.21.15.18:0/3922459755 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11801d070 con 0x7fd11c100ba0 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.590+0000 7fd112ffd700 1 -- 172.21.15.18:0/3922459755 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd118003e10 con 0x7fd11c100ba0 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.590+0000 7fd112ffd700 1 -- 172.21.15.18:0/3922459755 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118005430 con 0x7fd11c100ba0 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.590+0000 7fd123bd5700 1 -- 172.21.15.18:0/3922459755 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11c1a1f60 con 0x7fd11c100ba0 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.590+0000 7fd123bd5700 1 -- 172.21.15.18:0/3922459755 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11c1a2370 con 0x7fd11c100ba0 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.590+0000 7fd112ffd700 1 -- 172.21.15.18:0/3922459755 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fd118005590 con 0x7fd11c100ba0 2024-02-17T12:42:01.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.591+0000 7fd123bd5700 1 -- 172.21.15.18:0/3922459755 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11c10c4e0 con 0x7fd11c100ba0 2024-02-17T12:42:01.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.591+0000 7fd112ffd700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd1080391c0 0x7fd10803b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:42:01.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.591+0000 7fd112ffd700 1 -- 172.21.15.18:0/3922459755 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd11804cc50 con 0x7fd11c100ba0 2024-02-17T12:42:01.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.591+0000 7fd121170700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd1080391c0 0x7fd10803b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:42:01.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.591+0000 7fd121170700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd1080391c0 0x7fd10803b680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd10c007c60 tx=0x7fd10c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:42:01.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.596+0000 7fd112ffd700 1 -- 172.21.15.18:0/3922459755 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd11800f880 con 0x7fd11c100ba0 2024-02-17T12:42:01.704 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:01.535+0000 7ffa6c1df200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T12:42:01.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:01.811+0000 7fd123bd5700 1 -- 172.21.15.18:0/3922459755 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd11c0fe320 con 0x7fd1080391c0 2024-02-17T12:42:01.847 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:01 smithi119 podman[37553]: 2024-02-17 12:42:01.572860885 +0000 UTC m=+0.316965636 container start 68953ec3dd37828f08a33aab1711b2a0c38f33a38388b14ce7abb84bb373bad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-17T12:42:01.847 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:01 smithi119 ceph-mgr[37572]: -- 172.21.15.119:0/4037209728 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d4a605e1a0 con 0x55d4a6034800 2024-02-17T12:42:01.847 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:01 smithi119 bash[37553]: 68953ec3dd37828f08a33aab1711b2a0c38f33a38388b14ce7abb84bb373bad8 2024-02-17T12:42:01.847 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:01 smithi119 systemd[1]: Started Ceph mgr.y for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:42:02.270 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:02.132+0000 7f81a26e1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T12:42:02.270 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:02.232+0000 7f81a26e1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T12:42:02.849 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:02 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:02.556+0000 7ffa6c1df200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T12:42:02.849 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:02 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:02.695+0000 7ffa6c1df200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T12:42:02.850 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:02.850 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:02.850 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:02.850 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:02.850 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:02 smithi146 ceph-mon[36454]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:42:02.850 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:42:02.850 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:42:02.850 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:02.850 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:03.025 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:02 smithi119 ceph-mon[36188]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:42:03.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:42:03.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:42:03.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:03.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:03.026 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:02.934+0000 7f81a26e1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T12:42:03.026 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:03.030+0000 7f81a26e1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T12:42:03.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:02 smithi018 ceph-mon[32625]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:42:03.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:42:03.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:42:03.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:03.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:03.131 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:03.045+0000 7ffa6c1df200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T12:42:03.131 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:03.135+0000 7ffa6c1df200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T12:42:03.290 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:03.177+0000 7f81a26e1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T12:42:03.291 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:03.294+0000 7f81a26e1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T12:42:03.454 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:03.342+0000 7ffa6c1df200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:42:03.596 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:03.377+0000 7f81a26e1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T12:42:03.780 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:03.529+0000 7ffa6c1df200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T12:42:03.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-mon[36188]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:03.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:03.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:03.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:03.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T12:42:03.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T12:42:03.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:03.926 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:03.641+0000 7f81a26e1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T12:42:03.926 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:03.748+0000 7f81a26e1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T12:42:03.926 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:03.834+0000 7f81a26e1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T12:42:03.926 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:03.929+0000 7f81a26e1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T12:42:04.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:03 smithi018 ceph-mon[32625]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:04.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:03 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:04.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:03 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:04.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:03 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:04.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:03 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:04.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:03 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:04.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:03 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:04.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:03 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:04.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:03 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T12:42:04.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:03 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T12:42:04.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:03 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:04.071 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:03.814+0000 7ffa6c1df200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:42:04.071 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:04 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:04.075+0000 7ffa6c1df200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T12:42:04.071 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-mon[36454]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:04.071 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:04.071 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:04.072 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:04.072 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:04.072 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:04.072 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:04.072 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:04.072 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T12:42:04.072 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T12:42:04.072 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:03 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:04.346 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:04 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:04.103+0000 7f81a26e1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T12:42:04.454 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:04 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:04.309+0000 7ffa6c1df200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T12:42:05.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:04 smithi018 ceph-mon[32625]: Reconfiguring mgr.x (unknown last config time)... 2024-02-17T12:42:05.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:04 smithi018 ceph-mon[32625]: Reconfiguring daemon mgr.x on smithi018 2024-02-17T12:42:05.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:04 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1723689523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31ca3f0e-c84e-4779-984f-daae32ce96f3"}]: dispatch 2024-02-17T12:42:05.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:04 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1723689523' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31ca3f0e-c84e-4779-984f-daae32ce96f3"}]': finished 2024-02-17T12:42:05.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:04 smithi018 ceph-mon[32625]: osdmap e5: 1 total, 0 up, 1 in 2024-02-17T12:42:05.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:04 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:05.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:04 smithi119 ceph-mon[36188]: Reconfiguring mgr.x (unknown last config time)... 2024-02-17T12:42:05.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:04 smithi119 ceph-mon[36188]: Reconfiguring daemon mgr.x on smithi018 2024-02-17T12:42:05.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:04 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1723689523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31ca3f0e-c84e-4779-984f-daae32ce96f3"}]: dispatch 2024-02-17T12:42:05.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:04 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1723689523' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31ca3f0e-c84e-4779-984f-daae32ce96f3"}]': finished 2024-02-17T12:42:05.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:04 smithi119 ceph-mon[36188]: osdmap e5: 1 total, 0 up, 1 in 2024-02-17T12:42:05.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:04 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:05.110 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:04 smithi146 ceph-mon[36454]: Reconfiguring mgr.x (unknown last config time)... 2024-02-17T12:42:05.110 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:04 smithi146 ceph-mon[36454]: Reconfiguring daemon mgr.x on smithi018 2024-02-17T12:42:05.110 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:04 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1723689523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31ca3f0e-c84e-4779-984f-daae32ce96f3"}]: dispatch 2024-02-17T12:42:05.110 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:04 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1723689523' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31ca3f0e-c84e-4779-984f-daae32ce96f3"}]': finished 2024-02-17T12:42:05.110 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:04 smithi146 ceph-mon[36454]: osdmap e5: 1 total, 0 up, 1 in 2024-02-17T12:42:05.110 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:04 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:05.346 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:05 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:05.064+0000 7f81a26e1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T12:42:05.347 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:05 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:05.194+0000 7f81a26e1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T12:42:05.454 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:05.113+0000 7ffa6c1df200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T12:42:05.783 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:05 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:05.522+0000 7f81a26e1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T12:42:05.784 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:05 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:05.611+0000 7f81a26e1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T12:42:05.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:05 smithi146 ceph-mon[36454]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:05.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:05 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/76958145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:42:05.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:05.954 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:05.686+0000 7ffa6c1df200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T12:42:05.954 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:05.783+0000 7ffa6c1df200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T12:42:06.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:05 smithi119 ceph-mon[36188]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:06.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:05 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/76958145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:42:06.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:06.097 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:05 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:05.803+0000 7f81a26e1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:42:06.097 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:05 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:05.985+0000 7f81a26e1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T12:42:06.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:05 smithi018 ceph-mon[32625]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:06.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:05 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/76958145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:42:06.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:06.454 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:42:06 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:42:06.000+0000 7ffa6c1df200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T12:42:06.519 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:06 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:06.264+0000 7f81a26e1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:42:06.519 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:06 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:06.523+0000 7f81a26e1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T12:42:06.788 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:06 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:06.754+0000 7f81a26e1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T12:42:06.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:06.796+0000 7fd112ffd700 1 -- 172.21.15.18:0/3922459755 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 14) v1 ==== 91350+0+0 (secure 0 0 0) 0x7fd11802b440 con 0x7fd11c100ba0 2024-02-17T12:42:07.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:06 smithi119 ceph-mon[36188]: Standby manager daemon z started 2024-02-17T12:42:07.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:06 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.146:0/3660243495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-17T12:42:07.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:06 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.146:0/3660243495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T12:42:07.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:06 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.146:0/3660243495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-17T12:42:07.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:06 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.146:0/3660243495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T12:42:07.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:06 smithi146 ceph-mon[36454]: Standby manager daemon z started 2024-02-17T12:42:07.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:06 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.146:0/3660243495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-17T12:42:07.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:06 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.146:0/3660243495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T12:42:07.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:06 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.146:0/3660243495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-17T12:42:07.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:06 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.146:0/3660243495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T12:42:07.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:06 smithi018 ceph-mon[32625]: Standby manager daemon z started 2024-02-17T12:42:07.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:06 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.146:0/3660243495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-17T12:42:07.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:06 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.146:0/3660243495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T12:42:07.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:06 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.146:0/3660243495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-17T12:42:07.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:06 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.146:0/3660243495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T12:42:07.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:07 smithi119 ceph-mon[36188]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:07.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:07 smithi119 ceph-mon[36188]: mgrmap e14: x(active, since 76s), standbys: z 2024-02-17T12:42:07.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:07 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-17T12:42:07.847 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:07 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:07.544+0000 7f81a26e1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T12:42:08.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:07 smithi146 ceph-mon[36454]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:08.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:07 smithi146 ceph-mon[36454]: mgrmap e14: x(active, since 76s), standbys: z 2024-02-17T12:42:08.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:07 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-17T12:42:08.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:07 smithi018 ceph-mon[32625]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:08.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:07 smithi018 ceph-mon[32625]: mgrmap e14: x(active, since 76s), standbys: z 2024-02-17T12:42:08.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:07 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-17T12:42:08.415 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:08 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:08.107+0000 7f81a26e1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T12:42:08.415 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:08 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:08.204+0000 7f81a26e1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T12:42:08.415 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:42:08 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:42:08.419+0000 7f81a26e1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T12:42:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:08.806+0000 7fd112ffd700 1 -- 172.21.15.18:0/3922459755 <== mon.0 v2:172.21.15.18:3300/0 8 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7fd118003930 con 0x7fd11c100ba0 2024-02-17T12:42:08.991 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:08 smithi018 ceph-mon[32625]: Standby manager daemon y started 2024-02-17T12:42:08.991 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:08 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.119:0/89750680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-17T12:42:08.991 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:08 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.119:0/89750680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T12:42:08.992 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:08 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.119:0/89750680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-17T12:42:08.992 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:08 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.119:0/89750680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T12:42:09.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:08 smithi119 ceph-mon[36188]: Standby manager daemon y started 2024-02-17T12:42:09.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:08 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.119:0/89750680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-17T12:42:09.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:08 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.119:0/89750680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T12:42:09.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:08 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.119:0/89750680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-17T12:42:09.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:08 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.119:0/89750680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T12:42:09.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:08 smithi146 ceph-mon[36454]: Standby manager daemon y started 2024-02-17T12:42:09.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:08 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.119:0/89750680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-17T12:42:09.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:08 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.119:0/89750680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T12:42:09.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:08 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.119:0/89750680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-17T12:42:09.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:08 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.119:0/89750680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T12:42:09.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:09 smithi018 ceph-mon[32625]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:09.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:09 smithi018 ceph-mon[32625]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-02-17T12:42:09.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:09 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-17T12:42:10.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:09 smithi119 ceph-mon[36188]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:10.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:09 smithi119 ceph-mon[36188]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-02-17T12:42:10.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:09 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-17T12:42:10.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:09 smithi146 ceph-mon[36454]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:10.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:09 smithi146 ceph-mon[36454]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-02-17T12:42:10.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:09 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-17T12:42:11.104 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:10 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:11.104 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:10 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:11.104 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:10 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:11.104 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:10 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:11.104 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:10 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:11.104 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:10 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:11.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:10 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:11.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:10 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:11.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:10 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:11.205 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:10 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:11.205 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:10 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:11.205 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:10 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:11.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:10 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:11.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:10 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:11.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:10 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:11.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:10 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:11.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:10 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:11.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:10 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:12.146 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:11 smithi018 ceph-mon[32625]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:12.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:11 smithi119 ceph-mon[36188]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:12.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:11 smithi146 ceph-mon[36454]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:13.074 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-17T12:42:13.074 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:13.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-17T12:42:13.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:13.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-17T12:42:13.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:14.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:13 smithi018 ceph-mon[32625]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:14.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:13 smithi018 ceph-mon[32625]: Deploying daemon osd.0 on smithi018 2024-02-17T12:42:14.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:13 smithi119 ceph-mon[36188]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:14.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:13 smithi119 ceph-mon[36188]: Deploying daemon osd.0 on smithi018 2024-02-17T12:42:14.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:13 smithi146 ceph-mon[36454]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:14.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:13 smithi146 ceph-mon[36454]: Deploying daemon osd.0 on smithi018 2024-02-17T12:42:16.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:15 smithi018 ceph-mon[32625]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:16.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:15 smithi119 ceph-mon[36188]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:16.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:15 smithi146 ceph-mon[36454]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:18.171 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:17 smithi018 ceph-mon[32625]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:18.171 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:18.171 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:18.171 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:18 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:18.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:18 smithi119 ceph-mon[36188]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:18.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:18 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:18.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:18 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:18.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:18 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:18.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:18 smithi146 ceph-mon[36454]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:18.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:18 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:18.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:18 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:18.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:18 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:20.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:20 smithi018 ceph-mon[32625]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:20.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:20 smithi119 ceph-mon[36188]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:20.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:20 smithi146 ceph-mon[36454]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.023+0000 7fd112ffd700 1 -- 172.21.15.18:0/3922459755 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd11c0fe320 con 0x7fd1080391c0 2024-02-17T12:42:21.019 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 0 on host 'smithi018' 2024-02-17T12:42:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.026+0000 7fd123bd5700 1 -- 172.21.15.18:0/3922459755 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd1080391c0 msgr2=0x7fd10803b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.026+0000 7fd123bd5700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd1080391c0 0x7fd10803b680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd10c007c60 tx=0x7fd10c0073d0 comp rx=0 tx=0).stop 2024-02-17T12:42:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.026+0000 7fd123bd5700 1 -- 172.21.15.18:0/3922459755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd11c100ba0 msgr2=0x7fd11c104ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.026+0000 7fd123bd5700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd11c100ba0 0x7fd11c104ac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd1180036c0 tx=0x7fd11800f690 comp rx=0 tx=0).stop 2024-02-17T12:42:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.026+0000 7fd123bd5700 1 -- 172.21.15.18:0/3922459755 shutdown_connections 2024-02-17T12:42:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.026+0000 7fd123bd5700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd11c102740 0x7fd11c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:21.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.026+0000 7fd123bd5700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd11c100ba0 0x7fd11c104ac0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:21.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.026+0000 7fd123bd5700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd1080391c0 0x7fd10803b680 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:21.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.026+0000 7fd123bd5700 1 --2- 172.21.15.18:0/3922459755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd11c101da0 0x7fd11c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:21.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.026+0000 7fd123bd5700 1 -- 172.21.15.18:0/3922459755 >> 172.21.15.18:0/3922459755 conn(0x7fd11c0fc160 msgr2=0x7fd11c0fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:42:21.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.026+0000 7fd123bd5700 1 -- 172.21.15.18:0/3922459755 shutdown_connections 2024-02-17T12:42:21.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:42:21.026+0000 7fd123bd5700 1 -- 172.21.15.18:0/3922459755 wait complete. 2024-02-17T12:42:21.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:21 smithi146 ceph-mon[36454]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:21.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:21.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:21.457 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.457 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.457 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.457 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:21.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:21 smithi018 ceph-mon[32625]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:21.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:21.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:21.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:21.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:21 smithi119 ceph-mon[36188]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:21.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:21.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:21.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:21.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:21.654 DEBUG:teuthology.orchestra.run.smithi018:osd.0> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.0.service 2024-02-17T12:42:21.664 INFO:tasks.cephadm:Deploying osd.1 on smithi119 with /dev/vg_nvme/lv_4... 2024-02-17T12:42:21.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-17T12:42:21.910 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T12:42:22.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:22 smithi018 ceph-mon[32625]: from='osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-17T12:42:22.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:22 smithi119 ceph-mon[36188]: from='osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-17T12:42:22.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:22 smithi146 ceph-mon[36454]: from='osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-17T12:42:23.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:23 smithi018 ceph-mon[32625]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:23.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:23 smithi018 ceph-mon[32625]: from='osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-17T12:42:23.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:23 smithi018 ceph-mon[32625]: osdmap e6: 1 total, 0 up, 1 in 2024-02-17T12:42:23.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:23 smithi018 ceph-mon[32625]: from='osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:42:23.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:23 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:23.529 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 12:42:23 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0[47251]: 2024-02-17T12:42:23.218+0000 7f770643c700 -1 osd.0 0 waiting for initial osdmap 2024-02-17T12:42:23.529 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 12:42:23 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0[47251]: 2024-02-17T12:42:23.220+0000 7f7702a30700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T12:42:23.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:23 smithi119 ceph-mon[36188]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:23.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:23 smithi119 ceph-mon[36188]: from='osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-17T12:42:23.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:23 smithi119 ceph-mon[36188]: osdmap e6: 1 total, 0 up, 1 in 2024-02-17T12:42:23.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:23 smithi119 ceph-mon[36188]: from='osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:42:23.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:23 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:23.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:23 smithi146 ceph-mon[36454]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:23.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:23 smithi146 ceph-mon[36454]: from='osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-17T12:42:23.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:23 smithi146 ceph-mon[36454]: osdmap e6: 1 total, 0 up, 1 in 2024-02-17T12:42:23.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:23 smithi146 ceph-mon[36454]: from='osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:42:23.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:23 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:23.760 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-17T12:42:23.760 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-17T12:42:23.760 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-02-17T12:42:23.760 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-02-17T12:42:23.760 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111482 s, 941 MB/s 2024-02-17T12:42:23.760 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-02-17T12:42:24.329 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:24 smithi119 ceph-mon[36188]: from='osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-02-17T12:42:24.329 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:24 smithi119 ceph-mon[36188]: osdmap e7: 1 total, 0 up, 1 in 2024-02-17T12:42:24.329 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:24 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:24.329 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:24 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:24.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_4 2024-02-17T12:42:24.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:24 smithi018 ceph-mon[32625]: from='osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-02-17T12:42:24.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:24 smithi018 ceph-mon[32625]: osdmap e7: 1 total, 0 up, 1 in 2024-02-17T12:42:24.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:24 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:24.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:24 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:24.598 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T12:42:24.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:24 smithi146 ceph-mon[36454]: from='osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-02-17T12:42:24.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:24 smithi146 ceph-mon[36454]: osdmap e7: 1 total, 0 up, 1 in 2024-02-17T12:42:24.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:24 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:24.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:24 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:25.468 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: purged_snaps scrub starts 2024-02-17T12:42:25.468 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: purged_snaps scrub ok 2024-02-17T12:42:25.468 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941] boot 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: osdmap e8: 1 total, 1 up, 1 in 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: Detected new or changed devices on smithi018 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi018 to 17093M 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:25.469 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:25.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: purged_snaps scrub starts 2024-02-17T12:42:25.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: purged_snaps scrub ok 2024-02-17T12:42:25.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:25.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:25.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941] boot 2024-02-17T12:42:25.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: osdmap e8: 1 total, 1 up, 1 in 2024-02-17T12:42:25.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:25.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: Detected new or changed devices on smithi018 2024-02-17T12:42:25.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:25.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:25.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:42:25.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi018 to 17093M 2024-02-17T12:42:25.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:25.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:25.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:25.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:25.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: purged_snaps scrub starts 2024-02-17T12:42:25.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: purged_snaps scrub ok 2024-02-17T12:42:25.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T12:42:25.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:25.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: osd.0 [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941] boot 2024-02-17T12:42:25.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: osdmap e8: 1 total, 1 up, 1 in 2024-02-17T12:42:25.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:42:25.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: Detected new or changed devices on smithi018 2024-02-17T12:42:25.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:25.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:25.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:42:25.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi018 to 17093M 2024-02-17T12:42:25.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:25.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:25.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:25.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:26.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.028+0000 7eff71116700 1 -- 172.21.15.119:0/3282744753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c102ea0 msgr2=0x7eff6c103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:26.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.028+0000 7eff71116700 1 --2- 172.21.15.119:0/3282744753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c102ea0 0x7eff6c103300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0099c0 tx=0x7eff5c009cd0 comp rx=0 tx=0).stop 2024-02-17T12:42:26.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.028+0000 7eff71116700 1 -- 172.21.15.119:0/3282744753 shutdown_connections 2024-02-17T12:42:26.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.028+0000 7eff71116700 1 --2- 172.21.15.119:0/3282744753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c103840 0x7eff6c10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:26.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.028+0000 7eff71116700 1 --2- 172.21.15.119:0/3282744753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff6c101ca0 0x7eff6c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:26.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.028+0000 7eff71116700 1 --2- 172.21.15.119:0/3282744753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c102ea0 0x7eff6c103300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:26.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.028+0000 7eff71116700 1 -- 172.21.15.119:0/3282744753 >> 172.21.15.119:0/3282744753 conn(0x7eff6c0fd220 msgr2=0x7eff6c0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:42:26.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.028+0000 7eff71116700 1 -- 172.21.15.119:0/3282744753 shutdown_connections 2024-02-17T12:42:26.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.028+0000 7eff71116700 1 -- 172.21.15.119:0/3282744753 wait complete. 2024-02-17T12:42:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff71116700 1 Processor -- start 2024-02-17T12:42:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff71116700 1 -- start start 2024-02-17T12:42:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff71116700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c101ca0 0x7eff6c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:42:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff71116700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c102ea0 0x7eff6c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:42:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff71116700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff6c103840 0x7eff6c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:42:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff71116700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c19f750 con 0x7eff6c103840 2024-02-17T12:42:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff71116700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c19f890 con 0x7eff6c102ea0 2024-02-17T12:42:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff71116700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c19fa20 con 0x7eff6c101ca0 2024-02-17T12:42:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff6a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c102ea0 0x7eff6c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:42:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff6a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c102ea0 0x7eff6c198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51470/0 (socket says 172.21.15.119:51470) 2024-02-17T12:42:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff6a59c700 1 -- 172.21.15.119:0/469368096 learned_addr learned my addr 172.21.15.119:0/469368096 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-02-17T12:42:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff6b59e700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff6c103840 0x7eff6c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:42:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff6ad9d700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c101ca0 0x7eff6c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:42:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff6a59c700 1 -- 172.21.15.119:0/469368096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c101ca0 msgr2=0x7eff6c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff6a59c700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c101ca0 0x7eff6c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff6a59c700 1 -- 172.21.15.119:0/469368096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff6c103840 msgr2=0x7eff6c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff6a59c700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff6c103840 0x7eff6c19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.029+0000 7eff6a59c700 1 -- 172.21.15.119:0/469368096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c0095f0 con 0x7eff6c102ea0 2024-02-17T12:42:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.030+0000 7eff6a59c700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c102ea0 0x7eff6c198100 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff5c003790 tx=0x7eff5c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:42:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.030+0000 7eff53fff700 1 -- 172.21.15.119:0/469368096 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c01d070 con 0x7eff6c102ea0 2024-02-17T12:42:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.030+0000 7eff71116700 1 -- 172.21.15.119:0/469368096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6c19fca0 con 0x7eff6c102ea0 2024-02-17T12:42:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.030+0000 7eff53fff700 1 -- 172.21.15.119:0/469368096 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff5c022470 con 0x7eff6c102ea0 2024-02-17T12:42:26.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.030+0000 7eff53fff700 1 -- 172.21.15.119:0/469368096 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00f6a0 con 0x7eff6c102ea0 2024-02-17T12:42:26.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.030+0000 7eff71116700 1 -- 172.21.15.119:0/469368096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c1a0190 con 0x7eff6c102ea0 2024-02-17T12:42:26.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.031+0000 7eff71116700 1 -- 172.21.15.119:0/469368096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6c109060 con 0x7eff6c102ea0 2024-02-17T12:42:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.032+0000 7eff53fff700 1 -- 172.21.15.119:0/469368096 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7eff5c0225e0 con 0x7eff6c102ea0 2024-02-17T12:42:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.033+0000 7eff53fff700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7eff540a20c0 0x7eff540a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:42:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.033+0000 7eff53fff700 1 -- 172.21.15.119:0/469368096 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7eff5c0cdd50 con 0x7eff6c102ea0 2024-02-17T12:42:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.033+0000 7eff6ad9d700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7eff540a20c0 0x7eff540a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:42:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.034+0000 7eff6ad9d700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7eff540a20c0 0x7eff540a4580 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7eff60005fd0 tx=0x7eff60005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:42:26.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.037+0000 7eff53fff700 1 -- 172.21.15.119:0/469368096 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff5c098800 con 0x7eff6c102ea0 2024-02-17T12:42:26.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:26.238+0000 7eff71116700 1 -- 172.21.15.119:0/469368096 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7eff6c02d8e0 con 0x7eff540a20c0 2024-02-17T12:42:26.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:26 smithi018 ceph-mon[32625]: osdmap e9: 1 total, 1 up, 1 in 2024-02-17T12:42:26.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:26 smithi119 ceph-mon[36188]: osdmap e9: 1 total, 1 up, 1 in 2024-02-17T12:42:26.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:26 smithi146 ceph-mon[36454]: osdmap e9: 1 total, 1 up, 1 in 2024-02-17T12:42:27.360 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:27 smithi119 ceph-mon[36188]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:27.361 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:27 smithi119 ceph-mon[36188]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:42:27.361 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:27 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:42:27.361 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:27 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:42:27.361 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:27 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:27.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:27 smithi018 ceph-mon[32625]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:27.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:27 smithi018 ceph-mon[32625]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:42:27.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:27 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:42:27.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:27 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:42:27.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:27 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:27.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:27 smithi146 ceph-mon[36454]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:27.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:27 smithi146 ceph-mon[36454]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:42:27.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:27 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:42:27.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:27 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:42:27.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:27 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:29.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:29 smithi119 ceph-mon[36188]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:29.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:29 smithi119 ceph-mon[36188]: from='client.? 172.21.15.119:0/187430202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c69905b-1e0f-4b63-90d6-bb6d7a5f752f"}]: dispatch 2024-02-17T12:42:29.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:29 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c69905b-1e0f-4b63-90d6-bb6d7a5f752f"}]: dispatch 2024-02-17T12:42:29.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:29 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c69905b-1e0f-4b63-90d6-bb6d7a5f752f"}]': finished 2024-02-17T12:42:29.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:29 smithi119 ceph-mon[36188]: osdmap e10: 2 total, 1 up, 2 in 2024-02-17T12:42:29.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:29 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:29.506 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:29 smithi146 ceph-mon[36454]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:29.506 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:29 smithi146 ceph-mon[36454]: from='client.? 172.21.15.119:0/187430202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c69905b-1e0f-4b63-90d6-bb6d7a5f752f"}]: dispatch 2024-02-17T12:42:29.506 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:29 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c69905b-1e0f-4b63-90d6-bb6d7a5f752f"}]: dispatch 2024-02-17T12:42:29.506 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:29 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c69905b-1e0f-4b63-90d6-bb6d7a5f752f"}]': finished 2024-02-17T12:42:29.506 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:29 smithi146 ceph-mon[36454]: osdmap e10: 2 total, 1 up, 2 in 2024-02-17T12:42:29.506 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:29 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:29.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:29 smithi018 ceph-mon[32625]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:29.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:29 smithi018 ceph-mon[32625]: from='client.? 172.21.15.119:0/187430202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c69905b-1e0f-4b63-90d6-bb6d7a5f752f"}]: dispatch 2024-02-17T12:42:29.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:29 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c69905b-1e0f-4b63-90d6-bb6d7a5f752f"}]: dispatch 2024-02-17T12:42:29.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:29 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c69905b-1e0f-4b63-90d6-bb6d7a5f752f"}]': finished 2024-02-17T12:42:29.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:29 smithi018 ceph-mon[32625]: osdmap e10: 2 total, 1 up, 2 in 2024-02-17T12:42:29.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:29 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:30.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:30 smithi018 ceph-mon[32625]: from='client.? 172.21.15.119:0/2922934084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:42:30.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:30 smithi119 ceph-mon[36188]: from='client.? 172.21.15.119:0/2922934084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:42:30.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:30 smithi146 ceph-mon[36454]: from='client.? 172.21.15.119:0/2922934084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:42:31.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:31 smithi018 ceph-mon[32625]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:31.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:31 smithi119 ceph-mon[36188]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:31.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:31 smithi146 ceph-mon[36454]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:33.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:33 smithi018 ceph-mon[32625]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:33.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:33 smithi119 ceph-mon[36188]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:33.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:33 smithi146 ceph-mon[36454]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:35.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:35 smithi018 ceph-mon[32625]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:35.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:35 smithi119 ceph-mon[36188]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:35.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:35 smithi146 ceph-mon[36454]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:36.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:36 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-17T12:42:36.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:36 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:36.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:36 smithi018 ceph-mon[32625]: Deploying daemon osd.1 on smithi119 2024-02-17T12:42:36.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:36 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-17T12:42:36.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:36 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:36.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:36 smithi119 ceph-mon[36188]: Deploying daemon osd.1 on smithi119 2024-02-17T12:42:36.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:36 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-17T12:42:36.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:36 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:36.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:36 smithi146 ceph-mon[36454]: Deploying daemon osd.1 on smithi119 2024-02-17T12:42:37.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:37 smithi119 ceph-mon[36188]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:37.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:37 smithi018 ceph-mon[32625]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:37.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:37 smithi146 ceph-mon[36454]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:39.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:39 smithi018 ceph-mon[32625]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:39.569 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:39 smithi119 ceph-mon[36188]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:39.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:39 smithi146 ceph-mon[36454]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:41.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:41 smithi018 ceph-mon[32625]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:41.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:41 smithi119 ceph-mon[36188]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:41.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:41 smithi146 ceph-mon[36454]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:42.598 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:42.599 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:42.599 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:42.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:42.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:42.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:42.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:42.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:42.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:43.584 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:43 smithi119 ceph-mon[36188]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:43.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:43 smithi146 ceph-mon[36454]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:43.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:43 smithi018 ceph-mon[32625]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:44.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.754+0000 7eff53fff700 1 -- 172.21.15.119:0/469368096 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7eff6c02d8e0 con 0x7eff540a20c0 2024-02-17T12:42:44.749 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 1 on host 'smithi119' 2024-02-17T12:42:44.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.756+0000 7eff71116700 1 -- 172.21.15.119:0/469368096 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7eff540a20c0 msgr2=0x7eff540a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:44.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.756+0000 7eff71116700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7eff540a20c0 0x7eff540a4580 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7eff60005fd0 tx=0x7eff60005e50 comp rx=0 tx=0).stop 2024-02-17T12:42:44.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.756+0000 7eff71116700 1 -- 172.21.15.119:0/469368096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c102ea0 msgr2=0x7eff6c198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:44.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.756+0000 7eff71116700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c102ea0 0x7eff6c198100 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff5c003790 tx=0x7eff5c0047b0 comp rx=0 tx=0).stop 2024-02-17T12:42:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.756+0000 7eff71116700 1 -- 172.21.15.119:0/469368096 shutdown_connections 2024-02-17T12:42:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.756+0000 7eff71116700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff6c103840 0x7eff6c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.756+0000 7eff71116700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7eff540a20c0 0x7eff540a4580 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.756+0000 7eff71116700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c101ca0 0x7eff6c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.756+0000 7eff71116700 1 --2- 172.21.15.119:0/469368096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c102ea0 0x7eff6c198100 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.756+0000 7eff71116700 1 -- 172.21.15.119:0/469368096 >> 172.21.15.119:0/469368096 conn(0x7eff6c0fd220 msgr2=0x7eff6c1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:42:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.756+0000 7eff71116700 1 -- 172.21.15.119:0/469368096 shutdown_connections 2024-02-17T12:42:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:42:44.756+0000 7eff71116700 1 -- 172.21.15.119:0/469368096 wait complete. 2024-02-17T12:42:44.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:44 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:44.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:44 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:44.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:44 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:44.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:44 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:44.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:44 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:45.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:44 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:45.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:44 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:45.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:44 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:45.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:44 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:45.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:44 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:45.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:44 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:45.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:44 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:45.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:44 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:45.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:44 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:45.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:44 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:45.398 DEBUG:teuthology.orchestra.run.smithi119:osd.1> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.1.service 2024-02-17T12:42:45.400 INFO:tasks.cephadm:Deploying osd.2 on smithi146 with /dev/vg_nvme/lv_4... 2024-02-17T12:42:45.400 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-17T12:42:45.646 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:42:45.988 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:45 smithi146 ceph-mon[36454]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:45.988 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:45.988 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:45.989 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:45.989 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:45 smithi146 ceph-mon[36454]: from='osd.1 [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-17T12:42:45.989 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:45 smithi146 ceph-mon[36454]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-17T12:42:46.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:45 smithi018 ceph-mon[32625]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:46.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:46.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:46.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:46.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:45 smithi018 ceph-mon[32625]: from='osd.1 [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-17T12:42:46.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:45 smithi018 ceph-mon[32625]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-17T12:42:46.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:45 smithi119 ceph-mon[36188]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:46.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:46.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:46.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:42:46.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:45 smithi119 ceph-mon[36188]: from='osd.1 [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-17T12:42:46.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:45 smithi119 ceph-mon[36188]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-17T12:42:46.976 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:46 smithi146 ceph-mon[36454]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-17T12:42:46.977 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:46 smithi146 ceph-mon[36454]: from='osd.1 [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:42:46.977 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:46 smithi146 ceph-mon[36454]: osdmap e11: 2 total, 1 up, 2 in 2024-02-17T12:42:46.977 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:46 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:46.977 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:46 smithi146 ceph-mon[36454]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:42:47.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:46 smithi018 ceph-mon[32625]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-17T12:42:47.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:46 smithi018 ceph-mon[32625]: from='osd.1 [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:42:47.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:46 smithi018 ceph-mon[32625]: osdmap e11: 2 total, 1 up, 2 in 2024-02-17T12:42:47.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:46 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:47.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:46 smithi018 ceph-mon[32625]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:42:47.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:46 smithi119 ceph-mon[36188]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-17T12:42:47.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:46 smithi119 ceph-mon[36188]: from='osd.1 [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:42:47.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:46 smithi119 ceph-mon[36188]: osdmap e11: 2 total, 1 up, 2 in 2024-02-17T12:42:47.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:46 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:47.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:46 smithi119 ceph-mon[36188]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:42:47.558 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-17T12:42:47.558 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-17T12:42:47.558 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2024-02-17T12:42:47.558 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2024-02-17T12:42:47.558 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113944 s, 920 MB/s 2024-02-17T12:42:47.558 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2024-02-17T12:42:47.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:47 smithi119 ceph-mon[36188]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:47.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:47 smithi119 ceph-mon[36188]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-02-17T12:42:47.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:47 smithi119 ceph-mon[36188]: osdmap e12: 2 total, 1 up, 2 in 2024-02-17T12:42:47.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:47 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:47.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:47 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:47.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:47 smithi119 ceph-mon[36188]: from='osd.1 ' entity='osd.1' 2024-02-17T12:42:47.848 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 12:42:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1[40780]: 2024-02-17T12:42:47.544+0000 7f40b2791700 -1 osd.1 0 waiting for initial osdmap 2024-02-17T12:42:47.848 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 12:42:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1[40780]: 2024-02-17T12:42:47.552+0000 7f40aad7d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T12:42:48.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:47 smithi018 ceph-mon[32625]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:48.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:47 smithi018 ceph-mon[32625]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-02-17T12:42:48.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:47 smithi018 ceph-mon[32625]: osdmap e12: 2 total, 1 up, 2 in 2024-02-17T12:42:48.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:47 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:48.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:47 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:48.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:47 smithi018 ceph-mon[32625]: from='osd.1 ' entity='osd.1' 2024-02-17T12:42:48.106 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:47 smithi146 ceph-mon[36454]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:48.106 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:47 smithi146 ceph-mon[36454]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-02-17T12:42:48.106 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:47 smithi146 ceph-mon[36454]: osdmap e12: 2 total, 1 up, 2 in 2024-02-17T12:42:48.106 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:47 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:48.106 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:47 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:48.106 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:47 smithi146 ceph-mon[36454]: from='osd.1 ' entity='osd.1' 2024-02-17T12:42:48.132 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi146:vg_nvme/lv_4 2024-02-17T12:42:48.381 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:42:48.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: purged_snaps scrub starts 2024-02-17T12:42:48.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: purged_snaps scrub ok 2024-02-17T12:42:48.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:48.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:48.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:48.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:42:48.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:48.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:48.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:48.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: osd.1 [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131] boot 2024-02-17T12:42:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: osdmap e13: 2 total, 2 up, 2 in 2024-02-17T12:42:48.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:49.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: purged_snaps scrub starts 2024-02-17T12:42:49.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: purged_snaps scrub ok 2024-02-17T12:42:49.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:49.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:49.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:49.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:42:49.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:49.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:49.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:49.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:49.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: osd.1 [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131] boot 2024-02-17T12:42:49.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: osdmap e13: 2 total, 2 up, 2 in 2024-02-17T12:42:49.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:49.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: purged_snaps scrub starts 2024-02-17T12:42:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: purged_snaps scrub ok 2024-02-17T12:42:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:42:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:42:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:42:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: osd.1 [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131] boot 2024-02-17T12:42:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: osdmap e13: 2 total, 2 up, 2 in 2024-02-17T12:42:49.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:42:49.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.679+0000 7f34e5ae3700 1 -- 172.21.15.146:0/2103837669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0103830 msgr2=0x7f34e010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:49.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.679+0000 7f34e5ae3700 1 --2- 172.21.15.146:0/2103837669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0103830 0x7f34e010a0e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f34d800b0a0 tx=0x7f34d800b3b0 comp rx=0 tx=0).stop 2024-02-17T12:42:49.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.680+0000 7f34e5ae3700 1 -- 172.21.15.146:0/2103837669 shutdown_connections 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.680+0000 7f34e5ae3700 1 --2- 172.21.15.146:0/2103837669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0103830 0x7f34e010a0e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.680+0000 7f34e5ae3700 1 --2- 172.21.15.146:0/2103837669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34e0101c90 0x7f34e01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.680+0000 7f34e5ae3700 1 --2- 172.21.15.146:0/2103837669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34e0102e90 0x7f34e01032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.680+0000 7f34e5ae3700 1 -- 172.21.15.146:0/2103837669 >> 172.21.15.146:0/2103837669 conn(0x7f34e00fd250 msgr2=0x7f34e00ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.680+0000 7f34e5ae3700 1 -- 172.21.15.146:0/2103837669 shutdown_connections 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.680+0000 7f34e5ae3700 1 -- 172.21.15.146:0/2103837669 wait complete. 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.680+0000 7f34e5ae3700 1 Processor -- start 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.680+0000 7f34e5ae3700 1 -- start start 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34e5ae3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34e0101c90 0x7f34e0199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34e5ae3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0102e90 0x7f34e019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34e5ae3700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34e0103830 0x7f34e01a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34e5ae3700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e01a1a70 con 0x7f34e0103830 2024-02-17T12:42:49.678 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34e5ae3700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e01a1bb0 con 0x7f34e0101c90 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34e5ae3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e01a1d40 con 0x7f34e0102e90 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34deffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0102e90 0x7f34e019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34deffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0102e90 0x7f34e019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:56736/0 (socket says 172.21.15.146:56736) 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34deffd700 1 -- 172.21.15.146:0/3143866033 learned_addr learned my addr 172.21.15.146:0/3143866033 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34dffff700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34e0103830 0x7f34e01a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34df7fe700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34e0101c90 0x7f34e0199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34deffd700 1 -- 172.21.15.146:0/3143866033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34e0101c90 msgr2=0x7f34e0199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34deffd700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34e0101c90 0x7f34e0199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34deffd700 1 -- 172.21.15.146:0/3143866033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34e0103830 msgr2=0x7f34e01a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34deffd700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34e0103830 0x7f34e01a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34deffd700 1 -- 172.21.15.146:0/3143866033 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d8009d00 con 0x7f34e0102e90 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34deffd700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0102e90 0x7f34e019a420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f34d400b560 tx=0x7f34d400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34dcff9700 1 -- 172.21.15.146:0/3143866033 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4011830 con 0x7f34e0102e90 2024-02-17T12:42:49.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34dcff9700 1 -- 172.21.15.146:0/3143866033 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34d4011e70 con 0x7f34e0102e90 2024-02-17T12:42:49.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34e5ae3700 1 -- 172.21.15.146:0/3143866033 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e01a1fc0 con 0x7f34e0102e90 2024-02-17T12:42:49.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34dcff9700 1 -- 172.21.15.146:0/3143866033 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d400f2e0 con 0x7f34e0102e90 2024-02-17T12:42:49.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.681+0000 7f34e5ae3700 1 -- 172.21.15.146:0/3143866033 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e01a2560 con 0x7f34e0102e90 2024-02-17T12:42:49.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.683+0000 7f34e5ae3700 1 -- 172.21.15.146:0/3143866033 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e0068930 con 0x7f34e0102e90 2024-02-17T12:42:49.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.683+0000 7f34dcff9700 1 -- 172.21.15.146:0/3143866033 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f34d40105f0 con 0x7f34e0102e90 2024-02-17T12:42:49.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.684+0000 7f34dcff9700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f34c80a2140 0x7f34c80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:42:49.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.684+0000 7f34dcff9700 1 -- 172.21.15.146:0/3143866033 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f34d40cbe00 con 0x7f34e0102e90 2024-02-17T12:42:49.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.684+0000 7f34df7fe700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f34c80a2140 0x7f34c80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:42:49.681 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.685+0000 7f34df7fe700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f34c80a2140 0x7f34c80a4600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34d0005fd0 tx=0x7f34d0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:42:49.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.689+0000 7f34dcff9700 1 -- 172.21.15.146:0/3143866033 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f34d4096620 con 0x7f34e0102e90 2024-02-17T12:42:49.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:42:49.892+0000 7f34e5ae3700 1 -- 172.21.15.146:0/3143866033 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f34e00755c0 con 0x7f34c80a2140 2024-02-17T12:42:49.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:49 smithi146 ceph-mon[36454]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:49.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:49 smithi146 ceph-mon[36454]: Detected new or changed devices on smithi119 2024-02-17T12:42:49.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:49 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi119 to 17080M 2024-02-17T12:42:50.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:49 smithi018 ceph-mon[32625]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:50.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:49 smithi018 ceph-mon[32625]: Detected new or changed devices on smithi119 2024-02-17T12:42:50.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:49 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi119 to 17080M 2024-02-17T12:42:50.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:49 smithi119 ceph-mon[36188]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-17T12:42:50.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:49 smithi119 ceph-mon[36188]: Detected new or changed devices on smithi119 2024-02-17T12:42:50.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:49 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi119 to 17080M 2024-02-17T12:42:51.019 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:50 smithi146 ceph-mon[36454]: osdmap e14: 2 total, 2 up, 2 in 2024-02-17T12:42:51.019 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:50 smithi146 ceph-mon[36454]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:42:51.019 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:50 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:42:51.020 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:50 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:42:51.020 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:50 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:51.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:50 smithi018 ceph-mon[32625]: osdmap e14: 2 total, 2 up, 2 in 2024-02-17T12:42:51.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:50 smithi018 ceph-mon[32625]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:42:51.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:50 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:42:51.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:50 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:42:51.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:50 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:51.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:50 smithi119 ceph-mon[36188]: osdmap e14: 2 total, 2 up, 2 in 2024-02-17T12:42:51.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:50 smithi119 ceph-mon[36188]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:42:51.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:50 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:42:51.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:50 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:42:51.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:50 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:42:51.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:51 smithi146 ceph-mon[36454]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:42:52.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:51 smithi018 ceph-mon[32625]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:42:52.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:51 smithi119 ceph-mon[36188]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:42:53.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:52 smithi018 ceph-mon[32625]: from='client.? 172.21.15.146:0/1393280520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5964f903-92b4-4f81-b9d7-861f14ef1d38"}]: dispatch 2024-02-17T12:42:53.298 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:52 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5964f903-92b4-4f81-b9d7-861f14ef1d38"}]: dispatch 2024-02-17T12:42:53.298 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:52 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5964f903-92b4-4f81-b9d7-861f14ef1d38"}]': finished 2024-02-17T12:42:53.298 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:52 smithi018 ceph-mon[32625]: osdmap e15: 3 total, 2 up, 3 in 2024-02-17T12:42:53.299 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:42:53.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:52 smithi146 ceph-mon[36454]: from='client.? 172.21.15.146:0/1393280520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5964f903-92b4-4f81-b9d7-861f14ef1d38"}]: dispatch 2024-02-17T12:42:53.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:52 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5964f903-92b4-4f81-b9d7-861f14ef1d38"}]: dispatch 2024-02-17T12:42:53.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:52 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5964f903-92b4-4f81-b9d7-861f14ef1d38"}]': finished 2024-02-17T12:42:53.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:52 smithi146 ceph-mon[36454]: osdmap e15: 3 total, 2 up, 3 in 2024-02-17T12:42:53.300 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:42:53.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:52 smithi119 ceph-mon[36188]: from='client.? 172.21.15.146:0/1393280520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5964f903-92b4-4f81-b9d7-861f14ef1d38"}]: dispatch 2024-02-17T12:42:53.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:52 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5964f903-92b4-4f81-b9d7-861f14ef1d38"}]: dispatch 2024-02-17T12:42:53.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:52 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5964f903-92b4-4f81-b9d7-861f14ef1d38"}]': finished 2024-02-17T12:42:53.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:52 smithi119 ceph-mon[36188]: osdmap e15: 3 total, 2 up, 3 in 2024-02-17T12:42:53.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:42:54.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:53 smithi018 ceph-mon[32625]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:42:54.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:53 smithi018 ceph-mon[32625]: from='client.? 172.21.15.146:0/609017290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:42:54.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:53 smithi119 ceph-mon[36188]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:42:54.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:53 smithi119 ceph-mon[36188]: from='client.? 172.21.15.146:0/609017290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:42:54.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:53 smithi146 ceph-mon[36454]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:42:54.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:53 smithi146 ceph-mon[36454]: from='client.? 172.21.15.146:0/609017290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:42:56.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:55 smithi018 ceph-mon[32625]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:42:56.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:55 smithi119 ceph-mon[36188]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:42:56.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:55 smithi146 ceph-mon[36454]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:42:58.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:57 smithi018 ceph-mon[32625]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:42:58.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:57 smithi119 ceph-mon[36188]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:42:58.140 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:57 smithi146 ceph-mon[36454]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:42:59.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:42:59 smithi146 ceph-mon[36454]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:00.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:42:59 smithi018 ceph-mon[32625]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:00.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:42:59 smithi119 ceph-mon[36188]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:00.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-17T12:43:00.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:00.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:00 smithi146 ceph-mon[36454]: Deploying daemon osd.2 on smithi146 2024-02-17T12:43:01.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-17T12:43:01.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:01.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:00 smithi018 ceph-mon[32625]: Deploying daemon osd.2 on smithi146 2024-02-17T12:43:01.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-17T12:43:01.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:01.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:00 smithi119 ceph-mon[36188]: Deploying daemon osd.2 on smithi146 2024-02-17T12:43:02.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:01 smithi018 ceph-mon[32625]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:02.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:01 smithi119 ceph-mon[36188]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:02.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:01 smithi146 ceph-mon[36454]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:04.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:03 smithi018 ceph-mon[32625]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:04.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:03 smithi119 ceph-mon[36188]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:04.173 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:03 smithi146 ceph-mon[36454]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:06 smithi146 ceph-mon[36454]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:06.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:06 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:06.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:06 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:06.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:06 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:06.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:06 smithi018 ceph-mon[32625]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:06.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:06 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:06.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:06 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:06.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:06 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:06.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:06 smithi119 ceph-mon[36188]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:06.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:06 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:06.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:06 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:06.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:06 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:08.232 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:08 smithi146 ceph-mon[36454]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:08.232 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:08 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:08.232 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:08 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:08.233 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:08 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:08.233 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:08 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:08.233 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:08 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:08.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:08 smithi018 ceph-mon[32625]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:08.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:08 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:08.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:08 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:08.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:08 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:08.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:08 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:08.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:08 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:08.308 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 2 on host 'smithi146' 2024-02-17T12:43:08.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.312+0000 7f34dcff9700 1 -- 172.21.15.146:0/3143866033 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f34e00755c0 con 0x7f34c80a2140 2024-02-17T12:43:08.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.314+0000 7f34e5ae3700 1 -- 172.21.15.146:0/3143866033 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f34c80a2140 msgr2=0x7f34c80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:43:08.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.314+0000 7f34e5ae3700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f34c80a2140 0x7f34c80a4600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34d0005fd0 tx=0x7f34d0005e50 comp rx=0 tx=0).stop 2024-02-17T12:43:08.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.314+0000 7f34e5ae3700 1 -- 172.21.15.146:0/3143866033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0102e90 msgr2=0x7f34e019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:43:08.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.314+0000 7f34e5ae3700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0102e90 0x7f34e019a420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f34d400b560 tx=0x7f34d400b870 comp rx=0 tx=0).stop 2024-02-17T12:43:08.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.314+0000 7f34e5ae3700 1 -- 172.21.15.146:0/3143866033 shutdown_connections 2024-02-17T12:43:08.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.314+0000 7f34e5ae3700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34e0103830 0x7f34e01a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:08.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.314+0000 7f34e5ae3700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34e0101c90 0x7f34e0199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:08.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.314+0000 7f34e5ae3700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f34c80a2140 0x7f34c80a4600 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:08.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.314+0000 7f34e5ae3700 1 --2- 172.21.15.146:0/3143866033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0102e90 0x7f34e019a420 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:08.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.314+0000 7f34e5ae3700 1 -- 172.21.15.146:0/3143866033 >> 172.21.15.146:0/3143866033 conn(0x7f34e00fd250 msgr2=0x7f34e0073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:43:08.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.315+0000 7f34e5ae3700 1 -- 172.21.15.146:0/3143866033 shutdown_connections 2024-02-17T12:43:08.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:43:08.315+0000 7f34e5ae3700 1 -- 172.21.15.146:0/3143866033 wait complete. 2024-02-17T12:43:08.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:08 smithi119 ceph-mon[36188]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:08.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:08 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:08.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:08 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:08.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:08 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:08.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:08 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:08.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:08 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:08.877 DEBUG:teuthology.orchestra.run.smithi146:osd.2> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.2.service 2024-02-17T12:43:08.880 INFO:tasks.cephadm:Deploying osd.3 on smithi018 with /dev/vg_nvme/lv_3... 2024-02-17T12:43:08.880 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-17T12:43:09.133 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:43:09.147 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 12:43:08 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2[41608]: 2024-02-17T12:43:08.935+0000 7fa5abe707c0 -1 osd.2 0 log_to_monitors true 2024-02-17T12:43:09.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:09 smithi146 ceph-mon[36454]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:09.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:09 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:09.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:09 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:09.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:09 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:09.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:09 smithi146 ceph-mon[36454]: from='osd.2 [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-17T12:43:09.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:09 smithi146 ceph-mon[36454]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-17T12:43:09.462 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:09 smithi018 ceph-mon[32625]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:09.462 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:09 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:09.462 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:09 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:09.462 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:09 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:09.463 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:09 smithi018 ceph-mon[32625]: from='osd.2 [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-17T12:43:09.463 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:09 smithi018 ceph-mon[32625]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-17T12:43:09.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:09 smithi119 ceph-mon[36188]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:09.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:09 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:09.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:09 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:09.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:09 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:09.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:09 smithi119 ceph-mon[36188]: from='osd.2 [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-17T12:43:09.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:09 smithi119 ceph-mon[36188]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-17T12:43:10.416 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:10 smithi018 ceph-mon[32625]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-17T12:43:10.416 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:10 smithi018 ceph-mon[32625]: osdmap e16: 3 total, 2 up, 3 in 2024-02-17T12:43:10.416 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:10 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:10.416 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:10 smithi018 ceph-mon[32625]: from='osd.2 [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:43:10.416 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:10 smithi018 ceph-mon[32625]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:43:10.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:10 smithi119 ceph-mon[36188]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-17T12:43:10.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:10 smithi119 ceph-mon[36188]: osdmap e16: 3 total, 2 up, 3 in 2024-02-17T12:43:10.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:10 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:10.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:10 smithi119 ceph-mon[36188]: from='osd.2 [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:43:10.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:10 smithi119 ceph-mon[36188]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:43:10.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:10 smithi146 ceph-mon[36454]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-17T12:43:10.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:10 smithi146 ceph-mon[36454]: osdmap e16: 3 total, 2 up, 3 in 2024-02-17T12:43:10.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:10 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:10.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:10 smithi146 ceph-mon[36454]: from='osd.2 [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:43:10.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:10 smithi146 ceph-mon[36454]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:43:10.958 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-17T12:43:10.958 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-17T12:43:10.958 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-02-17T12:43:10.958 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-02-17T12:43:10.958 INFO:teuthology.orchestra.run.smithi018.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00858693 s, 1.2 GB/s 2024-02-17T12:43:10.958 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-02-17T12:43:11.454 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 12:43:11 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2[41608]: 2024-02-17T12:43:11.142+0000 7fa5a24d7700 -1 osd.2 0 waiting for initial osdmap 2024-02-17T12:43:11.454 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 12:43:11 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2[41608]: 2024-02-17T12:43:11.151+0000 7fa5992c0700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T12:43:11.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:11 smithi146 ceph-mon[36454]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:11.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:11 smithi146 ceph-mon[36454]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-02-17T12:43:11.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:11 smithi146 ceph-mon[36454]: osdmap e17: 3 total, 2 up, 3 in 2024-02-17T12:43:11.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:11.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:11.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:11 smithi146 ceph-mon[36454]: from='osd.2 ' entity='osd.2' 2024-02-17T12:43:11.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:11 smithi018 ceph-mon[32625]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:11.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:11 smithi018 ceph-mon[32625]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-02-17T12:43:11.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:11 smithi018 ceph-mon[32625]: osdmap e17: 3 total, 2 up, 3 in 2024-02-17T12:43:11.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:11.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:11.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:11 smithi018 ceph-mon[32625]: from='osd.2 ' entity='osd.2' 2024-02-17T12:43:11.593 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi018:vg_nvme/lv_3 2024-02-17T12:43:11.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:11 smithi119 ceph-mon[36188]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:11.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:11 smithi119 ceph-mon[36188]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-02-17T12:43:11.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:11 smithi119 ceph-mon[36188]: osdmap e17: 3 total, 2 up, 3 in 2024-02-17T12:43:11.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:11.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:11.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:11 smithi119 ceph-mon[36188]: from='osd.2 ' entity='osd.2' 2024-02-17T12:43:11.839 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:43:12.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: purged_snaps scrub starts 2024-02-17T12:43:12.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: purged_snaps scrub ok 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: Detected new or changed devices on smithi146 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi146 to 17080M 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: osd.2 [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578] boot 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: osdmap e18: 3 total, 3 up, 3 in 2024-02-17T12:43:12.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:12.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: purged_snaps scrub starts 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: purged_snaps scrub ok 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: Detected new or changed devices on smithi146 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi146 to 17080M 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: osd.2 [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578] boot 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: osdmap e18: 3 total, 3 up, 3 in 2024-02-17T12:43:12.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:12.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: purged_snaps scrub starts 2024-02-17T12:43:12.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: purged_snaps scrub ok 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: Detected new or changed devices on smithi146 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi146 to 17080M 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: osd.2 [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578] boot 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: osdmap e18: 3 total, 3 up, 3 in 2024-02-17T12:43:12.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:43:13.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.416+0000 7f343aaaf700 1 -- 172.21.15.18:0/174792186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3434100720 msgr2=0x7f343410a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:43:13.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.416+0000 7f343aaaf700 1 --2- 172.21.15.18:0/174792186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3434100720 0x7f343410a2d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f342c00b0a0 tx=0x7f342c00b3b0 comp rx=0 tx=0).stop 2024-02-17T12:43:13.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.416+0000 7f343aaaf700 1 -- 172.21.15.18:0/174792186 shutdown_connections 2024-02-17T12:43:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.416+0000 7f343aaaf700 1 --2- 172.21.15.18:0/174792186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3434100720 0x7f343410a2d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.416+0000 7f343aaaf700 1 --2- 172.21.15.18:0/174792186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34341032e0 0x7f3434103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.416+0000 7f343aaaf700 1 --2- 172.21.15.18:0/174792186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34340ffd60 0x7f34341001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.416+0000 7f343aaaf700 1 -- 172.21.15.18:0/174792186 >> 172.21.15.18:0/174792186 conn(0x7f34340fb900 msgr2=0x7f34340fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:43:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.416+0000 7f343aaaf700 1 -- 172.21.15.18:0/174792186 shutdown_connections 2024-02-17T12:43:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.416+0000 7f343aaaf700 1 -- 172.21.15.18:0/174792186 wait complete. 2024-02-17T12:43:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343aaaf700 1 Processor -- start 2024-02-17T12:43:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343aaaf700 1 -- start start 2024-02-17T12:43:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343aaaf700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34340ffd60 0x7f3434197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:43:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343aaaf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3434100720 0x7f3434198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:43:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343aaaf700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34341032e0 0x7f343419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:43:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343aaaf700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343419fa60 con 0x7f34340ffd60 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343aaaf700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343419fba0 con 0x7f34341032e0 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343aaaf700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343419fd30 con 0x7f3434100720 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343884b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34340ffd60 0x7f3434197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343884b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34340ffd60 0x7f3434197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33868/0 (socket says 172.21.15.18:33868) 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343884b700 1 -- 172.21.15.18:0/567147975 learned_addr learned my addr 172.21.15.18:0/567147975 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f3433fff700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3434100720 0x7f3434198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343884b700 1 -- 172.21.15.18:0/567147975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3434100720 msgr2=0x7f3434198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343884b700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3434100720 0x7f3434198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343904c700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34341032e0 0x7f343419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.417+0000 7f343884b700 1 -- 172.21.15.18:0/567147975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34341032e0 msgr2=0x7f343419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.418+0000 7f343884b700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34341032e0 0x7f343419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.418+0000 7f343884b700 1 -- 172.21.15.18:0/567147975 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c009d00 con 0x7f34340ffd60 2024-02-17T12:43:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.418+0000 7f343904c700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34341032e0 0x7f343419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:43:13.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.418+0000 7f343884b700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34340ffd60 0x7f3434197ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f342800b530 tx=0x7f342800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:43:13.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.418+0000 7f3431ffb700 1 -- 172.21.15.18:0/567147975 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428011830 con 0x7f34340ffd60 2024-02-17T12:43:13.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.418+0000 7f343aaaf700 1 -- 172.21.15.18:0/567147975 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343419ffb0 con 0x7f34340ffd60 2024-02-17T12:43:13.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.418+0000 7f3431ffb700 1 -- 172.21.15.18:0/567147975 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3428011e70 con 0x7f34340ffd60 2024-02-17T12:43:13.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.418+0000 7f3431ffb700 1 -- 172.21.15.18:0/567147975 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342800f2e0 con 0x7f34340ffd60 2024-02-17T12:43:13.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.418+0000 7f343aaaf700 1 -- 172.21.15.18:0/567147975 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34341a04a0 con 0x7f34340ffd60 2024-02-17T12:43:13.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.419+0000 7f343aaaf700 1 -- 172.21.15.18:0/567147975 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34341091e0 con 0x7f34340ffd60 2024-02-17T12:43:13.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.420+0000 7f3431ffb700 1 -- 172.21.15.18:0/567147975 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f34280105f0 con 0x7f34340ffd60 2024-02-17T12:43:13.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.421+0000 7f3431ffb700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f34240a20c0 0x7f34240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:43:13.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.421+0000 7f3433fff700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f34240a20c0 0x7f34240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:43:13.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.421+0000 7f3431ffb700 1 -- 172.21.15.18:0/567147975 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f34280cc800 con 0x7f34340ffd60 2024-02-17T12:43:13.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.421+0000 7f3433fff700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f34240a20c0 0x7f34240a4580 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3420006fd0 tx=0x7f3420008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:43:13.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.425+0000 7f3431ffb700 1 -- 172.21.15.18:0/567147975 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3428096bf0 con 0x7f34340ffd60 2024-02-17T12:43:13.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:13.641+0000 7f343aaaf700 1 -- 172.21.15.18:0/567147975 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f343402d910 con 0x7f34240a20c0 2024-02-17T12:43:13.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:13 smithi146 ceph-mon[36454]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:13.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:13 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-17T12:43:13.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:13 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-17T12:43:13.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:13 smithi146 ceph-mon[36454]: osdmap e19: 3 total, 3 up, 3 in 2024-02-17T12:43:13.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:13 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-17T12:43:13.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:13 smithi018 ceph-mon[32625]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:13.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-17T12:43:13.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-17T12:43:13.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:13 smithi018 ceph-mon[32625]: osdmap e19: 3 total, 3 up, 3 in 2024-02-17T12:43:13.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-17T12:43:13.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:13 smithi119 ceph-mon[36188]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-17T12:43:13.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:13 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-17T12:43:13.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:13 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-17T12:43:13.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:13 smithi119 ceph-mon[36188]: osdmap e19: 3 total, 3 up, 3 in 2024-02-17T12:43:13.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:13 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-17T12:43:14.510 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 12:43:14 smithi018 sudo[51790]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-17T12:43:14.510 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 12:43:14 smithi018 sudo[51790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T12:43:14.510 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 12:43:14 smithi018 sudo[51790]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T12:43:14.510 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 12:43:14 smithi018 sudo[51790]: pam_unix(sudo:session): session closed for user root 2024-02-17T12:43:14.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:14 smithi018 ceph-mon[32625]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:43:14.511 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:14 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:43:14.511 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:14 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:43:14.511 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:14 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:14.511 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:14 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-17T12:43:14.511 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:14 smithi018 ceph-mon[32625]: osdmap e20: 3 total, 3 up, 3 in 2024-02-17T12:43:14.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:14 smithi146 ceph-mon[36454]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:43:14.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:14 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:43:14.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:14 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:43:14.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:14 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:14.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:14 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-17T12:43:14.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:14 smithi146 ceph-mon[36454]: osdmap e20: 3 total, 3 up, 3 in 2024-02-17T12:43:14.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:14 smithi119 ceph-mon[36188]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:43:14.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:14 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:43:14.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:14 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:43:14.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:14 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:14.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:14 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-17T12:43:14.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:14 smithi119 ceph-mon[36188]: osdmap e20: 3 total, 3 up, 3 in 2024-02-17T12:43:15.027 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 12:43:14 smithi018 sudo[51806]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-17T12:43:15.027 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 12:43:14 smithi018 sudo[51806]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T12:43:15.027 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 12:43:14 smithi018 sudo[51806]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T12:43:15.027 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 12:43:14 smithi018 sudo[51806]: pam_unix(sudo:session): session closed for user root 2024-02-17T12:43:15.283 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 12:43:15 smithi119 sudo[44823]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-17T12:43:15.283 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 12:43:15 smithi119 sudo[44823]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T12:43:15.283 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 12:43:15 smithi119 sudo[44823]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T12:43:15.284 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 12:43:15 smithi119 sudo[44823]: pam_unix(sudo:session): session closed for user root 2024-02-17T12:43:15.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:15 smithi018 ceph-mon[32625]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:15.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:15 smithi018 ceph-mon[32625]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-17T12:43:15.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:15.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:15 smithi018 ceph-mon[32625]: osdmap e21: 3 total, 3 up, 3 in 2024-02-17T12:43:15.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:15.596 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 12:43:15 smithi119 sudo[44826]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-17T12:43:15.596 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 12:43:15 smithi119 sudo[44826]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T12:43:15.596 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 12:43:15 smithi119 sudo[44826]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T12:43:15.596 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 12:43:15 smithi119 sudo[44826]: pam_unix(sudo:session): session closed for user root 2024-02-17T12:43:15.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:15 smithi119 ceph-mon[36188]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:15.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:15 smithi119 ceph-mon[36188]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-17T12:43:15.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:15.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:15 smithi119 ceph-mon[36188]: osdmap e21: 3 total, 3 up, 3 in 2024-02-17T12:43:15.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:15.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:15.803+0000 7f3431ffb700 1 -- 172.21.15.18:0/567147975 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3428096fc0 con 0x7f34340ffd60 2024-02-17T12:43:15.806 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 12:43:15 smithi146 sudo[45691]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-17T12:43:15.806 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 12:43:15 smithi146 sudo[45691]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T12:43:15.806 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 12:43:15 smithi146 sudo[45691]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T12:43:15.806 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 12:43:15 smithi146 sudo[45691]: pam_unix(sudo:session): session closed for user root 2024-02-17T12:43:15.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:15 smithi146 ceph-mon[36454]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:15.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:15 smithi146 ceph-mon[36454]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-17T12:43:15.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:15.806 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:15 smithi146 ceph-mon[36454]: osdmap e21: 3 total, 3 up, 3 in 2024-02-17T12:43:15.807 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:16.203 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 12:43:15 smithi146 sudo[45694]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-17T12:43:16.203 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 12:43:15 smithi146 sudo[45694]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T12:43:16.204 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 12:43:15 smithi146 sudo[45694]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T12:43:16.204 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 12:43:15 smithi146 sudo[45694]: pam_unix(sudo:session): session closed for user root 2024-02-17T12:43:16.350 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 sudo[51865]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-17T12:43:16.350 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 sudo[51865]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T12:43:16.350 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 sudo[51865]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T12:43:16.351 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 sudo[51865]: pam_unix(sudo:session): session closed for user root 2024-02-17T12:43:16.717 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 sudo[51887]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-17T12:43:16.717 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 sudo[51887]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T12:43:16.717 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 sudo[51887]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T12:43:16.717 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 sudo[51887]: pam_unix(sudo:session): session closed for user root 2024-02-17T12:43:16.851 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 sudo[45697]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-17T12:43:16.851 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 sudo[45697]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T12:43:16.851 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 sudo[45697]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T12:43:16.851 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 sudo[45697]: pam_unix(sudo:session): session closed for user root 2024-02-17T12:43:17.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-17T12:43:17.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:17.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T12:43:17.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3273485513' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ef3a4f-6073-4998-812e-dc57d4760ecb"}]: dispatch 2024-02-17T12:43:17.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T12:43:17.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:43:17.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ef3a4f-6073-4998-812e-dc57d4760ecb"}]: dispatch 2024-02-17T12:43:17.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:43:17.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:43:17.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7ef3a4f-6073-4998-812e-dc57d4760ecb"}]': finished 2024-02-17T12:43:17.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: osdmap e22: 4 total, 3 up, 4 in 2024-02-17T12:43:17.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:17.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:43:17.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:43:17.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:16 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:43:17.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-17T12:43:17.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:17.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T12:43:17.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3273485513' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ef3a4f-6073-4998-812e-dc57d4760ecb"}]: dispatch 2024-02-17T12:43:17.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T12:43:17.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:43:17.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ef3a4f-6073-4998-812e-dc57d4760ecb"}]: dispatch 2024-02-17T12:43:17.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:43:17.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:43:17.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7ef3a4f-6073-4998-812e-dc57d4760ecb"}]': finished 2024-02-17T12:43:17.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: osdmap e22: 4 total, 3 up, 4 in 2024-02-17T12:43:17.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:17.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:43:17.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:43:17.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:16 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:43:17.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 sudo[44829]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-17T12:43:17.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 sudo[44829]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T12:43:17.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 sudo[44829]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T12:43:17.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 sudo[44829]: pam_unix(sudo:session): session closed for user root 2024-02-17T12:43:17.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 sudo[45700]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-17T12:43:17.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 sudo[45700]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T12:43:17.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 sudo[45700]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T12:43:17.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 sudo[45700]: pam_unix(sudo:session): session closed for user root 2024-02-17T12:43:17.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-17T12:43:17.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:17.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T12:43:17.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3273485513' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ef3a4f-6073-4998-812e-dc57d4760ecb"}]: dispatch 2024-02-17T12:43:17.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T12:43:17.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:43:17.205 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ef3a4f-6073-4998-812e-dc57d4760ecb"}]: dispatch 2024-02-17T12:43:17.205 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:43:17.205 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:43:17.205 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7ef3a4f-6073-4998-812e-dc57d4760ecb"}]': finished 2024-02-17T12:43:17.205 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: osdmap e22: 4 total, 3 up, 4 in 2024-02-17T12:43:17.205 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:17.205 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:43:17.205 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:43:17.205 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:16 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:43:17.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 sudo[44832]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-17T12:43:17.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 sudo[44832]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T12:43:17.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 sudo[44832]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T12:43:17.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 sudo[44832]: pam_unix(sudo:session): session closed for user root 2024-02-17T12:43:18.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:17 smithi146 ceph-mon[36454]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:18.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:17 smithi146 ceph-mon[36454]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T12:43:18.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:17 smithi146 ceph-mon[36454]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-17T12:43:18.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:17 smithi146 ceph-mon[36454]: Cluster is now healthy 2024-02-17T12:43:18.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:17 smithi146 ceph-mon[36454]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T12:43:18.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:17 smithi146 ceph-mon[36454]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T12:43:18.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:17 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:43:18.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:17 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:43:18.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:17 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:43:18.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:17 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2207099041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:43:18.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:17 smithi146 ceph-mon[36454]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T12:43:18.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:17 smithi018 ceph-mon[32625]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:18.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:17 smithi018 ceph-mon[32625]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T12:43:18.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:17 smithi018 ceph-mon[32625]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-17T12:43:18.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:17 smithi018 ceph-mon[32625]: Cluster is now healthy 2024-02-17T12:43:18.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:17 smithi018 ceph-mon[32625]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T12:43:18.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:17 smithi018 ceph-mon[32625]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T12:43:18.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:17 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:43:18.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:17 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:43:18.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:17 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:43:18.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:17 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2207099041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:43:18.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:17 smithi018 ceph-mon[32625]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T12:43:18.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 ceph-mon[36188]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:18.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 ceph-mon[36188]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T12:43:18.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 ceph-mon[36188]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-17T12:43:18.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 ceph-mon[36188]: Cluster is now healthy 2024-02-17T12:43:18.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 ceph-mon[36188]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T12:43:18.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 ceph-mon[36188]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T12:43:18.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:43:18.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:43:18.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:43:18.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2207099041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:43:18.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:17 smithi119 ceph-mon[36188]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T12:43:20.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:19 smithi146 ceph-mon[36454]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:20.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:19 smithi018 ceph-mon[32625]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:20.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:19 smithi119 ceph-mon[36188]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:22.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:21 smithi146 ceph-mon[36454]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:22.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:21 smithi018 ceph-mon[32625]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:22.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:21 smithi119 ceph-mon[36188]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:24.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:23 smithi146 ceph-mon[36454]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:24.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:23 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-17T12:43:24.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:23 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:24.216 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:23 smithi018 ceph-mon[32625]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:24.216 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:23 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-17T12:43:24.216 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:23 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:24.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:23 smithi119 ceph-mon[36188]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:24.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:23 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-17T12:43:24.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:23 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:25.125 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:24 smithi018 ceph-mon[32625]: Deploying daemon osd.3 on smithi018 2024-02-17T12:43:25.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:24 smithi146 ceph-mon[36454]: Deploying daemon osd.3 on smithi018 2024-02-17T12:43:25.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:24 smithi119 ceph-mon[36188]: Deploying daemon osd.3 on smithi018 2024-02-17T12:43:26.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:25 smithi018 ceph-mon[32625]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:26.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:25 smithi146 ceph-mon[36454]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:26.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:25 smithi119 ceph-mon[36188]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:28.147 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:27 smithi018 ceph-mon[32625]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:28.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:27 smithi146 ceph-mon[36454]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:28.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:27 smithi119 ceph-mon[36188]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:30.280 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:30 smithi018 ceph-mon[32625]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:30.280 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:30.280 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:30.280 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:30.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:30 smithi119 ceph-mon[36188]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:30.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:30.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:30.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:30.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:30 smithi146 ceph-mon[36454]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:30.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:30.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:30.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:32.120 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:32 smithi018 ceph-mon[32625]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:32.120 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:32 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:32.120 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:32 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:32.120 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:32 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:32.120 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:32 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:32.120 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:32 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:32.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:32 smithi119 ceph-mon[36188]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:32.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:32 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:32.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:32 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:32.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:32 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:32.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:32 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:32.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:32 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:32.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:32 smithi146 ceph-mon[36454]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:32.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:32 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:32.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:32 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:32.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:32 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:32.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:32 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:32.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:32 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:32.513 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 3 on host 'smithi018' 2024-02-17T12:43:32.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.518+0000 7f3431ffb700 1 -- 172.21.15.18:0/567147975 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f343402d910 con 0x7f34240a20c0 2024-02-17T12:43:32.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.520+0000 7f343aaaf700 1 -- 172.21.15.18:0/567147975 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f34240a20c0 msgr2=0x7f34240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:43:32.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.520+0000 7f343aaaf700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f34240a20c0 0x7f34240a4580 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3420006fd0 tx=0x7f3420008040 comp rx=0 tx=0).stop 2024-02-17T12:43:32.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.520+0000 7f343aaaf700 1 -- 172.21.15.18:0/567147975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34340ffd60 msgr2=0x7f3434197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:43:32.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.520+0000 7f343aaaf700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34340ffd60 0x7f3434197ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f342800b530 tx=0x7f342800b8f0 comp rx=0 tx=0).stop 2024-02-17T12:43:32.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.521+0000 7f343aaaf700 1 -- 172.21.15.18:0/567147975 shutdown_connections 2024-02-17T12:43:32.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.521+0000 7f343aaaf700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34341032e0 0x7f343419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:32.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.521+0000 7f343aaaf700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34340ffd60 0x7f3434197ed0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:32.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.521+0000 7f343aaaf700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f34240a20c0 0x7f34240a4580 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:32.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.521+0000 7f343aaaf700 1 --2- 172.21.15.18:0/567147975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3434100720 0x7f3434198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:32.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.521+0000 7f343aaaf700 1 -- 172.21.15.18:0/567147975 >> 172.21.15.18:0/567147975 conn(0x7f34340fb900 msgr2=0x7f3434106320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:43:32.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.521+0000 7f343aaaf700 1 -- 172.21.15.18:0/567147975 shutdown_connections 2024-02-17T12:43:32.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:43:32.521+0000 7f343aaaf700 1 -- 172.21.15.18:0/567147975 wait complete. 2024-02-17T12:43:33.189 DEBUG:teuthology.orchestra.run.smithi018:osd.3> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.3.service 2024-02-17T12:43:33.191 INFO:tasks.cephadm:Deploying osd.4 on smithi119 with /dev/vg_nvme/lv_3... 2024-02-17T12:43:33.191 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-17T12:43:33.434 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T12:43:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:33 smithi119 ceph-mon[36188]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:33 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:33 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:33 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:33 smithi119 ceph-mon[36188]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-17T12:43:33.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:33 smithi018 ceph-mon[32625]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:33.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:33 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:33.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:33 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:33.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:33 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:33.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:33 smithi018 ceph-mon[32625]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-17T12:43:33.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:33 smithi146 ceph-mon[36454]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:33.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:33 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:33.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:33 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:33.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:33 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:33.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:33 smithi146 ceph-mon[36454]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-17T12:43:34.628 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:34 smithi018 ceph-mon[32625]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-17T12:43:34.628 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:34 smithi018 ceph-mon[32625]: osdmap e23: 4 total, 3 up, 4 in 2024-02-17T12:43:34.628 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:34 smithi018 ceph-mon[32625]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:43:34.628 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:34 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:34.737 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:34 smithi119 ceph-mon[36188]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-17T12:43:34.737 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:34 smithi119 ceph-mon[36188]: osdmap e23: 4 total, 3 up, 4 in 2024-02-17T12:43:34.737 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:34 smithi119 ceph-mon[36188]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:43:34.737 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:34 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:34.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:34 smithi146 ceph-mon[36454]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-17T12:43:34.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:34 smithi146 ceph-mon[36454]: osdmap e23: 4 total, 3 up, 4 in 2024-02-17T12:43:34.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:34 smithi146 ceph-mon[36454]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:43:34.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:34 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:35.475 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 12:43:35 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3[53931]: 2024-02-17T12:43:35.184+0000 7f999ba80700 -1 osd.3 0 waiting for initial osdmap 2024-02-17T12:43:35.475 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 12:43:35 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3[53931]: 2024-02-17T12:43:35.191+0000 7f999406c700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T12:43:35.624 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:35 smithi119 ceph-mon[36188]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:35.625 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:35 smithi119 ceph-mon[36188]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-02-17T12:43:35.625 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:35 smithi119 ceph-mon[36188]: osdmap e24: 4 total, 3 up, 4 in 2024-02-17T12:43:35.625 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:35 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:35.625 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:35 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:35.625 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:35 smithi119 ceph-mon[36188]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' 2024-02-17T12:43:35.625 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:35 smithi119 ceph-mon[36188]: osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256] boot 2024-02-17T12:43:35.625 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:35 smithi119 ceph-mon[36188]: osdmap e25: 4 total, 4 up, 4 in 2024-02-17T12:43:35.625 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:35 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:35.625 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-17T12:43:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-17T12:43:35.626 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-02-17T12:43:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-02-17T12:43:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110727 s, 947 MB/s 2024-02-17T12:43:35.626 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 2024-02-17T12:43:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-02-17T12:43:35.770 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:35 smithi018 ceph-mon[32625]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:35.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:35 smithi018 ceph-mon[32625]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-02-17T12:43:35.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:35 smithi018 ceph-mon[32625]: osdmap e24: 4 total, 3 up, 4 in 2024-02-17T12:43:35.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:35 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:35.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:35 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:35.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:35 smithi018 ceph-mon[32625]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' 2024-02-17T12:43:35.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:35 smithi018 ceph-mon[32625]: osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256] boot 2024-02-17T12:43:35.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:35 smithi018 ceph-mon[32625]: osdmap e25: 4 total, 4 up, 4 in 2024-02-17T12:43:35.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:35 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:35.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:35 smithi146 ceph-mon[36454]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-17T12:43:35.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:35 smithi146 ceph-mon[36454]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-02-17T12:43:35.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:35 smithi146 ceph-mon[36454]: osdmap e24: 4 total, 3 up, 4 in 2024-02-17T12:43:35.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:35 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:35.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:35 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:35.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:35 smithi146 ceph-mon[36454]: from='osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256]' entity='osd.3' 2024-02-17T12:43:35.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:35 smithi146 ceph-mon[36454]: osd.3 [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256] boot 2024-02-17T12:43:35.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:35 smithi146 ceph-mon[36454]: osdmap e25: 4 total, 4 up, 4 in 2024-02-17T12:43:35.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:35 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:43:36.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_3 2024-02-17T12:43:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T12:43:36.771 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:36 smithi119 ceph-mon[36188]: purged_snaps scrub starts 2024-02-17T12:43:36.772 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:36 smithi119 ceph-mon[36188]: purged_snaps scrub ok 2024-02-17T12:43:36.772 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:36 smithi119 ceph-mon[36188]: osdmap e26: 4 total, 4 up, 4 in 2024-02-17T12:43:36.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:36 smithi018 ceph-mon[32625]: purged_snaps scrub starts 2024-02-17T12:43:36.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:36 smithi018 ceph-mon[32625]: purged_snaps scrub ok 2024-02-17T12:43:36.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:36 smithi018 ceph-mon[32625]: osdmap e26: 4 total, 4 up, 4 in 2024-02-17T12:43:36.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:36 smithi146 ceph-mon[36454]: purged_snaps scrub starts 2024-02-17T12:43:36.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:36 smithi146 ceph-mon[36454]: purged_snaps scrub ok 2024-02-17T12:43:36.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:36 smithi146 ceph-mon[36454]: osdmap e26: 4 total, 4 up, 4 in 2024-02-17T12:43:37.725 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:37 smithi119 ceph-mon[36188]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 525 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:37.725 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:37 smithi119 ceph-mon[36188]: Detected new or changed devices on smithi018 2024-02-17T12:43:37.726 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:37.726 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:37.726 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:43:37.726 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:43:37.726 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:37 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi018 to 8546M 2024-02-17T12:43:37.726 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:37.726 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:37.726 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:37.726 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:37.726 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:37 smithi119 ceph-mon[36188]: osdmap e27: 4 total, 4 up, 4 in 2024-02-17T12:43:37.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:37 smithi018 ceph-mon[32625]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 525 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:37.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:37 smithi018 ceph-mon[32625]: Detected new or changed devices on smithi018 2024-02-17T12:43:37.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:37.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:37.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:43:37.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:43:37.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:37 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi018 to 8546M 2024-02-17T12:43:37.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:37.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:37.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:37.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:37.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:37 smithi018 ceph-mon[32625]: osdmap e27: 4 total, 4 up, 4 in 2024-02-17T12:43:37.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.925+0000 7f5f0db26700 1 -- 172.21.15.119:0/1800903738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f08102730 msgr2=0x7f5f08104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:43:37.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.925+0000 7f5f0db26700 1 --2- 172.21.15.119:0/1800903738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f08102730 0x7f5f08104b20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009990 tx=0x7f5ef8009ca0 comp rx=0 tx=0).stop 2024-02-17T12:43:37.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.925+0000 7f5f0db26700 1 -- 172.21.15.119:0/1800903738 shutdown_connections 2024-02-17T12:43:37.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.925+0000 7f5f0db26700 1 --2- 172.21.15.119:0/1800903738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f08107990 0x7f5f08109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:37.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.925+0000 7f5f0db26700 1 --2- 172.21.15.119:0/1800903738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f08102730 0x7f5f08104b20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:37.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.925+0000 7f5f0db26700 1 --2- 172.21.15.119:0/1800903738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f08105060 0x7f5f08107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:37.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.925+0000 7f5f0db26700 1 -- 172.21.15.119:0/1800903738 >> 172.21.15.119:0/1800903738 conn(0x7f5f080fc160 msgr2=0x7f5f080fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:43:37.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.925+0000 7f5f0db26700 1 -- 172.21.15.119:0/1800903738 shutdown_connections 2024-02-17T12:43:37.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.925+0000 7f5f0db26700 1 -- 172.21.15.119:0/1800903738 wait complete. 2024-02-17T12:43:37.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.926+0000 7f5f0db26700 1 Processor -- start 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.926+0000 7f5f0db26700 1 -- start start 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.926+0000 7f5f0db26700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f08102730 0x7f5f08197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.926+0000 7f5f0db26700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f08105060 0x7f5f081981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.926+0000 7f5f0db26700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f08107990 0x7f5f0819f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.926+0000 7f5f0db26700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0819f800 con 0x7f5f08107990 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.926+0000 7f5f0db26700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0819f940 con 0x7f5f08102730 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.926+0000 7f5f0db26700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0819fb00 con 0x7f5f08105060 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.926+0000 7f5f077fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f08102730 0x7f5f08197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.926+0000 7f5f077fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f08102730 0x7f5f08197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43986/0 (socket says 172.21.15.119:43986) 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.926+0000 7f5f077fe700 1 -- 172.21.15.119:0/747325249 learned_addr learned my addr 172.21.15.119:0/747325249 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f07fff700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f08107990 0x7f5f0819f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f06ffd700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f08105060 0x7f5f081981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f077fe700 1 -- 172.21.15.119:0/747325249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f08105060 msgr2=0x7f5f081981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:43:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f077fe700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f08105060 0x7f5f081981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:37.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f077fe700 1 -- 172.21.15.119:0/747325249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f08107990 msgr2=0x7f5f0819f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:43:37.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f077fe700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f08107990 0x7f5f0819f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:37.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f077fe700 1 -- 172.21.15.119:0/747325249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef80095f0 con 0x7f5f08102730 2024-02-17T12:43:37.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f06ffd700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f08105060 0x7f5f081981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:43:37.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f077fe700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f08102730 0x7f5f08197c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800b5c0 tx=0x7f5ef800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:43:37.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f04ff9700 1 -- 172.21.15.119:0/747325249 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef801d070 con 0x7f5f08102730 2024-02-17T12:43:37.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f0db26700 1 -- 172.21.15.119:0/747325249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0819fd80 con 0x7f5f08102730 2024-02-17T12:43:37.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f0db26700 1 -- 172.21.15.119:0/747325249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f081a01a0 con 0x7f5f08102730 2024-02-17T12:43:37.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f04ff9700 1 -- 172.21.15.119:0/747325249 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ef800fb80 con 0x7f5f08102730 2024-02-17T12:43:37.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.927+0000 7f5f04ff9700 1 -- 172.21.15.119:0/747325249 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8017650 con 0x7f5f08102730 2024-02-17T12:43:37.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.928+0000 7f5f0db26700 1 -- 172.21.15.119:0/747325249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef4005360 con 0x7f5f08102730 2024-02-17T12:43:37.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.929+0000 7f5f04ff9700 1 -- 172.21.15.119:0/747325249 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5ef800fcf0 con 0x7f5f08102730 2024-02-17T12:43:37.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.929+0000 7f5f04ff9700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5ef00a21d0 0x7f5ef00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:43:37.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.930+0000 7f5f04ff9700 1 -- 172.21.15.119:0/747325249 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f5ef80cde40 con 0x7f5f08102730 2024-02-17T12:43:37.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.930+0000 7f5f06ffd700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5ef00a21d0 0x7f5ef00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:43:37.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.930+0000 7f5f06ffd700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5ef00a21d0 0x7f5ef00a4690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5efc005950 tx=0x7f5efc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:43:37.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:37.934+0000 7f5f04ff9700 1 -- 172.21.15.119:0/747325249 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ef80980a0 con 0x7f5f08102730 2024-02-17T12:43:37.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:37 smithi146 ceph-mon[36454]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 525 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:37.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:37 smithi146 ceph-mon[36454]: Detected new or changed devices on smithi018 2024-02-17T12:43:37.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:37.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:37.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:43:37.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:43:37.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:37 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi018 to 8546M 2024-02-17T12:43:37.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:37.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:37.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:37.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:37.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:37 smithi146 ceph-mon[36454]: osdmap e27: 4 total, 4 up, 4 in 2024-02-17T12:43:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:38.137+0000 7f5f0db26700 1 -- 172.21.15.119:0/747325249 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ef4000bf0 con 0x7f5ef00a21d0 2024-02-17T12:43:38.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:38 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:43:38.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:38 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:43:38.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:38 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:38.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:38 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:43:38.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:38 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:43:38.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:38 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:38.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:38 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:43:38.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:38 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:43:38.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:38 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:39.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:39 smithi018 ceph-mon[32625]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:43:39.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:39 smithi018 ceph-mon[32625]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:39.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:39 smithi018 ceph-mon[32625]: osdmap e28: 4 total, 4 up, 4 in 2024-02-17T12:43:39.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:39 smithi018 ceph-mon[32625]: osdmap e29: 4 total, 4 up, 4 in 2024-02-17T12:43:39.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:39 smithi119 ceph-mon[36188]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:43:39.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:39 smithi119 ceph-mon[36188]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:39.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:39 smithi119 ceph-mon[36188]: osdmap e28: 4 total, 4 up, 4 in 2024-02-17T12:43:39.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:39 smithi119 ceph-mon[36188]: osdmap e29: 4 total, 4 up, 4 in 2024-02-17T12:43:39.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:39 smithi146 ceph-mon[36454]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:43:39.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:39 smithi146 ceph-mon[36454]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:39.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:39 smithi146 ceph-mon[36454]: osdmap e28: 4 total, 4 up, 4 in 2024-02-17T12:43:39.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:39 smithi146 ceph-mon[36454]: osdmap e29: 4 total, 4 up, 4 in 2024-02-17T12:43:41.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:41 smithi018 ceph-mon[32625]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-17T12:43:41.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:41 smithi018 ceph-mon[32625]: from='client.? 172.21.15.119:0/2746819911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1c67fe4-8ba9-4767-89a7-edab8c686496"}]: dispatch 2024-02-17T12:43:41.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:41 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1c67fe4-8ba9-4767-89a7-edab8c686496"}]: dispatch 2024-02-17T12:43:41.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:41 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1c67fe4-8ba9-4767-89a7-edab8c686496"}]': finished 2024-02-17T12:43:41.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:41 smithi018 ceph-mon[32625]: osdmap e30: 5 total, 4 up, 5 in 2024-02-17T12:43:41.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:41 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:43:41.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:41 smithi018 ceph-mon[32625]: from='client.? 172.21.15.119:0/2339553487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:43:41.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:41 smithi119 ceph-mon[36188]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-17T12:43:41.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:41 smithi119 ceph-mon[36188]: from='client.? 172.21.15.119:0/2746819911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1c67fe4-8ba9-4767-89a7-edab8c686496"}]: dispatch 2024-02-17T12:43:41.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:41 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1c67fe4-8ba9-4767-89a7-edab8c686496"}]: dispatch 2024-02-17T12:43:41.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:41 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1c67fe4-8ba9-4767-89a7-edab8c686496"}]': finished 2024-02-17T12:43:41.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:41 smithi119 ceph-mon[36188]: osdmap e30: 5 total, 4 up, 5 in 2024-02-17T12:43:41.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:41 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:43:41.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:41 smithi119 ceph-mon[36188]: from='client.? 172.21.15.119:0/2339553487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:43:41.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:41 smithi146 ceph-mon[36454]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-17T12:43:41.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:41 smithi146 ceph-mon[36454]: from='client.? 172.21.15.119:0/2746819911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1c67fe4-8ba9-4767-89a7-edab8c686496"}]: dispatch 2024-02-17T12:43:41.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:41 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1c67fe4-8ba9-4767-89a7-edab8c686496"}]: dispatch 2024-02-17T12:43:41.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:41 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1c67fe4-8ba9-4767-89a7-edab8c686496"}]': finished 2024-02-17T12:43:41.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:41 smithi146 ceph-mon[36454]: osdmap e30: 5 total, 4 up, 5 in 2024-02-17T12:43:41.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:41 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:43:41.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:41 smithi146 ceph-mon[36454]: from='client.? 172.21.15.119:0/2339553487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:43:43.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:43 smithi018 ceph-mon[32625]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-02-17T12:43:43.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:43 smithi119 ceph-mon[36188]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-02-17T12:43:43.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:43 smithi146 ceph-mon[36454]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-02-17T12:43:45.772 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:45 smithi119 ceph-mon[36188]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-17T12:43:45.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:45 smithi018 ceph-mon[32625]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-17T12:43:45.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:45 smithi146 ceph-mon[36454]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-17T12:43:47.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:47 smithi119 ceph-mon[36188]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-17T12:43:47.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:47 smithi018 ceph-mon[32625]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-17T12:43:47.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:47 smithi146 ceph-mon[36454]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-17T12:43:48.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-17T12:43:48.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:48.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:48 smithi018 ceph-mon[32625]: Deploying daemon osd.4 on smithi119 2024-02-17T12:43:48.833 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-17T12:43:48.834 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:48.834 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:48 smithi119 ceph-mon[36188]: Deploying daemon osd.4 on smithi119 2024-02-17T12:43:48.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-17T12:43:48.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:48.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:48 smithi146 ceph-mon[36454]: Deploying daemon osd.4 on smithi119 2024-02-17T12:43:49.768 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:49 smithi119 ceph-mon[36188]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:49.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:49 smithi146 ceph-mon[36454]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:50.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:49 smithi018 ceph-mon[32625]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:51.630 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:51 smithi119 ceph-mon[36188]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:51.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:51 smithi146 ceph-mon[36454]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:52.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:51 smithi018 ceph-mon[32625]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:53.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:53 smithi119 ceph-mon[36188]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:53.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:53 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:53.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:53 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:53.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:53 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:53.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:53 smithi146 ceph-mon[36454]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:53.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:53 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:53.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:53 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:53.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:53 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:54.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:53 smithi018 ceph-mon[32625]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:54.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:54.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:54.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:55.732 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:55 smithi119 ceph-mon[36188]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:55.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:55 smithi146 ceph-mon[36454]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:56.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:55 smithi018 ceph-mon[32625]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.704+0000 7f5f04ff9700 1 -- 172.21.15.119:0/747325249 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5ef4000bf0 con 0x7f5ef00a21d0 2024-02-17T12:43:56.699 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 4 on host 'smithi119' 2024-02-17T12:43:56.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.707+0000 7f5f0db26700 1 -- 172.21.15.119:0/747325249 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5ef00a21d0 msgr2=0x7f5ef00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:43:56.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.707+0000 7f5f0db26700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5ef00a21d0 0x7f5ef00a4690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5efc005950 tx=0x7f5efc0058e0 comp rx=0 tx=0).stop 2024-02-17T12:43:56.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.708+0000 7f5f0db26700 1 -- 172.21.15.119:0/747325249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f08102730 msgr2=0x7f5f08197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:43:56.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.708+0000 7f5f0db26700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f08102730 0x7f5f08197c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800b5c0 tx=0x7f5ef800f690 comp rx=0 tx=0).stop 2024-02-17T12:43:56.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.708+0000 7f5f0db26700 1 -- 172.21.15.119:0/747325249 shutdown_connections 2024-02-17T12:43:56.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.708+0000 7f5f0db26700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f08107990 0x7f5f0819f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:56.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.708+0000 7f5f0db26700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f08102730 0x7f5f08197c80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:56.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.708+0000 7f5f0db26700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5ef00a21d0 0x7f5ef00a4690 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:56.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.708+0000 7f5f0db26700 1 --2- 172.21.15.119:0/747325249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f08105060 0x7f5f081981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:43:56.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.708+0000 7f5f0db26700 1 -- 172.21.15.119:0/747325249 >> 172.21.15.119:0/747325249 conn(0x7f5f080fc160 msgr2=0x7f5f08103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:43:56.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.708+0000 7f5f0db26700 1 -- 172.21.15.119:0/747325249 shutdown_connections 2024-02-17T12:43:56.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:43:56.708+0000 7f5f0db26700 1 -- 172.21.15.119:0/747325249 wait complete. 2024-02-17T12:43:56.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:56 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:56.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:56 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:56.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:56 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:56.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:56 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:56.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:56 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:56.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:56 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:56.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:56 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:56.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:56 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:57.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:56 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:57.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:56 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:57.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:56 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:57.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:56 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:57.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:56 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:57.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:56 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:57.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:56 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:57.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:56 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:57.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:56 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:57.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:56 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:57.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:56 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:43:57.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:56 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:43:57.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:56 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:57.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:56 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:57.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:56 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:43:57.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:56 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:43:57.380 DEBUG:teuthology.orchestra.run.smithi119:osd.4> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.4.service 2024-02-17T12:43:57.382 INFO:tasks.cephadm:Deploying osd.5 on smithi146 with /dev/vg_nvme/lv_3... 2024-02-17T12:43:57.383 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-17T12:43:57.626 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:43:57.933 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:57 smithi119 ceph-mon[36188]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:57.934 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:57 smithi119 ceph-mon[36188]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-17T12:43:57.979 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:57 smithi146 ceph-mon[36454]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:57.979 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:57 smithi146 ceph-mon[36454]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-17T12:43:58.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:57 smithi018 ceph-mon[32625]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:58.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:57 smithi018 ceph-mon[32625]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-17T12:43:58.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:58 smithi119 ceph-mon[36188]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-17T12:43:58.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:58 smithi119 ceph-mon[36188]: osdmap e31: 5 total, 4 up, 5 in 2024-02-17T12:43:58.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:58 smithi119 ceph-mon[36188]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:43:58.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:58 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:43:59.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:58 smithi018 ceph-mon[32625]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-17T12:43:59.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:58 smithi018 ceph-mon[32625]: osdmap e31: 5 total, 4 up, 5 in 2024-02-17T12:43:59.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:58 smithi018 ceph-mon[32625]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:43:59.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:58 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:43:59.041 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:58 smithi146 ceph-mon[36454]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-17T12:43:59.041 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:58 smithi146 ceph-mon[36454]: osdmap e31: 5 total, 4 up, 5 in 2024-02-17T12:43:59.041 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:58 smithi146 ceph-mon[36454]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:43:59.041 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:58 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:43:59.612 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-17T12:43:59.612 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-17T12:43:59.612 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2024-02-17T12:43:59.612 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2024-02-17T12:43:59.612 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113053 s, 928 MB/s 2024-02-17T12:43:59.613 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2024-02-17T12:43:59.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:59 smithi119 ceph-mon[36188]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:43:59.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:59 smithi119 ceph-mon[36188]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-02-17T12:43:59.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:59 smithi119 ceph-mon[36188]: osdmap e32: 5 total, 4 up, 5 in 2024-02-17T12:43:59.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:59 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:43:59.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:59 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:43:59.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:43:59 smithi119 ceph-mon[36188]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' 2024-02-17T12:43:59.846 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 12:43:59 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4[47651]: 2024-02-17T12:43:59.564+0000 7fc90d9ed700 -1 osd.4 0 waiting for initial osdmap 2024-02-17T12:43:59.846 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 12:43:59 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4[47651]: 2024-02-17T12:43:59.576+0000 7fc905fd9700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T12:44:00.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:59 smithi018 ceph-mon[32625]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:44:00.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:59 smithi018 ceph-mon[32625]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-02-17T12:44:00.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:59 smithi018 ceph-mon[32625]: osdmap e32: 5 total, 4 up, 5 in 2024-02-17T12:44:00.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:59 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:44:00.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:59 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:44:00.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:43:59 smithi018 ceph-mon[32625]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' 2024-02-17T12:44:00.188 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:59 smithi146 ceph-mon[36454]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:44:00.188 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:59 smithi146 ceph-mon[36454]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-02-17T12:44:00.188 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:59 smithi146 ceph-mon[36454]: osdmap e32: 5 total, 4 up, 5 in 2024-02-17T12:44:00.188 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:59 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:44:00.188 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:59 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:44:00.188 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:43:59 smithi146 ceph-mon[36454]: from='osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818]' entity='osd.4' 2024-02-17T12:44:00.216 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi146:vg_nvme/lv_3 2024-02-17T12:44:00.461 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:44:00.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: purged_snaps scrub starts 2024-02-17T12:44:00.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: purged_snaps scrub ok 2024-02-17T12:44:00.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:44:00.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:00.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:00.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:00.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:00.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:00.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:00.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:00.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:00.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818] boot 2024-02-17T12:44:00.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: osdmap e33: 5 total, 5 up, 5 in 2024-02-17T12:44:00.817 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:00 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:44:01.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: purged_snaps scrub starts 2024-02-17T12:44:01.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: purged_snaps scrub ok 2024-02-17T12:44:01.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:44:01.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:01.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:01.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:01.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:01.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:01.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:01.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:01.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:01.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818] boot 2024-02-17T12:44:01.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: osdmap e33: 5 total, 5 up, 5 in 2024-02-17T12:44:01.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:00 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:44:01.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: purged_snaps scrub starts 2024-02-17T12:44:01.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: purged_snaps scrub ok 2024-02-17T12:44:01.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:44:01.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:01.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:01.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:01.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:01.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:01.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:01.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:01.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:01.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: osd.4 [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818] boot 2024-02-17T12:44:01.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: osdmap e33: 5 total, 5 up, 5 in 2024-02-17T12:44:01.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:00 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:44:01.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.946+0000 7f865d223700 1 -- 172.21.15.146:0/2381103497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8658102ea0 msgr2=0x7f8658103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:01.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.946+0000 7f865d223700 1 --2- 172.21.15.146:0/2381103497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8658102ea0 0x7f8658103300 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f864c009990 tx=0x7f864c009ca0 comp rx=0 tx=0).stop 2024-02-17T12:44:01.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.946+0000 7f865d223700 1 -- 172.21.15.146:0/2381103497 shutdown_connections 2024-02-17T12:44:01.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.946+0000 7f865d223700 1 --2- 172.21.15.146:0/2381103497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8658103840 0x7f865810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:01.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.946+0000 7f865d223700 1 --2- 172.21.15.146:0/2381103497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8658101ca0 0x7f86581020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:01.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.946+0000 7f865d223700 1 --2- 172.21.15.146:0/2381103497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8658102ea0 0x7f8658103300 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:01.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.946+0000 7f865d223700 1 -- 172.21.15.146:0/2381103497 >> 172.21.15.146:0/2381103497 conn(0x7f86580fd280 msgr2=0x7f86580ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:44:01.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.946+0000 7f865d223700 1 -- 172.21.15.146:0/2381103497 shutdown_connections 2024-02-17T12:44:01.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.946+0000 7f865d223700 1 -- 172.21.15.146:0/2381103497 wait complete. 2024-02-17T12:44:01.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.946+0000 7f865d223700 1 Processor -- start 2024-02-17T12:44:01.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865d223700 1 -- start start 2024-02-17T12:44:01.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865d223700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8658101ca0 0x7f8658197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:44:01.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865d223700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8658102ea0 0x7f8658198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:44:01.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865d223700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8658103840 0x7f865819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:44:01.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865d223700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865819f840 con 0x7f8658102ea0 2024-02-17T12:44:01.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865d223700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865819f980 con 0x7f8658101ca0 2024-02-17T12:44:01.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865d223700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865819fb40 con 0x7f8658103840 2024-02-17T12:44:01.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f8656d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8658101ca0 0x7f8658197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:44:01.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f8656d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8658101ca0 0x7f8658197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.146:36922/0 (socket says 172.21.15.146:36922) 2024-02-17T12:44:01.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f8656d9d700 1 -- 172.21.15.146:0/3201562363 learned_addr learned my addr 172.21.15.146:0/3201562363 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-02-17T12:44:01.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865759e700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8658103840 0x7f865819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865659c700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8658102ea0 0x7f8658198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865759e700 1 -- 172.21.15.146:0/3201562363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8658101ca0 msgr2=0x7f8658197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865759e700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8658101ca0 0x7f8658197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865759e700 1 -- 172.21.15.146:0/3201562363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8658102ea0 msgr2=0x7f8658198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865759e700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8658102ea0 0x7f8658198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f865759e700 1 -- 172.21.15.146:0/3201562363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c0095f0 con 0x7f8658103840 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.947+0000 7f8656d9d700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8658101ca0 0x7f8658197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.948+0000 7f865759e700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8658103840 0x7f865819f270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8650007f10 tx=0x7f865000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.948+0000 7f863ffff700 1 -- 172.21.15.146:0/3201562363 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650004db0 con 0x7f8658103840 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.948+0000 7f865d223700 1 -- 172.21.15.146:0/3201562363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865819fe20 con 0x7f8658103840 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.948+0000 7f863ffff700 1 -- 172.21.15.146:0/3201562363 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f865000f040 con 0x7f8658103840 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.948+0000 7f863ffff700 1 -- 172.21.15.146:0/3201562363 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650005a90 con 0x7f8658103840 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.948+0000 7f865d223700 1 -- 172.21.15.146:0/3201562363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86581a0260 con 0x7f8658103840 2024-02-17T12:44:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.949+0000 7f865d223700 1 -- 172.21.15.146:0/3201562363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8658068930 con 0x7f8658103840 2024-02-17T12:44:01.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.950+0000 7f863ffff700 1 -- 172.21.15.146:0/3201562363 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f86500050b0 con 0x7f8658103840 2024-02-17T12:44:01.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.950+0000 7f863ffff700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f86400a2150 0x7f86400a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:44:01.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.951+0000 7f863ffff700 1 -- 172.21.15.146:0/3201562363 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f86500cccf0 con 0x7f8658103840 2024-02-17T12:44:01.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.951+0000 7f8656d9d700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f86400a2150 0x7f86400a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:44:01.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.951+0000 7f8656d9d700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f86400a2150 0x7f86400a4610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f86581991d0 tx=0x7f8648009240 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:44:01.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:01.954+0000 7f863ffff700 1 -- 172.21.15.146:0/3201562363 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8650096d30 con 0x7f8658103840 2024-02-17T12:44:01.998 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:01 smithi119 ceph-mon[36188]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:44:01.998 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:01 smithi119 ceph-mon[36188]: Detected new or changed devices on smithi119 2024-02-17T12:44:01.998 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:01 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi119 to 8540M 2024-02-17T12:44:02.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:01 smithi018 ceph-mon[32625]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:44:02.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:01 smithi018 ceph-mon[32625]: Detected new or changed devices on smithi119 2024-02-17T12:44:02.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:01 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi119 to 8540M 2024-02-17T12:44:02.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:02.157+0000 7f865d223700 1 -- 172.21.15.146:0/3201562363 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f865802d8a0 con 0x7f86400a2150 2024-02-17T12:44:02.153 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:01 smithi146 ceph-mon[36454]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-17T12:44:02.153 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:01 smithi146 ceph-mon[36454]: Detected new or changed devices on smithi119 2024-02-17T12:44:02.153 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:01 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi119 to 8540M 2024-02-17T12:44:03.015 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:02 smithi119 ceph-mon[36188]: osdmap e34: 5 total, 5 up, 5 in 2024-02-17T12:44:03.015 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:44:03.016 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:44:03.016 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:03.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:02 smithi018 ceph-mon[32625]: osdmap e34: 5 total, 5 up, 5 in 2024-02-17T12:44:03.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:44:03.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:44:03.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:03.066 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:02 smithi146 ceph-mon[36454]: osdmap e34: 5 total, 5 up, 5 in 2024-02-17T12:44:03.066 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:44:03.066 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:44:03.066 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:03.991 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:03 smithi146 ceph-mon[36454]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:03.991 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:03 smithi146 ceph-mon[36454]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:44:04.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:03 smithi018 ceph-mon[32625]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:04.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:03 smithi018 ceph-mon[32625]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:44:04.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:03 smithi119 ceph-mon[36188]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:04.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:03 smithi119 ceph-mon[36188]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:44:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:05 smithi018 ceph-mon[32625]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:05 smithi018 ceph-mon[32625]: from='client.? 172.21.15.146:0/3131546622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33c58b7e-7d68-40e7-b8bb-b441343fb6de"}]: dispatch 2024-02-17T12:44:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:05 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33c58b7e-7d68-40e7-b8bb-b441343fb6de"}]: dispatch 2024-02-17T12:44:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:05 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33c58b7e-7d68-40e7-b8bb-b441343fb6de"}]': finished 2024-02-17T12:44:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:05 smithi018 ceph-mon[32625]: osdmap e35: 6 total, 5 up, 6 in 2024-02-17T12:44:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:06.028 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:05 smithi018 ceph-mon[32625]: from='client.? 172.21.15.146:0/1716576767' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:44:06.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:05 smithi119 ceph-mon[36188]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:06.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:05 smithi119 ceph-mon[36188]: from='client.? 172.21.15.146:0/3131546622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33c58b7e-7d68-40e7-b8bb-b441343fb6de"}]: dispatch 2024-02-17T12:44:06.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:05 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33c58b7e-7d68-40e7-b8bb-b441343fb6de"}]: dispatch 2024-02-17T12:44:06.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:05 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33c58b7e-7d68-40e7-b8bb-b441343fb6de"}]': finished 2024-02-17T12:44:06.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:05 smithi119 ceph-mon[36188]: osdmap e35: 6 total, 5 up, 6 in 2024-02-17T12:44:06.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:06.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:05 smithi119 ceph-mon[36188]: from='client.? 172.21.15.146:0/1716576767' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:44:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:05 smithi146 ceph-mon[36454]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:05 smithi146 ceph-mon[36454]: from='client.? 172.21.15.146:0/3131546622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33c58b7e-7d68-40e7-b8bb-b441343fb6de"}]: dispatch 2024-02-17T12:44:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:05 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33c58b7e-7d68-40e7-b8bb-b441343fb6de"}]: dispatch 2024-02-17T12:44:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:05 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33c58b7e-7d68-40e7-b8bb-b441343fb6de"}]': finished 2024-02-17T12:44:06.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:05 smithi146 ceph-mon[36454]: osdmap e35: 6 total, 5 up, 6 in 2024-02-17T12:44:06.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:06.204 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:05 smithi146 ceph-mon[36454]: from='client.? 172.21.15.146:0/1716576767' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:44:08.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:07 smithi018 ceph-mon[32625]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:08.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:07 smithi119 ceph-mon[36188]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:08.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:07 smithi146 ceph-mon[36454]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:10.018 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:09 smithi146 ceph-mon[36454]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:10.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:09 smithi018 ceph-mon[32625]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:10.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:09 smithi119 ceph-mon[36188]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:11.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:11 smithi146 ceph-mon[36454]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:12.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:11 smithi018 ceph-mon[32625]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:12.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:11 smithi119 ceph-mon[36188]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:13.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-17T12:44:13.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:12 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:13.090 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-17T12:44:13.090 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:12 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:13.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-17T12:44:13.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:12 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:14.010 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:13 smithi146 ceph-mon[36454]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:14.010 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:13 smithi146 ceph-mon[36454]: Deploying daemon osd.5 on smithi146 2024-02-17T12:44:14.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:13 smithi018 ceph-mon[32625]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:14.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:13 smithi018 ceph-mon[32625]: Deploying daemon osd.5 on smithi146 2024-02-17T12:44:14.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:13 smithi119 ceph-mon[36188]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:14.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:13 smithi119 ceph-mon[36188]: Deploying daemon osd.5 on smithi146 2024-02-17T12:44:16.020 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:15 smithi146 ceph-mon[36454]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:16.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:15 smithi018 ceph-mon[32625]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:16.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:15 smithi119 ceph-mon[36188]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:17.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:17 smithi146 ceph-mon[36454]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:17.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:17 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:17.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:17 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:17.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:17 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:44:18.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:17 smithi018 ceph-mon[32625]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:18.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:17 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:18.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:17 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:18.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:17 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:44:18.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:17 smithi119 ceph-mon[36188]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:18.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:17 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:18.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:17 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:18.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:17 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:44:20.019 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:19 smithi146 ceph-mon[36454]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:20.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:19 smithi018 ceph-mon[32625]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:20.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:19 smithi119 ceph-mon[36188]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:21.220 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 5 on host 'smithi146' 2024-02-17T12:44:21.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.225+0000 7f863ffff700 1 -- 172.21.15.146:0/3201562363 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f865802d8a0 con 0x7f86400a2150 2024-02-17T12:44:21.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.228+0000 7f865d223700 1 -- 172.21.15.146:0/3201562363 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f86400a2150 msgr2=0x7f86400a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:21.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.228+0000 7f865d223700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f86400a2150 0x7f86400a4610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f86581991d0 tx=0x7f8648009240 comp rx=0 tx=0).stop 2024-02-17T12:44:21.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.228+0000 7f865d223700 1 -- 172.21.15.146:0/3201562363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8658103840 msgr2=0x7f865819f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:21.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.228+0000 7f865d223700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8658103840 0x7f865819f270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8650007f10 tx=0x7f865000d3b0 comp rx=0 tx=0).stop 2024-02-17T12:44:21.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.228+0000 7f865d223700 1 -- 172.21.15.146:0/3201562363 shutdown_connections 2024-02-17T12:44:21.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.228+0000 7f865d223700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f86400a2150 0x7f86400a4610 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:21.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.228+0000 7f865d223700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8658103840 0x7f865819f270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:21.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.228+0000 7f865d223700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8658101ca0 0x7f8658197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:21.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.228+0000 7f865d223700 1 --2- 172.21.15.146:0/3201562363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8658102ea0 0x7f8658198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:21.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.228+0000 7f865d223700 1 -- 172.21.15.146:0/3201562363 >> 172.21.15.146:0/3201562363 conn(0x7f86580fd280 msgr2=0x7f8658106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:44:21.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.228+0000 7f865d223700 1 -- 172.21.15.146:0/3201562363 shutdown_connections 2024-02-17T12:44:21.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:44:21.228+0000 7f865d223700 1 -- 172.21.15.146:0/3201562363 wait complete. 2024-02-17T12:44:21.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:21 smithi146 ceph-mon[36454]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:21.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:21.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:21.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:44:21.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:21 smithi018 ceph-mon[32625]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:21.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:21.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:21.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:44:21.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:21 smithi119 ceph-mon[36188]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:21.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:21.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:21.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:21.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:44:21.862 DEBUG:teuthology.orchestra.run.smithi146:osd.5> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.5.service 2024-02-17T12:44:21.867 INFO:tasks.cephadm:Deploying osd.6 on smithi018 with /dev/vg_nvme/lv_2... 2024-02-17T12:44:21.867 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-17T12:44:22.124 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:44:22.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:22 smithi146 ceph-mon[36454]: from='osd.5 [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-17T12:44:22.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:22 smithi146 ceph-mon[36454]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-17T12:44:22.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:22 smithi018 ceph-mon[32625]: from='osd.5 [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-17T12:44:22.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:22 smithi018 ceph-mon[32625]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-17T12:44:22.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:22 smithi119 ceph-mon[36188]: from='osd.5 [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-17T12:44:22.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:22 smithi119 ceph-mon[36188]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-17T12:44:23.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:23 smithi018 ceph-mon[32625]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:23.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:23 smithi018 ceph-mon[32625]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-17T12:44:23.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:23 smithi018 ceph-mon[32625]: osdmap e36: 6 total, 5 up, 6 in 2024-02-17T12:44:23.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:23 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:23.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:23 smithi018 ceph-mon[32625]: from='osd.5 [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:44:23.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:23 smithi018 ceph-mon[32625]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:44:23.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:23 smithi119 ceph-mon[36188]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:23.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:23 smithi119 ceph-mon[36188]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-17T12:44:23.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:23 smithi119 ceph-mon[36188]: osdmap e36: 6 total, 5 up, 6 in 2024-02-17T12:44:23.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:23 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:23.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:23 smithi119 ceph-mon[36188]: from='osd.5 [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:44:23.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:23 smithi119 ceph-mon[36188]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:44:23.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:23 smithi146 ceph-mon[36454]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:23.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:23 smithi146 ceph-mon[36454]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-17T12:44:23.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:23 smithi146 ceph-mon[36454]: osdmap e36: 6 total, 5 up, 6 in 2024-02-17T12:44:23.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:23 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:23.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:23 smithi146 ceph-mon[36454]: from='osd.5 [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:44:23.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:23 smithi146 ceph-mon[36454]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:44:23.704 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 12:44:23 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T12:44:23.279+0000 7fad2cfcc700 -1 osd.5 0 waiting for initial osdmap 2024-02-17T12:44:23.704 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 12:44:23 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T12:44:23.286+0000 7fad295c0700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T12:44:24.225 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-17T12:44:24.225 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-17T12:44:24.225 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-02-17T12:44:24.225 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-02-17T12:44:24.225 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00818341 s, 1.3 GB/s 2024-02-17T12:44:24.225 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-02-17T12:44:24.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:24 smithi146 ceph-mon[36454]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-02-17T12:44:24.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:24 smithi146 ceph-mon[36454]: osdmap e37: 6 total, 5 up, 6 in 2024-02-17T12:44:24.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:24 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:24.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:24 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:24.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:24 smithi146 ceph-mon[36454]: osd.5 [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651] boot 2024-02-17T12:44:24.456 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:24 smithi146 ceph-mon[36454]: osdmap e38: 6 total, 6 up, 6 in 2024-02-17T12:44:24.457 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:24 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:24.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:24 smithi018 ceph-mon[32625]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-02-17T12:44:24.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:24 smithi018 ceph-mon[32625]: osdmap e37: 6 total, 5 up, 6 in 2024-02-17T12:44:24.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:24 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:24.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:24 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:24.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:24 smithi018 ceph-mon[32625]: osd.5 [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651] boot 2024-02-17T12:44:24.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:24 smithi018 ceph-mon[32625]: osdmap e38: 6 total, 6 up, 6 in 2024-02-17T12:44:24.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:24 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:24.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:24 smithi119 ceph-mon[36188]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-02-17T12:44:24.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:24 smithi119 ceph-mon[36188]: osdmap e37: 6 total, 5 up, 6 in 2024-02-17T12:44:24.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:24 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:24.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:24 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:24.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:24 smithi119 ceph-mon[36188]: osd.5 [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651] boot 2024-02-17T12:44:24.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:24 smithi119 ceph-mon[36188]: osdmap e38: 6 total, 6 up, 6 in 2024-02-17T12:44:24.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:24 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:44:24.848 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi018:vg_nvme/lv_2 2024-02-17T12:44:25.101 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:44:25.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: purged_snaps scrub starts 2024-02-17T12:44:25.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: purged_snaps scrub ok 2024-02-17T12:44:25.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:25.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: osdmap e39: 6 total, 6 up, 6 in 2024-02-17T12:44:25.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: Detected new or changed devices on smithi146 2024-02-17T12:44:25.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:25.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:25.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:25.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:25.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi146 to 8540M 2024-02-17T12:44:25.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:25.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:25.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:25.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:25.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: purged_snaps scrub starts 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: purged_snaps scrub ok 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: osdmap e39: 6 total, 6 up, 6 in 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: Detected new or changed devices on smithi146 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi146 to 8540M 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:25.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:25.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: purged_snaps scrub starts 2024-02-17T12:44:25.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: purged_snaps scrub ok 2024-02-17T12:44:25.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-02-17T12:44:25.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: osdmap e39: 6 total, 6 up, 6 in 2024-02-17T12:44:25.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: Detected new or changed devices on smithi146 2024-02-17T12:44:25.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:25.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:25.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:25.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:25.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi146 to 8540M 2024-02-17T12:44:25.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:25.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:25.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:25.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:26.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.717+0000 7f159e731700 1 -- 172.21.15.18:0/949270198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1598103840 msgr2=0x7f159810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:26.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.717+0000 7f159e731700 1 --2- 172.21.15.18:0/949270198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1598103840 0x7f159810a0f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f159000b0d0 tx=0x7f159000b3e0 comp rx=0 tx=0).stop 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.717+0000 7f159e731700 1 -- 172.21.15.18:0/949270198 shutdown_connections 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 --2- 172.21.15.18:0/949270198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1598103840 0x7f159810a0f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 --2- 172.21.15.18:0/949270198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598101ca0 0x7f15981020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 --2- 172.21.15.18:0/949270198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1598102ea0 0x7f1598103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 -- 172.21.15.18:0/949270198 >> 172.21.15.18:0/949270198 conn(0x7f15980fd280 msgr2=0x7f15980ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 -- 172.21.15.18:0/949270198 shutdown_connections 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 -- 172.21.15.18:0/949270198 wait complete. 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 Processor -- start 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 -- start start 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598101ca0 0x7f1598199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1598102ea0 0x7f159819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1598103840 0x7f15981a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:44:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15981a1a90 con 0x7f1598102ea0 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.718+0000 7f159e731700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15981a1bd0 con 0x7f1598103840 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f159e731700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15981a1d60 con 0x7f1598101ca0 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f15977fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1598102ea0 0x7f159819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f15977fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1598102ea0 0x7f159819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57966/0 (socket says 172.21.15.18:57966) 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f15977fe700 1 -- 172.21.15.18:0/2692686303 learned_addr learned my addr 172.21.15.18:0/2692686303 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f159ccce700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1598103840 0x7f15981a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f1597fff700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598101ca0 0x7f1598199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f15977fe700 1 -- 172.21.15.18:0/2692686303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598101ca0 msgr2=0x7f1598199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f15977fe700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598101ca0 0x7f1598199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f15977fe700 1 -- 172.21.15.18:0/2692686303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1598103840 msgr2=0x7f15981a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f15977fe700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1598103840 0x7f15981a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f15977fe700 1 -- 172.21.15.18:0/2692686303 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1590009d00 con 0x7f1598102ea0 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f1597fff700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598101ca0 0x7f1598199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:44:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f15977fe700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1598102ea0 0x7f159819a440 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f158c00d980 tx=0x7f158c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:44:26.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f15957fa700 1 -- 172.21.15.18:0/2692686303 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c009b40 con 0x7f1598102ea0 2024-02-17T12:44:26.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f15957fa700 1 -- 172.21.15.18:0/2692686303 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f158c00b6e0 con 0x7f1598102ea0 2024-02-17T12:44:26.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f159e731700 1 -- 172.21.15.18:0/2692686303 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15981a1fe0 con 0x7f1598102ea0 2024-02-17T12:44:26.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f15957fa700 1 -- 172.21.15.18:0/2692686303 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c00f620 con 0x7f1598102ea0 2024-02-17T12:44:26.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.719+0000 7f159e731700 1 -- 172.21.15.18:0/2692686303 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15981a24b0 con 0x7f1598102ea0 2024-02-17T12:44:26.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.720+0000 7f159e731700 1 -- 172.21.15.18:0/2692686303 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1598068930 con 0x7f1598102ea0 2024-02-17T12:44:26.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.722+0000 7f15957fa700 1 -- 172.21.15.18:0/2692686303 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f158c010460 con 0x7f1598102ea0 2024-02-17T12:44:26.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.722+0000 7f15957fa700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f15800a2150 0x7f15800a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:44:26.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.722+0000 7f1597fff700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f15800a2150 0x7f15800a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:44:26.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.722+0000 7f15957fa700 1 -- 172.21.15.18:0/2692686303 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f158c0ccbb0 con 0x7f1598102ea0 2024-02-17T12:44:26.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.723+0000 7f1597fff700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f15800a2150 0x7f15800a4610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1588005fd0 tx=0x7f1588005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:44:26.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.726+0000 7f15957fa700 1 -- 172.21.15.18:0/2692686303 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f158c096a40 con 0x7f1598102ea0 2024-02-17T12:44:26.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:26 smithi018 ceph-mon[32625]: osdmap e40: 6 total, 6 up, 6 in 2024-02-17T12:44:26.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:26 smithi119 ceph-mon[36188]: osdmap e40: 6 total, 6 up, 6 in 2024-02-17T12:44:26.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:26.939+0000 7f159e731700 1 -- 172.21.15.18:0/2692686303 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f15980755c0 con 0x7f15800a2150 2024-02-17T12:44:26.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:26 smithi146 ceph-mon[36454]: osdmap e40: 6 total, 6 up, 6 in 2024-02-17T12:44:27.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:27 smithi018 ceph-mon[32625]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:27.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:27 smithi018 ceph-mon[32625]: osdmap e41: 6 total, 6 up, 6 in 2024-02-17T12:44:27.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:27 smithi018 ceph-mon[32625]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:44:27.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:27 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:44:27.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:27 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:44:27.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:27 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:27.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:27 smithi119 ceph-mon[36188]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:27.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:27 smithi119 ceph-mon[36188]: osdmap e41: 6 total, 6 up, 6 in 2024-02-17T12:44:27.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:27 smithi119 ceph-mon[36188]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:44:27.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:27 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:44:27.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:27 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:44:27.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:27 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:27.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:27 smithi146 ceph-mon[36454]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:27.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:27 smithi146 ceph-mon[36454]: osdmap e41: 6 total, 6 up, 6 in 2024-02-17T12:44:27.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:27 smithi146 ceph-mon[36454]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:44:27.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:27 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:44:27.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:27 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:44:27.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:27 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:28.755 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:28 smithi018 ceph-mon[32625]: osdmap e42: 6 total, 6 up, 6 in 2024-02-17T12:44:28.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:28 smithi119 ceph-mon[36188]: osdmap e42: 6 total, 6 up, 6 in 2024-02-17T12:44:28.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:28 smithi146 ceph-mon[36454]: osdmap e42: 6 total, 6 up, 6 in 2024-02-17T12:44:29.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:29 smithi119 ceph-mon[36188]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:29 smithi018 ceph-mon[32625]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:29.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:29 smithi146 ceph-mon[36454]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:30 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2993274013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3ef630d-efa7-4b1f-98f5-6235d1081fc1"}]: dispatch 2024-02-17T12:44:30.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:30 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2993274013' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3ef630d-efa7-4b1f-98f5-6235d1081fc1"}]': finished 2024-02-17T12:44:30.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:30 smithi119 ceph-mon[36188]: osdmap e43: 7 total, 6 up, 7 in 2024-02-17T12:44:30.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:30.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:30 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2348623153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:44:30.854 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:30 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2993274013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3ef630d-efa7-4b1f-98f5-6235d1081fc1"}]: dispatch 2024-02-17T12:44:30.854 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:30 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2993274013' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3ef630d-efa7-4b1f-98f5-6235d1081fc1"}]': finished 2024-02-17T12:44:30.854 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:30 smithi146 ceph-mon[36454]: osdmap e43: 7 total, 6 up, 7 in 2024-02-17T12:44:30.854 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:30.854 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:30 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2348623153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:44:31.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:30 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2993274013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3ef630d-efa7-4b1f-98f5-6235d1081fc1"}]: dispatch 2024-02-17T12:44:31.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:30 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2993274013' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3ef630d-efa7-4b1f-98f5-6235d1081fc1"}]': finished 2024-02-17T12:44:31.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:30 smithi018 ceph-mon[32625]: osdmap e43: 7 total, 6 up, 7 in 2024-02-17T12:44:31.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:31.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:30 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2348623153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:44:31.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:31 smithi119 ceph-mon[36188]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-02-17T12:44:31.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:31 smithi146 ceph-mon[36454]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-02-17T12:44:32.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:31 smithi018 ceph-mon[32625]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-02-17T12:44:33.845 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:33 smithi018 ceph-mon[32625]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-17T12:44:33.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:33 smithi119 ceph-mon[36188]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-17T12:44:33.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:33 smithi146 ceph-mon[36454]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-17T12:44:35.704 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:35 smithi018 ceph-mon[32625]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-17T12:44:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:35 smithi119 ceph-mon[36188]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-17T12:44:35.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:35 smithi146 ceph-mon[36454]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-17T12:44:37.703 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:37 smithi018 ceph-mon[32625]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-17T12:44:37.704 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-17T12:44:37.704 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:37.704 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:37 smithi018 ceph-mon[32625]: Deploying daemon osd.6 on smithi018 2024-02-17T12:44:37.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:37 smithi146 ceph-mon[36454]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-17T12:44:37.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-17T12:44:37.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:37.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:37 smithi146 ceph-mon[36454]: Deploying daemon osd.6 on smithi018 2024-02-17T12:44:38.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:37 smithi119 ceph-mon[36188]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-17T12:44:38.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-17T12:44:38.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:38.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:37 smithi119 ceph-mon[36188]: Deploying daemon osd.6 on smithi018 2024-02-17T12:44:39.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:39 smithi146 ceph-mon[36454]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-17T12:44:39.971 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:39 smithi018 ceph-mon[32625]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-17T12:44:40.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:39 smithi119 ceph-mon[36188]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-17T12:44:41.956 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:41 smithi146 ceph-mon[36454]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:41.973 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:41 smithi018 ceph-mon[32625]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:42.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:41 smithi119 ceph-mon[36188]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:43.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:43 smithi119 ceph-mon[36188]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:43.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:43.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:43.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:44:43.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:43 smithi146 ceph-mon[36454]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:43.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:43 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:43.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:43 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:43.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:43 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:44:44.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:43 smithi018 ceph-mon[32625]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:44.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:43 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:44.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:43 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:44.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:43 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:44:45.744 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:45 smithi018 ceph-mon[32625]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:45.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:45 smithi119 ceph-mon[36188]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:45.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:45 smithi146 ceph-mon[36454]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:46.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.199+0000 7f15957fa700 1 -- 172.21.15.18:0/2692686303 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f15980755c0 con 0x7f15800a2150 2024-02-17T12:44:46.194 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 6 on host 'smithi018' 2024-02-17T12:44:46.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.201+0000 7f159e731700 1 -- 172.21.15.18:0/2692686303 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f15800a2150 msgr2=0x7f15800a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:46.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.201+0000 7f159e731700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f15800a2150 0x7f15800a4610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1588005fd0 tx=0x7f1588005f00 comp rx=0 tx=0).stop 2024-02-17T12:44:46.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.201+0000 7f159e731700 1 -- 172.21.15.18:0/2692686303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1598102ea0 msgr2=0x7f159819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:46.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.201+0000 7f159e731700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1598102ea0 0x7f159819a440 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f158c00d980 tx=0x7f158c00dc90 comp rx=0 tx=0).stop 2024-02-17T12:44:46.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.201+0000 7f159e731700 1 -- 172.21.15.18:0/2692686303 shutdown_connections 2024-02-17T12:44:46.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.201+0000 7f159e731700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1598103840 0x7f15981a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:46.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.201+0000 7f159e731700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f15800a2150 0x7f15800a4610 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:46.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.202+0000 7f159e731700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1598101ca0 0x7f1598199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:46.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.202+0000 7f159e731700 1 --2- 172.21.15.18:0/2692686303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1598102ea0 0x7f159819a440 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:46.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.202+0000 7f159e731700 1 -- 172.21.15.18:0/2692686303 >> 172.21.15.18:0/2692686303 conn(0x7f15980fd280 msgr2=0x7f1598073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:44:46.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.202+0000 7f159e731700 1 -- 172.21.15.18:0/2692686303 shutdown_connections 2024-02-17T12:44:46.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:44:46.202+0000 7f159e731700 1 -- 172.21.15.18:0/2692686303 wait complete. 2024-02-17T12:44:46.979 DEBUG:teuthology.orchestra.run.smithi018:osd.6> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.6.service 2024-02-17T12:44:46.982 INFO:tasks.cephadm:Deploying osd.7 on smithi119 with /dev/vg_nvme/lv_2... 2024-02-17T12:44:46.982 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-17T12:44:47.229 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T12:44:47.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:47 smithi018 ceph-mon[32625]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:47.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:47 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:47.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:47 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:47.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:47 smithi018 ceph-mon[32625]: from='osd.6 [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-17T12:44:47.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:47 smithi018 ceph-mon[32625]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-17T12:44:47.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:47 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:47.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:47 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:47.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:47 smithi119 ceph-mon[36188]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:47.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:47 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:47.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:47 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:47.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:47 smithi119 ceph-mon[36188]: from='osd.6 [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-17T12:44:47.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:47 smithi119 ceph-mon[36188]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-17T12:44:47.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:47 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:47.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:47 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:47.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:47 smithi146 ceph-mon[36454]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:47.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:47 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:47.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:47 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:47.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:47 smithi146 ceph-mon[36454]: from='osd.6 [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-17T12:44:47.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:47 smithi146 ceph-mon[36454]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-17T12:44:47.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:47 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:47.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:47 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:48.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:48 smithi018 ceph-mon[32625]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-17T12:44:48.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:48 smithi018 ceph-mon[32625]: osdmap e44: 7 total, 6 up, 7 in 2024-02-17T12:44:48.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:48 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:48.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:48 smithi018 ceph-mon[32625]: from='osd.6 [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:44:48.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:48 smithi018 ceph-mon[32625]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:44:48.202 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 12:44:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T12:44:48.197+0000 7f202ef73700 -1 osd.6 0 waiting for initial osdmap 2024-02-17T12:44:48.202 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 12:44:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T12:44:48.207+0000 7f202b567700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T12:44:48.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:48 smithi119 ceph-mon[36188]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-17T12:44:48.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:48 smithi119 ceph-mon[36188]: osdmap e44: 7 total, 6 up, 7 in 2024-02-17T12:44:48.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:48 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:48.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:48 smithi119 ceph-mon[36188]: from='osd.6 [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:44:48.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:48 smithi119 ceph-mon[36188]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:44:48.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:48 smithi146 ceph-mon[36454]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-17T12:44:48.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:48 smithi146 ceph-mon[36454]: osdmap e44: 7 total, 6 up, 7 in 2024-02-17T12:44:48.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:48 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:48.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:48 smithi146 ceph-mon[36454]: from='osd.6 [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:44:48.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:48 smithi146 ceph-mon[36454]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:44:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-17T12:44:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-17T12:44:49.226 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-02-17T12:44:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-02-17T12:44:49.227 INFO:teuthology.orchestra.run.smithi119.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112145 s, 935 MB/s 2024-02-17T12:44:49.227 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-02-17T12:44:49.381 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:49 smithi119 ceph-mon[36188]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:49.381 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:49 smithi119 ceph-mon[36188]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-02-17T12:44:49.382 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:49 smithi119 ceph-mon[36188]: osdmap e45: 7 total, 6 up, 7 in 2024-02-17T12:44:49.382 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:49 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:49.382 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:49 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:49.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:49 smithi146 ceph-mon[36454]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:49.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:49 smithi146 ceph-mon[36454]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-02-17T12:44:49.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:49 smithi146 ceph-mon[36454]: osdmap e45: 7 total, 6 up, 7 in 2024-02-17T12:44:49.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:49 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:49.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:49 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:49.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:49 smithi018 ceph-mon[32625]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:49.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:49 smithi018 ceph-mon[32625]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-02-17T12:44:49.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:49 smithi018 ceph-mon[32625]: osdmap e45: 7 total, 6 up, 7 in 2024-02-17T12:44:49.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:49 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:49.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:49 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:49.878 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_2 2024-02-17T12:44:50.123 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T12:44:50.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:50 smithi018 ceph-mon[32625]: purged_snaps scrub starts 2024-02-17T12:44:50.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:50 smithi018 ceph-mon[32625]: purged_snaps scrub ok 2024-02-17T12:44:50.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:50 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:50.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:50 smithi018 ceph-mon[32625]: osd.6 [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734] boot 2024-02-17T12:44:50.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:50 smithi018 ceph-mon[32625]: osdmap e46: 7 total, 7 up, 7 in 2024-02-17T12:44:50.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:50 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:50.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:50 smithi146 ceph-mon[36454]: purged_snaps scrub starts 2024-02-17T12:44:50.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:50 smithi146 ceph-mon[36454]: purged_snaps scrub ok 2024-02-17T12:44:50.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:50 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:50.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:50 smithi146 ceph-mon[36454]: osd.6 [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734] boot 2024-02-17T12:44:50.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:50 smithi146 ceph-mon[36454]: osdmap e46: 7 total, 7 up, 7 in 2024-02-17T12:44:50.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:50 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:50.463 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:50 smithi119 ceph-mon[36188]: purged_snaps scrub starts 2024-02-17T12:44:50.463 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:50 smithi119 ceph-mon[36188]: purged_snaps scrub ok 2024-02-17T12:44:50.463 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:50 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:50.463 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:50 smithi119 ceph-mon[36188]: osd.6 [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734] boot 2024-02-17T12:44:50.463 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:50 smithi119 ceph-mon[36188]: osdmap e46: 7 total, 7 up, 7 in 2024-02-17T12:44:50.463 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:50 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:44:51.452 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:51.452 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: osdmap e47: 7 total, 7 up, 7 in 2024-02-17T12:44:51.452 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: Detected new or changed devices on smithi018 2024-02-17T12:44:51.452 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi018 to 5697M 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:51.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:51.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: osdmap e47: 7 total, 7 up, 7 in 2024-02-17T12:44:51.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: Detected new or changed devices on smithi018 2024-02-17T12:44:51.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:51.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:51.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:51.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi018 to 5697M 2024-02-17T12:44:51.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:44:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:51.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-17T12:44:51.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: osdmap e47: 7 total, 7 up, 7 in 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: Detected new or changed devices on smithi018 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi018 to 5697M 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:44:51.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:44:51.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.671+0000 7efe6dd91700 1 -- 172.21.15.119:0/242146655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe68102f90 msgr2=0x7efe681033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:51.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.671+0000 7efe6dd91700 1 --2- 172.21.15.119:0/242146655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe68102f90 0x7efe681033f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009990 tx=0x7efe5c009ca0 comp rx=0 tx=0).stop 2024-02-17T12:44:51.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.671+0000 7efe6dd91700 1 -- 172.21.15.119:0/242146655 shutdown_connections 2024-02-17T12:44:51.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.671+0000 7efe6dd91700 1 --2- 172.21.15.119:0/242146655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe68103930 0x7efe6810a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:51.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.671+0000 7efe6dd91700 1 --2- 172.21.15.119:0/242146655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe680fffb0 0x7efe681003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:51.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.671+0000 7efe6dd91700 1 --2- 172.21.15.119:0/242146655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe68102f90 0x7efe681033f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:51.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.671+0000 7efe6dd91700 1 -- 172.21.15.119:0/242146655 >> 172.21.15.119:0/242146655 conn(0x7efe680fb910 msgr2=0x7efe680fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:44:51.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.671+0000 7efe6dd91700 1 -- 172.21.15.119:0/242146655 shutdown_connections 2024-02-17T12:44:51.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.671+0000 7efe6dd91700 1 -- 172.21.15.119:0/242146655 wait complete. 2024-02-17T12:44:51.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.671+0000 7efe6dd91700 1 Processor -- start 2024-02-17T12:44:51.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe6dd91700 1 -- start start 2024-02-17T12:44:51.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe6dd91700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe680fffb0 0x7efe68197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:44:51.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe6dd91700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe68102f90 0x7efe681981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:44:51.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe6dd91700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe68103930 0x7efe6819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:44:51.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe6dd91700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6819f820 con 0x7efe680fffb0 2024-02-17T12:44:51.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe6dd91700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6819f960 con 0x7efe68103930 2024-02-17T12:44:51.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe6dd91700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6819fb20 con 0x7efe68102f90 2024-02-17T12:44:51.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe67fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe68103930 0x7efe6819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:44:51.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe67fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe68103930 0x7efe6819f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35950/0 (socket says 172.21.15.119:35950) 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe67fff700 1 -- 172.21.15.119:0/4253064090 learned_addr learned my addr 172.21.15.119:0/4253064090 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe677fe700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe680fffb0 0x7efe68197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe66ffd700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe68102f90 0x7efe681981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe67fff700 1 -- 172.21.15.119:0/4253064090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe68102f90 msgr2=0x7efe681981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe67fff700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe68102f90 0x7efe681981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe67fff700 1 -- 172.21.15.119:0/4253064090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe680fffb0 msgr2=0x7efe68197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe67fff700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe680fffb0 0x7efe68197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe67fff700 1 -- 172.21.15.119:0/4253064090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c0095f0 con 0x7efe68103930 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe66ffd700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe68102f90 0x7efe681981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe677fe700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe680fffb0 0x7efe68197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.672+0000 7efe67fff700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe68103930 0x7efe6819f250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe6000b970 tx=0x7efe6000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.673+0000 7efe64ff9700 1 -- 172.21.15.119:0/4253064090 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6000c930 con 0x7efe68103930 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.673+0000 7efe64ff9700 1 -- 172.21.15.119:0/4253064090 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe60014440 con 0x7efe68103930 2024-02-17T12:44:51.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.673+0000 7efe6dd91700 1 -- 172.21.15.119:0/4253064090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6819fe00 con 0x7efe68103930 2024-02-17T12:44:51.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.673+0000 7efe64ff9700 1 -- 172.21.15.119:0/4253064090 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe600075d0 con 0x7efe68103930 2024-02-17T12:44:51.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.673+0000 7efe6dd91700 1 -- 172.21.15.119:0/4253064090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe681a0270 con 0x7efe68103930 2024-02-17T12:44:51.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.674+0000 7efe6dd91700 1 -- 172.21.15.119:0/4253064090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe68109150 con 0x7efe68103930 2024-02-17T12:44:51.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.675+0000 7efe64ff9700 1 -- 172.21.15.119:0/4253064090 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7efe600145b0 con 0x7efe68103930 2024-02-17T12:44:51.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.675+0000 7efe64ff9700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7efe500a2150 0x7efe500a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:44:51.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.675+0000 7efe64ff9700 1 -- 172.21.15.119:0/4253064090 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7efe60006e40 con 0x7efe68103930 2024-02-17T12:44:51.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.676+0000 7efe677fe700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7efe500a2150 0x7efe500a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:44:51.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.676+0000 7efe677fe700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7efe500a2150 0x7efe500a4610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe68198b80 tx=0x7efe5800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:44:51.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.679+0000 7efe64ff9700 1 -- 172.21.15.119:0/4253064090 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe60096490 con 0x7efe68103930 2024-02-17T12:44:51.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:44:51.884+0000 7efe6dd91700 1 -- 172.21.15.119:0/4253064090 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7efe6802da40 con 0x7efe500a2150 2024-02-17T12:44:52.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:52 smithi119 ceph-mon[36188]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:44:52.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:44:52.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:44:52.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:52.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:52 smithi018 ceph-mon[32625]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:44:52.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:44:52.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:44:52.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:52.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:52 smithi146 ceph-mon[36454]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:44:52.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:44:52.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:44:52.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:44:53.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:53 smithi119 ceph-mon[36188]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:44:53.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:53 smithi018 ceph-mon[32625]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:44:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:53 smithi146 ceph-mon[36454]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:44:55.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:55 smithi018 ceph-mon[32625]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:44:55.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:55 smithi018 ceph-mon[32625]: from='client.? 172.21.15.119:0/2157913842' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a"}]: dispatch 2024-02-17T12:44:55.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:55 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a"}]: dispatch 2024-02-17T12:44:55.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:55 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a"}]': finished 2024-02-17T12:44:55.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:55 smithi018 ceph-mon[32625]: osdmap e48: 8 total, 7 up, 8 in 2024-02-17T12:44:55.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:55 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:44:55.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:55 smithi018 ceph-mon[32625]: from='client.? 172.21.15.119:0/3136815666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:44:55.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:55 smithi119 ceph-mon[36188]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:44:55.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:55 smithi119 ceph-mon[36188]: from='client.? 172.21.15.119:0/2157913842' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a"}]: dispatch 2024-02-17T12:44:55.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:55 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a"}]: dispatch 2024-02-17T12:44:55.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:55 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a"}]': finished 2024-02-17T12:44:55.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:55 smithi119 ceph-mon[36188]: osdmap e48: 8 total, 7 up, 8 in 2024-02-17T12:44:55.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:55 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:44:55.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:55 smithi119 ceph-mon[36188]: from='client.? 172.21.15.119:0/3136815666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:44:55.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:55 smithi146 ceph-mon[36454]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:44:55.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:55 smithi146 ceph-mon[36454]: from='client.? 172.21.15.119:0/2157913842' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a"}]: dispatch 2024-02-17T12:44:55.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:55 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a"}]: dispatch 2024-02-17T12:44:55.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:55 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a"}]': finished 2024-02-17T12:44:55.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:55 smithi146 ceph-mon[36454]: osdmap e48: 8 total, 7 up, 8 in 2024-02-17T12:44:55.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:55 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:44:55.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:55 smithi146 ceph-mon[36454]: from='client.? 172.21.15.119:0/3136815666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:44:57.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:57 smithi018 ceph-mon[32625]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:44:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:57 smithi119 ceph-mon[36188]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:44:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:57 smithi146 ceph-mon[36454]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:44:59.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:44:59 smithi119 ceph-mon[36188]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:44:59.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:44:59 smithi018 ceph-mon[32625]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:44:59.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:44:59 smithi146 ceph-mon[36454]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:01.505 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:01 smithi146 ceph-mon[36454]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:01.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:01 smithi018 ceph-mon[32625]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:01.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:01 smithi119 ceph-mon[36188]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:02.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-17T12:45:02.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:02 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:02.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:02 smithi018 ceph-mon[32625]: Deploying daemon osd.7 on smithi119 2024-02-17T12:45:02.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-17T12:45:02.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:02 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:02.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:02 smithi119 ceph-mon[36188]: Deploying daemon osd.7 on smithi119 2024-02-17T12:45:02.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-17T12:45:02.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:02 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:02.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:02 smithi146 ceph-mon[36454]: Deploying daemon osd.7 on smithi119 2024-02-17T12:45:03.393 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:03 smithi119 ceph-mon[36188]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:03.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:03 smithi018 ceph-mon[32625]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:03.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:03 smithi146 ceph-mon[36454]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:05.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:05 smithi119 ceph-mon[36188]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:05.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:05 smithi018 ceph-mon[32625]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:05.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:05 smithi146 ceph-mon[36454]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:07.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:07 smithi018 ceph-mon[32625]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:07.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:07 smithi119 ceph-mon[36188]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:07.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:07 smithi146 ceph-mon[36454]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:08.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:08 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:08.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:08 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:08.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:08 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:08.576 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:08 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:08.576 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:08 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:08.576 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:08 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:08.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:08 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:08.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:08 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:08.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:08 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:09.445 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:09 smithi119 ceph-mon[36188]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:09.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:09 smithi018 ceph-mon[32625]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:09.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:09 smithi146 ceph-mon[36454]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:10.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.905+0000 7efe64ff9700 1 -- 172.21.15.119:0/4253064090 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efe6802da40 con 0x7efe500a2150 2024-02-17T12:45:10.900 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 7 on host 'smithi119' 2024-02-17T12:45:10.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.907+0000 7efe6dd91700 1 -- 172.21.15.119:0/4253064090 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7efe500a2150 msgr2=0x7efe500a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:45:10.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.907+0000 7efe6dd91700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7efe500a2150 0x7efe500a4610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe68198b80 tx=0x7efe5800b410 comp rx=0 tx=0).stop 2024-02-17T12:45:10.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.907+0000 7efe6dd91700 1 -- 172.21.15.119:0/4253064090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe68103930 msgr2=0x7efe6819f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:45:10.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.907+0000 7efe6dd91700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe68103930 0x7efe6819f250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe6000b970 tx=0x7efe6000bc80 comp rx=0 tx=0).stop 2024-02-17T12:45:10.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.907+0000 7efe6dd91700 1 -- 172.21.15.119:0/4253064090 shutdown_connections 2024-02-17T12:45:10.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.907+0000 7efe6dd91700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe68103930 0x7efe6819f250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:10.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.907+0000 7efe6dd91700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe680fffb0 0x7efe68197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:10.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.907+0000 7efe6dd91700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7efe500a2150 0x7efe500a4610 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.907+0000 7efe6dd91700 1 --2- 172.21.15.119:0/4253064090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe68102f90 0x7efe681981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.907+0000 7efe6dd91700 1 -- 172.21.15.119:0/4253064090 >> 172.21.15.119:0/4253064090 conn(0x7efe680fb910 msgr2=0x7efe68106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:45:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.907+0000 7efe6dd91700 1 -- 172.21.15.119:0/4253064090 shutdown_connections 2024-02-17T12:45:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:45:10.907+0000 7efe6dd91700 1 -- 172.21.15.119:0/4253064090 wait complete. 2024-02-17T12:45:11.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:11.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:45:11.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:11.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:11.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:45:11.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:11.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:11.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:45:11.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:11.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:11.582 DEBUG:teuthology.orchestra.run.smithi119:osd.7> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.7.service 2024-02-17T12:45:11.584 INFO:tasks.cephadm:Deploying osd.8 on smithi146 with /dev/vg_nvme/lv_2... 2024-02-17T12:45:11.584 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-17T12:45:11.832 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:45:12.325 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:12 smithi146 ceph-mon[36454]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:12.325 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:12 smithi146 ceph-mon[36454]: from='osd.7 [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-17T12:45:12.325 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:12 smithi146 ceph-mon[36454]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-17T12:45:12.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:12 smithi119 ceph-mon[36188]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:12.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:12 smithi119 ceph-mon[36188]: from='osd.7 [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-17T12:45:12.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:12 smithi119 ceph-mon[36188]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-17T12:45:12.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:12 smithi018 ceph-mon[32625]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:12.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:12 smithi018 ceph-mon[32625]: from='osd.7 [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-17T12:45:12.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:12 smithi018 ceph-mon[32625]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-17T12:45:13.324 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:13 smithi146 ceph-mon[36454]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-17T12:45:13.324 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:13 smithi146 ceph-mon[36454]: osdmap e49: 8 total, 7 up, 8 in 2024-02-17T12:45:13.325 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:13 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:13.325 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:13 smithi146 ceph-mon[36454]: from='osd.7 [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:45:13.325 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:13 smithi146 ceph-mon[36454]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:45:13.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:13 smithi119 ceph-mon[36188]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-17T12:45:13.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:13 smithi119 ceph-mon[36188]: osdmap e49: 8 total, 7 up, 8 in 2024-02-17T12:45:13.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:13 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:13.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:13 smithi119 ceph-mon[36188]: from='osd.7 [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:45:13.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:13 smithi119 ceph-mon[36188]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:45:13.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:13 smithi018 ceph-mon[32625]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-17T12:45:13.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:13 smithi018 ceph-mon[32625]: osdmap e49: 8 total, 7 up, 8 in 2024-02-17T12:45:13.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:13.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:13 smithi018 ceph-mon[32625]: from='osd.7 [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:45:13.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:13 smithi018 ceph-mon[32625]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:45:13.878 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-17T12:45:13.878 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-17T12:45:13.878 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2024-02-17T12:45:13.878 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2024-02-17T12:45:13.878 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117052 s, 896 MB/s 2024-02-17T12:45:13.878 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2024-02-17T12:45:14.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:14 smithi119 ceph-mon[36188]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:14.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:14 smithi119 ceph-mon[36188]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-02-17T12:45:14.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:14 smithi119 ceph-mon[36188]: osdmap e50: 8 total, 7 up, 8 in 2024-02-17T12:45:14.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:14 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:14.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:14 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:14.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:14 smithi119 ceph-mon[36188]: osdmap e51: 8 total, 7 up, 8 in 2024-02-17T12:45:14.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:14 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:14.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:14 smithi119 ceph-mon[36188]: from='osd.7 ' entity='osd.7' 2024-02-17T12:45:14.311 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 12:45:13 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T12:45:13.978+0000 7f2e61033700 -1 osd.7 0 waiting for initial osdmap 2024-02-17T12:45:14.311 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 12:45:13 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T12:45:13.994+0000 7f2e57e1c700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T12:45:14.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:14 smithi146 ceph-mon[36454]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:14.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:14 smithi146 ceph-mon[36454]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-02-17T12:45:14.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:14 smithi146 ceph-mon[36454]: osdmap e50: 8 total, 7 up, 8 in 2024-02-17T12:45:14.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:14 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:14.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:14 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:14.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:14 smithi146 ceph-mon[36454]: osdmap e51: 8 total, 7 up, 8 in 2024-02-17T12:45:14.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:14 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:14.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:14 smithi146 ceph-mon[36454]: from='osd.7 ' entity='osd.7' 2024-02-17T12:45:14.486 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi146:vg_nvme/lv_2 2024-02-17T12:45:14.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:14 smithi018 ceph-mon[32625]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:14.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:14 smithi018 ceph-mon[32625]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-02-17T12:45:14.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:14 smithi018 ceph-mon[32625]: osdmap e50: 8 total, 7 up, 8 in 2024-02-17T12:45:14.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:14 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:14.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:14 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:14.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:14 smithi018 ceph-mon[32625]: osdmap e51: 8 total, 7 up, 8 in 2024-02-17T12:45:14.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:14 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:14.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:14 smithi018 ceph-mon[32625]: from='osd.7 ' entity='osd.7' 2024-02-17T12:45:14.730 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:45:15.110 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: purged_snaps scrub starts 2024-02-17T12:45:15.405 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: purged_snaps scrub starts 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: purged_snaps scrub ok 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: osd.7 [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737] boot 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: osdmap e52: 8 total, 8 up, 8 in 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:15.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:45:15.407 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:15.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: purged_snaps scrub ok 2024-02-17T12:45:15.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:15.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: osd.7 [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737] boot 2024-02-17T12:45:15.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: osdmap e52: 8 total, 8 up, 8 in 2024-02-17T12:45:15.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:15.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:15.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:15.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:15.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:15.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:15.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:15.454 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:15.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:45:15.455 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:15.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: purged_snaps scrub starts 2024-02-17T12:45:15.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: purged_snaps scrub ok 2024-02-17T12:45:15.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:15.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: osd.7 [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737] boot 2024-02-17T12:45:15.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: osdmap e52: 8 total, 8 up, 8 in 2024-02-17T12:45:15.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:45:15.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:15.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:15.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:15.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:15.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:15.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:15.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:15.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:45:15.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:16.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.337+0000 7f2808872700 1 -- 172.21.15.146:0/3747003910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2804102730 msgr2=0x7f2804108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:45:16.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.337+0000 7f2808872700 1 --2- 172.21.15.146:0/3747003910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2804102730 0x7f2804108fe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00b0a0 tx=0x7f27fc00b3b0 comp rx=0 tx=0).stop 2024-02-17T12:45:16.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.338+0000 7f2808872700 1 -- 172.21.15.146:0/3747003910 shutdown_connections 2024-02-17T12:45:16.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.338+0000 7f2808872700 1 --2- 172.21.15.146:0/3747003910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2804102730 0x7f2804108fe0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.338+0000 7f2808872700 1 --2- 172.21.15.146:0/3747003910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2804100b90 0x7f2804100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.338+0000 7f2808872700 1 --2- 172.21.15.146:0/3747003910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2804101d90 0x7f28041021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.338+0000 7f2808872700 1 -- 172.21.15.146:0/3747003910 >> 172.21.15.146:0/3747003910 conn(0x7f28040fc130 msgr2=0x7f28040fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.338+0000 7f2808872700 1 -- 172.21.15.146:0/3747003910 shutdown_connections 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.338+0000 7f2808872700 1 -- 172.21.15.146:0/3747003910 wait complete. 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.338+0000 7f2808872700 1 Processor -- start 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.338+0000 7f2808872700 1 -- start start 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2808872700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2804100b90 0x7f2804197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2808872700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2804101d90 0x7f2804198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2808872700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2804102730 0x7f280419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2808872700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280419f8e0 con 0x7f2804102730 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2808872700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280419fa20 con 0x7f2804100b90 2024-02-17T12:45:16.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2808872700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280419fbb0 con 0x7f2804101d90 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2801d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2804101d90 0x7f2804198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2801d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2804101d90 0x7f2804198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50256/0 (socket says 172.21.15.146:50256) 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2801d9b700 1 -- 172.21.15.146:0/3777413142 learned_addr learned my addr 172.21.15.146:0/3777413142 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f280259c700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2804100b90 0x7f2804197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2801d9b700 1 -- 172.21.15.146:0/3777413142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2804100b90 msgr2=0x7f2804197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2801d9b700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2804100b90 0x7f2804197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2801d9b700 1 -- 172.21.15.146:0/3777413142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2804102730 msgr2=0x7f280419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2801d9b700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2804102730 0x7f280419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2801d9b700 1 -- 172.21.15.146:0/3777413142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc009d00 con 0x7f2804101d90 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f280259c700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2804100b90 0x7f2804197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2801d9b700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2804101d90 0x7f2804198290 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f27f800d980 tx=0x7f27f800dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f27f37fe700 1 -- 172.21.15.146:0/3777413142 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8009b40 con 0x7f2804101d90 2024-02-17T12:45:16.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f27f37fe700 1 -- 172.21.15.146:0/3777413142 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27f800b6e0 con 0x7f2804101d90 2024-02-17T12:45:16.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2808872700 1 -- 172.21.15.146:0/3777413142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280419fe90 con 0x7f2804101d90 2024-02-17T12:45:16.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f27f37fe700 1 -- 172.21.15.146:0/3777413142 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800f620 con 0x7f2804101d90 2024-02-17T12:45:16.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.339+0000 7f2808872700 1 -- 172.21.15.146:0/3777413142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28041a0380 con 0x7f2804101d90 2024-02-17T12:45:16.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.340+0000 7f2808872700 1 -- 172.21.15.146:0/3777413142 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2804107e80 con 0x7f2804101d90 2024-02-17T12:45:16.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.341+0000 7f27f37fe700 1 -- 172.21.15.146:0/3777413142 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f27f8010460 con 0x7f2804101d90 2024-02-17T12:45:16.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.342+0000 7f27f37fe700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f27ec0a2150 0x7f27ec0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:45:16.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.342+0000 7f27f37fe700 1 -- 172.21.15.146:0/3777413142 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f27f80cd030 con 0x7f2804101d90 2024-02-17T12:45:16.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.342+0000 7f280259c700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f27ec0a2150 0x7f27ec0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:45:16.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.343+0000 7f280259c700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f27ec0a2150 0x7f27ec0a4610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2804101bf0 tx=0x7f27f4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:45:16.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.346+0000 7f27f37fe700 1 -- 172.21.15.146:0/3777413142 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27f8096b40 con 0x7f2804101d90 2024-02-17T12:45:16.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:16 smithi146 ceph-mon[36454]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:16.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:16 smithi146 ceph-mon[36454]: Detected new or changed devices on smithi119 2024-02-17T12:45:16.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:16 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi119 to 5693M 2024-02-17T12:45:16.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:16 smithi146 ceph-mon[36454]: osdmap e53: 8 total, 8 up, 8 in 2024-02-17T12:45:16.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:16 smithi018 ceph-mon[32625]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:16.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:16 smithi018 ceph-mon[32625]: Detected new or changed devices on smithi119 2024-02-17T12:45:16.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:16 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi119 to 5693M 2024-02-17T12:45:16.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:16 smithi018 ceph-mon[32625]: osdmap e53: 8 total, 8 up, 8 in 2024-02-17T12:45:16.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:16.550+0000 7f2808872700 1 -- 172.21.15.146:0/3777413142 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f280402d950 con 0x7f27ec0a2150 2024-02-17T12:45:16.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:16 smithi119 ceph-mon[36188]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-17T12:45:16.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:16 smithi119 ceph-mon[36188]: Detected new or changed devices on smithi119 2024-02-17T12:45:16.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:16 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi119 to 5693M 2024-02-17T12:45:16.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:16 smithi119 ceph-mon[36188]: osdmap e53: 8 total, 8 up, 8 in 2024-02-17T12:45:17.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:17 smithi018 ceph-mon[32625]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:17.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:17 smithi018 ceph-mon[32625]: osdmap e54: 8 total, 8 up, 8 in 2024-02-17T12:45:17.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:17 smithi018 ceph-mon[32625]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:45:17.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:17 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:45:17.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:17 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:45:17.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:17 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:17.815 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:17 smithi146 ceph-mon[36454]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:17.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:17 smithi146 ceph-mon[36454]: osdmap e54: 8 total, 8 up, 8 in 2024-02-17T12:45:17.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:17 smithi146 ceph-mon[36454]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:45:17.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:17 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:45:17.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:17 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:45:17.816 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:17 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:17.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:17 smithi119 ceph-mon[36188]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:17.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:17 smithi119 ceph-mon[36188]: osdmap e54: 8 total, 8 up, 8 in 2024-02-17T12:45:17.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:17 smithi119 ceph-mon[36188]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:45:17.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:17 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:45:17.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:17 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:45:17.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:17 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:19.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:19 smithi146 ceph-mon[36454]: osdmap e55: 8 total, 8 up, 8 in 2024-02-17T12:45:19.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:19 smithi018 ceph-mon[32625]: osdmap e55: 8 total, 8 up, 8 in 2024-02-17T12:45:19.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:19 smithi119 ceph-mon[36188]: osdmap e55: 8 total, 8 up, 8 in 2024-02-17T12:45:20.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:20 smithi146 ceph-mon[36454]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:20.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:20 smithi146 ceph-mon[36454]: osdmap e56: 8 total, 8 up, 8 in 2024-02-17T12:45:20.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:20 smithi146 ceph-mon[36454]: from='client.? 172.21.15.146:0/1750392214' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e49e3406-56e4-422e-b157-e5c12ee087a8"}]: dispatch 2024-02-17T12:45:20.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:20 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e49e3406-56e4-422e-b157-e5c12ee087a8"}]: dispatch 2024-02-17T12:45:20.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:20 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e49e3406-56e4-422e-b157-e5c12ee087a8"}]': finished 2024-02-17T12:45:20.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:20 smithi146 ceph-mon[36454]: osdmap e57: 9 total, 8 up, 9 in 2024-02-17T12:45:20.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:20 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:20.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:20 smithi146 ceph-mon[36454]: from='client.? 172.21.15.146:0/272083582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:45:20.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:20 smithi018 ceph-mon[32625]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:20.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:20 smithi018 ceph-mon[32625]: osdmap e56: 8 total, 8 up, 8 in 2024-02-17T12:45:20.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:20 smithi018 ceph-mon[32625]: from='client.? 172.21.15.146:0/1750392214' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e49e3406-56e4-422e-b157-e5c12ee087a8"}]: dispatch 2024-02-17T12:45:20.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:20 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e49e3406-56e4-422e-b157-e5c12ee087a8"}]: dispatch 2024-02-17T12:45:20.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:20 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e49e3406-56e4-422e-b157-e5c12ee087a8"}]': finished 2024-02-17T12:45:20.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:20 smithi018 ceph-mon[32625]: osdmap e57: 9 total, 8 up, 9 in 2024-02-17T12:45:20.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:20 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:20.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:20 smithi018 ceph-mon[32625]: from='client.? 172.21.15.146:0/272083582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:45:20.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:20 smithi119 ceph-mon[36188]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:20.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:20 smithi119 ceph-mon[36188]: osdmap e56: 8 total, 8 up, 8 in 2024-02-17T12:45:20.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:20 smithi119 ceph-mon[36188]: from='client.? 172.21.15.146:0/1750392214' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e49e3406-56e4-422e-b157-e5c12ee087a8"}]: dispatch 2024-02-17T12:45:20.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:20 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e49e3406-56e4-422e-b157-e5c12ee087a8"}]: dispatch 2024-02-17T12:45:20.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:20 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e49e3406-56e4-422e-b157-e5c12ee087a8"}]': finished 2024-02-17T12:45:20.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:20 smithi119 ceph-mon[36188]: osdmap e57: 9 total, 8 up, 9 in 2024-02-17T12:45:20.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:20 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:20.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:20 smithi119 ceph-mon[36188]: from='client.? 172.21.15.146:0/272083582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:45:21.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:21 smithi146 ceph-mon[36454]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:21.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:21 smithi018 ceph-mon[32625]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:21.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:21 smithi119 ceph-mon[36188]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:23.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:23 smithi146 ceph-mon[36454]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:23.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:23 smithi018 ceph-mon[32625]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:23.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:23 smithi119 ceph-mon[36188]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:25.520 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:25 smithi146 ceph-mon[36454]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:25.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:25 smithi018 ceph-mon[32625]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:25.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:25 smithi119 ceph-mon[36188]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:27.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:27 smithi146 ceph-mon[36454]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:27.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:27 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-17T12:45:27.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:27 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:27.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:27 smithi146 ceph-mon[36454]: Deploying daemon osd.8 on smithi146 2024-02-17T12:45:27.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:27 smithi018 ceph-mon[32625]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:27.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:27 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-17T12:45:27.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:27 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:27.527 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:27 smithi018 ceph-mon[32625]: Deploying daemon osd.8 on smithi146 2024-02-17T12:45:27.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:27 smithi119 ceph-mon[36188]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:27.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:27 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-17T12:45:27.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:27 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:27.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:27 smithi119 ceph-mon[36188]: Deploying daemon osd.8 on smithi146 2024-02-17T12:45:29.479 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:29 smithi146 ceph-mon[36454]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:29.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:29 smithi018 ceph-mon[32625]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:29.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:29 smithi119 ceph-mon[36188]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:31.514 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:31 smithi146 ceph-mon[36454]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:31.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:31 smithi018 ceph-mon[32625]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:31.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:31 smithi119 ceph-mon[36188]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:33.409 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:33 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:33.409 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:33 smithi119 ceph-mon[36188]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:33.410 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:33 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:33.410 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:33 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:33.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:33 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:33.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:33 smithi146 ceph-mon[36454]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:33.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:33 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:33.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:33 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:33.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:33 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:33.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:33 smithi018 ceph-mon[32625]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:33.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:33 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:33.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:33 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:35.391 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:35 smithi146 ceph-mon[36454]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:35.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:35 smithi018 ceph-mon[32625]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:35.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:35 smithi119 ceph-mon[36188]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:35.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.669+0000 7f27f37fe700 1 -- 172.21.15.146:0/3777413142 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f280402d950 con 0x7f27ec0a2150 2024-02-17T12:45:35.663 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 8 on host 'smithi146' 2024-02-17T12:45:35.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.671+0000 7f2808872700 1 -- 172.21.15.146:0/3777413142 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f27ec0a2150 msgr2=0x7f27ec0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:45:35.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.671+0000 7f2808872700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f27ec0a2150 0x7f27ec0a4610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2804101bf0 tx=0x7f27f4008040 comp rx=0 tx=0).stop 2024-02-17T12:45:35.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.671+0000 7f2808872700 1 -- 172.21.15.146:0/3777413142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2804101d90 msgr2=0x7f2804198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:45:35.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.671+0000 7f2808872700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2804101d90 0x7f2804198290 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f27f800d980 tx=0x7f27f800dd40 comp rx=0 tx=0).stop 2024-02-17T12:45:35.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.671+0000 7f2808872700 1 -- 172.21.15.146:0/3777413142 shutdown_connections 2024-02-17T12:45:35.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.671+0000 7f2808872700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2804102730 0x7f280419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:35.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.671+0000 7f2808872700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2804100b90 0x7f2804197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:35.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.671+0000 7f2808872700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f27ec0a2150 0x7f27ec0a4610 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:35.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.671+0000 7f2808872700 1 --2- 172.21.15.146:0/3777413142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2804101d90 0x7f2804198290 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:35.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.671+0000 7f2808872700 1 -- 172.21.15.146:0/3777413142 >> 172.21.15.146:0/3777413142 conn(0x7f28040fc130 msgr2=0x7f2804104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:45:35.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.671+0000 7f2808872700 1 -- 172.21.15.146:0/3777413142 shutdown_connections 2024-02-17T12:45:35.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:45:35.671+0000 7f2808872700 1 -- 172.21.15.146:0/3777413142 wait complete. 2024-02-17T12:45:36.270 DEBUG:teuthology.orchestra.run.smithi146:osd.8> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.8.service 2024-02-17T12:45:36.272 INFO:tasks.cephadm:Deploying osd.9 on smithi018 with /dev/vg_nvme/lv_1... 2024-02-17T12:45:36.272 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-17T12:45:36.527 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:45:36.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:36 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:36.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:36 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:36.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:36 smithi018 ceph-mon[32625]: from='osd.8 [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-17T12:45:36.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:36 smithi018 ceph-mon[32625]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-17T12:45:36.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:36 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:36.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:36 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:36.916 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:36 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:36.916 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:36 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:36.916 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:36 smithi146 ceph-mon[36454]: from='osd.8 [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-17T12:45:36.916 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:36 smithi146 ceph-mon[36454]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-17T12:45:36.916 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:36 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:36.916 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:36 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:37.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:36 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:37.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:36 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:37.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:36 smithi119 ceph-mon[36188]: from='osd.8 [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-17T12:45:37.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:36 smithi119 ceph-mon[36188]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-17T12:45:37.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:36 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:37.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:36 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:37.932 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:37 smithi146 ceph-mon[36454]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:37.932 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:37 smithi146 ceph-mon[36454]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-17T12:45:37.932 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:37 smithi146 ceph-mon[36454]: osdmap e58: 9 total, 8 up, 9 in 2024-02-17T12:45:37.932 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:37.932 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:37 smithi146 ceph-mon[36454]: from='osd.8 [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:45:37.932 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:37 smithi146 ceph-mon[36454]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:45:37.932 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 12:45:37 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T12:45:37.712+0000 7ff985977700 -1 osd.8 0 waiting for initial osdmap 2024-02-17T12:45:37.932 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 12:45:37 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T12:45:37.725+0000 7ff981f6b700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T12:45:37.968 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:37 smithi018 ceph-mon[32625]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:37.968 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:37 smithi018 ceph-mon[32625]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-17T12:45:37.968 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:37 smithi018 ceph-mon[32625]: osdmap e58: 9 total, 8 up, 9 in 2024-02-17T12:45:37.968 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:37.968 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:37 smithi018 ceph-mon[32625]: from='osd.8 [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:45:37.968 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:37 smithi018 ceph-mon[32625]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:45:38.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:37 smithi119 ceph-mon[36188]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:38.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:37 smithi119 ceph-mon[36188]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-17T12:45:38.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:37 smithi119 ceph-mon[36188]: osdmap e58: 9 total, 8 up, 9 in 2024-02-17T12:45:38.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:38.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:37 smithi119 ceph-mon[36188]: from='osd.8 [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:45:38.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:37 smithi119 ceph-mon[36188]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:45:38.852 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-17T12:45:38.852 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-17T12:45:38.852 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-02-17T12:45:38.852 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-02-17T12:45:38.852 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00846187 s, 1.2 GB/s 2024-02-17T12:45:38.852 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-02-17T12:45:38.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:38 smithi146 ceph-mon[36454]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-02-17T12:45:38.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:38 smithi146 ceph-mon[36454]: osdmap e59: 9 total, 8 up, 9 in 2024-02-17T12:45:38.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:38 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:38.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:38 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:38.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:38 smithi146 ceph-mon[36454]: osd.8 [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241] boot 2024-02-17T12:45:38.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:38 smithi146 ceph-mon[36454]: osdmap e60: 9 total, 9 up, 9 in 2024-02-17T12:45:38.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:38 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:39.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:38 smithi018 ceph-mon[32625]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-02-17T12:45:39.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:38 smithi018 ceph-mon[32625]: osdmap e59: 9 total, 8 up, 9 in 2024-02-17T12:45:39.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:38 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:39.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:38 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:39.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:38 smithi018 ceph-mon[32625]: osd.8 [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241] boot 2024-02-17T12:45:39.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:38 smithi018 ceph-mon[32625]: osdmap e60: 9 total, 9 up, 9 in 2024-02-17T12:45:39.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:38 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:39.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:38 smithi119 ceph-mon[36188]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-02-17T12:45:39.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:38 smithi119 ceph-mon[36188]: osdmap e59: 9 total, 8 up, 9 in 2024-02-17T12:45:39.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:38 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:39.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:38 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:39.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:38 smithi119 ceph-mon[36188]: osd.8 [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241] boot 2024-02-17T12:45:39.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:38 smithi119 ceph-mon[36188]: osdmap e60: 9 total, 9 up, 9 in 2024-02-17T12:45:39.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:38 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:45:39.718 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi018:vg_nvme/lv_1 2024-02-17T12:45:39.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:39 smithi146 ceph-mon[36454]: purged_snaps scrub starts 2024-02-17T12:45:39.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:39 smithi146 ceph-mon[36454]: purged_snaps scrub ok 2024-02-17T12:45:39.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:39 smithi146 ceph-mon[36454]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:39.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:39 smithi146 ceph-mon[36454]: osdmap e61: 9 total, 9 up, 9 in 2024-02-17T12:45:39.977 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:45:40.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:39 smithi018 ceph-mon[32625]: purged_snaps scrub starts 2024-02-17T12:45:40.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:39 smithi018 ceph-mon[32625]: purged_snaps scrub ok 2024-02-17T12:45:40.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:39 smithi018 ceph-mon[32625]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:40.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:39 smithi018 ceph-mon[32625]: osdmap e61: 9 total, 9 up, 9 in 2024-02-17T12:45:40.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:39 smithi119 ceph-mon[36188]: purged_snaps scrub starts 2024-02-17T12:45:40.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:39 smithi119 ceph-mon[36188]: purged_snaps scrub ok 2024-02-17T12:45:40.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:39 smithi119 ceph-mon[36188]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-02-17T12:45:40.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:39 smithi119 ceph-mon[36188]: osdmap e61: 9 total, 9 up, 9 in 2024-02-17T12:45:40.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: Detected new or changed devices on smithi146 2024-02-17T12:45:40.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:40.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:40.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:40.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:40.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:40.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi146 to 5693M 2024-02-17T12:45:40.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:40.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:40.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:45:40.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:40.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:40.954 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:40.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:45:40.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:40.955 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:40 smithi146 ceph-mon[36454]: osdmap e62: 9 total, 9 up, 9 in 2024-02-17T12:45:41.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: Detected new or changed devices on smithi146 2024-02-17T12:45:41.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:41.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:41.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi146 to 5693M 2024-02-17T12:45:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:45:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:45:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:40 smithi018 ceph-mon[32625]: osdmap e62: 9 total, 9 up, 9 in 2024-02-17T12:45:41.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: Detected new or changed devices on smithi146 2024-02-17T12:45:41.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:41.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:41.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:41.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:41.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:45:41.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi146 to 5693M 2024-02-17T12:45:41.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:41.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:41.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:45:41.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:41.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:41.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:41.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:45:41.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:41.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:40 smithi119 ceph-mon[36188]: osdmap e62: 9 total, 9 up, 9 in 2024-02-17T12:45:41.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.644+0000 7f534fa38700 1 -- 172.21.15.18:0/3703916619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53480ffd20 msgr2=0x7f5348100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:45:41.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.644+0000 7f534fa38700 1 --2- 172.21.15.18:0/3703916619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53480ffd20 0x7f5348100180 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5338009990 tx=0x7f5338009ca0 comp rx=0 tx=0).stop 2024-02-17T12:45:41.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.644+0000 7f534fa38700 1 -- 172.21.15.18:0/3703916619 shutdown_connections 2024-02-17T12:45:41.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.644+0000 7f534fa38700 1 --2- 172.21.15.18:0/3703916619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53481006c0 0x7f534810a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.644+0000 7f534fa38700 1 --2- 172.21.15.18:0/3703916619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53481032a0 0x7f53481036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.644+0000 7f534fa38700 1 --2- 172.21.15.18:0/3703916619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53480ffd20 0x7f5348100180 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.644+0000 7f534fa38700 1 -- 172.21.15.18:0/3703916619 >> 172.21.15.18:0/3703916619 conn(0x7f53480fb8c0 msgr2=0x7f53480fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.645+0000 7f534fa38700 1 -- 172.21.15.18:0/3703916619 shutdown_connections 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.645+0000 7f534fa38700 1 -- 172.21.15.18:0/3703916619 wait complete. 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.645+0000 7f534fa38700 1 Processor -- start 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.645+0000 7f534fa38700 1 -- start start 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.645+0000 7f534fa38700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53480ffd20 0x7f53481029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.645+0000 7f534fa38700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53481006c0 0x7f5348100ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.645+0000 7f534fa38700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53481032a0 0x7f5348101530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.645+0000 7f534fa38700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5348101cb0 con 0x7f53481032a0 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.645+0000 7f534fa38700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5348101e20 con 0x7f53480ffd20 2024-02-17T12:45:41.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.645+0000 7f534fa38700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53481a1ca0 con 0x7f53481006c0 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534dfd5700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53481032a0 0x7f5348101530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534dfd5700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53481032a0 0x7f5348101530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34874/0 (socket says 172.21.15.18:34874) 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534dfd5700 1 -- 172.21.15.18:0/415211615 learned_addr learned my addr 172.21.15.18:0/415211615 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534d7d4700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53480ffd20 0x7f53481029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534cfd3700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53481006c0 0x7f5348100ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534dfd5700 1 -- 172.21.15.18:0/415211615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53481006c0 msgr2=0x7f5348100ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534dfd5700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53481006c0 0x7f5348100ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534dfd5700 1 -- 172.21.15.18:0/415211615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53480ffd20 msgr2=0x7f53481029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534dfd5700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53480ffd20 0x7f53481029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534dfd5700 1 -- 172.21.15.18:0/415211615 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53380095f0 con 0x7f53481032a0 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534dfd5700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53481032a0 0x7f5348101530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f534000ef30 tx=0x7f534000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f533e7fc700 1 -- 172.21.15.18:0/415211615 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340018070 con 0x7f53481032a0 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f533e7fc700 1 -- 172.21.15.18:0/415211615 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f534000f040 con 0x7f53481032a0 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534fa38700 1 -- 172.21.15.18:0/415211615 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53481a1f20 con 0x7f53481032a0 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f533e7fc700 1 -- 172.21.15.18:0/415211615 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534000cd50 con 0x7f53481032a0 2024-02-17T12:45:41.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.646+0000 7f534fa38700 1 -- 172.21.15.18:0/415211615 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53481a2390 con 0x7f53481032a0 2024-02-17T12:45:41.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.647+0000 7f534fa38700 1 -- 172.21.15.18:0/415211615 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5348109200 con 0x7f53481032a0 2024-02-17T12:45:41.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.648+0000 7f533e7fc700 1 -- 172.21.15.18:0/415211615 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5340007500 con 0x7f53481032a0 2024-02-17T12:45:41.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.649+0000 7f533e7fc700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f53340a2150 0x7f53340a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:45:41.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.649+0000 7f534d7d4700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f53340a2150 0x7f53340a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:45:41.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.649+0000 7f533e7fc700 1 -- 172.21.15.18:0/415211615 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f53400ccdd0 con 0x7f53481032a0 2024-02-17T12:45:41.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.649+0000 7f534d7d4700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f53340a2150 0x7f53340a4610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f534806a110 tx=0x7f5344009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:45:41.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.654+0000 7f533e7fc700 1 -- 172.21.15.18:0/415211615 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f53400967c0 con 0x7f53481032a0 2024-02-17T12:45:41.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:45:41.868+0000 7f534fa38700 1 -- 172.21.15.18:0/415211615 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f534802d910 con 0x7f53340a2150 2024-02-17T12:45:42.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:41 smithi146 ceph-mon[36454]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:45:42.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:41 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:45:42.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:41 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:45:42.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:41 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:42.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:41 smithi018 ceph-mon[32625]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:45:42.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:41 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:45:42.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:41 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:45:42.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:41 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:42.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:41 smithi119 ceph-mon[36188]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:45:42.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:41 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:45:42.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:41 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:45:42.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:41 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:42 smithi146 ceph-mon[36454]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:45:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:42 smithi146 ceph-mon[36454]: osdmap e63: 9 total, 9 up, 9 in 2024-02-17T12:45:43.217 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:42 smithi018 ceph-mon[32625]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:45:43.217 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:42 smithi018 ceph-mon[32625]: osdmap e63: 9 total, 9 up, 9 in 2024-02-17T12:45:43.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:42 smithi119 ceph-mon[36188]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:45:43.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:42 smithi119 ceph-mon[36188]: osdmap e63: 9 total, 9 up, 9 in 2024-02-17T12:45:44.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:43 smithi146 ceph-mon[36454]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:45:44.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:43 smithi146 ceph-mon[36454]: osdmap e64: 9 total, 9 up, 9 in 2024-02-17T12:45:44.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:43 smithi018 ceph-mon[32625]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:45:44.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:43 smithi018 ceph-mon[32625]: osdmap e64: 9 total, 9 up, 9 in 2024-02-17T12:45:44.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:43 smithi119 ceph-mon[36188]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:45:44.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:43 smithi119 ceph-mon[36188]: osdmap e64: 9 total, 9 up, 9 in 2024-02-17T12:45:45.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:44 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2442801386' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a41cd245-5402-474b-9e7e-f15557523fd5"}]: dispatch 2024-02-17T12:45:45.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:44 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2442801386' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a41cd245-5402-474b-9e7e-f15557523fd5"}]': finished 2024-02-17T12:45:45.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:44 smithi146 ceph-mon[36454]: osdmap e65: 10 total, 9 up, 10 in 2024-02-17T12:45:45.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:44 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:45:45.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:44 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2442801386' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a41cd245-5402-474b-9e7e-f15557523fd5"}]: dispatch 2024-02-17T12:45:45.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:44 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2442801386' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a41cd245-5402-474b-9e7e-f15557523fd5"}]': finished 2024-02-17T12:45:45.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:44 smithi018 ceph-mon[32625]: osdmap e65: 10 total, 9 up, 10 in 2024-02-17T12:45:45.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:44 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:45:45.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:44 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2442801386' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a41cd245-5402-474b-9e7e-f15557523fd5"}]: dispatch 2024-02-17T12:45:45.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:44 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2442801386' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a41cd245-5402-474b-9e7e-f15557523fd5"}]': finished 2024-02-17T12:45:45.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:44 smithi119 ceph-mon[36188]: osdmap e65: 10 total, 9 up, 10 in 2024-02-17T12:45:45.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:44 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:45:46.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:45 smithi146 ceph-mon[36454]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:45:46.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:45 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1644237063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:45:46.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:45 smithi018 ceph-mon[32625]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:45:46.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:45 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1644237063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:45:46.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:45 smithi119 ceph-mon[36188]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:45:46.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:45 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1644237063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:45:48.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:47 smithi146 ceph-mon[36454]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-02-17T12:45:48.230 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:47 smithi018 ceph-mon[32625]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-02-17T12:45:48.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:47 smithi119 ceph-mon[36188]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-02-17T12:45:50.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:49 smithi146 ceph-mon[36454]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-02-17T12:45:50.229 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:49 smithi018 ceph-mon[32625]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-02-17T12:45:50.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:49 smithi119 ceph-mon[36188]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-02-17T12:45:52.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:51 smithi146 ceph-mon[36454]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-17T12:45:52.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:51 smithi018 ceph-mon[32625]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-17T12:45:52.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:51 smithi119 ceph-mon[36188]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-17T12:45:53.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-17T12:45:53.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:53.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:52 smithi018 ceph-mon[32625]: Deploying daemon osd.9 on smithi018 2024-02-17T12:45:53.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-17T12:45:53.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:53.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:52 smithi146 ceph-mon[36454]: Deploying daemon osd.9 on smithi018 2024-02-17T12:45:53.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-17T12:45:53.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:45:53.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:52 smithi119 ceph-mon[36188]: Deploying daemon osd.9 on smithi018 2024-02-17T12:45:54.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:53 smithi018 ceph-mon[32625]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-02-17T12:45:54.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:53 smithi146 ceph-mon[36454]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-02-17T12:45:54.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:53 smithi119 ceph-mon[36188]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-02-17T12:45:56.134 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:55 smithi018 ceph-mon[32625]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-17T12:45:56.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:55 smithi119 ceph-mon[36188]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-17T12:45:56.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:55 smithi146 ceph-mon[36454]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-17T12:45:58.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:57 smithi018 ceph-mon[32625]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-02-17T12:45:58.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:58.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:58.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:57 smithi119 ceph-mon[36188]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-02-17T12:45:58.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:58.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:58.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:57 smithi146 ceph-mon[36454]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-02-17T12:45:58.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:58.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:45:59.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:58 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:59.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:58 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:45:59.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:58 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:00.218 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:45:59 smithi018 ceph-mon[32625]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:46:00.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:45:59 smithi119 ceph-mon[36188]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:46:00.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:45:59 smithi146 ceph-mon[36454]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:46:01.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.785+0000 7f533e7fc700 1 -- 172.21.15.18:0/415211615 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f534802d910 con 0x7f53340a2150 2024-02-17T12:46:01.779 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 9 on host 'smithi018' 2024-02-17T12:46:01.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.788+0000 7f534fa38700 1 -- 172.21.15.18:0/415211615 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f53340a2150 msgr2=0x7f53340a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:01.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.788+0000 7f534fa38700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f53340a2150 0x7f53340a4610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f534806a110 tx=0x7f5344009260 comp rx=0 tx=0).stop 2024-02-17T12:46:01.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.788+0000 7f534fa38700 1 -- 172.21.15.18:0/415211615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53481032a0 msgr2=0x7f5348101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:01.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.788+0000 7f534fa38700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53481032a0 0x7f5348101530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f534000ef30 tx=0x7f534000c3a0 comp rx=0 tx=0).stop 2024-02-17T12:46:01.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.788+0000 7f534fa38700 1 -- 172.21.15.18:0/415211615 shutdown_connections 2024-02-17T12:46:01.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.788+0000 7f534fa38700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53481032a0 0x7f5348101530 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:01.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.788+0000 7f534fa38700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53480ffd20 0x7f53481029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:01.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.788+0000 7f534fa38700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f53340a2150 0x7f53340a4610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:01.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.788+0000 7f534fa38700 1 --2- 172.21.15.18:0/415211615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53481006c0 0x7f5348100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:01.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.788+0000 7f534fa38700 1 -- 172.21.15.18:0/415211615 >> 172.21.15.18:0/415211615 conn(0x7f53480fb8c0 msgr2=0x7f5348106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:46:01.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.788+0000 7f534fa38700 1 -- 172.21.15.18:0/415211615 shutdown_connections 2024-02-17T12:46:01.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:01.788+0000 7f534fa38700 1 -- 172.21.15.18:0/415211615 wait complete. 2024-02-17T12:46:02.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:01 smithi018 ceph-mon[32625]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:46:02.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:01 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:01 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:01 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:02.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:01 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:02.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:01 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:01 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:01 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:01 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:02.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:01 smithi018 ceph-mon[32625]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-17T12:46:02.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:01 smithi119 ceph-mon[36188]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:46:02.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:01 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:01 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:01 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:02.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:01 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:02.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:01 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:01 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:01 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:01 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:02.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:01 smithi119 ceph-mon[36188]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-17T12:46:02.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:01 smithi146 ceph-mon[36454]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:46:02.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:01 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:01 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:01 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:02.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:01 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:02.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:01 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:01 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:01 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:02.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:01 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:02.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:01 smithi146 ceph-mon[36454]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-17T12:46:02.398 DEBUG:teuthology.orchestra.run.smithi018:osd.9> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.9.service 2024-02-17T12:46:02.401 INFO:tasks.cephadm:Deploying osd.10 on smithi119 with /dev/vg_nvme/lv_1... 2024-02-17T12:46:02.402 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-17T12:46:02.647 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T12:46:03.916 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:03 smithi018 ceph-mon[32625]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:46:03.917 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:03 smithi018 ceph-mon[32625]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-17T12:46:03.917 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:03 smithi018 ceph-mon[32625]: osdmap e66: 10 total, 9 up, 10 in 2024-02-17T12:46:03.917 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:03 smithi018 ceph-mon[32625]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:46:03.917 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:03 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:03.938 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:03 smithi119 ceph-mon[36188]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:46:03.938 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:03 smithi119 ceph-mon[36188]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-17T12:46:03.938 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:03 smithi119 ceph-mon[36188]: osdmap e66: 10 total, 9 up, 10 in 2024-02-17T12:46:03.938 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:03 smithi119 ceph-mon[36188]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:46:03.938 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:03 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:04.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:03 smithi146 ceph-mon[36454]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:46:04.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:03 smithi146 ceph-mon[36454]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-17T12:46:04.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:03 smithi146 ceph-mon[36454]: osdmap e66: 10 total, 9 up, 10 in 2024-02-17T12:46:04.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:03 smithi146 ceph-mon[36454]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-02-17T12:46:04.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:03 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-17T12:46:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-17T12:46:04.478 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-02-17T12:46:04.478 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-02-17T12:46:04.478 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112484 s, 932 MB/s 2024-02-17T12:46:04.478 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-02-17T12:46:04.820 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:04 smithi018 ceph-mon[32625]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-02-17T12:46:04.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:04 smithi018 ceph-mon[32625]: osdmap e67: 10 total, 9 up, 10 in 2024-02-17T12:46:04.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:04 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:04.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:04 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:04.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:04 smithi018 ceph-mon[32625]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' 2024-02-17T12:46:04.821 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 12:46:04 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T12:46:04.561+0000 7f2e74957700 -1 osd.9 0 waiting for initial osdmap 2024-02-17T12:46:04.821 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 12:46:04 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T12:46:04.575+0000 7f2e6cf43700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T12:46:05.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:04 smithi119 ceph-mon[36188]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-02-17T12:46:05.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:04 smithi119 ceph-mon[36188]: osdmap e67: 10 total, 9 up, 10 in 2024-02-17T12:46:05.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:04 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:05.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:04 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:05.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:04 smithi119 ceph-mon[36188]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' 2024-02-17T12:46:05.112 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_1 2024-02-17T12:46:05.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:04 smithi146 ceph-mon[36454]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-02-17T12:46:05.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:04 smithi146 ceph-mon[36454]: osdmap e67: 10 total, 9 up, 10 in 2024-02-17T12:46:05.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:04 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:05.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:04 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:05.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:04 smithi146 ceph-mon[36454]: from='osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833]' entity='osd.9' 2024-02-17T12:46:05.352 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T12:46:06.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: purged_snaps scrub starts 2024-02-17T12:46:06.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: purged_snaps scrub ok 2024-02-17T12:46:06.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:46:06.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:06.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833] boot 2024-02-17T12:46:06.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: osdmap e68: 10 total, 10 up, 10 in 2024-02-17T12:46:06.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:06.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:06.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:06.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:06.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:05 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:06.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: purged_snaps scrub starts 2024-02-17T12:46:06.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: purged_snaps scrub ok 2024-02-17T12:46:06.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:46:06.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:06.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833] boot 2024-02-17T12:46:06.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: osdmap e68: 10 total, 10 up, 10 in 2024-02-17T12:46:06.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:06.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:06.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:06.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:06.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:06.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:06.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:06.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:06.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:06.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:06.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:05 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:06.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: purged_snaps scrub starts 2024-02-17T12:46:06.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: purged_snaps scrub ok 2024-02-17T12:46:06.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-02-17T12:46:06.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: osd.9 [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833] boot 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: osdmap e68: 10 total, 10 up, 10 in 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:06.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:05 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.912+0000 7f08fef61700 1 -- 172.21.15.119:0/3016341421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f8102720 msgr2=0x7f08f8104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.912+0000 7f08fef61700 1 --2- 172.21.15.119:0/3016341421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f8102720 0x7f08f8104b10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009990 tx=0x7f08ec009ca0 comp rx=0 tx=0).stop 2024-02-17T12:46:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.912+0000 7f08fef61700 1 -- 172.21.15.119:0/3016341421 shutdown_connections 2024-02-17T12:46:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.912+0000 7f08fef61700 1 --2- 172.21.15.119:0/3016341421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f8107980 0x7f08f8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.912+0000 7f08fef61700 1 --2- 172.21.15.119:0/3016341421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f8102720 0x7f08f8104b10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.912+0000 7f08fef61700 1 --2- 172.21.15.119:0/3016341421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f8105050 0x7f08f8107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.912+0000 7f08fef61700 1 -- 172.21.15.119:0/3016341421 >> 172.21.15.119:0/3016341421 conn(0x7f08f80fc130 msgr2=0x7f08f80fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:46:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.912+0000 7f08fef61700 1 -- 172.21.15.119:0/3016341421 shutdown_connections 2024-02-17T12:46:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.912+0000 7f08fef61700 1 -- 172.21.15.119:0/3016341421 wait complete. 2024-02-17T12:46:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fef61700 1 Processor -- start 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fef61700 1 -- start start 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fef61700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f8102720 0x7f08f8197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fef61700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f8105050 0x7f08f8198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fef61700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f8107980 0x7f08f819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fef61700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f819f760 con 0x7f08f8107980 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fef61700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f819f8a0 con 0x7f08f8102720 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fef61700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f819fa30 con 0x7f08f8105050 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fccfd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f8102720 0x7f08f8197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fccfd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f8102720 0x7f08f8197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54680/0 (socket says 172.21.15.119:54680) 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fccfd700 1 -- 172.21.15.119:0/2095282853 learned_addr learned my addr 172.21.15.119:0/2095282853 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fd4fe700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f8107980 0x7f08f819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08f7fff700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f8105050 0x7f08f8198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fccfd700 1 -- 172.21.15.119:0/2095282853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f8105050 msgr2=0x7f08f8198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fccfd700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f8105050 0x7f08f8198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:06.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fccfd700 1 -- 172.21.15.119:0/2095282853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f8107980 msgr2=0x7f08f819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:06.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fccfd700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f8107980 0x7f08f819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:06.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08fccfd700 1 -- 172.21.15.119:0/2095282853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec0095f0 con 0x7f08f8102720 2024-02-17T12:46:06.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.913+0000 7f08f7fff700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f8105050 0x7f08f8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:46:06.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.914+0000 7f08fccfd700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f8102720 0x7f08f8197bd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00b6f0 tx=0x7f08ec00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:46:06.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.914+0000 7f08f5ffb700 1 -- 172.21.15.119:0/2095282853 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec01d070 con 0x7f08f8102720 2024-02-17T12:46:06.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.914+0000 7f08fef61700 1 -- 172.21.15.119:0/2095282853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f819fcb0 con 0x7f08f8102720 2024-02-17T12:46:06.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.914+0000 7f08f5ffb700 1 -- 172.21.15.119:0/2095282853 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08ec00fe60 con 0x7f08f8102720 2024-02-17T12:46:06.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.914+0000 7f08f5ffb700 1 -- 172.21.15.119:0/2095282853 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec017650 con 0x7f08f8102720 2024-02-17T12:46:06.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.914+0000 7f08fef61700 1 -- 172.21.15.119:0/2095282853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f81a0120 con 0x7f08f8102720 2024-02-17T12:46:06.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.915+0000 7f08fef61700 1 -- 172.21.15.119:0/2095282853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f80fdcb0 con 0x7f08f8102720 2024-02-17T12:46:06.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.916+0000 7f08f5ffb700 1 -- 172.21.15.119:0/2095282853 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f08ec021980 con 0x7f08f8102720 2024-02-17T12:46:06.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.917+0000 7f08f5ffb700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f08e80a2150 0x7f08e80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:06.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.917+0000 7f08f5ffb700 1 -- 172.21.15.119:0/2095282853 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f08ec0cef20 con 0x7f08f8102720 2024-02-17T12:46:06.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.917+0000 7f08f7fff700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f08e80a2150 0x7f08e80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:06.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.917+0000 7f08f7fff700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f08e80a2150 0x7f08e80a4610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009cb0 tx=0x7f08e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:46:06.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:06.920+0000 7f08f5ffb700 1 -- 172.21.15.119:0/2095282853 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f08ec098780 con 0x7f08f8102720 2024-02-17T12:46:07.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:06 smithi018 ceph-mon[32625]: Detected new or changed devices on smithi018 2024-02-17T12:46:07.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:06 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi018 to 4273M 2024-02-17T12:46:07.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:06 smithi018 ceph-mon[32625]: osdmap e69: 10 total, 10 up, 10 in 2024-02-17T12:46:07.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:06 smithi119 ceph-mon[36188]: Detected new or changed devices on smithi018 2024-02-17T12:46:07.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:06 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi018 to 4273M 2024-02-17T12:46:07.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:06 smithi119 ceph-mon[36188]: osdmap e69: 10 total, 10 up, 10 in 2024-02-17T12:46:07.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:07.121+0000 7f08fef61700 1 -- 172.21.15.119:0/2095282853 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f08f802d7d0 con 0x7f08e80a2150 2024-02-17T12:46:07.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:06 smithi146 ceph-mon[36454]: Detected new or changed devices on smithi018 2024-02-17T12:46:07.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:06 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi018 to 4273M 2024-02-17T12:46:07.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:06 smithi146 ceph-mon[36454]: osdmap e69: 10 total, 10 up, 10 in 2024-02-17T12:46:08.048 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:07 smithi119 ceph-mon[36188]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:08.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:07 smithi119 ceph-mon[36188]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:46:08.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:07 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:46:08.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:07 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:46:08.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:07 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:08.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:07 smithi146 ceph-mon[36454]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:08.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:07 smithi146 ceph-mon[36454]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:46:08.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:07 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:46:08.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:07 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:46:08.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:07 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:07 smithi018 ceph-mon[32625]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:07 smithi018 ceph-mon[32625]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:46:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:07 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:46:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:07 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:46:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:07 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:10.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:09 smithi119 ceph-mon[36188]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:10.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:09 smithi119 ceph-mon[36188]: from='client.? 172.21.15.119:0/402739823' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d"}]: dispatch 2024-02-17T12:46:10.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:09 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d"}]: dispatch 2024-02-17T12:46:10.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:09 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d"}]': finished 2024-02-17T12:46:10.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:09 smithi119 ceph-mon[36188]: osdmap e70: 11 total, 10 up, 11 in 2024-02-17T12:46:10.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:09 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:10.078 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:09 smithi018 ceph-mon[32625]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:10.078 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:09 smithi018 ceph-mon[32625]: from='client.? 172.21.15.119:0/402739823' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d"}]: dispatch 2024-02-17T12:46:10.078 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:09 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d"}]: dispatch 2024-02-17T12:46:10.078 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:09 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d"}]': finished 2024-02-17T12:46:10.079 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:09 smithi018 ceph-mon[32625]: osdmap e70: 11 total, 10 up, 11 in 2024-02-17T12:46:10.079 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:09 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:10.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:09 smithi146 ceph-mon[36454]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:10.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:09 smithi146 ceph-mon[36454]: from='client.? 172.21.15.119:0/402739823' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d"}]: dispatch 2024-02-17T12:46:10.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:09 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d"}]: dispatch 2024-02-17T12:46:10.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:09 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d"}]': finished 2024-02-17T12:46:10.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:09 smithi146 ceph-mon[36454]: osdmap e70: 11 total, 10 up, 11 in 2024-02-17T12:46:10.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:09 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:11.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:10 smithi119 ceph-mon[36188]: from='client.? 172.21.15.119:0/1136345722' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:46:11.098 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:10 smithi018 ceph-mon[32625]: from='client.? 172.21.15.119:0/1136345722' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:46:11.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:10 smithi146 ceph-mon[36454]: from='client.? 172.21.15.119:0/1136345722' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:46:12.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:11 smithi119 ceph-mon[36188]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:12.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:11 smithi146 ceph-mon[36454]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:12.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:11 smithi018 ceph-mon[32625]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:13.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:13 smithi119 ceph-mon[36188]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:14.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:13 smithi146 ceph-mon[36454]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:14.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:13 smithi018 ceph-mon[32625]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:16.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:15 smithi119 ceph-mon[36188]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:16.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:15 smithi146 ceph-mon[36454]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:16.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:15 smithi018 ceph-mon[32625]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:18.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:17 smithi119 ceph-mon[36188]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:18.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:17 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-17T12:46:18.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:17 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:18.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:17 smithi146 ceph-mon[36454]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:18.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:17 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-17T12:46:18.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:17 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:18.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:17 smithi018 ceph-mon[32625]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:18.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:17 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-17T12:46:18.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:17 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:18.963 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:18 smithi119 ceph-mon[36188]: Deploying daemon osd.10 on smithi119 2024-02-17T12:46:19.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:18 smithi146 ceph-mon[36454]: Deploying daemon osd.10 on smithi119 2024-02-17T12:46:19.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:18 smithi018 ceph-mon[32625]: Deploying daemon osd.10 on smithi119 2024-02-17T12:46:20.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:19 smithi119 ceph-mon[36188]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:20.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:19 smithi146 ceph-mon[36454]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:20.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:19 smithi018 ceph-mon[32625]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:21.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:21 smithi119 ceph-mon[36188]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:22.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:21 smithi146 ceph-mon[36454]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:22.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:21 smithi018 ceph-mon[32625]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:24.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:23 smithi119 ceph-mon[36188]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:24.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:23 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:24.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:23 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:24.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:23 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:24.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:23 smithi146 ceph-mon[36454]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:24.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:23 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:24.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:23 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:24.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:23 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:24.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:23 smithi018 ceph-mon[32625]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:24.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:23 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:24.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:23 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:24.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:23 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:26.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:25 smithi119 ceph-mon[36188]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:26.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:26.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:26.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:26.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:26.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:25 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:26.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:25 smithi146 ceph-mon[36454]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:26.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:26.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:26.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:26.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:26.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:25 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:26.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:25 smithi018 ceph-mon[32625]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:26.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:26.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:26.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:26.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:26.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:25 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:26.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.492+0000 7f08f5ffb700 1 -- 172.21.15.119:0/2095282853 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f08f802d7d0 con 0x7f08e80a2150 2024-02-17T12:46:26.486 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 10 on host 'smithi119' 2024-02-17T12:46:26.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.494+0000 7f08fef61700 1 -- 172.21.15.119:0/2095282853 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f08e80a2150 msgr2=0x7f08e80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:26.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.494+0000 7f08fef61700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f08e80a2150 0x7f08e80a4610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009cb0 tx=0x7f08e4009260 comp rx=0 tx=0).stop 2024-02-17T12:46:26.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.494+0000 7f08fef61700 1 -- 172.21.15.119:0/2095282853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f8102720 msgr2=0x7f08f8197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:26.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.494+0000 7f08fef61700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f8102720 0x7f08f8197bd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00b6f0 tx=0x7f08ec00f690 comp rx=0 tx=0).stop 2024-02-17T12:46:26.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.494+0000 7f08fef61700 1 -- 172.21.15.119:0/2095282853 shutdown_connections 2024-02-17T12:46:26.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.494+0000 7f08fef61700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f8107980 0x7f08f819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:26.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.494+0000 7f08fef61700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f8102720 0x7f08f8197bd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:26.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.494+0000 7f08fef61700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f08e80a2150 0x7f08e80a4610 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:26.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.494+0000 7f08fef61700 1 --2- 172.21.15.119:0/2095282853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f8105050 0x7f08f8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:26.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.494+0000 7f08fef61700 1 -- 172.21.15.119:0/2095282853 >> 172.21.15.119:0/2095282853 conn(0x7f08f80fc130 msgr2=0x7f08f8103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:46:26.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.494+0000 7f08fef61700 1 -- 172.21.15.119:0/2095282853 shutdown_connections 2024-02-17T12:46:26.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-02-17T12:46:26.494+0000 7f08fef61700 1 -- 172.21.15.119:0/2095282853 wait complete. 2024-02-17T12:46:27.128 DEBUG:teuthology.orchestra.run.smithi119:osd.10> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.10.service 2024-02-17T12:46:27.131 INFO:tasks.cephadm:Deploying osd.11 on smithi146 with /dev/vg_nvme/lv_1... 2024-02-17T12:46:27.131 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-17T12:46:27.379 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:46:27.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:27 smithi146 ceph-mon[36454]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:27.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:27 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:27.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:27 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:27.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:27 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:27.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:27 smithi146 ceph-mon[36454]: from='osd.10 [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-17T12:46:27.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:27 smithi146 ceph-mon[36454]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-17T12:46:27.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:27 smithi018 ceph-mon[32625]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:27.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:27 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:27.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:27 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:27.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:27 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:27.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:27 smithi018 ceph-mon[32625]: from='osd.10 [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-17T12:46:27.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:27 smithi018 ceph-mon[32625]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-17T12:46:27.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:27 smithi119 ceph-mon[36188]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:27.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:27 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:27.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:27 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:27.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:27 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:27.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:27 smithi119 ceph-mon[36188]: from='osd.10 [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-17T12:46:27.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:27 smithi119 ceph-mon[36188]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-17T12:46:28.465 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:28 smithi119 ceph-mon[36188]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-17T12:46:28.466 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:28 smithi119 ceph-mon[36188]: from='osd.10 [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:46:28.466 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:28 smithi119 ceph-mon[36188]: osdmap e71: 11 total, 10 up, 11 in 2024-02-17T12:46:28.466 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:28 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:28.466 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:28 smithi119 ceph-mon[36188]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:46:28.745 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:28 smithi146 ceph-mon[36454]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-17T12:46:28.745 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:28 smithi146 ceph-mon[36454]: from='osd.10 [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:46:28.745 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:28 smithi146 ceph-mon[36454]: osdmap e71: 11 total, 10 up, 11 in 2024-02-17T12:46:28.745 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:28 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:28.745 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:28 smithi146 ceph-mon[36454]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:46:28.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:28 smithi018 ceph-mon[32625]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-17T12:46:28.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:28 smithi018 ceph-mon[32625]: from='osd.10 [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:46:28.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:28 smithi018 ceph-mon[32625]: osdmap e71: 11 total, 10 up, 11 in 2024-02-17T12:46:28.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:28 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:28.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:28 smithi018 ceph-mon[32625]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-02-17T12:46:28.844 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 12:46:28 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T12:46:28.472+0000 7f2d824d8700 -1 osd.10 0 waiting for initial osdmap 2024-02-17T12:46:28.845 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 12:46:28 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T12:46:28.487+0000 7f2d7d2c9700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T12:46:29.467 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-17T12:46:29.468 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-17T12:46:29.468 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2024-02-17T12:46:29.468 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2024-02-17T12:46:29.468 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112891 s, 929 MB/s 2024-02-17T12:46:29.468 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2024-02-17T12:46:29.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:29 smithi119 ceph-mon[36188]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:29.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:29 smithi119 ceph-mon[36188]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-02-17T12:46:29.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:29 smithi119 ceph-mon[36188]: osdmap e72: 11 total, 10 up, 11 in 2024-02-17T12:46:29.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:29 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:29.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:29 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:29.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:29 smithi119 ceph-mon[36188]: osd.10 [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609] boot 2024-02-17T12:46:29.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:29 smithi119 ceph-mon[36188]: osdmap e73: 11 total, 11 up, 11 in 2024-02-17T12:46:29.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:29 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:29.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:29 smithi018 ceph-mon[32625]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:29.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:29 smithi018 ceph-mon[32625]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-02-17T12:46:29.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:29 smithi018 ceph-mon[32625]: osdmap e72: 11 total, 10 up, 11 in 2024-02-17T12:46:29.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:29 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:29.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:29 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:29.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:29 smithi018 ceph-mon[32625]: osd.10 [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609] boot 2024-02-17T12:46:29.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:29 smithi018 ceph-mon[32625]: osdmap e73: 11 total, 11 up, 11 in 2024-02-17T12:46:29.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:29 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:29.777 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:29 smithi146 ceph-mon[36454]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-02-17T12:46:29.777 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:29 smithi146 ceph-mon[36454]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-02-17T12:46:29.777 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:29 smithi146 ceph-mon[36454]: osdmap e72: 11 total, 10 up, 11 in 2024-02-17T12:46:29.777 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:29 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:29.777 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:29 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:29.777 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:29 smithi146 ceph-mon[36454]: osd.10 [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609] boot 2024-02-17T12:46:29.777 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:29 smithi146 ceph-mon[36454]: osdmap e73: 11 total, 11 up, 11 in 2024-02-17T12:46:29.777 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:29 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:46:30.261 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi146:vg_nvme/lv_1 2024-02-17T12:46:30.511 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:46:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: purged_snaps scrub starts 2024-02-17T12:46:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: purged_snaps scrub ok 2024-02-17T12:46:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: osdmap e74: 11 total, 11 up, 11 in 2024-02-17T12:46:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: Detected new or changed devices on smithi119 2024-02-17T12:46:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi119 to 4270M 2024-02-17T12:46:30.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:30.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:30.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:30.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:30 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:30.871 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: purged_snaps scrub starts 2024-02-17T12:46:30.871 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: purged_snaps scrub ok 2024-02-17T12:46:30.871 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: osdmap e74: 11 total, 11 up, 11 in 2024-02-17T12:46:30.871 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: Detected new or changed devices on smithi119 2024-02-17T12:46:30.872 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:30.872 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:30.872 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:30.872 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:30.872 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:30.873 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:30.873 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi119 to 4270M 2024-02-17T12:46:30.873 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:30.873 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:30.873 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:30.873 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:30 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:31.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: purged_snaps scrub starts 2024-02-17T12:46:31.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: purged_snaps scrub ok 2024-02-17T12:46:31.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: osdmap e74: 11 total, 11 up, 11 in 2024-02-17T12:46:31.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: Detected new or changed devices on smithi119 2024-02-17T12:46:31.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:31.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:31.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:31.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:31.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:31.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:31.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi119 to 4270M 2024-02-17T12:46:31.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:31.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:31.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:31.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:30 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:31.791 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:31 smithi146 ceph-mon[36454]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:31.791 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:31 smithi146 ceph-mon[36454]: osdmap e75: 11 total, 11 up, 11 in 2024-02-17T12:46:31.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:31 smithi119 ceph-mon[36188]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:31.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:31 smithi119 ceph-mon[36188]: osdmap e75: 11 total, 11 up, 11 in 2024-02-17T12:46:32.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:31 smithi018 ceph-mon[32625]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:32.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:31 smithi018 ceph-mon[32625]: osdmap e75: 11 total, 11 up, 11 in 2024-02-17T12:46:32.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.139+0000 7f765f15b700 1 -- 172.21.15.146:0/1271196036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7658107990 msgr2=0x7f7658109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:32.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.139+0000 7f765f15b700 1 --2- 172.21.15.146:0/1271196036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7658107990 0x7f7658109e00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f765000b0a0 tx=0x7f765000b3b0 comp rx=0 tx=0).stop 2024-02-17T12:46:32.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.140+0000 7f765f15b700 1 -- 172.21.15.146:0/1271196036 shutdown_connections 2024-02-17T12:46:32.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.140+0000 7f765f15b700 1 --2- 172.21.15.146:0/1271196036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7658107990 0x7f7658109e00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:32.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.140+0000 7f765f15b700 1 --2- 172.21.15.146:0/1271196036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7658102730 0x7f7658104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:32.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.140+0000 7f765f15b700 1 --2- 172.21.15.146:0/1271196036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7658105060 0x7f7658107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:32.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.140+0000 7f765f15b700 1 -- 172.21.15.146:0/1271196036 >> 172.21.15.146:0/1271196036 conn(0x7f76580fc140 msgr2=0x7f76580fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:46:32.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.140+0000 7f765f15b700 1 -- 172.21.15.146:0/1271196036 shutdown_connections 2024-02-17T12:46:32.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.140+0000 7f765f15b700 1 -- 172.21.15.146:0/1271196036 wait complete. 2024-02-17T12:46:32.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.140+0000 7f765f15b700 1 Processor -- start 2024-02-17T12:46:32.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.140+0000 7f765f15b700 1 -- start start 2024-02-17T12:46:32.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.140+0000 7f765f15b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7658102730 0x7f7658197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:32.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.140+0000 7f765f15b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7658105060 0x7f76581982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:32.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f765f15b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7658107990 0x7f765819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:32.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f765f15b700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765819f900 con 0x7f7658102730 2024-02-17T12:46:32.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f765f15b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765819fa40 con 0x7f7658107990 2024-02-17T12:46:32.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f765f15b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765819fc00 con 0x7f7658105060 2024-02-17T12:46:32.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f7657fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7658105060 0x7f76581982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:32.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f7657fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7658105060 0x7f76581982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50272/0 (socket says 172.21.15.146:50272) 2024-02-17T12:46:32.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f7657fff700 1 -- 172.21.15.146:0/1763507397 learned_addr learned my addr 172.21.15.146:0/1763507397 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-02-17T12:46:32.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f765cef7700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7658102730 0x7f7658197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:32.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f765d6f8700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7658107990 0x7f765819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:32.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f7657fff700 1 -- 172.21.15.146:0/1763507397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7658107990 msgr2=0x7f765819f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:32.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f7657fff700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7658107990 0x7f765819f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:32.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f7657fff700 1 -- 172.21.15.146:0/1763507397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7658102730 msgr2=0x7f7658197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:32.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f7657fff700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7658102730 0x7f7658197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:32.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f7657fff700 1 -- 172.21.15.146:0/1763507397 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7650009d00 con 0x7f7658105060 2024-02-17T12:46:32.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f765d6f8700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7658107990 0x7f765819f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:46:32.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f7657fff700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7658105060 0x7f76581982f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f764400d950 tx=0x7f764400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:46:32.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f7655ffb700 1 -- 172.21.15.146:0/1763507397 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644009940 con 0x7f7658105060 2024-02-17T12:46:32.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f7655ffb700 1 -- 172.21.15.146:0/1763507397 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7644010460 con 0x7f7658105060 2024-02-17T12:46:32.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.141+0000 7f765f15b700 1 -- 172.21.15.146:0/1763507397 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765819fee0 con 0x7f7658105060 2024-02-17T12:46:32.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.142+0000 7f7655ffb700 1 -- 172.21.15.146:0/1763507397 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400f620 con 0x7f7658105060 2024-02-17T12:46:32.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.142+0000 7f765f15b700 1 -- 172.21.15.146:0/1763507397 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76581a02f0 con 0x7f7658105060 2024-02-17T12:46:32.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.143+0000 7f765f15b700 1 -- 172.21.15.146:0/1763507397 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76580fdcc0 con 0x7f7658105060 2024-02-17T12:46:32.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.144+0000 7f7655ffb700 1 -- 172.21.15.146:0/1763507397 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f76440105d0 con 0x7f7658105060 2024-02-17T12:46:32.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.145+0000 7f7655ffb700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f76480a2150 0x7f76480a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:32.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.145+0000 7f7655ffb700 1 -- 172.21.15.146:0/1763507397 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f76440cd4f0 con 0x7f7658105060 2024-02-17T12:46:32.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.145+0000 7f765cef7700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f76480a2150 0x7f76480a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:32.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.145+0000 7f765cef7700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f76480a2150 0x7f76480a4610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7658199290 tx=0x7f764c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:46:32.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.149+0000 7f7655ffb700 1 -- 172.21.15.146:0/1763507397 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f764409a640 con 0x7f7658105060 2024-02-17T12:46:32.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:32.353+0000 7f765f15b700 1 -- 172.21.15.146:0/1763507397 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f765802d950 con 0x7f76480a2150 2024-02-17T12:46:32.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:32 smithi119 ceph-mon[36188]: osdmap e76: 11 total, 11 up, 11 in 2024-02-17T12:46:32.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:32 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:46:32.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:32 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:46:32.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:32 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:32.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:32 smithi146 ceph-mon[36454]: osdmap e76: 11 total, 11 up, 11 in 2024-02-17T12:46:32.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:32 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:46:32.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:32 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:46:32.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:32 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:33.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:32 smithi018 ceph-mon[32625]: osdmap e76: 11 total, 11 up, 11 in 2024-02-17T12:46:33.090 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:32 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T12:46:33.090 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:32 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T12:46:33.090 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:32 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:33.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:33 smithi119 ceph-mon[36188]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:33.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:33 smithi119 ceph-mon[36188]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:46:33.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:33 smithi119 ceph-mon[36188]: osdmap e77: 11 total, 11 up, 11 in 2024-02-17T12:46:33.910 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:33 smithi146 ceph-mon[36454]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:33.910 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:33 smithi146 ceph-mon[36454]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:46:33.911 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:33 smithi146 ceph-mon[36454]: osdmap e77: 11 total, 11 up, 11 in 2024-02-17T12:46:34.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:33 smithi018 ceph-mon[32625]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:34.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:33 smithi018 ceph-mon[32625]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:46:34.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:33 smithi018 ceph-mon[32625]: osdmap e77: 11 total, 11 up, 11 in 2024-02-17T12:46:35.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:35 smithi146 ceph-mon[36454]: pgmap v239: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:35.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:35 smithi146 ceph-mon[36454]: from='client.? 172.21.15.146:0/2882123428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e507f2ad-80c0-44f5-be10-38d3da841477"}]: dispatch 2024-02-17T12:46:35.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:35 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e507f2ad-80c0-44f5-be10-38d3da841477"}]: dispatch 2024-02-17T12:46:35.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:35 smithi146 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e507f2ad-80c0-44f5-be10-38d3da841477"}]': finished 2024-02-17T12:46:35.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:35 smithi146 ceph-mon[36454]: osdmap e78: 12 total, 11 up, 12 in 2024-02-17T12:46:35.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:35 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:35.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:35 smithi119 ceph-mon[36188]: pgmap v239: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:35 smithi119 ceph-mon[36188]: from='client.? 172.21.15.146:0/2882123428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e507f2ad-80c0-44f5-be10-38d3da841477"}]: dispatch 2024-02-17T12:46:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:35 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e507f2ad-80c0-44f5-be10-38d3da841477"}]: dispatch 2024-02-17T12:46:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:35 smithi119 ceph-mon[36188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e507f2ad-80c0-44f5-be10-38d3da841477"}]': finished 2024-02-17T12:46:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:35 smithi119 ceph-mon[36188]: osdmap e78: 12 total, 11 up, 12 in 2024-02-17T12:46:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:35 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:36.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:35 smithi018 ceph-mon[32625]: pgmap v239: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:36.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:35 smithi018 ceph-mon[32625]: from='client.? 172.21.15.146:0/2882123428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e507f2ad-80c0-44f5-be10-38d3da841477"}]: dispatch 2024-02-17T12:46:36.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:35 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e507f2ad-80c0-44f5-be10-38d3da841477"}]: dispatch 2024-02-17T12:46:36.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:35 smithi018 ceph-mon[32625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e507f2ad-80c0-44f5-be10-38d3da841477"}]': finished 2024-02-17T12:46:36.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:35 smithi018 ceph-mon[32625]: osdmap e78: 12 total, 11 up, 12 in 2024-02-17T12:46:36.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:35 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:36.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:36 smithi119 ceph-mon[36188]: from='client.? 172.21.15.146:0/2736638596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:46:36.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:36 smithi146 ceph-mon[36454]: from='client.? 172.21.15.146:0/2736638596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:46:37.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:36 smithi018 ceph-mon[32625]: from='client.? 172.21.15.146:0/2736638596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T12:46:37.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:37 smithi119 ceph-mon[36188]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-02-17T12:46:37.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:37 smithi146 ceph-mon[36454]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-02-17T12:46:38.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:37 smithi018 ceph-mon[32625]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-02-17T12:46:39.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:39 smithi119 ceph-mon[36188]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-02-17T12:46:39.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:39 smithi146 ceph-mon[36454]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-02-17T12:46:40.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:39 smithi018 ceph-mon[32625]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-02-17T12:46:41.828 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:41 smithi018 ceph-mon[32625]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-17T12:46:41.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:41 smithi119 ceph-mon[36188]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-17T12:46:41.876 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:41 smithi146 ceph-mon[36454]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-17T12:46:43.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:43 smithi146 ceph-mon[36454]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-02-17T12:46:43.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:43 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-17T12:46:43.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:43 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:43.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:43 smithi146 ceph-mon[36454]: Deploying daemon osd.11 on smithi146 2024-02-17T12:46:43.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:43 smithi119 ceph-mon[36188]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-02-17T12:46:43.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-17T12:46:43.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:43 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:43.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:43 smithi119 ceph-mon[36188]: Deploying daemon osd.11 on smithi146 2024-02-17T12:46:44.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:43 smithi018 ceph-mon[32625]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-02-17T12:46:44.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:43 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-17T12:46:44.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:43 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:44.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:43 smithi018 ceph-mon[32625]: Deploying daemon osd.11 on smithi146 2024-02-17T12:46:45.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:45 smithi119 ceph-mon[36188]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-17T12:46:45.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:45 smithi146 ceph-mon[36454]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-17T12:46:46.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:45 smithi018 ceph-mon[32625]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-17T12:46:47.832 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:47 smithi146 ceph-mon[36454]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-02-17T12:46:47.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:47 smithi119 ceph-mon[36188]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-02-17T12:46:48.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:47 smithi018 ceph-mon[32625]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-02-17T12:46:50.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:49 smithi018 ceph-mon[32625]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:50.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:49 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:50.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:49 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:50.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:49 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:50.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:49 smithi119 ceph-mon[36188]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:50.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:49 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:50.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:49 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:50.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:49 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:50.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:49 smithi146 ceph-mon[36454]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:50.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:49 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:50.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:49 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:50.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:49 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:52.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:51 smithi018 ceph-mon[32625]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:52.054 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:51 smithi146 ceph-mon[36454]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:52.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:51 smithi119 ceph-mon[36188]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:52.118 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 11 on host 'smithi146' 2024-02-17T12:46:52.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.125+0000 7f7655ffb700 1 -- 172.21.15.146:0/1763507397 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f765802d950 con 0x7f76480a2150 2024-02-17T12:46:52.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.128+0000 7f765f15b700 1 -- 172.21.15.146:0/1763507397 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f76480a2150 msgr2=0x7f76480a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:52.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.128+0000 7f765f15b700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f76480a2150 0x7f76480a4610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7658199290 tx=0x7f764c009260 comp rx=0 tx=0).stop 2024-02-17T12:46:52.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.128+0000 7f765f15b700 1 -- 172.21.15.146:0/1763507397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7658105060 msgr2=0x7f76581982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:52.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.128+0000 7f765f15b700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7658105060 0x7f76581982f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f764400d950 tx=0x7f764400dc60 comp rx=0 tx=0).stop 2024-02-17T12:46:52.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.128+0000 7f765f15b700 1 -- 172.21.15.146:0/1763507397 shutdown_connections 2024-02-17T12:46:52.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.128+0000 7f765f15b700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7658107990 0x7f765819f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:52.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.129+0000 7f765f15b700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7658102730 0x7f7658197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:52.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.129+0000 7f765f15b700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f76480a2150 0x7f76480a4610 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:52.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.129+0000 7f765f15b700 1 --2- 172.21.15.146:0/1763507397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7658105060 0x7f76581982f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:52.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.129+0000 7f765f15b700 1 -- 172.21.15.146:0/1763507397 >> 172.21.15.146:0/1763507397 conn(0x7f76580fc140 msgr2=0x7f7658103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:46:52.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.129+0000 7f765f15b700 1 -- 172.21.15.146:0/1763507397 shutdown_connections 2024-02-17T12:46:52.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:46:52.129+0000 7f765f15b700 1 -- 172.21.15.146:0/1763507397 wait complete. 2024-02-17T12:46:52.741 DEBUG:teuthology.orchestra.run.smithi146:osd.11> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.11.service 2024-02-17T12:46:52.743 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-02-17T12:46:52.744 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-17T12:46:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:46:53.339 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:53.339 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:53.339 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:53 smithi018 ceph-mon[32625]: from='osd.11 [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-17T12:46:53.339 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:53 smithi018 ceph-mon[32625]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-17T12:46:53.339 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:53.340 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:53 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:53.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:53 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:53.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:53 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:53.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:53 smithi119 ceph-mon[36188]: from='osd.11 [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-17T12:46:53.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:53 smithi119 ceph-mon[36188]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-17T12:46:53.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:53 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:53.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:53 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:53.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:53 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:53.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:53 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:53.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:53 smithi146 ceph-mon[36454]: from='osd.11 [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-17T12:46:53.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:53 smithi146 ceph-mon[36454]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-17T12:46:53.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:53 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:53.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:53 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:54.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:54 smithi146 ceph-mon[36454]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:54.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:54 smithi146 ceph-mon[36454]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-17T12:46:54.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:54 smithi146 ceph-mon[36454]: osdmap e79: 12 total, 11 up, 12 in 2024-02-17T12:46:54.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:54 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:54.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:54 smithi146 ceph-mon[36454]: from='osd.11 [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:46:54.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:54 smithi146 ceph-mon[36454]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:46:54.203 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 12:46:54 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T12:46:54.108+0000 7fdec482a700 -1 osd.11 0 waiting for initial osdmap 2024-02-17T12:46:54.203 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 12:46:54 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T12:46:54.124+0000 7fdec0e1e700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T12:46:54.423 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:54 smithi018 ceph-mon[32625]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:54.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:54 smithi018 ceph-mon[32625]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-17T12:46:54.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:54 smithi018 ceph-mon[32625]: osdmap e79: 12 total, 11 up, 12 in 2024-02-17T12:46:54.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:54 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:54.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:54 smithi018 ceph-mon[32625]: from='osd.11 [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:46:54.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:54 smithi018 ceph-mon[32625]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:46:54.604 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:54 smithi119 ceph-mon[36188]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:54.604 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:54 smithi119 ceph-mon[36188]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-17T12:46:54.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:54 smithi119 ceph-mon[36188]: osdmap e79: 12 total, 11 up, 12 in 2024-02-17T12:46:54.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:54 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:54.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:54 smithi119 ceph-mon[36188]: from='osd.11 [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:46:54.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:54 smithi119 ceph-mon[36188]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-02-17T12:46:54.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.665+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/4166213362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf34100c10 msgr2=0x7fcf34101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:54.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.665+0000 7fcf3b1d1700 1 --2- 172.21.15.18:0/4166213362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf34100c10 0x7fcf34101030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00b0d0 tx=0x7fcf2c00b3e0 comp rx=0 tx=0).stop 2024-02-17T12:46:54.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.665+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/4166213362 shutdown_connections 2024-02-17T12:46:54.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.665+0000 7fcf3b1d1700 1 --2- 172.21.15.18:0/4166213362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341027b0 0x7fcf34109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:54.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.665+0000 7fcf3b1d1700 1 --2- 172.21.15.18:0/4166213362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf34100c10 0x7fcf34101030 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:54.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.665+0000 7fcf3b1d1700 1 --2- 172.21.15.18:0/4166213362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf34101e10 0x7fcf34102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:54.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.665+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/4166213362 >> 172.21.15.18:0/4166213362 conn(0x7fcf340fc1d0 msgr2=0x7fcf340fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:46:54.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.665+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/4166213362 shutdown_connections 2024-02-17T12:46:54.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.665+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/4166213362 wait complete. 2024-02-17T12:46:54.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3b1d1700 1 Processor -- start 2024-02-17T12:46:54.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3b1d1700 1 -- start start 2024-02-17T12:46:54.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3b1d1700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf34100c10 0x7fcf34197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:54.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3b1d1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf34101e10 0x7fcf341982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:54.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3b1d1700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf341027b0 0x7fcf3419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:54.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3b1d1700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3419f8e0 con 0x7fcf341027b0 2024-02-17T12:46:54.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3b1d1700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3419fa20 con 0x7fcf34100c10 2024-02-17T12:46:54.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3b1d1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3419fbe0 con 0x7fcf34101e10 2024-02-17T12:46:54.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3976e700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf341027b0 0x7fcf3419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:54.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3976e700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf341027b0 0x7fcf3419f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50552/0 (socket says 172.21.15.18:50552) 2024-02-17T12:46:54.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3976e700 1 -- 172.21.15.18:0/1417336491 learned_addr learned my addr 172.21.15.18:0/1417336491 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:46:54.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf38f6d700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf34100c10 0x7fcf34197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:54.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf33fff700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf34101e10 0x7fcf341982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:54.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3976e700 1 -- 172.21.15.18:0/1417336491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf34101e10 msgr2=0x7fcf341982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:54.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3976e700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf34101e10 0x7fcf341982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3976e700 1 -- 172.21.15.18:0/1417336491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf34100c10 msgr2=0x7fcf34197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3976e700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf34100c10 0x7fcf34197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3976e700 1 -- 172.21.15.18:0/1417336491 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c009d00 con 0x7fcf341027b0 2024-02-17T12:46:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf3976e700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf341027b0 0x7fcf3419f310 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400c900 tx=0x7fcf2400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:46:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.666+0000 7fcf31ffb700 1 -- 172.21.15.18:0/1417336491 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf240078d0 con 0x7fcf341027b0 2024-02-17T12:46:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.667+0000 7fcf31ffb700 1 -- 172.21.15.18:0/1417336491 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf2400f450 con 0x7fcf341027b0 2024-02-17T12:46:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.667+0000 7fcf31ffb700 1 -- 172.21.15.18:0/1417336491 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf24018660 con 0x7fcf341027b0 2024-02-17T12:46:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.667+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/1417336491 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3419fec0 con 0x7fcf341027b0 2024-02-17T12:46:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.667+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/1417336491 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf341a0280 con 0x7fcf341027b0 2024-02-17T12:46:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.668+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/1417336491 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3410c4e0 con 0x7fcf341027b0 2024-02-17T12:46:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.669+0000 7fcf31ffb700 1 -- 172.21.15.18:0/1417336491 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcf2400f5c0 con 0x7fcf341027b0 2024-02-17T12:46:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.669+0000 7fcf31ffb700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcf1c0a21d0 0x7fcf1c0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:54.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.669+0000 7fcf38f6d700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcf1c0a21d0 0x7fcf1c0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:54.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.669+0000 7fcf31ffb700 1 -- 172.21.15.18:0/1417336491 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fcf240cd490 con 0x7fcf341027b0 2024-02-17T12:46:54.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.670+0000 7fcf38f6d700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcf1c0a21d0 0x7fcf1c0a4690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009fd0 tx=0x7fcf2c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:46:54.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.674+0000 7fcf31ffb700 1 -- 172.21.15.18:0/1417336491 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf24096a20 con 0x7fcf341027b0 2024-02-17T12:46:54.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.879+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/1417336491 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fcf3410c7d0 con 0x7fcf341027b0 2024-02-17T12:46:54.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.879+0000 7fcf31ffb700 1 -- 172.21.15.18:0/1417336491 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7fcf2409a0e0 con 0x7fcf341027b0 2024-02-17T12:46:54.876 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:46:54.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.881+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/1417336491 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcf1c0a21d0 msgr2=0x7fcf1c0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:54.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.881+0000 7fcf3b1d1700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcf1c0a21d0 0x7fcf1c0a4690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009fd0 tx=0x7fcf2c00b040 comp rx=0 tx=0).stop 2024-02-17T12:46:54.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.881+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/1417336491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf341027b0 msgr2=0x7fcf3419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:54.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.881+0000 7fcf3b1d1700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf341027b0 0x7fcf3419f310 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400c900 tx=0x7fcf2400cc10 comp rx=0 tx=0).stop 2024-02-17T12:46:54.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.881+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/1417336491 shutdown_connections 2024-02-17T12:46:54.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.881+0000 7fcf3b1d1700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf341027b0 0x7fcf3419f310 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:54.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.881+0000 7fcf3b1d1700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf34100c10 0x7fcf34197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:54.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.881+0000 7fcf3b1d1700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcf1c0a21d0 0x7fcf1c0a4690 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:54.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.881+0000 7fcf3b1d1700 1 --2- 172.21.15.18:0/1417336491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf34101e10 0x7fcf341982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:54.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.881+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/1417336491 >> 172.21.15.18:0/1417336491 conn(0x7fcf340fc1d0 msgr2=0x7fcf340fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:46:54.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.882+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/1417336491 shutdown_connections 2024-02-17T12:46:54.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:54.882+0000 7fcf3b1d1700 1 -- 172.21.15.18:0/1417336491 wait complete. 2024-02-17T12:46:55.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:55 smithi018 ceph-mon[32625]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-02-17T12:46:55.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:55 smithi018 ceph-mon[32625]: osdmap e80: 12 total, 11 up, 12 in 2024-02-17T12:46:55.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:55 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:55.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:55 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:55.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:55 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1417336491' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-17T12:46:55.425 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1708173988,"num_in_osds":12,"osd_in_since":1708173995,"num_remapped_pgs":0} 2024-02-17T12:46:55.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:55 smithi146 ceph-mon[36454]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-02-17T12:46:55.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:55 smithi146 ceph-mon[36454]: osdmap e80: 12 total, 11 up, 12 in 2024-02-17T12:46:55.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:55 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:55.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:55 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:55.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:55 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1417336491' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-17T12:46:55.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:55 smithi119 ceph-mon[36188]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-02-17T12:46:55.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:55 smithi119 ceph-mon[36188]: osdmap e80: 12 total, 11 up, 12 in 2024-02-17T12:46:55.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:55 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:55.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:55 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:55.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:55 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1417336491' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-17T12:46:56.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:56 smithi146 ceph-mon[36454]: purged_snaps scrub starts 2024-02-17T12:46:56.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:56 smithi146 ceph-mon[36454]: purged_snaps scrub ok 2024-02-17T12:46:56.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:56 smithi146 ceph-mon[36454]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:56.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:56 smithi146 ceph-mon[36454]: osd.11 [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109] boot 2024-02-17T12:46:56.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:56 smithi146 ceph-mon[36454]: osdmap e81: 12 total, 12 up, 12 in 2024-02-17T12:46:56.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:56 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:56.426 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-17T12:46:56.456 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:56 smithi018 ceph-mon[32625]: purged_snaps scrub starts 2024-02-17T12:46:56.456 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:56 smithi018 ceph-mon[32625]: purged_snaps scrub ok 2024-02-17T12:46:56.456 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:56 smithi018 ceph-mon[32625]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:56.457 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:56 smithi018 ceph-mon[32625]: osd.11 [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109] boot 2024-02-17T12:46:56.457 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:56 smithi018 ceph-mon[32625]: osdmap e81: 12 total, 12 up, 12 in 2024-02-17T12:46:56.457 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:56 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:56.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:56 smithi119 ceph-mon[36188]: purged_snaps scrub starts 2024-02-17T12:46:56.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:56 smithi119 ceph-mon[36188]: purged_snaps scrub ok 2024-02-17T12:46:56.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:56 smithi119 ceph-mon[36188]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-02-17T12:46:56.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:56 smithi119 ceph-mon[36188]: osd.11 [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109] boot 2024-02-17T12:46:56.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:56 smithi119 ceph-mon[36188]: osdmap e81: 12 total, 12 up, 12 in 2024-02-17T12:46:56.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:56 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:46:56.676 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:46:57.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: osdmap e82: 12 total, 12 up, 12 in 2024-02-17T12:46:57.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:57.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:57.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:57.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:57.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:57.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:57.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:57.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:57.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:57.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:57 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.481 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: osdmap e82: 12 total, 12 up, 12 in 2024-02-17T12:46:57.481 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:57.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:57 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: osdmap e82: 12 total, 12 up, 12 in 2024-02-17T12:46:57.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:46:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:46:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:46:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:46:57.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:57 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:46:58.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.355+0000 7fec86478700 1 -- 172.21.15.18:0/913973231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec80101e10 msgr2=0x7fec80102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:58.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.355+0000 7fec86478700 1 --2- 172.21.15.18:0/913973231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec80101e10 0x7fec80102270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fec680099c0 tx=0x7fec68009cd0 comp rx=0 tx=0).stop 2024-02-17T12:46:58.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.355+0000 7fec86478700 1 -- 172.21.15.18:0/913973231 shutdown_connections 2024-02-17T12:46:58.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.355+0000 7fec86478700 1 --2- 172.21.15.18:0/913973231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec801027b0 0x7fec80109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:58.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.355+0000 7fec86478700 1 --2- 172.21.15.18:0/913973231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec80100c10 0x7fec80101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:58.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.355+0000 7fec86478700 1 --2- 172.21.15.18:0/913973231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec80101e10 0x7fec80102270 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:58.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.355+0000 7fec86478700 1 -- 172.21.15.18:0/913973231 >> 172.21.15.18:0/913973231 conn(0x7fec800fc1d0 msgr2=0x7fec800fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:46:58.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.355+0000 7fec86478700 1 -- 172.21.15.18:0/913973231 shutdown_connections 2024-02-17T12:46:58.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.355+0000 7fec86478700 1 -- 172.21.15.18:0/913973231 wait complete. 2024-02-17T12:46:58.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.355+0000 7fec86478700 1 Processor -- start 2024-02-17T12:46:58.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.355+0000 7fec86478700 1 -- start start 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec86478700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec80100c10 0x7fec80104ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec86478700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec80101e10 0x7fec801031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec86478700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec801027b0 0x7fec80103730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec86478700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec80103eb0 con 0x7fec80100c10 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec86478700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec80104020 con 0x7fec80101e10 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec86478700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec801a1c50 con 0x7fec801027b0 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec7ffff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec80100c10 0x7fec80104ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec7ffff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec80100c10 0x7fec80104ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50570/0 (socket says 172.21.15.18:50570) 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec7ffff700 1 -- 172.21.15.18:0/4004029108 learned_addr learned my addr 172.21.15.18:0/4004029108 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec7f7fe700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec80101e10 0x7fec801031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec7ffff700 1 -- 172.21.15.18:0/4004029108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec801027b0 msgr2=0x7fec80103730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec7ffff700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec801027b0 0x7fec80103730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec7ffff700 1 -- 172.21.15.18:0/4004029108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec80101e10 msgr2=0x7fec801031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec7ffff700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec80101e10 0x7fec801031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec7ffff700 1 -- 172.21.15.18:0/4004029108 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec680095f0 con 0x7fec80100c10 2024-02-17T12:46:58.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec7ffff700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec80100c10 0x7fec80104ba0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fec7000c990 tx=0x7fec7000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:46:58.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.356+0000 7fec7d7fa700 1 -- 172.21.15.18:0/4004029108 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70007b00 con 0x7fec80100c10 2024-02-17T12:46:58.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.357+0000 7fec7d7fa700 1 -- 172.21.15.18:0/4004029108 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec70007c60 con 0x7fec80100c10 2024-02-17T12:46:58.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.357+0000 7fec86478700 1 -- 172.21.15.18:0/4004029108 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec801a1f30 con 0x7fec80100c10 2024-02-17T12:46:58.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.357+0000 7fec7d7fa700 1 -- 172.21.15.18:0/4004029108 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70018660 con 0x7fec80100c10 2024-02-17T12:46:58.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.357+0000 7fec86478700 1 -- 172.21.15.18:0/4004029108 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec801a2340 con 0x7fec80100c10 2024-02-17T12:46:58.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.358+0000 7fec86478700 1 -- 172.21.15.18:0/4004029108 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8010c510 con 0x7fec80100c10 2024-02-17T12:46:58.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.359+0000 7fec7d7fa700 1 -- 172.21.15.18:0/4004029108 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fec70004020 con 0x7fec80100c10 2024-02-17T12:46:58.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.359+0000 7fec7d7fa700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fec6c0a2150 0x7fec6c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:46:58.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.359+0000 7fec7f7fe700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fec6c0a2150 0x7fec6c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:46:58.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.359+0000 7fec7d7fa700 1 -- 172.21.15.18:0/4004029108 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fec700cd860 con 0x7fec80100c10 2024-02-17T12:46:58.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.360+0000 7fec7f7fe700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fec6c0a2150 0x7fec6c0a4610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec68009990 tx=0x7fec680052c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:46:58.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.363+0000 7fec7d7fa700 1 -- 172.21.15.18:0/4004029108 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fec70096d60 con 0x7fec80100c10 2024-02-17T12:46:58.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:58 smithi146 ceph-mon[36454]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:46:58.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:58 smithi146 ceph-mon[36454]: Detected new or changed devices on smithi146 2024-02-17T12:46:58.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:58 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi146 to 4270M 2024-02-17T12:46:58.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:58 smithi018 ceph-mon[32625]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:46:58.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:58 smithi018 ceph-mon[32625]: Detected new or changed devices on smithi146 2024-02-17T12:46:58.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:58 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi146 to 4270M 2024-02-17T12:46:58.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.563+0000 7fec86478700 1 -- 172.21.15.18:0/4004029108 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fec8010c6d0 con 0x7fec80100c10 2024-02-17T12:46:58.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.563+0000 7fec7d7fa700 1 -- 172.21.15.18:0/4004029108 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) v1 ==== 74+0+133 (secure 0 0 0) 0x7fec7009a420 con 0x7fec80100c10 2024-02-17T12:46:58.557 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:46:58.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.565+0000 7fec86478700 1 -- 172.21.15.18:0/4004029108 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fec6c0a2150 msgr2=0x7fec6c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:58.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.565+0000 7fec86478700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fec6c0a2150 0x7fec6c0a4610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec68009990 tx=0x7fec680052c0 comp rx=0 tx=0).stop 2024-02-17T12:46:58.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.565+0000 7fec86478700 1 -- 172.21.15.18:0/4004029108 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec80100c10 msgr2=0x7fec80104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:46:58.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.565+0000 7fec86478700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec80100c10 0x7fec80104ba0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fec7000c990 tx=0x7fec7000cca0 comp rx=0 tx=0).stop 2024-02-17T12:46:58.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.565+0000 7fec86478700 1 -- 172.21.15.18:0/4004029108 shutdown_connections 2024-02-17T12:46:58.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.565+0000 7fec86478700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fec6c0a2150 0x7fec6c0a4610 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:58.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.565+0000 7fec86478700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec801027b0 0x7fec80103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:58.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.565+0000 7fec86478700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec80100c10 0x7fec80104ba0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:58.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.565+0000 7fec86478700 1 --2- 172.21.15.18:0/4004029108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec80101e10 0x7fec801031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:46:58.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.566+0000 7fec86478700 1 -- 172.21.15.18:0/4004029108 >> 172.21.15.18:0/4004029108 conn(0x7fec800fc1d0 msgr2=0x7fec800fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:46:58.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.566+0000 7fec86478700 1 -- 172.21.15.18:0/4004029108 shutdown_connections 2024-02-17T12:46:58.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:46:58.566+0000 7fec86478700 1 -- 172.21.15.18:0/4004029108 wait complete. 2024-02-17T12:46:58.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:58 smithi119 ceph-mon[36188]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:46:58.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:58 smithi119 ceph-mon[36188]: Detected new or changed devices on smithi146 2024-02-17T12:46:58.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:58 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi146 to 4270M 2024-02-17T12:46:59.188 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1708174015,"num_in_osds":12,"osd_in_since":1708173995,"num_remapped_pgs":0} 2024-02-17T12:46:59.189 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-17T12:46:59.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:46:59 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/4004029108' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-17T12:46:59.474 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:46:59.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:46:59 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/4004029108' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-17T12:46:59.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:46:59 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/4004029108' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-17T12:47:00.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:00 smithi146 ceph-mon[36454]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:00.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:00 smithi018 ceph-mon[32625]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:00.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:00 smithi119 ceph-mon[36188]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:01.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.175+0000 7f46aab9d700 1 -- 172.21.15.18:0/762566719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a4101e30 msgr2=0x7f46a4102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:01.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.175+0000 7f46aab9d700 1 --2- 172.21.15.18:0/762566719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a4101e30 0x7f46a4102210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f469c009990 tx=0x7f469c009ca0 comp rx=0 tx=0).stop 2024-02-17T12:47:01.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.175+0000 7f46aab9d700 1 -- 172.21.15.18:0/762566719 shutdown_connections 2024-02-17T12:47:01.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.175+0000 7f46aab9d700 1 --2- 172.21.15.18:0/762566719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46a4106f10 0x7f46a410eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:01.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.175+0000 7f46aab9d700 1 --2- 172.21.15.18:0/762566719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a4101e30 0x7f46a4102210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:01.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.175+0000 7f46aab9d700 1 --2- 172.21.15.18:0/762566719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46a41027e0 0x7f46a4106830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:01.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.175+0000 7f46aab9d700 1 -- 172.21.15.18:0/762566719 >> 172.21.15.18:0/762566719 conn(0x7f46a40fd720 msgr2=0x7f46a40ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:01.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.175+0000 7f46aab9d700 1 -- 172.21.15.18:0/762566719 shutdown_connections 2024-02-17T12:47:01.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.175+0000 7f46aab9d700 1 -- 172.21.15.18:0/762566719 wait complete. 2024-02-17T12:47:01.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.176+0000 7f46aab9d700 1 Processor -- start 2024-02-17T12:47:01.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.176+0000 7f46aab9d700 1 -- start start 2024-02-17T12:47:01.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.176+0000 7f46aab9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46a4101e30 0x7f46a4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:01.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.176+0000 7f46aab9d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a41027e0 0x7f46a4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.176+0000 7f46aab9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46a4106f10 0x7f46a419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.176+0000 7f46aab9d700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a419cfc0 con 0x7f46a41027e0 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.176+0000 7f46aab9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a419d130 con 0x7f46a4101e30 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.176+0000 7f46aab9d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a419d2f0 con 0x7f46a4106f10 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.176+0000 7f46a939a700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a41027e0 0x7f46a4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.176+0000 7f46a939a700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a41027e0 0x7f46a4198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43336/0 (socket says 172.21.15.18:43336) 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.176+0000 7f46a939a700 1 -- 172.21.15.18:0/200888129 learned_addr learned my addr 172.21.15.18:0/200888129 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.176+0000 7f46a9b9b700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46a4101e30 0x7f46a4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.177+0000 7f46aa39c700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46a4106f10 0x7f46a419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.177+0000 7f46a939a700 1 -- 172.21.15.18:0/200888129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46a4106f10 msgr2=0x7f46a419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.177+0000 7f46a939a700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46a4106f10 0x7f46a419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.177+0000 7f46a939a700 1 -- 172.21.15.18:0/200888129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46a4101e30 msgr2=0x7f46a4198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:01.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.177+0000 7f46a939a700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46a4101e30 0x7f46a4198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:01.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.177+0000 7f46a939a700 1 -- 172.21.15.18:0/200888129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c0095f0 con 0x7f46a41027e0 2024-02-17T12:47:01.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.177+0000 7f46a939a700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a41027e0 0x7f46a4198670 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f469800e970 tx=0x7f469800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:01.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.177+0000 7f46a2ffd700 1 -- 172.21.15.18:0/200888129 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698009790 con 0x7f46a41027e0 2024-02-17T12:47:01.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.177+0000 7f46aab9d700 1 -- 172.21.15.18:0/200888129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a419d5d0 con 0x7f46a41027e0 2024-02-17T12:47:01.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.177+0000 7f46a2ffd700 1 -- 172.21.15.18:0/200888129 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f469800eec0 con 0x7f46a41027e0 2024-02-17T12:47:01.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.177+0000 7f46a2ffd700 1 -- 172.21.15.18:0/200888129 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698005690 con 0x7f46a41027e0 2024-02-17T12:47:01.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.177+0000 7f46aab9d700 1 -- 172.21.15.18:0/200888129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a41a4e30 con 0x7f46a41027e0 2024-02-17T12:47:01.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.178+0000 7f46aab9d700 1 -- 172.21.15.18:0/200888129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a410a1d0 con 0x7f46a41027e0 2024-02-17T12:47:01.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.179+0000 7f46a2ffd700 1 -- 172.21.15.18:0/200888129 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f469800cbd0 con 0x7f46a41027e0 2024-02-17T12:47:01.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.180+0000 7f46a2ffd700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f468c0a2150 0x7f468c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:01.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.180+0000 7f46a9b9b700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f468c0a2150 0x7f468c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:01.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.180+0000 7f46a2ffd700 1 -- 172.21.15.18:0/200888129 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4698014070 con 0x7f46a41027e0 2024-02-17T12:47:01.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.181+0000 7f46a9b9b700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f468c0a2150 0x7f468c0a4610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f469c00b2e0 tx=0x7f469c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:01.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.184+0000 7f46a2ffd700 1 -- 172.21.15.18:0/200888129 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4698097160 con 0x7f46a41027e0 2024-02-17T12:47:01.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:01 smithi018 ceph-mon[32625]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:01.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.384+0000 7f46aab9d700 1 -- 172.21.15.18:0/200888129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46a4199440 con 0x7f46a41027e0 2024-02-17T12:47:01.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.386+0000 7f46a2ffd700 1 -- 172.21.15.18:0/200888129 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18458 (secure 0 0 0) 0x7f469809a820 con 0x7f46a41027e0 2024-02-17T12:47:01.379 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:01.379 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":82,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:46:56.115429+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:47:01.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.388+0000 7f46aab9d700 1 -- 172.21.15.18:0/200888129 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f468c0a2150 msgr2=0x7f468c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:01.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.388+0000 7f46aab9d700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f468c0a2150 0x7f468c0a4610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f469c00b2e0 tx=0x7f469c005960 comp rx=0 tx=0).stop 2024-02-17T12:47:01.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.388+0000 7f46aab9d700 1 -- 172.21.15.18:0/200888129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a41027e0 msgr2=0x7f46a4198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:01.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.388+0000 7f46aab9d700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a41027e0 0x7f46a4198670 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f469800e970 tx=0x7f469800ec80 comp rx=0 tx=0).stop 2024-02-17T12:47:01.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.388+0000 7f46aab9d700 1 -- 172.21.15.18:0/200888129 shutdown_connections 2024-02-17T12:47:01.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.388+0000 7f46aab9d700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f468c0a2150 0x7f468c0a4610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:01.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.388+0000 7f46aab9d700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46a4106f10 0x7f46a419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:01.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.388+0000 7f46aab9d700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46a4101e30 0x7f46a4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:01.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.388+0000 7f46aab9d700 1 --2- 172.21.15.18:0/200888129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a41027e0 0x7f46a4198670 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:01.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.388+0000 7f46aab9d700 1 -- 172.21.15.18:0/200888129 >> 172.21.15.18:0/200888129 conn(0x7f46a40fd720 msgr2=0x7f46a40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:01.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.388+0000 7f46aab9d700 1 -- 172.21.15.18:0/200888129 shutdown_connections 2024-02-17T12:47:01.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:01.388+0000 7f46aab9d700 1 -- 172.21.15.18:0/200888129 wait complete. 2024-02-17T12:47:01.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:01 smithi146 ceph-mon[36454]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:01.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:01 smithi119 ceph-mon[36188]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:02.017 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T12:43:12.450140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T12:47:02.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-17T12:47:02.259 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:47:02.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:02 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/200888129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:02.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:02 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/200888129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:02.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:02 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/200888129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:03.421 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:03 smithi018 ceph-mon[32625]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:03.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:03 smithi146 ceph-mon[36454]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:03.496 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:03 smithi119 ceph-mon[36188]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:03.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.972+0000 7fe96d9b5700 1 -- 172.21.15.18:0/4123037279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe968107cf0 msgr2=0x7fe9681080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:03.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.972+0000 7fe96d9b5700 1 --2- 172.21.15.18:0/4123037279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe968107cf0 0x7fe9681080d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe9600099c0 tx=0x7fe960009cd0 comp rx=0 tx=0).stop 2024-02-17T12:47:03.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.972+0000 7fe96d9b5700 1 -- 172.21.15.18:0/4123037279 shutdown_connections 2024-02-17T12:47:03.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.972+0000 7fe96d9b5700 1 --2- 172.21.15.18:0/4123037279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe96806ad10 0x7fe96810eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:03.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.972+0000 7fe96d9b5700 1 --2- 172.21.15.18:0/4123037279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe968107cf0 0x7fe9681080d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:03.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.972+0000 7fe96d9b5700 1 --2- 172.21.15.18:0/4123037279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe96806a370 0x7fe96806a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:03.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.972+0000 7fe96d9b5700 1 -- 172.21.15.18:0/4123037279 >> 172.21.15.18:0/4123037279 conn(0x7fe968076b90 msgr2=0x7fe968076fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:03.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.972+0000 7fe96d9b5700 1 -- 172.21.15.18:0/4123037279 shutdown_connections 2024-02-17T12:47:03.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.972+0000 7fe96d9b5700 1 -- 172.21.15.18:0/4123037279 wait complete. 2024-02-17T12:47:03.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.972+0000 7fe96d9b5700 1 Processor -- start 2024-02-17T12:47:03.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe96d9b5700 1 -- start start 2024-02-17T12:47:03.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe96d9b5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe96806a370 0x7fe968198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:03.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe96d9b5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe96806ad10 0x7fe968198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:03.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe96d9b5700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe968107cf0 0x7fe96819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:03.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe96d9b5700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96819d010 con 0x7fe968107cf0 2024-02-17T12:47:03.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe96d9b5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96819d180 con 0x7fe96806ad10 2024-02-17T12:47:03.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe96d9b5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96819d340 con 0x7fe96806a370 2024-02-17T12:47:03.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe9677fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe968107cf0 0x7fe96819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe9677fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe968107cf0 0x7fe96819ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43352/0 (socket says 172.21.15.18:43352) 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe9677fe700 1 -- 172.21.15.18:0/3384617903 learned_addr learned my addr 172.21.15.18:0/3384617903 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe966ffd700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe96806a370 0x7fe968198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe9667fc700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe96806ad10 0x7fe968198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe9677fe700 1 -- 172.21.15.18:0/3384617903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe96806a370 msgr2=0x7fe968198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe9677fe700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe96806a370 0x7fe968198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe9677fe700 1 -- 172.21.15.18:0/3384617903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe96806ad10 msgr2=0x7fe968198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe9677fe700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe96806ad10 0x7fe968198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.973+0000 7fe9677fe700 1 -- 172.21.15.18:0/3384617903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600095f0 con 0x7fe968107cf0 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.974+0000 7fe9677fe700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe968107cf0 0x7fe96819ca00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe95800ef30 tx=0x7fe95800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.974+0000 7fe96c9b3700 1 -- 172.21.15.18:0/3384617903 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958018070 con 0x7fe968107cf0 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.974+0000 7fe96d9b5700 1 -- 172.21.15.18:0/3384617903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96819d5c0 con 0x7fe968107cf0 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.974+0000 7fe96c9b3700 1 -- 172.21.15.18:0/3384617903 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe95800f040 con 0x7fe968107cf0 2024-02-17T12:47:03.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.974+0000 7fe96c9b3700 1 -- 172.21.15.18:0/3384617903 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95800cd50 con 0x7fe968107cf0 2024-02-17T12:47:03.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.974+0000 7fe96d9b5700 1 -- 172.21.15.18:0/3384617903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9681a4e80 con 0x7fe968107cf0 2024-02-17T12:47:03.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.975+0000 7fe96d9b5700 1 -- 172.21.15.18:0/3384617903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96810a210 con 0x7fe968107cf0 2024-02-17T12:47:03.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.976+0000 7fe96c9b3700 1 -- 172.21.15.18:0/3384617903 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe958007500 con 0x7fe968107cf0 2024-02-17T12:47:03.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.976+0000 7fe96c9b3700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe9540a2150 0x7fe9540a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:03.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.976+0000 7fe966ffd700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe9540a2150 0x7fe9540a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:03.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.977+0000 7fe96c9b3700 1 -- 172.21.15.18:0/3384617903 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe9580cd380 con 0x7fe968107cf0 2024-02-17T12:47:03.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.977+0000 7fe966ffd700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe9540a2150 0x7fe9540a4610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe960000c00 tx=0x7fe960005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:03.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:03.981+0000 7fe96c9b3700 1 -- 172.21.15.18:0/3384617903 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe9580967e0 con 0x7fe968107cf0 2024-02-17T12:47:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.181+0000 7fe96d9b5700 1 -- 172.21.15.18:0/3384617903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe968199490 con 0x7fe968107cf0 2024-02-17T12:47:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.182+0000 7fe96c9b3700 1 -- 172.21.15.18:0/3384617903 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe958099ea0 con 0x7fe968107cf0 2024-02-17T12:47:04.175 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-02-17T12:47:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.184+0000 7fe96d9b5700 1 -- 172.21.15.18:0/3384617903 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe9540a2150 msgr2=0x7fe9540a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.184+0000 7fe96d9b5700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe9540a2150 0x7fe9540a4610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe960000c00 tx=0x7fe960005fb0 comp rx=0 tx=0).stop 2024-02-17T12:47:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.184+0000 7fe96d9b5700 1 -- 172.21.15.18:0/3384617903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe968107cf0 msgr2=0x7fe96819ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.184+0000 7fe96d9b5700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe968107cf0 0x7fe96819ca00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe95800ef30 tx=0x7fe95800c3a0 comp rx=0 tx=0).stop 2024-02-17T12:47:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.184+0000 7fe96d9b5700 1 -- 172.21.15.18:0/3384617903 shutdown_connections 2024-02-17T12:47:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.184+0000 7fe96d9b5700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe968107cf0 0x7fe96819ca00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.184+0000 7fe96d9b5700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe9540a2150 0x7fe9540a4610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.184+0000 7fe96d9b5700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe96806a370 0x7fe968198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.184+0000 7fe96d9b5700 1 --2- 172.21.15.18:0/3384617903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe96806ad10 0x7fe968198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.184+0000 7fe96d9b5700 1 -- 172.21.15.18:0/3384617903 >> 172.21.15.18:0/3384617903 conn(0x7fe968076b90 msgr2=0x7fe9680ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.184+0000 7fe96d9b5700 1 -- 172.21.15.18:0/3384617903 shutdown_connections 2024-02-17T12:47:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:04.184+0000 7fe96d9b5700 1 -- 172.21.15.18:0/3384617903 wait complete. 2024-02-17T12:47:04.469 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:04 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3384617903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:47:04.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:04 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3384617903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:47:04.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:04 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3384617903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:47:04.792 INFO:tasks.cephadm:Adding mds.a on smithi018 2024-02-17T12:47:04.792 INFO:tasks.cephadm:Adding mds.d on smithi018 2024-02-17T12:47:04.792 INFO:tasks.cephadm:Adding mds.g on smithi018 2024-02-17T12:47:04.792 INFO:tasks.cephadm:Adding mds.j on smithi018 2024-02-17T12:47:04.792 INFO:tasks.cephadm:Adding mds.b on smithi119 2024-02-17T12:47:04.792 INFO:tasks.cephadm:Adding mds.e on smithi119 2024-02-17T12:47:04.792 INFO:tasks.cephadm:Adding mds.h on smithi119 2024-02-17T12:47:04.792 INFO:tasks.cephadm:Adding mds.k on smithi119 2024-02-17T12:47:04.793 INFO:tasks.cephadm:Adding mds.c on smithi146 2024-02-17T12:47:04.793 INFO:tasks.cephadm:Adding mds.f on smithi146 2024-02-17T12:47:04.793 INFO:tasks.cephadm:Adding mds.i on smithi146 2024-02-17T12:47:04.793 INFO:tasks.cephadm:Adding mds.l on smithi146 2024-02-17T12:47:04.793 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph orch apply mds all '12;smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l' 2024-02-17T12:47:05.045 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:47:05.425 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:05 smithi146 ceph-mon[36454]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:05.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:05 smithi018 ceph-mon[32625]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:05.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:05 smithi119 ceph-mon[36188]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:06.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.770+0000 7f3b92400700 1 -- 172.21.15.146:0/1316443636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b8c100ba0 msgr2=0x7f3b8c100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:06.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.770+0000 7f3b92400700 1 --2- 172.21.15.146:0/1316443636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b8c100ba0 0x7f3b8c100fc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009990 tx=0x7f3b7c009ca0 comp rx=0 tx=0).stop 2024-02-17T12:47:06.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.770+0000 7f3b92400700 1 -- 172.21.15.146:0/1316443636 shutdown_connections 2024-02-17T12:47:06.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.770+0000 7f3b92400700 1 --2- 172.21.15.146:0/1316443636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c102740 0x7f3b8c108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:06.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.770+0000 7f3b92400700 1 --2- 172.21.15.146:0/1316443636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b8c100ba0 0x7f3b8c100fc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:06.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.770+0000 7f3b92400700 1 --2- 172.21.15.146:0/1316443636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b8c101da0 0x7f3b8c102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:06.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.770+0000 7f3b92400700 1 -- 172.21.15.146:0/1316443636 >> 172.21.15.146:0/1316443636 conn(0x7f3b8c0fc140 msgr2=0x7f3b8c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:06.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.770+0000 7f3b92400700 1 -- 172.21.15.146:0/1316443636 shutdown_connections 2024-02-17T12:47:06.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.770+0000 7f3b92400700 1 -- 172.21.15.146:0/1316443636 wait complete. 2024-02-17T12:47:06.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.771+0000 7f3b92400700 1 Processor -- start 2024-02-17T12:47:06.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.771+0000 7f3b92400700 1 -- start start 2024-02-17T12:47:06.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.771+0000 7f3b92400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b8c100ba0 0x7f3b8c197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:06.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.771+0000 7f3b92400700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b8c101da0 0x7f3b8c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:06.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.771+0000 7f3b92400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c102740 0x7f3b8c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:06.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.771+0000 7f3b92400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c19f790 con 0x7f3b8c100ba0 2024-02-17T12:47:06.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.771+0000 7f3b92400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c19f8d0 con 0x7f3b8c102740 2024-02-17T12:47:06.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.771+0000 7f3b92400700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c19fa90 con 0x7f3b8c101da0 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.771+0000 7f3b8b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b8c101da0 0x7f3b8c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.771+0000 7f3b8b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b8c101da0 0x7f3b8c1981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37430/0 (socket says 172.21.15.146:37430) 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.771+0000 7f3b8bfff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b8c100ba0 0x7f3b8c197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.771+0000 7f3b8b7fe700 1 -- 172.21.15.146:0/4219801175 learned_addr learned my addr 172.21.15.146:0/4219801175 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b9099d700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c102740 0x7f3b8c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b8b7fe700 1 -- 172.21.15.146:0/4219801175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c102740 msgr2=0x7f3b8c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b8b7fe700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c102740 0x7f3b8c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b8b7fe700 1 -- 172.21.15.146:0/4219801175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b8c100ba0 msgr2=0x7f3b8c197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b8b7fe700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b8c100ba0 0x7f3b8c197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b8b7fe700 1 -- 172.21.15.146:0/4219801175 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c0095f0 con 0x7f3b8c101da0 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b9099d700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c102740 0x7f3b8c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b8bfff700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b8c100ba0 0x7f3b8c197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:47:06.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b8b7fe700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b8c101da0 0x7f3b8c1981d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000c9b0 tx=0x7f3b8000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:06.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b897fa700 1 -- 172.21.15.146:0/4219801175 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80007b00 con 0x7f3b8c101da0 2024-02-17T12:47:06.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b897fa700 1 -- 172.21.15.146:0/4219801175 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b80007c60 con 0x7f3b8c101da0 2024-02-17T12:47:06.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b92400700 1 -- 172.21.15.146:0/4219801175 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8c19fd70 con 0x7f3b8c101da0 2024-02-17T12:47:06.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b897fa700 1 -- 172.21.15.146:0/4219801175 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80018660 con 0x7f3b8c101da0 2024-02-17T12:47:06.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.772+0000 7f3b92400700 1 -- 172.21.15.146:0/4219801175 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8c1a01c0 con 0x7f3b8c101da0 2024-02-17T12:47:06.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.773+0000 7f3b92400700 1 -- 172.21.15.146:0/4219801175 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8c10c4e0 con 0x7f3b8c101da0 2024-02-17T12:47:06.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.774+0000 7f3b897fa700 1 -- 172.21.15.146:0/4219801175 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3b8000f450 con 0x7f3b8c101da0 2024-02-17T12:47:06.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.775+0000 7f3b897fa700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f3b740a2150 0x7f3b740a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:06.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.775+0000 7f3b897fa700 1 -- 172.21.15.146:0/4219801175 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3b800cda70 con 0x7f3b8c101da0 2024-02-17T12:47:06.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.775+0000 7f3b8bfff700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f3b740a2150 0x7f3b740a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:06.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.776+0000 7f3b8bfff700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f3b740a2150 0x7f3b740a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c199120 tx=0x7f3b7c0038d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:06.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.779+0000 7f3b897fa700 1 -- 172.21.15.146:0/4219801175 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3b80096f70 con 0x7f3b8c101da0 2024-02-17T12:47:06.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.980+0000 7f3b92400700 1 -- 172.21.15.146:0/4219801175 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b8c0fe2f0 con 0x7f3b740a2150 2024-02-17T12:47:06.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.988+0000 7f3b897fa700 1 -- 172.21.15.146:0/4219801175 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f3b8c0fe2f0 con 0x7f3b740a2150 2024-02-17T12:47:06.982 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled mds.all update... 2024-02-17T12:47:06.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.990+0000 7f3b92400700 1 -- 172.21.15.146:0/4219801175 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f3b740a2150 msgr2=0x7f3b740a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:06.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.990+0000 7f3b92400700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f3b740a2150 0x7f3b740a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c199120 tx=0x7f3b7c0038d0 comp rx=0 tx=0).stop 2024-02-17T12:47:06.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.990+0000 7f3b92400700 1 -- 172.21.15.146:0/4219801175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b8c101da0 msgr2=0x7f3b8c1981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:06.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.990+0000 7f3b92400700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b8c101da0 0x7f3b8c1981d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000c9b0 tx=0x7f3b8000ccc0 comp rx=0 tx=0).stop 2024-02-17T12:47:06.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.991+0000 7f3b92400700 1 -- 172.21.15.146:0/4219801175 shutdown_connections 2024-02-17T12:47:06.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.991+0000 7f3b92400700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c102740 0x7f3b8c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:06.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.991+0000 7f3b92400700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b8c100ba0 0x7f3b8c197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:06.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.991+0000 7f3b92400700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f3b740a2150 0x7f3b740a4610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:06.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.991+0000 7f3b92400700 1 --2- 172.21.15.146:0/4219801175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b8c101da0 0x7f3b8c1981d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:06.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.991+0000 7f3b92400700 1 -- 172.21.15.146:0/4219801175 >> 172.21.15.146:0/4219801175 conn(0x7f3b8c0fc140 msgr2=0x7f3b8c0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:06.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.991+0000 7f3b92400700 1 -- 172.21.15.146:0/4219801175 shutdown_connections 2024-02-17T12:47:06.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-02-17T12:47:06.991+0000 7f3b92400700 1 -- 172.21.15.146:0/4219801175 wait complete. 2024-02-17T12:47:07.690 DEBUG:teuthology.orchestra.run.smithi018:mds.a> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.a.service 2024-02-17T12:47:07.693 DEBUG:teuthology.orchestra.run.smithi018:mds.d> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.d.service 2024-02-17T12:47:07.736 DEBUG:teuthology.orchestra.run.smithi018:mds.g> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.g.service 2024-02-17T12:47:07.779 DEBUG:teuthology.orchestra.run.smithi018:mds.j> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.j.service 2024-02-17T12:47:07.822 DEBUG:teuthology.orchestra.run.smithi119:mds.b> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.b.service 2024-02-17T12:47:07.824 DEBUG:teuthology.orchestra.run.smithi119:mds.e> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.e.service 2024-02-17T12:47:07.868 DEBUG:teuthology.orchestra.run.smithi119:mds.h> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.h.service 2024-02-17T12:47:07.911 DEBUG:teuthology.orchestra.run.smithi119:mds.k> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.k.service 2024-02-17T12:47:07.954 DEBUG:teuthology.orchestra.run.smithi146:mds.c> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.c.service 2024-02-17T12:47:07.956 DEBUG:teuthology.orchestra.run.smithi146:mds.f> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.f.service 2024-02-17T12:47:07.999 DEBUG:teuthology.orchestra.run.smithi146:mds.i> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.i.service 2024-02-17T12:47:08.042 DEBUG:teuthology.orchestra.run.smithi146:mds.l> sudo journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.l.service 2024-02-17T12:47:08.085 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-02-17T12:47:08.086 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:47:08.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.216+0000 7f055733e640 1 -- 172.21.15.18:0/20532769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05501016b0 msgr2=0x7f0550101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.216+0000 7f055733e640 1 --2- 172.21.15.18:0/20532769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05501016b0 0x7f0550101ad0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f054c009900 tx=0x7f054c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:08.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.216+0000 7f055733e640 1 -- 172.21.15.18:0/20532769 shutdown_connections 2024-02-17T12:47:08.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.216+0000 7f055733e640 1 --2- 172.21.15.18:0/20532769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0550100d10 0x7f05501085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.216+0000 7f055733e640 1 --2- 172.21.15.18:0/20532769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0550100350 0x7f05501007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.216+0000 7f055733e640 1 --2- 172.21.15.18:0/20532769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05501016b0 0x7f0550101ad0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.216+0000 7f055733e640 1 -- 172.21.15.18:0/20532769 >> 172.21.15.18:0/20532769 conn(0x7f05500fada0 msgr2=0x7f05500fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:08.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.216+0000 7f055733e640 1 -- 172.21.15.18:0/20532769 shutdown_connections 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.216+0000 7f055733e640 1 -- 172.21.15.18:0/20532769 wait complete. 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f055733e640 1 Processor -- start 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f055733e640 1 -- start start 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f055733e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0550100350 0x7f05500ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f055733e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0550100d10 0x7f055019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f055733e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05501016b0 0x7f055019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f055733e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055019ed80 con 0x7f05501016b0 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f055733e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055019eef0 con 0x7f0550100350 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f055733e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055019f0b0 con 0x7f0550100d10 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f05558db640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05501016b0 0x7f055019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f05558db640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05501016b0 0x7f055019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43366/0 (socket says 172.21.15.18:43366) 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f05548d9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0550100d10 0x7f055019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f05558db640 1 -- 172.21.15.18:0/4251476391 learned_addr learned my addr 172.21.15.18:0/4251476391 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.217+0000 7f05550da640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0550100350 0x7f05500ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.218+0000 7f05558db640 1 -- 172.21.15.18:0/4251476391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0550100d10 msgr2=0x7f055019bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.218+0000 7f05558db640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0550100d10 0x7f055019bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.218+0000 7f05558db640 1 -- 172.21.15.18:0/4251476391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0550100350 msgr2=0x7f05500ffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.218+0000 7f05558db640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0550100350 0x7f05500ffdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.218+0000 7f05558db640 1 -- 172.21.15.18:0/4251476391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c009560 con 0x7f05501016b0 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.218+0000 7f05548d9640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0550100d10 0x7f055019bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.218+0000 7f05558db640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05501016b0 0x7f055019e5a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f054800eb10 tx=0x7f054800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.218+0000 7f05467fc640 1 -- 172.21.15.18:0/4251476391 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548004b20 con 0x7f05501016b0 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.218+0000 7f055733e640 1 -- 172.21.15.18:0/4251476391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055019f330 con 0x7f05501016b0 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.218+0000 7f05467fc640 1 -- 172.21.15.18:0/4251476391 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0548004c80 con 0x7f05501016b0 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.218+0000 7f05467fc640 1 -- 172.21.15.18:0/4251476391 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548010650 con 0x7f05501016b0 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.218+0000 7f055733e640 1 -- 172.21.15.18:0/4251476391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055019f840 con 0x7f05501016b0 2024-02-17T12:47:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.219+0000 7f055733e640 1 -- 172.21.15.18:0/4251476391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0550107510 con 0x7f05501016b0 2024-02-17T12:47:08.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.220+0000 7f05467fc640 1 -- 172.21.15.18:0/4251476391 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0548005020 con 0x7f05501016b0 2024-02-17T12:47:08.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.220+0000 7f05467fc640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f053c0a20c0 0x7f053c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:08.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.221+0000 7f05550da640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f053c0a20c0 0x7f053c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:08.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.221+0000 7f05467fc640 1 -- 172.21.15.18:0/4251476391 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0548014070 con 0x7f05501016b0 2024-02-17T12:47:08.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.221+0000 7f05550da640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f053c0a20c0 0x7f053c0a4580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f054c005ab0 tx=0x7f054c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:08.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.225+0000 7f05467fc640 1 -- 172.21.15.18:0/4251476391 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0548097190 con 0x7f05501016b0 2024-02-17T12:47:08.274 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:07 smithi146 ceph-mon[36454]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:08.275 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:07 smithi146 ceph-mon[36454]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:08.275 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:07 smithi146 ceph-mon[36454]: Saving service mds.all spec with placement smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l;count:12 2024-02-17T12:47:08.275 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:07 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:08.275 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:07 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:47:08.275 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:07 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:08.275 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:07 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:47:08.275 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:07 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:08.275 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:07 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:08.275 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:07 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:08.275 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:07 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:07 smithi018 ceph-mon[32625]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:07 smithi018 ceph-mon[32625]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:07 smithi018 ceph-mon[32625]: Saving service mds.all spec with placement smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l;count:12 2024-02-17T12:47:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:07 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:07 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:47:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:07 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:07 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:47:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:07 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:07 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:07 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:08.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:07 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:08.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:07 smithi119 ceph-mon[36188]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:08.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:07 smithi119 ceph-mon[36188]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:08.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:07 smithi119 ceph-mon[36188]: Saving service mds.all spec with placement smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l;count:12 2024-02-17T12:47:08.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:07 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:08.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:07 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:47:08.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:07 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:08.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:07 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:47:08.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:07 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:08.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:07 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:08.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:07 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:08.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:07 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:08.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.407+0000 7f055733e640 1 -- 172.21.15.18:0/4251476391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f055010bcf0 con 0x7f05501016b0 2024-02-17T12:47:08.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.408+0000 7f05467fc640 1 -- 172.21.15.18:0/4251476391 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18458 (secure 0 0 0) 0x7f054809a850 con 0x7f05501016b0 2024-02-17T12:47:08.402 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:08.402 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":82,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:46:56.115429+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:47:08.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.410+0000 7f055733e640 1 -- 172.21.15.18:0/4251476391 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f053c0a20c0 msgr2=0x7f053c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.410+0000 7f055733e640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f053c0a20c0 0x7f053c0a4580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f054c005ab0 tx=0x7f054c005fd0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.410+0000 7f055733e640 1 -- 172.21.15.18:0/4251476391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05501016b0 msgr2=0x7f055019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.410+0000 7f055733e640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05501016b0 0x7f055019e5a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f054800eb10 tx=0x7f054800c2d0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.410+0000 7f055733e640 1 -- 172.21.15.18:0/4251476391 shutdown_connections 2024-02-17T12:47:08.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.410+0000 7f055733e640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f053c0a20c0 0x7f053c0a4580 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.410+0000 7f055733e640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05501016b0 0x7f055019e5a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.410+0000 7f055733e640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0550100d10 0x7f055019bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.410+0000 7f055733e640 1 --2- 172.21.15.18:0/4251476391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0550100350 0x7f05500ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.410+0000 7f055733e640 1 -- 172.21.15.18:0/4251476391 >> 172.21.15.18:0/4251476391 conn(0x7f05500fada0 msgr2=0x7f05501046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:08.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.410+0000 7f055733e640 1 -- 172.21.15.18:0/4251476391 shutdown_connections 2024-02-17T12:47:08.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.410+0000 7f055733e640 1 -- 172.21.15.18:0/4251476391 wait complete. 2024-02-17T12:47:08.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T12:43:12.450140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T12:47:08.415 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T12:47:08.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.528+0000 7fad42de2640 1 -- 172.21.15.18:0/864807495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad3c0ff450 msgr2=0x7fad3c0ff870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.528+0000 7fad42de2640 1 --2- 172.21.15.18:0/864807495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad3c0ff450 0x7fad3c0ff870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad34009ca0 tx=0x7fad3402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.528+0000 7fad42de2640 1 -- 172.21.15.18:0/864807495 shutdown_connections 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.528+0000 7fad42de2640 1 --2- 172.21.15.18:0/864807495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad3c102de0 0x7fad3c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.528+0000 7fad42de2640 1 --2- 172.21.15.18:0/864807495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3c102420 0x7fad3c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.528+0000 7fad42de2640 1 --2- 172.21.15.18:0/864807495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad3c0ff450 0x7fad3c0ff870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.528+0000 7fad42de2640 1 -- 172.21.15.18:0/864807495 >> 172.21.15.18:0/864807495 conn(0x7fad3c0fada0 msgr2=0x7fad3c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.528+0000 7fad42de2640 1 -- 172.21.15.18:0/864807495 shutdown_connections 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.528+0000 7fad42de2640 1 -- 172.21.15.18:0/864807495 wait complete. 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.528+0000 7fad42de2640 1 Processor -- start 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.528+0000 7fad42de2640 1 -- start start 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad42de2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad3c0ff450 0x7fad3c197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad42de2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad3c102420 0x7fad3c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:08.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad42de2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3c102de0 0x7fad3c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad42de2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c19ed10 con 0x7fad3c0ff450 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad42de2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c19ee80 con 0x7fad3c102de0 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad42de2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c19f040 con 0x7fad3c102420 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad40b7e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad3c0ff450 0x7fad3c197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad40b7e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad3c0ff450 0x7fad3c197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43382/0 (socket says 172.21.15.18:43382) 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad40b7e640 1 -- 172.21.15.18:0/682212770 learned_addr learned my addr 172.21.15.18:0/682212770 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad4137f640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3c102de0 0x7fad3c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad3bfff640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad3c102420 0x7fad3c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad40b7e640 1 -- 172.21.15.18:0/682212770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad3c102420 msgr2=0x7fad3c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad40b7e640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad3c102420 0x7fad3c197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad40b7e640 1 -- 172.21.15.18:0/682212770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3c102de0 msgr2=0x7fad3c19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad40b7e640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3c102de0 0x7fad3c19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad40b7e640 1 -- 172.21.15.18:0/682212770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad34009900 con 0x7fad3c0ff450 2024-02-17T12:47:08.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad3bfff640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad3c102420 0x7fad3c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:08.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad40b7e640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad3c0ff450 0x7fad3c197220 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fad34037040 tx=0x7fad340346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:08.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad39ffb640 1 -- 172.21.15.18:0/682212770 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34032e70 con 0x7fad3c0ff450 2024-02-17T12:47:08.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad42de2640 1 -- 172.21.15.18:0/682212770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3c19f2c0 con 0x7fad3c0ff450 2024-02-17T12:47:08.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad39ffb640 1 -- 172.21.15.18:0/682212770 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad34003e50 con 0x7fad3c0ff450 2024-02-17T12:47:08.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.529+0000 7fad39ffb640 1 -- 172.21.15.18:0/682212770 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad340406b0 con 0x7fad3c0ff450 2024-02-17T12:47:08.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.530+0000 7fad42de2640 1 -- 172.21.15.18:0/682212770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3c19f770 con 0x7fad3c0ff450 2024-02-17T12:47:08.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.531+0000 7fad42de2640 1 -- 172.21.15.18:0/682212770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3c1085c0 con 0x7fad3c0ff450 2024-02-17T12:47:08.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.532+0000 7fad39ffb640 1 -- 172.21.15.18:0/682212770 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fad3400a040 con 0x7fad3c0ff450 2024-02-17T12:47:08.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.532+0000 7fad39ffb640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fad240a20c0 0x7fad240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:08.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.532+0000 7fad3bfff640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fad240a20c0 0x7fad240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:08.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.532+0000 7fad39ffb640 1 -- 172.21.15.18:0/682212770 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fad340f0690 con 0x7fad3c0ff450 2024-02-17T12:47:08.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.533+0000 7fad3bfff640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fad240a20c0 0x7fad240a4580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fad300058c0 tx=0x7fad30005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:08.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.537+0000 7fad39ffb640 1 -- 172.21.15.18:0/682212770 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad340b9b00 con 0x7fad3c0ff450 2024-02-17T12:47:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.716+0000 7fad42de2640 1 -- 172.21.15.18:0/682212770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad3c02cda0 con 0x7fad3c0ff450 2024-02-17T12:47:08.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.717+0000 7fad39ffb640 1 -- 172.21.15.18:0/682212770 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad340bd1c0 con 0x7fad3c0ff450 2024-02-17T12:47:08.710 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-02-17T12:47:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.719+0000 7fad42de2640 1 -- 172.21.15.18:0/682212770 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fad240a20c0 msgr2=0x7fad240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.719+0000 7fad42de2640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fad240a20c0 0x7fad240a4580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fad300058c0 tx=0x7fad30005850 comp rx=0 tx=0).stop 2024-02-17T12:47:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.719+0000 7fad42de2640 1 -- 172.21.15.18:0/682212770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad3c0ff450 msgr2=0x7fad3c197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.719+0000 7fad42de2640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad3c0ff450 0x7fad3c197220 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fad34037040 tx=0x7fad340346f0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.719+0000 7fad42de2640 1 -- 172.21.15.18:0/682212770 shutdown_connections 2024-02-17T12:47:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.719+0000 7fad42de2640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fad240a20c0 0x7fad240a4580 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.719+0000 7fad42de2640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3c102de0 0x7fad3c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.719+0000 7fad42de2640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad3c102420 0x7fad3c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.719+0000 7fad42de2640 1 --2- 172.21.15.18:0/682212770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad3c0ff450 0x7fad3c197220 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.719+0000 7fad42de2640 1 -- 172.21.15.18:0/682212770 >> 172.21.15.18:0/682212770 conn(0x7fad3c0fada0 msgr2=0x7fad3c105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.719+0000 7fad42de2640 1 -- 172.21.15.18:0/682212770 shutdown_connections 2024-02-17T12:47:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.719+0000 7fad42de2640 1 -- 172.21.15.18:0/682212770 wait complete. 2024-02-17T12:47:08.723 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:08.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.832+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/1586585242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5b0101290 msgr2=0x7ff5b0101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.832+0000 7ff5b5fd7640 1 --2- 172.21.15.18:0/1586585242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5b0101290 0x7ff5b0101710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009900 tx=0x7ff5a402bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:08.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.833+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/1586585242 shutdown_connections 2024-02-17T12:47:08.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.833+0000 7ff5b5fd7640 1 --2- 172.21.15.18:0/1586585242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5b0101c50 0x7ff5b01084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.833+0000 7ff5b5fd7640 1 --2- 172.21.15.18:0/1586585242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5b0101290 0x7ff5b0101710 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.833+0000 7ff5b5fd7640 1 --2- 172.21.15.18:0/1586585242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5b01000a0 0x7ff5b01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.833+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/1586585242 >> 172.21.15.18:0/1586585242 conn(0x7ff5b00fb610 msgr2=0x7ff5b00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.833+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/1586585242 shutdown_connections 2024-02-17T12:47:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.833+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/1586585242 wait complete. 2024-02-17T12:47:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.833+0000 7ff5b5fd7640 1 Processor -- start 2024-02-17T12:47:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.833+0000 7ff5b5fd7640 1 -- start start 2024-02-17T12:47:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.833+0000 7ff5b5fd7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5b01000a0 0x7ff5b01993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.833+0000 7ff5b5fd7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5b0101290 0x7ff5b0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.833+0000 7ff5b5fd7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5b0101c50 0x7ff5b01a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5b5fd7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b01a0ee0 con 0x7ff5b0101290 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5b5fd7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b01a1050 con 0x7ff5b01000a0 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5b5fd7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b01a1210 con 0x7ff5b0101c50 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5aeffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5b0101290 0x7ff5b0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5aeffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5b0101290 0x7ff5b0199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43412/0 (socket says 172.21.15.18:43412) 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5aeffd640 1 -- 172.21.15.18:0/891724430 learned_addr learned my addr 172.21.15.18:0/891724430 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5affff640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5b0101c50 0x7ff5b01a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5af7fe640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5b01000a0 0x7ff5b01993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5aeffd640 1 -- 172.21.15.18:0/891724430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5b0101c50 msgr2=0x7ff5b01a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5aeffd640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5b0101c50 0x7ff5b01a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5aeffd640 1 -- 172.21.15.18:0/891724430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5b01000a0 msgr2=0x7ff5b01993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5aeffd640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5b01000a0 0x7ff5b01993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5aeffd640 1 -- 172.21.15.18:0/891724430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a4009560 con 0x7ff5b0101290 2024-02-17T12:47:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5aeffd640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5b0101290 0x7ff5b0199930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4006010 tx=0x7ff5a40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5acff9640 1 -- 172.21.15.18:0/891724430 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40044c0 con 0x7ff5b0101290 2024-02-17T12:47:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5acff9640 1 -- 172.21.15.18:0/891724430 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5a40052d0 con 0x7ff5b0101290 2024-02-17T12:47:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/891724430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b01a1490 con 0x7ff5b0101290 2024-02-17T12:47:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5acff9640 1 -- 172.21.15.18:0/891724430 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4003e10 con 0x7ff5b0101290 2024-02-17T12:47:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.834+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/891724430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b01a19f0 con 0x7ff5b0101290 2024-02-17T12:47:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.835+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/891724430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b010bab0 con 0x7ff5b0101290 2024-02-17T12:47:08.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.836+0000 7ff5acff9640 1 -- 172.21.15.18:0/891724430 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff5a4005440 con 0x7ff5b0101290 2024-02-17T12:47:08.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.837+0000 7ff5acff9640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff5980a2140 0x7ff5980a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:08.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.837+0000 7ff5af7fe640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff5980a2140 0x7ff5980a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:08.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.837+0000 7ff5acff9640 1 -- 172.21.15.18:0/891724430 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff5a40efa40 con 0x7ff5b0101290 2024-02-17T12:47:08.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.837+0000 7ff5af7fe640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff5980a2140 0x7ff5980a4600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff5b01010f0 tx=0x7ff5a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:08.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:08.841+0000 7ff5acff9640 1 -- 172.21.15.18:0/891724430 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff5a40f1e60 con 0x7ff5b0101290 2024-02-17T12:47:09.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.060+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/891724430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5b010bc40 con 0x7ff5b0101290 2024-02-17T12:47:09.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.061+0000 7ff5acff9640 1 -- 172.21.15.18:0/891724430 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7ff5a40b8eb0 con 0x7ff5b0101290 2024-02-17T12:47:09.054 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:09.054 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-02-17T12:47:09.054 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 1 2024-02-17T12:47:09.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.063+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/891724430 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff5980a2140 msgr2=0x7ff5980a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:09.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.063+0000 7ff5b5fd7640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff5980a2140 0x7ff5980a4600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff5b01010f0 tx=0x7ff5a0005f90 comp rx=0 tx=0).stop 2024-02-17T12:47:09.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.063+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/891724430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5b0101290 msgr2=0x7ff5b0199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:09.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.063+0000 7ff5b5fd7640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5b0101290 0x7ff5b0199930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4006010 tx=0x7ff5a40040c0 comp rx=0 tx=0).stop 2024-02-17T12:47:09.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.063+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/891724430 shutdown_connections 2024-02-17T12:47:09.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.063+0000 7ff5b5fd7640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff5980a2140 0x7ff5980a4600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:09.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.063+0000 7ff5b5fd7640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5b0101c50 0x7ff5b01a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:09.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.063+0000 7ff5b5fd7640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5b0101290 0x7ff5b0199930 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:09.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.063+0000 7ff5b5fd7640 1 --2- 172.21.15.18:0/891724430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5b01000a0 0x7ff5b01993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:09.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.063+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/891724430 >> 172.21.15.18:0/891724430 conn(0x7ff5b00fb610 msgr2=0x7ff5b00fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:09.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.063+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/891724430 shutdown_connections 2024-02-17T12:47:09.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:09.063+0000 7ff5b5fd7640 1 -- 172.21.15.18:0/891724430 wait complete. 2024-02-17T12:47:09.251 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:08 smithi146 ceph-mon[36454]: Deploying daemon mds.l on smithi146 2024-02-17T12:47:09.251 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:08 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/4251476391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:09.251 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:08 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/682212770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:47:09.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:08 smithi018 ceph-mon[32625]: Deploying daemon mds.l on smithi146 2024-02-17T12:47:09.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:08 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/4251476391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:09.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:08 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/682212770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:47:09.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:08 smithi119 ceph-mon[36188]: Deploying daemon mds.l on smithi146 2024-02-17T12:47:09.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:08 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/4251476391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:09.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:08 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/682212770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:47:09.943 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 12:47:09 smithi146 systemd[1]: Starting Ceph mds.l for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:47:10.196 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:09 smithi146 ceph-mon[36454]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:10.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:09 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/891724430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:10.197 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 12:47:09 smithi146 podman[66705]: 2024-02-17 12:47:09.853053331 +0000 UTC m=+0.017976036 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:47:10.197 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 12:47:09 smithi146 podman[66705]: 2024-02-17 12:47:09.976839337 +0000 UTC m=+0.141762037 container create 79ede5d44042f490af8a075e46db7a0b0287851a86669ae79b72a16eda966e58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-17T12:47:10.197 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 12:47:10 smithi146 podman[66705]: 2024-02-17 12:47:10.20435308 +0000 UTC m=+0.369275790 container init 79ede5d44042f490af8a075e46db7a0b0287851a86669ae79b72a16eda966e58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-02-17T12:47:10.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:09 smithi018 ceph-mon[32625]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:10.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:09 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/891724430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:10.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:09 smithi119 ceph-mon[36188]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:10.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:09 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/891724430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:10.452 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 12:47:10 smithi146 podman[66705]: 2024-02-17 12:47:10.207146683 +0000 UTC m=+0.372069377 container start 79ede5d44042f490af8a075e46db7a0b0287851a86669ae79b72a16eda966e58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-17T12:47:10.452 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 12:47:10 smithi146 ceph-mds[66725]: -- 172.21.15.146:0/2973080838 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559eaa54e1a0 con 0x559eaa524c00 2024-02-17T12:47:10.452 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 12:47:10 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l[66721]: starting mds.l at 2024-02-17T12:47:10.452 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 12:47:10 smithi146 bash[66705]: 79ede5d44042f490af8a075e46db7a0b0287851a86669ae79b72a16eda966e58 2024-02-17T12:47:10.452 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 12:47:10 smithi146 systemd[1]: Started Ceph mds.l for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:47:11.068 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:11.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.190+0000 7f6afd6ce640 1 -- 172.21.15.18:0/2044838310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6af8101c20 msgr2=0x7f6af8104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:11.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.190+0000 7f6afd6ce640 1 --2- 172.21.15.18:0/2044838310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6af8101c20 0x7f6af8104010 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009900 tx=0x7f6ae802bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:11.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.190+0000 7f6afd6ce640 1 -- 172.21.15.18:0/2044838310 shutdown_connections 2024-02-17T12:47:11.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.190+0000 7f6afd6ce640 1 --2- 172.21.15.18:0/2044838310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af8106e80 0x7f6af81092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:11.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.190+0000 7f6afd6ce640 1 --2- 172.21.15.18:0/2044838310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6af8104550 0x7f6af8106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:11.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.190+0000 7f6afd6ce640 1 --2- 172.21.15.18:0/2044838310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6af8101c20 0x7f6af8104010 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:11.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.190+0000 7f6afd6ce640 1 -- 172.21.15.18:0/2044838310 >> 172.21.15.18:0/2044838310 conn(0x7f6af80fb610 msgr2=0x7f6af80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:11.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.190+0000 7f6afd6ce640 1 -- 172.21.15.18:0/2044838310 shutdown_connections 2024-02-17T12:47:11.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.190+0000 7f6afd6ce640 1 -- 172.21.15.18:0/2044838310 wait complete. 2024-02-17T12:47:11.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.191+0000 7f6afd6ce640 1 Processor -- start 2024-02-17T12:47:11.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.191+0000 7f6afd6ce640 1 -- start start 2024-02-17T12:47:11.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.191+0000 7f6afd6ce640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6af8101c20 0x7f6af8197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:11.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.191+0000 7f6afd6ce640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af8104550 0x7f6af8197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.191+0000 7f6afd6ce640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6af8106e80 0x7f6af819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.191+0000 7f6afd6ce640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af819ed30 con 0x7f6af8106e80 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.191+0000 7f6afd6ce640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af819eea0 con 0x7f6af8101c20 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.191+0000 7f6afd6ce640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af819f060 con 0x7f6af8104550 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.191+0000 7f6af77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6af8106e80 0x7f6af819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.191+0000 7f6af77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6af8106e80 0x7f6af819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57252/0 (socket says 172.21.15.18:57252) 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.191+0000 7f6af77fe640 1 -- 172.21.15.18:0/1712577983 learned_addr learned my addr 172.21.15.18:0/1712577983 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.191+0000 7f6af6ffd640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6af8101c20 0x7f6af8197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.192+0000 7f6aee7fc640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af8104550 0x7f6af8197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.192+0000 7f6af77fe640 1 -- 172.21.15.18:0/1712577983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af8104550 msgr2=0x7f6af8197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.192+0000 7f6af77fe640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af8104550 0x7f6af8197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.192+0000 7f6af77fe640 1 -- 172.21.15.18:0/1712577983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6af8101c20 msgr2=0x7f6af8197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.192+0000 7f6af77fe640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6af8101c20 0x7f6af8197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.192+0000 7f6af77fe640 1 -- 172.21.15.18:0/1712577983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae8009560 con 0x7f6af8106e80 2024-02-17T12:47:11.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.192+0000 7f6af77fe640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6af8106e80 0x7f6af819e7f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009fd0 tx=0x7f6af000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:11.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.192+0000 7f6af4ff9640 1 -- 172.21.15.18:0/1712577983 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af000bea0 con 0x7f6af8106e80 2024-02-17T12:47:11.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.192+0000 7f6af4ff9640 1 -- 172.21.15.18:0/1712577983 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6af0009940 con 0x7f6af8106e80 2024-02-17T12:47:11.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.192+0000 7f6afd6ce640 1 -- 172.21.15.18:0/1712577983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af819f340 con 0x7f6af8106e80 2024-02-17T12:47:11.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.192+0000 7f6af4ff9640 1 -- 172.21.15.18:0/1712577983 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af00106c0 con 0x7f6af8106e80 2024-02-17T12:47:11.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.192+0000 7f6afd6ce640 1 -- 172.21.15.18:0/1712577983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af819f800 con 0x7f6af8106e80 2024-02-17T12:47:11.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.193+0000 7f6afd6ce640 1 -- 172.21.15.18:0/1712577983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af80fd100 con 0x7f6af8106e80 2024-02-17T12:47:11.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.194+0000 7f6af4ff9640 1 -- 172.21.15.18:0/1712577983 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6af0004240 con 0x7f6af8106e80 2024-02-17T12:47:11.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.195+0000 7f6af4ff9640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6ae40a20c0 0x7f6ae40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:11.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.195+0000 7f6af6ffd640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6ae40a20c0 0x7f6ae40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:11.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.195+0000 7f6af4ff9640 1 -- 172.21.15.18:0/1712577983 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6af00cd480 con 0x7f6af8106e80 2024-02-17T12:47:11.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.195+0000 7f6af6ffd640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6ae40a20c0 0x7f6ae40a4580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009230 tx=0x7f6ae8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:11.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.200+0000 7f6af4ff9640 1 -- 172.21.15.18:0/1712577983 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6af00968f0 con 0x7f6af8106e80 2024-02-17T12:47:11.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:11 smithi119 ceph-mon[36188]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:11.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:11 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.146:6832/4241790141,v1:172.21.15.146:6833/4241790141] up:boot 2024-02-17T12:47:11.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:11 smithi119 ceph-mon[36188]: fsmap 1 up:standby 2024-02-17T12:47:11.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:47:11.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:11 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:11 smithi119 ceph-mon[36188]: Deploying daemon mds.k on smithi119 2024-02-17T12:47:11.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.420+0000 7f6afd6ce640 1 -- 172.21.15.18:0/1712577983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6af802ce40 con 0x7f6af8106e80 2024-02-17T12:47:11.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.420+0000 7f6af4ff9640 1 -- 172.21.15.18:0/1712577983 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f6af0099fb0 con 0x7f6af8106e80 2024-02-17T12:47:11.413 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:11.414 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-17T12:47:11.414 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 2 2024-02-17T12:47:11.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.422+0000 7f6afd6ce640 1 -- 172.21.15.18:0/1712577983 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6ae40a20c0 msgr2=0x7f6ae40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:11.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.422+0000 7f6afd6ce640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6ae40a20c0 0x7f6ae40a4580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009230 tx=0x7f6ae8005fd0 comp rx=0 tx=0).stop 2024-02-17T12:47:11.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.422+0000 7f6afd6ce640 1 -- 172.21.15.18:0/1712577983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6af8106e80 msgr2=0x7f6af819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:11.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.422+0000 7f6afd6ce640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6af8106e80 0x7f6af819e7f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009fd0 tx=0x7f6af000b260 comp rx=0 tx=0).stop 2024-02-17T12:47:11.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.422+0000 7f6afd6ce640 1 -- 172.21.15.18:0/1712577983 shutdown_connections 2024-02-17T12:47:11.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.422+0000 7f6afd6ce640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6ae40a20c0 0x7f6ae40a4580 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:11.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.422+0000 7f6afd6ce640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6af8106e80 0x7f6af819e7f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:11.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.422+0000 7f6afd6ce640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af8104550 0x7f6af8197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:11.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.422+0000 7f6afd6ce640 1 --2- 172.21.15.18:0/1712577983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6af8101c20 0x7f6af8197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:11.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.422+0000 7f6afd6ce640 1 -- 172.21.15.18:0/1712577983 >> 172.21.15.18:0/1712577983 conn(0x7f6af80fb610 msgr2=0x7f6af81024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:11.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.422+0000 7f6afd6ce640 1 -- 172.21.15.18:0/1712577983 shutdown_connections 2024-02-17T12:47:11.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:11.422+0000 7f6afd6ce640 1 -- 172.21.15.18:0/1712577983 wait complete. 2024-02-17T12:47:11.426 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:11 smithi018 ceph-mon[32625]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:11.426 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:11 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.146:6832/4241790141,v1:172.21.15.146:6833/4241790141] up:boot 2024-02-17T12:47:11.427 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:11 smithi018 ceph-mon[32625]: fsmap 1 up:standby 2024-02-17T12:47:11.427 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:47:11.427 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:11.427 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:11.427 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:11.427 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:11.427 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:11.427 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:11 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:11.427 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:11 smithi018 ceph-mon[32625]: Deploying daemon mds.k on smithi119 2024-02-17T12:47:11.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:11 smithi146 ceph-mon[36454]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:11.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:11 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.146:6832/4241790141,v1:172.21.15.146:6833/4241790141] up:boot 2024-02-17T12:47:11.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:11 smithi146 ceph-mon[36454]: fsmap 1 up:standby 2024-02-17T12:47:11.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:47:11.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:11.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:11.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:11.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:11.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:11.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:11 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:11.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:11 smithi146 ceph-mon[36454]: Deploying daemon mds.k on smithi119 2024-02-17T12:47:12.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:12 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1712577983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:12.572 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:12 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1712577983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:12.572 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 12:47:12 smithi119 systemd[1]: Starting Ceph mds.k for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:47:12.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:12 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1712577983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:12.825 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 12:47:12 smithi119 podman[66345]: 2024-02-17 12:47:12.481660212 +0000 UTC m=+0.017841823 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:47:12.825 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 12:47:12 smithi119 podman[66345]: 2024-02-17 12:47:12.579066868 +0000 UTC m=+0.115248484 container create 16c69fa7b6ed4011ace77e8c630d6cc6d051610a60a6ff51921ffc8991f93d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-02-17T12:47:12.825 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 12:47:12 smithi119 podman[66345]: 2024-02-17 12:47:12.806722435 +0000 UTC m=+0.342904041 container init 16c69fa7b6ed4011ace77e8c630d6cc6d051610a60a6ff51921ffc8991f93d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-17T12:47:12.825 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 12:47:12 smithi119 podman[66345]: 2024-02-17 12:47:12.809296408 +0000 UTC m=+0.345478019 container start 16c69fa7b6ed4011ace77e8c630d6cc6d051610a60a6ff51921ffc8991f93d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b) 2024-02-17T12:47:13.094 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 12:47:12 smithi119 ceph-mds[66365]: -- 172.21.15.119:0/3046937799 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5617bb1dc1a0 con 0x5617bb1b2c00 2024-02-17T12:47:13.094 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 12:47:12 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k[66361]: starting mds.k at 2024-02-17T12:47:13.094 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 12:47:12 smithi119 bash[66345]: 16c69fa7b6ed4011ace77e8c630d6cc6d051610a60a6ff51921ffc8991f93d33 2024-02-17T12:47:13.094 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 12:47:12 smithi119 systemd[1]: Started Ceph mds.k for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:47:13.429 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.548+0000 7fbdd24fb640 1 -- 172.21.15.18:0/111423030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdcc106e80 msgr2=0x7fbdcc1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.548+0000 7fbdd24fb640 1 --2- 172.21.15.18:0/111423030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdcc106e80 0x7fbdcc1092d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009ca0 tx=0x7fbdc402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:47:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.548+0000 7fbdd24fb640 1 -- 172.21.15.18:0/111423030 shutdown_connections 2024-02-17T12:47:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.548+0000 7fbdd24fb640 1 --2- 172.21.15.18:0/111423030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdcc106e80 0x7fbdcc1092d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.548+0000 7fbdd24fb640 1 --2- 172.21.15.18:0/111423030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc104550 0x7fbdcc106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.548+0000 7fbdd24fb640 1 --2- 172.21.15.18:0/111423030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdcc101c20 0x7fbdcc104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.548+0000 7fbdd24fb640 1 -- 172.21.15.18:0/111423030 >> 172.21.15.18:0/111423030 conn(0x7fbdcc0fb610 msgr2=0x7fbdcc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.548+0000 7fbdd24fb640 1 -- 172.21.15.18:0/111423030 shutdown_connections 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.548+0000 7fbdd24fb640 1 -- 172.21.15.18:0/111423030 wait complete. 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdd24fb640 1 Processor -- start 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdd24fb640 1 -- start start 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdd24fb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdcc101c20 0x7fbdcc197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdd24fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc104550 0x7fbdcc197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdd24fb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdcc106e80 0x7fbdcc19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdd24fb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc19ed10 con 0x7fbdcc101c20 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdd24fb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc19ee80 con 0x7fbdcc104550 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdd24fb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc19f040 con 0x7fbdcc106e80 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdcbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdcc101c20 0x7fbdcc197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdcbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdcc101c20 0x7fbdcc197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57272/0 (socket says 172.21.15.18:57272) 2024-02-17T12:47:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdcbfff640 1 -- 172.21.15.18:0/4138294697 learned_addr learned my addr 172.21.15.18:0/4138294697 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdd0a98640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdcc106e80 0x7fbdcc19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.549+0000 7fbdcb7fe640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc104550 0x7fbdcc197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdcbfff640 1 -- 172.21.15.18:0/4138294697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdcc106e80 msgr2=0x7fbdcc19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdcbfff640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdcc106e80 0x7fbdcc19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdcbfff640 1 -- 172.21.15.18:0/4138294697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc104550 msgr2=0x7fbdcc197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdcbfff640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc104550 0x7fbdcc197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdcbfff640 1 -- 172.21.15.18:0/4138294697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc4009900 con 0x7fbdcc101c20 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdcb7fe640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc104550 0x7fbdcc197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdd0a98640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdcc106e80 0x7fbdcc19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdcbfff640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdcc101c20 0x7fbdcc197220 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00d770 tx=0x7fbdbc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdc97fa640 1 -- 172.21.15.18:0/4138294697 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc009c30 con 0x7fbdcc101c20 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdd24fb640 1 -- 172.21.15.18:0/4138294697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdcc19f320 con 0x7fbdcc101c20 2024-02-17T12:47:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdc97fa640 1 -- 172.21.15.18:0/4138294697 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbdbc00de80 con 0x7fbdcc101c20 2024-02-17T12:47:13.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdc97fa640 1 -- 172.21.15.18:0/4138294697 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc004e80 con 0x7fbdcc101c20 2024-02-17T12:47:13.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.550+0000 7fbdd24fb640 1 -- 172.21.15.18:0/4138294697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdcc19f7e0 con 0x7fbdcc101c20 2024-02-17T12:47:13.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.551+0000 7fbdd24fb640 1 -- 172.21.15.18:0/4138294697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdcc0fd100 con 0x7fbdcc101c20 2024-02-17T12:47:13.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.552+0000 7fbdc97fa640 1 -- 172.21.15.18:0/4138294697 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fbdbc005020 con 0x7fbdcc101c20 2024-02-17T12:47:13.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.552+0000 7fbdc97fa640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fbdb80a20c0 0x7fbdb80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:13.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.552+0000 7fbdcb7fe640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fbdb80a20c0 0x7fbdb80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:13.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.553+0000 7fbdc97fa640 1 -- 172.21.15.18:0/4138294697 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbdbc0cd370 con 0x7fbdcc101c20 2024-02-17T12:47:13.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.553+0000 7fbdcb7fe640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fbdb80a20c0 0x7fbdb80a4580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc198730 tx=0x7fbdb4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:13.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.557+0000 7fbdc97fa640 1 -- 172.21.15.18:0/4138294697 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbdbc096740 con 0x7fbdcc101c20 2024-02-17T12:47:13.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.782+0000 7fbdd24fb640 1 -- 172.21.15.18:0/4138294697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdcc10cbb0 con 0x7fbdcc101c20 2024-02-17T12:47:13.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.783+0000 7fbdc97fa640 1 -- 172.21.15.18:0/4138294697 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2116 (secure 0 0 0) 0x7fbdbc099e00 con 0x7fbdcc101c20 2024-02-17T12:47:13.776 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:13.777 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-17T12:47:13.777 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 3 2024-02-17T12:47:13.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.785+0000 7fbdd24fb640 1 -- 172.21.15.18:0/4138294697 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fbdb80a20c0 msgr2=0x7fbdb80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:13.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.785+0000 7fbdd24fb640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fbdb80a20c0 0x7fbdb80a4580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc198730 tx=0x7fbdb4005d30 comp rx=0 tx=0).stop 2024-02-17T12:47:13.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.785+0000 7fbdd24fb640 1 -- 172.21.15.18:0/4138294697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdcc101c20 msgr2=0x7fbdcc197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:13.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.785+0000 7fbdd24fb640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdcc101c20 0x7fbdcc197220 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00d770 tx=0x7fbdbc00dc30 comp rx=0 tx=0).stop 2024-02-17T12:47:13.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.785+0000 7fbdd24fb640 1 -- 172.21.15.18:0/4138294697 shutdown_connections 2024-02-17T12:47:13.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.785+0000 7fbdd24fb640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fbdb80a20c0 0x7fbdb80a4580 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:13.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.785+0000 7fbdd24fb640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdcc106e80 0x7fbdcc19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:13.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.785+0000 7fbdd24fb640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdcc104550 0x7fbdcc197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:13.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.785+0000 7fbdd24fb640 1 --2- 172.21.15.18:0/4138294697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdcc101c20 0x7fbdcc197220 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:13.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.785+0000 7fbdd24fb640 1 -- 172.21.15.18:0/4138294697 >> 172.21.15.18:0/4138294697 conn(0x7fbdcc0fb610 msgr2=0x7fbdcc102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:13.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.785+0000 7fbdd24fb640 1 -- 172.21.15.18:0/4138294697 shutdown_connections 2024-02-17T12:47:13.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:13.785+0000 7fbdd24fb640 1 -- 172.21.15.18:0/4138294697 wait complete. 2024-02-17T12:47:14.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:13 smithi018 ceph-mon[32625]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:14.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:14.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:14.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:14.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:14.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:14.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:14.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:13 smithi018 ceph-mon[32625]: Deploying daemon mds.j on smithi018 2024-02-17T12:47:14.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:13 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] up:boot 2024-02-17T12:47:14.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:13 smithi018 ceph-mon[32625]: fsmap 2 up:standby 2024-02-17T12:47:14.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:13 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T12:47:14.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:13 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/4138294697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:14.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:13 smithi146 ceph-mon[36454]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:14.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:13 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:14.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:13 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:14.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:13 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:14.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:13 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:14.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:13 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:14.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:13 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:14.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:13 smithi146 ceph-mon[36454]: Deploying daemon mds.j on smithi018 2024-02-17T12:47:14.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:13 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] up:boot 2024-02-17T12:47:14.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:13 smithi146 ceph-mon[36454]: fsmap 2 up:standby 2024-02-17T12:47:14.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:13 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T12:47:14.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:13 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/4138294697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:14.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:13 smithi119 ceph-mon[36188]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:14.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:13 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:14.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:13 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:14.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:13 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:14.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:13 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:14.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:13 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:14.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:13 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:14.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:13 smithi119 ceph-mon[36188]: Deploying daemon mds.j on smithi018 2024-02-17T12:47:14.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:13 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] up:boot 2024-02-17T12:47:14.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:13 smithi119 ceph-mon[36188]: fsmap 2 up:standby 2024-02-17T12:47:14.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:13 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T12:47:14.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:13 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/4138294697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:15.275 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 12:47:15 smithi018 systemd[1]: Starting Ceph mds.j for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:47:15.657 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 12:47:15 smithi018 podman[73202]: 2024-02-17 12:47:15.275548713 +0000 UTC m=+0.017564794 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:47:15.657 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 12:47:15 smithi018 podman[73202]: 2024-02-17 12:47:15.378302821 +0000 UTC m=+0.120318939 container create 0514dd8cfd4ad6128f9131d8caf12ce024aece22e585152ac28fec3a2b0e0100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-j, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-17T12:47:15.657 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 12:47:15 smithi018 podman[73202]: 2024-02-17 12:47:15.589172363 +0000 UTC m=+0.331188440 container init 0514dd8cfd4ad6128f9131d8caf12ce024aece22e585152ac28fec3a2b0e0100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-j, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-17T12:47:15.657 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 12:47:15 smithi018 podman[73202]: 2024-02-17 12:47:15.591918754 +0000 UTC m=+0.333934835 container start 0514dd8cfd4ad6128f9131d8caf12ce024aece22e585152ac28fec3a2b0e0100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-j, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-17T12:47:15.657 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 12:47:15 smithi018 ceph-mds[73221]: -- 172.21.15.18:0/3054959612 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5643ec6401a0 con 0x5643ec616800 2024-02-17T12:47:15.657 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 12:47:15 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-j[73217]: starting mds.j at 2024-02-17T12:47:15.657 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 12:47:15 smithi018 bash[73202]: 0514dd8cfd4ad6128f9131d8caf12ce024aece22e585152ac28fec3a2b0e0100 2024-02-17T12:47:15.789 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:15.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.903+0000 7f5c8e888640 1 -- 172.21.15.18:0/915759331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c881045b0 msgr2=0x7f5c881069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:15.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.903+0000 7f5c8e888640 1 --2- 172.21.15.18:0/915759331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c881045b0 0x7f5c881069a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009900 tx=0x7f5c7c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:15.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.903+0000 7f5c8e888640 1 -- 172.21.15.18:0/915759331 shutdown_connections 2024-02-17T12:47:15.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.903+0000 7f5c8e888640 1 --2- 172.21.15.18:0/915759331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c88106ee0 0x7f5c881092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:15.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.903+0000 7f5c8e888640 1 --2- 172.21.15.18:0/915759331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c881045b0 0x7f5c881069a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:15.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.903+0000 7f5c8e888640 1 --2- 172.21.15.18:0/915759331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c88101c80 0x7f5c88104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:15.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.903+0000 7f5c8e888640 1 -- 172.21.15.18:0/915759331 >> 172.21.15.18:0/915759331 conn(0x7f5c880fb610 msgr2=0x7f5c880fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:15.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.903+0000 7f5c8e888640 1 -- 172.21.15.18:0/915759331 shutdown_connections 2024-02-17T12:47:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.903+0000 7f5c8e888640 1 -- 172.21.15.18:0/915759331 wait complete. 2024-02-17T12:47:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c8e888640 1 Processor -- start 2024-02-17T12:47:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c8e888640 1 -- start start 2024-02-17T12:47:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c8e888640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c88101c80 0x7f5c88197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c8e888640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c881045b0 0x7f5c88197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c8e888640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c88106ee0 0x7f5c8819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c8e888640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8819ed30 con 0x7f5c881045b0 2024-02-17T12:47:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c8e888640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8819eea0 con 0x7f5c88106ee0 2024-02-17T12:47:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c8e888640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8819f060 con 0x7f5c88101c80 2024-02-17T12:47:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c877fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c881045b0 0x7f5c88197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c877fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c881045b0 0x7f5c88197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57308/0 (socket says 172.21.15.18:57308) 2024-02-17T12:47:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c877fe640 1 -- 172.21.15.18:0/1712209892 learned_addr learned my addr 172.21.15.18:0/1712209892 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:15.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c8ce25640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c88106ee0 0x7f5c8819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:15.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c87fff640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c88101c80 0x7f5c88197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:15.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c877fe640 1 -- 172.21.15.18:0/1712209892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c88101c80 msgr2=0x7f5c88197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:15.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c877fe640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c88101c80 0x7f5c88197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:15.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c877fe640 1 -- 172.21.15.18:0/1712209892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c88106ee0 msgr2=0x7f5c8819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:15.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c877fe640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c88106ee0 0x7f5c8819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:15.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c877fe640 1 -- 172.21.15.18:0/1712209892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c009560 con 0x7f5c881045b0 2024-02-17T12:47:15.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.904+0000 7f5c87fff640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c88101c80 0x7f5c88197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:15.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.905+0000 7f5c877fe640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c881045b0 0x7f5c88197780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c005ab0 tx=0x7f5c7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:15.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.905+0000 7f5c857fa640 1 -- 172.21.15.18:0/1712209892 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c004aa0 con 0x7f5c881045b0 2024-02-17T12:47:15.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.905+0000 7f5c8e888640 1 -- 172.21.15.18:0/1712209892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8819f2e0 con 0x7f5c881045b0 2024-02-17T12:47:15.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.905+0000 7f5c857fa640 1 -- 172.21.15.18:0/1712209892 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c7c0052d0 con 0x7f5c881045b0 2024-02-17T12:47:15.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.905+0000 7f5c857fa640 1 -- 172.21.15.18:0/1712209892 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c003e80 con 0x7f5c881045b0 2024-02-17T12:47:15.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.905+0000 7f5c8e888640 1 -- 172.21.15.18:0/1712209892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8819f840 con 0x7f5c881045b0 2024-02-17T12:47:15.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.906+0000 7f5c8e888640 1 -- 172.21.15.18:0/1712209892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c880fd100 con 0x7f5c881045b0 2024-02-17T12:47:15.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.907+0000 7f5c857fa640 1 -- 172.21.15.18:0/1712209892 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5c7c005000 con 0x7f5c881045b0 2024-02-17T12:47:15.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.907+0000 7f5c857fa640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5c700a20c0 0x7f5c700a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:15.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.908+0000 7f5c87fff640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5c700a20c0 0x7f5c700a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:15.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.908+0000 7f5c857fa640 1 -- 172.21.15.18:0/1712209892 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5c7c0f0270 con 0x7f5c881045b0 2024-02-17T12:47:15.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.908+0000 7f5c87fff640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5c700a20c0 0x7f5c700a4580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009c20 tx=0x7f5c78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:15.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:15.911+0000 7f5c857fa640 1 -- 172.21.15.18:0/1712209892 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c7c0b96e0 con 0x7f5c881045b0 2024-02-17T12:47:15.946 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 12:47:15 smithi018 systemd[1]: Started Ceph mds.j for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:47:16.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.133+0000 7f5c8e888640 1 -- 172.21.15.18:0/1712209892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c8810cbb0 con 0x7f5c881045b0 2024-02-17T12:47:16.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.133+0000 7f5c857fa640 1 -- 172.21.15.18:0/1712209892 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2907 (secure 0 0 0) 0x7f5c7c0bcda0 con 0x7f5c881045b0 2024-02-17T12:47:16.127 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:16.127 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-17T12:47:16.128 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 4 2024-02-17T12:47:16.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.135+0000 7f5c8e888640 1 -- 172.21.15.18:0/1712209892 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5c700a20c0 msgr2=0x7f5c700a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:16.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.135+0000 7f5c8e888640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5c700a20c0 0x7f5c700a4580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009c20 tx=0x7f5c78005f90 comp rx=0 tx=0).stop 2024-02-17T12:47:16.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.135+0000 7f5c8e888640 1 -- 172.21.15.18:0/1712209892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c881045b0 msgr2=0x7f5c88197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:16.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.135+0000 7f5c8e888640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c881045b0 0x7f5c88197780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c005ab0 tx=0x7f5c7c004270 comp rx=0 tx=0).stop 2024-02-17T12:47:16.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.135+0000 7f5c8e888640 1 -- 172.21.15.18:0/1712209892 shutdown_connections 2024-02-17T12:47:16.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.135+0000 7f5c8e888640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5c700a20c0 0x7f5c700a4580 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:16.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.135+0000 7f5c8e888640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c88106ee0 0x7f5c8819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:16.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.135+0000 7f5c8e888640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c881045b0 0x7f5c88197780 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:16.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.135+0000 7f5c8e888640 1 --2- 172.21.15.18:0/1712209892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c88101c80 0x7f5c88197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:16.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.135+0000 7f5c8e888640 1 -- 172.21.15.18:0/1712209892 >> 172.21.15.18:0/1712209892 conn(0x7f5c880fb610 msgr2=0x7f5c881025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:16.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.135+0000 7f5c8e888640 1 -- 172.21.15.18:0/1712209892 shutdown_connections 2024-02-17T12:47:16.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:16.135+0000 7f5c8e888640 1 -- 172.21.15.18:0/1712209892 wait complete. 2024-02-17T12:47:16.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:15 smithi146 ceph-mon[36454]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:16.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:16.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:16.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:16.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:16.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:16.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:15 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:16.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:15 smithi018 ceph-mon[32625]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:16.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:16.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:16.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:16.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:16.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:16.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:15 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:16.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:15 smithi119 ceph-mon[36188]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:16.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:16.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:16.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:16.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:16.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:16.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:15 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:17.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:16 smithi018 ceph-mon[32625]: Deploying daemon mds.d on smithi018 2024-02-17T12:47:17.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:16 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.18:6834/3401600921,v1:172.21.15.18:6835/3401600921] up:boot 2024-02-17T12:47:17.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:16 smithi018 ceph-mon[32625]: fsmap 3 up:standby 2024-02-17T12:47:17.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:16 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-17T12:47:17.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:16 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1712209892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:17.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:16 smithi119 ceph-mon[36188]: Deploying daemon mds.d on smithi018 2024-02-17T12:47:17.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:16 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.18:6834/3401600921,v1:172.21.15.18:6835/3401600921] up:boot 2024-02-17T12:47:17.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:16 smithi119 ceph-mon[36188]: fsmap 3 up:standby 2024-02-17T12:47:17.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:16 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-17T12:47:17.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:16 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1712209892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:17.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:16 smithi146 ceph-mon[36454]: Deploying daemon mds.d on smithi018 2024-02-17T12:47:17.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:16 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.18:6834/3401600921,v1:172.21.15.18:6835/3401600921] up:boot 2024-02-17T12:47:17.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:16 smithi146 ceph-mon[36454]: fsmap 3 up:standby 2024-02-17T12:47:17.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:16 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-17T12:47:17.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:16 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1712209892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:17.953 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 12:47:17 smithi018 systemd[1]: Starting Ceph mds.d for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:47:18.141 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:18.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:17 smithi018 ceph-mon[32625]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:18.204 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 12:47:18 smithi018 podman[73553]: 2024-02-17 12:47:17.906471242 +0000 UTC m=+0.018310269 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:47:18.204 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 12:47:18 smithi018 podman[73553]: 2024-02-17 12:47:18.02395973 +0000 UTC m=+0.135798743 container create f1a3c63002a3f298b2f92426a5815d1ebfff4979cc5860491778bd0317bb4401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T12:47:18.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.293+0000 7f414eb9a640 1 -- 172.21.15.18:0/3470396182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148106f00 msgr2=0x7f41481092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:18.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.293+0000 7f414eb9a640 1 --2- 172.21.15.18:0/3470396182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148106f00 0x7f41481092f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4140009ca0 tx=0x7f414002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:47:18.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.293+0000 7f414eb9a640 1 -- 172.21.15.18:0/3470396182 shutdown_connections 2024-02-17T12:47:18.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.293+0000 7f414eb9a640 1 --2- 172.21.15.18:0/3470396182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148106f00 0x7f41481092f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:18.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.293+0000 7f414eb9a640 1 --2- 172.21.15.18:0/3470396182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41481045d0 0x7f41481069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:18.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.293+0000 7f414eb9a640 1 --2- 172.21.15.18:0/3470396182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4148101ca0 0x7f4148104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:18.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.293+0000 7f414eb9a640 1 -- 172.21.15.18:0/3470396182 >> 172.21.15.18:0/3470396182 conn(0x7f41480fb630 msgr2=0x7f41480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:18.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414eb9a640 1 -- 172.21.15.18:0/3470396182 shutdown_connections 2024-02-17T12:47:18.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414eb9a640 1 -- 172.21.15.18:0/3470396182 wait complete. 2024-02-17T12:47:18.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414eb9a640 1 Processor -- start 2024-02-17T12:47:18.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414eb9a640 1 -- start start 2024-02-17T12:47:18.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414eb9a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4148101ca0 0x7f4148197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:18.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414eb9a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41481045d0 0x7f41481977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:18.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414eb9a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148106f00 0x7f414819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:18.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414eb9a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414819ed80 con 0x7f4148106f00 2024-02-17T12:47:18.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414eb9a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414819eef0 con 0x7f41481045d0 2024-02-17T12:47:18.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414eb9a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414819f0b0 con 0x7f4148101ca0 2024-02-17T12:47:18.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414d137640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148106f00 0x7f414819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:18.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414d137640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148106f00 0x7f414819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57340/0 (socket says 172.21.15.18:57340) 2024-02-17T12:47:18.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f414d137640 1 -- 172.21.15.18:0/2528145120 learned_addr learned my addr 172.21.15.18:0/2528145120 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:18.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.294+0000 7f4147fff640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41481045d0 0x7f41481977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:18.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f414c936640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4148101ca0 0x7f4148197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:18.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f414d137640 1 -- 172.21.15.18:0/2528145120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4148101ca0 msgr2=0x7f4148197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:18.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f414d137640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4148101ca0 0x7f4148197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:18.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f414d137640 1 -- 172.21.15.18:0/2528145120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41481045d0 msgr2=0x7f41481977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:18.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f414d137640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41481045d0 0x7f41481977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:18.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f414d137640 1 -- 172.21.15.18:0/2528145120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4140009900 con 0x7f4148106f00 2024-02-17T12:47:18.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f4147fff640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41481045d0 0x7f41481977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:18.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f414d137640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148106f00 0x7f414819e840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4140005cc0 tx=0x7f4140034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:18.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f4145ffb640 1 -- 172.21.15.18:0/2528145120 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4140032480 con 0x7f4148106f00 2024-02-17T12:47:18.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f4145ffb640 1 -- 172.21.15.18:0/2528145120 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4140003e50 con 0x7f4148106f00 2024-02-17T12:47:18.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f414eb9a640 1 -- 172.21.15.18:0/2528145120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414819f330 con 0x7f4148106f00 2024-02-17T12:47:18.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f4145ffb640 1 -- 172.21.15.18:0/2528145120 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4140040470 con 0x7f4148106f00 2024-02-17T12:47:18.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.295+0000 7f414eb9a640 1 -- 172.21.15.18:0/2528145120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414819f7b0 con 0x7f4148106f00 2024-02-17T12:47:18.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.296+0000 7f414eb9a640 1 -- 172.21.15.18:0/2528145120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41480fd120 con 0x7f4148106f00 2024-02-17T12:47:18.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.297+0000 7f4145ffb640 1 -- 172.21.15.18:0/2528145120 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f414000a040 con 0x7f4148106f00 2024-02-17T12:47:18.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.298+0000 7f4145ffb640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f41380a20c0 0x7f41380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:18.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.298+0000 7f414c936640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f41380a20c0 0x7f41380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:18.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.298+0000 7f4145ffb640 1 -- 172.21.15.18:0/2528145120 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f41400f07b0 con 0x7f4148106f00 2024-02-17T12:47:18.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.298+0000 7f414c936640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f41380a20c0 0x7f41380a4580 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f413c005d00 tx=0x7f413c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:18.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.302+0000 7f4145ffb640 1 -- 172.21.15.18:0/2528145120 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f41400b9b80 con 0x7f4148106f00 2024-02-17T12:47:18.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:17 smithi119 ceph-mon[36188]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:18.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:17 smithi146 ceph-mon[36454]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:18.525 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 12:47:18 smithi018 podman[73553]: 2024-02-17 12:47:18.252009375 +0000 UTC m=+0.363848392 container init f1a3c63002a3f298b2f92426a5815d1ebfff4979cc5860491778bd0317bb4401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-17T12:47:18.525 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 12:47:18 smithi018 podman[73553]: 2024-02-17 12:47:18.255462928 +0000 UTC m=+0.367301955 container start f1a3c63002a3f298b2f92426a5815d1ebfff4979cc5860491778bd0317bb4401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-d, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-17T12:47:18.526 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 12:47:18 smithi018 ceph-mds[73592]: -- 172.21.15.18:0/3819972283 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559ae1be01a0 con 0x559ae1bb6800 2024-02-17T12:47:18.526 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 12:47:18 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-d[73568]: starting mds.d at 2024-02-17T12:47:18.526 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 12:47:18 smithi018 bash[73553]: f1a3c63002a3f298b2f92426a5815d1ebfff4979cc5860491778bd0317bb4401 2024-02-17T12:47:18.526 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 12:47:18 smithi018 systemd[1]: Started Ceph mds.d for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:47:18.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.532+0000 7f414eb9a640 1 -- 172.21.15.18:0/2528145120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f414802ce40 con 0x7f4148106f00 2024-02-17T12:47:18.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.533+0000 7f4145ffb640 1 -- 172.21.15.18:0/2528145120 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2907 (secure 0 0 0) 0x7f41400bd240 con 0x7f4148106f00 2024-02-17T12:47:18.527 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:18.527 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-17T12:47:18.527 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 4 2024-02-17T12:47:18.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.535+0000 7f414eb9a640 1 -- 172.21.15.18:0/2528145120 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f41380a20c0 msgr2=0x7f41380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:18.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.535+0000 7f414eb9a640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f41380a20c0 0x7f41380a4580 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f413c005d00 tx=0x7f413c00b410 comp rx=0 tx=0).stop 2024-02-17T12:47:18.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.535+0000 7f414eb9a640 1 -- 172.21.15.18:0/2528145120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148106f00 msgr2=0x7f414819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:18.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.535+0000 7f414eb9a640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148106f00 0x7f414819e840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4140005cc0 tx=0x7f4140034d40 comp rx=0 tx=0).stop 2024-02-17T12:47:18.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.535+0000 7f414eb9a640 1 -- 172.21.15.18:0/2528145120 shutdown_connections 2024-02-17T12:47:18.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.535+0000 7f414eb9a640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f41380a20c0 0x7f41380a4580 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:18.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.535+0000 7f414eb9a640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148106f00 0x7f414819e840 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:18.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.535+0000 7f414eb9a640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41481045d0 0x7f41481977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:18.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.535+0000 7f414eb9a640 1 --2- 172.21.15.18:0/2528145120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4148101ca0 0x7f4148197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:18.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.535+0000 7f414eb9a640 1 -- 172.21.15.18:0/2528145120 >> 172.21.15.18:0/2528145120 conn(0x7f41480fb630 msgr2=0x7f41481025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:18.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.535+0000 7f414eb9a640 1 -- 172.21.15.18:0/2528145120 shutdown_connections 2024-02-17T12:47:18.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:18.535+0000 7f414eb9a640 1 -- 172.21.15.18:0/2528145120 wait complete. 2024-02-17T12:47:19.584 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:19 smithi119 ceph-mon[36188]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:19.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:19 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:19.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:19 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:19.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:19 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:19.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:19 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:19.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:19 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:19.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:19 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:19.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:19 smithi119 ceph-mon[36188]: Deploying daemon mds.e on smithi119 2024-02-17T12:47:19.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:19 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2528145120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:19.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:19 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.18:6836/3018278996,v1:172.21.15.18:6837/3018278996] up:boot 2024-02-17T12:47:19.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:19 smithi119 ceph-mon[36188]: fsmap 4 up:standby 2024-02-17T12:47:19.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:19 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-17T12:47:19.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:19 smithi146 ceph-mon[36454]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:19.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:19 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:19.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:19 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:19.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:19 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:19.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:19 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:19.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:19 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:19.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:19 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:19.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:19 smithi146 ceph-mon[36454]: Deploying daemon mds.e on smithi119 2024-02-17T12:47:19.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:19 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2528145120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:19.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:19 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.18:6836/3018278996,v1:172.21.15.18:6837/3018278996] up:boot 2024-02-17T12:47:19.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:19 smithi146 ceph-mon[36454]: fsmap 4 up:standby 2024-02-17T12:47:19.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:19 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-17T12:47:19.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:19 smithi018 ceph-mon[32625]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:19.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:19 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:19.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:19 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:19.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:19 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:19.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:19 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:19.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:19 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:19.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:19 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:19.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:19 smithi018 ceph-mon[32625]: Deploying daemon mds.e on smithi119 2024-02-17T12:47:19.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:19 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2528145120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:19.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:19 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.18:6836/3018278996,v1:172.21.15.18:6837/3018278996] up:boot 2024-02-17T12:47:19.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:19 smithi018 ceph-mon[32625]: fsmap 4 up:standby 2024-02-17T12:47:19.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:19 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-17T12:47:20.541 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:20.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 12:47:20 smithi119 systemd[1]: Starting Ceph mds.e for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:47:20.567 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 12:47:20 smithi119 podman[66669]: 2024-02-17 12:47:20.476675516 +0000 UTC m=+0.018082513 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:47:20.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.662+0000 7f97b25bc640 1 -- 172.21.15.18:0/1044051624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97ac101cd0 msgr2=0x7f97ac108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:20.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.662+0000 7f97b25bc640 1 --2- 172.21.15.18:0/1044051624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97ac101cd0 0x7f97ac108560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009ca0 tx=0x7f97a402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:47:20.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.662+0000 7f97b25bc640 1 -- 172.21.15.18:0/1044051624 shutdown_connections 2024-02-17T12:47:20.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.662+0000 7f97b25bc640 1 --2- 172.21.15.18:0/1044051624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97ac101cd0 0x7f97ac108560 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:20.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.662+0000 7f97b25bc640 1 --2- 172.21.15.18:0/1044051624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac101310 0x7f97ac101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:20.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.662+0000 7f97b25bc640 1 --2- 172.21.15.18:0/1044051624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97ac100120 0x7f97ac100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:20.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.662+0000 7f97b25bc640 1 -- 172.21.15.18:0/1044051624 >> 172.21.15.18:0/1044051624 conn(0x7f97ac0fb6b0 msgr2=0x7f97ac0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:20.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.662+0000 7f97b25bc640 1 -- 172.21.15.18:0/1044051624 shutdown_connections 2024-02-17T12:47:20.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.662+0000 7f97b25bc640 1 -- 172.21.15.18:0/1044051624 wait complete. 2024-02-17T12:47:20.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.663+0000 7f97b25bc640 1 Processor -- start 2024-02-17T12:47:20.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.663+0000 7f97b25bc640 1 -- start start 2024-02-17T12:47:20.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.663+0000 7f97b25bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac100120 0x7f97ac197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:20.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.663+0000 7f97b25bc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97ac101310 0x7f97ac1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.663+0000 7f97b25bc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97ac101cd0 0x7f97ac19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.663+0000 7f97b25bc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac19ed60 con 0x7f97ac101310 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.663+0000 7f97b25bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac19eed0 con 0x7f97ac100120 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.663+0000 7f97b25bc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac19f090 con 0x7f97ac101cd0 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.663+0000 7f97ab7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97ac101310 0x7f97ac1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.663+0000 7f97ab7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97ac101310 0x7f97ac1977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40892/0 (socket says 172.21.15.18:40892) 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.663+0000 7f97ab7fe640 1 -- 172.21.15.18:0/3068988518 learned_addr learned my addr 172.21.15.18:0/3068988518 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.663+0000 7f97abfff640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac100120 0x7f97ac197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.664+0000 7f97b0b59640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97ac101cd0 0x7f97ac19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.664+0000 7f97ab7fe640 1 -- 172.21.15.18:0/3068988518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97ac101cd0 msgr2=0x7f97ac19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.664+0000 7f97ab7fe640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97ac101cd0 0x7f97ac19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.664+0000 7f97ab7fe640 1 -- 172.21.15.18:0/3068988518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac100120 msgr2=0x7f97ac197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.664+0000 7f97ab7fe640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac100120 0x7f97ac197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:20.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.664+0000 7f97ab7fe640 1 -- 172.21.15.18:0/3068988518 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a4009900 con 0x7f97ac101310 2024-02-17T12:47:20.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.664+0000 7f97ab7fe640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97ac101310 0x7f97ac1977b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f979400e9f0 tx=0x7f979400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:20.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.664+0000 7f97a97fa640 1 -- 172.21.15.18:0/3068988518 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794009700 con 0x7f97ac101310 2024-02-17T12:47:20.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.664+0000 7f97a97fa640 1 -- 172.21.15.18:0/3068988518 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f979400cd80 con 0x7f97ac101310 2024-02-17T12:47:20.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.664+0000 7f97b25bc640 1 -- 172.21.15.18:0/3068988518 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac19f370 con 0x7f97ac101310 2024-02-17T12:47:20.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.664+0000 7f97a97fa640 1 -- 172.21.15.18:0/3068988518 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794018920 con 0x7f97ac101310 2024-02-17T12:47:20.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.664+0000 7f97b25bc640 1 -- 172.21.15.18:0/3068988518 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac19f880 con 0x7f97ac101310 2024-02-17T12:47:20.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.665+0000 7f97b25bc640 1 -- 172.21.15.18:0/3068988518 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac10ba80 con 0x7f97ac101310 2024-02-17T12:47:20.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.666+0000 7f97a97fa640 1 -- 172.21.15.18:0/3068988518 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9794018a80 con 0x7f97ac101310 2024-02-17T12:47:20.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.667+0000 7f97a97fa640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f97980a20c0 0x7f97980a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:20.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.667+0000 7f97abfff640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f97980a20c0 0x7f97980a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:20.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.667+0000 7f97a97fa640 1 -- 172.21.15.18:0/3068988518 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9794014070 con 0x7f97ac101310 2024-02-17T12:47:20.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.668+0000 7f97abfff640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f97980a20c0 0x7f97980a4580 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f979c005fd0 tx=0x7f979c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:20.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.672+0000 7f97a97fa640 1 -- 172.21.15.18:0/3068988518 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9794097400 con 0x7f97ac101310 2024-02-17T12:47:20.843 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 12:47:20 smithi119 podman[66669]: 2024-02-17 12:47:20.581860409 +0000 UTC m=+0.123267432 container create 2e488f45841efc458b33320995459fd6163fda94e443733212abadb9399fe098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-e, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:47:20.843 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 12:47:20 smithi119 podman[66669]: 2024-02-17 12:47:20.792561565 +0000 UTC m=+0.333968554 container init 2e488f45841efc458b33320995459fd6163fda94e443733212abadb9399fe098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-e, maintainer=Guillaume Abrioux , ceph=True, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True) 2024-02-17T12:47:20.843 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 12:47:20 smithi119 podman[66669]: 2024-02-17 12:47:20.795178979 +0000 UTC m=+0.336585964 container start 2e488f45841efc458b33320995459fd6163fda94e443733212abadb9399fe098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-e, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-17T12:47:20.843 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 12:47:20 smithi119 ceph-mds[66688]: -- 172.21.15.119:0/3998418483 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557cd396a1a0 con 0x557cd3940800 2024-02-17T12:47:20.844 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 12:47:20 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-e[66684]: starting mds.e at 2024-02-17T12:47:20.844 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 12:47:20 smithi119 bash[66669]: 2e488f45841efc458b33320995459fd6163fda94e443733212abadb9399fe098 2024-02-17T12:47:20.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.889+0000 7f97b25bc640 1 -- 172.21.15.18:0/3068988518 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97ac19faa0 con 0x7f97ac101310 2024-02-17T12:47:20.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.890+0000 7f97a97fa640 1 -- 172.21.15.18:0/3068988518 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3698 (secure 0 0 0) 0x7f979409aac0 con 0x7f97ac101310 2024-02-17T12:47:20.884 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:20.884 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-17T12:47:20.884 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 5 2024-02-17T12:47:20.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.892+0000 7f97b25bc640 1 -- 172.21.15.18:0/3068988518 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f97980a20c0 msgr2=0x7f97980a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:20.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.892+0000 7f97b25bc640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f97980a20c0 0x7f97980a4580 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f979c005fd0 tx=0x7f979c00b410 comp rx=0 tx=0).stop 2024-02-17T12:47:20.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.892+0000 7f97b25bc640 1 -- 172.21.15.18:0/3068988518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97ac101310 msgr2=0x7f97ac1977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:20.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.892+0000 7f97b25bc640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97ac101310 0x7f97ac1977b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f979400e9f0 tx=0x7f979400eeb0 comp rx=0 tx=0).stop 2024-02-17T12:47:20.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.892+0000 7f97b25bc640 1 -- 172.21.15.18:0/3068988518 shutdown_connections 2024-02-17T12:47:20.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.892+0000 7f97b25bc640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f97980a20c0 0x7f97980a4580 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:20.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.892+0000 7f97b25bc640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97ac101cd0 0x7f97ac19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:20.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.892+0000 7f97b25bc640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97ac101310 0x7f97ac1977b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:20.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.892+0000 7f97b25bc640 1 --2- 172.21.15.18:0/3068988518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ac100120 0x7f97ac197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:20.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.892+0000 7f97b25bc640 1 -- 172.21.15.18:0/3068988518 >> 172.21.15.18:0/3068988518 conn(0x7f97ac0fb6b0 msgr2=0x7f97ac0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:20.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.892+0000 7f97b25bc640 1 -- 172.21.15.18:0/3068988518 shutdown_connections 2024-02-17T12:47:20.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:20.892+0000 7f97b25bc640 1 -- 172.21.15.18:0/3068988518 wait complete. 2024-02-17T12:47:21.094 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 12:47:20 smithi119 systemd[1]: Started Ceph mds.e for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:47:21.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:21 smithi146 ceph-mon[36454]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:21.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:21 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3068988518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:21.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:21.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:21.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:21.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:21.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:21.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:21 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:21.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:21 smithi146 ceph-mon[36454]: Deploying daemon mds.f on smithi146 2024-02-17T12:47:21.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:21 smithi018 ceph-mon[32625]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:21.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:21 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3068988518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:21.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:21.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:21.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:21.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:21.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:21.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:21 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:21.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:21 smithi018 ceph-mon[32625]: Deploying daemon mds.f on smithi146 2024-02-17T12:47:21.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:21 smithi119 ceph-mon[36188]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:21.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:21 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3068988518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:21.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:21.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:21.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:21.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:21.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:21.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:21 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:21.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:21 smithi119 ceph-mon[36188]: Deploying daemon mds.f on smithi146 2024-02-17T12:47:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:22 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] up:boot 2024-02-17T12:47:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:22 smithi146 ceph-mon[36454]: fsmap 5 up:standby 2024-02-17T12:47:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:22 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T12:47:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:22 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] up:boot 2024-02-17T12:47:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:22 smithi018 ceph-mon[32625]: fsmap 5 up:standby 2024-02-17T12:47:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:22 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T12:47:22.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:22 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] up:boot 2024-02-17T12:47:22.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:22 smithi119 ceph-mon[36188]: fsmap 5 up:standby 2024-02-17T12:47:22.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:22 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T12:47:22.898 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:23.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.021+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2467720115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7d4101ca0 msgr2=0x7fc7d4108530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:23.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.021+0000 7fc7da7e9640 1 --2- 172.21.15.18:0/2467720115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7d4101ca0 0x7fc7d4108530 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009ca0 tx=0x7fc7cc02c100 comp rx=0 tx=0).stop 2024-02-17T12:47:23.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.021+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2467720115 shutdown_connections 2024-02-17T12:47:23.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.021+0000 7fc7da7e9640 1 --2- 172.21.15.18:0/2467720115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7d4101ca0 0x7fc7d4108530 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:23.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.021+0000 7fc7da7e9640 1 --2- 172.21.15.18:0/2467720115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d41012e0 0x7fc7d4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:23.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.021+0000 7fc7da7e9640 1 --2- 172.21.15.18:0/2467720115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7d41005d0 0x7fc7d41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:23.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.021+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2467720115 >> 172.21.15.18:0/2467720115 conn(0x7fc7d40fada0 msgr2=0x7fc7d40fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:23.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.021+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2467720115 shutdown_connections 2024-02-17T12:47:23.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.021+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2467720115 wait complete. 2024-02-17T12:47:23.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.021+0000 7fc7da7e9640 1 Processor -- start 2024-02-17T12:47:23.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.021+0000 7fc7da7e9640 1 -- start start 2024-02-17T12:47:23.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7da7e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7d41005d0 0x7fc7d4104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:23.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7da7e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d41012e0 0x7fc7d41026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7da7e9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7d4101ca0 0x7fc7d4102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7da7e9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4103360 con 0x7fc7d41005d0 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7da7e9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d41034d0 con 0x7fc7d41012e0 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7da7e9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d41a1120 con 0x7fc7d4101ca0 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7d3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7d41005d0 0x7fc7d4104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7d3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7d41005d0 0x7fc7d4104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40910/0 (socket says 172.21.15.18:40910) 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7d3fff640 1 -- 172.21.15.18:0/2775627333 learned_addr learned my addr 172.21.15.18:0/2775627333 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7d8d86640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7d4101ca0 0x7fc7d4102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7d37fe640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d41012e0 0x7fc7d41026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7d3fff640 1 -- 172.21.15.18:0/2775627333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7d4101ca0 msgr2=0x7fc7d4102c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7d3fff640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7d4101ca0 0x7fc7d4102c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7d3fff640 1 -- 172.21.15.18:0/2775627333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d41012e0 msgr2=0x7fc7d41026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7d3fff640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d41012e0 0x7fc7d41026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:23.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7d3fff640 1 -- 172.21.15.18:0/2775627333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc009900 con 0x7fc7d41005d0 2024-02-17T12:47:23.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7d37fe640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d41012e0 0x7fc7d41026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:23.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.022+0000 7fc7d3fff640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7d41005d0 0x7fc7d4104080 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c9a0 tx=0x7fc7c400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:23.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.023+0000 7fc7d17fa640 1 -- 172.21.15.18:0/2775627333 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4007c10 con 0x7fc7d41005d0 2024-02-17T12:47:23.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.023+0000 7fc7d17fa640 1 -- 172.21.15.18:0/2775627333 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7c4007d70 con 0x7fc7d41005d0 2024-02-17T12:47:23.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.023+0000 7fc7d17fa640 1 -- 172.21.15.18:0/2775627333 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400f910 con 0x7fc7d41005d0 2024-02-17T12:47:23.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.023+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2775627333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d41a13d0 con 0x7fc7d41005d0 2024-02-17T12:47:23.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.023+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2775627333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d41a1930 con 0x7fc7d41005d0 2024-02-17T12:47:23.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.024+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2775627333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d410bab0 con 0x7fc7d41005d0 2024-02-17T12:47:23.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.025+0000 7fc7d17fa640 1 -- 172.21.15.18:0/2775627333 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc7c400fa70 con 0x7fc7d41005d0 2024-02-17T12:47:23.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.025+0000 7fc7d17fa640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc7bc0a2140 0x7fc7bc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:23.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.025+0000 7fc7d37fe640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc7bc0a2140 0x7fc7bc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:23.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.026+0000 7fc7d17fa640 1 -- 172.21.15.18:0/2775627333 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc7c4013070 con 0x7fc7d41005d0 2024-02-17T12:47:23.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.026+0000 7fc7d37fe640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc7bc0a2140 0x7fc7bc0a4600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4103df0 tx=0x7fc7c800a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:23.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.030+0000 7fc7d17fa640 1 -- 172.21.15.18:0/2775627333 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc7c4096fd0 con 0x7fc7d41005d0 2024-02-17T12:47:23.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.252+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2775627333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7d41a1bb0 con 0x7fc7d41005d0 2024-02-17T12:47:23.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.253+0000 7fc7d17fa640 1 -- 172.21.15.18:0/2775627333 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7fc7c409a690 con 0x7fc7d41005d0 2024-02-17T12:47:23.246 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:23.246 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-02-17T12:47:23.246 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 6 2024-02-17T12:47:23.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.255+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2775627333 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc7bc0a2140 msgr2=0x7fc7bc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:23.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.255+0000 7fc7da7e9640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc7bc0a2140 0x7fc7bc0a4600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4103df0 tx=0x7fc7c800a5c0 comp rx=0 tx=0).stop 2024-02-17T12:47:23.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.255+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2775627333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7d41005d0 msgr2=0x7fc7d4104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.255+0000 7fc7da7e9640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7d41005d0 0x7fc7d4104080 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c9a0 tx=0x7fc7c400ce60 comp rx=0 tx=0).stop 2024-02-17T12:47:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.255+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2775627333 shutdown_connections 2024-02-17T12:47:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.255+0000 7fc7da7e9640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc7bc0a2140 0x7fc7bc0a4600 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.255+0000 7fc7da7e9640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7d4101ca0 0x7fc7d4102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.255+0000 7fc7da7e9640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d41012e0 0x7fc7d41026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.255+0000 7fc7da7e9640 1 --2- 172.21.15.18:0/2775627333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7d41005d0 0x7fc7d4104080 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.255+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2775627333 >> 172.21.15.18:0/2775627333 conn(0x7fc7d40fada0 msgr2=0x7fc7d40fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.255+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2775627333 shutdown_connections 2024-02-17T12:47:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:23.256+0000 7fc7da7e9640 1 -- 172.21.15.18:0/2775627333 wait complete. 2024-02-17T12:47:23.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:23 smithi018 ceph-mon[32625]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:23.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:23 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2775627333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:23.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:23 smithi146 ceph-mon[36454]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:23.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:23 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2775627333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:23.702 INFO:journalctl@ceph.mds.f.smithi146.stdout:Feb 17 12:47:23 smithi146 systemd[1]: Started Ceph mds.f for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:47:23.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:23 smithi119 ceph-mon[36188]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:23.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:23 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2775627333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:24.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:24 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:24.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:24 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:24.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:24 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:24.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:24 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:24.829 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:24 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:24.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:24 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:24.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:24 smithi146 ceph-mon[36454]: Deploying daemon mds.c on smithi146 2024-02-17T12:47:24.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:24 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] up:boot 2024-02-17T12:47:24.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:24 smithi146 ceph-mon[36454]: fsmap 6 up:standby 2024-02-17T12:47:24.830 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:24 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T12:47:24.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:24 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:24.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:24 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:24.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:24 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:24.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:24 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:24.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:24 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:24.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:24 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:24.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:24 smithi119 ceph-mon[36188]: Deploying daemon mds.c on smithi146 2024-02-17T12:47:24.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:24 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] up:boot 2024-02-17T12:47:24.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:24 smithi119 ceph-mon[36188]: fsmap 6 up:standby 2024-02-17T12:47:24.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:24 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T12:47:25.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:24 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:25.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:24 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:25.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:24 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:25.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:24 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:25.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:24 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:25.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:24 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:25.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:24 smithi018 ceph-mon[32625]: Deploying daemon mds.c on smithi146 2024-02-17T12:47:25.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:24 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] up:boot 2024-02-17T12:47:25.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:24 smithi018 ceph-mon[32625]: fsmap 6 up:standby 2024-02-17T12:47:25.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:24 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T12:47:25.260 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:25.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.382+0000 7f6cb616b640 1 -- 172.21.15.18:0/2220529661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cb0101cd0 msgr2=0x7f6cb0108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:25.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.382+0000 7f6cb616b640 1 --2- 172.21.15.18:0/2220529661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cb0101cd0 0x7f6cb0108560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009ca0 tx=0x7f6ca802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:47:25.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.382+0000 7f6cb616b640 1 -- 172.21.15.18:0/2220529661 shutdown_connections 2024-02-17T12:47:25.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.382+0000 7f6cb616b640 1 --2- 172.21.15.18:0/2220529661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cb0101cd0 0x7f6cb0108560 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:25.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.382+0000 7f6cb616b640 1 --2- 172.21.15.18:0/2220529661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cb0101310 0x7f6cb0101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:25.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.382+0000 7f6cb616b640 1 --2- 172.21.15.18:0/2220529661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cb0100120 0x7f6cb0100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:25.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.382+0000 7f6cb616b640 1 -- 172.21.15.18:0/2220529661 >> 172.21.15.18:0/2220529661 conn(0x7f6cb00fb6b0 msgr2=0x7f6cb00fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:25.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.382+0000 7f6cb616b640 1 -- 172.21.15.18:0/2220529661 shutdown_connections 2024-02-17T12:47:25.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.382+0000 7f6cb616b640 1 -- 172.21.15.18:0/2220529661 wait complete. 2024-02-17T12:47:25.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.382+0000 7f6cb616b640 1 Processor -- start 2024-02-17T12:47:25.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6cb616b640 1 -- start start 2024-02-17T12:47:25.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6cb616b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cb0100120 0x7f6cb0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:25.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6cb616b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cb0101310 0x7f6cb0197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:25.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6cb616b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cb0101cd0 0x7f6cb019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:25.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6cb616b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb019ecf0 con 0x7f6cb0100120 2024-02-17T12:47:25.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6cb616b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb019ee60 con 0x7f6cb0101cd0 2024-02-17T12:47:25.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6cb616b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb019f020 con 0x7f6cb0101310 2024-02-17T12:47:25.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6caf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cb0100120 0x7f6cb0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:25.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6caf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cb0100120 0x7f6cb0197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40920/0 (socket says 172.21.15.18:40920) 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6caf7fe640 1 -- 172.21.15.18:0/3456274170 learned_addr learned my addr 172.21.15.18:0/3456274170 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6caeffd640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cb0101310 0x7f6cb0197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6caffff640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cb0101cd0 0x7f6cb019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6caf7fe640 1 -- 172.21.15.18:0/3456274170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cb0101310 msgr2=0x7f6cb0197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6caf7fe640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cb0101310 0x7f6cb0197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6caf7fe640 1 -- 172.21.15.18:0/3456274170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cb0101cd0 msgr2=0x7f6cb019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6caf7fe640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cb0101cd0 0x7f6cb019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6caf7fe640 1 -- 172.21.15.18:0/3456274170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8009900 con 0x7f6cb0100120 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6caffff640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cb0101cd0 0x7f6cb019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.383+0000 7f6caf7fe640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cb0100120 0x7f6cb0197200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000e9c0 tx=0x7f6ca000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.384+0000 7f6cacff9640 1 -- 172.21.15.18:0/3456274170 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00096e0 con 0x7f6cb0100120 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.384+0000 7f6cb616b640 1 -- 172.21.15.18:0/3456274170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb019f300 con 0x7f6cb0100120 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.384+0000 7f6cacff9640 1 -- 172.21.15.18:0/3456274170 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ca000cd80 con 0x7f6cb0100120 2024-02-17T12:47:25.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.384+0000 7f6cacff9640 1 -- 172.21.15.18:0/3456274170 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0010430 con 0x7f6cb0100120 2024-02-17T12:47:25.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.384+0000 7f6cb616b640 1 -- 172.21.15.18:0/3456274170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb019f7c0 con 0x7f6cb0100120 2024-02-17T12:47:25.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.385+0000 7f6cb616b640 1 -- 172.21.15.18:0/3456274170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb010bac0 con 0x7f6cb0100120 2024-02-17T12:47:25.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.386+0000 7f6cacff9640 1 -- 172.21.15.18:0/3456274170 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6ca0003680 con 0x7f6cb0100120 2024-02-17T12:47:25.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.386+0000 7f6cacff9640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6c980a20c0 0x7f6c980a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:25.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.387+0000 7f6caeffd640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6c980a20c0 0x7f6c980a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:25.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.387+0000 7f6cacff9640 1 -- 172.21.15.18:0/3456274170 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6ca0014070 con 0x7f6cb0100120 2024-02-17T12:47:25.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.387+0000 7f6caeffd640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6c980a20c0 0x7f6c980a4580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0198710 tx=0x7f6ca4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:25.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.390+0000 7f6cacff9640 1 -- 172.21.15.18:0/3456274170 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6ca0097150 con 0x7f6cb0100120 2024-02-17T12:47:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.610+0000 7f6cb616b640 1 -- 172.21.15.18:0/3456274170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cb019fa40 con 0x7f6cb0100120 2024-02-17T12:47:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.610+0000 7f6cacff9640 1 -- 172.21.15.18:0/3456274170 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5283 (secure 0 0 0) 0x7f6ca009a810 con 0x7f6cb0100120 2024-02-17T12:47:25.604 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:25.604 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-02-17T12:47:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 7 2024-02-17T12:47:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.612+0000 7f6cb616b640 1 -- 172.21.15.18:0/3456274170 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6c980a20c0 msgr2=0x7f6c980a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.612+0000 7f6cb616b640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6c980a20c0 0x7f6c980a4580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0198710 tx=0x7f6ca4005f90 comp rx=0 tx=0).stop 2024-02-17T12:47:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.612+0000 7f6cb616b640 1 -- 172.21.15.18:0/3456274170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cb0100120 msgr2=0x7f6cb0197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.612+0000 7f6cb616b640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cb0100120 0x7f6cb0197200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000e9c0 tx=0x7f6ca000ee80 comp rx=0 tx=0).stop 2024-02-17T12:47:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.612+0000 7f6cb616b640 1 -- 172.21.15.18:0/3456274170 shutdown_connections 2024-02-17T12:47:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.612+0000 7f6cb616b640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6c980a20c0 0x7f6c980a4580 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.612+0000 7f6cb616b640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cb0101cd0 0x7f6cb019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.612+0000 7f6cb616b640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cb0101310 0x7f6cb0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.612+0000 7f6cb616b640 1 --2- 172.21.15.18:0/3456274170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cb0100120 0x7f6cb0197200 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.612+0000 7f6cb616b640 1 -- 172.21.15.18:0/3456274170 >> 172.21.15.18:0/3456274170 conn(0x7f6cb00fb6b0 msgr2=0x7f6cb00fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.613+0000 7f6cb616b640 1 -- 172.21.15.18:0/3456274170 shutdown_connections 2024-02-17T12:47:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:25.613+0000 7f6cb616b640 1 -- 172.21.15.18:0/3456274170 wait complete. 2024-02-17T12:47:25.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:25 smithi119 ceph-mon[36188]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:25.883 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:25 smithi146 ceph-mon[36454]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:26.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:25 smithi018 ceph-mon[32625]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:26.202 INFO:journalctl@ceph.mds.c.smithi146.stdout:Feb 17 12:47:26 smithi146 systemd[1]: Started Ceph mds.c for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:47:26.836 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:26 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3456274170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:26.836 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:26 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:26.836 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:26 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:26.836 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:26 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:26.836 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:26 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:26.837 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:26 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:26.837 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:26 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:26.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:26 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3456274170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:26.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:26 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:26.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:26 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:26.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:26 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:26.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:26 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:26.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:26 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:26.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:26 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:27.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:26 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3456274170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:27.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:26 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:27.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:26 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:27.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:26 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:27.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:26 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:27.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:26 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:27.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:26 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:27.617 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:27.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.740+0000 7f9435278640 1 -- 172.21.15.18:0/2940529559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9430101cd0 msgr2=0x7f9430108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:27.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.740+0000 7f9435278640 1 --2- 172.21.15.18:0/2940529559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9430101cd0 0x7f9430108560 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9428009ca0 tx=0x7f942802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:47:27.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.740+0000 7f9435278640 1 -- 172.21.15.18:0/2940529559 shutdown_connections 2024-02-17T12:47:27.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.740+0000 7f9435278640 1 --2- 172.21.15.18:0/2940529559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9430101cd0 0x7f9430108560 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:27.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.740+0000 7f9435278640 1 --2- 172.21.15.18:0/2940529559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9430101310 0x7f9430101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:27.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.740+0000 7f9435278640 1 --2- 172.21.15.18:0/2940529559 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9430100120 0x7f9430100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:27.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.740+0000 7f9435278640 1 -- 172.21.15.18:0/2940529559 >> 172.21.15.18:0/2940529559 conn(0x7f94300fb6b0 msgr2=0x7f94300fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:27.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.740+0000 7f9435278640 1 -- 172.21.15.18:0/2940529559 shutdown_connections 2024-02-17T12:47:27.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.740+0000 7f9435278640 1 -- 172.21.15.18:0/2940529559 wait complete. 2024-02-17T12:47:27.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f9435278640 1 Processor -- start 2024-02-17T12:47:27.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f9435278640 1 -- start start 2024-02-17T12:47:27.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f9435278640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9430100120 0x7f94301971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:27.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f9435278640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9430101310 0x7f94301976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:27.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f9435278640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9430101cd0 0x7f943019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:27.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f9435278640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943019eca0 con 0x7f9430100120 2024-02-17T12:47:27.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f9435278640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943019ee10 con 0x7f9430101cd0 2024-02-17T12:47:27.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f9435278640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943019efd0 con 0x7f9430101310 2024-02-17T12:47:27.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f942effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9430100120 0x7f94301971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:27.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f942effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9430100120 0x7f94301971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40940/0 (socket says 172.21.15.18:40940) 2024-02-17T12:47:27.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f942effd640 1 -- 172.21.15.18:0/2735334657 learned_addr learned my addr 172.21.15.18:0/2735334657 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:27.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f9427fff640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9430101310 0x7f94301976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:27.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f942effd640 1 -- 172.21.15.18:0/2735334657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9430101310 msgr2=0x7f94301976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:27.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.741+0000 7f942effd640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9430101310 0x7f94301976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:27.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.742+0000 7f942effd640 1 -- 172.21.15.18:0/2735334657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9430101cd0 msgr2=0x7f943019e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:27.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.742+0000 7f942effd640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9430101cd0 0x7f943019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:27.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.742+0000 7f942effd640 1 -- 172.21.15.18:0/2735334657 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9428009900 con 0x7f9430100120 2024-02-17T12:47:27.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.742+0000 7f942effd640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9430100120 0x7f94301971b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f942000d950 tx=0x7f942000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:27.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.742+0000 7f942cff9640 1 -- 172.21.15.18:0/2735334657 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420009c30 con 0x7f9430100120 2024-02-17T12:47:27.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.742+0000 7f942cff9640 1 -- 172.21.15.18:0/2735334657 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9420009d90 con 0x7f9430100120 2024-02-17T12:47:27.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.742+0000 7f9435278640 1 -- 172.21.15.18:0/2735334657 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943019f2b0 con 0x7f9430100120 2024-02-17T12:47:27.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.742+0000 7f942cff9640 1 -- 172.21.15.18:0/2735334657 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942000f720 con 0x7f9430100120 2024-02-17T12:47:27.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.742+0000 7f9435278640 1 -- 172.21.15.18:0/2735334657 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943019f770 con 0x7f9430100120 2024-02-17T12:47:27.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.743+0000 7f9435278640 1 -- 172.21.15.18:0/2735334657 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9430107440 con 0x7f9430100120 2024-02-17T12:47:27.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.744+0000 7f942cff9640 1 -- 172.21.15.18:0/2735334657 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9420010a60 con 0x7f9430100120 2024-02-17T12:47:27.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.744+0000 7f942cff9640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f941c0a20c0 0x7f941c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:27.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.745+0000 7f9427fff640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f941c0a20c0 0x7f941c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:27.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.745+0000 7f942cff9640 1 -- 172.21.15.18:0/2735334657 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f94200cd590 con 0x7f9430100120 2024-02-17T12:47:27.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.745+0000 7f9427fff640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f941c0a20c0 0x7f941c0a4580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94301986c0 tx=0x7f941800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:27.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.748+0000 7f942cff9640 1 -- 172.21.15.18:0/2735334657 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9420096960 con 0x7f9430100120 2024-02-17T12:47:27.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:27 smithi119 ceph-mon[36188]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:27.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:27 smithi119 ceph-mon[36188]: Deploying daemon mds.h on smithi119 2024-02-17T12:47:27.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:27 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.146:6836/1732085011,v1:172.21.15.146:6837/1732085011] up:boot 2024-02-17T12:47:27.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:27 smithi119 ceph-mon[36188]: fsmap 7 up:standby 2024-02-17T12:47:27.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:27 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T12:47:27.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:27 smithi146 ceph-mon[36454]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:27.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:27 smithi146 ceph-mon[36454]: Deploying daemon mds.h on smithi119 2024-02-17T12:47:27.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:27 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.146:6836/1732085011,v1:172.21.15.146:6837/1732085011] up:boot 2024-02-17T12:47:27.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:27 smithi146 ceph-mon[36454]: fsmap 7 up:standby 2024-02-17T12:47:27.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:27 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T12:47:27.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.967+0000 7f9435278640 1 -- 172.21.15.18:0/2735334657 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f943010cbc0 con 0x7f9430100120 2024-02-17T12:47:27.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.968+0000 7f942cff9640 1 -- 172.21.15.18:0/2735334657 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6077 (secure 0 0 0) 0x7f942009a020 con 0x7f9430100120 2024-02-17T12:47:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:27.961 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-17T12:47:27.961 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 8 2024-02-17T12:47:27.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.970+0000 7f9435278640 1 -- 172.21.15.18:0/2735334657 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f941c0a20c0 msgr2=0x7f941c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:27.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.970+0000 7f9435278640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f941c0a20c0 0x7f941c0a4580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94301986c0 tx=0x7f941800b390 comp rx=0 tx=0).stop 2024-02-17T12:47:27.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.970+0000 7f9435278640 1 -- 172.21.15.18:0/2735334657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9430100120 msgr2=0x7f94301971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:27.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.970+0000 7f9435278640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9430100120 0x7f94301971b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f942000d950 tx=0x7f942000de10 comp rx=0 tx=0).stop 2024-02-17T12:47:27.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.970+0000 7f9435278640 1 -- 172.21.15.18:0/2735334657 shutdown_connections 2024-02-17T12:47:27.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.970+0000 7f9435278640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f941c0a20c0 0x7f941c0a4580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:27.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.970+0000 7f9435278640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9430101cd0 0x7f943019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:27.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.970+0000 7f9435278640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9430101310 0x7f94301976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:27.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.970+0000 7f9435278640 1 --2- 172.21.15.18:0/2735334657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9430100120 0x7f94301971b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:27.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.970+0000 7f9435278640 1 -- 172.21.15.18:0/2735334657 >> 172.21.15.18:0/2735334657 conn(0x7f94300fb6b0 msgr2=0x7f9430104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:27.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.970+0000 7f9435278640 1 -- 172.21.15.18:0/2735334657 shutdown_connections 2024-02-17T12:47:27.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:27.970+0000 7f9435278640 1 -- 172.21.15.18:0/2735334657 wait complete. 2024-02-17T12:47:27.973 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:27 smithi018 ceph-mon[32625]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:27.973 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:27 smithi018 ceph-mon[32625]: Deploying daemon mds.h on smithi119 2024-02-17T12:47:27.973 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:27 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.146:6836/1732085011,v1:172.21.15.146:6837/1732085011] up:boot 2024-02-17T12:47:27.973 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:27 smithi018 ceph-mon[32625]: fsmap 7 up:standby 2024-02-17T12:47:27.973 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:27 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T12:47:28.527 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 12:47:28 smithi119 systemd[1]: Starting Ceph mds.h for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:47:28.837 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:28 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2735334657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:28.838 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 12:47:28 smithi119 podman[66973]: 2024-02-17 12:47:28.438009556 +0000 UTC m=+0.017887684 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:47:28.838 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 12:47:28 smithi119 podman[66973]: 2024-02-17 12:47:28.593054193 +0000 UTC m=+0.172932308 container create a6a555ffabe7369da98155cd093947b26b39a5be8621ec8e281091dece695312 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-02-17T12:47:28.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:28 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2735334657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:29.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:28 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2735334657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:29.094 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 12:47:28 smithi119 podman[66973]: 2024-02-17 12:47:28.845474128 +0000 UTC m=+0.425352252 container init a6a555ffabe7369da98155cd093947b26b39a5be8621ec8e281091dece695312 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h, org.label-schema.build-date=20240214, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-17T12:47:29.094 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 12:47:28 smithi119 podman[66973]: 2024-02-17 12:47:28.848253959 +0000 UTC m=+0.428132082 container start a6a555ffabe7369da98155cd093947b26b39a5be8621ec8e281091dece695312 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T12:47:29.094 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 12:47:28 smithi119 ceph-mds[66992]: -- 172.21.15.119:0/3298444349 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e3c7781a0 con 0x560e3c74e800 2024-02-17T12:47:29.095 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 12:47:28 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: starting mds.h at 2024-02-17T12:47:29.095 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 12:47:28 smithi119 bash[66973]: a6a555ffabe7369da98155cd093947b26b39a5be8621ec8e281091dece695312 2024-02-17T12:47:29.095 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 12:47:28 smithi119 systemd[1]: Started Ceph mds.h for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:47:29.975 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:30.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.097+0000 7fb011302640 1 -- 172.21.15.18:0/2930277349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb00c101310 msgr2=0x7fb00c101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:30.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.097+0000 7fb011302640 1 --2- 172.21.15.18:0/2930277349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb00c101310 0x7fb00c101790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faff4009900 tx=0x7faff402bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:30.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.097+0000 7fb011302640 1 -- 172.21.15.18:0/2930277349 shutdown_connections 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.097+0000 7fb011302640 1 --2- 172.21.15.18:0/2930277349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb00c101cd0 0x7fb00c108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.097+0000 7fb011302640 1 --2- 172.21.15.18:0/2930277349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb00c101310 0x7fb00c101790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.097+0000 7fb011302640 1 --2- 172.21.15.18:0/2930277349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00c100120 0x7fb00c100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.097+0000 7fb011302640 1 -- 172.21.15.18:0/2930277349 >> 172.21.15.18:0/2930277349 conn(0x7fb00c0fb6b0 msgr2=0x7fb00c0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.097+0000 7fb011302640 1 -- 172.21.15.18:0/2930277349 shutdown_connections 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.097+0000 7fb011302640 1 -- 172.21.15.18:0/2930277349 wait complete. 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb011302640 1 Processor -- start 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb011302640 1 -- start start 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb011302640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb00c100120 0x7fb00c1040b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb011302640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00c101310 0x7fb00c1026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb011302640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb00c101cd0 0x7fb00c102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb011302640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c103390 con 0x7fb00c100120 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb011302640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c103500 con 0x7fb00c101310 2024-02-17T12:47:30.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb011302640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c1a1240 con 0x7fb00c101cd0 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb00affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb00c100120 0x7fb00c1040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb00affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb00c100120 0x7fb00c1040b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34148/0 (socket says 172.21.15.18:34148) 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb00affd640 1 -- 172.21.15.18:0/2935846912 learned_addr learned my addr 172.21.15.18:0/2935846912 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb00b7fe640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb00c101cd0 0x7fb00c102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb00affd640 1 -- 172.21.15.18:0/2935846912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb00c101cd0 msgr2=0x7fb00c102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb00affd640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb00c101cd0 0x7fb00c102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb00affd640 1 -- 172.21.15.18:0/2935846912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00c101310 msgr2=0x7fb00c1026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb00affd640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00c101310 0x7fb00c1026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb00affd640 1 -- 172.21.15.18:0/2935846912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff4009560 con 0x7fb00c100120 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.098+0000 7fb00affd640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb00c100120 0x7fb00c1040b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faffc00c9a0 tx=0x7faffc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.099+0000 7fb008ff9640 1 -- 172.21.15.18:0/2935846912 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc007c10 con 0x7fb00c100120 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.099+0000 7fb008ff9640 1 -- 172.21.15.18:0/2935846912 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faffc007d70 con 0x7fb00c100120 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.099+0000 7fb011302640 1 -- 172.21.15.18:0/2935846912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c1a14f0 con 0x7fb00c100120 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.099+0000 7fb008ff9640 1 -- 172.21.15.18:0/2935846912 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc00f910 con 0x7fb00c100120 2024-02-17T12:47:30.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.099+0000 7fb011302640 1 -- 172.21.15.18:0/2935846912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c1a1a50 con 0x7fb00c100120 2024-02-17T12:47:30.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.100+0000 7fb011302640 1 -- 172.21.15.18:0/2935846912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c10bac0 con 0x7fb00c100120 2024-02-17T12:47:30.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.101+0000 7fb008ff9640 1 -- 172.21.15.18:0/2935846912 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7faffc00fa70 con 0x7fb00c100120 2024-02-17T12:47:30.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.102+0000 7fb008ff9640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7faff80a2140 0x7faff80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:30.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.102+0000 7fb003fff640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7faff80a2140 0x7faff80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:30.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.102+0000 7fb008ff9640 1 -- 172.21.15.18:0/2935846912 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faffc013070 con 0x7fb00c100120 2024-02-17T12:47:30.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.102+0000 7fb003fff640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7faff80a2140 0x7faff80a4600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faff402c6a0 tx=0x7faff4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:30.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.106+0000 7fb008ff9640 1 -- 172.21.15.18:0/2935846912 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faffc096fd0 con 0x7fb00c100120 2024-02-17T12:47:30.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:29 smithi146 ceph-mon[36454]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:30.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:29 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:30.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:29 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:30.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:29 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:30.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:29 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:30.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:29 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:30.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:29 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:30.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:29 smithi146 ceph-mon[36454]: Deploying daemon mds.i on smithi146 2024-02-17T12:47:30.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:29 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] up:boot 2024-02-17T12:47:30.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:29 smithi146 ceph-mon[36454]: fsmap 8 up:standby 2024-02-17T12:47:30.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:29 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T12:47:30.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:29 smithi018 ceph-mon[32625]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:30.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:29 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:30.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:29 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:30.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:29 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:30.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:29 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:30.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:29 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:30.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:29 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:30.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:29 smithi018 ceph-mon[32625]: Deploying daemon mds.i on smithi146 2024-02-17T12:47:30.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:29 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] up:boot 2024-02-17T12:47:30.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:29 smithi018 ceph-mon[32625]: fsmap 8 up:standby 2024-02-17T12:47:30.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:29 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T12:47:30.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.326+0000 7fb011302640 1 -- 172.21.15.18:0/2935846912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb00c10bc50 con 0x7fb00c100120 2024-02-17T12:47:30.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.326+0000 7fb008ff9640 1 -- 172.21.15.18:0/2935846912 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6868 (secure 0 0 0) 0x7faffc09a690 con 0x7fb00c100120 2024-02-17T12:47:30.319 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:30.319 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-02-17T12:47:30.320 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 9 2024-02-17T12:47:30.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.328+0000 7fb011302640 1 -- 172.21.15.18:0/2935846912 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7faff80a2140 msgr2=0x7faff80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:30.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.328+0000 7fb011302640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7faff80a2140 0x7faff80a4600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faff402c6a0 tx=0x7faff4000bc0 comp rx=0 tx=0).stop 2024-02-17T12:47:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.328+0000 7fb011302640 1 -- 172.21.15.18:0/2935846912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb00c100120 msgr2=0x7fb00c1040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.328+0000 7fb011302640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb00c100120 0x7fb00c1040b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faffc00c9a0 tx=0x7faffc00ce60 comp rx=0 tx=0).stop 2024-02-17T12:47:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.328+0000 7fb011302640 1 -- 172.21.15.18:0/2935846912 shutdown_connections 2024-02-17T12:47:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.328+0000 7fb011302640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7faff80a2140 0x7faff80a4600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.328+0000 7fb011302640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb00c101cd0 0x7fb00c102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.328+0000 7fb011302640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00c101310 0x7fb00c1026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.328+0000 7fb011302640 1 --2- 172.21.15.18:0/2935846912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb00c100120 0x7fb00c1040b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.328+0000 7fb011302640 1 -- 172.21.15.18:0/2935846912 >> 172.21.15.18:0/2935846912 conn(0x7fb00c0fb6b0 msgr2=0x7fb00c0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.328+0000 7fb011302640 1 -- 172.21.15.18:0/2935846912 shutdown_connections 2024-02-17T12:47:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:30.328+0000 7fb011302640 1 -- 172.21.15.18:0/2935846912 wait complete. 2024-02-17T12:47:30.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:29 smithi119 ceph-mon[36188]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:30.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:29 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:30.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:29 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:30.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:29 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:30.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:29 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:30.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:29 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:30.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:29 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:30.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:29 smithi119 ceph-mon[36188]: Deploying daemon mds.i on smithi146 2024-02-17T12:47:30.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:29 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] up:boot 2024-02-17T12:47:30.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:29 smithi119 ceph-mon[36188]: fsmap 8 up:standby 2024-02-17T12:47:30.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:29 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T12:47:31.189 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:30 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2935846912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:31.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:30 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2935846912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:31.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:30 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2935846912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:31.701 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 12:47:31 smithi146 systemd[1]: Started Ceph mds.i for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:47:32.247 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:31 smithi018 ceph-mon[32625]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:32.247 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:31 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:32.247 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:31 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:32.247 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:31 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:32.247 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:31 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:32.247 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:31 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:32.247 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:31 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:32.333 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:32.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:31 smithi119 ceph-mon[36188]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:32.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:31 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:32.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:31 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:32.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:31 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:32.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:31 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:32.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:31 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:32.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:31 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.448+0000 7f3e7514b640 1 -- 172.21.15.18:0/2263537072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e70101cd0 msgr2=0x7f3e70108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.448+0000 7f3e7514b640 1 --2- 172.21.15.18:0/2263537072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e70101cd0 0x7f3e70108560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009900 tx=0x7f3e6802bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.448+0000 7f3e7514b640 1 -- 172.21.15.18:0/2263537072 shutdown_connections 2024-02-17T12:47:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.448+0000 7f3e7514b640 1 --2- 172.21.15.18:0/2263537072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e70101cd0 0x7f3e70108560 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:32.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.448+0000 7f3e7514b640 1 --2- 172.21.15.18:0/2263537072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e70101310 0x7f3e70101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:32.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.448+0000 7f3e7514b640 1 --2- 172.21.15.18:0/2263537072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e70100120 0x7f3e70100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:32.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.448+0000 7f3e7514b640 1 -- 172.21.15.18:0/2263537072 >> 172.21.15.18:0/2263537072 conn(0x7f3e700fb6b0 msgr2=0x7f3e700fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:32.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.448+0000 7f3e7514b640 1 -- 172.21.15.18:0/2263537072 shutdown_connections 2024-02-17T12:47:32.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.448+0000 7f3e7514b640 1 -- 172.21.15.18:0/2263537072 wait complete. 2024-02-17T12:47:32.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.448+0000 7f3e7514b640 1 Processor -- start 2024-02-17T12:47:32.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e7514b640 1 -- start start 2024-02-17T12:47:32.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e7514b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e70100120 0x7f3e7006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:32.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e7514b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e70101310 0x7f3e7019bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:32.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e7514b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e70101cd0 0x7f3e7019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:32.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e7514b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7019ed00 con 0x7f3e70101cd0 2024-02-17T12:47:32.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e7514b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7019ee70 con 0x7f3e70101310 2024-02-17T12:47:32.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e7514b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7019f030 con 0x7f3e70100120 2024-02-17T12:47:32.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e6f59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e70101cd0 0x7f3e7019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:32.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e6f59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e70101cd0 0x7f3e7019e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34174/0 (socket says 172.21.15.18:34174) 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e6f59e640 1 -- 172.21.15.18:0/691803607 learned_addr learned my addr 172.21.15.18:0/691803607 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e6ed9d640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e70100120 0x7f3e7006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e6e59c640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e70101310 0x7f3e7019bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e6f59e640 1 -- 172.21.15.18:0/691803607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e70100120 msgr2=0x7f3e7006a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e6f59e640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e70100120 0x7f3e7006a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e6f59e640 1 -- 172.21.15.18:0/691803607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e70101310 msgr2=0x7f3e7019bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e6f59e640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e70101310 0x7f3e7019bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e6f59e640 1 -- 172.21.15.18:0/691803607 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e68009560 con 0x7f3e70101cd0 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.449+0000 7f3e6f59e640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e70101cd0 0x7f3e7019e520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009230 tx=0x7f3e68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.450+0000 7f3e57fff640 1 -- 172.21.15.18:0/691803607 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e68004df0 con 0x7f3e70101cd0 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.450+0000 7f3e7514b640 1 -- 172.21.15.18:0/691803607 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7019f250 con 0x7f3e70101cd0 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.450+0000 7f3e57fff640 1 -- 172.21.15.18:0/691803607 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e68005430 con 0x7f3e70101cd0 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.450+0000 7f3e57fff640 1 -- 172.21.15.18:0/691803607 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e68003de0 con 0x7f3e70101cd0 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.450+0000 7f3e7514b640 1 -- 172.21.15.18:0/691803607 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7019f760 con 0x7f3e70101cd0 2024-02-17T12:47:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.451+0000 7f3e7514b640 1 -- 172.21.15.18:0/691803607 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e70107440 con 0x7f3e70101cd0 2024-02-17T12:47:32.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.452+0000 7f3e57fff640 1 -- 172.21.15.18:0/691803607 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3e680055a0 con 0x7f3e70101cd0 2024-02-17T12:47:32.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.452+0000 7f3e57fff640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f3e580a20c0 0x7f3e580a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:32.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.452+0000 7f3e6ed9d640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f3e580a20c0 0x7f3e580a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:32.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.453+0000 7f3e57fff640 1 -- 172.21.15.18:0/691803607 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3e680efd50 con 0x7f3e70101cd0 2024-02-17T12:47:32.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.453+0000 7f3e6ed9d640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f3e580a20c0 0x7f3e580a4580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3e60005fd0 tx=0x7f3e6000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:32.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.457+0000 7f3e57fff640 1 -- 172.21.15.18:0/691803607 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e680f3030 con 0x7f3e70101cd0 2024-02-17T12:47:32.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:31 smithi146 ceph-mon[36454]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:32.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:31 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:32.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:31 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:32.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:31 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:32.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:31 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:32.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:31 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:32.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:31 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:32.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.678+0000 7f3e7514b640 1 -- 172.21.15.18:0/691803607 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e7010bc70 con 0x7f3e70101cd0 2024-02-17T12:47:32.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.678+0000 7f3e57fff640 1 -- 172.21.15.18:0/691803607 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7664 (secure 0 0 0) 0x7f3e680b91c0 con 0x7f3e70101cd0 2024-02-17T12:47:32.672 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:32.672 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-02-17T12:47:32.673 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 10 2024-02-17T12:47:32.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.680+0000 7f3e7514b640 1 -- 172.21.15.18:0/691803607 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f3e580a20c0 msgr2=0x7f3e580a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:32.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.680+0000 7f3e7514b640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f3e580a20c0 0x7f3e580a4580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3e60005fd0 tx=0x7f3e6000b390 comp rx=0 tx=0).stop 2024-02-17T12:47:32.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.680+0000 7f3e7514b640 1 -- 172.21.15.18:0/691803607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e70101cd0 msgr2=0x7f3e7019e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:32.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.680+0000 7f3e7514b640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e70101cd0 0x7f3e7019e520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009230 tx=0x7f3e68004270 comp rx=0 tx=0).stop 2024-02-17T12:47:32.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.680+0000 7f3e7514b640 1 -- 172.21.15.18:0/691803607 shutdown_connections 2024-02-17T12:47:32.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.680+0000 7f3e7514b640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f3e580a20c0 0x7f3e580a4580 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:32.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.680+0000 7f3e7514b640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e70101cd0 0x7f3e7019e520 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:32.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.680+0000 7f3e7514b640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e70101310 0x7f3e7019bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:32.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.680+0000 7f3e7514b640 1 --2- 172.21.15.18:0/691803607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e70100120 0x7f3e7006a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:32.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.681+0000 7f3e7514b640 1 -- 172.21.15.18:0/691803607 >> 172.21.15.18:0/691803607 conn(0x7f3e700fb6b0 msgr2=0x7f3e70104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:32.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.681+0000 7f3e7514b640 1 -- 172.21.15.18:0/691803607 shutdown_connections 2024-02-17T12:47:32.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:32.681+0000 7f3e7514b640 1 -- 172.21.15.18:0/691803607 wait complete. 2024-02-17T12:47:33.245 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:32 smithi119 ceph-mon[36188]: Deploying daemon mds.g on smithi018 2024-02-17T12:47:33.245 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:32 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.146:6838/3061839650,v1:172.21.15.146:6839/3061839650] up:boot 2024-02-17T12:47:33.245 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:32 smithi119 ceph-mon[36188]: fsmap 9 up:standby 2024-02-17T12:47:33.245 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:32 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T12:47:33.245 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:32 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/691803607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:33.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:32 smithi018 ceph-mon[32625]: Deploying daemon mds.g on smithi018 2024-02-17T12:47:33.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:32 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.146:6838/3061839650,v1:172.21.15.146:6839/3061839650] up:boot 2024-02-17T12:47:33.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:32 smithi018 ceph-mon[32625]: fsmap 9 up:standby 2024-02-17T12:47:33.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:32 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T12:47:33.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:32 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/691803607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:33.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:32 smithi146 ceph-mon[36454]: Deploying daemon mds.g on smithi018 2024-02-17T12:47:33.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:32 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.146:6838/3061839650,v1:172.21.15.146:6839/3061839650] up:boot 2024-02-17T12:47:33.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:32 smithi146 ceph-mon[36454]: fsmap 9 up:standby 2024-02-17T12:47:33.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:32 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T12:47:33.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:32 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/691803607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:33.986 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 12:47:33 smithi018 systemd[1]: Starting Ceph mds.g for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:47:34.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:33 smithi018 ceph-mon[32625]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:34.275 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 12:47:34 smithi018 podman[74149]: 2024-02-17 12:47:33.963109817 +0000 UTC m=+0.017738534 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:47:34.275 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 12:47:34 smithi018 podman[74149]: 2024-02-17 12:47:34.210731619 +0000 UTC m=+0.265360331 container create b924061d027f877776c491a7ec820dea1687e8a8bb6dfaa9fe38ff1399472678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-17T12:47:34.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:33 smithi119 ceph-mon[36188]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:34.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:33 smithi146 ceph-mon[36454]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:34.688 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:34.714 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 12:47:34 smithi018 podman[74149]: 2024-02-17 12:47:34.455076244 +0000 UTC m=+0.509704957 container init b924061d027f877776c491a7ec820dea1687e8a8bb6dfaa9fe38ff1399472678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=) 2024-02-17T12:47:34.714 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 12:47:34 smithi018 podman[74149]: 2024-02-17 12:47:34.457690895 +0000 UTC m=+0.512319616 container start b924061d027f877776c491a7ec820dea1687e8a8bb6dfaa9fe38ff1399472678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:47:34.714 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 12:47:34 smithi018 ceph-mds[74168]: -- 172.21.15.18:0/4274441783 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5630b41541a0 con 0x5630b412a800 2024-02-17T12:47:34.714 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 12:47:34 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g[74164]: starting mds.g at 2024-02-17T12:47:34.714 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 12:47:34 smithi018 bash[74149]: b924061d027f877776c491a7ec820dea1687e8a8bb6dfaa9fe38ff1399472678 2024-02-17T12:47:34.714 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 12:47:34 smithi018 systemd[1]: Started Ceph mds.g for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:47:34.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.804+0000 7f6025b65640 1 -- 172.21.15.18:0/3743481257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6020106e80 msgr2=0x7f60201092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:34.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.804+0000 7f6025b65640 1 --2- 172.21.15.18:0/3743481257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6020106e80 0x7f60201092d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6018009ca0 tx=0x7f601802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:47:34.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.804+0000 7f6025b65640 1 -- 172.21.15.18:0/3743481257 shutdown_connections 2024-02-17T12:47:34.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.804+0000 7f6025b65640 1 --2- 172.21.15.18:0/3743481257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6020106e80 0x7f60201092d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:34.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.804+0000 7f6025b65640 1 --2- 172.21.15.18:0/3743481257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6020104550 0x7f6020106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:34.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.804+0000 7f6025b65640 1 --2- 172.21.15.18:0/3743481257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6020101c20 0x7f6020104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:34.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.804+0000 7f6025b65640 1 -- 172.21.15.18:0/3743481257 >> 172.21.15.18:0/3743481257 conn(0x7f60200fb630 msgr2=0x7f60200fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:34.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.804+0000 7f6025b65640 1 -- 172.21.15.18:0/3743481257 shutdown_connections 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.804+0000 7f6025b65640 1 -- 172.21.15.18:0/3743481257 wait complete. 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.804+0000 7f6025b65640 1 Processor -- start 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f6025b65640 1 -- start start 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f6025b65640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6020101c20 0x7f6020197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f6025b65640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6020104550 0x7f60201976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f6025b65640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6020106e80 0x7f602019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f6025b65640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602019ec50 con 0x7f6020104550 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f6025b65640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602019edc0 con 0x7f6020101c20 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f6025b65640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602019ef80 con 0x7f6020106e80 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6020104550 0x7f60201976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6020104550 0x7f60201976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34218/0 (socket says 172.21.15.18:34218) 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6020101c20 0x7f6020197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:34.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601effd640 1 -- 172.21.15.18:0/2227618571 learned_addr learned my addr 172.21.15.18:0/2227618571 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601ffff640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6020106e80 0x7f602019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601effd640 1 -- 172.21.15.18:0/2227618571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6020106e80 msgr2=0x7f602019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601effd640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6020106e80 0x7f602019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601effd640 1 -- 172.21.15.18:0/2227618571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6020101c20 msgr2=0x7f6020197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601effd640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6020101c20 0x7f6020197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601effd640 1 -- 172.21.15.18:0/2227618571 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6018009900 con 0x7f6020104550 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601ffff640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6020106e80 0x7f602019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601f7fe640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6020101c20 0x7f6020197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.805+0000 7f601effd640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6020104550 0x7f60201976a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f601000e9c0 tx=0x7f601000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.806+0000 7f601cff9640 1 -- 172.21.15.18:0/2227618571 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010009700 con 0x7f6020104550 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.806+0000 7f6025b65640 1 -- 172.21.15.18:0/2227618571 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602019f260 con 0x7f6020104550 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.806+0000 7f601cff9640 1 -- 172.21.15.18:0/2227618571 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6010005500 con 0x7f6020104550 2024-02-17T12:47:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.806+0000 7f601cff9640 1 -- 172.21.15.18:0/2227618571 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010010430 con 0x7f6020104550 2024-02-17T12:47:34.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.806+0000 7f6025b65640 1 -- 172.21.15.18:0/2227618571 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602019f7c0 con 0x7f6020104550 2024-02-17T12:47:34.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.807+0000 7f6025b65640 1 -- 172.21.15.18:0/2227618571 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60200fd120 con 0x7f6020104550 2024-02-17T12:47:34.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.808+0000 7f601cff9640 1 -- 172.21.15.18:0/2227618571 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f601000ccc0 con 0x7f6020104550 2024-02-17T12:47:34.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.809+0000 7f601cff9640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f60080a20c0 0x7f60080a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:34.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.809+0000 7f601cff9640 1 -- 172.21.15.18:0/2227618571 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6010014070 con 0x7f6020104550 2024-02-17T12:47:34.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.809+0000 7f601f7fe640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f60080a20c0 0x7f60080a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:34.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.809+0000 7f601f7fe640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f60080a20c0 0x7f60080a4580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f601400afb0 tx=0x7f601400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:34.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:34.812+0000 7f601cff9640 1 -- 172.21.15.18:0/2227618571 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6010097260 con 0x7f6020104550 2024-02-17T12:47:35.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.039+0000 7f6025b65640 1 -- 172.21.15.18:0/2227618571 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f602010cbb0 con 0x7f6020104550 2024-02-17T12:47:35.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.039+0000 7f601cff9640 1 -- 172.21.15.18:0/2227618571 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8456 (secure 0 0 0) 0x7f601009a920 con 0x7f6020104550 2024-02-17T12:47:35.032 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:35.032 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-02-17T12:47:35.033 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 11 2024-02-17T12:47:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.042+0000 7f6025b65640 1 -- 172.21.15.18:0/2227618571 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f60080a20c0 msgr2=0x7f60080a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.042+0000 7f6025b65640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f60080a20c0 0x7f60080a4580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f601400afb0 tx=0x7f601400a2d0 comp rx=0 tx=0).stop 2024-02-17T12:47:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.042+0000 7f6025b65640 1 -- 172.21.15.18:0/2227618571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6020104550 msgr2=0x7f60201976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.042+0000 7f6025b65640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6020104550 0x7f60201976a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f601000e9c0 tx=0x7f601000edc0 comp rx=0 tx=0).stop 2024-02-17T12:47:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.042+0000 7f6025b65640 1 -- 172.21.15.18:0/2227618571 shutdown_connections 2024-02-17T12:47:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.042+0000 7f6025b65640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f60080a20c0 0x7f60080a4580 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.042+0000 7f6025b65640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6020106e80 0x7f602019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.042+0000 7f6025b65640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6020104550 0x7f60201976a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.042+0000 7f6025b65640 1 --2- 172.21.15.18:0/2227618571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6020101c20 0x7f6020197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.042+0000 7f6025b65640 1 -- 172.21.15.18:0/2227618571 >> 172.21.15.18:0/2227618571 conn(0x7f60200fb630 msgr2=0x7f6020102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.042+0000 7f6025b65640 1 -- 172.21.15.18:0/2227618571 shutdown_connections 2024-02-17T12:47:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:35.042+0000 7f6025b65640 1 -- 172.21.15.18:0/2227618571 wait complete. 2024-02-17T12:47:35.587 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:35 smithi018 ceph-mon[32625]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:35.587 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:35 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:35.587 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:35 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:35.587 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:35 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:35.587 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:35 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:35.587 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:35 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:35.587 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:35 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:35.587 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:35 smithi018 ceph-mon[32625]: Deploying daemon mds.a on smithi018 2024-02-17T12:47:35.587 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:35 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.18:6838/1294470347,v1:172.21.15.18:6839/1294470347] up:boot 2024-02-17T12:47:35.587 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:35 smithi018 ceph-mon[32625]: fsmap 10 up:standby 2024-02-17T12:47:35.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:35 smithi119 ceph-mon[36188]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:35.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:35 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:35.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:35 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:35.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:35 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:35 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:35 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:35 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:35 smithi119 ceph-mon[36188]: Deploying daemon mds.a on smithi018 2024-02-17T12:47:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:35 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.18:6838/1294470347,v1:172.21.15.18:6839/1294470347] up:boot 2024-02-17T12:47:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:35 smithi119 ceph-mon[36188]: fsmap 10 up:standby 2024-02-17T12:47:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:35 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T12:47:35.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:35 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2227618571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:35.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:35 smithi146 ceph-mon[36454]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:35.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:35 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:35.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:35 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:35.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:35 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:35.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:35 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:35.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:35 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:35.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:35 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:35.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:35 smithi146 ceph-mon[36454]: Deploying daemon mds.a on smithi018 2024-02-17T12:47:35.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:35 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.18:6838/1294470347,v1:172.21.15.18:6839/1294470347] up:boot 2024-02-17T12:47:35.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:35 smithi146 ceph-mon[36454]: fsmap 10 up:standby 2024-02-17T12:47:35.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:35 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T12:47:35.953 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:35 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2227618571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:36.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:35 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T12:47:36.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:35 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2227618571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:36.993 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 12:47:36 smithi018 systemd[1]: Starting Ceph mds.a for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:47:37.047 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:37.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.163+0000 7feb96a81640 1 -- 172.21.15.18:0/3301014296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb901023a0 msgr2=0x7feb90102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:37.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.163+0000 7feb96a81640 1 --2- 172.21.15.18:0/3301014296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb901023a0 0x7feb90102820 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009900 tx=0x7feb7c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:37.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.163+0000 7feb96a81640 1 -- 172.21.15.18:0/3301014296 shutdown_connections 2024-02-17T12:47:37.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.163+0000 7feb96a81640 1 --2- 172.21.15.18:0/3301014296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb90102d60 0x7feb901095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:37.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.163+0000 7feb96a81640 1 --2- 172.21.15.18:0/3301014296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb901023a0 0x7feb90102820 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:37.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.163+0000 7feb96a81640 1 --2- 172.21.15.18:0/3301014296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb901011b0 0x7feb901015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:37.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.163+0000 7feb96a81640 1 -- 172.21.15.18:0/3301014296 >> 172.21.15.18:0/3301014296 conn(0x7feb900fc760 msgr2=0x7feb900feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:37.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.163+0000 7feb96a81640 1 -- 172.21.15.18:0/3301014296 shutdown_connections 2024-02-17T12:47:37.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.163+0000 7feb96a81640 1 -- 172.21.15.18:0/3301014296 wait complete. 2024-02-17T12:47:37.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb96a81640 1 Processor -- start 2024-02-17T12:47:37.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb96a81640 1 -- start start 2024-02-17T12:47:37.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb96a81640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb901011b0 0x7feb90074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:37.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb96a81640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb901023a0 0x7feb90073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:37.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb96a81640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb90102d60 0x7feb90073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:37.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb96a81640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb90074210 con 0x7feb901011b0 2024-02-17T12:47:37.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb96a81640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb90074380 con 0x7feb90102d60 2024-02-17T12:47:37.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb96a81640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb901a1260 con 0x7feb901023a0 2024-02-17T12:47:37.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb9481d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb901011b0 0x7feb90074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:37.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb9481d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb901011b0 0x7feb90074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34234/0 (socket says 172.21.15.18:34234) 2024-02-17T12:47:37.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb9481d640 1 -- 172.21.15.18:0/881012756 learned_addr learned my addr 172.21.15.18:0/881012756 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:37.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb9481d640 1 -- 172.21.15.18:0/881012756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb901023a0 msgr2=0x7feb90073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:37.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb9481d640 1 --2- 172.21.15.18:0/881012756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb901023a0 0x7feb90073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:37.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb9481d640 1 -- 172.21.15.18:0/881012756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb90102d60 msgr2=0x7feb90073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:37.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb9481d640 1 --2- 172.21.15.18:0/881012756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb90102d60 0x7feb90073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:37.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb9481d640 1 -- 172.21.15.18:0/881012756 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c009560 con 0x7feb901011b0 2024-02-17T12:47:37.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.164+0000 7feb9481d640 1 --2- 172.21.15.18:0/881012756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb901011b0 0x7feb90074f30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feb8400c9a0 tx=0x7feb8400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:37.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.165+0000 7feb8dffb640 1 -- 172.21.15.18:0/881012756 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84007c10 con 0x7feb901011b0 2024-02-17T12:47:37.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.165+0000 7feb96a81640 1 -- 172.21.15.18:0/881012756 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb901a1510 con 0x7feb901011b0 2024-02-17T12:47:37.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.165+0000 7feb8dffb640 1 -- 172.21.15.18:0/881012756 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb84007d70 con 0x7feb901011b0 2024-02-17T12:47:37.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.165+0000 7feb8dffb640 1 -- 172.21.15.18:0/881012756 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8400f870 con 0x7feb901011b0 2024-02-17T12:47:37.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.165+0000 7feb96a81640 1 -- 172.21.15.18:0/881012756 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb901a1a20 con 0x7feb901011b0 2024-02-17T12:47:37.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.166+0000 7feb96a81640 1 -- 172.21.15.18:0/881012756 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb90108400 con 0x7feb901011b0 2024-02-17T12:47:37.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.167+0000 7feb8dffb640 1 -- 172.21.15.18:0/881012756 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7feb8400f9f0 con 0x7feb901011b0 2024-02-17T12:47:37.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.167+0000 7feb8dffb640 1 --2- 172.21.15.18:0/881012756 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7feb800a20c0 0x7feb800a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:37.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.167+0000 7feb8ffff640 1 --2- 172.21.15.18:0/881012756 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7feb800a20c0 0x7feb800a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:37.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.167+0000 7feb8dffb640 1 -- 172.21.15.18:0/881012756 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feb84013070 con 0x7feb901011b0 2024-02-17T12:47:37.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.168+0000 7feb8ffff640 1 --2- 172.21.15.18:0/881012756 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7feb800a20c0 0x7feb800a4580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feb7c02c6a0 tx=0x7feb7c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:37.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.172+0000 7feb8dffb640 1 -- 172.21.15.18:0/881012756 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feb840970a0 con 0x7feb901011b0 2024-02-17T12:47:37.270 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 12:47:37 smithi018 podman[74514]: 2024-02-17 12:47:37.000182411 +0000 UTC m=+0.099593879 container create 49bad5597965c24f8e1c58ce7d64aa19fb61a5aeab0f59225f146f84c3b1de02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-17T12:47:37.270 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 12:47:37 smithi018 podman[74514]: 2024-02-17 12:47:36.91824493 +0000 UTC m=+0.017656327 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:47:37.270 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 12:47:37 smithi018 podman[74514]: 2024-02-17 12:47:37.194233591 +0000 UTC m=+0.293644984 container init 49bad5597965c24f8e1c58ce7d64aa19fb61a5aeab0f59225f146f84c3b1de02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-a, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-17T12:47:37.270 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 12:47:37 smithi018 podman[74514]: 2024-02-17 12:47:37.19746856 +0000 UTC m=+0.296879953 container start 49bad5597965c24f8e1c58ce7d64aa19fb61a5aeab0f59225f146f84c3b1de02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b) 2024-02-17T12:47:37.270 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 12:47:37 smithi018 ceph-mds[74572]: -- 172.21.15.18:0/2894461900 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557aa78741a0 con 0x557aa784ac00 2024-02-17T12:47:37.270 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 12:47:37 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-a[74545]: starting mds.a at 2024-02-17T12:47:37.270 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 12:47:37 smithi018 bash[74514]: 49bad5597965c24f8e1c58ce7d64aa19fb61a5aeab0f59225f146f84c3b1de02 2024-02-17T12:47:37.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.398+0000 7feb96a81640 1 -- 172.21.15.18:0/881012756 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb9010cbc0 con 0x7feb901011b0 2024-02-17T12:47:37.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.399+0000 7feb8dffb640 1 -- 172.21.15.18:0/881012756 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8456 (secure 0 0 0) 0x7feb8409a760 con 0x7feb901011b0 2024-02-17T12:47:37.392 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:37.392 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-02-17T12:47:37.392 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 11 2024-02-17T12:47:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.401+0000 7feb96a81640 1 -- 172.21.15.18:0/881012756 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7feb800a20c0 msgr2=0x7feb800a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.401+0000 7feb96a81640 1 --2- 172.21.15.18:0/881012756 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7feb800a20c0 0x7feb800a4580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feb7c02c6a0 tx=0x7feb7c000bc0 comp rx=0 tx=0).stop 2024-02-17T12:47:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.401+0000 7feb96a81640 1 -- 172.21.15.18:0/881012756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb901011b0 msgr2=0x7feb90074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.401+0000 7feb96a81640 1 --2- 172.21.15.18:0/881012756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb901011b0 0x7feb90074f30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feb8400c9a0 tx=0x7feb8400ce60 comp rx=0 tx=0).stop 2024-02-17T12:47:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.401+0000 7feb96a81640 1 -- 172.21.15.18:0/881012756 shutdown_connections 2024-02-17T12:47:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.401+0000 7feb96a81640 1 --2- 172.21.15.18:0/881012756 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7feb800a20c0 0x7feb800a4580 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.401+0000 7feb96a81640 1 --2- 172.21.15.18:0/881012756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb90102d60 0x7feb90073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.401+0000 7feb96a81640 1 --2- 172.21.15.18:0/881012756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb901023a0 0x7feb90073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.401+0000 7feb96a81640 1 --2- 172.21.15.18:0/881012756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb901011b0 0x7feb90074f30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.401+0000 7feb96a81640 1 -- 172.21.15.18:0/881012756 >> 172.21.15.18:0/881012756 conn(0x7feb900fc760 msgr2=0x7feb901055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.401+0000 7feb96a81640 1 -- 172.21.15.18:0/881012756 shutdown_connections 2024-02-17T12:47:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:37.401+0000 7feb96a81640 1 -- 172.21.15.18:0/881012756 wait complete. 2024-02-17T12:47:37.525 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 12:47:37 smithi018 systemd[1]: Started Ceph mds.a for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:47:37.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:37 smithi146 ceph-mon[36454]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:37.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:37.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:37.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:37.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:37.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:37.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:37 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:37.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:37 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/881012756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:37.981 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:37 smithi119 ceph-mon[36188]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:37.981 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:37.981 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:37.981 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:37.981 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:37.981 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:37.982 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:37 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:37.982 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:37 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/881012756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:38.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:37 smithi018 ceph-mon[32625]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:38.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:38.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:38.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:38.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T12:47:38.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T12:47:38.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:37 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:38.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:37 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/881012756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:38.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:38 smithi119 ceph-mon[36188]: Deploying daemon mds.b on smithi119 2024-02-17T12:47:38.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:38 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.18:6840/2569825427,v1:172.21.15.18:6841/2569825427] up:boot 2024-02-17T12:47:38.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:38 smithi119 ceph-mon[36188]: fsmap 11 up:standby 2024-02-17T12:47:38.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:38 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T12:47:38.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:38 smithi146 ceph-mon[36454]: Deploying daemon mds.b on smithi119 2024-02-17T12:47:38.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:38 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.18:6840/2569825427,v1:172.21.15.18:6841/2569825427] up:boot 2024-02-17T12:47:38.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:38 smithi146 ceph-mon[36454]: fsmap 11 up:standby 2024-02-17T12:47:38.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:38 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T12:47:39.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:38 smithi018 ceph-mon[32625]: Deploying daemon mds.b on smithi119 2024-02-17T12:47:39.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:38 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.18:6840/2569825427,v1:172.21.15.18:6841/2569825427] up:boot 2024-02-17T12:47:39.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:38 smithi018 ceph-mon[32625]: fsmap 11 up:standby 2024-02-17T12:47:39.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:38 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T12:47:39.406 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:39.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.529+0000 7f13e069d640 1 -- 172.21.15.18:0/3539531563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d8101ca0 msgr2=0x7f13d8104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:39.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.529+0000 7f13e069d640 1 --2- 172.21.15.18:0/3539531563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d8101ca0 0x7f13d8104090 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009900 tx=0x7f13d002bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:39.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.529+0000 7f13e069d640 1 -- 172.21.15.18:0/3539531563 shutdown_connections 2024-02-17T12:47:39.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.529+0000 7f13e069d640 1 --2- 172.21.15.18:0/3539531563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13d8106f00 0x7f13d8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:39.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.529+0000 7f13e069d640 1 --2- 172.21.15.18:0/3539531563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d81045d0 0x7f13d81069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:39.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.529+0000 7f13e069d640 1 --2- 172.21.15.18:0/3539531563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d8101ca0 0x7f13d8104090 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:39.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.529+0000 7f13e069d640 1 -- 172.21.15.18:0/3539531563 >> 172.21.15.18:0/3539531563 conn(0x7f13d80fb6b0 msgr2=0x7f13d80fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:39.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.529+0000 7f13e069d640 1 -- 172.21.15.18:0/3539531563 shutdown_connections 2024-02-17T12:47:39.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.529+0000 7f13e069d640 1 -- 172.21.15.18:0/3539531563 wait complete. 2024-02-17T12:47:39.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13e069d640 1 Processor -- start 2024-02-17T12:47:39.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13e069d640 1 -- start start 2024-02-17T12:47:39.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13e069d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13d8101ca0 0x7f13d8197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:39.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13e069d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d81045d0 0x7f13d81977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:39.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13e069d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d8106f00 0x7f13d819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:39.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13e069d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d819ed60 con 0x7f13d81045d0 2024-02-17T12:47:39.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13e069d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d819eed0 con 0x7f13d8106f00 2024-02-17T12:47:39.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13e069d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d819f090 con 0x7f13d8101ca0 2024-02-17T12:47:39.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13ddc38640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d81045d0 0x7f13d81977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:39.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13ddc38640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d81045d0 0x7f13d81977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35500/0 (socket says 172.21.15.18:35500) 2024-02-17T12:47:39.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13ddc38640 1 -- 172.21.15.18:0/2175058806 learned_addr learned my addr 172.21.15.18:0/2175058806 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:39.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13dec3a640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d8106f00 0x7f13d819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:39.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13de439640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13d8101ca0 0x7f13d8197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:39.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13ddc38640 1 -- 172.21.15.18:0/2175058806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13d8101ca0 msgr2=0x7f13d8197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:39.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13ddc38640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13d8101ca0 0x7f13d8197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:39.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13ddc38640 1 -- 172.21.15.18:0/2175058806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d8106f00 msgr2=0x7f13d819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:39.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13ddc38640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d8106f00 0x7f13d819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:39.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.530+0000 7f13ddc38640 1 -- 172.21.15.18:0/2175058806 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0009560 con 0x7f13d81045d0 2024-02-17T12:47:39.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.531+0000 7f13ddc38640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d81045d0 0x7f13d81977b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00e9c0 tx=0x7f13cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:39.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.531+0000 7f13d77fe640 1 -- 172.21.15.18:0/2175058806 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc009700 con 0x7f13d81045d0 2024-02-17T12:47:39.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.531+0000 7f13d77fe640 1 -- 172.21.15.18:0/2175058806 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13cc00cd80 con 0x7f13d81045d0 2024-02-17T12:47:39.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.531+0000 7f13e069d640 1 -- 172.21.15.18:0/2175058806 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d819f370 con 0x7f13d81045d0 2024-02-17T12:47:39.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.531+0000 7f13d77fe640 1 -- 172.21.15.18:0/2175058806 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc010430 con 0x7f13d81045d0 2024-02-17T12:47:39.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.531+0000 7f13e069d640 1 -- 172.21.15.18:0/2175058806 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d819f8d0 con 0x7f13d81045d0 2024-02-17T12:47:39.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.532+0000 7f13e069d640 1 -- 172.21.15.18:0/2175058806 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d80fd1a0 con 0x7f13d81045d0 2024-02-17T12:47:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.533+0000 7f13d77fe640 1 -- 172.21.15.18:0/2175058806 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f13cc005000 con 0x7f13d81045d0 2024-02-17T12:47:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.534+0000 7f13d77fe640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f13c00a2140 0x7f13c00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.534+0000 7f13de439640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f13c00a2140 0x7f13c00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.534+0000 7f13d77fe640 1 -- 172.21.15.18:0/2175058806 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f13cc014070 con 0x7f13d81045d0 2024-02-17T12:47:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.534+0000 7f13de439640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f13c00a2140 0x7f13c00a4600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f13d002c840 tx=0x7f13d0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:39.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.538+0000 7f13d77fe640 1 -- 172.21.15.18:0/2175058806 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f13cc0cfe60 con 0x7f13d81045d0 2024-02-17T12:47:39.594 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 12:47:39 smithi119 systemd[1]: Starting Ceph mds.b for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T12:47:39.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.759+0000 7f13e069d640 1 -- 172.21.15.18:0/2175058806 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13d810cbc0 con 0x7f13d81045d0 2024-02-17T12:47:39.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.760+0000 7f13d77fe640 1 -- 172.21.15.18:0/2175058806 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9248 (secure 0 0 0) 0x7f13cc097070 con 0x7f13d81045d0 2024-02-17T12:47:39.753 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:39.753 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-02-17T12:47:39.753 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 12 2024-02-17T12:47:39.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.762+0000 7f13e069d640 1 -- 172.21.15.18:0/2175058806 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f13c00a2140 msgr2=0x7f13c00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:39.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.762+0000 7f13e069d640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f13c00a2140 0x7f13c00a4600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f13d002c840 tx=0x7f13d0000bc0 comp rx=0 tx=0).stop 2024-02-17T12:47:39.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.762+0000 7f13e069d640 1 -- 172.21.15.18:0/2175058806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d81045d0 msgr2=0x7f13d81977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:39.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.762+0000 7f13e069d640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d81045d0 0x7f13d81977b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00e9c0 tx=0x7f13cc00ee80 comp rx=0 tx=0).stop 2024-02-17T12:47:39.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.762+0000 7f13e069d640 1 -- 172.21.15.18:0/2175058806 shutdown_connections 2024-02-17T12:47:39.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.762+0000 7f13e069d640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f13c00a2140 0x7f13c00a4600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:39.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.762+0000 7f13e069d640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d8106f00 0x7f13d819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:39.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.762+0000 7f13e069d640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d81045d0 0x7f13d81977b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:39.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.762+0000 7f13e069d640 1 --2- 172.21.15.18:0/2175058806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13d8101ca0 0x7f13d8197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:39.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.762+0000 7f13e069d640 1 -- 172.21.15.18:0/2175058806 >> 172.21.15.18:0/2175058806 conn(0x7f13d80fb6b0 msgr2=0x7f13d8102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:39.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.762+0000 7f13e069d640 1 -- 172.21.15.18:0/2175058806 shutdown_connections 2024-02-17T12:47:39.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:39.762+0000 7f13e069d640 1 -- 172.21.15.18:0/2175058806 wait complete. 2024-02-17T12:47:39.764 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:39 smithi018 ceph-mon[32625]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:39.891 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:39 smithi119 ceph-mon[36188]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:39.891 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 12:47:39 smithi119 podman[67295]: 2024-02-17 12:47:39.573981042 +0000 UTC m=+0.017226352 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:47:39.891 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 12:47:39 smithi119 podman[67295]: 2024-02-17 12:47:39.688186105 +0000 UTC m=+0.131431454 container create 5365242334d82ead07e8d572b8a18710b6fc389070628189fb083265d39c5435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:47:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:39 smithi146 ceph-mon[36454]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:40.344 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 12:47:39 smithi119 podman[67295]: 2024-02-17 12:47:39.898560265 +0000 UTC m=+0.341805575 container init 5365242334d82ead07e8d572b8a18710b6fc389070628189fb083265d39c5435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-17T12:47:40.344 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 12:47:39 smithi119 podman[67295]: 2024-02-17 12:47:39.901316154 +0000 UTC m=+0.344561466 container start 5365242334d82ead07e8d572b8a18710b6fc389070628189fb083265d39c5435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-17T12:47:40.344 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 12:47:39 smithi119 ceph-mds[67314]: -- 172.21.15.119:0/683683290 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556ad03ba1a0 con 0x556ad0391000 2024-02-17T12:47:40.345 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 12:47:39 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b[67310]: starting mds.b at 2024-02-17T12:47:40.345 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 12:47:39 smithi119 bash[67295]: 5365242334d82ead07e8d572b8a18710b6fc389070628189fb083265d39c5435 2024-02-17T12:47:40.345 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 12:47:40 smithi119 systemd[1]: Started Ceph mds.b for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T12:47:40.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:40 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2175058806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:40.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:40.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:40.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:40.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:40.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:40.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:40.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:40 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:47:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:40 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2175058806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:41.025 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:41.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:41.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:41.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:40 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:47:41.088 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:40 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2175058806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:41.088 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:41.089 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:41.089 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:41.089 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:41.089 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:41.089 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:41.089 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:40 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:47:41.766 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:41.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:41 smithi018 ceph-mon[32625]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:41.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:41 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] up:boot 2024-02-17T12:47:41.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:41 smithi018 ceph-mon[32625]: fsmap 12 up:standby 2024-02-17T12:47:41.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:41 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T12:47:41.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:41 smithi119 ceph-mon[36188]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:41.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:41 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] up:boot 2024-02-17T12:47:41.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:41 smithi119 ceph-mon[36188]: fsmap 12 up:standby 2024-02-17T12:47:41.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:41 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T12:47:41.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.887+0000 7fbee63d7640 1 -- 172.21.15.18:0/2519066657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee0100310 msgr2=0x7fbee0100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:41.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.887+0000 7fbee63d7640 1 --2- 172.21.15.18:0/2519066657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee0100310 0x7fbee0100790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009900 tx=0x7fbed402bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:41.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.888+0000 7fbee63d7640 1 -- 172.21.15.18:0/2519066657 shutdown_connections 2024-02-17T12:47:41.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.888+0000 7fbee63d7640 1 --2- 172.21.15.18:0/2519066657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee0100cd0 0x7fbee0108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:41.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.888+0000 7fbee63d7640 1 --2- 172.21.15.18:0/2519066657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee0100310 0x7fbee0100790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:41.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.888+0000 7fbee63d7640 1 --2- 172.21.15.18:0/2519066657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee0101670 0x7fbee0101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:41.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.888+0000 7fbee63d7640 1 -- 172.21.15.18:0/2519066657 >> 172.21.15.18:0/2519066657 conn(0x7fbee00fad80 msgr2=0x7fbee00fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:41.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.888+0000 7fbee63d7640 1 -- 172.21.15.18:0/2519066657 shutdown_connections 2024-02-17T12:47:41.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.888+0000 7fbee63d7640 1 -- 172.21.15.18:0/2519066657 wait complete. 2024-02-17T12:47:41.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.888+0000 7fbee63d7640 1 Processor -- start 2024-02-17T12:47:41.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.888+0000 7fbee63d7640 1 -- start start 2024-02-17T12:47:41.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbee63d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee0100310 0x7fbee00ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:41.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbee63d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee0100cd0 0x7fbee019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:41.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbee63d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee0101670 0x7fbee019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:41.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbee63d7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee019ed30 con 0x7fbee0100310 2024-02-17T12:47:41.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbee63d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee019eea0 con 0x7fbee0101670 2024-02-17T12:47:41.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbee63d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee019f060 con 0x7fbee0100cd0 2024-02-17T12:47:41.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbedffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee0100310 0x7fbee00ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:41.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbedffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee0100310 0x7fbee00ffd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35522/0 (socket says 172.21.15.18:35522) 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbedffff640 1 -- 172.21.15.18:0/3707489411 learned_addr learned my addr 172.21.15.18:0/3707489411 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbedf7fe640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee0100cd0 0x7fbee019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbee4974640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee0101670 0x7fbee019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbedffff640 1 -- 172.21.15.18:0/3707489411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee0100cd0 msgr2=0x7fbee019bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbedffff640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee0100cd0 0x7fbee019bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbedffff640 1 -- 172.21.15.18:0/3707489411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee0101670 msgr2=0x7fbee019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbedffff640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee0101670 0x7fbee019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbedffff640 1 -- 172.21.15.18:0/3707489411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed4009560 con 0x7fbee0100310 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.889+0000 7fbedffff640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee0100310 0x7fbee00ffd90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbed000e9c0 tx=0x7fbed000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.890+0000 7fbedd7fa640 1 -- 172.21.15.18:0/3707489411 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed00096e0 con 0x7fbee0100310 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.890+0000 7fbedd7fa640 1 -- 172.21.15.18:0/3707489411 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbed000cd80 con 0x7fbee0100310 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.890+0000 7fbee63d7640 1 -- 172.21.15.18:0/3707489411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee019f340 con 0x7fbee0100310 2024-02-17T12:47:41.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.890+0000 7fbedd7fa640 1 -- 172.21.15.18:0/3707489411 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0010430 con 0x7fbee0100310 2024-02-17T12:47:41.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.890+0000 7fbee63d7640 1 -- 172.21.15.18:0/3707489411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee019f820 con 0x7fbee0100310 2024-02-17T12:47:41.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.891+0000 7fbee63d7640 1 -- 172.21.15.18:0/3707489411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee00676e0 con 0x7fbee0100310 2024-02-17T12:47:41.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.892+0000 7fbedd7fa640 1 -- 172.21.15.18:0/3707489411 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fbed0003680 con 0x7fbee0100310 2024-02-17T12:47:41.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.893+0000 7fbedd7fa640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fbec80a2140 0x7fbec80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:41.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.893+0000 7fbedf7fe640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fbec80a2140 0x7fbec80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:41.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.893+0000 7fbedd7fa640 1 -- 172.21.15.18:0/3707489411 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbed0014070 con 0x7fbee0100310 2024-02-17T12:47:41.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.893+0000 7fbedf7fe640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fbec80a2140 0x7fbec80a4600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009230 tx=0x7fbed4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:41.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:41.897+0000 7fbedd7fa640 1 -- 172.21.15.18:0/3707489411 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbed0096f70 con 0x7fbee0100310 2024-02-17T12:47:41.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:41 smithi146 ceph-mon[36454]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:41.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:41 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] up:boot 2024-02-17T12:47:41.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:41 smithi146 ceph-mon[36454]: fsmap 12 up:standby 2024-02-17T12:47:41.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:41 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T12:47:42.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.125+0000 7fbee63d7640 1 -- 172.21.15.18:0/3707489411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbee0107450 con 0x7fbee0100310 2024-02-17T12:47:42.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.125+0000 7fbedd7fa640 1 -- 172.21.15.18:0/3707489411 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10043 (secure 0 0 0) 0x7fbed009a630 con 0x7fbee0100310 2024-02-17T12:47:42.118 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:42.118 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-02-17T12:47:42.119 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 13 2024-02-17T12:47:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.127+0000 7fbee63d7640 1 -- 172.21.15.18:0/3707489411 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fbec80a2140 msgr2=0x7fbec80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.127+0000 7fbee63d7640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fbec80a2140 0x7fbec80a4600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009230 tx=0x7fbed4005fd0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.127+0000 7fbee63d7640 1 -- 172.21.15.18:0/3707489411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee0100310 msgr2=0x7fbee00ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.127+0000 7fbee63d7640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee0100310 0x7fbee00ffd90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbed000e9c0 tx=0x7fbed000ee80 comp rx=0 tx=0).stop 2024-02-17T12:47:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.128+0000 7fbee63d7640 1 -- 172.21.15.18:0/3707489411 shutdown_connections 2024-02-17T12:47:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.128+0000 7fbee63d7640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fbec80a2140 0x7fbec80a4600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.128+0000 7fbee63d7640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee0101670 0x7fbee019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.128+0000 7fbee63d7640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee0100cd0 0x7fbee019bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.128+0000 7fbee63d7640 1 --2- 172.21.15.18:0/3707489411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee0100310 0x7fbee00ffd90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.128+0000 7fbee63d7640 1 -- 172.21.15.18:0/3707489411 >> 172.21.15.18:0/3707489411 conn(0x7fbee00fad80 msgr2=0x7fbee0104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.128+0000 7fbee63d7640 1 -- 172.21.15.18:0/3707489411 shutdown_connections 2024-02-17T12:47:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.128+0000 7fbee63d7640 1 -- 172.21.15.18:0/3707489411 wait complete. 2024-02-17T12:47:42.132 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:47:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.282+0000 7fb35c24d640 1 -- 172.21.15.18:0/3230175679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb354101290 msgr2=0x7fb354101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.282+0000 7fb35c24d640 1 --2- 172.21.15.18:0/3230175679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb354101290 0x7fb354101710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb344009900 tx=0x7fb34402bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.282+0000 7fb35c24d640 1 -- 172.21.15.18:0/3230175679 shutdown_connections 2024-02-17T12:47:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.282+0000 7fb35c24d640 1 --2- 172.21.15.18:0/3230175679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354101c50 0x7fb3541084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.282+0000 7fb35c24d640 1 --2- 172.21.15.18:0/3230175679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb354101290 0x7fb354101710 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.282+0000 7fb35c24d640 1 --2- 172.21.15.18:0/3230175679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3541000a0 0x7fb3541004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.282+0000 7fb35c24d640 1 -- 172.21.15.18:0/3230175679 >> 172.21.15.18:0/3230175679 conn(0x7fb3540fb630 msgr2=0x7fb3540fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.282+0000 7fb35c24d640 1 -- 172.21.15.18:0/3230175679 shutdown_connections 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.282+0000 7fb35c24d640 1 -- 172.21.15.18:0/3230175679 wait complete. 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb35c24d640 1 Processor -- start 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb35c24d640 1 -- start start 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb35c24d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3541000a0 0x7fb354197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb35c24d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354101290 0x7fb354197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb35c24d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb354101c50 0x7fb35419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb35c24d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35419ed10 con 0x7fb3541000a0 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb35c24d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35419ee80 con 0x7fb354101290 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb35c24d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35419f040 con 0x7fb354101c50 2024-02-17T12:47:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb359fe9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3541000a0 0x7fb354197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb359fe9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3541000a0 0x7fb354197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35548/0 (socket says 172.21.15.18:35548) 2024-02-17T12:47:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb359fe9640 1 -- 172.21.15.18:0/4287168293 learned_addr learned my addr 172.21.15.18:0/4287168293 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb35a7ea640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb354101c50 0x7fb35419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb3597e8640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354101290 0x7fb354197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb359fe9640 1 -- 172.21.15.18:0/4287168293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb354101c50 msgr2=0x7fb35419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb359fe9640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb354101c50 0x7fb35419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb359fe9640 1 -- 172.21.15.18:0/4287168293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354101290 msgr2=0x7fb354197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb359fe9640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354101290 0x7fb354197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb359fe9640 1 -- 172.21.15.18:0/4287168293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb344009560 con 0x7fb3541000a0 2024-02-17T12:47:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb35a7ea640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb354101c50 0x7fb35419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb3597e8640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354101290 0x7fb354197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.283+0000 7fb359fe9640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3541000a0 0x7fb354197220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb35000d950 tx=0x7fb35000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:42.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.284+0000 7fb34affd640 1 -- 172.21.15.18:0/4287168293 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350009c30 con 0x7fb3541000a0 2024-02-17T12:47:42.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.284+0000 7fb34affd640 1 -- 172.21.15.18:0/4287168293 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb350009d90 con 0x7fb3541000a0 2024-02-17T12:47:42.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.284+0000 7fb35c24d640 1 -- 172.21.15.18:0/4287168293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35419f320 con 0x7fb3541000a0 2024-02-17T12:47:42.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.284+0000 7fb34affd640 1 -- 172.21.15.18:0/4287168293 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35000f880 con 0x7fb3541000a0 2024-02-17T12:47:42.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.284+0000 7fb35c24d640 1 -- 172.21.15.18:0/4287168293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35419f880 con 0x7fb3541000a0 2024-02-17T12:47:42.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.285+0000 7fb35c24d640 1 -- 172.21.15.18:0/4287168293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3541073c0 con 0x7fb3541000a0 2024-02-17T12:47:42.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.286+0000 7fb34affd640 1 -- 172.21.15.18:0/4287168293 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb35000f9e0 con 0x7fb3541000a0 2024-02-17T12:47:42.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.286+0000 7fb34affd640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb3400a2140 0x7fb3400a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:42.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.287+0000 7fb3597e8640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb3400a2140 0x7fb3400a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:42.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.287+0000 7fb34affd640 1 -- 172.21.15.18:0/4287168293 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb3500cda90 con 0x7fb3541000a0 2024-02-17T12:47:42.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.287+0000 7fb3597e8640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb3400a2140 0x7fb3400a4600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb354198730 tx=0x7fb344005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:42.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.291+0000 7fb34affd640 1 -- 172.21.15.18:0/4287168293 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb350096f00 con 0x7fb3541000a0 2024-02-17T12:47:42.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.475+0000 7fb35c24d640 1 -- 172.21.15.18:0/4287168293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb35410cbb0 con 0x7fb3541000a0 2024-02-17T12:47:42.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.476+0000 7fb34affd640 1 -- 172.21.15.18:0/4287168293 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18458 (secure 0 0 0) 0x7fb35009a5c0 con 0x7fb3541000a0 2024-02-17T12:47:42.470 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:42.470 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":82,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:46:56.115429+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:47:42.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.478+0000 7fb35c24d640 1 -- 172.21.15.18:0/4287168293 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb3400a2140 msgr2=0x7fb3400a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.478+0000 7fb35c24d640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb3400a2140 0x7fb3400a4600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb354198730 tx=0x7fb344005be0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.478+0000 7fb35c24d640 1 -- 172.21.15.18:0/4287168293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3541000a0 msgr2=0x7fb354197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.478+0000 7fb35c24d640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3541000a0 0x7fb354197220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb35000d950 tx=0x7fb35000de10 comp rx=0 tx=0).stop 2024-02-17T12:47:42.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.478+0000 7fb35c24d640 1 -- 172.21.15.18:0/4287168293 shutdown_connections 2024-02-17T12:47:42.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.478+0000 7fb35c24d640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb3400a2140 0x7fb3400a4600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.478+0000 7fb35c24d640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb354101c50 0x7fb35419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.478+0000 7fb35c24d640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb354101290 0x7fb354197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.478+0000 7fb35c24d640 1 --2- 172.21.15.18:0/4287168293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3541000a0 0x7fb354197220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.478+0000 7fb35c24d640 1 -- 172.21.15.18:0/4287168293 >> 172.21.15.18:0/4287168293 conn(0x7fb3540fb630 msgr2=0x7fb354104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:42.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.479+0000 7fb35c24d640 1 -- 172.21.15.18:0/4287168293 shutdown_connections 2024-02-17T12:47:42.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.479+0000 7fb35c24d640 1 -- 172.21.15.18:0/4287168293 wait complete. 2024-02-17T12:47:42.484 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T12:43:12.450140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T12:47:42.484 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T12:47:42.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.604+0000 7fe0e08e1640 1 -- 172.21.15.18:0/4188466330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc100090 msgr2=0x7fe0dc1004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.604+0000 7fe0e08e1640 1 --2- 172.21.15.18:0/4188466330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc100090 0x7fe0dc1004b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009900 tx=0x7fe0d402bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:42.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.604+0000 7fe0e08e1640 1 -- 172.21.15.18:0/4188466330 shutdown_connections 2024-02-17T12:47:42.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.604+0000 7fe0e08e1640 1 --2- 172.21.15.18:0/4188466330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0dc101c40 0x7fe0dc1084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.604+0000 7fe0e08e1640 1 --2- 172.21.15.18:0/4188466330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0dc101280 0x7fe0dc101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.604+0000 7fe0e08e1640 1 --2- 172.21.15.18:0/4188466330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc100090 0x7fe0dc1004b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.604+0000 7fe0e08e1640 1 -- 172.21.15.18:0/4188466330 >> 172.21.15.18:0/4188466330 conn(0x7fe0dc0fb620 msgr2=0x7fe0dc0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:42.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.604+0000 7fe0e08e1640 1 -- 172.21.15.18:0/4188466330 shutdown_connections 2024-02-17T12:47:42.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.604+0000 7fe0e08e1640 1 -- 172.21.15.18:0/4188466330 wait complete. 2024-02-17T12:47:42.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0e08e1640 1 Processor -- start 2024-02-17T12:47:42.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0e08e1640 1 -- start start 2024-02-17T12:47:42.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0e08e1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0dc100090 0x7fe0dc06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:42.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0e08e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0dc101280 0x7fe0dc19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:42.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0e08e1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc101c40 0x7fe0dc19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:42.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0e08e1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc19ed60 con 0x7fe0dc101c40 2024-02-17T12:47:42.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0e08e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc19eed0 con 0x7fe0dc101280 2024-02-17T12:47:42.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0e08e1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc19f090 con 0x7fe0dc100090 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0dad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc101c40 0x7fe0dc19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0dad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc101c40 0x7fe0dc19e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35564/0 (socket says 172.21.15.18:35564) 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0dad9d640 1 -- 172.21.15.18:0/1011550315 learned_addr learned my addr 172.21.15.18:0/1011550315 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0da59c640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0dc100090 0x7fe0dc06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0d9d9b640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0dc101280 0x7fe0dc19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0dad9d640 1 -- 172.21.15.18:0/1011550315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0dc100090 msgr2=0x7fe0dc06a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0dad9d640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0dc100090 0x7fe0dc06a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0dad9d640 1 -- 172.21.15.18:0/1011550315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0dc101280 msgr2=0x7fe0dc19bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0dad9d640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0dc101280 0x7fe0dc19bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.605+0000 7fe0dad9d640 1 -- 172.21.15.18:0/1011550315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d4009560 con 0x7fe0dc101c40 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.606+0000 7fe0dad9d640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc101c40 0x7fe0dc19e5d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00eb10 tx=0x7fe0cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.606+0000 7fe0d37fe640 1 -- 172.21.15.18:0/1011550315 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc004b20 con 0x7fe0dc101c40 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.606+0000 7fe0d37fe640 1 -- 172.21.15.18:0/1011550315 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0cc004c80 con 0x7fe0dc101c40 2024-02-17T12:47:42.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.606+0000 7fe0e08e1640 1 -- 172.21.15.18:0/1011550315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc19f370 con 0x7fe0dc101c40 2024-02-17T12:47:42.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.606+0000 7fe0d37fe640 1 -- 172.21.15.18:0/1011550315 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc010650 con 0x7fe0dc101c40 2024-02-17T12:47:42.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.606+0000 7fe0e08e1640 1 -- 172.21.15.18:0/1011550315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc19f830 con 0x7fe0dc101c40 2024-02-17T12:47:42.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.607+0000 7fe0e08e1640 1 -- 172.21.15.18:0/1011550315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc107420 con 0x7fe0dc101c40 2024-02-17T12:47:42.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.608+0000 7fe0d37fe640 1 -- 172.21.15.18:0/1011550315 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe0cc0107f0 con 0x7fe0dc101c40 2024-02-17T12:47:42.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.608+0000 7fe0d37fe640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe0c80a20c0 0x7fe0c80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:42.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.608+0000 7fe0da59c640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe0c80a20c0 0x7fe0c80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:42.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.608+0000 7fe0d37fe640 1 -- 172.21.15.18:0/1011550315 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe0cc014070 con 0x7fe0dc101c40 2024-02-17T12:47:42.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.609+0000 7fe0da59c640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe0c80a20c0 0x7fe0c80a4580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4005ab0 tx=0x7fe0d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:42.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.612+0000 7fe0d37fe640 1 -- 172.21.15.18:0/1011550315 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe0cc097310 con 0x7fe0dc101c40 2024-02-17T12:47:42.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.799+0000 7fe0e08e1640 1 -- 172.21.15.18:0/1011550315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0dc10bcf0 con 0x7fe0dc101c40 2024-02-17T12:47:42.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.799+0000 7fe0d37fe640 1 -- 172.21.15.18:0/1011550315 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0cc09a9d0 con 0x7fe0dc101c40 2024-02-17T12:47:42.792 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-02-17T12:47:42.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.801+0000 7fe0e08e1640 1 -- 172.21.15.18:0/1011550315 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe0c80a20c0 msgr2=0x7fe0c80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.801+0000 7fe0e08e1640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe0c80a20c0 0x7fe0c80a4580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4005ab0 tx=0x7fe0d4005fd0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.801+0000 7fe0e08e1640 1 -- 172.21.15.18:0/1011550315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc101c40 msgr2=0x7fe0dc19e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.801+0000 7fe0e08e1640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc101c40 0x7fe0dc19e5d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00eb10 tx=0x7fe0cc00c2d0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.801+0000 7fe0e08e1640 1 -- 172.21.15.18:0/1011550315 shutdown_connections 2024-02-17T12:47:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.801+0000 7fe0e08e1640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe0c80a20c0 0x7fe0c80a4580 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.801+0000 7fe0e08e1640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc101c40 0x7fe0dc19e5d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.801+0000 7fe0e08e1640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0dc101280 0x7fe0dc19bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.801+0000 7fe0e08e1640 1 --2- 172.21.15.18:0/1011550315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0dc100090 0x7fe0dc06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.801+0000 7fe0e08e1640 1 -- 172.21.15.18:0/1011550315 >> 172.21.15.18:0/1011550315 conn(0x7fe0dc0fb620 msgr2=0x7fe0dc1045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.802+0000 7fe0e08e1640 1 -- 172.21.15.18:0/1011550315 shutdown_connections 2024-02-17T12:47:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.802+0000 7fe0e08e1640 1 -- 172.21.15.18:0/1011550315 wait complete. 2024-02-17T12:47:42.806 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-17T12:47:42.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.916+0000 7fed0d148640 1 -- 172.21.15.18:0/1114646104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08101c20 msgr2=0x7fed08104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.916+0000 7fed0d148640 1 --2- 172.21.15.18:0/1114646104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08101c20 0x7fed08104010 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009900 tx=0x7fecfc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.917+0000 7fed0d148640 1 -- 172.21.15.18:0/1114646104 shutdown_connections 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.917+0000 7fed0d148640 1 --2- 172.21.15.18:0/1114646104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed08106e80 0x7fed081092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.917+0000 7fed0d148640 1 --2- 172.21.15.18:0/1114646104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed08104550 0x7fed08106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.917+0000 7fed0d148640 1 --2- 172.21.15.18:0/1114646104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08101c20 0x7fed08104010 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.917+0000 7fed0d148640 1 -- 172.21.15.18:0/1114646104 >> 172.21.15.18:0/1114646104 conn(0x7fed080fb610 msgr2=0x7fed080fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.917+0000 7fed0d148640 1 -- 172.21.15.18:0/1114646104 shutdown_connections 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.917+0000 7fed0d148640 1 -- 172.21.15.18:0/1114646104 wait complete. 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.917+0000 7fed0d148640 1 Processor -- start 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.917+0000 7fed0d148640 1 -- start start 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0d148640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed08101c20 0x7fed08197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0d148640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08104550 0x7fed08197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:42.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0d148640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed08106e80 0x7fed0819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0d148640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0819ed50 con 0x7fed08104550 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0d148640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0819eec0 con 0x7fed08101c20 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0d148640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0819f080 con 0x7fed08106e80 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08104550 0x7fed08197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08104550 0x7fed08197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35582/0 (socket says 172.21.15.18:35582) 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0659c640 1 -- 172.21.15.18:0/1698545333 learned_addr learned my addr 172.21.15.18:0/1698545333 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed06d9d640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed08101c20 0x7fed08197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0759e640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed08106e80 0x7fed0819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0659c640 1 -- 172.21.15.18:0/1698545333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed08106e80 msgr2=0x7fed0819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0659c640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed08106e80 0x7fed0819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0659c640 1 -- 172.21.15.18:0/1698545333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed08101c20 msgr2=0x7fed08197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0659c640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed08101c20 0x7fed08197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0659c640 1 -- 172.21.15.18:0/1698545333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc009560 con 0x7fed08104550 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed06d9d640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed08101c20 0x7fed08197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:47:42.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0659c640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08104550 0x7fed08197840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fecf800e9c0 tx=0x7fecf800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:42.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7feceffff640 1 -- 172.21.15.18:0/1698545333 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf8009700 con 0x7fed08104550 2024-02-17T12:47:42.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7fed0d148640 1 -- 172.21.15.18:0/1698545333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0819f360 con 0x7fed08104550 2024-02-17T12:47:42.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7feceffff640 1 -- 172.21.15.18:0/1698545333 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecf800cd80 con 0x7fed08104550 2024-02-17T12:47:42.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.918+0000 7feceffff640 1 -- 172.21.15.18:0/1698545333 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf8010430 con 0x7fed08104550 2024-02-17T12:47:42.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.919+0000 7fed0d148640 1 -- 172.21.15.18:0/1698545333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0819f8c0 con 0x7fed08104550 2024-02-17T12:47:42.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.920+0000 7fed0d148640 1 -- 172.21.15.18:0/1698545333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed080fd100 con 0x7fed08104550 2024-02-17T12:47:42.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.921+0000 7feceffff640 1 -- 172.21.15.18:0/1698545333 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fecf8005000 con 0x7fed08104550 2024-02-17T12:47:42.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.922+0000 7feceffff640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fecf00a20c0 0x7fecf00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:42.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.922+0000 7fed06d9d640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fecf00a20c0 0x7fecf00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:42.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.922+0000 7feceffff640 1 -- 172.21.15.18:0/1698545333 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fecf8014070 con 0x7fed08104550 2024-02-17T12:47:42.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.922+0000 7fed06d9d640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fecf00a20c0 0x7fecf00a4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fecfc000c00 tx=0x7fecfc02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:42.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:42.925+0000 7feceffff640 1 -- 172.21.15.18:0/1698545333 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fecf8097270 con 0x7fed08104550 2024-02-17T12:47:43.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.110+0000 7fed0d148640 1 -- 172.21.15.18:0/1698545333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fed0810ce00 con 0x7fed08104550 2024-02-17T12:47:43.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.111+0000 7feceffff640 1 -- 172.21.15.18:0/1698545333 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7fecf809a930 con 0x7fed08104550 2024-02-17T12:47:43.104 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:43.104 INFO:teuthology.orchestra.run.smithi018.stdout:[ 2024-02-17T12:47:43.104 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T12:47:43.104 INFO:teuthology.orchestra.run.smithi018.stdout: "poolnum": 1, 2024-02-17T12:47:43.104 INFO:teuthology.orchestra.run.smithi018.stdout: "poolname": ".mgr" 2024-02-17T12:47:43.104 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:47:43.104 INFO:teuthology.orchestra.run.smithi018.stdout:] 2024-02-17T12:47:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.113+0000 7fed0d148640 1 -- 172.21.15.18:0/1698545333 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fecf00a20c0 msgr2=0x7fecf00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.113+0000 7fed0d148640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fecf00a20c0 0x7fecf00a4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fecfc000c00 tx=0x7fecfc02c7e0 comp rx=0 tx=0).stop 2024-02-17T12:47:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.113+0000 7fed0d148640 1 -- 172.21.15.18:0/1698545333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08104550 msgr2=0x7fed08197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.113+0000 7fed0d148640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08104550 0x7fed08197840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fecf800e9c0 tx=0x7fecf800ee80 comp rx=0 tx=0).stop 2024-02-17T12:47:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.113+0000 7fed0d148640 1 -- 172.21.15.18:0/1698545333 shutdown_connections 2024-02-17T12:47:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.113+0000 7fed0d148640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fecf00a20c0 0x7fecf00a4580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.113+0000 7fed0d148640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed08106e80 0x7fed0819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.113+0000 7fed0d148640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08104550 0x7fed08197840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.113+0000 7fed0d148640 1 --2- 172.21.15.18:0/1698545333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed08101c20 0x7fed08197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.113+0000 7fed0d148640 1 -- 172.21.15.18:0/1698545333 >> 172.21.15.18:0/1698545333 conn(0x7fed080fb610 msgr2=0x7fed08102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:43.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.113+0000 7fed0d148640 1 -- 172.21.15.18:0/1698545333 shutdown_connections 2024-02-17T12:47:43.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.113+0000 7fed0d148640 1 -- 172.21.15.18:0/1698545333 wait complete. 2024-02-17T12:47:43.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.118 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3707489411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:43.118 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:43.118 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:47:43.118 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.118 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/4287168293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:43.118 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:47:43.118 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:42 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1011550315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:47:43.119 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-17T12:47:43.119 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-17T12:47:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3707489411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:47:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/4287168293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:43.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:47:43.203 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:42 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1011550315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:47:43.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.243+0000 7fe462e81640 1 -- 172.21.15.18:0/480987074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe45c101290 msgr2=0x7fe45c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:43.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.243+0000 7fe462e81640 1 --2- 172.21.15.18:0/480987074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe45c101290 0x7fe45c101710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009900 tx=0x7fe44c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:43.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.243+0000 7fe462e81640 1 -- 172.21.15.18:0/480987074 shutdown_connections 2024-02-17T12:47:43.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.243+0000 7fe462e81640 1 --2- 172.21.15.18:0/480987074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c101c50 0x7fe45c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:43.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.243+0000 7fe462e81640 1 --2- 172.21.15.18:0/480987074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe45c101290 0x7fe45c101710 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:43.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.243+0000 7fe462e81640 1 --2- 172.21.15.18:0/480987074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c1000a0 0x7fe45c1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:43.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.243+0000 7fe462e81640 1 -- 172.21.15.18:0/480987074 >> 172.21.15.18:0/480987074 conn(0x7fe45c0fb630 msgr2=0x7fe45c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:43.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe462e81640 1 -- 172.21.15.18:0/480987074 shutdown_connections 2024-02-17T12:47:43.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe462e81640 1 -- 172.21.15.18:0/480987074 wait complete. 2024-02-17T12:47:43.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe462e81640 1 Processor -- start 2024-02-17T12:47:43.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe462e81640 1 -- start start 2024-02-17T12:47:43.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe462e81640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c1000a0 0x7fe45c104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:43.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe462e81640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c101290 0x7fe45c102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:43.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe462e81640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe45c101c50 0x7fe45c102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:43.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe462e81640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c103310 con 0x7fe45c101c50 2024-02-17T12:47:43.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe462e81640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c103480 con 0x7fe45c101290 2024-02-17T12:47:43.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe462e81640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c1a1330 con 0x7fe45c1000a0 2024-02-17T12:47:43.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe46141e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe45c101c50 0x7fe45c102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:43.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe46141e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe45c101c50 0x7fe45c102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35600/0 (socket says 172.21.15.18:35600) 2024-02-17T12:47:43.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe46141e640 1 -- 172.21.15.18:0/2573860306 learned_addr learned my addr 172.21.15.18:0/2573860306 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:43.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe453fff640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c101290 0x7fe45c102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:43.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.244+0000 7fe460c1d640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c1000a0 0x7fe45c104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:43.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.245+0000 7fe46141e640 1 -- 172.21.15.18:0/2573860306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c1000a0 msgr2=0x7fe45c104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:43.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.245+0000 7fe46141e640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c1000a0 0x7fe45c104030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:43.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.245+0000 7fe46141e640 1 -- 172.21.15.18:0/2573860306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c101290 msgr2=0x7fe45c102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:43.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.245+0000 7fe46141e640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c101290 0x7fe45c102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:43.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.245+0000 7fe46141e640 1 -- 172.21.15.18:0/2573860306 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c009560 con 0x7fe45c101c50 2024-02-17T12:47:43.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.245+0000 7fe453fff640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c101290 0x7fe45c102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:47:43.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.245+0000 7fe46141e640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe45c101c50 0x7fe45c102bb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe45400eb40 tx=0x7fe45400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:43.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.245+0000 7fe451ffb640 1 -- 172.21.15.18:0/2573860306 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454004b20 con 0x7fe45c101c50 2024-02-17T12:47:43.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.245+0000 7fe462e81640 1 -- 172.21.15.18:0/2573860306 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45c1a15e0 con 0x7fe45c101c50 2024-02-17T12:47:43.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.245+0000 7fe451ffb640 1 -- 172.21.15.18:0/2573860306 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe454004c80 con 0x7fe45c101c50 2024-02-17T12:47:43.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.245+0000 7fe451ffb640 1 -- 172.21.15.18:0/2573860306 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454010690 con 0x7fe45c101c50 2024-02-17T12:47:43.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.245+0000 7fe462e81640 1 -- 172.21.15.18:0/2573860306 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45c1a1ac0 con 0x7fe45c101c50 2024-02-17T12:47:43.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.246+0000 7fe462e81640 1 -- 172.21.15.18:0/2573860306 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45c10bb00 con 0x7fe45c101c50 2024-02-17T12:47:43.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.248+0000 7fe451ffb640 1 -- 172.21.15.18:0/2573860306 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe454005020 con 0x7fe45c101c50 2024-02-17T12:47:43.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.248+0000 7fe451ffb640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe4440a20c0 0x7fe4440a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:43.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.249+0000 7fe460c1d640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe4440a20c0 0x7fe4440a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:43.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.249+0000 7fe451ffb640 1 -- 172.21.15.18:0/2573860306 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe454014070 con 0x7fe45c101c50 2024-02-17T12:47:43.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.249+0000 7fe460c1d640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe4440a20c0 0x7fe4440a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe45c1010f0 tx=0x7fe458005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:43.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.253+0000 7fe451ffb640 1 -- 172.21.15.18:0/2573860306 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe454097330 con 0x7fe45c101c50 2024-02-17T12:47:43.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3707489411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:43.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:43.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:47:43.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:43.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/4287168293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:43.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:47:43.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:42 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1011550315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:47:43.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:43.442+0000 7fe462e81640 1 -- 172.21.15.18:0/2573860306 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe45c10bc90 con 0x7fe45c101c50 2024-02-17T12:47:44.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.038+0000 7fe451ffb640 1 -- 172.21.15.18:0/2573860306 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe45409a9f0 con 0x7fe45c101c50 2024-02-17T12:47:44.031 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'cephfs_metadata' created 2024-02-17T12:47:44.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.041+0000 7fe462e81640 1 -- 172.21.15.18:0/2573860306 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe4440a20c0 msgr2=0x7fe4440a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:44.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.041+0000 7fe462e81640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe4440a20c0 0x7fe4440a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe45c1010f0 tx=0x7fe458005f90 comp rx=0 tx=0).stop 2024-02-17T12:47:44.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.041+0000 7fe462e81640 1 -- 172.21.15.18:0/2573860306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe45c101c50 msgr2=0x7fe45c102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:44.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.041+0000 7fe462e81640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe45c101c50 0x7fe45c102bb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe45400eb40 tx=0x7fe45400c2d0 comp rx=0 tx=0).stop 2024-02-17T12:47:44.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.042+0000 7fe462e81640 1 -- 172.21.15.18:0/2573860306 shutdown_connections 2024-02-17T12:47:44.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.042+0000 7fe462e81640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe4440a20c0 0x7fe4440a4580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:44.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.042+0000 7fe462e81640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe45c101c50 0x7fe45c102bb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:44.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.042+0000 7fe462e81640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c101290 0x7fe45c102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:44.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.042+0000 7fe462e81640 1 --2- 172.21.15.18:0/2573860306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c1000a0 0x7fe45c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:44.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.042+0000 7fe462e81640 1 -- 172.21.15.18:0/2573860306 >> 172.21.15.18:0/2573860306 conn(0x7fe45c0fb630 msgr2=0x7fe45c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:44.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.042+0000 7fe462e81640 1 -- 172.21.15.18:0/2573860306 shutdown_connections 2024-02-17T12:47:44.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.043+0000 7fe462e81640 1 -- 172.21.15.18:0/2573860306 wait complete. 2024-02-17T12:47:44.053 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-17T12:47:44.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.177+0000 7f2f2632e640 1 -- 172.21.15.18:0/217728978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f20106e70 msgr2=0x7f2f201092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:44.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.177+0000 7f2f2632e640 1 --2- 172.21.15.18:0/217728978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f20106e70 0x7f2f201092c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009ca0 tx=0x7f2f1802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:47:44.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.177+0000 7f2f2632e640 1 -- 172.21.15.18:0/217728978 shutdown_connections 2024-02-17T12:47:44.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.177+0000 7f2f2632e640 1 --2- 172.21.15.18:0/217728978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f20106e70 0x7f2f201092c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.177+0000 7f2f2632e640 1 --2- 172.21.15.18:0/217728978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f20104540 0x7f2f20106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.177+0000 7f2f2632e640 1 --2- 172.21.15.18:0/217728978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f20101c10 0x7f2f20104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.177+0000 7f2f2632e640 1 -- 172.21.15.18:0/217728978 >> 172.21.15.18:0/217728978 conn(0x7f2f200fb600 msgr2=0x7f2f200fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.177+0000 7f2f2632e640 1 -- 172.21.15.18:0/217728978 shutdown_connections 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.177+0000 7f2f2632e640 1 -- 172.21.15.18:0/217728978 wait complete. 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f2632e640 1 Processor -- start 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f2632e640 1 -- start start 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f2632e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f20101c10 0x7f2f201993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f2632e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f20104540 0x7f2f201998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f2632e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f20106e70 0x7f2f201a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f2632e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f201a0ea0 con 0x7f2f20101c10 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f2632e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f201a1010 con 0x7f2f20104540 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f2632e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f201a11d0 con 0x7f2f20106e70 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f1ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f20101c10 0x7f2f201993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:44.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f1ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f20101c10 0x7f2f201993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35620/0 (socket says 172.21.15.18:35620) 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f248cb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f20106e70 0x7f2f201a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f1ffff640 1 -- 172.21.15.18:0/1706946054 learned_addr learned my addr 172.21.15.18:0/1706946054 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.178+0000 7f2f1f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f20104540 0x7f2f201998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f1ffff640 1 -- 172.21.15.18:0/1706946054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f20106e70 msgr2=0x7f2f201a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f1ffff640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f20106e70 0x7f2f201a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f1ffff640 1 -- 172.21.15.18:0/1706946054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f20104540 msgr2=0x7f2f201998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f1ffff640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f20104540 0x7f2f201998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f1ffff640 1 -- 172.21.15.18:0/1706946054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f18009900 con 0x7f2f20101c10 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f1f7fe640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f20104540 0x7f2f201998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f248cb640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f20106e70 0x7f2f201a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f1ffff640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f20101c10 0x7f2f201993b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000e9f0 tx=0x7f2f1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f1d7fa640 1 -- 172.21.15.18:0/1706946054 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10009720 con 0x7f2f20101c10 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f1d7fa640 1 -- 172.21.15.18:0/1706946054 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f1000cd80 con 0x7f2f20101c10 2024-02-17T12:47:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f2632e640 1 -- 172.21.15.18:0/1706946054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f201a14b0 con 0x7f2f20101c10 2024-02-17T12:47:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f1d7fa640 1 -- 172.21.15.18:0/1706946054 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10010690 con 0x7f2f20101c10 2024-02-17T12:47:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.179+0000 7f2f2632e640 1 -- 172.21.15.18:0/1706946054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f201a1970 con 0x7f2f20101c10 2024-02-17T12:47:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.180+0000 7f2f2632e640 1 -- 172.21.15.18:0/1706946054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f200ffd30 con 0x7f2f20101c10 2024-02-17T12:47:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.182+0000 7f2f1d7fa640 1 -- 172.21.15.18:0/1706946054 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2f100046c0 con 0x7f2f20101c10 2024-02-17T12:47:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.182+0000 7f2f1d7fa640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2f080a20c0 0x7f2f080a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.183+0000 7f2f1d7fa640 1 -- 172.21.15.18:0/1706946054 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f2f10014070 con 0x7f2f20101c10 2024-02-17T12:47:44.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.183+0000 7f2f1f7fe640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2f080a20c0 0x7f2f080a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:44.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.183+0000 7f2f1f7fe640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2f080a20c0 0x7f2f080a4580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2f2019a8c0 tx=0x7f2f14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:44.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.186+0000 7f2f1d7fa640 1 -- 172.21.15.18:0/1706946054 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f100971d0 con 0x7f2f20101c10 2024-02-17T12:47:44.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:44 smithi119 ceph-mon[36188]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:44.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:44 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1698545333' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-17T12:47:44.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:44 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2573860306' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-17T12:47:44.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:44.407+0000 7f2f2632e640 1 -- 172.21.15.18:0/1706946054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2f200fff10 con 0x7f2f20101c10 2024-02-17T12:47:44.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:44 smithi146 ceph-mon[36454]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:44.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:44 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1698545333' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-17T12:47:44.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:44 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2573860306' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-17T12:47:44.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:44 smithi018 ceph-mon[32625]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:44.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:44 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1698545333' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-17T12:47:44.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:44 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2573860306' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-17T12:47:45.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.043+0000 7f2f1d7fa640 1 -- 172.21.15.18:0/1706946054 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2f1009a890 con 0x7f2f20101c10 2024-02-17T12:47:45.036 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'cephfs_data' created 2024-02-17T12:47:45.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.046+0000 7f2f2632e640 1 -- 172.21.15.18:0/1706946054 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2f080a20c0 msgr2=0x7f2f080a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:45.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.046+0000 7f2f2632e640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2f080a20c0 0x7f2f080a4580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2f2019a8c0 tx=0x7f2f14005f90 comp rx=0 tx=0).stop 2024-02-17T12:47:45.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.046+0000 7f2f2632e640 1 -- 172.21.15.18:0/1706946054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f20101c10 msgr2=0x7f2f201993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:45.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.046+0000 7f2f2632e640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f20101c10 0x7f2f201993b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000e9f0 tx=0x7f2f1000edf0 comp rx=0 tx=0).stop 2024-02-17T12:47:45.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.046+0000 7f2f2632e640 1 -- 172.21.15.18:0/1706946054 shutdown_connections 2024-02-17T12:47:45.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.046+0000 7f2f2632e640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2f080a20c0 0x7f2f080a4580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:45.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.046+0000 7f2f2632e640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f20106e70 0x7f2f201a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:45.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.046+0000 7f2f2632e640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f20104540 0x7f2f201998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:45.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.046+0000 7f2f2632e640 1 --2- 172.21.15.18:0/1706946054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f20101c10 0x7f2f201993b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:45.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.046+0000 7f2f2632e640 1 -- 172.21.15.18:0/1706946054 >> 172.21.15.18:0/1706946054 conn(0x7f2f200fb600 msgr2=0x7f2f20104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:45.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.046+0000 7f2f2632e640 1 -- 172.21.15.18:0/1706946054 shutdown_connections 2024-02-17T12:47:45.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.046+0000 7f2f2632e640 1 -- 172.21.15.18:0/1706946054 wait complete. 2024-02-17T12:47:45.058 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-17T12:47:45.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.174+0000 7f00c5da0640 1 -- 172.21.15.18:0/1062671425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00c0102250 msgr2=0x7f00c01026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:45.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.174+0000 7f00c5da0640 1 --2- 172.21.15.18:0/1062671425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00c0102250 0x7f00c01026d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009900 tx=0x7f00ac02bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.174+0000 7f00c5da0640 1 -- 172.21.15.18:0/1062671425 shutdown_connections 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.174+0000 7f00c5da0640 1 --2- 172.21.15.18:0/1062671425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00c0102c10 0x7f00c01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.174+0000 7f00c5da0640 1 --2- 172.21.15.18:0/1062671425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00c0102250 0x7f00c01026d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.174+0000 7f00c5da0640 1 --2- 172.21.15.18:0/1062671425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00c0101060 0x7f00c0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.174+0000 7f00c5da0640 1 -- 172.21.15.18:0/1062671425 >> 172.21.15.18:0/1062671425 conn(0x7f00c00fc5f0 msgr2=0x7f00c00fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c5da0640 1 -- 172.21.15.18:0/1062671425 shutdown_connections 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c5da0640 1 -- 172.21.15.18:0/1062671425 wait complete. 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c5da0640 1 Processor -- start 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c5da0640 1 -- start start 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c5da0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00c0101060 0x7f00c0197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c5da0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00c0102250 0x7f00c0197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c5da0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00c0102c10 0x7f00c019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c5da0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c019ebf0 con 0x7f00c0102c10 2024-02-17T12:47:45.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c5da0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c019ed60 con 0x7f00c0102250 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c5da0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c019ef20 con 0x7f00c0101060 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c559f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00c0102c10 0x7f00c019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c4d9e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00c0101060 0x7f00c0197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00bffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00c0102250 0x7f00c0197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c559f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00c0102c10 0x7f00c019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35638/0 (socket says 172.21.15.18:35638) 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.175+0000 7f00c559f640 1 -- 172.21.15.18:0/2497355000 learned_addr learned my addr 172.21.15.18:0/2497355000 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00c4d9e640 1 -- 172.21.15.18:0/2497355000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00c0102250 msgr2=0x7f00c0197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00c4d9e640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00c0102250 0x7f00c0197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00c4d9e640 1 -- 172.21.15.18:0/2497355000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00c0102c10 msgr2=0x7f00c019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00c4d9e640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00c0102c10 0x7f00c019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00c4d9e640 1 -- 172.21.15.18:0/2497355000 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac009560 con 0x7f00c0101060 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00c559f640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00c0102c10 0x7f00c019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00bffff640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00c0102250 0x7f00c0197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:47:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00c4d9e640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00c0101060 0x7f00c0197100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f00b400e9c0 tx=0x7f00b400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:45.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00bdffb640 1 -- 172.21.15.18:0/2497355000 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b400ccc0 con 0x7f00c0101060 2024-02-17T12:47:45.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00c5da0640 1 -- 172.21.15.18:0/2497355000 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c019f200 con 0x7f00c0101060 2024-02-17T12:47:45.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00bdffb640 1 -- 172.21.15.18:0/2497355000 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00b400ce20 con 0x7f00c0101060 2024-02-17T12:47:45.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00bdffb640 1 -- 172.21.15.18:0/2497355000 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4010430 con 0x7f00c0101060 2024-02-17T12:47:45.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.176+0000 7f00c5da0640 1 -- 172.21.15.18:0/2497355000 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c019f730 con 0x7f00c0101060 2024-02-17T12:47:45.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.177+0000 7f00c5da0640 1 -- 172.21.15.18:0/2497355000 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c0108380 con 0x7f00c0101060 2024-02-17T12:47:45.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.178+0000 7f00bdffb640 1 -- 172.21.15.18:0/2497355000 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f00b4003f60 con 0x7f00c0101060 2024-02-17T12:47:45.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.179+0000 7f00bdffb640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f00b00a20c0 0x7f00b00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:45.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.179+0000 7f00bffff640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f00b00a20c0 0x7f00b00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:45.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.179+0000 7f00bdffb640 1 -- 172.21.15.18:0/2497355000 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f00b4014070 con 0x7f00c0101060 2024-02-17T12:47:45.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.179+0000 7f00bffff640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f00b00a20c0 0x7f00b00a4580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009230 tx=0x7f00ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:45.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.182+0000 7f00bdffb640 1 -- 172.21.15.18:0/2497355000 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f00b40970e0 con 0x7f00c0101060 2024-02-17T12:47:45.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:45 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2573860306' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-17T12:47:45.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:45 smithi119 ceph-mon[36188]: osdmap e83: 12 total, 12 up, 12 in 2024-02-17T12:47:45.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:45.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:47:45.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:45 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1706946054' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-17T12:47:45.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:45.405+0000 7f00c5da0640 1 -- 172.21.15.18:0/2497355000 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f00c010ca50 con 0x7f00c0101060 2024-02-17T12:47:45.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:45 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2573860306' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-17T12:47:45.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:45 smithi146 ceph-mon[36454]: osdmap e83: 12 total, 12 up, 12 in 2024-02-17T12:47:45.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:45.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:47:45.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:45 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1706946054' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-17T12:47:45.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:45 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2573860306' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-17T12:47:45.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:45 smithi018 ceph-mon[32625]: osdmap e83: 12 total, 12 up, 12 in 2024-02-17T12:47:45.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:47:45.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:47:45.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:47:45.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:45 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1706946054' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-17T12:47:46.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.052+0000 7f00bdffb640 1 -- 172.21.15.18:0/2497355000 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f00b409a7a0 con 0x7f00c0101060 2024-02-17T12:47:46.045 INFO:teuthology.orchestra.run.smithi018.stderr:new fs with metadata pool 2 and data pool 3 2024-02-17T12:47:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.056+0000 7f00c5da0640 1 -- 172.21.15.18:0/2497355000 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f00b00a20c0 msgr2=0x7f00b00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.056+0000 7f00c5da0640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f00b00a20c0 0x7f00b00a4580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009230 tx=0x7f00ac005fd0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.056+0000 7f00c5da0640 1 -- 172.21.15.18:0/2497355000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00c0101060 msgr2=0x7f00c0197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.056+0000 7f00c5da0640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00c0101060 0x7f00c0197100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f00b400e9c0 tx=0x7f00b400ee80 comp rx=0 tx=0).stop 2024-02-17T12:47:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.056+0000 7f00c5da0640 1 -- 172.21.15.18:0/2497355000 shutdown_connections 2024-02-17T12:47:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.056+0000 7f00c5da0640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f00b00a20c0 0x7f00b00a4580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.056+0000 7f00c5da0640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00c0102c10 0x7f00c019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.056+0000 7f00c5da0640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00c0102250 0x7f00c0197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.056+0000 7f00c5da0640 1 --2- 172.21.15.18:0/2497355000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00c0101060 0x7f00c0197100 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.056+0000 7f00c5da0640 1 -- 172.21.15.18:0/2497355000 >> 172.21.15.18:0/2497355000 conn(0x7f00c00fc5f0 msgr2=0x7f00c0105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.057+0000 7f00c5da0640 1 -- 172.21.15.18:0/2497355000 shutdown_connections 2024-02-17T12:47:46.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.057+0000 7f00c5da0640 1 -- 172.21.15.18:0/2497355000 wait complete. 2024-02-17T12:47:46.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:47:46.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.184+0000 7fd67f481640 1 -- 172.21.15.18:0/2533427408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd678101310 msgr2=0x7fd678101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.184+0000 7fd67f481640 1 --2- 172.21.15.18:0/2533427408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd678101310 0x7fd678101790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd668009900 tx=0x7fd66802bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:46.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.184+0000 7fd67f481640 1 -- 172.21.15.18:0/2533427408 shutdown_connections 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.184+0000 7fd67f481640 1 --2- 172.21.15.18:0/2533427408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd678101cd0 0x7fd678108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.184+0000 7fd67f481640 1 --2- 172.21.15.18:0/2533427408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd678101310 0x7fd678101790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.184+0000 7fd67f481640 1 --2- 172.21.15.18:0/2533427408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd678100120 0x7fd678100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.184+0000 7fd67f481640 1 -- 172.21.15.18:0/2533427408 >> 172.21.15.18:0/2533427408 conn(0x7fd6780fb6b0 msgr2=0x7fd6780fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.184+0000 7fd67f481640 1 -- 172.21.15.18:0/2533427408 shutdown_connections 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.184+0000 7fd67f481640 1 -- 172.21.15.18:0/2533427408 wait complete. 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67f481640 1 Processor -- start 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67f481640 1 -- start start 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67f481640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd678100120 0x7fd678199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67f481640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd678101310 0x7fd6781999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67f481640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd678101cd0 0x7fd6781a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67f481640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781a0f50 con 0x7fd678101310 2024-02-17T12:47:46.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67f481640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781a10c0 con 0x7fd678101cd0 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67f481640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781a1280 con 0x7fd678100120 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67ca1c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd678101310 0x7fd6781999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67ca1c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd678101310 0x7fd6781999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35660/0 (socket says 172.21.15.18:35660) 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67ca1c640 1 -- 172.21.15.18:0/1315874757 learned_addr learned my addr 172.21.15.18:0/1315874757 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67d21d640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd678100120 0x7fd678199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67da1e640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd678101cd0 0x7fd6781a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67ca1c640 1 -- 172.21.15.18:0/1315874757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd678100120 msgr2=0x7fd678199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67ca1c640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd678100120 0x7fd678199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67ca1c640 1 -- 172.21.15.18:0/1315874757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd678101cd0 msgr2=0x7fd6781a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67ca1c640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd678101cd0 0x7fd6781a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67ca1c640 1 -- 172.21.15.18:0/1315874757 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd668009560 con 0x7fd678101310 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67da1e640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd678101cd0 0x7fd6781a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:47:46.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.185+0000 7fd67ca1c640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd678101310 0x7fd6781999a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd668006010 tx=0x7fd6680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:46.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.186+0000 7fd66e7fc640 1 -- 172.21.15.18:0/1315874757 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6680044e0 con 0x7fd678101310 2024-02-17T12:47:46.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.186+0000 7fd66e7fc640 1 -- 172.21.15.18:0/1315874757 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6680054e0 con 0x7fd678101310 2024-02-17T12:47:46.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.186+0000 7fd67f481640 1 -- 172.21.15.18:0/1315874757 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6781a1500 con 0x7fd678101310 2024-02-17T12:47:46.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.186+0000 7fd66e7fc640 1 -- 172.21.15.18:0/1315874757 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66803e450 con 0x7fd678101310 2024-02-17T12:47:46.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.186+0000 7fd67f481640 1 -- 172.21.15.18:0/1315874757 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6781a1a10 con 0x7fd678101310 2024-02-17T12:47:46.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.187+0000 7fd67f481640 1 -- 172.21.15.18:0/1315874757 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67810bac0 con 0x7fd678101310 2024-02-17T12:47:46.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.188+0000 7fd66e7fc640 1 -- 172.21.15.18:0/1315874757 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd6680035f0 con 0x7fd678101310 2024-02-17T12:47:46.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.189+0000 7fd66e7fc640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd6640a20c0 0x7fd6640a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:46.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.189+0000 7fd67d21d640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd6640a20c0 0x7fd6640a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:46.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.189+0000 7fd66e7fc640 1 -- 172.21.15.18:0/1315874757 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd6680f05b0 con 0x7fd678101310 2024-02-17T12:47:46.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.189+0000 7fd67d21d640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd6640a20c0 0x7fd6640a4580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd674005fd0 tx=0x7fd674005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:46.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.193+0000 7fd66e7fc640 1 -- 172.21.15.18:0/1315874757 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd6680b96d0 con 0x7fd678101310 2024-02-17T12:47:46.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:46 smithi119 ceph-mon[36188]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:46.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:46 smithi119 ceph-mon[36188]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-17T12:47:46.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:46 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1706946054' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-17T12:47:46.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:46 smithi119 ceph-mon[36188]: osdmap e84: 12 total, 12 up, 12 in 2024-02-17T12:47:46.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:46 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2497355000' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-17T12:47:46.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:46 smithi119 ceph-mon[36188]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-17T12:47:46.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:46 smithi119 ceph-mon[36188]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-17T12:47:46.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.375+0000 7fd67f481640 1 -- 172.21.15.18:0/1315874757 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd67810bc50 con 0x7fd678101310 2024-02-17T12:47:46.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.376+0000 7fd66e7fc640 1 -- 172.21.15.18:0/1315874757 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22234 (secure 0 0 0) 0x7fd6680bcd90 con 0x7fd678101310 2024-02-17T12:47:46.369 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:46.369 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":85,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:47:45.407544+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:47:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.378+0000 7fd67f481640 1 -- 172.21.15.18:0/1315874757 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd6640a20c0 msgr2=0x7fd6640a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.378+0000 7fd67f481640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd6640a20c0 0x7fd6640a4580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd674005fd0 tx=0x7fd674005850 comp rx=0 tx=0).stop 2024-02-17T12:47:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.378+0000 7fd67f481640 1 -- 172.21.15.18:0/1315874757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd678101310 msgr2=0x7fd6781999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.378+0000 7fd67f481640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd678101310 0x7fd6781999a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd668006010 tx=0x7fd6680042f0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.378+0000 7fd67f481640 1 -- 172.21.15.18:0/1315874757 shutdown_connections 2024-02-17T12:47:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.378+0000 7fd67f481640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd6640a20c0 0x7fd6640a4580 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.378+0000 7fd67f481640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd678101cd0 0x7fd6781a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.378+0000 7fd67f481640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd678101310 0x7fd6781999a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.378+0000 7fd67f481640 1 --2- 172.21.15.18:0/1315874757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd678100120 0x7fd678199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.378+0000 7fd67f481640 1 -- 172.21.15.18:0/1315874757 >> 172.21.15.18:0/1315874757 conn(0x7fd6780fb6b0 msgr2=0x7fd6780fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:46.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.378+0000 7fd67f481640 1 -- 172.21.15.18:0/1315874757 shutdown_connections 2024-02-17T12:47:46.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.378+0000 7fd67f481640 1 -- 172.21.15.18:0/1315874757 wait complete. 2024-02-17T12:47:46.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:46 smithi018 ceph-mon[32625]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:46.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:46 smithi018 ceph-mon[32625]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-17T12:47:46.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:46 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1706946054' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-17T12:47:46.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:46 smithi018 ceph-mon[32625]: osdmap e84: 12 total, 12 up, 12 in 2024-02-17T12:47:46.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:46 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2497355000' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-17T12:47:46.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:46 smithi018 ceph-mon[32625]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-17T12:47:46.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:46 smithi018 ceph-mon[32625]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-17T12:47:46.383 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:47:46.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:46 smithi146 ceph-mon[36454]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:46.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:46 smithi146 ceph-mon[36454]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-17T12:47:46.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:46 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1706946054' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-17T12:47:46.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:46 smithi146 ceph-mon[36454]: osdmap e84: 12 total, 12 up, 12 in 2024-02-17T12:47:46.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:46 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2497355000' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-17T12:47:46.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:46 smithi146 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-17T12:47:46.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:46 smithi146 ceph-mon[36454]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-17T12:47:46.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.493+0000 7f2e79eb5640 1 -- 172.21.15.18:0/2914726858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e74100410 msgr2=0x7f2e74100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.493+0000 7f2e79eb5640 1 --2- 172.21.15.18:0/2914726858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e74100410 0x7f2e74100870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2e68009900 tx=0x7f2e6802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.493+0000 7f2e79eb5640 1 -- 172.21.15.18:0/2914726858 shutdown_connections 2024-02-17T12:47:46.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.493+0000 7f2e79eb5640 1 --2- 172.21.15.18:0/2914726858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e74100db0 0x7f2e74101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.493+0000 7f2e79eb5640 1 --2- 172.21.15.18:0/2914726858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e74100410 0x7f2e74100870 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.493+0000 7f2e79eb5640 1 --2- 172.21.15.18:0/2914726858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e740ffab0 0x7f2e740ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.493+0000 7f2e79eb5640 1 -- 172.21.15.18:0/2914726858 >> 172.21.15.18:0/2914726858 conn(0x7f2e740fb630 msgr2=0x7f2e740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.493+0000 7f2e79eb5640 1 -- 172.21.15.18:0/2914726858 shutdown_connections 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.493+0000 7f2e79eb5640 1 -- 172.21.15.18:0/2914726858 wait complete. 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.493+0000 7f2e79eb5640 1 Processor -- start 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.493+0000 7f2e79eb5640 1 -- start start 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e79eb5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e740ffab0 0x7f2e741993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e79eb5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e74100410 0x7f2e74199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e79eb5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e74100db0 0x7f2e741a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e79eb5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e741a0ec0 con 0x7f2e740ffab0 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e79eb5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e741a1030 con 0x7f2e74100410 2024-02-17T12:47:46.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e79eb5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e741a11f0 con 0x7f2e74100db0 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e737fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e740ffab0 0x7f2e741993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e737fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e740ffab0 0x7f2e741993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35678/0 (socket says 172.21.15.18:35678) 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e737fe640 1 -- 172.21.15.18:0/1331525064 learned_addr learned my addr 172.21.15.18:0/1331525064 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e72ffd640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e74100410 0x7f2e74199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e73fff640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e74100db0 0x7f2e741a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e737fe640 1 -- 172.21.15.18:0/1331525064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e74100db0 msgr2=0x7f2e741a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e737fe640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e74100db0 0x7f2e741a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e737fe640 1 -- 172.21.15.18:0/1331525064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e74100410 msgr2=0x7f2e74199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e737fe640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e74100410 0x7f2e74199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e737fe640 1 -- 172.21.15.18:0/1331525064 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e68009560 con 0x7f2e740ffab0 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e72ffd640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e74100410 0x7f2e74199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e73fff640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e74100db0 0x7f2e741a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e737fe640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e740ffab0 0x7f2e741993d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400c920 tx=0x7f2e6400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:46.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e70ff9640 1 -- 172.21.15.18:0/1331525064 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e64007c70 con 0x7f2e740ffab0 2024-02-17T12:47:46.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e70ff9640 1 -- 172.21.15.18:0/1331525064 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e64007dd0 con 0x7f2e740ffab0 2024-02-17T12:47:46.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e79eb5640 1 -- 172.21.15.18:0/1331525064 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e741a14d0 con 0x7f2e740ffab0 2024-02-17T12:47:46.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e70ff9640 1 -- 172.21.15.18:0/1331525064 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e64018860 con 0x7f2e740ffab0 2024-02-17T12:47:46.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.494+0000 7f2e79eb5640 1 -- 172.21.15.18:0/1331525064 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e741a1a30 con 0x7f2e740ffab0 2024-02-17T12:47:46.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.495+0000 7f2e79eb5640 1 -- 172.21.15.18:0/1331525064 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7410a950 con 0x7f2e740ffab0 2024-02-17T12:47:46.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.496+0000 7f2e70ff9640 1 -- 172.21.15.18:0/1331525064 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2e6400fac0 con 0x7f2e740ffab0 2024-02-17T12:47:46.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.497+0000 7f2e70ff9640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2e5c0a2140 0x7f2e5c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:46.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.497+0000 7f2e72ffd640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2e5c0a2140 0x7f2e5c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:46.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.497+0000 7f2e70ff9640 1 -- 172.21.15.18:0/1331525064 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2e640cdcf0 con 0x7f2e740ffab0 2024-02-17T12:47:46.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.497+0000 7f2e72ffd640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2e5c0a2140 0x7f2e5c0a4600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e68036040 tx=0x7f2e68000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:46.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.501+0000 7f2e70ff9640 1 -- 172.21.15.18:0/1331525064 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e640d2070 con 0x7f2e740ffab0 2024-02-17T12:47:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.687+0000 7f2e79eb5640 1 -- 172.21.15.18:0/1331525064 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e7402ce40 con 0x7f2e740ffab0 2024-02-17T12:47:46.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.688+0000 7f2e70ff9640 1 -- 172.21.15.18:0/1331525064 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22234 (secure 0 0 0) 0x7f2e64096e10 con 0x7f2e740ffab0 2024-02-17T12:47:46.681 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:46.681 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":85,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:47:45.407544+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:47:46.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.691+0000 7f2e79eb5640 1 -- 172.21.15.18:0/1331525064 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2e5c0a2140 msgr2=0x7f2e5c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.691+0000 7f2e79eb5640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2e5c0a2140 0x7f2e5c0a4600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e68036040 tx=0x7f2e68000bc0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.691+0000 7f2e79eb5640 1 -- 172.21.15.18:0/1331525064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e740ffab0 msgr2=0x7f2e741993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.691+0000 7f2e79eb5640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e740ffab0 0x7f2e741993d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400c920 tx=0x7f2e6400cde0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.691+0000 7f2e79eb5640 1 -- 172.21.15.18:0/1331525064 shutdown_connections 2024-02-17T12:47:46.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.691+0000 7f2e79eb5640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2e5c0a2140 0x7f2e5c0a4600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.691+0000 7f2e79eb5640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e74100db0 0x7f2e741a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.691+0000 7f2e79eb5640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e74100410 0x7f2e74199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.691+0000 7f2e79eb5640 1 --2- 172.21.15.18:0/1331525064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e740ffab0 0x7f2e741993d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.691+0000 7f2e79eb5640 1 -- 172.21.15.18:0/1331525064 >> 172.21.15.18:0/1331525064 conn(0x7f2e740fb630 msgr2=0x7f2e740fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:46.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.691+0000 7f2e79eb5640 1 -- 172.21.15.18:0/1331525064 shutdown_connections 2024-02-17T12:47:46.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.691+0000 7f2e79eb5640 1 -- 172.21.15.18:0/1331525064 wait complete. 2024-02-17T12:47:46.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-17T12:47:46.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.808+0000 7f5e89c64640 1 -- 172.21.15.18:0/3092671069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84101c40 msgr2=0x7f5e841084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.808+0000 7f5e89c64640 1 --2- 172.21.15.18:0/3092671069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84101c40 0x7f5e841084d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009ca0 tx=0x7f5e7c02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.808+0000 7f5e89c64640 1 -- 172.21.15.18:0/3092671069 shutdown_connections 2024-02-17T12:47:46.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.808+0000 7f5e89c64640 1 --2- 172.21.15.18:0/3092671069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84101c40 0x7f5e841084d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.808+0000 7f5e89c64640 1 --2- 172.21.15.18:0/3092671069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e84101280 0x7f5e84101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.808+0000 7f5e89c64640 1 --2- 172.21.15.18:0/3092671069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84100090 0x7f5e841004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.808+0000 7f5e89c64640 1 -- 172.21.15.18:0/3092671069 >> 172.21.15.18:0/3092671069 conn(0x7f5e840fb620 msgr2=0x7f5e840fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:46.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.808+0000 7f5e89c64640 1 -- 172.21.15.18:0/3092671069 shutdown_connections 2024-02-17T12:47:46.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.808+0000 7f5e89c64640 1 -- 172.21.15.18:0/3092671069 wait complete. 2024-02-17T12:47:46.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.808+0000 7f5e89c64640 1 Processor -- start 2024-02-17T12:47:46.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e89c64640 1 -- start start 2024-02-17T12:47:46.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e89c64640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84100090 0x7f5e84196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:46.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e89c64640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84101280 0x7f5e84196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:46.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e89c64640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e84101c40 0x7f5e8419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:46.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e89c64640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8419dbf0 con 0x7f5e84100090 2024-02-17T12:47:46.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e89c64640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8419dd60 con 0x7f5e84101280 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e89c64640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8419df20 con 0x7f5e84101c40 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e837fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84100090 0x7f5e84196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e837fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84100090 0x7f5e84196100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35696/0 (socket says 172.21.15.18:35696) 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e837fe640 1 -- 172.21.15.18:0/2578962515 learned_addr learned my addr 172.21.15.18:0/2578962515 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e82ffd640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84101280 0x7f5e84196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e83fff640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e84101c40 0x7f5e8419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e837fe640 1 -- 172.21.15.18:0/2578962515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e84101c40 msgr2=0x7f5e8419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e837fe640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e84101c40 0x7f5e8419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e837fe640 1 -- 172.21.15.18:0/2578962515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84101280 msgr2=0x7f5e84196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e837fe640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84101280 0x7f5e84196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e837fe640 1 -- 172.21.15.18:0/2578962515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c009900 con 0x7f5e84100090 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e83fff640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e84101c40 0x7f5e8419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.809+0000 7f5e837fe640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84100090 0x7f5e84196100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400d770 tx=0x7f5e7400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.810+0000 7f5e80ff9640 1 -- 172.21.15.18:0/2578962515 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74009c30 con 0x7f5e84100090 2024-02-17T12:47:46.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.810+0000 7f5e80ff9640 1 -- 172.21.15.18:0/2578962515 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e7400de80 con 0x7f5e84100090 2024-02-17T12:47:46.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.810+0000 7f5e89c64640 1 -- 172.21.15.18:0/2578962515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8419e200 con 0x7f5e84100090 2024-02-17T12:47:46.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.810+0000 7f5e80ff9640 1 -- 172.21.15.18:0/2578962515 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74010460 con 0x7f5e84100090 2024-02-17T12:47:46.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.810+0000 7f5e89c64640 1 -- 172.21.15.18:0/2578962515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8419e710 con 0x7f5e84100090 2024-02-17T12:47:46.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.811+0000 7f5e89c64640 1 -- 172.21.15.18:0/2578962515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e841073b0 con 0x7f5e84100090 2024-02-17T12:47:46.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.812+0000 7f5e80ff9640 1 -- 172.21.15.18:0/2578962515 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5e74003f60 con 0x7f5e84100090 2024-02-17T12:47:46.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.812+0000 7f5e80ff9640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5e6c0a20c0 0x7f5e6c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:46.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.812+0000 7f5e82ffd640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5e6c0a20c0 0x7f5e6c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:46.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.812+0000 7f5e80ff9640 1 -- 172.21.15.18:0/2578962515 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5e740cd850 con 0x7f5e84100090 2024-02-17T12:47:46.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.813+0000 7f5e82ffd640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5e6c0a20c0 0x7f5e6c0a4580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5e84197610 tx=0x7f5e7800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:46.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:46.817+0000 7f5e80ff9640 1 -- 172.21.15.18:0/2578962515 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e74096970 con 0x7f5e84100090 2024-02-17T12:47:47.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.036+0000 7f5e89c64640 1 -- 172.21.15.18:0/2578962515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5e8402cda0 con 0x7f5e84100090 2024-02-17T12:47:47.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.062+0000 7f5e80ff9640 1 -- 172.21.15.18:0/2578962515 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5e7409a030 con 0x7f5e84100090 2024-02-17T12:47:47.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.064+0000 7f5e89c64640 1 -- 172.21.15.18:0/2578962515 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5e6c0a20c0 msgr2=0x7f5e6c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:47.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.064+0000 7f5e89c64640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5e6c0a20c0 0x7f5e6c0a4580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5e84197610 tx=0x7f5e7800b390 comp rx=0 tx=0).stop 2024-02-17T12:47:47.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.064+0000 7f5e89c64640 1 -- 172.21.15.18:0/2578962515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84100090 msgr2=0x7f5e84196100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:47.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.064+0000 7f5e89c64640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84100090 0x7f5e84196100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400d770 tx=0x7f5e7400dc30 comp rx=0 tx=0).stop 2024-02-17T12:47:47.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.064+0000 7f5e89c64640 1 -- 172.21.15.18:0/2578962515 shutdown_connections 2024-02-17T12:47:47.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.064+0000 7f5e89c64640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5e6c0a20c0 0x7f5e6c0a4580 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:47.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.064+0000 7f5e89c64640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e84101c40 0x7f5e8419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:47.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.064+0000 7f5e89c64640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84101280 0x7f5e84196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:47.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.064+0000 7f5e89c64640 1 --2- 172.21.15.18:0/2578962515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e84100090 0x7f5e84196100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:47.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.064+0000 7f5e89c64640 1 -- 172.21.15.18:0/2578962515 >> 172.21.15.18:0/2578962515 conn(0x7f5e840fb620 msgr2=0x7f5e84104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:47.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.065+0000 7f5e89c64640 1 -- 172.21.15.18:0/2578962515 shutdown_connections 2024-02-17T12:47:47.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.065+0000 7f5e89c64640 1 -- 172.21.15.18:0/2578962515 wait complete. 2024-02-17T12:47:47.068 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-02-17T12:47:47.069 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-02-17T12:47:47.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.183+0000 7fe74fed4640 1 -- 172.21.15.18:0/316711419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7480ff830 msgr2=0x7fe7480ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:47.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.183+0000 7fe74fed4640 1 --2- 172.21.15.18:0/316711419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7480ff830 0x7fe7480ffc50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe740009900 tx=0x7fe74002bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:47.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.183+0000 7fe74fed4640 1 -- 172.21.15.18:0/316711419 shutdown_connections 2024-02-17T12:47:47.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.183+0000 7fe74fed4640 1 --2- 172.21.15.18:0/316711419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7481013e0 0x7fe748107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:47.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.183+0000 7fe74fed4640 1 --2- 172.21.15.18:0/316711419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe748100a20 0x7fe748100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:47.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.183+0000 7fe74fed4640 1 --2- 172.21.15.18:0/316711419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7480ff830 0x7fe7480ffc50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:47.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.183+0000 7fe74fed4640 1 -- 172.21.15.18:0/316711419 >> 172.21.15.18:0/316711419 conn(0x7fe7480fad80 msgr2=0x7fe7480fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.183+0000 7fe74fed4640 1 -- 172.21.15.18:0/316711419 shutdown_connections 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.183+0000 7fe74fed4640 1 -- 172.21.15.18:0/316711419 wait complete. 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74fed4640 1 Processor -- start 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74fed4640 1 -- start start 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74fed4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7480ff830 0x7fe748109100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74fed4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe748100a20 0x7fe74819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74fed4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7481013e0 0x7fe74819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74fed4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74819ed40 con 0x7fe7481013e0 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74fed4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74819eeb0 con 0x7fe748100a20 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74fed4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74819f070 con 0x7fe7480ff830 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74e471640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7481013e0 0x7fe74819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74e471640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7481013e0 0x7fe74819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35704/0 (socket says 172.21.15.18:35704) 2024-02-17T12:47:47.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74e471640 1 -- 172.21.15.18:0/3352038532 learned_addr learned my addr 172.21.15.18:0/3352038532 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74dc70640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7480ff830 0x7fe748109100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.184+0000 7fe74d46f640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe748100a20 0x7fe74819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.185+0000 7fe74e471640 1 -- 172.21.15.18:0/3352038532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7480ff830 msgr2=0x7fe748109100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.185+0000 7fe74e471640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7480ff830 0x7fe748109100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.185+0000 7fe74e471640 1 -- 172.21.15.18:0/3352038532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe748100a20 msgr2=0x7fe74819bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.185+0000 7fe74e471640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe748100a20 0x7fe74819bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.185+0000 7fe74e471640 1 -- 172.21.15.18:0/3352038532 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe740009560 con 0x7fe7481013e0 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.185+0000 7fe74e471640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7481013e0 0x7fe74819e5b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe73800eb10 tx=0x7fe73800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.185+0000 7fe746ffd640 1 -- 172.21.15.18:0/3352038532 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738004b20 con 0x7fe7481013e0 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.185+0000 7fe746ffd640 1 -- 172.21.15.18:0/3352038532 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe738004c80 con 0x7fe7481013e0 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.185+0000 7fe74fed4640 1 -- 172.21.15.18:0/3352038532 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74819f350 con 0x7fe7481013e0 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.185+0000 7fe746ffd640 1 -- 172.21.15.18:0/3352038532 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738010650 con 0x7fe7481013e0 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.185+0000 7fe74fed4640 1 -- 172.21.15.18:0/3352038532 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74819f8b0 con 0x7fe7481013e0 2024-02-17T12:47:47.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.186+0000 7fe74fed4640 1 -- 172.21.15.18:0/3352038532 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe748106b50 con 0x7fe7481013e0 2024-02-17T12:47:47.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.187+0000 7fe746ffd640 1 -- 172.21.15.18:0/3352038532 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe738005020 con 0x7fe7481013e0 2024-02-17T12:47:47.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.187+0000 7fe746ffd640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe7300a2140 0x7fe7300a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:47.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.188+0000 7fe74dc70640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe7300a2140 0x7fe7300a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:47.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.188+0000 7fe746ffd640 1 -- 172.21.15.18:0/3352038532 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe738014070 con 0x7fe7481013e0 2024-02-17T12:47:47.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.188+0000 7fe74dc70640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe7300a2140 0x7fe7300a4600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe740005ab0 tx=0x7fe740005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:47.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.192+0000 7fe746ffd640 1 -- 172.21.15.18:0/3352038532 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe7380970a0 con 0x7fe7481013e0 2024-02-17T12:47:47.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:47 smithi119 ceph-mon[36188]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-17T12:47:47.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:47 smithi119 ceph-mon[36188]: osdmap e85: 12 total, 12 up, 12 in 2024-02-17T12:47:47.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:47 smithi119 ceph-mon[36188]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-17T12:47:47.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:47 smithi119 ceph-mon[36188]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-17T12:47:47.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:47 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1315874757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:47.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:47 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1331525064' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:47.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:47 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2578962515' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-17T12:47:47.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:47.416+0000 7fe74fed4640 1 -- 172.21.15.18:0/3352038532 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fe74810bd30 con 0x7fe7481013e0 2024-02-17T12:47:47.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:47 smithi146 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-17T12:47:47.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:47 smithi146 ceph-mon[36454]: osdmap e85: 12 total, 12 up, 12 in 2024-02-17T12:47:47.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:47 smithi146 ceph-mon[36454]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-17T12:47:47.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:47 smithi146 ceph-mon[36454]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-17T12:47:47.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:47 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1315874757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:47.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:47 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1331525064' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:47.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:47 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2578962515' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-17T12:47:47.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:47 smithi018 ceph-mon[32625]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-17T12:47:47.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:47 smithi018 ceph-mon[32625]: osdmap e85: 12 total, 12 up, 12 in 2024-02-17T12:47:47.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:47 smithi018 ceph-mon[32625]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-17T12:47:47.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:47 smithi018 ceph-mon[32625]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-17T12:47:47.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:47 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1315874757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:47.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:47 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1331525064' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:47.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:47 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2578962515' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-17T12:47:48.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.068+0000 7fe746ffd640 1 -- 172.21.15.18:0/3352038532 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v16) v1 ==== 105+0+0 (secure 0 0 0) 0x7fe73809a760 con 0x7fe7481013e0 2024-02-17T12:47:48.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.070+0000 7fe74fed4640 1 -- 172.21.15.18:0/3352038532 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe7300a2140 msgr2=0x7fe7300a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:48.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.070+0000 7fe74fed4640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe7300a2140 0x7fe7300a4600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe740005ab0 tx=0x7fe740005fd0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.070+0000 7fe74fed4640 1 -- 172.21.15.18:0/3352038532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7481013e0 msgr2=0x7fe74819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:48.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.070+0000 7fe74fed4640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7481013e0 0x7fe74819e5b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe73800eb10 tx=0x7fe73800c2d0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.070+0000 7fe74fed4640 1 -- 172.21.15.18:0/3352038532 shutdown_connections 2024-02-17T12:47:48.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.070+0000 7fe74fed4640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe7300a2140 0x7fe7300a4600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.070+0000 7fe74fed4640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7481013e0 0x7fe74819e5b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.070+0000 7fe74fed4640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe748100a20 0x7fe74819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.070+0000 7fe74fed4640 1 --2- 172.21.15.18:0/3352038532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7480ff830 0x7fe748109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.070+0000 7fe74fed4640 1 -- 172.21.15.18:0/3352038532 >> 172.21.15.18:0/3352038532 conn(0x7fe7480fad80 msgr2=0x7fe748103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:48.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.071+0000 7fe74fed4640 1 -- 172.21.15.18:0/3352038532 shutdown_connections 2024-02-17T12:47:48.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.071+0000 7fe74fed4640 1 -- 172.21.15.18:0/3352038532 wait complete. 2024-02-17T12:47:48.079 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-02-17T12:47:48.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.196+0000 7f9114f0f640 1 -- 172.21.15.18:0/570087472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9110106f20 msgr2=0x7f9110109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:48.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.196+0000 7f9114f0f640 1 --2- 172.21.15.18:0/570087472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9110106f20 0x7f9110109310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9100009ca0 tx=0x7f910002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.196+0000 7f9114f0f640 1 -- 172.21.15.18:0/570087472 shutdown_connections 2024-02-17T12:47:48.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.196+0000 7f9114f0f640 1 --2- 172.21.15.18:0/570087472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9110106f20 0x7f9110109310 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.196+0000 7f9114f0f640 1 --2- 172.21.15.18:0/570087472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91101045f0 0x7f91101069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.196+0000 7f9114f0f640 1 --2- 172.21.15.18:0/570087472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9110101cc0 0x7f91101040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.196+0000 7f9114f0f640 1 -- 172.21.15.18:0/570087472 >> 172.21.15.18:0/570087472 conn(0x7f91100fb630 msgr2=0x7f91100fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.196+0000 7f9114f0f640 1 -- 172.21.15.18:0/570087472 shutdown_connections 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.196+0000 7f9114f0f640 1 -- 172.21.15.18:0/570087472 wait complete. 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.197+0000 7f9114f0f640 1 Processor -- start 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.197+0000 7f9114f0f640 1 -- start start 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.197+0000 7f9114f0f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9110101cc0 0x7f9110197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.197+0000 7f9114f0f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91101045f0 0x7f91101977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.197+0000 7f9114f0f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9110106f20 0x7f911019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.197+0000 7f9114f0f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911019ed70 con 0x7f9110106f20 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.197+0000 7f9114f0f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911019eee0 con 0x7f9110101cc0 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.197+0000 7f9114f0f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911019f0a0 con 0x7f91101045f0 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.197+0000 7f910ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9110106f20 0x7f911019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.197+0000 7f910ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9110106f20 0x7f911019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35716/0 (socket says 172.21.15.18:35716) 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.197+0000 7f910ed9d640 1 -- 172.21.15.18:0/3062913099 learned_addr learned my addr 172.21.15.18:0/3062913099 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.197+0000 7f910dd9b640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91101045f0 0x7f91101977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.198+0000 7f910e59c640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9110101cc0 0x7f9110197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.198+0000 7f910ed9d640 1 -- 172.21.15.18:0/3062913099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91101045f0 msgr2=0x7f91101977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.198+0000 7f910ed9d640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91101045f0 0x7f91101977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.198+0000 7f910ed9d640 1 -- 172.21.15.18:0/3062913099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9110101cc0 msgr2=0x7f9110197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.198+0000 7f910ed9d640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9110101cc0 0x7f9110197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.198+0000 7f910ed9d640 1 -- 172.21.15.18:0/3062913099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9100009900 con 0x7f9110106f20 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.198+0000 7f910ed9d640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9110106f20 0x7f911019e830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9100037040 tx=0x7f9100034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.198+0000 7f91077fe640 1 -- 172.21.15.18:0/3062913099 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100032eb0 con 0x7f9110106f20 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.198+0000 7f91077fe640 1 -- 172.21.15.18:0/3062913099 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9100003e50 con 0x7f9110106f20 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.198+0000 7f9114f0f640 1 -- 172.21.15.18:0/3062913099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911019f320 con 0x7f9110106f20 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.198+0000 7f91077fe640 1 -- 172.21.15.18:0/3062913099 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100040450 con 0x7f9110106f20 2024-02-17T12:47:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.198+0000 7f9114f0f640 1 -- 172.21.15.18:0/3062913099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911019f7d0 con 0x7f9110106f20 2024-02-17T12:47:48.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.199+0000 7f9114f0f640 1 -- 172.21.15.18:0/3062913099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91100fd120 con 0x7f9110106f20 2024-02-17T12:47:48.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.200+0000 7f91077fe640 1 -- 172.21.15.18:0/3062913099 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f910000a040 con 0x7f9110106f20 2024-02-17T12:47:48.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.201+0000 7f91077fe640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f90fc0a20c0 0x7f90fc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:48.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.201+0000 7f910e59c640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f90fc0a20c0 0x7f90fc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:48.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.201+0000 7f91077fe640 1 -- 172.21.15.18:0/3062913099 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f91000f0f70 con 0x7f9110106f20 2024-02-17T12:47:48.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.201+0000 7f910e59c640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f90fc0a20c0 0x7f90fc0a4580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9108009890 tx=0x7f9108008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.204+0000 7f91077fe640 1 -- 172.21.15.18:0/3062913099 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f91000ba090 con 0x7f9110106f20 2024-02-17T12:47:48.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:48 smithi119 ceph-mon[36188]: pgmap v283: 97 pgs: 22 active+clean, 42 creating+peering, 33 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:48.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:48 smithi119 ceph-mon[36188]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-17T12:47:48.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:48 smithi119 ceph-mon[36188]: Cluster is now healthy 2024-02-17T12:47:48.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:48 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2578962515' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-17T12:47:48.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:48 smithi119 ceph-mon[36188]: osdmap e86: 12 total, 12 up, 12 in 2024-02-17T12:47:48.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:48 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] up:active 2024-02-17T12:47:48.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:48 smithi119 ceph-mon[36188]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-17T12:47:48.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:48 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3352038532' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-02-17T12:47:48.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.428+0000 7f9114f0f640 1 -- 172.21.15.18:0/3062913099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f911010cc30 con 0x7f9110106f20 2024-02-17T12:47:48.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:48 smithi146 ceph-mon[36454]: pgmap v283: 97 pgs: 22 active+clean, 42 creating+peering, 33 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:48.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:48 smithi146 ceph-mon[36454]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-17T12:47:48.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:48 smithi146 ceph-mon[36454]: Cluster is now healthy 2024-02-17T12:47:48.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:48 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2578962515' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-17T12:47:48.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:48 smithi146 ceph-mon[36454]: osdmap e86: 12 total, 12 up, 12 in 2024-02-17T12:47:48.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:48 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] up:active 2024-02-17T12:47:48.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:48 smithi146 ceph-mon[36454]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-17T12:47:48.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:48 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3352038532' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-02-17T12:47:48.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:48 smithi018 ceph-mon[32625]: pgmap v283: 97 pgs: 22 active+clean, 42 creating+peering, 33 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:47:48.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:48 smithi018 ceph-mon[32625]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-17T12:47:48.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:48 smithi018 ceph-mon[32625]: Cluster is now healthy 2024-02-17T12:47:48.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:48 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2578962515' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-17T12:47:48.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:48 smithi018 ceph-mon[32625]: osdmap e86: 12 total, 12 up, 12 in 2024-02-17T12:47:48.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:48 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] up:active 2024-02-17T12:47:48.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:48 smithi018 ceph-mon[32625]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-17T12:47:48.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:48 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3352038532' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-02-17T12:47:48.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.696+0000 7f91077fe640 1 -- 172.21.15.18:0/3062913099 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7f91000bd750 con 0x7f9110106f20 2024-02-17T12:47:48.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.698+0000 7f9114f0f640 1 -- 172.21.15.18:0/3062913099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f90fc0a20c0 msgr2=0x7f90fc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:48.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.698+0000 7f9114f0f640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f90fc0a20c0 0x7f90fc0a4580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9108009890 tx=0x7f9108008040 comp rx=0 tx=0).stop 2024-02-17T12:47:48.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.698+0000 7f9114f0f640 1 -- 172.21.15.18:0/3062913099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9110106f20 msgr2=0x7f911019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:48.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.698+0000 7f9114f0f640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9110106f20 0x7f911019e830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9100037040 tx=0x7f9100034d40 comp rx=0 tx=0).stop 2024-02-17T12:47:48.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.698+0000 7f9114f0f640 1 -- 172.21.15.18:0/3062913099 shutdown_connections 2024-02-17T12:47:48.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.698+0000 7f9114f0f640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f90fc0a20c0 0x7f90fc0a4580 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.698+0000 7f9114f0f640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9110106f20 0x7f911019e830 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.698+0000 7f9114f0f640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91101045f0 0x7f91101977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.698+0000 7f9114f0f640 1 --2- 172.21.15.18:0/3062913099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9110101cc0 0x7f9110197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.698+0000 7f9114f0f640 1 -- 172.21.15.18:0/3062913099 >> 172.21.15.18:0/3062913099 conn(0x7f91100fb630 msgr2=0x7f9110102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:48.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.699+0000 7f9114f0f640 1 -- 172.21.15.18:0/3062913099 shutdown_connections 2024-02-17T12:47:48.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.699+0000 7f9114f0f640 1 -- 172.21.15.18:0/3062913099 wait complete. 2024-02-17T12:47:48.706 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-02-17T12:47:48.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.820+0000 7fdb792f4640 1 -- 172.21.15.18:0/1021675336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb74101c50 msgr2=0x7fdb741084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:48.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.820+0000 7fdb792f4640 1 --2- 172.21.15.18:0/1021675336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb74101c50 0x7fdb741084e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009ca0 tx=0x7fdb6c02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.820+0000 7fdb792f4640 1 -- 172.21.15.18:0/1021675336 shutdown_connections 2024-02-17T12:47:48.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.820+0000 7fdb792f4640 1 --2- 172.21.15.18:0/1021675336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb74101c50 0x7fdb741084e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.820+0000 7fdb792f4640 1 --2- 172.21.15.18:0/1021675336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74101290 0x7fdb74101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.820+0000 7fdb792f4640 1 --2- 172.21.15.18:0/1021675336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb741000a0 0x7fdb741004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.820+0000 7fdb792f4640 1 -- 172.21.15.18:0/1021675336 >> 172.21.15.18:0/1021675336 conn(0x7fdb740fb630 msgr2=0x7fdb740fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.820+0000 7fdb792f4640 1 -- 172.21.15.18:0/1021675336 shutdown_connections 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.820+0000 7fdb792f4640 1 -- 172.21.15.18:0/1021675336 wait complete. 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb792f4640 1 Processor -- start 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb792f4640 1 -- start start 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb792f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb741000a0 0x7fdb74197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb792f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb74101290 0x7fdb74197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb792f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74101c50 0x7fdb7419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb792f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7419ec30 con 0x7fdb74101290 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb792f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7419eda0 con 0x7fdb74101c50 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb792f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7419ef60 con 0x7fdb741000a0 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb6bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb74101290 0x7fdb74197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb6bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb74101290 0x7fdb74197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35730/0 (socket says 172.21.15.18:35730) 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb6bfff640 1 -- 172.21.15.18:0/1939965645 learned_addr learned my addr 172.21.15.18:0/1939965645 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb72ffd640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb741000a0 0x7fdb74197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb737fe640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74101c50 0x7fdb7419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb6bfff640 1 -- 172.21.15.18:0/1939965645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb741000a0 msgr2=0x7fdb74197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb6bfff640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb741000a0 0x7fdb74197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb6bfff640 1 -- 172.21.15.18:0/1939965645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74101c50 msgr2=0x7fdb7419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb6bfff640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74101c50 0x7fdb7419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb6bfff640 1 -- 172.21.15.18:0/1939965645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6c009900 con 0x7fdb74101290 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb737fe640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74101c50 0x7fdb7419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb6bfff640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb74101290 0x7fdb74197680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c920 tx=0x7fdb5c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb70ff9640 1 -- 172.21.15.18:0/1939965645 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c007c70 con 0x7fdb74101290 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb70ff9640 1 -- 172.21.15.18:0/1939965645 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb5c007dd0 con 0x7fdb74101290 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.821+0000 7fdb792f4640 1 -- 172.21.15.18:0/1939965645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7419f240 con 0x7fdb74101290 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.822+0000 7fdb70ff9640 1 -- 172.21.15.18:0/1939965645 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c018860 con 0x7fdb74101290 2024-02-17T12:47:48.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.822+0000 7fdb792f4640 1 -- 172.21.15.18:0/1939965645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7419f7a0 con 0x7fdb74101290 2024-02-17T12:47:48.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.823+0000 7fdb792f4640 1 -- 172.21.15.18:0/1939965645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb74107360 con 0x7fdb74101290 2024-02-17T12:47:48.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.824+0000 7fdb70ff9640 1 -- 172.21.15.18:0/1939965645 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fdb5c00fac0 con 0x7fdb74101290 2024-02-17T12:47:48.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.824+0000 7fdb70ff9640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdb600a2140 0x7fdb600a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:48.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.825+0000 7fdb72ffd640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdb600a2140 0x7fdb600a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:48.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.825+0000 7fdb70ff9640 1 -- 172.21.15.18:0/1939965645 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdb5c0cdc00 con 0x7fdb74101290 2024-02-17T12:47:48.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.825+0000 7fdb72ffd640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdb600a2140 0x7fdb600a4600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdb640058c0 tx=0x7fdb6400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:48.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:48.829+0000 7fdb70ff9640 1 -- 172.21.15.18:0/1939965645 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb5c0d4080 con 0x7fdb74101290 2024-02-17T12:47:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.048+0000 7fdb792f4640 1 -- 172.21.15.18:0/1939965645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fdb7402ce40 con 0x7fdb74101290 2024-02-17T12:47:49.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-17T12:47:49.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3352038532' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-02-17T12:47:49.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-17T12:47:49.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-17T12:47:49.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-02-17T12:47:49.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-02-17T12:47:49.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3062913099' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-17T12:47:49.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3062913099' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-17T12:47:49.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] up:active 2024-02-17T12:47:49.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-17T12:47:49.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2024-02-17T12:47:49.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2024-02-17T12:47:49.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-02-17T12:47:49.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:49 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1939965645' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-17T12:47:49.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-17T12:47:49.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3352038532' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-02-17T12:47:49.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-17T12:47:49.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-17T12:47:49.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-02-17T12:47:49.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-02-17T12:47:49.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3062913099' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-17T12:47:49.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3062913099' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-17T12:47:49.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] up:active 2024-02-17T12:47:49.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-17T12:47:49.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2024-02-17T12:47:49.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2024-02-17T12:47:49.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-02-17T12:47:49.453 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:49 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1939965645' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-17T12:47:49.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-17T12:47:49.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3352038532' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-02-17T12:47:49.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-17T12:47:49.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-17T12:47:49.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-02-17T12:47:49.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-02-17T12:47:49.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3062913099' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-17T12:47:49.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3062913099' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-17T12:47:49.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] up:active 2024-02-17T12:47:49.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-17T12:47:49.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2024-02-17T12:47:49.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2024-02-17T12:47:49.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-02-17T12:47:49.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:49 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1939965645' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-17T12:47:49.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.702+0000 7fdb70ff9640 1 -- 172.21.15.18:0/1939965645 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7fdb5c096d20 con 0x7fdb74101290 2024-02-17T12:47:49.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.705+0000 7fdb792f4640 1 -- 172.21.15.18:0/1939965645 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdb600a2140 msgr2=0x7fdb600a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:49.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.705+0000 7fdb792f4640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdb600a2140 0x7fdb600a4600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdb640058c0 tx=0x7fdb6400a2d0 comp rx=0 tx=0).stop 2024-02-17T12:47:49.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.705+0000 7fdb792f4640 1 -- 172.21.15.18:0/1939965645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb74101290 msgr2=0x7fdb74197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:49.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.705+0000 7fdb792f4640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb74101290 0x7fdb74197680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c920 tx=0x7fdb5c00cde0 comp rx=0 tx=0).stop 2024-02-17T12:47:49.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.705+0000 7fdb792f4640 1 -- 172.21.15.18:0/1939965645 shutdown_connections 2024-02-17T12:47:49.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.705+0000 7fdb792f4640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdb600a2140 0x7fdb600a4600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:49.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.705+0000 7fdb792f4640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74101c50 0x7fdb7419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:49.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.705+0000 7fdb792f4640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb74101290 0x7fdb74197680 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:49.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.705+0000 7fdb792f4640 1 --2- 172.21.15.18:0/1939965645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb741000a0 0x7fdb74197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:49.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.705+0000 7fdb792f4640 1 -- 172.21.15.18:0/1939965645 >> 172.21.15.18:0/1939965645 conn(0x7fdb740fb630 msgr2=0x7fdb74104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:49.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.705+0000 7fdb792f4640 1 -- 172.21.15.18:0/1939965645 shutdown_connections 2024-02-17T12:47:49.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.705+0000 7fdb792f4640 1 -- 172.21.15.18:0/1939965645 wait complete. 2024-02-17T12:47:49.714 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-02-17T12:47:49.714 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-02-17T12:47:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.832+0000 7f5c19ec0640 1 -- 172.21.15.18:0/2785688780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c14104580 msgr2=0x7f5c14106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.832+0000 7f5c19ec0640 1 --2- 172.21.15.18:0/2785688780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c14104580 0x7f5c14106970 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5c08009900 tx=0x7f5c0802bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.832+0000 7f5c19ec0640 1 -- 172.21.15.18:0/2785688780 shutdown_connections 2024-02-17T12:47:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.832+0000 7f5c19ec0640 1 --2- 172.21.15.18:0/2785688780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c14106eb0 0x7f5c141092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.832+0000 7f5c19ec0640 1 --2- 172.21.15.18:0/2785688780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c14104580 0x7f5c14106970 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.833+0000 7f5c19ec0640 1 --2- 172.21.15.18:0/2785688780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c14101c50 0x7f5c14104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.833+0000 7f5c19ec0640 1 -- 172.21.15.18:0/2785688780 >> 172.21.15.18:0/2785688780 conn(0x7f5c140fb620 msgr2=0x7f5c140fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.833+0000 7f5c19ec0640 1 -- 172.21.15.18:0/2785688780 shutdown_connections 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.833+0000 7f5c19ec0640 1 -- 172.21.15.18:0/2785688780 wait complete. 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.833+0000 7f5c19ec0640 1 Processor -- start 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.833+0000 7f5c19ec0640 1 -- start start 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.833+0000 7f5c19ec0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c14101c50 0x7f5c14197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.833+0000 7f5c19ec0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c14104580 0x7f5c141977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.833+0000 7f5c19ec0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c14106eb0 0x7f5c1419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.833+0000 7f5c19ec0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1419ed80 con 0x7f5c14101c50 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.833+0000 7f5c19ec0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1419eef0 con 0x7f5c14104580 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.833+0000 7f5c19ec0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1419f0b0 con 0x7f5c14106eb0 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c137fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c14101c50 0x7f5c14197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c137fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c14101c50 0x7f5c14197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39088/0 (socket says 172.21.15.18:39088) 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c137fe640 1 -- 172.21.15.18:0/889214166 learned_addr learned my addr 172.21.15.18:0/889214166 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c12ffd640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c14104580 0x7f5c141977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c137fe640 1 -- 172.21.15.18:0/889214166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c14106eb0 msgr2=0x7f5c1419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c137fe640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c14106eb0 0x7f5c1419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c137fe640 1 -- 172.21.15.18:0/889214166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c14104580 msgr2=0x7f5c141977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c137fe640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c14104580 0x7f5c141977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c137fe640 1 -- 172.21.15.18:0/889214166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c08009560 con 0x7f5c14101c50 2024-02-17T12:47:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c137fe640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c14101c50 0x7f5c14197290 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400d950 tx=0x7f5c0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c10ff9640 1 -- 172.21.15.18:0/889214166 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04009c30 con 0x7f5c14101c50 2024-02-17T12:47:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c19ec0640 1 -- 172.21.15.18:0/889214166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1419f390 con 0x7f5c14101c50 2024-02-17T12:47:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c10ff9640 1 -- 172.21.15.18:0/889214166 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c04009d90 con 0x7f5c14101c50 2024-02-17T12:47:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c10ff9640 1 -- 172.21.15.18:0/889214166 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400f7e0 con 0x7f5c14101c50 2024-02-17T12:47:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.834+0000 7f5c19ec0640 1 -- 172.21.15.18:0/889214166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1419f8f0 con 0x7f5c14101c50 2024-02-17T12:47:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.835+0000 7f5c19ec0640 1 -- 172.21.15.18:0/889214166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c140fd110 con 0x7f5c14101c50 2024-02-17T12:47:49.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.836+0000 7f5c10ff9640 1 -- 172.21.15.18:0/889214166 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5c04010ad0 con 0x7f5c14101c50 2024-02-17T12:47:49.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.837+0000 7f5c10ff9640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5bfc0a20c0 0x7f5bfc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:49.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.837+0000 7f5c12ffd640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5bfc0a20c0 0x7f5bfc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:49.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.837+0000 7f5c10ff9640 1 -- 172.21.15.18:0/889214166 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5c040cd8d0 con 0x7f5c14101c50 2024-02-17T12:47:49.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.837+0000 7f5c12ffd640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5bfc0a20c0 0x7f5bfc0a4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c08036040 tx=0x7f5c08005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:49.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:49.841+0000 7f5c10ff9640 1 -- 172.21.15.18:0/889214166 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c04096950 con 0x7f5c14101c50 2024-02-17T12:47:50.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.050+0000 7f5c19ec0640 1 -- 172.21.15.18:0/889214166 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c1402cee0 con 0x7f5bfc0a20c0 2024-02-17T12:47:50.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:50 smithi119 ceph-mon[36188]: pgmap v285: 97 pgs: 29 active+clean, 42 creating+peering, 26 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-02-17T12:47:50.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:50 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1939965645' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-17T12:47:50.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:50 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] up:active 2024-02-17T12:47:50.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:50 smithi119 ceph-mon[36188]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-02-17T12:47:50.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:50 smithi119 ceph-mon[36188]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-02-17T12:47:50.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:50 smithi119 ceph-mon[36188]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:creating} 2 up:standby-replay 6 up:standby 2024-02-17T12:47:50.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:50 smithi119 ceph-mon[36188]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-02-17T12:47:50.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:50 smithi146 ceph-mon[36454]: pgmap v285: 97 pgs: 29 active+clean, 42 creating+peering, 26 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-02-17T12:47:50.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:50 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1939965645' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-17T12:47:50.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:50 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] up:active 2024-02-17T12:47:50.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:50 smithi146 ceph-mon[36454]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-02-17T12:47:50.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:50 smithi146 ceph-mon[36454]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-02-17T12:47:50.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:50 smithi146 ceph-mon[36454]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:creating} 2 up:standby-replay 6 up:standby 2024-02-17T12:47:50.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:50 smithi146 ceph-mon[36454]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-02-17T12:47:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:50 smithi018 ceph-mon[32625]: pgmap v285: 97 pgs: 29 active+clean, 42 creating+peering, 26 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-02-17T12:47:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:50 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1939965645' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-17T12:47:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:50 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] up:active 2024-02-17T12:47:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:50 smithi018 ceph-mon[32625]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-02-17T12:47:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:50 smithi018 ceph-mon[32625]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-02-17T12:47:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:50 smithi018 ceph-mon[32625]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:creating} 2 up:standby-replay 6 up:standby 2024-02-17T12:47:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:50 smithi018 ceph-mon[32625]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-02-17T12:47:50.526 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:47:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:47:50.055+0000 7efe12816700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:47:50.526 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:47:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:47:50.055+0000 7efe12816700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:47:50.526 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:47:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:47:50.055+0000 7efe12816700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:47:50.526 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:47:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:47:50.055+0000 7efe12816700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:47:50.526 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:47:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:47:50.055+0000 7efe12816700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:47:50.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.733+0000 7f5c10ff9640 1 -- 172.21.15.18:0/889214166 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5c1402cee0 con 0x7f5bfc0a20c0 2024-02-17T12:47:50.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.735+0000 7f5c19ec0640 1 -- 172.21.15.18:0/889214166 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5bfc0a20c0 msgr2=0x7f5bfc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:50.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.735+0000 7f5c19ec0640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5bfc0a20c0 0x7f5bfc0a4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c08036040 tx=0x7f5c08005fd0 comp rx=0 tx=0).stop 2024-02-17T12:47:50.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.735+0000 7f5c19ec0640 1 -- 172.21.15.18:0/889214166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c14101c50 msgr2=0x7f5c14197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:50.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.735+0000 7f5c19ec0640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c14101c50 0x7f5c14197290 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400d950 tx=0x7f5c0400de10 comp rx=0 tx=0).stop 2024-02-17T12:47:50.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.735+0000 7f5c19ec0640 1 -- 172.21.15.18:0/889214166 shutdown_connections 2024-02-17T12:47:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.735+0000 7f5c19ec0640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5bfc0a20c0 0x7f5bfc0a4580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.735+0000 7f5c19ec0640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c14106eb0 0x7f5c1419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.735+0000 7f5c19ec0640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c14104580 0x7f5c141977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.735+0000 7f5c19ec0640 1 --2- 172.21.15.18:0/889214166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c14101c50 0x7f5c14197290 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.735+0000 7f5c19ec0640 1 -- 172.21.15.18:0/889214166 >> 172.21.15.18:0/889214166 conn(0x7f5c140fb620 msgr2=0x7f5c14102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.735+0000 7f5c19ec0640 1 -- 172.21.15.18:0/889214166 shutdown_connections 2024-02-17T12:47:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.735+0000 7f5c19ec0640 1 -- 172.21.15.18:0/889214166 wait complete. 2024-02-17T12:47:50.740 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-02-17T12:47:50.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 -- 172.21.15.18:0/4008418134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51a4100080 msgr2=0x7f51a41004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:50.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 --2- 172.21.15.18:0/4008418134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51a4100080 0x7f51a41004a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5194009900 tx=0x7f519402bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:50.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 -- 172.21.15.18:0/4008418134 shutdown_connections 2024-02-17T12:47:50.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 --2- 172.21.15.18:0/4008418134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51a4101c30 0x7f51a41084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:50.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 --2- 172.21.15.18:0/4008418134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51a4101270 0x7f51a41016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:50.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 --2- 172.21.15.18:0/4008418134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51a4100080 0x7f51a41004a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:50.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 -- 172.21.15.18:0/4008418134 >> 172.21.15.18:0/4008418134 conn(0x7f51a40fb610 msgr2=0x7f51a40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:50.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 -- 172.21.15.18:0/4008418134 shutdown_connections 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 -- 172.21.15.18:0/4008418134 wait complete. 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 Processor -- start 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 -- start start 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51a4100080 0x7f51a4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51a4101270 0x7f51a41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51a4101c30 0x7f51a419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a419ed80 con 0x7f51a4101270 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a419eef0 con 0x7f51a4100080 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.858+0000 7f51a8e39640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a419f0b0 con 0x7f51a4101c30 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f519bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51a4101270 0x7f51a41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f519bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51a4101270 0x7f51a41977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39138/0 (socket says 172.21.15.18:39138) 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f519bfff640 1 -- 172.21.15.18:0/3411581213 learned_addr learned my addr 172.21.15.18:0/3411581213 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f51a2d9d640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51a4101c30 0x7f51a419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f51a259c640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51a4100080 0x7f51a4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f519bfff640 1 -- 172.21.15.18:0/3411581213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51a4101c30 msgr2=0x7f51a419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f519bfff640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51a4101c30 0x7f51a419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f519bfff640 1 -- 172.21.15.18:0/3411581213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51a4100080 msgr2=0x7f51a4197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f519bfff640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51a4100080 0x7f51a4197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f519bfff640 1 -- 172.21.15.18:0/3411581213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194009560 con 0x7f51a4101270 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f51a259c640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51a4100080 0x7f51a4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f519bfff640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51a4101270 0x7f51a41977d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f518c00d770 tx=0x7f518c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f519b7fe640 1 -- 172.21.15.18:0/3411581213 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c009c30 con 0x7f51a4101270 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f519b7fe640 1 -- 172.21.15.18:0/3411581213 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f518c00de80 con 0x7f51a4101270 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f51a8e39640 1 -- 172.21.15.18:0/3411581213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a419f390 con 0x7f51a4101270 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f519b7fe640 1 -- 172.21.15.18:0/3411581213 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c004e80 con 0x7f51a4101270 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.859+0000 7f51a8e39640 1 -- 172.21.15.18:0/3411581213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a419f8a0 con 0x7f51a4101270 2024-02-17T12:47:50.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.860+0000 7f51a8e39640 1 -- 172.21.15.18:0/3411581213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a41074c0 con 0x7f51a4101270 2024-02-17T12:47:50.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.861+0000 7f519b7fe640 1 -- 172.21.15.18:0/3411581213 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f518c005020 con 0x7f51a4101270 2024-02-17T12:47:50.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.862+0000 7f519b7fe640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f51900a2140 0x7f51900a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:50.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.862+0000 7f51a259c640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f51900a2140 0x7f51900a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:50.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.862+0000 7f519b7fe640 1 -- 172.21.15.18:0/3411581213 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f518c0cd600 con 0x7f51a4101270 2024-02-17T12:47:50.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.862+0000 7f51a259c640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f51900a2140 0x7f51900a4600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5194006010 tx=0x7f5194005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:50.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:50.866+0000 7f519b7fe640 1 -- 172.21.15.18:0/3411581213 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f518c096720 con 0x7f51a4101270 2024-02-17T12:47:51.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.077+0000 7f51a8e39640 1 -- 172.21.15.18:0/3411581213 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f51a402ce40 con 0x7f51900a2140 2024-02-17T12:47:51.075 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 12:47:50 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b[67310]: 2024-02-17T12:47:50.617+0000 7f38c7bbc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-17T12:47:51.075 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 12:47:50 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b[67310]: 2024-02-17T12:47:50.617+0000 7f38c7bbc700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-02-17T12:47:51.075 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 12:47:50 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b[67310]: 2024-02-17T12:47:50.617+0000 7f38c7bbc700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-02-17T12:47:51.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.100+0000 7f519b7fe640 1 -- 172.21.15.18:0/3411581213 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f51a402ce40 con 0x7f51900a2140 2024-02-17T12:47:51.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.102+0000 7f51a8e39640 1 -- 172.21.15.18:0/3411581213 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f51900a2140 msgr2=0x7f51900a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.102+0000 7f51a8e39640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f51900a2140 0x7f51900a4600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5194006010 tx=0x7f5194005be0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.102+0000 7f51a8e39640 1 -- 172.21.15.18:0/3411581213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51a4101270 msgr2=0x7f51a41977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.102+0000 7f51a8e39640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51a4101270 0x7f51a41977d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f518c00d770 tx=0x7f518c00dc30 comp rx=0 tx=0).stop 2024-02-17T12:47:51.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.102+0000 7f51a8e39640 1 -- 172.21.15.18:0/3411581213 shutdown_connections 2024-02-17T12:47:51.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.102+0000 7f51a8e39640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f51900a2140 0x7f51900a4600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.102+0000 7f51a8e39640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51a4101c30 0x7f51a419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.102+0000 7f51a8e39640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51a4101270 0x7f51a41977d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.102+0000 7f51a8e39640 1 --2- 172.21.15.18:0/3411581213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51a4100080 0x7f51a4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.102+0000 7f51a8e39640 1 -- 172.21.15.18:0/3411581213 >> 172.21.15.18:0/3411581213 conn(0x7f51a40fb610 msgr2=0x7f51a4104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:51.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.102+0000 7f51a8e39640 1 -- 172.21.15.18:0/3411581213 shutdown_connections 2024-02-17T12:47:51.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.102+0000 7f51a8e39640 1 -- 172.21.15.18:0/3411581213 wait complete. 2024-02-17T12:47:51.106 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-02-17T12:47:51.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.220+0000 7f5f3296c640 1 -- 172.21.15.18:0/3822116867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f2c104540 msgr2=0x7f5f2c106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.220+0000 7f5f3296c640 1 --2- 172.21.15.18:0/3822116867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f2c104540 0x7f5f2c106930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009900 tx=0x7f5f2002bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:51.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.220+0000 7f5f3296c640 1 -- 172.21.15.18:0/3822116867 shutdown_connections 2024-02-17T12:47:51.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.220+0000 7f5f3296c640 1 --2- 172.21.15.18:0/3822116867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f2c106e70 0x7f5f2c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.220+0000 7f5f3296c640 1 --2- 172.21.15.18:0/3822116867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f2c104540 0x7f5f2c106930 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.220+0000 7f5f3296c640 1 --2- 172.21.15.18:0/3822116867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f2c101c10 0x7f5f2c104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.220+0000 7f5f3296c640 1 -- 172.21.15.18:0/3822116867 >> 172.21.15.18:0/3822116867 conn(0x7f5f2c0fb620 msgr2=0x7f5f2c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:51.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.220+0000 7f5f3296c640 1 -- 172.21.15.18:0/3822116867 shutdown_connections 2024-02-17T12:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.220+0000 7f5f3296c640 1 -- 172.21.15.18:0/3822116867 wait complete. 2024-02-17T12:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.220+0000 7f5f3296c640 1 Processor -- start 2024-02-17T12:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.220+0000 7f5f3296c640 1 -- start start 2024-02-17T12:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f3296c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f2c101c10 0x7f5f2c101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f3296c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f2c104540 0x7f5f2c0ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f3296c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f2c106e70 0x7f5f2c100100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f3296c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c100860 con 0x7f5f2c101c10 2024-02-17T12:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f3296c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c1009d0 con 0x7f5f2c104540 2024-02-17T12:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f3296c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c1a1310 con 0x7f5f2c106e70 2024-02-17T12:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f2bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f2c101c10 0x7f5f2c101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f2bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f2c101c10 0x7f5f2c101580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39154/0 (socket says 172.21.15.18:39154) 2024-02-17T12:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f2bfff640 1 -- 172.21.15.18:0/2083860238 learned_addr learned my addr 172.21.15.18:0/2083860238 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f30f09640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f2c106e70 0x7f5f2c100100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f2b7fe640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f2c104540 0x7f5f2c0ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f2bfff640 1 -- 172.21.15.18:0/2083860238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f2c106e70 msgr2=0x7f5f2c100100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f2bfff640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f2c106e70 0x7f5f2c100100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f2bfff640 1 -- 172.21.15.18:0/2083860238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f2c104540 msgr2=0x7f5f2c0ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f2bfff640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f2c104540 0x7f5f2c0ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f2bfff640 1 -- 172.21.15.18:0/2083860238 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20009560 con 0x7f5f2c101c10 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f2b7fe640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f2c104540 0x7f5f2c0ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f2bfff640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f2c101c10 0x7f5f2c101580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00d7a0 tx=0x7f5f1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f297fa640 1 -- 172.21.15.18:0/2083860238 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c009c30 con 0x7f5f2c101c10 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f297fa640 1 -- 172.21.15.18:0/2083860238 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f1c00deb0 con 0x7f5f2c101c10 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f3296c640 1 -- 172.21.15.18:0/2083860238 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c1a15c0 con 0x7f5f2c101c10 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.221+0000 7f5f297fa640 1 -- 172.21.15.18:0/2083860238 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c004ea0 con 0x7f5f2c101c10 2024-02-17T12:47:51.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.222+0000 7f5f3296c640 1 -- 172.21.15.18:0/2083860238 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c1a1a50 con 0x7f5f2c101c10 2024-02-17T12:47:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.222+0000 7f5f3296c640 1 -- 172.21.15.18:0/2083860238 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c0fd110 con 0x7f5f2c101c10 2024-02-17T12:47:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.223+0000 7f5f297fa640 1 -- 172.21.15.18:0/2083860238 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5f1c005020 con 0x7f5f2c101c10 2024-02-17T12:47:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.224+0000 7f5f297fa640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5f140a20c0 0x7f5f140a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.224+0000 7f5f2b7fe640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5f140a20c0 0x7f5f140a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:51.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.224+0000 7f5f297fa640 1 -- 172.21.15.18:0/2083860238 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5f1c0cd590 con 0x7f5f2c101c10 2024-02-17T12:47:51.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.224+0000 7f5f2b7fe640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5f140a20c0 0x7f5f140a4580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5f20006010 tx=0x7f5f20005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:51.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.228+0000 7f5f297fa640 1 -- 172.21.15.18:0/2083860238 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f1c0966b0 con 0x7f5f2c101c10 2024-02-17T12:47:51.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:51 smithi119 ceph-mon[36188]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:51.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:51 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] up:active 2024-02-17T12:47:51.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:51 smithi119 ceph-mon[36188]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-02-17T12:47:51.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:51 smithi119 ceph-mon[36188]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-17T12:47:51.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:51 smithi119 ceph-mon[36188]: Cluster is now healthy 2024-02-17T12:47:51.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:51 smithi119 ceph-mon[36188]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:active} 2 up:standby-replay 6 up:standby 2024-02-17T12:47:51.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:51 smithi119 ceph-mon[36188]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-02-17T12:47:51.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:51 smithi119 ceph-mon[36188]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-02-17T12:47:51.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.439+0000 7f5f3296c640 1 -- 172.21.15.18:0/2083860238 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f2c02ceb0 con 0x7f5f140a20c0 2024-02-17T12:47:51.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.443+0000 7f5f297fa640 1 -- 172.21.15.18:0/2083860238 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f5f2c02ceb0 con 0x7f5f140a20c0 2024-02-17T12:47:51.436 INFO:teuthology.orchestra.run.smithi018.stdout:/volumes/qa/sv_0/bd5a66fe-15be-45c4-9b54-499e3dbf0e9f 2024-02-17T12:47:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.445+0000 7f5f3296c640 1 -- 172.21.15.18:0/2083860238 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5f140a20c0 msgr2=0x7f5f140a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.445+0000 7f5f3296c640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5f140a20c0 0x7f5f140a4580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5f20006010 tx=0x7f5f20005be0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.445+0000 7f5f3296c640 1 -- 172.21.15.18:0/2083860238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f2c101c10 msgr2=0x7f5f2c101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.445+0000 7f5f3296c640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f2c101c10 0x7f5f2c101580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00d7a0 tx=0x7f5f1c00dc60 comp rx=0 tx=0).stop 2024-02-17T12:47:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.445+0000 7f5f3296c640 1 -- 172.21.15.18:0/2083860238 shutdown_connections 2024-02-17T12:47:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.445+0000 7f5f3296c640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5f140a20c0 0x7f5f140a4580 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.445+0000 7f5f3296c640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f2c106e70 0x7f5f2c100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.445+0000 7f5f3296c640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f2c104540 0x7f5f2c0ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.445+0000 7f5f3296c640 1 --2- 172.21.15.18:0/2083860238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f2c101c10 0x7f5f2c101580 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.445+0000 7f5f3296c640 1 -- 172.21.15.18:0/2083860238 >> 172.21.15.18:0/2083860238 conn(0x7f5f2c0fb620 msgr2=0x7f5f2c1026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.445+0000 7f5f3296c640 1 -- 172.21.15.18:0/2083860238 shutdown_connections 2024-02-17T12:47:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.445+0000 7f5f3296c640 1 -- 172.21.15.18:0/2083860238 wait complete. 2024-02-17T12:47:51.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:51 smithi018 ceph-mon[32625]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:51.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:51 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] up:active 2024-02-17T12:47:51.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:51 smithi018 ceph-mon[32625]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-02-17T12:47:51.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:51 smithi018 ceph-mon[32625]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-17T12:47:51.449 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:51 smithi018 ceph-mon[32625]: Cluster is now healthy 2024-02-17T12:47:51.449 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:51 smithi018 ceph-mon[32625]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:active} 2 up:standby-replay 6 up:standby 2024-02-17T12:47:51.449 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:51 smithi018 ceph-mon[32625]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-02-17T12:47:51.449 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:51 smithi018 ceph-mon[32625]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-02-17T12:47:51.450 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-02-17T12:47:51.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:51 smithi146 ceph-mon[36454]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:51.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:51 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] up:active 2024-02-17T12:47:51.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:51 smithi146 ceph-mon[36454]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-02-17T12:47:51.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:51 smithi146 ceph-mon[36454]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-17T12:47:51.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:51 smithi146 ceph-mon[36454]: Cluster is now healthy 2024-02-17T12:47:51.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:51 smithi146 ceph-mon[36454]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:active} 2 up:standby-replay 6 up:standby 2024-02-17T12:47:51.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:51 smithi146 ceph-mon[36454]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-02-17T12:47:51.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:51 smithi146 ceph-mon[36454]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-02-17T12:47:51.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.564+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3753180920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d84102d40 msgr2=0x7f2d841095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.564+0000 7f2d8c41c640 1 --2- 172.21.15.18:0/3753180920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d84102d40 0x7f2d841095d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009ca0 tx=0x7f2d7402c100 comp rx=0 tx=0).stop 2024-02-17T12:47:51.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.564+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3753180920 shutdown_connections 2024-02-17T12:47:51.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.564+0000 7f2d8c41c640 1 --2- 172.21.15.18:0/3753180920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d84102d40 0x7f2d841095d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.564+0000 7f2d8c41c640 1 --2- 172.21.15.18:0/3753180920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d84102380 0x7f2d84102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.564+0000 7f2d8c41c640 1 --2- 172.21.15.18:0/3753180920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d84101190 0x7f2d841015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.564+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3753180920 >> 172.21.15.18:0/3753180920 conn(0x7f2d840fc720 msgr2=0x7f2d840feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.564+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3753180920 shutdown_connections 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.564+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3753180920 wait complete. 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.564+0000 7f2d8c41c640 1 Processor -- start 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8c41c640 1 -- start start 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8c41c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d84101190 0x7f2d84197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8c41c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d84102380 0x7f2d841975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8c41c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d84102d40 0x7f2d8419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8c41c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8419eb80 con 0x7f2d84102d40 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8c41c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8419ecf0 con 0x7f2d84102380 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8c41c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8419eeb0 con 0x7f2d84101190 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8a9b9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d84102d40 0x7f2d8419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8a9b9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d84102d40 0x7f2d8419e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39176/0 (socket says 172.21.15.18:39176) 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8a9b9640 1 -- 172.21.15.18:0/3868993696 learned_addr learned my addr 172.21.15.18:0/3868993696 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8a1b8640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d84101190 0x7f2d84197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d899b7640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d84102380 0x7f2d841975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8a9b9640 1 -- 172.21.15.18:0/3868993696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d84101190 msgr2=0x7f2d84197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8a9b9640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d84101190 0x7f2d84197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8a9b9640 1 -- 172.21.15.18:0/3868993696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d84102380 msgr2=0x7f2d841975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8a9b9640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d84102380 0x7f2d841975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8a9b9640 1 -- 172.21.15.18:0/3868993696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d74009900 con 0x7f2d84102d40 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8a1b8640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d84101190 0x7f2d84197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8a9b9640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d84102d40 0x7f2d8419e640 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d74037040 tx=0x7f2d74034e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d837fe640 1 -- 172.21.15.18:0/3868993696 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74003970 con 0x7f2d84102d40 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d837fe640 1 -- 172.21.15.18:0/3868993696 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d74004040 con 0x7f2d84102d40 2024-02-17T12:47:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3868993696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8419f130 con 0x7f2d84102d40 2024-02-17T12:47:51.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.565+0000 7f2d837fe640 1 -- 172.21.15.18:0/3868993696 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d740406b0 con 0x7f2d84102d40 2024-02-17T12:47:51.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.566+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3868993696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8419f630 con 0x7f2d84102d40 2024-02-17T12:47:51.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.567+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3868993696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d841083e0 con 0x7f2d84102d40 2024-02-17T12:47:51.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.568+0000 7f2d837fe640 1 -- 172.21.15.18:0/3868993696 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2d7400a040 con 0x7f2d84102d40 2024-02-17T12:47:51.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.568+0000 7f2d837fe640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2d6c0a2140 0x7f2d6c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:51.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.568+0000 7f2d8a1b8640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2d6c0a2140 0x7f2d6c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:51.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.568+0000 7f2d837fe640 1 -- 172.21.15.18:0/3868993696 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2d740f08f0 con 0x7f2d84102d40 2024-02-17T12:47:51.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.569+0000 7f2d8a1b8640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2d6c0a2140 0x7f2d6c0a4600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2d841021e0 tx=0x7f2d7c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:51.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.572+0000 7f2d837fe640 1 -- 172.21.15.18:0/3868993696 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d740b9a10 con 0x7f2d84102d40 2024-02-17T12:47:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.788+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3868993696 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d8402ceb0 con 0x7f2d6c0a2140 2024-02-17T12:47:51.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.811+0000 7f2d837fe640 1 -- 172.21.15.18:0/3868993696 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2d8402ceb0 con 0x7f2d6c0a2140 2024-02-17T12:47:51.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.813+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3868993696 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2d6c0a2140 msgr2=0x7f2d6c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.813+0000 7f2d8c41c640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2d6c0a2140 0x7f2d6c0a4600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2d841021e0 tx=0x7f2d7c005db0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.813+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3868993696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d84102d40 msgr2=0x7f2d8419e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.813+0000 7f2d8c41c640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d84102d40 0x7f2d8419e640 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d74037040 tx=0x7f2d74034e20 comp rx=0 tx=0).stop 2024-02-17T12:47:51.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.813+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3868993696 shutdown_connections 2024-02-17T12:47:51.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.813+0000 7f2d8c41c640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2d6c0a2140 0x7f2d6c0a4600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.813+0000 7f2d8c41c640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d84102d40 0x7f2d8419e640 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.813+0000 7f2d8c41c640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d84102380 0x7f2d841975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.813+0000 7f2d8c41c640 1 --2- 172.21.15.18:0/3868993696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d84101190 0x7f2d84197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.813+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3868993696 >> 172.21.15.18:0/3868993696 conn(0x7f2d840fc720 msgr2=0x7f2d841055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:51.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.813+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3868993696 shutdown_connections 2024-02-17T12:47:51.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.813+0000 7f2d8c41c640 1 -- 172.21.15.18:0/3868993696 wait complete. 2024-02-17T12:47:51.818 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-02-17T12:47:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.935+0000 7fcaea7c9640 1 -- 172.21.15.18:0/3871574936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcae4101190 msgr2=0x7fcae41015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.935+0000 7fcaea7c9640 1 --2- 172.21.15.18:0/3871574936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcae4101190 0x7fcae41015b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009900 tx=0x7fcad402bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.935+0000 7fcaea7c9640 1 -- 172.21.15.18:0/3871574936 shutdown_connections 2024-02-17T12:47:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.935+0000 7fcaea7c9640 1 --2- 172.21.15.18:0/3871574936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcae4102d40 0x7fcae41095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.935+0000 7fcaea7c9640 1 --2- 172.21.15.18:0/3871574936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae4102380 0x7fcae4102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.935+0000 7fcaea7c9640 1 --2- 172.21.15.18:0/3871574936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcae4101190 0x7fcae41015b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.935+0000 7fcaea7c9640 1 -- 172.21.15.18:0/3871574936 >> 172.21.15.18:0/3871574936 conn(0x7fcae40fc720 msgr2=0x7fcae40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.935+0000 7fcaea7c9640 1 -- 172.21.15.18:0/3871574936 shutdown_connections 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.935+0000 7fcaea7c9640 1 -- 172.21.15.18:0/3871574936 wait complete. 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcaea7c9640 1 Processor -- start 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcaea7c9640 1 -- start start 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcaea7c9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcae4101190 0x7fcae41971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcaea7c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcae4102380 0x7fcae4197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcaea7c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae4102d40 0x7fcae419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcaea7c9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae419ecb0 con 0x7fcae4101190 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcaea7c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae419ee20 con 0x7fcae4102d40 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcaea7c9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae419efe0 con 0x7fcae4102380 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcae3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcae4101190 0x7fcae41971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcae3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcae4101190 0x7fcae41971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39196/0 (socket says 172.21.15.18:39196) 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcae3fff640 1 -- 172.21.15.18:0/1255022339 learned_addr learned my addr 172.21.15.18:0/1255022339 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcae37fe640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcae4102380 0x7fcae4197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcae3fff640 1 -- 172.21.15.18:0/1255022339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcae4102380 msgr2=0x7fcae4197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcae8d66640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae4102d40 0x7fcae419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcae3fff640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcae4102380 0x7fcae4197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcae3fff640 1 -- 172.21.15.18:0/1255022339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae4102d40 msgr2=0x7fcae419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcae3fff640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae4102d40 0x7fcae419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcae3fff640 1 -- 172.21.15.18:0/1255022339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad4009560 con 0x7fcae4101190 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.936+0000 7fcae37fe640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcae4102380 0x7fcae4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.937+0000 7fcae3fff640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcae4101190 0x7fcae41971c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcad4000c00 tx=0x7fcad4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.937+0000 7fcae17fa640 1 -- 172.21.15.18:0/1255022339 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4004df0 con 0x7fcae4101190 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.937+0000 7fcaea7c9640 1 -- 172.21.15.18:0/1255022339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae419f260 con 0x7fcae4101190 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.937+0000 7fcae17fa640 1 -- 172.21.15.18:0/1255022339 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcad4005430 con 0x7fcae4101190 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.937+0000 7fcae17fa640 1 -- 172.21.15.18:0/1255022339 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad403e450 con 0x7fcae4101190 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.937+0000 7fcaea7c9640 1 -- 172.21.15.18:0/1255022339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae419f770 con 0x7fcae4101190 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.938+0000 7fcaea7c9640 1 -- 172.21.15.18:0/1255022339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae4108500 con 0x7fcae4101190 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.939+0000 7fcae17fa640 1 -- 172.21.15.18:0/1255022339 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcad4003680 con 0x7fcae4101190 2024-02-17T12:47:51.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.939+0000 7fcae17fa640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcacc0a20c0 0x7fcacc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:51.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.939+0000 7fcae37fe640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcacc0a20c0 0x7fcacc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:51.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.939+0000 7fcae17fa640 1 -- 172.21.15.18:0/1255022339 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcad40f00f0 con 0x7fcae4101190 2024-02-17T12:47:51.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.940+0000 7fcae37fe640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcacc0a20c0 0x7fcacc0a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcae41986d0 tx=0x7fcad800a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:51.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:51.944+0000 7fcae17fa640 1 -- 172.21.15.18:0/1255022339 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcad40b9210 con 0x7fcae4101190 2024-02-17T12:47:52.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.155+0000 7fcaea7c9640 1 -- 172.21.15.18:0/1255022339 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcae402ceb0 con 0x7fcacc0a20c0 2024-02-17T12:47:52.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.159+0000 7fcae17fa640 1 -- 172.21.15.18:0/1255022339 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fcae402ceb0 con 0x7fcacc0a20c0 2024-02-17T12:47:52.152 INFO:teuthology.orchestra.run.smithi018.stdout:/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df 2024-02-17T12:47:52.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.161+0000 7fcaea7c9640 1 -- 172.21.15.18:0/1255022339 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcacc0a20c0 msgr2=0x7fcacc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.161+0000 7fcaea7c9640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcacc0a20c0 0x7fcacc0a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcae41986d0 tx=0x7fcad800a5c0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.161+0000 7fcaea7c9640 1 -- 172.21.15.18:0/1255022339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcae4101190 msgr2=0x7fcae41971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.161+0000 7fcaea7c9640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcae4101190 0x7fcae41971c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcad4000c00 tx=0x7fcad4004270 comp rx=0 tx=0).stop 2024-02-17T12:47:52.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.162+0000 7fcaea7c9640 1 -- 172.21.15.18:0/1255022339 shutdown_connections 2024-02-17T12:47:52.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.162+0000 7fcaea7c9640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcacc0a20c0 0x7fcacc0a4580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.162+0000 7fcaea7c9640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcae4102d40 0x7fcae419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.162+0000 7fcaea7c9640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcae4102380 0x7fcae4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.162+0000 7fcaea7c9640 1 --2- 172.21.15.18:0/1255022339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcae4101190 0x7fcae41971c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.162+0000 7fcaea7c9640 1 -- 172.21.15.18:0/1255022339 >> 172.21.15.18:0/1255022339 conn(0x7fcae40fc720 msgr2=0x7fcae41056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:52.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.162+0000 7fcaea7c9640 1 -- 172.21.15.18:0/1255022339 shutdown_connections 2024-02-17T12:47:52.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.162+0000 7fcaea7c9640 1 -- 172.21.15.18:0/1255022339 wait complete. 2024-02-17T12:47:52.165 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:47:52.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.275+0000 7f164f0aa640 1 -- 172.21.15.18:0/1943312137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1648101ca0 msgr2=0x7f1648104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.275+0000 7f164f0aa640 1 --2- 172.21.15.18:0/1943312137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1648101ca0 0x7f1648104090 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1630009900 tx=0x7f163002bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:52.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.275+0000 7f164f0aa640 1 -- 172.21.15.18:0/1943312137 shutdown_connections 2024-02-17T12:47:52.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.275+0000 7f164f0aa640 1 --2- 172.21.15.18:0/1943312137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1648106f00 0x7f1648109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.275+0000 7f164f0aa640 1 --2- 172.21.15.18:0/1943312137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16481045d0 0x7f16481069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.275+0000 7f164f0aa640 1 --2- 172.21.15.18:0/1943312137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1648101ca0 0x7f1648104090 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.275+0000 7f164f0aa640 1 -- 172.21.15.18:0/1943312137 >> 172.21.15.18:0/1943312137 conn(0x7f16480fb6b0 msgr2=0x7f16480fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.275+0000 7f164f0aa640 1 -- 172.21.15.18:0/1943312137 shutdown_connections 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.275+0000 7f164f0aa640 1 -- 172.21.15.18:0/1943312137 wait complete. 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.275+0000 7f164f0aa640 1 Processor -- start 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.275+0000 7f164f0aa640 1 -- start start 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164f0aa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1648101ca0 0x7f16481972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164f0aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16481045d0 0x7f1648197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164f0aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1648106f00 0x7f164819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164f0aa640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819edc0 con 0x7f1648106f00 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164f0aa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819ef30 con 0x7f16481045d0 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164f0aa640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819f0f0 con 0x7f1648101ca0 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164d647640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1648106f00 0x7f164819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:52.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164d647640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1648106f00 0x7f164819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39220/0 (socket says 172.21.15.18:39220) 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164d647640 1 -- 172.21.15.18:0/293319611 learned_addr learned my addr 172.21.15.18:0/293319611 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f1647fff640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16481045d0 0x7f1648197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164ce46640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1648101ca0 0x7f16481972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164d647640 1 -- 172.21.15.18:0/293319611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1648101ca0 msgr2=0x7f16481972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164d647640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1648101ca0 0x7f16481972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164d647640 1 -- 172.21.15.18:0/293319611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16481045d0 msgr2=0x7f1648197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164d647640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16481045d0 0x7f1648197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164d647640 1 -- 172.21.15.18:0/293319611 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1630009560 con 0x7f1648106f00 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f1647fff640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16481045d0 0x7f1648197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164ce46640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1648101ca0 0x7f16481972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164d647640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1648106f00 0x7f164819e880 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f164000eaa0 tx=0x7f164000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f1645ffb640 1 -- 172.21.15.18:0/293319611 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164000cdb0 con 0x7f1648106f00 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f164f0aa640 1 -- 172.21.15.18:0/293319611 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164819f3d0 con 0x7f1648106f00 2024-02-17T12:47:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f1645ffb640 1 -- 172.21.15.18:0/293319611 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1640005020 con 0x7f1648106f00 2024-02-17T12:47:52.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.276+0000 7f1645ffb640 1 -- 172.21.15.18:0/293319611 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16400116b0 con 0x7f1648106f00 2024-02-17T12:47:52.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.277+0000 7f164f0aa640 1 -- 172.21.15.18:0/293319611 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164819f910 con 0x7f1648106f00 2024-02-17T12:47:52.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.278+0000 7f164f0aa640 1 -- 172.21.15.18:0/293319611 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16480fd1a0 con 0x7f1648106f00 2024-02-17T12:47:52.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.278+0000 7f1645ffb640 1 -- 172.21.15.18:0/293319611 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1640005520 con 0x7f1648106f00 2024-02-17T12:47:52.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.279+0000 7f1645ffb640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f16340a20c0 0x7f16340a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:52.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.279+0000 7f164ce46640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f16340a20c0 0x7f16340a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:52.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.279+0000 7f1645ffb640 1 -- 172.21.15.18:0/293319611 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f16400cde90 con 0x7f1648106f00 2024-02-17T12:47:52.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.280+0000 7f164ce46640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f16340a20c0 0x7f16340a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1630036040 tx=0x7f1630000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:52.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.284+0000 7f1645ffb640 1 -- 172.21.15.18:0/293319611 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1640096f90 con 0x7f1648106f00 2024-02-17T12:47:52.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:52 smithi119 ceph-mon[36188]: pgmap v286: 97 pgs: 71 active+clean, 26 creating+peering; 580 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-02-17T12:47:52.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:52 smithi119 ceph-mon[36188]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:52.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:52 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] up:active 2024-02-17T12:47:52.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:52 smithi119 ceph-mon[36188]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-02-17T12:47:52.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:52 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:47:52.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:52 smithi146 ceph-mon[36454]: pgmap v286: 97 pgs: 71 active+clean, 26 creating+peering; 580 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-02-17T12:47:52.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:52 smithi146 ceph-mon[36454]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:52.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:52 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] up:active 2024-02-17T12:47:52.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:52 smithi146 ceph-mon[36454]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-02-17T12:47:52.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:52 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:47:52.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.503+0000 7f164f0aa640 1 -- 172.21.15.18:0/293319611 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f164802ce40 con 0x7f1648106f00 2024-02-17T12:47:52.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.504+0000 7f1645ffb640 1 -- 172.21.15.18:0/293319611 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11392 (secure 0 0 0) 0x7f164001f020 con 0x7f1648106f00 2024-02-17T12:47:52.497 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:52.497 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:47:51.719041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:47:52.497 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-02-17T12:47:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.506+0000 7f164f0aa640 1 -- 172.21.15.18:0/293319611 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f16340a20c0 msgr2=0x7f16340a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.506+0000 7f164f0aa640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f16340a20c0 0x7f16340a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1630036040 tx=0x7f1630000bc0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.506+0000 7f164f0aa640 1 -- 172.21.15.18:0/293319611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1648106f00 msgr2=0x7f164819e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.506+0000 7f164f0aa640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1648106f00 0x7f164819e880 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f164000eaa0 tx=0x7f164000ef60 comp rx=0 tx=0).stop 2024-02-17T12:47:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.506+0000 7f164f0aa640 1 -- 172.21.15.18:0/293319611 shutdown_connections 2024-02-17T12:47:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.506+0000 7f164f0aa640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f16340a20c0 0x7f16340a4580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.506+0000 7f164f0aa640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1648106f00 0x7f164819e880 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.506+0000 7f164f0aa640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16481045d0 0x7f1648197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.506+0000 7f164f0aa640 1 --2- 172.21.15.18:0/293319611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1648101ca0 0x7f16481972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.506+0000 7f164f0aa640 1 -- 172.21.15.18:0/293319611 >> 172.21.15.18:0/293319611 conn(0x7f16480fb6b0 msgr2=0x7f1648102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.506+0000 7f164f0aa640 1 -- 172.21.15.18:0/293319611 shutdown_connections 2024-02-17T12:47:52.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.506+0000 7f164f0aa640 1 -- 172.21.15.18:0/293319611 wait complete. 2024-02-17T12:47:52.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:52 smithi018 ceph-mon[32625]: pgmap v286: 97 pgs: 71 active+clean, 26 creating+peering; 580 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-02-17T12:47:52.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:52 smithi018 ceph-mon[32625]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:52.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:52 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] up:active 2024-02-17T12:47:52.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:52 smithi018 ceph-mon[32625]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-02-17T12:47:52.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:52 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:47:52.510 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:47:52.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.620+0000 7ff14722e640 1 -- 172.21.15.18:0/3071048330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff140104560 msgr2=0x7ff140106950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.620+0000 7ff14722e640 1 --2- 172.21.15.18:0/3071048330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff140104560 0x7ff140106950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009900 tx=0x7ff12c02bc50 comp rx=0 tx=0).stop 2024-02-17T12:47:52.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.620+0000 7ff14722e640 1 -- 172.21.15.18:0/3071048330 shutdown_connections 2024-02-17T12:47:52.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.620+0000 7ff14722e640 1 --2- 172.21.15.18:0/3071048330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff140106e90 0x7ff1401092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.620+0000 7ff14722e640 1 --2- 172.21.15.18:0/3071048330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff140104560 0x7ff140106950 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.620+0000 7ff14722e640 1 --2- 172.21.15.18:0/3071048330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff140101c30 0x7ff140104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.620+0000 7ff14722e640 1 -- 172.21.15.18:0/3071048330 >> 172.21.15.18:0/3071048330 conn(0x7ff1400fb660 msgr2=0x7ff1400fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:52.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.620+0000 7ff14722e640 1 -- 172.21.15.18:0/3071048330 shutdown_connections 2024-02-17T12:47:52.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.620+0000 7ff14722e640 1 -- 172.21.15.18:0/3071048330 wait complete. 2024-02-17T12:47:52.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.620+0000 7ff14722e640 1 Processor -- start 2024-02-17T12:47:52.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.620+0000 7ff14722e640 1 -- start start 2024-02-17T12:47:52.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff14722e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff140101c30 0x7ff140197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:52.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff14722e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff140104560 0x7ff140197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:52.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff14722e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff140106e90 0x7ff14019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:52.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff14722e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14019ec40 con 0x7ff140106e90 2024-02-17T12:47:52.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff14722e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14019edb0 con 0x7ff140101c30 2024-02-17T12:47:52.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff14722e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14019ef70 con 0x7ff140104560 2024-02-17T12:47:52.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff1457cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff140106e90 0x7ff14019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:52.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff1457cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff140106e90 0x7ff14019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39236/0 (socket says 172.21.15.18:39236) 2024-02-17T12:47:52.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff1457cb640 1 -- 172.21.15.18:0/2813451632 learned_addr learned my addr 172.21.15.18:0/2813451632 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:52.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff13ffff640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff140104560 0x7ff140197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:52.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff144fca640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff140101c30 0x7ff140197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:52.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff1457cb640 1 -- 172.21.15.18:0/2813451632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff140104560 msgr2=0x7ff140197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff1457cb640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff140104560 0x7ff140197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff1457cb640 1 -- 172.21.15.18:0/2813451632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff140101c30 msgr2=0x7ff140197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff1457cb640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff140101c30 0x7ff140197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff1457cb640 1 -- 172.21.15.18:0/2813451632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c009560 con 0x7ff140106e90 2024-02-17T12:47:52.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff1457cb640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff140106e90 0x7ff14019e700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff13800cce0 tx=0x7ff13800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:52.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff13dffb640 1 -- 172.21.15.18:0/2813451632 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13800f450 con 0x7ff140106e90 2024-02-17T12:47:52.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff13dffb640 1 -- 172.21.15.18:0/2813451632 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff13800fa90 con 0x7ff140106e90 2024-02-17T12:47:52.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff14722e640 1 -- 172.21.15.18:0/2813451632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14019f250 con 0x7ff140106e90 2024-02-17T12:47:52.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.621+0000 7ff13dffb640 1 -- 172.21.15.18:0/2813451632 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138018ae0 con 0x7ff140106e90 2024-02-17T12:47:52.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.622+0000 7ff14722e640 1 -- 172.21.15.18:0/2813451632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14019f730 con 0x7ff140106e90 2024-02-17T12:47:52.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.623+0000 7ff14722e640 1 -- 172.21.15.18:0/2813451632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1400fd150 con 0x7ff140106e90 2024-02-17T12:47:52.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.624+0000 7ff13dffb640 1 -- 172.21.15.18:0/2813451632 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff13800fc00 con 0x7ff140106e90 2024-02-17T12:47:52.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.624+0000 7ff13dffb640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff1300a2140 0x7ff1300a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:52.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.624+0000 7ff13dffb640 1 -- 172.21.15.18:0/2813451632 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff1380ce2f0 con 0x7ff140106e90 2024-02-17T12:47:52.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.624+0000 7ff144fca640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff1300a2140 0x7ff1300a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:52.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.625+0000 7ff144fca640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff1300a2140 0x7ff1300a4600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff13400abe0 tx=0x7ff13400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:52.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.628+0000 7ff13dffb640 1 -- 172.21.15.18:0/2813451632 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff138097490 con 0x7ff140106e90 2024-02-17T12:47:52.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.811+0000 7ff14722e640 1 -- 172.21.15.18:0/2813451632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff14010cb80 con 0x7ff140106e90 2024-02-17T12:47:52.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.813+0000 7ff13dffb640 1 -- 172.21.15.18:0/2813451632 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22221 (secure 0 0 0) 0x7ff13809ab50 con 0x7ff140106e90 2024-02-17T12:47:52.806 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:52.806 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":86,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:47:47.058106+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:47:52.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.815+0000 7ff14722e640 1 -- 172.21.15.18:0/2813451632 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff1300a2140 msgr2=0x7ff1300a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.815+0000 7ff14722e640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff1300a2140 0x7ff1300a4600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff13400abe0 tx=0x7ff13400a2d0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.815+0000 7ff14722e640 1 -- 172.21.15.18:0/2813451632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff140106e90 msgr2=0x7ff14019e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.815+0000 7ff14722e640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff140106e90 0x7ff14019e700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff13800cce0 tx=0x7ff13800a620 comp rx=0 tx=0).stop 2024-02-17T12:47:52.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.815+0000 7ff14722e640 1 -- 172.21.15.18:0/2813451632 shutdown_connections 2024-02-17T12:47:52.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.815+0000 7ff14722e640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff1300a2140 0x7ff1300a4600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.815+0000 7ff14722e640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff140106e90 0x7ff14019e700 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.815+0000 7ff14722e640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff140104560 0x7ff140197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.815+0000 7ff14722e640 1 --2- 172.21.15.18:0/2813451632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff140101c30 0x7ff140197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.815+0000 7ff14722e640 1 -- 172.21.15.18:0/2813451632 >> 172.21.15.18:0/2813451632 conn(0x7ff1400fb660 msgr2=0x7ff140102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:52.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.815+0000 7ff14722e640 1 -- 172.21.15.18:0/2813451632 shutdown_connections 2024-02-17T12:47:52.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.815+0000 7ff14722e640 1 -- 172.21.15.18:0/2813451632 wait complete. 2024-02-17T12:47:52.819 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-17T12:47:52.819 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-17T12:47:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.929+0000 7fadc77d7640 1 -- 172.21.15.18:0/2149449912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc01000c0 msgr2=0x7fadc01004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.929+0000 7fadc77d7640 1 --2- 172.21.15.18:0/2149449912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc01000c0 0x7fadc01004e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009900 tx=0x7fadbc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.929+0000 7fadc77d7640 1 -- 172.21.15.18:0/2149449912 shutdown_connections 2024-02-17T12:47:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.929+0000 7fadc77d7640 1 --2- 172.21.15.18:0/2149449912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc0101c70 0x7fadc0108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.929+0000 7fadc77d7640 1 --2- 172.21.15.18:0/2149449912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadc01012b0 0x7fadc0101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.929+0000 7fadc77d7640 1 --2- 172.21.15.18:0/2149449912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc01000c0 0x7fadc01004e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.929+0000 7fadc77d7640 1 -- 172.21.15.18:0/2149449912 >> 172.21.15.18:0/2149449912 conn(0x7fadc00fb630 msgr2=0x7fadc00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.929+0000 7fadc77d7640 1 -- 172.21.15.18:0/2149449912 shutdown_connections 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.929+0000 7fadc77d7640 1 -- 172.21.15.18:0/2149449912 wait complete. 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.929+0000 7fadc77d7640 1 Processor -- start 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.929+0000 7fadc77d7640 1 -- start start 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc77d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadc01000c0 0x7fadc0199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc77d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc01012b0 0x7fadc0199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc77d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc0101c70 0x7fadc01a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc77d7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc01a0f90 con 0x7fadc01012b0 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc77d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc01a1100 con 0x7fadc0101c70 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc77d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc01a12c0 con 0x7fadc01000c0 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc4d72640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc01012b0 0x7fadc0199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc4d72640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc01012b0 0x7fadc0199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39268/0 (socket says 172.21.15.18:39268) 2024-02-17T12:47:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc4d72640 1 -- 172.21.15.18:0/2168117879 learned_addr learned my addr 172.21.15.18:0/2168117879 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc5573640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadc01000c0 0x7fadc0199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc5d74640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc0101c70 0x7fadc01a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc4d72640 1 -- 172.21.15.18:0/2168117879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadc01000c0 msgr2=0x7fadc0199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc4d72640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadc01000c0 0x7fadc0199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc4d72640 1 -- 172.21.15.18:0/2168117879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc0101c70 msgr2=0x7fadc01a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc4d72640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc0101c70 0x7fadc01a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc4d72640 1 -- 172.21.15.18:0/2168117879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc009560 con 0x7fadc01012b0 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc5d74640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc0101c70 0x7fadc01a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc4d72640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc01012b0 0x7fadc0199a50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fadb000d770 tx=0x7fadb000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadb67fc640 1 -- 172.21.15.18:0/2168117879 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0009c30 con 0x7fadc01012b0 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadb67fc640 1 -- 172.21.15.18:0/2168117879 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadb000de80 con 0x7fadc01012b0 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadb67fc640 1 -- 172.21.15.18:0/2168117879 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0004e80 con 0x7fadc01012b0 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc77d7640 1 -- 172.21.15.18:0/2168117879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc01a15a0 con 0x7fadc01012b0 2024-02-17T12:47:52.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.930+0000 7fadc77d7640 1 -- 172.21.15.18:0/2168117879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc01a1ab0 con 0x7fadc01012b0 2024-02-17T12:47:52.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.931+0000 7fadc77d7640 1 -- 172.21.15.18:0/2168117879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc010bad0 con 0x7fadc01012b0 2024-02-17T12:47:52.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.932+0000 7fadb67fc640 1 -- 172.21.15.18:0/2168117879 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fadb0005020 con 0x7fadc01012b0 2024-02-17T12:47:52.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.933+0000 7fadb67fc640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fadac0a2140 0x7fadac0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:52.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.933+0000 7fadc5573640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fadac0a2140 0x7fadac0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:52.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.933+0000 7fadb67fc640 1 -- 172.21.15.18:0/2168117879 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fadb00cd550 con 0x7fadc01012b0 2024-02-17T12:47:52.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.933+0000 7fadc5573640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fadac0a2140 0x7fadac0a4600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fadbc005ab0 tx=0x7fadbc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:52.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:52.937+0000 7fadb67fc640 1 -- 172.21.15.18:0/2168117879 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadb0096670 con 0x7fadc01012b0 2024-02-17T12:47:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.097+0000 7fadb67fc640 1 -- 172.21.15.18:0/2168117879 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fadb0096a40 con 0x7fadc01012b0 2024-02-17T12:47:53.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.121+0000 7fadc77d7640 1 -- 172.21.15.18:0/2168117879 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fadc00fd810 con 0x7fadac0a2140 2024-02-17T12:47:53.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.132+0000 7fadb67fc640 1 -- 172.21.15.18:0/2168117879 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335034 (secure 0 0 0) 0x7fadc00fd810 con 0x7fadac0a2140 2024-02-17T12:47:53.129 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:53.131 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-02-17T12:47:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.135+0000 7fadc77d7640 1 -- 172.21.15.18:0/2168117879 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fadac0a2140 msgr2=0x7fadac0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.135+0000 7fadc77d7640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fadac0a2140 0x7fadac0a4600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fadbc005ab0 tx=0x7fadbc005fd0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.135+0000 7fadc77d7640 1 -- 172.21.15.18:0/2168117879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc01012b0 msgr2=0x7fadc0199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.135+0000 7fadc77d7640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc01012b0 0x7fadc0199a50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fadb000d770 tx=0x7fadb000dc30 comp rx=0 tx=0).stop 2024-02-17T12:47:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.135+0000 7fadc77d7640 1 -- 172.21.15.18:0/2168117879 shutdown_connections 2024-02-17T12:47:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.135+0000 7fadc77d7640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fadac0a2140 0x7fadac0a4600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.135+0000 7fadc77d7640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc0101c70 0x7fadc01a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.135+0000 7fadc77d7640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadc01012b0 0x7fadc0199a50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.135+0000 7fadc77d7640 1 --2- 172.21.15.18:0/2168117879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadc01000c0 0x7fadc0199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.135+0000 7fadc77d7640 1 -- 172.21.15.18:0/2168117879 >> 172.21.15.18:0/2168117879 conn(0x7fadc00fb630 msgr2=0x7fadc00fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.135+0000 7fadc77d7640 1 -- 172.21.15.18:0/2168117879 shutdown_connections 2024-02-17T12:47:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.135+0000 7fadc77d7640 1 -- 172.21.15.18:0/2168117879 wait complete. 2024-02-17T12:47:53.139 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-02-17T12:47:52.191838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595433,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":149,"num_read_kb":130,"num_write":312,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":251,"ondisk_log_size":251,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":233,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374488,"kb_used_data":12824,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567336,"statfs":{"total":1151940427776,"available":1151556952064,"internally_reserved":0,"allocated":13131776,"data_stored":7105251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5065,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":131,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.147072"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180316+0000","last_change":"2024-02-17T12:47:46.069941+0000","last_active":"2024-02-17T12:47:47.180316+0000","last_peered":"2024-02-17T12:47:47.180316+0000","last_clean":"2024-02-17T12:47:47.180316+0000","last_became_active":"2024-02-17T12:47:46.069192+0000","last_became_peered":"2024-02-17T12:47:46.069192+0000","last_unstale":"2024-02-17T12:47:47.180316+0000","last_undegraded":"2024-02-17T12:47:47.180316+0000","last_fullsized":"2024-02-17T12:47:47.180316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:27.102839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179745+0000","last_change":"2024-02-17T12:47:46.064872+0000","last_active":"2024-02-17T12:47:47.179745+0000","last_peered":"2024-02-17T12:47:47.179745+0000","last_clean":"2024-02-17T12:47:47.179745+0000","last_became_active":"2024-02-17T12:47:46.064498+0000","last_became_peered":"2024-02-17T12:47:46.064498+0000","last_unstale":"2024-02-17T12:47:47.179745+0000","last_undegraded":"2024-02-17T12:47:47.179745+0000","last_fullsized":"2024-02-17T12:47:47.179745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:30:41.432657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179528+0000","last_change":"2024-02-17T12:47:46.067504+0000","last_active":"2024-02-17T12:47:47.179528+0000","last_peered":"2024-02-17T12:47:47.179528+0000","last_clean":"2024-02-17T12:47:47.179528+0000","last_became_active":"2024-02-17T12:47:46.067155+0000","last_became_peered":"2024-02-17T12:47:46.067155+0000","last_unstale":"2024-02-17T12:47:47.179528+0000","last_undegraded":"2024-02-17T12:47:47.179528+0000","last_fullsized":"2024-02-17T12:47:47.179528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:38:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180329+0000","last_change":"2024-02-17T12:47:46.062356+0000","last_active":"2024-02-17T12:47:47.180329+0000","last_peered":"2024-02-17T12:47:47.180329+0000","last_clean":"2024-02-17T12:47:47.180329+0000","last_became_active":"2024-02-17T12:47:46.061988+0000","last_became_peered":"2024-02-17T12:47:46.061988+0000","last_unstale":"2024-02-17T12:47:47.180329+0000","last_undegraded":"2024-02-17T12:47:47.180329+0000","last_fullsized":"2024-02-17T12:47:47.180329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:25:09.120770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243368+0000","last_change":"2024-02-17T12:47:46.072411+0000","last_active":"2024-02-17T12:47:47.243368+0000","last_peered":"2024-02-17T12:47:47.243368+0000","last_clean":"2024-02-17T12:47:47.243368+0000","last_became_active":"2024-02-17T12:47:46.072067+0000","last_became_peered":"2024-02-17T12:47:46.072067+0000","last_unstale":"2024-02-17T12:47:47.243368+0000","last_undegraded":"2024-02-17T12:47:47.243368+0000","last_fullsized":"2024-02-17T12:47:47.243368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:29:33.051513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179899+0000","last_change":"2024-02-17T12:47:46.057425+0000","last_active":"2024-02-17T12:47:47.179899+0000","last_peered":"2024-02-17T12:47:47.179899+0000","last_clean":"2024-02-17T12:47:47.179899+0000","last_became_active":"2024-02-17T12:47:46.057001+0000","last_became_peered":"2024-02-17T12:47:46.057001+0000","last_unstale":"2024-02-17T12:47:47.179899+0000","last_undegraded":"2024-02-17T12:47:47.179899+0000","last_fullsized":"2024-02-17T12:47:47.179899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:48:24.135879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179726+0000","last_change":"2024-02-17T12:47:46.069143+0000","last_active":"2024-02-17T12:47:47.179726+0000","last_peered":"2024-02-17T12:47:47.179726+0000","last_clean":"2024-02-17T12:47:47.179726+0000","last_became_active":"2024-02-17T12:47:46.068775+0000","last_became_peered":"2024-02-17T12:47:46.068775+0000","last_unstale":"2024-02-17T12:47:47.179726+0000","last_undegraded":"2024-02-17T12:47:47.179726+0000","last_fullsized":"2024-02-17T12:47:47.179726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:32:39.518205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065050+0000","last_change":"2024-02-17T12:47:46.069579+0000","last_active":"2024-02-17T12:47:47.065050+0000","last_peered":"2024-02-17T12:47:47.065050+0000","last_clean":"2024-02-17T12:47:47.065050+0000","last_became_active":"2024-02-17T12:47:46.069114+0000","last_became_peered":"2024-02-17T12:47:46.069114+0000","last_unstale":"2024-02-17T12:47:47.065050+0000","last_undegraded":"2024-02-17T12:47:47.065050+0000","last_fullsized":"2024-02-17T12:47:47.065050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:54:58.100511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066245+0000","last_change":"2024-02-17T12:47:46.079103+0000","last_active":"2024-02-17T12:47:47.066245+0000","last_peered":"2024-02-17T12:47:47.066245+0000","last_clean":"2024-02-17T12:47:47.066245+0000","last_became_active":"2024-02-17T12:47:46.077992+0000","last_became_peered":"2024-02-17T12:47:46.077992+0000","last_unstale":"2024-02-17T12:47:47.066245+0000","last_undegraded":"2024-02-17T12:47:47.066245+0000","last_fullsized":"2024-02-17T12:47:47.066245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:12:59.337341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180026+0000","last_change":"2024-02-17T12:47:46.062897+0000","last_active":"2024-02-17T12:47:47.180026+0000","last_peered":"2024-02-17T12:47:47.180026+0000","last_clean":"2024-02-17T12:47:47.180026+0000","last_became_active":"2024-02-17T12:47:46.062571+0000","last_became_peered":"2024-02-17T12:47:46.062571+0000","last_unstale":"2024-02-17T12:47:47.180026+0000","last_undegraded":"2024-02-17T12:47:47.180026+0000","last_fullsized":"2024-02-17T12:47:47.180026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:57:02.845400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180144+0000","last_change":"2024-02-17T12:47:46.080183+0000","last_active":"2024-02-17T12:47:47.180144+0000","last_peered":"2024-02-17T12:47:47.180144+0000","last_clean":"2024-02-17T12:47:47.180144+0000","last_became_active":"2024-02-17T12:47:46.079827+0000","last_became_peered":"2024-02-17T12:47:46.079827+0000","last_unstale":"2024-02-17T12:47:47.180144+0000","last_undegraded":"2024-02-17T12:47:47.180144+0000","last_fullsized":"2024-02-17T12:47:47.180144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:56:10.193200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180650+0000","last_change":"2024-02-17T12:47:46.061849+0000","last_active":"2024-02-17T12:47:47.180650+0000","last_peered":"2024-02-17T12:47:47.180650+0000","last_clean":"2024-02-17T12:47:47.180650+0000","last_became_active":"2024-02-17T12:47:46.061458+0000","last_became_peered":"2024-02-17T12:47:46.061458+0000","last_unstale":"2024-02-17T12:47:47.180650+0000","last_undegraded":"2024-02-17T12:47:47.180650+0000","last_fullsized":"2024-02-17T12:47:47.180650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:02:09.141292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179320+0000","last_change":"2024-02-17T12:47:46.069356+0000","last_active":"2024-02-17T12:47:47.179320+0000","last_peered":"2024-02-17T12:47:47.179320+0000","last_clean":"2024-02-17T12:47:47.179320+0000","last_became_active":"2024-02-17T12:47:46.068567+0000","last_became_peered":"2024-02-17T12:47:46.068567+0000","last_unstale":"2024-02-17T12:47:47.179320+0000","last_undegraded":"2024-02-17T12:47:47.179320+0000","last_fullsized":"2024-02-17T12:47:47.179320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:08:01.872558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179478+0000","last_change":"2024-02-17T12:47:46.062684+0000","last_active":"2024-02-17T12:47:47.179478+0000","last_peered":"2024-02-17T12:47:47.179478+0000","last_clean":"2024-02-17T12:47:47.179478+0000","last_became_active":"2024-02-17T12:47:46.062221+0000","last_became_peered":"2024-02-17T12:47:46.062221+0000","last_unstale":"2024-02-17T12:47:47.179478+0000","last_undegraded":"2024-02-17T12:47:47.179478+0000","last_fullsized":"2024-02-17T12:47:47.179478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:37:11.513336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179310+0000","last_change":"2024-02-17T12:47:46.065961+0000","last_active":"2024-02-17T12:47:47.179310+0000","last_peered":"2024-02-17T12:47:47.179310+0000","last_clean":"2024-02-17T12:47:47.179310+0000","last_became_active":"2024-02-17T12:47:46.065600+0000","last_became_peered":"2024-02-17T12:47:46.065600+0000","last_unstale":"2024-02-17T12:47:47.179310+0000","last_undegraded":"2024-02-17T12:47:47.179310+0000","last_fullsized":"2024-02-17T12:47:47.179310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:24:46.205077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066101+0000","last_change":"2024-02-17T12:47:46.080150+0000","last_active":"2024-02-17T12:47:47.066101+0000","last_peered":"2024-02-17T12:47:47.066101+0000","last_clean":"2024-02-17T12:47:47.066101+0000","last_became_active":"2024-02-17T12:47:46.079813+0000","last_became_peered":"2024-02-17T12:47:46.079813+0000","last_unstale":"2024-02-17T12:47:47.066101+0000","last_undegraded":"2024-02-17T12:47:47.066101+0000","last_fullsized":"2024-02-17T12:47:47.066101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:59:00.868964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180476+0000","last_change":"2024-02-17T12:47:46.070202+0000","last_active":"2024-02-17T12:47:47.180476+0000","last_peered":"2024-02-17T12:47:47.180476+0000","last_clean":"2024-02-17T12:47:47.180476+0000","last_became_active":"2024-02-17T12:47:46.069628+0000","last_became_peered":"2024-02-17T12:47:46.069628+0000","last_unstale":"2024-02-17T12:47:47.180476+0000","last_undegraded":"2024-02-17T12:47:47.180476+0000","last_fullsized":"2024-02-17T12:47:47.180476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:44:17.481681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.150613+0000","last_change":"2024-02-17T12:47:46.067810+0000","last_active":"2024-02-17T12:47:47.150613+0000","last_peered":"2024-02-17T12:47:47.150613+0000","last_clean":"2024-02-17T12:47:47.150613+0000","last_became_active":"2024-02-17T12:47:46.067456+0000","last_became_peered":"2024-02-17T12:47:46.067456+0000","last_unstale":"2024-02-17T12:47:47.150613+0000","last_undegraded":"2024-02-17T12:47:47.150613+0000","last_fullsized":"2024-02-17T12:47:47.150613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:27:03.646597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179813+0000","last_change":"2024-02-17T12:47:46.076268+0000","last_active":"2024-02-17T12:47:47.179813+0000","last_peered":"2024-02-17T12:47:47.179813+0000","last_clean":"2024-02-17T12:47:47.179813+0000","last_became_active":"2024-02-17T12:47:46.075606+0000","last_became_peered":"2024-02-17T12:47:46.075606+0000","last_unstale":"2024-02-17T12:47:47.179813+0000","last_undegraded":"2024-02-17T12:47:47.179813+0000","last_fullsized":"2024-02-17T12:47:47.179813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:16:35.805008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:50.725688+0000","last_change":"2024-02-17T12:47:45.066780+0000","last_active":"2024-02-17T12:47:50.725688+0000","last_peered":"2024-02-17T12:47:50.725688+0000","last_clean":"2024-02-17T12:47:50.725688+0000","last_became_active":"2024-02-17T12:47:45.066039+0000","last_became_peered":"2024-02-17T12:47:45.066039+0000","last_unstale":"2024-02-17T12:47:50.725688+0000","last_undegraded":"2024-02-17T12:47:50.725688+0000","last_fullsized":"2024-02-17T12:47:50.725688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:36:25.420534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065470+0000","last_change":"2024-02-17T12:47:46.079603+0000","last_active":"2024-02-17T12:47:47.065470+0000","last_peered":"2024-02-17T12:47:47.065470+0000","last_clean":"2024-02-17T12:47:47.065470+0000","last_became_active":"2024-02-17T12:47:46.079246+0000","last_became_peered":"2024-02-17T12:47:46.079246+0000","last_unstale":"2024-02-17T12:47:47.065470+0000","last_undegraded":"2024-02-17T12:47:47.065470+0000","last_fullsized":"2024-02-17T12:47:47.065470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:32:04.975778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065654+0000","last_change":"2024-02-17T12:47:45.061868+0000","last_active":"2024-02-17T12:47:47.065654+0000","last_peered":"2024-02-17T12:47:47.065654+0000","last_clean":"2024-02-17T12:47:47.065654+0000","last_became_active":"2024-02-17T12:47:45.061512+0000","last_became_peered":"2024-02-17T12:47:45.061512+0000","last_unstale":"2024-02-17T12:47:47.065654+0000","last_undegraded":"2024-02-17T12:47:47.065654+0000","last_fullsized":"2024-02-17T12:47:47.065654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:53:48.405184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179664+0000","last_change":"2024-02-17T12:47:46.071055+0000","last_active":"2024-02-17T12:47:47.179664+0000","last_peered":"2024-02-17T12:47:47.179664+0000","last_clean":"2024-02-17T12:47:47.179664+0000","last_became_active":"2024-02-17T12:47:46.070649+0000","last_became_peered":"2024-02-17T12:47:46.070649+0000","last_unstale":"2024-02-17T12:47:47.179664+0000","last_undegraded":"2024-02-17T12:47:47.179664+0000","last_fullsized":"2024-02-17T12:47:47.179664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:10:22.078746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243694+0000","last_change":"2024-02-17T12:47:45.066292+0000","last_active":"2024-02-17T12:47:47.243694+0000","last_peered":"2024-02-17T12:47:47.243694+0000","last_clean":"2024-02-17T12:47:47.243694+0000","last_became_active":"2024-02-17T12:47:45.065839+0000","last_became_peered":"2024-02-17T12:47:45.065839+0000","last_unstale":"2024-02-17T12:47:47.243694+0000","last_undegraded":"2024-02-17T12:47:47.243694+0000","last_fullsized":"2024-02-17T12:47:47.243694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:12:57.076774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066291+0000","last_change":"2024-02-17T12:47:46.076448+0000","last_active":"2024-02-17T12:47:47.066291+0000","last_peered":"2024-02-17T12:47:47.066291+0000","last_clean":"2024-02-17T12:47:47.066291+0000","last_became_active":"2024-02-17T12:47:46.075250+0000","last_became_peered":"2024-02-17T12:47:46.075250+0000","last_unstale":"2024-02-17T12:47:47.066291+0000","last_undegraded":"2024-02-17T12:47:47.066291+0000","last_fullsized":"2024-02-17T12:47:47.066291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:11:45.846764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243557+0000","last_change":"2024-02-17T12:47:45.062699+0000","last_active":"2024-02-17T12:47:47.243557+0000","last_peered":"2024-02-17T12:47:47.243557+0000","last_clean":"2024-02-17T12:47:47.243557+0000","last_became_active":"2024-02-17T12:47:45.062322+0000","last_became_peered":"2024-02-17T12:47:45.062322+0000","last_unstale":"2024-02-17T12:47:47.243557+0000","last_undegraded":"2024-02-17T12:47:47.243557+0000","last_fullsized":"2024-02-17T12:47:47.243557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:14:06.112393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066193+0000","last_change":"2024-02-17T12:47:46.080278+0000","last_active":"2024-02-17T12:47:47.066193+0000","last_peered":"2024-02-17T12:47:47.066193+0000","last_clean":"2024-02-17T12:47:47.066193+0000","last_became_active":"2024-02-17T12:47:46.079924+0000","last_became_peered":"2024-02-17T12:47:46.079924+0000","last_unstale":"2024-02-17T12:47:47.066193+0000","last_undegraded":"2024-02-17T12:47:47.066193+0000","last_fullsized":"2024-02-17T12:47:47.066193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:54:08.759445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.705945+0000","last_change":"2024-02-17T12:47:45.058098+0000","last_active":"2024-02-17T12:47:48.705945+0000","last_peered":"2024-02-17T12:47:48.705945+0000","last_clean":"2024-02-17T12:47:48.705945+0000","last_became_active":"2024-02-17T12:47:45.057763+0000","last_became_peered":"2024-02-17T12:47:45.057763+0000","last_unstale":"2024-02-17T12:47:48.705945+0000","last_undegraded":"2024-02-17T12:47:48.705945+0000","last_fullsized":"2024-02-17T12:47:48.705945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:26:36.739723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243347+0000","last_change":"2024-02-17T12:47:46.065957+0000","last_active":"2024-02-17T12:47:47.243347+0000","last_peered":"2024-02-17T12:47:47.243347+0000","last_clean":"2024-02-17T12:47:47.243347+0000","last_became_active":"2024-02-17T12:47:46.065638+0000","last_became_peered":"2024-02-17T12:47:46.065638+0000","last_unstale":"2024-02-17T12:47:47.243347+0000","last_undegraded":"2024-02-17T12:47:47.243347+0000","last_fullsized":"2024-02-17T12:47:47.243347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:02:39.295990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.711703+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:47:48.711703+0000","last_peered":"2024-02-17T12:47:48.711703+0000","last_clean":"2024-02-17T12:47:48.711703+0000","last_became_active":"2024-02-17T12:47:45.066235+0000","last_became_peered":"2024-02-17T12:47:45.066235+0000","last_unstale":"2024-02-17T12:47:48.711703+0000","last_undegraded":"2024-02-17T12:47:48.711703+0000","last_fullsized":"2024-02-17T12:47:48.711703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:46:53.731920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066083+0000","last_change":"2024-02-17T12:47:46.072029+0000","last_active":"2024-02-17T12:47:47.066083+0000","last_peered":"2024-02-17T12:47:47.066083+0000","last_clean":"2024-02-17T12:47:47.066083+0000","last_became_active":"2024-02-17T12:47:46.071716+0000","last_became_peered":"2024-02-17T12:47:46.071716+0000","last_unstale":"2024-02-17T12:47:47.066083+0000","last_undegraded":"2024-02-17T12:47:47.066083+0000","last_fullsized":"2024-02-17T12:47:47.066083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:33.076266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.713573+0000","last_change":"2024-02-17T12:47:45.067443+0000","last_active":"2024-02-17T12:47:48.713573+0000","last_peered":"2024-02-17T12:47:48.713573+0000","last_clean":"2024-02-17T12:47:48.713573+0000","last_became_active":"2024-02-17T12:47:45.067090+0000","last_became_peered":"2024-02-17T12:47:45.067090+0000","last_unstale":"2024-02-17T12:47:48.713573+0000","last_undegraded":"2024-02-17T12:47:48.713573+0000","last_fullsized":"2024-02-17T12:47:48.713573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:17:39.874810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065310+0000","last_change":"2024-02-17T12:47:46.072275+0000","last_active":"2024-02-17T12:47:47.065310+0000","last_peered":"2024-02-17T12:47:47.065310+0000","last_clean":"2024-02-17T12:47:47.065310+0000","last_became_active":"2024-02-17T12:47:46.071951+0000","last_became_peered":"2024-02-17T12:47:46.071951+0000","last_unstale":"2024-02-17T12:47:47.065310+0000","last_undegraded":"2024-02-17T12:47:47.065310+0000","last_fullsized":"2024-02-17T12:47:47.065310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:45:58.580854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065538+0000","last_change":"2024-02-17T12:47:45.058875+0000","last_active":"2024-02-17T12:47:47.065538+0000","last_peered":"2024-02-17T12:47:47.065538+0000","last_clean":"2024-02-17T12:47:47.065538+0000","last_became_active":"2024-02-17T12:47:45.058433+0000","last_became_peered":"2024-02-17T12:47:45.058433+0000","last_unstale":"2024-02-17T12:47:47.065538+0000","last_undegraded":"2024-02-17T12:47:47.065538+0000","last_fullsized":"2024-02-17T12:47:47.065538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:49.932205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.150333+0000","last_change":"2024-02-17T12:47:46.070233+0000","last_active":"2024-02-17T12:47:47.150333+0000","last_peered":"2024-02-17T12:47:47.150333+0000","last_clean":"2024-02-17T12:47:47.150333+0000","last_became_active":"2024-02-17T12:47:46.069811+0000","last_became_peered":"2024-02-17T12:47:46.069811+0000","last_unstale":"2024-02-17T12:47:47.150333+0000","last_undegraded":"2024-02-17T12:47:47.150333+0000","last_fullsized":"2024-02-17T12:47:47.150333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:50:00.885517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.084465+0000","last_change":"2024-02-17T12:47:45.068778+0000","last_active":"2024-02-17T12:47:48.084465+0000","last_peered":"2024-02-17T12:47:48.084465+0000","last_clean":"2024-02-17T12:47:48.084465+0000","last_became_active":"2024-02-17T12:47:45.068375+0000","last_became_peered":"2024-02-17T12:47:45.068375+0000","last_unstale":"2024-02-17T12:47:48.084465+0000","last_undegraded":"2024-02-17T12:47:48.084465+0000","last_fullsized":"2024-02-17T12:47:48.084465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:42:57.600138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066225+0000","last_change":"2024-02-17T12:47:46.076802+0000","last_active":"2024-02-17T12:47:47.066225+0000","last_peered":"2024-02-17T12:47:47.066225+0000","last_clean":"2024-02-17T12:47:47.066225+0000","last_became_active":"2024-02-17T12:47:46.076433+0000","last_became_peered":"2024-02-17T12:47:46.076433+0000","last_unstale":"2024-02-17T12:47:47.066225+0000","last_undegraded":"2024-02-17T12:47:47.066225+0000","last_fullsized":"2024-02-17T12:47:47.066225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.352488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.084520+0000","last_change":"2024-02-17T12:47:45.059100+0000","last_active":"2024-02-17T12:47:48.084520+0000","last_peered":"2024-02-17T12:47:48.084520+0000","last_clean":"2024-02-17T12:47:48.084520+0000","last_became_active":"2024-02-17T12:47:45.058707+0000","last_became_peered":"2024-02-17T12:47:45.058707+0000","last_unstale":"2024-02-17T12:47:48.084520+0000","last_undegraded":"2024-02-17T12:47:48.084520+0000","last_fullsized":"2024-02-17T12:47:48.084520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:28:27.044127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179710+0000","last_change":"2024-02-17T12:47:46.065088+0000","last_active":"2024-02-17T12:47:47.179710+0000","last_peered":"2024-02-17T12:47:47.179710+0000","last_clean":"2024-02-17T12:47:47.179710+0000","last_became_active":"2024-02-17T12:47:46.064754+0000","last_became_peered":"2024-02-17T12:47:46.064754+0000","last_unstale":"2024-02-17T12:47:47.179710+0000","last_undegraded":"2024-02-17T12:47:47.179710+0000","last_fullsized":"2024-02-17T12:47:47.179710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:33:10.652881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.084701+0000","last_change":"2024-02-17T12:47:45.055403+0000","last_active":"2024-02-17T12:47:48.084701+0000","last_peered":"2024-02-17T12:47:48.084701+0000","last_clean":"2024-02-17T12:47:48.084701+0000","last_became_active":"2024-02-17T12:47:45.055083+0000","last_became_peered":"2024-02-17T12:47:45.055083+0000","last_unstale":"2024-02-17T12:47:48.084701+0000","last_undegraded":"2024-02-17T12:47:48.084701+0000","last_fullsized":"2024-02-17T12:47:48.084701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:05:51.295074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066267+0000","last_change":"2024-02-17T12:47:46.074883+0000","last_active":"2024-02-17T12:47:47.066267+0000","last_peered":"2024-02-17T12:47:47.066267+0000","last_clean":"2024-02-17T12:47:47.066267+0000","last_became_active":"2024-02-17T12:47:46.074414+0000","last_became_peered":"2024-02-17T12:47:46.074414+0000","last_unstale":"2024-02-17T12:47:47.066267+0000","last_undegraded":"2024-02-17T12:47:47.066267+0000","last_fullsized":"2024-02-17T12:47:47.066267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:35:55.026587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.706730+0000","last_change":"2024-02-17T12:47:45.057627+0000","last_active":"2024-02-17T12:47:48.706730+0000","last_peered":"2024-02-17T12:47:48.706730+0000","last_clean":"2024-02-17T12:47:48.706730+0000","last_became_active":"2024-02-17T12:47:45.057297+0000","last_became_peered":"2024-02-17T12:47:45.057297+0000","last_unstale":"2024-02-17T12:47:48.706730+0000","last_undegraded":"2024-02-17T12:47:48.706730+0000","last_fullsized":"2024-02-17T12:47:48.706730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:00:17.159939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243521+0000","last_change":"2024-02-17T12:47:46.076854+0000","last_active":"2024-02-17T12:47:47.243521+0000","last_peered":"2024-02-17T12:47:47.243521+0000","last_clean":"2024-02-17T12:47:47.243521+0000","last_became_active":"2024-02-17T12:47:46.076534+0000","last_became_peered":"2024-02-17T12:47:46.076534+0000","last_unstale":"2024-02-17T12:47:47.243521+0000","last_undegraded":"2024-02-17T12:47:47.243521+0000","last_fullsized":"2024-02-17T12:47:47.243521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:43:01.979075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179714+0000","last_change":"2024-02-17T12:47:45.060090+0000","last_active":"2024-02-17T12:47:47.179714+0000","last_peered":"2024-02-17T12:47:47.179714+0000","last_clean":"2024-02-17T12:47:47.179714+0000","last_became_active":"2024-02-17T12:47:45.058942+0000","last_became_peered":"2024-02-17T12:47:45.058942+0000","last_unstale":"2024-02-17T12:47:47.179714+0000","last_undegraded":"2024-02-17T12:47:47.179714+0000","last_fullsized":"2024-02-17T12:47:47.179714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:30:52.247140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179092+0000","last_change":"2024-02-17T12:47:46.071982+0000","last_active":"2024-02-17T12:47:47.179092+0000","last_peered":"2024-02-17T12:47:47.179092+0000","last_clean":"2024-02-17T12:47:47.179092+0000","last_became_active":"2024-02-17T12:47:46.071639+0000","last_became_peered":"2024-02-17T12:47:46.071639+0000","last_unstale":"2024-02-17T12:47:47.179092+0000","last_undegraded":"2024-02-17T12:47:47.179092+0000","last_fullsized":"2024-02-17T12:47:47.179092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:43:50.658132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.084214+0000","last_change":"2024-02-17T12:47:45.060017+0000","last_active":"2024-02-17T12:47:48.084214+0000","last_peered":"2024-02-17T12:47:48.084214+0000","last_clean":"2024-02-17T12:47:48.084214+0000","last_became_active":"2024-02-17T12:47:45.059104+0000","last_became_peered":"2024-02-17T12:47:45.059104+0000","last_unstale":"2024-02-17T12:47:48.084214+0000","last_undegraded":"2024-02-17T12:47:48.084214+0000","last_fullsized":"2024-02-17T12:47:48.084214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:56:50.457297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180374+0000","last_change":"2024-02-17T12:47:46.072250+0000","last_active":"2024-02-17T12:47:47.180374+0000","last_peered":"2024-02-17T12:47:47.180374+0000","last_clean":"2024-02-17T12:47:47.180374+0000","last_became_active":"2024-02-17T12:47:46.071856+0000","last_became_peered":"2024-02-17T12:47:46.071856+0000","last_unstale":"2024-02-17T12:47:47.180374+0000","last_undegraded":"2024-02-17T12:47:47.180374+0000","last_fullsized":"2024-02-17T12:47:47.180374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:46:04.628146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:49.717914+0000","last_change":"2024-02-17T12:47:45.056386+0000","last_active":"2024-02-17T12:47:49.717914+0000","last_peered":"2024-02-17T12:47:49.717914+0000","last_clean":"2024-02-17T12:47:49.717914+0000","last_became_active":"2024-02-17T12:47:45.055630+0000","last_became_peered":"2024-02-17T12:47:45.055630+0000","last_unstale":"2024-02-17T12:47:49.717914+0000","last_undegraded":"2024-02-17T12:47:49.717914+0000","last_fullsized":"2024-02-17T12:47:49.717914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:16.388950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179359+0000","last_change":"2024-02-17T12:47:46.071612+0000","last_active":"2024-02-17T12:47:47.179359+0000","last_peered":"2024-02-17T12:47:47.179359+0000","last_clean":"2024-02-17T12:47:47.179359+0000","last_became_active":"2024-02-17T12:47:46.071237+0000","last_became_peered":"2024-02-17T12:47:46.071237+0000","last_unstale":"2024-02-17T12:47:47.179359+0000","last_undegraded":"2024-02-17T12:47:47.179359+0000","last_fullsized":"2024-02-17T12:47:47.179359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:24:01.099679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'10","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.083192+0000","last_change":"2024-02-17T12:47:45.066258+0000","last_active":"2024-02-17T12:47:48.083192+0000","last_peered":"2024-02-17T12:47:48.083192+0000","last_clean":"2024-02-17T12:47:48.083192+0000","last_became_active":"2024-02-17T12:47:45.065813+0000","last_became_peered":"2024-02-17T12:47:45.065813+0000","last_unstale":"2024-02-17T12:47:48.083192+0000","last_undegraded":"2024-02-17T12:47:48.083192+0000","last_fullsized":"2024-02-17T12:47:48.083192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:06:43.298573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065640+0000","last_change":"2024-02-17T12:47:46.080347+0000","last_active":"2024-02-17T12:47:47.065640+0000","last_peered":"2024-02-17T12:47:47.065640+0000","last_clean":"2024-02-17T12:47:47.065640+0000","last_became_active":"2024-02-17T12:47:46.080083+0000","last_became_peered":"2024-02-17T12:47:46.080083+0000","last_unstale":"2024-02-17T12:47:47.065640+0000","last_undegraded":"2024-02-17T12:47:47.065640+0000","last_fullsized":"2024-02-17T12:47:47.065640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:24:41.807633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243639+0000","last_change":"2024-02-17T12:47:45.062814+0000","last_active":"2024-02-17T12:47:47.243639+0000","last_peered":"2024-02-17T12:47:47.243639+0000","last_clean":"2024-02-17T12:47:47.243639+0000","last_became_active":"2024-02-17T12:47:45.062418+0000","last_became_peered":"2024-02-17T12:47:45.062418+0000","last_unstale":"2024-02-17T12:47:47.243639+0000","last_undegraded":"2024-02-17T12:47:47.243639+0000","last_fullsized":"2024-02-17T12:47:47.243639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:49:34.577275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180380+0000","last_change":"2024-02-17T12:47:46.067742+0000","last_active":"2024-02-17T12:47:47.180380+0000","last_peered":"2024-02-17T12:47:47.180380+0000","last_clean":"2024-02-17T12:47:47.180380+0000","last_became_active":"2024-02-17T12:47:46.067329+0000","last_became_peered":"2024-02-17T12:47:46.067329+0000","last_unstale":"2024-02-17T12:47:47.180380+0000","last_undegraded":"2024-02-17T12:47:47.180380+0000","last_fullsized":"2024-02-17T12:47:47.180380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:19.432999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.150534+0000","last_change":"2024-02-17T12:47:45.065461+0000","last_active":"2024-02-17T12:47:47.150534+0000","last_peered":"2024-02-17T12:47:47.150534+0000","last_clean":"2024-02-17T12:47:47.150534+0000","last_became_active":"2024-02-17T12:47:45.065066+0000","last_became_peered":"2024-02-17T12:47:45.065066+0000","last_unstale":"2024-02-17T12:47:47.150534+0000","last_undegraded":"2024-02-17T12:47:47.150534+0000","last_fullsized":"2024-02-17T12:47:47.150534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:25:47.292901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180138+0000","last_change":"2024-02-17T12:47:46.074222+0000","last_active":"2024-02-17T12:47:47.180138+0000","last_peered":"2024-02-17T12:47:47.180138+0000","last_clean":"2024-02-17T12:47:47.180138+0000","last_became_active":"2024-02-17T12:47:46.073886+0000","last_became_peered":"2024-02-17T12:47:46.073886+0000","last_unstale":"2024-02-17T12:47:47.180138+0000","last_undegraded":"2024-02-17T12:47:47.180138+0000","last_fullsized":"2024-02-17T12:47:47.180138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:46:18.063422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180601+0000","last_change":"2024-02-17T12:47:45.071414+0000","last_active":"2024-02-17T12:47:47.180601+0000","last_peered":"2024-02-17T12:47:47.180601+0000","last_clean":"2024-02-17T12:47:47.180601+0000","last_became_active":"2024-02-17T12:47:45.071127+0000","last_became_peered":"2024-02-17T12:47:45.071127+0000","last_unstale":"2024-02-17T12:47:47.180601+0000","last_undegraded":"2024-02-17T12:47:47.180601+0000","last_fullsized":"2024-02-17T12:47:47.180601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:13:21.187718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179825+0000","last_change":"2024-02-17T12:47:46.059412+0000","last_active":"2024-02-17T12:47:47.179825+0000","last_peered":"2024-02-17T12:47:47.179825+0000","last_clean":"2024-02-17T12:47:47.179825+0000","last_became_active":"2024-02-17T12:47:46.059100+0000","last_became_peered":"2024-02-17T12:47:46.059100+0000","last_unstale":"2024-02-17T12:47:47.179825+0000","last_undegraded":"2024-02-17T12:47:47.179825+0000","last_fullsized":"2024-02-17T12:47:47.179825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:35.930613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065990+0000","last_change":"2024-02-17T12:47:46.071467+0000","last_active":"2024-02-17T12:47:47.065990+0000","last_peered":"2024-02-17T12:47:47.065990+0000","last_clean":"2024-02-17T12:47:47.065990+0000","last_became_active":"2024-02-17T12:47:46.070879+0000","last_became_peered":"2024-02-17T12:47:46.070879+0000","last_unstale":"2024-02-17T12:47:47.065990+0000","last_undegraded":"2024-02-17T12:47:47.065990+0000","last_fullsized":"2024-02-17T12:47:47.065990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.682842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.714859+0000","last_change":"2024-02-17T12:47:45.070545+0000","last_active":"2024-02-17T12:47:48.714859+0000","last_peered":"2024-02-17T12:47:48.714859+0000","last_clean":"2024-02-17T12:47:48.714859+0000","last_became_active":"2024-02-17T12:47:45.070265+0000","last_became_peered":"2024-02-17T12:47:45.070265+0000","last_unstale":"2024-02-17T12:47:48.714859+0000","last_undegraded":"2024-02-17T12:47:48.714859+0000","last_fullsized":"2024-02-17T12:47:48.714859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.763735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065446+0000","last_change":"2024-02-17T12:47:46.068539+0000","last_active":"2024-02-17T12:47:47.065446+0000","last_peered":"2024-02-17T12:47:47.065446+0000","last_clean":"2024-02-17T12:47:47.065446+0000","last_became_active":"2024-02-17T12:47:46.068169+0000","last_became_peered":"2024-02-17T12:47:46.068169+0000","last_unstale":"2024-02-17T12:47:47.065446+0000","last_undegraded":"2024-02-17T12:47:47.065446+0000","last_fullsized":"2024-02-17T12:47:47.065446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:27:28.485868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:49.718563+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:47:49.718563+0000","last_peered":"2024-02-17T12:47:49.718563+0000","last_clean":"2024-02-17T12:47:49.718563+0000","last_became_active":"2024-02-17T12:47:45.066299+0000","last_became_peered":"2024-02-17T12:47:45.066299+0000","last_unstale":"2024-02-17T12:47:49.718563+0000","last_undegraded":"2024-02-17T12:47:49.718563+0000","last_fullsized":"2024-02-17T12:47:49.718563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:34:02.040002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180075+0000","last_change":"2024-02-17T12:46:32.556576+0000","last_active":"2024-02-17T12:47:47.180075+0000","last_peered":"2024-02-17T12:47:47.180075+0000","last_clean":"2024-02-17T12:47:47.180075+0000","last_became_active":"2024-02-17T12:46:32.556160+0000","last_became_peered":"2024-02-17T12:46:32.556160+0000","last_unstale":"2024-02-17T12:47:47.180075+0000","last_undegraded":"2024-02-17T12:47:47.180075+0000","last_fullsized":"2024-02-17T12:47:47.180075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_clean_scrub_stamp":"2024-02-17T12:43:13.147248+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:57:52.244951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.178706+0000","last_change":"2024-02-17T12:47:46.073254+0000","last_active":"2024-02-17T12:47:47.178706+0000","last_peered":"2024-02-17T12:47:47.178706+0000","last_clean":"2024-02-17T12:47:47.178706+0000","last_became_active":"2024-02-17T12:47:46.072925+0000","last_became_peered":"2024-02-17T12:47:46.072925+0000","last_unstale":"2024-02-17T12:47:47.178706+0000","last_undegraded":"2024-02-17T12:47:47.178706+0000","last_fullsized":"2024-02-17T12:47:47.178706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:01:27.498671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.711138+0000","last_change":"2024-02-17T12:47:45.061112+0000","last_active":"2024-02-17T12:47:48.711138+0000","last_peered":"2024-02-17T12:47:48.711138+0000","last_clean":"2024-02-17T12:47:48.711138+0000","last_became_active":"2024-02-17T12:47:45.060633+0000","last_became_peered":"2024-02-17T12:47:45.060633+0000","last_unstale":"2024-02-17T12:47:48.711138+0000","last_undegraded":"2024-02-17T12:47:48.711138+0000","last_fullsized":"2024-02-17T12:47:48.711138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:29:29.421744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.082115+0000","last_change":"2024-02-17T12:47:45.066897+0000","last_active":"2024-02-17T12:47:48.082115+0000","last_peered":"2024-02-17T12:47:48.082115+0000","last_clean":"2024-02-17T12:47:48.082115+0000","last_became_active":"2024-02-17T12:47:45.066357+0000","last_became_peered":"2024-02-17T12:47:45.066357+0000","last_unstale":"2024-02-17T12:47:48.082115+0000","last_undegraded":"2024-02-17T12:47:48.082115+0000","last_fullsized":"2024-02-17T12:47:48.082115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:50:09.840414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.178844+0000","last_change":"2024-02-17T12:47:46.072412+0000","last_active":"2024-02-17T12:47:47.178844+0000","last_peered":"2024-02-17T12:47:47.178844+0000","last_clean":"2024-02-17T12:47:47.178844+0000","last_became_active":"2024-02-17T12:47:46.072105+0000","last_became_peered":"2024-02-17T12:47:46.072105+0000","last_unstale":"2024-02-17T12:47:47.178844+0000","last_undegraded":"2024-02-17T12:47:47.178844+0000","last_fullsized":"2024-02-17T12:47:47.178844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:33:19.367588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:49.719949+0000","last_change":"2024-02-17T12:47:45.067673+0000","last_active":"2024-02-17T12:47:49.719949+0000","last_peered":"2024-02-17T12:47:49.719949+0000","last_clean":"2024-02-17T12:47:49.719949+0000","last_became_active":"2024-02-17T12:47:45.067212+0000","last_became_peered":"2024-02-17T12:47:45.067212+0000","last_unstale":"2024-02-17T12:47:49.719949+0000","last_undegraded":"2024-02-17T12:47:49.719949+0000","last_fullsized":"2024-02-17T12:47:49.719949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:15:04.236158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065634+0000","last_change":"2024-02-17T12:47:46.077476+0000","last_active":"2024-02-17T12:47:47.065634+0000","last_peered":"2024-02-17T12:47:47.065634+0000","last_clean":"2024-02-17T12:47:47.065634+0000","last_became_active":"2024-02-17T12:47:46.077101+0000","last_became_peered":"2024-02-17T12:47:46.077101+0000","last_unstale":"2024-02-17T12:47:47.065634+0000","last_undegraded":"2024-02-17T12:47:47.065634+0000","last_fullsized":"2024-02-17T12:47:47.065634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:41:41.402101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:50.724191+0000","last_change":"2024-02-17T12:47:45.066693+0000","last_active":"2024-02-17T12:47:50.724191+0000","last_peered":"2024-02-17T12:47:50.724191+0000","last_clean":"2024-02-17T12:47:50.724191+0000","last_became_active":"2024-02-17T12:47:45.066349+0000","last_became_peered":"2024-02-17T12:47:45.066349+0000","last_unstale":"2024-02-17T12:47:50.724191+0000","last_undegraded":"2024-02-17T12:47:50.724191+0000","last_fullsized":"2024-02-17T12:47:50.724191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:30.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179783+0000","last_change":"2024-02-17T12:47:46.072899+0000","last_active":"2024-02-17T12:47:47.179783+0000","last_peered":"2024-02-17T12:47:47.179783+0000","last_clean":"2024-02-17T12:47:47.179783+0000","last_became_active":"2024-02-17T12:47:46.072488+0000","last_became_peered":"2024-02-17T12:47:46.072488+0000","last_unstale":"2024-02-17T12:47:47.179783+0000","last_undegraded":"2024-02-17T12:47:47.179783+0000","last_fullsized":"2024-02-17T12:47:47.179783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:05:43.675086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.084158+0000","last_change":"2024-02-17T12:47:45.056995+0000","last_active":"2024-02-17T12:47:48.084158+0000","last_peered":"2024-02-17T12:47:48.084158+0000","last_clean":"2024-02-17T12:47:48.084158+0000","last_became_active":"2024-02-17T12:47:45.056021+0000","last_became_peered":"2024-02-17T12:47:45.056021+0000","last_unstale":"2024-02-17T12:47:48.084158+0000","last_undegraded":"2024-02-17T12:47:48.084158+0000","last_fullsized":"2024-02-17T12:47:48.084158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:41.444513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243235+0000","last_change":"2024-02-17T12:47:46.065859+0000","last_active":"2024-02-17T12:47:47.243235+0000","last_peered":"2024-02-17T12:47:47.243235+0000","last_clean":"2024-02-17T12:47:47.243235+0000","last_became_active":"2024-02-17T12:47:46.065469+0000","last_became_peered":"2024-02-17T12:47:46.065469+0000","last_unstale":"2024-02-17T12:47:47.243235+0000","last_undegraded":"2024-02-17T12:47:47.243235+0000","last_fullsized":"2024-02-17T12:47:47.243235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:51:25.105361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:49.719022+0000","last_change":"2024-02-17T12:47:45.066524+0000","last_active":"2024-02-17T12:47:49.719022+0000","last_peered":"2024-02-17T12:47:49.719022+0000","last_clean":"2024-02-17T12:47:49.719022+0000","last_became_active":"2024-02-17T12:47:45.066218+0000","last_became_peered":"2024-02-17T12:47:45.066218+0000","last_unstale":"2024-02-17T12:47:49.719022+0000","last_undegraded":"2024-02-17T12:47:49.719022+0000","last_fullsized":"2024-02-17T12:47:49.719022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:56:47.244262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179664+0000","last_change":"2024-02-17T12:47:46.065779+0000","last_active":"2024-02-17T12:47:47.179664+0000","last_peered":"2024-02-17T12:47:47.179664+0000","last_clean":"2024-02-17T12:47:47.179664+0000","last_became_active":"2024-02-17T12:47:46.065459+0000","last_became_peered":"2024-02-17T12:47:46.065459+0000","last_unstale":"2024-02-17T12:47:47.179664+0000","last_undegraded":"2024-02-17T12:47:47.179664+0000","last_fullsized":"2024-02-17T12:47:47.179664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:08.944487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179624+0000","last_change":"2024-02-17T12:47:45.059639+0000","last_active":"2024-02-17T12:47:47.179624+0000","last_peered":"2024-02-17T12:47:47.179624+0000","last_clean":"2024-02-17T12:47:47.179624+0000","last_became_active":"2024-02-17T12:47:45.059085+0000","last_became_peered":"2024-02-17T12:47:45.059085+0000","last_unstale":"2024-02-17T12:47:47.179624+0000","last_undegraded":"2024-02-17T12:47:47.179624+0000","last_fullsized":"2024-02-17T12:47:47.179624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:48:55.847405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065248+0000","last_change":"2024-02-17T12:47:46.079623+0000","last_active":"2024-02-17T12:47:47.065248+0000","last_peered":"2024-02-17T12:47:47.065248+0000","last_clean":"2024-02-17T12:47:47.065248+0000","last_became_active":"2024-02-17T12:47:46.079285+0000","last_became_peered":"2024-02-17T12:47:46.079285+0000","last_unstale":"2024-02-17T12:47:47.065248+0000","last_undegraded":"2024-02-17T12:47:47.065248+0000","last_fullsized":"2024-02-17T12:47:47.065248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:45:14.441218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.084372+0000","last_change":"2024-02-17T12:47:45.064935+0000","last_active":"2024-02-17T12:47:48.084372+0000","last_peered":"2024-02-17T12:47:48.084372+0000","last_clean":"2024-02-17T12:47:48.084372+0000","last_became_active":"2024-02-17T12:47:45.064592+0000","last_became_peered":"2024-02-17T12:47:45.064592+0000","last_unstale":"2024-02-17T12:47:48.084372+0000","last_undegraded":"2024-02-17T12:47:48.084372+0000","last_fullsized":"2024-02-17T12:47:48.084372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:36.943316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180756+0000","last_change":"2024-02-17T12:47:46.074429+0000","last_active":"2024-02-17T12:47:47.180756+0000","last_peered":"2024-02-17T12:47:47.180756+0000","last_clean":"2024-02-17T12:47:47.180756+0000","last_became_active":"2024-02-17T12:47:46.074123+0000","last_became_peered":"2024-02-17T12:47:46.074123+0000","last_unstale":"2024-02-17T12:47:47.180756+0000","last_undegraded":"2024-02-17T12:47:47.180756+0000","last_fullsized":"2024-02-17T12:47:47.180756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:50:17.633831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179942+0000","last_change":"2024-02-17T12:47:45.061362+0000","last_active":"2024-02-17T12:47:47.179942+0000","last_peered":"2024-02-17T12:47:47.179942+0000","last_clean":"2024-02-17T12:47:47.179942+0000","last_became_active":"2024-02-17T12:47:45.061037+0000","last_became_peered":"2024-02-17T12:47:45.061037+0000","last_unstale":"2024-02-17T12:47:47.179942+0000","last_undegraded":"2024-02-17T12:47:47.179942+0000","last_fullsized":"2024-02-17T12:47:47.179942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:01.987157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180582+0000","last_change":"2024-02-17T12:47:46.073862+0000","last_active":"2024-02-17T12:47:47.180582+0000","last_peered":"2024-02-17T12:47:47.180582+0000","last_clean":"2024-02-17T12:47:47.180582+0000","last_became_active":"2024-02-17T12:47:46.073543+0000","last_became_peered":"2024-02-17T12:47:46.073543+0000","last_unstale":"2024-02-17T12:47:47.180582+0000","last_undegraded":"2024-02-17T12:47:47.180582+0000","last_fullsized":"2024-02-17T12:47:47.180582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:57:57.567726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065963+0000","last_change":"2024-02-17T12:47:45.060616+0000","last_active":"2024-02-17T12:47:47.065963+0000","last_peered":"2024-02-17T12:47:47.065963+0000","last_clean":"2024-02-17T12:47:47.065963+0000","last_became_active":"2024-02-17T12:47:45.060296+0000","last_became_peered":"2024-02-17T12:47:45.060296+0000","last_unstale":"2024-02-17T12:47:47.065963+0000","last_undegraded":"2024-02-17T12:47:47.065963+0000","last_fullsized":"2024-02-17T12:47:47.065963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:22:44.791363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179670+0000","last_change":"2024-02-17T12:47:46.072738+0000","last_active":"2024-02-17T12:47:47.179670+0000","last_peered":"2024-02-17T12:47:47.179670+0000","last_clean":"2024-02-17T12:47:47.179670+0000","last_became_active":"2024-02-17T12:47:46.072421+0000","last_became_peered":"2024-02-17T12:47:46.072421+0000","last_unstale":"2024-02-17T12:47:47.179670+0000","last_undegraded":"2024-02-17T12:47:47.179670+0000","last_fullsized":"2024-02-17T12:47:47.179670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:50:23.501787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179040+0000","last_change":"2024-02-17T12:47:45.071201+0000","last_active":"2024-02-17T12:47:47.179040+0000","last_peered":"2024-02-17T12:47:47.179040+0000","last_clean":"2024-02-17T12:47:47.179040+0000","last_became_active":"2024-02-17T12:47:45.070777+0000","last_became_peered":"2024-02-17T12:47:45.070777+0000","last_unstale":"2024-02-17T12:47:47.179040+0000","last_undegraded":"2024-02-17T12:47:47.179040+0000","last_fullsized":"2024-02-17T12:47:47.179040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:14:02.807288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180622+0000","last_change":"2024-02-17T12:47:46.066072+0000","last_active":"2024-02-17T12:47:47.180622+0000","last_peered":"2024-02-17T12:47:47.180622+0000","last_clean":"2024-02-17T12:47:47.180622+0000","last_became_active":"2024-02-17T12:47:46.065755+0000","last_became_peered":"2024-02-17T12:47:46.065755+0000","last_unstale":"2024-02-17T12:47:47.180622+0000","last_undegraded":"2024-02-17T12:47:47.180622+0000","last_fullsized":"2024-02-17T12:47:47.180622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:09:29.534097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179984+0000","last_change":"2024-02-17T12:47:46.067268+0000","last_active":"2024-02-17T12:47:47.179984+0000","last_peered":"2024-02-17T12:47:47.179984+0000","last_clean":"2024-02-17T12:47:47.179984+0000","last_became_active":"2024-02-17T12:47:46.066918+0000","last_became_peered":"2024-02-17T12:47:46.066918+0000","last_unstale":"2024-02-17T12:47:47.179984+0000","last_undegraded":"2024-02-17T12:47:47.179984+0000","last_fullsized":"2024-02-17T12:47:47.179984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:11:09.844954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180012+0000","last_change":"2024-02-17T12:47:46.068143+0000","last_active":"2024-02-17T12:47:47.180012+0000","last_peered":"2024-02-17T12:47:47.180012+0000","last_clean":"2024-02-17T12:47:47.180012+0000","last_became_active":"2024-02-17T12:47:46.065112+0000","last_became_peered":"2024-02-17T12:47:46.065112+0000","last_unstale":"2024-02-17T12:47:47.180012+0000","last_undegraded":"2024-02-17T12:47:47.180012+0000","last_fullsized":"2024-02-17T12:47:47.180012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:43:04.982009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179804+0000","last_change":"2024-02-17T12:47:46.076139+0000","last_active":"2024-02-17T12:47:47.179804+0000","last_peered":"2024-02-17T12:47:47.179804+0000","last_clean":"2024-02-17T12:47:47.179804+0000","last_became_active":"2024-02-17T12:47:46.075826+0000","last_became_peered":"2024-02-17T12:47:46.075826+0000","last_unstale":"2024-02-17T12:47:47.179804+0000","last_undegraded":"2024-02-17T12:47:47.179804+0000","last_fullsized":"2024-02-17T12:47:47.179804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:39:08.460716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179700+0000","last_change":"2024-02-17T12:47:46.063792+0000","last_active":"2024-02-17T12:47:47.179700+0000","last_peered":"2024-02-17T12:47:47.179700+0000","last_clean":"2024-02-17T12:47:47.179700+0000","last_became_active":"2024-02-17T12:47:46.063407+0000","last_became_peered":"2024-02-17T12:47:46.063407+0000","last_unstale":"2024-02-17T12:47:47.179700+0000","last_undegraded":"2024-02-17T12:47:47.179700+0000","last_fullsized":"2024-02-17T12:47:47.179700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:26:26.973435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.178899+0000","last_change":"2024-02-17T12:47:46.066202+0000","last_active":"2024-02-17T12:47:47.178899+0000","last_peered":"2024-02-17T12:47:47.178899+0000","last_clean":"2024-02-17T12:47:47.178899+0000","last_became_active":"2024-02-17T12:47:46.065820+0000","last_became_peered":"2024-02-17T12:47:46.065820+0000","last_unstale":"2024-02-17T12:47:47.178899+0000","last_undegraded":"2024-02-17T12:47:47.178899+0000","last_fullsized":"2024-02-17T12:47:47.178899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:36:25.201697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180376+0000","last_change":"2024-02-17T12:47:46.065092+0000","last_active":"2024-02-17T12:47:47.180376+0000","last_peered":"2024-02-17T12:47:47.180376+0000","last_clean":"2024-02-17T12:47:47.180376+0000","last_became_active":"2024-02-17T12:47:46.064710+0000","last_became_peered":"2024-02-17T12:47:46.064710+0000","last_unstale":"2024-02-17T12:47:47.180376+0000","last_undegraded":"2024-02-17T12:47:47.180376+0000","last_fullsized":"2024-02-17T12:47:47.180376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:00:13.715042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180239+0000","last_change":"2024-02-17T12:47:46.070811+0000","last_active":"2024-02-17T12:47:47.180239+0000","last_peered":"2024-02-17T12:47:47.180239+0000","last_clean":"2024-02-17T12:47:47.180239+0000","last_became_active":"2024-02-17T12:47:46.070462+0000","last_became_peered":"2024-02-17T12:47:46.070462+0000","last_unstale":"2024-02-17T12:47:47.180239+0000","last_undegraded":"2024-02-17T12:47:47.180239+0000","last_fullsized":"2024-02-17T12:47:47.180239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:03.671869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065737+0000","last_change":"2024-02-17T12:47:46.077692+0000","last_active":"2024-02-17T12:47:47.065737+0000","last_peered":"2024-02-17T12:47:47.065737+0000","last_clean":"2024-02-17T12:47:47.065737+0000","last_became_active":"2024-02-17T12:47:46.077363+0000","last_became_peered":"2024-02-17T12:47:46.077363+0000","last_unstale":"2024-02-17T12:47:47.065737+0000","last_undegraded":"2024-02-17T12:47:47.065737+0000","last_fullsized":"2024-02-17T12:47:47.065737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:02:49.068782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065666+0000","last_change":"2024-02-17T12:47:46.073507+0000","last_active":"2024-02-17T12:47:47.065666+0000","last_peered":"2024-02-17T12:47:47.065666+0000","last_clean":"2024-02-17T12:47:47.065666+0000","last_became_active":"2024-02-17T12:47:46.073107+0000","last_became_peered":"2024-02-17T12:47:46.073107+0000","last_unstale":"2024-02-17T12:47:47.065666+0000","last_undegraded":"2024-02-17T12:47:47.065666+0000","last_fullsized":"2024-02-17T12:47:47.065666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:32:48.799300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065077+0000","last_change":"2024-02-17T12:47:46.073384+0000","last_active":"2024-02-17T12:47:47.065077+0000","last_peered":"2024-02-17T12:47:47.065077+0000","last_clean":"2024-02-17T12:47:47.065077+0000","last_became_active":"2024-02-17T12:47:46.073017+0000","last_became_peered":"2024-02-17T12:47:46.073017+0000","last_unstale":"2024-02-17T12:47:47.065077+0000","last_undegraded":"2024-02-17T12:47:47.065077+0000","last_fullsized":"2024-02-17T12:47:47.065077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:43:50.091667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066154+0000","last_change":"2024-02-17T12:47:46.073758+0000","last_active":"2024-02-17T12:47:47.066154+0000","last_peered":"2024-02-17T12:47:47.066154+0000","last_clean":"2024-02-17T12:47:47.066154+0000","last_became_active":"2024-02-17T12:47:46.072781+0000","last_became_peered":"2024-02-17T12:47:46.072781+0000","last_unstale":"2024-02-17T12:47:47.066154+0000","last_undegraded":"2024-02-17T12:47:47.066154+0000","last_fullsized":"2024-02-17T12:47:47.066154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:10:24.811100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179319+0000","last_change":"2024-02-17T12:47:46.072025+0000","last_active":"2024-02-17T12:47:47.179319+0000","last_peered":"2024-02-17T12:47:47.179319+0000","last_clean":"2024-02-17T12:47:47.179319+0000","last_became_active":"2024-02-17T12:47:46.071662+0000","last_became_peered":"2024-02-17T12:47:46.071662+0000","last_unstale":"2024-02-17T12:47:47.179319+0000","last_undegraded":"2024-02-17T12:47:47.179319+0000","last_fullsized":"2024-02-17T12:47:47.179319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:04:07.188552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180071+0000","last_change":"2024-02-17T12:47:46.068710+0000","last_active":"2024-02-17T12:47:47.180071+0000","last_peered":"2024-02-17T12:47:47.180071+0000","last_clean":"2024-02-17T12:47:47.180071+0000","last_became_active":"2024-02-17T12:47:46.068368+0000","last_became_peered":"2024-02-17T12:47:46.068368+0000","last_unstale":"2024-02-17T12:47:47.180071+0000","last_undegraded":"2024-02-17T12:47:47.180071+0000","last_fullsized":"2024-02-17T12:47:47.180071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:29:57.982649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5065,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":131,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":17396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":445071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]}]},{"osd":10,"up_from":73,"seq":313532612625,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1037437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":1,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48699999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.432}]},{"osd":4,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38100000000000001}]},{"osd":5,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57999999999999996}]},{"osd":6,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.502}]},{"osd":7,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.54900000000000004}]},{"osd":8,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.67800000000000005}]},{"osd":9,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.51500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40424,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704728,"statfs":{"total":95995035648,"available":95953641472,"internally_reserved":0,"allocated":950272,"data_stored":444787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48299999999999998}]},{"osd":1,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52400000000000002}]},{"osd":2,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.68999999999999995}]},{"osd":5,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76400000000000001}]},{"osd":6,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.56999999999999995}]},{"osd":7,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.753},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.78500000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.80300000000000005}]},{"osd":10,"last update":"Sat Feb 17 12:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.74299999999999999}]}]},{"osd":8,"up_from":60,"seq":257698037788,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":444505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.439,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.32400000000000001}]},{"osd":1,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.0089999999999999,"15min":1.0840000000000001},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73799999999999999,"5min":10.955,"15min":10.955},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.5}]},{"osd":2,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.442,"15min":0.442},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48999999999999999,"15min":0.496},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.78400000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.27900000000000003}]},{"osd":3,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.40200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.436}]},{"osd":4,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":1.0289999999999999,"15min":1.1000000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.874,"5min":10.972,"15min":10.972},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.47299999999999998}]},{"osd":5,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.376},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56899999999999995,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79800000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48399999999999999}]},{"osd":6,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.48099999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.77200000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.85199999999999998,"15min":0.89000000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88,"5min":4.9269999999999996,"15min":4.9269999999999996},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.624},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70999999999999996},"min":{"1min":0.49299999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.52300000000000002}]},{"osd":9,"last update":"Sat Feb 17 12:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53300000000000003}]},{"osd":10,"last update":"Sat Feb 17 12:47:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":10.987,"5min":10.987,"15min":10.987},"last":0.55200000000000005}]}]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32684,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712468,"statfs":{"total":95995035648,"available":95961567232,"internally_reserved":0,"allocated":1544192,"data_stored":1036711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.41899999999999998,"15min":0.40999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.60599999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.438,"15min":0.43099999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.315}]},{"osd":2,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.417,"5min":0.73999999999999999,"15min":0.627},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":15.003,"5min":15.003,"15min":15.003},"last":0.58799999999999997}]},{"osd":3,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.627,"15min":0.627},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.496},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86799999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.433}]},{"osd":4,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.51100000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.52100000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":0.80100000000000005,"15min":0.68500000000000005},"min":{"1min":0.48399999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":15.028,"5min":15.028,"15min":15.028},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.70999999999999996}]},{"osd":6,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.53800000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45100000000000001}]},{"osd":8,"last update":"Sat Feb 17 12:47:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.47699999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.74399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54700000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Sat Feb 17 12:47:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.60499999999999998}]},{"osd":10,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.63400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495654,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":443991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.375,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.55100000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40100000000000002,"15min":0.39700000000000002},"min":{"1min":0.26000000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59699999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.55800000000000005,"15min":0.54500000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.81799999999999995}]},{"osd":2,"last update":"Sat Feb 17 12:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.54600000000000004,"15min":0.53600000000000003},"min":{"1min":0.48599999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.50600000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.752,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.48399999999999999}]},{"osd":3,"last update":"Sat Feb 17 12:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.45400000000000001,"15min":0.44800000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.48499999999999999,"15min":0.47499999999999998},"min":{"1min":0.377,"5min":0.373,"15min":0.373},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46200000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Sat Feb 17 12:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61399999999999999,"15min":0.59999999999999998},"min":{"1min":0.51200000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.52700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.50800000000000001}]},{"osd":7,"last update":"Sat Feb 17 12:47:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.59599999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98399999999999999,"5min":1.21,"15min":1.21},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.65000000000000002,"15min":0.63900000000000001},"min":{"1min":0.58699999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.84299999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.73899999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.006,"5min":1.252,"15min":1.252},"last":0.72199999999999998}]},{"osd":9,"last update":"Sat Feb 17 12:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66000000000000003}]},{"osd":10,"last update":"Sat Feb 17 12:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.78800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":445491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:46:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47599999999999998,"15min":0.47799999999999998},"min":{"1min":0.29699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72599999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.34999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71599999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.66900000000000004}]},{"osd":2,"last update":"Sat Feb 17 12:47:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.503},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56699999999999995}]},{"osd":3,"last update":"Sat Feb 17 12:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77000000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.54100000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73299999999999998,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60499999999999998}]},{"osd":4,"last update":"Sat Feb 17 12:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39600000000000002,"15min":0.38200000000000001},"min":{"1min":0.28599999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64400000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.41699999999999998,"15min":0.41199999999999998},"min":{"1min":0.309,"5min":0.255,"15min":0.255},"max":{"1min":0.63400000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.68400000000000005}]},{"osd":5,"last update":"Sat Feb 17 12:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90400000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.502},{"interface":"front","average":{"1min":0.628,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.62}]},{"osd":6,"last update":"Sat Feb 17 12:46:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86699999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78200000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat Feb 17 12:47:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79800000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.755},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.374,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.871,"5min":1.079,"15min":1.079},"last":0.78400000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.65100000000000002}]},{"osd":9,"last update":"Sat Feb 17 12:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81299999999999994}]},{"osd":10,"last update":"Sat Feb 17 12:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":445357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 12:46:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.50800000000000001,"15min":0.47999999999999998},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.51900000000000002,"15min":0.48699999999999999},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45500000000000002}]},{"osd":2,"last update":"Sat Feb 17 12:47:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70399999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64400000000000002,"15min":0.73399999999999999},"min":{"1min":0.377,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81599999999999995,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.55800000000000005}]},{"osd":3,"last update":"Sat Feb 17 12:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.30299999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64200000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.443,"15min":0.42699999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Feb 17 12:47:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.42299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81799999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.436},{"interface":"front","average":{"1min":0.629,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80300000000000005,"5min":0.872,"15min":0.872},"last":0.54400000000000004}]},{"osd":5,"last update":"Sat Feb 17 12:47:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60099999999999998,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.48099999999999998}]},{"osd":6,"last update":"Sat Feb 17 12:46:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46600000000000003,"15min":0.45800000000000002},"min":{"1min":0.32100000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.311},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Sat Feb 17 12:47:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.443},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64600000000000002,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:47:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.627,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.44900000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77800000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.60999999999999999}]},{"osd":9,"last update":"Sat Feb 17 12:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.378},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46200000000000002}]},{"osd":10,"last update":"Sat Feb 17 12:47:35 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.495},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.624}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":929792,"data_stored":436955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48599999999999999,"15min":0.46200000000000002},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40899999999999997}]},{"osd":1,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52300000000000002,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.312},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51600000000000001,"15min":0.498},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.51300000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46500000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.55500000000000005,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.36899999999999999}]},{"osd":5,"last update":"Sat Feb 17 12:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.441,"15min":0.433},"min":{"1min":0.28699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.35999999999999999}]},{"osd":6,"last update":"Sat Feb 17 12:46:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53800000000000003}]},{"osd":7,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.60999999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.56399999999999995,"5min":0.998,"15min":0.998},"last":0.5},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72699999999999998,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.49199999999999999}]},{"osd":9,"last update":"Sat Feb 17 12:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.63800000000000001}]},{"osd":10,"last update":"Sat Feb 17 12:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.67100000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":444287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.31900000000000001},"min":{"1min":0.25900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.50700000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35499999999999998,"15min":0.33500000000000002},"min":{"1min":0.27200000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.67300000000000004}]},{"osd":1,"last update":"Sat Feb 17 12:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.51800000000000002},"min":{"1min":0.39300000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.84599999999999997,"5min":0.879,"15min":0.879},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57399999999999995}]},{"osd":2,"last update":"Sat Feb 17 12:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85899999999999999,"5min":0.88,"15min":0.88},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.43099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82599999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68799999999999994}]},{"osd":4,"last update":"Sat Feb 17 12:47:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53700000000000003}]},{"osd":5,"last update":"Sat Feb 17 12:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64100000000000001,"15min":0.627},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84799999999999998}]},{"osd":6,"last update":"Sat Feb 17 12:46:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.51200000000000001,"15min":0.499},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.46500000000000002}]},{"osd":7,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.629,"15min":0.623},"min":{"1min":0.50900000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.755},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.57599999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92400000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61399999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:47:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91600000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.71299999999999997,"15min":0.72599999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0069999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.56899999999999995}]},{"osd":9,"last update":"Sat Feb 17 12:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.59399999999999997}]},{"osd":10,"last update":"Sat Feb 17 12:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.90100000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":444936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57899999999999996}]},{"osd":1,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.35299999999999998},"min":{"1min":0.32300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.56100000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.38300000000000001,"15min":0.36699999999999999},"min":{"1min":0.29799999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.434}]},{"osd":2,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.76500000000000001,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76700000000000002,"5min":18.73,"15min":18.73},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.45500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78800000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52100000000000002}]},{"osd":3,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.39800000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.46400000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.54700000000000004,"5min":0.378,"15min":0.378},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.754}]},{"osd":6,"last update":"Sat Feb 17 12:46:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.63200000000000001}]},{"osd":7,"last update":"Sat Feb 17 12:47:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.35199999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38900000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59099999999999997}]},{"osd":8,"last update":"Sat Feb 17 12:47:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.504,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82899999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.73699999999999999}]},{"osd":9,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66900000000000004}]},{"osd":10,"last update":"Sat Feb 17 12:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1548288,"data_stored":1035723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.315,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78600000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.50700000000000001,"15min":0.497},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.52500000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.51300000000000001},"min":{"1min":0.372,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54100000000000004,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.748,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.61199999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38900000000000001,"15min":0.377},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.53400000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.36699999999999999},"min":{"1min":0.27000000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59899999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.38300000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73499999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.437},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54000000000000004,"15min":0.53000000000000003},"min":{"1min":0.373,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.503}]},{"osd":4,"last update":"Sat Feb 17 12:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.72299999999999998,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.51600000000000001},"min":{"1min":0.40400000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84199999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65500000000000003}]},{"osd":6,"last update":"Sat Feb 17 12:46:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57899999999999996,"15min":0.57399999999999995},"min":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.45600000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.629}]},{"osd":7,"last update":"Sat Feb 17 12:47:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.84199999999999997,"5min":1,"15min":1},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85799999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77700000000000002}]},{"osd":8,"last update":"Sat Feb 17 12:47:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.58199999999999996}]},{"osd":9,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.745}]},{"osd":10,"last update":"Sat Feb 17 12:47:29 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.80500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T12:47:53.146 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-17T12:47:53.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.289+0000 7fa67e683640 1 -- 172.21.15.18:0/1033104002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781000a0 msgr2=0x7fa6781004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:53.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.289+0000 7fa67e683640 1 --2- 172.21.15.18:0/1033104002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781000a0 0x7fa6781004c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa668009900 tx=0x7fa66802bc30 comp rx=0 tx=0).stop 2024-02-17T12:47:53.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.289+0000 7fa67e683640 1 -- 172.21.15.18:0/1033104002 shutdown_connections 2024-02-17T12:47:53.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.289+0000 7fa67e683640 1 --2- 172.21.15.18:0/1033104002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa678101c50 0x7fa6781084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 --2- 172.21.15.18:0/1033104002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa678101290 0x7fa678101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 --2- 172.21.15.18:0/1033104002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781000a0 0x7fa6781004c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 -- 172.21.15.18:0/1033104002 >> 172.21.15.18:0/1033104002 conn(0x7fa6780fb610 msgr2=0x7fa6780fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 -- 172.21.15.18:0/1033104002 shutdown_connections 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 -- 172.21.15.18:0/1033104002 wait complete. 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 Processor -- start 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 -- start start 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781000a0 0x7fa678197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa678101290 0x7fa6781977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa678101c50 0x7fa67819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67819ed50 con 0x7fa6781000a0 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67819eec0 con 0x7fa678101c50 2024-02-17T12:47:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa67e683640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67819f080 con 0x7fa678101290 2024-02-17T12:47:53.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.290+0000 7fa677fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781000a0 0x7fa678197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:53.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa677fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781000a0 0x7fa678197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39288/0 (socket says 172.21.15.18:39288) 2024-02-17T12:47:53.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa677fff640 1 -- 172.21.15.18:0/2105671519 learned_addr learned my addr 172.21.15.18:0/2105671519 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:53.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa6777fe640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa678101290 0x7fa6781977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:53.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa67cc20640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa678101c50 0x7fa67819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:53.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa677fff640 1 -- 172.21.15.18:0/2105671519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa678101290 msgr2=0x7fa6781977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:53.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa677fff640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa678101290 0x7fa6781977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa677fff640 1 -- 172.21.15.18:0/2105671519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa678101c50 msgr2=0x7fa67819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:53.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa677fff640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa678101c50 0x7fa67819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa677fff640 1 -- 172.21.15.18:0/2105671519 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa668009560 con 0x7fa6781000a0 2024-02-17T12:47:53.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa67cc20640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa678101c50 0x7fa67819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:47:53.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa6777fe640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa678101290 0x7fa6781977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:53.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa677fff640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781000a0 0x7fa678197260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa668005ab0 tx=0x7fa668004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:53.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa6757fa640 1 -- 172.21.15.18:0/2105671519 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668004aa0 con 0x7fa6781000a0 2024-02-17T12:47:53.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa6757fa640 1 -- 172.21.15.18:0/2105671519 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6680052d0 con 0x7fa6781000a0 2024-02-17T12:47:53.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa67e683640 1 -- 172.21.15.18:0/2105671519 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67819f300 con 0x7fa6781000a0 2024-02-17T12:47:53.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa6757fa640 1 -- 172.21.15.18:0/2105671519 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668003e80 con 0x7fa6781000a0 2024-02-17T12:47:53.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.291+0000 7fa67e683640 1 -- 172.21.15.18:0/2105671519 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67819f810 con 0x7fa6781000a0 2024-02-17T12:47:53.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.292+0000 7fa67e683640 1 -- 172.21.15.18:0/2105671519 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67810bab0 con 0x7fa6781000a0 2024-02-17T12:47:53.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.293+0000 7fa6757fa640 1 -- 172.21.15.18:0/2105671519 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa668005440 con 0x7fa6781000a0 2024-02-17T12:47:53.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.294+0000 7fa6757fa640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa6600a2150 0x7fa6600a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:53.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.294+0000 7fa6777fe640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa6600a2150 0x7fa6600a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:53.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.294+0000 7fa6757fa640 1 -- 172.21.15.18:0/2105671519 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa6680f07d0 con 0x7fa6781000a0 2024-02-17T12:47:53.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.294+0000 7fa6777fe640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa6600a2150 0x7fa6600a4610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa678198770 tx=0x7fa66c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:53.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.297+0000 7fa6757fa640 1 -- 172.21.15.18:0/2105671519 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa6680b9800 con 0x7fa6781000a0 2024-02-17T12:47:53.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:53 smithi119 ceph-mon[36188]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:53.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:53 smithi119 ceph-mon[36188]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:53.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:53 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/293319611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:53.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:53 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2813451632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:53.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:53 smithi146 ceph-mon[36454]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:53.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:53 smithi146 ceph-mon[36454]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:53.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:53 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/293319611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:53.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:53 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2813451632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.479+0000 7fa67e683640 1 -- 172.21.15.18:0/2105671519 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6780689f0 con 0x7fa6600a2150 2024-02-17T12:47:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.489+0000 7fa6757fa640 1 -- 172.21.15.18:0/2105671519 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335034 (secure 0 0 0) 0x7fa6780689f0 con 0x7fa6600a2150 2024-02-17T12:47:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:53.489 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-02-17T12:47:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.493+0000 7fa67e683640 1 -- 172.21.15.18:0/2105671519 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa6600a2150 msgr2=0x7fa6600a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.493+0000 7fa67e683640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa6600a2150 0x7fa6600a4610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa678198770 tx=0x7fa66c005f90 comp rx=0 tx=0).stop 2024-02-17T12:47:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.493+0000 7fa67e683640 1 -- 172.21.15.18:0/2105671519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781000a0 msgr2=0x7fa678197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.493+0000 7fa67e683640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781000a0 0x7fa678197260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa668005ab0 tx=0x7fa668004270 comp rx=0 tx=0).stop 2024-02-17T12:47:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.493+0000 7fa67e683640 1 -- 172.21.15.18:0/2105671519 shutdown_connections 2024-02-17T12:47:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.493+0000 7fa67e683640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa6600a2150 0x7fa6600a4610 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.493+0000 7fa67e683640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa678101c50 0x7fa67819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.493+0000 7fa67e683640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa678101290 0x7fa6781977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.493+0000 7fa67e683640 1 --2- 172.21.15.18:0/2105671519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781000a0 0x7fa678197260 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.493+0000 7fa67e683640 1 -- 172.21.15.18:0/2105671519 >> 172.21.15.18:0/2105671519 conn(0x7fa6780fb610 msgr2=0x7fa6780682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.493+0000 7fa67e683640 1 -- 172.21.15.18:0/2105671519 shutdown_connections 2024-02-17T12:47:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:53.493+0000 7fa67e683640 1 -- 172.21.15.18:0/2105671519 wait complete. 2024-02-17T12:47:53.495 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:53 smithi018 ceph-mon[32625]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:53.496 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:53 smithi018 ceph-mon[32625]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:53.496 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:53 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/293319611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:47:53.496 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:53 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2813451632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:47:53.497 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-02-17T12:47:52.191838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595433,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":149,"num_read_kb":130,"num_write":312,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":251,"ondisk_log_size":251,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":233,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374488,"kb_used_data":12824,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567336,"statfs":{"total":1151940427776,"available":1151556952064,"internally_reserved":0,"allocated":13131776,"data_stored":7105251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5065,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":131,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.147072"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180316+0000","last_change":"2024-02-17T12:47:46.069941+0000","last_active":"2024-02-17T12:47:47.180316+0000","last_peered":"2024-02-17T12:47:47.180316+0000","last_clean":"2024-02-17T12:47:47.180316+0000","last_became_active":"2024-02-17T12:47:46.069192+0000","last_became_peered":"2024-02-17T12:47:46.069192+0000","last_unstale":"2024-02-17T12:47:47.180316+0000","last_undegraded":"2024-02-17T12:47:47.180316+0000","last_fullsized":"2024-02-17T12:47:47.180316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:27.102839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179745+0000","last_change":"2024-02-17T12:47:46.064872+0000","last_active":"2024-02-17T12:47:47.179745+0000","last_peered":"2024-02-17T12:47:47.179745+0000","last_clean":"2024-02-17T12:47:47.179745+0000","last_became_active":"2024-02-17T12:47:46.064498+0000","last_became_peered":"2024-02-17T12:47:46.064498+0000","last_unstale":"2024-02-17T12:47:47.179745+0000","last_undegraded":"2024-02-17T12:47:47.179745+0000","last_fullsized":"2024-02-17T12:47:47.179745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:30:41.432657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179528+0000","last_change":"2024-02-17T12:47:46.067504+0000","last_active":"2024-02-17T12:47:47.179528+0000","last_peered":"2024-02-17T12:47:47.179528+0000","last_clean":"2024-02-17T12:47:47.179528+0000","last_became_active":"2024-02-17T12:47:46.067155+0000","last_became_peered":"2024-02-17T12:47:46.067155+0000","last_unstale":"2024-02-17T12:47:47.179528+0000","last_undegraded":"2024-02-17T12:47:47.179528+0000","last_fullsized":"2024-02-17T12:47:47.179528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:38:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180329+0000","last_change":"2024-02-17T12:47:46.062356+0000","last_active":"2024-02-17T12:47:47.180329+0000","last_peered":"2024-02-17T12:47:47.180329+0000","last_clean":"2024-02-17T12:47:47.180329+0000","last_became_active":"2024-02-17T12:47:46.061988+0000","last_became_peered":"2024-02-17T12:47:46.061988+0000","last_unstale":"2024-02-17T12:47:47.180329+0000","last_undegraded":"2024-02-17T12:47:47.180329+0000","last_fullsized":"2024-02-17T12:47:47.180329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:25:09.120770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243368+0000","last_change":"2024-02-17T12:47:46.072411+0000","last_active":"2024-02-17T12:47:47.243368+0000","last_peered":"2024-02-17T12:47:47.243368+0000","last_clean":"2024-02-17T12:47:47.243368+0000","last_became_active":"2024-02-17T12:47:46.072067+0000","last_became_peered":"2024-02-17T12:47:46.072067+0000","last_unstale":"2024-02-17T12:47:47.243368+0000","last_undegraded":"2024-02-17T12:47:47.243368+0000","last_fullsized":"2024-02-17T12:47:47.243368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:29:33.051513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179899+0000","last_change":"2024-02-17T12:47:46.057425+0000","last_active":"2024-02-17T12:47:47.179899+0000","last_peered":"2024-02-17T12:47:47.179899+0000","last_clean":"2024-02-17T12:47:47.179899+0000","last_became_active":"2024-02-17T12:47:46.057001+0000","last_became_peered":"2024-02-17T12:47:46.057001+0000","last_unstale":"2024-02-17T12:47:47.179899+0000","last_undegraded":"2024-02-17T12:47:47.179899+0000","last_fullsized":"2024-02-17T12:47:47.179899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:48:24.135879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179726+0000","last_change":"2024-02-17T12:47:46.069143+0000","last_active":"2024-02-17T12:47:47.179726+0000","last_peered":"2024-02-17T12:47:47.179726+0000","last_clean":"2024-02-17T12:47:47.179726+0000","last_became_active":"2024-02-17T12:47:46.068775+0000","last_became_peered":"2024-02-17T12:47:46.068775+0000","last_unstale":"2024-02-17T12:47:47.179726+0000","last_undegraded":"2024-02-17T12:47:47.179726+0000","last_fullsized":"2024-02-17T12:47:47.179726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:32:39.518205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065050+0000","last_change":"2024-02-17T12:47:46.069579+0000","last_active":"2024-02-17T12:47:47.065050+0000","last_peered":"2024-02-17T12:47:47.065050+0000","last_clean":"2024-02-17T12:47:47.065050+0000","last_became_active":"2024-02-17T12:47:46.069114+0000","last_became_peered":"2024-02-17T12:47:46.069114+0000","last_unstale":"2024-02-17T12:47:47.065050+0000","last_undegraded":"2024-02-17T12:47:47.065050+0000","last_fullsized":"2024-02-17T12:47:47.065050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:54:58.100511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066245+0000","last_change":"2024-02-17T12:47:46.079103+0000","last_active":"2024-02-17T12:47:47.066245+0000","last_peered":"2024-02-17T12:47:47.066245+0000","last_clean":"2024-02-17T12:47:47.066245+0000","last_became_active":"2024-02-17T12:47:46.077992+0000","last_became_peered":"2024-02-17T12:47:46.077992+0000","last_unstale":"2024-02-17T12:47:47.066245+0000","last_undegraded":"2024-02-17T12:47:47.066245+0000","last_fullsized":"2024-02-17T12:47:47.066245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:12:59.337341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180026+0000","last_change":"2024-02-17T12:47:46.062897+0000","last_active":"2024-02-17T12:47:47.180026+0000","last_peered":"2024-02-17T12:47:47.180026+0000","last_clean":"2024-02-17T12:47:47.180026+0000","last_became_active":"2024-02-17T12:47:46.062571+0000","last_became_peered":"2024-02-17T12:47:46.062571+0000","last_unstale":"2024-02-17T12:47:47.180026+0000","last_undegraded":"2024-02-17T12:47:47.180026+0000","last_fullsized":"2024-02-17T12:47:47.180026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:57:02.845400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180144+0000","last_change":"2024-02-17T12:47:46.080183+0000","last_active":"2024-02-17T12:47:47.180144+0000","last_peered":"2024-02-17T12:47:47.180144+0000","last_clean":"2024-02-17T12:47:47.180144+0000","last_became_active":"2024-02-17T12:47:46.079827+0000","last_became_peered":"2024-02-17T12:47:46.079827+0000","last_unstale":"2024-02-17T12:47:47.180144+0000","last_undegraded":"2024-02-17T12:47:47.180144+0000","last_fullsized":"2024-02-17T12:47:47.180144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:56:10.193200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180650+0000","last_change":"2024-02-17T12:47:46.061849+0000","last_active":"2024-02-17T12:47:47.180650+0000","last_peered":"2024-02-17T12:47:47.180650+0000","last_clean":"2024-02-17T12:47:47.180650+0000","last_became_active":"2024-02-17T12:47:46.061458+0000","last_became_peered":"2024-02-17T12:47:46.061458+0000","last_unstale":"2024-02-17T12:47:47.180650+0000","last_undegraded":"2024-02-17T12:47:47.180650+0000","last_fullsized":"2024-02-17T12:47:47.180650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:02:09.141292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179320+0000","last_change":"2024-02-17T12:47:46.069356+0000","last_active":"2024-02-17T12:47:47.179320+0000","last_peered":"2024-02-17T12:47:47.179320+0000","last_clean":"2024-02-17T12:47:47.179320+0000","last_became_active":"2024-02-17T12:47:46.068567+0000","last_became_peered":"2024-02-17T12:47:46.068567+0000","last_unstale":"2024-02-17T12:47:47.179320+0000","last_undegraded":"2024-02-17T12:47:47.179320+0000","last_fullsized":"2024-02-17T12:47:47.179320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:08:01.872558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179478+0000","last_change":"2024-02-17T12:47:46.062684+0000","last_active":"2024-02-17T12:47:47.179478+0000","last_peered":"2024-02-17T12:47:47.179478+0000","last_clean":"2024-02-17T12:47:47.179478+0000","last_became_active":"2024-02-17T12:47:46.062221+0000","last_became_peered":"2024-02-17T12:47:46.062221+0000","last_unstale":"2024-02-17T12:47:47.179478+0000","last_undegraded":"2024-02-17T12:47:47.179478+0000","last_fullsized":"2024-02-17T12:47:47.179478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:37:11.513336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179310+0000","last_change":"2024-02-17T12:47:46.065961+0000","last_active":"2024-02-17T12:47:47.179310+0000","last_peered":"2024-02-17T12:47:47.179310+0000","last_clean":"2024-02-17T12:47:47.179310+0000","last_became_active":"2024-02-17T12:47:46.065600+0000","last_became_peered":"2024-02-17T12:47:46.065600+0000","last_unstale":"2024-02-17T12:47:47.179310+0000","last_undegraded":"2024-02-17T12:47:47.179310+0000","last_fullsized":"2024-02-17T12:47:47.179310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:24:46.205077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066101+0000","last_change":"2024-02-17T12:47:46.080150+0000","last_active":"2024-02-17T12:47:47.066101+0000","last_peered":"2024-02-17T12:47:47.066101+0000","last_clean":"2024-02-17T12:47:47.066101+0000","last_became_active":"2024-02-17T12:47:46.079813+0000","last_became_peered":"2024-02-17T12:47:46.079813+0000","last_unstale":"2024-02-17T12:47:47.066101+0000","last_undegraded":"2024-02-17T12:47:47.066101+0000","last_fullsized":"2024-02-17T12:47:47.066101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:59:00.868964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180476+0000","last_change":"2024-02-17T12:47:46.070202+0000","last_active":"2024-02-17T12:47:47.180476+0000","last_peered":"2024-02-17T12:47:47.180476+0000","last_clean":"2024-02-17T12:47:47.180476+0000","last_became_active":"2024-02-17T12:47:46.069628+0000","last_became_peered":"2024-02-17T12:47:46.069628+0000","last_unstale":"2024-02-17T12:47:47.180476+0000","last_undegraded":"2024-02-17T12:47:47.180476+0000","last_fullsized":"2024-02-17T12:47:47.180476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:44:17.481681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.150613+0000","last_change":"2024-02-17T12:47:46.067810+0000","last_active":"2024-02-17T12:47:47.150613+0000","last_peered":"2024-02-17T12:47:47.150613+0000","last_clean":"2024-02-17T12:47:47.150613+0000","last_became_active":"2024-02-17T12:47:46.067456+0000","last_became_peered":"2024-02-17T12:47:46.067456+0000","last_unstale":"2024-02-17T12:47:47.150613+0000","last_undegraded":"2024-02-17T12:47:47.150613+0000","last_fullsized":"2024-02-17T12:47:47.150613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:27:03.646597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179813+0000","last_change":"2024-02-17T12:47:46.076268+0000","last_active":"2024-02-17T12:47:47.179813+0000","last_peered":"2024-02-17T12:47:47.179813+0000","last_clean":"2024-02-17T12:47:47.179813+0000","last_became_active":"2024-02-17T12:47:46.075606+0000","last_became_peered":"2024-02-17T12:47:46.075606+0000","last_unstale":"2024-02-17T12:47:47.179813+0000","last_undegraded":"2024-02-17T12:47:47.179813+0000","last_fullsized":"2024-02-17T12:47:47.179813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:16:35.805008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:50.725688+0000","last_change":"2024-02-17T12:47:45.066780+0000","last_active":"2024-02-17T12:47:50.725688+0000","last_peered":"2024-02-17T12:47:50.725688+0000","last_clean":"2024-02-17T12:47:50.725688+0000","last_became_active":"2024-02-17T12:47:45.066039+0000","last_became_peered":"2024-02-17T12:47:45.066039+0000","last_unstale":"2024-02-17T12:47:50.725688+0000","last_undegraded":"2024-02-17T12:47:50.725688+0000","last_fullsized":"2024-02-17T12:47:50.725688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:36:25.420534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065470+0000","last_change":"2024-02-17T12:47:46.079603+0000","last_active":"2024-02-17T12:47:47.065470+0000","last_peered":"2024-02-17T12:47:47.065470+0000","last_clean":"2024-02-17T12:47:47.065470+0000","last_became_active":"2024-02-17T12:47:46.079246+0000","last_became_peered":"2024-02-17T12:47:46.079246+0000","last_unstale":"2024-02-17T12:47:47.065470+0000","last_undegraded":"2024-02-17T12:47:47.065470+0000","last_fullsized":"2024-02-17T12:47:47.065470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:32:04.975778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065654+0000","last_change":"2024-02-17T12:47:45.061868+0000","last_active":"2024-02-17T12:47:47.065654+0000","last_peered":"2024-02-17T12:47:47.065654+0000","last_clean":"2024-02-17T12:47:47.065654+0000","last_became_active":"2024-02-17T12:47:45.061512+0000","last_became_peered":"2024-02-17T12:47:45.061512+0000","last_unstale":"2024-02-17T12:47:47.065654+0000","last_undegraded":"2024-02-17T12:47:47.065654+0000","last_fullsized":"2024-02-17T12:47:47.065654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:53:48.405184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179664+0000","last_change":"2024-02-17T12:47:46.071055+0000","last_active":"2024-02-17T12:47:47.179664+0000","last_peered":"2024-02-17T12:47:47.179664+0000","last_clean":"2024-02-17T12:47:47.179664+0000","last_became_active":"2024-02-17T12:47:46.070649+0000","last_became_peered":"2024-02-17T12:47:46.070649+0000","last_unstale":"2024-02-17T12:47:47.179664+0000","last_undegraded":"2024-02-17T12:47:47.179664+0000","last_fullsized":"2024-02-17T12:47:47.179664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:10:22.078746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243694+0000","last_change":"2024-02-17T12:47:45.066292+0000","last_active":"2024-02-17T12:47:47.243694+0000","last_peered":"2024-02-17T12:47:47.243694+0000","last_clean":"2024-02-17T12:47:47.243694+0000","last_became_active":"2024-02-17T12:47:45.065839+0000","last_became_peered":"2024-02-17T12:47:45.065839+0000","last_unstale":"2024-02-17T12:47:47.243694+0000","last_undegraded":"2024-02-17T12:47:47.243694+0000","last_fullsized":"2024-02-17T12:47:47.243694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:12:57.076774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066291+0000","last_change":"2024-02-17T12:47:46.076448+0000","last_active":"2024-02-17T12:47:47.066291+0000","last_peered":"2024-02-17T12:47:47.066291+0000","last_clean":"2024-02-17T12:47:47.066291+0000","last_became_active":"2024-02-17T12:47:46.075250+0000","last_became_peered":"2024-02-17T12:47:46.075250+0000","last_unstale":"2024-02-17T12:47:47.066291+0000","last_undegraded":"2024-02-17T12:47:47.066291+0000","last_fullsized":"2024-02-17T12:47:47.066291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:11:45.846764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243557+0000","last_change":"2024-02-17T12:47:45.062699+0000","last_active":"2024-02-17T12:47:47.243557+0000","last_peered":"2024-02-17T12:47:47.243557+0000","last_clean":"2024-02-17T12:47:47.243557+0000","last_became_active":"2024-02-17T12:47:45.062322+0000","last_became_peered":"2024-02-17T12:47:45.062322+0000","last_unstale":"2024-02-17T12:47:47.243557+0000","last_undegraded":"2024-02-17T12:47:47.243557+0000","last_fullsized":"2024-02-17T12:47:47.243557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:14:06.112393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066193+0000","last_change":"2024-02-17T12:47:46.080278+0000","last_active":"2024-02-17T12:47:47.066193+0000","last_peered":"2024-02-17T12:47:47.066193+0000","last_clean":"2024-02-17T12:47:47.066193+0000","last_became_active":"2024-02-17T12:47:46.079924+0000","last_became_peered":"2024-02-17T12:47:46.079924+0000","last_unstale":"2024-02-17T12:47:47.066193+0000","last_undegraded":"2024-02-17T12:47:47.066193+0000","last_fullsized":"2024-02-17T12:47:47.066193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:54:08.759445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.705945+0000","last_change":"2024-02-17T12:47:45.058098+0000","last_active":"2024-02-17T12:47:48.705945+0000","last_peered":"2024-02-17T12:47:48.705945+0000","last_clean":"2024-02-17T12:47:48.705945+0000","last_became_active":"2024-02-17T12:47:45.057763+0000","last_became_peered":"2024-02-17T12:47:45.057763+0000","last_unstale":"2024-02-17T12:47:48.705945+0000","last_undegraded":"2024-02-17T12:47:48.705945+0000","last_fullsized":"2024-02-17T12:47:48.705945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:26:36.739723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243347+0000","last_change":"2024-02-17T12:47:46.065957+0000","last_active":"2024-02-17T12:47:47.243347+0000","last_peered":"2024-02-17T12:47:47.243347+0000","last_clean":"2024-02-17T12:47:47.243347+0000","last_became_active":"2024-02-17T12:47:46.065638+0000","last_became_peered":"2024-02-17T12:47:46.065638+0000","last_unstale":"2024-02-17T12:47:47.243347+0000","last_undegraded":"2024-02-17T12:47:47.243347+0000","last_fullsized":"2024-02-17T12:47:47.243347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:02:39.295990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.711703+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:47:48.711703+0000","last_peered":"2024-02-17T12:47:48.711703+0000","last_clean":"2024-02-17T12:47:48.711703+0000","last_became_active":"2024-02-17T12:47:45.066235+0000","last_became_peered":"2024-02-17T12:47:45.066235+0000","last_unstale":"2024-02-17T12:47:48.711703+0000","last_undegraded":"2024-02-17T12:47:48.711703+0000","last_fullsized":"2024-02-17T12:47:48.711703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:46:53.731920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066083+0000","last_change":"2024-02-17T12:47:46.072029+0000","last_active":"2024-02-17T12:47:47.066083+0000","last_peered":"2024-02-17T12:47:47.066083+0000","last_clean":"2024-02-17T12:47:47.066083+0000","last_became_active":"2024-02-17T12:47:46.071716+0000","last_became_peered":"2024-02-17T12:47:46.071716+0000","last_unstale":"2024-02-17T12:47:47.066083+0000","last_undegraded":"2024-02-17T12:47:47.066083+0000","last_fullsized":"2024-02-17T12:47:47.066083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:33.076266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.713573+0000","last_change":"2024-02-17T12:47:45.067443+0000","last_active":"2024-02-17T12:47:48.713573+0000","last_peered":"2024-02-17T12:47:48.713573+0000","last_clean":"2024-02-17T12:47:48.713573+0000","last_became_active":"2024-02-17T12:47:45.067090+0000","last_became_peered":"2024-02-17T12:47:45.067090+0000","last_unstale":"2024-02-17T12:47:48.713573+0000","last_undegraded":"2024-02-17T12:47:48.713573+0000","last_fullsized":"2024-02-17T12:47:48.713573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:17:39.874810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065310+0000","last_change":"2024-02-17T12:47:46.072275+0000","last_active":"2024-02-17T12:47:47.065310+0000","last_peered":"2024-02-17T12:47:47.065310+0000","last_clean":"2024-02-17T12:47:47.065310+0000","last_became_active":"2024-02-17T12:47:46.071951+0000","last_became_peered":"2024-02-17T12:47:46.071951+0000","last_unstale":"2024-02-17T12:47:47.065310+0000","last_undegraded":"2024-02-17T12:47:47.065310+0000","last_fullsized":"2024-02-17T12:47:47.065310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:45:58.580854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065538+0000","last_change":"2024-02-17T12:47:45.058875+0000","last_active":"2024-02-17T12:47:47.065538+0000","last_peered":"2024-02-17T12:47:47.065538+0000","last_clean":"2024-02-17T12:47:47.065538+0000","last_became_active":"2024-02-17T12:47:45.058433+0000","last_became_peered":"2024-02-17T12:47:45.058433+0000","last_unstale":"2024-02-17T12:47:47.065538+0000","last_undegraded":"2024-02-17T12:47:47.065538+0000","last_fullsized":"2024-02-17T12:47:47.065538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:49.932205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.150333+0000","last_change":"2024-02-17T12:47:46.070233+0000","last_active":"2024-02-17T12:47:47.150333+0000","last_peered":"2024-02-17T12:47:47.150333+0000","last_clean":"2024-02-17T12:47:47.150333+0000","last_became_active":"2024-02-17T12:47:46.069811+0000","last_became_peered":"2024-02-17T12:47:46.069811+0000","last_unstale":"2024-02-17T12:47:47.150333+0000","last_undegraded":"2024-02-17T12:47:47.150333+0000","last_fullsized":"2024-02-17T12:47:47.150333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:50:00.885517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.084465+0000","last_change":"2024-02-17T12:47:45.068778+0000","last_active":"2024-02-17T12:47:48.084465+0000","last_peered":"2024-02-17T12:47:48.084465+0000","last_clean":"2024-02-17T12:47:48.084465+0000","last_became_active":"2024-02-17T12:47:45.068375+0000","last_became_peered":"2024-02-17T12:47:45.068375+0000","last_unstale":"2024-02-17T12:47:48.084465+0000","last_undegraded":"2024-02-17T12:47:48.084465+0000","last_fullsized":"2024-02-17T12:47:48.084465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:42:57.600138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066225+0000","last_change":"2024-02-17T12:47:46.076802+0000","last_active":"2024-02-17T12:47:47.066225+0000","last_peered":"2024-02-17T12:47:47.066225+0000","last_clean":"2024-02-17T12:47:47.066225+0000","last_became_active":"2024-02-17T12:47:46.076433+0000","last_became_peered":"2024-02-17T12:47:46.076433+0000","last_unstale":"2024-02-17T12:47:47.066225+0000","last_undegraded":"2024-02-17T12:47:47.066225+0000","last_fullsized":"2024-02-17T12:47:47.066225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.352488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.084520+0000","last_change":"2024-02-17T12:47:45.059100+0000","last_active":"2024-02-17T12:47:48.084520+0000","last_peered":"2024-02-17T12:47:48.084520+0000","last_clean":"2024-02-17T12:47:48.084520+0000","last_became_active":"2024-02-17T12:47:45.058707+0000","last_became_peered":"2024-02-17T12:47:45.058707+0000","last_unstale":"2024-02-17T12:47:48.084520+0000","last_undegraded":"2024-02-17T12:47:48.084520+0000","last_fullsized":"2024-02-17T12:47:48.084520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:28:27.044127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179710+0000","last_change":"2024-02-17T12:47:46.065088+0000","last_active":"2024-02-17T12:47:47.179710+0000","last_peered":"2024-02-17T12:47:47.179710+0000","last_clean":"2024-02-17T12:47:47.179710+0000","last_became_active":"2024-02-17T12:47:46.064754+0000","last_became_peered":"2024-02-17T12:47:46.064754+0000","last_unstale":"2024-02-17T12:47:47.179710+0000","last_undegraded":"2024-02-17T12:47:47.179710+0000","last_fullsized":"2024-02-17T12:47:47.179710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:33:10.652881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.084701+0000","last_change":"2024-02-17T12:47:45.055403+0000","last_active":"2024-02-17T12:47:48.084701+0000","last_peered":"2024-02-17T12:47:48.084701+0000","last_clean":"2024-02-17T12:47:48.084701+0000","last_became_active":"2024-02-17T12:47:45.055083+0000","last_became_peered":"2024-02-17T12:47:45.055083+0000","last_unstale":"2024-02-17T12:47:48.084701+0000","last_undegraded":"2024-02-17T12:47:48.084701+0000","last_fullsized":"2024-02-17T12:47:48.084701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:05:51.295074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066267+0000","last_change":"2024-02-17T12:47:46.074883+0000","last_active":"2024-02-17T12:47:47.066267+0000","last_peered":"2024-02-17T12:47:47.066267+0000","last_clean":"2024-02-17T12:47:47.066267+0000","last_became_active":"2024-02-17T12:47:46.074414+0000","last_became_peered":"2024-02-17T12:47:46.074414+0000","last_unstale":"2024-02-17T12:47:47.066267+0000","last_undegraded":"2024-02-17T12:47:47.066267+0000","last_fullsized":"2024-02-17T12:47:47.066267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:35:55.026587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.706730+0000","last_change":"2024-02-17T12:47:45.057627+0000","last_active":"2024-02-17T12:47:48.706730+0000","last_peered":"2024-02-17T12:47:48.706730+0000","last_clean":"2024-02-17T12:47:48.706730+0000","last_became_active":"2024-02-17T12:47:45.057297+0000","last_became_peered":"2024-02-17T12:47:45.057297+0000","last_unstale":"2024-02-17T12:47:48.706730+0000","last_undegraded":"2024-02-17T12:47:48.706730+0000","last_fullsized":"2024-02-17T12:47:48.706730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:00:17.159939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243521+0000","last_change":"2024-02-17T12:47:46.076854+0000","last_active":"2024-02-17T12:47:47.243521+0000","last_peered":"2024-02-17T12:47:47.243521+0000","last_clean":"2024-02-17T12:47:47.243521+0000","last_became_active":"2024-02-17T12:47:46.076534+0000","last_became_peered":"2024-02-17T12:47:46.076534+0000","last_unstale":"2024-02-17T12:47:47.243521+0000","last_undegraded":"2024-02-17T12:47:47.243521+0000","last_fullsized":"2024-02-17T12:47:47.243521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:43:01.979075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179714+0000","last_change":"2024-02-17T12:47:45.060090+0000","last_active":"2024-02-17T12:47:47.179714+0000","last_peered":"2024-02-17T12:47:47.179714+0000","last_clean":"2024-02-17T12:47:47.179714+0000","last_became_active":"2024-02-17T12:47:45.058942+0000","last_became_peered":"2024-02-17T12:47:45.058942+0000","last_unstale":"2024-02-17T12:47:47.179714+0000","last_undegraded":"2024-02-17T12:47:47.179714+0000","last_fullsized":"2024-02-17T12:47:47.179714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:30:52.247140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179092+0000","last_change":"2024-02-17T12:47:46.071982+0000","last_active":"2024-02-17T12:47:47.179092+0000","last_peered":"2024-02-17T12:47:47.179092+0000","last_clean":"2024-02-17T12:47:47.179092+0000","last_became_active":"2024-02-17T12:47:46.071639+0000","last_became_peered":"2024-02-17T12:47:46.071639+0000","last_unstale":"2024-02-17T12:47:47.179092+0000","last_undegraded":"2024-02-17T12:47:47.179092+0000","last_fullsized":"2024-02-17T12:47:47.179092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:43:50.658132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.084214+0000","last_change":"2024-02-17T12:47:45.060017+0000","last_active":"2024-02-17T12:47:48.084214+0000","last_peered":"2024-02-17T12:47:48.084214+0000","last_clean":"2024-02-17T12:47:48.084214+0000","last_became_active":"2024-02-17T12:47:45.059104+0000","last_became_peered":"2024-02-17T12:47:45.059104+0000","last_unstale":"2024-02-17T12:47:48.084214+0000","last_undegraded":"2024-02-17T12:47:48.084214+0000","last_fullsized":"2024-02-17T12:47:48.084214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:56:50.457297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180374+0000","last_change":"2024-02-17T12:47:46.072250+0000","last_active":"2024-02-17T12:47:47.180374+0000","last_peered":"2024-02-17T12:47:47.180374+0000","last_clean":"2024-02-17T12:47:47.180374+0000","last_became_active":"2024-02-17T12:47:46.071856+0000","last_became_peered":"2024-02-17T12:47:46.071856+0000","last_unstale":"2024-02-17T12:47:47.180374+0000","last_undegraded":"2024-02-17T12:47:47.180374+0000","last_fullsized":"2024-02-17T12:47:47.180374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:46:04.628146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:49.717914+0000","last_change":"2024-02-17T12:47:45.056386+0000","last_active":"2024-02-17T12:47:49.717914+0000","last_peered":"2024-02-17T12:47:49.717914+0000","last_clean":"2024-02-17T12:47:49.717914+0000","last_became_active":"2024-02-17T12:47:45.055630+0000","last_became_peered":"2024-02-17T12:47:45.055630+0000","last_unstale":"2024-02-17T12:47:49.717914+0000","last_undegraded":"2024-02-17T12:47:49.717914+0000","last_fullsized":"2024-02-17T12:47:49.717914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:16.388950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179359+0000","last_change":"2024-02-17T12:47:46.071612+0000","last_active":"2024-02-17T12:47:47.179359+0000","last_peered":"2024-02-17T12:47:47.179359+0000","last_clean":"2024-02-17T12:47:47.179359+0000","last_became_active":"2024-02-17T12:47:46.071237+0000","last_became_peered":"2024-02-17T12:47:46.071237+0000","last_unstale":"2024-02-17T12:47:47.179359+0000","last_undegraded":"2024-02-17T12:47:47.179359+0000","last_fullsized":"2024-02-17T12:47:47.179359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:24:01.099679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'10","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.083192+0000","last_change":"2024-02-17T12:47:45.066258+0000","last_active":"2024-02-17T12:47:48.083192+0000","last_peered":"2024-02-17T12:47:48.083192+0000","last_clean":"2024-02-17T12:47:48.083192+0000","last_became_active":"2024-02-17T12:47:45.065813+0000","last_became_peered":"2024-02-17T12:47:45.065813+0000","last_unstale":"2024-02-17T12:47:48.083192+0000","last_undegraded":"2024-02-17T12:47:48.083192+0000","last_fullsized":"2024-02-17T12:47:48.083192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:06:43.298573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065640+0000","last_change":"2024-02-17T12:47:46.080347+0000","last_active":"2024-02-17T12:47:47.065640+0000","last_peered":"2024-02-17T12:47:47.065640+0000","last_clean":"2024-02-17T12:47:47.065640+0000","last_became_active":"2024-02-17T12:47:46.080083+0000","last_became_peered":"2024-02-17T12:47:46.080083+0000","last_unstale":"2024-02-17T12:47:47.065640+0000","last_undegraded":"2024-02-17T12:47:47.065640+0000","last_fullsized":"2024-02-17T12:47:47.065640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:24:41.807633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243639+0000","last_change":"2024-02-17T12:47:45.062814+0000","last_active":"2024-02-17T12:47:47.243639+0000","last_peered":"2024-02-17T12:47:47.243639+0000","last_clean":"2024-02-17T12:47:47.243639+0000","last_became_active":"2024-02-17T12:47:45.062418+0000","last_became_peered":"2024-02-17T12:47:45.062418+0000","last_unstale":"2024-02-17T12:47:47.243639+0000","last_undegraded":"2024-02-17T12:47:47.243639+0000","last_fullsized":"2024-02-17T12:47:47.243639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:49:34.577275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180380+0000","last_change":"2024-02-17T12:47:46.067742+0000","last_active":"2024-02-17T12:47:47.180380+0000","last_peered":"2024-02-17T12:47:47.180380+0000","last_clean":"2024-02-17T12:47:47.180380+0000","last_became_active":"2024-02-17T12:47:46.067329+0000","last_became_peered":"2024-02-17T12:47:46.067329+0000","last_unstale":"2024-02-17T12:47:47.180380+0000","last_undegraded":"2024-02-17T12:47:47.180380+0000","last_fullsized":"2024-02-17T12:47:47.180380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:19.432999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.150534+0000","last_change":"2024-02-17T12:47:45.065461+0000","last_active":"2024-02-17T12:47:47.150534+0000","last_peered":"2024-02-17T12:47:47.150534+0000","last_clean":"2024-02-17T12:47:47.150534+0000","last_became_active":"2024-02-17T12:47:45.065066+0000","last_became_peered":"2024-02-17T12:47:45.065066+0000","last_unstale":"2024-02-17T12:47:47.150534+0000","last_undegraded":"2024-02-17T12:47:47.150534+0000","last_fullsized":"2024-02-17T12:47:47.150534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:25:47.292901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180138+0000","last_change":"2024-02-17T12:47:46.074222+0000","last_active":"2024-02-17T12:47:47.180138+0000","last_peered":"2024-02-17T12:47:47.180138+0000","last_clean":"2024-02-17T12:47:47.180138+0000","last_became_active":"2024-02-17T12:47:46.073886+0000","last_became_peered":"2024-02-17T12:47:46.073886+0000","last_unstale":"2024-02-17T12:47:47.180138+0000","last_undegraded":"2024-02-17T12:47:47.180138+0000","last_fullsized":"2024-02-17T12:47:47.180138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:46:18.063422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180601+0000","last_change":"2024-02-17T12:47:45.071414+0000","last_active":"2024-02-17T12:47:47.180601+0000","last_peered":"2024-02-17T12:47:47.180601+0000","last_clean":"2024-02-17T12:47:47.180601+0000","last_became_active":"2024-02-17T12:47:45.071127+0000","last_became_peered":"2024-02-17T12:47:45.071127+0000","last_unstale":"2024-02-17T12:47:47.180601+0000","last_undegraded":"2024-02-17T12:47:47.180601+0000","last_fullsized":"2024-02-17T12:47:47.180601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:13:21.187718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179825+0000","last_change":"2024-02-17T12:47:46.059412+0000","last_active":"2024-02-17T12:47:47.179825+0000","last_peered":"2024-02-17T12:47:47.179825+0000","last_clean":"2024-02-17T12:47:47.179825+0000","last_became_active":"2024-02-17T12:47:46.059100+0000","last_became_peered":"2024-02-17T12:47:46.059100+0000","last_unstale":"2024-02-17T12:47:47.179825+0000","last_undegraded":"2024-02-17T12:47:47.179825+0000","last_fullsized":"2024-02-17T12:47:47.179825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:35.930613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065990+0000","last_change":"2024-02-17T12:47:46.071467+0000","last_active":"2024-02-17T12:47:47.065990+0000","last_peered":"2024-02-17T12:47:47.065990+0000","last_clean":"2024-02-17T12:47:47.065990+0000","last_became_active":"2024-02-17T12:47:46.070879+0000","last_became_peered":"2024-02-17T12:47:46.070879+0000","last_unstale":"2024-02-17T12:47:47.065990+0000","last_undegraded":"2024-02-17T12:47:47.065990+0000","last_fullsized":"2024-02-17T12:47:47.065990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.682842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.714859+0000","last_change":"2024-02-17T12:47:45.070545+0000","last_active":"2024-02-17T12:47:48.714859+0000","last_peered":"2024-02-17T12:47:48.714859+0000","last_clean":"2024-02-17T12:47:48.714859+0000","last_became_active":"2024-02-17T12:47:45.070265+0000","last_became_peered":"2024-02-17T12:47:45.070265+0000","last_unstale":"2024-02-17T12:47:48.714859+0000","last_undegraded":"2024-02-17T12:47:48.714859+0000","last_fullsized":"2024-02-17T12:47:48.714859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.763735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065446+0000","last_change":"2024-02-17T12:47:46.068539+0000","last_active":"2024-02-17T12:47:47.065446+0000","last_peered":"2024-02-17T12:47:47.065446+0000","last_clean":"2024-02-17T12:47:47.065446+0000","last_became_active":"2024-02-17T12:47:46.068169+0000","last_became_peered":"2024-02-17T12:47:46.068169+0000","last_unstale":"2024-02-17T12:47:47.065446+0000","last_undegraded":"2024-02-17T12:47:47.065446+0000","last_fullsized":"2024-02-17T12:47:47.065446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:27:28.485868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:49.718563+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:47:49.718563+0000","last_peered":"2024-02-17T12:47:49.718563+0000","last_clean":"2024-02-17T12:47:49.718563+0000","last_became_active":"2024-02-17T12:47:45.066299+0000","last_became_peered":"2024-02-17T12:47:45.066299+0000","last_unstale":"2024-02-17T12:47:49.718563+0000","last_undegraded":"2024-02-17T12:47:49.718563+0000","last_fullsized":"2024-02-17T12:47:49.718563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:34:02.040002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180075+0000","last_change":"2024-02-17T12:46:32.556576+0000","last_active":"2024-02-17T12:47:47.180075+0000","last_peered":"2024-02-17T12:47:47.180075+0000","last_clean":"2024-02-17T12:47:47.180075+0000","last_became_active":"2024-02-17T12:46:32.556160+0000","last_became_peered":"2024-02-17T12:46:32.556160+0000","last_unstale":"2024-02-17T12:47:47.180075+0000","last_undegraded":"2024-02-17T12:47:47.180075+0000","last_fullsized":"2024-02-17T12:47:47.180075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_clean_scrub_stamp":"2024-02-17T12:43:13.147248+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:57:52.244951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.178706+0000","last_change":"2024-02-17T12:47:46.073254+0000","last_active":"2024-02-17T12:47:47.178706+0000","last_peered":"2024-02-17T12:47:47.178706+0000","last_clean":"2024-02-17T12:47:47.178706+0000","last_became_active":"2024-02-17T12:47:46.072925+0000","last_became_peered":"2024-02-17T12:47:46.072925+0000","last_unstale":"2024-02-17T12:47:47.178706+0000","last_undegraded":"2024-02-17T12:47:47.178706+0000","last_fullsized":"2024-02-17T12:47:47.178706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:01:27.498671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.711138+0000","last_change":"2024-02-17T12:47:45.061112+0000","last_active":"2024-02-17T12:47:48.711138+0000","last_peered":"2024-02-17T12:47:48.711138+0000","last_clean":"2024-02-17T12:47:48.711138+0000","last_became_active":"2024-02-17T12:47:45.060633+0000","last_became_peered":"2024-02-17T12:47:45.060633+0000","last_unstale":"2024-02-17T12:47:48.711138+0000","last_undegraded":"2024-02-17T12:47:48.711138+0000","last_fullsized":"2024-02-17T12:47:48.711138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:29:29.421744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.082115+0000","last_change":"2024-02-17T12:47:45.066897+0000","last_active":"2024-02-17T12:47:48.082115+0000","last_peered":"2024-02-17T12:47:48.082115+0000","last_clean":"2024-02-17T12:47:48.082115+0000","last_became_active":"2024-02-17T12:47:45.066357+0000","last_became_peered":"2024-02-17T12:47:45.066357+0000","last_unstale":"2024-02-17T12:47:48.082115+0000","last_undegraded":"2024-02-17T12:47:48.082115+0000","last_fullsized":"2024-02-17T12:47:48.082115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:50:09.840414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.178844+0000","last_change":"2024-02-17T12:47:46.072412+0000","last_active":"2024-02-17T12:47:47.178844+0000","last_peered":"2024-02-17T12:47:47.178844+0000","last_clean":"2024-02-17T12:47:47.178844+0000","last_became_active":"2024-02-17T12:47:46.072105+0000","last_became_peered":"2024-02-17T12:47:46.072105+0000","last_unstale":"2024-02-17T12:47:47.178844+0000","last_undegraded":"2024-02-17T12:47:47.178844+0000","last_fullsized":"2024-02-17T12:47:47.178844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:33:19.367588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:49.719949+0000","last_change":"2024-02-17T12:47:45.067673+0000","last_active":"2024-02-17T12:47:49.719949+0000","last_peered":"2024-02-17T12:47:49.719949+0000","last_clean":"2024-02-17T12:47:49.719949+0000","last_became_active":"2024-02-17T12:47:45.067212+0000","last_became_peered":"2024-02-17T12:47:45.067212+0000","last_unstale":"2024-02-17T12:47:49.719949+0000","last_undegraded":"2024-02-17T12:47:49.719949+0000","last_fullsized":"2024-02-17T12:47:49.719949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:15:04.236158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065634+0000","last_change":"2024-02-17T12:47:46.077476+0000","last_active":"2024-02-17T12:47:47.065634+0000","last_peered":"2024-02-17T12:47:47.065634+0000","last_clean":"2024-02-17T12:47:47.065634+0000","last_became_active":"2024-02-17T12:47:46.077101+0000","last_became_peered":"2024-02-17T12:47:46.077101+0000","last_unstale":"2024-02-17T12:47:47.065634+0000","last_undegraded":"2024-02-17T12:47:47.065634+0000","last_fullsized":"2024-02-17T12:47:47.065634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:41:41.402101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:50.724191+0000","last_change":"2024-02-17T12:47:45.066693+0000","last_active":"2024-02-17T12:47:50.724191+0000","last_peered":"2024-02-17T12:47:50.724191+0000","last_clean":"2024-02-17T12:47:50.724191+0000","last_became_active":"2024-02-17T12:47:45.066349+0000","last_became_peered":"2024-02-17T12:47:45.066349+0000","last_unstale":"2024-02-17T12:47:50.724191+0000","last_undegraded":"2024-02-17T12:47:50.724191+0000","last_fullsized":"2024-02-17T12:47:50.724191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:30.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179783+0000","last_change":"2024-02-17T12:47:46.072899+0000","last_active":"2024-02-17T12:47:47.179783+0000","last_peered":"2024-02-17T12:47:47.179783+0000","last_clean":"2024-02-17T12:47:47.179783+0000","last_became_active":"2024-02-17T12:47:46.072488+0000","last_became_peered":"2024-02-17T12:47:46.072488+0000","last_unstale":"2024-02-17T12:47:47.179783+0000","last_undegraded":"2024-02-17T12:47:47.179783+0000","last_fullsized":"2024-02-17T12:47:47.179783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:05:43.675086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.084158+0000","last_change":"2024-02-17T12:47:45.056995+0000","last_active":"2024-02-17T12:47:48.084158+0000","last_peered":"2024-02-17T12:47:48.084158+0000","last_clean":"2024-02-17T12:47:48.084158+0000","last_became_active":"2024-02-17T12:47:45.056021+0000","last_became_peered":"2024-02-17T12:47:45.056021+0000","last_unstale":"2024-02-17T12:47:48.084158+0000","last_undegraded":"2024-02-17T12:47:48.084158+0000","last_fullsized":"2024-02-17T12:47:48.084158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:41.444513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.243235+0000","last_change":"2024-02-17T12:47:46.065859+0000","last_active":"2024-02-17T12:47:47.243235+0000","last_peered":"2024-02-17T12:47:47.243235+0000","last_clean":"2024-02-17T12:47:47.243235+0000","last_became_active":"2024-02-17T12:47:46.065469+0000","last_became_peered":"2024-02-17T12:47:46.065469+0000","last_unstale":"2024-02-17T12:47:47.243235+0000","last_undegraded":"2024-02-17T12:47:47.243235+0000","last_fullsized":"2024-02-17T12:47:47.243235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:51:25.105361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:49.719022+0000","last_change":"2024-02-17T12:47:45.066524+0000","last_active":"2024-02-17T12:47:49.719022+0000","last_peered":"2024-02-17T12:47:49.719022+0000","last_clean":"2024-02-17T12:47:49.719022+0000","last_became_active":"2024-02-17T12:47:45.066218+0000","last_became_peered":"2024-02-17T12:47:45.066218+0000","last_unstale":"2024-02-17T12:47:49.719022+0000","last_undegraded":"2024-02-17T12:47:49.719022+0000","last_fullsized":"2024-02-17T12:47:49.719022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:56:47.244262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179664+0000","last_change":"2024-02-17T12:47:46.065779+0000","last_active":"2024-02-17T12:47:47.179664+0000","last_peered":"2024-02-17T12:47:47.179664+0000","last_clean":"2024-02-17T12:47:47.179664+0000","last_became_active":"2024-02-17T12:47:46.065459+0000","last_became_peered":"2024-02-17T12:47:46.065459+0000","last_unstale":"2024-02-17T12:47:47.179664+0000","last_undegraded":"2024-02-17T12:47:47.179664+0000","last_fullsized":"2024-02-17T12:47:47.179664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:08.944487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179624+0000","last_change":"2024-02-17T12:47:45.059639+0000","last_active":"2024-02-17T12:47:47.179624+0000","last_peered":"2024-02-17T12:47:47.179624+0000","last_clean":"2024-02-17T12:47:47.179624+0000","last_became_active":"2024-02-17T12:47:45.059085+0000","last_became_peered":"2024-02-17T12:47:45.059085+0000","last_unstale":"2024-02-17T12:47:47.179624+0000","last_undegraded":"2024-02-17T12:47:47.179624+0000","last_fullsized":"2024-02-17T12:47:47.179624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:48:55.847405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065248+0000","last_change":"2024-02-17T12:47:46.079623+0000","last_active":"2024-02-17T12:47:47.065248+0000","last_peered":"2024-02-17T12:47:47.065248+0000","last_clean":"2024-02-17T12:47:47.065248+0000","last_became_active":"2024-02-17T12:47:46.079285+0000","last_became_peered":"2024-02-17T12:47:46.079285+0000","last_unstale":"2024-02-17T12:47:47.065248+0000","last_undegraded":"2024-02-17T12:47:47.065248+0000","last_fullsized":"2024-02-17T12:47:47.065248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:45:14.441218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:48.084372+0000","last_change":"2024-02-17T12:47:45.064935+0000","last_active":"2024-02-17T12:47:48.084372+0000","last_peered":"2024-02-17T12:47:48.084372+0000","last_clean":"2024-02-17T12:47:48.084372+0000","last_became_active":"2024-02-17T12:47:45.064592+0000","last_became_peered":"2024-02-17T12:47:45.064592+0000","last_unstale":"2024-02-17T12:47:48.084372+0000","last_undegraded":"2024-02-17T12:47:48.084372+0000","last_fullsized":"2024-02-17T12:47:48.084372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:36.943316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180756+0000","last_change":"2024-02-17T12:47:46.074429+0000","last_active":"2024-02-17T12:47:47.180756+0000","last_peered":"2024-02-17T12:47:47.180756+0000","last_clean":"2024-02-17T12:47:47.180756+0000","last_became_active":"2024-02-17T12:47:46.074123+0000","last_became_peered":"2024-02-17T12:47:46.074123+0000","last_unstale":"2024-02-17T12:47:47.180756+0000","last_undegraded":"2024-02-17T12:47:47.180756+0000","last_fullsized":"2024-02-17T12:47:47.180756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:50:17.633831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179942+0000","last_change":"2024-02-17T12:47:45.061362+0000","last_active":"2024-02-17T12:47:47.179942+0000","last_peered":"2024-02-17T12:47:47.179942+0000","last_clean":"2024-02-17T12:47:47.179942+0000","last_became_active":"2024-02-17T12:47:45.061037+0000","last_became_peered":"2024-02-17T12:47:45.061037+0000","last_unstale":"2024-02-17T12:47:47.179942+0000","last_undegraded":"2024-02-17T12:47:47.179942+0000","last_fullsized":"2024-02-17T12:47:47.179942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:01.987157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180582+0000","last_change":"2024-02-17T12:47:46.073862+0000","last_active":"2024-02-17T12:47:47.180582+0000","last_peered":"2024-02-17T12:47:47.180582+0000","last_clean":"2024-02-17T12:47:47.180582+0000","last_became_active":"2024-02-17T12:47:46.073543+0000","last_became_peered":"2024-02-17T12:47:46.073543+0000","last_unstale":"2024-02-17T12:47:47.180582+0000","last_undegraded":"2024-02-17T12:47:47.180582+0000","last_fullsized":"2024-02-17T12:47:47.180582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:57:57.567726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065963+0000","last_change":"2024-02-17T12:47:45.060616+0000","last_active":"2024-02-17T12:47:47.065963+0000","last_peered":"2024-02-17T12:47:47.065963+0000","last_clean":"2024-02-17T12:47:47.065963+0000","last_became_active":"2024-02-17T12:47:45.060296+0000","last_became_peered":"2024-02-17T12:47:45.060296+0000","last_unstale":"2024-02-17T12:47:47.065963+0000","last_undegraded":"2024-02-17T12:47:47.065963+0000","last_fullsized":"2024-02-17T12:47:47.065963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:22:44.791363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179670+0000","last_change":"2024-02-17T12:47:46.072738+0000","last_active":"2024-02-17T12:47:47.179670+0000","last_peered":"2024-02-17T12:47:47.179670+0000","last_clean":"2024-02-17T12:47:47.179670+0000","last_became_active":"2024-02-17T12:47:46.072421+0000","last_became_peered":"2024-02-17T12:47:46.072421+0000","last_unstale":"2024-02-17T12:47:47.179670+0000","last_undegraded":"2024-02-17T12:47:47.179670+0000","last_fullsized":"2024-02-17T12:47:47.179670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:50:23.501787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179040+0000","last_change":"2024-02-17T12:47:45.071201+0000","last_active":"2024-02-17T12:47:47.179040+0000","last_peered":"2024-02-17T12:47:47.179040+0000","last_clean":"2024-02-17T12:47:47.179040+0000","last_became_active":"2024-02-17T12:47:45.070777+0000","last_became_peered":"2024-02-17T12:47:45.070777+0000","last_unstale":"2024-02-17T12:47:47.179040+0000","last_undegraded":"2024-02-17T12:47:47.179040+0000","last_fullsized":"2024-02-17T12:47:47.179040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:14:02.807288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180622+0000","last_change":"2024-02-17T12:47:46.066072+0000","last_active":"2024-02-17T12:47:47.180622+0000","last_peered":"2024-02-17T12:47:47.180622+0000","last_clean":"2024-02-17T12:47:47.180622+0000","last_became_active":"2024-02-17T12:47:46.065755+0000","last_became_peered":"2024-02-17T12:47:46.065755+0000","last_unstale":"2024-02-17T12:47:47.180622+0000","last_undegraded":"2024-02-17T12:47:47.180622+0000","last_fullsized":"2024-02-17T12:47:47.180622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:09:29.534097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179984+0000","last_change":"2024-02-17T12:47:46.067268+0000","last_active":"2024-02-17T12:47:47.179984+0000","last_peered":"2024-02-17T12:47:47.179984+0000","last_clean":"2024-02-17T12:47:47.179984+0000","last_became_active":"2024-02-17T12:47:46.066918+0000","last_became_peered":"2024-02-17T12:47:46.066918+0000","last_unstale":"2024-02-17T12:47:47.179984+0000","last_undegraded":"2024-02-17T12:47:47.179984+0000","last_fullsized":"2024-02-17T12:47:47.179984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:11:09.844954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180012+0000","last_change":"2024-02-17T12:47:46.068143+0000","last_active":"2024-02-17T12:47:47.180012+0000","last_peered":"2024-02-17T12:47:47.180012+0000","last_clean":"2024-02-17T12:47:47.180012+0000","last_became_active":"2024-02-17T12:47:46.065112+0000","last_became_peered":"2024-02-17T12:47:46.065112+0000","last_unstale":"2024-02-17T12:47:47.180012+0000","last_undegraded":"2024-02-17T12:47:47.180012+0000","last_fullsized":"2024-02-17T12:47:47.180012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:43:04.982009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179804+0000","last_change":"2024-02-17T12:47:46.076139+0000","last_active":"2024-02-17T12:47:47.179804+0000","last_peered":"2024-02-17T12:47:47.179804+0000","last_clean":"2024-02-17T12:47:47.179804+0000","last_became_active":"2024-02-17T12:47:46.075826+0000","last_became_peered":"2024-02-17T12:47:46.075826+0000","last_unstale":"2024-02-17T12:47:47.179804+0000","last_undegraded":"2024-02-17T12:47:47.179804+0000","last_fullsized":"2024-02-17T12:47:47.179804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:39:08.460716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179700+0000","last_change":"2024-02-17T12:47:46.063792+0000","last_active":"2024-02-17T12:47:47.179700+0000","last_peered":"2024-02-17T12:47:47.179700+0000","last_clean":"2024-02-17T12:47:47.179700+0000","last_became_active":"2024-02-17T12:47:46.063407+0000","last_became_peered":"2024-02-17T12:47:46.063407+0000","last_unstale":"2024-02-17T12:47:47.179700+0000","last_undegraded":"2024-02-17T12:47:47.179700+0000","last_fullsized":"2024-02-17T12:47:47.179700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:26:26.973435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.178899+0000","last_change":"2024-02-17T12:47:46.066202+0000","last_active":"2024-02-17T12:47:47.178899+0000","last_peered":"2024-02-17T12:47:47.178899+0000","last_clean":"2024-02-17T12:47:47.178899+0000","last_became_active":"2024-02-17T12:47:46.065820+0000","last_became_peered":"2024-02-17T12:47:46.065820+0000","last_unstale":"2024-02-17T12:47:47.178899+0000","last_undegraded":"2024-02-17T12:47:47.178899+0000","last_fullsized":"2024-02-17T12:47:47.178899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:36:25.201697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180376+0000","last_change":"2024-02-17T12:47:46.065092+0000","last_active":"2024-02-17T12:47:47.180376+0000","last_peered":"2024-02-17T12:47:47.180376+0000","last_clean":"2024-02-17T12:47:47.180376+0000","last_became_active":"2024-02-17T12:47:46.064710+0000","last_became_peered":"2024-02-17T12:47:46.064710+0000","last_unstale":"2024-02-17T12:47:47.180376+0000","last_undegraded":"2024-02-17T12:47:47.180376+0000","last_fullsized":"2024-02-17T12:47:47.180376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:00:13.715042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180239+0000","last_change":"2024-02-17T12:47:46.070811+0000","last_active":"2024-02-17T12:47:47.180239+0000","last_peered":"2024-02-17T12:47:47.180239+0000","last_clean":"2024-02-17T12:47:47.180239+0000","last_became_active":"2024-02-17T12:47:46.070462+0000","last_became_peered":"2024-02-17T12:47:46.070462+0000","last_unstale":"2024-02-17T12:47:47.180239+0000","last_undegraded":"2024-02-17T12:47:47.180239+0000","last_fullsized":"2024-02-17T12:47:47.180239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:03.671869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065737+0000","last_change":"2024-02-17T12:47:46.077692+0000","last_active":"2024-02-17T12:47:47.065737+0000","last_peered":"2024-02-17T12:47:47.065737+0000","last_clean":"2024-02-17T12:47:47.065737+0000","last_became_active":"2024-02-17T12:47:46.077363+0000","last_became_peered":"2024-02-17T12:47:46.077363+0000","last_unstale":"2024-02-17T12:47:47.065737+0000","last_undegraded":"2024-02-17T12:47:47.065737+0000","last_fullsized":"2024-02-17T12:47:47.065737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:02:49.068782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065666+0000","last_change":"2024-02-17T12:47:46.073507+0000","last_active":"2024-02-17T12:47:47.065666+0000","last_peered":"2024-02-17T12:47:47.065666+0000","last_clean":"2024-02-17T12:47:47.065666+0000","last_became_active":"2024-02-17T12:47:46.073107+0000","last_became_peered":"2024-02-17T12:47:46.073107+0000","last_unstale":"2024-02-17T12:47:47.065666+0000","last_undegraded":"2024-02-17T12:47:47.065666+0000","last_fullsized":"2024-02-17T12:47:47.065666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:32:48.799300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.065077+0000","last_change":"2024-02-17T12:47:46.073384+0000","last_active":"2024-02-17T12:47:47.065077+0000","last_peered":"2024-02-17T12:47:47.065077+0000","last_clean":"2024-02-17T12:47:47.065077+0000","last_became_active":"2024-02-17T12:47:46.073017+0000","last_became_peered":"2024-02-17T12:47:46.073017+0000","last_unstale":"2024-02-17T12:47:47.065077+0000","last_undegraded":"2024-02-17T12:47:47.065077+0000","last_fullsized":"2024-02-17T12:47:47.065077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:43:50.091667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.066154+0000","last_change":"2024-02-17T12:47:46.073758+0000","last_active":"2024-02-17T12:47:47.066154+0000","last_peered":"2024-02-17T12:47:47.066154+0000","last_clean":"2024-02-17T12:47:47.066154+0000","last_became_active":"2024-02-17T12:47:46.072781+0000","last_became_peered":"2024-02-17T12:47:46.072781+0000","last_unstale":"2024-02-17T12:47:47.066154+0000","last_undegraded":"2024-02-17T12:47:47.066154+0000","last_fullsized":"2024-02-17T12:47:47.066154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:10:24.811100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.179319+0000","last_change":"2024-02-17T12:47:46.072025+0000","last_active":"2024-02-17T12:47:47.179319+0000","last_peered":"2024-02-17T12:47:47.179319+0000","last_clean":"2024-02-17T12:47:47.179319+0000","last_became_active":"2024-02-17T12:47:46.071662+0000","last_became_peered":"2024-02-17T12:47:46.071662+0000","last_unstale":"2024-02-17T12:47:47.179319+0000","last_undegraded":"2024-02-17T12:47:47.179319+0000","last_fullsized":"2024-02-17T12:47:47.179319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:04:07.188552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-17T12:47:47.180071+0000","last_change":"2024-02-17T12:47:46.068710+0000","last_active":"2024-02-17T12:47:47.180071+0000","last_peered":"2024-02-17T12:47:47.180071+0000","last_clean":"2024-02-17T12:47:47.180071+0000","last_became_active":"2024-02-17T12:47:46.068368+0000","last_became_peered":"2024-02-17T12:47:46.068368+0000","last_unstale":"2024-02-17T12:47:47.180071+0000","last_undegraded":"2024-02-17T12:47:47.180071+0000","last_fullsized":"2024-02-17T12:47:47.180071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:29:57.982649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5065,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":131,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":17396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":445071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]}]},{"osd":10,"up_from":73,"seq":313532612625,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1037437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":1,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48699999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.432}]},{"osd":4,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38100000000000001}]},{"osd":5,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57999999999999996}]},{"osd":6,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.502}]},{"osd":7,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.54900000000000004}]},{"osd":8,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.67800000000000005}]},{"osd":9,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.51500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40424,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704728,"statfs":{"total":95995035648,"available":95953641472,"internally_reserved":0,"allocated":950272,"data_stored":444787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48299999999999998}]},{"osd":1,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52400000000000002}]},{"osd":2,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.68999999999999995}]},{"osd":5,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76400000000000001}]},{"osd":6,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.56999999999999995}]},{"osd":7,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.753},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.78500000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.80300000000000005}]},{"osd":10,"last update":"Sat Feb 17 12:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.74299999999999999}]}]},{"osd":8,"up_from":60,"seq":257698037788,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":444505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.439,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.32400000000000001}]},{"osd":1,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.0089999999999999,"15min":1.0840000000000001},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73799999999999999,"5min":10.955,"15min":10.955},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.5}]},{"osd":2,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.442,"15min":0.442},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48999999999999999,"15min":0.496},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.78400000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.27900000000000003}]},{"osd":3,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.40200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.436}]},{"osd":4,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":1.0289999999999999,"15min":1.1000000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.874,"5min":10.972,"15min":10.972},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.47299999999999998}]},{"osd":5,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.376},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56899999999999995,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79800000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48399999999999999}]},{"osd":6,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.48099999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.77200000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.85199999999999998,"15min":0.89000000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88,"5min":4.9269999999999996,"15min":4.9269999999999996},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.624},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70999999999999996},"min":{"1min":0.49299999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.52300000000000002}]},{"osd":9,"last update":"Sat Feb 17 12:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53300000000000003}]},{"osd":10,"last update":"Sat Feb 17 12:47:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":10.987,"5min":10.987,"15min":10.987},"last":0.55200000000000005}]}]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32684,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712468,"statfs":{"total":95995035648,"available":95961567232,"internally_reserved":0,"allocated":1544192,"data_stored":1036711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.41899999999999998,"15min":0.40999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.60599999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.438,"15min":0.43099999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.315}]},{"osd":2,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.417,"5min":0.73999999999999999,"15min":0.627},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":15.003,"5min":15.003,"15min":15.003},"last":0.58799999999999997}]},{"osd":3,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.627,"15min":0.627},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.496},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86799999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.433}]},{"osd":4,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.51100000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.52100000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":0.80100000000000005,"15min":0.68500000000000005},"min":{"1min":0.48399999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":15.028,"5min":15.028,"15min":15.028},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.70999999999999996}]},{"osd":6,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.53800000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45100000000000001}]},{"osd":8,"last update":"Sat Feb 17 12:47:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.47699999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.74399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54700000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Sat Feb 17 12:47:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.60499999999999998}]},{"osd":10,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.63400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495654,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":443991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.375,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.55100000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40100000000000002,"15min":0.39700000000000002},"min":{"1min":0.26000000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59699999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.55800000000000005,"15min":0.54500000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.81799999999999995}]},{"osd":2,"last update":"Sat Feb 17 12:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.54600000000000004,"15min":0.53600000000000003},"min":{"1min":0.48599999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.50600000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.752,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.48399999999999999}]},{"osd":3,"last update":"Sat Feb 17 12:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.45400000000000001,"15min":0.44800000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.48499999999999999,"15min":0.47499999999999998},"min":{"1min":0.377,"5min":0.373,"15min":0.373},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46200000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Sat Feb 17 12:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61399999999999999,"15min":0.59999999999999998},"min":{"1min":0.51200000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.52700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.50800000000000001}]},{"osd":7,"last update":"Sat Feb 17 12:47:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.59599999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98399999999999999,"5min":1.21,"15min":1.21},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.65000000000000002,"15min":0.63900000000000001},"min":{"1min":0.58699999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.84299999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.73899999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.006,"5min":1.252,"15min":1.252},"last":0.72199999999999998}]},{"osd":9,"last update":"Sat Feb 17 12:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66000000000000003}]},{"osd":10,"last update":"Sat Feb 17 12:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.78800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":445491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:46:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47599999999999998,"15min":0.47799999999999998},"min":{"1min":0.29699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72599999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.34999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71599999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.66900000000000004}]},{"osd":2,"last update":"Sat Feb 17 12:47:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.503},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56699999999999995}]},{"osd":3,"last update":"Sat Feb 17 12:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77000000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.54100000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73299999999999998,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60499999999999998}]},{"osd":4,"last update":"Sat Feb 17 12:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39600000000000002,"15min":0.38200000000000001},"min":{"1min":0.28599999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64400000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.41699999999999998,"15min":0.41199999999999998},"min":{"1min":0.309,"5min":0.255,"15min":0.255},"max":{"1min":0.63400000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.68400000000000005}]},{"osd":5,"last update":"Sat Feb 17 12:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90400000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.502},{"interface":"front","average":{"1min":0.628,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.62}]},{"osd":6,"last update":"Sat Feb 17 12:46:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86699999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78200000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat Feb 17 12:47:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79800000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.755},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.374,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.871,"5min":1.079,"15min":1.079},"last":0.78400000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.65100000000000002}]},{"osd":9,"last update":"Sat Feb 17 12:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81299999999999994}]},{"osd":10,"last update":"Sat Feb 17 12:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":445357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 12:46:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.50800000000000001,"15min":0.47999999999999998},"min":{"1min":0.39800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.51900000000000002,"15min":0.48699999999999999},"min":{"1min":0.36599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45500000000000002}]},{"osd":2,"last update":"Sat Feb 17 12:47:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70399999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64400000000000002,"15min":0.73399999999999999},"min":{"1min":0.377,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81599999999999995,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.55800000000000005}]},{"osd":3,"last update":"Sat Feb 17 12:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39300000000000002,"15min":0.377},"min":{"1min":0.30299999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64200000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.443,"15min":0.42699999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Feb 17 12:47:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.42299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81799999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.436},{"interface":"front","average":{"1min":0.629,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80300000000000005,"5min":0.872,"15min":0.872},"last":0.54400000000000004}]},{"osd":5,"last update":"Sat Feb 17 12:47:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60099999999999998,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.48099999999999998}]},{"osd":6,"last update":"Sat Feb 17 12:46:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46600000000000003,"15min":0.45800000000000002},"min":{"1min":0.32100000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.311},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Sat Feb 17 12:47:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.443},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64600000000000002,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:47:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.627,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.44900000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77800000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.60999999999999999}]},{"osd":9,"last update":"Sat Feb 17 12:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.378},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46200000000000002}]},{"osd":10,"last update":"Sat Feb 17 12:47:35 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.495},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.624}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":929792,"data_stored":436955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48599999999999999,"15min":0.46200000000000002},"min":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40899999999999997}]},{"osd":1,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52300000000000002,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.312},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51600000000000001,"15min":0.498},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.51300000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46500000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.55500000000000005,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.36899999999999999}]},{"osd":5,"last update":"Sat Feb 17 12:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.441,"15min":0.433},"min":{"1min":0.28699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.35999999999999999}]},{"osd":6,"last update":"Sat Feb 17 12:46:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53800000000000003}]},{"osd":7,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.60999999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.56399999999999995,"5min":0.998,"15min":0.998},"last":0.5},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72699999999999998,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.49199999999999999}]},{"osd":9,"last update":"Sat Feb 17 12:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.63800000000000001}]},{"osd":10,"last update":"Sat Feb 17 12:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.67100000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":444287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.31900000000000001},"min":{"1min":0.25900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.50700000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35499999999999998,"15min":0.33500000000000002},"min":{"1min":0.27200000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.67300000000000004}]},{"osd":1,"last update":"Sat Feb 17 12:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.51800000000000002},"min":{"1min":0.39300000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.84599999999999997,"5min":0.879,"15min":0.879},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57399999999999995}]},{"osd":2,"last update":"Sat Feb 17 12:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85899999999999999,"5min":0.88,"15min":0.88},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.43099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82599999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68799999999999994}]},{"osd":4,"last update":"Sat Feb 17 12:47:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53700000000000003}]},{"osd":5,"last update":"Sat Feb 17 12:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64100000000000001,"15min":0.627},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84799999999999998}]},{"osd":6,"last update":"Sat Feb 17 12:46:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.51200000000000001,"15min":0.499},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.46500000000000002}]},{"osd":7,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.629,"15min":0.623},"min":{"1min":0.50900000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.755},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.57599999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92400000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61399999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:47:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91600000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.71299999999999997,"15min":0.72599999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0069999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.56899999999999995}]},{"osd":9,"last update":"Sat Feb 17 12:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.59399999999999997}]},{"osd":10,"last update":"Sat Feb 17 12:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.90100000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":444936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57899999999999996}]},{"osd":1,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.35299999999999998},"min":{"1min":0.32300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.56100000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.38300000000000001,"15min":0.36699999999999999},"min":{"1min":0.29799999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.434}]},{"osd":2,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.76500000000000001,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76700000000000002,"5min":18.73,"15min":18.73},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.45500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78800000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52100000000000002}]},{"osd":3,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.39800000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.46400000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.54700000000000004,"5min":0.378,"15min":0.378},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.754}]},{"osd":6,"last update":"Sat Feb 17 12:46:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.63200000000000001}]},{"osd":7,"last update":"Sat Feb 17 12:47:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.35199999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38900000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59099999999999997}]},{"osd":8,"last update":"Sat Feb 17 12:47:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.504,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82899999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.73699999999999999}]},{"osd":9,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66900000000000004}]},{"osd":10,"last update":"Sat Feb 17 12:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1548288,"data_stored":1035723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.315,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78600000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.50700000000000001,"15min":0.497},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.52500000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.51300000000000001},"min":{"1min":0.372,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54100000000000004,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.748,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.61199999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38900000000000001,"15min":0.377},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.53400000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.36699999999999999},"min":{"1min":0.27000000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59899999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.38300000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73499999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.437},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54000000000000004,"15min":0.53000000000000003},"min":{"1min":0.373,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.503}]},{"osd":4,"last update":"Sat Feb 17 12:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.72299999999999998,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.51600000000000001},"min":{"1min":0.40400000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84199999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65500000000000003}]},{"osd":6,"last update":"Sat Feb 17 12:46:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57899999999999996,"15min":0.57399999999999995},"min":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.45600000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.629}]},{"osd":7,"last update":"Sat Feb 17 12:47:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.84199999999999997,"5min":1,"15min":1},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85799999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77700000000000002}]},{"osd":8,"last update":"Sat Feb 17 12:47:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.58199999999999996}]},{"osd":9,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.745}]},{"osd":10,"last update":"Sat Feb 17 12:47:29 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.80500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T12:47:53.504 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-17T12:47:53.504 INFO:tasks.cephadm:Setting up client nodes... 2024-02-17T12:47:53.504 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-17T12:47:53.780 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:47:54.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:54 smithi018 ceph-mon[32625]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:54.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:54 smithi018 ceph-mon[32625]: pgmap v287: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.6 KiB/s wr, 20 op/s 2024-02-17T12:47:54.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:54 smithi018 ceph-mon[32625]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-17T12:47:54.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:54 smithi018 ceph-mon[32625]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:47:54.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:54 smithi119 ceph-mon[36188]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:54.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:54 smithi119 ceph-mon[36188]: pgmap v287: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.6 KiB/s wr, 20 op/s 2024-02-17T12:47:54.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:54 smithi119 ceph-mon[36188]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-17T12:47:54.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:54 smithi119 ceph-mon[36188]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:47:54.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:54 smithi146 ceph-mon[36454]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:47:54.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:54 smithi146 ceph-mon[36454]: pgmap v287: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.6 KiB/s wr, 20 op/s 2024-02-17T12:47:54.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:54 smithi146 ceph-mon[36454]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-17T12:47:54.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:54 smithi146 ceph-mon[36454]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:47:55.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:55 smithi119 ceph-mon[36188]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:47:55.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.353+0000 7f15a4557700 1 -- 172.21.15.18:0/3825493397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f159c0fff80 msgr2=0x7f159c1003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:55.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.353+0000 7f15a4557700 1 --2- 172.21.15.18:0/3825493397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f159c0fff80 0x7f159c1003a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1598009990 tx=0x7f1598009ca0 comp rx=0 tx=0).stop 2024-02-17T12:47:55.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.353+0000 7f15a4557700 1 -- 172.21.15.18:0/3825493397 shutdown_connections 2024-02-17T12:47:55.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.353+0000 7f15a4557700 1 --2- 172.21.15.18:0/3825493397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f159c103900 0x7f159c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:55.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.353+0000 7f15a4557700 1 --2- 172.21.15.18:0/3825493397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f159c0fff80 0x7f159c1003a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:55.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.353+0000 7f15a4557700 1 --2- 172.21.15.18:0/3825493397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f159c102f60 0x7f159c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:55.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.353+0000 7f15a4557700 1 -- 172.21.15.18:0/3825493397 >> 172.21.15.18:0/3825493397 conn(0x7f159c0fb900 msgr2=0x7f159c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:55.347 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:55 smithi018 ceph-mon[32625]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:47:55.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.353+0000 7f15a4557700 1 -- 172.21.15.18:0/3825493397 shutdown_connections 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.353+0000 7f15a4557700 1 -- 172.21.15.18:0/3825493397 wait complete. 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a4557700 1 Processor -- start 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a4557700 1 -- start start 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a4557700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f159c0fff80 0x7f159c102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a4557700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f159c102f60 0x7f159c100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a4557700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f159c103900 0x7f159c101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a4557700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c101bf0 con 0x7f159c0fff80 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a4557700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c101d60 con 0x7f159c102f60 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a4557700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c1a1c30 con 0x7f159c103900 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a22f3700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f159c0fff80 0x7f159c102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a22f3700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f159c0fff80 0x7f159c102870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39306/0 (socket says 172.21.15.18:39306) 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a22f3700 1 -- 172.21.15.18:0/3233938365 learned_addr learned my addr 172.21.15.18:0/3233938365 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:55.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a2af4700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f159c103900 0x7f159c101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.354+0000 7f15a1af2700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f159c102f60 0x7f159c100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.355+0000 7f15a22f3700 1 -- 172.21.15.18:0/3233938365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f159c103900 msgr2=0x7f159c101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.355+0000 7f15a22f3700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f159c103900 0x7f159c101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.355+0000 7f15a22f3700 1 -- 172.21.15.18:0/3233938365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f159c102f60 msgr2=0x7f159c100f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.355+0000 7f15a22f3700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f159c102f60 0x7f159c100f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.355+0000 7f15a22f3700 1 -- 172.21.15.18:0/3233938365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15980095f0 con 0x7f159c0fff80 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.355+0000 7f15a2af4700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f159c103900 0x7f159c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.355+0000 7f15a1af2700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f159c102f60 0x7f159c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.355+0000 7f15a22f3700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f159c0fff80 0x7f159c102870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f15980038c0 tx=0x7f159800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.355+0000 7f15937fe700 1 -- 172.21.15.18:0/3233938365 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159801d070 con 0x7f159c0fff80 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.355+0000 7f15a4557700 1 -- 172.21.15.18:0/3233938365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159c1a1e60 con 0x7f159c0fff80 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.355+0000 7f15937fe700 1 -- 172.21.15.18:0/3233938365 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1598005510 con 0x7f159c0fff80 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.355+0000 7f15937fe700 1 -- 172.21.15.18:0/3233938365 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598004b90 con 0x7f159c0fff80 2024-02-17T12:47:55.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.356+0000 7f15a4557700 1 -- 172.21.15.18:0/3233938365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c1a2210 con 0x7f159c0fff80 2024-02-17T12:47:55.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.356+0000 7f15a4557700 1 -- 172.21.15.18:0/3233938365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c1091a0 con 0x7f159c0fff80 2024-02-17T12:47:55.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.357+0000 7f15937fe700 1 -- 172.21.15.18:0/3233938365 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f1598004cf0 con 0x7f159c0fff80 2024-02-17T12:47:55.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.358+0000 7f15937fe700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f15880a2290 0x7f15880a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:55.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.358+0000 7f15a1af2700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f15880a2290 0x7f15880a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:55.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.358+0000 7f15937fe700 1 -- 172.21.15.18:0/3233938365 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f15980ceba0 con 0x7f159c0fff80 2024-02-17T12:47:55.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.358+0000 7f15a1af2700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f15880a2290 0x7f15880a4750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f158c005950 tx=0x7f158c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:55.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.362+0000 7f15937fe700 1 -- 172.21.15.18:0/3233938365 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1598097c50 con 0x7f159c0fff80 2024-02-17T12:47:55.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:55 smithi146 ceph-mon[36454]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:47:55.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.647+0000 7f15a4557700 1 -- 172.21.15.18:0/3233938365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f159c108b80 con 0x7f159c0fff80 2024-02-17T12:47:55.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.650+0000 7f15937fe700 1 -- 172.21.15.18:0/3233938365 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f159809b310 con 0x7f159c0fff80 2024-02-17T12:47:55.642 INFO:teuthology.orchestra.run.smithi018.stdout:[client.0] 2024-02-17T12:47:55.643 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQD7qtBlnTayJhAApOqLtWGer/JV7MHSjQYixA== 2024-02-17T12:47:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.652+0000 7f15a4557700 1 -- 172.21.15.18:0/3233938365 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f15880a2290 msgr2=0x7f15880a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.652+0000 7f15a4557700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f15880a2290 0x7f15880a4750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f158c005950 tx=0x7f158c0058e0 comp rx=0 tx=0).stop 2024-02-17T12:47:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.652+0000 7f15a4557700 1 -- 172.21.15.18:0/3233938365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f159c0fff80 msgr2=0x7f159c102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.652+0000 7f15a4557700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f159c0fff80 0x7f159c102870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f15980038c0 tx=0x7f159800f740 comp rx=0 tx=0).stop 2024-02-17T12:47:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.652+0000 7f15a4557700 1 -- 172.21.15.18:0/3233938365 shutdown_connections 2024-02-17T12:47:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.652+0000 7f15a4557700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f15880a2290 0x7f15880a4750 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:55.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.652+0000 7f15a4557700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f159c103900 0x7f159c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:55.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.652+0000 7f15a4557700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f159c0fff80 0x7f159c102870 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:55.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.652+0000 7f15a4557700 1 --2- 172.21.15.18:0/3233938365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f159c102f60 0x7f159c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:55.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.652+0000 7f15a4557700 1 -- 172.21.15.18:0/3233938365 >> 172.21.15.18:0/3233938365 conn(0x7f159c0fb900 msgr2=0x7f159c1062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:55.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.652+0000 7f15a4557700 1 -- 172.21.15.18:0/3233938365 shutdown_connections 2024-02-17T12:47:55.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:55.652+0000 7f15a4557700 1 -- 172.21.15.18:0/3233938365 wait complete. 2024-02-17T12:47:56.259 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:56 smithi018 ceph-mon[32625]: pgmap v288: 97 pgs: 97 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5.6 KiB/s wr, 24 op/s 2024-02-17T12:47:56.259 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:56 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3233938365' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-17T12:47:56.259 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:56 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3233938365' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-17T12:47:56.286 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:47:56.286 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-17T12:47:56.287 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-17T12:47:56.329 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-17T12:47:56.329 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-17T12:47:56.330 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph mgr dump --format=json 2024-02-17T12:47:56.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:56 smithi119 ceph-mon[36188]: pgmap v288: 97 pgs: 97 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5.6 KiB/s wr, 24 op/s 2024-02-17T12:47:56.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:56 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3233938365' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-17T12:47:56.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:56 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3233938365' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-17T12:47:56.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:56 smithi146 ceph-mon[36454]: pgmap v288: 97 pgs: 97 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5.6 KiB/s wr, 24 op/s 2024-02-17T12:47:56.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:56 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3233938365' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-17T12:47:56.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:56 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3233938365' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-17T12:47:56.611 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:47:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.185+0000 7fa7ec898700 1 -- 172.21.15.18:0/2725099902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8102680 msgr2=0x7fa7e810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.185+0000 7fa7ec898700 1 --2- 172.21.15.18:0/2725099902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8102680 0x7fa7e810eac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000b0d0 tx=0x7fa7e000b3e0 comp rx=0 tx=0).stop 2024-02-17T12:47:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.185+0000 7fa7ec898700 1 -- 172.21.15.18:0/2725099902 shutdown_connections 2024-02-17T12:47:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.185+0000 7fa7ec898700 1 --2- 172.21.15.18:0/2725099902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8102680 0x7fa7e810eac0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.185+0000 7fa7ec898700 1 --2- 172.21.15.18:0/2725099902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8107ca0 0x7fa7e8108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.185+0000 7fa7ec898700 1 --2- 172.21.15.18:0/2725099902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7e8101cc0 0x7fa7e8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.185+0000 7fa7ec898700 1 -- 172.21.15.18:0/2725099902 >> 172.21.15.18:0/2725099902 conn(0x7fa7e8078da0 msgr2=0x7fa7e80791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.185+0000 7fa7ec898700 1 -- 172.21.15.18:0/2725099902 shutdown_connections 2024-02-17T12:47:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.185+0000 7fa7ec898700 1 -- 172.21.15.18:0/2725099902 wait complete. 2024-02-17T12:47:58.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.185+0000 7fa7ec898700 1 Processor -- start 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7ec898700 1 -- start start 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7ec898700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101cc0 0x7fa7e81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7ec898700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7e8102680 0x7fa7e81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7ec898700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8107ca0 0x7fa7e819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7ec898700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e819d0e0 con 0x7fa7e8101cc0 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7ec898700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e819d250 con 0x7fa7e8102680 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7ec898700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e819d410 con 0x7fa7e8107ca0 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7e659c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101cc0 0x7fa7e81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7e659c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101cc0 0x7fa7e81982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39322/0 (socket says 172.21.15.18:39322) 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7e659c700 1 -- 172.21.15.18:0/279781990 learned_addr learned my addr 172.21.15.18:0/279781990 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7e5d9b700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7e8102680 0x7fa7e81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7e6d9d700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8107ca0 0x7fa7e819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7e659c700 1 -- 172.21.15.18:0/279781990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8107ca0 msgr2=0x7fa7e819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7e659c700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8107ca0 0x7fa7e819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7e659c700 1 -- 172.21.15.18:0/279781990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7e8102680 msgr2=0x7fa7e81987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7e659c700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7e8102680 0x7fa7e81987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7e659c700 1 -- 172.21.15.18:0/279781990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e0009d00 con 0x7fa7e8101cc0 2024-02-17T12:47:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7e6d9d700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8107ca0 0x7fa7e819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:47:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.186+0000 7fa7e659c700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101cc0 0x7fa7e81982a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000b530 tx=0x7fa7d000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.187+0000 7fa7df7fe700 1 -- 172.21.15.18:0/279781990 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0011830 con 0x7fa7e8101cc0 2024-02-17T12:47:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.187+0000 7fa7ec898700 1 -- 172.21.15.18:0/279781990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e819d6f0 con 0x7fa7e8101cc0 2024-02-17T12:47:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.187+0000 7fa7df7fe700 1 -- 172.21.15.18:0/279781990 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7d0011e70 con 0x7fa7e8101cc0 2024-02-17T12:47:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.187+0000 7fa7df7fe700 1 -- 172.21.15.18:0/279781990 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d000f340 con 0x7fa7e8101cc0 2024-02-17T12:47:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.187+0000 7fa7ec898700 1 -- 172.21.15.18:0/279781990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e81a4f50 con 0x7fa7e8101cc0 2024-02-17T12:47:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.188+0000 7fa7ec898700 1 -- 172.21.15.18:0/279781990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e810a180 con 0x7fa7e8101cc0 2024-02-17T12:47:58.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.189+0000 7fa7df7fe700 1 -- 172.21.15.18:0/279781990 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa7d00106d0 con 0x7fa7e8101cc0 2024-02-17T12:47:58.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.189+0000 7fa7df7fe700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa7d40a21e0 0x7fa7d40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:47:58.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.189+0000 7fa7e5d9b700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa7d40a21e0 0x7fa7d40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:47:58.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.190+0000 7fa7df7fe700 1 -- 172.21.15.18:0/279781990 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa7d00cd800 con 0x7fa7e8101cc0 2024-02-17T12:47:58.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.190+0000 7fa7e5d9b700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa7d40a21e0 0x7fa7d40a46a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8079310 tx=0x7fa7d800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:47:58.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.194+0000 7fa7df7fe700 1 -- 172.21.15.18:0/279781990 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7d0096830 con 0x7fa7e8101cc0 2024-02-17T12:47:58.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:58 smithi119 ceph-mon[36188]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 30 op/s 2024-02-17T12:47:58.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.455+0000 7fa7ec898700 1 -- 172.21.15.18:0/279781990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa7e81995c0 con 0x7fa7e8101cc0 2024-02-17T12:47:58.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:58 smithi018 ceph-mon[32625]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 30 op/s 2024-02-17T12:47:58.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:58 smithi146 ceph-mon[36454]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 30 op/s 2024-02-17T12:47:58.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.460+0000 7fa7df7fe700 1 -- 172.21.15.18:0/279781990 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261485 (secure 0 0 0) 0x7fa7d0099ef0 con 0x7fa7e8101cc0 2024-02-17T12:47:58.456 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:47:58.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.463+0000 7fa7ec898700 1 -- 172.21.15.18:0/279781990 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa7d40a21e0 msgr2=0x7fa7d40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:58.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.463+0000 7fa7ec898700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa7d40a21e0 0x7fa7d40a46a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8079310 tx=0x7fa7d800a380 comp rx=0 tx=0).stop 2024-02-17T12:47:58.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.463+0000 7fa7ec898700 1 -- 172.21.15.18:0/279781990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101cc0 msgr2=0x7fa7e81982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:47:58.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.463+0000 7fa7ec898700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101cc0 0x7fa7e81982a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000b530 tx=0x7fa7d000b840 comp rx=0 tx=0).stop 2024-02-17T12:47:58.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.463+0000 7fa7ec898700 1 -- 172.21.15.18:0/279781990 shutdown_connections 2024-02-17T12:47:58.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.463+0000 7fa7ec898700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa7d40a21e0 0x7fa7d40a46a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:58.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.463+0000 7fa7ec898700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8107ca0 0x7fa7e819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:58.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.463+0000 7fa7ec898700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101cc0 0x7fa7e81982a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:58.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.463+0000 7fa7ec898700 1 --2- 172.21.15.18:0/279781990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7e8102680 0x7fa7e81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:47:58.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.463+0000 7fa7ec898700 1 -- 172.21.15.18:0/279781990 >> 172.21.15.18:0/279781990 conn(0x7fa7e8078da0 msgr2=0x7fa7e81007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:47:58.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.463+0000 7fa7ec898700 1 -- 172.21.15.18:0/279781990 shutdown_connections 2024-02-17T12:47:58.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:47:58.463+0000 7fa7ec898700 1 -- 172.21.15.18:0/279781990 wait complete. 2024-02-17T12:47:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":3026633658},{"type":"v1","addr":"172.21.15.18:6801","nonce":3026633658}]},"active_addr":"172.21.15.18:6801/3026633658","active_change":"2024-02-17T12:40:50.114194+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.18:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":2513093443}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":1103049228}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":1074404031}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":3776356008}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":1128360183}]}]} 2024-02-17T12:47:59.157 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-17T12:47:59.157 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-17T12:47:59.157 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-17T12:47:59.398 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:47:59.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:47:59 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/279781990' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-17T12:47:59.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:47:59 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/279781990' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-17T12:47:59.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:47:59 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/279781990' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-17T12:48:00.407 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:00 smithi018 ceph-mon[32625]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.6 KiB/s wr, 28 op/s 2024-02-17T12:48:00.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:00 smithi146 ceph-mon[36454]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.6 KiB/s wr, 28 op/s 2024-02-17T12:48:00.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:00 smithi119 ceph-mon[36188]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.6 KiB/s wr, 28 op/s 2024-02-17T12:48:01.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.038+0000 7f0fbbc30700 1 -- 172.21.15.18:0/2317686926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fb4101530 msgr2=0x7f0fb410d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:01.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.038+0000 7f0fbbc30700 1 --2- 172.21.15.18:0/2317686926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fb4101530 0x7f0fb410d9c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00b0a0 tx=0x7f0fac00b3b0 comp rx=0 tx=0).stop 2024-02-17T12:48:01.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.038+0000 7f0fbbc30700 1 -- 172.21.15.18:0/2317686926 shutdown_connections 2024-02-17T12:48:01.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.038+0000 7f0fbbc30700 1 --2- 172.21.15.18:0/2317686926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fb4101530 0x7f0fb410d9c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:01.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.038+0000 7f0fbbc30700 1 --2- 172.21.15.18:0/2317686926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fb4106b80 0x7f0fb4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:01.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.038+0000 7f0fbbc30700 1 --2- 172.21.15.18:0/2317686926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fb4100b70 0x7f0fb4100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:01.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.038+0000 7f0fbbc30700 1 -- 172.21.15.18:0/2317686926 >> 172.21.15.18:0/2317686926 conn(0x7f0fb4077aa0 msgr2=0x7f0fb4077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:01.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.038+0000 7f0fbbc30700 1 -- 172.21.15.18:0/2317686926 shutdown_connections 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.038+0000 7f0fbbc30700 1 -- 172.21.15.18:0/2317686926 wait complete. 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fbbc30700 1 Processor -- start 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fbbc30700 1 -- start start 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fbbc30700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fb4100b70 0x7f0fb41027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fbbc30700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fb4101530 0x7f0fb4102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fbbc30700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fb4106b80 0x7f0fb4103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fbbc30700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41a6510 con 0x7f0fb4100b70 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fbbc30700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41a6680 con 0x7f0fb4101530 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fbbc30700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41a6840 con 0x7f0fb4106b80 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fb99cc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fb4100b70 0x7f0fb41027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fb99cc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fb4100b70 0x7f0fb41027b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35958/0 (socket says 172.21.15.18:35958) 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fb99cc700 1 -- 172.21.15.18:0/1957201386 learned_addr learned my addr 172.21.15.18:0/1957201386 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:01.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fba1cd700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fb4106b80 0x7f0fb4103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.039+0000 7f0fb91cb700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fb4101530 0x7f0fb4102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.040+0000 7f0fb99cc700 1 -- 172.21.15.18:0/1957201386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fb4106b80 msgr2=0x7f0fb4103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.040+0000 7f0fb99cc700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fb4106b80 0x7f0fb4103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.040+0000 7f0fb99cc700 1 -- 172.21.15.18:0/1957201386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fb4101530 msgr2=0x7f0fb4102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.040+0000 7f0fb99cc700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fb4101530 0x7f0fb4102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.040+0000 7f0fb99cc700 1 -- 172.21.15.18:0/1957201386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac009d00 con 0x7f0fb4100b70 2024-02-17T12:48:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.040+0000 7f0fb99cc700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fb4100b70 0x7f0fb41027b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800c9b0 tx=0x7f0fa800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.040+0000 7f0fa6ffd700 1 -- 172.21.15.18:0/1957201386 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8007b00 con 0x7f0fb4100b70 2024-02-17T12:48:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.040+0000 7f0fa6ffd700 1 -- 172.21.15.18:0/1957201386 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fa800ce50 con 0x7f0fb4100b70 2024-02-17T12:48:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.040+0000 7f0fbbc30700 1 -- 172.21.15.18:0/1957201386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb41a6b20 con 0x7f0fb4100b70 2024-02-17T12:48:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.040+0000 7f0fa6ffd700 1 -- 172.21.15.18:0/1957201386 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8018660 con 0x7f0fb4100b70 2024-02-17T12:48:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.040+0000 7f0fbbc30700 1 -- 172.21.15.18:0/1957201386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb41a7020 con 0x7f0fb4100b70 2024-02-17T12:48:01.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.041+0000 7f0fbbc30700 1 -- 172.21.15.18:0/1957201386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb4068930 con 0x7f0fb4100b70 2024-02-17T12:48:01.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.042+0000 7f0fa6ffd700 1 -- 172.21.15.18:0/1957201386 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f0fa800f450 con 0x7f0fb4100b70 2024-02-17T12:48:01.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.043+0000 7f0fa6ffd700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0fa00a21e0 0x7f0fa00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:01.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.043+0000 7f0fb91cb700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0fa00a21e0 0x7f0fa00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:01.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.043+0000 7f0fa6ffd700 1 -- 172.21.15.18:0/1957201386 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0fa80cdec0 con 0x7f0fb4100b70 2024-02-17T12:48:01.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.044+0000 7f0fb91cb700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0fa00a21e0 0x7f0fa00a46a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40780a0 tx=0x7f0fb000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:01.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.047+0000 7f0fa6ffd700 1 -- 172.21.15.18:0/1957201386 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0fa8096e70 con 0x7f0fb4100b70 2024-02-17T12:48:01.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.252+0000 7f0fbbc30700 1 -- 172.21.15.18:0/1957201386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fb4103c90 con 0x7f0fb4100b70 2024-02-17T12:48:01.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.254+0000 7f0fa6ffd700 1 -- 172.21.15.18:0/1957201386 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22221 (secure 0 0 0) 0x7f0fa809a530 con 0x7f0fb4100b70 2024-02-17T12:48:01.247 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:48:01.247 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":86,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:47:47.058106+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:48:01.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.256+0000 7f0fbbc30700 1 -- 172.21.15.18:0/1957201386 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0fa00a21e0 msgr2=0x7f0fa00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:01.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.256+0000 7f0fbbc30700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0fa00a21e0 0x7f0fa00a46a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40780a0 tx=0x7f0fb000b410 comp rx=0 tx=0).stop 2024-02-17T12:48:01.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.256+0000 7f0fbbc30700 1 -- 172.21.15.18:0/1957201386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fb4100b70 msgr2=0x7f0fb41027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:01.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.256+0000 7f0fbbc30700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fb4100b70 0x7f0fb41027b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800c9b0 tx=0x7f0fa800ccc0 comp rx=0 tx=0).stop 2024-02-17T12:48:01.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.256+0000 7f0fbbc30700 1 -- 172.21.15.18:0/1957201386 shutdown_connections 2024-02-17T12:48:01.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.256+0000 7f0fbbc30700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0fa00a21e0 0x7f0fa00a46a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:01.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.256+0000 7f0fbbc30700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fb4106b80 0x7f0fb4103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:01.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.256+0000 7f0fbbc30700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fb4100b70 0x7f0fb41027b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:01.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.256+0000 7f0fbbc30700 1 --2- 172.21.15.18:0/1957201386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fb4101530 0x7f0fb4102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:01.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.256+0000 7f0fbbc30700 1 -- 172.21.15.18:0/1957201386 >> 172.21.15.18:0/1957201386 conn(0x7f0fb4077aa0 msgr2=0x7f0fb40ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:01.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.256+0000 7f0fbbc30700 1 -- 172.21.15.18:0/1957201386 shutdown_connections 2024-02-17T12:48:01.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:01.256+0000 7f0fbbc30700 1 -- 172.21.15.18:0/1957201386 wait complete. 2024-02-17T12:48:01.794 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-17T12:48:01.794 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-17T12:48:02.040 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:02.410 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:02 smithi018 ceph-mon[32625]: pgmap v291: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.8 KiB/s wr, 28 op/s 2024-02-17T12:48:02.410 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:02 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1957201386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:48:02.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:02 smithi146 ceph-mon[36454]: pgmap v291: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.8 KiB/s wr, 28 op/s 2024-02-17T12:48:02.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:02 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1957201386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:48:02.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:02 smithi119 ceph-mon[36188]: pgmap v291: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.8 KiB/s wr, 28 op/s 2024-02-17T12:48:02.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:02 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1957201386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:48:03.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.468+0000 7f86b56c4700 1 -- 172.21.15.18:0/1668196523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b0101eb0 msgr2=0x7f86b0102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:03.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.468+0000 7f86b56c4700 1 --2- 172.21.15.18:0/1668196523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b0101eb0 0x7f86b0102290 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009990 tx=0x7f86a8009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:03.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.468+0000 7f86b56c4700 1 -- 172.21.15.18:0/1668196523 shutdown_connections 2024-02-17T12:48:03.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.468+0000 7f86b56c4700 1 --2- 172.21.15.18:0/1668196523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b0106f90 0x7f86b010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:03.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.468+0000 7f86b56c4700 1 --2- 172.21.15.18:0/1668196523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b0101eb0 0x7f86b0102290 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:03.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.468+0000 7f86b56c4700 1 --2- 172.21.15.18:0/1668196523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86b0102860 0x7f86b01068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:03.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.468+0000 7f86b56c4700 1 -- 172.21.15.18:0/1668196523 >> 172.21.15.18:0/1668196523 conn(0x7f86b00fd720 msgr2=0x7f86b00ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:03.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.468+0000 7f86b56c4700 1 -- 172.21.15.18:0/1668196523 shutdown_connections 2024-02-17T12:48:03.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.468+0000 7f86b56c4700 1 -- 172.21.15.18:0/1668196523 wait complete. 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86b56c4700 1 Processor -- start 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86b56c4700 1 -- start start 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86b56c4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86b0101eb0 0x7f86b019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86b56c4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b0102860 0x7f86b019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86b56c4700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b0106f90 0x7f86b019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86b56c4700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b019f2e0 con 0x7f86b0106f90 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86b56c4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b019f450 con 0x7f86b0102860 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86b56c4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b019f610 con 0x7f86b0101eb0 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86af7fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b0106f90 0x7f86b019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86af7fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b0106f90 0x7f86b019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35990/0 (socket says 172.21.15.18:35990) 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86af7fe700 1 -- 172.21.15.18:0/908772077 learned_addr learned my addr 172.21.15.18:0/908772077 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86ae7fc700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b0102860 0x7f86b019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:03.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86aeffd700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86b0101eb0 0x7f86b019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:03.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86af7fe700 1 -- 172.21.15.18:0/908772077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86b0101eb0 msgr2=0x7f86b019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:03.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86af7fe700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86b0101eb0 0x7f86b019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:03.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86af7fe700 1 -- 172.21.15.18:0/908772077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b0102860 msgr2=0x7f86b019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:03.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86af7fe700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b0102860 0x7f86b019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:03.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.469+0000 7f86af7fe700 1 -- 172.21.15.18:0/908772077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a80095f0 con 0x7f86b0106f90 2024-02-17T12:48:03.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.470+0000 7f86af7fe700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b0106f90 0x7f86b019ecd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f86a0007ee0 tx=0x7f86a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:03.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.470+0000 7f86a7fff700 1 -- 172.21.15.18:0/908772077 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a0017070 con 0x7f86b0106f90 2024-02-17T12:48:03.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.470+0000 7f86a7fff700 1 -- 172.21.15.18:0/908772077 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86a000f040 con 0x7f86b0106f90 2024-02-17T12:48:03.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.470+0000 7f86b56c4700 1 -- 172.21.15.18:0/908772077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b019f8f0 con 0x7f86b0106f90 2024-02-17T12:48:03.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.470+0000 7f86a7fff700 1 -- 172.21.15.18:0/908772077 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a0005650 con 0x7f86b0106f90 2024-02-17T12:48:03.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.470+0000 7f86b56c4700 1 -- 172.21.15.18:0/908772077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b01a7150 con 0x7f86b0106f90 2024-02-17T12:48:03.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.471+0000 7f86b56c4700 1 -- 172.21.15.18:0/908772077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b0068930 con 0x7f86b0106f90 2024-02-17T12:48:03.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.472+0000 7f86a7fff700 1 -- 172.21.15.18:0/908772077 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f86a0004cd0 con 0x7f86b0106f90 2024-02-17T12:48:03.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.472+0000 7f86a7fff700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f869c0a21e0 0x7f869c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:03.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.473+0000 7f86aeffd700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f869c0a21e0 0x7f869c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:03.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.473+0000 7f86a7fff700 1 -- 172.21.15.18:0/908772077 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f86a00cd490 con 0x7f86b0106f90 2024-02-17T12:48:03.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.473+0000 7f86aeffd700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f869c0a21e0 0x7f869c0a46a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f86a800b5c0 tx=0x7f86a8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:03.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.477+0000 7f86a7fff700 1 -- 172.21.15.18:0/908772077 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f86a00963b0 con 0x7f86b0106f90 2024-02-17T12:48:03.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.679+0000 7f86b56c4700 1 -- 172.21.15.18:0/908772077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86b0111cf0 con 0x7f86b0106f90 2024-02-17T12:48:03.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.680+0000 7f86a7fff700 1 -- 172.21.15.18:0/908772077 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22221 (secure 0 0 0) 0x7f86a0099a70 con 0x7f86b0106f90 2024-02-17T12:48:03.673 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:48:03.673 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":86,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:47:47.058106+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:48:03.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.682+0000 7f86b56c4700 1 -- 172.21.15.18:0/908772077 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f869c0a21e0 msgr2=0x7f869c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:03.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.682+0000 7f86b56c4700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f869c0a21e0 0x7f869c0a46a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f86a800b5c0 tx=0x7f86a8005960 comp rx=0 tx=0).stop 2024-02-17T12:48:03.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.682+0000 7f86b56c4700 1 -- 172.21.15.18:0/908772077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b0106f90 msgr2=0x7f86b019ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:03.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.682+0000 7f86b56c4700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b0106f90 0x7f86b019ecd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f86a0007ee0 tx=0x7f86a000d3b0 comp rx=0 tx=0).stop 2024-02-17T12:48:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.682+0000 7f86b56c4700 1 -- 172.21.15.18:0/908772077 shutdown_connections 2024-02-17T12:48:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.682+0000 7f86b56c4700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b0106f90 0x7f86b019ecd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.682+0000 7f86b56c4700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f869c0a21e0 0x7f869c0a46a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.682+0000 7f86b56c4700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86b0101eb0 0x7f86b019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.682+0000 7f86b56c4700 1 --2- 172.21.15.18:0/908772077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b0102860 0x7f86b019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.683+0000 7f86b56c4700 1 -- 172.21.15.18:0/908772077 >> 172.21.15.18:0/908772077 conn(0x7f86b00fd720 msgr2=0x7f86b010de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.683+0000 7f86b56c4700 1 -- 172.21.15.18:0/908772077 shutdown_connections 2024-02-17T12:48:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:03.683+0000 7f86b56c4700 1 -- 172.21.15.18:0/908772077 wait complete. 2024-02-17T12:48:04.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:04 smithi018 ceph-mon[32625]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-02-17T12:48:04.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:04 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/908772077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:48:04.325 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-17T12:48:04.325 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-17T12:48:04.326 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-17T12:48:04.326 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-17T12:48:04.326 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-17T12:48:04.327 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-17T12:48:04.327 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-17T12:48:04.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-17T12:48:04.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-02-17T12:48:04.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-02-17T12:48:04.329 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-02-17T12:48:04.329 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-02-17T12:48:04.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:04 smithi119 ceph-mon[36188]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-02-17T12:48:04.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:04 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/908772077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:48:04.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:04 smithi146 ceph-mon[36454]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-02-17T12:48:04.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:04 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/908772077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:48:05.052 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:05.094 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:05.132 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:05.133 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:05.133 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:05.134 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:05.135 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:06.251 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:06 smithi018 ceph-mon[32625]: pgmap v293: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.7 KiB/s wr, 18 op/s 2024-02-17T12:48:06.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:06 smithi146 ceph-mon[36454]: pgmap v293: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.7 KiB/s wr, 18 op/s 2024-02-17T12:48:06.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:06 smithi119 ceph-mon[36188]: pgmap v293: 97 pgs: 97 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.7 KiB/s wr, 18 op/s 2024-02-17T12:48:07.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.631+0000 7fceee863700 1 -- 172.21.15.18:0/3140493774 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcee8102680 msgr2=0x7fcee810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:07.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.631+0000 7fceee863700 1 --2- 172.21.15.18:0/3140493774 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcee8102680 0x7fcee810eac0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcee000b0d0 tx=0x7fcee000b3e0 comp rx=0 tx=0).stop 2024-02-17T12:48:07.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.632+0000 7fceee863700 1 -- 172.21.15.18:0/3140493774 shutdown_connections 2024-02-17T12:48:07.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.632+0000 7fceee863700 1 --2- 172.21.15.18:0/3140493774 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcee8102680 0x7fcee810eac0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:07.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.632+0000 7fceee863700 1 --2- 172.21.15.18:0/3140493774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee8107ca0 0x7fcee8108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:07.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.632+0000 7fceee863700 1 --2- 172.21.15.18:0/3140493774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcee8101cc0 0x7fcee8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:07.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.632+0000 7fceee863700 1 -- 172.21.15.18:0/3140493774 >> 172.21.15.18:0/3140493774 conn(0x7fcee8078da0 msgr2=0x7fcee80791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:07.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.632+0000 7fceee863700 1 -- 172.21.15.18:0/3140493774 shutdown_connections 2024-02-17T12:48:07.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.632+0000 7fceee863700 1 -- 172.21.15.18:0/3140493774 wait complete. 2024-02-17T12:48:07.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.632+0000 7fceee863700 1 Processor -- start 2024-02-17T12:48:07.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.632+0000 7fceee863700 1 -- start start 2024-02-17T12:48:07.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceee863700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcee8101cc0 0x7fcee8073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:07.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceee863700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee8102680 0x7fcee80744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:07.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceee863700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcee8107ca0 0x7fcee80749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:07.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceee863700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee81a6590 con 0x7fcee8107ca0 2024-02-17T12:48:07.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceee863700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee81a6700 con 0x7fcee8102680 2024-02-17T12:48:07.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceee863700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee81a68c0 con 0x7fcee8101cc0 2024-02-17T12:48:07.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceece00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcee8107ca0 0x7fcee80749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:07.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceece00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcee8107ca0 0x7fcee80749f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36008/0 (socket says 172.21.15.18:36008) 2024-02-17T12:48:07.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceece00700 1 -- 172.21.15.18:0/1381062467 learned_addr learned my addr 172.21.15.18:0/1381062467 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:07.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fcee7fff700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcee8101cc0 0x7fcee8073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:07.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fcee77fe700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee8102680 0x7fcee80744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:07.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceece00700 1 -- 172.21.15.18:0/1381062467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcee8101cc0 msgr2=0x7fcee8073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:07.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceece00700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcee8101cc0 0x7fcee8073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:07.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceece00700 1 -- 172.21.15.18:0/1381062467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee8102680 msgr2=0x7fcee80744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceece00700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee8102680 0x7fcee80744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceece00700 1 -- 172.21.15.18:0/1381062467 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee0009d00 con 0x7fcee8107ca0 2024-02-17T12:48:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fceece00700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcee8107ca0 0x7fcee80749f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcee000d010 tx=0x7fcee0007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.633+0000 7fcee57fa700 1 -- 172.21.15.18:0/1381062467 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee000e040 con 0x7fcee8107ca0 2024-02-17T12:48:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.634+0000 7fcee57fa700 1 -- 172.21.15.18:0/1381062467 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcee0003e10 con 0x7fcee8107ca0 2024-02-17T12:48:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.634+0000 7fceee863700 1 -- 172.21.15.18:0/1381062467 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee81a6b40 con 0x7fcee8107ca0 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.634+0000 7fcee57fa700 1 -- 172.21.15.18:0/1381062467 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee001b9d0 con 0x7fcee8107ca0 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.634+0000 7fceee863700 1 -- 172.21.15.18:0/1381062467 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee81a6f50 con 0x7fcee8107ca0 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.635+0000 7fceee863700 1 -- 172.21.15.18:0/1381062467 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcee8111fd0 con 0x7fcee8107ca0 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.636+0000 7fcee57fa700 1 -- 172.21.15.18:0/1381062467 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fcee00057e0 con 0x7fcee8107ca0 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.636+0000 7fcee57fa700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fced00a21e0 0x7fced00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.636+0000 7fcee7fff700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fced00a21e0 0x7fced00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.636+0000 7fcee57fa700 1 -- 172.21.15.18:0/1381062467 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcee00cf5e0 con 0x7fcee8107ca0 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.636+0000 7fcee57fa700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609] conn(0x7fced00ab890 0x7fced00adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.636+0000 7fcee57fa700 1 -- 172.21.15.18:0/1381062467 --> [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fced00ae360 con 0x7fced00ab890 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.636+0000 7fcee57fa700 1 -- 172.21.15.18:0/1381062467 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcee00cfa80 con 0x7fcee8107ca0 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.637+0000 7fcee7fff700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fced00a21e0 0x7fced00a46a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fced800abd0 tx=0x7fced800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.637+0000 7fcee77fe700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609] conn(0x7fced00ab890 0x7fced00adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:07.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.637+0000 7fcee77fe700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609] conn(0x7fced00ab890 0x7fced00adcb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:07.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.638+0000 7fcee57fa700 1 -- 172.21.15.18:0/1381062467 <== osd.10 v2:172.21.15.119:6824/3116452609 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fced00ae360 con 0x7fced00ab890 2024-02-17T12:48:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.654+0000 7fceee863700 1 -- 172.21.15.18:0/1381062467 --> [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcee81073b0 con 0x7fced00ab890 2024-02-17T12:48:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.655+0000 7fcee57fa700 1 -- 172.21.15.18:0/1381062467 <== osd.10 v2:172.21.15.119:6824/3116452609 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcee81073b0 con 0x7fced00ab890 2024-02-17T12:48:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.655+0000 7fceee863700 1 -- 172.21.15.18:0/1381062467 >> [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609] conn(0x7fced00ab890 msgr2=0x7fced00adcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.655+0000 7fceee863700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609] conn(0x7fced00ab890 0x7fced00adcb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.655+0000 7fceee863700 1 -- 172.21.15.18:0/1381062467 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fced00a21e0 msgr2=0x7fced00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.655+0000 7fceee863700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fced00a21e0 0x7fced00a46a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fced800abd0 tx=0x7fced800a380 comp rx=0 tx=0).stop 2024-02-17T12:48:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.655+0000 7fceee863700 1 -- 172.21.15.18:0/1381062467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcee8107ca0 msgr2=0x7fcee80749f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.655+0000 7fceee863700 1 --2- 172.21.15.18:0/1381062467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcee8107ca0 0x7fcee80749f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcee000d010 tx=0x7fcee0007930 comp rx=0 tx=0).stop 2024-02-17T12:48:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.656+0000 7fceece00700 1 -- 172.21.15.18:0/1381062467 reap_dead start 2024-02-17T12:48:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.656+0000 7fceee863700 1 -- 172.21.15.18:0/1381062467 shutdown_connections 2024-02-17T12:48:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.656+0000 7fceee863700 1 -- 172.21.15.18:0/1381062467 >> 172.21.15.18:0/1381062467 conn(0x7fcee8078da0 msgr2=0x7fcee8105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.656+0000 7fceee863700 1 -- 172.21.15.18:0/1381062467 shutdown_connections 2024-02-17T12:48:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:07.656+0000 7fceee863700 1 -- 172.21.15.18:0/1381062467 wait complete. 2024-02-17T12:48:08.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:08 smithi018 ceph-mon[32625]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 18 op/s 2024-02-17T12:48:08.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:08 smithi146 ceph-mon[36454]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 18 op/s 2024-02-17T12:48:08.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:08 smithi119 ceph-mon[36188]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 18 op/s 2024-02-17T12:48:10.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:10 smithi146 ceph-mon[36454]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-02-17T12:48:10.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:10 smithi018 ceph-mon[32625]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-02-17T12:48:10.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:10 smithi119 ceph-mon[36188]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-02-17T12:48:10.721 INFO:teuthology.orchestra.run.smithi018.stdout:313532612629 2024-02-17T12:48:10.722 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-02-17T12:48:11.128 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:11.128 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:11.131 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:11.132 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:12.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:12 smithi018 ceph-mon[32625]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-02-17T12:48:12.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:12 smithi146 ceph-mon[36454]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-02-17T12:48:12.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:12 smithi119 ceph-mon[36188]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-02-17T12:48:13.144 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:14.425 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:14 smithi018 ceph-mon[32625]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-02-17T12:48:14.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:14 smithi146 ceph-mon[36454]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-02-17T12:48:14.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:14 smithi119 ceph-mon[36188]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-02-17T12:48:15.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.114+0000 7f012b68a700 1 -- 172.21.15.18:0/417003470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124101510 msgr2=0x7f012410d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:15.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.114+0000 7f012b68a700 1 --2- 172.21.15.18:0/417003470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124101510 0x7f012410d9b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f011c00b0a0 tx=0x7f011c00b3b0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.114+0000 7f012b68a700 1 -- 172.21.15.18:0/417003470 shutdown_connections 2024-02-17T12:48:15.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.114+0000 7f012b68a700 1 --2- 172.21.15.18:0/417003470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124101510 0x7f012410d9b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.114+0000 7f012b68a700 1 --2- 172.21.15.18:0/417003470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0124105ab0 0x7f0124105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.114+0000 7f012b68a700 1 --2- 172.21.15.18:0/417003470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0124100b50 0x7f0124100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.114+0000 7f012b68a700 1 -- 172.21.15.18:0/417003470 >> 172.21.15.18:0/417003470 conn(0x7f0124077b80 msgr2=0x7f0124077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:15.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.114+0000 7f012b68a700 1 -- 172.21.15.18:0/417003470 shutdown_connections 2024-02-17T12:48:15.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.114+0000 7f012b68a700 1 -- 172.21.15.18:0/417003470 wait complete. 2024-02-17T12:48:15.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.114+0000 7f012b68a700 1 Processor -- start 2024-02-17T12:48:15.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.114+0000 7f012b68a700 1 -- start start 2024-02-17T12:48:15.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f012b68a700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124100b50 0x7f01241990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:15.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f012b68a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0124101510 0x7f0124199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:15.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f012b68a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0124105ab0 0x7f01241a4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:15.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f012b68a700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124199ed0 con 0x7f0124100b50 2024-02-17T12:48:15.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f012b68a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01241a4690 con 0x7f0124101510 2024-02-17T12:48:15.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f012b68a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01241a47d0 con 0x7f0124105ab0 2024-02-17T12:48:15.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f0129426700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124100b50 0x7f01241990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f0129426700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124100b50 0x7f01241990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41554/0 (socket says 172.21.15.18:41554) 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f0129426700 1 -- 172.21.15.18:0/1085228658 learned_addr learned my addr 172.21.15.18:0/1085228658 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f0129c27700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0124105ab0 0x7f01241a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f0128c25700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0124101510 0x7f0124199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f0129426700 1 -- 172.21.15.18:0/1085228658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0124105ab0 msgr2=0x7f01241a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f0129426700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0124105ab0 0x7f01241a4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f0129426700 1 -- 172.21.15.18:0/1085228658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0124101510 msgr2=0x7f0124199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f0129426700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0124101510 0x7f0124199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f0129426700 1 -- 172.21.15.18:0/1085228658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c009d00 con 0x7f0124100b50 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f0129426700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124100b50 0x7f01241990d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f012000d950 tx=0x7f012000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.115+0000 7f011a7fc700 1 -- 172.21.15.18:0/1085228658 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012000f890 con 0x7f0124100b50 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.116+0000 7f012b68a700 1 -- 172.21.15.18:0/1085228658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01241a4ab0 con 0x7f0124100b50 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.116+0000 7f011a7fc700 1 -- 172.21.15.18:0/1085228658 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0120010460 con 0x7f0124100b50 2024-02-17T12:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.116+0000 7f011a7fc700 1 -- 172.21.15.18:0/1085228658 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012000e610 con 0x7f0124100b50 2024-02-17T12:48:15.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.116+0000 7f012b68a700 1 -- 172.21.15.18:0/1085228658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01241a5000 con 0x7f0124100b50 2024-02-17T12:48:15.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.116+0000 7f012b68a700 1 -- 172.21.15.18:0/1085228658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0124110e60 con 0x7f0124100b50 2024-02-17T12:48:15.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.118+0000 7f011a7fc700 1 -- 172.21.15.18:0/1085228658 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f0120010960 con 0x7f0124100b50 2024-02-17T12:48:15.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.118+0000 7f011a7fc700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f01100a2260 0x7f01100a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:15.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.118+0000 7f0128c25700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f01100a2260 0x7f01100a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:15.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.118+0000 7f011a7fc700 1 -- 172.21.15.18:0/1085228658 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f01200cddb0 con 0x7f0124100b50 2024-02-17T12:48:15.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.118+0000 7f011a7fc700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131] conn(0x7f01100aba40 0x7f01100ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:15.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.118+0000 7f011a7fc700 1 -- 172.21.15.18:0/1085228658 --> [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f01100ae510 con 0x7f01100aba40 2024-02-17T12:48:15.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.118+0000 7f011a7fc700 1 -- 172.21.15.18:0/1085228658 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01200ce250 con 0x7f0124100b50 2024-02-17T12:48:15.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.119+0000 7f0128c25700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f01100a2260 0x7f01100a4720 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f01240780f0 tx=0x7f011400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:15.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.119+0000 7f0129c27700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131] conn(0x7f01100aba40 0x7f01100ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:15.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.119+0000 7f0129c27700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131] conn(0x7f01100aba40 0x7f01100ade60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:15.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.120+0000 7f011a7fc700 1 -- 172.21.15.18:0/1085228658 <== osd.1 v2:172.21.15.119:6800/3915030131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f01100ae510 con 0x7f01100aba40 2024-02-17T12:48:15.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.137+0000 7f012b68a700 1 -- 172.21.15.18:0/1085228658 --> [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0124062330 con 0x7f01100aba40 2024-02-17T12:48:15.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.137+0000 7f011a7fc700 1 -- 172.21.15.18:0/1085228658 <== osd.1 v2:172.21.15.119:6800/3915030131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0124062330 con 0x7f01100aba40 2024-02-17T12:48:15.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.138+0000 7f012b68a700 1 -- 172.21.15.18:0/1085228658 >> [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131] conn(0x7f01100aba40 msgr2=0x7f01100ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:15.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.138+0000 7f012b68a700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131] conn(0x7f01100aba40 0x7f01100ade60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.138+0000 7f012b68a700 1 -- 172.21.15.18:0/1085228658 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f01100a2260 msgr2=0x7f01100a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:15.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.138+0000 7f012b68a700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f01100a2260 0x7f01100a4720 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f01240780f0 tx=0x7f011400b410 comp rx=0 tx=0).stop 2024-02-17T12:48:15.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.138+0000 7f012b68a700 1 -- 172.21.15.18:0/1085228658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124100b50 msgr2=0x7f01241990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:15.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.138+0000 7f012b68a700 1 --2- 172.21.15.18:0/1085228658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124100b50 0x7f01241990d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f012000d950 tx=0x7f012000dc60 comp rx=0 tx=0).stop 2024-02-17T12:48:15.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.138+0000 7f0129c27700 1 -- 172.21.15.18:0/1085228658 reap_dead start 2024-02-17T12:48:15.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.138+0000 7f012b68a700 1 -- 172.21.15.18:0/1085228658 shutdown_connections 2024-02-17T12:48:15.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.138+0000 7f012b68a700 1 -- 172.21.15.18:0/1085228658 >> 172.21.15.18:0/1085228658 conn(0x7f0124077b80 msgr2=0x7f01240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:15.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.138+0000 7f012b68a700 1 -- 172.21.15.18:0/1085228658 shutdown_connections 2024-02-17T12:48:15.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.138+0000 7f012b68a700 1 -- 172.21.15.18:0/1085228658 wait complete. 2024-02-17T12:48:15.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.296+0000 7f94c2989700 1 -- 172.21.15.18:0/4087327544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94bc106b80 msgr2=0x7f94bc106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:15.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.296+0000 7f94c2989700 1 --2- 172.21.15.18:0/4087327544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94bc106b80 0x7f94bc106f60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f94ac009990 tx=0x7f94ac009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.296+0000 7f94c2989700 1 -- 172.21.15.18:0/4087327544 shutdown_connections 2024-02-17T12:48:15.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.296+0000 7f94c2989700 1 --2- 172.21.15.18:0/4087327544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94bc101530 0x7f94bc10d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.296+0000 7f94c2989700 1 --2- 172.21.15.18:0/4087327544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94bc106b80 0x7f94bc106f60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.296+0000 7f94c2989700 1 --2- 172.21.15.18:0/4087327544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94bc100b70 0x7f94bc100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.296+0000 7f94c2989700 1 -- 172.21.15.18:0/4087327544 >> 172.21.15.18:0/4087327544 conn(0x7f94bc077aa0 msgr2=0x7f94bc077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.296+0000 7f94c2989700 1 -- 172.21.15.18:0/4087327544 shutdown_connections 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.296+0000 7f94c2989700 1 -- 172.21.15.18:0/4087327544 wait complete. 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94c2989700 1 Processor -- start 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94c2989700 1 -- start start 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94c2989700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94bc100b70 0x7f94bc1027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94c2989700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94bc101530 0x7f94bc102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94c2989700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94bc106b80 0x7f94bc103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94c2989700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc1a65c0 con 0x7f94bc100b70 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94c2989700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc1a6730 con 0x7f94bc106b80 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94c2989700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc1a68f0 con 0x7f94bc101530 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94bbfff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94bc100b70 0x7f94bc1027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94bbfff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94bc100b70 0x7f94bc1027b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41578/0 (socket says 172.21.15.18:41578) 2024-02-17T12:48:15.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94bbfff700 1 -- 172.21.15.18:0/1336464307 learned_addr learned my addr 172.21.15.18:0/1336464307 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94bb7fe700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94bc101530 0x7f94bc102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94c0f26700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94bc106b80 0x7f94bc103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94bbfff700 1 -- 172.21.15.18:0/1336464307 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94bc101530 msgr2=0x7f94bc102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94bbfff700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94bc101530 0x7f94bc102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94bbfff700 1 -- 172.21.15.18:0/1336464307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94bc106b80 msgr2=0x7f94bc103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94bbfff700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94bc106b80 0x7f94bc103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94bbfff700 1 -- 172.21.15.18:0/1336464307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac0095f0 con 0x7f94bc100b70 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.297+0000 7f94c0f26700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94bc106b80 0x7f94bc103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.298+0000 7f94bbfff700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94bc100b70 0x7f94bc1027b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f94ac000c00 tx=0x7f94ac00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.298+0000 7f94b97fa700 1 -- 172.21.15.18:0/1336464307 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac01d070 con 0x7f94bc100b70 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.298+0000 7f94b97fa700 1 -- 172.21.15.18:0/1336464307 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94ac00fc30 con 0x7f94bc100b70 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.298+0000 7f94c2989700 1 -- 172.21.15.18:0/1336464307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc1a6b70 con 0x7f94bc100b70 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.298+0000 7f94b97fa700 1 -- 172.21.15.18:0/1336464307 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0176e0 con 0x7f94bc100b70 2024-02-17T12:48:15.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.298+0000 7f94c2989700 1 -- 172.21.15.18:0/1336464307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc1a6fe0 con 0x7f94bc100b70 2024-02-17T12:48:15.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.299+0000 7f94c2989700 1 -- 172.21.15.18:0/1336464307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94bc110fd0 con 0x7f94bc100b70 2024-02-17T12:48:15.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.300+0000 7f94b97fa700 1 -- 172.21.15.18:0/1336464307 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f94ac0229e0 con 0x7f94bc100b70 2024-02-17T12:48:15.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.300+0000 7f94b97fa700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f94a40a2260 0x7f94a40a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:15.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.300+0000 7f94bb7fe700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f94a40a2260 0x7f94a40a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:15.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.300+0000 7f94b97fa700 1 -- 172.21.15.18:0/1336464307 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f94ac0ceee0 con 0x7f94bc100b70 2024-02-17T12:48:15.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.301+0000 7f94b97fa700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941] conn(0x7f94a40aba40 0x7f94a40ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:15.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.301+0000 7f94b97fa700 1 -- 172.21.15.18:0/1336464307 --> [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94a40ae510 con 0x7f94a40aba40 2024-02-17T12:48:15.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.301+0000 7f94b97fa700 1 -- 172.21.15.18:0/1336464307 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94ac0d15f0 con 0x7f94bc100b70 2024-02-17T12:48:15.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.301+0000 7f94c0f26700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941] conn(0x7f94a40aba40 0x7f94a40ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:15.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.301+0000 7f94bb7fe700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f94a40a2260 0x7f94a40a4720 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94b0006fd0 tx=0x7f94b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:15.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.301+0000 7f94c0f26700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941] conn(0x7f94a40aba40 0x7f94a40ade60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:15.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.302+0000 7f94b97fa700 1 -- 172.21.15.18:0/1336464307 <== osd.0 v2:172.21.15.18:6802/417036941 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f94a40ae510 con 0x7f94a40aba40 2024-02-17T12:48:15.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.319+0000 7f94c2989700 1 -- 172.21.15.18:0/1336464307 --> [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f94bc104bd0 con 0x7f94a40aba40 2024-02-17T12:48:15.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.320+0000 7f94b97fa700 1 -- 172.21.15.18:0/1336464307 <== osd.0 v2:172.21.15.18:6802/417036941 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f94bc104bd0 con 0x7f94a40aba40 2024-02-17T12:48:15.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.320+0000 7f94c2989700 1 -- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941] conn(0x7f94a40aba40 msgr2=0x7f94a40ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:15.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.320+0000 7f94c2989700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941] conn(0x7f94a40aba40 0x7f94a40ade60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:15.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.320+0000 7f94c2989700 1 -- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f94a40a2260 msgr2=0x7f94a40a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:15.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.320+0000 7f94c2989700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f94a40a2260 0x7f94a40a4720 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94b0006fd0 tx=0x7f94b0008040 comp rx=0 tx=0).stop 2024-02-17T12:48:15.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.320+0000 7f94c2989700 1 -- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94bc100b70 msgr2=0x7f94bc1027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:15.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.320+0000 7f94c2989700 1 --2- 172.21.15.18:0/1336464307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94bc100b70 0x7f94bc1027b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f94ac000c00 tx=0x7f94ac00f740 comp rx=0 tx=0).stop 2024-02-17T12:48:15.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.320+0000 7f94c0f26700 1 -- 172.21.15.18:0/1336464307 reap_dead start 2024-02-17T12:48:15.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.320+0000 7f94c2989700 1 -- 172.21.15.18:0/1336464307 shutdown_connections 2024-02-17T12:48:15.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.320+0000 7f94c2989700 1 -- 172.21.15.18:0/1336464307 >> 172.21.15.18:0/1336464307 conn(0x7f94bc077aa0 msgr2=0x7f94bc0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:15.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.320+0000 7f94c2989700 1 -- 172.21.15.18:0/1336464307 shutdown_connections 2024-02-17T12:48:15.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:15.320+0000 7f94c2989700 1 -- 172.21.15.18:0/1336464307 wait complete. 2024-02-17T12:48:16.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:16 smithi018 ceph-mon[32625]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-02-17T12:48:16.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.439+0000 7fa3d97dd700 1 -- 172.21.15.18:0/2750178587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3d4105ed0 msgr2=0x7fa3d410daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:16.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.439+0000 7fa3d97dd700 1 --2- 172.21.15.18:0/2750178587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3d4105ed0 0x7fa3d410daa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00b0a0 tx=0x7fa3cc00b3b0 comp rx=0 tx=0).stop 2024-02-17T12:48:16.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.439+0000 7fa3d97dd700 1 -- 172.21.15.18:0/2750178587 shutdown_connections 2024-02-17T12:48:16.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.439+0000 7fa3d97dd700 1 --2- 172.21.15.18:0/2750178587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3d4105ed0 0x7fa3d410daa0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:16.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.439+0000 7fa3d97dd700 1 --2- 172.21.15.18:0/2750178587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3d4100df0 0x7fa3d41011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:16.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.439+0000 7fa3d97dd700 1 --2- 172.21.15.18:0/2750178587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d41017a0 0x7fa3d41057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:16.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.439+0000 7fa3d97dd700 1 -- 172.21.15.18:0/2750178587 >> 172.21.15.18:0/2750178587 conn(0x7fa3d4077c70 msgr2=0x7fa3d4078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:16.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.439+0000 7fa3d97dd700 1 -- 172.21.15.18:0/2750178587 shutdown_connections 2024-02-17T12:48:16.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.439+0000 7fa3d97dd700 1 -- 172.21.15.18:0/2750178587 wait complete. 2024-02-17T12:48:16.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.440+0000 7fa3d97dd700 1 Processor -- start 2024-02-17T12:48:16.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.440+0000 7fa3d97dd700 1 -- start start 2024-02-17T12:48:16.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.440+0000 7fa3d97dd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3d4100df0 0x7fa3d419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:16.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.440+0000 7fa3d97dd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3d41017a0 0x7fa3d419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:16.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.440+0000 7fa3d97dd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d4105ed0 0x7fa3d419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:16.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.440+0000 7fa3d97dd700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d419f3e0 con 0x7fa3d41017a0 2024-02-17T12:48:16.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.440+0000 7fa3d97dd700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d419f550 con 0x7fa3d4105ed0 2024-02-17T12:48:16.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.440+0000 7fa3d97dd700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d419f710 con 0x7fa3d4100df0 2024-02-17T12:48:16.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.440+0000 7fa3d27fc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3d41017a0 0x7fa3d419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:16.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.440+0000 7fa3d27fc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3d41017a0 0x7fa3d419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41594/0 (socket says 172.21.15.18:41594) 2024-02-17T12:48:16.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.440+0000 7fa3d27fc700 1 -- 172.21.15.18:0/26537910 learned_addr learned my addr 172.21.15.18:0/26537910 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:16.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3d2ffd700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3d4100df0 0x7fa3d419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:16.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3d37fe700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d4105ed0 0x7fa3d419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:16.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3d27fc700 1 -- 172.21.15.18:0/26537910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3d4100df0 msgr2=0x7fa3d419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:16.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3d27fc700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3d4100df0 0x7fa3d419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:16.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3d27fc700 1 -- 172.21.15.18:0/26537910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d4105ed0 msgr2=0x7fa3d419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:16.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3d27fc700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d4105ed0 0x7fa3d419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:16.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3d27fc700 1 -- 172.21.15.18:0/26537910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc009d00 con 0x7fa3d41017a0 2024-02-17T12:48:16.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3d37fe700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d4105ed0 0x7fa3d419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:48:16.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3d27fc700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3d41017a0 0x7fa3d419aa40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800b500 tx=0x7fa3c800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:16.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3bbfff700 1 -- 172.21.15.18:0/26537910 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8011830 con 0x7fa3d41017a0 2024-02-17T12:48:16.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3d97dd700 1 -- 172.21.15.18:0/26537910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d419f9f0 con 0x7fa3d41017a0 2024-02-17T12:48:16.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3bbfff700 1 -- 172.21.15.18:0/26537910 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3c8011e70 con 0x7fa3d41017a0 2024-02-17T12:48:16.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3bbfff700 1 -- 172.21.15.18:0/26537910 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800f2e0 con 0x7fa3d41017a0 2024-02-17T12:48:16.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.441+0000 7fa3d97dd700 1 -- 172.21.15.18:0/26537910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d41a7250 con 0x7fa3d41017a0 2024-02-17T12:48:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.442+0000 7fa3d97dd700 1 -- 172.21.15.18:0/26537910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa3d4110fb0 con 0x7fa3d41017a0 2024-02-17T12:48:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.443+0000 7fa3bbfff700 1 -- 172.21.15.18:0/26537910 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa3c8010610 con 0x7fa3d41017a0 2024-02-17T12:48:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.444+0000 7fa3bbfff700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa3bc0a21e0 0x7fa3bc0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.444+0000 7fa3d2ffd700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa3bc0a21e0 0x7fa3bc0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.444+0000 7fa3bbfff700 1 -- 172.21.15.18:0/26537910 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa3c80cdd50 con 0x7fa3d41017a0 2024-02-17T12:48:16.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.444+0000 7fa3bbfff700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734] conn(0x7fa3bc0ab9c0 0x7fa3bc0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:16.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.444+0000 7fa3bbfff700 1 -- 172.21.15.18:0/26537910 --> [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3bc0ae490 con 0x7fa3bc0ab9c0 2024-02-17T12:48:16.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.444+0000 7fa3bbfff700 1 -- 172.21.15.18:0/26537910 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3c80ce0c0 con 0x7fa3d41017a0 2024-02-17T12:48:16.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.444+0000 7fa3d37fe700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734] conn(0x7fa3bc0ab9c0 0x7fa3bc0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:16.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.444+0000 7fa3d2ffd700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa3bc0a21e0 0x7fa3bc0a46a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4005fd0 tx=0x7fa3c400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:16.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.444+0000 7fa3d37fe700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734] conn(0x7fa3bc0ab9c0 0x7fa3bc0adde0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:16.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.445+0000 7fa3bbfff700 1 -- 172.21.15.18:0/26537910 <== osd.6 v2:172.21.15.18:6818/171901734 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa3bc0ae490 con 0x7fa3bc0ab9c0 2024-02-17T12:48:16.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:16 smithi146 ceph-mon[36454]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-02-17T12:48:16.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.461+0000 7fa3d97dd700 1 -- 172.21.15.18:0/26537910 --> [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa3d4062300 con 0x7fa3bc0ab9c0 2024-02-17T12:48:16.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.462+0000 7fa3bbfff700 1 -- 172.21.15.18:0/26537910 <== osd.6 v2:172.21.15.18:6818/171901734 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa3d4062300 con 0x7fa3bc0ab9c0 2024-02-17T12:48:16.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.462+0000 7fa3d97dd700 1 -- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734] conn(0x7fa3bc0ab9c0 msgr2=0x7fa3bc0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:16.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.462+0000 7fa3d97dd700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734] conn(0x7fa3bc0ab9c0 0x7fa3bc0adde0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:16.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.462+0000 7fa3d97dd700 1 -- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa3bc0a21e0 msgr2=0x7fa3bc0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:16.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.462+0000 7fa3d97dd700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa3bc0a21e0 0x7fa3bc0a46a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4005fd0 tx=0x7fa3c400a560 comp rx=0 tx=0).stop 2024-02-17T12:48:16.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.462+0000 7fa3d97dd700 1 -- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3d41017a0 msgr2=0x7fa3d419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:16.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.462+0000 7fa3d97dd700 1 --2- 172.21.15.18:0/26537910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3d41017a0 0x7fa3d419aa40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800b500 tx=0x7fa3c800b8c0 comp rx=0 tx=0).stop 2024-02-17T12:48:16.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.462+0000 7fa3d37fe700 1 -- 172.21.15.18:0/26537910 reap_dead start 2024-02-17T12:48:16.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.462+0000 7fa3d97dd700 1 -- 172.21.15.18:0/26537910 shutdown_connections 2024-02-17T12:48:16.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.462+0000 7fa3d97dd700 1 -- 172.21.15.18:0/26537910 >> 172.21.15.18:0/26537910 conn(0x7fa3d4077c70 msgr2=0x7fa3d410c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:16.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.462+0000 7fa3d97dd700 1 -- 172.21.15.18:0/26537910 shutdown_connections 2024-02-17T12:48:16.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:16.462+0000 7fa3d97dd700 1 -- 172.21.15.18:0/26537910 wait complete. 2024-02-17T12:48:16.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:16 smithi119 ceph-mon[36188]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-02-17T12:48:18.259 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:18 smithi018 ceph-mon[32625]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-02-17T12:48:18.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:18 smithi146 ceph-mon[36454]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-02-17T12:48:18.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:18 smithi119 ceph-mon[36188]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-02-17T12:48:19.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.109+0000 7f8b0a407700 1 -- 172.21.15.18:0/454511809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b04107cf0 msgr2=0x7f8b041080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:19.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.109+0000 7f8b0a407700 1 --2- 172.21.15.18:0/454511809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b04107cf0 0x7f8b041080d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8af40099c0 tx=0x7f8af4009cd0 comp rx=0 tx=0).stop 2024-02-17T12:48:19.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.109+0000 7f8b0a407700 1 -- 172.21.15.18:0/454511809 shutdown_connections 2024-02-17T12:48:19.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.109+0000 7f8b0a407700 1 --2- 172.21.15.18:0/454511809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0406ad10 0x7f8b0410eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:19.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.109+0000 7f8b0a407700 1 --2- 172.21.15.18:0/454511809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b04107cf0 0x7f8b041080d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:19.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.109+0000 7f8b0a407700 1 --2- 172.21.15.18:0/454511809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b0406a370 0x7f8b0406a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:19.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.109+0000 7f8b0a407700 1 -- 172.21.15.18:0/454511809 >> 172.21.15.18:0/454511809 conn(0x7f8b04076b90 msgr2=0x7f8b04076fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.110+0000 7f8b0a407700 1 -- 172.21.15.18:0/454511809 shutdown_connections 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.110+0000 7f8b0a407700 1 -- 172.21.15.18:0/454511809 wait complete. 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.110+0000 7f8b0a407700 1 Processor -- start 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.110+0000 7f8b0a407700 1 -- start start 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.110+0000 7f8b0a407700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0406a370 0x7f8b041981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.110+0000 7f8b0a407700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b0406ad10 0x7f8b04198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.110+0000 7f8b0a407700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b04107cf0 0x7f8b0419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b0a407700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0419d0d0 con 0x7f8b0406ad10 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b0a407700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0419d240 con 0x7f8b04107cf0 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b0a407700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0419d400 con 0x7f8b0406a370 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b037fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b0406ad10 0x7f8b04198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b037fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b0406ad10 0x7f8b04198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41626/0 (socket says 172.21.15.18:41626) 2024-02-17T12:48:19.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b037fe700 1 -- 172.21.15.18:0/404374025 learned_addr learned my addr 172.21.15.18:0/404374025 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b03fff700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0406a370 0x7f8b041981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b089a4700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b04107cf0 0x7f8b0419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b037fe700 1 -- 172.21.15.18:0/404374025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0406a370 msgr2=0x7f8b041981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b037fe700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0406a370 0x7f8b041981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b037fe700 1 -- 172.21.15.18:0/404374025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b04107cf0 msgr2=0x7f8b0419cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b037fe700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b04107cf0 0x7f8b0419cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b037fe700 1 -- 172.21.15.18:0/404374025 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af40095f0 con 0x7f8b0406ad10 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b037fe700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b0406ad10 0x7f8b04198730 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8af800d950 tx=0x7f8af800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b017fa700 1 -- 172.21.15.18:0/404374025 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8009940 con 0x7f8b0406ad10 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b017fa700 1 -- 172.21.15.18:0/404374025 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8af8010460 con 0x7f8b0406ad10 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b0a407700 1 -- 172.21.15.18:0/404374025 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0419d6e0 con 0x7f8b0406ad10 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b017fa700 1 -- 172.21.15.18:0/404374025 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af800f620 con 0x7f8b0406ad10 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.111+0000 7f8b0a407700 1 -- 172.21.15.18:0/404374025 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b041a4f40 con 0x7f8b0406ad10 2024-02-17T12:48:19.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.112+0000 7f8b0a407700 1 -- 172.21.15.18:0/404374025 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b04111f50 con 0x7f8b0406ad10 2024-02-17T12:48:19.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.113+0000 7f8b017fa700 1 -- 172.21.15.18:0/404374025 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f8af80105d0 con 0x7f8b0406ad10 2024-02-17T12:48:19.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.114+0000 7f8b017fa700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8aec0a21e0 0x7f8aec0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:19.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.114+0000 7f8b03fff700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8aec0a21e0 0x7f8aec0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:19.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.114+0000 7f8b017fa700 1 -- 172.21.15.18:0/404374025 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8af80ce070 con 0x7f8b0406ad10 2024-02-17T12:48:19.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.114+0000 7f8b017fa700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818] conn(0x7f8aec0ab9c0 0x7f8aec0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:19.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.114+0000 7f8b017fa700 1 -- 172.21.15.18:0/404374025 --> [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8aec0ae490 con 0x7f8aec0ab9c0 2024-02-17T12:48:19.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.114+0000 7f8b017fa700 1 -- 172.21.15.18:0/404374025 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8af80ce510 con 0x7f8b0406ad10 2024-02-17T12:48:19.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.114+0000 7f8b03fff700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8aec0a21e0 0x7f8aec0a46a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009990 tx=0x7f8af4005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:19.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.114+0000 7f8b089a4700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818] conn(0x7f8aec0ab9c0 0x7f8aec0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:19.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.115+0000 7f8b089a4700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818] conn(0x7f8aec0ab9c0 0x7f8aec0adde0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:19.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.116+0000 7f8b017fa700 1 -- 172.21.15.18:0/404374025 <== osd.4 v2:172.21.15.119:6808/1444866818 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8aec0ae490 con 0x7f8aec0ab9c0 2024-02-17T12:48:19.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.132+0000 7f8b0a407700 1 -- 172.21.15.18:0/404374025 --> [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8b04062300 con 0x7f8aec0ab9c0 2024-02-17T12:48:19.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.133+0000 7f8b017fa700 1 -- 172.21.15.18:0/404374025 <== osd.4 v2:172.21.15.119:6808/1444866818 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8b04062300 con 0x7f8aec0ab9c0 2024-02-17T12:48:19.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.133+0000 7f8b0a407700 1 -- 172.21.15.18:0/404374025 >> [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818] conn(0x7f8aec0ab9c0 msgr2=0x7f8aec0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:19.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.133+0000 7f8b0a407700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818] conn(0x7f8aec0ab9c0 0x7f8aec0adde0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:19.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.133+0000 7f8b0a407700 1 -- 172.21.15.18:0/404374025 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8aec0a21e0 msgr2=0x7f8aec0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:19.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.133+0000 7f8b0a407700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8aec0a21e0 0x7f8aec0a46a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009990 tx=0x7f8af4005bc0 comp rx=0 tx=0).stop 2024-02-17T12:48:19.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.133+0000 7f8b0a407700 1 -- 172.21.15.18:0/404374025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b0406ad10 msgr2=0x7f8b04198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:19.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.133+0000 7f8b0a407700 1 --2- 172.21.15.18:0/404374025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b0406ad10 0x7f8b04198730 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8af800d950 tx=0x7f8af800dc60 comp rx=0 tx=0).stop 2024-02-17T12:48:19.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.133+0000 7f8b089a4700 1 -- 172.21.15.18:0/404374025 reap_dead start 2024-02-17T12:48:19.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.133+0000 7f8b0a407700 1 -- 172.21.15.18:0/404374025 shutdown_connections 2024-02-17T12:48:19.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.133+0000 7f8b0a407700 1 -- 172.21.15.18:0/404374025 >> 172.21.15.18:0/404374025 conn(0x7f8b04076b90 msgr2=0x7f8b040ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:19.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.133+0000 7f8b0a407700 1 -- 172.21.15.18:0/404374025 shutdown_connections 2024-02-17T12:48:19.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:19.133+0000 7f8b0a407700 1 -- 172.21.15.18:0/404374025 wait complete. 2024-02-17T12:48:19.395 INFO:teuthology.orchestra.run.smithi018.stdout:34359738442 2024-02-17T12:48:19.396 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-17T12:48:19.976 INFO:teuthology.orchestra.run.smithi018.stdout:197568495660 2024-02-17T12:48:19.976 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-17T12:48:20.252 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:20 smithi018 ceph-mon[32625]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-17T12:48:20.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:20 smithi146 ceph-mon[36454]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-17T12:48:20.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:20 smithi119 ceph-mon[36188]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-17T12:48:20.643 INFO:teuthology.orchestra.run.smithi018.stdout:141733920823 2024-02-17T12:48:20.644 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-17T12:48:22.226 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:22 smithi018 ceph-mon[32625]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-02-17T12:48:22.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:22 smithi146 ceph-mon[36454]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-02-17T12:48:22.563 INFO:teuthology.orchestra.run.smithi018.stdout:55834574917 2024-02-17T12:48:22.563 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-17T12:48:22.581 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:22.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:22 smithi119 ceph-mon[36188]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-02-17T12:48:23.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.834+0000 7f438c4fd700 1 -- 172.21.15.18:0/1819506970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4384100e00 msgr2=0x7f43841011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:23.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.834+0000 7f438c4fd700 1 --2- 172.21.15.18:0/1819506970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4384100e00 0x7f43841011e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4380009990 tx=0x7f4380009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:23.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.834+0000 7f438c4fd700 1 -- 172.21.15.18:0/1819506970 shutdown_connections 2024-02-17T12:48:23.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.834+0000 7f438c4fd700 1 --2- 172.21.15.18:0/1819506970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4384105e90 0x7f438410da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:23.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.834+0000 7f438c4fd700 1 --2- 172.21.15.18:0/1819506970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4384100e00 0x7f43841011e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:23.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.834+0000 7f438c4fd700 1 --2- 172.21.15.18:0/1819506970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43841017b0 0x7f43841057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.834+0000 7f438c4fd700 1 -- 172.21.15.18:0/1819506970 >> 172.21.15.18:0/1819506970 conn(0x7f4384076b90 msgr2=0x7f4384076fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.835+0000 7f438c4fd700 1 -- 172.21.15.18:0/1819506970 shutdown_connections 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.835+0000 7f438c4fd700 1 -- 172.21.15.18:0/1819506970 wait complete. 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.835+0000 7f438c4fd700 1 Processor -- start 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.835+0000 7f438c4fd700 1 -- start start 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438c4fd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4384100e00 0x7f4384199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438c4fd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43841017b0 0x7f4384199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438c4fd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4384105e90 0x7f43841a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438c4fd700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4384199dc0 con 0x7f4384100e00 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438c4fd700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43841a45f0 con 0x7f4384105e90 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438c4fd700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43841a4730 con 0x7f43841017b0 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438a299700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4384100e00 0x7f4384199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438a299700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4384100e00 0x7f4384199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49814/0 (socket says 172.21.15.18:49814) 2024-02-17T12:48:23.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438a299700 1 -- 172.21.15.18:0/1652116563 learned_addr learned my addr 172.21.15.18:0/1652116563 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438aa9a700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4384105e90 0x7f43841a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f4389a98700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43841017b0 0x7f4384199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438a299700 1 -- 172.21.15.18:0/1652116563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43841017b0 msgr2=0x7f4384199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438a299700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43841017b0 0x7f4384199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438a299700 1 -- 172.21.15.18:0/1652116563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4384105e90 msgr2=0x7f43841a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438a299700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4384105e90 0x7f43841a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438a299700 1 -- 172.21.15.18:0/1652116563 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43800095f0 con 0x7f4384100e00 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.836+0000 7f438a299700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4384100e00 0x7f4384199030 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4380000c00 tx=0x7f438000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.837+0000 7f437b7fe700 1 -- 172.21.15.18:0/1652116563 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438001d070 con 0x7f4384100e00 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.837+0000 7f438c4fd700 1 -- 172.21.15.18:0/1652116563 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4384196e20 con 0x7f4384100e00 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.837+0000 7f437b7fe700 1 -- 172.21.15.18:0/1652116563 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4380003680 con 0x7f4384100e00 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.837+0000 7f437b7fe700 1 -- 172.21.15.18:0/1652116563 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43800173f0 con 0x7f4384100e00 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.837+0000 7f438c4fd700 1 -- 172.21.15.18:0/1652116563 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43841972c0 con 0x7f4384100e00 2024-02-17T12:48:23.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.837+0000 7f438c4fd700 1 -- 172.21.15.18:0/1652116563 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4384110ea0 con 0x7f4384100e00 2024-02-17T12:48:23.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.839+0000 7f437b7fe700 1 -- 172.21.15.18:0/1652116563 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4380022720 con 0x7f4384100e00 2024-02-17T12:48:23.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.839+0000 7f437b7fe700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f43700a21e0 0x7f43700a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:23.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.840+0000 7f4389a98700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f43700a21e0 0x7f43700a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:23.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.840+0000 7f437b7fe700 1 -- 172.21.15.18:0/1652116563 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f43800cf500 con 0x7f4384100e00 2024-02-17T12:48:23.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.840+0000 7f437b7fe700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109] conn(0x7f43700ab9c0 0x7f43700adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:23.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.840+0000 7f437b7fe700 1 -- 172.21.15.18:0/1652116563 --> [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43700ae490 con 0x7f43700ab9c0 2024-02-17T12:48:23.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.840+0000 7f437b7fe700 1 -- 172.21.15.18:0/1652116563 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43800cf9a0 con 0x7f4384100e00 2024-02-17T12:48:23.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.840+0000 7f4389a98700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f43700a21e0 0x7f43700a46a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4374005950 tx=0x7f43740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:23.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.840+0000 7f438aa9a700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109] conn(0x7f43700ab9c0 0x7f43700adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:23.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.841+0000 7f438aa9a700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109] conn(0x7f43700ab9c0 0x7f43700adde0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:23.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.842+0000 7f437b7fe700 1 -- 172.21.15.18:0/1652116563 <== osd.11 v2:172.21.15.146:6824/2169300109 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f43700ae490 con 0x7f43700ab9c0 2024-02-17T12:48:23.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.858+0000 7f438c4fd700 1 -- 172.21.15.18:0/1652116563 --> [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4384062300 con 0x7f43700ab9c0 2024-02-17T12:48:23.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.858+0000 7f437b7fe700 1 -- 172.21.15.18:0/1652116563 <== osd.11 v2:172.21.15.146:6824/2169300109 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4384062300 con 0x7f43700ab9c0 2024-02-17T12:48:23.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.859+0000 7f438c4fd700 1 -- 172.21.15.18:0/1652116563 >> [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109] conn(0x7f43700ab9c0 msgr2=0x7f43700adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:23.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.859+0000 7f438c4fd700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109] conn(0x7f43700ab9c0 0x7f43700adde0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:23.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.859+0000 7f438c4fd700 1 -- 172.21.15.18:0/1652116563 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f43700a21e0 msgr2=0x7f43700a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:23.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.859+0000 7f438c4fd700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f43700a21e0 0x7f43700a46a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4374005950 tx=0x7f43740058e0 comp rx=0 tx=0).stop 2024-02-17T12:48:23.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.859+0000 7f438c4fd700 1 -- 172.21.15.18:0/1652116563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4384100e00 msgr2=0x7f4384199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:23.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.859+0000 7f438c4fd700 1 --2- 172.21.15.18:0/1652116563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4384100e00 0x7f4384199030 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4380000c00 tx=0x7f438000f690 comp rx=0 tx=0).stop 2024-02-17T12:48:23.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.859+0000 7f438aa9a700 1 -- 172.21.15.18:0/1652116563 reap_dead start 2024-02-17T12:48:23.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.859+0000 7f438c4fd700 1 -- 172.21.15.18:0/1652116563 shutdown_connections 2024-02-17T12:48:23.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.859+0000 7f438c4fd700 1 -- 172.21.15.18:0/1652116563 >> 172.21.15.18:0/1652116563 conn(0x7f4384076b90 msgr2=0x7f4384100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:23.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.859+0000 7f438c4fd700 1 -- 172.21.15.18:0/1652116563 shutdown_connections 2024-02-17T12:48:23.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:23.859+0000 7f438c4fd700 1 -- 172.21.15.18:0/1652116563 wait complete. 2024-02-17T12:48:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.304+0000 7f2097376700 1 -- 172.21.15.18:0/2339730351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2090102900 msgr2=0x7f20901068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.304+0000 7f2097376700 1 --2- 172.21.15.18:0/2339730351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2090102900 0x7f20901068a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2080009990 tx=0x7f2080009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.304+0000 7f2097376700 1 -- 172.21.15.18:0/2339730351 shutdown_connections 2024-02-17T12:48:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.304+0000 7f2097376700 1 --2- 172.21.15.18:0/2339730351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2090106f80 0x7f209010eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.304+0000 7f2097376700 1 --2- 172.21.15.18:0/2339730351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2090101f50 0x7f2090102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.304+0000 7f2097376700 1 --2- 172.21.15.18:0/2339730351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2090102900 0x7f20901068a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.304+0000 7f2097376700 1 -- 172.21.15.18:0/2339730351 >> 172.21.15.18:0/2339730351 conn(0x7f2090078d00 msgr2=0x7f2090079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.304+0000 7f2097376700 1 -- 172.21.15.18:0/2339730351 shutdown_connections 2024-02-17T12:48:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.304+0000 7f2097376700 1 -- 172.21.15.18:0/2339730351 wait complete. 2024-02-17T12:48:24.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2097376700 1 Processor -- start 2024-02-17T12:48:24.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2097376700 1 -- start start 2024-02-17T12:48:24.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2097376700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2090101f50 0x7f2090198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2097376700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2090102900 0x7f2090198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2097376700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2090106f80 0x7f209019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2097376700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209019d0f0 con 0x7f2090106f80 2024-02-17T12:48:24.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2097376700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209019d260 con 0x7f2090102900 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2097376700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209019d420 con 0x7f2090101f50 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2095913700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2090106f80 0x7f209019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2095913700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2090106f80 0x7f209019cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49832/0 (socket says 172.21.15.18:49832) 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2095913700 1 -- 172.21.15.18:0/870148871 learned_addr learned my addr 172.21.15.18:0/870148871 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2095112700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2090101f50 0x7f2090198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2094911700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2090102900 0x7f2090198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2095913700 1 -- 172.21.15.18:0/870148871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2090101f50 msgr2=0x7f2090198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2095913700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2090101f50 0x7f2090198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2095913700 1 -- 172.21.15.18:0/870148871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2090102900 msgr2=0x7f2090198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2095913700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2090102900 0x7f2090198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2095913700 1 -- 172.21.15.18:0/870148871 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20800095f0 con 0x7f2090106f80 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.305+0000 7f2094911700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2090102900 0x7f2090198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.306+0000 7f2095913700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2090106f80 0x7f209019cae0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2088007ee0 tx=0x7f208800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:24.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.306+0000 7f20867fc700 1 -- 172.21.15.18:0/870148871 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088017070 con 0x7f2090106f80 2024-02-17T12:48:24.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.306+0000 7f2097376700 1 -- 172.21.15.18:0/870148871 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209019d700 con 0x7f2090106f80 2024-02-17T12:48:24.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.306+0000 7f20867fc700 1 -- 172.21.15.18:0/870148871 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f208800f040 con 0x7f2090106f80 2024-02-17T12:48:24.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.306+0000 7f20867fc700 1 -- 172.21.15.18:0/870148871 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088005760 con 0x7f2090106f80 2024-02-17T12:48:24.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.306+0000 7f2097376700 1 -- 172.21.15.18:0/870148871 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20901a4f60 con 0x7f2090106f80 2024-02-17T12:48:24.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.307+0000 7f2097376700 1 -- 172.21.15.18:0/870148871 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2090112030 con 0x7f2090106f80 2024-02-17T12:48:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.308+0000 7f20867fc700 1 -- 172.21.15.18:0/870148871 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f20880058c0 con 0x7f2090106f80 2024-02-17T12:48:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.308+0000 7f20867fc700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f207c0a21e0 0x7f207c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.309+0000 7f2095112700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f207c0a21e0 0x7f207c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.309+0000 7f20867fc700 1 -- 172.21.15.18:0/870148871 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f20880cdcc0 con 0x7f2090106f80 2024-02-17T12:48:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.309+0000 7f20867fc700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241] conn(0x7f207c0ab9c0 0x7f207c0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.309+0000 7f20867fc700 1 -- 172.21.15.18:0/870148871 --> [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f207c0ae490 con 0x7f207c0ab9c0 2024-02-17T12:48:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.309+0000 7f20867fc700 1 -- 172.21.15.18:0/870148871 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20880ce030 con 0x7f2090106f80 2024-02-17T12:48:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.309+0000 7f2095112700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f207c0a21e0 0x7f207c0a46a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f208c009a10 tx=0x7f208c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:24.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.309+0000 7f2094911700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241] conn(0x7f207c0ab9c0 0x7f207c0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.310+0000 7f2094911700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241] conn(0x7f207c0ab9c0 0x7f207c0adde0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:24.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.310+0000 7f20867fc700 1 -- 172.21.15.18:0/870148871 <== osd.8 v2:172.21.15.146:6816/1027539241 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f207c0ae490 con 0x7f207c0ab9c0 2024-02-17T12:48:24.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.327+0000 7f2097376700 1 -- 172.21.15.18:0/870148871 --> [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2090062380 con 0x7f207c0ab9c0 2024-02-17T12:48:24.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.327+0000 7f20867fc700 1 -- 172.21.15.18:0/870148871 <== osd.8 v2:172.21.15.146:6816/1027539241 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2090062380 con 0x7f207c0ab9c0 2024-02-17T12:48:24.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.328+0000 7f2097376700 1 -- 172.21.15.18:0/870148871 >> [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241] conn(0x7f207c0ab9c0 msgr2=0x7f207c0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.328+0000 7f2097376700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241] conn(0x7f207c0ab9c0 0x7f207c0adde0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.328+0000 7f2097376700 1 -- 172.21.15.18:0/870148871 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f207c0a21e0 msgr2=0x7f207c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.328+0000 7f2097376700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f207c0a21e0 0x7f207c0a46a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f208c009a10 tx=0x7f208c009190 comp rx=0 tx=0).stop 2024-02-17T12:48:24.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.328+0000 7f2097376700 1 -- 172.21.15.18:0/870148871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2090106f80 msgr2=0x7f209019cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.328+0000 7f2097376700 1 --2- 172.21.15.18:0/870148871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2090106f80 0x7f209019cae0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2088007ee0 tx=0x7f208800d3b0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.328+0000 7f2095913700 1 -- 172.21.15.18:0/870148871 reap_dead start 2024-02-17T12:48:24.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.328+0000 7f2097376700 1 -- 172.21.15.18:0/870148871 shutdown_connections 2024-02-17T12:48:24.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.328+0000 7f2097376700 1 -- 172.21.15.18:0/870148871 >> 172.21.15.18:0/870148871 conn(0x7f2090078d00 msgr2=0x7f2090101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:24.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.328+0000 7f2097376700 1 -- 172.21.15.18:0/870148871 shutdown_connections 2024-02-17T12:48:24.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.328+0000 7f2097376700 1 -- 172.21.15.18:0/870148871 wait complete. 2024-02-17T12:48:24.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.873+0000 7f37d3032700 1 -- 172.21.15.18:0/3243600565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37cc105e80 msgr2=0x7f37cc10da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.873+0000 7f37d3032700 1 --2- 172.21.15.18:0/3243600565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37cc105e80 0x7f37cc10da50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f37c400b0a0 tx=0x7f37c400b3b0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.873+0000 7f37d3032700 1 -- 172.21.15.18:0/3243600565 shutdown_connections 2024-02-17T12:48:24.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.873+0000 7f37d3032700 1 --2- 172.21.15.18:0/3243600565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37cc105e80 0x7f37cc10da50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.873+0000 7f37d3032700 1 --2- 172.21.15.18:0/3243600565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc100dc0 0x7f37cc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.873+0000 7f37d3032700 1 --2- 172.21.15.18:0/3243600565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37cc101770 0x7f37cc1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.873+0000 7f37d3032700 1 -- 172.21.15.18:0/3243600565 >> 172.21.15.18:0/3243600565 conn(0x7f37cc077ae0 msgr2=0x7f37cc077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:24.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.873+0000 7f37d3032700 1 -- 172.21.15.18:0/3243600565 shutdown_connections 2024-02-17T12:48:24.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.873+0000 7f37d3032700 1 -- 172.21.15.18:0/3243600565 wait complete. 2024-02-17T12:48:24.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.873+0000 7f37d3032700 1 Processor -- start 2024-02-17T12:48:24.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d3032700 1 -- start start 2024-02-17T12:48:24.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d3032700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37cc100dc0 0x7f37cc199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d3032700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc101770 0x7f37cc199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d3032700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37cc105e80 0x7f37cc1a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d3032700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc19a020 con 0x7f37cc100dc0 2024-02-17T12:48:24.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d3032700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc1a4800 con 0x7f37cc105e80 2024-02-17T12:48:24.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d3032700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc1a4940 con 0x7f37cc101770 2024-02-17T12:48:24.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d0dce700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37cc100dc0 0x7f37cc199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d0dce700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37cc100dc0 0x7f37cc199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49794/0 (socket says 172.21.15.18:49794) 2024-02-17T12:48:24.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d0dce700 1 -- 172.21.15.18:0/4174652902 learned_addr learned my addr 172.21.15.18:0/4174652902 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:24.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37cbfff700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc101770 0x7f37cc199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d15cf700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37cc105e80 0x7f37cc1a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d0dce700 1 -- 172.21.15.18:0/4174652902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc101770 msgr2=0x7f37cc199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d0dce700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc101770 0x7f37cc199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d0dce700 1 -- 172.21.15.18:0/4174652902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37cc105e80 msgr2=0x7f37cc1a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d0dce700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37cc105e80 0x7f37cc1a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d0dce700 1 -- 172.21.15.18:0/4174652902 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c4009d00 con 0x7f37cc100dc0 2024-02-17T12:48:24.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37cbfff700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37cc101770 0x7f37cc199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:48:24.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d15cf700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37cc105e80 0x7f37cc1a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:48:24.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37d0dce700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37cc100dc0 0x7f37cc199240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00b560 tx=0x7f37bc00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:24.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.874+0000 7f37c9ffb700 1 -- 172.21.15.18:0/4174652902 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc011830 con 0x7f37cc100dc0 2024-02-17T12:48:24.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.875+0000 7f37c9ffb700 1 -- 172.21.15.18:0/4174652902 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37bc011e70 con 0x7f37cc100dc0 2024-02-17T12:48:24.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.875+0000 7f37d3032700 1 -- 172.21.15.18:0/4174652902 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37cc1a4bf0 con 0x7f37cc100dc0 2024-02-17T12:48:24.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.875+0000 7f37c9ffb700 1 -- 172.21.15.18:0/4174652902 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00f2e0 con 0x7f37cc100dc0 2024-02-17T12:48:24.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.875+0000 7f37d3032700 1 -- 172.21.15.18:0/4174652902 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc1a5140 con 0x7f37cc100dc0 2024-02-17T12:48:24.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.876+0000 7f37d3032700 1 -- 172.21.15.18:0/4174652902 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f37cc110f50 con 0x7f37cc100dc0 2024-02-17T12:48:24.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.877+0000 7f37c9ffb700 1 -- 172.21.15.18:0/4174652902 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f37bc011990 con 0x7f37cc100dc0 2024-02-17T12:48:24.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.877+0000 7f37c9ffb700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f37b40a2260 0x7f37b40a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.877+0000 7f37cbfff700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f37b40a2260 0x7f37b40a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.877+0000 7f37c9ffb700 1 -- 172.21.15.18:0/4174652902 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f37bc0cd630 con 0x7f37cc100dc0 2024-02-17T12:48:24.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.877+0000 7f37c9ffb700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833] conn(0x7f37b40aba40 0x7f37b40ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.877+0000 7f37c9ffb700 1 -- 172.21.15.18:0/4174652902 --> [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37b40ae510 con 0x7f37b40aba40 2024-02-17T12:48:24.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.877+0000 7f37c9ffb700 1 -- 172.21.15.18:0/4174652902 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37bc0cfd40 con 0x7f37cc100dc0 2024-02-17T12:48:24.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.878+0000 7f37d15cf700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833] conn(0x7f37b40aba40 0x7f37b40ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.878+0000 7f37cbfff700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f37b40a2260 0x7f37b40a4720 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0ff680 tx=0x7f37c0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:24.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.878+0000 7f37d15cf700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833] conn(0x7f37b40aba40 0x7f37b40ade60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:24.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.879+0000 7f37c9ffb700 1 -- 172.21.15.18:0/4174652902 <== osd.9 v2:172.21.15.18:6826/3939472833 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f37b40ae510 con 0x7f37b40aba40 2024-02-17T12:48:24.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.895+0000 7f37d3032700 1 -- 172.21.15.18:0/4174652902 --> [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f37cc062330 con 0x7f37b40aba40 2024-02-17T12:48:24.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.895+0000 7f37c9ffb700 1 -- 172.21.15.18:0/4174652902 <== osd.9 v2:172.21.15.18:6826/3939472833 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f37cc062330 con 0x7f37b40aba40 2024-02-17T12:48:24.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.895+0000 7f37d3032700 1 -- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833] conn(0x7f37b40aba40 msgr2=0x7f37b40ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.895+0000 7f37d3032700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833] conn(0x7f37b40aba40 0x7f37b40ade60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.895+0000 7f37d3032700 1 -- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f37b40a2260 msgr2=0x7f37b40a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.895+0000 7f37d3032700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f37b40a2260 0x7f37b40a4720 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0ff680 tx=0x7f37c0008040 comp rx=0 tx=0).stop 2024-02-17T12:48:24.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.895+0000 7f37d3032700 1 -- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37cc100dc0 msgr2=0x7f37cc199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.895+0000 7f37d3032700 1 --2- 172.21.15.18:0/4174652902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37cc100dc0 0x7f37cc199240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00b560 tx=0x7f37bc00b920 comp rx=0 tx=0).stop 2024-02-17T12:48:24.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.895+0000 7f37d15cf700 1 -- 172.21.15.18:0/4174652902 reap_dead start 2024-02-17T12:48:24.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.896+0000 7f37d3032700 1 -- 172.21.15.18:0/4174652902 shutdown_connections 2024-02-17T12:48:24.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.896+0000 7f37d3032700 1 -- 172.21.15.18:0/4174652902 >> 172.21.15.18:0/4174652902 conn(0x7f37cc077ae0 msgr2=0x7f37cc1002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:24.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.896+0000 7f37d3032700 1 -- 172.21.15.18:0/4174652902 shutdown_connections 2024-02-17T12:48:24.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:22.896+0000 7f37d3032700 1 -- 172.21.15.18:0/4174652902 wait complete. 2024-02-17T12:48:24.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:24 smithi018 ceph-mon[32625]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-17T12:48:24.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:24 smithi146 ceph-mon[36454]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-17T12:48:24.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:24 smithi119 ceph-mon[36188]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-17T12:48:24.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.908+0000 7fe468f72700 1 -- 172.21.15.18:0/2822332630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4641070b0 msgr2=0x7fe46410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.908+0000 7fe468f72700 1 --2- 172.21.15.18:0/2822332630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4641070b0 0x7fe46410ec30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00b0d0 tx=0x7fe45c00b3e0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.909+0000 7fe468f72700 1 -- 172.21.15.18:0/2822332630 shutdown_connections 2024-02-17T12:48:24.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.909+0000 7fe468f72700 1 --2- 172.21.15.18:0/2822332630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4641070b0 0x7fe46410ec30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.909+0000 7fe468f72700 1 --2- 172.21.15.18:0/2822332630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe46406a610 0x7fe46406a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.909+0000 7fe468f72700 1 --2- 172.21.15.18:0/2822332630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46406afc0 0x7fe464106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.909+0000 7fe468f72700 1 -- 172.21.15.18:0/2822332630 >> 172.21.15.18:0/2822332630 conn(0x7fe4640779d0 msgr2=0x7fe464077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:24.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.909+0000 7fe468f72700 1 -- 172.21.15.18:0/2822332630 shutdown_connections 2024-02-17T12:48:24.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.909+0000 7fe468f72700 1 -- 172.21.15.18:0/2822332630 wait complete. 2024-02-17T12:48:24.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.909+0000 7fe468f72700 1 Processor -- start 2024-02-17T12:48:24.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.909+0000 7fe468f72700 1 -- start start 2024-02-17T12:48:24.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe468f72700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe46406a610 0x7fe464198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe468f72700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46406afc0 0x7fe464198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe468f72700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4641070b0 0x7fe46419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe468f72700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46419d110 con 0x7fe46406a610 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe468f72700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46419d280 con 0x7fe46406afc0 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe468f72700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46419d440 con 0x7fe4641070b0 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe46259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe46406a610 0x7fe464198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe46259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe46406a610 0x7fe464198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49848/0 (socket says 172.21.15.18:49848) 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe46259c700 1 -- 172.21.15.18:0/3552434006 learned_addr learned my addr 172.21.15.18:0/3552434006 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe461d9b700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46406afc0 0x7fe464198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe462d9d700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4641070b0 0x7fe46419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe46259c700 1 -- 172.21.15.18:0/3552434006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4641070b0 msgr2=0x7fe46419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe46259c700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4641070b0 0x7fe46419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe46259c700 1 -- 172.21.15.18:0/3552434006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46406afc0 msgr2=0x7fe464198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe46259c700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46406afc0 0x7fe464198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe46259c700 1 -- 172.21.15.18:0/3552434006 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c009d00 con 0x7fe46406a610 2024-02-17T12:48:24.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe46259c700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe46406a610 0x7fe464198230 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe45400c9b0 tx=0x7fe45400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:24.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe4537fe700 1 -- 172.21.15.18:0/3552434006 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454007b00 con 0x7fe46406a610 2024-02-17T12:48:24.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe468f72700 1 -- 172.21.15.18:0/3552434006 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46419d720 con 0x7fe46406a610 2024-02-17T12:48:24.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe4537fe700 1 -- 172.21.15.18:0/3552434006 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe454007c60 con 0x7fe46406a610 2024-02-17T12:48:24.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.910+0000 7fe4537fe700 1 -- 172.21.15.18:0/3552434006 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454018660 con 0x7fe46406a610 2024-02-17T12:48:24.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.911+0000 7fe468f72700 1 -- 172.21.15.18:0/3552434006 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4641a4f80 con 0x7fe46406a610 2024-02-17T12:48:24.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.911+0000 7fe468f72700 1 -- 172.21.15.18:0/3552434006 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe4641120d0 con 0x7fe46406a610 2024-02-17T12:48:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.913+0000 7fe4537fe700 1 -- 172.21.15.18:0/3552434006 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe45400f450 con 0x7fe46406a610 2024-02-17T12:48:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.913+0000 7fe4537fe700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe44c0a21e0 0x7fe44c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.913+0000 7fe461d9b700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe44c0a21e0 0x7fe44c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.913+0000 7fe4537fe700 1 -- 172.21.15.18:0/3552434006 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe4540ce0c0 con 0x7fe46406a610 2024-02-17T12:48:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.913+0000 7fe4537fe700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651] conn(0x7fe44c0ab9c0 0x7fe44c0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.913+0000 7fe4537fe700 1 -- 172.21.15.18:0/3552434006 --> [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe44c0ae490 con 0x7fe44c0ab9c0 2024-02-17T12:48:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.913+0000 7fe4537fe700 1 -- 172.21.15.18:0/3552434006 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4540ce560 con 0x7fe46406a610 2024-02-17T12:48:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.914+0000 7fe461d9b700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe44c0a21e0 0x7fe44c0a46a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe464108240 tx=0x7fe458009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.914+0000 7fe462d9d700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651] conn(0x7fe44c0ab9c0 0x7fe44c0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:24.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.914+0000 7fe462d9d700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651] conn(0x7fe44c0ab9c0 0x7fe44c0adde0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:24.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.915+0000 7fe4537fe700 1 -- 172.21.15.18:0/3552434006 <== osd.5 v2:172.21.15.146:6808/4227453651 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe44c0ae490 con 0x7fe44c0ab9c0 2024-02-17T12:48:24.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.931+0000 7fe468f72700 1 -- 172.21.15.18:0/3552434006 --> [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe4640ff620 con 0x7fe44c0ab9c0 2024-02-17T12:48:24.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.932+0000 7fe4537fe700 1 -- 172.21.15.18:0/3552434006 <== osd.5 v2:172.21.15.146:6808/4227453651 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe4640ff620 con 0x7fe44c0ab9c0 2024-02-17T12:48:24.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.932+0000 7fe468f72700 1 -- 172.21.15.18:0/3552434006 >> [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651] conn(0x7fe44c0ab9c0 msgr2=0x7fe44c0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.932+0000 7fe468f72700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651] conn(0x7fe44c0ab9c0 0x7fe44c0adde0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.932+0000 7fe468f72700 1 -- 172.21.15.18:0/3552434006 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe44c0a21e0 msgr2=0x7fe44c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.932+0000 7fe468f72700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe44c0a21e0 0x7fe44c0a46a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe464108240 tx=0x7fe458009190 comp rx=0 tx=0).stop 2024-02-17T12:48:24.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.932+0000 7fe468f72700 1 -- 172.21.15.18:0/3552434006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe46406a610 msgr2=0x7fe464198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:24.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.932+0000 7fe468f72700 1 --2- 172.21.15.18:0/3552434006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe46406a610 0x7fe464198230 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe45400c9b0 tx=0x7fe45400ccc0 comp rx=0 tx=0).stop 2024-02-17T12:48:24.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.932+0000 7fe462d9d700 1 -- 172.21.15.18:0/3552434006 reap_dead start 2024-02-17T12:48:24.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.932+0000 7fe468f72700 1 -- 172.21.15.18:0/3552434006 shutdown_connections 2024-02-17T12:48:24.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.932+0000 7fe468f72700 1 -- 172.21.15.18:0/3552434006 >> 172.21.15.18:0/3552434006 conn(0x7fe4640779d0 msgr2=0x7fe4640fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:24.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.932+0000 7fe468f72700 1 -- 172.21.15.18:0/3552434006 shutdown_connections 2024-02-17T12:48:24.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:24.932+0000 7fe468f72700 1 -- 172.21.15.18:0/3552434006 wait complete. 2024-02-17T12:48:25.420 INFO:teuthology.orchestra.run.smithi018.stdout:292057776159 2024-02-17T12:48:25.420 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-02-17T12:48:25.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:25.999+0000 7f8e2edfd700 1 -- 172.21.15.18:0/425870421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e28106b80 msgr2=0x7f8e28106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:25.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:25.999+0000 7f8e2edfd700 1 --2- 172.21.15.18:0/425870421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e28106b80 0x7f8e28106f60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009990 tx=0x7f8e1c009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:25.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:25.999+0000 7f8e2edfd700 1 -- 172.21.15.18:0/425870421 shutdown_connections 2024-02-17T12:48:25.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:25.999+0000 7f8e2edfd700 1 --2- 172.21.15.18:0/425870421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e28101530 0x7f8e2810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:25.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:25.999+0000 7f8e2edfd700 1 --2- 172.21.15.18:0/425870421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e28106b80 0x7f8e28106f60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:25.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:25.999+0000 7f8e2edfd700 1 --2- 172.21.15.18:0/425870421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28100b70 0x7f8e28100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:25.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:25.999+0000 7f8e2edfd700 1 -- 172.21.15.18:0/425870421 >> 172.21.15.18:0/425870421 conn(0x7f8e28077aa0 msgr2=0x7f8e28077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:25.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:25.999+0000 7f8e2edfd700 1 -- 172.21.15.18:0/425870421 shutdown_connections 2024-02-17T12:48:25.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:25.999+0000 7f8e2edfd700 1 -- 172.21.15.18:0/425870421 wait complete. 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e2edfd700 1 Processor -- start 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e2edfd700 1 -- start start 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e2edfd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e28100b70 0x7f8e28198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e2edfd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e28101530 0x7f8e28198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e2edfd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28106b80 0x7f8e2819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e2edfd700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2819cfc0 con 0x7f8e28101530 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e2edfd700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2819d130 con 0x7f8e28106b80 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e2edfd700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2819d2f0 con 0x7f8e28100b70 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e28101530 0x7f8e28198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e28101530 0x7f8e28198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49878/0 (socket says 172.21.15.18:49878) 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e27fff700 1 -- 172.21.15.18:0/3973210385 learned_addr learned my addr 172.21.15.18:0/3973210385 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e2cb99700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e28100b70 0x7f8e28198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:25.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e27fff700 1 -- 172.21.15.18:0/3973210385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e28100b70 msgr2=0x7f8e28198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:25.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e27fff700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e28100b70 0x7f8e28198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:25.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e2d39a700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28106b80 0x7f8e2819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:25.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e27fff700 1 -- 172.21.15.18:0/3973210385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28106b80 msgr2=0x7f8e2819c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:25.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e27fff700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28106b80 0x7f8e2819c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:25.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e27fff700 1 -- 172.21.15.18:0/3973210385 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c0095f0 con 0x7f8e28101530 2024-02-17T12:48:25.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.000+0000 7f8e2d39a700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28106b80 0x7f8e2819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:48:25.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.001+0000 7f8e27fff700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e28101530 0x7f8e28198670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400c920 tx=0x7f8e1400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:25.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.001+0000 7f8e25ffb700 1 -- 172.21.15.18:0/3973210385 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e14007b00 con 0x7f8e28101530 2024-02-17T12:48:25.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.001+0000 7f8e25ffb700 1 -- 172.21.15.18:0/3973210385 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e14004d10 con 0x7f8e28101530 2024-02-17T12:48:25.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.001+0000 7f8e2edfd700 1 -- 172.21.15.18:0/3973210385 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2819d5d0 con 0x7f8e28101530 2024-02-17T12:48:25.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.001+0000 7f8e25ffb700 1 -- 172.21.15.18:0/3973210385 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e140056c0 con 0x7f8e28101530 2024-02-17T12:48:25.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.001+0000 7f8e2edfd700 1 -- 172.21.15.18:0/3973210385 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e281a4e30 con 0x7f8e28101530 2024-02-17T12:48:25.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.002+0000 7f8e2edfd700 1 -- 172.21.15.18:0/3973210385 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e28110e60 con 0x7f8e28101530 2024-02-17T12:48:25.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.003+0000 7f8e25ffb700 1 -- 172.21.15.18:0/3973210385 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f8e14007c60 con 0x7f8e28101530 2024-02-17T12:48:25.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.003+0000 7f8e25ffb700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8e180a21e0 0x7f8e180a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:25.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.003+0000 7f8e2cb99700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8e180a21e0 0x7f8e180a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:25.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.004+0000 7f8e25ffb700 1 -- 172.21.15.18:0/3973210385 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8e140cdd30 con 0x7f8e28101530 2024-02-17T12:48:25.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.004+0000 7f8e25ffb700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737] conn(0x7f8e180ab9c0 0x7f8e180adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:25.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.004+0000 7f8e25ffb700 1 -- 172.21.15.18:0/3973210385 --> [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e180ae490 con 0x7f8e180ab9c0 2024-02-17T12:48:25.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.004+0000 7f8e25ffb700 1 -- 172.21.15.18:0/3973210385 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e140ce1d0 con 0x7f8e28101530 2024-02-17T12:48:25.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.004+0000 7f8e2cb99700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8e180a21e0 0x7f8e180a46a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c006010 tx=0x7f8e1c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:25.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.004+0000 7f8e2d39a700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737] conn(0x7f8e180ab9c0 0x7f8e180adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:25.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.004+0000 7f8e2d39a700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737] conn(0x7f8e180ab9c0 0x7f8e180adde0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:25.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.005+0000 7f8e25ffb700 1 -- 172.21.15.18:0/3973210385 <== osd.7 v2:172.21.15.119:6816/833114737 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8e180ae490 con 0x7f8e180ab9c0 2024-02-17T12:48:26.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.022+0000 7f8e2edfd700 1 -- 172.21.15.18:0/3973210385 --> [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e28104bc0 con 0x7f8e180ab9c0 2024-02-17T12:48:26.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.023+0000 7f8e25ffb700 1 -- 172.21.15.18:0/3973210385 <== osd.7 v2:172.21.15.119:6816/833114737 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8e28104bc0 con 0x7f8e180ab9c0 2024-02-17T12:48:26.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.023+0000 7f8e2edfd700 1 -- 172.21.15.18:0/3973210385 >> [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737] conn(0x7f8e180ab9c0 msgr2=0x7f8e180adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:26.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.023+0000 7f8e2edfd700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737] conn(0x7f8e180ab9c0 0x7f8e180adde0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:26.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.023+0000 7f8e2edfd700 1 -- 172.21.15.18:0/3973210385 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8e180a21e0 msgr2=0x7f8e180a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:26.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.023+0000 7f8e2edfd700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8e180a21e0 0x7f8e180a46a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c006010 tx=0x7f8e1c01a040 comp rx=0 tx=0).stop 2024-02-17T12:48:26.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.023+0000 7f8e2edfd700 1 -- 172.21.15.18:0/3973210385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e28101530 msgr2=0x7f8e28198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:26.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.023+0000 7f8e2edfd700 1 --2- 172.21.15.18:0/3973210385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e28101530 0x7f8e28198670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400c920 tx=0x7f8e1400cce0 comp rx=0 tx=0).stop 2024-02-17T12:48:26.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.023+0000 7f8e2d39a700 1 -- 172.21.15.18:0/3973210385 reap_dead start 2024-02-17T12:48:26.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.023+0000 7f8e2edfd700 1 -- 172.21.15.18:0/3973210385 shutdown_connections 2024-02-17T12:48:26.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.023+0000 7f8e2edfd700 1 -- 172.21.15.18:0/3973210385 >> 172.21.15.18:0/3973210385 conn(0x7f8e28077aa0 msgr2=0x7f8e280ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:26.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.023+0000 7f8e2edfd700 1 -- 172.21.15.18:0/3973210385 shutdown_connections 2024-02-17T12:48:26.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.023+0000 7f8e2edfd700 1 -- 172.21.15.18:0/3973210385 wait complete. 2024-02-17T12:48:26.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:26 smithi018 ceph-mon[32625]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-17T12:48:26.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:26 smithi146 ceph-mon[36454]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-17T12:48:26.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.530+0000 7fa889b90700 1 -- 172.21.15.18:0/2813106543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa884102680 msgr2=0x7fa88410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:26.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.530+0000 7fa889b90700 1 --2- 172.21.15.18:0/2813106543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa884102680 0x7fa88410eac0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00b0d0 tx=0x7fa87c00b3e0 comp rx=0 tx=0).stop 2024-02-17T12:48:26.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.530+0000 7fa889b90700 1 -- 172.21.15.18:0/2813106543 shutdown_connections 2024-02-17T12:48:26.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.530+0000 7fa889b90700 1 --2- 172.21.15.18:0/2813106543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa884102680 0x7fa88410eac0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:26.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.530+0000 7fa889b90700 1 --2- 172.21.15.18:0/2813106543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa884107ca0 0x7fa884108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:26.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.530+0000 7fa889b90700 1 --2- 172.21.15.18:0/2813106543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa884101cc0 0x7fa884102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:26.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.530+0000 7fa889b90700 1 -- 172.21.15.18:0/2813106543 >> 172.21.15.18:0/2813106543 conn(0x7fa884078da0 msgr2=0x7fa8840791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:26.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.530+0000 7fa889b90700 1 -- 172.21.15.18:0/2813106543 shutdown_connections 2024-02-17T12:48:26.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.530+0000 7fa889b90700 1 -- 172.21.15.18:0/2813106543 wait complete. 2024-02-17T12:48:26.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.530+0000 7fa889b90700 1 Processor -- start 2024-02-17T12:48:26.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.530+0000 7fa889b90700 1 -- start start 2024-02-17T12:48:26.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa889b90700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa884101cc0 0x7fa884198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:26.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa889b90700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa884102680 0x7fa884198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:26.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa889b90700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa884107ca0 0x7fa88419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:26.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa889b90700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88419d120 con 0x7fa884101cc0 2024-02-17T12:48:26.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa889b90700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88419d290 con 0x7fa884102680 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa889b90700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88419d450 con 0x7fa884107ca0 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa8837fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa884101cc0 0x7fa884198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa8837fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa884101cc0 0x7fa884198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49890/0 (socket says 172.21.15.18:49890) 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa8837fe700 1 -- 172.21.15.18:0/2812211203 learned_addr learned my addr 172.21.15.18:0/2812211203 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa882ffd700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa884102680 0x7fa884198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa883fff700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa884107ca0 0x7fa88419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa8837fe700 1 -- 172.21.15.18:0/2812211203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa884107ca0 msgr2=0x7fa88419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa8837fe700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa884107ca0 0x7fa88419cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa8837fe700 1 -- 172.21.15.18:0/2812211203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa884102680 msgr2=0x7fa884198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa8837fe700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa884102680 0x7fa884198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa8837fe700 1 -- 172.21.15.18:0/2812211203 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c009d00 con 0x7fa884101cc0 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.531+0000 7fa8837fe700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa884101cc0 0x7fa884198240 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa87400b530 tx=0x7fa87400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.532+0000 7fa880ff9700 1 -- 172.21.15.18:0/2812211203 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874011830 con 0x7fa884101cc0 2024-02-17T12:48:26.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.532+0000 7fa880ff9700 1 -- 172.21.15.18:0/2812211203 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa874011e70 con 0x7fa884101cc0 2024-02-17T12:48:26.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.532+0000 7fa889b90700 1 -- 172.21.15.18:0/2812211203 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88419d730 con 0x7fa884101cc0 2024-02-17T12:48:26.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.532+0000 7fa880ff9700 1 -- 172.21.15.18:0/2812211203 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400f2e0 con 0x7fa884101cc0 2024-02-17T12:48:26.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.532+0000 7fa889b90700 1 -- 172.21.15.18:0/2812211203 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8841a4f90 con 0x7fa884101cc0 2024-02-17T12:48:26.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.532+0000 7fa889b90700 1 -- 172.21.15.18:0/2812211203 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa884112040 con 0x7fa884101cc0 2024-02-17T12:48:26.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.534+0000 7fa880ff9700 1 -- 172.21.15.18:0/2812211203 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa874010610 con 0x7fa884101cc0 2024-02-17T12:48:26.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.534+0000 7fa880ff9700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa86c0a2260 0x7fa86c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:26.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.534+0000 7fa882ffd700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa86c0a2260 0x7fa86c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:26.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.534+0000 7fa880ff9700 1 -- 172.21.15.18:0/2812211203 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa8740cda10 con 0x7fa884101cc0 2024-02-17T12:48:26.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.534+0000 7fa880ff9700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578] conn(0x7fa86c0aba40 0x7fa86c0ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:26.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.534+0000 7fa880ff9700 1 -- 172.21.15.18:0/2812211203 --> [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa86c0ae510 con 0x7fa86c0aba40 2024-02-17T12:48:26.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.534+0000 7fa880ff9700 1 -- 172.21.15.18:0/2812211203 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8740cdeb0 con 0x7fa884101cc0 2024-02-17T12:48:26.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.535+0000 7fa882ffd700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa86c0a2260 0x7fa86c0a4720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa884079330 tx=0x7fa87800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:26.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.535+0000 7fa883fff700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578] conn(0x7fa86c0aba40 0x7fa86c0ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:26.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.535+0000 7fa883fff700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578] conn(0x7fa86c0aba40 0x7fa86c0ade60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:26.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.536+0000 7fa880ff9700 1 -- 172.21.15.18:0/2812211203 <== osd.2 v2:172.21.15.146:6800/3369250578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa86c0ae510 con 0x7fa86c0aba40 2024-02-17T12:48:26.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.553+0000 7fa889b90700 1 -- 172.21.15.18:0/2812211203 --> [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa884062300 con 0x7fa86c0aba40 2024-02-17T12:48:26.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.554+0000 7fa880ff9700 1 -- 172.21.15.18:0/2812211203 <== osd.2 v2:172.21.15.146:6800/3369250578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa884062300 con 0x7fa86c0aba40 2024-02-17T12:48:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.554+0000 7fa889b90700 1 -- 172.21.15.18:0/2812211203 >> [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578] conn(0x7fa86c0aba40 msgr2=0x7fa86c0ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.554+0000 7fa889b90700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578] conn(0x7fa86c0aba40 0x7fa86c0ade60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.554+0000 7fa889b90700 1 -- 172.21.15.18:0/2812211203 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa86c0a2260 msgr2=0x7fa86c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.554+0000 7fa889b90700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fa86c0a2260 0x7fa86c0a4720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa884079330 tx=0x7fa87800b410 comp rx=0 tx=0).stop 2024-02-17T12:48:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.554+0000 7fa889b90700 1 -- 172.21.15.18:0/2812211203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa884101cc0 msgr2=0x7fa884198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.554+0000 7fa889b90700 1 --2- 172.21.15.18:0/2812211203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa884101cc0 0x7fa884198240 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa87400b530 tx=0x7fa87400b840 comp rx=0 tx=0).stop 2024-02-17T12:48:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.554+0000 7fa883fff700 1 -- 172.21.15.18:0/2812211203 reap_dead start 2024-02-17T12:48:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.554+0000 7fa889b90700 1 -- 172.21.15.18:0/2812211203 shutdown_connections 2024-02-17T12:48:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.554+0000 7fa889b90700 1 -- 172.21.15.18:0/2812211203 >> 172.21.15.18:0/2812211203 conn(0x7fa884078da0 msgr2=0x7fa884100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.554+0000 7fa889b90700 1 -- 172.21.15.18:0/2812211203 shutdown_connections 2024-02-17T12:48:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:26.554+0000 7fa889b90700 1 -- 172.21.15.18:0/2812211203 wait complete. 2024-02-17T12:48:26.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:26 smithi119 ceph-mon[36188]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-02-17T12:48:28.035 INFO:teuthology.orchestra.run.smithi018.stdout:163208757299 2024-02-17T12:48:28.036 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-17T12:48:28.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:28 smithi018 ceph-mon[32625]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 12 op/s 2024-02-17T12:48:28.658 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:28 smithi119 ceph-mon[36188]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 12 op/s 2024-02-17T12:48:28.658 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:28 smithi146 ceph-mon[36454]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 12 op/s 2024-02-17T12:48:28.659 INFO:teuthology.orchestra.run.smithi018.stdout:77309411394 2024-02-17T12:48:28.659 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-17T12:48:29.120 INFO:teuthology.orchestra.run.smithi018.stdout:257698037796 2024-02-17T12:48:29.121 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-02-17T12:48:29.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.361+0000 7f692f03b700 1 -- 172.21.15.18:0/1956659250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6928100c20 msgr2=0x7f692810eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:29.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.361+0000 7f692f03b700 1 --2- 172.21.15.18:0/1956659250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6928100c20 0x7f692810eb80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f692000b0a0 tx=0x7f692000b3b0 comp rx=0 tx=0).stop 2024-02-17T12:48:29.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.361+0000 7f692f03b700 1 -- 172.21.15.18:0/1956659250 shutdown_connections 2024-02-17T12:48:29.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.361+0000 7f692f03b700 1 --2- 172.21.15.18:0/1956659250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6928100c20 0x7f692810eb80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:29.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.361+0000 7f692f03b700 1 --2- 172.21.15.18:0/1956659250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6928106ce0 0x7f69281070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:29.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.361+0000 7f692f03b700 1 --2- 172.21.15.18:0/1956659250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6928100260 0x7f69281006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:29.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.361+0000 7f692f03b700 1 -- 172.21.15.18:0/1956659250 >> 172.21.15.18:0/1956659250 conn(0x7f6928077170 msgr2=0x7f69280fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.361+0000 7f692f03b700 1 -- 172.21.15.18:0/1956659250 shutdown_connections 2024-02-17T12:48:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.361+0000 7f692f03b700 1 -- 172.21.15.18:0/1956659250 wait complete. 2024-02-17T12:48:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.361+0000 7f692f03b700 1 Processor -- start 2024-02-17T12:48:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.361+0000 7f692f03b700 1 -- start start 2024-02-17T12:48:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f692f03b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6928100260 0x7f692819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f692f03b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6928100c20 0x7f692819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f692f03b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6928106ce0 0x7f692819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f692f03b700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692819f500 con 0x7f6928100c20 2024-02-17T12:48:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f692f03b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692819f670 con 0x7f6928100260 2024-02-17T12:48:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f692f03b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692819f830 con 0x7f6928106ce0 2024-02-17T12:48:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f6927fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6928100c20 0x7f692819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f6927fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6928100c20 0x7f692819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49906/0 (socket says 172.21.15.18:49906) 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f6927fff700 1 -- 172.21.15.18:0/3244700540 learned_addr learned my addr 172.21.15.18:0/3244700540 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f692cdd7700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6928100260 0x7f692819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f692d5d8700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6928106ce0 0x7f692819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f6927fff700 1 -- 172.21.15.18:0/3244700540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6928106ce0 msgr2=0x7f692819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f6927fff700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6928106ce0 0x7f692819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f6927fff700 1 -- 172.21.15.18:0/3244700540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6928100260 msgr2=0x7f692819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f6927fff700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6928100260 0x7f692819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.362+0000 7f6927fff700 1 -- 172.21.15.18:0/3244700540 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6920009d00 con 0x7f6928100c20 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.363+0000 7f6927fff700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6928100c20 0x7f692819abb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f691400d980 tx=0x7f691400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.363+0000 7f6925ffb700 1 -- 172.21.15.18:0/3244700540 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6914004220 con 0x7f6928100c20 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.363+0000 7f6925ffb700 1 -- 172.21.15.18:0/3244700540 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6914007640 con 0x7f6928100c20 2024-02-17T12:48:29.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.363+0000 7f692f03b700 1 -- 172.21.15.18:0/3244700540 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692819fb10 con 0x7f6928100c20 2024-02-17T12:48:29.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.363+0000 7f6925ffb700 1 -- 172.21.15.18:0/3244700540 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691400f690 con 0x7f6928100c20 2024-02-17T12:48:29.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.363+0000 7f692f03b700 1 -- 172.21.15.18:0/3244700540 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69281a7370 con 0x7f6928100c20 2024-02-17T12:48:29.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.364+0000 7f692f03b700 1 -- 172.21.15.18:0/3244700540 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6928112180 con 0x7f6928100c20 2024-02-17T12:48:29.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.365+0000 7f6925ffb700 1 -- 172.21.15.18:0/3244700540 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f6914004410 con 0x7f6928100c20 2024-02-17T12:48:29.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.366+0000 7f6925ffb700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f69180a2260 0x7f69180a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:29.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.366+0000 7f692cdd7700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f69180a2260 0x7f69180a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:29.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.366+0000 7f6925ffb700 1 -- 172.21.15.18:0/3244700540 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f69140ce150 con 0x7f6928100c20 2024-02-17T12:48:29.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.366+0000 7f6925ffb700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256] conn(0x7f69180aba40 0x7f69180ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:29.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.366+0000 7f6925ffb700 1 -- 172.21.15.18:0/3244700540 --> [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69180ae510 con 0x7f69180aba40 2024-02-17T12:48:29.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.366+0000 7f6925ffb700 1 -- 172.21.15.18:0/3244700540 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69140ce4c0 con 0x7f6928100c20 2024-02-17T12:48:29.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.367+0000 7f692d5d8700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256] conn(0x7f69180aba40 0x7f69180ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:29.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.367+0000 7f692cdd7700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f69180a2260 0x7f69180a4720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f691c009a10 tx=0x7f691c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:29.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.367+0000 7f692d5d8700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256] conn(0x7f69180aba40 0x7f69180ade60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:29.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.368+0000 7f6925ffb700 1 -- 172.21.15.18:0/3244700540 <== osd.3 v2:172.21.15.18:6810/3584006256 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f69180ae510 con 0x7f69180aba40 2024-02-17T12:48:29.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.385+0000 7f692f03b700 1 -- 172.21.15.18:0/3244700540 --> [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6928105430 con 0x7f69180aba40 2024-02-17T12:48:29.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.386+0000 7f6925ffb700 1 -- 172.21.15.18:0/3244700540 <== osd.3 v2:172.21.15.18:6810/3584006256 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6928105430 con 0x7f69180aba40 2024-02-17T12:48:29.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.386+0000 7f692f03b700 1 -- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256] conn(0x7f69180aba40 msgr2=0x7f69180ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:29.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.386+0000 7f692f03b700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256] conn(0x7f69180aba40 0x7f69180ade60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:29.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.386+0000 7f692f03b700 1 -- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f69180a2260 msgr2=0x7f69180a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:29.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.386+0000 7f692f03b700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f69180a2260 0x7f69180a4720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f691c009a10 tx=0x7f691c009190 comp rx=0 tx=0).stop 2024-02-17T12:48:29.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.386+0000 7f692f03b700 1 -- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6928100c20 msgr2=0x7f692819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:29.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.386+0000 7f692f03b700 1 --2- 172.21.15.18:0/3244700540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6928100c20 0x7f692819abb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f691400d980 tx=0x7f691400dc90 comp rx=0 tx=0).stop 2024-02-17T12:48:29.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.386+0000 7f692d5d8700 1 -- 172.21.15.18:0/3244700540 reap_dead start 2024-02-17T12:48:29.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.386+0000 7f692f03b700 1 -- 172.21.15.18:0/3244700540 shutdown_connections 2024-02-17T12:48:29.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.386+0000 7f692f03b700 1 -- 172.21.15.18:0/3244700540 >> 172.21.15.18:0/3244700540 conn(0x7f6928077170 msgr2=0x7f6928104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:29.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.386+0000 7f692f03b700 1 -- 172.21.15.18:0/3244700540 shutdown_connections 2024-02-17T12:48:29.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:29.386+0000 7f692f03b700 1 -- 172.21.15.18:0/3244700540 wait complete. 2024-02-17T12:48:29.976 INFO:teuthology.orchestra.run.smithi018.stdout:223338299434 2024-02-17T12:48:29.977 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-17T12:48:30.031 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:30.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:30 smithi018 ceph-mon[32625]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:48:30.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:30 smithi146 ceph-mon[36454]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:48:30.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:30 smithi119 ceph-mon[36188]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:48:31.496 INFO:teuthology.orchestra.run.smithi018.stdout:107374182461 2024-02-17T12:48:31.497 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-17T12:48:31.950 INFO:teuthology.orchestra.run.smithi018.stdout:347892350996 2024-02-17T12:48:31.950 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-02-17T12:48:32.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:32 smithi018 ceph-mon[32625]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-02-17T12:48:32.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:32 smithi146 ceph-mon[36454]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-02-17T12:48:32.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:32 smithi119 ceph-mon[36188]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-02-17T12:48:33.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.235+0000 7fc0feaf8700 1 -- 172.21.15.18:0/4252528491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8106f90 msgr2=0x7fc0f810eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:33.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.235+0000 7fc0feaf8700 1 --2- 172.21.15.18:0/4252528491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8106f90 0x7fc0f810eb60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000b0a0 tx=0x7fc0f000b3b0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.235+0000 7fc0feaf8700 1 -- 172.21.15.18:0/4252528491 shutdown_connections 2024-02-17T12:48:33.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.235+0000 7fc0feaf8700 1 --2- 172.21.15.18:0/4252528491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8106f90 0x7fc0f810eb60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.235+0000 7fc0feaf8700 1 --2- 172.21.15.18:0/4252528491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0f8101eb0 0x7fc0f8102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.235+0000 7fc0feaf8700 1 --2- 172.21.15.18:0/4252528491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f8102860 0x7fc0f81068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.235+0000 7fc0feaf8700 1 -- 172.21.15.18:0/4252528491 >> 172.21.15.18:0/4252528491 conn(0x7fc0f80fd720 msgr2=0x7fc0f80ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:33.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.235+0000 7fc0feaf8700 1 -- 172.21.15.18:0/4252528491 shutdown_connections 2024-02-17T12:48:33.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.235+0000 7fc0feaf8700 1 -- 172.21.15.18:0/4252528491 wait complete. 2024-02-17T12:48:33.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.235+0000 7fc0feaf8700 1 Processor -- start 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0feaf8700 1 -- start start 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0feaf8700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8101eb0 0x7fc0f81982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0feaf8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f8102860 0x7fc0f81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0feaf8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0f8106f90 0x7fc0f819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0feaf8700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f819d190 con 0x7fc0f8101eb0 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0feaf8700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f819d300 con 0x7fc0f8102860 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0feaf8700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f819d4c0 con 0x7fc0f8106f90 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0fc894700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8101eb0 0x7fc0f81982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0fc894700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8101eb0 0x7fc0f81982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44948/0 (socket says 172.21.15.18:44948) 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0fc894700 1 -- 172.21.15.18:0/2735797354 learned_addr learned my addr 172.21.15.18:0/2735797354 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0f7fff700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f8102860 0x7fc0f81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0fd095700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0f8106f90 0x7fc0f819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0fc894700 1 -- 172.21.15.18:0/2735797354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0f8106f90 msgr2=0x7fc0f819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:33.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0fc894700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0f8106f90 0x7fc0f819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0fc894700 1 -- 172.21.15.18:0/2735797354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f8102860 msgr2=0x7fc0f81987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:33.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0fc894700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f8102860 0x7fc0f81987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0fc894700 1 -- 172.21.15.18:0/2735797354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f0009d00 con 0x7fc0f8101eb0 2024-02-17T12:48:33.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0fd095700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0f8106f90 0x7fc0f819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:48:33.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0f7fff700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f8102860 0x7fc0f81987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:48:33.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0fc894700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8101eb0 0x7fc0f81982b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00b530 tx=0x7fc0ec00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:33.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.236+0000 7fc0f5ffb700 1 -- 172.21.15.18:0/2735797354 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec011830 con 0x7fc0f8101eb0 2024-02-17T12:48:33.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.237+0000 7fc0feaf8700 1 -- 172.21.15.18:0/2735797354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f819d7a0 con 0x7fc0f8101eb0 2024-02-17T12:48:33.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.237+0000 7fc0f5ffb700 1 -- 172.21.15.18:0/2735797354 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0ec011e70 con 0x7fc0f8101eb0 2024-02-17T12:48:33.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.237+0000 7fc0f5ffb700 1 -- 172.21.15.18:0/2735797354 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec00f2e0 con 0x7fc0f8101eb0 2024-02-17T12:48:33.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.237+0000 7fc0feaf8700 1 -- 172.21.15.18:0/2735797354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f81a5000 con 0x7fc0f8101eb0 2024-02-17T12:48:33.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.238+0000 7fc0feaf8700 1 -- 172.21.15.18:0/2735797354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f8068930 con 0x7fc0f8101eb0 2024-02-17T12:48:33.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.239+0000 7fc0f5ffb700 1 -- 172.21.15.18:0/2735797354 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fc0ec0105f0 con 0x7fc0f8101eb0 2024-02-17T12:48:33.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.239+0000 7fc0f5ffb700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc0e80a21e0 0x7fc0e80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:33.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.239+0000 7fc0f7fff700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc0e80a21e0 0x7fc0e80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:33.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.239+0000 7fc0f5ffb700 1 -- 172.21.15.18:0/2735797354 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc0ec0cd670 con 0x7fc0f8101eb0 2024-02-17T12:48:33.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.240+0000 7fc0f7fff700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc0e80a21e0 0x7fc0e80a46a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4005fd0 tx=0x7fc0e4005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:33.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.244+0000 7fc0f5ffb700 1 -- 172.21.15.18:0/2735797354 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc0ec0967a0 con 0x7fc0f8101eb0 2024-02-17T12:48:33.403 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:33.405 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:33.408 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:33.408 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:33.424 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:33.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.457+0000 7f18b1428700 1 -- 172.21.15.18:0/2655337585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ac100f40 msgr2=0x7f18ac104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:33.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.457+0000 7f18b1428700 1 --2- 172.21.15.18:0/2655337585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ac100f40 0x7f18ac104f90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f189c0098f0 tx=0x7f189c009c00 comp rx=0 tx=0).stop 2024-02-17T12:48:33.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.458+0000 7f18b1428700 1 -- 172.21.15.18:0/2655337585 shutdown_connections 2024-02-17T12:48:33.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.458+0000 7f18b1428700 1 --2- 172.21.15.18:0/2655337585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac105670 0x7f18ac10d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.458+0000 7f18b1428700 1 --2- 172.21.15.18:0/2655337585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18ac100590 0x7f18ac100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.458+0000 7f18b1428700 1 --2- 172.21.15.18:0/2655337585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ac100f40 0x7f18ac104f90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.458+0000 7f18b1428700 1 -- 172.21.15.18:0/2655337585 >> 172.21.15.18:0/2655337585 conn(0x7f18ac077170 msgr2=0x7f18ac0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:33.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.458+0000 7f18b1428700 1 -- 172.21.15.18:0/2655337585 shutdown_connections 2024-02-17T12:48:33.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.458+0000 7f18b1428700 1 -- 172.21.15.18:0/2655337585 wait complete. 2024-02-17T12:48:33.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.459+0000 7f18b1428700 1 Processor -- start 2024-02-17T12:48:33.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.459+0000 7f18b1428700 1 -- start start 2024-02-17T12:48:33.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18b1428700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18ac100590 0x7f18ac0733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:33.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18b1428700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac100f40 0x7f18ac073920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:33.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18b1428700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ac105670 0x7f18ac06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:33.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18b1428700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac06cf30 con 0x7f18ac105670 2024-02-17T12:48:33.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18b1428700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac06d0a0 con 0x7f18ac100590 2024-02-17T12:48:33.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18b1428700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac06d260 con 0x7f18ac100f40 2024-02-17T12:48:33.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18ab7fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ac105670 0x7f18ac06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:33.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18ab7fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ac105670 0x7f18ac06c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44962/0 (socket says 172.21.15.18:44962) 2024-02-17T12:48:33.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18ab7fe700 1 -- 172.21.15.18:0/774857797 learned_addr learned my addr 172.21.15.18:0/774857797 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:33.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18ab7fe700 1 -- 172.21.15.18:0/774857797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac100f40 msgr2=0x7f18ac073920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:48:33.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18ab7fe700 1 --2- 172.21.15.18:0/774857797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac100f40 0x7f18ac073920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18ab7fe700 1 -- 172.21.15.18:0/774857797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18ac100590 msgr2=0x7f18ac0733e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:48:33.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18ab7fe700 1 --2- 172.21.15.18:0/774857797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18ac100590 0x7f18ac0733e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.460+0000 7f18ab7fe700 1 -- 172.21.15.18:0/774857797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c009520 con 0x7f18ac105670 2024-02-17T12:48:33.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.461+0000 7f18ab7fe700 1 --2- 172.21.15.18:0/774857797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ac105670 0x7f18ac06c7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f18a000d980 tx=0x7f18a000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:33.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.461+0000 7f1893fff700 1 -- 172.21.15.18:0/774857797 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0009940 con 0x7f18ac105670 2024-02-17T12:48:33.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.462+0000 7f1893fff700 1 -- 172.21.15.18:0/774857797 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18a0004830 con 0x7f18ac105670 2024-02-17T12:48:33.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.462+0000 7f1893fff700 1 -- 172.21.15.18:0/774857797 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a00054f0 con 0x7f18ac105670 2024-02-17T12:48:33.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.462+0000 7f18b1428700 1 -- 172.21.15.18:0/774857797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac070f70 con 0x7f18ac105670 2024-02-17T12:48:33.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.462+0000 7f18b1428700 1 -- 172.21.15.18:0/774857797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac071420 con 0x7f18ac105670 2024-02-17T12:48:33.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.469+0000 7f1893fff700 1 -- 172.21.15.18:0/774857797 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f18a00049a0 con 0x7f18ac105670 2024-02-17T12:48:33.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.469+0000 7f1893fff700 1 --2- 172.21.15.18:0/774857797 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f18940a6800 0x7f18940a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:33.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.470+0000 7f18b1428700 1 -- 172.21.15.18:0/774857797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ac101a70 con 0x7f18ac105670 2024-02-17T12:48:33.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.476+0000 7f1893fff700 1 -- 172.21.15.18:0/774857797 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f18a00cdbb0 con 0x7f18ac105670 2024-02-17T12:48:33.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.479+0000 7f18aaffd700 1 --2- 172.21.15.18:0/774857797 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f18940a6800 0x7f18940a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:33.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.479+0000 7f1893fff700 1 -- 172.21.15.18:0/774857797 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18a0096ad0 con 0x7f18ac105670 2024-02-17T12:48:33.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.483+0000 7f18aaffd700 1 --2- 172.21.15.18:0/774857797 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f18940a6800 0x7f18940a8cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f18a400afd0 tx=0x7f18a400aaf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:33.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.505+0000 7fc0feaf8700 1 -- 172.21.15.18:0/2735797354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fc0f8111d90 con 0x7fc0f8101eb0 2024-02-17T12:48:33.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.505+0000 7fc0f5ffb700 1 -- 172.21.15.18:0/2735797354 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc0ec099e60 con 0x7fc0f8101eb0 2024-02-17T12:48:33.498 INFO:teuthology.orchestra.run.smithi018.stdout:313532612634 2024-02-17T12:48:33.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.507+0000 7fc0feaf8700 1 -- 172.21.15.18:0/2735797354 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc0e80a21e0 msgr2=0x7fc0e80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:33.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.507+0000 7fc0feaf8700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc0e80a21e0 0x7fc0e80a46a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4005fd0 tx=0x7fc0e4005e20 comp rx=0 tx=0).stop 2024-02-17T12:48:33.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.507+0000 7fc0feaf8700 1 -- 172.21.15.18:0/2735797354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8101eb0 msgr2=0x7fc0f81982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:33.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.507+0000 7fc0feaf8700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8101eb0 0x7fc0f81982b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00b530 tx=0x7fc0ec00b8f0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.507+0000 7fc0feaf8700 1 -- 172.21.15.18:0/2735797354 shutdown_connections 2024-02-17T12:48:33.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.507+0000 7fc0feaf8700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc0e80a21e0 0x7fc0e80a46a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.507+0000 7fc0feaf8700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0f8106f90 0x7fc0f819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.507+0000 7fc0feaf8700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8101eb0 0x7fc0f81982b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.507+0000 7fc0feaf8700 1 --2- 172.21.15.18:0/2735797354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f8102860 0x7fc0f81987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.507+0000 7fc0feaf8700 1 -- 172.21.15.18:0/2735797354 >> 172.21.15.18:0/2735797354 conn(0x7fc0f80fd720 msgr2=0x7fc0f80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:33.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.507+0000 7fc0feaf8700 1 -- 172.21.15.18:0/2735797354 shutdown_connections 2024-02-17T12:48:33.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.508+0000 7fc0feaf8700 1 -- 172.21.15.18:0/2735797354 wait complete. 2024-02-17T12:48:33.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.705+0000 7f18b1428700 1 -- 172.21.15.18:0/774857797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f18ac06d9a0 con 0x7f18ac105670 2024-02-17T12:48:33.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.705+0000 7f1893fff700 1 -- 172.21.15.18:0/774857797 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18a0017020 con 0x7f18ac105670 2024-02-17T12:48:33.699 INFO:teuthology.orchestra.run.smithi018.stdout:197568495663 2024-02-17T12:48:33.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.707+0000 7f18b1428700 1 -- 172.21.15.18:0/774857797 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f18940a6800 msgr2=0x7f18940a8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:33.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.707+0000 7f18b1428700 1 --2- 172.21.15.18:0/774857797 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f18940a6800 0x7f18940a8cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f18a400afd0 tx=0x7f18a400aaf0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.707+0000 7f18b1428700 1 -- 172.21.15.18:0/774857797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ac105670 msgr2=0x7f18ac06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:33.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.707+0000 7f18b1428700 1 --2- 172.21.15.18:0/774857797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ac105670 0x7f18ac06c7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f18a000d980 tx=0x7f18a000dd40 comp rx=0 tx=0).stop 2024-02-17T12:48:33.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.707+0000 7f18b1428700 1 -- 172.21.15.18:0/774857797 shutdown_connections 2024-02-17T12:48:33.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.707+0000 7f18b1428700 1 --2- 172.21.15.18:0/774857797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ac105670 0x7f18ac06c7a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.707+0000 7f18b1428700 1 --2- 172.21.15.18:0/774857797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18ac100590 0x7f18ac0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.707+0000 7f18b1428700 1 --2- 172.21.15.18:0/774857797 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f18940a6800 0x7f18940a8cc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.707+0000 7f18b1428700 1 --2- 172.21.15.18:0/774857797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac100f40 0x7f18ac073920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:33.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.707+0000 7f18b1428700 1 -- 172.21.15.18:0/774857797 >> 172.21.15.18:0/774857797 conn(0x7f18ac077170 msgr2=0x7f18ac103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:33.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.707+0000 7f18b1428700 1 -- 172.21.15.18:0/774857797 shutdown_connections 2024-02-17T12:48:33.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:33.707+0000 7f18b1428700 1 -- 172.21.15.18:0/774857797 wait complete. 2024-02-17T12:48:34.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:34 smithi119 ceph-mon[36188]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-02-17T12:48:34.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:34 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2735797354' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-17T12:48:34.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:34 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/774857797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-17T12:48:34.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:34 smithi146 ceph-mon[36454]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-02-17T12:48:34.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:34 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2735797354' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-17T12:48:34.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:34 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/774857797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-17T12:48:34.452 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:34 smithi018 ceph-mon[32625]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-02-17T12:48:34.452 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:34 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2735797354' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-17T12:48:34.452 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:34 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/774857797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-17T12:48:36.400 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:36 smithi018 ceph-mon[32625]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:48:36.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:36 smithi146 ceph-mon[36454]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:48:36.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:36 smithi119 ceph-mon[36188]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:48:38.263 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:38 smithi018 ceph-mon[32625]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-02-17T12:48:38.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:38 smithi146 ceph-mon[36454]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-02-17T12:48:38.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:38 smithi119 ceph-mon[36188]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-02-17T12:48:38.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.954+0000 7f5d051f2700 1 -- 172.21.15.18:0/1818695792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d001026b0 msgr2=0x7f5d0010eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:38.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.954+0000 7f5d051f2700 1 --2- 172.21.15.18:0/1818695792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d001026b0 0x7f5d0010eb10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800b0a0 tx=0x7f5cf800b3b0 comp rx=0 tx=0).stop 2024-02-17T12:48:38.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.954+0000 7f5d051f2700 1 -- 172.21.15.18:0/1818695792 shutdown_connections 2024-02-17T12:48:38.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.954+0000 7f5d051f2700 1 --2- 172.21.15.18:0/1818695792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d001026b0 0x7f5d0010eb10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:38.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.954+0000 7f5d051f2700 1 --2- 172.21.15.18:0/1818695792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d00107dd0 0x7f5d001081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:38.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.954+0000 7f5d051f2700 1 --2- 172.21.15.18:0/1818695792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d00101cf0 0x7f5d00102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:38.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.954+0000 7f5d051f2700 1 -- 172.21.15.18:0/1818695792 >> 172.21.15.18:0/1818695792 conn(0x7f5d00078d30 msgr2=0x7f5d00079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:38.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.955+0000 7f5d051f2700 1 -- 172.21.15.18:0/1818695792 shutdown_connections 2024-02-17T12:48:38.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.955+0000 7f5d051f2700 1 -- 172.21.15.18:0/1818695792 wait complete. 2024-02-17T12:48:38.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.955+0000 7f5d051f2700 1 Processor -- start 2024-02-17T12:48:38.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.956+0000 7f5d051f2700 1 -- start start 2024-02-17T12:48:38.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.956+0000 7f5d051f2700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d00101cf0 0x7f5d001982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:38.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.956+0000 7f5d051f2700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d001026b0 0x7f5d00198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:38.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.956+0000 7f5d051f2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d00107dd0 0x7f5d0019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:38.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.956+0000 7f5d051f2700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0019d1b0 con 0x7f5d00101cf0 2024-02-17T12:48:38.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.956+0000 7f5d051f2700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0019d320 con 0x7f5d001026b0 2024-02-17T12:48:38.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.956+0000 7f5d051f2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0019d4e0 con 0x7f5d00107dd0 2024-02-17T12:48:38.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.956+0000 7f5cfed9d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d00101cf0 0x7f5d001982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.956+0000 7f5cfed9d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d00101cf0 0x7f5d001982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44976/0 (socket says 172.21.15.18:44976) 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.956+0000 7f5cfed9d700 1 -- 172.21.15.18:0/3338974799 learned_addr learned my addr 172.21.15.18:0/3338974799 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.956+0000 7f5cff59e700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d00107dd0 0x7f5d0019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.956+0000 7f5cfe59c700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d001026b0 0x7f5d00198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.957+0000 7f5cfed9d700 1 -- 172.21.15.18:0/3338974799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d00107dd0 msgr2=0x7f5d0019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.957+0000 7f5cfed9d700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d00107dd0 0x7f5d0019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.957+0000 7f5cfed9d700 1 -- 172.21.15.18:0/3338974799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d001026b0 msgr2=0x7f5d00198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.957+0000 7f5cfed9d700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d001026b0 0x7f5d00198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.957+0000 7f5cfed9d700 1 -- 172.21.15.18:0/3338974799 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf8009d00 con 0x7f5d00101cf0 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.957+0000 7f5cfed9d700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d00101cf0 0x7f5d001982d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000d980 tx=0x7f5cf000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.957+0000 7f5ce7fff700 1 -- 172.21.15.18:0/3338974799 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0004220 con 0x7f5d00101cf0 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.957+0000 7f5ce7fff700 1 -- 172.21.15.18:0/3338974799 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cf0007640 con 0x7f5d00101cf0 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.957+0000 7f5ce7fff700 1 -- 172.21.15.18:0/3338974799 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000f690 con 0x7f5d00101cf0 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.957+0000 7f5d051f2700 1 -- 172.21.15.18:0/3338974799 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0019d7c0 con 0x7f5d00101cf0 2024-02-17T12:48:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.957+0000 7f5d051f2700 1 -- 172.21.15.18:0/3338974799 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d00074070 con 0x7f5d00101cf0 2024-02-17T12:48:38.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.958+0000 7f5d051f2700 1 -- 172.21.15.18:0/3338974799 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0010a260 con 0x7f5d00101cf0 2024-02-17T12:48:38.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.959+0000 7f5ce7fff700 1 -- 172.21.15.18:0/3338974799 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f5cf0004410 con 0x7f5d00101cf0 2024-02-17T12:48:38.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.960+0000 7f5ce7fff700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5ce80a2260 0x7f5ce80a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:38.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.960+0000 7f5ce7fff700 1 -- 172.21.15.18:0/3338974799 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5cf00cddd0 con 0x7f5d00101cf0 2024-02-17T12:48:38.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.960+0000 7f5cfe59c700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5ce80a2260 0x7f5ce80a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:38.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.961+0000 7f5cfe59c700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5ce80a2260 0x7f5ce80a4720 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d00079290 tx=0x7f5cf400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:38.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:38.964+0000 7f5ce7fff700 1 -- 172.21.15.18:0/3338974799 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5cf0096df0 con 0x7f5d00101cf0 2024-02-17T12:48:39.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.170+0000 7f5d051f2700 1 -- 172.21.15.18:0/3338974799 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5d00068930 con 0x7f5d00101cf0 2024-02-17T12:48:39.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.170+0000 7f5ce7fff700 1 -- 172.21.15.18:0/3338974799 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5cf009a4b0 con 0x7f5d00101cf0 2024-02-17T12:48:39.164 INFO:teuthology.orchestra.run.smithi018.stdout:77309411397 2024-02-17T12:48:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.172+0000 7f5d051f2700 1 -- 172.21.15.18:0/3338974799 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5ce80a2260 msgr2=0x7f5ce80a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.172+0000 7f5d051f2700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5ce80a2260 0x7f5ce80a4720 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d00079290 tx=0x7f5cf400a300 comp rx=0 tx=0).stop 2024-02-17T12:48:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.172+0000 7f5d051f2700 1 -- 172.21.15.18:0/3338974799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d00101cf0 msgr2=0x7f5d001982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.172+0000 7f5d051f2700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d00101cf0 0x7f5d001982d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000d980 tx=0x7f5cf000dc90 comp rx=0 tx=0).stop 2024-02-17T12:48:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.172+0000 7f5d051f2700 1 -- 172.21.15.18:0/3338974799 shutdown_connections 2024-02-17T12:48:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.172+0000 7f5d051f2700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f5ce80a2260 0x7f5ce80a4720 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.172+0000 7f5d051f2700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d00107dd0 0x7f5d0019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.172+0000 7f5d051f2700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d00101cf0 0x7f5d001982d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.172+0000 7f5d051f2700 1 --2- 172.21.15.18:0/3338974799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d001026b0 0x7f5d00198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.172+0000 7f5d051f2700 1 -- 172.21.15.18:0/3338974799 >> 172.21.15.18:0/3338974799 conn(0x7f5d00078d30 msgr2=0x7f5d001008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.173+0000 7f5d051f2700 1 -- 172.21.15.18:0/3338974799 shutdown_connections 2024-02-17T12:48:39.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:39.173+0000 7f5d051f2700 1 -- 172.21.15.18:0/3338974799 wait complete. 2024-02-17T12:48:39.364 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495660 got 197568495663 for osd.6 2024-02-17T12:48:39.365 DEBUG:teuthology.parallel:result is None 2024-02-17T12:48:40.445 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:40.445 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:40.446 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:40.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:40 smithi146 ceph-mon[36454]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-17T12:48:40.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:40 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3338974799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-17T12:48:40.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:40 smithi018 ceph-mon[32625]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-17T12:48:40.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:40 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3338974799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-17T12:48:40.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:40 smithi119 ceph-mon[36188]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-17T12:48:40.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:40 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3338974799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-17T12:48:42.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:42 smithi146 ceph-mon[36454]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-02-17T12:48:42.469 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:42 smithi018 ceph-mon[32625]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-02-17T12:48:42.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:42 smithi119 ceph-mon[36188]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-02-17T12:48:43.418 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:43 smithi018 ceph-mon[32625]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-02-17T12:48:43.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:43 smithi146 ceph-mon[36454]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-02-17T12:48:43.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:43 smithi119 ceph-mon[36188]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-02-17T12:48:43.970 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411397 for osd.2 2024-02-17T12:48:43.970 DEBUG:teuthology.parallel:result is None 2024-02-17T12:48:44.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.387+0000 7fe58beb7700 1 -- 172.21.15.18:0/3974908644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe584100dd0 msgr2=0x7fe5841011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:44.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.387+0000 7fe58beb7700 1 --2- 172.21.15.18:0/3974908644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe584100dd0 0x7fe5841011b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009990 tx=0x7fe57c009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:44.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.387+0000 7fe58beb7700 1 -- 172.21.15.18:0/3974908644 shutdown_connections 2024-02-17T12:48:44.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.387+0000 7fe58beb7700 1 --2- 172.21.15.18:0/3974908644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe584105eb0 0x7fe58410da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:44.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.387+0000 7fe58beb7700 1 --2- 172.21.15.18:0/3974908644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe584100dd0 0x7fe5841011b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:44.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.387+0000 7fe58beb7700 1 --2- 172.21.15.18:0/3974908644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe584101780 0x7fe5841057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:44.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.387+0000 7fe58beb7700 1 -- 172.21.15.18:0/3974908644 >> 172.21.15.18:0/3974908644 conn(0x7fe584077c40 msgr2=0x7fe584078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:44.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.387+0000 7fe58beb7700 1 -- 172.21.15.18:0/3974908644 shutdown_connections 2024-02-17T12:48:44.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.387+0000 7fe58beb7700 1 -- 172.21.15.18:0/3974908644 wait complete. 2024-02-17T12:48:44.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe58beb7700 1 Processor -- start 2024-02-17T12:48:44.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe58beb7700 1 -- start start 2024-02-17T12:48:44.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe58beb7700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe584100dd0 0x7fe5841980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:44.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe58beb7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe584101780 0x7fe584198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:44.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe58beb7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe584105eb0 0x7fe58419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:44.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe58beb7700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58419cfa0 con 0x7fe584100dd0 2024-02-17T12:48:44.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe58beb7700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58419d110 con 0x7fe584105eb0 2024-02-17T12:48:44.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe58beb7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58419d2d0 con 0x7fe584101780 2024-02-17T12:48:44.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe589c53700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe584100dd0 0x7fe5841980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:44.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe589c53700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe584100dd0 0x7fe5841980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44242/0 (socket says 172.21.15.18:44242) 2024-02-17T12:48:44.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe589c53700 1 -- 172.21.15.18:0/3340963740 learned_addr learned my addr 172.21.15.18:0/3340963740 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:44.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe58a454700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe584105eb0 0x7fe58419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:44.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe589452700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe584101780 0x7fe584198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:44.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe589c53700 1 -- 172.21.15.18:0/3340963740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe584101780 msgr2=0x7fe584198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:44.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe589c53700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe584101780 0x7fe584198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:44.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe589c53700 1 -- 172.21.15.18:0/3340963740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe584105eb0 msgr2=0x7fe58419c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:44.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe589c53700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe584105eb0 0x7fe58419c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:44.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe589c53700 1 -- 172.21.15.18:0/3340963740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c0095f0 con 0x7fe584100dd0 2024-02-17T12:48:44.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe589452700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe584101780 0x7fe584198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:48:44.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.388+0000 7fe589c53700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe584100dd0 0x7fe5841980c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00f690 tx=0x7fe57c00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:44.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.389+0000 7fe582ffd700 1 -- 172.21.15.18:0/3340963740 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c01d070 con 0x7fe584100dd0 2024-02-17T12:48:44.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.389+0000 7fe582ffd700 1 -- 172.21.15.18:0/3340963740 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe57c003680 con 0x7fe584100dd0 2024-02-17T12:48:44.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.389+0000 7fe58beb7700 1 -- 172.21.15.18:0/3340963740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58419d550 con 0x7fe584100dd0 2024-02-17T12:48:44.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.389+0000 7fe582ffd700 1 -- 172.21.15.18:0/3340963740 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0173f0 con 0x7fe584100dd0 2024-02-17T12:48:44.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.389+0000 7fe58beb7700 1 -- 172.21.15.18:0/3340963740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5841a4e10 con 0x7fe584100dd0 2024-02-17T12:48:44.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.390+0000 7fe58beb7700 1 -- 172.21.15.18:0/3340963740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58410a0e0 con 0x7fe584100dd0 2024-02-17T12:48:44.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.391+0000 7fe582ffd700 1 -- 172.21.15.18:0/3340963740 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe57c00fc70 con 0x7fe584100dd0 2024-02-17T12:48:44.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.391+0000 7fe582ffd700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe56c0a21e0 0x7fe56c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:44.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.391+0000 7fe589452700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe56c0a21e0 0x7fe56c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:44.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.391+0000 7fe582ffd700 1 -- 172.21.15.18:0/3340963740 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe57c0cf140 con 0x7fe584100dd0 2024-02-17T12:48:44.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.392+0000 7fe589452700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe56c0a21e0 0x7fe56c0a46a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe578005fd0 tx=0x7fe578005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:44.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.395+0000 7fe582ffd700 1 -- 172.21.15.18:0/3340963740 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe57c098200 con 0x7fe584100dd0 2024-02-17T12:48:44.530 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612629 got 313532612634 for osd.10 2024-02-17T12:48:44.530 DEBUG:teuthology.parallel:result is None 2024-02-17T12:48:44.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.602+0000 7fe58beb7700 1 -- 172.21.15.18:0/3340963740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe584199420 con 0x7fe584100dd0 2024-02-17T12:48:44.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.602+0000 7fe582ffd700 1 -- 172.21.15.18:0/3340963740 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe57c09b8c0 con 0x7fe584100dd0 2024-02-17T12:48:44.596 INFO:teuthology.orchestra.run.smithi018.stdout:141733920828 2024-02-17T12:48:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.604+0000 7fe58beb7700 1 -- 172.21.15.18:0/3340963740 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe56c0a21e0 msgr2=0x7fe56c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.604+0000 7fe58beb7700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe56c0a21e0 0x7fe56c0a46a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe578005fd0 tx=0x7fe578005e20 comp rx=0 tx=0).stop 2024-02-17T12:48:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.604+0000 7fe58beb7700 1 -- 172.21.15.18:0/3340963740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe584100dd0 msgr2=0x7fe5841980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.604+0000 7fe58beb7700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe584100dd0 0x7fe5841980c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00f690 tx=0x7fe57c00f6c0 comp rx=0 tx=0).stop 2024-02-17T12:48:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.604+0000 7fe58beb7700 1 -- 172.21.15.18:0/3340963740 shutdown_connections 2024-02-17T12:48:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.604+0000 7fe58beb7700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe584105eb0 0x7fe58419c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.604+0000 7fe58beb7700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe584100dd0 0x7fe5841980c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:44.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.604+0000 7fe58beb7700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe56c0a21e0 0x7fe56c0a46a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:44.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.604+0000 7fe58beb7700 1 --2- 172.21.15.18:0/3340963740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe584101780 0x7fe584198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:44.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.604+0000 7fe58beb7700 1 -- 172.21.15.18:0/3340963740 >> 172.21.15.18:0/3340963740 conn(0x7fe584077c40 msgr2=0x7fe584104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:44.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.605+0000 7fe58beb7700 1 -- 172.21.15.18:0/3340963740 shutdown_connections 2024-02-17T12:48:44.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:44.605+0000 7fe58beb7700 1 -- 172.21.15.18:0/3340963740 wait complete. 2024-02-17T12:48:45.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:45 smithi018 ceph-mon[32625]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-17T12:48:45.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:45 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:48:45.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:45 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3340963740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-17T12:48:45.507 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:45 smithi146 ceph-mon[36454]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-17T12:48:45.507 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:45 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:48:45.507 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:45 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3340963740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-17T12:48:45.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:45 smithi119 ceph-mon[36188]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-02-17T12:48:45.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:45 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:48:45.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:45 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3340963740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-17T12:48:46.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.144+0000 7fc06d4e3700 1 -- 172.21.15.18:0/2325455254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc068106ce0 msgr2=0x7fc0681070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:46.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.144+0000 7fc06d4e3700 1 --2- 172.21.15.18:0/2325455254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc068106ce0 0x7fc0681070c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc058009990 tx=0x7fc058009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:46.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.144+0000 7fc06d4e3700 1 -- 172.21.15.18:0/2325455254 shutdown_connections 2024-02-17T12:48:46.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.144+0000 7fc06d4e3700 1 --2- 172.21.15.18:0/2325455254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc068100c20 0x7fc06810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:46.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.144+0000 7fc06d4e3700 1 --2- 172.21.15.18:0/2325455254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc068106ce0 0x7fc0681070c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:46.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.144+0000 7fc06d4e3700 1 --2- 172.21.15.18:0/2325455254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc068100260 0x7fc0681006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:46.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.144+0000 7fc06d4e3700 1 -- 172.21.15.18:0/2325455254 >> 172.21.15.18:0/2325455254 conn(0x7fc068077170 msgr2=0x7fc0680fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:46.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.145+0000 7fc06d4e3700 1 -- 172.21.15.18:0/2325455254 shutdown_connections 2024-02-17T12:48:46.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.145+0000 7fc06d4e3700 1 -- 172.21.15.18:0/2325455254 wait complete. 2024-02-17T12:48:46.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.145+0000 7fc06d4e3700 1 Processor -- start 2024-02-17T12:48:46.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.145+0000 7fc06d4e3700 1 -- start start 2024-02-17T12:48:46.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.145+0000 7fc06d4e3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc068100260 0x7fc0681982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:46.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.145+0000 7fc06d4e3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc068100c20 0x7fc0681987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:46.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.145+0000 7fc06d4e3700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc068106ce0 0x7fc06819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:46.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.145+0000 7fc06d4e3700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06819d180 con 0x7fc068106ce0 2024-02-17T12:48:46.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.145+0000 7fc06d4e3700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06819d2f0 con 0x7fc068100260 2024-02-17T12:48:46.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.145+0000 7fc06d4e3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06819d4b0 con 0x7fc068100c20 2024-02-17T12:48:46.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc0667fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc068100c20 0x7fc0681987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:46.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc0677fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc068106ce0 0x7fc06819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:46.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc0667fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc068100c20 0x7fc0681987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:46668/0 (socket says 172.21.15.18:46668) 2024-02-17T12:48:46.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc066ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc068100260 0x7fc0681982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:46.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc0667fc700 1 -- 172.21.15.18:0/246365775 learned_addr learned my addr 172.21.15.18:0/246365775 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc0667fc700 1 -- 172.21.15.18:0/246365775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc068100260 msgr2=0x7fc0681982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc0667fc700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc068100260 0x7fc0681982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc0667fc700 1 -- 172.21.15.18:0/246365775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc068106ce0 msgr2=0x7fc06819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc0667fc700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc068106ce0 0x7fc06819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc0667fc700 1 -- 172.21.15.18:0/246365775 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0580095f0 con 0x7fc068100c20 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc0677fe700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc068106ce0 0x7fc06819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc0667fc700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc068100c20 0x7fc0681987e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00e9a0 tx=0x7fc05c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc04ffff700 1 -- 172.21.15.18:0/246365775 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c009790 con 0x7fc068100c20 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc04ffff700 1 -- 172.21.15.18:0/246365775 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc05c010430 con 0x7fc068100c20 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc06d4e3700 1 -- 172.21.15.18:0/246365775 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06819d790 con 0x7fc068100c20 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc04ffff700 1 -- 172.21.15.18:0/246365775 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c018680 con 0x7fc068100c20 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.146+0000 7fc06d4e3700 1 -- 172.21.15.18:0/246365775 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0681a4ff0 con 0x7fc068100c20 2024-02-17T12:48:46.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.147+0000 7fc06d4e3700 1 -- 172.21.15.18:0/246365775 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc068103580 con 0x7fc068100c20 2024-02-17T12:48:46.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.148+0000 7fc04ffff700 1 -- 172.21.15.18:0/246365775 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fc05c010910 con 0x7fc068100c20 2024-02-17T12:48:46.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.149+0000 7fc04ffff700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc0500a21e0 0x7fc0500a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:46.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.149+0000 7fc066ffd700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc0500a21e0 0x7fc0500a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:46.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.149+0000 7fc04ffff700 1 -- 172.21.15.18:0/246365775 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc05c014070 con 0x7fc068100c20 2024-02-17T12:48:46.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.149+0000 7fc066ffd700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc0500a21e0 0x7fc0500a46a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc05800b2e0 tx=0x7fc058005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:46.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.153+0000 7fc04ffff700 1 -- 172.21.15.18:0/246365775 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc05c096e60 con 0x7fc068100c20 2024-02-17T12:48:46.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.360+0000 7fc06d4e3700 1 -- 172.21.15.18:0/246365775 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc068068930 con 0x7fc068100c20 2024-02-17T12:48:46.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.361+0000 7fc04ffff700 1 -- 172.21.15.18:0/246365775 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc05c0105a0 con 0x7fc068100c20 2024-02-17T12:48:46.354 INFO:teuthology.orchestra.run.smithi018.stdout:34359738447 2024-02-17T12:48:46.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.363+0000 7fc06d4e3700 1 -- 172.21.15.18:0/246365775 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc0500a21e0 msgr2=0x7fc0500a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:46.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.363+0000 7fc06d4e3700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc0500a21e0 0x7fc0500a46a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc05800b2e0 tx=0x7fc058005fd0 comp rx=0 tx=0).stop 2024-02-17T12:48:46.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.363+0000 7fc06d4e3700 1 -- 172.21.15.18:0/246365775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc068100c20 msgr2=0x7fc0681987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:46.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.363+0000 7fc06d4e3700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc068100c20 0x7fc0681987e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00e9a0 tx=0x7fc05c00ecb0 comp rx=0 tx=0).stop 2024-02-17T12:48:46.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.363+0000 7fc06d4e3700 1 -- 172.21.15.18:0/246365775 shutdown_connections 2024-02-17T12:48:46.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.363+0000 7fc06d4e3700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc068106ce0 0x7fc06819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:46.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.363+0000 7fc06d4e3700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc068100260 0x7fc0681982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:46.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.363+0000 7fc06d4e3700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc0500a21e0 0x7fc0500a46a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:46.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.363+0000 7fc06d4e3700 1 --2- 172.21.15.18:0/246365775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc068100c20 0x7fc0681987e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:46.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.363+0000 7fc06d4e3700 1 -- 172.21.15.18:0/246365775 >> 172.21.15.18:0/246365775 conn(0x7fc068077170 msgr2=0x7fc068104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:46.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.363+0000 7fc06d4e3700 1 -- 172.21.15.18:0/246365775 shutdown_connections 2024-02-17T12:48:46.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.363+0000 7fc06d4e3700 1 -- 172.21.15.18:0/246365775 wait complete. 2024-02-17T12:48:47.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.552+0000 7fb860aa1700 1 -- 172.21.15.18:0/3279646166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb85c106f80 msgr2=0x7fb85c10eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.552+0000 7fb860aa1700 1 --2- 172.21.15.18:0/3279646166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb85c106f80 0x7fb85c10eb30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb854009990 tx=0x7fb854009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.552+0000 7fb860aa1700 1 -- 172.21.15.18:0/3279646166 shutdown_connections 2024-02-17T12:48:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.552+0000 7fb860aa1700 1 --2- 172.21.15.18:0/3279646166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb85c106f80 0x7fb85c10eb30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.552+0000 7fb860aa1700 1 --2- 172.21.15.18:0/3279646166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb85c06a410 0x7fb85c06a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:47.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.552+0000 7fb860aa1700 1 --2- 172.21.15.18:0/3279646166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb85c06adc0 0x7fb85c1068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:47.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.552+0000 7fb860aa1700 1 -- 172.21.15.18:0/3279646166 >> 172.21.15.18:0/3279646166 conn(0x7fb85c076bc0 msgr2=0x7fb85c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:47.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.552+0000 7fb860aa1700 1 -- 172.21.15.18:0/3279646166 shutdown_connections 2024-02-17T12:48:47.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.552+0000 7fb860aa1700 1 -- 172.21.15.18:0/3279646166 wait complete. 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb860aa1700 1 Processor -- start 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb860aa1700 1 -- start start 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb860aa1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb85c06a410 0x7fb85c100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb860aa1700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb85c06adc0 0x7fb85c1011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb860aa1700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb85c106f80 0x7fb85c1016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb860aa1700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1a6490 con 0x7fb85c06adc0 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb860aa1700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1a6600 con 0x7fb85c106f80 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb860aa1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1a67c0 con 0x7fb85c06a410 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb859d9b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb85c06adc0 0x7fb85c1011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb85a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb85c06a410 0x7fb85c100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb859d9b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb85c06adc0 0x7fb85c1011b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44284/0 (socket says 172.21.15.18:44284) 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb859d9b700 1 -- 172.21.15.18:0/789545812 learned_addr learned my addr 172.21.15.18:0/789545812 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.553+0000 7fb85ad9d700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb85c106f80 0x7fb85c1016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:47.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.554+0000 7fb85a59c700 1 -- 172.21.15.18:0/789545812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb85c106f80 msgr2=0x7fb85c1016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:47.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.554+0000 7fb85a59c700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb85c106f80 0x7fb85c1016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:47.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.554+0000 7fb85a59c700 1 -- 172.21.15.18:0/789545812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb85c06adc0 msgr2=0x7fb85c1011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:47.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.554+0000 7fb85a59c700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb85c06adc0 0x7fb85c1011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:47.169 INFO:teuthology.orchestra.run.smithi018.stdout:55834574923 2024-02-17T12:48:47.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.554+0000 7fb85a59c700 1 -- 172.21.15.18:0/789545812 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8540095f0 con 0x7fb85c06a410 2024-02-17T12:48:47.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.554+0000 7fb859d9b700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb85c06adc0 0x7fb85c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.554+0000 7fb85a59c700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb85c06a410 0x7fb85c100c70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00ef10 tx=0x7fb84c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.554+0000 7fb84b7fe700 1 -- 172.21.15.18:0/789545812 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c00c9c0 con 0x7fb85c06a410 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.554+0000 7fb860aa1700 1 -- 172.21.15.18:0/789545812 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85c1a6a40 con 0x7fb85c06a410 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.554+0000 7fb84b7fe700 1 -- 172.21.15.18:0/789545812 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb84c00f040 con 0x7fb85c06a410 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.554+0000 7fb84b7fe700 1 -- 172.21.15.18:0/789545812 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0146e0 con 0x7fb85c06a410 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.554+0000 7fb860aa1700 1 -- 172.21.15.18:0/789545812 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c1a6f90 con 0x7fb85c06a410 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.555+0000 7fb860aa1700 1 -- 172.21.15.18:0/789545812 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85c10a280 con 0x7fb85c06a410 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.556+0000 7fb84b7fe700 1 -- 172.21.15.18:0/789545812 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fb84c007720 con 0x7fb85c06a410 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.557+0000 7fb84b7fe700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb8440a21e0 0x7fb8440a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.557+0000 7fb859d9b700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb8440a21e0 0x7fb8440a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.557+0000 7fb84b7fe700 1 -- 172.21.15.18:0/789545812 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb84c0ce590 con 0x7fb85c06a410 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.557+0000 7fb859d9b700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb8440a21e0 0x7fb8440a46a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb85c108370 tx=0x7fb850009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.560+0000 7fb84b7fe700 1 -- 172.21.15.18:0/789545812 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb84c0975b0 con 0x7fb85c06a410 2024-02-17T12:48:47.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.787+0000 7fb860aa1700 1 -- 172.21.15.18:0/789545812 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb85c111ce0 con 0x7fb85c06a410 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.788+0000 7fb84b7fe700 1 -- 172.21.15.18:0/789545812 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb84c09ac70 con 0x7fb85c06a410 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.790+0000 7fb860aa1700 1 -- 172.21.15.18:0/789545812 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb8440a21e0 msgr2=0x7fb8440a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.790+0000 7fb860aa1700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb8440a21e0 0x7fb8440a46a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb85c108370 tx=0x7fb850009260 comp rx=0 tx=0).stop 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.790+0000 7fb860aa1700 1 -- 172.21.15.18:0/789545812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb85c06a410 msgr2=0x7fb85c100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.790+0000 7fb860aa1700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb85c06a410 0x7fb85c100c70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00ef10 tx=0x7fb84c00c3a0 comp rx=0 tx=0).stop 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.790+0000 7fb860aa1700 1 -- 172.21.15.18:0/789545812 shutdown_connections 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.790+0000 7fb860aa1700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb85c106f80 0x7fb85c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.790+0000 7fb860aa1700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb8440a21e0 0x7fb8440a46a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.790+0000 7fb860aa1700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb85c06a410 0x7fb85c100c70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.790+0000 7fb860aa1700 1 --2- 172.21.15.18:0/789545812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb85c06adc0 0x7fb85c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.790+0000 7fb860aa1700 1 -- 172.21.15.18:0/789545812 >> 172.21.15.18:0/789545812 conn(0x7fb85c076bc0 msgr2=0x7fb85c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.790+0000 7fb860aa1700 1 -- 172.21.15.18:0/789545812 shutdown_connections 2024-02-17T12:48:47.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:46.790+0000 7fb860aa1700 1 -- 172.21.15.18:0/789545812 wait complete. 2024-02-17T12:48:47.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:47 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:47.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:47 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:47.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:47 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:47.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:47 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:47.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:47 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/246365775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-17T12:48:47.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:47 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/789545812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-17T12:48:47.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:47 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:47.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:47 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:47.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:47 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:47.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:47 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:47.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:47 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/246365775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-17T12:48:47.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:47 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/789545812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-17T12:48:47.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:47 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:47.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:47 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:47.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:47 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:47.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:47 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:47.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:47 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/246365775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-17T12:48:47.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:47 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/789545812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-17T12:48:47.736 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574923 for osd.1 2024-02-17T12:48:47.736 DEBUG:teuthology.parallel:result is None 2024-02-17T12:48:48.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:48 smithi018 ceph-mon[32625]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-02-17T12:48:48.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.410+0000 7f7172441700 1 -- 172.21.15.18:0/2221867236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f716c101eb0 msgr2=0x7f716c102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:48.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.410+0000 7f7172441700 1 --2- 172.21.15.18:0/2221867236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f716c101eb0 0x7f716c102290 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f715c009990 tx=0x7f715c009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:48.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.410+0000 7f7172441700 1 -- 172.21.15.18:0/2221867236 shutdown_connections 2024-02-17T12:48:48.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.410+0000 7f7172441700 1 --2- 172.21.15.18:0/2221867236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716c106f90 0x7f716c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.410+0000 7f7172441700 1 --2- 172.21.15.18:0/2221867236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f716c101eb0 0x7f716c102290 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.410+0000 7f7172441700 1 --2- 172.21.15.18:0/2221867236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c102860 0x7f716c1068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.410+0000 7f7172441700 1 -- 172.21.15.18:0/2221867236 >> 172.21.15.18:0/2221867236 conn(0x7f716c0fd720 msgr2=0x7f716c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.410+0000 7f7172441700 1 -- 172.21.15.18:0/2221867236 shutdown_connections 2024-02-17T12:48:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.410+0000 7f7172441700 1 -- 172.21.15.18:0/2221867236 wait complete. 2024-02-17T12:48:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.411+0000 7f7172441700 1 Processor -- start 2024-02-17T12:48:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.411+0000 7f7172441700 1 -- start start 2024-02-17T12:48:48.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.411+0000 7f7172441700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f716c101eb0 0x7f716c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.411+0000 7f7172441700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c102860 0x7f716c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.411+0000 7f7172441700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716c106f90 0x7f716c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.411+0000 7f7172441700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c19d160 con 0x7f716c101eb0 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.411+0000 7f7172441700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c19d2d0 con 0x7f716c102860 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.411+0000 7f7172441700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c19d490 con 0x7f716c106f90 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.411+0000 7f716bfff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f716c101eb0 0x7f716c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.411+0000 7f716bfff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f716c101eb0 0x7f716c1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44310/0 (socket says 172.21.15.18:44310) 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.411+0000 7f716bfff700 1 -- 172.21.15.18:0/3147248889 learned_addr learned my addr 172.21.15.18:0/3147248889 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f71709de700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716c106f90 0x7f716c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f716b7fe700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c102860 0x7f716c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f716bfff700 1 -- 172.21.15.18:0/3147248889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716c106f90 msgr2=0x7f716c19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f716bfff700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716c106f90 0x7f716c19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f716bfff700 1 -- 172.21.15.18:0/3147248889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c102860 msgr2=0x7f716c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f716bfff700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c102860 0x7f716c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f716bfff700 1 -- 172.21.15.18:0/3147248889 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c0095f0 con 0x7f716c101eb0 2024-02-17T12:48:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f716b7fe700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c102860 0x7f716c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:48:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f716bfff700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f716c101eb0 0x7f716c1982d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f715c009fd0 tx=0x7f715c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f71697fa700 1 -- 172.21.15.18:0/3147248889 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c01d070 con 0x7f716c101eb0 2024-02-17T12:48:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f71697fa700 1 -- 172.21.15.18:0/3147248889 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f715c005510 con 0x7f716c101eb0 2024-02-17T12:48:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f7172441700 1 -- 172.21.15.18:0/3147248889 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c19d710 con 0x7f716c101eb0 2024-02-17T12:48:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f71697fa700 1 -- 172.21.15.18:0/3147248889 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c004d20 con 0x7f716c101eb0 2024-02-17T12:48:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.412+0000 7f7172441700 1 -- 172.21.15.18:0/3147248889 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c1a4fd0 con 0x7f716c101eb0 2024-02-17T12:48:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.413+0000 7f7172441700 1 -- 172.21.15.18:0/3147248889 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c10a250 con 0x7f716c101eb0 2024-02-17T12:48:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.414+0000 7f71697fa700 1 -- 172.21.15.18:0/3147248889 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f715c00fc10 con 0x7f716c101eb0 2024-02-17T12:48:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.415+0000 7f71697fa700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f71540a2260 0x7f71540a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.415+0000 7f716b7fe700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f71540a2260 0x7f71540a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.415+0000 7f71697fa700 1 -- 172.21.15.18:0/3147248889 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f715c0cedf0 con 0x7f716c101eb0 2024-02-17T12:48:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.416+0000 7f716b7fe700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f71540a2260 0x7f71540a4720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f716c1084e0 tx=0x7f7160008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.419+0000 7f71697fa700 1 -- 172.21.15.18:0/3147248889 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f715c097da0 con 0x7f716c101eb0 2024-02-17T12:48:48.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:48 smithi146 ceph-mon[36454]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-02-17T12:48:48.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:48 smithi119 ceph-mon[36188]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-02-17T12:48:48.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.623+0000 7f7172441700 1 -- 172.21.15.18:0/3147248889 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f716c1995e0 con 0x7f716c101eb0 2024-02-17T12:48:48.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.624+0000 7f71697fa700 1 -- 172.21.15.18:0/3147248889 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f715c09b460 con 0x7f716c101eb0 2024-02-17T12:48:48.616 INFO:teuthology.orchestra.run.smithi018.stdout:292057776164 2024-02-17T12:48:48.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.626+0000 7f7172441700 1 -- 172.21.15.18:0/3147248889 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f71540a2260 msgr2=0x7f71540a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:48.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.626+0000 7f7172441700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f71540a2260 0x7f71540a4720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f716c1084e0 tx=0x7f7160008040 comp rx=0 tx=0).stop 2024-02-17T12:48:48.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.626+0000 7f7172441700 1 -- 172.21.15.18:0/3147248889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f716c101eb0 msgr2=0x7f716c1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:48.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.626+0000 7f7172441700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f716c101eb0 0x7f716c1982d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f715c009fd0 tx=0x7f715c00f690 comp rx=0 tx=0).stop 2024-02-17T12:48:48.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.626+0000 7f7172441700 1 -- 172.21.15.18:0/3147248889 shutdown_connections 2024-02-17T12:48:48.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.626+0000 7f7172441700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f71540a2260 0x7f71540a4720 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:48.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.626+0000 7f7172441700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716c106f90 0x7f716c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:48.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.626+0000 7f7172441700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f716c101eb0 0x7f716c1982d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:48.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.626+0000 7f7172441700 1 --2- 172.21.15.18:0/3147248889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c102860 0x7f716c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:48.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.626+0000 7f7172441700 1 -- 172.21.15.18:0/3147248889 >> 172.21.15.18:0/3147248889 conn(0x7f716c0fd720 msgr2=0x7f716c0ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:48.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.626+0000 7f7172441700 1 -- 172.21.15.18:0/3147248889 shutdown_connections 2024-02-17T12:48:48.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:48.626+0000 7f7172441700 1 -- 172.21.15.18:0/3147248889 wait complete. 2024-02-17T12:48:49.237 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:49 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3147248889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-17T12:48:49.262 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776159 got 292057776164 for osd.9 2024-02-17T12:48:49.262 DEBUG:teuthology.parallel:result is None 2024-02-17T12:48:49.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:49 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3147248889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-17T12:48:49.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:49 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3147248889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-17T12:48:49.655 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920823 got 141733920828 for osd.4 2024-02-17T12:48:49.656 DEBUG:teuthology.parallel:result is None 2024-02-17T12:48:50.208 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:50 smithi018 ceph-mon[32625]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:48:50.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:50 smithi146 ceph-mon[36454]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:48:50.584 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738442 got 34359738447 for osd.0 2024-02-17T12:48:50.585 DEBUG:teuthology.parallel:result is None 2024-02-17T12:48:50.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:50 smithi119 ceph-mon[36188]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:48:50.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.898+0000 7f83c80a4700 1 -- 172.21.15.18:0/1884188208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83c0101530 msgr2=0x7f83c010d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:50.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.898+0000 7f83c80a4700 1 --2- 172.21.15.18:0/1884188208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83c0101530 0x7f83c010d9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f83b800b0a0 tx=0x7f83b800b3b0 comp rx=0 tx=0).stop 2024-02-17T12:48:50.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.898+0000 7f83c80a4700 1 -- 172.21.15.18:0/1884188208 shutdown_connections 2024-02-17T12:48:50.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.898+0000 7f83c80a4700 1 --2- 172.21.15.18:0/1884188208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83c0101530 0x7f83c010d9c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:50.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.898+0000 7f83c80a4700 1 --2- 172.21.15.18:0/1884188208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c0106b80 0x7f83c0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:50.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.898+0000 7f83c80a4700 1 --2- 172.21.15.18:0/1884188208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83c0100b70 0x7f83c0100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:50.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.898+0000 7f83c80a4700 1 -- 172.21.15.18:0/1884188208 >> 172.21.15.18:0/1884188208 conn(0x7f83c0077aa0 msgr2=0x7f83c0077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:50.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.898+0000 7f83c80a4700 1 -- 172.21.15.18:0/1884188208 shutdown_connections 2024-02-17T12:48:50.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.898+0000 7f83c80a4700 1 -- 172.21.15.18:0/1884188208 wait complete. 2024-02-17T12:48:50.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c80a4700 1 Processor -- start 2024-02-17T12:48:50.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c80a4700 1 -- start start 2024-02-17T12:48:50.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c80a4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c0100b70 0x7f83c0198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:50.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c80a4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83c0101530 0x7f83c0199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:50.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c80a4700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83c0106b80 0x7f83c01a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:50.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c80a4700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c0199dc0 con 0x7f83c0106b80 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c80a4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c01a45a0 con 0x7f83c0100b70 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c80a4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c01a46e0 con 0x7f83c0101530 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c6641700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83c0106b80 0x7f83c01a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c6641700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83c0106b80 0x7f83c01a4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44638/0 (socket says 172.21.15.18:44638) 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c6641700 1 -- 172.21.15.18:0/962696200 learned_addr learned my addr 172.21.15.18:0/962696200 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c563f700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83c0101530 0x7f83c0199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c6641700 1 -- 172.21.15.18:0/962696200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83c0101530 msgr2=0x7f83c0199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c6641700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83c0101530 0x7f83c0199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c6641700 1 -- 172.21.15.18:0/962696200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c0100b70 msgr2=0x7f83c0198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c5e40700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c0100b70 0x7f83c0198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c6641700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c0100b70 0x7f83c0198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c6641700 1 -- 172.21.15.18:0/962696200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b8009d00 con 0x7f83c0106b80 2024-02-17T12:48:50.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.899+0000 7f83c563f700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83c0101530 0x7f83c0199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:48:50.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.900+0000 7f83c6641700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83c0106b80 0x7f83c01a4060 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f83b8004e40 tx=0x7f83b8004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:50.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.900+0000 7f83b6ffd700 1 -- 172.21.15.18:0/962696200 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b800e040 con 0x7f83c0106b80 2024-02-17T12:48:50.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.900+0000 7f83b6ffd700 1 -- 172.21.15.18:0/962696200 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83b8007a30 con 0x7f83c0106b80 2024-02-17T12:48:50.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.900+0000 7f83c80a4700 1 -- 172.21.15.18:0/962696200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c01a4930 con 0x7f83c0106b80 2024-02-17T12:48:50.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.900+0000 7f83b6ffd700 1 -- 172.21.15.18:0/962696200 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b801b920 con 0x7f83c0106b80 2024-02-17T12:48:50.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.900+0000 7f83c80a4700 1 -- 172.21.15.18:0/962696200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c01a4da0 con 0x7f83c0106b80 2024-02-17T12:48:50.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.901+0000 7f83c80a4700 1 -- 172.21.15.18:0/962696200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c0109020 con 0x7f83c0106b80 2024-02-17T12:48:50.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.902+0000 7f83b6ffd700 1 -- 172.21.15.18:0/962696200 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f83b801dbf0 con 0x7f83c0106b80 2024-02-17T12:48:50.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.903+0000 7f83b6ffd700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f83ac0a21e0 0x7f83ac0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:50.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.903+0000 7f83c5e40700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f83ac0a21e0 0x7f83ac0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:50.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.903+0000 7f83b6ffd700 1 -- 172.21.15.18:0/962696200 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f83b80cfd50 con 0x7f83c0106b80 2024-02-17T12:48:50.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.903+0000 7f83c5e40700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f83ac0a21e0 0x7f83ac0a46a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005fd0 tx=0x7f83b0005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:50.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:50.906+0000 7f83b6ffd700 1 -- 172.21.15.18:0/962696200 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f83b8098d40 con 0x7f83c0106b80 2024-02-17T12:48:51.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.116+0000 7f83c80a4700 1 -- 172.21.15.18:0/962696200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f83c0110b80 con 0x7f83c0106b80 2024-02-17T12:48:51.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.117+0000 7f83b6ffd700 1 -- 172.21.15.18:0/962696200 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f83b809c400 con 0x7f83c0106b80 2024-02-17T12:48:51.109 INFO:teuthology.orchestra.run.smithi018.stdout:163208757303 2024-02-17T12:48:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.120+0000 7f83c80a4700 1 -- 172.21.15.18:0/962696200 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f83ac0a21e0 msgr2=0x7f83ac0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.120+0000 7f83c80a4700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f83ac0a21e0 0x7f83ac0a46a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005fd0 tx=0x7f83b0005ee0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.120+0000 7f83c80a4700 1 -- 172.21.15.18:0/962696200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83c0106b80 msgr2=0x7f83c01a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.120+0000 7f83c80a4700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83c0106b80 0x7f83c01a4060 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f83b8004e40 tx=0x7f83b8004f20 comp rx=0 tx=0).stop 2024-02-17T12:48:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.120+0000 7f83c80a4700 1 -- 172.21.15.18:0/962696200 shutdown_connections 2024-02-17T12:48:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.120+0000 7f83c80a4700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83c0106b80 0x7f83c01a4060 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.120+0000 7f83c80a4700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c0100b70 0x7f83c0198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.120+0000 7f83c80a4700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f83ac0a21e0 0x7f83ac0a46a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.120+0000 7f83c80a4700 1 --2- 172.21.15.18:0/962696200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83c0101530 0x7f83c0199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.121+0000 7f83c80a4700 1 -- 172.21.15.18:0/962696200 >> 172.21.15.18:0/962696200 conn(0x7f83c0077aa0 msgr2=0x7f83c00ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.121+0000 7f83c80a4700 1 -- 172.21.15.18:0/962696200 shutdown_connections 2024-02-17T12:48:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.121+0000 7f83c80a4700 1 -- 172.21.15.18:0/962696200 wait complete. 2024-02-17T12:48:51.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.206+0000 7f6f9889a700 1 -- 172.21.15.18:0/3671433272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f94106ce0 msgr2=0x7f6f941070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:51.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.206+0000 7f6f9889a700 1 --2- 172.21.15.18:0/3671433272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f94106ce0 0x7f6f941070c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009990 tx=0x7f6f84009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.207+0000 7f6f9889a700 1 -- 172.21.15.18:0/3671433272 shutdown_connections 2024-02-17T12:48:51.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.207+0000 7f6f9889a700 1 --2- 172.21.15.18:0/3671433272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f94100c20 0x7f6f9410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.207+0000 7f6f9889a700 1 --2- 172.21.15.18:0/3671433272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f94106ce0 0x7f6f941070c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.207+0000 7f6f9889a700 1 --2- 172.21.15.18:0/3671433272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f94100260 0x7f6f941006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.207+0000 7f6f9889a700 1 -- 172.21.15.18:0/3671433272 >> 172.21.15.18:0/3671433272 conn(0x7f6f94077170 msgr2=0x7f6f940fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:51.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.207+0000 7f6f9889a700 1 -- 172.21.15.18:0/3671433272 shutdown_connections 2024-02-17T12:48:51.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.207+0000 7f6f9889a700 1 -- 172.21.15.18:0/3671433272 wait complete. 2024-02-17T12:48:51.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f9889a700 1 Processor -- start 2024-02-17T12:48:51.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f9889a700 1 -- start start 2024-02-17T12:48:51.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f9889a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f94100260 0x7f6f94101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:51.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f9889a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f94100c20 0x7f6f94101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:51.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f9889a700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f94106ce0 0x7f6f94101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:51.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f9889a700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f941a6670 con 0x7f6f94106ce0 2024-02-17T12:48:51.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f9889a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f941a67e0 con 0x7f6f94100260 2024-02-17T12:48:51.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f9889a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f941a69a0 con 0x7f6f94100c20 2024-02-17T12:48:51.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f92d9d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f94106ce0 0x7f6f94101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:51.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f92d9d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f94106ce0 0x7f6f94101ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44646/0 (socket says 172.21.15.18:44646) 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f92d9d700 1 -- 172.21.15.18:0/2829180662 learned_addr learned my addr 172.21.15.18:0/2829180662 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f91d9b700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f94100c20 0x7f6f94101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.208+0000 7f6f9259c700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f94100260 0x7f6f94101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.209+0000 7f6f91d9b700 1 -- 172.21.15.18:0/2829180662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f94100260 msgr2=0x7f6f94101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.209+0000 7f6f91d9b700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f94100260 0x7f6f94101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.209+0000 7f6f91d9b700 1 -- 172.21.15.18:0/2829180662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f94106ce0 msgr2=0x7f6f94101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.209+0000 7f6f91d9b700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f94106ce0 0x7f6f94101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.209+0000 7f6f91d9b700 1 -- 172.21.15.18:0/2829180662 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f840095f0 con 0x7f6f94100c20 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.209+0000 7f6f92d9d700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f94106ce0 0x7f6f94101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.209+0000 7f6f91d9b700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f94100c20 0x7f6f94101980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800d750 tx=0x7f6f8800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.209+0000 7f6f837fe700 1 -- 172.21.15.18:0/2829180662 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88004020 con 0x7f6f94100c20 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.209+0000 7f6f9889a700 1 -- 172.21.15.18:0/2829180662 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f941a6c80 con 0x7f6f94100c20 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.209+0000 7f6f9889a700 1 -- 172.21.15.18:0/2829180662 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f941a7180 con 0x7f6f94100c20 2024-02-17T12:48:51.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.209+0000 7f6f837fe700 1 -- 172.21.15.18:0/2829180662 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f88004830 con 0x7f6f94100c20 2024-02-17T12:48:51.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.209+0000 7f6f837fe700 1 -- 172.21.15.18:0/2829180662 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880055d0 con 0x7f6f94100c20 2024-02-17T12:48:51.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.210+0000 7f6f9889a700 1 -- 172.21.15.18:0/2829180662 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f74005330 con 0x7f6f94100c20 2024-02-17T12:48:51.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.212+0000 7f6f837fe700 1 -- 172.21.15.18:0/2829180662 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f6f88004180 con 0x7f6f94100c20 2024-02-17T12:48:51.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.213+0000 7f6f837fe700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6f7c0a21e0 0x7f6f7c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:51.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.213+0000 7f6f837fe700 1 -- 172.21.15.18:0/2829180662 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6f880cd9b0 con 0x7f6f94100c20 2024-02-17T12:48:51.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.213+0000 7f6f9259c700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6f7c0a21e0 0x7f6f7c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:51.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.214+0000 7f6f9259c700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6f7c0a21e0 0x7f6f7c0a46a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400b2e0 tx=0x7f6f84005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:51.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.216+0000 7f6f837fe700 1 -- 172.21.15.18:0/2829180662 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6f880967f0 con 0x7f6f94100c20 2024-02-17T12:48:51.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-02-17T12:48:51.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-02-17T12:48:51.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-02-17T12:48:51.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-02-17T12:48:51.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-02-17T12:48:51.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-02-17T12:48:51.236 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:51 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-02-17T12:48:51.236 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:51 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/962696200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-17T12:48:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.421+0000 7f6f9889a700 1 -- 172.21.15.18:0/2829180662 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6f740058b0 con 0x7f6f94100c20 2024-02-17T12:48:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.422+0000 7f6f837fe700 1 -- 172.21.15.18:0/2829180662 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f8801d020 con 0x7f6f94100c20 2024-02-17T12:48:51.414 INFO:teuthology.orchestra.run.smithi018.stdout:107374182465 2024-02-17T12:48:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.424+0000 7f6f9889a700 1 -- 172.21.15.18:0/2829180662 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6f7c0a21e0 msgr2=0x7f6f7c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.424+0000 7f6f9889a700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6f7c0a21e0 0x7f6f7c0a46a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400b2e0 tx=0x7f6f84005960 comp rx=0 tx=0).stop 2024-02-17T12:48:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.424+0000 7f6f9889a700 1 -- 172.21.15.18:0/2829180662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f94100c20 msgr2=0x7f6f94101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.424+0000 7f6f9889a700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f94100c20 0x7f6f94101980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800d750 tx=0x7f6f8800da60 comp rx=0 tx=0).stop 2024-02-17T12:48:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.424+0000 7f6f9889a700 1 -- 172.21.15.18:0/2829180662 shutdown_connections 2024-02-17T12:48:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.424+0000 7f6f9889a700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f94106ce0 0x7f6f94101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.424+0000 7f6f9889a700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f94100260 0x7f6f94101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.424+0000 7f6f9889a700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6f7c0a21e0 0x7f6f7c0a46a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.424+0000 7f6f9889a700 1 --2- 172.21.15.18:0/2829180662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f94100c20 0x7f6f94101980 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.424+0000 7f6f9889a700 1 -- 172.21.15.18:0/2829180662 >> 172.21.15.18:0/2829180662 conn(0x7f6f94077170 msgr2=0x7f6f94104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.424+0000 7f6f9889a700 1 -- 172.21.15.18:0/2829180662 shutdown_connections 2024-02-17T12:48:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:51.424+0000 7f6f9889a700 1 -- 172.21.15.18:0/2829180662 wait complete. 2024-02-17T12:48:51.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-02-17T12:48:51.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-02-17T12:48:51.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-02-17T12:48:51.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-02-17T12:48:51.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-02-17T12:48:51.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-02-17T12:48:51.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:51 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-02-17T12:48:51.452 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:51 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/962696200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-17T12:48:51.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-02-17T12:48:51.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-02-17T12:48:51.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-02-17T12:48:51.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-02-17T12:48:51.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-02-17T12:48:51.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-02-17T12:48:51.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:51 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-02-17T12:48:51.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:51 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/962696200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-17T12:48:51.785 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757299 got 163208757303 for osd.5 2024-02-17T12:48:51.786 DEBUG:teuthology.parallel:result is None 2024-02-17T12:48:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.019+0000 7f74aa415700 1 -- 172.21.15.18:0/626318001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74a406a610 msgr2=0x7f74a406a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.019+0000 7f74aa415700 1 --2- 172.21.15.18:0/626318001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74a406a610 0x7f74a406a9f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f74940099c0 tx=0x7f7494009cd0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.020+0000 7f74aa415700 1 -- 172.21.15.18:0/626318001 shutdown_connections 2024-02-17T12:48:52.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.020+0000 7f74aa415700 1 --2- 172.21.15.18:0/626318001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74a41070b0 0x7f74a410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.020+0000 7f74aa415700 1 --2- 172.21.15.18:0/626318001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74a406a610 0x7f74a406a9f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.020+0000 7f74aa415700 1 --2- 172.21.15.18:0/626318001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74a406afc0 0x7f74a4106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.020+0000 7f74aa415700 1 -- 172.21.15.18:0/626318001 >> 172.21.15.18:0/626318001 conn(0x7f74a40779d0 msgr2=0x7f74a4077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:52.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.020+0000 7f74aa415700 1 -- 172.21.15.18:0/626318001 shutdown_connections 2024-02-17T12:48:52.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.020+0000 7f74aa415700 1 -- 172.21.15.18:0/626318001 wait complete. 2024-02-17T12:48:52.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.020+0000 7f74aa415700 1 Processor -- start 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.020+0000 7f74aa415700 1 -- start start 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74aa415700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74a406a610 0x7f74a4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74aa415700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74a406afc0 0x7f74a41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74aa415700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74a41070b0 0x7f74a419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74aa415700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a419d100 con 0x7f74a41070b0 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74aa415700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a419d270 con 0x7f74a406a610 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74aa415700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a419d430 con 0x7f74a406afc0 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74a89b2700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74a41070b0 0x7f74a419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74a89b2700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74a41070b0 0x7f74a419caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44664/0 (socket says 172.21.15.18:44664) 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74a89b2700 1 -- 172.21.15.18:0/304311854 learned_addr learned my addr 172.21.15.18:0/304311854 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74a3fff700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74a406a610 0x7f74a4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74a37fe700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74a406afc0 0x7f74a41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74a89b2700 1 -- 172.21.15.18:0/304311854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74a406afc0 msgr2=0x7f74a41987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74a89b2700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74a406afc0 0x7f74a41987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74a89b2700 1 -- 172.21.15.18:0/304311854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74a406a610 msgr2=0x7f74a4198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74a89b2700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74a406a610 0x7f74a4198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.021+0000 7f74a89b2700 1 -- 172.21.15.18:0/304311854 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940095f0 con 0x7f74a41070b0 2024-02-17T12:48:52.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.022+0000 7f74a89b2700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74a41070b0 0x7f74a419caf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f749c00b9e0 tx=0x7f749c00bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:52.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.022+0000 7f74a17fa700 1 -- 172.21.15.18:0/304311854 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00d8e0 con 0x7f74a41070b0 2024-02-17T12:48:52.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.022+0000 7f74aa415700 1 -- 172.21.15.18:0/304311854 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a419d710 con 0x7f74a41070b0 2024-02-17T12:48:52.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.022+0000 7f74a17fa700 1 -- 172.21.15.18:0/304311854 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f749c015420 con 0x7f74a41070b0 2024-02-17T12:48:52.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.022+0000 7f74a17fa700 1 -- 172.21.15.18:0/304311854 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c014630 con 0x7f74a41070b0 2024-02-17T12:48:52.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.022+0000 7f74aa415700 1 -- 172.21.15.18:0/304311854 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a4101020 con 0x7f74a41070b0 2024-02-17T12:48:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.023+0000 7f74aa415700 1 -- 172.21.15.18:0/304311854 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a410a2f0 con 0x7f74a41070b0 2024-02-17T12:48:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.024+0000 7f74a17fa700 1 -- 172.21.15.18:0/304311854 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f749c015900 con 0x7f74a41070b0 2024-02-17T12:48:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.024+0000 7f74a17fa700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f74900a21e0 0x7f74900a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.025+0000 7f74a3fff700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f74900a21e0 0x7f74900a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.025+0000 7f74a17fa700 1 -- 172.21.15.18:0/304311854 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f749c011070 con 0x7f74a41070b0 2024-02-17T12:48:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.025+0000 7f74a3fff700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f74900a21e0 0x7f74900a46a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7494009f50 tx=0x7f7494005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:52.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.029+0000 7f74a17fa700 1 -- 172.21.15.18:0/304311854 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f749c04fd20 con 0x7f74a41070b0 2024-02-17T12:48:52.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.210+0000 7f665a382700 1 -- 172.21.15.18:0/3597945543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6654100260 msgr2=0x7f66541006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.210+0000 7f665a382700 1 --2- 172.21.15.18:0/3597945543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6654100260 0x7f66541006e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6648009990 tx=0x7f6648009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.210+0000 7f665a382700 1 -- 172.21.15.18:0/3597945543 shutdown_connections 2024-02-17T12:48:52.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.210+0000 7f665a382700 1 --2- 172.21.15.18:0/3597945543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6654100c20 0x7f665410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.210+0000 7f665a382700 1 --2- 172.21.15.18:0/3597945543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6654106ce0 0x7f66541070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.210+0000 7f665a382700 1 --2- 172.21.15.18:0/3597945543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6654100260 0x7f66541006e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.210+0000 7f665a382700 1 -- 172.21.15.18:0/3597945543 >> 172.21.15.18:0/3597945543 conn(0x7f6654077170 msgr2=0x7f66540fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:52.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.210+0000 7f665a382700 1 -- 172.21.15.18:0/3597945543 shutdown_connections 2024-02-17T12:48:52.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.210+0000 7f665a382700 1 -- 172.21.15.18:0/3597945543 wait complete. 2024-02-17T12:48:52.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.211+0000 7f665a382700 1 Processor -- start 2024-02-17T12:48:52.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.211+0000 7f665a382700 1 -- start start 2024-02-17T12:48:52.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.211+0000 7f665a382700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6654100260 0x7f6654101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:52.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.211+0000 7f665a382700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6654100c20 0x7f6654101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:52.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665a382700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6654106ce0 0x7f6654101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:52.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665a382700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66541a6520 con 0x7f6654106ce0 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665a382700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66541a6690 con 0x7f6654100c20 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665a382700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66541a6850 con 0x7f6654100260 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665891f700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6654106ce0 0x7f6654101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665891f700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6654106ce0 0x7f6654101ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44678/0 (socket says 172.21.15.18:44678) 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665891f700 1 -- 172.21.15.18:0/2606736348 learned_addr learned my addr 172.21.15.18:0/2606736348 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f6653fff700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6654100260 0x7f6654101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f66537fe700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6654100c20 0x7f6654101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665891f700 1 -- 172.21.15.18:0/2606736348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6654100260 msgr2=0x7f6654101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665891f700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6654100260 0x7f6654101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665891f700 1 -- 172.21.15.18:0/2606736348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6654100c20 msgr2=0x7f6654101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665891f700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6654100c20 0x7f6654101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665891f700 1 -- 172.21.15.18:0/2606736348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66480095f0 con 0x7f6654106ce0 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.212+0000 7f665891f700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6654106ce0 0x7f6654101ec0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f664c0094f0 tx=0x7f664c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:52.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.213+0000 7f66517fa700 1 -- 172.21.15.18:0/2606736348 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c004db0 con 0x7f6654106ce0 2024-02-17T12:48:52.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.213+0000 7f66517fa700 1 -- 172.21.15.18:0/2606736348 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f664c00f040 con 0x7f6654106ce0 2024-02-17T12:48:52.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.213+0000 7f66517fa700 1 -- 172.21.15.18:0/2606736348 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c005a10 con 0x7f6654106ce0 2024-02-17T12:48:52.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.213+0000 7f665a382700 1 -- 172.21.15.18:0/2606736348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66541a6b30 con 0x7f6654106ce0 2024-02-17T12:48:52.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.213+0000 7f665a382700 1 -- 172.21.15.18:0/2606736348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66541a6f20 con 0x7f6654106ce0 2024-02-17T12:48:52.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.214+0000 7f665a382700 1 -- 172.21.15.18:0/2606736348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6654103580 con 0x7f6654106ce0 2024-02-17T12:48:52.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.215+0000 7f66517fa700 1 -- 172.21.15.18:0/2606736348 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f664c0050b0 con 0x7f6654106ce0 2024-02-17T12:48:52.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.215+0000 7f66517fa700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f663c0a2260 0x7f663c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:52.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.215+0000 7f6653fff700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f663c0a2260 0x7f663c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:52.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.216+0000 7f66517fa700 1 -- 172.21.15.18:0/2606736348 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f664c0cd6d0 con 0x7f6654106ce0 2024-02-17T12:48:52.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.216+0000 7f6653fff700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f663c0a2260 0x7f663c0a4720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6644005950 tx=0x7f664400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:52.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.220+0000 7f66517fa700 1 -- 172.21.15.18:0/2606736348 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f664c04fd20 con 0x7f6654106ce0 2024-02-17T12:48:52.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.246+0000 7f74aa415700 1 -- 172.21.15.18:0/304311854 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f74a4103430 con 0x7f74a41070b0 2024-02-17T12:48:52.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.247+0000 7f74a17fa700 1 -- 172.21.15.18:0/304311854 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f749c09a130 con 0x7f74a41070b0 2024-02-17T12:48:52.239 INFO:teuthology.orchestra.run.smithi018.stdout:257698037801 2024-02-17T12:48:52.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.249+0000 7f74aa415700 1 -- 172.21.15.18:0/304311854 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f74900a21e0 msgr2=0x7f74900a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.249+0000 7f74aa415700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f74900a21e0 0x7f74900a46a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7494009f50 tx=0x7f7494005e80 comp rx=0 tx=0).stop 2024-02-17T12:48:52.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.249+0000 7f74aa415700 1 -- 172.21.15.18:0/304311854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74a41070b0 msgr2=0x7f74a419caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.249+0000 7f74aa415700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74a41070b0 0x7f74a419caf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f749c00b9e0 tx=0x7f749c00bcf0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.249+0000 7f74aa415700 1 -- 172.21.15.18:0/304311854 shutdown_connections 2024-02-17T12:48:52.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.249+0000 7f74aa415700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74a41070b0 0x7f74a419caf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.249+0000 7f74aa415700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74a406a610 0x7f74a4198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.249+0000 7f74aa415700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f74900a21e0 0x7f74900a46a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.249+0000 7f74aa415700 1 --2- 172.21.15.18:0/304311854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74a406afc0 0x7f74a41987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.249+0000 7f74aa415700 1 -- 172.21.15.18:0/304311854 >> 172.21.15.18:0/304311854 conn(0x7f74a40779d0 msgr2=0x7f74a41005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:52.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.249+0000 7f74aa415700 1 -- 172.21.15.18:0/304311854 shutdown_connections 2024-02-17T12:48:52.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.249+0000 7f74aa415700 1 -- 172.21.15.18:0/304311854 wait complete. 2024-02-17T12:48:52.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:52 smithi018 ceph-mon[32625]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-17T12:48:52.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-02-17T12:48:52.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-02-17T12:48:52.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-02-17T12:48:52.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-02-17T12:48:52.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-02-17T12:48:52.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-02-17T12:48:52.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:52 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-02-17T12:48:52.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:52 smithi018 ceph-mon[32625]: osdmap e87: 12 total, 12 up, 12 in 2024-02-17T12:48:52.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:52 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2829180662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-17T12:48:52.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.427+0000 7f665a382700 1 -- 172.21.15.18:0/2606736348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f6654111e90 con 0x7f6654106ce0 2024-02-17T12:48:52.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.427+0000 7f66517fa700 1 -- 172.21.15.18:0/2606736348 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f664c096510 con 0x7f6654106ce0 2024-02-17T12:48:52.420 INFO:teuthology.orchestra.run.smithi018.stdout:347892351001 2024-02-17T12:48:52.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.429+0000 7f665a382700 1 -- 172.21.15.18:0/2606736348 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f663c0a2260 msgr2=0x7f663c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.429+0000 7f665a382700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f663c0a2260 0x7f663c0a4720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6644005950 tx=0x7f664400b410 comp rx=0 tx=0).stop 2024-02-17T12:48:52.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.429+0000 7f665a382700 1 -- 172.21.15.18:0/2606736348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6654106ce0 msgr2=0x7f6654101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.429+0000 7f665a382700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6654106ce0 0x7f6654101ec0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f664c0094f0 tx=0x7f664c00d3b0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.429+0000 7f665a382700 1 -- 172.21.15.18:0/2606736348 shutdown_connections 2024-02-17T12:48:52.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.429+0000 7f665a382700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6654106ce0 0x7f6654101ec0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.429+0000 7f665a382700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f663c0a2260 0x7f663c0a4720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.429+0000 7f665a382700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6654100260 0x7f6654101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.429+0000 7f665a382700 1 --2- 172.21.15.18:0/2606736348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6654100c20 0x7f6654101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.429+0000 7f665a382700 1 -- 172.21.15.18:0/2606736348 >> 172.21.15.18:0/2606736348 conn(0x7f6654077170 msgr2=0x7f66541047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:52.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.429+0000 7f665a382700 1 -- 172.21.15.18:0/2606736348 shutdown_connections 2024-02-17T12:48:52.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.429+0000 7f665a382700 1 -- 172.21.15.18:0/2606736348 wait complete. 2024-02-17T12:48:52.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:52 smithi146 ceph-mon[36454]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-17T12:48:52.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-02-17T12:48:52.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-02-17T12:48:52.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-02-17T12:48:52.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-02-17T12:48:52.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-02-17T12:48:52.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-02-17T12:48:52.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:52 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-02-17T12:48:52.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:52 smithi146 ceph-mon[36454]: osdmap e87: 12 total, 12 up, 12 in 2024-02-17T12:48:52.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:52 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2829180662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-17T12:48:52.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.479+0000 7f227e937700 1 -- 172.21.15.18:0/3255166170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2278100590 msgr2=0x7f2278100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.479+0000 7f227e937700 1 --2- 172.21.15.18:0/3255166170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2278100590 0x7f2278100970 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f226c009990 tx=0x7f226c009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.479+0000 7f227e937700 1 -- 172.21.15.18:0/3255166170 shutdown_connections 2024-02-17T12:48:52.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.479+0000 7f227e937700 1 --2- 172.21.15.18:0/3255166170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2278105670 0x7f227810d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.479+0000 7f227e937700 1 --2- 172.21.15.18:0/3255166170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2278100590 0x7f2278100970 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.479+0000 7f227e937700 1 --2- 172.21.15.18:0/3255166170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2278100f40 0x7f2278104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.479+0000 7f227e937700 1 -- 172.21.15.18:0/3255166170 >> 172.21.15.18:0/3255166170 conn(0x7f2278077170 msgr2=0x7f22780fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:52.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.479+0000 7f227e937700 1 -- 172.21.15.18:0/3255166170 shutdown_connections 2024-02-17T12:48:52.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.479+0000 7f227e937700 1 -- 172.21.15.18:0/3255166170 wait complete. 2024-02-17T12:48:52.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.480+0000 7f227e937700 1 Processor -- start 2024-02-17T12:48:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.480+0000 7f227e937700 1 -- start start 2024-02-17T12:48:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.480+0000 7f227e937700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2278100590 0x7f2278198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.480+0000 7f227e937700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2278100f40 0x7f22781987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.480+0000 7f227e937700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2278105670 0x7f227819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.480+0000 7f227e937700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227819d170 con 0x7f2278105670 2024-02-17T12:48:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.480+0000 7f227e937700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227819d2e0 con 0x7f2278100590 2024-02-17T12:48:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.480+0000 7f227e937700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227819d4a0 con 0x7f2278100f40 2024-02-17T12:48:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.480+0000 7f227ced4700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2278105670 0x7f227819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.480+0000 7f22777fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2278100f40 0x7f22781987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.480+0000 7f227ced4700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2278105670 0x7f227819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44696/0 (socket says 172.21.15.18:44696) 2024-02-17T12:48:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.480+0000 7f227ced4700 1 -- 172.21.15.18:0/1318465708 learned_addr learned my addr 172.21.15.18:0/1318465708 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f2277fff700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2278100590 0x7f2278198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f227ced4700 1 -- 172.21.15.18:0/1318465708 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2278100f40 msgr2=0x7f22781987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f227ced4700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2278100f40 0x7f22781987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f227ced4700 1 -- 172.21.15.18:0/1318465708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2278100590 msgr2=0x7f2278198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f227ced4700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2278100590 0x7f2278198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f227ced4700 1 -- 172.21.15.18:0/1318465708 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c0095f0 con 0x7f2278105670 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f2277fff700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2278100590 0x7f2278198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f227ced4700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2278105670 0x7f227819cb60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f227000b970 tx=0x7f227000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f22757fa700 1 -- 172.21.15.18:0/1318465708 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227000c930 con 0x7f2278105670 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f22757fa700 1 -- 172.21.15.18:0/1318465708 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2270014440 con 0x7f2278105670 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f22757fa700 1 -- 172.21.15.18:0/1318465708 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22700075d0 con 0x7f2278105670 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f227e937700 1 -- 172.21.15.18:0/1318465708 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227819d780 con 0x7f2278105670 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.481+0000 7f227e937700 1 -- 172.21.15.18:0/1318465708 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22781a4fe0 con 0x7f2278105670 2024-02-17T12:48:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.482+0000 7f227e937700 1 -- 172.21.15.18:0/1318465708 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2278108930 con 0x7f2278105670 2024-02-17T12:48:52.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.483+0000 7f22757fa700 1 -- 172.21.15.18:0/1318465708 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f22700145b0 con 0x7f2278105670 2024-02-17T12:48:52.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.484+0000 7f22757fa700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f22600a2260 0x7f22600a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:52.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.484+0000 7f2277fff700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f22600a2260 0x7f22600a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:52.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.484+0000 7f22757fa700 1 -- 172.21.15.18:0/1318465708 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2270006e40 con 0x7f2278105670 2024-02-17T12:48:52.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.484+0000 7f2277fff700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f22600a2260 0x7f22600a4720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f226c00b5c0 tx=0x7f226c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:52.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.488+0000 7f22757fa700 1 -- 172.21.15.18:0/1318465708 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2270096760 con 0x7f2278105670 2024-02-17T12:48:52.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:52 smithi119 ceph-mon[36188]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-17T12:48:52.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-02-17T12:48:52.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-02-17T12:48:52.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-02-17T12:48:52.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-02-17T12:48:52.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-02-17T12:48:52.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-02-17T12:48:52.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:52 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-02-17T12:48:52.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:52 smithi119 ceph-mon[36188]: osdmap e87: 12 total, 12 up, 12 in 2024-02-17T12:48:52.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:52 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2829180662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-17T12:48:52.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.692+0000 7f227e937700 1 -- 172.21.15.18:0/1318465708 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2278101a70 con 0x7f2278105670 2024-02-17T12:48:52.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.693+0000 7f22757fa700 1 -- 172.21.15.18:0/1318465708 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2270099e20 con 0x7f2278105670 2024-02-17T12:48:52.685 INFO:teuthology.orchestra.run.smithi018.stdout:223338299439 2024-02-17T12:48:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.695+0000 7f227e937700 1 -- 172.21.15.18:0/1318465708 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f22600a2260 msgr2=0x7f22600a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.695+0000 7f227e937700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f22600a2260 0x7f22600a4720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f226c00b5c0 tx=0x7f226c005fb0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.695+0000 7f227e937700 1 -- 172.21.15.18:0/1318465708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2278105670 msgr2=0x7f227819cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.695+0000 7f227e937700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2278105670 0x7f227819cb60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f227000b970 tx=0x7f227000bc80 comp rx=0 tx=0).stop 2024-02-17T12:48:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.695+0000 7f227e937700 1 -- 172.21.15.18:0/1318465708 shutdown_connections 2024-02-17T12:48:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.695+0000 7f227e937700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2278105670 0x7f227819cb60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.695+0000 7f227e937700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2278100590 0x7f2278198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.695+0000 7f227e937700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f22600a2260 0x7f22600a4720 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.695+0000 7f227e937700 1 --2- 172.21.15.18:0/1318465708 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2278100f40 0x7f22781987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.695+0000 7f227e937700 1 -- 172.21.15.18:0/1318465708 >> 172.21.15.18:0/1318465708 conn(0x7f2278077170 msgr2=0x7f22780fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.695+0000 7f227e937700 1 -- 172.21.15.18:0/1318465708 shutdown_connections 2024-02-17T12:48:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:52.695+0000 7f227e937700 1 -- 172.21.15.18:0/1318465708 wait complete. 2024-02-17T12:48:53.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:53 smithi018 ceph-mon[32625]: osdmap e88: 12 total, 12 up, 12 in 2024-02-17T12:48:53.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:53 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/304311854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-17T12:48:53.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:53 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2606736348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-17T12:48:53.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:53 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1318465708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-17T12:48:53.437 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299434 got 223338299439 for osd.7 2024-02-17T12:48:53.437 DEBUG:teuthology.parallel:result is None 2024-02-17T12:48:53.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:53 smithi146 ceph-mon[36454]: osdmap e88: 12 total, 12 up, 12 in 2024-02-17T12:48:53.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:53 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/304311854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-17T12:48:53.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:53 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2606736348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-17T12:48:53.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:53 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1318465708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-17T12:48:53.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:53 smithi119 ceph-mon[36188]: osdmap e88: 12 total, 12 up, 12 in 2024-02-17T12:48:53.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:53 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/304311854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-17T12:48:53.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:53 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2606736348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-17T12:48:53.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:53 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1318465708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-17T12:48:54.109 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350996 got 347892351001 for osd.11 2024-02-17T12:48:54.109 DEBUG:teuthology.parallel:result is None 2024-02-17T12:48:54.409 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:54 smithi018 ceph-mon[32625]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-02-17T12:48:54.435 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182465 for osd.3 2024-02-17T12:48:54.436 DEBUG:teuthology.parallel:result is None 2024-02-17T12:48:54.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:54 smithi146 ceph-mon[36454]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-02-17T12:48:54.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:54 smithi119 ceph-mon[36188]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-02-17T12:48:54.959 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037796 got 257698037801 for osd.8 2024-02-17T12:48:54.959 DEBUG:teuthology.parallel:result is None 2024-02-17T12:48:54.959 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-17T12:48:54.960 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-17T12:48:55.647 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:56.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:56 smithi146 ceph-mon[36454]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-02-17T12:48:56.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:56 smithi018 ceph-mon[32625]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-02-17T12:48:56.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:56 smithi119 ceph-mon[36188]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-02-17T12:48:57.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.326+0000 7f9a30cb5700 1 -- 172.21.15.18:0/4284054642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c06adc0 msgr2=0x7f9a2c1068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:57.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.326+0000 7f9a30cb5700 1 --2- 172.21.15.18:0/4284054642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c06adc0 0x7f9a2c1068c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009990 tx=0x7f9a20009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:57.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.327+0000 7f9a30cb5700 1 -- 172.21.15.18:0/4284054642 shutdown_connections 2024-02-17T12:48:57.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.327+0000 7f9a30cb5700 1 --2- 172.21.15.18:0/4284054642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a2c106ff0 0x7f9a2c10eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:57.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.327+0000 7f9a30cb5700 1 --2- 172.21.15.18:0/4284054642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2c06a410 0x7f9a2c06a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:57.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.327+0000 7f9a30cb5700 1 --2- 172.21.15.18:0/4284054642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c06adc0 0x7f9a2c1068c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:57.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.327+0000 7f9a30cb5700 1 -- 172.21.15.18:0/4284054642 >> 172.21.15.18:0/4284054642 conn(0x7f9a2c076bc0 msgr2=0x7f9a2c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:57.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.327+0000 7f9a30cb5700 1 -- 172.21.15.18:0/4284054642 shutdown_connections 2024-02-17T12:48:57.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.327+0000 7f9a30cb5700 1 -- 172.21.15.18:0/4284054642 wait complete. 2024-02-17T12:48:57.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a30cb5700 1 Processor -- start 2024-02-17T12:48:57.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a30cb5700 1 -- start start 2024-02-17T12:48:57.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a30cb5700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c06a410 0x7f9a2c100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:57.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a30cb5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a2c06adc0 0x7f9a2c1011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:57.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a30cb5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2c106ff0 0x7f9a2c1016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:57.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a30cb5700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c1a6550 con 0x7f9a2c06a410 2024-02-17T12:48:57.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a30cb5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c1a66c0 con 0x7f9a2c06adc0 2024-02-17T12:48:57.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a30cb5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c1a6880 con 0x7f9a2c106ff0 2024-02-17T12:48:57.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a2a59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c06a410 0x7f9a2c100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:57.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a2a59c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c06a410 0x7f9a2c100c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44714/0 (socket says 172.21.15.18:44714) 2024-02-17T12:48:57.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a2a59c700 1 -- 172.21.15.18:0/53032430 learned_addr learned my addr 172.21.15.18:0/53032430 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:57.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a29d9b700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a2c06adc0 0x7f9a2c1011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:57.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.328+0000 7f9a2ad9d700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2c106ff0 0x7f9a2c1016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:57.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.329+0000 7f9a2a59c700 1 -- 172.21.15.18:0/53032430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2c106ff0 msgr2=0x7f9a2c1016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:57.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.329+0000 7f9a2a59c700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2c106ff0 0x7f9a2c1016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:57.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.329+0000 7f9a2a59c700 1 -- 172.21.15.18:0/53032430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a2c06adc0 msgr2=0x7f9a2c1011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:57.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.329+0000 7f9a2a59c700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a2c06adc0 0x7f9a2c1011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:57.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.329+0000 7f9a2a59c700 1 -- 172.21.15.18:0/53032430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a200095f0 con 0x7f9a2c06a410 2024-02-17T12:48:57.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.329+0000 7f9a29d9b700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a2c06adc0 0x7f9a2c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:48:57.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.329+0000 7f9a2a59c700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c06a410 0x7f9a2c100c70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00c9b0 tx=0x7f9a1c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:57.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.329+0000 7f9a1b7fe700 1 -- 172.21.15.18:0/53032430 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c007b00 con 0x7f9a2c06a410 2024-02-17T12:48:57.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.329+0000 7f9a1b7fe700 1 -- 172.21.15.18:0/53032430 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c00ce50 con 0x7f9a2c06a410 2024-02-17T12:48:57.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.329+0000 7f9a30cb5700 1 -- 172.21.15.18:0/53032430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2c1a6b60 con 0x7f9a2c06a410 2024-02-17T12:48:57.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.329+0000 7f9a30cb5700 1 -- 172.21.15.18:0/53032430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c1a7030 con 0x7f9a2c06a410 2024-02-17T12:48:57.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.329+0000 7f9a1b7fe700 1 -- 172.21.15.18:0/53032430 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c018660 con 0x7f9a2c06a410 2024-02-17T12:48:57.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.331+0000 7f9a30cb5700 1 -- 172.21.15.18:0/53032430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a2c10a1e0 con 0x7f9a2c06a410 2024-02-17T12:48:57.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.331+0000 7f9a1b7fe700 1 -- 172.21.15.18:0/53032430 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f9a1c00f450 con 0x7f9a2c06a410 2024-02-17T12:48:57.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.332+0000 7f9a1b7fe700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9a140a21e0 0x7f9a140a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:57.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.332+0000 7f9a29d9b700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9a140a21e0 0x7f9a140a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:57.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.332+0000 7f9a1b7fe700 1 -- 172.21.15.18:0/53032430 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9a1c0cdd20 con 0x7f9a2c06a410 2024-02-17T12:48:57.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.333+0000 7f9a29d9b700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9a140a21e0 0x7f9a140a46a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a20006010 tx=0x7f9a20005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:57.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.340+0000 7f9a1b7fe700 1 -- 172.21.15.18:0/53032430 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a1c096bc0 con 0x7f9a2c06a410 2024-02-17T12:48:57.333 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:57 smithi018 ceph-mon[32625]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-02-17T12:48:57.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.551+0000 7f9a30cb5700 1 -- 172.21.15.18:0/53032430 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a2c102210 con 0x7f9a140a21e0 2024-02-17T12:48:57.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.562+0000 7f9a1b7fe700 1 -- 172.21.15.18:0/53032430 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344290 (secure 0 0 0) 0x7f9a2c102210 con 0x7f9a140a21e0 2024-02-17T12:48:57.559 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.566+0000 7f9a30cb5700 1 -- 172.21.15.18:0/53032430 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9a140a21e0 msgr2=0x7f9a140a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.566+0000 7f9a30cb5700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9a140a21e0 0x7f9a140a46a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a20006010 tx=0x7f9a20005960 comp rx=0 tx=0).stop 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.566+0000 7f9a30cb5700 1 -- 172.21.15.18:0/53032430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c06a410 msgr2=0x7f9a2c100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.566+0000 7f9a30cb5700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c06a410 0x7f9a2c100c70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00c9b0 tx=0x7f9a1c00ccc0 comp rx=0 tx=0).stop 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.566+0000 7f9a30cb5700 1 -- 172.21.15.18:0/53032430 shutdown_connections 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.566+0000 7f9a30cb5700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9a140a21e0 0x7f9a140a46a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.566+0000 7f9a30cb5700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2c106ff0 0x7f9a2c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.566+0000 7f9a30cb5700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c06a410 0x7f9a2c100c70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.566+0000 7f9a30cb5700 1 --2- 172.21.15.18:0/53032430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a2c06adc0 0x7f9a2c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.566+0000 7f9a30cb5700 1 -- 172.21.15.18:0/53032430 >> 172.21.15.18:0/53032430 conn(0x7f9a2c076bc0 msgr2=0x7f9a2c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.567+0000 7f9a30cb5700 1 -- 172.21.15.18:0/53032430 shutdown_connections 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:57.567+0000 7f9a30cb5700 1 -- 172.21.15.18:0/53032430 wait complete. 2024-02-17T12:48:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-02-17T12:48:57.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:57 smithi119 ceph-mon[36188]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-02-17T12:48:57.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:57 smithi146 ceph-mon[36454]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-02-17T12:48:58.142 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-02-17T12:48:56.208330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657200,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":811,"num_read_kb":530,"num_write":419,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181749,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":296,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379312,"kb_used_data":13448,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562512,"statfs":{"total":1151940427776,"available":1151552012288,"internally_reserved":0,"allocated":13770752,"data_stored":7494647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":46,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1013,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001892"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159822+0000","last_change":"2024-02-17T12:47:46.069941+0000","last_active":"2024-02-17T12:48:52.159822+0000","last_peered":"2024-02-17T12:48:52.159822+0000","last_clean":"2024-02-17T12:48:52.159822+0000","last_became_active":"2024-02-17T12:47:46.069192+0000","last_became_peered":"2024-02-17T12:47:46.069192+0000","last_unstale":"2024-02-17T12:48:52.159822+0000","last_undegraded":"2024-02-17T12:48:52.159822+0000","last_fullsized":"2024-02-17T12:48:52.159822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:27.102839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463276+0000","last_change":"2024-02-17T12:47:46.064872+0000","last_active":"2024-02-17T12:48:52.463276+0000","last_peered":"2024-02-17T12:48:52.463276+0000","last_clean":"2024-02-17T12:48:52.463276+0000","last_became_active":"2024-02-17T12:47:46.064498+0000","last_became_peered":"2024-02-17T12:47:46.064498+0000","last_unstale":"2024-02-17T12:48:52.463276+0000","last_undegraded":"2024-02-17T12:48:52.463276+0000","last_fullsized":"2024-02-17T12:48:52.463276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:30:41.432657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463466+0000","last_change":"2024-02-17T12:47:46.067504+0000","last_active":"2024-02-17T12:48:52.463466+0000","last_peered":"2024-02-17T12:48:52.463466+0000","last_clean":"2024-02-17T12:48:52.463466+0000","last_became_active":"2024-02-17T12:47:46.067155+0000","last_became_peered":"2024-02-17T12:47:46.067155+0000","last_unstale":"2024-02-17T12:48:52.463466+0000","last_undegraded":"2024-02-17T12:48:52.463466+0000","last_fullsized":"2024-02-17T12:48:52.463466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:38:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464939+0000","last_change":"2024-02-17T12:47:46.062356+0000","last_active":"2024-02-17T12:48:52.464939+0000","last_peered":"2024-02-17T12:48:52.464939+0000","last_clean":"2024-02-17T12:48:52.464939+0000","last_became_active":"2024-02-17T12:47:46.061988+0000","last_became_peered":"2024-02-17T12:47:46.061988+0000","last_unstale":"2024-02-17T12:48:52.464939+0000","last_undegraded":"2024-02-17T12:48:52.464939+0000","last_fullsized":"2024-02-17T12:48:52.464939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:25:09.120770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.162896+0000","last_change":"2024-02-17T12:47:46.072411+0000","last_active":"2024-02-17T12:48:52.162896+0000","last_peered":"2024-02-17T12:48:52.162896+0000","last_clean":"2024-02-17T12:48:52.162896+0000","last_became_active":"2024-02-17T12:47:46.072067+0000","last_became_peered":"2024-02-17T12:47:46.072067+0000","last_unstale":"2024-02-17T12:48:52.162896+0000","last_undegraded":"2024-02-17T12:48:52.162896+0000","last_fullsized":"2024-02-17T12:48:52.162896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:29:33.051513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464236+0000","last_change":"2024-02-17T12:47:46.057425+0000","last_active":"2024-02-17T12:48:52.464236+0000","last_peered":"2024-02-17T12:48:52.464236+0000","last_clean":"2024-02-17T12:48:52.464236+0000","last_became_active":"2024-02-17T12:47:46.057001+0000","last_became_peered":"2024-02-17T12:47:46.057001+0000","last_unstale":"2024-02-17T12:48:52.464236+0000","last_undegraded":"2024-02-17T12:48:52.464236+0000","last_fullsized":"2024-02-17T12:48:52.464236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:48:24.135879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157182+0000","last_change":"2024-02-17T12:47:46.069143+0000","last_active":"2024-02-17T12:48:52.157182+0000","last_peered":"2024-02-17T12:48:52.157182+0000","last_clean":"2024-02-17T12:48:52.157182+0000","last_became_active":"2024-02-17T12:47:46.068775+0000","last_became_peered":"2024-02-17T12:47:46.068775+0000","last_unstale":"2024-02-17T12:48:52.157182+0000","last_undegraded":"2024-02-17T12:48:52.157182+0000","last_fullsized":"2024-02-17T12:48:52.157182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:32:39.518205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165272+0000","last_change":"2024-02-17T12:47:46.069579+0000","last_active":"2024-02-17T12:48:52.165272+0000","last_peered":"2024-02-17T12:48:52.165272+0000","last_clean":"2024-02-17T12:48:52.165272+0000","last_became_active":"2024-02-17T12:47:46.069114+0000","last_became_peered":"2024-02-17T12:47:46.069114+0000","last_unstale":"2024-02-17T12:48:52.165272+0000","last_undegraded":"2024-02-17T12:48:52.165272+0000","last_fullsized":"2024-02-17T12:48:52.165272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:54:58.100511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.465498+0000","last_change":"2024-02-17T12:47:46.079103+0000","last_active":"2024-02-17T12:48:52.465498+0000","last_peered":"2024-02-17T12:48:52.465498+0000","last_clean":"2024-02-17T12:48:52.465498+0000","last_became_active":"2024-02-17T12:47:46.077992+0000","last_became_peered":"2024-02-17T12:47:46.077992+0000","last_unstale":"2024-02-17T12:48:52.465498+0000","last_undegraded":"2024-02-17T12:48:52.465498+0000","last_fullsized":"2024-02-17T12:48:52.465498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:12:59.337341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464970+0000","last_change":"2024-02-17T12:47:46.062897+0000","last_active":"2024-02-17T12:48:52.464970+0000","last_peered":"2024-02-17T12:48:52.464970+0000","last_clean":"2024-02-17T12:48:52.464970+0000","last_became_active":"2024-02-17T12:47:46.062571+0000","last_became_peered":"2024-02-17T12:47:46.062571+0000","last_unstale":"2024-02-17T12:48:52.464970+0000","last_undegraded":"2024-02-17T12:48:52.464970+0000","last_fullsized":"2024-02-17T12:48:52.464970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:57:02.845400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157655+0000","last_change":"2024-02-17T12:47:46.080183+0000","last_active":"2024-02-17T12:48:52.157655+0000","last_peered":"2024-02-17T12:48:52.157655+0000","last_clean":"2024-02-17T12:48:52.157655+0000","last_became_active":"2024-02-17T12:47:46.079827+0000","last_became_peered":"2024-02-17T12:47:46.079827+0000","last_unstale":"2024-02-17T12:48:52.157655+0000","last_undegraded":"2024-02-17T12:48:52.157655+0000","last_fullsized":"2024-02-17T12:48:52.157655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:56:10.193200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464991+0000","last_change":"2024-02-17T12:47:46.061849+0000","last_active":"2024-02-17T12:48:52.464991+0000","last_peered":"2024-02-17T12:48:52.464991+0000","last_clean":"2024-02-17T12:48:52.464991+0000","last_became_active":"2024-02-17T12:47:46.061458+0000","last_became_peered":"2024-02-17T12:47:46.061458+0000","last_unstale":"2024-02-17T12:48:52.464991+0000","last_undegraded":"2024-02-17T12:48:52.464991+0000","last_fullsized":"2024-02-17T12:48:52.464991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:02:09.141292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163184+0000","last_change":"2024-02-17T12:47:46.069356+0000","last_active":"2024-02-17T12:48:52.163184+0000","last_peered":"2024-02-17T12:48:52.163184+0000","last_clean":"2024-02-17T12:48:52.163184+0000","last_became_active":"2024-02-17T12:47:46.068567+0000","last_became_peered":"2024-02-17T12:47:46.068567+0000","last_unstale":"2024-02-17T12:48:52.163184+0000","last_undegraded":"2024-02-17T12:48:52.163184+0000","last_fullsized":"2024-02-17T12:48:52.163184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:08:01.872558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463404+0000","last_change":"2024-02-17T12:47:46.062684+0000","last_active":"2024-02-17T12:48:52.463404+0000","last_peered":"2024-02-17T12:48:52.463404+0000","last_clean":"2024-02-17T12:48:52.463404+0000","last_became_active":"2024-02-17T12:47:46.062221+0000","last_became_peered":"2024-02-17T12:47:46.062221+0000","last_unstale":"2024-02-17T12:48:52.463404+0000","last_undegraded":"2024-02-17T12:48:52.463404+0000","last_fullsized":"2024-02-17T12:48:52.463404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:37:11.513336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.462993+0000","last_change":"2024-02-17T12:47:46.065961+0000","last_active":"2024-02-17T12:48:52.462993+0000","last_peered":"2024-02-17T12:48:52.462993+0000","last_clean":"2024-02-17T12:48:52.462993+0000","last_became_active":"2024-02-17T12:47:46.065600+0000","last_became_peered":"2024-02-17T12:47:46.065600+0000","last_unstale":"2024-02-17T12:48:52.462993+0000","last_undegraded":"2024-02-17T12:48:52.462993+0000","last_fullsized":"2024-02-17T12:48:52.462993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:24:46.205077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463715+0000","last_change":"2024-02-17T12:47:46.080150+0000","last_active":"2024-02-17T12:48:52.463715+0000","last_peered":"2024-02-17T12:48:52.463715+0000","last_clean":"2024-02-17T12:48:52.463715+0000","last_became_active":"2024-02-17T12:47:46.079813+0000","last_became_peered":"2024-02-17T12:47:46.079813+0000","last_unstale":"2024-02-17T12:48:52.463715+0000","last_undegraded":"2024-02-17T12:48:52.463715+0000","last_fullsized":"2024-02-17T12:48:52.463715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:59:00.868964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159950+0000","last_change":"2024-02-17T12:47:46.070202+0000","last_active":"2024-02-17T12:48:52.159950+0000","last_peered":"2024-02-17T12:48:52.159950+0000","last_clean":"2024-02-17T12:48:52.159950+0000","last_became_active":"2024-02-17T12:47:46.069628+0000","last_became_peered":"2024-02-17T12:47:46.069628+0000","last_unstale":"2024-02-17T12:48:52.159950+0000","last_undegraded":"2024-02-17T12:48:52.159950+0000","last_fullsized":"2024-02-17T12:48:52.159950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:44:17.481681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156569+0000","last_change":"2024-02-17T12:47:46.067810+0000","last_active":"2024-02-17T12:48:52.156569+0000","last_peered":"2024-02-17T12:48:52.156569+0000","last_clean":"2024-02-17T12:48:52.156569+0000","last_became_active":"2024-02-17T12:47:46.067456+0000","last_became_peered":"2024-02-17T12:47:46.067456+0000","last_unstale":"2024-02-17T12:48:52.156569+0000","last_undegraded":"2024-02-17T12:48:52.156569+0000","last_fullsized":"2024-02-17T12:48:52.156569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:27:03.646597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.164082+0000","last_change":"2024-02-17T12:47:46.076268+0000","last_active":"2024-02-17T12:48:52.164082+0000","last_peered":"2024-02-17T12:48:52.164082+0000","last_clean":"2024-02-17T12:48:52.164082+0000","last_became_active":"2024-02-17T12:47:46.075606+0000","last_became_peered":"2024-02-17T12:47:46.075606+0000","last_unstale":"2024-02-17T12:48:52.164082+0000","last_undegraded":"2024-02-17T12:48:52.164082+0000","last_fullsized":"2024-02-17T12:48:52.164082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:16:35.805008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:53.811536+0000","last_change":"2024-02-17T12:47:45.066780+0000","last_active":"2024-02-17T12:48:53.811536+0000","last_peered":"2024-02-17T12:48:53.811536+0000","last_clean":"2024-02-17T12:48:53.811536+0000","last_became_active":"2024-02-17T12:47:45.066039+0000","last_became_peered":"2024-02-17T12:47:45.066039+0000","last_unstale":"2024-02-17T12:48:53.811536+0000","last_undegraded":"2024-02-17T12:48:53.811536+0000","last_fullsized":"2024-02-17T12:48:53.811536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:36:25.420534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165559+0000","last_change":"2024-02-17T12:47:46.079603+0000","last_active":"2024-02-17T12:48:52.165559+0000","last_peered":"2024-02-17T12:48:52.165559+0000","last_clean":"2024-02-17T12:48:52.165559+0000","last_became_active":"2024-02-17T12:47:46.079246+0000","last_became_peered":"2024-02-17T12:47:46.079246+0000","last_unstale":"2024-02-17T12:48:52.165559+0000","last_undegraded":"2024-02-17T12:48:52.165559+0000","last_fullsized":"2024-02-17T12:48:52.165559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:32:04.975778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165983+0000","last_change":"2024-02-17T12:47:45.061868+0000","last_active":"2024-02-17T12:48:52.165983+0000","last_peered":"2024-02-17T12:48:52.165983+0000","last_clean":"2024-02-17T12:48:52.165983+0000","last_became_active":"2024-02-17T12:47:45.061512+0000","last_became_peered":"2024-02-17T12:47:45.061512+0000","last_unstale":"2024-02-17T12:48:52.165983+0000","last_undegraded":"2024-02-17T12:48:52.165983+0000","last_fullsized":"2024-02-17T12:48:52.165983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:53:48.405184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463452+0000","last_change":"2024-02-17T12:47:46.071055+0000","last_active":"2024-02-17T12:48:52.463452+0000","last_peered":"2024-02-17T12:48:52.463452+0000","last_clean":"2024-02-17T12:48:52.463452+0000","last_became_active":"2024-02-17T12:47:46.070649+0000","last_became_peered":"2024-02-17T12:47:46.070649+0000","last_unstale":"2024-02-17T12:48:52.463452+0000","last_undegraded":"2024-02-17T12:48:52.463452+0000","last_fullsized":"2024-02-17T12:48:52.463452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:10:22.078746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.178268+0000","last_change":"2024-02-17T12:48:52.177755+0000","last_active":"2024-02-17T12:48:52.178268+0000","last_peered":"2024-02-17T12:48:52.178268+0000","last_clean":"2024-02-17T12:48:52.178268+0000","last_became_active":"2024-02-17T12:48:52.165119+0000","last_became_peered":"2024-02-17T12:48:52.165119+0000","last_unstale":"2024-02-17T12:48:52.178268+0000","last_undegraded":"2024-02-17T12:48:52.178268+0000","last_fullsized":"2024-02-17T12:48:52.178268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:38.834337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464243+0000","last_change":"2024-02-17T12:47:46.076448+0000","last_active":"2024-02-17T12:48:52.464243+0000","last_peered":"2024-02-17T12:48:52.464243+0000","last_clean":"2024-02-17T12:48:52.464243+0000","last_became_active":"2024-02-17T12:47:46.075250+0000","last_became_peered":"2024-02-17T12:47:46.075250+0000","last_unstale":"2024-02-17T12:48:52.464243+0000","last_undegraded":"2024-02-17T12:48:52.464243+0000","last_fullsized":"2024-02-17T12:48:52.464243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:11:45.846764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163335+0000","last_change":"2024-02-17T12:47:45.062699+0000","last_active":"2024-02-17T12:48:52.163335+0000","last_peered":"2024-02-17T12:48:52.163335+0000","last_clean":"2024-02-17T12:48:52.163335+0000","last_became_active":"2024-02-17T12:47:45.062322+0000","last_became_peered":"2024-02-17T12:47:45.062322+0000","last_unstale":"2024-02-17T12:48:52.163335+0000","last_undegraded":"2024-02-17T12:48:52.163335+0000","last_fullsized":"2024-02-17T12:48:52.163335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:14:06.112393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160749+0000","last_change":"2024-02-17T12:47:46.080278+0000","last_active":"2024-02-17T12:48:52.160749+0000","last_peered":"2024-02-17T12:48:52.160749+0000","last_clean":"2024-02-17T12:48:52.160749+0000","last_became_active":"2024-02-17T12:47:46.079924+0000","last_became_peered":"2024-02-17T12:47:46.079924+0000","last_unstale":"2024-02-17T12:48:52.160749+0000","last_undegraded":"2024-02-17T12:48:52.160749+0000","last_fullsized":"2024-02-17T12:48:52.160749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:54:08.759445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.470282+0000","last_change":"2024-02-17T12:48:52.469734+0000","last_active":"2024-02-17T12:48:52.470282+0000","last_peered":"2024-02-17T12:48:52.470282+0000","last_clean":"2024-02-17T12:48:52.470282+0000","last_became_active":"2024-02-17T12:48:52.465945+0000","last_became_peered":"2024-02-17T12:48:52.465945+0000","last_unstale":"2024-02-17T12:48:52.470282+0000","last_undegraded":"2024-02-17T12:48:52.470282+0000","last_fullsized":"2024-02-17T12:48:52.470282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:05:27.931619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163489+0000","last_change":"2024-02-17T12:47:46.065957+0000","last_active":"2024-02-17T12:48:52.163489+0000","last_peered":"2024-02-17T12:48:52.163489+0000","last_clean":"2024-02-17T12:48:52.163489+0000","last_became_active":"2024-02-17T12:47:46.065638+0000","last_became_peered":"2024-02-17T12:47:46.065638+0000","last_unstale":"2024-02-17T12:48:52.163489+0000","last_undegraded":"2024-02-17T12:48:52.163489+0000","last_fullsized":"2024-02-17T12:48:52.163489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:02:39.295990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463600+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:48:52.463600+0000","last_peered":"2024-02-17T12:48:52.463600+0000","last_clean":"2024-02-17T12:48:52.463600+0000","last_became_active":"2024-02-17T12:47:45.066235+0000","last_became_peered":"2024-02-17T12:47:45.066235+0000","last_unstale":"2024-02-17T12:48:52.463600+0000","last_undegraded":"2024-02-17T12:48:52.463600+0000","last_fullsized":"2024-02-17T12:48:52.463600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:46:53.731920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160252+0000","last_change":"2024-02-17T12:47:46.072029+0000","last_active":"2024-02-17T12:48:52.160252+0000","last_peered":"2024-02-17T12:48:52.160252+0000","last_clean":"2024-02-17T12:48:52.160252+0000","last_became_active":"2024-02-17T12:47:46.071716+0000","last_became_peered":"2024-02-17T12:47:46.071716+0000","last_unstale":"2024-02-17T12:48:52.160252+0000","last_undegraded":"2024-02-17T12:48:52.160252+0000","last_fullsized":"2024-02-17T12:48:52.160252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:33.076266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163802+0000","last_change":"2024-02-17T12:47:45.067443+0000","last_active":"2024-02-17T12:48:52.163802+0000","last_peered":"2024-02-17T12:48:52.163802+0000","last_clean":"2024-02-17T12:48:52.163802+0000","last_became_active":"2024-02-17T12:47:45.067090+0000","last_became_peered":"2024-02-17T12:47:45.067090+0000","last_unstale":"2024-02-17T12:48:52.163802+0000","last_undegraded":"2024-02-17T12:48:52.163802+0000","last_fullsized":"2024-02-17T12:48:52.163802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:17:39.874810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165386+0000","last_change":"2024-02-17T12:47:46.072275+0000","last_active":"2024-02-17T12:48:52.165386+0000","last_peered":"2024-02-17T12:48:52.165386+0000","last_clean":"2024-02-17T12:48:52.165386+0000","last_became_active":"2024-02-17T12:47:46.071951+0000","last_became_peered":"2024-02-17T12:47:46.071951+0000","last_unstale":"2024-02-17T12:48:52.165386+0000","last_undegraded":"2024-02-17T12:48:52.165386+0000","last_fullsized":"2024-02-17T12:48:52.165386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:45:58.580854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159772+0000","last_change":"2024-02-17T12:47:45.058875+0000","last_active":"2024-02-17T12:48:52.159772+0000","last_peered":"2024-02-17T12:48:52.159772+0000","last_clean":"2024-02-17T12:48:52.159772+0000","last_became_active":"2024-02-17T12:47:45.058433+0000","last_became_peered":"2024-02-17T12:47:45.058433+0000","last_unstale":"2024-02-17T12:48:52.159772+0000","last_undegraded":"2024-02-17T12:48:52.159772+0000","last_fullsized":"2024-02-17T12:48:52.159772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:49.932205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156324+0000","last_change":"2024-02-17T12:47:46.070233+0000","last_active":"2024-02-17T12:48:52.156324+0000","last_peered":"2024-02-17T12:48:52.156324+0000","last_clean":"2024-02-17T12:48:52.156324+0000","last_became_active":"2024-02-17T12:47:46.069811+0000","last_became_peered":"2024-02-17T12:47:46.069811+0000","last_unstale":"2024-02-17T12:48:52.156324+0000","last_undegraded":"2024-02-17T12:48:52.156324+0000","last_fullsized":"2024-02-17T12:48:52.156324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:50:00.885517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":152,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:53.812483+0000","last_change":"2024-02-17T12:47:45.068778+0000","last_active":"2024-02-17T12:48:53.812483+0000","last_peered":"2024-02-17T12:48:53.812483+0000","last_clean":"2024-02-17T12:48:53.812483+0000","last_became_active":"2024-02-17T12:47:45.068375+0000","last_became_peered":"2024-02-17T12:47:45.068375+0000","last_unstale":"2024-02-17T12:48:53.812483+0000","last_undegraded":"2024-02-17T12:48:53.812483+0000","last_fullsized":"2024-02-17T12:48:53.812483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:42:57.600138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.465164+0000","last_change":"2024-02-17T12:47:46.076802+0000","last_active":"2024-02-17T12:48:52.465164+0000","last_peered":"2024-02-17T12:48:52.465164+0000","last_clean":"2024-02-17T12:48:52.465164+0000","last_became_active":"2024-02-17T12:47:46.076433+0000","last_became_peered":"2024-02-17T12:47:46.076433+0000","last_unstale":"2024-02-17T12:48:52.465164+0000","last_undegraded":"2024-02-17T12:48:52.465164+0000","last_fullsized":"2024-02-17T12:48:52.465164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.352488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157571+0000","last_change":"2024-02-17T12:47:45.059100+0000","last_active":"2024-02-17T12:48:52.157571+0000","last_peered":"2024-02-17T12:48:52.157571+0000","last_clean":"2024-02-17T12:48:52.157571+0000","last_became_active":"2024-02-17T12:47:45.058707+0000","last_became_peered":"2024-02-17T12:47:45.058707+0000","last_unstale":"2024-02-17T12:48:52.157571+0000","last_undegraded":"2024-02-17T12:48:52.157571+0000","last_fullsized":"2024-02-17T12:48:52.157571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:28:27.044127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463554+0000","last_change":"2024-02-17T12:47:46.065088+0000","last_active":"2024-02-17T12:48:52.463554+0000","last_peered":"2024-02-17T12:48:52.463554+0000","last_clean":"2024-02-17T12:48:52.463554+0000","last_became_active":"2024-02-17T12:47:46.064754+0000","last_became_peered":"2024-02-17T12:47:46.064754+0000","last_unstale":"2024-02-17T12:48:52.463554+0000","last_undegraded":"2024-02-17T12:48:52.463554+0000","last_fullsized":"2024-02-17T12:48:52.463554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:33:10.652881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160868+0000","last_change":"2024-02-17T12:47:45.055403+0000","last_active":"2024-02-17T12:48:52.160868+0000","last_peered":"2024-02-17T12:48:52.160868+0000","last_clean":"2024-02-17T12:48:52.160868+0000","last_became_active":"2024-02-17T12:47:45.055083+0000","last_became_peered":"2024-02-17T12:47:45.055083+0000","last_unstale":"2024-02-17T12:48:52.160868+0000","last_undegraded":"2024-02-17T12:48:52.160868+0000","last_fullsized":"2024-02-17T12:48:52.160868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:05:51.295074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464126+0000","last_change":"2024-02-17T12:47:46.074883+0000","last_active":"2024-02-17T12:48:52.464126+0000","last_peered":"2024-02-17T12:48:52.464126+0000","last_clean":"2024-02-17T12:48:52.464126+0000","last_became_active":"2024-02-17T12:47:46.074414+0000","last_became_peered":"2024-02-17T12:47:46.074414+0000","last_unstale":"2024-02-17T12:48:52.464126+0000","last_undegraded":"2024-02-17T12:48:52.464126+0000","last_fullsized":"2024-02-17T12:48:52.464126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:35:55.026587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":187,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:54.469769+0000","last_change":"2024-02-17T12:47:45.057627+0000","last_active":"2024-02-17T12:48:54.469769+0000","last_peered":"2024-02-17T12:48:54.469769+0000","last_clean":"2024-02-17T12:48:54.469769+0000","last_became_active":"2024-02-17T12:47:45.057297+0000","last_became_peered":"2024-02-17T12:47:45.057297+0000","last_unstale":"2024-02-17T12:48:54.469769+0000","last_undegraded":"2024-02-17T12:48:54.469769+0000","last_fullsized":"2024-02-17T12:48:54.469769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:00:17.159939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59394,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":139,"num_read_kb":127,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163803+0000","last_change":"2024-02-17T12:47:46.076854+0000","last_active":"2024-02-17T12:48:52.163803+0000","last_peered":"2024-02-17T12:48:52.163803+0000","last_clean":"2024-02-17T12:48:52.163803+0000","last_became_active":"2024-02-17T12:47:46.076534+0000","last_became_peered":"2024-02-17T12:47:46.076534+0000","last_unstale":"2024-02-17T12:48:52.163803+0000","last_undegraded":"2024-02-17T12:48:52.163803+0000","last_fullsized":"2024-02-17T12:48:52.163803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:43:01.979075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163878+0000","last_change":"2024-02-17T12:47:45.060090+0000","last_active":"2024-02-17T12:48:52.163878+0000","last_peered":"2024-02-17T12:48:52.163878+0000","last_clean":"2024-02-17T12:48:52.163878+0000","last_became_active":"2024-02-17T12:47:45.058942+0000","last_became_peered":"2024-02-17T12:47:45.058942+0000","last_unstale":"2024-02-17T12:48:52.163878+0000","last_undegraded":"2024-02-17T12:48:52.163878+0000","last_fullsized":"2024-02-17T12:48:52.163878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:30:52.247140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.162861+0000","last_change":"2024-02-17T12:47:46.071982+0000","last_active":"2024-02-17T12:48:52.162861+0000","last_peered":"2024-02-17T12:48:52.162861+0000","last_clean":"2024-02-17T12:48:52.162861+0000","last_became_active":"2024-02-17T12:47:46.071639+0000","last_became_peered":"2024-02-17T12:47:46.071639+0000","last_unstale":"2024-02-17T12:48:52.162861+0000","last_undegraded":"2024-02-17T12:48:52.162861+0000","last_fullsized":"2024-02-17T12:48:52.162861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:43:50.658132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160287+0000","last_change":"2024-02-17T12:47:45.060017+0000","last_active":"2024-02-17T12:48:52.160287+0000","last_peered":"2024-02-17T12:48:52.160287+0000","last_clean":"2024-02-17T12:48:52.160287+0000","last_became_active":"2024-02-17T12:47:45.059104+0000","last_became_peered":"2024-02-17T12:47:45.059104+0000","last_unstale":"2024-02-17T12:48:52.160287+0000","last_undegraded":"2024-02-17T12:48:52.160287+0000","last_fullsized":"2024-02-17T12:48:52.160287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:56:50.457297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157576+0000","last_change":"2024-02-17T12:47:46.072250+0000","last_active":"2024-02-17T12:48:52.157576+0000","last_peered":"2024-02-17T12:48:52.157576+0000","last_clean":"2024-02-17T12:48:52.157576+0000","last_became_active":"2024-02-17T12:47:46.071856+0000","last_became_peered":"2024-02-17T12:47:46.071856+0000","last_unstale":"2024-02-17T12:48:52.157576+0000","last_undegraded":"2024-02-17T12:48:52.157576+0000","last_fullsized":"2024-02-17T12:48:52.157576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:46:04.628146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:53.837182+0000","last_change":"2024-02-17T12:47:45.056386+0000","last_active":"2024-02-17T12:48:53.837182+0000","last_peered":"2024-02-17T12:48:53.837182+0000","last_clean":"2024-02-17T12:48:53.837182+0000","last_became_active":"2024-02-17T12:47:45.055630+0000","last_became_peered":"2024-02-17T12:47:45.055630+0000","last_unstale":"2024-02-17T12:48:53.837182+0000","last_undegraded":"2024-02-17T12:48:53.837182+0000","last_fullsized":"2024-02-17T12:48:53.837182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:16.388950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157086+0000","last_change":"2024-02-17T12:47:46.071612+0000","last_active":"2024-02-17T12:48:52.157086+0000","last_peered":"2024-02-17T12:48:52.157086+0000","last_clean":"2024-02-17T12:48:52.157086+0000","last_became_active":"2024-02-17T12:47:46.071237+0000","last_became_peered":"2024-02-17T12:47:46.071237+0000","last_unstale":"2024-02-17T12:48:52.157086+0000","last_undegraded":"2024-02-17T12:48:52.157086+0000","last_fullsized":"2024-02-17T12:48:52.157086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:24:01.099679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163081+0000","last_change":"2024-02-17T12:47:45.066258+0000","last_active":"2024-02-17T12:48:52.163081+0000","last_peered":"2024-02-17T12:48:52.163081+0000","last_clean":"2024-02-17T12:48:52.163081+0000","last_became_active":"2024-02-17T12:47:45.065813+0000","last_became_peered":"2024-02-17T12:47:45.065813+0000","last_unstale":"2024-02-17T12:48:52.163081+0000","last_undegraded":"2024-02-17T12:48:52.163081+0000","last_fullsized":"2024-02-17T12:48:52.163081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:06:43.298573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160251+0000","last_change":"2024-02-17T12:47:46.080347+0000","last_active":"2024-02-17T12:48:52.160251+0000","last_peered":"2024-02-17T12:48:52.160251+0000","last_clean":"2024-02-17T12:48:52.160251+0000","last_became_active":"2024-02-17T12:47:46.080083+0000","last_became_peered":"2024-02-17T12:47:46.080083+0000","last_unstale":"2024-02-17T12:48:52.160251+0000","last_undegraded":"2024-02-17T12:48:52.160251+0000","last_fullsized":"2024-02-17T12:48:52.160251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:24:41.807633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163573+0000","last_change":"2024-02-17T12:47:45.062814+0000","last_active":"2024-02-17T12:48:52.163573+0000","last_peered":"2024-02-17T12:48:52.163573+0000","last_clean":"2024-02-17T12:48:52.163573+0000","last_became_active":"2024-02-17T12:47:45.062418+0000","last_became_peered":"2024-02-17T12:47:45.062418+0000","last_unstale":"2024-02-17T12:48:52.163573+0000","last_undegraded":"2024-02-17T12:48:52.163573+0000","last_fullsized":"2024-02-17T12:48:52.163573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:49:34.577275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159642+0000","last_change":"2024-02-17T12:47:46.067742+0000","last_active":"2024-02-17T12:48:52.159642+0000","last_peered":"2024-02-17T12:48:52.159642+0000","last_clean":"2024-02-17T12:48:52.159642+0000","last_became_active":"2024-02-17T12:47:46.067329+0000","last_became_peered":"2024-02-17T12:47:46.067329+0000","last_unstale":"2024-02-17T12:48:52.159642+0000","last_undegraded":"2024-02-17T12:48:52.159642+0000","last_fullsized":"2024-02-17T12:48:52.159642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:19.432999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156444+0000","last_change":"2024-02-17T12:47:45.065461+0000","last_active":"2024-02-17T12:48:52.156444+0000","last_peered":"2024-02-17T12:48:52.156444+0000","last_clean":"2024-02-17T12:48:52.156444+0000","last_became_active":"2024-02-17T12:47:45.065066+0000","last_became_peered":"2024-02-17T12:47:45.065066+0000","last_unstale":"2024-02-17T12:48:52.156444+0000","last_undegraded":"2024-02-17T12:48:52.156444+0000","last_fullsized":"2024-02-17T12:48:52.156444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:25:47.292901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463958+0000","last_change":"2024-02-17T12:47:46.074222+0000","last_active":"2024-02-17T12:48:52.463958+0000","last_peered":"2024-02-17T12:48:52.463958+0000","last_clean":"2024-02-17T12:48:52.463958+0000","last_became_active":"2024-02-17T12:47:46.073886+0000","last_became_peered":"2024-02-17T12:47:46.073886+0000","last_unstale":"2024-02-17T12:48:52.463958+0000","last_undegraded":"2024-02-17T12:48:52.463958+0000","last_fullsized":"2024-02-17T12:48:52.463958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:46:18.063422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:53.812144+0000","last_change":"2024-02-17T12:47:45.071414+0000","last_active":"2024-02-17T12:48:53.812144+0000","last_peered":"2024-02-17T12:48:53.812144+0000","last_clean":"2024-02-17T12:48:53.812144+0000","last_became_active":"2024-02-17T12:47:45.071127+0000","last_became_peered":"2024-02-17T12:47:45.071127+0000","last_unstale":"2024-02-17T12:48:53.812144+0000","last_undegraded":"2024-02-17T12:48:53.812144+0000","last_fullsized":"2024-02-17T12:48:53.812144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:13:21.187718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464019+0000","last_change":"2024-02-17T12:47:46.059412+0000","last_active":"2024-02-17T12:48:52.464019+0000","last_peered":"2024-02-17T12:48:52.464019+0000","last_clean":"2024-02-17T12:48:52.464019+0000","last_became_active":"2024-02-17T12:47:46.059100+0000","last_became_peered":"2024-02-17T12:47:46.059100+0000","last_unstale":"2024-02-17T12:48:52.464019+0000","last_undegraded":"2024-02-17T12:48:52.464019+0000","last_fullsized":"2024-02-17T12:48:52.464019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:35.930613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160000+0000","last_change":"2024-02-17T12:47:46.071467+0000","last_active":"2024-02-17T12:48:52.160000+0000","last_peered":"2024-02-17T12:48:52.160000+0000","last_clean":"2024-02-17T12:48:52.160000+0000","last_became_active":"2024-02-17T12:47:46.070879+0000","last_became_peered":"2024-02-17T12:47:46.070879+0000","last_unstale":"2024-02-17T12:48:52.160000+0000","last_undegraded":"2024-02-17T12:48:52.160000+0000","last_fullsized":"2024-02-17T12:48:52.160000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.682842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:53.836290+0000","last_change":"2024-02-17T12:47:45.070545+0000","last_active":"2024-02-17T12:48:53.836290+0000","last_peered":"2024-02-17T12:48:53.836290+0000","last_clean":"2024-02-17T12:48:53.836290+0000","last_became_active":"2024-02-17T12:47:45.070265+0000","last_became_peered":"2024-02-17T12:47:45.070265+0000","last_unstale":"2024-02-17T12:48:53.836290+0000","last_undegraded":"2024-02-17T12:48:53.836290+0000","last_fullsized":"2024-02-17T12:48:53.836290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.763735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165524+0000","last_change":"2024-02-17T12:47:46.068539+0000","last_active":"2024-02-17T12:48:52.165524+0000","last_peered":"2024-02-17T12:48:52.165524+0000","last_clean":"2024-02-17T12:48:52.165524+0000","last_became_active":"2024-02-17T12:47:46.068169+0000","last_became_peered":"2024-02-17T12:47:46.068169+0000","last_unstale":"2024-02-17T12:48:52.165524+0000","last_undegraded":"2024-02-17T12:48:52.165524+0000","last_fullsized":"2024-02-17T12:48:52.165524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:27:28.485868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464598+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:48:52.464598+0000","last_peered":"2024-02-17T12:48:52.464598+0000","last_clean":"2024-02-17T12:48:52.464598+0000","last_became_active":"2024-02-17T12:47:45.066299+0000","last_became_peered":"2024-02-17T12:47:45.066299+0000","last_unstale":"2024-02-17T12:48:52.464598+0000","last_undegraded":"2024-02-17T12:48:52.464598+0000","last_fullsized":"2024-02-17T12:48:52.464598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:34:02.040002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157204+0000","last_change":"2024-02-17T12:46:32.556576+0000","last_active":"2024-02-17T12:48:52.157204+0000","last_peered":"2024-02-17T12:48:52.157204+0000","last_clean":"2024-02-17T12:48:52.157204+0000","last_became_active":"2024-02-17T12:46:32.556160+0000","last_became_peered":"2024-02-17T12:46:32.556160+0000","last_unstale":"2024-02-17T12:48:52.157204+0000","last_undegraded":"2024-02-17T12:48:52.157204+0000","last_fullsized":"2024-02-17T12:48:52.157204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_clean_scrub_stamp":"2024-02-17T12:43:13.147248+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:57:52.244951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.166240+0000","last_change":"2024-02-17T12:48:52.166240+0000","last_active":"2024-02-17T12:48:52.166240+0000","last_peered":"2024-02-17T12:48:52.166240+0000","last_clean":"2024-02-17T12:48:52.166240+0000","last_became_active":"2024-02-17T12:48:52.165882+0000","last_became_peered":"2024-02-17T12:48:52.165882+0000","last_unstale":"2024-02-17T12:48:52.166240+0000","last_undegraded":"2024-02-17T12:48:52.166240+0000","last_fullsized":"2024-02-17T12:48:52.166240+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:56:44.360362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157273+0000","last_change":"2024-02-17T12:47:45.061112+0000","last_active":"2024-02-17T12:48:52.157273+0000","last_peered":"2024-02-17T12:48:52.157273+0000","last_clean":"2024-02-17T12:48:52.157273+0000","last_became_active":"2024-02-17T12:47:45.060633+0000","last_became_peered":"2024-02-17T12:47:45.060633+0000","last_unstale":"2024-02-17T12:48:52.157273+0000","last_undegraded":"2024-02-17T12:48:52.157273+0000","last_fullsized":"2024-02-17T12:48:52.157273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:29:29.421744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163918+0000","last_change":"2024-02-17T12:47:45.066897+0000","last_active":"2024-02-17T12:48:52.163918+0000","last_peered":"2024-02-17T12:48:52.163918+0000","last_clean":"2024-02-17T12:48:52.163918+0000","last_became_active":"2024-02-17T12:47:45.066357+0000","last_became_peered":"2024-02-17T12:47:45.066357+0000","last_unstale":"2024-02-17T12:48:52.163918+0000","last_undegraded":"2024-02-17T12:48:52.163918+0000","last_fullsized":"2024-02-17T12:48:52.163918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:50:09.840414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163515+0000","last_change":"2024-02-17T12:47:46.072412+0000","last_active":"2024-02-17T12:48:52.163515+0000","last_peered":"2024-02-17T12:48:52.163515+0000","last_clean":"2024-02-17T12:48:52.163515+0000","last_became_active":"2024-02-17T12:47:46.072105+0000","last_became_peered":"2024-02-17T12:47:46.072105+0000","last_unstale":"2024-02-17T12:48:52.163515+0000","last_undegraded":"2024-02-17T12:48:52.163515+0000","last_fullsized":"2024-02-17T12:48:52.163515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:33:19.367588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156606+0000","last_change":"2024-02-17T12:47:45.067673+0000","last_active":"2024-02-17T12:48:52.156606+0000","last_peered":"2024-02-17T12:48:52.156606+0000","last_clean":"2024-02-17T12:48:52.156606+0000","last_became_active":"2024-02-17T12:47:45.067212+0000","last_became_peered":"2024-02-17T12:47:45.067212+0000","last_unstale":"2024-02-17T12:48:52.156606+0000","last_undegraded":"2024-02-17T12:48:52.156606+0000","last_fullsized":"2024-02-17T12:48:52.156606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:15:04.236158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.166062+0000","last_change":"2024-02-17T12:47:46.077476+0000","last_active":"2024-02-17T12:48:52.166062+0000","last_peered":"2024-02-17T12:48:52.166062+0000","last_clean":"2024-02-17T12:48:52.166062+0000","last_became_active":"2024-02-17T12:47:46.077101+0000","last_became_peered":"2024-02-17T12:47:46.077101+0000","last_unstale":"2024-02-17T12:48:52.166062+0000","last_undegraded":"2024-02-17T12:48:52.166062+0000","last_fullsized":"2024-02-17T12:48:52.166062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:41:41.402101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464561+0000","last_change":"2024-02-17T12:47:45.066693+0000","last_active":"2024-02-17T12:48:52.464561+0000","last_peered":"2024-02-17T12:48:52.464561+0000","last_clean":"2024-02-17T12:48:52.464561+0000","last_became_active":"2024-02-17T12:47:45.066349+0000","last_became_peered":"2024-02-17T12:47:45.066349+0000","last_unstale":"2024-02-17T12:48:52.464561+0000","last_undegraded":"2024-02-17T12:48:52.464561+0000","last_fullsized":"2024-02-17T12:48:52.464561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:30.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463335+0000","last_change":"2024-02-17T12:47:46.072899+0000","last_active":"2024-02-17T12:48:52.463335+0000","last_peered":"2024-02-17T12:48:52.463335+0000","last_clean":"2024-02-17T12:48:52.463335+0000","last_became_active":"2024-02-17T12:47:46.072488+0000","last_became_peered":"2024-02-17T12:47:46.072488+0000","last_unstale":"2024-02-17T12:48:52.463335+0000","last_undegraded":"2024-02-17T12:48:52.463335+0000","last_fullsized":"2024-02-17T12:48:52.463335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:05:43.675086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463459+0000","last_change":"2024-02-17T12:47:45.056995+0000","last_active":"2024-02-17T12:48:52.463459+0000","last_peered":"2024-02-17T12:48:52.463459+0000","last_clean":"2024-02-17T12:48:52.463459+0000","last_became_active":"2024-02-17T12:47:45.056021+0000","last_became_peered":"2024-02-17T12:47:45.056021+0000","last_unstale":"2024-02-17T12:48:52.463459+0000","last_undegraded":"2024-02-17T12:48:52.463459+0000","last_fullsized":"2024-02-17T12:48:52.463459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:41.444513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163223+0000","last_change":"2024-02-17T12:47:46.065859+0000","last_active":"2024-02-17T12:48:52.163223+0000","last_peered":"2024-02-17T12:48:52.163223+0000","last_clean":"2024-02-17T12:48:52.163223+0000","last_became_active":"2024-02-17T12:47:46.065469+0000","last_became_peered":"2024-02-17T12:47:46.065469+0000","last_unstale":"2024-02-17T12:48:52.163223+0000","last_undegraded":"2024-02-17T12:48:52.163223+0000","last_fullsized":"2024-02-17T12:48:52.163223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:51:25.105361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165363+0000","last_change":"2024-02-17T12:47:45.066524+0000","last_active":"2024-02-17T12:48:52.165363+0000","last_peered":"2024-02-17T12:48:52.165363+0000","last_clean":"2024-02-17T12:48:52.165363+0000","last_became_active":"2024-02-17T12:47:45.066218+0000","last_became_peered":"2024-02-17T12:47:45.066218+0000","last_unstale":"2024-02-17T12:48:52.165363+0000","last_undegraded":"2024-02-17T12:48:52.165363+0000","last_fullsized":"2024-02-17T12:48:52.165363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:56:47.244262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159558+0000","last_change":"2024-02-17T12:47:46.065779+0000","last_active":"2024-02-17T12:48:52.159558+0000","last_peered":"2024-02-17T12:48:52.159558+0000","last_clean":"2024-02-17T12:48:52.159558+0000","last_became_active":"2024-02-17T12:47:46.065459+0000","last_became_peered":"2024-02-17T12:47:46.065459+0000","last_unstale":"2024-02-17T12:48:52.159558+0000","last_undegraded":"2024-02-17T12:48:52.159558+0000","last_fullsized":"2024-02-17T12:48:52.159558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:08.944487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163588+0000","last_change":"2024-02-17T12:47:45.059639+0000","last_active":"2024-02-17T12:48:52.163588+0000","last_peered":"2024-02-17T12:48:52.163588+0000","last_clean":"2024-02-17T12:48:52.163588+0000","last_became_active":"2024-02-17T12:47:45.059085+0000","last_became_peered":"2024-02-17T12:47:45.059085+0000","last_unstale":"2024-02-17T12:48:52.163588+0000","last_undegraded":"2024-02-17T12:48:52.163588+0000","last_fullsized":"2024-02-17T12:48:52.163588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:48:55.847405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165668+0000","last_change":"2024-02-17T12:47:46.079623+0000","last_active":"2024-02-17T12:48:52.165668+0000","last_peered":"2024-02-17T12:48:52.165668+0000","last_clean":"2024-02-17T12:48:52.165668+0000","last_became_active":"2024-02-17T12:47:46.079285+0000","last_became_peered":"2024-02-17T12:47:46.079285+0000","last_unstale":"2024-02-17T12:48:52.165668+0000","last_undegraded":"2024-02-17T12:48:52.165668+0000","last_fullsized":"2024-02-17T12:48:52.165668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:45:14.441218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.181532+0000","last_change":"2024-02-17T12:48:52.181057+0000","last_active":"2024-02-17T12:48:52.181532+0000","last_peered":"2024-02-17T12:48:52.181532+0000","last_clean":"2024-02-17T12:48:52.181532+0000","last_became_active":"2024-02-17T12:48:52.165968+0000","last_became_peered":"2024-02-17T12:48:52.165968+0000","last_unstale":"2024-02-17T12:48:52.181532+0000","last_undegraded":"2024-02-17T12:48:52.181532+0000","last_fullsized":"2024-02-17T12:48:52.181532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:14:06.153057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157361+0000","last_change":"2024-02-17T12:47:46.074429+0000","last_active":"2024-02-17T12:48:52.157361+0000","last_peered":"2024-02-17T12:48:52.157361+0000","last_clean":"2024-02-17T12:48:52.157361+0000","last_became_active":"2024-02-17T12:47:46.074123+0000","last_became_peered":"2024-02-17T12:47:46.074123+0000","last_unstale":"2024-02-17T12:48:52.157361+0000","last_undegraded":"2024-02-17T12:48:52.157361+0000","last_fullsized":"2024-02-17T12:48:52.157361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:50:17.633831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":87,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.164746+0000","last_change":"2024-02-17T12:47:45.061362+0000","last_active":"2024-02-17T12:48:52.164746+0000","last_peered":"2024-02-17T12:48:52.164746+0000","last_clean":"2024-02-17T12:48:52.164746+0000","last_became_active":"2024-02-17T12:47:45.061037+0000","last_became_peered":"2024-02-17T12:47:45.061037+0000","last_unstale":"2024-02-17T12:48:52.164746+0000","last_undegraded":"2024-02-17T12:48:52.164746+0000","last_fullsized":"2024-02-17T12:48:52.164746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:01.987157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":63,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157838+0000","last_change":"2024-02-17T12:47:46.073862+0000","last_active":"2024-02-17T12:48:52.157838+0000","last_peered":"2024-02-17T12:48:52.157838+0000","last_clean":"2024-02-17T12:48:52.157838+0000","last_became_active":"2024-02-17T12:47:46.073543+0000","last_became_peered":"2024-02-17T12:47:46.073543+0000","last_unstale":"2024-02-17T12:48:52.157838+0000","last_undegraded":"2024-02-17T12:48:52.157838+0000","last_fullsized":"2024-02-17T12:48:52.157838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:57:57.567726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160633+0000","last_change":"2024-02-17T12:47:45.060616+0000","last_active":"2024-02-17T12:48:52.160633+0000","last_peered":"2024-02-17T12:48:52.160633+0000","last_clean":"2024-02-17T12:48:52.160633+0000","last_became_active":"2024-02-17T12:47:45.060296+0000","last_became_peered":"2024-02-17T12:47:45.060296+0000","last_unstale":"2024-02-17T12:48:52.160633+0000","last_undegraded":"2024-02-17T12:48:52.160633+0000","last_fullsized":"2024-02-17T12:48:52.160633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:22:44.791363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157453+0000","last_change":"2024-02-17T12:47:46.072738+0000","last_active":"2024-02-17T12:48:52.157453+0000","last_peered":"2024-02-17T12:48:52.157453+0000","last_clean":"2024-02-17T12:48:52.157453+0000","last_became_active":"2024-02-17T12:47:46.072421+0000","last_became_peered":"2024-02-17T12:47:46.072421+0000","last_unstale":"2024-02-17T12:48:52.157453+0000","last_undegraded":"2024-02-17T12:48:52.157453+0000","last_fullsized":"2024-02-17T12:48:52.157453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:50:23.501787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.470091+0000","last_change":"2024-02-17T12:48:52.470091+0000","last_active":"2024-02-17T12:48:52.470091+0000","last_peered":"2024-02-17T12:48:52.470091+0000","last_clean":"2024-02-17T12:48:52.470091+0000","last_became_active":"2024-02-17T12:48:52.466153+0000","last_became_peered":"2024-02-17T12:48:52.466153+0000","last_unstale":"2024-02-17T12:48:52.470091+0000","last_undegraded":"2024-02-17T12:48:52.470091+0000","last_fullsized":"2024-02-17T12:48:52.470091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:03:45.027983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157617+0000","last_change":"2024-02-17T12:47:46.066072+0000","last_active":"2024-02-17T12:48:52.157617+0000","last_peered":"2024-02-17T12:48:52.157617+0000","last_clean":"2024-02-17T12:48:52.157617+0000","last_became_active":"2024-02-17T12:47:46.065755+0000","last_became_peered":"2024-02-17T12:47:46.065755+0000","last_unstale":"2024-02-17T12:48:52.157617+0000","last_undegraded":"2024-02-17T12:48:52.157617+0000","last_fullsized":"2024-02-17T12:48:52.157617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:09:29.534097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159652+0000","last_change":"2024-02-17T12:47:46.067268+0000","last_active":"2024-02-17T12:48:52.159652+0000","last_peered":"2024-02-17T12:48:52.159652+0000","last_clean":"2024-02-17T12:48:52.159652+0000","last_became_active":"2024-02-17T12:47:46.066918+0000","last_became_peered":"2024-02-17T12:47:46.066918+0000","last_unstale":"2024-02-17T12:48:52.159652+0000","last_undegraded":"2024-02-17T12:48:52.159652+0000","last_fullsized":"2024-02-17T12:48:52.159652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:11:09.844954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163365+0000","last_change":"2024-02-17T12:47:46.068143+0000","last_active":"2024-02-17T12:48:52.163365+0000","last_peered":"2024-02-17T12:48:52.163365+0000","last_clean":"2024-02-17T12:48:52.163365+0000","last_became_active":"2024-02-17T12:47:46.065112+0000","last_became_peered":"2024-02-17T12:47:46.065112+0000","last_unstale":"2024-02-17T12:48:52.163365+0000","last_undegraded":"2024-02-17T12:48:52.163365+0000","last_fullsized":"2024-02-17T12:48:52.163365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:43:04.982009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163481+0000","last_change":"2024-02-17T12:47:46.076139+0000","last_active":"2024-02-17T12:48:52.163481+0000","last_peered":"2024-02-17T12:48:52.163481+0000","last_clean":"2024-02-17T12:48:52.163481+0000","last_became_active":"2024-02-17T12:47:46.075826+0000","last_became_peered":"2024-02-17T12:47:46.075826+0000","last_unstale":"2024-02-17T12:48:52.163481+0000","last_undegraded":"2024-02-17T12:48:52.163481+0000","last_fullsized":"2024-02-17T12:48:52.163481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:39:08.460716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463447+0000","last_change":"2024-02-17T12:47:46.063792+0000","last_active":"2024-02-17T12:48:52.463447+0000","last_peered":"2024-02-17T12:48:52.463447+0000","last_clean":"2024-02-17T12:48:52.463447+0000","last_became_active":"2024-02-17T12:47:46.063407+0000","last_became_peered":"2024-02-17T12:47:46.063407+0000","last_unstale":"2024-02-17T12:48:52.463447+0000","last_undegraded":"2024-02-17T12:48:52.463447+0000","last_fullsized":"2024-02-17T12:48:52.463447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:26:26.973435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163601+0000","last_change":"2024-02-17T12:47:46.066202+0000","last_active":"2024-02-17T12:48:52.163601+0000","last_peered":"2024-02-17T12:48:52.163601+0000","last_clean":"2024-02-17T12:48:52.163601+0000","last_became_active":"2024-02-17T12:47:46.065820+0000","last_became_peered":"2024-02-17T12:47:46.065820+0000","last_unstale":"2024-02-17T12:48:52.163601+0000","last_undegraded":"2024-02-17T12:48:52.163601+0000","last_fullsized":"2024-02-17T12:48:52.163601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:36:25.201697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.166996+0000","last_change":"2024-02-17T12:48:52.166996+0000","last_active":"2024-02-17T12:48:52.166996+0000","last_peered":"2024-02-17T12:48:52.166996+0000","last_clean":"2024-02-17T12:48:52.166996+0000","last_became_active":"2024-02-17T12:48:52.166589+0000","last_became_peered":"2024-02-17T12:48:52.166589+0000","last_unstale":"2024-02-17T12:48:52.166996+0000","last_undegraded":"2024-02-17T12:48:52.166996+0000","last_fullsized":"2024-02-17T12:48:52.166996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:16:30.392183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.164351+0000","last_change":"2024-02-17T12:47:46.070811+0000","last_active":"2024-02-17T12:48:52.164351+0000","last_peered":"2024-02-17T12:48:52.164351+0000","last_clean":"2024-02-17T12:48:52.164351+0000","last_became_active":"2024-02-17T12:47:46.070462+0000","last_became_peered":"2024-02-17T12:47:46.070462+0000","last_unstale":"2024-02-17T12:48:52.164351+0000","last_undegraded":"2024-02-17T12:48:52.164351+0000","last_fullsized":"2024-02-17T12:48:52.164351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:03.671869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165953+0000","last_change":"2024-02-17T12:47:46.077692+0000","last_active":"2024-02-17T12:48:52.165953+0000","last_peered":"2024-02-17T12:48:52.165953+0000","last_clean":"2024-02-17T12:48:52.165953+0000","last_became_active":"2024-02-17T12:47:46.077363+0000","last_became_peered":"2024-02-17T12:47:46.077363+0000","last_unstale":"2024-02-17T12:48:52.165953+0000","last_undegraded":"2024-02-17T12:48:52.165953+0000","last_fullsized":"2024-02-17T12:48:52.165953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:02:49.068782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159896+0000","last_change":"2024-02-17T12:47:46.073507+0000","last_active":"2024-02-17T12:48:52.159896+0000","last_peered":"2024-02-17T12:48:52.159896+0000","last_clean":"2024-02-17T12:48:52.159896+0000","last_became_active":"2024-02-17T12:47:46.073107+0000","last_became_peered":"2024-02-17T12:47:46.073107+0000","last_unstale":"2024-02-17T12:48:52.159896+0000","last_undegraded":"2024-02-17T12:48:52.159896+0000","last_fullsized":"2024-02-17T12:48:52.159896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:32:48.799300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165279+0000","last_change":"2024-02-17T12:47:46.073384+0000","last_active":"2024-02-17T12:48:52.165279+0000","last_peered":"2024-02-17T12:48:52.165279+0000","last_clean":"2024-02-17T12:48:52.165279+0000","last_became_active":"2024-02-17T12:47:46.073017+0000","last_became_peered":"2024-02-17T12:47:46.073017+0000","last_unstale":"2024-02-17T12:48:52.165279+0000","last_undegraded":"2024-02-17T12:48:52.165279+0000","last_fullsized":"2024-02-17T12:48:52.165279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:43:50.091667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.465319+0000","last_change":"2024-02-17T12:47:46.073758+0000","last_active":"2024-02-17T12:48:52.465319+0000","last_peered":"2024-02-17T12:48:52.465319+0000","last_clean":"2024-02-17T12:48:52.465319+0000","last_became_active":"2024-02-17T12:47:46.072781+0000","last_became_peered":"2024-02-17T12:47:46.072781+0000","last_unstale":"2024-02-17T12:48:52.465319+0000","last_undegraded":"2024-02-17T12:48:52.465319+0000","last_fullsized":"2024-02-17T12:48:52.465319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:10:24.811100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163113+0000","last_change":"2024-02-17T12:47:46.072025+0000","last_active":"2024-02-17T12:48:52.163113+0000","last_peered":"2024-02-17T12:48:52.163113+0000","last_clean":"2024-02-17T12:48:52.163113+0000","last_became_active":"2024-02-17T12:47:46.071662+0000","last_became_peered":"2024-02-17T12:47:46.071662+0000","last_unstale":"2024-02-17T12:48:52.163113+0000","last_undegraded":"2024-02-17T12:48:52.163113+0000","last_fullsized":"2024-02-17T12:48:52.163113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:04:07.188552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.166041+0000","last_change":"2024-02-17T12:48:52.166041+0000","last_active":"2024-02-17T12:48:52.166041+0000","last_peered":"2024-02-17T12:48:52.166041+0000","last_clean":"2024-02-17T12:48:52.166041+0000","last_became_active":"2024-02-17T12:48:52.165576+0000","last_became_peered":"2024-02-17T12:48:52.165576+0000","last_unstale":"2024-02-17T12:48:52.166041+0000","last_undegraded":"2024-02-17T12:48:52.166041+0000","last_fullsized":"2024-02-17T12:48:52.166041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:26.810385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66598,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":675,"num_read_kb":411,"num_write":236,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1013,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":200807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351003,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":462062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.47499999999999998}]},{"osd":1,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.81299999999999994}]},{"osd":2,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65700000000000003}]},{"osd":5,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.436},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.52500000000000002}]},{"osd":7,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.70999999999999996}]},{"osd":8,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.58799999999999997}]},{"osd":9,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.69399999999999995}]},{"osd":10,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.78200000000000003}]}]},{"osd":10,"up_from":73,"seq":313532612639,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1112876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":13.85,"5min":13.85,"15min":13.85},"last":0.39200000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.495,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70699999999999996,"5min":0.751,"15min":0.751},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.51500000000000001,"15min":0.52700000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.66400000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.49199999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83999999999999997,"5min":1.002,"15min":1.002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.68999999999999995,"15min":0.70399999999999996},"min":{"1min":0.47199999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.73899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.63600000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.59399999999999997,"15min":0.60899999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79400000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.438},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83899999999999997,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58799999999999997}]},{"osd":4,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64100000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.60199999999999998,"15min":0.62},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81799999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.73099999999999998,"15min":0.745},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.88300000000000001,"5min":1.022,"15min":1.022},"last":0.747},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.68700000000000006,"15min":0.70099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77100000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.625}]},{"osd":6,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.82499999999999996,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.752,"15min":0.76600000000000001},"min":{"1min":0.502,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.87,"5min":1.085,"15min":1.085},"last":0.72499999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.54900000000000004}]},{"osd":8,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.76300000000000001,"15min":0.78100000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.78500000000000003,"5min":1.107,"15min":1.107},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89500000000000002,"5min":1.119,"15min":1.119},"last":0.78700000000000003}]},{"osd":9,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.83499999999999996,"15min":0.85799999999999998},"min":{"1min":0.57799999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91800000000000004,"5min":1.151,"15min":1.151},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.81200000000000006,"15min":0.83199999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.85499999999999998,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.70499999999999996}]},{"osd":11,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77400000000000002}]}]},{"osd":9,"up_from":68,"seq":292057776166,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40468,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704684,"statfs":{"total":95995035648,"available":95953596416,"internally_reserved":0,"allocated":987136,"data_stored":461992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.45500000000000002,"15min":0.46100000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73199999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.36199999999999999}]},{"osd":1,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.64900000000000002,"15min":0.65900000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98699999999999999,"5min":1.04,"15min":1.04},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.55500000000000005}]},{"osd":2,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.40100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0249999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64400000000000002,"15min":0.65100000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.502}]},{"osd":3,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60999999999999999,"15min":0.621},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76300000000000001,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66700000000000004,"15min":0.67300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83399999999999996,"5min":1.159,"15min":1.159},"last":0.57099999999999995}]},{"osd":5,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.44800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96699999999999997,"5min":1.323,"15min":1.323},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.76500000000000001,"15min":0.77600000000000002},"min":{"1min":0.47899999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.101,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.64500000000000002}]},{"osd":6,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1379999999999999,"5min":1.268,"15min":1.268},"last":0.624},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92300000000000004,"5min":1.27,"15min":1.27},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80200000000000005,"15min":0.81599999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97299999999999998,"5min":1.351,"15min":1.351},"last":0.66600000000000004}]},{"osd":8,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.503,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.79000000000000004,"15min":0.80200000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95599999999999996,"5min":1.363,"15min":1.363},"last":0.63600000000000001}]},{"osd":10,"last update":"Sat Feb 17 12:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0429999999999999,"5min":1.228,"15min":1.228},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.81899999999999995},"min":{"1min":0.61399999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2849999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.68799999999999994}]}]},{"osd":8,"up_from":60,"seq":257698037802,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":462370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.35899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.91800000000000004,"15min":1.054},"min":{"1min":0.441,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89000000000000001,"5min":10.955,"15min":10.955},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.434,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.71099999999999997}]},{"osd":2,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.44900000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.755},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58199999999999996}]},{"osd":3,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.36499999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.51600000000000001}]},{"osd":4,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.93300000000000005,"15min":1.0680000000000001},"min":{"1min":0.46100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":10.972,"15min":10.972},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.92800000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64500000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.372,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.34899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.69199999999999995}]},{"osd":6,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.879},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.878},"min":{"1min":0.53200000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97499999999999998,"5min":4.9269999999999996,"15min":4.9269999999999996},"last":0.86299999999999999}]},{"osd":7,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.49399999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":1.022,"15min":1.022},"last":0.77700000000000002}]},{"osd":9,"last update":"Sat Feb 17 12:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.69999999999999996,"15min":0.68799999999999994},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":1.038},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":1.006}]},{"osd":10,"last update":"Sat Feb 17 12:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95299999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.2330000000000001,"15min":1.3009999999999999},"min":{"1min":0.55200000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.98399999999999999,"5min":10.987,"15min":10.987},"last":0.98699999999999999}]},{"osd":11,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":1.024},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.97199999999999998}]}]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1581056,"data_stored":1055279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42499999999999999,"15min":0.41199999999999998},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.434},"min":{"1min":0.315,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Feb 17 12:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77500000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.748,"15min":0.63},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":15.003,"15min":15.003},"last":0.64400000000000002}]},{"osd":3,"last update":"Sat Feb 17 12:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61199999999999999,"15min":0.622},"min":{"1min":0.40100000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.746,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.62,"15min":0.629},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57199999999999995}]},{"osd":4,"last update":"Sat Feb 17 12:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.73899999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.75600000000000001,"5min":0.872,"15min":0.872},"last":0.53100000000000003}]},{"osd":5,"last update":"Sat Feb 17 12:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.80200000000000005,"15min":0.68600000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":15.028,"15min":15.028},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63,"15min":0.61299999999999999},"min":{"1min":0.42199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90600000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59599999999999997}]},{"osd":6,"last update":"Sat Feb 17 12:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.82399999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.8799999999999999,"5min":0.93999999999999995,"15min":0.77400000000000002},"min":{"1min":0.45100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":14.579000000000001,"5min":14.579000000000001,"15min":14.579000000000001},"last":0.626}]},{"osd":8,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.60999999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96199999999999997,"5min":1.02,"15min":1.02},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75700000000000001,"15min":0.73799999999999999},"min":{"1min":0.65500000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.72699999999999998}]},{"osd":9,"last update":"Sat Feb 17 12:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.754,"15min":0.76800000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88600000000000001,"5min":1.04,"15min":1.04},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.403,"5min":0.89500000000000002,"15min":0.81100000000000005},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":14.598000000000001,"5min":14.598000000000001,"15min":14.598000000000001},"last":0.67200000000000004}]},{"osd":10,"last update":"Sat Feb 17 12:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.751,"15min":0.75600000000000001},"min":{"1min":0.45300000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0309999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94399999999999995,"5min":1.103,"15min":1.103},"last":0.745}]},{"osd":11,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.77900000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495668,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":461709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.372},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.31},{"interface":"front","average":{"1min":0.379,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.313,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54500000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.34699999999999998}]},{"osd":1,"last update":"Sat Feb 17 12:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.495,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76700000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.46700000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84099999999999997,"5min":1.173,"15min":1.173},"last":0.46200000000000002}]},{"osd":2,"last update":"Sat Feb 17 12:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.63100000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.32700000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63700000000000001,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42199999999999999}]},{"osd":4,"last update":"Sat Feb 17 12:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80200000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.45200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.84199999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.435}]},{"osd":5,"last update":"Sat Feb 17 12:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.62,"15min":0.60199999999999998},"min":{"1min":0.379,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.873,"5min":1.284,"15min":1.284},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.753,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.72399999999999998,"15min":0.67100000000000004},"min":{"1min":0.57599999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":5.069,"5min":5.069,"15min":5.069},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65300000000000002,"15min":0.64000000000000001},"min":{"1min":0.46700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47699999999999998}]},{"osd":8,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.73399999999999999,"15min":0.70799999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":5.0940000000000003,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90800000000000003,"5min":1.252,"15min":1.252},"last":0.66200000000000003}]},{"osd":9,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85699999999999998,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.48199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58099999999999996}]},{"osd":10,"last update":"Sat Feb 17 12:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.749,"15min":0.76300000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.88500000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":522016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48199999999999998,"15min":0.47999999999999998},"min":{"1min":0.35399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.62,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":1.393,"5min":0.67000000000000004,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":14.91,"5min":14.91,"15min":14.91},"last":0.68000000000000005}]},{"osd":2,"last update":"Sat Feb 17 12:48:20 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55400000000000005,"15min":0.51200000000000001},"min":{"1min":0.44900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56200000000000006,"15min":0.51600000000000001},"min":{"1min":0.45300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79400000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.70599999999999996}]},{"osd":3,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.48599999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78500000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.621},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.432,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73599999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54500000000000004}]},{"osd":4,"last update":"Sat Feb 17 12:48:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.41499999999999998,"15min":0.38900000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.71199999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.432},{"interface":"front","average":{"1min":0.502,"5min":0.436,"15min":0.41799999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.73599999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56699999999999995}]},{"osd":5,"last update":"Sat Feb 17 12:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.61499999999999999,"15min":0.57899999999999996},"min":{"1min":0.46300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90500000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.626,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64900000000000002}]},{"osd":6,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88800000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84199999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.60699999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:48:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57299999999999995,"15min":0.54900000000000004},"min":{"1min":0.32300000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.626,"15min":0.59699999999999998},"min":{"1min":0.374,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0329999999999999,"5min":1.079,"15min":1.079},"last":0.76600000000000001}]},{"osd":8,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.67400000000000004,"15min":0.63400000000000001},"min":{"1min":0.63800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.58999999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.74199999999999999}]},{"osd":9,"last update":"Sat Feb 17 12:48:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.89900000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.505,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46000000000000002}]},{"osd":10,"last update":"Sat Feb 17 12:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73299999999999998,"15min":0.71899999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.877,"5min":0.75700000000000001,"15min":0.73699999999999999},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.79800000000000004}]},{"osd":11,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.754}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":463048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 12:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53400000000000003,"15min":0.48899999999999999},"min":{"1min":0.34499999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81299999999999994,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54500000000000004,"15min":0.495},"min":{"1min":0.41699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64400000000000002}]},{"osd":2,"last update":"Sat Feb 17 12:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.68899999999999995,"15min":0.749},"min":{"1min":0.42899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":8.3919999999999995,"5min":8.3919999999999995,"15min":8.3919999999999995},"last":0.76400000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.51100000000000001}]},{"osd":4,"last update":"Sat Feb 17 12:48:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.629},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59399999999999997,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85199999999999998,"5min":0.872,"15min":0.872},"last":0.69599999999999995}]},{"osd":5,"last update":"Sat Feb 17 12:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.42099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68999999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.751},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.48099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.748,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.67700000000000005}]},{"osd":6,"last update":"Sat Feb 17 12:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.28999999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.626,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Sat Feb 17 12:47:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.443},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64600000000000002,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.65300000000000002,"15min":0.66500000000000004},"min":{"1min":0.45000000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79500000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.79900000000000004}]},{"osd":9,"last update":"Sat Feb 17 12:48:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.64100000000000001,"15min":0.66200000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77300000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.59799999999999998}]},{"osd":10,"last update":"Sat Feb 17 12:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77700000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.71999999999999997,"15min":0.73599999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.72199999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72699999999999998}]},{"osd":11,"last update":"Sat Feb 17 12:48:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.82699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":514221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.46300000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70499999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.29099999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67700000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40400000000000003}]},{"osd":1,"last update":"Sat Feb 17 12:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53400000000000003,"15min":0.504},"min":{"1min":0.312,"5min":0.309,"15min":0.309},"max":{"1min":0.746,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53700000000000003,"15min":0.505},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.68999999999999995}]},{"osd":3,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56000000000000005,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.77200000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.499}]},{"osd":4,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.62}]},{"osd":5,"last update":"Sat Feb 17 12:48:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.46000000000000002,"15min":0.44},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79000000000000004,"5min":1.325,"15min":1.325},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.47899999999999998,"15min":0.48399999999999999},"min":{"1min":0.32800000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84699999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53200000000000003}]},{"osd":6,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91100000000000003,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.93700000000000006,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58799999999999997}]},{"osd":7,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.60999999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.85699999999999998,"5min":0.998,"15min":0.998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60899999999999999,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95899999999999996,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.76100000000000001}]},{"osd":9,"last update":"Sat Feb 17 12:48:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92300000000000004,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71599999999999997,"15min":0.72199999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98399999999999999,"5min":1.284,"15min":1.284},"last":0.89700000000000002}]},{"osd":10,"last update":"Sat Feb 17 12:48:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96799999999999997,"5min":1.347,"15min":1.347},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.006,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.92500000000000004}]},{"osd":11,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.79800000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":461534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.32300000000000001},"min":{"1min":0.26000000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59799999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.33700000000000002},"min":{"1min":0.26300000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.379}]},{"osd":1,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77100000000000002,"5min":0.879,"15min":0.879},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.44800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.74199999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.752,"5min":0.88,"15min":0.88},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77900000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.58999999999999997}]},{"osd":4,"last update":"Sat Feb 17 12:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.70699999999999996,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.38200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.86799999999999999,"5min":0.876,"15min":0.876},"last":0.57599999999999996}]},{"osd":5,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.505,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86699999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84799999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64300000000000002}]},{"osd":6,"last update":"Sat Feb 17 12:48:53 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.504,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73799999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.441},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.495,"15min":0.505},"min":{"1min":0.35099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.40300000000000002}]},{"osd":7,"last update":"Sat Feb 17 12:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":0.876,"15min":0.876},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.505,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.751}]},{"osd":8,"last update":"Sat Feb 17 12:47:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91600000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.71299999999999997,"15min":0.72599999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0069999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.56899999999999995}]},{"osd":9,"last update":"Sat Feb 17 12:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.66900000000000004,"15min":0.68999999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93999999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69399999999999995,"15min":0.71299999999999997},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82599999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.50800000000000001}]},{"osd":10,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.83199999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.76400000000000001,"15min":0.77400000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.77900000000000003}]},{"osd":11,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.73299999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":462974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57899999999999996}]},{"osd":1,"last update":"Sat Feb 17 12:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.38900000000000001,"15min":0.35899999999999999},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.372},"min":{"1min":0.28799999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.71299999999999997,"5min":0.751,"15min":0.751},"last":0.39900000000000002}]},{"osd":2,"last update":"Sat Feb 17 12:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.77800000000000002,"15min":0.63200000000000001},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86299999999999999,"5min":18.73,"15min":18.73},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.45500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73799999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Feb 17 12:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86499999999999999,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":0.871,"15min":0.871},"last":0.42199999999999999}]},{"osd":5,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.49199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69499999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60999999999999999,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.81799999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.747}]},{"osd":6,"last update":"Sat Feb 17 12:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58699999999999997,"15min":0.57199999999999995},"min":{"1min":0.436,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.74299999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57199999999999995,"15min":0.54700000000000004},"min":{"1min":0.45600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84699999999999998,"5min":0.877,"15min":0.877},"last":0.56299999999999994}]},{"osd":7,"last update":"Sat Feb 17 12:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53600000000000003,"15min":0.52100000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.65200000000000002}]},{"osd":8,"last update":"Sat Feb 17 12:48:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.83699999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76300000000000001}]},{"osd":9,"last update":"Sat Feb 17 12:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.60999999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.89900000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68300000000000005}]},{"osd":10,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85899999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.72199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.86899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73199999999999998}]},{"osd":11,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.78200000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1054566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.435,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82499999999999996,"5min":1.02,"15min":1.02},"last":0.47599999999999998}]},{"osd":1,"last update":"Sat Feb 17 12:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.55600000000000005,"15min":0.52400000000000002},"min":{"1min":0.53200000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.57599999999999996,"15min":0.53000000000000003},"min":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.71999999999999997}]},{"osd":2,"last update":"Sat Feb 17 12:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.41099999999999998,"15min":0.38500000000000001},"min":{"1min":0.34000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.626,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.42099999999999999,"15min":0.377},"min":{"1min":0.32600000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.50600000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.57399999999999995}]},{"osd":4,"last update":"Sat Feb 17 12:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.59099999999999997,"15min":0.55000000000000004},"min":{"1min":0.48799999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.58199999999999996,"15min":0.53000000000000003},"min":{"1min":0.52200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82899999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.54000000000000004}]},{"osd":7,"last update":"Sat Feb 17 12:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68600000000000005,"15min":0.67700000000000005},"min":{"1min":0.53800000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92700000000000005,"5min":1,"15min":1},"last":0.628},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.495,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68700000000000006}]},{"osd":8,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.629,"15min":0.625},"min":{"1min":0.51200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.46600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70499999999999996}]},{"osd":9,"last update":"Sat Feb 17 12:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66900000000000004,"15min":0.67000000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88800000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.747},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70199999999999996,"15min":0.70499999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77500000000000002}]},{"osd":10,"last update":"Sat Feb 17 12:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.755,"15min":0.74099999999999999},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.97499999999999998,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.70399999999999996,"15min":0.68400000000000005},"min":{"1min":0.59099999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T12:48:58.150 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-17T12:48:58.431 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:48:58.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:58 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:58.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:58 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:58.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:58 smithi018 ceph-mon[32625]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:48:58.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:58 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:48:58.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:58 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:48:58.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:58 smithi018 ceph-mon[32625]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:58 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:58 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:58 smithi146 ceph-mon[36454]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:48:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:58 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:48:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:58 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:48:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:58 smithi146 ceph-mon[36454]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:58.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:58 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:58.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:58 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:58.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:58 smithi119 ceph-mon[36188]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:48:58.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:58 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:48:58.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:58 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:48:58.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:58 smithi119 ceph-mon[36188]: from='mgr.14152 172.21.15.18:0/653921761' entity='mgr.x' 2024-02-17T12:48:59.464 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:48:59 smithi018 ceph-mon[32625]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-02-17T12:48:59.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:48:59 smithi146 ceph-mon[36454]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-02-17T12:48:59.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:48:59 smithi119 ceph-mon[36188]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-02-17T12:48:59.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.984+0000 7ff1f2c72700 1 -- 172.21.15.18:0/735977865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1ec107d90 msgr2=0x7ff1ec108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:59.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.984+0000 7ff1f2c72700 1 --2- 172.21.15.18:0/735977865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1ec107d90 0x7ff1ec108170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009990 tx=0x7ff1e0009ca0 comp rx=0 tx=0).stop 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.984+0000 7ff1f2c72700 1 -- 172.21.15.18:0/735977865 shutdown_connections 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.984+0000 7ff1f2c72700 1 --2- 172.21.15.18:0/735977865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1ec1026e0 0x7ff1ec10ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.984+0000 7ff1f2c72700 1 --2- 172.21.15.18:0/735977865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1ec107d90 0x7ff1ec108170 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.984+0000 7ff1f2c72700 1 --2- 172.21.15.18:0/735977865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1ec101d20 0x7ff1ec1021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.984+0000 7ff1f2c72700 1 -- 172.21.15.18:0/735977865 >> 172.21.15.18:0/735977865 conn(0x7ff1ec078d40 msgr2=0x7ff1ec079150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.984+0000 7ff1f2c72700 1 -- 172.21.15.18:0/735977865 shutdown_connections 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.984+0000 7ff1f2c72700 1 -- 172.21.15.18:0/735977865 wait complete. 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1f2c72700 1 Processor -- start 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1f2c72700 1 -- start start 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1f2c72700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1ec101d20 0x7ff1ec19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1f2c72700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1ec1026e0 0x7ff1ec19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1f2c72700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1ec107d90 0x7ff1ec19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:59.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1f2c72700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec19f200 con 0x7ff1ec1026e0 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1f2c72700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec19f370 con 0x7ff1ec107d90 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1f2c72700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec19f530 con 0x7ff1ec101d20 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1ebfff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1ec1026e0 0x7ff1ec19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1ebfff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1ec1026e0 0x7ff1ec19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53116/0 (socket says 172.21.15.18:53116) 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1ebfff700 1 -- 172.21.15.18:0/597227520 learned_addr learned my addr 172.21.15.18:0/597227520 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1f0a0e700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1ec101d20 0x7ff1ec19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1f120f700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1ec107d90 0x7ff1ec19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1ebfff700 1 -- 172.21.15.18:0/597227520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1ec101d20 msgr2=0x7ff1ec19a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1ebfff700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1ec101d20 0x7ff1ec19a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1ebfff700 1 -- 172.21.15.18:0/597227520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1ec107d90 msgr2=0x7ff1ec19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1ebfff700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1ec107d90 0x7ff1ec19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1ebfff700 1 -- 172.21.15.18:0/597227520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00095f0 con 0x7ff1ec1026e0 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1f120f700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1ec107d90 0x7ff1ec19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:48:59.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.985+0000 7ff1ebfff700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1ec1026e0 0x7ff1ec19a860 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800c9b0 tx=0x7ff1d800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:59.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.986+0000 7ff1e9ffb700 1 -- 172.21.15.18:0/597227520 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8007b00 con 0x7ff1ec1026e0 2024-02-17T12:48:59.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.986+0000 7ff1f2c72700 1 -- 172.21.15.18:0/597227520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec19f810 con 0x7ff1ec1026e0 2024-02-17T12:48:59.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.986+0000 7ff1e9ffb700 1 -- 172.21.15.18:0/597227520 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1d8007c60 con 0x7ff1ec1026e0 2024-02-17T12:48:59.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.986+0000 7ff1e9ffb700 1 -- 172.21.15.18:0/597227520 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8018730 con 0x7ff1ec1026e0 2024-02-17T12:48:59.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.986+0000 7ff1f2c72700 1 -- 172.21.15.18:0/597227520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec1a7070 con 0x7ff1ec1026e0 2024-02-17T12:48:59.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.987+0000 7ff1f2c72700 1 -- 172.21.15.18:0/597227520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec068930 con 0x7ff1ec1026e0 2024-02-17T12:48:59.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.988+0000 7ff1e9ffb700 1 -- 172.21.15.18:0/597227520 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ff1d8018890 con 0x7ff1ec1026e0 2024-02-17T12:48:59.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.988+0000 7ff1e9ffb700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff1dc0a21e0 0x7ff1dc0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:48:59.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.988+0000 7ff1f0a0e700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff1dc0a21e0 0x7ff1dc0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:48:59.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.988+0000 7ff1e9ffb700 1 -- 172.21.15.18:0/597227520 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff1d80cdf00 con 0x7ff1ec1026e0 2024-02-17T12:48:59.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.989+0000 7ff1f0a0e700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff1dc0a21e0 0x7ff1dc0a46a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0006010 tx=0x7ff1e000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:48:59.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:48:59.993+0000 7ff1e9ffb700 1 -- 172.21.15.18:0/597227520 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff1d8096da0 con 0x7ff1ec1026e0 2024-02-17T12:49:00.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.191+0000 7ff1f2c72700 1 -- 172.21.15.18:0/597227520 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1ec111df0 con 0x7ff1dc0a21e0 2024-02-17T12:49:00.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.201+0000 7ff1e9ffb700 1 -- 172.21.15.18:0/597227520 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344280 (secure 0 0 0) 0x7ff1ec111df0 con 0x7ff1dc0a21e0 2024-02-17T12:49:00.198 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.204+0000 7ff1f2c72700 1 -- 172.21.15.18:0/597227520 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff1dc0a21e0 msgr2=0x7ff1dc0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.204+0000 7ff1f2c72700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff1dc0a21e0 0x7ff1dc0a46a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0006010 tx=0x7ff1e000b540 comp rx=0 tx=0).stop 2024-02-17T12:49:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.204+0000 7ff1f2c72700 1 -- 172.21.15.18:0/597227520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1ec1026e0 msgr2=0x7ff1ec19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.204+0000 7ff1f2c72700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1ec1026e0 0x7ff1ec19a860 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800c9b0 tx=0x7ff1d800cd70 comp rx=0 tx=0).stop 2024-02-17T12:49:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.205+0000 7ff1f2c72700 1 -- 172.21.15.18:0/597227520 shutdown_connections 2024-02-17T12:49:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.205+0000 7ff1f2c72700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1ec107d90 0x7ff1ec19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.205+0000 7ff1f2c72700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff1dc0a21e0 0x7ff1dc0a46a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.205+0000 7ff1f2c72700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1ec101d20 0x7ff1ec19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.205+0000 7ff1f2c72700 1 --2- 172.21.15.18:0/597227520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1ec1026e0 0x7ff1ec19a860 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.205+0000 7ff1f2c72700 1 -- 172.21.15.18:0/597227520 >> 172.21.15.18:0/597227520 conn(0x7ff1ec078d40 msgr2=0x7ff1ec0740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:00.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.205+0000 7ff1f2c72700 1 -- 172.21.15.18:0/597227520 shutdown_connections 2024-02-17T12:49:00.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:00.205+0000 7ff1f2c72700 1 -- 172.21.15.18:0/597227520 wait complete. 2024-02-17T12:49:00.202 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-02-17T12:49:00.924 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-02-17T12:48:58.208833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657200,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":816,"num_read_kb":530,"num_write":419,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181749,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":294,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379316,"kb_used_data":13452,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562508,"statfs":{"total":1151940427776,"available":1151552008192,"internally_reserved":0,"allocated":13774848,"data_stored":7502040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":31,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1013,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001957"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159822+0000","last_change":"2024-02-17T12:47:46.069941+0000","last_active":"2024-02-17T12:48:52.159822+0000","last_peered":"2024-02-17T12:48:52.159822+0000","last_clean":"2024-02-17T12:48:52.159822+0000","last_became_active":"2024-02-17T12:47:46.069192+0000","last_became_peered":"2024-02-17T12:47:46.069192+0000","last_unstale":"2024-02-17T12:48:52.159822+0000","last_undegraded":"2024-02-17T12:48:52.159822+0000","last_fullsized":"2024-02-17T12:48:52.159822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:27.102839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463276+0000","last_change":"2024-02-17T12:47:46.064872+0000","last_active":"2024-02-17T12:48:52.463276+0000","last_peered":"2024-02-17T12:48:52.463276+0000","last_clean":"2024-02-17T12:48:52.463276+0000","last_became_active":"2024-02-17T12:47:46.064498+0000","last_became_peered":"2024-02-17T12:47:46.064498+0000","last_unstale":"2024-02-17T12:48:52.463276+0000","last_undegraded":"2024-02-17T12:48:52.463276+0000","last_fullsized":"2024-02-17T12:48:52.463276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:30:41.432657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463466+0000","last_change":"2024-02-17T12:47:46.067504+0000","last_active":"2024-02-17T12:48:52.463466+0000","last_peered":"2024-02-17T12:48:52.463466+0000","last_clean":"2024-02-17T12:48:52.463466+0000","last_became_active":"2024-02-17T12:47:46.067155+0000","last_became_peered":"2024-02-17T12:47:46.067155+0000","last_unstale":"2024-02-17T12:48:52.463466+0000","last_undegraded":"2024-02-17T12:48:52.463466+0000","last_fullsized":"2024-02-17T12:48:52.463466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:38:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464939+0000","last_change":"2024-02-17T12:47:46.062356+0000","last_active":"2024-02-17T12:48:52.464939+0000","last_peered":"2024-02-17T12:48:52.464939+0000","last_clean":"2024-02-17T12:48:52.464939+0000","last_became_active":"2024-02-17T12:47:46.061988+0000","last_became_peered":"2024-02-17T12:47:46.061988+0000","last_unstale":"2024-02-17T12:48:52.464939+0000","last_undegraded":"2024-02-17T12:48:52.464939+0000","last_fullsized":"2024-02-17T12:48:52.464939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:25:09.120770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.162896+0000","last_change":"2024-02-17T12:47:46.072411+0000","last_active":"2024-02-17T12:48:52.162896+0000","last_peered":"2024-02-17T12:48:52.162896+0000","last_clean":"2024-02-17T12:48:52.162896+0000","last_became_active":"2024-02-17T12:47:46.072067+0000","last_became_peered":"2024-02-17T12:47:46.072067+0000","last_unstale":"2024-02-17T12:48:52.162896+0000","last_undegraded":"2024-02-17T12:48:52.162896+0000","last_fullsized":"2024-02-17T12:48:52.162896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:29:33.051513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464236+0000","last_change":"2024-02-17T12:47:46.057425+0000","last_active":"2024-02-17T12:48:52.464236+0000","last_peered":"2024-02-17T12:48:52.464236+0000","last_clean":"2024-02-17T12:48:52.464236+0000","last_became_active":"2024-02-17T12:47:46.057001+0000","last_became_peered":"2024-02-17T12:47:46.057001+0000","last_unstale":"2024-02-17T12:48:52.464236+0000","last_undegraded":"2024-02-17T12:48:52.464236+0000","last_fullsized":"2024-02-17T12:48:52.464236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:48:24.135879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157182+0000","last_change":"2024-02-17T12:47:46.069143+0000","last_active":"2024-02-17T12:48:52.157182+0000","last_peered":"2024-02-17T12:48:52.157182+0000","last_clean":"2024-02-17T12:48:52.157182+0000","last_became_active":"2024-02-17T12:47:46.068775+0000","last_became_peered":"2024-02-17T12:47:46.068775+0000","last_unstale":"2024-02-17T12:48:52.157182+0000","last_undegraded":"2024-02-17T12:48:52.157182+0000","last_fullsized":"2024-02-17T12:48:52.157182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:32:39.518205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165272+0000","last_change":"2024-02-17T12:47:46.069579+0000","last_active":"2024-02-17T12:48:52.165272+0000","last_peered":"2024-02-17T12:48:52.165272+0000","last_clean":"2024-02-17T12:48:52.165272+0000","last_became_active":"2024-02-17T12:47:46.069114+0000","last_became_peered":"2024-02-17T12:47:46.069114+0000","last_unstale":"2024-02-17T12:48:52.165272+0000","last_undegraded":"2024-02-17T12:48:52.165272+0000","last_fullsized":"2024-02-17T12:48:52.165272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:54:58.100511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.465498+0000","last_change":"2024-02-17T12:47:46.079103+0000","last_active":"2024-02-17T12:48:52.465498+0000","last_peered":"2024-02-17T12:48:52.465498+0000","last_clean":"2024-02-17T12:48:52.465498+0000","last_became_active":"2024-02-17T12:47:46.077992+0000","last_became_peered":"2024-02-17T12:47:46.077992+0000","last_unstale":"2024-02-17T12:48:52.465498+0000","last_undegraded":"2024-02-17T12:48:52.465498+0000","last_fullsized":"2024-02-17T12:48:52.465498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:12:59.337341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464970+0000","last_change":"2024-02-17T12:47:46.062897+0000","last_active":"2024-02-17T12:48:52.464970+0000","last_peered":"2024-02-17T12:48:52.464970+0000","last_clean":"2024-02-17T12:48:52.464970+0000","last_became_active":"2024-02-17T12:47:46.062571+0000","last_became_peered":"2024-02-17T12:47:46.062571+0000","last_unstale":"2024-02-17T12:48:52.464970+0000","last_undegraded":"2024-02-17T12:48:52.464970+0000","last_fullsized":"2024-02-17T12:48:52.464970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:57:02.845400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157655+0000","last_change":"2024-02-17T12:47:46.080183+0000","last_active":"2024-02-17T12:48:52.157655+0000","last_peered":"2024-02-17T12:48:52.157655+0000","last_clean":"2024-02-17T12:48:52.157655+0000","last_became_active":"2024-02-17T12:47:46.079827+0000","last_became_peered":"2024-02-17T12:47:46.079827+0000","last_unstale":"2024-02-17T12:48:52.157655+0000","last_undegraded":"2024-02-17T12:48:52.157655+0000","last_fullsized":"2024-02-17T12:48:52.157655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:56:10.193200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464991+0000","last_change":"2024-02-17T12:47:46.061849+0000","last_active":"2024-02-17T12:48:52.464991+0000","last_peered":"2024-02-17T12:48:52.464991+0000","last_clean":"2024-02-17T12:48:52.464991+0000","last_became_active":"2024-02-17T12:47:46.061458+0000","last_became_peered":"2024-02-17T12:47:46.061458+0000","last_unstale":"2024-02-17T12:48:52.464991+0000","last_undegraded":"2024-02-17T12:48:52.464991+0000","last_fullsized":"2024-02-17T12:48:52.464991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:02:09.141292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163184+0000","last_change":"2024-02-17T12:47:46.069356+0000","last_active":"2024-02-17T12:48:52.163184+0000","last_peered":"2024-02-17T12:48:52.163184+0000","last_clean":"2024-02-17T12:48:52.163184+0000","last_became_active":"2024-02-17T12:47:46.068567+0000","last_became_peered":"2024-02-17T12:47:46.068567+0000","last_unstale":"2024-02-17T12:48:52.163184+0000","last_undegraded":"2024-02-17T12:48:52.163184+0000","last_fullsized":"2024-02-17T12:48:52.163184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:08:01.872558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463404+0000","last_change":"2024-02-17T12:47:46.062684+0000","last_active":"2024-02-17T12:48:52.463404+0000","last_peered":"2024-02-17T12:48:52.463404+0000","last_clean":"2024-02-17T12:48:52.463404+0000","last_became_active":"2024-02-17T12:47:46.062221+0000","last_became_peered":"2024-02-17T12:47:46.062221+0000","last_unstale":"2024-02-17T12:48:52.463404+0000","last_undegraded":"2024-02-17T12:48:52.463404+0000","last_fullsized":"2024-02-17T12:48:52.463404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:37:11.513336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.462993+0000","last_change":"2024-02-17T12:47:46.065961+0000","last_active":"2024-02-17T12:48:52.462993+0000","last_peered":"2024-02-17T12:48:52.462993+0000","last_clean":"2024-02-17T12:48:52.462993+0000","last_became_active":"2024-02-17T12:47:46.065600+0000","last_became_peered":"2024-02-17T12:47:46.065600+0000","last_unstale":"2024-02-17T12:48:52.462993+0000","last_undegraded":"2024-02-17T12:48:52.462993+0000","last_fullsized":"2024-02-17T12:48:52.462993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:24:46.205077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463715+0000","last_change":"2024-02-17T12:47:46.080150+0000","last_active":"2024-02-17T12:48:52.463715+0000","last_peered":"2024-02-17T12:48:52.463715+0000","last_clean":"2024-02-17T12:48:52.463715+0000","last_became_active":"2024-02-17T12:47:46.079813+0000","last_became_peered":"2024-02-17T12:47:46.079813+0000","last_unstale":"2024-02-17T12:48:52.463715+0000","last_undegraded":"2024-02-17T12:48:52.463715+0000","last_fullsized":"2024-02-17T12:48:52.463715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:59:00.868964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159950+0000","last_change":"2024-02-17T12:47:46.070202+0000","last_active":"2024-02-17T12:48:52.159950+0000","last_peered":"2024-02-17T12:48:52.159950+0000","last_clean":"2024-02-17T12:48:52.159950+0000","last_became_active":"2024-02-17T12:47:46.069628+0000","last_became_peered":"2024-02-17T12:47:46.069628+0000","last_unstale":"2024-02-17T12:48:52.159950+0000","last_undegraded":"2024-02-17T12:48:52.159950+0000","last_fullsized":"2024-02-17T12:48:52.159950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:44:17.481681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156569+0000","last_change":"2024-02-17T12:47:46.067810+0000","last_active":"2024-02-17T12:48:52.156569+0000","last_peered":"2024-02-17T12:48:52.156569+0000","last_clean":"2024-02-17T12:48:52.156569+0000","last_became_active":"2024-02-17T12:47:46.067456+0000","last_became_peered":"2024-02-17T12:47:46.067456+0000","last_unstale":"2024-02-17T12:48:52.156569+0000","last_undegraded":"2024-02-17T12:48:52.156569+0000","last_fullsized":"2024-02-17T12:48:52.156569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:27:03.646597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.164082+0000","last_change":"2024-02-17T12:47:46.076268+0000","last_active":"2024-02-17T12:48:52.164082+0000","last_peered":"2024-02-17T12:48:52.164082+0000","last_clean":"2024-02-17T12:48:52.164082+0000","last_became_active":"2024-02-17T12:47:46.075606+0000","last_became_peered":"2024-02-17T12:47:46.075606+0000","last_unstale":"2024-02-17T12:48:52.164082+0000","last_undegraded":"2024-02-17T12:48:52.164082+0000","last_fullsized":"2024-02-17T12:48:52.164082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:16:35.805008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:53.811536+0000","last_change":"2024-02-17T12:47:45.066780+0000","last_active":"2024-02-17T12:48:53.811536+0000","last_peered":"2024-02-17T12:48:53.811536+0000","last_clean":"2024-02-17T12:48:53.811536+0000","last_became_active":"2024-02-17T12:47:45.066039+0000","last_became_peered":"2024-02-17T12:47:45.066039+0000","last_unstale":"2024-02-17T12:48:53.811536+0000","last_undegraded":"2024-02-17T12:48:53.811536+0000","last_fullsized":"2024-02-17T12:48:53.811536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:36:25.420534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165559+0000","last_change":"2024-02-17T12:47:46.079603+0000","last_active":"2024-02-17T12:48:52.165559+0000","last_peered":"2024-02-17T12:48:52.165559+0000","last_clean":"2024-02-17T12:48:52.165559+0000","last_became_active":"2024-02-17T12:47:46.079246+0000","last_became_peered":"2024-02-17T12:47:46.079246+0000","last_unstale":"2024-02-17T12:48:52.165559+0000","last_undegraded":"2024-02-17T12:48:52.165559+0000","last_fullsized":"2024-02-17T12:48:52.165559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:32:04.975778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165983+0000","last_change":"2024-02-17T12:47:45.061868+0000","last_active":"2024-02-17T12:48:52.165983+0000","last_peered":"2024-02-17T12:48:52.165983+0000","last_clean":"2024-02-17T12:48:52.165983+0000","last_became_active":"2024-02-17T12:47:45.061512+0000","last_became_peered":"2024-02-17T12:47:45.061512+0000","last_unstale":"2024-02-17T12:48:52.165983+0000","last_undegraded":"2024-02-17T12:48:52.165983+0000","last_fullsized":"2024-02-17T12:48:52.165983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:53:48.405184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463452+0000","last_change":"2024-02-17T12:47:46.071055+0000","last_active":"2024-02-17T12:48:52.463452+0000","last_peered":"2024-02-17T12:48:52.463452+0000","last_clean":"2024-02-17T12:48:52.463452+0000","last_became_active":"2024-02-17T12:47:46.070649+0000","last_became_peered":"2024-02-17T12:47:46.070649+0000","last_unstale":"2024-02-17T12:48:52.463452+0000","last_undegraded":"2024-02-17T12:48:52.463452+0000","last_fullsized":"2024-02-17T12:48:52.463452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:10:22.078746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.178268+0000","last_change":"2024-02-17T12:48:52.177755+0000","last_active":"2024-02-17T12:48:52.178268+0000","last_peered":"2024-02-17T12:48:52.178268+0000","last_clean":"2024-02-17T12:48:52.178268+0000","last_became_active":"2024-02-17T12:48:52.165119+0000","last_became_peered":"2024-02-17T12:48:52.165119+0000","last_unstale":"2024-02-17T12:48:52.178268+0000","last_undegraded":"2024-02-17T12:48:52.178268+0000","last_fullsized":"2024-02-17T12:48:52.178268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:38.834337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464243+0000","last_change":"2024-02-17T12:47:46.076448+0000","last_active":"2024-02-17T12:48:52.464243+0000","last_peered":"2024-02-17T12:48:52.464243+0000","last_clean":"2024-02-17T12:48:52.464243+0000","last_became_active":"2024-02-17T12:47:46.075250+0000","last_became_peered":"2024-02-17T12:47:46.075250+0000","last_unstale":"2024-02-17T12:48:52.464243+0000","last_undegraded":"2024-02-17T12:48:52.464243+0000","last_fullsized":"2024-02-17T12:48:52.464243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:11:45.846764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163335+0000","last_change":"2024-02-17T12:47:45.062699+0000","last_active":"2024-02-17T12:48:52.163335+0000","last_peered":"2024-02-17T12:48:52.163335+0000","last_clean":"2024-02-17T12:48:52.163335+0000","last_became_active":"2024-02-17T12:47:45.062322+0000","last_became_peered":"2024-02-17T12:47:45.062322+0000","last_unstale":"2024-02-17T12:48:52.163335+0000","last_undegraded":"2024-02-17T12:48:52.163335+0000","last_fullsized":"2024-02-17T12:48:52.163335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:14:06.112393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160749+0000","last_change":"2024-02-17T12:47:46.080278+0000","last_active":"2024-02-17T12:48:52.160749+0000","last_peered":"2024-02-17T12:48:52.160749+0000","last_clean":"2024-02-17T12:48:52.160749+0000","last_became_active":"2024-02-17T12:47:46.079924+0000","last_became_peered":"2024-02-17T12:47:46.079924+0000","last_unstale":"2024-02-17T12:48:52.160749+0000","last_undegraded":"2024-02-17T12:48:52.160749+0000","last_fullsized":"2024-02-17T12:48:52.160749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:54:08.759445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.470282+0000","last_change":"2024-02-17T12:48:52.469734+0000","last_active":"2024-02-17T12:48:52.470282+0000","last_peered":"2024-02-17T12:48:52.470282+0000","last_clean":"2024-02-17T12:48:52.470282+0000","last_became_active":"2024-02-17T12:48:52.465945+0000","last_became_peered":"2024-02-17T12:48:52.465945+0000","last_unstale":"2024-02-17T12:48:52.470282+0000","last_undegraded":"2024-02-17T12:48:52.470282+0000","last_fullsized":"2024-02-17T12:48:52.470282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:05:27.931619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163489+0000","last_change":"2024-02-17T12:47:46.065957+0000","last_active":"2024-02-17T12:48:52.163489+0000","last_peered":"2024-02-17T12:48:52.163489+0000","last_clean":"2024-02-17T12:48:52.163489+0000","last_became_active":"2024-02-17T12:47:46.065638+0000","last_became_peered":"2024-02-17T12:47:46.065638+0000","last_unstale":"2024-02-17T12:48:52.163489+0000","last_undegraded":"2024-02-17T12:48:52.163489+0000","last_fullsized":"2024-02-17T12:48:52.163489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:02:39.295990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463600+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:48:52.463600+0000","last_peered":"2024-02-17T12:48:52.463600+0000","last_clean":"2024-02-17T12:48:52.463600+0000","last_became_active":"2024-02-17T12:47:45.066235+0000","last_became_peered":"2024-02-17T12:47:45.066235+0000","last_unstale":"2024-02-17T12:48:52.463600+0000","last_undegraded":"2024-02-17T12:48:52.463600+0000","last_fullsized":"2024-02-17T12:48:52.463600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:46:53.731920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160252+0000","last_change":"2024-02-17T12:47:46.072029+0000","last_active":"2024-02-17T12:48:52.160252+0000","last_peered":"2024-02-17T12:48:52.160252+0000","last_clean":"2024-02-17T12:48:52.160252+0000","last_became_active":"2024-02-17T12:47:46.071716+0000","last_became_peered":"2024-02-17T12:47:46.071716+0000","last_unstale":"2024-02-17T12:48:52.160252+0000","last_undegraded":"2024-02-17T12:48:52.160252+0000","last_fullsized":"2024-02-17T12:48:52.160252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:33.076266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163802+0000","last_change":"2024-02-17T12:47:45.067443+0000","last_active":"2024-02-17T12:48:52.163802+0000","last_peered":"2024-02-17T12:48:52.163802+0000","last_clean":"2024-02-17T12:48:52.163802+0000","last_became_active":"2024-02-17T12:47:45.067090+0000","last_became_peered":"2024-02-17T12:47:45.067090+0000","last_unstale":"2024-02-17T12:48:52.163802+0000","last_undegraded":"2024-02-17T12:48:52.163802+0000","last_fullsized":"2024-02-17T12:48:52.163802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:17:39.874810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165386+0000","last_change":"2024-02-17T12:47:46.072275+0000","last_active":"2024-02-17T12:48:52.165386+0000","last_peered":"2024-02-17T12:48:52.165386+0000","last_clean":"2024-02-17T12:48:52.165386+0000","last_became_active":"2024-02-17T12:47:46.071951+0000","last_became_peered":"2024-02-17T12:47:46.071951+0000","last_unstale":"2024-02-17T12:48:52.165386+0000","last_undegraded":"2024-02-17T12:48:52.165386+0000","last_fullsized":"2024-02-17T12:48:52.165386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:45:58.580854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159772+0000","last_change":"2024-02-17T12:47:45.058875+0000","last_active":"2024-02-17T12:48:52.159772+0000","last_peered":"2024-02-17T12:48:52.159772+0000","last_clean":"2024-02-17T12:48:52.159772+0000","last_became_active":"2024-02-17T12:47:45.058433+0000","last_became_peered":"2024-02-17T12:47:45.058433+0000","last_unstale":"2024-02-17T12:48:52.159772+0000","last_undegraded":"2024-02-17T12:48:52.159772+0000","last_fullsized":"2024-02-17T12:48:52.159772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:49.932205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156324+0000","last_change":"2024-02-17T12:47:46.070233+0000","last_active":"2024-02-17T12:48:52.156324+0000","last_peered":"2024-02-17T12:48:52.156324+0000","last_clean":"2024-02-17T12:48:52.156324+0000","last_became_active":"2024-02-17T12:47:46.069811+0000","last_became_peered":"2024-02-17T12:47:46.069811+0000","last_unstale":"2024-02-17T12:48:52.156324+0000","last_undegraded":"2024-02-17T12:48:52.156324+0000","last_fullsized":"2024-02-17T12:48:52.156324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:50:00.885517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":152,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:53.812483+0000","last_change":"2024-02-17T12:47:45.068778+0000","last_active":"2024-02-17T12:48:53.812483+0000","last_peered":"2024-02-17T12:48:53.812483+0000","last_clean":"2024-02-17T12:48:53.812483+0000","last_became_active":"2024-02-17T12:47:45.068375+0000","last_became_peered":"2024-02-17T12:47:45.068375+0000","last_unstale":"2024-02-17T12:48:53.812483+0000","last_undegraded":"2024-02-17T12:48:53.812483+0000","last_fullsized":"2024-02-17T12:48:53.812483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:42:57.600138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.465164+0000","last_change":"2024-02-17T12:47:46.076802+0000","last_active":"2024-02-17T12:48:52.465164+0000","last_peered":"2024-02-17T12:48:52.465164+0000","last_clean":"2024-02-17T12:48:52.465164+0000","last_became_active":"2024-02-17T12:47:46.076433+0000","last_became_peered":"2024-02-17T12:47:46.076433+0000","last_unstale":"2024-02-17T12:48:52.465164+0000","last_undegraded":"2024-02-17T12:48:52.465164+0000","last_fullsized":"2024-02-17T12:48:52.465164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.352488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157571+0000","last_change":"2024-02-17T12:47:45.059100+0000","last_active":"2024-02-17T12:48:52.157571+0000","last_peered":"2024-02-17T12:48:52.157571+0000","last_clean":"2024-02-17T12:48:52.157571+0000","last_became_active":"2024-02-17T12:47:45.058707+0000","last_became_peered":"2024-02-17T12:47:45.058707+0000","last_unstale":"2024-02-17T12:48:52.157571+0000","last_undegraded":"2024-02-17T12:48:52.157571+0000","last_fullsized":"2024-02-17T12:48:52.157571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:28:27.044127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463554+0000","last_change":"2024-02-17T12:47:46.065088+0000","last_active":"2024-02-17T12:48:52.463554+0000","last_peered":"2024-02-17T12:48:52.463554+0000","last_clean":"2024-02-17T12:48:52.463554+0000","last_became_active":"2024-02-17T12:47:46.064754+0000","last_became_peered":"2024-02-17T12:47:46.064754+0000","last_unstale":"2024-02-17T12:48:52.463554+0000","last_undegraded":"2024-02-17T12:48:52.463554+0000","last_fullsized":"2024-02-17T12:48:52.463554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:33:10.652881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160868+0000","last_change":"2024-02-17T12:47:45.055403+0000","last_active":"2024-02-17T12:48:52.160868+0000","last_peered":"2024-02-17T12:48:52.160868+0000","last_clean":"2024-02-17T12:48:52.160868+0000","last_became_active":"2024-02-17T12:47:45.055083+0000","last_became_peered":"2024-02-17T12:47:45.055083+0000","last_unstale":"2024-02-17T12:48:52.160868+0000","last_undegraded":"2024-02-17T12:48:52.160868+0000","last_fullsized":"2024-02-17T12:48:52.160868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:05:51.295074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464126+0000","last_change":"2024-02-17T12:47:46.074883+0000","last_active":"2024-02-17T12:48:52.464126+0000","last_peered":"2024-02-17T12:48:52.464126+0000","last_clean":"2024-02-17T12:48:52.464126+0000","last_became_active":"2024-02-17T12:47:46.074414+0000","last_became_peered":"2024-02-17T12:47:46.074414+0000","last_unstale":"2024-02-17T12:48:52.464126+0000","last_undegraded":"2024-02-17T12:48:52.464126+0000","last_fullsized":"2024-02-17T12:48:52.464126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:35:55.026587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":187,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:54.469769+0000","last_change":"2024-02-17T12:47:45.057627+0000","last_active":"2024-02-17T12:48:54.469769+0000","last_peered":"2024-02-17T12:48:54.469769+0000","last_clean":"2024-02-17T12:48:54.469769+0000","last_became_active":"2024-02-17T12:47:45.057297+0000","last_became_peered":"2024-02-17T12:47:45.057297+0000","last_unstale":"2024-02-17T12:48:54.469769+0000","last_undegraded":"2024-02-17T12:48:54.469769+0000","last_fullsized":"2024-02-17T12:48:54.469769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:00:17.159939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59394,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":139,"num_read_kb":127,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163803+0000","last_change":"2024-02-17T12:47:46.076854+0000","last_active":"2024-02-17T12:48:52.163803+0000","last_peered":"2024-02-17T12:48:52.163803+0000","last_clean":"2024-02-17T12:48:52.163803+0000","last_became_active":"2024-02-17T12:47:46.076534+0000","last_became_peered":"2024-02-17T12:47:46.076534+0000","last_unstale":"2024-02-17T12:48:52.163803+0000","last_undegraded":"2024-02-17T12:48:52.163803+0000","last_fullsized":"2024-02-17T12:48:52.163803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:43:01.979075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163878+0000","last_change":"2024-02-17T12:47:45.060090+0000","last_active":"2024-02-17T12:48:52.163878+0000","last_peered":"2024-02-17T12:48:52.163878+0000","last_clean":"2024-02-17T12:48:52.163878+0000","last_became_active":"2024-02-17T12:47:45.058942+0000","last_became_peered":"2024-02-17T12:47:45.058942+0000","last_unstale":"2024-02-17T12:48:52.163878+0000","last_undegraded":"2024-02-17T12:48:52.163878+0000","last_fullsized":"2024-02-17T12:48:52.163878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:30:52.247140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.162861+0000","last_change":"2024-02-17T12:47:46.071982+0000","last_active":"2024-02-17T12:48:52.162861+0000","last_peered":"2024-02-17T12:48:52.162861+0000","last_clean":"2024-02-17T12:48:52.162861+0000","last_became_active":"2024-02-17T12:47:46.071639+0000","last_became_peered":"2024-02-17T12:47:46.071639+0000","last_unstale":"2024-02-17T12:48:52.162861+0000","last_undegraded":"2024-02-17T12:48:52.162861+0000","last_fullsized":"2024-02-17T12:48:52.162861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:43:50.658132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160287+0000","last_change":"2024-02-17T12:47:45.060017+0000","last_active":"2024-02-17T12:48:52.160287+0000","last_peered":"2024-02-17T12:48:52.160287+0000","last_clean":"2024-02-17T12:48:52.160287+0000","last_became_active":"2024-02-17T12:47:45.059104+0000","last_became_peered":"2024-02-17T12:47:45.059104+0000","last_unstale":"2024-02-17T12:48:52.160287+0000","last_undegraded":"2024-02-17T12:48:52.160287+0000","last_fullsized":"2024-02-17T12:48:52.160287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:56:50.457297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157576+0000","last_change":"2024-02-17T12:47:46.072250+0000","last_active":"2024-02-17T12:48:52.157576+0000","last_peered":"2024-02-17T12:48:52.157576+0000","last_clean":"2024-02-17T12:48:52.157576+0000","last_became_active":"2024-02-17T12:47:46.071856+0000","last_became_peered":"2024-02-17T12:47:46.071856+0000","last_unstale":"2024-02-17T12:48:52.157576+0000","last_undegraded":"2024-02-17T12:48:52.157576+0000","last_fullsized":"2024-02-17T12:48:52.157576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:46:04.628146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:53.837182+0000","last_change":"2024-02-17T12:47:45.056386+0000","last_active":"2024-02-17T12:48:53.837182+0000","last_peered":"2024-02-17T12:48:53.837182+0000","last_clean":"2024-02-17T12:48:53.837182+0000","last_became_active":"2024-02-17T12:47:45.055630+0000","last_became_peered":"2024-02-17T12:47:45.055630+0000","last_unstale":"2024-02-17T12:48:53.837182+0000","last_undegraded":"2024-02-17T12:48:53.837182+0000","last_fullsized":"2024-02-17T12:48:53.837182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:16.388950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157086+0000","last_change":"2024-02-17T12:47:46.071612+0000","last_active":"2024-02-17T12:48:52.157086+0000","last_peered":"2024-02-17T12:48:52.157086+0000","last_clean":"2024-02-17T12:48:52.157086+0000","last_became_active":"2024-02-17T12:47:46.071237+0000","last_became_peered":"2024-02-17T12:47:46.071237+0000","last_unstale":"2024-02-17T12:48:52.157086+0000","last_undegraded":"2024-02-17T12:48:52.157086+0000","last_fullsized":"2024-02-17T12:48:52.157086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:24:01.099679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163081+0000","last_change":"2024-02-17T12:47:45.066258+0000","last_active":"2024-02-17T12:48:52.163081+0000","last_peered":"2024-02-17T12:48:52.163081+0000","last_clean":"2024-02-17T12:48:52.163081+0000","last_became_active":"2024-02-17T12:47:45.065813+0000","last_became_peered":"2024-02-17T12:47:45.065813+0000","last_unstale":"2024-02-17T12:48:52.163081+0000","last_undegraded":"2024-02-17T12:48:52.163081+0000","last_fullsized":"2024-02-17T12:48:52.163081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:06:43.298573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160251+0000","last_change":"2024-02-17T12:47:46.080347+0000","last_active":"2024-02-17T12:48:52.160251+0000","last_peered":"2024-02-17T12:48:52.160251+0000","last_clean":"2024-02-17T12:48:52.160251+0000","last_became_active":"2024-02-17T12:47:46.080083+0000","last_became_peered":"2024-02-17T12:47:46.080083+0000","last_unstale":"2024-02-17T12:48:52.160251+0000","last_undegraded":"2024-02-17T12:48:52.160251+0000","last_fullsized":"2024-02-17T12:48:52.160251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:24:41.807633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163573+0000","last_change":"2024-02-17T12:47:45.062814+0000","last_active":"2024-02-17T12:48:52.163573+0000","last_peered":"2024-02-17T12:48:52.163573+0000","last_clean":"2024-02-17T12:48:52.163573+0000","last_became_active":"2024-02-17T12:47:45.062418+0000","last_became_peered":"2024-02-17T12:47:45.062418+0000","last_unstale":"2024-02-17T12:48:52.163573+0000","last_undegraded":"2024-02-17T12:48:52.163573+0000","last_fullsized":"2024-02-17T12:48:52.163573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:49:34.577275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159642+0000","last_change":"2024-02-17T12:47:46.067742+0000","last_active":"2024-02-17T12:48:52.159642+0000","last_peered":"2024-02-17T12:48:52.159642+0000","last_clean":"2024-02-17T12:48:52.159642+0000","last_became_active":"2024-02-17T12:47:46.067329+0000","last_became_peered":"2024-02-17T12:47:46.067329+0000","last_unstale":"2024-02-17T12:48:52.159642+0000","last_undegraded":"2024-02-17T12:48:52.159642+0000","last_fullsized":"2024-02-17T12:48:52.159642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:19.432999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156444+0000","last_change":"2024-02-17T12:47:45.065461+0000","last_active":"2024-02-17T12:48:52.156444+0000","last_peered":"2024-02-17T12:48:52.156444+0000","last_clean":"2024-02-17T12:48:52.156444+0000","last_became_active":"2024-02-17T12:47:45.065066+0000","last_became_peered":"2024-02-17T12:47:45.065066+0000","last_unstale":"2024-02-17T12:48:52.156444+0000","last_undegraded":"2024-02-17T12:48:52.156444+0000","last_fullsized":"2024-02-17T12:48:52.156444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:25:47.292901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463958+0000","last_change":"2024-02-17T12:47:46.074222+0000","last_active":"2024-02-17T12:48:52.463958+0000","last_peered":"2024-02-17T12:48:52.463958+0000","last_clean":"2024-02-17T12:48:52.463958+0000","last_became_active":"2024-02-17T12:47:46.073886+0000","last_became_peered":"2024-02-17T12:47:46.073886+0000","last_unstale":"2024-02-17T12:48:52.463958+0000","last_undegraded":"2024-02-17T12:48:52.463958+0000","last_fullsized":"2024-02-17T12:48:52.463958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:46:18.063422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:53.812144+0000","last_change":"2024-02-17T12:47:45.071414+0000","last_active":"2024-02-17T12:48:53.812144+0000","last_peered":"2024-02-17T12:48:53.812144+0000","last_clean":"2024-02-17T12:48:53.812144+0000","last_became_active":"2024-02-17T12:47:45.071127+0000","last_became_peered":"2024-02-17T12:47:45.071127+0000","last_unstale":"2024-02-17T12:48:53.812144+0000","last_undegraded":"2024-02-17T12:48:53.812144+0000","last_fullsized":"2024-02-17T12:48:53.812144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:13:21.187718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464019+0000","last_change":"2024-02-17T12:47:46.059412+0000","last_active":"2024-02-17T12:48:52.464019+0000","last_peered":"2024-02-17T12:48:52.464019+0000","last_clean":"2024-02-17T12:48:52.464019+0000","last_became_active":"2024-02-17T12:47:46.059100+0000","last_became_peered":"2024-02-17T12:47:46.059100+0000","last_unstale":"2024-02-17T12:48:52.464019+0000","last_undegraded":"2024-02-17T12:48:52.464019+0000","last_fullsized":"2024-02-17T12:48:52.464019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:35.930613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160000+0000","last_change":"2024-02-17T12:47:46.071467+0000","last_active":"2024-02-17T12:48:52.160000+0000","last_peered":"2024-02-17T12:48:52.160000+0000","last_clean":"2024-02-17T12:48:52.160000+0000","last_became_active":"2024-02-17T12:47:46.070879+0000","last_became_peered":"2024-02-17T12:47:46.070879+0000","last_unstale":"2024-02-17T12:48:52.160000+0000","last_undegraded":"2024-02-17T12:48:52.160000+0000","last_fullsized":"2024-02-17T12:48:52.160000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.682842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:53.836290+0000","last_change":"2024-02-17T12:47:45.070545+0000","last_active":"2024-02-17T12:48:53.836290+0000","last_peered":"2024-02-17T12:48:53.836290+0000","last_clean":"2024-02-17T12:48:53.836290+0000","last_became_active":"2024-02-17T12:47:45.070265+0000","last_became_peered":"2024-02-17T12:47:45.070265+0000","last_unstale":"2024-02-17T12:48:53.836290+0000","last_undegraded":"2024-02-17T12:48:53.836290+0000","last_fullsized":"2024-02-17T12:48:53.836290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.763735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165524+0000","last_change":"2024-02-17T12:47:46.068539+0000","last_active":"2024-02-17T12:48:52.165524+0000","last_peered":"2024-02-17T12:48:52.165524+0000","last_clean":"2024-02-17T12:48:52.165524+0000","last_became_active":"2024-02-17T12:47:46.068169+0000","last_became_peered":"2024-02-17T12:47:46.068169+0000","last_unstale":"2024-02-17T12:48:52.165524+0000","last_undegraded":"2024-02-17T12:48:52.165524+0000","last_fullsized":"2024-02-17T12:48:52.165524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:27:28.485868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464598+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:48:52.464598+0000","last_peered":"2024-02-17T12:48:52.464598+0000","last_clean":"2024-02-17T12:48:52.464598+0000","last_became_active":"2024-02-17T12:47:45.066299+0000","last_became_peered":"2024-02-17T12:47:45.066299+0000","last_unstale":"2024-02-17T12:48:52.464598+0000","last_undegraded":"2024-02-17T12:48:52.464598+0000","last_fullsized":"2024-02-17T12:48:52.464598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:34:02.040002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157204+0000","last_change":"2024-02-17T12:46:32.556576+0000","last_active":"2024-02-17T12:48:52.157204+0000","last_peered":"2024-02-17T12:48:52.157204+0000","last_clean":"2024-02-17T12:48:52.157204+0000","last_became_active":"2024-02-17T12:46:32.556160+0000","last_became_peered":"2024-02-17T12:46:32.556160+0000","last_unstale":"2024-02-17T12:48:52.157204+0000","last_undegraded":"2024-02-17T12:48:52.157204+0000","last_fullsized":"2024-02-17T12:48:52.157204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_clean_scrub_stamp":"2024-02-17T12:43:13.147248+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:57:52.244951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.166240+0000","last_change":"2024-02-17T12:48:52.166240+0000","last_active":"2024-02-17T12:48:52.166240+0000","last_peered":"2024-02-17T12:48:52.166240+0000","last_clean":"2024-02-17T12:48:52.166240+0000","last_became_active":"2024-02-17T12:48:52.165882+0000","last_became_peered":"2024-02-17T12:48:52.165882+0000","last_unstale":"2024-02-17T12:48:52.166240+0000","last_undegraded":"2024-02-17T12:48:52.166240+0000","last_fullsized":"2024-02-17T12:48:52.166240+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:56:44.360362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157273+0000","last_change":"2024-02-17T12:47:45.061112+0000","last_active":"2024-02-17T12:48:52.157273+0000","last_peered":"2024-02-17T12:48:52.157273+0000","last_clean":"2024-02-17T12:48:52.157273+0000","last_became_active":"2024-02-17T12:47:45.060633+0000","last_became_peered":"2024-02-17T12:47:45.060633+0000","last_unstale":"2024-02-17T12:48:52.157273+0000","last_undegraded":"2024-02-17T12:48:52.157273+0000","last_fullsized":"2024-02-17T12:48:52.157273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:29:29.421744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163918+0000","last_change":"2024-02-17T12:47:45.066897+0000","last_active":"2024-02-17T12:48:52.163918+0000","last_peered":"2024-02-17T12:48:52.163918+0000","last_clean":"2024-02-17T12:48:52.163918+0000","last_became_active":"2024-02-17T12:47:45.066357+0000","last_became_peered":"2024-02-17T12:47:45.066357+0000","last_unstale":"2024-02-17T12:48:52.163918+0000","last_undegraded":"2024-02-17T12:48:52.163918+0000","last_fullsized":"2024-02-17T12:48:52.163918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:50:09.840414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163515+0000","last_change":"2024-02-17T12:47:46.072412+0000","last_active":"2024-02-17T12:48:52.163515+0000","last_peered":"2024-02-17T12:48:52.163515+0000","last_clean":"2024-02-17T12:48:52.163515+0000","last_became_active":"2024-02-17T12:47:46.072105+0000","last_became_peered":"2024-02-17T12:47:46.072105+0000","last_unstale":"2024-02-17T12:48:52.163515+0000","last_undegraded":"2024-02-17T12:48:52.163515+0000","last_fullsized":"2024-02-17T12:48:52.163515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:33:19.367588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156606+0000","last_change":"2024-02-17T12:47:45.067673+0000","last_active":"2024-02-17T12:48:52.156606+0000","last_peered":"2024-02-17T12:48:52.156606+0000","last_clean":"2024-02-17T12:48:52.156606+0000","last_became_active":"2024-02-17T12:47:45.067212+0000","last_became_peered":"2024-02-17T12:47:45.067212+0000","last_unstale":"2024-02-17T12:48:52.156606+0000","last_undegraded":"2024-02-17T12:48:52.156606+0000","last_fullsized":"2024-02-17T12:48:52.156606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:15:04.236158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.166062+0000","last_change":"2024-02-17T12:47:46.077476+0000","last_active":"2024-02-17T12:48:52.166062+0000","last_peered":"2024-02-17T12:48:52.166062+0000","last_clean":"2024-02-17T12:48:52.166062+0000","last_became_active":"2024-02-17T12:47:46.077101+0000","last_became_peered":"2024-02-17T12:47:46.077101+0000","last_unstale":"2024-02-17T12:48:52.166062+0000","last_undegraded":"2024-02-17T12:48:52.166062+0000","last_fullsized":"2024-02-17T12:48:52.166062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:41:41.402101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464561+0000","last_change":"2024-02-17T12:47:45.066693+0000","last_active":"2024-02-17T12:48:52.464561+0000","last_peered":"2024-02-17T12:48:52.464561+0000","last_clean":"2024-02-17T12:48:52.464561+0000","last_became_active":"2024-02-17T12:47:45.066349+0000","last_became_peered":"2024-02-17T12:47:45.066349+0000","last_unstale":"2024-02-17T12:48:52.464561+0000","last_undegraded":"2024-02-17T12:48:52.464561+0000","last_fullsized":"2024-02-17T12:48:52.464561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:30.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463335+0000","last_change":"2024-02-17T12:47:46.072899+0000","last_active":"2024-02-17T12:48:52.463335+0000","last_peered":"2024-02-17T12:48:52.463335+0000","last_clean":"2024-02-17T12:48:52.463335+0000","last_became_active":"2024-02-17T12:47:46.072488+0000","last_became_peered":"2024-02-17T12:47:46.072488+0000","last_unstale":"2024-02-17T12:48:52.463335+0000","last_undegraded":"2024-02-17T12:48:52.463335+0000","last_fullsized":"2024-02-17T12:48:52.463335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:05:43.675086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463459+0000","last_change":"2024-02-17T12:47:45.056995+0000","last_active":"2024-02-17T12:48:52.463459+0000","last_peered":"2024-02-17T12:48:52.463459+0000","last_clean":"2024-02-17T12:48:52.463459+0000","last_became_active":"2024-02-17T12:47:45.056021+0000","last_became_peered":"2024-02-17T12:47:45.056021+0000","last_unstale":"2024-02-17T12:48:52.463459+0000","last_undegraded":"2024-02-17T12:48:52.463459+0000","last_fullsized":"2024-02-17T12:48:52.463459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:41.444513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163223+0000","last_change":"2024-02-17T12:47:46.065859+0000","last_active":"2024-02-17T12:48:52.163223+0000","last_peered":"2024-02-17T12:48:52.163223+0000","last_clean":"2024-02-17T12:48:52.163223+0000","last_became_active":"2024-02-17T12:47:46.065469+0000","last_became_peered":"2024-02-17T12:47:46.065469+0000","last_unstale":"2024-02-17T12:48:52.163223+0000","last_undegraded":"2024-02-17T12:48:52.163223+0000","last_fullsized":"2024-02-17T12:48:52.163223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:51:25.105361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165363+0000","last_change":"2024-02-17T12:47:45.066524+0000","last_active":"2024-02-17T12:48:52.165363+0000","last_peered":"2024-02-17T12:48:52.165363+0000","last_clean":"2024-02-17T12:48:52.165363+0000","last_became_active":"2024-02-17T12:47:45.066218+0000","last_became_peered":"2024-02-17T12:47:45.066218+0000","last_unstale":"2024-02-17T12:48:52.165363+0000","last_undegraded":"2024-02-17T12:48:52.165363+0000","last_fullsized":"2024-02-17T12:48:52.165363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:56:47.244262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159558+0000","last_change":"2024-02-17T12:47:46.065779+0000","last_active":"2024-02-17T12:48:52.159558+0000","last_peered":"2024-02-17T12:48:52.159558+0000","last_clean":"2024-02-17T12:48:52.159558+0000","last_became_active":"2024-02-17T12:47:46.065459+0000","last_became_peered":"2024-02-17T12:47:46.065459+0000","last_unstale":"2024-02-17T12:48:52.159558+0000","last_undegraded":"2024-02-17T12:48:52.159558+0000","last_fullsized":"2024-02-17T12:48:52.159558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:08.944487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163588+0000","last_change":"2024-02-17T12:47:45.059639+0000","last_active":"2024-02-17T12:48:52.163588+0000","last_peered":"2024-02-17T12:48:52.163588+0000","last_clean":"2024-02-17T12:48:52.163588+0000","last_became_active":"2024-02-17T12:47:45.059085+0000","last_became_peered":"2024-02-17T12:47:45.059085+0000","last_unstale":"2024-02-17T12:48:52.163588+0000","last_undegraded":"2024-02-17T12:48:52.163588+0000","last_fullsized":"2024-02-17T12:48:52.163588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:48:55.847405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165668+0000","last_change":"2024-02-17T12:47:46.079623+0000","last_active":"2024-02-17T12:48:52.165668+0000","last_peered":"2024-02-17T12:48:52.165668+0000","last_clean":"2024-02-17T12:48:52.165668+0000","last_became_active":"2024-02-17T12:47:46.079285+0000","last_became_peered":"2024-02-17T12:47:46.079285+0000","last_unstale":"2024-02-17T12:48:52.165668+0000","last_undegraded":"2024-02-17T12:48:52.165668+0000","last_fullsized":"2024-02-17T12:48:52.165668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:45:14.441218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.181532+0000","last_change":"2024-02-17T12:48:52.181057+0000","last_active":"2024-02-17T12:48:52.181532+0000","last_peered":"2024-02-17T12:48:52.181532+0000","last_clean":"2024-02-17T12:48:52.181532+0000","last_became_active":"2024-02-17T12:48:52.165968+0000","last_became_peered":"2024-02-17T12:48:52.165968+0000","last_unstale":"2024-02-17T12:48:52.181532+0000","last_undegraded":"2024-02-17T12:48:52.181532+0000","last_fullsized":"2024-02-17T12:48:52.181532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:14:06.153057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157361+0000","last_change":"2024-02-17T12:47:46.074429+0000","last_active":"2024-02-17T12:48:52.157361+0000","last_peered":"2024-02-17T12:48:52.157361+0000","last_clean":"2024-02-17T12:48:52.157361+0000","last_became_active":"2024-02-17T12:47:46.074123+0000","last_became_peered":"2024-02-17T12:47:46.074123+0000","last_unstale":"2024-02-17T12:48:52.157361+0000","last_undegraded":"2024-02-17T12:48:52.157361+0000","last_fullsized":"2024-02-17T12:48:52.157361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:50:17.633831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:55.815687+0000","last_change":"2024-02-17T12:47:45.061362+0000","last_active":"2024-02-17T12:48:55.815687+0000","last_peered":"2024-02-17T12:48:55.815687+0000","last_clean":"2024-02-17T12:48:55.815687+0000","last_became_active":"2024-02-17T12:47:45.061037+0000","last_became_peered":"2024-02-17T12:47:45.061037+0000","last_unstale":"2024-02-17T12:48:55.815687+0000","last_undegraded":"2024-02-17T12:48:55.815687+0000","last_fullsized":"2024-02-17T12:48:55.815687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:01.987157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157838+0000","last_change":"2024-02-17T12:47:46.073862+0000","last_active":"2024-02-17T12:48:52.157838+0000","last_peered":"2024-02-17T12:48:52.157838+0000","last_clean":"2024-02-17T12:48:52.157838+0000","last_became_active":"2024-02-17T12:47:46.073543+0000","last_became_peered":"2024-02-17T12:47:46.073543+0000","last_unstale":"2024-02-17T12:48:52.157838+0000","last_undegraded":"2024-02-17T12:48:52.157838+0000","last_fullsized":"2024-02-17T12:48:52.157838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:57:57.567726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160633+0000","last_change":"2024-02-17T12:47:45.060616+0000","last_active":"2024-02-17T12:48:52.160633+0000","last_peered":"2024-02-17T12:48:52.160633+0000","last_clean":"2024-02-17T12:48:52.160633+0000","last_became_active":"2024-02-17T12:47:45.060296+0000","last_became_peered":"2024-02-17T12:47:45.060296+0000","last_unstale":"2024-02-17T12:48:52.160633+0000","last_undegraded":"2024-02-17T12:48:52.160633+0000","last_fullsized":"2024-02-17T12:48:52.160633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:22:44.791363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157453+0000","last_change":"2024-02-17T12:47:46.072738+0000","last_active":"2024-02-17T12:48:52.157453+0000","last_peered":"2024-02-17T12:48:52.157453+0000","last_clean":"2024-02-17T12:48:52.157453+0000","last_became_active":"2024-02-17T12:47:46.072421+0000","last_became_peered":"2024-02-17T12:47:46.072421+0000","last_unstale":"2024-02-17T12:48:52.157453+0000","last_undegraded":"2024-02-17T12:48:52.157453+0000","last_fullsized":"2024-02-17T12:48:52.157453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:50:23.501787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.470091+0000","last_change":"2024-02-17T12:48:52.470091+0000","last_active":"2024-02-17T12:48:52.470091+0000","last_peered":"2024-02-17T12:48:52.470091+0000","last_clean":"2024-02-17T12:48:52.470091+0000","last_became_active":"2024-02-17T12:48:52.466153+0000","last_became_peered":"2024-02-17T12:48:52.466153+0000","last_unstale":"2024-02-17T12:48:52.470091+0000","last_undegraded":"2024-02-17T12:48:52.470091+0000","last_fullsized":"2024-02-17T12:48:52.470091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:03:45.027983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157617+0000","last_change":"2024-02-17T12:47:46.066072+0000","last_active":"2024-02-17T12:48:52.157617+0000","last_peered":"2024-02-17T12:48:52.157617+0000","last_clean":"2024-02-17T12:48:52.157617+0000","last_became_active":"2024-02-17T12:47:46.065755+0000","last_became_peered":"2024-02-17T12:47:46.065755+0000","last_unstale":"2024-02-17T12:48:52.157617+0000","last_undegraded":"2024-02-17T12:48:52.157617+0000","last_fullsized":"2024-02-17T12:48:52.157617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:09:29.534097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159652+0000","last_change":"2024-02-17T12:47:46.067268+0000","last_active":"2024-02-17T12:48:52.159652+0000","last_peered":"2024-02-17T12:48:52.159652+0000","last_clean":"2024-02-17T12:48:52.159652+0000","last_became_active":"2024-02-17T12:47:46.066918+0000","last_became_peered":"2024-02-17T12:47:46.066918+0000","last_unstale":"2024-02-17T12:48:52.159652+0000","last_undegraded":"2024-02-17T12:48:52.159652+0000","last_fullsized":"2024-02-17T12:48:52.159652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:11:09.844954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163365+0000","last_change":"2024-02-17T12:47:46.068143+0000","last_active":"2024-02-17T12:48:52.163365+0000","last_peered":"2024-02-17T12:48:52.163365+0000","last_clean":"2024-02-17T12:48:52.163365+0000","last_became_active":"2024-02-17T12:47:46.065112+0000","last_became_peered":"2024-02-17T12:47:46.065112+0000","last_unstale":"2024-02-17T12:48:52.163365+0000","last_undegraded":"2024-02-17T12:48:52.163365+0000","last_fullsized":"2024-02-17T12:48:52.163365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:43:04.982009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163481+0000","last_change":"2024-02-17T12:47:46.076139+0000","last_active":"2024-02-17T12:48:52.163481+0000","last_peered":"2024-02-17T12:48:52.163481+0000","last_clean":"2024-02-17T12:48:52.163481+0000","last_became_active":"2024-02-17T12:47:46.075826+0000","last_became_peered":"2024-02-17T12:47:46.075826+0000","last_unstale":"2024-02-17T12:48:52.163481+0000","last_undegraded":"2024-02-17T12:48:52.163481+0000","last_fullsized":"2024-02-17T12:48:52.163481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:39:08.460716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463447+0000","last_change":"2024-02-17T12:47:46.063792+0000","last_active":"2024-02-17T12:48:52.463447+0000","last_peered":"2024-02-17T12:48:52.463447+0000","last_clean":"2024-02-17T12:48:52.463447+0000","last_became_active":"2024-02-17T12:47:46.063407+0000","last_became_peered":"2024-02-17T12:47:46.063407+0000","last_unstale":"2024-02-17T12:48:52.463447+0000","last_undegraded":"2024-02-17T12:48:52.463447+0000","last_fullsized":"2024-02-17T12:48:52.463447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:26:26.973435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163601+0000","last_change":"2024-02-17T12:47:46.066202+0000","last_active":"2024-02-17T12:48:52.163601+0000","last_peered":"2024-02-17T12:48:52.163601+0000","last_clean":"2024-02-17T12:48:52.163601+0000","last_became_active":"2024-02-17T12:47:46.065820+0000","last_became_peered":"2024-02-17T12:47:46.065820+0000","last_unstale":"2024-02-17T12:48:52.163601+0000","last_undegraded":"2024-02-17T12:48:52.163601+0000","last_fullsized":"2024-02-17T12:48:52.163601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:36:25.201697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.166996+0000","last_change":"2024-02-17T12:48:52.166996+0000","last_active":"2024-02-17T12:48:52.166996+0000","last_peered":"2024-02-17T12:48:52.166996+0000","last_clean":"2024-02-17T12:48:52.166996+0000","last_became_active":"2024-02-17T12:48:52.166589+0000","last_became_peered":"2024-02-17T12:48:52.166589+0000","last_unstale":"2024-02-17T12:48:52.166996+0000","last_undegraded":"2024-02-17T12:48:52.166996+0000","last_fullsized":"2024-02-17T12:48:52.166996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:16:30.392183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.164351+0000","last_change":"2024-02-17T12:47:46.070811+0000","last_active":"2024-02-17T12:48:52.164351+0000","last_peered":"2024-02-17T12:48:52.164351+0000","last_clean":"2024-02-17T12:48:52.164351+0000","last_became_active":"2024-02-17T12:47:46.070462+0000","last_became_peered":"2024-02-17T12:47:46.070462+0000","last_unstale":"2024-02-17T12:48:52.164351+0000","last_undegraded":"2024-02-17T12:48:52.164351+0000","last_fullsized":"2024-02-17T12:48:52.164351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:03.671869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165953+0000","last_change":"2024-02-17T12:47:46.077692+0000","last_active":"2024-02-17T12:48:52.165953+0000","last_peered":"2024-02-17T12:48:52.165953+0000","last_clean":"2024-02-17T12:48:52.165953+0000","last_became_active":"2024-02-17T12:47:46.077363+0000","last_became_peered":"2024-02-17T12:47:46.077363+0000","last_unstale":"2024-02-17T12:48:52.165953+0000","last_undegraded":"2024-02-17T12:48:52.165953+0000","last_fullsized":"2024-02-17T12:48:52.165953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:02:49.068782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159896+0000","last_change":"2024-02-17T12:47:46.073507+0000","last_active":"2024-02-17T12:48:52.159896+0000","last_peered":"2024-02-17T12:48:52.159896+0000","last_clean":"2024-02-17T12:48:52.159896+0000","last_became_active":"2024-02-17T12:47:46.073107+0000","last_became_peered":"2024-02-17T12:47:46.073107+0000","last_unstale":"2024-02-17T12:48:52.159896+0000","last_undegraded":"2024-02-17T12:48:52.159896+0000","last_fullsized":"2024-02-17T12:48:52.159896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:32:48.799300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165279+0000","last_change":"2024-02-17T12:47:46.073384+0000","last_active":"2024-02-17T12:48:52.165279+0000","last_peered":"2024-02-17T12:48:52.165279+0000","last_clean":"2024-02-17T12:48:52.165279+0000","last_became_active":"2024-02-17T12:47:46.073017+0000","last_became_peered":"2024-02-17T12:47:46.073017+0000","last_unstale":"2024-02-17T12:48:52.165279+0000","last_undegraded":"2024-02-17T12:48:52.165279+0000","last_fullsized":"2024-02-17T12:48:52.165279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:43:50.091667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.465319+0000","last_change":"2024-02-17T12:47:46.073758+0000","last_active":"2024-02-17T12:48:52.465319+0000","last_peered":"2024-02-17T12:48:52.465319+0000","last_clean":"2024-02-17T12:48:52.465319+0000","last_became_active":"2024-02-17T12:47:46.072781+0000","last_became_peered":"2024-02-17T12:47:46.072781+0000","last_unstale":"2024-02-17T12:48:52.465319+0000","last_undegraded":"2024-02-17T12:48:52.465319+0000","last_fullsized":"2024-02-17T12:48:52.465319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:10:24.811100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163113+0000","last_change":"2024-02-17T12:47:46.072025+0000","last_active":"2024-02-17T12:48:52.163113+0000","last_peered":"2024-02-17T12:48:52.163113+0000","last_clean":"2024-02-17T12:48:52.163113+0000","last_became_active":"2024-02-17T12:47:46.071662+0000","last_became_peered":"2024-02-17T12:47:46.071662+0000","last_unstale":"2024-02-17T12:48:52.163113+0000","last_undegraded":"2024-02-17T12:48:52.163113+0000","last_fullsized":"2024-02-17T12:48:52.163113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:04:07.188552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.166041+0000","last_change":"2024-02-17T12:48:52.166041+0000","last_active":"2024-02-17T12:48:52.166041+0000","last_peered":"2024-02-17T12:48:52.166041+0000","last_clean":"2024-02-17T12:48:52.166041+0000","last_became_active":"2024-02-17T12:48:52.165576+0000","last_became_peered":"2024-02-17T12:48:52.165576+0000","last_unstale":"2024-02-17T12:48:52.166041+0000","last_undegraded":"2024-02-17T12:48:52.166041+0000","last_fullsized":"2024-02-17T12:48:52.166041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:26.810385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66598,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":680,"num_read_kb":411,"num_write":236,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1013,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351003,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":462062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.47499999999999998}]},{"osd":1,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.81299999999999994}]},{"osd":2,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65700000000000003}]},{"osd":5,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.436},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.52500000000000002}]},{"osd":7,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.70999999999999996}]},{"osd":8,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.58799999999999997}]},{"osd":9,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.69399999999999995}]},{"osd":10,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.78200000000000003}]}]},{"osd":10,"up_from":73,"seq":313532612639,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1112876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":13.85,"5min":13.85,"15min":13.85},"last":0.39200000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.495,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70699999999999996,"5min":0.751,"15min":0.751},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.51500000000000001,"15min":0.52700000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.66400000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.49199999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83999999999999997,"5min":1.002,"15min":1.002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.68999999999999995,"15min":0.70399999999999996},"min":{"1min":0.47199999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.73899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.63600000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.59399999999999997,"15min":0.60899999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79400000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.438},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83899999999999997,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58799999999999997}]},{"osd":4,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64100000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.60199999999999998,"15min":0.62},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81799999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.73099999999999998,"15min":0.745},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.88300000000000001,"5min":1.022,"15min":1.022},"last":0.747},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.68700000000000006,"15min":0.70099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77100000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.625}]},{"osd":6,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.82499999999999996,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.752,"15min":0.76600000000000001},"min":{"1min":0.502,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.87,"5min":1.085,"15min":1.085},"last":0.72499999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.54900000000000004}]},{"osd":8,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.76300000000000001,"15min":0.78100000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.78500000000000003,"5min":1.107,"15min":1.107},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89500000000000002,"5min":1.119,"15min":1.119},"last":0.78700000000000003}]},{"osd":9,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.83499999999999996,"15min":0.85799999999999998},"min":{"1min":0.57799999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91800000000000004,"5min":1.151,"15min":1.151},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.81200000000000006,"15min":0.83199999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.85499999999999998,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.70499999999999996}]},{"osd":11,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77400000000000002}]}]},{"osd":9,"up_from":68,"seq":292057776166,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40468,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704684,"statfs":{"total":95995035648,"available":95953596416,"internally_reserved":0,"allocated":987136,"data_stored":461992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.45500000000000002,"15min":0.46100000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73199999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.36199999999999999}]},{"osd":1,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.64900000000000002,"15min":0.65900000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98699999999999999,"5min":1.04,"15min":1.04},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.55500000000000005}]},{"osd":2,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.40100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0249999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64400000000000002,"15min":0.65100000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.502}]},{"osd":3,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60999999999999999,"15min":0.621},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76300000000000001,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66700000000000004,"15min":0.67300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83399999999999996,"5min":1.159,"15min":1.159},"last":0.57099999999999995}]},{"osd":5,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.44800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96699999999999997,"5min":1.323,"15min":1.323},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.76500000000000001,"15min":0.77600000000000002},"min":{"1min":0.47899999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.101,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.64500000000000002}]},{"osd":6,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1379999999999999,"5min":1.268,"15min":1.268},"last":0.624},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92300000000000004,"5min":1.27,"15min":1.27},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80200000000000005,"15min":0.81599999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97299999999999998,"5min":1.351,"15min":1.351},"last":0.66600000000000004}]},{"osd":8,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.503,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.79000000000000004,"15min":0.80200000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95599999999999996,"5min":1.363,"15min":1.363},"last":0.63600000000000001}]},{"osd":10,"last update":"Sat Feb 17 12:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0429999999999999,"5min":1.228,"15min":1.228},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.81899999999999995},"min":{"1min":0.61399999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2849999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.68799999999999994}]}]},{"osd":8,"up_from":60,"seq":257698037802,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":462370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.35899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.91800000000000004,"15min":1.054},"min":{"1min":0.441,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89000000000000001,"5min":10.955,"15min":10.955},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.434,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.71099999999999997}]},{"osd":2,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.44900000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.755},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58199999999999996}]},{"osd":3,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.36499999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.51600000000000001}]},{"osd":4,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.93300000000000005,"15min":1.0680000000000001},"min":{"1min":0.46100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":10.972,"15min":10.972},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.92800000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64500000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.372,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.34899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.69199999999999995}]},{"osd":6,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.879},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.878},"min":{"1min":0.53200000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97499999999999998,"5min":4.9269999999999996,"15min":4.9269999999999996},"last":0.86299999999999999}]},{"osd":7,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.49399999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":1.022,"15min":1.022},"last":0.77700000000000002}]},{"osd":9,"last update":"Sat Feb 17 12:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.69999999999999996,"15min":0.68799999999999994},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":1.038},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":1.006}]},{"osd":10,"last update":"Sat Feb 17 12:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95299999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.2330000000000001,"15min":1.3009999999999999},"min":{"1min":0.55200000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.98399999999999999,"5min":10.987,"15min":10.987},"last":0.98699999999999999}]},{"osd":11,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":1.024},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.97199999999999998}]}]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1581056,"data_stored":1055279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42499999999999999,"15min":0.41199999999999998},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.434},"min":{"1min":0.315,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Feb 17 12:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77500000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.748,"15min":0.63},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":15.003,"15min":15.003},"last":0.64400000000000002}]},{"osd":3,"last update":"Sat Feb 17 12:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61199999999999999,"15min":0.622},"min":{"1min":0.40100000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.746,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.62,"15min":0.629},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57199999999999995}]},{"osd":4,"last update":"Sat Feb 17 12:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.73899999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.75600000000000001,"5min":0.872,"15min":0.872},"last":0.53100000000000003}]},{"osd":5,"last update":"Sat Feb 17 12:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.80200000000000005,"15min":0.68600000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":15.028,"15min":15.028},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63,"15min":0.61299999999999999},"min":{"1min":0.42199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90600000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59599999999999997}]},{"osd":6,"last update":"Sat Feb 17 12:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.82399999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.8799999999999999,"5min":0.93999999999999995,"15min":0.77400000000000002},"min":{"1min":0.45100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":14.579000000000001,"5min":14.579000000000001,"15min":14.579000000000001},"last":0.626}]},{"osd":8,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.60999999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96199999999999997,"5min":1.02,"15min":1.02},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75700000000000001,"15min":0.73799999999999999},"min":{"1min":0.65500000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.72699999999999998}]},{"osd":9,"last update":"Sat Feb 17 12:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.754,"15min":0.76800000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88600000000000001,"5min":1.04,"15min":1.04},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.403,"5min":0.89500000000000002,"15min":0.81100000000000005},"min":{"1min":0.47599999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":14.598000000000001,"5min":14.598000000000001,"15min":14.598000000000001},"last":0.67200000000000004}]},{"osd":10,"last update":"Sat Feb 17 12:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.751,"15min":0.75600000000000001},"min":{"1min":0.45300000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0309999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94399999999999995,"5min":1.103,"15min":1.103},"last":0.745}]},{"osd":11,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.77900000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495668,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":461709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.372},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.31},{"interface":"front","average":{"1min":0.379,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.313,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54500000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.34699999999999998}]},{"osd":1,"last update":"Sat Feb 17 12:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.495,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76700000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.46700000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84099999999999997,"5min":1.173,"15min":1.173},"last":0.46200000000000002}]},{"osd":2,"last update":"Sat Feb 17 12:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58399999999999996,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.63100000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.32700000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63700000000000001,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42199999999999999}]},{"osd":4,"last update":"Sat Feb 17 12:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80200000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.45200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.84199999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.435}]},{"osd":5,"last update":"Sat Feb 17 12:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.62,"15min":0.60199999999999998},"min":{"1min":0.379,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.873,"5min":1.284,"15min":1.284},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.753,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.72399999999999998,"15min":0.67100000000000004},"min":{"1min":0.57599999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":5.069,"5min":5.069,"15min":5.069},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65300000000000002,"15min":0.64000000000000001},"min":{"1min":0.46700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47699999999999998}]},{"osd":8,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.73399999999999999,"15min":0.70799999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":5.0940000000000003,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90800000000000003,"5min":1.252,"15min":1.252},"last":0.66200000000000003}]},{"osd":9,"last update":"Sat Feb 17 12:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85699999999999998,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.48199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58099999999999996}]},{"osd":10,"last update":"Sat Feb 17 12:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.749,"15min":0.76300000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.88500000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":522016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48199999999999998,"15min":0.47999999999999998},"min":{"1min":0.35399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.62,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":1.393,"5min":0.67000000000000004,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":14.91,"5min":14.91,"15min":14.91},"last":0.68000000000000005}]},{"osd":2,"last update":"Sat Feb 17 12:48:20 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55400000000000005,"15min":0.51200000000000001},"min":{"1min":0.44900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56200000000000006,"15min":0.51600000000000001},"min":{"1min":0.45300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79400000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.70599999999999996}]},{"osd":3,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.48599999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78500000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.621},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.432,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73599999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54500000000000004}]},{"osd":4,"last update":"Sat Feb 17 12:48:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.41499999999999998,"15min":0.38900000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.71199999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.432},{"interface":"front","average":{"1min":0.502,"5min":0.436,"15min":0.41799999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.73599999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56699999999999995}]},{"osd":5,"last update":"Sat Feb 17 12:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.61499999999999999,"15min":0.57899999999999996},"min":{"1min":0.46300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90500000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.626,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64900000000000002}]},{"osd":6,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88800000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84199999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.60699999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:48:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57299999999999995,"15min":0.54900000000000004},"min":{"1min":0.32300000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.626,"15min":0.59699999999999998},"min":{"1min":0.374,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0329999999999999,"5min":1.079,"15min":1.079},"last":0.76600000000000001}]},{"osd":8,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.67400000000000004,"15min":0.63400000000000001},"min":{"1min":0.63800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.58999999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.74199999999999999}]},{"osd":9,"last update":"Sat Feb 17 12:48:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.89900000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.505,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46000000000000002}]},{"osd":10,"last update":"Sat Feb 17 12:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73299999999999998,"15min":0.71899999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.877,"5min":0.75700000000000001,"15min":0.73699999999999999},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.79800000000000004}]},{"osd":11,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.754}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":463048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 12:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53400000000000003,"15min":0.48899999999999999},"min":{"1min":0.34499999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81299999999999994,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54500000000000004,"15min":0.495},"min":{"1min":0.41699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64400000000000002}]},{"osd":2,"last update":"Sat Feb 17 12:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.68899999999999995,"15min":0.749},"min":{"1min":0.42899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":8.3919999999999995,"5min":8.3919999999999995,"15min":8.3919999999999995},"last":0.76400000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.51100000000000001}]},{"osd":4,"last update":"Sat Feb 17 12:48:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.629},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59399999999999997,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85199999999999998,"5min":0.872,"15min":0.872},"last":0.69599999999999995}]},{"osd":5,"last update":"Sat Feb 17 12:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.42099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68999999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.751},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.48099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.748,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.67700000000000005}]},{"osd":6,"last update":"Sat Feb 17 12:47:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.28999999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.626,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Sat Feb 17 12:47:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.443},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64600000000000002,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.65300000000000002,"15min":0.66500000000000004},"min":{"1min":0.45000000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79500000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.79900000000000004}]},{"osd":9,"last update":"Sat Feb 17 12:48:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.64100000000000001,"15min":0.66200000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77300000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.59799999999999998}]},{"osd":10,"last update":"Sat Feb 17 12:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77700000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.71999999999999997,"15min":0.73599999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.72199999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72699999999999998}]},{"osd":11,"last update":"Sat Feb 17 12:48:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.82699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1044480,"data_stored":521614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.46300000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70499999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.29099999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67700000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54300000000000004}]},{"osd":1,"last update":"Sat Feb 17 12:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53400000000000003,"15min":0.504},"min":{"1min":0.312,"5min":0.309,"15min":0.309},"max":{"1min":0.746,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53700000000000003,"15min":0.505},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.71899999999999997}]},{"osd":3,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56000000000000005,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.77200000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.79800000000000004}]},{"osd":4,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.82699999999999996}]},{"osd":5,"last update":"Sat Feb 17 12:48:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.46000000000000002,"15min":0.44},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79000000000000004,"5min":1.325,"15min":1.325},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.47899999999999998,"15min":0.48399999999999999},"min":{"1min":0.32800000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84699999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.69499999999999995}]},{"osd":6,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91100000000000003,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.93700000000000006,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.73899999999999999}]},{"osd":7,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.60999999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.85699999999999998,"5min":0.998,"15min":0.998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60899999999999999,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95899999999999996,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.76500000000000001}]},{"osd":9,"last update":"Sat Feb 17 12:48:09 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92300000000000004,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71599999999999997,"15min":0.72199999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98399999999999999,"5min":1.284,"15min":1.284},"last":0.91000000000000003}]},{"osd":10,"last update":"Sat Feb 17 12:48:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96799999999999997,"5min":1.347,"15min":1.347},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.006,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.876}]},{"osd":11,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.88800000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":461534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.32300000000000001},"min":{"1min":0.26000000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59799999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.33700000000000002},"min":{"1min":0.26300000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.379}]},{"osd":1,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77100000000000002,"5min":0.879,"15min":0.879},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.44800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.74199999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.752,"5min":0.88,"15min":0.88},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77900000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.58999999999999997}]},{"osd":4,"last update":"Sat Feb 17 12:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.70699999999999996,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.38200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.86799999999999999,"5min":0.876,"15min":0.876},"last":0.57599999999999996}]},{"osd":5,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.505,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86699999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84799999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64300000000000002}]},{"osd":6,"last update":"Sat Feb 17 12:48:53 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.504,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73799999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.441},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.495,"15min":0.505},"min":{"1min":0.35099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.40300000000000002}]},{"osd":7,"last update":"Sat Feb 17 12:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":0.876,"15min":0.876},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.505,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.751}]},{"osd":8,"last update":"Sat Feb 17 12:47:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91600000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.71299999999999997,"15min":0.72599999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0069999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.56899999999999995}]},{"osd":9,"last update":"Sat Feb 17 12:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.66900000000000004,"15min":0.68999999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93999999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69399999999999995,"15min":0.71299999999999997},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82599999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.50800000000000001}]},{"osd":10,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.83199999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.76400000000000001,"15min":0.77400000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.77900000000000003}]},{"osd":11,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.73299999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":462974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57899999999999996}]},{"osd":1,"last update":"Sat Feb 17 12:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.38900000000000001,"15min":0.35899999999999999},"min":{"1min":0.28399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.372},"min":{"1min":0.28799999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.71299999999999997,"5min":0.751,"15min":0.751},"last":0.39900000000000002}]},{"osd":2,"last update":"Sat Feb 17 12:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.77800000000000002,"15min":0.63200000000000001},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86299999999999999,"5min":18.73,"15min":18.73},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.45500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73799999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Feb 17 12:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86499999999999999,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":0.871,"15min":0.871},"last":0.42199999999999999}]},{"osd":5,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.49199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69499999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60999999999999999,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.81799999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.747}]},{"osd":6,"last update":"Sat Feb 17 12:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58699999999999997,"15min":0.57199999999999995},"min":{"1min":0.436,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.74299999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57199999999999995,"15min":0.54700000000000004},"min":{"1min":0.45600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84699999999999998,"5min":0.877,"15min":0.877},"last":0.56299999999999994}]},{"osd":7,"last update":"Sat Feb 17 12:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53600000000000003,"15min":0.52100000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.65200000000000002}]},{"osd":8,"last update":"Sat Feb 17 12:48:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.83699999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76300000000000001}]},{"osd":9,"last update":"Sat Feb 17 12:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.60999999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.89900000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68300000000000005}]},{"osd":10,"last update":"Sat Feb 17 12:48:31 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85899999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.72199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.86899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73199999999999998}]},{"osd":11,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.78200000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1054566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.435,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82499999999999996,"5min":1.02,"15min":1.02},"last":0.47599999999999998}]},{"osd":1,"last update":"Sat Feb 17 12:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.55600000000000005,"15min":0.52400000000000002},"min":{"1min":0.53200000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.57599999999999996,"15min":0.53000000000000003},"min":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.71999999999999997}]},{"osd":2,"last update":"Sat Feb 17 12:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.41099999999999998,"15min":0.38500000000000001},"min":{"1min":0.34000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.626,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.42099999999999999,"15min":0.377},"min":{"1min":0.32600000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.50600000000000001}]},{"osd":3,"last update":"Sat Feb 17 12:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.57399999999999995}]},{"osd":4,"last update":"Sat Feb 17 12:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.59099999999999997,"15min":0.55000000000000004},"min":{"1min":0.48799999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.58199999999999996,"15min":0.53000000000000003},"min":{"1min":0.52200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Feb 17 12:47:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82899999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.54000000000000004}]},{"osd":7,"last update":"Sat Feb 17 12:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68600000000000005,"15min":0.67700000000000005},"min":{"1min":0.53800000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92700000000000005,"5min":1,"15min":1},"last":0.628},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.495,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68700000000000006}]},{"osd":8,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.629,"15min":0.625},"min":{"1min":0.51200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.46600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70499999999999996}]},{"osd":9,"last update":"Sat Feb 17 12:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66900000000000004,"15min":0.67000000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88800000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.747},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70199999999999996,"15min":0.70499999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77500000000000002}]},{"osd":10,"last update":"Sat Feb 17 12:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.755,"15min":0.74099999999999999},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.97499999999999998,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.70399999999999996,"15min":0.68400000000000005},"min":{"1min":0.59099999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T12:49:00.930 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-17T12:49:00.930 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-17T12:49:00.930 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-17T12:49:00.930 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph health --format=json 2024-02-17T12:49:01.175 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:49:01.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:01 smithi018 ceph-mon[32625]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:49:01.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:01 smithi018 ceph-mon[32625]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 111 B/s, 1 keys/s, 1 objects/s recovering 2024-02-17T12:49:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:01 smithi146 ceph-mon[36454]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:49:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:01 smithi146 ceph-mon[36454]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 111 B/s, 1 keys/s, 1 objects/s recovering 2024-02-17T12:49:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:01 smithi119 ceph-mon[36188]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:49:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:01 smithi119 ceph-mon[36188]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 111 B/s, 1 keys/s, 1 objects/s recovering 2024-02-17T12:49:02.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.715+0000 7fd61ca2c700 1 -- 172.21.15.18:0/275416192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd618107ca0 msgr2=0x7fd618108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:02.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.715+0000 7fd61ca2c700 1 --2- 172.21.15.18:0/275416192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd618107ca0 0x7fd618108080 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd6080099c0 tx=0x7fd608009cd0 comp rx=0 tx=0).stop 2024-02-17T12:49:02.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.715+0000 7fd61ca2c700 1 -- 172.21.15.18:0/275416192 shutdown_connections 2024-02-17T12:49:02.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.715+0000 7fd61ca2c700 1 --2- 172.21.15.18:0/275416192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd61806ad80 0x7fd61810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:02.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.715+0000 7fd61ca2c700 1 --2- 172.21.15.18:0/275416192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd618107ca0 0x7fd618108080 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:02.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.715+0000 7fd61ca2c700 1 --2- 172.21.15.18:0/275416192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd61806a3c0 0x7fd61806a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:02.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.715+0000 7fd61ca2c700 1 -- 172.21.15.18:0/275416192 >> 172.21.15.18:0/275416192 conn(0x7fd618077b80 msgr2=0x7fd618077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:02.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.715+0000 7fd61ca2c700 1 -- 172.21.15.18:0/275416192 shutdown_connections 2024-02-17T12:49:02.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.715+0000 7fd61ca2c700 1 -- 172.21.15.18:0/275416192 wait complete. 2024-02-17T12:49:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.716+0000 7fd61ca2c700 1 Processor -- start 2024-02-17T12:49:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd61ca2c700 1 -- start start 2024-02-17T12:49:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd61ca2c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd61806a3c0 0x7fd61819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd61ca2c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd61806ad80 0x7fd61819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd61ca2c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd618107ca0 0x7fd61819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd61ca2c700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61819f2b0 con 0x7fd61806ad80 2024-02-17T12:49:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd61ca2c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61819f420 con 0x7fd618107ca0 2024-02-17T12:49:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd61ca2c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61819f5e0 con 0x7fd61806a3c0 2024-02-17T12:49:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd60dd9b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd61806ad80 0x7fd61819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd60dd9b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd61806ad80 0x7fd61819a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53132/0 (socket says 172.21.15.18:53132) 2024-02-17T12:49:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd60dd9b700 1 -- 172.21.15.18:0/907842723 learned_addr learned my addr 172.21.15.18:0/907842723 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd61659c700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd61806a3c0 0x7fd61819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd616d9d700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd618107ca0 0x7fd61819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd60dd9b700 1 -- 172.21.15.18:0/907842723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd61806a3c0 msgr2=0x7fd61819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd60dd9b700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd61806a3c0 0x7fd61819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd60dd9b700 1 -- 172.21.15.18:0/907842723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd618107ca0 msgr2=0x7fd61819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd60dd9b700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd618107ca0 0x7fd61819eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd60dd9b700 1 -- 172.21.15.18:0/907842723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6080095f0 con 0x7fd61806ad80 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.717+0000 7fd60dd9b700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd61806ad80 0x7fd61819a910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd60000b530 tx=0x7fd60000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.718+0000 7fd60ffff700 1 -- 172.21.15.18:0/907842723 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600011830 con 0x7fd61806ad80 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.718+0000 7fd61ca2c700 1 -- 172.21.15.18:0/907842723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61819f8c0 con 0x7fd61806ad80 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.718+0000 7fd60ffff700 1 -- 172.21.15.18:0/907842723 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd600011e70 con 0x7fd61806ad80 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.718+0000 7fd60ffff700 1 -- 172.21.15.18:0/907842723 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60000f300 con 0x7fd61806ad80 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.718+0000 7fd61ca2c700 1 -- 172.21.15.18:0/907842723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6181a7120 con 0x7fd61806ad80 2024-02-17T12:49:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.719+0000 7fd61ca2c700 1 -- 172.21.15.18:0/907842723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd618068930 con 0x7fd61806ad80 2024-02-17T12:49:02.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.720+0000 7fd60ffff700 1 -- 172.21.15.18:0/907842723 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fd6000106d0 con 0x7fd61806ad80 2024-02-17T12:49:02.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.720+0000 7fd60ffff700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd6040a21e0 0x7fd6040a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:02.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.720+0000 7fd61659c700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd6040a21e0 0x7fd6040a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:02.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.720+0000 7fd60ffff700 1 -- 172.21.15.18:0/907842723 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd6000cdce0 con 0x7fd61806ad80 2024-02-17T12:49:02.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.721+0000 7fd61659c700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd6040a21e0 0x7fd6040a46a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd608009990 tx=0x7fd608005260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:02.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.724+0000 7fd60ffff700 1 -- 172.21.15.18:0/907842723 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd600096c90 con 0x7fd61806ad80 2024-02-17T12:49:02.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.982+0000 7fd61ca2c700 1 -- 172.21.15.18:0/907842723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd61819b730 con 0x7fd61806ad80 2024-02-17T12:49:02.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.983+0000 7fd60ffff700 1 -- 172.21.15.18:0/907842723 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd60009a350 con 0x7fd61806ad80 2024-02-17T12:49:02.975 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:02.975 INFO:teuthology.orchestra.run.smithi018.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-17T12:49:02.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.985+0000 7fd61ca2c700 1 -- 172.21.15.18:0/907842723 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd6040a21e0 msgr2=0x7fd6040a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:02.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.985+0000 7fd61ca2c700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd6040a21e0 0x7fd6040a46a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd608009990 tx=0x7fd608005260 comp rx=0 tx=0).stop 2024-02-17T12:49:02.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.985+0000 7fd61ca2c700 1 -- 172.21.15.18:0/907842723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd61806ad80 msgr2=0x7fd61819a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:02.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.985+0000 7fd61ca2c700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd61806ad80 0x7fd61819a910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd60000b530 tx=0x7fd60000b840 comp rx=0 tx=0).stop 2024-02-17T12:49:02.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.985+0000 7fd61ca2c700 1 -- 172.21.15.18:0/907842723 shutdown_connections 2024-02-17T12:49:02.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.985+0000 7fd61ca2c700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd618107ca0 0x7fd61819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:02.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.985+0000 7fd61ca2c700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd6040a21e0 0x7fd6040a46a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:02.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.985+0000 7fd61ca2c700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd61806a3c0 0x7fd61819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:02.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.985+0000 7fd61ca2c700 1 --2- 172.21.15.18:0/907842723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd61806ad80 0x7fd61819a910 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:02.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.985+0000 7fd61ca2c700 1 -- 172.21.15.18:0/907842723 >> 172.21.15.18:0/907842723 conn(0x7fd618077b80 msgr2=0x7fd6180ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:02.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.985+0000 7fd61ca2c700 1 -- 172.21.15.18:0/907842723 shutdown_connections 2024-02-17T12:49:02.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:02.985+0000 7fd61ca2c700 1 -- 172.21.15.18:0/907842723 wait complete. 2024-02-17T12:49:03.548 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-17T12:49:03.549 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:49:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.662+0000 7fcfa6d65640 1 -- 172.21.15.18:0/106704904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfa01000c0 msgr2=0x7fcfa01004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.662+0000 7fcfa6d65640 1 --2- 172.21.15.18:0/106704904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfa01000c0 0x7fcfa01004e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009900 tx=0x7fcf9402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.662+0000 7fcfa6d65640 1 -- 172.21.15.18:0/106704904 shutdown_connections 2024-02-17T12:49:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.662+0000 7fcfa6d65640 1 --2- 172.21.15.18:0/106704904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfa0101c70 0x7fcfa0108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.662+0000 7fcfa6d65640 1 --2- 172.21.15.18:0/106704904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa01012b0 0x7fcfa0101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.662+0000 7fcfa6d65640 1 --2- 172.21.15.18:0/106704904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfa01000c0 0x7fcfa01004e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.662+0000 7fcfa6d65640 1 -- 172.21.15.18:0/106704904 >> 172.21.15.18:0/106704904 conn(0x7fcfa00fb630 msgr2=0x7fcfa00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.662+0000 7fcfa6d65640 1 -- 172.21.15.18:0/106704904 shutdown_connections 2024-02-17T12:49:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.662+0000 7fcfa6d65640 1 -- 172.21.15.18:0/106704904 wait complete. 2024-02-17T12:49:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcfa6d65640 1 Processor -- start 2024-02-17T12:49:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcfa6d65640 1 -- start start 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcfa6d65640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfa01000c0 0x7fcfa0196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcfa6d65640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfa01012b0 0x7fcfa0196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcfa6d65640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa0101c70 0x7fcfa019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcfa6d65640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa019dc40 con 0x7fcfa01012b0 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcfa6d65640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa019ddb0 con 0x7fcfa0101c70 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcfa6d65640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa019df70 con 0x7fcfa01000c0 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcf9ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfa01012b0 0x7fcfa0196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcf9ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfa01012b0 0x7fcfa0196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53156/0 (socket says 172.21.15.18:53156) 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcf9ffff640 1 -- 172.21.15.18:0/3459821154 learned_addr learned my addr 172.21.15.18:0/3459821154 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcfa4b01640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfa01000c0 0x7fcfa0196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcf9ffff640 1 -- 172.21.15.18:0/3459821154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfa01000c0 msgr2=0x7fcfa0196150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcf9ffff640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfa01000c0 0x7fcfa0196150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcf9ffff640 1 -- 172.21.15.18:0/3459821154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa0101c70 msgr2=0x7fcfa019d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:03.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcfa5302640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa0101c70 0x7fcfa019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:03.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcf9ffff640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa0101c70 0x7fcfa019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcf9ffff640 1 -- 172.21.15.18:0/3459821154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf94009560 con 0x7fcfa01012b0 2024-02-17T12:49:03.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcfa4b01640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfa01000c0 0x7fcfa0196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:03.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.663+0000 7fcf9ffff640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfa01012b0 0x7fcfa0196690 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00c920 tx=0x7fcf8c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:03.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.664+0000 7fcf9dffb640 1 -- 172.21.15.18:0/3459821154 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c007c70 con 0x7fcfa01012b0 2024-02-17T12:49:03.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.664+0000 7fcfa6d65640 1 -- 172.21.15.18:0/3459821154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa019e250 con 0x7fcfa01012b0 2024-02-17T12:49:03.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.664+0000 7fcf9dffb640 1 -- 172.21.15.18:0/3459821154 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf8c007dd0 con 0x7fcfa01012b0 2024-02-17T12:49:03.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.664+0000 7fcf9dffb640 1 -- 172.21.15.18:0/3459821154 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c018860 con 0x7fcfa01012b0 2024-02-17T12:49:03.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.664+0000 7fcfa6d65640 1 -- 172.21.15.18:0/3459821154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa019e830 con 0x7fcfa01012b0 2024-02-17T12:49:03.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.665+0000 7fcfa6d65640 1 -- 172.21.15.18:0/3459821154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa0107380 con 0x7fcfa01012b0 2024-02-17T12:49:03.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.666+0000 7fcf9dffb640 1 -- 172.21.15.18:0/3459821154 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fcf8c00fac0 con 0x7fcfa01012b0 2024-02-17T12:49:03.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.666+0000 7fcf9dffb640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcf900a2150 0x7fcf900a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:03.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.666+0000 7fcf9dffb640 1 -- 172.21.15.18:0/3459821154 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcf8c0cdff0 con 0x7fcfa01012b0 2024-02-17T12:49:03.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.666+0000 7fcfa4b01640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcf900a2150 0x7fcf900a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:03.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.667+0000 7fcfa4b01640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcf900a2150 0x7fcf900a4610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcf94036040 tx=0x7fcf94000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:03.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.671+0000 7fcf9dffb640 1 -- 172.21.15.18:0/3459821154 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf8c096e90 con 0x7fcfa01012b0 2024-02-17T12:49:03.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:03 smithi146 ceph-mon[36454]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 100 B/s, 0 keys/s, 1 objects/s recovering 2024-02-17T12:49:03.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:03 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/907842723' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-17T12:49:03.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:03 smithi018 ceph-mon[32625]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 100 B/s, 0 keys/s, 1 objects/s recovering 2024-02-17T12:49:03.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:03 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/907842723' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-17T12:49:03.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:03 smithi119 ceph-mon[36188]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 100 B/s, 0 keys/s, 1 objects/s recovering 2024-02-17T12:49:03.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:03 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/907842723' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-17T12:49:03.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.855+0000 7fcfa6d65640 1 -- 172.21.15.18:0/3459821154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfa010bd30 con 0x7fcfa01012b0 2024-02-17T12:49:03.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.857+0000 7fcf9dffb640 1 -- 172.21.15.18:0/3459821154 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22607 (secure 0 0 0) 0x7fcf8c09a550 con 0x7fcfa01012b0 2024-02-17T12:49:03.850 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:03.850 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":88,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:48:52.149237+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:49:03.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.859+0000 7fcfa6d65640 1 -- 172.21.15.18:0/3459821154 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcf900a2150 msgr2=0x7fcf900a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:03.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.859+0000 7fcfa6d65640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcf900a2150 0x7fcf900a4610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcf94036040 tx=0x7fcf94000bc0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.859+0000 7fcfa6d65640 1 -- 172.21.15.18:0/3459821154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfa01012b0 msgr2=0x7fcfa0196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:03.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.859+0000 7fcfa6d65640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfa01012b0 0x7fcfa0196690 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00c920 tx=0x7fcf8c00cde0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.859+0000 7fcfa6d65640 1 -- 172.21.15.18:0/3459821154 shutdown_connections 2024-02-17T12:49:03.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.859+0000 7fcfa6d65640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fcf900a2150 0x7fcf900a4610 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.859+0000 7fcfa6d65640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa0101c70 0x7fcfa019d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.859+0000 7fcfa6d65640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfa01012b0 0x7fcfa0196690 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.859+0000 7fcfa6d65640 1 --2- 172.21.15.18:0/3459821154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfa01000c0 0x7fcfa0196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.859+0000 7fcfa6d65640 1 -- 172.21.15.18:0/3459821154 >> 172.21.15.18:0/3459821154 conn(0x7fcfa00fb630 msgr2=0x7fcfa0104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:03.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.859+0000 7fcfa6d65640 1 -- 172.21.15.18:0/3459821154 shutdown_connections 2024-02-17T12:49:03.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.859+0000 7fcfa6d65640 1 -- 172.21.15.18:0/3459821154 wait complete. 2024-02-17T12:49:03.863 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T12:43:12.450140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T12:47:43.690543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T12:47:44.653229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T12:49:03.863 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T12:49:03.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.977+0000 7f22b09a0640 1 -- 172.21.15.18:0/3809496945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22ac104410 msgr2=0x7f22ac106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:03.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.977+0000 7f22b09a0640 1 --2- 172.21.15.18:0/3809496945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22ac104410 0x7f22ac106800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009900 tx=0x7f22a002bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:03.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.977+0000 7f22b09a0640 1 -- 172.21.15.18:0/3809496945 shutdown_connections 2024-02-17T12:49:03.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.977+0000 7f22b09a0640 1 --2- 172.21.15.18:0/3809496945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22ac106d40 0x7f22ac109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.977+0000 7f22b09a0640 1 --2- 172.21.15.18:0/3809496945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22ac104410 0x7f22ac106800 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.977+0000 7f22b09a0640 1 --2- 172.21.15.18:0/3809496945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22ac101ae0 0x7f22ac103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.977+0000 7f22b09a0640 1 -- 172.21.15.18:0/3809496945 >> 172.21.15.18:0/3809496945 conn(0x7f22ac0fb4f0 msgr2=0x7f22ac0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.977+0000 7f22b09a0640 1 -- 172.21.15.18:0/3809496945 shutdown_connections 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.977+0000 7f22b09a0640 1 -- 172.21.15.18:0/3809496945 wait complete. 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.977+0000 7f22b09a0640 1 Processor -- start 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.977+0000 7f22b09a0640 1 -- start start 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22b09a0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22ac101ae0 0x7f22ac1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22b09a0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22ac104410 0x7f22ac1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22b09a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22ac106d40 0x7f22ac19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22b09a0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac19ec90 con 0x7f22ac101ae0 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22b09a0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac19ee00 con 0x7f22ac106d40 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22b09a0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac19efc0 con 0x7f22ac104410 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22ab7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22ac101ae0 0x7f22ac1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:03.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22ab7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22ac101ae0 0x7f22ac1971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53176/0 (socket says 172.21.15.18:53176) 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22ab7fe640 1 -- 172.21.15.18:0/3136305180 learned_addr learned my addr 172.21.15.18:0/3136305180 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22aaffd640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22ac104410 0x7f22ac1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22abfff640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22ac106d40 0x7f22ac19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22ab7fe640 1 -- 172.21.15.18:0/3136305180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22ac104410 msgr2=0x7f22ac1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22ab7fe640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22ac104410 0x7f22ac1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22ab7fe640 1 -- 172.21.15.18:0/3136305180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22ac106d40 msgr2=0x7f22ac19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22ab7fe640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22ac106d40 0x7f22ac19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22ab7fe640 1 -- 172.21.15.18:0/3136305180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a0009560 con 0x7f22ac101ae0 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22abfff640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22ac106d40 0x7f22ac19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22ab7fe640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22ac101ae0 0x7f22ac1971a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f229c00b4a0 tx=0x7f229c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22a8ff9640 1 -- 172.21.15.18:0/3136305180 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c010610 con 0x7f22ac101ae0 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22a8ff9640 1 -- 172.21.15.18:0/3136305180 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f229c010c50 con 0x7f22ac101ae0 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22b09a0640 1 -- 172.21.15.18:0/3136305180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac19f2a0 con 0x7f22ac101ae0 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22a8ff9640 1 -- 172.21.15.18:0/3136305180 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c017e20 con 0x7f22ac101ae0 2024-02-17T12:49:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.978+0000 7f22b09a0640 1 -- 172.21.15.18:0/3136305180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac19f800 con 0x7f22ac101ae0 2024-02-17T12:49:03.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.979+0000 7f22b09a0640 1 -- 172.21.15.18:0/3136305180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac0fcfe0 con 0x7f22ac101ae0 2024-02-17T12:49:03.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.980+0000 7f22a8ff9640 1 -- 172.21.15.18:0/3136305180 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f229c017480 con 0x7f22ac101ae0 2024-02-17T12:49:03.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.981+0000 7f22a8ff9640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f22940a2150 0x7f22940a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:03.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.981+0000 7f22aaffd640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f22940a2150 0x7f22940a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:03.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.981+0000 7f22a8ff9640 1 -- 172.21.15.18:0/3136305180 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f229c0cd3b0 con 0x7f22ac101ae0 2024-02-17T12:49:03.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.981+0000 7f22aaffd640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f22940a2150 0x7f22940a4610 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f22a0005ab0 tx=0x7f22a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:03.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:03.986+0000 7f22a8ff9640 1 -- 172.21.15.18:0/3136305180 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f229c096300 con 0x7f22ac101ae0 2024-02-17T12:49:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.166+0000 7f22b09a0640 1 -- 172.21.15.18:0/3136305180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22ac10cad0 con 0x7f22ac101ae0 2024-02-17T12:49:04.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.166+0000 7f22a8ff9640 1 -- 172.21.15.18:0/3136305180 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f229c0999c0 con 0x7f22ac101ae0 2024-02-17T12:49:04.161 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-02-17T12:49:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.168+0000 7f22b09a0640 1 -- 172.21.15.18:0/3136305180 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f22940a2150 msgr2=0x7f22940a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.168+0000 7f22b09a0640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f22940a2150 0x7f22940a4610 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f22a0005ab0 tx=0x7f22a0005fd0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.168+0000 7f22b09a0640 1 -- 172.21.15.18:0/3136305180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22ac101ae0 msgr2=0x7f22ac1971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.168+0000 7f22b09a0640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22ac101ae0 0x7f22ac1971a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f229c00b4a0 tx=0x7f229c00b960 comp rx=0 tx=0).stop 2024-02-17T12:49:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.168+0000 7f22b09a0640 1 -- 172.21.15.18:0/3136305180 shutdown_connections 2024-02-17T12:49:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.168+0000 7f22b09a0640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f22940a2150 0x7f22940a4610 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.168+0000 7f22b09a0640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22ac106d40 0x7f22ac19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.168+0000 7f22b09a0640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22ac104410 0x7f22ac1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.168+0000 7f22b09a0640 1 --2- 172.21.15.18:0/3136305180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22ac101ae0 0x7f22ac1971a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.168+0000 7f22b09a0640 1 -- 172.21.15.18:0/3136305180 >> 172.21.15.18:0/3136305180 conn(0x7f22ac0fb4f0 msgr2=0x7f22ac102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:04.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.168+0000 7f22b09a0640 1 -- 172.21.15.18:0/3136305180 shutdown_connections 2024-02-17T12:49:04.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.168+0000 7f22b09a0640 1 -- 172.21.15.18:0/3136305180 wait complete. 2024-02-17T12:49:04.172 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T12:49:04.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.284+0000 7fd9c641c640 1 -- 172.21.15.18:0/2273689046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c00ff820 msgr2=0x7fd9c00ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.284+0000 7fd9c641c640 1 --2- 172.21.15.18:0/2273689046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c00ff820 0x7fd9c00ffc40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009900 tx=0x7fd9b402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:04.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.284+0000 7fd9c641c640 1 -- 172.21.15.18:0/2273689046 shutdown_connections 2024-02-17T12:49:04.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.284+0000 7fd9c641c640 1 --2- 172.21.15.18:0/2273689046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c01013d0 0x7fd9c0107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.284+0000 7fd9c641c640 1 --2- 172.21.15.18:0/2273689046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9c0100a10 0x7fd9c0100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.284+0000 7fd9c641c640 1 --2- 172.21.15.18:0/2273689046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c00ff820 0x7fd9c00ffc40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.284+0000 7fd9c641c640 1 -- 172.21.15.18:0/2273689046 >> 172.21.15.18:0/2273689046 conn(0x7fd9c00fadb0 msgr2=0x7fd9c00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:04.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.284+0000 7fd9c641c640 1 -- 172.21.15.18:0/2273689046 shutdown_connections 2024-02-17T12:49:04.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.284+0000 7fd9c641c640 1 -- 172.21.15.18:0/2273689046 wait complete. 2024-02-17T12:49:04.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.285+0000 7fd9c641c640 1 Processor -- start 2024-02-17T12:49:04.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.285+0000 7fd9c641c640 1 -- start start 2024-02-17T12:49:04.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.285+0000 7fd9c641c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c00ff820 0x7fd9c01037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.285+0000 7fd9c641c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9c0100a10 0x7fd9c0101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.285+0000 7fd9c641c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c01013d0 0x7fd9c0102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.285+0000 7fd9c641c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0102a90 con 0x7fd9c00ff820 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.285+0000 7fd9c641c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0102c00 con 0x7fd9c01013d0 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.285+0000 7fd9c641c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c01a12f0 con 0x7fd9c0100a10 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.285+0000 7fd9bffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c00ff820 0x7fd9c01037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.285+0000 7fd9bffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c00ff820 0x7fd9c01037b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53188/0 (socket says 172.21.15.18:53188) 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.285+0000 7fd9bffff640 1 -- 172.21.15.18:0/3571130197 learned_addr learned my addr 172.21.15.18:0/3571130197 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.285+0000 7fd9bf7fe640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9c0100a10 0x7fd9c0101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.286+0000 7fd9c49b9640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c01013d0 0x7fd9c0102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.286+0000 7fd9bffff640 1 -- 172.21.15.18:0/3571130197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9c0100a10 msgr2=0x7fd9c0101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.286+0000 7fd9bffff640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9c0100a10 0x7fd9c0101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.286+0000 7fd9bffff640 1 -- 172.21.15.18:0/3571130197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c01013d0 msgr2=0x7fd9c0102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.286+0000 7fd9bffff640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c01013d0 0x7fd9c0102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.286+0000 7fd9bffff640 1 -- 172.21.15.18:0/3571130197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b4009560 con 0x7fd9c00ff820 2024-02-17T12:49:04.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.286+0000 7fd9bffff640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c00ff820 0x7fd9c01037b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4000c00 tx=0x7fd9b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:04.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.286+0000 7fd9bd7fa640 1 -- 172.21.15.18:0/3571130197 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4004aa0 con 0x7fd9c00ff820 2024-02-17T12:49:04.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.286+0000 7fd9bd7fa640 1 -- 172.21.15.18:0/3571130197 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9b40052d0 con 0x7fd9c00ff820 2024-02-17T12:49:04.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.286+0000 7fd9c641c640 1 -- 172.21.15.18:0/3571130197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c01a1540 con 0x7fd9c00ff820 2024-02-17T12:49:04.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.286+0000 7fd9bd7fa640 1 -- 172.21.15.18:0/3571130197 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4003e60 con 0x7fd9c00ff820 2024-02-17T12:49:04.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.286+0000 7fd9c641c640 1 -- 172.21.15.18:0/3571130197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c01a1a50 con 0x7fd9c00ff820 2024-02-17T12:49:04.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.287+0000 7fd9c641c640 1 -- 172.21.15.18:0/3571130197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c010bac0 con 0x7fd9c00ff820 2024-02-17T12:49:04.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.288+0000 7fd9bd7fa640 1 -- 172.21.15.18:0/3571130197 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fd9b4005440 con 0x7fd9c00ff820 2024-02-17T12:49:04.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.288+0000 7fd9bd7fa640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd9a80a2150 0x7fd9a80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:04.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.289+0000 7fd9bf7fe640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd9a80a2150 0x7fd9a80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:04.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.289+0000 7fd9bd7fa640 1 -- 172.21.15.18:0/3571130197 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd9b40f0250 con 0x7fd9c00ff820 2024-02-17T12:49:04.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.289+0000 7fd9bf7fe640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd9a80a2150 0x7fd9a80a4610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0103520 tx=0x7fd9b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:04.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.293+0000 7fd9bd7fa640 1 -- 172.21.15.18:0/3571130197 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd9b40b9200 con 0x7fd9c00ff820 2024-02-17T12:49:04.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.477+0000 7fd9c641c640 1 -- 172.21.15.18:0/3571130197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9c01a1ec0 con 0x7fd9c00ff820 2024-02-17T12:49:04.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.478+0000 7fd9bd7fa640 1 -- 172.21.15.18:0/3571130197 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9b40bc8c0 con 0x7fd9c00ff820 2024-02-17T12:49:04.471 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-02-17T12:49:04.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.480+0000 7fd9c641c640 1 -- 172.21.15.18:0/3571130197 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd9a80a2150 msgr2=0x7fd9a80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.480+0000 7fd9c641c640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd9a80a2150 0x7fd9a80a4610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0103520 tx=0x7fd9b0005f90 comp rx=0 tx=0).stop 2024-02-17T12:49:04.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.480+0000 7fd9c641c640 1 -- 172.21.15.18:0/3571130197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c00ff820 msgr2=0x7fd9c01037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.480+0000 7fd9c641c640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c00ff820 0x7fd9c01037b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4000c00 tx=0x7fd9b4004270 comp rx=0 tx=0).stop 2024-02-17T12:49:04.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.480+0000 7fd9c641c640 1 -- 172.21.15.18:0/3571130197 shutdown_connections 2024-02-17T12:49:04.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.480+0000 7fd9c641c640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd9a80a2150 0x7fd9a80a4610 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.480+0000 7fd9c641c640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c01013d0 0x7fd9c0102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.480+0000 7fd9c641c640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9c0100a10 0x7fd9c0101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.480+0000 7fd9c641c640 1 --2- 172.21.15.18:0/3571130197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c00ff820 0x7fd9c01037b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.480+0000 7fd9c641c640 1 -- 172.21.15.18:0/3571130197 >> 172.21.15.18:0/3571130197 conn(0x7fd9c00fadb0 msgr2=0x7fd9c00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:04.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.480+0000 7fd9c641c640 1 -- 172.21.15.18:0/3571130197 shutdown_connections 2024-02-17T12:49:04.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.480+0000 7fd9c641c640 1 -- 172.21.15.18:0/3571130197 wait complete. 2024-02-17T12:49:04.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:04 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3459821154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:04.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:04 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3136305180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:04.483 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T12:49:04.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.596+0000 7fb64f52e640 1 -- 172.21.15.18:0/4082193620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb648101c50 msgr2=0x7fb6481084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.596+0000 7fb64f52e640 1 --2- 172.21.15.18:0/4082193620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb648101c50 0x7fb6481084e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb640009ca0 tx=0x7fb64002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.596+0000 7fb64f52e640 1 -- 172.21.15.18:0/4082193620 shutdown_connections 2024-02-17T12:49:04.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.596+0000 7fb64f52e640 1 --2- 172.21.15.18:0/4082193620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb648101c50 0x7fb6481084e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.596+0000 7fb64f52e640 1 --2- 172.21.15.18:0/4082193620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb648101290 0x7fb648101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.596+0000 7fb64f52e640 1 --2- 172.21.15.18:0/4082193620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6481000a0 0x7fb6481004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.596+0000 7fb64f52e640 1 -- 172.21.15.18:0/4082193620 >> 172.21.15.18:0/4082193620 conn(0x7fb6480fb630 msgr2=0x7fb6480fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:04.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.596+0000 7fb64f52e640 1 -- 172.21.15.18:0/4082193620 shutdown_connections 2024-02-17T12:49:04.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.596+0000 7fb64f52e640 1 -- 172.21.15.18:0/4082193620 wait complete. 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64f52e640 1 Processor -- start 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64f52e640 1 -- start start 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64f52e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6481000a0 0x7fb6481993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64f52e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb648101290 0x7fb648199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64f52e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb648101c50 0x7fb6481a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64f52e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481a0ee0 con 0x7fb648101290 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64f52e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481a1050 con 0x7fb648101c50 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64f52e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481a1210 con 0x7fb6481000a0 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64cac9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb648101290 0x7fb648199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64cac9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb648101290 0x7fb648199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53200/0 (socket says 172.21.15.18:53200) 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64cac9640 1 -- 172.21.15.18:0/3178678991 learned_addr learned my addr 172.21.15.18:0/3178678991 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64d2ca640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6481000a0 0x7fb6481993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64dacb640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb648101c50 0x7fb6481a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:04.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64cac9640 1 -- 172.21.15.18:0/3178678991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6481000a0 msgr2=0x7fb6481993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64cac9640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6481000a0 0x7fb6481993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64cac9640 1 -- 172.21.15.18:0/3178678991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb648101c50 msgr2=0x7fb6481a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64cac9640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb648101c50 0x7fb6481a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64cac9640 1 -- 172.21.15.18:0/3178678991 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb640009900 con 0x7fb648101290 2024-02-17T12:49:04.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.597+0000 7fb64cac9640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb648101290 0x7fb648199930 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e9f0 tx=0x7fb63800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:04.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.598+0000 7fb63e7fc640 1 -- 172.21.15.18:0/3178678991 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638009720 con 0x7fb648101290 2024-02-17T12:49:04.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.598+0000 7fb63e7fc640 1 -- 172.21.15.18:0/3178678991 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb63800cd80 con 0x7fb648101290 2024-02-17T12:49:04.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.598+0000 7fb64f52e640 1 -- 172.21.15.18:0/3178678991 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6481a14f0 con 0x7fb648101290 2024-02-17T12:49:04.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.598+0000 7fb63e7fc640 1 -- 172.21.15.18:0/3178678991 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638010690 con 0x7fb648101290 2024-02-17T12:49:04.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.598+0000 7fb64f52e640 1 -- 172.21.15.18:0/3178678991 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6481a19b0 con 0x7fb648101290 2024-02-17T12:49:04.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.599+0000 7fb64f52e640 1 -- 172.21.15.18:0/3178678991 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64810bab0 con 0x7fb648101290 2024-02-17T12:49:04.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.600+0000 7fb63e7fc640 1 -- 172.21.15.18:0/3178678991 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fb6380046c0 con 0x7fb648101290 2024-02-17T12:49:04.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.600+0000 7fb63e7fc640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb6340a2150 0x7fb6340a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:04.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.600+0000 7fb64d2ca640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb6340a2150 0x7fb6340a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:04.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.600+0000 7fb63e7fc640 1 -- 172.21.15.18:0/3178678991 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb638014070 con 0x7fb648101290 2024-02-17T12:49:04.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.601+0000 7fb64d2ca640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb6340a2150 0x7fb6340a4610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb6481010f0 tx=0x7fb644005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:04.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.604+0000 7fb63e7fc640 1 -- 172.21.15.18:0/3178678991 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb6380972f0 con 0x7fb648101290 2024-02-17T12:49:04.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:04 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3459821154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:04.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:04 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3136305180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:04.745 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:04 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3459821154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:04.745 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:04 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3136305180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:04.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.785+0000 7fb64f52e640 1 -- 172.21.15.18:0/3178678991 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6481a1ed0 con 0x7fb648101290 2024-02-17T12:49:04.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.785+0000 7fb63e7fc640 1 -- 172.21.15.18:0/3178678991 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb63809a9b0 con 0x7fb648101290 2024-02-17T12:49:04.778 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-02-17T12:49:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.787+0000 7fb64f52e640 1 -- 172.21.15.18:0/3178678991 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb6340a2150 msgr2=0x7fb6340a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.787+0000 7fb64f52e640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb6340a2150 0x7fb6340a4610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb6481010f0 tx=0x7fb644005f90 comp rx=0 tx=0).stop 2024-02-17T12:49:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.787+0000 7fb64f52e640 1 -- 172.21.15.18:0/3178678991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb648101290 msgr2=0x7fb648199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.787+0000 7fb64f52e640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb648101290 0x7fb648199930 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e9f0 tx=0x7fb63800edf0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.787+0000 7fb64f52e640 1 -- 172.21.15.18:0/3178678991 shutdown_connections 2024-02-17T12:49:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.787+0000 7fb64f52e640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb6340a2150 0x7fb6340a4610 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.787+0000 7fb64f52e640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb648101c50 0x7fb6481a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.787+0000 7fb64f52e640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb648101290 0x7fb648199930 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.787+0000 7fb64f52e640 1 --2- 172.21.15.18:0/3178678991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6481000a0 0x7fb6481993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.787+0000 7fb64f52e640 1 -- 172.21.15.18:0/3178678991 >> 172.21.15.18:0/3178678991 conn(0x7fb6480fb630 msgr2=0x7fb648069790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:04.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.787+0000 7fb64f52e640 1 -- 172.21.15.18:0/3178678991 shutdown_connections 2024-02-17T12:49:04.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.787+0000 7fb64f52e640 1 -- 172.21.15.18:0/3178678991 wait complete. 2024-02-17T12:49:04.790 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:49:04.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.901+0000 7f662984e640 1 -- 172.21.15.18:0/2347822445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624100120 msgr2=0x7f6624100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.901+0000 7f662984e640 1 --2- 172.21.15.18:0/2347822445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624100120 0x7f6624100540 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6614009900 tx=0x7f661402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.901+0000 7f662984e640 1 -- 172.21.15.18:0/2347822445 shutdown_connections 2024-02-17T12:49:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.901+0000 7f662984e640 1 --2- 172.21.15.18:0/2347822445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6624101cd0 0x7f6624108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.901+0000 7f662984e640 1 --2- 172.21.15.18:0/2347822445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6624101310 0x7f6624101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.901+0000 7f662984e640 1 --2- 172.21.15.18:0/2347822445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624100120 0x7f6624100540 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.901+0000 7f662984e640 1 -- 172.21.15.18:0/2347822445 >> 172.21.15.18:0/2347822445 conn(0x7f66240fb6b0 msgr2=0x7f66240fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.901+0000 7f662984e640 1 -- 172.21.15.18:0/2347822445 shutdown_connections 2024-02-17T12:49:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.901+0000 7f662984e640 1 -- 172.21.15.18:0/2347822445 wait complete. 2024-02-17T12:49:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f662984e640 1 Processor -- start 2024-02-17T12:49:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f662984e640 1 -- start start 2024-02-17T12:49:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f662984e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624100120 0x7f66241972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f662984e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6624101310 0x7f6624197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f662984e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6624101cd0 0x7f662419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f662984e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662419ede0 con 0x7f6624100120 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f662984e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662419ef50 con 0x7f6624101cd0 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f662984e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662419f110 con 0x7f6624101310 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f6622ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624100120 0x7f66241972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f6622ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624100120 0x7f66241972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53222/0 (socket says 172.21.15.18:53222) 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f6622ffd640 1 -- 172.21.15.18:0/3220677946 learned_addr learned my addr 172.21.15.18:0/3220677946 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f661bfff640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6624101310 0x7f6624197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f6622ffd640 1 -- 172.21.15.18:0/3220677946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6624101310 msgr2=0x7f6624197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f6622ffd640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6624101310 0x7f6624197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f66237fe640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6624101cd0 0x7f662419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f6622ffd640 1 -- 172.21.15.18:0/3220677946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6624101cd0 msgr2=0x7f662419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f6622ffd640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6624101cd0 0x7f662419e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.902+0000 7f6622ffd640 1 -- 172.21.15.18:0/3220677946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614009560 con 0x7f6624100120 2024-02-17T12:49:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.903+0000 7f6622ffd640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624100120 0x7f66241972f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6614036040 tx=0x7f6614005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.903+0000 7f662884c640 1 -- 172.21.15.18:0/3220677946 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614004410 con 0x7f6624100120 2024-02-17T12:49:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.903+0000 7f662884c640 1 -- 172.21.15.18:0/3220677946 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66140052d0 con 0x7f6624100120 2024-02-17T12:49:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.903+0000 7f662884c640 1 -- 172.21.15.18:0/3220677946 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614003d70 con 0x7f6624100120 2024-02-17T12:49:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.903+0000 7f662984e640 1 -- 172.21.15.18:0/3220677946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662419f390 con 0x7f6624100120 2024-02-17T12:49:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.903+0000 7f662984e640 1 -- 172.21.15.18:0/3220677946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662419f8f0 con 0x7f6624100120 2024-02-17T12:49:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.904+0000 7f662984e640 1 -- 172.21.15.18:0/3220677946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662410bac0 con 0x7f6624100120 2024-02-17T12:49:04.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.905+0000 7f662884c640 1 -- 172.21.15.18:0/3220677946 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f6614005440 con 0x7f6624100120 2024-02-17T12:49:04.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.905+0000 7f662884c640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f66100a21d0 0x7f66100a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:04.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.906+0000 7f661bfff640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f66100a21d0 0x7f66100a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:04.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.906+0000 7f662884c640 1 -- 172.21.15.18:0/3220677946 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f66140efe30 con 0x7f6624100120 2024-02-17T12:49:04.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.906+0000 7f661bfff640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f66100a21d0 0x7f66100a4690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6624198800 tx=0x7f660c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:04.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:04.910+0000 7f662884c640 1 -- 172.21.15.18:0/3220677946 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f66140b8d80 con 0x7f6624100120 2024-02-17T12:49:05.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.131+0000 7f662984e640 1 -- 172.21.15.18:0/3220677946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f662410bca0 con 0x7f6624100120 2024-02-17T12:49:05.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.132+0000 7f662884c640 1 -- 172.21.15.18:0/3220677946 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11392 (secure 0 0 0) 0x7f66140bc440 con 0x7f6624100120 2024-02-17T12:49:05.124 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:05.124 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:47:51.719041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:49:05.125 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-02-17T12:49:05.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.134+0000 7f662984e640 1 -- 172.21.15.18:0/3220677946 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f66100a21d0 msgr2=0x7f66100a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.134+0000 7f662984e640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f66100a21d0 0x7f66100a4690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6624198800 tx=0x7f660c005f90 comp rx=0 tx=0).stop 2024-02-17T12:49:05.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.134+0000 7f662984e640 1 -- 172.21.15.18:0/3220677946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624100120 msgr2=0x7f66241972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.134+0000 7f662984e640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624100120 0x7f66241972f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6614036040 tx=0x7f6614005dd0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.134+0000 7f662984e640 1 -- 172.21.15.18:0/3220677946 shutdown_connections 2024-02-17T12:49:05.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.134+0000 7f662984e640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f66100a21d0 0x7f66100a4690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.134+0000 7f662984e640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6624101cd0 0x7f662419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.134+0000 7f662984e640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6624101310 0x7f6624197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.134+0000 7f662984e640 1 --2- 172.21.15.18:0/3220677946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624100120 0x7f66241972f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.134+0000 7f662984e640 1 -- 172.21.15.18:0/3220677946 >> 172.21.15.18:0/3220677946 conn(0x7f66240fb6b0 msgr2=0x7f66240fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:05.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.134+0000 7f662984e640 1 -- 172.21.15.18:0/3220677946 shutdown_connections 2024-02-17T12:49:05.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.134+0000 7f662984e640 1 -- 172.21.15.18:0/3220677946 wait complete. 2024-02-17T12:49:05.138 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:49:05.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.248+0000 7fc449548640 1 -- 172.21.15.18:0/2493703614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4441023c0 msgr2=0x7fc444102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.248+0000 7fc449548640 1 --2- 172.21.15.18:0/2493703614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4441023c0 0x7fc444102840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009900 tx=0x7fc42c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:05.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.248+0000 7fc449548640 1 -- 172.21.15.18:0/2493703614 shutdown_connections 2024-02-17T12:49:05.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.248+0000 7fc449548640 1 --2- 172.21.15.18:0/2493703614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc444102d80 0x7fc444109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.248+0000 7fc449548640 1 --2- 172.21.15.18:0/2493703614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4441023c0 0x7fc444102840 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.248+0000 7fc449548640 1 --2- 172.21.15.18:0/2493703614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4441011d0 0x7fc4441015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.248+0000 7fc449548640 1 -- 172.21.15.18:0/2493703614 >> 172.21.15.18:0/2493703614 conn(0x7fc4440fc720 msgr2=0x7fc4440feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.248+0000 7fc449548640 1 -- 172.21.15.18:0/2493703614 shutdown_connections 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.248+0000 7fc449548640 1 -- 172.21.15.18:0/2493703614 wait complete. 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc449548640 1 Processor -- start 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc449548640 1 -- start start 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc449548640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4441011d0 0x7fc444197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc449548640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4441023c0 0x7fc444197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc449548640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc444102d80 0x7fc44419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc449548640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44419ed00 con 0x7fc4441011d0 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc449548640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44419ee70 con 0x7fc4441023c0 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc449548640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44419f030 con 0x7fc444102d80 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc442ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4441011d0 0x7fc444197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:05.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc442ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4441011d0 0x7fc444197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53240/0 (socket says 172.21.15.18:53240) 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc442ffd640 1 -- 172.21.15.18:0/1626119038 learned_addr learned my addr 172.21.15.18:0/1626119038 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc4437fe640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc444102d80 0x7fc44419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.249+0000 7fc4427fc640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4441023c0 0x7fc444197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.250+0000 7fc442ffd640 1 -- 172.21.15.18:0/1626119038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc444102d80 msgr2=0x7fc44419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.250+0000 7fc442ffd640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc444102d80 0x7fc44419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.250+0000 7fc442ffd640 1 -- 172.21.15.18:0/1626119038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4441023c0 msgr2=0x7fc444197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.250+0000 7fc442ffd640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4441023c0 0x7fc444197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.250+0000 7fc442ffd640 1 -- 172.21.15.18:0/1626119038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c009560 con 0x7fc4441011d0 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.250+0000 7fc4427fc640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4441023c0 0x7fc444197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.250+0000 7fc442ffd640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4441011d0 0x7fc444197210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00d770 tx=0x7fc43c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.250+0000 7fc43bfff640 1 -- 172.21.15.18:0/1626119038 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c009c30 con 0x7fc4441011d0 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.250+0000 7fc43bfff640 1 -- 172.21.15.18:0/1626119038 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc43c00de80 con 0x7fc4441011d0 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.250+0000 7fc43bfff640 1 -- 172.21.15.18:0/1626119038 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c010460 con 0x7fc4441011d0 2024-02-17T12:49:05.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.250+0000 7fc449548640 1 -- 172.21.15.18:0/1626119038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44419f310 con 0x7fc4441011d0 2024-02-17T12:49:05.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.250+0000 7fc449548640 1 -- 172.21.15.18:0/1626119038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44419f7d0 con 0x7fc4441011d0 2024-02-17T12:49:05.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.251+0000 7fc449548640 1 -- 172.21.15.18:0/1626119038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4441084f0 con 0x7fc4441011d0 2024-02-17T12:49:05.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.252+0000 7fc43bfff640 1 -- 172.21.15.18:0/1626119038 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fc43c003f60 con 0x7fc4441011d0 2024-02-17T12:49:05.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.252+0000 7fc43bfff640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc4300a21d0 0x7fc4300a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:05.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.253+0000 7fc4427fc640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc4300a21d0 0x7fc4300a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:05.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.253+0000 7fc43bfff640 1 -- 172.21.15.18:0/1626119038 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc43c0cdaa0 con 0x7fc4441011d0 2024-02-17T12:49:05.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.253+0000 7fc4427fc640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc4300a21d0 0x7fc4300a4690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc444198720 tx=0x7fc42c005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:05.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.257+0000 7fc43bfff640 1 -- 172.21.15.18:0/1626119038 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc43c096940 con 0x7fc4441011d0 2024-02-17T12:49:05.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.440+0000 7fc449548640 1 -- 172.21.15.18:0/1626119038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc44410ce50 con 0x7fc4441011d0 2024-02-17T12:49:05.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.441+0000 7fc43bfff640 1 -- 172.21.15.18:0/1626119038 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22607 (secure 0 0 0) 0x7fc43c09a000 con 0x7fc4441011d0 2024-02-17T12:49:05.434 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:05.434 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":88,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:48:52.149237+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:49:05.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.443+0000 7fc449548640 1 -- 172.21.15.18:0/1626119038 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc4300a21d0 msgr2=0x7fc4300a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.443+0000 7fc449548640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc4300a21d0 0x7fc4300a4690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc444198720 tx=0x7fc42c005be0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.443+0000 7fc449548640 1 -- 172.21.15.18:0/1626119038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4441011d0 msgr2=0x7fc444197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.443+0000 7fc449548640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4441011d0 0x7fc444197210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00d770 tx=0x7fc43c00dc30 comp rx=0 tx=0).stop 2024-02-17T12:49:05.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.443+0000 7fc449548640 1 -- 172.21.15.18:0/1626119038 shutdown_connections 2024-02-17T12:49:05.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.443+0000 7fc449548640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc4300a21d0 0x7fc4300a4690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.443+0000 7fc449548640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc444102d80 0x7fc44419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.443+0000 7fc449548640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4441023c0 0x7fc444197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.443+0000 7fc449548640 1 --2- 172.21.15.18:0/1626119038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4441011d0 0x7fc444197210 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.443+0000 7fc449548640 1 -- 172.21.15.18:0/1626119038 >> 172.21.15.18:0/1626119038 conn(0x7fc4440fc720 msgr2=0x7fc4441056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:05.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.444+0000 7fc449548640 1 -- 172.21.15.18:0/1626119038 shutdown_connections 2024-02-17T12:49:05.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.444+0000 7fc449548640 1 -- 172.21.15.18:0/1626119038 wait complete. 2024-02-17T12:49:05.447 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T12:43:12.450140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T12:47:43.690543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T12:47:44.653229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T12:49:05.448 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T12:49:05.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.561+0000 7f42a08f4640 1 -- 172.21.15.18:0/4109093569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f429c104550 msgr2=0x7f429c106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.561+0000 7f42a08f4640 1 --2- 172.21.15.18:0/4109093569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f429c104550 0x7f429c106940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4284009900 tx=0x7f428402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:05.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.561+0000 7f42a08f4640 1 -- 172.21.15.18:0/4109093569 shutdown_connections 2024-02-17T12:49:05.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.561+0000 7f42a08f4640 1 --2- 172.21.15.18:0/4109093569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f429c106e80 0x7f429c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.561+0000 7f42a08f4640 1 --2- 172.21.15.18:0/4109093569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f429c104550 0x7f429c106940 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.561+0000 7f42a08f4640 1 --2- 172.21.15.18:0/4109093569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f429c101c20 0x7f429c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.561+0000 7f42a08f4640 1 -- 172.21.15.18:0/4109093569 >> 172.21.15.18:0/4109093569 conn(0x7f429c0fb630 msgr2=0x7f429c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:05.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.561+0000 7f42a08f4640 1 -- 172.21.15.18:0/4109093569 shutdown_connections 2024-02-17T12:49:05.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.561+0000 7f42a08f4640 1 -- 172.21.15.18:0/4109093569 wait complete. 2024-02-17T12:49:05.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f42a08f4640 1 Processor -- start 2024-02-17T12:49:05.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f42a08f4640 1 -- start start 2024-02-17T12:49:05.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f42a08f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f429c101c20 0x7f429c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:05.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f42a08f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f429c104550 0x7f429c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:05.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f42a08f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f429c106e80 0x7f429c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:05.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f42a08f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c19ecc0 con 0x7f429c106e80 2024-02-17T12:49:05.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f42a08f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c19ee30 con 0x7f429c104550 2024-02-17T12:49:05.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f42a08f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c19eff0 con 0x7f429c101c20 2024-02-17T12:49:05.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f429ad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f429c106e80 0x7f429c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:05.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f429ad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f429c106e80 0x7f429c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53268/0 (socket says 172.21.15.18:53268) 2024-02-17T12:49:05.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f429ad9d640 1 -- 172.21.15.18:0/1717351623 learned_addr learned my addr 172.21.15.18:0/1717351623 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:05.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f429a59c640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f429c101c20 0x7f429c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:05.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f4293fff640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f429c104550 0x7f429c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:05.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f429ad9d640 1 -- 172.21.15.18:0/1717351623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f429c101c20 msgr2=0x7f429c1971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f429ad9d640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f429c101c20 0x7f429c1971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f429ad9d640 1 -- 172.21.15.18:0/1717351623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f429c104550 msgr2=0x7f429c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f429ad9d640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f429c104550 0x7f429c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f429ad9d640 1 -- 172.21.15.18:0/1717351623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4284009560 con 0x7f429c106e80 2024-02-17T12:49:05.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f4293fff640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f429c104550 0x7f429c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:05.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.562+0000 7f429ad9d640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f429c106e80 0x7f429c19e780 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f429400eb10 tx=0x7f429400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:05.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.563+0000 7f42937fe640 1 -- 172.21.15.18:0/1717351623 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400ce20 con 0x7f429c106e80 2024-02-17T12:49:05.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.563+0000 7f42937fe640 1 -- 172.21.15.18:0/1717351623 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4294005020 con 0x7f429c106e80 2024-02-17T12:49:05.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.563+0000 7f42a08f4640 1 -- 172.21.15.18:0/1717351623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429c19f2d0 con 0x7f429c106e80 2024-02-17T12:49:05.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.563+0000 7f42937fe640 1 -- 172.21.15.18:0/1717351623 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294010690 con 0x7f429c106e80 2024-02-17T12:49:05.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.563+0000 7f42a08f4640 1 -- 172.21.15.18:0/1717351623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429c19f7e0 con 0x7f429c106e80 2024-02-17T12:49:05.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.564+0000 7f42a08f4640 1 -- 172.21.15.18:0/1717351623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429c0fd120 con 0x7f429c106e80 2024-02-17T12:49:05.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.565+0000 7f42937fe640 1 -- 172.21.15.18:0/1717351623 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4294005500 con 0x7f429c106e80 2024-02-17T12:49:05.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.565+0000 7f42937fe640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f42880a2150 0x7f42880a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:05.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.565+0000 7f429a59c640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f42880a2150 0x7f42880a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:05.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.565+0000 7f42937fe640 1 -- 172.21.15.18:0/1717351623 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4294014070 con 0x7f429c106e80 2024-02-17T12:49:05.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.566+0000 7f429a59c640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f42880a2150 0x7f42880a4610 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f428c00abe0 tx=0x7f428c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:05.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.570+0000 7f42937fe640 1 -- 172.21.15.18:0/1717351623 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4294097330 con 0x7f429c106e80 2024-02-17T12:49:05.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:05 smithi146 ceph-mon[36454]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s; 84 B/s, 0 keys/s, 1 objects/s recovering 2024-02-17T12:49:05.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:05 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3571130197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:05.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:05 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3178678991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:05.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:05 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3220677946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:05.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.749+0000 7f42a08f4640 1 -- 172.21.15.18:0/1717351623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f429c02ce40 con 0x7f429c106e80 2024-02-17T12:49:05.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.749+0000 7f42937fe640 1 -- 172.21.15.18:0/1717351623 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4294005190 con 0x7f429c106e80 2024-02-17T12:49:05.741 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-02-17T12:49:05.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.751+0000 7f42a08f4640 1 -- 172.21.15.18:0/1717351623 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f42880a2150 msgr2=0x7f42880a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.751+0000 7f42a08f4640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f42880a2150 0x7f42880a4610 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f428c00abe0 tx=0x7f428c00a2d0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.751+0000 7f42a08f4640 1 -- 172.21.15.18:0/1717351623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f429c106e80 msgr2=0x7f429c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.751+0000 7f42a08f4640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f429c106e80 0x7f429c19e780 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f429400eb10 tx=0x7f429400ef10 comp rx=0 tx=0).stop 2024-02-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.751+0000 7f42a08f4640 1 -- 172.21.15.18:0/1717351623 shutdown_connections 2024-02-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.751+0000 7f42a08f4640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f42880a2150 0x7f42880a4610 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.751+0000 7f42a08f4640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f429c106e80 0x7f429c19e780 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.751+0000 7f42a08f4640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f429c104550 0x7f429c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.751+0000 7f42a08f4640 1 --2- 172.21.15.18:0/1717351623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f429c101c20 0x7f429c1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.751+0000 7f42a08f4640 1 -- 172.21.15.18:0/1717351623 >> 172.21.15.18:0/1717351623 conn(0x7f429c0fb630 msgr2=0x7f429c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.751+0000 7f42a08f4640 1 -- 172.21.15.18:0/1717351623 shutdown_connections 2024-02-17T12:49:05.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.751+0000 7f42a08f4640 1 -- 172.21.15.18:0/1717351623 wait complete. 2024-02-17T12:49:05.754 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:05 smithi018 ceph-mon[32625]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s; 84 B/s, 0 keys/s, 1 objects/s recovering 2024-02-17T12:49:05.754 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:05 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3571130197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:05.754 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:05 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3178678991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:05.754 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:05 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3220677946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:05.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T12:49:05.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:05 smithi119 ceph-mon[36188]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s; 84 B/s, 0 keys/s, 1 objects/s recovering 2024-02-17T12:49:05.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:05 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3571130197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:05.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:05 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3178678991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:05.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:05 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3220677946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.868+0000 7fdfd7916640 1 -- 172.21.15.18:0/1177952025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfd0100ce0 msgr2=0x7fdfd0108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.868+0000 7fdfd7916640 1 --2- 172.21.15.18:0/1177952025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfd0100ce0 0x7fdfd0108580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009ca0 tx=0x7fdfc802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 -- 172.21.15.18:0/1177952025 shutdown_connections 2024-02-17T12:49:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 --2- 172.21.15.18:0/1177952025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfd0100ce0 0x7fdfd0108580 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 --2- 172.21.15.18:0/1177952025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd0100320 0x7fdfd01007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 --2- 172.21.15.18:0/1177952025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfd0101680 0x7fdfd0101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 -- 172.21.15.18:0/1177952025 >> 172.21.15.18:0/1177952025 conn(0x7fdfd00fad90 msgr2=0x7fdfd00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 -- 172.21.15.18:0/1177952025 shutdown_connections 2024-02-17T12:49:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 -- 172.21.15.18:0/1177952025 wait complete. 2024-02-17T12:49:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 Processor -- start 2024-02-17T12:49:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 -- start start 2024-02-17T12:49:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfd0100320 0x7fdfd01960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd0100ce0 0x7fdfd0196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:05.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfd0101680 0x7fdfd019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:05.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd019dbd0 con 0x7fdfd0100320 2024-02-17T12:49:05.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd019dd40 con 0x7fdfd0100ce0 2024-02-17T12:49:05.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd7916640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd019df00 con 0x7fdfd0101680 2024-02-17T12:49:05.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd56b2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfd0100320 0x7fdfd01960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:05.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd56b2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfd0100320 0x7fdfd01960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53278/0 (socket says 172.21.15.18:53278) 2024-02-17T12:49:05.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.869+0000 7fdfd56b2640 1 -- 172.21.15.18:0/3869641677 learned_addr learned my addr 172.21.15.18:0/3869641677 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:05.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfd5eb3640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfd0101680 0x7fdfd019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfd4eb1640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd0100ce0 0x7fdfd0196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfd56b2640 1 -- 172.21.15.18:0/3869641677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfd0101680 msgr2=0x7fdfd019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfd56b2640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfd0101680 0x7fdfd019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfd56b2640 1 -- 172.21.15.18:0/3869641677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd0100ce0 msgr2=0x7fdfd0196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfd56b2640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd0100ce0 0x7fdfd0196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfd56b2640 1 -- 172.21.15.18:0/3869641677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc8009900 con 0x7fdfd0100320 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfd56b2640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfd0100320 0x7fdfd01960e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00d950 tx=0x7fdfcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfc67fc640 1 -- 172.21.15.18:0/3869641677 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc009c30 con 0x7fdfd0100320 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfd7916640 1 -- 172.21.15.18:0/3869641677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd019e1e0 con 0x7fdfd0100320 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfc67fc640 1 -- 172.21.15.18:0/3869641677 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfcc009d90 con 0x7fdfd0100320 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfc67fc640 1 -- 172.21.15.18:0/3869641677 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc00f720 con 0x7fdfd0100320 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.870+0000 7fdfd7916640 1 -- 172.21.15.18:0/3869641677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd019e6a0 con 0x7fdfd0100320 2024-02-17T12:49:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.871+0000 7fdfd7916640 1 -- 172.21.15.18:0/3869641677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd0107400 con 0x7fdfd0100320 2024-02-17T12:49:05.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.872+0000 7fdfc67fc640 1 -- 172.21.15.18:0/3869641677 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fdfcc010a60 con 0x7fdfd0100320 2024-02-17T12:49:05.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.873+0000 7fdfc67fc640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdfbc0a2150 0x7fdfbc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:05.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.873+0000 7fdfd4eb1640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdfbc0a2150 0x7fdfbc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:05.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.873+0000 7fdfc67fc640 1 -- 172.21.15.18:0/3869641677 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdfcc0cdcd0 con 0x7fdfd0100320 2024-02-17T12:49:05.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.873+0000 7fdfd4eb1640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdfbc0a2150 0x7fdfbc0a4610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0005fd0 tx=0x7fdfc0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:05.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:05.877+0000 7fdfc67fc640 1 -- 172.21.15.18:0/3869641677 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdfcc096b10 con 0x7fdfd0100320 2024-02-17T12:49:06.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.060+0000 7fdfd7916640 1 -- 172.21.15.18:0/3869641677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfd002ce40 con 0x7fdfd0100320 2024-02-17T12:49:06.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.060+0000 7fdfc67fc640 1 -- 172.21.15.18:0/3869641677 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdfcc09a1d0 con 0x7fdfd0100320 2024-02-17T12:49:06.053 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-02-17T12:49:06.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.062+0000 7fdfd7916640 1 -- 172.21.15.18:0/3869641677 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdfbc0a2150 msgr2=0x7fdfbc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.062+0000 7fdfd7916640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdfbc0a2150 0x7fdfbc0a4610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0005fd0 tx=0x7fdfc0005850 comp rx=0 tx=0).stop 2024-02-17T12:49:06.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.062+0000 7fdfd7916640 1 -- 172.21.15.18:0/3869641677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfd0100320 msgr2=0x7fdfd01960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.062+0000 7fdfd7916640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfd0100320 0x7fdfd01960e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00d950 tx=0x7fdfcc00de10 comp rx=0 tx=0).stop 2024-02-17T12:49:06.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.062+0000 7fdfd7916640 1 -- 172.21.15.18:0/3869641677 shutdown_connections 2024-02-17T12:49:06.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.062+0000 7fdfd7916640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdfbc0a2150 0x7fdfbc0a4610 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.062+0000 7fdfd7916640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfd0101680 0x7fdfd019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.062+0000 7fdfd7916640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd0100ce0 0x7fdfd0196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.062+0000 7fdfd7916640 1 --2- 172.21.15.18:0/3869641677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfd0100320 0x7fdfd01960e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.062+0000 7fdfd7916640 1 -- 172.21.15.18:0/3869641677 >> 172.21.15.18:0/3869641677 conn(0x7fdfd00fad90 msgr2=0x7fdfd01045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:06.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.063+0000 7fdfd7916640 1 -- 172.21.15.18:0/3869641677 shutdown_connections 2024-02-17T12:49:06.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.063+0000 7fdfd7916640 1 -- 172.21.15.18:0/3869641677 wait complete. 2024-02-17T12:49:06.067 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T12:49:06.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.180+0000 7f2f4f163640 1 -- 172.21.15.18:0/3643469627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f48106ec0 msgr2=0x7f2f481092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.180+0000 7f2f4f163640 1 --2- 172.21.15.18:0/3643469627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f48106ec0 0x7f2f481092b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009900 tx=0x7f2f3802bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:06.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.180+0000 7f2f4f163640 1 -- 172.21.15.18:0/3643469627 shutdown_connections 2024-02-17T12:49:06.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.180+0000 7f2f4f163640 1 --2- 172.21.15.18:0/3643469627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f48106ec0 0x7f2f481092b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.180+0000 7f2f4f163640 1 --2- 172.21.15.18:0/3643469627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f48104590 0x7f2f48106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.180+0000 7f2f4f163640 1 --2- 172.21.15.18:0/3643469627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f48101c60 0x7f2f48104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.180+0000 7f2f4f163640 1 -- 172.21.15.18:0/3643469627 >> 172.21.15.18:0/3643469627 conn(0x7f2f480fb610 msgr2=0x7f2f480fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:06.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.180+0000 7f2f4f163640 1 -- 172.21.15.18:0/3643469627 shutdown_connections 2024-02-17T12:49:06.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.180+0000 7f2f4f163640 1 -- 172.21.15.18:0/3643469627 wait complete. 2024-02-17T12:49:06.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4f163640 1 Processor -- start 2024-02-17T12:49:06.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4f163640 1 -- start start 2024-02-17T12:49:06.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4f163640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f48101c60 0x7f2f481971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:06.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4f163640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f48104590 0x7f2f481976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:06.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4f163640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f48106ec0 0x7f2f4819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:06.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4f163640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4819ec90 con 0x7f2f48106ec0 2024-02-17T12:49:06.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4f163640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4819ee00 con 0x7f2f48101c60 2024-02-17T12:49:06.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4f163640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4819efc0 con 0x7f2f48104590 2024-02-17T12:49:06.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4d700640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f48106ec0 0x7f2f4819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:06.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4d700640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f48106ec0 0x7f2f4819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53296/0 (socket says 172.21.15.18:53296) 2024-02-17T12:49:06.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4d700640 1 -- 172.21.15.18:0/91409742 learned_addr learned my addr 172.21.15.18:0/91409742 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:06.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f47fff640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f48104590 0x7f2f481976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:06.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4ceff640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f48101c60 0x7f2f481971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:06.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4d700640 1 -- 172.21.15.18:0/91409742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f48104590 msgr2=0x7f2f481976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4d700640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f48104590 0x7f2f481976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4d700640 1 -- 172.21.15.18:0/91409742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f48101c60 msgr2=0x7f2f481971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4d700640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f48101c60 0x7f2f481971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.181+0000 7f2f4d700640 1 -- 172.21.15.18:0/91409742 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f38009560 con 0x7f2f48106ec0 2024-02-17T12:49:06.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.182+0000 7f2f4d700640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f48106ec0 0x7f2f4819e750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009230 tx=0x7f2f38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:06.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.182+0000 7f2f45ffb640 1 -- 172.21.15.18:0/91409742 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38004df0 con 0x7f2f48106ec0 2024-02-17T12:49:06.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.182+0000 7f2f4f163640 1 -- 172.21.15.18:0/91409742 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4819f240 con 0x7f2f48106ec0 2024-02-17T12:49:06.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.182+0000 7f2f45ffb640 1 -- 172.21.15.18:0/91409742 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f38005430 con 0x7f2f48106ec0 2024-02-17T12:49:06.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.182+0000 7f2f45ffb640 1 -- 172.21.15.18:0/91409742 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38003d20 con 0x7f2f48106ec0 2024-02-17T12:49:06.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.182+0000 7f2f4f163640 1 -- 172.21.15.18:0/91409742 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4819f750 con 0x7f2f48106ec0 2024-02-17T12:49:06.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.183+0000 7f2f4f163640 1 -- 172.21.15.18:0/91409742 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f480fd100 con 0x7f2f48106ec0 2024-02-17T12:49:06.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.184+0000 7f2f45ffb640 1 -- 172.21.15.18:0/91409742 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f2f380055a0 con 0x7f2f48106ec0 2024-02-17T12:49:06.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.185+0000 7f2f45ffb640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2f300a2150 0x7f2f300a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:06.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.185+0000 7f2f4ceff640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2f300a2150 0x7f2f300a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:06.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.185+0000 7f2f45ffb640 1 -- 172.21.15.18:0/91409742 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2f380f00f0 con 0x7f2f48106ec0 2024-02-17T12:49:06.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.185+0000 7f2f4ceff640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2f300a2150 0x7f2f300a4610 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2f40005fd0 tx=0x7f2f40009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:06.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.189+0000 7f2f45ffb640 1 -- 172.21.15.18:0/91409742 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f380b9040 con 0x7f2f48106ec0 2024-02-17T12:49:06.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.371+0000 7f2f4f163640 1 -- 172.21.15.18:0/91409742 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f4802cda0 con 0x7f2f48106ec0 2024-02-17T12:49:06.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.372+0000 7f2f45ffb640 1 -- 172.21.15.18:0/91409742 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f380bc700 con 0x7f2f48106ec0 2024-02-17T12:49:06.364 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-02-17T12:49:06.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.374+0000 7f2f4f163640 1 -- 172.21.15.18:0/91409742 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2f300a2150 msgr2=0x7f2f300a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.374+0000 7f2f4f163640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2f300a2150 0x7f2f300a4610 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2f40005fd0 tx=0x7f2f40009870 comp rx=0 tx=0).stop 2024-02-17T12:49:06.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.374+0000 7f2f4f163640 1 -- 172.21.15.18:0/91409742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f48106ec0 msgr2=0x7f2f4819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.374+0000 7f2f4f163640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f48106ec0 0x7f2f4819e750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009230 tx=0x7f2f38004270 comp rx=0 tx=0).stop 2024-02-17T12:49:06.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.374+0000 7f2f4f163640 1 -- 172.21.15.18:0/91409742 shutdown_connections 2024-02-17T12:49:06.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.374+0000 7f2f4f163640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2f300a2150 0x7f2f300a4610 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.374+0000 7f2f4f163640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f48106ec0 0x7f2f4819e750 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.374+0000 7f2f4f163640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f48104590 0x7f2f481976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.374+0000 7f2f4f163640 1 --2- 172.21.15.18:0/91409742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f48101c60 0x7f2f481971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.374+0000 7f2f4f163640 1 -- 172.21.15.18:0/91409742 >> 172.21.15.18:0/91409742 conn(0x7f2f480fb610 msgr2=0x7f2f481025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:06.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.374+0000 7f2f4f163640 1 -- 172.21.15.18:0/91409742 shutdown_connections 2024-02-17T12:49:06.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.374+0000 7f2f4f163640 1 -- 172.21.15.18:0/91409742 wait complete. 2024-02-17T12:49:06.377 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 -- 172.21.15.18:0/2271710132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b5c1027d0 msgr2=0x7f2b5c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 --2- 172.21.15.18:0/2271710132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b5c1027d0 0x7f2b5c102bf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009ca0 tx=0x7f2b5402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 -- 172.21.15.18:0/2271710132 shutdown_connections 2024-02-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 --2- 172.21.15.18:0/2271710132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b5c0ffc00 0x7f2b5c1097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 --2- 172.21.15.18:0/2271710132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b5c0ff240 0x7f2b5c0ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 --2- 172.21.15.18:0/2271710132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b5c1027d0 0x7f2b5c102bf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 -- 172.21.15.18:0/2271710132 >> 172.21.15.18:0/2271710132 conn(0x7f2b5c0fadc0 msgr2=0x7f2b5c0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 -- 172.21.15.18:0/2271710132 shutdown_connections 2024-02-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 -- 172.21.15.18:0/2271710132 wait complete. 2024-02-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 Processor -- start 2024-02-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 -- start start 2024-02-17T12:49:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b5c0ff240 0x7f2b5c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b5c0ffc00 0x7f2b5c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b5c1027d0 0x7f2b5c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c19ed60 con 0x7f2b5c0ff240 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c19eed0 con 0x7f2b5c1027d0 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.491+0000 7f2b63215640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c19f090 con 0x7f2b5c0ffc00 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b60fb1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b5c0ff240 0x7f2b5c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b60fb1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b5c0ff240 0x7f2b5c197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53324/0 (socket says 172.21.15.18:53324) 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b60fb1640 1 -- 172.21.15.18:0/3629138981 learned_addr learned my addr 172.21.15.18:0/3629138981 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b5bfff640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b5c0ffc00 0x7f2b5c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b617b2640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b5c1027d0 0x7f2b5c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b60fb1640 1 -- 172.21.15.18:0/3629138981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b5c0ffc00 msgr2=0x7f2b5c1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b60fb1640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b5c0ffc00 0x7f2b5c1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b60fb1640 1 -- 172.21.15.18:0/3629138981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b5c1027d0 msgr2=0x7f2b5c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b60fb1640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b5c1027d0 0x7f2b5c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b60fb1640 1 -- 172.21.15.18:0/3629138981 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b54009900 con 0x7f2b5c0ff240 2024-02-17T12:49:06.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b617b2640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b5c1027d0 0x7f2b5c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:06.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b60fb1640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b5c0ff240 0x7f2b5c197270 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2b54000f80 tx=0x7f2b54034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:06.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b59ffb640 1 -- 172.21.15.18:0/3629138981 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b540329b0 con 0x7f2b5c0ff240 2024-02-17T12:49:06.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b59ffb640 1 -- 172.21.15.18:0/3629138981 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b54003e50 con 0x7f2b5c0ff240 2024-02-17T12:49:06.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b59ffb640 1 -- 172.21.15.18:0/3629138981 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54040450 con 0x7f2b5c0ff240 2024-02-17T12:49:06.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b63215640 1 -- 172.21.15.18:0/3629138981 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5c19f310 con 0x7f2b5c0ff240 2024-02-17T12:49:06.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.492+0000 7f2b63215640 1 -- 172.21.15.18:0/3629138981 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c19f790 con 0x7f2b5c0ff240 2024-02-17T12:49:06.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.494+0000 7f2b63215640 1 -- 172.21.15.18:0/3629138981 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5c108690 con 0x7f2b5c0ff240 2024-02-17T12:49:06.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.495+0000 7f2b59ffb640 1 -- 172.21.15.18:0/3629138981 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f2b5400a040 con 0x7f2b5c0ff240 2024-02-17T12:49:06.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.495+0000 7f2b59ffb640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2b480a2150 0x7f2b480a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:06.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.496+0000 7f2b5bfff640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2b480a2150 0x7f2b480a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:06.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.496+0000 7f2b59ffb640 1 -- 172.21.15.18:0/3629138981 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2b540f0ef0 con 0x7f2b5c0ff240 2024-02-17T12:49:06.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.496+0000 7f2b5bfff640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2b480a2150 0x7f2b480a4610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c198780 tx=0x7f2b44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:06.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.499+0000 7f2b59ffb640 1 -- 172.21.15.18:0/3629138981 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b540b9fc0 con 0x7f2b5c0ff240 2024-02-17T12:49:06.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:06 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1626119038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:06.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:06 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1717351623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:06.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:06 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3869641677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:06.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:06 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/91409742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:06.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.721+0000 7f2b63215640 1 -- 172.21.15.18:0/3629138981 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b5c02ce40 con 0x7f2b5c0ff240 2024-02-17T12:49:06.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.721+0000 7f2b59ffb640 1 -- 172.21.15.18:0/3629138981 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11392 (secure 0 0 0) 0x7f2b54044090 con 0x7f2b5c0ff240 2024-02-17T12:49:06.714 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:06.714 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:47:51.719041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:49:06.714 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-02-17T12:49:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.724+0000 7f2b63215640 1 -- 172.21.15.18:0/3629138981 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2b480a2150 msgr2=0x7f2b480a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.724+0000 7f2b63215640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2b480a2150 0x7f2b480a4610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c198780 tx=0x7f2b44005f90 comp rx=0 tx=0).stop 2024-02-17T12:49:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.724+0000 7f2b63215640 1 -- 172.21.15.18:0/3629138981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b5c0ff240 msgr2=0x7f2b5c197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.724+0000 7f2b63215640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b5c0ff240 0x7f2b5c197270 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2b54000f80 tx=0x7f2b54034a90 comp rx=0 tx=0).stop 2024-02-17T12:49:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.724+0000 7f2b63215640 1 -- 172.21.15.18:0/3629138981 shutdown_connections 2024-02-17T12:49:06.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.724+0000 7f2b63215640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2b480a2150 0x7f2b480a4610 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.724+0000 7f2b63215640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b5c1027d0 0x7f2b5c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.724+0000 7f2b63215640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b5c0ffc00 0x7f2b5c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.724+0000 7f2b63215640 1 --2- 172.21.15.18:0/3629138981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b5c0ff240 0x7f2b5c197270 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.724+0000 7f2b63215640 1 -- 172.21.15.18:0/3629138981 >> 172.21.15.18:0/3629138981 conn(0x7f2b5c0fadc0 msgr2=0x7f2b5c105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:06.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.724+0000 7f2b63215640 1 -- 172.21.15.18:0/3629138981 shutdown_connections 2024-02-17T12:49:06.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.724+0000 7f2b63215640 1 -- 172.21.15.18:0/3629138981 wait complete. 2024-02-17T12:49:06.726 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:06 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1626119038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:06.726 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:06 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1717351623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:06.726 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:06 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3869641677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:06.726 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:06 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/91409742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:06.728 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:49:06.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.838+0000 7f586d58c640 1 -- 172.21.15.18:0/2194270097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5868102390 msgr2=0x7f5868102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.838+0000 7f586d58c640 1 --2- 172.21.15.18:0/2194270097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5868102390 0x7f5868102810 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5850009900 tx=0x7f585002bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:06.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.838+0000 7f586d58c640 1 -- 172.21.15.18:0/2194270097 shutdown_connections 2024-02-17T12:49:06.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.838+0000 7f586d58c640 1 --2- 172.21.15.18:0/2194270097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5868102d50 0x7f58681095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.838+0000 7f586d58c640 1 --2- 172.21.15.18:0/2194270097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5868102390 0x7f5868102810 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.838+0000 7f586d58c640 1 --2- 172.21.15.18:0/2194270097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58681011a0 0x7f58681015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.838+0000 7f586d58c640 1 -- 172.21.15.18:0/2194270097 >> 172.21.15.18:0/2194270097 conn(0x7f58680fc730 msgr2=0x7f58680feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:06.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.838+0000 7f586d58c640 1 -- 172.21.15.18:0/2194270097 shutdown_connections 2024-02-17T12:49:06.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.838+0000 7f586d58c640 1 -- 172.21.15.18:0/2194270097 wait complete. 2024-02-17T12:49:06.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.838+0000 7f586d58c640 1 Processor -- start 2024-02-17T12:49:06.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f586d58c640 1 -- start start 2024-02-17T12:49:06.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f586d58c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58681011a0 0x7f5868074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f586d58c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5868102390 0x7f5868073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f586d58c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5868102d50 0x7f5868073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f586d58c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5868074210 con 0x7f5868102d50 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f586d58c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5868074380 con 0x7f58681011a0 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f586d58c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58681a1270 con 0x7f5868102390 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f58677fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5868102d50 0x7f5868073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f58677fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5868102d50 0x7f5868073ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53352/0 (socket says 172.21.15.18:53352) 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f58677fe640 1 -- 172.21.15.18:0/103374070 learned_addr learned my addr 172.21.15.18:0/103374070 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f5866ffd640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58681011a0 0x7f5868074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f585e7fc640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5868102390 0x7f5868073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f58677fe640 1 -- 172.21.15.18:0/103374070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5868102390 msgr2=0x7f5868073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f58677fe640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5868102390 0x7f5868073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f58677fe640 1 -- 172.21.15.18:0/103374070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58681011a0 msgr2=0x7f5868074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:06.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f58677fe640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58681011a0 0x7f5868074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:06.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f58677fe640 1 -- 172.21.15.18:0/103374070 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5850009560 con 0x7f5868102d50 2024-02-17T12:49:06.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.839+0000 7f58677fe640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5868102d50 0x7f5868073ab0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f586000ead0 tx=0x7f586000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:06.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.840+0000 7f5864ff9640 1 -- 172.21.15.18:0/103374070 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586000cde0 con 0x7f5868102d50 2024-02-17T12:49:06.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.840+0000 7f5864ff9640 1 -- 172.21.15.18:0/103374070 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5860005020 con 0x7f5868102d50 2024-02-17T12:49:06.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.840+0000 7f586d58c640 1 -- 172.21.15.18:0/103374070 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58681a1520 con 0x7f5868102d50 2024-02-17T12:49:06.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.840+0000 7f5864ff9640 1 -- 172.21.15.18:0/103374070 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58600116b0 con 0x7f5868102d50 2024-02-17T12:49:06.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.840+0000 7f586d58c640 1 -- 172.21.15.18:0/103374070 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58681a1a30 con 0x7f5868102d50 2024-02-17T12:49:06.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.841+0000 7f586d58c640 1 -- 172.21.15.18:0/103374070 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58681084c0 con 0x7f5868102d50 2024-02-17T12:49:06.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.842+0000 7f5864ff9640 1 -- 172.21.15.18:0/103374070 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f5860005520 con 0x7f5868102d50 2024-02-17T12:49:06.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.842+0000 7f5864ff9640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f58540a21d0 0x7f58540a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:06.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.843+0000 7f5866ffd640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f58540a21d0 0x7f58540a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:06.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.843+0000 7f5864ff9640 1 -- 172.21.15.18:0/103374070 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f58600cdfb0 con 0x7f5868102d50 2024-02-17T12:49:06.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.843+0000 7f5866ffd640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f58540a21d0 0x7f58540a4690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f58681021f0 tx=0x7f585800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:06.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:06.847+0000 7f5864ff9640 1 -- 172.21.15.18:0/103374070 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5860101d20 con 0x7f5868102d50 2024-02-17T12:49:06.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:06 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1626119038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:06.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:06 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1717351623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:06.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:06 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3869641677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:06.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:06 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/91409742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:07.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.030+0000 7f586d58c640 1 -- 172.21.15.18:0/103374070 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f586802ce40 con 0x7f5868102d50 2024-02-17T12:49:07.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.032+0000 7f5864ff9640 1 -- 172.21.15.18:0/103374070 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22607 (secure 0 0 0) 0x7f586001f020 con 0x7f5868102d50 2024-02-17T12:49:07.024 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:07.025 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":88,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:48:52.149237+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:49:07.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.034+0000 7f586d58c640 1 -- 172.21.15.18:0/103374070 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f58540a21d0 msgr2=0x7f58540a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.034+0000 7f586d58c640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f58540a21d0 0x7f58540a4690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f58681021f0 tx=0x7f585800a2d0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.034+0000 7f586d58c640 1 -- 172.21.15.18:0/103374070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5868102d50 msgr2=0x7f5868073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.034+0000 7f586d58c640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5868102d50 0x7f5868073ab0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f586000ead0 tx=0x7f586000ef90 comp rx=0 tx=0).stop 2024-02-17T12:49:07.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.034+0000 7f586d58c640 1 -- 172.21.15.18:0/103374070 shutdown_connections 2024-02-17T12:49:07.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.034+0000 7f586d58c640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f58540a21d0 0x7f58540a4690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.034+0000 7f586d58c640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5868102d50 0x7f5868073ab0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.034+0000 7f586d58c640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5868102390 0x7f5868073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.034+0000 7f586d58c640 1 --2- 172.21.15.18:0/103374070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58681011a0 0x7f5868074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.034+0000 7f586d58c640 1 -- 172.21.15.18:0/103374070 >> 172.21.15.18:0/103374070 conn(0x7f58680fc730 msgr2=0x7f5868105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.034+0000 7f586d58c640 1 -- 172.21.15.18:0/103374070 shutdown_connections 2024-02-17T12:49:07.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.034+0000 7f586d58c640 1 -- 172.21.15.18:0/103374070 wait complete. 2024-02-17T12:49:07.038 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:49:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.148+0000 7f00d9688640 1 -- 172.21.15.18:0/3954258059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d41012b0 msgr2=0x7f00d4101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.148+0000 7f00d9688640 1 --2- 172.21.15.18:0/3954258059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d41012b0 0x7f00d4101730 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009900 tx=0x7f00bc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.148+0000 7f00d9688640 1 -- 172.21.15.18:0/3954258059 shutdown_connections 2024-02-17T12:49:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.148+0000 7f00d9688640 1 --2- 172.21.15.18:0/3954258059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4101c70 0x7f00d4108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.148+0000 7f00d9688640 1 --2- 172.21.15.18:0/3954258059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d41012b0 0x7f00d4101730 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.148+0000 7f00d9688640 1 --2- 172.21.15.18:0/3954258059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d41000c0 0x7f00d41004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.148+0000 7f00d9688640 1 -- 172.21.15.18:0/3954258059 >> 172.21.15.18:0/3954258059 conn(0x7f00d40fb630 msgr2=0x7f00d40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.148+0000 7f00d9688640 1 -- 172.21.15.18:0/3954258059 shutdown_connections 2024-02-17T12:49:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.148+0000 7f00d9688640 1 -- 172.21.15.18:0/3954258059 wait complete. 2024-02-17T12:49:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d9688640 1 Processor -- start 2024-02-17T12:49:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d9688640 1 -- start start 2024-02-17T12:49:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d9688640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d41000c0 0x7f00d41960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d9688640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d41012b0 0x7f00d4196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d9688640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d4101c70 0x7f00d419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d9688640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d419dbb0 con 0x7f00d4101c70 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d9688640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d419dd20 con 0x7f00d41000c0 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d9688640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d419dee0 con 0x7f00d41012b0 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d4101c70 0x7f00d419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d4101c70 0x7f00d419d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53360/0 (socket says 172.21.15.18:53360) 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d37fe640 1 -- 172.21.15.18:0/1095076514 learned_addr learned my addr 172.21.15.18:0/1095076514 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00cbfff640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d41012b0 0x7f00d4196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d2ffd640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d41000c0 0x7f00d41960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d37fe640 1 -- 172.21.15.18:0/1095076514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d41012b0 msgr2=0x7f00d4196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d37fe640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d41012b0 0x7f00d4196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d37fe640 1 -- 172.21.15.18:0/1095076514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d41000c0 msgr2=0x7f00d41960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d37fe640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d41000c0 0x7f00d41960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.149+0000 7f00d37fe640 1 -- 172.21.15.18:0/1095076514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc009560 con 0x7f00d4101c70 2024-02-17T12:49:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.150+0000 7f00d37fe640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d4101c70 0x7f00d419d670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00cce0 tx=0x7f00cc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.150+0000 7f00d0ff9640 1 -- 172.21.15.18:0/1095076514 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc00f450 con 0x7f00d4101c70 2024-02-17T12:49:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.150+0000 7f00d9688640 1 -- 172.21.15.18:0/1095076514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d419e1c0 con 0x7f00d4101c70 2024-02-17T12:49:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.150+0000 7f00d0ff9640 1 -- 172.21.15.18:0/1095076514 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00cc00fa90 con 0x7f00d4101c70 2024-02-17T12:49:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.150+0000 7f00d0ff9640 1 -- 172.21.15.18:0/1095076514 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc018b90 con 0x7f00d4101c70 2024-02-17T12:49:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.150+0000 7f00d9688640 1 -- 172.21.15.18:0/1095076514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d419e6a0 con 0x7f00d4101c70 2024-02-17T12:49:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.151+0000 7f00d9688640 1 -- 172.21.15.18:0/1095076514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d4107380 con 0x7f00d4101c70 2024-02-17T12:49:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.152+0000 7f00d0ff9640 1 -- 172.21.15.18:0/1095076514 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f00cc00fc00 con 0x7f00d4101c70 2024-02-17T12:49:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.152+0000 7f00d0ff9640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f00c00a21d0 0x7f00c00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.153+0000 7f00d2ffd640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f00c00a21d0 0x7f00c00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.153+0000 7f00d0ff9640 1 -- 172.21.15.18:0/1095076514 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f00cc0ce650 con 0x7f00d4101c70 2024-02-17T12:49:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.153+0000 7f00d2ffd640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f00c00a21d0 0x7f00c00a4690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f00c40058c0 tx=0x7f00c400a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.157+0000 7f00d0ff9640 1 -- 172.21.15.18:0/1095076514 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f00cc097490 con 0x7f00d4101c70 2024-02-17T12:49:07.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.376+0000 7f00d9688640 1 -- 172.21.15.18:0/1095076514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00d4106df0 con 0x7f00d4101c70 2024-02-17T12:49:07.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.377+0000 7f00d0ff9640 1 -- 172.21.15.18:0/1095076514 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11392 (secure 0 0 0) 0x7f00cc09ab50 con 0x7f00d4101c70 2024-02-17T12:49:07.370 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:07.370 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:47:51.719041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:49:07.371 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-02-17T12:49:07.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.379+0000 7f00d9688640 1 -- 172.21.15.18:0/1095076514 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f00c00a21d0 msgr2=0x7f00c00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.379+0000 7f00d9688640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f00c00a21d0 0x7f00c00a4690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f00c40058c0 tx=0x7f00c400a4c0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.379+0000 7f00d9688640 1 -- 172.21.15.18:0/1095076514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d4101c70 msgr2=0x7f00d419d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.379+0000 7f00d9688640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d4101c70 0x7f00d419d670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00cce0 tx=0x7f00cc00a620 comp rx=0 tx=0).stop 2024-02-17T12:49:07.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.379+0000 7f00d9688640 1 -- 172.21.15.18:0/1095076514 shutdown_connections 2024-02-17T12:49:07.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.379+0000 7f00d9688640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f00c00a21d0 0x7f00c00a4690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.379+0000 7f00d9688640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d4101c70 0x7f00d419d670 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.379+0000 7f00d9688640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d41012b0 0x7f00d4196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.379+0000 7f00d9688640 1 --2- 172.21.15.18:0/1095076514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d41000c0 0x7f00d41960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.379+0000 7f00d9688640 1 -- 172.21.15.18:0/1095076514 >> 172.21.15.18:0/1095076514 conn(0x7f00d40fb630 msgr2=0x7f00d4104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.379+0000 7f00d9688640 1 -- 172.21.15.18:0/1095076514 shutdown_connections 2024-02-17T12:49:07.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.379+0000 7f00d9688640 1 -- 172.21.15.18:0/1095076514 wait complete. 2024-02-17T12:49:07.383 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-17T12:47:45.407920+0000', 'modified': '2024-02-17T12:47:51.719041+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24512, 'mds_1': 24476, 'mds_2': 24455, 'mds_3': 24448, 'mds_4': 24422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.18:6839/1294470347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6838', 'nonce': 1294470347}, {'type': 'v1', 'addr': '172.21.15.18:6839', 'nonce': 1294470347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14655': {'gid': 14655, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.18:6841/2569825427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6840', 'nonce': 2569825427}, {'type': 'v1', 'addr': '172.21.15.18:6841', 'nonce': 2569825427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6833/4241790141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 4241790141}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 4241790141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.119:6833/21392902', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6832', 'nonce': 21392902}, {'type': 'v1', 'addr': '172.21.15.119:6833', 'nonce': 21392902}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.146:6835/405923223', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 405923223}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 405923223}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.119:6835/2304139172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 2304139172}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 2304139172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6837/1732085011', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 1732085011}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 1732085011}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6839/3061839650', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6838', 'nonce': 3061839650}, {'type': 'v1', 'addr': '172.21.15.146:6839', 'nonce': 3061839650}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.119:6837/248613392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 248613392}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 248613392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6839/2236251663', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 2236251663}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 2236251663}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-17T12:49:07.383 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-02-17T12:49:07.383 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-02-17T12:49:07.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.495+0000 7f19a899e640 1 -- 172.21.15.18:0/2524735482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a4101060 msgr2=0x7f19a4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.495+0000 7f19a899e640 1 --2- 172.21.15.18:0/2524735482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a4101060 0x7f19a4101480 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1994009900 tx=0x7f199402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:07.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.495+0000 7f19a899e640 1 -- 172.21.15.18:0/2524735482 shutdown_connections 2024-02-17T12:49:07.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.495+0000 7f19a899e640 1 --2- 172.21.15.18:0/2524735482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19a4102c10 0x7f19a41094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.495+0000 7f19a899e640 1 --2- 172.21.15.18:0/2524735482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19a4102250 0x7f19a41026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.495+0000 7f19a899e640 1 --2- 172.21.15.18:0/2524735482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a4101060 0x7f19a4101480 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.495+0000 7f19a899e640 1 -- 172.21.15.18:0/2524735482 >> 172.21.15.18:0/2524735482 conn(0x7f19a40fc5f0 msgr2=0x7f19a40fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.495+0000 7f19a899e640 1 -- 172.21.15.18:0/2524735482 shutdown_connections 2024-02-17T12:49:07.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.495+0000 7f19a899e640 1 -- 172.21.15.18:0/2524735482 wait complete. 2024-02-17T12:49:07.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a899e640 1 Processor -- start 2024-02-17T12:49:07.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a899e640 1 -- start start 2024-02-17T12:49:07.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a899e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19a4101060 0x7f19a4199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a899e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a4102250 0x7f19a4199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a899e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19a4102c10 0x7f19a41a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a899e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a41a0e10 con 0x7f19a4102250 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a899e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a41a0f80 con 0x7f19a4102c10 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a899e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a41a1140 con 0x7f19a4101060 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a4102250 0x7f19a4199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a4102250 0x7f19a4199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53386/0 (socket says 172.21.15.18:53386) 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a2ffd640 1 -- 172.21.15.18:0/2321820414 learned_addr learned my addr 172.21.15.18:0/2321820414 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a37fe640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19a4101060 0x7f19a4199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a3fff640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19a4102c10 0x7f19a41a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a2ffd640 1 -- 172.21.15.18:0/2321820414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19a4101060 msgr2=0x7f19a4199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a2ffd640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19a4101060 0x7f19a4199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a2ffd640 1 -- 172.21.15.18:0/2321820414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19a4102c10 msgr2=0x7f19a41a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a2ffd640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19a4102c10 0x7f19a41a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.496+0000 7f19a2ffd640 1 -- 172.21.15.18:0/2321820414 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1994009560 con 0x7f19a4102250 2024-02-17T12:49:07.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.497+0000 7f19a2ffd640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a4102250 0x7f19a4199860 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f199800d770 tx=0x7f199800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.497+0000 7f19a0ff9640 1 -- 172.21.15.18:0/2321820414 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998009c30 con 0x7f19a4102250 2024-02-17T12:49:07.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.497+0000 7f19a0ff9640 1 -- 172.21.15.18:0/2321820414 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f199800de80 con 0x7f19a4102250 2024-02-17T12:49:07.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.497+0000 7f19a899e640 1 -- 172.21.15.18:0/2321820414 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a41a1420 con 0x7f19a4102250 2024-02-17T12:49:07.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.497+0000 7f19a0ff9640 1 -- 172.21.15.18:0/2321820414 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998004e80 con 0x7f19a4102250 2024-02-17T12:49:07.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.497+0000 7f19a899e640 1 -- 172.21.15.18:0/2321820414 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a41a18e0 con 0x7f19a4102250 2024-02-17T12:49:07.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.499+0000 7f19a0ff9640 1 -- 172.21.15.18:0/2321820414 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f1998005020 con 0x7f19a4102250 2024-02-17T12:49:07.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.499+0000 7f19ac7cd740 1 Processor -- start 2024-02-17T12:49:07.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.499+0000 7f19ac7cd740 1 -- start start 2024-02-17T12:49:07.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19ac7cd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d097180 0x56500d0975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19ac7cd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097b70 0x56500d09c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19ac7cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d09cce0 0x56500d09f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19ac7cd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500d052cd0 con 0x56500d097b70 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19ac7cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500d005f20 con 0x56500d09cce0 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19ac7cd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500d003450 con 0x56500d097180 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097b70 0x56500d09c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097b70 0x56500d09c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53400/0 (socket says 172.21.15.18:53400) 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a37fe640 1 -- 172.21.15.18:0/1378720263 learned_addr learned my addr 172.21.15.18:0/1378720263 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a0ff9640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f198c0a2150 0x7f198c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a2ffd640 1 --2- 172.21.15.18:0/1378720263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d09cce0 0x56500d09f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a3fff640 1 --2- 172.21.15.18:0/1378720263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d097180 0x56500d0975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a3fff640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f198c0a2150 0x7f198c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a37fe640 1 -- 172.21.15.18:0/1378720263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d097180 msgr2=0x56500d0975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a37fe640 1 --2- 172.21.15.18:0/1378720263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d097180 0x56500d0975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a37fe640 1 -- 172.21.15.18:0/1378720263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d09cce0 msgr2=0x56500d09f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a37fe640 1 --2- 172.21.15.18:0/1378720263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d09cce0 0x56500d09f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a37fe640 1 -- 172.21.15.18:0/1378720263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56500cfb3f20 con 0x56500d097b70 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a0ff9640 1 -- 172.21.15.18:0/2321820414 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f19980cd850 con 0x7f19a4102250 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a37fe640 1 --2- 172.21.15.18:0/1378720263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097b70 0x56500d09c670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1994009900 tx=0x7f1994005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f975ad1353aee1f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.500+0000 7f19a3fff640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f198c0a2150 0x7f198c0a4610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x56500d0983d0 tx=0x7f199c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.501+0000 7f1989ffb640 1 -- 172.21.15.18:0/1378720263 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199402caa0 con 0x56500d097b70 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.501+0000 7f1989ffb640 1 -- 172.21.15.18:0/1378720263 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f199402cc00 con 0x56500d097b70 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.501+0000 7f1989ffb640 1 -- 172.21.15.18:0/1378720263 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994005210 con 0x56500d097b70 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.501+0000 7f19ac7cd740 1 -- 172.21.15.18:0/1378720263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097b70 msgr2=0x56500d09c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.501+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/1378720263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097b70 0x56500d09c670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1994009900 tx=0x7f1994005e00 comp rx=0 tx=0).stop 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.501+0000 7f19ac7cd740 1 -- 172.21.15.18:0/1378720263 shutdown_connections 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.501+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/1378720263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d09cce0 0x56500d09f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.501+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/1378720263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097b70 0x56500d09c670 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.501+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/1378720263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d097180 0x56500d0975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.501+0000 7f19ac7cd740 1 -- 172.21.15.18:0/1378720263 >> 172.21.15.18:0/1378720263 conn(0x56500d0926f0 msgr2=0x56500d092af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.501+0000 7f19ac7cd740 1 -- 172.21.15.18:0/1378720263 shutdown_connections 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.501+0000 7f19ac7cd740 1 -- 172.21.15.18:0/1378720263 wait complete. 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19ac7cd740 1 -- start start 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19ac7cd740 1 Processor -- start 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19ac7cd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d097180 0x56500d0a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19ac7cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19ac7cd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d09cce0 0x56500d0a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19ac7cd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500cffec30 con 0x56500d09cce0 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19ac7cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500cfa3d30 con 0x56500d097b70 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19ac7cd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500ccd2310 con 0x56500d097180 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19ac7cd740 10 client.0 _ll_register_callbacks cb 0x56500cfabd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19a3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d09cce0 0x56500d0a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19a3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d09cce0 0x56500d0a8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53412/0 (socket says 172.21.15.18:53412) 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19a3fff640 1 -- 172.21.15.18:0/2115019552 learned_addr learned my addr 172.21.15.18:0/2115019552 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19a37fe640 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d097180 0x56500d0a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19a3fff640 1 -- 172.21.15.18:0/2115019552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d097180 msgr2=0x56500d0a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19a2ffd640 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19a3fff640 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d097180 0x56500d0a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19a3fff640 1 -- 172.21.15.18:0/2115019552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 msgr2=0x56500d0a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19a3fff640 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.502+0000 7f19a3fff640 1 -- 172.21.15.18:0/2115019552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a4109da0 con 0x56500d09cce0 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f19a3fff640 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d09cce0 0x56500d0a8850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f199c0042d0 tx=0x7f199c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74194d0a81f960 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f1982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f1982ffd640 1 -- 172.21.15.18:0/2115019552 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c011eb0 con 0x56500d09cce0 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f1982ffd640 1 -- 172.21.15.18:0/2115019552 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f199c00f420 con 0x56500d09cce0 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2115019552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56500ccfe6c0 con 0x56500d09cce0 2024-02-17T12:49:07.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f1982ffd640 1 -- 172.21.15.18:0/2115019552 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f199c004850 con 0x56500d09cce0 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f19ac7cd740 10 client.15069 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2115019552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56500cffd2b0 con 0x56500d09cce0 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f1982ffd640 1 -- 172.21.15.18:0/2115019552 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56500ccfe6c0 con 0x56500d09cce0 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f1982ffd640 1 -- 172.21.15.18:0/2115019552 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f199c025020 con 0x56500d09cce0 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f19ac7cd740 10 client.15069 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f19ac7cd740 10 client.15069 resolve_mds: resolved name 'k' to daemon mds.k 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f19ac7cd740 20 client.15069 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56500d0add80 0x56500d0acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f19ac7cd740 4 client.15069 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.503+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2115019552 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56500cd4f270 con 0x56500d0add80 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.504+0000 7f19a37fe640 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56500d0add80 0x56500d0acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.504+0000 7f1982ffd640 10 client.15069 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.504+0000 7f19a37fe640 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56500d0add80 0x56500d0acfd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=19d43c7a924c870e server_cookie=fe6bddd6bdb6fc42 in_seq=0 out_seq=0 2024-02-17T12:49:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f1982ffd640 1 -- 172.21.15.18:0/2115019552 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56500cd4f270 con 0x56500d0add80 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f1982ffd640 10 client.15069 handle_command_reply: tid=0 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f19ac7cd740 1 client.15069 shutdown 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2115019552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d09cce0 msgr2=0x56500d0a8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d09cce0 0x56500d0a8850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f199c0042d0 tx=0x7f199c004a40 comp rx=0 tx=0).stop 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2115019552 shutdown_connections 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d09cce0 0x56500d0a8850 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56500d0add80 0x56500d0acfd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f1982ffd640 0 client.15069 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2115019552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d097180 0x56500d0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2115019552 >> 172.21.15.18:0/2115019552 conn(0x56500d0926f0 msgr2=0x56500d091930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2115019552 shutdown_connections 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2115019552 wait complete. 2024-02-17T12:49:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.505+0000 7f19ac7cd740 20 client.15069 trim_cache size 0 max 16384 2024-02-17T12:49:07.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19ac7cd740 1 Processor -- start 2024-02-17T12:49:07.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19ac7cd740 1 -- start start 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19ac7cd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 0x56500d092ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19ac7cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19ac7cd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d09cce0 0x56500d0910a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19ac7cd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500cf84490 con 0x56500d097180 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19ac7cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500d0a94d0 con 0x56500d097b70 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19ac7cd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500d0ad200 con 0x56500d09cce0 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19a37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 0x56500d092ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19a37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 0x56500d092ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53414/0 (socket says 172.21.15.18:53414) 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19a37fe640 1 -- 172.21.15.18:0/2562940290 learned_addr learned my addr 172.21.15.18:0/2562940290 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19a37fe640 1 -- 172.21.15.18:0/2562940290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d09cce0 msgr2=0x56500d0910a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19a37fe640 1 --2- 172.21.15.18:0/2562940290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d09cce0 0x56500d0910a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19a2ffd640 1 --2- 172.21.15.18:0/2562940290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19a37fe640 1 -- 172.21.15.18:0/2562940290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 msgr2=0x56500d0a8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19a37fe640 1 --2- 172.21.15.18:0/2562940290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19a37fe640 1 -- 172.21.15.18:0/2562940290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56500cf99a20 con 0x56500d097180 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19a2ffd640 1 --2- 172.21.15.18:0/2562940290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19a37fe640 1 --2- 172.21.15.18:0/2562940290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 0x56500d092ad0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x56500d0985e0 tx=0x7f1994000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd9cd66f51371d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19897fa640 1 -- 172.21.15.18:0/2562940290 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19940046c0 con 0x56500d097180 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19897fa640 1 -- 172.21.15.18:0/2562940290 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1994004940 con 0x56500d097180 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.521+0000 7f19897fa640 1 -- 172.21.15.18:0/2562940290 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19940505f0 con 0x56500d097180 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.522+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2562940290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 msgr2=0x56500d092ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.522+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2562940290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 0x56500d092ad0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x56500d0985e0 tx=0x7f1994000bc0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.522+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2562940290 shutdown_connections 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.522+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2562940290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d09cce0 0x56500d0910a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.522+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2562940290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.522+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2562940290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 0x56500d092ad0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.522+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2562940290 >> 172.21.15.18:0/2562940290 conn(0x56500d0926f0 msgr2=0x56500d098610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.522+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2562940290 shutdown_connections 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.522+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2562940290 wait complete. 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.522+0000 7f19ac7cd740 1 -- start start 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.522+0000 7f19ac7cd740 1 Processor -- start 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19ac7cd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 0x56500d0a8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19ac7cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19ac7cd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d09cce0 0x56500d0a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19ac7cd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500d0a5b10 con 0x56500d097180 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19ac7cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500d0a5c80 con 0x56500d097b70 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19ac7cd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500d1039b0 con 0x56500d09cce0 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19ac7cd740 10 client.0 _ll_register_callbacks cb 0x7f1994009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19a37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 0x56500d0a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19a37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 0x56500d0a8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53424/0 (socket says 172.21.15.18:53424) 2024-02-17T12:49:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19a37fe640 1 -- 172.21.15.18:0/2797837567 learned_addr learned my addr 172.21.15.18:0/2797837567 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19a37fe640 1 -- 172.21.15.18:0/2797837567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d09cce0 msgr2=0x56500d0a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19a2ffd640 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19a37fe640 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d09cce0 0x56500d0a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19a37fe640 1 -- 172.21.15.18:0/2797837567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 msgr2=0x56500d0a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19a37fe640 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19a37fe640 1 -- 172.21.15.18:0/2797837567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56500cfb3f20 con 0x56500d097180 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19a2ffd640 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19a37fe640 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 0x56500d0a8570 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f199402ce20 tx=0x7f1994050d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a369c88e60df6c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f1982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f1982ffd640 1 -- 172.21.15.18:0/2797837567 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994050580 con 0x56500d097180 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2797837567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56500d0ada90 con 0x56500d097180 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.523+0000 7f1982ffd640 1 -- 172.21.15.18:0/2797837567 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f19940506e0 con 0x56500d097180 2024-02-17T12:49:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.524+0000 7f1982ffd640 1 -- 172.21.15.18:0/2797837567 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19940508a0 con 0x56500d097180 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.524+0000 7f19ac7cd740 10 client.15075 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.524+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2797837567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56500cfbf780 con 0x56500d097180 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.524+0000 7f1982ffd640 1 -- 172.21.15.18:0/2797837567 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56500d0ada90 con 0x56500d097180 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.524+0000 7f1982ffd640 1 -- 172.21.15.18:0/2797837567 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f199403c050 con 0x56500d097180 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.524+0000 7f19ac7cd740 10 client.15075 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.524+0000 7f19ac7cd740 10 client.15075 resolve_mds: resolved name 'k' to daemon mds.k 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.524+0000 7f19ac7cd740 20 client.15075 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.524+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56500d1087b0 0x56500d104bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.524+0000 7f19ac7cd740 4 client.15075 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.524+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2797837567 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f199402cc00 con 0x56500d1087b0 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.524+0000 7f19a2ffd640 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56500d1087b0 0x56500d104bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f1982ffd640 10 client.15075 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19a2ffd640 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56500d1087b0 0x56500d104bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dbcea41347b4da18 server_cookie=6b5cb21377ed2a82 in_seq=0 out_seq=0 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f1982ffd640 1 -- 172.21.15.18:0/2797837567 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f199402cc00 con 0x56500d1087b0 2024-02-17T12:49:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f1982ffd640 10 client.15075 handle_command_reply: tid=0 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19ac7cd740 1 client.15075 shutdown 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2797837567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 msgr2=0x56500d0a8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 0x56500d0a8570 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f199402ce20 tx=0x7f1994050d70 comp rx=0 tx=0).stop 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2797837567 shutdown_connections 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56500d09cce0 0x56500d0a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56500d1087b0 0x56500d104bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56500d097b70 0x56500d0a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f1982ffd640 0 client.15075 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19ac7cd740 1 --2- 172.21.15.18:0/2797837567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56500d097180 0x56500d0a8570 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2797837567 >> 172.21.15.18:0/2797837567 conn(0x56500d0926f0 msgr2=0x56500d092ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2797837567 shutdown_connections 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19ac7cd740 1 -- 172.21.15.18:0/2797837567 wait complete. 2024-02-17T12:49:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.525+0000 7f19ac7cd740 20 client.15075 trim_cache size 0 max 16384 2024-02-17T12:49:07.525 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:49:07.525 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:49:07.525 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 4, 2024-02-17T12:49:07.525 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24422, 2024-02-17T12:49:07.525 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-02-17T12:49:07.525 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-02-17T12:49:07.525 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-02-17T12:49:07.525 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 76.810020894000004, 2024-02-17T12:49:07.525 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 25, 2024-02-17T12:49:07.525 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 88, 2024-02-17T12:49:07.526 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 86, 2024-02-17T12:49:07.526 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 114.691167963 2024-02-17T12:49:07.526 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:49:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.526+0000 7f19a899e640 1 -- 172.21.15.18:0/2321820414 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f198c0a2150 msgr2=0x7f198c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.526+0000 7f19a899e640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f198c0a2150 0x7f198c0a4610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x56500d0983d0 tx=0x7f199c009500 comp rx=0 tx=0).stop 2024-02-17T12:49:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.526+0000 7f19a899e640 1 -- 172.21.15.18:0/2321820414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a4102250 msgr2=0x7f19a4199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.526+0000 7f19a899e640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a4102250 0x7f19a4199860 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f199800d770 tx=0x7f199800dc30 comp rx=0 tx=0).stop 2024-02-17T12:49:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.526+0000 7f19a899e640 1 -- 172.21.15.18:0/2321820414 shutdown_connections 2024-02-17T12:49:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.526+0000 7f19a899e640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f198c0a2150 0x7f198c0a4610 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.526+0000 7f19a899e640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19a4102c10 0x7f19a41a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.526+0000 7f19a899e640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a4102250 0x7f19a4199860 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.526+0000 7f19a899e640 1 --2- 172.21.15.18:0/2321820414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19a4101060 0x7f19a4199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.526+0000 7f19a899e640 1 -- 172.21.15.18:0/2321820414 >> 172.21.15.18:0/2321820414 conn(0x7f19a40fc5f0 msgr2=0x7f19a40733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.526+0000 7f19a899e640 1 -- 172.21.15.18:0/2321820414 shutdown_connections 2024-02-17T12:49:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.526+0000 7f19a899e640 1 -- 172.21.15.18:0/2321820414 wait complete. 2024-02-17T12:49:07.529 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-02-17T12:49:07.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.640+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/1000656828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdcec100080 msgr2=0x7fdcec1004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.640+0000 7fdcf1e0f640 1 --2- 172.21.15.18:0/1000656828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdcec100080 0x7fdcec1004a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009ca0 tx=0x7fdcdc02c100 comp rx=0 tx=0).stop 2024-02-17T12:49:07.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.640+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/1000656828 shutdown_connections 2024-02-17T12:49:07.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.640+0000 7fdcf1e0f640 1 --2- 172.21.15.18:0/1000656828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcec101c30 0x7fdcec1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.640+0000 7fdcf1e0f640 1 --2- 172.21.15.18:0/1000656828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcec101270 0x7fdcec1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.640+0000 7fdcf1e0f640 1 --2- 172.21.15.18:0/1000656828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdcec100080 0x7fdcec1004a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.640+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/1000656828 >> 172.21.15.18:0/1000656828 conn(0x7fdcec0fb610 msgr2=0x7fdcec0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.640+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/1000656828 shutdown_connections 2024-02-17T12:49:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.641+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/1000656828 wait complete. 2024-02-17T12:49:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.641+0000 7fdcf1e0f640 1 Processor -- start 2024-02-17T12:49:07.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.641+0000 7fdcf1e0f640 1 -- start start 2024-02-17T12:49:07.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.641+0000 7fdcf1e0f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcec100080 0x7fdcec197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.641+0000 7fdcf1e0f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcec101270 0x7fdcec1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.641+0000 7fdcf1e0f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdcec101c30 0x7fdcec19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.641+0000 7fdcf1e0f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec19ec60 con 0x7fdcec101c30 2024-02-17T12:49:07.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.641+0000 7fdcf1e0f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec19edd0 con 0x7fdcec101270 2024-02-17T12:49:07.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.641+0000 7fdcf1e0f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec19ef90 con 0x7fdcec100080 2024-02-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.641+0000 7fdcebfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdcec101c30 0x7fdcec19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.641+0000 7fdcebfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdcec101c30 0x7fdcec19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53448/0 (socket says 172.21.15.18:53448) 2024-02-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.641+0000 7fdcebfff640 1 -- 172.21.15.18:0/683795182 learned_addr learned my addr 172.21.15.18:0/683795182 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdceaffd640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcec101270 0x7fdcec1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdceb7fe640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcec100080 0x7fdcec197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdcebfff640 1 -- 172.21.15.18:0/683795182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcec100080 msgr2=0x7fdcec197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdcebfff640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcec100080 0x7fdcec197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdcebfff640 1 -- 172.21.15.18:0/683795182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcec101270 msgr2=0x7fdcec1976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdcebfff640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcec101270 0x7fdcec1976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdcebfff640 1 -- 172.21.15.18:0/683795182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc009900 con 0x7fdcec101c30 2024-02-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdceb7fe640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcec100080 0x7fdcec197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdcebfff640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdcec101c30 0x7fdcec19e720 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c970 tx=0x7fdce400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdce8ff9640 1 -- 172.21.15.18:0/683795182 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4007bb0 con 0x7fdcec101c30 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdce8ff9640 1 -- 172.21.15.18:0/683795182 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdce4007d10 con 0x7fdcec101c30 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/683795182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec19f270 con 0x7fdcec101c30 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdce8ff9640 1 -- 172.21.15.18:0/683795182 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce40186c0 con 0x7fdcec101c30 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.642+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/683795182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec19f730 con 0x7fdcec101c30 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.644+0000 7fdce8ff9640 1 -- 172.21.15.18:0/683795182 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fdce400f450 con 0x7fdcec101c30 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.644+0000 7fdcf49db740 1 Processor -- start 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdcf49db740 1 -- start start 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdcf49db740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f31f80 0x55f764f323a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdcf49db740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 0x55f764f37470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdcf49db740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 0x55f764f39f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdcf49db740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764eedcd0 con 0x55f764f37ae0 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdcf49db740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764ea0f20 con 0x55f764f31f80 2024-02-17T12:49:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdcf49db740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764e9e450 con 0x55f764f32970 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdce8ff9640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdcd40a2150 0x7fdcd40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdceaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 0x55f764f39f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdceb7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 0x55f764f37470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdceaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 0x55f764f39f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53462/0 (socket says 172.21.15.18:53462) 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdcebfff640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdcd40a2150 0x7fdcd40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdceaffd640 1 -- 172.21.15.18:0/2188243229 learned_addr learned my addr 172.21.15.18:0/2188243229 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdceaffd640 1 --2- 172.21.15.18:0/2188243229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f31f80 0x55f764f323a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdce8ff9640 1 -- 172.21.15.18:0/683795182 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdce40ce040 con 0x7fdcec101c30 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdceaffd640 1 -- 172.21.15.18:0/2188243229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 msgr2=0x55f764f37470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdceaffd640 1 --2- 172.21.15.18:0/2188243229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 0x55f764f37470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdceaffd640 1 -- 172.21.15.18:0/2188243229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f31f80 msgr2=0x55f764f323a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdceaffd640 1 --2- 172.21.15.18:0/2188243229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f31f80 0x55f764f323a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdceaffd640 1 -- 172.21.15.18:0/2188243229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f764e4ef20 con 0x55f764f37ae0 2024-02-17T12:49:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdceaffd640 1 --2- 172.21.15.18:0/2188243229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 0x55f764f39f70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdce000a5c0 tx=0x7fdce000a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275b196d7e590dd2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdcebfff640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdcd40a2150 0x7fdcd40a4610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdce4097040 tx=0x7fdce406cbf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.645+0000 7fdcd1ffb640 1 -- 172.21.15.18:0/2188243229 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0005020 con 0x55f764f37ae0 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.646+0000 7fdcd1ffb640 1 -- 172.21.15.18:0/2188243229 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdce0005660 con 0x55f764f37ae0 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.646+0000 7fdcd1ffb640 1 -- 172.21.15.18:0/2188243229 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce000fa20 con 0x55f764f37ae0 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.646+0000 7fdcf49db740 1 -- 172.21.15.18:0/2188243229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 msgr2=0x55f764f39f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.646+0000 7fdcf49db740 1 --2- 172.21.15.18:0/2188243229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 0x55f764f39f70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdce000a5c0 tx=0x7fdce000a760 comp rx=0 tx=0).stop 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.646+0000 7fdcf49db740 1 -- 172.21.15.18:0/2188243229 shutdown_connections 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.646+0000 7fdcf49db740 1 --2- 172.21.15.18:0/2188243229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 0x55f764f39f70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.646+0000 7fdcf49db740 1 --2- 172.21.15.18:0/2188243229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 0x55f764f37470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.646+0000 7fdcf49db740 1 --2- 172.21.15.18:0/2188243229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f31f80 0x55f764f323a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.646+0000 7fdcf49db740 1 -- 172.21.15.18:0/2188243229 >> 172.21.15.18:0/2188243229 conn(0x55f764f2d4f0 msgr2=0x55f764f2d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.646+0000 7fdcf49db740 1 -- 172.21.15.18:0/2188243229 shutdown_connections 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.646+0000 7fdcf49db740 1 -- 172.21.15.18:0/2188243229 wait complete. 2024-02-17T12:49:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdcf49db740 1 -- start start 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdcf49db740 1 Processor -- start 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdcf49db740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 0x55f764f3c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdcf49db740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 0x55f764f3c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdcf49db740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f37ae0 0x55f764f43710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdcf49db740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764e99c30 con 0x55f764f31f80 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdcf49db740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764e3ed30 con 0x55f764f37ae0 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdcf49db740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764b6d310 con 0x55f764f32970 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdceaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 0x55f764f3c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdcf49db740 10 client.0 _ll_register_callbacks cb 0x55f764e46d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdceaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 0x55f764f3c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53474/0 (socket says 172.21.15.18:53474) 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdceaffd640 1 -- 172.21.15.18:0/3245213652 learned_addr learned my addr 172.21.15.18:0/3245213652 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdceaffd640 1 -- 172.21.15.18:0/3245213652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 msgr2=0x55f764f3c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdceaffd640 1 --2- 172.21.15.18:0/3245213652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 0x55f764f3c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdceaffd640 1 -- 172.21.15.18:0/3245213652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f37ae0 msgr2=0x55f764f43710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdceaffd640 1 --2- 172.21.15.18:0/3245213652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f37ae0 0x55f764f43710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.647+0000 7fdceaffd640 1 -- 172.21.15.18:0/3245213652 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce0019040 con 0x55f764f31f80 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdceaffd640 1 --2- 172.21.15.18:0/3245213652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 0x55f764f3c190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdce000ad90 tx=0x7fdce000adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca09006781d27425 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdccaffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdccaffd640 1 -- 172.21.15.18:0/3245213652 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0004990 con 0x55f764f31f80 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdccaffd640 1 -- 172.21.15.18:0/3245213652 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdce0000e40 con 0x55f764f31f80 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdcf49db740 1 -- 172.21.15.18:0/3245213652 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f764b996c0 con 0x55f764f31f80 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdccaffd640 1 -- 172.21.15.18:0/3245213652 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdce0004af0 con 0x55f764f31f80 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdcf49db740 10 client.15087 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdcf49db740 1 -- 172.21.15.18:0/3245213652 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f764e982b0 con 0x55f764f31f80 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdccaffd640 1 -- 172.21.15.18:0/3245213652 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f764b996c0 con 0x55f764f31f80 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdccaffd640 1 -- 172.21.15.18:0/3245213652 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdce001e930 con 0x55f764f31f80 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdcf49db740 10 client.15087 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdcf49db740 10 client.15087 resolve_mds: resolved name 'f' to daemon mds.f 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdcf49db740 20 client.15087 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdcf49db740 1 --2- 172.21.15.18:0/3245213652 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f764f47c60 0x55f764f4cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdcf49db740 4 client.15087 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.648+0000 7fdcf49db740 1 -- 172.21.15.18:0/3245213652 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f764bea270 con 0x55f764f47c60 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.649+0000 7fdceb7fe640 1 --2- 172.21.15.18:0/3245213652 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f764f47c60 0x55f764f4cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.649+0000 7fdccaffd640 10 client.15087 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.649+0000 7fdceb7fe640 1 --2- 172.21.15.18:0/3245213652 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f764f47c60 0x55f764f4cd80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32d74d5a5ed2631c server_cookie=427487ef573fe2d9 in_seq=0 out_seq=0 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdccaffd640 1 -- 172.21.15.18:0/3245213652 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f764bea270 con 0x55f764f47c60 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdccaffd640 10 client.15087 handle_command_reply: tid=0 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdcf49db740 1 client.15087 shutdown 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdcf49db740 1 -- 172.21.15.18:0/3245213652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 msgr2=0x55f764f3c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdcf49db740 1 --2- 172.21.15.18:0/3245213652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 0x55f764f3c190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdce000ad90 tx=0x7fdce000adc0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdcf49db740 1 -- 172.21.15.18:0/3245213652 shutdown_connections 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdcf49db740 1 --2- 172.21.15.18:0/3245213652 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f764f47c60 0x55f764f4cd80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdcf49db740 1 --2- 172.21.15.18:0/3245213652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f37ae0 0x55f764f43710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdcf49db740 1 --2- 172.21.15.18:0/3245213652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 0x55f764f3c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdccaffd640 0 client.15087 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:49:07.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdcf49db740 1 --2- 172.21.15.18:0/3245213652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 0x55f764f3c190 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdcf49db740 1 -- 172.21.15.18:0/3245213652 >> 172.21.15.18:0/3245213652 conn(0x55f764f2d4f0 msgr2=0x55f764f2c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdcf49db740 1 -- 172.21.15.18:0/3245213652 shutdown_connections 2024-02-17T12:49:07.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdcf49db740 1 -- 172.21.15.18:0/3245213652 wait complete. 2024-02-17T12:49:07.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.650+0000 7fdcf49db740 20 client.15087 trim_cache size 0 max 16384 2024-02-17T12:49:07.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdcf49db740 1 Processor -- start 2024-02-17T12:49:07.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdcf49db740 1 -- start start 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdcf49db740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f31f80 0x55f764f2d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdcf49db740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 0x55f764f436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdcf49db740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 0x55f764f39ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdcf49db740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764e1f490 con 0x55f764f37ae0 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdcf49db740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764f376e0 con 0x55f764f31f80 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdcf49db740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764f3b9e0 con 0x55f764f32970 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdceaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 0x55f764f39ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdceaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 0x55f764f39ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53484/0 (socket says 172.21.15.18:53484) 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdceaffd640 1 -- 172.21.15.18:0/3960742547 learned_addr learned my addr 172.21.15.18:0/3960742547 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdceaffd640 1 -- 172.21.15.18:0/3960742547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 msgr2=0x55f764f436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdceb7fe640 1 --2- 172.21.15.18:0/3960742547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 0x55f764f436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdceaffd640 1 --2- 172.21.15.18:0/3960742547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 0x55f764f436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdceaffd640 1 -- 172.21.15.18:0/3960742547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f31f80 msgr2=0x55f764f2d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdceaffd640 1 --2- 172.21.15.18:0/3960742547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f31f80 0x55f764f2d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdceaffd640 1 -- 172.21.15.18:0/3960742547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f764e34a20 con 0x55f764f37ae0 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdceaffd640 1 --2- 172.21.15.18:0/3960742547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 0x55f764f39ed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdce0000c00 tx=0x7fdce000ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23284c5048b3a67c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdcd17fa640 1 -- 172.21.15.18:0/3960742547 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce000ae30 con 0x55f764f37ae0 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdcd17fa640 1 -- 172.21.15.18:0/3960742547 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdce0015040 con 0x55f764f37ae0 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.666+0000 7fdcd17fa640 1 -- 172.21.15.18:0/3960742547 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce001e6a0 con 0x55f764f37ae0 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.667+0000 7fdcf49db740 1 -- 172.21.15.18:0/3960742547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 msgr2=0x55f764f39ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.667+0000 7fdcf49db740 1 --2- 172.21.15.18:0/3960742547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 0x55f764f39ed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdce0000c00 tx=0x7fdce000ffc0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.667+0000 7fdcf49db740 1 -- 172.21.15.18:0/3960742547 shutdown_connections 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.667+0000 7fdcf49db740 1 --2- 172.21.15.18:0/3960742547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f37ae0 0x55f764f39ed0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.667+0000 7fdcf49db740 1 --2- 172.21.15.18:0/3960742547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f32970 0x55f764f436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.667+0000 7fdcf49db740 1 --2- 172.21.15.18:0/3960742547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f31f80 0x55f764f2d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.667+0000 7fdcf49db740 1 -- 172.21.15.18:0/3960742547 >> 172.21.15.18:0/3960742547 conn(0x55f764f2d4f0 msgr2=0x55f764f33060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.667+0000 7fdcf49db740 1 -- 172.21.15.18:0/3960742547 shutdown_connections 2024-02-17T12:49:07.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.667+0000 7fdcf49db740 1 -- 172.21.15.18:0/3960742547 wait complete. 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.667+0000 7fdcf49db740 1 -- start start 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdcf49db740 1 Processor -- start 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdcf49db740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 0x55f764f43110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdcf49db740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f32970 0x55f764f43650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdcf49db740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f37ae0 0x55f764f3fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdcf49db740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764f40330 con 0x55f764f31f80 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdcf49db740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764f404a0 con 0x55f764f32970 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdcf49db740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f764f40660 con 0x55f764f37ae0 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdcf49db740 10 client.0 _ll_register_callbacks cb 0x7fdce000aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdceaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 0x55f764f43110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdceaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 0x55f764f43110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53490/0 (socket says 172.21.15.18:53490) 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdceaffd640 1 -- 172.21.15.18:0/1139262621 learned_addr learned my addr 172.21.15.18:0/1139262621 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdceaffd640 1 --2- 172.21.15.18:0/1139262621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f37ae0 0x55f764f3fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdceaffd640 1 -- 172.21.15.18:0/1139262621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f37ae0 msgr2=0x55f764f3fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdceaffd640 1 --2- 172.21.15.18:0/1139262621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f37ae0 0x55f764f3fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdceaffd640 1 -- 172.21.15.18:0/1139262621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f32970 msgr2=0x55f764f43650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdceaffd640 1 --2- 172.21.15.18:0/1139262621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f32970 0x55f764f43650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdceaffd640 1 -- 172.21.15.18:0/1139262621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f764e982b0 con 0x55f764f31f80 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdceaffd640 1 --2- 172.21.15.18:0/1139262621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 0x55f764f43110 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdce000a960 tx=0x7fdce000a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6114bef7ade49ad server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdccaffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdccaffd640 1 -- 172.21.15.18:0/1139262621 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0011050 con 0x55f764f31f80 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdccaffd640 1 -- 172.21.15.18:0/1139262621 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdce00111b0 con 0x55f764f31f80 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.668+0000 7fdcf49db740 1 -- 172.21.15.18:0/1139262621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f764f48610 con 0x55f764f31f80 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.669+0000 7fdccaffd640 1 -- 172.21.15.18:0/1139262621 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdce001e620 con 0x55f764f31f80 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.669+0000 7fdcf49db740 10 client.15093 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.669+0000 7fdcf49db740 1 -- 172.21.15.18:0/1139262621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f764e5a780 con 0x55f764f31f80 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.669+0000 7fdccaffd640 1 -- 172.21.15.18:0/1139262621 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f764f48610 con 0x55f764f31f80 2024-02-17T12:49:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.669+0000 7fdccaffd640 1 -- 172.21.15.18:0/1139262621 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdce000f8b0 con 0x55f764f31f80 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.669+0000 7fdcf49db740 10 client.15093 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.669+0000 7fdcf49db740 10 client.15093 resolve_mds: resolved name 'f' to daemon mds.f 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.669+0000 7fdcf49db740 20 client.15093 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.669+0000 7fdcf49db740 1 --2- 172.21.15.18:0/1139262621 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f764fa3560 0x55f764f9f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.669+0000 7fdcf49db740 4 client.15093 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.669+0000 7fdcf49db740 1 -- 172.21.15.18:0/1139262621 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f764f492d0 con 0x55f764fa3560 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.669+0000 7fdceb7fe640 1 --2- 172.21.15.18:0/1139262621 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f764fa3560 0x55f764f9f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdceb7fe640 1 --2- 172.21.15.18:0/1139262621 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f764fa3560 0x55f764f9f9c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=434cfb0a58b7fa85 server_cookie=df9ae98ee84f2630 in_seq=0 out_seq=0 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdccaffd640 10 client.15093 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdccaffd640 1 -- 172.21.15.18:0/1139262621 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f764f492d0 con 0x55f764fa3560 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdccaffd640 10 client.15093 handle_command_reply: tid=0 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdcf49db740 1 client.15093 shutdown 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdcf49db740 1 -- 172.21.15.18:0/1139262621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 msgr2=0x55f764f43110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdcf49db740 1 --2- 172.21.15.18:0/1139262621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 0x55f764f43110 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdce000a960 tx=0x7fdce000a8f0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdcf49db740 1 -- 172.21.15.18:0/1139262621 shutdown_connections 2024-02-17T12:49:07.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdcf49db740 1 --2- 172.21.15.18:0/1139262621 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f764fa3560 0x55f764f9f9c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdcf49db740 1 --2- 172.21.15.18:0/1139262621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f764f37ae0 0x55f764f3fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdcf49db740 1 --2- 172.21.15.18:0/1139262621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f764f32970 0x55f764f43650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdccaffd640 0 client.15093 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:49:07.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdcf49db740 1 --2- 172.21.15.18:0/1139262621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f764f31f80 0x55f764f43110 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdcf49db740 1 -- 172.21.15.18:0/1139262621 >> 172.21.15.18:0/1139262621 conn(0x55f764f2d4f0 msgr2=0x55f764f3bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdcf49db740 1 -- 172.21.15.18:0/1139262621 shutdown_connections 2024-02-17T12:49:07.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdcf49db740 1 -- 172.21.15.18:0/1139262621 wait complete. 2024-02-17T12:49:07.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.670+0000 7fdcf49db740 20 client.15093 trim_cache size 0 max 16384 2024-02-17T12:49:07.670 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:49:07.670 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:49:07.670 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 3, 2024-02-17T12:49:07.671 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24448, 2024-02-17T12:49:07.671 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-02-17T12:49:07.671 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-02-17T12:49:07.671 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-02-17T12:49:07.671 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 77.961560188999997, 2024-02-17T12:49:07.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 25, 2024-02-17T12:49:07.671 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 88, 2024-02-17T12:49:07.671 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 86, 2024-02-17T12:49:07.671 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 104.21180306700001 2024-02-17T12:49:07.671 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:49:07.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.671+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/683795182 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdcd40a2150 msgr2=0x7fdcd40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.671+0000 7fdcf1e0f640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdcd40a2150 0x7fdcd40a4610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdce4097040 tx=0x7fdce406cbf0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.671+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/683795182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdcec101c30 msgr2=0x7fdcec19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.671+0000 7fdcf1e0f640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdcec101c30 0x7fdcec19e720 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c970 tx=0x7fdce400cd70 comp rx=0 tx=0).stop 2024-02-17T12:49:07.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.671+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/683795182 shutdown_connections 2024-02-17T12:49:07.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.671+0000 7fdcf1e0f640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdcd40a2150 0x7fdcd40a4610 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.671+0000 7fdcf1e0f640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdcec101c30 0x7fdcec19e720 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.671+0000 7fdcf1e0f640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcec101270 0x7fdcec1976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.671+0000 7fdcf1e0f640 1 --2- 172.21.15.18:0/683795182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcec100080 0x7fdcec197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.671+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/683795182 >> 172.21.15.18:0/683795182 conn(0x7fdcec0fb610 msgr2=0x7fdcec0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.671+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/683795182 shutdown_connections 2024-02-17T12:49:07.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.671+0000 7fdcf1e0f640 1 -- 172.21.15.18:0/683795182 wait complete. 2024-02-17T12:49:07.673 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:07 smithi018 ceph-mon[32625]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s; 84 B/s, 0 keys/s, 1 objects/s recovering 2024-02-17T12:49:07.673 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:07 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3629138981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:07.673 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:07 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/103374070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:07.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:07 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1095076514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:07.674 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-02-17T12:49:07.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:07 smithi146 ceph-mon[36454]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s; 84 B/s, 0 keys/s, 1 objects/s recovering 2024-02-17T12:49:07.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:07 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3629138981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:07.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:07 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/103374070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:07.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:07 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1095076514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:07.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.786+0000 7f7513a4e640 1 -- 172.21.15.18:0/519005121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c100a10 msgr2=0x7f750c100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.786+0000 7f7513a4e640 1 --2- 172.21.15.18:0/519005121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c100a10 0x7f750c100e90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7500009900 tx=0x7f750002bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.786+0000 7f7513a4e640 1 -- 172.21.15.18:0/519005121 shutdown_connections 2024-02-17T12:49:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.786+0000 7f7513a4e640 1 --2- 172.21.15.18:0/519005121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f750c1013d0 0x7f750c107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.786+0000 7f7513a4e640 1 --2- 172.21.15.18:0/519005121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c100a10 0x7f750c100e90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.786+0000 7f7513a4e640 1 --2- 172.21.15.18:0/519005121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f750c0ff820 0x7f750c0ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.786+0000 7f7513a4e640 1 -- 172.21.15.18:0/519005121 >> 172.21.15.18:0/519005121 conn(0x7f750c0fadb0 msgr2=0x7f750c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.786+0000 7f7513a4e640 1 -- 172.21.15.18:0/519005121 shutdown_connections 2024-02-17T12:49:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.786+0000 7f7513a4e640 1 -- 172.21.15.18:0/519005121 wait complete. 2024-02-17T12:49:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.786+0000 7f7513a4e640 1 Processor -- start 2024-02-17T12:49:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.786+0000 7f7513a4e640 1 -- start start 2024-02-17T12:49:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f7513a4e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0ff820 0x7f750c1037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f7513a4e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f750c100a10 0x7f750c101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f7513a4e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f750c1013d0 0x7f750c102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f7513a4e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c102a90 con 0x7f750c0ff820 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f7513a4e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c102c00 con 0x7f750c1013d0 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f7513a4e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c1a1270 con 0x7f750c100a10 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f75117ea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0ff820 0x7f750c1037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f75117ea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0ff820 0x7f750c1037b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53522/0 (socket says 172.21.15.18:53522) 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f75117ea640 1 -- 172.21.15.18:0/2400938229 learned_addr learned my addr 172.21.15.18:0/2400938229 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f7510fe9640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f750c100a10 0x7f750c101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f7511feb640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f750c1013d0 0x7f750c102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f75117ea640 1 -- 172.21.15.18:0/2400938229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f750c100a10 msgr2=0x7f750c101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f75117ea640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f750c100a10 0x7f750c101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f75117ea640 1 -- 172.21.15.18:0/2400938229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f750c1013d0 msgr2=0x7f750c102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f75117ea640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f750c1013d0 0x7f750c102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f75117ea640 1 -- 172.21.15.18:0/2400938229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7500009560 con 0x7f750c0ff820 2024-02-17T12:49:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f7511feb640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f750c1013d0 0x7f750c102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:07.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f75117ea640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0ff820 0x7f750c1037b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f750400eab0 tx=0x7f750400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f750a7fc640 1 -- 172.21.15.18:0/2400938229 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750400ce40 con 0x7f750c0ff820 2024-02-17T12:49:07.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f7513a4e640 1 -- 172.21.15.18:0/2400938229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f750c1a1520 con 0x7f750c0ff820 2024-02-17T12:49:07.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f750a7fc640 1 -- 172.21.15.18:0/2400938229 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7504011950 con 0x7f750c0ff820 2024-02-17T12:49:07.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f750a7fc640 1 -- 172.21.15.18:0/2400938229 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750401ab40 con 0x7f750c0ff820 2024-02-17T12:49:07.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.787+0000 7f7513a4e640 1 -- 172.21.15.18:0/2400938229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750c1a1a00 con 0x7f750c0ff820 2024-02-17T12:49:07.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.790+0000 7f750a7fc640 1 -- 172.21.15.18:0/2400938229 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f7504011ac0 con 0x7f750c0ff820 2024-02-17T12:49:07.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.790+0000 7f7516628740 1 Processor -- start 2024-02-17T12:49:07.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7516628740 1 -- start start 2024-02-17T12:49:07.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7516628740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7516628740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936ba470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7516628740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 0x5647936bcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7516628740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564793670cd0 con 0x5647936baae0 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7516628740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564793623f20 con 0x5647936b4f80 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7516628740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564793621450 con 0x5647936b5970 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f750a7fc640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f74f40a2150 0x7f74f40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7510fe9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 0x5647936bcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7510fe9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 0x5647936bcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53524/0 (socket says 172.21.15.18:53524) 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7510fe9640 1 -- 172.21.15.18:0/2090015842 learned_addr learned my addr 172.21.15.18:0/2090015842 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7511feb640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f74f40a2150 0x7f74f40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f750a7fc640 1 -- 172.21.15.18:0/2400938229 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f75040ce430 con 0x7f750c0ff820 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f75117ea640 1 --2- 172.21.15.18:0/2090015842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936ba470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7511feb640 1 --2- 172.21.15.18:0/2090015842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7510fe9640 1 -- 172.21.15.18:0/2090015842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 msgr2=0x5647936ba470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7510fe9640 1 --2- 172.21.15.18:0/2090015842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936ba470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7510fe9640 1 -- 172.21.15.18:0/2090015842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 msgr2=0x5647936b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7510fe9640 1 --2- 172.21.15.18:0/2090015842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7510fe9640 1 -- 172.21.15.18:0/2090015842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647935d1f20 con 0x5647936baae0 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7511feb640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f74f40a2150 0x7f74f40a4610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5647936b61d0 tx=0x7f74fc00c2f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.791+0000 7f7510fe9640 1 --2- 172.21.15.18:0/2090015842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 0x5647936bcf70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7500009900 tx=0x7f7500005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ac216636da7caf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.792+0000 7f74f37fe640 1 -- 172.21.15.18:0/2090015842 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750002cbd0 con 0x5647936baae0 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.792+0000 7f74f37fe640 1 -- 172.21.15.18:0/2090015842 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7500004030 con 0x5647936baae0 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.792+0000 7f74f37fe640 1 -- 172.21.15.18:0/2090015842 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500005270 con 0x5647936baae0 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.792+0000 7f7516628740 1 -- 172.21.15.18:0/2090015842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 msgr2=0x5647936bcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.792+0000 7f7516628740 1 --2- 172.21.15.18:0/2090015842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 0x5647936bcf70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7500009900 tx=0x7f7500005c00 comp rx=0 tx=0).stop 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.792+0000 7f7516628740 1 -- 172.21.15.18:0/2090015842 shutdown_connections 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.792+0000 7f7516628740 1 --2- 172.21.15.18:0/2090015842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 0x5647936bcf70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.792+0000 7f7516628740 1 --2- 172.21.15.18:0/2090015842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936ba470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.792+0000 7f7516628740 1 --2- 172.21.15.18:0/2090015842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.792+0000 7f7516628740 1 -- 172.21.15.18:0/2090015842 >> 172.21.15.18:0/2090015842 conn(0x5647936b04f0 msgr2=0x5647936b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.792+0000 7f7516628740 1 -- 172.21.15.18:0/2090015842 shutdown_connections 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.792+0000 7f7516628740 1 -- 172.21.15.18:0/2090015842 wait complete. 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f7516628740 1 -- start start 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f7516628740 1 Processor -- start 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f7516628740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b4f80 0x5647936bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f7516628740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936bf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f7516628740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936baae0 0x5647936c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f7516628740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479361cc30 con 0x5647936b4f80 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f7516628740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647935c1d30 con 0x5647936baae0 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f7516628740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647932f0310 con 0x5647936b5970 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f75117ea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b4f80 0x5647936bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f7516628740 10 client.0 _ll_register_callbacks cb 0x5647935c9d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:07.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f75117ea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b4f80 0x5647936bf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53538/0 (socket says 172.21.15.18:53538) 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f75117ea640 1 -- 172.21.15.18:0/2905660979 learned_addr learned my addr 172.21.15.18:0/2905660979 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f75117ea640 1 -- 172.21.15.18:0/2905660979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 msgr2=0x5647936bf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f7511feb640 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936baae0 0x5647936c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.793+0000 7f7510fe9640 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936bf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f75117ea640 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936bf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f75117ea640 1 -- 172.21.15.18:0/2905660979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936baae0 msgr2=0x5647936c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f75117ea640 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936baae0 0x5647936c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f75117ea640 1 -- 172.21.15.18:0/2905660979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750c068f40 con 0x5647936b4f80 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f7510fe9640 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936bf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f7511feb640 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936baae0 0x5647936c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f75117ea640 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b4f80 0x5647936bf0a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f750409acd0 tx=0x7f750409a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad698e9a247ec595 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f74f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:07.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f74f0ff9640 1 -- 172.21.15.18:0/2905660979 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040b7070 con 0x5647936b4f80 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f74f0ff9640 1 -- 172.21.15.18:0/2905660979 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f750409f120 con 0x5647936b4f80 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f7516628740 1 -- 172.21.15.18:0/2905660979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56479331c6c0 con 0x5647936b4f80 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f74f0ff9640 1 -- 172.21.15.18:0/2905660979 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75040a3770 con 0x5647936b4f80 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f7516628740 10 client.15105 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f7516628740 1 -- 172.21.15.18:0/2905660979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56479361b2b0 con 0x5647936b4f80 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f74f0ff9640 1 -- 172.21.15.18:0/2905660979 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75040c9080 con 0x5647936b4f80 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f74f0ff9640 1 -- 172.21.15.18:0/2905660979 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7504018020 con 0x5647936b4f80 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f7516628740 10 client.15105 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f7516628740 10 client.15105 resolve_mds: resolved name 'e' to daemon mds.e 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f7516628740 20 client.15105 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f7516628740 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5647936cb9a0 0x5647936cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f7516628740 4 client.15105 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.794+0000 7f7516628740 1 -- 172.21.15.18:0/2905660979 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56479336d270 con 0x5647936cb9a0 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.795+0000 7f7510fe9640 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5647936cb9a0 0x5647936cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.795+0000 7f7510fe9640 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5647936cb9a0 0x5647936cfe20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69890d52a370c5d8 server_cookie=f8c489367b0a5e90 in_seq=0 out_seq=0 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.795+0000 7f74f0ff9640 10 client.15105 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f74f0ff9640 1 -- 172.21.15.18:0/2905660979 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56479336d270 con 0x5647936cb9a0 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f74f0ff9640 10 client.15105 handle_command_reply: tid=0 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f7516628740 1 client.15105 shutdown 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f7516628740 1 -- 172.21.15.18:0/2905660979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b4f80 msgr2=0x5647936bf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f7516628740 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b4f80 0x5647936bf0a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f750409acd0 tx=0x7f750409a9f0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f7516628740 1 -- 172.21.15.18:0/2905660979 shutdown_connections 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f7516628740 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5647936cb9a0 0x5647936cfe20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f7516628740 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936baae0 0x5647936c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f7516628740 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936bf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f74f0ff9640 0 client.15105 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f7516628740 1 --2- 172.21.15.18:0/2905660979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b4f80 0x5647936bf0a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f7516628740 1 -- 172.21.15.18:0/2905660979 >> 172.21.15.18:0/2905660979 conn(0x5647936b04f0 msgr2=0x5647936af7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f7516628740 1 -- 172.21.15.18:0/2905660979 shutdown_connections 2024-02-17T12:49:07.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f7516628740 1 -- 172.21.15.18:0/2905660979 wait complete. 2024-02-17T12:49:07.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.796+0000 7f7516628740 20 client.15105 trim_cache size 0 max 16384 2024-02-17T12:49:07.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7516628740 1 Processor -- start 2024-02-17T12:49:07.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7516628740 1 -- start start 2024-02-17T12:49:07.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7516628740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7516628740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b5970 0x5647936c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7516628740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936baae0 0x5647936aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7516628740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647935a2490 con 0x5647936b5970 2024-02-17T12:49:07.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7516628740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647936cd980 con 0x5647936b4f80 2024-02-17T12:49:07.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7516628740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647936868c0 con 0x5647936baae0 2024-02-17T12:49:07.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7510fe9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b5970 0x5647936c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7510fe9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b5970 0x5647936c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53548/0 (socket says 172.21.15.18:53548) 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7510fe9640 1 -- 172.21.15.18:0/1917167851 learned_addr learned my addr 172.21.15.18:0/1917167851 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f75117ea640 1 --2- 172.21.15.18:0/1917167851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7510fe9640 1 -- 172.21.15.18:0/1917167851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936baae0 msgr2=0x5647936aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7510fe9640 1 --2- 172.21.15.18:0/1917167851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936baae0 0x5647936aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7510fe9640 1 -- 172.21.15.18:0/1917167851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 msgr2=0x5647936d4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7510fe9640 1 --2- 172.21.15.18:0/1917167851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936d4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f7510fe9640 1 -- 172.21.15.18:0/1917167851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647935b7a20 con 0x5647936b5970 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.812+0000 7f75117ea640 1 --2- 172.21.15.18:0/1917167851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936d4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f7510fe9640 1 --2- 172.21.15.18:0/1917167851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b5970 0x5647936c65e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7500009900 tx=0x7f7500005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db866e1abf371679 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f74f2ffd640 1 -- 172.21.15.18:0/1917167851 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75000041a0 con 0x5647936b5970 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f74f2ffd640 1 -- 172.21.15.18:0/1917167851 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7500004df0 con 0x5647936b5970 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f74f2ffd640 1 -- 172.21.15.18:0/1917167851 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500003850 con 0x5647936b5970 2024-02-17T12:49:07.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f7516628740 1 -- 172.21.15.18:0/1917167851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b5970 msgr2=0x5647936c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f7516628740 1 --2- 172.21.15.18:0/1917167851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b5970 0x5647936c65e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7500009900 tx=0x7f7500005e80 comp rx=0 tx=0).stop 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f7516628740 1 -- 172.21.15.18:0/1917167851 shutdown_connections 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f7516628740 1 --2- 172.21.15.18:0/1917167851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936baae0 0x5647936aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f7516628740 1 --2- 172.21.15.18:0/1917167851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936b5970 0x5647936c65e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f7516628740 1 --2- 172.21.15.18:0/1917167851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936d4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f7516628740 1 -- 172.21.15.18:0/1917167851 >> 172.21.15.18:0/1917167851 conn(0x5647936b04f0 msgr2=0x5647936d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f7516628740 1 -- 172.21.15.18:0/1917167851 shutdown_connections 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f7516628740 1 -- 172.21.15.18:0/1917167851 wait complete. 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.813+0000 7f7516628740 1 -- start start 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7516628740 1 Processor -- start 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7516628740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936c3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7516628740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7516628740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 0x564793721ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7516628740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647936c6760 con 0x5647936baae0 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7516628740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647936c68d0 con 0x5647936b4f80 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7516628740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564793722020 con 0x5647936b5970 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7516628740 10 client.0 _ll_register_callbacks cb 0x7f7500005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7511feb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 0x564793721ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7511feb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 0x564793721ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53564/0 (socket says 172.21.15.18:53564) 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7511feb640 1 -- 172.21.15.18:0/1376937974 learned_addr learned my addr 172.21.15.18:0/1376937974 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f75117ea640 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7510fe9640 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7511feb640 1 -- 172.21.15.18:0/1376937974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 msgr2=0x5647936c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7511feb640 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936c3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7511feb640 1 -- 172.21.15.18:0/1376937974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 msgr2=0x5647936c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7511feb640 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936c3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7511feb640 1 -- 172.21.15.18:0/1376937974 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647935d1f20 con 0x5647936baae0 2024-02-17T12:49:07.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f75117ea640 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.814+0000 7f7511feb640 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 0x564793721ae0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f74fc0127f0 tx=0x7f74fc00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b292b7deb292ef server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f74f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f74f0ff9640 1 -- 172.21.15.18:0/1376937974 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc011b40 con 0x5647936baae0 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f74f0ff9640 1 -- 172.21.15.18:0/1376937974 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f74fc011ca0 con 0x5647936baae0 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f7516628740 1 -- 172.21.15.18:0/1376937974 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647936cb6b0 con 0x5647936baae0 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f74f0ff9640 1 -- 172.21.15.18:0/1376937974 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74fc011e60 con 0x5647936baae0 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f7516628740 10 client.15111 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f7516628740 1 -- 172.21.15.18:0/1376937974 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5647935dd780 con 0x5647936baae0 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f74f0ff9640 1 -- 172.21.15.18:0/1376937974 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5647936cb6b0 con 0x5647936baae0 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f74f0ff9640 1 -- 172.21.15.18:0/1376937974 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f74fc020610 con 0x5647936baae0 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f7516628740 10 client.15111 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f7516628740 10 client.15111 resolve_mds: resolved name 'e' to daemon mds.e 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f7516628740 20 client.15111 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:07.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f7516628740 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x564793727290 0x564793723710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f7516628740 4 client.15111 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.815+0000 7f7516628740 1 -- 172.21.15.18:0/1376937974 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f7500004030 con 0x564793727290 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f75117ea640 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x564793727290 0x564793723710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f74f0ff9640 10 client.15111 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f75117ea640 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x564793727290 0x564793723710 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60460df82d679caf server_cookie=40a903b22dc07060 in_seq=0 out_seq=0 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f74f0ff9640 1 -- 172.21.15.18:0/1376937974 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f7500004030 con 0x564793727290 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f74f0ff9640 10 client.15111 handle_command_reply: tid=0 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f7516628740 1 client.15111 shutdown 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f7516628740 1 -- 172.21.15.18:0/1376937974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 msgr2=0x564793721ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f7516628740 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 0x564793721ae0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f74fc0127f0 tx=0x7f74fc00cba0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f7516628740 1 -- 172.21.15.18:0/1376937974 shutdown_connections 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f7516628740 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x564793727290 0x564793723710 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f7516628740 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647936baae0 0x564793721ae0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f74f0ff9640 0 client.15111 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:49:07.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f7516628740 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5647936b5970 0x5647936c3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f7516628740 1 --2- 172.21.15.18:0/1376937974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5647936b4f80 0x5647936c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.816+0000 7f7516628740 1 -- 172.21.15.18:0/1376937974 >> 172.21.15.18:0/1376937974 conn(0x5647936b04f0 msgr2=0x5647936d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7516628740 1 -- 172.21.15.18:0/1376937974 shutdown_connections 2024-02-17T12:49:07.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7516628740 1 -- 172.21.15.18:0/1376937974 wait complete. 2024-02-17T12:49:07.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7516628740 20 client.15111 trim_cache size 0 max 16384 2024-02-17T12:49:07.818 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:49:07.818 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:49:07.818 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 2, 2024-02-17T12:49:07.818 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24455, 2024-02-17T12:49:07.819 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-02-17T12:49:07.819 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-02-17T12:49:07.819 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-02-17T12:49:07.819 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 79.114353863000005, 2024-02-17T12:49:07.819 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 25, 2024-02-17T12:49:07.819 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 88, 2024-02-17T12:49:07.819 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 86, 2024-02-17T12:49:07.819 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 106.99763860100001 2024-02-17T12:49:07.819 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:49:07.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7513a4e640 1 -- 172.21.15.18:0/2400938229 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f74f40a2150 msgr2=0x7f74f40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7513a4e640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f74f40a2150 0x7f74f40a4610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5647936b61d0 tx=0x7f74fc00c2f0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7513a4e640 1 -- 172.21.15.18:0/2400938229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0ff820 msgr2=0x7f750c1037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7513a4e640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0ff820 0x7f750c1037b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f750400eab0 tx=0x7f750400ef70 comp rx=0 tx=0).stop 2024-02-17T12:49:07.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7513a4e640 1 -- 172.21.15.18:0/2400938229 shutdown_connections 2024-02-17T12:49:07.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7513a4e640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f74f40a2150 0x7f74f40a4610 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7513a4e640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f750c1013d0 0x7f750c102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7513a4e640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f750c100a10 0x7f750c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7513a4e640 1 --2- 172.21.15.18:0/2400938229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0ff820 0x7f750c1037b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7513a4e640 1 -- 172.21.15.18:0/2400938229 >> 172.21.15.18:0/2400938229 conn(0x7f750c0fadb0 msgr2=0x7f750c0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7513a4e640 1 -- 172.21.15.18:0/2400938229 shutdown_connections 2024-02-17T12:49:07.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.817+0000 7f7513a4e640 1 -- 172.21.15.18:0/2400938229 wait complete. 2024-02-17T12:49:07.822 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-02-17T12:49:07.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:07 smithi119 ceph-mon[36188]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s; 84 B/s, 0 keys/s, 1 objects/s recovering 2024-02-17T12:49:07.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:07 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3629138981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:07.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:07 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/103374070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:07.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:07 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1095076514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:07.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.932+0000 7f94a33c2640 1 -- 172.21.15.18:0/3165440302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c101270 msgr2=0x7f949c1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.932+0000 7f94a33c2640 1 --2- 172.21.15.18:0/3165440302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c101270 0x7f949c1016f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9490009900 tx=0x7f949002bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:07.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.932+0000 7f94a33c2640 1 -- 172.21.15.18:0/3165440302 shutdown_connections 2024-02-17T12:49:07.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 --2- 172.21.15.18:0/3165440302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f949c101c30 0x7f949c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 --2- 172.21.15.18:0/3165440302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c101270 0x7f949c1016f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 --2- 172.21.15.18:0/3165440302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f949c100080 0x7f949c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 -- 172.21.15.18:0/3165440302 >> 172.21.15.18:0/3165440302 conn(0x7f949c0fb610 msgr2=0x7f949c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 -- 172.21.15.18:0/3165440302 shutdown_connections 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 -- 172.21.15.18:0/3165440302 wait complete. 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 Processor -- start 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 -- start start 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f949c100080 0x7f949c104010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f949c101270 0x7f949c102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c101c30 0x7f949c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c1032f0 con 0x7f949c101c30 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c103460 con 0x7f949c100080 2024-02-17T12:49:07.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a33c2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c1a1240 con 0x7f949c101270 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a195f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c101c30 0x7f949c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a195f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c101c30 0x7f949c102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53572/0 (socket says 172.21.15.18:53572) 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.933+0000 7f94a195f640 1 -- 172.21.15.18:0/2083477339 learned_addr learned my addr 172.21.15.18:0/2083477339 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f94a095d640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f949c101270 0x7f949c102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f94a115e640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f949c100080 0x7f949c104010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f94a195f640 1 -- 172.21.15.18:0/2083477339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f949c101270 msgr2=0x7f949c102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f94a195f640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f949c101270 0x7f949c102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f94a195f640 1 -- 172.21.15.18:0/2083477339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f949c100080 msgr2=0x7f949c104010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f94a195f640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f949c100080 0x7f949c104010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f94a195f640 1 -- 172.21.15.18:0/2083477339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9490009560 con 0x7f949c101c30 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f94a195f640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c101c30 0x7f949c102b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f949400c9a0 tx=0x7f949400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f949a7fc640 1 -- 172.21.15.18:0/2083477339 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494007bb0 con 0x7f949c101c30 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f949a7fc640 1 -- 172.21.15.18:0/2083477339 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9494007d10 con 0x7f949c101c30 2024-02-17T12:49:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f94a33c2640 1 -- 172.21.15.18:0/2083477339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c1a14f0 con 0x7f949c101c30 2024-02-17T12:49:07.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f949a7fc640 1 -- 172.21.15.18:0/2083477339 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494018800 con 0x7f949c101c30 2024-02-17T12:49:07.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.934+0000 7f94a33c2640 1 -- 172.21.15.18:0/2083477339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c1a19b0 con 0x7f949c101c30 2024-02-17T12:49:07.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.936+0000 7f949a7fc640 1 -- 172.21.15.18:0/2083477339 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f949400fa30 con 0x7f949c101c30 2024-02-17T12:49:07.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a5f8b740 1 Processor -- start 2024-02-17T12:49:07.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a5f8b740 1 -- start start 2024-02-17T12:49:07.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a5f8b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c806f80 0x55ee4c8073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a5f8b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c807970 0x55ee4c80c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a5f8b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c80cae0 0x55ee4c80ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a5f8b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c7c2cd0 con 0x55ee4c80cae0 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a5f8b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c775f20 con 0x55ee4c806f80 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a5f8b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c773450 con 0x55ee4c807970 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f949a7fc640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f94840a2150 0x7f94840a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a095d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c80cae0 0x55ee4c80ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a115e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c807970 0x55ee4c80c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a095d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c80cae0 0x55ee4c80ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53586/0 (socket says 172.21.15.18:53586) 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a095d640 1 -- 172.21.15.18:0/913562894 learned_addr learned my addr 172.21.15.18:0/913562894 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a195f640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f94840a2150 0x7f94840a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.937+0000 7f94a095d640 1 --2- 172.21.15.18:0/913562894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c806f80 0x55ee4c8073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.938+0000 7f949a7fc640 1 -- 172.21.15.18:0/2083477339 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f94940cdf70 con 0x7f949c101c30 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.938+0000 7f94a095d640 1 -- 172.21.15.18:0/913562894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c807970 msgr2=0x55ee4c80c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.938+0000 7f94a095d640 1 --2- 172.21.15.18:0/913562894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c807970 0x55ee4c80c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.938+0000 7f94a095d640 1 -- 172.21.15.18:0/913562894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c806f80 msgr2=0x55ee4c8073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.938+0000 7f94a095d640 1 --2- 172.21.15.18:0/913562894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c806f80 0x55ee4c8073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.938+0000 7f94a095d640 1 -- 172.21.15.18:0/913562894 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee4c723f20 con 0x55ee4c80cae0 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.938+0000 7f94a115e640 1 --2- 172.21.15.18:0/913562894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c807970 0x55ee4c80c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.938+0000 7f94a095d640 1 --2- 172.21.15.18:0/913562894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c80cae0 0x55ee4c80ef70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9490005ec0 tx=0x7f9490005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54545da58c501f70 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.938+0000 7f94a195f640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f94840a2150 0x7f94840a4610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f949400c970 tx=0x7f9494069d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.938+0000 7f94837fe640 1 -- 172.21.15.18:0/913562894 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490047070 con 0x55ee4c80cae0 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.938+0000 7f94837fe640 1 -- 172.21.15.18:0/913562894 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9490004510 con 0x55ee4c80cae0 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.938+0000 7f94837fe640 1 -- 172.21.15.18:0/913562894 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490005420 con 0x55ee4c80cae0 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.939+0000 7f94a5f8b740 1 -- 172.21.15.18:0/913562894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c80cae0 msgr2=0x55ee4c80ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.939+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/913562894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c80cae0 0x55ee4c80ef70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9490005ec0 tx=0x7f9490005a40 comp rx=0 tx=0).stop 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.939+0000 7f94a5f8b740 1 -- 172.21.15.18:0/913562894 shutdown_connections 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.939+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/913562894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c80cae0 0x55ee4c80ef70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.939+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/913562894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c807970 0x55ee4c80c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.939+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/913562894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c806f80 0x55ee4c8073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.939+0000 7f94a5f8b740 1 -- 172.21.15.18:0/913562894 >> 172.21.15.18:0/913562894 conn(0x55ee4c8024f0 msgr2=0x55ee4c8028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.939+0000 7f94a5f8b740 1 -- 172.21.15.18:0/913562894 shutdown_connections 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.939+0000 7f94a5f8b740 1 -- 172.21.15.18:0/913562894 wait complete. 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.939+0000 7f94a5f8b740 1 -- start start 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.939+0000 7f94a5f8b740 1 Processor -- start 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a5f8b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 0x55ee4c811230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a5f8b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c807970 0x55ee4c811770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a5f8b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c80cae0 0x55ee4c8187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a5f8b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c76ec30 con 0x55ee4c806f80 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a5f8b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c713d30 con 0x55ee4c80cae0 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a5f8b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c442310 con 0x55ee4c807970 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a5f8b740 10 client.0 _ll_register_callbacks cb 0x55ee4c71bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a095d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 0x55ee4c811230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a095d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 0x55ee4c811230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53590/0 (socket says 172.21.15.18:53590) 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a095d640 1 -- 172.21.15.18:0/618094064 learned_addr learned my addr 172.21.15.18:0/618094064 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a095d640 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c80cae0 0x55ee4c8187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a095d640 1 -- 172.21.15.18:0/618094064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c807970 msgr2=0x55ee4c811770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a115e640 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c807970 0x55ee4c811770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a095d640 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c807970 0x55ee4c811770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a095d640 1 -- 172.21.15.18:0/618094064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c80cae0 msgr2=0x55ee4c8187b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a095d640 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c80cae0 0x55ee4c8187b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a095d640 1 -- 172.21.15.18:0/618094064 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c108dc0 con 0x55ee4c806f80 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a095d640 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 0x55ee4c811230 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f949002c170 tx=0x7f9490005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde5b45cefb489dc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f9480ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f9480ff9640 1 -- 172.21.15.18:0/618094064 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490047070 con 0x55ee4c806f80 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f9480ff9640 1 -- 172.21.15.18:0/618094064 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9490005540 con 0x55ee4c806f80 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a5f8b740 1 -- 172.21.15.18:0/618094064 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee4c46e6c0 con 0x55ee4c806f80 2024-02-17T12:49:07.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f9480ff9640 1 -- 172.21.15.18:0/618094064 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9490005700 con 0x55ee4c806f80 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a5f8b740 10 client.15123 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.940+0000 7f94a5f8b740 1 -- 172.21.15.18:0/618094064 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ee4c76d2b0 con 0x55ee4c806f80 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.941+0000 7f9480ff9640 1 -- 172.21.15.18:0/618094064 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ee4c46e6c0 con 0x55ee4c806f80 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.941+0000 7f9480ff9640 1 -- 172.21.15.18:0/618094064 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f949004e070 con 0x55ee4c806f80 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.941+0000 7f94a5f8b740 10 client.15123 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.941+0000 7f94a5f8b740 10 client.15123 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.941+0000 7f94a5f8b740 20 client.15123 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.941+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ee4c81cca0 0x55ee4c821ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.941+0000 7f94a5f8b740 4 client.15123 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.941+0000 7f94a5f8b740 1 -- 172.21.15.18:0/618094064 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee4c4bf270 con 0x55ee4c81cca0 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.941+0000 7f94a115e640 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ee4c81cca0 0x55ee4c821ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.941+0000 7f94a115e640 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ee4c81cca0 0x55ee4c821ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c32949cc8ad1406 server_cookie=e52d2a4e5bb7e1c2 in_seq=0 out_seq=0 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.941+0000 7f9480ff9640 10 client.15123 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:49:07.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f9480ff9640 1 -- 172.21.15.18:0/618094064 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee4c4bf270 con 0x55ee4c81cca0 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f9480ff9640 10 client.15123 handle_command_reply: tid=0 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f94a5f8b740 1 client.15123 shutdown 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f94a5f8b740 1 -- 172.21.15.18:0/618094064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 msgr2=0x55ee4c811230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 0x55ee4c811230 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f949002c170 tx=0x7f9490005d50 comp rx=0 tx=0).stop 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f94a5f8b740 1 -- 172.21.15.18:0/618094064 shutdown_connections 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ee4c81cca0 0x55ee4c821ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c80cae0 0x55ee4c8187b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c807970 0x55ee4c811770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f9480ff9640 0 client.15123 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/618094064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 0x55ee4c811230 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f94a5f8b740 1 -- 172.21.15.18:0/618094064 >> 172.21.15.18:0/618094064 conn(0x55ee4c8024f0 msgr2=0x55ee4c8017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f94a5f8b740 1 -- 172.21.15.18:0/618094064 shutdown_connections 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.942+0000 7f94a5f8b740 1 -- 172.21.15.18:0/618094064 wait complete. 2024-02-17T12:49:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.943+0000 7f94a5f8b740 20 client.15123 trim_cache size 0 max 16384 2024-02-17T12:49:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.958+0000 7f94a5f8b740 1 Processor -- start 2024-02-17T12:49:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.958+0000 7f94a5f8b740 1 -- start start 2024-02-17T12:49:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.958+0000 7f94a5f8b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c806f80 0x55ee4c7fd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.958+0000 7f94a5f8b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c807970 0x55ee4c818740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.958+0000 7f94a5f8b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c80cae0 0x55ee4c808fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.958+0000 7f94a5f8b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c6f4490 con 0x55ee4c807970 2024-02-17T12:49:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.958+0000 7f94a5f8b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c819430 con 0x55ee4c806f80 2024-02-17T12:49:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.958+0000 7f94a5f8b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c802f40 con 0x55ee4c80cae0 2024-02-17T12:49:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.958+0000 7f94a115e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c807970 0x55ee4c818740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.958+0000 7f94a115e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c807970 0x55ee4c818740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53600/0 (socket says 172.21.15.18:53600) 2024-02-17T12:49:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.958+0000 7f94a115e640 1 -- 172.21.15.18:0/4003654374 learned_addr learned my addr 172.21.15.18:0/4003654374 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a115e640 1 -- 172.21.15.18:0/4003654374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c80cae0 msgr2=0x55ee4c808fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a115e640 1 --2- 172.21.15.18:0/4003654374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c80cae0 0x55ee4c808fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a115e640 1 -- 172.21.15.18:0/4003654374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c806f80 msgr2=0x55ee4c7fd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a115e640 1 --2- 172.21.15.18:0/4003654374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c806f80 0x55ee4c7fd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a115e640 1 -- 172.21.15.18:0/4003654374 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee4c709a20 con 0x55ee4c807970 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a115e640 1 --2- 172.21.15.18:0/4003654374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c807970 0x55ee4c818740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55ee4c8083e0 tx=0x7f948c0097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2302a6c4fa1bab28 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f9482ffd640 1 -- 172.21.15.18:0/4003654374 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c018040 con 0x55ee4c807970 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f9482ffd640 1 -- 172.21.15.18:0/4003654374 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f948c0140d0 con 0x55ee4c807970 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f9482ffd640 1 -- 172.21.15.18:0/4003654374 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c005020 con 0x55ee4c807970 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a5f8b740 1 -- 172.21.15.18:0/4003654374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c807970 msgr2=0x55ee4c818740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/4003654374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c807970 0x55ee4c818740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55ee4c8083e0 tx=0x7f948c0097c0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a5f8b740 1 -- 172.21.15.18:0/4003654374 shutdown_connections 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/4003654374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c80cae0 0x55ee4c808fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/4003654374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c807970 0x55ee4c818740 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/4003654374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c806f80 0x55ee4c7fd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a5f8b740 1 -- 172.21.15.18:0/4003654374 >> 172.21.15.18:0/4003654374 conn(0x55ee4c8024f0 msgr2=0x55ee4c808470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.959+0000 7f94a5f8b740 1 -- 172.21.15.18:0/4003654374 shutdown_connections 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a5f8b740 1 -- 172.21.15.18:0/4003654374 wait complete. 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a5f8b740 1 -- start start 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a5f8b740 1 Processor -- start 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a5f8b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 0x55ee4c8181d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a5f8b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c807970 0x55ee4c818790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a5f8b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c80cae0 0x55ee4c815000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a5f8b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c80f100 con 0x55ee4c806f80 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a5f8b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c81d430 con 0x55ee4c807970 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a5f8b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c8157b0 con 0x55ee4c80cae0 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a5f8b740 10 client.0 _ll_register_callbacks cb 0x7f9490009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a095d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 0x55ee4c8181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a095d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 0x55ee4c8181d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53612/0 (socket says 172.21.15.18:53612) 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a095d640 1 -- 172.21.15.18:0/1910711182 learned_addr learned my addr 172.21.15.18:0/1910711182 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a095d640 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c80cae0 0x55ee4c815000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a115e640 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c807970 0x55ee4c818790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a095d640 1 -- 172.21.15.18:0/1910711182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c80cae0 msgr2=0x55ee4c815000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a095d640 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c80cae0 0x55ee4c815000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a095d640 1 -- 172.21.15.18:0/1910711182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c807970 msgr2=0x55ee4c818790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a095d640 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c807970 0x55ee4c818790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.960+0000 7f94a095d640 1 -- 172.21.15.18:0/1910711182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c009d00 con 0x55ee4c806f80 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f94a115e640 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c807970 0x55ee4c818790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f94a095d640 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 0x55ee4c8181d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9490009850 tx=0x7f9490005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e9f09295bc4411 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f9480ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f9480ff9640 1 -- 172.21.15.18:0/1910711182 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490047040 con 0x55ee4c806f80 2024-02-17T12:49:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f9480ff9640 1 -- 172.21.15.18:0/1910711182 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f949002cb90 con 0x55ee4c806f80 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f94a5f8b740 1 -- 172.21.15.18:0/1910711182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee4c81d570 con 0x55ee4c806f80 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f9480ff9640 1 -- 172.21.15.18:0/1910711182 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9490005540 con 0x55ee4c806f80 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f94a5f8b740 10 client.15129 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f94a5f8b740 1 -- 172.21.15.18:0/1910711182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ee4c72f780 con 0x55ee4c806f80 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f9480ff9640 1 -- 172.21.15.18:0/1910711182 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9490067020 con 0x55ee4c806f80 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f9480ff9640 1 -- 172.21.15.18:0/1910711182 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f949004e070 con 0x55ee4c806f80 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f94a5f8b740 10 client.15129 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f94a5f8b740 10 client.15129 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f94a5f8b740 20 client.15129 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ee4c878340 0x55ee4c8747c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f94a5f8b740 4 client.15129 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.961+0000 7f94a5f8b740 1 -- 172.21.15.18:0/1910711182 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f9490004510 con 0x55ee4c878340 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f94a115e640 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ee4c878340 0x55ee4c8747c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f94a115e640 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ee4c878340 0x55ee4c8747c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=874a9ff7cb703a5c server_cookie=da94368c86c51acf in_seq=0 out_seq=0 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f9480ff9640 10 client.15129 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f9480ff9640 1 -- 172.21.15.18:0/1910711182 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f9490004510 con 0x55ee4c878340 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f9480ff9640 10 client.15129 handle_command_reply: tid=0 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f94a5f8b740 1 client.15129 shutdown 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f94a5f8b740 1 -- 172.21.15.18:0/1910711182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 msgr2=0x55ee4c8181d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 0x55ee4c8181d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9490009850 tx=0x7f9490005e80 comp rx=0 tx=0).stop 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f94a5f8b740 1 -- 172.21.15.18:0/1910711182 shutdown_connections 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ee4c878340 0x55ee4c8747c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ee4c80cae0 0x55ee4c815000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee4c807970 0x55ee4c818790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f9480ff9640 0 client.15129 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f94a5f8b740 1 --2- 172.21.15.18:0/1910711182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee4c806f80 0x55ee4c8181d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.962+0000 7f94a5f8b740 1 -- 172.21.15.18:0/1910711182 >> 172.21.15.18:0/1910711182 conn(0x55ee4c8024f0 msgr2=0x55ee4c7fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a5f8b740 1 -- 172.21.15.18:0/1910711182 shutdown_connections 2024-02-17T12:49:07.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a5f8b740 1 -- 172.21.15.18:0/1910711182 wait complete. 2024-02-17T12:49:07.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a5f8b740 20 client.15129 trim_cache size 0 max 16384 2024-02-17T12:49:07.962 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:49:07.962 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:49:07.962 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 1, 2024-02-17T12:49:07.962 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24476, 2024-02-17T12:49:07.962 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-02-17T12:49:07.962 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-02-17T12:49:07.962 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-02-17T12:49:07.963 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 79.889016580000003, 2024-02-17T12:49:07.963 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 25, 2024-02-17T12:49:07.963 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 88, 2024-02-17T12:49:07.963 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 86, 2024-02-17T12:49:07.963 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 99.090666952999996 2024-02-17T12:49:07.963 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:49:07.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a33c2640 1 -- 172.21.15.18:0/2083477339 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f94840a2150 msgr2=0x7f94840a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a33c2640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f94840a2150 0x7f94840a4610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f949400c970 tx=0x7f9494069d00 comp rx=0 tx=0).stop 2024-02-17T12:49:07.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a33c2640 1 -- 172.21.15.18:0/2083477339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c101c30 msgr2=0x7f949c102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:07.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a33c2640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c101c30 0x7f949c102b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f949400c9a0 tx=0x7f949400ce60 comp rx=0 tx=0).stop 2024-02-17T12:49:07.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a33c2640 1 -- 172.21.15.18:0/2083477339 shutdown_connections 2024-02-17T12:49:07.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a33c2640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f94840a2150 0x7f94840a4610 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a33c2640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c101c30 0x7f949c102b90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a33c2640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f949c101270 0x7f949c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a33c2640 1 --2- 172.21.15.18:0/2083477339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f949c100080 0x7f949c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:07.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a33c2640 1 -- 172.21.15.18:0/2083477339 >> 172.21.15.18:0/2083477339 conn(0x7f949c0fb610 msgr2=0x7f949c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:07.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a33c2640 1 -- 172.21.15.18:0/2083477339 shutdown_connections 2024-02-17T12:49:07.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:07.963+0000 7f94a33c2640 1 -- 172.21.15.18:0/2083477339 wait complete. 2024-02-17T12:49:07.966 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-17T12:49:08.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.077+0000 7f65eac76640 1 -- 172.21.15.18:0/3073315027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e4101c30 msgr2=0x7f65e41084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:08.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.077+0000 7f65eac76640 1 --2- 172.21.15.18:0/3073315027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e4101c30 0x7f65e41084c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009900 tx=0x7f65cc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:08.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.077+0000 7f65eac76640 1 -- 172.21.15.18:0/3073315027 shutdown_connections 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.077+0000 7f65eac76640 1 --2- 172.21.15.18:0/3073315027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e4101c30 0x7f65e41084c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.077+0000 7f65eac76640 1 --2- 172.21.15.18:0/3073315027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e4101270 0x7f65e41016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.077+0000 7f65eac76640 1 --2- 172.21.15.18:0/3073315027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65e4100080 0x7f65e41004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.077+0000 7f65eac76640 1 -- 172.21.15.18:0/3073315027 >> 172.21.15.18:0/3073315027 conn(0x7f65e40fb610 msgr2=0x7f65e40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.077+0000 7f65eac76640 1 -- 172.21.15.18:0/3073315027 shutdown_connections 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.077+0000 7f65eac76640 1 -- 172.21.15.18:0/3073315027 wait complete. 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65eac76640 1 Processor -- start 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65eac76640 1 -- start start 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65eac76640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e4100080 0x7f65e4104010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65eac76640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65e4101270 0x7f65e4102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65eac76640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e4101c30 0x7f65e4102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65eac76640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e41032f0 con 0x7f65e4101c30 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65eac76640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4103460 con 0x7f65e4100080 2024-02-17T12:49:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65eac76640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e41a1210 con 0x7f65e4101270 2024-02-17T12:49:08.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65e9213640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e4101c30 0x7f65e4102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65e9213640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e4101c30 0x7f65e4102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53638/0 (socket says 172.21.15.18:53638) 2024-02-17T12:49:08.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65e9213640 1 -- 172.21.15.18:0/1216682175 learned_addr learned my addr 172.21.15.18:0/1216682175 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:08.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65e8a12640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e4100080 0x7f65e4104010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65e3fff640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65e4101270 0x7f65e4102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65e9213640 1 -- 172.21.15.18:0/1216682175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65e4101270 msgr2=0x7f65e4102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:08.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65e9213640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65e4101270 0x7f65e4102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65e9213640 1 -- 172.21.15.18:0/1216682175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e4100080 msgr2=0x7f65e4104010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:08.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65e9213640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e4100080 0x7f65e4104010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.078+0000 7f65e9213640 1 -- 172.21.15.18:0/1216682175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc009560 con 0x7f65e4101c30 2024-02-17T12:49:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.079+0000 7f65e8a12640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e4100080 0x7f65e4104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.079+0000 7f65e9213640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e4101c30 0x7f65e4102b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009430 tx=0x7f65cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.079+0000 7f65e1ffb640 1 -- 172.21.15.18:0/1216682175 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0044e0 con 0x7f65e4101c30 2024-02-17T12:49:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.079+0000 7f65e1ffb640 1 -- 172.21.15.18:0/1216682175 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65cc0054e0 con 0x7f65e4101c30 2024-02-17T12:49:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.079+0000 7f65eac76640 1 -- 172.21.15.18:0/1216682175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e41a1460 con 0x7f65e4101c30 2024-02-17T12:49:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.079+0000 7f65e1ffb640 1 -- 172.21.15.18:0/1216682175 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc03e450 con 0x7f65e4101c30 2024-02-17T12:49:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.079+0000 7f65eac76640 1 -- 172.21.15.18:0/1216682175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e41a19c0 con 0x7f65e4101c30 2024-02-17T12:49:08.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.081+0000 7f65e1ffb640 1 -- 172.21.15.18:0/1216682175 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f65cc003680 con 0x7f65e4101c30 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.081+0000 7f65ed849740 1 Processor -- start 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.081+0000 7f65ed849740 1 -- start start 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65ed849740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b0f80 0x5555a43b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65ed849740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b1970 0x5555a43b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e1ffb640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65d00a2250 0x7f65d00a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65ed849740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b6ae0 0x5555a43b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65ed849740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a436ccd0 con 0x5555a43b6ae0 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65ed849740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a431ff20 con 0x5555a43b1970 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65ed849740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a431d450 con 0x5555a43b0f80 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e9213640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b6ae0 0x5555a43b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e3fff640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65d00a2250 0x7f65d00a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e8a12640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b1970 0x5555a43b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e8a12640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b1970 0x5555a43b6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:46248/0 (socket says 172.21.15.18:46248) 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e8a12640 1 -- 172.21.15.18:0/1936860461 learned_addr learned my addr 172.21.15.18:0/1936860461 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e1ffb640 1 -- 172.21.15.18:0/1216682175 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f65cc0f05b0 con 0x7f65e4101c30 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e3fff640 1 --2- 172.21.15.18:0/1936860461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b0f80 0x5555a43b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e9213640 1 -- 172.21.15.18:0/1936860461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b0f80 msgr2=0x5555a43b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e9213640 1 --2- 172.21.15.18:0/1936860461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b0f80 0x5555a43b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e9213640 1 -- 172.21.15.18:0/1936860461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b1970 msgr2=0x5555a43b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e9213640 1 --2- 172.21.15.18:0/1936860461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b1970 0x5555a43b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e9213640 1 -- 172.21.15.18:0/1936860461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555a42cdf20 con 0x5555a43b6ae0 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e8a12640 1 --2- 172.21.15.18:0/1936860461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b1970 0x5555a43b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e3fff640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65d00a2250 0x7f65d00a4710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f65e4103d80 tx=0x7f65d4013a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65e9213640 1 --2- 172.21.15.18:0/1936860461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b6ae0 0x5555a43b8f70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0ee040 tx=0x7f65cc0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e07ef4ebdc0fa4b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65daffd640 1 -- 172.21.15.18:0/1936860461 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0bcbc0 con 0x5555a43b6ae0 2024-02-17T12:49:08.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65daffd640 1 -- 172.21.15.18:0/1936860461 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65cc0c28d0 con 0x5555a43b6ae0 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.082+0000 7f65daffd640 1 -- 172.21.15.18:0/1936860461 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0c36e0 con 0x5555a43b6ae0 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.083+0000 7f65ed849740 1 -- 172.21.15.18:0/1936860461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b6ae0 msgr2=0x5555a43b8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.083+0000 7f65ed849740 1 --2- 172.21.15.18:0/1936860461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b6ae0 0x5555a43b8f70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0ee040 tx=0x7f65cc0058b0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.083+0000 7f65ed849740 1 -- 172.21.15.18:0/1936860461 shutdown_connections 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.083+0000 7f65ed849740 1 --2- 172.21.15.18:0/1936860461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b6ae0 0x5555a43b8f70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.083+0000 7f65ed849740 1 --2- 172.21.15.18:0/1936860461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b1970 0x5555a43b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.083+0000 7f65ed849740 1 --2- 172.21.15.18:0/1936860461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b0f80 0x5555a43b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.083+0000 7f65ed849740 1 -- 172.21.15.18:0/1936860461 >> 172.21.15.18:0/1936860461 conn(0x5555a43ac4f0 msgr2=0x5555a43ac8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.083+0000 7f65ed849740 1 -- 172.21.15.18:0/1936860461 shutdown_connections 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.083+0000 7f65ed849740 1 -- 172.21.15.18:0/1936860461 wait complete. 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.083+0000 7f65ed849740 1 -- start start 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.083+0000 7f65ed849740 1 Processor -- start 2024-02-17T12:49:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65ed849740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 0x5555a43baf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65ed849740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b1970 0x5555a43bb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65ed849740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b6ae0 0x5555a43c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65ed849740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a4318c30 con 0x5555a43b0f80 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65ed849740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a42bdd30 con 0x5555a43b6ae0 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65ed849740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a3fec310 con 0x5555a43b1970 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65ed849740 10 client.0 _ll_register_callbacks cb 0x5555a42c5d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65e8a12640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 0x5555a43baf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65e8a12640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 0x5555a43baf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53668/0 (socket says 172.21.15.18:53668) 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65e8a12640 1 -- 172.21.15.18:0/2511514223 learned_addr learned my addr 172.21.15.18:0/2511514223 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65e8a12640 1 -- 172.21.15.18:0/2511514223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b1970 msgr2=0x5555a43bb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65e8a12640 1 --2- 172.21.15.18:0/2511514223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b1970 0x5555a43bb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65e8a12640 1 -- 172.21.15.18:0/2511514223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b6ae0 msgr2=0x5555a43c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65e8a12640 1 --2- 172.21.15.18:0/2511514223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b6ae0 0x5555a43c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65e8a12640 1 -- 172.21.15.18:0/2511514223 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e41a19c0 con 0x5555a43b0f80 2024-02-17T12:49:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65e8a12640 1 --2- 172.21.15.18:0/2511514223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 0x5555a43baf20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f65dc011a30 tx=0x7f65dc011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140f62322bb24eae server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.084+0000 7f65c3fff640 1 -- 172.21.15.18:0/2511514223 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc004520 con 0x5555a43b0f80 2024-02-17T12:49:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65c3fff640 1 -- 172.21.15.18:0/2511514223 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f65dc00f940 con 0x5555a43b0f80 2024-02-17T12:49:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65ed849740 1 -- 172.21.15.18:0/2511514223 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555a40186c0 con 0x5555a43b0f80 2024-02-17T12:49:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65c3fff640 1 -- 172.21.15.18:0/2511514223 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65dc00fb00 con 0x5555a43b0f80 2024-02-17T12:49:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65ed849740 10 client.15141 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65ed849740 1 -- 172.21.15.18:0/2511514223 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5555a43172b0 con 0x5555a43b0f80 2024-02-17T12:49:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65c3fff640 1 -- 172.21.15.18:0/2511514223 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5555a40186c0 con 0x5555a43b0f80 2024-02-17T12:49:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65c3fff640 1 -- 172.21.15.18:0/2511514223 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f65dc022050 con 0x5555a43b0f80 2024-02-17T12:49:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65ed849740 10 client.15141 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65ed849740 10 client.15141 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-17T12:49:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65ed849740 20 client.15141 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65ed849740 1 --2- 172.21.15.18:0/2511514223 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555a43c7a20 0x5555a43c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65ed849740 4 client.15141 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:49:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65ed849740 1 -- 172.21.15.18:0/2511514223 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555a4069270 con 0x5555a43c7a20 2024-02-17T12:49:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.085+0000 7f65e3fff640 1 --2- 172.21.15.18:0/2511514223 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555a43c7a20 0x5555a43c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.086+0000 7f65e3fff640 1 --2- 172.21.15.18:0/2511514223 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555a43c7a20 0x5555a43c6bd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ecb3abc65cdd13a server_cookie=289fc19a7e40b321 in_seq=0 out_seq=0 2024-02-17T12:49:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.086+0000 7f65c3fff640 10 client.15141 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.086+0000 7f65c3fff640 1 -- 172.21.15.18:0/2511514223 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5555a4069270 con 0x5555a43c7a20 2024-02-17T12:49:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.086+0000 7f65c3fff640 10 client.15141 handle_command_reply: tid=0 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.086+0000 7f65ed849740 1 client.15141 shutdown 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.086+0000 7f65ed849740 1 -- 172.21.15.18:0/2511514223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 msgr2=0x5555a43baf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.086+0000 7f65ed849740 1 --2- 172.21.15.18:0/2511514223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 0x5555a43baf20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f65dc011a30 tx=0x7f65dc011ef0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.087+0000 7f65ed849740 1 -- 172.21.15.18:0/2511514223 shutdown_connections 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.087+0000 7f65ed849740 1 --2- 172.21.15.18:0/2511514223 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555a43c7a20 0x5555a43c6bd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.087+0000 7f65ed849740 1 --2- 172.21.15.18:0/2511514223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b6ae0 0x5555a43c24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.087+0000 7f65ed849740 1 --2- 172.21.15.18:0/2511514223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b1970 0x5555a43bb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.087+0000 7f65ed849740 1 --2- 172.21.15.18:0/2511514223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 0x5555a43baf20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.087+0000 7f65c3fff640 0 client.15141 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.087+0000 7f65ed849740 1 -- 172.21.15.18:0/2511514223 >> 172.21.15.18:0/2511514223 conn(0x5555a43ac4f0 msgr2=0x5555a43ab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.087+0000 7f65ed849740 1 -- 172.21.15.18:0/2511514223 shutdown_connections 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.087+0000 7f65ed849740 1 -- 172.21.15.18:0/2511514223 wait complete. 2024-02-17T12:49:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.087+0000 7f65ed849740 20 client.15141 trim_cache size 0 max 16384 2024-02-17T12:49:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.102+0000 7f65ed849740 1 Processor -- start 2024-02-17T12:49:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65ed849740 1 -- start start 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65ed849740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b0f80 0x5555a43aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65ed849740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b1970 0x5555a43c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65ed849740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b6ae0 0x5555a43ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65ed849740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a429e490 con 0x5555a43b1970 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65ed849740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a42d1f40 con 0x5555a43b0f80 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65ed849740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a4311ad0 con 0x5555a43b6ae0 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b1970 0x5555a43c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b1970 0x5555a43c23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53674/0 (socket says 172.21.15.18:53674) 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e3fff640 1 -- 172.21.15.18:0/287100524 learned_addr learned my addr 172.21.15.18:0/287100524 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e9213640 1 --2- 172.21.15.18:0/287100524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b6ae0 0x5555a43ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e8a12640 1 --2- 172.21.15.18:0/287100524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b0f80 0x5555a43aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e3fff640 1 -- 172.21.15.18:0/287100524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b6ae0 msgr2=0x5555a43ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e3fff640 1 --2- 172.21.15.18:0/287100524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b6ae0 0x5555a43ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e3fff640 1 -- 172.21.15.18:0/287100524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b0f80 msgr2=0x5555a43aaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e3fff640 1 --2- 172.21.15.18:0/287100524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b0f80 0x5555a43aaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e3fff640 1 -- 172.21.15.18:0/287100524 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555a42b3a20 con 0x5555a43b1970 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e9213640 1 --2- 172.21.15.18:0/287100524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b6ae0 0x5555a43ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e8a12640 1 --2- 172.21.15.18:0/287100524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b0f80 0x5555a43aaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65e3fff640 1 --2- 172.21.15.18:0/287100524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b1970 0x5555a43c23c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f65d4005c40 tx=0x7f65d400a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4776140d1465c2a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65da7fc640 1 -- 172.21.15.18:0/287100524 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4026d00 con 0x5555a43b1970 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65da7fc640 1 -- 172.21.15.18:0/287100524 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65d4019070 con 0x5555a43b1970 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.103+0000 7f65da7fc640 1 -- 172.21.15.18:0/287100524 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4012a60 con 0x5555a43b1970 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.104+0000 7f65ed849740 1 -- 172.21.15.18:0/287100524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b1970 msgr2=0x5555a43c23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.104+0000 7f65ed849740 1 --2- 172.21.15.18:0/287100524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b1970 0x5555a43c23c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f65d4005c40 tx=0x7f65d400a840 comp rx=0 tx=0).stop 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.104+0000 7f65ed849740 1 -- 172.21.15.18:0/287100524 shutdown_connections 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.104+0000 7f65ed849740 1 --2- 172.21.15.18:0/287100524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b6ae0 0x5555a43ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.104+0000 7f65ed849740 1 --2- 172.21.15.18:0/287100524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b1970 0x5555a43c23c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.104+0000 7f65ed849740 1 --2- 172.21.15.18:0/287100524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b0f80 0x5555a43aaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.104+0000 7f65ed849740 1 -- 172.21.15.18:0/287100524 >> 172.21.15.18:0/287100524 conn(0x5555a43ac4f0 msgr2=0x5555a43b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.104+0000 7f65ed849740 1 -- 172.21.15.18:0/287100524 shutdown_connections 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.104+0000 7f65ed849740 1 -- 172.21.15.18:0/287100524 wait complete. 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.104+0000 7f65ed849740 1 -- start start 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.104+0000 7f65ed849740 1 Processor -- start 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65ed849740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 0x5555a43bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65ed849740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b1970 0x5555a43bfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65ed849740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b6ae0 0x5555a43c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65ed849740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a43c10a0 con 0x5555a43b0f80 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65ed849740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a43c1210 con 0x5555a43b6ae0 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65ed849740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a43c13d0 con 0x5555a43b1970 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65ed849740 10 client.0 _ll_register_callbacks cb 0x7f65cc003930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65e8a12640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 0x5555a43bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65e8a12640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 0x5555a43bf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53690/0 (socket says 172.21.15.18:53690) 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65e8a12640 1 -- 172.21.15.18:0/871343515 learned_addr learned my addr 172.21.15.18:0/871343515 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65e9213640 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b6ae0 0x5555a43c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65e3fff640 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b1970 0x5555a43bfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65e8a12640 1 -- 172.21.15.18:0/871343515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b1970 msgr2=0x5555a43bfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65e8a12640 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b1970 0x5555a43bfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65e8a12640 1 -- 172.21.15.18:0/871343515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b6ae0 msgr2=0x5555a43c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65e8a12640 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b6ae0 0x5555a43c0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65e8a12640 1 -- 172.21.15.18:0/871343515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555a42d9780 con 0x5555a43b0f80 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65e8a12640 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 0x5555a43bf610 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f65dc004c30 tx=0x7f65dc011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3728ea2be546b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65c3fff640 1 -- 172.21.15.18:0/871343515 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc005220 con 0x5555a43b0f80 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65c3fff640 1 -- 172.21.15.18:0/871343515 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f65dc000e40 con 0x5555a43b0f80 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65ed849740 1 -- 172.21.15.18:0/871343515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65cc0c28d0 con 0x5555a43b0f80 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65c3fff640 1 -- 172.21.15.18:0/871343515 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65dc005380 con 0x5555a43b0f80 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65ed849740 10 client.15147 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.105+0000 7f65ed849740 1 -- 172.21.15.18:0/871343515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5555a42bff10 con 0x5555a43b0f80 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.106+0000 7f65c3fff640 1 -- 172.21.15.18:0/871343515 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65cc0c28d0 con 0x5555a43b0f80 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.106+0000 7f65c3fff640 1 -- 172.21.15.18:0/871343515 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f65dc022070 con 0x5555a43b0f80 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.106+0000 7f65ed849740 10 client.15147 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.106+0000 7f65ed849740 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.106+0000 7f65ed849740 20 client.15147 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.106+0000 7f65ed849740 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555a44232d0 0x5555a441f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.106+0000 7f65ed849740 4 client.15147 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.106+0000 7f65ed849740 1 -- 172.21.15.18:0/871343515 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5555a4055610 con 0x5555a44232d0 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.106+0000 7f65e3fff640 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555a44232d0 0x5555a441f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65c3fff640 10 client.15147 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65e3fff640 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555a44232d0 0x5555a441f750 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15492f129a3e6a55 server_cookie=1a5a3345c65000eb in_seq=0 out_seq=0 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65c3fff640 1 -- 172.21.15.18:0/871343515 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5555a4055610 con 0x5555a44232d0 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65c3fff640 10 client.15147 handle_command_reply: tid=0 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65ed849740 1 client.15147 shutdown 2024-02-17T12:49:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65ed849740 1 -- 172.21.15.18:0/871343515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 msgr2=0x5555a43bf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:08.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65ed849740 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 0x5555a43bf610 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f65dc004c30 tx=0x7f65dc011650 comp rx=0 tx=0).stop 2024-02-17T12:49:08.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65ed849740 1 -- 172.21.15.18:0/871343515 shutdown_connections 2024-02-17T12:49:08.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65ed849740 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555a44232d0 0x5555a441f750 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65ed849740 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555a43b6ae0 0x5555a43c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65ed849740 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555a43b1970 0x5555a43bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65c3fff640 0 client.15147 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:08.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65ed849740 1 --2- 172.21.15.18:0/871343515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555a43b0f80 0x5555a43bf610 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65ed849740 1 -- 172.21.15.18:0/871343515 >> 172.21.15.18:0/871343515 conn(0x5555a43ac4f0 msgr2=0x5555a43af750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:08.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65ed849740 1 -- 172.21.15.18:0/871343515 shutdown_connections 2024-02-17T12:49:08.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65ed849740 1 -- 172.21.15.18:0/871343515 wait complete. 2024-02-17T12:49:08.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.107+0000 7f65ed849740 20 client.15147 trim_cache size 0 max 16384 2024-02-17T12:49:08.107 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:49:08.107 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:49:08.107 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 0, 2024-02-17T12:49:08.107 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24512, 2024-02-17T12:49:08.107 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-02-17T12:49:08.107 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-02-17T12:49:08.107 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-02-17T12:49:08.107 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 82.059980264000004, 2024-02-17T12:49:08.107 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 25, 2024-02-17T12:49:08.107 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 88, 2024-02-17T12:49:08.107 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 86, 2024-02-17T12:49:08.107 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 88.181778370999993 2024-02-17T12:49:08.108 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:49:08.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.108+0000 7f65eac76640 1 -- 172.21.15.18:0/1216682175 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65d00a2250 msgr2=0x7f65d00a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.108+0000 7f65eac76640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65d00a2250 0x7f65d00a4710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f65e4103d80 tx=0x7f65d4013a40 comp rx=0 tx=0).stop 2024-02-17T12:49:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.108+0000 7f65eac76640 1 -- 172.21.15.18:0/1216682175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e4101c30 msgr2=0x7f65e4102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.108+0000 7f65eac76640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e4101c30 0x7f65e4102b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009430 tx=0x7f65cc0042f0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.108+0000 7f65eac76640 1 -- 172.21.15.18:0/1216682175 shutdown_connections 2024-02-17T12:49:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.108+0000 7f65eac76640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65d00a2250 0x7f65d00a4710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.108+0000 7f65eac76640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e4101c30 0x7f65e4102b90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.108+0000 7f65eac76640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65e4101270 0x7f65e4102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.108+0000 7f65eac76640 1 --2- 172.21.15.18:0/1216682175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e4100080 0x7f65e4104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.108+0000 7f65eac76640 1 -- 172.21.15.18:0/1216682175 >> 172.21.15.18:0/1216682175 conn(0x7f65e40fb610 msgr2=0x7f65e40fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.108+0000 7f65eac76640 1 -- 172.21.15.18:0/1216682175 shutdown_connections 2024-02-17T12:49:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:08.108+0000 7f65eac76640 1 -- 172.21.15.18:0/1216682175 wait complete. 2024-02-17T12:49:08.111 INFO:tasks.cephadm:Setup complete, yielding 2024-02-17T12:49:08.111 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-17T12:49:08.120 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-02-17T12:49:08.121 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- bash -c 'ceph orch status' 2024-02-17T12:49:08.362 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:49:09.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:09 smithi146 ceph-mon[36454]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:49:09.722 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:09 smithi018 ceph-mon[32625]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:49:09.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:09 smithi119 ceph-mon[36188]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:49:09.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.993+0000 7f20daa90700 1 -- 172.21.15.18:0/1321361538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4107990 msgr2=0x7f20d4109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:09.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.993+0000 7f20daa90700 1 --2- 172.21.15.18:0/1321361538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4107990 0x7f20d4109e00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00b0a0 tx=0x7f20cc00b3b0 comp rx=0 tx=0).stop 2024-02-17T12:49:09.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.993+0000 7f20daa90700 1 -- 172.21.15.18:0/1321361538 shutdown_connections 2024-02-17T12:49:09.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.993+0000 7f20daa90700 1 --2- 172.21.15.18:0/1321361538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4107990 0x7f20d4109e00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:09.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.993+0000 7f20daa90700 1 --2- 172.21.15.18:0/1321361538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d4102730 0x7f20d4104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:09.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.993+0000 7f20daa90700 1 --2- 172.21.15.18:0/1321361538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d4105060 0x7f20d4107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:09.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.993+0000 7f20daa90700 1 -- 172.21.15.18:0/1321361538 >> 172.21.15.18:0/1321361538 conn(0x7f20d40fc140 msgr2=0x7f20d40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:09.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.993+0000 7f20daa90700 1 -- 172.21.15.18:0/1321361538 shutdown_connections 2024-02-17T12:49:09.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.993+0000 7f20daa90700 1 -- 172.21.15.18:0/1321361538 wait complete. 2024-02-17T12:49:09.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20daa90700 1 Processor -- start 2024-02-17T12:49:09.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20daa90700 1 -- start start 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20daa90700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d4102730 0x7f20d4199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20daa90700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d4105060 0x7f20d419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20daa90700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4107990 0x7f20d41a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20daa90700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d41a1b00 con 0x7f20d4107990 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20daa90700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d41a1c40 con 0x7f20d4105060 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20daa90700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d41a1dd0 con 0x7f20d4102730 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20d902d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4107990 0x7f20d41a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20d902d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4107990 0x7f20d41a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38526/0 (socket says 172.21.15.18:38526) 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20d3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d4105060 0x7f20d419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20d902d700 1 -- 172.21.15.18:0/215954004 learned_addr learned my addr 172.21.15.18:0/215954004 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.994+0000 7f20d882c700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d4102730 0x7f20d4199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.995+0000 7f20d902d700 1 -- 172.21.15.18:0/215954004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d4102730 msgr2=0x7f20d4199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.995+0000 7f20d902d700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d4102730 0x7f20d4199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:09.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.995+0000 7f20d902d700 1 -- 172.21.15.18:0/215954004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d4105060 msgr2=0x7f20d419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:09.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.995+0000 7f20d902d700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d4105060 0x7f20d419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:09.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.995+0000 7f20d902d700 1 -- 172.21.15.18:0/215954004 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc009d00 con 0x7f20d4107990 2024-02-17T12:49:09.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.995+0000 7f20d3fff700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d4105060 0x7f20d419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:49:09.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.995+0000 7f20d902d700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4107990 0x7f20d41a1530 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f20cc007920 tx=0x7f20cc007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:09.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.995+0000 7f20d1ffb700 1 -- 172.21.15.18:0/215954004 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00e040 con 0x7f20d4107990 2024-02-17T12:49:09.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.995+0000 7f20daa90700 1 -- 172.21.15.18:0/215954004 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d41a2050 con 0x7f20d4107990 2024-02-17T12:49:09.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.995+0000 7f20d1ffb700 1 -- 172.21.15.18:0/215954004 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20cc01d470 con 0x7f20d4107990 2024-02-17T12:49:09.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.995+0000 7f20d1ffb700 1 -- 172.21.15.18:0/215954004 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc005280 con 0x7f20d4107990 2024-02-17T12:49:09.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.995+0000 7f20daa90700 1 -- 172.21.15.18:0/215954004 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d41a24c0 con 0x7f20d4107990 2024-02-17T12:49:09.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.996+0000 7f20daa90700 1 -- 172.21.15.18:0/215954004 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d4109510 con 0x7f20d4107990 2024-02-17T12:49:09.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.998+0000 7f20d1ffb700 1 -- 172.21.15.18:0/215954004 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f20cc01d5e0 con 0x7f20d4107990 2024-02-17T12:49:09.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.999+0000 7f20d1ffb700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f20c00a21e0 0x7f20c00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:09.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.999+0000 7f20d882c700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f20c00a21e0 0x7f20c00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:09.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.999+0000 7f20d1ffb700 1 -- 172.21.15.18:0/215954004 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f20cc0cf690 con 0x7f20d4107990 2024-02-17T12:49:09.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:09.999+0000 7f20d882c700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f20c00a21e0 0x7f20c00a46a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009c60 tx=0x7f20c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:09.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.002+0000 7f20d1ffb700 1 -- 172.21.15.18:0/215954004 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20cc0985c0 con 0x7f20d4107990 2024-02-17T12:49:10.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.224+0000 7f20daa90700 1 -- 172.21.15.18:0/215954004 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f20d40620b0 con 0x7f20c00a21e0 2024-02-17T12:49:10.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.225+0000 7f20d1ffb700 1 -- 172.21.15.18:0/215954004 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f20d40620b0 con 0x7f20c00a21e0 2024-02-17T12:49:10.217 INFO:teuthology.orchestra.run.smithi018.stdout:Backend: cephadm 2024-02-17T12:49:10.217 INFO:teuthology.orchestra.run.smithi018.stdout:Available: Yes 2024-02-17T12:49:10.218 INFO:teuthology.orchestra.run.smithi018.stdout:Paused: No 2024-02-17T12:49:10.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.227+0000 7f20daa90700 1 -- 172.21.15.18:0/215954004 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f20c00a21e0 msgr2=0x7f20c00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:10.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.227+0000 7f20daa90700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f20c00a21e0 0x7f20c00a46a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009c60 tx=0x7f20c4009260 comp rx=0 tx=0).stop 2024-02-17T12:49:10.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.227+0000 7f20daa90700 1 -- 172.21.15.18:0/215954004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4107990 msgr2=0x7f20d41a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:10.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.227+0000 7f20daa90700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4107990 0x7f20d41a1530 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f20cc007920 tx=0x7f20cc007950 comp rx=0 tx=0).stop 2024-02-17T12:49:10.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.227+0000 7f20daa90700 1 -- 172.21.15.18:0/215954004 shutdown_connections 2024-02-17T12:49:10.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.227+0000 7f20daa90700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4107990 0x7f20d41a1530 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:10.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.227+0000 7f20daa90700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f20c00a21e0 0x7f20c00a46a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:10.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.227+0000 7f20daa90700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20d4102730 0x7f20d4199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:10.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.227+0000 7f20daa90700 1 --2- 172.21.15.18:0/215954004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20d4105060 0x7f20d419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:10.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.227+0000 7f20daa90700 1 -- 172.21.15.18:0/215954004 >> 172.21.15.18:0/215954004 conn(0x7f20d40fc140 msgr2=0x7f20d4101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:10.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.227+0000 7f20daa90700 1 -- 172.21.15.18:0/215954004 shutdown_connections 2024-02-17T12:49:10.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:10.227+0000 7f20daa90700 1 -- 172.21.15.18:0/215954004 wait complete. 2024-02-17T12:49:10.853 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-17T12:49:11.107 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:49:11.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:11 smithi018 ceph-mon[32625]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-02-17T12:49:11.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:11 smithi018 ceph-mon[32625]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:11.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:11 smithi146 ceph-mon[36454]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-02-17T12:49:11.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:11 smithi146 ceph-mon[36454]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:11.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:11 smithi119 ceph-mon[36188]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-02-17T12:49:11.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:11 smithi119 ceph-mon[36188]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:12.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.648+0000 7f2b7bae8700 1 -- 172.21.15.18:0/1029478318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b74102e90 msgr2=0x7f2b74105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:12.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.648+0000 7f2b7bae8700 1 --2- 172.21.15.18:0/1029478318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b74102e90 0x7f2b74105320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009990 tx=0x7f2b68009ca0 comp rx=0 tx=0).stop 2024-02-17T12:49:12.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.648+0000 7f2b7bae8700 1 -- 172.21.15.18:0/1029478318 shutdown_connections 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.648+0000 7f2b7bae8700 1 --2- 172.21.15.18:0/1029478318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b74105860 0x7f2b74105ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.648+0000 7f2b7bae8700 1 --2- 172.21.15.18:0/1029478318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b74100520 0x7f2b74102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.648+0000 7f2b7bae8700 1 --2- 172.21.15.18:0/1029478318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b74102e90 0x7f2b74105320 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.648+0000 7f2b7bae8700 1 -- 172.21.15.18:0/1029478318 >> 172.21.15.18:0/1029478318 conn(0x7f2b740fc140 msgr2=0x7f2b740fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.648+0000 7f2b7bae8700 1 -- 172.21.15.18:0/1029478318 shutdown_connections 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.648+0000 7f2b7bae8700 1 -- 172.21.15.18:0/1029478318 wait complete. 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b7bae8700 1 Processor -- start 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b7bae8700 1 -- start start 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b7bae8700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b74100520 0x7f2b7419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b7bae8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b74102e90 0x7f2b7419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b7bae8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b74105860 0x7f2b741a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b7bae8700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b741a5df0 con 0x7f2b74100520 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b7bae8700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b741a5f30 con 0x7f2b74102e90 2024-02-17T12:49:12.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b7bae8700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b741a60c0 con 0x7f2b74105860 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b79884700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b74100520 0x7f2b7419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b79884700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b74100520 0x7f2b7419e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38542/0 (socket says 172.21.15.18:38542) 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b79884700 1 -- 172.21.15.18:0/3503746154 learned_addr learned my addr 172.21.15.18:0/3503746154 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b7a085700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b74105860 0x7f2b741a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b79083700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b74102e90 0x7f2b7419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b79884700 1 -- 172.21.15.18:0/3503746154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b74105860 msgr2=0x7f2b741a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b79884700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b74105860 0x7f2b741a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b79884700 1 -- 172.21.15.18:0/3503746154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b74102e90 msgr2=0x7f2b7419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b79884700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b74102e90 0x7f2b7419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.649+0000 7f2b79884700 1 -- 172.21.15.18:0/3503746154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b680095f0 con 0x7f2b74100520 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.650+0000 7f2b79884700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b74100520 0x7f2b7419e2b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2b64007ee0 tx=0x7f2b6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.650+0000 7f2b72ffd700 1 -- 172.21.15.18:0/3503746154 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400dd20 con 0x7f2b74100520 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.650+0000 7f2b7bae8700 1 -- 172.21.15.18:0/3503746154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b741a63a0 con 0x7f2b74100520 2024-02-17T12:49:12.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.650+0000 7f2b72ffd700 1 -- 172.21.15.18:0/3503746154 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b6400f040 con 0x7f2b74100520 2024-02-17T12:49:12.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.650+0000 7f2b72ffd700 1 -- 172.21.15.18:0/3503746154 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64014600 con 0x7f2b74100520 2024-02-17T12:49:12.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.650+0000 7f2b7bae8700 1 -- 172.21.15.18:0/3503746154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b741a68f0 con 0x7f2b74100520 2024-02-17T12:49:12.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.651+0000 7f2b7bae8700 1 -- 172.21.15.18:0/3503746154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b740fdcc0 con 0x7f2b74100520 2024-02-17T12:49:12.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.652+0000 7f2b72ffd700 1 -- 172.21.15.18:0/3503746154 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f2b64015400 con 0x7f2b74100520 2024-02-17T12:49:12.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.653+0000 7f2b72ffd700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2b5c0a2260 0x7f2b5c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:12.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.653+0000 7f2b79083700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2b5c0a2260 0x7f2b5c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:12.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.653+0000 7f2b72ffd700 1 -- 172.21.15.18:0/3503746154 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2b640cd700 con 0x7f2b74100520 2024-02-17T12:49:12.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.653+0000 7f2b79083700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2b5c0a2260 0x7f2b5c0a4720 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800b5c0 tx=0x7f2b68003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:12.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.657+0000 7f2b72ffd700 1 -- 172.21.15.18:0/3503746154 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b64096540 con 0x7f2b74100520 2024-02-17T12:49:12.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.868+0000 7f2b7bae8700 1 -- 172.21.15.18:0/3503746154 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b7402d8a0 con 0x7f2b5c0a2260 2024-02-17T12:49:12.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.882+0000 7f2b72ffd700 1 -- 172.21.15.18:0/3503746154 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f2b7402d8a0 con 0x7f2b5c0a2260 2024-02-17T12:49:12.874 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mds.a smithi018 running (95s) 15s ago 95s 16.7M - 19.0.0-1515-g6822c7b5 32fa8af9a902 49bad5597965 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mds.b smithi119 running (93s) 26s ago 92s 28.4M - 19.0.0-1515-g6822c7b5 32fa8af9a902 5365242334d8 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mds.c smithi146 running (107s) 26s ago 106s 19.1M - 19.0.0-1515-g6822c7b5 32fa8af9a902 bcf9fbbd17ff 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mds.d smithi018 running (114s) 15s ago 114s 16.2M - 19.0.0-1515-g6822c7b5 32fa8af9a902 f1a3c63002a3 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mds.e smithi119 running (112s) 26s ago 112s 17.7M - 19.0.0-1515-g6822c7b5 32fa8af9a902 2e488f45841e 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mds.f smithi146 running (109s) 26s ago 109s 18.7M - 19.0.0-1515-g6822c7b5 32fa8af9a902 da48079880bd 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mds.g smithi018 running (98s) 15s ago 98s 17.1M - 19.0.0-1515-g6822c7b5 32fa8af9a902 b924061d027f 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mds.h smithi119 running (104s) 26s ago 103s 17.4M - 19.0.0-1515-g6822c7b5 32fa8af9a902 a6a555ffabe7 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mds.i smithi146 running (101s) 26s ago 101s 18.3M - 19.0.0-1515-g6822c7b5 32fa8af9a902 db4ba8766bbe 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mds.j smithi018 running (117s) 15s ago 117s 13.8M - 19.0.0-1515-g6822c7b5 32fa8af9a902 0514dd8cfd4a 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mds.k smithi119 running (2m) 26s ago 119s 19.6M - 19.0.0-1515-g6822c7b5 32fa8af9a902 16c69fa7b6ed 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mds.l smithi146 running (2m) 26s ago 2m 16.9M - 19.0.0-1515-g6822c7b5 32fa8af9a902 79ede5d44042 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.x smithi018 *:9283,8765 running (9m) 15s ago 9m 512M - 19.0.0-1515-g6822c7b5 32fa8af9a902 186b5dfab411 2024-02-17T12:49:12.875 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.y smithi119 *:8443,8765 running (7m) 26s ago 7m 450M - 19.0.0-1515-g6822c7b5 32fa8af9a902 68953ec3dd37 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.z smithi146 *:8443,8765 running (7m) 26s ago 7m 453M - 19.0.0-1515-g6822c7b5 32fa8af9a902 f16d38e89334 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:mon.a smithi018 running (9m) 15s ago 9m 54.1M 2048M 19.0.0-1515-g6822c7b5 32fa8af9a902 0786d0b495a8 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:mon.b smithi119 running (7m) 26s ago 7m 38.5M 2048M 19.0.0-1515-g6822c7b5 32fa8af9a902 42ee1f6ae412 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:mon.c smithi146 running (7m) 26s ago 7m 41.3M 2048M 19.0.0-1515-g6822c7b5 32fa8af9a902 a924da2e36e4 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (6m) 15s ago 6m 48.6M 4273M 19.0.0-1515-g6822c7b5 32fa8af9a902 45f66f351be2 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi119 running (6m) 26s ago 6m 57.5M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 870e46b5497d 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi146 running (6m) 26s ago 6m 62.3M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 d8efbf996202 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi018 running (5m) 15s ago 5m 57.1M 4273M 19.0.0-1515-g6822c7b5 32fa8af9a902 ab91728db216 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi119 running (5m) 26s ago 5m 52.4M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 670d395af678 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi146 running (4m) 26s ago 4m 50.2M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 24e0914f9ffd 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6 smithi018 running (4m) 15s ago 4m 47.5M 4273M 19.0.0-1515-g6822c7b5 32fa8af9a902 07846cd7f3a5 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7 smithi119 running (4m) 26s ago 4m 53.9M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 dde9188ecbfb 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:osd.8 smithi146 running (3m) 26s ago 3m 49.0M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 628e781e5fa1 2024-02-17T12:49:12.876 INFO:teuthology.orchestra.run.smithi018.stdout:osd.9 smithi018 running (3m) 15s ago 3m 50.0M 4273M 19.0.0-1515-g6822c7b5 32fa8af9a902 dad64f14ca7d 2024-02-17T12:49:12.877 INFO:teuthology.orchestra.run.smithi018.stdout:osd.10 smithi119 running (2m) 26s ago 2m 48.8M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 1a624659c666 2024-02-17T12:49:12.877 INFO:teuthology.orchestra.run.smithi018.stdout:osd.11 smithi146 running (2m) 26s ago 2m 43.0M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 ad3f44772022 2024-02-17T12:49:12.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.884+0000 7f2b7bae8700 1 -- 172.21.15.18:0/3503746154 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2b5c0a2260 msgr2=0x7f2b5c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:12.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.884+0000 7f2b7bae8700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2b5c0a2260 0x7f2b5c0a4720 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800b5c0 tx=0x7f2b68003860 comp rx=0 tx=0).stop 2024-02-17T12:49:12.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.884+0000 7f2b7bae8700 1 -- 172.21.15.18:0/3503746154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b74100520 msgr2=0x7f2b7419e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:12.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.884+0000 7f2b7bae8700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b74100520 0x7f2b7419e2b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2b64007ee0 tx=0x7f2b6400d3b0 comp rx=0 tx=0).stop 2024-02-17T12:49:12.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.884+0000 7f2b7bae8700 1 -- 172.21.15.18:0/3503746154 shutdown_connections 2024-02-17T12:49:12.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.884+0000 7f2b7bae8700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2b5c0a2260 0x7f2b5c0a4720 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:12.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.884+0000 7f2b7bae8700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b74105860 0x7f2b741a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:12.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.884+0000 7f2b7bae8700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b74100520 0x7f2b7419e2b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:12.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.884+0000 7f2b7bae8700 1 --2- 172.21.15.18:0/3503746154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b74102e90 0x7f2b7419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:12.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.884+0000 7f2b7bae8700 1 -- 172.21.15.18:0/3503746154 >> 172.21.15.18:0/3503746154 conn(0x7f2b740fc140 msgr2=0x7f2b74101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:12.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.884+0000 7f2b7bae8700 1 -- 172.21.15.18:0/3503746154 shutdown_connections 2024-02-17T12:49:12.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:12.884+0000 7f2b7bae8700 1 -- 172.21.15.18:0/3503746154 wait complete. 2024-02-17T12:49:13.485 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-17T12:49:13.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:13 smithi146 ceph-mon[36454]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-02-17T12:49:13.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:13 smithi146 ceph-mon[36454]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:13.730 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:49:13.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:13 smithi018 ceph-mon[32625]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-02-17T12:49:13.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:13 smithi018 ceph-mon[32625]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:13.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:13 smithi119 ceph-mon[36188]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-02-17T12:49:13.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:13 smithi119 ceph-mon[36188]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:15.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.315+0000 7f7d85214700 1 -- 172.21.15.18:0/2733649176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d80100ba0 msgr2=0x7f7d80100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:15.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.315+0000 7f7d85214700 1 --2- 172.21.15.18:0/2733649176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d80100ba0 0x7f7d80100fc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000b0d0 tx=0x7f7d7000b3e0 comp rx=0 tx=0).stop 2024-02-17T12:49:15.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.315+0000 7f7d85214700 1 -- 172.21.15.18:0/2733649176 shutdown_connections 2024-02-17T12:49:15.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.315+0000 7f7d85214700 1 --2- 172.21.15.18:0/2733649176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d80102740 0x7f7d80108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:15.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.315+0000 7f7d85214700 1 --2- 172.21.15.18:0/2733649176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d80100ba0 0x7f7d80100fc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:15.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.315+0000 7f7d85214700 1 --2- 172.21.15.18:0/2733649176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d80101da0 0x7f7d80102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:15.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.315+0000 7f7d85214700 1 -- 172.21.15.18:0/2733649176 >> 172.21.15.18:0/2733649176 conn(0x7f7d800fc160 msgr2=0x7f7d800fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:15.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.315+0000 7f7d85214700 1 -- 172.21.15.18:0/2733649176 shutdown_connections 2024-02-17T12:49:15.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.315+0000 7f7d85214700 1 -- 172.21.15.18:0/2733649176 wait complete. 2024-02-17T12:49:15.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d85214700 1 Processor -- start 2024-02-17T12:49:15.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d85214700 1 -- start start 2024-02-17T12:49:15.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d85214700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d80100ba0 0x7f7d8010eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:15.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d85214700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d80101da0 0x7f7d8010d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:15.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d85214700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d80102740 0x7f7d8010d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:15.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d85214700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8010deb0 con 0x7f7d80100ba0 2024-02-17T12:49:15.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d85214700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8010e020 con 0x7f7d80101da0 2024-02-17T12:49:15.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d85214700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d801a1e10 con 0x7f7d80102740 2024-02-17T12:49:15.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d7ed9d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d80100ba0 0x7f7d8010eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:15.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d7ed9d700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d80100ba0 0x7f7d8010eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38556/0 (socket says 172.21.15.18:38556) 2024-02-17T12:49:15.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d7ed9d700 1 -- 172.21.15.18:0/810129205 learned_addr learned my addr 172.21.15.18:0/810129205 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:15.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d7f59e700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d80102740 0x7f7d8010d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:15.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d7e59c700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d80101da0 0x7f7d8010d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:15.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d7ed9d700 1 -- 172.21.15.18:0/810129205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d80102740 msgr2=0x7f7d8010d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:15.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d7ed9d700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d80102740 0x7f7d8010d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:15.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d7ed9d700 1 -- 172.21.15.18:0/810129205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d80101da0 msgr2=0x7f7d8010d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:15.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d7ed9d700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d80101da0 0x7f7d8010d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:15.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.316+0000 7f7d7ed9d700 1 -- 172.21.15.18:0/810129205 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d70009d00 con 0x7f7d80100ba0 2024-02-17T12:49:15.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.317+0000 7f7d7ed9d700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d80100ba0 0x7f7d8010eba0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7d70000f80 tx=0x7f7d70009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:15.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.317+0000 7f7d77fff700 1 -- 172.21.15.18:0/810129205 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7000e040 con 0x7f7d80100ba0 2024-02-17T12:49:15.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.317+0000 7f7d77fff700 1 -- 172.21.15.18:0/810129205 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d7001dd70 con 0x7f7d80100ba0 2024-02-17T12:49:15.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.317+0000 7f7d85214700 1 -- 172.21.15.18:0/810129205 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d801a2090 con 0x7f7d80100ba0 2024-02-17T12:49:15.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.317+0000 7f7d77fff700 1 -- 172.21.15.18:0/810129205 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7001be80 con 0x7f7d80100ba0 2024-02-17T12:49:15.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.317+0000 7f7d85214700 1 -- 172.21.15.18:0/810129205 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d801a2500 con 0x7f7d80100ba0 2024-02-17T12:49:15.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.318+0000 7f7d85214700 1 -- 172.21.15.18:0/810129205 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d80068930 con 0x7f7d80100ba0 2024-02-17T12:49:15.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.320+0000 7f7d77fff700 1 -- 172.21.15.18:0/810129205 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f7d7001b5e0 con 0x7f7d80100ba0 2024-02-17T12:49:15.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.320+0000 7f7d77fff700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f7d6c0a21e0 0x7f7d6c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:15.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.320+0000 7f7d77fff700 1 -- 172.21.15.18:0/810129205 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7d700cfe10 con 0x7f7d80100ba0 2024-02-17T12:49:15.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.320+0000 7f7d7e59c700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f7d6c0a21e0 0x7f7d6c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:15.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.321+0000 7f7d7e59c700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f7d6c0a21e0 0x7f7d6c0a46a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7d68005950 tx=0x7f7d680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:15.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.324+0000 7f7d77fff700 1 -- 172.21.15.18:0/810129205 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7d70098d40 con 0x7f7d80100ba0 2024-02-17T12:49:15.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.536+0000 7f7d85214700 1 -- 172.21.15.18:0/810129205 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d800fe350 con 0x7f7d6c0a21e0 2024-02-17T12:49:15.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.542+0000 7f7d77fff700 1 -- 172.21.15.18:0/810129205 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f7d800fe350 con 0x7f7d6c0a21e0 2024-02-17T12:49:15.535 INFO:teuthology.orchestra.run.smithi018.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-17T12:49:15.535 INFO:teuthology.orchestra.run.smithi018.stdout:mds.all 12/12 29s ago 2m smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi146=c;smithi146=f;smithi146=i;smithi146=l;count:12 2024-02-17T12:49:15.535 INFO:teuthology.orchestra.run.smithi018.stdout:mgr 3/3 29s ago 7m smithi018=x;smithi119=y;smithi146=z;count:3 2024-02-17T12:49:15.535 INFO:teuthology.orchestra.run.smithi018.stdout:mon 3/3 29s ago 7m smithi018:172.21.15.18=a;smithi119:172.21.15.119=b;smithi146:172.21.15.146=c;count:3 2024-02-17T12:49:15.536 INFO:teuthology.orchestra.run.smithi018.stdout:osd 12 29s ago - 2024-02-17T12:49:15.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.544+0000 7f7d85214700 1 -- 172.21.15.18:0/810129205 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f7d6c0a21e0 msgr2=0x7f7d6c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:15.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.544+0000 7f7d85214700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f7d6c0a21e0 0x7f7d6c0a46a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7d68005950 tx=0x7f7d680058e0 comp rx=0 tx=0).stop 2024-02-17T12:49:15.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.544+0000 7f7d85214700 1 -- 172.21.15.18:0/810129205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d80100ba0 msgr2=0x7f7d8010eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:15.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.544+0000 7f7d85214700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d80100ba0 0x7f7d8010eba0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7d70000f80 tx=0x7f7d70009380 comp rx=0 tx=0).stop 2024-02-17T12:49:15.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.544+0000 7f7d85214700 1 -- 172.21.15.18:0/810129205 shutdown_connections 2024-02-17T12:49:15.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.544+0000 7f7d85214700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f7d6c0a21e0 0x7f7d6c0a46a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:15.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.544+0000 7f7d85214700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d80102740 0x7f7d8010d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:15.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.544+0000 7f7d85214700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d80100ba0 0x7f7d8010eba0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:15.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.544+0000 7f7d85214700 1 --2- 172.21.15.18:0/810129205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d80101da0 0x7f7d8010d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:15.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.544+0000 7f7d85214700 1 -- 172.21.15.18:0/810129205 >> 172.21.15.18:0/810129205 conn(0x7f7d800fc160 msgr2=0x7f7d800fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:15.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.544+0000 7f7d85214700 1 -- 172.21.15.18:0/810129205 shutdown_connections 2024-02-17T12:49:15.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:15.544+0000 7f7d85214700 1 -- 172.21.15.18:0/810129205 wait complete. 2024-02-17T12:49:15.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:15 smithi146 ceph-mon[36454]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-02-17T12:49:15.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:15 smithi018 ceph-mon[32625]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-02-17T12:49:15.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:15 smithi119 ceph-mon[36188]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-02-17T12:49:16.183 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-17T12:49:16.428 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:49:16.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:16 smithi146 ceph-mon[36454]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:16.748 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:16 smithi018 ceph-mon[32625]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:16.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:16 smithi119 ceph-mon[36188]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:17.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:17 smithi018 ceph-mon[32625]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-02-17T12:49:17.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:17 smithi146 ceph-mon[36454]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-02-17T12:49:17.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:17 smithi119 ceph-mon[36188]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-02-17T12:49:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.915+0000 7fae03799700 1 -- 172.21.15.18:0/2313900144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadfc101da0 msgr2=0x7fadfc102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.915+0000 7fae03799700 1 --2- 172.21.15.18:0/2313900144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadfc101da0 0x7fadfc102200 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fadf80099c0 tx=0x7fadf8009cd0 comp rx=0 tx=0).stop 2024-02-17T12:49:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.915+0000 7fae03799700 1 -- 172.21.15.18:0/2313900144 shutdown_connections 2024-02-17T12:49:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.915+0000 7fae03799700 1 --2- 172.21.15.18:0/2313900144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadfc102740 0x7fadfc108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.915+0000 7fae03799700 1 --2- 172.21.15.18:0/2313900144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadfc100ba0 0x7fadfc100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.915+0000 7fae03799700 1 --2- 172.21.15.18:0/2313900144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadfc101da0 0x7fadfc102200 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.915+0000 7fae03799700 1 -- 172.21.15.18:0/2313900144 >> 172.21.15.18:0/2313900144 conn(0x7fadfc0fc140 msgr2=0x7fadfc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.915+0000 7fae03799700 1 -- 172.21.15.18:0/2313900144 shutdown_connections 2024-02-17T12:49:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.915+0000 7fae03799700 1 -- 172.21.15.18:0/2313900144 wait complete. 2024-02-17T12:49:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae03799700 1 Processor -- start 2024-02-17T12:49:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae03799700 1 -- start start 2024-02-17T12:49:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae03799700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadfc100ba0 0x7fadfc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae03799700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadfc101da0 0x7fadfc19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae03799700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadfc102740 0x7fadfc19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae03799700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc19f810 con 0x7fadfc101da0 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae03799700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc19f980 con 0x7fadfc100ba0 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae03799700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc19fb40 con 0x7fadfc102740 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae00d34700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadfc101da0 0x7fadfc19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae00d34700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadfc101da0 0x7fadfc19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38586/0 (socket says 172.21.15.18:38586) 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae00d34700 1 -- 172.21.15.18:0/2744914813 learned_addr learned my addr 172.21.15.18:0/2744914813 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae01d36700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadfc102740 0x7fadfc19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae01535700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadfc100ba0 0x7fadfc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae00d34700 1 -- 172.21.15.18:0/2744914813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadfc102740 msgr2=0x7fadfc19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.916+0000 7fae00d34700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadfc102740 0x7fadfc19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.917+0000 7fae00d34700 1 -- 172.21.15.18:0/2744914813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadfc100ba0 msgr2=0x7fadfc06a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:17.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.917+0000 7fae00d34700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadfc100ba0 0x7fadfc06a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:17.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.917+0000 7fae00d34700 1 -- 172.21.15.18:0/2744914813 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80095f0 con 0x7fadfc101da0 2024-02-17T12:49:17.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.917+0000 7fae01535700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadfc100ba0 0x7fadfc06a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:17.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.917+0000 7fae00d34700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadfc101da0 0x7fadfc19ca30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fadf800b580 tx=0x7fadf80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:17.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.917+0000 7fadee7fc700 1 -- 172.21.15.18:0/2744914813 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf801d070 con 0x7fadfc101da0 2024-02-17T12:49:17.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.917+0000 7fadee7fc700 1 -- 172.21.15.18:0/2744914813 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadf80041d0 con 0x7fadfc101da0 2024-02-17T12:49:17.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.917+0000 7fae03799700 1 -- 172.21.15.18:0/2744914813 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc19fdc0 con 0x7fadfc101da0 2024-02-17T12:49:17.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.917+0000 7fadee7fc700 1 -- 172.21.15.18:0/2744914813 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf800f460 con 0x7fadfc101da0 2024-02-17T12:49:17.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.917+0000 7fae03799700 1 -- 172.21.15.18:0/2744914813 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc1a0200 con 0x7fadfc101da0 2024-02-17T12:49:17.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.918+0000 7fae03799700 1 -- 172.21.15.18:0/2744914813 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc068930 con 0x7fadfc101da0 2024-02-17T12:49:17.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.919+0000 7fadee7fc700 1 -- 172.21.15.18:0/2744914813 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fadf8022690 con 0x7fadfc101da0 2024-02-17T12:49:17.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.920+0000 7fadee7fc700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fade80a21e0 0x7fade80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:17.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.920+0000 7fae01535700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fade80a21e0 0x7fade80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:17.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.920+0000 7fadee7fc700 1 -- 172.21.15.18:0/2744914813 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fadf80cf280 con 0x7fadfc101da0 2024-02-17T12:49:17.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.920+0000 7fae01535700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fade80a21e0 0x7fade80a46a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fadfc101c00 tx=0x7fadf0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:17.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:17.924+0000 7fadee7fc700 1 -- 172.21.15.18:0/2744914813 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadf80981b0 con 0x7fadfc101da0 2024-02-17T12:49:18.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.134+0000 7fae03799700 1 -- 172.21.15.18:0/2744914813 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fadfc02d910 con 0x7fade80a21e0 2024-02-17T12:49:18.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.136+0000 7fadee7fc700 1 -- 172.21.15.18:0/2744914813 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fadfc02d910 con 0x7fade80a21e0 2024-02-17T12:49:18.128 INFO:teuthology.orchestra.run.smithi018.stdout:HOST ADDR LABELS STATUS 2024-02-17T12:49:18.128 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018 172.21.15.18 2024-02-17T12:49:18.128 INFO:teuthology.orchestra.run.smithi018.stdout:smithi119 172.21.15.119 2024-02-17T12:49:18.128 INFO:teuthology.orchestra.run.smithi018.stdout:smithi146 172.21.15.146 2024-02-17T12:49:18.128 INFO:teuthology.orchestra.run.smithi018.stdout:3 hosts in cluster 2024-02-17T12:49:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.138+0000 7fae03799700 1 -- 172.21.15.18:0/2744914813 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fade80a21e0 msgr2=0x7fade80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.138+0000 7fae03799700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fade80a21e0 0x7fade80a46a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fadfc101c00 tx=0x7fadf0008040 comp rx=0 tx=0).stop 2024-02-17T12:49:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.138+0000 7fae03799700 1 -- 172.21.15.18:0/2744914813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadfc101da0 msgr2=0x7fadfc19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.138+0000 7fae03799700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadfc101da0 0x7fadfc19ca30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fadf800b580 tx=0x7fadf80047b0 comp rx=0 tx=0).stop 2024-02-17T12:49:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.138+0000 7fae03799700 1 -- 172.21.15.18:0/2744914813 shutdown_connections 2024-02-17T12:49:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.138+0000 7fae03799700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fade80a21e0 0x7fade80a46a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:18.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.138+0000 7fae03799700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadfc102740 0x7fadfc19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:18.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.138+0000 7fae03799700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadfc100ba0 0x7fadfc06a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:18.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.138+0000 7fae03799700 1 --2- 172.21.15.18:0/2744914813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadfc101da0 0x7fadfc19ca30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:18.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.138+0000 7fae03799700 1 -- 172.21.15.18:0/2744914813 >> 172.21.15.18:0/2744914813 conn(0x7fadfc0fc140 msgr2=0x7fadfc1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:18.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.138+0000 7fae03799700 1 -- 172.21.15.18:0/2744914813 shutdown_connections 2024-02-17T12:49:18.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:18.138+0000 7fae03799700 1 -- 172.21.15.18:0/2744914813 wait complete. 2024-02-17T12:49:18.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:18 smithi146 ceph-mon[36454]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:18.717 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:18 smithi018 ceph-mon[32625]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:18.746 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-17T12:49:18.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:18 smithi119 ceph-mon[36188]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:18.990 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:49:19.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:19 smithi146 ceph-mon[36454]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-02-17T12:49:19.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:19 smithi018 ceph-mon[32625]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-02-17T12:49:19.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:19 smithi119 ceph-mon[36188]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-02-17T12:49:20.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.629+0000 7f658de50700 1 -- 172.21.15.18:0/2613423465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6588101dd0 msgr2=0x7f6588102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:20.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.629+0000 7f658de50700 1 --2- 172.21.15.18:0/2613423465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6588101dd0 0x7f6588102230 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f65700099c0 tx=0x7f6570009cd0 comp rx=0 tx=0).stop 2024-02-17T12:49:20.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.629+0000 7f658de50700 1 -- 172.21.15.18:0/2613423465 shutdown_connections 2024-02-17T12:49:20.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.629+0000 7f658de50700 1 --2- 172.21.15.18:0/2613423465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6588102770 0x7f6588109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:20.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.629+0000 7f658de50700 1 --2- 172.21.15.18:0/2613423465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6588100bd0 0x7f6588100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:20.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.629+0000 7f658de50700 1 --2- 172.21.15.18:0/2613423465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6588101dd0 0x7f6588102230 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:20.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.629+0000 7f658de50700 1 -- 172.21.15.18:0/2613423465 >> 172.21.15.18:0/2613423465 conn(0x7f65880fc190 msgr2=0x7f65880fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:20.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.629+0000 7f658de50700 1 -- 172.21.15.18:0/2613423465 shutdown_connections 2024-02-17T12:49:20.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.629+0000 7f658de50700 1 -- 172.21.15.18:0/2613423465 wait complete. 2024-02-17T12:49:20.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.629+0000 7f658de50700 1 Processor -- start 2024-02-17T12:49:20.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.629+0000 7f658de50700 1 -- start start 2024-02-17T12:49:20.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f658de50700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6588100bd0 0x7f658806af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:20.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f658de50700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6588101dd0 0x7f658819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:20.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f658de50700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6588102770 0x7f658819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:20.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f658de50700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658819f820 con 0x7f6588101dd0 2024-02-17T12:49:20.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f658de50700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658819f990 con 0x7f6588100bd0 2024-02-17T12:49:20.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f658de50700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658819fb50 con 0x7f6588102770 2024-02-17T12:49:20.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f6586ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6588101dd0 0x7f658819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:20.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f6586ffd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6588101dd0 0x7f658819c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40248/0 (socket says 172.21.15.18:40248) 2024-02-17T12:49:20.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f6586ffd700 1 -- 172.21.15.18:0/2226015360 learned_addr learned my addr 172.21.15.18:0/2226015360 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:20.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f65877fe700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6588100bd0 0x7f658806af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:20.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f6587fff700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6588102770 0x7f658819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:20.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f6586ffd700 1 -- 172.21.15.18:0/2226015360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6588102770 msgr2=0x7f658819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:20.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f6586ffd700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6588102770 0x7f658819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:20.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f6586ffd700 1 -- 172.21.15.18:0/2226015360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6588100bd0 msgr2=0x7f658806af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:20.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f6586ffd700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6588100bd0 0x7f658806af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f6586ffd700 1 -- 172.21.15.18:0/2226015360 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65700095f0 con 0x7f6588101dd0 2024-02-17T12:49:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f65877fe700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6588100bd0 0x7f658806af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f6587fff700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6588102770 0x7f658819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f6586ffd700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6588101dd0 0x7f658819c9f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6570003790 tx=0x7f6570003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.630+0000 7f6584ff9700 1 -- 172.21.15.18:0/2226015360 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657001d070 con 0x7f6588101dd0 2024-02-17T12:49:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.631+0000 7f658de50700 1 -- 172.21.15.18:0/2226015360 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658819fdd0 con 0x7f6588101dd0 2024-02-17T12:49:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.631+0000 7f6584ff9700 1 -- 172.21.15.18:0/2226015360 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6570003f00 con 0x7f6588101dd0 2024-02-17T12:49:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.631+0000 7f6584ff9700 1 -- 172.21.15.18:0/2226015360 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657000f460 con 0x7f6588101dd0 2024-02-17T12:49:20.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.631+0000 7f658de50700 1 -- 172.21.15.18:0/2226015360 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65881a0260 con 0x7f6588101dd0 2024-02-17T12:49:20.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.632+0000 7f658de50700 1 -- 172.21.15.18:0/2226015360 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6588068930 con 0x7f6588101dd0 2024-02-17T12:49:20.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.633+0000 7f6584ff9700 1 -- 172.21.15.18:0/2226015360 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f6570004070 con 0x7f6588101dd0 2024-02-17T12:49:20.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.634+0000 7f6584ff9700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65740a2260 0x7f65740a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:20.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.634+0000 7f65877fe700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65740a2260 0x7f65740a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:20.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.634+0000 7f6584ff9700 1 -- 172.21.15.18:0/2226015360 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f65700cf650 con 0x7f6588101dd0 2024-02-17T12:49:20.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.634+0000 7f65877fe700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65740a2260 0x7f65740a4720 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6588101c30 tx=0x7f6578009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:20.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.638+0000 7f6584ff9700 1 -- 172.21.15.18:0/2226015360 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6570098580 con 0x7f6588101dd0 2024-02-17T12:49:20.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.878+0000 7f658de50700 1 -- 172.21.15.18:0/2226015360 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f658802d910 con 0x7f65740a2260 2024-02-17T12:49:20.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.880+0000 7f6584ff9700 1 -- 172.21.15.18:0/2226015360 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f658802d910 con 0x7f65740a2260 2024-02-17T12:49:20.872 INFO:teuthology.orchestra.run.smithi018.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-17T12:49:20.872 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120091480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-02-17T12:49:20.872 INFO:teuthology.orchestra.run.smithi018.stdout:smithi119 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y1400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-17T12:49:20.872 INFO:teuthology.orchestra.run.smithi018.stdout:smithi146 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-17T12:49:20.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.882+0000 7f658de50700 1 -- 172.21.15.18:0/2226015360 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65740a2260 msgr2=0x7f65740a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:20.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.882+0000 7f658de50700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65740a2260 0x7f65740a4720 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6588101c30 tx=0x7f6578009570 comp rx=0 tx=0).stop 2024-02-17T12:49:20.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.882+0000 7f658de50700 1 -- 172.21.15.18:0/2226015360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6588101dd0 msgr2=0x7f658819c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:20.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.882+0000 7f658de50700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6588101dd0 0x7f658819c9f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6570003790 tx=0x7f6570003a10 comp rx=0 tx=0).stop 2024-02-17T12:49:20.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.882+0000 7f658de50700 1 -- 172.21.15.18:0/2226015360 shutdown_connections 2024-02-17T12:49:20.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.882+0000 7f658de50700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65740a2260 0x7f65740a4720 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:20.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.882+0000 7f658de50700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6588102770 0x7f658819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:20.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.882+0000 7f658de50700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6588100bd0 0x7f658806af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:20.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.882+0000 7f658de50700 1 --2- 172.21.15.18:0/2226015360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6588101dd0 0x7f658819c9f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:20.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.882+0000 7f658de50700 1 -- 172.21.15.18:0/2226015360 >> 172.21.15.18:0/2226015360 conn(0x7f65880fc190 msgr2=0x7f65881050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:20.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.882+0000 7f658de50700 1 -- 172.21.15.18:0/2226015360 shutdown_connections 2024-02-17T12:49:20.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:20.882+0000 7f658de50700 1 -- 172.21.15.18:0/2226015360 wait complete. 2024-02-17T12:49:21.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:21 smithi018 ceph-mon[32625]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-02-17T12:49:21.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:21 smithi018 ceph-mon[32625]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:21.540 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-17T12:49:21.549 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-02-17T12:49:21.549 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-17T12:49:21.793 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:49:21.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:21 smithi119 ceph-mon[36188]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-02-17T12:49:21.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:21 smithi119 ceph-mon[36188]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:21.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:21 smithi146 ceph-mon[36454]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-02-17T12:49:21.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:21 smithi146 ceph-mon[36454]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:22 smithi018 ceph-mon[32625]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-02-17T12:49:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:22 smithi119 ceph-mon[36188]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-02-17T12:49:22.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:22 smithi146 ceph-mon[36454]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-02-17T12:49:23.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.468+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/938504427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c78101ca0 msgr2=0x7f8c781020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:23.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.468+0000 7f8c7d5fc700 1 --2- 172.21.15.18:0/938504427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c78101ca0 0x7f8c781020c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8c680099c0 tx=0x7f8c68009cd0 comp rx=0 tx=0).stop 2024-02-17T12:49:23.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.468+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/938504427 shutdown_connections 2024-02-17T12:49:23.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.468+0000 7f8c7d5fc700 1 --2- 172.21.15.18:0/938504427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c78103840 0x7f8c7810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:23.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.468+0000 7f8c7d5fc700 1 --2- 172.21.15.18:0/938504427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c78101ca0 0x7f8c781020c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:23.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.468+0000 7f8c7d5fc700 1 --2- 172.21.15.18:0/938504427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c78102ea0 0x7f8c78103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:23.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.468+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/938504427 >> 172.21.15.18:0/938504427 conn(0x7f8c780fd280 msgr2=0x7f8c780ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:23.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.468+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/938504427 shutdown_connections 2024-02-17T12:49:23.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.468+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/938504427 wait complete. 2024-02-17T12:49:23.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.469+0000 7f8c7d5fc700 1 Processor -- start 2024-02-17T12:49:23.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.469+0000 7f8c7d5fc700 1 -- start start 2024-02-17T12:49:23.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.469+0000 7f8c7d5fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c78101ca0 0x7f8c78195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.469+0000 7f8c7d5fc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c78102ea0 0x7f8c78195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.469+0000 7f8c7d5fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c78103840 0x7f8c7819d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.469+0000 7f8c7d5fc700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7819d630 con 0x7f8c78102ea0 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.469+0000 7f8c7d5fc700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7819d770 con 0x7f8c78101ca0 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.469+0000 7f8c7d5fc700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7819d900 con 0x7f8c78103840 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.469+0000 7f8c767fc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c78102ea0 0x7f8c78195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.469+0000 7f8c767fc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c78102ea0 0x7f8c78195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40260/0 (socket says 172.21.15.18:40260) 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.469+0000 7f8c767fc700 1 -- 172.21.15.18:0/2982311099 learned_addr learned my addr 172.21.15.18:0/2982311099 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c777fe700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c78103840 0x7f8c7819d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c76ffd700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c78101ca0 0x7f8c78195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c767fc700 1 -- 172.21.15.18:0/2982311099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c78103840 msgr2=0x7f8c7819d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c767fc700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c78103840 0x7f8c7819d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c767fc700 1 -- 172.21.15.18:0/2982311099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c78101ca0 msgr2=0x7f8c78195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:23.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c767fc700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c78101ca0 0x7f8c78195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:23.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c767fc700 1 -- 172.21.15.18:0/2982311099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c680095f0 con 0x7f8c78102ea0 2024-02-17T12:49:23.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c767fc700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c78102ea0 0x7f8c78195fe0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00d980 tx=0x7f8c6c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:23.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c5ffff700 1 -- 172.21.15.18:0/2982311099 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0098d0 con 0x7f8c78102ea0 2024-02-17T12:49:23.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/2982311099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7819dbe0 con 0x7f8c78102ea0 2024-02-17T12:49:23.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c5ffff700 1 -- 172.21.15.18:0/2982311099 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c6c010460 con 0x7f8c78102ea0 2024-02-17T12:49:23.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c5ffff700 1 -- 172.21.15.18:0/2982311099 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c00f620 con 0x7f8c78102ea0 2024-02-17T12:49:23.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.470+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/2982311099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7819e130 con 0x7f8c78102ea0 2024-02-17T12:49:23.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.471+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/2982311099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c781090d0 con 0x7f8c78102ea0 2024-02-17T12:49:23.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.472+0000 7f8c5ffff700 1 -- 172.21.15.18:0/2982311099 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8c6c0108f0 con 0x7f8c78102ea0 2024-02-17T12:49:23.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.473+0000 7f8c5ffff700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8c600a6560 0x7f8c600a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:23.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.473+0000 7f8c76ffd700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8c600a6560 0x7f8c600a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:23.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.473+0000 7f8c5ffff700 1 -- 172.21.15.18:0/2982311099 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8c6c0cd9d0 con 0x7f8c78102ea0 2024-02-17T12:49:23.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.474+0000 7f8c76ffd700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8c600a6560 0x7f8c600a8a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009f50 tx=0x7f8c6800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:23.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.478+0000 7f8c5ffff700 1 -- 172.21.15.18:0/2982311099 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c6c0969d0 con 0x7f8c78102ea0 2024-02-17T12:49:23.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.725+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/2982311099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8c78068930 con 0x7f8c78102ea0 2024-02-17T12:49:23.718 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:23 smithi018 ceph-mon[32625]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-02-17T12:49:23.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.725+0000 7f8c5ffff700 1 -- 172.21.15.18:0/2982311099 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+2970 (secure 0 0 0) 0x7f8c6c016bb0 con 0x7f8c78102ea0 2024-02-17T12:49:23.718 INFO:teuthology.orchestra.run.smithi018.stdout:e25 2024-02-17T12:49:23.718 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-17T12:49:23.718 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 25 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-02-17T12:47:45.407920+0000 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-02-17T12:47:51.719041+0000 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 300 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-02-17T12:49:23.719 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 0 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 5 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1,2,3,4 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=24512,1=24476,2=24455,3=24448,4=24422} 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-02-17T12:49:23.720 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 0 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.b{0:24512} state up:active seq 3 addr [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.i{0:24475} state up:standby-replay seq 1 addr [v2:172.21.15.146:6838/3061839650,v1:172.21.15.146:6839/3061839650] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.h{1:24476} state up:active seq 6 addr [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.c{1:24460} state up:standby-replay seq 1 addr [v2:172.21.15.146:6836/1732085011,v1:172.21.15.146:6837/1732085011] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.e{2:24455} state up:active seq 8 addr [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.l{2:24403} state up:standby-replay seq 1 addr [v2:172.21.15.146:6832/4241790141,v1:172.21.15.146:6833/4241790141] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.f{3:24448} state up:active seq 8 addr [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.a{3:14655} state up:standby-replay seq 1 addr [v2:172.21.15.18:6840/2569825427,v1:172.21.15.18:6841/2569825427] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.k{4:24422} state up:active seq 11 addr [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T12:49:23.721 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.g{4:14637} state up:standby-replay seq 1 addr [v2:172.21.15.18:6838/1294470347,v1:172.21.15.18:6839/1294470347] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T12:49:23.722 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:23.722 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:23.722 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-02-17T12:49:23.722 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:23.722 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.j{-1:14550} state up:standby seq 1 addr [v2:172.21.15.18:6834/3401600921,v1:172.21.15.18:6835/3401600921] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T12:49:23.722 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.18:6836/3018278996,v1:172.21.15.18:6837/3018278996] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T12:49:23.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.728+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/2982311099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8c600a6560 msgr2=0x7f8c600a8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:23.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.728+0000 7f8c7d5fc700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8c600a6560 0x7f8c600a8a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009f50 tx=0x7f8c6800b540 comp rx=0 tx=0).stop 2024-02-17T12:49:23.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.728+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/2982311099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c78102ea0 msgr2=0x7f8c78195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:23.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.728+0000 7f8c7d5fc700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c78102ea0 0x7f8c78195fe0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00d980 tx=0x7f8c6c00dc90 comp rx=0 tx=0).stop 2024-02-17T12:49:23.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.728+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/2982311099 shutdown_connections 2024-02-17T12:49:23.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.729+0000 7f8c7d5fc700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8c600a6560 0x7f8c600a8a20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:23.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.729+0000 7f8c7d5fc700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c78103840 0x7f8c7819d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:23.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.729+0000 7f8c7d5fc700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c78101ca0 0x7f8c78195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:23.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.729+0000 7f8c7d5fc700 1 --2- 172.21.15.18:0/2982311099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c78102ea0 0x7f8c78195fe0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:23.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.729+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/2982311099 >> 172.21.15.18:0/2982311099 conn(0x7f8c780fd280 msgr2=0x7f8c78106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:23.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.729+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/2982311099 shutdown_connections 2024-02-17T12:49:23.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:23.729+0000 7f8c7d5fc700 1 -- 172.21.15.18:0/2982311099 wait complete. 2024-02-17T12:49:23.724 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-02-17T12:49:23.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:23 smithi119 ceph-mon[36188]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-02-17T12:49:23.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:23 smithi146 ceph-mon[36454]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-02-17T12:49:24.426 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- bash -c 'ceph osd dump' 2024-02-17T12:49:24.674 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:49:24.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:24 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2982311099' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-17T12:49:24.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:24 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2982311099' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-17T12:49:24.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:24 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2982311099' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-17T12:49:25.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:25 smithi018 ceph-mon[32625]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-17T12:49:25.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:25 smithi119 ceph-mon[36188]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-17T12:49:25.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:25 smithi146 ceph-mon[36454]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-17T12:49:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.568+0000 7f8c85498700 1 -- 172.21.15.18:0/603368627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c80102ec0 msgr2=0x7f8c80103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.568+0000 7f8c85498700 1 --2- 172.21.15.18:0/603368627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c80102ec0 0x7f8c80103320 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8c740099c0 tx=0x7f8c74009cd0 comp rx=0 tx=0).stop 2024-02-17T12:49:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.568+0000 7f8c85498700 1 -- 172.21.15.18:0/603368627 shutdown_connections 2024-02-17T12:49:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.568+0000 7f8c85498700 1 --2- 172.21.15.18:0/603368627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80103860 0x7f8c8010a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.568+0000 7f8c85498700 1 --2- 172.21.15.18:0/603368627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c80101cc0 0x7f8c801020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.568+0000 7f8c85498700 1 --2- 172.21.15.18:0/603368627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c80102ec0 0x7f8c80103320 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.568+0000 7f8c85498700 1 -- 172.21.15.18:0/603368627 >> 172.21.15.18:0/603368627 conn(0x7f8c800fd220 msgr2=0x7f8c800ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.568+0000 7f8c85498700 1 -- 172.21.15.18:0/603368627 shutdown_connections 2024-02-17T12:49:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.568+0000 7f8c85498700 1 -- 172.21.15.18:0/603368627 wait complete. 2024-02-17T12:49:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.568+0000 7f8c85498700 1 Processor -- start 2024-02-17T12:49:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c85498700 1 -- start start 2024-02-17T12:49:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c85498700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c80101cc0 0x7f8c80197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c85498700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c80102ec0 0x7f8c801980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c85498700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80103860 0x7f8c8019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c85498700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8019f710 con 0x7f8c80102ec0 2024-02-17T12:49:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c85498700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8019f850 con 0x7f8c80101cc0 2024-02-17T12:49:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c85498700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8019f9e0 con 0x7f8c80103860 2024-02-17T12:49:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c7e7fc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c80102ec0 0x7f8c801980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c7e7fc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c80102ec0 0x7f8c801980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40276/0 (socket says 172.21.15.18:40276) 2024-02-17T12:49:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c7e7fc700 1 -- 172.21.15.18:0/2733913960 learned_addr learned my addr 172.21.15.18:0/2733913960 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c7effd700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c80101cc0 0x7f8c80197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c7e7fc700 1 -- 172.21.15.18:0/2733913960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80103860 msgr2=0x7f8c8019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c7f7fe700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80103860 0x7f8c8019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c7e7fc700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80103860 0x7f8c8019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c7e7fc700 1 -- 172.21.15.18:0/2733913960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c80101cc0 msgr2=0x7f8c80197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c7e7fc700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c80101cc0 0x7f8c80197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c7e7fc700 1 -- 172.21.15.18:0/2733913960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c740095f0 con 0x7f8c80102ec0 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c7f7fe700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80103860 0x7f8c8019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.569+0000 7f8c7effd700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c80101cc0 0x7f8c80197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.570+0000 7f8c7e7fc700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c80102ec0 0x7f8c801980c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005910 tx=0x7f8c74003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.570+0000 7f8c67fff700 1 -- 172.21.15.18:0/2733913960 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7401d070 con 0x7f8c80102ec0 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.570+0000 7f8c85498700 1 -- 172.21.15.18:0/2733913960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8019fc60 con 0x7f8c80102ec0 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.570+0000 7f8c67fff700 1 -- 172.21.15.18:0/2733913960 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c74022470 con 0x7f8c80102ec0 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.570+0000 7f8c67fff700 1 -- 172.21.15.18:0/2733913960 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7400f6c0 con 0x7f8c80102ec0 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.570+0000 7f8c85498700 1 -- 172.21.15.18:0/2733913960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c801a0100 con 0x7f8c80102ec0 2024-02-17T12:49:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.571+0000 7f8c85498700 1 -- 172.21.15.18:0/2733913960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c80109020 con 0x7f8c80102ec0 2024-02-17T12:49:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.572+0000 7f8c67fff700 1 -- 172.21.15.18:0/2733913960 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8c74022890 con 0x7f8c80102ec0 2024-02-17T12:49:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.573+0000 7f8c67fff700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8c680a2150 0x7f8c680a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.573+0000 7f8c7effd700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8c680a2150 0x7f8c680a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.573+0000 7f8c67fff700 1 -- 172.21.15.18:0/2733913960 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8c740cf840 con 0x7f8c80102ec0 2024-02-17T12:49:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.573+0000 7f8c7effd700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8c680a2150 0x7f8c680a4610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c70005fd0 tx=0x7f8c70005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:26.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.577+0000 7f8c67fff700 1 -- 172.21.15.18:0/2733913960 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c740988d0 con 0x7f8c80102ec0 2024-02-17T12:49:26.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.781+0000 7f8c85498700 1 -- 172.21.15.18:0/2733913960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f8c8010d600 con 0x7f8c80102ec0 2024-02-17T12:49:26.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.782+0000 7f8c67fff700 1 -- 172.21.15.18:0/2733913960 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5259 (secure 0 0 0) 0x7f8c7409bf90 con 0x7f8c80102ec0 2024-02-17T12:49:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 88 2024-02-17T12:49:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T12:49:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-02-17T12:39:48.882857+0000 2024-02-17T12:49:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-02-17T12:48:52.149237+0000 2024-02-17T12:49:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-17T12:49:26.774 INFO:teuthology.orchestra.run.smithi018.stdout:crush_version 26 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:full_ratio 0.95 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:backfillfull_ratio 0.9 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:nearfull_ratio 0.85 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:require_min_compat_client luminous 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:min_compat_client luminous 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:require_osd_release squid 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:stretch_mode_enabled false 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:max_osd 12 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.18:6802/417036941,v1:172.21.15.18:6803/417036941] [v2:172.21.15.18:6804/417036941,v1:172.21.15.18:6805/417036941] exists,up 31ca3f0e-c84e-4779-984f-daae32ce96f3 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6800/3915030131,v1:172.21.15.119:6801/3915030131] [v2:172.21.15.119:6802/3915030131,v1:172.21.15.119:6803/3915030131] exists,up 8c69905b-1e0f-4b63-90d6-bb6d7a5f752f 2024-02-17T12:49:26.775 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6800/3369250578,v1:172.21.15.146:6801/3369250578] [v2:172.21.15.146:6802/3369250578,v1:172.21.15.146:6803/3369250578] exists,up 5964f903-92b4-4f81-b9d7-861f14ef1d38 2024-02-17T12:49:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.18:6810/3584006256,v1:172.21.15.18:6811/3584006256] [v2:172.21.15.18:6812/3584006256,v1:172.21.15.18:6813/3584006256] exists,up c7ef3a4f-6073-4998-812e-dc57d4760ecb 2024-02-17T12:49:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6808/1444866818,v1:172.21.15.119:6809/1444866818] [v2:172.21.15.119:6810/1444866818,v1:172.21.15.119:6811/1444866818] exists,up c1c67fe4-8ba9-4767-89a7-edab8c686496 2024-02-17T12:49:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6808/4227453651,v1:172.21.15.146:6809/4227453651] [v2:172.21.15.146:6810/4227453651,v1:172.21.15.146:6811/4227453651] exists,up 33c58b7e-7d68-40e7-b8bb-b441343fb6de 2024-02-17T12:49:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.18:6818/171901734,v1:172.21.15.18:6819/171901734] [v2:172.21.15.18:6820/171901734,v1:172.21.15.18:6821/171901734] exists,up c3ef630d-efa7-4b1f-98f5-6235d1081fc1 2024-02-17T12:49:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6816/833114737,v1:172.21.15.119:6817/833114737] [v2:172.21.15.119:6818/833114737,v1:172.21.15.119:6819/833114737] exists,up 4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a 2024-02-17T12:49:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6816/1027539241,v1:172.21.15.146:6817/1027539241] [v2:172.21.15.146:6818/1027539241,v1:172.21.15.146:6819/1027539241] exists,up e49e3406-56e4-422e-b157-e5c12ee087a8 2024-02-17T12:49:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:osd.9 up in weight 1 up_from 68 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.18:6826/3939472833,v1:172.21.15.18:6827/3939472833] [v2:172.21.15.18:6828/3939472833,v1:172.21.15.18:6829/3939472833] exists,up a41cd245-5402-474b-9e7e-f15557523fd5 2024-02-17T12:49:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6824/3116452609,v1:172.21.15.119:6825/3116452609] [v2:172.21.15.119:6826/3116452609,v1:172.21.15.119:6827/3116452609] exists,up 5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d 2024-02-17T12:49:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6824/2169300109,v1:172.21.15.146:6825/2169300109] [v2:172.21.15.146:6826/2169300109,v1:172.21.15.146:6827/2169300109] exists,up e507f2ad-80c0-44f5-be10-38d3da841477 2024-02-17T12:49:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 2.b [8,11] 2024-02-17T12:49:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 2.e [2,1] 2024-02-17T12:49:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 2.1b [2,4] 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 2.1d [8,11] 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 3.1 [4,9] 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 3.25 [5,9] 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 3.2c [4,9] 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/2724676594 expires 2024-02-18T12:40:50.114081+0000 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/1333332933 expires 2024-02-18T12:40:50.114081+0000 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/1092699235 expires 2024-02-18T12:40:50.114081+0000 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/1745633107 expires 2024-02-18T12:40:22.805777+0000 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:6801/1543388552 expires 2024-02-18T12:40:22.805777+0000 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/3602994927 expires 2024-02-18T12:40:22.805777+0000 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/2150265194 expires 2024-02-18T12:40:22.805777+0000 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:6800/1086757713 expires 2024-02-18T12:40:50.114081+0000 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:6801/1086757713 expires 2024-02-18T12:40:50.114081+0000 2024-02-17T12:49:26.777 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:6800/1543388552 expires 2024-02-18T12:40:22.805777+0000 2024-02-17T12:49:26.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.784+0000 7f8c85498700 1 -- 172.21.15.18:0/2733913960 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8c680a2150 msgr2=0x7f8c680a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:26.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.784+0000 7f8c85498700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8c680a2150 0x7f8c680a4610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c70005fd0 tx=0x7f8c70005e40 comp rx=0 tx=0).stop 2024-02-17T12:49:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.784+0000 7f8c85498700 1 -- 172.21.15.18:0/2733913960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c80102ec0 msgr2=0x7f8c801980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.784+0000 7f8c85498700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c80102ec0 0x7f8c801980c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005910 tx=0x7f8c74003a10 comp rx=0 tx=0).stop 2024-02-17T12:49:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.784+0000 7f8c85498700 1 -- 172.21.15.18:0/2733913960 shutdown_connections 2024-02-17T12:49:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.784+0000 7f8c85498700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8c680a2150 0x7f8c680a4610 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.784+0000 7f8c85498700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80103860 0x7f8c8019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.784+0000 7f8c85498700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c80101cc0 0x7f8c80197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.784+0000 7f8c85498700 1 --2- 172.21.15.18:0/2733913960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c80102ec0 0x7f8c801980c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.784+0000 7f8c85498700 1 -- 172.21.15.18:0/2733913960 >> 172.21.15.18:0/2733913960 conn(0x7f8c800fd220 msgr2=0x7f8c80106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.785+0000 7f8c85498700 1 -- 172.21.15.18:0/2733913960 shutdown_connections 2024-02-17T12:49:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:26.785+0000 7f8c85498700 1 -- 172.21.15.18:0/2733913960 wait complete. 2024-02-17T12:49:27.390 INFO:teuthology.run_tasks:Running task fs.ready... 2024-02-17T12:49:27.402 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:49:27.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.518+0000 7f70f48d4640 1 -- 172.21.15.18:0/1780036597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70f01011a0 msgr2=0x7f70f01015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:27.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.518+0000 7f70f48d4640 1 --2- 172.21.15.18:0/1780036597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70f01011a0 0x7f70f01015c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009900 tx=0x7f70e002bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:27.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.518+0000 7f70f48d4640 1 -- 172.21.15.18:0/1780036597 shutdown_connections 2024-02-17T12:49:27.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.518+0000 7f70f48d4640 1 --2- 172.21.15.18:0/1780036597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f0102d50 0x7f70f01095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.518+0000 7f70f48d4640 1 --2- 172.21.15.18:0/1780036597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f0102390 0x7f70f0102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.518+0000 7f70f48d4640 1 --2- 172.21.15.18:0/1780036597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70f01011a0 0x7f70f01015c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.518+0000 7f70f48d4640 1 -- 172.21.15.18:0/1780036597 >> 172.21.15.18:0/1780036597 conn(0x7f70f00fc730 msgr2=0x7f70f00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:27.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.518+0000 7f70f48d4640 1 -- 172.21.15.18:0/1780036597 shutdown_connections 2024-02-17T12:49:27.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.518+0000 7f70f48d4640 1 -- 172.21.15.18:0/1780036597 wait complete. 2024-02-17T12:49:27.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.519+0000 7f70f48d4640 1 Processor -- start 2024-02-17T12:49:27.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.519+0000 7f70f48d4640 1 -- start start 2024-02-17T12:49:27.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.519+0000 7f70f48d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f01011a0 0x7f70f01971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:27.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.519+0000 7f70f48d4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f0102390 0x7f70f01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:27.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70f48d4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70f0102d50 0x7f70f019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:27.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70f48d4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f019ec90 con 0x7f70f0102d50 2024-02-17T12:49:27.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70f48d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f019ee00 con 0x7f70f01011a0 2024-02-17T12:49:27.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70f48d4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f019efc0 con 0x7f70f0102390 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70eed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70f0102d50 0x7f70f019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70eed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70f0102d50 0x7f70f019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40298/0 (socket says 172.21.15.18:40298) 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70eed9d640 1 -- 172.21.15.18:0/1898214213 learned_addr learned my addr 172.21.15.18:0/1898214213 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70ee59c640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f01011a0 0x7f70f01971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70e7fff640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f0102390 0x7f70f01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70eed9d640 1 -- 172.21.15.18:0/1898214213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f0102390 msgr2=0x7f70f01976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70eed9d640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f0102390 0x7f70f01976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70eed9d640 1 -- 172.21.15.18:0/1898214213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f01011a0 msgr2=0x7f70f01971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70eed9d640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f01011a0 0x7f70f01971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70eed9d640 1 -- 172.21.15.18:0/1898214213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e0009560 con 0x7f70f0102d50 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70eed9d640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70f0102d50 0x7f70f019e750 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f70e800eaa0 tx=0x7f70e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70e77fe640 1 -- 172.21.15.18:0/1898214213 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e800cdb0 con 0x7f70f0102d50 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70f48d4640 1 -- 172.21.15.18:0/1898214213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f019f2a0 con 0x7f70f0102d50 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70e77fe640 1 -- 172.21.15.18:0/1898214213 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70e8005020 con 0x7f70f0102d50 2024-02-17T12:49:27.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70e77fe640 1 -- 172.21.15.18:0/1898214213 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e80116b0 con 0x7f70f0102d50 2024-02-17T12:49:27.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.520+0000 7f70f48d4640 1 -- 172.21.15.18:0/1898214213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f019f760 con 0x7f70f0102d50 2024-02-17T12:49:27.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.521+0000 7f70f48d4640 1 -- 172.21.15.18:0/1898214213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f01084c0 con 0x7f70f0102d50 2024-02-17T12:49:27.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.522+0000 7f70e77fe640 1 -- 172.21.15.18:0/1898214213 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f70e8005520 con 0x7f70f0102d50 2024-02-17T12:49:27.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.523+0000 7f70e77fe640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f70dc0a20c0 0x7f70dc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:27.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.523+0000 7f70ee59c640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f70dc0a20c0 0x7f70dc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:27.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.524+0000 7f70e77fe640 1 -- 172.21.15.18:0/1898214213 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f70e80ce2a0 con 0x7f70f0102d50 2024-02-17T12:49:27.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.524+0000 7f70ee59c640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f70dc0a20c0 0x7f70dc0a4580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70e0005ab0 tx=0x7f70e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:27.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.527+0000 7f70e77fe640 1 -- 172.21.15.18:0/1898214213 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f70e80972a0 con 0x7f70f0102d50 2024-02-17T12:49:27.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.710+0000 7f70f48d4640 1 -- 172.21.15.18:0/1898214213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70f002ce40 con 0x7f70f0102d50 2024-02-17T12:49:27.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.711+0000 7f70e77fe640 1 -- 172.21.15.18:0/1898214213 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22607 (secure 0 0 0) 0x7f70e801f020 con 0x7f70f0102d50 2024-02-17T12:49:27.705 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:27.705 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":88,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:48:52.149237+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:49:27.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.713+0000 7f70f48d4640 1 -- 172.21.15.18:0/1898214213 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f70dc0a20c0 msgr2=0x7f70dc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:27.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.713+0000 7f70f48d4640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f70dc0a20c0 0x7f70dc0a4580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70e0005ab0 tx=0x7f70e0005fd0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.713+0000 7f70f48d4640 1 -- 172.21.15.18:0/1898214213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70f0102d50 msgr2=0x7f70f019e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:27.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.713+0000 7f70f48d4640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70f0102d50 0x7f70f019e750 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f70e800eaa0 tx=0x7f70e800ef60 comp rx=0 tx=0).stop 2024-02-17T12:49:27.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.714+0000 7f70f48d4640 1 -- 172.21.15.18:0/1898214213 shutdown_connections 2024-02-17T12:49:27.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.714+0000 7f70f48d4640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f70dc0a20c0 0x7f70dc0a4580 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.714+0000 7f70f48d4640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70f0102d50 0x7f70f019e750 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.714+0000 7f70f48d4640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f0102390 0x7f70f01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.714+0000 7f70f48d4640 1 --2- 172.21.15.18:0/1898214213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f01011a0 0x7f70f01971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.714+0000 7f70f48d4640 1 -- 172.21.15.18:0/1898214213 >> 172.21.15.18:0/1898214213 conn(0x7f70f00fc730 msgr2=0x7f70f0105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:27.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.714+0000 7f70f48d4640 1 -- 172.21.15.18:0/1898214213 shutdown_connections 2024-02-17T12:49:27.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.714+0000 7f70f48d4640 1 -- 172.21.15.18:0/1898214213 wait complete. 2024-02-17T12:49:27.716 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:27 smithi018 ceph-mon[32625]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-02-17T12:49:27.716 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:27 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2733913960' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-17T12:49:27.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T12:43:12.450140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T12:47:43.690543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T12:47:44.653229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T12:49:27.718 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T12:49:27.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.831+0000 7f1a25de2640 1 -- 172.21.15.18:0/1413669280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a20101270 msgr2=0x7f1a201016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:27.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.831+0000 7f1a25de2640 1 --2- 172.21.15.18:0/1413669280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a20101270 0x7f1a201016f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009900 tx=0x7f1a1402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.831+0000 7f1a25de2640 1 -- 172.21.15.18:0/1413669280 shutdown_connections 2024-02-17T12:49:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.831+0000 7f1a25de2640 1 --2- 172.21.15.18:0/1413669280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a20101c30 0x7f1a201084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.831+0000 7f1a25de2640 1 --2- 172.21.15.18:0/1413669280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a20101270 0x7f1a201016f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.831+0000 7f1a25de2640 1 --2- 172.21.15.18:0/1413669280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a20100080 0x7f1a201004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.831+0000 7f1a25de2640 1 -- 172.21.15.18:0/1413669280 >> 172.21.15.18:0/1413669280 conn(0x7f1a200fb610 msgr2=0x7f1a200fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a25de2640 1 -- 172.21.15.18:0/1413669280 shutdown_connections 2024-02-17T12:49:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a25de2640 1 -- 172.21.15.18:0/1413669280 wait complete. 2024-02-17T12:49:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a25de2640 1 Processor -- start 2024-02-17T12:49:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a25de2640 1 -- start start 2024-02-17T12:49:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a25de2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a20100080 0x7f1a20196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a25de2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a20101270 0x7f1a201966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a25de2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a20101c30 0x7f1a2019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a25de2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2019dc60 con 0x7f1a20101270 2024-02-17T12:49:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a25de2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2019ddd0 con 0x7f1a20100080 2024-02-17T12:49:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a25de2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2019df90 con 0x7f1a20101c30 2024-02-17T12:49:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a1effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a20101270 0x7f1a201966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a1effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a20101270 0x7f1a201966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40318/0 (socket says 172.21.15.18:40318) 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.832+0000 7f1a1effd640 1 -- 172.21.15.18:0/2080377910 learned_addr learned my addr 172.21.15.18:0/2080377910 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a1f7fe640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a20100080 0x7f1a20196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a1ffff640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a20101c30 0x7f1a2019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a1effd640 1 -- 172.21.15.18:0/2080377910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a20101c30 msgr2=0x7f1a2019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a1effd640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a20101c30 0x7f1a2019d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a1effd640 1 -- 172.21.15.18:0/2080377910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a20100080 msgr2=0x7f1a20196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a1effd640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a20100080 0x7f1a20196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a1effd640 1 -- 172.21.15.18:0/2080377910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a14009560 con 0x7f1a20101270 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a1f7fe640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a20100080 0x7f1a20196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a1effd640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a20101270 0x7f1a201966b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1a14006010 tx=0x7f1a140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a1cff9640 1 -- 172.21.15.18:0/2080377910 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a140044e0 con 0x7f1a20101270 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a1cff9640 1 -- 172.21.15.18:0/2080377910 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a140054e0 con 0x7f1a20101270 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a25de2640 1 -- 172.21.15.18:0/2080377910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2019e210 con 0x7f1a20101270 2024-02-17T12:49:27.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a1cff9640 1 -- 172.21.15.18:0/2080377910 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1403e450 con 0x7f1a20101270 2024-02-17T12:49:27.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.833+0000 7f1a25de2640 1 -- 172.21.15.18:0/2080377910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2019e720 con 0x7f1a20101270 2024-02-17T12:49:27.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.834+0000 7f1a25de2640 1 -- 172.21.15.18:0/2080377910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a201074e0 con 0x7f1a20101270 2024-02-17T12:49:27.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.835+0000 7f1a1cff9640 1 -- 172.21.15.18:0/2080377910 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1a140035f0 con 0x7f1a20101270 2024-02-17T12:49:27.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.836+0000 7f1a1cff9640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1a080a20c0 0x7f1a080a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:27.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.836+0000 7f1a1f7fe640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1a080a20c0 0x7f1a080a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:27.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.836+0000 7f1a1cff9640 1 -- 172.21.15.18:0/2080377910 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1a140f00d0 con 0x7f1a20101270 2024-02-17T12:49:27.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.836+0000 7f1a1f7fe640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1a080a20c0 0x7f1a080a4580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1a100058c0 tx=0x7f1a10005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:27.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:27.840+0000 7f1a1cff9640 1 -- 172.21.15.18:0/2080377910 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a140b90d0 con 0x7f1a20101270 2024-02-17T12:49:27.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:27 smithi119 ceph-mon[36188]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-02-17T12:49:27.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:27 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2733913960' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-17T12:49:27.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:27 smithi146 ceph-mon[36454]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-02-17T12:49:27.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:27 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2733913960' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-17T12:49:28.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.020+0000 7f1a25de2640 1 -- 172.21.15.18:0/2080377910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a2002cda0 con 0x7f1a20101270 2024-02-17T12:49:28.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.021+0000 7f1a1cff9640 1 -- 172.21.15.18:0/2080377910 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a140bc790 con 0x7f1a20101270 2024-02-17T12:49:28.013 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-02-17T12:49:28.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.023+0000 7f1a25de2640 1 -- 172.21.15.18:0/2080377910 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1a080a20c0 msgr2=0x7f1a080a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.023+0000 7f1a25de2640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1a080a20c0 0x7f1a080a4580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1a100058c0 tx=0x7f1a10005850 comp rx=0 tx=0).stop 2024-02-17T12:49:28.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.023+0000 7f1a25de2640 1 -- 172.21.15.18:0/2080377910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a20101270 msgr2=0x7f1a201966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.023+0000 7f1a25de2640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a20101270 0x7f1a201966b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1a14006010 tx=0x7f1a140042f0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.023+0000 7f1a25de2640 1 -- 172.21.15.18:0/2080377910 shutdown_connections 2024-02-17T12:49:28.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.023+0000 7f1a25de2640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1a080a20c0 0x7f1a080a4580 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.023+0000 7f1a25de2640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a20101c30 0x7f1a2019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.023+0000 7f1a25de2640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a20101270 0x7f1a201966b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.023+0000 7f1a25de2640 1 --2- 172.21.15.18:0/2080377910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a20100080 0x7f1a20196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.023+0000 7f1a25de2640 1 -- 172.21.15.18:0/2080377910 >> 172.21.15.18:0/2080377910 conn(0x7f1a200fb610 msgr2=0x7f1a201046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:28.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.023+0000 7f1a25de2640 1 -- 172.21.15.18:0/2080377910 shutdown_connections 2024-02-17T12:49:28.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.023+0000 7f1a25de2640 1 -- 172.21.15.18:0/2080377910 wait complete. 2024-02-17T12:49:28.026 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T12:49:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.139+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/2273685870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f240ffbb0 msgr2=0x7f6f24109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.139+0000 7f6f2b5e2640 1 --2- 172.21.15.18:0/2273685870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f240ffbb0 0x7f6f24109760 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009ca0 tx=0x7f6f1c02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.139+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/2273685870 shutdown_connections 2024-02-17T12:49:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.139+0000 7f6f2b5e2640 1 --2- 172.21.15.18:0/2273685870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f240ffbb0 0x7f6f24109760 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.139+0000 7f6f2b5e2640 1 --2- 172.21.15.18:0/2273685870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f240ff1f0 0x7f6f240ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.139+0000 7f6f2b5e2640 1 --2- 172.21.15.18:0/2273685870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f24102780 0x7f6f24102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.139+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/2273685870 >> 172.21.15.18:0/2273685870 conn(0x7f6f240fad70 msgr2=0x7f6f240fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.139+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/2273685870 shutdown_connections 2024-02-17T12:49:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.139+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/2273685870 wait complete. 2024-02-17T12:49:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.139+0000 7f6f2b5e2640 1 Processor -- start 2024-02-17T12:49:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f2b5e2640 1 -- start start 2024-02-17T12:49:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f2b5e2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f240ff1f0 0x7f6f24197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f2b5e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f240ffbb0 0x7f6f24197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f2b5e2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f24102780 0x7f6f2419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f2b5e2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2419ec20 con 0x7f6f24102780 2024-02-17T12:49:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f2b5e2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2419ed90 con 0x7f6f240ffbb0 2024-02-17T12:49:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f2b5e2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2419ef50 con 0x7f6f240ff1f0 2024-02-17T12:49:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f29b7f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f24102780 0x7f6f2419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f29b7f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f24102780 0x7f6f2419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40350/0 (socket says 172.21.15.18:40350) 2024-02-17T12:49:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f29b7f640 1 -- 172.21.15.18:0/3956835069 learned_addr learned my addr 172.21.15.18:0/3956835069 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f2937e640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f240ff1f0 0x7f6f24197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f28b7d640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f240ffbb0 0x7f6f24197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f29b7f640 1 -- 172.21.15.18:0/3956835069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f240ff1f0 msgr2=0x7f6f24197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f29b7f640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f240ff1f0 0x7f6f24197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f29b7f640 1 -- 172.21.15.18:0/3956835069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f240ffbb0 msgr2=0x7f6f24197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f29b7f640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f240ffbb0 0x7f6f24197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f29b7f640 1 -- 172.21.15.18:0/3956835069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c009900 con 0x7f6f24102780 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f28b7d640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f240ffbb0 0x7f6f24197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f29b7f640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f24102780 0x7f6f2419e6e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c037040 tx=0x7f6f1c0346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f1a7fc640 1 -- 172.21.15.18:0/3956835069 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c003970 con 0x7f6f24102780 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f1a7fc640 1 -- 172.21.15.18:0/3956835069 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f1c004040 con 0x7f6f24102780 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f1a7fc640 1 -- 172.21.15.18:0/3956835069 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0406b0 con 0x7f6f24102780 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.140+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/3956835069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2419f1d0 con 0x7f6f24102780 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.141+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/3956835069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2419f6d0 con 0x7f6f24102780 2024-02-17T12:49:28.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.141+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/3956835069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f24108570 con 0x7f6f24102780 2024-02-17T12:49:28.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.142+0000 7f6f1a7fc640 1 -- 172.21.15.18:0/3956835069 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6f1c00a040 con 0x7f6f24102780 2024-02-17T12:49:28.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.143+0000 7f6f1a7fc640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6f100a2140 0x7f6f100a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:28.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.143+0000 7f6f2937e640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6f100a2140 0x7f6f100a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:28.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.143+0000 7f6f1a7fc640 1 -- 172.21.15.18:0/3956835069 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6f1c0f12a0 con 0x7f6f24102780 2024-02-17T12:49:28.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.143+0000 7f6f2937e640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6f100a2140 0x7f6f100a4600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f200058c0 tx=0x7f6f20005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:28.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.147+0000 7f6f1a7fc640 1 -- 172.21.15.18:0/3956835069 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6f1c0ba330 con 0x7f6f24102780 2024-02-17T12:49:28.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.327+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/3956835069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f2410cbe0 con 0x7f6f24102780 2024-02-17T12:49:28.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.328+0000 7f6f1a7fc640 1 -- 172.21.15.18:0/3956835069 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f1c0bd9f0 con 0x7f6f24102780 2024-02-17T12:49:28.320 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-02-17T12:49:28.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.329+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/3956835069 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6f100a2140 msgr2=0x7f6f100a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.329+0000 7f6f2b5e2640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6f100a2140 0x7f6f100a4600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f200058c0 tx=0x7f6f20005850 comp rx=0 tx=0).stop 2024-02-17T12:49:28.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.329+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/3956835069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f24102780 msgr2=0x7f6f2419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.329+0000 7f6f2b5e2640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f24102780 0x7f6f2419e6e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c037040 tx=0x7f6f1c0346f0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.330+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/3956835069 shutdown_connections 2024-02-17T12:49:28.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.330+0000 7f6f2b5e2640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6f100a2140 0x7f6f100a4600 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.330+0000 7f6f2b5e2640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f24102780 0x7f6f2419e6e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.330+0000 7f6f2b5e2640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f240ffbb0 0x7f6f24197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.330+0000 7f6f2b5e2640 1 --2- 172.21.15.18:0/3956835069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f240ff1f0 0x7f6f24197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.330+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/3956835069 >> 172.21.15.18:0/3956835069 conn(0x7f6f240fad70 msgr2=0x7f6f24105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:28.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.330+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/3956835069 shutdown_connections 2024-02-17T12:49:28.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.330+0000 7f6f2b5e2640 1 -- 172.21.15.18:0/3956835069 wait complete. 2024-02-17T12:49:28.334 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T12:49:28.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.447+0000 7f51595d0640 1 -- 172.21.15.18:0/2932754083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5154102d40 msgr2=0x7f51541095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.447+0000 7f51595d0640 1 --2- 172.21.15.18:0/2932754083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5154102d40 0x7f51541095d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f514c009ca0 tx=0x7f514c02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.447+0000 7f51595d0640 1 -- 172.21.15.18:0/2932754083 shutdown_connections 2024-02-17T12:49:28.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.447+0000 7f51595d0640 1 --2- 172.21.15.18:0/2932754083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5154102d40 0x7f51541095d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.447+0000 7f51595d0640 1 --2- 172.21.15.18:0/2932754083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5154102380 0x7f5154102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.447+0000 7f51595d0640 1 --2- 172.21.15.18:0/2932754083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5154101190 0x7f51541015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.447+0000 7f51595d0640 1 -- 172.21.15.18:0/2932754083 >> 172.21.15.18:0/2932754083 conn(0x7f51540fc720 msgr2=0x7f51540feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.447+0000 7f51595d0640 1 -- 172.21.15.18:0/2932754083 shutdown_connections 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.447+0000 7f51595d0640 1 -- 172.21.15.18:0/2932754083 wait complete. 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.447+0000 7f51595d0640 1 Processor -- start 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.447+0000 7f51595d0640 1 -- start start 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.447+0000 7f51595d0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5154101190 0x7f51541971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f51595d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5154102380 0x7f5154197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f51595d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5154102d40 0x7f515419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f51595d0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515419ecd0 con 0x7f5154101190 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f51595d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515419ee40 con 0x7f5154102380 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f51595d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515419f000 con 0x7f5154102d40 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f5152ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5154101190 0x7f51541971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f5152ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5154101190 0x7f51541971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40374/0 (socket says 172.21.15.18:40374) 2024-02-17T12:49:28.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f5152ffd640 1 -- 172.21.15.18:0/3204809420 learned_addr learned my addr 172.21.15.18:0/3204809420 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f51527fc640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5154102380 0x7f5154197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f51537fe640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5154102d40 0x7f515419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f5152ffd640 1 -- 172.21.15.18:0/3204809420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5154102d40 msgr2=0x7f515419e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f5152ffd640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5154102d40 0x7f515419e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f5152ffd640 1 -- 172.21.15.18:0/3204809420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5154102380 msgr2=0x7f5154197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f5152ffd640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5154102380 0x7f5154197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f5152ffd640 1 -- 172.21.15.18:0/3204809420 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c009900 con 0x7f5154101190 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f51537fe640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5154102d40 0x7f515419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f5152ffd640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5154101190 0x7f51541971e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f514400d770 tx=0x7f514400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f513bfff640 1 -- 172.21.15.18:0/3204809420 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144009c30 con 0x7f5154101190 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f513bfff640 1 -- 172.21.15.18:0/3204809420 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f514400de80 con 0x7f5154101190 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f51595d0640 1 -- 172.21.15.18:0/3204809420 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515419f2e0 con 0x7f5154101190 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f513bfff640 1 -- 172.21.15.18:0/3204809420 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144010460 con 0x7f5154101190 2024-02-17T12:49:28.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.448+0000 7f51595d0640 1 -- 172.21.15.18:0/3204809420 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515419f7f0 con 0x7f5154101190 2024-02-17T12:49:28.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.449+0000 7f51595d0640 1 -- 172.21.15.18:0/3204809420 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51541083e0 con 0x7f5154101190 2024-02-17T12:49:28.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.450+0000 7f513bfff640 1 -- 172.21.15.18:0/3204809420 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5144003f60 con 0x7f5154101190 2024-02-17T12:49:28.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.451+0000 7f513bfff640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f513c0a20c0 0x7f513c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:28.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.451+0000 7f51527fc640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f513c0a20c0 0x7f513c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:28.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.451+0000 7f513bfff640 1 -- 172.21.15.18:0/3204809420 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f51440cda20 con 0x7f5154101190 2024-02-17T12:49:28.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.452+0000 7f51527fc640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f513c0a20c0 0x7f513c0a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f51541986f0 tx=0x7f514800c4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:28.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.456+0000 7f513bfff640 1 -- 172.21.15.18:0/3204809420 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5144096a20 con 0x7f5154101190 2024-02-17T12:49:28.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.639+0000 7f51595d0640 1 -- 172.21.15.18:0/3204809420 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f515402ce40 con 0x7f5154101190 2024-02-17T12:49:28.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.639+0000 7f513bfff640 1 -- 172.21.15.18:0/3204809420 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f514409a0e0 con 0x7f5154101190 2024-02-17T12:49:28.632 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-02-17T12:49:28.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.641+0000 7f51595d0640 1 -- 172.21.15.18:0/3204809420 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f513c0a20c0 msgr2=0x7f513c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.641+0000 7f51595d0640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f513c0a20c0 0x7f513c0a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f51541986f0 tx=0x7f514800c4a0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.641+0000 7f51595d0640 1 -- 172.21.15.18:0/3204809420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5154101190 msgr2=0x7f51541971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.641+0000 7f51595d0640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5154101190 0x7f51541971e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f514400d770 tx=0x7f514400dc30 comp rx=0 tx=0).stop 2024-02-17T12:49:28.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.641+0000 7f51595d0640 1 -- 172.21.15.18:0/3204809420 shutdown_connections 2024-02-17T12:49:28.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.641+0000 7f51595d0640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f513c0a20c0 0x7f513c0a4580 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.641+0000 7f51595d0640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5154102d40 0x7f515419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.641+0000 7f51595d0640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5154102380 0x7f5154197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.641+0000 7f51595d0640 1 --2- 172.21.15.18:0/3204809420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5154101190 0x7f51541971e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.641+0000 7f51595d0640 1 -- 172.21.15.18:0/3204809420 >> 172.21.15.18:0/3204809420 conn(0x7f51540fc720 msgr2=0x7f51541055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:28.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.641+0000 7f51595d0640 1 -- 172.21.15.18:0/3204809420 shutdown_connections 2024-02-17T12:49:28.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.641+0000 7f51595d0640 1 -- 172.21.15.18:0/3204809420 wait complete. 2024-02-17T12:49:28.643 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:28 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1898214213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:28.644 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:28 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2080377910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:28.644 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:28 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3956835069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:28.645 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:49:28.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.755+0000 7fe1d242a640 1 -- 172.21.15.18:0/2118073942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc1045b0 msgr2=0x7fe1cc1069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.755+0000 7fe1d242a640 1 --2- 172.21.15.18:0/2118073942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc1045b0 0x7fe1cc1069a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009900 tx=0x7fe1c002bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.755+0000 7fe1d242a640 1 -- 172.21.15.18:0/2118073942 shutdown_connections 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.755+0000 7fe1d242a640 1 --2- 172.21.15.18:0/2118073942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1cc106ee0 0x7fe1cc1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.755+0000 7fe1d242a640 1 --2- 172.21.15.18:0/2118073942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc1045b0 0x7fe1cc1069a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.755+0000 7fe1d242a640 1 --2- 172.21.15.18:0/2118073942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1cc101c80 0x7fe1cc104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.755+0000 7fe1d242a640 1 -- 172.21.15.18:0/2118073942 >> 172.21.15.18:0/2118073942 conn(0x7fe1cc0fb610 msgr2=0x7fe1cc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.755+0000 7fe1d242a640 1 -- 172.21.15.18:0/2118073942 shutdown_connections 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.756+0000 7fe1d242a640 1 -- 172.21.15.18:0/2118073942 wait complete. 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.756+0000 7fe1d242a640 1 Processor -- start 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.756+0000 7fe1d242a640 1 -- start start 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.756+0000 7fe1d242a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc101c80 0x7fe1cc197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.756+0000 7fe1d242a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1cc1045b0 0x7fe1cc197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.756+0000 7fe1d242a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1cc106ee0 0x7fe1cc19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.756+0000 7fe1d242a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc19ed30 con 0x7fe1cc101c80 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.756+0000 7fe1d242a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc19eea0 con 0x7fe1cc1045b0 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.756+0000 7fe1d242a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc19f060 con 0x7fe1cc106ee0 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.756+0000 7fe1cbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc101c80 0x7fe1cc197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.756+0000 7fe1cbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc101c80 0x7fe1cc197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40392/0 (socket says 172.21.15.18:40392) 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.756+0000 7fe1cbfff640 1 -- 172.21.15.18:0/3864052399 learned_addr learned my addr 172.21.15.18:0/3864052399 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.757+0000 7fe1d09c7640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1cc106ee0 0x7fe1cc19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.757+0000 7fe1cbfff640 1 -- 172.21.15.18:0/3864052399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1cc106ee0 msgr2=0x7fe1cc19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.757+0000 7fe1cbfff640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1cc106ee0 0x7fe1cc19e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.757+0000 7fe1cbfff640 1 -- 172.21.15.18:0/3864052399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1cc1045b0 msgr2=0x7fe1cc197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.757+0000 7fe1cbfff640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1cc1045b0 0x7fe1cc197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.757+0000 7fe1cbfff640 1 -- 172.21.15.18:0/3864052399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c0009560 con 0x7fe1cc101c80 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.757+0000 7fe1cbfff640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc101c80 0x7fe1cc197240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00d950 tx=0x7fe1bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.757+0000 7fe1c97fa640 1 -- 172.21.15.18:0/3864052399 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc009c30 con 0x7fe1cc101c80 2024-02-17T12:49:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.757+0000 7fe1c97fa640 1 -- 172.21.15.18:0/3864052399 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1bc009d90 con 0x7fe1cc101c80 2024-02-17T12:49:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.757+0000 7fe1d242a640 1 -- 172.21.15.18:0/3864052399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc19f340 con 0x7fe1cc101c80 2024-02-17T12:49:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.757+0000 7fe1c97fa640 1 -- 172.21.15.18:0/3864052399 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00f7e0 con 0x7fe1cc101c80 2024-02-17T12:49:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.757+0000 7fe1d242a640 1 -- 172.21.15.18:0/3864052399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc19f8a0 con 0x7fe1cc101c80 2024-02-17T12:49:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.758+0000 7fe1d242a640 1 -- 172.21.15.18:0/3864052399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc0fd100 con 0x7fe1cc101c80 2024-02-17T12:49:28.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.759+0000 7fe1c97fa640 1 -- 172.21.15.18:0/3864052399 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe1bc010ad0 con 0x7fe1cc101c80 2024-02-17T12:49:28.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.760+0000 7fe1c97fa640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe1b40a2140 0x7fe1b40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:28.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.760+0000 7fe1cb7fe640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe1b40a2140 0x7fe1b40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:28.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.760+0000 7fe1c97fa640 1 -- 172.21.15.18:0/3864052399 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe1bc0cd920 con 0x7fe1cc101c80 2024-02-17T12:49:28.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.760+0000 7fe1cb7fe640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe1b40a2140 0x7fe1b40a4600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0036040 tx=0x7fe1c0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:28.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.765+0000 7fe1c97fa640 1 -- 172.21.15.18:0/3864052399 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1bc096920 con 0x7fe1cc101c80 2024-02-17T12:49:28.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:28 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1898214213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:28.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:28 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2080377910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:28.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:28 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3956835069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:28.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:28 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1898214213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:28.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:28 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2080377910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:28.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:28 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3956835069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:28.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.988+0000 7fe1d242a640 1 -- 172.21.15.18:0/3864052399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1cc02ce70 con 0x7fe1cc101c80 2024-02-17T12:49:28.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.989+0000 7fe1c97fa640 1 -- 172.21.15.18:0/3864052399 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11392 (secure 0 0 0) 0x7fe1bc099fe0 con 0x7fe1cc101c80 2024-02-17T12:49:28.982 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:28.982 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:47:51.719041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:49:28.982 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-02-17T12:49:28.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.991+0000 7fe1d242a640 1 -- 172.21.15.18:0/3864052399 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe1b40a2140 msgr2=0x7fe1b40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.991+0000 7fe1d242a640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe1b40a2140 0x7fe1b40a4600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0036040 tx=0x7fe1c0000bc0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.991+0000 7fe1d242a640 1 -- 172.21.15.18:0/3864052399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc101c80 msgr2=0x7fe1cc197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:28.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.991+0000 7fe1d242a640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc101c80 0x7fe1cc197240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00d950 tx=0x7fe1bc00de10 comp rx=0 tx=0).stop 2024-02-17T12:49:28.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.991+0000 7fe1d242a640 1 -- 172.21.15.18:0/3864052399 shutdown_connections 2024-02-17T12:49:28.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.991+0000 7fe1d242a640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fe1b40a2140 0x7fe1b40a4600 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.991+0000 7fe1d242a640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1cc106ee0 0x7fe1cc19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.991+0000 7fe1d242a640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1cc1045b0 0x7fe1cc197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.991+0000 7fe1d242a640 1 --2- 172.21.15.18:0/3864052399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc101c80 0x7fe1cc197240 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:28.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.991+0000 7fe1d242a640 1 -- 172.21.15.18:0/3864052399 >> 172.21.15.18:0/3864052399 conn(0x7fe1cc0fb610 msgr2=0x7fe1cc1026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:28.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.991+0000 7fe1d242a640 1 -- 172.21.15.18:0/3864052399 shutdown_connections 2024-02-17T12:49:28.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:28.991+0000 7fe1d242a640 1 -- 172.21.15.18:0/3864052399 wait complete. 2024-02-17T12:49:28.995 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:49:29.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.107+0000 7f33ca7c7640 1 -- 172.21.15.18:0/3218384533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33c40ff490 msgr2=0x7f33c40ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.107+0000 7f33ca7c7640 1 --2- 172.21.15.18:0/3218384533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33c40ff490 0x7f33c40ff8b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009900 tx=0x7f33b402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:29.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.107+0000 7f33ca7c7640 1 -- 172.21.15.18:0/3218384533 shutdown_connections 2024-02-17T12:49:29.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.107+0000 7f33ca7c7640 1 --2- 172.21.15.18:0/3218384533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33c4102ec0 0x7f33c4109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.107+0000 7f33ca7c7640 1 --2- 172.21.15.18:0/3218384533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33c4102500 0x7f33c4102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.107+0000 7f33ca7c7640 1 --2- 172.21.15.18:0/3218384533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33c40ff490 0x7f33c40ff8b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.107+0000 7f33ca7c7640 1 -- 172.21.15.18:0/3218384533 >> 172.21.15.18:0/3218384533 conn(0x7f33c40fada0 msgr2=0x7f33c40fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:29.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.107+0000 7f33ca7c7640 1 -- 172.21.15.18:0/3218384533 shutdown_connections 2024-02-17T12:49:29.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.107+0000 7f33ca7c7640 1 -- 172.21.15.18:0/3218384533 wait complete. 2024-02-17T12:49:29.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.107+0000 7f33ca7c7640 1 Processor -- start 2024-02-17T12:49:29.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.107+0000 7f33ca7c7640 1 -- start start 2024-02-17T12:49:29.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33ca7c7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33c40ff490 0x7f33c4101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:29.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33ca7c7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33c4102500 0x7f33c41004b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:29.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33ca7c7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33c4102ec0 0x7f33c41009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:29.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33ca7c7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4101150 con 0x7f33c40ff490 2024-02-17T12:49:29.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33ca7c7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c41012c0 con 0x7f33c4102500 2024-02-17T12:49:29.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33ca7c7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c41a12d0 con 0x7f33c4102ec0 2024-02-17T12:49:29.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33c3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33c40ff490 0x7f33c4101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:29.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33c3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33c40ff490 0x7f33c4101e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40398/0 (socket says 172.21.15.18:40398) 2024-02-17T12:49:29.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33c3fff640 1 -- 172.21.15.18:0/1060354206 learned_addr learned my addr 172.21.15.18:0/1060354206 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:29.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33c8d64640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33c4102ec0 0x7f33c41009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:29.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33c3fff640 1 -- 172.21.15.18:0/1060354206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33c4102ec0 msgr2=0x7f33c41009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33c37fe640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33c4102500 0x7f33c41004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:29.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33c3fff640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33c4102ec0 0x7f33c41009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33c3fff640 1 -- 172.21.15.18:0/1060354206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33c4102500 msgr2=0x7f33c41004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33c3fff640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33c4102500 0x7f33c41004b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33c3fff640 1 -- 172.21.15.18:0/1060354206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b4009560 con 0x7f33c40ff490 2024-02-17T12:49:29.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.108+0000 7f33c3fff640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33c40ff490 0x7f33c4101e70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f33b4036040 tx=0x7f33b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:29.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.109+0000 7f33c17fa640 1 -- 172.21.15.18:0/1060354206 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4004df0 con 0x7f33c40ff490 2024-02-17T12:49:29.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.109+0000 7f33c17fa640 1 -- 172.21.15.18:0/1060354206 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33b4005430 con 0x7f33c40ff490 2024-02-17T12:49:29.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.109+0000 7f33c17fa640 1 -- 172.21.15.18:0/1060354206 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b403e450 con 0x7f33c40ff490 2024-02-17T12:49:29.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.109+0000 7f33ca7c7640 1 -- 172.21.15.18:0/1060354206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c41a1520 con 0x7f33c40ff490 2024-02-17T12:49:29.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.109+0000 7f33ca7c7640 1 -- 172.21.15.18:0/1060354206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c41a1a30 con 0x7f33c40ff490 2024-02-17T12:49:29.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.110+0000 7f33ca7c7640 1 -- 172.21.15.18:0/1060354206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c4108680 con 0x7f33c40ff490 2024-02-17T12:49:29.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.111+0000 7f33c17fa640 1 -- 172.21.15.18:0/1060354206 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f33b40055a0 con 0x7f33c40ff490 2024-02-17T12:49:29.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.111+0000 7f33c17fa640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f33ac0a20c0 0x7f33ac0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:29.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.111+0000 7f33c37fe640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f33ac0a20c0 0x7f33ac0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:29.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.111+0000 7f33c17fa640 1 -- 172.21.15.18:0/1060354206 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f33b40f0260 con 0x7f33c40ff490 2024-02-17T12:49:29.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.112+0000 7f33c37fe640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f33ac0a20c0 0x7f33ac0a4580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005fd0 tx=0x7f33b8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:29.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.116+0000 7f33c17fa640 1 -- 172.21.15.18:0/1060354206 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f33b40b9260 con 0x7f33c40ff490 2024-02-17T12:49:29.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.300+0000 7f33ca7c7640 1 -- 172.21.15.18:0/1060354206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33c402ce40 con 0x7f33c40ff490 2024-02-17T12:49:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.302+0000 7f33c17fa640 1 -- 172.21.15.18:0/1060354206 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22607 (secure 0 0 0) 0x7f33b40bc920 con 0x7f33c40ff490 2024-02-17T12:49:29.294 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:29.294 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":88,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:48:52.149237+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:49:29.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.303+0000 7f33ca7c7640 1 -- 172.21.15.18:0/1060354206 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f33ac0a20c0 msgr2=0x7f33ac0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.303+0000 7f33ca7c7640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f33ac0a20c0 0x7f33ac0a4580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005fd0 tx=0x7f33b8005850 comp rx=0 tx=0).stop 2024-02-17T12:49:29.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.303+0000 7f33ca7c7640 1 -- 172.21.15.18:0/1060354206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33c40ff490 msgr2=0x7f33c4101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.303+0000 7f33ca7c7640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33c40ff490 0x7f33c4101e70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f33b4036040 tx=0x7f33b4004270 comp rx=0 tx=0).stop 2024-02-17T12:49:29.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.304+0000 7f33ca7c7640 1 -- 172.21.15.18:0/1060354206 shutdown_connections 2024-02-17T12:49:29.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.304+0000 7f33ca7c7640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f33ac0a20c0 0x7f33ac0a4580 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.304+0000 7f33ca7c7640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33c4102ec0 0x7f33c41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.304+0000 7f33ca7c7640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33c4102500 0x7f33c41004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.304+0000 7f33ca7c7640 1 --2- 172.21.15.18:0/1060354206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33c40ff490 0x7f33c4101e70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.304+0000 7f33ca7c7640 1 -- 172.21.15.18:0/1060354206 >> 172.21.15.18:0/1060354206 conn(0x7f33c40fada0 msgr2=0x7f33c4105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:29.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.304+0000 7f33ca7c7640 1 -- 172.21.15.18:0/1060354206 shutdown_connections 2024-02-17T12:49:29.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.304+0000 7f33ca7c7640 1 -- 172.21.15.18:0/1060354206 wait complete. 2024-02-17T12:49:29.307 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T12:43:12.450140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T12:47:43.690543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T12:47:44.653229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T12:49:29.308 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T12:49:29.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.421+0000 7f4bdb688640 1 -- 172.21.15.18:0/2938780505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4102d40 msgr2=0x7f4bd41095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.421+0000 7f4bdb688640 1 --2- 172.21.15.18:0/2938780505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4102d40 0x7f4bd41095d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009ca0 tx=0x7f4bcc02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.421+0000 7f4bdb688640 1 -- 172.21.15.18:0/2938780505 shutdown_connections 2024-02-17T12:49:29.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.421+0000 7f4bdb688640 1 --2- 172.21.15.18:0/2938780505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4102d40 0x7f4bd41095d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.421+0000 7f4bdb688640 1 --2- 172.21.15.18:0/2938780505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd4102380 0x7f4bd4102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.421+0000 7f4bdb688640 1 --2- 172.21.15.18:0/2938780505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bd4101190 0x7f4bd41015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.421+0000 7f4bdb688640 1 -- 172.21.15.18:0/2938780505 >> 172.21.15.18:0/2938780505 conn(0x7f4bd40fc720 msgr2=0x7f4bd40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.421+0000 7f4bdb688640 1 -- 172.21.15.18:0/2938780505 shutdown_connections 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.421+0000 7f4bdb688640 1 -- 172.21.15.18:0/2938780505 wait complete. 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.421+0000 7f4bdb688640 1 Processor -- start 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.421+0000 7f4bdb688640 1 -- start start 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bdb688640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4101190 0x7f4bd4197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bdb688640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd4102380 0x7f4bd41976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bdb688640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bd4102d40 0x7f4bd419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bdb688640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd419ec60 con 0x7f4bd4101190 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bdb688640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd419edd0 con 0x7f4bd4102380 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bdb688640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd419ef90 con 0x7f4bd4102d40 2024-02-17T12:49:29.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bd9424640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4101190 0x7f4bd4197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:29.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bd9424640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4101190 0x7f4bd4197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40420/0 (socket says 172.21.15.18:40420) 2024-02-17T12:49:29.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bd9424640 1 -- 172.21.15.18:0/2735343543 learned_addr learned my addr 172.21.15.18:0/2735343543 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:29.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bd8c23640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd4102380 0x7f4bd41976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:29.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bd9c25640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bd4102d40 0x7f4bd419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:29.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bd9424640 1 -- 172.21.15.18:0/2735343543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bd4102d40 msgr2=0x7f4bd419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bd9424640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bd4102d40 0x7f4bd419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bd9424640 1 -- 172.21.15.18:0/2735343543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd4102380 msgr2=0x7f4bd41976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bd9424640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd4102380 0x7f4bd41976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bd9424640 1 -- 172.21.15.18:0/2735343543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc009900 con 0x7f4bd4101190 2024-02-17T12:49:29.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bd9424640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4101190 0x7f4bd4197170 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000d950 tx=0x7f4bd000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:29.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bca7fc640 1 -- 172.21.15.18:0/2735343543 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0009c30 con 0x7f4bd4101190 2024-02-17T12:49:29.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bca7fc640 1 -- 172.21.15.18:0/2735343543 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bd000be10 con 0x7f4bd4101190 2024-02-17T12:49:29.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bdb688640 1 -- 172.21.15.18:0/2735343543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd419f270 con 0x7f4bd4101190 2024-02-17T12:49:29.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bca7fc640 1 -- 172.21.15.18:0/2735343543 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000f6b0 con 0x7f4bd4101190 2024-02-17T12:49:29.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.422+0000 7f4bdb688640 1 -- 172.21.15.18:0/2735343543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd419f730 con 0x7f4bd4101190 2024-02-17T12:49:29.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.423+0000 7f4bdb688640 1 -- 172.21.15.18:0/2735343543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd4108450 con 0x7f4bd4101190 2024-02-17T12:49:29.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.424+0000 7f4bca7fc640 1 -- 172.21.15.18:0/2735343543 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4bd0010460 con 0x7f4bd4101190 2024-02-17T12:49:29.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.425+0000 7f4bca7fc640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4bc00a20c0 0x7f4bc00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:29.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.425+0000 7f4bd8c23640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4bc00a20c0 0x7f4bc00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:29.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.425+0000 7f4bca7fc640 1 -- 172.21.15.18:0/2735343543 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4bd00cdad0 con 0x7f4bd4101190 2024-02-17T12:49:29.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.425+0000 7f4bd8c23640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4bc00a20c0 0x7f4bc00a4580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4198680 tx=0x7f4bc400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:29.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.429+0000 7f4bca7fc640 1 -- 172.21.15.18:0/2735343543 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4bd0096ad0 con 0x7f4bd4101190 2024-02-17T12:49:29.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.610+0000 7f4bdb688640 1 -- 172.21.15.18:0/2735343543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bd402ce40 con 0x7f4bd4101190 2024-02-17T12:49:29.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.610+0000 7f4bca7fc640 1 -- 172.21.15.18:0/2735343543 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4bd009a190 con 0x7f4bd4101190 2024-02-17T12:49:29.603 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-02-17T12:49:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.612+0000 7f4bdb688640 1 -- 172.21.15.18:0/2735343543 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4bc00a20c0 msgr2=0x7f4bc00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.612+0000 7f4bdb688640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4bc00a20c0 0x7f4bc00a4580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4198680 tx=0x7f4bc400a2d0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.612+0000 7f4bdb688640 1 -- 172.21.15.18:0/2735343543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4101190 msgr2=0x7f4bd4197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.612+0000 7f4bdb688640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4101190 0x7f4bd4197170 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000d950 tx=0x7f4bd000dd50 comp rx=0 tx=0).stop 2024-02-17T12:49:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.612+0000 7f4bdb688640 1 -- 172.21.15.18:0/2735343543 shutdown_connections 2024-02-17T12:49:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.612+0000 7f4bdb688640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4bc00a20c0 0x7f4bc00a4580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.612+0000 7f4bdb688640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bd4102d40 0x7f4bd419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.612+0000 7f4bdb688640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd4102380 0x7f4bd41976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.612+0000 7f4bdb688640 1 --2- 172.21.15.18:0/2735343543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bd4101190 0x7f4bd4197170 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.612+0000 7f4bdb688640 1 -- 172.21.15.18:0/2735343543 >> 172.21.15.18:0/2735343543 conn(0x7f4bd40fc720 msgr2=0x7f4bd4105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.612+0000 7f4bdb688640 1 -- 172.21.15.18:0/2735343543 shutdown_connections 2024-02-17T12:49:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.612+0000 7f4bdb688640 1 -- 172.21.15.18:0/2735343543 wait complete. 2024-02-17T12:49:29.614 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:29 smithi018 ceph-mon[32625]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-02-17T12:49:29.614 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:29 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3204809420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:29.614 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:29 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3864052399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:29.615 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:29 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1060354206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:29.616 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T12:49:29.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.728+0000 7fab308d4640 1 -- 172.21.15.18:0/2270479518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c101190 msgr2=0x7fab2c1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.728+0000 7fab308d4640 1 --2- 172.21.15.18:0/2270479518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c101190 0x7fab2c1015b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fab24009900 tx=0x7fab2402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:29.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 -- 172.21.15.18:0/2270479518 shutdown_connections 2024-02-17T12:49:29.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 --2- 172.21.15.18:0/2270479518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c102d40 0x7fab2c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 --2- 172.21.15.18:0/2270479518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c102380 0x7fab2c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 --2- 172.21.15.18:0/2270479518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c101190 0x7fab2c1015b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 -- 172.21.15.18:0/2270479518 >> 172.21.15.18:0/2270479518 conn(0x7fab2c0fc720 msgr2=0x7fab2c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 -- 172.21.15.18:0/2270479518 shutdown_connections 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 -- 172.21.15.18:0/2270479518 wait complete. 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 Processor -- start 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 -- start start 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c101190 0x7fab2c1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c102380 0x7fab2c1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c102d40 0x7fab2c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c19ec90 con 0x7fab2c101190 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c19ee00 con 0x7fab2c102d40 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.729+0000 7fab308d4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c19efc0 con 0x7fab2c102380 2024-02-17T12:49:29.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab2a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c101190 0x7fab2c1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:29.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab2a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c101190 0x7fab2c1971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36268/0 (socket says 172.21.15.18:36268) 2024-02-17T12:49:29.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab2a59c640 1 -- 172.21.15.18:0/433752649 learned_addr learned my addr 172.21.15.18:0/433752649 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:29.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab29d9b640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c102380 0x7fab2c1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:29.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab2ad9d640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c102d40 0x7fab2c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:29.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab2a59c640 1 -- 172.21.15.18:0/433752649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c102380 msgr2=0x7fab2c1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab2a59c640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c102380 0x7fab2c1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab2a59c640 1 -- 172.21.15.18:0/433752649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c102d40 msgr2=0x7fab2c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab2a59c640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c102d40 0x7fab2c19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab2a59c640 1 -- 172.21.15.18:0/433752649 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab24009560 con 0x7fab2c101190 2024-02-17T12:49:29.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab2a59c640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c101190 0x7fab2c1971a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fab24036040 tx=0x7fab240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:29.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab237fe640 1 -- 172.21.15.18:0/433752649 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab240044e0 con 0x7fab2c101190 2024-02-17T12:49:29.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab237fe640 1 -- 172.21.15.18:0/433752649 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab240054e0 con 0x7fab2c101190 2024-02-17T12:49:29.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab308d4640 1 -- 172.21.15.18:0/433752649 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c19f240 con 0x7fab2c101190 2024-02-17T12:49:29.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab237fe640 1 -- 172.21.15.18:0/433752649 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2403e450 con 0x7fab2c101190 2024-02-17T12:49:29.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.730+0000 7fab308d4640 1 -- 172.21.15.18:0/433752649 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c19f750 con 0x7fab2c101190 2024-02-17T12:49:29.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.731+0000 7fab308d4640 1 -- 172.21.15.18:0/433752649 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c1084b0 con 0x7fab2c101190 2024-02-17T12:49:29.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.732+0000 7fab237fe640 1 -- 172.21.15.18:0/433752649 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fab24003680 con 0x7fab2c101190 2024-02-17T12:49:29.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.733+0000 7fab237fe640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fab180a20c0 0x7fab180a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:29.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.733+0000 7fab29d9b640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fab180a20c0 0x7fab180a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:29.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.733+0000 7fab237fe640 1 -- 172.21.15.18:0/433752649 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fab240f0660 con 0x7fab2c101190 2024-02-17T12:49:29.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.733+0000 7fab29d9b640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fab180a20c0 0x7fab180a4580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fab2c1986b0 tx=0x7fab1400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:29.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.736+0000 7fab237fe640 1 -- 172.21.15.18:0/433752649 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab240b9600 con 0x7fab2c101190 2024-02-17T12:49:29.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:29 smithi119 ceph-mon[36188]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-02-17T12:49:29.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:29 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3204809420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:29.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:29 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3864052399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:29.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:29 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1060354206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:29.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.917+0000 7fab308d4640 1 -- 172.21.15.18:0/433752649 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab2c10cb60 con 0x7fab2c101190 2024-02-17T12:49:29.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.918+0000 7fab237fe640 1 -- 172.21.15.18:0/433752649 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab240bccc0 con 0x7fab2c101190 2024-02-17T12:49:29.910 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-02-17T12:49:29.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.920+0000 7fab308d4640 1 -- 172.21.15.18:0/433752649 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fab180a20c0 msgr2=0x7fab180a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.920+0000 7fab308d4640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fab180a20c0 0x7fab180a4580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fab2c1986b0 tx=0x7fab1400a2d0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.920+0000 7fab308d4640 1 -- 172.21.15.18:0/433752649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c101190 msgr2=0x7fab2c1971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:29.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.920+0000 7fab308d4640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c101190 0x7fab2c1971a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fab24036040 tx=0x7fab240042f0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.920+0000 7fab308d4640 1 -- 172.21.15.18:0/433752649 shutdown_connections 2024-02-17T12:49:29.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.920+0000 7fab308d4640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fab180a20c0 0x7fab180a4580 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.920+0000 7fab308d4640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c102d40 0x7fab2c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.920+0000 7fab308d4640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab2c102380 0x7fab2c1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.920+0000 7fab308d4640 1 --2- 172.21.15.18:0/433752649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c101190 0x7fab2c1971a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:29.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.920+0000 7fab308d4640 1 -- 172.21.15.18:0/433752649 >> 172.21.15.18:0/433752649 conn(0x7fab2c0fc720 msgr2=0x7fab2c105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:29.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.920+0000 7fab308d4640 1 -- 172.21.15.18:0/433752649 shutdown_connections 2024-02-17T12:49:29.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:29.920+0000 7fab308d4640 1 -- 172.21.15.18:0/433752649 wait complete. 2024-02-17T12:49:29.924 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T12:49:29.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:29 smithi146 ceph-mon[36454]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-02-17T12:49:29.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:29 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3204809420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:29.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:29 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3864052399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:29.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:29 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1060354206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:30.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.037+0000 7ffb174ae640 1 -- 172.21.15.18:0/4075738368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb10102440 msgr2=0x7ffb101028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.037+0000 7ffb174ae640 1 --2- 172.21.15.18:0/4075738368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb10102440 0x7ffb101028c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009900 tx=0x7ffb0002bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:30.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.037+0000 7ffb174ae640 1 -- 172.21.15.18:0/4075738368 shutdown_connections 2024-02-17T12:49:30.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.037+0000 7ffb174ae640 1 --2- 172.21.15.18:0/4075738368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb10102e00 0x7ffb10109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.037+0000 7ffb174ae640 1 --2- 172.21.15.18:0/4075738368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb10102440 0x7ffb101028c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.037+0000 7ffb174ae640 1 --2- 172.21.15.18:0/4075738368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb100ff470 0x7ffb100ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.037+0000 7ffb174ae640 1 -- 172.21.15.18:0/4075738368 >> 172.21.15.18:0/4075738368 conn(0x7ffb100fad80 msgr2=0x7ffb100fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:30.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb174ae640 1 -- 172.21.15.18:0/4075738368 shutdown_connections 2024-02-17T12:49:30.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb174ae640 1 -- 172.21.15.18:0/4075738368 wait complete. 2024-02-17T12:49:30.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb174ae640 1 Processor -- start 2024-02-17T12:49:30.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb174ae640 1 -- start start 2024-02-17T12:49:30.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb174ae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb100ff470 0x7ffb10197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb174ae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb10102440 0x7ffb101977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb174ae640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb10102e00 0x7ffb1019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb174ae640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1019ed70 con 0x7ffb10102e00 2024-02-17T12:49:30.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb174ae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1019eee0 con 0x7ffb100ff470 2024-02-17T12:49:30.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb174ae640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1019f0a0 con 0x7ffb10102440 2024-02-17T12:49:30.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb15a4b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb10102e00 0x7ffb1019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb15a4b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb10102e00 0x7ffb1019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36298/0 (socket says 172.21.15.18:36298) 2024-02-17T12:49:30.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb15a4b640 1 -- 172.21.15.18:0/1530893022 learned_addr learned my addr 172.21.15.18:0/1530893022 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:30.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb14a49640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb10102440 0x7ffb101977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.038+0000 7ffb1524a640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb100ff470 0x7ffb10197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.039+0000 7ffb15a4b640 1 -- 172.21.15.18:0/1530893022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb10102440 msgr2=0x7ffb101977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.039+0000 7ffb15a4b640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb10102440 0x7ffb101977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.039+0000 7ffb15a4b640 1 -- 172.21.15.18:0/1530893022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb100ff470 msgr2=0x7ffb10197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.039+0000 7ffb15a4b640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb100ff470 0x7ffb10197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.039+0000 7ffb15a4b640 1 -- 172.21.15.18:0/1530893022 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb00009560 con 0x7ffb10102e00 2024-02-17T12:49:30.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.039+0000 7ffb15a4b640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb10102e00 0x7ffb1019e830 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800c970 tx=0x7ffb0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:30.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.039+0000 7ffb067fc640 1 -- 172.21.15.18:0/1530893022 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08007bb0 con 0x7ffb10102e00 2024-02-17T12:49:30.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.039+0000 7ffb067fc640 1 -- 172.21.15.18:0/1530893022 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb08007d10 con 0x7ffb10102e00 2024-02-17T12:49:30.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.039+0000 7ffb174ae640 1 -- 172.21.15.18:0/1530893022 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1019f380 con 0x7ffb10102e00 2024-02-17T12:49:30.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.039+0000 7ffb067fc640 1 -- 172.21.15.18:0/1530893022 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0800f450 con 0x7ffb10102e00 2024-02-17T12:49:30.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.039+0000 7ffb174ae640 1 -- 172.21.15.18:0/1530893022 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1019f810 con 0x7ffb10102e00 2024-02-17T12:49:30.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.040+0000 7ffb174ae640 1 -- 172.21.15.18:0/1530893022 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb10108570 con 0x7ffb10102e00 2024-02-17T12:49:30.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.041+0000 7ffb067fc640 1 -- 172.21.15.18:0/1530893022 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ffb08004f00 con 0x7ffb10102e00 2024-02-17T12:49:30.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.042+0000 7ffb067fc640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ffafc0a20c0 0x7ffafc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.042+0000 7ffb1524a640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ffafc0a20c0 0x7ffafc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.042+0000 7ffb067fc640 1 -- 172.21.15.18:0/1530893022 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ffb080cd860 con 0x7ffb10102e00 2024-02-17T12:49:30.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.042+0000 7ffb1524a640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ffafc0a20c0 0x7ffafc0a4580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffb10069590 tx=0x7ffb0c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:30.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.046+0000 7ffb067fc640 1 -- 172.21.15.18:0/1530893022 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb08096910 con 0x7ffb10102e00 2024-02-17T12:49:30.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.231+0000 7ffb174ae640 1 -- 172.21.15.18:0/1530893022 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb1002ce40 con 0x7ffb10102e00 2024-02-17T12:49:30.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.231+0000 7ffb067fc640 1 -- 172.21.15.18:0/1530893022 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb08099fd0 con 0x7ffb10102e00 2024-02-17T12:49:30.224 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-02-17T12:49:30.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.233+0000 7ffb174ae640 1 -- 172.21.15.18:0/1530893022 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ffafc0a20c0 msgr2=0x7ffafc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.233+0000 7ffb174ae640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ffafc0a20c0 0x7ffafc0a4580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffb10069590 tx=0x7ffb0c009680 comp rx=0 tx=0).stop 2024-02-17T12:49:30.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.233+0000 7ffb174ae640 1 -- 172.21.15.18:0/1530893022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb10102e00 msgr2=0x7ffb1019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.233+0000 7ffb174ae640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb10102e00 0x7ffb1019e830 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800c970 tx=0x7ffb0800ce30 comp rx=0 tx=0).stop 2024-02-17T12:49:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.233+0000 7ffb174ae640 1 -- 172.21.15.18:0/1530893022 shutdown_connections 2024-02-17T12:49:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.233+0000 7ffb174ae640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ffafc0a20c0 0x7ffafc0a4580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.233+0000 7ffb174ae640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb10102e00 0x7ffb1019e830 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.233+0000 7ffb174ae640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb10102440 0x7ffb101977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.233+0000 7ffb174ae640 1 --2- 172.21.15.18:0/1530893022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb100ff470 0x7ffb10197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.233+0000 7ffb174ae640 1 -- 172.21.15.18:0/1530893022 >> 172.21.15.18:0/1530893022 conn(0x7ffb100fad80 msgr2=0x7ffb10105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.233+0000 7ffb174ae640 1 -- 172.21.15.18:0/1530893022 shutdown_connections 2024-02-17T12:49:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.233+0000 7ffb174ae640 1 -- 172.21.15.18:0/1530893022 wait complete. 2024-02-17T12:49:30.237 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:49:30.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.347+0000 7fdc90dd0640 1 -- 172.21.15.18:0/2652030912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc8c101310 msgr2=0x7fdc8c101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.347+0000 7fdc90dd0640 1 --2- 172.21.15.18:0/2652030912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc8c101310 0x7fdc8c101790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009900 tx=0x7fdc7402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:30.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.347+0000 7fdc90dd0640 1 -- 172.21.15.18:0/2652030912 shutdown_connections 2024-02-17T12:49:30.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.347+0000 7fdc90dd0640 1 --2- 172.21.15.18:0/2652030912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8c101cd0 0x7fdc8c108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.347+0000 7fdc90dd0640 1 --2- 172.21.15.18:0/2652030912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc8c101310 0x7fdc8c101790 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.347+0000 7fdc90dd0640 1 --2- 172.21.15.18:0/2652030912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc8c100120 0x7fdc8c100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.347+0000 7fdc90dd0640 1 -- 172.21.15.18:0/2652030912 >> 172.21.15.18:0/2652030912 conn(0x7fdc8c0fb6b0 msgr2=0x7fdc8c0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:30.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.347+0000 7fdc90dd0640 1 -- 172.21.15.18:0/2652030912 shutdown_connections 2024-02-17T12:49:30.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.347+0000 7fdc90dd0640 1 -- 172.21.15.18:0/2652030912 wait complete. 2024-02-17T12:49:30.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc90dd0640 1 Processor -- start 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc90dd0640 1 -- start start 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc90dd0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc8c100120 0x7fdc8c197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc90dd0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc8c101310 0x7fdc8c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc90dd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8c101cd0 0x7fdc8c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc90dd0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c19ed10 con 0x7fdc8c100120 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc90dd0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c19ee80 con 0x7fdc8c101cd0 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc90dd0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c19f040 con 0x7fdc8c101310 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc8a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc8c100120 0x7fdc8c197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc8a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc8c100120 0x7fdc8c197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36318/0 (socket says 172.21.15.18:36318) 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc8a59c640 1 -- 172.21.15.18:0/476176946 learned_addr learned my addr 172.21.15.18:0/476176946 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc81d9b640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc8c101310 0x7fdc8c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc8a59c640 1 -- 172.21.15.18:0/476176946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc8c101310 msgr2=0x7fdc8c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc8a59c640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc8c101310 0x7fdc8c197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc8a59c640 1 -- 172.21.15.18:0/476176946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8c101cd0 msgr2=0x7fdc8c19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc8ad9d640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8c101cd0 0x7fdc8c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc8a59c640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8c101cd0 0x7fdc8c19e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc8a59c640 1 -- 172.21.15.18:0/476176946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc74009560 con 0x7fdc8c100120 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.348+0000 7fdc81d9b640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc8c101310 0x7fdc8c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.349+0000 7fdc8a59c640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc8c100120 0x7fdc8c197220 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00c970 tx=0x7fdc7c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.349+0000 7fdc83fff640 1 -- 172.21.15.18:0/476176946 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c004220 con 0x7fdc8c100120 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.349+0000 7fdc83fff640 1 -- 172.21.15.18:0/476176946 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc7c007e90 con 0x7fdc8c100120 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.349+0000 7fdc90dd0640 1 -- 172.21.15.18:0/476176946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8c19f320 con 0x7fdc8c100120 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.349+0000 7fdc83fff640 1 -- 172.21.15.18:0/476176946 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c004a00 con 0x7fdc8c100120 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.349+0000 7fdc90dd0640 1 -- 172.21.15.18:0/476176946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8c19f7e0 con 0x7fdc8c100120 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.350+0000 7fdc90dd0640 1 -- 172.21.15.18:0/476176946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8c107440 con 0x7fdc8c100120 2024-02-17T12:49:30.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.351+0000 7fdc83fff640 1 -- 172.21.15.18:0/476176946 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fdc7c0079b0 con 0x7fdc8c100120 2024-02-17T12:49:30.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.351+0000 7fdc83fff640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdc780a20c0 0x7fdc780a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.351+0000 7fdc81d9b640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdc780a20c0 0x7fdc780a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.351+0000 7fdc83fff640 1 -- 172.21.15.18:0/476176946 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdc7c0cdde0 con 0x7fdc8c100120 2024-02-17T12:49:30.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.352+0000 7fdc81d9b640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdc780a20c0 0x7fdc780a4580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdc74036040 tx=0x7fdc74000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:30.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.356+0000 7fdc83fff640 1 -- 172.21.15.18:0/476176946 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdc7c096de0 con 0x7fdc8c100120 2024-02-17T12:49:30.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.575+0000 7fdc90dd0640 1 -- 172.21.15.18:0/476176946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc8c10cbc0 con 0x7fdc8c100120 2024-02-17T12:49:30.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.576+0000 7fdc83fff640 1 -- 172.21.15.18:0/476176946 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11392 (secure 0 0 0) 0x7fdc7c01d020 con 0x7fdc8c100120 2024-02-17T12:49:30.568 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:30.568 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:47:51.719041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:49:30.569 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-02-17T12:49:30.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.578+0000 7fdc90dd0640 1 -- 172.21.15.18:0/476176946 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdc780a20c0 msgr2=0x7fdc780a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.578+0000 7fdc90dd0640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdc780a20c0 0x7fdc780a4580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdc74036040 tx=0x7fdc74000bc0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.578+0000 7fdc90dd0640 1 -- 172.21.15.18:0/476176946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc8c100120 msgr2=0x7fdc8c197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.578+0000 7fdc90dd0640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc8c100120 0x7fdc8c197220 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00c970 tx=0x7fdc7c00ce30 comp rx=0 tx=0).stop 2024-02-17T12:49:30.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.578+0000 7fdc90dd0640 1 -- 172.21.15.18:0/476176946 shutdown_connections 2024-02-17T12:49:30.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.578+0000 7fdc90dd0640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fdc780a20c0 0x7fdc780a4580 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.578+0000 7fdc90dd0640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8c101cd0 0x7fdc8c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.578+0000 7fdc90dd0640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc8c101310 0x7fdc8c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.578+0000 7fdc90dd0640 1 --2- 172.21.15.18:0/476176946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc8c100120 0x7fdc8c197220 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.578+0000 7fdc90dd0640 1 -- 172.21.15.18:0/476176946 >> 172.21.15.18:0/476176946 conn(0x7fdc8c0fb6b0 msgr2=0x7fdc8c104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:30.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.578+0000 7fdc90dd0640 1 -- 172.21.15.18:0/476176946 shutdown_connections 2024-02-17T12:49:30.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.578+0000 7fdc90dd0640 1 -- 172.21.15.18:0/476176946 wait complete. 2024-02-17T12:49:30.581 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:49:30.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.692+0000 7f691f039640 1 -- 172.21.15.18:0/3016477620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6918101290 msgr2=0x7f6918101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.692+0000 7f691f039640 1 --2- 172.21.15.18:0/3016477620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6918101290 0x7f6918101710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6908009ca0 tx=0x7f690802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.692+0000 7f691f039640 1 -- 172.21.15.18:0/3016477620 shutdown_connections 2024-02-17T12:49:30.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.692+0000 7f691f039640 1 --2- 172.21.15.18:0/3016477620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6918101c50 0x7f69181084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.692+0000 7f691f039640 1 --2- 172.21.15.18:0/3016477620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6918101290 0x7f6918101710 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.692+0000 7f691f039640 1 --2- 172.21.15.18:0/3016477620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69181000a0 0x7f69181004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.692+0000 7f691f039640 1 -- 172.21.15.18:0/3016477620 >> 172.21.15.18:0/3016477620 conn(0x7f69180fb610 msgr2=0x7f69180fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:30.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691f039640 1 -- 172.21.15.18:0/3016477620 shutdown_connections 2024-02-17T12:49:30.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691f039640 1 -- 172.21.15.18:0/3016477620 wait complete. 2024-02-17T12:49:30.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691f039640 1 Processor -- start 2024-02-17T12:49:30.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691f039640 1 -- start start 2024-02-17T12:49:30.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691f039640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69181000a0 0x7f6918196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691f039640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6918101290 0x7f69181966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691f039640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6918101c50 0x7f691819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691f039640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691819dc60 con 0x7f6918101c50 2024-02-17T12:49:30.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691f039640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691819ddd0 con 0x7f69181000a0 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691f039640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691819df90 con 0x7f6918101290 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691d5d6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6918101c50 0x7f691819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691d5d6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6918101c50 0x7f691819d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36332/0 (socket says 172.21.15.18:36332) 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.693+0000 7f691d5d6640 1 -- 172.21.15.18:0/2954363049 learned_addr learned my addr 172.21.15.18:0/2954363049 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f6917fff640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6918101290 0x7f69181966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f691cdd5640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69181000a0 0x7f6918196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f691d5d6640 1 -- 172.21.15.18:0/2954363049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6918101290 msgr2=0x7f69181966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f691d5d6640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6918101290 0x7f69181966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f691d5d6640 1 -- 172.21.15.18:0/2954363049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69181000a0 msgr2=0x7f6918196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f691d5d6640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69181000a0 0x7f6918196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f691d5d6640 1 -- 172.21.15.18:0/2954363049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6908009900 con 0x7f6918101c50 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f691cdd5640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69181000a0 0x7f6918196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f691d5d6640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6918101c50 0x7f691819d720 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f690000d950 tx=0x7f690000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:30.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f6915ffb640 1 -- 172.21.15.18:0/2954363049 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900009a30 con 0x7f6918101c50 2024-02-17T12:49:30.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f6915ffb640 1 -- 172.21.15.18:0/2954363049 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6900009b90 con 0x7f6918101c50 2024-02-17T12:49:30.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f691f039640 1 -- 172.21.15.18:0/2954363049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6918068340 con 0x7f6918101c50 2024-02-17T12:49:30.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f6915ffb640 1 -- 172.21.15.18:0/2954363049 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900003d60 con 0x7f6918101c50 2024-02-17T12:49:30.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.694+0000 7f691f039640 1 -- 172.21.15.18:0/2954363049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6918068820 con 0x7f6918101c50 2024-02-17T12:49:30.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.695+0000 7f691f039640 1 -- 172.21.15.18:0/2954363049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6918067f70 con 0x7f6918101c50 2024-02-17T12:49:30.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.696+0000 7f6915ffb640 1 -- 172.21.15.18:0/2954363049 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6900009d00 con 0x7f6918101c50 2024-02-17T12:49:30.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.696+0000 7f6915ffb640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f69040a2140 0x7f69040a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.697+0000 7f691cdd5640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f69040a2140 0x7f69040a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.697+0000 7f6915ffb640 1 -- 172.21.15.18:0/2954363049 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f69000cd780 con 0x7f6918101c50 2024-02-17T12:49:30.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.697+0000 7f691cdd5640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f69040a2140 0x7f69040a4600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f69181010f0 tx=0x7f6910005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:30.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.701+0000 7f6915ffb640 1 -- 172.21.15.18:0/2954363049 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6900096720 con 0x7f6918101c50 2024-02-17T12:49:30.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:30 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2735343543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:30.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:30 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/433752649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:30.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:30 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1530893022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:30.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:30 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2735343543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:30.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:30 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/433752649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:30.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:30 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1530893022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:30.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.883+0000 7f691f039640 1 -- 172.21.15.18:0/2954363049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6918107000 con 0x7f6918101c50 2024-02-17T12:49:30.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.884+0000 7f6915ffb640 1 -- 172.21.15.18:0/2954363049 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22607 (secure 0 0 0) 0x7f6900099de0 con 0x7f6918101c50 2024-02-17T12:49:30.877 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:30.877 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":88,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:48:52.149237+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:49:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.886+0000 7f691f039640 1 -- 172.21.15.18:0/2954363049 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f69040a2140 msgr2=0x7f69040a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.886+0000 7f691f039640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f69040a2140 0x7f69040a4600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f69181010f0 tx=0x7f6910005e70 comp rx=0 tx=0).stop 2024-02-17T12:49:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.886+0000 7f691f039640 1 -- 172.21.15.18:0/2954363049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6918101c50 msgr2=0x7f691819d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.886+0000 7f691f039640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6918101c50 0x7f691819d720 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f690000d950 tx=0x7f690000de10 comp rx=0 tx=0).stop 2024-02-17T12:49:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.887+0000 7f691f039640 1 -- 172.21.15.18:0/2954363049 shutdown_connections 2024-02-17T12:49:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.887+0000 7f691f039640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f69040a2140 0x7f69040a4600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.887+0000 7f691f039640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6918101c50 0x7f691819d720 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.887+0000 7f691f039640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6918101290 0x7f69181966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.887+0000 7f691f039640 1 --2- 172.21.15.18:0/2954363049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69181000a0 0x7f6918196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.887+0000 7f691f039640 1 -- 172.21.15.18:0/2954363049 >> 172.21.15.18:0/2954363049 conn(0x7f69180fb610 msgr2=0x7f6918104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.887+0000 7f691f039640 1 -- 172.21.15.18:0/2954363049 shutdown_connections 2024-02-17T12:49:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:30.887+0000 7f691f039640 1 -- 172.21.15.18:0/2954363049 wait complete. 2024-02-17T12:49:30.891 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-17T12:47:45.407920+0000', 'modified': '2024-02-17T12:47:51.719041+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24512, 'mds_1': 24476, 'mds_2': 24455, 'mds_3': 24448, 'mds_4': 24422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.18:6839/1294470347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6838', 'nonce': 1294470347}, {'type': 'v1', 'addr': '172.21.15.18:6839', 'nonce': 1294470347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14655': {'gid': 14655, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.18:6841/2569825427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6840', 'nonce': 2569825427}, {'type': 'v1', 'addr': '172.21.15.18:6841', 'nonce': 2569825427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6833/4241790141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 4241790141}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 4241790141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.119:6833/21392902', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6832', 'nonce': 21392902}, {'type': 'v1', 'addr': '172.21.15.119:6833', 'nonce': 21392902}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.146:6835/405923223', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 405923223}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 405923223}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.119:6835/2304139172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 2304139172}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 2304139172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6837/1732085011', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 1732085011}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 1732085011}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6839/3061839650', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6838', 'nonce': 3061839650}, {'type': 'v1', 'addr': '172.21.15.146:6839', 'nonce': 3061839650}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.119:6837/248613392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 248613392}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 248613392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6839/2236251663', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 2236251663}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 2236251663}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-17T12:49:30.891 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-02-17T12:49:30.891 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-02-17T12:49:30.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:30 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2735343543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:30.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:30 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/433752649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:30.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:30 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1530893022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:30.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.002+0000 7f3002113640 1 -- 172.21.15.18:0/3810204900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ffc101c50 msgr2=0x7f2ffc1084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.002+0000 7f3002113640 1 --2- 172.21.15.18:0/3810204900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ffc101c50 0x7f2ffc1084e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009ca0 tx=0x7f2ff402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.002+0000 7f3002113640 1 -- 172.21.15.18:0/3810204900 shutdown_connections 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.002+0000 7f3002113640 1 --2- 172.21.15.18:0/3810204900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ffc101c50 0x7f2ffc1084e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.002+0000 7f3002113640 1 --2- 172.21.15.18:0/3810204900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc101290 0x7f2ffc101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.002+0000 7f3002113640 1 --2- 172.21.15.18:0/3810204900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ffc1000a0 0x7f2ffc1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.002+0000 7f3002113640 1 -- 172.21.15.18:0/3810204900 >> 172.21.15.18:0/3810204900 conn(0x7f2ffc0fb630 msgr2=0x7f2ffc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.002+0000 7f3002113640 1 -- 172.21.15.18:0/3810204900 shutdown_connections 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.002+0000 7f3002113640 1 -- 172.21.15.18:0/3810204900 wait complete. 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.002+0000 7f3002113640 1 Processor -- start 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f3002113640 1 -- start start 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f3002113640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc1000a0 0x7f2ffc1971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f3002113640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ffc101290 0x7f2ffc197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f3002113640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ffc101c50 0x7f2ffc19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:30.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f3002113640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc19ecb0 con 0x7f2ffc101c50 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f3002113640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc19ee20 con 0x7f2ffc1000a0 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f3002113640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc19efe0 con 0x7f2ffc101290 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f2ffbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ffc101c50 0x7f2ffc19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f2ffbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ffc101c50 0x7f2ffc19e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36348/0 (socket says 172.21.15.18:36348) 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f2ffbfff640 1 -- 172.21.15.18:0/2300326660 learned_addr learned my addr 172.21.15.18:0/2300326660 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f2ffb7fe640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc1000a0 0x7f2ffc1971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ffc101290 0x7f2ffc197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f2ffbfff640 1 -- 172.21.15.18:0/2300326660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ffc101290 msgr2=0x7f2ffc197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f2ffbfff640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ffc101290 0x7f2ffc197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f2ffbfff640 1 -- 172.21.15.18:0/2300326660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc1000a0 msgr2=0x7f2ffc1971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f2ffbfff640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc1000a0 0x7f2ffc1971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.003+0000 7f2ffbfff640 1 -- 172.21.15.18:0/2300326660 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff4009900 con 0x7f2ffc101c50 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.004+0000 7f2ffbfff640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ffc101c50 0x7f2ffc19e770 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4037040 tx=0x7f2ff4034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:30.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.004+0000 7f2ff8ff9640 1 -- 172.21.15.18:0/2300326660 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4032eb0 con 0x7f2ffc101c50 2024-02-17T12:49:30.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.004+0000 7f2ff8ff9640 1 -- 172.21.15.18:0/2300326660 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ff4003e50 con 0x7f2ffc101c50 2024-02-17T12:49:30.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.004+0000 7f3002113640 1 -- 172.21.15.18:0/2300326660 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ffc19f260 con 0x7f2ffc101c50 2024-02-17T12:49:30.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.004+0000 7f2ff8ff9640 1 -- 172.21.15.18:0/2300326660 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4040450 con 0x7f2ffc101c50 2024-02-17T12:49:30.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.004+0000 7f3002113640 1 -- 172.21.15.18:0/2300326660 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc19f710 con 0x7f2ffc101c50 2024-02-17T12:49:31.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.006+0000 7f2ff8ff9640 1 -- 172.21.15.18:0/2300326660 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2ff400a040 con 0x7f2ffc101c50 2024-02-17T12:49:31.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f3004dca740 1 Processor -- start 2024-02-17T12:49:31.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f3004dca740 1 -- start start 2024-02-17T12:49:31.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f3004dca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c0cf80 0x55d146c0d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f3004dca740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 0x55d146c12470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f3004dca740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c12ae0 0x55d146c14f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f3004dca740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d146bc8cd0 con 0x55d146c0d970 2024-02-17T12:49:31.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f3004dca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d146b7bf20 con 0x55d146c0cf80 2024-02-17T12:49:31.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f3004dca740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d146b79450 con 0x55d146c12ae0 2024-02-17T12:49:31.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 0x55d146c12470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 0x55d146c12470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36350/0 (socket says 172.21.15.18:36350) 2024-02-17T12:49:31.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ffb7fe640 1 -- 172.21.15.18:0/998753634 learned_addr learned my addr 172.21.15.18:0/998753634 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/998753634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c12ae0 0x55d146c14f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ff8ff9640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2fe40a20c0 0x7f2fe40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/998753634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c0cf80 0x55d146c0d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ffb7fe640 1 -- 172.21.15.18:0/998753634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c12ae0 msgr2=0x55d146c14f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ffb7fe640 1 --2- 172.21.15.18:0/998753634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c12ae0 0x55d146c14f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ffb7fe640 1 -- 172.21.15.18:0/998753634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c0cf80 msgr2=0x55d146c0d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ffb7fe640 1 --2- 172.21.15.18:0/998753634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c0cf80 0x55d146c0d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ffb7fe640 1 -- 172.21.15.18:0/998753634 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d146b29f20 con 0x55d146c0d970 2024-02-17T12:49:31.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ffbfff640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2fe40a20c0 0x7f2fe40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ffb7fe640 1 --2- 172.21.15.18:0/998753634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 0x55d146c12470 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00bcb0 tx=0x7f2fec0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d620122fce973e01 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.007+0000 7f2ff8ff9640 1 -- 172.21.15.18:0/2300326660 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2ff40f0bb0 con 0x7f2ffc101c50 2024-02-17T12:49:31.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f2fe1ffb640 1 -- 172.21.15.18:0/998753634 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec005020 con 0x55d146c0d970 2024-02-17T12:49:31.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f2fe1ffb640 1 -- 172.21.15.18:0/998753634 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fec015070 con 0x55d146c0d970 2024-02-17T12:49:31.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f2fe1ffb640 1 -- 172.21.15.18:0/998753634 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0116a0 con 0x55d146c0d970 2024-02-17T12:49:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f3004dca740 1 -- 172.21.15.18:0/998753634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 msgr2=0x55d146c12470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f2ffbfff640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2fe40a20c0 0x7f2fe40a4580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40bd8a0 tx=0x7f2ff40e8210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f3004dca740 1 --2- 172.21.15.18:0/998753634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 0x55d146c12470 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00bcb0 tx=0x7f2fec0107a0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f3004dca740 1 -- 172.21.15.18:0/998753634 shutdown_connections 2024-02-17T12:49:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f3004dca740 1 --2- 172.21.15.18:0/998753634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c12ae0 0x55d146c14f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f3004dca740 1 --2- 172.21.15.18:0/998753634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 0x55d146c12470 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f3004dca740 1 --2- 172.21.15.18:0/998753634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c0cf80 0x55d146c0d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f3004dca740 1 -- 172.21.15.18:0/998753634 >> 172.21.15.18:0/998753634 conn(0x55d146c084f0 msgr2=0x55d146c088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f3004dca740 1 -- 172.21.15.18:0/998753634 shutdown_connections 2024-02-17T12:49:31.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.008+0000 7f3004dca740 1 -- 172.21.15.18:0/998753634 wait complete. 2024-02-17T12:49:31.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f3004dca740 1 -- start start 2024-02-17T12:49:31.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f3004dca740 1 Processor -- start 2024-02-17T12:49:31.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f3004dca740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0cf80 0x55d146c17160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f3004dca740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 0x55d146c176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f3004dca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c12ae0 0x55d146c1e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f3004dca740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d146b74c30 con 0x55d146c0d970 2024-02-17T12:49:31.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f3004dca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d146b19d30 con 0x55d146c12ae0 2024-02-17T12:49:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f3004dca740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d146848310 con 0x55d146c0cf80 2024-02-17T12:49:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f3004dca740 10 client.0 _ll_register_callbacks cb 0x55d146b21d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 0x55d146c176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 0x55d146c176a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36356/0 (socket says 172.21.15.18:36356) 2024-02-17T12:49:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f2ffb7fe640 1 -- 172.21.15.18:0/1838196772 learned_addr learned my addr 172.21.15.18:0/1838196772 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/1838196772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0cf80 0x55d146c17160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f2ffb7fe640 1 -- 172.21.15.18:0/1838196772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0cf80 msgr2=0x55d146c17160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f2ffb7fe640 1 --2- 172.21.15.18:0/1838196772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0cf80 0x55d146c17160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f2ffb7fe640 1 -- 172.21.15.18:0/1838196772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c12ae0 msgr2=0x55d146c1e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f2ffb7fe640 1 --2- 172.21.15.18:0/1838196772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c12ae0 0x55d146c1e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.009+0000 7f2ffb7fe640 1 -- 172.21.15.18:0/1838196772 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fec00b8e0 con 0x55d146c0d970 2024-02-17T12:49:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f2ffb7fe640 1 --2- 172.21.15.18:0/1838196772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 0x55d146c176a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2fec011010 tx=0x7f2fec0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3b9fe6b611367a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f2fdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f2fdaffd640 1 -- 172.21.15.18:0/1838196772 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec00bee0 con 0x55d146c0d970 2024-02-17T12:49:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f2fdaffd640 1 -- 172.21.15.18:0/1838196772 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2fec017070 con 0x55d146c0d970 2024-02-17T12:49:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f3004dca740 1 -- 172.21.15.18:0/1838196772 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1468746c0 con 0x55d146c0d970 2024-02-17T12:49:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f2fdaffd640 1 -- 172.21.15.18:0/1838196772 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fec0052f0 con 0x55d146c0d970 2024-02-17T12:49:31.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f3004dca740 10 client.15267 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:31.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f3004dca740 1 -- 172.21.15.18:0/1838196772 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d146b732b0 con 0x55d146c0d970 2024-02-17T12:49:31.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f2fdaffd640 1 -- 172.21.15.18:0/1838196772 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fec035020 con 0x55d146c0d970 2024-02-17T12:49:31.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f2fdaffd640 1 -- 172.21.15.18:0/1838196772 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2fec02f020 con 0x55d146c0d970 2024-02-17T12:49:31.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f3004dca740 10 client.15267 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:31.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f3004dca740 10 client.15267 resolve_mds: resolved name 'k' to daemon mds.k 2024-02-17T12:49:31.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f3004dca740 20 client.15267 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:31.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f3004dca740 1 --2- 172.21.15.18:0/1838196772 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d146c23300 0x55d146c27dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f3004dca740 4 client.15267 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:49:31.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.010+0000 7f3004dca740 1 -- 172.21.15.18:0/1838196772 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1468c5270 con 0x55d146c23300 2024-02-17T12:49:31.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.011+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/1838196772 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d146c23300 0x55d146c27dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.011+0000 7f2fdaffd640 10 client.15267 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:49:31.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.011+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/1838196772 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d146c23300 0x55d146c27dc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f3f0bb8e3393ef0d server_cookie=a81351b2f866f056 in_seq=0 out_seq=0 2024-02-17T12:49:31.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f2fdaffd640 1 -- 172.21.15.18:0/1838196772 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d1468c5270 con 0x55d146c23300 2024-02-17T12:49:31.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f2fdaffd640 10 client.15267 handle_command_reply: tid=0 2024-02-17T12:49:31.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f3004dca740 1 client.15267 shutdown 2024-02-17T12:49:31.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f3004dca740 1 -- 172.21.15.18:0/1838196772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 msgr2=0x55d146c176a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f3004dca740 1 --2- 172.21.15.18:0/1838196772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 0x55d146c176a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2fec011010 tx=0x7f2fec0048f0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f3004dca740 1 -- 172.21.15.18:0/1838196772 shutdown_connections 2024-02-17T12:49:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f3004dca740 1 --2- 172.21.15.18:0/1838196772 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d146c23300 0x55d146c27dc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f3004dca740 1 --2- 172.21.15.18:0/1838196772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c12ae0 0x55d146c1e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f3004dca740 1 --2- 172.21.15.18:0/1838196772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0d970 0x55d146c176a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f2fdaffd640 0 client.15267 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:49:31.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f3004dca740 1 --2- 172.21.15.18:0/1838196772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0cf80 0x55d146c17160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f3004dca740 1 -- 172.21.15.18:0/1838196772 >> 172.21.15.18:0/1838196772 conn(0x55d146c084f0 msgr2=0x55d146c077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f3004dca740 1 -- 172.21.15.18:0/1838196772 shutdown_connections 2024-02-17T12:49:31.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f3004dca740 1 -- 172.21.15.18:0/1838196772 wait complete. 2024-02-17T12:49:31.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.012+0000 7f3004dca740 20 client.15267 trim_cache size 0 max 16384 2024-02-17T12:49:31.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.027+0000 7f3004dca740 1 Processor -- start 2024-02-17T12:49:31.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.027+0000 7f3004dca740 1 -- start start 2024-02-17T12:49:31.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f3004dca740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0cf80 0x55d146c2c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f3004dca740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0d970 0x55d146c1e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f3004dca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c12ae0 0x55d146c06f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f3004dca740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d146afa490 con 0x55d146c0cf80 2024-02-17T12:49:31.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f3004dca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1468c52a0 con 0x55d146c12ae0 2024-02-17T12:49:31.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f3004dca740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d146b6dad0 con 0x55d146c0d970 2024-02-17T12:49:31.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f2ffaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0cf80 0x55d146c2c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f2ffaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0cf80 0x55d146c2c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36370/0 (socket says 172.21.15.18:36370) 2024-02-17T12:49:31.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f2ffaffd640 1 -- 172.21.15.18:0/1832386582 learned_addr learned my addr 172.21.15.18:0/1832386582 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f2ffaffd640 1 -- 172.21.15.18:0/1832386582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0d970 msgr2=0x55d146c1e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/1832386582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0d970 0x55d146c1e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f2ffaffd640 1 -- 172.21.15.18:0/1832386582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c12ae0 msgr2=0x55d146c06f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/1832386582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c12ae0 0x55d146c06f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f2ffaffd640 1 -- 172.21.15.18:0/1832386582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d146b0fa20 con 0x55d146c0cf80 2024-02-17T12:49:31.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/1832386582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0cf80 0x55d146c2c4f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc1986d0 tx=0x7f2ff00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb04cdd964f4716 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f2fe17fa640 1 -- 172.21.15.18:0/1832386582 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0016040 con 0x55d146c0cf80 2024-02-17T12:49:31.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f2fe17fa640 1 -- 172.21.15.18:0/1832386582 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ff001c860 con 0x55d146c0cf80 2024-02-17T12:49:31.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.028+0000 7f2fe17fa640 1 -- 172.21.15.18:0/1832386582 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0003d10 con 0x55d146c0cf80 2024-02-17T12:49:31.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 -- 172.21.15.18:0/1832386582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0cf80 msgr2=0x55d146c2c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 --2- 172.21.15.18:0/1832386582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0cf80 0x55d146c2c4f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc1986d0 tx=0x7f2ff00118d0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 -- 172.21.15.18:0/1832386582 shutdown_connections 2024-02-17T12:49:31.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 --2- 172.21.15.18:0/1832386582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c12ae0 0x55d146c06f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 --2- 172.21.15.18:0/1832386582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0d970 0x55d146c1e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 --2- 172.21.15.18:0/1832386582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c0cf80 0x55d146c2c4f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 -- 172.21.15.18:0/1832386582 >> 172.21.15.18:0/1832386582 conn(0x55d146c084f0 msgr2=0x55d146c2bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 -- 172.21.15.18:0/1832386582 shutdown_connections 2024-02-17T12:49:31.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 -- 172.21.15.18:0/1832386582 wait complete. 2024-02-17T12:49:31.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 -- start start 2024-02-17T12:49:31.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 Processor -- start 2024-02-17T12:49:31.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0cf80 0x55d146c1b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c0d970 0x55d146c1bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c12ae0 0x55d146c79a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.029+0000 7f3004dca740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d146c1c270 con 0x55d146c12ae0 2024-02-17T12:49:31.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f3004dca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d146c79fc0 con 0x55d146c0d970 2024-02-17T12:49:31.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f3004dca740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d146c7a180 con 0x55d146c0cf80 2024-02-17T12:49:31.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f3004dca740 10 client.0 _ll_register_callbacks cb 0x7f2fec00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:31.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2ffaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c12ae0 0x55d146c79a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2ffaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c12ae0 0x55d146c79a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36380/0 (socket says 172.21.15.18:36380) 2024-02-17T12:49:31.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2ffaffd640 1 -- 172.21.15.18:0/3968849113 learned_addr learned my addr 172.21.15.18:0/3968849113 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2ffb7fe640 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c0d970 0x55d146c1bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0cf80 0x55d146c1b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2ffaffd640 1 -- 172.21.15.18:0/3968849113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0cf80 msgr2=0x55d146c1b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0cf80 0x55d146c1b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2ffaffd640 1 -- 172.21.15.18:0/3968849113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c0d970 msgr2=0x55d146c1bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c0d970 0x55d146c1bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2ffaffd640 1 -- 172.21.15.18:0/3968849113 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff000ea50 con 0x55d146c12ae0 2024-02-17T12:49:31.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2ffb7fe640 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c0d970 0x55d146c1bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:31.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2ffaffd640 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c12ae0 0x55d146c79a80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0005fd0 tx=0x7f2ff0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d49dd0a6377fce2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2fdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2fdaffd640 1 -- 172.21.15.18:0/3968849113 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0016040 con 0x55d146c12ae0 2024-02-17T12:49:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2fdaffd640 1 -- 172.21.15.18:0/3968849113 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2ff001c690 con 0x55d146c12ae0 2024-02-17T12:49:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f3004dca740 1 -- 172.21.15.18:0/3968849113 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d146c23cb0 con 0x55d146c12ae0 2024-02-17T12:49:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f2fdaffd640 1 -- 172.21.15.18:0/3968849113 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ff0004000 con 0x55d146c12ae0 2024-02-17T12:49:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f3004dca740 10 client.15273 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:31.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.030+0000 7f3004dca740 1 -- 172.21.15.18:0/3968849113 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d146b35780 con 0x55d146c12ae0 2024-02-17T12:49:31.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.031+0000 7f2fdaffd640 1 -- 172.21.15.18:0/3968849113 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d146c23cb0 con 0x55d146c12ae0 2024-02-17T12:49:31.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.031+0000 7f2fdaffd640 1 -- 172.21.15.18:0/3968849113 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2ff00047e0 con 0x55d146c12ae0 2024-02-17T12:49:31.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.031+0000 7f3004dca740 10 client.15273 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.031+0000 7f3004dca740 10 client.15273 resolve_mds: resolved name 'k' to daemon mds.k 2024-02-17T12:49:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.031+0000 7f3004dca740 20 client.15273 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.031+0000 7f3004dca740 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d146c7f3e0 0x55d146c7b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.031+0000 7f3004dca740 4 client.15273 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T12:49:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.031+0000 7f3004dca740 1 -- 172.21.15.18:0/3968849113 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f2fec015070 con 0x55d146c7f3e0 2024-02-17T12:49:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.031+0000 7f2ffb7fe640 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d146c7f3e0 0x55d146c7b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.031+0000 7f2ffb7fe640 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d146c7f3e0 0x55d146c7b8a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1c00f064134948ee server_cookie=fc17e3f7189360c4 in_seq=0 out_seq=0 2024-02-17T12:49:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.031+0000 7f2fdaffd640 10 client.15273 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:49:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f2fdaffd640 1 -- 172.21.15.18:0/3968849113 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f2fec015070 con 0x55d146c7f3e0 2024-02-17T12:49:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f2fdaffd640 10 client.15273 handle_command_reply: tid=0 2024-02-17T12:49:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3004dca740 1 client.15273 shutdown 2024-02-17T12:49:31.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3004dca740 1 -- 172.21.15.18:0/3968849113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c12ae0 msgr2=0x55d146c79a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3004dca740 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c12ae0 0x55d146c79a80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0005fd0 tx=0x7f2ff0013040 comp rx=0 tx=0).stop 2024-02-17T12:49:31.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3004dca740 1 -- 172.21.15.18:0/3968849113 shutdown_connections 2024-02-17T12:49:31.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3004dca740 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d146c12ae0 0x55d146c79a80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3004dca740 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d146c7f3e0 0x55d146c7b8a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3004dca740 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d146c0d970 0x55d146c1bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3004dca740 1 --2- 172.21.15.18:0/3968849113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d146c0cf80 0x55d146c1b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f2fdaffd640 0 client.15273 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:49:31.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3004dca740 1 -- 172.21.15.18:0/3968849113 >> 172.21.15.18:0/3968849113 conn(0x55d146c084f0 msgr2=0x55d146c2c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3004dca740 1 -- 172.21.15.18:0/3968849113 shutdown_connections 2024-02-17T12:49:31.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3004dca740 1 -- 172.21.15.18:0/3968849113 wait complete. 2024-02-17T12:49:31.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3004dca740 20 client.15273 trim_cache size 0 max 16384 2024-02-17T12:49:31.055 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:49:31.055 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:49:31.055 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 4, 2024-02-17T12:49:31.056 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24422, 2024-02-17T12:49:31.056 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-02-17T12:49:31.056 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-02-17T12:49:31.056 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-02-17T12:49:31.056 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 100.316713308, 2024-02-17T12:49:31.056 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 25, 2024-02-17T12:49:31.057 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 88, 2024-02-17T12:49:31.057 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 86, 2024-02-17T12:49:31.057 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 138.19785795000001 2024-02-17T12:49:31.057 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:49:31.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3002113640 1 -- 172.21.15.18:0/2300326660 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2fe40a20c0 msgr2=0x7f2fe40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3002113640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2fe40a20c0 0x7f2fe40a4580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40bd8a0 tx=0x7f2ff40e8210 comp rx=0 tx=0).stop 2024-02-17T12:49:31.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3002113640 1 -- 172.21.15.18:0/2300326660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ffc101c50 msgr2=0x7f2ffc19e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3002113640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ffc101c50 0x7f2ffc19e770 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4037040 tx=0x7f2ff4034d40 comp rx=0 tx=0).stop 2024-02-17T12:49:31.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3002113640 1 -- 172.21.15.18:0/2300326660 shutdown_connections 2024-02-17T12:49:31.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3002113640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2fe40a20c0 0x7f2fe40a4580 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3002113640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ffc101c50 0x7f2ffc19e770 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3002113640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ffc101290 0x7f2ffc197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3002113640 1 --2- 172.21.15.18:0/2300326660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc1000a0 0x7f2ffc1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.032+0000 7f3002113640 1 -- 172.21.15.18:0/2300326660 >> 172.21.15.18:0/2300326660 conn(0x7f2ffc0fb630 msgr2=0x7f2ffc0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.033+0000 7f3002113640 1 -- 172.21.15.18:0/2300326660 shutdown_connections 2024-02-17T12:49:31.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.033+0000 7f3002113640 1 -- 172.21.15.18:0/2300326660 wait complete. 2024-02-17T12:49:31.062 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-02-17T12:49:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 -- 172.21.15.18:0/2412491012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9004102d50 msgr2=0x7f90041095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 --2- 172.21.15.18:0/2412491012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9004102d50 0x7f90041095e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009ca0 tx=0x7f8ff402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 -- 172.21.15.18:0/2412491012 shutdown_connections 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 --2- 172.21.15.18:0/2412491012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9004102d50 0x7f90041095e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 --2- 172.21.15.18:0/2412491012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9004102390 0x7f9004102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 --2- 172.21.15.18:0/2412491012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90041011a0 0x7f90041015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 -- 172.21.15.18:0/2412491012 >> 172.21.15.18:0/2412491012 conn(0x7f90040fc710 msgr2=0x7f90040feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 -- 172.21.15.18:0/2412491012 shutdown_connections 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 -- 172.21.15.18:0/2412491012 wait complete. 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 Processor -- start 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 -- start start 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90041011a0 0x7f9004197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9004102390 0x7f90041977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9004102d50 0x7f900419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900419ed70 con 0x7f90041011a0 2024-02-17T12:49:31.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900419eee0 con 0x7f9004102390 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.174+0000 7f900b31e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900419f0a0 con 0x7f9004102d50 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90090ba640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90041011a0 0x7f9004197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90090ba640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90041011a0 0x7f9004197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36406/0 (socket says 172.21.15.18:36406) 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90090ba640 1 -- 172.21.15.18:0/317968901 learned_addr learned my addr 172.21.15.18:0/317968901 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90098bb640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9004102d50 0x7f900419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90090ba640 1 -- 172.21.15.18:0/317968901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9004102d50 msgr2=0x7f900419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90090ba640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9004102d50 0x7f900419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90088b9640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9004102390 0x7f90041977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90090ba640 1 -- 172.21.15.18:0/317968901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9004102390 msgr2=0x7f90041977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90090ba640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9004102390 0x7f90041977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90090ba640 1 -- 172.21.15.18:0/317968901 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff4009900 con 0x7f90041011a0 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90088b9640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9004102390 0x7f90041977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90090ba640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90041011a0 0x7f9004197280 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c970 tx=0x7f8ffc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90027fc640 1 -- 172.21.15.18:0/317968901 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc004220 con 0x7f90041011a0 2024-02-17T12:49:31.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90027fc640 1 -- 172.21.15.18:0/317968901 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ffc007e90 con 0x7f90041011a0 2024-02-17T12:49:31.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f900b31e640 1 -- 172.21.15.18:0/317968901 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900419f380 con 0x7f90041011a0 2024-02-17T12:49:31.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f90027fc640 1 -- 172.21.15.18:0/317968901 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc004a00 con 0x7f90041011a0 2024-02-17T12:49:31.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.175+0000 7f900b31e640 1 -- 172.21.15.18:0/317968901 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900419f890 con 0x7f90041011a0 2024-02-17T12:49:31.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.177+0000 7f90027fc640 1 -- 172.21.15.18:0/317968901 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8ffc0079b0 con 0x7f90041011a0 2024-02-17T12:49:31.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f900dfcd740 1 Processor -- start 2024-02-17T12:49:31.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f900dfcd740 1 -- start start 2024-02-17T12:49:31.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f900dfcd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed6f80 0x557a77ed73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f900dfcd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed7970 0x557a77edc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f900dfcd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 0x557a77edef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f900dfcd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a77e92cd0 con 0x557a77edcae0 2024-02-17T12:49:31.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f900dfcd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a77e45f20 con 0x557a77ed7970 2024-02-17T12:49:31.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f900dfcd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a77e43450 con 0x557a77ed6f80 2024-02-17T12:49:31.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90027fc640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8fec0a2140 0x7f8fec0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90088b9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 0x557a77edef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90088b9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 0x557a77edef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36416/0 (socket says 172.21.15.18:36416) 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90088b9640 1 -- 172.21.15.18:0/1964810266 learned_addr learned my addr 172.21.15.18:0/1964810266 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90098bb640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8fec0a2140 0x7f8fec0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90090ba640 1 --2- 172.21.15.18:0/1964810266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed7970 0x557a77edc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90098bb640 1 --2- 172.21.15.18:0/1964810266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed6f80 0x557a77ed73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90027fc640 1 -- 172.21.15.18:0/317968901 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8ffc0cdd80 con 0x7f90041011a0 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90088b9640 1 -- 172.21.15.18:0/1964810266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed6f80 msgr2=0x557a77ed73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90088b9640 1 --2- 172.21.15.18:0/1964810266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed6f80 0x557a77ed73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90088b9640 1 -- 172.21.15.18:0/1964810266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed7970 msgr2=0x557a77edc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90088b9640 1 --2- 172.21.15.18:0/1964810266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed7970 0x557a77edc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90088b9640 1 -- 172.21.15.18:0/1964810266 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a77df3f20 con 0x557a77edcae0 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90098bb640 1 --2- 172.21.15.18:0/1964810266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed6f80 0x557a77ed73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.178+0000 7f90088b9640 1 --2- 172.21.15.18:0/1964810266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 0x557a77edef70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800a9d0 tx=0x7f8ff800ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a84ca68ddf6fcad server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.179+0000 7f90098bb640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8fec0a2140 0x7f8fec0a4600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4006010 tx=0x7f8ff4032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.179+0000 7f8feb7fe640 1 -- 172.21.15.18:0/1964810266 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800cc60 con 0x557a77edcae0 2024-02-17T12:49:31.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.179+0000 7f8feb7fe640 1 -- 172.21.15.18:0/1964810266 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ff800cdc0 con 0x557a77edcae0 2024-02-17T12:49:31.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.179+0000 7f8feb7fe640 1 -- 172.21.15.18:0/1964810266 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8018700 con 0x557a77edcae0 2024-02-17T12:49:31.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.179+0000 7f900dfcd740 1 -- 172.21.15.18:0/1964810266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 msgr2=0x557a77edef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.179+0000 7f900dfcd740 1 --2- 172.21.15.18:0/1964810266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 0x557a77edef70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800a9d0 tx=0x7f8ff800ae90 comp rx=0 tx=0).stop 2024-02-17T12:49:31.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.179+0000 7f900dfcd740 1 -- 172.21.15.18:0/1964810266 shutdown_connections 2024-02-17T12:49:31.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.179+0000 7f900dfcd740 1 --2- 172.21.15.18:0/1964810266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 0x557a77edef70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.179+0000 7f900dfcd740 1 --2- 172.21.15.18:0/1964810266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed7970 0x557a77edc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.179+0000 7f900dfcd740 1 --2- 172.21.15.18:0/1964810266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed6f80 0x557a77ed73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.179+0000 7f900dfcd740 1 -- 172.21.15.18:0/1964810266 >> 172.21.15.18:0/1964810266 conn(0x557a77ed24f0 msgr2=0x557a77ed28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.179+0000 7f900dfcd740 1 -- 172.21.15.18:0/1964810266 shutdown_connections 2024-02-17T12:49:31.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f900dfcd740 1 -- 172.21.15.18:0/1964810266 wait complete. 2024-02-17T12:49:31.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f900dfcd740 1 -- start start 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f900dfcd740 1 Processor -- start 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f900dfcd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed6f80 0x557a77ee1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f900dfcd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed7970 0x557a77ee1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f900dfcd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 0x557a77ee8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f900dfcd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a77e3ec30 con 0x557a77edcae0 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f900dfcd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a77de3d30 con 0x557a77ed6f80 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f900dfcd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a77b12310 con 0x557a77ed7970 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f900dfcd740 10 client.0 _ll_register_callbacks cb 0x557a77debd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f90098bb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 0x557a77ee8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f90098bb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 0x557a77ee8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36420/0 (socket says 172.21.15.18:36420) 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f90098bb640 1 -- 172.21.15.18:0/3873634625 learned_addr learned my addr 172.21.15.18:0/3873634625 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.180+0000 7f90088b9640 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed7970 0x557a77ee1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f90090ba640 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed6f80 0x557a77ee1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f90098bb640 1 -- 172.21.15.18:0/3873634625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed7970 msgr2=0x557a77ee1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f90098bb640 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed7970 0x557a77ee1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f90098bb640 1 -- 172.21.15.18:0/3873634625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed6f80 msgr2=0x557a77ee1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f90098bb640 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed6f80 0x557a77ee1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f90098bb640 1 -- 172.21.15.18:0/3873634625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff800a600 con 0x557a77edcae0 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f90098bb640 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 0x557a77ee8680 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4036000 tx=0x7f8ff40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bda80503cd4621f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f8fe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f8fe8ff9640 1 -- 172.21.15.18:0/3873634625 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4003b80 con 0x557a77edcae0 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f8fe8ff9640 1 -- 172.21.15.18:0/3873634625 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8ff4003ce0 con 0x557a77edcae0 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f900dfcd740 1 -- 172.21.15.18:0/3873634625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a77b3e6c0 con 0x557a77edcae0 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f8fe8ff9640 1 -- 172.21.15.18:0/3873634625 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ff4003ea0 con 0x557a77edcae0 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f900dfcd740 10 client.15285 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f900dfcd740 1 -- 172.21.15.18:0/3873634625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557a77e3d2b0 con 0x557a77edcae0 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f8fe8ff9640 1 -- 172.21.15.18:0/3873634625 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557a77b3e6c0 con 0x557a77edcae0 2024-02-17T12:49:31.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f8fe8ff9640 1 -- 172.21.15.18:0/3873634625 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ff404c870 con 0x557a77edcae0 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.181+0000 7f900dfcd740 10 client.15285 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.182+0000 7f900dfcd740 10 client.15285 resolve_mds: resolved name 'f' to daemon mds.f 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.182+0000 7f900dfcd740 20 client.15285 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.182+0000 7f900dfcd740 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557a77eeda50 0x557a77eecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.182+0000 7f900dfcd740 4 client.15285 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.182+0000 7f900dfcd740 1 -- 172.21.15.18:0/3873634625 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a77b8f270 con 0x557a77eeda50 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.182+0000 7f90090ba640 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557a77eeda50 0x557a77eecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.182+0000 7f8fe8ff9640 10 client.15285 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.182+0000 7f90090ba640 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557a77eeda50 0x557a77eecc70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bd77e2229d6216a8 server_cookie=6188f1f545683dd in_seq=0 out_seq=0 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f8fe8ff9640 1 -- 172.21.15.18:0/3873634625 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a77b8f270 con 0x557a77eeda50 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f8fe8ff9640 10 client.15285 handle_command_reply: tid=0 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f900dfcd740 1 client.15285 shutdown 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f900dfcd740 1 -- 172.21.15.18:0/3873634625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 msgr2=0x557a77ee8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f900dfcd740 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 0x557a77ee8680 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4036000 tx=0x7f8ff40095c0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f900dfcd740 1 -- 172.21.15.18:0/3873634625 shutdown_connections 2024-02-17T12:49:31.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f900dfcd740 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557a77eeda50 0x557a77eecc70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f900dfcd740 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77edcae0 0x557a77ee8680 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f900dfcd740 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed7970 0x557a77ee1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f900dfcd740 1 --2- 172.21.15.18:0/3873634625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed6f80 0x557a77ee1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f8fe8ff9640 0 client.15285 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:49:31.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f900dfcd740 1 -- 172.21.15.18:0/3873634625 >> 172.21.15.18:0/3873634625 conn(0x557a77ed24f0 msgr2=0x557a77ed17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f900dfcd740 1 -- 172.21.15.18:0/3873634625 shutdown_connections 2024-02-17T12:49:31.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f900dfcd740 1 -- 172.21.15.18:0/3873634625 wait complete. 2024-02-17T12:49:31.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.183+0000 7f900dfcd740 20 client.15285 trim_cache size 0 max 16384 2024-02-17T12:49:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.199+0000 7f900dfcd740 1 Processor -- start 2024-02-17T12:49:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.199+0000 7f900dfcd740 1 -- start start 2024-02-17T12:49:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.199+0000 7f900dfcd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed24f0 0x557a77ed8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.199+0000 7f900dfcd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed6f80 0x557a77ed85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.199+0000 7f900dfcd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed7970 0x557a77ee85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.199+0000 7f900dfcd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a77dc4490 con 0x557a77ed7970 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.199+0000 7f900dfcd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a77b8f270 con 0x557a77ed6f80 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.199+0000 7f900dfcd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a77b3e6c0 con 0x557a77ed24f0 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.199+0000 7f90098bb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed7970 0x557a77ee85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.199+0000 7f90098bb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed7970 0x557a77ee85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36426/0 (socket says 172.21.15.18:36426) 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.199+0000 7f90098bb640 1 -- 172.21.15.18:0/1287898466 learned_addr learned my addr 172.21.15.18:0/1287898466 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.200+0000 7f90090ba640 1 --2- 172.21.15.18:0/1287898466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed24f0 0x557a77ed8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.200+0000 7f90098bb640 1 -- 172.21.15.18:0/1287898466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed24f0 msgr2=0x557a77ed8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.200+0000 7f90098bb640 1 --2- 172.21.15.18:0/1287898466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed24f0 0x557a77ed8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.200+0000 7f90098bb640 1 -- 172.21.15.18:0/1287898466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed6f80 msgr2=0x557a77ed85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.200+0000 7f90098bb640 1 --2- 172.21.15.18:0/1287898466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed6f80 0x557a77ed85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.200+0000 7f90098bb640 1 -- 172.21.15.18:0/1287898466 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a77dd9a20 con 0x557a77ed7970 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.200+0000 7f90090ba640 1 --2- 172.21.15.18:0/1287898466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed24f0 0x557a77ed8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.200+0000 7f90098bb640 1 --2- 172.21.15.18:0/1287898466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed7970 0x557a77ee85a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4005d50 tx=0x7f8ff4034f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f0d05f46409882 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.200+0000 7f8feaffd640 1 -- 172.21.15.18:0/1287898466 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff404c630 con 0x557a77ed7970 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.200+0000 7f8feaffd640 1 -- 172.21.15.18:0/1287898466 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ff404c790 con 0x557a77ed7970 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.200+0000 7f8feaffd640 1 -- 172.21.15.18:0/1287898466 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4055440 con 0x557a77ed7970 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 -- 172.21.15.18:0/1287898466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed7970 msgr2=0x557a77ee85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 --2- 172.21.15.18:0/1287898466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed7970 0x557a77ee85a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4005d50 tx=0x7f8ff4034f10 comp rx=0 tx=0).stop 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 -- 172.21.15.18:0/1287898466 shutdown_connections 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 --2- 172.21.15.18:0/1287898466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed7970 0x557a77ee85a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 --2- 172.21.15.18:0/1287898466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed6f80 0x557a77ed85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 --2- 172.21.15.18:0/1287898466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed24f0 0x557a77ed8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 -- 172.21.15.18:0/1287898466 >> 172.21.15.18:0/1287898466 conn(0x557a77ed2c40 msgr2=0x557a77ef5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 -- 172.21.15.18:0/1287898466 shutdown_connections 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 -- 172.21.15.18:0/1287898466 wait complete. 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 -- start start 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 Processor -- start 2024-02-17T12:49:31.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed24f0 0x557a77ee83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed6f80 0x557a77ee4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed7970 0x557a77ee53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a77ee58f0 con 0x557a77ed24f0 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a77ee5a60 con 0x557a77ed7970 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a77f43650 con 0x557a77ed6f80 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f900dfcd740 10 client.0 _ll_register_callbacks cb 0x7f8ff800a1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f90090ba640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed24f0 0x557a77ee83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f90090ba640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed24f0 0x557a77ee83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36436/0 (socket says 172.21.15.18:36436) 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.201+0000 7f90090ba640 1 -- 172.21.15.18:0/192421154 learned_addr learned my addr 172.21.15.18:0/192421154 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f90088b9640 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed6f80 0x557a77ee4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f90098bb640 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed7970 0x557a77ee53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f90090ba640 1 -- 172.21.15.18:0/192421154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed6f80 msgr2=0x557a77ee4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f90090ba640 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed6f80 0x557a77ee4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f90090ba640 1 -- 172.21.15.18:0/192421154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed7970 msgr2=0x557a77ee53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f90090ba640 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed7970 0x557a77ee53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f90090ba640 1 -- 172.21.15.18:0/192421154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a77e3d2b0 con 0x557a77ed24f0 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f90098bb640 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed7970 0x557a77ee53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f90090ba640 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed24f0 0x557a77ee83d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc09aa10 tx=0x7f8ffc01eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fabddef7a4214a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f8fe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f8fe8ff9640 1 -- 172.21.15.18:0/192421154 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc0cf030 con 0x557a77ed24f0 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f8fe8ff9640 1 -- 172.21.15.18:0/192421154 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8ffc09ebd0 con 0x557a77ed24f0 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f900dfcd740 1 -- 172.21.15.18:0/192421154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ff800cdc0 con 0x557a77ed24f0 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f8fe8ff9640 1 -- 172.21.15.18:0/192421154 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ffc0a3220 con 0x557a77ed24f0 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f900dfcd740 10 client.15291 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f900dfcd740 1 -- 172.21.15.18:0/192421154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557a77dff780 con 0x557a77ed24f0 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f8fe8ff9640 1 -- 172.21.15.18:0/192421154 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ffc07d080 con 0x557a77ed24f0 2024-02-17T12:49:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f8fe8ff9640 1 -- 172.21.15.18:0/192421154 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ffc020070 con 0x557a77ed24f0 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.202+0000 7f900dfcd740 10 client.15291 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.203+0000 7f900dfcd740 10 client.15291 resolve_mds: resolved name 'f' to daemon mds.f 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.203+0000 7f900dfcd740 20 client.15291 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.203+0000 7f900dfcd740 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557a77edcae0 0x557a77f44d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.203+0000 7f900dfcd740 4 client.15291 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.203+0000 7f900dfcd740 1 -- 172.21.15.18:0/192421154 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557a77b7b610 con 0x557a77edcae0 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.203+0000 7f90088b9640 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557a77edcae0 0x557a77f44d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.203+0000 7f8fe8ff9640 10 client.15291 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.203+0000 7f90088b9640 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557a77edcae0 0x557a77f44d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef68954f38a65d8b server_cookie=cf1a265cbf33d129 in_seq=0 out_seq=0 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.203+0000 7f8fe8ff9640 1 -- 172.21.15.18:0/192421154 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557a77b7b610 con 0x557a77edcae0 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.203+0000 7f8fe8ff9640 10 client.15291 handle_command_reply: tid=0 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.203+0000 7f900dfcd740 1 client.15291 shutdown 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900dfcd740 1 -- 172.21.15.18:0/192421154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed24f0 msgr2=0x557a77ee83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900dfcd740 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed24f0 0x557a77ee83d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc09aa10 tx=0x7f8ffc01eea0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900dfcd740 1 -- 172.21.15.18:0/192421154 shutdown_connections 2024-02-17T12:49:31.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900dfcd740 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557a77edcae0 0x557a77f44d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900dfcd740 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a77ed7970 0x557a77ee53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900dfcd740 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a77ed6f80 0x557a77ee4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f8fe8ff9640 0 client.15291 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:49:31.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900dfcd740 1 --2- 172.21.15.18:0/192421154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a77ed24f0 0x557a77ee83d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900dfcd740 1 -- 172.21.15.18:0/192421154 >> 172.21.15.18:0/192421154 conn(0x557a77ed2c40 msgr2=0x557a77ed8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900dfcd740 1 -- 172.21.15.18:0/192421154 shutdown_connections 2024-02-17T12:49:31.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900dfcd740 1 -- 172.21.15.18:0/192421154 wait complete. 2024-02-17T12:49:31.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900dfcd740 20 client.15291 trim_cache size 0 max 16384 2024-02-17T12:49:31.203 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:49:31.203 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:49:31.204 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 3, 2024-02-17T12:49:31.204 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24448, 2024-02-17T12:49:31.204 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-02-17T12:49:31.204 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-02-17T12:49:31.204 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-02-17T12:49:31.204 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 101.49500726700001, 2024-02-17T12:49:31.204 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 25, 2024-02-17T12:49:31.204 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 88, 2024-02-17T12:49:31.204 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 86, 2024-02-17T12:49:31.204 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 127.74524975999999 2024-02-17T12:49:31.204 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:49:31.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900b31e640 1 -- 172.21.15.18:0/317968901 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8fec0a2140 msgr2=0x7f8fec0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900b31e640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8fec0a2140 0x7f8fec0a4600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4006010 tx=0x7f8ff4032340 comp rx=0 tx=0).stop 2024-02-17T12:49:31.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900b31e640 1 -- 172.21.15.18:0/317968901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90041011a0 msgr2=0x7f9004197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900b31e640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90041011a0 0x7f9004197280 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c970 tx=0x7f8ffc00ce30 comp rx=0 tx=0).stop 2024-02-17T12:49:31.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900b31e640 1 -- 172.21.15.18:0/317968901 shutdown_connections 2024-02-17T12:49:31.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900b31e640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f8fec0a2140 0x7f8fec0a4600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900b31e640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9004102d50 0x7f900419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900b31e640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9004102390 0x7f90041977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900b31e640 1 --2- 172.21.15.18:0/317968901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90041011a0 0x7f9004197280 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900b31e640 1 -- 172.21.15.18:0/317968901 >> 172.21.15.18:0/317968901 conn(0x7f90040fc710 msgr2=0x7f9004105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900b31e640 1 -- 172.21.15.18:0/317968901 shutdown_connections 2024-02-17T12:49:31.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.204+0000 7f900b31e640 1 -- 172.21.15.18:0/317968901 wait complete. 2024-02-17T12:49:31.207 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-02-17T12:49:31.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.318+0000 7f500ba40640 1 -- 172.21.15.18:0/3708622368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50041011b0 msgr2=0x7f50041015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.318+0000 7f500ba40640 1 --2- 172.21.15.18:0/3708622368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50041011b0 0x7f50041015d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009900 tx=0x7f4ffc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:31.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.318+0000 7f500ba40640 1 -- 172.21.15.18:0/3708622368 shutdown_connections 2024-02-17T12:49:31.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.318+0000 7f500ba40640 1 --2- 172.21.15.18:0/3708622368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5004102d60 0x7f50041095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 --2- 172.21.15.18:0/3708622368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50041023a0 0x7f5004102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 --2- 172.21.15.18:0/3708622368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50041011b0 0x7f50041015d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 -- 172.21.15.18:0/3708622368 >> 172.21.15.18:0/3708622368 conn(0x7f50040fc720 msgr2=0x7f50040feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 -- 172.21.15.18:0/3708622368 shutdown_connections 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 -- 172.21.15.18:0/3708622368 wait complete. 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 Processor -- start 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 -- start start 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50041011b0 0x7f5004074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50041023a0 0x7f5004073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5004102d60 0x7f5004073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5004074210 con 0x7f50041023a0 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5004074380 con 0x7f50041011b0 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f500ba40640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50041a12e0 con 0x7f5004102d60 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f5008fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50041023a0 0x7f5004073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.319+0000 7f5008fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50041023a0 0x7f5004073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36460/0 (socket says 172.21.15.18:36460) 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f5008fdb640 1 -- 172.21.15.18:0/2539747157 learned_addr learned my addr 172.21.15.18:0/2539747157 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f50097dc640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50041011b0 0x7f5004074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f5009fdd640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5004102d60 0x7f5004073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f5008fdb640 1 -- 172.21.15.18:0/2539747157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5004102d60 msgr2=0x7f5004073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f5008fdb640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5004102d60 0x7f5004073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f5008fdb640 1 -- 172.21.15.18:0/2539747157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50041011b0 msgr2=0x7f5004074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f5008fdb640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50041011b0 0x7f5004074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f5008fdb640 1 -- 172.21.15.18:0/2539747157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc009560 con 0x7f50041023a0 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f5009fdd640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5004102d60 0x7f5004073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f5008fdb640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50041023a0 0x7f5004073570 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800c8c0 tx=0x7f4ff800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f50027fc640 1 -- 172.21.15.18:0/2539747157 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8004220 con 0x7f50041023a0 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f500ba40640 1 -- 172.21.15.18:0/2539747157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50041a1590 con 0x7f50041023a0 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f50027fc640 1 -- 172.21.15.18:0/2539747157 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ff8007e90 con 0x7f50041023a0 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f50027fc640 1 -- 172.21.15.18:0/2539747157 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8004a00 con 0x7f50041023a0 2024-02-17T12:49:31.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.320+0000 7f500ba40640 1 -- 172.21.15.18:0/2539747157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50041a1aa0 con 0x7f50041023a0 2024-02-17T12:49:31.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.322+0000 7f50027fc640 1 -- 172.21.15.18:0/2539747157 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4ff80079b0 con 0x7f50041023a0 2024-02-17T12:49:31.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f500e60e740 1 Processor -- start 2024-02-17T12:49:31.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f500e60e740 1 -- start start 2024-02-17T12:49:31.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f500e60e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 0x555fce7c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f500e60e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 0x555fce7c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f500e60e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c7ae0 0x555fce7c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f500e60e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fce77dcd0 con 0x555fce7c7ae0 2024-02-17T12:49:31.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f500e60e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fce730f20 con 0x555fce7c1f80 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f500e60e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fce72e450 con 0x555fce7c2970 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f50027fc640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4fec0a20c0 0x7f4fec0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f5008fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c7ae0 0x555fce7c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f50097dc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 0x555fce7c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f50097dc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 0x555fce7c7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:55676/0 (socket says 172.21.15.18:55676) 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f5008fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c7ae0 0x555fce7c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36464/0 (socket says 172.21.15.18:36464) 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f50097dc640 1 -- 172.21.15.18:0/4270614250 learned_addr learned my addr 172.21.15.18:0/4270614250 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.323+0000 7f5009fdd640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4fec0a20c0 0x7f4fec0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.324+0000 7f5009fdd640 1 --2- 172.21.15.18:0/4270614250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 0x555fce7c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.324+0000 7f50027fc640 1 -- 172.21.15.18:0/2539747157 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4ff80cda70 con 0x7f50041023a0 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.324+0000 7f50097dc640 1 -- 172.21.15.18:0/4270614250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 msgr2=0x555fce7c23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.324+0000 7f50097dc640 1 --2- 172.21.15.18:0/4270614250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 0x555fce7c23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.324+0000 7f50097dc640 1 -- 172.21.15.18:0/4270614250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c7ae0 msgr2=0x555fce7c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.324+0000 7f50097dc640 1 --2- 172.21.15.18:0/4270614250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c7ae0 0x555fce7c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.324+0000 7f50097dc640 1 -- 172.21.15.18:0/4270614250 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fce6def20 con 0x555fce7c2970 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.324+0000 7f5009fdd640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4fec0a20c0 0x7f4fec0a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x555fce7c31d0 tx=0x7f4ff4009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.324+0000 7f50097dc640 1 --2- 172.21.15.18:0/4270614250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 0x555fce7c7470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009900 tx=0x7f4ffc005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=479b73ec3f429644 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.324+0000 7f4feb7fe640 1 -- 172.21.15.18:0/4270614250 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc02cc60 con 0x555fce7c2970 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.324+0000 7f4feb7fe640 1 -- 172.21.15.18:0/4270614250 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ffc004030 con 0x555fce7c2970 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.324+0000 7f4feb7fe640 1 -- 172.21.15.18:0/4270614250 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc005360 con 0x555fce7c2970 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.325+0000 7f500e60e740 1 -- 172.21.15.18:0/4270614250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 msgr2=0x555fce7c7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.325+0000 7f500e60e740 1 --2- 172.21.15.18:0/4270614250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 0x555fce7c7470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009900 tx=0x7f4ffc005e60 comp rx=0 tx=0).stop 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.325+0000 7f500e60e740 1 -- 172.21.15.18:0/4270614250 shutdown_connections 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.325+0000 7f500e60e740 1 --2- 172.21.15.18:0/4270614250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c7ae0 0x555fce7c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.325+0000 7f500e60e740 1 --2- 172.21.15.18:0/4270614250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 0x555fce7c7470 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.325+0000 7f500e60e740 1 --2- 172.21.15.18:0/4270614250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 0x555fce7c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.325+0000 7f500e60e740 1 -- 172.21.15.18:0/4270614250 >> 172.21.15.18:0/4270614250 conn(0x555fce7bd4f0 msgr2=0x555fce7bd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.325+0000 7f500e60e740 1 -- 172.21.15.18:0/4270614250 shutdown_connections 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.325+0000 7f500e60e740 1 -- 172.21.15.18:0/4270614250 wait complete. 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.325+0000 7f500e60e740 1 -- start start 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.325+0000 7f500e60e740 1 Processor -- start 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f500e60e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 0x555fce7cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f500e60e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 0x555fce7cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f500e60e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c7ae0 0x555fce7d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f500e60e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fce729c30 con 0x555fce7c2970 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f500e60e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fce6ced30 con 0x555fce7c1f80 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f500e60e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fce3fd310 con 0x555fce7c7ae0 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f500e60e740 10 client.0 _ll_register_callbacks cb 0x555fce6d6d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f5008fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 0x555fce7cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f5008fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 0x555fce7cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36480/0 (socket says 172.21.15.18:36480) 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f5008fdb640 1 -- 172.21.15.18:0/917493685 learned_addr learned my addr 172.21.15.18:0/917493685 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f50097dc640 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 0x555fce7cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f5009fdd640 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c7ae0 0x555fce7d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f5008fdb640 1 -- 172.21.15.18:0/917493685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c7ae0 msgr2=0x555fce7d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f5008fdb640 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c7ae0 0x555fce7d3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f5008fdb640 1 -- 172.21.15.18:0/917493685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 msgr2=0x555fce7cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f5008fdb640 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 0x555fce7cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f5008fdb640 1 -- 172.21.15.18:0/917493685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5004109ef0 con 0x555fce7c2970 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f5008fdb640 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 0x555fce7cc610 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8097140 tx=0x7f4ff806d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89875c340589416 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f4fe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f4fe8ff9640 1 -- 172.21.15.18:0/917493685 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff809ae70 con 0x555fce7c2970 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f4fe8ff9640 1 -- 172.21.15.18:0/917493685 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4ff809a770 con 0x555fce7c2970 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f500e60e740 1 -- 172.21.15.18:0/917493685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fce4296c0 con 0x555fce7c2970 2024-02-17T12:49:31.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f4fe8ff9640 1 -- 172.21.15.18:0/917493685 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff809e8c0 con 0x555fce7c2970 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.326+0000 7f500e60e740 10 client.15303 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.327+0000 7f500e60e740 1 -- 172.21.15.18:0/917493685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555fce7282b0 con 0x555fce7c2970 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.327+0000 7f4fe8ff9640 1 -- 172.21.15.18:0/917493685 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555fce4296c0 con 0x555fce7c2970 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.327+0000 7f4fe8ff9640 1 -- 172.21.15.18:0/917493685 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4ff8016030 con 0x555fce7c2970 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.327+0000 7f500e60e740 10 client.15303 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.327+0000 7f500e60e740 10 client.15303 resolve_mds: resolved name 'e' to daemon mds.e 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.327+0000 7f500e60e740 20 client.15303 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.327+0000 7f500e60e740 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x555fce7d89c0 0x555fce7dce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.327+0000 7f500e60e740 4 client.15303 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.327+0000 7f500e60e740 1 -- 172.21.15.18:0/917493685 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fce47a270 con 0x555fce7d89c0 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.327+0000 7f50097dc640 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x555fce7d89c0 0x555fce7dce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f4fe8ff9640 10 client.15303 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f50097dc640 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x555fce7d89c0 0x555fce7dce40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bd2f7bbffc12e89 server_cookie=14b005368d8cbc20 in_seq=0 out_seq=0 2024-02-17T12:49:31.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f4fe8ff9640 1 -- 172.21.15.18:0/917493685 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555fce47a270 con 0x555fce7d89c0 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f4fe8ff9640 10 client.15303 handle_command_reply: tid=0 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f500e60e740 1 client.15303 shutdown 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f500e60e740 1 -- 172.21.15.18:0/917493685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 msgr2=0x555fce7cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f500e60e740 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 0x555fce7cc610 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8097140 tx=0x7f4ff806d830 comp rx=0 tx=0).stop 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f500e60e740 1 -- 172.21.15.18:0/917493685 shutdown_connections 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f500e60e740 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x555fce7d89c0 0x555fce7dce40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f500e60e740 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c7ae0 0x555fce7d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f500e60e740 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 0x555fce7cc610 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f500e60e740 1 --2- 172.21.15.18:0/917493685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 0x555fce7cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f4fe8ff9640 0 client.15303 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f500e60e740 1 -- 172.21.15.18:0/917493685 >> 172.21.15.18:0/917493685 conn(0x555fce7bd4f0 msgr2=0x555fce7bc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f500e60e740 1 -- 172.21.15.18:0/917493685 shutdown_connections 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.328+0000 7f500e60e740 1 -- 172.21.15.18:0/917493685 wait complete. 2024-02-17T12:49:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.329+0000 7f500e60e740 20 client.15303 trim_cache size 0 max 16384 2024-02-17T12:49:31.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f500e60e740 1 Processor -- start 2024-02-17T12:49:31.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f500e60e740 1 -- start start 2024-02-17T12:49:31.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f500e60e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 0x555fce7c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f500e60e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 0x555fce7d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f500e60e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c7ae0 0x555fce7bd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f500e60e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fce6af490 con 0x555fce7c2970 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f500e60e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fce7da9a0 con 0x555fce7c1f80 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f500e60e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fce7ca630 con 0x555fce7c7ae0 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f5008fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 0x555fce7d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f5008fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 0x555fce7d3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36490/0 (socket says 172.21.15.18:36490) 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f5008fdb640 1 -- 172.21.15.18:0/1381828711 learned_addr learned my addr 172.21.15.18:0/1381828711 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f5008fdb640 1 -- 172.21.15.18:0/1381828711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c7ae0 msgr2=0x555fce7bd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.344+0000 7f5008fdb640 1 --2- 172.21.15.18:0/1381828711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c7ae0 0x555fce7bd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f5008fdb640 1 -- 172.21.15.18:0/1381828711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 msgr2=0x555fce7c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f5008fdb640 1 --2- 172.21.15.18:0/1381828711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 0x555fce7c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f5008fdb640 1 -- 172.21.15.18:0/1381828711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fce6c4a20 con 0x555fce7c2970 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f5008fdb640 1 --2- 172.21.15.18:0/1381828711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 0x555fce7d3610 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80057c0 tx=0x7f4ff8096ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0990177d826e07 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f4feaffd640 1 -- 172.21.15.18:0/1381828711 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff809ae70 con 0x555fce7c2970 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f4feaffd640 1 -- 172.21.15.18:0/1381828711 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ff806b030 con 0x555fce7c2970 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f4feaffd640 1 -- 172.21.15.18:0/1381828711 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80a2f10 con 0x555fce7c2970 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f500e60e740 1 -- 172.21.15.18:0/1381828711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 msgr2=0x555fce7d3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f500e60e740 1 --2- 172.21.15.18:0/1381828711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 0x555fce7d3610 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80057c0 tx=0x7f4ff8096ac0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f500e60e740 1 -- 172.21.15.18:0/1381828711 shutdown_connections 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f500e60e740 1 --2- 172.21.15.18:0/1381828711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c7ae0 0x555fce7bd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f500e60e740 1 --2- 172.21.15.18:0/1381828711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c2970 0x555fce7d3610 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f500e60e740 1 --2- 172.21.15.18:0/1381828711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c1f80 0x555fce7c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f500e60e740 1 -- 172.21.15.18:0/1381828711 >> 172.21.15.18:0/1381828711 conn(0x555fce7bd4f0 msgr2=0x555fce7c3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f500e60e740 1 -- 172.21.15.18:0/1381828711 shutdown_connections 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.345+0000 7f500e60e740 1 -- 172.21.15.18:0/1381828711 wait complete. 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f500e60e740 1 -- start start 2024-02-17T12:49:31.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f500e60e740 1 Processor -- start 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f500e60e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c1f80 0x555fce7d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f500e60e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 0x555fce7cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f500e60e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c7ae0 0x555fce7d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f500e60e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fce7d3a00 con 0x555fce7c1f80 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f500e60e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fce7d08e0 con 0x555fce7c7ae0 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f500e60e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fce7d0a70 con 0x555fce7c2970 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f500e60e740 10 client.0 _ll_register_callbacks cb 0x7f4ff40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f50097dc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c1f80 0x555fce7d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f50097dc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c1f80 0x555fce7d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36500/0 (socket says 172.21.15.18:36500) 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f50097dc640 1 -- 172.21.15.18:0/4042190885 learned_addr learned my addr 172.21.15.18:0/4042190885 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f5009fdd640 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c7ae0 0x555fce7d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f5008fdb640 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 0x555fce7cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f50097dc640 1 -- 172.21.15.18:0/4042190885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 msgr2=0x555fce7cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f50097dc640 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 0x555fce7cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f50097dc640 1 -- 172.21.15.18:0/4042190885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c7ae0 msgr2=0x555fce7d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.346+0000 7f50097dc640 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c7ae0 0x555fce7d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f50097dc640 1 -- 172.21.15.18:0/4042190885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fce7282b0 con 0x555fce7c1f80 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f5009fdd640 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c7ae0 0x555fce7d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f5008fdb640 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 0x555fce7cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f50097dc640 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c1f80 0x555fce7d31f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009900 tx=0x7f4ffc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789759484e743cf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f4fe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f4fe8ff9640 1 -- 172.21.15.18:0/4042190885 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc054070 con 0x555fce7c1f80 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f4fe8ff9640 1 -- 172.21.15.18:0/4042190885 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4ffc0042c0 con 0x555fce7c1f80 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f500e60e740 1 -- 172.21.15.18:0/4042190885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ffc004030 con 0x555fce7c1f80 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f4fe8ff9640 1 -- 172.21.15.18:0/4042190885 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ffc004480 con 0x555fce7c1f80 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f500e60e740 10 client.15309 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f500e60e740 1 -- 172.21.15.18:0/4042190885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555fce6ea780 con 0x555fce7c1f80 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f4fe8ff9640 1 -- 172.21.15.18:0/4042190885 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ffc004030 con 0x555fce7c1f80 2024-02-17T12:49:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f4fe8ff9640 1 -- 172.21.15.18:0/4042190885 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4ffc057030 con 0x555fce7c1f80 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f500e60e740 10 client.15309 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f500e60e740 10 client.15309 resolve_mds: resolved name 'e' to daemon mds.e 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f500e60e740 20 client.15309 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f500e60e740 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x555fce833860 0x555fce82fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f500e60e740 4 client.15309 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.347+0000 7f500e60e740 1 -- 172.21.15.18:0/4042190885 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555fce466610 con 0x555fce833860 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f5008fdb640 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x555fce833860 0x555fce82fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f4fe8ff9640 10 client.15309 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f5008fdb640 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x555fce833860 0x555fce82fd60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7963679d74c350c9 server_cookie=f6a81317ce5e9c15 in_seq=0 out_seq=0 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f4fe8ff9640 1 -- 172.21.15.18:0/4042190885 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x555fce466610 con 0x555fce833860 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f4fe8ff9640 10 client.15309 handle_command_reply: tid=0 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f500e60e740 1 client.15309 shutdown 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f500e60e740 1 -- 172.21.15.18:0/4042190885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c1f80 msgr2=0x555fce7d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f500e60e740 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c1f80 0x555fce7d31f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009900 tx=0x7f4ffc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f500e60e740 1 -- 172.21.15.18:0/4042190885 shutdown_connections 2024-02-17T12:49:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f500e60e740 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x555fce833860 0x555fce82fd60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f500e60e740 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555fce7c7ae0 0x555fce7d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f4fe8ff9640 0 client.15309 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:49:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f500e60e740 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555fce7c2970 0x555fce7cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f500e60e740 1 --2- 172.21.15.18:0/4042190885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fce7c1f80 0x555fce7d31f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f500e60e740 1 -- 172.21.15.18:0/4042190885 >> 172.21.15.18:0/4042190885 conn(0x555fce7bd4f0 msgr2=0x555fce7c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f500e60e740 1 -- 172.21.15.18:0/4042190885 shutdown_connections 2024-02-17T12:49:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.348+0000 7f500e60e740 1 -- 172.21.15.18:0/4042190885 wait complete. 2024-02-17T12:49:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500e60e740 20 client.15309 trim_cache size 0 max 16384 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 2, 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24455, 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 102.646237289, 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 25, 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 88, 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 86, 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 130.52952180099999 2024-02-17T12:49:31.349 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:49:31.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500ba40640 1 -- 172.21.15.18:0/2539747157 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4fec0a20c0 msgr2=0x7f4fec0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500ba40640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4fec0a20c0 0x7f4fec0a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x555fce7c31d0 tx=0x7f4ff4009760 comp rx=0 tx=0).stop 2024-02-17T12:49:31.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500ba40640 1 -- 172.21.15.18:0/2539747157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50041023a0 msgr2=0x7f5004073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500ba40640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50041023a0 0x7f5004073570 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800c8c0 tx=0x7f4ff800cd80 comp rx=0 tx=0).stop 2024-02-17T12:49:31.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500ba40640 1 -- 172.21.15.18:0/2539747157 shutdown_connections 2024-02-17T12:49:31.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500ba40640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f4fec0a20c0 0x7f4fec0a4580 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500ba40640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5004102d60 0x7f5004073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500ba40640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50041023a0 0x7f5004073570 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500ba40640 1 --2- 172.21.15.18:0/2539747157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50041011b0 0x7f5004074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500ba40640 1 -- 172.21.15.18:0/2539747157 >> 172.21.15.18:0/2539747157 conn(0x7f50040fc720 msgr2=0x7f5004105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500ba40640 1 -- 172.21.15.18:0/2539747157 shutdown_connections 2024-02-17T12:49:31.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.349+0000 7f500ba40640 1 -- 172.21.15.18:0/2539747157 wait complete. 2024-02-17T12:49:31.352 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-02-17T12:49:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.463+0000 7f9b75577640 1 -- 172.21.15.18:0/781097239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b70100090 msgr2=0x7f9b701004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.463+0000 7f9b75577640 1 --2- 172.21.15.18:0/781097239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b70100090 0x7f9b701004b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009900 tx=0x7f9b6002bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.463+0000 7f9b75577640 1 -- 172.21.15.18:0/781097239 shutdown_connections 2024-02-17T12:49:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.463+0000 7f9b75577640 1 --2- 172.21.15.18:0/781097239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b70101c40 0x7f9b701084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.463+0000 7f9b75577640 1 --2- 172.21.15.18:0/781097239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101280 0x7f9b70101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.463+0000 7f9b75577640 1 --2- 172.21.15.18:0/781097239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b70100090 0x7f9b701004b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.463+0000 7f9b75577640 1 -- 172.21.15.18:0/781097239 >> 172.21.15.18:0/781097239 conn(0x7f9b700fb620 msgr2=0x7f9b700fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.463+0000 7f9b75577640 1 -- 172.21.15.18:0/781097239 shutdown_connections 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.463+0000 7f9b75577640 1 -- 172.21.15.18:0/781097239 wait complete. 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b75577640 1 Processor -- start 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b75577640 1 -- start start 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b75577640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b70100090 0x7f9b70197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b75577640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101280 0x7f9b70197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b75577640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b70101c40 0x7f9b7019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b75577640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7019edf0 con 0x7f9b70100090 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b75577640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7019ef60 con 0x7f9b70101280 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b75577640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7019f120 con 0x7f9b70101c40 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b6effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b70100090 0x7f9b70197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b6effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b70100090 0x7f9b70197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36504/0 (socket says 172.21.15.18:36504) 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b6effd640 1 -- 172.21.15.18:0/776898087 learned_addr learned my addr 172.21.15.18:0/776898087 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b6f7fe640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b70101c40 0x7f9b7019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b67fff640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101280 0x7f9b70197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b6effd640 1 -- 172.21.15.18:0/776898087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b70101c40 msgr2=0x7f9b7019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b6effd640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b70101c40 0x7f9b7019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b6effd640 1 -- 172.21.15.18:0/776898087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101280 msgr2=0x7f9b70197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b6effd640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101280 0x7f9b70197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b6effd640 1 -- 172.21.15.18:0/776898087 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b60009560 con 0x7f9b70100090 2024-02-17T12:49:31.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.464+0000 7f9b67fff640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101280 0x7f9b70197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:31.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.465+0000 7f9b6effd640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b70100090 0x7f9b70197300 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9b60000c00 tx=0x7f9b60004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.465+0000 7f9b6cff9640 1 -- 172.21.15.18:0/776898087 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60004df0 con 0x7f9b70100090 2024-02-17T12:49:31.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.465+0000 7f9b75577640 1 -- 172.21.15.18:0/776898087 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7019f3a0 con 0x7f9b70100090 2024-02-17T12:49:31.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.465+0000 7f9b6cff9640 1 -- 172.21.15.18:0/776898087 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b60005430 con 0x7f9b70100090 2024-02-17T12:49:31.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.465+0000 7f9b6cff9640 1 -- 172.21.15.18:0/776898087 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6003e450 con 0x7f9b70100090 2024-02-17T12:49:31.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.465+0000 7f9b75577640 1 -- 172.21.15.18:0/776898087 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7019f900 con 0x7f9b70100090 2024-02-17T12:49:31.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.467+0000 7f9b6cff9640 1 -- 172.21.15.18:0/776898087 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9b600055a0 con 0x7f9b70100090 2024-02-17T12:49:31.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.467+0000 7f9b78228740 1 Processor -- start 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.467+0000 7f9b78228740 1 -- start start 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.467+0000 7f9b78228740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b23f80 0x55c931b243a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.467+0000 7f9b78228740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 0x55c931b29470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.467+0000 7f9b78228740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b29ae0 0x55c931b2bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.467+0000 7f9b78228740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c931adfcd0 con 0x55c931b24970 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.467+0000 7f9b78228740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c931a92f20 con 0x55c931b23f80 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.467+0000 7f9b78228740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c931a90450 con 0x55c931b29ae0 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.467+0000 7f9b6cff9640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9b5c0a20c0 0x7f9b5c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.467+0000 7f9b6effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 0x55c931b29470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b6effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 0x55c931b29470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36516/0 (socket says 172.21.15.18:36516) 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b6effd640 1 -- 172.21.15.18:0/2421872470 learned_addr learned my addr 172.21.15.18:0/2421872470 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b67fff640 1 --2- 172.21.15.18:0/2421872470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b29ae0 0x55c931b2bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b6f7fe640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9b5c0a20c0 0x7f9b5c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b6f7fe640 1 --2- 172.21.15.18:0/2421872470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b23f80 0x55c931b243a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b6cff9640 1 -- 172.21.15.18:0/776898087 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9b600f0000 con 0x7f9b70100090 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b6effd640 1 -- 172.21.15.18:0/2421872470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b29ae0 msgr2=0x55c931b2bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b6effd640 1 --2- 172.21.15.18:0/2421872470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b29ae0 0x55c931b2bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b6effd640 1 -- 172.21.15.18:0/2421872470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b23f80 msgr2=0x55c931b243a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b6effd640 1 --2- 172.21.15.18:0/2421872470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b23f80 0x55c931b243a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b6effd640 1 -- 172.21.15.18:0/2421872470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c931a40f20 con 0x55c931b24970 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b6f7fe640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9b5c0a20c0 0x7f9b5c0a4580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9b68013a90 tx=0x7f9b6800bc50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b6effd640 1 --2- 172.21.15.18:0/2421872470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 0x55c931b29470 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9b60048000 tx=0x7f9b60003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf43a5c62790446 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b657fa640 1 -- 172.21.15.18:0/2421872470 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b600bc6c0 con 0x55c931b24970 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b657fa640 1 -- 172.21.15.18:0/2421872470 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b600bd460 con 0x55c931b24970 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.468+0000 7f9b657fa640 1 -- 172.21.15.18:0/2421872470 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b600c5070 con 0x55c931b24970 2024-02-17T12:49:31.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.469+0000 7f9b78228740 1 -- 172.21.15.18:0/2421872470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 msgr2=0x55c931b29470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.469+0000 7f9b78228740 1 --2- 172.21.15.18:0/2421872470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 0x55c931b29470 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9b60048000 tx=0x7f9b60003a20 comp rx=0 tx=0).stop 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.469+0000 7f9b78228740 1 -- 172.21.15.18:0/2421872470 shutdown_connections 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.469+0000 7f9b78228740 1 --2- 172.21.15.18:0/2421872470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b29ae0 0x55c931b2bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.469+0000 7f9b78228740 1 --2- 172.21.15.18:0/2421872470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 0x55c931b29470 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.469+0000 7f9b78228740 1 --2- 172.21.15.18:0/2421872470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b23f80 0x55c931b243a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.469+0000 7f9b78228740 1 -- 172.21.15.18:0/2421872470 >> 172.21.15.18:0/2421872470 conn(0x55c931b1f4f0 msgr2=0x55c931b1f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.469+0000 7f9b78228740 1 -- 172.21.15.18:0/2421872470 shutdown_connections 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.469+0000 7f9b78228740 1 -- 172.21.15.18:0/2421872470 wait complete. 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.469+0000 7f9b78228740 1 -- start start 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.469+0000 7f9b78228740 1 Processor -- start 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b78228740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 0x55c931b2df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b78228740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b24970 0x55c931b2e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b78228740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 0x55c931b354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b78228740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c931a8bc30 con 0x55c931b29ae0 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b78228740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c931a30d30 con 0x55c931b24970 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b78228740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93175f310 con 0x55c931b23f80 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b78228740 10 client.0 _ll_register_callbacks cb 0x55c931a38d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b6f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 0x55c931b354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b6f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 0x55c931b354b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36532/0 (socket says 172.21.15.18:36532) 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b6f7fe640 1 -- 172.21.15.18:0/1070349455 learned_addr learned my addr 172.21.15.18:0/1070349455 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b6effd640 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 0x55c931b2df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b6f7fe640 1 -- 172.21.15.18:0/1070349455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 msgr2=0x55c931b2df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b6f7fe640 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 0x55c931b2df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b67fff640 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b24970 0x55c931b2e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b6f7fe640 1 -- 172.21.15.18:0/1070349455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b24970 msgr2=0x55c931b2e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b6f7fe640 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b24970 0x55c931b2e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b6f7fe640 1 -- 172.21.15.18:0/1070349455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7019f900 con 0x55c931b29ae0 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b6f7fe640 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 0x55c931b354b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9b68010dc0 tx=0x7f9b68007220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a31629b77bdc89f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b567fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b567fc640 1 -- 172.21.15.18:0/1070349455 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68028660 con 0x55c931b29ae0 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b567fc640 1 -- 172.21.15.18:0/1070349455 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9b680287c0 con 0x55c931b29ae0 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b78228740 1 -- 172.21.15.18:0/1070349455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c93178b6c0 con 0x55c931b29ae0 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.470+0000 7f9b567fc640 1 -- 172.21.15.18:0/1070349455 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b68004b20 con 0x55c931b29ae0 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.471+0000 7f9b78228740 10 client.15321 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.471+0000 7f9b78228740 1 -- 172.21.15.18:0/1070349455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c931a8a2b0 con 0x55c931b29ae0 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.471+0000 7f9b567fc640 1 -- 172.21.15.18:0/1070349455 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c93178b6c0 con 0x55c931b29ae0 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.471+0000 7f9b567fc640 1 -- 172.21.15.18:0/1070349455 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9b6801f450 con 0x55c931b29ae0 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.471+0000 7f9b78228740 10 client.15321 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.471+0000 7f9b78228740 10 client.15321 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-17T12:49:31.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.471+0000 7f9b78228740 20 client.15321 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.471+0000 7f9b78228740 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c931b3a910 0x55c931b39b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.471+0000 7f9b78228740 4 client.15321 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.471+0000 7f9b78228740 1 -- 172.21.15.18:0/1070349455 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9317dc270 con 0x55c931b3a910 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.471+0000 7f9b6effd640 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c931b3a910 0x55c931b39b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b567fc640 10 client.15321 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b6effd640 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c931b3a910 0x55c931b39b80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55c19140d7499e5a server_cookie=4e33fb9234c3c504 in_seq=0 out_seq=0 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b567fc640 1 -- 172.21.15.18:0/1070349455 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c9317dc270 con 0x55c931b3a910 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b567fc640 10 client.15321 handle_command_reply: tid=0 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b78228740 1 client.15321 shutdown 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b78228740 1 -- 172.21.15.18:0/1070349455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 msgr2=0x55c931b354b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b78228740 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 0x55c931b354b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9b68010dc0 tx=0x7f9b68007220 comp rx=0 tx=0).stop 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b78228740 1 -- 172.21.15.18:0/1070349455 shutdown_connections 2024-02-17T12:49:31.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b78228740 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c931b3a910 0x55c931b39b80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b78228740 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 0x55c931b354b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b78228740 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b24970 0x55c931b2e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b78228740 1 --2- 172.21.15.18:0/1070349455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 0x55c931b2df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.472+0000 7f9b567fc640 0 client.15321 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:49:31.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.473+0000 7f9b78228740 1 -- 172.21.15.18:0/1070349455 >> 172.21.15.18:0/1070349455 conn(0x55c931b1f4f0 msgr2=0x55c931b1e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.473+0000 7f9b78228740 1 -- 172.21.15.18:0/1070349455 shutdown_connections 2024-02-17T12:49:31.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.473+0000 7f9b78228740 1 -- 172.21.15.18:0/1070349455 wait complete. 2024-02-17T12:49:31.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.473+0000 7f9b78228740 20 client.15321 trim_cache size 0 max 16384 2024-02-17T12:49:31.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.488+0000 7f9b78228740 1 Processor -- start 2024-02-17T12:49:31.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.488+0000 7f9b78228740 1 -- start start 2024-02-17T12:49:31.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.488+0000 7f9b78228740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 0x55c931b39b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.488+0000 7f9b78228740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b24970 0x55c931b35470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.488+0000 7f9b78228740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 0x55c931b25e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b78228740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c931a11490 con 0x55c931b29ae0 2024-02-17T12:49:31.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b78228740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c931b3b1a0 con 0x55c931b24970 2024-02-17T12:49:31.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b78228740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c931b36130 con 0x55c931b23f80 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b6f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 0x55c931b25e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b6f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 0x55c931b25e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36534/0 (socket says 172.21.15.18:36534) 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b6f7fe640 1 -- 172.21.15.18:0/4259929023 learned_addr learned my addr 172.21.15.18:0/4259929023 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b6effd640 1 --2- 172.21.15.18:0/4259929023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 0x55c931b39b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b67fff640 1 --2- 172.21.15.18:0/4259929023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b24970 0x55c931b35470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b6f7fe640 1 -- 172.21.15.18:0/4259929023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 msgr2=0x55c931b39b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b6f7fe640 1 --2- 172.21.15.18:0/4259929023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 0x55c931b39b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b6f7fe640 1 -- 172.21.15.18:0/4259929023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b24970 msgr2=0x55c931b35470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b6f7fe640 1 --2- 172.21.15.18:0/4259929023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b24970 0x55c931b35470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b6f7fe640 1 -- 172.21.15.18:0/4259929023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c931a26a20 con 0x55c931b29ae0 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b67fff640 1 --2- 172.21.15.18:0/4259929023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b24970 0x55c931b35470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b6f7fe640 1 --2- 172.21.15.18:0/4259929023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 0x55c931b25e70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9b68005c40 tx=0x7f9b68013500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338414d3fad7396c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b64ff9640 1 -- 172.21.15.18:0/4259929023 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6801f650 con 0x55c931b29ae0 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b64ff9640 1 -- 172.21.15.18:0/4259929023 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b68019040 con 0x55c931b29ae0 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.489+0000 7f9b64ff9640 1 -- 172.21.15.18:0/4259929023 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68004020 con 0x55c931b29ae0 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 -- 172.21.15.18:0/4259929023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 msgr2=0x55c931b25e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 --2- 172.21.15.18:0/4259929023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 0x55c931b25e70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9b68005c40 tx=0x7f9b68013500 comp rx=0 tx=0).stop 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 -- 172.21.15.18:0/4259929023 shutdown_connections 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 --2- 172.21.15.18:0/4259929023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b29ae0 0x55c931b25e70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 --2- 172.21.15.18:0/4259929023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b24970 0x55c931b35470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 --2- 172.21.15.18:0/4259929023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 0x55c931b39b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 -- 172.21.15.18:0/4259929023 >> 172.21.15.18:0/4259929023 conn(0x55c931b1f4f0 msgr2=0x55c931b1f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 -- 172.21.15.18:0/4259929023 shutdown_connections 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 -- 172.21.15.18:0/4259929023 wait complete. 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 -- start start 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 Processor -- start 2024-02-17T12:49:31.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 0x55c931b352b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 0x55c931b31d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b29ae0 0x55c931b322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c931b32900 con 0x55c931b24970 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c931b90510 con 0x55c931b29ae0 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c931b90650 con 0x55c931b23f80 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.490+0000 7f9b78228740 10 client.0 _ll_register_callbacks cb 0x7f9b600b93d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b67fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 0x55c931b31d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b67fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 0x55c931b31d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36550/0 (socket says 172.21.15.18:36550) 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b67fff640 1 -- 172.21.15.18:0/1947865457 learned_addr learned my addr 172.21.15.18:0/1947865457 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b6effd640 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 0x55c931b352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b67fff640 1 -- 172.21.15.18:0/1947865457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 msgr2=0x55c931b352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b6f7fe640 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b29ae0 0x55c931b322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b67fff640 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 0x55c931b352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b67fff640 1 -- 172.21.15.18:0/1947865457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b29ae0 msgr2=0x55c931b322c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b67fff640 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b29ae0 0x55c931b322c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b67fff640 1 -- 172.21.15.18:0/1947865457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c931a8a2b0 con 0x55c931b24970 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b67fff640 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 0x55c931b31d80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005fd0 tx=0x7f9b580137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea4f10923069ede server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b567fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b567fc640 1 -- 172.21.15.18:0/1947865457 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b58011870 con 0x55c931b24970 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b567fc640 1 -- 172.21.15.18:0/1947865457 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9b580119d0 con 0x55c931b24970 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b78228740 1 -- 172.21.15.18:0/1947865457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c931b3b2e0 con 0x55c931b24970 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b567fc640 1 -- 172.21.15.18:0/1947865457 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b58011b90 con 0x55c931b24970 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b78228740 10 client.15327 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b78228740 1 -- 172.21.15.18:0/1947865457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c931a4c780 con 0x55c931b24970 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.491+0000 7f9b567fc640 1 -- 172.21.15.18:0/1947865457 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c931b3b2e0 con 0x55c931b24970 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b567fc640 1 -- 172.21.15.18:0/1947865457 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9b58025050 con 0x55c931b24970 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b78228740 10 client.15327 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b78228740 10 client.15327 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b78228740 20 client.15327 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b78228740 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c931b3a910 0x55c931b91eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b78228740 4 client.15327 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b78228740 1 -- 172.21.15.18:0/1947865457 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f9b600bd460 con 0x55c931b3a910 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b6effd640 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c931b3a910 0x55c931b91eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b6effd640 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c931b3a910 0x55c931b91eb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=914ffc0e6f1ca84a server_cookie=403a8ad085f61a40 in_seq=0 out_seq=0 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b567fc640 10 client.15327 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b567fc640 1 -- 172.21.15.18:0/1947865457 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7f9b600bd460 con 0x55c931b3a910 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b567fc640 10 client.15327 handle_command_reply: tid=0 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.492+0000 7f9b78228740 1 client.15327 shutdown 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b78228740 1 -- 172.21.15.18:0/1947865457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 msgr2=0x55c931b31d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b78228740 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 0x55c931b31d80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005fd0 tx=0x7f9b580137d0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b78228740 1 -- 172.21.15.18:0/1947865457 shutdown_connections 2024-02-17T12:49:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b78228740 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c931b3a910 0x55c931b91eb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b78228740 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c931b29ae0 0x55c931b322c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b567fc640 0 client.15327 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:49:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b78228740 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c931b24970 0x55c931b31d80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b78228740 1 --2- 172.21.15.18:0/1947865457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c931b23f80 0x55c931b352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b78228740 1 -- 172.21.15.18:0/1947865457 >> 172.21.15.18:0/1947865457 conn(0x55c931b1f4f0 msgr2=0x55c931b25e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b78228740 1 -- 172.21.15.18:0/1947865457 shutdown_connections 2024-02-17T12:49:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b78228740 1 -- 172.21.15.18:0/1947865457 wait complete. 2024-02-17T12:49:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b78228740 20 client.15327 trim_cache size 0 max 16384 2024-02-17T12:49:31.492 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:49:31.493 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:49:31.493 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 1, 2024-02-17T12:49:31.493 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24476, 2024-02-17T12:49:31.493 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-02-17T12:49:31.493 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-02-17T12:49:31.493 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-02-17T12:49:31.493 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 103.419230156, 2024-02-17T12:49:31.493 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 25, 2024-02-17T12:49:31.493 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 88, 2024-02-17T12:49:31.493 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 86, 2024-02-17T12:49:31.493 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 122.620880038 2024-02-17T12:49:31.493 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:49:31.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b75577640 1 -- 172.21.15.18:0/776898087 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9b5c0a20c0 msgr2=0x7f9b5c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b75577640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9b5c0a20c0 0x7f9b5c0a4580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9b68013a90 tx=0x7f9b6800bc50 comp rx=0 tx=0).stop 2024-02-17T12:49:31.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b75577640 1 -- 172.21.15.18:0/776898087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b70100090 msgr2=0x7f9b70197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b75577640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b70100090 0x7f9b70197300 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9b60000c00 tx=0x7f9b60004270 comp rx=0 tx=0).stop 2024-02-17T12:49:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b75577640 1 -- 172.21.15.18:0/776898087 shutdown_connections 2024-02-17T12:49:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b75577640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f9b5c0a20c0 0x7f9b5c0a4580 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b75577640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b70101c40 0x7f9b7019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b75577640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101280 0x7f9b70197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b75577640 1 --2- 172.21.15.18:0/776898087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b70100090 0x7f9b70197300 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b75577640 1 -- 172.21.15.18:0/776898087 >> 172.21.15.18:0/776898087 conn(0x7f9b700fb620 msgr2=0x7f9b70069840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b75577640 1 -- 172.21.15.18:0/776898087 shutdown_connections 2024-02-17T12:49:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.493+0000 7f9b75577640 1 -- 172.21.15.18:0/776898087 wait complete. 2024-02-17T12:49:31.497 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-17T12:49:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.608+0000 7fd240098640 1 -- 172.21.15.18:0/2378874115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238102380 msgr2=0x7fd238102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.608+0000 7fd240098640 1 --2- 172.21.15.18:0/2378874115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238102380 0x7fd238102800 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd228009900 tx=0x7fd22802bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.608+0000 7fd240098640 1 -- 172.21.15.18:0/2378874115 shutdown_connections 2024-02-17T12:49:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.608+0000 7fd240098640 1 --2- 172.21.15.18:0/2378874115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd238102d40 0x7fd2381095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.608+0000 7fd240098640 1 --2- 172.21.15.18:0/2378874115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238102380 0x7fd238102800 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.608+0000 7fd240098640 1 --2- 172.21.15.18:0/2378874115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd238101190 0x7fd2381015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.608+0000 7fd240098640 1 -- 172.21.15.18:0/2378874115 >> 172.21.15.18:0/2378874115 conn(0x7fd2380fc720 msgr2=0x7fd2380feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.608+0000 7fd240098640 1 -- 172.21.15.18:0/2378874115 shutdown_connections 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.608+0000 7fd240098640 1 -- 172.21.15.18:0/2378874115 wait complete. 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd240098640 1 Processor -- start 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd240098640 1 -- start start 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd240098640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd238101190 0x7fd238197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd240098640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238102380 0x7fd2381977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd240098640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd238102d40 0x7fd23819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd240098640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23819ed50 con 0x7fd238102380 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd240098640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23819eec0 con 0x7fd238102d40 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd240098640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23819f080 con 0x7fd238101190 2024-02-17T12:49:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd23d633640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238102380 0x7fd2381977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd23d633640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238102380 0x7fd2381977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36558/0 (socket says 172.21.15.18:36558) 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd23d633640 1 -- 172.21.15.18:0/1700634596 learned_addr learned my addr 172.21.15.18:0/1700634596 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd23de34640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd238101190 0x7fd238197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.609+0000 7fd23e635640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd238102d40 0x7fd23819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.610+0000 7fd23d633640 1 -- 172.21.15.18:0/1700634596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd238101190 msgr2=0x7fd238197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.610+0000 7fd23d633640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd238101190 0x7fd238197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.610+0000 7fd23d633640 1 -- 172.21.15.18:0/1700634596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd238102d40 msgr2=0x7fd23819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.610+0000 7fd23d633640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd238102d40 0x7fd23819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.610+0000 7fd23d633640 1 -- 172.21.15.18:0/1700634596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd228009560 con 0x7fd238102380 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.610+0000 7fd23e635640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd238102d40 0x7fd23819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.610+0000 7fd23d633640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238102380 0x7fd2381977a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd228006010 tx=0x7fd2280040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.610+0000 7fd22effd640 1 -- 172.21.15.18:0/1700634596 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2280044c0 con 0x7fd238102380 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.610+0000 7fd240098640 1 -- 172.21.15.18:0/1700634596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23819f300 con 0x7fd238102380 2024-02-17T12:49:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.610+0000 7fd22effd640 1 -- 172.21.15.18:0/1700634596 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2280052d0 con 0x7fd238102380 2024-02-17T12:49:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.610+0000 7fd22effd640 1 -- 172.21.15.18:0/1700634596 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228003e10 con 0x7fd238102380 2024-02-17T12:49:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.610+0000 7fd240098640 1 -- 172.21.15.18:0/1700634596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23819f810 con 0x7fd238102380 2024-02-17T12:49:31.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.612+0000 7fd22effd640 1 -- 172.21.15.18:0/1700634596 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd228005440 con 0x7fd238102380 2024-02-17T12:49:31.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.612+0000 7fd242c60740 1 Processor -- start 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.612+0000 7fd242c60740 1 -- start start 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd242c60740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd242c60740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a0970 0x5555844a5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd242c60740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a5ae0 0x5555844a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd242c60740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558445bcd0 con 0x55558449ff80 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd242c60740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558440ef20 con 0x5555844a0970 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd242c60740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558440c450 con 0x5555844a5ae0 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd23e635640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd23e635640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36564/0 (socket says 172.21.15.18:36564) 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd23e635640 1 -- 172.21.15.18:0/2372434142 learned_addr learned my addr 172.21.15.18:0/2372434142 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd23de34640 1 --2- 172.21.15.18:0/2372434142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a0970 0x5555844a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd23d633640 1 --2- 172.21.15.18:0/2372434142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a5ae0 0x5555844a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd22effd640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd2240a20c0 0x7fd2240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd23e635640 1 -- 172.21.15.18:0/2372434142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a5ae0 msgr2=0x5555844a7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd23e635640 1 --2- 172.21.15.18:0/2372434142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a5ae0 0x5555844a7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd23e635640 1 -- 172.21.15.18:0/2372434142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a0970 msgr2=0x5555844a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd23e635640 1 --2- 172.21.15.18:0/2372434142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a0970 0x5555844a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd23e635640 1 -- 172.21.15.18:0/2372434142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555843bcf20 con 0x55558449ff80 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd22effd640 1 -- 172.21.15.18:0/1700634596 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd2280f0060 con 0x7fd238102380 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd23e635640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd2240a20c0 0x7fd2240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd23e635640 1 --2- 172.21.15.18:0/2372434142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844a03a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd23000f4b0 tx=0x7fd23000f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b4437d32a72b2a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.613+0000 7fd223fff640 1 -- 172.21.15.18:0/2372434142 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230012590 con 0x55558449ff80 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.614+0000 7fd223fff640 1 -- 172.21.15.18:0/2372434142 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd23001cdd0 con 0x55558449ff80 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.614+0000 7fd223fff640 1 -- 172.21.15.18:0/2372434142 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230025d70 con 0x55558449ff80 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.614+0000 7fd242c60740 1 -- 172.21.15.18:0/2372434142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 msgr2=0x5555844a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.614+0000 7fd242c60740 1 --2- 172.21.15.18:0/2372434142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844a03a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd23000f4b0 tx=0x7fd23000f970 comp rx=0 tx=0).stop 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.614+0000 7fd23e635640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd2240a20c0 0x7fd2240a4580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd230020000 tx=0x7fd23001e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.614+0000 7fd242c60740 1 -- 172.21.15.18:0/2372434142 shutdown_connections 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.614+0000 7fd242c60740 1 --2- 172.21.15.18:0/2372434142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a5ae0 0x5555844a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.614+0000 7fd242c60740 1 --2- 172.21.15.18:0/2372434142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a0970 0x5555844a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.614+0000 7fd242c60740 1 --2- 172.21.15.18:0/2372434142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844a03a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.614+0000 7fd242c60740 1 -- 172.21.15.18:0/2372434142 >> 172.21.15.18:0/2372434142 conn(0x55558449b4f0 msgr2=0x55558449b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.614+0000 7fd242c60740 1 -- 172.21.15.18:0/2372434142 shutdown_connections 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.614+0000 7fd242c60740 1 -- 172.21.15.18:0/2372434142 wait complete. 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd242c60740 1 -- start start 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd242c60740 1 Processor -- start 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd242c60740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd242c60740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a0970 0x5555844aa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd242c60740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a5ae0 0x5555844b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd242c60740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555584407c30 con 0x55558449ff80 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd242c60740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555843acd30 con 0x5555844a0970 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd242c60740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555840db310 con 0x5555844a5ae0 2024-02-17T12:49:31.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd23de34640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd242c60740 10 client.0 _ll_register_callbacks cb 0x5555843b4d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd23de34640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844aa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36570/0 (socket says 172.21.15.18:36570) 2024-02-17T12:49:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd23de34640 1 -- 172.21.15.18:0/3739191156 learned_addr learned my addr 172.21.15.18:0/3739191156 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd23e635640 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a5ae0 0x5555844b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd23d633640 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a0970 0x5555844aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd23de34640 1 -- 172.21.15.18:0/3739191156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a5ae0 msgr2=0x5555844b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd23de34640 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a5ae0 0x5555844b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd23de34640 1 -- 172.21.15.18:0/3739191156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a0970 msgr2=0x5555844aa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd23de34640 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a0970 0x5555844aa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.615+0000 7fd23de34640 1 -- 172.21.15.18:0/3739191156 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd230009c90 con 0x55558449ff80 2024-02-17T12:49:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd23d633640 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a0970 0x5555844aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd23de34640 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844aa1c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd23400bf20 tx=0x7fd2340118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c616b946e3664c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd2217fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd2217fa640 1 -- 172.21.15.18:0/3739191156 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234016070 con 0x55558449ff80 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd2217fa640 1 -- 172.21.15.18:0/3739191156 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd234005020 con 0x55558449ff80 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd242c60740 1 -- 172.21.15.18:0/3739191156 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555841076c0 con 0x55558449ff80 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd2217fa640 1 -- 172.21.15.18:0/3739191156 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2340051e0 con 0x55558449ff80 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd242c60740 10 client.15339 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd242c60740 1 -- 172.21.15.18:0/3739191156 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5555844062b0 con 0x55558449ff80 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd2217fa640 1 -- 172.21.15.18:0/3739191156 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5555841076c0 con 0x55558449ff80 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd2217fa640 1 -- 172.21.15.18:0/3739191156 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd2340107d0 con 0x55558449ff80 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd242c60740 10 client.15339 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd242c60740 10 client.15339 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd242c60740 20 client.15339 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd242c60740 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555844b6b80 0x5555844b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd242c60740 4 client.15339 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:49:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.616+0000 7fd242c60740 1 -- 172.21.15.18:0/3739191156 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555584158270 con 0x5555844b6b80 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.617+0000 7fd23d633640 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555844b6b80 0x5555844b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.617+0000 7fd23d633640 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555844b6b80 0x5555844b5dc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e0d3fd0377ce6b8 server_cookie=87a5d371112613f in_seq=0 out_seq=0 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.617+0000 7fd2217fa640 10 client.15339 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd2217fa640 1 -- 172.21.15.18:0/3739191156 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555584158270 con 0x5555844b6b80 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd2217fa640 10 client.15339 handle_command_reply: tid=0 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd242c60740 1 client.15339 shutdown 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd242c60740 1 -- 172.21.15.18:0/3739191156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 msgr2=0x5555844aa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd242c60740 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844aa1c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd23400bf20 tx=0x7fd2340118d0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd242c60740 1 -- 172.21.15.18:0/3739191156 shutdown_connections 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd242c60740 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555844b6b80 0x5555844b5dc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd242c60740 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a5ae0 0x5555844b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd2217fa640 0 client.15339 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd242c60740 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a0970 0x5555844aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd242c60740 1 --2- 172.21.15.18:0/3739191156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844aa1c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd242c60740 1 -- 172.21.15.18:0/3739191156 >> 172.21.15.18:0/3739191156 conn(0x55558449b4f0 msgr2=0x55558449a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd242c60740 1 -- 172.21.15.18:0/3739191156 shutdown_connections 2024-02-17T12:49:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd242c60740 1 -- 172.21.15.18:0/3739191156 wait complete. 2024-02-17T12:49:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.618+0000 7fd242c60740 20 client.15339 trim_cache size 0 max 16384 2024-02-17T12:49:31.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd242c60740 1 Processor -- start 2024-02-17T12:49:31.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd242c60740 1 -- start start 2024-02-17T12:49:31.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd242c60740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd242c60740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a0970 0x5555844a2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd242c60740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a5ae0 0x5555844b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd242c60740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558438d490 con 0x55558449ff80 2024-02-17T12:49:31.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd242c60740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558449fb40 con 0x5555844a5ae0 2024-02-17T12:49:31.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd242c60740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555844b49c0 con 0x5555844a0970 2024-02-17T12:49:31.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd23de34640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd23de34640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36580/0 (socket says 172.21.15.18:36580) 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd23de34640 1 -- 172.21.15.18:0/1957179638 learned_addr learned my addr 172.21.15.18:0/1957179638 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd23de34640 1 -- 172.21.15.18:0/1957179638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a0970 msgr2=0x5555844a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd23de34640 1 --2- 172.21.15.18:0/1957179638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a0970 0x5555844a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd23de34640 1 -- 172.21.15.18:0/1957179638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a5ae0 msgr2=0x5555844b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd23de34640 1 --2- 172.21.15.18:0/1957179638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a5ae0 0x5555844b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd23de34640 1 -- 172.21.15.18:0/1957179638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555843a2a20 con 0x55558449ff80 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd23de34640 1 --2- 172.21.15.18:0/1957179638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844a1ec0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd234000c00 tx=0x7fd234005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4821965d89a8a01d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.634+0000 7fd2237fe640 1 -- 172.21.15.18:0/1957179638 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23402c050 con 0x55558449ff80 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.635+0000 7fd2237fe640 1 -- 172.21.15.18:0/1957179638 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd234010dc0 con 0x55558449ff80 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.635+0000 7fd2237fe640 1 -- 172.21.15.18:0/1957179638 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234016040 con 0x55558449ff80 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.635+0000 7fd242c60740 1 -- 172.21.15.18:0/1957179638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 msgr2=0x5555844a1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.635+0000 7fd242c60740 1 --2- 172.21.15.18:0/1957179638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844a1ec0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd234000c00 tx=0x7fd234005bd0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.635+0000 7fd242c60740 1 -- 172.21.15.18:0/1957179638 shutdown_connections 2024-02-17T12:49:31.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.635+0000 7fd242c60740 1 --2- 172.21.15.18:0/1957179638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a5ae0 0x5555844b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.635+0000 7fd242c60740 1 --2- 172.21.15.18:0/1957179638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555844a0970 0x5555844a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.635+0000 7fd242c60740 1 --2- 172.21.15.18:0/1957179638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55558449ff80 0x5555844a1ec0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.635+0000 7fd242c60740 1 -- 172.21.15.18:0/1957179638 >> 172.21.15.18:0/1957179638 conn(0x55558449b4f0 msgr2=0x55558449b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.635+0000 7fd242c60740 1 -- 172.21.15.18:0/1957179638 shutdown_connections 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.635+0000 7fd242c60740 1 -- 172.21.15.18:0/1957179638 wait complete. 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.635+0000 7fd242c60740 1 -- start start 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd242c60740 1 Processor -- start 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd242c60740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55558449ff80 0x5555844ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd242c60740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555844a0970 0x5555844aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd242c60740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a5ae0 0x55558450c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd242c60740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555844af170 con 0x5555844a0970 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd242c60740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558450ccc0 con 0x5555844a5ae0 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd242c60740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558450ce00 con 0x55558449ff80 2024-02-17T12:49:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd242c60740 10 client.0 _ll_register_callbacks cb 0x7fd2300099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd23d633640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555844a0970 0x5555844aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd23d633640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555844a0970 0x5555844aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36586/0 (socket says 172.21.15.18:36586) 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd23d633640 1 -- 172.21.15.18:0/2976571230 learned_addr learned my addr 172.21.15.18:0/2976571230 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd23e635640 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a5ae0 0x55558450c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd23de34640 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55558449ff80 0x5555844ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd23d633640 1 -- 172.21.15.18:0/2976571230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55558449ff80 msgr2=0x5555844ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd23d633640 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55558449ff80 0x5555844ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd23d633640 1 -- 172.21.15.18:0/2976571230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a5ae0 msgr2=0x55558450c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd23d633640 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a5ae0 0x55558450c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd23d633640 1 -- 172.21.15.18:0/2976571230 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555844062b0 con 0x5555844a0970 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd23d633640 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555844a0970 0x5555844aeb60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd2280b9730 tx=0x7fd2280bcc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9eb1ec4eeb86f6e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd2217fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.636+0000 7fd2217fa640 1 -- 172.21.15.18:0/2976571230 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2280c5500 con 0x5555844a0970 2024-02-17T12:49:31.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd2217fa640 1 -- 172.21.15.18:0/2976571230 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd2280edc40 con 0x5555844a0970 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd242c60740 1 -- 172.21.15.18:0/2976571230 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555844b7bb0 con 0x5555844a0970 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd2217fa640 1 -- 172.21.15.18:0/2976571230 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2280035f0 con 0x5555844a0970 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd242c60740 10 client.15345 fetch_fsmap learned FSMap version 25 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd242c60740 1 -- 172.21.15.18:0/2976571230 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5555843c8780 con 0x5555844a0970 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd2217fa640 1 -- 172.21.15.18:0/2976571230 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5555844b7bb0 con 0x5555844a0970 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd2217fa640 1 -- 172.21.15.18:0/2976571230 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd22803c070 con 0x5555844a0970 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd242c60740 10 client.15345 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd242c60740 10 client.15345 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd242c60740 20 client.15345 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd242c60740 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555584511e40 0x55558450e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd242c60740 4 client.15345 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd242c60740 1 -- 172.21.15.18:0/2976571230 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fd23001cdd0 con 0x555584511e40 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.637+0000 7fd23de34640 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555584511e40 0x55558450e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd23de34640 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555584511e40 0x55558450e420 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0bfeec32eac447 server_cookie=9f6ebe54416ab102 in_seq=0 out_seq=0 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd2217fa640 10 client.15345 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd2217fa640 1 -- 172.21.15.18:0/2976571230 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7fd23001cdd0 con 0x555584511e40 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd2217fa640 10 client.15345 handle_command_reply: tid=0 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd242c60740 1 client.15345 shutdown 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd242c60740 1 -- 172.21.15.18:0/2976571230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555844a0970 msgr2=0x5555844aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd242c60740 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555844a0970 0x5555844aeb60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd2280b9730 tx=0x7fd2280bcc50 comp rx=0 tx=0).stop 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd242c60740 1 -- 172.21.15.18:0/2976571230 shutdown_connections 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd242c60740 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555584511e40 0x55558450e420 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd242c60740 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555844a5ae0 0x55558450c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd242c60740 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555844a0970 0x5555844aeb60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd2217fa640 0 client.15345 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd242c60740 1 --2- 172.21.15.18:0/2976571230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55558449ff80 0x5555844ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd242c60740 1 -- 172.21.15.18:0/2976571230 >> 172.21.15.18:0/2976571230 conn(0x55558449b4f0 msgr2=0x5555844a1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd242c60740 1 -- 172.21.15.18:0/2976571230 shutdown_connections 2024-02-17T12:49:31.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd242c60740 1 -- 172.21.15.18:0/2976571230 wait complete. 2024-02-17T12:49:31.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.638+0000 7fd242c60740 20 client.15345 trim_cache size 0 max 16384 2024-02-17T12:49:31.638 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:49:31.638 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "7fca71fe-cd91-11ee-95ba-87774f69a715", 2024-02-17T12:49:31.638 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 0, 2024-02-17T12:49:31.638 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24512, 2024-02-17T12:49:31.638 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-02-17T12:49:31.638 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-02-17T12:49:31.638 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-02-17T12:49:31.638 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 105.59109661799999, 2024-02-17T12:49:31.639 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 25, 2024-02-17T12:49:31.639 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 88, 2024-02-17T12:49:31.639 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 86, 2024-02-17T12:49:31.639 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 111.71289478200001 2024-02-17T12:49:31.639 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:49:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.639+0000 7fd240098640 1 -- 172.21.15.18:0/1700634596 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd2240a20c0 msgr2=0x7fd2240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.639+0000 7fd240098640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd2240a20c0 0x7fd2240a4580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd230020000 tx=0x7fd23001e000 comp rx=0 tx=0).stop 2024-02-17T12:49:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.639+0000 7fd240098640 1 -- 172.21.15.18:0/1700634596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238102380 msgr2=0x7fd2381977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.639+0000 7fd240098640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238102380 0x7fd2381977a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd228006010 tx=0x7fd2280040c0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.639+0000 7fd240098640 1 -- 172.21.15.18:0/1700634596 shutdown_connections 2024-02-17T12:49:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.639+0000 7fd240098640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd2240a20c0 0x7fd2240a4580 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.639+0000 7fd240098640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd238102d40 0x7fd23819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.639+0000 7fd240098640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238102380 0x7fd2381977a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.639+0000 7fd240098640 1 --2- 172.21.15.18:0/1700634596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd238101190 0x7fd238197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.639+0000 7fd240098640 1 -- 172.21.15.18:0/1700634596 >> 172.21.15.18:0/1700634596 conn(0x7fd2380fc720 msgr2=0x7fd238105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.639+0000 7fd240098640 1 -- 172.21.15.18:0/1700634596 shutdown_connections 2024-02-17T12:49:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:31.639+0000 7fd240098640 1 -- 172.21.15.18:0/1700634596 wait complete. 2024-02-17T12:49:31.642 INFO:teuthology.run_tasks:Running task kclient... 2024-02-17T12:49:31.665 INFO:tasks.kclient:Mounting kernel clients... 2024-02-17T12:49:31.665 INFO:tasks.kclient:config is {'client.0': {}} 2024-02-17T12:49:31.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:31.665 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-02-17T12:49:31.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:31.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link delete ceph-brx 2024-02-17T12:49:31.744 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:31 smithi018 ceph-mon[32625]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-02-17T12:49:31.744 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:31 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/476176946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:31.744 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:31 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2954363049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:Cannot find device "ceph-brx" 2024-02-17T12:49:31.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:49:31.750 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-02-17T12:49:31.750 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-17T12:49:31.750 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-17T12:49:31.751 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph fs ls 2024-02-17T12:49:31.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:31 smithi119 ceph-mon[36188]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-02-17T12:49:31.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:31 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/476176946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:31.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:31 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2954363049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:31.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:31 smithi146 ceph-mon[36454]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-02-17T12:49:31.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:31 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/476176946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:31.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:31 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2954363049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:49:33.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.624+0000 7f267e5a2700 1 -- 172.21.15.18:0/793188523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2678105e90 msgr2=0x7f267810da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:33.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.624+0000 7f267e5a2700 1 --2- 172.21.15.18:0/793188523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2678105e90 0x7f267810da60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f267000b0a0 tx=0x7f267000b3b0 comp rx=0 tx=0).stop 2024-02-17T12:49:33.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.625+0000 7f267e5a2700 1 -- 172.21.15.18:0/793188523 shutdown_connections 2024-02-17T12:49:33.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.625+0000 7f267e5a2700 1 --2- 172.21.15.18:0/793188523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2678105e90 0x7f267810da60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:33.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.625+0000 7f267e5a2700 1 --2- 172.21.15.18:0/793188523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2678100e00 0x7f26781011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:33.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.625+0000 7f267e5a2700 1 --2- 172.21.15.18:0/793188523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26781017b0 0x7f26781057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:33.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.625+0000 7f267e5a2700 1 -- 172.21.15.18:0/793188523 >> 172.21.15.18:0/793188523 conn(0x7f2678076b90 msgr2=0x7f2678076fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:33.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.625+0000 7f267e5a2700 1 -- 172.21.15.18:0/793188523 shutdown_connections 2024-02-17T12:49:33.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.625+0000 7f267e5a2700 1 -- 172.21.15.18:0/793188523 wait complete. 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.625+0000 7f267e5a2700 1 Processor -- start 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.625+0000 7f267e5a2700 1 -- start start 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f267e5a2700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2678100e00 0x7f2678198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f267e5a2700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26781017b0 0x7f26781987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f267e5a2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2678105e90 0x7f267819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f267e5a2700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267819d120 con 0x7f2678100e00 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f267e5a2700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267819d290 con 0x7f26781017b0 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f267e5a2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267819d450 con 0x7f2678105e90 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f2677fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2678100e00 0x7f2678198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f2677fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2678100e00 0x7f2678198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36614/0 (socket says 172.21.15.18:36614) 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f2677fff700 1 -- 172.21.15.18:0/463859821 learned_addr learned my addr 172.21.15.18:0/463859821 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f26777fe700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26781017b0 0x7f26781987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:33.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f267cb3f700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2678105e90 0x7f267819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:33.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f2677fff700 1 -- 172.21.15.18:0/463859821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2678105e90 msgr2=0x7f267819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:33.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f2677fff700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2678105e90 0x7f267819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:33.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f2677fff700 1 -- 172.21.15.18:0/463859821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26781017b0 msgr2=0x7f26781987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:33.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f2677fff700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26781017b0 0x7f26781987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:33.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f2677fff700 1 -- 172.21.15.18:0/463859821 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2670009d00 con 0x7f2678100e00 2024-02-17T12:49:33.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f2677fff700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2678100e00 0x7f2678198290 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f266800d950 tx=0x7f266800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:33.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f26757fa700 1 -- 172.21.15.18:0/463859821 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2668009b40 con 0x7f2678100e00 2024-02-17T12:49:33.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f267e5a2700 1 -- 172.21.15.18:0/463859821 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267819d730 con 0x7f2678100e00 2024-02-17T12:49:33.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f26757fa700 1 -- 172.21.15.18:0/463859821 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2668009ca0 con 0x7f2678100e00 2024-02-17T12:49:33.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.626+0000 7f26757fa700 1 -- 172.21.15.18:0/463859821 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266800f6b0 con 0x7f2678100e00 2024-02-17T12:49:33.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.627+0000 7f267e5a2700 1 -- 172.21.15.18:0/463859821 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26781a4f90 con 0x7f2678100e00 2024-02-17T12:49:33.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.627+0000 7f267e5a2700 1 -- 172.21.15.18:0/463859821 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2678068930 con 0x7f2678100e00 2024-02-17T12:49:33.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.628+0000 7f26757fa700 1 -- 172.21.15.18:0/463859821 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2668010460 con 0x7f2678100e00 2024-02-17T12:49:33.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.629+0000 7f26757fa700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f26600a2150 0x7f26600a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:33.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.629+0000 7f26777fe700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f26600a2150 0x7f26600a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:33.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.629+0000 7f26757fa700 1 -- 172.21.15.18:0/463859821 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f26680cdf90 con 0x7f2678100e00 2024-02-17T12:49:33.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.629+0000 7f26777fe700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f26600a2150 0x7f26600a4610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f266c005950 tx=0x7f266c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:33.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.634+0000 7f26757fa700 1 -- 172.21.15.18:0/463859821 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2668101d20 con 0x7f2678100e00 2024-02-17T12:49:33.753 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:33 smithi146 ceph-mon[36454]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-02-17T12:49:33.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:33 smithi018 ceph-mon[32625]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-02-17T12:49:33.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:33 smithi119 ceph-mon[36188]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-02-17T12:49:33.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.883+0000 7f267e5a2700 1 -- 172.21.15.18:0/463859821 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f26781995a0 con 0x7f2678100e00 2024-02-17T12:49:33.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.883+0000 7f26757fa700 1 -- 172.21.15.18:0/463859821 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2668020070 con 0x7f2678100e00 2024-02-17T12:49:33.876 INFO:teuthology.orchestra.run.smithi018.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-17T12:49:33.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.886+0000 7f267e5a2700 1 -- 172.21.15.18:0/463859821 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f26600a2150 msgr2=0x7f26600a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:33.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.886+0000 7f267e5a2700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f26600a2150 0x7f26600a4610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f266c005950 tx=0x7f266c0058e0 comp rx=0 tx=0).stop 2024-02-17T12:49:33.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.886+0000 7f267e5a2700 1 -- 172.21.15.18:0/463859821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2678100e00 msgr2=0x7f2678198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:33.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.886+0000 7f267e5a2700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2678100e00 0x7f2678198290 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f266800d950 tx=0x7f266800dd10 comp rx=0 tx=0).stop 2024-02-17T12:49:33.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.886+0000 7f267e5a2700 1 -- 172.21.15.18:0/463859821 shutdown_connections 2024-02-17T12:49:33.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.886+0000 7f267e5a2700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f26600a2150 0x7f26600a4610 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:33.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.886+0000 7f267e5a2700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2678105e90 0x7f267819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:33.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.886+0000 7f267e5a2700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2678100e00 0x7f2678198290 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:33.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.886+0000 7f267e5a2700 1 --2- 172.21.15.18:0/463859821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26781017b0 0x7f26781987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:33.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.886+0000 7f267e5a2700 1 -- 172.21.15.18:0/463859821 >> 172.21.15.18:0/463859821 conn(0x7f2678076b90 msgr2=0x7f2678104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:33.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.886+0000 7f267e5a2700 1 -- 172.21.15.18:0/463859821 shutdown_connections 2024-02-17T12:49:33.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:33.886+0000 7f267e5a2700 1 -- 172.21.15.18:0/463859821 wait complete. 2024-02-17T12:49:34.676 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-17T12:49:34.676 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-17T12:49:34.676 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi018.front.sepia.ceph.com 2024-02-17T12:49:34.676 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-17T12:49:34.676 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-17T12:49:34.676 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-17T12:49:34.676 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-17T12:49:34.676 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-17T12:49:34.676 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-17T12:49:34.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:34.677 DEBUG:teuthology.orchestra.run.smithi018:> ip addr 2024-02-17T12:49:34.694 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-17T12:49:34.694 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-17T12:49:34.694 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2024-02-17T12:49:34.694 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-02-17T12:49:34.694 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2024-02-17T12:49:34.694 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-02-17T12:49:34.694 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-17T12:49:34.694 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2024-02-17T12:49:34.694 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp7s0f0 2024-02-17T12:49:34.694 INFO:teuthology.orchestra.run.smithi018.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-17T12:49:34.694 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2024-02-17T12:49:34.694 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp7s0f1 2024-02-17T12:49:34.695 INFO:teuthology.orchestra.run.smithi018.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-17T12:49:34.695 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2024-02-17T12:49:34.695 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp1s0f0 2024-02-17T12:49:34.695 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-17T12:49:34.695 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42112sec preferred_lft 42112sec 2024-02-17T12:49:34.695 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link noprefixroute 2024-02-17T12:49:34.695 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-02-17T12:49:34.695 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-17T12:49:34.695 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2024-02-17T12:49:34.695 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp1s0f1 2024-02-17T12:49:34.696 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-17T12:49:34.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:34.696 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec bash -c ' 2024-02-17T12:49:34.696 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-02-17T12:49:34.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link add name ceph-brx type bridge 2024-02-17T12:49:34.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip addr flush dev ceph-brx 2024-02-17T12:49:34.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set ceph-brx up 2024-02-17T12:49:34.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-17T12:49:34.696 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-02-17T12:49:34.760 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:34 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/463859821' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-17T12:49:34.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:34 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/463859821' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-17T12:49:34.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:34.861 DEBUG:teuthology.orchestra.run.smithi018:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-17T12:49:34.928 INFO:teuthology.orchestra.run.smithi018.stdout:1 2024-02-17T12:49:34.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:34.930 DEBUG:teuthology.orchestra.run.smithi018:> route 2024-02-17T12:49:34.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:34 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/463859821' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-17T12:49:35.005 INFO:teuthology.orchestra.run.smithi018.stdout:Kernel IP routing table 2024-02-17T12:49:35.005 INFO:teuthology.orchestra.run.smithi018.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-17T12:49:35.005 INFO:teuthology.orchestra.run.smithi018.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-17T12:49:35.005 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-17T12:49:35.005 INFO:teuthology.orchestra.run.smithi018.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-17T12:49:35.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:35.006 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec bash -c ' 2024-02-17T12:49:35.006 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-02-17T12:49:35.006 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-17T12:49:35.006 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-17T12:49:35.006 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-17T12:49:35.006 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-02-17T12:49:35.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:35.281 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-02-17T12:49:35.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:35.297 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list-id 2024-02-17T12:49:35.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:35.352 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec bash -c ' 2024-02-17T12:49:35.352 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-02-17T12:49:35.352 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-17T12:49:35.352 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-17T12:49:35.352 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-02-17T12:49:35.435 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-17T12:49:35.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:35.435 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec bash -c ' 2024-02-17T12:49:35.435 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-02-17T12:49:35.435 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-17T12:49:35.435 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-17T12:49:35.435 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-17T12:49:35.436 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-17T12:49:35.436 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-17T12:49:35.436 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-02-17T12:49:35.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:35.575 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec bash -c ' 2024-02-17T12:49:35.575 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-02-17T12:49:35.576 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set brx.0 up 2024-02-17T12:49:35.576 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set dev brx.0 master ceph-brx 2024-02-17T12:49:35.576 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-02-17T12:49:35.617 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-17T12:49:35.617 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-17T12:49:35.672 INFO:teuthology.orchestra.run.smithi018.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-17T12:49:35.673 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-17T12:49:35.673 DEBUG:teuthology.orchestra.run.smithi018:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-17T12:49:35.727 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-02-17T12:49:35.727 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-02-17T12:49:35.727 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df 2024-02-17T12:49:35.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:35.727 DEBUG:teuthology.orchestra.run.smithi018:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-02-17T12:49:35.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:35 smithi018 ceph-mon[32625]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-02-17T12:49:35.810 INFO:teuthology.orchestra.run.smithi018.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-02-17T12:49:35.810 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-02-17T12:49:35.810 INFO:teuthology.orchestra.run.smithi018.stdout:invalid new device string format 2024-02-17T12:49:35.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:35 smithi119 ceph-mon[36188]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-02-17T12:49:35.920 INFO:teuthology.orchestra.run.smithi018.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-02-17T12:49:35.921 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-02-17T12:49:35.921 INFO:teuthology.orchestra.run.smithi018.stdout:invalid new device string format 2024-02-17T12:49:35.921 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: resolved to: "172.21.15.18:3300,172.21.15.119:3300,172.21.15.146:3300" 2024-02-17T12:49:35.921 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.18:3300,172.21.15.119:3300,172.21.15.146:3300:/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df 2024-02-17T12:49:35.921 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=7fca71fe-cd91-11ee-95ba-87774f69a715" will pass to kernel 2024-02-17T12:49:35.922 INFO:teuthology.orchestra.run.smithi018.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-17T12:49:35.922 INFO:teuthology.orchestra.run.smithi018.stdout: You just mounted a file system that supports labels which does not 2024-02-17T12:49:35.922 INFO:teuthology.orchestra.run.smithi018.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-17T12:49:35.922 INFO:teuthology.orchestra.run.smithi018.stdout: applications will generate AVC messages and not be allowed access to 2024-02-17T12:49:35.922 INFO:teuthology.orchestra.run.smithi018.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-17T12:49:35.923 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-17T12:49:35.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:35.924 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-17T12:49:35.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:35 smithi146 ceph-mon[36454]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-02-17T12:49:35.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:35.993 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-17T12:49:36.049 INFO:teuthology.orchestra.run.smithi018.stdout:client15354 2024-02-17T12:49:36.049 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:49:36.049 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-17T12:49:36.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:36.105 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-17T12:49:36.161 INFO:teuthology.orchestra.run.smithi018.stdout:client15354 2024-02-17T12:49:36.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:36.162 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-17T12:49:36.218 INFO:teuthology.orchestra.run.smithi018.stdout:client15354 2024-02-17T12:49:36.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:36.219 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715.client15354/status') 2024-02-17T12:49:36.285 INFO:teuthology.orchestra.run.smithi018.stdout:instance: client.15354 (3)192.168.0.1:0/4231858671 2024-02-17T12:49:36.285 INFO:teuthology.orchestra.run.smithi018.stdout:blocklisted: false 2024-02-17T12:49:36.285 INFO:teuthology.orchestra.run.smithi018.stderr:0+1 records in 2024-02-17T12:49:36.286 INFO:teuthology.orchestra.run.smithi018.stderr:0+1 records out 2024-02-17T12:49:36.286 INFO:teuthology.orchestra.run.smithi018.stderr:70 bytes copied, 2.1834e-05 s, 3.2 MB/s 2024-02-17T12:49:36.287 INFO:teuthology.run_tasks:Running task exec... 2024-02-17T12:49:36.298 INFO:teuthology.task.exec:Executing custom commands... 2024-02-17T12:49:36.298 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-02-17T12:49:36.299 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-02-17T12:49:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.438+0000 7f60c0713640 1 -- 172.21.15.18:0/981507461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60b80ff4a0 msgr2=0x7f60b80ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.438+0000 7f60c0713640 1 --2- 172.21.15.18:0/981507461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60b80ff4a0 0x7f60b80ff8c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009900 tx=0x7f60b402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.438+0000 7f60c0713640 1 -- 172.21.15.18:0/981507461 shutdown_connections 2024-02-17T12:49:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.438+0000 7f60c0713640 1 --2- 172.21.15.18:0/981507461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60b8102e30 0x7f60b81096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.438+0000 7f60c0713640 1 --2- 172.21.15.18:0/981507461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b8102470 0x7f60b81028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.438+0000 7f60c0713640 1 --2- 172.21.15.18:0/981507461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60b80ff4a0 0x7f60b80ff8c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.438+0000 7f60c0713640 1 -- 172.21.15.18:0/981507461 >> 172.21.15.18:0/981507461 conn(0x7f60b80fadd0 msgr2=0x7f60b80fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.438+0000 7f60c0713640 1 -- 172.21.15.18:0/981507461 shutdown_connections 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60c0713640 1 -- 172.21.15.18:0/981507461 wait complete. 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60c0713640 1 Processor -- start 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60c0713640 1 -- start start 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60c0713640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60b80ff4a0 0x7f60b81971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60c0713640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b8102470 0x7f60b81976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60c0713640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60b8102e30 0x7f60b819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60c0713640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b819eca0 con 0x7f60b8102e30 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60c0713640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b819ee10 con 0x7f60b8102470 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60c0713640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b819efd0 con 0x7f60b80ff4a0 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60becb0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60b8102e30 0x7f60b819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60becb0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60b8102e30 0x7f60b819e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36628/0 (socket says 172.21.15.18:36628) 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60becb0640 1 -- 172.21.15.18:0/362345029 learned_addr learned my addr 172.21.15.18:0/362345029 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.439+0000 7f60be4af640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60b80ff4a0 0x7f60b81971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60bdcae640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b8102470 0x7f60b81976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60becb0640 1 -- 172.21.15.18:0/362345029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60b80ff4a0 msgr2=0x7f60b81971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60becb0640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60b80ff4a0 0x7f60b81971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60becb0640 1 -- 172.21.15.18:0/362345029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b8102470 msgr2=0x7f60b81976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60becb0640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b8102470 0x7f60b81976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60becb0640 1 -- 172.21.15.18:0/362345029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b4009560 con 0x7f60b8102e30 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60bdcae640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b8102470 0x7f60b81976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60becb0640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60b8102e30 0x7f60b819e760 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f60b000eaa0 tx=0x7f60b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60af7fe640 1 -- 172.21.15.18:0/362345029 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b000cdb0 con 0x7f60b8102e30 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60af7fe640 1 -- 172.21.15.18:0/362345029 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60b0005020 con 0x7f60b8102e30 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60c0713640 1 -- 172.21.15.18:0/362345029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b819f2b0 con 0x7f60b8102e30 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60af7fe640 1 -- 172.21.15.18:0/362345029 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b00116b0 con 0x7f60b8102e30 2024-02-17T12:49:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.440+0000 7f60c0713640 1 -- 172.21.15.18:0/362345029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b819f770 con 0x7f60b8102e30 2024-02-17T12:49:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.441+0000 7f60c0713640 1 -- 172.21.15.18:0/362345029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b81084d0 con 0x7f60b8102e30 2024-02-17T12:49:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.442+0000 7f60af7fe640 1 -- 172.21.15.18:0/362345029 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f60b0005520 con 0x7f60b8102e30 2024-02-17T12:49:36.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.442+0000 7f60af7fe640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f60a40a2160 0x7f60a40a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:36.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.443+0000 7f60be4af640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f60a40a2160 0x7f60a40a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:36.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.443+0000 7f60af7fe640 1 -- 172.21.15.18:0/362345029 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f60b00ce310 con 0x7f60b8102e30 2024-02-17T12:49:36.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.443+0000 7f60be4af640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f60a40a2160 0x7f60a40a4620 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f60b4005ab0 tx=0x7f60b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:36.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.446+0000 7f60af7fe640 1 -- 172.21.15.18:0/362345029 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f60b00972b0 con 0x7f60b8102e30 2024-02-17T12:49:36.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:36.671+0000 7f60c0713640 1 -- 172.21.15.18:0/362345029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f60b810cc00 con 0x7f60b8102e30 2024-02-17T12:49:37.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.518+0000 7f60af7fe640 1 -- 172.21.15.18:0/362345029 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) v1 ==== 117+0+0 (secure 0 0 0) 0x7f60b001f020 con 0x7f60b8102e30 2024-02-17T12:49:37.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.521+0000 7f60c0713640 1 -- 172.21.15.18:0/362345029 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f60a40a2160 msgr2=0x7f60a40a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:37.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.521+0000 7f60c0713640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f60a40a2160 0x7f60a40a4620 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f60b4005ab0 tx=0x7f60b4005fd0 comp rx=0 tx=0).stop 2024-02-17T12:49:37.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.521+0000 7f60c0713640 1 -- 172.21.15.18:0/362345029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60b8102e30 msgr2=0x7f60b819e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.521+0000 7f60c0713640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60b8102e30 0x7f60b819e760 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f60b000eaa0 tx=0x7f60b000ef60 comp rx=0 tx=0).stop 2024-02-17T12:49:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.521+0000 7f60c0713640 1 -- 172.21.15.18:0/362345029 shutdown_connections 2024-02-17T12:49:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.521+0000 7f60c0713640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f60a40a2160 0x7f60a40a4620 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.521+0000 7f60c0713640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60b8102e30 0x7f60b819e760 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.521+0000 7f60c0713640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b8102470 0x7f60b81976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.521+0000 7f60c0713640 1 --2- 172.21.15.18:0/362345029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60b80ff4a0 0x7f60b81971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.521+0000 7f60c0713640 1 -- 172.21.15.18:0/362345029 >> 172.21.15.18:0/362345029 conn(0x7f60b80fadd0 msgr2=0x7f60b81056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.521+0000 7f60c0713640 1 -- 172.21.15.18:0/362345029 shutdown_connections 2024-02-17T12:49:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:37.521+0000 7f60c0713640 1 -- 172.21.15.18:0/362345029 wait complete. 2024-02-17T12:49:37.527 INFO:teuthology.run_tasks:Running task check-counter... 2024-02-17T12:49:37.552 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-02-17T12:49:38.542 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:49:38.544 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:37 smithi146 ceph-mon[36454]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-02-17T12:49:38.544 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:37 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/362345029' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-02-17T12:49:38.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:37 smithi119 ceph-mon[36188]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-02-17T12:49:38.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:37 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/362345029' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-02-17T12:49:38.545 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:37 smithi018 ceph-mon[32625]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-02-17T12:49:38.545 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:37 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/362345029' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-02-17T12:49:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.668+0000 7f2752bc9640 1 -- 172.21.15.18:0/1485783928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f274c101690 msgr2=0x7f274c101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.668+0000 7f2752bc9640 1 --2- 172.21.15.18:0/1485783928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f274c101690 0x7f274c101ab0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2734009900 tx=0x7f273402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.668+0000 7f2752bc9640 1 -- 172.21.15.18:0/1485783928 shutdown_connections 2024-02-17T12:49:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.668+0000 7f2752bc9640 1 --2- 172.21.15.18:0/1485783928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c100cf0 0x7f274c108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.668+0000 7f2752bc9640 1 --2- 172.21.15.18:0/1485783928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274c100330 0x7f274c1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.668+0000 7f2752bc9640 1 --2- 172.21.15.18:0/1485783928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f274c101690 0x7f274c101ab0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.668+0000 7f2752bc9640 1 -- 172.21.15.18:0/1485783928 >> 172.21.15.18:0/1485783928 conn(0x7f274c0fada0 msgr2=0x7f274c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.668+0000 7f2752bc9640 1 -- 172.21.15.18:0/1485783928 shutdown_connections 2024-02-17T12:49:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.668+0000 7f2752bc9640 1 -- 172.21.15.18:0/1485783928 wait complete. 2024-02-17T12:49:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2752bc9640 1 Processor -- start 2024-02-17T12:49:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2752bc9640 1 -- start start 2024-02-17T12:49:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2752bc9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f274c100330 0x7f274c0ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2752bc9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274c100cf0 0x7f274c19bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2752bc9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c101690 0x7f274c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2752bc9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c19ed80 con 0x7f274c100330 2024-02-17T12:49:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2752bc9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c19eef0 con 0x7f274c101690 2024-02-17T12:49:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2752bc9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c19f0b0 con 0x7f274c100cf0 2024-02-17T12:49:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2750965640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f274c100330 0x7f274c0ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2750965640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f274c100330 0x7f274c0ffd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36648/0 (socket says 172.21.15.18:36648) 2024-02-17T12:49:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2750965640 1 -- 172.21.15.18:0/384816410 learned_addr learned my addr 172.21.15.18:0/384816410 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f274bfff640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274c100cf0 0x7f274c19bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2751166640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c101690 0x7f274c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2750965640 1 -- 172.21.15.18:0/384816410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274c100cf0 msgr2=0x7f274c19bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2750965640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274c100cf0 0x7f274c19bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2750965640 1 -- 172.21.15.18:0/384816410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c101690 msgr2=0x7f274c19e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2750965640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c101690 0x7f274c19e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2750965640 1 -- 172.21.15.18:0/384816410 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2734009560 con 0x7f274c100330 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f274bfff640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274c100cf0 0x7f274c19bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.669+0000 7f2751166640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c101690 0x7f274c19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.670+0000 7f2750965640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f274c100330 0x7f274c0ffd90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2734000c00 tx=0x7f2734004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.670+0000 7f2749ffb640 1 -- 172.21.15.18:0/384816410 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2734004df0 con 0x7f274c100330 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.670+0000 7f2749ffb640 1 -- 172.21.15.18:0/384816410 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2734005430 con 0x7f274c100330 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.670+0000 7f2752bc9640 1 -- 172.21.15.18:0/384816410 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274c19f2d0 con 0x7f274c100330 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.670+0000 7f2749ffb640 1 -- 172.21.15.18:0/384816410 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273403e450 con 0x7f274c100330 2024-02-17T12:49:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.670+0000 7f2752bc9640 1 -- 172.21.15.18:0/384816410 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274c19f7e0 con 0x7f274c100330 2024-02-17T12:49:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.671+0000 7f2752bc9640 1 -- 172.21.15.18:0/384816410 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274c1074f0 con 0x7f274c100330 2024-02-17T12:49:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.672+0000 7f2749ffb640 1 -- 172.21.15.18:0/384816410 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f27340055a0 con 0x7f274c100330 2024-02-17T12:49:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.673+0000 7f2749ffb640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f27380a20c0 0x7f27380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.673+0000 7f274bfff640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f27380a20c0 0x7f27380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.673+0000 7f2749ffb640 1 -- 172.21.15.18:0/384816410 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f27340f06a0 con 0x7f274c100330 2024-02-17T12:49:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.674+0000 7f274bfff640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f27380a20c0 0x7f27380a4580 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f273c005d00 tx=0x7f273c005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.677+0000 7f2749ffb640 1 -- 172.21.15.18:0/384816410 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27340b96a0 con 0x7f274c100330 2024-02-17T12:49:38.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:38 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/362345029' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-02-17T12:49:38.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:38 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:49:38.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:38 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/362345029' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-02-17T12:49:38.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:38 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:49:38.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.861+0000 7f2752bc9640 1 -- 172.21.15.18:0/384816410 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f274c02cdd0 con 0x7f274c100330 2024-02-17T12:49:38.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.863+0000 7f2749ffb640 1 -- 172.21.15.18:0/384816410 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22607 (secure 0 0 0) 0x7f27340bcd60 con 0x7f274c100330 2024-02-17T12:49:38.855 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:38.855 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":88,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:48:52.149237+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:49:38.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.864+0000 7f2752bc9640 1 -- 172.21.15.18:0/384816410 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f27380a20c0 msgr2=0x7f27380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:38.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.864+0000 7f2752bc9640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f27380a20c0 0x7f27380a4580 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f273c005d00 tx=0x7f273c005ed0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.864+0000 7f2752bc9640 1 -- 172.21.15.18:0/384816410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f274c100330 msgr2=0x7f274c0ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:38.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.864+0000 7f2752bc9640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f274c100330 0x7f274c0ffd90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2734000c00 tx=0x7f2734004270 comp rx=0 tx=0).stop 2024-02-17T12:49:38.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.865+0000 7f2752bc9640 1 -- 172.21.15.18:0/384816410 shutdown_connections 2024-02-17T12:49:38.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.865+0000 7f2752bc9640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f27380a20c0 0x7f27380a4580 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.865+0000 7f2752bc9640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f274c101690 0x7f274c19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.865+0000 7f2752bc9640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274c100cf0 0x7f274c19bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.865+0000 7f2752bc9640 1 --2- 172.21.15.18:0/384816410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f274c100330 0x7f274c0ffd90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.865+0000 7f2752bc9640 1 -- 172.21.15.18:0/384816410 >> 172.21.15.18:0/384816410 conn(0x7f274c0fada0 msgr2=0x7f274c1046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:38.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.865+0000 7f2752bc9640 1 -- 172.21.15.18:0/384816410 shutdown_connections 2024-02-17T12:49:38.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.865+0000 7f2752bc9640 1 -- 172.21.15.18:0/384816410 wait complete. 2024-02-17T12:49:38.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T12:43:12.450140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T12:47:43.690543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T12:47:44.653229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T12:49:38.869 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T12:49:38.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:38 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/362345029' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-02-17T12:49:38.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:38 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:49:38.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.983+0000 7f2deb2ce640 1 -- 172.21.15.18:0/3107029074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2de4102390 msgr2=0x7f2de4102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:38.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.983+0000 7f2deb2ce640 1 --2- 172.21.15.18:0/3107029074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2de4102390 0x7f2de4102810 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009900 tx=0x7f2dd802bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:38.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.983+0000 7f2deb2ce640 1 -- 172.21.15.18:0/3107029074 shutdown_connections 2024-02-17T12:49:38.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.983+0000 7f2deb2ce640 1 --2- 172.21.15.18:0/3107029074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2de4102d50 0x7f2de41095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.983+0000 7f2deb2ce640 1 --2- 172.21.15.18:0/3107029074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2de4102390 0x7f2de4102810 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.983+0000 7f2deb2ce640 1 --2- 172.21.15.18:0/3107029074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de41011a0 0x7f2de41015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.983+0000 7f2deb2ce640 1 -- 172.21.15.18:0/3107029074 >> 172.21.15.18:0/3107029074 conn(0x7f2de40fc710 msgr2=0x7f2de40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.983+0000 7f2deb2ce640 1 -- 172.21.15.18:0/3107029074 shutdown_connections 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.983+0000 7f2deb2ce640 1 -- 172.21.15.18:0/3107029074 wait complete. 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.983+0000 7f2deb2ce640 1 Processor -- start 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.983+0000 7f2deb2ce640 1 -- start start 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.983+0000 7f2deb2ce640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2de41011a0 0x7f2de4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2deb2ce640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2de4102390 0x7f2de4197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2deb2ce640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de4102d50 0x7f2de419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2deb2ce640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de419ecf0 con 0x7f2de41011a0 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2deb2ce640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de419ee60 con 0x7f2de4102d50 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2deb2ce640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de419f020 con 0x7f2de4102390 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de906a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2de41011a0 0x7f2de4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:38.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de906a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2de41011a0 0x7f2de4197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36664/0 (socket says 172.21.15.18:36664) 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de906a640 1 -- 172.21.15.18:0/418814937 learned_addr learned my addr 172.21.15.18:0/418814937 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de986b640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de4102d50 0x7f2de419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de8869640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2de4102390 0x7f2de4197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de906a640 1 -- 172.21.15.18:0/418814937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2de4102390 msgr2=0x7f2de4197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de906a640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2de4102390 0x7f2de4197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de906a640 1 -- 172.21.15.18:0/418814937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de4102d50 msgr2=0x7f2de419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de906a640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de4102d50 0x7f2de419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de906a640 1 -- 172.21.15.18:0/418814937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd8009560 con 0x7f2de41011a0 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de906a640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2de41011a0 0x7f2de4197200 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00d950 tx=0x7f2ddc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de27fc640 1 -- 172.21.15.18:0/418814937 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc009c30 con 0x7f2de41011a0 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de27fc640 1 -- 172.21.15.18:0/418814937 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc009d90 con 0x7f2de41011a0 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2deb2ce640 1 -- 172.21.15.18:0/418814937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de419f300 con 0x7f2de41011a0 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2de27fc640 1 -- 172.21.15.18:0/418814937 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc00f850 con 0x7f2de41011a0 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.984+0000 7f2deb2ce640 1 -- 172.21.15.18:0/418814937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de419f830 con 0x7f2de41011a0 2024-02-17T12:49:38.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.985+0000 7f2deb2ce640 1 -- 172.21.15.18:0/418814937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de4108460 con 0x7f2de41011a0 2024-02-17T12:49:38.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.986+0000 7f2de27fc640 1 -- 172.21.15.18:0/418814937 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2ddc00f9b0 con 0x7f2de41011a0 2024-02-17T12:49:38.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.987+0000 7f2de27fc640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2dcc0a2140 0x7f2dcc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:38.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.987+0000 7f2de27fc640 1 -- 172.21.15.18:0/418814937 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2ddc0cdc10 con 0x7f2de41011a0 2024-02-17T12:49:38.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.987+0000 7f2de8869640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2dcc0a2140 0x7f2dcc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:38.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.987+0000 7f2de8869640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2dcc0a2140 0x7f2dcc0a4600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8006010 tx=0x7f2dd8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:38.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:38.991+0000 7f2de27fc640 1 -- 172.21.15.18:0/418814937 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2ddc096c90 con 0x7f2de41011a0 2024-02-17T12:49:39.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.171+0000 7f2deb2ce640 1 -- 172.21.15.18:0/418814937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2de410cc00 con 0x7f2de41011a0 2024-02-17T12:49:39.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.171+0000 7f2de27fc640 1 -- 172.21.15.18:0/418814937 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ddc017020 con 0x7f2de41011a0 2024-02-17T12:49:39.164 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-02-17T12:49:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.173+0000 7f2deb2ce640 1 -- 172.21.15.18:0/418814937 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2dcc0a2140 msgr2=0x7f2dcc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.173+0000 7f2deb2ce640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2dcc0a2140 0x7f2dcc0a4600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8006010 tx=0x7f2dd8000bc0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.173+0000 7f2deb2ce640 1 -- 172.21.15.18:0/418814937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2de41011a0 msgr2=0x7f2de4197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.173+0000 7f2deb2ce640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2de41011a0 0x7f2de4197200 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00d950 tx=0x7f2ddc00de10 comp rx=0 tx=0).stop 2024-02-17T12:49:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.173+0000 7f2deb2ce640 1 -- 172.21.15.18:0/418814937 shutdown_connections 2024-02-17T12:49:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.173+0000 7f2deb2ce640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f2dcc0a2140 0x7f2dcc0a4600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.173+0000 7f2deb2ce640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2de4102d50 0x7f2de419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.173+0000 7f2deb2ce640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2de4102390 0x7f2de4197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.173+0000 7f2deb2ce640 1 --2- 172.21.15.18:0/418814937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2de41011a0 0x7f2de4197200 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.173+0000 7f2deb2ce640 1 -- 172.21.15.18:0/418814937 >> 172.21.15.18:0/418814937 conn(0x7f2de40fc710 msgr2=0x7f2de4105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.173+0000 7f2deb2ce640 1 -- 172.21.15.18:0/418814937 shutdown_connections 2024-02-17T12:49:39.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.173+0000 7f2deb2ce640 1 -- 172.21.15.18:0/418814937 wait complete. 2024-02-17T12:49:39.177 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T12:49:39.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.291+0000 7f65bc725640 1 -- 172.21.15.18:0/1705711208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b4101280 msgr2=0x7f65b4101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.291+0000 7f65bc725640 1 --2- 172.21.15.18:0/1705711208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b4101280 0x7f65b4101700 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009900 tx=0x7f65a402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.291+0000 7f65bc725640 1 -- 172.21.15.18:0/1705711208 shutdown_connections 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.291+0000 7f65bc725640 1 --2- 172.21.15.18:0/1705711208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65b4101c40 0x7f65b41084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.291+0000 7f65bc725640 1 --2- 172.21.15.18:0/1705711208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b4101280 0x7f65b4101700 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.291+0000 7f65bc725640 1 --2- 172.21.15.18:0/1705711208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65b4100090 0x7f65b41004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.291+0000 7f65bc725640 1 -- 172.21.15.18:0/1705711208 >> 172.21.15.18:0/1705711208 conn(0x7f65b40fb620 msgr2=0x7f65b40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.291+0000 7f65bc725640 1 -- 172.21.15.18:0/1705711208 shutdown_connections 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.291+0000 7f65bc725640 1 -- 172.21.15.18:0/1705711208 wait complete. 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65bc725640 1 Processor -- start 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65bc725640 1 -- start start 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65bc725640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65b4100090 0x7f65b4197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65bc725640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b4101280 0x7f65b4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65bc725640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65b4101c40 0x7f65b419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65bc725640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b419ed00 con 0x7f65b4101280 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65bc725640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b419ee70 con 0x7f65b4100090 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65bc725640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b419f030 con 0x7f65b4101c40 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65b9cc0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b4101280 0x7f65b4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65b9cc0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b4101280 0x7f65b4197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36682/0 (socket says 172.21.15.18:36682) 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65b9cc0640 1 -- 172.21.15.18:0/1199239719 learned_addr learned my addr 172.21.15.18:0/1199239719 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65ba4c1640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65b4100090 0x7f65b4197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65bacc2640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65b4101c40 0x7f65b419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65b9cc0640 1 -- 172.21.15.18:0/1199239719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65b4101c40 msgr2=0x7f65b419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65b9cc0640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65b4101c40 0x7f65b419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65b9cc0640 1 -- 172.21.15.18:0/1199239719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65b4100090 msgr2=0x7f65b4197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65b9cc0640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65b4100090 0x7f65b4197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.292+0000 7f65b9cc0640 1 -- 172.21.15.18:0/1199239719 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a4009560 con 0x7f65b4101280 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.293+0000 7f65b9cc0640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b4101280 0x7f65b4197750 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f65a4006010 tx=0x7f65a40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.293+0000 7f65ab7fe640 1 -- 172.21.15.18:0/1199239719 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a40044c0 con 0x7f65b4101280 2024-02-17T12:49:39.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.293+0000 7f65ab7fe640 1 -- 172.21.15.18:0/1199239719 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65a40052d0 con 0x7f65b4101280 2024-02-17T12:49:39.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.293+0000 7f65ab7fe640 1 -- 172.21.15.18:0/1199239719 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a4003e10 con 0x7f65b4101280 2024-02-17T12:49:39.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.293+0000 7f65bc725640 1 -- 172.21.15.18:0/1199239719 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b419f2b0 con 0x7f65b4101280 2024-02-17T12:49:39.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.293+0000 7f65bc725640 1 -- 172.21.15.18:0/1199239719 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b419f7c0 con 0x7f65b4101280 2024-02-17T12:49:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.294+0000 7f65bc725640 1 -- 172.21.15.18:0/1199239719 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b410baa0 con 0x7f65b4101280 2024-02-17T12:49:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.295+0000 7f65ab7fe640 1 -- 172.21.15.18:0/1199239719 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f65a4005440 con 0x7f65b4101280 2024-02-17T12:49:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.295+0000 7f65ab7fe640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65a00a20c0 0x7f65a00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.295+0000 7f65ba4c1640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65a00a20c0 0x7f65a00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.295+0000 7f65ab7fe640 1 -- 172.21.15.18:0/1199239719 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f65a40eff50 con 0x7f65b4101280 2024-02-17T12:49:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.296+0000 7f65ba4c1640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65a00a20c0 0x7f65a00a4580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f65b0005fd0 tx=0x7f65b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:39.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.300+0000 7f65ab7fe640 1 -- 172.21.15.18:0/1199239719 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65a40b8f50 con 0x7f65b4101280 2024-02-17T12:49:39.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.480+0000 7f65bc725640 1 -- 172.21.15.18:0/1199239719 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65b419fb80 con 0x7f65b4101280 2024-02-17T12:49:39.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.480+0000 7f65ab7fe640 1 -- 172.21.15.18:0/1199239719 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65a40bc610 con 0x7f65b4101280 2024-02-17T12:49:39.473 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-02-17T12:49:39.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.482+0000 7f65bc725640 1 -- 172.21.15.18:0/1199239719 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65a00a20c0 msgr2=0x7f65a00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.482+0000 7f65bc725640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65a00a20c0 0x7f65a00a4580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f65b0005fd0 tx=0x7f65b000b390 comp rx=0 tx=0).stop 2024-02-17T12:49:39.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.482+0000 7f65bc725640 1 -- 172.21.15.18:0/1199239719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b4101280 msgr2=0x7f65b4197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.482+0000 7f65bc725640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b4101280 0x7f65b4197750 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f65a4006010 tx=0x7f65a40040c0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.482+0000 7f65bc725640 1 -- 172.21.15.18:0/1199239719 shutdown_connections 2024-02-17T12:49:39.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.482+0000 7f65bc725640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f65a00a20c0 0x7f65a00a4580 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.482+0000 7f65bc725640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65b4101c40 0x7f65b419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.482+0000 7f65bc725640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b4101280 0x7f65b4197750 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.482+0000 7f65bc725640 1 --2- 172.21.15.18:0/1199239719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65b4100090 0x7f65b4197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.482+0000 7f65bc725640 1 -- 172.21.15.18:0/1199239719 >> 172.21.15.18:0/1199239719 conn(0x7f65b40fb620 msgr2=0x7f65b4069770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:39.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.482+0000 7f65bc725640 1 -- 172.21.15.18:0/1199239719 shutdown_connections 2024-02-17T12:49:39.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.482+0000 7f65bc725640 1 -- 172.21.15.18:0/1199239719 wait complete. 2024-02-17T12:49:39.485 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T12:49:39.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.599+0000 7f672694f640 1 -- 172.21.15.18:0/2664018765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6720101c50 msgr2=0x7f67201084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.599+0000 7f672694f640 1 --2- 172.21.15.18:0/2664018765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6720101c50 0x7f67201084e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6718009ca0 tx=0x7f671802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.599+0000 7f672694f640 1 -- 172.21.15.18:0/2664018765 shutdown_connections 2024-02-17T12:49:39.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.599+0000 7f672694f640 1 --2- 172.21.15.18:0/2664018765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6720101c50 0x7f67201084e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.599+0000 7f672694f640 1 --2- 172.21.15.18:0/2664018765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6720101290 0x7f6720101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.599+0000 7f672694f640 1 --2- 172.21.15.18:0/2664018765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67201000a0 0x7f67201004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.599+0000 7f672694f640 1 -- 172.21.15.18:0/2664018765 >> 172.21.15.18:0/2664018765 conn(0x7f67200fb630 msgr2=0x7f67200fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:39.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.599+0000 7f672694f640 1 -- 172.21.15.18:0/2664018765 shutdown_connections 2024-02-17T12:49:39.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.599+0000 7f672694f640 1 -- 172.21.15.18:0/2664018765 wait complete. 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f672694f640 1 Processor -- start 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f672694f640 1 -- start start 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f672694f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67201000a0 0x7f6720104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f672694f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6720101290 0x7f6720102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f672694f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6720101c50 0x7f6720102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f672694f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6720103310 con 0x7f6720101290 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f672694f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6720103480 con 0x7f67201000a0 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f672694f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67201a12b0 con 0x7f6720101c50 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f671f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6720101290 0x7f6720102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f671f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6720101290 0x7f6720102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34336/0 (socket says 172.21.15.18:34336) 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f671f7fe640 1 -- 172.21.15.18:0/4283421451 learned_addr learned my addr 172.21.15.18:0/4283421451 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f671ffff640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67201000a0 0x7f6720104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f671f7fe640 1 -- 172.21.15.18:0/4283421451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6720101c50 msgr2=0x7f6720102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f671f7fe640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6720101c50 0x7f6720102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f671f7fe640 1 -- 172.21.15.18:0/4283421451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67201000a0 msgr2=0x7f6720104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f671f7fe640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67201000a0 0x7f6720104030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f671f7fe640 1 -- 172.21.15.18:0/4283421451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6718009900 con 0x7f6720101290 2024-02-17T12:49:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.600+0000 7f671ffff640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67201000a0 0x7f6720104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.601+0000 7f671f7fe640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6720101290 0x7f6720102670 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f671400d7a0 tx=0x7f671400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.601+0000 7f671d7fa640 1 -- 172.21.15.18:0/4283421451 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714009c30 con 0x7f6720101290 2024-02-17T12:49:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.601+0000 7f672694f640 1 -- 172.21.15.18:0/4283421451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67201a1560 con 0x7f6720101290 2024-02-17T12:49:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.601+0000 7f671d7fa640 1 -- 172.21.15.18:0/4283421451 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f671400deb0 con 0x7f6720101290 2024-02-17T12:49:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.601+0000 7f671d7fa640 1 -- 172.21.15.18:0/4283421451 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714004ea0 con 0x7f6720101290 2024-02-17T12:49:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.601+0000 7f672694f640 1 -- 172.21.15.18:0/4283421451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67201a1a70 con 0x7f6720101290 2024-02-17T12:49:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.602+0000 7f672694f640 1 -- 172.21.15.18:0/4283421451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672010bab0 con 0x7f6720101290 2024-02-17T12:49:39.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.603+0000 7f671d7fa640 1 -- 172.21.15.18:0/4283421451 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6714005020 con 0x7f6720101290 2024-02-17T12:49:39.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.603+0000 7f671d7fa640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f67080a20c0 0x7f67080a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:39.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.604+0000 7f671ffff640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f67080a20c0 0x7f67080a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:39.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.604+0000 7f671d7fa640 1 -- 172.21.15.18:0/4283421451 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f67140cd7b0 con 0x7f6720101290 2024-02-17T12:49:39.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.604+0000 7f671ffff640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f67080a20c0 0x7f67080a4580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f67100058c0 tx=0x7f6710005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:39.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.608+0000 7f671d7fa640 1 -- 172.21.15.18:0/4283421451 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6714096750 con 0x7f6720101290 2024-02-17T12:49:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:39 smithi018 ceph-mon[32625]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:49:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:39 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/384816410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:39 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/418814937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:39.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:39 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1199239719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:39.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.787+0000 7f672694f640 1 -- 172.21.15.18:0/4283421451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f672010bc40 con 0x7f6720101290 2024-02-17T12:49:39.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.787+0000 7f671d7fa640 1 -- 172.21.15.18:0/4283421451 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6714017020 con 0x7f6720101290 2024-02-17T12:49:39.780 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-02-17T12:49:39.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.789+0000 7f672694f640 1 -- 172.21.15.18:0/4283421451 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f67080a20c0 msgr2=0x7f67080a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.789+0000 7f672694f640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f67080a20c0 0x7f67080a4580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f67100058c0 tx=0x7f6710005850 comp rx=0 tx=0).stop 2024-02-17T12:49:39.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.789+0000 7f672694f640 1 -- 172.21.15.18:0/4283421451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6720101290 msgr2=0x7f6720102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.789+0000 7f672694f640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6720101290 0x7f6720102670 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f671400d7a0 tx=0x7f671400dc60 comp rx=0 tx=0).stop 2024-02-17T12:49:39.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.789+0000 7f672694f640 1 -- 172.21.15.18:0/4283421451 shutdown_connections 2024-02-17T12:49:39.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.789+0000 7f672694f640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f67080a20c0 0x7f67080a4580 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.789+0000 7f672694f640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6720101c50 0x7f6720102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.789+0000 7f672694f640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6720101290 0x7f6720102670 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.789+0000 7f672694f640 1 --2- 172.21.15.18:0/4283421451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67201000a0 0x7f6720104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.789+0000 7f672694f640 1 -- 172.21.15.18:0/4283421451 >> 172.21.15.18:0/4283421451 conn(0x7f67200fb630 msgr2=0x7f67200fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:39.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.790+0000 7f672694f640 1 -- 172.21.15.18:0/4283421451 shutdown_connections 2024-02-17T12:49:39.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.790+0000 7f672694f640 1 -- 172.21.15.18:0/4283421451 wait complete. 2024-02-17T12:49:39.793 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:49:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:39 smithi119 ceph-mon[36188]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:49:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:39 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/384816410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:39 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/418814937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:39 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1199239719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:39.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.903+0000 7f26d677a640 1 -- 172.21.15.18:0/528506279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26d01011b0 msgr2=0x7f26d01015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.903+0000 7f26d677a640 1 --2- 172.21.15.18:0/528506279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26d01011b0 0x7f26d01015d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009900 tx=0x7f26c002bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.903+0000 7f26d677a640 1 -- 172.21.15.18:0/528506279 shutdown_connections 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.903+0000 7f26d677a640 1 --2- 172.21.15.18:0/528506279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d0102d60 0x7f26d01095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.903+0000 7f26d677a640 1 --2- 172.21.15.18:0/528506279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d01023a0 0x7f26d0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.903+0000 7f26d677a640 1 --2- 172.21.15.18:0/528506279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26d01011b0 0x7f26d01015d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.903+0000 7f26d677a640 1 -- 172.21.15.18:0/528506279 >> 172.21.15.18:0/528506279 conn(0x7f26d00fc760 msgr2=0x7f26d00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.903+0000 7f26d677a640 1 -- 172.21.15.18:0/528506279 shutdown_connections 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.903+0000 7f26d677a640 1 -- 172.21.15.18:0/528506279 wait complete. 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26d677a640 1 Processor -- start 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26d677a640 1 -- start start 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26d677a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d01011b0 0x7f26d0197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26d677a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d01023a0 0x7f26d0197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26d677a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26d0102d60 0x7f26d019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:39.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26d677a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d019ed00 con 0x7f26d0102d60 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26d677a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d019ee70 con 0x7f26d01023a0 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26d677a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d019f030 con 0x7f26d01011b0 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26d4d17640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26d0102d60 0x7f26d019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26d4d17640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26d0102d60 0x7f26d019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34366/0 (socket says 172.21.15.18:34366) 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26d4d17640 1 -- 172.21.15.18:0/1009935363 learned_addr learned my addr 172.21.15.18:0/1009935363 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26cffff640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d01011b0 0x7f26d0197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.904+0000 7f26cf7fe640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d01023a0 0x7f26d0197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.905+0000 7f26d4d17640 1 -- 172.21.15.18:0/1009935363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d01011b0 msgr2=0x7f26d0197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.905+0000 7f26d4d17640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d01011b0 0x7f26d0197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.905+0000 7f26d4d17640 1 -- 172.21.15.18:0/1009935363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d01023a0 msgr2=0x7f26d0197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.905+0000 7f26d4d17640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d01023a0 0x7f26d0197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.905+0000 7f26d4d17640 1 -- 172.21.15.18:0/1009935363 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c0009560 con 0x7f26d0102d60 2024-02-17T12:49:39.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.905+0000 7f26cf7fe640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d01023a0 0x7f26d0197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:39.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.905+0000 7f26d4d17640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26d0102d60 0x7f26d019e7c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f26c800cce0 tx=0x7f26c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:39.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.905+0000 7f26cd7fa640 1 -- 172.21.15.18:0/1009935363 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c800f450 con 0x7f26d0102d60 2024-02-17T12:49:39.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.905+0000 7f26cd7fa640 1 -- 172.21.15.18:0/1009935363 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26c800fa90 con 0x7f26d0102d60 2024-02-17T12:49:39.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.905+0000 7f26d677a640 1 -- 172.21.15.18:0/1009935363 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d019f310 con 0x7f26d0102d60 2024-02-17T12:49:39.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.905+0000 7f26cd7fa640 1 -- 172.21.15.18:0/1009935363 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8018b90 con 0x7f26d0102d60 2024-02-17T12:49:39.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.905+0000 7f26d677a640 1 -- 172.21.15.18:0/1009935363 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d019f7d0 con 0x7f26d0102d60 2024-02-17T12:49:39.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.906+0000 7f26d677a640 1 -- 172.21.15.18:0/1009935363 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d0108470 con 0x7f26d0102d60 2024-02-17T12:49:39.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.907+0000 7f26cd7fa640 1 -- 172.21.15.18:0/1009935363 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f26c800fc00 con 0x7f26d0102d60 2024-02-17T12:49:39.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.908+0000 7f26cd7fa640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f26bc0a20c0 0x7f26bc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:39.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.908+0000 7f26cffff640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f26bc0a20c0 0x7f26bc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:39.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.908+0000 7f26cd7fa640 1 -- 172.21.15.18:0/1009935363 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f26c80ce490 con 0x7f26d0102d60 2024-02-17T12:49:39.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.908+0000 7f26cffff640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f26bc0a20c0 0x7f26bc0a4580 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f26c0005ab0 tx=0x7f26c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:39.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:39.912+0000 7f26cd7fa640 1 -- 172.21.15.18:0/1009935363 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f26c8097430 con 0x7f26d0102d60 2024-02-17T12:49:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:39 smithi146 ceph-mon[36454]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-17T12:49:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:39 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/384816410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:39 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/418814937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:39 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1199239719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:40.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.094+0000 7f26d677a640 1 -- 172.21.15.18:0/1009935363 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26d002ce40 con 0x7f26d0102d60 2024-02-17T12:49:40.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.096+0000 7f26cd7fa640 1 -- 172.21.15.18:0/1009935363 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22607 (secure 0 0 0) 0x7f26c809aaf0 con 0x7f26d0102d60 2024-02-17T12:49:40.088 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:40.088 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":88,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:48:52.149237+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:49:40.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.098+0000 7f26d677a640 1 -- 172.21.15.18:0/1009935363 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f26bc0a20c0 msgr2=0x7f26bc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.098+0000 7f26d677a640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f26bc0a20c0 0x7f26bc0a4580 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f26c0005ab0 tx=0x7f26c0005fd0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.098+0000 7f26d677a640 1 -- 172.21.15.18:0/1009935363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26d0102d60 msgr2=0x7f26d019e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.098+0000 7f26d677a640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26d0102d60 0x7f26d019e7c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f26c800cce0 tx=0x7f26c800a620 comp rx=0 tx=0).stop 2024-02-17T12:49:40.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.098+0000 7f26d677a640 1 -- 172.21.15.18:0/1009935363 shutdown_connections 2024-02-17T12:49:40.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.098+0000 7f26d677a640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f26bc0a20c0 0x7f26bc0a4580 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.098+0000 7f26d677a640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26d0102d60 0x7f26d019e7c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.098+0000 7f26d677a640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26d01023a0 0x7f26d0197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.098+0000 7f26d677a640 1 --2- 172.21.15.18:0/1009935363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26d01011b0 0x7f26d0197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.098+0000 7f26d677a640 1 -- 172.21.15.18:0/1009935363 >> 172.21.15.18:0/1009935363 conn(0x7f26d00fc760 msgr2=0x7f26d0105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:40.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.098+0000 7f26d677a640 1 -- 172.21.15.18:0/1009935363 shutdown_connections 2024-02-17T12:49:40.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.098+0000 7f26d677a640 1 -- 172.21.15.18:0/1009935363 wait complete. 2024-02-17T12:49:40.102 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T12:43:12.450140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T12:47:43.690543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T12:47:44.653229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T12:49:40.102 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T12:49:40.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.216+0000 7fc1231b9640 1 -- 172.21.15.18:0/2335894461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc11c100d20 msgr2=0x7fc11c1085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.216+0000 7fc1231b9640 1 --2- 172.21.15.18:0/2335894461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc11c100d20 0x7fc11c1085c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc104009930 tx=0x7fc10402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.216+0000 7fc1231b9640 1 -- 172.21.15.18:0/2335894461 shutdown_connections 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.216+0000 7fc1231b9640 1 --2- 172.21.15.18:0/2335894461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc11c100d20 0x7fc11c1085c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.216+0000 7fc1231b9640 1 --2- 172.21.15.18:0/2335894461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c100360 0x7fc11c1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.216+0000 7fc1231b9640 1 --2- 172.21.15.18:0/2335894461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc11c101630 0x7fc11c101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.216+0000 7fc1231b9640 1 -- 172.21.15.18:0/2335894461 >> 172.21.15.18:0/2335894461 conn(0x7fc11c0fadb0 msgr2=0x7fc11c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.216+0000 7fc1231b9640 1 -- 172.21.15.18:0/2335894461 shutdown_connections 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.216+0000 7fc1231b9640 1 -- 172.21.15.18:0/2335894461 wait complete. 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.216+0000 7fc1231b9640 1 Processor -- start 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.216+0000 7fc1231b9640 1 -- start start 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc1231b9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc11c100360 0x7fc11c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc1231b9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c100d20 0x7fc11c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc1231b9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc11c101630 0x7fc11c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc1231b9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19ed00 con 0x7fc11c101630 2024-02-17T12:49:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc1231b9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19ee70 con 0x7fc11c100d20 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc1231b9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19f030 con 0x7fc11c100360 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc121756640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc11c101630 0x7fc11c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc121756640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc11c101630 0x7fc11c19e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34384/0 (socket says 172.21.15.18:34384) 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc121756640 1 -- 172.21.15.18:0/3319334737 learned_addr learned my addr 172.21.15.18:0/3319334737 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc11bfff640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c100d20 0x7fc11c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc120f55640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc11c100360 0x7fc11c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc121756640 1 -- 172.21.15.18:0/3319334737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc11c100360 msgr2=0x7fc11c197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc121756640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc11c100360 0x7fc11c197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc121756640 1 -- 172.21.15.18:0/3319334737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c100d20 msgr2=0x7fc11c197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc121756640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c100d20 0x7fc11c197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc121756640 1 -- 172.21.15.18:0/3319334737 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc104009560 con 0x7fc11c101630 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc121756640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc11c101630 0x7fc11c19e7c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc104009900 tx=0x7fc1040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc119ffb640 1 -- 172.21.15.18:0/3319334737 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1040044e0 con 0x7fc11c101630 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc119ffb640 1 -- 172.21.15.18:0/3319334737 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1040054e0 con 0x7fc11c101630 2024-02-17T12:49:40.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc1231b9640 1 -- 172.21.15.18:0/3319334737 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c19f2b0 con 0x7fc11c101630 2024-02-17T12:49:40.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.217+0000 7fc119ffb640 1 -- 172.21.15.18:0/3319334737 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10403e450 con 0x7fc11c101630 2024-02-17T12:49:40.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.218+0000 7fc1231b9640 1 -- 172.21.15.18:0/3319334737 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c19f7c0 con 0x7fc11c101630 2024-02-17T12:49:40.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.219+0000 7fc1231b9640 1 -- 172.21.15.18:0/3319334737 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c10bad0 con 0x7fc11c101630 2024-02-17T12:49:40.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.220+0000 7fc119ffb640 1 -- 172.21.15.18:0/3319334737 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc104003680 con 0x7fc11c101630 2024-02-17T12:49:40.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.220+0000 7fc119ffb640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc1080a20c0 0x7fc1080a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:40.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.221+0000 7fc120f55640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc1080a20c0 0x7fc1080a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:40.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.221+0000 7fc119ffb640 1 -- 172.21.15.18:0/3319334737 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc1040f01d0 con 0x7fc11c101630 2024-02-17T12:49:40.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.221+0000 7fc120f55640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc1080a20c0 0x7fc1080a4580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc114005fd0 tx=0x7fc11400b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:40.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.225+0000 7fc119ffb640 1 -- 172.21.15.18:0/3319334737 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1040b9170 con 0x7fc11c101630 2024-02-17T12:49:40.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.408+0000 7fc1231b9640 1 -- 172.21.15.18:0/3319334737 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc11c10bc60 con 0x7fc11c101630 2024-02-17T12:49:40.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.408+0000 7fc119ffb640 1 -- 172.21.15.18:0/3319334737 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1040bc830 con 0x7fc11c101630 2024-02-17T12:49:40.400 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-02-17T12:49:40.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.410+0000 7fc1231b9640 1 -- 172.21.15.18:0/3319334737 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc1080a20c0 msgr2=0x7fc1080a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.410+0000 7fc1231b9640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc1080a20c0 0x7fc1080a4580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc114005fd0 tx=0x7fc11400b5b0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.410+0000 7fc1231b9640 1 -- 172.21.15.18:0/3319334737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc11c101630 msgr2=0x7fc11c19e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.410+0000 7fc1231b9640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc11c101630 0x7fc11c19e7c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc104009900 tx=0x7fc1040042f0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.410+0000 7fc1231b9640 1 -- 172.21.15.18:0/3319334737 shutdown_connections 2024-02-17T12:49:40.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.410+0000 7fc1231b9640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc1080a20c0 0x7fc1080a4580 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.410+0000 7fc1231b9640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc11c101630 0x7fc11c19e7c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.410+0000 7fc1231b9640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c100d20 0x7fc11c197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.410+0000 7fc1231b9640 1 --2- 172.21.15.18:0/3319334737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc11c100360 0x7fc11c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.410+0000 7fc1231b9640 1 -- 172.21.15.18:0/3319334737 >> 172.21.15.18:0/3319334737 conn(0x7fc11c0fadb0 msgr2=0x7fc11c0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:40.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.410+0000 7fc1231b9640 1 -- 172.21.15.18:0/3319334737 shutdown_connections 2024-02-17T12:49:40.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.410+0000 7fc1231b9640 1 -- 172.21.15.18:0/3319334737 wait complete. 2024-02-17T12:49:40.414 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T12:49:40.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.529+0000 7fb37a93a640 1 -- 172.21.15.18:0/2292867272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb374101c00 msgr2=0x7fb374103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.529+0000 7fb37a93a640 1 --2- 172.21.15.18:0/2292867272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb374101c00 0x7fb374103ff0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb364009900 tx=0x7fb36402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:40.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.529+0000 7fb37a93a640 1 -- 172.21.15.18:0/2292867272 shutdown_connections 2024-02-17T12:49:40.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.529+0000 7fb37a93a640 1 --2- 172.21.15.18:0/2292867272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb374106e60 0x7fb3741092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.529+0000 7fb37a93a640 1 --2- 172.21.15.18:0/2292867272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb374104530 0x7fb374106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.529+0000 7fb37a93a640 1 --2- 172.21.15.18:0/2292867272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb374101c00 0x7fb374103ff0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.529+0000 7fb37a93a640 1 -- 172.21.15.18:0/2292867272 >> 172.21.15.18:0/2292867272 conn(0x7fb3740fb610 msgr2=0x7fb3740fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:40.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.529+0000 7fb37a93a640 1 -- 172.21.15.18:0/2292867272 shutdown_connections 2024-02-17T12:49:40.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.529+0000 7fb37a93a640 1 -- 172.21.15.18:0/2292867272 wait complete. 2024-02-17T12:49:40.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb37a93a640 1 Processor -- start 2024-02-17T12:49:40.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb37a93a640 1 -- start start 2024-02-17T12:49:40.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb37a93a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb374101c00 0x7fb374197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:40.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb37a93a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb374104530 0x7fb3741977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:40.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb37a93a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb374106e60 0x7fb37419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:40.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb37a93a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37419ed70 con 0x7fb374104530 2024-02-17T12:49:40.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb37a93a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37419eee0 con 0x7fb374101c00 2024-02-17T12:49:40.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb37a93a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37419f0a0 con 0x7fb374106e60 2024-02-17T12:49:40.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb3737fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb374104530 0x7fb3741977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:40.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb3737fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb374104530 0x7fb3741977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34406/0 (socket says 172.21.15.18:34406) 2024-02-17T12:49:40.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb3737fe640 1 -- 172.21.15.18:0/3313977282 learned_addr learned my addr 172.21.15.18:0/3313977282 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:40.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb373fff640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb374101c00 0x7fb374197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:40.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb378ed7640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb374106e60 0x7fb37419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:40.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb3737fe640 1 -- 172.21.15.18:0/3313977282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb374106e60 msgr2=0x7fb37419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb3737fe640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb374106e60 0x7fb37419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb3737fe640 1 -- 172.21.15.18:0/3313977282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb374101c00 msgr2=0x7fb374197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb3737fe640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb374101c00 0x7fb374197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.530+0000 7fb3737fe640 1 -- 172.21.15.18:0/3313977282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb364009560 con 0x7fb374104530 2024-02-17T12:49:40.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.531+0000 7fb378ed7640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb374106e60 0x7fb37419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:40.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.531+0000 7fb3737fe640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb374104530 0x7fb3741977c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb36800e9c0 tx=0x7fb36800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.532+0000 7fb3717fa640 1 -- 172.21.15.18:0/3313977282 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368009700 con 0x7fb374104530 2024-02-17T12:49:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.532+0000 7fb3717fa640 1 -- 172.21.15.18:0/3313977282 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb36800cd80 con 0x7fb374104530 2024-02-17T12:49:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.532+0000 7fb3717fa640 1 -- 172.21.15.18:0/3313977282 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368010430 con 0x7fb374104530 2024-02-17T12:49:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.532+0000 7fb37a93a640 1 -- 172.21.15.18:0/3313977282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37419f380 con 0x7fb374104530 2024-02-17T12:49:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.532+0000 7fb37a93a640 1 -- 172.21.15.18:0/3313977282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37419f890 con 0x7fb374104530 2024-02-17T12:49:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.533+0000 7fb37a93a640 1 -- 172.21.15.18:0/3313977282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3740fd100 con 0x7fb374104530 2024-02-17T12:49:40.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.534+0000 7fb3717fa640 1 -- 172.21.15.18:0/3313977282 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb368005000 con 0x7fb374104530 2024-02-17T12:49:40.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.535+0000 7fb3717fa640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb35c0a20c0 0x7fb35c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:40.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.535+0000 7fb373fff640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb35c0a20c0 0x7fb35c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:40.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.535+0000 7fb3717fa640 1 -- 172.21.15.18:0/3313977282 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb368014070 con 0x7fb374104530 2024-02-17T12:49:40.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.535+0000 7fb373fff640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb35c0a20c0 0x7fb35c0a4580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb364000c00 tx=0x7fb36402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:40.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.539+0000 7fb3717fa640 1 -- 172.21.15.18:0/3313977282 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb3680970f0 con 0x7fb374104530 2024-02-17T12:49:40.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.718+0000 7fb37a93a640 1 -- 172.21.15.18:0/3313977282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb37402cda0 con 0x7fb374104530 2024-02-17T12:49:40.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.719+0000 7fb3717fa640 1 -- 172.21.15.18:0/3313977282 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb36809a7b0 con 0x7fb374104530 2024-02-17T12:49:40.711 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-02-17T12:49:40.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.721+0000 7fb37a93a640 1 -- 172.21.15.18:0/3313977282 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb35c0a20c0 msgr2=0x7fb35c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.721+0000 7fb37a93a640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb35c0a20c0 0x7fb35c0a4580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb364000c00 tx=0x7fb36402c7e0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.721+0000 7fb37a93a640 1 -- 172.21.15.18:0/3313977282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb374104530 msgr2=0x7fb3741977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.721+0000 7fb37a93a640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb374104530 0x7fb3741977c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb36800e9c0 tx=0x7fb36800ee80 comp rx=0 tx=0).stop 2024-02-17T12:49:40.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.721+0000 7fb37a93a640 1 -- 172.21.15.18:0/3313977282 shutdown_connections 2024-02-17T12:49:40.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.721+0000 7fb37a93a640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fb35c0a20c0 0x7fb35c0a4580 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.721+0000 7fb37a93a640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb374106e60 0x7fb37419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.721+0000 7fb37a93a640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb374104530 0x7fb3741977c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.721+0000 7fb37a93a640 1 --2- 172.21.15.18:0/3313977282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb374101c00 0x7fb374197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.721+0000 7fb37a93a640 1 -- 172.21.15.18:0/3313977282 >> 172.21.15.18:0/3313977282 conn(0x7fb3740fb610 msgr2=0x7fb3741026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:40.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.721+0000 7fb37a93a640 1 -- 172.21.15.18:0/3313977282 shutdown_connections 2024-02-17T12:49:40.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.721+0000 7fb37a93a640 1 -- 172.21.15.18:0/3313977282 wait complete. 2024-02-17T12:49:40.723 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:40 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/4283421451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:40.723 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:40 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1009935363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:40.723 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:40 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3319334737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:40.725 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T12:49:40.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3195256073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38f4102380 msgr2=0x7f38f4102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 --2- 172.21.15.18:0/3195256073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38f4102380 0x7f38f4102800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009900 tx=0x7f38e002bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3195256073 shutdown_connections 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 --2- 172.21.15.18:0/3195256073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38f4102d40 0x7f38f41095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 --2- 172.21.15.18:0/3195256073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38f4102380 0x7f38f4102800 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 --2- 172.21.15.18:0/3195256073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38f4101190 0x7f38f41015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3195256073 >> 172.21.15.18:0/3195256073 conn(0x7f38f40fc720 msgr2=0x7f38f40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3195256073 shutdown_connections 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3195256073 wait complete. 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 Processor -- start 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 -- start start 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38f4101190 0x7f38f41971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38f4102380 0x7f38f41976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38f4102d40 0x7f38f419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f419ec90 con 0x7f38f4102d40 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f419ee00 con 0x7f38f4102380 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.838+0000 7f38fb0a3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f419efc0 con 0x7f38f4101190 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f9640640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38f4102d40 0x7f38f419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38f4102380 0x7f38f41976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f9640640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38f4102d40 0x7f38f419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34422/0 (socket says 172.21.15.18:34422) 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f9640640 1 -- 172.21.15.18:0/3103363957 learned_addr learned my addr 172.21.15.18:0/3103363957 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f8e3f640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38f4101190 0x7f38f41971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f9640640 1 -- 172.21.15.18:0/3103363957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38f4101190 msgr2=0x7f38f41971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f9640640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38f4101190 0x7f38f41971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f9640640 1 -- 172.21.15.18:0/3103363957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38f4102380 msgr2=0x7f38f41976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f9640640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38f4102380 0x7f38f41976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f9640640 1 -- 172.21.15.18:0/3103363957 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e0009560 con 0x7f38f4102d40 2024-02-17T12:49:40.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f9640640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38f4102d40 0x7f38f419e750 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00cce0 tx=0x7f38ec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:40.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f1ffb640 1 -- 172.21.15.18:0/3103363957 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec00f450 con 0x7f38f4102d40 2024-02-17T12:49:40.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f1ffb640 1 -- 172.21.15.18:0/3103363957 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38ec00fa90 con 0x7f38f4102d40 2024-02-17T12:49:40.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3103363957 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f419f2a0 con 0x7f38f4102d40 2024-02-17T12:49:40.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38f1ffb640 1 -- 172.21.15.18:0/3103363957 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec018b90 con 0x7f38f4102d40 2024-02-17T12:49:40.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.839+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3103363957 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f419f730 con 0x7f38f4102d40 2024-02-17T12:49:40.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.840+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3103363957 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f4108530 con 0x7f38f4102d40 2024-02-17T12:49:40.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.841+0000 7f38f1ffb640 1 -- 172.21.15.18:0/3103363957 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f38ec00fc00 con 0x7f38f4102d40 2024-02-17T12:49:40.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.842+0000 7f38f1ffb640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f38e40a20c0 0x7f38e40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:40.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.842+0000 7f38f8e3f640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f38e40a20c0 0x7f38e40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:40.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.842+0000 7f38f1ffb640 1 -- 172.21.15.18:0/3103363957 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f38ec0ce380 con 0x7f38f4102d40 2024-02-17T12:49:40.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.842+0000 7f38f8e3f640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f38e40a20c0 0x7f38e40a4580 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f38f41021e0 tx=0x7f38e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:40.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:40.846+0000 7f38f1ffb640 1 -- 172.21.15.18:0/3103363957 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f38ec097380 con 0x7f38f4102d40 2024-02-17T12:49:40.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:40 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/4283421451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:40.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:40 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1009935363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:40.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:40 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3319334737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:40.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:40 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/4283421451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:40.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:40 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1009935363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:40.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:40 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3319334737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:41.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.026+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3103363957 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f38f410cba0 con 0x7f38f4102d40 2024-02-17T12:49:41.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.026+0000 7f38f1ffb640 1 -- 172.21.15.18:0/3103363957 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f38f410cba0 con 0x7f38f4102d40 2024-02-17T12:49:41.019 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-02-17T12:49:41.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.028+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3103363957 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f38e40a20c0 msgr2=0x7f38e40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.028+0000 7f38fb0a3640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f38e40a20c0 0x7f38e40a4580 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f38f41021e0 tx=0x7f38e8005f90 comp rx=0 tx=0).stop 2024-02-17T12:49:41.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.028+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3103363957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38f4102d40 msgr2=0x7f38f419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.028+0000 7f38fb0a3640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38f4102d40 0x7f38f419e750 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00cce0 tx=0x7f38ec00a620 comp rx=0 tx=0).stop 2024-02-17T12:49:41.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.028+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3103363957 shutdown_connections 2024-02-17T12:49:41.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.028+0000 7f38fb0a3640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f38e40a20c0 0x7f38e40a4580 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.028+0000 7f38fb0a3640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38f4102d40 0x7f38f419e750 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.028+0000 7f38fb0a3640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38f4102380 0x7f38f41976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.028+0000 7f38fb0a3640 1 --2- 172.21.15.18:0/3103363957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38f4101190 0x7f38f41971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.028+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3103363957 >> 172.21.15.18:0/3103363957 conn(0x7f38f40fc720 msgr2=0x7f38f4105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:41.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.029+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3103363957 shutdown_connections 2024-02-17T12:49:41.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.029+0000 7f38fb0a3640 1 -- 172.21.15.18:0/3103363957 wait complete. 2024-02-17T12:49:41.032 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-02-17T12:49:41.033 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:49:41.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.146+0000 7f66766a7640 1 -- 172.21.15.18:0/1513226359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6670100d70 msgr2=0x7f66701011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.146+0000 7f66766a7640 1 --2- 172.21.15.18:0/1513226359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6670100d70 0x7f66701011f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6668009ca0 tx=0x7f666802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.146+0000 7f66766a7640 1 -- 172.21.15.18:0/1513226359 shutdown_connections 2024-02-17T12:49:41.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.146+0000 7f66766a7640 1 --2- 172.21.15.18:0/1513226359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6670100d70 0x7f66701011f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.146+0000 7f66766a7640 1 --2- 172.21.15.18:0/1513226359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66701003d0 0x7f6670100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.146+0000 7f66766a7640 1 --2- 172.21.15.18:0/1513226359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66700ffa70 0x7f66700ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.146+0000 7f66766a7640 1 -- 172.21.15.18:0/1513226359 >> 172.21.15.18:0/1513226359 conn(0x7f66700fb610 msgr2=0x7f66700fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:41.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.146+0000 7f66766a7640 1 -- 172.21.15.18:0/1513226359 shutdown_connections 2024-02-17T12:49:41.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.146+0000 7f66766a7640 1 -- 172.21.15.18:0/1513226359 wait complete. 2024-02-17T12:49:41.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.146+0000 7f66766a7640 1 Processor -- start 2024-02-17T12:49:41.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.146+0000 7f66766a7640 1 -- start start 2024-02-17T12:49:41.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f66766a7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66700ffa70 0x7f66701993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:41.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f66766a7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66701003d0 0x7f66701998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:41.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f66766a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6670100d70 0x7f66701a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:41.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f66766a7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66701a0e90 con 0x7f66700ffa70 2024-02-17T12:49:41.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f66766a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66701a1000 con 0x7f6670100d70 2024-02-17T12:49:41.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f66766a7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66701a11c0 con 0x7f66701003d0 2024-02-17T12:49:41.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f666ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66700ffa70 0x7f66701993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:41.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f666ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66700ffa70 0x7f66701993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34450/0 (socket says 172.21.15.18:34450) 2024-02-17T12:49:41.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f666ffff640 1 -- 172.21.15.18:0/2747482111 learned_addr learned my addr 172.21.15.18:0/2747482111 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:41.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f666f7fe640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66701003d0 0x7f66701998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:41.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f6674c44640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6670100d70 0x7f66701a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:41.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f666ffff640 1 -- 172.21.15.18:0/2747482111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66701003d0 msgr2=0x7f66701998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f666ffff640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66701003d0 0x7f66701998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f666ffff640 1 -- 172.21.15.18:0/2747482111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6670100d70 msgr2=0x7f66701a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f666ffff640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6670100d70 0x7f66701a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f666ffff640 1 -- 172.21.15.18:0/2747482111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6668009900 con 0x7f66700ffa70 2024-02-17T12:49:41.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.147+0000 7f666ffff640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66700ffa70 0x7f66701993a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f666000d950 tx=0x7f666000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:41.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.148+0000 7f666d7fa640 1 -- 172.21.15.18:0/2747482111 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660009a30 con 0x7f66700ffa70 2024-02-17T12:49:41.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.148+0000 7f666d7fa640 1 -- 172.21.15.18:0/2747482111 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6660009b90 con 0x7f66700ffa70 2024-02-17T12:49:41.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.148+0000 7f66766a7640 1 -- 172.21.15.18:0/2747482111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66701a14a0 con 0x7f66700ffa70 2024-02-17T12:49:41.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.148+0000 7f666d7fa640 1 -- 172.21.15.18:0/2747482111 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660004a00 con 0x7f66700ffa70 2024-02-17T12:49:41.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.148+0000 7f66766a7640 1 -- 172.21.15.18:0/2747482111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66701a19b0 con 0x7f66700ffa70 2024-02-17T12:49:41.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.149+0000 7f66766a7640 1 -- 172.21.15.18:0/2747482111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667010a920 con 0x7f66700ffa70 2024-02-17T12:49:41.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.151+0000 7f666d7fa640 1 -- 172.21.15.18:0/2747482111 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6660009d00 con 0x7f66700ffa70 2024-02-17T12:49:41.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.151+0000 7f666d7fa640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f665c0a20c0 0x7f665c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:41.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.151+0000 7f666f7fe640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f665c0a20c0 0x7f665c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:41.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.151+0000 7f666d7fa640 1 -- 172.21.15.18:0/2747482111 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f66600cd7b0 con 0x7f66700ffa70 2024-02-17T12:49:41.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.152+0000 7f666f7fe640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f665c0a20c0 0x7f665c0a4580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6658005e00 tx=0x7f6658005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:41.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.156+0000 7f666d7fa640 1 -- 172.21.15.18:0/2747482111 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f66600967b0 con 0x7f66700ffa70 2024-02-17T12:49:41.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.376+0000 7f66766a7640 1 -- 172.21.15.18:0/2747482111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f667010edc0 con 0x7f66700ffa70 2024-02-17T12:49:41.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.378+0000 7f666d7fa640 1 -- 172.21.15.18:0/2747482111 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11392 (secure 0 0 0) 0x7f6660099e70 con 0x7f66700ffa70 2024-02-17T12:49:41.373 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:41.373 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:49:36.672377+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:49:41.373 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 26 2024-02-17T12:49:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.379+0000 7f66766a7640 1 -- 172.21.15.18:0/2747482111 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f665c0a20c0 msgr2=0x7f665c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.379+0000 7f66766a7640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f665c0a20c0 0x7f665c0a4580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6658005e00 tx=0x7f6658005c50 comp rx=0 tx=0).stop 2024-02-17T12:49:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.380+0000 7f66766a7640 1 -- 172.21.15.18:0/2747482111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66700ffa70 msgr2=0x7f66701993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.380+0000 7f66766a7640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66700ffa70 0x7f66701993a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f666000d950 tx=0x7f666000de10 comp rx=0 tx=0).stop 2024-02-17T12:49:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.380+0000 7f66766a7640 1 -- 172.21.15.18:0/2747482111 shutdown_connections 2024-02-17T12:49:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.380+0000 7f66766a7640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f665c0a20c0 0x7f665c0a4580 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.380+0000 7f66766a7640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6670100d70 0x7f66701a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.380+0000 7f66766a7640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66701003d0 0x7f66701998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.380+0000 7f66766a7640 1 --2- 172.21.15.18:0/2747482111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66700ffa70 0x7f66701993a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.380+0000 7f66766a7640 1 -- 172.21.15.18:0/2747482111 >> 172.21.15.18:0/2747482111 conn(0x7f66700fb610 msgr2=0x7f66700fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:41.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.380+0000 7f66766a7640 1 -- 172.21.15.18:0/2747482111 shutdown_connections 2024-02-17T12:49:41.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.380+0000 7f66766a7640 1 -- 172.21.15.18:0/2747482111 wait complete. 2024-02-17T12:49:41.385 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-02-17T12:49:41.385 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-02-17T12:49:41.385 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-17T12:49:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.500+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3267251933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc81000a0 msgr2=0x7f1fc81004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.500+0000 7f1fcf2f4640 1 --2- 172.21.15.18:0/3267251933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc81000a0 0x7f1fc81004c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009900 tx=0x7f1fc402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.500+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3267251933 shutdown_connections 2024-02-17T12:49:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.500+0000 7f1fcf2f4640 1 --2- 172.21.15.18:0/3267251933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc8101c50 0x7f1fc81084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.500+0000 7f1fcf2f4640 1 --2- 172.21.15.18:0/3267251933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc8101290 0x7f1fc8101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.500+0000 7f1fcf2f4640 1 --2- 172.21.15.18:0/3267251933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc81000a0 0x7f1fc81004c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.500+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3267251933 >> 172.21.15.18:0/3267251933 conn(0x7f1fc80fb610 msgr2=0x7f1fc80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.500+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3267251933 shutdown_connections 2024-02-17T12:49:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.500+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3267251933 wait complete. 2024-02-17T12:49:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcf2f4640 1 Processor -- start 2024-02-17T12:49:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcf2f4640 1 -- start start 2024-02-17T12:49:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcf2f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc81000a0 0x7f1fc8104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcf2f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc8101290 0x7f1fc8102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcf2f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc8101c50 0x7f1fc8102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcf2f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc8103310 con 0x7f1fc8101290 2024-02-17T12:49:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcf2f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc8103480 con 0x7f1fc8101c50 2024-02-17T12:49:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcf2f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc81a12e0 con 0x7f1fc81000a0 2024-02-17T12:49:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcc88f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc8101290 0x7f1fc8102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcc88f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc8101290 0x7f1fc8102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34476/0 (socket says 172.21.15.18:34476) 2024-02-17T12:49:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcc88f640 1 -- 172.21.15.18:0/3317022877 learned_addr learned my addr 172.21.15.18:0/3317022877 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcd090640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc81000a0 0x7f1fc8104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcc88f640 1 -- 172.21.15.18:0/3317022877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc81000a0 msgr2=0x7f1fc8104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcc88f640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc81000a0 0x7f1fc8104030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcc88f640 1 -- 172.21.15.18:0/3317022877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc8101c50 msgr2=0x7f1fc8102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.501+0000 7f1fcd891640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc8101c50 0x7f1fc8102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.502+0000 7f1fcc88f640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc8101c50 0x7f1fc8102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.502+0000 7f1fcc88f640 1 -- 172.21.15.18:0/3317022877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc4009560 con 0x7f1fc8101290 2024-02-17T12:49:41.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.502+0000 7f1fcc88f640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc8101290 0x7f1fc8102670 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800ea20 tx=0x7f1fb800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:41.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.502+0000 7f1fbe7fc640 1 -- 172.21.15.18:0/3317022877 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8009700 con 0x7f1fc8101290 2024-02-17T12:49:41.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.502+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3317022877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc81a1590 con 0x7f1fc8101290 2024-02-17T12:49:41.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.502+0000 7f1fbe7fc640 1 -- 172.21.15.18:0/3317022877 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fb800cd80 con 0x7f1fc8101290 2024-02-17T12:49:41.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.502+0000 7f1fbe7fc640 1 -- 172.21.15.18:0/3317022877 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8018840 con 0x7f1fc8101290 2024-02-17T12:49:41.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.502+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3317022877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc81a1af0 con 0x7f1fc8101290 2024-02-17T12:49:41.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.503+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3317022877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc810bab0 con 0x7f1fc8101290 2024-02-17T12:49:41.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.504+0000 7f1fbe7fc640 1 -- 172.21.15.18:0/3317022877 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1fb8010a80 con 0x7f1fc8101290 2024-02-17T12:49:41.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.505+0000 7f1fbe7fc640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1fb40a20c0 0x7f1fb40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:41.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.505+0000 7f1fcd090640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1fb40a20c0 0x7f1fb40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:41.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.505+0000 7f1fbe7fc640 1 -- 172.21.15.18:0/3317022877 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1fb8014070 con 0x7f1fc8101290 2024-02-17T12:49:41.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.505+0000 7f1fcd090640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1fb40a20c0 0x7f1fb40a4580 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4006010 tx=0x7f1fc4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:41.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.509+0000 7f1fbe7fc640 1 -- 172.21.15.18:0/3317022877 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1fb8052d20 con 0x7f1fc8101290 2024-02-17T12:49:41.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.690+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3317022877 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1fc80fd810 con 0x7f1fb40a20c0 2024-02-17T12:49:41.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.701+0000 7f1fbe7fc640 1 -- 172.21.15.18:0/3317022877 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344756 (secure 0 0 0) 0x7f1fc80fd810 con 0x7f1fb40a20c0 2024-02-17T12:49:41.698 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:41.700 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-02-17T12:49:41.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.704+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3317022877 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1fb40a20c0 msgr2=0x7f1fb40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.704+0000 7f1fcf2f4640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1fb40a20c0 0x7f1fb40a4580 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4006010 tx=0x7f1fc4000bc0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.704+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3317022877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc8101290 msgr2=0x7f1fc8102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.704+0000 7f1fcf2f4640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc8101290 0x7f1fc8102670 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800ea20 tx=0x7f1fb800eee0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.704+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3317022877 shutdown_connections 2024-02-17T12:49:41.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.704+0000 7f1fcf2f4640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f1fb40a20c0 0x7f1fb40a4580 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.704+0000 7f1fcf2f4640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc8101c50 0x7f1fc8102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.704+0000 7f1fcf2f4640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc8101290 0x7f1fc8102670 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.704+0000 7f1fcf2f4640 1 --2- 172.21.15.18:0/3317022877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc81000a0 0x7f1fc8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.704+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3317022877 >> 172.21.15.18:0/3317022877 conn(0x7f1fc80fb610 msgr2=0x7f1fc80fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:41.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.705+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3317022877 shutdown_connections 2024-02-17T12:49:41.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.705+0000 7f1fcf2f4640 1 -- 172.21.15.18:0/3317022877 wait complete. 2024-02-17T12:49:41.707 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:41 smithi018 ceph-mon[32625]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 11 op/s 2024-02-17T12:49:41.707 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:41 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3313977282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:41.707 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:41 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3103363957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:41.707 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:41 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2747482111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:41.708 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-02-17T12:49:40.219511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658039,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1247,"num_read_kb":747,"num_write":430,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181749,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":854,"ondisk_log_size":854,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383404,"kb_used_data":13452,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558420,"statfs":{"total":1151940427776,"available":1151547822080,"internally_reserved":0,"allocated":13774848,"data_stored":7504557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":66,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002916"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768991+0000","last_change":"2024-02-17T12:47:46.069941+0000","last_active":"2024-02-17T12:49:20.768991+0000","last_peered":"2024-02-17T12:49:20.768991+0000","last_clean":"2024-02-17T12:49:20.768991+0000","last_became_active":"2024-02-17T12:47:46.069192+0000","last_became_peered":"2024-02-17T12:47:46.069192+0000","last_unstale":"2024-02-17T12:49:20.768991+0000","last_undegraded":"2024-02-17T12:49:20.768991+0000","last_fullsized":"2024-02-17T12:49:20.768991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:27.102839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.760213+0000","last_change":"2024-02-17T12:47:46.064872+0000","last_active":"2024-02-17T12:49:20.760213+0000","last_peered":"2024-02-17T12:49:20.760213+0000","last_clean":"2024-02-17T12:49:20.760213+0000","last_became_active":"2024-02-17T12:47:46.064498+0000","last_became_peered":"2024-02-17T12:47:46.064498+0000","last_unstale":"2024-02-17T12:49:20.760213+0000","last_undegraded":"2024-02-17T12:49:20.760213+0000","last_fullsized":"2024-02-17T12:49:20.760213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:30:41.432657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762825+0000","last_change":"2024-02-17T12:47:46.067504+0000","last_active":"2024-02-17T12:49:20.762825+0000","last_peered":"2024-02-17T12:49:20.762825+0000","last_clean":"2024-02-17T12:49:20.762825+0000","last_became_active":"2024-02-17T12:47:46.067155+0000","last_became_peered":"2024-02-17T12:47:46.067155+0000","last_unstale":"2024-02-17T12:49:20.762825+0000","last_undegraded":"2024-02-17T12:49:20.762825+0000","last_fullsized":"2024-02-17T12:49:20.762825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:38:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764046+0000","last_change":"2024-02-17T12:47:46.062356+0000","last_active":"2024-02-17T12:49:20.764046+0000","last_peered":"2024-02-17T12:49:20.764046+0000","last_clean":"2024-02-17T12:49:20.764046+0000","last_became_active":"2024-02-17T12:47:46.061988+0000","last_became_peered":"2024-02-17T12:47:46.061988+0000","last_unstale":"2024-02-17T12:49:20.764046+0000","last_undegraded":"2024-02-17T12:49:20.764046+0000","last_fullsized":"2024-02-17T12:49:20.764046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:25:09.120770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.746039+0000","last_change":"2024-02-17T12:47:46.072411+0000","last_active":"2024-02-17T12:49:20.746039+0000","last_peered":"2024-02-17T12:49:20.746039+0000","last_clean":"2024-02-17T12:49:20.746039+0000","last_became_active":"2024-02-17T12:47:46.072067+0000","last_became_peered":"2024-02-17T12:47:46.072067+0000","last_unstale":"2024-02-17T12:49:20.746039+0000","last_undegraded":"2024-02-17T12:49:20.746039+0000","last_fullsized":"2024-02-17T12:49:20.746039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:29:33.051513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.754893+0000","last_change":"2024-02-17T12:47:46.057425+0000","last_active":"2024-02-17T12:49:20.754893+0000","last_peered":"2024-02-17T12:49:20.754893+0000","last_clean":"2024-02-17T12:49:20.754893+0000","last_became_active":"2024-02-17T12:47:46.057001+0000","last_became_peered":"2024-02-17T12:47:46.057001+0000","last_unstale":"2024-02-17T12:49:20.754893+0000","last_undegraded":"2024-02-17T12:49:20.754893+0000","last_fullsized":"2024-02-17T12:49:20.754893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:48:24.135879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758496+0000","last_change":"2024-02-17T12:47:46.069143+0000","last_active":"2024-02-17T12:49:20.758496+0000","last_peered":"2024-02-17T12:49:20.758496+0000","last_clean":"2024-02-17T12:49:20.758496+0000","last_became_active":"2024-02-17T12:47:46.068775+0000","last_became_peered":"2024-02-17T12:47:46.068775+0000","last_unstale":"2024-02-17T12:49:20.758496+0000","last_undegraded":"2024-02-17T12:49:20.758496+0000","last_fullsized":"2024-02-17T12:49:20.758496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:32:39.518205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764182+0000","last_change":"2024-02-17T12:47:46.069579+0000","last_active":"2024-02-17T12:49:20.764182+0000","last_peered":"2024-02-17T12:49:20.764182+0000","last_clean":"2024-02-17T12:49:20.764182+0000","last_became_active":"2024-02-17T12:47:46.069114+0000","last_became_peered":"2024-02-17T12:47:46.069114+0000","last_unstale":"2024-02-17T12:49:20.764182+0000","last_undegraded":"2024-02-17T12:49:20.764182+0000","last_fullsized":"2024-02-17T12:49:20.764182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:54:58.100511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768271+0000","last_change":"2024-02-17T12:47:46.079103+0000","last_active":"2024-02-17T12:49:20.768271+0000","last_peered":"2024-02-17T12:49:20.768271+0000","last_clean":"2024-02-17T12:49:20.768271+0000","last_became_active":"2024-02-17T12:47:46.077992+0000","last_became_peered":"2024-02-17T12:47:46.077992+0000","last_unstale":"2024-02-17T12:49:20.768271+0000","last_undegraded":"2024-02-17T12:49:20.768271+0000","last_fullsized":"2024-02-17T12:49:20.768271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:12:59.337341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758225+0000","last_change":"2024-02-17T12:47:46.062897+0000","last_active":"2024-02-17T12:49:20.758225+0000","last_peered":"2024-02-17T12:49:20.758225+0000","last_clean":"2024-02-17T12:49:20.758225+0000","last_became_active":"2024-02-17T12:47:46.062571+0000","last_became_peered":"2024-02-17T12:47:46.062571+0000","last_unstale":"2024-02-17T12:49:20.758225+0000","last_undegraded":"2024-02-17T12:49:20.758225+0000","last_fullsized":"2024-02-17T12:49:20.758225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:57:02.845400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.760478+0000","last_change":"2024-02-17T12:47:46.080183+0000","last_active":"2024-02-17T12:49:20.760478+0000","last_peered":"2024-02-17T12:49:20.760478+0000","last_clean":"2024-02-17T12:49:20.760478+0000","last_became_active":"2024-02-17T12:47:46.079827+0000","last_became_peered":"2024-02-17T12:47:46.079827+0000","last_unstale":"2024-02-17T12:49:20.760478+0000","last_undegraded":"2024-02-17T12:49:20.760478+0000","last_fullsized":"2024-02-17T12:49:20.760478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:56:10.193200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762695+0000","last_change":"2024-02-17T12:47:46.061849+0000","last_active":"2024-02-17T12:49:20.762695+0000","last_peered":"2024-02-17T12:49:20.762695+0000","last_clean":"2024-02-17T12:49:20.762695+0000","last_became_active":"2024-02-17T12:47:46.061458+0000","last_became_peered":"2024-02-17T12:47:46.061458+0000","last_unstale":"2024-02-17T12:49:20.762695+0000","last_undegraded":"2024-02-17T12:49:20.762695+0000","last_fullsized":"2024-02-17T12:49:20.762695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:02:09.141292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.765932+0000","last_change":"2024-02-17T12:47:46.069356+0000","last_active":"2024-02-17T12:49:20.765932+0000","last_peered":"2024-02-17T12:49:20.765932+0000","last_clean":"2024-02-17T12:49:20.765932+0000","last_became_active":"2024-02-17T12:47:46.068567+0000","last_became_peered":"2024-02-17T12:47:46.068567+0000","last_unstale":"2024-02-17T12:49:20.765932+0000","last_undegraded":"2024-02-17T12:49:20.765932+0000","last_fullsized":"2024-02-17T12:49:20.765932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:08:01.872558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.757541+0000","last_change":"2024-02-17T12:47:46.062684+0000","last_active":"2024-02-17T12:49:20.757541+0000","last_peered":"2024-02-17T12:49:20.757541+0000","last_clean":"2024-02-17T12:49:20.757541+0000","last_became_active":"2024-02-17T12:47:46.062221+0000","last_became_peered":"2024-02-17T12:47:46.062221+0000","last_unstale":"2024-02-17T12:49:20.757541+0000","last_undegraded":"2024-02-17T12:49:20.757541+0000","last_fullsized":"2024-02-17T12:49:20.757541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:37:11.513336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.760133+0000","last_change":"2024-02-17T12:47:46.065961+0000","last_active":"2024-02-17T12:49:20.760133+0000","last_peered":"2024-02-17T12:49:20.760133+0000","last_clean":"2024-02-17T12:49:20.760133+0000","last_became_active":"2024-02-17T12:47:46.065600+0000","last_became_peered":"2024-02-17T12:47:46.065600+0000","last_unstale":"2024-02-17T12:49:20.760133+0000","last_undegraded":"2024-02-17T12:49:20.760133+0000","last_fullsized":"2024-02-17T12:49:20.760133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:24:46.205077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762547+0000","last_change":"2024-02-17T12:47:46.080150+0000","last_active":"2024-02-17T12:49:20.762547+0000","last_peered":"2024-02-17T12:49:20.762547+0000","last_clean":"2024-02-17T12:49:20.762547+0000","last_became_active":"2024-02-17T12:47:46.079813+0000","last_became_peered":"2024-02-17T12:47:46.079813+0000","last_unstale":"2024-02-17T12:49:20.762547+0000","last_undegraded":"2024-02-17T12:49:20.762547+0000","last_fullsized":"2024-02-17T12:49:20.762547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:59:00.868964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.769045+0000","last_change":"2024-02-17T12:47:46.070202+0000","last_active":"2024-02-17T12:49:20.769045+0000","last_peered":"2024-02-17T12:49:20.769045+0000","last_clean":"2024-02-17T12:49:20.769045+0000","last_became_active":"2024-02-17T12:47:46.069628+0000","last_became_peered":"2024-02-17T12:47:46.069628+0000","last_unstale":"2024-02-17T12:49:20.769045+0000","last_undegraded":"2024-02-17T12:49:20.769045+0000","last_fullsized":"2024-02-17T12:49:20.769045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:44:17.481681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763818+0000","last_change":"2024-02-17T12:47:46.067810+0000","last_active":"2024-02-17T12:49:20.763818+0000","last_peered":"2024-02-17T12:49:20.763818+0000","last_clean":"2024-02-17T12:49:20.763818+0000","last_became_active":"2024-02-17T12:47:46.067456+0000","last_became_peered":"2024-02-17T12:47:46.067456+0000","last_unstale":"2024-02-17T12:49:20.763818+0000","last_undegraded":"2024-02-17T12:49:20.763818+0000","last_fullsized":"2024-02-17T12:49:20.763818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:27:03.646597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.753377+0000","last_change":"2024-02-17T12:47:46.076268+0000","last_active":"2024-02-17T12:49:20.753377+0000","last_peered":"2024-02-17T12:49:20.753377+0000","last_clean":"2024-02-17T12:49:20.753377+0000","last_became_active":"2024-02-17T12:47:46.075606+0000","last_became_peered":"2024-02-17T12:47:46.075606+0000","last_unstale":"2024-02-17T12:49:20.753377+0000","last_undegraded":"2024-02-17T12:49:20.753377+0000","last_fullsized":"2024-02-17T12:49:20.753377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:16:35.805008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":137,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:33.865136+0000","last_change":"2024-02-17T12:47:45.066780+0000","last_active":"2024-02-17T12:49:33.865136+0000","last_peered":"2024-02-17T12:49:33.865136+0000","last_clean":"2024-02-17T12:49:33.865136+0000","last_became_active":"2024-02-17T12:47:45.066039+0000","last_became_peered":"2024-02-17T12:47:45.066039+0000","last_unstale":"2024-02-17T12:49:33.865136+0000","last_undegraded":"2024-02-17T12:49:33.865136+0000","last_fullsized":"2024-02-17T12:49:33.865136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:36:25.420534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763169+0000","last_change":"2024-02-17T12:47:46.079603+0000","last_active":"2024-02-17T12:49:20.763169+0000","last_peered":"2024-02-17T12:49:20.763169+0000","last_clean":"2024-02-17T12:49:20.763169+0000","last_became_active":"2024-02-17T12:47:46.079246+0000","last_became_peered":"2024-02-17T12:47:46.079246+0000","last_unstale":"2024-02-17T12:49:20.763169+0000","last_undegraded":"2024-02-17T12:49:20.763169+0000","last_fullsized":"2024-02-17T12:49:20.763169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:32:04.975778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165983+0000","last_change":"2024-02-17T12:47:45.061868+0000","last_active":"2024-02-17T12:48:52.165983+0000","last_peered":"2024-02-17T12:48:52.165983+0000","last_clean":"2024-02-17T12:48:52.165983+0000","last_became_active":"2024-02-17T12:47:45.061512+0000","last_became_peered":"2024-02-17T12:47:45.061512+0000","last_unstale":"2024-02-17T12:48:52.165983+0000","last_undegraded":"2024-02-17T12:48:52.165983+0000","last_fullsized":"2024-02-17T12:48:52.165983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:53:48.405184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763185+0000","last_change":"2024-02-17T12:47:46.071055+0000","last_active":"2024-02-17T12:49:20.763185+0000","last_peered":"2024-02-17T12:49:20.763185+0000","last_clean":"2024-02-17T12:49:20.763185+0000","last_became_active":"2024-02-17T12:47:46.070649+0000","last_became_peered":"2024-02-17T12:47:46.070649+0000","last_unstale":"2024-02-17T12:49:20.763185+0000","last_undegraded":"2024-02-17T12:49:20.763185+0000","last_fullsized":"2024-02-17T12:49:20.763185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:10:22.078746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.178268+0000","last_change":"2024-02-17T12:48:52.177755+0000","last_active":"2024-02-17T12:48:52.178268+0000","last_peered":"2024-02-17T12:48:52.178268+0000","last_clean":"2024-02-17T12:48:52.178268+0000","last_became_active":"2024-02-17T12:48:52.165119+0000","last_became_peered":"2024-02-17T12:48:52.165119+0000","last_unstale":"2024-02-17T12:48:52.178268+0000","last_undegraded":"2024-02-17T12:48:52.178268+0000","last_fullsized":"2024-02-17T12:48:52.178268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:38.834337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767093+0000","last_change":"2024-02-17T12:47:46.076448+0000","last_active":"2024-02-17T12:49:20.767093+0000","last_peered":"2024-02-17T12:49:20.767093+0000","last_clean":"2024-02-17T12:49:20.767093+0000","last_became_active":"2024-02-17T12:47:46.075250+0000","last_became_peered":"2024-02-17T12:47:46.075250+0000","last_unstale":"2024-02-17T12:49:20.767093+0000","last_undegraded":"2024-02-17T12:49:20.767093+0000","last_fullsized":"2024-02-17T12:49:20.767093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:11:45.846764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163335+0000","last_change":"2024-02-17T12:47:45.062699+0000","last_active":"2024-02-17T12:48:52.163335+0000","last_peered":"2024-02-17T12:48:52.163335+0000","last_clean":"2024-02-17T12:48:52.163335+0000","last_became_active":"2024-02-17T12:47:45.062322+0000","last_became_peered":"2024-02-17T12:47:45.062322+0000","last_unstale":"2024-02-17T12:48:52.163335+0000","last_undegraded":"2024-02-17T12:48:52.163335+0000","last_fullsized":"2024-02-17T12:48:52.163335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:14:06.112393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767082+0000","last_change":"2024-02-17T12:47:46.080278+0000","last_active":"2024-02-17T12:49:20.767082+0000","last_peered":"2024-02-17T12:49:20.767082+0000","last_clean":"2024-02-17T12:49:20.767082+0000","last_became_active":"2024-02-17T12:47:46.079924+0000","last_became_peered":"2024-02-17T12:47:46.079924+0000","last_unstale":"2024-02-17T12:49:20.767082+0000","last_undegraded":"2024-02-17T12:49:20.767082+0000","last_fullsized":"2024-02-17T12:49:20.767082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:54:08.759445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":152,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:37.543298+0000","last_change":"2024-02-17T12:48:52.469734+0000","last_active":"2024-02-17T12:49:37.543298+0000","last_peered":"2024-02-17T12:49:37.543298+0000","last_clean":"2024-02-17T12:49:37.543298+0000","last_became_active":"2024-02-17T12:48:52.465945+0000","last_became_peered":"2024-02-17T12:48:52.465945+0000","last_unstale":"2024-02-17T12:49:37.543298+0000","last_undegraded":"2024-02-17T12:49:37.543298+0000","last_fullsized":"2024-02-17T12:49:37.543298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:05:27.931619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.749939+0000","last_change":"2024-02-17T12:47:46.065957+0000","last_active":"2024-02-17T12:49:20.749939+0000","last_peered":"2024-02-17T12:49:20.749939+0000","last_clean":"2024-02-17T12:49:20.749939+0000","last_became_active":"2024-02-17T12:47:46.065638+0000","last_became_peered":"2024-02-17T12:47:46.065638+0000","last_unstale":"2024-02-17T12:49:20.749939+0000","last_undegraded":"2024-02-17T12:49:20.749939+0000","last_fullsized":"2024-02-17T12:49:20.749939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:02:39.295990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463600+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:48:52.463600+0000","last_peered":"2024-02-17T12:48:52.463600+0000","last_clean":"2024-02-17T12:48:52.463600+0000","last_became_active":"2024-02-17T12:47:45.066235+0000","last_became_peered":"2024-02-17T12:47:45.066235+0000","last_unstale":"2024-02-17T12:48:52.463600+0000","last_undegraded":"2024-02-17T12:48:52.463600+0000","last_fullsized":"2024-02-17T12:48:52.463600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:46:53.731920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763770+0000","last_change":"2024-02-17T12:47:46.072029+0000","last_active":"2024-02-17T12:49:20.763770+0000","last_peered":"2024-02-17T12:49:20.763770+0000","last_clean":"2024-02-17T12:49:20.763770+0000","last_became_active":"2024-02-17T12:47:46.071716+0000","last_became_peered":"2024-02-17T12:47:46.071716+0000","last_unstale":"2024-02-17T12:49:20.763770+0000","last_undegraded":"2024-02-17T12:49:20.763770+0000","last_fullsized":"2024-02-17T12:49:20.763770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:33.076266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163802+0000","last_change":"2024-02-17T12:47:45.067443+0000","last_active":"2024-02-17T12:48:52.163802+0000","last_peered":"2024-02-17T12:48:52.163802+0000","last_clean":"2024-02-17T12:48:52.163802+0000","last_became_active":"2024-02-17T12:47:45.067090+0000","last_became_peered":"2024-02-17T12:47:45.067090+0000","last_unstale":"2024-02-17T12:48:52.163802+0000","last_undegraded":"2024-02-17T12:48:52.163802+0000","last_fullsized":"2024-02-17T12:48:52.163802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:17:39.874810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768189+0000","last_change":"2024-02-17T12:47:46.072275+0000","last_active":"2024-02-17T12:49:20.768189+0000","last_peered":"2024-02-17T12:49:20.768189+0000","last_clean":"2024-02-17T12:49:20.768189+0000","last_became_active":"2024-02-17T12:47:46.071951+0000","last_became_peered":"2024-02-17T12:47:46.071951+0000","last_unstale":"2024-02-17T12:49:20.768189+0000","last_undegraded":"2024-02-17T12:49:20.768189+0000","last_fullsized":"2024-02-17T12:49:20.768189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:45:58.580854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159772+0000","last_change":"2024-02-17T12:47:45.058875+0000","last_active":"2024-02-17T12:48:52.159772+0000","last_peered":"2024-02-17T12:48:52.159772+0000","last_clean":"2024-02-17T12:48:52.159772+0000","last_became_active":"2024-02-17T12:47:45.058433+0000","last_became_peered":"2024-02-17T12:47:45.058433+0000","last_unstale":"2024-02-17T12:48:52.159772+0000","last_undegraded":"2024-02-17T12:48:52.159772+0000","last_fullsized":"2024-02-17T12:48:52.159772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:49.932205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762407+0000","last_change":"2024-02-17T12:47:46.070233+0000","last_active":"2024-02-17T12:49:20.762407+0000","last_peered":"2024-02-17T12:49:20.762407+0000","last_clean":"2024-02-17T12:49:20.762407+0000","last_became_active":"2024-02-17T12:47:46.069811+0000","last_became_peered":"2024-02-17T12:47:46.069811+0000","last_unstale":"2024-02-17T12:49:20.762407+0000","last_undegraded":"2024-02-17T12:49:20.762407+0000","last_fullsized":"2024-02-17T12:49:20.762407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:50:00.885517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":242,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:38.544493+0000","last_change":"2024-02-17T12:47:45.068778+0000","last_active":"2024-02-17T12:49:38.544493+0000","last_peered":"2024-02-17T12:49:38.544493+0000","last_clean":"2024-02-17T12:49:38.544493+0000","last_became_active":"2024-02-17T12:47:45.068375+0000","last_became_peered":"2024-02-17T12:47:45.068375+0000","last_unstale":"2024-02-17T12:49:38.544493+0000","last_undegraded":"2024-02-17T12:49:38.544493+0000","last_fullsized":"2024-02-17T12:49:38.544493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:42:57.600138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":218,"num_read_kb":218,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.765647+0000","last_change":"2024-02-17T12:47:46.076802+0000","last_active":"2024-02-17T12:49:20.765647+0000","last_peered":"2024-02-17T12:49:20.765647+0000","last_clean":"2024-02-17T12:49:20.765647+0000","last_became_active":"2024-02-17T12:47:46.076433+0000","last_became_peered":"2024-02-17T12:47:46.076433+0000","last_unstale":"2024-02-17T12:49:20.765647+0000","last_undegraded":"2024-02-17T12:49:20.765647+0000","last_fullsized":"2024-02-17T12:49:20.765647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.352488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157571+0000","last_change":"2024-02-17T12:47:45.059100+0000","last_active":"2024-02-17T12:48:52.157571+0000","last_peered":"2024-02-17T12:48:52.157571+0000","last_clean":"2024-02-17T12:48:52.157571+0000","last_became_active":"2024-02-17T12:47:45.058707+0000","last_became_peered":"2024-02-17T12:47:45.058707+0000","last_unstale":"2024-02-17T12:48:52.157571+0000","last_undegraded":"2024-02-17T12:48:52.157571+0000","last_fullsized":"2024-02-17T12:48:52.157571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:28:27.044127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.757936+0000","last_change":"2024-02-17T12:47:46.065088+0000","last_active":"2024-02-17T12:49:20.757936+0000","last_peered":"2024-02-17T12:49:20.757936+0000","last_clean":"2024-02-17T12:49:20.757936+0000","last_became_active":"2024-02-17T12:47:46.064754+0000","last_became_peered":"2024-02-17T12:47:46.064754+0000","last_unstale":"2024-02-17T12:49:20.757936+0000","last_undegraded":"2024-02-17T12:49:20.757936+0000","last_fullsized":"2024-02-17T12:49:20.757936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:33:10.652881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160868+0000","last_change":"2024-02-17T12:47:45.055403+0000","last_active":"2024-02-17T12:48:52.160868+0000","last_peered":"2024-02-17T12:48:52.160868+0000","last_clean":"2024-02-17T12:48:52.160868+0000","last_became_active":"2024-02-17T12:47:45.055083+0000","last_became_peered":"2024-02-17T12:47:45.055083+0000","last_unstale":"2024-02-17T12:48:52.160868+0000","last_undegraded":"2024-02-17T12:48:52.160868+0000","last_fullsized":"2024-02-17T12:48:52.160868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:05:51.295074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767519+0000","last_change":"2024-02-17T12:47:46.074883+0000","last_active":"2024-02-17T12:49:20.767519+0000","last_peered":"2024-02-17T12:49:20.767519+0000","last_clean":"2024-02-17T12:49:20.767519+0000","last_became_active":"2024-02-17T12:47:46.074414+0000","last_became_peered":"2024-02-17T12:47:46.074414+0000","last_unstale":"2024-02-17T12:49:20.767519+0000","last_undegraded":"2024-02-17T12:49:20.767519+0000","last_fullsized":"2024-02-17T12:49:20.767519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:35:55.026587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":282,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:38.544476+0000","last_change":"2024-02-17T12:47:45.057627+0000","last_active":"2024-02-17T12:49:38.544476+0000","last_peered":"2024-02-17T12:49:38.544476+0000","last_clean":"2024-02-17T12:49:38.544476+0000","last_became_active":"2024-02-17T12:47:45.057297+0000","last_became_peered":"2024-02-17T12:47:45.057297+0000","last_unstale":"2024-02-17T12:49:38.544476+0000","last_undegraded":"2024-02-17T12:49:38.544476+0000","last_fullsized":"2024-02-17T12:49:38.544476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:00:17.159939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59951,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":229,"num_read_kb":173,"num_write":36,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.761631+0000","last_change":"2024-02-17T12:47:46.076854+0000","last_active":"2024-02-17T12:49:20.761631+0000","last_peered":"2024-02-17T12:49:20.761631+0000","last_clean":"2024-02-17T12:49:20.761631+0000","last_became_active":"2024-02-17T12:47:46.076534+0000","last_became_peered":"2024-02-17T12:47:46.076534+0000","last_unstale":"2024-02-17T12:49:20.761631+0000","last_undegraded":"2024-02-17T12:49:20.761631+0000","last_fullsized":"2024-02-17T12:49:20.761631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:43:01.979075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163878+0000","last_change":"2024-02-17T12:47:45.060090+0000","last_active":"2024-02-17T12:48:52.163878+0000","last_peered":"2024-02-17T12:48:52.163878+0000","last_clean":"2024-02-17T12:48:52.163878+0000","last_became_active":"2024-02-17T12:47:45.058942+0000","last_became_peered":"2024-02-17T12:47:45.058942+0000","last_unstale":"2024-02-17T12:48:52.163878+0000","last_undegraded":"2024-02-17T12:48:52.163878+0000","last_fullsized":"2024-02-17T12:48:52.163878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:30:52.247140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767896+0000","last_change":"2024-02-17T12:47:46.071982+0000","last_active":"2024-02-17T12:49:20.767896+0000","last_peered":"2024-02-17T12:49:20.767896+0000","last_clean":"2024-02-17T12:49:20.767896+0000","last_became_active":"2024-02-17T12:47:46.071639+0000","last_became_peered":"2024-02-17T12:47:46.071639+0000","last_unstale":"2024-02-17T12:49:20.767896+0000","last_undegraded":"2024-02-17T12:49:20.767896+0000","last_fullsized":"2024-02-17T12:49:20.767896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:43:50.658132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160287+0000","last_change":"2024-02-17T12:47:45.060017+0000","last_active":"2024-02-17T12:48:52.160287+0000","last_peered":"2024-02-17T12:48:52.160287+0000","last_clean":"2024-02-17T12:48:52.160287+0000","last_became_active":"2024-02-17T12:47:45.059104+0000","last_became_peered":"2024-02-17T12:47:45.059104+0000","last_unstale":"2024-02-17T12:48:52.160287+0000","last_undegraded":"2024-02-17T12:48:52.160287+0000","last_fullsized":"2024-02-17T12:48:52.160287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:56:50.457297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768545+0000","last_change":"2024-02-17T12:47:46.072250+0000","last_active":"2024-02-17T12:49:20.768545+0000","last_peered":"2024-02-17T12:49:20.768545+0000","last_clean":"2024-02-17T12:49:20.768545+0000","last_became_active":"2024-02-17T12:47:46.071856+0000","last_became_peered":"2024-02-17T12:47:46.071856+0000","last_unstale":"2024-02-17T12:49:20.768545+0000","last_undegraded":"2024-02-17T12:49:20.768545+0000","last_fullsized":"2024-02-17T12:49:20.768545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:46:04.628146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":133,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:33.907678+0000","last_change":"2024-02-17T12:47:45.056386+0000","last_active":"2024-02-17T12:49:33.907678+0000","last_peered":"2024-02-17T12:49:33.907678+0000","last_clean":"2024-02-17T12:49:33.907678+0000","last_became_active":"2024-02-17T12:47:45.055630+0000","last_became_peered":"2024-02-17T12:47:45.055630+0000","last_unstale":"2024-02-17T12:49:33.907678+0000","last_undegraded":"2024-02-17T12:49:33.907678+0000","last_fullsized":"2024-02-17T12:49:33.907678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:16.388950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.759754+0000","last_change":"2024-02-17T12:47:46.071612+0000","last_active":"2024-02-17T12:49:20.759754+0000","last_peered":"2024-02-17T12:49:20.759754+0000","last_clean":"2024-02-17T12:49:20.759754+0000","last_became_active":"2024-02-17T12:47:46.071237+0000","last_became_peered":"2024-02-17T12:47:46.071237+0000","last_unstale":"2024-02-17T12:49:20.759754+0000","last_undegraded":"2024-02-17T12:49:20.759754+0000","last_fullsized":"2024-02-17T12:49:20.759754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:24:01.099679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163081+0000","last_change":"2024-02-17T12:47:45.066258+0000","last_active":"2024-02-17T12:48:52.163081+0000","last_peered":"2024-02-17T12:48:52.163081+0000","last_clean":"2024-02-17T12:48:52.163081+0000","last_became_active":"2024-02-17T12:47:45.065813+0000","last_became_peered":"2024-02-17T12:47:45.065813+0000","last_unstale":"2024-02-17T12:48:52.163081+0000","last_undegraded":"2024-02-17T12:48:52.163081+0000","last_fullsized":"2024-02-17T12:48:52.163081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:06:43.298573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763292+0000","last_change":"2024-02-17T12:47:46.080347+0000","last_active":"2024-02-17T12:49:20.763292+0000","last_peered":"2024-02-17T12:49:20.763292+0000","last_clean":"2024-02-17T12:49:20.763292+0000","last_became_active":"2024-02-17T12:47:46.080083+0000","last_became_peered":"2024-02-17T12:47:46.080083+0000","last_unstale":"2024-02-17T12:49:20.763292+0000","last_undegraded":"2024-02-17T12:49:20.763292+0000","last_fullsized":"2024-02-17T12:49:20.763292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:24:41.807633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163573+0000","last_change":"2024-02-17T12:47:45.062814+0000","last_active":"2024-02-17T12:48:52.163573+0000","last_peered":"2024-02-17T12:48:52.163573+0000","last_clean":"2024-02-17T12:48:52.163573+0000","last_became_active":"2024-02-17T12:47:45.062418+0000","last_became_peered":"2024-02-17T12:47:45.062418+0000","last_unstale":"2024-02-17T12:48:52.163573+0000","last_undegraded":"2024-02-17T12:48:52.163573+0000","last_fullsized":"2024-02-17T12:48:52.163573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:49:34.577275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.759382+0000","last_change":"2024-02-17T12:47:46.067742+0000","last_active":"2024-02-17T12:49:20.759382+0000","last_peered":"2024-02-17T12:49:20.759382+0000","last_clean":"2024-02-17T12:49:20.759382+0000","last_became_active":"2024-02-17T12:47:46.067329+0000","last_became_peered":"2024-02-17T12:47:46.067329+0000","last_unstale":"2024-02-17T12:49:20.759382+0000","last_undegraded":"2024-02-17T12:49:20.759382+0000","last_fullsized":"2024-02-17T12:49:20.759382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:19.432999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156444+0000","last_change":"2024-02-17T12:47:45.065461+0000","last_active":"2024-02-17T12:48:52.156444+0000","last_peered":"2024-02-17T12:48:52.156444+0000","last_clean":"2024-02-17T12:48:52.156444+0000","last_became_active":"2024-02-17T12:47:45.065066+0000","last_became_peered":"2024-02-17T12:47:45.065066+0000","last_unstale":"2024-02-17T12:48:52.156444+0000","last_undegraded":"2024-02-17T12:48:52.156444+0000","last_fullsized":"2024-02-17T12:48:52.156444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:25:47.292901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764291+0000","last_change":"2024-02-17T12:47:46.074222+0000","last_active":"2024-02-17T12:49:20.764291+0000","last_peered":"2024-02-17T12:49:20.764291+0000","last_clean":"2024-02-17T12:49:20.764291+0000","last_became_active":"2024-02-17T12:47:46.073886+0000","last_became_peered":"2024-02-17T12:47:46.073886+0000","last_unstale":"2024-02-17T12:49:20.764291+0000","last_undegraded":"2024-02-17T12:49:20.764291+0000","last_fullsized":"2024-02-17T12:49:20.764291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:46:18.063422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":134,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:38.872177+0000","last_change":"2024-02-17T12:47:45.071414+0000","last_active":"2024-02-17T12:49:38.872177+0000","last_peered":"2024-02-17T12:49:38.872177+0000","last_clean":"2024-02-17T12:49:38.872177+0000","last_became_active":"2024-02-17T12:47:45.071127+0000","last_became_peered":"2024-02-17T12:47:45.071127+0000","last_unstale":"2024-02-17T12:49:38.872177+0000","last_undegraded":"2024-02-17T12:49:38.872177+0000","last_fullsized":"2024-02-17T12:49:38.872177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:13:21.187718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.750322+0000","last_change":"2024-02-17T12:47:46.059412+0000","last_active":"2024-02-17T12:49:20.750322+0000","last_peered":"2024-02-17T12:49:20.750322+0000","last_clean":"2024-02-17T12:49:20.750322+0000","last_became_active":"2024-02-17T12:47:46.059100+0000","last_became_peered":"2024-02-17T12:47:46.059100+0000","last_unstale":"2024-02-17T12:49:20.750322+0000","last_undegraded":"2024-02-17T12:49:20.750322+0000","last_fullsized":"2024-02-17T12:49:20.750322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:35.930613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763100+0000","last_change":"2024-02-17T12:47:46.071467+0000","last_active":"2024-02-17T12:49:20.763100+0000","last_peered":"2024-02-17T12:49:20.763100+0000","last_clean":"2024-02-17T12:49:20.763100+0000","last_became_active":"2024-02-17T12:47:46.070879+0000","last_became_peered":"2024-02-17T12:47:46.070879+0000","last_unstale":"2024-02-17T12:49:20.763100+0000","last_undegraded":"2024-02-17T12:49:20.763100+0000","last_fullsized":"2024-02-17T12:49:20.763100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.682842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":130,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:33.906943+0000","last_change":"2024-02-17T12:47:45.070545+0000","last_active":"2024-02-17T12:49:33.906943+0000","last_peered":"2024-02-17T12:49:33.906943+0000","last_clean":"2024-02-17T12:49:33.906943+0000","last_became_active":"2024-02-17T12:47:45.070265+0000","last_became_peered":"2024-02-17T12:47:45.070265+0000","last_unstale":"2024-02-17T12:49:33.906943+0000","last_undegraded":"2024-02-17T12:49:33.906943+0000","last_fullsized":"2024-02-17T12:49:33.906943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.763735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766601+0000","last_change":"2024-02-17T12:47:46.068539+0000","last_active":"2024-02-17T12:49:20.766601+0000","last_peered":"2024-02-17T12:49:20.766601+0000","last_clean":"2024-02-17T12:49:20.766601+0000","last_became_active":"2024-02-17T12:47:46.068169+0000","last_became_peered":"2024-02-17T12:47:46.068169+0000","last_unstale":"2024-02-17T12:49:20.766601+0000","last_undegraded":"2024-02-17T12:49:20.766601+0000","last_fullsized":"2024-02-17T12:49:20.766601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:27:28.485868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464598+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:48:52.464598+0000","last_peered":"2024-02-17T12:48:52.464598+0000","last_clean":"2024-02-17T12:48:52.464598+0000","last_became_active":"2024-02-17T12:47:45.066299+0000","last_became_peered":"2024-02-17T12:47:45.066299+0000","last_unstale":"2024-02-17T12:48:52.464598+0000","last_undegraded":"2024-02-17T12:48:52.464598+0000","last_fullsized":"2024-02-17T12:48:52.464598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:34:02.040002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157254+0000","last_change":"2024-02-17T12:46:32.556576+0000","last_active":"2024-02-17T12:48:52.157254+0000","last_peered":"2024-02-17T12:48:52.157254+0000","last_clean":"2024-02-17T12:48:52.157254+0000","last_became_active":"2024-02-17T12:46:32.556160+0000","last_became_peered":"2024-02-17T12:46:32.556160+0000","last_unstale":"2024-02-17T12:48:52.157254+0000","last_undegraded":"2024-02-17T12:48:52.157254+0000","last_fullsized":"2024-02-17T12:48:52.157254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_clean_scrub_stamp":"2024-02-17T12:43:13.147248+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:57:52.244951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767280+0000","last_change":"2024-02-17T12:48:52.166240+0000","last_active":"2024-02-17T12:49:20.767280+0000","last_peered":"2024-02-17T12:49:20.767280+0000","last_clean":"2024-02-17T12:49:20.767280+0000","last_became_active":"2024-02-17T12:48:52.165882+0000","last_became_peered":"2024-02-17T12:48:52.165882+0000","last_unstale":"2024-02-17T12:49:20.767280+0000","last_undegraded":"2024-02-17T12:49:20.767280+0000","last_fullsized":"2024-02-17T12:49:20.767280+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:56:44.360362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157273+0000","last_change":"2024-02-17T12:47:45.061112+0000","last_active":"2024-02-17T12:48:52.157273+0000","last_peered":"2024-02-17T12:48:52.157273+0000","last_clean":"2024-02-17T12:48:52.157273+0000","last_became_active":"2024-02-17T12:47:45.060633+0000","last_became_peered":"2024-02-17T12:47:45.060633+0000","last_unstale":"2024-02-17T12:48:52.157273+0000","last_undegraded":"2024-02-17T12:48:52.157273+0000","last_fullsized":"2024-02-17T12:48:52.157273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:29:29.421744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163918+0000","last_change":"2024-02-17T12:47:45.066897+0000","last_active":"2024-02-17T12:48:52.163918+0000","last_peered":"2024-02-17T12:48:52.163918+0000","last_clean":"2024-02-17T12:48:52.163918+0000","last_became_active":"2024-02-17T12:47:45.066357+0000","last_became_peered":"2024-02-17T12:47:45.066357+0000","last_unstale":"2024-02-17T12:48:52.163918+0000","last_undegraded":"2024-02-17T12:48:52.163918+0000","last_fullsized":"2024-02-17T12:48:52.163918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:50:09.840414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767577+0000","last_change":"2024-02-17T12:47:46.072412+0000","last_active":"2024-02-17T12:49:20.767577+0000","last_peered":"2024-02-17T12:49:20.767577+0000","last_clean":"2024-02-17T12:49:20.767577+0000","last_became_active":"2024-02-17T12:47:46.072105+0000","last_became_peered":"2024-02-17T12:47:46.072105+0000","last_unstale":"2024-02-17T12:49:20.767577+0000","last_undegraded":"2024-02-17T12:49:20.767577+0000","last_fullsized":"2024-02-17T12:49:20.767577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:33:19.367588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'13","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156660+0000","last_change":"2024-02-17T12:47:45.067673+0000","last_active":"2024-02-17T12:48:52.156660+0000","last_peered":"2024-02-17T12:48:52.156660+0000","last_clean":"2024-02-17T12:48:52.156660+0000","last_became_active":"2024-02-17T12:47:45.067212+0000","last_became_peered":"2024-02-17T12:47:45.067212+0000","last_unstale":"2024-02-17T12:48:52.156660+0000","last_undegraded":"2024-02-17T12:48:52.156660+0000","last_fullsized":"2024-02-17T12:48:52.156660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:15:04.236158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763306+0000","last_change":"2024-02-17T12:47:46.077476+0000","last_active":"2024-02-17T12:49:20.763306+0000","last_peered":"2024-02-17T12:49:20.763306+0000","last_clean":"2024-02-17T12:49:20.763306+0000","last_became_active":"2024-02-17T12:47:46.077101+0000","last_became_peered":"2024-02-17T12:47:46.077101+0000","last_unstale":"2024-02-17T12:49:20.763306+0000","last_undegraded":"2024-02-17T12:49:20.763306+0000","last_fullsized":"2024-02-17T12:49:20.763306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:41:41.402101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464561+0000","last_change":"2024-02-17T12:47:45.066693+0000","last_active":"2024-02-17T12:48:52.464561+0000","last_peered":"2024-02-17T12:48:52.464561+0000","last_clean":"2024-02-17T12:48:52.464561+0000","last_became_active":"2024-02-17T12:47:45.066349+0000","last_became_peered":"2024-02-17T12:47:45.066349+0000","last_unstale":"2024-02-17T12:48:52.464561+0000","last_undegraded":"2024-02-17T12:48:52.464561+0000","last_fullsized":"2024-02-17T12:48:52.464561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:30.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.759614+0000","last_change":"2024-02-17T12:47:46.072899+0000","last_active":"2024-02-17T12:49:20.759614+0000","last_peered":"2024-02-17T12:49:20.759614+0000","last_clean":"2024-02-17T12:49:20.759614+0000","last_became_active":"2024-02-17T12:47:46.072488+0000","last_became_peered":"2024-02-17T12:47:46.072488+0000","last_unstale":"2024-02-17T12:49:20.759614+0000","last_undegraded":"2024-02-17T12:49:20.759614+0000","last_fullsized":"2024-02-17T12:49:20.759614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:05:43.675086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463459+0000","last_change":"2024-02-17T12:47:45.056995+0000","last_active":"2024-02-17T12:48:52.463459+0000","last_peered":"2024-02-17T12:48:52.463459+0000","last_clean":"2024-02-17T12:48:52.463459+0000","last_became_active":"2024-02-17T12:47:45.056021+0000","last_became_peered":"2024-02-17T12:47:45.056021+0000","last_unstale":"2024-02-17T12:48:52.463459+0000","last_undegraded":"2024-02-17T12:48:52.463459+0000","last_fullsized":"2024-02-17T12:48:52.463459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:41.444513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.722229+0000","last_change":"2024-02-17T12:47:46.065859+0000","last_active":"2024-02-17T12:49:20.722229+0000","last_peered":"2024-02-17T12:49:20.722229+0000","last_clean":"2024-02-17T12:49:20.722229+0000","last_became_active":"2024-02-17T12:47:46.065469+0000","last_became_peered":"2024-02-17T12:47:46.065469+0000","last_unstale":"2024-02-17T12:49:20.722229+0000","last_undegraded":"2024-02-17T12:49:20.722229+0000","last_fullsized":"2024-02-17T12:49:20.722229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:51:25.105361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165363+0000","last_change":"2024-02-17T12:47:45.066524+0000","last_active":"2024-02-17T12:48:52.165363+0000","last_peered":"2024-02-17T12:48:52.165363+0000","last_clean":"2024-02-17T12:48:52.165363+0000","last_became_active":"2024-02-17T12:47:45.066218+0000","last_became_peered":"2024-02-17T12:47:45.066218+0000","last_unstale":"2024-02-17T12:48:52.165363+0000","last_undegraded":"2024-02-17T12:48:52.165363+0000","last_fullsized":"2024-02-17T12:48:52.165363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:56:47.244262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764154+0000","last_change":"2024-02-17T12:47:46.065779+0000","last_active":"2024-02-17T12:49:20.764154+0000","last_peered":"2024-02-17T12:49:20.764154+0000","last_clean":"2024-02-17T12:49:20.764154+0000","last_became_active":"2024-02-17T12:47:46.065459+0000","last_became_peered":"2024-02-17T12:47:46.065459+0000","last_unstale":"2024-02-17T12:49:20.764154+0000","last_undegraded":"2024-02-17T12:49:20.764154+0000","last_fullsized":"2024-02-17T12:49:20.764154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:08.944487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163588+0000","last_change":"2024-02-17T12:47:45.059639+0000","last_active":"2024-02-17T12:48:52.163588+0000","last_peered":"2024-02-17T12:48:52.163588+0000","last_clean":"2024-02-17T12:48:52.163588+0000","last_became_active":"2024-02-17T12:47:45.059085+0000","last_became_peered":"2024-02-17T12:47:45.059085+0000","last_unstale":"2024-02-17T12:48:52.163588+0000","last_undegraded":"2024-02-17T12:48:52.163588+0000","last_fullsized":"2024-02-17T12:48:52.163588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:48:55.847405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.756470+0000","last_change":"2024-02-17T12:47:46.079623+0000","last_active":"2024-02-17T12:49:20.756470+0000","last_peered":"2024-02-17T12:49:20.756470+0000","last_clean":"2024-02-17T12:49:20.756470+0000","last_became_active":"2024-02-17T12:47:46.079285+0000","last_became_peered":"2024-02-17T12:47:46.079285+0000","last_unstale":"2024-02-17T12:49:20.756470+0000","last_undegraded":"2024-02-17T12:49:20.756470+0000","last_fullsized":"2024-02-17T12:49:20.756470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:45:14.441218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.181532+0000","last_change":"2024-02-17T12:48:52.181057+0000","last_active":"2024-02-17T12:48:52.181532+0000","last_peered":"2024-02-17T12:48:52.181532+0000","last_clean":"2024-02-17T12:48:52.181532+0000","last_became_active":"2024-02-17T12:48:52.165968+0000","last_became_peered":"2024-02-17T12:48:52.165968+0000","last_unstale":"2024-02-17T12:48:52.181532+0000","last_undegraded":"2024-02-17T12:48:52.181532+0000","last_fullsized":"2024-02-17T12:48:52.181532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:14:06.153057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766320+0000","last_change":"2024-02-17T12:47:46.074429+0000","last_active":"2024-02-17T12:49:20.766320+0000","last_peered":"2024-02-17T12:49:20.766320+0000","last_clean":"2024-02-17T12:49:20.766320+0000","last_became_active":"2024-02-17T12:47:46.074123+0000","last_became_peered":"2024-02-17T12:47:46.074123+0000","last_unstale":"2024-02-17T12:49:20.766320+0000","last_undegraded":"2024-02-17T12:49:20.766320+0000","last_fullsized":"2024-02-17T12:49:20.766320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:50:17.633831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":132,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:35.869454+0000","last_change":"2024-02-17T12:47:45.061362+0000","last_active":"2024-02-17T12:49:35.869454+0000","last_peered":"2024-02-17T12:49:35.869454+0000","last_clean":"2024-02-17T12:49:35.869454+0000","last_became_active":"2024-02-17T12:47:45.061037+0000","last_became_peered":"2024-02-17T12:47:45.061037+0000","last_unstale":"2024-02-17T12:49:35.869454+0000","last_undegraded":"2024-02-17T12:49:35.869454+0000","last_fullsized":"2024-02-17T12:49:35.869454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:01.987157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764578+0000","last_change":"2024-02-17T12:47:46.073862+0000","last_active":"2024-02-17T12:49:20.764578+0000","last_peered":"2024-02-17T12:49:20.764578+0000","last_clean":"2024-02-17T12:49:20.764578+0000","last_became_active":"2024-02-17T12:47:46.073543+0000","last_became_peered":"2024-02-17T12:47:46.073543+0000","last_unstale":"2024-02-17T12:49:20.764578+0000","last_undegraded":"2024-02-17T12:49:20.764578+0000","last_fullsized":"2024-02-17T12:49:20.764578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:57:57.567726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160633+0000","last_change":"2024-02-17T12:47:45.060616+0000","last_active":"2024-02-17T12:48:52.160633+0000","last_peered":"2024-02-17T12:48:52.160633+0000","last_clean":"2024-02-17T12:48:52.160633+0000","last_became_active":"2024-02-17T12:47:45.060296+0000","last_became_peered":"2024-02-17T12:47:45.060296+0000","last_unstale":"2024-02-17T12:48:52.160633+0000","last_undegraded":"2024-02-17T12:48:52.160633+0000","last_fullsized":"2024-02-17T12:48:52.160633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:22:44.791363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758731+0000","last_change":"2024-02-17T12:47:46.072738+0000","last_active":"2024-02-17T12:49:20.758731+0000","last_peered":"2024-02-17T12:49:20.758731+0000","last_clean":"2024-02-17T12:49:20.758731+0000","last_became_active":"2024-02-17T12:47:46.072421+0000","last_became_peered":"2024-02-17T12:47:46.072421+0000","last_unstale":"2024-02-17T12:49:20.758731+0000","last_undegraded":"2024-02-17T12:49:20.758731+0000","last_fullsized":"2024-02-17T12:49:20.758731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:50:23.501787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.470091+0000","last_change":"2024-02-17T12:48:52.470091+0000","last_active":"2024-02-17T12:48:52.470091+0000","last_peered":"2024-02-17T12:48:52.470091+0000","last_clean":"2024-02-17T12:48:52.470091+0000","last_became_active":"2024-02-17T12:48:52.466153+0000","last_became_peered":"2024-02-17T12:48:52.466153+0000","last_unstale":"2024-02-17T12:48:52.470091+0000","last_undegraded":"2024-02-17T12:48:52.470091+0000","last_fullsized":"2024-02-17T12:48:52.470091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:03:45.027983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764385+0000","last_change":"2024-02-17T12:47:46.066072+0000","last_active":"2024-02-17T12:49:20.764385+0000","last_peered":"2024-02-17T12:49:20.764385+0000","last_clean":"2024-02-17T12:49:20.764385+0000","last_became_active":"2024-02-17T12:47:46.065755+0000","last_became_peered":"2024-02-17T12:47:46.065755+0000","last_unstale":"2024-02-17T12:49:20.764385+0000","last_undegraded":"2024-02-17T12:49:20.764385+0000","last_fullsized":"2024-02-17T12:49:20.764385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:09:29.534097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763980+0000","last_change":"2024-02-17T12:47:46.067268+0000","last_active":"2024-02-17T12:49:20.763980+0000","last_peered":"2024-02-17T12:49:20.763980+0000","last_clean":"2024-02-17T12:49:20.763980+0000","last_became_active":"2024-02-17T12:47:46.066918+0000","last_became_peered":"2024-02-17T12:47:46.066918+0000","last_unstale":"2024-02-17T12:49:20.763980+0000","last_undegraded":"2024-02-17T12:49:20.763980+0000","last_fullsized":"2024-02-17T12:49:20.763980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:11:09.844954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.757364+0000","last_change":"2024-02-17T12:47:46.068143+0000","last_active":"2024-02-17T12:49:20.757364+0000","last_peered":"2024-02-17T12:49:20.757364+0000","last_clean":"2024-02-17T12:49:20.757364+0000","last_became_active":"2024-02-17T12:47:46.065112+0000","last_became_peered":"2024-02-17T12:47:46.065112+0000","last_unstale":"2024-02-17T12:49:20.757364+0000","last_undegraded":"2024-02-17T12:49:20.757364+0000","last_fullsized":"2024-02-17T12:49:20.757364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:43:04.982009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.748955+0000","last_change":"2024-02-17T12:47:46.076139+0000","last_active":"2024-02-17T12:49:20.748955+0000","last_peered":"2024-02-17T12:49:20.748955+0000","last_clean":"2024-02-17T12:49:20.748955+0000","last_became_active":"2024-02-17T12:47:46.075826+0000","last_became_peered":"2024-02-17T12:47:46.075826+0000","last_unstale":"2024-02-17T12:49:20.748955+0000","last_undegraded":"2024-02-17T12:49:20.748955+0000","last_fullsized":"2024-02-17T12:49:20.748955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:39:08.460716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766396+0000","last_change":"2024-02-17T12:47:46.063792+0000","last_active":"2024-02-17T12:49:20.766396+0000","last_peered":"2024-02-17T12:49:20.766396+0000","last_clean":"2024-02-17T12:49:20.766396+0000","last_became_active":"2024-02-17T12:47:46.063407+0000","last_became_peered":"2024-02-17T12:47:46.063407+0000","last_unstale":"2024-02-17T12:49:20.766396+0000","last_undegraded":"2024-02-17T12:49:20.766396+0000","last_fullsized":"2024-02-17T12:49:20.766396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:26:26.973435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768046+0000","last_change":"2024-02-17T12:47:46.066202+0000","last_active":"2024-02-17T12:49:20.768046+0000","last_peered":"2024-02-17T12:49:20.768046+0000","last_clean":"2024-02-17T12:49:20.768046+0000","last_became_active":"2024-02-17T12:47:46.065820+0000","last_became_peered":"2024-02-17T12:47:46.065820+0000","last_unstale":"2024-02-17T12:49:20.768046+0000","last_undegraded":"2024-02-17T12:49:20.768046+0000","last_fullsized":"2024-02-17T12:49:20.768046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:36:25.201697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.759899+0000","last_change":"2024-02-17T12:48:52.166996+0000","last_active":"2024-02-17T12:49:20.759899+0000","last_peered":"2024-02-17T12:49:20.759899+0000","last_clean":"2024-02-17T12:49:20.759899+0000","last_became_active":"2024-02-17T12:48:52.166589+0000","last_became_peered":"2024-02-17T12:48:52.166589+0000","last_unstale":"2024-02-17T12:49:20.759899+0000","last_undegraded":"2024-02-17T12:49:20.759899+0000","last_fullsized":"2024-02-17T12:49:20.759899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:16:30.392183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763854+0000","last_change":"2024-02-17T12:47:46.070811+0000","last_active":"2024-02-17T12:49:20.763854+0000","last_peered":"2024-02-17T12:49:20.763854+0000","last_clean":"2024-02-17T12:49:20.763854+0000","last_became_active":"2024-02-17T12:47:46.070462+0000","last_became_peered":"2024-02-17T12:47:46.070462+0000","last_unstale":"2024-02-17T12:49:20.763854+0000","last_undegraded":"2024-02-17T12:49:20.763854+0000","last_fullsized":"2024-02-17T12:49:20.763854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:03.671869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758608+0000","last_change":"2024-02-17T12:47:46.077692+0000","last_active":"2024-02-17T12:49:20.758608+0000","last_peered":"2024-02-17T12:49:20.758608+0000","last_clean":"2024-02-17T12:49:20.758608+0000","last_became_active":"2024-02-17T12:47:46.077363+0000","last_became_peered":"2024-02-17T12:47:46.077363+0000","last_unstale":"2024-02-17T12:49:20.758608+0000","last_undegraded":"2024-02-17T12:49:20.758608+0000","last_fullsized":"2024-02-17T12:49:20.758608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:02:49.068782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758794+0000","last_change":"2024-02-17T12:47:46.073507+0000","last_active":"2024-02-17T12:49:20.758794+0000","last_peered":"2024-02-17T12:49:20.758794+0000","last_clean":"2024-02-17T12:49:20.758794+0000","last_became_active":"2024-02-17T12:47:46.073107+0000","last_became_peered":"2024-02-17T12:47:46.073107+0000","last_unstale":"2024-02-17T12:49:20.758794+0000","last_undegraded":"2024-02-17T12:49:20.758794+0000","last_fullsized":"2024-02-17T12:49:20.758794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:32:48.799300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762538+0000","last_change":"2024-02-17T12:47:46.073384+0000","last_active":"2024-02-17T12:49:20.762538+0000","last_peered":"2024-02-17T12:49:20.762538+0000","last_clean":"2024-02-17T12:49:20.762538+0000","last_became_active":"2024-02-17T12:47:46.073017+0000","last_became_peered":"2024-02-17T12:47:46.073017+0000","last_unstale":"2024-02-17T12:49:20.762538+0000","last_undegraded":"2024-02-17T12:49:20.762538+0000","last_fullsized":"2024-02-17T12:49:20.762538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:43:50.091667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766804+0000","last_change":"2024-02-17T12:47:46.073758+0000","last_active":"2024-02-17T12:49:20.766804+0000","last_peered":"2024-02-17T12:49:20.766804+0000","last_clean":"2024-02-17T12:49:20.766804+0000","last_became_active":"2024-02-17T12:47:46.072781+0000","last_became_peered":"2024-02-17T12:47:46.072781+0000","last_unstale":"2024-02-17T12:49:20.766804+0000","last_undegraded":"2024-02-17T12:49:20.766804+0000","last_fullsized":"2024-02-17T12:49:20.766804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:10:24.811100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766956+0000","last_change":"2024-02-17T12:47:46.072025+0000","last_active":"2024-02-17T12:49:20.766956+0000","last_peered":"2024-02-17T12:49:20.766956+0000","last_clean":"2024-02-17T12:49:20.766956+0000","last_became_active":"2024-02-17T12:47:46.071662+0000","last_became_peered":"2024-02-17T12:47:46.071662+0000","last_unstale":"2024-02-17T12:49:20.766956+0000","last_undegraded":"2024-02-17T12:49:20.766956+0000","last_fullsized":"2024-02-17T12:49:20.766956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:04:07.188552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762073+0000","last_change":"2024-02-17T12:48:52.166041+0000","last_active":"2024-02-17T12:49:20.762073+0000","last_peered":"2024-02-17T12:49:20.762073+0000","last_clean":"2024-02-17T12:49:20.762073+0000","last_became_active":"2024-02-17T12:48:52.165576+0000","last_became_peered":"2024-02-17T12:48:52.165576+0000","last_unstale":"2024-02-17T12:49:20.762073+0000","last_undegraded":"2024-02-17T12:49:20.762073+0000","last_fullsized":"2024-02-17T12:49:20.762073+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:26.810385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67437,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1111,"num_read_kb":628,"num_write":247,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1013,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":462062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.40699999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.443}]},{"osd":1,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.56399999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82899999999999996,"5min":0.874,"15min":0.874},"last":0.67100000000000004}]},{"osd":2,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69999999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.54900000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90700000000000003,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.51600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83699999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80000000000000004,"5min":1.002,"15min":1.002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93100000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84599999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.67100000000000004,"15min":0.67800000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":1.121,"15min":1.121},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77600000000000002,"15min":0.77300000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.748,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91800000000000004,"5min":1.103,"15min":1.103},"last":0.873}]},{"osd":8,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.77800000000000002}]},{"osd":9,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.157,"15min":1.157},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.78800000000000003,"15min":0.78300000000000003},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.054,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.82399999999999995}]},{"osd":10,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79400000000000004,"15min":0.78800000000000003},"min":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98699999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.80900000000000005}]}]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1113574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":13.85,"5min":13.85,"15min":13.85},"last":0.56999999999999995}]},{"osd":1,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.48199999999999998,"15min":0.503},"min":{"1min":0.31900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.629,"5min":0.751,"15min":0.751},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.52300000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65600000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.43099999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68100000000000005,"15min":0.69099999999999995},"min":{"1min":0.49099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76600000000000001,"5min":1.002,"15min":1.002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.52800000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83499999999999996,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.65600000000000003}]},{"osd":3,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.57199999999999995,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.64400000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.499},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.64200000000000002,"15min":0.66800000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.74299999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.51400000000000001,"15min":0.53700000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64500000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.58099999999999996,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78000000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.52400000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71499999999999997,"15min":0.73999999999999999},"min":{"1min":0.47999999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.88,"5min":1.022,"15min":1.022},"last":0.623},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67500000000000004,"15min":0.69699999999999995},"min":{"1min":0.47299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83599999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65800000000000003,"15min":0.68999999999999995},"min":{"1min":0.438,"5min":0.437,"15min":0.437},"max":{"1min":0.748,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.73399999999999999,"15min":0.76000000000000001},"min":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85299999999999998,"5min":1.085,"15min":1.085},"last":0.47699999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.54900000000000004}]},{"osd":8,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.74099999999999999,"15min":0.77400000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.81000000000000005,"5min":1.107,"15min":1.107},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.80300000000000005,"15min":0.83299999999999996},"min":{"1min":0.61099999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92000000000000004,"5min":1.119,"15min":1.119},"last":0.753}]},{"osd":9,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.80400000000000005,"15min":0.84799999999999998},"min":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94399999999999995,"5min":1.151,"15min":1.151},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.78000000000000003,"15min":0.82199999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.91300000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.59599999999999997}]},{"osd":11,"last update":"Sat Feb 17 12:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77700000000000002,"15min":0.77900000000000003},"min":{"1min":0.63800000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.93899999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75,"15min":0.752},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68799999999999994}]}]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40460,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704692,"statfs":{"total":95995035648,"available":95953604608,"internally_reserved":0,"allocated":987136,"data_stored":461992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.436,"15min":0.45400000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.60199999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.39900000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63100000000000001,"15min":0.65300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.5}]},{"osd":2,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63300000000000001,"15min":0.64700000000000002},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.0409999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.316,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.879,"5min":1.488,"15min":1.488},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.58299999999999996,"15min":0.61199999999999999},"min":{"1min":0.29999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83999999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2030000000000001,"5min":1.218,"15min":1.218},"last":0.622},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65100000000000002,"15min":0.66700000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95999999999999996,"5min":1.159,"15min":1.159},"last":0.58799999999999997}]},{"osd":5,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.68200000000000005,"15min":0.70599999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0029999999999999,"5min":1.323,"15min":1.323},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.73999999999999999,"15min":0.76800000000000002},"min":{"1min":0.46100000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92100000000000004,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.77700000000000002}]},{"osd":6,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67300000000000004,"15min":0.70699999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.79700000000000004,"5min":1.268,"15min":1.268},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.60999999999999999,"15min":0.63400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85799999999999998,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.56699999999999995}]},{"osd":7,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.73899999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0760000000000001,"5min":1.27,"15min":1.27},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77800000000000002,"15min":0.80800000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.236,"5min":1.351,"15min":1.351},"last":0.66600000000000004}]},{"osd":8,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73599999999999999,"15min":0.754},"min":{"1min":0.498,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.749},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.76600000000000001,"15min":0.79400000000000004},"min":{"1min":0.52100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1160000000000001,"5min":1.363,"15min":1.363},"last":0.76200000000000001}]},{"osd":10,"last update":"Sat Feb 17 12:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.80700000000000005,"5min":1.228,"15min":1.228},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78300000000000003,"15min":0.81000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.73099999999999998}]}]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":462370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.35899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.52400000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.91800000000000004,"15min":1.054},"min":{"1min":0.441,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89000000000000001,"5min":10.955,"15min":10.955},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.434,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.84999999999999998}]},{"osd":2,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.44900000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.443}]},{"osd":3,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.36499999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.71899999999999997}]},{"osd":4,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.93300000000000005,"15min":1.0680000000000001},"min":{"1min":0.46100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":10.972,"15min":10.972},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.92800000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.372,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.34899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.60099999999999998}]},{"osd":6,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.878},"min":{"1min":0.53200000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97499999999999998,"5min":4.9269999999999996,"15min":4.9269999999999996},"last":0.81599999999999995}]},{"osd":7,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.49399999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.628},{"interface":"front","average":{"1min":0.755,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":1.022,"15min":1.022},"last":0.68500000000000005}]},{"osd":9,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.72999999999999998,"15min":0.69799999999999995},"min":{"1min":0.64500000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.77700000000000002,"15min":0.75},"min":{"1min":0.65000000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.98799999999999999}]},{"osd":10,"last update":"Sat Feb 17 12:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.753,"15min":0.74399999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.748},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.1339999999999999,"15min":1.268},"min":{"1min":0.54700000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.083,"5min":10.987,"15min":10.987},"last":0.89900000000000002}]},{"osd":11,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.875},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.91600000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1055561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42899999999999999,"15min":0.41299999999999998},"min":{"1min":0.33700000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57699999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.46100000000000002,"15min":0.439},"min":{"1min":0.38200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68000000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.318}]},{"osd":2,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.48899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.75800000000000001,"15min":0.63300000000000001},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77900000000000003,"5min":15.003,"15min":15.003},"last":0.54400000000000004}]},{"osd":3,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60199999999999998,"15min":0.61799999999999999},"min":{"1min":0.45600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68600000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.60299999999999998,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76100000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56000000000000005}]},{"osd":4,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.36599999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71999999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.39900000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82399999999999995,"5min":0.872,"15min":0.872},"last":0.35899999999999999}]},{"osd":5,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.81100000000000005,"15min":0.68899999999999995},"min":{"1min":0.44600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.92700000000000005,"5min":15.028,"15min":15.028},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.54500000000000004,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86199999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59099999999999997}]},{"osd":6,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66700000000000004,"15min":0.68200000000000005},"min":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.80600000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.499},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.93300000000000005,"15min":0.77100000000000002},"min":{"1min":0.51700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88400000000000001,"5min":14.579000000000001,"15min":14.579000000000001},"last":0.45300000000000001}]},{"osd":8,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.60999999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96199999999999997,"5min":1.02,"15min":1.02},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75700000000000001,"15min":0.73799999999999999},"min":{"1min":0.65500000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.77400000000000002}]},{"osd":9,"last update":"Sat Feb 17 12:49:14 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":0.84699999999999998,"15min":0.79900000000000004},"min":{"1min":0.48999999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.403,"5min":10.403,"15min":10.403},"last":0.51900000000000002},{"interface":"front","average":{"1min":1.2410000000000001,"5min":0.98999999999999999,"15min":0.84199999999999997},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":10.471,"5min":14.598000000000001,"15min":14.598000000000001},"last":0.65800000000000003}]},{"osd":10,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.68000000000000005,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91400000000000003,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.59299999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82799999999999996,"5min":1.103,"15min":1.103},"last":0.64200000000000002}]},{"osd":11,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.876}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":461709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.38100000000000001,"15min":0.371},"min":{"1min":0.26100000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.49399999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.28100000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53600000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.32600000000000001}]},{"osd":1,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.40400000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.62,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":5.0179999999999998,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":0.59499999999999997}]},{"osd":2,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.65300000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.63200000000000001,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":5.0469999999999997,"5min":5.0469999999999997,"15min":5.0469999999999997},"last":0.57199999999999995}]},{"osd":3,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.441,"15min":0.443},"min":{"1min":0.29099999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.52900000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.624,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42899999999999999}]},{"osd":4,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73699999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.623},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.53900000000000003}]},{"osd":5,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74199999999999999,"5min":1.284,"15min":1.284},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75900000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.65600000000000003}]},{"osd":7,"last update":"Sat Feb 17 12:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72099999999999997,"15min":0.67000000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92200000000000004,"5min":5.069,"15min":5.069},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.64100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.58099999999999996}]},{"osd":8,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.73399999999999999,"15min":0.70799999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":5.0940000000000003,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90800000000000003,"5min":1.252,"15min":1.252},"last":0.68799999999999994}]},{"osd":9,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.65500000000000003,"15min":0.68500000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90600000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.67700000000000005,"15min":0.70699999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85499999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.49299999999999999}]},{"osd":10,"last update":"Sat Feb 17 12:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.749,"15min":0.76300000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.88500000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":522573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.40500000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.67800000000000005,"5min":0.73999999999999999,"15min":0.78100000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.69499999999999995,"15min":0.55500000000000005},"min":{"1min":0.42699999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79600000000000004,"5min":14.91,"15min":14.91},"last":0.67600000000000005}]},{"osd":2,"last update":"Sat Feb 17 12:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.59899999999999998,"15min":0.52700000000000002},"min":{"1min":0.51000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.434},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.60499999999999998,"15min":0.53000000000000003},"min":{"1min":0.61599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61899999999999999}]},{"osd":3,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.48599999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78500000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.432,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73599999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.74199999999999999}]},{"osd":4,"last update":"Sat Feb 17 12:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.44600000000000001,"15min":0.39900000000000002},"min":{"1min":0.34699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.77600000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.46700000000000003,"15min":0.42899999999999999},"min":{"1min":0.35699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.80700000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46000000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.65300000000000002,"15min":0.59199999999999997},"min":{"1min":0.63200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98799999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.60699999999999998},"min":{"1min":0.50900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82399999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72299999999999998}]},{"osd":6,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61599999999999999,"15min":0.59499999999999997},"min":{"1min":0.59999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86699999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.53700000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.77200000000000002}]},{"osd":7,"last update":"Sat Feb 17 12:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.60099999999999998,"15min":0.55900000000000005},"min":{"1min":0.41099999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.66200000000000003,"15min":0.60899999999999999},"min":{"1min":0.53200000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95899999999999996,"5min":1.079,"15min":1.079},"last":0.65300000000000002}]},{"osd":8,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.67400000000000004,"15min":0.63400000000000001},"min":{"1min":0.63800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.58999999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59499999999999997}]},{"osd":9,"last update":"Sat Feb 17 12:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72499999999999998,"15min":0.70699999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.747,"15min":0.71899999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.85899999999999999}]},{"osd":10,"last update":"Sat Feb 17 12:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.57899999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97799999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.78100000000000003,"15min":0.745},"min":{"1min":0.52800000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83199999999999996}]},{"osd":11,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.81799999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":463048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53900000000000003,"15min":0.49099999999999999},"min":{"1min":0.36899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55300000000000005,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64400000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60899999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.437,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.748,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.73699999999999999},"min":{"1min":0.42299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81999999999999995,"5min":8.3919999999999995,"15min":8.3919999999999995},"last":0.67300000000000004}]},{"osd":3,"last update":"Sat Feb 17 12:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.39000000000000001}]},{"osd":4,"last update":"Sat Feb 17 12:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.58899999999999997,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":0.872,"15min":0.872},"last":0.59499999999999997}]},{"osd":5,"last update":"Sat Feb 17 12:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.42099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68999999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.48099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.748,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63}]},{"osd":6,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.46600000000000003,"15min":0.45800000000000002},"min":{"1min":0.32500000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58699999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.49399999999999999,"15min":0.51000000000000001},"min":{"1min":0.30099999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65600000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.502}]},{"osd":7,"last update":"Sat Feb 17 12:47:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.443},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64600000000000002,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.65300000000000002,"15min":0.66500000000000004},"min":{"1min":0.45000000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79500000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.75800000000000001}]},{"osd":9,"last update":"Sat Feb 17 12:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.58499999999999996,"15min":0.622},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61799999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.60299999999999998,"15min":0.64900000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.59799999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.64500000000000002}]},{"osd":10,"last update":"Sat Feb 17 12:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77700000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.71999999999999997,"15min":0.73599999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.72199999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.77400000000000002}]},{"osd":11,"last update":"Sat Feb 17 12:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.60799999999999998,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.84299999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.57599999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.79700000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":522171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.50700000000000001,"15min":0.46899999999999997},"min":{"1min":0.40200000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.79600000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.48699999999999999,"15min":0.47099999999999997},"min":{"1min":0.34599999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68799999999999994,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57899999999999996}]},{"osd":1,"last update":"Sat Feb 17 12:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.56399999999999995,"15min":0.51400000000000001},"min":{"1min":0.45200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.83299999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.57599999999999996,"15min":0.51800000000000002},"min":{"1min":0.41199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.84799999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.78100000000000003}]},{"osd":3,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56000000000000005,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.77200000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66300000000000003}]},{"osd":4,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.76400000000000001}]},{"osd":5,"last update":"Sat Feb 17 12:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.48899999999999999,"15min":0.44900000000000001},"min":{"1min":0.35799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.92100000000000004,"5min":1.325,"15min":1.325},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.505,"15min":0.49299999999999999},"min":{"1min":0.42099999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83799999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Feb 17 12:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.47299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.59199999999999997,"15min":0.60099999999999998},"min":{"1min":0.39100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80600000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63100000000000001}]},{"osd":7,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.60999999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.85699999999999998,"5min":0.998,"15min":0.998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60899999999999999,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95899999999999996,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.83799999999999997}]},{"osd":9,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.502,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86399999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.51400000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93600000000000005,"5min":1.284,"15min":1.284},"last":0.747}]},{"osd":10,"last update":"Sat Feb 17 12:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.55200000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92500000000000004,"5min":1.347,"15min":1.347},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.72899999999999998,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97199999999999998,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.80300000000000005}]},{"osd":11,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.81200000000000006}]}]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40460,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704692,"statfs":{"total":95995035648,"available":95953604608,"internally_reserved":0,"allocated":987136,"data_stored":461534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.32300000000000001},"min":{"1min":0.26000000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59799999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.255},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.33700000000000002},"min":{"1min":0.26300000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.28499999999999998}]},{"osd":1,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77100000000000002,"5min":0.879,"15min":0.879},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.44800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.752,"5min":0.88,"15min":0.88},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77900000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.38100000000000001}]},{"osd":4,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.39600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.44600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.74299999999999999,"5min":0.876,"15min":0.876},"last":0.59499999999999997}]},{"osd":5,"last update":"Sat Feb 17 12:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.50600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72999999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81399999999999995,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.41099999999999998}]},{"osd":6,"last update":"Sat Feb 17 12:48:53 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.504,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73799999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.495,"15min":0.505},"min":{"1min":0.35099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.316}]},{"osd":7,"last update":"Sat Feb 17 12:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.625,"15min":0.621},"min":{"1min":0.47099999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78600000000000003,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57599999999999996}]},{"osd":8,"last update":"Sat Feb 17 12:47:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91600000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.71299999999999997,"15min":0.72599999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0069999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.56899999999999995}]},{"osd":9,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.64300000000000002,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.81799999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.67200000000000004,"15min":0.70599999999999996},"min":{"1min":0.44900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85899999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.497}]},{"osd":10,"last update":"Sat Feb 17 12:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.75,"15min":0.78300000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84199999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.622},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.76700000000000002},"min":{"1min":0.60499999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.77900000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64200000000000002}]},{"osd":11,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.54000000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":463256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57899999999999996}]},{"osd":1,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.36299999999999999},"min":{"1min":0.28499999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.60899999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.41599999999999998,"15min":0.378},"min":{"1min":0.35099999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.58799999999999997,"5min":0.751,"15min":0.751},"last":0.42999999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.80000000000000004,"15min":0.64000000000000001},"min":{"1min":0.42699999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85899999999999999,"5min":18.73,"15min":18.73},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55100000000000005}]},{"osd":3,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.47799999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85199999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.627},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40000000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.51000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64000000000000001,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Feb 17 12:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.57499999999999996},"min":{"1min":0.47899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79800000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.60499999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.54900000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83699999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.53700000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:48:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.83699999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.69299999999999995}]},{"osd":9,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89200000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.71099999999999997}]},{"osd":10,"last update":"Sat Feb 17 12:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.499,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92600000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.49299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76600000000000001}]},{"osd":11,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.75},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.79800000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1054707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.371,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.82599999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.625,"5min":1.02,"15min":1.02},"last":0.59299999999999997}]},{"osd":1,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.53000000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60099999999999998,"15min":0.53900000000000003},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.81699999999999995}]},{"osd":2,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.435,"15min":0.39300000000000002},"min":{"1min":0.34300000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.45000000000000001,"15min":0.38700000000000001},"min":{"1min":0.34399999999999997,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.71199999999999997}]},{"osd":3,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.54000000000000004},"min":{"1min":0.48299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84299999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.64800000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61799999999999999,"15min":0.55900000000000005},"min":{"1min":0.41899999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93999999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61099999999999999,"15min":0.54000000000000004},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.78700000000000003}]},{"osd":6,"last update":"Sat Feb 17 12:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.621,"15min":0.58799999999999997},"min":{"1min":0.56299999999999994,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98799999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.628,"15min":0.61099999999999999},"min":{"1min":0.53400000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79900000000000004}]},{"osd":7,"last update":"Sat Feb 17 12:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70699999999999996,"15min":0.68400000000000005},"min":{"1min":0.47799999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.86199999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.629,"15min":0.625},"min":{"1min":0.51200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.46600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.82999999999999996}]},{"osd":9,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68899999999999995,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.878},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.58799999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77900000000000003}]},{"osd":10,"last update":"Sat Feb 17 12:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77100000000000002,"15min":0.746},"min":{"1min":0.57999999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.1100000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72899999999999998,"15min":0.69199999999999995},"min":{"1min":0.55700000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.89100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T12:49:41.715 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-17T12:49:41.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:41 smithi119 ceph-mon[36188]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 11 op/s 2024-02-17T12:49:41.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:41 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3313977282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:41.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:41 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3103363957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:41.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:41 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2747482111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:41.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.860+0000 7f3112bbc640 1 -- 172.21.15.18:0/3696368882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c106e80 msgr2=0x7f310c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.860+0000 7f3112bbc640 1 --2- 172.21.15.18:0/3696368882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c106e80 0x7f310c1092d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3104009ca0 tx=0x7f310402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.860+0000 7f3112bbc640 1 -- 172.21.15.18:0/3696368882 shutdown_connections 2024-02-17T12:49:41.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.860+0000 7f3112bbc640 1 --2- 172.21.15.18:0/3696368882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c106e80 0x7f310c1092d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.860+0000 7f3112bbc640 1 --2- 172.21.15.18:0/3696368882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f310c104550 0x7f310c106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.860+0000 7f3112bbc640 1 --2- 172.21.15.18:0/3696368882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c101c20 0x7f310c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.860+0000 7f3112bbc640 1 -- 172.21.15.18:0/3696368882 >> 172.21.15.18:0/3696368882 conn(0x7f310c0fb610 msgr2=0x7f310c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.860+0000 7f3112bbc640 1 -- 172.21.15.18:0/3696368882 shutdown_connections 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.860+0000 7f3112bbc640 1 -- 172.21.15.18:0/3696368882 wait complete. 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.860+0000 7f3112bbc640 1 Processor -- start 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.860+0000 7f3112bbc640 1 -- start start 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3112bbc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c101c20 0x7f310c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3112bbc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f310c104550 0x7f310c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3112bbc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c106e80 0x7f310c1a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3112bbc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c1a0f10 con 0x7f310c101c20 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3112bbc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c1a1080 con 0x7f310c106e80 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3112bbc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c1a1240 con 0x7f310c104550 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3110958640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c101c20 0x7f310c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3110958640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c101c20 0x7f310c199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34482/0 (socket says 172.21.15.18:34482) 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3110958640 1 -- 172.21.15.18:0/1602167764 learned_addr learned my addr 172.21.15.18:0/1602167764 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f310bfff640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f310c104550 0x7f310c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3111159640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c106e80 0x7f310c1a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3110958640 1 -- 172.21.15.18:0/1602167764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f310c104550 msgr2=0x7f310c199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3110958640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f310c104550 0x7f310c199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3110958640 1 -- 172.21.15.18:0/1602167764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c106e80 msgr2=0x7f310c1a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3110958640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c106e80 0x7f310c1a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3110958640 1 -- 172.21.15.18:0/1602167764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3104009900 con 0x7f310c101c20 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f310bfff640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f310c104550 0x7f310c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3110958640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c101c20 0x7f310c199420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f310000d950 tx=0x7f310000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3109ffb640 1 -- 172.21.15.18:0/1602167764 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100009a30 con 0x7f310c101c20 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3109ffb640 1 -- 172.21.15.18:0/1602167764 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3100009b90 con 0x7f310c101c20 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3109ffb640 1 -- 172.21.15.18:0/1602167764 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100004a00 con 0x7f310c101c20 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.861+0000 7f3112bbc640 1 -- 172.21.15.18:0/1602167764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c1a1520 con 0x7f310c101c20 2024-02-17T12:49:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.862+0000 7f3112bbc640 1 -- 172.21.15.18:0/1602167764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c1a1a80 con 0x7f310c101c20 2024-02-17T12:49:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.863+0000 7f3112bbc640 1 -- 172.21.15.18:0/1602167764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c108970 con 0x7f310c101c20 2024-02-17T12:49:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.864+0000 7f3109ffb640 1 -- 172.21.15.18:0/1602167764 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3100009d00 con 0x7f310c101c20 2024-02-17T12:49:41.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.864+0000 7f3109ffb640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f30fc0a2140 0x7f30fc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:41.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.865+0000 7f310bfff640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f30fc0a2140 0x7f30fc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:41.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.865+0000 7f3109ffb640 1 -- 172.21.15.18:0/1602167764 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f31000cd710 con 0x7f310c101c20 2024-02-17T12:49:41.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.865+0000 7f310bfff640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f30fc0a2140 0x7f30fc0a4600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f30f8005fd0 tx=0x7f30f8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:41.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:41.869+0000 7f3109ffb640 1 -- 172.21.15.18:0/1602167764 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3100096710 con 0x7f310c101c20 2024-02-17T12:49:41.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:41 smithi146 ceph-mon[36454]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 11 op/s 2024-02-17T12:49:41.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:41 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3313977282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:41.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:41 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3103363957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:41.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:41 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2747482111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:42.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.049+0000 7f3112bbc640 1 -- 172.21.15.18:0/1602167764 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f310c02ceb0 con 0x7f30fc0a2140 2024-02-17T12:49:42.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.059+0000 7f3109ffb640 1 -- 172.21.15.18:0/1602167764 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344756 (secure 0 0 0) 0x7f310c02ceb0 con 0x7f30fc0a2140 2024-02-17T12:49:42.055 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:42.057 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-02-17T12:49:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.062+0000 7f3112bbc640 1 -- 172.21.15.18:0/1602167764 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f30fc0a2140 msgr2=0x7f30fc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.062+0000 7f3112bbc640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f30fc0a2140 0x7f30fc0a4600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f30f8005fd0 tx=0x7f30f8005850 comp rx=0 tx=0).stop 2024-02-17T12:49:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.062+0000 7f3112bbc640 1 -- 172.21.15.18:0/1602167764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c101c20 msgr2=0x7f310c199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.062+0000 7f3112bbc640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c101c20 0x7f310c199420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f310000d950 tx=0x7f310000de10 comp rx=0 tx=0).stop 2024-02-17T12:49:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.062+0000 7f3112bbc640 1 -- 172.21.15.18:0/1602167764 shutdown_connections 2024-02-17T12:49:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.062+0000 7f3112bbc640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f30fc0a2140 0x7f30fc0a4600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.062+0000 7f3112bbc640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c106e80 0x7f310c1a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.062+0000 7f3112bbc640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f310c104550 0x7f310c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.062+0000 7f3112bbc640 1 --2- 172.21.15.18:0/1602167764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c101c20 0x7f310c199420 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.062+0000 7f3112bbc640 1 -- 172.21.15.18:0/1602167764 >> 172.21.15.18:0/1602167764 conn(0x7f310c0fb610 msgr2=0x7f310c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:42.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.063+0000 7f3112bbc640 1 -- 172.21.15.18:0/1602167764 shutdown_connections 2024-02-17T12:49:42.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.063+0000 7f3112bbc640 1 -- 172.21.15.18:0/1602167764 wait complete. 2024-02-17T12:49:42.066 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-02-17T12:49:40.219511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658039,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1247,"num_read_kb":747,"num_write":430,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181749,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":854,"ondisk_log_size":854,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383404,"kb_used_data":13452,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558420,"statfs":{"total":1151940427776,"available":1151547822080,"internally_reserved":0,"allocated":13774848,"data_stored":7504557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":66,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002916"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768991+0000","last_change":"2024-02-17T12:47:46.069941+0000","last_active":"2024-02-17T12:49:20.768991+0000","last_peered":"2024-02-17T12:49:20.768991+0000","last_clean":"2024-02-17T12:49:20.768991+0000","last_became_active":"2024-02-17T12:47:46.069192+0000","last_became_peered":"2024-02-17T12:47:46.069192+0000","last_unstale":"2024-02-17T12:49:20.768991+0000","last_undegraded":"2024-02-17T12:49:20.768991+0000","last_fullsized":"2024-02-17T12:49:20.768991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:27.102839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.760213+0000","last_change":"2024-02-17T12:47:46.064872+0000","last_active":"2024-02-17T12:49:20.760213+0000","last_peered":"2024-02-17T12:49:20.760213+0000","last_clean":"2024-02-17T12:49:20.760213+0000","last_became_active":"2024-02-17T12:47:46.064498+0000","last_became_peered":"2024-02-17T12:47:46.064498+0000","last_unstale":"2024-02-17T12:49:20.760213+0000","last_undegraded":"2024-02-17T12:49:20.760213+0000","last_fullsized":"2024-02-17T12:49:20.760213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:30:41.432657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762825+0000","last_change":"2024-02-17T12:47:46.067504+0000","last_active":"2024-02-17T12:49:20.762825+0000","last_peered":"2024-02-17T12:49:20.762825+0000","last_clean":"2024-02-17T12:49:20.762825+0000","last_became_active":"2024-02-17T12:47:46.067155+0000","last_became_peered":"2024-02-17T12:47:46.067155+0000","last_unstale":"2024-02-17T12:49:20.762825+0000","last_undegraded":"2024-02-17T12:49:20.762825+0000","last_fullsized":"2024-02-17T12:49:20.762825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:38:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764046+0000","last_change":"2024-02-17T12:47:46.062356+0000","last_active":"2024-02-17T12:49:20.764046+0000","last_peered":"2024-02-17T12:49:20.764046+0000","last_clean":"2024-02-17T12:49:20.764046+0000","last_became_active":"2024-02-17T12:47:46.061988+0000","last_became_peered":"2024-02-17T12:47:46.061988+0000","last_unstale":"2024-02-17T12:49:20.764046+0000","last_undegraded":"2024-02-17T12:49:20.764046+0000","last_fullsized":"2024-02-17T12:49:20.764046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:25:09.120770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.746039+0000","last_change":"2024-02-17T12:47:46.072411+0000","last_active":"2024-02-17T12:49:20.746039+0000","last_peered":"2024-02-17T12:49:20.746039+0000","last_clean":"2024-02-17T12:49:20.746039+0000","last_became_active":"2024-02-17T12:47:46.072067+0000","last_became_peered":"2024-02-17T12:47:46.072067+0000","last_unstale":"2024-02-17T12:49:20.746039+0000","last_undegraded":"2024-02-17T12:49:20.746039+0000","last_fullsized":"2024-02-17T12:49:20.746039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:29:33.051513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.754893+0000","last_change":"2024-02-17T12:47:46.057425+0000","last_active":"2024-02-17T12:49:20.754893+0000","last_peered":"2024-02-17T12:49:20.754893+0000","last_clean":"2024-02-17T12:49:20.754893+0000","last_became_active":"2024-02-17T12:47:46.057001+0000","last_became_peered":"2024-02-17T12:47:46.057001+0000","last_unstale":"2024-02-17T12:49:20.754893+0000","last_undegraded":"2024-02-17T12:49:20.754893+0000","last_fullsized":"2024-02-17T12:49:20.754893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:48:24.135879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758496+0000","last_change":"2024-02-17T12:47:46.069143+0000","last_active":"2024-02-17T12:49:20.758496+0000","last_peered":"2024-02-17T12:49:20.758496+0000","last_clean":"2024-02-17T12:49:20.758496+0000","last_became_active":"2024-02-17T12:47:46.068775+0000","last_became_peered":"2024-02-17T12:47:46.068775+0000","last_unstale":"2024-02-17T12:49:20.758496+0000","last_undegraded":"2024-02-17T12:49:20.758496+0000","last_fullsized":"2024-02-17T12:49:20.758496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:32:39.518205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764182+0000","last_change":"2024-02-17T12:47:46.069579+0000","last_active":"2024-02-17T12:49:20.764182+0000","last_peered":"2024-02-17T12:49:20.764182+0000","last_clean":"2024-02-17T12:49:20.764182+0000","last_became_active":"2024-02-17T12:47:46.069114+0000","last_became_peered":"2024-02-17T12:47:46.069114+0000","last_unstale":"2024-02-17T12:49:20.764182+0000","last_undegraded":"2024-02-17T12:49:20.764182+0000","last_fullsized":"2024-02-17T12:49:20.764182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:54:58.100511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768271+0000","last_change":"2024-02-17T12:47:46.079103+0000","last_active":"2024-02-17T12:49:20.768271+0000","last_peered":"2024-02-17T12:49:20.768271+0000","last_clean":"2024-02-17T12:49:20.768271+0000","last_became_active":"2024-02-17T12:47:46.077992+0000","last_became_peered":"2024-02-17T12:47:46.077992+0000","last_unstale":"2024-02-17T12:49:20.768271+0000","last_undegraded":"2024-02-17T12:49:20.768271+0000","last_fullsized":"2024-02-17T12:49:20.768271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:12:59.337341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758225+0000","last_change":"2024-02-17T12:47:46.062897+0000","last_active":"2024-02-17T12:49:20.758225+0000","last_peered":"2024-02-17T12:49:20.758225+0000","last_clean":"2024-02-17T12:49:20.758225+0000","last_became_active":"2024-02-17T12:47:46.062571+0000","last_became_peered":"2024-02-17T12:47:46.062571+0000","last_unstale":"2024-02-17T12:49:20.758225+0000","last_undegraded":"2024-02-17T12:49:20.758225+0000","last_fullsized":"2024-02-17T12:49:20.758225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:57:02.845400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.760478+0000","last_change":"2024-02-17T12:47:46.080183+0000","last_active":"2024-02-17T12:49:20.760478+0000","last_peered":"2024-02-17T12:49:20.760478+0000","last_clean":"2024-02-17T12:49:20.760478+0000","last_became_active":"2024-02-17T12:47:46.079827+0000","last_became_peered":"2024-02-17T12:47:46.079827+0000","last_unstale":"2024-02-17T12:49:20.760478+0000","last_undegraded":"2024-02-17T12:49:20.760478+0000","last_fullsized":"2024-02-17T12:49:20.760478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:56:10.193200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762695+0000","last_change":"2024-02-17T12:47:46.061849+0000","last_active":"2024-02-17T12:49:20.762695+0000","last_peered":"2024-02-17T12:49:20.762695+0000","last_clean":"2024-02-17T12:49:20.762695+0000","last_became_active":"2024-02-17T12:47:46.061458+0000","last_became_peered":"2024-02-17T12:47:46.061458+0000","last_unstale":"2024-02-17T12:49:20.762695+0000","last_undegraded":"2024-02-17T12:49:20.762695+0000","last_fullsized":"2024-02-17T12:49:20.762695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:02:09.141292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.765932+0000","last_change":"2024-02-17T12:47:46.069356+0000","last_active":"2024-02-17T12:49:20.765932+0000","last_peered":"2024-02-17T12:49:20.765932+0000","last_clean":"2024-02-17T12:49:20.765932+0000","last_became_active":"2024-02-17T12:47:46.068567+0000","last_became_peered":"2024-02-17T12:47:46.068567+0000","last_unstale":"2024-02-17T12:49:20.765932+0000","last_undegraded":"2024-02-17T12:49:20.765932+0000","last_fullsized":"2024-02-17T12:49:20.765932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:08:01.872558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.757541+0000","last_change":"2024-02-17T12:47:46.062684+0000","last_active":"2024-02-17T12:49:20.757541+0000","last_peered":"2024-02-17T12:49:20.757541+0000","last_clean":"2024-02-17T12:49:20.757541+0000","last_became_active":"2024-02-17T12:47:46.062221+0000","last_became_peered":"2024-02-17T12:47:46.062221+0000","last_unstale":"2024-02-17T12:49:20.757541+0000","last_undegraded":"2024-02-17T12:49:20.757541+0000","last_fullsized":"2024-02-17T12:49:20.757541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:37:11.513336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.760133+0000","last_change":"2024-02-17T12:47:46.065961+0000","last_active":"2024-02-17T12:49:20.760133+0000","last_peered":"2024-02-17T12:49:20.760133+0000","last_clean":"2024-02-17T12:49:20.760133+0000","last_became_active":"2024-02-17T12:47:46.065600+0000","last_became_peered":"2024-02-17T12:47:46.065600+0000","last_unstale":"2024-02-17T12:49:20.760133+0000","last_undegraded":"2024-02-17T12:49:20.760133+0000","last_fullsized":"2024-02-17T12:49:20.760133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:24:46.205077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762547+0000","last_change":"2024-02-17T12:47:46.080150+0000","last_active":"2024-02-17T12:49:20.762547+0000","last_peered":"2024-02-17T12:49:20.762547+0000","last_clean":"2024-02-17T12:49:20.762547+0000","last_became_active":"2024-02-17T12:47:46.079813+0000","last_became_peered":"2024-02-17T12:47:46.079813+0000","last_unstale":"2024-02-17T12:49:20.762547+0000","last_undegraded":"2024-02-17T12:49:20.762547+0000","last_fullsized":"2024-02-17T12:49:20.762547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:59:00.868964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.769045+0000","last_change":"2024-02-17T12:47:46.070202+0000","last_active":"2024-02-17T12:49:20.769045+0000","last_peered":"2024-02-17T12:49:20.769045+0000","last_clean":"2024-02-17T12:49:20.769045+0000","last_became_active":"2024-02-17T12:47:46.069628+0000","last_became_peered":"2024-02-17T12:47:46.069628+0000","last_unstale":"2024-02-17T12:49:20.769045+0000","last_undegraded":"2024-02-17T12:49:20.769045+0000","last_fullsized":"2024-02-17T12:49:20.769045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:44:17.481681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763818+0000","last_change":"2024-02-17T12:47:46.067810+0000","last_active":"2024-02-17T12:49:20.763818+0000","last_peered":"2024-02-17T12:49:20.763818+0000","last_clean":"2024-02-17T12:49:20.763818+0000","last_became_active":"2024-02-17T12:47:46.067456+0000","last_became_peered":"2024-02-17T12:47:46.067456+0000","last_unstale":"2024-02-17T12:49:20.763818+0000","last_undegraded":"2024-02-17T12:49:20.763818+0000","last_fullsized":"2024-02-17T12:49:20.763818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:27:03.646597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.753377+0000","last_change":"2024-02-17T12:47:46.076268+0000","last_active":"2024-02-17T12:49:20.753377+0000","last_peered":"2024-02-17T12:49:20.753377+0000","last_clean":"2024-02-17T12:49:20.753377+0000","last_became_active":"2024-02-17T12:47:46.075606+0000","last_became_peered":"2024-02-17T12:47:46.075606+0000","last_unstale":"2024-02-17T12:49:20.753377+0000","last_undegraded":"2024-02-17T12:49:20.753377+0000","last_fullsized":"2024-02-17T12:49:20.753377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:16:35.805008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":137,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:33.865136+0000","last_change":"2024-02-17T12:47:45.066780+0000","last_active":"2024-02-17T12:49:33.865136+0000","last_peered":"2024-02-17T12:49:33.865136+0000","last_clean":"2024-02-17T12:49:33.865136+0000","last_became_active":"2024-02-17T12:47:45.066039+0000","last_became_peered":"2024-02-17T12:47:45.066039+0000","last_unstale":"2024-02-17T12:49:33.865136+0000","last_undegraded":"2024-02-17T12:49:33.865136+0000","last_fullsized":"2024-02-17T12:49:33.865136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:36:25.420534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763169+0000","last_change":"2024-02-17T12:47:46.079603+0000","last_active":"2024-02-17T12:49:20.763169+0000","last_peered":"2024-02-17T12:49:20.763169+0000","last_clean":"2024-02-17T12:49:20.763169+0000","last_became_active":"2024-02-17T12:47:46.079246+0000","last_became_peered":"2024-02-17T12:47:46.079246+0000","last_unstale":"2024-02-17T12:49:20.763169+0000","last_undegraded":"2024-02-17T12:49:20.763169+0000","last_fullsized":"2024-02-17T12:49:20.763169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:32:04.975778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165983+0000","last_change":"2024-02-17T12:47:45.061868+0000","last_active":"2024-02-17T12:48:52.165983+0000","last_peered":"2024-02-17T12:48:52.165983+0000","last_clean":"2024-02-17T12:48:52.165983+0000","last_became_active":"2024-02-17T12:47:45.061512+0000","last_became_peered":"2024-02-17T12:47:45.061512+0000","last_unstale":"2024-02-17T12:48:52.165983+0000","last_undegraded":"2024-02-17T12:48:52.165983+0000","last_fullsized":"2024-02-17T12:48:52.165983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:53:48.405184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763185+0000","last_change":"2024-02-17T12:47:46.071055+0000","last_active":"2024-02-17T12:49:20.763185+0000","last_peered":"2024-02-17T12:49:20.763185+0000","last_clean":"2024-02-17T12:49:20.763185+0000","last_became_active":"2024-02-17T12:47:46.070649+0000","last_became_peered":"2024-02-17T12:47:46.070649+0000","last_unstale":"2024-02-17T12:49:20.763185+0000","last_undegraded":"2024-02-17T12:49:20.763185+0000","last_fullsized":"2024-02-17T12:49:20.763185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:10:22.078746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.178268+0000","last_change":"2024-02-17T12:48:52.177755+0000","last_active":"2024-02-17T12:48:52.178268+0000","last_peered":"2024-02-17T12:48:52.178268+0000","last_clean":"2024-02-17T12:48:52.178268+0000","last_became_active":"2024-02-17T12:48:52.165119+0000","last_became_peered":"2024-02-17T12:48:52.165119+0000","last_unstale":"2024-02-17T12:48:52.178268+0000","last_undegraded":"2024-02-17T12:48:52.178268+0000","last_fullsized":"2024-02-17T12:48:52.178268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:38.834337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767093+0000","last_change":"2024-02-17T12:47:46.076448+0000","last_active":"2024-02-17T12:49:20.767093+0000","last_peered":"2024-02-17T12:49:20.767093+0000","last_clean":"2024-02-17T12:49:20.767093+0000","last_became_active":"2024-02-17T12:47:46.075250+0000","last_became_peered":"2024-02-17T12:47:46.075250+0000","last_unstale":"2024-02-17T12:49:20.767093+0000","last_undegraded":"2024-02-17T12:49:20.767093+0000","last_fullsized":"2024-02-17T12:49:20.767093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:11:45.846764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163335+0000","last_change":"2024-02-17T12:47:45.062699+0000","last_active":"2024-02-17T12:48:52.163335+0000","last_peered":"2024-02-17T12:48:52.163335+0000","last_clean":"2024-02-17T12:48:52.163335+0000","last_became_active":"2024-02-17T12:47:45.062322+0000","last_became_peered":"2024-02-17T12:47:45.062322+0000","last_unstale":"2024-02-17T12:48:52.163335+0000","last_undegraded":"2024-02-17T12:48:52.163335+0000","last_fullsized":"2024-02-17T12:48:52.163335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:14:06.112393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767082+0000","last_change":"2024-02-17T12:47:46.080278+0000","last_active":"2024-02-17T12:49:20.767082+0000","last_peered":"2024-02-17T12:49:20.767082+0000","last_clean":"2024-02-17T12:49:20.767082+0000","last_became_active":"2024-02-17T12:47:46.079924+0000","last_became_peered":"2024-02-17T12:47:46.079924+0000","last_unstale":"2024-02-17T12:49:20.767082+0000","last_undegraded":"2024-02-17T12:49:20.767082+0000","last_fullsized":"2024-02-17T12:49:20.767082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:54:08.759445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":152,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:37.543298+0000","last_change":"2024-02-17T12:48:52.469734+0000","last_active":"2024-02-17T12:49:37.543298+0000","last_peered":"2024-02-17T12:49:37.543298+0000","last_clean":"2024-02-17T12:49:37.543298+0000","last_became_active":"2024-02-17T12:48:52.465945+0000","last_became_peered":"2024-02-17T12:48:52.465945+0000","last_unstale":"2024-02-17T12:49:37.543298+0000","last_undegraded":"2024-02-17T12:49:37.543298+0000","last_fullsized":"2024-02-17T12:49:37.543298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:05:27.931619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.749939+0000","last_change":"2024-02-17T12:47:46.065957+0000","last_active":"2024-02-17T12:49:20.749939+0000","last_peered":"2024-02-17T12:49:20.749939+0000","last_clean":"2024-02-17T12:49:20.749939+0000","last_became_active":"2024-02-17T12:47:46.065638+0000","last_became_peered":"2024-02-17T12:47:46.065638+0000","last_unstale":"2024-02-17T12:49:20.749939+0000","last_undegraded":"2024-02-17T12:49:20.749939+0000","last_fullsized":"2024-02-17T12:49:20.749939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:02:39.295990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463600+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:48:52.463600+0000","last_peered":"2024-02-17T12:48:52.463600+0000","last_clean":"2024-02-17T12:48:52.463600+0000","last_became_active":"2024-02-17T12:47:45.066235+0000","last_became_peered":"2024-02-17T12:47:45.066235+0000","last_unstale":"2024-02-17T12:48:52.463600+0000","last_undegraded":"2024-02-17T12:48:52.463600+0000","last_fullsized":"2024-02-17T12:48:52.463600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:46:53.731920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763770+0000","last_change":"2024-02-17T12:47:46.072029+0000","last_active":"2024-02-17T12:49:20.763770+0000","last_peered":"2024-02-17T12:49:20.763770+0000","last_clean":"2024-02-17T12:49:20.763770+0000","last_became_active":"2024-02-17T12:47:46.071716+0000","last_became_peered":"2024-02-17T12:47:46.071716+0000","last_unstale":"2024-02-17T12:49:20.763770+0000","last_undegraded":"2024-02-17T12:49:20.763770+0000","last_fullsized":"2024-02-17T12:49:20.763770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:33.076266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163802+0000","last_change":"2024-02-17T12:47:45.067443+0000","last_active":"2024-02-17T12:48:52.163802+0000","last_peered":"2024-02-17T12:48:52.163802+0000","last_clean":"2024-02-17T12:48:52.163802+0000","last_became_active":"2024-02-17T12:47:45.067090+0000","last_became_peered":"2024-02-17T12:47:45.067090+0000","last_unstale":"2024-02-17T12:48:52.163802+0000","last_undegraded":"2024-02-17T12:48:52.163802+0000","last_fullsized":"2024-02-17T12:48:52.163802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:17:39.874810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768189+0000","last_change":"2024-02-17T12:47:46.072275+0000","last_active":"2024-02-17T12:49:20.768189+0000","last_peered":"2024-02-17T12:49:20.768189+0000","last_clean":"2024-02-17T12:49:20.768189+0000","last_became_active":"2024-02-17T12:47:46.071951+0000","last_became_peered":"2024-02-17T12:47:46.071951+0000","last_unstale":"2024-02-17T12:49:20.768189+0000","last_undegraded":"2024-02-17T12:49:20.768189+0000","last_fullsized":"2024-02-17T12:49:20.768189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:45:58.580854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159772+0000","last_change":"2024-02-17T12:47:45.058875+0000","last_active":"2024-02-17T12:48:52.159772+0000","last_peered":"2024-02-17T12:48:52.159772+0000","last_clean":"2024-02-17T12:48:52.159772+0000","last_became_active":"2024-02-17T12:47:45.058433+0000","last_became_peered":"2024-02-17T12:47:45.058433+0000","last_unstale":"2024-02-17T12:48:52.159772+0000","last_undegraded":"2024-02-17T12:48:52.159772+0000","last_fullsized":"2024-02-17T12:48:52.159772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:49.932205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762407+0000","last_change":"2024-02-17T12:47:46.070233+0000","last_active":"2024-02-17T12:49:20.762407+0000","last_peered":"2024-02-17T12:49:20.762407+0000","last_clean":"2024-02-17T12:49:20.762407+0000","last_became_active":"2024-02-17T12:47:46.069811+0000","last_became_peered":"2024-02-17T12:47:46.069811+0000","last_unstale":"2024-02-17T12:49:20.762407+0000","last_undegraded":"2024-02-17T12:49:20.762407+0000","last_fullsized":"2024-02-17T12:49:20.762407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:50:00.885517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":242,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:38.544493+0000","last_change":"2024-02-17T12:47:45.068778+0000","last_active":"2024-02-17T12:49:38.544493+0000","last_peered":"2024-02-17T12:49:38.544493+0000","last_clean":"2024-02-17T12:49:38.544493+0000","last_became_active":"2024-02-17T12:47:45.068375+0000","last_became_peered":"2024-02-17T12:47:45.068375+0000","last_unstale":"2024-02-17T12:49:38.544493+0000","last_undegraded":"2024-02-17T12:49:38.544493+0000","last_fullsized":"2024-02-17T12:49:38.544493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:42:57.600138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":218,"num_read_kb":218,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.765647+0000","last_change":"2024-02-17T12:47:46.076802+0000","last_active":"2024-02-17T12:49:20.765647+0000","last_peered":"2024-02-17T12:49:20.765647+0000","last_clean":"2024-02-17T12:49:20.765647+0000","last_became_active":"2024-02-17T12:47:46.076433+0000","last_became_peered":"2024-02-17T12:47:46.076433+0000","last_unstale":"2024-02-17T12:49:20.765647+0000","last_undegraded":"2024-02-17T12:49:20.765647+0000","last_fullsized":"2024-02-17T12:49:20.765647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.352488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157571+0000","last_change":"2024-02-17T12:47:45.059100+0000","last_active":"2024-02-17T12:48:52.157571+0000","last_peered":"2024-02-17T12:48:52.157571+0000","last_clean":"2024-02-17T12:48:52.157571+0000","last_became_active":"2024-02-17T12:47:45.058707+0000","last_became_peered":"2024-02-17T12:47:45.058707+0000","last_unstale":"2024-02-17T12:48:52.157571+0000","last_undegraded":"2024-02-17T12:48:52.157571+0000","last_fullsized":"2024-02-17T12:48:52.157571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:28:27.044127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.757936+0000","last_change":"2024-02-17T12:47:46.065088+0000","last_active":"2024-02-17T12:49:20.757936+0000","last_peered":"2024-02-17T12:49:20.757936+0000","last_clean":"2024-02-17T12:49:20.757936+0000","last_became_active":"2024-02-17T12:47:46.064754+0000","last_became_peered":"2024-02-17T12:47:46.064754+0000","last_unstale":"2024-02-17T12:49:20.757936+0000","last_undegraded":"2024-02-17T12:49:20.757936+0000","last_fullsized":"2024-02-17T12:49:20.757936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:33:10.652881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160868+0000","last_change":"2024-02-17T12:47:45.055403+0000","last_active":"2024-02-17T12:48:52.160868+0000","last_peered":"2024-02-17T12:48:52.160868+0000","last_clean":"2024-02-17T12:48:52.160868+0000","last_became_active":"2024-02-17T12:47:45.055083+0000","last_became_peered":"2024-02-17T12:47:45.055083+0000","last_unstale":"2024-02-17T12:48:52.160868+0000","last_undegraded":"2024-02-17T12:48:52.160868+0000","last_fullsized":"2024-02-17T12:48:52.160868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:05:51.295074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767519+0000","last_change":"2024-02-17T12:47:46.074883+0000","last_active":"2024-02-17T12:49:20.767519+0000","last_peered":"2024-02-17T12:49:20.767519+0000","last_clean":"2024-02-17T12:49:20.767519+0000","last_became_active":"2024-02-17T12:47:46.074414+0000","last_became_peered":"2024-02-17T12:47:46.074414+0000","last_unstale":"2024-02-17T12:49:20.767519+0000","last_undegraded":"2024-02-17T12:49:20.767519+0000","last_fullsized":"2024-02-17T12:49:20.767519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:35:55.026587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":282,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:38.544476+0000","last_change":"2024-02-17T12:47:45.057627+0000","last_active":"2024-02-17T12:49:38.544476+0000","last_peered":"2024-02-17T12:49:38.544476+0000","last_clean":"2024-02-17T12:49:38.544476+0000","last_became_active":"2024-02-17T12:47:45.057297+0000","last_became_peered":"2024-02-17T12:47:45.057297+0000","last_unstale":"2024-02-17T12:49:38.544476+0000","last_undegraded":"2024-02-17T12:49:38.544476+0000","last_fullsized":"2024-02-17T12:49:38.544476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:00:17.159939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59951,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":229,"num_read_kb":173,"num_write":36,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.761631+0000","last_change":"2024-02-17T12:47:46.076854+0000","last_active":"2024-02-17T12:49:20.761631+0000","last_peered":"2024-02-17T12:49:20.761631+0000","last_clean":"2024-02-17T12:49:20.761631+0000","last_became_active":"2024-02-17T12:47:46.076534+0000","last_became_peered":"2024-02-17T12:47:46.076534+0000","last_unstale":"2024-02-17T12:49:20.761631+0000","last_undegraded":"2024-02-17T12:49:20.761631+0000","last_fullsized":"2024-02-17T12:49:20.761631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:43:01.979075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163878+0000","last_change":"2024-02-17T12:47:45.060090+0000","last_active":"2024-02-17T12:48:52.163878+0000","last_peered":"2024-02-17T12:48:52.163878+0000","last_clean":"2024-02-17T12:48:52.163878+0000","last_became_active":"2024-02-17T12:47:45.058942+0000","last_became_peered":"2024-02-17T12:47:45.058942+0000","last_unstale":"2024-02-17T12:48:52.163878+0000","last_undegraded":"2024-02-17T12:48:52.163878+0000","last_fullsized":"2024-02-17T12:48:52.163878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:30:52.247140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767896+0000","last_change":"2024-02-17T12:47:46.071982+0000","last_active":"2024-02-17T12:49:20.767896+0000","last_peered":"2024-02-17T12:49:20.767896+0000","last_clean":"2024-02-17T12:49:20.767896+0000","last_became_active":"2024-02-17T12:47:46.071639+0000","last_became_peered":"2024-02-17T12:47:46.071639+0000","last_unstale":"2024-02-17T12:49:20.767896+0000","last_undegraded":"2024-02-17T12:49:20.767896+0000","last_fullsized":"2024-02-17T12:49:20.767896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:43:50.658132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160287+0000","last_change":"2024-02-17T12:47:45.060017+0000","last_active":"2024-02-17T12:48:52.160287+0000","last_peered":"2024-02-17T12:48:52.160287+0000","last_clean":"2024-02-17T12:48:52.160287+0000","last_became_active":"2024-02-17T12:47:45.059104+0000","last_became_peered":"2024-02-17T12:47:45.059104+0000","last_unstale":"2024-02-17T12:48:52.160287+0000","last_undegraded":"2024-02-17T12:48:52.160287+0000","last_fullsized":"2024-02-17T12:48:52.160287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:56:50.457297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768545+0000","last_change":"2024-02-17T12:47:46.072250+0000","last_active":"2024-02-17T12:49:20.768545+0000","last_peered":"2024-02-17T12:49:20.768545+0000","last_clean":"2024-02-17T12:49:20.768545+0000","last_became_active":"2024-02-17T12:47:46.071856+0000","last_became_peered":"2024-02-17T12:47:46.071856+0000","last_unstale":"2024-02-17T12:49:20.768545+0000","last_undegraded":"2024-02-17T12:49:20.768545+0000","last_fullsized":"2024-02-17T12:49:20.768545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:46:04.628146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":133,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:33.907678+0000","last_change":"2024-02-17T12:47:45.056386+0000","last_active":"2024-02-17T12:49:33.907678+0000","last_peered":"2024-02-17T12:49:33.907678+0000","last_clean":"2024-02-17T12:49:33.907678+0000","last_became_active":"2024-02-17T12:47:45.055630+0000","last_became_peered":"2024-02-17T12:47:45.055630+0000","last_unstale":"2024-02-17T12:49:33.907678+0000","last_undegraded":"2024-02-17T12:49:33.907678+0000","last_fullsized":"2024-02-17T12:49:33.907678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:16.388950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.759754+0000","last_change":"2024-02-17T12:47:46.071612+0000","last_active":"2024-02-17T12:49:20.759754+0000","last_peered":"2024-02-17T12:49:20.759754+0000","last_clean":"2024-02-17T12:49:20.759754+0000","last_became_active":"2024-02-17T12:47:46.071237+0000","last_became_peered":"2024-02-17T12:47:46.071237+0000","last_unstale":"2024-02-17T12:49:20.759754+0000","last_undegraded":"2024-02-17T12:49:20.759754+0000","last_fullsized":"2024-02-17T12:49:20.759754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:24:01.099679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163081+0000","last_change":"2024-02-17T12:47:45.066258+0000","last_active":"2024-02-17T12:48:52.163081+0000","last_peered":"2024-02-17T12:48:52.163081+0000","last_clean":"2024-02-17T12:48:52.163081+0000","last_became_active":"2024-02-17T12:47:45.065813+0000","last_became_peered":"2024-02-17T12:47:45.065813+0000","last_unstale":"2024-02-17T12:48:52.163081+0000","last_undegraded":"2024-02-17T12:48:52.163081+0000","last_fullsized":"2024-02-17T12:48:52.163081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:06:43.298573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763292+0000","last_change":"2024-02-17T12:47:46.080347+0000","last_active":"2024-02-17T12:49:20.763292+0000","last_peered":"2024-02-17T12:49:20.763292+0000","last_clean":"2024-02-17T12:49:20.763292+0000","last_became_active":"2024-02-17T12:47:46.080083+0000","last_became_peered":"2024-02-17T12:47:46.080083+0000","last_unstale":"2024-02-17T12:49:20.763292+0000","last_undegraded":"2024-02-17T12:49:20.763292+0000","last_fullsized":"2024-02-17T12:49:20.763292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:24:41.807633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163573+0000","last_change":"2024-02-17T12:47:45.062814+0000","last_active":"2024-02-17T12:48:52.163573+0000","last_peered":"2024-02-17T12:48:52.163573+0000","last_clean":"2024-02-17T12:48:52.163573+0000","last_became_active":"2024-02-17T12:47:45.062418+0000","last_became_peered":"2024-02-17T12:47:45.062418+0000","last_unstale":"2024-02-17T12:48:52.163573+0000","last_undegraded":"2024-02-17T12:48:52.163573+0000","last_fullsized":"2024-02-17T12:48:52.163573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:49:34.577275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.759382+0000","last_change":"2024-02-17T12:47:46.067742+0000","last_active":"2024-02-17T12:49:20.759382+0000","last_peered":"2024-02-17T12:49:20.759382+0000","last_clean":"2024-02-17T12:49:20.759382+0000","last_became_active":"2024-02-17T12:47:46.067329+0000","last_became_peered":"2024-02-17T12:47:46.067329+0000","last_unstale":"2024-02-17T12:49:20.759382+0000","last_undegraded":"2024-02-17T12:49:20.759382+0000","last_fullsized":"2024-02-17T12:49:20.759382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:19.432999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156444+0000","last_change":"2024-02-17T12:47:45.065461+0000","last_active":"2024-02-17T12:48:52.156444+0000","last_peered":"2024-02-17T12:48:52.156444+0000","last_clean":"2024-02-17T12:48:52.156444+0000","last_became_active":"2024-02-17T12:47:45.065066+0000","last_became_peered":"2024-02-17T12:47:45.065066+0000","last_unstale":"2024-02-17T12:48:52.156444+0000","last_undegraded":"2024-02-17T12:48:52.156444+0000","last_fullsized":"2024-02-17T12:48:52.156444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:25:47.292901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764291+0000","last_change":"2024-02-17T12:47:46.074222+0000","last_active":"2024-02-17T12:49:20.764291+0000","last_peered":"2024-02-17T12:49:20.764291+0000","last_clean":"2024-02-17T12:49:20.764291+0000","last_became_active":"2024-02-17T12:47:46.073886+0000","last_became_peered":"2024-02-17T12:47:46.073886+0000","last_unstale":"2024-02-17T12:49:20.764291+0000","last_undegraded":"2024-02-17T12:49:20.764291+0000","last_fullsized":"2024-02-17T12:49:20.764291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:46:18.063422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":134,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:38.872177+0000","last_change":"2024-02-17T12:47:45.071414+0000","last_active":"2024-02-17T12:49:38.872177+0000","last_peered":"2024-02-17T12:49:38.872177+0000","last_clean":"2024-02-17T12:49:38.872177+0000","last_became_active":"2024-02-17T12:47:45.071127+0000","last_became_peered":"2024-02-17T12:47:45.071127+0000","last_unstale":"2024-02-17T12:49:38.872177+0000","last_undegraded":"2024-02-17T12:49:38.872177+0000","last_fullsized":"2024-02-17T12:49:38.872177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:13:21.187718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.750322+0000","last_change":"2024-02-17T12:47:46.059412+0000","last_active":"2024-02-17T12:49:20.750322+0000","last_peered":"2024-02-17T12:49:20.750322+0000","last_clean":"2024-02-17T12:49:20.750322+0000","last_became_active":"2024-02-17T12:47:46.059100+0000","last_became_peered":"2024-02-17T12:47:46.059100+0000","last_unstale":"2024-02-17T12:49:20.750322+0000","last_undegraded":"2024-02-17T12:49:20.750322+0000","last_fullsized":"2024-02-17T12:49:20.750322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:35.930613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763100+0000","last_change":"2024-02-17T12:47:46.071467+0000","last_active":"2024-02-17T12:49:20.763100+0000","last_peered":"2024-02-17T12:49:20.763100+0000","last_clean":"2024-02-17T12:49:20.763100+0000","last_became_active":"2024-02-17T12:47:46.070879+0000","last_became_peered":"2024-02-17T12:47:46.070879+0000","last_unstale":"2024-02-17T12:49:20.763100+0000","last_undegraded":"2024-02-17T12:49:20.763100+0000","last_fullsized":"2024-02-17T12:49:20.763100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.682842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":130,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:33.906943+0000","last_change":"2024-02-17T12:47:45.070545+0000","last_active":"2024-02-17T12:49:33.906943+0000","last_peered":"2024-02-17T12:49:33.906943+0000","last_clean":"2024-02-17T12:49:33.906943+0000","last_became_active":"2024-02-17T12:47:45.070265+0000","last_became_peered":"2024-02-17T12:47:45.070265+0000","last_unstale":"2024-02-17T12:49:33.906943+0000","last_undegraded":"2024-02-17T12:49:33.906943+0000","last_fullsized":"2024-02-17T12:49:33.906943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.763735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766601+0000","last_change":"2024-02-17T12:47:46.068539+0000","last_active":"2024-02-17T12:49:20.766601+0000","last_peered":"2024-02-17T12:49:20.766601+0000","last_clean":"2024-02-17T12:49:20.766601+0000","last_became_active":"2024-02-17T12:47:46.068169+0000","last_became_peered":"2024-02-17T12:47:46.068169+0000","last_unstale":"2024-02-17T12:49:20.766601+0000","last_undegraded":"2024-02-17T12:49:20.766601+0000","last_fullsized":"2024-02-17T12:49:20.766601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:27:28.485868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464598+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:48:52.464598+0000","last_peered":"2024-02-17T12:48:52.464598+0000","last_clean":"2024-02-17T12:48:52.464598+0000","last_became_active":"2024-02-17T12:47:45.066299+0000","last_became_peered":"2024-02-17T12:47:45.066299+0000","last_unstale":"2024-02-17T12:48:52.464598+0000","last_undegraded":"2024-02-17T12:48:52.464598+0000","last_fullsized":"2024-02-17T12:48:52.464598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:34:02.040002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157254+0000","last_change":"2024-02-17T12:46:32.556576+0000","last_active":"2024-02-17T12:48:52.157254+0000","last_peered":"2024-02-17T12:48:52.157254+0000","last_clean":"2024-02-17T12:48:52.157254+0000","last_became_active":"2024-02-17T12:46:32.556160+0000","last_became_peered":"2024-02-17T12:46:32.556160+0000","last_unstale":"2024-02-17T12:48:52.157254+0000","last_undegraded":"2024-02-17T12:48:52.157254+0000","last_fullsized":"2024-02-17T12:48:52.157254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_clean_scrub_stamp":"2024-02-17T12:43:13.147248+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:57:52.244951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767280+0000","last_change":"2024-02-17T12:48:52.166240+0000","last_active":"2024-02-17T12:49:20.767280+0000","last_peered":"2024-02-17T12:49:20.767280+0000","last_clean":"2024-02-17T12:49:20.767280+0000","last_became_active":"2024-02-17T12:48:52.165882+0000","last_became_peered":"2024-02-17T12:48:52.165882+0000","last_unstale":"2024-02-17T12:49:20.767280+0000","last_undegraded":"2024-02-17T12:49:20.767280+0000","last_fullsized":"2024-02-17T12:49:20.767280+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:56:44.360362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157273+0000","last_change":"2024-02-17T12:47:45.061112+0000","last_active":"2024-02-17T12:48:52.157273+0000","last_peered":"2024-02-17T12:48:52.157273+0000","last_clean":"2024-02-17T12:48:52.157273+0000","last_became_active":"2024-02-17T12:47:45.060633+0000","last_became_peered":"2024-02-17T12:47:45.060633+0000","last_unstale":"2024-02-17T12:48:52.157273+0000","last_undegraded":"2024-02-17T12:48:52.157273+0000","last_fullsized":"2024-02-17T12:48:52.157273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:29:29.421744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163918+0000","last_change":"2024-02-17T12:47:45.066897+0000","last_active":"2024-02-17T12:48:52.163918+0000","last_peered":"2024-02-17T12:48:52.163918+0000","last_clean":"2024-02-17T12:48:52.163918+0000","last_became_active":"2024-02-17T12:47:45.066357+0000","last_became_peered":"2024-02-17T12:47:45.066357+0000","last_unstale":"2024-02-17T12:48:52.163918+0000","last_undegraded":"2024-02-17T12:48:52.163918+0000","last_fullsized":"2024-02-17T12:48:52.163918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:50:09.840414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767577+0000","last_change":"2024-02-17T12:47:46.072412+0000","last_active":"2024-02-17T12:49:20.767577+0000","last_peered":"2024-02-17T12:49:20.767577+0000","last_clean":"2024-02-17T12:49:20.767577+0000","last_became_active":"2024-02-17T12:47:46.072105+0000","last_became_peered":"2024-02-17T12:47:46.072105+0000","last_unstale":"2024-02-17T12:49:20.767577+0000","last_undegraded":"2024-02-17T12:49:20.767577+0000","last_fullsized":"2024-02-17T12:49:20.767577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:33:19.367588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'13","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156660+0000","last_change":"2024-02-17T12:47:45.067673+0000","last_active":"2024-02-17T12:48:52.156660+0000","last_peered":"2024-02-17T12:48:52.156660+0000","last_clean":"2024-02-17T12:48:52.156660+0000","last_became_active":"2024-02-17T12:47:45.067212+0000","last_became_peered":"2024-02-17T12:47:45.067212+0000","last_unstale":"2024-02-17T12:48:52.156660+0000","last_undegraded":"2024-02-17T12:48:52.156660+0000","last_fullsized":"2024-02-17T12:48:52.156660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:15:04.236158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763306+0000","last_change":"2024-02-17T12:47:46.077476+0000","last_active":"2024-02-17T12:49:20.763306+0000","last_peered":"2024-02-17T12:49:20.763306+0000","last_clean":"2024-02-17T12:49:20.763306+0000","last_became_active":"2024-02-17T12:47:46.077101+0000","last_became_peered":"2024-02-17T12:47:46.077101+0000","last_unstale":"2024-02-17T12:49:20.763306+0000","last_undegraded":"2024-02-17T12:49:20.763306+0000","last_fullsized":"2024-02-17T12:49:20.763306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:41:41.402101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464561+0000","last_change":"2024-02-17T12:47:45.066693+0000","last_active":"2024-02-17T12:48:52.464561+0000","last_peered":"2024-02-17T12:48:52.464561+0000","last_clean":"2024-02-17T12:48:52.464561+0000","last_became_active":"2024-02-17T12:47:45.066349+0000","last_became_peered":"2024-02-17T12:47:45.066349+0000","last_unstale":"2024-02-17T12:48:52.464561+0000","last_undegraded":"2024-02-17T12:48:52.464561+0000","last_fullsized":"2024-02-17T12:48:52.464561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:30.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.759614+0000","last_change":"2024-02-17T12:47:46.072899+0000","last_active":"2024-02-17T12:49:20.759614+0000","last_peered":"2024-02-17T12:49:20.759614+0000","last_clean":"2024-02-17T12:49:20.759614+0000","last_became_active":"2024-02-17T12:47:46.072488+0000","last_became_peered":"2024-02-17T12:47:46.072488+0000","last_unstale":"2024-02-17T12:49:20.759614+0000","last_undegraded":"2024-02-17T12:49:20.759614+0000","last_fullsized":"2024-02-17T12:49:20.759614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:05:43.675086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463459+0000","last_change":"2024-02-17T12:47:45.056995+0000","last_active":"2024-02-17T12:48:52.463459+0000","last_peered":"2024-02-17T12:48:52.463459+0000","last_clean":"2024-02-17T12:48:52.463459+0000","last_became_active":"2024-02-17T12:47:45.056021+0000","last_became_peered":"2024-02-17T12:47:45.056021+0000","last_unstale":"2024-02-17T12:48:52.463459+0000","last_undegraded":"2024-02-17T12:48:52.463459+0000","last_fullsized":"2024-02-17T12:48:52.463459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:41.444513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.722229+0000","last_change":"2024-02-17T12:47:46.065859+0000","last_active":"2024-02-17T12:49:20.722229+0000","last_peered":"2024-02-17T12:49:20.722229+0000","last_clean":"2024-02-17T12:49:20.722229+0000","last_became_active":"2024-02-17T12:47:46.065469+0000","last_became_peered":"2024-02-17T12:47:46.065469+0000","last_unstale":"2024-02-17T12:49:20.722229+0000","last_undegraded":"2024-02-17T12:49:20.722229+0000","last_fullsized":"2024-02-17T12:49:20.722229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:51:25.105361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165363+0000","last_change":"2024-02-17T12:47:45.066524+0000","last_active":"2024-02-17T12:48:52.165363+0000","last_peered":"2024-02-17T12:48:52.165363+0000","last_clean":"2024-02-17T12:48:52.165363+0000","last_became_active":"2024-02-17T12:47:45.066218+0000","last_became_peered":"2024-02-17T12:47:45.066218+0000","last_unstale":"2024-02-17T12:48:52.165363+0000","last_undegraded":"2024-02-17T12:48:52.165363+0000","last_fullsized":"2024-02-17T12:48:52.165363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:56:47.244262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764154+0000","last_change":"2024-02-17T12:47:46.065779+0000","last_active":"2024-02-17T12:49:20.764154+0000","last_peered":"2024-02-17T12:49:20.764154+0000","last_clean":"2024-02-17T12:49:20.764154+0000","last_became_active":"2024-02-17T12:47:46.065459+0000","last_became_peered":"2024-02-17T12:47:46.065459+0000","last_unstale":"2024-02-17T12:49:20.764154+0000","last_undegraded":"2024-02-17T12:49:20.764154+0000","last_fullsized":"2024-02-17T12:49:20.764154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:08.944487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163588+0000","last_change":"2024-02-17T12:47:45.059639+0000","last_active":"2024-02-17T12:48:52.163588+0000","last_peered":"2024-02-17T12:48:52.163588+0000","last_clean":"2024-02-17T12:48:52.163588+0000","last_became_active":"2024-02-17T12:47:45.059085+0000","last_became_peered":"2024-02-17T12:47:45.059085+0000","last_unstale":"2024-02-17T12:48:52.163588+0000","last_undegraded":"2024-02-17T12:48:52.163588+0000","last_fullsized":"2024-02-17T12:48:52.163588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:48:55.847405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.756470+0000","last_change":"2024-02-17T12:47:46.079623+0000","last_active":"2024-02-17T12:49:20.756470+0000","last_peered":"2024-02-17T12:49:20.756470+0000","last_clean":"2024-02-17T12:49:20.756470+0000","last_became_active":"2024-02-17T12:47:46.079285+0000","last_became_peered":"2024-02-17T12:47:46.079285+0000","last_unstale":"2024-02-17T12:49:20.756470+0000","last_undegraded":"2024-02-17T12:49:20.756470+0000","last_fullsized":"2024-02-17T12:49:20.756470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:45:14.441218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.181532+0000","last_change":"2024-02-17T12:48:52.181057+0000","last_active":"2024-02-17T12:48:52.181532+0000","last_peered":"2024-02-17T12:48:52.181532+0000","last_clean":"2024-02-17T12:48:52.181532+0000","last_became_active":"2024-02-17T12:48:52.165968+0000","last_became_peered":"2024-02-17T12:48:52.165968+0000","last_unstale":"2024-02-17T12:48:52.181532+0000","last_undegraded":"2024-02-17T12:48:52.181532+0000","last_fullsized":"2024-02-17T12:48:52.181532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:14:06.153057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766320+0000","last_change":"2024-02-17T12:47:46.074429+0000","last_active":"2024-02-17T12:49:20.766320+0000","last_peered":"2024-02-17T12:49:20.766320+0000","last_clean":"2024-02-17T12:49:20.766320+0000","last_became_active":"2024-02-17T12:47:46.074123+0000","last_became_peered":"2024-02-17T12:47:46.074123+0000","last_unstale":"2024-02-17T12:49:20.766320+0000","last_undegraded":"2024-02-17T12:49:20.766320+0000","last_fullsized":"2024-02-17T12:49:20.766320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:50:17.633831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":132,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:35.869454+0000","last_change":"2024-02-17T12:47:45.061362+0000","last_active":"2024-02-17T12:49:35.869454+0000","last_peered":"2024-02-17T12:49:35.869454+0000","last_clean":"2024-02-17T12:49:35.869454+0000","last_became_active":"2024-02-17T12:47:45.061037+0000","last_became_peered":"2024-02-17T12:47:45.061037+0000","last_unstale":"2024-02-17T12:49:35.869454+0000","last_undegraded":"2024-02-17T12:49:35.869454+0000","last_fullsized":"2024-02-17T12:49:35.869454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:01.987157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764578+0000","last_change":"2024-02-17T12:47:46.073862+0000","last_active":"2024-02-17T12:49:20.764578+0000","last_peered":"2024-02-17T12:49:20.764578+0000","last_clean":"2024-02-17T12:49:20.764578+0000","last_became_active":"2024-02-17T12:47:46.073543+0000","last_became_peered":"2024-02-17T12:47:46.073543+0000","last_unstale":"2024-02-17T12:49:20.764578+0000","last_undegraded":"2024-02-17T12:49:20.764578+0000","last_fullsized":"2024-02-17T12:49:20.764578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:57:57.567726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160633+0000","last_change":"2024-02-17T12:47:45.060616+0000","last_active":"2024-02-17T12:48:52.160633+0000","last_peered":"2024-02-17T12:48:52.160633+0000","last_clean":"2024-02-17T12:48:52.160633+0000","last_became_active":"2024-02-17T12:47:45.060296+0000","last_became_peered":"2024-02-17T12:47:45.060296+0000","last_unstale":"2024-02-17T12:48:52.160633+0000","last_undegraded":"2024-02-17T12:48:52.160633+0000","last_fullsized":"2024-02-17T12:48:52.160633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:22:44.791363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758731+0000","last_change":"2024-02-17T12:47:46.072738+0000","last_active":"2024-02-17T12:49:20.758731+0000","last_peered":"2024-02-17T12:49:20.758731+0000","last_clean":"2024-02-17T12:49:20.758731+0000","last_became_active":"2024-02-17T12:47:46.072421+0000","last_became_peered":"2024-02-17T12:47:46.072421+0000","last_unstale":"2024-02-17T12:49:20.758731+0000","last_undegraded":"2024-02-17T12:49:20.758731+0000","last_fullsized":"2024-02-17T12:49:20.758731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:50:23.501787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.470091+0000","last_change":"2024-02-17T12:48:52.470091+0000","last_active":"2024-02-17T12:48:52.470091+0000","last_peered":"2024-02-17T12:48:52.470091+0000","last_clean":"2024-02-17T12:48:52.470091+0000","last_became_active":"2024-02-17T12:48:52.466153+0000","last_became_peered":"2024-02-17T12:48:52.466153+0000","last_unstale":"2024-02-17T12:48:52.470091+0000","last_undegraded":"2024-02-17T12:48:52.470091+0000","last_fullsized":"2024-02-17T12:48:52.470091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:03:45.027983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764385+0000","last_change":"2024-02-17T12:47:46.066072+0000","last_active":"2024-02-17T12:49:20.764385+0000","last_peered":"2024-02-17T12:49:20.764385+0000","last_clean":"2024-02-17T12:49:20.764385+0000","last_became_active":"2024-02-17T12:47:46.065755+0000","last_became_peered":"2024-02-17T12:47:46.065755+0000","last_unstale":"2024-02-17T12:49:20.764385+0000","last_undegraded":"2024-02-17T12:49:20.764385+0000","last_fullsized":"2024-02-17T12:49:20.764385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:09:29.534097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763980+0000","last_change":"2024-02-17T12:47:46.067268+0000","last_active":"2024-02-17T12:49:20.763980+0000","last_peered":"2024-02-17T12:49:20.763980+0000","last_clean":"2024-02-17T12:49:20.763980+0000","last_became_active":"2024-02-17T12:47:46.066918+0000","last_became_peered":"2024-02-17T12:47:46.066918+0000","last_unstale":"2024-02-17T12:49:20.763980+0000","last_undegraded":"2024-02-17T12:49:20.763980+0000","last_fullsized":"2024-02-17T12:49:20.763980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:11:09.844954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.757364+0000","last_change":"2024-02-17T12:47:46.068143+0000","last_active":"2024-02-17T12:49:20.757364+0000","last_peered":"2024-02-17T12:49:20.757364+0000","last_clean":"2024-02-17T12:49:20.757364+0000","last_became_active":"2024-02-17T12:47:46.065112+0000","last_became_peered":"2024-02-17T12:47:46.065112+0000","last_unstale":"2024-02-17T12:49:20.757364+0000","last_undegraded":"2024-02-17T12:49:20.757364+0000","last_fullsized":"2024-02-17T12:49:20.757364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:43:04.982009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.748955+0000","last_change":"2024-02-17T12:47:46.076139+0000","last_active":"2024-02-17T12:49:20.748955+0000","last_peered":"2024-02-17T12:49:20.748955+0000","last_clean":"2024-02-17T12:49:20.748955+0000","last_became_active":"2024-02-17T12:47:46.075826+0000","last_became_peered":"2024-02-17T12:47:46.075826+0000","last_unstale":"2024-02-17T12:49:20.748955+0000","last_undegraded":"2024-02-17T12:49:20.748955+0000","last_fullsized":"2024-02-17T12:49:20.748955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:39:08.460716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766396+0000","last_change":"2024-02-17T12:47:46.063792+0000","last_active":"2024-02-17T12:49:20.766396+0000","last_peered":"2024-02-17T12:49:20.766396+0000","last_clean":"2024-02-17T12:49:20.766396+0000","last_became_active":"2024-02-17T12:47:46.063407+0000","last_became_peered":"2024-02-17T12:47:46.063407+0000","last_unstale":"2024-02-17T12:49:20.766396+0000","last_undegraded":"2024-02-17T12:49:20.766396+0000","last_fullsized":"2024-02-17T12:49:20.766396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:26:26.973435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768046+0000","last_change":"2024-02-17T12:47:46.066202+0000","last_active":"2024-02-17T12:49:20.768046+0000","last_peered":"2024-02-17T12:49:20.768046+0000","last_clean":"2024-02-17T12:49:20.768046+0000","last_became_active":"2024-02-17T12:47:46.065820+0000","last_became_peered":"2024-02-17T12:47:46.065820+0000","last_unstale":"2024-02-17T12:49:20.768046+0000","last_undegraded":"2024-02-17T12:49:20.768046+0000","last_fullsized":"2024-02-17T12:49:20.768046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:36:25.201697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.759899+0000","last_change":"2024-02-17T12:48:52.166996+0000","last_active":"2024-02-17T12:49:20.759899+0000","last_peered":"2024-02-17T12:49:20.759899+0000","last_clean":"2024-02-17T12:49:20.759899+0000","last_became_active":"2024-02-17T12:48:52.166589+0000","last_became_peered":"2024-02-17T12:48:52.166589+0000","last_unstale":"2024-02-17T12:49:20.759899+0000","last_undegraded":"2024-02-17T12:49:20.759899+0000","last_fullsized":"2024-02-17T12:49:20.759899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:16:30.392183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763854+0000","last_change":"2024-02-17T12:47:46.070811+0000","last_active":"2024-02-17T12:49:20.763854+0000","last_peered":"2024-02-17T12:49:20.763854+0000","last_clean":"2024-02-17T12:49:20.763854+0000","last_became_active":"2024-02-17T12:47:46.070462+0000","last_became_peered":"2024-02-17T12:47:46.070462+0000","last_unstale":"2024-02-17T12:49:20.763854+0000","last_undegraded":"2024-02-17T12:49:20.763854+0000","last_fullsized":"2024-02-17T12:49:20.763854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:03.671869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758608+0000","last_change":"2024-02-17T12:47:46.077692+0000","last_active":"2024-02-17T12:49:20.758608+0000","last_peered":"2024-02-17T12:49:20.758608+0000","last_clean":"2024-02-17T12:49:20.758608+0000","last_became_active":"2024-02-17T12:47:46.077363+0000","last_became_peered":"2024-02-17T12:47:46.077363+0000","last_unstale":"2024-02-17T12:49:20.758608+0000","last_undegraded":"2024-02-17T12:49:20.758608+0000","last_fullsized":"2024-02-17T12:49:20.758608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:02:49.068782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758794+0000","last_change":"2024-02-17T12:47:46.073507+0000","last_active":"2024-02-17T12:49:20.758794+0000","last_peered":"2024-02-17T12:49:20.758794+0000","last_clean":"2024-02-17T12:49:20.758794+0000","last_became_active":"2024-02-17T12:47:46.073107+0000","last_became_peered":"2024-02-17T12:47:46.073107+0000","last_unstale":"2024-02-17T12:49:20.758794+0000","last_undegraded":"2024-02-17T12:49:20.758794+0000","last_fullsized":"2024-02-17T12:49:20.758794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:32:48.799300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762538+0000","last_change":"2024-02-17T12:47:46.073384+0000","last_active":"2024-02-17T12:49:20.762538+0000","last_peered":"2024-02-17T12:49:20.762538+0000","last_clean":"2024-02-17T12:49:20.762538+0000","last_became_active":"2024-02-17T12:47:46.073017+0000","last_became_peered":"2024-02-17T12:47:46.073017+0000","last_unstale":"2024-02-17T12:49:20.762538+0000","last_undegraded":"2024-02-17T12:49:20.762538+0000","last_fullsized":"2024-02-17T12:49:20.762538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:43:50.091667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766804+0000","last_change":"2024-02-17T12:47:46.073758+0000","last_active":"2024-02-17T12:49:20.766804+0000","last_peered":"2024-02-17T12:49:20.766804+0000","last_clean":"2024-02-17T12:49:20.766804+0000","last_became_active":"2024-02-17T12:47:46.072781+0000","last_became_peered":"2024-02-17T12:47:46.072781+0000","last_unstale":"2024-02-17T12:49:20.766804+0000","last_undegraded":"2024-02-17T12:49:20.766804+0000","last_fullsized":"2024-02-17T12:49:20.766804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:10:24.811100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766956+0000","last_change":"2024-02-17T12:47:46.072025+0000","last_active":"2024-02-17T12:49:20.766956+0000","last_peered":"2024-02-17T12:49:20.766956+0000","last_clean":"2024-02-17T12:49:20.766956+0000","last_became_active":"2024-02-17T12:47:46.071662+0000","last_became_peered":"2024-02-17T12:47:46.071662+0000","last_unstale":"2024-02-17T12:49:20.766956+0000","last_undegraded":"2024-02-17T12:49:20.766956+0000","last_fullsized":"2024-02-17T12:49:20.766956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:04:07.188552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762073+0000","last_change":"2024-02-17T12:48:52.166041+0000","last_active":"2024-02-17T12:49:20.762073+0000","last_peered":"2024-02-17T12:49:20.762073+0000","last_clean":"2024-02-17T12:49:20.762073+0000","last_became_active":"2024-02-17T12:48:52.165576+0000","last_became_peered":"2024-02-17T12:48:52.165576+0000","last_unstale":"2024-02-17T12:49:20.762073+0000","last_undegraded":"2024-02-17T12:49:20.762073+0000","last_fullsized":"2024-02-17T12:49:20.762073+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:26.810385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67437,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1111,"num_read_kb":628,"num_write":247,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1013,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":462062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.40699999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.443}]},{"osd":1,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.56399999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82899999999999996,"5min":0.874,"15min":0.874},"last":0.67100000000000004}]},{"osd":2,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69999999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.54900000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90700000000000003,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.51600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83699999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80000000000000004,"5min":1.002,"15min":1.002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93100000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84599999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.67100000000000004,"15min":0.67800000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":1.121,"15min":1.121},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77600000000000002,"15min":0.77300000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.748,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91800000000000004,"5min":1.103,"15min":1.103},"last":0.873}]},{"osd":8,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.77800000000000002}]},{"osd":9,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.157,"15min":1.157},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.78800000000000003,"15min":0.78300000000000003},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.054,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.82399999999999995}]},{"osd":10,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79400000000000004,"15min":0.78800000000000003},"min":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98699999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.80900000000000005}]}]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1113574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":13.85,"5min":13.85,"15min":13.85},"last":0.56999999999999995}]},{"osd":1,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.48199999999999998,"15min":0.503},"min":{"1min":0.31900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.629,"5min":0.751,"15min":0.751},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.52300000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65600000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.43099999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68100000000000005,"15min":0.69099999999999995},"min":{"1min":0.49099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76600000000000001,"5min":1.002,"15min":1.002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.52800000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83499999999999996,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.65600000000000003}]},{"osd":3,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.57199999999999995,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.64400000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.499},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.64200000000000002,"15min":0.66800000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.74299999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.51400000000000001,"15min":0.53700000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64500000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.58099999999999996,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78000000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.52400000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71499999999999997,"15min":0.73999999999999999},"min":{"1min":0.47999999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.88,"5min":1.022,"15min":1.022},"last":0.623},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67500000000000004,"15min":0.69699999999999995},"min":{"1min":0.47299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83599999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65800000000000003,"15min":0.68999999999999995},"min":{"1min":0.438,"5min":0.437,"15min":0.437},"max":{"1min":0.748,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.73399999999999999,"15min":0.76000000000000001},"min":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85299999999999998,"5min":1.085,"15min":1.085},"last":0.47699999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.54900000000000004}]},{"osd":8,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.74099999999999999,"15min":0.77400000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.81000000000000005,"5min":1.107,"15min":1.107},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.80300000000000005,"15min":0.83299999999999996},"min":{"1min":0.61099999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92000000000000004,"5min":1.119,"15min":1.119},"last":0.753}]},{"osd":9,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.80400000000000005,"15min":0.84799999999999998},"min":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94399999999999995,"5min":1.151,"15min":1.151},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.78000000000000003,"15min":0.82199999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.91300000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.59599999999999997}]},{"osd":11,"last update":"Sat Feb 17 12:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77700000000000002,"15min":0.77900000000000003},"min":{"1min":0.63800000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.93899999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75,"15min":0.752},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68799999999999994}]}]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40460,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704692,"statfs":{"total":95995035648,"available":95953604608,"internally_reserved":0,"allocated":987136,"data_stored":461992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.436,"15min":0.45400000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.60199999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.39900000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63100000000000001,"15min":0.65300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.5}]},{"osd":2,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63300000000000001,"15min":0.64700000000000002},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.0409999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.316,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.879,"5min":1.488,"15min":1.488},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.58299999999999996,"15min":0.61199999999999999},"min":{"1min":0.29999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83999999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2030000000000001,"5min":1.218,"15min":1.218},"last":0.622},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65100000000000002,"15min":0.66700000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95999999999999996,"5min":1.159,"15min":1.159},"last":0.58799999999999997}]},{"osd":5,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.68200000000000005,"15min":0.70599999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0029999999999999,"5min":1.323,"15min":1.323},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.73999999999999999,"15min":0.76800000000000002},"min":{"1min":0.46100000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92100000000000004,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.77700000000000002}]},{"osd":6,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67300000000000004,"15min":0.70699999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.79700000000000004,"5min":1.268,"15min":1.268},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.60999999999999999,"15min":0.63400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85799999999999998,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.56699999999999995}]},{"osd":7,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.73899999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0760000000000001,"5min":1.27,"15min":1.27},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77800000000000002,"15min":0.80800000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.236,"5min":1.351,"15min":1.351},"last":0.66600000000000004}]},{"osd":8,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73599999999999999,"15min":0.754},"min":{"1min":0.498,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.749},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.76600000000000001,"15min":0.79400000000000004},"min":{"1min":0.52100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1160000000000001,"5min":1.363,"15min":1.363},"last":0.76200000000000001}]},{"osd":10,"last update":"Sat Feb 17 12:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.80700000000000005,"5min":1.228,"15min":1.228},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78300000000000003,"15min":0.81000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.73099999999999998}]}]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":462370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.35899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.52400000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.91800000000000004,"15min":1.054},"min":{"1min":0.441,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89000000000000001,"5min":10.955,"15min":10.955},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.434,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.84999999999999998}]},{"osd":2,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.44900000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.443}]},{"osd":3,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.36499999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.71899999999999997}]},{"osd":4,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.93300000000000005,"15min":1.0680000000000001},"min":{"1min":0.46100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":10.972,"15min":10.972},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.92800000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.372,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.34899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.60099999999999998}]},{"osd":6,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.878},"min":{"1min":0.53200000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97499999999999998,"5min":4.9269999999999996,"15min":4.9269999999999996},"last":0.81599999999999995}]},{"osd":7,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.49399999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.628},{"interface":"front","average":{"1min":0.755,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":1.022,"15min":1.022},"last":0.68500000000000005}]},{"osd":9,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.72999999999999998,"15min":0.69799999999999995},"min":{"1min":0.64500000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.77700000000000002,"15min":0.75},"min":{"1min":0.65000000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.98799999999999999}]},{"osd":10,"last update":"Sat Feb 17 12:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.753,"15min":0.74399999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.748},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.1339999999999999,"15min":1.268},"min":{"1min":0.54700000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.083,"5min":10.987,"15min":10.987},"last":0.89900000000000002}]},{"osd":11,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.875},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.91600000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1055561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42899999999999999,"15min":0.41299999999999998},"min":{"1min":0.33700000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57699999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.46100000000000002,"15min":0.439},"min":{"1min":0.38200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68000000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.318}]},{"osd":2,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.48899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.75800000000000001,"15min":0.63300000000000001},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77900000000000003,"5min":15.003,"15min":15.003},"last":0.54400000000000004}]},{"osd":3,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60199999999999998,"15min":0.61799999999999999},"min":{"1min":0.45600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68600000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.60299999999999998,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76100000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56000000000000005}]},{"osd":4,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.36599999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71999999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.39900000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82399999999999995,"5min":0.872,"15min":0.872},"last":0.35899999999999999}]},{"osd":5,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.81100000000000005,"15min":0.68899999999999995},"min":{"1min":0.44600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.92700000000000005,"5min":15.028,"15min":15.028},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.54500000000000004,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86199999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59099999999999997}]},{"osd":6,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66700000000000004,"15min":0.68200000000000005},"min":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.80600000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.499},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.93300000000000005,"15min":0.77100000000000002},"min":{"1min":0.51700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88400000000000001,"5min":14.579000000000001,"15min":14.579000000000001},"last":0.45300000000000001}]},{"osd":8,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.60999999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96199999999999997,"5min":1.02,"15min":1.02},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75700000000000001,"15min":0.73799999999999999},"min":{"1min":0.65500000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.77400000000000002}]},{"osd":9,"last update":"Sat Feb 17 12:49:14 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":0.84699999999999998,"15min":0.79900000000000004},"min":{"1min":0.48999999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.403,"5min":10.403,"15min":10.403},"last":0.51900000000000002},{"interface":"front","average":{"1min":1.2410000000000001,"5min":0.98999999999999999,"15min":0.84199999999999997},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":10.471,"5min":14.598000000000001,"15min":14.598000000000001},"last":0.65800000000000003}]},{"osd":10,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.68000000000000005,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91400000000000003,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.59299999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82799999999999996,"5min":1.103,"15min":1.103},"last":0.64200000000000002}]},{"osd":11,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.876}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":461709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.38100000000000001,"15min":0.371},"min":{"1min":0.26100000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.49399999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.28100000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53600000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.32600000000000001}]},{"osd":1,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.40400000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.62,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":5.0179999999999998,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":0.59499999999999997}]},{"osd":2,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.65300000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.63200000000000001,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":5.0469999999999997,"5min":5.0469999999999997,"15min":5.0469999999999997},"last":0.57199999999999995}]},{"osd":3,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.441,"15min":0.443},"min":{"1min":0.29099999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.52900000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.624,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42899999999999999}]},{"osd":4,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73699999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.623},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.53900000000000003}]},{"osd":5,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74199999999999999,"5min":1.284,"15min":1.284},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75900000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.65600000000000003}]},{"osd":7,"last update":"Sat Feb 17 12:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72099999999999997,"15min":0.67000000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92200000000000004,"5min":5.069,"15min":5.069},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.64100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.58099999999999996}]},{"osd":8,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.73399999999999999,"15min":0.70799999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":5.0940000000000003,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90800000000000003,"5min":1.252,"15min":1.252},"last":0.68799999999999994}]},{"osd":9,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.65500000000000003,"15min":0.68500000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90600000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.67700000000000005,"15min":0.70699999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85499999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.49299999999999999}]},{"osd":10,"last update":"Sat Feb 17 12:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.749,"15min":0.76300000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.88500000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":522573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.40500000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.67800000000000005,"5min":0.73999999999999999,"15min":0.78100000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.69499999999999995,"15min":0.55500000000000005},"min":{"1min":0.42699999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79600000000000004,"5min":14.91,"15min":14.91},"last":0.67600000000000005}]},{"osd":2,"last update":"Sat Feb 17 12:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.59899999999999998,"15min":0.52700000000000002},"min":{"1min":0.51000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.434},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.60499999999999998,"15min":0.53000000000000003},"min":{"1min":0.61599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61899999999999999}]},{"osd":3,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.48599999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78500000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.432,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73599999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.74199999999999999}]},{"osd":4,"last update":"Sat Feb 17 12:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.44600000000000001,"15min":0.39900000000000002},"min":{"1min":0.34699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.77600000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.46700000000000003,"15min":0.42899999999999999},"min":{"1min":0.35699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.80700000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46000000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.65300000000000002,"15min":0.59199999999999997},"min":{"1min":0.63200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98799999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.60699999999999998},"min":{"1min":0.50900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82399999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72299999999999998}]},{"osd":6,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61599999999999999,"15min":0.59499999999999997},"min":{"1min":0.59999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86699999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.53700000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.77200000000000002}]},{"osd":7,"last update":"Sat Feb 17 12:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.60099999999999998,"15min":0.55900000000000005},"min":{"1min":0.41099999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.66200000000000003,"15min":0.60899999999999999},"min":{"1min":0.53200000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95899999999999996,"5min":1.079,"15min":1.079},"last":0.65300000000000002}]},{"osd":8,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.67400000000000004,"15min":0.63400000000000001},"min":{"1min":0.63800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.58999999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59499999999999997}]},{"osd":9,"last update":"Sat Feb 17 12:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72499999999999998,"15min":0.70699999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.747,"15min":0.71899999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.85899999999999999}]},{"osd":10,"last update":"Sat Feb 17 12:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.57899999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97799999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.78100000000000003,"15min":0.745},"min":{"1min":0.52800000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83199999999999996}]},{"osd":11,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.81799999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":463048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53900000000000003,"15min":0.49099999999999999},"min":{"1min":0.36899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55300000000000005,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64400000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60899999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.437,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.748,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.73699999999999999},"min":{"1min":0.42299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81999999999999995,"5min":8.3919999999999995,"15min":8.3919999999999995},"last":0.67300000000000004}]},{"osd":3,"last update":"Sat Feb 17 12:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.39000000000000001}]},{"osd":4,"last update":"Sat Feb 17 12:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.58899999999999997,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":0.872,"15min":0.872},"last":0.59499999999999997}]},{"osd":5,"last update":"Sat Feb 17 12:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.42099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68999999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.48099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.748,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63}]},{"osd":6,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.46600000000000003,"15min":0.45800000000000002},"min":{"1min":0.32500000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58699999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.49399999999999999,"15min":0.51000000000000001},"min":{"1min":0.30099999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65600000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.502}]},{"osd":7,"last update":"Sat Feb 17 12:47:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.443},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64600000000000002,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.65300000000000002,"15min":0.66500000000000004},"min":{"1min":0.45000000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79500000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.75800000000000001}]},{"osd":9,"last update":"Sat Feb 17 12:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.58499999999999996,"15min":0.622},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61799999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.60299999999999998,"15min":0.64900000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.59799999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.64500000000000002}]},{"osd":10,"last update":"Sat Feb 17 12:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77700000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.71999999999999997,"15min":0.73599999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.72199999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.77400000000000002}]},{"osd":11,"last update":"Sat Feb 17 12:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.60799999999999998,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.84299999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.57599999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.79700000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":522171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.50700000000000001,"15min":0.46899999999999997},"min":{"1min":0.40200000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.79600000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.48699999999999999,"15min":0.47099999999999997},"min":{"1min":0.34599999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68799999999999994,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57899999999999996}]},{"osd":1,"last update":"Sat Feb 17 12:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.56399999999999995,"15min":0.51400000000000001},"min":{"1min":0.45200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.83299999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.57599999999999996,"15min":0.51800000000000002},"min":{"1min":0.41199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.84799999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.78100000000000003}]},{"osd":3,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56000000000000005,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.77200000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66300000000000003}]},{"osd":4,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.76400000000000001}]},{"osd":5,"last update":"Sat Feb 17 12:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.48899999999999999,"15min":0.44900000000000001},"min":{"1min":0.35799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.92100000000000004,"5min":1.325,"15min":1.325},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.505,"15min":0.49299999999999999},"min":{"1min":0.42099999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83799999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Feb 17 12:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.47299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.59199999999999997,"15min":0.60099999999999998},"min":{"1min":0.39100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80600000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63100000000000001}]},{"osd":7,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.60999999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.85699999999999998,"5min":0.998,"15min":0.998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60899999999999999,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95899999999999996,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.83799999999999997}]},{"osd":9,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.502,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86399999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.51400000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93600000000000005,"5min":1.284,"15min":1.284},"last":0.747}]},{"osd":10,"last update":"Sat Feb 17 12:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.55200000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92500000000000004,"5min":1.347,"15min":1.347},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.72899999999999998,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97199999999999998,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.80300000000000005}]},{"osd":11,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.81200000000000006}]}]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40460,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704692,"statfs":{"total":95995035648,"available":95953604608,"internally_reserved":0,"allocated":987136,"data_stored":461534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.32300000000000001},"min":{"1min":0.26000000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59799999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.255},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.33700000000000002},"min":{"1min":0.26300000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.28499999999999998}]},{"osd":1,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77100000000000002,"5min":0.879,"15min":0.879},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.44800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.752,"5min":0.88,"15min":0.88},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77900000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.38100000000000001}]},{"osd":4,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.39600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.44600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.74299999999999999,"5min":0.876,"15min":0.876},"last":0.59499999999999997}]},{"osd":5,"last update":"Sat Feb 17 12:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.50600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72999999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81399999999999995,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.41099999999999998}]},{"osd":6,"last update":"Sat Feb 17 12:48:53 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.504,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73799999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.495,"15min":0.505},"min":{"1min":0.35099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.316}]},{"osd":7,"last update":"Sat Feb 17 12:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.625,"15min":0.621},"min":{"1min":0.47099999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78600000000000003,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57599999999999996}]},{"osd":8,"last update":"Sat Feb 17 12:47:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91600000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.71299999999999997,"15min":0.72599999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0069999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.56899999999999995}]},{"osd":9,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.64300000000000002,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.81799999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.67200000000000004,"15min":0.70599999999999996},"min":{"1min":0.44900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85899999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.497}]},{"osd":10,"last update":"Sat Feb 17 12:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.75,"15min":0.78300000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84199999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.622},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.76700000000000002},"min":{"1min":0.60499999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.77900000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64200000000000002}]},{"osd":11,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.54000000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":463256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57899999999999996}]},{"osd":1,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.36299999999999999},"min":{"1min":0.28499999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.60899999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.41599999999999998,"15min":0.378},"min":{"1min":0.35099999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.58799999999999997,"5min":0.751,"15min":0.751},"last":0.42999999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.80000000000000004,"15min":0.64000000000000001},"min":{"1min":0.42699999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85899999999999999,"5min":18.73,"15min":18.73},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55100000000000005}]},{"osd":3,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.47799999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85199999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.627},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40000000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.51000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64000000000000001,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Feb 17 12:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.57499999999999996},"min":{"1min":0.47899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79800000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.60499999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.54900000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83699999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.53700000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:48:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.83699999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.69299999999999995}]},{"osd":9,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89200000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.71099999999999997}]},{"osd":10,"last update":"Sat Feb 17 12:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.499,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92600000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.49299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76600000000000001}]},{"osd":11,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.75},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.79800000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1054707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.371,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.82599999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.625,"5min":1.02,"15min":1.02},"last":0.59299999999999997}]},{"osd":1,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.53000000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60099999999999998,"15min":0.53900000000000003},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.81699999999999995}]},{"osd":2,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.435,"15min":0.39300000000000002},"min":{"1min":0.34300000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.45000000000000001,"15min":0.38700000000000001},"min":{"1min":0.34399999999999997,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.71199999999999997}]},{"osd":3,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.54000000000000004},"min":{"1min":0.48299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84299999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.64800000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61799999999999999,"15min":0.55900000000000005},"min":{"1min":0.41899999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93999999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61099999999999999,"15min":0.54000000000000004},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.78700000000000003}]},{"osd":6,"last update":"Sat Feb 17 12:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.621,"15min":0.58799999999999997},"min":{"1min":0.56299999999999994,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98799999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.628,"15min":0.61099999999999999},"min":{"1min":0.53400000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79900000000000004}]},{"osd":7,"last update":"Sat Feb 17 12:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70699999999999996,"15min":0.68400000000000005},"min":{"1min":0.47799999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.86199999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.629,"15min":0.625},"min":{"1min":0.51200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.46600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.82999999999999996}]},{"osd":9,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68899999999999995,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.878},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.58799999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77900000000000003}]},{"osd":10,"last update":"Sat Feb 17 12:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77100000000000002,"15min":0.746},"min":{"1min":0.57999999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.1100000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72899999999999998,"15min":0.69199999999999995},"min":{"1min":0.55700000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.89100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T12:49:42.072 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-02-17T12:49:42.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-17T12:49:42.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.216+0000 7f82bccff640 1 -- 172.21.15.18:0/2712960188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82b8101cc0 msgr2=0x7f82b81040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.216+0000 7f82bccff640 1 --2- 172.21.15.18:0/2712960188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82b8101cc0 0x7f82b81040b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009900 tx=0x7f82a802bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:42.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.216+0000 7f82bccff640 1 -- 172.21.15.18:0/2712960188 shutdown_connections 2024-02-17T12:49:42.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.216+0000 7f82bccff640 1 --2- 172.21.15.18:0/2712960188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b8106f20 0x7f82b8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.216+0000 7f82bccff640 1 --2- 172.21.15.18:0/2712960188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82b81045f0 0x7f82b81069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.216+0000 7f82bccff640 1 --2- 172.21.15.18:0/2712960188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82b8101cc0 0x7f82b81040b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.216+0000 7f82bccff640 1 -- 172.21.15.18:0/2712960188 >> 172.21.15.18:0/2712960188 conn(0x7f82b80fb610 msgr2=0x7f82b80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:42.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.216+0000 7f82bccff640 1 -- 172.21.15.18:0/2712960188 shutdown_connections 2024-02-17T12:49:42.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.216+0000 7f82bccff640 1 -- 172.21.15.18:0/2712960188 wait complete. 2024-02-17T12:49:42.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.217+0000 7f82bccff640 1 Processor -- start 2024-02-17T12:49:42.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.217+0000 7f82bccff640 1 -- start start 2024-02-17T12:49:42.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.217+0000 7f82bccff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82b8101cc0 0x7f82b8101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:42.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.217+0000 7f82bccff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81045f0 0x7f82b80ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:42.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.217+0000 7f82bccff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82b8106f20 0x7f82b81001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:42.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.217+0000 7f82bccff640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b8100910 con 0x7f82b8101cc0 2024-02-17T12:49:42.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.217+0000 7f82bccff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b8100a80 con 0x7f82b81045f0 2024-02-17T12:49:42.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.217+0000 7f82bccff640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b81a12f0 con 0x7f82b8106f20 2024-02-17T12:49:42.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.217+0000 7f82b659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82b8101cc0 0x7f82b8101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:42.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.217+0000 7f82b659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82b8101cc0 0x7f82b8101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34500/0 (socket says 172.21.15.18:34500) 2024-02-17T12:49:42.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.217+0000 7f82b659c640 1 -- 172.21.15.18:0/907109948 learned_addr learned my addr 172.21.15.18:0/907109948 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:42.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82b6d9d640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82b8106f20 0x7f82b81001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:42.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82affff640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81045f0 0x7f82b80ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:42.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82b659c640 1 -- 172.21.15.18:0/907109948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82b8106f20 msgr2=0x7f82b81001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82b659c640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82b8106f20 0x7f82b81001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82b659c640 1 -- 172.21.15.18:0/907109948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81045f0 msgr2=0x7f82b80ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82b659c640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81045f0 0x7f82b80ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82b659c640 1 -- 172.21.15.18:0/907109948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a8009560 con 0x7f82b8101cc0 2024-02-17T12:49:42.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82b659c640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82b8101cc0 0x7f82b8101630 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f82a8036040 tx=0x7f82a80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:42.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82af7fe640 1 -- 172.21.15.18:0/907109948 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a80044c0 con 0x7f82b8101cc0 2024-02-17T12:49:42.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82af7fe640 1 -- 172.21.15.18:0/907109948 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82a80052d0 con 0x7f82b8101cc0 2024-02-17T12:49:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82bccff640 1 -- 172.21.15.18:0/907109948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b81a1540 con 0x7f82b8101cc0 2024-02-17T12:49:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82af7fe640 1 -- 172.21.15.18:0/907109948 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8003e10 con 0x7f82b8101cc0 2024-02-17T12:49:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.218+0000 7f82bccff640 1 -- 172.21.15.18:0/907109948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b81a1aa0 con 0x7f82b8101cc0 2024-02-17T12:49:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.219+0000 7f82bccff640 1 -- 172.21.15.18:0/907109948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b80fd100 con 0x7f82b8101cc0 2024-02-17T12:49:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.220+0000 7f82af7fe640 1 -- 172.21.15.18:0/907109948 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f82a8005440 con 0x7f82b8101cc0 2024-02-17T12:49:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.221+0000 7f82af7fe640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f82a40a2140 0x7f82a40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.221+0000 7f82affff640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f82a40a2140 0x7f82a40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.221+0000 7f82af7fe640 1 -- 172.21.15.18:0/907109948 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f82a80effb0 con 0x7f82b8101cc0 2024-02-17T12:49:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.222+0000 7f82affff640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f82a40a2140 0x7f82a40a4600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f82b81013a0 tx=0x7f82a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.226+0000 7f82af7fe640 1 -- 172.21.15.18:0/907109948 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f82a80b9050 con 0x7f82b8101cc0 2024-02-17T12:49:42.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.404+0000 7f82bccff640 1 -- 172.21.15.18:0/907109948 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82b802ce40 con 0x7f82a40a2140 2024-02-17T12:49:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.414+0000 7f82af7fe640 1 -- 172.21.15.18:0/907109948 <== mgr.14152 v2:172.21.15.18:6800/3026633658 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344773 (secure 0 0 0) 0x7f82b802ce40 con 0x7f82a40a2140 2024-02-17T12:49:42.411 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:42.413 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-02-17T12:49:42.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.417+0000 7f82bccff640 1 -- 172.21.15.18:0/907109948 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f82a40a2140 msgr2=0x7f82a40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.417+0000 7f82bccff640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f82a40a2140 0x7f82a40a4600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f82b81013a0 tx=0x7f82a0005f90 comp rx=0 tx=0).stop 2024-02-17T12:49:42.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.417+0000 7f82bccff640 1 -- 172.21.15.18:0/907109948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82b8101cc0 msgr2=0x7f82b8101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.417+0000 7f82bccff640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82b8101cc0 0x7f82b8101630 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f82a8036040 tx=0x7f82a80040c0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.417+0000 7f82bccff640 1 -- 172.21.15.18:0/907109948 shutdown_connections 2024-02-17T12:49:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.417+0000 7f82bccff640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f82a40a2140 0x7f82a40a4600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.417+0000 7f82bccff640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82b8106f20 0x7f82b81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.417+0000 7f82bccff640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81045f0 0x7f82b80ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.417+0000 7f82bccff640 1 --2- 172.21.15.18:0/907109948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82b8101cc0 0x7f82b8101630 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.417+0000 7f82bccff640 1 -- 172.21.15.18:0/907109948 >> 172.21.15.18:0/907109948 conn(0x7f82b80fb610 msgr2=0x7f82b8102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.417+0000 7f82bccff640 1 -- 172.21.15.18:0/907109948 shutdown_connections 2024-02-17T12:49:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.418+0000 7f82bccff640 1 -- 172.21.15.18:0/907109948 wait complete. 2024-02-17T12:49:42.421 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-02-17T12:49:42.220071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658039,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1262,"num_read_kb":757,"num_write":432,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181749,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383404,"kb_used_data":13452,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558420,"statfs":{"total":1151940427776,"available":1151547822080,"internally_reserved":0,"allocated":13774848,"data_stored":7504557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":61,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003115"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768991+0000","last_change":"2024-02-17T12:47:46.069941+0000","last_active":"2024-02-17T12:49:20.768991+0000","last_peered":"2024-02-17T12:49:20.768991+0000","last_clean":"2024-02-17T12:49:20.768991+0000","last_became_active":"2024-02-17T12:47:46.069192+0000","last_became_peered":"2024-02-17T12:47:46.069192+0000","last_unstale":"2024-02-17T12:49:20.768991+0000","last_undegraded":"2024-02-17T12:49:20.768991+0000","last_fullsized":"2024-02-17T12:49:20.768991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:27.102839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.760213+0000","last_change":"2024-02-17T12:47:46.064872+0000","last_active":"2024-02-17T12:49:20.760213+0000","last_peered":"2024-02-17T12:49:20.760213+0000","last_clean":"2024-02-17T12:49:20.760213+0000","last_became_active":"2024-02-17T12:47:46.064498+0000","last_became_peered":"2024-02-17T12:47:46.064498+0000","last_unstale":"2024-02-17T12:49:20.760213+0000","last_undegraded":"2024-02-17T12:49:20.760213+0000","last_fullsized":"2024-02-17T12:49:20.760213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:30:41.432657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762825+0000","last_change":"2024-02-17T12:47:46.067504+0000","last_active":"2024-02-17T12:49:20.762825+0000","last_peered":"2024-02-17T12:49:20.762825+0000","last_clean":"2024-02-17T12:49:20.762825+0000","last_became_active":"2024-02-17T12:47:46.067155+0000","last_became_peered":"2024-02-17T12:47:46.067155+0000","last_unstale":"2024-02-17T12:49:20.762825+0000","last_undegraded":"2024-02-17T12:49:20.762825+0000","last_fullsized":"2024-02-17T12:49:20.762825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:38:56.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764046+0000","last_change":"2024-02-17T12:47:46.062356+0000","last_active":"2024-02-17T12:49:20.764046+0000","last_peered":"2024-02-17T12:49:20.764046+0000","last_clean":"2024-02-17T12:49:20.764046+0000","last_became_active":"2024-02-17T12:47:46.061988+0000","last_became_peered":"2024-02-17T12:47:46.061988+0000","last_unstale":"2024-02-17T12:49:20.764046+0000","last_undegraded":"2024-02-17T12:49:20.764046+0000","last_fullsized":"2024-02-17T12:49:20.764046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:25:09.120770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.746039+0000","last_change":"2024-02-17T12:47:46.072411+0000","last_active":"2024-02-17T12:49:20.746039+0000","last_peered":"2024-02-17T12:49:20.746039+0000","last_clean":"2024-02-17T12:49:20.746039+0000","last_became_active":"2024-02-17T12:47:46.072067+0000","last_became_peered":"2024-02-17T12:47:46.072067+0000","last_unstale":"2024-02-17T12:49:20.746039+0000","last_undegraded":"2024-02-17T12:49:20.746039+0000","last_fullsized":"2024-02-17T12:49:20.746039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:29:33.051513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.754893+0000","last_change":"2024-02-17T12:47:46.057425+0000","last_active":"2024-02-17T12:49:20.754893+0000","last_peered":"2024-02-17T12:49:20.754893+0000","last_clean":"2024-02-17T12:49:20.754893+0000","last_became_active":"2024-02-17T12:47:46.057001+0000","last_became_peered":"2024-02-17T12:47:46.057001+0000","last_unstale":"2024-02-17T12:49:20.754893+0000","last_undegraded":"2024-02-17T12:49:20.754893+0000","last_fullsized":"2024-02-17T12:49:20.754893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:48:24.135879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758496+0000","last_change":"2024-02-17T12:47:46.069143+0000","last_active":"2024-02-17T12:49:20.758496+0000","last_peered":"2024-02-17T12:49:20.758496+0000","last_clean":"2024-02-17T12:49:20.758496+0000","last_became_active":"2024-02-17T12:47:46.068775+0000","last_became_peered":"2024-02-17T12:47:46.068775+0000","last_unstale":"2024-02-17T12:49:20.758496+0000","last_undegraded":"2024-02-17T12:49:20.758496+0000","last_fullsized":"2024-02-17T12:49:20.758496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:32:39.518205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764182+0000","last_change":"2024-02-17T12:47:46.069579+0000","last_active":"2024-02-17T12:49:20.764182+0000","last_peered":"2024-02-17T12:49:20.764182+0000","last_clean":"2024-02-17T12:49:20.764182+0000","last_became_active":"2024-02-17T12:47:46.069114+0000","last_became_peered":"2024-02-17T12:47:46.069114+0000","last_unstale":"2024-02-17T12:49:20.764182+0000","last_undegraded":"2024-02-17T12:49:20.764182+0000","last_fullsized":"2024-02-17T12:49:20.764182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:54:58.100511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768271+0000","last_change":"2024-02-17T12:47:46.079103+0000","last_active":"2024-02-17T12:49:20.768271+0000","last_peered":"2024-02-17T12:49:20.768271+0000","last_clean":"2024-02-17T12:49:20.768271+0000","last_became_active":"2024-02-17T12:47:46.077992+0000","last_became_peered":"2024-02-17T12:47:46.077992+0000","last_unstale":"2024-02-17T12:49:20.768271+0000","last_undegraded":"2024-02-17T12:49:20.768271+0000","last_fullsized":"2024-02-17T12:49:20.768271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:12:59.337341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758225+0000","last_change":"2024-02-17T12:47:46.062897+0000","last_active":"2024-02-17T12:49:20.758225+0000","last_peered":"2024-02-17T12:49:20.758225+0000","last_clean":"2024-02-17T12:49:20.758225+0000","last_became_active":"2024-02-17T12:47:46.062571+0000","last_became_peered":"2024-02-17T12:47:46.062571+0000","last_unstale":"2024-02-17T12:49:20.758225+0000","last_undegraded":"2024-02-17T12:49:20.758225+0000","last_fullsized":"2024-02-17T12:49:20.758225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:57:02.845400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.760478+0000","last_change":"2024-02-17T12:47:46.080183+0000","last_active":"2024-02-17T12:49:20.760478+0000","last_peered":"2024-02-17T12:49:20.760478+0000","last_clean":"2024-02-17T12:49:20.760478+0000","last_became_active":"2024-02-17T12:47:46.079827+0000","last_became_peered":"2024-02-17T12:47:46.079827+0000","last_unstale":"2024-02-17T12:49:20.760478+0000","last_undegraded":"2024-02-17T12:49:20.760478+0000","last_fullsized":"2024-02-17T12:49:20.760478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:56:10.193200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762695+0000","last_change":"2024-02-17T12:47:46.061849+0000","last_active":"2024-02-17T12:49:20.762695+0000","last_peered":"2024-02-17T12:49:20.762695+0000","last_clean":"2024-02-17T12:49:20.762695+0000","last_became_active":"2024-02-17T12:47:46.061458+0000","last_became_peered":"2024-02-17T12:47:46.061458+0000","last_unstale":"2024-02-17T12:49:20.762695+0000","last_undegraded":"2024-02-17T12:49:20.762695+0000","last_fullsized":"2024-02-17T12:49:20.762695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:02:09.141292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.765932+0000","last_change":"2024-02-17T12:47:46.069356+0000","last_active":"2024-02-17T12:49:20.765932+0000","last_peered":"2024-02-17T12:49:20.765932+0000","last_clean":"2024-02-17T12:49:20.765932+0000","last_became_active":"2024-02-17T12:47:46.068567+0000","last_became_peered":"2024-02-17T12:47:46.068567+0000","last_unstale":"2024-02-17T12:49:20.765932+0000","last_undegraded":"2024-02-17T12:49:20.765932+0000","last_fullsized":"2024-02-17T12:49:20.765932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:08:01.872558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.757541+0000","last_change":"2024-02-17T12:47:46.062684+0000","last_active":"2024-02-17T12:49:20.757541+0000","last_peered":"2024-02-17T12:49:20.757541+0000","last_clean":"2024-02-17T12:49:20.757541+0000","last_became_active":"2024-02-17T12:47:46.062221+0000","last_became_peered":"2024-02-17T12:47:46.062221+0000","last_unstale":"2024-02-17T12:49:20.757541+0000","last_undegraded":"2024-02-17T12:49:20.757541+0000","last_fullsized":"2024-02-17T12:49:20.757541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:37:11.513336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.760133+0000","last_change":"2024-02-17T12:47:46.065961+0000","last_active":"2024-02-17T12:49:20.760133+0000","last_peered":"2024-02-17T12:49:20.760133+0000","last_clean":"2024-02-17T12:49:20.760133+0000","last_became_active":"2024-02-17T12:47:46.065600+0000","last_became_peered":"2024-02-17T12:47:46.065600+0000","last_unstale":"2024-02-17T12:49:20.760133+0000","last_undegraded":"2024-02-17T12:49:20.760133+0000","last_fullsized":"2024-02-17T12:49:20.760133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:24:46.205077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762547+0000","last_change":"2024-02-17T12:47:46.080150+0000","last_active":"2024-02-17T12:49:20.762547+0000","last_peered":"2024-02-17T12:49:20.762547+0000","last_clean":"2024-02-17T12:49:20.762547+0000","last_became_active":"2024-02-17T12:47:46.079813+0000","last_became_peered":"2024-02-17T12:47:46.079813+0000","last_unstale":"2024-02-17T12:49:20.762547+0000","last_undegraded":"2024-02-17T12:49:20.762547+0000","last_fullsized":"2024-02-17T12:49:20.762547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:59:00.868964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.769045+0000","last_change":"2024-02-17T12:47:46.070202+0000","last_active":"2024-02-17T12:49:20.769045+0000","last_peered":"2024-02-17T12:49:20.769045+0000","last_clean":"2024-02-17T12:49:20.769045+0000","last_became_active":"2024-02-17T12:47:46.069628+0000","last_became_peered":"2024-02-17T12:47:46.069628+0000","last_unstale":"2024-02-17T12:49:20.769045+0000","last_undegraded":"2024-02-17T12:49:20.769045+0000","last_fullsized":"2024-02-17T12:49:20.769045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:44:17.481681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763818+0000","last_change":"2024-02-17T12:47:46.067810+0000","last_active":"2024-02-17T12:49:20.763818+0000","last_peered":"2024-02-17T12:49:20.763818+0000","last_clean":"2024-02-17T12:49:20.763818+0000","last_became_active":"2024-02-17T12:47:46.067456+0000","last_became_peered":"2024-02-17T12:47:46.067456+0000","last_unstale":"2024-02-17T12:49:20.763818+0000","last_undegraded":"2024-02-17T12:49:20.763818+0000","last_fullsized":"2024-02-17T12:49:20.763818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:27:03.646597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.753377+0000","last_change":"2024-02-17T12:47:46.076268+0000","last_active":"2024-02-17T12:49:20.753377+0000","last_peered":"2024-02-17T12:49:20.753377+0000","last_clean":"2024-02-17T12:49:20.753377+0000","last_became_active":"2024-02-17T12:47:46.075606+0000","last_became_peered":"2024-02-17T12:47:46.075606+0000","last_unstale":"2024-02-17T12:49:20.753377+0000","last_undegraded":"2024-02-17T12:49:20.753377+0000","last_fullsized":"2024-02-17T12:49:20.753377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:16:35.805008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":142,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:38.871544+0000","last_change":"2024-02-17T12:47:45.066780+0000","last_active":"2024-02-17T12:49:38.871544+0000","last_peered":"2024-02-17T12:49:38.871544+0000","last_clean":"2024-02-17T12:49:38.871544+0000","last_became_active":"2024-02-17T12:47:45.066039+0000","last_became_peered":"2024-02-17T12:47:45.066039+0000","last_unstale":"2024-02-17T12:49:38.871544+0000","last_undegraded":"2024-02-17T12:49:38.871544+0000","last_fullsized":"2024-02-17T12:49:38.871544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:36:25.420534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763169+0000","last_change":"2024-02-17T12:47:46.079603+0000","last_active":"2024-02-17T12:49:20.763169+0000","last_peered":"2024-02-17T12:49:20.763169+0000","last_clean":"2024-02-17T12:49:20.763169+0000","last_became_active":"2024-02-17T12:47:46.079246+0000","last_became_peered":"2024-02-17T12:47:46.079246+0000","last_unstale":"2024-02-17T12:49:20.763169+0000","last_undegraded":"2024-02-17T12:49:20.763169+0000","last_fullsized":"2024-02-17T12:49:20.763169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:32:04.975778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165983+0000","last_change":"2024-02-17T12:47:45.061868+0000","last_active":"2024-02-17T12:48:52.165983+0000","last_peered":"2024-02-17T12:48:52.165983+0000","last_clean":"2024-02-17T12:48:52.165983+0000","last_became_active":"2024-02-17T12:47:45.061512+0000","last_became_peered":"2024-02-17T12:47:45.061512+0000","last_unstale":"2024-02-17T12:48:52.165983+0000","last_undegraded":"2024-02-17T12:48:52.165983+0000","last_fullsized":"2024-02-17T12:48:52.165983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:53:48.405184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763185+0000","last_change":"2024-02-17T12:47:46.071055+0000","last_active":"2024-02-17T12:49:20.763185+0000","last_peered":"2024-02-17T12:49:20.763185+0000","last_clean":"2024-02-17T12:49:20.763185+0000","last_became_active":"2024-02-17T12:47:46.070649+0000","last_became_peered":"2024-02-17T12:47:46.070649+0000","last_unstale":"2024-02-17T12:49:20.763185+0000","last_undegraded":"2024-02-17T12:49:20.763185+0000","last_fullsized":"2024-02-17T12:49:20.763185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:10:22.078746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.178268+0000","last_change":"2024-02-17T12:48:52.177755+0000","last_active":"2024-02-17T12:48:52.178268+0000","last_peered":"2024-02-17T12:48:52.178268+0000","last_clean":"2024-02-17T12:48:52.178268+0000","last_became_active":"2024-02-17T12:48:52.165119+0000","last_became_peered":"2024-02-17T12:48:52.165119+0000","last_unstale":"2024-02-17T12:48:52.178268+0000","last_undegraded":"2024-02-17T12:48:52.178268+0000","last_fullsized":"2024-02-17T12:48:52.178268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:38.834337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767093+0000","last_change":"2024-02-17T12:47:46.076448+0000","last_active":"2024-02-17T12:49:20.767093+0000","last_peered":"2024-02-17T12:49:20.767093+0000","last_clean":"2024-02-17T12:49:20.767093+0000","last_became_active":"2024-02-17T12:47:46.075250+0000","last_became_peered":"2024-02-17T12:47:46.075250+0000","last_unstale":"2024-02-17T12:49:20.767093+0000","last_undegraded":"2024-02-17T12:49:20.767093+0000","last_fullsized":"2024-02-17T12:49:20.767093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:11:45.846764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163335+0000","last_change":"2024-02-17T12:47:45.062699+0000","last_active":"2024-02-17T12:48:52.163335+0000","last_peered":"2024-02-17T12:48:52.163335+0000","last_clean":"2024-02-17T12:48:52.163335+0000","last_became_active":"2024-02-17T12:47:45.062322+0000","last_became_peered":"2024-02-17T12:47:45.062322+0000","last_unstale":"2024-02-17T12:48:52.163335+0000","last_undegraded":"2024-02-17T12:48:52.163335+0000","last_fullsized":"2024-02-17T12:48:52.163335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:14:06.112393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767082+0000","last_change":"2024-02-17T12:47:46.080278+0000","last_active":"2024-02-17T12:49:20.767082+0000","last_peered":"2024-02-17T12:49:20.767082+0000","last_clean":"2024-02-17T12:49:20.767082+0000","last_became_active":"2024-02-17T12:47:46.079924+0000","last_became_peered":"2024-02-17T12:47:46.079924+0000","last_unstale":"2024-02-17T12:49:20.767082+0000","last_undegraded":"2024-02-17T12:49:20.767082+0000","last_fullsized":"2024-02-17T12:49:20.767082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:54:08.759445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":152,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:37.543298+0000","last_change":"2024-02-17T12:48:52.469734+0000","last_active":"2024-02-17T12:49:37.543298+0000","last_peered":"2024-02-17T12:49:37.543298+0000","last_clean":"2024-02-17T12:49:37.543298+0000","last_became_active":"2024-02-17T12:48:52.465945+0000","last_became_peered":"2024-02-17T12:48:52.465945+0000","last_unstale":"2024-02-17T12:49:37.543298+0000","last_undegraded":"2024-02-17T12:49:37.543298+0000","last_fullsized":"2024-02-17T12:49:37.543298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:05:27.931619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.749939+0000","last_change":"2024-02-17T12:47:46.065957+0000","last_active":"2024-02-17T12:49:20.749939+0000","last_peered":"2024-02-17T12:49:20.749939+0000","last_clean":"2024-02-17T12:49:20.749939+0000","last_became_active":"2024-02-17T12:47:46.065638+0000","last_became_peered":"2024-02-17T12:47:46.065638+0000","last_unstale":"2024-02-17T12:49:20.749939+0000","last_undegraded":"2024-02-17T12:49:20.749939+0000","last_fullsized":"2024-02-17T12:49:20.749939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:02:39.295990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463600+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:48:52.463600+0000","last_peered":"2024-02-17T12:48:52.463600+0000","last_clean":"2024-02-17T12:48:52.463600+0000","last_became_active":"2024-02-17T12:47:45.066235+0000","last_became_peered":"2024-02-17T12:47:45.066235+0000","last_unstale":"2024-02-17T12:48:52.463600+0000","last_undegraded":"2024-02-17T12:48:52.463600+0000","last_fullsized":"2024-02-17T12:48:52.463600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:46:53.731920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763770+0000","last_change":"2024-02-17T12:47:46.072029+0000","last_active":"2024-02-17T12:49:20.763770+0000","last_peered":"2024-02-17T12:49:20.763770+0000","last_clean":"2024-02-17T12:49:20.763770+0000","last_became_active":"2024-02-17T12:47:46.071716+0000","last_became_peered":"2024-02-17T12:47:46.071716+0000","last_unstale":"2024-02-17T12:49:20.763770+0000","last_undegraded":"2024-02-17T12:49:20.763770+0000","last_fullsized":"2024-02-17T12:49:20.763770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:33.076266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163802+0000","last_change":"2024-02-17T12:47:45.067443+0000","last_active":"2024-02-17T12:48:52.163802+0000","last_peered":"2024-02-17T12:48:52.163802+0000","last_clean":"2024-02-17T12:48:52.163802+0000","last_became_active":"2024-02-17T12:47:45.067090+0000","last_became_peered":"2024-02-17T12:47:45.067090+0000","last_unstale":"2024-02-17T12:48:52.163802+0000","last_undegraded":"2024-02-17T12:48:52.163802+0000","last_fullsized":"2024-02-17T12:48:52.163802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:17:39.874810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768189+0000","last_change":"2024-02-17T12:47:46.072275+0000","last_active":"2024-02-17T12:49:20.768189+0000","last_peered":"2024-02-17T12:49:20.768189+0000","last_clean":"2024-02-17T12:49:20.768189+0000","last_became_active":"2024-02-17T12:47:46.071951+0000","last_became_peered":"2024-02-17T12:47:46.071951+0000","last_unstale":"2024-02-17T12:49:20.768189+0000","last_undegraded":"2024-02-17T12:49:20.768189+0000","last_fullsized":"2024-02-17T12:49:20.768189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:45:58.580854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.159772+0000","last_change":"2024-02-17T12:47:45.058875+0000","last_active":"2024-02-17T12:48:52.159772+0000","last_peered":"2024-02-17T12:48:52.159772+0000","last_clean":"2024-02-17T12:48:52.159772+0000","last_became_active":"2024-02-17T12:47:45.058433+0000","last_became_peered":"2024-02-17T12:47:45.058433+0000","last_unstale":"2024-02-17T12:48:52.159772+0000","last_undegraded":"2024-02-17T12:48:52.159772+0000","last_fullsized":"2024-02-17T12:48:52.159772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:49.932205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762407+0000","last_change":"2024-02-17T12:47:46.070233+0000","last_active":"2024-02-17T12:49:20.762407+0000","last_peered":"2024-02-17T12:49:20.762407+0000","last_clean":"2024-02-17T12:49:20.762407+0000","last_became_active":"2024-02-17T12:47:46.069811+0000","last_became_peered":"2024-02-17T12:47:46.069811+0000","last_unstale":"2024-02-17T12:49:20.762407+0000","last_undegraded":"2024-02-17T12:49:20.762407+0000","last_fullsized":"2024-02-17T12:49:20.762407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:50:00.885517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":242,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:38.544493+0000","last_change":"2024-02-17T12:47:45.068778+0000","last_active":"2024-02-17T12:49:38.544493+0000","last_peered":"2024-02-17T12:49:38.544493+0000","last_clean":"2024-02-17T12:49:38.544493+0000","last_became_active":"2024-02-17T12:47:45.068375+0000","last_became_peered":"2024-02-17T12:47:45.068375+0000","last_unstale":"2024-02-17T12:49:38.544493+0000","last_undegraded":"2024-02-17T12:49:38.544493+0000","last_fullsized":"2024-02-17T12:49:38.544493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:42:57.600138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":218,"num_read_kb":218,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.765647+0000","last_change":"2024-02-17T12:47:46.076802+0000","last_active":"2024-02-17T12:49:20.765647+0000","last_peered":"2024-02-17T12:49:20.765647+0000","last_clean":"2024-02-17T12:49:20.765647+0000","last_became_active":"2024-02-17T12:47:46.076433+0000","last_became_peered":"2024-02-17T12:47:46.076433+0000","last_unstale":"2024-02-17T12:49:20.765647+0000","last_undegraded":"2024-02-17T12:49:20.765647+0000","last_fullsized":"2024-02-17T12:49:20.765647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.352488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157571+0000","last_change":"2024-02-17T12:47:45.059100+0000","last_active":"2024-02-17T12:48:52.157571+0000","last_peered":"2024-02-17T12:48:52.157571+0000","last_clean":"2024-02-17T12:48:52.157571+0000","last_became_active":"2024-02-17T12:47:45.058707+0000","last_became_peered":"2024-02-17T12:47:45.058707+0000","last_unstale":"2024-02-17T12:48:52.157571+0000","last_undegraded":"2024-02-17T12:48:52.157571+0000","last_fullsized":"2024-02-17T12:48:52.157571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:28:27.044127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.757936+0000","last_change":"2024-02-17T12:47:46.065088+0000","last_active":"2024-02-17T12:49:20.757936+0000","last_peered":"2024-02-17T12:49:20.757936+0000","last_clean":"2024-02-17T12:49:20.757936+0000","last_became_active":"2024-02-17T12:47:46.064754+0000","last_became_peered":"2024-02-17T12:47:46.064754+0000","last_unstale":"2024-02-17T12:49:20.757936+0000","last_undegraded":"2024-02-17T12:49:20.757936+0000","last_fullsized":"2024-02-17T12:49:20.757936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:33:10.652881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160868+0000","last_change":"2024-02-17T12:47:45.055403+0000","last_active":"2024-02-17T12:48:52.160868+0000","last_peered":"2024-02-17T12:48:52.160868+0000","last_clean":"2024-02-17T12:48:52.160868+0000","last_became_active":"2024-02-17T12:47:45.055083+0000","last_became_peered":"2024-02-17T12:47:45.055083+0000","last_unstale":"2024-02-17T12:48:52.160868+0000","last_undegraded":"2024-02-17T12:48:52.160868+0000","last_fullsized":"2024-02-17T12:48:52.160868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:05:51.295074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767519+0000","last_change":"2024-02-17T12:47:46.074883+0000","last_active":"2024-02-17T12:49:20.767519+0000","last_peered":"2024-02-17T12:49:20.767519+0000","last_clean":"2024-02-17T12:49:20.767519+0000","last_became_active":"2024-02-17T12:47:46.074414+0000","last_became_peered":"2024-02-17T12:47:46.074414+0000","last_unstale":"2024-02-17T12:49:20.767519+0000","last_undegraded":"2024-02-17T12:49:20.767519+0000","last_fullsized":"2024-02-17T12:49:20.767519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:35:55.026587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":282,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:38.544476+0000","last_change":"2024-02-17T12:47:45.057627+0000","last_active":"2024-02-17T12:49:38.544476+0000","last_peered":"2024-02-17T12:49:38.544476+0000","last_clean":"2024-02-17T12:49:38.544476+0000","last_became_active":"2024-02-17T12:47:45.057297+0000","last_became_peered":"2024-02-17T12:47:45.057297+0000","last_unstale":"2024-02-17T12:49:38.544476+0000","last_undegraded":"2024-02-17T12:49:38.544476+0000","last_fullsized":"2024-02-17T12:49:38.544476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:00:17.159939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59951,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":229,"num_read_kb":173,"num_write":36,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.761631+0000","last_change":"2024-02-17T12:47:46.076854+0000","last_active":"2024-02-17T12:49:20.761631+0000","last_peered":"2024-02-17T12:49:20.761631+0000","last_clean":"2024-02-17T12:49:20.761631+0000","last_became_active":"2024-02-17T12:47:46.076534+0000","last_became_peered":"2024-02-17T12:47:46.076534+0000","last_unstale":"2024-02-17T12:49:20.761631+0000","last_undegraded":"2024-02-17T12:49:20.761631+0000","last_fullsized":"2024-02-17T12:49:20.761631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:43:01.979075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163878+0000","last_change":"2024-02-17T12:47:45.060090+0000","last_active":"2024-02-17T12:48:52.163878+0000","last_peered":"2024-02-17T12:48:52.163878+0000","last_clean":"2024-02-17T12:48:52.163878+0000","last_became_active":"2024-02-17T12:47:45.058942+0000","last_became_peered":"2024-02-17T12:47:45.058942+0000","last_unstale":"2024-02-17T12:48:52.163878+0000","last_undegraded":"2024-02-17T12:48:52.163878+0000","last_fullsized":"2024-02-17T12:48:52.163878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:30:52.247140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767896+0000","last_change":"2024-02-17T12:47:46.071982+0000","last_active":"2024-02-17T12:49:20.767896+0000","last_peered":"2024-02-17T12:49:20.767896+0000","last_clean":"2024-02-17T12:49:20.767896+0000","last_became_active":"2024-02-17T12:47:46.071639+0000","last_became_peered":"2024-02-17T12:47:46.071639+0000","last_unstale":"2024-02-17T12:49:20.767896+0000","last_undegraded":"2024-02-17T12:49:20.767896+0000","last_fullsized":"2024-02-17T12:49:20.767896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:43:50.658132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160287+0000","last_change":"2024-02-17T12:47:45.060017+0000","last_active":"2024-02-17T12:48:52.160287+0000","last_peered":"2024-02-17T12:48:52.160287+0000","last_clean":"2024-02-17T12:48:52.160287+0000","last_became_active":"2024-02-17T12:47:45.059104+0000","last_became_peered":"2024-02-17T12:47:45.059104+0000","last_unstale":"2024-02-17T12:48:52.160287+0000","last_undegraded":"2024-02-17T12:48:52.160287+0000","last_fullsized":"2024-02-17T12:48:52.160287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:56:50.457297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768545+0000","last_change":"2024-02-17T12:47:46.072250+0000","last_active":"2024-02-17T12:49:20.768545+0000","last_peered":"2024-02-17T12:49:20.768545+0000","last_clean":"2024-02-17T12:49:20.768545+0000","last_became_active":"2024-02-17T12:47:46.071856+0000","last_became_peered":"2024-02-17T12:47:46.071856+0000","last_unstale":"2024-02-17T12:49:20.768545+0000","last_undegraded":"2024-02-17T12:49:20.768545+0000","last_fullsized":"2024-02-17T12:49:20.768545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:46:04.628146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":138,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:38.916285+0000","last_change":"2024-02-17T12:47:45.056386+0000","last_active":"2024-02-17T12:49:38.916285+0000","last_peered":"2024-02-17T12:49:38.916285+0000","last_clean":"2024-02-17T12:49:38.916285+0000","last_became_active":"2024-02-17T12:47:45.055630+0000","last_became_peered":"2024-02-17T12:47:45.055630+0000","last_unstale":"2024-02-17T12:49:38.916285+0000","last_undegraded":"2024-02-17T12:49:38.916285+0000","last_fullsized":"2024-02-17T12:49:38.916285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:16.388950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.759754+0000","last_change":"2024-02-17T12:47:46.071612+0000","last_active":"2024-02-17T12:49:20.759754+0000","last_peered":"2024-02-17T12:49:20.759754+0000","last_clean":"2024-02-17T12:49:20.759754+0000","last_became_active":"2024-02-17T12:47:46.071237+0000","last_became_peered":"2024-02-17T12:47:46.071237+0000","last_unstale":"2024-02-17T12:49:20.759754+0000","last_undegraded":"2024-02-17T12:49:20.759754+0000","last_fullsized":"2024-02-17T12:49:20.759754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:24:01.099679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163081+0000","last_change":"2024-02-17T12:47:45.066258+0000","last_active":"2024-02-17T12:48:52.163081+0000","last_peered":"2024-02-17T12:48:52.163081+0000","last_clean":"2024-02-17T12:48:52.163081+0000","last_became_active":"2024-02-17T12:47:45.065813+0000","last_became_peered":"2024-02-17T12:47:45.065813+0000","last_unstale":"2024-02-17T12:48:52.163081+0000","last_undegraded":"2024-02-17T12:48:52.163081+0000","last_fullsized":"2024-02-17T12:48:52.163081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:06:43.298573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763292+0000","last_change":"2024-02-17T12:47:46.080347+0000","last_active":"2024-02-17T12:49:20.763292+0000","last_peered":"2024-02-17T12:49:20.763292+0000","last_clean":"2024-02-17T12:49:20.763292+0000","last_became_active":"2024-02-17T12:47:46.080083+0000","last_became_peered":"2024-02-17T12:47:46.080083+0000","last_unstale":"2024-02-17T12:49:20.763292+0000","last_undegraded":"2024-02-17T12:49:20.763292+0000","last_fullsized":"2024-02-17T12:49:20.763292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:24:41.807633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163573+0000","last_change":"2024-02-17T12:47:45.062814+0000","last_active":"2024-02-17T12:48:52.163573+0000","last_peered":"2024-02-17T12:48:52.163573+0000","last_clean":"2024-02-17T12:48:52.163573+0000","last_became_active":"2024-02-17T12:47:45.062418+0000","last_became_peered":"2024-02-17T12:47:45.062418+0000","last_unstale":"2024-02-17T12:48:52.163573+0000","last_undegraded":"2024-02-17T12:48:52.163573+0000","last_fullsized":"2024-02-17T12:48:52.163573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:49:34.577275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.759382+0000","last_change":"2024-02-17T12:47:46.067742+0000","last_active":"2024-02-17T12:49:20.759382+0000","last_peered":"2024-02-17T12:49:20.759382+0000","last_clean":"2024-02-17T12:49:20.759382+0000","last_became_active":"2024-02-17T12:47:46.067329+0000","last_became_peered":"2024-02-17T12:47:46.067329+0000","last_unstale":"2024-02-17T12:49:20.759382+0000","last_undegraded":"2024-02-17T12:49:20.759382+0000","last_fullsized":"2024-02-17T12:49:20.759382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:58:19.432999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.156444+0000","last_change":"2024-02-17T12:47:45.065461+0000","last_active":"2024-02-17T12:48:52.156444+0000","last_peered":"2024-02-17T12:48:52.156444+0000","last_clean":"2024-02-17T12:48:52.156444+0000","last_became_active":"2024-02-17T12:47:45.065066+0000","last_became_peered":"2024-02-17T12:47:45.065066+0000","last_unstale":"2024-02-17T12:48:52.156444+0000","last_undegraded":"2024-02-17T12:48:52.156444+0000","last_fullsized":"2024-02-17T12:48:52.156444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:25:47.292901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764291+0000","last_change":"2024-02-17T12:47:46.074222+0000","last_active":"2024-02-17T12:49:20.764291+0000","last_peered":"2024-02-17T12:49:20.764291+0000","last_clean":"2024-02-17T12:49:20.764291+0000","last_became_active":"2024-02-17T12:47:46.073886+0000","last_became_peered":"2024-02-17T12:47:46.073886+0000","last_unstale":"2024-02-17T12:49:20.764291+0000","last_undegraded":"2024-02-17T12:49:20.764291+0000","last_fullsized":"2024-02-17T12:49:20.764291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:46:18.063422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":134,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:38.872177+0000","last_change":"2024-02-17T12:47:45.071414+0000","last_active":"2024-02-17T12:49:38.872177+0000","last_peered":"2024-02-17T12:49:38.872177+0000","last_clean":"2024-02-17T12:49:38.872177+0000","last_became_active":"2024-02-17T12:47:45.071127+0000","last_became_peered":"2024-02-17T12:47:45.071127+0000","last_unstale":"2024-02-17T12:49:38.872177+0000","last_undegraded":"2024-02-17T12:49:38.872177+0000","last_fullsized":"2024-02-17T12:49:38.872177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:13:21.187718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.750322+0000","last_change":"2024-02-17T12:47:46.059412+0000","last_active":"2024-02-17T12:49:20.750322+0000","last_peered":"2024-02-17T12:49:20.750322+0000","last_clean":"2024-02-17T12:49:20.750322+0000","last_became_active":"2024-02-17T12:47:46.059100+0000","last_became_peered":"2024-02-17T12:47:46.059100+0000","last_unstale":"2024-02-17T12:49:20.750322+0000","last_undegraded":"2024-02-17T12:49:20.750322+0000","last_fullsized":"2024-02-17T12:49:20.750322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:35.930613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763100+0000","last_change":"2024-02-17T12:47:46.071467+0000","last_active":"2024-02-17T12:49:20.763100+0000","last_peered":"2024-02-17T12:49:20.763100+0000","last_clean":"2024-02-17T12:49:20.763100+0000","last_became_active":"2024-02-17T12:47:46.070879+0000","last_became_peered":"2024-02-17T12:47:46.070879+0000","last_unstale":"2024-02-17T12:49:20.763100+0000","last_undegraded":"2024-02-17T12:49:20.763100+0000","last_fullsized":"2024-02-17T12:49:20.763100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:18:49.682842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":135,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:38.915518+0000","last_change":"2024-02-17T12:47:45.070545+0000","last_active":"2024-02-17T12:49:38.915518+0000","last_peered":"2024-02-17T12:49:38.915518+0000","last_clean":"2024-02-17T12:49:38.915518+0000","last_became_active":"2024-02-17T12:47:45.070265+0000","last_became_peered":"2024-02-17T12:47:45.070265+0000","last_unstale":"2024-02-17T12:49:38.915518+0000","last_undegraded":"2024-02-17T12:49:38.915518+0000","last_fullsized":"2024-02-17T12:49:38.915518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.763735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766601+0000","last_change":"2024-02-17T12:47:46.068539+0000","last_active":"2024-02-17T12:49:20.766601+0000","last_peered":"2024-02-17T12:49:20.766601+0000","last_clean":"2024-02-17T12:49:20.766601+0000","last_became_active":"2024-02-17T12:47:46.068169+0000","last_became_peered":"2024-02-17T12:47:46.068169+0000","last_unstale":"2024-02-17T12:49:20.766601+0000","last_undegraded":"2024-02-17T12:49:20.766601+0000","last_fullsized":"2024-02-17T12:49:20.766601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:27:28.485868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464598+0000","last_change":"2024-02-17T12:47:45.066615+0000","last_active":"2024-02-17T12:48:52.464598+0000","last_peered":"2024-02-17T12:48:52.464598+0000","last_clean":"2024-02-17T12:48:52.464598+0000","last_became_active":"2024-02-17T12:47:45.066299+0000","last_became_peered":"2024-02-17T12:47:45.066299+0000","last_unstale":"2024-02-17T12:48:52.464598+0000","last_undegraded":"2024-02-17T12:48:52.464598+0000","last_fullsized":"2024-02-17T12:48:52.464598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:34:02.040002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157254+0000","last_change":"2024-02-17T12:46:32.556576+0000","last_active":"2024-02-17T12:48:52.157254+0000","last_peered":"2024-02-17T12:48:52.157254+0000","last_clean":"2024-02-17T12:48:52.157254+0000","last_became_active":"2024-02-17T12:46:32.556160+0000","last_became_peered":"2024-02-17T12:46:32.556160+0000","last_unstale":"2024-02-17T12:48:52.157254+0000","last_undegraded":"2024-02-17T12:48:52.157254+0000","last_fullsized":"2024-02-17T12:48:52.157254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:43:13.147248+0000","last_clean_scrub_stamp":"2024-02-17T12:43:13.147248+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:57:52.244951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767280+0000","last_change":"2024-02-17T12:48:52.166240+0000","last_active":"2024-02-17T12:49:20.767280+0000","last_peered":"2024-02-17T12:49:20.767280+0000","last_clean":"2024-02-17T12:49:20.767280+0000","last_became_active":"2024-02-17T12:48:52.165882+0000","last_became_peered":"2024-02-17T12:48:52.165882+0000","last_unstale":"2024-02-17T12:49:20.767280+0000","last_undegraded":"2024-02-17T12:49:20.767280+0000","last_fullsized":"2024-02-17T12:49:20.767280+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:56:44.360362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.157273+0000","last_change":"2024-02-17T12:47:45.061112+0000","last_active":"2024-02-17T12:48:52.157273+0000","last_peered":"2024-02-17T12:48:52.157273+0000","last_clean":"2024-02-17T12:48:52.157273+0000","last_became_active":"2024-02-17T12:47:45.060633+0000","last_became_peered":"2024-02-17T12:47:45.060633+0000","last_unstale":"2024-02-17T12:48:52.157273+0000","last_undegraded":"2024-02-17T12:48:52.157273+0000","last_fullsized":"2024-02-17T12:48:52.157273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:29:29.421744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163918+0000","last_change":"2024-02-17T12:47:45.066897+0000","last_active":"2024-02-17T12:48:52.163918+0000","last_peered":"2024-02-17T12:48:52.163918+0000","last_clean":"2024-02-17T12:48:52.163918+0000","last_became_active":"2024-02-17T12:47:45.066357+0000","last_became_peered":"2024-02-17T12:47:45.066357+0000","last_unstale":"2024-02-17T12:48:52.163918+0000","last_undegraded":"2024-02-17T12:48:52.163918+0000","last_fullsized":"2024-02-17T12:48:52.163918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:50:09.840414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.767577+0000","last_change":"2024-02-17T12:47:46.072412+0000","last_active":"2024-02-17T12:49:20.767577+0000","last_peered":"2024-02-17T12:49:20.767577+0000","last_clean":"2024-02-17T12:49:20.767577+0000","last_became_active":"2024-02-17T12:47:46.072105+0000","last_became_peered":"2024-02-17T12:47:46.072105+0000","last_unstale":"2024-02-17T12:49:20.767577+0000","last_undegraded":"2024-02-17T12:49:20.767577+0000","last_fullsized":"2024-02-17T12:49:20.767577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:33:19.367588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'14","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:21.070020+0000","last_change":"2024-02-17T12:47:45.067673+0000","last_active":"2024-02-17T12:49:21.070020+0000","last_peered":"2024-02-17T12:49:21.070020+0000","last_clean":"2024-02-17T12:49:21.070020+0000","last_became_active":"2024-02-17T12:47:45.067212+0000","last_became_peered":"2024-02-17T12:47:45.067212+0000","last_unstale":"2024-02-17T12:49:21.070020+0000","last_undegraded":"2024-02-17T12:49:21.070020+0000","last_fullsized":"2024-02-17T12:49:21.070020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:15:04.236158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763306+0000","last_change":"2024-02-17T12:47:46.077476+0000","last_active":"2024-02-17T12:49:20.763306+0000","last_peered":"2024-02-17T12:49:20.763306+0000","last_clean":"2024-02-17T12:49:20.763306+0000","last_became_active":"2024-02-17T12:47:46.077101+0000","last_became_peered":"2024-02-17T12:47:46.077101+0000","last_unstale":"2024-02-17T12:49:20.763306+0000","last_undegraded":"2024-02-17T12:49:20.763306+0000","last_fullsized":"2024-02-17T12:49:20.763306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:41:41.402101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.464561+0000","last_change":"2024-02-17T12:47:45.066693+0000","last_active":"2024-02-17T12:48:52.464561+0000","last_peered":"2024-02-17T12:48:52.464561+0000","last_clean":"2024-02-17T12:48:52.464561+0000","last_became_active":"2024-02-17T12:47:45.066349+0000","last_became_peered":"2024-02-17T12:47:45.066349+0000","last_unstale":"2024-02-17T12:48:52.464561+0000","last_undegraded":"2024-02-17T12:48:52.464561+0000","last_fullsized":"2024-02-17T12:48:52.464561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:30.633577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.759614+0000","last_change":"2024-02-17T12:47:46.072899+0000","last_active":"2024-02-17T12:49:20.759614+0000","last_peered":"2024-02-17T12:49:20.759614+0000","last_clean":"2024-02-17T12:49:20.759614+0000","last_became_active":"2024-02-17T12:47:46.072488+0000","last_became_peered":"2024-02-17T12:47:46.072488+0000","last_unstale":"2024-02-17T12:49:20.759614+0000","last_undegraded":"2024-02-17T12:49:20.759614+0000","last_fullsized":"2024-02-17T12:49:20.759614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:05:43.675086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.463459+0000","last_change":"2024-02-17T12:47:45.056995+0000","last_active":"2024-02-17T12:48:52.463459+0000","last_peered":"2024-02-17T12:48:52.463459+0000","last_clean":"2024-02-17T12:48:52.463459+0000","last_became_active":"2024-02-17T12:47:45.056021+0000","last_became_peered":"2024-02-17T12:47:45.056021+0000","last_unstale":"2024-02-17T12:48:52.463459+0000","last_undegraded":"2024-02-17T12:48:52.463459+0000","last_fullsized":"2024-02-17T12:48:52.463459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:40:41.444513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.722229+0000","last_change":"2024-02-17T12:47:46.065859+0000","last_active":"2024-02-17T12:49:20.722229+0000","last_peered":"2024-02-17T12:49:20.722229+0000","last_clean":"2024-02-17T12:49:20.722229+0000","last_became_active":"2024-02-17T12:47:46.065469+0000","last_became_peered":"2024-02-17T12:47:46.065469+0000","last_unstale":"2024-02-17T12:49:20.722229+0000","last_undegraded":"2024-02-17T12:49:20.722229+0000","last_fullsized":"2024-02-17T12:49:20.722229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:51:25.105361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.165363+0000","last_change":"2024-02-17T12:47:45.066524+0000","last_active":"2024-02-17T12:48:52.165363+0000","last_peered":"2024-02-17T12:48:52.165363+0000","last_clean":"2024-02-17T12:48:52.165363+0000","last_became_active":"2024-02-17T12:47:45.066218+0000","last_became_peered":"2024-02-17T12:47:45.066218+0000","last_unstale":"2024-02-17T12:48:52.165363+0000","last_undegraded":"2024-02-17T12:48:52.165363+0000","last_fullsized":"2024-02-17T12:48:52.165363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:56:47.244262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764154+0000","last_change":"2024-02-17T12:47:46.065779+0000","last_active":"2024-02-17T12:49:20.764154+0000","last_peered":"2024-02-17T12:49:20.764154+0000","last_clean":"2024-02-17T12:49:20.764154+0000","last_became_active":"2024-02-17T12:47:46.065459+0000","last_became_peered":"2024-02-17T12:47:46.065459+0000","last_unstale":"2024-02-17T12:49:20.764154+0000","last_undegraded":"2024-02-17T12:49:20.764154+0000","last_fullsized":"2024-02-17T12:49:20.764154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:08.944487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.163588+0000","last_change":"2024-02-17T12:47:45.059639+0000","last_active":"2024-02-17T12:48:52.163588+0000","last_peered":"2024-02-17T12:48:52.163588+0000","last_clean":"2024-02-17T12:48:52.163588+0000","last_became_active":"2024-02-17T12:47:45.059085+0000","last_became_peered":"2024-02-17T12:47:45.059085+0000","last_unstale":"2024-02-17T12:48:52.163588+0000","last_undegraded":"2024-02-17T12:48:52.163588+0000","last_fullsized":"2024-02-17T12:48:52.163588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:48:55.847405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.756470+0000","last_change":"2024-02-17T12:47:46.079623+0000","last_active":"2024-02-17T12:49:20.756470+0000","last_peered":"2024-02-17T12:49:20.756470+0000","last_clean":"2024-02-17T12:49:20.756470+0000","last_became_active":"2024-02-17T12:47:46.079285+0000","last_became_peered":"2024-02-17T12:47:46.079285+0000","last_unstale":"2024-02-17T12:49:20.756470+0000","last_undegraded":"2024-02-17T12:49:20.756470+0000","last_fullsized":"2024-02-17T12:49:20.756470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:45:14.441218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.181532+0000","last_change":"2024-02-17T12:48:52.181057+0000","last_active":"2024-02-17T12:48:52.181532+0000","last_peered":"2024-02-17T12:48:52.181532+0000","last_clean":"2024-02-17T12:48:52.181532+0000","last_became_active":"2024-02-17T12:48:52.165968+0000","last_became_peered":"2024-02-17T12:48:52.165968+0000","last_unstale":"2024-02-17T12:48:52.181532+0000","last_undegraded":"2024-02-17T12:48:52.181532+0000","last_fullsized":"2024-02-17T12:48:52.181532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:14:06.153057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766320+0000","last_change":"2024-02-17T12:47:46.074429+0000","last_active":"2024-02-17T12:49:20.766320+0000","last_peered":"2024-02-17T12:49:20.766320+0000","last_clean":"2024-02-17T12:49:20.766320+0000","last_became_active":"2024-02-17T12:47:46.074123+0000","last_became_peered":"2024-02-17T12:47:46.074123+0000","last_unstale":"2024-02-17T12:49:20.766320+0000","last_undegraded":"2024-02-17T12:49:20.766320+0000","last_fullsized":"2024-02-17T12:49:20.766320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:50:17.633831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":132,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:35.869454+0000","last_change":"2024-02-17T12:47:45.061362+0000","last_active":"2024-02-17T12:49:35.869454+0000","last_peered":"2024-02-17T12:49:35.869454+0000","last_clean":"2024-02-17T12:49:35.869454+0000","last_became_active":"2024-02-17T12:47:45.061037+0000","last_became_peered":"2024-02-17T12:47:45.061037+0000","last_unstale":"2024-02-17T12:49:35.869454+0000","last_undegraded":"2024-02-17T12:49:35.869454+0000","last_fullsized":"2024-02-17T12:49:35.869454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:01.987157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764578+0000","last_change":"2024-02-17T12:47:46.073862+0000","last_active":"2024-02-17T12:49:20.764578+0000","last_peered":"2024-02-17T12:49:20.764578+0000","last_clean":"2024-02-17T12:49:20.764578+0000","last_became_active":"2024-02-17T12:47:46.073543+0000","last_became_peered":"2024-02-17T12:47:46.073543+0000","last_unstale":"2024-02-17T12:49:20.764578+0000","last_undegraded":"2024-02-17T12:49:20.764578+0000","last_fullsized":"2024-02-17T12:49:20.764578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:57:57.567726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.160633+0000","last_change":"2024-02-17T12:47:45.060616+0000","last_active":"2024-02-17T12:48:52.160633+0000","last_peered":"2024-02-17T12:48:52.160633+0000","last_clean":"2024-02-17T12:48:52.160633+0000","last_became_active":"2024-02-17T12:47:45.060296+0000","last_became_peered":"2024-02-17T12:47:45.060296+0000","last_unstale":"2024-02-17T12:48:52.160633+0000","last_undegraded":"2024-02-17T12:48:52.160633+0000","last_fullsized":"2024-02-17T12:48:52.160633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:22:44.791363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758731+0000","last_change":"2024-02-17T12:47:46.072738+0000","last_active":"2024-02-17T12:49:20.758731+0000","last_peered":"2024-02-17T12:49:20.758731+0000","last_clean":"2024-02-17T12:49:20.758731+0000","last_became_active":"2024-02-17T12:47:46.072421+0000","last_became_peered":"2024-02-17T12:47:46.072421+0000","last_unstale":"2024-02-17T12:49:20.758731+0000","last_undegraded":"2024-02-17T12:49:20.758731+0000","last_fullsized":"2024-02-17T12:49:20.758731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:50:23.501787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:48:52.470091+0000","last_change":"2024-02-17T12:48:52.470091+0000","last_active":"2024-02-17T12:48:52.470091+0000","last_peered":"2024-02-17T12:48:52.470091+0000","last_clean":"2024-02-17T12:48:52.470091+0000","last_became_active":"2024-02-17T12:48:52.466153+0000","last_became_peered":"2024-02-17T12:48:52.466153+0000","last_unstale":"2024-02-17T12:48:52.470091+0000","last_undegraded":"2024-02-17T12:48:52.470091+0000","last_fullsized":"2024-02-17T12:48:52.470091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:44.032441+0000","last_clean_scrub_stamp":"2024-02-17T12:47:44.032441+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:03:45.027983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.764385+0000","last_change":"2024-02-17T12:47:46.066072+0000","last_active":"2024-02-17T12:49:20.764385+0000","last_peered":"2024-02-17T12:49:20.764385+0000","last_clean":"2024-02-17T12:49:20.764385+0000","last_became_active":"2024-02-17T12:47:46.065755+0000","last_became_peered":"2024-02-17T12:47:46.065755+0000","last_unstale":"2024-02-17T12:49:20.764385+0000","last_undegraded":"2024-02-17T12:49:20.764385+0000","last_fullsized":"2024-02-17T12:49:20.764385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:09:29.534097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763980+0000","last_change":"2024-02-17T12:47:46.067268+0000","last_active":"2024-02-17T12:49:20.763980+0000","last_peered":"2024-02-17T12:49:20.763980+0000","last_clean":"2024-02-17T12:49:20.763980+0000","last_became_active":"2024-02-17T12:47:46.066918+0000","last_became_peered":"2024-02-17T12:47:46.066918+0000","last_unstale":"2024-02-17T12:49:20.763980+0000","last_undegraded":"2024-02-17T12:49:20.763980+0000","last_fullsized":"2024-02-17T12:49:20.763980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:11:09.844954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.757364+0000","last_change":"2024-02-17T12:47:46.068143+0000","last_active":"2024-02-17T12:49:20.757364+0000","last_peered":"2024-02-17T12:49:20.757364+0000","last_clean":"2024-02-17T12:49:20.757364+0000","last_became_active":"2024-02-17T12:47:46.065112+0000","last_became_peered":"2024-02-17T12:47:46.065112+0000","last_unstale":"2024-02-17T12:49:20.757364+0000","last_undegraded":"2024-02-17T12:49:20.757364+0000","last_fullsized":"2024-02-17T12:49:20.757364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:43:04.982009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.748955+0000","last_change":"2024-02-17T12:47:46.076139+0000","last_active":"2024-02-17T12:49:20.748955+0000","last_peered":"2024-02-17T12:49:20.748955+0000","last_clean":"2024-02-17T12:49:20.748955+0000","last_became_active":"2024-02-17T12:47:46.075826+0000","last_became_peered":"2024-02-17T12:47:46.075826+0000","last_unstale":"2024-02-17T12:49:20.748955+0000","last_undegraded":"2024-02-17T12:49:20.748955+0000","last_fullsized":"2024-02-17T12:49:20.748955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:39:08.460716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766396+0000","last_change":"2024-02-17T12:47:46.063792+0000","last_active":"2024-02-17T12:49:20.766396+0000","last_peered":"2024-02-17T12:49:20.766396+0000","last_clean":"2024-02-17T12:49:20.766396+0000","last_became_active":"2024-02-17T12:47:46.063407+0000","last_became_peered":"2024-02-17T12:47:46.063407+0000","last_unstale":"2024-02-17T12:49:20.766396+0000","last_undegraded":"2024-02-17T12:49:20.766396+0000","last_fullsized":"2024-02-17T12:49:20.766396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:26:26.973435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.768046+0000","last_change":"2024-02-17T12:47:46.066202+0000","last_active":"2024-02-17T12:49:20.768046+0000","last_peered":"2024-02-17T12:49:20.768046+0000","last_clean":"2024-02-17T12:49:20.768046+0000","last_became_active":"2024-02-17T12:47:46.065820+0000","last_became_peered":"2024-02-17T12:47:46.065820+0000","last_unstale":"2024-02-17T12:49:20.768046+0000","last_undegraded":"2024-02-17T12:49:20.768046+0000","last_fullsized":"2024-02-17T12:49:20.768046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:36:25.201697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.759899+0000","last_change":"2024-02-17T12:48:52.166996+0000","last_active":"2024-02-17T12:49:20.759899+0000","last_peered":"2024-02-17T12:49:20.759899+0000","last_clean":"2024-02-17T12:49:20.759899+0000","last_became_active":"2024-02-17T12:48:52.166589+0000","last_became_peered":"2024-02-17T12:48:52.166589+0000","last_unstale":"2024-02-17T12:49:20.759899+0000","last_undegraded":"2024-02-17T12:49:20.759899+0000","last_fullsized":"2024-02-17T12:49:20.759899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:16:30.392183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.763854+0000","last_change":"2024-02-17T12:47:46.070811+0000","last_active":"2024-02-17T12:49:20.763854+0000","last_peered":"2024-02-17T12:49:20.763854+0000","last_clean":"2024-02-17T12:49:20.763854+0000","last_became_active":"2024-02-17T12:47:46.070462+0000","last_became_peered":"2024-02-17T12:47:46.070462+0000","last_unstale":"2024-02-17T12:49:20.763854+0000","last_undegraded":"2024-02-17T12:49:20.763854+0000","last_fullsized":"2024-02-17T12:49:20.763854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:27:03.671869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758608+0000","last_change":"2024-02-17T12:47:46.077692+0000","last_active":"2024-02-17T12:49:20.758608+0000","last_peered":"2024-02-17T12:49:20.758608+0000","last_clean":"2024-02-17T12:49:20.758608+0000","last_became_active":"2024-02-17T12:47:46.077363+0000","last_became_peered":"2024-02-17T12:47:46.077363+0000","last_unstale":"2024-02-17T12:49:20.758608+0000","last_undegraded":"2024-02-17T12:49:20.758608+0000","last_fullsized":"2024-02-17T12:49:20.758608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:02:49.068782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.758794+0000","last_change":"2024-02-17T12:47:46.073507+0000","last_active":"2024-02-17T12:49:20.758794+0000","last_peered":"2024-02-17T12:49:20.758794+0000","last_clean":"2024-02-17T12:49:20.758794+0000","last_became_active":"2024-02-17T12:47:46.073107+0000","last_became_peered":"2024-02-17T12:47:46.073107+0000","last_unstale":"2024-02-17T12:49:20.758794+0000","last_undegraded":"2024-02-17T12:49:20.758794+0000","last_fullsized":"2024-02-17T12:49:20.758794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T00:32:48.799300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762538+0000","last_change":"2024-02-17T12:47:46.073384+0000","last_active":"2024-02-17T12:49:20.762538+0000","last_peered":"2024-02-17T12:49:20.762538+0000","last_clean":"2024-02-17T12:49:20.762538+0000","last_became_active":"2024-02-17T12:47:46.073017+0000","last_became_peered":"2024-02-17T12:47:46.073017+0000","last_unstale":"2024-02-17T12:49:20.762538+0000","last_undegraded":"2024-02-17T12:49:20.762538+0000","last_fullsized":"2024-02-17T12:49:20.762538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:43:50.091667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766804+0000","last_change":"2024-02-17T12:47:46.073758+0000","last_active":"2024-02-17T12:49:20.766804+0000","last_peered":"2024-02-17T12:49:20.766804+0000","last_clean":"2024-02-17T12:49:20.766804+0000","last_became_active":"2024-02-17T12:47:46.072781+0000","last_became_peered":"2024-02-17T12:47:46.072781+0000","last_unstale":"2024-02-17T12:49:20.766804+0000","last_undegraded":"2024-02-17T12:49:20.766804+0000","last_fullsized":"2024-02-17T12:49:20.766804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:10:24.811100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.766956+0000","last_change":"2024-02-17T12:47:46.072025+0000","last_active":"2024-02-17T12:49:20.766956+0000","last_peered":"2024-02-17T12:49:20.766956+0000","last_clean":"2024-02-17T12:49:20.766956+0000","last_became_active":"2024-02-17T12:47:46.071662+0000","last_became_peered":"2024-02-17T12:47:46.071662+0000","last_unstale":"2024-02-17T12:49:20.766956+0000","last_undegraded":"2024-02-17T12:49:20.766956+0000","last_fullsized":"2024-02-17T12:49:20.766956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:04:07.188552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-17T12:49:20.762073+0000","last_change":"2024-02-17T12:48:52.166041+0000","last_active":"2024-02-17T12:49:20.762073+0000","last_peered":"2024-02-17T12:49:20.762073+0000","last_clean":"2024-02-17T12:49:20.762073+0000","last_became_active":"2024-02-17T12:48:52.165576+0000","last_became_peered":"2024-02-17T12:48:52.165576+0000","last_unstale":"2024-02-17T12:49:20.762073+0000","last_undegraded":"2024-02-17T12:49:20.762073+0000","last_fullsized":"2024-02-17T12:49:20.762073+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T12:47:45.039138+0000","last_clean_scrub_stamp":"2024-02-17T12:47:45.039138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:26.810385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67437,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1126,"num_read_kb":638,"num_write":249,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1013,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351012,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":462062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.40699999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.55600000000000005}]},{"osd":1,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.56399999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82899999999999996,"5min":0.874,"15min":0.874},"last":0.48599999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.504},{"interface":"front","average":{"1min":0.5,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69999999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.54900000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90700000000000003,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.622},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.51600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83699999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80000000000000004,"5min":1.002,"15min":1.002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93100000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.53300000000000003}]},{"osd":6,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84599999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.67100000000000004,"15min":0.67800000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":1.121,"15min":1.121},"last":0.67400000000000004}]},{"osd":7,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77600000000000002,"15min":0.77300000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.748,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91800000000000004,"5min":1.103,"15min":1.103},"last":0.748}]},{"osd":8,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.79000000000000004}]},{"osd":9,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.157,"15min":1.157},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.78800000000000003,"15min":0.78300000000000003},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.054,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77700000000000002}]},{"osd":10,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79400000000000004,"15min":0.78800000000000003},"min":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98699999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76600000000000001}]}]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1113574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":13.85,"5min":13.85,"15min":13.85},"last":0.56999999999999995}]},{"osd":1,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.48199999999999998,"15min":0.503},"min":{"1min":0.31900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.629,"5min":0.751,"15min":0.751},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.502,"15min":0.52300000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65600000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.43099999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68100000000000005,"15min":0.69099999999999995},"min":{"1min":0.49099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76600000000000001,"5min":1.002,"15min":1.002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.52800000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83499999999999996,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.65600000000000003}]},{"osd":3,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.57199999999999995,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.64400000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.499},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.64200000000000002,"15min":0.66800000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.74299999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.51400000000000001,"15min":0.53700000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64500000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.58099999999999996,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78000000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.52400000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71499999999999997,"15min":0.73999999999999999},"min":{"1min":0.47999999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.88,"5min":1.022,"15min":1.022},"last":0.623},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67500000000000004,"15min":0.69699999999999995},"min":{"1min":0.47299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83599999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65800000000000003,"15min":0.68999999999999995},"min":{"1min":0.438,"5min":0.437,"15min":0.437},"max":{"1min":0.748,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.73399999999999999,"15min":0.76000000000000001},"min":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85299999999999998,"5min":1.085,"15min":1.085},"last":0.47699999999999998}]},{"osd":7,"last update":"Sat Feb 17 12:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.54900000000000004}]},{"osd":8,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.74099999999999999,"15min":0.77400000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.81000000000000005,"5min":1.107,"15min":1.107},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.80300000000000005,"15min":0.83299999999999996},"min":{"1min":0.61099999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92000000000000004,"5min":1.119,"15min":1.119},"last":0.753}]},{"osd":9,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.80400000000000005,"15min":0.84799999999999998},"min":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94399999999999995,"5min":1.151,"15min":1.151},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.78000000000000003,"15min":0.82199999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.91300000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.59599999999999997}]},{"osd":11,"last update":"Sat Feb 17 12:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77700000000000002,"15min":0.77900000000000003},"min":{"1min":0.63800000000000001,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.93899999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75,"15min":0.752},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68799999999999994}]}]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40460,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704692,"statfs":{"total":95995035648,"available":95953604608,"internally_reserved":0,"allocated":987136,"data_stored":461992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.436,"15min":0.45400000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.60199999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.35699999999999998}]},{"osd":1,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63100000000000001,"15min":0.65300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63300000000000001,"15min":0.64700000000000002},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.0409999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.54200000000000004}]},{"osd":3,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.316,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.879,"5min":1.488,"15min":1.488},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.58299999999999996,"15min":0.61199999999999999},"min":{"1min":0.29999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83999999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.375}]},{"osd":4,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2030000000000001,"5min":1.218,"15min":1.218},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65100000000000002,"15min":0.66700000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95999999999999996,"5min":1.159,"15min":1.159},"last":0.438}]},{"osd":5,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.68200000000000005,"15min":0.70599999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0029999999999999,"5min":1.323,"15min":1.323},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.73999999999999999,"15min":0.76800000000000002},"min":{"1min":0.46100000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92100000000000004,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.66000000000000003}]},{"osd":6,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67300000000000004,"15min":0.70699999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.79700000000000004,"5min":1.268,"15min":1.268},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.60999999999999999,"15min":0.63400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85799999999999998,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.45500000000000002}]},{"osd":7,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.73899999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0760000000000001,"5min":1.27,"15min":1.27},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77800000000000002,"15min":0.80800000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.236,"5min":1.351,"15min":1.351},"last":0.626}]},{"osd":8,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73599999999999999,"15min":0.754},"min":{"1min":0.498,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.76600000000000001,"15min":0.79400000000000004},"min":{"1min":0.52100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1160000000000001,"5min":1.363,"15min":1.363},"last":0.69199999999999995}]},{"osd":10,"last update":"Sat Feb 17 12:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.80700000000000005,"5min":1.228,"15min":1.228},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78300000000000003,"15min":0.81000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.58199999999999996}]}]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":462370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.35899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.52400000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.91800000000000004,"15min":1.054},"min":{"1min":0.441,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89000000000000001,"5min":10.955,"15min":10.955},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.434,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.84999999999999998}]},{"osd":2,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.44900000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.443}]},{"osd":3,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.36499999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.71899999999999997}]},{"osd":4,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.93300000000000005,"15min":1.0680000000000001},"min":{"1min":0.46100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":10.972,"15min":10.972},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.92800000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.372,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.34899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.60099999999999998}]},{"osd":6,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.878},"min":{"1min":0.53200000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97499999999999998,"5min":4.9269999999999996,"15min":4.9269999999999996},"last":0.81599999999999995}]},{"osd":7,"last update":"Sat Feb 17 12:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.49399999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.628},{"interface":"front","average":{"1min":0.755,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":1.022,"15min":1.022},"last":0.68500000000000005}]},{"osd":9,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.72999999999999998,"15min":0.69799999999999995},"min":{"1min":0.64500000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.77700000000000002,"15min":0.75},"min":{"1min":0.65000000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.98799999999999999}]},{"osd":10,"last update":"Sat Feb 17 12:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.753,"15min":0.74399999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.748},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.1339999999999999,"15min":1.268},"min":{"1min":0.54700000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.083,"5min":10.987,"15min":10.987},"last":0.89900000000000002}]},{"osd":11,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.875},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.91600000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1055561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42899999999999999,"15min":0.41299999999999998},"min":{"1min":0.33700000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57699999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.46100000000000002,"15min":0.439},"min":{"1min":0.38200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68000000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.318}]},{"osd":2,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.48899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.75800000000000001,"15min":0.63300000000000001},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77900000000000003,"5min":15.003,"15min":15.003},"last":0.54400000000000004}]},{"osd":3,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60199999999999998,"15min":0.61799999999999999},"min":{"1min":0.45600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68600000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.60299999999999998,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76100000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56000000000000005}]},{"osd":4,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.36599999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71999999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.39900000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82399999999999995,"5min":0.872,"15min":0.872},"last":0.35899999999999999}]},{"osd":5,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.81100000000000005,"15min":0.68899999999999995},"min":{"1min":0.44600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.92700000000000005,"5min":15.028,"15min":15.028},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.54500000000000004,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86199999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59099999999999997}]},{"osd":6,"last update":"Sat Feb 17 12:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66700000000000004,"15min":0.68200000000000005},"min":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.80600000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.499},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.93300000000000005,"15min":0.77100000000000002},"min":{"1min":0.51700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88400000000000001,"5min":14.579000000000001,"15min":14.579000000000001},"last":0.45300000000000001}]},{"osd":8,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.60999999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96199999999999997,"5min":1.02,"15min":1.02},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75700000000000001,"15min":0.73799999999999999},"min":{"1min":0.65500000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.77400000000000002}]},{"osd":9,"last update":"Sat Feb 17 12:49:14 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":0.84699999999999998,"15min":0.79900000000000004},"min":{"1min":0.48999999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.403,"5min":10.403,"15min":10.403},"last":0.51900000000000002},{"interface":"front","average":{"1min":1.2410000000000001,"5min":0.98999999999999999,"15min":0.84199999999999997},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":10.471,"5min":14.598000000000001,"15min":14.598000000000001},"last":0.65800000000000003}]},{"osd":10,"last update":"Sat Feb 17 12:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.68000000000000005,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91400000000000003,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.59299999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82799999999999996,"5min":1.103,"15min":1.103},"last":0.64200000000000002}]},{"osd":11,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.876}]}]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":461709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.38100000000000001,"15min":0.371},"min":{"1min":0.26100000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.49399999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.28100000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53600000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.50800000000000001}]},{"osd":1,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.40400000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.62,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":5.0179999999999998,"5min":5.0179999999999998,"15min":5.0179999999999998},"last":0.91600000000000004}]},{"osd":2,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.65300000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.63200000000000001,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":5.0469999999999997,"5min":5.0469999999999997,"15min":5.0469999999999997},"last":0.71499999999999997}]},{"osd":3,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.441,"15min":0.443},"min":{"1min":0.29099999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.52900000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.624,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.81699999999999995}]},{"osd":4,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73699999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.79300000000000004}]},{"osd":5,"last update":"Sat Feb 17 12:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74199999999999999,"5min":1.284,"15min":1.284},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75900000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.83699999999999997}]},{"osd":7,"last update":"Sat Feb 17 12:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72099999999999997,"15min":0.67000000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92200000000000004,"5min":5.069,"15min":5.069},"last":0.871},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.64100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.76500000000000001}]},{"osd":8,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.73399999999999999,"15min":0.70799999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":5.0940000000000003,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90800000000000003,"5min":1.252,"15min":1.252},"last":0.94399999999999995}]},{"osd":9,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.65500000000000003,"15min":0.68500000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90600000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.67700000000000005,"15min":0.70699999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85499999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.96899999999999997}]},{"osd":10,"last update":"Sat Feb 17 12:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.74199999999999999,"15min":0.76000000000000001},"min":{"1min":0.56599999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77600000000000002,"15min":0.80700000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.98899999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.95499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":522573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.40500000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.67800000000000005,"5min":0.73999999999999999,"15min":0.78100000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.69499999999999995,"15min":0.55500000000000005},"min":{"1min":0.42699999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79600000000000004,"5min":14.91,"15min":14.91},"last":0.67600000000000005}]},{"osd":2,"last update":"Sat Feb 17 12:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.59899999999999998,"15min":0.52700000000000002},"min":{"1min":0.51000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.434},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.60499999999999998,"15min":0.53000000000000003},"min":{"1min":0.61599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61899999999999999}]},{"osd":3,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.48599999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78500000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.432,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73599999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.74199999999999999}]},{"osd":4,"last update":"Sat Feb 17 12:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.44600000000000001,"15min":0.39900000000000002},"min":{"1min":0.34699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.77600000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.46700000000000003,"15min":0.42899999999999999},"min":{"1min":0.35699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.80700000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46000000000000002}]},{"osd":5,"last update":"Sat Feb 17 12:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.65300000000000002,"15min":0.59199999999999997},"min":{"1min":0.63200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98799999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.60699999999999998},"min":{"1min":0.50900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82399999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72299999999999998}]},{"osd":6,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61599999999999999,"15min":0.59499999999999997},"min":{"1min":0.59999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86699999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.53700000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.77200000000000002}]},{"osd":7,"last update":"Sat Feb 17 12:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.60099999999999998,"15min":0.55900000000000005},"min":{"1min":0.41099999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.66200000000000003,"15min":0.60899999999999999},"min":{"1min":0.53200000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95899999999999996,"5min":1.079,"15min":1.079},"last":0.65300000000000002}]},{"osd":8,"last update":"Sat Feb 17 12:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.67400000000000004,"15min":0.63400000000000001},"min":{"1min":0.63800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.58999999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59499999999999997}]},{"osd":9,"last update":"Sat Feb 17 12:49:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72499999999999998,"15min":0.70699999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.747,"15min":0.71899999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.85899999999999999}]},{"osd":10,"last update":"Sat Feb 17 12:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.57899999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97799999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.78100000000000003,"15min":0.745},"min":{"1min":0.52800000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83199999999999996}]},{"osd":11,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.81799999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":463048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53900000000000003,"15min":0.49099999999999999},"min":{"1min":0.36899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.503},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55300000000000005,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64400000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64700000000000002}]},{"osd":2,"last update":"Sat Feb 17 12:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.437,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.748,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.73699999999999999},"min":{"1min":0.42299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81999999999999995,"5min":8.3919999999999995,"15min":8.3919999999999995},"last":0.72499999999999998}]},{"osd":3,"last update":"Sat Feb 17 12:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Sat Feb 17 12:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75600000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.58899999999999997,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":0.872,"15min":0.872},"last":0.66700000000000004}]},{"osd":5,"last update":"Sat Feb 17 12:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.42099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68999999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.623},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.48099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.748,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68200000000000005}]},{"osd":6,"last update":"Sat Feb 17 12:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.46600000000000003,"15min":0.45800000000000002},"min":{"1min":0.32500000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58699999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.49399999999999999,"15min":0.51000000000000001},"min":{"1min":0.30099999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65600000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55700000000000005}]},{"osd":7,"last update":"Sat Feb 17 12:47:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85299999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.443},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64600000000000002,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003}]},{"osd":8,"last update":"Sat Feb 17 12:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.65300000000000002,"15min":0.66500000000000004},"min":{"1min":0.45000000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79500000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63600000000000001}]},{"osd":9,"last update":"Sat Feb 17 12:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.58499999999999996,"15min":0.622},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61799999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.60299999999999998,"15min":0.64900000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.59799999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.70299999999999996}]},{"osd":10,"last update":"Sat Feb 17 12:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77700000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.746},{"interface":"front","average":{"1min":0.621,"5min":0.71999999999999997,"15min":0.73599999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.72199999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.75900000000000001}]},{"osd":11,"last update":"Sat Feb 17 12:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.60799999999999998,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.84299999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.57599999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.77900000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":522171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.50700000000000001,"15min":0.46899999999999997},"min":{"1min":0.40200000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.79600000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.48699999999999999,"15min":0.47099999999999997},"min":{"1min":0.34599999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68799999999999994,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57899999999999996}]},{"osd":1,"last update":"Sat Feb 17 12:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.56399999999999995,"15min":0.51400000000000001},"min":{"1min":0.45200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.83299999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.57599999999999996,"15min":0.51800000000000002},"min":{"1min":0.41199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.84799999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.78100000000000003}]},{"osd":3,"last update":"Sat Feb 17 12:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56000000000000005,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.77200000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66300000000000003}]},{"osd":4,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.76400000000000001}]},{"osd":5,"last update":"Sat Feb 17 12:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.48899999999999999,"15min":0.44900000000000001},"min":{"1min":0.35799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.92100000000000004,"5min":1.325,"15min":1.325},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.505,"15min":0.49299999999999999},"min":{"1min":0.42099999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83799999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Sat Feb 17 12:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.47299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.59199999999999997,"15min":0.60099999999999998},"min":{"1min":0.39100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80600000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63100000000000001}]},{"osd":7,"last update":"Sat Feb 17 12:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.60999999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.85699999999999998,"5min":0.998,"15min":0.998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60899999999999999,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95899999999999996,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.83799999999999997}]},{"osd":9,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67000000000000004,"15min":0.68300000000000005},"min":{"1min":0.502,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86399999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.51400000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93600000000000005,"5min":1.284,"15min":1.284},"last":0.747}]},{"osd":10,"last update":"Sat Feb 17 12:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.55200000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92500000000000004,"5min":1.347,"15min":1.347},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.72899999999999998,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97199999999999998,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.80300000000000005}]},{"osd":11,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.81200000000000006}]}]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40460,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704692,"statfs":{"total":95995035648,"available":95953604608,"internally_reserved":0,"allocated":987136,"data_stored":461534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.32300000000000001},"min":{"1min":0.26000000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59799999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.33700000000000002},"min":{"1min":0.26300000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.41499999999999998}]},{"osd":1,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77100000000000002,"5min":0.879,"15min":0.879},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.44800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.57599999999999996}]},{"osd":2,"last update":"Sat Feb 17 12:48:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.752,"5min":0.88,"15min":0.88},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77900000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.626}]},{"osd":4,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.39600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.44600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.74299999999999999,"5min":0.876,"15min":0.876},"last":0.83199999999999996}]},{"osd":5,"last update":"Sat Feb 17 12:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.50600000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72999999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81399999999999995,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.86399999999999999}]},{"osd":6,"last update":"Sat Feb 17 12:48:53 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.504,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73799999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.495,"15min":0.505},"min":{"1min":0.35099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.49399999999999999}]},{"osd":7,"last update":"Sat Feb 17 12:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.625,"15min":0.621},"min":{"1min":0.47099999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78600000000000003,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76100000000000001}]},{"osd":8,"last update":"Sat Feb 17 12:47:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91600000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.71299999999999997,"15min":0.72599999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0069999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.56899999999999995}]},{"osd":9,"last update":"Sat Feb 17 12:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.64300000000000002,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.81799999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.67200000000000004,"15min":0.70599999999999996},"min":{"1min":0.44900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85899999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73599999999999999}]},{"osd":10,"last update":"Sat Feb 17 12:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.75,"15min":0.78300000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84199999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.76700000000000002},"min":{"1min":0.60499999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.77900000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.879}]},{"osd":11,"last update":"Sat Feb 17 12:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.94199999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":463256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57899999999999996}]},{"osd":1,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.36299999999999999},"min":{"1min":0.28499999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.60899999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.436},{"interface":"front","average":{"1min":0.437,"5min":0.41599999999999998,"15min":0.378},"min":{"1min":0.35099999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.58799999999999997,"5min":0.751,"15min":0.751},"last":0.34200000000000003}]},{"osd":2,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.80000000000000004,"15min":0.64000000000000001},"min":{"1min":0.42699999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85899999999999999,"5min":18.73,"15min":18.73},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57299999999999995}]},{"osd":3,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.47799999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85199999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.58799999999999997}]},{"osd":5,"last update":"Sat Feb 17 12:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.51000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64000000000000001,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.55900000000000005}]},{"osd":6,"last update":"Sat Feb 17 12:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.57499999999999996},"min":{"1min":0.47899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79800000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51600000000000001}]},{"osd":7,"last update":"Sat Feb 17 12:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.54900000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83699999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.41199999999999998}]},{"osd":8,"last update":"Sat Feb 17 12:48:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.83699999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.65800000000000003}]},{"osd":9,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.62},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89200000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.74399999999999999}]},{"osd":10,"last update":"Sat Feb 17 12:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.755,"15min":0.75800000000000001},"min":{"1min":0.499,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92600000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.49299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.77200000000000002}]},{"osd":11,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.79400000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1054707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.371,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.82599999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.625,"5min":1.02,"15min":1.02},"last":0.59299999999999997}]},{"osd":1,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.53000000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60099999999999998,"15min":0.53900000000000003},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.81699999999999995}]},{"osd":2,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.435,"15min":0.39300000000000002},"min":{"1min":0.34300000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.45000000000000001,"15min":0.38700000000000001},"min":{"1min":0.34399999999999997,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.71199999999999997}]},{"osd":3,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.54000000000000004},"min":{"1min":0.48299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84299999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.64800000000000002}]},{"osd":4,"last update":"Sat Feb 17 12:49:27 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61799999999999999,"15min":0.55900000000000005},"min":{"1min":0.41899999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93999999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61099999999999999,"15min":0.54000000000000004},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.78700000000000003}]},{"osd":6,"last update":"Sat Feb 17 12:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.621,"15min":0.58799999999999997},"min":{"1min":0.56299999999999994,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98799999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.628,"15min":0.61099999999999999},"min":{"1min":0.53400000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79900000000000004}]},{"osd":7,"last update":"Sat Feb 17 12:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70699999999999996,"15min":0.68400000000000005},"min":{"1min":0.47799999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.86199999999999999}]},{"osd":8,"last update":"Sat Feb 17 12:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.629,"15min":0.625},"min":{"1min":0.51200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.46600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.82999999999999996}]},{"osd":9,"last update":"Sat Feb 17 12:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68899999999999995,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.878},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.58799999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77900000000000003}]},{"osd":10,"last update":"Sat Feb 17 12:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77100000000000002,"15min":0.746},"min":{"1min":0.57999999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.1100000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72899999999999998,"15min":0.69199999999999995},"min":{"1min":0.55700000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.89100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T12:49:42.427 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:49:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.572+0000 7f0ad7edd640 1 -- 172.21.15.18:0/1602781073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ad01016c0 msgr2=0x7f0ad0101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.572+0000 7f0ad7edd640 1 --2- 172.21.15.18:0/1602781073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ad01016c0 0x7f0ad0101ae0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009900 tx=0x7f0acc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.572+0000 7f0ad7edd640 1 -- 172.21.15.18:0/1602781073 shutdown_connections 2024-02-17T12:49:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.572+0000 7f0ad7edd640 1 --2- 172.21.15.18:0/1602781073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ad0100cd0 0x7f0ad0108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.572+0000 7f0ad7edd640 1 --2- 172.21.15.18:0/1602781073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ad0100310 0x7f0ad0100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.572+0000 7f0ad7edd640 1 --2- 172.21.15.18:0/1602781073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ad01016c0 0x7f0ad0101ae0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.572+0000 7f0ad7edd640 1 -- 172.21.15.18:0/1602781073 >> 172.21.15.18:0/1602781073 conn(0x7f0ad00fada0 msgr2=0x7f0ad00fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.572+0000 7f0ad7edd640 1 -- 172.21.15.18:0/1602781073 shutdown_connections 2024-02-17T12:49:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.572+0000 7f0ad7edd640 1 -- 172.21.15.18:0/1602781073 wait complete. 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad7edd640 1 Processor -- start 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad7edd640 1 -- start start 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad7edd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ad0100310 0x7f0ad01972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad7edd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ad0100cd0 0x7f0ad0197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad7edd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ad01016c0 0x7f0ad019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad7edd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad019ed70 con 0x7f0ad01016c0 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad7edd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad019eee0 con 0x7f0ad0100cd0 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad7edd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad019f0a0 con 0x7f0ad0100310 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad647a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ad01016c0 0x7f0ad019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad647a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ad01016c0 0x7f0ad019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34516/0 (socket says 172.21.15.18:34516) 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad647a640 1 -- 172.21.15.18:0/3780875885 learned_addr learned my addr 172.21.15.18:0/3780875885 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad5c79640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ad0100310 0x7f0ad01972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.573+0000 7f0ad5478640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ad0100cd0 0x7f0ad0197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.574+0000 7f0ad647a640 1 -- 172.21.15.18:0/3780875885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ad0100310 msgr2=0x7f0ad01972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.574+0000 7f0ad647a640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ad0100310 0x7f0ad01972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.574+0000 7f0ad647a640 1 -- 172.21.15.18:0/3780875885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ad0100cd0 msgr2=0x7f0ad0197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.574+0000 7f0ad647a640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ad0100cd0 0x7f0ad0197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.574+0000 7f0ad647a640 1 -- 172.21.15.18:0/3780875885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc009560 con 0x7f0ad01016c0 2024-02-17T12:49:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.574+0000 7f0ad647a640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ad01016c0 0x7f0ad019e830 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800eaa0 tx=0x7f0ac800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.574+0000 7f0ac6ffd640 1 -- 172.21.15.18:0/3780875885 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac800cdb0 con 0x7f0ad01016c0 2024-02-17T12:49:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.574+0000 7f0ad7edd640 1 -- 172.21.15.18:0/3780875885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad019f380 con 0x7f0ad01016c0 2024-02-17T12:49:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.574+0000 7f0ac6ffd640 1 -- 172.21.15.18:0/3780875885 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ac8005020 con 0x7f0ad01016c0 2024-02-17T12:49:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.574+0000 7f0ac6ffd640 1 -- 172.21.15.18:0/3780875885 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac80116b0 con 0x7f0ad01016c0 2024-02-17T12:49:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.574+0000 7f0ad7edd640 1 -- 172.21.15.18:0/3780875885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad019f840 con 0x7f0ad01016c0 2024-02-17T12:49:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.575+0000 7f0ad7edd640 1 -- 172.21.15.18:0/3780875885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad010bb00 con 0x7f0ad01016c0 2024-02-17T12:49:42.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.576+0000 7f0ac6ffd640 1 -- 172.21.15.18:0/3780875885 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0ac8005520 con 0x7f0ad01016c0 2024-02-17T12:49:42.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.577+0000 7f0ac6ffd640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0abc0a20c0 0x7f0abc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:42.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.577+0000 7f0ad5c79640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0abc0a20c0 0x7f0abc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:42.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.577+0000 7f0ac6ffd640 1 -- 172.21.15.18:0/3780875885 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0ac80cdf20 con 0x7f0ad01016c0 2024-02-17T12:49:42.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.577+0000 7f0ad5c79640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0abc0a20c0 0x7f0abc0a4580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0acc005ab0 tx=0x7f0acc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.581+0000 7f0ac6ffd640 1 -- 172.21.15.18:0/3780875885 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ac8096ec0 con 0x7f0ad01016c0 2024-02-17T12:49:42.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.763+0000 7f0ad7edd640 1 -- 172.21.15.18:0/3780875885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ad019fd40 con 0x7f0ad01016c0 2024-02-17T12:49:42.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.765+0000 7f0ac6ffd640 1 -- 172.21.15.18:0/3780875885 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22607 (secure 0 0 0) 0x7f0ac801f020 con 0x7f0ad01016c0 2024-02-17T12:49:42.757 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:42.757 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":88,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:48:52.149237+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:49:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.766+0000 7f0ad7edd640 1 -- 172.21.15.18:0/3780875885 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0abc0a20c0 msgr2=0x7f0abc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.766+0000 7f0ad7edd640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0abc0a20c0 0x7f0abc0a4580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0acc005ab0 tx=0x7f0acc005fd0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.766+0000 7f0ad7edd640 1 -- 172.21.15.18:0/3780875885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ad01016c0 msgr2=0x7f0ad019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.766+0000 7f0ad7edd640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ad01016c0 0x7f0ad019e830 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800eaa0 tx=0x7f0ac800ef60 comp rx=0 tx=0).stop 2024-02-17T12:49:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.767+0000 7f0ad7edd640 1 -- 172.21.15.18:0/3780875885 shutdown_connections 2024-02-17T12:49:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.767+0000 7f0ad7edd640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f0abc0a20c0 0x7f0abc0a4580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.767+0000 7f0ad7edd640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ad01016c0 0x7f0ad019e830 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.767+0000 7f0ad7edd640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ad0100cd0 0x7f0ad0197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.767+0000 7f0ad7edd640 1 --2- 172.21.15.18:0/3780875885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ad0100310 0x7f0ad01972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.767+0000 7f0ad7edd640 1 -- 172.21.15.18:0/3780875885 >> 172.21.15.18:0/3780875885 conn(0x7f0ad00fada0 msgr2=0x7f0ad00ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:42.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.767+0000 7f0ad7edd640 1 -- 172.21.15.18:0/3780875885 shutdown_connections 2024-02-17T12:49:42.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.767+0000 7f0ad7edd640 1 -- 172.21.15.18:0/3780875885 wait complete. 2024-02-17T12:49:42.769 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:42 smithi018 ceph-mon[32625]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:49:42.769 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:42 smithi018 ceph-mon[32625]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:49:42.770 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T12:43:12.450140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T12:47:43.690543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T12:47:44.653229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T12:49:42.770 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T12:49:42.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:42 smithi119 ceph-mon[36188]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:49:42.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:42 smithi119 ceph-mon[36188]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:49:42.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.885+0000 7ffa21c8f640 1 -- 172.21.15.18:0/319377244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c101c30 msgr2=0x7ffa1c1084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.885+0000 7ffa21c8f640 1 --2- 172.21.15.18:0/319377244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c101c30 0x7ffa1c1084c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009ca0 tx=0x7ffa1402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.885+0000 7ffa21c8f640 1 -- 172.21.15.18:0/319377244 shutdown_connections 2024-02-17T12:49:42.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.885+0000 7ffa21c8f640 1 --2- 172.21.15.18:0/319377244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c101c30 0x7ffa1c1084c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.885+0000 7ffa21c8f640 1 --2- 172.21.15.18:0/319377244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa1c101270 0x7ffa1c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.885+0000 7ffa21c8f640 1 --2- 172.21.15.18:0/319377244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c100080 0x7ffa1c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.885+0000 7ffa21c8f640 1 -- 172.21.15.18:0/319377244 >> 172.21.15.18:0/319377244 conn(0x7ffa1c0fb610 msgr2=0x7ffa1c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.885+0000 7ffa21c8f640 1 -- 172.21.15.18:0/319377244 shutdown_connections 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.885+0000 7ffa21c8f640 1 -- 172.21.15.18:0/319377244 wait complete. 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.885+0000 7ffa21c8f640 1 Processor -- start 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.885+0000 7ffa21c8f640 1 -- start start 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa21c8f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa1c100080 0x7ffa1c068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa21c8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c101270 0x7ffa1c19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa21c8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c101c30 0x7ffa1c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa21c8f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19ecf0 con 0x7ffa1c101270 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa21c8f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19ee60 con 0x7ffa1c101c30 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa21c8f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19f020 con 0x7ffa1c100080 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa1affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c101270 0x7ffa1c19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa1affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c101270 0x7ffa1c19bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34540/0 (socket says 172.21.15.18:34540) 2024-02-17T12:49:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa1affd640 1 -- 172.21.15.18:0/1762546797 learned_addr learned my addr 172.21.15.18:0/1762546797 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa1bfff640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c101c30 0x7ffa1c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa1b7fe640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa1c100080 0x7ffa1c068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa1affd640 1 -- 172.21.15.18:0/1762546797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa1c100080 msgr2=0x7ffa1c068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa1affd640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa1c100080 0x7ffa1c068e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa1affd640 1 -- 172.21.15.18:0/1762546797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c101c30 msgr2=0x7ffa1c19e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa1affd640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c101c30 0x7ffa1c19e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa1affd640 1 -- 172.21.15.18:0/1762546797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa14009900 con 0x7ffa1c101270 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa1affd640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c101270 0x7ffa1c19bf20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000d770 tx=0x7ffa1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa18ff9640 1 -- 172.21.15.18:0/1762546797 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10009c30 con 0x7ffa1c101270 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa18ff9640 1 -- 172.21.15.18:0/1762546797 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa1000de80 con 0x7ffa1c101270 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa21c8f640 1 -- 172.21.15.18:0/1762546797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1c19f300 con 0x7ffa1c101270 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa18ff9640 1 -- 172.21.15.18:0/1762546797 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10004e80 con 0x7ffa1c101270 2024-02-17T12:49:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.886+0000 7ffa21c8f640 1 -- 172.21.15.18:0/1762546797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c19f7c0 con 0x7ffa1c101270 2024-02-17T12:49:42.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.888+0000 7ffa21c8f640 1 -- 172.21.15.18:0/1762546797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1c1073a0 con 0x7ffa1c101270 2024-02-17T12:49:42.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.889+0000 7ffa18ff9640 1 -- 172.21.15.18:0/1762546797 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ffa10005000 con 0x7ffa1c101270 2024-02-17T12:49:42.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.889+0000 7ffa18ff9640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ffa040a20c0 0x7ffa040a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:42.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.889+0000 7ffa18ff9640 1 -- 172.21.15.18:0/1762546797 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ffa100cdb60 con 0x7ffa1c101270 2024-02-17T12:49:42.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.889+0000 7ffa1b7fe640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ffa040a20c0 0x7ffa040a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:42.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.890+0000 7ffa1b7fe640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ffa040a20c0 0x7ffa040a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c1010d0 tx=0x7ffa0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:42.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:42.893+0000 7ffa18ff9640 1 -- 172.21.15.18:0/1762546797 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa10096b60 con 0x7ffa1c101270 2024-02-17T12:49:42.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:42 smithi146 ceph-mon[36454]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:49:42.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:42 smithi146 ceph-mon[36454]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:49:43.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.074+0000 7ffa21c8f640 1 -- 172.21.15.18:0/1762546797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa1c10bc20 con 0x7ffa1c101270 2024-02-17T12:49:43.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.074+0000 7ffa18ff9640 1 -- 172.21.15.18:0/1762546797 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa10017020 con 0x7ffa1c101270 2024-02-17T12:49:43.067 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-02-17T12:49:43.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.077+0000 7ffa21c8f640 1 -- 172.21.15.18:0/1762546797 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ffa040a20c0 msgr2=0x7ffa040a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.077+0000 7ffa21c8f640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ffa040a20c0 0x7ffa040a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c1010d0 tx=0x7ffa0c005f90 comp rx=0 tx=0).stop 2024-02-17T12:49:43.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.077+0000 7ffa21c8f640 1 -- 172.21.15.18:0/1762546797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c101270 msgr2=0x7ffa1c19bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.077+0000 7ffa21c8f640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c101270 0x7ffa1c19bf20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000d770 tx=0x7ffa1000dc30 comp rx=0 tx=0).stop 2024-02-17T12:49:43.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.077+0000 7ffa21c8f640 1 -- 172.21.15.18:0/1762546797 shutdown_connections 2024-02-17T12:49:43.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.077+0000 7ffa21c8f640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ffa040a20c0 0x7ffa040a4580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.077+0000 7ffa21c8f640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c101c30 0x7ffa1c19e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.077+0000 7ffa21c8f640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa1c101270 0x7ffa1c19bf20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.077+0000 7ffa21c8f640 1 --2- 172.21.15.18:0/1762546797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa1c100080 0x7ffa1c068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.077+0000 7ffa21c8f640 1 -- 172.21.15.18:0/1762546797 >> 172.21.15.18:0/1762546797 conn(0x7ffa1c0fb610 msgr2=0x7ffa1c104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:43.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.077+0000 7ffa21c8f640 1 -- 172.21.15.18:0/1762546797 shutdown_connections 2024-02-17T12:49:43.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.077+0000 7ffa21c8f640 1 -- 172.21.15.18:0/1762546797 wait complete. 2024-02-17T12:49:43.080 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T12:49:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.194+0000 7f523ca4e640 1 -- 172.21.15.18:0/198627546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52381011b0 msgr2=0x7f52381015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.194+0000 7f523ca4e640 1 --2- 172.21.15.18:0/198627546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52381011b0 0x7f52381015d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5230009900 tx=0x7f523002bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.194+0000 7f523ca4e640 1 -- 172.21.15.18:0/198627546 shutdown_connections 2024-02-17T12:49:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.194+0000 7f523ca4e640 1 --2- 172.21.15.18:0/198627546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5238102d60 0x7f52381095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.194+0000 7f523ca4e640 1 --2- 172.21.15.18:0/198627546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52381023a0 0x7f5238102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.194+0000 7f523ca4e640 1 --2- 172.21.15.18:0/198627546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52381011b0 0x7f52381015d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.194+0000 7f523ca4e640 1 -- 172.21.15.18:0/198627546 >> 172.21.15.18:0/198627546 conn(0x7f52380fc720 msgr2=0x7f52380feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.194+0000 7f523ca4e640 1 -- 172.21.15.18:0/198627546 shutdown_connections 2024-02-17T12:49:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.194+0000 7f523ca4e640 1 -- 172.21.15.18:0/198627546 wait complete. 2024-02-17T12:49:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.195+0000 7f523ca4e640 1 Processor -- start 2024-02-17T12:49:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.195+0000 7f523ca4e640 1 -- start start 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.195+0000 7f523ca4e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52381011b0 0x7f52381973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.195+0000 7f523ca4e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52381023a0 0x7f5238197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.195+0000 7f523ca4e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5238102d60 0x7f523819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.195+0000 7f523ca4e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523819eed0 con 0x7f52381023a0 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.195+0000 7f523ca4e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523819f040 con 0x7f5238102d60 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.195+0000 7f523ca4e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523819f200 con 0x7f52381011b0 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.195+0000 7f5235d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52381023a0 0x7f5238197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.195+0000 7f5235d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52381023a0 0x7f5238197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34554/0 (socket says 172.21.15.18:34554) 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.195+0000 7f5235d9b640 1 -- 172.21.15.18:0/2296238979 learned_addr learned my addr 172.21.15.18:0/2296238979 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.195+0000 7f523659c640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52381011b0 0x7f52381973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f5236d9d640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5238102d60 0x7f523819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f5235d9b640 1 -- 172.21.15.18:0/2296238979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52381011b0 msgr2=0x7f52381973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f5235d9b640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52381011b0 0x7f52381973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f5235d9b640 1 -- 172.21.15.18:0/2296238979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5238102d60 msgr2=0x7f523819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f5235d9b640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5238102d60 0x7f523819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f5235d9b640 1 -- 172.21.15.18:0/2296238979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5230009560 con 0x7f52381023a0 2024-02-17T12:49:43.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f5236d9d640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5238102d60 0x7f523819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:43.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f5235d9b640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52381023a0 0x7f5238197920 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f522000b4a0 tx=0x7f522000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:43.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f522f7fe640 1 -- 172.21.15.18:0/2296238979 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220010610 con 0x7f52381023a0 2024-02-17T12:49:43.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f523ca4e640 1 -- 172.21.15.18:0/2296238979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523819f4e0 con 0x7f52381023a0 2024-02-17T12:49:43.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f522f7fe640 1 -- 172.21.15.18:0/2296238979 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5220010c50 con 0x7f52381023a0 2024-02-17T12:49:43.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f522f7fe640 1 -- 172.21.15.18:0/2296238979 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220017e20 con 0x7f52381023a0 2024-02-17T12:49:43.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.196+0000 7f523ca4e640 1 -- 172.21.15.18:0/2296238979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523819fac0 con 0x7f52381023a0 2024-02-17T12:49:43.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.197+0000 7f523ca4e640 1 -- 172.21.15.18:0/2296238979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52381085e0 con 0x7f52381023a0 2024-02-17T12:49:43.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.198+0000 7f522f7fe640 1 -- 172.21.15.18:0/2296238979 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5220017480 con 0x7f52381023a0 2024-02-17T12:49:43.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.199+0000 7f522f7fe640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f52240a20c0 0x7f52240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:43.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.199+0000 7f523659c640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f52240a20c0 0x7f52240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:43.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.199+0000 7f522f7fe640 1 -- 172.21.15.18:0/2296238979 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f52200cd2b0 con 0x7f52381023a0 2024-02-17T12:49:43.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.199+0000 7f523659c640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f52240a20c0 0x7f52240a4580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5230009230 tx=0x7f5230005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:43.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.203+0000 7f522f7fe640 1 -- 172.21.15.18:0/2296238979 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f52200962b0 con 0x7f52381023a0 2024-02-17T12:49:43.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.384+0000 7f523ca4e640 1 -- 172.21.15.18:0/2296238979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5238108050 con 0x7f52381023a0 2024-02-17T12:49:43.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.384+0000 7f522f7fe640 1 -- 172.21.15.18:0/2296238979 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5220099970 con 0x7f52381023a0 2024-02-17T12:49:43.376 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-02-17T12:49:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.386+0000 7f523ca4e640 1 -- 172.21.15.18:0/2296238979 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f52240a20c0 msgr2=0x7f52240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.386+0000 7f523ca4e640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f52240a20c0 0x7f52240a4580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5230009230 tx=0x7f5230005fd0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.386+0000 7f523ca4e640 1 -- 172.21.15.18:0/2296238979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52381023a0 msgr2=0x7f5238197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.386+0000 7f523ca4e640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52381023a0 0x7f5238197920 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f522000b4a0 tx=0x7f522000b960 comp rx=0 tx=0).stop 2024-02-17T12:49:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.386+0000 7f523ca4e640 1 -- 172.21.15.18:0/2296238979 shutdown_connections 2024-02-17T12:49:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.386+0000 7f523ca4e640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f52240a20c0 0x7f52240a4580 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.386+0000 7f523ca4e640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5238102d60 0x7f523819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.386+0000 7f523ca4e640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52381023a0 0x7f5238197920 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.386+0000 7f523ca4e640 1 --2- 172.21.15.18:0/2296238979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52381011b0 0x7f52381973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.386+0000 7f523ca4e640 1 -- 172.21.15.18:0/2296238979 >> 172.21.15.18:0/2296238979 conn(0x7f52380fc720 msgr2=0x7f52381057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.386+0000 7f523ca4e640 1 -- 172.21.15.18:0/2296238979 shutdown_connections 2024-02-17T12:49:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.386+0000 7f523ca4e640 1 -- 172.21.15.18:0/2296238979 wait complete. 2024-02-17T12:49:43.389 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T12:49:43.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.502+0000 7f6ba9881640 1 -- 172.21.15.18:0/3898942303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba4101270 msgr2=0x7f6ba41016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.502+0000 7f6ba9881640 1 --2- 172.21.15.18:0/3898942303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba4101270 0x7f6ba41016f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009900 tx=0x7f6b9802bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.502+0000 7f6ba9881640 1 -- 172.21.15.18:0/3898942303 shutdown_connections 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.502+0000 7f6ba9881640 1 --2- 172.21.15.18:0/3898942303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4101c30 0x7f6ba41084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.502+0000 7f6ba9881640 1 --2- 172.21.15.18:0/3898942303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba4101270 0x7f6ba41016f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.502+0000 7f6ba9881640 1 --2- 172.21.15.18:0/3898942303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ba4100080 0x7f6ba41004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.502+0000 7f6ba9881640 1 -- 172.21.15.18:0/3898942303 >> 172.21.15.18:0/3898942303 conn(0x7f6ba40fb610 msgr2=0x7f6ba40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.502+0000 7f6ba9881640 1 -- 172.21.15.18:0/3898942303 shutdown_connections 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.502+0000 7f6ba9881640 1 -- 172.21.15.18:0/3898942303 wait complete. 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba9881640 1 Processor -- start 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba9881640 1 -- start start 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba9881640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ba4100080 0x7f6ba4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba9881640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba4101270 0x7f6ba41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:43.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba9881640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4101c30 0x7f6ba419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba9881640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba419ed80 con 0x7f6ba4101270 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba9881640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba419eef0 con 0x7f6ba4101c30 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba9881640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba419f0b0 con 0x7f6ba4100080 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba27fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba4101270 0x7f6ba41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba27fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba4101270 0x7f6ba41977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34576/0 (socket says 172.21.15.18:34576) 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba27fc640 1 -- 172.21.15.18:0/3793532704 learned_addr learned my addr 172.21.15.18:0/3793532704 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba2ffd640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ba4100080 0x7f6ba4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba37fe640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4101c30 0x7f6ba419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.503+0000 7f6ba27fc640 1 -- 172.21.15.18:0/3793532704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ba4100080 msgr2=0x7f6ba4197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.504+0000 7f6ba27fc640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ba4100080 0x7f6ba4197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.504+0000 7f6ba27fc640 1 -- 172.21.15.18:0/3793532704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4101c30 msgr2=0x7f6ba419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.504+0000 7f6ba27fc640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4101c30 0x7f6ba419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.504+0000 7f6ba27fc640 1 -- 172.21.15.18:0/3793532704 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b98009560 con 0x7f6ba4101270 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.504+0000 7f6ba37fe640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4101c30 0x7f6ba419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:43.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.504+0000 7f6ba27fc640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba4101270 0x7f6ba41977d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6b98006010 tx=0x7f6b980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:43.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.504+0000 7f6ba887f640 1 -- 172.21.15.18:0/3793532704 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b980044e0 con 0x7f6ba4101270 2024-02-17T12:49:43.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.504+0000 7f6ba9881640 1 -- 172.21.15.18:0/3793532704 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba419f330 con 0x7f6ba4101270 2024-02-17T12:49:43.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.504+0000 7f6ba887f640 1 -- 172.21.15.18:0/3793532704 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b980054e0 con 0x7f6ba4101270 2024-02-17T12:49:43.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.504+0000 7f6ba887f640 1 -- 172.21.15.18:0/3793532704 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9803e450 con 0x7f6ba4101270 2024-02-17T12:49:43.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.504+0000 7f6ba9881640 1 -- 172.21.15.18:0/3793532704 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba419f840 con 0x7f6ba4101270 2024-02-17T12:49:43.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.505+0000 7f6ba9881640 1 -- 172.21.15.18:0/3793532704 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba410ba90 con 0x7f6ba4101270 2024-02-17T12:49:43.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.506+0000 7f6ba887f640 1 -- 172.21.15.18:0/3793532704 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6b980035f0 con 0x7f6ba4101270 2024-02-17T12:49:43.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.507+0000 7f6ba887f640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6b8c0a20c0 0x7f6b8c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:43.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.507+0000 7f6ba2ffd640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6b8c0a20c0 0x7f6b8c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:43.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.507+0000 7f6ba887f640 1 -- 172.21.15.18:0/3793532704 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6b980f00f0 con 0x7f6ba4101270 2024-02-17T12:49:43.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.507+0000 7f6ba2ffd640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6b8c0a20c0 0x7f6b8c0a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c005fd0 tx=0x7f6b9c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:43.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.511+0000 7f6ba887f640 1 -- 172.21.15.18:0/3793532704 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b980b90f0 con 0x7f6ba4101270 2024-02-17T12:49:43.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.692+0000 7f6ba9881640 1 -- 172.21.15.18:0/3793532704 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ba419fcd0 con 0x7f6ba4101270 2024-02-17T12:49:43.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.692+0000 7f6ba887f640 1 -- 172.21.15.18:0/3793532704 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b980bc7b0 con 0x7f6ba4101270 2024-02-17T12:49:43.685 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-02-17T12:49:43.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.694+0000 7f6ba9881640 1 -- 172.21.15.18:0/3793532704 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6b8c0a20c0 msgr2=0x7f6b8c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.694+0000 7f6ba9881640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6b8c0a20c0 0x7f6b8c0a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c005fd0 tx=0x7f6b9c005850 comp rx=0 tx=0).stop 2024-02-17T12:49:43.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.694+0000 7f6ba9881640 1 -- 172.21.15.18:0/3793532704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba4101270 msgr2=0x7f6ba41977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.694+0000 7f6ba9881640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba4101270 0x7f6ba41977d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6b98006010 tx=0x7f6b980042f0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.694+0000 7f6ba9881640 1 -- 172.21.15.18:0/3793532704 shutdown_connections 2024-02-17T12:49:43.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.694+0000 7f6ba9881640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f6b8c0a20c0 0x7f6b8c0a4580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.694+0000 7f6ba9881640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4101c30 0x7f6ba419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.694+0000 7f6ba9881640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba4101270 0x7f6ba41977d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.694+0000 7f6ba9881640 1 --2- 172.21.15.18:0/3793532704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ba4100080 0x7f6ba4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.694+0000 7f6ba9881640 1 -- 172.21.15.18:0/3793532704 >> 172.21.15.18:0/3793532704 conn(0x7f6ba40fb610 msgr2=0x7f6ba40fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:43.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.695+0000 7f6ba9881640 1 -- 172.21.15.18:0/3793532704 shutdown_connections 2024-02-17T12:49:43.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.695+0000 7f6ba9881640 1 -- 172.21.15.18:0/3793532704 wait complete. 2024-02-17T12:49:43.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:43 smithi018 ceph-mon[32625]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-17T12:49:43.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:43 smithi018 ceph-mon[32625]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:49:43.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:43 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3780875885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:43.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:43 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1762546797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:43.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:43 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2296238979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:43.698 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:49:43.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.808+0000 7f30af0f4640 1 -- 172.21.15.18:0/1467336131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30a8101290 msgr2=0x7f30a8101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.808+0000 7f30af0f4640 1 --2- 172.21.15.18:0/1467336131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30a8101290 0x7f30a8101710 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3094009900 tx=0x7f309402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:43.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.808+0000 7f30af0f4640 1 -- 172.21.15.18:0/1467336131 shutdown_connections 2024-02-17T12:49:43.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.808+0000 7f30af0f4640 1 --2- 172.21.15.18:0/1467336131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a8101c50 0x7f30a81084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.808+0000 7f30af0f4640 1 --2- 172.21.15.18:0/1467336131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30a8101290 0x7f30a8101710 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.808+0000 7f30af0f4640 1 --2- 172.21.15.18:0/1467336131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30a81000a0 0x7f30a81004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.808+0000 7f30af0f4640 1 -- 172.21.15.18:0/1467336131 >> 172.21.15.18:0/1467336131 conn(0x7f30a80fb610 msgr2=0x7f30a80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:43.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.808+0000 7f30af0f4640 1 -- 172.21.15.18:0/1467336131 shutdown_connections 2024-02-17T12:49:43.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.808+0000 7f30af0f4640 1 -- 172.21.15.18:0/1467336131 wait complete. 2024-02-17T12:49:43.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.808+0000 7f30af0f4640 1 Processor -- start 2024-02-17T12:49:43.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.808+0000 7f30af0f4640 1 -- start start 2024-02-17T12:49:43.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30af0f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a81000a0 0x7f30a8068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:43.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30af0f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30a8101290 0x7f30a819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:43.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30af0f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30a8101c50 0x7f30a819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:43.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30af0f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a819ed80 con 0x7f30a8101290 2024-02-17T12:49:43.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30af0f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a819eef0 con 0x7f30a8101c50 2024-02-17T12:49:43.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30af0f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a819f0b0 con 0x7f30a81000a0 2024-02-17T12:49:43.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30a8101290 0x7f30a819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:43.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30a8101290 0x7f30a819bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34588/0 (socket says 172.21.15.18:34588) 2024-02-17T12:49:43.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30a7fff640 1 -- 172.21.15.18:0/1596506406 learned_addr learned my addr 172.21.15.18:0/1596506406 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:43.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30ad691640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30a8101c50 0x7f30a819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:43.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30ace90640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a81000a0 0x7f30a8068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30a7fff640 1 -- 172.21.15.18:0/1596506406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a81000a0 msgr2=0x7f30a8068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30a7fff640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a81000a0 0x7f30a8068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30a7fff640 1 -- 172.21.15.18:0/1596506406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30a8101c50 msgr2=0x7f30a819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30a7fff640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30a8101c50 0x7f30a819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30a7fff640 1 -- 172.21.15.18:0/1596506406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3094009560 con 0x7f30a8101290 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30ace90640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a81000a0 0x7f30a8068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30a7fff640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30a8101290 0x7f30a819bf80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3094006010 tx=0x7f30940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30a5ffb640 1 -- 172.21.15.18:0/1596506406 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30940044e0 con 0x7f30a8101290 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30a5ffb640 1 -- 172.21.15.18:0/1596506406 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30940054e0 con 0x7f30a8101290 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30a5ffb640 1 -- 172.21.15.18:0/1596506406 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309403e450 con 0x7f30a8101290 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.809+0000 7f30af0f4640 1 -- 172.21.15.18:0/1596506406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a819f2d0 con 0x7f30a8101290 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.810+0000 7f30af0f4640 1 -- 172.21.15.18:0/1596506406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a819f7e0 con 0x7f30a8101290 2024-02-17T12:49:43.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.811+0000 7f30af0f4640 1 -- 172.21.15.18:0/1596506406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a81073c0 con 0x7f30a8101290 2024-02-17T12:49:43.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.812+0000 7f30a5ffb640 1 -- 172.21.15.18:0/1596506406 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f30940035f0 con 0x7f30a8101290 2024-02-17T12:49:43.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.812+0000 7f30a5ffb640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f30980a20c0 0x7f30980a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:43.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.812+0000 7f30a5ffb640 1 -- 172.21.15.18:0/1596506406 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f30940f0070 con 0x7f30a8101290 2024-02-17T12:49:43.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.812+0000 7f30ace90640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f30980a20c0 0x7f30980a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:43.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.813+0000 7f30ace90640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f30980a20c0 0x7f30980a4580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f309c0058c0 tx=0x7f309c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:43.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:43.817+0000 7f30a5ffb640 1 -- 172.21.15.18:0/1596506406 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f30940b9070 con 0x7f30a8101290 2024-02-17T12:49:43.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:43 smithi119 ceph-mon[36188]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-17T12:49:43.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:43 smithi119 ceph-mon[36188]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:49:43.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:43 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3780875885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:43.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:43 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1762546797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:43.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:43 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2296238979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:43.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:43 smithi146 ceph-mon[36454]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-17T12:49:43.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:43 smithi146 ceph-mon[36454]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T12:49:43.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:43 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3780875885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:43.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:43 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1762546797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:49:43.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:43 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2296238979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:49:44.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.034+0000 7f30af0f4640 1 -- 172.21.15.18:0/1596506406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30a802ce40 con 0x7f30a8101290 2024-02-17T12:49:44.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.035+0000 7f30a5ffb640 1 -- 172.21.15.18:0/1596506406 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11392 (secure 0 0 0) 0x7f30940bc730 con 0x7f30a8101290 2024-02-17T12:49:44.027 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:44.027 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:49:36.672377+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:49:44.028 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 26 2024-02-17T12:49:44.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.037+0000 7f30af0f4640 1 -- 172.21.15.18:0/1596506406 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f30980a20c0 msgr2=0x7f30980a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.037+0000 7f30af0f4640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f30980a20c0 0x7f30980a4580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f309c0058c0 tx=0x7f309c005850 comp rx=0 tx=0).stop 2024-02-17T12:49:44.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.037+0000 7f30af0f4640 1 -- 172.21.15.18:0/1596506406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30a8101290 msgr2=0x7f30a819bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.037+0000 7f30af0f4640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30a8101290 0x7f30a819bf80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3094006010 tx=0x7f30940042f0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.037+0000 7f30af0f4640 1 -- 172.21.15.18:0/1596506406 shutdown_connections 2024-02-17T12:49:44.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.037+0000 7f30af0f4640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f30980a20c0 0x7f30980a4580 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.037+0000 7f30af0f4640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30a8101c50 0x7f30a819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.037+0000 7f30af0f4640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30a8101290 0x7f30a819bf80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.037+0000 7f30af0f4640 1 --2- 172.21.15.18:0/1596506406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a81000a0 0x7f30a8068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.037+0000 7f30af0f4640 1 -- 172.21.15.18:0/1596506406 >> 172.21.15.18:0/1596506406 conn(0x7f30a80fb610 msgr2=0x7f30a8104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.037+0000 7f30af0f4640 1 -- 172.21.15.18:0/1596506406 shutdown_connections 2024-02-17T12:49:44.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.037+0000 7f30af0f4640 1 -- 172.21.15.18:0/1596506406 wait complete. 2024-02-17T12:49:44.040 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:49:44.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.150+0000 7fefb6fe9640 1 -- 172.21.15.18:0/2396154667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefb0102d50 msgr2=0x7fefb01095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.150+0000 7fefb6fe9640 1 --2- 172.21.15.18:0/2396154667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefb0102d50 0x7fefb01095e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fef98009900 tx=0x7fef9802bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:44.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.150+0000 7fefb6fe9640 1 -- 172.21.15.18:0/2396154667 shutdown_connections 2024-02-17T12:49:44.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.150+0000 7fefb6fe9640 1 --2- 172.21.15.18:0/2396154667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefb0102d50 0x7fefb01095e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.150+0000 7fefb6fe9640 1 --2- 172.21.15.18:0/2396154667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefb0102390 0x7fefb0102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.150+0000 7fefb6fe9640 1 --2- 172.21.15.18:0/2396154667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefb01011a0 0x7fefb01015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.150+0000 7fefb6fe9640 1 -- 172.21.15.18:0/2396154667 >> 172.21.15.18:0/2396154667 conn(0x7fefb00fc730 msgr2=0x7fefb00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.150+0000 7fefb6fe9640 1 -- 172.21.15.18:0/2396154667 shutdown_connections 2024-02-17T12:49:44.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.150+0000 7fefb6fe9640 1 -- 172.21.15.18:0/2396154667 wait complete. 2024-02-17T12:49:44.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefb6fe9640 1 Processor -- start 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefb6fe9640 1 -- start start 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefb6fe9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefb01011a0 0x7fefb01971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefb6fe9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefb0102390 0x7fefb0197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefb6fe9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefb0102d50 0x7fefb019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefb6fe9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb019ecb0 con 0x7fefb0102390 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefb6fe9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb019ee20 con 0x7fefb01011a0 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefb6fe9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb019efe0 con 0x7fefb0102d50 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefaffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefb0102390 0x7fefb0197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefaffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefb0102390 0x7fefb0197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34608/0 (socket says 172.21.15.18:34608) 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefaffff640 1 -- 172.21.15.18:0/3552014445 learned_addr learned my addr 172.21.15.18:0/3552014445 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefb4d85640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefb01011a0 0x7fefb01971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.151+0000 7fefb5586640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefb0102d50 0x7fefb019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.152+0000 7fefaffff640 1 -- 172.21.15.18:0/3552014445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefb0102d50 msgr2=0x7fefb019e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.152+0000 7fefaffff640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefb0102d50 0x7fefb019e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.152+0000 7fefaffff640 1 -- 172.21.15.18:0/3552014445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefb01011a0 msgr2=0x7fefb01971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.152+0000 7fefaffff640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefb01011a0 0x7fefb01971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.152+0000 7fefaffff640 1 -- 172.21.15.18:0/3552014445 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef98009560 con 0x7fefb0102390 2024-02-17T12:49:44.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.152+0000 7fefaffff640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefb0102390 0x7fefb0197700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fefa000e9f0 tx=0x7fefa000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.152+0000 7fefadffb640 1 -- 172.21.15.18:0/3552014445 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0009700 con 0x7fefb0102390 2024-02-17T12:49:44.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.152+0000 7fefb6fe9640 1 -- 172.21.15.18:0/3552014445 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb019f2c0 con 0x7fefb0102390 2024-02-17T12:49:44.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.152+0000 7fefadffb640 1 -- 172.21.15.18:0/3552014445 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefa000cd80 con 0x7fefb0102390 2024-02-17T12:49:44.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.152+0000 7fefadffb640 1 -- 172.21.15.18:0/3552014445 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0018920 con 0x7fefb0102390 2024-02-17T12:49:44.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.152+0000 7fefb6fe9640 1 -- 172.21.15.18:0/3552014445 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb019f820 con 0x7fefb0102390 2024-02-17T12:49:44.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.153+0000 7fefb6fe9640 1 -- 172.21.15.18:0/3552014445 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb01084c0 con 0x7fefb0102390 2024-02-17T12:49:44.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.154+0000 7fefadffb640 1 -- 172.21.15.18:0/3552014445 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fefa0018a80 con 0x7fefb0102390 2024-02-17T12:49:44.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.155+0000 7fefadffb640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fef9c0a20c0 0x7fef9c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.155+0000 7fefb4d85640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fef9c0a20c0 0x7fef9c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.155+0000 7fefadffb640 1 -- 172.21.15.18:0/3552014445 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fefa0014070 con 0x7fefb0102390 2024-02-17T12:49:44.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.155+0000 7fefb4d85640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fef9c0a20c0 0x7fef9c0a4580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fefb01021f0 tx=0x7fefa8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.159+0000 7fefadffb640 1 -- 172.21.15.18:0/3552014445 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fefa0052d20 con 0x7fefb0102390 2024-02-17T12:49:44.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.342+0000 7fefb6fe9640 1 -- 172.21.15.18:0/3552014445 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefb010cdd0 con 0x7fefb0102390 2024-02-17T12:49:44.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.344+0000 7fefadffb640 1 -- 172.21.15.18:0/3552014445 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22607 (secure 0 0 0) 0x7fefa0097120 con 0x7fefb0102390 2024-02-17T12:49:44.336 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:49:44.336 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":88,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T12:48:52.149237+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:49:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.345+0000 7fefb6fe9640 1 -- 172.21.15.18:0/3552014445 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fef9c0a20c0 msgr2=0x7fef9c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.345+0000 7fefb6fe9640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fef9c0a20c0 0x7fef9c0a4580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fefb01021f0 tx=0x7fefa8009500 comp rx=0 tx=0).stop 2024-02-17T12:49:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.345+0000 7fefb6fe9640 1 -- 172.21.15.18:0/3552014445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefb0102390 msgr2=0x7fefb0197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.345+0000 7fefb6fe9640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefb0102390 0x7fefb0197700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fefa000e9f0 tx=0x7fefa000eeb0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.345+0000 7fefb6fe9640 1 -- 172.21.15.18:0/3552014445 shutdown_connections 2024-02-17T12:49:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.345+0000 7fefb6fe9640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fef9c0a20c0 0x7fef9c0a4580 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.345+0000 7fefb6fe9640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefb0102d50 0x7fefb019e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.346+0000 7fefb6fe9640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefb0102390 0x7fefb0197700 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.346+0000 7fefb6fe9640 1 --2- 172.21.15.18:0/3552014445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefb01011a0 0x7fefb01971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.346+0000 7fefb6fe9640 1 -- 172.21.15.18:0/3552014445 >> 172.21.15.18:0/3552014445 conn(0x7fefb00fc730 msgr2=0x7fefb0105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.346+0000 7fefb6fe9640 1 -- 172.21.15.18:0/3552014445 shutdown_connections 2024-02-17T12:49:44.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.346+0000 7fefb6fe9640 1 -- 172.21.15.18:0/3552014445 wait complete. 2024-02-17T12:49:44.349 DEBUG:tasks.fwd_scrub:Yielding 2024-02-17T12:49:44.349 INFO:teuthology.run_tasks:Running task exec... 2024-02-17T12:49:44.359 INFO:teuthology.task.exec:Executing custom commands... 2024-02-17T12:49:44.360 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-02-17T12:49:44.360 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-02-17T12:49:44.360 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-02-17T12:49:44.360 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:49:44.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:49:44.361 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:49:44.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.472+0000 7fceb83f4640 1 -- 172.21.15.18:0/1126272216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fceb0101c50 msgr2=0x7fceb01084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.472+0000 7fceb83f4640 1 --2- 172.21.15.18:0/1126272216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fceb0101c50 0x7fceb01084e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009980 tx=0x7fcea0030060 comp rx=0 tx=0).stop 2024-02-17T12:49:44.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.472+0000 7fceb83f4640 1 -- 172.21.15.18:0/1126272216 shutdown_connections 2024-02-17T12:49:44.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.472+0000 7fceb83f4640 1 --2- 172.21.15.18:0/1126272216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fceb0101c50 0x7fceb01084e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.472+0000 7fceb83f4640 1 --2- 172.21.15.18:0/1126272216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fceb0101290 0x7fceb0101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.472+0000 7fceb83f4640 1 --2- 172.21.15.18:0/1126272216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb01000a0 0x7fceb01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.472+0000 7fceb83f4640 1 -- 172.21.15.18:0/1126272216 >> 172.21.15.18:0/1126272216 conn(0x7fceb00fb610 msgr2=0x7fceb00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.472+0000 7fceb83f4640 1 -- 172.21.15.18:0/1126272216 shutdown_connections 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.472+0000 7fceb83f4640 1 -- 172.21.15.18:0/1126272216 wait complete. 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.472+0000 7fceb83f4640 1 Processor -- start 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb83f4640 1 -- start start 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb83f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fceb01000a0 0x7fceb0194f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb83f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb0101290 0x7fceb0195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb83f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fceb0101c50 0x7fceb019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb83f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb019ca20 con 0x7fceb01000a0 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb83f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb019cb90 con 0x7fceb0101c50 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb83f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb019cd50 con 0x7fceb0101290 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb6190640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fceb01000a0 0x7fceb0194f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb6190640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fceb01000a0 0x7fceb0194f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34636/0 (socket says 172.21.15.18:34636) 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb6190640 1 -- 172.21.15.18:0/726569984 learned_addr learned my addr 172.21.15.18:0/726569984 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb6991640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fceb0101c50 0x7fceb019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb598f640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb0101290 0x7fceb0195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb6190640 1 -- 172.21.15.18:0/726569984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb0101290 msgr2=0x7fceb0195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb6190640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb0101290 0x7fceb0195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb6190640 1 -- 172.21.15.18:0/726569984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fceb0101c50 msgr2=0x7fceb019c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb6190640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fceb0101c50 0x7fceb019c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb6190640 1 -- 172.21.15.18:0/726569984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea0009560 con 0x7fceb01000a0 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb598f640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb0101290 0x7fceb0195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb6991640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fceb0101c50 0x7fceb019c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.473+0000 7fceb6190640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fceb01000a0 0x7fceb0194f30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009fd0 tx=0x7fcea800c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.474+0000 7fceaf7fe640 1 -- 172.21.15.18:0/726569984 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8011470 con 0x7fceb01000a0 2024-02-17T12:49:44.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.474+0000 7fceb83f4640 1 -- 172.21.15.18:0/726569984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb019cfd0 con 0x7fceb01000a0 2024-02-17T12:49:44.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.474+0000 7fceaf7fe640 1 -- 172.21.15.18:0/726569984 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcea8011ab0 con 0x7fceb01000a0 2024-02-17T12:49:44.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.474+0000 7fceaf7fe640 1 -- 172.21.15.18:0/726569984 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea801ab00 con 0x7fceb01000a0 2024-02-17T12:49:44.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.474+0000 7fceb83f4640 1 -- 172.21.15.18:0/726569984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb019d4a0 con 0x7fceb01000a0 2024-02-17T12:49:44.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.475+0000 7fceb83f4640 1 -- 172.21.15.18:0/726569984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb01073c0 con 0x7fceb01000a0 2024-02-17T12:49:44.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.475+0000 7fee95b8c640 1 -- 172.21.15.18:0/173800298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee90100080 msgr2=0x7fee901004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.475+0000 7fee95b8c640 1 --2- 172.21.15.18:0/173800298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee90100080 0x7fee901004a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fee80009900 tx=0x7fee8002bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:44.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.475+0000 7fee95b8c640 1 -- 172.21.15.18:0/173800298 shutdown_connections 2024-02-17T12:49:44.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.475+0000 7fee95b8c640 1 --2- 172.21.15.18:0/173800298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee90101c30 0x7fee901084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.475+0000 7fee95b8c640 1 --2- 172.21.15.18:0/173800298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee90101270 0x7fee901016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.475+0000 7fee95b8c640 1 --2- 172.21.15.18:0/173800298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee90100080 0x7fee901004a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.475+0000 7fee95b8c640 1 -- 172.21.15.18:0/173800298 >> 172.21.15.18:0/173800298 conn(0x7fee900fb610 msgr2=0x7fee900fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.476+0000 7fee95b8c640 1 -- 172.21.15.18:0/173800298 shutdown_connections 2024-02-17T12:49:44.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.476+0000 7fee95b8c640 1 -- 172.21.15.18:0/173800298 wait complete. 2024-02-17T12:49:44.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.476+0000 7fceaf7fe640 1 -- 172.21.15.18:0/726569984 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcea8011c20 con 0x7fceb01000a0 2024-02-17T12:49:44.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.476+0000 7fee95b8c640 1 Processor -- start 2024-02-17T12:49:44.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.476+0000 7fee95b8c640 1 -- start start 2024-02-17T12:49:44.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.476+0000 7fee95b8c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee90100080 0x7fee90104160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.476+0000 7fee95b8c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee90101270 0x7fee901027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.476+0000 7fee95b8c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee90101c30 0x7fee90102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.476+0000 7fee95b8c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee90103440 con 0x7fee90101c30 2024-02-17T12:49:44.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.476+0000 7fee95b8c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee901035b0 con 0x7fee90100080 2024-02-17T12:49:44.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.476+0000 7fee95b8c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee901a12f0 con 0x7fee90101270 2024-02-17T12:49:44.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fceaf7fe640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fce980a2160 0x7fce980a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fee8ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee90101c30 0x7fee90102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fee8effd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee90101270 0x7fee901027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fee8ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee90101c30 0x7fee90102ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34646/0 (socket says 172.21.15.18:34646) 2024-02-17T12:49:44.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fee8ffff640 1 -- 172.21.15.18:0/1313894000 learned_addr learned my addr 172.21.15.18:0/1313894000 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:44.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fceb598f640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fce980a2160 0x7fce980a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fceaf7fe640 1 -- 172.21.15.18:0/726569984 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcea80ce280 con 0x7fceb01000a0 2024-02-17T12:49:44.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee90100080 0x7fee90104160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fee8ffff640 1 -- 172.21.15.18:0/1313894000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee90101270 msgr2=0x7fee901027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fee8ffff640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee90101270 0x7fee901027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fee8ffff640 1 -- 172.21.15.18:0/1313894000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee90100080 msgr2=0x7fee90104160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fee8ffff640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee90100080 0x7fee90104160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fee8ffff640 1 -- 172.21.15.18:0/1313894000 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee80009560 con 0x7fee90101c30 2024-02-17T12:49:44.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee90100080 0x7fee90104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:49:44.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fceb598f640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fce980a2160 0x7fce980a4620 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fceb0196440 tx=0x7fcea400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.477+0000 7fee8ffff640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee90101c30 0x7fee90102ce0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fee8800eb40 tx=0x7fee8800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.481+0000 7fee8cff9640 1 -- 172.21.15.18:0/1313894000 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88004b20 con 0x7fee90101c30 2024-02-17T12:49:44.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.481+0000 7fee8cff9640 1 -- 172.21.15.18:0/1313894000 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee88004c80 con 0x7fee90101c30 2024-02-17T12:49:44.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.481+0000 7fee95b8c640 1 -- 172.21.15.18:0/1313894000 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee901a15a0 con 0x7fee90101c30 2024-02-17T12:49:44.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.481+0000 7fee8cff9640 1 -- 172.21.15.18:0/1313894000 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88010670 con 0x7fee90101c30 2024-02-17T12:49:44.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.481+0000 7fee95b8c640 1 -- 172.21.15.18:0/1313894000 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee901a1ab0 con 0x7fee90101c30 2024-02-17T12:49:44.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.481+0000 7fceaf7fe640 1 -- 172.21.15.18:0/726569984 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcea8097280 con 0x7fceb01000a0 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.484+0000 7fee98765740 1 Processor -- start 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.484+0000 7fee8cff9640 1 -- 172.21.15.18:0/1313894000 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fee88005020 con 0x7fee90101c30 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.484+0000 7fee98765740 1 -- start start 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.484+0000 7fee98765740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 0x556750ead8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.484+0000 7fee98765740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eae690 0x556750eaeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.484+0000 7fee98765740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 0x556750eb58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.484+0000 7fee98765740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556750e1cf90 con 0x556750ead4c0 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.484+0000 7fee98765740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556750e1a4c0 con 0x556750eaf050 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.484+0000 7fee98765740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556750e15ca0 con 0x556750eae690 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 0x556750ead8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 0x556750ead8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34650/0 (socket says 172.21.15.18:34650) 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8effd640 1 -- 172.21.15.18:0/3304418793 learned_addr learned my addr 172.21.15.18:0/3304418793 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:44.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8cff9640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fee780a2140 0x7fee780a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/3304418793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eae690 0x556750eaeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8effd640 1 --2- 172.21.15.18:0/3304418793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 0x556750eb58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8effd640 1 -- 172.21.15.18:0/3304418793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eae690 msgr2=0x556750eaeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8effd640 1 --2- 172.21.15.18:0/3304418793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eae690 0x556750eaeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8effd640 1 -- 172.21.15.18:0/3304418793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 msgr2=0x556750eb58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8effd640 1 --2- 172.21.15.18:0/3304418793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 0x556750eb58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8effd640 1 -- 172.21.15.18:0/3304418793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556750dd80b0 con 0x556750ead4c0 2024-02-17T12:49:44.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8ffff640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fee780a2140 0x7fee780a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8effd640 1 --2- 172.21.15.18:0/3304418793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 0x556750ead8e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fee84013620 tx=0x7fee84009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f201792b3571d1cd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee8cff9640 1 -- 172.21.15.18:0/1313894000 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fee88014070 con 0x7fee90101c30 2024-02-17T12:49:44.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee75ffb640 1 -- 172.21.15.18:0/3304418793 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84004050 con 0x556750ead4c0 2024-02-17T12:49:44.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee75ffb640 1 -- 172.21.15.18:0/3304418793 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee840041b0 con 0x556750ead4c0 2024-02-17T12:49:44.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.485+0000 7fee75ffb640 1 -- 172.21.15.18:0/3304418793 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8400f7a0 con 0x556750ead4c0 2024-02-17T12:49:44.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.486+0000 7fee98765740 1 -- 172.21.15.18:0/3304418793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 msgr2=0x556750ead8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.486+0000 7fee98765740 1 --2- 172.21.15.18:0/3304418793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 0x556750ead8e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fee84013620 tx=0x7fee84009830 comp rx=0 tx=0).stop 2024-02-17T12:49:44.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.486+0000 7fee8ffff640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fee780a2140 0x7fee780a4600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fee8809abc0 tx=0x7fee8801b6d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.486+0000 7fee98765740 1 -- 172.21.15.18:0/3304418793 shutdown_connections 2024-02-17T12:49:44.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.486+0000 7fee98765740 1 --2- 172.21.15.18:0/3304418793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 0x556750eb58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.486+0000 7fee98765740 1 --2- 172.21.15.18:0/3304418793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eae690 0x556750eaeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.486+0000 7fee98765740 1 --2- 172.21.15.18:0/3304418793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 0x556750ead8e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.486+0000 7fee98765740 1 -- 172.21.15.18:0/3304418793 >> 172.21.15.18:0/3304418793 conn(0x556750ea86a0 msgr2=0x556750ea8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.486+0000 7fee98765740 1 -- 172.21.15.18:0/3304418793 shutdown_connections 2024-02-17T12:49:44.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.486+0000 7fee98765740 1 -- 172.21.15.18:0/3304418793 wait complete. 2024-02-17T12:49:44.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.486+0000 7fee98765740 1 -- start start 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee98765740 1 Processor -- start 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee98765740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750ead4c0 0x556750eb76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee98765740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 0x556750eb7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee98765740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eaf050 0x556750ebec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee98765740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556750dbaf40 con 0x556750eae690 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee98765740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556750ae94b0 con 0x556750ead4c0 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee98765740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556750d9b670 con 0x556750eaf050 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee98765740 10 client.0 _ll_register_callbacks cb 0x556750e08f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee8f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 0x556750eb7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee8f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 0x556750eb7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34664/0 (socket says 172.21.15.18:34664) 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee8f7fe640 1 -- 172.21.15.18:0/1422498869 learned_addr learned my addr 172.21.15.18:0/1422498869 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee8effd640 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750ead4c0 0x556750eb76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee8effd640 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eaf050 0x556750ebec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee8f7fe640 1 -- 172.21.15.18:0/1422498869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eaf050 msgr2=0x556750ebec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eaf050 0x556750ebec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee8f7fe640 1 -- 172.21.15.18:0/1422498869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750ead4c0 msgr2=0x556750eb76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750ead4c0 0x556750eb76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee8f7fe640 1 -- 172.21.15.18:0/1422498869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee84009d70 con 0x556750eae690 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.487+0000 7fee8effd640 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eaf050 0x556750ebec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.488+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 0x556750eb7c20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fee80009900 tx=0x7fee80004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48add19c7482fde3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.488+0000 7fee6effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.488+0000 7fee6effd640 1 -- 172.21.15.18:0/1422498869 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8002cc90 con 0x556750eae690 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.488+0000 7fee6effd640 1 -- 172.21.15.18:0/1422498869 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fee80045070 con 0x556750eae690 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.488+0000 7fee98765740 1 -- 172.21.15.18:0/1422498869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556750deff40 con 0x556750eae690 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.488+0000 7fee6effd640 1 -- 172.21.15.18:0/1422498869 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee80005d50 con 0x556750eae690 2024-02-17T12:49:44.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.488+0000 7fee98765740 10 client.15486 fetch_fsmap learned FSMap version 26 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.488+0000 7fee98765740 1 -- 172.21.15.18:0/1422498869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556750e14320 con 0x556750eae690 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.488+0000 7fee6effd640 1 -- 172.21.15.18:0/1422498869 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556750deff40 con 0x556750eae690 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.488+0000 7fee6effd640 1 -- 172.21.15.18:0/1422498869 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fee8003c040 con 0x556750eae690 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.489+0000 7fee98765740 10 client.15486 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.489+0000 7fee98765740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.489+0000 7fee98765740 20 client.15486 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.489+0000 7fee98765740 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556750ec3900 0x556750ec5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.489+0000 7fee98765740 4 client.15486 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.489+0000 7fee98765740 1 -- 172.21.15.18:0/1422498869 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556750e0c310 con 0x556750ec3900 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.489+0000 7fee8effd640 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556750ec3900 0x556750ec5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.489+0000 7fee8effd640 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556750ec3900 0x556750ec5d20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c7a431bebaf6b04 server_cookie=159468152e296e81 in_seq=0 out_seq=0 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.489+0000 7fee6effd640 10 client.15486 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee6effd640 1 -- 172.21.15.18:0/1422498869 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fee8402ca90 con 0x556750ec3900 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee6effd640 10 client.15486 handle_command_reply: tid=0 2024-02-17T12:49:44.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee98765740 1 client.15486 shutdown 2024-02-17T12:49:44.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee98765740 1 -- 172.21.15.18:0/1422498869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 msgr2=0x556750eb7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee98765740 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 0x556750eb7c20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fee80009900 tx=0x7fee80004270 comp rx=0 tx=0).stop 2024-02-17T12:49:44.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee98765740 1 -- 172.21.15.18:0/1422498869 shutdown_connections 2024-02-17T12:49:44.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee98765740 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556750ec3900 0x556750ec5d20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee98765740 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eaf050 0x556750ebec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee98765740 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 0x556750eb7c20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee98765740 1 --2- 172.21.15.18:0/1422498869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750ead4c0 0x556750eb76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee6effd640 0 client.15486 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:44.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee98765740 1 -- 172.21.15.18:0/1422498869 >> 172.21.15.18:0/1422498869 conn(0x556750ea86a0 msgr2=0x556750ea78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee98765740 1 -- 172.21.15.18:0/1422498869 shutdown_connections 2024-02-17T12:49:44.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee98765740 1 -- 172.21.15.18:0/1422498869 wait complete. 2024-02-17T12:49:44.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.490+0000 7fee98765740 20 client.15486 trim_cache size 0 max 16384 2024-02-17T12:49:44.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.505+0000 7fee98765740 1 Processor -- start 2024-02-17T12:49:44.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.505+0000 7fee98765740 1 -- start start 2024-02-17T12:49:44.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee98765740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750ead4c0 0x556750eca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee98765740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 0x556750ebec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee98765740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 0x556750eb22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee98765740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556750eaeed0 con 0x556750eae690 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee98765740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556750ec0150 con 0x556750eaf050 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee98765740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556750eaf430 con 0x556750ead4c0 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee8f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 0x556750ebec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee8f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 0x556750ebec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34666/0 (socket says 172.21.15.18:34666) 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee8f7fe640 1 -- 172.21.15.18:0/1501484989 learned_addr learned my addr 172.21.15.18:0/1501484989 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee8effd640 1 --2- 172.21.15.18:0/1501484989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750ead4c0 0x556750eca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee8effd640 1 --2- 172.21.15.18:0/1501484989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 0x556750eb22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee8f7fe640 1 -- 172.21.15.18:0/1501484989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750ead4c0 msgr2=0x556750eca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/1501484989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750ead4c0 0x556750eca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee8f7fe640 1 -- 172.21.15.18:0/1501484989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 msgr2=0x556750eb22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/1501484989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 0x556750eb22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee8f7fe640 1 -- 172.21.15.18:0/1501484989 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556750db0c00 con 0x556750eae690 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee8effd640 1 --2- 172.21.15.18:0/1501484989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 0x556750eb22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/1501484989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 0x556750ebec20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fee80005ab0 tx=0x7fee80009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf3e33c01c91a52 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee757fa640 1 -- 172.21.15.18:0/1501484989 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80059070 con 0x556750eae690 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee757fa640 1 -- 172.21.15.18:0/1501484989 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee80043070 con 0x556750eae690 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.506+0000 7fee757fa640 1 -- 172.21.15.18:0/1501484989 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8004ab50 con 0x556750eae690 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.507+0000 7fee98765740 1 -- 172.21.15.18:0/1501484989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 msgr2=0x556750ebec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.507+0000 7fee98765740 1 --2- 172.21.15.18:0/1501484989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 0x556750ebec20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fee80005ab0 tx=0x7fee80009800 comp rx=0 tx=0).stop 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.507+0000 7fee98765740 1 -- 172.21.15.18:0/1501484989 shutdown_connections 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.507+0000 7fee98765740 1 --2- 172.21.15.18:0/1501484989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 0x556750eb22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.507+0000 7fee98765740 1 --2- 172.21.15.18:0/1501484989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750eae690 0x556750ebec20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.507+0000 7fee98765740 1 --2- 172.21.15.18:0/1501484989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750ead4c0 0x556750eca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.507+0000 7fee98765740 1 -- 172.21.15.18:0/1501484989 >> 172.21.15.18:0/1501484989 conn(0x556750ea86a0 msgr2=0x556750eb6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.507+0000 7fee98765740 1 -- 172.21.15.18:0/1501484989 shutdown_connections 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.507+0000 7fee98765740 1 -- 172.21.15.18:0/1501484989 wait complete. 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.507+0000 7fee98765740 1 -- start start 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee98765740 1 Processor -- start 2024-02-17T12:49:44.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee98765740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 0x556750ebe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee98765740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eae690 0x556750ebb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee98765740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 0x556750ebb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee98765740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556750ebbed0 con 0x556750ead4c0 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee98765740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556750ebc040 con 0x556750eaf050 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee98765740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556750f19e10 con 0x556750eae690 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee98765740 10 client.0 _ll_register_callbacks cb 0x7fee84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee8effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 0x556750ebe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee8effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 0x556750ebe8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34680/0 (socket says 172.21.15.18:34680) 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee8effd640 1 -- 172.21.15.18:0/1277996868 learned_addr learned my addr 172.21.15.18:0/1277996868 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eae690 0x556750ebb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee8effd640 1 -- 172.21.15.18:0/1277996868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eae690 msgr2=0x556750ebb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee8effd640 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eae690 0x556750ebb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee8effd640 1 -- 172.21.15.18:0/1277996868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 msgr2=0x556750ebb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee8effd640 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 0x556750ebb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee8effd640 1 -- 172.21.15.18:0/1277996868 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556750e14320 con 0x556750ead4c0 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eae690 0x556750ebb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee8effd640 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 0x556750ebe8d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fee84013620 tx=0x7fee84005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670f89333db473b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.508+0000 7fee6effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee6effd640 1 -- 172.21.15.18:0/1277996868 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84003800 con 0x556750ead4c0 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee98765740 1 -- 172.21.15.18:0/1277996868 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556750e3b900 con 0x556750ead4c0 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee6effd640 1 -- 172.21.15.18:0/1277996868 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fee84003960 con 0x556750ead4c0 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee6effd640 1 -- 172.21.15.18:0/1277996868 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee84003c90 con 0x556750ead4c0 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee98765740 10 client.15492 fetch_fsmap learned FSMap version 26 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee98765740 1 -- 172.21.15.18:0/1277996868 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556750dd6910 con 0x556750ead4c0 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee6effd640 1 -- 172.21.15.18:0/1277996868 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee8403e020 con 0x556750ead4c0 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee6effd640 1 -- 172.21.15.18:0/1277996868 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fee8400fa30 con 0x556750ead4c0 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee98765740 10 client.15492 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee98765740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:49:44.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee98765740 20 client.15492 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:44.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee98765740 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556750f1b7f0 0x556750f1dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee98765740 4 client.15492 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:49:44.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.509+0000 7fee98765740 1 -- 172.21.15.18:0/1277996868 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556750b15860 con 0x556750f1b7f0 2024-02-17T12:49:44.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.510+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556750f1b7f0 0x556750f1dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.510+0000 7fee8f7fe640 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556750f1b7f0 0x556750f1dc10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51a8f0dd0e18113e server_cookie=78c34803d64ea146 in_seq=0 out_seq=0 2024-02-17T12:49:44.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.510+0000 7fee6effd640 10 client.15492 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:44.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee6effd640 1 -- 172.21.15.18:0/1277996868 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556750b15860 con 0x556750f1b7f0 2024-02-17T12:49:44.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee6effd640 10 client.15492 handle_command_reply: tid=0 2024-02-17T12:49:44.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee98765740 1 client.15492 shutdown 2024-02-17T12:49:44.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee98765740 1 -- 172.21.15.18:0/1277996868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 msgr2=0x556750ebe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee98765740 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 0x556750ebe8d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fee84013620 tx=0x7fee84005690 comp rx=0 tx=0).stop 2024-02-17T12:49:44.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee98765740 1 -- 172.21.15.18:0/1277996868 shutdown_connections 2024-02-17T12:49:44.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee98765740 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556750f1b7f0 0x556750f1dc10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee6effd640 0 client.15492 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:44.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee98765740 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556750eaf050 0x556750ebb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee98765740 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556750eae690 0x556750ebb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee98765740 1 --2- 172.21.15.18:0/1277996868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556750ead4c0 0x556750ebe8d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee98765740 1 -- 172.21.15.18:0/1277996868 >> 172.21.15.18:0/1277996868 conn(0x556750ea86a0 msgr2=0x556750eb18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.511+0000 7fee98765740 1 -- 172.21.15.18:0/1277996868 shutdown_connections 2024-02-17T12:49:44.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee98765740 1 -- 172.21.15.18:0/1277996868 wait complete. 2024-02-17T12:49:44.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee98765740 20 client.15492 trim_cache size 0 max 16384 2024-02-17T12:49:44.510 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:49:44.510 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:49:44.510 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "5c252577-2ec5-4d35-b972-e2024489aae4", 2024-02-17T12:49:44.510 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:49:44.510 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:49:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee95b8c640 1 -- 172.21.15.18:0/1313894000 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fee780a2140 msgr2=0x7fee780a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee95b8c640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fee780a2140 0x7fee780a4600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fee8809abc0 tx=0x7fee8801b6d0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee95b8c640 1 -- 172.21.15.18:0/1313894000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee90101c30 msgr2=0x7fee90102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee95b8c640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee90101c30 0x7fee90102ce0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fee8800eb40 tx=0x7fee8800c2d0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee95b8c640 1 -- 172.21.15.18:0/1313894000 shutdown_connections 2024-02-17T12:49:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee95b8c640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fee780a2140 0x7fee780a4600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee95b8c640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee90101c30 0x7fee90102ce0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee95b8c640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee90101270 0x7fee901027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee95b8c640 1 --2- 172.21.15.18:0/1313894000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee90100080 0x7fee90104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee95b8c640 1 -- 172.21.15.18:0/1313894000 >> 172.21.15.18:0/1313894000 conn(0x7fee900fb610 msgr2=0x7fee900682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee95b8c640 1 -- 172.21.15.18:0/1313894000 shutdown_connections 2024-02-17T12:49:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.512+0000 7fee95b8c640 1 -- 172.21.15.18:0/1313894000 wait complete. 2024-02-17T12:49:44.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:49:44.515 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:49:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.628+0000 7fc5549c7640 1 -- 172.21.15.18:0/4014023788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc550100080 msgr2=0x7fc5501004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.628+0000 7fc5549c7640 1 --2- 172.21.15.18:0/4014023788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc550100080 0x7fc5501004a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc538009900 tx=0x7fc53802bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 -- 172.21.15.18:0/4014023788 shutdown_connections 2024-02-17T12:49:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 --2- 172.21.15.18:0/4014023788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc550101c30 0x7fc5501084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 --2- 172.21.15.18:0/4014023788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc550101270 0x7fc5501016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 --2- 172.21.15.18:0/4014023788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc550100080 0x7fc5501004a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 -- 172.21.15.18:0/4014023788 >> 172.21.15.18:0/4014023788 conn(0x7fc5500fb5f0 msgr2=0x7fc5500fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 -- 172.21.15.18:0/4014023788 shutdown_connections 2024-02-17T12:49:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 -- 172.21.15.18:0/4014023788 wait complete. 2024-02-17T12:49:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 Processor -- start 2024-02-17T12:49:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 -- start start 2024-02-17T12:49:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc550100080 0x7fc550199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc550101270 0x7fc5501999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc550101c30 0x7fc5501a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5501a0f70 con 0x7fc550101c30 2024-02-17T12:49:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5501a10e0 con 0x7fc550100080 2024-02-17T12:49:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.629+0000 7fc5549c7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5501a12a0 con 0x7fc550101270 2024-02-17T12:49:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc54ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc550101c30 0x7fc5501a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc54ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc550101c30 0x7fc5501a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34698/0 (socket says 172.21.15.18:34698) 2024-02-17T12:49:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc54ed9d640 1 -- 172.21.15.18:0/2831691707 learned_addr learned my addr 172.21.15.18:0/2831691707 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc54e59c640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc550100080 0x7fc550199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc54dd9b640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc550101270 0x7fc5501999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc54ed9d640 1 -- 172.21.15.18:0/2831691707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc550101270 msgr2=0x7fc5501999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc54ed9d640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc550101270 0x7fc5501999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc54ed9d640 1 -- 172.21.15.18:0/2831691707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc550100080 msgr2=0x7fc550199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc54ed9d640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc550100080 0x7fc550199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc54ed9d640 1 -- 172.21.15.18:0/2831691707 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc538009560 con 0x7fc550101c30 2024-02-17T12:49:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc54ed9d640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc550101c30 0x7fc5501a0a30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc548009fd0 tx=0x7fc54800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc5477fe640 1 -- 172.21.15.18:0/2831691707 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54800bea0 con 0x7fc550101c30 2024-02-17T12:49:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc5477fe640 1 -- 172.21.15.18:0/2831691707 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc548009940 con 0x7fc550101c30 2024-02-17T12:49:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc5477fe640 1 -- 172.21.15.18:0/2831691707 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480106c0 con 0x7fc550101c30 2024-02-17T12:49:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc5549c7640 1 -- 172.21.15.18:0/2831691707 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5501a1580 con 0x7fc550101c30 2024-02-17T12:49:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.630+0000 7fc5549c7640 1 -- 172.21.15.18:0/2831691707 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5501a1a40 con 0x7fc550101c30 2024-02-17T12:49:44.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.633+0000 7fc5477fe640 1 -- 172.21.15.18:0/2831691707 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc548004240 con 0x7fc550101c30 2024-02-17T12:49:44.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.633+0000 7fc5477fe640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc53c0a20c0 0x7fc53c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.633+0000 7fc54e59c640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc53c0a20c0 0x7fc53c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.633+0000 7fc5477fe640 1 -- 172.21.15.18:0/2831691707 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc5480cd980 con 0x7fc550101c30 2024-02-17T12:49:44.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.634+0000 7fc54e59c640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc53c0a20c0 0x7fc53c0a4580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc538005ab0 tx=0x7fc538005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.634+0000 7fc557596740 1 Processor -- start 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.634+0000 7fc557596740 1 -- start start 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.634+0000 7fc557596740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4a2d0 0x55d42eb4a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.634+0000 7fc557596740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4acc0 0x55d42eb4f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.634+0000 7fc557596740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 0x55d42eb522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.634+0000 7fc557596740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42eaba280 con 0x55d42eb4fe30 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.634+0000 7fc557596740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42eab77b0 con 0x55d42eb4a2d0 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.634+0000 7fc557596740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42eab2f90 con 0x55d42eb4acc0 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.634+0000 7fc54ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 0x55d42eb522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.634+0000 7fc54ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 0x55d42eb522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34706/0 (socket says 172.21.15.18:34706) 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.634+0000 7fc54ed9d640 1 -- 172.21.15.18:0/1667967027 learned_addr learned my addr 172.21.15.18:0/1667967027 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc54dd9b640 1 --2- 172.21.15.18:0/1667967027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4acc0 0x55d42eb4f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc54e59c640 1 --2- 172.21.15.18:0/1667967027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4a2d0 0x55d42eb4a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc54ed9d640 1 -- 172.21.15.18:0/1667967027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4acc0 msgr2=0x55d42eb4f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc54ed9d640 1 --2- 172.21.15.18:0/1667967027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4acc0 0x55d42eb4f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc54ed9d640 1 -- 172.21.15.18:0/1667967027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4a2d0 msgr2=0x55d42eb4a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc54ed9d640 1 --2- 172.21.15.18:0/1667967027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4a2d0 0x55d42eb4a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc54ed9d640 1 -- 172.21.15.18:0/1667967027 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d42ea71570 con 0x55d42eb4fe30 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc54dd9b640 1 --2- 172.21.15.18:0/1667967027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4acc0 0x55d42eb4f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc54e59c640 1 --2- 172.21.15.18:0/1667967027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4a2d0 0x55d42eb4a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc54ed9d640 1 --2- 172.21.15.18:0/1667967027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 0x55d42eb522c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc5480a2950 tx=0x7fc54809a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6662d6c42cb1346 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc544ff9640 1 -- 172.21.15.18:0/1667967027 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54809e830 con 0x55d42eb4fe30 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc544ff9640 1 -- 172.21.15.18:0/1667967027 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5480a2e80 con 0x55d42eb4fe30 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.635+0000 7fc544ff9640 1 -- 172.21.15.18:0/1667967027 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480ba0c0 con 0x55d42eb4fe30 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.636+0000 7fc557596740 1 -- 172.21.15.18:0/1667967027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 msgr2=0x55d42eb522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.636+0000 7fc557596740 1 --2- 172.21.15.18:0/1667967027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 0x55d42eb522c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc5480a2950 tx=0x7fc54809a140 comp rx=0 tx=0).stop 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.636+0000 7fc557596740 1 -- 172.21.15.18:0/1667967027 shutdown_connections 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.636+0000 7fc557596740 1 --2- 172.21.15.18:0/1667967027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 0x55d42eb522c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.636+0000 7fc557596740 1 --2- 172.21.15.18:0/1667967027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4acc0 0x55d42eb4f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.636+0000 7fc557596740 1 --2- 172.21.15.18:0/1667967027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4a2d0 0x55d42eb4a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.636+0000 7fc557596740 1 -- 172.21.15.18:0/1667967027 >> 172.21.15.18:0/1667967027 conn(0x55d42eb457f0 msgr2=0x55d42eb45bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.636+0000 7fc557596740 1 -- 172.21.15.18:0/1667967027 shutdown_connections 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.636+0000 7fc557596740 1 -- 172.21.15.18:0/1667967027 wait complete. 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.636+0000 7fc557596740 1 -- start start 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc557596740 1 Processor -- start 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc557596740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4a2d0 0x55d42eb54210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc557596740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4acc0 0x55d42eb54750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc557596740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4fe30 0x55d42eb5b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc557596740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42e786430 con 0x55d42eb4a2d0 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc557596740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42e742e00 con 0x55d42eb4fe30 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc557596740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42ea37fd0 con 0x55d42eb4acc0 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc557596740 10 client.0 _ll_register_callbacks cb 0x55d42ea8dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc54e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4a2d0 0x55d42eb54210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc54e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4a2d0 0x55d42eb54210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34712/0 (socket says 172.21.15.18:34712) 2024-02-17T12:49:44.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc54e59c640 1 -- 172.21.15.18:0/1245993039 learned_addr learned my addr 172.21.15.18:0/1245993039 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:44.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc54ed9d640 1 --2- 172.21.15.18:0/1245993039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4fe30 0x55d42eb5b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc54e59c640 1 -- 172.21.15.18:0/1245993039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4acc0 msgr2=0x55d42eb54750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc54e59c640 1 --2- 172.21.15.18:0/1245993039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4acc0 0x55d42eb54750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc54e59c640 1 -- 172.21.15.18:0/1245993039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4fe30 msgr2=0x55d42eb5b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc54e59c640 1 --2- 172.21.15.18:0/1245993039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4fe30 0x55d42eb5b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc54e59c640 1 -- 172.21.15.18:0/1245993039 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5501a1a40 con 0x55d42eb4a2d0 2024-02-17T12:49:44.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc54e59c640 1 --2- 172.21.15.18:0/1245993039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4a2d0 0x55d42eb54210 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc538009230 tx=0x7fc538009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ba7d44de03e795 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.637+0000 7fc535ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:44.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc535ffb640 1 -- 172.21.15.18:0/1245993039 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53802cbc0 con 0x55d42eb4a2d0 2024-02-17T12:49:44.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc535ffb640 1 -- 172.21.15.18:0/1245993039 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc538047050 con 0x55d42eb4a2d0 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc557596740 1 -- 172.21.15.18:0/1245993039 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d42e7b27e0 con 0x55d42eb4a2d0 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc535ffb640 1 -- 172.21.15.18:0/1245993039 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc538005000 con 0x55d42eb4a2d0 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc557596740 10 client.15504 fetch_fsmap learned FSMap version 26 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc557596740 1 -- 172.21.15.18:0/1245993039 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d42eab1610 con 0x55d42eb4a2d0 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc535ffb640 1 -- 172.21.15.18:0/1245993039 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d42e7b27e0 con 0x55d42eb4a2d0 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc535ffb640 1 -- 172.21.15.18:0/1245993039 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc538003ca0 con 0x55d42eb4a2d0 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc557596740 10 client.15504 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc557596740 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc557596740 20 client.15504 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc557596740 1 --2- 172.21.15.18:0/1245993039 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d42eb60400 0x55d42eb62840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc557596740 4 client.15504 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.638+0000 7fc557596740 1 -- 172.21.15.18:0/1245993039 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d42e7ef720 con 0x55d42eb60400 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.639+0000 7fc54dd9b640 1 --2- 172.21.15.18:0/1245993039 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d42eb60400 0x55d42eb62840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.639+0000 7fc535ffb640 10 client.15504 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:44.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.639+0000 7fc54dd9b640 1 --2- 172.21.15.18:0/1245993039 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d42eb60400 0x55d42eb62840 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717ff8a3cb3b8813 server_cookie=99d1abc74185277e in_seq=0 out_seq=0 2024-02-17T12:49:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc535ffb640 1 -- 172.21.15.18:0/1245993039 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d42e7ef720 con 0x55d42eb60400 2024-02-17T12:49:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc535ffb640 10 client.15504 handle_command_reply: tid=0 2024-02-17T12:49:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc557596740 1 client.15504 shutdown 2024-02-17T12:49:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc557596740 1 -- 172.21.15.18:0/1245993039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4a2d0 msgr2=0x55d42eb54210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc557596740 1 --2- 172.21.15.18:0/1245993039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4a2d0 0x55d42eb54210 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc538009230 tx=0x7fc538009260 comp rx=0 tx=0).stop 2024-02-17T12:49:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc557596740 1 -- 172.21.15.18:0/1245993039 shutdown_connections 2024-02-17T12:49:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc557596740 1 --2- 172.21.15.18:0/1245993039 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d42eb60400 0x55d42eb62840 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc557596740 1 --2- 172.21.15.18:0/1245993039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4fe30 0x55d42eb5b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc535ffb640 0 client.15504 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc557596740 1 --2- 172.21.15.18:0/1245993039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4acc0 0x55d42eb54750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc557596740 1 --2- 172.21.15.18:0/1245993039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4a2d0 0x55d42eb54210 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc557596740 1 -- 172.21.15.18:0/1245993039 >> 172.21.15.18:0/1245993039 conn(0x55d42eb457f0 msgr2=0x55d42eb44a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc557596740 1 -- 172.21.15.18:0/1245993039 shutdown_connections 2024-02-17T12:49:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc557596740 1 -- 172.21.15.18:0/1245993039 wait complete. 2024-02-17T12:49:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.640+0000 7fc557596740 20 client.15504 trim_cache size 0 max 16384 2024-02-17T12:49:44.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.674+0000 7fc557596740 1 Processor -- start 2024-02-17T12:49:44.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.674+0000 7fc557596740 1 -- start start 2024-02-17T12:49:44.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc557596740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4a2d0 0x55d42eb4bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc557596740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4acc0 0x55d42eb4c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc557596740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 0x55d42eb5b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc557596740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42eb5e980 con 0x55d42eb4fe30 2024-02-17T12:49:44.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc557596740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42e7ef750 con 0x55d42eb4acc0 2024-02-17T12:49:44.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc557596740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42eb4b500 con 0x55d42eb4a2d0 2024-02-17T12:49:44.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc54ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 0x55d42eb5b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc54ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 0x55d42eb5b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34714/0 (socket says 172.21.15.18:34714) 2024-02-17T12:49:44.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc54ed9d640 1 -- 172.21.15.18:0/852822348 learned_addr learned my addr 172.21.15.18:0/852822348 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:44.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc54e59c640 1 --2- 172.21.15.18:0/852822348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4a2d0 0x55d42eb4bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc54dd9b640 1 --2- 172.21.15.18:0/852822348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4acc0 0x55d42eb4c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc54ed9d640 1 -- 172.21.15.18:0/852822348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4a2d0 msgr2=0x55d42eb4bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc54ed9d640 1 --2- 172.21.15.18:0/852822348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4a2d0 0x55d42eb4bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc54ed9d640 1 -- 172.21.15.18:0/852822348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4acc0 msgr2=0x55d42eb4c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc54ed9d640 1 --2- 172.21.15.18:0/852822348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4acc0 0x55d42eb4c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc54ed9d640 1 -- 172.21.15.18:0/852822348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d42ea4dac0 con 0x55d42eb4fe30 2024-02-17T12:49:44.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc54ed9d640 1 --2- 172.21.15.18:0/852822348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 0x55d42eb5b750 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc54809a3e0 tx=0x7fc5480097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0dde2940b4f511 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc537fff640 1 -- 172.21.15.18:0/852822348 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548098780 con 0x55d42eb4fe30 2024-02-17T12:49:44.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc537fff640 1 -- 172.21.15.18:0/852822348 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc54809e830 con 0x55d42eb4fe30 2024-02-17T12:49:44.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.675+0000 7fc537fff640 1 -- 172.21.15.18:0/852822348 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480b3240 con 0x55d42eb4fe30 2024-02-17T12:49:44.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 -- 172.21.15.18:0/852822348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 msgr2=0x55d42eb5b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 --2- 172.21.15.18:0/852822348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 0x55d42eb5b750 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc54809a3e0 tx=0x7fc5480097f0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 -- 172.21.15.18:0/852822348 shutdown_connections 2024-02-17T12:49:44.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 --2- 172.21.15.18:0/852822348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4fe30 0x55d42eb5b750 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 --2- 172.21.15.18:0/852822348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4acc0 0x55d42eb4c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 --2- 172.21.15.18:0/852822348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4a2d0 0x55d42eb4bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 -- 172.21.15.18:0/852822348 >> 172.21.15.18:0/852822348 conn(0x55d42eb457f0 msgr2=0x55d42eb45bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 -- 172.21.15.18:0/852822348 shutdown_connections 2024-02-17T12:49:44.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 -- 172.21.15.18:0/852822348 wait complete. 2024-02-17T12:49:44.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 -- start start 2024-02-17T12:49:44.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 Processor -- start 2024-02-17T12:49:44.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4a2d0 0x55d42eb5afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4acc0 0x55d42eb5b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4fe30 0x55d42eb595e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42eb59b20 con 0x55d42eb4acc0 2024-02-17T12:49:44.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42eb59c90 con 0x55d42eb4fe30 2024-02-17T12:49:44.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42eb59e50 con 0x55d42eb4a2d0 2024-02-17T12:49:44.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.676+0000 7fc557596740 10 client.0 _ll_register_callbacks cb 0x7fc548096340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:49:44.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc54dd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4acc0 0x55d42eb5b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc54dd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4acc0 0x55d42eb5b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34728/0 (socket says 172.21.15.18:34728) 2024-02-17T12:49:44.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc54dd9b640 1 -- 172.21.15.18:0/3129292217 learned_addr learned my addr 172.21.15.18:0/3129292217 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:44.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc54ed9d640 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4fe30 0x55d42eb595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc54e59c640 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4a2d0 0x55d42eb5afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc54dd9b640 1 -- 172.21.15.18:0/3129292217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4a2d0 msgr2=0x55d42eb5afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc54dd9b640 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4a2d0 0x55d42eb5afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc54dd9b640 1 -- 172.21.15.18:0/3129292217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4fe30 msgr2=0x55d42eb595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:49:44.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc54dd9b640 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4fe30 0x55d42eb595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc54dd9b640 1 -- 172.21.15.18:0/3129292217 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d42ea71570 con 0x55d42eb4acc0 2024-02-17T12:49:44.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc54dd9b640 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4acc0 0x55d42eb5b510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55d42eb4b730 tx=0x7fc540018ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8791d70ed16b3ca server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:44.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc535ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:49:44.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc535ffb640 1 -- 172.21.15.18:0/3129292217 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540018910 con 0x55d42eb4acc0 2024-02-17T12:49:44.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc557596740 1 -- 172.21.15.18:0/3129292217 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5480a2e80 con 0x55d42eb4acc0 2024-02-17T12:49:44.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc535ffb640 1 -- 172.21.15.18:0/3129292217 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc540012450 con 0x55d42eb4acc0 2024-02-17T12:49:44.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc535ffb640 1 -- 172.21.15.18:0/3129292217 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc54000be60 con 0x55d42eb4acc0 2024-02-17T12:49:44.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc557596740 10 client.15510 fetch_fsmap learned FSMap version 26 2024-02-17T12:49:44.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.677+0000 7fc557596740 1 -- 172.21.15.18:0/3129292217 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d42ea736a0 con 0x55d42eb4acc0 2024-02-17T12:49:44.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.678+0000 7fc535ffb640 1 -- 172.21.15.18:0/3129292217 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5480a2e80 con 0x55d42eb4acc0 2024-02-17T12:49:44.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.678+0000 7fc535ffb640 1 -- 172.21.15.18:0/3129292217 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc54002a050 con 0x55d42eb4acc0 2024-02-17T12:49:44.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.678+0000 7fc557596740 10 client.15510 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:49:44.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.678+0000 7fc557596740 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:49:44.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.678+0000 7fc557596740 20 client.15510 populate_metadata read hostname 'smithi018' 2024-02-17T12:49:44.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.678+0000 7fc557596740 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d42eb60400 0x55d42ebb9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:44.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.678+0000 7fc557596740 4 client.15510 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:49:44.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.678+0000 7fc557596740 1 -- 172.21.15.18:0/3129292217 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d42e83e890 con 0x55d42eb60400 2024-02-17T12:49:44.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.678+0000 7fc54e59c640 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d42eb60400 0x55d42ebb9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:44.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.678+0000 7fc54e59c640 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d42eb60400 0x55d42ebb9d40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2262f0be92035ee server_cookie=e47dfd5f051696f9 in_seq=0 out_seq=0 2024-02-17T12:49:44.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.678+0000 7fc535ffb640 10 client.15510 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:44.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc535ffb640 1 -- 172.21.15.18:0/3129292217 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d42e83e890 con 0x55d42eb60400 2024-02-17T12:49:44.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc535ffb640 10 client.15510 handle_command_reply: tid=0 2024-02-17T12:49:44.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc557596740 1 client.15510 shutdown 2024-02-17T12:49:44.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc557596740 1 -- 172.21.15.18:0/3129292217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4acc0 msgr2=0x55d42eb5b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc557596740 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4acc0 0x55d42eb5b510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55d42eb4b730 tx=0x7fc540018ed0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc557596740 1 -- 172.21.15.18:0/3129292217 shutdown_connections 2024-02-17T12:49:44.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc557596740 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d42eb60400 0x55d42ebb9d40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc557596740 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d42eb4fe30 0x55d42eb595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc535ffb640 0 client.15510 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:49:44.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc557596740 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d42eb4acc0 0x55d42eb5b510 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc557596740 1 --2- 172.21.15.18:0/3129292217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d42eb4a2d0 0x55d42eb5afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc557596740 1 -- 172.21.15.18:0/3129292217 >> 172.21.15.18:0/3129292217 conn(0x55d42eb457f0 msgr2=0x55d42eb4bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc557596740 1 -- 172.21.15.18:0/3129292217 shutdown_connections 2024-02-17T12:49:44.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc557596740 1 -- 172.21.15.18:0/3129292217 wait complete. 2024-02-17T12:49:44.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc557596740 20 client.15510 trim_cache size 0 max 16384 2024-02-17T12:49:44.687 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:49:44.687 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:49:44.688 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:49:44.688 INFO:teuthology.orchestra.run.smithi018.stdout: "5c252577-2ec5-4d35-b972-e2024489aae4": { 2024-02-17T12:49:44.688 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:49:44.688 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "5c252577-2ec5-4d35-b972-e2024489aae4", 2024-02-17T12:49:44.688 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:49:44.688 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:49:44.688 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:49:44.688 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:49:44.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc5549c7640 1 -- 172.21.15.18:0/2831691707 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc53c0a20c0 msgr2=0x7fc53c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc5549c7640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc53c0a20c0 0x7fc53c0a4580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc538005ab0 tx=0x7fc538005fd0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc5549c7640 1 -- 172.21.15.18:0/2831691707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc550101c30 msgr2=0x7fc5501a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:44.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc5549c7640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc550101c30 0x7fc5501a0a30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc548009fd0 tx=0x7fc54800b260 comp rx=0 tx=0).stop 2024-02-17T12:49:44.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc5549c7640 1 -- 172.21.15.18:0/2831691707 shutdown_connections 2024-02-17T12:49:44.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc5549c7640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fc53c0a20c0 0x7fc53c0a4580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc5549c7640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc550101c30 0x7fc5501a0a30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc5549c7640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc550101270 0x7fc5501999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc5549c7640 1 --2- 172.21.15.18:0/2831691707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc550100080 0x7fc550199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:44.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc5549c7640 1 -- 172.21.15.18:0/2831691707 >> 172.21.15.18:0/2831691707 conn(0x7fc5500fb5f0 msgr2=0x7fc550108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:44.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc5549c7640 1 -- 172.21.15.18:0/2831691707 shutdown_connections 2024-02-17T12:49:44.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.679+0000 7fc5549c7640 1 -- 172.21.15.18:0/2831691707 wait complete. 2024-02-17T12:49:44.691 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:44 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3793532704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:44.691 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:44 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1596506406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:44.691 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:44 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3552014445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:44.693 INFO:tasks.cephfs.filesystem:scrub status for tag:5c252577-2ec5-4d35-b972-e2024489aae4 - {'path': '/', 'tag': '5c252577-2ec5-4d35-b972-e2024489aae4', 'options': 'recursive,force'} 2024-02-17T12:49:44.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:44.746+0000 7fceb83f4640 1 -- 172.21.15.18:0/726569984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fceb002cee0 con 0x7fceb01000a0 2024-02-17T12:49:44.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:44 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3793532704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:44.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:44 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1596506406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:44.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:44 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3552014445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:44 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3793532704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:49:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:44 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1596506406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:49:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:44 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3552014445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:49:45.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.550+0000 7fceaf7fe640 1 -- 172.21.15.18:0/726569984 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7fcea809a940 con 0x7fceb01000a0 2024-02-17T12:49:45.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.554+0000 7fceb83f4640 1 -- 172.21.15.18:0/726569984 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fce980a2160 msgr2=0x7fce980a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:45.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.554+0000 7fceb83f4640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fce980a2160 0x7fce980a4620 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fceb0196440 tx=0x7fcea400d040 comp rx=0 tx=0).stop 2024-02-17T12:49:45.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.554+0000 7fceb83f4640 1 -- 172.21.15.18:0/726569984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fceb01000a0 msgr2=0x7fceb0194f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:45.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.554+0000 7fceb83f4640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fceb01000a0 0x7fceb0194f30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009fd0 tx=0x7fcea800c270 comp rx=0 tx=0).stop 2024-02-17T12:49:45.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.554+0000 7fceb83f4640 1 -- 172.21.15.18:0/726569984 shutdown_connections 2024-02-17T12:49:45.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.554+0000 7fceb83f4640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fce980a2160 0x7fce980a4620 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.554+0000 7fceb83f4640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fceb0101c50 0x7fceb019c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.554+0000 7fceb83f4640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fceb0101290 0x7fceb0195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.554+0000 7fceb83f4640 1 --2- 172.21.15.18:0/726569984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fceb01000a0 0x7fceb0194f30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.554+0000 7fceb83f4640 1 -- 172.21.15.18:0/726569984 >> 172.21.15.18:0/726569984 conn(0x7fceb00fb610 msgr2=0x7fceb0104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:45.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.554+0000 7fceb83f4640 1 -- 172.21.15.18:0/726569984 shutdown_connections 2024-02-17T12:49:45.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.555+0000 7fceb83f4640 1 -- 172.21.15.18:0/726569984 wait complete. 2024-02-17T12:49:45.567 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-02-17T12:49:45.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.687+0000 7f47cae40640 1 -- 172.21.15.18:0/1250352065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47c41000a0 msgr2=0x7f47c41004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:45.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.687+0000 7f47cae40640 1 --2- 172.21.15.18:0/1250352065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47c41000a0 0x7f47c41004c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009900 tx=0x7f47b802bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:45.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.687+0000 7f47cae40640 1 -- 172.21.15.18:0/1250352065 shutdown_connections 2024-02-17T12:49:45.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.687+0000 7f47cae40640 1 --2- 172.21.15.18:0/1250352065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47c4101c50 0x7f47c41084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.687+0000 7f47cae40640 1 --2- 172.21.15.18:0/1250352065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c4101290 0x7f47c4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.687+0000 7f47cae40640 1 --2- 172.21.15.18:0/1250352065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47c41000a0 0x7f47c41004c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.687+0000 7f47cae40640 1 -- 172.21.15.18:0/1250352065 >> 172.21.15.18:0/1250352065 conn(0x7f47c40fb610 msgr2=0x7f47c40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:45.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.687+0000 7f47cae40640 1 -- 172.21.15.18:0/1250352065 shutdown_connections 2024-02-17T12:49:45.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.687+0000 7f47cae40640 1 -- 172.21.15.18:0/1250352065 wait complete. 2024-02-17T12:49:45.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47cae40640 1 Processor -- start 2024-02-17T12:49:45.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47cae40640 1 -- start start 2024-02-17T12:49:45.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47cae40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c41000a0 0x7f47c4069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:45.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47cae40640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47c4101290 0x7f47c419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:45.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47cae40640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47c4101c50 0x7f47c41a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:45.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47cae40640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c41a0f10 con 0x7f47c4101c50 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47cae40640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c41a1080 con 0x7f47c41000a0 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47cae40640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c41a1240 con 0x7f47c4101290 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47c93dd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47c4101c50 0x7f47c41a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47c93dd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47c4101c50 0x7f47c41a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34778/0 (socket says 172.21.15.18:34778) 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47c93dd640 1 -- 172.21.15.18:0/1283057700 learned_addr learned my addr 172.21.15.18:0/1283057700 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47c3fff640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47c4101290 0x7f47c419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.688+0000 7f47c8bdc640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c41000a0 0x7f47c4069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.689+0000 7f47c8bdc640 1 -- 172.21.15.18:0/1283057700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47c4101290 msgr2=0x7f47c419e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.689+0000 7f47c8bdc640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47c4101290 0x7f47c419e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.689+0000 7f47c8bdc640 1 -- 172.21.15.18:0/1283057700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47c4101c50 msgr2=0x7f47c41a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.689+0000 7f47c8bdc640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47c4101c50 0x7f47c41a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.689+0000 7f47c8bdc640 1 -- 172.21.15.18:0/1283057700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b8009560 con 0x7f47c41000a0 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.689+0000 7f47c93dd640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47c4101c50 0x7f47c41a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:49:45.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.689+0000 7f47c8bdc640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c41000a0 0x7f47c4069370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f47b8036040 tx=0x7f47b8005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:45.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.689+0000 7f47c1ffb640 1 -- 172.21.15.18:0/1283057700 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80043d0 con 0x7f47c41000a0 2024-02-17T12:49:45.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.689+0000 7f47c1ffb640 1 -- 172.21.15.18:0/1283057700 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47b80052d0 con 0x7f47c41000a0 2024-02-17T12:49:45.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.689+0000 7f47cae40640 1 -- 172.21.15.18:0/1283057700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c41a1460 con 0x7f47c41000a0 2024-02-17T12:49:45.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.689+0000 7f47c1ffb640 1 -- 172.21.15.18:0/1283057700 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8003cd0 con 0x7f47c41000a0 2024-02-17T12:49:45.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.689+0000 7f47cae40640 1 -- 172.21.15.18:0/1283057700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c41a1940 con 0x7f47c41000a0 2024-02-17T12:49:45.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.690+0000 7f47cae40640 1 -- 172.21.15.18:0/1283057700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c4107360 con 0x7f47c41000a0 2024-02-17T12:49:45.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.691+0000 7f47c1ffb640 1 -- 172.21.15.18:0/1283057700 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 136785+0+0 (secure 0 0 0) 0x7f47b8005440 con 0x7f47c41000a0 2024-02-17T12:49:45.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.692+0000 7f47c1ffb640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f47b40a21e0 0x7f47b40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:45.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.692+0000 7f47c3fff640 1 -- 172.21.15.18:0/1283057700 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f47b40a21e0 msgr2=0x7f47b40a46a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3026633658 2024-02-17T12:49:45.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.692+0000 7f47c1ffb640 1 -- 172.21.15.18:0/1283057700 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f47b80effe0 con 0x7f47c41000a0 2024-02-17T12:49:45.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.692+0000 7f47c3fff640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f47b40a21e0 0x7f47b40a46a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T12:49:45.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.695+0000 7f47c1ffb640 1 -- 172.21.15.18:0/1283057700 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f47b80b8fa0 con 0x7f47c41000a0 2024-02-17T12:49:45.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:45 smithi119 ceph-mon[36188]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-17T12:49:45.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:45 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:49:45.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:45 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:49:45.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:45 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:49:45.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:45 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:49:45.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:45 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/726569984' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-17T12:49:45.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:45 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:49:45.844 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:45 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: ignoring --setuser ceph since I am not root 2024-02-17T12:49:45.844 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:45 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: ignoring --setgroup ceph since I am not root 2024-02-17T12:49:45.844 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:45 smithi119 ceph-mgr[37572]: -- 172.21.15.119:0/3527754379 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555843d121a0 con 0x555843ce9000 2024-02-17T12:49:45.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.880+0000 7f47cae40640 1 -- 172.21.15.18:0/1283057700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f47c402ce40 con 0x7f47c41000a0 2024-02-17T12:49:45.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.885+0000 7f47c1ffb640 1 -- 172.21.15.18:0/1283057700 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f47b80bc660 con 0x7f47c41000a0 2024-02-17T12:49:45.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.888+0000 7f47cae40640 1 -- 172.21.15.18:0/1283057700 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f47b40a21e0 msgr2=0x7f47b40a46a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:49:45.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.888+0000 7f47cae40640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f47b40a21e0 0x7f47b40a46a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.888+0000 7f47cae40640 1 -- 172.21.15.18:0/1283057700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c41000a0 msgr2=0x7f47c4069370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:45.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.888+0000 7f47cae40640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c41000a0 0x7f47c4069370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f47b8036040 tx=0x7f47b8005e50 comp rx=0 tx=0).stop 2024-02-17T12:49:45.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.888+0000 7f47cae40640 1 -- 172.21.15.18:0/1283057700 shutdown_connections 2024-02-17T12:49:45.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.888+0000 7f47cae40640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7f47b40a21e0 0x7f47b40a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.888+0000 7f47cae40640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47c4101c50 0x7f47c41a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.888+0000 7f47cae40640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47c4101290 0x7f47c419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.888+0000 7f47cae40640 1 --2- 172.21.15.18:0/1283057700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c41000a0 0x7f47c4069370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.888+0000 7f47cae40640 1 -- 172.21.15.18:0/1283057700 >> 172.21.15.18:0/1283057700 conn(0x7f47c40fb610 msgr2=0x7f47c4104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:45.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.888+0000 7f47cae40640 1 -- 172.21.15.18:0/1283057700 shutdown_connections 2024-02-17T12:49:45.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:45.888+0000 7f47cae40640 1 -- 172.21.15.18:0/1283057700 wait complete. 2024-02-17T12:49:45.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:45 smithi018 ceph-mon[32625]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-17T12:49:45.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:45 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:49:45.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:45 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:49:45.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:45 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:49:45.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:45 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:49:45.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:45 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/726569984' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-17T12:49:45.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:45 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:49:45.891 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:45 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: ignoring --setuser ceph since I am not root 2024-02-17T12:49:45.891 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:45 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: ignoring --setgroup ceph since I am not root 2024-02-17T12:49:45.892 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-02-17T12:49:45.951 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:45 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: ignoring --setuser ceph since I am not root 2024-02-17T12:49:45.951 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:45 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: ignoring --setgroup ceph since I am not root 2024-02-17T12:49:45.951 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:45 smithi146 ceph-mgr[38310]: -- 172.21.15.146:0/1090240107 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f5556461a0 con 0x55f55561c800 2024-02-17T12:49:45.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:45 smithi146 ceph-mon[36454]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-17T12:49:45.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:45 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:49:45.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:45 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:49:45.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:45 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:49:45.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:45 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:49:45.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:45 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/726569984' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-17T12:49:45.952 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:45 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:49:45.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.005+0000 7ff60bf8a640 1 -- 172.21.15.18:0/1711465295 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6041000a0 msgr2=0x7ff6041004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:45.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.005+0000 7ff60bf8a640 1 --2- 172.21.15.18:0/1711465295 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6041000a0 0x7ff6041004c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009900 tx=0x7ff5f402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:45.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.005+0000 7ff60bf8a640 1 -- 172.21.15.18:0/1711465295 shutdown_connections 2024-02-17T12:49:45.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.005+0000 7ff60bf8a640 1 --2- 172.21.15.18:0/1711465295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff604101c50 0x7ff6041084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.005+0000 7ff60bf8a640 1 --2- 172.21.15.18:0/1711465295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff604101290 0x7ff604101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.005+0000 7ff60bf8a640 1 --2- 172.21.15.18:0/1711465295 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6041000a0 0x7ff6041004c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:45.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.005+0000 7ff60bf8a640 1 -- 172.21.15.18:0/1711465295 >> 172.21.15.18:0/1711465295 conn(0x7ff6040fb610 msgr2=0x7ff6040fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.005+0000 7ff60bf8a640 1 -- 172.21.15.18:0/1711465295 shutdown_connections 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.005+0000 7ff60bf8a640 1 -- 172.21.15.18:0/1711465295 wait complete. 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff60bf8a640 1 Processor -- start 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff60bf8a640 1 -- start start 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff60bf8a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6041000a0 0x7ff6041971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff60bf8a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff604101290 0x7ff604197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff60bf8a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff604101c50 0x7ff60419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff60bf8a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60419ecc0 con 0x7ff604101290 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff60bf8a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60419ee30 con 0x7ff6041000a0 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff60bf8a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60419eff0 con 0x7ff604101c50 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff609525640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff604101290 0x7ff604197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff609d26640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6041000a0 0x7ff6041971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff609525640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff604101290 0x7ff604197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34796/0 (socket says 172.21.15.18:34796) 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff609525640 1 -- 172.21.15.18:0/783520869 learned_addr learned my addr 172.21.15.18:0/783520869 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff60a527640 1 --2- 172.21.15.18:0/783520869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff604101c50 0x7ff60419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff609525640 1 -- 172.21.15.18:0/783520869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff604101c50 msgr2=0x7ff60419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff609525640 1 --2- 172.21.15.18:0/783520869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff604101c50 0x7ff60419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff609525640 1 -- 172.21.15.18:0/783520869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6041000a0 msgr2=0x7ff6041971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff609525640 1 --2- 172.21.15.18:0/783520869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6041000a0 0x7ff6041971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff609525640 1 -- 172.21.15.18:0/783520869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4009560 con 0x7ff604101290 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.006+0000 7ff609525640 1 --2- 172.21.15.18:0/783520869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff604101290 0x7ff604197710 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff60000e9f0 tx=0x7ff60000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.007+0000 7ff5faffd640 1 -- 172.21.15.18:0/783520869 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600009720 con 0x7ff604101290 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.007+0000 7ff60bf8a640 1 -- 172.21.15.18:0/783520869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60419f2d0 con 0x7ff604101290 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.007+0000 7ff5faffd640 1 -- 172.21.15.18:0/783520869 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff60000cd80 con 0x7ff604101290 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.007+0000 7ff5faffd640 1 -- 172.21.15.18:0/783520869 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600010690 con 0x7ff604101290 2024-02-17T12:49:46.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.007+0000 7ff60bf8a640 1 -- 172.21.15.18:0/783520869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60419f790 con 0x7ff604101290 2024-02-17T12:49:46.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.008+0000 7ff60bf8a640 1 -- 172.21.15.18:0/783520869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff604107360 con 0x7ff604101290 2024-02-17T12:49:46.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.009+0000 7ff5faffd640 1 -- 172.21.15.18:0/783520869 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 136785+0+0 (secure 0 0 0) 0x7ff6000046c0 con 0x7ff604101290 2024-02-17T12:49:46.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.009+0000 7ff5faffd640 1 --2- 172.21.15.18:0/783520869 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff5f00a21b0 0x7ff5f00a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:46.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.009+0000 7ff609d26640 1 -- 172.21.15.18:0/783520869 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff5f00a21b0 msgr2=0x7ff5f00a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3026633658 2024-02-17T12:49:46.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.010+0000 7ff5faffd640 1 -- 172.21.15.18:0/783520869 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff600014070 con 0x7ff604101290 2024-02-17T12:49:46.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.010+0000 7ff609d26640 1 --2- 172.21.15.18:0/783520869 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff5f00a21b0 0x7ff5f00a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T12:49:46.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.014+0000 7ff5faffd640 1 -- 172.21.15.18:0/783520869 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff600097050 con 0x7ff604101290 2024-02-17T12:49:46.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.202+0000 7ff60bf8a640 1 -- 172.21.15.18:0/783520869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7ff60410cbb0 con 0x7ff604101290 2024-02-17T12:49:46.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.206+0000 7ff5faffd640 1 -- 172.21.15.18:0/783520869 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff60009a710 con 0x7ff604101290 2024-02-17T12:49:46.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.208+0000 7ff60bf8a640 1 -- 172.21.15.18:0/783520869 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff5f00a21b0 msgr2=0x7ff5f00a4670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:49:46.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.208+0000 7ff60bf8a640 1 --2- 172.21.15.18:0/783520869 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff5f00a21b0 0x7ff5f00a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.208+0000 7ff60bf8a640 1 -- 172.21.15.18:0/783520869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff604101290 msgr2=0x7ff604197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:46.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.208+0000 7ff60bf8a640 1 --2- 172.21.15.18:0/783520869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff604101290 0x7ff604197710 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff60000e9f0 tx=0x7ff60000edf0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.208+0000 7ff60bf8a640 1 -- 172.21.15.18:0/783520869 shutdown_connections 2024-02-17T12:49:46.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.208+0000 7ff60bf8a640 1 --2- 172.21.15.18:0/783520869 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7ff5f00a21b0 0x7ff5f00a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.208+0000 7ff60bf8a640 1 --2- 172.21.15.18:0/783520869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff604101c50 0x7ff60419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.208+0000 7ff60bf8a640 1 --2- 172.21.15.18:0/783520869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff604101290 0x7ff604197710 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.208+0000 7ff60bf8a640 1 --2- 172.21.15.18:0/783520869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6041000a0 0x7ff6041971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.208+0000 7ff60bf8a640 1 -- 172.21.15.18:0/783520869 >> 172.21.15.18:0/783520869 conn(0x7ff6040fb610 msgr2=0x7ff604104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:46.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.208+0000 7ff60bf8a640 1 -- 172.21.15.18:0/783520869 shutdown_connections 2024-02-17T12:49:46.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.208+0000 7ff60bf8a640 1 -- 172.21.15.18:0/783520869 wait complete. 2024-02-17T12:49:46.212 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-02-17T12:49:46.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.327+0000 7fd5b1302640 1 -- 172.21.15.18:0/414970419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5ac101ca0 msgr2=0x7fd5ac108530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:46.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.327+0000 7fd5b1302640 1 --2- 172.21.15.18:0/414970419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5ac101ca0 0x7fd5ac108530 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4009ca0 tx=0x7fd5a402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.327+0000 7fd5b1302640 1 -- 172.21.15.18:0/414970419 shutdown_connections 2024-02-17T12:49:46.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.327+0000 7fd5b1302640 1 --2- 172.21.15.18:0/414970419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5ac101ca0 0x7fd5ac108530 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.327+0000 7fd5b1302640 1 --2- 172.21.15.18:0/414970419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac1012e0 0x7fd5ac101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.327+0000 7fd5b1302640 1 --2- 172.21.15.18:0/414970419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5ac1000f0 0x7fd5ac100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.327+0000 7fd5b1302640 1 -- 172.21.15.18:0/414970419 >> 172.21.15.18:0/414970419 conn(0x7fd5ac0fb660 msgr2=0x7fd5ac0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:46.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.327+0000 7fd5b1302640 1 -- 172.21.15.18:0/414970419 shutdown_connections 2024-02-17T12:49:46.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.327+0000 7fd5b1302640 1 -- 172.21.15.18:0/414970419 wait complete. 2024-02-17T12:49:46.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5b1302640 1 Processor -- start 2024-02-17T12:49:46.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5b1302640 1 -- start start 2024-02-17T12:49:46.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5b1302640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5ac1000f0 0x7fd5ac199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:46.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5b1302640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5ac1012e0 0x7fd5ac1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:46.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5b1302640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac101ca0 0x7fd5ac1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:46.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5b1302640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac1a0e60 con 0x7fd5ac1012e0 2024-02-17T12:49:46.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5b1302640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac1a0fd0 con 0x7fd5ac1000f0 2024-02-17T12:49:46.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5b1302640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac1a1190 con 0x7fd5ac101ca0 2024-02-17T12:49:46.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5aa7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5ac1012e0 0x7fd5ac1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:46.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5aa7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5ac1012e0 0x7fd5ac1998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34804/0 (socket says 172.21.15.18:34804) 2024-02-17T12:49:46.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5aa7fc640 1 -- 172.21.15.18:0/3519064905 learned_addr learned my addr 172.21.15.18:0/3519064905 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:46.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5aaffd640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5ac1000f0 0x7fd5ac199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5ab7fe640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac101ca0 0x7fd5ac1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5aa7fc640 1 -- 172.21.15.18:0/3519064905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac101ca0 msgr2=0x7fd5ac1a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5aa7fc640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5ac101ca0 0x7fd5ac1a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5aa7fc640 1 -- 172.21.15.18:0/3519064905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5ac1000f0 msgr2=0x7fd5ac199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5aa7fc640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5ac1000f0 0x7fd5ac199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.328+0000 7fd5aa7fc640 1 -- 172.21.15.18:0/3519064905 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a4009900 con 0x7fd5ac1012e0 2024-02-17T12:49:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.329+0000 7fd5aa7fc640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5ac1012e0 0x7fd5ac1998b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000e9f0 tx=0x7fd5a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.329+0000 7fd593fff640 1 -- 172.21.15.18:0/3519064905 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0009720 con 0x7fd5ac1012e0 2024-02-17T12:49:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.329+0000 7fd593fff640 1 -- 172.21.15.18:0/3519064905 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5a000cd80 con 0x7fd5ac1012e0 2024-02-17T12:49:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.329+0000 7fd5b1302640 1 -- 172.21.15.18:0/3519064905 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac1a1470 con 0x7fd5ac1012e0 2024-02-17T12:49:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.329+0000 7fd593fff640 1 -- 172.21.15.18:0/3519064905 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0010690 con 0x7fd5ac1012e0 2024-02-17T12:49:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.329+0000 7fd5b1302640 1 -- 172.21.15.18:0/3519064905 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac1a1930 con 0x7fd5ac1012e0 2024-02-17T12:49:46.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.330+0000 7fd5b1302640 1 -- 172.21.15.18:0/3519064905 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac10bb00 con 0x7fd5ac1012e0 2024-02-17T12:49:46.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.331+0000 7fd593fff640 1 -- 172.21.15.18:0/3519064905 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 136785+0+0 (secure 0 0 0) 0x7fd5a00046c0 con 0x7fd5ac1012e0 2024-02-17T12:49:46.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.331+0000 7fd593fff640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 0x7fd5940a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:46.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.332+0000 7fd5aaffd640 1 -- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 msgr2=0x7fd5940a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3026633658 2024-02-17T12:49:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.332+0000 7fd593fff640 1 -- 172.21.15.18:0/3519064905 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd5a0014070 con 0x7fd5ac1012e0 2024-02-17T12:49:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.332+0000 7fd5aaffd640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 0x7fd5940a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T12:49:46.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.336+0000 7fd593fff640 1 -- 172.21.15.18:0/3519064905 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5a0097050 con 0x7fd5ac1012e0 2024-02-17T12:49:46.450 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:46 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:46.122+0000 7f2573813200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T12:49:46.451 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:46 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:46.216+0000 7f2573813200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T12:49:46.524 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:46 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:46.158+0000 7fe72b873200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T12:49:46.525 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:46 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:46.259+0000 7fe72b873200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T12:49:46.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.533+0000 7fd5aaffd640 1 -- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 msgr2=0x7fd5940a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3026633658 2024-02-17T12:49:46.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.533+0000 7fd5aaffd640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 0x7fd5940a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-17T12:49:46.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.539+0000 7fd5b1302640 1 -- 172.21.15.18:0/3519064905 --> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5ac0fd6e0 con 0x7fd5940a21b0 2024-02-17T12:49:46.543 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:46 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:46.126+0000 7fc5e6178200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T12:49:46.544 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:46 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:46.220+0000 7fc5e6178200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T12:49:46.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:46 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/726569984' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-17T12:49:46.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:46 smithi119 ceph-mon[36188]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-02-17T12:49:46.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:46 smithi119 ceph-mon[36188]: from='client.? ' entity='client.admin' 2024-02-17T12:49:46.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:46 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/783520869' entity='client.admin' 2024-02-17T12:49:46.844 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:46 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/726569984' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-17T12:49:46.844 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:46 smithi146 ceph-mon[36454]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-02-17T12:49:46.844 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:46 smithi146 ceph-mon[36454]: from='client.? ' entity='client.admin' 2024-02-17T12:49:46.844 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:46 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/783520869' entity='client.admin' 2024-02-17T12:49:46.917 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:46 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/726569984' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-17T12:49:46.917 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:46 smithi018 ceph-mon[32625]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-02-17T12:49:46.917 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:46 smithi018 ceph-mon[32625]: from='client.? ' entity='client.admin' 2024-02-17T12:49:46.917 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:46 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/783520869' entity='client.admin' 2024-02-17T12:49:46.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.934+0000 7fd5aaffd640 1 -- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 msgr2=0x7fd5940a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3026633658 2024-02-17T12:49:46.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:46.934+0000 7fd5aaffd640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 0x7fd5940a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-17T12:49:47.173 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:46 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:46.851+0000 7f2573813200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T12:49:47.173 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:46 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:46.944+0000 7f2573813200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T12:49:47.173 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:47.085+0000 7f2573813200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T12:49:47.173 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:47.180+0000 7f2573813200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T12:49:47.184 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:46 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:46.860+0000 7fc5e6178200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T12:49:47.184 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:46 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:46.955+0000 7fc5e6178200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T12:49:47.184 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:47.096+0000 7fc5e6178200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T12:49:47.184 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:47.191+0000 7fc5e6178200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T12:49:47.256 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:46 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:46.925+0000 7fe72b873200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T12:49:47.256 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:47.022+0000 7fe72b873200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T12:49:47.256 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:47.165+0000 7fe72b873200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T12:49:47.256 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:47.264+0000 7fe72b873200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T12:49:47.451 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:47.263+0000 7f2573813200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T12:49:47.524 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:47.350+0000 7fe72b873200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T12:49:47.528 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:47.275+0000 7fc5e6178200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T12:49:47.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:47.735+0000 7fd5aaffd640 1 -- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 msgr2=0x7fd5940a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3026633658 2024-02-17T12:49:47.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:47.735+0000 7fd5aaffd640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 0x7fd5940a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-17T12:49:47.807 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:47.527+0000 7f2573813200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T12:49:47.808 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:47.636+0000 7f2573813200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T12:49:47.808 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:47.721+0000 7f2573813200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T12:49:47.808 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:47.815+0000 7f2573813200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T12:49:47.819 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:47.536+0000 7fc5e6178200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T12:49:47.820 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:47.645+0000 7fc5e6178200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T12:49:47.820 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:47.732+0000 7fc5e6178200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T12:49:47.820 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:47.827+0000 7fc5e6178200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T12:49:47.913 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:47.622+0000 7fe72b873200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T12:49:47.913 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:47.734+0000 7fe72b873200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T12:49:47.913 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:47.823+0000 7fe72b873200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T12:49:47.913 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:47.921+0000 7fe72b873200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T12:49:48.093 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:48 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:48.003+0000 7fc5e6178200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T12:49:48.201 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:47.993+0000 7f2573813200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T12:49:48.274 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:48.100+0000 7fe72b873200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T12:49:49.201 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:48.958+0000 7f2573813200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T12:49:49.201 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:49 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:49.089+0000 7f2573813200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T12:49:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:49.336+0000 7fd5aaffd640 1 -- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 msgr2=0x7fd5940a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3026633658 2024-02-17T12:49:49.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:49.336+0000 7fd5aaffd640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 0x7fd5940a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-17T12:49:49.343 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:48 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:48.968+0000 7fc5e6178200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T12:49:49.343 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:49.098+0000 7fc5e6178200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T12:49:49.524 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:49.097+0000 7fe72b873200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T12:49:49.524 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:49.232+0000 7fe72b873200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T12:49:49.700 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:49 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:49.424+0000 7f2573813200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T12:49:49.701 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:49 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:49.513+0000 7f2573813200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T12:49:49.701 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:49 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:49.708+0000 7f2573813200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:49:49.705 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:49.428+0000 7fc5e6178200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T12:49:49.705 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:49.518+0000 7fc5e6178200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T12:49:49.705 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:49.713+0000 7fc5e6178200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:49:49.860 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:49.572+0000 7fe72b873200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T12:49:49.860 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:49.665+0000 7fe72b873200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T12:49:49.951 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:49 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:49.895+0000 7f2573813200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T12:49:50.093 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:49.897+0000 7fc5e6178200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T12:49:50.274 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:49.867+0000 7fe72b873200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:49:50.274 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:50.059+0000 7fe72b873200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T12:49:50.431 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:50 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:50.181+0000 7fc5e6178200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:49:50.431 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:50 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:50.178+0000 7f2573813200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:49:50.610 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:50.354+0000 7fe72b873200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T12:49:50.701 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:50 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:50.439+0000 7f2573813200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T12:49:50.701 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:50 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:50.673+0000 7f2573813200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T12:49:50.843 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:50 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:50.439+0000 7fc5e6178200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T12:49:50.843 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:50 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:50.673+0000 7fc5e6178200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T12:49:51.024 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:50.617+0000 7fe72b873200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T12:49:51.024 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:50.863+0000 7fe72b873200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T12:49:51.843 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:51 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:51.472+0000 7fc5e6178200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T12:49:51.950 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:51 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:51.473+0000 7f2573813200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T12:49:52.024 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:51 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:51.701+0000 7fe72b873200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T12:49:52.343 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:52.040+0000 7fc5e6178200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T12:49:52.343 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:52.138+0000 7fc5e6178200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T12:49:52.355 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:52.049+0000 7f2573813200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T12:49:52.355 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:52.148+0000 7f2573813200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T12:49:52.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:52.429+0000 7fd593fff640 1 -- 172.21.15.18:0/3519064905 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 20) v1 ==== 136785+0+0 (secure 0 0 0) 0x7fd5a00d1030 con 0x7fd5ac1012e0 2024-02-17T12:49:52.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:52.540+0000 7fd5aaffd640 1 -- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 msgr2=0x7fd5940a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3026633658 2024-02-17T12:49:52.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:52.540+0000 7fd5aaffd640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 0x7fd5940a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-02-17T12:49:52.626 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.290+0000 7fe72b873200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T12:49:52.626 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.400+0000 7fe72b873200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T12:49:52.626 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-mon[32625]: Standby manager daemon y restarted 2024-02-17T12:49:52.626 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-mon[32625]: Standby manager daemon y started 2024-02-17T12:49:52.626 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.119:0/3560578821' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-17T12:49:52.626 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.119:0/3560578821' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T12:49:52.626 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.119:0/3560578821' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-17T12:49:52.626 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.119:0/3560578821' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T12:49:52.627 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-mon[32625]: Standby manager daemon z restarted 2024-02-17T12:49:52.627 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-mon[32625]: Standby manager daemon z started 2024-02-17T12:49:52.627 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.146:0/17231320' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-17T12:49:52.627 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.146:0/17231320' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T12:49:52.627 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.146:0/17231320' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-17T12:49:52.627 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-mon[32625]: from='mgr.? 172.21.15.146:0/17231320' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T12:49:52.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:52.656+0000 7fd593fff640 1 -- 172.21.15.18:0/3519064905 <== mon.0 v2:172.21.15.18:3300/0 8 ==== mgrmap(e 21) v1 ==== 136453+0+0 (secure 0 0 0) 0x7fd5a00cbbd0 con 0x7fd5ac1012e0 2024-02-17T12:49:52.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:52.657+0000 7fd593fff640 1 -- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 msgr2=0x7fd5940a4670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:49:52.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:52.657+0000 7fd593fff640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/3026633658,v1:172.21.15.18:6801/3026633658] conn(0x7fd5940a21b0 0x7fd5940a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:52.701 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z[38306]: 2024-02-17T12:49:52.364+0000 7f2573813200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T12:49:52.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-mon[36454]: Standby manager daemon y restarted 2024-02-17T12:49:52.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-mon[36454]: Standby manager daemon y started 2024-02-17T12:49:52.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.119:0/3560578821' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-17T12:49:52.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.119:0/3560578821' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T12:49:52.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.119:0/3560578821' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-17T12:49:52.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.119:0/3560578821' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T12:49:52.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-mon[36454]: Standby manager daemon z restarted 2024-02-17T12:49:52.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-mon[36454]: Standby manager daemon z started 2024-02-17T12:49:52.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.146:0/17231320' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-17T12:49:52.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.146:0/17231320' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T12:49:52.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.146:0/17231320' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-17T12:49:52.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:52 smithi146 ceph-mon[36454]: from='mgr.? 172.21.15.146:0/17231320' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T12:49:52.827 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y[37568]: 2024-02-17T12:49:52.355+0000 7fc5e6178200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T12:49:52.827 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-mon[36188]: Standby manager daemon y restarted 2024-02-17T12:49:52.827 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-mon[36188]: Standby manager daemon y started 2024-02-17T12:49:52.827 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.119:0/3560578821' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-17T12:49:52.827 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.119:0/3560578821' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T12:49:52.827 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.119:0/3560578821' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-17T12:49:52.827 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.119:0/3560578821' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T12:49:52.827 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-mon[36188]: Standby manager daemon z restarted 2024-02-17T12:49:52.828 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-mon[36188]: Standby manager daemon z started 2024-02-17T12:49:52.828 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.146:0/17231320' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-17T12:49:52.828 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.146:0/17231320' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T12:49:52.828 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.146:0/17231320' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-17T12:49:52.828 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:52 smithi119 ceph-mon[36188]: from='mgr.? 172.21.15.146:0/17231320' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T12:49:53.025 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.633+0000 7fe72b873200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T12:49:53.025 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.882+0000 7fe68daa1700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:49:53.025 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.882+0000 7fe68daa1700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:49:53.025 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.882+0000 7fe68daa1700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:49:53.026 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.882+0000 7fe68daa1700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:49:53.026 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.882+0000 7fe68daa1700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:49:53.026 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.882+0000 7fe68b29c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:49:53.026 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.882+0000 7fe68b29c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:49:53.026 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.882+0000 7fe68b29c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:49:53.026 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.882+0000 7fe68b29c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:49:53.026 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:49:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:49:52.882+0000 7fe68b29c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:49:53.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.662+0000 7fd593fff640 1 -- 172.21.15.18:0/3519064905 <== mon.0 v2:172.21.15.18:3300/0 9 ==== mgrmap(e 22) v1 ==== 136580+0+0 (secure 0 0 0) 0x7fd5a0097420 con 0x7fd5ac1012e0 2024-02-17T12:49:53.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.662+0000 7fd593fff640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd594035630 0x7fd594035a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:53.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.662+0000 7fd593fff640 1 -- 172.21.15.18:0/3519064905 --> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5ac0fd6e0 con 0x7fd594035630 2024-02-17T12:49:53.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.667+0000 7fd5aaffd640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd594035630 0x7fd594035a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:53.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.671+0000 7fd5aaffd640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd594035630 0x7fd594035a10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac101140 tx=0x7fd59c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:53.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-02-17T12:49:53.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: Active manager daemon x restarted 2024-02-17T12:49:53.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: Activating manager daemon x 2024-02-17T12:49:53.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: osdmap e89: 12 total, 12 up, 12 in 2024-02-17T12:49:53.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: mgrmap e21: x(active, starting, since 0.0113557s), standbys: y, z 2024-02-17T12:49:53.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:49:53.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:49:53.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:49:53.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-17T12:49:53.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-17T12:49:53.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T12:49:53.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T12:49:53.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:49:53.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:49:53.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:49:53.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:49:53.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:49:53.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:49:53.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:49:53.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:49:53.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:49:53.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-17T12:49:53.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-17T12:49:53.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-17T12:49:53.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: Manager daemon x is now available 2024-02-17T12:49:53.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:49:53.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:49:53.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-17T12:49:53.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:53 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-17T12:49:53.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.741+0000 7fd593fff640 1 -- 172.21.15.18:0/3519064905 <== mgr.15528 v2:172.21.15.18:6800/2186095810 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd5ac0fd6e0 con 0x7fd594035630 2024-02-17T12:49:53.734 INFO:teuthology.orchestra.run.smithi018.stdout:Schedule set for path / 2024-02-17T12:49:53.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.743+0000 7fd5b1302640 1 -- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd594035630 msgr2=0x7fd594035a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:53.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.744+0000 7fd5b1302640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd594035630 0x7fd594035a10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac101140 tx=0x7fd59c005f90 comp rx=0 tx=0).stop 2024-02-17T12:49:53.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.744+0000 7fd5b1302640 1 -- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5ac1012e0 msgr2=0x7fd5ac1998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:53.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.744+0000 7fd5b1302640 1 --2- 172.21.15.18:0/3519064905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5ac1012e0 0x7fd5ac1998b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000e9f0 tx=0x7fd5a000edf0 comp rx=0 tx=0).stop 2024-02-17T12:49:53.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.744+0000 7fd5ab7fe640 1 -- 172.21.15.18:0/3519064905 reap_dead start 2024-02-17T12:49:53.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.744+0000 7fd5b1302640 1 -- 172.21.15.18:0/3519064905 shutdown_connections 2024-02-17T12:49:53.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.744+0000 7fd5b1302640 1 -- 172.21.15.18:0/3519064905 >> 172.21.15.18:0/3519064905 conn(0x7fd5ac0fb660 msgr2=0x7fd5ac0fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:53.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.744+0000 7fd5b1302640 1 -- 172.21.15.18:0/3519064905 shutdown_connections 2024-02-17T12:49:53.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.744+0000 7fd5b1302640 1 -- 172.21.15.18:0/3519064905 wait complete. 2024-02-17T12:49:53.748 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-02-17T12:49:53.748 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: Active manager daemon x restarted 2024-02-17T12:49:53.748 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: Activating manager daemon x 2024-02-17T12:49:53.748 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: osdmap e89: 12 total, 12 up, 12 in 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: mgrmap e21: x(active, starting, since 0.0113557s), standbys: y, z 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T12:49:53.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T12:49:53.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T12:49:53.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T12:49:53.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T12:49:53.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-17T12:49:53.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-17T12:49:53.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-17T12:49:53.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:49:53.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:49:53.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:49:53.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:49:53.750 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:49:53.751 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-02-17T12:49:53.752 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:49:53.752 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:49:53.752 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:49:53.752 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:49:53.752 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:49:53.752 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:49:53.752 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:49:53.752 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-17T12:49:53.752 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-17T12:49:53.752 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-17T12:49:53.753 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: Manager daemon x is now available 2024-02-17T12:49:53.753 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:49:53.753 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:49:53.753 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-17T12:49:53.753 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:53 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-17T12:49:53.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-02-17T12:49:53.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: Active manager daemon x restarted 2024-02-17T12:49:53.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: Activating manager daemon x 2024-02-17T12:49:53.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: osdmap e89: 12 total, 12 up, 12 in 2024-02-17T12:49:53.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: mgrmap e21: x(active, starting, since 0.0113557s), standbys: y, z 2024-02-17T12:49:53.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T12:49:53.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T12:49:53.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T12:49:53.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T12:49:53.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-17T12:49:53.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-17T12:49:53.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-17T12:49:53.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: Manager daemon x is now available 2024-02-17T12:49:53.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:49:53.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:49:53.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-17T12:49:53.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:53 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-17T12:49:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.868+0000 7fdb05a2e640 1 -- 172.21.15.18:0/2479519828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb00101290 msgr2=0x7fdb00101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.868+0000 7fdb05a2e640 1 --2- 172.21.15.18:0/2479519828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb00101290 0x7fdb00101710 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009900 tx=0x7fdae802bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 -- 172.21.15.18:0/2479519828 shutdown_connections 2024-02-17T12:49:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 --2- 172.21.15.18:0/2479519828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb00101c50 0x7fdb001084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 --2- 172.21.15.18:0/2479519828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb00101290 0x7fdb00101710 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 --2- 172.21.15.18:0/2479519828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb001000a0 0x7fdb001004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 -- 172.21.15.18:0/2479519828 >> 172.21.15.18:0/2479519828 conn(0x7fdb000fb610 msgr2=0x7fdb000fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 -- 172.21.15.18:0/2479519828 shutdown_connections 2024-02-17T12:49:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 -- 172.21.15.18:0/2479519828 wait complete. 2024-02-17T12:49:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 Processor -- start 2024-02-17T12:49:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 -- start start 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb001000a0 0x7fdb00069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb00101290 0x7fdb0019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb00101c50 0x7fdb0019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0019ecf0 con 0x7fdb00101290 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0019ee60 con 0x7fdb00101c50 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.869+0000 7fdb05a2e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0019f020 con 0x7fdb001000a0 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdafe7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb00101290 0x7fdb0019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdafe7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb00101290 0x7fdb0019bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60356/0 (socket says 172.21.15.18:60356) 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdafe7fc640 1 -- 172.21.15.18:0/457961265 learned_addr learned my addr 172.21.15.18:0/457961265 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdafeffd640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb001000a0 0x7fdb00069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdaff7fe640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb00101c50 0x7fdb0019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdafe7fc640 1 -- 172.21.15.18:0/457961265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb001000a0 msgr2=0x7fdb00069370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdafe7fc640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb001000a0 0x7fdb00069370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdafe7fc640 1 -- 172.21.15.18:0/457961265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb00101c50 msgr2=0x7fdb0019e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdafe7fc640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb00101c50 0x7fdb0019e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdafe7fc640 1 -- 172.21.15.18:0/457961265 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae8009560 con 0x7fdb00101290 2024-02-17T12:49:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdafe7fc640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb00101290 0x7fdb0019bef0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdae8006010 tx=0x7fdae80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdb04a2c640 1 -- 172.21.15.18:0/457961265 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae80044e0 con 0x7fdb00101290 2024-02-17T12:49:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdb05a2e640 1 -- 172.21.15.18:0/457961265 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0019f240 con 0x7fdb00101290 2024-02-17T12:49:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdb04a2c640 1 -- 172.21.15.18:0/457961265 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdae80054e0 con 0x7fdb00101290 2024-02-17T12:49:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdb04a2c640 1 -- 172.21.15.18:0/457961265 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae803e450 con 0x7fdb00101290 2024-02-17T12:49:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.870+0000 7fdb05a2e640 1 -- 172.21.15.18:0/457961265 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0019f750 con 0x7fdb00101290 2024-02-17T12:49:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.871+0000 7fdb05a2e640 1 -- 172.21.15.18:0/457961265 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb001073c0 con 0x7fdb00101290 2024-02-17T12:49:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.872+0000 7fdb04a2c640 1 -- 172.21.15.18:0/457961265 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 22) v1 ==== 136580+0+0 (secure 0 0 0) 0x7fdae80035f0 con 0x7fdb00101290 2024-02-17T12:49:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.873+0000 7fdb04a2c640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdaec0a1f70 0x7fdaec0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.873+0000 7fdafeffd640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdaec0a1f70 0x7fdaec0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.873+0000 7fdb04a2c640 1 -- 172.21.15.18:0/457961265 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdae80efcb0 con 0x7fdb00101290 2024-02-17T12:49:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.873+0000 7fdafeffd640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdaec0a1f70 0x7fdaec0a4430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdb001010f0 tx=0x7fdaf800b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:53.877+0000 7fdb04a2c640 1 -- 172.21.15.18:0/457961265 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdae80b8ea0 con 0x7fdb00101290 2024-02-17T12:49:54.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.089+0000 7fdb05a2e640 1 -- 172.21.15.18:0/457961265 --> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb0002ce10 con 0x7fdaec0a1f70 2024-02-17T12:49:54.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.149+0000 7fdb04a2c640 1 -- 172.21.15.18:0/457961265 <== mgr.15528 v2:172.21.15.18:6800/2186095810 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fdb0002ce10 con 0x7fdaec0a1f70 2024-02-17T12:49:54.142 INFO:teuthology.orchestra.run.smithi018.stdout:Retention added to path / 2024-02-17T12:49:54.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.151+0000 7fdb05a2e640 1 -- 172.21.15.18:0/457961265 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdaec0a1f70 msgr2=0x7fdaec0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:54.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.151+0000 7fdb05a2e640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdaec0a1f70 0x7fdaec0a4430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdb001010f0 tx=0x7fdaf800b390 comp rx=0 tx=0).stop 2024-02-17T12:49:54.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.151+0000 7fdb05a2e640 1 -- 172.21.15.18:0/457961265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb00101290 msgr2=0x7fdb0019bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:54.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.151+0000 7fdb05a2e640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb00101290 0x7fdb0019bef0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdae8006010 tx=0x7fdae80042f0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.152+0000 7fdb05a2e640 1 -- 172.21.15.18:0/457961265 shutdown_connections 2024-02-17T12:49:54.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.152+0000 7fdb05a2e640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdaec0a1f70 0x7fdaec0a4430 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.152+0000 7fdb05a2e640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb00101c50 0x7fdb0019e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.152+0000 7fdb05a2e640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb00101290 0x7fdb0019bef0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.152+0000 7fdb05a2e640 1 --2- 172.21.15.18:0/457961265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb001000a0 0x7fdb00069370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.152+0000 7fdb05a2e640 1 -- 172.21.15.18:0/457961265 >> 172.21.15.18:0/457961265 conn(0x7fdb000fb610 msgr2=0x7fdb00104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:54.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.152+0000 7fdb05a2e640 1 -- 172.21.15.18:0/457961265 shutdown_connections 2024-02-17T12:49:54.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.152+0000 7fdb05a2e640 1 -- 172.21.15.18:0/457961265 wait complete. 2024-02-17T12:49:54.155 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-17T12:49:54.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.267+0000 7f71ffdac640 1 -- 172.21.15.18:0/3911195261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71f80ff4a0 msgr2=0x7f71f80ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:54.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.267+0000 7f71ffdac640 1 --2- 172.21.15.18:0/3911195261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71f80ff4a0 0x7f71f80ff8c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009900 tx=0x7f71f402bc30 comp rx=0 tx=0).stop 2024-02-17T12:49:54.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.267+0000 7f71ffdac640 1 -- 172.21.15.18:0/3911195261 shutdown_connections 2024-02-17T12:49:54.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.267+0000 7f71ffdac640 1 --2- 172.21.15.18:0/3911195261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f8102e30 0x7f71f81096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.267+0000 7f71ffdac640 1 --2- 172.21.15.18:0/3911195261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f8102470 0x7f71f81028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.267+0000 7f71ffdac640 1 --2- 172.21.15.18:0/3911195261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71f80ff4a0 0x7f71f80ff8c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.267+0000 7f71ffdac640 1 -- 172.21.15.18:0/3911195261 >> 172.21.15.18:0/3911195261 conn(0x7f71f80fadd0 msgr2=0x7f71f80fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:54.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.267+0000 7f71ffdac640 1 -- 172.21.15.18:0/3911195261 shutdown_connections 2024-02-17T12:49:54.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.267+0000 7f71ffdac640 1 -- 172.21.15.18:0/3911195261 wait complete. 2024-02-17T12:49:54.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71ffdac640 1 Processor -- start 2024-02-17T12:49:54.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71ffdac640 1 -- start start 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71ffdac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f80ff4a0 0x7f71f81971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71ffdac640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f8102470 0x7f71f81976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71ffdac640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71f8102e30 0x7f71f819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71ffdac640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f819eca0 con 0x7f71f8102e30 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71ffdac640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f819ee10 con 0x7f71f80ff4a0 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71ffdac640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f819efd0 con 0x7f71f8102470 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71fe349640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71f8102e30 0x7f71f819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71fe349640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71f8102e30 0x7f71f819e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60382/0 (socket says 172.21.15.18:60382) 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71fe349640 1 -- 172.21.15.18:0/2193680843 learned_addr learned my addr 172.21.15.18:0/2193680843 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71fd347640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f8102470 0x7f71f81976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71fdb48640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f80ff4a0 0x7f71f81971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71fe349640 1 -- 172.21.15.18:0/2193680843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f8102470 msgr2=0x7f71f81976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71fe349640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f8102470 0x7f71f81976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71fe349640 1 -- 172.21.15.18:0/2193680843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f80ff4a0 msgr2=0x7f71f81971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:54.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71fe349640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f80ff4a0 0x7f71f81971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71fe349640 1 -- 172.21.15.18:0/2193680843 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f4009560 con 0x7f71f8102e30 2024-02-17T12:49:54.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.268+0000 7f71fe349640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71f8102e30 0x7f71f819e760 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f71f000eb10 tx=0x7f71f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:54.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.269+0000 7f71eeffd640 1 -- 172.21.15.18:0/2193680843 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0004b20 con 0x7f71f8102e30 2024-02-17T12:49:54.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.269+0000 7f71eeffd640 1 -- 172.21.15.18:0/2193680843 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71f0004c80 con 0x7f71f8102e30 2024-02-17T12:49:54.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.269+0000 7f71ffdac640 1 -- 172.21.15.18:0/2193680843 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f819f2b0 con 0x7f71f8102e30 2024-02-17T12:49:54.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.269+0000 7f71eeffd640 1 -- 172.21.15.18:0/2193680843 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0010650 con 0x7f71f8102e30 2024-02-17T12:49:54.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.269+0000 7f71ffdac640 1 -- 172.21.15.18:0/2193680843 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f819f770 con 0x7f71f8102e30 2024-02-17T12:49:54.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.270+0000 7f71ffdac640 1 -- 172.21.15.18:0/2193680843 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f8108540 con 0x7f71f8102e30 2024-02-17T12:49:54.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.271+0000 7f71eeffd640 1 -- 172.21.15.18:0/2193680843 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 22) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f71f00107f0 con 0x7f71f8102e30 2024-02-17T12:49:54.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.271+0000 7f71eeffd640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f71e40a1f70 0x7f71e40a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:54.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.271+0000 7f71eeffd640 1 -- 172.21.15.18:0/2193680843 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f71f0014070 con 0x7f71f8102e30 2024-02-17T12:49:54.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.271+0000 7f71fdb48640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f71e40a1f70 0x7f71e40a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:54.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.272+0000 7f71fdb48640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f71e40a1f70 0x7f71e40a4430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71f4005ab0 tx=0x7f71f4005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:54.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.275+0000 7f71eeffd640 1 -- 172.21.15.18:0/2193680843 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f71f00972d0 con 0x7f71f8102e30 2024-02-17T12:49:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.481+0000 7f71ffdac640 1 -- 172.21.15.18:0/2193680843 --> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f71f802ceb0 con 0x7f71e40a1f70 2024-02-17T12:49:54.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.489+0000 7f71eeffd640 1 -- 172.21.15.18:0/2193680843 <== mgr.15528 v2:172.21.15.18:6800/2186095810 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f71f802ceb0 con 0x7f71e40a1f70 2024-02-17T12:49:54.481 INFO:teuthology.orchestra.run.smithi018.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-17T00:00:00", "created": "2024-02-17T12:49:53", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-02-17T12:49:54.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.491+0000 7f71ffdac640 1 -- 172.21.15.18:0/2193680843 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f71e40a1f70 msgr2=0x7f71e40a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:54.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.491+0000 7f71ffdac640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f71e40a1f70 0x7f71e40a4430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71f4005ab0 tx=0x7f71f4005fd0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.491+0000 7f71ffdac640 1 -- 172.21.15.18:0/2193680843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71f8102e30 msgr2=0x7f71f819e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:54.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.491+0000 7f71ffdac640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71f8102e30 0x7f71f819e760 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f71f000eb10 tx=0x7f71f000c2d0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.491+0000 7f71ffdac640 1 -- 172.21.15.18:0/2193680843 shutdown_connections 2024-02-17T12:49:54.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.491+0000 7f71ffdac640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f71e40a1f70 0x7f71e40a4430 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.491+0000 7f71ffdac640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71f8102e30 0x7f71f819e760 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.491+0000 7f71ffdac640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f8102470 0x7f71f81976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.491+0000 7f71ffdac640 1 --2- 172.21.15.18:0/2193680843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f80ff4a0 0x7f71f81971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.491+0000 7f71ffdac640 1 -- 172.21.15.18:0/2193680843 >> 172.21.15.18:0/2193680843 conn(0x7f71f80fadd0 msgr2=0x7f71f8105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:54.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.492+0000 7f71ffdac640 1 -- 172.21.15.18:0/2193680843 shutdown_connections 2024-02-17T12:49:54.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.492+0000 7f71ffdac640 1 -- 172.21.15.18:0/2193680843 wait complete. 2024-02-17T12:49:54.496 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-17T12:49:54.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:54 smithi119 ceph-mon[36188]: [17/Feb/2024:12:49:53] ENGINE Bus STARTING 2024-02-17T12:49:54.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:54 smithi119 ceph-mon[36188]: mgrmap e22: x(active, since 1.01765s), standbys: y, z 2024-02-17T12:49:54.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.634+0000 7f2f6721c640 1 -- 172.21.15.18:0/2418589600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f60102eb0 msgr2=0x7f2f60109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.634+0000 7f2f6721c640 1 --2- 172.21.15.18:0/2418589600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f60102eb0 0x7f2f60109740 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009ca0 tx=0x7f2f5802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.634+0000 7f2f6721c640 1 -- 172.21.15.18:0/2418589600 shutdown_connections 2024-02-17T12:49:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.634+0000 7f2f6721c640 1 --2- 172.21.15.18:0/2418589600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f60102eb0 0x7f2f60109740 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.634+0000 7f2f6721c640 1 --2- 172.21.15.18:0/2418589600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f601024f0 0x7f2f60102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.634+0000 7f2f6721c640 1 --2- 172.21.15.18:0/2418589600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f600ff480 0x7f2f600ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.634+0000 7f2f6721c640 1 -- 172.21.15.18:0/2418589600 >> 172.21.15.18:0/2418589600 conn(0x7f2f600fadd0 msgr2=0x7f2f600fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.634+0000 7f2f6721c640 1 -- 172.21.15.18:0/2418589600 shutdown_connections 2024-02-17T12:49:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.634+0000 7f2f6721c640 1 -- 172.21.15.18:0/2418589600 wait complete. 2024-02-17T12:49:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f6721c640 1 Processor -- start 2024-02-17T12:49:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f6721c640 1 -- start start 2024-02-17T12:49:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f6721c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f600ff480 0x7f2f60101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f6721c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f601024f0 0x7f2f601004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f6721c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f60102eb0 0x7f2f601009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f6721c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f60101170 con 0x7f2f601024f0 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f6721c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f601012e0 con 0x7f2f600ff480 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f6721c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f601a11f0 con 0x7f2f60102eb0 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f5ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f601024f0 0x7f2f601004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f5ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f601024f0 0x7f2f601004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60408/0 (socket says 172.21.15.18:60408) 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f5ffff640 1 -- 172.21.15.18:0/1452384598 learned_addr learned my addr 172.21.15.18:0/1452384598 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f657b9640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f60102eb0 0x7f2f601009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.635+0000 7f2f64fb8640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f600ff480 0x7f2f60101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.636+0000 7f2f5ffff640 1 -- 172.21.15.18:0/1452384598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f60102eb0 msgr2=0x7f2f601009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.636+0000 7f2f5ffff640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f60102eb0 0x7f2f601009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.636+0000 7f2f5ffff640 1 -- 172.21.15.18:0/1452384598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f600ff480 msgr2=0x7f2f60101e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:54.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.636+0000 7f2f5ffff640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f600ff480 0x7f2f60101e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.636+0000 7f2f5ffff640 1 -- 172.21.15.18:0/1452384598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f58009900 con 0x7f2f601024f0 2024-02-17T12:49:54.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.636+0000 7f2f5ffff640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f601024f0 0x7f2f601004a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00ea20 tx=0x7f2f4c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:54.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.636+0000 7f2f5dffb640 1 -- 172.21.15.18:0/1452384598 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c009700 con 0x7f2f601024f0 2024-02-17T12:49:54.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.636+0000 7f2f5dffb640 1 -- 172.21.15.18:0/1452384598 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f4c00cd80 con 0x7f2f601024f0 2024-02-17T12:49:54.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.636+0000 7f2f6721c640 1 -- 172.21.15.18:0/1452384598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f601a1450 con 0x7f2f601024f0 2024-02-17T12:49:54.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.636+0000 7f2f5dffb640 1 -- 172.21.15.18:0/1452384598 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c018840 con 0x7f2f601024f0 2024-02-17T12:49:54.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.636+0000 7f2f6721c640 1 -- 172.21.15.18:0/1452384598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f601a1960 con 0x7f2f601024f0 2024-02-17T12:49:54.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.637+0000 7f2f6721c640 1 -- 172.21.15.18:0/1452384598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f60108620 con 0x7f2f601024f0 2024-02-17T12:49:54.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.638+0000 7f2f5dffb640 1 -- 172.21.15.18:0/1452384598 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 22) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f2f4c010a80 con 0x7f2f601024f0 2024-02-17T12:49:54.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.639+0000 7f2f5dffb640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2f500a1f70 0x7f2f500a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:49:54.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.639+0000 7f2f5dffb640 1 -- 172.21.15.18:0/1452384598 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2f4c014070 con 0x7f2f601024f0 2024-02-17T12:49:54.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.639+0000 7f2f64fb8640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2f500a1f70 0x7f2f500a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:49:54.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.639+0000 7f2f64fb8640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2f500a1f70 0x7f2f500a4430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f60068540 tx=0x7f2f54005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:49:54.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.643+0000 7f2f5dffb640 1 -- 172.21.15.18:0/1452384598 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f4c052d20 con 0x7f2f601024f0 2024-02-17T12:49:54.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:54 smithi146 ceph-mon[36454]: [17/Feb/2024:12:49:53] ENGINE Bus STARTING 2024-02-17T12:49:54.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:54 smithi146 ceph-mon[36454]: mgrmap e22: x(active, since 1.01765s), standbys: y, z 2024-02-17T12:49:54.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:54 smithi018 ceph-mon[32625]: [17/Feb/2024:12:49:53] ENGINE Bus STARTING 2024-02-17T12:49:54.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:54 smithi018 ceph-mon[32625]: mgrmap e22: x(active, since 1.01765s), standbys: y, z 2024-02-17T12:49:54.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.896+0000 7f2f6721c640 1 -- 172.21.15.18:0/1452384598 --> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2f6002ceb0 con 0x7f2f500a1f70 2024-02-17T12:49:54.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.902+0000 7f2f5dffb640 1 -- 172.21.15.18:0/1452384598 <== mgr.15528 v2:172.21.15.18:6800/2186095810 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f2f6002ceb0 con 0x7f2f500a1f70 2024-02-17T12:49:54.895 INFO:teuthology.orchestra.run.smithi018.stdout:/ 1m 6m3h 2024-02-17T12:49:54.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.904+0000 7f2f6721c640 1 -- 172.21.15.18:0/1452384598 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2f500a1f70 msgr2=0x7f2f500a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:54.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.904+0000 7f2f6721c640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2f500a1f70 0x7f2f500a4430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f60068540 tx=0x7f2f54005f90 comp rx=0 tx=0).stop 2024-02-17T12:49:54.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.904+0000 7f2f6721c640 1 -- 172.21.15.18:0/1452384598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f601024f0 msgr2=0x7f2f601004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:49:54.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.904+0000 7f2f6721c640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f601024f0 0x7f2f601004a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00ea20 tx=0x7f2f4c00eee0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.905+0000 7f2f6721c640 1 -- 172.21.15.18:0/1452384598 shutdown_connections 2024-02-17T12:49:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.905+0000 7f2f6721c640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2f500a1f70 0x7f2f500a4430 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.905+0000 7f2f6721c640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f60102eb0 0x7f2f601009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.905+0000 7f2f6721c640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f601024f0 0x7f2f601004a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.905+0000 7f2f6721c640 1 --2- 172.21.15.18:0/1452384598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f600ff480 0x7f2f60101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:49:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.905+0000 7f2f6721c640 1 -- 172.21.15.18:0/1452384598 >> 172.21.15.18:0/1452384598 conn(0x7f2f600fadd0 msgr2=0x7f2f601057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:49:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.905+0000 7f2f6721c640 1 -- 172.21.15.18:0/1452384598 shutdown_connections 2024-02-17T12:49:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:49:54.905+0000 7f2f6721c640 1 -- 172.21.15.18:0/1452384598 wait complete. 2024-02-17T12:49:54.908 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-02-17T12:49:54.937 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-02-17T12:49:55.008 INFO:teuthology.run_tasks:Running task background_exec... 2024-02-17T12:49:55.038 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-02-17T12:49:55.039 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-02-17T12:49:55.041 INFO:teuthology.run_tasks:Running task workunit... 2024-02-17T12:49:55.054 INFO:tasks.workunit:Pulling workunits from ref 6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:49:55.055 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-17T12:49:55.055 INFO:tasks.workunit:timeout=3h 2024-02-17T12:49:55.055 INFO:tasks.workunit:cleanup=True 2024-02-17T12:49:55.055 DEBUG:teuthology.orchestra.run.smithi018:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-17T12:49:55.070 INFO:teuthology.orchestra.run.smithi018.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-17T12:49:55.070 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-02-17T12:49:55.070 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-02-17T12:49:55.070 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T12:49:55.071 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-02-17T12:49:55.071 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-02-17 12:47:51.793931620 +0000 2024-02-17T12:49:55.071 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-02-17 12:47:51.793931620 +0000 2024-02-17T12:49:55.071 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-02-17 12:49:35.998307385 +0000 2024-02-17T12:49:55.071 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-02-17 12:47:51.793931620 +0000 2024-02-17T12:49:55.071 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-17T12:49:55.071 DEBUG:teuthology.orchestra.run.smithi018:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-17T12:49:55.148 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T12:49:55.204 INFO:tasks.workunit.client.0.smithi018.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-17T12:49:55.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:55.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:49:55.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: [17/Feb/2024:12:49:53] ENGINE Serving on https://172.21.15.18:7150 2024-02-17T12:49:55.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: [17/Feb/2024:12:49:53] ENGINE Serving on http://172.21.15.18:8765 2024-02-17T12:49:55.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: [17/Feb/2024:12:49:53] ENGINE Bus STARTED 2024-02-17T12:49:55.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:55.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:55.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:55 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:55.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:49:55.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: [17/Feb/2024:12:49:53] ENGINE Serving on https://172.21.15.18:7150 2024-02-17T12:49:55.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: [17/Feb/2024:12:49:53] ENGINE Serving on http://172.21.15.18:8765 2024-02-17T12:49:55.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: [17/Feb/2024:12:49:53] ENGINE Bus STARTED 2024-02-17T12:49:55.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:55.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:55.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:55 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:55.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:49:55.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: [17/Feb/2024:12:49:53] ENGINE Serving on https://172.21.15.18:7150 2024-02-17T12:49:55.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: [17/Feb/2024:12:49:53] ENGINE Serving on http://172.21.15.18:8765 2024-02-17T12:49:55.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: [17/Feb/2024:12:49:53] ENGINE Bus STARTED 2024-02-17T12:49:55.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:55.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:55.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:55.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:55 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:49:56.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:56.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: mgrmap e23: x(active, since 3s), standbys: y, z 2024-02-17T12:49:56.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.706 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.706 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.706 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:49:56.706 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:56 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:49:56.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:49:56.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:56.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: mgrmap e23: x(active, since 3s), standbys: y, z 2024-02-17T12:49:56.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:49:56.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:56 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:49:56.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:49:56.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:49:56.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: mgrmap e23: x(active, since 3s), standbys: y, z 2024-02-17T12:49:56.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:49:56.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:56 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:49:57.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi119 to 174.1M 2024-02-17T12:49:57.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: Unable to set osd_memory_target on smithi119 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-02-17T12:49:57.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi146 to 174.1M 2024-02-17T12:49:57.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: Unable to set osd_memory_target on smithi146 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-02-17T12:49:57.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi018 to 177.2M 2024-02-17T12:49:57.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: Unable to set osd_memory_target on smithi018 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-02-17T12:49:57.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: Updating smithi018:/etc/ceph/ceph.conf 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: Updating smithi119:/etc/ceph/ceph.conf 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: Updating smithi146:/etc/ceph/ceph.conf 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: Updating smithi018:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:49:57.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:49:57.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:49:57.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:57 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi119 to 174.1M 2024-02-17T12:49:57.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: Unable to set osd_memory_target on smithi119 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-02-17T12:49:57.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi146 to 174.1M 2024-02-17T12:49:57.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: Unable to set osd_memory_target on smithi146 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-02-17T12:49:57.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi018 to 177.2M 2024-02-17T12:49:57.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: Unable to set osd_memory_target on smithi018 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-02-17T12:49:57.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: Updating smithi018:/etc/ceph/ceph.conf 2024-02-17T12:49:57.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: Updating smithi119:/etc/ceph/ceph.conf 2024-02-17T12:49:57.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: Updating smithi146:/etc/ceph/ceph.conf 2024-02-17T12:49:57.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: Updating smithi018:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:49:57.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:49:57.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:49:57.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:49:57.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:49:57.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:49:57.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:57 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi119 to 174.1M 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: Unable to set osd_memory_target on smithi119 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi146 to 174.1M 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: Unable to set osd_memory_target on smithi146 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi018 to 177.2M 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: Unable to set osd_memory_target on smithi018 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: Updating smithi018:/etc/ceph/ceph.conf 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: Updating smithi119:/etc/ceph/ceph.conf 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: Updating smithi146:/etc/ceph/ceph.conf 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: Updating smithi018:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:57.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:49:57.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:49:57.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:49:57.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:57 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:49:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:58 smithi146 ceph-mon[36454]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:49:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:58 smithi146 ceph-mon[36454]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:49:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:58 smithi146 ceph-mon[36454]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:49:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:58 smithi146 ceph-mon[36454]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:49:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:58 smithi146 ceph-mon[36454]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:49:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:58 smithi146 ceph-mon[36454]: Updating smithi018:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:49:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:58 smithi146 ceph-mon[36454]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:49:58.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:49:58 smithi146 ceph-mon[36454]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-02-17T12:49:58.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:58 smithi018 ceph-mon[32625]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:49:58.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:58 smithi018 ceph-mon[32625]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:49:58.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:58 smithi018 ceph-mon[32625]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:49:58.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:58 smithi018 ceph-mon[32625]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:49:58.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:58 smithi018 ceph-mon[32625]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:49:58.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:58 smithi018 ceph-mon[32625]: Updating smithi018:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:49:58.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:58 smithi018 ceph-mon[32625]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:49:58.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:49:58 smithi018 ceph-mon[32625]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-02-17T12:49:58.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:58 smithi119 ceph-mon[36188]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:49:58.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:58 smithi119 ceph-mon[36188]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:49:58.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:58 smithi119 ceph-mon[36188]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-02-17T12:49:58.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:58 smithi119 ceph-mon[36188]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:49:58.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:58 smithi119 ceph-mon[36188]: Updating smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:49:58.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:58 smithi119 ceph-mon[36188]: Updating smithi018:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:49:58.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:58 smithi119 ceph-mon[36188]: Updating smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T12:49:58.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:49:58 smithi119 ceph-mon[36188]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-02-17T12:50:00.439 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:50:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:50:00.158+0000 7fe66be9e700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:50:00.439 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:50:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:50:00.158+0000 7fe66be9e700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:50:00.439 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:50:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:50:00.158+0000 7fe66be9e700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:50:00.439 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:50:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:50:00.158+0000 7fe66be9e700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:50:00.439 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 12:50:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T12:50:00.158+0000 7fe66be9e700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T12:50:00.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:00 smithi146 ceph-mon[36454]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:50:00.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:00 smithi146 ceph-mon[36454]: overall HEALTH_OK 2024-02-17T12:50:00.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:00 smithi018 ceph-mon[32625]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:50:00.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:00 smithi018 ceph-mon[32625]: overall HEALTH_OK 2024-02-17T12:50:00.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:00 smithi119 ceph-mon[36188]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T12:50:00.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:00 smithi119 ceph-mon[36188]: overall HEALTH_OK 2024-02-17T12:50:02.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:02 smithi146 ceph-mon[36454]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-02-17T12:50:02.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:02 smithi146 ceph-mon[36454]: mgrmap e25: x(active, since 8s), standbys: y, z 2024-02-17T12:50:02.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:02 smithi018 ceph-mon[32625]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-02-17T12:50:02.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:02 smithi018 ceph-mon[32625]: mgrmap e25: x(active, since 8s), standbys: y, z 2024-02-17T12:50:02.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:02 smithi119 ceph-mon[36188]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-02-17T12:50:02.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:02 smithi119 ceph-mon[36188]: mgrmap e25: x(active, since 8s), standbys: y, z 2024-02-17T12:50:04.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:04 smithi146 ceph-mon[36454]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-02-17T12:50:04.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:04 smithi018 ceph-mon[32625]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-02-17T12:50:04.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:04 smithi119 ceph-mon[36188]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-02-17T12:50:06.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:06 smithi146 ceph-mon[36454]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 109 KiB/s wr, 72 op/s 2024-02-17T12:50:06.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:06 smithi018 ceph-mon[32625]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 109 KiB/s wr, 72 op/s 2024-02-17T12:50:06.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:06 smithi119 ceph-mon[36188]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 109 KiB/s wr, 72 op/s 2024-02-17T12:50:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.187+0000 7f13c424e640 1 -- 172.21.15.18:0/322775690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13bc100c90 msgr2=0x7f13bc10ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.187+0000 7f13c424e640 1 --2- 172.21.15.18:0/322775690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13bc100c90 0x7f13bc10ceb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009ca0 tx=0x7f13b402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 -- 172.21.15.18:0/322775690 shutdown_connections 2024-02-17T12:50:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 --2- 172.21.15.18:0/322775690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13bc100c90 0x7f13bc10ceb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 --2- 172.21.15.18:0/322775690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13bc1002d0 0x7f13bc100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 --2- 172.21.15.18:0/322775690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13bc105020 0x7f13bc105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 -- 172.21.15.18:0/322775690 >> 172.21.15.18:0/322775690 conn(0x7f13bc076ed0 msgr2=0x7f13bc0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 -- 172.21.15.18:0/322775690 shutdown_connections 2024-02-17T12:50:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 -- 172.21.15.18:0/322775690 wait complete. 2024-02-17T12:50:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 Processor -- start 2024-02-17T12:50:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 -- start start 2024-02-17T12:50:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13bc1002d0 0x7f13bc197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13bc100c90 0x7f13bc197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13bc105020 0x7f13bc19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc19c600 con 0x7f13bc1002d0 2024-02-17T12:50:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.188+0000 7f13c424e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc19c770 con 0x7f13bc100c90 2024-02-17T12:50:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c424e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc19c930 con 0x7f13bc105020 2024-02-17T12:50:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c1fea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13bc1002d0 0x7f13bc197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c1fea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13bc1002d0 0x7f13bc197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51438/0 (socket says 172.21.15.18:51438) 2024-02-17T12:50:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c1fea640 1 -- 172.21.15.18:0/1038456188 learned_addr learned my addr 172.21.15.18:0/1038456188 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c17e9640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13bc100c90 0x7f13bc197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c27eb640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13bc105020 0x7f13bc19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c1fea640 1 -- 172.21.15.18:0/1038456188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13bc105020 msgr2=0x7f13bc19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c1fea640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13bc105020 0x7f13bc19c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c1fea640 1 -- 172.21.15.18:0/1038456188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13bc100c90 msgr2=0x7f13bc197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c1fea640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13bc100c90 0x7f13bc197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c1fea640 1 -- 172.21.15.18:0/1038456188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b4009900 con 0x7f13bc1002d0 2024-02-17T12:50:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c27eb640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13bc105020 0x7f13bc19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:50:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c17e9640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13bc100c90 0x7f13bc197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c1fea640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13bc1002d0 0x7f13bc197740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f13b800d770 tx=0x7f13b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13b2ffd640 1 -- 172.21.15.18:0/1038456188 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8009c30 con 0x7f13bc1002d0 2024-02-17T12:50:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c424e640 1 -- 172.21.15.18:0/1038456188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13bc19cc10 con 0x7f13bc1002d0 2024-02-17T12:50:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13b2ffd640 1 -- 172.21.15.18:0/1038456188 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13b800de80 con 0x7f13bc1002d0 2024-02-17T12:50:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13c424e640 1 -- 172.21.15.18:0/1038456188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13bc1a4490 con 0x7f13bc1002d0 2024-02-17T12:50:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.189+0000 7f13b2ffd640 1 -- 172.21.15.18:0/1038456188 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8004e80 con 0x7f13bc1002d0 2024-02-17T12:50:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.192+0000 7f13b2ffd640 1 -- 172.21.15.18:0/1038456188 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f13b8005020 con 0x7f13bc1002d0 2024-02-17T12:50:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.192+0000 7f13c6e26740 1 Processor -- start 2024-02-17T12:50:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c6e26740 1 -- start start 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c6e26740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc46820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c6e26740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 0x55e8cfc471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c6e26740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc502e0 0x55e8cfc526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c6e26740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cfbb59f0 con 0x55e8cfc46d60 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c6e26740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cfbb2f20 con 0x55e8cfc47ad0 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c6e26740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cfbae700 con 0x55e8cfc502e0 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c1fea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 0x55e8cfc471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c1fea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 0x55e8cfc471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51450/0 (socket says 172.21.15.18:51450) 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c1fea640 1 -- 172.21.15.18:0/238513588 learned_addr learned my addr 172.21.15.18:0/238513588 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c17e9640 1 --2- 172.21.15.18:0/238513588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc502e0 0x55e8cfc526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c27eb640 1 --2- 172.21.15.18:0/238513588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc46820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c1fea640 1 -- 172.21.15.18:0/238513588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc502e0 msgr2=0x55e8cfc526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c1fea640 1 --2- 172.21.15.18:0/238513588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc502e0 0x55e8cfc526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c1fea640 1 -- 172.21.15.18:0/238513588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc47ad0 msgr2=0x55e8cfc46820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c1fea640 1 --2- 172.21.15.18:0/238513588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc46820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13b2ffd640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13a80a23f0 0x7f13a80a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c1fea640 1 -- 172.21.15.18:0/238513588 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8cfb447b0 con 0x55e8cfc46d60 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.193+0000 7f13c27eb640 1 --2- 172.21.15.18:0/238513588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc46820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.194+0000 7f13c27eb640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13a80a23f0 0x7f13a80a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.194+0000 7f13c1fea640 1 --2- 172.21.15.18:0/238513588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 0x55e8cfc471e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f13b80cb4d0 tx=0x7f13b80d4820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f921e2a85ae3279 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.194+0000 7f13b2ffd640 1 -- 172.21.15.18:0/1038456188 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f13b80cdda0 con 0x7f13bc1002d0 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.194+0000 7f13a7fff640 1 -- 172.21.15.18:0/238513588 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b809a360 con 0x55e8cfc46d60 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.194+0000 7f13a7fff640 1 -- 172.21.15.18:0/238513588 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13b809a0d0 con 0x55e8cfc46d60 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.194+0000 7f13a7fff640 1 -- 172.21.15.18:0/238513588 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b809a590 con 0x55e8cfc46d60 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.195+0000 7f13c27eb640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13a80a23f0 0x7f13a80a48b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13b4037040 tx=0x7f13b4032d60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.195+0000 7f13c6e26740 1 -- 172.21.15.18:0/238513588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 msgr2=0x55e8cfc471e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.195+0000 7f13c6e26740 1 --2- 172.21.15.18:0/238513588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 0x55e8cfc471e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f13b80cb4d0 tx=0x7f13b80d4820 comp rx=0 tx=0).stop 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.195+0000 7f13c6e26740 1 -- 172.21.15.18:0/238513588 shutdown_connections 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.195+0000 7f13c6e26740 1 --2- 172.21.15.18:0/238513588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc502e0 0x55e8cfc526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.195+0000 7f13c6e26740 1 --2- 172.21.15.18:0/238513588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 0x55e8cfc471e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.195+0000 7f13c6e26740 1 --2- 172.21.15.18:0/238513588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc46820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.195+0000 7f13c6e26740 1 -- 172.21.15.18:0/238513588 >> 172.21.15.18:0/238513588 conn(0x55e8cfc0e550 msgr2=0x55e8cfc41a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.195+0000 7f13c6e26740 1 -- 172.21.15.18:0/238513588 shutdown_connections 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.195+0000 7f13c6e26740 1 -- 172.21.15.18:0/238513588 wait complete. 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.195+0000 7f13c6e26740 1 -- start start 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.195+0000 7f13c6e26740 1 Processor -- start 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c6e26740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc46d60 0x55e8cfc4d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c6e26740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc4d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c6e26740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 0x55e8cfc4dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c6e26740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cfb53bf0 con 0x55e8cfc502e0 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c6e26740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cf882250 con 0x55e8cfc46d60 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c6e26740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cfb34200 con 0x55e8cfc47ad0 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c6e26740 10 client.0 _ll_register_callbacks cb 0x55e8cfba2d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c27eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 0x55e8cfc4dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c27eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 0x55e8cfc4dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51464/0 (socket says 172.21.15.18:51464) 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c27eb640 1 -- 172.21.15.18:0/1568966157 learned_addr learned my addr 172.21.15.18:0/1568966157 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c17e9640 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc4d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c1fea640 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc46d60 0x55e8cfc4d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c27eb640 1 -- 172.21.15.18:0/1568966157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 msgr2=0x55e8cfc4d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c27eb640 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc4d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c27eb640 1 -- 172.21.15.18:0/1568966157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc46d60 msgr2=0x55e8cfc4d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c27eb640 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc46d60 0x55e8cfc4d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.196+0000 7f13c27eb640 1 -- 172.21.15.18:0/1568966157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13bc1a4490 con 0x55e8cfc502e0 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13c27eb640 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 0x55e8cfc4dda0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f13b4034d10 tx=0x7f13b40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8ab5157ac420b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13a57fa640 1 -- 172.21.15.18:0/1568966157 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b4045040 con 0x55e8cfc502e0 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13a57fa640 1 -- 172.21.15.18:0/1568966157 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f13b4003b70 con 0x55e8cfc502e0 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13c6e26740 1 -- 172.21.15.18:0/1568966157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8cf8ae600 con 0x55e8cfc502e0 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13a57fa640 1 -- 172.21.15.18:0/1568966157 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13b4003d30 con 0x55e8cfc502e0 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13c6e26740 10 client.15624 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13c6e26740 1 -- 172.21.15.18:0/1568966157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e8cfbacd80 con 0x55e8cfc502e0 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13a57fa640 1 -- 172.21.15.18:0/1568966157 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e8cf8ae600 con 0x55e8cfc502e0 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13a57fa640 1 -- 172.21.15.18:0/1568966157 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f13b404b490 con 0x55e8cfc502e0 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13c6e26740 10 client.15624 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13c6e26740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13c6e26740 20 client.15624 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13c6e26740 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e8cfc606c0 0x55e8cfc62ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13c6e26740 4 client.15624 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.197+0000 7f13c6e26740 1 -- 172.21.15.18:0/1568966157 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8cf8ff1b0 con 0x55e8cfc606c0 2024-02-17T12:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.198+0000 7f13c1fea640 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e8cfc606c0 0x55e8cfc62ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.198+0000 7f13c1fea640 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e8cfc606c0 0x55e8cfc62ae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be085606b2714324 server_cookie=ec3f6f84c06ace44 in_seq=0 out_seq=0 2024-02-17T12:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.198+0000 7f13a57fa640 10 client.15624 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13a57fa640 1 -- 172.21.15.18:0/1568966157 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e8cf8ff1b0 con 0x55e8cfc606c0 2024-02-17T12:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13a57fa640 10 client.15624 handle_command_reply: tid=0 2024-02-17T12:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13c6e26740 1 client.15624 shutdown 2024-02-17T12:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13c6e26740 1 -- 172.21.15.18:0/1568966157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 msgr2=0x55e8cfc4dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13c6e26740 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 0x55e8cfc4dda0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f13b4034d10 tx=0x7f13b40095c0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13c6e26740 1 -- 172.21.15.18:0/1568966157 shutdown_connections 2024-02-17T12:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13c6e26740 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e8cfc606c0 0x55e8cfc62ae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13c6e26740 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 0x55e8cfc4dda0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13c6e26740 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc4d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13a57fa640 0 client.15624 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13c6e26740 1 --2- 172.21.15.18:0/1568966157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc46d60 0x55e8cfc4d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13c6e26740 1 -- 172.21.15.18:0/1568966157 >> 172.21.15.18:0/1568966157 conn(0x55e8cfc0e550 msgr2=0x55e8cfc437a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13c6e26740 1 -- 172.21.15.18:0/1568966157 shutdown_connections 2024-02-17T12:50:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13c6e26740 1 -- 172.21.15.18:0/1568966157 wait complete. 2024-02-17T12:50:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.199+0000 7f13c6e26740 20 client.15624 trim_cache size 0 max 16384 2024-02-17T12:50:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.226+0000 7f13c6e26740 1 Processor -- start 2024-02-17T12:50:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.226+0000 7f13c6e26740 1 -- start start 2024-02-17T12:50:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.226+0000 7f13c6e26740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc46d60 0x55e8cfc57110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.226+0000 7f13c6e26740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc57650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.226+0000 7f13c6e26740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 0x55e8cfc48600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.226+0000 7f13c6e26740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cfc42c80 con 0x55e8cfc502e0 2024-02-17T12:50:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.226+0000 7f13c6e26740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cfc41310 con 0x55e8cfc46d60 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.226+0000 7f13c6e26740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cfc42a50 con 0x55e8cfc47ad0 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.226+0000 7f13c27eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 0x55e8cfc48600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.226+0000 7f13c27eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 0x55e8cfc48600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51472/0 (socket says 172.21.15.18:51472) 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.226+0000 7f13c27eb640 1 -- 172.21.15.18:0/4278954148 learned_addr learned my addr 172.21.15.18:0/4278954148 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.227+0000 7f13c27eb640 1 -- 172.21.15.18:0/4278954148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 msgr2=0x55e8cfc57650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.227+0000 7f13c17e9640 1 --2- 172.21.15.18:0/4278954148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc57650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.227+0000 7f13c27eb640 1 --2- 172.21.15.18:0/4278954148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc57650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.227+0000 7f13c1fea640 1 --2- 172.21.15.18:0/4278954148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc46d60 0x55e8cfc57110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.227+0000 7f13c27eb640 1 -- 172.21.15.18:0/4278954148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc46d60 msgr2=0x55e8cfc57110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.227+0000 7f13c27eb640 1 --2- 172.21.15.18:0/4278954148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc46d60 0x55e8cfc57110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.227+0000 7f13c27eb640 1 -- 172.21.15.18:0/4278954148 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8cfb497b0 con 0x55e8cfc502e0 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.227+0000 7f13c17e9640 1 --2- 172.21.15.18:0/4278954148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc57650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:50:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.227+0000 7f13c27eb640 1 --2- 172.21.15.18:0/4278954148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 0x55e8cfc48600 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f13b40093d0 tx=0x7f13b4032600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e468edb5b7fc5364 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.227+0000 7f13a77fe640 1 -- 172.21.15.18:0/4278954148 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b404b470 con 0x55e8cfc502e0 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.227+0000 7f13a77fe640 1 -- 172.21.15.18:0/4278954148 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13b404b5d0 con 0x55e8cfc502e0 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.227+0000 7f13a77fe640 1 -- 172.21.15.18:0/4278954148 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b4004430 con 0x55e8cfc502e0 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.228+0000 7f13c6e26740 1 -- 172.21.15.18:0/4278954148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 msgr2=0x55e8cfc48600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.228+0000 7f13c6e26740 1 --2- 172.21.15.18:0/4278954148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 0x55e8cfc48600 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f13b40093d0 tx=0x7f13b4032600 comp rx=0 tx=0).stop 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.228+0000 7f13c6e26740 1 -- 172.21.15.18:0/4278954148 shutdown_connections 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.228+0000 7f13c6e26740 1 --2- 172.21.15.18:0/4278954148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc502e0 0x55e8cfc48600 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.228+0000 7f13c6e26740 1 --2- 172.21.15.18:0/4278954148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfc57650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.228+0000 7f13c6e26740 1 --2- 172.21.15.18:0/4278954148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc46d60 0x55e8cfc57110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.228+0000 7f13c6e26740 1 -- 172.21.15.18:0/4278954148 >> 172.21.15.18:0/4278954148 conn(0x55e8cfc0e550 msgr2=0x55e8cfc67400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.228+0000 7f13c6e26740 1 -- 172.21.15.18:0/4278954148 shutdown_connections 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.228+0000 7f13c6e26740 1 -- 172.21.15.18:0/4278954148 wait complete. 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.228+0000 7f13c6e26740 1 -- start start 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.228+0000 7f13c6e26740 1 Processor -- start 2024-02-17T12:50:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c6e26740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 0x55e8cfcaf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c6e26740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfcaf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c6e26740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc502e0 0x55e8cfcb9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c6e26740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cfcba4e0 con 0x55e8cfc46d60 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c6e26740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cfcba620 con 0x55e8cfc502e0 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c6e26740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cfcba760 con 0x55e8cfc47ad0 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c6e26740 10 client.0 _ll_register_callbacks cb 0x7f13b80d1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c1fea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 0x55e8cfcaf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c1fea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 0x55e8cfcaf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51482/0 (socket says 172.21.15.18:51482) 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c1fea640 1 -- 172.21.15.18:0/3414606433 learned_addr learned my addr 172.21.15.18:0/3414606433 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c27eb640 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc502e0 0x55e8cfcb9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c17e9640 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfcaf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c1fea640 1 -- 172.21.15.18:0/3414606433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 msgr2=0x55e8cfcaf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c1fea640 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfcaf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c1fea640 1 -- 172.21.15.18:0/3414606433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc502e0 msgr2=0x55e8cfcb9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c1fea640 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc502e0 0x55e8cfcb9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c1fea640 1 -- 172.21.15.18:0/3414606433 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8cfbacd80 con 0x55e8cfc46d60 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c1fea640 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 0x55e8cfcaf110 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f13b800d740 tx=0x7f13b80cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7854e59c3a37d74 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13a57fa640 1 -- 172.21.15.18:0/3414606433 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8099ef0 con 0x55e8cfc46d60 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13a57fa640 1 -- 172.21.15.18:0/3414606433 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f13b80d1990 con 0x55e8cfc46d60 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.229+0000 7f13c6e26740 1 -- 172.21.15.18:0/3414606433 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13b809a0d0 con 0x55e8cfc46d60 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.230+0000 7f13a57fa640 1 -- 172.21.15.18:0/3414606433 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13b809a360 con 0x55e8cfc46d60 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.230+0000 7f13c6e26740 10 client.15630 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.230+0000 7f13c6e26740 1 -- 172.21.15.18:0/3414606433 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e8cfb6f540 con 0x55e8cfc46d60 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.230+0000 7f13a57fa640 1 -- 172.21.15.18:0/3414606433 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13b809a0d0 con 0x55e8cfc46d60 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.230+0000 7f13a57fa640 1 -- 172.21.15.18:0/3414606433 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f13b80d1d10 con 0x55e8cfc46d60 2024-02-17T12:50:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.230+0000 7f13c6e26740 10 client.15630 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.230+0000 7f13c6e26740 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.230+0000 7f13c6e26740 20 client.15630 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.230+0000 7f13c6e26740 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e8cfcbba70 0x55e8cfcbde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.230+0000 7f13c6e26740 4 client.15630 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:50:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.230+0000 7f13c6e26740 1 -- 172.21.15.18:0/3414606433 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e8cf8eb550 con 0x55e8cfcbba70 2024-02-17T12:50:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.230+0000 7f13c17e9640 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e8cfcbba70 0x55e8cfcbde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.231+0000 7f13a57fa640 10 client.15630 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.231+0000 7f13c17e9640 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e8cfcbba70 0x55e8cfcbde60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d57c1299b36599fd server_cookie=eded95955a3f147 in_seq=0 out_seq=0 2024-02-17T12:50:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.242+0000 7f13a57fa640 1 -- 172.21.15.18:0/3414606433 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e8cf8eb550 con 0x55e8cfcbba70 2024-02-17T12:50:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.242+0000 7f13a57fa640 10 client.15630 handle_command_reply: tid=0 2024-02-17T12:50:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.242+0000 7f13c6e26740 1 client.15630 shutdown 2024-02-17T12:50:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.242+0000 7f13c6e26740 1 -- 172.21.15.18:0/3414606433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 msgr2=0x55e8cfcaf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.242+0000 7f13c6e26740 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 0x55e8cfcaf110 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f13b800d740 tx=0x7f13b80cd040 comp rx=0 tx=0).stop 2024-02-17T12:50:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.242+0000 7f13c6e26740 1 -- 172.21.15.18:0/3414606433 shutdown_connections 2024-02-17T12:50:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.242+0000 7f13c6e26740 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e8cfcbba70 0x55e8cfcbde60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.242+0000 7f13c6e26740 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8cfc502e0 0x55e8cfcb9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.242+0000 7f13c6e26740 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e8cfc47ad0 0x55e8cfcaf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.242+0000 7f13a57fa640 0 client.15630 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.242+0000 7f13c6e26740 1 --2- 172.21.15.18:0/3414606433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e8cfc46d60 0x55e8cfcaf110 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.242+0000 7f13c6e26740 1 -- 172.21.15.18:0/3414606433 >> 172.21.15.18:0/3414606433 conn(0x55e8cfc0e550 msgr2=0x55e8cfc48600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c6e26740 1 -- 172.21.15.18:0/3414606433 shutdown_connections 2024-02-17T12:50:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c6e26740 1 -- 172.21.15.18:0/3414606433 wait complete. 2024-02-17T12:50:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c6e26740 20 client.15630 trim_cache size 0 max 16384 2024-02-17T12:50:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:50:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:50:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:50:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:50:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c424e640 1 -- 172.21.15.18:0/1038456188 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13a80a23f0 msgr2=0x7f13a80a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c424e640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13a80a23f0 0x7f13a80a48b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13b4037040 tx=0x7f13b4032d60 comp rx=0 tx=0).stop 2024-02-17T12:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c424e640 1 -- 172.21.15.18:0/1038456188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13bc1002d0 msgr2=0x7f13bc197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c424e640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13bc1002d0 0x7f13bc197740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f13b800d770 tx=0x7f13b800dc30 comp rx=0 tx=0).stop 2024-02-17T12:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c424e640 1 -- 172.21.15.18:0/1038456188 shutdown_connections 2024-02-17T12:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c424e640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13a80a23f0 0x7f13a80a48b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c424e640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13bc105020 0x7f13bc19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c424e640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13bc100c90 0x7f13bc197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c424e640 1 --2- 172.21.15.18:0/1038456188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13bc1002d0 0x7f13bc197740 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c424e640 1 -- 172.21.15.18:0/1038456188 >> 172.21.15.18:0/1038456188 conn(0x7f13bc076ed0 msgr2=0x7f13bc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c424e640 1 -- 172.21.15.18:0/1038456188 shutdown_connections 2024-02-17T12:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:08.243+0000 7f13c424e640 1 -- 172.21.15.18:0/1038456188 wait complete. 2024-02-17T12:50:08.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:08 smithi018 ceph-mon[32625]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-02-17T12:50:08.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:08 smithi119 ceph-mon[36188]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-02-17T12:50:08.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:08 smithi146 ceph-mon[36454]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-02-17T12:50:10.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:10 smithi018 ceph-mon[32625]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-02-17T12:50:10.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:10 smithi119 ceph-mon[36188]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-02-17T12:50:10.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:10 smithi146 ceph-mon[36454]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-02-17T12:50:12.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:12 smithi018 ceph-mon[32625]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 102 KiB/s wr, 75 op/s 2024-02-17T12:50:12.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:12 smithi119 ceph-mon[36188]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 102 KiB/s wr, 75 op/s 2024-02-17T12:50:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:12 smithi146 ceph-mon[36454]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 102 KiB/s wr, 75 op/s 2024-02-17T12:50:14.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:50:14.694 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:50:14.731 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:14 smithi018 ceph-mon[32625]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-02-17T12:50:14.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:14 smithi119 ceph-mon[36188]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-02-17T12:50:14.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.873+0000 7f2b59655640 1 -- 172.21.15.18:0/920295667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b54105340 msgr2=0x7f2b5410cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:14.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.873+0000 7f2b59655640 1 --2- 172.21.15.18:0/920295667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b54105340 0x7f2b5410cf10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009cd0 tx=0x7f2b4c02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.873+0000 7f2b59655640 1 -- 172.21.15.18:0/920295667 shutdown_connections 2024-02-17T12:50:14.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.873+0000 7f2b59655640 1 --2- 172.21.15.18:0/920295667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b54105340 0x7f2b5410cf10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.873+0000 7f2b59655640 1 --2- 172.21.15.18:0/920295667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b54100c20 0x7f2b54104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.873+0000 7f2b59655640 1 --2- 172.21.15.18:0/920295667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b54100270 0x7f2b54100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.873+0000 7f2b59655640 1 -- 172.21.15.18:0/920295667 >> 172.21.15.18:0/920295667 conn(0x7f2b54077230 msgr2=0x7f2b54077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:14.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.873+0000 7f2b59655640 1 -- 172.21.15.18:0/920295667 shutdown_connections 2024-02-17T12:50:14.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.873+0000 7f2b59655640 1 -- 172.21.15.18:0/920295667 wait complete. 2024-02-17T12:50:14.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.874+0000 7f2b59655640 1 Processor -- start 2024-02-17T12:50:14.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.874+0000 7f2b59655640 1 -- start start 2024-02-17T12:50:14.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.874+0000 7f2b59655640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b54100270 0x7f2b54197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.874+0000 7f2b59655640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b54105340 0x7f2b54197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.874+0000 7f2b59655640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b5419bf70 0x7f2b5419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.874+0000 7f2b59655640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5419c960 con 0x7f2b54100270 2024-02-17T12:50:14.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.874+0000 7f2b59655640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5419cad0 con 0x7f2b54105340 2024-02-17T12:50:14.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.874+0000 7f2b59655640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5419cc40 con 0x7f2b5419bf70 2024-02-17T12:50:14.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.874+0000 7f2b52ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b54100270 0x7f2b54197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b52ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b54100270 0x7f2b54197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52094/0 (socket says 172.21.15.18:52094) 2024-02-17T12:50:14.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b52ffd640 1 -- 172.21.15.18:0/1002912165 learned_addr learned my addr 172.21.15.18:0/1002912165 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:14.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b537fe640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b5419bf70 0x7f2b5419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b527fc640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b54105340 0x7f2b54197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b52ffd640 1 -- 172.21.15.18:0/1002912165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b5419bf70 msgr2=0x7f2b5419c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b52ffd640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b5419bf70 0x7f2b5419c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b52ffd640 1 -- 172.21.15.18:0/1002912165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b54105340 msgr2=0x7f2b54197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b52ffd640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b54105340 0x7f2b54197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b52ffd640 1 -- 172.21.15.18:0/1002912165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4c009900 con 0x7f2b54100270 2024-02-17T12:50:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b527fc640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b54105340 0x7f2b54197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b52ffd640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b54100270 0x7f2b54197700 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400d770 tx=0x7f2b4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b3bfff640 1 -- 172.21.15.18:0/1002912165 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44009c30 con 0x7f2b54100270 2024-02-17T12:50:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b3bfff640 1 -- 172.21.15.18:0/1002912165 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b4400de80 con 0x7f2b54100270 2024-02-17T12:50:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b3bfff640 1 -- 172.21.15.18:0/1002912165 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44004e80 con 0x7f2b54100270 2024-02-17T12:50:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.875+0000 7f2b59655640 1 -- 172.21.15.18:0/1002912165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b541a4220 con 0x7f2b54100270 2024-02-17T12:50:14.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.876+0000 7f2b59655640 1 -- 172.21.15.18:0/1002912165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b541a4680 con 0x7f2b54100270 2024-02-17T12:50:14.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.879+0000 7f2b5c221740 1 Processor -- start 2024-02-17T12:50:14.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.879+0000 7f2b3bfff640 1 -- 172.21.15.18:0/1002912165 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f2b44005020 con 0x7f2b54100270 2024-02-17T12:50:14.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.879+0000 7f2b5c221740 1 -- start start 2024-02-17T12:50:14.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.879+0000 7f2b5c221740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df920690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.879+0000 7f2b5c221740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df920bd0 0x55b7df929270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.879+0000 7f2b537fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df920690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.879+0000 7f2b537fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df920690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52102/0 (socket says 172.21.15.18:52102) 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.880+0000 7f2b52ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df920bd0 0x55b7df929270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.880+0000 7f2b52ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df920bd0 0x55b7df929270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:45614/0 (socket says 172.21.15.18:45614) 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.880+0000 7f2b3bfff640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b3c0a2450 0x7f2b3c0a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.879+0000 7f2b5c221740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9297b0 0x55b7df92bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.880+0000 7f2b5c221740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7df88f280 con 0x55b7df9202b0 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.880+0000 7f2b5c221740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7df88c7b0 con 0x55b7df920bd0 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.880+0000 7f2b5c221740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7df887f90 con 0x55b7df9297b0 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.880+0000 7f2b537fe640 1 -- 172.21.15.18:0/3139651944 learned_addr learned my addr 172.21.15.18:0/3139651944 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.880+0000 7f2b537fe640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b3c0a2450 0x7f2b3c0a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.880+0000 7f2b3bfff640 1 -- 172.21.15.18:0/1002912165 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2b440cdd40 con 0x7f2b54100270 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.881+0000 7f2b537fe640 1 -- 172.21.15.18:0/3139651944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9297b0 msgr2=0x55b7df92bba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.881+0000 7f2b537fe640 1 --2- 172.21.15.18:0/3139651944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9297b0 0x55b7df92bba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.881+0000 7f2b537fe640 1 -- 172.21.15.18:0/3139651944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df920bd0 msgr2=0x55b7df929270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:14.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.881+0000 7f2b537fe640 1 --2- 172.21.15.18:0/3139651944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df920bd0 0x55b7df929270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.881+0000 7f2b537fe640 1 -- 172.21.15.18:0/3139651944 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7df846570 con 0x55b7df9202b0 2024-02-17T12:50:14.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.881+0000 7f2b537fe640 1 --2- 172.21.15.18:0/3139651944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df920690 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009890 tx=0x7f2b4c034600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5d9143c0659c9b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.881+0000 7f2b397fa640 1 -- 172.21.15.18:0/3139651944 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c0328b0 con 0x55b7df9202b0 2024-02-17T12:50:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.881+0000 7f2b397fa640 1 -- 172.21.15.18:0/3139651944 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b4c003e50 con 0x55b7df9202b0 2024-02-17T12:50:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.881+0000 7f2b397fa640 1 -- 172.21.15.18:0/3139651944 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c0045b0 con 0x55b7df9202b0 2024-02-17T12:50:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.881+0000 7f2b537fe640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b3c0a2450 0x7f2b3c0a4910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c006010 tx=0x7f2b4c04b000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.882+0000 7f2b5c221740 1 -- 172.21.15.18:0/3139651944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 msgr2=0x55b7df920690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.882+0000 7f2b5c221740 1 --2- 172.21.15.18:0/3139651944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df920690 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009890 tx=0x7f2b4c034600 comp rx=0 tx=0).stop 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.882+0000 7f2b5c221740 1 -- 172.21.15.18:0/3139651944 shutdown_connections 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.882+0000 7f2b5c221740 1 --2- 172.21.15.18:0/3139651944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9297b0 0x55b7df92bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.882+0000 7f2b5c221740 1 --2- 172.21.15.18:0/3139651944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df920bd0 0x55b7df929270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.882+0000 7f2b5c221740 1 --2- 172.21.15.18:0/3139651944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df920690 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.882+0000 7f2b5c221740 1 -- 172.21.15.18:0/3139651944 >> 172.21.15.18:0/3139651944 conn(0x55b7df91a990 msgr2=0x55b7df91ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.882+0000 7f2b5c221740 1 -- 172.21.15.18:0/3139651944 shutdown_connections 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.882+0000 7f2b5c221740 1 -- 172.21.15.18:0/3139651944 wait complete. 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.883+0000 7f2b5c221740 1 -- start start 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.883+0000 7f2b5c221740 1 Processor -- start 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.883+0000 7f2b5c221740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df929200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.883+0000 7f2b5c221740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df920bd0 0x55b7df926590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.883+0000 7f2b5c221740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9297b0 0x55b7df926cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.883+0000 7f2b5c221740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7df55b430 con 0x55b7df9202b0 2024-02-17T12:50:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.883+0000 7f2b5c221740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7df517e00 con 0x55b7df920bd0 2024-02-17T12:50:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.883+0000 7f2b5c221740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7df80cfd0 con 0x55b7df9297b0 2024-02-17T12:50:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.883+0000 7f2b5c221740 10 client.0 _ll_register_callbacks cb 0x55b7df862fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.883+0000 7f2b52ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df929200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.883+0000 7f2b52ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df929200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52106/0 (socket says 172.21.15.18:52106) 2024-02-17T12:50:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.883+0000 7f2b52ffd640 1 -- 172.21.15.18:0/986911455 learned_addr learned my addr 172.21.15.18:0/986911455 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.884+0000 7f2b527fc640 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df920bd0 0x55b7df926590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.884+0000 7f2b537fe640 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9297b0 0x55b7df926cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.884+0000 7f2b52ffd640 1 -- 172.21.15.18:0/986911455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9297b0 msgr2=0x55b7df926cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.884+0000 7f2b52ffd640 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9297b0 0x55b7df926cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.884+0000 7f2b52ffd640 1 -- 172.21.15.18:0/986911455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df920bd0 msgr2=0x55b7df926590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.884+0000 7f2b52ffd640 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df920bd0 0x55b7df926590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.884+0000 7f2b52ffd640 1 -- 172.21.15.18:0/986911455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5410d720 con 0x55b7df9202b0 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.884+0000 7f2b52ffd640 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df929200 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2b4409a460 tx=0x7f2b4409a090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e1b45a6688cbcf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.884+0000 7f2b36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.884+0000 7f2b36ff5640 1 -- 172.21.15.18:0/986911455 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44014020 con 0x55b7df9202b0 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.884+0000 7f2b36ff5640 1 -- 172.21.15.18:0/986911455 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2b4409a4d0 con 0x55b7df9202b0 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.885+0000 7f2b5c221740 1 -- 172.21.15.18:0/986911455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7df5877e0 con 0x55b7df9202b0 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.885+0000 7f2b36ff5640 1 -- 172.21.15.18:0/986911455 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b4409e620 con 0x55b7df9202b0 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.885+0000 7f2b5c221740 10 client.15642 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.885+0000 7f2b5c221740 1 -- 172.21.15.18:0/986911455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b7df886610 con 0x55b7df9202b0 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.885+0000 7f2b36ff5640 1 -- 172.21.15.18:0/986911455 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b7df5877e0 con 0x55b7df9202b0 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.885+0000 7f2b36ff5640 1 -- 172.21.15.18:0/986911455 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2b440cb5e0 con 0x55b7df9202b0 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.885+0000 7f2b5c221740 10 client.15642 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.885+0000 7f2b5c221740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.885+0000 7f2b5c221740 20 client.15642 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:14.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.885+0000 7f2b5c221740 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b7df939a00 0x55b7df93be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.885+0000 7f2b5c221740 4 client.15642 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:14.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.885+0000 7f2b5c221740 1 -- 172.21.15.18:0/986911455 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7df5c4720 con 0x55b7df939a00 2024-02-17T12:50:14.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.886+0000 7f2b527fc640 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b7df939a00 0x55b7df93be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.886+0000 7f2b36ff5640 10 client.15642 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:14.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.886+0000 7f2b527fc640 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b7df939a00 0x55b7df93be20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5f97eb045f5b85f server_cookie=7f812ef976630ba7 in_seq=0 out_seq=0 2024-02-17T12:50:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b36ff5640 1 -- 172.21.15.18:0/986911455 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b7df5c4720 con 0x55b7df939a00 2024-02-17T12:50:14.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b36ff5640 10 client.15642 handle_command_reply: tid=0 2024-02-17T12:50:14.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b5c221740 1 client.15642 shutdown 2024-02-17T12:50:14.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b5c221740 1 -- 172.21.15.18:0/986911455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 msgr2=0x55b7df929200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:14.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b5c221740 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df929200 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2b4409a460 tx=0x7f2b4409a090 comp rx=0 tx=0).stop 2024-02-17T12:50:14.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b5c221740 1 -- 172.21.15.18:0/986911455 shutdown_connections 2024-02-17T12:50:14.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b5c221740 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b7df939a00 0x55b7df93be20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b5c221740 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9297b0 0x55b7df926cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b5c221740 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df920bd0 0x55b7df926590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b5c221740 1 --2- 172.21.15.18:0/986911455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df929200 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b36ff5640 0 client.15642 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b5c221740 1 -- 172.21.15.18:0/986911455 >> 172.21.15.18:0/986911455 conn(0x55b7df91a990 msgr2=0x55b7df91c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b5c221740 1 -- 172.21.15.18:0/986911455 shutdown_connections 2024-02-17T12:50:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.887+0000 7f2b5c221740 1 -- 172.21.15.18:0/986911455 wait complete. 2024-02-17T12:50:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.888+0000 7f2b5c221740 20 client.15642 trim_cache size 0 max 16384 2024-02-17T12:50:14.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.918+0000 7f2b5c221740 1 Processor -- start 2024-02-17T12:50:14.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.918+0000 7f2b5c221740 1 -- start start 2024-02-17T12:50:14.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.918+0000 7f2b5c221740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9202b0 0x55b7df928cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.918+0000 7f2b5c221740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df920bd0 0x55b7df929200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.918+0000 7f2b5c221740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df9297b0 0x55b7df925e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.918+0000 7f2b5c221740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7df91f470 con 0x55b7df920bd0 2024-02-17T12:50:14.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.918+0000 7f2b5c221740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7df91ff10 con 0x55b7df9297b0 2024-02-17T12:50:14.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.918+0000 7f2b5c221740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7df9162b0 con 0x55b7df9202b0 2024-02-17T12:50:14.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.918+0000 7f2b527fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df920bd0 0x55b7df929200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.918+0000 7f2b527fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df920bd0 0x55b7df929200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52112/0 (socket says 172.21.15.18:52112) 2024-02-17T12:50:14.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.918+0000 7f2b527fc640 1 -- 172.21.15.18:0/2909345411 learned_addr learned my addr 172.21.15.18:0/2909345411 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:14.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.918+0000 7f2b52ffd640 1 --2- 172.21.15.18:0/2909345411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9202b0 0x55b7df928cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.919+0000 7f2b537fe640 1 --2- 172.21.15.18:0/2909345411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df9297b0 0x55b7df925e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.919+0000 7f2b527fc640 1 -- 172.21.15.18:0/2909345411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9202b0 msgr2=0x55b7df928cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:14.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.919+0000 7f2b527fc640 1 --2- 172.21.15.18:0/2909345411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9202b0 0x55b7df928cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.919+0000 7f2b527fc640 1 -- 172.21.15.18:0/2909345411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df9297b0 msgr2=0x55b7df925e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:14.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.919+0000 7f2b527fc640 1 --2- 172.21.15.18:0/2909345411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df9297b0 0x55b7df925e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.919+0000 7f2b527fc640 1 -- 172.21.15.18:0/2909345411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7df822ac0 con 0x55b7df920bd0 2024-02-17T12:50:14.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.919+0000 7f2b527fc640 1 --2- 172.21.15.18:0/2909345411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df920bd0 0x55b7df929200 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2b48008430 tx=0x7f2b48008310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee404735c322e4a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:14.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.919+0000 7f2b38ff9640 1 -- 172.21.15.18:0/2909345411 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48016040 con 0x55b7df920bd0 2024-02-17T12:50:14.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.919+0000 7f2b38ff9640 1 -- 172.21.15.18:0/2909345411 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b4801c070 con 0x55b7df920bd0 2024-02-17T12:50:14.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.919+0000 7f2b38ff9640 1 -- 172.21.15.18:0/2909345411 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48018580 con 0x55b7df920bd0 2024-02-17T12:50:14.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.920+0000 7f2b5c221740 1 -- 172.21.15.18:0/2909345411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df920bd0 msgr2=0x55b7df929200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:14.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.920+0000 7f2b5c221740 1 --2- 172.21.15.18:0/2909345411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df920bd0 0x55b7df929200 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2b48008430 tx=0x7f2b48008310 comp rx=0 tx=0).stop 2024-02-17T12:50:14.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.920+0000 7f2b5c221740 1 -- 172.21.15.18:0/2909345411 shutdown_connections 2024-02-17T12:50:14.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.920+0000 7f2b5c221740 1 --2- 172.21.15.18:0/2909345411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df9297b0 0x55b7df925e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.920+0000 7f2b5c221740 1 --2- 172.21.15.18:0/2909345411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df920bd0 0x55b7df929200 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.920+0000 7f2b5c221740 1 --2- 172.21.15.18:0/2909345411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df9202b0 0x55b7df928cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.920+0000 7f2b5c221740 1 -- 172.21.15.18:0/2909345411 >> 172.21.15.18:0/2909345411 conn(0x55b7df91a990 msgr2=0x55b7df924e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.920+0000 7f2b5c221740 1 -- 172.21.15.18:0/2909345411 shutdown_connections 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.920+0000 7f2b5c221740 1 -- 172.21.15.18:0/2909345411 wait complete. 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.921+0000 7f2b5c221740 1 -- start start 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.921+0000 7f2b5c221740 1 Processor -- start 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.921+0000 7f2b5c221740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df98bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.921+0000 7f2b5c221740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df920bd0 0x55b7df989140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.921+0000 7f2b5c221740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df9297b0 0x55b7df989880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.921+0000 7f2b5c221740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7df989f50 con 0x55b7df9202b0 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.921+0000 7f2b5c221740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7df92f660 con 0x55b7df9297b0 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.921+0000 7f2b52ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df98bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.921+0000 7f2b52ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df98bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52114/0 (socket says 172.21.15.18:52114) 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.921+0000 7f2b52ffd640 1 -- 172.21.15.18:0/213123039 learned_addr learned my addr 172.21.15.18:0/213123039 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:14.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.922+0000 7f2b5c221740 1 -- 172.21.15.18:0/213123039 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7df92f820 con 0x55b7df920bd0 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.922+0000 7f2b527fc640 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df920bd0 0x55b7df989140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.922+0000 7f2b5c221740 10 client.0 _ll_register_callbacks cb 0x7f2b4c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.922+0000 7f2b537fe640 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df9297b0 0x55b7df989880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.922+0000 7f2b527fc640 1 -- 172.21.15.18:0/213123039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df9297b0 msgr2=0x55b7df989880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.922+0000 7f2b527fc640 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df9297b0 0x55b7df989880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.922+0000 7f2b527fc640 1 -- 172.21.15.18:0/213123039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 msgr2=0x55b7df98bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.922+0000 7f2b527fc640 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df98bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.922+0000 7f2b527fc640 1 -- 172.21.15.18:0/213123039 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7df8486a0 con 0x55b7df920bd0 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.922+0000 7f2b52ffd640 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df98bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.922+0000 7f2b527fc640 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df920bd0 0x55b7df989140 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2b48008430 tx=0x7f2b48012980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac7ca4b7a45441a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.922+0000 7f2b36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b36ff5640 1 -- 172.21.15.18:0/213123039 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48004460 con 0x55b7df920bd0 2024-02-17T12:50:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b5c221740 1 -- 172.21.15.18:0/213123039 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b4c003e50 con 0x55b7df920bd0 2024-02-17T12:50:14.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b36ff5640 1 -- 172.21.15.18:0/213123039 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2b48012c30 con 0x55b7df920bd0 2024-02-17T12:50:14.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b36ff5640 1 -- 172.21.15.18:0/213123039 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b480182b0 con 0x55b7df920bd0 2024-02-17T12:50:14.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b5c221740 10 client.25222 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:14.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b5c221740 1 -- 172.21.15.18:0/213123039 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b7df82ee30 con 0x55b7df920bd0 2024-02-17T12:50:14.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b36ff5640 1 -- 172.21.15.18:0/213123039 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b4c003e50 con 0x55b7df920bd0 2024-02-17T12:50:14.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b36ff5640 1 -- 172.21.15.18:0/213123039 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2b48003cc0 con 0x55b7df920bd0 2024-02-17T12:50:14.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b5c221740 10 client.25222 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:14.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b5c221740 10 client.25222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:14.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b5c221740 20 client.25222 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b5c221740 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b7df99ab40 0x55b7df99cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b5c221740 4 client.25222 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.923+0000 7f2b5c221740 1 -- 172.21.15.18:0/213123039 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b7df613890 con 0x55b7df99ab40 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.924+0000 7f2b52ffd640 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b7df99ab40 0x55b7df99cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.924+0000 7f2b36ff5640 10 client.25222 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.924+0000 7f2b52ffd640 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b7df99ab40 0x55b7df99cf30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a513bc51511498ee server_cookie=968dfdb3ade0d802 in_seq=0 out_seq=0 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.924+0000 7f2b36ff5640 1 -- 172.21.15.18:0/213123039 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b7df613890 con 0x55b7df99ab40 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.924+0000 7f2b36ff5640 10 client.25222 handle_command_reply: tid=0 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.924+0000 7f2b5c221740 1 client.25222 shutdown 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.924+0000 7f2b5c221740 1 -- 172.21.15.18:0/213123039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df920bd0 msgr2=0x55b7df989140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b5c221740 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df920bd0 0x55b7df989140 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2b48008430 tx=0x7f2b48012980 comp rx=0 tx=0).stop 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b5c221740 1 -- 172.21.15.18:0/213123039 shutdown_connections 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b5c221740 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b7df99ab40 0x55b7df99cf30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b5c221740 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7df9297b0 0x55b7df989880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b5c221740 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b7df920bd0 0x55b7df989140 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b5c221740 1 --2- 172.21.15.18:0/213123039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7df9202b0 0x55b7df98bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b5c221740 1 -- 172.21.15.18:0/213123039 >> 172.21.15.18:0/213123039 conn(0x55b7df91a990 msgr2=0x55b7df925df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b36ff5640 0 client.25222 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b5c221740 1 -- 172.21.15.18:0/213123039 shutdown_connections 2024-02-17T12:50:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b5c221740 1 -- 172.21.15.18:0/213123039 wait complete. 2024-02-17T12:50:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b5c221740 20 client.25222 trim_cache size 0 max 16384 2024-02-17T12:50:14.923 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:50:14.924 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:50:14.924 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:50:14.924 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:50:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b59655640 1 -- 172.21.15.18:0/1002912165 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b3c0a2450 msgr2=0x7f2b3c0a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b59655640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b3c0a2450 0x7f2b3c0a4910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c006010 tx=0x7f2b4c04b000 comp rx=0 tx=0).stop 2024-02-17T12:50:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b59655640 1 -- 172.21.15.18:0/1002912165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b54100270 msgr2=0x7f2b54197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.925+0000 7f2b59655640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b54100270 0x7f2b54197700 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400d770 tx=0x7f2b4400dc30 comp rx=0 tx=0).stop 2024-02-17T12:50:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.926+0000 7f2b59655640 1 -- 172.21.15.18:0/1002912165 shutdown_connections 2024-02-17T12:50:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.926+0000 7f2b59655640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b3c0a2450 0x7f2b3c0a4910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.926+0000 7f2b59655640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b5419bf70 0x7f2b5419c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.926+0000 7f2b59655640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b54105340 0x7f2b54197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.926+0000 7f2b59655640 1 --2- 172.21.15.18:0/1002912165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b54100270 0x7f2b54197700 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.926+0000 7f2b59655640 1 -- 172.21.15.18:0/1002912165 >> 172.21.15.18:0/1002912165 conn(0x7f2b54077230 msgr2=0x7f2b54068560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.926+0000 7f2b59655640 1 -- 172.21.15.18:0/1002912165 shutdown_connections 2024-02-17T12:50:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:14.926+0000 7f2b59655640 1 -- 172.21.15.18:0/1002912165 wait complete. 2024-02-17T12:50:14.933 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:50:14.934 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:50:14.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:14 smithi146 ceph-mon[36454]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-02-17T12:50:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.089+0000 7fad2bedf640 1 -- 172.21.15.18:0/432763447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad24101d30 msgr2=0x7fad24105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.089+0000 7fad2bedf640 1 --2- 172.21.15.18:0/432763447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad24101d30 0x7fad24105d70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fad14009930 tx=0x7fad1402bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.089+0000 7fad2bedf640 1 -- 172.21.15.18:0/432763447 shutdown_connections 2024-02-17T12:50:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.089+0000 7fad2bedf640 1 --2- 172.21.15.18:0/432763447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad24106450 0x7fad2410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.089+0000 7fad2bedf640 1 --2- 172.21.15.18:0/432763447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad24101d30 0x7fad24105d70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.089+0000 7fad2bedf640 1 --2- 172.21.15.18:0/432763447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad24101380 0x7fad24101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.089+0000 7fad2bedf640 1 -- 172.21.15.18:0/432763447 >> 172.21.15.18:0/432763447 conn(0x7fad240fcc20 msgr2=0x7fad240ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.089+0000 7fad2bedf640 1 -- 172.21.15.18:0/432763447 shutdown_connections 2024-02-17T12:50:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.089+0000 7fad2bedf640 1 -- 172.21.15.18:0/432763447 wait complete. 2024-02-17T12:50:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad2bedf640 1 Processor -- start 2024-02-17T12:50:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad2bedf640 1 -- start start 2024-02-17T12:50:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad2bedf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad24101380 0x7fad2419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad2bedf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad24106450 0x7fad2419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad2bedf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad24199860 0x7fad24199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad2bedf640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2419a250 con 0x7fad24199860 2024-02-17T12:50:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad2bedf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2419a3c0 con 0x7fad24106450 2024-02-17T12:50:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad2bedf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2419a580 con 0x7fad24101380 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad2a47c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad24199860 0x7fad24199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad2a47c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad24199860 0x7fad24199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52130/0 (socket says 172.21.15.18:52130) 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad2a47c640 1 -- 172.21.15.18:0/105361699 learned_addr learned my addr 172.21.15.18:0/105361699 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad29c7b640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad24101380 0x7fad2419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.090+0000 7fad2947a640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad24106450 0x7fad2419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.091+0000 7fad2a47c640 1 -- 172.21.15.18:0/105361699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad24101380 msgr2=0x7fad2419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.091+0000 7fad2a47c640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad24101380 0x7fad2419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.091+0000 7fad2a47c640 1 -- 172.21.15.18:0/105361699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad24106450 msgr2=0x7fad2419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.091+0000 7fad2a47c640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad24106450 0x7fad2419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.091+0000 7fad2a47c640 1 -- 172.21.15.18:0/105361699 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad14009560 con 0x7fad24199860 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.091+0000 7fad2a47c640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad24199860 0x7fad24199d10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009fd0 tx=0x7fad1c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.091+0000 7fad1affd640 1 -- 172.21.15.18:0/105361699 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c00bea0 con 0x7fad24199860 2024-02-17T12:50:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.091+0000 7fad2bedf640 1 -- 172.21.15.18:0/105361699 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad241a6570 con 0x7fad24199860 2024-02-17T12:50:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.091+0000 7fad1affd640 1 -- 172.21.15.18:0/105361699 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad1c009940 con 0x7fad24199860 2024-02-17T12:50:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.091+0000 7fad1affd640 1 -- 172.21.15.18:0/105361699 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c0106c0 con 0x7fad24199860 2024-02-17T12:50:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.091+0000 7fad2bedf640 1 -- 172.21.15.18:0/105361699 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad241a6990 con 0x7fad24199860 2024-02-17T12:50:15.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.093+0000 7fad2bedf640 1 -- 172.21.15.18:0/105361699 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad24067f70 con 0x7fad24199860 2024-02-17T12:50:15.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.094+0000 7fad1affd640 1 -- 172.21.15.18:0/105361699 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fad1c004220 con 0x7fad24199860 2024-02-17T12:50:15.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.094+0000 7fad1affd640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fad100a23d0 0x7fad100a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.094+0000 7fad1affd640 1 -- 172.21.15.18:0/105361699 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fad1c0cdb90 con 0x7fad24199860 2024-02-17T12:50:15.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.095+0000 7fad29c7b640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fad100a23d0 0x7fad100a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.095+0000 7fad29c7b640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fad100a23d0 0x7fad100a4890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fad2419a9a0 tx=0x7fad2000b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.103+0000 7fad1affd640 1 -- 172.21.15.18:0/105361699 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad1c0966d0 con 0x7fad24199860 2024-02-17T12:50:15.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.552+0000 7fad2bedf640 1 -- 172.21.15.18:0/105361699 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad24101dc0 con 0x7fad24199860 2024-02-17T12:50:15.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.553+0000 7fad1affd640 1 -- 172.21.15.18:0/105361699 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11392 (secure 0 0 0) 0x7fad1c099d90 con 0x7fad24199860 2024-02-17T12:50:15.545 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:50:15.545 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:49:36.672377+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:50:15.546 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 26 2024-02-17T12:50:15.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.556+0000 7fad2bedf640 1 -- 172.21.15.18:0/105361699 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fad100a23d0 msgr2=0x7fad100a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.556+0000 7fad2bedf640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fad100a23d0 0x7fad100a4890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fad2419a9a0 tx=0x7fad2000b410 comp rx=0 tx=0).stop 2024-02-17T12:50:15.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.556+0000 7fad2bedf640 1 -- 172.21.15.18:0/105361699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad24199860 msgr2=0x7fad24199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.556+0000 7fad2bedf640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad24199860 0x7fad24199d10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009fd0 tx=0x7fad1c00b260 comp rx=0 tx=0).stop 2024-02-17T12:50:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.556+0000 7fad2bedf640 1 -- 172.21.15.18:0/105361699 shutdown_connections 2024-02-17T12:50:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.556+0000 7fad2bedf640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fad100a23d0 0x7fad100a4890 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.556+0000 7fad2bedf640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad24199860 0x7fad24199d10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.556+0000 7fad2bedf640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad24106450 0x7fad2419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.556+0000 7fad2bedf640 1 --2- 172.21.15.18:0/105361699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad24101380 0x7fad2419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.556+0000 7fad2bedf640 1 -- 172.21.15.18:0/105361699 >> 172.21.15.18:0/105361699 conn(0x7fad240fcc20 msgr2=0x7fad240fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.556+0000 7fad2bedf640 1 -- 172.21.15.18:0/105361699 shutdown_connections 2024-02-17T12:50:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.556+0000 7fad2bedf640 1 -- 172.21.15.18:0/105361699 wait complete. 2024-02-17T12:50:15.559 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:50:15.559 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:50:15.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.696+0000 7f910256c640 1 -- 172.21.15.18:0/2282075057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90fc06a0f0 msgr2=0x7f90fc10dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.696+0000 7f910256c640 1 --2- 172.21.15.18:0/2282075057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90fc06a0f0 0x7f90fc10dfd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f90f4009cd0 tx=0x7f90f402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.697+0000 7f910256c640 1 -- 172.21.15.18:0/2282075057 shutdown_connections 2024-02-17T12:50:15.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.697+0000 7f910256c640 1 --2- 172.21.15.18:0/2282075057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90fc06a0f0 0x7f90fc10dfd0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.697+0000 7f910256c640 1 --2- 172.21.15.18:0/2282075057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90fc069750 0x7f90fc069bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.697+0000 7f910256c640 1 --2- 172.21.15.18:0/2282075057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90fc1071c0 0x7f90fc1075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.697+0000 7f910256c640 1 -- 172.21.15.18:0/2282075057 >> 172.21.15.18:0/2282075057 conn(0x7f90fc075e80 msgr2=0x7f90fc076290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.697+0000 7f910256c640 1 -- 172.21.15.18:0/2282075057 shutdown_connections 2024-02-17T12:50:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.697+0000 7f910256c640 1 -- 172.21.15.18:0/2282075057 wait complete. 2024-02-17T12:50:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.697+0000 7f910256c640 1 Processor -- start 2024-02-17T12:50:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.697+0000 7f910256c640 1 -- start start 2024-02-17T12:50:15.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f910256c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90fc069750 0x7f90fc19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f910256c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90fc06a0f0 0x7f90fc19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f910256c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90fc1071c0 0x7f90fc1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f910256c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc199e40 con 0x7f90fc1071c0 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f910256c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc199fb0 con 0x7f90fc06a0f0 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f910256c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc19a170 con 0x7f90fc069750 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f9100b09640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90fc1071c0 0x7f90fc1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f9100b09640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90fc1071c0 0x7f90fc1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52154/0 (socket says 172.21.15.18:52154) 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f9100b09640 1 -- 172.21.15.18:0/2542839438 learned_addr learned my addr 172.21.15.18:0/2542839438 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f90fb7fe640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90fc06a0f0 0x7f90fc19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f90fbfff640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90fc069750 0x7f90fc19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f9100b09640 1 -- 172.21.15.18:0/2542839438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90fc069750 msgr2=0x7f90fc19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f9100b09640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90fc069750 0x7f90fc19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f9100b09640 1 -- 172.21.15.18:0/2542839438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90fc06a0f0 msgr2=0x7f90fc19fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f9100b09640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90fc06a0f0 0x7f90fc19fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.698+0000 7f9100b09640 1 -- 172.21.15.18:0/2542839438 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4009900 con 0x7f90fc1071c0 2024-02-17T12:50:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.699+0000 7f9100b09640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90fc1071c0 0x7f90fc1997a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f90f4004af0 tx=0x7f90f4032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.699+0000 7f90f97fa640 1 -- 172.21.15.18:0/2542839438 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f40328b0 con 0x7f90fc1071c0 2024-02-17T12:50:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.699+0000 7f90f97fa640 1 -- 172.21.15.18:0/2542839438 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90f4003e50 con 0x7f90fc1071c0 2024-02-17T12:50:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.699+0000 7f910256c640 1 -- 172.21.15.18:0/2542839438 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90fc19a3f0 con 0x7f90fc1071c0 2024-02-17T12:50:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.699+0000 7f90f97fa640 1 -- 172.21.15.18:0/2542839438 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4004740 con 0x7f90fc1071c0 2024-02-17T12:50:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.699+0000 7f910256c640 1 -- 172.21.15.18:0/2542839438 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90fc1a6660 con 0x7f90fc1071c0 2024-02-17T12:50:15.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.701+0000 7f90f97fa640 1 -- 172.21.15.18:0/2542839438 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f90f400a040 con 0x7f90fc1071c0 2024-02-17T12:50:15.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.702+0000 7f90f97fa640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f90e40a2450 0x7f90e40a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.702+0000 7f90f97fa640 1 -- 172.21.15.18:0/2542839438 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f90f40f0df0 con 0x7f90fc1071c0 2024-02-17T12:50:15.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.702+0000 7f90fbfff640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f90e40a2450 0x7f90e40a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.702+0000 7f910521e740 1 Processor -- start 2024-02-17T12:50:15.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.702+0000 7f910521e740 1 -- start start 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f910521e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cf250 0x55afed1cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f910521e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cfb70 0x55afed1d8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f910521e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed1dab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f910521e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afed18a960 con 0x55afed1cf250 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f910521e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afed13e310 con 0x55afed1cfb70 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f910521e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afed13b840 con 0x55afed1d8750 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fbfff640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f90e40a2450 0x7f90e40a4910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009ae0 tx=0x7f90ec005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cf250 0x55afed1cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cfb70 0x55afed1d8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cf250 0x55afed1cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52156/0 (socket says 172.21.15.18:52156) 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cfb70 0x55afed1d8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:45674/0 (socket says 172.21.15.18:45674) 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fbfff640 1 -- 172.21.15.18:0/3551101357 learned_addr learned my addr 172.21.15.18:0/3551101357 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f9100b09640 1 --2- 172.21.15.18:0/3551101357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed1dab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fbfff640 1 -- 172.21.15.18:0/3551101357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 msgr2=0x55afed1dab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fbfff640 1 --2- 172.21.15.18:0/3551101357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed1dab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fbfff640 1 -- 172.21.15.18:0/3551101357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cfb70 msgr2=0x55afed1d8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fbfff640 1 --2- 172.21.15.18:0/3551101357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cfb70 0x55afed1d8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fbfff640 1 -- 172.21.15.18:0/3551101357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afed0f9060 con 0x55afed1cf250 2024-02-17T12:50:15.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fb7fe640 1 --2- 172.21.15.18:0/3551101357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cfb70 0x55afed1d8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:15.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.703+0000 7f90fbfff640 1 --2- 172.21.15.18:0/3551101357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cf250 0x55afed1cf630 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f90ec011f30 tx=0x7f90ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdbb116c3215b7f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.704+0000 7f90e27fc640 1 -- 172.21.15.18:0/3551101357 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec01f620 con 0x55afed1cf250 2024-02-17T12:50:15.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.704+0000 7f90e27fc640 1 -- 172.21.15.18:0/3551101357 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90ec016070 con 0x55afed1cf250 2024-02-17T12:50:15.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.704+0000 7f90e27fc640 1 -- 172.21.15.18:0/3551101357 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec01ebe0 con 0x55afed1cf250 2024-02-17T12:50:15.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.704+0000 7f910521e740 1 -- 172.21.15.18:0/3551101357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cf250 msgr2=0x55afed1cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.704+0000 7f910521e740 1 --2- 172.21.15.18:0/3551101357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cf250 0x55afed1cf630 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f90ec011f30 tx=0x7f90ec014000 comp rx=0 tx=0).stop 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.704+0000 7f910521e740 1 -- 172.21.15.18:0/3551101357 shutdown_connections 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.704+0000 7f910521e740 1 --2- 172.21.15.18:0/3551101357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed1dab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.704+0000 7f910521e740 1 --2- 172.21.15.18:0/3551101357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cfb70 0x55afed1d8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.704+0000 7f910521e740 1 --2- 172.21.15.18:0/3551101357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cf250 0x55afed1cf630 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.704+0000 7f910521e740 1 -- 172.21.15.18:0/3551101357 >> 172.21.15.18:0/3551101357 conn(0x55afed1c9980 msgr2=0x55afed1c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.704+0000 7f910521e740 1 -- 172.21.15.18:0/3551101357 shutdown_connections 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.705+0000 7f910521e740 1 -- 172.21.15.18:0/3551101357 wait complete. 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.705+0000 7f910521e740 1 -- start start 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.705+0000 7f910521e740 1 Processor -- start 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f910521e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cf250 0x55afed1d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f910521e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1d5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f910521e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed1d61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f910521e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afed137020 con 0x55afed1cfb70 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f910521e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afece0a430 con 0x55afed1cf250 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f910521e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afecdc6e00 con 0x55afed1d8750 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f910521e740 10 client.0 _ll_register_callbacks cb 0x55afed0decb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f90fbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cf250 0x55afed1d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f90fb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f90fb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1d5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52170/0 (socket says 172.21.15.18:52170) 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f90fb7fe640 1 -- 172.21.15.18:0/3155883525 learned_addr learned my addr 172.21.15.18:0/3155883525 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f90fbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cf250 0x55afed1d5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:45682/0 (socket says 172.21.15.18:45682) 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f9100b09640 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed1d61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f90fb7fe640 1 -- 172.21.15.18:0/3155883525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 msgr2=0x55afed1d61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f90fb7fe640 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed1d61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f90fb7fe640 1 -- 172.21.15.18:0/3155883525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cf250 msgr2=0x55afed1d5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f90fb7fe640 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cf250 0x55afed1d5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f90fb7fe640 1 -- 172.21.15.18:0/3155883525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ec011be0 con 0x55afed1cfb70 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f9100b09640 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed1d61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f90fb7fe640 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1d5c80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f90f00126e0 tx=0x7f90f0012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96397c435736dba6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.706+0000 7f90db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f90db7fe640 1 -- 172.21.15.18:0/3155883525 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0017070 con 0x55afed1cfb70 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f90db7fe640 1 -- 172.21.15.18:0/3155883525 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f90f00121d0 con 0x55afed1cfb70 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f910521e740 1 -- 172.21.15.18:0/3155883525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afece367e0 con 0x55afed1cfb70 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f90db7fe640 1 -- 172.21.15.18:0/3155883525 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90f0004050 con 0x55afed1cfb70 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f910521e740 10 client.15666 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f910521e740 1 -- 172.21.15.18:0/3155883525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55afed1356a0 con 0x55afed1cfb70 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f90db7fe640 1 -- 172.21.15.18:0/3155883525 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55afece367e0 con 0x55afed1cfb70 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f90db7fe640 1 -- 172.21.15.18:0/3155883525 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f90f0003820 con 0x55afed1cfb70 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f910521e740 10 client.15666 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f910521e740 10 client.15666 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f910521e740 20 client.15666 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f910521e740 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55afed1e8b50 0x55afed1eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f910521e740 4 client.15666 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.707+0000 7f910521e740 1 -- 172.21.15.18:0/3155883525 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afece73720 con 0x55afed1e8b50 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.708+0000 7f90fbfff640 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55afed1e8b50 0x55afed1eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.708+0000 7f90fbfff640 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55afed1e8b50 0x55afed1eaf70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d75cad422d768a1f server_cookie=640bc594f1840af0 in_seq=0 out_seq=0 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.708+0000 7f90db7fe640 10 client.15666 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f90db7fe640 1 -- 172.21.15.18:0/3155883525 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55afece73720 con 0x55afed1e8b50 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f90db7fe640 10 client.15666 handle_command_reply: tid=0 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f910521e740 1 client.15666 shutdown 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f910521e740 1 -- 172.21.15.18:0/3155883525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 msgr2=0x55afed1d5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f910521e740 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1d5c80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f90f00126e0 tx=0x7f90f0012ba0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f910521e740 1 -- 172.21.15.18:0/3155883525 shutdown_connections 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f910521e740 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed1d61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f910521e740 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1d5c80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f910521e740 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55afed1e8b50 0x55afed1eaf70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f910521e740 1 --2- 172.21.15.18:0/3155883525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cf250 0x55afed1d5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f90db7fe640 0 client.15666 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:50:15.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f910521e740 1 -- 172.21.15.18:0/3155883525 >> 172.21.15.18:0/3155883525 conn(0x55afed1c9980 msgr2=0x55afed1cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f910521e740 1 -- 172.21.15.18:0/3155883525 shutdown_connections 2024-02-17T12:50:15.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.709+0000 7f910521e740 1 -- 172.21.15.18:0/3155883525 wait complete. 2024-02-17T12:50:15.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.710+0000 7f910521e740 20 client.15666 trim_cache size 0 max 16384 2024-02-17T12:50:15.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.732+0000 7f910521e740 1 Processor -- start 2024-02-17T12:50:15.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.732+0000 7f910521e740 1 -- start start 2024-02-17T12:50:15.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.732+0000 7f910521e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1cf250 0x55afed1d7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.732+0000 7f910521e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1d5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.732+0000 7f910521e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1d8750 0x55afed1d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.732+0000 7f90fb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1d5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f90fb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1d5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52186/0 (socket says 172.21.15.18:52186) 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f90fb7fe640 1 -- 172.21.15.18:0/823650963 learned_addr learned my addr 172.21.15.18:0/823650963 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f910521e740 1 -- 172.21.15.18:0/823650963 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afed0bc130 con 0x55afed1cfb70 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f910521e740 1 -- 172.21.15.18:0/823650963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afed1ce570 con 0x55afed1d8750 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f910521e740 1 -- 172.21.15.18:0/823650963 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afed1ceee0 con 0x55afed1cf250 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f90fbfff640 1 --2- 172.21.15.18:0/823650963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1cf250 0x55afed1d7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f9100b09640 1 --2- 172.21.15.18:0/823650963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1d8750 0x55afed1d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f90fb7fe640 1 -- 172.21.15.18:0/823650963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1cf250 msgr2=0x55afed1d7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f90fb7fe640 1 --2- 172.21.15.18:0/823650963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1cf250 0x55afed1d7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f90fb7fe640 1 -- 172.21.15.18:0/823650963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1d8750 msgr2=0x55afed1d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f90fb7fe640 1 --2- 172.21.15.18:0/823650963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1d8750 0x55afed1d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f90fb7fe640 1 -- 172.21.15.18:0/823650963 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afed0d1c20 con 0x55afed1cfb70 2024-02-17T12:50:15.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f9100b09640 1 --2- 172.21.15.18:0/823650963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1d8750 0x55afed1d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:50:15.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f90fb7fe640 1 --2- 172.21.15.18:0/823650963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1d5fe0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f90f0000c00 tx=0x7f90f00098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4c2c1b06983acd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f90e1ffb640 1 -- 172.21.15.18:0/823650963 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0017040 con 0x55afed1cfb70 2024-02-17T12:50:15.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f90e1ffb640 1 -- 172.21.15.18:0/823650963 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90f0009e80 con 0x55afed1cfb70 2024-02-17T12:50:15.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.733+0000 7f90e1ffb640 1 -- 172.21.15.18:0/823650963 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f000fc50 con 0x55afed1cfb70 2024-02-17T12:50:15.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.734+0000 7f910521e740 1 -- 172.21.15.18:0/823650963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 msgr2=0x55afed1d5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.734+0000 7f910521e740 1 --2- 172.21.15.18:0/823650963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1d5fe0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f90f0000c00 tx=0x7f90f00098b0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.734+0000 7f910521e740 1 -- 172.21.15.18:0/823650963 shutdown_connections 2024-02-17T12:50:15.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.734+0000 7f910521e740 1 --2- 172.21.15.18:0/823650963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1d8750 0x55afed1d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.734+0000 7f910521e740 1 --2- 172.21.15.18:0/823650963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1d5fe0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.734+0000 7f910521e740 1 --2- 172.21.15.18:0/823650963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1cf250 0x55afed1d7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.734+0000 7f910521e740 1 -- 172.21.15.18:0/823650963 >> 172.21.15.18:0/823650963 conn(0x55afed1c9980 msgr2=0x55afed1c4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.734+0000 7f910521e740 1 -- 172.21.15.18:0/823650963 shutdown_connections 2024-02-17T12:50:15.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.734+0000 7f910521e740 1 -- 172.21.15.18:0/823650963 wait complete. 2024-02-17T12:50:15.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f910521e740 1 -- start start 2024-02-17T12:50:15.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f910521e740 1 Processor -- start 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f910521e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cf250 0x55afed1de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f910521e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1de910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f910521e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed23f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f910521e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afed1df1f0 con 0x55afed1cfb70 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f910521e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afed23f5a0 con 0x55afed1cf250 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f910521e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afed23f6e0 con 0x55afed1d8750 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f910521e740 10 client.0 _ll_register_callbacks cb 0x7f90ec0095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f90fb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1de910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f90fb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1de910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52198/0 (socket says 172.21.15.18:52198) 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f90fb7fe640 1 -- 172.21.15.18:0/3212287175 learned_addr learned my addr 172.21.15.18:0/3212287175 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.735+0000 7f90fbfff640 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cf250 0x55afed1de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f9100b09640 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed23f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f90fb7fe640 1 -- 172.21.15.18:0/3212287175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 msgr2=0x55afed23f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f90fb7fe640 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed23f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f90fb7fe640 1 -- 172.21.15.18:0/3212287175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cf250 msgr2=0x55afed1de3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f90fb7fe640 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cf250 0x55afed1de3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f90fb7fe640 1 -- 172.21.15.18:0/3212287175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afed1356a0 con 0x55afed1cfb70 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f9100b09640 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed23f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f90fb7fe640 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1de910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f90f00038a0 tx=0x7f90f00038d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96b553b65a6d0ac server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f90db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f90db7fe640 1 -- 172.21.15.18:0/3212287175 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0017030 con 0x55afed1cfb70 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f90db7fe640 1 -- 172.21.15.18:0/3212287175 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f90f0000e40 con 0x55afed1cfb70 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f910521e740 1 -- 172.21.15.18:0/3212287175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90ec016070 con 0x55afed1cfb70 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f90db7fe640 1 -- 172.21.15.18:0/3212287175 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90f00053a0 con 0x55afed1cfb70 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f910521e740 10 client.15672 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.736+0000 7f910521e740 1 -- 172.21.15.18:0/3212287175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55afed0f78c0 con 0x55afed1cfb70 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.737+0000 7f90db7fe640 1 -- 172.21.15.18:0/3212287175 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90ec016070 con 0x55afed1cfb70 2024-02-17T12:50:15.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.737+0000 7f90db7fe640 1 -- 172.21.15.18:0/3212287175 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f90f002c030 con 0x55afed1cfb70 2024-02-17T12:50:15.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.737+0000 7f910521e740 10 client.15672 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:15.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.737+0000 7f910521e740 10 client.15672 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:50:15.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.737+0000 7f910521e740 20 client.15672 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:15.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.737+0000 7f910521e740 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55afed240a60 0x55afed242e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.737+0000 7f910521e740 4 client.15672 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:50:15.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.737+0000 7f910521e740 1 -- 172.21.15.18:0/3212287175 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55afecec2890 con 0x55afed240a60 2024-02-17T12:50:15.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.737+0000 7f90fbfff640 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55afed240a60 0x55afed242e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f90fbfff640 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55afed240a60 0x55afed242e80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d4dc5aa0ca16005f server_cookie=11a12b8b11df0aea in_seq=0 out_seq=0 2024-02-17T12:50:15.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f90db7fe640 10 client.15672 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:50:15.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f90db7fe640 1 -- 172.21.15.18:0/3212287175 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55afecec2890 con 0x55afed240a60 2024-02-17T12:50:15.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f90db7fe640 10 client.15672 handle_command_reply: tid=0 2024-02-17T12:50:15.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f910521e740 1 client.15672 shutdown 2024-02-17T12:50:15.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f910521e740 1 -- 172.21.15.18:0/3212287175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 msgr2=0x55afed1de910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f910521e740 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1de910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f90f00038a0 tx=0x7f90f00038d0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f910521e740 1 -- 172.21.15.18:0/3212287175 shutdown_connections 2024-02-17T12:50:15.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f910521e740 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55afed1d8750 0x55afed23f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f910521e740 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afed1cfb70 0x55afed1de910 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f910521e740 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55afed240a60 0x55afed242e80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f910521e740 1 --2- 172.21.15.18:0/3212287175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55afed1cf250 0x55afed1de3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f910521e740 1 -- 172.21.15.18:0/3212287175 >> 172.21.15.18:0/3212287175 conn(0x55afed1c9980 msgr2=0x55afed1d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f90db7fe640 0 client.15672 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:50:15.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f910521e740 1 -- 172.21.15.18:0/3212287175 shutdown_connections 2024-02-17T12:50:15.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f910521e740 1 -- 172.21.15.18:0/3212287175 wait complete. 2024-02-17T12:50:15.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.738+0000 7f910521e740 20 client.15672 trim_cache size 0 max 16384 2024-02-17T12:50:15.737 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:50:15.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.739+0000 7f910256c640 1 -- 172.21.15.18:0/2542839438 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f90e40a2450 msgr2=0x7f90e40a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.739+0000 7f910256c640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f90e40a2450 0x7f90e40a4910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009ae0 tx=0x7f90ec005f90 comp rx=0 tx=0).stop 2024-02-17T12:50:15.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.739+0000 7f910256c640 1 -- 172.21.15.18:0/2542839438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90fc1071c0 msgr2=0x7f90fc1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.739+0000 7f910256c640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90fc1071c0 0x7f90fc1997a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f90f4004af0 tx=0x7f90f4032570 comp rx=0 tx=0).stop 2024-02-17T12:50:15.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.739+0000 7f910256c640 1 -- 172.21.15.18:0/2542839438 shutdown_connections 2024-02-17T12:50:15.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.739+0000 7f910256c640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f90e40a2450 0x7f90e40a4910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.739+0000 7f910256c640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90fc1071c0 0x7f90fc1997a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.739+0000 7f910256c640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90fc06a0f0 0x7f90fc19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.739+0000 7f910256c640 1 --2- 172.21.15.18:0/2542839438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90fc069750 0x7f90fc19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.739+0000 7f910256c640 1 -- 172.21.15.18:0/2542839438 >> 172.21.15.18:0/2542839438 conn(0x7f90fc075e80 msgr2=0x7f90fc10c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.739+0000 7f910256c640 1 -- 172.21.15.18:0/2542839438 shutdown_connections 2024-02-17T12:50:15.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.739+0000 7f910256c640 1 -- 172.21.15.18:0/2542839438 wait complete. 2024-02-17T12:50:15.742 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:50:15.742 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:50:15.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.856+0000 7f079653a640 1 -- 172.21.15.18:0/1227618502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0790100260 msgr2=0x7f0790100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.856+0000 7f079653a640 1 --2- 172.21.15.18:0/1227618502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0790100260 0x7f0790100640 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0780009930 tx=0x7f078002bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:15.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.857+0000 7f079653a640 1 -- 172.21.15.18:0/1227618502 shutdown_connections 2024-02-17T12:50:15.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.857+0000 7f079653a640 1 --2- 172.21.15.18:0/1227618502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105330 0x7f079010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.857+0000 7f079653a640 1 --2- 172.21.15.18:0/1227618502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0790100c10 0x7f0790104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.857+0000 7f079653a640 1 --2- 172.21.15.18:0/1227618502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0790100260 0x7f0790100640 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.857+0000 7f079653a640 1 -- 172.21.15.18:0/1227618502 >> 172.21.15.18:0/1227618502 conn(0x7f0790077180 msgr2=0x7f0790077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.857+0000 7f079653a640 1 -- 172.21.15.18:0/1227618502 shutdown_connections 2024-02-17T12:50:15.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.857+0000 7f079653a640 1 -- 172.21.15.18:0/1227618502 wait complete. 2024-02-17T12:50:15.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.857+0000 7f079653a640 1 Processor -- start 2024-02-17T12:50:15.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.857+0000 7f079653a640 1 -- start start 2024-02-17T12:50:15.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.858+0000 7f079653a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790100260 0x7f0790195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.858+0000 7f079653a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0790105330 0x7f0790195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.858+0000 7f079653a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0790199f10 0x7f079019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.858+0000 7f078f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0790105330 0x7f0790195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.858+0000 7f078f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0790105330 0x7f0790195b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52206/0 (socket says 172.21.15.18:52206) 2024-02-17T12:50:15.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.858+0000 7f078f7fe640 1 -- 172.21.15.18:0/2246624227 learned_addr learned my addr 172.21.15.18:0/2246624227 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:15.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.858+0000 7f079653a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079019a900 con 0x7f0790105330 2024-02-17T12:50:15.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.858+0000 7f079653a640 1 -- 172.21.15.18:0/2246624227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079019aa70 con 0x7f0790100260 2024-02-17T12:50:15.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.858+0000 7f079653a640 1 -- 172.21.15.18:0/2246624227 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079019abe0 con 0x7f0790199f10 2024-02-17T12:50:15.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.858+0000 7f078ffff640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790100260 0x7f0790195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.858+0000 7f0794ad7640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0790199f10 0x7f079019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.859+0000 7f078f7fe640 1 -- 172.21.15.18:0/2246624227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0790199f10 msgr2=0x7f079019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.859+0000 7f078f7fe640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0790199f10 0x7f079019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.859+0000 7f078f7fe640 1 -- 172.21.15.18:0/2246624227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790100260 msgr2=0x7f0790195650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.859+0000 7f078f7fe640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790100260 0x7f0790195650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.859+0000 7f078f7fe640 1 -- 172.21.15.18:0/2246624227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0780009560 con 0x7f0790105330 2024-02-17T12:50:15.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.859+0000 7f078f7fe640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0790105330 0x7f0790195b90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f078400e9f0 tx=0x7f078400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.859+0000 7f078d7fa640 1 -- 172.21.15.18:0/2246624227 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07840096c0 con 0x7f0790105330 2024-02-17T12:50:15.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.859+0000 7f078d7fa640 1 -- 172.21.15.18:0/2246624227 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0784005500 con 0x7f0790105330 2024-02-17T12:50:15.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.859+0000 7f078d7fa640 1 -- 172.21.15.18:0/2246624227 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784010430 con 0x7f0790105330 2024-02-17T12:50:15.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.859+0000 7f079653a640 1 -- 172.21.15.18:0/2246624227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07901a65d0 con 0x7f0790105330 2024-02-17T12:50:15.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.859+0000 7f079653a640 1 -- 172.21.15.18:0/2246624227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07901a6a30 con 0x7f0790105330 2024-02-17T12:50:15.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.862+0000 7f07991e9740 1 Processor -- start 2024-02-17T12:50:15.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.862+0000 7f07991e9740 1 -- start start 2024-02-17T12:50:15.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.862+0000 7f078d7fa640 1 -- 172.21.15.18:0/2246624227 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f0784005020 con 0x7f0790105330 2024-02-17T12:50:15.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.863+0000 7f07991e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bf250 0x563f596bf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.863+0000 7f07991e9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bfb70 0x563f596c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.863+0000 7f07991e9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596c8750 0x563f596cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.863+0000 7f07991e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5967a960 con 0x563f596bf250 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.863+0000 7f07991e9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5962e310 con 0x563f596c8750 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.863+0000 7f07991e9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5962b840 con 0x563f596bfb70 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.863+0000 7f078f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596c8750 0x563f596cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.863+0000 7f078f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596c8750 0x563f596cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:45736/0 (socket says 172.21.15.18:45736) 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.863+0000 7f078f7fe640 1 -- 172.21.15.18:0/2975959300 learned_addr learned my addr 172.21.15.18:0/2975959300 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.863+0000 7f0794ad7640 1 --2- 172.21.15.18:0/2975959300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bf250 0x563f596bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.863+0000 7f078ffff640 1 --2- 172.21.15.18:0/2975959300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bfb70 0x563f596c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.863+0000 7f0794ad7640 1 -- 172.21.15.18:0/2975959300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bfb70 msgr2=0x563f596c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.864+0000 7f078d7fa640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07780a23d0 0x7f07780a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.864+0000 7f0794ad7640 1 --2- 172.21.15.18:0/2975959300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bfb70 0x563f596c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.864+0000 7f0794ad7640 1 -- 172.21.15.18:0/2975959300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596c8750 msgr2=0x563f596cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.864+0000 7f0794ad7640 1 --2- 172.21.15.18:0/2975959300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596c8750 0x563f596cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.864+0000 7f0794ad7640 1 -- 172.21.15.18:0/2975959300 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f595e9060 con 0x563f596bf250 2024-02-17T12:50:15.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.864+0000 7f078f7fe640 1 --2- 172.21.15.18:0/2975959300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596c8750 0x563f596cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:15.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.864+0000 7f078ffff640 1 --2- 172.21.15.18:0/2975959300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bfb70 0x563f596c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:15.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.864+0000 7f078d7fa640 1 -- 172.21.15.18:0/2246624227 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0784014070 con 0x7f0790105330 2024-02-17T12:50:15.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.864+0000 7f0794ad7640 1 --2- 172.21.15.18:0/2975959300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bf250 0x563f596bf630 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0788009f60 tx=0x7f0788009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1374ae90553a552 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.864+0000 7f0794ad7640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07780a23d0 0x7f07780a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.865+0000 7f07767fc640 1 -- 172.21.15.18:0/2975959300 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788012040 con 0x563f596bf250 2024-02-17T12:50:15.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.865+0000 7f07767fc640 1 -- 172.21.15.18:0/2975959300 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0788018940 con 0x563f596bf250 2024-02-17T12:50:15.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.865+0000 7f07767fc640 1 -- 172.21.15.18:0/2975959300 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788017c50 con 0x563f596bf250 2024-02-17T12:50:15.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f07991e9740 1 -- 172.21.15.18:0/2975959300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bf250 msgr2=0x563f596bf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f07991e9740 1 --2- 172.21.15.18:0/2975959300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bf250 0x563f596bf630 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0788009f60 tx=0x7f0788009500 comp rx=0 tx=0).stop 2024-02-17T12:50:15.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f0794ad7640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07780a23d0 0x7f07780a4890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f078801b000 tx=0x7f0788017e90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f07991e9740 1 -- 172.21.15.18:0/2975959300 shutdown_connections 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f07991e9740 1 --2- 172.21.15.18:0/2975959300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596c8750 0x563f596cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f07991e9740 1 --2- 172.21.15.18:0/2975959300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bfb70 0x563f596c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f07991e9740 1 --2- 172.21.15.18:0/2975959300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bf250 0x563f596bf630 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f07991e9740 1 -- 172.21.15.18:0/2975959300 >> 172.21.15.18:0/2975959300 conn(0x563f596b9980 msgr2=0x563f596b9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f07991e9740 1 -- 172.21.15.18:0/2975959300 shutdown_connections 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f07991e9740 1 -- 172.21.15.18:0/2975959300 wait complete. 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f07991e9740 1 -- start start 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f07991e9740 1 Processor -- start 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.866+0000 7f07991e9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bf250 0x563f596c5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f07991e9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bfb70 0x563f596c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f07991e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 0x563f596c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f07991e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f59627020 con 0x563f596c8750 2024-02-17T12:50:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f07991e9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f592fa430 con 0x563f596bf250 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f07991e9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f592b6e00 con 0x563f596bfb70 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f07991e9740 10 client.0 _ll_register_callbacks cb 0x563f595cecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f078ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bf250 0x563f596c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f0794ad7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 0x563f596c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f078f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bfb70 0x563f596c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f078ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bf250 0x563f596c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:45742/0 (socket says 172.21.15.18:45742) 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f078ffff640 1 -- 172.21.15.18:0/2821259248 learned_addr learned my addr 172.21.15.18:0/2821259248 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f078f7fe640 1 -- 172.21.15.18:0/2821259248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bf250 msgr2=0x563f596c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f078f7fe640 1 --2- 172.21.15.18:0/2821259248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bf250 0x563f596c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f078f7fe640 1 -- 172.21.15.18:0/2821259248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 msgr2=0x563f596c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f078f7fe640 1 --2- 172.21.15.18:0/2821259248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 0x563f596c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f078f7fe640 1 -- 172.21.15.18:0/2821259248 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0788009c40 con 0x563f596bfb70 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f0794ad7640 1 --2- 172.21.15.18:0/2821259248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 0x563f596c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f078f7fe640 1 --2- 172.21.15.18:0/2821259248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bfb70 0x563f596c5cb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f07840cbc90 tx=0x7f078409a9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c61e1c38ad414d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.867+0000 7f076f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f076f7fe640 1 -- 172.21.15.18:0/2821259248 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078409b4d0 con 0x563f596bfb70 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f076f7fe640 1 -- 172.21.15.18:0/2821259248 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f078409add0 con 0x563f596bfb70 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f07991e9740 1 -- 172.21.15.18:0/2821259248 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f593267e0 con 0x563f596bfb70 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f076f7fe640 1 -- 172.21.15.18:0/2821259248 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078409ef20 con 0x563f596bfb70 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f07991e9740 10 client.25258 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f07991e9740 1 -- 172.21.15.18:0/2821259248 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563f596256a0 con 0x563f596bfb70 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f076f7fe640 1 -- 172.21.15.18:0/2821259248 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07840b7080 con 0x563f596bfb70 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f076f7fe640 1 -- 172.21.15.18:0/2821259248 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f078401f060 con 0x563f596bfb70 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f07991e9740 10 client.25258 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f07991e9740 10 client.25258 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f07991e9740 20 client.25258 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f07991e9740 1 --2- 172.21.15.18:0/2821259248 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x563f596d8b00 0x563f596daf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f07991e9740 4 client.25258 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f07991e9740 1 -- 172.21.15.18:0/2821259248 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f593267e0 con 0x563f596d8b00 2024-02-17T12:50:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.868+0000 7f078ffff640 1 --2- 172.21.15.18:0/2821259248 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x563f596d8b00 0x563f596daf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.869+0000 7f076f7fe640 10 client.25258 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:50:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.869+0000 7f078ffff640 1 --2- 172.21.15.18:0/2821259248 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x563f596d8b00 0x563f596daf60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5e80daa0c120fd13 server_cookie=3d1f7bea47839348 in_seq=0 out_seq=0 2024-02-17T12:50:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f076f7fe640 1 -- 172.21.15.18:0/2821259248 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563f593267e0 con 0x563f596d8b00 2024-02-17T12:50:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f076f7fe640 10 client.25258 handle_command_reply: tid=0 2024-02-17T12:50:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f07991e9740 1 client.25258 shutdown 2024-02-17T12:50:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f07991e9740 1 -- 172.21.15.18:0/2821259248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bfb70 msgr2=0x563f596c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f07991e9740 1 --2- 172.21.15.18:0/2821259248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bfb70 0x563f596c5cb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f07840cbc90 tx=0x7f078409a9d0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f07991e9740 1 -- 172.21.15.18:0/2821259248 shutdown_connections 2024-02-17T12:50:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f07991e9740 1 --2- 172.21.15.18:0/2821259248 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x563f596d8b00 0x563f596daf60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f076f7fe640 0 client.25258 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:50:15.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f07991e9740 1 --2- 172.21.15.18:0/2821259248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 0x563f596c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f07991e9740 1 --2- 172.21.15.18:0/2821259248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bfb70 0x563f596c5cb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f07991e9740 1 --2- 172.21.15.18:0/2821259248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bf250 0x563f596c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f07991e9740 1 -- 172.21.15.18:0/2821259248 >> 172.21.15.18:0/2821259248 conn(0x563f596b9980 msgr2=0x563f596bb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f07991e9740 1 -- 172.21.15.18:0/2821259248 shutdown_connections 2024-02-17T12:50:15.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f07991e9740 1 -- 172.21.15.18:0/2821259248 wait complete. 2024-02-17T12:50:15.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.870+0000 7f07991e9740 20 client.25258 trim_cache size 0 max 16384 2024-02-17T12:50:15.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f07991e9740 1 Processor -- start 2024-02-17T12:50:15.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f07991e9740 1 -- start start 2024-02-17T12:50:15.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f07991e9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bf250 0x563f596c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f07991e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bfb70 0x563f596c0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f07991e9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596c8750 0x563f596c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f07991e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f595ac130 con 0x563f596bfb70 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f07991e9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f59326810 con 0x563f596bf250 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f07991e9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f596b52b0 con 0x563f596c8750 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f078f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bfb70 0x563f596c0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f078f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bfb70 0x563f596c0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52222/0 (socket says 172.21.15.18:52222) 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f078f7fe640 1 -- 172.21.15.18:0/1223091812 learned_addr learned my addr 172.21.15.18:0/1223091812 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f078ffff640 1 --2- 172.21.15.18:0/1223091812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bf250 0x563f596c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.882+0000 7f0794ad7640 1 --2- 172.21.15.18:0/1223091812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596c8750 0x563f596c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.883+0000 7f078f7fe640 1 -- 172.21.15.18:0/1223091812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596c8750 msgr2=0x563f596c29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.883+0000 7f078f7fe640 1 --2- 172.21.15.18:0/1223091812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596c8750 0x563f596c29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.883+0000 7f078f7fe640 1 -- 172.21.15.18:0/1223091812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bf250 msgr2=0x563f596c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.883+0000 7f078f7fe640 1 --2- 172.21.15.18:0/1223091812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bf250 0x563f596c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.883+0000 7f078f7fe640 1 -- 172.21.15.18:0/1223091812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f595c1c20 con 0x563f596bfb70 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.883+0000 7f078f7fe640 1 --2- 172.21.15.18:0/1223091812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bfb70 0x563f596c0ad0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f07840cbd10 tx=0x7f078409aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689c4cf63891d4a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.883+0000 7f0775ffb640 1 -- 172.21.15.18:0/1223091812 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07840d1a40 con 0x563f596bfb70 2024-02-17T12:50:15.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.883+0000 7f0775ffb640 1 -- 172.21.15.18:0/1223091812 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07840a30e0 con 0x563f596bfb70 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.883+0000 7f0775ffb640 1 -- 172.21.15.18:0/1223091812 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07840a3570 con 0x563f596bfb70 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.884+0000 7f07991e9740 1 -- 172.21.15.18:0/1223091812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bfb70 msgr2=0x563f596c0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.884+0000 7f07991e9740 1 --2- 172.21.15.18:0/1223091812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bfb70 0x563f596c0ad0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f07840cbd10 tx=0x7f078409aba0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.884+0000 7f07991e9740 1 -- 172.21.15.18:0/1223091812 shutdown_connections 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.884+0000 7f07991e9740 1 --2- 172.21.15.18:0/1223091812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596c8750 0x563f596c29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.884+0000 7f07991e9740 1 --2- 172.21.15.18:0/1223091812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596bfb70 0x563f596c0ad0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.884+0000 7f07991e9740 1 --2- 172.21.15.18:0/1223091812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bf250 0x563f596c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.884+0000 7f07991e9740 1 -- 172.21.15.18:0/1223091812 >> 172.21.15.18:0/1223091812 conn(0x563f596b9980 msgr2=0x563f596b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.884+0000 7f07991e9740 1 -- 172.21.15.18:0/1223091812 shutdown_connections 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.884+0000 7f07991e9740 1 -- 172.21.15.18:0/1223091812 wait complete. 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.884+0000 7f07991e9740 1 -- start start 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f07991e9740 1 Processor -- start 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f07991e9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bf250 0x563f596ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f07991e9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bfb70 0x563f596cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f07991e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 0x563f597378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f07991e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f59737de0 con 0x563f596c8750 2024-02-17T12:50:15.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f07991e9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f59737f20 con 0x563f596bfb70 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f07991e9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f59738060 con 0x563f596bf250 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f07991e9740 10 client.0 _ll_register_callbacks cb 0x7f0788009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f0794ad7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 0x563f597378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f078f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bfb70 0x563f596cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f0794ad7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 0x563f597378a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52226/0 (socket says 172.21.15.18:52226) 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f0794ad7640 1 -- 172.21.15.18:0/4133394250 learned_addr learned my addr 172.21.15.18:0/4133394250 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f078ffff640 1 --2- 172.21.15.18:0/4133394250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bf250 0x563f596ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f078f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bfb70 0x563f596cec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:45756/0 (socket says 172.21.15.18:45756) 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f0794ad7640 1 -- 172.21.15.18:0/4133394250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bf250 msgr2=0x563f596ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f0794ad7640 1 --2- 172.21.15.18:0/4133394250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bf250 0x563f596ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f0794ad7640 1 -- 172.21.15.18:0/4133394250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bfb70 msgr2=0x563f596cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f0794ad7640 1 --2- 172.21.15.18:0/4133394250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bfb70 0x563f596cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.885+0000 7f0794ad7640 1 -- 172.21.15.18:0/4133394250 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f596256a0 con 0x563f596c8750 2024-02-17T12:50:15.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.886+0000 7f0794ad7640 1 --2- 172.21.15.18:0/4133394250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 0x563f597378a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f078800a010 tx=0x7f0788018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd92f5c1003f9c6f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:15.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.886+0000 7f076f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:15.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.886+0000 7f076f7fe640 1 -- 172.21.15.18:0/4133394250 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078802b040 con 0x563f596c8750 2024-02-17T12:50:15.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.886+0000 7f076f7fe640 1 -- 172.21.15.18:0/4133394250 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0788016480 con 0x563f596c8750 2024-02-17T12:50:15.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.886+0000 7f07991e9740 1 -- 172.21.15.18:0/4133394250 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0788018940 con 0x563f596c8750 2024-02-17T12:50:15.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.886+0000 7f076f7fe640 1 -- 172.21.15.18:0/4133394250 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07880096c0 con 0x563f596c8750 2024-02-17T12:50:15.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.886+0000 7f07991e9740 10 client.15690 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:15.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.886+0000 7f07991e9740 1 -- 172.21.15.18:0/4133394250 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563f595e78c0 con 0x563f596c8750 2024-02-17T12:50:15.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.887+0000 7f076f7fe640 1 -- 172.21.15.18:0/4133394250 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0788018940 con 0x563f596c8750 2024-02-17T12:50:15.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.887+0000 7f076f7fe640 1 -- 172.21.15.18:0/4133394250 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0788031040 con 0x563f596c8750 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.887+0000 7f07991e9740 10 client.15690 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.887+0000 7f07991e9740 10 client.15690 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.887+0000 7f07991e9740 20 client.15690 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.887+0000 7f07991e9740 1 --2- 172.21.15.18:0/4133394250 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x563f596d8b00 0x563f5973b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.887+0000 7f07991e9740 4 client.15690 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.887+0000 7f07991e9740 1 -- 172.21.15.18:0/4133394250 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f59363720 con 0x563f596d8b00 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.887+0000 7f078ffff640 1 --2- 172.21.15.18:0/4133394250 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x563f596d8b00 0x563f5973b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.887+0000 7f076f7fe640 10 client.15690 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.887+0000 7f078ffff640 1 --2- 172.21.15.18:0/4133394250 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x563f596d8b00 0x563f5973b880 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aec8440d80b954e0 server_cookie=a7a171d011ed1c8 in_seq=0 out_seq=0 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f076f7fe640 1 -- 172.21.15.18:0/4133394250 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f59363720 con 0x563f596d8b00 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f076f7fe640 10 client.15690 handle_command_reply: tid=0 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f07991e9740 1 client.15690 shutdown 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f07991e9740 1 -- 172.21.15.18:0/4133394250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 msgr2=0x563f597378a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f07991e9740 1 --2- 172.21.15.18:0/4133394250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 0x563f597378a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f078800a010 tx=0x7f0788018000 comp rx=0 tx=0).stop 2024-02-17T12:50:15.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f07991e9740 1 -- 172.21.15.18:0/4133394250 shutdown_connections 2024-02-17T12:50:15.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f07991e9740 1 --2- 172.21.15.18:0/4133394250 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x563f596d8b00 0x563f5973b880 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f076f7fe640 0 client.15690 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:50:15.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f07991e9740 1 --2- 172.21.15.18:0/4133394250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f596c8750 0x563f597378a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f07991e9740 1 --2- 172.21.15.18:0/4133394250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f596bfb70 0x563f596cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f07991e9740 1 --2- 172.21.15.18:0/4133394250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f596bf250 0x563f596ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f07991e9740 1 -- 172.21.15.18:0/4133394250 >> 172.21.15.18:0/4133394250 conn(0x563f596b9980 msgr2=0x563f596c3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f07991e9740 1 -- 172.21.15.18:0/4133394250 shutdown_connections 2024-02-17T12:50:15.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f07991e9740 1 -- 172.21.15.18:0/4133394250 wait complete. 2024-02-17T12:50:15.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.888+0000 7f07991e9740 20 client.15690 trim_cache size 0 max 16384 2024-02-17T12:50:15.885 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:50:15.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.889+0000 7f079653a640 1 -- 172.21.15.18:0/2246624227 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07780a23d0 msgr2=0x7f07780a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.889+0000 7f079653a640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07780a23d0 0x7f07780a4890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f078801b000 tx=0x7f0788017e90 comp rx=0 tx=0).stop 2024-02-17T12:50:15.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.889+0000 7f079653a640 1 -- 172.21.15.18:0/2246624227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0790105330 msgr2=0x7f0790195b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:15.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.889+0000 7f079653a640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0790105330 0x7f0790195b90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f078400e9f0 tx=0x7f078400eeb0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.889+0000 7f079653a640 1 -- 172.21.15.18:0/2246624227 shutdown_connections 2024-02-17T12:50:15.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.889+0000 7f079653a640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07780a23d0 0x7f07780a4890 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.889+0000 7f079653a640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0790199f10 0x7f079019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.889+0000 7f079653a640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0790105330 0x7f0790195b90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.889+0000 7f079653a640 1 --2- 172.21.15.18:0/2246624227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790100260 0x7f0790195650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:15.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.889+0000 7f079653a640 1 -- 172.21.15.18:0/2246624227 >> 172.21.15.18:0/2246624227 conn(0x7f0790077180 msgr2=0x7f07900fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:15.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.889+0000 7f079653a640 1 -- 172.21.15.18:0/2246624227 shutdown_connections 2024-02-17T12:50:15.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:15.889+0000 7f079653a640 1 -- 172.21.15.18:0/2246624227 wait complete. 2024-02-17T12:50:15.892 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:50:15.892 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:50:16.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.031+0000 7f0aca2ae640 1 -- 172.21.15.18:0/1756089745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac4100260 msgr2=0x7f0ac4100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.031+0000 7f0aca2ae640 1 --2- 172.21.15.18:0/1756089745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac4100260 0x7f0ac4100640 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009930 tx=0x7f0ab402bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:16.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.031+0000 7f0aca2ae640 1 -- 172.21.15.18:0/1756089745 shutdown_connections 2024-02-17T12:50:16.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.031+0000 7f0aca2ae640 1 --2- 172.21.15.18:0/1756089745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac4105330 0x7f0ac410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.031+0000 7f0aca2ae640 1 --2- 172.21.15.18:0/1756089745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4100c10 0x7f0ac4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.031+0000 7f0aca2ae640 1 --2- 172.21.15.18:0/1756089745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac4100260 0x7f0ac4100640 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.031+0000 7f0aca2ae640 1 -- 172.21.15.18:0/1756089745 >> 172.21.15.18:0/1756089745 conn(0x7f0ac4077180 msgr2=0x7f0ac4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.031+0000 7f0aca2ae640 1 -- 172.21.15.18:0/1756089745 shutdown_connections 2024-02-17T12:50:16.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.031+0000 7f0aca2ae640 1 -- 172.21.15.18:0/1756089745 wait complete. 2024-02-17T12:50:16.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.032+0000 7f0aca2ae640 1 Processor -- start 2024-02-17T12:50:16.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.032+0000 7f0aca2ae640 1 -- start start 2024-02-17T12:50:16.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.032+0000 7f0aca2ae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac4100260 0x7f0ac41966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.032+0000 7f0aca2ae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4105330 0x7f0ac4196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.032+0000 7f0aca2ae640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac419af90 0x7f0ac419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.032+0000 7f0aca2ae640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac419b980 con 0x7f0ac419af90 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.032+0000 7f0aca2ae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac419baf0 con 0x7f0ac4100260 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.032+0000 7f0aca2ae640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac419bc60 con 0x7f0ac4105330 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac884b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac419af90 0x7f0ac419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac884b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac419af90 0x7f0ac419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52240/0 (socket says 172.21.15.18:52240) 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac884b640 1 -- 172.21.15.18:0/135094117 learned_addr learned my addr 172.21.15.18:0/135094117 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac4100260 0x7f0ac41966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac37fe640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4105330 0x7f0ac4196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac3fff640 1 -- 172.21.15.18:0/135094117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4105330 msgr2=0x7f0ac4196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4105330 0x7f0ac4196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac3fff640 1 -- 172.21.15.18:0/135094117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac419af90 msgr2=0x7f0ac419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac419af90 0x7f0ac419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac3fff640 1 -- 172.21.15.18:0/135094117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab4009560 con 0x7f0ac4100260 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac884b640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac419af90 0x7f0ac419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac4100260 0x7f0ac41966d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009230 tx=0x7f0ab402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.033+0000 7f0ac17fa640 1 -- 172.21.15.18:0/135094117 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40042d0 con 0x7f0ac4100260 2024-02-17T12:50:16.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.034+0000 7f0ac17fa640 1 -- 172.21.15.18:0/135094117 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ab4004430 con 0x7f0ac4100260 2024-02-17T12:50:16.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.034+0000 7f0aca2ae640 1 -- 172.21.15.18:0/135094117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac41a3240 con 0x7f0ac4100260 2024-02-17T12:50:16.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.034+0000 7f0aca2ae640 1 -- 172.21.15.18:0/135094117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac41a36c0 con 0x7f0ac4100260 2024-02-17T12:50:16.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.034+0000 7f0ac17fa640 1 -- 172.21.15.18:0/135094117 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4003bd0 con 0x7f0ac4100260 2024-02-17T12:50:16.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.037+0000 7f0ac17fa640 1 -- 172.21.15.18:0/135094117 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f0ab4003d30 con 0x7f0ac4100260 2024-02-17T12:50:16.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.037+0000 7f0accb22740 1 Processor -- start 2024-02-17T12:50:16.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.037+0000 7f0accb22740 1 -- start start 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.037+0000 7f0accb22740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def785cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.037+0000 7f0accb22740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def786a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.037+0000 7f0ac17fa640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0aac0a23d0 0x7f0aac0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.037+0000 7f0accb22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def786d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.037+0000 7f0accb22740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def7818960 con 0x55def785cac0 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.037+0000 7f0accb22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def77cc310 con 0x55def786ad50 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.037+0000 7f0accb22740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def77c9840 con 0x55def785d3e0 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac884b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def785cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac884b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def785cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52254/0 (socket says 172.21.15.18:52254) 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac884b640 1 -- 172.21.15.18:0/3960855538 learned_addr learned my addr 172.21.15.18:0/3960855538 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/3960855538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def786a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac17fa640 1 -- 172.21.15.18:0/135094117 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0ab40f00b0 con 0x7f0ac4100260 2024-02-17T12:50:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac884b640 1 --2- 172.21.15.18:0/3960855538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def786d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac3fff640 1 -- 172.21.15.18:0/3960855538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 msgr2=0x55def786d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/3960855538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def786d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac3fff640 1 -- 172.21.15.18:0/3960855538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 msgr2=0x55def785cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/3960855538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def785cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac37fe640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0aac0a23d0 0x7f0aac0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac3fff640 1 -- 172.21.15.18:0/3960855538 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55def7787060 con 0x55def785d3e0 2024-02-17T12:50:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac884b640 1 --2- 172.21.15.18:0/3960855538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def785cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.038+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/3960855538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def786a810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40ed7e0 tx=0x7f0ab402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e1cef7ce2eb87db server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.039+0000 7f0aaa7fc640 1 -- 172.21.15.18:0/3960855538 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40bd0b0 con 0x55def785d3e0 2024-02-17T12:50:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.039+0000 7f0aaa7fc640 1 -- 172.21.15.18:0/3960855538 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ab40bc9b0 con 0x55def785d3e0 2024-02-17T12:50:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.039+0000 7f0aaa7fc640 1 -- 172.21.15.18:0/3960855538 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40db8a0 con 0x55def785d3e0 2024-02-17T12:50:16.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.039+0000 7f0accb22740 1 -- 172.21.15.18:0/3960855538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 msgr2=0x55def786a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.039+0000 7f0ac37fe640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0aac0a23d0 0x7f0aac0a4890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40ffba0 tx=0x7f0ab8005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.039+0000 7f0accb22740 1 --2- 172.21.15.18:0/3960855538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def786a810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40ed7e0 tx=0x7f0ab402bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.040+0000 7f0accb22740 1 -- 172.21.15.18:0/3960855538 shutdown_connections 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.040+0000 7f0accb22740 1 --2- 172.21.15.18:0/3960855538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def786d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.040+0000 7f0accb22740 1 --2- 172.21.15.18:0/3960855538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def786a810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.040+0000 7f0accb22740 1 --2- 172.21.15.18:0/3960855538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def785cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.040+0000 7f0accb22740 1 -- 172.21.15.18:0/3960855538 >> 172.21.15.18:0/3960855538 conn(0x55def7857980 msgr2=0x55def7857d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.040+0000 7f0accb22740 1 -- 172.21.15.18:0/3960855538 shutdown_connections 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.040+0000 7f0accb22740 1 -- 172.21.15.18:0/3960855538 wait complete. 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.040+0000 7f0accb22740 1 -- start start 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.040+0000 7f0accb22740 1 Processor -- start 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0accb22740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def7867d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0accb22740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def78682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0accb22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7868800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def7867d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0accb22740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def77c5020 con 0x55def785cac0 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def7867d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52264/0 (socket says 172.21.15.18:52264) 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0accb22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def7498430 con 0x55def786ad50 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0ac3fff640 1 -- 172.21.15.18:0/3536504274 learned_addr learned my addr 172.21.15.18:0/3536504274 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0accb22740 1 -- 172.21.15.18:0/3536504274 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def7454e00 con 0x55def785d3e0 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0ac37fe640 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def78682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0ac884b640 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7868800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0ac3fff640 1 -- 172.21.15.18:0/3536504274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 msgr2=0x55def78682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def78682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0accb22740 10 client.0 _ll_register_callbacks cb 0x55def776ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0ac3fff640 1 -- 172.21.15.18:0/3536504274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 msgr2=0x55def7868800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7868800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0ac3fff640 1 -- 172.21.15.18:0/3536504274 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac41a36c0 con 0x55def785cac0 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.041+0000 7f0accb22740 1 -- 172.21.15.18:0/3536504274 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55def74c47e0 con 0x55def785cac0 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def7867d80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4006010 tx=0x7f0ab4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca1c476ce4d777c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0aa37fe640 10 client.15702 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0aa37fe640 1 -- 172.21.15.18:0/3536504274 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40d54a0 con 0x55def785cac0 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0aa37fe640 1 -- 172.21.15.18:0/3536504274 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0ab40c5150 con 0x55def785cac0 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0aa37fe640 1 -- 172.21.15.18:0/3536504274 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ab40c4cc0 con 0x55def785cac0 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0accb22740 10 client.15702 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0accb22740 1 -- 172.21.15.18:0/3536504274 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({fsmap=26}) v3 -- 0x55def77c36a0 con 0x55def785cac0 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0aa37fe640 1 -- 172.21.15.18:0/3536504274 <== mon.0 v2:172.21.15.18:3300/0 4 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ab403c070 con 0x55def785cac0 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0accb22740 10 client.15702 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0accb22740 10 client.15702 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0accb22740 20 client.15702 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:16.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0accb22740 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55def787a640 0x55def787ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0accb22740 4 client.15702 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.042+0000 7f0accb22740 1 -- 172.21.15.18:0/3536504274 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55def7501720 con 0x55def787a640 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.043+0000 7f0ac37fe640 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55def787a640 0x55def787ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.043+0000 7f0aa37fe640 10 client.15702 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.043+0000 7f0ac37fe640 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55def787a640 0x55def787ca30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4ee29f58c5377d5 server_cookie=1bb26b554e802547 in_seq=0 out_seq=0 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.043+0000 7f0aa37fe640 1 -- 172.21.15.18:0/3536504274 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55def7501720 con 0x55def787a640 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0aa37fe640 10 client.15702 handle_command_reply: tid=0 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0accb22740 1 client.15702 shutdown 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0accb22740 1 -- 172.21.15.18:0/3536504274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 msgr2=0x55def7867d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0accb22740 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def7867d80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4006010 tx=0x7f0ab4009260 comp rx=0 tx=0).stop 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0accb22740 1 -- 172.21.15.18:0/3536504274 shutdown_connections 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0accb22740 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55def787a640 0x55def787ca30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0aa37fe640 0 client.15702 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:50:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0accb22740 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7868800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0accb22740 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def78682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0accb22740 1 --2- 172.21.15.18:0/3536504274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def7867d80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0accb22740 1 -- 172.21.15.18:0/3536504274 >> 172.21.15.18:0/3536504274 conn(0x55def7857980 msgr2=0x55def78596d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0accb22740 1 -- 172.21.15.18:0/3536504274 shutdown_connections 2024-02-17T12:50:16.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0accb22740 1 -- 172.21.15.18:0/3536504274 wait complete. 2024-02-17T12:50:16.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.044+0000 7f0accb22740 20 client.15702 trim_cache size 0 max 16384 2024-02-17T12:50:16.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0accb22740 1 Processor -- start 2024-02-17T12:50:16.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0accb22740 1 -- start start 2024-02-17T12:50:16.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0accb22740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def786a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0accb22740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def78657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0accb22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7866d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0accb22740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def774a130 con 0x55def785cac0 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0accb22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def785c340 con 0x55def786ad50 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0accb22740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def786b130 con 0x55def785d3e0 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def786a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def786a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52276/0 (socket says 172.21.15.18:52276) 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0ac3fff640 1 -- 172.21.15.18:0/762441517 learned_addr learned my addr 172.21.15.18:0/762441517 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0ac884b640 1 --2- 172.21.15.18:0/762441517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7866d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.056+0000 7f0ac37fe640 1 --2- 172.21.15.18:0/762441517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def78657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.057+0000 7f0ac3fff640 1 -- 172.21.15.18:0/762441517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 msgr2=0x55def78657e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.057+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/762441517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def78657e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.057+0000 7f0ac3fff640 1 -- 172.21.15.18:0/762441517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 msgr2=0x55def7866d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.057+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/762441517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7866d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.057+0000 7f0ac3fff640 1 -- 172.21.15.18:0/762441517 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55def775fc20 con 0x55def785cac0 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.057+0000 7f0ac884b640 1 --2- 172.21.15.18:0/762441517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7866d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.057+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/762441517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def786a4b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0ab402cde0 tx=0x7f0ab4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd4057f321b3e40 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.057+0000 7f0aa9ffb640 1 -- 172.21.15.18:0/762441517 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40bc4f0 con 0x55def785cac0 2024-02-17T12:50:16.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.057+0000 7f0aa9ffb640 1 -- 172.21.15.18:0/762441517 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55def74c47e0 con 0x55def785cac0 2024-02-17T12:50:16.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.057+0000 7f0aa9ffb640 1 -- 172.21.15.18:0/762441517 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40a70b0 con 0x55def785cac0 2024-02-17T12:50:16.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.058+0000 7f0accb22740 1 -- 172.21.15.18:0/762441517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 msgr2=0x55def786a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.058+0000 7f0accb22740 1 --2- 172.21.15.18:0/762441517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def786a4b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0ab402cde0 tx=0x7f0ab4040000 comp rx=0 tx=0).stop 2024-02-17T12:50:16.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.058+0000 7f0accb22740 1 -- 172.21.15.18:0/762441517 shutdown_connections 2024-02-17T12:50:16.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.058+0000 7f0accb22740 1 --2- 172.21.15.18:0/762441517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7866d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.058+0000 7f0accb22740 1 --2- 172.21.15.18:0/762441517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def78657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.058+0000 7f0accb22740 1 --2- 172.21.15.18:0/762441517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def786a4b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.058+0000 7f0accb22740 1 -- 172.21.15.18:0/762441517 >> 172.21.15.18:0/762441517 conn(0x55def7857980 msgr2=0x55def7866650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.058+0000 7f0accb22740 1 -- 172.21.15.18:0/762441517 shutdown_connections 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.058+0000 7f0accb22740 1 -- 172.21.15.18:0/762441517 wait complete. 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.059+0000 7f0accb22740 1 -- start start 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.059+0000 7f0accb22740 1 Processor -- start 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.059+0000 7f0accb22740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def7871b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.059+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def7871b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.059+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def7871b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52290/0 (socket says 172.21.15.18:52290) 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.059+0000 7f0accb22740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def786ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.059+0000 7f0ac3fff640 1 -- 172.21.15.18:0/1136406246 learned_addr learned my addr 172.21.15.18:0/1136406246 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.059+0000 7f0accb22740 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7870450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0accb22740 1 -- 172.21.15.18:0/1136406246 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def7870b50 con 0x55def785cac0 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0accb22740 1 -- 172.21.15.18:0/1136406246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def7870cc0 con 0x55def786ad50 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0accb22740 1 -- 172.21.15.18:0/1136406246 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def78d0a10 con 0x55def785d3e0 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0ac37fe640 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def786ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0ac884b640 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7870450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0ac3fff640 1 -- 172.21.15.18:0/1136406246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 msgr2=0x55def786ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def786ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0ac3fff640 1 -- 172.21.15.18:0/1136406246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 msgr2=0x55def7870450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7870450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0ac3fff640 1 -- 172.21.15.18:0/1136406246 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc0128b0 con 0x55def785cac0 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0ac884b640 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7870450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0ac3fff640 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def7871b70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40b9380 tx=0x7f0ab4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77470781d38b40e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0aa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0accb22740 10 client.0 _ll_register_callbacks cb 0x7f0ab40045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:16.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0accb22740 1 -- 172.21.15.18:0/1136406246 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ab40bc9b0 con 0x55def785cac0 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.060+0000 7f0aa37fe640 1 -- 172.21.15.18:0/1136406246 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40bc4f0 con 0x55def785cac0 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.061+0000 7f0aa37fe640 1 -- 172.21.15.18:0/1136406246 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0ab40a7390 con 0x55def785cac0 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.061+0000 7f0aa37fe640 1 -- 172.21.15.18:0/1136406246 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ab40c4f10 con 0x55def785cac0 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.061+0000 7f0accb22740 10 client.15708 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.061+0000 7f0accb22740 1 -- 172.21.15.18:0/1136406246 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({fsmap=26}) v3 -- 0x7f0ac41a36c0 con 0x55def785cac0 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.061+0000 7f0aa37fe640 1 -- 172.21.15.18:0/1136406246 <== mon.0 v2:172.21.15.18:3300/0 4 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ab403c070 con 0x55def785cac0 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.061+0000 7f0accb22740 10 client.15708 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.061+0000 7f0accb22740 10 client.15708 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.061+0000 7f0accb22740 20 client.15708 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.061+0000 7f0accb22740 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55def78d1970 0x55def78d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.061+0000 7f0accb22740 4 client.15708 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.061+0000 7f0accb22740 1 -- 172.21.15.18:0/1136406246 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f0ab40bc9b0 con 0x55def78d1970 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.061+0000 7f0ac37fe640 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55def78d1970 0x55def78d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.062+0000 7f0aa37fe640 10 client.15708 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.062+0000 7f0ac37fe640 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55def78d1970 0x55def78d3db0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d26f4bebc8091e90 server_cookie=a9c252dd829607d9 in_seq=0 out_seq=0 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.062+0000 7f0aa37fe640 1 -- 172.21.15.18:0/1136406246 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f0ab40bc9b0 con 0x55def78d1970 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.062+0000 7f0aa37fe640 10 client.15708 handle_command_reply: tid=0 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.062+0000 7f0accb22740 1 client.15708 shutdown 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.062+0000 7f0accb22740 1 -- 172.21.15.18:0/1136406246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 msgr2=0x55def7871b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.062+0000 7f0accb22740 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def7871b70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40b9380 tx=0x7f0ab4009260 comp rx=0 tx=0).stop 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.063+0000 7f0accb22740 1 -- 172.21.15.18:0/1136406246 shutdown_connections 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.063+0000 7f0accb22740 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55def78d1970 0x55def78d3db0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.063+0000 7f0accb22740 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55def786ad50 0x55def7870450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.063+0000 7f0accb22740 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55def785d3e0 0x55def786ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.063+0000 7f0accb22740 1 --2- 172.21.15.18:0/1136406246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55def785cac0 0x55def7871b70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.063+0000 7f0accb22740 1 -- 172.21.15.18:0/1136406246 >> 172.21.15.18:0/1136406246 conn(0x55def7857980 msgr2=0x55def78562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.063+0000 7f0aa37fe640 0 client.15708 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.063+0000 7f0accb22740 1 -- 172.21.15.18:0/1136406246 shutdown_connections 2024-02-17T12:50:16.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.063+0000 7f0accb22740 1 -- 172.21.15.18:0/1136406246 wait complete. 2024-02-17T12:50:16.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.063+0000 7f0accb22740 20 client.15708 trim_cache size 0 max 16384 2024-02-17T12:50:16.060 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:50:16.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.063+0000 7f0aca2ae640 1 -- 172.21.15.18:0/135094117 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0aac0a23d0 msgr2=0x7f0aac0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.063+0000 7f0aca2ae640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0aac0a23d0 0x7f0aac0a4890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40ffba0 tx=0x7f0ab8005f90 comp rx=0 tx=0).stop 2024-02-17T12:50:16.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.064+0000 7f0aca2ae640 1 -- 172.21.15.18:0/135094117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac4100260 msgr2=0x7f0ac41966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.064+0000 7f0aca2ae640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac4100260 0x7f0ac41966d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009230 tx=0x7f0ab402c5d0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.064+0000 7f0aca2ae640 1 -- 172.21.15.18:0/135094117 shutdown_connections 2024-02-17T12:50:16.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.064+0000 7f0aca2ae640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0aac0a23d0 0x7f0aac0a4890 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.064+0000 7f0aca2ae640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ac419af90 0x7f0ac419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.064+0000 7f0aca2ae640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4105330 0x7f0ac4196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.064+0000 7f0aca2ae640 1 --2- 172.21.15.18:0/135094117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac4100260 0x7f0ac41966d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.064+0000 7f0aca2ae640 1 -- 172.21.15.18:0/135094117 >> 172.21.15.18:0/135094117 conn(0x7f0ac4077180 msgr2=0x7f0ac40fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.064+0000 7f0aca2ae640 1 -- 172.21.15.18:0/135094117 shutdown_connections 2024-02-17T12:50:16.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.064+0000 7f0aca2ae640 1 -- 172.21.15.18:0/135094117 wait complete. 2024-02-17T12:50:16.070 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:50:16.070 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:50:16.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.184+0000 7f19f31bc640 1 -- 172.21.15.18:0/2086276168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19ec1064c0 msgr2=0x7f19ec10e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.184+0000 7f19f31bc640 1 --2- 172.21.15.18:0/2086276168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19ec1064c0 0x7f19ec10e070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009be0 tx=0x7f19e402bef0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.184+0000 7f19f31bc640 1 -- 172.21.15.18:0/2086276168 shutdown_connections 2024-02-17T12:50:16.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.184+0000 7f19f31bc640 1 --2- 172.21.15.18:0/2086276168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19ec1064c0 0x7f19ec10e070 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.184+0000 7f19f31bc640 1 --2- 172.21.15.18:0/2086276168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec068fa0 0x7f19ec105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.184+0000 7f19f31bc640 1 --2- 172.21.15.18:0/2086276168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec0685f0 0x7f19ec0689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.184+0000 7f19f31bc640 1 -- 172.21.15.18:0/2086276168 >> 172.21.15.18:0/2086276168 conn(0x7f19ec077180 msgr2=0x7f19ec077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.184+0000 7f19f31bc640 1 -- 172.21.15.18:0/2086276168 shutdown_connections 2024-02-17T12:50:16.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.184+0000 7f19f31bc640 1 -- 172.21.15.18:0/2086276168 wait complete. 2024-02-17T12:50:16.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.185+0000 7f19f31bc640 1 Processor -- start 2024-02-17T12:50:16.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.185+0000 7f19f31bc640 1 -- start start 2024-02-17T12:50:16.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.185+0000 7f19f31bc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec0685f0 0x7f19ec197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.185+0000 7f19f31bc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19ec1064c0 0x7f19ec197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.185+0000 7f19f31bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec19bfe0 0x7f19ec19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.185+0000 7f19f0f58640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec0685f0 0x7f19ec197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f0f58640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec0685f0 0x7f19ec197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52304/0 (socket says 172.21.15.18:52304) 2024-02-17T12:50:16.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f0f58640 1 -- 172.21.15.18:0/3335914101 learned_addr learned my addr 172.21.15.18:0/3335914101 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f31bc640 1 -- 172.21.15.18:0/3335914101 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec19c9d0 con 0x7f19ec0685f0 2024-02-17T12:50:16.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f31bc640 1 -- 172.21.15.18:0/3335914101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec19cb40 con 0x7f19ec19bfe0 2024-02-17T12:50:16.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f31bc640 1 -- 172.21.15.18:0/3335914101 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec19ccb0 con 0x7f19ec1064c0 2024-02-17T12:50:16.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19ebfff640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19ec1064c0 0x7f19ec197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f1759640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec19bfe0 0x7f19ec19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f0f58640 1 -- 172.21.15.18:0/3335914101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19ec1064c0 msgr2=0x7f19ec197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f0f58640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19ec1064c0 0x7f19ec197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f0f58640 1 -- 172.21.15.18:0/3335914101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec19bfe0 msgr2=0x7f19ec19c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f0f58640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec19bfe0 0x7f19ec19c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f0f58640 1 -- 172.21.15.18:0/3335914101 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4009840 con 0x7f19ec0685f0 2024-02-17T12:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f1759640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec19bfe0 0x7f19ec19c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19f0f58640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec0685f0 0x7f19ec197770 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f19e000e9f0 tx=0x7f19e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.186+0000 7f19e9ffb640 1 -- 172.21.15.18:0/3335914101 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0009700 con 0x7f19ec0685f0 2024-02-17T12:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.187+0000 7f19f31bc640 1 -- 172.21.15.18:0/3335914101 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec1a4290 con 0x7f19ec0685f0 2024-02-17T12:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.187+0000 7f19e9ffb640 1 -- 172.21.15.18:0/3335914101 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19e000ccc0 con 0x7f19ec0685f0 2024-02-17T12:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.187+0000 7f19e9ffb640 1 -- 172.21.15.18:0/3335914101 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0018820 con 0x7f19ec0685f0 2024-02-17T12:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.187+0000 7f19f31bc640 1 -- 172.21.15.18:0/3335914101 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec1a4740 con 0x7f19ec0685f0 2024-02-17T12:50:16.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.189+0000 7f19e9ffb640 1 -- 172.21.15.18:0/3335914101 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f19e0010430 con 0x7f19ec0685f0 2024-02-17T12:50:16.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19e9ffb640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f19dc0a2350 0x7f19dc0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19f5d8d740 1 Processor -- start 2024-02-17T12:50:16.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19f5d8d740 1 -- start start 2024-02-17T12:50:16.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19ebfff640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f19dc0a2350 0x7f19dc0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19e9ffb640 1 -- 172.21.15.18:0/3335914101 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f19e0014070 con 0x7f19ec0685f0 2024-02-17T12:50:16.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19f5d8d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bac30 0x5623255bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19f5d8d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bb5e0 0x5623255c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19f5d8d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19f5d8d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562325576960 con 0x5623255bac30 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19f5d8d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232552a310 con 0x5623255c8b60 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19f5d8d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562325527840 con 0x5623255bb5e0 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19f0f58640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bac30 0x5623255bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19f0f58640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bac30 0x5623255bb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52316/0 (socket says 172.21.15.18:52316) 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19f0f58640 1 -- 172.21.15.18:0/2627159381 learned_addr learned my addr 172.21.15.18:0/2627159381 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.190+0000 7f19f1759640 1 --2- 172.21.15.18:0/2627159381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19ebfff640 1 --2- 172.21.15.18:0/2627159381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bb5e0 0x5623255c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19ebfff640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f19dc0a2350 0x7f19dc0a4810 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0ffb00 tx=0x7f19d8009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19f0f58640 1 -- 172.21.15.18:0/2627159381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bb5e0 msgr2=0x5623255c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19f0f58640 1 --2- 172.21.15.18:0/2627159381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bb5e0 0x5623255c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19f0f58640 1 -- 172.21.15.18:0/2627159381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 msgr2=0x5623255caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19f0f58640 1 --2- 172.21.15.18:0/2627159381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19f0f58640 1 -- 172.21.15.18:0/2627159381 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623254e5060 con 0x5623255bac30 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19ebfff640 1 --2- 172.21.15.18:0/2627159381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bb5e0 0x5623255c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19f1759640 1 --2- 172.21.15.18:0/2627159381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19f0f58640 1 --2- 172.21.15.18:0/2627159381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bac30 0x5623255bb010 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f19e000e9c0 tx=0x7f19e009ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bc09ff157ac7e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19d6ffd640 1 -- 172.21.15.18:0/2627159381 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e009af40 con 0x5623255bac30 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19d6ffd640 1 -- 172.21.15.18:0/2627159381 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19e009f090 con 0x5623255bac30 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.191+0000 7f19d6ffd640 1 -- 172.21.15.18:0/2627159381 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e00cc2b0 con 0x5623255bac30 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.192+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2627159381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bac30 msgr2=0x5623255bb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.192+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/2627159381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bac30 0x5623255bb010 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f19e000e9c0 tx=0x7f19e009ab80 comp rx=0 tx=0).stop 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.192+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2627159381 shutdown_connections 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.192+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/2627159381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.192+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/2627159381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bb5e0 0x5623255c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.192+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/2627159381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bac30 0x5623255bb010 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.192+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2627159381 >> 172.21.15.18:0/2627159381 conn(0x5623255b5980 msgr2=0x5623255b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.192+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2627159381 shutdown_connections 2024-02-17T12:50:16.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.192+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2627159381 wait complete. 2024-02-17T12:50:16.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.192+0000 7f19f5d8d740 1 -- start start 2024-02-17T12:50:16.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.192+0000 7f19f5d8d740 1 Processor -- start 2024-02-17T12:50:16.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19f5d8d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bac30 0x5623255c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19f5d8d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19f5d8d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19f5d8d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562325523020 con 0x5623255bb5e0 2024-02-17T12:50:16.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19f5d8d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623251f6430 con 0x5623255c8b60 2024-02-17T12:50:16.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19f5d8d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623251b2e00 con 0x5623255bac30 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19f5d8d740 10 client.0 _ll_register_callbacks cb 0x5623254cacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19ebfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19ebfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52320/0 (socket says 172.21.15.18:52320) 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19ebfff640 1 -- 172.21.15.18:0/4256647251 learned_addr learned my addr 172.21.15.18:0/4256647251 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19f1759640 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19f0f58640 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bac30 0x5623255c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19ebfff640 1 -- 172.21.15.18:0/4256647251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bac30 msgr2=0x5623255c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19ebfff640 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bac30 0x5623255c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19ebfff640 1 -- 172.21.15.18:0/4256647251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 msgr2=0x5623255c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19ebfff640 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19ebfff640 1 -- 172.21.15.18:0/4256647251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec1a4740 con 0x5623255bb5e0 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.193+0000 7f19f1759640 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19ebfff640 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255c6100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f19d80045a0 tx=0x7f19d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e18f7e57a2f320 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:16.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19cbfff640 1 -- 172.21.15.18:0/4256647251 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d8005220 con 0x5623255bb5e0 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19cbfff640 1 -- 172.21.15.18:0/4256647251 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f19d8005380 con 0x5623255bb5e0 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19f5d8d740 1 -- 172.21.15.18:0/4256647251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623252227e0 con 0x5623255bb5e0 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19cbfff640 1 -- 172.21.15.18:0/4256647251 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19d8005540 con 0x5623255bb5e0 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19f5d8d740 10 client.15720 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19f5d8d740 1 -- 172.21.15.18:0/4256647251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5623255216a0 con 0x5623255bb5e0 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19cbfff640 1 -- 172.21.15.18:0/4256647251 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19d8004c80 con 0x5623255bb5e0 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19cbfff640 1 -- 172.21.15.18:0/4256647251 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f19d801eba0 con 0x5623255bb5e0 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19f5d8d740 10 client.15720 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19f5d8d740 10 client.15720 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19f5d8d740 20 client.15720 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5623255d8e50 0x5623255db270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19f5d8d740 4 client.15720 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.194+0000 7f19f5d8d740 1 -- 172.21.15.18:0/4256647251 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623252227e0 con 0x5623255d8e50 2024-02-17T12:50:16.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.195+0000 7f19f0f58640 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5623255d8e50 0x5623255db270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.195+0000 7f19f0f58640 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5623255d8e50 0x5623255db270 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8eb63549fdf2a352 server_cookie=da8374baeebedfaf in_seq=0 out_seq=0 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.195+0000 7f19cbfff640 10 client.15720 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19cbfff640 1 -- 172.21.15.18:0/4256647251 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623252227e0 con 0x5623255d8e50 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19cbfff640 10 client.15720 handle_command_reply: tid=0 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19f5d8d740 1 client.15720 shutdown 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19f5d8d740 1 -- 172.21.15.18:0/4256647251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 msgr2=0x5623255c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255c6100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f19d80045a0 tx=0x7f19d8004730 comp rx=0 tx=0).stop 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19f5d8d740 1 -- 172.21.15.18:0/4256647251 shutdown_connections 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5623255d8e50 0x5623255db270 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19cbfff640 0 client.15720 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255c6100 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/4256647251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bac30 0x5623255c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19f5d8d740 1 -- 172.21.15.18:0/4256647251 >> 172.21.15.18:0/4256647251 conn(0x5623255b5980 msgr2=0x5623255b76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19f5d8d740 1 -- 172.21.15.18:0/4256647251 shutdown_connections 2024-02-17T12:50:16.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19f5d8d740 1 -- 172.21.15.18:0/4256647251 wait complete. 2024-02-17T12:50:16.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.196+0000 7f19f5d8d740 20 client.15720 trim_cache size 0 max 16384 2024-02-17T12:50:16.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.208+0000 7f19f5d8d740 1 Processor -- start 2024-02-17T12:50:16.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.208+0000 7f19f5d8d740 1 -- start start 2024-02-17T12:50:16.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.208+0000 7f19f5d8d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255bac30 0x5623255b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.208+0000 7f19f5d8d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255c6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.208+0000 7f19f5d8d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255c8b60 0x5623255c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.208+0000 7f19f5d8d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623254a8130 con 0x5623255bb5e0 2024-02-17T12:50:16.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.208+0000 7f19f5d8d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623255ba590 con 0x5623255bac30 2024-02-17T12:50:16.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.208+0000 7f19f5d8d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623255e04c0 con 0x5623255c8b60 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.208+0000 7f19ebfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.208+0000 7f19ebfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52324/0 (socket says 172.21.15.18:52324) 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.208+0000 7f19ebfff640 1 -- 172.21.15.18:0/490437248 learned_addr learned my addr 172.21.15.18:0/490437248 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.209+0000 7f19f1759640 1 --2- 172.21.15.18:0/490437248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255c8b60 0x5623255c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.209+0000 7f19f0f58640 1 --2- 172.21.15.18:0/490437248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255bac30 0x5623255b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.209+0000 7f19ebfff640 1 -- 172.21.15.18:0/490437248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255c8b60 msgr2=0x5623255c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.209+0000 7f19ebfff640 1 --2- 172.21.15.18:0/490437248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255c8b60 0x5623255c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.209+0000 7f19ebfff640 1 -- 172.21.15.18:0/490437248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255bac30 msgr2=0x5623255b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.209+0000 7f19ebfff640 1 --2- 172.21.15.18:0/490437248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255bac30 0x5623255b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.209+0000 7f19ebfff640 1 -- 172.21.15.18:0/490437248 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623254bdc20 con 0x5623255bb5e0 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.209+0000 7f19ebfff640 1 --2- 172.21.15.18:0/490437248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255c6460 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f19d801ef30 tx=0x7f19d8003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6167642010b15e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.209+0000 7f19d67fc640 1 -- 172.21.15.18:0/490437248 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d80043e0 con 0x5623255bb5e0 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.209+0000 7f19d67fc640 1 -- 172.21.15.18:0/490437248 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19d802fde0 con 0x5623255bb5e0 2024-02-17T12:50:16.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.209+0000 7f19d67fc640 1 -- 172.21.15.18:0/490437248 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d8039e90 con 0x5623255bb5e0 2024-02-17T12:50:16.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.210+0000 7f19f5d8d740 1 -- 172.21.15.18:0/490437248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 msgr2=0x5623255c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.210+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/490437248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255c6460 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f19d801ef30 tx=0x7f19d8003e30 comp rx=0 tx=0).stop 2024-02-17T12:50:16.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.210+0000 7f19f5d8d740 1 -- 172.21.15.18:0/490437248 shutdown_connections 2024-02-17T12:50:16.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.210+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/490437248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255c8b60 0x5623255c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.210+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/490437248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255c6460 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.210+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/490437248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255bac30 0x5623255b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.210+0000 7f19f5d8d740 1 -- 172.21.15.18:0/490437248 >> 172.21.15.18:0/490437248 conn(0x5623255b5980 msgr2=0x5623255c72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.210+0000 7f19f5d8d740 1 -- 172.21.15.18:0/490437248 shutdown_connections 2024-02-17T12:50:16.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.210+0000 7f19f5d8d740 1 -- 172.21.15.18:0/490437248 wait complete. 2024-02-17T12:50:16.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.211+0000 7f19f5d8d740 1 -- start start 2024-02-17T12:50:16.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.211+0000 7f19f5d8d740 1 Processor -- start 2024-02-17T12:50:16.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.211+0000 7f19f5d8d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bac30 0x5623255cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.211+0000 7f19f5d8d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.211+0000 7f19f5d8d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.211+0000 7f19f5d8d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623255d0280 con 0x5623255bb5e0 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.211+0000 7f19ebfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.211+0000 7f19f5d8d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623255cc3b0 con 0x5623255c8b60 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.211+0000 7f19f5d8d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623255cc570 con 0x5623255bac30 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.211+0000 7f19ebfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52328/0 (socket says 172.21.15.18:52328) 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.211+0000 7f19ebfff640 1 -- 172.21.15.18:0/2749369145 learned_addr learned my addr 172.21.15.18:0/2749369145 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.211+0000 7f19f5d8d740 10 client.0 _ll_register_callbacks cb 0x7f19e0096ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19f0f58640 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bac30 0x5623255cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19f1759640 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19ebfff640 1 -- 172.21.15.18:0/2749369145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bac30 msgr2=0x5623255cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19ebfff640 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bac30 0x5623255cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19ebfff640 1 -- 172.21.15.18:0/2749369145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 msgr2=0x5623255cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19ebfff640 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19ebfff640 1 -- 172.21.15.18:0/2749369145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623255216a0 con 0x5623255bb5e0 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19f1759640 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19ebfff640 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255cb930 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f19d8043040 tx=0x7f19d8039990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513ff14db4f7c2d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19cbfff640 1 -- 172.21.15.18:0/2749369145 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d8005450 con 0x5623255bb5e0 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19cbfff640 1 -- 172.21.15.18:0/2749369145 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f19d803d070 con 0x5623255bb5e0 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2749369145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19e009f090 con 0x5623255bb5e0 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19cbfff640 1 -- 172.21.15.18:0/2749369145 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19d80055b0 con 0x5623255bb5e0 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19f5d8d740 10 client.15726 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.212+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2749369145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5623254e38c0 con 0x5623255bb5e0 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.213+0000 7f19cbfff640 1 -- 172.21.15.18:0/2749369145 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19d8058020 con 0x5623255bb5e0 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.213+0000 7f19cbfff640 1 -- 172.21.15.18:0/2749369145 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f19d802f8a0 con 0x5623255bb5e0 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.213+0000 7f19f5d8d740 10 client.15726 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:16.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.213+0000 7f19f5d8d740 10 client.15726 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.213+0000 7f19f5d8d740 20 client.15726 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.213+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56232562d3a0 0x56232562f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.213+0000 7f19f5d8d740 4 client.15726 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.213+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2749369145 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f19e009f090 con 0x56232562d3a0 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.213+0000 7f19f0f58640 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56232562d3a0 0x56232562f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19cbfff640 10 client.15726 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19f0f58640 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56232562d3a0 0x56232562f790 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40ce23aab7d7d7db server_cookie=fc613f17b76959f4 in_seq=0 out_seq=0 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19cbfff640 1 -- 172.21.15.18:0/2749369145 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f19e009f090 con 0x56232562d3a0 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19cbfff640 10 client.15726 handle_command_reply: tid=0 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19f5d8d740 1 client.15726 shutdown 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2749369145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 msgr2=0x5623255cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255cb930 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f19d8043040 tx=0x7f19d8039990 comp rx=0 tx=0).stop 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2749369145 shutdown_connections 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56232562d3a0 0x56232562f790 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19cbfff640 0 client.15726 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:50:16.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623255c8b60 0x5623255cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5623255bb5e0 0x5623255cb930 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19f5d8d740 1 --2- 172.21.15.18:0/2749369145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623255bac30 0x5623255cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2749369145 >> 172.21.15.18:0/2749369145 conn(0x5623255b5980 msgr2=0x5623255b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2749369145 shutdown_connections 2024-02-17T12:50:16.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.214+0000 7f19f5d8d740 1 -- 172.21.15.18:0/2749369145 wait complete. 2024-02-17T12:50:16.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.215+0000 7f19f5d8d740 20 client.15726 trim_cache size 0 max 16384 2024-02-17T12:50:16.211 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:50:16.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.215+0000 7f19f31bc640 1 -- 172.21.15.18:0/3335914101 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f19dc0a2350 msgr2=0x7f19dc0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.215+0000 7f19f31bc640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f19dc0a2350 0x7f19dc0a4810 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0ffb00 tx=0x7f19d8009260 comp rx=0 tx=0).stop 2024-02-17T12:50:16.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.215+0000 7f19f31bc640 1 -- 172.21.15.18:0/3335914101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec0685f0 msgr2=0x7f19ec197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.215+0000 7f19f31bc640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec0685f0 0x7f19ec197770 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f19e000e9f0 tx=0x7f19e000edf0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.215+0000 7f19f31bc640 1 -- 172.21.15.18:0/3335914101 shutdown_connections 2024-02-17T12:50:16.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.215+0000 7f19f31bc640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f19dc0a2350 0x7f19dc0a4810 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.215+0000 7f19f31bc640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec19bfe0 0x7f19ec19c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.215+0000 7f19f31bc640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19ec1064c0 0x7f19ec197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.215+0000 7f19f31bc640 1 --2- 172.21.15.18:0/3335914101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec0685f0 0x7f19ec197770 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.215+0000 7f19f31bc640 1 -- 172.21.15.18:0/3335914101 >> 172.21.15.18:0/3335914101 conn(0x7f19ec077180 msgr2=0x7f19ec0fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.215+0000 7f19f31bc640 1 -- 172.21.15.18:0/3335914101 shutdown_connections 2024-02-17T12:50:16.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.216+0000 7f19f31bc640 1 -- 172.21.15.18:0/3335914101 wait complete. 2024-02-17T12:50:16.221 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:50:16.221 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:50:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.342+0000 7f605b683640 1 -- 172.21.15.18:0/3178937938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6054068fa0 msgr2=0x7f6054105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.342+0000 7f605b683640 1 --2- 172.21.15.18:0/3178937938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6054068fa0 0x7f6054105de0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6044009930 tx=0x7f604402bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.342+0000 7f605b683640 1 -- 172.21.15.18:0/3178937938 shutdown_connections 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.342+0000 7f605b683640 1 --2- 172.21.15.18:0/3178937938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60541064c0 0x7f605410e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.342+0000 7f605b683640 1 --2- 172.21.15.18:0/3178937938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6054068fa0 0x7f6054105de0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.342+0000 7f605b683640 1 --2- 172.21.15.18:0/3178937938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60540685f0 0x7f60540689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.342+0000 7f605b683640 1 -- 172.21.15.18:0/3178937938 >> 172.21.15.18:0/3178937938 conn(0x7f6054077180 msgr2=0x7f6054077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.343+0000 7f605b683640 1 -- 172.21.15.18:0/3178937938 shutdown_connections 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.343+0000 7f605b683640 1 -- 172.21.15.18:0/3178937938 wait complete. 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.343+0000 7f605b683640 1 Processor -- start 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.343+0000 7f605b683640 1 -- start start 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.343+0000 7f605b683640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60540685f0 0x7f60541955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.343+0000 7f605b683640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60541064c0 0x7f6054195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.343+0000 7f605b683640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054199e90 0x7f605419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.343+0000 7f605b683640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605419a880 con 0x7f60541064c0 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.343+0000 7f605b683640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605419a9f0 con 0x7f6054199e90 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.343+0000 7f605b683640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605419ab60 con 0x7f60540685f0 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f6058c1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60541064c0 0x7f6054195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f6058c1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60541064c0 0x7f6054195b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52344/0 (socket says 172.21.15.18:52344) 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f6058c1e640 1 -- 172.21.15.18:0/3369071177 learned_addr learned my addr 172.21.15.18:0/3369071177 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f6059c20640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054199e90 0x7f605419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f605941f640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60540685f0 0x7f60541955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f6058c1e640 1 -- 172.21.15.18:0/3369071177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60540685f0 msgr2=0x7f60541955d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f6058c1e640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60540685f0 0x7f60541955d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f6058c1e640 1 -- 172.21.15.18:0/3369071177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054199e90 msgr2=0x7f605419a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f6058c1e640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054199e90 0x7f605419a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f6058c1e640 1 -- 172.21.15.18:0/3369071177 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6044009560 con 0x7f60541064c0 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f6059c20640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054199e90 0x7f605419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f6058c1e640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60541064c0 0x7f6054195b10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6044000c00 tx=0x7f6044004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f604a7fc640 1 -- 172.21.15.18:0/3369071177 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60440043e0 con 0x7f60541064c0 2024-02-17T12:50:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.344+0000 7f605b683640 1 -- 172.21.15.18:0/3369071177 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60541a2140 con 0x7f60541064c0 2024-02-17T12:50:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.345+0000 7f604a7fc640 1 -- 172.21.15.18:0/3369071177 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60440052d0 con 0x7f60541064c0 2024-02-17T12:50:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.345+0000 7f604a7fc640 1 -- 172.21.15.18:0/3369071177 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6044003d70 con 0x7f60541064c0 2024-02-17T12:50:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.345+0000 7f605b683640 1 -- 172.21.15.18:0/3369071177 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60541a25f0 con 0x7f60541064c0 2024-02-17T12:50:16.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.347+0000 7f605e24b740 1 Processor -- start 2024-02-17T12:50:16.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.347+0000 7f605e24b740 1 -- start start 2024-02-17T12:50:16.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f605e24b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894f060 0x55eca894f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f605e24b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894fa10 0x55eca8960a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f605e24b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca8960f60 0x55eca8963350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f605e24b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca890a960 con 0x55eca894f060 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f605e24b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca88be310 con 0x55eca8960f60 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f605e24b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca88bb840 con 0x55eca894fa10 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f6059c20640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894f060 0x55eca894f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f6059c20640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894f060 0x55eca894f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52354/0 (socket says 172.21.15.18:52354) 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f6059c20640 1 -- 172.21.15.18:0/1684709353 learned_addr learned my addr 172.21.15.18:0/1684709353 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f605941f640 1 --2- 172.21.15.18:0/1684709353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894fa10 0x55eca8960a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f6059c20640 1 -- 172.21.15.18:0/1684709353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894fa10 msgr2=0x55eca8960a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f6059c20640 1 --2- 172.21.15.18:0/1684709353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894fa10 0x55eca8960a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f6059c20640 1 -- 172.21.15.18:0/1684709353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca8960f60 msgr2=0x55eca8963350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f6059c20640 1 --2- 172.21.15.18:0/1684709353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca8960f60 0x55eca8963350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f6059c20640 1 -- 172.21.15.18:0/1684709353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eca8879060 con 0x55eca894f060 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f605941f640 1 --2- 172.21.15.18:0/1684709353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894fa10 0x55eca8960a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:50:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.348+0000 7f6059c20640 1 --2- 172.21.15.18:0/1684709353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894f060 0x55eca894f440 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f604c00c620 tx=0x7f604c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fe3ead019253d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.349+0000 7f603f7fe640 1 -- 172.21.15.18:0/1684709353 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c014070 con 0x55eca894f060 2024-02-17T12:50:16.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.349+0000 7f603f7fe640 1 -- 172.21.15.18:0/1684709353 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f604c003e50 con 0x55eca894f060 2024-02-17T12:50:16.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.349+0000 7f603f7fe640 1 -- 172.21.15.18:0/1684709353 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c00a6f0 con 0x55eca894f060 2024-02-17T12:50:16.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.349+0000 7f605e24b740 1 -- 172.21.15.18:0/1684709353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894f060 msgr2=0x55eca894f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.349+0000 7f605e24b740 1 --2- 172.21.15.18:0/1684709353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894f060 0x55eca894f440 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f604c00c620 tx=0x7f604c009500 comp rx=0 tx=0).stop 2024-02-17T12:50:16.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.349+0000 7f605e24b740 1 -- 172.21.15.18:0/1684709353 shutdown_connections 2024-02-17T12:50:16.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.349+0000 7f605e24b740 1 --2- 172.21.15.18:0/1684709353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca8960f60 0x55eca8963350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.349+0000 7f604a7fc640 1 -- 172.21.15.18:0/3369071177 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f6044005440 con 0x7f60541064c0 2024-02-17T12:50:16.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.349+0000 7f605e24b740 1 --2- 172.21.15.18:0/1684709353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894fa10 0x55eca8960a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.349+0000 7f605e24b740 1 --2- 172.21.15.18:0/1684709353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894f060 0x55eca894f440 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.349+0000 7f605e24b740 1 -- 172.21.15.18:0/1684709353 >> 172.21.15.18:0/1684709353 conn(0x55eca8949980 msgr2=0x55eca8949d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.350+0000 7f605e24b740 1 -- 172.21.15.18:0/1684709353 shutdown_connections 2024-02-17T12:50:16.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.350+0000 7f605e24b740 1 -- 172.21.15.18:0/1684709353 wait complete. 2024-02-17T12:50:16.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.350+0000 7f605e24b740 1 -- start start 2024-02-17T12:50:16.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.350+0000 7f605e24b740 1 Processor -- start 2024-02-17T12:50:16.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.350+0000 7f604a7fc640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f60400a2260 0x7f60400a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.350+0000 7f604a7fc640 1 -- 172.21.15.18:0/3369071177 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f60440eff50 con 0x7f60541064c0 2024-02-17T12:50:16.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.350+0000 7f605e24b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894f060 0x55eca895e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f605e24b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894fa10 0x55eca895e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f605e24b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca895eb00 0x55eca896cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f605e24b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca88b7020 con 0x55eca894fa10 2024-02-17T12:50:16.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f605e24b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca858a430 con 0x55eca895eb00 2024-02-17T12:50:16.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f605e24b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca8546e00 con 0x55eca894f060 2024-02-17T12:50:16.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f6058c1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894fa10 0x55eca895e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f605e24b740 10 client.0 _ll_register_callbacks cb 0x55eca885ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:16.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f6058c1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894fa10 0x55eca895e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52360/0 (socket says 172.21.15.18:52360) 2024-02-17T12:50:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f6058c1e640 1 -- 172.21.15.18:0/4224831180 learned_addr learned my addr 172.21.15.18:0/4224831180 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f6059c20640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f60400a2260 0x7f60400a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f6058c1e640 1 -- 172.21.15.18:0/4224831180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894f060 msgr2=0x55eca895e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f6058c1e640 1 --2- 172.21.15.18:0/4224831180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894f060 0x55eca895e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f6058c1e640 1 -- 172.21.15.18:0/4224831180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca895eb00 msgr2=0x55eca896cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f6058c1e640 1 --2- 172.21.15.18:0/4224831180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca895eb00 0x55eca896cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f6058c1e640 1 -- 172.21.15.18:0/4224831180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c00c2d0 con 0x55eca894fa10 2024-02-17T12:50:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f6059c20640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f60400a2260 0x7f60400a4720 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f604c004740 tx=0x7f604c0047a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f6058c1e640 1 --2- 172.21.15.18:0/4224831180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894fa10 0x55eca895e5c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f60440bc550 tx=0x7f604402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d47991d7a0eb77d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.351+0000 7f603cff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:16.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.352+0000 7f603cff9640 1 -- 172.21.15.18:0/4224831180 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60440bcf50 con 0x55eca894fa10 2024-02-17T12:50:16.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.352+0000 7f603cff9640 1 -- 172.21.15.18:0/4224831180 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f60440ede40 con 0x55eca894fa10 2024-02-17T12:50:16.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.352+0000 7f605e24b740 1 -- 172.21.15.18:0/4224831180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eca85b67e0 con 0x55eca894fa10 2024-02-17T12:50:16.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.352+0000 7f603cff9640 1 -- 172.21.15.18:0/4224831180 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60440bc850 con 0x55eca894fa10 2024-02-17T12:50:16.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.352+0000 7f605e24b740 10 client.15738 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:16.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.352+0000 7f605e24b740 1 -- 172.21.15.18:0/4224831180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55eca88b56a0 con 0x55eca894fa10 2024-02-17T12:50:16.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.352+0000 7f603cff9640 1 -- 172.21.15.18:0/4224831180 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55eca85b67e0 con 0x55eca894fa10 2024-02-17T12:50:16.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.352+0000 7f603cff9640 1 -- 172.21.15.18:0/4224831180 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f604403c070 con 0x55eca894fa10 2024-02-17T12:50:16.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.352+0000 7f605e24b740 10 client.15738 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:16.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.352+0000 7f605e24b740 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:16.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.353+0000 7f605e24b740 20 client.15738 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:16.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.353+0000 7f605e24b740 1 --2- 172.21.15.18:0/4224831180 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eca8971330 0x55eca8973720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.353+0000 7f605e24b740 4 client.15738 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:16.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.353+0000 7f605e24b740 1 -- 172.21.15.18:0/4224831180 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eca85f3720 con 0x55eca8971330 2024-02-17T12:50:16.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.353+0000 7f605941f640 1 --2- 172.21.15.18:0/4224831180 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eca8971330 0x55eca8973720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.353+0000 7f603cff9640 10 client.15738 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:16.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.353+0000 7f605941f640 1 --2- 172.21.15.18:0/4224831180 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eca8971330 0x55eca8973720 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d526689ec4a70617 server_cookie=24c2632398ec7bc4 in_seq=0 out_seq=0 2024-02-17T12:50:16.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.354+0000 7f603cff9640 1 -- 172.21.15.18:0/4224831180 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eca85f3720 con 0x55eca8971330 2024-02-17T12:50:16.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.354+0000 7f603cff9640 10 client.15738 handle_command_reply: tid=0 2024-02-17T12:50:16.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.354+0000 7f605e24b740 1 client.15738 shutdown 2024-02-17T12:50:16.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.354+0000 7f605e24b740 1 -- 172.21.15.18:0/4224831180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894fa10 msgr2=0x55eca895e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.354+0000 7f605e24b740 1 --2- 172.21.15.18:0/4224831180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894fa10 0x55eca895e5c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f60440bc550 tx=0x7f604402bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:16.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.354+0000 7f605e24b740 1 -- 172.21.15.18:0/4224831180 shutdown_connections 2024-02-17T12:50:16.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.354+0000 7f605e24b740 1 --2- 172.21.15.18:0/4224831180 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eca8971330 0x55eca8973720 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.354+0000 7f603cff9640 0 client.15738 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:16.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.354+0000 7f605e24b740 1 --2- 172.21.15.18:0/4224831180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca895eb00 0x55eca896cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.354+0000 7f605e24b740 1 --2- 172.21.15.18:0/4224831180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca894fa10 0x55eca895e5c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.354+0000 7f605e24b740 1 --2- 172.21.15.18:0/4224831180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894f060 0x55eca895e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.354+0000 7f605e24b740 1 -- 172.21.15.18:0/4224831180 >> 172.21.15.18:0/4224831180 conn(0x55eca8949980 msgr2=0x55eca894b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.355+0000 7f605e24b740 1 -- 172.21.15.18:0/4224831180 shutdown_connections 2024-02-17T12:50:16.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.355+0000 7f605e24b740 1 -- 172.21.15.18:0/4224831180 wait complete. 2024-02-17T12:50:16.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.355+0000 7f605e24b740 20 client.15738 trim_cache size 0 max 16384 2024-02-17T12:50:16.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f605e24b740 1 Processor -- start 2024-02-17T12:50:16.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f605e24b740 1 -- start start 2024-02-17T12:50:16.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f605e24b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca894f060 0x55eca895b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f605e24b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca895eb00 0x55eca895b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f605e24b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca8971330 0x55eca89482f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f605e24b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca883c130 con 0x55eca895eb00 2024-02-17T12:50:16.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f605e24b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca894ec10 con 0x55eca894f060 2024-02-17T12:50:16.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f605e24b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca894e5b0 con 0x55eca8971330 2024-02-17T12:50:16.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f6058c1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca895eb00 0x55eca895b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f6058c1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca895eb00 0x55eca895b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52372/0 (socket says 172.21.15.18:52372) 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f6058c1e640 1 -- 172.21.15.18:0/3638449694 learned_addr learned my addr 172.21.15.18:0/3638449694 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f6059c20640 1 --2- 172.21.15.18:0/3638449694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca8971330 0x55eca89482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.367+0000 7f605941f640 1 --2- 172.21.15.18:0/3638449694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca894f060 0x55eca895b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.368+0000 7f6058c1e640 1 -- 172.21.15.18:0/3638449694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca8971330 msgr2=0x55eca89482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.368+0000 7f6058c1e640 1 --2- 172.21.15.18:0/3638449694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca8971330 0x55eca89482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.368+0000 7f6058c1e640 1 -- 172.21.15.18:0/3638449694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca894f060 msgr2=0x55eca895b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.368+0000 7f6058c1e640 1 --2- 172.21.15.18:0/3638449694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca894f060 0x55eca895b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.368+0000 7f6058c1e640 1 -- 172.21.15.18:0/3638449694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eca8851c20 con 0x55eca895eb00 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.368+0000 7f6058c1e640 1 --2- 172.21.15.18:0/3638449694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca895eb00 0x55eca895b860 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f60440b8cb0 tx=0x7f60440bad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0af61c431c2684 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.368+0000 7f603effd640 1 -- 172.21.15.18:0/3638449694 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60440bc390 con 0x55eca895eb00 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.368+0000 7f603effd640 1 -- 172.21.15.18:0/3638449694 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60440ed5c0 con 0x55eca895eb00 2024-02-17T12:50:16.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.368+0000 7f603effd640 1 -- 172.21.15.18:0/3638449694 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60440c09a0 con 0x55eca895eb00 2024-02-17T12:50:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.369+0000 7f605e24b740 1 -- 172.21.15.18:0/3638449694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca895eb00 msgr2=0x55eca895b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.369+0000 7f605e24b740 1 --2- 172.21.15.18:0/3638449694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca895eb00 0x55eca895b860 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f60440b8cb0 tx=0x7f60440bad60 comp rx=0 tx=0).stop 2024-02-17T12:50:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.369+0000 7f605e24b740 1 -- 172.21.15.18:0/3638449694 shutdown_connections 2024-02-17T12:50:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.369+0000 7f605e24b740 1 --2- 172.21.15.18:0/3638449694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca8971330 0x55eca89482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.369+0000 7f605e24b740 1 --2- 172.21.15.18:0/3638449694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca895eb00 0x55eca895b860 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.369+0000 7f605e24b740 1 --2- 172.21.15.18:0/3638449694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca894f060 0x55eca895b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.369+0000 7f605e24b740 1 -- 172.21.15.18:0/3638449694 >> 172.21.15.18:0/3638449694 conn(0x55eca8949980 msgr2=0x55eca896c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.370+0000 7f605e24b740 1 -- 172.21.15.18:0/3638449694 shutdown_connections 2024-02-17T12:50:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.370+0000 7f605e24b740 1 -- 172.21.15.18:0/3638449694 wait complete. 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.370+0000 7f605e24b740 1 -- start start 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.370+0000 7f605e24b740 1 Processor -- start 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.370+0000 7f605e24b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894f060 0x55eca8966a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f605e24b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca8971330 0x55eca8966f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f605e24b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca8967490 0x55eca89c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f605e24b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca89c7d70 con 0x55eca8967490 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f605e24b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca89c7eb0 con 0x55eca8971330 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f605e24b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca89c8020 con 0x55eca894f060 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f605e24b740 10 client.0 _ll_register_callbacks cb 0x7f604c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f6059c20640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca8967490 0x55eca89c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f605941f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894f060 0x55eca8966a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f6059c20640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca8967490 0x55eca89c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52388/0 (socket says 172.21.15.18:52388) 2024-02-17T12:50:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f6059c20640 1 -- 172.21.15.18:0/2308744306 learned_addr learned my addr 172.21.15.18:0/2308744306 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f6058c1e640 1 --2- 172.21.15.18:0/2308744306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca8971330 0x55eca8966f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f6059c20640 1 -- 172.21.15.18:0/2308744306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894f060 msgr2=0x55eca8966a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f6059c20640 1 --2- 172.21.15.18:0/2308744306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894f060 0x55eca8966a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f6059c20640 1 -- 172.21.15.18:0/2308744306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca8971330 msgr2=0x55eca8966f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f6059c20640 1 --2- 172.21.15.18:0/2308744306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca8971330 0x55eca8966f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f6059c20640 1 -- 172.21.15.18:0/2308744306 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eca88b56a0 con 0x55eca8967490 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f6059c20640 1 --2- 172.21.15.18:0/2308744306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca8967490 0x55eca89c7830 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f604c00cf70 tx=0x7f604c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28fec1d03707fdb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.371+0000 7f603cff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.372+0000 7f603cff9640 1 -- 172.21.15.18:0/2308744306 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c014040 con 0x55eca8967490 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.372+0000 7f603cff9640 1 -- 172.21.15.18:0/2308744306 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f604c010460 con 0x55eca8967490 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.372+0000 7f605e24b740 1 -- 172.21.15.18:0/2308744306 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f604c003e50 con 0x55eca8967490 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.372+0000 7f603cff9640 1 -- 172.21.15.18:0/2308744306 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f604c010620 con 0x55eca8967490 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.372+0000 7f605e24b740 10 client.15744 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:16.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.372+0000 7f605e24b740 1 -- 172.21.15.18:0/2308744306 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55eca88778c0 con 0x55eca8967490 2024-02-17T12:50:16.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.372+0000 7f603cff9640 1 -- 172.21.15.18:0/2308744306 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f604c003e50 con 0x55eca8967490 2024-02-17T12:50:16.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.373+0000 7f603cff9640 1 -- 172.21.15.18:0/2308744306 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f604c01f040 con 0x55eca8967490 2024-02-17T12:50:16.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.373+0000 7f605e24b740 10 client.15744 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:16.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.373+0000 7f605e24b740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.373+0000 7f605e24b740 20 client.15744 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.373+0000 7f605e24b740 1 --2- 172.21.15.18:0/2308744306 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eca89c9820 0x55eca89cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.373+0000 7f605e24b740 4 client.15744 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.373+0000 7f605e24b740 1 -- 172.21.15.18:0/2308744306 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eca8642890 con 0x55eca89c9820 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.373+0000 7f605941f640 1 --2- 172.21.15.18:0/2308744306 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eca89c9820 0x55eca89cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.373+0000 7f603cff9640 10 client.15744 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.373+0000 7f605941f640 1 --2- 172.21.15.18:0/2308744306 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eca89c9820 0x55eca89cbc40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5930cc5f40b15431 server_cookie=6e4099d5ed66b332 in_seq=0 out_seq=0 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f603cff9640 1 -- 172.21.15.18:0/2308744306 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eca8642890 con 0x55eca89c9820 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f603cff9640 10 client.15744 handle_command_reply: tid=0 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f605e24b740 1 client.15744 shutdown 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f605e24b740 1 -- 172.21.15.18:0/2308744306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca8967490 msgr2=0x55eca89c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f605e24b740 1 --2- 172.21.15.18:0/2308744306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca8967490 0x55eca89c7830 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f604c00cf70 tx=0x7f604c009500 comp rx=0 tx=0).stop 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f605e24b740 1 -- 172.21.15.18:0/2308744306 shutdown_connections 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f605e24b740 1 --2- 172.21.15.18:0/2308744306 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eca89c9820 0x55eca89cbc40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f603cff9640 0 client.15744 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:16.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f605e24b740 1 --2- 172.21.15.18:0/2308744306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eca8967490 0x55eca89c7830 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f605e24b740 1 --2- 172.21.15.18:0/2308744306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eca8971330 0x55eca8966f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f605e24b740 1 --2- 172.21.15.18:0/2308744306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eca894f060 0x55eca8966a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f605e24b740 1 -- 172.21.15.18:0/2308744306 >> 172.21.15.18:0/2308744306 conn(0x55eca8949980 msgr2=0x55eca894ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f605e24b740 1 -- 172.21.15.18:0/2308744306 shutdown_connections 2024-02-17T12:50:16.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f605e24b740 1 -- 172.21.15.18:0/2308744306 wait complete. 2024-02-17T12:50:16.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.374+0000 7f605e24b740 20 client.15744 trim_cache size 0 max 16384 2024-02-17T12:50:16.371 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:50:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.375+0000 7f605b683640 1 -- 172.21.15.18:0/3369071177 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f60400a2260 msgr2=0x7f60400a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.375+0000 7f605b683640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f60400a2260 0x7f60400a4720 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f604c004740 tx=0x7f604c0047a0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.375+0000 7f605b683640 1 -- 172.21.15.18:0/3369071177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60541064c0 msgr2=0x7f6054195b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.375+0000 7f605b683640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60541064c0 0x7f6054195b10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6044000c00 tx=0x7f6044004270 comp rx=0 tx=0).stop 2024-02-17T12:50:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.375+0000 7f605b683640 1 -- 172.21.15.18:0/3369071177 shutdown_connections 2024-02-17T12:50:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.375+0000 7f605b683640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f60400a2260 0x7f60400a4720 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.375+0000 7f605b683640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054199e90 0x7f605419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.375+0000 7f605b683640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60541064c0 0x7f6054195b10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.375+0000 7f605b683640 1 --2- 172.21.15.18:0/3369071177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60540685f0 0x7f60541955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.375+0000 7f605b683640 1 -- 172.21.15.18:0/3369071177 >> 172.21.15.18:0/3369071177 conn(0x7f6054077180 msgr2=0x7f60540fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.375+0000 7f605b683640 1 -- 172.21.15.18:0/3369071177 shutdown_connections 2024-02-17T12:50:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:16.375+0000 7f605b683640 1 -- 172.21.15.18:0/3369071177 wait complete. 2024-02-17T12:50:16.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:16 smithi018 ceph-mon[32625]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-02-17T12:50:16.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:16 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/105361699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:50:16.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:16 smithi119 ceph-mon[36188]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-02-17T12:50:16.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:16 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/105361699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:50:16.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:16 smithi146 ceph-mon[36454]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-02-17T12:50:16.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:16 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/105361699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:50:17.382 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:50:17.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:50:17.383 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:50:17.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.502+0000 7fe86959f640 1 -- 172.21.15.18:0/3305349226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe864073e70 msgr2=0x7fe86410d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.502+0000 7fe86959f640 1 --2- 172.21.15.18:0/3305349226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe864073e70 0x7fe86410d350 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe858009900 tx=0x7fe85802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.503+0000 7fe86959f640 1 -- 172.21.15.18:0/3305349226 shutdown_connections 2024-02-17T12:50:17.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.503+0000 7fe86959f640 1 --2- 172.21.15.18:0/3305349226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe86410d890 0x7fe86410fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.503+0000 7fe86959f640 1 --2- 172.21.15.18:0/3305349226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe864073e70 0x7fe86410d350 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.503+0000 7fe86959f640 1 --2- 172.21.15.18:0/3305349226 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe864073550 0x7fe864073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.503+0000 7fe86959f640 1 -- 172.21.15.18:0/3305349226 >> 172.21.15.18:0/3305349226 conn(0x7fe8640fcb20 msgr2=0x7fe8640fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:17.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.503+0000 7fe86959f640 1 -- 172.21.15.18:0/3305349226 shutdown_connections 2024-02-17T12:50:17.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.503+0000 7fe86959f640 1 -- 172.21.15.18:0/3305349226 wait complete. 2024-02-17T12:50:17.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.503+0000 7fe86959f640 1 Processor -- start 2024-02-17T12:50:17.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.503+0000 7fe86959f640 1 -- start start 2024-02-17T12:50:17.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe86959f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe864073550 0x7fe86419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe86959f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe864073e70 0x7fe86419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe86959f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe86410d890 0x7fe864199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe86959f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe864199ee0 con 0x7fe86410d890 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe86959f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86419a050 con 0x7fe864073550 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe86959f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86419a210 con 0x7fe864073e70 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe868d9e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe86410d890 0x7fe864199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe868d9e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe86410d890 0x7fe864199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52412/0 (socket says 172.21.15.18:52412) 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe868d9e640 1 -- 172.21.15.18:0/3938465944 learned_addr learned my addr 172.21.15.18:0/3938465944 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe863fff640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe864073550 0x7fe86419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe8637fe640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe864073e70 0x7fe86419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe863fff640 1 -- 172.21.15.18:0/3938465944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe864073e70 msgr2=0x7fe86419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe863fff640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe864073e70 0x7fe86419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe863fff640 1 -- 172.21.15.18:0/3938465944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe86410d890 msgr2=0x7fe864199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe863fff640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe86410d890 0x7fe864199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe863fff640 1 -- 172.21.15.18:0/3938465944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe858009560 con 0x7fe864073550 2024-02-17T12:50:17.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.504+0000 7fe868d9e640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe86410d890 0x7fe864199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:17.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.505+0000 7fe863fff640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe864073550 0x7fe86419f640 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe85400d6d0 tx=0x7fe85400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:17.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.505+0000 7fe8617fa640 1 -- 172.21.15.18:0/3938465944 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854005220 con 0x7fe864073550 2024-02-17T12:50:17.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.505+0000 7fe8617fa640 1 -- 172.21.15.18:0/3938465944 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe854009d00 con 0x7fe864073550 2024-02-17T12:50:17.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.505+0000 7fe86959f640 1 -- 172.21.15.18:0/3938465944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86419a4f0 con 0x7fe864073550 2024-02-17T12:50:17.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.505+0000 7fe8617fa640 1 -- 172.21.15.18:0/3938465944 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854004dd0 con 0x7fe864073550 2024-02-17T12:50:17.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.505+0000 7fe86959f640 1 -- 172.21.15.18:0/3938465944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8641a65c0 con 0x7fe864073550 2024-02-17T12:50:17.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe8617fa640 1 -- 172.21.15.18:0/3938465944 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fe854010670 con 0x7fe864073550 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe86d3cc740 1 Processor -- start 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe86d3cc740 1 -- start start 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe86d3cc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318550 0x556b82318930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe86d3cc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318f00 0x556b823214b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe86d3cc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82321a80 0x556b82323ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe86d3cc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b82286f90 con 0x556b82318550 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe86d3cc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b822844c0 con 0x556b82318f00 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe86d3cc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8227fca0 con 0x556b82321a80 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe868d9e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318550 0x556b82318930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe868d9e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318550 0x556b82318930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52420/0 (socket says 172.21.15.18:52420) 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe868d9e640 1 -- 172.21.15.18:0/1056409051 learned_addr learned my addr 172.21.15.18:0/1056409051 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.508+0000 7fe863fff640 1 --2- 172.21.15.18:0/1056409051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318f00 0x556b823214b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe8637fe640 1 --2- 172.21.15.18:0/1056409051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82321a80 0x556b82323ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe868d9e640 1 -- 172.21.15.18:0/1056409051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82321a80 msgr2=0x556b82323ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe868d9e640 1 --2- 172.21.15.18:0/1056409051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82321a80 0x556b82323ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe868d9e640 1 -- 172.21.15.18:0/1056409051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318f00 msgr2=0x556b823214b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe868d9e640 1 --2- 172.21.15.18:0/1056409051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318f00 0x556b823214b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe868d9e640 1 -- 172.21.15.18:0/1056409051 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b822420b0 con 0x556b82318550 2024-02-17T12:50:17.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe8617fa640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe84c0a2350 0x7fe84c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe8637fe640 1 --2- 172.21.15.18:0/1056409051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82321a80 0x556b82323ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:50:17.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe863fff640 1 --2- 172.21.15.18:0/1056409051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318f00 0x556b823214b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:17.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe868d9e640 1 --2- 172.21.15.18:0/1056409051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318550 0x556b82318930 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe85c011780 tx=0x7fe85c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cb6bf9a50e9d19 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:17.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe8617fa640 1 -- 172.21.15.18:0/3938465944 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe8540cde10 con 0x7fe864073550 2024-02-17T12:50:17.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe863fff640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe84c0a2350 0x7fe84c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.509+0000 7fe84a7fc640 1 -- 172.21.15.18:0/1056409051 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00fae0 con 0x556b82318550 2024-02-17T12:50:17.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.510+0000 7fe84a7fc640 1 -- 172.21.15.18:0/1056409051 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe85c011e90 con 0x556b82318550 2024-02-17T12:50:17.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.510+0000 7fe84a7fc640 1 -- 172.21.15.18:0/1056409051 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c01a650 con 0x556b82318550 2024-02-17T12:50:17.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.510+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1056409051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318550 msgr2=0x556b82318930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.510+0000 7fe863fff640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe84c0a2350 0x7fe84c0a4810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe8540cb490 tx=0x7fe8540ba040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:17.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.510+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/1056409051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318550 0x556b82318930 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe85c011780 tx=0x7fe85c011c40 comp rx=0 tx=0).stop 2024-02-17T12:50:17.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.510+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1056409051 shutdown_connections 2024-02-17T12:50:17.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.510+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/1056409051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82321a80 0x556b82323ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.510+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/1056409051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318f00 0x556b823214b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.510+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/1056409051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318550 0x556b82318930 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.510+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1056409051 >> 172.21.15.18:0/1056409051 conn(0x556b82312a10 msgr2=0x556b82312df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.510+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1056409051 shutdown_connections 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.511+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1056409051 wait complete. 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.511+0000 7fe86d3cc740 1 -- start start 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.511+0000 7fe86d3cc740 1 Processor -- start 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.511+0000 7fe86d3cc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b8231e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.511+0000 7fe86d3cc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 0x556b8231ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe86d3cc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82321a80 0x556b8231f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe86d3cc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b82224f40 con 0x556b82318f00 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe86d3cc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b81f534b0 con 0x556b82318550 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe86d3cc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b82205670 con 0x556b82321a80 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe86d3cc740 10 client.0 _ll_register_callbacks cb 0x556b82272f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe8637fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 0x556b8231ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe8637fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 0x556b8231ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52430/0 (socket says 172.21.15.18:52430) 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe8637fe640 1 -- 172.21.15.18:0/1034825047 learned_addr learned my addr 172.21.15.18:0/1034825047 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe868d9e640 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b8231e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe868d9e640 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82321a80 0x556b8231f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe8637fe640 1 -- 172.21.15.18:0/1034825047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82321a80 msgr2=0x556b8231f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe8637fe640 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82321a80 0x556b8231f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe8637fe640 1 -- 172.21.15.18:0/1034825047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 msgr2=0x556b8231e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe8637fe640 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b8231e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe8637fe640 1 -- 172.21.15.18:0/1034825047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c0113e0 con 0x556b82318f00 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe868d9e640 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82321a80 0x556b8231f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe8637fe640 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 0x556b8231ef30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe858009900 tx=0x7fe85802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bae3a6d28501ca server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.512+0000 7fe8437fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe8437fe640 1 -- 172.21.15.18:0/1034825047 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858004040 con 0x556b82318f00 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe8437fe640 1 -- 172.21.15.18:0/1034825047 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe8580041a0 con 0x556b82318f00 2024-02-17T12:50:17.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1034825047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b82259f40 con 0x556b82318f00 2024-02-17T12:50:17.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe8437fe640 1 -- 172.21.15.18:0/1034825047 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe858004360 con 0x556b82318f00 2024-02-17T12:50:17.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe86d3cc740 10 client.15756 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:17.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1034825047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556b8227e320 con 0x556b82318f00 2024-02-17T12:50:17.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe8437fe640 1 -- 172.21.15.18:0/1034825047 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556b82259f40 con 0x556b82318f00 2024-02-17T12:50:17.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe8437fe640 1 -- 172.21.15.18:0/1034825047 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe85803c040 con 0x556b82318f00 2024-02-17T12:50:17.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe86d3cc740 10 client.15756 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:17.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe86d3cc740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:17.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe86d3cc740 20 client.15756 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:17.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556b82331f00 0x556b82334320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe86d3cc740 4 client.15756 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:17.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.513+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1034825047 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b82276310 con 0x556b82331f00 2024-02-17T12:50:17.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.514+0000 7fe868d9e640 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556b82331f00 0x556b82334320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.514+0000 7fe8437fe640 10 client.15756 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:17.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.514+0000 7fe868d9e640 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556b82331f00 0x556b82334320 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a860dcb03d0a39a server_cookie=99320720621e1994 in_seq=0 out_seq=0 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.515+0000 7fe8437fe640 1 -- 172.21.15.18:0/1034825047 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b82276310 con 0x556b82331f00 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.515+0000 7fe8437fe640 10 client.15756 handle_command_reply: tid=0 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.515+0000 7fe86d3cc740 1 client.15756 shutdown 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.515+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1034825047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 msgr2=0x556b8231ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.515+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 0x556b8231ef30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe858009900 tx=0x7fe85802c7e0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.515+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1034825047 shutdown_connections 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.515+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556b82331f00 0x556b82334320 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.515+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82321a80 0x556b8231f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.515+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 0x556b8231ef30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.515+0000 7fe8437fe640 0 client.15756 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.515+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/1034825047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b8231e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.515+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1034825047 >> 172.21.15.18:0/1034825047 conn(0x556b82312a10 msgr2=0x556b82311c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.516+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1034825047 shutdown_connections 2024-02-17T12:50:17.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.516+0000 7fe86d3cc740 1 -- 172.21.15.18:0/1034825047 wait complete. 2024-02-17T12:50:17.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.516+0000 7fe86d3cc740 20 client.15756 trim_cache size 0 max 16384 2024-02-17T12:50:17.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.531+0000 7fe86d3cc740 1 Processor -- start 2024-02-17T12:50:17.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.531+0000 7fe86d3cc740 1 -- start start 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.531+0000 7fe86d3cc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82312a10 0x556b8231f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.531+0000 7fe86d3cc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b82311b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.531+0000 7fe86d3cc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 0x556b82328b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.531+0000 7fe86d3cc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8232eb50 con 0x556b82318f00 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.531+0000 7fe86d3cc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8231a5e0 con 0x556b82318550 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.531+0000 7fe86d3cc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b82314f30 con 0x556b82312a10 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.531+0000 7fe868d9e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 0x556b82328b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.531+0000 7fe868d9e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 0x556b82328b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52434/0 (socket says 172.21.15.18:52434) 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.531+0000 7fe868d9e640 1 -- 172.21.15.18:0/2643673100 learned_addr learned my addr 172.21.15.18:0/2643673100 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.532+0000 7fe868d9e640 1 --2- 172.21.15.18:0/2643673100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82312a10 0x556b8231f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.532+0000 7fe8637fe640 1 --2- 172.21.15.18:0/2643673100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b82311b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.532+0000 7fe868d9e640 1 -- 172.21.15.18:0/2643673100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82312a10 msgr2=0x556b8231f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.532+0000 7fe868d9e640 1 --2- 172.21.15.18:0/2643673100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82312a10 0x556b8231f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.532+0000 7fe868d9e640 1 -- 172.21.15.18:0/2643673100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 msgr2=0x556b82311b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.532+0000 7fe868d9e640 1 --2- 172.21.15.18:0/2643673100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b82311b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.532+0000 7fe868d9e640 1 -- 172.21.15.18:0/2643673100 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b8221ac00 con 0x556b82318f00 2024-02-17T12:50:17.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.532+0000 7fe8637fe640 1 --2- 172.21.15.18:0/2643673100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b82311b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:17.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.532+0000 7fe868d9e640 1 --2- 172.21.15.18:0/2643673100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 0x556b82328b00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe85c004a10 tx=0x7fe85c012d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820aa451092d0ef server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:17.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.532+0000 7fe849ffb640 1 -- 172.21.15.18:0/2643673100 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00feb0 con 0x556b82318f00 2024-02-17T12:50:17.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.532+0000 7fe849ffb640 1 -- 172.21.15.18:0/2643673100 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe85c012450 con 0x556b82318f00 2024-02-17T12:50:17.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.532+0000 7fe849ffb640 1 -- 172.21.15.18:0/2643673100 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c0052a0 con 0x556b82318f00 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.533+0000 7fe86d3cc740 1 -- 172.21.15.18:0/2643673100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 msgr2=0x556b82328b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.533+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/2643673100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 0x556b82328b00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe85c004a10 tx=0x7fe85c012d60 comp rx=0 tx=0).stop 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.533+0000 7fe86d3cc740 1 -- 172.21.15.18:0/2643673100 shutdown_connections 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.533+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/2643673100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82318f00 0x556b82328b00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.533+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/2643673100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b82311b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.533+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/2643673100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82312a10 0x556b8231f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.533+0000 7fe86d3cc740 1 -- 172.21.15.18:0/2643673100 >> 172.21.15.18:0/2643673100 conn(0x556b82319650 msgr2=0x556b82311400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.533+0000 7fe86d3cc740 1 -- 172.21.15.18:0/2643673100 shutdown_connections 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.533+0000 7fe86d3cc740 1 -- 172.21.15.18:0/2643673100 wait complete. 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.534+0000 7fe86d3cc740 1 -- start start 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.534+0000 7fe86d3cc740 1 Processor -- start 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.534+0000 7fe86d3cc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82312a10 0x556b82382c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.534+0000 7fe86d3cc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b82383180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.534+0000 7fe86d3cc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82318f00 0x556b82328240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.534+0000 7fe86d3cc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b82328800 con 0x556b82312a10 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.534+0000 7fe868d9e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82312a10 0x556b82382c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe868d9e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82312a10 0x556b82382c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52436/0 (socket says 172.21.15.18:52436) 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe868d9e640 1 -- 172.21.15.18:0/3085028700 learned_addr learned my addr 172.21.15.18:0/3085028700 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe86d3cc740 1 -- 172.21.15.18:0/3085028700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b82328970 con 0x556b82318550 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe86d3cc740 1 -- 172.21.15.18:0/3085028700 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b82328b30 con 0x556b82318f00 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe86d3cc740 10 client.0 _ll_register_callbacks cb 0x7fe85c009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe8637fe640 1 --2- 172.21.15.18:0/3085028700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b82383180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe868d9e640 1 -- 172.21.15.18:0/3085028700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82318f00 msgr2=0x556b82328240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe868d9e640 1 --2- 172.21.15.18:0/3085028700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82318f00 0x556b82328240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe868d9e640 1 -- 172.21.15.18:0/3085028700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 msgr2=0x556b82383180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe868d9e640 1 --2- 172.21.15.18:0/3085028700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b82383180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe868d9e640 1 -- 172.21.15.18:0/3085028700 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b822420b0 con 0x556b82312a10 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe868d9e640 1 --2- 172.21.15.18:0/3085028700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82312a10 0x556b82382c40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe85c01c040 tx=0x7fe85c00feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5155fe7a844ec89 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe8437fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe8437fe640 1 -- 172.21.15.18:0/3085028700 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c0125c0 con 0x556b82312a10 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe8437fe640 1 -- 172.21.15.18:0/3085028700 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe85c012720 con 0x556b82312a10 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.535+0000 7fe86d3cc740 1 -- 172.21.15.18:0/3085028700 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b822a5900 con 0x556b82312a10 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.536+0000 7fe8437fe640 1 -- 172.21.15.18:0/3085028700 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe85c0128e0 con 0x556b82312a10 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.536+0000 7fe86d3cc740 10 client.15762 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.536+0000 7fe86d3cc740 1 -- 172.21.15.18:0/3085028700 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556b82240910 con 0x556b82312a10 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.536+0000 7fe8437fe640 1 -- 172.21.15.18:0/3085028700 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556b822a5900 con 0x556b82312a10 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.536+0000 7fe8437fe640 1 -- 172.21.15.18:0/3085028700 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe85c00c070 con 0x556b82312a10 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.536+0000 7fe86d3cc740 10 client.15762 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.536+0000 7fe86d3cc740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.536+0000 7fe86d3cc740 20 client.15762 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:17.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.536+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/3085028700 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556b82321a80 0x556b8238c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.536+0000 7fe86d3cc740 4 client.15762 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:50:17.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.536+0000 7fe86d3cc740 1 -- 172.21.15.18:0/3085028700 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556b81f7f860 con 0x556b82321a80 2024-02-17T12:50:17.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.536+0000 7fe8637fe640 1 --2- 172.21.15.18:0/3085028700 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556b82321a80 0x556b8238c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.537+0000 7fe8437fe640 10 client.15762 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:17.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.537+0000 7fe8637fe640 1 --2- 172.21.15.18:0/3085028700 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556b82321a80 0x556b8238c790 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15e7a6815f2abe0e server_cookie=86b19740cfbe0fd9 in_seq=0 out_seq=0 2024-02-17T12:50:17.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.538+0000 7fe8437fe640 1 -- 172.21.15.18:0/3085028700 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556b81f7f860 con 0x556b82321a80 2024-02-17T12:50:17.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.538+0000 7fe8437fe640 10 client.15762 handle_command_reply: tid=0 2024-02-17T12:50:17.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.538+0000 7fe86d3cc740 1 client.15762 shutdown 2024-02-17T12:50:17.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.538+0000 7fe86d3cc740 1 -- 172.21.15.18:0/3085028700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82312a10 msgr2=0x556b82382c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.538+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/3085028700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82312a10 0x556b82382c40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe85c01c040 tx=0x7fe85c00feb0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86d3cc740 1 -- 172.21.15.18:0/3085028700 shutdown_connections 2024-02-17T12:50:17.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/3085028700 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556b82321a80 0x556b8238c790 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/3085028700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556b82318f00 0x556b82328240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/3085028700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b82318550 0x556b82383180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86d3cc740 1 --2- 172.21.15.18:0/3085028700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b82312a10 0x556b82382c40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86d3cc740 1 -- 172.21.15.18:0/3085028700 >> 172.21.15.18:0/3085028700 conn(0x556b82319650 msgr2=0x556b82311c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:17.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe8437fe640 0 client.15762 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:17.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86d3cc740 1 -- 172.21.15.18:0/3085028700 shutdown_connections 2024-02-17T12:50:17.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86d3cc740 1 -- 172.21.15.18:0/3085028700 wait complete. 2024-02-17T12:50:17.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86d3cc740 20 client.15762 trim_cache size 0 max 16384 2024-02-17T12:50:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:50:17.535 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:50:17.535 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "a4216602-a3a9-4500-a1fb-20f2e3aa2c0d", 2024-02-17T12:50:17.535 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:50:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:50:17.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86959f640 1 -- 172.21.15.18:0/3938465944 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe84c0a2350 msgr2=0x7fe84c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86959f640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe84c0a2350 0x7fe84c0a4810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe8540cb490 tx=0x7fe8540ba040 comp rx=0 tx=0).stop 2024-02-17T12:50:17.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86959f640 1 -- 172.21.15.18:0/3938465944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe864073550 msgr2=0x7fe86419f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.539+0000 7fe86959f640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe864073550 0x7fe86419f640 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe85400d6d0 tx=0x7fe85400db90 comp rx=0 tx=0).stop 2024-02-17T12:50:17.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.540+0000 7fe86959f640 1 -- 172.21.15.18:0/3938465944 shutdown_connections 2024-02-17T12:50:17.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.540+0000 7fe86959f640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe84c0a2350 0x7fe84c0a4810 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.540+0000 7fe86959f640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe86410d890 0x7fe864199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.540+0000 7fe86959f640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe864073e70 0x7fe86419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.540+0000 7fe86959f640 1 --2- 172.21.15.18:0/3938465944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe864073550 0x7fe86419f640 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.540+0000 7fe86959f640 1 -- 172.21.15.18:0/3938465944 >> 172.21.15.18:0/3938465944 conn(0x7fe8640fcb20 msgr2=0x7fe8640fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:17.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.540+0000 7fe86959f640 1 -- 172.21.15.18:0/3938465944 shutdown_connections 2024-02-17T12:50:17.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.540+0000 7fe86959f640 1 -- 172.21.15.18:0/3938465944 wait complete. 2024-02-17T12:50:17.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:50:17.543 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:50:17.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.657+0000 7f780aad4640 1 -- 172.21.15.18:0/4183620743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7804105340 msgr2=0x7f780410cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.657+0000 7f780aad4640 1 --2- 172.21.15.18:0/4183620743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7804105340 0x7f780410cf10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009cd0 tx=0x7f77fc02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 -- 172.21.15.18:0/4183620743 shutdown_connections 2024-02-17T12:50:17.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 --2- 172.21.15.18:0/4183620743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7804105340 0x7f780410cf10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 --2- 172.21.15.18:0/4183620743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7804100c20 0x7f7804104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 --2- 172.21.15.18:0/4183620743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7804100270 0x7f7804100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 -- 172.21.15.18:0/4183620743 >> 172.21.15.18:0/4183620743 conn(0x7f7804077230 msgr2=0x7f7804077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:17.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 -- 172.21.15.18:0/4183620743 shutdown_connections 2024-02-17T12:50:17.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 -- 172.21.15.18:0/4183620743 wait complete. 2024-02-17T12:50:17.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 Processor -- start 2024-02-17T12:50:17.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 -- start start 2024-02-17T12:50:17.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7804100270 0x7f7804198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7804105340 0x7f7804198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7804199060 0x7f78041a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78041a3cb0 con 0x7f7804199060 2024-02-17T12:50:17.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78041a3e20 con 0x7f7804105340 2024-02-17T12:50:17.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.658+0000 7f780aad4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78041a3fe0 con 0x7f7804100270 2024-02-17T12:50:17.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.659+0000 7f7809071640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7804199060 0x7f78041a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.659+0000 7f7809071640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7804199060 0x7f78041a3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52454/0 (socket says 172.21.15.18:52454) 2024-02-17T12:50:17.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.659+0000 7f7809071640 1 -- 172.21.15.18:0/262550222 learned_addr learned my addr 172.21.15.18:0/262550222 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:17.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.659+0000 7f7808870640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7804100270 0x7f7804198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.659+0000 7f7803fff640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7804105340 0x7f7804198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.659+0000 7f7809071640 1 -- 172.21.15.18:0/262550222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7804100270 msgr2=0x7f7804198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.659+0000 7f7809071640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7804100270 0x7f7804198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.659+0000 7f7809071640 1 -- 172.21.15.18:0/262550222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7804105340 msgr2=0x7f7804198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.659+0000 7f7809071640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7804105340 0x7f7804198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.659+0000 7f7809071640 1 -- 172.21.15.18:0/262550222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc009900 con 0x7f7804199060 2024-02-17T12:50:17.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.659+0000 7f7803fff640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7804105340 0x7f7804198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:17.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.659+0000 7f7809071640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7804199060 0x7f78041a3610 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f77fc004af0 tx=0x7f77fc034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:17.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.660+0000 7f7801ffb640 1 -- 172.21.15.18:0/262550222 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0328c0 con 0x7f7804199060 2024-02-17T12:50:17.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.660+0000 7f780aad4640 1 -- 172.21.15.18:0/262550222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78041a4260 con 0x7f7804199060 2024-02-17T12:50:17.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.660+0000 7f7801ffb640 1 -- 172.21.15.18:0/262550222 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77fc003e50 con 0x7f7804199060 2024-02-17T12:50:17.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.660+0000 7f7801ffb640 1 -- 172.21.15.18:0/262550222 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc004860 con 0x7f7804199060 2024-02-17T12:50:17.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.660+0000 7f780aad4640 1 -- 172.21.15.18:0/262550222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78041a4760 con 0x7f7804199060 2024-02-17T12:50:17.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.663+0000 7f7801ffb640 1 -- 172.21.15.18:0/262550222 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f77fc00a040 con 0x7f7804199060 2024-02-17T12:50:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.663+0000 7f780d780740 1 Processor -- start 2024-02-17T12:50:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.663+0000 7f780d780740 1 -- start start 2024-02-17T12:50:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.663+0000 7f780d780740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036dee9e0 0x565036deedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.663+0000 7f780d780740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036ddb440 0x565036ddb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.663+0000 7f780d780740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036ddbe00 0x565036df57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.663+0000 7f780d780740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x565036d50280 con 0x565036dee9e0 2024-02-17T12:50:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.663+0000 7f780d780740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565036d4d7b0 con 0x565036ddb440 2024-02-17T12:50:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.663+0000 7f780d780740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565036d48f90 con 0x565036ddbe00 2024-02-17T12:50:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7803fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036dee9e0 0x565036deedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7803fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036dee9e0 0x565036deedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52464/0 (socket says 172.21.15.18:52464) 2024-02-17T12:50:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7803fff640 1 -- 172.21.15.18:0/1817135277 learned_addr learned my addr 172.21.15.18:0/1817135277 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7808870640 1 --2- 172.21.15.18:0/1817135277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036ddb440 0x565036ddb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7801ffb640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f77f40a23d0 0x7f77f40a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7803fff640 1 -- 172.21.15.18:0/1817135277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036ddbe00 msgr2=0x565036df57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7803fff640 1 --2- 172.21.15.18:0/1817135277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036ddbe00 0x565036df57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7803fff640 1 -- 172.21.15.18:0/1817135277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036ddb440 msgr2=0x565036ddb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7803fff640 1 --2- 172.21.15.18:0/1817135277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036ddb440 0x565036ddb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7803fff640 1 -- 172.21.15.18:0/1817135277 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565036d07570 con 0x565036dee9e0 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7809071640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f77f40a23d0 0x7f77f40a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7801ffb640 1 -- 172.21.15.18:0/262550222 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f77fc0f0d80 con 0x7f7804199060 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.664+0000 7f7803fff640 1 --2- 172.21.15.18:0/1817135277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036dee9e0 0x565036deedc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009f80 tx=0x7f77f00139f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bea1144b881665c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.665+0000 7f7809071640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f77f40a23d0 0x7f77f40a4890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0ee510 tx=0x7f77fc02c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.665+0000 7f77eeffd640 1 -- 172.21.15.18:0/1817135277 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0004050 con 0x565036dee9e0 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.665+0000 7f77eeffd640 1 -- 172.21.15.18:0/1817135277 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77f00041b0 con 0x565036dee9e0 2024-02-17T12:50:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.665+0000 7f77eeffd640 1 -- 172.21.15.18:0/1817135277 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f000f940 con 0x565036dee9e0 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.666+0000 7f780d780740 1 -- 172.21.15.18:0/1817135277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036dee9e0 msgr2=0x565036deedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.666+0000 7f780d780740 1 --2- 172.21.15.18:0/1817135277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036dee9e0 0x565036deedc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009f80 tx=0x7f77f00139f0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.666+0000 7f780d780740 1 -- 172.21.15.18:0/1817135277 shutdown_connections 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.666+0000 7f780d780740 1 --2- 172.21.15.18:0/1817135277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036ddbe00 0x565036df57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.666+0000 7f780d780740 1 --2- 172.21.15.18:0/1817135277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036ddb440 0x565036ddb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.666+0000 7f780d780740 1 --2- 172.21.15.18:0/1817135277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036dee9e0 0x565036deedc0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.666+0000 7f780d780740 1 -- 172.21.15.18:0/1817135277 >> 172.21.15.18:0/1817135277 conn(0x565036dddfb0 msgr2=0x565036dde390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.666+0000 7f780d780740 1 -- 172.21.15.18:0/1817135277 shutdown_connections 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.666+0000 7f780d780740 1 -- 172.21.15.18:0/1817135277 wait complete. 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.666+0000 7f780d780740 1 -- start start 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.666+0000 7f780d780740 1 Processor -- start 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.667+0000 7f780d780740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddb440 0x565036df2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.667+0000 7f780d780740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036ddbe00 0x565036df3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.667+0000 7f780d780740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036df3570 0x565036dfefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.667+0000 7f780d780740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x565036a1c430 con 0x565036ddb440 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.667+0000 7f780d780740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650369d8e00 con 0x565036ddbe00 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.667+0000 7f780d780740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565036ccdfd0 con 0x565036df3570 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.667+0000 7f780d780740 10 client.0 _ll_register_callbacks cb 0x565036d23fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.667+0000 7f7803fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddb440 0x565036df2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.667+0000 7f7803fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddb440 0x565036df2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52468/0 (socket says 172.21.15.18:52468) 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.667+0000 7f7803fff640 1 -- 172.21.15.18:0/2003999897 learned_addr learned my addr 172.21.15.18:0/2003999897 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.667+0000 7f7808870640 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036ddbe00 0x565036df3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.668+0000 7f7803fff640 1 -- 172.21.15.18:0/2003999897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036df3570 msgr2=0x565036dfefb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.668+0000 7f7803fff640 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036df3570 0x565036dfefb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.668+0000 7f7803fff640 1 -- 172.21.15.18:0/2003999897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036ddbe00 msgr2=0x565036df3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.668+0000 7f7803fff640 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036ddbe00 0x565036df3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.668+0000 7f7803fff640 1 -- 172.21.15.18:0/2003999897 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f0009d60 con 0x565036ddb440 2024-02-17T12:50:17.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.668+0000 7f7808870640 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036ddbe00 0x565036df3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.668+0000 7f7803fff640 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddb440 0x565036df2af0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009fd0 tx=0x7f77f0012050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e78223c085191d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.668+0000 7f77e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.668+0000 7f77e3fff640 1 -- 172.21.15.18:0/2003999897 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0022070 con 0x565036ddb440 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.668+0000 7f77e3fff640 1 -- 172.21.15.18:0/2003999897 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f77f00035f0 con 0x565036ddb440 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.668+0000 7f780d780740 1 -- 172.21.15.18:0/2003999897 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565036a487e0 con 0x565036ddb440 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.669+0000 7f77e3fff640 1 -- 172.21.15.18:0/2003999897 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77f0003d60 con 0x565036ddb440 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.669+0000 7f780d780740 10 client.15774 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.669+0000 7f780d780740 1 -- 172.21.15.18:0/2003999897 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x565036d47610 con 0x565036ddb440 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.669+0000 7f77e3fff640 1 -- 172.21.15.18:0/2003999897 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x565036a487e0 con 0x565036ddb440 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.669+0000 7f77e3fff640 1 -- 172.21.15.18:0/2003999897 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77f001c450 con 0x565036ddb440 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.669+0000 7f780d780740 10 client.15774 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.669+0000 7f780d780740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.669+0000 7f780d780740 20 client.15774 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.669+0000 7f780d780740 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x565036e03610 0x565036e05a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.669+0000 7f780d780740 4 client.15774 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.669+0000 7f780d780740 1 -- 172.21.15.18:0/2003999897 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565036a85720 con 0x565036e03610 2024-02-17T12:50:17.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.670+0000 7f7808870640 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x565036e03610 0x565036e05a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.670+0000 7f7808870640 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x565036e03610 0x565036e05a30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9095f7f8093ff58a server_cookie=458d61de73f473c2 in_seq=0 out_seq=0 2024-02-17T12:50:17.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.670+0000 7f77e3fff640 10 client.15774 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:17.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f77e3fff640 1 -- 172.21.15.18:0/2003999897 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565036a85720 con 0x565036e03610 2024-02-17T12:50:17.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f77e3fff640 10 client.15774 handle_command_reply: tid=0 2024-02-17T12:50:17.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f780d780740 1 client.15774 shutdown 2024-02-17T12:50:17.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f780d780740 1 -- 172.21.15.18:0/2003999897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddb440 msgr2=0x565036df2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f780d780740 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddb440 0x565036df2af0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009fd0 tx=0x7f77f0012050 comp rx=0 tx=0).stop 2024-02-17T12:50:17.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f780d780740 1 -- 172.21.15.18:0/2003999897 shutdown_connections 2024-02-17T12:50:17.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f780d780740 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x565036e03610 0x565036e05a30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f780d780740 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036df3570 0x565036dfefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f780d780740 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036ddbe00 0x565036df3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f77e3fff640 0 client.15774 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:17.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f780d780740 1 --2- 172.21.15.18:0/2003999897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddb440 0x565036df2af0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f780d780740 1 -- 172.21.15.18:0/2003999897 >> 172.21.15.18:0/2003999897 conn(0x565036dddfb0 msgr2=0x565036ddd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:17.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f780d780740 1 -- 172.21.15.18:0/2003999897 shutdown_connections 2024-02-17T12:50:17.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f780d780740 1 -- 172.21.15.18:0/2003999897 wait complete. 2024-02-17T12:50:17.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.671+0000 7f780d780740 20 client.15774 trim_cache size 0 max 16384 2024-02-17T12:50:17.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f780d780740 1 Processor -- start 2024-02-17T12:50:17.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f780d780740 1 -- start start 2024-02-17T12:50:17.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f780d780740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddbe00 0x565036df06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f780d780740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036df3570 0x565036df23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f780d780740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036df5330 0x565036df9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f780d780740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x565036df28f0 con 0x565036ddbe00 2024-02-17T12:50:17.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f780d780740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565036e0ac20 con 0x565036df5330 2024-02-17T12:50:17.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f780d780740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565036ddc960 con 0x565036df3570 2024-02-17T12:50:17.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f7803fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddbe00 0x565036df06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f7803fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddbe00 0x565036df06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52478/0 (socket says 172.21.15.18:52478) 2024-02-17T12:50:17.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f7803fff640 1 -- 172.21.15.18:0/1226834424 learned_addr learned my addr 172.21.15.18:0/1226834424 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:17.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f7808870640 1 --2- 172.21.15.18:0/1226834424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036df3570 0x565036df23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.687+0000 7f7803fff640 1 --2- 172.21.15.18:0/1226834424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036df5330 0x565036df9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.688+0000 7f7803fff640 1 -- 172.21.15.18:0/1226834424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036df3570 msgr2=0x565036df23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.688+0000 7f7803fff640 1 --2- 172.21.15.18:0/1226834424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036df3570 0x565036df23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.688+0000 7f7803fff640 1 -- 172.21.15.18:0/1226834424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036df5330 msgr2=0x565036df9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.688+0000 7f7803fff640 1 --2- 172.21.15.18:0/1226834424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036df5330 0x565036df9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.688+0000 7f7803fff640 1 -- 172.21.15.18:0/1226834424 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565036ce3ac0 con 0x565036ddbe00 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.688+0000 7f7803fff640 1 --2- 172.21.15.18:0/1226834424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddbe00 0x565036df06e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f77f0013530 tx=0x7f77f0013460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8511a846803c1fc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.688+0000 7f77ee7fc640 1 -- 172.21.15.18:0/1226834424 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0022030 con 0x565036ddbe00 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.688+0000 7f77ee7fc640 1 -- 172.21.15.18:0/1226834424 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77f001c450 con 0x565036ddbe00 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.688+0000 7f77ee7fc640 1 -- 172.21.15.18:0/1226834424 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0003da0 con 0x565036ddbe00 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.689+0000 7f780d780740 1 -- 172.21.15.18:0/1226834424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddbe00 msgr2=0x565036df06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.689+0000 7f780d780740 1 --2- 172.21.15.18:0/1226834424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddbe00 0x565036df06e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f77f0013530 tx=0x7f77f0013460 comp rx=0 tx=0).stop 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.689+0000 7f780d780740 1 -- 172.21.15.18:0/1226834424 shutdown_connections 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.689+0000 7f780d780740 1 --2- 172.21.15.18:0/1226834424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036df5330 0x565036df9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.689+0000 7f780d780740 1 --2- 172.21.15.18:0/1226834424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036df3570 0x565036df23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.689+0000 7f780d780740 1 --2- 172.21.15.18:0/1226834424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036ddbe00 0x565036df06e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.689+0000 7f780d780740 1 -- 172.21.15.18:0/1226834424 >> 172.21.15.18:0/1226834424 conn(0x565036dddfb0 msgr2=0x565036dde390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:17.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.689+0000 7f780d780740 1 -- 172.21.15.18:0/1226834424 shutdown_connections 2024-02-17T12:50:17.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.689+0000 7f780d780740 1 -- 172.21.15.18:0/1226834424 wait complete. 2024-02-17T12:50:17.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.689+0000 7f780d780740 1 -- start start 2024-02-17T12:50:17.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.689+0000 7f780d780740 1 Processor -- start 2024-02-17T12:50:17.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f780d780740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036ddbe00 0x565036df8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f780d780740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036df3570 0x565036df86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f780d780740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036df5330 0x565036e52b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f780d780740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x565036e53090 con 0x565036df5330 2024-02-17T12:50:17.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f780d780740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565036e53200 con 0x565036df3570 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f780d780740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565036e533c0 con 0x565036ddbe00 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f780d780740 10 client.0 _ll_register_callbacks cb 0x7f77f0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f7803fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036df5330 0x565036e52b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f7803fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036df5330 0x565036e52b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52494/0 (socket says 172.21.15.18:52494) 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f7803fff640 1 -- 172.21.15.18:0/4063894144 learned_addr learned my addr 172.21.15.18:0/4063894144 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f7803fff640 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036ddbe00 0x565036df8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f7808870640 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036df3570 0x565036df86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f7803fff640 1 -- 172.21.15.18:0/4063894144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036ddbe00 msgr2=0x565036df8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f7803fff640 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036ddbe00 0x565036df8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f7803fff640 1 -- 172.21.15.18:0/4063894144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036df3570 msgr2=0x565036df86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f7803fff640 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036df3570 0x565036df86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f7803fff640 1 -- 172.21.15.18:0/4063894144 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565036d47610 con 0x565036df5330 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f7803fff640 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036df5330 0x565036e52b50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f77f001ccb0 tx=0x7f77f0003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce229b3d4b9f8f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.690+0000 7f77e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f77e3fff640 1 -- 172.21.15.18:0/4063894144 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0003a00 con 0x565036df5330 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f77e3fff640 1 -- 172.21.15.18:0/4063894144 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f77f0023070 con 0x565036df5330 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f780d780740 1 -- 172.21.15.18:0/4063894144 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77f00041b0 con 0x565036df5330 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f77e3fff640 1 -- 172.21.15.18:0/4063894144 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77f001c5c0 con 0x565036df5330 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f780d780740 10 client.15780 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f780d780740 1 -- 172.21.15.18:0/4063894144 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x565036d096a0 con 0x565036df5330 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f77e3fff640 1 -- 172.21.15.18:0/4063894144 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77f004b020 con 0x565036df5330 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f77e3fff640 1 -- 172.21.15.18:0/4063894144 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77f0036020 con 0x565036df5330 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f780d780740 10 client.15780 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f780d780740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f780d780740 20 client.15780 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f780d780740 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x565036e5a6b0 0x565036e5caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.691+0000 7f780d780740 4 client.15780 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:50:17.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.692+0000 7f780d780740 1 -- 172.21.15.18:0/4063894144 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f77f00041b0 con 0x565036e5a6b0 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.692+0000 7f7808870640 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x565036e5a6b0 0x565036e5caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.692+0000 7f77e3fff640 10 client.15780 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.692+0000 7f7808870640 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x565036e5a6b0 0x565036e5caa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbe373dd740d4891 server_cookie=8ea424beba42786c in_seq=0 out_seq=0 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.692+0000 7f77e3fff640 1 -- 172.21.15.18:0/4063894144 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f77f00041b0 con 0x565036e5a6b0 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.692+0000 7f77e3fff640 10 client.15780 handle_command_reply: tid=0 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.692+0000 7f780d780740 1 client.15780 shutdown 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.692+0000 7f780d780740 1 -- 172.21.15.18:0/4063894144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036df5330 msgr2=0x565036e52b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.692+0000 7f780d780740 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036df5330 0x565036e52b50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f77f001ccb0 tx=0x7f77f0003be0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.693+0000 7f780d780740 1 -- 172.21.15.18:0/4063894144 shutdown_connections 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.693+0000 7f780d780740 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x565036e5a6b0 0x565036e5caa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.693+0000 7f780d780740 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565036df5330 0x565036e52b50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.693+0000 7f780d780740 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565036df3570 0x565036df86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.693+0000 7f780d780740 1 --2- 172.21.15.18:0/4063894144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565036ddbe00 0x565036df8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.693+0000 7f780d780740 1 -- 172.21.15.18:0/4063894144 >> 172.21.15.18:0/4063894144 conn(0x565036dddfb0 msgr2=0x565036decc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:17.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.693+0000 7f77e3fff640 0 client.15780 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:17.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.693+0000 7f780d780740 1 -- 172.21.15.18:0/4063894144 shutdown_connections 2024-02-17T12:50:17.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.693+0000 7f780d780740 1 -- 172.21.15.18:0/4063894144 wait complete. 2024-02-17T12:50:17.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.693+0000 7f780d780740 20 client.15780 trim_cache size 0 max 16384 2024-02-17T12:50:17.694 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:50:17.694 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:50:17.694 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:50:17.694 INFO:teuthology.orchestra.run.smithi018.stdout: "a4216602-a3a9-4500-a1fb-20f2e3aa2c0d": { 2024-02-17T12:50:17.694 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:50:17.694 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "a4216602-a3a9-4500-a1fb-20f2e3aa2c0d", 2024-02-17T12:50:17.694 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:50:17.695 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:50:17.695 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:50:17.695 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:50:17.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.694+0000 7f780aad4640 1 -- 172.21.15.18:0/262550222 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f77f40a23d0 msgr2=0x7f77f40a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.694+0000 7f780aad4640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f77f40a23d0 0x7f77f40a4890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0ee510 tx=0x7f77fc02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.694+0000 7f780aad4640 1 -- 172.21.15.18:0/262550222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7804199060 msgr2=0x7f78041a3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:17.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.694+0000 7f780aad4640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7804199060 0x7f78041a3610 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f77fc004af0 tx=0x7f77fc034cc0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.694+0000 7f780aad4640 1 -- 172.21.15.18:0/262550222 shutdown_connections 2024-02-17T12:50:17.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.694+0000 7f780aad4640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f77f40a23d0 0x7f77f40a4890 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.694+0000 7f780aad4640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7804199060 0x7f78041a3610 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.694+0000 7f780aad4640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7804105340 0x7f7804198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.694+0000 7f780aad4640 1 --2- 172.21.15.18:0/262550222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7804100270 0x7f7804198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:17.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.694+0000 7f780aad4640 1 -- 172.21.15.18:0/262550222 >> 172.21.15.18:0/262550222 conn(0x7f7804077230 msgr2=0x7f78040fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:17.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.694+0000 7f780aad4640 1 -- 172.21.15.18:0/262550222 shutdown_connections 2024-02-17T12:50:17.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:17.694+0000 7f780aad4640 1 -- 172.21.15.18:0/262550222 wait complete. 2024-02-17T12:50:17.697 INFO:tasks.cephfs.filesystem:scrub status for tag:a4216602-a3a9-4500-a1fb-20f2e3aa2c0d - {'path': '/', 'tag': 'a4216602-a3a9-4500-a1fb-20f2e3aa2c0d', 'options': 'recursive,force'} 2024-02-17T12:50:18.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:18 smithi018 ceph-mon[32625]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-02-17T12:50:18.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:18 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:50:18.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:18 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:50:18.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:18 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:50:18.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:18 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:50:18.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:18 smithi119 ceph-mon[36188]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-02-17T12:50:18.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:18 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:50:18.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:18 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:50:18.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:18 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:50:18.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:18 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:50:18.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:18 smithi146 ceph-mon[36454]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-02-17T12:50:18.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:18 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:50:18.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:18 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:50:18.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:18 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:50:18.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:18 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:50:20.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:20 smithi018 ceph-mon[32625]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-02-17T12:50:20.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:20 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:50:20.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:20 smithi119 ceph-mon[36188]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-02-17T12:50:20.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:20 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:50:20.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:20 smithi146 ceph-mon[36454]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-02-17T12:50:20.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:20 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:50:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.376+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1884170524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0080ff790 msgr2=0x7ff0080ffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.376+0000 7ff00eb5c640 1 --2- 172.21.15.18:0/1884170524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0080ff790 0x7ff0080ffc10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feff4009930 tx=0x7feff402bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.376+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1884170524 shutdown_connections 2024-02-17T12:50:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.376+0000 7ff00eb5c640 1 --2- 172.21.15.18:0/1884170524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff008100150 0x7ff00810c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.376+0000 7ff00eb5c640 1 --2- 172.21.15.18:0/1884170524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0080ff790 0x7ff0080ffc10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.376+0000 7ff00eb5c640 1 --2- 172.21.15.18:0/1884170524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0081057a0 0x7ff008105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.376+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1884170524 >> 172.21.15.18:0/1884170524 conn(0x7ff008076610 msgr2=0x7ff0080fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.377+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1884170524 shutdown_connections 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.377+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1884170524 wait complete. 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.377+0000 7ff00eb5c640 1 Processor -- start 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.377+0000 7ff00eb5c640 1 -- start start 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00eb5c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0080ff790 0x7ff008196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00eb5c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff008100150 0x7ff0081968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00eb5c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0081057a0 0x7ff00819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00eb5c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0081970e0 con 0x7ff0081057a0 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00eb5c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff008197250 con 0x7ff008100150 2024-02-17T12:50:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00eb5c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00819b9a0 con 0x7ff0080ff790 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00d0f9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0081057a0 0x7ff00819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00d0f9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0081057a0 0x7ff00819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57514/0 (socket says 172.21.15.18:57514) 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00d0f9640 1 -- 172.21.15.18:0/1220130061 learned_addr learned my addr 172.21.15.18:0/1220130061 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff007fff640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff008100150 0x7ff0081968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0080ff790 0x7ff008196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00d0f9640 1 -- 172.21.15.18:0/1220130061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0080ff790 msgr2=0x7ff008196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00d0f9640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0080ff790 0x7ff008196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00d0f9640 1 -- 172.21.15.18:0/1220130061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff008100150 msgr2=0x7ff0081968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00d0f9640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff008100150 0x7ff0081968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00d0f9640 1 -- 172.21.15.18:0/1220130061 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4009560 con 0x7ff0081057a0 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff007fff640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff008100150 0x7ff0081968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.378+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0080ff790 0x7ff008196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.379+0000 7ff00d0f9640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0081057a0 0x7ff00819b3e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff00000eaa0 tx=0x7ff00000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.379+0000 7ff005ffb640 1 -- 172.21.15.18:0/1220130061 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00000cdb0 con 0x7ff0081057a0 2024-02-17T12:50:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.379+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1220130061 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00819bc80 con 0x7ff0081057a0 2024-02-17T12:50:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.379+0000 7ff005ffb640 1 -- 172.21.15.18:0/1220130061 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff000005020 con 0x7ff0081057a0 2024-02-17T12:50:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.379+0000 7ff005ffb640 1 -- 172.21.15.18:0/1220130061 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0000116d0 con 0x7ff0081057a0 2024-02-17T12:50:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.379+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1220130061 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00819c160 con 0x7ff0081057a0 2024-02-17T12:50:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.382+0000 7ff011805740 1 Processor -- start 2024-02-17T12:50:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.382+0000 7ff005ffb640 1 -- 172.21.15.18:0/1220130061 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7ff000011830 con 0x7ff0081057a0 2024-02-17T12:50:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.382+0000 7ff011805740 1 -- start start 2024-02-17T12:50:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.382+0000 7ff011805740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595b9ad0 0x5634595b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.382+0000 7ff011805740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b8d60 0x5634595b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.382+0000 7ff011805740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 0x5634595c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.382+0000 7ff011805740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634595279f0 con 0x5634595b8d60 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.382+0000 7ff011805740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563459524f20 con 0x5634595b9ad0 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.382+0000 7ff011805740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563459520700 con 0x5634595c22e0 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff00c8f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b8d60 0x5634595b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff00c8f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b8d60 0x5634595b91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57520/0 (socket says 172.21.15.18:57520) 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff00c8f8640 1 -- 172.21.15.18:0/560779021 learned_addr learned my addr 172.21.15.18:0/560779021 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff007fff640 1 --2- 172.21.15.18:0/560779021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595b9ad0 0x5634595b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff007fff640 1 --2- 172.21.15.18:0/560779021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 0x5634595c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff005ffb640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7feff80a2470 0x7feff80a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff00c8f8640 1 -- 172.21.15.18:0/560779021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 msgr2=0x5634595c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/560779021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 0x5634595c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff00c8f8640 1 -- 172.21.15.18:0/560779021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595b9ad0 msgr2=0x5634595b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/560779021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595b9ad0 0x5634595b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff00c8f8640 1 -- 172.21.15.18:0/560779021 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634594b67b0 con 0x5634595b8d60 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff005ffb640 1 -- 172.21.15.18:0/1220130061 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff0000ce5a0 con 0x7ff0081057a0 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff00d0f9640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7feff80a2470 0x7feff80a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.383+0000 7ff007fff640 1 --2- 172.21.15.18:0/560779021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 0x5634595c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.384+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/560779021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b8d60 0x5634595b91e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7feffc009bc0 tx=0x7feffc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c5fb3bc2ccbc9f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.384+0000 7ff00d0f9640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7feff80a2470 0x7feff80a4930 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0000cbc20 tx=0x7ff000005800 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.384+0000 7feff2ffd640 1 -- 172.21.15.18:0/560779021 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc005020 con 0x5634595b8d60 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.384+0000 7feff2ffd640 1 -- 172.21.15.18:0/560779021 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feffc005660 con 0x5634595b8d60 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.384+0000 7feff2ffd640 1 -- 172.21.15.18:0/560779021 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc0116d0 con 0x5634595b8d60 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.385+0000 7ff011805740 1 -- 172.21.15.18:0/560779021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b8d60 msgr2=0x5634595b91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.385+0000 7ff011805740 1 --2- 172.21.15.18:0/560779021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b8d60 0x5634595b91e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7feffc009bc0 tx=0x7feffc00e6c0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.385+0000 7ff011805740 1 -- 172.21.15.18:0/560779021 shutdown_connections 2024-02-17T12:50:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.385+0000 7ff011805740 1 --2- 172.21.15.18:0/560779021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 0x5634595c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.385+0000 7ff011805740 1 --2- 172.21.15.18:0/560779021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b8d60 0x5634595b91e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.385+0000 7ff011805740 1 --2- 172.21.15.18:0/560779021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595b9ad0 0x5634595b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.385+0000 7ff011805740 1 -- 172.21.15.18:0/560779021 >> 172.21.15.18:0/560779021 conn(0x563459580550 msgr2=0x5634595b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.385+0000 7ff011805740 1 -- 172.21.15.18:0/560779021 shutdown_connections 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.385+0000 7ff011805740 1 -- 172.21.15.18:0/560779021 wait complete. 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.385+0000 7ff011805740 1 -- start start 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.385+0000 7ff011805740 1 Processor -- start 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.386+0000 7ff011805740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b9ad0 0x5634595bf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.386+0000 7ff011805740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595c22e0 0x5634595bf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.386+0000 7ff011805740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595bfe90 0x5634595ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.386+0000 7ff011805740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634594c5bf0 con 0x5634595b9ad0 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.386+0000 7ff011805740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634591f4250 con 0x5634595c22e0 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.386+0000 7ff011805740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634594a6200 con 0x5634595bfe90 2024-02-17T12:50:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.386+0000 7ff011805740 10 client.0 _ll_register_callbacks cb 0x563459514d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.386+0000 7ff007fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b9ad0 0x5634595bf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.386+0000 7ff007fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b9ad0 0x5634595bf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57528/0 (socket says 172.21.15.18:57528) 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.386+0000 7ff007fff640 1 -- 172.21.15.18:0/2763526021 learned_addr learned my addr 172.21.15.18:0/2763526021 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.386+0000 7ff007fff640 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595bfe90 0x5634595ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.387+0000 7ff007fff640 1 -- 172.21.15.18:0/2763526021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595bfe90 msgr2=0x5634595ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.387+0000 7ff007fff640 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595bfe90 0x5634595ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.387+0000 7ff007fff640 1 -- 172.21.15.18:0/2763526021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595c22e0 msgr2=0x5634595bf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.386+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595c22e0 0x5634595bf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.387+0000 7ff007fff640 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595c22e0 0x5634595bf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.387+0000 7ff007fff640 1 -- 172.21.15.18:0/2763526021 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feffc009820 con 0x5634595b9ad0 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.387+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595c22e0 0x5634595bf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.387+0000 7ff007fff640 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b9ad0 0x5634595bf410 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feff4005940 tx=0x7feff402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdaae755e80538e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.387+0000 7fefe7fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.387+0000 7fefe7fff640 1 -- 172.21.15.18:0/2763526021 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4004df0 con 0x5634595b9ad0 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.387+0000 7fefe7fff640 1 -- 172.21.15.18:0/2763526021 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7feff4004f50 con 0x5634595b9ad0 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.387+0000 7ff011805740 1 -- 172.21.15.18:0/2763526021 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563459220600 con 0x5634595b9ad0 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.388+0000 7fefe7fff640 1 -- 172.21.15.18:0/2763526021 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feff4005110 con 0x5634595b9ad0 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.388+0000 7ff011805740 10 client.15792 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.388+0000 7ff011805740 1 -- 172.21.15.18:0/2763526021 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56345951ed80 con 0x5634595b9ad0 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.388+0000 7fefe7fff640 1 -- 172.21.15.18:0/2763526021 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563459220600 con 0x5634595b9ad0 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.388+0000 7fefe7fff640 1 -- 172.21.15.18:0/2763526021 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feff4043450 con 0x5634595b9ad0 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.388+0000 7ff011805740 10 client.15792 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.388+0000 7ff011805740 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.388+0000 7ff011805740 20 client.15792 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.389+0000 7ff011805740 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5634595d2c00 0x5634595d4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.389+0000 7ff011805740 4 client.15792 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.389+0000 7ff011805740 1 -- 172.21.15.18:0/2763526021 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634592711b0 con 0x5634595d2c00 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.389+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5634595d2c00 0x5634595d4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.390+0000 7fefe7fff640 10 client.15792 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.390+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5634595d2c00 0x5634595d4ff0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc3647f02e84b1dd server_cookie=a44b4b4a054d571b in_seq=0 out_seq=0 2024-02-17T12:50:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.390+0000 7fefe7fff640 1 -- 172.21.15.18:0/2763526021 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634592711b0 con 0x5634595d2c00 2024-02-17T12:50:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.390+0000 7fefe7fff640 10 client.15792 handle_command_reply: tid=0 2024-02-17T12:50:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.390+0000 7ff011805740 1 client.15792 shutdown 2024-02-17T12:50:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.390+0000 7ff011805740 1 -- 172.21.15.18:0/2763526021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b9ad0 msgr2=0x5634595bf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.390+0000 7ff011805740 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b9ad0 0x5634595bf410 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feff4005940 tx=0x7feff402bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.390+0000 7ff011805740 1 -- 172.21.15.18:0/2763526021 shutdown_connections 2024-02-17T12:50:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.390+0000 7ff011805740 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5634595d2c00 0x5634595d4ff0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.391+0000 7ff011805740 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595bfe90 0x5634595ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.391+0000 7ff011805740 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595c22e0 0x5634595bf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.391+0000 7ff011805740 1 --2- 172.21.15.18:0/2763526021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595b9ad0 0x5634595bf410 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.391+0000 7fefe7fff640 0 client.15792 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.391+0000 7ff011805740 1 -- 172.21.15.18:0/2763526021 >> 172.21.15.18:0/2763526021 conn(0x563459580550 msgr2=0x5634595b57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.391+0000 7ff011805740 1 -- 172.21.15.18:0/2763526021 shutdown_connections 2024-02-17T12:50:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.391+0000 7ff011805740 1 -- 172.21.15.18:0/2763526021 wait complete. 2024-02-17T12:50:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.391+0000 7ff011805740 20 client.15792 trim_cache size 0 max 16384 2024-02-17T12:50:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.404+0000 7ff011805740 1 Processor -- start 2024-02-17T12:50:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.404+0000 7ff011805740 1 -- start start 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.404+0000 7ff011805740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595bfe90 0x5634595d9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff011805740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595c22e0 0x5634595b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff011805740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595ce2d0 0x5634595b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff011805740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634595af180 con 0x5634595c22e0 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff011805740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634595bb090 con 0x5634595ce2d0 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff011805740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634595c4ef0 con 0x5634595bfe90 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff00c8f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595c22e0 0x5634595b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff00c8f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595c22e0 0x5634595b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57540/0 (socket says 172.21.15.18:57540) 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff00c8f8640 1 -- 172.21.15.18:0/2058215402 learned_addr learned my addr 172.21.15.18:0/2058215402 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff007fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595bfe90 0x5634595d9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff007fff640 1 --2- 172.21.15.18:0/2058215402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595ce2d0 0x5634595b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff00c8f8640 1 -- 172.21.15.18:0/2058215402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595bfe90 msgr2=0x5634595d9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/2058215402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595bfe90 0x5634595d9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff00c8f8640 1 -- 172.21.15.18:0/2058215402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595ce2d0 msgr2=0x5634595b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/2058215402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595ce2d0 0x5634595b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff00c8f8640 1 -- 172.21.15.18:0/2058215402 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634594bb7b0 con 0x5634595c22e0 2024-02-17T12:50:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff007fff640 1 --2- 172.21.15.18:0/2058215402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595ce2d0 0x5634595b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.405+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/2058215402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595c22e0 0x5634595b3580 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7feffc004d60 tx=0x7feffc004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ad158f7810feca server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.406+0000 7feff27fc640 1 -- 172.21.15.18:0/2058215402 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc02c030 con 0x5634595c22e0 2024-02-17T12:50:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.406+0000 7feff27fc640 1 -- 172.21.15.18:0/2058215402 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feffc009eb0 con 0x5634595c22e0 2024-02-17T12:50:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.406+0000 7feff27fc640 1 -- 172.21.15.18:0/2058215402 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc028030 con 0x5634595c22e0 2024-02-17T12:50:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.406+0000 7ff011805740 1 -- 172.21.15.18:0/2058215402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595c22e0 msgr2=0x5634595b3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.406+0000 7ff011805740 1 --2- 172.21.15.18:0/2058215402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595c22e0 0x5634595b3580 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7feffc004d60 tx=0x7feffc004d90 comp rx=0 tx=0).stop 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.407+0000 7ff011805740 1 -- 172.21.15.18:0/2058215402 shutdown_connections 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.407+0000 7ff011805740 1 --2- 172.21.15.18:0/2058215402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595ce2d0 0x5634595b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.407+0000 7ff011805740 1 --2- 172.21.15.18:0/2058215402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595c22e0 0x5634595b3580 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.407+0000 7ff011805740 1 --2- 172.21.15.18:0/2058215402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595bfe90 0x5634595d9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.407+0000 7ff011805740 1 -- 172.21.15.18:0/2058215402 >> 172.21.15.18:0/2058215402 conn(0x563459580550 msgr2=0x5634595cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.407+0000 7ff011805740 1 -- 172.21.15.18:0/2058215402 shutdown_connections 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.407+0000 7ff011805740 1 -- 172.21.15.18:0/2058215402 wait complete. 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.407+0000 7ff011805740 1 -- start start 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.407+0000 7ff011805740 1 Processor -- start 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.407+0000 7ff011805740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595bfe90 0x5634595bc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.407+0000 7ff011805740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 0x5634595bc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff011805740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595ce2d0 0x5634596290b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff011805740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634595bd1e0 con 0x5634595ce2d0 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff011805740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634596295f0 con 0x5634595bfe90 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff011805740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563459629730 con 0x5634595c22e0 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff011805740 10 client.0 _ll_register_callbacks cb 0x7feffc009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff00c8f8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 0x5634595bc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff007fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595ce2d0 0x5634596290b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff00c8f8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 0x5634595bc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:33500/0 (socket says 172.21.15.18:33500) 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff00c8f8640 1 -- 172.21.15.18:0/3717742489 learned_addr learned my addr 172.21.15.18:0/3717742489 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff007fff640 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595bfe90 0x5634595bc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff00c8f8640 1 -- 172.21.15.18:0/3717742489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595bfe90 msgr2=0x5634595bc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595bfe90 0x5634595bc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff00c8f8640 1 -- 172.21.15.18:0/3717742489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595ce2d0 msgr2=0x5634596290b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595ce2d0 0x5634596290b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff00c8f8640 1 -- 172.21.15.18:0/3717742489 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634594b67b0 con 0x5634595c22e0 2024-02-17T12:50:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff007fff640 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595ce2d0 0x5634596290b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7ff00c8f8640 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 0x5634595bc910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feffc001eb0 tx=0x7feffc005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=810c553c1d30a742 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.408+0000 7fefe7fff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.409+0000 7fefe7fff640 1 -- 172.21.15.18:0/3717742489 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc02c070 con 0x5634595c22e0 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.409+0000 7ff011805740 1 -- 172.21.15.18:0/3717742489 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feffc005660 con 0x5634595c22e0 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.409+0000 7fefe7fff640 1 -- 172.21.15.18:0/3717742489 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7feffc000e40 con 0x5634595c22e0 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.409+0000 7fefe7fff640 1 -- 172.21.15.18:0/3717742489 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feffc009460 con 0x5634595c22e0 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.409+0000 7ff011805740 10 client.25348 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.409+0000 7ff011805740 1 -- 172.21.15.18:0/3717742489 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5634594e1540 con 0x5634595c22e0 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.409+0000 7fefe7fff640 1 -- 172.21.15.18:0/3717742489 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feffc005660 con 0x5634595c22e0 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.409+0000 7fefe7fff640 1 -- 172.21.15.18:0/3717742489 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feffc025040 con 0x5634595c22e0 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.409+0000 7ff011805740 10 client.25348 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.409+0000 7ff011805740 10 client.25348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.409+0000 7ff011805740 20 client.25348 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.409+0000 7ff011805740 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56345962aa50 0x56345962ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.410+0000 7ff011805740 4 client.25348 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:50:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.410+0000 7ff011805740 1 -- 172.21.15.18:0/3717742489 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56345925d550 con 0x56345962aa50 2024-02-17T12:50:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.410+0000 7ff007fff640 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56345962aa50 0x56345962ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.410+0000 7fefe7fff640 10 client.25348 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.410+0000 7ff007fff640 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56345962aa50 0x56345962ce70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d31804f04f7be80a server_cookie=ab517715b57f29a2 in_seq=0 out_seq=0 2024-02-17T12:50:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7fefe7fff640 1 -- 172.21.15.18:0/3717742489 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56345925d550 con 0x56345962aa50 2024-02-17T12:50:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7fefe7fff640 10 client.25348 handle_command_reply: tid=0 2024-02-17T12:50:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7ff011805740 1 client.25348 shutdown 2024-02-17T12:50:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7ff011805740 1 -- 172.21.15.18:0/3717742489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 msgr2=0x5634595bc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7ff011805740 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 0x5634595bc910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feffc001eb0 tx=0x7feffc005ab0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7ff011805740 1 -- 172.21.15.18:0/3717742489 shutdown_connections 2024-02-17T12:50:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7ff011805740 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56345962aa50 0x56345962ce70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7ff011805740 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5634595ce2d0 0x5634596290b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7ff011805740 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634595c22e0 0x5634595bc910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7ff011805740 1 --2- 172.21.15.18:0/3717742489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634595bfe90 0x5634595bc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7fefe7fff640 0 client.25348 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7ff011805740 1 -- 172.21.15.18:0/3717742489 >> 172.21.15.18:0/3717742489 conn(0x563459580550 msgr2=0x5634595bf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7ff011805740 1 -- 172.21.15.18:0/3717742489 shutdown_connections 2024-02-17T12:50:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7ff011805740 1 -- 172.21.15.18:0/3717742489 wait complete. 2024-02-17T12:50:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.414+0000 7ff011805740 20 client.25348 trim_cache size 0 max 16384 2024-02-17T12:50:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:50:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:50:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:50:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:50:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.415+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1220130061 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7feff80a2470 msgr2=0x7feff80a4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.415+0000 7ff00eb5c640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7feff80a2470 0x7feff80a4930 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0000cbc20 tx=0x7ff000005800 comp rx=0 tx=0).stop 2024-02-17T12:50:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.415+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1220130061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0081057a0 msgr2=0x7ff00819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.415+0000 7ff00eb5c640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0081057a0 0x7ff00819b3e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff00000eaa0 tx=0x7ff00000ef60 comp rx=0 tx=0).stop 2024-02-17T12:50:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.415+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1220130061 shutdown_connections 2024-02-17T12:50:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.415+0000 7ff00eb5c640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7feff80a2470 0x7feff80a4930 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.415+0000 7ff00eb5c640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0081057a0 0x7ff00819b3e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.415+0000 7ff00eb5c640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff008100150 0x7ff0081968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.415+0000 7ff00eb5c640 1 --2- 172.21.15.18:0/1220130061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0080ff790 0x7ff008196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.415+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1220130061 >> 172.21.15.18:0/1220130061 conn(0x7ff008076610 msgr2=0x7ff008102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.415+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1220130061 shutdown_connections 2024-02-17T12:50:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:21.415+0000 7ff00eb5c640 1 -- 172.21.15.18:0/1220130061 wait complete. 2024-02-17T12:50:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:22 smithi018 ceph-mon[32625]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.4 KiB/s wr, 18 op/s 2024-02-17T12:50:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:22 smithi119 ceph-mon[36188]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.4 KiB/s wr, 18 op/s 2024-02-17T12:50:22.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:22 smithi146 ceph-mon[36454]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.4 KiB/s wr, 18 op/s 2024-02-17T12:50:24.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:24 smithi018 ceph-mon[32625]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-02-17T12:50:24.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:24 smithi119 ceph-mon[36188]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-02-17T12:50:24.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:24 smithi146 ceph-mon[36454]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-02-17T12:50:26.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:26 smithi018 ceph-mon[32625]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-02-17T12:50:26.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:26 smithi119 ceph-mon[36188]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-02-17T12:50:26.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:26 smithi146 ceph-mon[36454]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-02-17T12:50:28.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:28 smithi018 ceph-mon[32625]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-17T12:50:28.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:28 smithi119 ceph-mon[36188]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-17T12:50:28.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:28 smithi146 ceph-mon[36454]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-17T12:50:30.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:30 smithi018 ceph-mon[32625]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-02-17T12:50:30.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:30 smithi119 ceph-mon[36188]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-02-17T12:50:30.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:30 smithi146 ceph-mon[36454]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-02-17T12:50:32.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:32 smithi018 ceph-mon[32625]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-02-17T12:50:32.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:32 smithi119 ceph-mon[36188]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-02-17T12:50:32.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:32 smithi146 ceph-mon[36454]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-02-17T12:50:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.544+0000 7fcb36c64640 1 -- 172.21.15.18:0/1345194064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb30100c90 msgr2=0x7fcb3010ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.544+0000 7fcb36c64640 1 --2- 172.21.15.18:0/1345194064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb30100c90 0x7fcb3010ceb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009ca0 tx=0x7fcb2802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.544+0000 7fcb36c64640 1 -- 172.21.15.18:0/1345194064 shutdown_connections 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.544+0000 7fcb36c64640 1 --2- 172.21.15.18:0/1345194064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb30100c90 0x7fcb3010ceb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.544+0000 7fcb36c64640 1 --2- 172.21.15.18:0/1345194064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb301002d0 0x7fcb30100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.544+0000 7fcb36c64640 1 --2- 172.21.15.18:0/1345194064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb30105020 0x7fcb30105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.544+0000 7fcb36c64640 1 -- 172.21.15.18:0/1345194064 >> 172.21.15.18:0/1345194064 conn(0x7fcb30076ed0 msgr2=0x7fcb300772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.544+0000 7fcb36c64640 1 -- 172.21.15.18:0/1345194064 shutdown_connections 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.544+0000 7fcb36c64640 1 -- 172.21.15.18:0/1345194064 wait complete. 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.544+0000 7fcb36c64640 1 Processor -- start 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.544+0000 7fcb36c64640 1 -- start start 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb36c64640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb301002d0 0x7fcb30101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb36c64640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb30100c90 0x7fcb30102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb36c64640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb30105020 0x7fcb30102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb36c64640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb301a5aa0 con 0x7fcb301002d0 2024-02-17T12:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb36c64640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb301a5c10 con 0x7fcb30105020 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb36c64640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb301a5dd0 con 0x7fcb30100c90 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb34a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb301002d0 0x7fcb30101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb34a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb301002d0 0x7fcb30101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45502/0 (socket says 172.21.15.18:45502) 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb34a00640 1 -- 172.21.15.18:0/3505456752 learned_addr learned my addr 172.21.15.18:0/3505456752 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb35201640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb30105020 0x7fcb30102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb27fff640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb30100c90 0x7fcb30102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb34a00640 1 -- 172.21.15.18:0/3505456752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb30100c90 msgr2=0x7fcb30102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb34a00640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb30100c90 0x7fcb30102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb34a00640 1 -- 172.21.15.18:0/3505456752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb30105020 msgr2=0x7fcb30102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb34a00640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb30105020 0x7fcb30102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb34a00640 1 -- 172.21.15.18:0/3505456752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb28009900 con 0x7fcb301002d0 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.545+0000 7fcb34a00640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb301002d0 0x7fcb30101cf0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00e9f0 tx=0x7fcb2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.546+0000 7fcb25ffb640 1 -- 172.21.15.18:0/3505456752 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c009700 con 0x7fcb301002d0 2024-02-17T12:50:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.546+0000 7fcb25ffb640 1 -- 172.21.15.18:0/3505456752 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb2c00cd80 con 0x7fcb301002d0 2024-02-17T12:50:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.546+0000 7fcb25ffb640 1 -- 172.21.15.18:0/3505456752 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c018920 con 0x7fcb301002d0 2024-02-17T12:50:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.546+0000 7fcb36c64640 1 -- 172.21.15.18:0/3505456752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb301a60b0 con 0x7fcb301002d0 2024-02-17T12:50:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.546+0000 7fcb36c64640 1 -- 172.21.15.18:0/3505456752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb301a6570 con 0x7fcb301002d0 2024-02-17T12:50:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.549+0000 7fcb25ffb640 1 -- 172.21.15.18:0/3505456752 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fcb2c018a80 con 0x7fcb301002d0 2024-02-17T12:50:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.549+0000 7fcb39836740 1 Processor -- start 2024-02-17T12:50:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.549+0000 7fcb39836740 1 -- start start 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.549+0000 7fcb39836740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b998ad0 0x55e96b997820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.549+0000 7fcb39836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b997d60 0x55e96b9981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.549+0000 7fcb39836740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 0x55e96b9a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.549+0000 7fcb39836740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96b9069f0 con 0x55e96b9a12e0 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.549+0000 7fcb39836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96b903f20 con 0x55e96b997d60 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.549+0000 7fcb39836740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96b8ff700 con 0x55e96b998ad0 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.549+0000 7fcb25ffb640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb180a24a0 0x7fcb180a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.549+0000 7fcb27fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 0x55e96b9a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb27fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 0x55e96b9a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45512/0 (socket says 172.21.15.18:45512) 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb27fff640 1 -- 172.21.15.18:0/3160932044 learned_addr learned my addr 172.21.15.18:0/3160932044 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb35201640 1 --2- 172.21.15.18:0/3160932044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b998ad0 0x55e96b997820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb25ffb640 1 -- 172.21.15.18:0/3505456752 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb2c014070 con 0x7fcb301002d0 2024-02-17T12:50:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb27fff640 1 -- 172.21.15.18:0/3160932044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b998ad0 msgr2=0x55e96b997820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb27fff640 1 --2- 172.21.15.18:0/3160932044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b998ad0 0x55e96b997820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb35201640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb180a24a0 0x7fcb180a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb27fff640 1 -- 172.21.15.18:0/3160932044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b997d60 msgr2=0x55e96b9981e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:50:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb27fff640 1 --2- 172.21.15.18:0/3160932044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b997d60 0x55e96b9981e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb27fff640 1 -- 172.21.15.18:0/3160932044 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e96b8957b0 con 0x55e96b9a12e0 2024-02-17T12:50:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb27fff640 1 --2- 172.21.15.18:0/3160932044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 0x55e96b9a36d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009c30 tx=0x7fcb200117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814a9d008deaf3cf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb16ffd640 1 -- 172.21.15.18:0/3160932044 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20012450 con 0x55e96b9a12e0 2024-02-17T12:50:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb16ffd640 1 -- 172.21.15.18:0/3160932044 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb20012a90 con 0x55e96b9a12e0 2024-02-17T12:50:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.550+0000 7fcb16ffd640 1 -- 172.21.15.18:0/3160932044 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2000fac0 con 0x55e96b9a12e0 2024-02-17T12:50:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.551+0000 7fcb39836740 1 -- 172.21.15.18:0/3160932044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 msgr2=0x55e96b9a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.551+0000 7fcb39836740 1 --2- 172.21.15.18:0/3160932044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 0x55e96b9a36d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009c30 tx=0x7fcb200117b0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.551+0000 7fcb35201640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb180a24a0 0x7fcb180a4960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb28004af0 tx=0x7fcb2802c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.551+0000 7fcb39836740 1 -- 172.21.15.18:0/3160932044 shutdown_connections 2024-02-17T12:50:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.551+0000 7fcb39836740 1 --2- 172.21.15.18:0/3160932044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 0x55e96b9a36d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.551+0000 7fcb39836740 1 --2- 172.21.15.18:0/3160932044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b997d60 0x55e96b9981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.551+0000 7fcb39836740 1 --2- 172.21.15.18:0/3160932044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b998ad0 0x55e96b997820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.551+0000 7fcb39836740 1 -- 172.21.15.18:0/3160932044 >> 172.21.15.18:0/3160932044 conn(0x55e96b95f550 msgr2=0x55e96b992a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.551+0000 7fcb39836740 1 -- 172.21.15.18:0/3160932044 shutdown_connections 2024-02-17T12:50:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.551+0000 7fcb39836740 1 -- 172.21.15.18:0/3160932044 wait complete. 2024-02-17T12:50:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb39836740 1 -- start start 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb39836740 1 Processor -- start 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb39836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b997d60 0x55e96b99e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb39836740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b998ad0 0x55e96b99e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb39836740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 0x55e96b99ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb39836740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96b8a4bf0 con 0x55e96b9a12e0 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb39836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96b5d3250 con 0x55e96b997d60 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb39836740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96b885200 con 0x55e96b998ad0 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb39836740 10 client.0 _ll_register_callbacks cb 0x55e96b8f3d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb35201640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 0x55e96b99ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb35201640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 0x55e96b99ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45514/0 (socket says 172.21.15.18:45514) 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb35201640 1 -- 172.21.15.18:0/1070267987 learned_addr learned my addr 172.21.15.18:0/1070267987 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.552+0000 7fcb27fff640 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b998ad0 0x55e96b99e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb34a00640 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b997d60 0x55e96b99e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb35201640 1 -- 172.21.15.18:0/1070267987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b998ad0 msgr2=0x55e96b99e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb35201640 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b998ad0 0x55e96b99e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb35201640 1 -- 172.21.15.18:0/1070267987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b997d60 msgr2=0x55e96b99e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb35201640 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b997d60 0x55e96b99e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb35201640 1 -- 172.21.15.18:0/1070267987 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb20009860 con 0x55e96b9a12e0 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb35201640 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 0x55e96b99ee40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009ca0 tx=0x7fcb280095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72abf49ba3e20551 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb0ffff640 1 -- 172.21.15.18:0/1070267987 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28032970 con 0x55e96b9a12e0 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb0ffff640 1 -- 172.21.15.18:0/1070267987 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb28032ad0 con 0x55e96b9a12e0 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb39836740 1 -- 172.21.15.18:0/1070267987 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e96b5ff600 con 0x55e96b9a12e0 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb0ffff640 1 -- 172.21.15.18:0/1070267987 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb28003970 con 0x55e96b9a12e0 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb39836740 10 client.15810 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.553+0000 7fcb39836740 1 -- 172.21.15.18:0/1070267987 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e96b8fdd80 con 0x55e96b9a12e0 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.554+0000 7fcb0ffff640 1 -- 172.21.15.18:0/1070267987 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb2805d020 con 0x55e96b9a12e0 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.554+0000 7fcb0ffff640 1 -- 172.21.15.18:0/1070267987 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcb2804d470 con 0x55e96b9a12e0 2024-02-17T12:50:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.554+0000 7fcb39836740 10 client.15810 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.554+0000 7fcb39836740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.554+0000 7fcb39836740 20 client.15810 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.554+0000 7fcb39836740 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e96b9b1730 0x55e96b9b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.554+0000 7fcb39836740 4 client.15810 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.554+0000 7fcb39836740 1 -- 172.21.15.18:0/1070267987 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e96b6501b0 con 0x55e96b9b1730 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.554+0000 7fcb34a00640 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e96b9b1730 0x55e96b9b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.555+0000 7fcb34a00640 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e96b9b1730 0x55e96b9b3b20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdec04572c74d7cc server_cookie=f15b647008d42215 in_seq=0 out_seq=0 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.555+0000 7fcb0ffff640 10 client.15810 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.555+0000 7fcb0ffff640 1 -- 172.21.15.18:0/1070267987 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e96b6501b0 con 0x55e96b9b1730 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.555+0000 7fcb0ffff640 10 client.15810 handle_command_reply: tid=0 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.555+0000 7fcb39836740 1 client.15810 shutdown 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.556+0000 7fcb39836740 1 -- 172.21.15.18:0/1070267987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 msgr2=0x55e96b99ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.556+0000 7fcb39836740 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 0x55e96b99ee40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009ca0 tx=0x7fcb280095c0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.556+0000 7fcb39836740 1 -- 172.21.15.18:0/1070267987 shutdown_connections 2024-02-17T12:50:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.556+0000 7fcb39836740 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e96b9b1730 0x55e96b9b3b20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.556+0000 7fcb0ffff640 0 client.15810 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.556+0000 7fcb39836740 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b9a12e0 0x55e96b99ee40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.556+0000 7fcb39836740 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b998ad0 0x55e96b99e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.556+0000 7fcb39836740 1 --2- 172.21.15.18:0/1070267987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b997d60 0x55e96b99e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.556+0000 7fcb39836740 1 -- 172.21.15.18:0/1070267987 >> 172.21.15.18:0/1070267987 conn(0x55e96b95f550 msgr2=0x55e96b9947a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.556+0000 7fcb39836740 1 -- 172.21.15.18:0/1070267987 shutdown_connections 2024-02-17T12:50:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.556+0000 7fcb39836740 1 -- 172.21.15.18:0/1070267987 wait complete. 2024-02-17T12:50:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.556+0000 7fcb39836740 20 client.15810 trim_cache size 0 max 16384 2024-02-17T12:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb39836740 1 Processor -- start 2024-02-17T12:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb39836740 1 -- start start 2024-02-17T12:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb39836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b95f550 0x55e96b9a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb39836740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b997d60 0x55e96b9a8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb39836740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b998ad0 0x55e96b9a8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb39836740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96b9ae230 con 0x55e96b998ad0 2024-02-17T12:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb39836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96b993c10 con 0x55e96b95f550 2024-02-17T12:50:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb39836740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96b993a50 con 0x55e96b997d60 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb35201640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b998ad0 0x55e96b9a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb35201640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b998ad0 0x55e96b9a8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45522/0 (socket says 172.21.15.18:45522) 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb35201640 1 -- 172.21.15.18:0/3562973809 learned_addr learned my addr 172.21.15.18:0/3562973809 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb34a00640 1 --2- 172.21.15.18:0/3562973809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b95f550 0x55e96b9a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.581+0000 7fcb27fff640 1 --2- 172.21.15.18:0/3562973809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b997d60 0x55e96b9a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.582+0000 7fcb35201640 1 -- 172.21.15.18:0/3562973809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b997d60 msgr2=0x55e96b9a8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.582+0000 7fcb35201640 1 --2- 172.21.15.18:0/3562973809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b997d60 0x55e96b9a8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.582+0000 7fcb35201640 1 -- 172.21.15.18:0/3562973809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b95f550 msgr2=0x55e96b9a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.582+0000 7fcb35201640 1 --2- 172.21.15.18:0/3562973809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b95f550 0x55e96b9a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.582+0000 7fcb35201640 1 -- 172.21.15.18:0/3562973809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e96b89a7b0 con 0x55e96b998ad0 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.582+0000 7fcb35201640 1 --2- 172.21.15.18:0/3562973809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b998ad0 0x55e96b9a8640 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcb2802c700 tx=0x7fcb28005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17568fb40ac82f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.582+0000 7fcb167fc640 1 -- 172.21.15.18:0/3562973809 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2804d470 con 0x55e96b998ad0 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.582+0000 7fcb167fc640 1 -- 172.21.15.18:0/3562973809 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb2805d040 con 0x55e96b998ad0 2024-02-17T12:50:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.582+0000 7fcb167fc640 1 -- 172.21.15.18:0/3562973809 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb280042a0 con 0x55e96b998ad0 2024-02-17T12:50:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.583+0000 7fcb39836740 1 -- 172.21.15.18:0/3562973809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b998ad0 msgr2=0x55e96b9a8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.583+0000 7fcb39836740 1 --2- 172.21.15.18:0/3562973809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b998ad0 0x55e96b9a8640 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcb2802c700 tx=0x7fcb28005ce0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.583+0000 7fcb39836740 1 -- 172.21.15.18:0/3562973809 shutdown_connections 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.583+0000 7fcb39836740 1 --2- 172.21.15.18:0/3562973809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b998ad0 0x55e96b9a8640 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.583+0000 7fcb39836740 1 --2- 172.21.15.18:0/3562973809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b997d60 0x55e96b9a8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.583+0000 7fcb39836740 1 --2- 172.21.15.18:0/3562973809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b95f550 0x55e96b9a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.583+0000 7fcb39836740 1 -- 172.21.15.18:0/3562973809 >> 172.21.15.18:0/3562973809 conn(0x55e96b9a38b0 msgr2=0x55e96b99ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.583+0000 7fcb39836740 1 -- 172.21.15.18:0/3562973809 shutdown_connections 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.583+0000 7fcb39836740 1 -- 172.21.15.18:0/3562973809 wait complete. 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.583+0000 7fcb39836740 1 -- start start 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb39836740 1 Processor -- start 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb39836740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b95f550 0x55e96b9a8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb39836740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b997d60 0x55e96b99aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb39836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b998ad0 0x55e96b99af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb39836740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96b9a87c0 con 0x55e96b997d60 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb39836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96b9a8930 con 0x55e96b998ad0 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb39836740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96b99b4c0 con 0x55e96b95f550 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb39836740 10 client.0 _ll_register_callbacks cb 0x7fcb20009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb27fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b997d60 0x55e96b99aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb27fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b997d60 0x55e96b99aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45538/0 (socket says 172.21.15.18:45538) 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb27fff640 1 -- 172.21.15.18:0/2861316372 learned_addr learned my addr 172.21.15.18:0/2861316372 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb34a00640 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b95f550 0x55e96b9a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.584+0000 7fcb35201640 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b998ad0 0x55e96b99af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb27fff640 1 -- 172.21.15.18:0/2861316372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b95f550 msgr2=0x55e96b9a8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb27fff640 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b95f550 0x55e96b9a8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb27fff640 1 -- 172.21.15.18:0/2861316372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b998ad0 msgr2=0x55e96b99af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb27fff640 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b998ad0 0x55e96b99af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb27fff640 1 -- 172.21.15.18:0/2861316372 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e96b8fdd80 con 0x55e96b997d60 2024-02-17T12:50:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb27fff640 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b997d60 0x55e96b99aa40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x55e96b998800 tx=0x7fcb20012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa98ee62b24487 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb0ffff640 1 -- 172.21.15.18:0/2861316372 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20005240 con 0x55e96b997d60 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb0ffff640 1 -- 172.21.15.18:0/2861316372 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb200053a0 con 0x55e96b997d60 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb39836740 1 -- 172.21.15.18:0/2861316372 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb20012a90 con 0x55e96b997d60 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb0ffff640 1 -- 172.21.15.18:0/2861316372 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb20005560 con 0x55e96b997d60 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb39836740 10 client.15816 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.585+0000 7fcb39836740 1 -- 172.21.15.18:0/2861316372 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e96b8c0540 con 0x55e96b997d60 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.586+0000 7fcb0ffff640 1 -- 172.21.15.18:0/2861316372 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb20012a90 con 0x55e96b997d60 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.586+0000 7fcb0ffff640 1 -- 172.21.15.18:0/2861316372 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcb20022040 con 0x55e96b997d60 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.586+0000 7fcb39836740 10 client.15816 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.586+0000 7fcb39836740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.586+0000 7fcb39836740 20 client.15816 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.586+0000 7fcb39836740 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e96b9a12e0 0x55e96ba0ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.586+0000 7fcb39836740 4 client.15816 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:50:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.586+0000 7fcb39836740 1 -- 172.21.15.18:0/2861316372 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e96b63c550 con 0x55e96b9a12e0 2024-02-17T12:50:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.586+0000 7fcb34a00640 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e96b9a12e0 0x55e96ba0ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.587+0000 7fcb0ffff640 10 client.15816 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.587+0000 7fcb34a00640 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e96b9a12e0 0x55e96ba0ade0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01b7443d471de06 server_cookie=94547b841ba2a3d9 in_seq=0 out_seq=0 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.594+0000 7fcb0ffff640 1 -- 172.21.15.18:0/2861316372 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e96b63c550 con 0x55e96b9a12e0 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.594+0000 7fcb0ffff640 10 client.15816 handle_command_reply: tid=0 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.594+0000 7fcb39836740 1 client.15816 shutdown 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.594+0000 7fcb39836740 1 -- 172.21.15.18:0/2861316372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b997d60 msgr2=0x55e96b99aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.594+0000 7fcb39836740 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b997d60 0x55e96b99aa40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x55e96b998800 tx=0x7fcb20012680 comp rx=0 tx=0).stop 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.594+0000 7fcb39836740 1 -- 172.21.15.18:0/2861316372 shutdown_connections 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.594+0000 7fcb39836740 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e96b9a12e0 0x55e96ba0ade0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.594+0000 7fcb39836740 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e96b998ad0 0x55e96b99af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.594+0000 7fcb39836740 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e96b997d60 0x55e96b99aa40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.594+0000 7fcb39836740 1 --2- 172.21.15.18:0/2861316372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e96b95f550 0x55e96b9a8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb39836740 1 -- 172.21.15.18:0/2861316372 >> 172.21.15.18:0/2861316372 conn(0x55e96b9a38b0 msgr2=0x55e96b993da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb0ffff640 0 client.15816 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb39836740 1 -- 172.21.15.18:0/2861316372 shutdown_connections 2024-02-17T12:50:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb39836740 1 -- 172.21.15.18:0/2861316372 wait complete. 2024-02-17T12:50:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb39836740 20 client.15816 trim_cache size 0 max 16384 2024-02-17T12:50:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:50:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:50:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:50:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:50:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb36c64640 1 -- 172.21.15.18:0/3505456752 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb180a24a0 msgr2=0x7fcb180a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb36c64640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb180a24a0 0x7fcb180a4960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb28004af0 tx=0x7fcb2802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb36c64640 1 -- 172.21.15.18:0/3505456752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb301002d0 msgr2=0x7fcb30101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb36c64640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb301002d0 0x7fcb30101cf0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00e9f0 tx=0x7fcb2c00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb36c64640 1 -- 172.21.15.18:0/3505456752 shutdown_connections 2024-02-17T12:50:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb36c64640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb180a24a0 0x7fcb180a4960 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb36c64640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb30105020 0x7fcb30102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb36c64640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb30100c90 0x7fcb30102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb36c64640 1 --2- 172.21.15.18:0/3505456752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb301002d0 0x7fcb30101cf0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.595+0000 7fcb36c64640 1 -- 172.21.15.18:0/3505456752 >> 172.21.15.18:0/3505456752 conn(0x7fcb30076ed0 msgr2=0x7fcb300689e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.596+0000 7fcb36c64640 1 -- 172.21.15.18:0/3505456752 shutdown_connections 2024-02-17T12:50:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:34.596+0000 7fcb36c64640 1 -- 172.21.15.18:0/3505456752 wait complete. 2024-02-17T12:50:34.753 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:34 smithi146 ceph-mon[36454]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-17T12:50:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:34 smithi018 ceph-mon[32625]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-17T12:50:34.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:34 smithi119 ceph-mon[36188]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-17T12:50:36.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:36 smithi018 ceph-mon[32625]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-02-17T12:50:36.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:36 smithi119 ceph-mon[36188]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-02-17T12:50:36.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:36 smithi146 ceph-mon[36454]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-02-17T12:50:38.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:38 smithi018 ceph-mon[32625]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-02-17T12:50:38.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:38 smithi119 ceph-mon[36188]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-02-17T12:50:38.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:38 smithi146 ceph-mon[36454]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-02-17T12:50:40.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:40 smithi018 ceph-mon[32625]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-02-17T12:50:40.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:40 smithi119 ceph-mon[36188]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-02-17T12:50:40.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:40 smithi146 ceph-mon[36454]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-02-17T12:50:42.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:42 smithi018 ceph-mon[32625]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-02-17T12:50:42.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:42 smithi119 ceph-mon[36188]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-02-17T12:50:42.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:42 smithi146 ceph-mon[36454]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-02-17T12:50:43.432 INFO:tasks.workunit.client.0.smithi018.stderr:Updating files: 97% (11770/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-02-17T12:50:44.146 INFO:tasks.workunit.client.0.smithi018.stderr:Note: switching to '6822c7b54e949006be7a0802650d01723d834d1d'. 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr:state without impacting any branches by switching back to a branch. 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr: git switch -c 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr:Or undo this operation with: 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr: git switch - 2024-02-17T12:50:44.147 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-02-17T12:50:44.148 INFO:tasks.workunit.client.0.smithi018.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-17T12:50:44.148 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-02-17T12:50:44.148 INFO:tasks.workunit.client.0.smithi018.stderr:HEAD is now at 6822c7b54e9 Merge pull request #55453 from ronen-fr/wip-rf-0741-logs 2024-02-17T12:50:44.151 DEBUG:teuthology.orchestra.run.smithi018:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-17T12:50:44.182 INFO:tasks.workunit.client.0.smithi018.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-17T12:50:44.183 INFO:tasks.workunit.client.0.smithi018.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-17T12:50:44.184 INFO:tasks.workunit.client.0.smithi018.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-17T12:50:44.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:44 smithi018 ceph-mon[32625]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-02-17T12:50:44.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:44 smithi119 ceph-mon[36188]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-02-17T12:50:44.889 INFO:tasks.workunit.client.0.smithi018.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-17T12:50:44.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:44 smithi146 ceph-mon[36454]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-02-17T12:50:44.951 INFO:tasks.workunit.client.0.smithi018.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-17T12:50:44.990 INFO:tasks.workunit.client.0.smithi018.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-17T12:50:44.991 INFO:tasks.workunit.client.0.smithi018.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-17T12:50:44.991 INFO:tasks.workunit.client.0.smithi018.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-17T12:50:45.029 INFO:tasks.workunit.client.0.smithi018.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-17T12:50:45.033 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T12:50:45.033 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-17T12:50:45.088 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-02-17T12:50:45.089 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-02-17T12:50:45.089 DEBUG:teuthology.orchestra.run.smithi018:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6822c7b54e949006be7a0802650d01723d834d1d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-02-17T12:50:45.148 INFO:tasks.workunit.client.0.smithi018.stderr:+ dbench 1 2024-02-17T12:50:45.152 INFO:tasks.workunit.client.0.smithi018.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-02-17T12:50:45.153 INFO:tasks.workunit.client.0.smithi018.stdout: 2024-02-17T12:50:45.153 INFO:tasks.workunit.client.0.smithi018.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-02-17T12:50:45.157 INFO:tasks.workunit.client.0.smithi018.stdout:0 of 1 processes prepared for launch 0 sec 2024-02-17T12:50:45.258 INFO:tasks.workunit.client.0.smithi018.stdout:1 of 1 processes prepared for launch 0 sec 2024-02-17T12:50:45.258 INFO:tasks.workunit.client.0.smithi018.stdout:releasing clients 2024-02-17T12:50:46.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 3702 38.78 MB/sec warmup 1 sec latency 6.440 ms 2024-02-17T12:50:46.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:46 smithi018 ceph-mon[32625]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-02-17T12:50:46.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:46 smithi119 ceph-mon[36188]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-02-17T12:50:46.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:46 smithi146 ceph-mon[36454]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-02-17T12:50:47.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 7190 28.88 MB/sec warmup 2 sec latency 5.584 ms 2024-02-17T12:50:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.686+0000 7f87111c9640 1 -- 172.21.15.18:0/1530005619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c1002d0 msgr2=0x7f870c100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.686+0000 7f87111c9640 1 --2- 172.21.15.18:0/1530005619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c1002d0 0x7f870c100730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8700009900 tx=0x7f870002bc80 comp rx=0 tx=0).stop 2024-02-17T12:50:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.686+0000 7f87111c9640 1 -- 172.21.15.18:0/1530005619 shutdown_connections 2024-02-17T12:50:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.686+0000 7f87111c9640 1 --2- 172.21.15.18:0/1530005619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f870c100c70 0x7f870c10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.686+0000 7f87111c9640 1 --2- 172.21.15.18:0/1530005619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c1002d0 0x7f870c100730 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.686+0000 7f87111c9640 1 --2- 172.21.15.18:0/1530005619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f870c105040 0x7f870c105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.686+0000 7f87111c9640 1 -- 172.21.15.18:0/1530005619 >> 172.21.15.18:0/1530005619 conn(0x7f870c076ee0 msgr2=0x7f870c0772f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.686+0000 7f87111c9640 1 -- 172.21.15.18:0/1530005619 shutdown_connections 2024-02-17T12:50:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.686+0000 7f87111c9640 1 -- 172.21.15.18:0/1530005619 wait complete. 2024-02-17T12:50:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f87111c9640 1 Processor -- start 2024-02-17T12:50:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f87111c9640 1 -- start start 2024-02-17T12:50:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f87111c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f870c1002d0 0x7f870c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f87111c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f870c100c70 0x7f870c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f87111c9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c105040 0x7f870c197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f87111c9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c197dc0 con 0x7f870c105040 2024-02-17T12:50:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f87111c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c197f30 con 0x7f870c1002d0 2024-02-17T12:50:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f87111c9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c1980f0 con 0x7f870c100c70 2024-02-17T12:50:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f870b59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c105040 0x7f870c197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f870b59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c105040 0x7f870c197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44248/0 (socket says 172.21.15.18:44248) 2024-02-17T12:50:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f870b59e640 1 -- 172.21.15.18:0/795914174 learned_addr learned my addr 172.21.15.18:0/795914174 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f870ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f870c1002d0 0x7f870c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f870a59c640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f870c100c70 0x7f870c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f870b59e640 1 -- 172.21.15.18:0/795914174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f870c100c70 msgr2=0x7f870c19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f870b59e640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f870c100c70 0x7f870c19dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f870b59e640 1 -- 172.21.15.18:0/795914174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f870c1002d0 msgr2=0x7f870c19d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f870b59e640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f870c1002d0 0x7f870c19d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f870b59e640 1 -- 172.21.15.18:0/795914174 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8700009560 con 0x7f870c105040 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.687+0000 7f870ad9d640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f870c1002d0 0x7f870c19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.688+0000 7f870b59e640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c105040 0x7f870c197630 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f870400d680 tx=0x7f870400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.688+0000 7f86f3fff640 1 -- 172.21.15.18:0/795914174 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704005220 con 0x7f870c105040 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.688+0000 7f86f3fff640 1 -- 172.21.15.18:0/795914174 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8704009d00 con 0x7f870c105040 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.688+0000 7f87111c9640 1 -- 172.21.15.18:0/795914174 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870c1983d0 con 0x7f870c105040 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.688+0000 7f86f3fff640 1 -- 172.21.15.18:0/795914174 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704004dd0 con 0x7f870c105040 2024-02-17T12:50:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.688+0000 7f87111c9640 1 -- 172.21.15.18:0/795914174 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870c1a44f0 con 0x7f870c105040 2024-02-17T12:50:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.690+0000 7f86f3fff640 1 -- 172.21.15.18:0/795914174 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f8704009820 con 0x7f870c105040 2024-02-17T12:50:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.690+0000 7f8713d9e740 1 Processor -- start 2024-02-17T12:50:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.690+0000 7f8713d9e740 1 -- start start 2024-02-17T12:50:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f8713d9e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bdad0 0x55a52c9bc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f8713d9e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bcd60 0x55a52c9bd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f8713d9e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f8713d9e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52c92b9f0 con 0x55a52c9bdad0 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f8713d9e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52c928f20 con 0x55a52c9c62e0 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f8713d9e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52c924700 con 0x55a52c9bcd60 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f86f3fff640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f86f40a23f0 0x7f86f40a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bdad0 0x55a52c9bc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bdad0 0x55a52c9bc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44252/0 (socket says 172.21.15.18:44252) 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870a59c640 1 -- 172.21.15.18:0/4094062197 learned_addr learned my addr 172.21.15.18:0/4094062197 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870b59e640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f86f40a23f0 0x7f86f40a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870ad9d640 1 --2- 172.21.15.18:0/4094062197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bcd60 0x55a52c9bd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870a59c640 1 --2- 172.21.15.18:0/4094062197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f86f3fff640 1 -- 172.21.15.18:0/795914174 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f87040cdc50 con 0x7f870c105040 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870a59c640 1 -- 172.21.15.18:0/4094062197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bcd60 msgr2=0x55a52c9bd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870a59c640 1 --2- 172.21.15.18:0/4094062197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bcd60 0x55a52c9bd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870a59c640 1 -- 172.21.15.18:0/4094062197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 msgr2=0x55a52c9c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870a59c640 1 --2- 172.21.15.18:0/4094062197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870a59c640 1 -- 172.21.15.18:0/4094062197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a52c8ba7b0 con 0x55a52c9bdad0 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870ad9d640 1 --2- 172.21.15.18:0/4094062197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bcd60 0x55a52c9bd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.691+0000 7f870a59c640 1 --2- 172.21.15.18:0/4094062197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bdad0 0x55a52c9bc820 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f870002c1c0 tx=0x7f8700005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8ce49a65b5d31b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f870b59e640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f86f40a23f0 0x7f86f40a48b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f87040cb270 tx=0x7f87040048a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f86f17fa640 1 -- 172.21.15.18:0/4094062197 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87000046c0 con 0x55a52c9bdad0 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f86f17fa640 1 -- 172.21.15.18:0/4094062197 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8700004510 con 0x55a52c9bdad0 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f86f17fa640 1 -- 172.21.15.18:0/4094062197 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700005160 con 0x55a52c9bdad0 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f8713d9e740 1 -- 172.21.15.18:0/4094062197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bdad0 msgr2=0x55a52c9bc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f8713d9e740 1 --2- 172.21.15.18:0/4094062197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bdad0 0x55a52c9bc820 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f870002c1c0 tx=0x7f8700005b50 comp rx=0 tx=0).stop 2024-02-17T12:50:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f8713d9e740 1 -- 172.21.15.18:0/4094062197 shutdown_connections 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f8713d9e740 1 --2- 172.21.15.18:0/4094062197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f8713d9e740 1 --2- 172.21.15.18:0/4094062197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bcd60 0x55a52c9bd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f8713d9e740 1 --2- 172.21.15.18:0/4094062197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bdad0 0x55a52c9bc820 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f8713d9e740 1 -- 172.21.15.18:0/4094062197 >> 172.21.15.18:0/4094062197 conn(0x55a52c984550 msgr2=0x55a52c9b7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f8713d9e740 1 -- 172.21.15.18:0/4094062197 shutdown_connections 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.692+0000 7f8713d9e740 1 -- 172.21.15.18:0/4094062197 wait complete. 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f8713d9e740 1 -- start start 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f8713d9e740 1 Processor -- start 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f8713d9e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 0x55a52c9c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f8713d9e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bdad0 0x55a52c9c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f8713d9e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f8713d9e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52c8c9bf0 con 0x55a52c9bcd60 2024-02-17T12:50:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f8713d9e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52c5f8250 con 0x55a52c9c62e0 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f8713d9e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52c8aa200 con 0x55a52c9bdad0 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f8713d9e740 10 client.0 _ll_register_callbacks cb 0x55a52c918d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f870a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 0x55a52c9c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f870a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 0x55a52c9c32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44256/0 (socket says 172.21.15.18:44256) 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f870a59c640 1 -- 172.21.15.18:0/2970886283 learned_addr learned my addr 172.21.15.18:0/2970886283 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f870ad9d640 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bdad0 0x55a52c9c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f870a59c640 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f870a59c640 1 -- 172.21.15.18:0/2970886283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bdad0 msgr2=0x55a52c9c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f870a59c640 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bdad0 0x55a52c9c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f870a59c640 1 -- 172.21.15.18:0/2970886283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 msgr2=0x55a52c9c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f870a59c640 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9c3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.693+0000 7f870a59c640 1 -- 172.21.15.18:0/2970886283 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870c10d860 con 0x55a52c9bcd60 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f870a59c640 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 0x55a52c9c32e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f870002c3f0 tx=0x7f870002c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d096bea17268f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f86eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f86eeff5640 1 -- 172.21.15.18:0/2970886283 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700053070 con 0x55a52c9bcd60 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f86eeff5640 1 -- 172.21.15.18:0/2970886283 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f87000052f0 con 0x55a52c9bcd60 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f8713d9e740 1 -- 172.21.15.18:0/2970886283 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a52c624600 con 0x55a52c9bcd60 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f86eeff5640 1 -- 172.21.15.18:0/2970886283 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f870002c600 con 0x55a52c9bcd60 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f8713d9e740 10 client.15828 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f8713d9e740 1 -- 172.21.15.18:0/2970886283 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a52c922d80 con 0x55a52c9bcd60 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f86eeff5640 1 -- 172.21.15.18:0/2970886283 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a52c624600 con 0x55a52c9bcd60 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f86eeff5640 1 -- 172.21.15.18:0/2970886283 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f870003c030 con 0x55a52c9bcd60 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f8713d9e740 10 client.15828 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f8713d9e740 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f8713d9e740 20 client.15828 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f8713d9e740 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a52c9d6320 0x55a52c9d8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f8713d9e740 4 client.15828 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f8713d9e740 1 -- 172.21.15.18:0/2970886283 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a52c6751b0 con 0x55a52c9d6320 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.694+0000 7f870ad9d640 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a52c9d6320 0x55a52c9d8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.695+0000 7f870ad9d640 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a52c9d6320 0x55a52c9d8710 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d665cf8c12a58ba server_cookie=4c8c759500c69130 in_seq=0 out_seq=0 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.695+0000 7f86eeff5640 10 client.15828 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f86eeff5640 1 -- 172.21.15.18:0/2970886283 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a52c6751b0 con 0x55a52c9d6320 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f86eeff5640 10 client.15828 handle_command_reply: tid=0 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f8713d9e740 1 client.15828 shutdown 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f8713d9e740 1 -- 172.21.15.18:0/2970886283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 msgr2=0x55a52c9c32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f8713d9e740 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 0x55a52c9c32e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f870002c3f0 tx=0x7f870002c380 comp rx=0 tx=0).stop 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f8713d9e740 1 -- 172.21.15.18:0/2970886283 shutdown_connections 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f8713d9e740 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a52c9d6320 0x55a52c9d8710 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f8713d9e740 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f8713d9e740 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bdad0 0x55a52c9c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f86eeff5640 0 client.15828 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f8713d9e740 1 --2- 172.21.15.18:0/2970886283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 0x55a52c9c32e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f8713d9e740 1 -- 172.21.15.18:0/2970886283 >> 172.21.15.18:0/2970886283 conn(0x55a52c984550 msgr2=0x55a52c9b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f8713d9e740 1 -- 172.21.15.18:0/2970886283 shutdown_connections 2024-02-17T12:50:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f8713d9e740 1 -- 172.21.15.18:0/2970886283 wait complete. 2024-02-17T12:50:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.696+0000 7f8713d9e740 20 client.15828 trim_cache size 0 max 16384 2024-02-17T12:50:47.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:50:47.698 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:50:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.709+0000 7f8713d9e740 1 Processor -- start 2024-02-17T12:50:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.709+0000 7f8713d9e740 1 -- start start 2024-02-17T12:50:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.709+0000 7f8713d9e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bcd60 0x55a52c9c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.709+0000 7f8713d9e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9bdad0 0x55a52c9c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.709+0000 7f8713d9e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9c62e0 0x55a52c9c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.709+0000 7f8713d9e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52c9d6120 con 0x55a52c9c62e0 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.709+0000 7f8713d9e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52c9bc030 con 0x55a52c9bdad0 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.709+0000 7f8713d9e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52c9c8b30 con 0x55a52c9bcd60 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f870a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9c62e0 0x55a52c9c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f870ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9bdad0 0x55a52c9c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f870ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9bdad0 0x55a52c9c3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:43672/0 (socket says 172.21.15.18:43672) 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f870ad9d640 1 -- 172.21.15.18:0/3874497950 learned_addr learned my addr 172.21.15.18:0/3874497950 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f870a59c640 1 --2- 172.21.15.18:0/3874497950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bcd60 0x55a52c9c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f870ad9d640 1 -- 172.21.15.18:0/3874497950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bcd60 msgr2=0x55a52c9c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f870ad9d640 1 --2- 172.21.15.18:0/3874497950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bcd60 0x55a52c9c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f870ad9d640 1 -- 172.21.15.18:0/3874497950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9c62e0 msgr2=0x55a52c9c0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f870ad9d640 1 --2- 172.21.15.18:0/3874497950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9c62e0 0x55a52c9c0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f870ad9d640 1 -- 172.21.15.18:0/3874497950 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a52c8bf7b0 con 0x55a52c9bdad0 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f870a59c640 1 --2- 172.21.15.18:0/3874497950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9c62e0 0x55a52c9c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f870ad9d640 1 --2- 172.21.15.18:0/3874497950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9bdad0 0x55a52c9c3b80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005fd0 tx=0x7f86fc009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe9c88bbb36c094 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f86f0ff9640 1 -- 172.21.15.18:0/3874497950 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc017070 con 0x55a52c9bdad0 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f86f0ff9640 1 -- 172.21.15.18:0/3874497950 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86fc018040 con 0x55a52c9bdad0 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.710+0000 7f86f0ff9640 1 -- 172.21.15.18:0/3874497950 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc010430 con 0x55a52c9bdad0 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.711+0000 7f8713d9e740 1 -- 172.21.15.18:0/3874497950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9bdad0 msgr2=0x55a52c9c3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.711+0000 7f8713d9e740 1 --2- 172.21.15.18:0/3874497950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9bdad0 0x55a52c9c3b80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005fd0 tx=0x7f86fc009500 comp rx=0 tx=0).stop 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.711+0000 7f8713d9e740 1 -- 172.21.15.18:0/3874497950 shutdown_connections 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.711+0000 7f8713d9e740 1 --2- 172.21.15.18:0/3874497950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9c62e0 0x55a52c9c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.711+0000 7f8713d9e740 1 --2- 172.21.15.18:0/3874497950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9bdad0 0x55a52c9c3b80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.711+0000 7f8713d9e740 1 --2- 172.21.15.18:0/3874497950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bcd60 0x55a52c9c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.711+0000 7f8713d9e740 1 -- 172.21.15.18:0/3874497950 >> 172.21.15.18:0/3874497950 conn(0x55a52c984550 msgr2=0x55a52c9c1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.711+0000 7f8713d9e740 1 -- 172.21.15.18:0/3874497950 shutdown_connections 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.711+0000 7f8713d9e740 1 -- 172.21.15.18:0/3874497950 wait complete. 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.711+0000 7f8713d9e740 1 -- start start 2024-02-17T12:50:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.711+0000 7f8713d9e740 1 Processor -- start 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f8713d9e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 0x55a52c9cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f8713d9e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bdad0 0x55a52c9cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f8713d9e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f8713d9e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52ca2c880 con 0x55a52c9bcd60 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f8713d9e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52ca2c9f0 con 0x55a52c9c62e0 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f8713d9e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52ca2cbb0 con 0x55a52c9bdad0 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f8713d9e740 10 client.0 _ll_register_callbacks cb 0x7f8700009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f870a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 0x55a52c9cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f870ad9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bdad0 0x55a52c9cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f870a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 0x55a52c9cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44274/0 (socket says 172.21.15.18:44274) 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f870a59c640 1 -- 172.21.15.18:0/2972249331 learned_addr learned my addr 172.21.15.18:0/2972249331 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f870a59c640 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f870ad9d640 1 -- 172.21.15.18:0/2972249331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 msgr2=0x55a52c9cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f870ad9d640 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f870ad9d640 1 -- 172.21.15.18:0/2972249331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 msgr2=0x55a52c9cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f870ad9d640 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 0x55a52c9cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f870ad9d640 1 -- 172.21.15.18:0/2972249331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a52c922d80 con 0x55a52c9bdad0 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f870a59c640 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f870ad9d640 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bdad0 0x55a52c9cc150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55a52c9c4580 tx=0x7f86fc00ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5adb587101aac775 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f86eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.712+0000 7f86eeff5640 1 -- 172.21.15.18:0/2972249331 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc017070 con 0x55a52c9bdad0 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f86eeff5640 1 -- 172.21.15.18:0/2972249331 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f86fc0046c0 con 0x55a52c9bdad0 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f8713d9e740 1 -- 172.21.15.18:0/2972249331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8700004510 con 0x55a52c9bdad0 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f86eeff5640 1 -- 172.21.15.18:0/2972249331 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86fc004880 con 0x55a52c9bdad0 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f8713d9e740 10 client.25384 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f8713d9e740 1 -- 172.21.15.18:0/2972249331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f86fc00fab0 con 0x55a52c9bdad0 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f86eeff5640 1 -- 172.21.15.18:0/2972249331 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8700004510 con 0x55a52c9bdad0 2024-02-17T12:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f86eeff5640 1 -- 172.21.15.18:0/2972249331 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f86fc03bbb0 con 0x55a52c9bdad0 2024-02-17T12:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f8713d9e740 10 client.25384 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f8713d9e740 10 client.25384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f8713d9e740 20 client.25384 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f8713d9e740 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a52ca2e390 0x55a52ca307b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f8713d9e740 4 client.25384 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f8713d9e740 1 -- 172.21.15.18:0/2972249331 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a52c661550 con 0x55a52ca2e390 2024-02-17T12:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.713+0000 7f870a59c640 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a52ca2e390 0x55a52ca307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.714+0000 7f870a59c640 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a52ca2e390 0x55a52ca307b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0663dddaf39f2d7 server_cookie=ead9515817754e78 in_seq=0 out_seq=0 2024-02-17T12:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.714+0000 7f86eeff5640 10 client.25384 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f86eeff5640 1 -- 172.21.15.18:0/2972249331 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a52c661550 con 0x55a52ca2e390 2024-02-17T12:50:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f86eeff5640 10 client.25384 handle_command_reply: tid=0 2024-02-17T12:50:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f8713d9e740 1 client.25384 shutdown 2024-02-17T12:50:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f8713d9e740 1 -- 172.21.15.18:0/2972249331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bdad0 msgr2=0x55a52c9cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f8713d9e740 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bdad0 0x55a52c9cc150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55a52c9c4580 tx=0x7f86fc00ff80 comp rx=0 tx=0).stop 2024-02-17T12:50:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f8713d9e740 1 -- 172.21.15.18:0/2972249331 shutdown_connections 2024-02-17T12:50:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f8713d9e740 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a52ca2e390 0x55a52ca307b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f8713d9e740 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a52c9c62e0 0x55a52c9cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f86eeff5640 0 client.25384 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f8713d9e740 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a52c9bdad0 0x55a52c9cc150 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f8713d9e740 1 --2- 172.21.15.18:0/2972249331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a52c9bcd60 0x55a52c9cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f8713d9e740 1 -- 172.21.15.18:0/2972249331 >> 172.21.15.18:0/2972249331 conn(0x55a52c984550 msgr2=0x55a52c9b6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f8713d9e740 1 -- 172.21.15.18:0/2972249331 shutdown_connections 2024-02-17T12:50:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f8713d9e740 1 -- 172.21.15.18:0/2972249331 wait complete. 2024-02-17T12:50:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f8713d9e740 20 client.25384 trim_cache size 0 max 16384 2024-02-17T12:50:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:50:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:50:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:50:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:50:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f87111c9640 1 -- 172.21.15.18:0/795914174 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f86f40a23f0 msgr2=0x7f86f40a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f87111c9640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f86f40a23f0 0x7f86f40a48b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f87040cb270 tx=0x7f87040048a0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f87111c9640 1 -- 172.21.15.18:0/795914174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c105040 msgr2=0x7f870c197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.791+0000 7f87111c9640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c105040 0x7f870c197630 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f870400d680 tx=0x7f870400db40 comp rx=0 tx=0).stop 2024-02-17T12:50:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.792+0000 7f87111c9640 1 -- 172.21.15.18:0/795914174 shutdown_connections 2024-02-17T12:50:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.792+0000 7f87111c9640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f86f40a23f0 0x7f86f40a48b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.792+0000 7f87111c9640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c105040 0x7f870c197630 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.792+0000 7f87111c9640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f870c100c70 0x7f870c19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.792+0000 7f87111c9640 1 --2- 172.21.15.18:0/795914174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f870c1002d0 0x7f870c19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.792+0000 7f87111c9640 1 -- 172.21.15.18:0/795914174 >> 172.21.15.18:0/795914174 conn(0x7f870c076ee0 msgr2=0x7f870c069c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.792+0000 7f87111c9640 1 -- 172.21.15.18:0/795914174 shutdown_connections 2024-02-17T12:50:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:50:47.792+0000 7f87111c9640 1 -- 172.21.15.18:0/795914174 wait complete. 2024-02-17T12:50:47.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.834+0000 7f02878a7640 1 -- 172.21.15.18:0/50679138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0280100270 msgr2=0x7f0280100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.834+0000 7f02878a7640 1 --2- 172.21.15.18:0/50679138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0280100270 0x7f0280100650 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f027c009930 tx=0x7f027c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:47.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.834+0000 7f02878a7640 1 -- 172.21.15.18:0/50679138 shutdown_connections 2024-02-17T12:50:47.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.834+0000 7f02878a7640 1 --2- 172.21.15.18:0/50679138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0280105340 0x7f028010cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.834+0000 7f02878a7640 1 --2- 172.21.15.18:0/50679138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0280100c20 0x7f0280104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.834+0000 7f02878a7640 1 --2- 172.21.15.18:0/50679138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0280100270 0x7f0280100650 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.834+0000 7f02878a7640 1 -- 172.21.15.18:0/50679138 >> 172.21.15.18:0/50679138 conn(0x7f0280077230 msgr2=0x7f0280077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:47.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.834+0000 7f02878a7640 1 -- 172.21.15.18:0/50679138 shutdown_connections 2024-02-17T12:50:47.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.834+0000 7f02878a7640 1 -- 172.21.15.18:0/50679138 wait complete. 2024-02-17T12:50:47.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.834+0000 7f02878a7640 1 Processor -- start 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.834+0000 7f02878a7640 1 -- start start 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f02878a7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0280100270 0x7f02801985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f02878a7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0280105340 0x7f0280198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f02878a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02801990c0 0x7f02801a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f02878a7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02801a3d10 con 0x7f0280100270 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f02878a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02801a3e80 con 0x7f02801990c0 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f02878a7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02801a4040 con 0x7f0280105340 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f0285643640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0280100270 0x7f02801985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f0285643640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0280100270 0x7f02801985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44298/0 (socket says 172.21.15.18:44298) 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f0285643640 1 -- 172.21.15.18:0/1329356677 learned_addr learned my addr 172.21.15.18:0/1329356677 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f0284e42640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0280105340 0x7f0280198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f0285e44640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02801990c0 0x7f02801a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f0285643640 1 -- 172.21.15.18:0/1329356677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0280105340 msgr2=0x7f0280198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f0285643640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0280105340 0x7f0280198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f0285643640 1 -- 172.21.15.18:0/1329356677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02801990c0 msgr2=0x7f02801a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f0285643640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02801990c0 0x7f02801a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f0285643640 1 -- 172.21.15.18:0/1329356677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c009560 con 0x7f0280100270 2024-02-17T12:50:47.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f0285643640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0280100270 0x7f02801985f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f027c009230 tx=0x7f027c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:47.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f02767fc640 1 -- 172.21.15.18:0/1329356677 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c004350 con 0x7f0280100270 2024-02-17T12:50:47.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f02767fc640 1 -- 172.21.15.18:0/1329356677 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f027c0044b0 con 0x7f0280100270 2024-02-17T12:50:47.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f02878a7640 1 -- 172.21.15.18:0/1329356677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02801a42c0 con 0x7f0280100270 2024-02-17T12:50:47.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f02767fc640 1 -- 172.21.15.18:0/1329356677 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c003d00 con 0x7f0280100270 2024-02-17T12:50:47.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.835+0000 7f02878a7640 1 -- 172.21.15.18:0/1329356677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02801a4820 con 0x7f0280100270 2024-02-17T12:50:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.838+0000 7f02767fc640 1 -- 172.21.15.18:0/1329356677 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f027c004df0 con 0x7f0280100270 2024-02-17T12:50:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.838+0000 7f028a47e740 1 Processor -- start 2024-02-17T12:50:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.838+0000 7f028a47e740 1 -- start start 2024-02-17T12:50:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.838+0000 7f028a47e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb442b0 0x56368eb44690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.838+0000 7f028a47e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb44bd0 0x56368eb4d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.838+0000 7f028a47e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 0x56368eb4fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.838+0000 7f028a47e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368eab3280 con 0x56368eb4d7b0 2024-02-17T12:50:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.838+0000 7f028a47e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368eab07b0 con 0x56368eb44bd0 2024-02-17T12:50:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.838+0000 7f028a47e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368eaabf90 con 0x56368eb442b0 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.838+0000 7f0284e42640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 0x56368eb4fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.838+0000 7f0285643640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb44bd0 0x56368eb4d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f0284e42640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 0x56368eb4fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44304/0 (socket says 172.21.15.18:44304) 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f0285e44640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb442b0 0x56368eb44690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f0284e42640 1 -- 172.21.15.18:0/4231625449 learned_addr learned my addr 172.21.15.18:0/4231625449 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f02767fc640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f026c0a23d0 0x7f026c0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f0285643640 1 -- 172.21.15.18:0/4231625449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb442b0 msgr2=0x56368eb44690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f0285643640 1 --2- 172.21.15.18:0/4231625449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb442b0 0x56368eb44690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f0285643640 1 -- 172.21.15.18:0/4231625449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 msgr2=0x56368eb4fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f0285643640 1 --2- 172.21.15.18:0/4231625449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 0x56368eb4fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f0285643640 1 -- 172.21.15.18:0/4231625449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56368ea6a570 con 0x56368eb44bd0 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f0284e42640 1 --2- 172.21.15.18:0/4231625449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 0x56368eb4fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f0285e44640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f026c0a23d0 0x7f026c0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f02767fc640 1 -- 172.21.15.18:0/1329356677 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f027c0f0190 con 0x7f0280100270 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f0285643640 1 --2- 172.21.15.18:0/4231625449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb44bd0 0x56368eb4d270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f027c0ed8c0 tx=0x7f027c0b2160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e05b061bee738eb7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f026b7fe640 1 -- 172.21.15.18:0/4231625449 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c0f3d90 con 0x56368eb44bd0 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f026b7fe640 1 -- 172.21.15.18:0/4231625449 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f027c0bc860 con 0x56368eb44bd0 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.839+0000 7f026b7fe640 1 -- 172.21.15.18:0/4231625449 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c0035f0 con 0x56368eb44bd0 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.840+0000 7f028a47e740 1 -- 172.21.15.18:0/4231625449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb44bd0 msgr2=0x56368eb4d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.840+0000 7f0285e44640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f026c0a23d0 0x7f026c0a4890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0278005c40 tx=0x7f0278004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.840+0000 7f028a47e740 1 --2- 172.21.15.18:0/4231625449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb44bd0 0x56368eb4d270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f027c0ed8c0 tx=0x7f027c0b2160 comp rx=0 tx=0).stop 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.840+0000 7f028a47e740 1 -- 172.21.15.18:0/4231625449 shutdown_connections 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.840+0000 7f028a47e740 1 --2- 172.21.15.18:0/4231625449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 0x56368eb4fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.840+0000 7f028a47e740 1 --2- 172.21.15.18:0/4231625449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb44bd0 0x56368eb4d270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.840+0000 7f028a47e740 1 --2- 172.21.15.18:0/4231625449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb442b0 0x56368eb44690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.840+0000 7f028a47e740 1 -- 172.21.15.18:0/4231625449 >> 172.21.15.18:0/4231625449 conn(0x56368eb3e990 msgr2=0x56368eb3ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.840+0000 7f028a47e740 1 -- 172.21.15.18:0/4231625449 shutdown_connections 2024-02-17T12:50:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.840+0000 7f028a47e740 1 -- 172.21.15.18:0/4231625449 wait complete. 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.840+0000 7f028a47e740 1 -- start start 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f028a47e740 1 Processor -- start 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f028a47e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb442b0 0x56368eb4d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f028a47e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb44bd0 0x56368eb4a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f028a47e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 0x56368eb4ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f028a47e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368e77f430 con 0x56368eb4d7b0 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f028a47e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368e73be00 con 0x56368eb442b0 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f028a47e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368ea30fd0 con 0x56368eb44bd0 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f028a47e740 10 client.0 _ll_register_callbacks cb 0x56368ea86fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f0285e44640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 0x56368eb4ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f0285643640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb442b0 0x56368eb4d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f0285e44640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 0x56368eb4ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44318/0 (socket says 172.21.15.18:44318) 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f0285e44640 1 -- 172.21.15.18:0/786667436 learned_addr learned my addr 172.21.15.18:0/786667436 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f0284e42640 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb44bd0 0x56368eb4a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f0285643640 1 -- 172.21.15.18:0/786667436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb44bd0 msgr2=0x56368eb4a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f0285643640 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb44bd0 0x56368eb4a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f0285643640 1 -- 172.21.15.18:0/786667436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 msgr2=0x56368eb4ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f0285643640 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 0x56368eb4ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f0285643640 1 -- 172.21.15.18:0/786667436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02801a4820 con 0x56368eb442b0 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.841+0000 7f0285e44640 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 0x56368eb4ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f0285643640 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb442b0 0x56368eb4d230 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f027c005580 tx=0x7f027c02c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc3ad0d601e1b4b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f0268ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f0268ff9640 1 -- 172.21.15.18:0/786667436 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c0f0030 con 0x56368eb442b0 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f0268ff9640 1 -- 172.21.15.18:0/786667436 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f027c0035f0 con 0x56368eb442b0 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f028a47e740 1 -- 172.21.15.18:0/786667436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56368e7ab7e0 con 0x56368eb442b0 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f0268ff9640 1 -- 172.21.15.18:0/786667436 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f027c0f3d90 con 0x56368eb442b0 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f028a47e740 10 client.25310 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f028a47e740 1 -- 172.21.15.18:0/786667436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56368eaaa610 con 0x56368eb442b0 2024-02-17T12:50:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f0268ff9640 1 -- 172.21.15.18:0/786667436 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56368e7ab7e0 con 0x56368eb442b0 2024-02-17T12:50:47.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f0268ff9640 1 -- 172.21.15.18:0/786667436 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f027c03c070 con 0x56368eb442b0 2024-02-17T12:50:47.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f028a47e740 10 client.25310 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:47.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f028a47e740 10 client.25310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:47.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f028a47e740 20 client.25310 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:47.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f028a47e740 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56368eb5d240 0x56368eb5f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f028a47e740 4 client.25310 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:47.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.842+0000 7f028a47e740 1 -- 172.21.15.18:0/786667436 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56368e7e8720 con 0x56368eb5d240 2024-02-17T12:50:47.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.843+0000 7f0284e42640 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56368eb5d240 0x56368eb5f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.843+0000 7f0268ff9640 10 client.25310 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:47.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.843+0000 7f0284e42640 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56368eb5d240 0x56368eb5f630 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795afbf0233aa9bf server_cookie=402b471b31e6a739 in_seq=0 out_seq=0 2024-02-17T12:50:47.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f0268ff9640 1 -- 172.21.15.18:0/786667436 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56368e7e8720 con 0x56368eb5d240 2024-02-17T12:50:47.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f0268ff9640 10 client.25310 handle_command_reply: tid=0 2024-02-17T12:50:47.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f028a47e740 1 client.25310 shutdown 2024-02-17T12:50:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f028a47e740 1 -- 172.21.15.18:0/786667436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb442b0 msgr2=0x56368eb4d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f028a47e740 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb442b0 0x56368eb4d230 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f027c005580 tx=0x7f027c02c740 comp rx=0 tx=0).stop 2024-02-17T12:50:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f028a47e740 1 -- 172.21.15.18:0/786667436 shutdown_connections 2024-02-17T12:50:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f028a47e740 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56368eb5d240 0x56368eb5f630 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f028a47e740 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb4d7b0 0x56368eb4ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f0268ff9640 0 client.25310 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f028a47e740 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb44bd0 0x56368eb4a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f028a47e740 1 --2- 172.21.15.18:0/786667436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb442b0 0x56368eb4d230 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f028a47e740 1 -- 172.21.15.18:0/786667436 >> 172.21.15.18:0/786667436 conn(0x56368eb3e990 msgr2=0x56368eb40730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f028a47e740 1 -- 172.21.15.18:0/786667436 shutdown_connections 2024-02-17T12:50:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f028a47e740 1 -- 172.21.15.18:0/786667436 wait complete. 2024-02-17T12:50:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.845+0000 7f028a47e740 20 client.25310 trim_cache size 0 max 16384 2024-02-17T12:50:47.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f028a47e740 1 Processor -- start 2024-02-17T12:50:47.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f028a47e740 1 -- start start 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f028a47e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb442b0 0x56368eb4a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f028a47e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb44bd0 0x56368eb4ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f028a47e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb4d7b0 0x56368eb4d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f028a47e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368eb433a0 con 0x56368eb44bd0 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f028a47e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368eb3f9e0 con 0x56368eb442b0 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f028a47e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368eb43cf0 con 0x56368eb4d7b0 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f0284e42640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb44bd0 0x56368eb4ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f0284e42640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb44bd0 0x56368eb4ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44332/0 (socket says 172.21.15.18:44332) 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f0284e42640 1 -- 172.21.15.18:0/1848223587 learned_addr learned my addr 172.21.15.18:0/1848223587 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f0285643640 1 --2- 172.21.15.18:0/1848223587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb442b0 0x56368eb4a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f0285e44640 1 --2- 172.21.15.18:0/1848223587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb4d7b0 0x56368eb4d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f0284e42640 1 -- 172.21.15.18:0/1848223587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb4d7b0 msgr2=0x56368eb4d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f0284e42640 1 --2- 172.21.15.18:0/1848223587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb4d7b0 0x56368eb4d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f0284e42640 1 -- 172.21.15.18:0/1848223587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb442b0 msgr2=0x56368eb4a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f0284e42640 1 --2- 172.21.15.18:0/1848223587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb442b0 0x56368eb4a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f0284e42640 1 -- 172.21.15.18:0/1848223587 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56368ea46ac0 con 0x56368eb44bd0 2024-02-17T12:50:47.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f0285e44640 1 --2- 172.21.15.18:0/1848223587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb4d7b0 0x56368eb4d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:47.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.861+0000 7f0284e42640 1 --2- 172.21.15.18:0/1848223587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb44bd0 0x56368eb4ccf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x56368eb49e20 tx=0x7f0270005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a67aaf6ee759633 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:47.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.862+0000 7f026affd640 1 -- 172.21.15.18:0/1848223587 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270017040 con 0x56368eb44bd0 2024-02-17T12:50:47.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.862+0000 7f026affd640 1 -- 172.21.15.18:0/1848223587 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0270018040 con 0x56368eb44bd0 2024-02-17T12:50:47.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.862+0000 7f026affd640 1 -- 172.21.15.18:0/1848223587 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027001f390 con 0x56368eb44bd0 2024-02-17T12:50:47.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.862+0000 7f028a47e740 1 -- 172.21.15.18:0/1848223587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb44bd0 msgr2=0x56368eb4ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.862+0000 7f028a47e740 1 --2- 172.21.15.18:0/1848223587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb44bd0 0x56368eb4ccf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x56368eb49e20 tx=0x7f0270005f90 comp rx=0 tx=0).stop 2024-02-17T12:50:47.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.862+0000 7f028a47e740 1 -- 172.21.15.18:0/1848223587 shutdown_connections 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.862+0000 7f028a47e740 1 --2- 172.21.15.18:0/1848223587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb4d7b0 0x56368eb4d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.862+0000 7f028a47e740 1 --2- 172.21.15.18:0/1848223587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb44bd0 0x56368eb4ccf0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.862+0000 7f028a47e740 1 --2- 172.21.15.18:0/1848223587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb442b0 0x56368eb4a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.862+0000 7f028a47e740 1 -- 172.21.15.18:0/1848223587 >> 172.21.15.18:0/1848223587 conn(0x56368eb3e990 msgr2=0x56368eb49e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.862+0000 7f028a47e740 1 -- 172.21.15.18:0/1848223587 shutdown_connections 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.862+0000 7f028a47e740 1 -- 172.21.15.18:0/1848223587 wait complete. 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f028a47e740 1 -- start start 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f028a47e740 1 Processor -- start 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f028a47e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb442b0 0x56368eb510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f028a47e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb44bd0 0x56368eb515e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f028a47e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb4d7b0 0x56368eb51b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f028a47e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368ebb37e0 con 0x56368eb442b0 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f028a47e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368ebb3950 con 0x56368eb44bd0 2024-02-17T12:50:47.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f028a47e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368ebb3b10 con 0x56368eb4d7b0 2024-02-17T12:50:47.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f028a47e740 10 client.0 _ll_register_callbacks cb 0x7f027c0f3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:47.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f0285643640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb442b0 0x56368eb510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f0285643640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb442b0 0x56368eb510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44334/0 (socket says 172.21.15.18:44334) 2024-02-17T12:50:47.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f0285643640 1 -- 172.21.15.18:0/3504556908 learned_addr learned my addr 172.21.15.18:0/3504556908 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:47.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f0285e44640 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb4d7b0 0x56368eb51b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f0284e42640 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb44bd0 0x56368eb515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.863+0000 7f0285643640 1 -- 172.21.15.18:0/3504556908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb4d7b0 msgr2=0x56368eb51b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f0285643640 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb4d7b0 0x56368eb51b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f0285643640 1 -- 172.21.15.18:0/3504556908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb44bd0 msgr2=0x56368eb515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f0285643640 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb44bd0 0x56368eb515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f0285643640 1 -- 172.21.15.18:0/3504556908 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0270009b00 con 0x56368eb442b0 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f0285643640 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb442b0 0x56368eb510a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f027c0b8db0 tx=0x7f027c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cea5f2e6a7606e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f0268ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f0268ff9640 1 -- 172.21.15.18:0/3504556908 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c0f3d90 con 0x56368eb442b0 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f0268ff9640 1 -- 172.21.15.18:0/3504556908 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f027c0035f0 con 0x56368eb442b0 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f028a47e740 1 -- 172.21.15.18:0/3504556908 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f027c0bc860 con 0x56368eb442b0 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f0268ff9640 1 -- 172.21.15.18:0/3504556908 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f027c0c0be0 con 0x56368eb442b0 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f028a47e740 10 client.15852 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f028a47e740 1 -- 172.21.15.18:0/3504556908 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56368ea6c6a0 con 0x56368eb442b0 2024-02-17T12:50:47.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f0268ff9640 1 -- 172.21.15.18:0/3504556908 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f027c0bc860 con 0x56368eb442b0 2024-02-17T12:50:47.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f0268ff9640 1 -- 172.21.15.18:0/3504556908 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f027c03c070 con 0x56368eb442b0 2024-02-17T12:50:47.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f028a47e740 10 client.15852 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:47.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f028a47e740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:47.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f028a47e740 20 client.15852 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:47.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f028a47e740 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56368ebb4df0 0x56368ebb71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:47.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f028a47e740 4 client.15852 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:50:47.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.864+0000 7f028a47e740 1 -- 172.21.15.18:0/3504556908 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56368e837890 con 0x56368ebb4df0 2024-02-17T12:50:47.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.865+0000 7f0284e42640 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56368ebb4df0 0x56368ebb71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:47.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.865+0000 7f0268ff9640 10 client.15852 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:47.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.865+0000 7f0284e42640 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56368ebb4df0 0x56368ebb71e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c45663eef92c3 server_cookie=1eae4d70900396ca in_seq=0 out_seq=0 2024-02-17T12:50:47.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f0268ff9640 1 -- 172.21.15.18:0/3504556908 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56368e837890 con 0x56368ebb4df0 2024-02-17T12:50:47.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f0268ff9640 10 client.15852 handle_command_reply: tid=0 2024-02-17T12:50:47.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f028a47e740 1 client.15852 shutdown 2024-02-17T12:50:47.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f028a47e740 1 -- 172.21.15.18:0/3504556908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb442b0 msgr2=0x56368eb510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f028a47e740 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb442b0 0x56368eb510a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f027c0b8db0 tx=0x7f027c009260 comp rx=0 tx=0).stop 2024-02-17T12:50:47.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f028a47e740 1 -- 172.21.15.18:0/3504556908 shutdown_connections 2024-02-17T12:50:47.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f028a47e740 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56368ebb4df0 0x56368ebb71e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f028a47e740 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56368eb4d7b0 0x56368eb51b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f028a47e740 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56368eb44bd0 0x56368eb515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f0268ff9640 0 client.15852 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:47.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f028a47e740 1 --2- 172.21.15.18:0/3504556908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56368eb442b0 0x56368eb510a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f028a47e740 1 -- 172.21.15.18:0/3504556908 >> 172.21.15.18:0/3504556908 conn(0x56368eb3e990 msgr2=0x56368eb48d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:47.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f028a47e740 1 -- 172.21.15.18:0/3504556908 shutdown_connections 2024-02-17T12:50:47.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f028a47e740 1 -- 172.21.15.18:0/3504556908 wait complete. 2024-02-17T12:50:47.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.866+0000 7f028a47e740 20 client.15852 trim_cache size 0 max 16384 2024-02-17T12:50:47.866 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:50:47.867 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:50:47.867 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:50:47.867 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:50:47.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.867+0000 7f02878a7640 1 -- 172.21.15.18:0/1329356677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f026c0a23d0 msgr2=0x7f026c0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.867+0000 7f02878a7640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f026c0a23d0 0x7f026c0a4890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0278005c40 tx=0x7f0278004a40 comp rx=0 tx=0).stop 2024-02-17T12:50:47.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.867+0000 7f02878a7640 1 -- 172.21.15.18:0/1329356677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0280100270 msgr2=0x7f02801985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:47.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.867+0000 7f02878a7640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0280100270 0x7f02801985f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f027c009230 tx=0x7f027c02c650 comp rx=0 tx=0).stop 2024-02-17T12:50:47.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.867+0000 7f02878a7640 1 -- 172.21.15.18:0/1329356677 shutdown_connections 2024-02-17T12:50:47.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.867+0000 7f02878a7640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f026c0a23d0 0x7f026c0a4890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.867+0000 7f02878a7640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02801990c0 0x7f02801a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.867+0000 7f02878a7640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0280105340 0x7f0280198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.867+0000 7f02878a7640 1 --2- 172.21.15.18:0/1329356677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0280100270 0x7f02801985f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:47.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.867+0000 7f02878a7640 1 -- 172.21.15.18:0/1329356677 >> 172.21.15.18:0/1329356677 conn(0x7f0280077230 msgr2=0x7f02800fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:47.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.867+0000 7f02878a7640 1 -- 172.21.15.18:0/1329356677 shutdown_connections 2024-02-17T12:50:47.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:47.867+0000 7f02878a7640 1 -- 172.21.15.18:0/1329356677 wait complete. 2024-02-17T12:50:47.870 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:50:47.870 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:50:48.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.025+0000 7f5e379eb640 1 -- 172.21.15.18:0/417431979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e301053c0 msgr2=0x7f5e3010cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.025+0000 7f5e379eb640 1 --2- 172.21.15.18:0/417431979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e301053c0 0x7f5e3010cf90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009be0 tx=0x7f5e2802bef0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.025+0000 7f5e379eb640 1 -- 172.21.15.18:0/417431979 shutdown_connections 2024-02-17T12:50:48.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.025+0000 7f5e379eb640 1 --2- 172.21.15.18:0/417431979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e301053c0 0x7f5e3010cf90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.025+0000 7f5e379eb640 1 --2- 172.21.15.18:0/417431979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e30100ca0 0x7f5e30104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.025+0000 7f5e379eb640 1 --2- 172.21.15.18:0/417431979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e301002f0 0x7f5e301006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.025+0000 7f5e379eb640 1 -- 172.21.15.18:0/417431979 >> 172.21.15.18:0/417431979 conn(0x7f5e30077180 msgr2=0x7f5e30077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.025+0000 7f5e379eb640 1 -- 172.21.15.18:0/417431979 shutdown_connections 2024-02-17T12:50:48.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.025+0000 7f5e379eb640 1 -- 172.21.15.18:0/417431979 wait complete. 2024-02-17T12:50:48.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e379eb640 1 Processor -- start 2024-02-17T12:50:48.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e379eb640 1 -- start start 2024-02-17T12:50:48.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e379eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e301002f0 0x7f5e3019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e379eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e301053c0 0x7f5e3019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e379eb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e301975e0 0x7f5e30197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e379eb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e30197fd0 con 0x7f5e301053c0 2024-02-17T12:50:48.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e379eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e30198140 con 0x7f5e301002f0 2024-02-17T12:50:48.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e379eb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e30198300 con 0x7f5e301975e0 2024-02-17T12:50:48.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e34f86640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e301053c0 0x7f5e3019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e35787640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e301002f0 0x7f5e3019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e35f88640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e301975e0 0x7f5e30197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e34f86640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e301053c0 0x7f5e3019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44350/0 (socket says 172.21.15.18:44350) 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e35787640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e301002f0 0x7f5e3019d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:43758/0 (socket says 172.21.15.18:43758) 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.026+0000 7f5e34f86640 1 -- 172.21.15.18:0/3426285369 learned_addr learned my addr 172.21.15.18:0/3426285369 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.027+0000 7f5e34f86640 1 -- 172.21.15.18:0/3426285369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e301975e0 msgr2=0x7f5e30197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.027+0000 7f5e34f86640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e301975e0 0x7f5e30197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.027+0000 7f5e34f86640 1 -- 172.21.15.18:0/3426285369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e301002f0 msgr2=0x7f5e3019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.027+0000 7f5e34f86640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e301002f0 0x7f5e3019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.027+0000 7f5e34f86640 1 -- 172.21.15.18:0/3426285369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e20009560 con 0x7f5e301053c0 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.027+0000 7f5e35787640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e301002f0 0x7f5e3019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.027+0000 7f5e34f86640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e301053c0 0x7f5e3019d970 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000e9a0 tx=0x7f5e2000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.027+0000 7f5e267fc640 1 -- 172.21.15.18:0/3426285369 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2000cd80 con 0x7f5e301053c0 2024-02-17T12:50:48.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.027+0000 7f5e379eb640 1 -- 172.21.15.18:0/3426285369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e28009840 con 0x7f5e301053c0 2024-02-17T12:50:48.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.027+0000 7f5e267fc640 1 -- 172.21.15.18:0/3426285369 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e200035f0 con 0x7f5e301053c0 2024-02-17T12:50:48.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.027+0000 7f5e267fc640 1 -- 172.21.15.18:0/3426285369 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20010690 con 0x7f5e301053c0 2024-02-17T12:50:48.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.027+0000 7f5e379eb640 1 -- 172.21.15.18:0/3426285369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e301a4670 con 0x7f5e301053c0 2024-02-17T12:50:48.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.028+0000 7f5e379eb640 1 -- 172.21.15.18:0/3426285369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e30067f70 con 0x7f5e301053c0 2024-02-17T12:50:48.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.029+0000 7f5e267fc640 1 -- 172.21.15.18:0/3426285369 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f5e20003ad0 con 0x7f5e301053c0 2024-02-17T12:50:48.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.030+0000 7f5e267fc640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5e1c0a2350 0x7f5e1c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.030+0000 7f5e35787640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5e1c0a2350 0x7f5e1c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.030+0000 7f5e267fc640 1 -- 172.21.15.18:0/3426285369 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5e20014070 con 0x7f5e301053c0 2024-02-17T12:50:48.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.030+0000 7f5e35787640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5e1c0a2350 0x7f5e1c0a4810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009fc0 tx=0x7f5e2c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.033+0000 7f5e267fc640 1 -- 172.21.15.18:0/3426285369 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e20097220 con 0x7f5e301053c0 2024-02-17T12:50:48.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 10033 23.91 MB/sec warmup 3 sec latency 33.211 ms 2024-02-17T12:50:48.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.276+0000 7f5e379eb640 1 -- 172.21.15.18:0/3426285369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e30198b10 con 0x7f5e301053c0 2024-02-17T12:50:48.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.277+0000 7f5e267fc640 1 -- 172.21.15.18:0/3426285369 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11392 (secure 0 0 0) 0x7f5e20003760 con 0x7f5e301053c0 2024-02-17T12:50:48.269 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:50:48.269 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:49:36.672377+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:50:48.269 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 26 2024-02-17T12:50:48.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.279+0000 7f5e379eb640 1 -- 172.21.15.18:0/3426285369 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5e1c0a2350 msgr2=0x7f5e1c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.279+0000 7f5e379eb640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5e1c0a2350 0x7f5e1c0a4810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009fc0 tx=0x7f5e2c005f90 comp rx=0 tx=0).stop 2024-02-17T12:50:48.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.279+0000 7f5e379eb640 1 -- 172.21.15.18:0/3426285369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e301053c0 msgr2=0x7f5e3019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.279+0000 7f5e379eb640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e301053c0 0x7f5e3019d970 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000e9a0 tx=0x7f5e2000ee60 comp rx=0 tx=0).stop 2024-02-17T12:50:48.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.279+0000 7f5e379eb640 1 -- 172.21.15.18:0/3426285369 shutdown_connections 2024-02-17T12:50:48.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.279+0000 7f5e379eb640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5e1c0a2350 0x7f5e1c0a4810 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.279+0000 7f5e379eb640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e301975e0 0x7f5e30197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.279+0000 7f5e379eb640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e301053c0 0x7f5e3019d970 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.279+0000 7f5e379eb640 1 --2- 172.21.15.18:0/3426285369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e301002f0 0x7f5e3019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.279+0000 7f5e379eb640 1 -- 172.21.15.18:0/3426285369 >> 172.21.15.18:0/3426285369 conn(0x7f5e30077180 msgr2=0x7f5e300fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.279+0000 7f5e379eb640 1 -- 172.21.15.18:0/3426285369 shutdown_connections 2024-02-17T12:50:48.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.279+0000 7f5e379eb640 1 -- 172.21.15.18:0/3426285369 wait complete. 2024-02-17T12:50:48.282 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:50:48.283 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:50:48.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.400+0000 7fb32f7f4640 1 -- 172.21.15.18:0/159093582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb328100c10 msgr2=0x7fb328104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.400+0000 7fb32f7f4640 1 --2- 172.21.15.18:0/159093582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb328100c10 0x7fb328104c50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009930 tx=0x7fb31c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:48.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.400+0000 7fb32f7f4640 1 -- 172.21.15.18:0/159093582 shutdown_connections 2024-02-17T12:50:48.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.400+0000 7fb32f7f4640 1 --2- 172.21.15.18:0/159093582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb328105330 0x7fb32810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.400+0000 7fb32f7f4640 1 --2- 172.21.15.18:0/159093582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb328100c10 0x7fb328104c50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.400+0000 7fb32f7f4640 1 --2- 172.21.15.18:0/159093582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb328100260 0x7fb328100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.400+0000 7fb32f7f4640 1 -- 172.21.15.18:0/159093582 >> 172.21.15.18:0/159093582 conn(0x7fb328077180 msgr2=0x7fb328077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.400+0000 7fb32f7f4640 1 -- 172.21.15.18:0/159093582 shutdown_connections 2024-02-17T12:50:48.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.400+0000 7fb32f7f4640 1 -- 172.21.15.18:0/159093582 wait complete. 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32f7f4640 1 Processor -- start 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32f7f4640 1 -- start start 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32f7f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb328100260 0x7fb3281966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32f7f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb328105330 0x7fb328196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32f7f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb32819af70 0x7fb32819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32f7f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32819b960 con 0x7fb328100260 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32f7f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32819bad0 con 0x7fb328105330 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32f7f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32819bc40 con 0x7fb32819af70 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32d590640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb328100260 0x7fb3281966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32d590640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb328100260 0x7fb3281966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44364/0 (socket says 172.21.15.18:44364) 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32d590640 1 -- 172.21.15.18:0/1616282114 learned_addr learned my addr 172.21.15.18:0/1616282114 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32cd8f640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb328105330 0x7fb328196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32dd91640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb32819af70 0x7fb32819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32d590640 1 -- 172.21.15.18:0/1616282114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb32819af70 msgr2=0x7fb32819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32d590640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb32819af70 0x7fb32819b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32d590640 1 -- 172.21.15.18:0/1616282114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb328105330 msgr2=0x7fb328196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32d590640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb328105330 0x7fb328196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32d590640 1 -- 172.21.15.18:0/1616282114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c009560 con 0x7fb328100260 2024-02-17T12:50:48.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32cd8f640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb328105330 0x7fb328196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:48.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.401+0000 7fb32d590640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb328100260 0x7fb3281966b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb32000e9f0 tx=0x7fb32000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.402+0000 7fb3267fc640 1 -- 172.21.15.18:0/1616282114 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320009700 con 0x7fb328100260 2024-02-17T12:50:48.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.402+0000 7fb3267fc640 1 -- 172.21.15.18:0/1616282114 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb32000cd80 con 0x7fb328100260 2024-02-17T12:50:48.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.402+0000 7fb32f7f4640 1 -- 172.21.15.18:0/1616282114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3281a3220 con 0x7fb328100260 2024-02-17T12:50:48.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.402+0000 7fb3267fc640 1 -- 172.21.15.18:0/1616282114 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320018920 con 0x7fb328100260 2024-02-17T12:50:48.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.402+0000 7fb32f7f4640 1 -- 172.21.15.18:0/1616282114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3281a36a0 con 0x7fb328100260 2024-02-17T12:50:48.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.404+0000 7fb3267fc640 1 -- 172.21.15.18:0/1616282114 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fb320018a80 con 0x7fb328100260 2024-02-17T12:50:48.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.404+0000 7fb3323c6740 1 Processor -- start 2024-02-17T12:50:48.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.404+0000 7fb3323c6740 1 -- start start 2024-02-17T12:50:48.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb3323c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 0x562300806630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb3323c6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806b70 0x56230080f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb3323c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56230080f750 0x562300811b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb3267fc640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb3100a2350 0x7fb3100a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb3323c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623007c1960 con 0x562300806250 2024-02-17T12:50:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb3323c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562300775310 con 0x56230080f750 2024-02-17T12:50:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb3323c6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562300772840 con 0x562300806b70 2024-02-17T12:50:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32dd91640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 0x562300806630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32dd91640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 0x562300806630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44378/0 (socket says 172.21.15.18:44378) 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32dd91640 1 -- 172.21.15.18:0/2286014377 learned_addr learned my addr 172.21.15.18:0/2286014377 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32cd8f640 1 --2- 172.21.15.18:0/2286014377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56230080f750 0x562300811b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32dd91640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb3100a2350 0x7fb3100a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb3267fc640 1 -- 172.21.15.18:0/1616282114 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb320014070 con 0x7fb328100260 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32d590640 1 --2- 172.21.15.18:0/2286014377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806b70 0x56230080f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32dd91640 1 -- 172.21.15.18:0/2286014377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806b70 msgr2=0x56230080f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32dd91640 1 --2- 172.21.15.18:0/2286014377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806b70 0x56230080f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32dd91640 1 -- 172.21.15.18:0/2286014377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56230080f750 msgr2=0x562300811b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32dd91640 1 --2- 172.21.15.18:0/2286014377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56230080f750 0x562300811b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32dd91640 1 -- 172.21.15.18:0/2286014377 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562300730060 con 0x562300806250 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32d590640 1 --2- 172.21.15.18:0/2286014377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806b70 0x56230080f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32cd8f640 1 --2- 172.21.15.18:0/2286014377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56230080f750 0x562300811b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32dd91640 1 --2- 172.21.15.18:0/2286014377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 0x562300806630 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb318013fb0 tx=0x7fb31800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a19e3162ac160e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb32dd91640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb3100a2350 0x7fb3100a4810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb31800fd50 tx=0x7fb318020000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.405+0000 7fb30f7fe640 1 -- 172.21.15.18:0/2286014377 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31800ed60 con 0x562300806250 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.406+0000 7fb30f7fe640 1 -- 172.21.15.18:0/2286014377 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb318009dc0 con 0x562300806250 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.406+0000 7fb30f7fe640 1 -- 172.21.15.18:0/2286014377 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318004430 con 0x562300806250 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.406+0000 7fb3323c6740 1 -- 172.21.15.18:0/2286014377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 msgr2=0x562300806630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.406+0000 7fb3323c6740 1 --2- 172.21.15.18:0/2286014377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 0x562300806630 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb318013fb0 tx=0x7fb31800f850 comp rx=0 tx=0).stop 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.406+0000 7fb3323c6740 1 -- 172.21.15.18:0/2286014377 shutdown_connections 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.406+0000 7fb3323c6740 1 --2- 172.21.15.18:0/2286014377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56230080f750 0x562300811b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.406+0000 7fb3323c6740 1 --2- 172.21.15.18:0/2286014377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806b70 0x56230080f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.406+0000 7fb3323c6740 1 --2- 172.21.15.18:0/2286014377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 0x562300806630 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.406+0000 7fb3323c6740 1 -- 172.21.15.18:0/2286014377 >> 172.21.15.18:0/2286014377 conn(0x562300800980 msgr2=0x562300800d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.406+0000 7fb3323c6740 1 -- 172.21.15.18:0/2286014377 shutdown_connections 2024-02-17T12:50:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.406+0000 7fb3323c6740 1 -- 172.21.15.18:0/2286014377 wait complete. 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb3323c6740 1 -- start start 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb3323c6740 1 Processor -- start 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb3323c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 0x56230080c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb3323c6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806b70 0x56230080cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb3323c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56230080f750 0x56230080d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb3323c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230076e020 con 0x562300806250 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb3323c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562300441430 con 0x56230080f750 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb3323c6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623003fde00 con 0x562300806b70 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb3323c6740 10 client.0 _ll_register_callbacks cb 0x562300715cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb32d590640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 0x56230080c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb32d590640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 0x56230080c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44388/0 (socket says 172.21.15.18:44388) 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb32d590640 1 -- 172.21.15.18:0/1162826594 learned_addr learned my addr 172.21.15.18:0/1162826594 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb32cd8f640 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806b70 0x56230080cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.407+0000 7fb32dd91640 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56230080f750 0x56230080d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb32d590640 1 -- 172.21.15.18:0/1162826594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806b70 msgr2=0x56230080cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb32d590640 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806b70 0x56230080cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb32d590640 1 -- 172.21.15.18:0/1162826594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56230080f750 msgr2=0x56230080d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb32d590640 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56230080f750 0x56230080d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb32d590640 1 -- 172.21.15.18:0/1162826594 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31801b040 con 0x562300806250 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb32d590640 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 0x56230080c7d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb32000e9c0 tx=0x7fb32009b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4495842ed5dd4de3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb30cff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb30cff9640 1 -- 172.21.15.18:0/1162826594 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3200cc190 con 0x562300806250 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb30cff9640 1 -- 172.21.15.18:0/1162826594 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb320010c40 con 0x562300806250 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb3323c6740 1 -- 172.21.15.18:0/1162826594 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56230046d7e0 con 0x562300806250 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb30cff9640 1 -- 172.21.15.18:0/1162826594 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3200aaa30 con 0x562300806250 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb3323c6740 10 client.15870 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb3323c6740 1 -- 172.21.15.18:0/1162826594 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56230076c6a0 con 0x562300806250 2024-02-17T12:50:48.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb30cff9640 1 -- 172.21.15.18:0/1162826594 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56230046d7e0 con 0x562300806250 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb30cff9640 1 -- 172.21.15.18:0/1162826594 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb320099df0 con 0x562300806250 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb3323c6740 10 client.15870 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb3323c6740 10 client.15870 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb3323c6740 20 client.15870 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb3323c6740 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56230081f940 0x562300821d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb3323c6740 4 client.15870 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.408+0000 7fb3323c6740 1 -- 172.21.15.18:0/1162826594 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623004aa720 con 0x56230081f940 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.409+0000 7fb32cd8f640 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56230081f940 0x562300821d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.409+0000 7fb30cff9640 10 client.15870 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.409+0000 7fb32cd8f640 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56230081f940 0x562300821d30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f2d36f9a633e1d60 server_cookie=9710cd6d43fcd0a7 in_seq=0 out_seq=0 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb30cff9640 1 -- 172.21.15.18:0/1162826594 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623004aa720 con 0x56230081f940 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb30cff9640 10 client.15870 handle_command_reply: tid=0 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb3323c6740 1 client.15870 shutdown 2024-02-17T12:50:48.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb3323c6740 1 -- 172.21.15.18:0/1162826594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 msgr2=0x56230080c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb3323c6740 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 0x56230080c7d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb32000e9c0 tx=0x7fb32009b070 comp rx=0 tx=0).stop 2024-02-17T12:50:48.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb3323c6740 1 -- 172.21.15.18:0/1162826594 shutdown_connections 2024-02-17T12:50:48.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb3323c6740 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56230081f940 0x562300821d30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb3323c6740 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56230080f750 0x56230080d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb3323c6740 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806b70 0x56230080cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb30cff9640 0 client.15870 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:50:48.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb3323c6740 1 --2- 172.21.15.18:0/1162826594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806250 0x56230080c7d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb3323c6740 1 -- 172.21.15.18:0/1162826594 >> 172.21.15.18:0/1162826594 conn(0x562300800980 msgr2=0x5623008026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb3323c6740 1 -- 172.21.15.18:0/1162826594 shutdown_connections 2024-02-17T12:50:48.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb3323c6740 1 -- 172.21.15.18:0/1162826594 wait complete. 2024-02-17T12:50:48.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.410+0000 7fb3323c6740 20 client.15870 trim_cache size 0 max 16384 2024-02-17T12:50:48.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb3323c6740 1 Processor -- start 2024-02-17T12:50:48.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb3323c6740 1 -- start start 2024-02-17T12:50:48.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb3323c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562300806250 0x56230080ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb3323c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806b70 0x56230080d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb3323c6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56230080f750 0x562300809a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb3323c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623006f3130 con 0x562300806b70 2024-02-17T12:50:48.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb3323c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230081b550 con 0x562300806250 2024-02-17T12:50:48.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb3323c6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623008069d0 con 0x56230080f750 2024-02-17T12:50:48.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb32cd8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806b70 0x56230080d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb32cd8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806b70 0x56230080d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44404/0 (socket says 172.21.15.18:44404) 2024-02-17T12:50:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb32cd8f640 1 -- 172.21.15.18:0/1369769411 learned_addr learned my addr 172.21.15.18:0/1369769411 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb32d590640 1 --2- 172.21.15.18:0/1369769411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562300806250 0x56230080ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.423+0000 7fb32dd91640 1 --2- 172.21.15.18:0/1369769411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56230080f750 0x562300809a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.424+0000 7fb32cd8f640 1 -- 172.21.15.18:0/1369769411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56230080f750 msgr2=0x562300809a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.424+0000 7fb32cd8f640 1 --2- 172.21.15.18:0/1369769411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56230080f750 0x562300809a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.424+0000 7fb32cd8f640 1 -- 172.21.15.18:0/1369769411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562300806250 msgr2=0x56230080ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.424+0000 7fb32cd8f640 1 --2- 172.21.15.18:0/1369769411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562300806250 0x56230080ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.424+0000 7fb32cd8f640 1 -- 172.21.15.18:0/1369769411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562300708c20 con 0x562300806b70 2024-02-17T12:50:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.424+0000 7fb32cd8f640 1 --2- 172.21.15.18:0/1369769411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806b70 0x56230080d070 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb31c005e00 tx=0x7fb31c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5528b65d7317ed7a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.424+0000 7fb30effd640 1 -- 172.21.15.18:0/1369769411 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c049be0 con 0x562300806b70 2024-02-17T12:50:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.424+0000 7fb30effd640 1 -- 172.21.15.18:0/1369769411 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb31c049d40 con 0x562300806b70 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.424+0000 7fb30effd640 1 -- 172.21.15.18:0/1369769411 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c049700 con 0x562300806b70 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.424+0000 7fb3323c6740 1 -- 172.21.15.18:0/1369769411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806b70 msgr2=0x56230080d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.424+0000 7fb3323c6740 1 --2- 172.21.15.18:0/1369769411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806b70 0x56230080d070 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb31c005e00 tx=0x7fb31c005d90 comp rx=0 tx=0).stop 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 -- 172.21.15.18:0/1369769411 shutdown_connections 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 --2- 172.21.15.18:0/1369769411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56230080f750 0x562300809a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 --2- 172.21.15.18:0/1369769411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562300806b70 0x56230080d070 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 --2- 172.21.15.18:0/1369769411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562300806250 0x56230080ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 -- 172.21.15.18:0/1369769411 >> 172.21.15.18:0/1369769411 conn(0x562300800980 msgr2=0x562300826650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 -- 172.21.15.18:0/1369769411 shutdown_connections 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 -- 172.21.15.18:0/1369769411 wait complete. 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 -- start start 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 Processor -- start 2024-02-17T12:50:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806250 0x562300815880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562300806b70 0x562300815dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56230080f750 0x56230086fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562300870360 con 0x56230080f750 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623008704a0 con 0x562300806b70 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562300870660 con 0x562300806250 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb3323c6740 10 client.0 _ll_register_callbacks cb 0x7fb318013650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb32dd91640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56230080f750 0x56230086fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb32d590640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806250 0x562300815880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb32dd91640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56230080f750 0x56230086fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44418/0 (socket says 172.21.15.18:44418) 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb32dd91640 1 -- 172.21.15.18:0/4292080310 learned_addr learned my addr 172.21.15.18:0/4292080310 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb32d590640 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806250 0x562300815880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34092/0 (socket says 172.21.15.18:34092) 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.425+0000 7fb32cd8f640 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562300806b70 0x562300815dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb32dd91640 1 -- 172.21.15.18:0/4292080310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806250 msgr2=0x562300815880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb32dd91640 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806250 0x562300815880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb32dd91640 1 -- 172.21.15.18:0/4292080310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562300806b70 msgr2=0x562300815dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb32dd91640 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562300806b70 0x562300815dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb32dd91640 1 -- 172.21.15.18:0/4292080310 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31c030070 con 0x56230080f750 2024-02-17T12:50:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb32d590640 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806250 0x562300815880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb32dd91640 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56230080f750 0x56230086fe20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb318003f10 tx=0x7fb318004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f038b87876db140 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb30cff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb30cff9640 1 -- 172.21.15.18:0/4292080310 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318003b70 con 0x56230080f750 2024-02-17T12:50:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb30cff9640 1 -- 172.21.15.18:0/4292080310 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb318003cd0 con 0x56230080f750 2024-02-17T12:50:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb3323c6740 1 -- 172.21.15.18:0/4292080310 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb318009dc0 con 0x56230080f750 2024-02-17T12:50:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb30cff9640 1 -- 172.21.15.18:0/4292080310 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb318004660 con 0x56230080f750 2024-02-17T12:50:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb3323c6740 10 client.15876 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb3323c6740 1 -- 172.21.15.18:0/4292080310 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56230072e8c0 con 0x56230080f750 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb30cff9640 1 -- 172.21.15.18:0/4292080310 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb318009dc0 con 0x56230080f750 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.426+0000 7fb30cff9640 1 -- 172.21.15.18:0/4292080310 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb318039020 con 0x56230080f750 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 10 client.15876 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 10 client.15876 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 20 client.15876 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x562300877e50 0x56230087a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 4 client.15876 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 1 -- 172.21.15.18:0/4292080310 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5623004f9890 con 0x562300877e50 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb32d590640 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x562300877e50 0x56230087a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb30cff9640 10 client.15876 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb32d590640 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x562300877e50 0x56230087a270 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a06c22d7b51c600 server_cookie=853270ded058cffc in_seq=0 out_seq=0 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb30cff9640 1 -- 172.21.15.18:0/4292080310 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5623004f9890 con 0x562300877e50 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb30cff9640 10 client.15876 handle_command_reply: tid=0 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 1 client.15876 shutdown 2024-02-17T12:50:48.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 1 -- 172.21.15.18:0/4292080310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56230080f750 msgr2=0x56230086fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56230080f750 0x56230086fe20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb318003f10 tx=0x7fb318004050 comp rx=0 tx=0).stop 2024-02-17T12:50:48.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 1 -- 172.21.15.18:0/4292080310 shutdown_connections 2024-02-17T12:50:48.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56230080f750 0x56230086fe20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x562300877e50 0x56230087a270 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562300806b70 0x562300815dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb30cff9640 0 client.15876 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:50:48.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 1 --2- 172.21.15.18:0/4292080310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562300806250 0x562300815880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.427+0000 7fb3323c6740 1 -- 172.21.15.18:0/4292080310 >> 172.21.15.18:0/4292080310 conn(0x562300800980 msgr2=0x5623007ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb3323c6740 1 -- 172.21.15.18:0/4292080310 shutdown_connections 2024-02-17T12:50:48.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb3323c6740 1 -- 172.21.15.18:0/4292080310 wait complete. 2024-02-17T12:50:48.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb3323c6740 20 client.15876 trim_cache size 0 max 16384 2024-02-17T12:50:48.427 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:50:48.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb32f7f4640 1 -- 172.21.15.18:0/1616282114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb3100a2350 msgr2=0x7fb3100a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb32f7f4640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb3100a2350 0x7fb3100a4810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb31800fd50 tx=0x7fb318020000 comp rx=0 tx=0).stop 2024-02-17T12:50:48.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb32f7f4640 1 -- 172.21.15.18:0/1616282114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb328100260 msgr2=0x7fb3281966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb32f7f4640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb328100260 0x7fb3281966b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb32000e9f0 tx=0x7fb32000eeb0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb32f7f4640 1 -- 172.21.15.18:0/1616282114 shutdown_connections 2024-02-17T12:50:48.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb32f7f4640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb3100a2350 0x7fb3100a4810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb32f7f4640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb32819af70 0x7fb32819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb32f7f4640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb328105330 0x7fb328196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb32f7f4640 1 --2- 172.21.15.18:0/1616282114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb328100260 0x7fb3281966b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb32f7f4640 1 -- 172.21.15.18:0/1616282114 >> 172.21.15.18:0/1616282114 conn(0x7fb328077180 msgr2=0x7fb3280fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb32f7f4640 1 -- 172.21.15.18:0/1616282114 shutdown_connections 2024-02-17T12:50:48.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.428+0000 7fb32f7f4640 1 -- 172.21.15.18:0/1616282114 wait complete. 2024-02-17T12:50:48.430 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:50:48.431 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:50:48.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.545+0000 7f0a8548d640 1 -- 172.21.15.18:0/1303363973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a80100c10 msgr2=0x7f0a80104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.545+0000 7f0a8548d640 1 --2- 172.21.15.18:0/1303363973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a80100c10 0x7f0a80104c50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009930 tx=0x7f0a6802bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:48.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.545+0000 7f0a8548d640 1 -- 172.21.15.18:0/1303363973 shutdown_connections 2024-02-17T12:50:48.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.545+0000 7f0a8548d640 1 --2- 172.21.15.18:0/1303363973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a80105330 0x7f0a8010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.545+0000 7f0a8548d640 1 --2- 172.21.15.18:0/1303363973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a80100c10 0x7f0a80104c50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.545+0000 7f0a8548d640 1 --2- 172.21.15.18:0/1303363973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a80100260 0x7f0a80100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.545+0000 7f0a8548d640 1 -- 172.21.15.18:0/1303363973 >> 172.21.15.18:0/1303363973 conn(0x7f0a80077180 msgr2=0x7f0a80077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.545+0000 7f0a8548d640 1 -- 172.21.15.18:0/1303363973 shutdown_connections 2024-02-17T12:50:48.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.545+0000 7f0a8548d640 1 -- 172.21.15.18:0/1303363973 wait complete. 2024-02-17T12:50:48.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.545+0000 7f0a8548d640 1 Processor -- start 2024-02-17T12:50:48.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.545+0000 7f0a8548d640 1 -- start start 2024-02-17T12:50:48.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a8548d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a80100260 0x7f0a8019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a8548d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a80105330 0x7f0a8019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a8548d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a80197630 0x7f0a80197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a8548d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80198020 con 0x7f0a80100260 2024-02-17T12:50:48.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a8548d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80198190 con 0x7f0a80197630 2024-02-17T12:50:48.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a8548d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80198350 con 0x7f0a80105330 2024-02-17T12:50:48.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a7effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a80100260 0x7f0a8019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a7effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a80100260 0x7f0a8019d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44432/0 (socket says 172.21.15.18:44432) 2024-02-17T12:50:48.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a7effd640 1 -- 172.21.15.18:0/2052169074 learned_addr learned my addr 172.21.15.18:0/2052169074 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a7f7fe640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a80197630 0x7f0a80197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a7e7fc640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a80105330 0x7f0a8019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a7effd640 1 -- 172.21.15.18:0/2052169074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a80105330 msgr2=0x7f0a8019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a7effd640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a80105330 0x7f0a8019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a7effd640 1 -- 172.21.15.18:0/2052169074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a80197630 msgr2=0x7f0a80197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a7effd640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a80197630 0x7f0a80197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a7effd640 1 -- 172.21.15.18:0/2052169074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a68009560 con 0x7f0a80100260 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a7f7fe640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a80197630 0x7f0a80197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a7effd640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a80100260 0x7f0a8019d430 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800e9f0 tx=0x7f0a7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a77fff640 1 -- 172.21.15.18:0/2052169074 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78009700 con 0x7f0a80100260 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a8548d640 1 -- 172.21.15.18:0/2052169074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a801a42f0 con 0x7f0a80100260 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a77fff640 1 -- 172.21.15.18:0/2052169074 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a7800cd80 con 0x7f0a80100260 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a77fff640 1 -- 172.21.15.18:0/2052169074 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78018920 con 0x7f0a80100260 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.546+0000 7f0a8548d640 1 -- 172.21.15.18:0/2052169074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a801a47d0 con 0x7f0a80100260 2024-02-17T12:50:48.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a77fff640 1 -- 172.21.15.18:0/2052169074 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f0a78018a80 con 0x7f0a80100260 2024-02-17T12:50:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a8805a740 1 Processor -- start 2024-02-17T12:50:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a8805a740 1 -- start start 2024-02-17T12:50:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a8805a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bd93360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a8805a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bda0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a8805a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 0x56513bda2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a8805a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513bd4e960 con 0x56513bda0b60 2024-02-17T12:50:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a8805a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513bd02310 con 0x56513bd938a0 2024-02-17T12:50:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a8805a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513bcff840 con 0x56513bd92f80 2024-02-17T12:50:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a7e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 0x56513bda2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a7f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bd93360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a7effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bda0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a7f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bd93360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34122/0 (socket says 172.21.15.18:34122) 2024-02-17T12:50:48.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a7e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 0x56513bda2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44446/0 (socket says 172.21.15.18:44446) 2024-02-17T12:50:48.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a7f7fe640 1 -- 172.21.15.18:0/1621306661 learned_addr learned my addr 172.21.15.18:0/1621306661 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.549+0000 7f0a77fff640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0a6c0a2350 0x7f0a6c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a7f7fe640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0a6c0a2350 0x7f0a6c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a77fff640 1 -- 172.21.15.18:0/2052169074 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0a78014070 con 0x7f0a80100260 2024-02-17T12:50:48.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a7effd640 1 -- 172.21.15.18:0/1621306661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 msgr2=0x56513bd93360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a7effd640 1 --2- 172.21.15.18:0/1621306661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bd93360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a7effd640 1 -- 172.21.15.18:0/1621306661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 msgr2=0x56513bda2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a7effd640 1 --2- 172.21.15.18:0/1621306661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 0x56513bda2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a7effd640 1 -- 172.21.15.18:0/1621306661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56513bcbd060 con 0x56513bd938a0 2024-02-17T12:50:48.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a7e7fc640 1 --2- 172.21.15.18:0/1621306661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 0x56513bda2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:48.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a7f7fe640 1 --2- 172.21.15.18:0/1621306661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bd93360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:48.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a7effd640 1 --2- 172.21.15.18:0/1621306661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bda0620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0a780cbcb0 tx=0x7f0a78010f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=575fbd75bfa18655 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a7f7fe640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0a6c0a2350 0x7f0a6c0a4810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a700092f0 tx=0x7f0a70004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a757fa640 1 -- 172.21.15.18:0/1621306661 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7809af30 con 0x56513bd938a0 2024-02-17T12:50:48.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a757fa640 1 -- 172.21.15.18:0/1621306661 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a7809efd0 con 0x56513bd938a0 2024-02-17T12:50:48.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.550+0000 7f0a757fa640 1 -- 172.21.15.18:0/1621306661 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a780d1cb0 con 0x56513bd938a0 2024-02-17T12:50:48.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.551+0000 7f0a8805a740 1 -- 172.21.15.18:0/1621306661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 msgr2=0x56513bda0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.551+0000 7f0a8805a740 1 --2- 172.21.15.18:0/1621306661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bda0620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0a780cbcb0 tx=0x7f0a78010f50 comp rx=0 tx=0).stop 2024-02-17T12:50:48.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.551+0000 7f0a8805a740 1 -- 172.21.15.18:0/1621306661 shutdown_connections 2024-02-17T12:50:48.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.551+0000 7f0a8805a740 1 --2- 172.21.15.18:0/1621306661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 0x56513bda2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.551+0000 7f0a8805a740 1 --2- 172.21.15.18:0/1621306661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bda0620 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.551+0000 7f0a8805a740 1 --2- 172.21.15.18:0/1621306661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bd93360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.551+0000 7f0a8805a740 1 -- 172.21.15.18:0/1621306661 >> 172.21.15.18:0/1621306661 conn(0x56513bd8d980 msgr2=0x56513bd8dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.551+0000 7f0a8805a740 1 -- 172.21.15.18:0/1621306661 shutdown_connections 2024-02-17T12:50:48.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.551+0000 7f0a8805a740 1 -- 172.21.15.18:0/1621306661 wait complete. 2024-02-17T12:50:48.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.551+0000 7f0a8805a740 1 -- start start 2024-02-17T12:50:48.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.551+0000 7f0a8805a740 1 Processor -- start 2024-02-17T12:50:48.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a8805a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bd9dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a8805a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bd9e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a8805a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 0x56513bd9e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a8805a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513bcfb020 con 0x56513bda0b60 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a8805a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513b9ce430 con 0x56513bd938a0 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a8805a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513b98ae00 con 0x56513bd92f80 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a8805a740 10 client.0 _ll_register_callbacks cb 0x56513bca2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a7f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 0x56513bd9e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a7f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 0x56513bd9e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44458/0 (socket says 172.21.15.18:44458) 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a7f7fe640 1 -- 172.21.15.18:0/919609581 learned_addr learned my addr 172.21.15.18:0/919609581 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a7e7fc640 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bd9e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a7effd640 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bd9dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a7f7fe640 1 -- 172.21.15.18:0/919609581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 msgr2=0x56513bd9dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a7f7fe640 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bd9dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a7f7fe640 1 -- 172.21.15.18:0/919609581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 msgr2=0x56513bd9e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a7f7fe640 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bd9e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a7f7fe640 1 -- 172.21.15.18:0/919609581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a801a47d0 con 0x56513bda0b60 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a7f7fe640 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 0x56513bd9e660 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0a70005bd0 tx=0x7f0a70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcef61e31c894bf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a667fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a667fc640 1 -- 172.21.15.18:0/919609581 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70004d20 con 0x56513bda0b60 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.552+0000 7f0a8805a740 1 -- 172.21.15.18:0/919609581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56513b9fa7e0 con 0x56513bda0b60 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a667fc640 1 -- 172.21.15.18:0/919609581 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0a70021610 con 0x56513bda0b60 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a667fc640 1 -- 172.21.15.18:0/919609581 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a70004e80 con 0x56513bda0b60 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a8805a740 10 client.15888 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a8805a740 1 -- 172.21.15.18:0/919609581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56513bcf96a0 con 0x56513bda0b60 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a667fc640 1 -- 172.21.15.18:0/919609581 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56513b9fa7e0 con 0x56513bda0b60 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a667fc640 1 -- 172.21.15.18:0/919609581 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0a70027030 con 0x56513bda0b60 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a8805a740 10 client.15888 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:48.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a8805a740 10 client.15888 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:50:48.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a8805a740 20 client.15888 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:48.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a8805a740 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56513bdb0fd0 0x56513bdb33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a8805a740 4 client.15888 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:48.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a8805a740 1 -- 172.21.15.18:0/919609581 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56513ba37720 con 0x56513bdb0fd0 2024-02-17T12:50:48.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.553+0000 7f0a7effd640 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56513bdb0fd0 0x56513bdb33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a667fc640 10 client.15888 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:50:48.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a7effd640 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56513bdb0fd0 0x56513bdb33f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ae15d5017e5216b1 server_cookie=c70684045d7391cb in_seq=0 out_seq=0 2024-02-17T12:50:48.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a667fc640 1 -- 172.21.15.18:0/919609581 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56513ba37720 con 0x56513bdb0fd0 2024-02-17T12:50:48.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a667fc640 10 client.15888 handle_command_reply: tid=0 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a8805a740 1 client.15888 shutdown 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a8805a740 1 -- 172.21.15.18:0/919609581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 msgr2=0x56513bd9e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a8805a740 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 0x56513bd9e660 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0a70005bd0 tx=0x7f0a70004a40 comp rx=0 tx=0).stop 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a8805a740 1 -- 172.21.15.18:0/919609581 shutdown_connections 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a8805a740 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56513bdb0fd0 0x56513bdb33f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a8805a740 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bda0b60 0x56513bd9e660 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a667fc640 0 client.15888 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a8805a740 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bd9e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a8805a740 1 --2- 172.21.15.18:0/919609581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bd9dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a8805a740 1 -- 172.21.15.18:0/919609581 >> 172.21.15.18:0/919609581 conn(0x56513bd8d980 msgr2=0x56513bd8f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.554+0000 7f0a8805a740 1 -- 172.21.15.18:0/919609581 shutdown_connections 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.555+0000 7f0a8805a740 1 -- 172.21.15.18:0/919609581 wait complete. 2024-02-17T12:50:48.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.555+0000 7f0a8805a740 20 client.15888 trim_cache size 0 max 16384 2024-02-17T12:50:48.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.566+0000 7f0a8805a740 1 Processor -- start 2024-02-17T12:50:48.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.566+0000 7f0a8805a740 1 -- start start 2024-02-17T12:50:48.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a8805a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bda0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a8805a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd938a0 0x56513bd9e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a8805a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bda0b60 0x56513bd98ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a8805a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513bc80130 con 0x56513bd938a0 2024-02-17T12:50:48.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a8805a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513bd8e780 con 0x56513bda0b60 2024-02-17T12:50:48.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a8805a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513bdadad0 con 0x56513bd92f80 2024-02-17T12:50:48.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a7e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd938a0 0x56513bd9e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a7e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd938a0 0x56513bd9e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44468/0 (socket says 172.21.15.18:44468) 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a7e7fc640 1 -- 172.21.15.18:0/2076869875 learned_addr learned my addr 172.21.15.18:0/2076869875 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a7effd640 1 --2- 172.21.15.18:0/2076869875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bda0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a7f7fe640 1 --2- 172.21.15.18:0/2076869875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bda0b60 0x56513bd98ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a7e7fc640 1 -- 172.21.15.18:0/2076869875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 msgr2=0x56513bda0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a7e7fc640 1 --2- 172.21.15.18:0/2076869875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bda0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a7e7fc640 1 -- 172.21.15.18:0/2076869875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bda0b60 msgr2=0x56513bd98ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a7e7fc640 1 --2- 172.21.15.18:0/2076869875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bda0b60 0x56513bd98ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a7e7fc640 1 -- 172.21.15.18:0/2076869875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56513bc95c20 con 0x56513bd938a0 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a7e7fc640 1 --2- 172.21.15.18:0/2076869875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd938a0 0x56513bd9e480 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009900 tx=0x7f0a680048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1834ce52050c02e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.567+0000 7f0a74ff9640 1 -- 172.21.15.18:0/2076869875 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68004a60 con 0x56513bd938a0 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.568+0000 7f0a74ff9640 1 -- 172.21.15.18:0/2076869875 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a68004030 con 0x56513bd938a0 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.568+0000 7f0a74ff9640 1 -- 172.21.15.18:0/2076869875 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68005130 con 0x56513bd938a0 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.568+0000 7f0a8805a740 1 -- 172.21.15.18:0/2076869875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd938a0 msgr2=0x56513bd9e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.568+0000 7f0a8805a740 1 --2- 172.21.15.18:0/2076869875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd938a0 0x56513bd9e480 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009900 tx=0x7f0a680048b0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.568+0000 7f0a8805a740 1 -- 172.21.15.18:0/2076869875 shutdown_connections 2024-02-17T12:50:48.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.568+0000 7f0a8805a740 1 --2- 172.21.15.18:0/2076869875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bda0b60 0x56513bd98ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.568+0000 7f0a8805a740 1 --2- 172.21.15.18:0/2076869875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd938a0 0x56513bd9e480 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.568+0000 7f0a8805a740 1 --2- 172.21.15.18:0/2076869875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bd92f80 0x56513bda0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.568+0000 7f0a8805a740 1 -- 172.21.15.18:0/2076869875 >> 172.21.15.18:0/2076869875 conn(0x56513bd8d980 msgr2=0x56513bd8dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.568+0000 7f0a8805a740 1 -- 172.21.15.18:0/2076869875 shutdown_connections 2024-02-17T12:50:48.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.568+0000 7f0a8805a740 1 -- 172.21.15.18:0/2076869875 wait complete. 2024-02-17T12:50:48.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a8805a740 1 -- start start 2024-02-17T12:50:48.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a8805a740 1 Processor -- start 2024-02-17T12:50:48.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a8805a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd92f80 0x56513bda7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a8805a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bda6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a8805a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bda0b60 0x56513bda6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a8805a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513bda6f30 con 0x56513bd92f80 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a8805a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513bda70a0 con 0x56513bd938a0 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a8805a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513be074e0 con 0x56513bda0b60 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a8805a740 10 client.0 _ll_register_callbacks cb 0x7f0a78097000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a7effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd92f80 0x56513bda7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a7effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd92f80 0x56513bda7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44470/0 (socket says 172.21.15.18:44470) 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a7effd640 1 -- 172.21.15.18:0/3222256614 learned_addr learned my addr 172.21.15.18:0/3222256614 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a7e7fc640 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bda6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a7f7fe640 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bda0b60 0x56513bda6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a7effd640 1 -- 172.21.15.18:0/3222256614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bda0b60 msgr2=0x56513bda6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a7effd640 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bda0b60 0x56513bda6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a7effd640 1 -- 172.21.15.18:0/3222256614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 msgr2=0x56513bda6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a7effd640 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bda6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a7effd640 1 -- 172.21.15.18:0/3222256614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a68030070 con 0x56513bd92f80 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a7effd640 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd92f80 0x56513bda7e90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0a780991e0 tx=0x7f0a7801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954cabc5e4125dec server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.569+0000 7f0a667fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a667fc640 1 -- 172.21.15.18:0/3222256614 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a780ce790 con 0x56513bd92f80 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a667fc640 1 -- 172.21.15.18:0/3222256614 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0a78010b80 con 0x56513bd92f80 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a8805a740 1 -- 172.21.15.18:0/3222256614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a7809efd0 con 0x56513bd92f80 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a667fc640 1 -- 172.21.15.18:0/3222256614 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a780971d0 con 0x56513bd92f80 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a8805a740 10 client.15894 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a8805a740 1 -- 172.21.15.18:0/3222256614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56513bcbb8c0 con 0x56513bd92f80 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a667fc640 1 -- 172.21.15.18:0/3222256614 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a78068020 con 0x56513bd92f80 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a667fc640 1 -- 172.21.15.18:0/3222256614 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0a78096c80 con 0x56513bd92f80 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a8805a740 10 client.15894 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:48.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a8805a740 10 client.15894 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:50:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a8805a740 20 client.15894 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a8805a740 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56513be088e0 0x56513be0acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a8805a740 4 client.15894 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:50:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a8805a740 1 -- 172.21.15.18:0/3222256614 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56513ba86890 con 0x56513be088e0 2024-02-17T12:50:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.570+0000 7f0a7e7fc640 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56513be088e0 0x56513be0acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a7e7fc640 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56513be088e0 0x56513be0acd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cdee70868a84be1a server_cookie=ae0d72b6b1994bfa in_seq=0 out_seq=0 2024-02-17T12:50:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a667fc640 10 client.15894 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:50:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a667fc640 1 -- 172.21.15.18:0/3222256614 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56513ba86890 con 0x56513be088e0 2024-02-17T12:50:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a667fc640 10 client.15894 handle_command_reply: tid=0 2024-02-17T12:50:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a8805a740 1 client.15894 shutdown 2024-02-17T12:50:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a8805a740 1 -- 172.21.15.18:0/3222256614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd92f80 msgr2=0x56513bda7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a8805a740 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd92f80 0x56513bda7e90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0a780991e0 tx=0x7f0a7801f000 comp rx=0 tx=0).stop 2024-02-17T12:50:48.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a8805a740 1 -- 172.21.15.18:0/3222256614 shutdown_connections 2024-02-17T12:50:48.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a8805a740 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56513be088e0 0x56513be0acd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a8805a740 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56513bda0b60 0x56513bda6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a8805a740 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56513bd938a0 0x56513bda6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a667fc640 0 client.15894 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:50:48.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a8805a740 1 --2- 172.21.15.18:0/3222256614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56513bd92f80 0x56513bda7e90 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a8805a740 1 -- 172.21.15.18:0/3222256614 >> 172.21.15.18:0/3222256614 conn(0x56513bd8d980 msgr2=0x56513bd98ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a8805a740 1 -- 172.21.15.18:0/3222256614 shutdown_connections 2024-02-17T12:50:48.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a8805a740 1 -- 172.21.15.18:0/3222256614 wait complete. 2024-02-17T12:50:48.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.571+0000 7f0a8805a740 20 client.15894 trim_cache size 0 max 16384 2024-02-17T12:50:48.575 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:50:48.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.572+0000 7f0a8548d640 1 -- 172.21.15.18:0/2052169074 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0a6c0a2350 msgr2=0x7f0a6c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.572+0000 7f0a8548d640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0a6c0a2350 0x7f0a6c0a4810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a700092f0 tx=0x7f0a70004ab0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.572+0000 7f0a8548d640 1 -- 172.21.15.18:0/2052169074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a80100260 msgr2=0x7f0a8019d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.572+0000 7f0a8548d640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a80100260 0x7f0a8019d430 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800e9f0 tx=0x7f0a7800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.572+0000 7f0a8548d640 1 -- 172.21.15.18:0/2052169074 shutdown_connections 2024-02-17T12:50:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.572+0000 7f0a8548d640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0a6c0a2350 0x7f0a6c0a4810 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.572+0000 7f0a8548d640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a80197630 0x7f0a80197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.572+0000 7f0a8548d640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a80105330 0x7f0a8019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.572+0000 7f0a8548d640 1 --2- 172.21.15.18:0/2052169074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a80100260 0x7f0a8019d430 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.572+0000 7f0a8548d640 1 -- 172.21.15.18:0/2052169074 >> 172.21.15.18:0/2052169074 conn(0x7f0a80077180 msgr2=0x7f0a800fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.572+0000 7f0a8548d640 1 -- 172.21.15.18:0/2052169074 shutdown_connections 2024-02-17T12:50:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.572+0000 7f0a8548d640 1 -- 172.21.15.18:0/2052169074 wait complete. 2024-02-17T12:50:48.577 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:48 smithi018 ceph-mon[32625]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-02-17T12:50:48.577 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:48 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3426285369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:50:48.578 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:50:48.578 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:50:48.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.694+0000 7ff4d725b640 1 -- 172.21.15.18:0/3177017440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d0101d30 msgr2=0x7ff4d0105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.694+0000 7ff4d725b640 1 --2- 172.21.15.18:0/3177017440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d0101d30 0x7ff4d0105d70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009930 tx=0x7ff4c402bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:48.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.694+0000 7ff4d725b640 1 -- 172.21.15.18:0/3177017440 shutdown_connections 2024-02-17T12:50:48.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.694+0000 7ff4d725b640 1 --2- 172.21.15.18:0/3177017440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4d0106450 0x7ff4d010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.694+0000 7ff4d725b640 1 --2- 172.21.15.18:0/3177017440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d0101d30 0x7ff4d0105d70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.694+0000 7ff4d725b640 1 --2- 172.21.15.18:0/3177017440 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4d0101380 0x7ff4d0101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.694+0000 7ff4d725b640 1 -- 172.21.15.18:0/3177017440 >> 172.21.15.18:0/3177017440 conn(0x7ff4d0078580 msgr2=0x7ff4d0078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.694+0000 7ff4d725b640 1 -- 172.21.15.18:0/3177017440 shutdown_connections 2024-02-17T12:50:48.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.694+0000 7ff4d725b640 1 -- 172.21.15.18:0/3177017440 wait complete. 2024-02-17T12:50:48.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4d725b640 1 Processor -- start 2024-02-17T12:50:48.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4d725b640 1 -- start start 2024-02-17T12:50:48.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4d725b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4d0101380 0x7ff4d019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4d725b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d0106450 0x7ff4d019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4d725b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4d0197630 0x7ff4d0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4d725b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0198100 con 0x7ff4d0106450 2024-02-17T12:50:48.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4d725b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0198270 con 0x7ff4d0197630 2024-02-17T12:50:48.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4d725b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0198430 con 0x7ff4d0101380 2024-02-17T12:50:48.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4cffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d0106450 0x7ff4d019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4cffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d0106450 0x7ff4d019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44494/0 (socket says 172.21.15.18:44494) 2024-02-17T12:50:48.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4cffff640 1 -- 172.21.15.18:0/699214382 learned_addr learned my addr 172.21.15.18:0/699214382 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4d57f8640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4d0197630 0x7ff4d0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4d4ff7640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4d0101380 0x7ff4d019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4cffff640 1 -- 172.21.15.18:0/699214382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4d0101380 msgr2=0x7ff4d019d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4cffff640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4d0101380 0x7ff4d019d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4cffff640 1 -- 172.21.15.18:0/699214382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4d0197630 msgr2=0x7ff4d0197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4cffff640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4d0197630 0x7ff4d0197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.695+0000 7ff4cffff640 1 -- 172.21.15.18:0/699214382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4009560 con 0x7ff4d0106450 2024-02-17T12:50:48.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.696+0000 7ff4cffff640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d0106450 0x7ff4d019dab0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009230 tx=0x7ff4c402c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.696+0000 7ff4cdffb640 1 -- 172.21.15.18:0/699214382 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4004350 con 0x7ff4d0106450 2024-02-17T12:50:48.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.696+0000 7ff4cdffb640 1 -- 172.21.15.18:0/699214382 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4c40044b0 con 0x7ff4d0106450 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.696+0000 7ff4d725b640 1 -- 172.21.15.18:0/699214382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d01a4370 con 0x7ff4d0106450 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.696+0000 7ff4cdffb640 1 -- 172.21.15.18:0/699214382 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4003da0 con 0x7ff4d0106450 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.696+0000 7ff4d725b640 1 -- 172.21.15.18:0/699214382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d01a4850 con 0x7ff4d0106450 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.698+0000 7ff4cdffb640 1 -- 172.21.15.18:0/699214382 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7ff4c4004df0 con 0x7ff4d0106450 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.698+0000 7ff4d9e2a740 1 Processor -- start 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.698+0000 7ff4d9e2a740 1 -- start start 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d9e2a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11ad660 0x55aeb11ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d9e2a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11adf80 0x55aeb11b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d9e2a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11b8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d9e2a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb1164960 con 0x55aeb11b6b60 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d9e2a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb1118310 con 0x55aeb11ad660 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d9e2a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb1115840 con 0x55aeb11adf80 2024-02-17T12:50:48.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4cdffb640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4b80a2350 0x7ff4b80a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4cffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11b8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d4ff7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11adf80 0x55aeb11b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d57f8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11ad660 0x55aeb11ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d4ff7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11adf80 0x55aeb11b6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34204/0 (socket says 172.21.15.18:34204) 2024-02-17T12:50:48.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d4ff7640 1 -- 172.21.15.18:0/719590599 learned_addr learned my addr 172.21.15.18:0/719590599 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d57f8640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4b80a2350 0x7ff4b80a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4cdffb640 1 -- 172.21.15.18:0/699214382 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff4c40f02b0 con 0x7ff4d0106450 2024-02-17T12:50:48.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d4ff7640 1 -- 172.21.15.18:0/719590599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11ad660 msgr2=0x55aeb11ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d4ff7640 1 --2- 172.21.15.18:0/719590599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11ad660 0x55aeb11ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d4ff7640 1 -- 172.21.15.18:0/719590599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 msgr2=0x55aeb11b8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d4ff7640 1 --2- 172.21.15.18:0/719590599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11b8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d4ff7640 1 -- 172.21.15.18:0/719590599 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeb10d3060 con 0x55aeb11adf80 2024-02-17T12:50:48.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d57f8640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4b80a2350 0x7ff4b80a4810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005f60 tx=0x7ff4c0005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.699+0000 7ff4d4ff7640 1 --2- 172.21.15.18:0/719590599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11adf80 0x55aeb11b6620 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80114f0 tx=0x7ff4c80118f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc163afa3e9f10a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.700+0000 7ff4b6ffd640 1 -- 172.21.15.18:0/719590599 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8004d20 con 0x55aeb11adf80 2024-02-17T12:50:48.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.700+0000 7ff4b6ffd640 1 -- 172.21.15.18:0/719590599 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4c8004e80 con 0x55aeb11adf80 2024-02-17T12:50:48.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.700+0000 7ff4b6ffd640 1 -- 172.21.15.18:0/719590599 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8005280 con 0x55aeb11adf80 2024-02-17T12:50:48.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.700+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/719590599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11adf80 msgr2=0x55aeb11b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.700+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/719590599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11adf80 0x55aeb11b6620 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80114f0 tx=0x7ff4c80118f0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.700+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/719590599 shutdown_connections 2024-02-17T12:50:48.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.700+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/719590599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11b8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.700+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/719590599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11adf80 0x55aeb11b6620 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.700+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/719590599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11ad660 0x55aeb11ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.700+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/719590599 >> 172.21.15.18:0/719590599 conn(0x55aeb11a7d90 msgr2=0x55aeb11a8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.700+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/719590599 shutdown_connections 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.700+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/719590599 wait complete. 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d9e2a740 1 -- start start 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d9e2a740 1 Processor -- start 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d9e2a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 0x55aeb11b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d9e2a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11adf80 0x55aeb11b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d9e2a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d9e2a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb1111020 con 0x55aeb11b6b60 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d9e2a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb0de4430 con 0x55aeb11adf80 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d9e2a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb0da0e00 con 0x55aeb11ad660 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d9e2a740 10 client.0 _ll_register_callbacks cb 0x55aeb10b8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d57f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4cffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11adf80 0x55aeb11b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d57f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44524/0 (socket says 172.21.15.18:44524) 2024-02-17T12:50:48.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d57f8640 1 -- 172.21.15.18:0/4020690744 learned_addr learned my addr 172.21.15.18:0/4020690744 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.701+0000 7ff4d4ff7640 1 --2- 172.21.15.18:0/4020690744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 0x55aeb11b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4d57f8640 1 -- 172.21.15.18:0/4020690744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 msgr2=0x55aeb11b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4d57f8640 1 --2- 172.21.15.18:0/4020690744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 0x55aeb11b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4d57f8640 1 -- 172.21.15.18:0/4020690744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11adf80 msgr2=0x55aeb11b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4d57f8640 1 --2- 172.21.15.18:0/4020690744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11adf80 0x55aeb11b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4d57f8640 1 -- 172.21.15.18:0/4020690744 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c8009c80 con 0x55aeb11b6b60 2024-02-17T12:50:48.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4d57f8640 1 --2- 172.21.15.18:0/4020690744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11b4570 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0011e10 tx=0x7ff4c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1886c2f90cba21c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4affff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:48.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4affff640 1 -- 172.21.15.18:0/4020690744 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0005220 con 0x55aeb11b6b60 2024-02-17T12:50:48.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/4020690744 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aeb0e4d720 con 0x55aeb11b6b60 2024-02-17T12:50:48.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4affff640 1 -- 172.21.15.18:0/4020690744 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff4c0011e40 con 0x55aeb11b6b60 2024-02-17T12:50:48.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4affff640 1 -- 172.21.15.18:0/4020690744 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c0005380 con 0x55aeb11b6b60 2024-02-17T12:50:48.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4d9e2a740 10 client.15906 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:48.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/4020690744 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55aeb110f6a0 con 0x55aeb11b6b60 2024-02-17T12:50:48.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4affff640 1 -- 172.21.15.18:0/4020690744 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aeb0e4d720 con 0x55aeb11b6b60 2024-02-17T12:50:48.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4affff640 1 -- 172.21.15.18:0/4020690744 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff4c0003a60 con 0x55aeb11b6b60 2024-02-17T12:50:48.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4d9e2a740 10 client.15906 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:48.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.702+0000 7ff4d9e2a740 10 client.15906 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:50:48.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.703+0000 7ff4d9e2a740 20 client.15906 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:48.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.703+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/4020690744 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55aeb11c6ec0 0x55aeb11c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.703+0000 7ff4d9e2a740 4 client.15906 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:48.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.703+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/4020690744 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aeb0e9c890 con 0x55aeb11c6ec0 2024-02-17T12:50:48.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.703+0000 7ff4d4ff7640 1 --2- 172.21.15.18:0/4020690744 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55aeb11c6ec0 0x55aeb11c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.703+0000 7ff4d4ff7640 1 --2- 172.21.15.18:0/4020690744 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55aeb11c6ec0 0x55aeb11c92e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4025b9c5d8f90bfc server_cookie=44df9a5cd4a1417b in_seq=0 out_seq=0 2024-02-17T12:50:48.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.703+0000 7ff4affff640 10 client.15906 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:50:48.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4affff640 1 -- 172.21.15.18:0/4020690744 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aeb0e9c890 con 0x55aeb11c6ec0 2024-02-17T12:50:48.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4affff640 10 client.15906 handle_command_reply: tid=0 2024-02-17T12:50:48.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4d9e2a740 1 client.15906 shutdown 2024-02-17T12:50:48.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/4020690744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 msgr2=0x55aeb11b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/4020690744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11b4570 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0011e10 tx=0x7ff4c00046c0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/4020690744 shutdown_connections 2024-02-17T12:50:48.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/4020690744 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55aeb11c6ec0 0x55aeb11c92e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/4020690744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11b4570 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/4020690744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11adf80 0x55aeb11b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4affff640 0 client.15906 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:50:48.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/4020690744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 0x55aeb11b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/4020690744 >> 172.21.15.18:0/4020690744 conn(0x55aeb11a7d90 msgr2=0x55aeb11a9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/4020690744 shutdown_connections 2024-02-17T12:50:48.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/4020690744 wait complete. 2024-02-17T12:50:48.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.704+0000 7ff4d9e2a740 20 client.15906 trim_cache size 0 max 16384 2024-02-17T12:50:48.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.716+0000 7ff4d9e2a740 1 Processor -- start 2024-02-17T12:50:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.716+0000 7ff4d9e2a740 1 -- start start 2024-02-17T12:50:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.716+0000 7ff4d9e2a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 0x55aeb11b4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.716+0000 7ff4d9e2a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11adf80 0x55aeb11bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.716+0000 7ff4d9e2a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11b6b60 0x55aeb11bdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.716+0000 7ff4d9e2a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb1096130 con 0x55aeb11adf80 2024-02-17T12:50:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.716+0000 7ff4d9e2a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb11a7640 con 0x55aeb11b6b60 2024-02-17T12:50:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.716+0000 7ff4d9e2a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb119ea10 con 0x55aeb11ad660 2024-02-17T12:50:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4cffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11adf80 0x55aeb11bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4cffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11adf80 0x55aeb11bda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44528/0 (socket says 172.21.15.18:44528) 2024-02-17T12:50:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4cffff640 1 -- 172.21.15.18:0/3720790364 learned_addr learned my addr 172.21.15.18:0/3720790364 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4d4ff7640 1 --2- 172.21.15.18:0/3720790364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 0x55aeb11b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4d57f8640 1 --2- 172.21.15.18:0/3720790364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11b6b60 0x55aeb11bdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4cffff640 1 -- 172.21.15.18:0/3720790364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 msgr2=0x55aeb11b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4cffff640 1 --2- 172.21.15.18:0/3720790364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 0x55aeb11b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4cffff640 1 -- 172.21.15.18:0/3720790364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11b6b60 msgr2=0x55aeb11bdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4cffff640 1 --2- 172.21.15.18:0/3720790364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11b6b60 0x55aeb11bdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4cffff640 1 -- 172.21.15.18:0/3720790364 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeb10abc20 con 0x55aeb11adf80 2024-02-17T12:50:48.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4cffff640 1 --2- 172.21.15.18:0/3720790364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11adf80 0x55aeb11bda10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005780 tx=0x7ff4c404a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb10ac357af77bc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4b67fc640 1 -- 172.21.15.18:0/3720790364 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c40bcbb0 con 0x55aeb11adf80 2024-02-17T12:50:48.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4b67fc640 1 -- 172.21.15.18:0/3720790364 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4c40c0d00 con 0x55aeb11adf80 2024-02-17T12:50:48.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.717+0000 7ff4b67fc640 1 -- 172.21.15.18:0/3720790364 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c40c5120 con 0x55aeb11adf80 2024-02-17T12:50:48.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.718+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3720790364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11adf80 msgr2=0x55aeb11bda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.718+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/3720790364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11adf80 0x55aeb11bda10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005780 tx=0x7ff4c404a040 comp rx=0 tx=0).stop 2024-02-17T12:50:48.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.718+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3720790364 shutdown_connections 2024-02-17T12:50:48.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.718+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/3720790364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11b6b60 0x55aeb11bdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.718+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/3720790364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11adf80 0x55aeb11bda10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.718+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/3720790364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 0x55aeb11b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.718+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3720790364 >> 172.21.15.18:0/3720790364 conn(0x55aeb11a7d90 msgr2=0x55aeb11aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.718+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3720790364 shutdown_connections 2024-02-17T12:50:48.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.718+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3720790364 wait complete. 2024-02-17T12:50:48.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.718+0000 7ff4d9e2a740 1 -- start start 2024-02-17T12:50:48.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.718+0000 7ff4d9e2a740 1 Processor -- start 2024-02-17T12:50:48.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d9e2a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 0x55aeb11bc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d9e2a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11adf80 0x55aeb11bca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d9e2a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11bcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d9e2a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb121d3f0 con 0x55aeb11b6b60 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d9e2a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb121d530 con 0x55aeb11adf80 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d9e2a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb121d6f0 con 0x55aeb11ad660 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d9e2a740 10 client.0 _ll_register_callbacks cb 0x7ff4c8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d57f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11bcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d57f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11bcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44530/0 (socket says 172.21.15.18:44530) 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d57f8640 1 -- 172.21.15.18:0/3218511953 learned_addr learned my addr 172.21.15.18:0/3218511953 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d4ff7640 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 0x55aeb11bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4cffff640 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11adf80 0x55aeb11bca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d57f8640 1 -- 172.21.15.18:0/3218511953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 msgr2=0x55aeb11bc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d57f8640 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 0x55aeb11bc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d57f8640 1 -- 172.21.15.18:0/3218511953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11adf80 msgr2=0x55aeb11bca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d57f8640 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11adf80 0x55aeb11bca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d57f8640 1 -- 172.21.15.18:0/3218511953 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d01a4850 con 0x55aeb11b6b60 2024-02-17T12:50:48.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d57f8640 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11bcfc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0011e10 tx=0x7ff4c0003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6d1a5f4208dbe4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4affff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4affff640 1 -- 172.21.15.18:0/3218511953 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0011680 con 0x55aeb11b6b60 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4affff640 1 -- 172.21.15.18:0/3218511953 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff4c0011e40 con 0x55aeb11b6b60 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3218511953 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aeb0e107e0 con 0x55aeb11b6b60 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4affff640 1 -- 172.21.15.18:0/3218511953 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c000fc30 con 0x55aeb11b6b60 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d9e2a740 10 client.15912 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.719+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3218511953 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55aeb10d18c0 con 0x55aeb11b6b60 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.720+0000 7ff4affff640 1 -- 172.21.15.18:0/3218511953 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aeb0e107e0 con 0x55aeb11b6b60 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.720+0000 7ff4affff640 1 -- 172.21.15.18:0/3218511953 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff4c00465c0 con 0x55aeb11b6b60 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.720+0000 7ff4d9e2a740 10 client.15912 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.720+0000 7ff4d9e2a740 10 client.15912 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.720+0000 7ff4d9e2a740 20 client.15912 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.720+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55aeb121ee30 0x55aeb1221250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.720+0000 7ff4d9e2a740 4 client.15912 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:50:48.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.720+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3218511953 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aeb0dbc700 con 0x55aeb121ee30 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.720+0000 7ff4d4ff7640 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55aeb121ee30 0x55aeb1221250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.720+0000 7ff4affff640 10 client.15912 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.720+0000 7ff4d4ff7640 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55aeb121ee30 0x55aeb1221250 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f1a8ac732b0a623 server_cookie=1d7485bd241f0125 in_seq=0 out_seq=0 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4affff640 1 -- 172.21.15.18:0/3218511953 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aeb0dbc700 con 0x55aeb121ee30 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4affff640 10 client.15912 handle_command_reply: tid=0 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d9e2a740 1 client.15912 shutdown 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3218511953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 msgr2=0x55aeb11bcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11bcfc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0011e10 tx=0x7ff4c0003c30 comp rx=0 tx=0).stop 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3218511953 shutdown_connections 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55aeb121ee30 0x55aeb1221250 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aeb11b6b60 0x55aeb11bcfc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aeb11adf80 0x55aeb11bca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4affff640 0 client.15912 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:50:48.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d9e2a740 1 --2- 172.21.15.18:0/3218511953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aeb11ad660 0x55aeb11bc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3218511953 >> 172.21.15.18:0/3218511953 conn(0x55aeb11a7d90 msgr2=0x55aeb11a22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3218511953 shutdown_connections 2024-02-17T12:50:48.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d9e2a740 1 -- 172.21.15.18:0/3218511953 wait complete. 2024-02-17T12:50:48.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d9e2a740 20 client.15912 trim_cache size 0 max 16384 2024-02-17T12:50:48.729 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:50:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d725b640 1 -- 172.21.15.18:0/699214382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4b80a2350 msgr2=0x7ff4b80a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d725b640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4b80a2350 0x7ff4b80a4810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005f60 tx=0x7ff4c0005f90 comp rx=0 tx=0).stop 2024-02-17T12:50:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d725b640 1 -- 172.21.15.18:0/699214382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d0106450 msgr2=0x7ff4d019dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.721+0000 7ff4d725b640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d0106450 0x7ff4d019dab0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009230 tx=0x7ff4c402c680 comp rx=0 tx=0).stop 2024-02-17T12:50:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.722+0000 7ff4d725b640 1 -- 172.21.15.18:0/699214382 shutdown_connections 2024-02-17T12:50:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.722+0000 7ff4d725b640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4b80a2350 0x7ff4b80a4810 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.722+0000 7ff4d725b640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4d0197630 0x7ff4d0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.722+0000 7ff4d725b640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d0106450 0x7ff4d019dab0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.722+0000 7ff4d725b640 1 --2- 172.21.15.18:0/699214382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4d0101380 0x7ff4d019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.722+0000 7ff4d725b640 1 -- 172.21.15.18:0/699214382 >> 172.21.15.18:0/699214382 conn(0x7ff4d0078580 msgr2=0x7ff4d00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.722+0000 7ff4d725b640 1 -- 172.21.15.18:0/699214382 shutdown_connections 2024-02-17T12:50:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.722+0000 7ff4d725b640 1 -- 172.21.15.18:0/699214382 wait complete. 2024-02-17T12:50:48.732 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:50:48.732 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:50:48.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:48 smithi119 ceph-mon[36188]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-02-17T12:50:48.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:48 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3426285369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:50:48.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.857+0000 7f24417a0640 1 -- 172.21.15.18:0/4052307192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f243c101d40 msgr2=0x7f243c105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.857+0000 7f24417a0640 1 --2- 172.21.15.18:0/4052307192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f243c101d40 0x7f243c105d80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2430009930 tx=0x7f243002bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:48.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.857+0000 7f24417a0640 1 -- 172.21.15.18:0/4052307192 shutdown_connections 2024-02-17T12:50:48.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.857+0000 7f24417a0640 1 --2- 172.21.15.18:0/4052307192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f243c106460 0x7f243c10e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.857+0000 7f24417a0640 1 --2- 172.21.15.18:0/4052307192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f243c101d40 0x7f243c105d80 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.857+0000 7f24417a0640 1 --2- 172.21.15.18:0/4052307192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f243c101390 0x7f243c101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.858+0000 7f24417a0640 1 -- 172.21.15.18:0/4052307192 >> 172.21.15.18:0/4052307192 conn(0x7f243c0fcc10 msgr2=0x7f243c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.858+0000 7f24417a0640 1 -- 172.21.15.18:0/4052307192 shutdown_connections 2024-02-17T12:50:48.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.858+0000 7f24417a0640 1 -- 172.21.15.18:0/4052307192 wait complete. 2024-02-17T12:50:48.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.858+0000 7f24417a0640 1 Processor -- start 2024-02-17T12:50:48.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.858+0000 7f24417a0640 1 -- start start 2024-02-17T12:50:48.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.858+0000 7f24417a0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f243c101390 0x7f243c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.858+0000 7f24417a0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f243c106460 0x7f243c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.858+0000 7f24417a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f243c197680 0x7f243c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.858+0000 7f24417a0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c198150 con 0x7f243c101390 2024-02-17T12:50:48.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.858+0000 7f24417a0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c1982c0 con 0x7f243c197680 2024-02-17T12:50:48.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.858+0000 7f24417a0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c198480 con 0x7f243c106460 2024-02-17T12:50:48.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f243affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f243c101390 0x7f243c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f243affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f243c101390 0x7f243c19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44544/0 (socket says 172.21.15.18:44544) 2024-02-17T12:50:48.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f243affd640 1 -- 172.21.15.18:0/1495155107 learned_addr learned my addr 172.21.15.18:0/1495155107 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f243b7fe640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f243c197680 0x7f243c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f243a7fc640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f243c106460 0x7f243c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f243affd640 1 -- 172.21.15.18:0/1495155107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f243c106460 msgr2=0x7f243c19db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f243affd640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f243c106460 0x7f243c19db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f243affd640 1 -- 172.21.15.18:0/1495155107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f243c197680 msgr2=0x7f243c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f243affd640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f243c197680 0x7f243c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f243affd640 1 -- 172.21.15.18:0/1495155107 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430009560 con 0x7f243c101390 2024-02-17T12:50:48.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f243affd640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f243c101390 0x7f243c19d5c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f242c00d950 tx=0x7f242c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f2423fff640 1 -- 172.21.15.18:0/1495155107 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c009a30 con 0x7f243c101390 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f2423fff640 1 -- 172.21.15.18:0/1495155107 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f242c009b90 con 0x7f243c101390 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f24417a0640 1 -- 172.21.15.18:0/1495155107 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243c1a4420 con 0x7f243c101390 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f2423fff640 1 -- 172.21.15.18:0/1495155107 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c004a00 con 0x7f243c101390 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.859+0000 7f24417a0640 1 -- 172.21.15.18:0/1495155107 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c1a4900 con 0x7f243c101390 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f2423fff640 1 -- 172.21.15.18:0/1495155107 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f242c009d00 con 0x7f243c101390 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f2444370740 1 Processor -- start 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f2444370740 1 -- start start 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f2423fff640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f24240a2350 0x7f24240a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f2444370740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a2c30 0x5592fb2a3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f2444370740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2a35e0 0x5592fb2b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f2444370740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 0x5592fb2b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f2444370740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fb25e960 con 0x5592fb2b0b60 2024-02-17T12:50:48.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f2444370740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fb212310 con 0x5592fb2a2c30 2024-02-17T12:50:48.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f2444370740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fb20f840 con 0x5592fb2a35e0 2024-02-17T12:50:48.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f243b7fe640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f24240a2350 0x7f24240a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f243affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a2c30 0x5592fb2a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f243a7fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2a35e0 0x5592fb2b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f243b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 0x5592fb2b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f243a7fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2a35e0 0x5592fb2b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34252/0 (socket says 172.21.15.18:34252) 2024-02-17T12:50:48.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f243a7fc640 1 -- 172.21.15.18:0/1390364949 learned_addr learned my addr 172.21.15.18:0/1390364949 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f243affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a2c30 0x5592fb2a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:43940/0 (socket says 172.21.15.18:43940) 2024-02-17T12:50:48.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f243b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 0x5592fb2b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44550/0 (socket says 172.21.15.18:44550) 2024-02-17T12:50:48.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.862+0000 7f2423fff640 1 -- 172.21.15.18:0/1495155107 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f242c0cdc80 con 0x7f243c101390 2024-02-17T12:50:48.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.863+0000 7f243a7fc640 1 -- 172.21.15.18:0/1390364949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a2c30 msgr2=0x5592fb2a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.863+0000 7f243a7fc640 1 --2- 172.21.15.18:0/1390364949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a2c30 0x5592fb2a3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.863+0000 7f243a7fc640 1 -- 172.21.15.18:0/1390364949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 msgr2=0x5592fb2b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.863+0000 7f243a7fc640 1 --2- 172.21.15.18:0/1390364949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 0x5592fb2b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.863+0000 7f243a7fc640 1 -- 172.21.15.18:0/1390364949 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592fb1cd060 con 0x5592fb2a35e0 2024-02-17T12:50:48.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.863+0000 7f243b7fe640 1 --2- 172.21.15.18:0/1390364949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 0x5592fb2b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:48.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.863+0000 7f243affd640 1 --2- 172.21.15.18:0/1390364949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a2c30 0x5592fb2a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:48.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.863+0000 7f243b7fe640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f24240a2350 0x7f24240a4810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2434009fd0 tx=0x7f2434009da0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.863+0000 7f243a7fc640 1 --2- 172.21.15.18:0/1390364949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2a35e0 0x5592fb2b0620 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2430005ab0 tx=0x7f2430005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2951fc3de5c7519b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.863+0000 7f24217fa640 1 -- 172.21.15.18:0/1390364949 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430004900 con 0x5592fb2a35e0 2024-02-17T12:50:48.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.863+0000 7f24217fa640 1 -- 172.21.15.18:0/1390364949 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2430004a60 con 0x5592fb2a35e0 2024-02-17T12:50:48.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.863+0000 7f24217fa640 1 -- 172.21.15.18:0/1390364949 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430005150 con 0x5592fb2a35e0 2024-02-17T12:50:48.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 -- 172.21.15.18:0/1390364949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2a35e0 msgr2=0x5592fb2b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 --2- 172.21.15.18:0/1390364949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2a35e0 0x5592fb2b0620 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2430005ab0 tx=0x7f2430005c00 comp rx=0 tx=0).stop 2024-02-17T12:50:48.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 -- 172.21.15.18:0/1390364949 shutdown_connections 2024-02-17T12:50:48.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 --2- 172.21.15.18:0/1390364949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 0x5592fb2b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 --2- 172.21.15.18:0/1390364949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2a35e0 0x5592fb2b0620 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 --2- 172.21.15.18:0/1390364949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a2c30 0x5592fb2a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 -- 172.21.15.18:0/1390364949 >> 172.21.15.18:0/1390364949 conn(0x5592fb29d980 msgr2=0x5592fb29dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 -- 172.21.15.18:0/1390364949 shutdown_connections 2024-02-17T12:50:48.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 -- 172.21.15.18:0/1390364949 wait complete. 2024-02-17T12:50:48.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 -- start start 2024-02-17T12:50:48.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 Processor -- start 2024-02-17T12:50:48.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 0x5592fb2adb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a35e0 0x5592fb2ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.864+0000 7f2444370740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2b0b60 0x5592fb2ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f2444370740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fb20b020 con 0x5592fb2a2c30 2024-02-17T12:50:48.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f2444370740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592faede430 con 0x5592fb2a35e0 2024-02-17T12:50:48.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f2444370740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fae9ae00 con 0x5592fb2b0b60 2024-02-17T12:50:48.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f2444370740 10 client.0 _ll_register_callbacks cb 0x5592fb1b2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:48.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f243affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 0x5592fb2adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f243affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 0x5592fb2adb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44558/0 (socket says 172.21.15.18:44558) 2024-02-17T12:50:48.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f243affd640 1 -- 172.21.15.18:0/731592521 learned_addr learned my addr 172.21.15.18:0/731592521 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f243a7fc640 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a35e0 0x5592fb2ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f243b7fe640 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2b0b60 0x5592fb2ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f243affd640 1 -- 172.21.15.18:0/731592521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2b0b60 msgr2=0x5592fb2ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f243affd640 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2b0b60 0x5592fb2ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f243affd640 1 -- 172.21.15.18:0/731592521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a35e0 msgr2=0x5592fb2ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f243affd640 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a35e0 0x5592fb2ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f243affd640 1 -- 172.21.15.18:0/731592521 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c10e840 con 0x5592fb2a2c30 2024-02-17T12:50:48.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f243affd640 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 0x5592fb2adb30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f242c0968d0 tx=0x7f242c098980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bea47cf949e533 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f241eff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:48.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f241eff5640 1 -- 172.21.15.18:0/731592521 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0b72d0 con 0x5592fb2a2c30 2024-02-17T12:50:48.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f241eff5640 1 -- 172.21.15.18:0/731592521 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f242c09ab70 con 0x5592fb2a2c30 2024-02-17T12:50:48.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f2444370740 1 -- 172.21.15.18:0/731592521 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592faf0a7e0 con 0x5592fb2a2c30 2024-02-17T12:50:48.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f241eff5640 1 -- 172.21.15.18:0/731592521 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f242c0ba0d0 con 0x5592fb2a2c30 2024-02-17T12:50:48.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f2444370740 10 client.15924 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:48.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.865+0000 7f2444370740 1 -- 172.21.15.18:0/731592521 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5592fb2096a0 con 0x5592fb2a2c30 2024-02-17T12:50:48.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.866+0000 7f241eff5640 1 -- 172.21.15.18:0/731592521 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5592faf0a7e0 con 0x5592fb2a2c30 2024-02-17T12:50:48.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.866+0000 7f241eff5640 1 -- 172.21.15.18:0/731592521 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f242c0b8be0 con 0x5592fb2a2c30 2024-02-17T12:50:48.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.866+0000 7f2444370740 10 client.15924 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:48.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.866+0000 7f2444370740 10 client.15924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:50:48.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.866+0000 7f2444370740 20 client.15924 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:48.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.866+0000 7f2444370740 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5592fb2c0e70 0x5592fb2c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.866+0000 7f2444370740 4 client.15924 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:48.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.866+0000 7f2444370740 1 -- 172.21.15.18:0/731592521 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592faf47720 con 0x5592fb2c0e70 2024-02-17T12:50:48.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.866+0000 7f243a7fc640 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5592fb2c0e70 0x5592fb2c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.866+0000 7f241eff5640 10 client.15924 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:50:48.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.866+0000 7f243a7fc640 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5592fb2c0e70 0x5592fb2c3290 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef60dcd64305612f server_cookie=b19fe92aac8ce92c in_seq=0 out_seq=0 2024-02-17T12:50:48.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f241eff5640 1 -- 172.21.15.18:0/731592521 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592faf47720 con 0x5592fb2c0e70 2024-02-17T12:50:48.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f241eff5640 10 client.15924 handle_command_reply: tid=0 2024-02-17T12:50:48.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f2444370740 1 client.15924 shutdown 2024-02-17T12:50:48.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f2444370740 1 -- 172.21.15.18:0/731592521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 msgr2=0x5592fb2adb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f2444370740 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 0x5592fb2adb30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f242c0968d0 tx=0x7f242c098980 comp rx=0 tx=0).stop 2024-02-17T12:50:48.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f2444370740 1 -- 172.21.15.18:0/731592521 shutdown_connections 2024-02-17T12:50:48.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f2444370740 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5592fb2c0e70 0x5592fb2c3290 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f2444370740 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2b0b60 0x5592fb2ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f2444370740 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a35e0 0x5592fb2ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f2444370740 1 --2- 172.21.15.18:0/731592521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 0x5592fb2adb30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f2444370740 1 -- 172.21.15.18:0/731592521 >> 172.21.15.18:0/731592521 conn(0x5592fb29d980 msgr2=0x5592fb29f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f241eff5640 0 client.15924 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:50:48.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f2444370740 1 -- 172.21.15.18:0/731592521 shutdown_connections 2024-02-17T12:50:48.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f2444370740 1 -- 172.21.15.18:0/731592521 wait complete. 2024-02-17T12:50:48.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.867+0000 7f2444370740 20 client.15924 trim_cache size 0 max 16384 2024-02-17T12:50:48.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f2444370740 1 Processor -- start 2024-02-17T12:50:48.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f2444370740 1 -- start start 2024-02-17T12:50:48.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f2444370740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a2c30 0x5592fb2ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f2444370740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2a35e0 0x5592fb29f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f2444370740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 0x5592fb2b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f2444370740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fb190130 con 0x5592fb2b0b60 2024-02-17T12:50:48.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f2444370740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fb2a4030 con 0x5592fb2a2c30 2024-02-17T12:50:48.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f2444370740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fb2bd9c0 con 0x5592fb2a35e0 2024-02-17T12:50:48.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f243b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 0x5592fb2b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f243b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 0x5592fb2b79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44568/0 (socket says 172.21.15.18:44568) 2024-02-17T12:50:48.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f243b7fe640 1 -- 172.21.15.18:0/1648526572 learned_addr learned my addr 172.21.15.18:0/1648526572 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f243affd640 1 --2- 172.21.15.18:0/1648526572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a2c30 0x5592fb2ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.879+0000 7f243a7fc640 1 --2- 172.21.15.18:0/1648526572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2a35e0 0x5592fb29f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.880+0000 7f243b7fe640 1 -- 172.21.15.18:0/1648526572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2a35e0 msgr2=0x5592fb29f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.880+0000 7f243b7fe640 1 --2- 172.21.15.18:0/1648526572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2a35e0 0x5592fb29f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.880+0000 7f243b7fe640 1 -- 172.21.15.18:0/1648526572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a2c30 msgr2=0x5592fb2ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.880+0000 7f243b7fe640 1 --2- 172.21.15.18:0/1648526572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a2c30 0x5592fb2ae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.880+0000 7f243b7fe640 1 -- 172.21.15.18:0/1648526572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592fb1a5c20 con 0x5592fb2b0b60 2024-02-17T12:50:48.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.880+0000 7f243b7fe640 1 --2- 172.21.15.18:0/1648526572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 0x5592fb2b79c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2434010d80 tx=0x7f2434012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd75fde9450d0397 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.880+0000 7f2420ff9640 1 -- 172.21.15.18:0/1648526572 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434010db0 con 0x5592fb2b0b60 2024-02-17T12:50:48.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.880+0000 7f2420ff9640 1 -- 172.21.15.18:0/1648526572 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f243401e8d0 con 0x5592fb2b0b60 2024-02-17T12:50:48.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.880+0000 7f2420ff9640 1 -- 172.21.15.18:0/1648526572 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340118a0 con 0x5592fb2b0b60 2024-02-17T12:50:48.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 -- 172.21.15.18:0/1648526572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 msgr2=0x5592fb2b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 --2- 172.21.15.18:0/1648526572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 0x5592fb2b79c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2434010d80 tx=0x7f2434012f90 comp rx=0 tx=0).stop 2024-02-17T12:50:48.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 -- 172.21.15.18:0/1648526572 shutdown_connections 2024-02-17T12:50:48.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 --2- 172.21.15.18:0/1648526572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2b0b60 0x5592fb2b79c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 --2- 172.21.15.18:0/1648526572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2a35e0 0x5592fb29f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 --2- 172.21.15.18:0/1648526572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a2c30 0x5592fb2ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 -- 172.21.15.18:0/1648526572 >> 172.21.15.18:0/1648526572 conn(0x5592fb29d980 msgr2=0x5592fb29edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 -- 172.21.15.18:0/1648526572 shutdown_connections 2024-02-17T12:50:48.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 -- 172.21.15.18:0/1648526572 wait complete. 2024-02-17T12:50:48.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 -- start start 2024-02-17T12:50:48.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 Processor -- start 2024-02-17T12:50:48.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 0x5592fb2b6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a35e0 0x5592fb2b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2b0b60 0x5592fb2b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fb3173c0 con 0x5592fb2a2c30 2024-02-17T12:50:48.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fb317500 con 0x5592fb2a35e0 2024-02-17T12:50:48.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fb317670 con 0x5592fb2b0b60 2024-02-17T12:50:48.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f2444370740 10 client.0 _ll_register_callbacks cb 0x7f2430005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:48.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f243affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 0x5592fb2b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f243a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a35e0 0x5592fb2b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f243affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 0x5592fb2b6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44574/0 (socket says 172.21.15.18:44574) 2024-02-17T12:50:48.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.881+0000 7f243affd640 1 -- 172.21.15.18:0/2657126840 learned_addr learned my addr 172.21.15.18:0/2657126840 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:48.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f243b7fe640 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2b0b60 0x5592fb2b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f243a7fc640 1 -- 172.21.15.18:0/2657126840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2b0b60 msgr2=0x5592fb2b6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f243a7fc640 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2b0b60 0x5592fb2b6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f243a7fc640 1 -- 172.21.15.18:0/2657126840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 msgr2=0x5592fb2b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:48.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f243a7fc640 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 0x5592fb2b6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f243a7fc640 1 -- 172.21.15.18:0/2657126840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2434010ac0 con 0x5592fb2a35e0 2024-02-17T12:50:48.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f243affd640 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 0x5592fb2b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:48.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f243b7fe640 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2b0b60 0x5592fb2b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:48.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f243a7fc640 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a35e0 0x5592fb2b6a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2430009900 tx=0x7f2430004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e56103e4f310e56 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:48.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f241eff5640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:50:48.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f241eff5640 1 -- 172.21.15.18:0/2657126840 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430004740 con 0x5592fb2a35e0 2024-02-17T12:50:48.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f241eff5640 1 -- 172.21.15.18:0/2657126840 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f243002cc10 con 0x5592fb2a35e0 2024-02-17T12:50:48.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f2444370740 1 -- 172.21.15.18:0/2657126840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2430004a60 con 0x5592fb2a35e0 2024-02-17T12:50:48.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f241eff5640 1 -- 172.21.15.18:0/2657126840 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2430051b00 con 0x5592fb2a35e0 2024-02-17T12:50:48.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f2444370740 10 client.25394 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:48.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f2444370740 1 -- 172.21.15.18:0/2657126840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5592fb1cb8c0 con 0x5592fb2a35e0 2024-02-17T12:50:48.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.882+0000 7f241eff5640 1 -- 172.21.15.18:0/2657126840 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2430004a60 con 0x5592fb2a35e0 2024-02-17T12:50:48.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f241eff5640 1 -- 172.21.15.18:0/2657126840 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f243003c050 con 0x5592fb2a35e0 2024-02-17T12:50:48.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f2444370740 10 client.25394 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:48.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f2444370740 10 client.25394 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:50:48.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f2444370740 20 client.25394 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:48.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f2444370740 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5592fb318b40 0x5592fb31af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:48.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f2444370740 4 client.25394 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:50:48.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f2444370740 1 -- 172.21.15.18:0/2657126840 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5592faf96890 con 0x5592fb318b40 2024-02-17T12:50:48.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f243affd640 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5592fb318b40 0x5592fb31af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:48.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f241eff5640 10 client.25394 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:50:48.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f243affd640 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5592fb318b40 0x5592fb31af60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6afdaf9ebe382c7 server_cookie=a7ac37c49854657b in_seq=0 out_seq=0 2024-02-17T12:50:48.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f241eff5640 1 -- 172.21.15.18:0/2657126840 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5592faf96890 con 0x5592fb318b40 2024-02-17T12:50:48.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f241eff5640 10 client.25394 handle_command_reply: tid=0 2024-02-17T12:50:48.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.883+0000 7f2444370740 1 client.25394 shutdown 2024-02-17T12:50:48.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f2444370740 1 -- 172.21.15.18:0/2657126840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a35e0 msgr2=0x5592fb2b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f2444370740 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a35e0 0x5592fb2b6a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2430009900 tx=0x7f2430004610 comp rx=0 tx=0).stop 2024-02-17T12:50:48.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f2444370740 1 -- 172.21.15.18:0/2657126840 shutdown_connections 2024-02-17T12:50:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f2444370740 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5592fb318b40 0x5592fb31af60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f2444370740 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5592fb2b0b60 0x5592fb2b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f241eff5640 0 client.25394 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:50:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f2444370740 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fb2a35e0 0x5592fb2b6a80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f2444370740 1 --2- 172.21.15.18:0/2657126840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592fb2a2c30 0x5592fb2b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f2444370740 1 -- 172.21.15.18:0/2657126840 >> 172.21.15.18:0/2657126840 conn(0x5592fb29d980 msgr2=0x5592fb29cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f2444370740 1 -- 172.21.15.18:0/2657126840 shutdown_connections 2024-02-17T12:50:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f2444370740 1 -- 172.21.15.18:0/2657126840 wait complete. 2024-02-17T12:50:48.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f2444370740 20 client.25394 trim_cache size 0 max 16384 2024-02-17T12:50:48.911 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:50:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f24417a0640 1 -- 172.21.15.18:0/1495155107 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f24240a2350 msgr2=0x7f24240a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f24417a0640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f24240a2350 0x7f24240a4810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2434009fd0 tx=0x7f2434009da0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f24417a0640 1 -- 172.21.15.18:0/1495155107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f243c101390 msgr2=0x7f243c19d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:48.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f24417a0640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f243c101390 0x7f243c19d5c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f242c00d950 tx=0x7f242c00de10 comp rx=0 tx=0).stop 2024-02-17T12:50:48.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f24417a0640 1 -- 172.21.15.18:0/1495155107 shutdown_connections 2024-02-17T12:50:48.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f24417a0640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f24240a2350 0x7f24240a4810 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f24417a0640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f243c197680 0x7f243c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f24417a0640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f243c106460 0x7f243c19db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f24417a0640 1 --2- 172.21.15.18:0/1495155107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f243c101390 0x7f243c19d5c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:48.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f24417a0640 1 -- 172.21.15.18:0/1495155107 >> 172.21.15.18:0/1495155107 conn(0x7f243c0fcc10 msgr2=0x7f243c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:48.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f24417a0640 1 -- 172.21.15.18:0/1495155107 shutdown_connections 2024-02-17T12:50:48.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:48.884+0000 7f24417a0640 1 -- 172.21.15.18:0/1495155107 wait complete. 2024-02-17T12:50:48.915 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:50:48.915 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:50:48.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:48 smithi146 ceph-mon[36454]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-02-17T12:50:48.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:48 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3426285369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:50:49.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.033+0000 7f52f82eb640 1 -- 172.21.15.18:0/1158812893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52f0101380 msgr2=0x7f52f0101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.033+0000 7f52f82eb640 1 --2- 172.21.15.18:0/1158812893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52f0101380 0x7f52f0101760 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f52ec009930 tx=0x7f52ec02bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.033+0000 7f52f82eb640 1 -- 172.21.15.18:0/1158812893 shutdown_connections 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.033+0000 7f52f82eb640 1 --2- 172.21.15.18:0/1158812893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52f0106450 0x7f52f010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.033+0000 7f52f82eb640 1 --2- 172.21.15.18:0/1158812893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52f0101d30 0x7f52f0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.033+0000 7f52f82eb640 1 --2- 172.21.15.18:0/1158812893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52f0101380 0x7f52f0101760 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.033+0000 7f52f82eb640 1 -- 172.21.15.18:0/1158812893 >> 172.21.15.18:0/1158812893 conn(0x7f52f0078580 msgr2=0x7f52f0078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.033+0000 7f52f82eb640 1 -- 172.21.15.18:0/1158812893 shutdown_connections 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.033+0000 7f52f82eb640 1 -- 172.21.15.18:0/1158812893 wait complete. 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.033+0000 7f52f82eb640 1 Processor -- start 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.033+0000 7f52f82eb640 1 -- start start 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f82eb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52f0101380 0x7f52f019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f82eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52f0106450 0x7f52f019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f82eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52f0197620 0x7f52f0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f82eb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f0198010 con 0x7f52f0197620 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f82eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f0198180 con 0x7f52f0106450 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f82eb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f0198340 con 0x7f52f0101380 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f6888640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52f0197620 0x7f52f0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f6888640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52f0197620 0x7f52f0197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44598/0 (socket says 172.21.15.18:44598) 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f6888640 1 -- 172.21.15.18:0/1767315357 learned_addr learned my addr 172.21.15.18:0/1767315357 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f5886640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52f0106450 0x7f52f019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f6087640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52f0101380 0x7f52f019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f6888640 1 -- 172.21.15.18:0/1767315357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52f0101380 msgr2=0x7f52f019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f6888640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52f0101380 0x7f52f019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f6888640 1 -- 172.21.15.18:0/1767315357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52f0106450 msgr2=0x7f52f019d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f6888640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52f0106450 0x7f52f019d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f6888640 1 -- 172.21.15.18:0/1767315357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e0009490 con 0x7f52f0197620 2024-02-17T12:50:49.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f6888640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52f0197620 0x7f52f0197ad0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f52e800eb10 tx=0x7f52e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:49.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52e77fe640 1 -- 172.21.15.18:0/1767315357 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8004b20 con 0x7f52f0197620 2024-02-17T12:50:49.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f82eb640 1 -- 172.21.15.18:0/1767315357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec009560 con 0x7f52f0197620 2024-02-17T12:50:49.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52e77fe640 1 -- 172.21.15.18:0/1767315357 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52e8004c80 con 0x7f52f0197620 2024-02-17T12:50:49.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52e77fe640 1 -- 172.21.15.18:0/1767315357 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8010670 con 0x7f52f0197620 2024-02-17T12:50:49.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.034+0000 7f52f82eb640 1 -- 172.21.15.18:0/1767315357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f01a46b0 con 0x7f52f0197620 2024-02-17T12:50:49.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.036+0000 7f52e77fe640 1 -- 172.21.15.18:0/1767315357 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f52e8005020 con 0x7f52f0197620 2024-02-17T12:50:49.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.037+0000 7f52e77fe640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52dc0a2350 0x7f52dc0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.037+0000 7f52faf96740 1 Processor -- start 2024-02-17T12:50:49.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.037+0000 7f52e77fe640 1 -- 172.21.15.18:0/1767315357 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f52e8014070 con 0x7f52f0197620 2024-02-17T12:50:49.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.037+0000 7f52f6087640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52dc0a2350 0x7f52dc0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.037+0000 7f52faf96740 1 -- start start 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52faf96740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649250 0x55b574649630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52faf96740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649b70 0x55b574652210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52faf96740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574652750 0x55b574654b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52faf96740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b574604960 con 0x55b574649250 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52faf96740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5745b8310 con 0x55b574649b70 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52faf96740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5745b5840 con 0x55b574652750 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f6087640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649250 0x55b574649630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f6087640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649250 0x55b574649630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44614/0 (socket says 172.21.15.18:44614) 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f6087640 1 -- 172.21.15.18:0/1100259456 learned_addr learned my addr 172.21.15.18:0/1100259456 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f5886640 1 --2- 172.21.15.18:0/1100259456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649b70 0x55b574652210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f6888640 1 --2- 172.21.15.18:0/1100259456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574652750 0x55b574654b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f6087640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52dc0a2350 0x7f52dc0a4810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f52ec006010 tx=0x7f52ec0046c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f5886640 1 -- 172.21.15.18:0/1100259456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574652750 msgr2=0x55b574654b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f5886640 1 --2- 172.21.15.18:0/1100259456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574652750 0x55b574654b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f5886640 1 -- 172.21.15.18:0/1100259456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649250 msgr2=0x55b574649630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f5886640 1 --2- 172.21.15.18:0/1100259456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649250 0x55b574649630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f5886640 1 -- 172.21.15.18:0/1100259456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b574573060 con 0x55b574649b70 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f6888640 1 --2- 172.21.15.18:0/1100259456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574652750 0x55b574654b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f6087640 1 --2- 172.21.15.18:0/1100259456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649250 0x55b574649630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.038+0000 7f52f5886640 1 --2- 172.21.15.18:0/1100259456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649b70 0x55b574652210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52f0198760 tx=0x7f52e0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7131fc244c7b4c68 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.039+0000 7f52e4bfe640 1 -- 172.21.15.18:0/1100259456 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0019410 con 0x55b574649b70 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.039+0000 7f52e4bfe640 1 -- 172.21.15.18:0/1100259456 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52e0019a50 con 0x55b574649b70 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.039+0000 7f52e4bfe640 1 -- 172.21.15.18:0/1100259456 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e00189f0 con 0x55b574649b70 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.039+0000 7f52faf96740 1 -- 172.21.15.18:0/1100259456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649b70 msgr2=0x55b574652210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.039+0000 7f52faf96740 1 --2- 172.21.15.18:0/1100259456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649b70 0x55b574652210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52f0198760 tx=0x7f52e0011810 comp rx=0 tx=0).stop 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.039+0000 7f52faf96740 1 -- 172.21.15.18:0/1100259456 shutdown_connections 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.039+0000 7f52faf96740 1 --2- 172.21.15.18:0/1100259456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574652750 0x55b574654b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.039+0000 7f52faf96740 1 --2- 172.21.15.18:0/1100259456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649b70 0x55b574652210 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.039+0000 7f52faf96740 1 --2- 172.21.15.18:0/1100259456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649250 0x55b574649630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.039+0000 7f52faf96740 1 -- 172.21.15.18:0/1100259456 >> 172.21.15.18:0/1100259456 conn(0x55b574643980 msgr2=0x55b574643d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.039+0000 7f52faf96740 1 -- 172.21.15.18:0/1100259456 shutdown_connections 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.039+0000 7f52faf96740 1 -- 172.21.15.18:0/1100259456 wait complete. 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52faf96740 1 -- start start 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52faf96740 1 Processor -- start 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52faf96740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649250 0x55b57464f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52faf96740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574649b70 0x55b57464fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52faf96740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574652750 0x55b5746501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52faf96740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5745b1020 con 0x55b574652750 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52faf96740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b574284430 con 0x55b574649250 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52faf96740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b574240e00 con 0x55b574649b70 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52faf96740 10 client.0 _ll_register_callbacks cb 0x55b574558cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52f6888640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574652750 0x55b5746501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52f6888640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574652750 0x55b5746501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44624/0 (socket says 172.21.15.18:44624) 2024-02-17T12:50:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52f6888640 1 -- 172.21.15.18:0/2513710089 learned_addr learned my addr 172.21.15.18:0/2513710089 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52f6087640 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649250 0x55b57464f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52f5886640 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574649b70 0x55b57464fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52f6888640 1 -- 172.21.15.18:0/2513710089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574649b70 msgr2=0x55b57464fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52f6888640 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574649b70 0x55b57464fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52f6888640 1 -- 172.21.15.18:0/2513710089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649250 msgr2=0x55b57464f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52f6888640 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649250 0x55b57464f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.040+0000 7f52f6888640 1 -- 172.21.15.18:0/2513710089 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e00099e0 con 0x55b574652750 2024-02-17T12:50:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52f6888640 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574652750 0x55b5746501e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f52e8097a60 tx=0x7f52e809aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1726ec0d9c513f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52d9ffb640 1 -- 172.21.15.18:0/2513710089 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e80b1030 con 0x55b574652750 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52d9ffb640 1 -- 172.21.15.18:0/2513710089 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f52e80b2070 con 0x55b574652750 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52faf96740 1 -- 172.21.15.18:0/2513710089 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5742b07e0 con 0x55b574652750 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52d9ffb640 1 -- 172.21.15.18:0/2513710089 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52e80ac990 con 0x55b574652750 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52faf96740 10 client.15942 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52faf96740 1 -- 172.21.15.18:0/2513710089 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b5745af6a0 con 0x55b574652750 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52d9ffb640 1 -- 172.21.15.18:0/2513710089 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b5742b07e0 con 0x55b574652750 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52d9ffb640 1 -- 172.21.15.18:0/2513710089 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f52e801f060 con 0x55b574652750 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52faf96740 10 client.15942 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52faf96740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52faf96740 20 client.15942 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52faf96740 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b574662b90 0x55b574664f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52faf96740 4 client.15942 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.041+0000 7f52faf96740 1 -- 172.21.15.18:0/2513710089 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5742ed720 con 0x55b574662b90 2024-02-17T12:50:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.042+0000 7f52f6087640 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b574662b90 0x55b574664f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.042+0000 7f52d9ffb640 10 client.15942 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.042+0000 7f52f6087640 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b574662b90 0x55b574664f80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10dc73e7af094a5d server_cookie=ffaf4d699d4d770c in_seq=0 out_seq=0 2024-02-17T12:50:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52d9ffb640 1 -- 172.21.15.18:0/2513710089 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b5742ed720 con 0x55b574662b90 2024-02-17T12:50:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52d9ffb640 10 client.15942 handle_command_reply: tid=0 2024-02-17T12:50:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52faf96740 1 client.15942 shutdown 2024-02-17T12:50:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52faf96740 1 -- 172.21.15.18:0/2513710089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574652750 msgr2=0x55b5746501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52faf96740 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574652750 0x55b5746501e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f52e8097a60 tx=0x7f52e809aab0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52faf96740 1 -- 172.21.15.18:0/2513710089 shutdown_connections 2024-02-17T12:50:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52faf96740 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b574662b90 0x55b574664f80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52faf96740 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574652750 0x55b5746501e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52d9ffb640 0 client.15942 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:49.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52faf96740 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574649b70 0x55b57464fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52faf96740 1 --2- 172.21.15.18:0/2513710089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649250 0x55b57464f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52faf96740 1 -- 172.21.15.18:0/2513710089 >> 172.21.15.18:0/2513710089 conn(0x55b574643980 msgr2=0x55b5746456d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:49.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52faf96740 1 -- 172.21.15.18:0/2513710089 shutdown_connections 2024-02-17T12:50:49.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.043+0000 7f52faf96740 1 -- 172.21.15.18:0/2513710089 wait complete. 2024-02-17T12:50:49.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.044+0000 7f52faf96740 20 client.15942 trim_cache size 0 max 16384 2024-02-17T12:50:49.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.055+0000 7f52faf96740 1 Processor -- start 2024-02-17T12:50:49.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.055+0000 7f52faf96740 1 -- start start 2024-02-17T12:50:49.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52faf96740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649250 0x55b574651e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52faf96740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 0x55b574650000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52faf96740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574652750 0x55b57464aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52faf96740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b574536130 con 0x55b574649b70 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52faf96740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5742b07e0 con 0x55b574649250 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52faf96740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57465f690 con 0x55b574652750 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52f5886640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 0x55b574650000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52f5886640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 0x55b574650000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44628/0 (socket says 172.21.15.18:44628) 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52f5886640 1 -- 172.21.15.18:0/365327170 learned_addr learned my addr 172.21.15.18:0/365327170 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52f6087640 1 --2- 172.21.15.18:0/365327170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649250 0x55b574651e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52f6888640 1 --2- 172.21.15.18:0/365327170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574652750 0x55b57464aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52f5886640 1 -- 172.21.15.18:0/365327170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574652750 msgr2=0x55b57464aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52f5886640 1 --2- 172.21.15.18:0/365327170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574652750 0x55b57464aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52f5886640 1 -- 172.21.15.18:0/365327170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649250 msgr2=0x55b574651e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52f5886640 1 --2- 172.21.15.18:0/365327170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649250 0x55b574651e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52f5886640 1 -- 172.21.15.18:0/365327170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b57454bc20 con 0x55b574649b70 2024-02-17T12:50:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52f5886640 1 --2- 172.21.15.18:0/365327170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 0x55b574650000 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009cd0 tx=0x7f52e0018460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49cb4f9a5ec2c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52dbfff640 1 -- 172.21.15.18:0/365327170 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0009ec0 con 0x55b574649b70 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52dbfff640 1 -- 172.21.15.18:0/365327170 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52e0019bc0 con 0x55b574649b70 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.056+0000 7f52dbfff640 1 -- 172.21.15.18:0/365327170 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0003850 con 0x55b574649b70 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.057+0000 7f52faf96740 1 -- 172.21.15.18:0/365327170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 msgr2=0x55b574650000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.057+0000 7f52faf96740 1 --2- 172.21.15.18:0/365327170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 0x55b574650000 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009cd0 tx=0x7f52e0018460 comp rx=0 tx=0).stop 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.057+0000 7f52faf96740 1 -- 172.21.15.18:0/365327170 shutdown_connections 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.057+0000 7f52faf96740 1 --2- 172.21.15.18:0/365327170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574652750 0x55b57464aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.057+0000 7f52faf96740 1 --2- 172.21.15.18:0/365327170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 0x55b574650000 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.057+0000 7f52faf96740 1 --2- 172.21.15.18:0/365327170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574649250 0x55b574651e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.057+0000 7f52faf96740 1 -- 172.21.15.18:0/365327170 >> 172.21.15.18:0/365327170 conn(0x55b574643980 msgr2=0x55b57463ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.057+0000 7f52faf96740 1 -- 172.21.15.18:0/365327170 shutdown_connections 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.057+0000 7f52faf96740 1 -- 172.21.15.18:0/365327170 wait complete. 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.057+0000 7f52faf96740 1 -- start start 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.057+0000 7f52faf96740 1 Processor -- start 2024-02-17T12:50:49.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52faf96740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574649250 0x55b574658380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52faf96740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 0x55b5746588c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52faf96740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574652750 0x55b5746b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52faf96740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b574659080 con 0x55b574649b70 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52faf96740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5746591f0 con 0x55b574652750 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52faf96740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5746b95c0 con 0x55b574649250 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52faf96740 10 client.0 _ll_register_callbacks cb 0x7f52e0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52f5886640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 0x55b5746588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52f5886640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 0x55b5746588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44638/0 (socket says 172.21.15.18:44638) 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52f5886640 1 -- 172.21.15.18:0/3088932879 learned_addr learned my addr 172.21.15.18:0/3088932879 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52f6087640 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574649250 0x55b574658380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52f6888640 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574652750 0x55b5746b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52f5886640 1 -- 172.21.15.18:0/3088932879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574649250 msgr2=0x55b574658380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52f5886640 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574649250 0x55b574658380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52f5886640 1 -- 172.21.15.18:0/3088932879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574652750 msgr2=0x55b5746b9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52f5886640 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574652750 0x55b5746b9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52f5886640 1 -- 172.21.15.18:0/3088932879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b574573060 con 0x55b574649b70 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52f5886640 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 0x55b5746588c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f52e001b040 tx=0x7f52e0019f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff91d11dfe2ca4f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52d9ffb640 1 -- 172.21.15.18:0/3088932879 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0019410 con 0x55b574649b70 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52d9ffb640 1 -- 172.21.15.18:0/3088932879 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f52e0019570 con 0x55b574649b70 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52faf96740 1 -- 172.21.15.18:0/3088932879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52e0019a50 con 0x55b574649b70 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52d9ffb640 1 -- 172.21.15.18:0/3088932879 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52e0019730 con 0x55b574649b70 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52faf96740 10 client.15948 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.058+0000 7f52faf96740 1 -- 172.21.15.18:0/3088932879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b5745718c0 con 0x55b574649b70 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.059+0000 7f52d9ffb640 1 -- 172.21.15.18:0/3088932879 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52e003d080 con 0x55b574649b70 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.059+0000 7f52d9ffb640 1 -- 172.21.15.18:0/3088932879 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f52e002a020 con 0x55b574649b70 2024-02-17T12:50:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.059+0000 7f52faf96740 10 client.15948 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.059+0000 7f52faf96740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.059+0000 7f52faf96740 20 client.15948 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.059+0000 7f52faf96740 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b5746ba900 0x55b5746bcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.059+0000 7f52faf96740 4 client.15948 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.059+0000 7f52faf96740 1 -- 172.21.15.18:0/3088932879 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b57433c890 con 0x55b5746ba900 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.059+0000 7f52f6087640 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b5746ba900 0x55b5746bcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.059+0000 7f52f6087640 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b5746ba900 0x55b5746bcd20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320085783873f4ec server_cookie=2f866d7d06e3512e in_seq=0 out_seq=0 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.059+0000 7f52d9ffb640 10 client.15948 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52d9ffb640 1 -- 172.21.15.18:0/3088932879 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b57433c890 con 0x55b5746ba900 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52d9ffb640 10 client.15948 handle_command_reply: tid=0 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52faf96740 1 client.15948 shutdown 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52faf96740 1 -- 172.21.15.18:0/3088932879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 msgr2=0x55b5746588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52faf96740 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 0x55b5746588c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f52e001b040 tx=0x7f52e0019f60 comp rx=0 tx=0).stop 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52faf96740 1 -- 172.21.15.18:0/3088932879 shutdown_connections 2024-02-17T12:50:49.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52faf96740 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b5746ba900 0x55b5746bcd20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52faf96740 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b574652750 0x55b5746b9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52faf96740 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b574649b70 0x55b5746588c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52d9ffb640 0 client.15948 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:49.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52faf96740 1 --2- 172.21.15.18:0/3088932879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b574649250 0x55b574658380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52faf96740 1 -- 172.21.15.18:0/3088932879 >> 172.21.15.18:0/3088932879 conn(0x55b574643980 msgr2=0x55b57464aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:49.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52faf96740 1 -- 172.21.15.18:0/3088932879 shutdown_connections 2024-02-17T12:50:49.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52faf96740 1 -- 172.21.15.18:0/3088932879 wait complete. 2024-02-17T12:50:49.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.060+0000 7f52faf96740 20 client.15948 trim_cache size 0 max 16384 2024-02-17T12:50:49.059 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:50:49.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.061+0000 7f52f82eb640 1 -- 172.21.15.18:0/1767315357 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52dc0a2350 msgr2=0x7f52dc0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:49.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.061+0000 7f52f82eb640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52dc0a2350 0x7f52dc0a4810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f52ec006010 tx=0x7f52ec0046c0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.061+0000 7f52f82eb640 1 -- 172.21.15.18:0/1767315357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52f0197620 msgr2=0x7f52f0197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:49.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.061+0000 7f52f82eb640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52f0197620 0x7f52f0197ad0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f52e800eb10 tx=0x7f52e800c2d0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.061+0000 7f52f82eb640 1 -- 172.21.15.18:0/1767315357 shutdown_connections 2024-02-17T12:50:49.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.061+0000 7f52f82eb640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52dc0a2350 0x7f52dc0a4810 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.061+0000 7f52f82eb640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52f0197620 0x7f52f0197ad0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.061+0000 7f52f82eb640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52f0106450 0x7f52f019d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.061+0000 7f52f82eb640 1 --2- 172.21.15.18:0/1767315357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52f0101380 0x7f52f019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:49.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.061+0000 7f52f82eb640 1 -- 172.21.15.18:0/1767315357 >> 172.21.15.18:0/1767315357 conn(0x7f52f0078580 msgr2=0x7f52f00ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:49.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.061+0000 7f52f82eb640 1 -- 172.21.15.18:0/1767315357 shutdown_connections 2024-02-17T12:50:49.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:49.061+0000 7f52f82eb640 1 -- 172.21.15.18:0/1767315357 wait complete. 2024-02-17T12:50:49.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 13548 22.55 MB/sec warmup 4 sec latency 5.332 ms 2024-02-17T12:50:50.065 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:50:50.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:50:50.065 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:50:50.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.183+0000 7fdabb464640 1 -- 172.21.15.18:0/3476089063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdab4101d30 msgr2=0x7fdab4105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.183+0000 7fdabb464640 1 --2- 172.21.15.18:0/3476089063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdab4101d30 0x7fdab4105d70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009930 tx=0x7fdab002bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.183+0000 7fdabb464640 1 -- 172.21.15.18:0/3476089063 shutdown_connections 2024-02-17T12:50:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.183+0000 7fdabb464640 1 --2- 172.21.15.18:0/3476089063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdab4106450 0x7fdab410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.183+0000 7fdabb464640 1 --2- 172.21.15.18:0/3476089063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdab4101d30 0x7fdab4105d70 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.183+0000 7fdabb464640 1 --2- 172.21.15.18:0/3476089063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab4101380 0x7fdab4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.183+0000 7fdabb464640 1 -- 172.21.15.18:0/3476089063 >> 172.21.15.18:0/3476089063 conn(0x7fdab40fcc20 msgr2=0x7fdab40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.183+0000 7fdabb464640 1 -- 172.21.15.18:0/3476089063 shutdown_connections 2024-02-17T12:50:50.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.183+0000 7fdabb464640 1 -- 172.21.15.18:0/3476089063 wait complete. 2024-02-17T12:50:50.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.183+0000 7fdabb464640 1 Processor -- start 2024-02-17T12:50:50.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdabb464640 1 -- start start 2024-02-17T12:50:50.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdabb464640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdab4101380 0x7fdab419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdabb464640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdab4106450 0x7fdab419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdabb464640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab41976a0 0x7fdab4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdabb464640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab4198170 con 0x7fdab4106450 2024-02-17T12:50:50.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdabb464640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab41982e0 con 0x7fdab41976a0 2024-02-17T12:50:50.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdabb464640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab41984a0 con 0x7fdab4101380 2024-02-17T12:50:50.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdab89ff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdab4106450 0x7fdab419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdab89ff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdab4106450 0x7fdab419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57270/0 (socket says 172.21.15.18:57270) 2024-02-17T12:50:50.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdab9200640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdab4101380 0x7fdab419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdab89ff640 1 -- 172.21.15.18:0/3015764693 learned_addr learned my addr 172.21.15.18:0/3015764693 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:50.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdab9a01640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab41976a0 0x7fdab4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdab89ff640 1 -- 172.21.15.18:0/3015764693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdab4101380 msgr2=0x7fdab419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdab89ff640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdab4101380 0x7fdab419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdab89ff640 1 -- 172.21.15.18:0/3015764693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab41976a0 msgr2=0x7fdab4197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdab89ff640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab41976a0 0x7fdab4197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdab89ff640 1 -- 172.21.15.18:0/3015764693 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa4009490 con 0x7fdab4106450 2024-02-17T12:50:50.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdab89ff640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdab4106450 0x7fdab419dad0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdab0005ab0 tx=0x7fdab002cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:50.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.184+0000 7fdaaa7fc640 1 -- 172.21.15.18:0/3015764693 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab0004350 con 0x7fdab4106450 2024-02-17T12:50:50.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.185+0000 7fdabb464640 1 -- 172.21.15.18:0/3015764693 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab0009560 con 0x7fdab4106450 2024-02-17T12:50:50.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.185+0000 7fdaaa7fc640 1 -- 172.21.15.18:0/3015764693 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdab00044b0 con 0x7fdab4106450 2024-02-17T12:50:50.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.185+0000 7fdaaa7fc640 1 -- 172.21.15.18:0/3015764693 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab0003cc0 con 0x7fdab4106450 2024-02-17T12:50:50.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.185+0000 7fdabb464640 1 -- 172.21.15.18:0/3015764693 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab41a46e0 con 0x7fdab4106450 2024-02-17T12:50:50.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.187+0000 7fdaaa7fc640 1 -- 172.21.15.18:0/3015764693 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fdab003e450 con 0x7fdab4106450 2024-02-17T12:50:50.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.187+0000 7fdabe033740 1 Processor -- start 2024-02-17T12:50:50.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.187+0000 7fdabe033740 1 -- start start 2024-02-17T12:50:50.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.187+0000 7fdabe033740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58380 0x556052b58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.187+0000 7fdabe033740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58d30 0x556052b612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.187+0000 7fdabe033740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 0x556052b63d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.187+0000 7fdabe033740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556052ac6f90 con 0x556052b618b0 2024-02-17T12:50:50.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.187+0000 7fdabe033740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556052ac44c0 con 0x556052b58d30 2024-02-17T12:50:50.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.187+0000 7fdabe033740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556052abfca0 con 0x556052b58380 2024-02-17T12:50:50.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9a01640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58380 0x556052b58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58d30 0x556052b612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab89ff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 0x556052b63d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58d30 0x556052b612e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:48398/0 (socket says 172.21.15.18:48398) 2024-02-17T12:50:50.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9a01640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58380 0x556052b58760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:55902/0 (socket says 172.21.15.18:55902) 2024-02-17T12:50:50.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9200640 1 -- 172.21.15.18:0/1000131550 learned_addr learned my addr 172.21.15.18:0/1000131550 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:50.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdaaa7fc640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdaa00a2350 0x7fdaa00a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9200640 1 -- 172.21.15.18:0/1000131550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58380 msgr2=0x556052b58760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9200640 1 --2- 172.21.15.18:0/1000131550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58380 0x556052b58760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9200640 1 -- 172.21.15.18:0/1000131550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 msgr2=0x556052b63d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9200640 1 --2- 172.21.15.18:0/1000131550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 0x556052b63d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9200640 1 -- 172.21.15.18:0/1000131550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556052a820b0 con 0x556052b58d30 2024-02-17T12:50:50.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab89ff640 1 --2- 172.21.15.18:0/1000131550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 0x556052b63d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:50.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9a01640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdaa00a2350 0x7fdaa00a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdaaa7fc640 1 -- 172.21.15.18:0/3015764693 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdab00f08f0 con 0x7fdab4106450 2024-02-17T12:50:50.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9200640 1 --2- 172.21.15.18:0/1000131550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58d30 0x556052b612e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009e80 tx=0x7fdaa4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c53cc7bd1dcace70 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:50.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fdab9a01640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdaa00a2350 0x7fdaa00a4810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdaac005fd0 tx=0x7fdaac004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:50.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fda9f7fe640 1 -- 172.21.15.18:0/1000131550 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4019410 con 0x556052b58d30 2024-02-17T12:50:50.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fda9f7fe640 1 -- 172.21.15.18:0/1000131550 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdaa4019a50 con 0x556052b58d30 2024-02-17T12:50:50.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.188+0000 7fda9f7fe640 1 -- 172.21.15.18:0/1000131550 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4018b30 con 0x556052b58d30 2024-02-17T12:50:50.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.189+0000 7fdabe033740 1 -- 172.21.15.18:0/1000131550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58d30 msgr2=0x556052b612e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.189+0000 7fdabe033740 1 --2- 172.21.15.18:0/1000131550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58d30 0x556052b612e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009e80 tx=0x7fdaa4011810 comp rx=0 tx=0).stop 2024-02-17T12:50:50.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.189+0000 7fdabe033740 1 -- 172.21.15.18:0/1000131550 shutdown_connections 2024-02-17T12:50:50.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.189+0000 7fdabe033740 1 --2- 172.21.15.18:0/1000131550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 0x556052b63d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.189+0000 7fdabe033740 1 --2- 172.21.15.18:0/1000131550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58d30 0x556052b612e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.189+0000 7fdabe033740 1 --2- 172.21.15.18:0/1000131550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58380 0x556052b58760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.189+0000 7fdabe033740 1 -- 172.21.15.18:0/1000131550 >> 172.21.15.18:0/1000131550 conn(0x556052b52840 msgr2=0x556052b52c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:50.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.189+0000 7fdabe033740 1 -- 172.21.15.18:0/1000131550 shutdown_connections 2024-02-17T12:50:50.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.189+0000 7fdabe033740 1 -- 172.21.15.18:0/1000131550 wait complete. 2024-02-17T12:50:50.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.189+0000 7fdabe033740 1 -- start start 2024-02-17T12:50:50.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.189+0000 7fdabe033740 1 Processor -- start 2024-02-17T12:50:50.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdabe033740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58380 0x556052b5e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdabe033740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b5edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdabe033740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 0x556052b5f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdabe033740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556052a64f40 con 0x556052b618b0 2024-02-17T12:50:50.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdabe033740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560527934b0 con 0x556052b58380 2024-02-17T12:50:50.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdabe033740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556052a45670 con 0x556052b58d30 2024-02-17T12:50:50.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdabe033740 10 client.0 _ll_register_callbacks cb 0x556052ab2f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:50.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab9a01640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 0x556052b5f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58380 0x556052b5e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab89ff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b5edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab9a01640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 0x556052b5f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57288/0 (socket says 172.21.15.18:57288) 2024-02-17T12:50:50.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58380 0x556052b5e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:48404/0 (socket says 172.21.15.18:48404) 2024-02-17T12:50:50.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab9a01640 1 -- 172.21.15.18:0/686255601 learned_addr learned my addr 172.21.15.18:0/686255601 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:50.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab9a01640 1 -- 172.21.15.18:0/686255601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 msgr2=0x556052b5edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab9a01640 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b5edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab9a01640 1 -- 172.21.15.18:0/686255601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58380 msgr2=0x556052b5e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab9a01640 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58380 0x556052b5e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab9a01640 1 -- 172.21.15.18:0/686255601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa4009ae0 con 0x556052b618b0 2024-02-17T12:50:50.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab9200640 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58380 0x556052b5e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:50.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.190+0000 7fdab89ff640 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b5edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:50.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fdab9a01640 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 0x556052b5f310 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0126f0 tx=0x7fdaac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4431025cca9979d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:50.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fda9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:50.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fda9cff9640 1 -- 172.21.15.18:0/686255601 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0107d0 con 0x556052b618b0 2024-02-17T12:50:50.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fda9cff9640 1 -- 172.21.15.18:0/686255601 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdaac010930 con 0x556052b618b0 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fdabe033740 1 -- 172.21.15.18:0/686255601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556052a99f40 con 0x556052b618b0 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fda9cff9640 1 -- 172.21.15.18:0/686255601 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaac010af0 con 0x556052b618b0 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fdabe033740 10 client.15960 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fdabe033740 1 -- 172.21.15.18:0/686255601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556052abe320 con 0x556052b618b0 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fda9cff9640 1 -- 172.21.15.18:0/686255601 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556052a99f40 con 0x556052b618b0 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fda9cff9640 1 -- 172.21.15.18:0/686255601 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdaac0113e0 con 0x556052b618b0 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fdabe033740 10 client.15960 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fdabe033740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fdabe033740 20 client.15960 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fdabe033740 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556052b71f80 0x556052b743a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fdabe033740 4 client.15960 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.191+0000 7fdabe033740 1 -- 172.21.15.18:0/686255601 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556052ab6310 con 0x556052b71f80 2024-02-17T12:50:50.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.192+0000 7fdab9200640 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556052b71f80 0x556052b743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.192+0000 7fdab9200640 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556052b71f80 0x556052b743a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b531f3a8f02a626 server_cookie=837e7c463c2a10a8 in_seq=0 out_seq=0 2024-02-17T12:50:50.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.192+0000 7fda9cff9640 10 client.15960 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:50.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.193+0000 7fda9cff9640 1 -- 172.21.15.18:0/686255601 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556052ab6310 con 0x556052b71f80 2024-02-17T12:50:50.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.193+0000 7fda9cff9640 10 client.15960 handle_command_reply: tid=0 2024-02-17T12:50:50.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.193+0000 7fdabe033740 1 client.15960 shutdown 2024-02-17T12:50:50.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.193+0000 7fdabe033740 1 -- 172.21.15.18:0/686255601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 msgr2=0x556052b5f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.193+0000 7fdabe033740 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 0x556052b5f310 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0126f0 tx=0x7fdaac004ab0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.194+0000 7fdabe033740 1 -- 172.21.15.18:0/686255601 shutdown_connections 2024-02-17T12:50:50.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.194+0000 7fdabe033740 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556052b71f80 0x556052b743a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.194+0000 7fdabe033740 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b618b0 0x556052b5f310 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.194+0000 7fda9cff9640 0 client.15960 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:50.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.194+0000 7fdabe033740 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b5edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.194+0000 7fdabe033740 1 --2- 172.21.15.18:0/686255601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b58380 0x556052b5e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.194+0000 7fdabe033740 1 -- 172.21.15.18:0/686255601 >> 172.21.15.18:0/686255601 conn(0x556052b52840 msgr2=0x556052b547a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:50.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.194+0000 7fdabe033740 1 -- 172.21.15.18:0/686255601 shutdown_connections 2024-02-17T12:50:50.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.194+0000 7fdabe033740 1 -- 172.21.15.18:0/686255601 wait complete. 2024-02-17T12:50:50.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.194+0000 7fdabe033740 20 client.15960 trim_cache size 0 max 16384 2024-02-17T12:50:50.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdabe033740 1 Processor -- start 2024-02-17T12:50:50.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdabe033740 1 -- start start 2024-02-17T12:50:50.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdabe033740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 0x556052b59ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdabe033740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b5a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdabe033740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b618b0 0x556052b5ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdabe033740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556052b57610 con 0x556052b58380 2024-02-17T12:50:50.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdabe033740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556052b57cd0 con 0x556052b618b0 2024-02-17T12:50:50.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdabe033740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556052b612f0 con 0x556052b58d30 2024-02-17T12:50:50.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdab9200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 0x556052b59ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdab9200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 0x556052b59ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57294/0 (socket says 172.21.15.18:57294) 2024-02-17T12:50:50.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdab9200640 1 -- 172.21.15.18:0/408220807 learned_addr learned my addr 172.21.15.18:0/408220807 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:50.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdab9a01640 1 --2- 172.21.15.18:0/408220807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b618b0 0x556052b5ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdab89ff640 1 --2- 172.21.15.18:0/408220807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b5a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdab9a01640 1 -- 172.21.15.18:0/408220807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 msgr2=0x556052b5a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdab9a01640 1 --2- 172.21.15.18:0/408220807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b5a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdab9a01640 1 -- 172.21.15.18:0/408220807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 msgr2=0x556052b59ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdab9a01640 1 --2- 172.21.15.18:0/408220807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 0x556052b59ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdab9a01640 1 -- 172.21.15.18:0/408220807 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556052a5ac00 con 0x556052b618b0 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.210+0000 7fdab9200640 1 --2- 172.21.15.18:0/408220807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 0x556052b59ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.211+0000 7fdab9a01640 1 --2- 172.21.15.18:0/408220807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b618b0 0x556052b5ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0126c0 tx=0x7fdaac010400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77262469f5aa871a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.211+0000 7fda9effd640 1 -- 172.21.15.18:0/408220807 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0047b0 con 0x556052b618b0 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.211+0000 7fda9effd640 1 -- 172.21.15.18:0/408220807 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdaac0113e0 con 0x556052b618b0 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.211+0000 7fda9effd640 1 -- 172.21.15.18:0/408220807 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac010e40 con 0x556052b618b0 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 -- 172.21.15.18:0/408220807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b618b0 msgr2=0x556052b5ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 --2- 172.21.15.18:0/408220807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b618b0 0x556052b5ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0126c0 tx=0x7fdaac010400 comp rx=0 tx=0).stop 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 -- 172.21.15.18:0/408220807 shutdown_connections 2024-02-17T12:50:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 --2- 172.21.15.18:0/408220807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b618b0 0x556052b5ede0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 --2- 172.21.15.18:0/408220807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b5a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 --2- 172.21.15.18:0/408220807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 0x556052b59ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 -- 172.21.15.18:0/408220807 >> 172.21.15.18:0/408220807 conn(0x556052b52840 msgr2=0x556052b52c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:50.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 -- 172.21.15.18:0/408220807 shutdown_connections 2024-02-17T12:50:50.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 -- 172.21.15.18:0/408220807 wait complete. 2024-02-17T12:50:50.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 -- start start 2024-02-17T12:50:50.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 Processor -- start 2024-02-17T12:50:50.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 0x556052b68cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b67090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b618b0 0x556052b67610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556052b67d10 con 0x556052b58380 2024-02-17T12:50:50.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556052b67e80 con 0x556052b618b0 2024-02-17T12:50:50.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556052bcc8a0 con 0x556052b58d30 2024-02-17T12:50:50.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdabe033740 10 client.0 _ll_register_callbacks cb 0x7fdaa4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:50.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.212+0000 7fdab9200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 0x556052b68cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdab9200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 0x556052b68cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57310/0 (socket says 172.21.15.18:57310) 2024-02-17T12:50:50.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdab9200640 1 -- 172.21.15.18:0/3434907779 learned_addr learned my addr 172.21.15.18:0/3434907779 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:50.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdab9a01640 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b618b0 0x556052b67610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdab89ff640 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b67090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdab9200640 1 -- 172.21.15.18:0/3434907779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 msgr2=0x556052b67090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdab9200640 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b67090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdab9200640 1 -- 172.21.15.18:0/3434907779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b618b0 msgr2=0x556052b67610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdab9200640 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b618b0 0x556052b67610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdab9200640 1 -- 172.21.15.18:0/3434907779 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556052a820b0 con 0x556052b58380 2024-02-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdab9200640 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 0x556052b68cf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdaa401c040 tx=0x7fdaa4005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7205730fbfb16ca server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fda9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fda9cff9640 1 -- 172.21.15.18:0/3434907779 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40186c0 con 0x556052b58380 2024-02-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fda9cff9640 1 -- 172.21.15.18:0/3434907779 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdaa4005040 con 0x556052b58380 2024-02-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdabe033740 1 -- 172.21.15.18:0/3434907779 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556052ae5900 con 0x556052b58380 2024-02-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fda9cff9640 1 -- 172.21.15.18:0/3434907779 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaa4005200 con 0x556052b58380 2024-02-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdabe033740 10 client.15966 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:50.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fdabe033740 1 -- 172.21.15.18:0/3434907779 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556052abe320 con 0x556052b58380 2024-02-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.213+0000 7fda9cff9640 1 -- 172.21.15.18:0/3434907779 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556052ae5900 con 0x556052b58380 2024-02-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.214+0000 7fda9cff9640 1 -- 172.21.15.18:0/3434907779 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdaa4025450 con 0x556052b58380 2024-02-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.214+0000 7fdabe033740 10 client.15966 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.214+0000 7fdabe033740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.214+0000 7fdabe033740 20 client.15966 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.214+0000 7fdabe033740 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556052bce010 0x556052bd0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.214+0000 7fdabe033740 4 client.15966 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.214+0000 7fdabe033740 1 -- 172.21.15.18:0/3434907779 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5560527bf860 con 0x556052bce010 2024-02-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.214+0000 7fdab89ff640 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556052bce010 0x556052bd0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.214+0000 7fda9cff9640 10 client.15966 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:50.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.214+0000 7fdab89ff640 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556052bce010 0x556052bd0400 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c1fb04218636d3 server_cookie=3d9ec9ea113a1fcb in_seq=0 out_seq=0 2024-02-17T12:50:50.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fda9cff9640 1 -- 172.21.15.18:0/3434907779 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5560527bf860 con 0x556052bce010 2024-02-17T12:50:50.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fda9cff9640 10 client.15966 handle_command_reply: tid=0 2024-02-17T12:50:50.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fdabe033740 1 client.15966 shutdown 2024-02-17T12:50:50.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fdabe033740 1 -- 172.21.15.18:0/3434907779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 msgr2=0x556052b68cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fdabe033740 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 0x556052b68cf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdaa401c040 tx=0x7fdaa4005d30 comp rx=0 tx=0).stop 2024-02-17T12:50:50.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fdabe033740 1 -- 172.21.15.18:0/3434907779 shutdown_connections 2024-02-17T12:50:50.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fdabe033740 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556052bce010 0x556052bd0400 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fdabe033740 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556052b618b0 0x556052b67610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fdabe033740 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556052b58d30 0x556052b67090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fda9cff9640 0 client.15966 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:50.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fdabe033740 1 --2- 172.21.15.18:0/3434907779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556052b58380 0x556052b68cf0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fdabe033740 1 -- 172.21.15.18:0/3434907779 >> 172.21.15.18:0/3434907779 conn(0x556052b52840 msgr2=0x556052b59ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:50.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fdabe033740 1 -- 172.21.15.18:0/3434907779 shutdown_connections 2024-02-17T12:50:50.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fdabe033740 1 -- 172.21.15.18:0/3434907779 wait complete. 2024-02-17T12:50:50.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.216+0000 7fdabe033740 20 client.15966 trim_cache size 0 max 16384 2024-02-17T12:50:50.216 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:50:50.216 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:50:50.217 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "b49c04fd-75c8-49bf-9208-f7de316aa88d", 2024-02-17T12:50:50.217 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:50:50.217 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:50:50.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.217+0000 7fdabb464640 1 -- 172.21.15.18:0/3015764693 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdaa00a2350 msgr2=0x7fdaa00a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.217+0000 7fdabb464640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdaa00a2350 0x7fdaa00a4810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdaac005fd0 tx=0x7fdaac004a40 comp rx=0 tx=0).stop 2024-02-17T12:50:50.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.217+0000 7fdabb464640 1 -- 172.21.15.18:0/3015764693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdab4106450 msgr2=0x7fdab419dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.217+0000 7fdabb464640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdab4106450 0x7fdab419dad0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdab0005ab0 tx=0x7fdab002cd80 comp rx=0 tx=0).stop 2024-02-17T12:50:50.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.217+0000 7fdabb464640 1 -- 172.21.15.18:0/3015764693 shutdown_connections 2024-02-17T12:50:50.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.217+0000 7fdabb464640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdaa00a2350 0x7fdaa00a4810 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.217+0000 7fdabb464640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab41976a0 0x7fdab4197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.217+0000 7fdabb464640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdab4106450 0x7fdab419dad0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.217+0000 7fdabb464640 1 --2- 172.21.15.18:0/3015764693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdab4101380 0x7fdab419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.217+0000 7fdabb464640 1 -- 172.21.15.18:0/3015764693 >> 172.21.15.18:0/3015764693 conn(0x7fdab40fcc20 msgr2=0x7fdab40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:50.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.217+0000 7fdabb464640 1 -- 172.21.15.18:0/3015764693 shutdown_connections 2024-02-17T12:50:50.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.217+0000 7fdabb464640 1 -- 172.21.15.18:0/3015764693 wait complete. 2024-02-17T12:50:50.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:50:50.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:50:50.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 16630 20.92 MB/sec warmup 5 sec latency 6.149 ms 2024-02-17T12:50:50.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.335+0000 7f27f6c40640 1 -- 172.21.15.18:0/2224187161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27f0106110 msgr2=0x7f27f01064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.335+0000 7f27f6c40640 1 --2- 172.21.15.18:0/2224187161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27f0106110 0x7f27f01064f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009900 tx=0x7f27e002bc30 comp rx=0 tx=0).stop 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.335+0000 7f27f6c40640 1 -- 172.21.15.18:0/2224187161 shutdown_connections 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.335+0000 7f27f6c40640 1 --2- 172.21.15.18:0/2224187161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27f0100c90 0x7f27f010ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.335+0000 7f27f6c40640 1 --2- 172.21.15.18:0/2224187161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27f01002d0 0x7f27f0100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.335+0000 7f27f6c40640 1 --2- 172.21.15.18:0/2224187161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27f0106110 0x7f27f01064f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.335+0000 7f27f6c40640 1 -- 172.21.15.18:0/2224187161 >> 172.21.15.18:0/2224187161 conn(0x7f27f0076e20 msgr2=0x7f27f0077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.335+0000 7f27f6c40640 1 -- 172.21.15.18:0/2224187161 shutdown_connections 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.335+0000 7f27f6c40640 1 -- 172.21.15.18:0/2224187161 wait complete. 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27f6c40640 1 Processor -- start 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27f6c40640 1 -- start start 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27f6c40640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27f01002d0 0x7f27f010f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27f6c40640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27f0100c90 0x7f27f010fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27f6c40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27f0106110 0x7f27f01102d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27f6c40640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f01a5b40 con 0x7f27f0100c90 2024-02-17T12:50:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27f6c40640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f01a5cb0 con 0x7f27f0106110 2024-02-17T12:50:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27f6c40640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f01a5e70 con 0x7f27f01002d0 2024-02-17T12:50:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27effff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27f0100c90 0x7f27f010fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27effff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27f0100c90 0x7f27f010fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57328/0 (socket says 172.21.15.18:57328) 2024-02-17T12:50:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27effff640 1 -- 172.21.15.18:0/210029703 learned_addr learned my addr 172.21.15.18:0/210029703 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27f49dc640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27f01002d0 0x7f27f010f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27f51dd640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27f0106110 0x7f27f01102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27effff640 1 -- 172.21.15.18:0/210029703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27f01002d0 msgr2=0x7f27f010f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27effff640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27f01002d0 0x7f27f010f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.336+0000 7f27effff640 1 -- 172.21.15.18:0/210029703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27f0106110 msgr2=0x7f27f01102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.337+0000 7f27effff640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27f0106110 0x7f27f01102d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.337+0000 7f27effff640 1 -- 172.21.15.18:0/210029703 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e0009560 con 0x7f27f0100c90 2024-02-17T12:50:50.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.337+0000 7f27effff640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27f0100c90 0x7f27f010fd90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f27d800e9f0 tx=0x7f27d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:50.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.337+0000 7f27edffb640 1 -- 172.21.15.18:0/210029703 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8009700 con 0x7f27f0100c90 2024-02-17T12:50:50.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.337+0000 7f27edffb640 1 -- 172.21.15.18:0/210029703 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27d800cd80 con 0x7f27f0100c90 2024-02-17T12:50:50.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.337+0000 7f27f6c40640 1 -- 172.21.15.18:0/210029703 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f01a6150 con 0x7f27f0100c90 2024-02-17T12:50:50.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.337+0000 7f27edffb640 1 -- 172.21.15.18:0/210029703 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8018920 con 0x7f27f0100c90 2024-02-17T12:50:50.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.337+0000 7f27f6c40640 1 -- 172.21.15.18:0/210029703 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f01a6660 con 0x7f27f0100c90 2024-02-17T12:50:50.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.339+0000 7f27edffb640 1 -- 172.21.15.18:0/210029703 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f27d8018a80 con 0x7f27f0100c90 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f980d740 1 Processor -- start 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f980d740 1 -- start start 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f980d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f980d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278afbd0 0x55c4278b8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f980d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278b87b0 0x55c4278baba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f980d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42781e280 con 0x55c4278afbd0 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f980d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42781b7b0 con 0x55c4278b87b0 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f980d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c427816f90 con 0x55c4278af2b0 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f49dc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278afbd0 0x55c4278b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27edffb640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f27dc0a2350 0x7f27dc0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27effff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278b87b0 0x55c4278baba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f49dc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278afbd0 0x55c4278b8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57336/0 (socket says 172.21.15.18:57336) 2024-02-17T12:50:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f49dc640 1 -- 172.21.15.18:0/1490434972 learned_addr learned my addr 172.21.15.18:0/1490434972 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f51dd640 1 --2- 172.21.15.18:0/1490434972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f51dd640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f27dc0a2350 0x7f27dc0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.340+0000 7f27f49dc640 1 -- 172.21.15.18:0/1490434972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 msgr2=0x55c4278af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.341+0000 7f27f49dc640 1 --2- 172.21.15.18:0/1490434972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.341+0000 7f27f49dc640 1 -- 172.21.15.18:0/1490434972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278b87b0 msgr2=0x55c4278baba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.341+0000 7f27edffb640 1 -- 172.21.15.18:0/210029703 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f27d8014070 con 0x7f27f0100c90 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.341+0000 7f27f49dc640 1 --2- 172.21.15.18:0/1490434972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278b87b0 0x55c4278baba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.341+0000 7f27f49dc640 1 -- 172.21.15.18:0/1490434972 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4277d5570 con 0x55c4278afbd0 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.341+0000 7f27f49dc640 1 --2- 172.21.15.18:0/1490434972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278afbd0 0x55c4278b8270 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009900 tx=0x7f27e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2a113bf4fc1581 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.341+0000 7f27f51dd640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f27dc0a2350 0x7f27dc0a4810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f27e8005fd0 tx=0x7f27e8004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.341+0000 7f27e6ffd640 1 -- 172.21.15.18:0/1490434972 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00048b0 con 0x55c4278afbd0 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.341+0000 7f27e6ffd640 1 -- 172.21.15.18:0/1490434972 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27e0004a10 con 0x55c4278afbd0 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.341+0000 7f27e6ffd640 1 -- 172.21.15.18:0/1490434972 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00051c0 con 0x55c4278afbd0 2024-02-17T12:50:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.342+0000 7f27f980d740 1 -- 172.21.15.18:0/1490434972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278afbd0 msgr2=0x55c4278b8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.342+0000 7f27f980d740 1 --2- 172.21.15.18:0/1490434972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278afbd0 0x55c4278b8270 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009900 tx=0x7f27e00046c0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.342+0000 7f27f980d740 1 -- 172.21.15.18:0/1490434972 shutdown_connections 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.342+0000 7f27f980d740 1 --2- 172.21.15.18:0/1490434972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278b87b0 0x55c4278baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.342+0000 7f27f980d740 1 --2- 172.21.15.18:0/1490434972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278afbd0 0x55c4278b8270 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.342+0000 7f27f980d740 1 --2- 172.21.15.18:0/1490434972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.342+0000 7f27f980d740 1 -- 172.21.15.18:0/1490434972 >> 172.21.15.18:0/1490434972 conn(0x55c4278a9990 msgr2=0x55c4278a9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.342+0000 7f27f980d740 1 -- 172.21.15.18:0/1490434972 shutdown_connections 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.342+0000 7f27f980d740 1 -- 172.21.15.18:0/1490434972 wait complete. 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.342+0000 7f27f980d740 1 -- start start 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.342+0000 7f27f980d740 1 Processor -- start 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f980d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f980d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278afbd0 0x55c4278b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f980d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 0x55c4278b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f980d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4274ea430 con 0x55c4278b87b0 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f980d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4274a6e00 con 0x55c4278afbd0 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f980d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42779bfd0 con 0x55c4278af2b0 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f980d740 10 client.0 _ll_register_callbacks cb 0x55c4277f1fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f51dd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 0x55c4278b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27effff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278afbd0 0x55c4278b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f51dd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 0x55c4278b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57348/0 (socket says 172.21.15.18:57348) 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f51dd640 1 -- 172.21.15.18:0/1444724145 learned_addr learned my addr 172.21.15.18:0/1444724145 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f49dc640 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27effff640 1 -- 172.21.15.18:0/1444724145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 msgr2=0x55c4278b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27effff640 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27effff640 1 -- 172.21.15.18:0/1444724145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 msgr2=0x55c4278b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27effff640 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 0x55c4278b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27effff640 1 -- 172.21.15.18:0/1444724145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f010d790 con 0x55c4278afbd0 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f51dd640 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 0x55c4278b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27effff640 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278afbd0 0x55c4278b5570 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f27d80cbcb0 tx=0x7f27d8010f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc15f34819da412c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27cffff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27cffff640 1 -- 172.21.15.18:0/1444724145 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d80cc190 con 0x55c4278afbd0 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27f980d740 1 -- 172.21.15.18:0/1444724145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4275167e0 con 0x55c4278afbd0 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.343+0000 7f27cffff640 1 -- 172.21.15.18:0/1444724145 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f27d809af30 con 0x55c4278afbd0 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.344+0000 7f27cffff640 1 -- 172.21.15.18:0/1444724145 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27d809efd0 con 0x55c4278afbd0 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.344+0000 7f27f980d740 10 client.25442 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.344+0000 7f27f980d740 1 -- 172.21.15.18:0/1444724145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c427815610 con 0x55c4278afbd0 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.344+0000 7f27cffff640 1 -- 172.21.15.18:0/1444724145 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c4275167e0 con 0x55c4278afbd0 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.344+0000 7f27cffff640 1 -- 172.21.15.18:0/1444724145 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f27d8099df0 con 0x55c4278afbd0 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.344+0000 7f27f980d740 10 client.25442 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.344+0000 7f27f980d740 10 client.25442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.344+0000 7f27f980d740 20 client.25442 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.344+0000 7f27f980d740 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c4278c88c0 0x55c4278cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.344+0000 7f27f980d740 4 client.25442 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.344+0000 7f27f980d740 1 -- 172.21.15.18:0/1444724145 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c427553720 con 0x55c4278c88c0 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.344+0000 7f27f49dc640 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c4278c88c0 0x55c4278cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27cffff640 10 client.25442 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27f49dc640 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c4278c88c0 0x55c4278cacb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f66756901b413ce server_cookie=12b664a8be3f3e3 in_seq=0 out_seq=0 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27cffff640 1 -- 172.21.15.18:0/1444724145 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c427553720 con 0x55c4278c88c0 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27cffff640 10 client.25442 handle_command_reply: tid=0 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27f980d740 1 client.25442 shutdown 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27f980d740 1 -- 172.21.15.18:0/1444724145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278afbd0 msgr2=0x55c4278b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27f980d740 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278afbd0 0x55c4278b5570 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f27d80cbcb0 tx=0x7f27d8010f50 comp rx=0 tx=0).stop 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27f980d740 1 -- 172.21.15.18:0/1444724145 shutdown_connections 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27f980d740 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c4278c88c0 0x55c4278cacb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27f980d740 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 0x55c4278b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27f980d740 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278afbd0 0x55c4278b5570 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27cffff640 0 client.25442 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:50.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27f980d740 1 --2- 172.21.15.18:0/1444724145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.345+0000 7f27f980d740 1 -- 172.21.15.18:0/1444724145 >> 172.21.15.18:0/1444724145 conn(0x55c4278a9990 msgr2=0x55c4278ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:50.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.346+0000 7f27f980d740 1 -- 172.21.15.18:0/1444724145 shutdown_connections 2024-02-17T12:50:50.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.346+0000 7f27f980d740 1 -- 172.21.15.18:0/1444724145 wait complete. 2024-02-17T12:50:50.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.346+0000 7f27f980d740 20 client.25442 trim_cache size 0 max 16384 2024-02-17T12:50:50.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f980d740 1 Processor -- start 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f980d740 1 -- start start 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f980d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278af2b0 0x55c4278b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f980d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278afbd0 0x55c4278cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f980d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278b87b0 0x55c4278b7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f980d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4278b0420 con 0x55c4278af2b0 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f980d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4278aa9e0 con 0x55c4278b87b0 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f980d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4278b1160 con 0x55c4278afbd0 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f49dc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278af2b0 0x55c4278b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f49dc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278af2b0 0x55c4278b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57360/0 (socket says 172.21.15.18:57360) 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f49dc640 1 -- 172.21.15.18:0/3338173692 learned_addr learned my addr 172.21.15.18:0/3338173692 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f51dd640 1 --2- 172.21.15.18:0/3338173692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278b87b0 0x55c4278b7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27effff640 1 --2- 172.21.15.18:0/3338173692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278afbd0 0x55c4278cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f49dc640 1 -- 172.21.15.18:0/3338173692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278afbd0 msgr2=0x55c4278cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f49dc640 1 --2- 172.21.15.18:0/3338173692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278afbd0 0x55c4278cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f49dc640 1 -- 172.21.15.18:0/3338173692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278b87b0 msgr2=0x55c4278b7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f49dc640 1 --2- 172.21.15.18:0/3338173692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278b87b0 0x55c4278b7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.361+0000 7f27f49dc640 1 -- 172.21.15.18:0/3338173692 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4277b1ac0 con 0x55c4278af2b0 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27f49dc640 1 --2- 172.21.15.18:0/3338173692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278af2b0 0x55c4278b5c70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55c4278aed80 tx=0x7f27e0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116e5f2404973ff0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27e67fc640 1 -- 172.21.15.18:0/3338173692 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0050d70 con 0x55c4278af2b0 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27e67fc640 1 -- 172.21.15.18:0/3338173692 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27e00047b0 con 0x55c4278af2b0 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27e67fc640 1 -- 172.21.15.18:0/3338173692 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0005170 con 0x55c4278af2b0 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27f980d740 1 -- 172.21.15.18:0/3338173692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278af2b0 msgr2=0x55c4278b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27f980d740 1 --2- 172.21.15.18:0/3338173692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278af2b0 0x55c4278b5c70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55c4278aed80 tx=0x7f27e0005fd0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27f980d740 1 -- 172.21.15.18:0/3338173692 shutdown_connections 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27f980d740 1 --2- 172.21.15.18:0/3338173692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278b87b0 0x55c4278b7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27f980d740 1 --2- 172.21.15.18:0/3338173692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278afbd0 0x55c4278cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27f980d740 1 --2- 172.21.15.18:0/3338173692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278af2b0 0x55c4278b5c70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27f980d740 1 -- 172.21.15.18:0/3338173692 >> 172.21.15.18:0/3338173692 conn(0x55c4278a9990 msgr2=0x55c4278a9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27f980d740 1 -- 172.21.15.18:0/3338173692 shutdown_connections 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.362+0000 7f27f980d740 1 -- 172.21.15.18:0/3338173692 wait complete. 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f980d740 1 -- start start 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f980d740 1 Processor -- start 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f980d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278bf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f980d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278afbd0 0x55c4278bbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f980d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 0x55c4278bc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f980d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4278bc8a0 con 0x55c4278b87b0 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f980d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4278bca10 con 0x55c4278afbd0 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f980d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42791edb0 con 0x55c4278af2b0 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f980d740 10 client.0 _ll_register_callbacks cb 0x7f27e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f51dd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 0x55c4278bc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f49dc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27effff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278afbd0 0x55c4278bbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f51dd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 0x55c4278bc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57364/0 (socket says 172.21.15.18:57364) 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f49dc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278bf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:55968/0 (socket says 172.21.15.18:55968) 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.363+0000 7f27f51dd640 1 -- 172.21.15.18:0/3908050113 learned_addr learned my addr 172.21.15.18:0/3908050113 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27f51dd640 1 -- 172.21.15.18:0/3908050113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 msgr2=0x55c4278bf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27f51dd640 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278bf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27f51dd640 1 -- 172.21.15.18:0/3908050113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278afbd0 msgr2=0x55c4278bbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27f51dd640 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278afbd0 0x55c4278bbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27f51dd640 1 -- 172.21.15.18:0/3908050113 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4277d5570 con 0x55c4278b87b0 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27f49dc640 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278bf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27effff640 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278afbd0 0x55c4278bbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27f51dd640 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 0x55c4278bc1d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f27e800f000 tx=0x7f27e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c314958e6c7e3da3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27cffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27cffff640 1 -- 172.21.15.18:0/3908050113 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8026070 con 0x55c4278b87b0 2024-02-17T12:50:50.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27cffff640 1 -- 172.21.15.18:0/3908050113 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f27e8021e40 con 0x55c4278b87b0 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27f980d740 1 -- 172.21.15.18:0/3908050113 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4275a2890 con 0x55c4278b87b0 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27cffff640 1 -- 172.21.15.18:0/3908050113 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27e802b420 con 0x55c4278b87b0 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27f980d740 10 client.15984 fetch_fsmap learned FSMap version 26 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27f980d740 1 -- 172.21.15.18:0/3908050113 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c4277d76a0 con 0x55c4278b87b0 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27cffff640 1 -- 172.21.15.18:0/3908050113 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c4275a2890 con 0x55c4278b87b0 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.364+0000 7f27cffff640 1 -- 172.21.15.18:0/3908050113 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f27e8010910 con 0x55c4278b87b0 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f980d740 10 client.15984 fetch_fsmap finished waiting for FSMap version 26 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f980d740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f980d740 20 client.15984 populate_metadata read hostname 'smithi018' 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f980d740 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c427920550 0x55c427922970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f980d740 4 client.15984 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f980d740 1 -- 172.21.15.18:0/3908050113 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c4275054e0 con 0x55c427920550 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f49dc640 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c427920550 0x55c427922970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f49dc640 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c427920550 0x55c427922970 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=975687718654a216 server_cookie=a70353e67ae7eca2 in_seq=0 out_seq=0 2024-02-17T12:50:50.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27cffff640 10 client.15984 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27cffff640 1 -- 172.21.15.18:0/3908050113 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c4275054e0 con 0x55c427920550 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27cffff640 10 client.15984 handle_command_reply: tid=0 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f980d740 1 client.15984 shutdown 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f980d740 1 -- 172.21.15.18:0/3908050113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 msgr2=0x55c4278bc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f980d740 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 0x55c4278bc1d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f27e800f000 tx=0x7f27e8004a40 comp rx=0 tx=0).stop 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f980d740 1 -- 172.21.15.18:0/3908050113 shutdown_connections 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.365+0000 7f27f980d740 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c427920550 0x55c427922970 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27f980d740 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4278b87b0 0x55c4278bc1d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27f980d740 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4278afbd0 0x55c4278bbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27cffff640 0 client.15984 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27f980d740 1 --2- 172.21.15.18:0/3908050113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c4278af2b0 0x55c4278bf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27f980d740 1 -- 172.21.15.18:0/3908050113 >> 172.21.15.18:0/3908050113 conn(0x55c4278a9990 msgr2=0x55c4278cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:50.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27f980d740 1 -- 172.21.15.18:0/3908050113 shutdown_connections 2024-02-17T12:50:50.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27f980d740 1 -- 172.21.15.18:0/3908050113 wait complete. 2024-02-17T12:50:50.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27f980d740 20 client.15984 trim_cache size 0 max 16384 2024-02-17T12:50:50.365 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:50:50.365 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:50:50.365 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:50:50.365 INFO:teuthology.orchestra.run.smithi018.stdout: "b49c04fd-75c8-49bf-9208-f7de316aa88d": { 2024-02-17T12:50:50.365 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:50:50.365 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "b49c04fd-75c8-49bf-9208-f7de316aa88d", 2024-02-17T12:50:50.365 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:50:50.366 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:50:50.366 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:50:50.366 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:50:50.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27f6c40640 1 -- 172.21.15.18:0/210029703 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f27dc0a2350 msgr2=0x7f27dc0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27f6c40640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f27dc0a2350 0x7f27dc0a4810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f27e8005fd0 tx=0x7f27e8004ab0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27f6c40640 1 -- 172.21.15.18:0/210029703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27f0100c90 msgr2=0x7f27f010fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:50:50.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27f6c40640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27f0100c90 0x7f27f010fd90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f27d800e9f0 tx=0x7f27d800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.366+0000 7f27f6c40640 1 -- 172.21.15.18:0/210029703 shutdown_connections 2024-02-17T12:50:50.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.367+0000 7f27f6c40640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f27dc0a2350 0x7f27dc0a4810 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.367+0000 7f27f6c40640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27f0106110 0x7f27f01102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.367+0000 7f27f6c40640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27f0100c90 0x7f27f010fd90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.367+0000 7f27f6c40640 1 --2- 172.21.15.18:0/210029703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27f01002d0 0x7f27f010f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:50:50.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.367+0000 7f27f6c40640 1 -- 172.21.15.18:0/210029703 >> 172.21.15.18:0/210029703 conn(0x7f27f0076e20 msgr2=0x7f27f00fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:50:50.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.367+0000 7f27f6c40640 1 -- 172.21.15.18:0/210029703 shutdown_connections 2024-02-17T12:50:50.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:50:50.367+0000 7f27f6c40640 1 -- 172.21.15.18:0/210029703 wait complete. 2024-02-17T12:50:50.369 INFO:tasks.cephfs.filesystem:scrub status for tag:b49c04fd-75c8-49bf-9208-f7de316aa88d - {'path': '/', 'tag': 'b49c04fd-75c8-49bf-9208-f7de316aa88d', 'options': 'recursive,force'} 2024-02-17T12:50:50.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:50 smithi018 ceph-mon[32625]: pgmap v31: 97 pgs: 97 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 432 KiB/s wr, 121 op/s 2024-02-17T12:50:50.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:50 smithi119 ceph-mon[36188]: pgmap v31: 97 pgs: 97 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 432 KiB/s wr, 121 op/s 2024-02-17T12:50:50.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:50 smithi146 ceph-mon[36454]: pgmap v31: 97 pgs: 97 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 432 KiB/s wr, 121 op/s 2024-02-17T12:50:51.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 19439 20.30 MB/sec warmup 6 sec latency 15.109 ms 2024-02-17T12:50:51.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:51 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:50:51.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:51 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:50:51.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:51 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:50:51.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:51 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:50:51.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:51 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:50:51.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:51 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:50:51.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:51 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:50:51.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:51 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:50:51.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:51 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:50:51.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:51 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:50:51.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:51 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:50:51.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:51 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:50:52.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 22749 20.11 MB/sec warmup 7 sec latency 5.763 ms 2024-02-17T12:50:52.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:52 smithi018 ceph-mon[32625]: pgmap v32: 97 pgs: 97 active+clean; 6.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 543 KiB/s wr, 151 op/s 2024-02-17T12:50:52.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:52 smithi119 ceph-mon[36188]: pgmap v32: 97 pgs: 97 active+clean; 6.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 543 KiB/s wr, 151 op/s 2024-02-17T12:50:52.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:52 smithi146 ceph-mon[36454]: pgmap v32: 97 pgs: 97 active+clean; 6.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 543 KiB/s wr, 151 op/s 2024-02-17T12:50:53.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 25848 19.46 MB/sec warmup 8 sec latency 13.449 ms 2024-02-17T12:50:54.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 28703 19.30 MB/sec warmup 9 sec latency 161.439 ms 2024-02-17T12:50:54.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:54 smithi018 ceph-mon[32625]: pgmap v33: 97 pgs: 97 active+clean; 6.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 542 KiB/s wr, 146 op/s 2024-02-17T12:50:54.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:54 smithi119 ceph-mon[36188]: pgmap v33: 97 pgs: 97 active+clean; 6.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 542 KiB/s wr, 146 op/s 2024-02-17T12:50:54.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:54 smithi146 ceph-mon[36454]: pgmap v33: 97 pgs: 97 active+clean; 6.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 542 KiB/s wr, 146 op/s 2024-02-17T12:50:55.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 31952 19.14 MB/sec warmup 10 sec latency 6.459 ms 2024-02-17T12:50:55.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:55 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:50:55.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:55 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:50:55.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:55 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:50:56.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 34747 18.55 MB/sec warmup 11 sec latency 6.411 ms 2024-02-17T12:50:56.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:56 smithi018 ceph-mon[32625]: pgmap v34: 97 pgs: 97 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.3 MiB/s wr, 252 op/s 2024-02-17T12:50:56.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:56 smithi119 ceph-mon[36188]: pgmap v34: 97 pgs: 97 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.3 MiB/s wr, 252 op/s 2024-02-17T12:50:56.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:56 smithi146 ceph-mon[36454]: pgmap v34: 97 pgs: 97 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.3 MiB/s wr, 252 op/s 2024-02-17T12:50:57.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 37334 18.13 MB/sec warmup 12 sec latency 15.853 ms 2024-02-17T12:50:57.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:57 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:50:57.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:57 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:50:57.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:57 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:50:58.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 40035 17.86 MB/sec warmup 13 sec latency 39.554 ms 2024-02-17T12:50:58.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:58 smithi018 ceph-mon[32625]: pgmap v35: 97 pgs: 97 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.3 MiB/s wr, 248 op/s 2024-02-17T12:50:58.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:58 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:50:58.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:58 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:50:58.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:58 smithi119 ceph-mon[36188]: pgmap v35: 97 pgs: 97 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.3 MiB/s wr, 248 op/s 2024-02-17T12:50:58.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:58 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:50:58.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:58 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:50:58.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:58 smithi146 ceph-mon[36454]: pgmap v35: 97 pgs: 97 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.3 MiB/s wr, 248 op/s 2024-02-17T12:50:58.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:58 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:50:58.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:58 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:50:59.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 43016 17.89 MB/sec warmup 14 sec latency 85.584 ms 2024-02-17T12:50:59.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:59 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:50:59.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:59 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:50:59.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:50:59 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:50:59.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:59 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:50:59.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:59 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:50:59.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:50:59 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:51:00.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:59 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:51:00.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:59 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:51:00.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:50:59 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:51:00.259 INFO:tasks.workunit.client.0.smithi018.stdout: 1 46315 17.89 MB/sec warmup 15 sec latency 17.496 ms 2024-02-17T12:51:00.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:00 smithi119 ceph-mon[36188]: pgmap v36: 97 pgs: 97 active+clean; 17 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.7 MiB/s wr, 274 op/s 2024-02-17T12:51:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.879+0000 7f2a40623640 1 -- 172.21.15.18:0/1774504100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a380699a0 msgr2=0x7f2a38069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.879+0000 7f2a40623640 1 --2- 172.21.15.18:0/1774504100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a380699a0 0x7f2a38069e20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009900 tx=0x7f2a2802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.879+0000 7f2a40623640 1 -- 172.21.15.18:0/1774504100 shutdown_connections 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.879+0000 7f2a40623640 1 --2- 172.21.15.18:0/1774504100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a3806a360 0x7f2a3810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.879+0000 7f2a40623640 1 --2- 172.21.15.18:0/1774504100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a380699a0 0x7f2a38069e20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.879+0000 7f2a40623640 1 --2- 172.21.15.18:0/1774504100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a381072b0 0x7f2a38107690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.879+0000 7f2a40623640 1 -- 172.21.15.18:0/1774504100 >> 172.21.15.18:0/1774504100 conn(0x7f2a38076ee0 msgr2=0x7f2a380772f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.879+0000 7f2a40623640 1 -- 172.21.15.18:0/1774504100 shutdown_connections 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.879+0000 7f2a40623640 1 -- 172.21.15.18:0/1774504100 wait complete. 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a40623640 1 Processor -- start 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a40623640 1 -- start start 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a40623640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a380699a0 0x7f2a38197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a40623640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a3806a360 0x7f2a38197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a40623640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a381072b0 0x7f2a3819bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a40623640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819c4f0 con 0x7f2a381072b0 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a40623640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819c660 con 0x7f2a380699a0 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a40623640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819c820 con 0x7f2a3806a360 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a3ebc0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a381072b0 0x7f2a3819bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a3ebc0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a381072b0 0x7f2a3819bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33272/0 (socket says 172.21.15.18:33272) 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a3ebc0640 1 -- 172.21.15.18:0/1467323033 learned_addr learned my addr 172.21.15.18:0/1467323033 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a3dbbe640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a3806a360 0x7f2a38197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.880+0000 7f2a3e3bf640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a380699a0 0x7f2a38197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.881+0000 7f2a3ebc0640 1 -- 172.21.15.18:0/1467323033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a3806a360 msgr2=0x7f2a38197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.881+0000 7f2a3ebc0640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a3806a360 0x7f2a38197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.881+0000 7f2a3ebc0640 1 -- 172.21.15.18:0/1467323033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a380699a0 msgr2=0x7f2a38197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.881+0000 7f2a3ebc0640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a380699a0 0x7f2a38197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.881+0000 7f2a3ebc0640 1 -- 172.21.15.18:0/1467323033 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a28009560 con 0x7f2a381072b0 2024-02-17T12:51:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.881+0000 7f2a3ebc0640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a381072b0 0x7f2a3819bef0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000d680 tx=0x7f2a3000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.881+0000 7f2a2f7fe640 1 -- 172.21.15.18:0/1467323033 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30005220 con 0x7f2a381072b0 2024-02-17T12:51:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.881+0000 7f2a2f7fe640 1 -- 172.21.15.18:0/1467323033 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a30009d00 con 0x7f2a381072b0 2024-02-17T12:51:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.881+0000 7f2a40623640 1 -- 172.21.15.18:0/1467323033 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3819cb00 con 0x7f2a381072b0 2024-02-17T12:51:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.881+0000 7f2a2f7fe640 1 -- 172.21.15.18:0/1467323033 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30004dd0 con 0x7f2a381072b0 2024-02-17T12:51:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.881+0000 7f2a40623640 1 -- 172.21.15.18:0/1467323033 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a381a4380 con 0x7f2a381072b0 2024-02-17T12:51:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.883+0000 7f2a2f7fe640 1 -- 172.21.15.18:0/1467323033 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f2a30009820 con 0x7f2a381072b0 2024-02-17T12:51:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a431f7740 1 Processor -- start 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a431f7740 1 -- start start 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a2f7fe640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2a240a23f0 0x7f2a240a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a431f7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550433ad0 0x560550432820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a431f7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550432d60 0x5605504331e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a431f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x56055043e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a431f7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605503a19f0 con 0x560550432d60 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a431f7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56055039ef20 con 0x560550433ad0 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a431f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56055039a700 con 0x56055043c2e0 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a3dbbe640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2a240a23f0 0x7f2a240a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a2f7fe640 1 -- 172.21.15.18:0/1467323033 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2a300cdc50 con 0x7f2a381072b0 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a3ebc0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x56055043e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a3ebc0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x56055043e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:59396/0 (socket says 172.21.15.18:59396) 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a3dbbe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550432d60 0x5605504331e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a3e3bf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550433ad0 0x560550432820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.884+0000 7f2a3ebc0640 1 -- 172.21.15.18:0/2076887751 learned_addr learned my addr 172.21.15.18:0/2076887751 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.885+0000 7f2a3ebc0640 1 -- 172.21.15.18:0/2076887751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550433ad0 msgr2=0x560550432820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.885+0000 7f2a3ebc0640 1 --2- 172.21.15.18:0/2076887751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550433ad0 0x560550432820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.885+0000 7f2a3ebc0640 1 -- 172.21.15.18:0/2076887751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550432d60 msgr2=0x5605504331e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.885+0000 7f2a3ebc0640 1 --2- 172.21.15.18:0/2076887751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550432d60 0x5605504331e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.885+0000 7f2a3ebc0640 1 -- 172.21.15.18:0/2076887751 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605503307b0 con 0x56055043c2e0 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.885+0000 7f2a3dbbe640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2a240a23f0 0x7f2a240a48b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a28006010 tx=0x7f2a2802c960 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.885+0000 7f2a3ebc0640 1 --2- 172.21.15.18:0/2076887751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x56055043e6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009610 tx=0x7f2a3009a090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1999c61516f89fd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.885+0000 7f2a2cbfe640 1 -- 172.21.15.18:0/2076887751 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a300aca50 con 0x56055043c2e0 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.885+0000 7f2a2cbfe640 1 -- 172.21.15.18:0/2076887751 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a30063b50 con 0x56055043c2e0 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.885+0000 7f2a2cbfe640 1 -- 172.21.15.18:0/2076887751 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a300ac820 con 0x56055043c2e0 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.886+0000 7f2a431f7740 1 -- 172.21.15.18:0/2076887751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 msgr2=0x56055043e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.886+0000 7f2a431f7740 1 --2- 172.21.15.18:0/2076887751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x56055043e6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009610 tx=0x7f2a3009a090 comp rx=0 tx=0).stop 2024-02-17T12:51:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.886+0000 7f2a431f7740 1 -- 172.21.15.18:0/2076887751 shutdown_connections 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.886+0000 7f2a431f7740 1 --2- 172.21.15.18:0/2076887751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x56055043e6d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.886+0000 7f2a431f7740 1 --2- 172.21.15.18:0/2076887751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550432d60 0x5605504331e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.886+0000 7f2a431f7740 1 --2- 172.21.15.18:0/2076887751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550433ad0 0x560550432820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.886+0000 7f2a431f7740 1 -- 172.21.15.18:0/2076887751 >> 172.21.15.18:0/2076887751 conn(0x5605503fa550 msgr2=0x56055042da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.886+0000 7f2a431f7740 1 -- 172.21.15.18:0/2076887751 shutdown_connections 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.886+0000 7f2a431f7740 1 -- 172.21.15.18:0/2076887751 wait complete. 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.886+0000 7f2a431f7740 1 -- start start 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.886+0000 7f2a431f7740 1 Processor -- start 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a431f7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x5605504392d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a431f7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 0x560550439810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a431f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x560550439d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a431f7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56055033fbf0 con 0x560550433ad0 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a431f7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56055006e250 con 0x560550432d60 2024-02-17T12:51:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a431f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560550320200 con 0x56055043c2e0 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a431f7740 10 client.0 _ll_register_callbacks cb 0x56055038ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a3dbbe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 0x560550439810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a3dbbe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 0x560550439810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33298/0 (socket says 172.21.15.18:33298) 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a3dbbe640 1 -- 172.21.15.18:0/4226324463 learned_addr learned my addr 172.21.15.18:0/4226324463 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a3e3bf640 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x5605504392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a3ebc0640 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x560550439d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a3dbbe640 1 -- 172.21.15.18:0/4226324463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 msgr2=0x560550439d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a3dbbe640 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x560550439d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a3dbbe640 1 -- 172.21.15.18:0/4226324463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 msgr2=0x5605504392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a3dbbe640 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x5605504392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a3dbbe640 1 -- 172.21.15.18:0/4226324463 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a381a4380 con 0x560550433ad0 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a3dbbe640 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 0x560550439810 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2a28004c50 tx=0x7f2a28005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659ed7b0284b195b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a21ffb640 1 -- 172.21.15.18:0/4226324463 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28048070 con 0x560550433ad0 2024-02-17T12:51:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a21ffb640 1 -- 172.21.15.18:0/4226324463 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2a28005000 con 0x560550433ad0 2024-02-17T12:51:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a431f7740 1 -- 172.21.15.18:0/4226324463 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56055009a600 con 0x560550433ad0 2024-02-17T12:51:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.887+0000 7f2a21ffb640 1 -- 172.21.15.18:0/4226324463 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a280051c0 con 0x560550433ad0 2024-02-17T12:51:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.888+0000 7f2a431f7740 10 client.15996 fetch_fsmap learned FSMap version 27 2024-02-17T12:51:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.888+0000 7f2a431f7740 1 -- 172.21.15.18:0/4226324463 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560550398d80 con 0x560550433ad0 2024-02-17T12:51:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.888+0000 7f2a21ffb640 1 -- 172.21.15.18:0/4226324463 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56055009a600 con 0x560550433ad0 2024-02-17T12:51:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.888+0000 7f2a21ffb640 1 -- 172.21.15.18:0/4226324463 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2a2804e440 con 0x560550433ad0 2024-02-17T12:51:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.888+0000 7f2a431f7740 10 client.15996 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:51:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.888+0000 7f2a431f7740 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.888+0000 7f2a431f7740 20 client.15996 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.888+0000 7f2a431f7740 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56055044c600 0x56055044ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.888+0000 7f2a431f7740 4 client.15996 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.888+0000 7f2a431f7740 1 -- 172.21.15.18:0/4226324463 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605500eb1b0 con 0x56055044c600 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.888+0000 7f2a3e3bf640 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56055044c600 0x56055044ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a3e3bf640 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56055044c600 0x56055044ea40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7597dd96344bd1df server_cookie=b90756fb3da4a22f in_seq=0 out_seq=0 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a21ffb640 10 client.15996 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a21ffb640 1 -- 172.21.15.18:0/4226324463 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5605500eb1b0 con 0x56055044c600 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a21ffb640 10 client.15996 handle_command_reply: tid=0 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a431f7740 1 client.15996 shutdown 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a431f7740 1 -- 172.21.15.18:0/4226324463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 msgr2=0x560550439810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a431f7740 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 0x560550439810 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2a28004c50 tx=0x7f2a28005e40 comp rx=0 tx=0).stop 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a431f7740 1 -- 172.21.15.18:0/4226324463 shutdown_connections 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a431f7740 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56055044c600 0x56055044ea40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a431f7740 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x560550439d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a431f7740 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 0x560550439810 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a21ffb640 0 client.15996 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a431f7740 1 --2- 172.21.15.18:0/4226324463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x5605504392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a431f7740 1 -- 172.21.15.18:0/4226324463 >> 172.21.15.18:0/4226324463 conn(0x5605503fa550 msgr2=0x56055042f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a431f7740 1 -- 172.21.15.18:0/4226324463 shutdown_connections 2024-02-17T12:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a431f7740 1 -- 172.21.15.18:0/4226324463 wait complete. 2024-02-17T12:51:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.889+0000 7f2a431f7740 20 client.15996 trim_cache size 0 max 16384 2024-02-17T12:51:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.902+0000 7f2a431f7740 1 Processor -- start 2024-02-17T12:51:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.902+0000 7f2a431f7740 1 -- start start 2024-02-17T12:51:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a431f7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x560550439b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a431f7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 0x560550436530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a431f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x560550436a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a431f7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56055009a600 con 0x560550433ad0 2024-02-17T12:51:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a431f7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605504490c0 con 0x560550432d60 2024-02-17T12:51:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a431f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56055043bda0 con 0x56055043c2e0 2024-02-17T12:51:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3dbbe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 0x560550436530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3dbbe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 0x560550436530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33306/0 (socket says 172.21.15.18:33306) 2024-02-17T12:51:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3dbbe640 1 -- 172.21.15.18:0/2787093823 learned_addr learned my addr 172.21.15.18:0/2787093823 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3e3bf640 1 --2- 172.21.15.18:0/2787093823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x560550439b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3ebc0640 1 --2- 172.21.15.18:0/2787093823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x560550436a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3dbbe640 1 -- 172.21.15.18:0/2787093823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 msgr2=0x560550436a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3dbbe640 1 --2- 172.21.15.18:0/2787093823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x560550436a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3dbbe640 1 -- 172.21.15.18:0/2787093823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 msgr2=0x560550439b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3dbbe640 1 --2- 172.21.15.18:0/2787093823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x560550439b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3dbbe640 1 -- 172.21.15.18:0/2787093823 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605503357b0 con 0x560550433ad0 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3e3bf640 1 --2- 172.21.15.18:0/2787093823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x560550439b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3ebc0640 1 --2- 172.21.15.18:0/2787093823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x560550436a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a3dbbe640 1 --2- 172.21.15.18:0/2787093823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 0x560550436530 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2a2804e8a0 tx=0x7f2a28004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5235b4c3bf07e795 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a23fff640 1 -- 172.21.15.18:0/2787093823 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28048040 con 0x560550433ad0 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a23fff640 1 -- 172.21.15.18:0/2787093823 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a28004410 con 0x560550433ad0 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.903+0000 7f2a23fff640 1 -- 172.21.15.18:0/2787093823 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28005500 con 0x560550433ad0 2024-02-17T12:51:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 -- 172.21.15.18:0/2787093823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 msgr2=0x560550436530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 --2- 172.21.15.18:0/2787093823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 0x560550436530 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2a2804e8a0 tx=0x7f2a28004ba0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 -- 172.21.15.18:0/2787093823 shutdown_connections 2024-02-17T12:51:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 --2- 172.21.15.18:0/2787093823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56055043c2e0 0x560550436a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 --2- 172.21.15.18:0/2787093823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560550433ad0 0x560550436530 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 --2- 172.21.15.18:0/2787093823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x560550439b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 -- 172.21.15.18:0/2787093823 >> 172.21.15.18:0/2787093823 conn(0x5605503fa550 msgr2=0x560550437ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 -- 172.21.15.18:0/2787093823 shutdown_connections 2024-02-17T12:51:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 -- 172.21.15.18:0/2787093823 wait complete. 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 -- start start 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 Processor -- start 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x560550443010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.904+0000 7f2a431f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560550433ad0 0x560550443550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a431f7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56055043c2e0 0x560550441540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a431f7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560550441c40 con 0x56055043c2e0 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a431f7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560550441db0 con 0x560550432d60 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a431f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560550441f70 con 0x560550433ad0 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a431f7740 10 client.0 _ll_register_callbacks cb 0x7f2a30009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a3ebc0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56055043c2e0 0x560550441540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a3ebc0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56055043c2e0 0x560550441540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33322/0 (socket says 172.21.15.18:33322) 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a3ebc0640 1 -- 172.21.15.18:0/3301170701 learned_addr learned my addr 172.21.15.18:0/3301170701 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a3e3bf640 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x560550443010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a3dbbe640 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560550433ad0 0x560550443550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a3ebc0640 1 -- 172.21.15.18:0/3301170701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560550433ad0 msgr2=0x560550443550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a3ebc0640 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560550433ad0 0x560550443550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a3ebc0640 1 -- 172.21.15.18:0/3301170701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 msgr2=0x560550443010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a3ebc0640 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x560550443010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a3ebc0640 1 -- 172.21.15.18:0/3301170701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560550398d80 con 0x56055043c2e0 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a3ebc0640 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56055043c2e0 0x560550441540 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000d650 tx=0x7f2a3000b350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef617b8214708b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a21ffb640 1 -- 172.21.15.18:0/3301170701 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3006cb30 con 0x56055043c2e0 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a21ffb640 1 -- 172.21.15.18:0/3301170701 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2a300046c0 con 0x56055043c2e0 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a431f7740 1 -- 172.21.15.18:0/3301170701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605500d7550 con 0x56055043c2e0 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a21ffb640 1 -- 172.21.15.18:0/3301170701 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a3006c700 con 0x56055043c2e0 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a431f7740 10 client.16002 fetch_fsmap learned FSMap version 27 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.905+0000 7f2a431f7740 1 -- 172.21.15.18:0/3301170701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56055035b540 con 0x56055043c2e0 2024-02-17T12:51:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.906+0000 7f2a21ffb640 1 -- 172.21.15.18:0/3301170701 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a300b7030 con 0x56055043c2e0 2024-02-17T12:51:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.906+0000 7f2a21ffb640 1 -- 172.21.15.18:0/3301170701 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2a30096b00 con 0x56055043c2e0 2024-02-17T12:51:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.906+0000 7f2a431f7740 10 client.16002 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:51:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.906+0000 7f2a431f7740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.906+0000 7f2a431f7740 20 client.16002 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.906+0000 7f2a431f7740 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5605504a3a80 0x5605504a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.906+0000 7f2a431f7740 4 client.16002 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:51:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.906+0000 7f2a431f7740 1 -- 172.21.15.18:0/3301170701 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560550046520 con 0x5605504a3a80 2024-02-17T12:51:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.906+0000 7f2a3e3bf640 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5605504a3a80 0x5605504a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.906+0000 7f2a3e3bf640 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5605504a3a80 0x5605504a5ea0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41dd4ee9c95b9627 server_cookie=921fec299315e1ed in_seq=0 out_seq=0 2024-02-17T12:51:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:00.906+0000 7f2a21ffb640 10 client.16002 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:00.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:00 smithi146 ceph-mon[36454]: pgmap v36: 97 pgs: 97 active+clean; 17 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.7 MiB/s wr, 274 op/s 2024-02-17T12:51:01.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:00 smithi018 ceph-mon[32625]: pgmap v36: 97 pgs: 97 active+clean; 17 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.7 MiB/s wr, 274 op/s 2024-02-17T12:51:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.069+0000 7f2a21ffb640 1 -- 172.21.15.18:0/3301170701 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560550046520 con 0x5605504a3a80 2024-02-17T12:51:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.069+0000 7f2a21ffb640 10 client.16002 handle_command_reply: tid=0 2024-02-17T12:51:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.069+0000 7f2a431f7740 1 client.16002 shutdown 2024-02-17T12:51:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.070+0000 7f2a431f7740 1 -- 172.21.15.18:0/3301170701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56055043c2e0 msgr2=0x560550441540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.070+0000 7f2a431f7740 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56055043c2e0 0x560550441540 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000d650 tx=0x7f2a3000b350 comp rx=0 tx=0).stop 2024-02-17T12:51:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.070+0000 7f2a431f7740 1 -- 172.21.15.18:0/3301170701 shutdown_connections 2024-02-17T12:51:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.070+0000 7f2a431f7740 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5605504a3a80 0x5605504a5ea0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.070+0000 7f2a21ffb640 0 client.16002 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.070+0000 7f2a431f7740 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56055043c2e0 0x560550441540 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.070+0000 7f2a431f7740 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560550433ad0 0x560550443550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.070+0000 7f2a431f7740 1 --2- 172.21.15.18:0/3301170701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560550432d60 0x560550443010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.070+0000 7f2a431f7740 1 -- 172.21.15.18:0/3301170701 >> 172.21.15.18:0/3301170701 conn(0x5605503fa550 msgr2=0x56055042c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.070+0000 7f2a431f7740 1 -- 172.21.15.18:0/3301170701 shutdown_connections 2024-02-17T12:51:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.070+0000 7f2a431f7740 1 -- 172.21.15.18:0/3301170701 wait complete. 2024-02-17T12:51:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.070+0000 7f2a431f7740 20 client.16002 trim_cache size 0 max 16384 2024-02-17T12:51:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:51:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:51:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:51:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:51:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.071+0000 7f2a40623640 1 -- 172.21.15.18:0/1467323033 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2a240a23f0 msgr2=0x7f2a240a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.071+0000 7f2a40623640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2a240a23f0 0x7f2a240a48b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a28006010 tx=0x7f2a2802c960 comp rx=0 tx=0).stop 2024-02-17T12:51:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.071+0000 7f2a40623640 1 -- 172.21.15.18:0/1467323033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a381072b0 msgr2=0x7f2a3819bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.071+0000 7f2a40623640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a381072b0 0x7f2a3819bef0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000d680 tx=0x7f2a3000db40 comp rx=0 tx=0).stop 2024-02-17T12:51:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.071+0000 7f2a40623640 1 -- 172.21.15.18:0/1467323033 shutdown_connections 2024-02-17T12:51:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.071+0000 7f2a40623640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2a240a23f0 0x7f2a240a48b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.071+0000 7f2a40623640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a381072b0 0x7f2a3819bef0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.071+0000 7f2a40623640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a3806a360 0x7f2a38197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.071+0000 7f2a40623640 1 --2- 172.21.15.18:0/1467323033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a380699a0 0x7f2a38197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.071+0000 7f2a40623640 1 -- 172.21.15.18:0/1467323033 >> 172.21.15.18:0/1467323033 conn(0x7f2a38076ee0 msgr2=0x7f2a380fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.071+0000 7f2a40623640 1 -- 172.21.15.18:0/1467323033 shutdown_connections 2024-02-17T12:51:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:01.071+0000 7f2a40623640 1 -- 172.21.15.18:0/1467323033 wait complete. 2024-02-17T12:51:01.258 INFO:tasks.workunit.client.0.smithi018.stdout: 1 46767 16.85 MB/sec warmup 16 sec latency 397.697 ms 2024-02-17T12:51:01.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:01 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:51:01.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:01 smithi146 ceph-mon[36454]: pgmap v37: 97 pgs: 97 active+clean; 21 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 230 op/s 2024-02-17T12:51:02.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:01 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:51:02.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:01 smithi018 ceph-mon[32625]: pgmap v37: 97 pgs: 97 active+clean; 21 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 230 op/s 2024-02-17T12:51:02.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:01 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:51:02.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:01 smithi119 ceph-mon[36188]: pgmap v37: 97 pgs: 97 active+clean; 21 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 230 op/s 2024-02-17T12:51:02.259 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49828 16.90 MB/sec warmup 17 sec latency 103.194 ms 2024-02-17T12:51:03.259 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49828 15.96 MB/sec warmup 18 sec latency 1103.236 ms 2024-02-17T12:51:04.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:03 smithi018 ceph-mon[32625]: pgmap v38: 97 pgs: 97 active+clean; 21 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-02-17T12:51:04.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:03 smithi119 ceph-mon[36188]: pgmap v38: 97 pgs: 97 active+clean; 21 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-02-17T12:51:04.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:03 smithi146 ceph-mon[36454]: pgmap v38: 97 pgs: 97 active+clean; 21 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-02-17T12:51:04.259 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49928 15.18 MB/sec warmup 19 sec latency 1743.086 ms 2024-02-17T12:51:05.259 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49928 14.42 MB/sec warmup 20 sec latency 1272.397 ms 2024-02-17T12:51:06.003 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:05 smithi146 ceph-mon[36454]: pgmap v39: 97 pgs: 97 active+clean; 14 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 420 op/s 2024-02-17T12:51:06.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:05 smithi018 ceph-mon[32625]: pgmap v39: 97 pgs: 97 active+clean; 14 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 420 op/s 2024-02-17T12:51:06.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:05 smithi119 ceph-mon[36188]: pgmap v39: 97 pgs: 97 active+clean; 14 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 420 op/s 2024-02-17T12:51:06.259 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49928 13.73 MB/sec warmup 21 sec latency 2272.489 ms 2024-02-17T12:51:07.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:06 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:51:07.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:06 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:51:07.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:06 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:51:07.259 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49928 13.11 MB/sec warmup 22 sec latency 3272.611 ms 2024-02-17T12:51:08.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:07 smithi018 ceph-mon[32625]: pgmap v40: 97 pgs: 97 active+clean; 14 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 1.3 MiB/s wr, 314 op/s 2024-02-17T12:51:08.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:07 smithi119 ceph-mon[36188]: pgmap v40: 97 pgs: 97 active+clean; 14 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 1.3 MiB/s wr, 314 op/s 2024-02-17T12:51:08.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:07 smithi146 ceph-mon[36454]: pgmap v40: 97 pgs: 97 active+clean; 14 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 1.3 MiB/s wr, 314 op/s 2024-02-17T12:51:08.259 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49928 12.54 MB/sec warmup 23 sec latency 4272.706 ms 2024-02-17T12:51:09.259 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49932 12.02 MB/sec warmup 24 sec latency 4901.514 ms 2024-02-17T12:51:10.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:09 smithi018 ceph-mon[32625]: pgmap v41: 97 pgs: 97 active+clean; 14 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.4 MiB/s wr, 323 op/s 2024-02-17T12:51:10.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:09 smithi119 ceph-mon[36188]: pgmap v41: 97 pgs: 97 active+clean; 14 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.4 MiB/s wr, 323 op/s 2024-02-17T12:51:10.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:09 smithi146 ceph-mon[36454]: pgmap v41: 97 pgs: 97 active+clean; 14 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.4 MiB/s wr, 323 op/s 2024-02-17T12:51:10.259 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49932 11.53 MB/sec warmup 25 sec latency 1368.912 ms 2024-02-17T12:51:11.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49932 11.09 MB/sec warmup 26 sec latency 2369.021 ms 2024-02-17T12:51:12.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:11 smithi018 ceph-mon[32625]: pgmap v42: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 994 KiB/s wr, 297 op/s 2024-02-17T12:51:12.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:11 smithi119 ceph-mon[36188]: pgmap v42: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 994 KiB/s wr, 297 op/s 2024-02-17T12:51:12.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:11 smithi146 ceph-mon[36454]: pgmap v42: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 994 KiB/s wr, 297 op/s 2024-02-17T12:51:12.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49932 10.68 MB/sec warmup 27 sec latency 3369.149 ms 2024-02-17T12:51:13.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49932 10.30 MB/sec warmup 28 sec latency 4369.239 ms 2024-02-17T12:51:14.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:13 smithi018 ceph-mon[32625]: pgmap v43: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 628 KiB/s wr, 230 op/s 2024-02-17T12:51:14.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:13 smithi119 ceph-mon[36188]: pgmap v43: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 628 KiB/s wr, 230 op/s 2024-02-17T12:51:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.173+0000 7fa23d920640 1 -- 172.21.15.18:0/1555150038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2381035d0 msgr2=0x7fa2381039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.173+0000 7fa23d920640 1 --2- 172.21.15.18:0/1555150038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2381035d0 0x7fa2381039b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa228009930 tx=0x7fa22802bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.173+0000 7fa23d920640 1 -- 172.21.15.18:0/1555150038 shutdown_connections 2024-02-17T12:51:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.173+0000 7fa23d920640 1 --2- 172.21.15.18:0/1555150038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2381086a0 0x7fa238110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.173+0000 7fa23d920640 1 --2- 172.21.15.18:0/1555150038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa238103f80 0x7fa238107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.173+0000 7fa23d920640 1 --2- 172.21.15.18:0/1555150038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2381035d0 0x7fa2381039b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.173+0000 7fa23d920640 1 -- 172.21.15.18:0/1555150038 >> 172.21.15.18:0/1555150038 conn(0x7fa2380fee30 msgr2=0x7fa238101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.173+0000 7fa23d920640 1 -- 172.21.15.18:0/1555150038 shutdown_connections 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.173+0000 7fa23d920640 1 -- 172.21.15.18:0/1555150038 wait complete. 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.174+0000 7fa23d920640 1 Processor -- start 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.174+0000 7fa23d920640 1 -- start start 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.174+0000 7fa23d920640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2381035d0 0x7fa23819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.174+0000 7fa23d920640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2381086a0 0x7fa23819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.174+0000 7fa23d920640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa238199800 0x7fa238199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.174+0000 7fa23d920640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23819a2b0 con 0x7fa238199800 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.174+0000 7fa23d920640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23819a420 con 0x7fa2381035d0 2024-02-17T12:51:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.174+0000 7fa23d920640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23819a590 con 0x7fa2381086a0 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.174+0000 7fa2377fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa238199800 0x7fa238199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.174+0000 7fa2377fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa238199800 0x7fa238199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34860/0 (socket says 172.21.15.18:34860) 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.174+0000 7fa2377fe640 1 -- 172.21.15.18:0/2079955916 learned_addr learned my addr 172.21.15.18:0/2079955916 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.174+0000 7fa2367fc640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2381086a0 0x7fa23819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.175+0000 7fa236ffd640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2381035d0 0x7fa23819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.175+0000 7fa2377fe640 1 -- 172.21.15.18:0/2079955916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2381086a0 msgr2=0x7fa23819fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.175+0000 7fa2377fe640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2381086a0 0x7fa23819fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.175+0000 7fa2377fe640 1 -- 172.21.15.18:0/2079955916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2381035d0 msgr2=0x7fa23819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.175+0000 7fa2377fe640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2381035d0 0x7fa23819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.175+0000 7fa2377fe640 1 -- 172.21.15.18:0/2079955916 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa230009840 con 0x7fa238199800 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.175+0000 7fa2377fe640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa238199800 0x7fa238199cb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa23000efd0 tx=0x7fa23000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.175+0000 7fa23c91e640 1 -- 172.21.15.18:0/2079955916 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230009b40 con 0x7fa238199800 2024-02-17T12:51:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.175+0000 7fa23d920640 1 -- 172.21.15.18:0/2079955916 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa228009560 con 0x7fa238199800 2024-02-17T12:51:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.175+0000 7fa23c91e640 1 -- 172.21.15.18:0/2079955916 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa230003b70 con 0x7fa238199800 2024-02-17T12:51:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.175+0000 7fa23c91e640 1 -- 172.21.15.18:0/2079955916 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2300108f0 con 0x7fa238199800 2024-02-17T12:51:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.175+0000 7fa23d920640 1 -- 172.21.15.18:0/2079955916 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2381a67f0 con 0x7fa238199800 2024-02-17T12:51:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa23c91e640 1 -- 172.21.15.18:0/2079955916 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fa230003ce0 con 0x7fa238199800 2024-02-17T12:51:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa2405cc740 1 Processor -- start 2024-02-17T12:51:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa2405cc740 1 -- start start 2024-02-17T12:51:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa23c91e640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa21c0a23f0 0x7fa21c0a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa2405cc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255cad0 0x557b7255b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa2405cc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255bd60 0x557b7255c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa2405cc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b725652e0 0x557b725676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa2405cc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b724ca9f0 con 0x557b7255cad0 2024-02-17T12:51:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa2405cc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b724c7f20 con 0x557b725652e0 2024-02-17T12:51:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa2405cc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b724c3700 con 0x557b7255bd60 2024-02-17T12:51:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa2367fc640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa21c0a23f0 0x7fa21c0a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa2377fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b725652e0 0x557b725676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa236ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255cad0 0x557b7255b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa236ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255cad0 0x557b7255b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34866/0 (socket says 172.21.15.18:34866) 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.178+0000 7fa236ffd640 1 -- 172.21.15.18:0/4014384749 learned_addr learned my addr 172.21.15.18:0/4014384749 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.179+0000 7fa23c91e640 1 -- 172.21.15.18:0/2079955916 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa230014070 con 0x7fa238199800 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.179+0000 7fa2367fc640 1 --2- 172.21.15.18:0/4014384749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255bd60 0x557b7255c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.179+0000 7fa2377fe640 1 -- 172.21.15.18:0/4014384749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255bd60 msgr2=0x557b7255c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.179+0000 7fa2377fe640 1 --2- 172.21.15.18:0/4014384749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255bd60 0x557b7255c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.179+0000 7fa2377fe640 1 -- 172.21.15.18:0/4014384749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255cad0 msgr2=0x557b7255b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.179+0000 7fa2377fe640 1 --2- 172.21.15.18:0/4014384749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255cad0 0x557b7255b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.179+0000 7fa2377fe640 1 -- 172.21.15.18:0/4014384749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b724597b0 con 0x557b725652e0 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.179+0000 7fa2367fc640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa21c0a23f0 0x7fa21c0a48b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa224009710 tx=0x7fa2240098b0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.179+0000 7fa2377fe640 1 --2- 172.21.15.18:0/4014384749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b725652e0 0x557b725676d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa2300cbe50 tx=0x7fa230008c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3259e0770bd87e58 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.179+0000 7fa22dffb640 1 -- 172.21.15.18:0/4014384749 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23009a980 con 0x557b725652e0 2024-02-17T12:51:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.179+0000 7fa22dffb640 1 -- 172.21.15.18:0/4014384749 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa23009b720 con 0x557b725652e0 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.179+0000 7fa22dffb640 1 -- 172.21.15.18:0/4014384749 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2300993b0 con 0x557b725652e0 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.180+0000 7fa2405cc740 1 -- 172.21.15.18:0/4014384749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b725652e0 msgr2=0x557b725676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.180+0000 7fa2405cc740 1 --2- 172.21.15.18:0/4014384749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b725652e0 0x557b725676d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa2300cbe50 tx=0x7fa230008c50 comp rx=0 tx=0).stop 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.180+0000 7fa2405cc740 1 -- 172.21.15.18:0/4014384749 shutdown_connections 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.180+0000 7fa2405cc740 1 --2- 172.21.15.18:0/4014384749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b725652e0 0x557b725676d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.180+0000 7fa2405cc740 1 --2- 172.21.15.18:0/4014384749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255bd60 0x557b7255c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.180+0000 7fa2405cc740 1 --2- 172.21.15.18:0/4014384749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255cad0 0x557b7255b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.180+0000 7fa2405cc740 1 -- 172.21.15.18:0/4014384749 >> 172.21.15.18:0/4014384749 conn(0x557b72523550 msgr2=0x557b72556a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.180+0000 7fa2405cc740 1 -- 172.21.15.18:0/4014384749 shutdown_connections 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.180+0000 7fa2405cc740 1 -- 172.21.15.18:0/4014384749 wait complete. 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.180+0000 7fa2405cc740 1 -- start start 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2405cc740 1 Processor -- start 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2405cc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b7255bd60 0x557b725622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2405cc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255cad0 0x557b72562800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2405cc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b725652e0 0x557b72562d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2405cc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b72468bf0 con 0x557b725652e0 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2405cc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b72197250 con 0x557b7255bd60 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2405cc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b72449200 con 0x557b7255cad0 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2405cc740 10 client.0 _ll_register_callbacks cb 0x557b724b7d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2377fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b725652e0 0x557b72562d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa236ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b7255bd60 0x557b725622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2377fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b725652e0 0x557b72562d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34874/0 (socket says 172.21.15.18:34874) 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2377fe640 1 -- 172.21.15.18:0/797390850 learned_addr learned my addr 172.21.15.18:0/797390850 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2367fc640 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255cad0 0x557b72562800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2377fe640 1 -- 172.21.15.18:0/797390850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255cad0 msgr2=0x557b72562800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2377fe640 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255cad0 0x557b72562800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2377fe640 1 -- 172.21.15.18:0/797390850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b7255bd60 msgr2=0x557b725622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2377fe640 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b7255bd60 0x557b725622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2377fe640 1 -- 172.21.15.18:0/797390850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2381a67f0 con 0x557b725652e0 2024-02-17T12:51:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2367fc640 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255cad0 0x557b72562800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa236ffd640 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b7255bd60 0x557b725622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa2377fe640 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b725652e0 0x557b72562d40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa2300cc5d0 tx=0x7fa23009b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33e91fe14d1272 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.181+0000 7fa21affd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa21affd640 1 -- 172.21.15.18:0/797390850 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23009f170 con 0x557b725652e0 2024-02-17T12:51:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa21affd640 1 -- 172.21.15.18:0/797390850 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa2300a37c0 con 0x557b725652e0 2024-02-17T12:51:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa2405cc740 1 -- 172.21.15.18:0/797390850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b721c3600 con 0x557b725652e0 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa21affd640 1 -- 172.21.15.18:0/797390850 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa23009b020 con 0x557b725652e0 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa2405cc740 10 client.16014 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa2405cc740 1 -- 172.21.15.18:0/797390850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557b724c1d80 con 0x557b725652e0 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa21affd640 1 -- 172.21.15.18:0/797390850 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557b721c3600 con 0x557b725652e0 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa21affd640 1 -- 172.21.15.18:0/797390850 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa23009a470 con 0x557b725652e0 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa2405cc740 10 client.16014 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa2405cc740 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa2405cc740 20 client.16014 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa2405cc740 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557b725753d0 0x557b725777f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa2405cc740 4 client.16014 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa2405cc740 1 -- 172.21.15.18:0/797390850 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b722141b0 con 0x557b725753d0 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.182+0000 7fa236ffd640 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557b725753d0 0x557b725777f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.183+0000 7fa236ffd640 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557b725753d0 0x557b725777f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ddfecbe990c560 server_cookie=b2520e388901c9ef in_seq=0 out_seq=0 2024-02-17T12:51:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.183+0000 7fa21affd640 10 client.16014 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.183+0000 7fa21affd640 1 -- 172.21.15.18:0/797390850 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b722141b0 con 0x557b725753d0 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.183+0000 7fa21affd640 10 client.16014 handle_command_reply: tid=0 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.183+0000 7fa2405cc740 1 client.16014 shutdown 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.183+0000 7fa2405cc740 1 -- 172.21.15.18:0/797390850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b725652e0 msgr2=0x557b72562d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.183+0000 7fa2405cc740 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b725652e0 0x557b72562d40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa2300cc5d0 tx=0x7fa23009b890 comp rx=0 tx=0).stop 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.184+0000 7fa2405cc740 1 -- 172.21.15.18:0/797390850 shutdown_connections 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.184+0000 7fa2405cc740 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557b725753d0 0x557b725777f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.184+0000 7fa2405cc740 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b725652e0 0x557b72562d40 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.184+0000 7fa2405cc740 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255cad0 0x557b72562800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.184+0000 7fa21affd640 0 client.16014 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.184+0000 7fa2405cc740 1 --2- 172.21.15.18:0/797390850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b7255bd60 0x557b725622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.184+0000 7fa2405cc740 1 -- 172.21.15.18:0/797390850 >> 172.21.15.18:0/797390850 conn(0x557b72523550 msgr2=0x557b725587a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.184+0000 7fa2405cc740 1 -- 172.21.15.18:0/797390850 shutdown_connections 2024-02-17T12:51:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.184+0000 7fa2405cc740 1 -- 172.21.15.18:0/797390850 wait complete. 2024-02-17T12:51:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.184+0000 7fa2405cc740 20 client.16014 trim_cache size 0 max 16384 2024-02-17T12:51:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.197+0000 7fa2405cc740 1 Processor -- start 2024-02-17T12:51:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.197+0000 7fa2405cc740 1 -- start start 2024-02-17T12:51:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.197+0000 7fa2405cc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 0x557b725649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.197+0000 7fa2405cc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b7255cad0 0x557b72562b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.197+0000 7fa2405cc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b725652e0 0x557b72560b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.197+0000 7fa2405cc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b721c3600 con 0x557b7255bd60 2024-02-17T12:51:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.197+0000 7fa2405cc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b7255d0c0 con 0x557b7255cad0 2024-02-17T12:51:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.197+0000 7fa2405cc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b7255b030 con 0x557b725652e0 2024-02-17T12:51:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.197+0000 7fa236ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 0x557b725649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.197+0000 7fa236ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 0x557b725649f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34888/0 (socket says 172.21.15.18:34888) 2024-02-17T12:51:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.197+0000 7fa236ffd640 1 -- 172.21.15.18:0/3952194882 learned_addr learned my addr 172.21.15.18:0/3952194882 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.197+0000 7fa2367fc640 1 --2- 172.21.15.18:0/3952194882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b7255cad0 0x557b72562b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.198+0000 7fa236ffd640 1 -- 172.21.15.18:0/3952194882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b725652e0 msgr2=0x557b72560b90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:51:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.198+0000 7fa236ffd640 1 --2- 172.21.15.18:0/3952194882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b725652e0 0x557b72560b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.198+0000 7fa236ffd640 1 -- 172.21.15.18:0/3952194882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b7255cad0 msgr2=0x557b72562b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.198+0000 7fa236ffd640 1 --2- 172.21.15.18:0/3952194882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b7255cad0 0x557b72562b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.198+0000 7fa236ffd640 1 -- 172.21.15.18:0/3952194882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b7245e7b0 con 0x557b7255bd60 2024-02-17T12:51:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.198+0000 7fa236ffd640 1 --2- 172.21.15.18:0/3952194882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 0x557b725649f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa228036040 tx=0x7fa228005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5645cd44a839040a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.198+0000 7fa22d7fa640 1 -- 172.21.15.18:0/3952194882 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228049ca0 con 0x557b7255bd60 2024-02-17T12:51:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.198+0000 7fa22d7fa640 1 -- 172.21.15.18:0/3952194882 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa228049e00 con 0x557b7255bd60 2024-02-17T12:51:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.198+0000 7fa22d7fa640 1 -- 172.21.15.18:0/3952194882 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228049600 con 0x557b7255bd60 2024-02-17T12:51:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 -- 172.21.15.18:0/3952194882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 msgr2=0x557b725649f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 --2- 172.21.15.18:0/3952194882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 0x557b725649f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa228036040 tx=0x7fa228005fd0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 -- 172.21.15.18:0/3952194882 shutdown_connections 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 --2- 172.21.15.18:0/3952194882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b725652e0 0x557b72560b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 --2- 172.21.15.18:0/3952194882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b7255cad0 0x557b72562b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 --2- 172.21.15.18:0/3952194882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 0x557b725649f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 -- 172.21.15.18:0/3952194882 >> 172.21.15.18:0/3952194882 conn(0x557b72523550 msgr2=0x557b7255d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 -- 172.21.15.18:0/3952194882 shutdown_connections 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 -- 172.21.15.18:0/3952194882 wait complete. 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 -- start start 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 Processor -- start 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 0x557b7256c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255cad0 0x557b7256a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b725652e0 0x557b7256ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b7256b0d0 con 0x557b7255bd60 2024-02-17T12:51:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.199+0000 7fa2405cc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b7256b240 con 0x557b725652e0 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa2405cc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b7256b400 con 0x557b7255cad0 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa2405cc740 10 client.0 _ll_register_callbacks cb 0x7fa230004800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa236ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 0x557b7256c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa236ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 0x557b7256c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34894/0 (socket says 172.21.15.18:34894) 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa236ffd640 1 -- 172.21.15.18:0/362387951 learned_addr learned my addr 172.21.15.18:0/362387951 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa2377fe640 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b725652e0 0x557b7256ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa236ffd640 1 -- 172.21.15.18:0/362387951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255cad0 msgr2=0x557b7256a620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa236ffd640 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255cad0 0x557b7256a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa236ffd640 1 -- 172.21.15.18:0/362387951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b725652e0 msgr2=0x557b7256ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa236ffd640 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b725652e0 0x557b7256ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa236ffd640 1 -- 172.21.15.18:0/362387951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa238110a80 con 0x557b7255bd60 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa2377fe640 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b725652e0 0x557b7256ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa236ffd640 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 0x557b7256c280 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa22802bc70 tx=0x7fa22802c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de4b9196ed650a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa21affd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa21affd640 1 -- 172.21.15.18:0/362387951 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228044070 con 0x557b7255bd60 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa21affd640 1 -- 172.21.15.18:0/362387951 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa228057030 con 0x557b7255bd60 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa2405cc740 1 -- 172.21.15.18:0/362387951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa23009b720 con 0x557b7255bd60 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.200+0000 7fa21affd640 1 -- 172.21.15.18:0/362387951 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2280493a0 con 0x557b7255bd60 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa2405cc740 10 client.16020 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa2405cc740 1 -- 172.21.15.18:0/362387951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557b72484540 con 0x557b7255bd60 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa21affd640 1 -- 172.21.15.18:0/362387951 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa23009b720 con 0x557b7255bd60 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa21affd640 1 -- 172.21.15.18:0/362387951 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa22803c040 con 0x557b7255bd60 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa2405cc740 10 client.16020 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa2405cc740 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa2405cc740 20 client.16020 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa2405cc740 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557b725ccbc0 0x557b725cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa2405cc740 4 client.16020 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:51:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa2405cc740 1 -- 172.21.15.18:0/362387951 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557b72200550 con 0x557b725ccbc0 2024-02-17T12:51:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa2367fc640 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557b725ccbc0 0x557b725cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa2367fc640 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557b725ccbc0 0x557b725cefe0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdcf16bed6832019 server_cookie=e06dd4d293aaf4b1 in_seq=0 out_seq=0 2024-02-17T12:51:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:14.201+0000 7fa21affd640 10 client.16020 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:14.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:13 smithi146 ceph-mon[36454]: pgmap v43: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 628 KiB/s wr, 230 op/s 2024-02-17T12:51:14.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50214 9.95 MB/sec warmup 29 sec latency 4995.306 ms 2024-02-17T12:51:15.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50214 9.62 MB/sec warmup 30 sec latency 1037.551 ms 2024-02-17T12:51:16.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:15 smithi018 ceph-mon[32625]: pgmap v44: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 635 KiB/s wr, 236 op/s 2024-02-17T12:51:16.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:15 smithi119 ceph-mon[36188]: pgmap v44: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 635 KiB/s wr, 236 op/s 2024-02-17T12:51:16.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:15 smithi146 ceph-mon[36454]: pgmap v44: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 635 KiB/s wr, 236 op/s 2024-02-17T12:51:16.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50214 9.31 MB/sec warmup 31 sec latency 2037.671 ms 2024-02-17T12:51:17.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50214 9.02 MB/sec warmup 32 sec latency 3037.767 ms 2024-02-17T12:51:18.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:17 smithi018 ceph-mon[32625]: pgmap v45: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-02-17T12:51:18.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:17 smithi119 ceph-mon[36188]: pgmap v45: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-02-17T12:51:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:17 smithi146 ceph-mon[36454]: pgmap v45: 97 pgs: 97 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-02-17T12:51:18.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50214 8.75 MB/sec warmup 33 sec latency 4037.867 ms 2024-02-17T12:51:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa21affd640 1 -- 172.21.15.18:0/362387951 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557b72200550 con 0x557b725ccbc0 2024-02-17T12:51:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa21affd640 10 client.16020 handle_command_reply: tid=0 2024-02-17T12:51:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa2405cc740 1 client.16020 shutdown 2024-02-17T12:51:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa2405cc740 1 -- 172.21.15.18:0/362387951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 msgr2=0x557b7256c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa2405cc740 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 0x557b7256c280 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa22802bc70 tx=0x7fa22802c3e0 comp rx=0 tx=0).stop 2024-02-17T12:51:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa2405cc740 1 -- 172.21.15.18:0/362387951 shutdown_connections 2024-02-17T12:51:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa2405cc740 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557b725ccbc0 0x557b725cefe0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa2405cc740 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557b725652e0 0x557b7256ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa21affd640 0 client.16020 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa2405cc740 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b7255cad0 0x557b7256a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa2405cc740 1 --2- 172.21.15.18:0/362387951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b7255bd60 0x557b7256c280 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa2405cc740 1 -- 172.21.15.18:0/362387951 >> 172.21.15.18:0/362387951 conn(0x557b72523550 msgr2=0x557b72560ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa2405cc740 1 -- 172.21.15.18:0/362387951 shutdown_connections 2024-02-17T12:51:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.940+0000 7fa2405cc740 1 -- 172.21.15.18:0/362387951 wait complete. 2024-02-17T12:51:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.941+0000 7fa2405cc740 20 client.16020 trim_cache size 0 max 16384 2024-02-17T12:51:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:51:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:51:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:51:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:51:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.941+0000 7fa23d920640 1 -- 172.21.15.18:0/2079955916 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa21c0a23f0 msgr2=0x7fa21c0a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.941+0000 7fa23d920640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa21c0a23f0 0x7fa21c0a48b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa224009710 tx=0x7fa2240098b0 comp rx=0 tx=0).stop 2024-02-17T12:51:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.941+0000 7fa23d920640 1 -- 172.21.15.18:0/2079955916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa238199800 msgr2=0x7fa238199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.941+0000 7fa23d920640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa238199800 0x7fa238199cb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa23000efd0 tx=0x7fa23000c6a0 comp rx=0 tx=0).stop 2024-02-17T12:51:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.941+0000 7fa23d920640 1 -- 172.21.15.18:0/2079955916 shutdown_connections 2024-02-17T12:51:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.941+0000 7fa23d920640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa21c0a23f0 0x7fa21c0a48b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.941+0000 7fa23d920640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa238199800 0x7fa238199cb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.941+0000 7fa23d920640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2381086a0 0x7fa23819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.941+0000 7fa23d920640 1 --2- 172.21.15.18:0/2079955916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2381035d0 0x7fa23819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.941+0000 7fa23d920640 1 -- 172.21.15.18:0/2079955916 >> 172.21.15.18:0/2079955916 conn(0x7fa2380fee30 msgr2=0x7fa238101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.942+0000 7fa23d920640 1 -- 172.21.15.18:0/2079955916 shutdown_connections 2024-02-17T12:51:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:18.942+0000 7fa23d920640 1 -- 172.21.15.18:0/2079955916 wait complete. 2024-02-17T12:51:19.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50355 8.49 MB/sec warmup 34 sec latency 4712.115 ms 2024-02-17T12:51:20.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:19 smithi018 ceph-mon[32625]: pgmap v46: 97 pgs: 97 active+clean; 37 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.0 MiB/s wr, 52 op/s 2024-02-17T12:51:20.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:19 smithi119 ceph-mon[36188]: pgmap v46: 97 pgs: 97 active+clean; 37 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.0 MiB/s wr, 52 op/s 2024-02-17T12:51:20.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:19 smithi146 ceph-mon[36454]: pgmap v46: 97 pgs: 97 active+clean; 37 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.0 MiB/s wr, 52 op/s 2024-02-17T12:51:20.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50355 8.25 MB/sec warmup 35 sec latency 1218.638 ms 2024-02-17T12:51:20.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:51:20.370 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:51:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.497+0000 7fda336eb640 1 -- 172.21.15.18:0/3353522229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda2c101360 msgr2=0x7fda2c101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.497+0000 7fda336eb640 1 --2- 172.21.15.18:0/3353522229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda2c101360 0x7fda2c101740 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fda24009cd0 tx=0x7fda2402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.497+0000 7fda336eb640 1 -- 172.21.15.18:0/3353522229 shutdown_connections 2024-02-17T12:51:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.497+0000 7fda336eb640 1 --2- 172.21.15.18:0/3353522229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c106430 0x7fda2c10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.497+0000 7fda336eb640 1 --2- 172.21.15.18:0/3353522229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda2c101d10 0x7fda2c105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.497+0000 7fda336eb640 1 --2- 172.21.15.18:0/3353522229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda2c101360 0x7fda2c101740 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.497+0000 7fda336eb640 1 -- 172.21.15.18:0/3353522229 >> 172.21.15.18:0/3353522229 conn(0x7fda2c0fcc20 msgr2=0x7fda2c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda336eb640 1 -- 172.21.15.18:0/3353522229 shutdown_connections 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda336eb640 1 -- 172.21.15.18:0/3353522229 wait complete. 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda336eb640 1 Processor -- start 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda336eb640 1 -- start start 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda336eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda2c101360 0x7fda2c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda336eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c106430 0x7fda2c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda336eb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda2c073f00 0x7fda2c1a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda336eb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c1a5f50 con 0x7fda2c101360 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda336eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c1a60c0 con 0x7fda2c106430 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda336eb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c1a6280 con 0x7fda2c073f00 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda31487640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda2c101360 0x7fda2c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda31487640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda2c101360 0x7fda2c073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43420/0 (socket says 172.21.15.18:43420) 2024-02-17T12:51:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.498+0000 7fda31487640 1 -- 172.21.15.18:0/2472662215 learned_addr learned my addr 172.21.15.18:0/2472662215 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda31c88640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda2c073f00 0x7fda2c1a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda30c86640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c106430 0x7fda2c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda31487640 1 -- 172.21.15.18:0/2472662215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda2c073f00 msgr2=0x7fda2c1a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda31487640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda2c073f00 0x7fda2c1a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda31487640 1 -- 172.21.15.18:0/2472662215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c106430 msgr2=0x7fda2c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda31487640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c106430 0x7fda2c0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda31487640 1 -- 172.21.15.18:0/2472662215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda24009900 con 0x7fda2c101360 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda30c86640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c106430 0x7fda2c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda31487640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda2c101360 0x7fda2c073480 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fda24004af0 tx=0x7fda24034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda2a7fc640 1 -- 172.21.15.18:0/2472662215 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24032ac0 con 0x7fda2c101360 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda2a7fc640 1 -- 172.21.15.18:0/2472662215 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda24003e50 con 0x7fda2c101360 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda336eb640 1 -- 172.21.15.18:0/2472662215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c1a6500 con 0x7fda2c101360 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda2a7fc640 1 -- 172.21.15.18:0/2472662215 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24040450 con 0x7fda2c101360 2024-02-17T12:51:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.499+0000 7fda336eb640 1 -- 172.21.15.18:0/2472662215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c1a6a00 con 0x7fda2c101360 2024-02-17T12:51:20.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.501+0000 7fda2a7fc640 1 -- 172.21.15.18:0/2472662215 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fda2400a040 con 0x7fda2c101360 2024-02-17T12:51:20.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda36399740 1 Processor -- start 2024-02-17T12:51:20.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda36399740 1 -- start start 2024-02-17T12:51:20.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda36399740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda36399740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6d8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda36399740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6d87b0 0x55aa2b6daba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda36399740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2b63e280 con 0x55aa2b6d87b0 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda36399740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2b63b7b0 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda36399740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2b636f90 con 0x55aa2b6cf2b0 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda2a7fc640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fda140a23d0 0x7fda140a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda30c86640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6d87b0 0x55aa2b6daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda30c86640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6d87b0 0x55aa2b6daba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43434/0 (socket says 172.21.15.18:43434) 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda30c86640 1 -- 172.21.15.18:0/3334476818 learned_addr learned my addr 172.21.15.18:0/3334476818 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda31c88640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fda140a23d0 0x7fda140a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda31487640 1 --2- 172.21.15.18:0/3334476818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6d8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.502+0000 7fda2a7fc640 1 -- 172.21.15.18:0/2472662215 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fda240f0c10 con 0x7fda2c101360 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.503+0000 7fda30c86640 1 -- 172.21.15.18:0/3334476818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cf2b0 msgr2=0x55aa2b6cf690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.503+0000 7fda30c86640 1 --2- 172.21.15.18:0/3334476818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6cf690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.503+0000 7fda30c86640 1 -- 172.21.15.18:0/3334476818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6cfbd0 msgr2=0x55aa2b6d8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.503+0000 7fda30c86640 1 --2- 172.21.15.18:0/3334476818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6d8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.503+0000 7fda30c86640 1 -- 172.21.15.18:0/3334476818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa2b5f5570 con 0x55aa2b6d87b0 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.503+0000 7fda31c88640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fda140a23d0 0x7fda140a4890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00ab10 tx=0x7fda1c00a250 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.503+0000 7fda30c86640 1 --2- 172.21.15.18:0/3334476818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6d87b0 0x55aa2b6daba0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fda2000bce0 tx=0x7fda2000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37540f0d0200bccf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.503+0000 7fda137fe640 1 -- 172.21.15.18:0/3334476818 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2001a460 con 0x55aa2b6d87b0 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.503+0000 7fda137fe640 1 -- 172.21.15.18:0/3334476818 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda2001aaa0 con 0x55aa2b6d87b0 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.503+0000 7fda137fe640 1 -- 172.21.15.18:0/3334476818 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda200119c0 con 0x55aa2b6d87b0 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.504+0000 7fda36399740 1 -- 172.21.15.18:0/3334476818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6d87b0 msgr2=0x55aa2b6daba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.504+0000 7fda36399740 1 --2- 172.21.15.18:0/3334476818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6d87b0 0x55aa2b6daba0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fda2000bce0 tx=0x7fda2000f730 comp rx=0 tx=0).stop 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.504+0000 7fda36399740 1 -- 172.21.15.18:0/3334476818 shutdown_connections 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.504+0000 7fda36399740 1 --2- 172.21.15.18:0/3334476818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6d87b0 0x55aa2b6daba0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.504+0000 7fda36399740 1 --2- 172.21.15.18:0/3334476818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6d8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.504+0000 7fda36399740 1 --2- 172.21.15.18:0/3334476818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.504+0000 7fda36399740 1 -- 172.21.15.18:0/3334476818 >> 172.21.15.18:0/3334476818 conn(0x55aa2b6c9990 msgr2=0x55aa2b6c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.504+0000 7fda36399740 1 -- 172.21.15.18:0/3334476818 shutdown_connections 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.504+0000 7fda36399740 1 -- 172.21.15.18:0/3334476818 wait complete. 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.504+0000 7fda36399740 1 -- start start 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.504+0000 7fda36399740 1 Processor -- start 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda36399740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6d81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda36399740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6d5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda36399740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6d87b0 0x55aa2b6d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda36399740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2b30a430 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda36399740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2b2c6e00 con 0x55aa2b6cf2b0 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda36399740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2b5bbfd0 con 0x55aa2b6d87b0 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda36399740 10 client.0 _ll_register_callbacks cb 0x55aa2b611fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda30c86640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6d5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda30c86640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6d5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43442/0 (socket says 172.21.15.18:43442) 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda30c86640 1 -- 172.21.15.18:0/2325036181 learned_addr learned my addr 172.21.15.18:0/2325036181 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda31487640 1 --2- 172.21.15.18:0/2325036181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6d81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda30c86640 1 -- 172.21.15.18:0/2325036181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6d87b0 msgr2=0x55aa2b6d5cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda30c86640 1 --2- 172.21.15.18:0/2325036181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6d87b0 0x55aa2b6d5cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda30c86640 1 -- 172.21.15.18:0/2325036181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6cf2b0 msgr2=0x55aa2b6d81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda30c86640 1 --2- 172.21.15.18:0/2325036181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6d81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda30c86640 1 -- 172.21.15.18:0/2325036181 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2000b940 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda30c86640 1 --2- 172.21.15.18:0/2325036181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6d5580 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fda2000b890 tx=0x7fda2001a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcef08b5ee917c31 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda10ff9640 1 -- 172.21.15.18:0/2325036181 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20020070 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda10ff9640 1 -- 172.21.15.18:0/2325036181 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fda20005040 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.505+0000 7fda36399740 1 -- 172.21.15.18:0/2325036181 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa2b3367e0 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.506+0000 7fda10ff9640 1 -- 172.21.15.18:0/2325036181 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda20005200 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.506+0000 7fda36399740 10 client.16032 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:20.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.506+0000 7fda36399740 1 -- 172.21.15.18:0/2325036181 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55aa2b635610 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.506+0000 7fda10ff9640 1 -- 172.21.15.18:0/2325036181 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aa2b3367e0 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.506+0000 7fda10ff9640 1 -- 172.21.15.18:0/2325036181 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fda20004a50 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.506+0000 7fda36399740 10 client.16032 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.506+0000 7fda36399740 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.506+0000 7fda36399740 20 client.16032 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.506+0000 7fda36399740 1 --2- 172.21.15.18:0/2325036181 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa2b6e88e0 0x55aa2b6ead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.506+0000 7fda36399740 4 client.16032 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.506+0000 7fda36399740 1 -- 172.21.15.18:0/2325036181 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa2b373720 con 0x55aa2b6e88e0 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.506+0000 7fda31487640 1 --2- 172.21.15.18:0/2325036181 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa2b6e88e0 0x55aa2b6ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.507+0000 7fda31487640 1 --2- 172.21.15.18:0/2325036181 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa2b6e88e0 0x55aa2b6ead00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f54dfe29d1a52d2 server_cookie=fd6028f972c7de0 in_seq=0 out_seq=0 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.507+0000 7fda10ff9640 10 client.16032 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.507+0000 7fda10ff9640 1 -- 172.21.15.18:0/2325036181 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa2b373720 con 0x55aa2b6e88e0 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.507+0000 7fda10ff9640 10 client.16032 handle_command_reply: tid=0 2024-02-17T12:51:20.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.507+0000 7fda36399740 1 client.16032 shutdown 2024-02-17T12:51:20.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.507+0000 7fda36399740 1 -- 172.21.15.18:0/2325036181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 msgr2=0x55aa2b6d5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.507+0000 7fda36399740 1 --2- 172.21.15.18:0/2325036181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6d5580 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fda2000b890 tx=0x7fda2001a690 comp rx=0 tx=0).stop 2024-02-17T12:51:20.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.507+0000 7fda36399740 1 -- 172.21.15.18:0/2325036181 shutdown_connections 2024-02-17T12:51:20.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.508+0000 7fda36399740 1 --2- 172.21.15.18:0/2325036181 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa2b6e88e0 0x55aa2b6ead00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.508+0000 7fda36399740 1 --2- 172.21.15.18:0/2325036181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6d87b0 0x55aa2b6d5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.508+0000 7fda36399740 1 --2- 172.21.15.18:0/2325036181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6d5580 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.508+0000 7fda36399740 1 --2- 172.21.15.18:0/2325036181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6d81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.508+0000 7fda10ff9640 0 client.16032 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:20.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.508+0000 7fda36399740 1 -- 172.21.15.18:0/2325036181 >> 172.21.15.18:0/2325036181 conn(0x55aa2b6c9990 msgr2=0x55aa2b6cb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:20.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.508+0000 7fda36399740 1 -- 172.21.15.18:0/2325036181 shutdown_connections 2024-02-17T12:51:20.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.508+0000 7fda36399740 1 -- 172.21.15.18:0/2325036181 wait complete. 2024-02-17T12:51:20.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.508+0000 7fda36399740 20 client.16032 trim_cache size 0 max 16384 2024-02-17T12:51:20.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda36399740 1 Processor -- start 2024-02-17T12:51:20.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda36399740 1 -- start start 2024-02-17T12:51:20.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda36399740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6df660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda36399740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6dfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda36399740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6d87b0 0x55aa2b6d7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda36399740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2b6c50f0 con 0x55aa2b6cf2b0 2024-02-17T12:51:20.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda36399740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2b6cecf0 con 0x55aa2b6d87b0 2024-02-17T12:51:20.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda36399740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2b6cef10 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda31487640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6df660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda31487640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6df660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43458/0 (socket says 172.21.15.18:43458) 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda31487640 1 -- 172.21.15.18:0/1320212862 learned_addr learned my addr 172.21.15.18:0/1320212862 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda31487640 1 -- 172.21.15.18:0/1320212862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cfbd0 msgr2=0x55aa2b6dfba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda31487640 1 --2- 172.21.15.18:0/1320212862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6dfba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda31487640 1 -- 172.21.15.18:0/1320212862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6d87b0 msgr2=0x55aa2b6d7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda31487640 1 --2- 172.21.15.18:0/1320212862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6d87b0 0x55aa2b6d7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.523+0000 7fda31487640 1 -- 172.21.15.18:0/1320212862 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa2b5d1ac0 con 0x55aa2b6cf2b0 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda31487640 1 --2- 172.21.15.18:0/1320212862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6df660 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fda240ee340 tx=0x7fda24039d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca8be93c24e48d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda12ffd640 1 -- 172.21.15.18:0/1320212862 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240bdc80 con 0x55aa2b6cf2b0 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda12ffd640 1 -- 172.21.15.18:0/1320212862 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda240bd580 con 0x55aa2b6cf2b0 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda12ffd640 1 -- 172.21.15.18:0/1320212862 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240dd0b0 con 0x55aa2b6cf2b0 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda36399740 1 -- 172.21.15.18:0/1320212862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cf2b0 msgr2=0x55aa2b6df660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda36399740 1 --2- 172.21.15.18:0/1320212862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6df660 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fda240ee340 tx=0x7fda24039d00 comp rx=0 tx=0).stop 2024-02-17T12:51:20.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda36399740 1 -- 172.21.15.18:0/1320212862 shutdown_connections 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda36399740 1 --2- 172.21.15.18:0/1320212862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6d87b0 0x55aa2b6d7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda36399740 1 --2- 172.21.15.18:0/1320212862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6dfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda36399740 1 --2- 172.21.15.18:0/1320212862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6df660 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda36399740 1 -- 172.21.15.18:0/1320212862 >> 172.21.15.18:0/1320212862 conn(0x55aa2b6c9990 msgr2=0x55aa2b6d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda36399740 1 -- 172.21.15.18:0/1320212862 shutdown_connections 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.524+0000 7fda36399740 1 -- 172.21.15.18:0/1320212862 wait complete. 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda36399740 1 -- start start 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda36399740 1 Processor -- start 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda36399740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6de920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda36399740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6dee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda36399740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6d87b0 0x55aa2b738e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda36399740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2b7393b0 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda36399740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2b7394f0 con 0x55aa2b6d87b0 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda36399740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2b7396b0 con 0x55aa2b6cf2b0 2024-02-17T12:51:20.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda36399740 10 client.0 _ll_register_callbacks cb 0x7fda2000b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda30c86640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda30c86640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6dee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43470/0 (socket says 172.21.15.18:43470) 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda30c86640 1 -- 172.21.15.18:0/3486347598 learned_addr learned my addr 172.21.15.18:0/3486347598 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda31487640 1 --2- 172.21.15.18:0/3486347598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6de920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda30c86640 1 -- 172.21.15.18:0/3486347598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cf2b0 msgr2=0x55aa2b6de920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda30c86640 1 --2- 172.21.15.18:0/3486347598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6de920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda30c86640 1 -- 172.21.15.18:0/3486347598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6d87b0 msgr2=0x55aa2b738e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.525+0000 7fda30c86640 1 --2- 172.21.15.18:0/3486347598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6d87b0 0x55aa2b738e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda30c86640 1 -- 172.21.15.18:0/3486347598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c1a6a00 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda30c86640 1 --2- 172.21.15.18:0/3486347598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6dee60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fda200103f0 tx=0x7fda20005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f552e96258d2c4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda10ff9640 1 -- 172.21.15.18:0/3486347598 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20020070 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda10ff9640 1 -- 172.21.15.18:0/3486347598 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fda20000e40 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda36399740 1 -- 172.21.15.18:0/3486347598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda2001aaa0 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda10ff9640 1 -- 172.21.15.18:0/3486347598 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda2001a700 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda36399740 10 client.16038 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda36399740 1 -- 172.21.15.18:0/3486347598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55aa2b5f76a0 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda10ff9640 1 -- 172.21.15.18:0/3486347598 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda2001aaa0 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda10ff9640 1 -- 172.21.15.18:0/3486347598 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fda200107c0 con 0x55aa2b6cfbd0 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda36399740 10 client.16038 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda36399740 10 client.16038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda36399740 20 client.16038 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda36399740 1 --2- 172.21.15.18:0/3486347598 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa2b740d20 0x55aa2b743140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda36399740 4 client.16038 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.526+0000 7fda36399740 1 -- 172.21.15.18:0/3486347598 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aa2b3c2890 con 0x55aa2b740d20 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda31487640 1 --2- 172.21.15.18:0/3486347598 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa2b740d20 0x55aa2b743140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda10ff9640 10 client.16038 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda31487640 1 --2- 172.21.15.18:0/3486347598 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa2b740d20 0x55aa2b743140 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6ad0122b59a78a server_cookie=17082755547c787e in_seq=0 out_seq=0 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda10ff9640 1 -- 172.21.15.18:0/3486347598 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55aa2b3c2890 con 0x55aa2b740d20 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda10ff9640 10 client.16038 handle_command_reply: tid=0 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda36399740 1 client.16038 shutdown 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda36399740 1 -- 172.21.15.18:0/3486347598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 msgr2=0x55aa2b6dee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda36399740 1 --2- 172.21.15.18:0/3486347598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6dee60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fda200103f0 tx=0x7fda20005ec0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda36399740 1 -- 172.21.15.18:0/3486347598 shutdown_connections 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda36399740 1 --2- 172.21.15.18:0/3486347598 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa2b740d20 0x55aa2b743140 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda36399740 1 --2- 172.21.15.18:0/3486347598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa2b6d87b0 0x55aa2b738e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda10ff9640 0 client.16038 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda36399740 1 --2- 172.21.15.18:0/3486347598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa2b6cfbd0 0x55aa2b6dee60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda36399740 1 --2- 172.21.15.18:0/3486347598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa2b6cf2b0 0x55aa2b6de920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.527+0000 7fda36399740 1 -- 172.21.15.18:0/3486347598 >> 172.21.15.18:0/3486347598 conn(0x55aa2b6c9990 msgr2=0x55aa2b6d3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda36399740 1 -- 172.21.15.18:0/3486347598 shutdown_connections 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda36399740 1 -- 172.21.15.18:0/3486347598 wait complete. 2024-02-17T12:51:20.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda36399740 20 client.16038 trim_cache size 0 max 16384 2024-02-17T12:51:20.526 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:51:20.526 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:51:20.526 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:51:20.526 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:51:20.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda336eb640 1 -- 172.21.15.18:0/2472662215 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fda140a23d0 msgr2=0x7fda140a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda336eb640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fda140a23d0 0x7fda140a4890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00ab10 tx=0x7fda1c00a250 comp rx=0 tx=0).stop 2024-02-17T12:51:20.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda336eb640 1 -- 172.21.15.18:0/2472662215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda2c101360 msgr2=0x7fda2c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda336eb640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda2c101360 0x7fda2c073480 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fda24004af0 tx=0x7fda24034cc0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda336eb640 1 -- 172.21.15.18:0/2472662215 shutdown_connections 2024-02-17T12:51:20.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda336eb640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fda140a23d0 0x7fda140a4890 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda336eb640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda2c073f00 0x7fda2c1a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda336eb640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c106430 0x7fda2c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda336eb640 1 --2- 172.21.15.18:0/2472662215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda2c101360 0x7fda2c073480 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda336eb640 1 -- 172.21.15.18:0/2472662215 >> 172.21.15.18:0/2472662215 conn(0x7fda2c0fcc20 msgr2=0x7fda2c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:20.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda336eb640 1 -- 172.21.15.18:0/2472662215 shutdown_connections 2024-02-17T12:51:20.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.528+0000 7fda336eb640 1 -- 172.21.15.18:0/2472662215 wait complete. 2024-02-17T12:51:20.530 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:51:20.531 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:51:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.642+0000 7f565974b640 1 -- 172.21.15.18:0/641783235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5654106480 msgr2=0x7f565410e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.642+0000 7f565974b640 1 --2- 172.21.15.18:0/641783235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5654106480 0x7f565410e050 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5644009cd0 tx=0x7f564402c100 comp rx=0 tx=0).stop 2024-02-17T12:51:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.642+0000 7f565974b640 1 -- 172.21.15.18:0/641783235 shutdown_connections 2024-02-17T12:51:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.642+0000 7f565974b640 1 --2- 172.21.15.18:0/641783235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5654106480 0x7f565410e050 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.642+0000 7f565974b640 1 --2- 172.21.15.18:0/641783235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654101d60 0x7f5654105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.642+0000 7f565974b640 1 --2- 172.21.15.18:0/641783235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56541013b0 0x7f5654101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.642+0000 7f565974b640 1 -- 172.21.15.18:0/641783235 >> 172.21.15.18:0/641783235 conn(0x7f56540fcbf0 msgr2=0x7f56540ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.642+0000 7f565974b640 1 -- 172.21.15.18:0/641783235 shutdown_connections 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.642+0000 7f565974b640 1 -- 172.21.15.18:0/641783235 wait complete. 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f565974b640 1 Processor -- start 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f565974b640 1 -- start start 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f565974b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56541013b0 0x7f5654073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f565974b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5654106480 0x7f56540739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f565974b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654073f00 0x7f56541a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f565974b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56541a6020 con 0x7f5654106480 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f565974b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56541a6190 con 0x7f5654073f00 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f565974b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56541a6350 con 0x7f56541013b0 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f56527fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5654106480 0x7f56540739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f56527fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5654106480 0x7f56540739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43480/0 (socket says 172.21.15.18:43480) 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f56527fc640 1 -- 172.21.15.18:0/1133686246 learned_addr learned my addr 172.21.15.18:0/1133686246 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f56537fe640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654073f00 0x7f56541a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f5652ffd640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56541013b0 0x7f5654073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f56527fc640 1 -- 172.21.15.18:0/1133686246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56541013b0 msgr2=0x7f5654073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f56527fc640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56541013b0 0x7f5654073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f56527fc640 1 -- 172.21.15.18:0/1133686246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654073f00 msgr2=0x7f56541a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.643+0000 7f56527fc640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654073f00 0x7f56541a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.644+0000 7f56527fc640 1 -- 172.21.15.18:0/1133686246 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644009900 con 0x7f5654106480 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.644+0000 7f5652ffd640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56541013b0 0x7f5654073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.644+0000 7f56527fc640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5654106480 0x7f56540739c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f563c00d770 tx=0x7f563c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.644+0000 7f564bfff640 1 -- 172.21.15.18:0/1133686246 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c009c30 con 0x7f5654106480 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.644+0000 7f564bfff640 1 -- 172.21.15.18:0/1133686246 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f563c00de80 con 0x7f5654106480 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.644+0000 7f565974b640 1 -- 172.21.15.18:0/1133686246 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56541a6630 con 0x7f5654106480 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.644+0000 7f564bfff640 1 -- 172.21.15.18:0/1133686246 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c00f6b0 con 0x7f5654106480 2024-02-17T12:51:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.644+0000 7f565974b640 1 -- 172.21.15.18:0/1133686246 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56541a6b40 con 0x7f5654106480 2024-02-17T12:51:20.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.645+0000 7f565974b640 1 -- 172.21.15.18:0/1133686246 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56540683c0 con 0x7f5654106480 2024-02-17T12:51:20.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.647+0000 7f564bfff640 1 -- 172.21.15.18:0/1133686246 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f563c005030 con 0x7f5654106480 2024-02-17T12:51:20.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.647+0000 7f564bfff640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56400a2380 0x7f56400a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.647+0000 7f564bfff640 1 -- 172.21.15.18:0/1133686246 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f563c0ce070 con 0x7f5654106480 2024-02-17T12:51:20.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.647+0000 7f5652ffd640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56400a2380 0x7f56400a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.648+0000 7f5652ffd640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56400a2380 0x7f56400a4840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f564c00a8b0 tx=0x7f564c008040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:20.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.651+0000 7f564bfff640 1 -- 172.21.15.18:0/1133686246 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f563c096ba0 con 0x7f5654106480 2024-02-17T12:51:20.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.876+0000 7f565974b640 1 -- 172.21.15.18:0/1133686246 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5654109710 con 0x7f5654106480 2024-02-17T12:51:20.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.876+0000 7f564bfff640 1 -- 172.21.15.18:0/1133686246 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11399 (secure 0 0 0) 0x7f563c021020 con 0x7f5654106480 2024-02-17T12:51:20.868 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:51:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:51:04.933427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:51:20.868 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 28 2024-02-17T12:51:20.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.878+0000 7f565974b640 1 -- 172.21.15.18:0/1133686246 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56400a2380 msgr2=0x7f56400a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.878+0000 7f565974b640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56400a2380 0x7f56400a4840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f564c00a8b0 tx=0x7f564c008040 comp rx=0 tx=0).stop 2024-02-17T12:51:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.878+0000 7f565974b640 1 -- 172.21.15.18:0/1133686246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5654106480 msgr2=0x7f56540739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.878+0000 7f565974b640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5654106480 0x7f56540739c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f563c00d770 tx=0x7f563c00dc30 comp rx=0 tx=0).stop 2024-02-17T12:51:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.878+0000 7f565974b640 1 -- 172.21.15.18:0/1133686246 shutdown_connections 2024-02-17T12:51:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.878+0000 7f565974b640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56400a2380 0x7f56400a4840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.878+0000 7f565974b640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5654073f00 0x7f56541a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.878+0000 7f565974b640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5654106480 0x7f56540739c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.878+0000 7f565974b640 1 --2- 172.21.15.18:0/1133686246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56541013b0 0x7f5654073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.878+0000 7f565974b640 1 -- 172.21.15.18:0/1133686246 >> 172.21.15.18:0/1133686246 conn(0x7f56540fcbf0 msgr2=0x7f56540feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.879+0000 7f565974b640 1 -- 172.21.15.18:0/1133686246 shutdown_connections 2024-02-17T12:51:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.879+0000 7f565974b640 1 -- 172.21.15.18:0/1133686246 wait complete. 2024-02-17T12:51:20.881 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:51:20.881 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:51:20.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.994+0000 7f5a0e556640 1 -- 172.21.15.18:0/547528357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a08100310 msgr2=0x7f5a081006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.994+0000 7f5a0e556640 1 --2- 172.21.15.18:0/547528357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a08100310 0x7f5a081006f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009930 tx=0x7f59fc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.994+0000 7f5a0e556640 1 -- 172.21.15.18:0/547528357 shutdown_connections 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.994+0000 7f5a0e556640 1 --2- 172.21.15.18:0/547528357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a081053e0 0x7f5a0810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.994+0000 7f5a0e556640 1 --2- 172.21.15.18:0/547528357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a08100cc0 0x7f5a08104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.994+0000 7f5a0e556640 1 --2- 172.21.15.18:0/547528357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a08100310 0x7f5a081006f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.994+0000 7f5a0e556640 1 -- 172.21.15.18:0/547528357 >> 172.21.15.18:0/547528357 conn(0x7f5a080771e0 msgr2=0x7f5a080775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.994+0000 7f5a0e556640 1 -- 172.21.15.18:0/547528357 shutdown_connections 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.994+0000 7f5a0e556640 1 -- 172.21.15.18:0/547528357 wait complete. 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.995+0000 7f5a0e556640 1 Processor -- start 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.995+0000 7f5a0e556640 1 -- start start 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.995+0000 7f5a0e556640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a08100310 0x7f5a0819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.995+0000 7f5a0e556640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081053e0 0x7f5a0819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.995+0000 7f5a0e556640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a08199840 0x7f5a08199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.995+0000 7f5a0e556640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0819a2f0 con 0x7f5a08100310 2024-02-17T12:51:20.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.995+0000 7f5a0e556640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0819a460 con 0x7f5a081053e0 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.995+0000 7f5a0e556640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0819a5d0 con 0x7f5a08199840 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a07fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a08100310 0x7f5a0819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a07fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a08100310 0x7f5a0819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43496/0 (socket says 172.21.15.18:43496) 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a07fff640 1 -- 172.21.15.18:0/2197794191 learned_addr learned my addr 172.21.15.18:0/2197794191 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a077fe640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081053e0 0x7f5a0819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a0caf3640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a08199840 0x7f5a08199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a07fff640 1 -- 172.21.15.18:0/2197794191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a08199840 msgr2=0x7f5a08199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a07fff640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a08199840 0x7f5a08199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a07fff640 1 -- 172.21.15.18:0/2197794191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081053e0 msgr2=0x7f5a0819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a07fff640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081053e0 0x7f5a0819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a07fff640 1 -- 172.21.15.18:0/2197794191 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc009560 con 0x7f5a08100310 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a07fff640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a08100310 0x7f5a0819f6d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009230 tx=0x7f59fc02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a057fa640 1 -- 172.21.15.18:0/2197794191 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc004350 con 0x7f5a08100310 2024-02-17T12:51:20.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a0e556640 1 -- 172.21.15.18:0/2197794191 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a081a6520 con 0x7f5a08100310 2024-02-17T12:51:20.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a057fa640 1 -- 172.21.15.18:0/2197794191 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59fc0044b0 con 0x7f5a08100310 2024-02-17T12:51:20.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a057fa640 1 -- 172.21.15.18:0/2197794191 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc003d70 con 0x7f5a08100310 2024-02-17T12:51:20.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.996+0000 7f5a0e556640 1 -- 172.21.15.18:0/2197794191 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a081a69e0 con 0x7f5a08100310 2024-02-17T12:51:20.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.998+0000 7f5a057fa640 1 -- 172.21.15.18:0/2197794191 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f59fc004df0 con 0x7f5a08100310 2024-02-17T12:51:20.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a11207740 1 Processor -- start 2024-02-17T12:51:20.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a11207740 1 -- start start 2024-02-17T12:51:20.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a057fa640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f59f00a2350 0x7f59f00a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a11207740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a11207740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 0x5642557bb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a11207740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 0x5642557bdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a11207740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425576d960 con 0x5642557bb750 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a11207740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564255721310 con 0x5642557b2250 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a11207740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425571e840 con 0x5642557b2b70 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a0caf3640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f59f00a2350 0x7f59f00a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a057fa640 1 -- 172.21.15.18:0/2197794191 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f59fc0f0360 con 0x7f5a08100310 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a0caf3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 0x5642557bdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a0caf3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 0x5642557bdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43504/0 (socket says 172.21.15.18:43504) 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a0caf3640 1 -- 172.21.15.18:0/3198354424 learned_addr learned my addr 172.21.15.18:0/3198354424 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a077fe640 1 --2- 172.21.15.18:0/3198354424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 0x5642557bb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:20.999+0000 7f5a07fff640 1 --2- 172.21.15.18:0/3198354424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.000+0000 7f5a0caf3640 1 -- 172.21.15.18:0/3198354424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 msgr2=0x5642557bb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.000+0000 7f5a0caf3640 1 --2- 172.21.15.18:0/3198354424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 0x5642557bb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.000+0000 7f5a0caf3640 1 -- 172.21.15.18:0/3198354424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 msgr2=0x5642557b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.000+0000 7f5a0caf3640 1 --2- 172.21.15.18:0/3198354424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.000+0000 7f5a0caf3640 1 -- 172.21.15.18:0/3198354424 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642556dc060 con 0x5642557bb750 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.000+0000 7f5a077fe640 1 --2- 172.21.15.18:0/3198354424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 0x5642557bb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.000+0000 7f5a0caf3640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f59f00a2350 0x7f59f00a4810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009fd0 tx=0x7f5a00009d00 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.000+0000 7f5a0caf3640 1 --2- 172.21.15.18:0/3198354424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 0x5642557bdb40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009280 tx=0x7f5a0001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94e40cf1bccbc58 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.000+0000 7f59ee7fc640 1 -- 172.21.15.18:0/3198354424 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00004b20 con 0x5642557bb750 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.000+0000 7f59ee7fc640 1 -- 172.21.15.18:0/3198354424 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a00004c80 con 0x5642557bb750 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.000+0000 7f59ee7fc640 1 -- 172.21.15.18:0/3198354424 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a000054a0 con 0x5642557bb750 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 -- 172.21.15.18:0/3198354424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 msgr2=0x5642557bdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 --2- 172.21.15.18:0/3198354424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 0x5642557bdb40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009280 tx=0x7f5a0001c000 comp rx=0 tx=0).stop 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 -- 172.21.15.18:0/3198354424 shutdown_connections 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 --2- 172.21.15.18:0/3198354424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 0x5642557bdb40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 --2- 172.21.15.18:0/3198354424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 0x5642557bb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 --2- 172.21.15.18:0/3198354424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 -- 172.21.15.18:0/3198354424 >> 172.21.15.18:0/3198354424 conn(0x5642557ac980 msgr2=0x5642557acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 -- 172.21.15.18:0/3198354424 shutdown_connections 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 -- 172.21.15.18:0/3198354424 wait complete. 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 -- start start 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 Processor -- start 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 0x5642557b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 0x5642557b9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425571a020 con 0x5642557bb750 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.001+0000 7f5a11207740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642553ed430 con 0x5642557b2250 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a11207740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642553a9e00 con 0x5642557b2b70 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a11207740 10 client.0 _ll_register_callbacks cb 0x5642556c1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a0caf3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 0x5642557b9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a07fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a0caf3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 0x5642557b9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43508/0 (socket says 172.21.15.18:43508) 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a07fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557b8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:40856/0 (socket says 172.21.15.18:40856) 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a0caf3640 1 -- 172.21.15.18:0/303156028 learned_addr learned my addr 172.21.15.18:0/303156028 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a077fe640 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 0x5642557b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a0caf3640 1 -- 172.21.15.18:0/303156028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 msgr2=0x5642557b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a0caf3640 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 0x5642557b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a0caf3640 1 -- 172.21.15.18:0/303156028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 msgr2=0x5642557b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a0caf3640 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a0caf3640 1 -- 172.21.15.18:0/303156028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0001a040 con 0x5642557bb750 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a07fff640 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a0caf3640 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 0x5642557b9280 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000cd80 tx=0x7f5a00004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fec805f600eae90 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f59e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f59e77fe640 1 -- 172.21.15.18:0/303156028 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000cde0 con 0x5642557bb750 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f59e77fe640 1 -- 172.21.15.18:0/303156028 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5a00004df0 con 0x5642557bb750 2024-02-17T12:51:21.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a11207740 1 -- 172.21.15.18:0/303156028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642554197e0 con 0x5642557bb750 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f59e77fe640 1 -- 172.21.15.18:0/303156028 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a0000c8c0 con 0x5642557bb750 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a11207740 10 client.16056 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.002+0000 7f5a11207740 1 -- 172.21.15.18:0/303156028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5642557186a0 con 0x5642557bb750 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.003+0000 7f59e77fe640 1 -- 172.21.15.18:0/303156028 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5642554197e0 con 0x5642557bb750 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.003+0000 7f59e77fe640 1 -- 172.21.15.18:0/303156028 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5a00026020 con 0x5642557bb750 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.003+0000 7f5a11207740 10 client.16056 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.003+0000 7f5a11207740 10 client.16056 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.003+0000 7f5a11207740 20 client.16056 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.003+0000 7f5a11207740 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5642557cba50 0x5642557cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.003+0000 7f5a11207740 4 client.16056 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.003+0000 7f5a11207740 1 -- 172.21.15.18:0/303156028 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564255456720 con 0x5642557cba50 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.003+0000 7f5a07fff640 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5642557cba50 0x5642557cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.003+0000 7f5a07fff640 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5642557cba50 0x5642557cde70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3ada6cb0c8c2bdd5 server_cookie=a1cc7e3269b2c39b in_seq=0 out_seq=0 2024-02-17T12:51:21.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.003+0000 7f59e77fe640 10 client.16056 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f59e77fe640 1 -- 172.21.15.18:0/303156028 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564255456720 con 0x5642557cba50 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f59e77fe640 10 client.16056 handle_command_reply: tid=0 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f5a11207740 1 client.16056 shutdown 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f5a11207740 1 -- 172.21.15.18:0/303156028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 msgr2=0x5642557b9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f5a11207740 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 0x5642557b9280 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000cd80 tx=0x7f5a00004c10 comp rx=0 tx=0).stop 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f5a11207740 1 -- 172.21.15.18:0/303156028 shutdown_connections 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f5a11207740 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557bb750 0x5642557b9280 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f5a11207740 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 0x5642557b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f5a11207740 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5642557cba50 0x5642557cde70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f5a11207740 1 --2- 172.21.15.18:0/303156028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f59e77fe640 0 client.16056 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f5a11207740 1 -- 172.21.15.18:0/303156028 >> 172.21.15.18:0/303156028 conn(0x5642557ac980 msgr2=0x5642557ae6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f5a11207740 1 -- 172.21.15.18:0/303156028 shutdown_connections 2024-02-17T12:51:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f5a11207740 1 -- 172.21.15.18:0/303156028 wait complete. 2024-02-17T12:51:21.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.004+0000 7f5a11207740 20 client.16056 trim_cache size 0 max 16384 2024-02-17T12:51:21.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.016+0000 7f5a11207740 1 Processor -- start 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.016+0000 7f5a11207740 1 -- start start 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.016+0000 7f5a11207740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557baf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.016+0000 7f5a11207740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2b70 0x5642557b90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.016+0000 7f5a11207740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557bb750 0x5642557b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.016+0000 7f5a11207740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425569f130 con 0x5642557b2b70 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.016+0000 7f5a11207740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642557b29d0 con 0x5642557b2250 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.016+0000 7f5a11207740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642557cb820 con 0x5642557bb750 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.016+0000 7f5a077fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2b70 0x5642557b90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.016+0000 7f5a077fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2b70 0x5642557b90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43524/0 (socket says 172.21.15.18:43524) 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.016+0000 7f5a077fe640 1 -- 172.21.15.18:0/3128242000 learned_addr learned my addr 172.21.15.18:0/3128242000 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.017+0000 7f5a0caf3640 1 --2- 172.21.15.18:0/3128242000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557bb750 0x5642557b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.017+0000 7f5a07fff640 1 --2- 172.21.15.18:0/3128242000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557baf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.017+0000 7f5a077fe640 1 -- 172.21.15.18:0/3128242000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557bb750 msgr2=0x5642557b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.017+0000 7f5a077fe640 1 --2- 172.21.15.18:0/3128242000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557bb750 0x5642557b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.017+0000 7f5a077fe640 1 -- 172.21.15.18:0/3128242000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 msgr2=0x5642557baf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.017+0000 7f5a077fe640 1 --2- 172.21.15.18:0/3128242000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557baf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.017+0000 7f5a077fe640 1 -- 172.21.15.18:0/3128242000 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642556b4c20 con 0x5642557b2b70 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.017+0000 7f5a0caf3640 1 --2- 172.21.15.18:0/3128242000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557bb750 0x5642557b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.017+0000 7f5a077fe640 1 --2- 172.21.15.18:0/3128242000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2b70 0x5642557b90c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5a08069620 tx=0x7f59f8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a41faabba820f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.017+0000 7f59edffb640 1 -- 172.21.15.18:0/3128242000 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8020070 con 0x5642557b2b70 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.017+0000 7f59edffb640 1 -- 172.21.15.18:0/3128242000 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59f8018040 con 0x5642557b2b70 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.017+0000 7f59edffb640 1 -- 172.21.15.18:0/3128242000 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8017050 con 0x5642557b2b70 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 -- 172.21.15.18:0/3128242000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2b70 msgr2=0x5642557b90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 --2- 172.21.15.18:0/3128242000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2b70 0x5642557b90c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5a08069620 tx=0x7f59f8005f90 comp rx=0 tx=0).stop 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 -- 172.21.15.18:0/3128242000 shutdown_connections 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 --2- 172.21.15.18:0/3128242000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557bb750 0x5642557b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 --2- 172.21.15.18:0/3128242000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2b70 0x5642557b90c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 --2- 172.21.15.18:0/3128242000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557b2250 0x5642557baf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 -- 172.21.15.18:0/3128242000 >> 172.21.15.18:0/3128242000 conn(0x5642557ac980 msgr2=0x5642557d27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 -- 172.21.15.18:0/3128242000 shutdown_connections 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 -- 172.21.15.18:0/3128242000 wait complete. 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 -- start start 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 Processor -- start 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2250 0x5642557c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 0x5642557c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557bb750 0x5642557c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564255822030 con 0x5642557b2250 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642558221a0 con 0x5642557bb750 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564255822360 con 0x5642557b2b70 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.018+0000 7f5a11207740 10 client.0 _ll_register_callbacks cb 0x7f5a00012650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a07fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2250 0x5642557c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a07fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2250 0x5642557c11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43540/0 (socket says 172.21.15.18:43540) 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a07fff640 1 -- 172.21.15.18:0/2152587339 learned_addr learned my addr 172.21.15.18:0/2152587339 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a0caf3640 1 --2- 172.21.15.18:0/2152587339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557bb750 0x5642557c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a07fff640 1 -- 172.21.15.18:0/2152587339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 msgr2=0x5642557c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a07fff640 1 --2- 172.21.15.18:0/2152587339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 0x5642557c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a07fff640 1 -- 172.21.15.18:0/2152587339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557bb750 msgr2=0x5642557c1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a07fff640 1 --2- 172.21.15.18:0/2152587339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557bb750 0x5642557c1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a07fff640 1 -- 172.21.15.18:0/2152587339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f80099e0 con 0x5642557b2250 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a07fff640 1 --2- 172.21.15.18:0/2152587339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2250 0x5642557c11d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f59fc005ec0 tx=0x7f59fc0bcb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b82afe3d3eb806 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f59e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f59e77fe640 1 -- 172.21.15.18:0/2152587339 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc0bcc60 con 0x5642557b2250 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f59e77fe640 1 -- 172.21.15.18:0/2152587339 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f59fc0c0db0 con 0x5642557b2250 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a11207740 1 -- 172.21.15.18:0/2152587339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a00004c80 con 0x5642557b2250 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f59e77fe640 1 -- 172.21.15.18:0/2152587339 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59fc0c5400 con 0x5642557b2250 2024-02-17T12:51:21.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a11207740 10 client.16062 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f5a11207740 1 -- 172.21.15.18:0/2152587339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5642556da8c0 con 0x5642557b2250 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.019+0000 7f59e77fe640 1 -- 172.21.15.18:0/2152587339 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a00004c80 con 0x5642557b2250 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f59e77fe640 1 -- 172.21.15.18:0/2152587339 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f59fc03c070 con 0x5642557b2250 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 10 client.16062 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 10 client.16062 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 20 client.16062 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 1 --2- 172.21.15.18:0/2152587339 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x564255823970 0x564255825d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 4 client.16062 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 1 -- 172.21.15.18:0/2152587339 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5642554a5890 con 0x564255823970 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a077fe640 1 --2- 172.21.15.18:0/2152587339 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x564255823970 0x564255825d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a077fe640 1 --2- 172.21.15.18:0/2152587339 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x564255823970 0x564255825d60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cd92af077bfad206 server_cookie=bdfecf434fba4417 in_seq=0 out_seq=0 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f59e77fe640 10 client.16062 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f59e77fe640 1 -- 172.21.15.18:0/2152587339 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5642554a5890 con 0x564255823970 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f59e77fe640 10 client.16062 handle_command_reply: tid=0 2024-02-17T12:51:21.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 1 client.16062 shutdown 2024-02-17T12:51:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 1 -- 172.21.15.18:0/2152587339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2250 msgr2=0x5642557c11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 1 --2- 172.21.15.18:0/2152587339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2250 0x5642557c11d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f59fc005ec0 tx=0x7f59fc0bcb50 comp rx=0 tx=0).stop 2024-02-17T12:51:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 1 -- 172.21.15.18:0/2152587339 shutdown_connections 2024-02-17T12:51:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 1 --2- 172.21.15.18:0/2152587339 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x564255823970 0x564255825d60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 1 --2- 172.21.15.18:0/2152587339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642557bb750 0x5642557c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 1 --2- 172.21.15.18:0/2152587339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642557b2b70 0x5642557c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f59e77fe640 0 client.16062 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:51:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.020+0000 7f5a11207740 1 --2- 172.21.15.18:0/2152587339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642557b2250 0x5642557c11d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a11207740 1 -- 172.21.15.18:0/2152587339 >> 172.21.15.18:0/2152587339 conn(0x5642557ac980 msgr2=0x5642557ab2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a11207740 1 -- 172.21.15.18:0/2152587339 shutdown_connections 2024-02-17T12:51:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a11207740 1 -- 172.21.15.18:0/2152587339 wait complete. 2024-02-17T12:51:21.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a11207740 20 client.16062 trim_cache size 0 max 16384 2024-02-17T12:51:21.020 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:51:21.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a0e556640 1 -- 172.21.15.18:0/2197794191 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f59f00a2350 msgr2=0x7f59f00a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a0e556640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f59f00a2350 0x7f59f00a4810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009fd0 tx=0x7f5a00009d00 comp rx=0 tx=0).stop 2024-02-17T12:51:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a0e556640 1 -- 172.21.15.18:0/2197794191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a08100310 msgr2=0x7f5a0819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a0e556640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a08100310 0x7f5a0819f6d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009230 tx=0x7f59fc02c6c0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a0e556640 1 -- 172.21.15.18:0/2197794191 shutdown_connections 2024-02-17T12:51:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a0e556640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f59f00a2350 0x7f59f00a4810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a0e556640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a08199840 0x7f5a08199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a0e556640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081053e0 0x7f5a0819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a0e556640 1 --2- 172.21.15.18:0/2197794191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a08100310 0x7f5a0819f6d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a0e556640 1 -- 172.21.15.18:0/2197794191 >> 172.21.15.18:0/2197794191 conn(0x7f5a080771e0 msgr2=0x7f5a080ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a0e556640 1 -- 172.21.15.18:0/2197794191 shutdown_connections 2024-02-17T12:51:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.021+0000 7f5a0e556640 1 -- 172.21.15.18:0/2197794191 wait complete. 2024-02-17T12:51:21.023 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:51:21.024 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:51:21.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.136+0000 7fc338276640 1 -- 172.21.15.18:0/3253447176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3301053e0 msgr2=0x7fc33010cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.136+0000 7fc338276640 1 --2- 172.21.15.18:0/3253447176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3301053e0 0x7fc33010cfb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc328009cd0 tx=0x7fc32802c100 comp rx=0 tx=0).stop 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 -- 172.21.15.18:0/3253447176 shutdown_connections 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 --2- 172.21.15.18:0/3253447176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3301053e0 0x7fc33010cfb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 --2- 172.21.15.18:0/3253447176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc330100cc0 0x7fc330104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 --2- 172.21.15.18:0/3253447176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc330100310 0x7fc3301006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 -- 172.21.15.18:0/3253447176 >> 172.21.15.18:0/3253447176 conn(0x7fc3300771e0 msgr2=0x7fc3300775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 -- 172.21.15.18:0/3253447176 shutdown_connections 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 -- 172.21.15.18:0/3253447176 wait complete. 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 Processor -- start 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 -- start start 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc330100310 0x7fc33019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3301053e0 0x7fc33019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3301976c0 0x7fc330197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc330198190 con 0x7fc3301053e0 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc330198300 con 0x7fc330100310 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.137+0000 7fc338276640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3301984c0 con 0x7fc3301976c0 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc335811640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3301053e0 0x7fc33019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc335811640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3301053e0 0x7fc33019db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43554/0 (socket says 172.21.15.18:43554) 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc335811640 1 -- 172.21.15.18:0/1073028225 learned_addr learned my addr 172.21.15.18:0/1073028225 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc336012640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc330100310 0x7fc33019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc335811640 1 -- 172.21.15.18:0/1073028225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3301976c0 msgr2=0x7fc330197b40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc335811640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3301976c0 0x7fc330197b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc335811640 1 -- 172.21.15.18:0/1073028225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc330100310 msgr2=0x7fc33019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc335811640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc330100310 0x7fc33019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc335811640 1 -- 172.21.15.18:0/1073028225 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc328009900 con 0x7fc3301053e0 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc335811640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3301053e0 0x7fc33019db40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc32000e9c0 tx=0x7fc32000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc3277fe640 1 -- 172.21.15.18:0/1073028225 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320009700 con 0x7fc3301053e0 2024-02-17T12:51:21.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc3277fe640 1 -- 172.21.15.18:0/1073028225 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc32000cd80 con 0x7fc3301053e0 2024-02-17T12:51:21.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc338276640 1 -- 172.21.15.18:0/1073028225 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3301a4460 con 0x7fc3301053e0 2024-02-17T12:51:21.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc3277fe640 1 -- 172.21.15.18:0/1073028225 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320010430 con 0x7fc3301053e0 2024-02-17T12:51:21.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.138+0000 7fc338276640 1 -- 172.21.15.18:0/1073028225 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3301a4990 con 0x7fc3301053e0 2024-02-17T12:51:21.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.141+0000 7fc3277fe640 1 -- 172.21.15.18:0/1073028225 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fc320005000 con 0x7fc3301053e0 2024-02-17T12:51:21.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.141+0000 7fc33ae49740 1 Processor -- start 2024-02-17T12:51:21.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.141+0000 7fc33ae49740 1 -- start start 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.141+0000 7fc33ae49740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c170630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.141+0000 7fc33ae49740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c170b70 0x55f97c179210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.141+0000 7fc33ae49740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c179750 0x55f97c17bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.141+0000 7fc33ae49740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c12b960 con 0x55f97c179750 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.141+0000 7fc33ae49740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c0df310 con 0x55f97c170250 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.141+0000 7fc33ae49740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c0dc840 con 0x55f97c170b70 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc335811640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c179750 0x55f97c17bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc335811640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c179750 0x55f97c17bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43564/0 (socket says 172.21.15.18:43564) 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc335811640 1 -- 172.21.15.18:0/1669730003 learned_addr learned my addr 172.21.15.18:0/1669730003 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc3277fe640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc31c0a23d0 0x7fc31c0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc336813640 1 --2- 172.21.15.18:0/1669730003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c170630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc336813640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc31c0a23d0 0x7fc31c0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc3277fe640 1 -- 172.21.15.18:0/1073028225 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc320014070 con 0x7fc3301053e0 2024-02-17T12:51:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc335811640 1 -- 172.21.15.18:0/1669730003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c170b70 msgr2=0x55f97c179210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc335811640 1 --2- 172.21.15.18:0/1669730003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c170b70 0x55f97c179210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc335811640 1 -- 172.21.15.18:0/1669730003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 msgr2=0x55f97c170630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc335811640 1 --2- 172.21.15.18:0/1669730003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c170630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc335811640 1 -- 172.21.15.18:0/1669730003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f97c09a060 con 0x55f97c179750 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc335811640 1 --2- 172.21.15.18:0/1669730003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c179750 0x55f97c17bb40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc3200cbe00 tx=0x7fc320005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bda5ae329d9ec1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc336813640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc31c0a23d0 0x7fc31c0a4890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc328009ca0 tx=0x7fc32802c100 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc324bfe640 1 -- 172.21.15.18:0/1669730003 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32009b640 con 0x55f97c179750 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc324bfe640 1 -- 172.21.15.18:0/1669730003 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc32009af40 con 0x55f97c179750 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.142+0000 7fc324bfe640 1 -- 172.21.15.18:0/1669730003 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3200d1e80 con 0x55f97c179750 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.143+0000 7fc33ae49740 1 -- 172.21.15.18:0/1669730003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c179750 msgr2=0x55f97c17bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.143+0000 7fc33ae49740 1 --2- 172.21.15.18:0/1669730003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c179750 0x55f97c17bb40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc3200cbe00 tx=0x7fc320005880 comp rx=0 tx=0).stop 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.143+0000 7fc33ae49740 1 -- 172.21.15.18:0/1669730003 shutdown_connections 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.143+0000 7fc33ae49740 1 --2- 172.21.15.18:0/1669730003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c179750 0x55f97c17bb40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.143+0000 7fc33ae49740 1 --2- 172.21.15.18:0/1669730003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c170b70 0x55f97c179210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.143+0000 7fc33ae49740 1 --2- 172.21.15.18:0/1669730003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c170630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.143+0000 7fc33ae49740 1 -- 172.21.15.18:0/1669730003 >> 172.21.15.18:0/1669730003 conn(0x55f97c16a980 msgr2=0x55f97c16ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.143+0000 7fc33ae49740 1 -- 172.21.15.18:0/1669730003 shutdown_connections 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.143+0000 7fc33ae49740 1 -- 172.21.15.18:0/1669730003 wait complete. 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.143+0000 7fc33ae49740 1 -- start start 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc33ae49740 1 Processor -- start 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc33ae49740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c1767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc33ae49740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 0x55f97c176d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc33ae49740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 0x55f97c177270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc33ae49740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c0d8020 con 0x55f97c170b70 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc33ae49740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97bdab430 con 0x55f97c170250 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc33ae49740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97bd67e00 con 0x55f97c179750 2024-02-17T12:51:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc33ae49740 10 client.0 _ll_register_callbacks cb 0x55f97c07fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc335811640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 0x55f97c176d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc335811640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 0x55f97c176d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43570/0 (socket says 172.21.15.18:43570) 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc335811640 1 -- 172.21.15.18:0/2972800610 learned_addr learned my addr 172.21.15.18:0/2972800610 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc336012640 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c1767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc336813640 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 0x55f97c177270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc335811640 1 -- 172.21.15.18:0/2972800610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 msgr2=0x55f97c177270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc335811640 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 0x55f97c177270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc335811640 1 -- 172.21.15.18:0/2972800610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 msgr2=0x55f97c1767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc335811640 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c1767f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc335811640 1 -- 172.21.15.18:0/2972800610 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3301a4990 con 0x55f97c170b70 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc335811640 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 0x55f97c176d30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc320000c00 tx=0x7fc3200971e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b27722268e2f33 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.144+0000 7fc319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc319ffb640 1 -- 172.21.15.18:0/2972800610 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32009b6c0 con 0x55f97c170b70 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc319ffb640 1 -- 172.21.15.18:0/2972800610 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc3200052b0 con 0x55f97c170b70 2024-02-17T12:51:21.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc33ae49740 1 -- 172.21.15.18:0/2972800610 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f97bdd77e0 con 0x55f97c170b70 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc319ffb640 1 -- 172.21.15.18:0/2972800610 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc320096e90 con 0x55f97c170b70 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc33ae49740 10 client.16074 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc33ae49740 1 -- 172.21.15.18:0/2972800610 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f97c0d66a0 con 0x55f97c170b70 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc319ffb640 1 -- 172.21.15.18:0/2972800610 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f97bdd77e0 con 0x55f97c170b70 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc319ffb640 1 -- 172.21.15.18:0/2972800610 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc32001f060 con 0x55f97c170b70 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc33ae49740 10 client.16074 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc33ae49740 10 client.16074 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc33ae49740 20 client.16074 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc33ae49740 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f97c1897f0 0x55f97c18bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc33ae49740 4 client.16074 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc33ae49740 1 -- 172.21.15.18:0/2972800610 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f97be14720 con 0x55f97c1897f0 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.145+0000 7fc336012640 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f97c1897f0 0x55f97c18bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.146+0000 7fc319ffb640 10 client.16074 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.146+0000 7fc336012640 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f97c1897f0 0x55f97c18bbe0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4391dce6ac351a09 server_cookie=4950e664174916b5 in_seq=0 out_seq=0 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc319ffb640 1 -- 172.21.15.18:0/2972800610 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f97be14720 con 0x55f97c1897f0 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc319ffb640 10 client.16074 handle_command_reply: tid=0 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc33ae49740 1 client.16074 shutdown 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc33ae49740 1 -- 172.21.15.18:0/2972800610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 msgr2=0x55f97c176d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc33ae49740 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 0x55f97c176d30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc320000c00 tx=0x7fc3200971e0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc33ae49740 1 -- 172.21.15.18:0/2972800610 shutdown_connections 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc33ae49740 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f97c1897f0 0x55f97c18bbe0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc33ae49740 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 0x55f97c177270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc33ae49740 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 0x55f97c176d30 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc319ffb640 0 client.16074 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc33ae49740 1 --2- 172.21.15.18:0/2972800610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c1767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc33ae49740 1 -- 172.21.15.18:0/2972800610 >> 172.21.15.18:0/2972800610 conn(0x55f97c16a980 msgr2=0x55f97c16c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc33ae49740 1 -- 172.21.15.18:0/2972800610 shutdown_connections 2024-02-17T12:51:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc33ae49740 1 -- 172.21.15.18:0/2972800610 wait complete. 2024-02-17T12:51:21.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.147+0000 7fc33ae49740 20 client.16074 trim_cache size 0 max 16384 2024-02-17T12:51:21.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc33ae49740 1 Processor -- start 2024-02-17T12:51:21.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc33ae49740 1 -- start start 2024-02-17T12:51:21.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc33ae49740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170250 0x55f97c178f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc33ae49740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170b70 0x55f97c177090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc33ae49740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 0x55f97c173a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc33ae49740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c05d130 con 0x55f97c170250 2024-02-17T12:51:21.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc33ae49740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c16b980 con 0x55f97c170b70 2024-02-17T12:51:21.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc33ae49740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c190de0 con 0x55f97c179750 2024-02-17T12:51:21.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc336012640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170250 0x55f97c178f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc336012640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170250 0x55f97c178f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43572/0 (socket says 172.21.15.18:43572) 2024-02-17T12:51:21.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc336012640 1 -- 172.21.15.18:0/3020567800 learned_addr learned my addr 172.21.15.18:0/3020567800 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc336012640 1 -- 172.21.15.18:0/3020567800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 msgr2=0x55f97c173a50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:51:21.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc336012640 1 --2- 172.21.15.18:0/3020567800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 0x55f97c173a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc336012640 1 -- 172.21.15.18:0/3020567800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170b70 msgr2=0x55f97c177090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc336012640 1 --2- 172.21.15.18:0/3020567800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170b70 0x55f97c177090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.159+0000 7fc336012640 1 -- 172.21.15.18:0/3020567800 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f97c072c20 con 0x55f97c170250 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc336012640 1 --2- 172.21.15.18:0/3020567800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170250 0x55f97c178f20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009960 tx=0x7fc32c016800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a51194a3b62dc18 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc31bfff640 1 -- 172.21.15.18:0/3020567800 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c009c70 con 0x55f97c170250 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc31bfff640 1 -- 172.21.15.18:0/3020567800 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc32c01e070 con 0x55f97c170250 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc31bfff640 1 -- 172.21.15.18:0/3020567800 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0215e0 con 0x55f97c170250 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc33ae49740 1 -- 172.21.15.18:0/3020567800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170250 msgr2=0x55f97c178f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc33ae49740 1 --2- 172.21.15.18:0/3020567800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170250 0x55f97c178f20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009960 tx=0x7fc32c016800 comp rx=0 tx=0).stop 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc33ae49740 1 -- 172.21.15.18:0/3020567800 shutdown_connections 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc33ae49740 1 --2- 172.21.15.18:0/3020567800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 0x55f97c173a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc33ae49740 1 --2- 172.21.15.18:0/3020567800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170b70 0x55f97c177090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc33ae49740 1 --2- 172.21.15.18:0/3020567800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170250 0x55f97c178f20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc33ae49740 1 -- 172.21.15.18:0/3020567800 >> 172.21.15.18:0/3020567800 conn(0x55f97c16a980 msgr2=0x55f97c171b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc33ae49740 1 -- 172.21.15.18:0/3020567800 shutdown_connections 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.160+0000 7fc33ae49740 1 -- 172.21.15.18:0/3020567800 wait complete. 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc33ae49740 1 -- start start 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc33ae49740 1 Processor -- start 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc33ae49740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c17f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc33ae49740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 0x55f97c17f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc33ae49740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 0x55f97c17fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc33ae49740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c1e85f0 con 0x55f97c170b70 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc33ae49740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c1e8760 con 0x55f97c170250 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc33ae49740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97c1e8920 con 0x55f97c179750 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc33ae49740 10 client.0 _ll_register_callbacks cb 0x7fc320097010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc335811640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 0x55f97c17f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc335811640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 0x55f97c17f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43588/0 (socket says 172.21.15.18:43588) 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc335811640 1 -- 172.21.15.18:0/2999740142 learned_addr learned my addr 172.21.15.18:0/2999740142 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc336012640 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c17f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc336813640 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 0x55f97c17fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc335811640 1 -- 172.21.15.18:0/2999740142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 msgr2=0x55f97c17fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc335811640 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 0x55f97c17fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc335811640 1 -- 172.21.15.18:0/2999740142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 msgr2=0x55f97c17f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc335811640 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c17f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc335811640 1 -- 172.21.15.18:0/2999740142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f97c0988c0 con 0x55f97c170b70 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc336012640 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c17f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc336813640 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 0x55f97c17fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc335811640 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 0x55f97c17f6e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc3200978c0 tx=0x7fc32009adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51651878c2c05811 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.161+0000 7fc319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc319ffb640 1 -- 172.21.15.18:0/2999740142 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32009a430 con 0x55f97c170b70 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc319ffb640 1 -- 172.21.15.18:0/2999740142 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc32009a690 con 0x55f97c170b70 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc33ae49740 1 -- 172.21.15.18:0/2999740142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc32009af40 con 0x55f97c170b70 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc319ffb640 1 -- 172.21.15.18:0/2999740142 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc32009aa80 con 0x55f97c170b70 2024-02-17T12:51:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc33ae49740 10 client.16080 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc33ae49740 1 -- 172.21.15.18:0/2999740142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f97c07f050 con 0x55f97c170b70 2024-02-17T12:51:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc319ffb640 1 -- 172.21.15.18:0/2999740142 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc32009af40 con 0x55f97c170b70 2024-02-17T12:51:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc319ffb640 1 -- 172.21.15.18:0/2999740142 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc32001f070 con 0x55f97c170b70 2024-02-17T12:51:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc33ae49740 10 client.16080 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc33ae49740 10 client.16080 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:51:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc33ae49740 20 client.16080 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc33ae49740 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f97c1e9c10 0x55f97c1ec000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc33ae49740 4 client.16080 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:51:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.162+0000 7fc33ae49740 1 -- 172.21.15.18:0/2999740142 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f97be63890 con 0x55f97c1e9c10 2024-02-17T12:51:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.163+0000 7fc336012640 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f97c1e9c10 0x55f97c1ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc336012640 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f97c1e9c10 0x55f97c1ec000 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7053609f228d066c server_cookie=aaafb5e840e30955 in_seq=0 out_seq=0 2024-02-17T12:51:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc319ffb640 10 client.16080 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:51:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc319ffb640 1 -- 172.21.15.18:0/2999740142 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f97be63890 con 0x55f97c1e9c10 2024-02-17T12:51:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc319ffb640 10 client.16080 handle_command_reply: tid=0 2024-02-17T12:51:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc33ae49740 1 client.16080 shutdown 2024-02-17T12:51:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc33ae49740 1 -- 172.21.15.18:0/2999740142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 msgr2=0x55f97c17f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc33ae49740 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 0x55f97c17f6e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc3200978c0 tx=0x7fc32009adc0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc33ae49740 1 -- 172.21.15.18:0/2999740142 shutdown_connections 2024-02-17T12:51:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc33ae49740 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55f97c1e9c10 0x55f97c1ec000 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc33ae49740 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f97c179750 0x55f97c17fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc319ffb640 0 client.16080 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:51:21.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc33ae49740 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f97c170b70 0x55f97c17f6e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc33ae49740 1 --2- 172.21.15.18:0/2999740142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f97c170250 0x55f97c17f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.165+0000 7fc33ae49740 1 -- 172.21.15.18:0/2999740142 >> 172.21.15.18:0/2999740142 conn(0x55f97c16a980 msgr2=0x55f97c1692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc33ae49740 1 -- 172.21.15.18:0/2999740142 shutdown_connections 2024-02-17T12:51:21.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc33ae49740 1 -- 172.21.15.18:0/2999740142 wait complete. 2024-02-17T12:51:21.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc33ae49740 20 client.16080 trim_cache size 0 max 16384 2024-02-17T12:51:21.163 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:51:21.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc338276640 1 -- 172.21.15.18:0/1073028225 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc31c0a23d0 msgr2=0x7fc31c0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc338276640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc31c0a23d0 0x7fc31c0a4890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc328009ca0 tx=0x7fc32802c100 comp rx=0 tx=0).stop 2024-02-17T12:51:21.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc338276640 1 -- 172.21.15.18:0/1073028225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3301053e0 msgr2=0x7fc33019db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc338276640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3301053e0 0x7fc33019db40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc32000e9c0 tx=0x7fc32000ee80 comp rx=0 tx=0).stop 2024-02-17T12:51:21.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc338276640 1 -- 172.21.15.18:0/1073028225 shutdown_connections 2024-02-17T12:51:21.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc338276640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc31c0a23d0 0x7fc31c0a4890 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc338276640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3301976c0 0x7fc330197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc338276640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3301053e0 0x7fc33019db40 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc338276640 1 --2- 172.21.15.18:0/1073028225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc330100310 0x7fc33019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc338276640 1 -- 172.21.15.18:0/1073028225 >> 172.21.15.18:0/1073028225 conn(0x7fc3300771e0 msgr2=0x7fc3300fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc338276640 1 -- 172.21.15.18:0/1073028225 shutdown_connections 2024-02-17T12:51:21.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.166+0000 7fc338276640 1 -- 172.21.15.18:0/1073028225 wait complete. 2024-02-17T12:51:21.176 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:51:21.176 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:51:21.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50411 8.03 MB/sec warmup 36 sec latency 1786.448 ms 2024-02-17T12:51:21.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.308+0000 7f6cb192c640 1 -- 172.21.15.18:0/3600694803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cac0ff7b0 msgr2=0x7f6cac0ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.308+0000 7f6cb192c640 1 --2- 172.21.15.18:0/3600694803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cac0ff7b0 0x7f6cac0ffc30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009930 tx=0x7f6c9402bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:21.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.308+0000 7f6cb192c640 1 -- 172.21.15.18:0/3600694803 shutdown_connections 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.308+0000 7f6cb192c640 1 --2- 172.21.15.18:0/3600694803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cac100170 0x7f6cac10c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.308+0000 7f6cb192c640 1 --2- 172.21.15.18:0/3600694803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cac0ff7b0 0x7f6cac0ffc30 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.308+0000 7f6cb192c640 1 --2- 172.21.15.18:0/3600694803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac1057c0 0x7f6cac105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.308+0000 7f6cb192c640 1 -- 172.21.15.18:0/3600694803 >> 172.21.15.18:0/3600694803 conn(0x7f6cac076640 msgr2=0x7f6cac0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.308+0000 7f6cb192c640 1 -- 172.21.15.18:0/3600694803 shutdown_connections 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.308+0000 7f6cb192c640 1 -- 172.21.15.18:0/3600694803 wait complete. 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.309+0000 7f6cb192c640 1 Processor -- start 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.309+0000 7f6cb192c640 1 -- start start 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.309+0000 7f6cb192c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cac0ff7b0 0x7f6cac197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.309+0000 7f6cb192c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cac100170 0x7f6cac197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.309+0000 7f6cb192c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac1057c0 0x7f6cac19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.309+0000 7f6cb192c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac19c600 con 0x7f6cac0ff7b0 2024-02-17T12:51:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.309+0000 7f6cb192c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac19c770 con 0x7f6cac1057c0 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.309+0000 7f6cb192c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac19c930 con 0x7f6cac100170 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.309+0000 7f6caaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cac0ff7b0 0x7f6cac197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.309+0000 7f6caaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cac0ff7b0 0x7f6cac197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43612/0 (socket says 172.21.15.18:43612) 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.309+0000 7f6caaffd640 1 -- 172.21.15.18:0/982601425 learned_addr learned my addr 172.21.15.18:0/982601425 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6cab7fe640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac1057c0 0x7f6cac19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6caa7fc640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cac100170 0x7f6cac197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6caaffd640 1 -- 172.21.15.18:0/982601425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cac100170 msgr2=0x7f6cac197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6caaffd640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cac100170 0x7f6cac197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6caaffd640 1 -- 172.21.15.18:0/982601425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac1057c0 msgr2=0x7f6cac19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6caaffd640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac1057c0 0x7f6cac19c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6caaffd640 1 -- 172.21.15.18:0/982601425 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c94009560 con 0x7f6cac0ff7b0 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6cab7fe640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac1057c0 0x7f6cac19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6caaffd640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cac0ff7b0 0x7f6cac197740 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400c920 tx=0x7f6ca400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6cb092a640 1 -- 172.21.15.18:0/982601425 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4007c70 con 0x7f6cac0ff7b0 2024-02-17T12:51:21.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6cb092a640 1 -- 172.21.15.18:0/982601425 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ca4007dd0 con 0x7f6cac0ff7b0 2024-02-17T12:51:21.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6cb192c640 1 -- 172.21.15.18:0/982601425 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac19cc10 con 0x7f6cac0ff7b0 2024-02-17T12:51:21.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6cb092a640 1 -- 172.21.15.18:0/982601425 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4018860 con 0x7f6cac0ff7b0 2024-02-17T12:51:21.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.310+0000 7f6cb192c640 1 -- 172.21.15.18:0/982601425 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac1a4490 con 0x7f6cac0ff7b0 2024-02-17T12:51:21.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.312+0000 7f6cb092a640 1 -- 172.21.15.18:0/982601425 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f6ca400fac0 con 0x7f6cac0ff7b0 2024-02-17T12:51:21.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.313+0000 7f6cb45dc740 1 Processor -- start 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.313+0000 7f6cb45dc740 1 -- start start 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.313+0000 7f6cb092a640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6c980a2350 0x7f6c980a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.313+0000 7f6cb45dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125ac250 0x55c2125ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.313+0000 7f6cb45dc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125acb70 0x55c2125b5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.313+0000 7f6cb45dc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125b5750 0x55c2125b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.313+0000 7f6cb45dc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c212567960 con 0x55c2125b5750 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.313+0000 7f6cb45dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21251b310 con 0x55c2125ac250 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.313+0000 7f6cb45dc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c212518840 con 0x55c2125acb70 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.313+0000 7f6cb092a640 1 -- 172.21.15.18:0/982601425 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6ca40ce470 con 0x7f6cac0ff7b0 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.313+0000 7f6cab7fe640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6c980a2350 0x7f6c980a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6cab7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125b5750 0x55c2125b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6cab7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125b5750 0x55c2125b7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43628/0 (socket says 172.21.15.18:43628) 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6cab7fe640 1 -- 172.21.15.18:0/123157885 learned_addr learned my addr 172.21.15.18:0/123157885 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6caa7fc640 1 --2- 172.21.15.18:0/123157885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125acb70 0x55c2125b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6caaffd640 1 --2- 172.21.15.18:0/123157885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125ac250 0x55c2125ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6cab7fe640 1 -- 172.21.15.18:0/123157885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125acb70 msgr2=0x55c2125b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6cab7fe640 1 --2- 172.21.15.18:0/123157885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125acb70 0x55c2125b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6cab7fe640 1 -- 172.21.15.18:0/123157885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125ac250 msgr2=0x55c2125ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6cab7fe640 1 --2- 172.21.15.18:0/123157885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125ac250 0x55c2125ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6cab7fe640 1 -- 172.21.15.18:0/123157885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2124d6060 con 0x55c2125b5750 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6caa7fc640 1 --2- 172.21.15.18:0/123157885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125acb70 0x55c2125b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6cab7fe640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6c980a2350 0x7f6c980a4810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00afd0 tx=0x7f6c9c00adc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6cab7fe640 1 --2- 172.21.15.18:0/123157885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125b5750 0x55c2125b7b40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00b7f0 tx=0x7f6c9c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7278f5c2ffa693 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6ca1ffb640 1 -- 172.21.15.18:0/123157885 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c004b20 con 0x55c2125b5750 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6ca1ffb640 1 -- 172.21.15.18:0/123157885 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c9c01dd60 con 0x55c2125b5750 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.314+0000 7f6ca1ffb640 1 -- 172.21.15.18:0/123157885 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c005550 con 0x55c2125b5750 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.315+0000 7f6cb45dc740 1 -- 172.21.15.18:0/123157885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125b5750 msgr2=0x55c2125b7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.315+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/123157885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125b5750 0x55c2125b7b40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00b7f0 tx=0x7f6c9c01a000 comp rx=0 tx=0).stop 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.315+0000 7f6cb45dc740 1 -- 172.21.15.18:0/123157885 shutdown_connections 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.315+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/123157885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125b5750 0x55c2125b7b40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.315+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/123157885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125acb70 0x55c2125b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.315+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/123157885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125ac250 0x55c2125ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.315+0000 7f6cb45dc740 1 -- 172.21.15.18:0/123157885 >> 172.21.15.18:0/123157885 conn(0x55c2125a6980 msgr2=0x55c2125a6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.315+0000 7f6cb45dc740 1 -- 172.21.15.18:0/123157885 shutdown_connections 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.315+0000 7f6cb45dc740 1 -- 172.21.15.18:0/123157885 wait complete. 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.315+0000 7f6cb45dc740 1 -- start start 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.315+0000 7f6cb45dc740 1 Processor -- start 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6cb45dc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6cb45dc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125acb70 0x55c2125b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6cb45dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125b5750 0x55c2125b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6cb45dc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c212514020 con 0x55c2125ac250 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6cb45dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2121e7430 con 0x55c2125b5750 2024-02-17T12:51:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6cb45dc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2121a3e00 con 0x55c2125acb70 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6cb45dc740 10 client.0 _ll_register_callbacks cb 0x55c2124bbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6caaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6caaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43640/0 (socket says 172.21.15.18:43640) 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6caaffd640 1 -- 172.21.15.18:0/3347226770 learned_addr learned my addr 172.21.15.18:0/3347226770 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6caa7fc640 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125acb70 0x55c2125b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6cab7fe640 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125b5750 0x55c2125b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6caaffd640 1 -- 172.21.15.18:0/3347226770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125acb70 msgr2=0x55c2125b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6caaffd640 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125acb70 0x55c2125b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6caaffd640 1 -- 172.21.15.18:0/3347226770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125b5750 msgr2=0x55c2125b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6caaffd640 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125b5750 0x55c2125b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6caaffd640 1 -- 172.21.15.18:0/3347226770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9c016040 con 0x55c2125ac250 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6caaffd640 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125b2720 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4097070 tx=0x7f6ca40780a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c874830710bdd321 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6c8affd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6c8affd640 1 -- 172.21.15.18:0/3347226770 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4077e70 con 0x55c2125ac250 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6c8affd640 1 -- 172.21.15.18:0/3347226770 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6ca409ac60 con 0x55c2125ac250 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.316+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3347226770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2122137e0 con 0x55c2125ac250 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.317+0000 7f6c8affd640 1 -- 172.21.15.18:0/3347226770 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ca409edb0 con 0x55c2125ac250 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.317+0000 7f6cb45dc740 10 client.16092 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.317+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3347226770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c2125126a0 con 0x55c2125ac250 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.317+0000 7f6c8affd640 1 -- 172.21.15.18:0/3347226770 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c2122137e0 con 0x55c2125ac250 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.317+0000 7f6c8affd640 1 -- 172.21.15.18:0/3347226770 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6ca4016020 con 0x55c2125ac250 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.317+0000 7f6cb45dc740 10 client.16092 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.317+0000 7f6cb45dc740 10 client.16092 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.317+0000 7f6cb45dc740 20 client.16092 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.317+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55c2125c5a80 0x55c2125c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.317+0000 7f6cb45dc740 4 client.16092 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.317+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3347226770 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c212250720 con 0x55c2125c5a80 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.317+0000 7f6caa7fc640 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55c2125c5a80 0x55c2125c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6c8affd640 10 client.16092 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6caa7fc640 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55c2125c5a80 0x55c2125c7e70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a572c8a98fe0bcb server_cookie=add3597ae5e48e0c in_seq=0 out_seq=0 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6c8affd640 1 -- 172.21.15.18:0/3347226770 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c212250720 con 0x55c2125c5a80 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6c8affd640 10 client.16092 handle_command_reply: tid=0 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6cb45dc740 1 client.16092 shutdown 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3347226770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 msgr2=0x55c2125b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125b2720 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4097070 tx=0x7f6ca40780a0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3347226770 shutdown_connections 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55c2125c5a80 0x55c2125c7e70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125b5750 0x55c2125b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125acb70 0x55c2125b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6c8affd640 0 client.16092 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3347226770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125b2720 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.318+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3347226770 >> 172.21.15.18:0/3347226770 conn(0x55c2125a6980 msgr2=0x55c2125a86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.319+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3347226770 shutdown_connections 2024-02-17T12:51:21.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.319+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3347226770 wait complete. 2024-02-17T12:51:21.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.319+0000 7f6cb45dc740 20 client.16092 trim_cache size 0 max 16384 2024-02-17T12:51:21.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.330+0000 7f6cb45dc740 1 Processor -- start 2024-02-17T12:51:21.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.330+0000 7f6cb45dc740 1 -- start start 2024-02-17T12:51:21.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.330+0000 7f6cb45dc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.330+0000 7f6cb45dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125acb70 0x55c2125b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.330+0000 7f6cb45dc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125b5750 0x55c2125ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.330+0000 7f6cb45dc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c212499130 con 0x55c2125ac250 2024-02-17T12:51:21.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.330+0000 7f6cb45dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2125c2620 con 0x55c2125acb70 2024-02-17T12:51:21.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.330+0000 7f6cb45dc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2125b7b40 con 0x55c2125b5750 2024-02-17T12:51:21.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6caaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6caaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43648/0 (socket says 172.21.15.18:43648) 2024-02-17T12:51:21.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6caaffd640 1 -- 172.21.15.18:0/3637905669 learned_addr learned my addr 172.21.15.18:0/3637905669 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6caaffd640 1 -- 172.21.15.18:0/3637905669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125b5750 msgr2=0x55c2125ad9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:51:21.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6caaffd640 1 --2- 172.21.15.18:0/3637905669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125b5750 0x55c2125ad9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6caa7fc640 1 --2- 172.21.15.18:0/3637905669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125acb70 0x55c2125b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6caaffd640 1 -- 172.21.15.18:0/3637905669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125acb70 msgr2=0x55c2125b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6caaffd640 1 --2- 172.21.15.18:0/3637905669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125acb70 0x55c2125b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6caaffd640 1 -- 172.21.15.18:0/3637905669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2124aec20 con 0x55c2125ac250 2024-02-17T12:51:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6caa7fc640 1 --2- 172.21.15.18:0/3637905669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125acb70 0x55c2125b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6caaffd640 1 --2- 172.21.15.18:0/3637905669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125b4e50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4097630 tx=0x7f6ca40cbd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cad21fc427808a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6ca17fa640 1 -- 172.21.15.18:0/3637905669 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4096f00 con 0x55c2125ac250 2024-02-17T12:51:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6ca17fa640 1 -- 172.21.15.18:0/3637905669 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ca40cb9d0 con 0x55c2125ac250 2024-02-17T12:51:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.331+0000 7f6ca17fa640 1 -- 172.21.15.18:0/3637905669 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40cc060 con 0x55c2125ac250 2024-02-17T12:51:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3637905669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 msgr2=0x55c2125b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3637905669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125b4e50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4097630 tx=0x7f6ca40cbd70 comp rx=0 tx=0).stop 2024-02-17T12:51:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3637905669 shutdown_connections 2024-02-17T12:51:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3637905669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125b5750 0x55c2125ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3637905669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125acb70 0x55c2125b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3637905669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125b4e50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3637905669 >> 172.21.15.18:0/3637905669 conn(0x55c2125a6980 msgr2=0x55c2125a1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3637905669 shutdown_connections 2024-02-17T12:51:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3637905669 wait complete. 2024-02-17T12:51:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 -- start start 2024-02-17T12:51:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 Processor -- start 2024-02-17T12:51:21.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125acb70 0x55c2125bac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125b5750 0x55c2125bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2125bb8c0 con 0x55c2125ac250 2024-02-17T12:51:21.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2125bba30 con 0x55c2125acb70 2024-02-17T12:51:21.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21261bf20 con 0x55c2125b5750 2024-02-17T12:51:21.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.332+0000 7f6cb45dc740 10 client.0 _ll_register_callbacks cb 0x7f6c9c00e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:21.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6caaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6caaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125bc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43664/0 (socket says 172.21.15.18:43664) 2024-02-17T12:51:21.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6caaffd640 1 -- 172.21.15.18:0/3449397689 learned_addr learned my addr 172.21.15.18:0/3449397689 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6caa7fc640 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125acb70 0x55c2125bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6cab7fe640 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125b5750 0x55c2125bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6caaffd640 1 -- 172.21.15.18:0/3449397689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125b5750 msgr2=0x55c2125bb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6caaffd640 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125b5750 0x55c2125bb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6caaffd640 1 -- 172.21.15.18:0/3449397689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125acb70 msgr2=0x55c2125bac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6caaffd640 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125acb70 0x55c2125bac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6caaffd640 1 -- 172.21.15.18:0/3449397689 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2125126a0 con 0x55c2125ac250 2024-02-17T12:51:21.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6caaffd640 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125bc8e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400ff00 tx=0x7f6ca400fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5754f5e6b865a9f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6c8affd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:21.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6c8affd640 1 -- 172.21.15.18:0/3449397689 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40b91f0 con 0x55c2125ac250 2024-02-17T12:51:21.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6c8affd640 1 -- 172.21.15.18:0/3449397689 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6ca409edb0 con 0x55c2125ac250 2024-02-17T12:51:21.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3449397689 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c9c01dd60 con 0x55c2125ac250 2024-02-17T12:51:21.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6c8affd640 1 -- 172.21.15.18:0/3449397689 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ca4096f00 con 0x55c2125ac250 2024-02-17T12:51:21.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6cb45dc740 10 client.16098 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:21.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3449397689 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c2124d48c0 con 0x55c2125ac250 2024-02-17T12:51:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.333+0000 7f6c8affd640 1 -- 172.21.15.18:0/3449397689 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c9c01dd60 con 0x55c2125ac250 2024-02-17T12:51:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6c8affd640 1 -- 172.21.15.18:0/3449397689 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6ca4016050 con 0x55c2125ac250 2024-02-17T12:51:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6cb45dc740 10 client.16098 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6cb45dc740 10 client.16098 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:51:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6cb45dc740 20 client.16098 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55c21261d060 0x55c21261f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6cb45dc740 4 client.16098 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:51:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3449397689 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c21229f890 con 0x55c21261d060 2024-02-17T12:51:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6caa7fc640 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55c21261d060 0x55c21261f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6c8affd640 10 client.16098 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:51:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6caa7fc640 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55c21261d060 0x55c21261f450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4ff5dccbed46f3b server_cookie=b56922a63d21fee7 in_seq=0 out_seq=0 2024-02-17T12:51:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6c8affd640 1 -- 172.21.15.18:0/3449397689 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c21229f890 con 0x55c21261d060 2024-02-17T12:51:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6c8affd640 10 client.16098 handle_command_reply: tid=0 2024-02-17T12:51:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6cb45dc740 1 client.16098 shutdown 2024-02-17T12:51:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3449397689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 msgr2=0x55c2125bc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.334+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125bc8e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400ff00 tx=0x7f6ca400fdc0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3449397689 shutdown_connections 2024-02-17T12:51:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55c21261d060 0x55c21261f450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2125b5750 0x55c2125bb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6c8affd640 0 client.16098 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:51:21.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2125acb70 0x55c2125bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb45dc740 1 --2- 172.21.15.18:0/3449397689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2125ac250 0x55c2125bc8e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3449397689 >> 172.21.15.18:0/3449397689 conn(0x55c2125a6980 msgr2=0x55c2125ad9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3449397689 shutdown_connections 2024-02-17T12:51:21.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb45dc740 1 -- 172.21.15.18:0/3449397689 wait complete. 2024-02-17T12:51:21.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb45dc740 20 client.16098 trim_cache size 0 max 16384 2024-02-17T12:51:21.337 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:51:21.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb192c640 1 -- 172.21.15.18:0/982601425 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6c980a2350 msgr2=0x7f6c980a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb192c640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6c980a2350 0x7f6c980a4810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00afd0 tx=0x7f6c9c00adc0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb192c640 1 -- 172.21.15.18:0/982601425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cac0ff7b0 msgr2=0x7f6cac197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb192c640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cac0ff7b0 0x7f6cac197740 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400c920 tx=0x7f6ca400cde0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb192c640 1 -- 172.21.15.18:0/982601425 shutdown_connections 2024-02-17T12:51:21.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb192c640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6c980a2350 0x7f6c980a4810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb192c640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac1057c0 0x7f6cac19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb192c640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cac100170 0x7f6cac197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb192c640 1 --2- 172.21.15.18:0/982601425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cac0ff7b0 0x7f6cac197740 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb192c640 1 -- 172.21.15.18:0/982601425 >> 172.21.15.18:0/982601425 conn(0x7f6cac076640 msgr2=0x7f6cac0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb192c640 1 -- 172.21.15.18:0/982601425 shutdown_connections 2024-02-17T12:51:21.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.335+0000 7f6cb192c640 1 -- 172.21.15.18:0/982601425 wait complete. 2024-02-17T12:51:21.341 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:51:21.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:51:21.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.455+0000 7fcbd5864640 1 -- 172.21.15.18:0/2555450480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbd0073e50 msgr2=0x7fcbd010d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.455+0000 7fcbd5864640 1 --2- 172.21.15.18:0/2555450480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbd0073e50 0x7fcbd010d490 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009900 tx=0x7fcbb802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.455+0000 7fcbd5864640 1 -- 172.21.15.18:0/2555450480 shutdown_connections 2024-02-17T12:51:21.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.455+0000 7fcbd5864640 1 --2- 172.21.15.18:0/2555450480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbd010d9d0 0x7fcbd010fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.455+0000 7fcbd5864640 1 --2- 172.21.15.18:0/2555450480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbd0073e50 0x7fcbd010d490 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.455+0000 7fcbd5864640 1 --2- 172.21.15.18:0/2555450480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd0073530 0x7fcbd0073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.455+0000 7fcbd5864640 1 -- 172.21.15.18:0/2555450480 >> 172.21.15.18:0/2555450480 conn(0x7fcbd0078580 msgr2=0x7fcbd0078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.455+0000 7fcbd5864640 1 -- 172.21.15.18:0/2555450480 shutdown_connections 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.455+0000 7fcbd5864640 1 -- 172.21.15.18:0/2555450480 wait complete. 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbd5864640 1 Processor -- start 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbd5864640 1 -- start start 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbd5864640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbd0073530 0x7fcbd019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbd5864640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbd0073e50 0x7fcbd019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbd5864640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd010d9d0 0x7fcbd0199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbd5864640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd0199ea0 con 0x7fcbd0073530 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbd5864640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd019a010 con 0x7fcbd010d9d0 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbd5864640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd019a1d0 con 0x7fcbd0073e50 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbceffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbd0073530 0x7fcbd019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbceffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbd0073530 0x7fcbd019f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43694/0 (socket says 172.21.15.18:43694) 2024-02-17T12:51:21.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbceffd640 1 -- 172.21.15.18:0/4256728722 learned_addr learned my addr 172.21.15.18:0/4256728722 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbcf7fe640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd010d9d0 0x7fcbd0199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbceffd640 1 -- 172.21.15.18:0/4256728722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbd0073e50 msgr2=0x7fcbd019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbceffd640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbd0073e50 0x7fcbd019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbceffd640 1 -- 172.21.15.18:0/4256728722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd010d9d0 msgr2=0x7fcbd0199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbceffd640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd010d9d0 0x7fcbd0199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbceffd640 1 -- 172.21.15.18:0/4256728722 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb8009560 con 0x7fcbd0073530 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.456+0000 7fcbcf7fe640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd010d9d0 0x7fcbd0199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.457+0000 7fcbceffd640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbd0073530 0x7fcbd019f640 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000d950 tx=0x7fcbc000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.457+0000 7fcbd4862640 1 -- 172.21.15.18:0/4256728722 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0009c30 con 0x7fcbd0073530 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.457+0000 7fcbd4862640 1 -- 172.21.15.18:0/4256728722 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbc0009d90 con 0x7fcbd0073530 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.457+0000 7fcbd5864640 1 -- 172.21.15.18:0/4256728722 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd019a4b0 con 0x7fcbd0073530 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.457+0000 7fcbd4862640 1 -- 172.21.15.18:0/4256728722 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000f7e0 con 0x7fcbd0073530 2024-02-17T12:51:21.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.457+0000 7fcbd5864640 1 -- 172.21.15.18:0/4256728722 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd01a6670 con 0x7fcbd0073530 2024-02-17T12:51:21.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.459+0000 7fcbd4862640 1 -- 172.21.15.18:0/4256728722 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fcbc0010ad0 con 0x7fcbd0073530 2024-02-17T12:51:21.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.459+0000 7fcbd8435740 1 Processor -- start 2024-02-17T12:51:21.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.459+0000 7fcbd8435740 1 -- start start 2024-02-17T12:51:21.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbd4862640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcbbc0a2350 0x7fcbbc0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbd8435740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787250 0x55c169787630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbcf7fe640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcbbc0a2350 0x7fcbbc0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbd8435740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787b70 0x55c169790210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbd8435740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169790750 0x55c169792b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbd8435740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c169742960 con 0x55c169787b70 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbd8435740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1696f6310 con 0x55c169790750 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbd8435740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1696f3840 con 0x55c169787250 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbd4862640 1 -- 172.21.15.18:0/4256728722 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcbc00ce0c0 con 0x7fcbd0073530 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbc7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787b70 0x55c169790210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbc7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787b70 0x55c169790210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43700/0 (socket says 172.21.15.18:43700) 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbc7fff640 1 -- 172.21.15.18:0/3177062754 learned_addr learned my addr 172.21.15.18:0/3177062754 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbceffd640 1 --2- 172.21.15.18:0/3177062754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787250 0x55c169787630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbcf7fe640 1 --2- 172.21.15.18:0/3177062754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169790750 0x55c169792b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbc7fff640 1 -- 172.21.15.18:0/3177062754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787250 msgr2=0x55c169787630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbc7fff640 1 --2- 172.21.15.18:0/3177062754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787250 0x55c169787630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbc7fff640 1 -- 172.21.15.18:0/3177062754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169790750 msgr2=0x55c169792b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbc7fff640 1 --2- 172.21.15.18:0/3177062754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169790750 0x55c169792b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbc7fff640 1 -- 172.21.15.18:0/3177062754 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1696b1060 con 0x55c169787b70 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbcf7fe640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcbbc0a2350 0x7fcbbc0a4810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80092f0 tx=0x7fcbc8010740 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.460+0000 7fcbc7fff640 1 --2- 172.21.15.18:0/3177062754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787b70 0x55c169790210 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcbb802c210 tx=0x7fcbb802c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64986b991354a6a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.461+0000 7fcbc5ffb640 1 -- 172.21.15.18:0/3177062754 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80047c0 con 0x55c169787b70 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.461+0000 7fcbc5ffb640 1 -- 172.21.15.18:0/3177062754 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbb8004920 con 0x55c169787b70 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.461+0000 7fcbc5ffb640 1 -- 172.21.15.18:0/3177062754 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8005030 con 0x55c169787b70 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.461+0000 7fcbd8435740 1 -- 172.21.15.18:0/3177062754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787b70 msgr2=0x55c169790210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.461+0000 7fcbd8435740 1 --2- 172.21.15.18:0/3177062754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787b70 0x55c169790210 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcbb802c210 tx=0x7fcbb802c6f0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.461+0000 7fcbd8435740 1 -- 172.21.15.18:0/3177062754 shutdown_connections 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.461+0000 7fcbd8435740 1 --2- 172.21.15.18:0/3177062754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169790750 0x55c169792b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.461+0000 7fcbd8435740 1 --2- 172.21.15.18:0/3177062754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787b70 0x55c169790210 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.461+0000 7fcbd8435740 1 --2- 172.21.15.18:0/3177062754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787250 0x55c169787630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.461+0000 7fcbd8435740 1 -- 172.21.15.18:0/3177062754 >> 172.21.15.18:0/3177062754 conn(0x55c169781980 msgr2=0x55c169781d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.461+0000 7fcbd8435740 1 -- 172.21.15.18:0/3177062754 shutdown_connections 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.461+0000 7fcbd8435740 1 -- 172.21.15.18:0/3177062754 wait complete. 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbd8435740 1 -- start start 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbd8435740 1 Processor -- start 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbd8435740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787250 0x55c16978d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbd8435740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169787b70 0x55c16978dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbd8435740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169790750 0x55c16978e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbd8435740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1696ef020 con 0x55c169790750 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbd8435740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1693c2430 con 0x55c169787b70 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbd8435740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16937ee00 con 0x55c169787250 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbd8435740 10 client.0 _ll_register_callbacks cb 0x55c169696cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbcf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169790750 0x55c16978e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbcf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169790750 0x55c16978e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43712/0 (socket says 172.21.15.18:43712) 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbcf7fe640 1 -- 172.21.15.18:0/4156189317 learned_addr learned my addr 172.21.15.18:0/4156189317 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbceffd640 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787250 0x55c16978d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbc7fff640 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169787b70 0x55c16978dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbcf7fe640 1 -- 172.21.15.18:0/4156189317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787250 msgr2=0x55c16978d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbcf7fe640 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787250 0x55c16978d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbcf7fe640 1 -- 172.21.15.18:0/4156189317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169787b70 msgr2=0x55c16978dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbcf7fe640 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169787b70 0x55c16978dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.462+0000 7fcbcf7fe640 1 -- 172.21.15.18:0/4156189317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd0110a70 con 0x55c169790750 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbc7fff640 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169787b70 0x55c16978dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbcf7fe640 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169790750 0x55c16978e160 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8013930 tx=0x7fcbc8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a674b8b1d9d444 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbaeffd640 1 -- 172.21.15.18:0/4156189317 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8013230 con 0x55c169790750 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbaeffd640 1 -- 172.21.15.18:0/4156189317 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcbc8019040 con 0x55c169790750 2024-02-17T12:51:21.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbd8435740 1 -- 172.21.15.18:0/4156189317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1693ee7e0 con 0x55c169790750 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbaeffd640 1 -- 172.21.15.18:0/4156189317 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbc801eb50 con 0x55c169790750 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbd8435740 10 client.16110 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbd8435740 1 -- 172.21.15.18:0/4156189317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c1696ed6a0 con 0x55c169790750 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbaeffd640 1 -- 172.21.15.18:0/4156189317 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c1693ee7e0 con 0x55c169790750 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbaeffd640 1 -- 172.21.15.18:0/4156189317 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcbc8012440 con 0x55c169790750 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbd8435740 10 client.16110 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbd8435740 10 client.16110 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbd8435740 20 client.16110 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbd8435740 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c1697a0c50 0x55c1697a3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbd8435740 4 client.16110 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.463+0000 7fcbd8435740 1 -- 172.21.15.18:0/4156189317 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c16942b720 con 0x55c1697a0c50 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.464+0000 7fcbceffd640 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c1697a0c50 0x55c1697a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.464+0000 7fcbaeffd640 10 client.16110 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.464+0000 7fcbceffd640 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c1697a0c50 0x55c1697a3070 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c751f22b0c9f5e5f server_cookie=25771565445f87ec in_seq=0 out_seq=0 2024-02-17T12:51:21.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbaeffd640 1 -- 172.21.15.18:0/4156189317 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c16942b720 con 0x55c1697a0c50 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbaeffd640 10 client.16110 handle_command_reply: tid=0 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbd8435740 1 client.16110 shutdown 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbd8435740 1 -- 172.21.15.18:0/4156189317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169790750 msgr2=0x55c16978e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbd8435740 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169790750 0x55c16978e160 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8013930 tx=0x7fcbc8004ab0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbd8435740 1 -- 172.21.15.18:0/4156189317 shutdown_connections 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbd8435740 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c1697a0c50 0x55c1697a3070 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbd8435740 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169790750 0x55c16978e160 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbd8435740 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169787b70 0x55c16978dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbaeffd640 0 client.16110 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbd8435740 1 --2- 172.21.15.18:0/4156189317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787250 0x55c16978d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbd8435740 1 -- 172.21.15.18:0/4156189317 >> 172.21.15.18:0/4156189317 conn(0x55c169781980 msgr2=0x55c1697836d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbd8435740 1 -- 172.21.15.18:0/4156189317 shutdown_connections 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbd8435740 1 -- 172.21.15.18:0/4156189317 wait complete. 2024-02-17T12:51:21.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.465+0000 7fcbd8435740 20 client.16110 trim_cache size 0 max 16384 2024-02-17T12:51:21.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbd8435740 1 Processor -- start 2024-02-17T12:51:21.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbd8435740 1 -- start start 2024-02-17T12:51:21.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbd8435740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 0x55c16978df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbd8435740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787b70 0x55c169780a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbd8435740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169790750 0x55c169797600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbd8435740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c169674130 con 0x55c169787250 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbd8435740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c169782980 con 0x55c169790750 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbd8435740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c169786b80 con 0x55c169787b70 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbceffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 0x55c16978df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbceffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 0x55c16978df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43722/0 (socket says 172.21.15.18:43722) 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbceffd640 1 -- 172.21.15.18:0/1998228193 learned_addr learned my addr 172.21.15.18:0/1998228193 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbc7fff640 1 --2- 172.21.15.18:0/1998228193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787b70 0x55c169780a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbceffd640 1 -- 172.21.15.18:0/1998228193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787b70 msgr2=0x55c169780a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbceffd640 1 --2- 172.21.15.18:0/1998228193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787b70 0x55c169780a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbceffd640 1 -- 172.21.15.18:0/1998228193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169790750 msgr2=0x55c169797600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbceffd640 1 --2- 172.21.15.18:0/1998228193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169790750 0x55c169797600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbceffd640 1 -- 172.21.15.18:0/1998228193 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c169689c20 con 0x55c169787250 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbc7fff640 1 --2- 172.21.15.18:0/1998228193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787b70 0x55c169780a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.477+0000 7fcbceffd640 1 --2- 172.21.15.18:0/1998228193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 0x55c16978df80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009610 tx=0x7fcbc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5906aefe277c32fc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbc57fa640 1 -- 172.21.15.18:0/1998228193 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc009e9a0 con 0x55c169787250 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbc57fa640 1 -- 172.21.15.18:0/1998228193 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbc00a2ff0 con 0x55c169787250 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbc57fa640 1 -- 172.21.15.18:0/1998228193 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0085070 con 0x55c169787250 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbd8435740 1 -- 172.21.15.18:0/1998228193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 msgr2=0x55c16978df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbd8435740 1 --2- 172.21.15.18:0/1998228193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 0x55c16978df80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009610 tx=0x7fcbc0005880 comp rx=0 tx=0).stop 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbd8435740 1 -- 172.21.15.18:0/1998228193 shutdown_connections 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbd8435740 1 --2- 172.21.15.18:0/1998228193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169790750 0x55c169797600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbd8435740 1 --2- 172.21.15.18:0/1998228193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169787b70 0x55c169780a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbd8435740 1 --2- 172.21.15.18:0/1998228193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 0x55c16978df80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbd8435740 1 -- 172.21.15.18:0/1998228193 >> 172.21.15.18:0/1998228193 conn(0x55c169781980 msgr2=0x55c1697802f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbd8435740 1 -- 172.21.15.18:0/1998228193 shutdown_connections 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbd8435740 1 -- 172.21.15.18:0/1998228193 wait complete. 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.478+0000 7fcbd8435740 1 -- start start 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbd8435740 1 Processor -- start 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbd8435740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 0x55c1697969b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbd8435740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169787b70 0x55c169796ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbd8435740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169790750 0x55c1697f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbd8435740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1697f16e0 con 0x55c169787250 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbd8435740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1697f1820 con 0x55c169787b70 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbd8435740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1697f19e0 con 0x55c169790750 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbd8435740 10 client.0 _ll_register_callbacks cb 0x7fcbb8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbceffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 0x55c1697969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbceffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 0x55c1697969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43724/0 (socket says 172.21.15.18:43724) 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbceffd640 1 -- 172.21.15.18:0/2843586347 learned_addr learned my addr 172.21.15.18:0/2843586347 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbc7fff640 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169787b70 0x55c169796ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbceffd640 1 -- 172.21.15.18:0/2843586347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169790750 msgr2=0x55c1697f11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbceffd640 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169790750 0x55c1697f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbceffd640 1 -- 172.21.15.18:0/2843586347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169787b70 msgr2=0x55c169796ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbceffd640 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169787b70 0x55c169796ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbceffd640 1 -- 172.21.15.18:0/2843586347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd01a6670 con 0x55c169787250 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbc7fff640 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169787b70 0x55c169796ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbceffd640 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 0x55c1697969b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0096cb0 tx=0x7fcbc00cb670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c0602322cfb9e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.479+0000 7fcbaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbaeffd640 1 -- 172.21.15.18:0/2843586347 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0065ce0 con 0x55c169787250 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbaeffd640 1 -- 172.21.15.18:0/2843586347 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcbc00850e0 con 0x55c169787250 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbd8435740 1 -- 172.21.15.18:0/2843586347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbb8004920 con 0x55c169787250 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbaeffd640 1 -- 172.21.15.18:0/2843586347 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbc00b3200 con 0x55c169787250 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbd8435740 10 client.16116 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbd8435740 1 -- 172.21.15.18:0/2843586347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c1696af8c0 con 0x55c169787250 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbaeffd640 1 -- 172.21.15.18:0/2843586347 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbc00b7080 con 0x55c169787250 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbaeffd640 1 -- 172.21.15.18:0/2843586347 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcbc00cb960 con 0x55c169787250 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbd8435740 10 client.16116 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:21.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbd8435740 10 client.16116 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbd8435740 20 client.16116 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbd8435740 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c1697f8e10 0x55c1697fb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbd8435740 4 client.16116 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbd8435740 1 -- 172.21.15.18:0/2843586347 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c16947a890 con 0x55c1697f8e10 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.480+0000 7fcbc7fff640 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c1697f8e10 0x55c1697fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbc7fff640 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c1697f8e10 0x55c1697fb200 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ef3590618e68131 server_cookie=2c8abe595f83835c in_seq=0 out_seq=0 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbaeffd640 10 client.16116 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbaeffd640 1 -- 172.21.15.18:0/2843586347 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c16947a890 con 0x55c1697f8e10 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbaeffd640 10 client.16116 handle_command_reply: tid=0 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbd8435740 1 client.16116 shutdown 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbd8435740 1 -- 172.21.15.18:0/2843586347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 msgr2=0x55c1697969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbd8435740 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 0x55c1697969b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0096cb0 tx=0x7fcbc00cb670 comp rx=0 tx=0).stop 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbd8435740 1 -- 172.21.15.18:0/2843586347 shutdown_connections 2024-02-17T12:51:21.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbd8435740 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55c1697f8e10 0x55c1697fb200 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbd8435740 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c169790750 0x55c1697f11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbd8435740 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169787b70 0x55c169796ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbaeffd640 0 client.16116 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:51:21.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbd8435740 1 --2- 172.21.15.18:0/2843586347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c169787250 0x55c1697969b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbd8435740 1 -- 172.21.15.18:0/2843586347 >> 172.21.15.18:0/2843586347 conn(0x55c169781980 msgr2=0x55c169780b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbd8435740 1 -- 172.21.15.18:0/2843586347 shutdown_connections 2024-02-17T12:51:21.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbd8435740 1 -- 172.21.15.18:0/2843586347 wait complete. 2024-02-17T12:51:21.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.481+0000 7fcbd8435740 20 client.16116 trim_cache size 0 max 16384 2024-02-17T12:51:21.480 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:51:21.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.482+0000 7fcbd5864640 1 -- 172.21.15.18:0/4256728722 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcbbc0a2350 msgr2=0x7fcbbc0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.482+0000 7fcbd5864640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcbbc0a2350 0x7fcbbc0a4810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80092f0 tx=0x7fcbc8010740 comp rx=0 tx=0).stop 2024-02-17T12:51:21.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.482+0000 7fcbd5864640 1 -- 172.21.15.18:0/4256728722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbd0073530 msgr2=0x7fcbd019f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.482+0000 7fcbd5864640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbd0073530 0x7fcbd019f640 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000d950 tx=0x7fcbc000de10 comp rx=0 tx=0).stop 2024-02-17T12:51:21.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.482+0000 7fcbd5864640 1 -- 172.21.15.18:0/4256728722 shutdown_connections 2024-02-17T12:51:21.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.482+0000 7fcbd5864640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcbbc0a2350 0x7fcbbc0a4810 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.482+0000 7fcbd5864640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbd010d9d0 0x7fcbd0199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.482+0000 7fcbd5864640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbd0073e50 0x7fcbd019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.482+0000 7fcbd5864640 1 --2- 172.21.15.18:0/4256728722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbd0073530 0x7fcbd019f640 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.482+0000 7fcbd5864640 1 -- 172.21.15.18:0/4256728722 >> 172.21.15.18:0/4256728722 conn(0x7fcbd0078580 msgr2=0x7fcbd00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.482+0000 7fcbd5864640 1 -- 172.21.15.18:0/4256728722 shutdown_connections 2024-02-17T12:51:21.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.482+0000 7fcbd5864640 1 -- 172.21.15.18:0/4256728722 wait complete. 2024-02-17T12:51:21.484 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:51:21.484 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:51:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.597+0000 7f65b3c2e640 1 -- 172.21.15.18:0/171191447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65ac106450 msgr2=0x7f65ac10e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.597+0000 7f65b3c2e640 1 --2- 172.21.15.18:0/171191447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65ac106450 0x7f65ac10e020 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009cd0 tx=0x7f65a402c100 comp rx=0 tx=0).stop 2024-02-17T12:51:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.597+0000 7f65b3c2e640 1 -- 172.21.15.18:0/171191447 shutdown_connections 2024-02-17T12:51:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.597+0000 7f65b3c2e640 1 --2- 172.21.15.18:0/171191447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65ac106450 0x7f65ac10e020 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.597+0000 7f65b3c2e640 1 --2- 172.21.15.18:0/171191447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ac101d30 0x7f65ac105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.597+0000 7f65b3c2e640 1 --2- 172.21.15.18:0/171191447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac101380 0x7f65ac101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.597+0000 7f65b3c2e640 1 -- 172.21.15.18:0/171191447 >> 172.21.15.18:0/171191447 conn(0x7f65ac078580 msgr2=0x7f65ac078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.597+0000 7f65b3c2e640 1 -- 172.21.15.18:0/171191447 shutdown_connections 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.597+0000 7f65b3c2e640 1 -- 172.21.15.18:0/171191447 wait complete. 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b3c2e640 1 Processor -- start 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b3c2e640 1 -- start start 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b3c2e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65ac101380 0x7f65ac19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b3c2e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac106450 0x7f65ac19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b3c2e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ac197670 0x7f65ac197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b3c2e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac198140 con 0x7f65ac101380 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b3c2e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac1982b0 con 0x7f65ac106450 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b3c2e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac198470 con 0x7f65ac197670 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b19ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65ac101380 0x7f65ac19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b19ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65ac101380 0x7f65ac19d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43752/0 (socket says 172.21.15.18:43752) 2024-02-17T12:51:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b19ca640 1 -- 172.21.15.18:0/1909101037 learned_addr learned my addr 172.21.15.18:0/1909101037 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b11c9640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac106450 0x7f65ac19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b19ca640 1 -- 172.21.15.18:0/1909101037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ac197670 msgr2=0x7f65ac197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b19ca640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ac197670 0x7f65ac197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b19ca640 1 -- 172.21.15.18:0/1909101037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac106450 msgr2=0x7f65ac19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b19ca640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac106450 0x7f65ac19daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b19ca640 1 -- 172.21.15.18:0/1909101037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a4009900 con 0x7f65ac101380 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.598+0000 7f65b11c9640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac106450 0x7f65ac19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.599+0000 7f65b19ca640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65ac101380 0x7f65ac19d5b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f659c00d950 tx=0x7f659c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.599+0000 7f65a2ffd640 1 -- 172.21.15.18:0/1909101037 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c009a30 con 0x7f65ac101380 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.599+0000 7f65b3c2e640 1 -- 172.21.15.18:0/1909101037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ac1a4410 con 0x7f65ac101380 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.599+0000 7f65a2ffd640 1 -- 172.21.15.18:0/1909101037 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f659c009b90 con 0x7f65ac101380 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.599+0000 7f65a2ffd640 1 -- 172.21.15.18:0/1909101037 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c004a00 con 0x7f65ac101380 2024-02-17T12:51:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.599+0000 7f65b3c2e640 1 -- 172.21.15.18:0/1909101037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ac1a4940 con 0x7f65ac101380 2024-02-17T12:51:21.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.601+0000 7f65a2ffd640 1 -- 172.21.15.18:0/1909101037 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f659c009d00 con 0x7f65ac101380 2024-02-17T12:51:21.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b6802740 1 Processor -- start 2024-02-17T12:51:21.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b6802740 1 -- start start 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b6802740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938250 0x563feb938630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b6802740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938b70 0x563feb941210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b6802740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 0x563feb943b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65a2ffd640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f65980a2350 0x7f65980a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b6802740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563feb8f3960 con 0x563feb941750 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b6802740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563feb8a7310 con 0x563feb938250 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b6802740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563feb8a4840 con 0x563feb938b70 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b21cb640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f65980a2350 0x7f65980a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b11c9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 0x563feb943b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b11c9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 0x563feb943b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43768/0 (socket says 172.21.15.18:43768) 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b11c9640 1 -- 172.21.15.18:0/561753564 learned_addr learned my addr 172.21.15.18:0/561753564 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65a2ffd640 1 -- 172.21.15.18:0/1909101037 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f659c0cddf0 con 0x7f65ac101380 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.602+0000 7f65b19ca640 1 --2- 172.21.15.18:0/561753564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938b70 0x563feb941210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.603+0000 7f65b21cb640 1 --2- 172.21.15.18:0/561753564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938250 0x563feb938630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.603+0000 7f65b11c9640 1 -- 172.21.15.18:0/561753564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938b70 msgr2=0x563feb941210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.603+0000 7f65b11c9640 1 --2- 172.21.15.18:0/561753564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938b70 0x563feb941210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.603+0000 7f65b11c9640 1 -- 172.21.15.18:0/561753564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938250 msgr2=0x563feb938630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.603+0000 7f65b11c9640 1 --2- 172.21.15.18:0/561753564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938250 0x563feb938630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.603+0000 7f65b11c9640 1 -- 172.21.15.18:0/561753564 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563feb862060 con 0x563feb941750 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.603+0000 7f65b21cb640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f65980a2350 0x7f65980a4810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f65a4006010 tx=0x7f65a402ee10 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.603+0000 7f65b11c9640 1 --2- 172.21.15.18:0/561753564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 0x563feb943b40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f65a800a950 tx=0x7f65a800ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fc4b15775c069e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.603+0000 7f6597fff640 1 -- 172.21.15.18:0/561753564 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a800cc00 con 0x563feb941750 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.603+0000 7f6597fff640 1 -- 172.21.15.18:0/561753564 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65a800cd60 con 0x563feb941750 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.603+0000 7f6597fff640 1 -- 172.21.15.18:0/561753564 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8018650 con 0x563feb941750 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.604+0000 7f65b6802740 1 -- 172.21.15.18:0/561753564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 msgr2=0x563feb943b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.604+0000 7f65b6802740 1 --2- 172.21.15.18:0/561753564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 0x563feb943b40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f65a800a950 tx=0x7f65a800ae10 comp rx=0 tx=0).stop 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.604+0000 7f65b6802740 1 -- 172.21.15.18:0/561753564 shutdown_connections 2024-02-17T12:51:21.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.604+0000 7f65b6802740 1 --2- 172.21.15.18:0/561753564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 0x563feb943b40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.604+0000 7f65b6802740 1 --2- 172.21.15.18:0/561753564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938b70 0x563feb941210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.604+0000 7f65b6802740 1 --2- 172.21.15.18:0/561753564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938250 0x563feb938630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.604+0000 7f65b6802740 1 -- 172.21.15.18:0/561753564 >> 172.21.15.18:0/561753564 conn(0x563feb932980 msgr2=0x563feb932d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.604+0000 7f65b6802740 1 -- 172.21.15.18:0/561753564 shutdown_connections 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.604+0000 7f65b6802740 1 -- 172.21.15.18:0/561753564 wait complete. 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.604+0000 7f65b6802740 1 -- start start 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.604+0000 7f65b6802740 1 Processor -- start 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b6802740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 0x563feb93e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b6802740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938b70 0x563feb93eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b6802740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb941750 0x563feb93f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b6802740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563feb8a0020 con 0x563feb938250 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b6802740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563feb573430 con 0x563feb938b70 2024-02-17T12:51:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b6802740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563feb52fe00 con 0x563feb941750 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b6802740 10 client.0 _ll_register_callbacks cb 0x563feb847cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b19ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 0x563feb93e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b19ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 0x563feb93e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43782/0 (socket says 172.21.15.18:43782) 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b19ca640 1 -- 172.21.15.18:0/3121608555 learned_addr learned my addr 172.21.15.18:0/3121608555 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b11c9640 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938b70 0x563feb93eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b19ca640 1 -- 172.21.15.18:0/3121608555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb941750 msgr2=0x563feb93f0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b19ca640 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb941750 0x563feb93f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b19ca640 1 -- 172.21.15.18:0/3121608555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938b70 msgr2=0x563feb93eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b19ca640 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938b70 0x563feb93eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b19ca640 1 -- 172.21.15.18:0/3121608555 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a800a580 con 0x563feb938250 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b11c9640 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938b70 0x563feb93eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b19ca640 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 0x563feb93e630 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f659c096a40 tx=0x7f659c09a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6eb6223d35b5f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65957fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65957fa640 1 -- 172.21.15.18:0/3121608555 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0a2d80 con 0x563feb938250 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65957fa640 1 -- 172.21.15.18:0/3121608555 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f659c005680 con 0x563feb938250 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b6802740 1 -- 172.21.15.18:0/3121608555 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563feb59f7e0 con 0x563feb938250 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65957fa640 1 -- 172.21.15.18:0/3121608555 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f659c0a9f80 con 0x563feb938250 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b6802740 10 client.16128 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.605+0000 7f65b6802740 1 -- 172.21.15.18:0/3121608555 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563feb89e6a0 con 0x563feb938250 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.606+0000 7f65957fa640 1 -- 172.21.15.18:0/3121608555 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563feb59f7e0 con 0x563feb938250 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.606+0000 7f65957fa640 1 -- 172.21.15.18:0/3121608555 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f659c0a2a10 con 0x563feb938250 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.606+0000 7f65b6802740 10 client.16128 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.606+0000 7f65b6802740 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.606+0000 7f65b6802740 20 client.16128 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.606+0000 7f65b6802740 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563feb951a70 0x563feb953eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.606+0000 7f65b6802740 4 client.16128 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.606+0000 7f65b6802740 1 -- 172.21.15.18:0/3121608555 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563feb5dc720 con 0x563feb951a70 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.606+0000 7f65b11c9640 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563feb951a70 0x563feb953eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.606+0000 7f65957fa640 10 client.16128 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.606+0000 7f65b11c9640 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563feb951a70 0x563feb953eb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc28a2132eb1a255 server_cookie=30384a27be385aad in_seq=0 out_seq=0 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65957fa640 1 -- 172.21.15.18:0/3121608555 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563feb5dc720 con 0x563feb951a70 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65957fa640 10 client.16128 handle_command_reply: tid=0 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65b6802740 1 client.16128 shutdown 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65b6802740 1 -- 172.21.15.18:0/3121608555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 msgr2=0x563feb93e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65b6802740 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 0x563feb93e630 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f659c096a40 tx=0x7f659c09a000 comp rx=0 tx=0).stop 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65b6802740 1 -- 172.21.15.18:0/3121608555 shutdown_connections 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65b6802740 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563feb951a70 0x563feb953eb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65b6802740 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb941750 0x563feb93f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65b6802740 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938b70 0x563feb93eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65b6802740 1 --2- 172.21.15.18:0/3121608555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 0x563feb93e630 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65957fa640 0 client.16128 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65b6802740 1 -- 172.21.15.18:0/3121608555 >> 172.21.15.18:0/3121608555 conn(0x563feb932980 msgr2=0x563feb9346d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65b6802740 1 -- 172.21.15.18:0/3121608555 shutdown_connections 2024-02-17T12:51:21.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65b6802740 1 -- 172.21.15.18:0/3121608555 wait complete. 2024-02-17T12:51:21.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.607+0000 7f65b6802740 20 client.16128 trim_cache size 0 max 16384 2024-02-17T12:51:21.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.619+0000 7f65b6802740 1 Processor -- start 2024-02-17T12:51:21.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.619+0000 7f65b6802740 1 -- start start 2024-02-17T12:51:21.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.619+0000 7f65b6802740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 0x563feb93d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.619+0000 7f65b6802740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938b70 0x563feb931a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.619+0000 7f65b6802740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb941750 0x563feb93eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.619+0000 7f65b6802740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563feb825130 con 0x563feb938250 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.619+0000 7f65b6802740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563feb5dc720 con 0x563feb941750 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.619+0000 7f65b6802740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563feb59f7e0 con 0x563feb938b70 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.619+0000 7f65b19ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 0x563feb93d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.619+0000 7f65b19ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 0x563feb93d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43796/0 (socket says 172.21.15.18:43796) 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.619+0000 7f65b19ca640 1 -- 172.21.15.18:0/3507844249 learned_addr learned my addr 172.21.15.18:0/3507844249 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b11c9640 1 --2- 172.21.15.18:0/3507844249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938b70 0x563feb931a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b21cb640 1 --2- 172.21.15.18:0/3507844249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb941750 0x563feb93eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b19ca640 1 -- 172.21.15.18:0/3507844249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938b70 msgr2=0x563feb931a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b19ca640 1 --2- 172.21.15.18:0/3507844249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938b70 0x563feb931a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b19ca640 1 -- 172.21.15.18:0/3507844249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb941750 msgr2=0x563feb93eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b19ca640 1 --2- 172.21.15.18:0/3507844249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb941750 0x563feb93eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b19ca640 1 -- 172.21.15.18:0/3507844249 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563feb83ac20 con 0x563feb938250 2024-02-17T12:51:21.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b11c9640 1 --2- 172.21.15.18:0/3507844249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938b70 0x563feb931a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b21cb640 1 --2- 172.21.15.18:0/3507844249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb941750 0x563feb93eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b19ca640 1 --2- 172.21.15.18:0/3507844249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 0x563feb93d790 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f659c0cb520 tx=0x7f659c09a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad59b5a503e4afb3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65977fe640 1 -- 172.21.15.18:0/3507844249 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0cb6f0 con 0x563feb938250 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65977fe640 1 -- 172.21.15.18:0/3507844249 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f659c09a120 con 0x563feb938250 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65977fe640 1 -- 172.21.15.18:0/3507844249 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0a2b50 con 0x563feb938250 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b6802740 1 -- 172.21.15.18:0/3507844249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 msgr2=0x563feb93d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b6802740 1 --2- 172.21.15.18:0/3507844249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 0x563feb93d790 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f659c0cb520 tx=0x7f659c09a070 comp rx=0 tx=0).stop 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b6802740 1 -- 172.21.15.18:0/3507844249 shutdown_connections 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b6802740 1 --2- 172.21.15.18:0/3507844249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb941750 0x563feb93eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b6802740 1 --2- 172.21.15.18:0/3507844249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938b70 0x563feb931a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b6802740 1 --2- 172.21.15.18:0/3507844249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb938250 0x563feb93d790 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.620+0000 7f65b6802740 1 -- 172.21.15.18:0/3507844249 >> 172.21.15.18:0/3507844249 conn(0x563feb932980 msgr2=0x563feb9312f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b6802740 1 -- 172.21.15.18:0/3507844249 shutdown_connections 2024-02-17T12:51:21.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b6802740 1 -- 172.21.15.18:0/3507844249 wait complete. 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b6802740 1 -- start start 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b6802740 1 Processor -- start 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b6802740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938250 0x563feb946fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b6802740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938b70 0x563feb947520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b6802740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 0x563feb947a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b6802740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563feb948f80 con 0x563feb941750 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b6802740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563feb9a8010 con 0x563feb938b70 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b6802740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563feb9a81d0 con 0x563feb938250 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b6802740 10 client.0 _ll_register_callbacks cb 0x7f65a800a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b21cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 0x563feb947a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b21cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 0x563feb947a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43806/0 (socket says 172.21.15.18:43806) 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b21cb640 1 -- 172.21.15.18:0/4194503883 learned_addr learned my addr 172.21.15.18:0/4194503883 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b19ca640 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938250 0x563feb946fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.621+0000 7f65b11c9640 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938b70 0x563feb947520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65b21cb640 1 -- 172.21.15.18:0/4194503883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938250 msgr2=0x563feb946fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65b21cb640 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938250 0x563feb946fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65b21cb640 1 -- 172.21.15.18:0/4194503883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938b70 msgr2=0x563feb947520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:21.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65b21cb640 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938b70 0x563feb947520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65b21cb640 1 -- 172.21.15.18:0/4194503883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563feb89e6a0 con 0x563feb941750 2024-02-17T12:51:21.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65b21cb640 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 0x563feb947a60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f65a4036000 tx=0x7f65a4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8946519daa198861 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:21.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65957fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:21.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65957fa640 1 -- 172.21.15.18:0/4194503883 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a4034850 con 0x563feb941750 2024-02-17T12:51:21.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65957fa640 1 -- 172.21.15.18:0/4194503883 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f65a404e450 con 0x563feb941750 2024-02-17T12:51:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65b6802740 1 -- 172.21.15.18:0/4194503883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563feb92e0f0 con 0x563feb941750 2024-02-17T12:51:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65957fa640 1 -- 172.21.15.18:0/4194503883 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65a404e610 con 0x563feb941750 2024-02-17T12:51:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65b6802740 10 client.16134 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65b6802740 1 -- 172.21.15.18:0/4194503883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563feb8608c0 con 0x563feb941750 2024-02-17T12:51:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65957fa640 1 -- 172.21.15.18:0/4194503883 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563feb92e0f0 con 0x563feb941750 2024-02-17T12:51:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.622+0000 7f65957fa640 1 -- 172.21.15.18:0/4194503883 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f65a403f750 con 0x563feb941750 2024-02-17T12:51:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 10 client.16134 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 20 client.16134 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563feb9a9aa0 0x563feb9abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 4 client.16134 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:51:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 1 -- 172.21.15.18:0/4194503883 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f65a800cd60 con 0x563feb9a9aa0 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b19ca640 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563feb9a9aa0 0x563feb9abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b19ca640 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563feb9a9aa0 0x563feb9abec0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b2be3dc708871c server_cookie=c8ecfb8459389ebe in_seq=0 out_seq=0 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65957fa640 10 client.16134 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65957fa640 1 -- 172.21.15.18:0/4194503883 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f65a800cd60 con 0x563feb9a9aa0 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65957fa640 10 client.16134 handle_command_reply: tid=0 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 1 client.16134 shutdown 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 1 -- 172.21.15.18:0/4194503883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 msgr2=0x563feb947a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 0x563feb947a60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f65a4036000 tx=0x7f65a4005c50 comp rx=0 tx=0).stop 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 1 -- 172.21.15.18:0/4194503883 shutdown_connections 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563feb9a9aa0 0x563feb9abec0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563feb941750 0x563feb947a60 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65957fa640 0 client.16134 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563feb938b70 0x563feb947520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 1 --2- 172.21.15.18:0/4194503883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563feb938250 0x563feb946fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 1 -- 172.21.15.18:0/4194503883 >> 172.21.15.18:0/4194503883 conn(0x563feb932980 msgr2=0x563feb933c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.623+0000 7f65b6802740 1 -- 172.21.15.18:0/4194503883 shutdown_connections 2024-02-17T12:51:21.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b6802740 1 -- 172.21.15.18:0/4194503883 wait complete. 2024-02-17T12:51:21.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b6802740 20 client.16134 trim_cache size 0 max 16384 2024-02-17T12:51:21.623 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:51:21.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b3c2e640 1 -- 172.21.15.18:0/1909101037 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f65980a2350 msgr2=0x7f65980a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b3c2e640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f65980a2350 0x7f65980a4810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f65a4006010 tx=0x7f65a402ee10 comp rx=0 tx=0).stop 2024-02-17T12:51:21.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b3c2e640 1 -- 172.21.15.18:0/1909101037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65ac101380 msgr2=0x7f65ac19d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:21.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b3c2e640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65ac101380 0x7f65ac19d5b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f659c00d950 tx=0x7f659c00de10 comp rx=0 tx=0).stop 2024-02-17T12:51:21.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b3c2e640 1 -- 172.21.15.18:0/1909101037 shutdown_connections 2024-02-17T12:51:21.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b3c2e640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f65980a2350 0x7f65980a4810 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b3c2e640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65ac197670 0x7f65ac197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b3c2e640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac106450 0x7f65ac19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b3c2e640 1 --2- 172.21.15.18:0/1909101037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65ac101380 0x7f65ac19d5b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:21.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b3c2e640 1 -- 172.21.15.18:0/1909101037 >> 172.21.15.18:0/1909101037 conn(0x7f65ac078580 msgr2=0x7f65ac0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:21.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b3c2e640 1 -- 172.21.15.18:0/1909101037 shutdown_connections 2024-02-17T12:51:21.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:21.624+0000 7f65b3c2e640 1 -- 172.21.15.18:0/1909101037 wait complete. 2024-02-17T12:51:22.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:21 smithi018 ceph-mon[32625]: pgmap v47: 97 pgs: 97 active+clean; 37 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 2.0 MiB/s wr, 44 op/s 2024-02-17T12:51:22.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:21 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1133686246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:51:22.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:21 smithi119 ceph-mon[36188]: pgmap v47: 97 pgs: 97 active+clean; 37 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 2.0 MiB/s wr, 44 op/s 2024-02-17T12:51:22.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:21 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1133686246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:51:22.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:21 smithi146 ceph-mon[36454]: pgmap v47: 97 pgs: 97 active+clean; 37 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 2.0 MiB/s wr, 44 op/s 2024-02-17T12:51:22.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:21 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1133686246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:51:22.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50411 7.81 MB/sec warmup 37 sec latency 1395.111 ms 2024-02-17T12:51:22.627 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:51:22.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:51:22.627 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:51:22.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.755+0000 7fc626113640 1 -- 172.21.15.18:0/2185820913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc620068f80 msgr2=0x7fc620105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.755+0000 7fc626113640 1 --2- 172.21.15.18:0/2185820913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc620068f80 0x7fc620105d50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc614009930 tx=0x7fc61402bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:22.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.755+0000 7fc626113640 1 -- 172.21.15.18:0/2185820913 shutdown_connections 2024-02-17T12:51:22.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.755+0000 7fc626113640 1 --2- 172.21.15.18:0/2185820913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc620106430 0x7fc62010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.755+0000 7fc626113640 1 --2- 172.21.15.18:0/2185820913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc620068f80 0x7fc620105d50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.755+0000 7fc626113640 1 --2- 172.21.15.18:0/2185820913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6200685d0 0x7fc6200689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.755+0000 7fc626113640 1 -- 172.21.15.18:0/2185820913 >> 172.21.15.18:0/2185820913 conn(0x7fc620077180 msgr2=0x7fc620077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:22.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.755+0000 7fc626113640 1 -- 172.21.15.18:0/2185820913 shutdown_connections 2024-02-17T12:51:22.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.755+0000 7fc626113640 1 -- 172.21.15.18:0/2185820913 wait complete. 2024-02-17T12:51:22.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.755+0000 7fc626113640 1 Processor -- start 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc626113640 1 -- start start 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc626113640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6200685d0 0x7fc620100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc626113640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc620106430 0x7fc620100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc626113640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc620100ac0 0x7fc6201a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc626113640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201a5f10 con 0x7fc620106430 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc626113640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201a6080 con 0x7fc6200685d0 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc626113640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201a6240 con 0x7fc620100ac0 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc61effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc620106430 0x7fc620100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc61effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc620106430 0x7fc620100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43822/0 (socket says 172.21.15.18:43822) 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc61effd640 1 -- 172.21.15.18:0/3590492752 learned_addr learned my addr 172.21.15.18:0/3590492752 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc61ffff640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc620100ac0 0x7fc6201a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6200685d0 0x7fc620100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc61effd640 1 -- 172.21.15.18:0/3590492752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc620100ac0 msgr2=0x7fc6201a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc61effd640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc620100ac0 0x7fc6201a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc61effd640 1 -- 172.21.15.18:0/3590492752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6200685d0 msgr2=0x7fc620100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc61effd640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6200685d0 0x7fc620100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.756+0000 7fc61effd640 1 -- 172.21.15.18:0/3590492752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc614009560 con 0x7fc620106430 2024-02-17T12:51:22.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.757+0000 7fc61effd640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc620106430 0x7fc620100580 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc614009230 tx=0x7fc614004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:22.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.757+0000 7fc61cff9640 1 -- 172.21.15.18:0/3590492752 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61402cbc0 con 0x7fc620106430 2024-02-17T12:51:22.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.757+0000 7fc61cff9640 1 -- 172.21.15.18:0/3590492752 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6140052d0 con 0x7fc620106430 2024-02-17T12:51:22.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.757+0000 7fc626113640 1 -- 172.21.15.18:0/3590492752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6201a64c0 con 0x7fc620106430 2024-02-17T12:51:22.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.757+0000 7fc61cff9640 1 -- 172.21.15.18:0/3590492752 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614003e30 con 0x7fc620106430 2024-02-17T12:51:22.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.757+0000 7fc626113640 1 -- 172.21.15.18:0/3590492752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6201a69d0 con 0x7fc620106430 2024-02-17T12:51:22.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.759+0000 7fc61cff9640 1 -- 172.21.15.18:0/3590492752 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fc6140035f0 con 0x7fc620106430 2024-02-17T12:51:22.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.759+0000 7fc61cff9640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc6080a2380 0x7fc6080a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.759+0000 7fc628dc7740 1 Processor -- start 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc628dc7740 1 -- start start 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc6080a2380 0x7fc6080a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61cff9640 1 -- 172.21.15.18:0/3590492752 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc6140f0730 con 0x7fc620106430 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc628dc7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2c380 0x5581a1f2c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc628dc7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc628dc7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 0x5581a1f37d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc628dc7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a1e9af90 con 0x5581a1f2cd30 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc628dc7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a1e984c0 con 0x5581a1f2c380 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc628dc7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a1e93ca0 con 0x5581a1f358b0 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f352e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43832/0 (socket says 172.21.15.18:43832) 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61effd640 1 -- 172.21.15.18:0/381087636 learned_addr learned my addr 172.21.15.18:0/381087636 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/381087636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2c380 0x5581a1f2c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61effd640 1 -- 172.21.15.18:0/381087636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 msgr2=0x5581a1f37d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61effd640 1 --2- 172.21.15.18:0/381087636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 0x5581a1f37d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc6080a2380 0x7fc6080a4840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc610005fd0 tx=0x7fc61000f360 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61effd640 1 -- 172.21.15.18:0/381087636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2c380 msgr2=0x5581a1f2c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61effd640 1 --2- 172.21.15.18:0/381087636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2c380 0x5581a1f2c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61effd640 1 -- 172.21.15.18:0/381087636 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a1e560b0 con 0x5581a1f2cd30 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.760+0000 7fc61effd640 1 --2- 172.21.15.18:0/381087636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f352e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc6140b9490 tx=0x7fc61402cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2da3d47007463e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.761+0000 7fc605ffb640 1 -- 172.21.15.18:0/381087636 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6140c1180 con 0x5581a1f2cd30 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.761+0000 7fc605ffb640 1 -- 172.21.15.18:0/381087636 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6140c57d0 con 0x5581a1f2cd30 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.761+0000 7fc605ffb640 1 -- 172.21.15.18:0/381087636 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6140c55a0 con 0x5581a1f2cd30 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.761+0000 7fc628dc7740 1 -- 172.21.15.18:0/381087636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 msgr2=0x5581a1f352e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.761+0000 7fc628dc7740 1 --2- 172.21.15.18:0/381087636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f352e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc6140b9490 tx=0x7fc61402cd20 comp rx=0 tx=0).stop 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.761+0000 7fc628dc7740 1 -- 172.21.15.18:0/381087636 shutdown_connections 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.761+0000 7fc628dc7740 1 --2- 172.21.15.18:0/381087636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 0x5581a1f37d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.761+0000 7fc628dc7740 1 --2- 172.21.15.18:0/381087636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f352e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.761+0000 7fc628dc7740 1 --2- 172.21.15.18:0/381087636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2c380 0x5581a1f2c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.761+0000 7fc628dc7740 1 -- 172.21.15.18:0/381087636 >> 172.21.15.18:0/381087636 conn(0x5581a1f26840 msgr2=0x5581a1f26c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.761+0000 7fc628dc7740 1 -- 172.21.15.18:0/381087636 shutdown_connections 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.761+0000 7fc628dc7740 1 -- 172.21.15.18:0/381087636 wait complete. 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc628dc7740 1 -- start start 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc628dc7740 1 Processor -- start 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc628dc7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2c380 0x5581a1f328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc628dc7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f32e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc628dc7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 0x5581a1f33360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc628dc7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a1e38f40 con 0x5581a1f2cd30 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc628dc7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a1b674b0 con 0x5581a1f2c380 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc628dc7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a1e19670 con 0x5581a1f358b0 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc628dc7740 10 client.0 _ll_register_callbacks cb 0x5581a1e86f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc61effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f32e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc61effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f32e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43848/0 (socket says 172.21.15.18:43848) 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc61effd640 1 -- 172.21.15.18:0/2542353596 learned_addr learned my addr 172.21.15.18:0/2542353596 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc61ffff640 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 0x5581a1f33360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.762+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2c380 0x5581a1f328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc61effd640 1 -- 172.21.15.18:0/2542353596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 msgr2=0x5581a1f33360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc61effd640 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 0x5581a1f33360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc61effd640 1 -- 172.21.15.18:0/2542353596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2c380 msgr2=0x5581a1f328e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc61effd640 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2c380 0x5581a1f328e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc61effd640 1 -- 172.21.15.18:0/2542353596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6201a69d0 con 0x5581a1f2cd30 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc61effd640 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f32e20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc614000c00 tx=0x7fc614005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136a3303f4ce20b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc5feffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc5feffd640 1 -- 172.21.15.18:0/2542353596 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6140c57d0 con 0x5581a1f2cd30 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc5feffd640 1 -- 172.21.15.18:0/2542353596 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc6140c1180 con 0x5581a1f2cd30 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc628dc7740 1 -- 172.21.15.18:0/2542353596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581a1e6df40 con 0x5581a1f2cd30 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc5feffd640 1 -- 172.21.15.18:0/2542353596 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6140ee240 con 0x5581a1f2cd30 2024-02-17T12:51:22.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc628dc7740 10 client.16146 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc628dc7740 1 -- 172.21.15.18:0/2542353596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5581a1e92320 con 0x5581a1f2cd30 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc5feffd640 1 -- 172.21.15.18:0/2542353596 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5581a1e6df40 con 0x5581a1f2cd30 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.763+0000 7fc5feffd640 1 -- 172.21.15.18:0/2542353596 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc61403c070 con 0x5581a1f2cd30 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.764+0000 7fc628dc7740 10 client.16146 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.764+0000 7fc628dc7740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.764+0000 7fc628dc7740 20 client.16146 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.764+0000 7fc628dc7740 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5581a1f45710 0x5581a1f47b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.764+0000 7fc628dc7740 4 client.16146 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.764+0000 7fc628dc7740 1 -- 172.21.15.18:0/2542353596 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581a1e8a310 con 0x5581a1f45710 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.764+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5581a1f45710 0x5581a1f47b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.764+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5581a1f45710 0x5581a1f47b30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e7e0b31503433b server_cookie=10a251eaded741a5 in_seq=0 out_seq=0 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.764+0000 7fc5feffd640 10 client.16146 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc5feffd640 1 -- 172.21.15.18:0/2542353596 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5581a1e8a310 con 0x5581a1f45710 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc5feffd640 10 client.16146 handle_command_reply: tid=0 2024-02-17T12:51:22.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc628dc7740 1 client.16146 shutdown 2024-02-17T12:51:22.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc628dc7740 1 -- 172.21.15.18:0/2542353596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 msgr2=0x5581a1f32e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc628dc7740 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f32e20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc614000c00 tx=0x7fc614005fd0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc628dc7740 1 -- 172.21.15.18:0/2542353596 shutdown_connections 2024-02-17T12:51:22.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc628dc7740 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5581a1f45710 0x5581a1f47b30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc628dc7740 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 0x5581a1f33360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc628dc7740 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f32e20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc5feffd640 0 client.16146 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:22.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc628dc7740 1 --2- 172.21.15.18:0/2542353596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2c380 0x5581a1f328e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc628dc7740 1 -- 172.21.15.18:0/2542353596 >> 172.21.15.18:0/2542353596 conn(0x5581a1f26840 msgr2=0x5581a1f287a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:22.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc628dc7740 1 -- 172.21.15.18:0/2542353596 shutdown_connections 2024-02-17T12:51:22.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.765+0000 7fc628dc7740 1 -- 172.21.15.18:0/2542353596 wait complete. 2024-02-17T12:51:22.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.766+0000 7fc628dc7740 20 client.16146 trim_cache size 0 max 16384 2024-02-17T12:51:22.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.781+0000 7fc628dc7740 1 Processor -- start 2024-02-17T12:51:22.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.781+0000 7fc628dc7740 1 -- start start 2024-02-17T12:51:22.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.781+0000 7fc628dc7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2c380 0x5581a1f31a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.781+0000 7fc628dc7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2cd30 0x5581a1f33180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.781+0000 7fc628dc7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 0x5581a1f25760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.781+0000 7fc628dc7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a1f31fa0 con 0x5581a1f2c380 2024-02-17T12:51:22.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.781+0000 7fc628dc7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a1f22110 con 0x5581a1f2cd30 2024-02-17T12:51:22.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.781+0000 7fc628dc7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a1f38030 con 0x5581a1f358b0 2024-02-17T12:51:22.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.781+0000 7fc61f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2c380 0x5581a1f31a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.781+0000 7fc61f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2c380 0x5581a1f31a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43850/0 (socket says 172.21.15.18:43850) 2024-02-17T12:51:22.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.781+0000 7fc61f7fe640 1 -- 172.21.15.18:0/3710041710 learned_addr learned my addr 172.21.15.18:0/3710041710 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:22.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.782+0000 7fc61f7fe640 1 -- 172.21.15.18:0/3710041710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 msgr2=0x5581a1f25760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.782+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/3710041710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 0x5581a1f25760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.782+0000 7fc61f7fe640 1 -- 172.21.15.18:0/3710041710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2cd30 msgr2=0x5581a1f33180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.782+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/3710041710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2cd30 0x5581a1f33180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.782+0000 7fc61f7fe640 1 -- 172.21.15.18:0/3710041710 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a1e2ec00 con 0x5581a1f2c380 2024-02-17T12:51:22.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.782+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/3710041710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2c380 0x5581a1f31a60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc61000b9f0 tx=0x7fc610014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a639f169b40a89 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:22.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.782+0000 7fc6057fa640 1 -- 172.21.15.18:0/3710041710 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc610023040 con 0x5581a1f2c380 2024-02-17T12:51:22.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.782+0000 7fc6057fa640 1 -- 172.21.15.18:0/3710041710 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc61000b820 con 0x5581a1f2c380 2024-02-17T12:51:22.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.782+0000 7fc6057fa640 1 -- 172.21.15.18:0/3710041710 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc610030630 con 0x5581a1f2c380 2024-02-17T12:51:22.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 -- 172.21.15.18:0/3710041710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2c380 msgr2=0x5581a1f31a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 --2- 172.21.15.18:0/3710041710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2c380 0x5581a1f31a60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc61000b9f0 tx=0x7fc610014000 comp rx=0 tx=0).stop 2024-02-17T12:51:22.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 -- 172.21.15.18:0/3710041710 shutdown_connections 2024-02-17T12:51:22.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 --2- 172.21.15.18:0/3710041710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f358b0 0x5581a1f25760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 --2- 172.21.15.18:0/3710041710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f2cd30 0x5581a1f33180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 --2- 172.21.15.18:0/3710041710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2c380 0x5581a1f31a60 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 -- 172.21.15.18:0/3710041710 >> 172.21.15.18:0/3710041710 conn(0x5581a1f26840 msgr2=0x5581a1f25050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:22.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 -- 172.21.15.18:0/3710041710 shutdown_connections 2024-02-17T12:51:22.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 -- 172.21.15.18:0/3710041710 wait complete. 2024-02-17T12:51:22.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 -- start start 2024-02-17T12:51:22.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 Processor -- start 2024-02-17T12:51:22.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f2c380 0x5581a1f96210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f96750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.783+0000 7fc628dc7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f358b0 0x5581a1f3bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc628dc7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a1f3c160 con 0x5581a1f2cd30 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc628dc7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a1f3c2d0 con 0x5581a1f358b0 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc628dc7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a1f3c490 con 0x5581a1f2c380 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc628dc7740 10 client.0 _ll_register_callbacks cb 0x7fc614005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc61effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f96750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc61effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f96750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43856/0 (socket says 172.21.15.18:43856) 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc61effd640 1 -- 172.21.15.18:0/708462345 learned_addr learned my addr 172.21.15.18:0/708462345 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f2c380 0x5581a1f96210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc61ffff640 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f358b0 0x5581a1f3bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc61effd640 1 -- 172.21.15.18:0/708462345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f2c380 msgr2=0x5581a1f96210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc61effd640 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f2c380 0x5581a1f96210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc61effd640 1 -- 172.21.15.18:0/708462345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f358b0 msgr2=0x5581a1f3bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc61effd640 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f358b0 0x5581a1f3bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc61effd640 1 -- 172.21.15.18:0/708462345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6100125e0 con 0x5581a1f2cd30 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc61effd640 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f96750 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc6140b9490 tx=0x7fc61402c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4f5d60ebbe2a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc5feffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc5feffd640 1 -- 172.21.15.18:0/708462345 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6140bd730 con 0x5581a1f2cd30 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc5feffd640 1 -- 172.21.15.18:0/708462345 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc6140c1180 con 0x5581a1f2cd30 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc628dc7740 1 -- 172.21.15.18:0/708462345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581a1eb9900 con 0x5581a1f2cd30 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc5feffd640 1 -- 172.21.15.18:0/708462345 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6140bcb70 con 0x5581a1f2cd30 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc628dc7740 10 client.16152 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.784+0000 7fc628dc7740 1 -- 172.21.15.18:0/708462345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5581a1e54910 con 0x5581a1f2cd30 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.785+0000 7fc5feffd640 1 -- 172.21.15.18:0/708462345 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5581a1eb9900 con 0x5581a1f2cd30 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.785+0000 7fc5feffd640 1 -- 172.21.15.18:0/708462345 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc61403c070 con 0x5581a1f2cd30 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.785+0000 7fc628dc7740 10 client.16152 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.785+0000 7fc628dc7740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.785+0000 7fc628dc7740 20 client.16152 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:22.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.785+0000 7fc628dc7740 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5581a1f9dbd0 0x5581a1f9ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.785+0000 7fc628dc7740 4 client.16152 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.785+0000 7fc628dc7740 1 -- 172.21.15.18:0/708462345 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5581a1b93860 con 0x5581a1f9dbd0 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.785+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5581a1f9dbd0 0x5581a1f9ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.786+0000 7fc61f7fe640 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5581a1f9dbd0 0x5581a1f9ffc0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c3cbc447e0855b server_cookie=948bc692f7070368 in_seq=0 out_seq=0 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.786+0000 7fc5feffd640 10 client.16152 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc5feffd640 1 -- 172.21.15.18:0/708462345 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5581a1b93860 con 0x5581a1f9dbd0 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc5feffd640 10 client.16152 handle_command_reply: tid=0 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc628dc7740 1 client.16152 shutdown 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc628dc7740 1 -- 172.21.15.18:0/708462345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 msgr2=0x5581a1f96750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc628dc7740 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f96750 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc6140b9490 tx=0x7fc61402c450 comp rx=0 tx=0).stop 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc628dc7740 1 -- 172.21.15.18:0/708462345 shutdown_connections 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc628dc7740 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5581a1f9dbd0 0x5581a1f9ffc0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc628dc7740 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581a1f358b0 0x5581a1f3bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc628dc7740 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581a1f2cd30 0x5581a1f96750 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc5feffd640 0 client.16152 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:22.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc628dc7740 1 --2- 172.21.15.18:0/708462345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5581a1f2c380 0x5581a1f96210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc628dc7740 1 -- 172.21.15.18:0/708462345 >> 172.21.15.18:0/708462345 conn(0x5581a1f26840 msgr2=0x5581a1f258e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:22.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc628dc7740 1 -- 172.21.15.18:0/708462345 shutdown_connections 2024-02-17T12:51:22.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc628dc7740 1 -- 172.21.15.18:0/708462345 wait complete. 2024-02-17T12:51:22.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.787+0000 7fc628dc7740 20 client.16152 trim_cache size 0 max 16384 2024-02-17T12:51:22.784 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:51:22.784 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:51:22.784 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "b972c2f2-2ecf-4f52-a9a8-091e249b5d86", 2024-02-17T12:51:22.784 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:51:22.785 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:51:22.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.788+0000 7fc626113640 1 -- 172.21.15.18:0/3590492752 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc6080a2380 msgr2=0x7fc6080a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.788+0000 7fc626113640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc6080a2380 0x7fc6080a4840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc610005fd0 tx=0x7fc61000f360 comp rx=0 tx=0).stop 2024-02-17T12:51:22.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.788+0000 7fc626113640 1 -- 172.21.15.18:0/3590492752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc620106430 msgr2=0x7fc620100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.788+0000 7fc626113640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc620106430 0x7fc620100580 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc614009230 tx=0x7fc614004270 comp rx=0 tx=0).stop 2024-02-17T12:51:22.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.788+0000 7fc626113640 1 -- 172.21.15.18:0/3590492752 shutdown_connections 2024-02-17T12:51:22.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.788+0000 7fc626113640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc6080a2380 0x7fc6080a4840 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.788+0000 7fc626113640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc620100ac0 0x7fc6201a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.788+0000 7fc626113640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc620106430 0x7fc620100580 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.788+0000 7fc626113640 1 --2- 172.21.15.18:0/3590492752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6200685d0 0x7fc620100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.788+0000 7fc626113640 1 -- 172.21.15.18:0/3590492752 >> 172.21.15.18:0/3590492752 conn(0x7fc620077180 msgr2=0x7fc6200fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:22.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.788+0000 7fc626113640 1 -- 172.21.15.18:0/3590492752 shutdown_connections 2024-02-17T12:51:22.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.788+0000 7fc626113640 1 -- 172.21.15.18:0/3590492752 wait complete. 2024-02-17T12:51:22.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:51:22.791 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:51:22.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.911+0000 7ff4595a1640 1 -- 172.21.15.18:0/2439018174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff454100270 msgr2=0x7ff454100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.911+0000 7ff4595a1640 1 --2- 172.21.15.18:0/2439018174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff454100270 0x7ff454100650 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff444009930 tx=0x7ff44402bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:22.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.911+0000 7ff4595a1640 1 -- 172.21.15.18:0/2439018174 shutdown_connections 2024-02-17T12:51:22.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.911+0000 7ff4595a1640 1 --2- 172.21.15.18:0/2439018174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff454105340 0x7ff45410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.911+0000 7ff4595a1640 1 --2- 172.21.15.18:0/2439018174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff454100c20 0x7ff454104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.911+0000 7ff4595a1640 1 --2- 172.21.15.18:0/2439018174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff454100270 0x7ff454100650 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.911+0000 7ff4595a1640 1 -- 172.21.15.18:0/2439018174 >> 172.21.15.18:0/2439018174 conn(0x7ff454077230 msgr2=0x7ff454077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:22.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.911+0000 7ff4595a1640 1 -- 172.21.15.18:0/2439018174 shutdown_connections 2024-02-17T12:51:22.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.911+0000 7ff4595a1640 1 -- 172.21.15.18:0/2439018174 wait complete. 2024-02-17T12:51:22.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.911+0000 7ff4595a1640 1 Processor -- start 2024-02-17T12:51:22.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff4595a1640 1 -- start start 2024-02-17T12:51:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff4595a1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff454100270 0x7ff45419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff4595a1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff454105340 0x7ff45419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff4595a1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4541975e0 0x7ff454197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff4595a1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff454197fd0 con 0x7ff454100270 2024-02-17T12:51:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff4595a1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff454198140 con 0x7ff454105340 2024-02-17T12:51:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff4595a1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff454198300 con 0x7ff4541975e0 2024-02-17T12:51:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff454100270 0x7ff45419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff454100270 0x7ff45419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43866/0 (socket says 172.21.15.18:43866) 2024-02-17T12:51:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff452ffd640 1 -- 172.21.15.18:0/3492532348 learned_addr learned my addr 172.21.15.18:0/3492532348 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff44a7fc640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff454105340 0x7ff45419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff4537fe640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4541975e0 0x7ff454197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff452ffd640 1 -- 172.21.15.18:0/3492532348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4541975e0 msgr2=0x7ff454197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff452ffd640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4541975e0 0x7ff454197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff452ffd640 1 -- 172.21.15.18:0/3492532348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff454105340 msgr2=0x7ff45419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff452ffd640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff454105340 0x7ff45419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff452ffd640 1 -- 172.21.15.18:0/3492532348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff444009560 con 0x7ff454100270 2024-02-17T12:51:22.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff44a7fc640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff454105340 0x7ff45419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:22.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff452ffd640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff454100270 0x7ff45419d430 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff444009230 tx=0x7ff44402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:22.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.912+0000 7ff450ff9640 1 -- 172.21.15.18:0/3492532348 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444004350 con 0x7ff454100270 2024-02-17T12:51:22.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.913+0000 7ff4595a1640 1 -- 172.21.15.18:0/3492532348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4541a42f0 con 0x7ff454100270 2024-02-17T12:51:22.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.913+0000 7ff450ff9640 1 -- 172.21.15.18:0/3492532348 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4440044b0 con 0x7ff454100270 2024-02-17T12:51:22.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.913+0000 7ff4595a1640 1 -- 172.21.15.18:0/3492532348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4541a47f0 con 0x7ff454100270 2024-02-17T12:51:22.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.913+0000 7ff450ff9640 1 -- 172.21.15.18:0/3492532348 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444003d00 con 0x7ff454100270 2024-02-17T12:51:22.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.915+0000 7ff450ff9640 1 -- 172.21.15.18:0/3492532348 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 137001+0+0 (secure 0 0 0) 0x7ff444004df0 con 0x7ff454100270 2024-02-17T12:51:22.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.915+0000 7ff45c178740 1 Processor -- start 2024-02-17T12:51:22.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.915+0000 7ff45c178740 1 -- start start 2024-02-17T12:51:22.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.915+0000 7ff45c178740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff9832b0 0x55d9ff983690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff45c178740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff98c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff45c178740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff98c7b0 0x55d9ff98eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff45c178740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ff8f2280 con 0x55d9ff983bd0 2024-02-17T12:51:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff45c178740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ff8ef7b0 con 0x55d9ff9832b0 2024-02-17T12:51:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff45c178740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ff8eaf90 con 0x55d9ff98c7b0 2024-02-17T12:51:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff450ff9640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4400a2350 0x7ff4400a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff98c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff4537fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff9832b0 0x55d9ff983690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff98c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43868/0 (socket says 172.21.15.18:43868) 2024-02-17T12:51:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff452ffd640 1 -- 172.21.15.18:0/2513799642 learned_addr learned my addr 172.21.15.18:0/2513799642 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff4537fe640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4400a2350 0x7ff4400a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff450ff9640 1 -- 172.21.15.18:0/3492532348 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff4440f0120 con 0x7ff454100270 2024-02-17T12:51:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff4537fe640 1 -- 172.21.15.18:0/2513799642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff98c7b0 msgr2=0x55d9ff98eba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:51:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff4537fe640 1 --2- 172.21.15.18:0/2513799642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff98c7b0 0x55d9ff98eba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff4537fe640 1 -- 172.21.15.18:0/2513799642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 msgr2=0x55d9ff98c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff4537fe640 1 --2- 172.21.15.18:0/2513799642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff98c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff4537fe640 1 -- 172.21.15.18:0/2513799642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9ff8a9570 con 0x55d9ff9832b0 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff452ffd640 1 --2- 172.21.15.18:0/2513799642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff98c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff4537fe640 1 --2- 172.21.15.18:0/2513799642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff9832b0 0x55d9ff983690 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff44c010d00 tx=0x7ff44c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256cc8137bd479d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff4537fe640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4400a2350 0x7ff4400a4810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00fc90 tx=0x7ff44c00e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.916+0000 7ff4497fa640 1 -- 172.21.15.18:0/2513799642 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c010600 con 0x55d9ff9832b0 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.917+0000 7ff4497fa640 1 -- 172.21.15.18:0/2513799642 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff44c010760 con 0x55d9ff9832b0 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.917+0000 7ff4497fa640 1 -- 172.21.15.18:0/2513799642 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c0053e0 con 0x55d9ff9832b0 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.917+0000 7ff45c178740 1 -- 172.21.15.18:0/2513799642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff9832b0 msgr2=0x55d9ff983690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.917+0000 7ff45c178740 1 --2- 172.21.15.18:0/2513799642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff9832b0 0x55d9ff983690 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff44c010d00 tx=0x7ff44c00f790 comp rx=0 tx=0).stop 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.917+0000 7ff45c178740 1 -- 172.21.15.18:0/2513799642 shutdown_connections 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.917+0000 7ff45c178740 1 --2- 172.21.15.18:0/2513799642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff98c7b0 0x55d9ff98eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.917+0000 7ff45c178740 1 --2- 172.21.15.18:0/2513799642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff98c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.917+0000 7ff45c178740 1 --2- 172.21.15.18:0/2513799642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff9832b0 0x55d9ff983690 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.917+0000 7ff45c178740 1 -- 172.21.15.18:0/2513799642 >> 172.21.15.18:0/2513799642 conn(0x55d9ff97d990 msgr2=0x55d9ff97dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.917+0000 7ff45c178740 1 -- 172.21.15.18:0/2513799642 shutdown_connections 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.917+0000 7ff45c178740 1 -- 172.21.15.18:0/2513799642 wait complete. 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff45c178740 1 -- start start 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff45c178740 1 Processor -- start 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff45c178740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff9832b0 0x55d9ff989630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff45c178740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff989b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff45c178740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff98c7b0 0x55d9ff98a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff45c178740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ff5be430 con 0x55d9ff983bd0 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff45c178740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ff57ae00 con 0x55d9ff98c7b0 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff45c178740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ff86ffd0 con 0x55d9ff9832b0 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff45c178740 10 client.0 _ll_register_callbacks cb 0x55d9ff8c5fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:22.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff44a7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff989b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff44a7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff989b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43870/0 (socket says 172.21.15.18:43870) 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff44a7fc640 1 -- 172.21.15.18:0/4114489632 learned_addr learned my addr 172.21.15.18:0/4114489632 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff4537fe640 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff98c7b0 0x55d9ff98a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff44a7fc640 1 -- 172.21.15.18:0/4114489632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff9832b0 msgr2=0x55d9ff989630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff44a7fc640 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff9832b0 0x55d9ff989630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff44a7fc640 1 -- 172.21.15.18:0/4114489632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff98c7b0 msgr2=0x55d9ff98a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff44a7fc640 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff98c7b0 0x55d9ff98a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff44a7fc640 1 -- 172.21.15.18:0/4114489632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c0108e0 con 0x55d9ff983bd0 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.918+0000 7ff4537fe640 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff98c7b0 0x55d9ff98a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff44a7fc640 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff989b70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff43c011970 tx=0x7ff43c011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e4dae3fa2cbac server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff43a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff43a7fc640 1 -- 172.21.15.18:0/4114489632 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c00fe50 con 0x55d9ff983bd0 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff43a7fc640 1 -- 172.21.15.18:0/4114489632 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff43c004030 con 0x55d9ff983bd0 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff45c178740 1 -- 172.21.15.18:0/4114489632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9ff627720 con 0x55d9ff983bd0 2024-02-17T12:51:22.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff43a7fc640 1 -- 172.21.15.18:0/4114489632 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff43c0041f0 con 0x55d9ff983bd0 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff45c178740 10 client.16164 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff45c178740 1 -- 172.21.15.18:0/4114489632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d9ff8e9610 con 0x55d9ff983bd0 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff43a7fc640 1 -- 172.21.15.18:0/4114489632 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff43c026020 con 0x55d9ff983bd0 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff43a7fc640 1 -- 172.21.15.18:0/4114489632 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff43c019420 con 0x55d9ff983bd0 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff45c178740 10 client.16164 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff45c178740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff45c178740 20 client.16164 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff45c178740 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ff99cb50 0x55d9ff99ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff45c178740 4 client.16164 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.919+0000 7ff45c178740 1 -- 172.21.15.18:0/4114489632 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9ff676890 con 0x55d9ff99cb50 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.920+0000 7ff452ffd640 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ff99cb50 0x55d9ff99ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.920+0000 7ff43a7fc640 10 client.16164 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.920+0000 7ff452ffd640 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ff99cb50 0x55d9ff99ef70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e7eb60e27cb31a server_cookie=e5e2e1df0d2e2567 in_seq=0 out_seq=0 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff43a7fc640 1 -- 172.21.15.18:0/4114489632 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d9ff676890 con 0x55d9ff99cb50 2024-02-17T12:51:22.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff43a7fc640 10 client.16164 handle_command_reply: tid=0 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff45c178740 1 client.16164 shutdown 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff45c178740 1 -- 172.21.15.18:0/4114489632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 msgr2=0x55d9ff989b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff45c178740 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff989b70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff43c011970 tx=0x7ff43c011e30 comp rx=0 tx=0).stop 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff45c178740 1 -- 172.21.15.18:0/4114489632 shutdown_connections 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff45c178740 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ff99cb50 0x55d9ff99ef70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff45c178740 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff98c7b0 0x55d9ff98a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff43a7fc640 0 client.16164 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff45c178740 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff989b70 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff45c178740 1 --2- 172.21.15.18:0/4114489632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff9832b0 0x55d9ff989630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff45c178740 1 -- 172.21.15.18:0/4114489632 >> 172.21.15.18:0/4114489632 conn(0x55d9ff97d990 msgr2=0x55d9ff97f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff45c178740 1 -- 172.21.15.18:0/4114489632 shutdown_connections 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff45c178740 1 -- 172.21.15.18:0/4114489632 wait complete. 2024-02-17T12:51:22.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.921+0000 7ff45c178740 20 client.16164 trim_cache size 0 max 16384 2024-02-17T12:51:22.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.937+0000 7ff45c178740 1 Processor -- start 2024-02-17T12:51:22.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.937+0000 7ff45c178740 1 -- start start 2024-02-17T12:51:22.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.937+0000 7ff45c178740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff9832b0 0x55d9ff98bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.937+0000 7ff45c178740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff988610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.937+0000 7ff45c178740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff98c7b0 0x55d9ff989ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.937+0000 7ff45c178740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ff97e9e0 con 0x55d9ff983bd0 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.937+0000 7ff45c178740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ff97d1d0 con 0x55d9ff98c7b0 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.937+0000 7ff45c178740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ff987b80 con 0x55d9ff9832b0 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.937+0000 7ff44a7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff988610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.937+0000 7ff44a7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff988610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43882/0 (socket says 172.21.15.18:43882) 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.937+0000 7ff44a7fc640 1 -- 172.21.15.18:0/3792985361 learned_addr learned my addr 172.21.15.18:0/3792985361 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.938+0000 7ff4537fe640 1 --2- 172.21.15.18:0/3792985361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff98c7b0 0x55d9ff989ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.938+0000 7ff452ffd640 1 --2- 172.21.15.18:0/3792985361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff9832b0 0x55d9ff98bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.938+0000 7ff44a7fc640 1 -- 172.21.15.18:0/3792985361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff9832b0 msgr2=0x55d9ff98bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.938+0000 7ff44a7fc640 1 --2- 172.21.15.18:0/3792985361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff9832b0 0x55d9ff98bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.938+0000 7ff44a7fc640 1 -- 172.21.15.18:0/3792985361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff98c7b0 msgr2=0x55d9ff989ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.938+0000 7ff44a7fc640 1 --2- 172.21.15.18:0/3792985361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff98c7b0 0x55d9ff989ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.938+0000 7ff44a7fc640 1 -- 172.21.15.18:0/3792985361 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9ff885ac0 con 0x55d9ff983bd0 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.938+0000 7ff44a7fc640 1 --2- 172.21.15.18:0/3792985361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff988610 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff43c000c00 tx=0x7ff43c00f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86eb07034488e5cd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.938+0000 7ff448ff9640 1 -- 172.21.15.18:0/3792985361 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c005cd0 con 0x55d9ff983bd0 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.938+0000 7ff448ff9640 1 -- 172.21.15.18:0/3792985361 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff43c026030 con 0x55d9ff983bd0 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.938+0000 7ff448ff9640 1 -- 172.21.15.18:0/3792985361 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0056f0 con 0x55d9ff983bd0 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 -- 172.21.15.18:0/3792985361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 msgr2=0x55d9ff988610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 --2- 172.21.15.18:0/3792985361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff988610 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff43c000c00 tx=0x7ff43c00f830 comp rx=0 tx=0).stop 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 -- 172.21.15.18:0/3792985361 shutdown_connections 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 --2- 172.21.15.18:0/3792985361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff98c7b0 0x55d9ff989ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 --2- 172.21.15.18:0/3792985361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff983bd0 0x55d9ff988610 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 --2- 172.21.15.18:0/3792985361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff9832b0 0x55d9ff98bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 -- 172.21.15.18:0/3792985361 >> 172.21.15.18:0/3792985361 conn(0x55d9ff97d990 msgr2=0x55d9ff987f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 -- 172.21.15.18:0/3792985361 shutdown_connections 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 -- 172.21.15.18:0/3792985361 wait complete. 2024-02-17T12:51:22.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 -- start start 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 Processor -- start 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff9832b0 0x55d9ff992230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff983bd0 0x55d9ff992770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff98c7b0 0x55d9ff992cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ff9f3190 con 0x55d9ff9832b0 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ff9f3300 con 0x55d9ff983bd0 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.939+0000 7ff45c178740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ff9f34c0 con 0x55d9ff98c7b0 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff45c178740 10 client.0 _ll_register_callbacks cb 0x7ff44c009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff9832b0 0x55d9ff992230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff9832b0 0x55d9ff992230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43892/0 (socket says 172.21.15.18:43892) 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff452ffd640 1 -- 172.21.15.18:0/3103907554 learned_addr learned my addr 172.21.15.18:0/3103907554 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff44a7fc640 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff983bd0 0x55d9ff992770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff4537fe640 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff98c7b0 0x55d9ff992cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff452ffd640 1 -- 172.21.15.18:0/3103907554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff98c7b0 msgr2=0x55d9ff992cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff452ffd640 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff98c7b0 0x55d9ff992cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff452ffd640 1 -- 172.21.15.18:0/3103907554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff983bd0 msgr2=0x55d9ff992770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff452ffd640 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff983bd0 0x55d9ff992770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff452ffd640 1 -- 172.21.15.18:0/3103907554 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9ff8e9610 con 0x55d9ff9832b0 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff452ffd640 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff9832b0 0x55d9ff992230 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff4440ed9a0 tx=0x7ff444005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd318a370a0ee827 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff43a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff43a7fc640 1 -- 172.21.15.18:0/3103907554 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4440035f0 con 0x55d9ff9832b0 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff43a7fc640 1 -- 172.21.15.18:0/3103907554 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff4440bd160 con 0x55d9ff9832b0 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff45c178740 1 -- 172.21.15.18:0/3103907554 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9ff5ea7e0 con 0x55d9ff9832b0 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff43a7fc640 1 -- 172.21.15.18:0/3103907554 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4440bca60 con 0x55d9ff9832b0 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff45c178740 10 client.16170 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff45c178740 1 -- 172.21.15.18:0/3103907554 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d9ff8ab6a0 con 0x55d9ff9832b0 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.940+0000 7ff43a7fc640 1 -- 172.21.15.18:0/3103907554 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4440d9080 con 0x55d9ff9832b0 2024-02-17T12:51:22.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.941+0000 7ff43a7fc640 1 -- 172.21.15.18:0/3103907554 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff44403c070 con 0x55d9ff9832b0 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.941+0000 7ff45c178740 10 client.16170 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.941+0000 7ff45c178740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.941+0000 7ff45c178740 20 client.16170 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.941+0000 7ff45c178740 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ff9f4b60 0x55d9ff9f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.941+0000 7ff45c178740 4 client.16170 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.941+0000 7ff45c178740 1 -- 172.21.15.18:0/3103907554 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d9ff5d94e0 con 0x55d9ff9f4b60 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.941+0000 7ff44a7fc640 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ff9f4b60 0x55d9ff9f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.941+0000 7ff44a7fc640 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ff9f4b60 0x55d9ff9f6f80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=894aa938642f2c34 server_cookie=65228ae7e9124d66 in_seq=0 out_seq=0 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.941+0000 7ff43a7fc640 10 client.16170 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff43a7fc640 1 -- 172.21.15.18:0/3103907554 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d9ff5d94e0 con 0x55d9ff9f4b60 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff43a7fc640 10 client.16170 handle_command_reply: tid=0 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff45c178740 1 client.16170 shutdown 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff45c178740 1 -- 172.21.15.18:0/3103907554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff9832b0 msgr2=0x55d9ff992230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff45c178740 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff9832b0 0x55d9ff992230 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff4440ed9a0 tx=0x7ff444005fd0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff45c178740 1 -- 172.21.15.18:0/3103907554 shutdown_connections 2024-02-17T12:51:22.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff45c178740 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ff9f4b60 0x55d9ff9f6f80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff45c178740 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ff98c7b0 0x55d9ff992cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff45c178740 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ff983bd0 0x55d9ff992770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff45c178740 1 --2- 172.21.15.18:0/3103907554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ff9832b0 0x55d9ff992230 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff45c178740 1 -- 172.21.15.18:0/3103907554 >> 172.21.15.18:0/3103907554 conn(0x55d9ff97d990 msgr2=0x55d9ff97ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:22.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff43a7fc640 0 client.16170 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:22.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff45c178740 1 -- 172.21.15.18:0/3103907554 shutdown_connections 2024-02-17T12:51:22.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff45c178740 1 -- 172.21.15.18:0/3103907554 wait complete. 2024-02-17T12:51:22.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff45c178740 20 client.16170 trim_cache size 0 max 16384 2024-02-17T12:51:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:51:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:51:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:51:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "b972c2f2-2ecf-4f52-a9a8-091e249b5d86": { 2024-02-17T12:51:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:51:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "b972c2f2-2ecf-4f52-a9a8-091e249b5d86", 2024-02-17T12:51:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:51:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:51:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:51:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:51:22.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff4595a1640 1 -- 172.21.15.18:0/3492532348 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4400a2350 msgr2=0x7ff4400a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff4595a1640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4400a2350 0x7ff4400a4810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00fc90 tx=0x7ff44c00e000 comp rx=0 tx=0).stop 2024-02-17T12:51:22.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff4595a1640 1 -- 172.21.15.18:0/3492532348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff454100270 msgr2=0x7ff45419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:22.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff4595a1640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff454100270 0x7ff45419d430 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff444009230 tx=0x7ff44402c650 comp rx=0 tx=0).stop 2024-02-17T12:51:22.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff4595a1640 1 -- 172.21.15.18:0/3492532348 shutdown_connections 2024-02-17T12:51:22.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff4595a1640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4400a2350 0x7ff4400a4810 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff4595a1640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4541975e0 0x7ff454197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff4595a1640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff454105340 0x7ff45419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.942+0000 7ff4595a1640 1 --2- 172.21.15.18:0/3492532348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff454100270 0x7ff45419d430 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:22.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.943+0000 7ff4595a1640 1 -- 172.21.15.18:0/3492532348 >> 172.21.15.18:0/3492532348 conn(0x7ff454077230 msgr2=0x7ff454068660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:22.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.943+0000 7ff4595a1640 1 -- 172.21.15.18:0/3492532348 shutdown_connections 2024-02-17T12:51:22.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:22.943+0000 7ff4595a1640 1 -- 172.21.15.18:0/3492532348 wait complete. 2024-02-17T12:51:22.946 INFO:tasks.cephfs.filesystem:scrub status for tag:b972c2f2-2ecf-4f52-a9a8-091e249b5d86 - {'path': '/', 'tag': 'b972c2f2-2ecf-4f52-a9a8-091e249b5d86', 'options': 'recursive,force'} 2024-02-17T12:51:23.260 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50601 7.62 MB/sec warmup 38 sec latency 1984.346 ms 2024-02-17T12:51:24.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: pgmap v48: 97 pgs: 97 active+clean; 37 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 2.0 MiB/s wr, 43 op/s 2024-02-17T12:51:24.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:51:24.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:24.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:24.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:24.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:24.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:24.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:24.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:23 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: pgmap v48: 97 pgs: 97 active+clean; 37 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 2.0 MiB/s wr, 43 op/s 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:23 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.261 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50601 7.42 MB/sec warmup 39 sec latency 1311.028 ms 2024-02-17T12:51:24.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: pgmap v48: 97 pgs: 97 active+clean; 37 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 2.0 MiB/s wr, 43 op/s 2024-02-17T12:51:24.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:51:24.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:24.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:24.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:24.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:24.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:24.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:24.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:24.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:23 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:25.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:24 smithi119 ceph-mon[36188]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-02-17T12:51:25.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:24 smithi146 ceph-mon[36454]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-02-17T12:51:25.261 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50601 7.24 MB/sec warmup 40 sec latency 2311.125 ms 2024-02-17T12:51:25.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:24 smithi018 ceph-mon[32625]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-02-17T12:51:26.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:25 smithi119 ceph-mon[36188]: pgmap v49: 97 pgs: 97 active+clean; 34 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 3.1 MiB/s wr, 110 op/s 2024-02-17T12:51:26.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:25 smithi146 ceph-mon[36454]: pgmap v49: 97 pgs: 97 active+clean; 34 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 3.1 MiB/s wr, 110 op/s 2024-02-17T12:51:26.261 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50667 7.06 MB/sec warmup 41 sec latency 2881.771 ms 2024-02-17T12:51:26.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:25 smithi018 ceph-mon[32625]: pgmap v49: 97 pgs: 97 active+clean; 34 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 3.1 MiB/s wr, 110 op/s 2024-02-17T12:51:27.261 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50667 6.89 MB/sec warmup 42 sec latency 1397.495 ms 2024-02-17T12:51:28.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:27 smithi119 ceph-mon[36188]: pgmap v50: 97 pgs: 97 active+clean; 34 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 3.1 MiB/s wr, 104 op/s 2024-02-17T12:51:28.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:27 smithi146 ceph-mon[36454]: pgmap v50: 97 pgs: 97 active+clean; 34 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 3.1 MiB/s wr, 104 op/s 2024-02-17T12:51:28.261 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50667 6.73 MB/sec warmup 43 sec latency 2397.619 ms 2024-02-17T12:51:28.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:27 smithi018 ceph-mon[32625]: pgmap v50: 97 pgs: 97 active+clean; 34 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 3.1 MiB/s wr, 104 op/s 2024-02-17T12:51:29.261 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50667 6.58 MB/sec warmup 44 sec latency 3397.736 ms 2024-02-17T12:51:30.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:29 smithi119 ceph-mon[36188]: pgmap v51: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.2 MiB/s wr, 134 op/s 2024-02-17T12:51:30.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:29 smithi146 ceph-mon[36454]: pgmap v51: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.2 MiB/s wr, 134 op/s 2024-02-17T12:51:30.261 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50667 6.43 MB/sec warmup 45 sec latency 4397.784 ms 2024-02-17T12:51:30.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:29 smithi018 ceph-mon[32625]: pgmap v51: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.2 MiB/s wr, 134 op/s 2024-02-17T12:51:31.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:30 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:51:31.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:30 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:51:31.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:30 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:51:31.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:30 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:51:31.261 INFO:tasks.workunit.client.0.smithi018.stdout: 1 52080 6.43 MB/sec warmup 46 sec latency 4981.909 ms 2024-02-17T12:51:31.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:30 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:51:31.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:30 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:51:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.035+0000 7f42ed7d7640 1 -- 172.21.15.18:0/971079295 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42e81002d0 msgr2=0x7f42e8100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.035+0000 7f42ed7d7640 1 --2- 172.21.15.18:0/971079295 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42e81002d0 0x7f42e8100750 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009900 tx=0x7f42d002bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.035+0000 7f42ed7d7640 1 -- 172.21.15.18:0/971079295 shutdown_connections 2024-02-17T12:51:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.035+0000 7f42ed7d7640 1 --2- 172.21.15.18:0/971079295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e8100c90 0x7f42e810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.035+0000 7f42ed7d7640 1 --2- 172.21.15.18:0/971079295 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42e81002d0 0x7f42e8100750 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.035+0000 7f42ed7d7640 1 --2- 172.21.15.18:0/971079295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e8105020 0x7f42e8105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.035+0000 7f42ed7d7640 1 -- 172.21.15.18:0/971079295 >> 172.21.15.18:0/971079295 conn(0x7f42e8076ed0 msgr2=0x7f42e80772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.035+0000 7f42ed7d7640 1 -- 172.21.15.18:0/971079295 shutdown_connections 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.035+0000 7f42ed7d7640 1 -- 172.21.15.18:0/971079295 wait complete. 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.036+0000 7f42ed7d7640 1 Processor -- start 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.036+0000 7f42ed7d7640 1 -- start start 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.036+0000 7f42ed7d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e81002d0 0x7f42e8196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.036+0000 7f42ed7d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42e8100c90 0x7f42e8196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.036+0000 7f42ed7d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e8105020 0x7f42e819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.036+0000 7f42ed7d7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e819b520 con 0x7f42e8100c90 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.036+0000 7f42ed7d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e819b690 con 0x7f42e81002d0 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.036+0000 7f42ed7d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e819b850 con 0x7f42e8105020 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.036+0000 7f42e67fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42e8100c90 0x7f42e8196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.036+0000 7f42e67fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42e8100c90 0x7f42e8196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46564/0 (socket says 172.21.15.18:46564) 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.036+0000 7f42e6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e81002d0 0x7f42e8196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.036+0000 7f42e67fc640 1 -- 172.21.15.18:0/1616049169 learned_addr learned my addr 172.21.15.18:0/1616049169 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42e77fe640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e8105020 0x7f42e819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42e67fc640 1 -- 172.21.15.18:0/1616049169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e8105020 msgr2=0x7f42e819af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42e67fc640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e8105020 0x7f42e819af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42e67fc640 1 -- 172.21.15.18:0/1616049169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e81002d0 msgr2=0x7f42e8196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42e67fc640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e81002d0 0x7f42e8196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42e67fc640 1 -- 172.21.15.18:0/1616049169 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d0009560 con 0x7f42e8100c90 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e81002d0 0x7f42e8196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42e77fe640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e8105020 0x7f42e819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42e67fc640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42e8100c90 0x7f42e8196ba0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f42d0005940 tx=0x7f42d002c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42dffff640 1 -- 172.21.15.18:0/1616049169 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00044d0 con 0x7f42e8100c90 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42ed7d7640 1 -- 172.21.15.18:0/1616049169 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e819bad0 con 0x7f42e8100c90 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42dffff640 1 -- 172.21.15.18:0/1616049169 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d0040070 con 0x7f42e8100c90 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42dffff640 1 -- 172.21.15.18:0/1616049169 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0003d30 con 0x7f42e8100c90 2024-02-17T12:51:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.037+0000 7f42ed7d7640 1 -- 172.21.15.18:0/1616049169 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e80689e0 con 0x7f42e8100c90 2024-02-17T12:51:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.039+0000 7f42dffff640 1 -- 172.21.15.18:0/1616049169 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f42d0004df0 con 0x7f42e8100c90 2024-02-17T12:51:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42f03a9740 1 Processor -- start 2024-02-17T12:51:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42f03a9740 1 -- start start 2024-02-17T12:51:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42f03a9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f6ad0 0x555db81f5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42f03a9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 0x555db81f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42f03a9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db8205ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42f03a9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db81649f0 con 0x555db81f5d60 2024-02-17T12:51:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42f03a9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db8161f20 con 0x555db82036f0 2024-02-17T12:51:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42dffff640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42d40a22d0 0x7f42d40a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42f03a9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db815d700 con 0x555db81f6ad0 2024-02-17T12:51:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42e6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 0x555db81f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42e6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 0x555db81f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46568/0 (socket says 172.21.15.18:46568) 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42e77fe640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42d40a22d0 0x7f42d40a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42e6ffd640 1 -- 172.21.15.18:0/162679616 learned_addr learned my addr 172.21.15.18:0/162679616 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42e67fc640 1 --2- 172.21.15.18:0/162679616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db8205ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42e77fe640 1 --2- 172.21.15.18:0/162679616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f6ad0 0x555db81f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42dffff640 1 -- 172.21.15.18:0/1616049169 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f42d0042070 con 0x7f42e8100c90 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42e6ffd640 1 -- 172.21.15.18:0/162679616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f6ad0 msgr2=0x555db81f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/162679616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f6ad0 0x555db81f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42e6ffd640 1 -- 172.21.15.18:0/162679616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 msgr2=0x555db8205ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/162679616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db8205ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.040+0000 7f42e6ffd640 1 -- 172.21.15.18:0/162679616 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db80f37b0 con 0x555db81f5d60 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.041+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/162679616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 0x555db81f61e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f42d8011860 tx=0x7f42d8011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd70cc4c1b49a4f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.041+0000 7f42e77fe640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42d40a22d0 0x7f42d40a4790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42e0012690 tx=0x7f42e0009660 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.041+0000 7f42dd7fa640 1 -- 172.21.15.18:0/162679616 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d80112f0 con 0x555db81f5d60 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.041+0000 7f42dd7fa640 1 -- 172.21.15.18:0/162679616 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d8003e50 con 0x555db81f5d60 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.041+0000 7f42dd7fa640 1 -- 172.21.15.18:0/162679616 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d80044f0 con 0x555db81f5d60 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 -- 172.21.15.18:0/162679616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 msgr2=0x555db81f61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 --2- 172.21.15.18:0/162679616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 0x555db81f61e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f42d8011860 tx=0x7f42d8011d20 comp rx=0 tx=0).stop 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 -- 172.21.15.18:0/162679616 shutdown_connections 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 --2- 172.21.15.18:0/162679616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db8205ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 --2- 172.21.15.18:0/162679616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 0x555db81f61e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 --2- 172.21.15.18:0/162679616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f6ad0 0x555db81f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 -- 172.21.15.18:0/162679616 >> 172.21.15.18:0/162679616 conn(0x555db81bd550 msgr2=0x555db81f0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 -- 172.21.15.18:0/162679616 shutdown_connections 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 -- 172.21.15.18:0/162679616 wait complete. 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 -- start start 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 Processor -- start 2024-02-17T12:51:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f5d60 0x555db8200750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 0x555db8200c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db82011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db8102bf0 con 0x555db81f6ad0 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db7e31250 con 0x555db82036f0 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db80e3200 con 0x555db81f5d60 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42f03a9740 10 client.0 _ll_register_callbacks cb 0x555db8151d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42e67fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 0x555db8200c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42e67fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 0x555db8200c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46578/0 (socket says 172.21.15.18:46578) 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.042+0000 7f42e67fc640 1 -- 172.21.15.18:0/4046326653 learned_addr learned my addr 172.21.15.18:0/4046326653 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f5d60 0x555db8200750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42e77fe640 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db82011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42e67fc640 1 -- 172.21.15.18:0/4046326653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f5d60 msgr2=0x555db8200750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42e67fc640 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f5d60 0x555db8200750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42e67fc640 1 -- 172.21.15.18:0/4046326653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 msgr2=0x555db82011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42e67fc640 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db82011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42e67fc640 1 -- 172.21.15.18:0/4046326653 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d80114c0 con 0x555db81f6ad0 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42e67fc640 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 0x555db8200c90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f42d00bead0 tx=0x7f42d008df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1047de4a7b86b363 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42ce7fc640 1 -- 172.21.15.18:0/4046326653 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00efea0 con 0x555db81f6ad0 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42ce7fc640 1 -- 172.21.15.18:0/4046326653 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f42d00bf4d0 con 0x555db81f6ad0 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42f03a9740 1 -- 172.21.15.18:0/4046326653 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555db7e5d600 con 0x555db81f6ad0 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42ce7fc640 1 -- 172.21.15.18:0/4046326653 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42d008a330 con 0x555db81f6ad0 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42f03a9740 10 client.16182 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42f03a9740 1 -- 172.21.15.18:0/4046326653 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555db815bd80 con 0x555db81f6ad0 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42ce7fc640 1 -- 172.21.15.18:0/4046326653 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d00d6020 con 0x555db81f6ad0 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.043+0000 7f42ce7fc640 1 -- 172.21.15.18:0/4046326653 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f42d00c4440 con 0x555db81f6ad0 2024-02-17T12:51:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.044+0000 7f42f03a9740 10 client.16182 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.044+0000 7f42f03a9740 10 client.16182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.044+0000 7f42f03a9740 20 client.16182 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.044+0000 7f42f03a9740 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555db82139d0 0x555db8215df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.044+0000 7f42f03a9740 4 client.16182 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.044+0000 7f42f03a9740 1 -- 172.21.15.18:0/4046326653 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555db7eae1b0 con 0x555db82139d0 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.044+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555db82139d0 0x555db8215df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.044+0000 7f42ce7fc640 10 client.16182 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.044+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555db82139d0 0x555db8215df0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c4398bf62b5dd27 server_cookie=494e007c1acce9a2 in_seq=0 out_seq=0 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42ce7fc640 1 -- 172.21.15.18:0/4046326653 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555db7eae1b0 con 0x555db82139d0 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42ce7fc640 10 client.16182 handle_command_reply: tid=0 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42f03a9740 1 client.16182 shutdown 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42f03a9740 1 -- 172.21.15.18:0/4046326653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 msgr2=0x555db8200c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42f03a9740 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 0x555db8200c90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f42d00bead0 tx=0x7f42d008df70 comp rx=0 tx=0).stop 2024-02-17T12:51:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42f03a9740 1 -- 172.21.15.18:0/4046326653 shutdown_connections 2024-02-17T12:51:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42f03a9740 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555db82139d0 0x555db8215df0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42f03a9740 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db82011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42f03a9740 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 0x555db8200c90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42ce7fc640 0 client.16182 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42f03a9740 1 --2- 172.21.15.18:0/4046326653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f5d60 0x555db8200750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42f03a9740 1 -- 172.21.15.18:0/4046326653 >> 172.21.15.18:0/4046326653 conn(0x555db81bd550 msgr2=0x555db81f27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42f03a9740 1 -- 172.21.15.18:0/4046326653 shutdown_connections 2024-02-17T12:51:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42f03a9740 1 -- 172.21.15.18:0/4046326653 wait complete. 2024-02-17T12:51:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.045+0000 7f42f03a9740 20 client.16182 trim_cache size 0 max 16384 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42f03a9740 1 Processor -- start 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42f03a9740 1 -- start start 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42f03a9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 0x555db8202e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42f03a9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f6ad0 0x555db8200ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42f03a9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db81fd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42f03a9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db821b000 con 0x555db81f5d60 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42f03a9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db81f5030 con 0x555db82036f0 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42f03a9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db81feca0 con 0x555db81f6ad0 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42e6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 0x555db8202e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42e6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 0x555db8202e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46590/0 (socket says 172.21.15.18:46590) 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42e6ffd640 1 -- 172.21.15.18:0/3685333886 learned_addr learned my addr 172.21.15.18:0/3685333886 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42e77fe640 1 --2- 172.21.15.18:0/3685333886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db81fd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.059+0000 7f42e67fc640 1 --2- 172.21.15.18:0/3685333886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f6ad0 0x555db8200ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.060+0000 7f42e6ffd640 1 -- 172.21.15.18:0/3685333886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f6ad0 msgr2=0x555db8200ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.060+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/3685333886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f6ad0 0x555db8200ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.060+0000 7f42e6ffd640 1 -- 172.21.15.18:0/3685333886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 msgr2=0x555db81fd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.060+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/3685333886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db81fd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.060+0000 7f42e6ffd640 1 -- 172.21.15.18:0/3685333886 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db80f87b0 con 0x555db81f5d60 2024-02-17T12:51:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.060+0000 7f42e77fe640 1 --2- 172.21.15.18:0/3685333886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db81fd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.060+0000 7f42e67fc640 1 --2- 172.21.15.18:0/3685333886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f6ad0 0x555db8200ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.060+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/3685333886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 0x555db8202e80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f42d8011860 tx=0x7f42d80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9252131a4b837d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.060+0000 7f42dcff9640 1 -- 172.21.15.18:0/3685333886 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d8003b70 con 0x555db81f5d60 2024-02-17T12:51:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.060+0000 7f42dcff9640 1 -- 172.21.15.18:0/3685333886 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d8020070 con 0x555db81f5d60 2024-02-17T12:51:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.060+0000 7f42dcff9640 1 -- 172.21.15.18:0/3685333886 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d80058d0 con 0x555db81f5d60 2024-02-17T12:51:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 -- 172.21.15.18:0/3685333886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 msgr2=0x555db8202e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 --2- 172.21.15.18:0/3685333886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 0x555db8202e80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f42d8011860 tx=0x7f42d80117b0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 -- 172.21.15.18:0/3685333886 shutdown_connections 2024-02-17T12:51:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 --2- 172.21.15.18:0/3685333886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db82036f0 0x555db81fd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 --2- 172.21.15.18:0/3685333886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db81f6ad0 0x555db8200ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 --2- 172.21.15.18:0/3685333886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f5d60 0x555db8202e80 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 -- 172.21.15.18:0/3685333886 >> 172.21.15.18:0/3685333886 conn(0x555db81bd550 msgr2=0x555db81ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 -- 172.21.15.18:0/3685333886 shutdown_connections 2024-02-17T12:51:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 -- 172.21.15.18:0/3685333886 wait complete. 2024-02-17T12:51:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 -- start start 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 Processor -- start 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db81f5d60 0x555db82096e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 0x555db8209c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db82036f0 0x555db8269ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db826a420 con 0x555db81f6ad0 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db826a560 con 0x555db81f5d60 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db826a6a0 con 0x555db82036f0 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.061+0000 7f42f03a9740 10 client.0 _ll_register_callbacks cb 0x7f42d8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42e67fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 0x555db8209c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42e6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db81f5d60 0x555db82096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42e67fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 0x555db8209c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46604/0 (socket says 172.21.15.18:46604) 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42e67fc640 1 -- 172.21.15.18:0/2131307364 learned_addr learned my addr 172.21.15.18:0/2131307364 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42e67fc640 1 -- 172.21.15.18:0/2131307364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db82036f0 msgr2=0x555db8269ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:51:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42e67fc640 1 --2- 172.21.15.18:0/2131307364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db82036f0 0x555db8269ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42e67fc640 1 -- 172.21.15.18:0/2131307364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db81f5d60 msgr2=0x555db82096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42e67fc640 1 --2- 172.21.15.18:0/2131307364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db81f5d60 0x555db82096e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42e67fc640 1 -- 172.21.15.18:0/2131307364 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555db815bd80 con 0x555db81f6ad0 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/2131307364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db81f5d60 0x555db82096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42e67fc640 1 --2- 172.21.15.18:0/2131307364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 0x555db8209c20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f42d00bb230 tx=0x7f42d00bf260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32df2a6ce738c1fd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42ce7fc640 1 -- 172.21.15.18:0/2131307364 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00c72e0 con 0x555db81f6ad0 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42ce7fc640 1 -- 172.21.15.18:0/2131307364 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f42d00d7030 con 0x555db81f6ad0 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42f03a9740 1 -- 172.21.15.18:0/2131307364 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42d8003e50 con 0x555db81f6ad0 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42ce7fc640 1 -- 172.21.15.18:0/2131307364 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42d00bf4d0 con 0x555db81f6ad0 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42f03a9740 10 client.16188 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42f03a9740 1 -- 172.21.15.18:0/2131307364 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f42e0012040 con 0x555db81f6ad0 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.062+0000 7f42ce7fc640 1 -- 172.21.15.18:0/2131307364 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d0088080 con 0x555db81f6ad0 2024-02-17T12:51:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.063+0000 7f42ce7fc640 1 -- 172.21.15.18:0/2131307364 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f42d008a130 con 0x555db81f6ad0 2024-02-17T12:51:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.063+0000 7f42f03a9740 10 client.16188 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.063+0000 7f42f03a9740 10 client.16188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.063+0000 7f42f03a9740 20 client.16188 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.063+0000 7f42f03a9740 1 --2- 172.21.15.18:0/2131307364 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555db826bc00 0x555db826e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.063+0000 7f42f03a9740 4 client.16188 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:51:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.063+0000 7f42f03a9740 1 -- 172.21.15.18:0/2131307364 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555db7e9a550 con 0x555db826bc00 2024-02-17T12:51:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.063+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/2131307364 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555db826bc00 0x555db826e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.063+0000 7f42e6ffd640 1 --2- 172.21.15.18:0/2131307364 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555db826bc00 0x555db826e020 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44731fafb27f440a server_cookie=1ce8c3adf76af99a in_seq=0 out_seq=0 2024-02-17T12:51:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.063+0000 7f42ce7fc640 10 client.16188 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:32.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:31 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:51:32.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:31 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:51:32.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:31 smithi119 ceph-mon[36188]: pgmap v52: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 2.2 MiB/s wr, 96 op/s 2024-02-17T12:51:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42ce7fc640 1 -- 172.21.15.18:0/2131307364 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555db7e9a550 con 0x555db826bc00 2024-02-17T12:51:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42ce7fc640 10 client.16188 handle_command_reply: tid=0 2024-02-17T12:51:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42f03a9740 1 client.16188 shutdown 2024-02-17T12:51:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42f03a9740 1 -- 172.21.15.18:0/2131307364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 msgr2=0x555db8209c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42f03a9740 1 --2- 172.21.15.18:0/2131307364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 0x555db8209c20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f42d00bb230 tx=0x7f42d00bf260 comp rx=0 tx=0).stop 2024-02-17T12:51:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42f03a9740 1 -- 172.21.15.18:0/2131307364 shutdown_connections 2024-02-17T12:51:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42f03a9740 1 --2- 172.21.15.18:0/2131307364 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555db826bc00 0x555db826e020 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42f03a9740 1 --2- 172.21.15.18:0/2131307364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db82036f0 0x555db8269ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42f03a9740 1 --2- 172.21.15.18:0/2131307364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db81f6ad0 0x555db8209c20 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42ce7fc640 0 client.16188 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42f03a9740 1 --2- 172.21.15.18:0/2131307364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db81f5d60 0x555db82096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42f03a9740 1 -- 172.21.15.18:0/2131307364 >> 172.21.15.18:0/2131307364 conn(0x555db81bd550 msgr2=0x555db81ef300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42f03a9740 1 -- 172.21.15.18:0/2131307364 shutdown_connections 2024-02-17T12:51:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42f03a9740 1 -- 172.21.15.18:0/2131307364 wait complete. 2024-02-17T12:51:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.102+0000 7f42f03a9740 20 client.16188 trim_cache size 0 max 16384 2024-02-17T12:51:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:51:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:51:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:51:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:51:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.103+0000 7f42ed7d7640 1 -- 172.21.15.18:0/1616049169 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42d40a22d0 msgr2=0x7f42d40a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.103+0000 7f42ed7d7640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42d40a22d0 0x7f42d40a4790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42e0012690 tx=0x7f42e0009660 comp rx=0 tx=0).stop 2024-02-17T12:51:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.103+0000 7f42ed7d7640 1 -- 172.21.15.18:0/1616049169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42e8100c90 msgr2=0x7f42e8196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.103+0000 7f42ed7d7640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42e8100c90 0x7f42e8196ba0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f42d0005940 tx=0x7f42d002c670 comp rx=0 tx=0).stop 2024-02-17T12:51:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.103+0000 7f42ed7d7640 1 -- 172.21.15.18:0/1616049169 shutdown_connections 2024-02-17T12:51:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.103+0000 7f42ed7d7640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42d40a22d0 0x7f42d40a4790 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.103+0000 7f42ed7d7640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e8105020 0x7f42e819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.103+0000 7f42ed7d7640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42e8100c90 0x7f42e8196ba0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.103+0000 7f42ed7d7640 1 --2- 172.21.15.18:0/1616049169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42e81002d0 0x7f42e8196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.103+0000 7f42ed7d7640 1 -- 172.21.15.18:0/1616049169 >> 172.21.15.18:0/1616049169 conn(0x7f42e8076ed0 msgr2=0x7f42e80fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.103+0000 7f42ed7d7640 1 -- 172.21.15.18:0/1616049169 shutdown_connections 2024-02-17T12:51:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:32.103+0000 7f42ed7d7640 1 -- 172.21.15.18:0/1616049169 wait complete. 2024-02-17T12:51:32.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:31 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:51:32.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:31 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:51:32.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:31 smithi146 ceph-mon[36454]: pgmap v52: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 2.2 MiB/s wr, 96 op/s 2024-02-17T12:51:32.261 INFO:tasks.workunit.client.0.smithi018.stdout: 1 53626 6.55 MB/sec warmup 47 sec latency 84.683 ms 2024-02-17T12:51:32.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:31 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:51:32.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:31 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:51:32.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:31 smithi018 ceph-mon[32625]: pgmap v52: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 2.2 MiB/s wr, 96 op/s 2024-02-17T12:51:33.261 INFO:tasks.workunit.client.0.smithi018.stdout: 1 54110 6.43 MB/sec warmup 48 sec latency 638.999 ms 2024-02-17T12:51:34.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:33 smithi119 ceph-mon[36188]: pgmap v53: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 2.2 MiB/s wr, 96 op/s 2024-02-17T12:51:34.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:33 smithi146 ceph-mon[36454]: pgmap v53: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 2.2 MiB/s wr, 96 op/s 2024-02-17T12:51:34.261 INFO:tasks.workunit.client.0.smithi018.stdout: 1 54110 6.29 MB/sec warmup 49 sec latency 1639.071 ms 2024-02-17T12:51:34.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:33 smithi018 ceph-mon[32625]: pgmap v53: 97 pgs: 97 active+clean; 47 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 2.2 MiB/s wr, 96 op/s 2024-02-17T12:51:35.261 INFO:tasks.workunit.client.0.smithi018.stdout: 1 54110 6.17 MB/sec warmup 50 sec latency 2639.138 ms 2024-02-17T12:51:36.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:35 smithi119 ceph-mon[36188]: pgmap v54: 97 pgs: 97 active+clean; 29 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 3.1 MiB/s wr, 198 op/s 2024-02-17T12:51:36.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:35 smithi146 ceph-mon[36454]: pgmap v54: 97 pgs: 97 active+clean; 29 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 3.1 MiB/s wr, 198 op/s 2024-02-17T12:51:36.262 INFO:tasks.workunit.client.0.smithi018.stdout: 1 54216 6.05 MB/sec warmup 51 sec latency 3206.681 ms 2024-02-17T12:51:36.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:35 smithi018 ceph-mon[32625]: pgmap v54: 97 pgs: 97 active+clean; 29 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 3.1 MiB/s wr, 198 op/s 2024-02-17T12:51:37.262 INFO:tasks.workunit.client.0.smithi018.stdout: 1 54216 5.94 MB/sec warmup 52 sec latency 1356.264 ms 2024-02-17T12:51:38.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:37 smithi119 ceph-mon[36188]: pgmap v55: 97 pgs: 97 active+clean; 29 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 2.0 MiB/s wr, 131 op/s 2024-02-17T12:51:38.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:37 smithi146 ceph-mon[36454]: pgmap v55: 97 pgs: 97 active+clean; 29 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 2.0 MiB/s wr, 131 op/s 2024-02-17T12:51:38.262 INFO:tasks.workunit.client.0.smithi018.stdout: 1 54216 5.82 MB/sec warmup 53 sec latency 2356.385 ms 2024-02-17T12:51:38.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:37 smithi018 ceph-mon[32625]: pgmap v55: 97 pgs: 97 active+clean; 29 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 2.0 MiB/s wr, 131 op/s 2024-02-17T12:51:39.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 54216 5.72 MB/sec warmup 54 sec latency 3356.481 ms 2024-02-17T12:51:40.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:39 smithi119 ceph-mon[36188]: pgmap v56: 97 pgs: 97 active+clean; 31 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.2 MiB/s wr, 145 op/s 2024-02-17T12:51:40.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:39 smithi146 ceph-mon[36454]: pgmap v56: 97 pgs: 97 active+clean; 31 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.2 MiB/s wr, 145 op/s 2024-02-17T12:51:40.262 INFO:tasks.workunit.client.0.smithi018.stdout: 1 54216 5.61 MB/sec warmup 55 sec latency 4356.593 ms 2024-02-17T12:51:40.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:39 smithi018 ceph-mon[32625]: pgmap v56: 97 pgs: 97 active+clean; 31 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.2 MiB/s wr, 145 op/s 2024-02-17T12:51:41.262 INFO:tasks.workunit.client.0.smithi018.stdout: 1 55679 5.62 MB/sec warmup 56 sec latency 4926.153 ms 2024-02-17T12:51:42.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:41 smithi119 ceph-mon[36188]: pgmap v57: 97 pgs: 97 active+clean; 31 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-02-17T12:51:42.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:41 smithi146 ceph-mon[36454]: pgmap v57: 97 pgs: 97 active+clean; 31 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-02-17T12:51:42.262 INFO:tasks.workunit.client.0.smithi018.stdout: 1 57503 5.74 MB/sec warmup 57 sec latency 10.372 ms 2024-02-17T12:51:42.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:41 smithi018 ceph-mon[32625]: pgmap v57: 97 pgs: 97 active+clean; 31 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-02-17T12:51:43.262 INFO:tasks.workunit.client.0.smithi018.stdout: 1 57571 5.64 MB/sec warmup 58 sec latency 963.930 ms 2024-02-17T12:51:44.076 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:43 smithi018 ceph-mon[32625]: pgmap v58: 97 pgs: 97 active+clean; 31 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-02-17T12:51:44.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:43 smithi119 ceph-mon[36188]: pgmap v58: 97 pgs: 97 active+clean; 31 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-02-17T12:51:44.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:43 smithi146 ceph-mon[36454]: pgmap v58: 97 pgs: 97 active+clean; 31 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-02-17T12:51:44.262 INFO:tasks.workunit.client.0.smithi018.stdout: 1 57571 5.55 MB/sec warmup 59 sec latency 1963.985 ms 2024-02-17T12:51:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.200+0000 7f42c72b5640 1 -- 172.21.15.18:0/2773298734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c0102c80 msgr2=0x7f42c010b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.200+0000 7f42c72b5640 1 --2- 172.21.15.18:0/2773298734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c0102c80 0x7f42c010b160 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009900 tx=0x7f42b002bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.200+0000 7f42c72b5640 1 -- 172.21.15.18:0/2773298734 shutdown_connections 2024-02-17T12:51:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.200+0000 7f42c72b5640 1 --2- 172.21.15.18:0/2773298734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c010b7d0 0x7f42c010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.200+0000 7f42c72b5640 1 --2- 172.21.15.18:0/2773298734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c0102c80 0x7f42c010b160 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.200+0000 7f42c72b5640 1 --2- 172.21.15.18:0/2773298734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c0102360 0x7f42c0102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.200+0000 7f42c72b5640 1 -- 172.21.15.18:0/2773298734 >> 172.21.15.18:0/2773298734 conn(0x7f42c00771b0 msgr2=0x7f42c00775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.201+0000 7f42c72b5640 1 -- 172.21.15.18:0/2773298734 shutdown_connections 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.201+0000 7f42c72b5640 1 -- 172.21.15.18:0/2773298734 wait complete. 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.201+0000 7f42c72b5640 1 Processor -- start 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.201+0000 7f42c72b5640 1 -- start start 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.201+0000 7f42c72b5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c0102360 0x7f42c01976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.201+0000 7f42c72b5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c0102c80 0x7f42c0197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.201+0000 7f42c72b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c010b7d0 0x7f42c019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.201+0000 7f42c72b5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c019c560 con 0x7f42c0102360 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.201+0000 7f42c72b5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c019c6d0 con 0x7f42c010b7d0 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.201+0000 7f42c72b5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c019c890 con 0x7f42c0102c80 2024-02-17T12:51:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.201+0000 7f42c5051640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c0102360 0x7f42c01976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42c5051640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c0102360 0x7f42c01976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52300/0 (socket says 172.21.15.18:52300) 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42c5051640 1 -- 172.21.15.18:0/2438444515 learned_addr learned my addr 172.21.15.18:0/2438444515 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42c4850640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c0102c80 0x7f42c0197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42c5852640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c010b7d0 0x7f42c019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42c5051640 1 -- 172.21.15.18:0/2438444515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c0102c80 msgr2=0x7f42c0197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42c5051640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c0102c80 0x7f42c0197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42c5051640 1 -- 172.21.15.18:0/2438444515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c010b7d0 msgr2=0x7f42c019bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42c5051640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c010b7d0 0x7f42c019bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42c5051640 1 -- 172.21.15.18:0/2438444515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b0009560 con 0x7f42c0102360 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42c5051640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c0102360 0x7f42c01976f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00d950 tx=0x7f42bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42b67fc640 1 -- 172.21.15.18:0/2438444515 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc009c30 con 0x7f42c0102360 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42c72b5640 1 -- 172.21.15.18:0/2438444515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c019cb70 con 0x7f42c0102360 2024-02-17T12:51:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42b67fc640 1 -- 172.21.15.18:0/2438444515 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42bc009d90 con 0x7f42c0102360 2024-02-17T12:51:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42b67fc640 1 -- 172.21.15.18:0/2438444515 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00f7e0 con 0x7f42c0102360 2024-02-17T12:51:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.202+0000 7f42c72b5640 1 -- 172.21.15.18:0/2438444515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c01a43f0 con 0x7f42c0102360 2024-02-17T12:51:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.204+0000 7f42b67fc640 1 -- 172.21.15.18:0/2438444515 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f42bc010ad0 con 0x7f42c0102360 2024-02-17T12:51:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.205+0000 7f42c9f63740 1 Processor -- start 2024-02-17T12:51:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.205+0000 7f42c9f63740 1 -- start start 2024-02-17T12:51:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.205+0000 7f42b67fc640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42ac0a22d0 0x7f42ac0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.205+0000 7f42c9f63740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb0ad0 0x563670faf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.205+0000 7f42c9f63740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fafd60 0x563670fb01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.205+0000 7f42c9f63740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb92e0 0x563670fbb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.205+0000 7f42c9f63740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563670f1e9f0 con 0x563670fb92e0 2024-02-17T12:51:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.205+0000 7f42c9f63740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563670f1bf20 con 0x563670fafd60 2024-02-17T12:51:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.205+0000 7f42c9f63740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563670f17700 con 0x563670fb0ad0 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.205+0000 7f42c5852640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42ac0a22d0 0x7f42ac0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.205+0000 7f42b67fc640 1 -- 172.21.15.18:0/2438444515 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f42bc0cdc40 con 0x7f42c0102360 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.205+0000 7f42c5852640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb92e0 0x563670fbb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42c4850640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fafd60 0x563670fb01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42c5852640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb92e0 0x563670fbb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52304/0 (socket says 172.21.15.18:52304) 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42c5852640 1 -- 172.21.15.18:0/2504181368 learned_addr learned my addr 172.21.15.18:0/2504181368 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42c5051640 1 --2- 172.21.15.18:0/2504181368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb0ad0 0x563670faf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42c5852640 1 -- 172.21.15.18:0/2504181368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb0ad0 msgr2=0x563670faf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42c5852640 1 --2- 172.21.15.18:0/2504181368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb0ad0 0x563670faf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42c5852640 1 -- 172.21.15.18:0/2504181368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fafd60 msgr2=0x563670fb01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42c5852640 1 --2- 172.21.15.18:0/2504181368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fafd60 0x563670fb01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42c5852640 1 -- 172.21.15.18:0/2504181368 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563670ead7b0 con 0x563670fb92e0 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42c4850640 1 --2- 172.21.15.18:0/2504181368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fafd60 0x563670fb01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42c5852640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42ac0a22d0 0x7f42ac0a4790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009fd0 tx=0x7f42b8009f00 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42c5852640 1 --2- 172.21.15.18:0/2504181368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb92e0 0x563670fbb6d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f42b800fd90 tx=0x7f42b801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f6e38c77d167a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42ab7fe640 1 -- 172.21.15.18:0/2504181368 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b800ebe0 con 0x563670fb92e0 2024-02-17T12:51:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42ab7fe640 1 -- 172.21.15.18:0/2504181368 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42b800ed40 con 0x563670fb92e0 2024-02-17T12:51:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.206+0000 7f42ab7fe640 1 -- 172.21.15.18:0/2504181368 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b80296a0 con 0x563670fb92e0 2024-02-17T12:51:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.207+0000 7f42c9f63740 1 -- 172.21.15.18:0/2504181368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb92e0 msgr2=0x563670fbb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.207+0000 7f42c9f63740 1 --2- 172.21.15.18:0/2504181368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb92e0 0x563670fbb6d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f42b800fd90 tx=0x7f42b801d000 comp rx=0 tx=0).stop 2024-02-17T12:51:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.207+0000 7f42c9f63740 1 -- 172.21.15.18:0/2504181368 shutdown_connections 2024-02-17T12:51:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.207+0000 7f42c9f63740 1 --2- 172.21.15.18:0/2504181368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb92e0 0x563670fbb6d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.207+0000 7f42c9f63740 1 --2- 172.21.15.18:0/2504181368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fafd60 0x563670fb01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.207+0000 7f42c9f63740 1 --2- 172.21.15.18:0/2504181368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb0ad0 0x563670faf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.207+0000 7f42c9f63740 1 -- 172.21.15.18:0/2504181368 >> 172.21.15.18:0/2504181368 conn(0x563670f77550 msgr2=0x563670faaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.207+0000 7f42c9f63740 1 -- 172.21.15.18:0/2504181368 shutdown_connections 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.207+0000 7f42c9f63740 1 -- 172.21.15.18:0/2504181368 wait complete. 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.207+0000 7f42c9f63740 1 -- start start 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.207+0000 7f42c9f63740 1 Processor -- start 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c9f63740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fafd60 0x563670fb6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c9f63740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 0x563670fb6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c9f63740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb92e0 0x563670fb6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c9f63740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563670ebcbf0 con 0x563670fb0ad0 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c9f63740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563670beb250 con 0x563670fafd60 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c9f63740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563670e9d200 con 0x563670fb92e0 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c9f63740 10 client.0 _ll_register_callbacks cb 0x563670f0bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c4850640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 0x563670fb6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c4850640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 0x563670fb6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52310/0 (socket says 172.21.15.18:52310) 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c4850640 1 -- 172.21.15.18:0/1366471748 learned_addr learned my addr 172.21.15.18:0/1366471748 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c5051640 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fafd60 0x563670fb6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c5852640 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb92e0 0x563670fb6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c4850640 1 -- 172.21.15.18:0/1366471748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb92e0 msgr2=0x563670fb6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c4850640 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb92e0 0x563670fb6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c4850640 1 -- 172.21.15.18:0/1366471748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fafd60 msgr2=0x563670fb6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c4850640 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fafd60 0x563670fb6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c4850640 1 -- 172.21.15.18:0/1366471748 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b800fb50 con 0x563670fb0ad0 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c5051640 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fafd60 0x563670fb6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.208+0000 7f42c5852640 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb92e0 0x563670fb6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42c4850640 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 0x563670fb6860 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f42b0006010 tx=0x7f42b002ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40d4e241ad86c68 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42a8ff9640 1 -- 172.21.15.18:0/1366471748 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b002cc00 con 0x563670fb0ad0 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42a8ff9640 1 -- 172.21.15.18:0/1366471748 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f42b0046070 con 0x563670fb0ad0 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42c9f63740 1 -- 172.21.15.18:0/1366471748 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563670c17600 con 0x563670fb0ad0 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42a8ff9640 1 -- 172.21.15.18:0/1366471748 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42b0004250 con 0x563670fb0ad0 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42c9f63740 10 client.16200 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42c9f63740 1 -- 172.21.15.18:0/1366471748 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563670f15d80 con 0x563670fb0ad0 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42a8ff9640 1 -- 172.21.15.18:0/1366471748 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563670c17600 con 0x563670fb0ad0 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42a8ff9640 1 -- 172.21.15.18:0/1366471748 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f42b003c040 con 0x563670fb0ad0 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42c9f63740 10 client.16200 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42c9f63740 10 client.16200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42c9f63740 20 client.16200 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42c9f63740 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563670fc96b0 0x563670fcbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42c9f63740 4 client.16200 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.209+0000 7f42c9f63740 1 -- 172.21.15.18:0/1366471748 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563670c681b0 con 0x563670fc96b0 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.210+0000 7f42c5051640 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563670fc96b0 0x563670fcbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.210+0000 7f42a8ff9640 10 client.16200 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.210+0000 7f42c5051640 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563670fc96b0 0x563670fcbad0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99913c3bb607259 server_cookie=cc5cab787e8b9c72 in_seq=0 out_seq=0 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42a8ff9640 1 -- 172.21.15.18:0/1366471748 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563670c681b0 con 0x563670fc96b0 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42a8ff9640 10 client.16200 handle_command_reply: tid=0 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42c9f63740 1 client.16200 shutdown 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42c9f63740 1 -- 172.21.15.18:0/1366471748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 msgr2=0x563670fb6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42c9f63740 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 0x563670fb6860 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f42b0006010 tx=0x7f42b002ca10 comp rx=0 tx=0).stop 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42c9f63740 1 -- 172.21.15.18:0/1366471748 shutdown_connections 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42c9f63740 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563670fc96b0 0x563670fcbad0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42c9f63740 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb92e0 0x563670fb6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42c9f63740 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 0x563670fb6860 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42c9f63740 1 --2- 172.21.15.18:0/1366471748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fafd60 0x563670fb6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42a8ff9640 0 client.16200 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42c9f63740 1 -- 172.21.15.18:0/1366471748 >> 172.21.15.18:0/1366471748 conn(0x563670f77550 msgr2=0x563670fac7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42c9f63740 1 -- 172.21.15.18:0/1366471748 shutdown_connections 2024-02-17T12:51:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42c9f63740 1 -- 172.21.15.18:0/1366471748 wait complete. 2024-02-17T12:51:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.211+0000 7f42c9f63740 20 client.16200 trim_cache size 0 max 16384 2024-02-17T12:51:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.224+0000 7f42c9f63740 1 Processor -- start 2024-02-17T12:51:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.224+0000 7f42c9f63740 1 -- start start 2024-02-17T12:51:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c9f63740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fafd60 0x563670fb8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c9f63740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 0x563670fc0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c9f63740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fb92e0 0x563670fb6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c9f63740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563670fc7de0 con 0x563670fb0ad0 2024-02-17T12:51:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c9f63740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563670fc9450 con 0x563670fb92e0 2024-02-17T12:51:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c9f63740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563670fafb10 con 0x563670fafd60 2024-02-17T12:51:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c4850640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 0x563670fc0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c4850640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 0x563670fc0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52320/0 (socket says 172.21.15.18:52320) 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c4850640 1 -- 172.21.15.18:0/3998393434 learned_addr learned my addr 172.21.15.18:0/3998393434 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c5852640 1 --2- 172.21.15.18:0/3998393434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fb92e0 0x563670fb6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c5051640 1 --2- 172.21.15.18:0/3998393434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fafd60 0x563670fb8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c4850640 1 -- 172.21.15.18:0/3998393434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fafd60 msgr2=0x563670fb8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c4850640 1 --2- 172.21.15.18:0/3998393434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fafd60 0x563670fb8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c4850640 1 -- 172.21.15.18:0/3998393434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fb92e0 msgr2=0x563670fb6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c4850640 1 --2- 172.21.15.18:0/3998393434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fb92e0 0x563670fb6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c4850640 1 -- 172.21.15.18:0/3998393434 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563670eb27b0 con 0x563670fb0ad0 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c5852640 1 --2- 172.21.15.18:0/3998393434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fb92e0 0x563670fb6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42c4850640 1 --2- 172.21.15.18:0/3998393434 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 0x563670fc0490 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f42b0004c50 tx=0x7f42b002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d945e8b79461d69 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42aaffd640 1 -- 172.21.15.18:0/3998393434 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b0003ed0 con 0x563670fb0ad0 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42aaffd640 1 -- 172.21.15.18:0/3998393434 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42b0044070 con 0x563670fb0ad0 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.225+0000 7f42aaffd640 1 -- 172.21.15.18:0/3998393434 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b0005530 con 0x563670fb0ad0 2024-02-17T12:51:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.226+0000 7f42c9f63740 1 -- 172.21.15.18:0/3998393434 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 msgr2=0x563670fc0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.226+0000 7f42c9f63740 1 --2- 172.21.15.18:0/3998393434 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 0x563670fc0490 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f42b0004c50 tx=0x7f42b002bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.226+0000 7f42c9f63740 1 -- 172.21.15.18:0/3998393434 shutdown_connections 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.226+0000 7f42c9f63740 1 --2- 172.21.15.18:0/3998393434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fb92e0 0x563670fb6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.226+0000 7f42c9f63740 1 --2- 172.21.15.18:0/3998393434 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fb0ad0 0x563670fc0490 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.226+0000 7f42c9f63740 1 --2- 172.21.15.18:0/3998393434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fafd60 0x563670fb8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.226+0000 7f42c9f63740 1 -- 172.21.15.18:0/3998393434 >> 172.21.15.18:0/3998393434 conn(0x563670f77550 msgr2=0x563670fd0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.226+0000 7f42c9f63740 1 -- 172.21.15.18:0/3998393434 shutdown_connections 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.226+0000 7f42c9f63740 1 -- 172.21.15.18:0/3998393434 wait complete. 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.226+0000 7f42c9f63740 1 -- start start 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.226+0000 7f42c9f63740 1 Processor -- start 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c9f63740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fafd60 0x56367101d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c9f63740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fb0ad0 0x56367101dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c9f63740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb92e0 0x563671017ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c9f63740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563671018160 con 0x563670fafd60 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c9f63740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636710182d0 con 0x563670fb0ad0 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c9f63740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563671018490 con 0x563670fb92e0 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c9f63740 10 client.0 _ll_register_callbacks cb 0x7f42b8010710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c5051640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fafd60 0x56367101d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c5051640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fafd60 0x56367101d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52328/0 (socket says 172.21.15.18:52328) 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c5051640 1 -- 172.21.15.18:0/3276909116 learned_addr learned my addr 172.21.15.18:0/3276909116 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c5852640 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb92e0 0x563671017ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c4850640 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fb0ad0 0x56367101dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c5051640 1 -- 172.21.15.18:0/3276909116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb92e0 msgr2=0x563671017ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c5051640 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb92e0 0x563671017ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c5051640 1 -- 172.21.15.18:0/3276909116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fb0ad0 msgr2=0x56367101dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c5051640 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fb0ad0 0x56367101dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c5051640 1 -- 172.21.15.18:0/3276909116 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563670f15d80 con 0x563670fafd60 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42c5051640 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fafd60 0x56367101d790 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f42bc097100 tx=0x7f42bc0a7910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9418e88f8e370b6f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.227+0000 7f42a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42a8ff9640 1 -- 172.21.15.18:0/3276909116 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0a2e20 con 0x563670fafd60 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42a8ff9640 1 -- 172.21.15.18:0/3276909116 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f42bc098a30 con 0x563670fafd60 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42c9f63740 1 -- 172.21.15.18:0/3276909116 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42b800ed40 con 0x563670fafd60 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42a8ff9640 1 -- 172.21.15.18:0/3276909116 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42bc0ad070 con 0x563670fafd60 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42c9f63740 10 client.16206 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42c9f63740 1 -- 172.21.15.18:0/3276909116 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563670ed8540 con 0x563670fafd60 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42a8ff9640 1 -- 172.21.15.18:0/3276909116 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42b800ed40 con 0x563670fafd60 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42a8ff9640 1 -- 172.21.15.18:0/3276909116 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f42bc0ba070 con 0x563670fafd60 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42c9f63740 10 client.16206 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42c9f63740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42c9f63740 20 client.16206 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42c9f63740 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563671025650 0x563671027a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42c9f63740 4 client.16206 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:51:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.228+0000 7f42c9f63740 1 -- 172.21.15.18:0/3276909116 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563670c54550 con 0x563671025650 2024-02-17T12:51:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.229+0000 7f42c4850640 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563671025650 0x563671027a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.229+0000 7f42c4850640 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563671025650 0x563671027a70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b027475404dd5c29 server_cookie=fee6f06d5bcd5fc1 in_seq=0 out_seq=0 2024-02-17T12:51:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.229+0000 7f42a8ff9640 10 client.16206 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42a8ff9640 1 -- 172.21.15.18:0/3276909116 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563670c54550 con 0x563671025650 2024-02-17T12:51:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42a8ff9640 10 client.16206 handle_command_reply: tid=0 2024-02-17T12:51:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c9f63740 1 client.16206 shutdown 2024-02-17T12:51:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c9f63740 1 -- 172.21.15.18:0/3276909116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fafd60 msgr2=0x56367101d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c9f63740 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fafd60 0x56367101d790 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f42bc097100 tx=0x7f42bc0a7910 comp rx=0 tx=0).stop 2024-02-17T12:51:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c9f63740 1 -- 172.21.15.18:0/3276909116 shutdown_connections 2024-02-17T12:51:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c9f63740 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563671025650 0x563671027a70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c9f63740 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563670fb92e0 0x563671017ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42a8ff9640 0 client.16206 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c9f63740 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563670fb0ad0 0x56367101dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c9f63740 1 --2- 172.21.15.18:0/3276909116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563670fafd60 0x56367101d790 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c9f63740 1 -- 172.21.15.18:0/3276909116 >> 172.21.15.18:0/3276909116 conn(0x563670f77550 msgr2=0x563670fb4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c9f63740 1 -- 172.21.15.18:0/3276909116 shutdown_connections 2024-02-17T12:51:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c9f63740 1 -- 172.21.15.18:0/3276909116 wait complete. 2024-02-17T12:51:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c9f63740 20 client.16206 trim_cache size 0 max 16384 2024-02-17T12:51:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:51:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:51:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:51:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:51:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c72b5640 1 -- 172.21.15.18:0/2438444515 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42ac0a22d0 msgr2=0x7f42ac0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.260+0000 7f42c72b5640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42ac0a22d0 0x7f42ac0a4790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009fd0 tx=0x7f42b8009f00 comp rx=0 tx=0).stop 2024-02-17T12:51:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.261+0000 7f42c72b5640 1 -- 172.21.15.18:0/2438444515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c0102360 msgr2=0x7f42c01976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.261+0000 7f42c72b5640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c0102360 0x7f42c01976f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00d950 tx=0x7f42bc00de10 comp rx=0 tx=0).stop 2024-02-17T12:51:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.261+0000 7f42c72b5640 1 -- 172.21.15.18:0/2438444515 shutdown_connections 2024-02-17T12:51:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.261+0000 7f42c72b5640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42ac0a22d0 0x7f42ac0a4790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.261+0000 7f42c72b5640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c010b7d0 0x7f42c019bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.261+0000 7f42c72b5640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42c0102c80 0x7f42c0197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.261+0000 7f42c72b5640 1 --2- 172.21.15.18:0/2438444515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c0102360 0x7f42c01976f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.261+0000 7f42c72b5640 1 -- 172.21.15.18:0/2438444515 >> 172.21.15.18:0/2438444515 conn(0x7f42c00771b0 msgr2=0x7f42c00fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.261+0000 7f42c72b5640 1 -- 172.21.15.18:0/2438444515 shutdown_connections 2024-02-17T12:51:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:45.261+0000 7f42c72b5640 1 -- 172.21.15.18:0/2438444515 wait complete. 2024-02-17T12:51:45.262 INFO:tasks.workunit.client.0.smithi018.stdout: 1 57571 5.46 MB/sec warmup 60 sec latency 2964.028 ms 2024-02-17T12:51:46.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:45 smithi119 ceph-mon[36188]: pgmap v59: 97 pgs: 97 active+clean; 30 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-02-17T12:51:46.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:45 smithi146 ceph-mon[36454]: pgmap v59: 97 pgs: 97 active+clean; 30 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-02-17T12:51:46.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 58840 5.45 MB/sec warmup 61 sec latency 3530.546 ms 2024-02-17T12:51:46.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:45 smithi018 ceph-mon[32625]: pgmap v59: 97 pgs: 97 active+clean; 30 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-02-17T12:51:47.262 INFO:tasks.workunit.client.0.smithi018.stdout: 1 60768 5.58 MB/sec warmup 62 sec latency 10.952 ms 2024-02-17T12:51:48.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:47 smithi119 ceph-mon[36188]: pgmap v60: 97 pgs: 97 active+clean; 30 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 432 KiB/s wr, 57 op/s 2024-02-17T12:51:48.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:47 smithi146 ceph-mon[36454]: pgmap v60: 97 pgs: 97 active+clean; 30 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 432 KiB/s wr, 57 op/s 2024-02-17T12:51:48.262 INFO:tasks.workunit.client.0.smithi018.stdout: 1 61119 5.50 MB/sec warmup 63 sec latency 761.487 ms 2024-02-17T12:51:48.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:47 smithi018 ceph-mon[32625]: pgmap v60: 97 pgs: 97 active+clean; 30 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 432 KiB/s wr, 57 op/s 2024-02-17T12:51:49.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 61119 5.41 MB/sec warmup 64 sec latency 1761.574 ms 2024-02-17T12:51:50.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:49 smithi119 ceph-mon[36188]: pgmap v61: 97 pgs: 97 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 768 KiB/s wr, 115 op/s 2024-02-17T12:51:50.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:49 smithi146 ceph-mon[36454]: pgmap v61: 97 pgs: 97 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 768 KiB/s wr, 115 op/s 2024-02-17T12:51:50.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 61119 5.33 MB/sec warmup 65 sec latency 2761.705 ms 2024-02-17T12:51:50.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:49 smithi018 ceph-mon[32625]: pgmap v61: 97 pgs: 97 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 768 KiB/s wr, 115 op/s 2024-02-17T12:51:51.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 61969 5.32 MB/sec warmup 66 sec latency 3330.341 ms 2024-02-17T12:51:52.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:51 smithi119 ceph-mon[36188]: pgmap v62: 97 pgs: 97 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 521 KiB/s wr, 101 op/s 2024-02-17T12:51:52.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:51 smithi146 ceph-mon[36454]: pgmap v62: 97 pgs: 97 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 521 KiB/s wr, 101 op/s 2024-02-17T12:51:52.262 INFO:tasks.workunit.client.0.smithi018.stdout: 1 64198 5.45 MB/sec warmup 67 sec latency 9.781 ms 2024-02-17T12:51:52.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:51 smithi018 ceph-mon[32625]: pgmap v62: 97 pgs: 97 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 521 KiB/s wr, 101 op/s 2024-02-17T12:51:52.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:51:52.947 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:51:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.060+0000 7f13cc8ae640 1 -- 172.21.15.18:0/4107929325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13c81060b0 msgr2=0x7f13c8106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.060+0000 7f13cc8ae640 1 --2- 172.21.15.18:0/4107929325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13c81060b0 0x7f13c8106490 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009900 tx=0x7f13c002bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.060+0000 7f13cc8ae640 1 -- 172.21.15.18:0/4107929325 shutdown_connections 2024-02-17T12:51:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.060+0000 7f13cc8ae640 1 --2- 172.21.15.18:0/4107929325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c8100c70 0x7f13c810ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.060+0000 7f13cc8ae640 1 --2- 172.21.15.18:0/4107929325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13c81002d0 0x7f13c8100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.060+0000 7f13cc8ae640 1 --2- 172.21.15.18:0/4107929325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13c81060b0 0x7f13c8106490 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.060+0000 7f13cc8ae640 1 -- 172.21.15.18:0/4107929325 >> 172.21.15.18:0/4107929325 conn(0x7f13c8076e20 msgr2=0x7f13c8077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.060+0000 7f13cc8ae640 1 -- 172.21.15.18:0/4107929325 shutdown_connections 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.060+0000 7f13cc8ae640 1 -- 172.21.15.18:0/4107929325 wait complete. 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.061+0000 7f13cc8ae640 1 Processor -- start 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.061+0000 7f13cc8ae640 1 -- start start 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.061+0000 7f13cc8ae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81002d0 0x7f13c810d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.061+0000 7f13cc8ae640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13c8100c70 0x7f13c810dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.061+0000 7f13cc8ae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13c81060b0 0x7f13c810e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.061+0000 7f13cc8ae640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c81a5b30 con 0x7f13c8100c70 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.061+0000 7f13cc8ae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c81a5ca0 con 0x7f13c81002d0 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.061+0000 7f13cc8ae640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c81a5e60 con 0x7f13c81060b0 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.061+0000 7f13c5d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13c8100c70 0x7f13c810dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13c5d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13c8100c70 0x7f13c810dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36106/0 (socket says 172.21.15.18:36106) 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13c5d9b640 1 -- 172.21.15.18:0/1790181252 learned_addr learned my addr 172.21.15.18:0/1790181252 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13c6d9d640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13c81060b0 0x7f13c810e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13c659c640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81002d0 0x7f13c810d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13c5d9b640 1 -- 172.21.15.18:0/1790181252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13c81060b0 msgr2=0x7f13c810e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13c5d9b640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13c81060b0 0x7f13c810e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13c5d9b640 1 -- 172.21.15.18:0/1790181252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81002d0 msgr2=0x7f13c810d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13c5d9b640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81002d0 0x7f13c810d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13c5d9b640 1 -- 172.21.15.18:0/1790181252 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c0009560 con 0x7f13c8100c70 2024-02-17T12:51:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13c5d9b640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13c8100c70 0x7f13c810dc20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f13b000d770 tx=0x7f13b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13bf7fe640 1 -- 172.21.15.18:0/1790181252 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0009c30 con 0x7f13c8100c70 2024-02-17T12:51:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13bf7fe640 1 -- 172.21.15.18:0/1790181252 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13b000de80 con 0x7f13c8100c70 2024-02-17T12:51:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13cc8ae640 1 -- 172.21.15.18:0/1790181252 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c81a6140 con 0x7f13c8100c70 2024-02-17T12:51:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13bf7fe640 1 -- 172.21.15.18:0/1790181252 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0004e80 con 0x7f13c8100c70 2024-02-17T12:51:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.062+0000 7f13cc8ae640 1 -- 172.21.15.18:0/1790181252 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c81a6600 con 0x7f13c8100c70 2024-02-17T12:51:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.064+0000 7f13bf7fe640 1 -- 172.21.15.18:0/1790181252 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f13b0005020 con 0x7f13c8100c70 2024-02-17T12:51:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13cf482740 1 Processor -- start 2024-02-17T12:51:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13cf482740 1 -- start start 2024-02-17T12:51:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13bf7fe640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13b40a2230 0x7f13b40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13cf482740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481a2b0 0x555b2481a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13cf482740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b24823270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13cf482740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b248237b0 0x555b24825ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13cf482740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b24789280 con 0x555b2481a2b0 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13cf482740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b247867b0 con 0x555b248237b0 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13cf482740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b24781f90 con 0x555b2481abd0 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13c6d9d640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13b40a2230 0x7f13b40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13bf7fe640 1 -- 172.21.15.18:0/1790181252 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f13b00cda60 con 0x7f13c8100c70 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13c659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481a2b0 0x555b2481a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13c5d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b24823270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13c6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b248237b0 0x555b24825ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13c5d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b24823270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:38806/0 (socket says 172.21.15.18:38806) 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.065+0000 7f13c5d9b640 1 -- 172.21.15.18:0/2641083183 learned_addr learned my addr 172.21.15.18:0/2641083183 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.066+0000 7f13c659c640 1 -- 172.21.15.18:0/2641083183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 msgr2=0x555b24823270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.066+0000 7f13c659c640 1 --2- 172.21.15.18:0/2641083183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b24823270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.066+0000 7f13c659c640 1 -- 172.21.15.18:0/2641083183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b248237b0 msgr2=0x555b24825ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.066+0000 7f13c659c640 1 --2- 172.21.15.18:0/2641083183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b248237b0 0x555b24825ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.066+0000 7f13c659c640 1 -- 172.21.15.18:0/2641083183 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b24740570 con 0x555b2481a2b0 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.066+0000 7f13c5d9b640 1 --2- 172.21.15.18:0/2641083183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b24823270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.066+0000 7f13c6d9d640 1 --2- 172.21.15.18:0/2641083183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b248237b0 0x555b24825ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.066+0000 7f13c6d9d640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13b40a2230 0x7f13b40a46f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f13b8005fd0 tx=0x7f13b8009e60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.066+0000 7f13c659c640 1 --2- 172.21.15.18:0/2641083183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481a2b0 0x555b2481a690 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f13c0000c00 tx=0x7f13c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cdf3ee315be676 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.066+0000 7f13bcff9640 1 -- 172.21.15.18:0/2641083183 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0004970 con 0x555b2481a2b0 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.066+0000 7f13bcff9640 1 -- 172.21.15.18:0/2641083183 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13c0004ad0 con 0x555b2481a2b0 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.066+0000 7f13bcff9640 1 -- 172.21.15.18:0/2641083183 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0005210 con 0x555b2481a2b0 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.067+0000 7f13cf482740 1 -- 172.21.15.18:0/2641083183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481a2b0 msgr2=0x555b2481a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.067+0000 7f13cf482740 1 --2- 172.21.15.18:0/2641083183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481a2b0 0x555b2481a690 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f13c0000c00 tx=0x7f13c00046c0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.067+0000 7f13cf482740 1 -- 172.21.15.18:0/2641083183 shutdown_connections 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.067+0000 7f13cf482740 1 --2- 172.21.15.18:0/2641083183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b248237b0 0x555b24825ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.067+0000 7f13cf482740 1 --2- 172.21.15.18:0/2641083183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b24823270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.067+0000 7f13cf482740 1 --2- 172.21.15.18:0/2641083183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481a2b0 0x555b2481a690 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.067+0000 7f13cf482740 1 -- 172.21.15.18:0/2641083183 >> 172.21.15.18:0/2641083183 conn(0x555b24814990 msgr2=0x555b24814d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.067+0000 7f13cf482740 1 -- 172.21.15.18:0/2641083183 shutdown_connections 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.067+0000 7f13cf482740 1 -- 172.21.15.18:0/2641083183 wait complete. 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.067+0000 7f13cf482740 1 -- start start 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.067+0000 7f13cf482740 1 Processor -- start 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13cf482740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481a2b0 0x555b24823150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13cf482740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481abd0 0x555b248204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13cf482740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b248237b0 0x555b24820a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13cf482740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b24455430 con 0x555b2481abd0 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13cf482740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b24411e00 con 0x555b248237b0 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13cf482740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b24706fd0 con 0x555b2481a2b0 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13cf482740 10 client.0 _ll_register_callbacks cb 0x555b2475cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13c5d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481abd0 0x555b248204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13c659c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481a2b0 0x555b24823150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13c5d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481abd0 0x555b248204e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36112/0 (socket says 172.21.15.18:36112) 2024-02-17T12:51:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13c5d9b640 1 -- 172.21.15.18:0/4082295484 learned_addr learned my addr 172.21.15.18:0/4082295484 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13c6d9d640 1 --2- 172.21.15.18:0/4082295484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b248237b0 0x555b24820a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13c5d9b640 1 -- 172.21.15.18:0/4082295484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481a2b0 msgr2=0x555b24823150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13c5d9b640 1 --2- 172.21.15.18:0/4082295484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481a2b0 0x555b24823150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13c5d9b640 1 -- 172.21.15.18:0/4082295484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b248237b0 msgr2=0x555b24820a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13c5d9b640 1 --2- 172.21.15.18:0/4082295484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b248237b0 0x555b24820a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13c5d9b640 1 -- 172.21.15.18:0/4082295484 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c8069970 con 0x555b2481abd0 2024-02-17T12:51:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13c5d9b640 1 --2- 172.21.15.18:0/4082295484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481abd0 0x555b248204e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f13b0096900 tx=0x7f13b009a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d9324e238926ba server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13adffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13adffb640 1 -- 172.21.15.18:0/4082295484 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b009a2c0 con 0x555b2481abd0 2024-02-17T12:51:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13adffb640 1 -- 172.21.15.18:0/4082295484 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f13b0012690 con 0x555b2481abd0 2024-02-17T12:51:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13cf482740 1 -- 172.21.15.18:0/4082295484 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b244817e0 con 0x555b2481abd0 2024-02-17T12:51:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13adffb640 1 -- 172.21.15.18:0/4082295484 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13b009a4f0 con 0x555b2481abd0 2024-02-17T12:51:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.068+0000 7f13cf482740 10 client.16218 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.069+0000 7f13cf482740 1 -- 172.21.15.18:0/4082295484 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555b24780610 con 0x555b2481abd0 2024-02-17T12:51:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.069+0000 7f13adffb640 1 -- 172.21.15.18:0/4082295484 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555b244817e0 con 0x555b2481abd0 2024-02-17T12:51:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.069+0000 7f13adffb640 1 -- 172.21.15.18:0/4082295484 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f13b00120a0 con 0x555b2481abd0 2024-02-17T12:51:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.069+0000 7f13cf482740 10 client.16218 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.069+0000 7f13cf482740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.069+0000 7f13cf482740 20 client.16218 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.069+0000 7f13cf482740 1 --2- 172.21.15.18:0/4082295484 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555b24833880 0x555b24835ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.069+0000 7f13cf482740 4 client.16218 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.069+0000 7f13cf482740 1 -- 172.21.15.18:0/4082295484 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b244be720 con 0x555b24833880 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.069+0000 7f13c659c640 1 --2- 172.21.15.18:0/4082295484 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555b24833880 0x555b24835ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.069+0000 7f13adffb640 10 client.16218 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.069+0000 7f13c659c640 1 --2- 172.21.15.18:0/4082295484 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555b24833880 0x555b24835ca0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0050ffd78ab6f0 server_cookie=be1f102f5a45cc70 in_seq=0 out_seq=0 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13adffb640 1 -- 172.21.15.18:0/4082295484 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555b244be720 con 0x555b24833880 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13adffb640 10 client.16218 handle_command_reply: tid=0 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13cf482740 1 client.16218 shutdown 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13cf482740 1 -- 172.21.15.18:0/4082295484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481abd0 msgr2=0x555b248204e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13cf482740 1 --2- 172.21.15.18:0/4082295484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481abd0 0x555b248204e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f13b0096900 tx=0x7f13b009a140 comp rx=0 tx=0).stop 2024-02-17T12:51:53.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13cf482740 1 -- 172.21.15.18:0/4082295484 shutdown_connections 2024-02-17T12:51:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13cf482740 1 --2- 172.21.15.18:0/4082295484 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555b24833880 0x555b24835ca0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13cf482740 1 --2- 172.21.15.18:0/4082295484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b248237b0 0x555b24820a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13adffb640 0 client.16218 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13cf482740 1 --2- 172.21.15.18:0/4082295484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b2481abd0 0x555b248204e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13cf482740 1 --2- 172.21.15.18:0/4082295484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481a2b0 0x555b24823150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13cf482740 1 -- 172.21.15.18:0/4082295484 >> 172.21.15.18:0/4082295484 conn(0x555b24814990 msgr2=0x555b24816730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13cf482740 1 -- 172.21.15.18:0/4082295484 shutdown_connections 2024-02-17T12:51:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13cf482740 1 -- 172.21.15.18:0/4082295484 wait complete. 2024-02-17T12:51:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.070+0000 7f13cf482740 20 client.16218 trim_cache size 0 max 16384 2024-02-17T12:51:53.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.086+0000 7f13cf482740 1 Processor -- start 2024-02-17T12:51:53.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.086+0000 7f13cf482740 1 -- start start 2024-02-17T12:51:53.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.086+0000 7f13cf482740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b2481a2b0 0x555b24822c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.086+0000 7f13cf482740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b24823150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.086+0000 7f13cf482740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 0x555b2481edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.086+0000 7f13cf482740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b24830370 con 0x555b248237b0 2024-02-17T12:51:53.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13cf482740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b2480ee20 con 0x555b2481a2b0 2024-02-17T12:51:53.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13cf482740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b2482f540 con 0x555b2481abd0 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13c6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 0x555b2481edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13c6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 0x555b2481edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36122/0 (socket says 172.21.15.18:36122) 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13c6d9d640 1 -- 172.21.15.18:0/3136889122 learned_addr learned my addr 172.21.15.18:0/3136889122 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13c5d9b640 1 --2- 172.21.15.18:0/3136889122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b24823150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13c659c640 1 --2- 172.21.15.18:0/3136889122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b2481a2b0 0x555b24822c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13c6d9d640 1 -- 172.21.15.18:0/3136889122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 msgr2=0x555b24823150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13c6d9d640 1 --2- 172.21.15.18:0/3136889122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b24823150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13c6d9d640 1 -- 172.21.15.18:0/3136889122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b2481a2b0 msgr2=0x555b24822c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13c6d9d640 1 --2- 172.21.15.18:0/3136889122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b2481a2b0 0x555b24822c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13c6d9d640 1 -- 172.21.15.18:0/3136889122 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b2471cac0 con 0x555b248237b0 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13c6d9d640 1 --2- 172.21.15.18:0/3136889122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 0x555b2481edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f13b8012db0 tx=0x7f13b801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea15ae4a427801a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13affff640 1 -- 172.21.15.18:0/3136889122 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8015110 con 0x555b248237b0 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13affff640 1 -- 172.21.15.18:0/3136889122 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13b801b040 con 0x555b248237b0 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.087+0000 7f13affff640 1 -- 172.21.15.18:0/3136889122 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8022d80 con 0x555b248237b0 2024-02-17T12:51:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.088+0000 7f13cf482740 1 -- 172.21.15.18:0/3136889122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 msgr2=0x555b2481edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.088+0000 7f13cf482740 1 --2- 172.21.15.18:0/3136889122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 0x555b2481edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f13b8012db0 tx=0x7f13b801f000 comp rx=0 tx=0).stop 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.088+0000 7f13cf482740 1 -- 172.21.15.18:0/3136889122 shutdown_connections 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.088+0000 7f13cf482740 1 --2- 172.21.15.18:0/3136889122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 0x555b2481edb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.088+0000 7f13cf482740 1 --2- 172.21.15.18:0/3136889122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b24823150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.088+0000 7f13cf482740 1 --2- 172.21.15.18:0/3136889122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b2481a2b0 0x555b24822c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.088+0000 7f13cf482740 1 -- 172.21.15.18:0/3136889122 >> 172.21.15.18:0/3136889122 conn(0x555b24814990 msgr2=0x555b2481b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.088+0000 7f13cf482740 1 -- 172.21.15.18:0/3136889122 shutdown_connections 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.088+0000 7f13cf482740 1 -- 172.21.15.18:0/3136889122 wait complete. 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.088+0000 7f13cf482740 1 -- start start 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.088+0000 7f13cf482740 1 Processor -- start 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13cf482740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b2481a2b0 0x555b2481d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13cf482740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b2481d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13cf482740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 0x555b2481dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13cf482740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b24889e00 con 0x555b248237b0 2024-02-17T12:51:53.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13cf482740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b24889f70 con 0x555b2481a2b0 2024-02-17T12:51:53.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13cf482740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b2488a130 con 0x555b2481abd0 2024-02-17T12:51:53.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13cf482740 10 client.0 _ll_register_callbacks cb 0x7f13c0005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:53.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13c6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 0x555b2481dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13c6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 0x555b2481dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36136/0 (socket says 172.21.15.18:36136) 2024-02-17T12:51:53.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13c6d9d640 1 -- 172.21.15.18:0/1399388455 learned_addr learned my addr 172.21.15.18:0/1399388455 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13c659c640 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b2481a2b0 0x555b2481d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13c5d9b640 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b2481d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13c6d9d640 1 -- 172.21.15.18:0/1399388455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 msgr2=0x555b2481d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13c6d9d640 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b2481d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13c6d9d640 1 -- 172.21.15.18:0/1399388455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b2481a2b0 msgr2=0x555b2481d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13c6d9d640 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b2481a2b0 0x555b2481d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13c6d9d640 1 -- 172.21.15.18:0/1399388455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b8012b70 con 0x555b248237b0 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13c5d9b640 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b2481d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13c6d9d640 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 0x555b2481dd50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f13b80049d0 tx=0x7f13b8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527e2859e71f133b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13adffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13adffb640 1 -- 172.21.15.18:0/1399388455 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8022d00 con 0x555b248237b0 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13adffb640 1 -- 172.21.15.18:0/1399388455 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f13b8022480 con 0x555b248237b0 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13cf482740 1 -- 172.21.15.18:0/1399388455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13c0004ad0 con 0x555b248237b0 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13adffb640 1 -- 172.21.15.18:0/1399388455 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13b8022e60 con 0x555b248237b0 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13cf482740 10 client.16224 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.089+0000 7f13cf482740 1 -- 172.21.15.18:0/1399388455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555b247426a0 con 0x555b248237b0 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.090+0000 7f13adffb640 1 -- 172.21.15.18:0/1399388455 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13b8041020 con 0x555b248237b0 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.090+0000 7f13adffb640 1 -- 172.21.15.18:0/1399388455 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f13b80052b0 con 0x555b248237b0 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.090+0000 7f13cf482740 10 client.16224 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.090+0000 7f13cf482740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:53.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.090+0000 7f13cf482740 20 client.16224 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.090+0000 7f13cf482740 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555b2488b920 0x555b2488dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.090+0000 7f13cf482740 4 client.16224 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.090+0000 7f13cf482740 1 -- 172.21.15.18:0/1399388455 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555b2450d890 con 0x555b2488b920 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.090+0000 7f13c659c640 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555b2488b920 0x555b2488dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.090+0000 7f13c659c640 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555b2488b920 0x555b2488dd40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5513c8d5377eac15 server_cookie=bbaff2eeca2f53cc in_seq=0 out_seq=0 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.090+0000 7f13adffb640 10 client.16224 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13adffb640 1 -- 172.21.15.18:0/1399388455 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555b2450d890 con 0x555b2488b920 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13adffb640 10 client.16224 handle_command_reply: tid=0 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cf482740 1 client.16224 shutdown 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cf482740 1 -- 172.21.15.18:0/1399388455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 msgr2=0x555b2481dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cf482740 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 0x555b2481dd50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f13b80049d0 tx=0x7f13b8009800 comp rx=0 tx=0).stop 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cf482740 1 -- 172.21.15.18:0/1399388455 shutdown_connections 2024-02-17T12:51:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cf482740 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555b2488b920 0x555b2488dd40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cf482740 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555b248237b0 0x555b2481dd50 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cf482740 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b2481abd0 0x555b2481d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cf482740 1 --2- 172.21.15.18:0/1399388455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555b2481a2b0 0x555b2481d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13adffb640 0 client.16224 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cf482740 1 -- 172.21.15.18:0/1399388455 >> 172.21.15.18:0/1399388455 conn(0x555b24814990 msgr2=0x555b2481ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cf482740 1 -- 172.21.15.18:0/1399388455 shutdown_connections 2024-02-17T12:51:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cf482740 1 -- 172.21.15.18:0/1399388455 wait complete. 2024-02-17T12:51:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cf482740 20 client.16224 trim_cache size 0 max 16384 2024-02-17T12:51:53.090 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:51:53.090 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:51:53.091 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:51:53.091 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:51:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cc8ae640 1 -- 172.21.15.18:0/1790181252 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13b40a2230 msgr2=0x7f13b40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cc8ae640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13b40a2230 0x7f13b40a46f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f13b8005fd0 tx=0x7f13b8009e60 comp rx=0 tx=0).stop 2024-02-17T12:51:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cc8ae640 1 -- 172.21.15.18:0/1790181252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13c8100c70 msgr2=0x7f13c810dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cc8ae640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13c8100c70 0x7f13c810dc20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f13b000d770 tx=0x7f13b000dc30 comp rx=0 tx=0).stop 2024-02-17T12:51:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cc8ae640 1 -- 172.21.15.18:0/1790181252 shutdown_connections 2024-02-17T12:51:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cc8ae640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13b40a2230 0x7f13b40a46f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cc8ae640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13c81060b0 0x7f13c810e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cc8ae640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13c8100c70 0x7f13c810dc20 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cc8ae640 1 --2- 172.21.15.18:0/1790181252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13c81002d0 0x7f13c810d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cc8ae640 1 -- 172.21.15.18:0/1790181252 >> 172.21.15.18:0/1790181252 conn(0x7f13c8076e20 msgr2=0x7f13c80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.091+0000 7f13cc8ae640 1 -- 172.21.15.18:0/1790181252 shutdown_connections 2024-02-17T12:51:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.092+0000 7f13cc8ae640 1 -- 172.21.15.18:0/1790181252 wait complete. 2024-02-17T12:51:53.094 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:51:53.094 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:51:53.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.208+0000 7fee4f94f640 1 -- 172.21.15.18:0/1806440802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee48106450 msgr2=0x7fee4810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.208+0000 7fee4f94f640 1 --2- 172.21.15.18:0/1806440802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee48106450 0x7fee4810e020 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fee44009930 tx=0x7fee4402bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:53.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.208+0000 7fee4f94f640 1 -- 172.21.15.18:0/1806440802 shutdown_connections 2024-02-17T12:51:53.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.208+0000 7fee4f94f640 1 --2- 172.21.15.18:0/1806440802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee48106450 0x7fee4810e020 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.208+0000 7fee4f94f640 1 --2- 172.21.15.18:0/1806440802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee48101d30 0x7fee48105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.208+0000 7fee4f94f640 1 --2- 172.21.15.18:0/1806440802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee48101380 0x7fee48101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.208+0000 7fee4f94f640 1 -- 172.21.15.18:0/1806440802 >> 172.21.15.18:0/1806440802 conn(0x7fee480fcc20 msgr2=0x7fee480ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.208+0000 7fee4f94f640 1 -- 172.21.15.18:0/1806440802 shutdown_connections 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.208+0000 7fee4f94f640 1 -- 172.21.15.18:0/1806440802 wait complete. 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4f94f640 1 Processor -- start 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4f94f640 1 -- start start 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4f94f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee48101380 0x7fee4819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4f94f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee48106450 0x7fee4819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4f94f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee48197660 0x7fee48197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4f94f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee48198050 con 0x7fee48101380 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4f94f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee481981c0 con 0x7fee48106450 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4f94f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee48198380 con 0x7fee48197660 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4d6eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee48101380 0x7fee4819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4d6eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee48101380 0x7fee4819d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36150/0 (socket says 172.21.15.18:36150) 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4d6eb640 1 -- 172.21.15.18:0/3576442906 learned_addr learned my addr 172.21.15.18:0/3576442906 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4ceea640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee48106450 0x7fee4819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4deec640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee48197660 0x7fee48197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4d6eb640 1 -- 172.21.15.18:0/3576442906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee48197660 msgr2=0x7fee48197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4d6eb640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee48197660 0x7fee48197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4d6eb640 1 -- 172.21.15.18:0/3576442906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee48106450 msgr2=0x7fee4819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4d6eb640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee48106450 0x7fee4819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4d6eb640 1 -- 172.21.15.18:0/3576442906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee44009560 con 0x7fee48101380 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.209+0000 7fee4d6eb640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee48101380 0x7fee4819d460 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fee4000eab0 tx=0x7fee4000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.210+0000 7fee3e7fc640 1 -- 172.21.15.18:0/3576442906 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4000ce40 con 0x7fee48101380 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.210+0000 7fee3e7fc640 1 -- 172.21.15.18:0/3576442906 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee40011950 con 0x7fee48101380 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.210+0000 7fee4f94f640 1 -- 172.21.15.18:0/3576442906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee481a4320 con 0x7fee48101380 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.210+0000 7fee3e7fc640 1 -- 172.21.15.18:0/3576442906 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee40010aa0 con 0x7fee48101380 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.210+0000 7fee4f94f640 1 -- 172.21.15.18:0/3576442906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee481a4880 con 0x7fee48101380 2024-02-17T12:51:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.211+0000 7fee4f94f640 1 -- 172.21.15.18:0/3576442906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee480683c0 con 0x7fee48101380 2024-02-17T12:51:53.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.212+0000 7fee3e7fc640 1 -- 172.21.15.18:0/3576442906 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fee40011ac0 con 0x7fee48101380 2024-02-17T12:51:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.212+0000 7fee3e7fc640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fee340a22b0 0x7fee340a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.212+0000 7fee4ceea640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fee340a22b0 0x7fee340a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.212+0000 7fee3e7fc640 1 -- 172.21.15.18:0/3576442906 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fee400ce560 con 0x7fee48101380 2024-02-17T12:51:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.213+0000 7fee4ceea640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fee340a22b0 0x7fee340a4770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee48100380 tx=0x7fee3800b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.216+0000 7fee3e7fc640 1 -- 172.21.15.18:0/3576442906 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee400972c0 con 0x7fee48101380 2024-02-17T12:51:53.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 64592 5.37 MB/sec warmup 68 sec latency 676.151 ms 2024-02-17T12:51:53.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.436+0000 7fee4f94f640 1 -- 172.21.15.18:0/3576442906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee481096e0 con 0x7fee48101380 2024-02-17T12:51:53.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.437+0000 7fee3e7fc640 1 -- 172.21.15.18:0/3576442906 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11399 (secure 0 0 0) 0x7fee4009a980 con 0x7fee48101380 2024-02-17T12:51:53.428 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:51:53.428 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:51:04.933427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:51:53.429 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 28 2024-02-17T12:51:53.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.438+0000 7fee4f94f640 1 -- 172.21.15.18:0/3576442906 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fee340a22b0 msgr2=0x7fee340a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.438+0000 7fee4f94f640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fee340a22b0 0x7fee340a4770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee48100380 tx=0x7fee3800b390 comp rx=0 tx=0).stop 2024-02-17T12:51:53.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.438+0000 7fee4f94f640 1 -- 172.21.15.18:0/3576442906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee48101380 msgr2=0x7fee4819d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.438+0000 7fee4f94f640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee48101380 0x7fee4819d460 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fee4000eab0 tx=0x7fee4000ef70 comp rx=0 tx=0).stop 2024-02-17T12:51:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.438+0000 7fee4f94f640 1 -- 172.21.15.18:0/3576442906 shutdown_connections 2024-02-17T12:51:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.439+0000 7fee4f94f640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fee340a22b0 0x7fee340a4770 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.439+0000 7fee4f94f640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee48197660 0x7fee48197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.439+0000 7fee4f94f640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee48106450 0x7fee4819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.439+0000 7fee4f94f640 1 --2- 172.21.15.18:0/3576442906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee48101380 0x7fee4819d460 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.439+0000 7fee4f94f640 1 -- 172.21.15.18:0/3576442906 >> 172.21.15.18:0/3576442906 conn(0x7fee480fcc20 msgr2=0x7fee480fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.439+0000 7fee4f94f640 1 -- 172.21.15.18:0/3576442906 shutdown_connections 2024-02-17T12:51:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.439+0000 7fee4f94f640 1 -- 172.21.15.18:0/3576442906 wait complete. 2024-02-17T12:51:53.441 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:51:53.441 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:51:53.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.554+0000 7ff31ba40640 1 -- 172.21.15.18:0/1310140243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff314101d30 msgr2=0x7ff314105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.554+0000 7ff31ba40640 1 --2- 172.21.15.18:0/1310140243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff314101d30 0x7ff314105d70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff308009930 tx=0x7ff30802bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:53.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 -- 172.21.15.18:0/1310140243 shutdown_connections 2024-02-17T12:51:53.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 --2- 172.21.15.18:0/1310140243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff314106450 0x7ff31410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 --2- 172.21.15.18:0/1310140243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff314101d30 0x7ff314105d70 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 --2- 172.21.15.18:0/1310140243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff314101380 0x7ff314101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 -- 172.21.15.18:0/1310140243 >> 172.21.15.18:0/1310140243 conn(0x7ff314078580 msgr2=0x7ff314078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 -- 172.21.15.18:0/1310140243 shutdown_connections 2024-02-17T12:51:53.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 -- 172.21.15.18:0/1310140243 wait complete. 2024-02-17T12:51:53.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 Processor -- start 2024-02-17T12:51:53.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 -- start start 2024-02-17T12:51:53.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff314101380 0x7ff31419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff314106450 0x7ff31419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff314197650 0x7ff314197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314198120 con 0x7ff314106450 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314198290 con 0x7ff314101380 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.555+0000 7ff31ba40640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314198450 con 0x7ff314197650 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff318fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff314106450 0x7ff31419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff318fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff314106450 0x7ff31419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36178/0 (socket says 172.21.15.18:36178) 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff318fdb640 1 -- 172.21.15.18:0/1454037143 learned_addr learned my addr 172.21.15.18:0/1454037143 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff3197dc640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff314101380 0x7ff31419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff319fdd640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff314197650 0x7ff314197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff318fdb640 1 -- 172.21.15.18:0/1454037143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff314197650 msgr2=0x7ff314197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff318fdb640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff314197650 0x7ff314197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff318fdb640 1 -- 172.21.15.18:0/1454037143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff314101380 msgr2=0x7ff31419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff318fdb640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff314101380 0x7ff31419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff318fdb640 1 -- 172.21.15.18:0/1454037143 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff308009560 con 0x7ff314106450 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff318fdb640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff314106450 0x7ff31419dad0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff308009230 tx=0x7ff308004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff3127fc640 1 -- 172.21.15.18:0/1454037143 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3080043d0 con 0x7ff314106450 2024-02-17T12:51:53.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff3127fc640 1 -- 172.21.15.18:0/1454037143 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3080052d0 con 0x7ff314106450 2024-02-17T12:51:53.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff31ba40640 1 -- 172.21.15.18:0/1454037143 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3141a4390 con 0x7ff314106450 2024-02-17T12:51:53.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff3127fc640 1 -- 172.21.15.18:0/1454037143 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308003d80 con 0x7ff314106450 2024-02-17T12:51:53.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.556+0000 7ff31ba40640 1 -- 172.21.15.18:0/1454037143 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3141a4870 con 0x7ff314106450 2024-02-17T12:51:53.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.558+0000 7ff3127fc640 1 -- 172.21.15.18:0/1454037143 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff308005440 con 0x7ff314106450 2024-02-17T12:51:53.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff31e60e740 1 Processor -- start 2024-02-17T12:51:53.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff31e60e740 1 -- start start 2024-02-17T12:51:53.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff3127fc640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff2fc0a2230 0x7ff2fc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff31e60e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610e5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff31e60e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610ee210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff31e60e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610ee750 0x558c610f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff31e60e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c610a0960 con 0x558c610e5b70 2024-02-17T12:51:53.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff31e60e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c61054310 con 0x558c610e5250 2024-02-17T12:51:53.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff31e60e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c61051840 con 0x558c610ee750 2024-02-17T12:51:53.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff3127fc640 1 -- 172.21.15.18:0/1454037143 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff3080f00a0 con 0x7ff314106450 2024-02-17T12:51:53.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff319fdd640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff2fc0a2230 0x7ff2fc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff318fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610ee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff318fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610ee210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36180/0 (socket says 172.21.15.18:36180) 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff318fdb640 1 -- 172.21.15.18:0/392054403 learned_addr learned my addr 172.21.15.18:0/392054403 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.559+0000 7ff319fdd640 1 --2- 172.21.15.18:0/392054403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610ee750 0x558c610f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff3197dc640 1 --2- 172.21.15.18:0/392054403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610e5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff318fdb640 1 -- 172.21.15.18:0/392054403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610ee750 msgr2=0x558c610f0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff318fdb640 1 --2- 172.21.15.18:0/392054403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610ee750 0x558c610f0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff318fdb640 1 -- 172.21.15.18:0/392054403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 msgr2=0x558c610e5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff318fdb640 1 --2- 172.21.15.18:0/392054403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610e5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff318fdb640 1 -- 172.21.15.18:0/392054403 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c6100f060 con 0x558c610e5b70 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff319fdd640 1 --2- 172.21.15.18:0/392054403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610ee750 0x558c610f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff3197dc640 1 --2- 172.21.15.18:0/392054403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610e5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff318fdb640 1 --2- 172.21.15.18:0/392054403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610ee210 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff3080bcb80 tx=0x7ff3080bca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1634ee27821753 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff319fdd640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff2fc0a2230 0x7ff2fc0a46f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x558c610e0b10 tx=0x7ff304009a60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff2fb7fe640 1 -- 172.21.15.18:0/392054403 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3080bd2f0 con 0x558c610e5b70 2024-02-17T12:51:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff2fb7fe640 1 -- 172.21.15.18:0/392054403 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3080bcbf0 con 0x558c610e5b70 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.560+0000 7ff2fb7fe640 1 -- 172.21.15.18:0/392054403 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3080036d0 con 0x558c610e5b70 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.561+0000 7ff31e60e740 1 -- 172.21.15.18:0/392054403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 msgr2=0x558c610ee210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.561+0000 7ff31e60e740 1 --2- 172.21.15.18:0/392054403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610ee210 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff3080bcb80 tx=0x7ff3080bca80 comp rx=0 tx=0).stop 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.561+0000 7ff31e60e740 1 -- 172.21.15.18:0/392054403 shutdown_connections 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.561+0000 7ff31e60e740 1 --2- 172.21.15.18:0/392054403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610ee750 0x558c610f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.561+0000 7ff31e60e740 1 --2- 172.21.15.18:0/392054403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610ee210 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.561+0000 7ff31e60e740 1 --2- 172.21.15.18:0/392054403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610e5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.561+0000 7ff31e60e740 1 -- 172.21.15.18:0/392054403 >> 172.21.15.18:0/392054403 conn(0x558c610df980 msgr2=0x558c610dfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.561+0000 7ff31e60e740 1 -- 172.21.15.18:0/392054403 shutdown_connections 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.561+0000 7ff31e60e740 1 -- 172.21.15.18:0/392054403 wait complete. 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.561+0000 7ff31e60e740 1 -- start start 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.561+0000 7ff31e60e740 1 Processor -- start 2024-02-17T12:51:53.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff31e60e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff31e60e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610e5b70 0x558c610ebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff31e60e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610ee750 0x558c610ec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff31e60e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c6104d020 con 0x558c610ee750 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff31e60e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c60d20430 con 0x558c610e5250 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff31e60e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c60cdce00 con 0x558c610e5b70 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff31e60e740 10 client.0 _ll_register_callbacks cb 0x558c60ff4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff319fdd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610ee750 0x558c610ec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff319fdd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610ee750 0x558c610ec0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36184/0 (socket says 172.21.15.18:36184) 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff319fdd640 1 -- 172.21.15.18:0/1591695723 learned_addr learned my addr 172.21.15.18:0/1591695723 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff3197dc640 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff318fdb640 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610e5b70 0x558c610ebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff319fdd640 1 -- 172.21.15.18:0/1591695723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610e5b70 msgr2=0x558c610ebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff319fdd640 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610e5b70 0x558c610ebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff319fdd640 1 -- 172.21.15.18:0/1591695723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 msgr2=0x558c610eb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff319fdd640 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610eb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff319fdd640 1 -- 172.21.15.18:0/1591695723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3141a4870 con 0x558c610ee750 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff319fdd640 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610ee750 0x558c610ec0b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff304009be0 tx=0x7ff3040096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee80aea8ca96bea2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff2f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff2f8ff9640 1 -- 172.21.15.18:0/1591695723 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400eee0 con 0x558c610ee750 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff2f8ff9640 1 -- 172.21.15.18:0/1591695723 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff304010920 con 0x558c610ee750 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.562+0000 7ff31e60e740 1 -- 172.21.15.18:0/1591695723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c60d4c7e0 con 0x558c610ee750 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.563+0000 7ff2f8ff9640 1 -- 172.21.15.18:0/1591695723 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff304010ae0 con 0x558c610ee750 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.563+0000 7ff31e60e740 10 client.16242 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.563+0000 7ff31e60e740 1 -- 172.21.15.18:0/1591695723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558c6104b6a0 con 0x558c610ee750 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.563+0000 7ff2f8ff9640 1 -- 172.21.15.18:0/1591695723 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558c60d4c7e0 con 0x558c610ee750 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.563+0000 7ff2f8ff9640 1 -- 172.21.15.18:0/1591695723 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff30401f400 con 0x558c610ee750 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.563+0000 7ff31e60e740 10 client.16242 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:53.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.563+0000 7ff31e60e740 10 client.16242 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.563+0000 7ff31e60e740 20 client.16242 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.563+0000 7ff31e60e740 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x558c610fe6e0 0x558c61100b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.563+0000 7ff31e60e740 4 client.16242 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.563+0000 7ff31e60e740 1 -- 172.21.15.18:0/1591695723 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c60d89720 con 0x558c610fe6e0 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.563+0000 7ff3197dc640 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x558c610fe6e0 0x558c61100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.564+0000 7ff3197dc640 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x558c610fe6e0 0x558c61100b00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bab61bcd28e0fe9d server_cookie=bf6209d9fd643884 in_seq=0 out_seq=0 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.564+0000 7ff2f8ff9640 10 client.16242 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.564+0000 7ff2f8ff9640 1 -- 172.21.15.18:0/1591695723 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c60d89720 con 0x558c610fe6e0 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.564+0000 7ff2f8ff9640 10 client.16242 handle_command_reply: tid=0 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff31e60e740 1 client.16242 shutdown 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff31e60e740 1 -- 172.21.15.18:0/1591695723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610ee750 msgr2=0x558c610ec0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff31e60e740 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610ee750 0x558c610ec0b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff304009be0 tx=0x7ff3040096a0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff31e60e740 1 -- 172.21.15.18:0/1591695723 shutdown_connections 2024-02-17T12:51:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff31e60e740 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610ee750 0x558c610ec0b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff31e60e740 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610e5b70 0x558c610ebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff31e60e740 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x558c610fe6e0 0x558c61100b00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff31e60e740 1 --2- 172.21.15.18:0/1591695723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff2f8ff9640 0 client.16242 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:51:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff31e60e740 1 -- 172.21.15.18:0/1591695723 >> 172.21.15.18:0/1591695723 conn(0x558c610df980 msgr2=0x558c610e16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff31e60e740 1 -- 172.21.15.18:0/1591695723 shutdown_connections 2024-02-17T12:51:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff31e60e740 1 -- 172.21.15.18:0/1591695723 wait complete. 2024-02-17T12:51:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.565+0000 7ff31e60e740 20 client.16242 trim_cache size 0 max 16384 2024-02-17T12:51:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff31e60e740 1 Processor -- start 2024-02-17T12:51:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff31e60e740 1 -- start start 2024-02-17T12:51:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff31e60e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610e5250 0x558c610e14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff31e60e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff31e60e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610ee750 0x558c610f5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff31e60e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c60fd2130 con 0x558c610e5b70 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff31e60e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c610e4ba0 con 0x558c610ee750 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff31e60e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c610e0980 con 0x558c610e5250 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff318fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff318fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610f53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36192/0 (socket says 172.21.15.18:36192) 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff318fdb640 1 -- 172.21.15.18:0/3990595931 learned_addr learned my addr 172.21.15.18:0/3990595931 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff318fdb640 1 -- 172.21.15.18:0/3990595931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610e5250 msgr2=0x558c610e14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff318fdb640 1 --2- 172.21.15.18:0/3990595931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610e5250 0x558c610e14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff318fdb640 1 -- 172.21.15.18:0/3990595931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610ee750 msgr2=0x558c610f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff318fdb640 1 --2- 172.21.15.18:0/3990595931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610ee750 0x558c610f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff318fdb640 1 -- 172.21.15.18:0/3990595931 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c60fe7c20 con 0x558c610e5b70 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff318fdb640 1 --2- 172.21.15.18:0/3990595931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610f53c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff308005ec0 tx=0x7ff30802c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85babda3b707d07a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff2faffd640 1 -- 172.21.15.18:0/3990595931 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3080bd2f0 con 0x558c610e5b70 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff2faffd640 1 -- 172.21.15.18:0/3990595931 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3080c5390 con 0x558c610e5b70 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.577+0000 7ff2faffd640 1 -- 172.21.15.18:0/3990595931 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308004df0 con 0x558c610e5b70 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.578+0000 7ff31e60e740 1 -- 172.21.15.18:0/3990595931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 msgr2=0x558c610f53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.578+0000 7ff31e60e740 1 --2- 172.21.15.18:0/3990595931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610f53c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff308005ec0 tx=0x7ff30802c450 comp rx=0 tx=0).stop 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.578+0000 7ff31e60e740 1 -- 172.21.15.18:0/3990595931 shutdown_connections 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.578+0000 7ff31e60e740 1 --2- 172.21.15.18:0/3990595931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610ee750 0x558c610f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.578+0000 7ff31e60e740 1 --2- 172.21.15.18:0/3990595931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610f53c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.578+0000 7ff31e60e740 1 --2- 172.21.15.18:0/3990595931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610e5250 0x558c610e14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.578+0000 7ff31e60e740 1 -- 172.21.15.18:0/3990595931 >> 172.21.15.18:0/3990595931 conn(0x558c610df980 msgr2=0x558c610e0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.578+0000 7ff31e60e740 1 -- 172.21.15.18:0/3990595931 shutdown_connections 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.578+0000 7ff31e60e740 1 -- 172.21.15.18:0/3990595931 wait complete. 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.578+0000 7ff31e60e740 1 -- start start 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.578+0000 7ff31e60e740 1 Processor -- start 2024-02-17T12:51:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff31e60e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff31e60e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610f4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff31e60e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610ee750 0x558c610f4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff31e60e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c61154c10 con 0x558c610e5b70 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff31e60e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c61154d50 con 0x558c610e5250 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff31e60e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c61154e90 con 0x558c610ee750 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff31e60e740 10 client.0 _ll_register_callbacks cb 0x7ff304009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff318fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff318fdb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610f4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36198/0 (socket says 172.21.15.18:36198) 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff318fdb640 1 -- 172.21.15.18:0/3556328516 learned_addr learned my addr 172.21.15.18:0/3556328516 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff319fdd640 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610ee750 0x558c610f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff3197dc640 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff318fdb640 1 -- 172.21.15.18:0/3556328516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610ee750 msgr2=0x558c610f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff318fdb640 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610ee750 0x558c610f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff318fdb640 1 -- 172.21.15.18:0/3556328516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 msgr2=0x558c610f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff318fdb640 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff318fdb640 1 -- 172.21.15.18:0/3556328516 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c6100f060 con 0x558c610e5b70 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff318fdb640 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610f4320 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff30802cde0 tx=0x7ff3080c0d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1ecca399a07779 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff2f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.579+0000 7ff2f8ff9640 1 -- 172.21.15.18:0/3556328516 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3080bd2f0 con 0x558c610e5b70 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff31e60e740 1 -- 172.21.15.18:0/3556328516 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c610db0f0 con 0x558c610e5b70 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff2f8ff9640 1 -- 172.21.15.18:0/3556328516 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff3080a73b0 con 0x558c610e5b70 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff2f8ff9640 1 -- 172.21.15.18:0/3556328516 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3080a75f0 con 0x558c610e5b70 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff31e60e740 10 client.16248 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff31e60e740 1 -- 172.21.15.18:0/3556328516 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558c6100d8c0 con 0x558c610e5b70 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff2f8ff9640 1 -- 172.21.15.18:0/3556328516 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558c610db0f0 con 0x558c610e5b70 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff2f8ff9640 1 -- 172.21.15.18:0/3556328516 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff30803c070 con 0x558c610e5b70 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff31e60e740 10 client.16248 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff31e60e740 10 client.16248 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:51:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff31e60e740 20 client.16248 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff31e60e740 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x558c611561e0 0x558c61158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff31e60e740 4 client.16248 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff31e60e740 1 -- 172.21.15.18:0/3556328516 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7ff3080bcbf0 con 0x558c611561e0 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.580+0000 7ff3197dc640 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x558c611561e0 0x558c61158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff2f8ff9640 10 client.16248 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff3197dc640 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x558c611561e0 0x558c61158600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9aefa70aebc354a6 server_cookie=b709ad2a5c6b96ba in_seq=0 out_seq=0 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff2f8ff9640 1 -- 172.21.15.18:0/3556328516 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7ff3080bcbf0 con 0x558c611561e0 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff2f8ff9640 10 client.16248 handle_command_reply: tid=0 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff31e60e740 1 client.16248 shutdown 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff31e60e740 1 -- 172.21.15.18:0/3556328516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 msgr2=0x558c610f4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff31e60e740 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610f4320 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff30802cde0 tx=0x7ff3080c0d40 comp rx=0 tx=0).stop 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff31e60e740 1 -- 172.21.15.18:0/3556328516 shutdown_connections 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff31e60e740 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610ee750 0x558c610f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff31e60e740 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610e5b70 0x558c610f4320 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff31e60e740 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x558c611561e0 0x558c61158600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff31e60e740 1 --2- 172.21.15.18:0/3556328516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610e5250 0x558c610f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff31e60e740 1 -- 172.21.15.18:0/3556328516 >> 172.21.15.18:0/3556328516 conn(0x558c610df980 msgr2=0x558c610ebf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff2f8ff9640 0 client.16248 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:51:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff31e60e740 1 -- 172.21.15.18:0/3556328516 shutdown_connections 2024-02-17T12:51:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff31e60e740 1 -- 172.21.15.18:0/3556328516 wait complete. 2024-02-17T12:51:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.581+0000 7ff31e60e740 20 client.16248 trim_cache size 0 max 16384 2024-02-17T12:51:53.579 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:51:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.582+0000 7ff31ba40640 1 -- 172.21.15.18:0/1454037143 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff2fc0a2230 msgr2=0x7ff2fc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.582+0000 7ff31ba40640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff2fc0a2230 0x7ff2fc0a46f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x558c610e0b10 tx=0x7ff304009a60 comp rx=0 tx=0).stop 2024-02-17T12:51:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.582+0000 7ff31ba40640 1 -- 172.21.15.18:0/1454037143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff314106450 msgr2=0x7ff31419dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.582+0000 7ff31ba40640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff314106450 0x7ff31419dad0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff308009230 tx=0x7ff308004c00 comp rx=0 tx=0).stop 2024-02-17T12:51:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.582+0000 7ff31ba40640 1 -- 172.21.15.18:0/1454037143 shutdown_connections 2024-02-17T12:51:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.582+0000 7ff31ba40640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff2fc0a2230 0x7ff2fc0a46f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.582+0000 7ff31ba40640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff314197650 0x7ff314197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.582+0000 7ff31ba40640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff314106450 0x7ff31419dad0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.582+0000 7ff31ba40640 1 --2- 172.21.15.18:0/1454037143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff314101380 0x7ff31419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.582+0000 7ff31ba40640 1 -- 172.21.15.18:0/1454037143 >> 172.21.15.18:0/1454037143 conn(0x7ff314078580 msgr2=0x7ff3140ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.582+0000 7ff31ba40640 1 -- 172.21.15.18:0/1454037143 shutdown_connections 2024-02-17T12:51:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.582+0000 7ff31ba40640 1 -- 172.21.15.18:0/1454037143 wait complete. 2024-02-17T12:51:53.584 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:51:53.584 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:51:53.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.697+0000 7f204d402640 1 -- 172.21.15.18:0/952013639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2048102240 msgr2=0x7f2048102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.697+0000 7f204d402640 1 --2- 172.21.15.18:0/952013639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2048102240 0x7f2048102620 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2038009900 tx=0x7f203802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.697+0000 7f204d402640 1 -- 172.21.15.18:0/952013639 shutdown_connections 2024-02-17T12:51:53.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.697+0000 7f204d402640 1 --2- 172.21.15.18:0/952013639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f204810b610 0x7f204810da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.697+0000 7f204d402640 1 --2- 172.21.15.18:0/952013639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2048102b60 0x7f204810b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.697+0000 7f204d402640 1 --2- 172.21.15.18:0/952013639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2048102240 0x7f2048102620 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.698+0000 7f204d402640 1 -- 172.21.15.18:0/952013639 >> 172.21.15.18:0/952013639 conn(0x7f20480fb9e0 msgr2=0x7f20480fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.698+0000 7f204d402640 1 -- 172.21.15.18:0/952013639 shutdown_connections 2024-02-17T12:51:53.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.698+0000 7f204d402640 1 -- 172.21.15.18:0/952013639 wait complete. 2024-02-17T12:51:53.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.698+0000 7f204d402640 1 Processor -- start 2024-02-17T12:51:53.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.698+0000 7f204d402640 1 -- start start 2024-02-17T12:51:53.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.698+0000 7f204d402640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2048102240 0x7f2048197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.698+0000 7f204d402640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2048102b60 0x7f2048197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f204d402640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f204810b610 0x7f204819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f204d402640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204819c5c0 con 0x7f2048102b60 2024-02-17T12:51:53.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f204d402640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204819c730 con 0x7f2048102240 2024-02-17T12:51:53.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f204d402640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204819c8f0 con 0x7f204810b610 2024-02-17T12:51:53.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f20467fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2048102b60 0x7f2048197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f20467fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2048102b60 0x7f2048197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36210/0 (socket says 172.21.15.18:36210) 2024-02-17T12:51:53.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f20467fc640 1 -- 172.21.15.18:0/3790093850 learned_addr learned my addr 172.21.15.18:0/3790093850 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f20477fe640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f204810b610 0x7f204819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f2046ffd640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2048102240 0x7f2048197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f20467fc640 1 -- 172.21.15.18:0/3790093850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f204810b610 msgr2=0x7f204819bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f20467fc640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f204810b610 0x7f204819bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f20467fc640 1 -- 172.21.15.18:0/3790093850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2048102240 msgr2=0x7f2048197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f20467fc640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2048102240 0x7f2048197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f20467fc640 1 -- 172.21.15.18:0/3790093850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2038009560 con 0x7f2048102b60 2024-02-17T12:51:53.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f20477fe640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f204810b610 0x7f204819bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:53.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f20467fc640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2048102b60 0x7f2048197c40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f203c00d950 tx=0x7f203c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f202ffff640 1 -- 172.21.15.18:0/3790093850 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c009a30 con 0x7f2048102b60 2024-02-17T12:51:53.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f204d402640 1 -- 172.21.15.18:0/3790093850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204819cbd0 con 0x7f2048102b60 2024-02-17T12:51:53.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f202ffff640 1 -- 172.21.15.18:0/3790093850 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f203c009b90 con 0x7f2048102b60 2024-02-17T12:51:53.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f202ffff640 1 -- 172.21.15.18:0/3790093850 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c004a00 con 0x7f2048102b60 2024-02-17T12:51:53.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.699+0000 7f204d402640 1 -- 172.21.15.18:0/3790093850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20481a4450 con 0x7f2048102b60 2024-02-17T12:51:53.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f202ffff640 1 -- 172.21.15.18:0/3790093850 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f203c009d00 con 0x7f2048102b60 2024-02-17T12:51:53.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f204ffcf740 1 Processor -- start 2024-02-17T12:51:53.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f204ffcf740 1 -- start start 2024-02-17T12:51:53.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f204ffcf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 0x55a509b75550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f204ffcf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b82810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f204ffcf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b82d50 0x55a509b85140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f204ffcf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a509b30960 con 0x55a509b75170 2024-02-17T12:51:53.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f204ffcf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a509ae4310 con 0x55a509b75a90 2024-02-17T12:51:53.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f204ffcf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a509ae1840 con 0x55a509b82d50 2024-02-17T12:51:53.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f20477fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 0x55a509b75550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f20477fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 0x55a509b75550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36226/0 (socket says 172.21.15.18:36226) 2024-02-17T12:51:53.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f20477fe640 1 -- 172.21.15.18:0/1684697653 learned_addr learned my addr 172.21.15.18:0/1684697653 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f2046ffd640 1 --2- 172.21.15.18:0/1684697653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b82810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f202ffff640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f20300a2230 0x7f20300a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f20477fe640 1 -- 172.21.15.18:0/1684697653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b82d50 msgr2=0x55a509b85140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f20477fe640 1 --2- 172.21.15.18:0/1684697653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b82d50 0x55a509b85140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f20477fe640 1 -- 172.21.15.18:0/1684697653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 msgr2=0x55a509b82810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f20477fe640 1 --2- 172.21.15.18:0/1684697653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b82810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f20477fe640 1 -- 172.21.15.18:0/1684697653 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a509a9f060 con 0x55a509b75170 2024-02-17T12:51:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.702+0000 7f202ffff640 1 -- 172.21.15.18:0/3790093850 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f203c0cdb40 con 0x7f2048102b60 2024-02-17T12:51:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.703+0000 7f20477fe640 1 --2- 172.21.15.18:0/1684697653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 0x55a509b75550 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2040009fd0 tx=0x7f2040010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb93c410b60fe8b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.703+0000 7f20477fe640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f20300a2230 0x7f20300a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.703+0000 7f202d7fa640 1 -- 172.21.15.18:0/1684697653 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040011ae0 con 0x55a509b75170 2024-02-17T12:51:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.703+0000 7f202d7fa640 1 -- 172.21.15.18:0/1684697653 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2040015070 con 0x55a509b75170 2024-02-17T12:51:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.703+0000 7f202d7fa640 1 -- 172.21.15.18:0/1684697653 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204001bd40 con 0x55a509b75170 2024-02-17T12:51:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.703+0000 7f20477fe640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f20300a2230 0x7f20300a46f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f204001f000 tx=0x7f2040020000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.703+0000 7f204ffcf740 1 -- 172.21.15.18:0/1684697653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 msgr2=0x55a509b75550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.703+0000 7f204ffcf740 1 --2- 172.21.15.18:0/1684697653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 0x55a509b75550 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2040009fd0 tx=0x7f2040010860 comp rx=0 tx=0).stop 2024-02-17T12:51:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 -- 172.21.15.18:0/1684697653 shutdown_connections 2024-02-17T12:51:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 --2- 172.21.15.18:0/1684697653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b82d50 0x55a509b85140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 --2- 172.21.15.18:0/1684697653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b82810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 --2- 172.21.15.18:0/1684697653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 0x55a509b75550 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 -- 172.21.15.18:0/1684697653 >> 172.21.15.18:0/1684697653 conn(0x55a509b6fb70 msgr2=0x55a509b6ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 -- 172.21.15.18:0/1684697653 shutdown_connections 2024-02-17T12:51:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 -- 172.21.15.18:0/1684697653 wait complete. 2024-02-17T12:51:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 -- start start 2024-02-17T12:51:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 Processor -- start 2024-02-17T12:51:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b75170 0x55a509b7fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b80350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 0x55a509b80890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a509add020 con 0x55a509b82d50 2024-02-17T12:51:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5097b0430 con 0x55a509b75a90 2024-02-17T12:51:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a50976ce00 con 0x55a509b75170 2024-02-17T12:51:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f204ffcf740 10 client.0 _ll_register_callbacks cb 0x55a509a84cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:53.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f20477fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 0x55a509b80890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f20477fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 0x55a509b80890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36236/0 (socket says 172.21.15.18:36236) 2024-02-17T12:51:53.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f20477fe640 1 -- 172.21.15.18:0/2829968502 learned_addr learned my addr 172.21.15.18:0/2829968502 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f20467fc640 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b80350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.704+0000 7f2046ffd640 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b75170 0x55a509b7fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f20477fe640 1 -- 172.21.15.18:0/2829968502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b75170 msgr2=0x55a509b7fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f20477fe640 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b75170 0x55a509b7fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f20477fe640 1 -- 172.21.15.18:0/2829968502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 msgr2=0x55a509b80350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f20477fe640 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b80350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f20477fe640 1 -- 172.21.15.18:0/2829968502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2040009c30 con 0x55a509b82d50 2024-02-17T12:51:53.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f20477fe640 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 0x55a509b80890 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f204001b9e0 tx=0x7f204001b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c9292ef23b58bf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f202aff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:53.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f202aff5640 1 -- 172.21.15.18:0/2829968502 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040011cf0 con 0x55a509b82d50 2024-02-17T12:51:53.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f202aff5640 1 -- 172.21.15.18:0/2829968502 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2040017070 con 0x55a509b82d50 2024-02-17T12:51:53.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f204ffcf740 1 -- 172.21.15.18:0/2829968502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5097dc7e0 con 0x55a509b82d50 2024-02-17T12:51:53.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f202aff5640 1 -- 172.21.15.18:0/2829968502 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20400049c0 con 0x55a509b82d50 2024-02-17T12:51:53.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f204ffcf740 10 client.16260 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:53.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f204ffcf740 1 -- 172.21.15.18:0/2829968502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a509adb6a0 con 0x55a509b82d50 2024-02-17T12:51:53.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f202aff5640 1 -- 172.21.15.18:0/2829968502 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a5097dc7e0 con 0x55a509b82d50 2024-02-17T12:51:53.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f202aff5640 1 -- 172.21.15.18:0/2829968502 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2040024030 con 0x55a509b82d50 2024-02-17T12:51:53.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f204ffcf740 10 client.16260 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:53.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.705+0000 7f204ffcf740 10 client.16260 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:51:53.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.706+0000 7f204ffcf740 20 client.16260 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:53.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.706+0000 7f204ffcf740 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a509b93230 0x55a509b95620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.706+0000 7f204ffcf740 4 client.16260 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:53.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.706+0000 7f204ffcf740 1 -- 172.21.15.18:0/2829968502 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a509819720 con 0x55a509b93230 2024-02-17T12:51:53.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.706+0000 7f2046ffd640 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a509b93230 0x55a509b95620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.706+0000 7f202aff5640 10 client.16260 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:51:53.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.706+0000 7f2046ffd640 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a509b93230 0x55a509b95620 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d8fe6e88ec41575 server_cookie=7a26b955200a18d in_seq=0 out_seq=0 2024-02-17T12:51:53.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f202aff5640 1 -- 172.21.15.18:0/2829968502 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a509819720 con 0x55a509b93230 2024-02-17T12:51:53.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f202aff5640 10 client.16260 handle_command_reply: tid=0 2024-02-17T12:51:53.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f204ffcf740 1 client.16260 shutdown 2024-02-17T12:51:53.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f204ffcf740 1 -- 172.21.15.18:0/2829968502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 msgr2=0x55a509b80890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f204ffcf740 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 0x55a509b80890 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f204001b9e0 tx=0x7f204001b630 comp rx=0 tx=0).stop 2024-02-17T12:51:53.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f204ffcf740 1 -- 172.21.15.18:0/2829968502 shutdown_connections 2024-02-17T12:51:53.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f204ffcf740 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a509b93230 0x55a509b95620 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f204ffcf740 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 0x55a509b80890 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f204ffcf740 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b80350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f204ffcf740 1 --2- 172.21.15.18:0/2829968502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b75170 0x55a509b7fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f202aff5640 0 client.16260 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:51:53.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f204ffcf740 1 -- 172.21.15.18:0/2829968502 >> 172.21.15.18:0/2829968502 conn(0x55a509b6fb70 msgr2=0x55a509b718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f204ffcf740 1 -- 172.21.15.18:0/2829968502 shutdown_connections 2024-02-17T12:51:53.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f204ffcf740 1 -- 172.21.15.18:0/2829968502 wait complete. 2024-02-17T12:51:53.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.707+0000 7f204ffcf740 20 client.16260 trim_cache size 0 max 16384 2024-02-17T12:51:53.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f204ffcf740 1 Processor -- start 2024-02-17T12:51:53.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f204ffcf740 1 -- start start 2024-02-17T12:51:53.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f204ffcf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 0x55a509b824d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f204ffcf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b806f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f204ffcf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b82d50 0x55a509b99f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f204ffcf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a509a62130 con 0x55a509b75170 2024-02-17T12:51:53.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f204ffcf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a509b8ec60 con 0x55a509b75a90 2024-02-17T12:51:53.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f204ffcf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a509b74db0 con 0x55a509b82d50 2024-02-17T12:51:53.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f2046ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 0x55a509b824d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f2046ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 0x55a509b824d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36240/0 (socket says 172.21.15.18:36240) 2024-02-17T12:51:53.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f2046ffd640 1 -- 172.21.15.18:0/524619281 learned_addr learned my addr 172.21.15.18:0/524619281 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f20467fc640 1 --2- 172.21.15.18:0/524619281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b806f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f2046ffd640 1 -- 172.21.15.18:0/524619281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b82d50 msgr2=0x55a509b99f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f2046ffd640 1 --2- 172.21.15.18:0/524619281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b82d50 0x55a509b99f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f2046ffd640 1 -- 172.21.15.18:0/524619281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 msgr2=0x55a509b806f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f2046ffd640 1 --2- 172.21.15.18:0/524619281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b806f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f2046ffd640 1 -- 172.21.15.18:0/524619281 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a509a77c20 con 0x55a509b75170 2024-02-17T12:51:53.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.719+0000 7f20467fc640 1 --2- 172.21.15.18:0/524619281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b806f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:53.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f2046ffd640 1 --2- 172.21.15.18:0/524619281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 0x55a509b824d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f203802c210 tx=0x7f2038005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6f6c947cc9357f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f202cff9640 1 -- 172.21.15.18:0/524619281 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038043040 con 0x55a509b75170 2024-02-17T12:51:53.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f202cff9640 1 -- 172.21.15.18:0/524619281 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f203804e070 con 0x55a509b75170 2024-02-17T12:51:53.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f202cff9640 1 -- 172.21.15.18:0/524619281 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203802cc30 con 0x55a509b75170 2024-02-17T12:51:53.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f204ffcf740 1 -- 172.21.15.18:0/524619281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 msgr2=0x55a509b824d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f204ffcf740 1 --2- 172.21.15.18:0/524619281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 0x55a509b824d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f203802c210 tx=0x7f2038005e00 comp rx=0 tx=0).stop 2024-02-17T12:51:53.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f204ffcf740 1 -- 172.21.15.18:0/524619281 shutdown_connections 2024-02-17T12:51:53.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f204ffcf740 1 --2- 172.21.15.18:0/524619281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b82d50 0x55a509b99f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f204ffcf740 1 --2- 172.21.15.18:0/524619281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b806f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f204ffcf740 1 --2- 172.21.15.18:0/524619281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b75170 0x55a509b824d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f204ffcf740 1 -- 172.21.15.18:0/524619281 >> 172.21.15.18:0/524619281 conn(0x55a509b6fb70 msgr2=0x55a509b6ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f204ffcf740 1 -- 172.21.15.18:0/524619281 shutdown_connections 2024-02-17T12:51:53.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.720+0000 7f204ffcf740 1 -- 172.21.15.18:0/524619281 wait complete. 2024-02-17T12:51:53.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f204ffcf740 1 -- start start 2024-02-17T12:51:53.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f204ffcf740 1 Processor -- start 2024-02-17T12:51:53.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f204ffcf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b75170 0x55a509b88710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f204ffcf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b88c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f204ffcf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 0x55a509b89190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f204ffcf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a509be9710 con 0x55a509b82d50 2024-02-17T12:51:53.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f204ffcf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a509be9850 con 0x55a509b75a90 2024-02-17T12:51:53.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f204ffcf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a509be9a10 con 0x55a509b75170 2024-02-17T12:51:53.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f204ffcf740 10 client.0 _ll_register_callbacks cb 0x7f2040009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:53.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f20477fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 0x55a509b89190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f20477fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 0x55a509b89190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36246/0 (socket says 172.21.15.18:36246) 2024-02-17T12:51:53.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f20477fe640 1 -- 172.21.15.18:0/3218565351 learned_addr learned my addr 172.21.15.18:0/3218565351 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f2046ffd640 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b75170 0x55a509b88710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f20467fc640 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b88c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f20477fe640 1 -- 172.21.15.18:0/3218565351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b75170 msgr2=0x55a509b88710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f20477fe640 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b75170 0x55a509b88710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f20477fe640 1 -- 172.21.15.18:0/3218565351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 msgr2=0x55a509b88c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f20477fe640 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b88c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f20477fe640 1 -- 172.21.15.18:0/3218565351 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204810e710 con 0x55a509b82d50 2024-02-17T12:51:53.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.721+0000 7f20467fc640 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b88c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:53.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f20477fe640 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 0x55a509b89190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2040001220 tx=0x7f2040011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83789c58b909d54 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f202aff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:53.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f202aff5640 1 -- 172.21.15.18:0/3218565351 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040009ad0 con 0x55a509b82d50 2024-02-17T12:51:53.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f202aff5640 1 -- 172.21.15.18:0/3218565351 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2040017040 con 0x55a509b82d50 2024-02-17T12:51:53.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f204ffcf740 1 -- 172.21.15.18:0/3218565351 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2040015070 con 0x55a509b82d50 2024-02-17T12:51:53.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f202aff5640 1 -- 172.21.15.18:0/3218565351 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2040009e50 con 0x55a509b82d50 2024-02-17T12:51:53.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f204ffcf740 10 client.16266 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:53.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f204ffcf740 1 -- 172.21.15.18:0/3218565351 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a509a9d8c0 con 0x55a509b82d50 2024-02-17T12:51:53.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f202aff5640 1 -- 172.21.15.18:0/3218565351 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2040015070 con 0x55a509b82d50 2024-02-17T12:51:53.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f202aff5640 1 -- 172.21.15.18:0/3218565351 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2040005020 con 0x55a509b82d50 2024-02-17T12:51:53.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f204ffcf740 10 client.16266 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:53.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f204ffcf740 10 client.16266 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:51:53.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f204ffcf740 20 client.16266 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:53.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f204ffcf740 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a509beae30 0x55a509bed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f204ffcf740 4 client.16266 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f204ffcf740 1 -- 172.21.15.18:0/3218565351 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a509868890 con 0x55a509beae30 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.722+0000 7f2046ffd640 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a509beae30 0x55a509bed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f202aff5640 10 client.16266 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f2046ffd640 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a509beae30 0x55a509bed220 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7469e0bfd49ef728 server_cookie=1698e8edc76577a1 in_seq=0 out_seq=0 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f202aff5640 1 -- 172.21.15.18:0/3218565351 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a509868890 con 0x55a509beae30 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f202aff5640 10 client.16266 handle_command_reply: tid=0 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f204ffcf740 1 client.16266 shutdown 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f204ffcf740 1 -- 172.21.15.18:0/3218565351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 msgr2=0x55a509b89190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f204ffcf740 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 0x55a509b89190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2040001220 tx=0x7f2040011e20 comp rx=0 tx=0).stop 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f204ffcf740 1 -- 172.21.15.18:0/3218565351 shutdown_connections 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f204ffcf740 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a509beae30 0x55a509bed220 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f204ffcf740 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a509b82d50 0x55a509b89190 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f202aff5640 0 client.16266 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:51:53.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f204ffcf740 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a509b75a90 0x55a509b88c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f204ffcf740 1 --2- 172.21.15.18:0/3218565351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a509b75170 0x55a509b88710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f204ffcf740 1 -- 172.21.15.18:0/3218565351 >> 172.21.15.18:0/3218565351 conn(0x55a509b6fb70 msgr2=0x55a509b99f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f204ffcf740 1 -- 172.21.15.18:0/3218565351 shutdown_connections 2024-02-17T12:51:53.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f204ffcf740 1 -- 172.21.15.18:0/3218565351 wait complete. 2024-02-17T12:51:53.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.723+0000 7f204ffcf740 20 client.16266 trim_cache size 0 max 16384 2024-02-17T12:51:53.724 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:51:53.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.724+0000 7f204d402640 1 -- 172.21.15.18:0/3790093850 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f20300a2230 msgr2=0x7f20300a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.724+0000 7f204d402640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f20300a2230 0x7f20300a46f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f204001f000 tx=0x7f2040020000 comp rx=0 tx=0).stop 2024-02-17T12:51:53.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.724+0000 7f204d402640 1 -- 172.21.15.18:0/3790093850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2048102b60 msgr2=0x7f2048197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.724+0000 7f204d402640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2048102b60 0x7f2048197c40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f203c00d950 tx=0x7f203c00de10 comp rx=0 tx=0).stop 2024-02-17T12:51:53.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.724+0000 7f204d402640 1 -- 172.21.15.18:0/3790093850 shutdown_connections 2024-02-17T12:51:53.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.724+0000 7f204d402640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f20300a2230 0x7f20300a46f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.724+0000 7f204d402640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f204810b610 0x7f204819bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.724+0000 7f204d402640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2048102b60 0x7f2048197c40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.724+0000 7f204d402640 1 --2- 172.21.15.18:0/3790093850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2048102240 0x7f2048197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.724+0000 7f204d402640 1 -- 172.21.15.18:0/3790093850 >> 172.21.15.18:0/3790093850 conn(0x7f20480fb9e0 msgr2=0x7f20480fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.724+0000 7f204d402640 1 -- 172.21.15.18:0/3790093850 shutdown_connections 2024-02-17T12:51:53.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.724+0000 7f204d402640 1 -- 172.21.15.18:0/3790093850 wait complete. 2024-02-17T12:51:53.727 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:51:53.727 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:51:53.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.842+0000 7faa3aedf640 1 -- 172.21.15.18:0/1576606406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa341002a0 msgr2=0x7faa34100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.842+0000 7faa3aedf640 1 --2- 172.21.15.18:0/1576606406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa341002a0 0x7faa34100680 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7faa28009900 tx=0x7faa2802bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:53.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.842+0000 7faa3aedf640 1 -- 172.21.15.18:0/1576606406 shutdown_connections 2024-02-17T12:51:53.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.842+0000 7faa3aedf640 1 --2- 172.21.15.18:0/1576606406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa34105370 0x7faa3410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.842+0000 7faa3aedf640 1 --2- 172.21.15.18:0/1576606406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa34100c50 0x7faa34104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.842+0000 7faa3aedf640 1 --2- 172.21.15.18:0/1576606406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa341002a0 0x7faa34100680 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.842+0000 7faa3aedf640 1 -- 172.21.15.18:0/1576606406 >> 172.21.15.18:0/1576606406 conn(0x7faa34076400 msgr2=0x7faa34076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.842+0000 7faa3aedf640 1 -- 172.21.15.18:0/1576606406 shutdown_connections 2024-02-17T12:51:53.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.842+0000 7faa3aedf640 1 -- 172.21.15.18:0/1576606406 wait complete. 2024-02-17T12:51:53.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.842+0000 7faa3aedf640 1 Processor -- start 2024-02-17T12:51:53.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa3aedf640 1 -- start start 2024-02-17T12:51:53.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa3aedf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa341002a0 0x7faa341977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa3aedf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa34105370 0x7faa34197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa3aedf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3419c010 0x7faa3419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa3aedf640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3419ca00 con 0x7faa34105370 2024-02-17T12:51:53.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa3aedf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3419cb70 con 0x7faa3419c010 2024-02-17T12:51:53.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa3aedf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3419cce0 con 0x7faa341002a0 2024-02-17T12:51:53.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa33fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa34105370 0x7faa34197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa33fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa34105370 0x7faa34197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36256/0 (socket says 172.21.15.18:36256) 2024-02-17T12:51:53.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa33fff640 1 -- 172.21.15.18:0/1245401947 learned_addr learned my addr 172.21.15.18:0/1245401947 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa38c7b640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa341002a0 0x7faa341977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa3947c640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3419c010 0x7faa3419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa33fff640 1 -- 172.21.15.18:0/1245401947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa341002a0 msgr2=0x7faa341977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa33fff640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa341002a0 0x7faa341977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa33fff640 1 -- 172.21.15.18:0/1245401947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3419c010 msgr2=0x7faa3419c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa33fff640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3419c010 0x7faa3419c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa33fff640 1 -- 172.21.15.18:0/1245401947 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa28009560 con 0x7faa34105370 2024-02-17T12:51:53.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa33fff640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa34105370 0x7faa34197ce0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7faa20005fd0 tx=0x7faa2000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa31ffb640 1 -- 172.21.15.18:0/1245401947 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20009c30 con 0x7faa34105370 2024-02-17T12:51:53.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa31ffb640 1 -- 172.21.15.18:0/1245401947 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa20009d90 con 0x7faa34105370 2024-02-17T12:51:53.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa3aedf640 1 -- 172.21.15.18:0/1245401947 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa341a42c0 con 0x7faa34105370 2024-02-17T12:51:53.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.843+0000 7faa31ffb640 1 -- 172.21.15.18:0/1245401947 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20010460 con 0x7faa34105370 2024-02-17T12:51:53.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.844+0000 7faa3aedf640 1 -- 172.21.15.18:0/1245401947 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa341a47c0 con 0x7faa34105370 2024-02-17T12:51:53.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.845+0000 7faa31ffb640 1 -- 172.21.15.18:0/1245401947 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7faa20003f20 con 0x7faa34105370 2024-02-17T12:51:53.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa3db94740 1 Processor -- start 2024-02-17T12:51:53.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa31ffb640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7faa240a22b0 0x7faa240a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa3db94740 1 -- start start 2024-02-17T12:51:53.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa3947c640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7faa240a22b0 0x7faa240a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa31ffb640 1 -- 172.21.15.18:0/1245401947 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faa200cdc50 con 0x7faa34105370 2024-02-17T12:51:53.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa3db94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793250 0x5639f4793630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa3db94740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793b70 0x5639f479c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa3db94740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f479c750 0x5639f479eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa3db94740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f474e960 con 0x5639f4793b70 2024-02-17T12:51:53.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa3db94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f4702310 con 0x5639f4793250 2024-02-17T12:51:53.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa3db94740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f46ff840 con 0x5639f479c750 2024-02-17T12:51:53.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa33fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793b70 0x5639f479c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa33fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793b70 0x5639f479c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36272/0 (socket says 172.21.15.18:36272) 2024-02-17T12:51:53.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.846+0000 7faa33fff640 1 -- 172.21.15.18:0/2593772999 learned_addr learned my addr 172.21.15.18:0/2593772999 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.847+0000 7faa3947c640 1 --2- 172.21.15.18:0/2593772999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f479c750 0x5639f479eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.847+0000 7faa38c7b640 1 --2- 172.21.15.18:0/2593772999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793250 0x5639f4793630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.847+0000 7faa33fff640 1 -- 172.21.15.18:0/2593772999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f479c750 msgr2=0x5639f479eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.847+0000 7faa33fff640 1 --2- 172.21.15.18:0/2593772999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f479c750 0x5639f479eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.847+0000 7faa33fff640 1 -- 172.21.15.18:0/2593772999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793250 msgr2=0x5639f4793630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.847+0000 7faa33fff640 1 --2- 172.21.15.18:0/2593772999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793250 0x5639f4793630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.847+0000 7faa33fff640 1 -- 172.21.15.18:0/2593772999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639f46bd060 con 0x5639f4793b70 2024-02-17T12:51:53.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.847+0000 7faa3947c640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7faa240a22b0 0x7faa240a4770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00acf0 tx=0x7faa2c00e720 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.847+0000 7faa33fff640 1 --2- 172.21.15.18:0/2593772999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793b70 0x5639f479c210 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7faa20097060 tx=0x7faa2009a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e913d22887977b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.847+0000 7faa1effd640 1 -- 172.21.15.18:0/2593772999 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa200850b0 con 0x5639f4793b70 2024-02-17T12:51:53.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.847+0000 7faa1effd640 1 -- 172.21.15.18:0/2593772999 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa2009a690 con 0x5639f4793b70 2024-02-17T12:51:53.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.847+0000 7faa1effd640 1 -- 172.21.15.18:0/2593772999 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20096d00 con 0x5639f4793b70 2024-02-17T12:51:53.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.848+0000 7faa3db94740 1 -- 172.21.15.18:0/2593772999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793b70 msgr2=0x5639f479c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.848+0000 7faa3db94740 1 --2- 172.21.15.18:0/2593772999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793b70 0x5639f479c210 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7faa20097060 tx=0x7faa2009a2c0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.848+0000 7faa3db94740 1 -- 172.21.15.18:0/2593772999 shutdown_connections 2024-02-17T12:51:53.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.848+0000 7faa3db94740 1 --2- 172.21.15.18:0/2593772999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f479c750 0x5639f479eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.848+0000 7faa3db94740 1 --2- 172.21.15.18:0/2593772999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793b70 0x5639f479c210 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.848+0000 7faa3db94740 1 --2- 172.21.15.18:0/2593772999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793250 0x5639f4793630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.848+0000 7faa3db94740 1 -- 172.21.15.18:0/2593772999 >> 172.21.15.18:0/2593772999 conn(0x5639f478d980 msgr2=0x5639f478dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.848+0000 7faa3db94740 1 -- 172.21.15.18:0/2593772999 shutdown_connections 2024-02-17T12:51:53.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.848+0000 7faa3db94740 1 -- 172.21.15.18:0/2593772999 wait complete. 2024-02-17T12:51:53.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.848+0000 7faa3db94740 1 -- start start 2024-02-17T12:51:53.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.848+0000 7faa3db94740 1 Processor -- start 2024-02-17T12:51:53.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3db94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793250 0x5639f4799790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3db94740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793b70 0x5639f4799cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3db94740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 0x5639f479a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3db94740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f46fb020 con 0x5639f479c750 2024-02-17T12:51:53.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3db94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f43ce430 con 0x5639f4793250 2024-02-17T12:51:53.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3db94740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f438ae00 con 0x5639f4793b70 2024-02-17T12:51:53.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3db94740 10 client.0 _ll_register_callbacks cb 0x5639f46a2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:53.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3947c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 0x5639f479a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3947c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 0x5639f479a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36288/0 (socket says 172.21.15.18:36288) 2024-02-17T12:51:53.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3947c640 1 -- 172.21.15.18:0/2050640163 learned_addr learned my addr 172.21.15.18:0/2050640163 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa38c7b640 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793250 0x5639f4799790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa33fff640 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793b70 0x5639f4799cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3947c640 1 -- 172.21.15.18:0/2050640163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793b70 msgr2=0x5639f4799cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3947c640 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793b70 0x5639f4799cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3947c640 1 -- 172.21.15.18:0/2050640163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793250 msgr2=0x5639f4799790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3947c640 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793250 0x5639f4799790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3947c640 1 -- 172.21.15.18:0/2050640163 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa341a47c0 con 0x5639f479c750 2024-02-17T12:51:53.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3947c640 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 0x5639f479a210 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7faa2c012e20 tx=0x7faa2c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a87c661e5c54442 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa13fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:53.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa13fff640 1 -- 172.21.15.18:0/2050640163 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c018030 con 0x5639f479c750 2024-02-17T12:51:53.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa13fff640 1 -- 172.21.15.18:0/2050640163 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faa2c019070 con 0x5639f479c750 2024-02-17T12:51:53.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.849+0000 7faa3db94740 1 -- 172.21.15.18:0/2050640163 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639f43fa7e0 con 0x5639f479c750 2024-02-17T12:51:53.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.850+0000 7faa13fff640 1 -- 172.21.15.18:0/2050640163 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa2c004b20 con 0x5639f479c750 2024-02-17T12:51:53.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.850+0000 7faa3db94740 10 client.16278 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:53.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.850+0000 7faa3db94740 1 -- 172.21.15.18:0/2050640163 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5639f46f96a0 con 0x5639f479c750 2024-02-17T12:51:53.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.850+0000 7faa13fff640 1 -- 172.21.15.18:0/2050640163 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5639f43fa7e0 con 0x5639f479c750 2024-02-17T12:51:53.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.850+0000 7faa13fff640 1 -- 172.21.15.18:0/2050640163 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7faa2c024070 con 0x5639f479c750 2024-02-17T12:51:53.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.850+0000 7faa3db94740 10 client.16278 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:53.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.850+0000 7faa3db94740 10 client.16278 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:51:53.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.850+0000 7faa3db94740 20 client.16278 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:53.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.850+0000 7faa3db94740 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5639f47acbc0 0x5639f47aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.850+0000 7faa3db94740 4 client.16278 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:53.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.850+0000 7faa3db94740 1 -- 172.21.15.18:0/2050640163 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639f4437720 con 0x5639f47acbc0 2024-02-17T12:51:53.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.850+0000 7faa38c7b640 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5639f47acbc0 0x5639f47aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.851+0000 7faa38c7b640 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5639f47acbc0 0x5639f47aefe0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b697da4e7cc2ba7 server_cookie=6a5d9bcca5ca63f6 in_seq=0 out_seq=0 2024-02-17T12:51:53.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.851+0000 7faa13fff640 10 client.16278 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:51:53.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.851+0000 7faa13fff640 1 -- 172.21.15.18:0/2050640163 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639f4437720 con 0x5639f47acbc0 2024-02-17T12:51:53.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.851+0000 7faa13fff640 10 client.16278 handle_command_reply: tid=0 2024-02-17T12:51:53.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.851+0000 7faa3db94740 1 client.16278 shutdown 2024-02-17T12:51:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.852+0000 7faa3db94740 1 -- 172.21.15.18:0/2050640163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 msgr2=0x5639f479a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.852+0000 7faa3db94740 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 0x5639f479a210 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7faa2c012e20 tx=0x7faa2c004ab0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.852+0000 7faa3db94740 1 -- 172.21.15.18:0/2050640163 shutdown_connections 2024-02-17T12:51:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.852+0000 7faa3db94740 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5639f47acbc0 0x5639f47aefe0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.852+0000 7faa3db94740 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 0x5639f479a210 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.852+0000 7faa3db94740 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793b70 0x5639f4799cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.852+0000 7faa3db94740 1 --2- 172.21.15.18:0/2050640163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793250 0x5639f4799790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.852+0000 7faa13fff640 0 client.16278 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:51:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.852+0000 7faa3db94740 1 -- 172.21.15.18:0/2050640163 >> 172.21.15.18:0/2050640163 conn(0x5639f478d980 msgr2=0x5639f478f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.852+0000 7faa3db94740 1 -- 172.21.15.18:0/2050640163 shutdown_connections 2024-02-17T12:51:53.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.852+0000 7faa3db94740 1 -- 172.21.15.18:0/2050640163 wait complete. 2024-02-17T12:51:53.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.852+0000 7faa3db94740 20 client.16278 trim_cache size 0 max 16384 2024-02-17T12:51:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.863+0000 7faa3db94740 1 Processor -- start 2024-02-17T12:51:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.863+0000 7faa3db94740 1 -- start start 2024-02-17T12:51:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.863+0000 7faa3db94740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793250 0x5639f479bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.863+0000 7faa3db94740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793b70 0x5639f479a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.863+0000 7faa3db94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f479c750 0x5639f4796a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.863+0000 7faa3db94740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f4680130 con 0x5639f4793250 2024-02-17T12:51:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.863+0000 7faa3db94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f478e980 con 0x5639f479c750 2024-02-17T12:51:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.863+0000 7faa3db94740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f47b41f0 con 0x5639f4793b70 2024-02-17T12:51:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa38c7b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793250 0x5639f479bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa38c7b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793250 0x5639f479bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36298/0 (socket says 172.21.15.18:36298) 2024-02-17T12:51:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa38c7b640 1 -- 172.21.15.18:0/3250511212 learned_addr learned my addr 172.21.15.18:0/3250511212 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa33fff640 1 --2- 172.21.15.18:0/3250511212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793b70 0x5639f479a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa3947c640 1 --2- 172.21.15.18:0/3250511212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f479c750 0x5639f4796a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa38c7b640 1 -- 172.21.15.18:0/3250511212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793b70 msgr2=0x5639f479a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa38c7b640 1 --2- 172.21.15.18:0/3250511212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793b70 0x5639f479a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa38c7b640 1 -- 172.21.15.18:0/3250511212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f479c750 msgr2=0x5639f4796a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa38c7b640 1 --2- 172.21.15.18:0/3250511212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f479c750 0x5639f4796a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa38c7b640 1 -- 172.21.15.18:0/3250511212 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639f4695c20 con 0x5639f4793250 2024-02-17T12:51:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa33fff640 1 --2- 172.21.15.18:0/3250511212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793b70 0x5639f479a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa38c7b640 1 --2- 172.21.15.18:0/3250511212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793250 0x5639f479bec0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7faa2802c3a0 tx=0x7faa2802c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545110f6db1ea97c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa1e7fc640 1 -- 172.21.15.18:0/3250511212 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28047be0 con 0x5639f4793250 2024-02-17T12:51:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa1e7fc640 1 -- 172.21.15.18:0/3250511212 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa28047d40 con 0x5639f4793250 2024-02-17T12:51:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.864+0000 7faa1e7fc640 1 -- 172.21.15.18:0/3250511212 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28047690 con 0x5639f4793250 2024-02-17T12:51:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.865+0000 7faa3db94740 1 -- 172.21.15.18:0/3250511212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793250 msgr2=0x5639f479bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.865+0000 7faa3db94740 1 --2- 172.21.15.18:0/3250511212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793250 0x5639f479bec0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7faa2802c3a0 tx=0x7faa2802c330 comp rx=0 tx=0).stop 2024-02-17T12:51:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.865+0000 7faa3db94740 1 -- 172.21.15.18:0/3250511212 shutdown_connections 2024-02-17T12:51:53.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.865+0000 7faa3db94740 1 --2- 172.21.15.18:0/3250511212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f479c750 0x5639f4796a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.865+0000 7faa3db94740 1 --2- 172.21.15.18:0/3250511212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793b70 0x5639f479a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.865+0000 7faa3db94740 1 --2- 172.21.15.18:0/3250511212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f4793250 0x5639f479bec0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.865+0000 7faa3db94740 1 -- 172.21.15.18:0/3250511212 >> 172.21.15.18:0/3250511212 conn(0x5639f478d980 msgr2=0x5639f4794a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.865+0000 7faa3db94740 1 -- 172.21.15.18:0/3250511212 shutdown_connections 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.865+0000 7faa3db94740 1 -- 172.21.15.18:0/3250511212 wait complete. 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.865+0000 7faa3db94740 1 -- start start 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.865+0000 7faa3db94740 1 Processor -- start 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.865+0000 7faa3db94740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793250 0x5639f47a27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3db94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793b70 0x5639f47a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3db94740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 0x5639f48030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3db94740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f4803610 con 0x5639f479c750 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3db94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f4803750 con 0x5639f4793b70 2024-02-17T12:51:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3db94740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f48038e0 con 0x5639f4793250 2024-02-17T12:51:53.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3db94740 10 client.0 _ll_register_callbacks cb 0x7faa20009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:53.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3947c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 0x5639f48030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3947c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 0x5639f48030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36306/0 (socket says 172.21.15.18:36306) 2024-02-17T12:51:53.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3947c640 1 -- 172.21.15.18:0/757182777 learned_addr learned my addr 172.21.15.18:0/757182777 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa38c7b640 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793250 0x5639f47a27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa33fff640 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793b70 0x5639f47a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3947c640 1 -- 172.21.15.18:0/757182777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793250 msgr2=0x5639f47a27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3947c640 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793250 0x5639f47a27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3947c640 1 -- 172.21.15.18:0/757182777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793b70 msgr2=0x5639f47a2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3947c640 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793b70 0x5639f47a2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3947c640 1 -- 172.21.15.18:0/757182777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa34069930 con 0x5639f479c750 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3947c640 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 0x5639f48030d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7faa2c026950 tx=0x7faa2c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc597ab93bae3b1b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa13fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa13fff640 1 -- 172.21.15.18:0/757182777 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c026ea0 con 0x5639f479c750 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3db94740 1 -- 172.21.15.18:0/757182777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa2009a690 con 0x5639f479c750 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa13fff640 1 -- 172.21.15.18:0/757182777 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faa2c018040 con 0x5639f479c750 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa13fff640 1 -- 172.21.15.18:0/757182777 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa2c02f430 con 0x5639f479c750 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.866+0000 7faa3db94740 10 client.16284 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.867+0000 7faa3db94740 1 -- 172.21.15.18:0/757182777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5639f46bb8c0 con 0x5639f479c750 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.867+0000 7faa13fff640 1 -- 172.21.15.18:0/757182777 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa2009a690 con 0x5639f479c750 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.867+0000 7faa13fff640 1 -- 172.21.15.18:0/757182777 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7faa2c024070 con 0x5639f479c750 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.867+0000 7faa3db94740 10 client.16284 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.867+0000 7faa3db94740 10 client.16284 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:51:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.867+0000 7faa3db94740 20 client.16284 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:53.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.867+0000 7faa3db94740 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5639f4805070 0x5639f4807460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.867+0000 7faa3db94740 4 client.16284 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:51:53.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.867+0000 7faa3db94740 1 -- 172.21.15.18:0/757182777 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5639f4486890 con 0x5639f4805070 2024-02-17T12:51:53.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.867+0000 7faa38c7b640 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5639f4805070 0x5639f4807460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa13fff640 10 client.16284 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:51:53.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa38c7b640 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5639f4805070 0x5639f4807460 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c158c2624f39baf server_cookie=5f5ffcc6b01a4ef4 in_seq=0 out_seq=0 2024-02-17T12:51:53.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa13fff640 1 -- 172.21.15.18:0/757182777 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5639f4486890 con 0x5639f4805070 2024-02-17T12:51:53.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa13fff640 10 client.16284 handle_command_reply: tid=0 2024-02-17T12:51:53.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa3db94740 1 client.16284 shutdown 2024-02-17T12:51:53.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa3db94740 1 -- 172.21.15.18:0/757182777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 msgr2=0x5639f48030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa3db94740 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 0x5639f48030d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7faa2c026950 tx=0x7faa2c011c40 comp rx=0 tx=0).stop 2024-02-17T12:51:53.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa3db94740 1 -- 172.21.15.18:0/757182777 shutdown_connections 2024-02-17T12:51:53.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa3db94740 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5639f4805070 0x5639f4807460 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa3db94740 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639f479c750 0x5639f48030d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa3db94740 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639f4793b70 0x5639f47a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa3db94740 1 --2- 172.21.15.18:0/757182777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639f4793250 0x5639f47a27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa13fff640 0 client.16284 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:51:53.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa3db94740 1 -- 172.21.15.18:0/757182777 >> 172.21.15.18:0/757182777 conn(0x5639f478d980 msgr2=0x5639f478c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa3db94740 1 -- 172.21.15.18:0/757182777 shutdown_connections 2024-02-17T12:51:53.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa3db94740 1 -- 172.21.15.18:0/757182777 wait complete. 2024-02-17T12:51:53.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.868+0000 7faa3db94740 20 client.16284 trim_cache size 0 max 16384 2024-02-17T12:51:53.872 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:51:53.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.869+0000 7faa3aedf640 1 -- 172.21.15.18:0/1245401947 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7faa240a22b0 msgr2=0x7faa240a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.869+0000 7faa3aedf640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7faa240a22b0 0x7faa240a4770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00acf0 tx=0x7faa2c00e720 comp rx=0 tx=0).stop 2024-02-17T12:51:53.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.869+0000 7faa3aedf640 1 -- 172.21.15.18:0/1245401947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa34105370 msgr2=0x7faa34197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.869+0000 7faa3aedf640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa34105370 0x7faa34197ce0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7faa20005fd0 tx=0x7faa2000dde0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.869+0000 7faa3aedf640 1 -- 172.21.15.18:0/1245401947 shutdown_connections 2024-02-17T12:51:53.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.869+0000 7faa3aedf640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7faa240a22b0 0x7faa240a4770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.869+0000 7faa3aedf640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3419c010 0x7faa3419c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.869+0000 7faa3aedf640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa34105370 0x7faa34197ce0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.869+0000 7faa3aedf640 1 --2- 172.21.15.18:0/1245401947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa341002a0 0x7faa341977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.869+0000 7faa3aedf640 1 -- 172.21.15.18:0/1245401947 >> 172.21.15.18:0/1245401947 conn(0x7faa34076400 msgr2=0x7faa340fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.869+0000 7faa3aedf640 1 -- 172.21.15.18:0/1245401947 shutdown_connections 2024-02-17T12:51:53.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.869+0000 7faa3aedf640 1 -- 172.21.15.18:0/1245401947 wait complete. 2024-02-17T12:51:53.874 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:51:53.875 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:51:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.987+0000 7f09f388f640 1 -- 172.21.15.18:0/3379211411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09ec068f80 msgr2=0x7f09ec105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.987+0000 7f09f388f640 1 --2- 172.21.15.18:0/3379211411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09ec068f80 0x7f09ec105d50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009930 tx=0x7f09e002bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.987+0000 7f09f388f640 1 -- 172.21.15.18:0/3379211411 shutdown_connections 2024-02-17T12:51:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.987+0000 7f09f388f640 1 --2- 172.21.15.18:0/3379211411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09ec106430 0x7f09ec10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.987+0000 7f09f388f640 1 --2- 172.21.15.18:0/3379211411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09ec068f80 0x7f09ec105d50 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.987+0000 7f09f388f640 1 --2- 172.21.15.18:0/3379211411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09ec0685d0 0x7f09ec0689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.987+0000 7f09f388f640 1 -- 172.21.15.18:0/3379211411 >> 172.21.15.18:0/3379211411 conn(0x7f09ec077180 msgr2=0x7f09ec077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f388f640 1 -- 172.21.15.18:0/3379211411 shutdown_connections 2024-02-17T12:51:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f388f640 1 -- 172.21.15.18:0/3379211411 wait complete. 2024-02-17T12:51:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f388f640 1 Processor -- start 2024-02-17T12:51:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f388f640 1 -- start start 2024-02-17T12:51:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f388f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09ec0685d0 0x7f09ec19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f388f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09ec106430 0x7f09ec19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f388f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09ec1975f0 0x7f09ec197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f388f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec197fe0 con 0x7f09ec0685d0 2024-02-17T12:51:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f388f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec198150 con 0x7f09ec1975f0 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f388f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec198310 con 0x7f09ec106430 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f162b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09ec0685d0 0x7f09ec19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f162b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09ec0685d0 0x7f09ec19d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36312/0 (socket says 172.21.15.18:36312) 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f162b640 1 -- 172.21.15.18:0/1373762504 learned_addr learned my addr 172.21.15.18:0/1373762504 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.988+0000 7f09f1e2c640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09ec1975f0 0x7f09ec197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.989+0000 7f09f0e2a640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09ec106430 0x7f09ec19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.989+0000 7f09f162b640 1 -- 172.21.15.18:0/1373762504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09ec106430 msgr2=0x7f09ec19d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.989+0000 7f09f162b640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09ec106430 0x7f09ec19d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.989+0000 7f09f162b640 1 -- 172.21.15.18:0/1373762504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09ec1975f0 msgr2=0x7f09ec197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.989+0000 7f09f162b640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09ec1975f0 0x7f09ec197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.989+0000 7f09f162b640 1 -- 172.21.15.18:0/1373762504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e0009560 con 0x7f09ec0685d0 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.989+0000 7f09f162b640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09ec0685d0 0x7f09ec19d440 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f09e400d950 tx=0x7f09e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.989+0000 7f09ea7fc640 1 -- 172.21.15.18:0/1373762504 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4009c30 con 0x7f09ec0685d0 2024-02-17T12:51:53.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.989+0000 7f09ea7fc640 1 -- 172.21.15.18:0/1373762504 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09e4009d90 con 0x7f09ec0685d0 2024-02-17T12:51:53.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.989+0000 7f09ea7fc640 1 -- 172.21.15.18:0/1373762504 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e400f7e0 con 0x7f09ec0685d0 2024-02-17T12:51:53.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.989+0000 7f09f388f640 1 -- 172.21.15.18:0/1373762504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ec1a4300 con 0x7f09ec0685d0 2024-02-17T12:51:53.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.989+0000 7f09f388f640 1 -- 172.21.15.18:0/1373762504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ec1a47e0 con 0x7f09ec0685d0 2024-02-17T12:51:53.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.991+0000 7f09ea7fc640 1 -- 172.21.15.18:0/1373762504 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f09e4010ad0 con 0x7f09ec0685d0 2024-02-17T12:51:53.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f645e740 1 Processor -- start 2024-02-17T12:51:53.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f645e740 1 -- start start 2024-02-17T12:51:53.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f645e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c31f80 0x560227c32360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f645e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c328a0 0x560227c3f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f645e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 0x560227c41f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f645e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560227bed960 con 0x560227c3fb60 2024-02-17T12:51:53.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f645e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560227ba1310 con 0x560227c328a0 2024-02-17T12:51:53.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f645e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560227b9e840 con 0x560227c31f80 2024-02-17T12:51:53.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f0e2a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 0x560227c41f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f0e2a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 0x560227c41f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36328/0 (socket says 172.21.15.18:36328) 2024-02-17T12:51:53.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f0e2a640 1 -- 172.21.15.18:0/1766657963 learned_addr learned my addr 172.21.15.18:0/1766657963 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09ea7fc640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f09d40a2230 0x7f09d40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f1e2c640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f09d40a2230 0x7f09d40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f162b640 1 --2- 172.21.15.18:0/1766657963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c328a0 0x560227c3f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f1e2c640 1 --2- 172.21.15.18:0/1766657963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c31f80 0x560227c32360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f0e2a640 1 -- 172.21.15.18:0/1766657963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c31f80 msgr2=0x560227c32360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f0e2a640 1 --2- 172.21.15.18:0/1766657963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c31f80 0x560227c32360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f0e2a640 1 -- 172.21.15.18:0/1766657963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c328a0 msgr2=0x560227c3f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09ea7fc640 1 -- 172.21.15.18:0/1373762504 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f09e40cdd00 con 0x7f09ec0685d0 2024-02-17T12:51:53.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f0e2a640 1 --2- 172.21.15.18:0/1766657963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c328a0 0x560227c3f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f0e2a640 1 -- 172.21.15.18:0/1766657963 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560227b5c060 con 0x560227c3fb60 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f1e2c640 1 --2- 172.21.15.18:0/1766657963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c31f80 0x560227c32360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.992+0000 7f09f162b640 1 --2- 172.21.15.18:0/1766657963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c328a0 0x560227c3f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.993+0000 7f09f0e2a640 1 --2- 172.21.15.18:0/1766657963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 0x560227c41f50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f09e0000c00 tx=0x7f09e002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12e8533659f52db server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.993+0000 7f09d37fe640 1 -- 172.21.15.18:0/1766657963 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e0004a80 con 0x560227c3fb60 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.993+0000 7f09f1e2c640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f09d40a2230 0x7f09d40a46f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09dc005fd0 tx=0x7f09dc004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.993+0000 7f09d37fe640 1 -- 172.21.15.18:0/1766657963 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09e0004030 con 0x560227c3fb60 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.993+0000 7f09d37fe640 1 -- 172.21.15.18:0/1766657963 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e0005360 con 0x560227c3fb60 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.993+0000 7f09f645e740 1 -- 172.21.15.18:0/1766657963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 msgr2=0x560227c41f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.993+0000 7f09f645e740 1 --2- 172.21.15.18:0/1766657963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 0x560227c41f50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f09e0000c00 tx=0x7f09e002c650 comp rx=0 tx=0).stop 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 -- 172.21.15.18:0/1766657963 shutdown_connections 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 --2- 172.21.15.18:0/1766657963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 0x560227c41f50 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 --2- 172.21.15.18:0/1766657963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c328a0 0x560227c3f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 --2- 172.21.15.18:0/1766657963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c31f80 0x560227c32360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 -- 172.21.15.18:0/1766657963 >> 172.21.15.18:0/1766657963 conn(0x560227c2c980 msgr2=0x560227c2cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 -- 172.21.15.18:0/1766657963 shutdown_connections 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 -- 172.21.15.18:0/1766657963 wait complete. 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 -- start start 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 Processor -- start 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 0x560227c3ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c328a0 0x560227c3cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c3fb60 0x560227c3d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560227b9a020 con 0x560227c31f80 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022786d430 con 0x560227c3fb60 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560227829e00 con 0x560227c328a0 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f645e740 10 client.0 _ll_register_callbacks cb 0x560227b41cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:53.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f162b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 0x560227c3ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f162b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 0x560227c3ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36342/0 (socket says 172.21.15.18:36342) 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.994+0000 7f09f162b640 1 -- 172.21.15.18:0/159386507 learned_addr learned my addr 172.21.15.18:0/159386507 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09f0e2a640 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c328a0 0x560227c3cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09f1e2c640 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c3fb60 0x560227c3d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09f162b640 1 -- 172.21.15.18:0/159386507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c328a0 msgr2=0x560227c3cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09f162b640 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c328a0 0x560227c3cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09f162b640 1 -- 172.21.15.18:0/159386507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c3fb60 msgr2=0x560227c3d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09f162b640 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c3fb60 0x560227c3d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09f162b640 1 -- 172.21.15.18:0/159386507 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ec10e810 con 0x560227c31f80 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09f1e2c640 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c3fb60 0x560227c3d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09f162b640 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 0x560227c3ca80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009610 tx=0x7f09e401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40e83a479930787 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09d0ff9640 1 -- 172.21.15.18:0/159386507 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e409a740 con 0x560227c31f80 2024-02-17T12:51:53.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09d0ff9640 1 -- 172.21.15.18:0/159386507 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f09e409e890 con 0x560227c31f80 2024-02-17T12:51:53.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09f645e740 1 -- 172.21.15.18:0/159386507 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602278997e0 con 0x560227c31f80 2024-02-17T12:51:53.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09d0ff9640 1 -- 172.21.15.18:0/159386507 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09e40a2ee0 con 0x560227c31f80 2024-02-17T12:51:53.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09f645e740 10 client.16296 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:53.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09f645e740 1 -- 172.21.15.18:0/159386507 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560227b986a0 con 0x560227c31f80 2024-02-17T12:51:53.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09d0ff9640 1 -- 172.21.15.18:0/159386507 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5602278997e0 con 0x560227c31f80 2024-02-17T12:51:53.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.995+0000 7f09d0ff9640 1 -- 172.21.15.18:0/159386507 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f09e4096040 con 0x560227c31f80 2024-02-17T12:51:53.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.996+0000 7f09f645e740 10 client.16296 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:53.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.996+0000 7f09f645e740 10 client.16296 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:51:53.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.996+0000 7f09f645e740 20 client.16296 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:53.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.996+0000 7f09f645e740 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x560227c4fdd0 0x560227c521c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:53.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.996+0000 7f09f645e740 4 client.16296 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.996+0000 7f09f645e740 1 -- 172.21.15.18:0/159386507 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602278d6720 con 0x560227c4fdd0 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.996+0000 7f09f0e2a640 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x560227c4fdd0 0x560227c521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.996+0000 7f09d0ff9640 10 client.16296 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.996+0000 7f09f0e2a640 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x560227c4fdd0 0x560227c521c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=486b6d576802b2ad server_cookie=ddd9a1f5fecd2729 in_seq=0 out_seq=0 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09d0ff9640 1 -- 172.21.15.18:0/159386507 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5602278d6720 con 0x560227c4fdd0 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09d0ff9640 10 client.16296 handle_command_reply: tid=0 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09f645e740 1 client.16296 shutdown 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09f645e740 1 -- 172.21.15.18:0/159386507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 msgr2=0x560227c3ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09f645e740 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 0x560227c3ca80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009610 tx=0x7f09e401e000 comp rx=0 tx=0).stop 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09f645e740 1 -- 172.21.15.18:0/159386507 shutdown_connections 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09f645e740 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x560227c4fdd0 0x560227c521c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09f645e740 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c3fb60 0x560227c3d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09f645e740 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c328a0 0x560227c3cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09f645e740 1 --2- 172.21.15.18:0/159386507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 0x560227c3ca80 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:53.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09d0ff9640 0 client.16296 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:51:53.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09f645e740 1 -- 172.21.15.18:0/159386507 >> 172.21.15.18:0/159386507 conn(0x560227c2c980 msgr2=0x560227c2e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:53.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09f645e740 1 -- 172.21.15.18:0/159386507 shutdown_connections 2024-02-17T12:51:53.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09f645e740 1 -- 172.21.15.18:0/159386507 wait complete. 2024-02-17T12:51:53.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:53.997+0000 7f09f645e740 20 client.16296 trim_cache size 0 max 16384 2024-02-17T12:51:54.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f645e740 1 Processor -- start 2024-02-17T12:51:54.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f645e740 1 -- start start 2024-02-17T12:51:54.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f645e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c31f80 0x560227c3d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f645e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c328a0 0x560227c2ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f645e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 0x560227c469c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f645e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560227b1f130 con 0x560227c3fb60 2024-02-17T12:51:54.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f645e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602278997e0 con 0x560227c31f80 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f645e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560227c3dee0 con 0x560227c328a0 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f1e2c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 0x560227c469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f1e2c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 0x560227c469c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36348/0 (socket says 172.21.15.18:36348) 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f1e2c640 1 -- 172.21.15.18:0/3095906891 learned_addr learned my addr 172.21.15.18:0/3095906891 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f0e2a640 1 --2- 172.21.15.18:0/3095906891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c328a0 0x560227c2ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f1e2c640 1 -- 172.21.15.18:0/3095906891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c328a0 msgr2=0x560227c2ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f162b640 1 --2- 172.21.15.18:0/3095906891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c31f80 0x560227c3d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f1e2c640 1 --2- 172.21.15.18:0/3095906891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c328a0 0x560227c2ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f1e2c640 1 -- 172.21.15.18:0/3095906891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c31f80 msgr2=0x560227c3d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f1e2c640 1 --2- 172.21.15.18:0/3095906891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c31f80 0x560227c3d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.009+0000 7f09f1e2c640 1 -- 172.21.15.18:0/3095906891 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560227b34c20 con 0x560227c3fb60 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09f1e2c640 1 --2- 172.21.15.18:0/3095906891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 0x560227c469c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f09dc012860 tx=0x7f09dc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cffcaedc719dc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09d2ffd640 1 -- 172.21.15.18:0/3095906891 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc004b20 con 0x560227c3fb60 2024-02-17T12:51:54.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09d2ffd640 1 -- 172.21.15.18:0/3095906891 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09dc016070 con 0x560227c3fb60 2024-02-17T12:51:54.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09d2ffd640 1 -- 172.21.15.18:0/3095906891 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0056a0 con 0x560227c3fb60 2024-02-17T12:51:54.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09f645e740 1 -- 172.21.15.18:0/3095906891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 msgr2=0x560227c469c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09f645e740 1 --2- 172.21.15.18:0/3095906891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 0x560227c469c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f09dc012860 tx=0x7f09dc014000 comp rx=0 tx=0).stop 2024-02-17T12:51:54.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09f645e740 1 -- 172.21.15.18:0/3095906891 shutdown_connections 2024-02-17T12:51:54.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09f645e740 1 --2- 172.21.15.18:0/3095906891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c3fb60 0x560227c469c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09f645e740 1 --2- 172.21.15.18:0/3095906891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c328a0 0x560227c2ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09f645e740 1 --2- 172.21.15.18:0/3095906891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c31f80 0x560227c3d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09f645e740 1 -- 172.21.15.18:0/3095906891 >> 172.21.15.18:0/3095906891 conn(0x560227c2c980 msgr2=0x560227c2b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:54.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09f645e740 1 -- 172.21.15.18:0/3095906891 shutdown_connections 2024-02-17T12:51:54.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.010+0000 7f09f645e740 1 -- 172.21.15.18:0/3095906891 wait complete. 2024-02-17T12:51:54.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f645e740 1 -- start start 2024-02-17T12:51:54.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f645e740 1 Processor -- start 2024-02-17T12:51:54.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f645e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 0x560227c45b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f645e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c328a0 0x560227c460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f645e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c3fb60 0x560227ca62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f645e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560227c46840 con 0x560227c31f80 2024-02-17T12:51:54.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f645e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560227ca67e0 con 0x560227c328a0 2024-02-17T12:51:54.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f645e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560227ca69a0 con 0x560227c3fb60 2024-02-17T12:51:54.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f645e740 10 client.0 _ll_register_callbacks cb 0x7f09e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:54.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f162b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 0x560227c45b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f162b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 0x560227c45b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36360/0 (socket says 172.21.15.18:36360) 2024-02-17T12:51:54.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f162b640 1 -- 172.21.15.18:0/4024377992 learned_addr learned my addr 172.21.15.18:0/4024377992 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:54.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f0e2a640 1 --2- 172.21.15.18:0/4024377992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c328a0 0x560227c460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f162b640 1 -- 172.21.15.18:0/4024377992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c3fb60 msgr2=0x560227ca62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f162b640 1 --2- 172.21.15.18:0/4024377992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c3fb60 0x560227ca62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f162b640 1 -- 172.21.15.18:0/4024377992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c328a0 msgr2=0x560227c460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f162b640 1 --2- 172.21.15.18:0/4024377992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c328a0 0x560227c460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.011+0000 7f09f162b640 1 -- 172.21.15.18:0/4024377992 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09dc012490 con 0x560227c31f80 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09f162b640 1 --2- 172.21.15.18:0/4024377992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 0x560227c45b70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f09e4099d00 tx=0x7f09e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae1df6a7aa1d75a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09d0ff9640 1 -- 172.21.15.18:0/4024377992 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e40b3290 con 0x560227c31f80 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09d0ff9640 1 -- 172.21.15.18:0/4024377992 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f09e40b2030 con 0x560227c31f80 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09f645e740 1 -- 172.21.15.18:0/4024377992 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09e0004030 con 0x560227c31f80 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09d0ff9640 1 -- 172.21.15.18:0/4024377992 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09e409a0a0 con 0x560227c31f80 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09f645e740 10 client.16302 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09f645e740 1 -- 172.21.15.18:0/4024377992 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560227b5a8c0 con 0x560227c31f80 2024-02-17T12:51:54.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09d0ff9640 1 -- 172.21.15.18:0/4024377992 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09e407f080 con 0x560227c31f80 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09d0ff9640 1 -- 172.21.15.18:0/4024377992 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f09e4096040 con 0x560227c31f80 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09f645e740 10 client.16302 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09f645e740 10 client.16302 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09f645e740 20 client.16302 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09f645e740 1 --2- 172.21.15.18:0/4024377992 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x560227ca7ee0 0x560227caa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09f645e740 4 client.16302 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09f645e740 1 -- 172.21.15.18:0/4024377992 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560227925890 con 0x560227ca7ee0 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.012+0000 7f09f0e2a640 1 --2- 172.21.15.18:0/4024377992 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x560227ca7ee0 0x560227caa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09d0ff9640 10 client.16302 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f0e2a640 1 --2- 172.21.15.18:0/4024377992 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x560227ca7ee0 0x560227caa300 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f90005a17ce716bb server_cookie=985d83f5e5916ef1 in_seq=0 out_seq=0 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09d0ff9640 1 -- 172.21.15.18:0/4024377992 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560227925890 con 0x560227ca7ee0 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09d0ff9640 10 client.16302 handle_command_reply: tid=0 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f645e740 1 client.16302 shutdown 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f645e740 1 -- 172.21.15.18:0/4024377992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 msgr2=0x560227c45b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f645e740 1 --2- 172.21.15.18:0/4024377992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 0x560227c45b70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f09e4099d00 tx=0x7f09e4005880 comp rx=0 tx=0).stop 2024-02-17T12:51:54.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f645e740 1 -- 172.21.15.18:0/4024377992 shutdown_connections 2024-02-17T12:51:54.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f645e740 1 --2- 172.21.15.18:0/4024377992 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x560227ca7ee0 0x560227caa300 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f645e740 1 --2- 172.21.15.18:0/4024377992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560227c3fb60 0x560227ca62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f645e740 1 --2- 172.21.15.18:0/4024377992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560227c328a0 0x560227c460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f645e740 1 --2- 172.21.15.18:0/4024377992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560227c31f80 0x560227c45b70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09d0ff9640 0 client.16302 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:51:54.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f645e740 1 -- 172.21.15.18:0/4024377992 >> 172.21.15.18:0/4024377992 conn(0x560227c2c980 msgr2=0x560227c2dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:54.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f645e740 1 -- 172.21.15.18:0/4024377992 shutdown_connections 2024-02-17T12:51:54.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f645e740 1 -- 172.21.15.18:0/4024377992 wait complete. 2024-02-17T12:51:54.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.013+0000 7f09f645e740 20 client.16302 trim_cache size 0 max 16384 2024-02-17T12:51:54.012 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:51:54.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.014+0000 7f09f388f640 1 -- 172.21.15.18:0/1373762504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f09d40a2230 msgr2=0x7f09d40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.014+0000 7f09f388f640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f09d40a2230 0x7f09d40a46f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09dc005fd0 tx=0x7f09dc004a40 comp rx=0 tx=0).stop 2024-02-17T12:51:54.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.014+0000 7f09f388f640 1 -- 172.21.15.18:0/1373762504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09ec0685d0 msgr2=0x7f09ec19d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.014+0000 7f09f388f640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09ec0685d0 0x7f09ec19d440 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f09e400d950 tx=0x7f09e400de10 comp rx=0 tx=0).stop 2024-02-17T12:51:54.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.014+0000 7f09f388f640 1 -- 172.21.15.18:0/1373762504 shutdown_connections 2024-02-17T12:51:54.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.014+0000 7f09f388f640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f09d40a2230 0x7f09d40a46f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.014+0000 7f09f388f640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09ec1975f0 0x7f09ec197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.014+0000 7f09f388f640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09ec106430 0x7f09ec19d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.014+0000 7f09f388f640 1 --2- 172.21.15.18:0/1373762504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09ec0685d0 0x7f09ec19d440 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.014+0000 7f09f388f640 1 -- 172.21.15.18:0/1373762504 >> 172.21.15.18:0/1373762504 conn(0x7f09ec077180 msgr2=0x7f09ec0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:54.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.014+0000 7f09f388f640 1 -- 172.21.15.18:0/1373762504 shutdown_connections 2024-02-17T12:51:54.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.014+0000 7f09f388f640 1 -- 172.21.15.18:0/1373762504 wait complete. 2024-02-17T12:51:54.015 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:53 smithi018 ceph-mon[32625]: pgmap v63: 97 pgs: 97 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 521 KiB/s wr, 101 op/s 2024-02-17T12:51:54.015 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:53 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3576442906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:51:54.016 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:51:54.016 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:51:54.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:53 smithi119 ceph-mon[36188]: pgmap v63: 97 pgs: 97 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 521 KiB/s wr, 101 op/s 2024-02-17T12:51:54.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:53 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3576442906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.131+0000 7f962ea9d640 1 -- 172.21.15.18:0/198594505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9628106450 msgr2=0x7f962810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.131+0000 7f962ea9d640 1 --2- 172.21.15.18:0/198594505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9628106450 0x7f962810e020 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9620009cd0 tx=0x7f962002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.131+0000 7f962ea9d640 1 -- 172.21.15.18:0/198594505 shutdown_connections 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.131+0000 7f962ea9d640 1 --2- 172.21.15.18:0/198594505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9628106450 0x7f962810e020 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.131+0000 7f962ea9d640 1 --2- 172.21.15.18:0/198594505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9628101d30 0x7f9628105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.131+0000 7f962ea9d640 1 --2- 172.21.15.18:0/198594505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9628101380 0x7f9628101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.131+0000 7f962ea9d640 1 -- 172.21.15.18:0/198594505 >> 172.21.15.18:0/198594505 conn(0x7f96280fcc20 msgr2=0x7f96280ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.131+0000 7f962ea9d640 1 -- 172.21.15.18:0/198594505 shutdown_connections 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.131+0000 7f962ea9d640 1 -- 172.21.15.18:0/198594505 wait complete. 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962ea9d640 1 Processor -- start 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962ea9d640 1 -- start start 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962ea9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9628101380 0x7f962819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962ea9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9628106450 0x7f962819fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962ea9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96281997e0 0x7f9628199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962ea9d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962819a290 con 0x7f9628101380 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962ea9d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962819a400 con 0x7f96281997e0 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962ea9d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962819a570 con 0x7f9628106450 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962c839640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9628101380 0x7f962819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962c839640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9628101380 0x7f962819f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36378/0 (socket says 172.21.15.18:36378) 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962c839640 1 -- 172.21.15.18:0/1438539538 learned_addr learned my addr 172.21.15.18:0/1438539538 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962c839640 1 -- 172.21.15.18:0/1438539538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9628106450 msgr2=0x7f962819fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962c839640 1 --2- 172.21.15.18:0/1438539538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9628106450 0x7f962819fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962c839640 1 -- 172.21.15.18:0/1438539538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96281997e0 msgr2=0x7f9628199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962c839640 1 --2- 172.21.15.18:0/1438539538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96281997e0 0x7f9628199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962c839640 1 -- 172.21.15.18:0/1438539538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9620009900 con 0x7f9628101380 2024-02-17T12:51:54.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.132+0000 7f962c839640 1 --2- 172.21.15.18:0/1438539538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9628101380 0x7f962819f670 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f961c00c920 tx=0x7f961c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:54.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.133+0000 7f9625ffb640 1 -- 172.21.15.18:0/1438539538 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c007c70 con 0x7f9628101380 2024-02-17T12:51:54.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.133+0000 7f962ea9d640 1 -- 172.21.15.18:0/1438539538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96281a6520 con 0x7f9628101380 2024-02-17T12:51:54.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.133+0000 7f9625ffb640 1 -- 172.21.15.18:0/1438539538 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f961c007dd0 con 0x7f9628101380 2024-02-17T12:51:54.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.133+0000 7f9625ffb640 1 -- 172.21.15.18:0/1438539538 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c018860 con 0x7f9628101380 2024-02-17T12:51:54.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.133+0000 7f962ea9d640 1 -- 172.21.15.18:0/1438539538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96281a6a30 con 0x7f9628101380 2024-02-17T12:51:54.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.135+0000 7f9625ffb640 1 -- 172.21.15.18:0/1438539538 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f961c00fac0 con 0x7f9628101380 2024-02-17T12:51:54.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.135+0000 7f963167b740 1 Processor -- start 2024-02-17T12:51:54.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.135+0000 7f9625ffb640 1 --2- 172.21.15.18:0/1438539538 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96180a2230 0x7f96180a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.135+0000 7f963167b740 1 -- start start 2024-02-17T12:51:54.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9625ffb640 1 -- 172.21.15.18:0/1438539538 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f961c0ce2b0 con 0x7f9628101380 2024-02-17T12:51:54.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f962d03a640 1 --2- 172.21.15.18:0/1438539538 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96180a2230 0x7f96180a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f963167b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1250 0x558ad77b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f963167b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1b70 0x558ad77ba210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f963167b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77ba750 0x558ad77bcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f963167b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad776c960 con 0x558ad77b1b70 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f963167b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad7720310 con 0x558ad77ba750 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f963167b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad771d840 con 0x558ad77b1250 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9627fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1b70 0x558ad77ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9627fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1b70 0x558ad77ba210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36388/0 (socket says 172.21.15.18:36388) 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9627fff640 1 -- 172.21.15.18:0/2057600339 learned_addr learned my addr 172.21.15.18:0/2057600339 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f962d03a640 1 --2- 172.21.15.18:0/2057600339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77ba750 0x558ad77bcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9627fff640 1 -- 172.21.15.18:0/2057600339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1250 msgr2=0x558ad77b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9627fff640 1 --2- 172.21.15.18:0/2057600339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1250 0x558ad77b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9627fff640 1 -- 172.21.15.18:0/2057600339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77ba750 msgr2=0x558ad77bcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:54.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9627fff640 1 --2- 172.21.15.18:0/2057600339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77ba750 0x558ad77bcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f962d03a640 1 --2- 172.21.15.18:0/1438539538 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96180a2230 0x7f96180a46f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9620009430 tx=0x7f9620032ed0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9627fff640 1 -- 172.21.15.18:0/2057600339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ad76db060 con 0x558ad77b1b70 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9627fff640 1 --2- 172.21.15.18:0/2057600339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1b70 0x558ad77ba210 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f961400b9f0 tx=0x7f961400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e29b72f50d80c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9612ffd640 1 -- 172.21.15.18:0/2057600339 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961400dbe0 con 0x558ad77b1b70 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9612ffd640 1 -- 172.21.15.18:0/2057600339 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f961400b4d0 con 0x558ad77b1b70 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.136+0000 7f9612ffd640 1 -- 172.21.15.18:0/2057600339 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961400f730 con 0x558ad77b1b70 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.137+0000 7f963167b740 1 -- 172.21.15.18:0/2057600339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1b70 msgr2=0x558ad77ba210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.137+0000 7f963167b740 1 --2- 172.21.15.18:0/2057600339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1b70 0x558ad77ba210 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f961400b9f0 tx=0x7f961400beb0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.137+0000 7f963167b740 1 -- 172.21.15.18:0/2057600339 shutdown_connections 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.137+0000 7f963167b740 1 --2- 172.21.15.18:0/2057600339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77ba750 0x558ad77bcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.137+0000 7f963167b740 1 --2- 172.21.15.18:0/2057600339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1b70 0x558ad77ba210 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.137+0000 7f963167b740 1 --2- 172.21.15.18:0/2057600339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1250 0x558ad77b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.137+0000 7f963167b740 1 -- 172.21.15.18:0/2057600339 >> 172.21.15.18:0/2057600339 conn(0x558ad77ab980 msgr2=0x558ad77abd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.137+0000 7f963167b740 1 -- 172.21.15.18:0/2057600339 shutdown_connections 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.137+0000 7f963167b740 1 -- 172.21.15.18:0/2057600339 wait complete. 2024-02-17T12:51:54.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.137+0000 7f963167b740 1 -- start start 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f963167b740 1 Processor -- start 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f963167b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1250 0x558ad77b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f963167b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77b1b70 0x558ad77b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f963167b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 0x558ad77b8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f963167b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad7719020 con 0x558ad77ba750 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f963167b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad73ec430 con 0x558ad77b1b70 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f963167b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad73a8e00 con 0x558ad77b1250 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f963167b740 10 client.0 _ll_register_callbacks cb 0x558ad76c0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f962d03a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 0x558ad77b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f962d03a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 0x558ad77b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36400/0 (socket says 172.21.15.18:36400) 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f962d03a640 1 -- 172.21.15.18:0/3789502820 learned_addr learned my addr 172.21.15.18:0/3789502820 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f9627fff640 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77b1b70 0x558ad77b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f962c839640 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1250 0x558ad77b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f962d03a640 1 -- 172.21.15.18:0/3789502820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1250 msgr2=0x558ad77b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f962d03a640 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1250 0x558ad77b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f962d03a640 1 -- 172.21.15.18:0/3789502820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77b1b70 msgr2=0x558ad77b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f962d03a640 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77b1b70 0x558ad77b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.138+0000 7f962d03a640 1 -- 172.21.15.18:0/3789502820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961400b650 con 0x558ad77ba750 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f962d03a640 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 0x558ad77b8260 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9620009cd0 tx=0x7f96200095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95931e418504a867 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f9607fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f9607fff640 1 -- 172.21.15.18:0/3789502820 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620003970 con 0x558ad77ba750 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f9607fff640 1 -- 172.21.15.18:0/3789502820 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9620003ad0 con 0x558ad77ba750 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f963167b740 1 -- 172.21.15.18:0/3789502820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ad7455720 con 0x558ad77ba750 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f9607fff640 1 -- 172.21.15.18:0/3789502820 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9620003c90 con 0x558ad77ba750 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f963167b740 10 client.16314 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f963167b740 1 -- 172.21.15.18:0/3789502820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558ad77176a0 con 0x558ad77ba750 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f9607fff640 1 -- 172.21.15.18:0/3789502820 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558ad7455720 con 0x558ad77ba750 2024-02-17T12:51:54.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f9607fff640 1 -- 172.21.15.18:0/3789502820 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9620052060 con 0x558ad77ba750 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f963167b740 10 client.16314 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f963167b740 10 client.16314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f963167b740 20 client.16314 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f963167b740 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558ad77cad30 0x558ad77cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f963167b740 4 client.16314 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.139+0000 7f963167b740 1 -- 172.21.15.18:0/3789502820 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ad74a4890 con 0x558ad77cad30 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.140+0000 7f962c839640 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558ad77cad30 0x558ad77cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.140+0000 7f962c839640 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558ad77cad30 0x558ad77cd150 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9ea9e5e2ce2b71 server_cookie=e8e67d2b44b0f076 in_seq=0 out_seq=0 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.140+0000 7f9607fff640 10 client.16314 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f9607fff640 1 -- 172.21.15.18:0/3789502820 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ad74a4890 con 0x558ad77cad30 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f9607fff640 10 client.16314 handle_command_reply: tid=0 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f963167b740 1 client.16314 shutdown 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f963167b740 1 -- 172.21.15.18:0/3789502820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 msgr2=0x558ad77b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f963167b740 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 0x558ad77b8260 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9620009cd0 tx=0x7f96200095c0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f963167b740 1 -- 172.21.15.18:0/3789502820 shutdown_connections 2024-02-17T12:51:54.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f963167b740 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558ad77cad30 0x558ad77cd150 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f963167b740 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 0x558ad77b8260 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f963167b740 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77b1b70 0x558ad77b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f963167b740 1 --2- 172.21.15.18:0/3789502820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1250 0x558ad77b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f9607fff640 0 client.16314 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:54.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f963167b740 1 -- 172.21.15.18:0/3789502820 >> 172.21.15.18:0/3789502820 conn(0x558ad77ab980 msgr2=0x558ad77ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:54.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f963167b740 1 -- 172.21.15.18:0/3789502820 shutdown_connections 2024-02-17T12:51:54.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f963167b740 1 -- 172.21.15.18:0/3789502820 wait complete. 2024-02-17T12:51:54.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.141+0000 7f963167b740 20 client.16314 trim_cache size 0 max 16384 2024-02-17T12:51:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.152+0000 7f963167b740 1 Processor -- start 2024-02-17T12:51:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f963167b740 1 -- start start 2024-02-17T12:51:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f963167b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1250 0x558ad77b9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f963167b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1b70 0x558ad77b8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f963167b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77ba750 0x558ad77b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f963167b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad769e130 con 0x558ad77b1250 2024-02-17T12:51:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f963167b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad77c76c0 con 0x558ad77ba750 2024-02-17T12:51:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f963167b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad77b0370 con 0x558ad77b1b70 2024-02-17T12:51:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f962c839640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1250 0x558ad77b9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f962c839640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1250 0x558ad77b9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36402/0 (socket says 172.21.15.18:36402) 2024-02-17T12:51:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f962c839640 1 -- 172.21.15.18:0/1570787819 learned_addr learned my addr 172.21.15.18:0/1570787819 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f9627fff640 1 --2- 172.21.15.18:0/1570787819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1b70 0x558ad77b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f962c839640 1 -- 172.21.15.18:0/1570787819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1b70 msgr2=0x558ad77b8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f962c839640 1 --2- 172.21.15.18:0/1570787819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1b70 0x558ad77b8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f962c839640 1 -- 172.21.15.18:0/1570787819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77ba750 msgr2=0x558ad77b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f962d03a640 1 --2- 172.21.15.18:0/1570787819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77ba750 0x558ad77b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f962c839640 1 --2- 172.21.15.18:0/1570787819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77ba750 0x558ad77b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f962c839640 1 -- 172.21.15.18:0/1570787819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ad76b3c20 con 0x558ad77b1250 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f962d03a640 1 --2- 172.21.15.18:0/1570787819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77ba750 0x558ad77b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f962c839640 1 --2- 172.21.15.18:0/1570787819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1250 0x558ad77b9f10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f961c01f4b0 tx=0x7f961c00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a86942588f46f2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f96127fc640 1 -- 172.21.15.18:0/1570787819 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c0a3430 con 0x558ad77b1250 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f96127fc640 1 -- 172.21.15.18:0/1570787819 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f961c0cd040 con 0x558ad77b1250 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.153+0000 7f96127fc640 1 -- 172.21.15.18:0/1570787819 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c097490 con 0x558ad77b1250 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.154+0000 7f963167b740 1 -- 172.21.15.18:0/1570787819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1250 msgr2=0x558ad77b9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.154+0000 7f963167b740 1 --2- 172.21.15.18:0/1570787819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1250 0x558ad77b9f10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f961c01f4b0 tx=0x7f961c00fec0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.154+0000 7f963167b740 1 -- 172.21.15.18:0/1570787819 shutdown_connections 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.154+0000 7f963167b740 1 --2- 172.21.15.18:0/1570787819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77ba750 0x558ad77b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.154+0000 7f963167b740 1 --2- 172.21.15.18:0/1570787819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1b70 0x558ad77b8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.154+0000 7f963167b740 1 --2- 172.21.15.18:0/1570787819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77b1250 0x558ad77b9f10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.154+0000 7f963167b740 1 -- 172.21.15.18:0/1570787819 >> 172.21.15.18:0/1570787819 conn(0x558ad77ab980 msgr2=0x558ad77b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.154+0000 7f963167b740 1 -- 172.21.15.18:0/1570787819 shutdown_connections 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.154+0000 7f963167b740 1 -- 172.21.15.18:0/1570787819 wait complete. 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.154+0000 7f963167b740 1 -- start start 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.154+0000 7f963167b740 1 Processor -- start 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f963167b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77b1250 0x558ad77c0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f963167b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1b70 0x558ad77c0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f963167b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 0x558ad77c0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f963167b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad78213b0 con 0x558ad77ba750 2024-02-17T12:51:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f963167b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad7821520 con 0x558ad77b1250 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f963167b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad78216e0 con 0x558ad77b1b70 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f963167b740 10 client.0 _ll_register_callbacks cb 0x7f9614005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f962d03a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 0x558ad77c0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f962d03a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 0x558ad77c0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36404/0 (socket says 172.21.15.18:36404) 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f962d03a640 1 -- 172.21.15.18:0/1592248971 learned_addr learned my addr 172.21.15.18:0/1592248971 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f962c839640 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77b1250 0x558ad77c0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f9627fff640 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1b70 0x558ad77c0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f962d03a640 1 -- 172.21.15.18:0/1592248971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1b70 msgr2=0x558ad77c0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f962d03a640 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1b70 0x558ad77c0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f962d03a640 1 -- 172.21.15.18:0/1592248971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77b1250 msgr2=0x558ad77c0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f962d03a640 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77b1250 0x558ad77c0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f962d03a640 1 -- 172.21.15.18:0/1592248971 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96281a6a30 con 0x558ad77ba750 2024-02-17T12:51:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f9627fff640 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1b70 0x558ad77c0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f962d03a640 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 0x558ad77c0cc0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9620032ea0 tx=0x7f9620005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45149912824a044e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f9607fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.155+0000 7f9607fff640 1 -- 172.21.15.18:0/1592248971 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962002cea0 con 0x558ad77ba750 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f9607fff640 1 -- 172.21.15.18:0/1592248971 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9620003970 con 0x558ad77ba750 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f963167b740 1 -- 172.21.15.18:0/1592248971 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ad74187e0 con 0x558ad77ba750 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f9607fff640 1 -- 172.21.15.18:0/1592248971 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9620032a80 con 0x558ad77ba750 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f963167b740 10 client.16320 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f963167b740 1 -- 172.21.15.18:0/1592248971 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558ad76d98c0 con 0x558ad77ba750 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f9607fff640 1 -- 172.21.15.18:0/1592248971 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558ad74187e0 con 0x558ad77ba750 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f9607fff640 1 -- 172.21.15.18:0/1592248971 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9620052040 con 0x558ad77ba750 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f963167b740 10 client.16320 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f963167b740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f963167b740 20 client.16320 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f963167b740 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558ad7822c90 0x558ad78250b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f963167b740 4 client.16320 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f963167b740 1 -- 172.21.15.18:0/1592248971 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558ad73c4700 con 0x558ad7822c90 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.156+0000 7f962c839640 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558ad7822c90 0x558ad78250b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f962c839640 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558ad7822c90 0x558ad78250b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7afdef1f7ef0126 server_cookie=25234608c9094159 in_seq=0 out_seq=0 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f9607fff640 10 client.16320 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f9607fff640 1 -- 172.21.15.18:0/1592248971 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558ad73c4700 con 0x558ad7822c90 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f9607fff640 10 client.16320 handle_command_reply: tid=0 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f963167b740 1 client.16320 shutdown 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f963167b740 1 -- 172.21.15.18:0/1592248971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 msgr2=0x558ad77c0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f963167b740 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 0x558ad77c0cc0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9620032ea0 tx=0x7f9620005bd0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f963167b740 1 -- 172.21.15.18:0/1592248971 shutdown_connections 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f963167b740 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558ad7822c90 0x558ad78250b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f963167b740 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558ad77ba750 0x558ad77c0cc0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f9607fff640 0 client.16320 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:54.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f963167b740 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ad77b1b70 0x558ad77c0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f963167b740 1 --2- 172.21.15.18:0/1592248971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ad77b1250 0x558ad77c0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f963167b740 1 -- 172.21.15.18:0/1592248971 >> 172.21.15.18:0/1592248971 conn(0x558ad77ab980 msgr2=0x558ad77acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:54.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f963167b740 1 -- 172.21.15.18:0/1592248971 shutdown_connections 2024-02-17T12:51:54.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f963167b740 1 -- 172.21.15.18:0/1592248971 wait complete. 2024-02-17T12:51:54.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.157+0000 7f963167b740 20 client.16320 trim_cache size 0 max 16384 2024-02-17T12:51:54.157 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:51:54.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.158+0000 7f962ea9d640 1 -- 172.21.15.18:0/1438539538 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96180a2230 msgr2=0x7f96180a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.158+0000 7f962ea9d640 1 --2- 172.21.15.18:0/1438539538 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96180a2230 0x7f96180a46f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9620009430 tx=0x7f9620032ed0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.158+0000 7f962ea9d640 1 -- 172.21.15.18:0/1438539538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9628101380 msgr2=0x7f962819f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:54.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.158+0000 7f962ea9d640 1 --2- 172.21.15.18:0/1438539538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9628101380 0x7f962819f670 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f961c00c920 tx=0x7f961c00cde0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.158+0000 7f962ea9d640 1 -- 172.21.15.18:0/1438539538 shutdown_connections 2024-02-17T12:51:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.158+0000 7f962ea9d640 1 --2- 172.21.15.18:0/1438539538 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96180a2230 0x7f96180a46f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.158+0000 7f962ea9d640 1 --2- 172.21.15.18:0/1438539538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96281997e0 0x7f9628199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.158+0000 7f962ea9d640 1 --2- 172.21.15.18:0/1438539538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9628106450 0x7f962819fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.158+0000 7f962ea9d640 1 --2- 172.21.15.18:0/1438539538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9628101380 0x7f962819f670 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.158+0000 7f962ea9d640 1 -- 172.21.15.18:0/1438539538 >> 172.21.15.18:0/1438539538 conn(0x7f96280fcc20 msgr2=0x7f96280fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.158+0000 7f962ea9d640 1 -- 172.21.15.18:0/1438539538 shutdown_connections 2024-02-17T12:51:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:54.158+0000 7f962ea9d640 1 -- 172.21.15.18:0/1438539538 wait complete. 2024-02-17T12:51:54.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:53 smithi146 ceph-mon[36454]: pgmap v63: 97 pgs: 97 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 521 KiB/s wr, 101 op/s 2024-02-17T12:51:54.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:53 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3576442906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:51:54.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 64592 5.30 MB/sec warmup 69 sec latency 1676.250 ms 2024-02-17T12:51:55.160 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:51:55.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:51:55.161 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:51:55.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 64592 5.22 MB/sec warmup 70 sec latency 2676.315 ms 2024-02-17T12:51:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.288+0000 7fa0a80eb640 1 -- 172.21.15.18:0/4053672881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a01023c0 msgr2=0x7fa0a01027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.288+0000 7fa0a80eb640 1 --2- 172.21.15.18:0/4053672881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a01023c0 0x7fa0a01027a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa098009ca0 tx=0x7fa09802c180 comp rx=0 tx=0).stop 2024-02-17T12:51:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.288+0000 7fa0a80eb640 1 -- 172.21.15.18:0/4053672881 shutdown_connections 2024-02-17T12:51:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.288+0000 7fa0a80eb640 1 --2- 172.21.15.18:0/4053672881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0a010b790 0x7fa0a010dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 --2- 172.21.15.18:0/4053672881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0a0102ce0 0x7fa0a010b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 --2- 172.21.15.18:0/4053672881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a01023c0 0x7fa0a01027a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 -- 172.21.15.18:0/4053672881 >> 172.21.15.18:0/4053672881 conn(0x7fa0a0076400 msgr2=0x7fa0a0076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 -- 172.21.15.18:0/4053672881 shutdown_connections 2024-02-17T12:51:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 -- 172.21.15.18:0/4053672881 wait complete. 2024-02-17T12:51:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 Processor -- start 2024-02-17T12:51:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 -- start start 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a01023c0 0x7fa0a0100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0a0102ce0 0x7fa0a0100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0a010b790 0x7fa0a0100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a01a5b80 con 0x7fa0a01023c0 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a01a5cf0 con 0x7fa0a010b790 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.289+0000 7fa0a80eb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a01a5eb0 con 0x7fa0a0102ce0 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa0a5e87640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a01023c0 0x7fa0a0100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa0a5e87640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a01023c0 0x7fa0a0100010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36422/0 (socket says 172.21.15.18:36422) 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa0a5e87640 1 -- 172.21.15.18:0/2216281149 learned_addr learned my addr 172.21.15.18:0/2216281149 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa0a6688640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0a010b790 0x7fa0a0100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa0a5e87640 1 -- 172.21.15.18:0/2216281149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0a0102ce0 msgr2=0x7fa0a0100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0a0102ce0 0x7fa0a0100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa0a5e87640 1 -- 172.21.15.18:0/2216281149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0a010b790 msgr2=0x7fa0a0100a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0a010b790 0x7fa0a0100a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa0a5e87640 1 -- 172.21.15.18:0/2216281149 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa098009900 con 0x7fa0a01023c0 2024-02-17T12:51:55.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a01023c0 0x7fa0a0100010 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa098005e30 tx=0x7fa098034dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:55.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa09effd640 1 -- 172.21.15.18:0/2216281149 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098032790 con 0x7fa0a01023c0 2024-02-17T12:51:55.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa09effd640 1 -- 172.21.15.18:0/2216281149 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa098032dd0 con 0x7fa0a01023c0 2024-02-17T12:51:55.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa09effd640 1 -- 172.21.15.18:0/2216281149 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098040450 con 0x7fa0a01023c0 2024-02-17T12:51:55.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa0a80eb640 1 -- 172.21.15.18:0/2216281149 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a01a6130 con 0x7fa0a01023c0 2024-02-17T12:51:55.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.290+0000 7fa0a80eb640 1 -- 172.21.15.18:0/2216281149 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a01a6560 con 0x7fa0a01023c0 2024-02-17T12:51:55.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.292+0000 7fa09effd640 1 -- 172.21.15.18:0/2216281149 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa09800a040 con 0x7fa0a01023c0 2024-02-17T12:51:55.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0aad97740 1 Processor -- start 2024-02-17T12:51:55.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0aad97740 1 -- start start 2024-02-17T12:51:55.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0aad97740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x55644935f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0aad97740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 0x5564493682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa09effd640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0880a2230 0x7fa0880a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0aad97740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564493688b0 0x55644936ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0aad97740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564492cdf90 con 0x55644935fd30 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0aad97740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564492cb4c0 con 0x5564493688b0 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0aad97740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564492c6ca0 con 0x55644935f380 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0a5e87640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 0x5564493682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0a5686640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0880a2230 0x7fa0880a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa09effd640 1 -- 172.21.15.18:0/2216281149 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa0980f0a60 con 0x7fa0a01023c0 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0a6688640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x55644935f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0a5e87640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 0x5564493682e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36438/0 (socket says 172.21.15.18:36438) 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0a5e87640 1 -- 172.21.15.18:0/1198159713 learned_addr learned my addr 172.21.15.18:0/1198159713 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.293+0000 7fa0a6688640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x55644935f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:39126/0 (socket says 172.21.15.18:39126) 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.294+0000 7fa0a6688640 1 --2- 172.21.15.18:0/1198159713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564493688b0 0x55644936ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.294+0000 7fa0a5e87640 1 -- 172.21.15.18:0/1198159713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 msgr2=0x55644935f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.294+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/1198159713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x55644935f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.294+0000 7fa0a5e87640 1 -- 172.21.15.18:0/1198159713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564493688b0 msgr2=0x55644936ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.294+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/1198159713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564493688b0 0x55644936ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.294+0000 7fa0a5e87640 1 -- 172.21.15.18:0/1198159713 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564492890b0 con 0x55644935fd30 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.294+0000 7fa0a6688640 1 --2- 172.21.15.18:0/1198159713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564493688b0 0x55644936ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.294+0000 7fa0a5686640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0880a2230 0x7fa0880a46f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa094005fd0 tx=0x7fa094005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.294+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/1198159713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 0x5564493682e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa0980ee800 tx=0x7fa098088aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01753a95690ad50 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.294+0000 7fa087fff640 1 -- 172.21.15.18:0/1198159713 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0980bd0f0 con 0x55644935fd30 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.294+0000 7fa087fff640 1 -- 172.21.15.18:0/1198159713 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0980bcf10 con 0x55644935fd30 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.294+0000 7fa087fff640 1 -- 172.21.15.18:0/1198159713 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0980c5d50 con 0x55644935fd30 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.295+0000 7fa0aad97740 1 -- 172.21.15.18:0/1198159713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 msgr2=0x5564493682e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.295+0000 7fa0aad97740 1 --2- 172.21.15.18:0/1198159713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 0x5564493682e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa0980ee800 tx=0x7fa098088aa0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.295+0000 7fa0aad97740 1 -- 172.21.15.18:0/1198159713 shutdown_connections 2024-02-17T12:51:55.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.295+0000 7fa0aad97740 1 --2- 172.21.15.18:0/1198159713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564493688b0 0x55644936ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.295+0000 7fa0aad97740 1 --2- 172.21.15.18:0/1198159713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 0x5564493682e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.295+0000 7fa0aad97740 1 --2- 172.21.15.18:0/1198159713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x55644935f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.295+0000 7fa0aad97740 1 -- 172.21.15.18:0/1198159713 >> 172.21.15.18:0/1198159713 conn(0x556449359840 msgr2=0x556449359c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.295+0000 7fa0aad97740 1 -- 172.21.15.18:0/1198159713 shutdown_connections 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.295+0000 7fa0aad97740 1 -- 172.21.15.18:0/1198159713 wait complete. 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.295+0000 7fa0aad97740 1 -- start start 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.295+0000 7fa0aad97740 1 Processor -- start 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0aad97740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935f380 0x556449365700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0aad97740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 0x556449365c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0aad97740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5564493688b0 0x556449366180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0aad97740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644926bf40 con 0x55644935fd30 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0aad97740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556448f9a4b0 con 0x55644935f380 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0aad97740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644924c670 con 0x5564493688b0 2024-02-17T12:51:55.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0aad97740 10 client.0 _ll_register_callbacks cb 0x5564492b9f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:55.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0a5686640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 0x556449365c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0a5686640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 0x556449365c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36454/0 (socket says 172.21.15.18:36454) 2024-02-17T12:51:55.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0a5686640 1 -- 172.21.15.18:0/3159513015 learned_addr learned my addr 172.21.15.18:0/3159513015 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:55.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0a6688640 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5564493688b0 0x556449366180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935f380 0x556449365700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0a5686640 1 -- 172.21.15.18:0/3159513015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5564493688b0 msgr2=0x556449366180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0a5686640 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5564493688b0 0x556449366180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0a5686640 1 -- 172.21.15.18:0/3159513015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935f380 msgr2=0x556449365700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0a5686640 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935f380 0x556449365700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.296+0000 7fa0a5686640 1 -- 172.21.15.18:0/3159513015 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a01a6560 con 0x55644935fd30 2024-02-17T12:51:55.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0a5686640 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 0x556449365c40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa09400df20 tx=0x7fa0940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4c05134a93d662 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:55.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0857fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0857fa640 1 -- 172.21.15.18:0/3159513015 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094018040 con 0x55644935fd30 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0857fa640 1 -- 172.21.15.18:0/3159513015 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa09401d7e0 con 0x55644935fd30 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0aad97740 1 -- 172.21.15.18:0/3159513015 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564492a0f40 con 0x55644935fd30 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0857fa640 1 -- 172.21.15.18:0/3159513015 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa09401d9a0 con 0x55644935fd30 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0aad97740 10 client.16332 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0aad97740 1 -- 172.21.15.18:0/3159513015 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5564492c5320 con 0x55644935fd30 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0857fa640 1 -- 172.21.15.18:0/3159513015 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5564492a0f40 con 0x55644935fd30 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0857fa640 1 -- 172.21.15.18:0/3159513015 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa09401ec60 con 0x55644935fd30 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0aad97740 10 client.16332 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0aad97740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0aad97740 20 client.16332 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0aad97740 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556449378b50 0x55644937af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0aad97740 4 client.16332 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.297+0000 7fa0aad97740 1 -- 172.21.15.18:0/3159513015 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564492bd310 con 0x556449378b50 2024-02-17T12:51:55.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.298+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556449378b50 0x55644937af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.298+0000 7fa0857fa640 10 client.16332 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.298+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556449378b50 0x55644937af70 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=461aa6eae71f8dca server_cookie=aaceff244c67e8cd in_seq=0 out_seq=0 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0857fa640 1 -- 172.21.15.18:0/3159513015 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5564492bd310 con 0x556449378b50 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0857fa640 10 client.16332 handle_command_reply: tid=0 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0aad97740 1 client.16332 shutdown 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0aad97740 1 -- 172.21.15.18:0/3159513015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 msgr2=0x556449365c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0aad97740 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 0x556449365c40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa09400df20 tx=0x7fa0940046c0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0aad97740 1 -- 172.21.15.18:0/3159513015 shutdown_connections 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0aad97740 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556449378b50 0x55644937af70 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0aad97740 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5564493688b0 0x556449366180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0aad97740 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55644935fd30 0x556449365c40 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0aad97740 1 --2- 172.21.15.18:0/3159513015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935f380 0x556449365700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0857fa640 0 client.16332 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0aad97740 1 -- 172.21.15.18:0/3159513015 >> 172.21.15.18:0/3159513015 conn(0x556449359840 msgr2=0x55644935b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:55.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0aad97740 1 -- 172.21.15.18:0/3159513015 shutdown_connections 2024-02-17T12:51:55.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0aad97740 1 -- 172.21.15.18:0/3159513015 wait complete. 2024-02-17T12:51:55.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.299+0000 7fa0aad97740 20 client.16332 trim_cache size 0 max 16384 2024-02-17T12:51:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.314+0000 7fa0aad97740 1 Processor -- start 2024-02-17T12:51:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.314+0000 7fa0aad97740 1 -- start start 2024-02-17T12:51:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.314+0000 7fa0aad97740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x556449364880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.314+0000 7fa0aad97740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935fd30 0x556449365fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.314+0000 7fa0aad97740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 0x55644936f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.314+0000 7fa0aad97740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564492a0f40 con 0x5564493688b0 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.314+0000 7fa0aad97740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564493596d0 con 0x55644935fd30 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.314+0000 7fa0aad97740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644935a640 con 0x55644935f380 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.314+0000 7fa0a6688640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 0x55644936f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.314+0000 7fa0a6688640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 0x55644936f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36468/0 (socket says 172.21.15.18:36468) 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.314+0000 7fa0a6688640 1 -- 172.21.15.18:0/3031516524 learned_addr learned my addr 172.21.15.18:0/3031516524 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.315+0000 7fa0a5686640 1 --2- 172.21.15.18:0/3031516524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935fd30 0x556449365fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.315+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/3031516524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x556449364880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.315+0000 7fa0a6688640 1 -- 172.21.15.18:0/3031516524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 msgr2=0x556449364880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.315+0000 7fa0a6688640 1 --2- 172.21.15.18:0/3031516524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x556449364880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.315+0000 7fa0a6688640 1 -- 172.21.15.18:0/3031516524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935fd30 msgr2=0x556449365fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.315+0000 7fa0a6688640 1 --2- 172.21.15.18:0/3031516524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935fd30 0x556449365fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.315+0000 7fa0a6688640 1 -- 172.21.15.18:0/3031516524 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55644935f1f0 con 0x5564493688b0 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.315+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/3031516524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x556449364880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.315+0000 7fa0a6688640 1 --2- 172.21.15.18:0/3031516524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 0x55644936f6d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa09000cfb0 tx=0x7fa0900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc348871491c2ce server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.315+0000 7fa0877fe640 1 -- 172.21.15.18:0/3031516524 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090005220 con 0x5564493688b0 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.315+0000 7fa0877fe640 1 -- 172.21.15.18:0/3031516524 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa090019070 con 0x5564493688b0 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.315+0000 7fa0877fe640 1 -- 172.21.15.18:0/3031516524 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0900118c0 con 0x5564493688b0 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 -- 172.21.15.18:0/3031516524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 msgr2=0x55644936f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 --2- 172.21.15.18:0/3031516524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 0x55644936f6d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa09000cfb0 tx=0x7fa0900046c0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 -- 172.21.15.18:0/3031516524 shutdown_connections 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 --2- 172.21.15.18:0/3031516524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 0x55644936f6d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 --2- 172.21.15.18:0/3031516524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935fd30 0x556449365fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 --2- 172.21.15.18:0/3031516524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x556449364880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 -- 172.21.15.18:0/3031516524 >> 172.21.15.18:0/3031516524 conn(0x556449359840 msgr2=0x556449358050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 -- 172.21.15.18:0/3031516524 shutdown_connections 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 -- 172.21.15.18:0/3031516524 wait complete. 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 -- start start 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 Processor -- start 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x55644936eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935fd30 0x55644936f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 0x5564493c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564493c96e0 con 0x5564493688b0 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564493c9850 con 0x55644935fd30 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564493c99c0 con 0x55644935f380 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.316+0000 7fa0aad97740 10 client.0 _ll_register_callbacks cb 0x7fa09400b6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0a6688640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 0x5564493c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0a6688640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 0x5564493c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36474/0 (socket says 172.21.15.18:36474) 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0a6688640 1 -- 172.21.15.18:0/2792875408 learned_addr learned my addr 172.21.15.18:0/2792875408 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0a5686640 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935fd30 0x55644936f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x55644936eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0a6688640 1 -- 172.21.15.18:0/2792875408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 msgr2=0x55644936eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0a6688640 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x55644936eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0a6688640 1 -- 172.21.15.18:0/2792875408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935fd30 msgr2=0x55644936f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0a6688640 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935fd30 0x55644936f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0a6688640 1 -- 172.21.15.18:0/2792875408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09000cd40 con 0x5564493688b0 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0a6688640 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 0x5564493c91a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa090000c00 tx=0x7fa090005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2e9a9a5298906e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0857fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0857fa640 1 -- 172.21.15.18:0/2792875408 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090004c00 con 0x5564493688b0 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0857fa640 1 -- 172.21.15.18:0/2792875408 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa090004d60 con 0x5564493688b0 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0aad97740 1 -- 172.21.15.18:0/2792875408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55644936b1c0 con 0x5564493688b0 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0857fa640 1 -- 172.21.15.18:0/2792875408 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa090011500 con 0x5564493688b0 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0aad97740 10 client.16338 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.317+0000 7fa0aad97740 1 -- 172.21.15.18:0/2792875408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556449261c00 con 0x5564493688b0 2024-02-17T12:51:55.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.318+0000 7fa0857fa640 1 -- 172.21.15.18:0/2792875408 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55644936b1c0 con 0x5564493688b0 2024-02-17T12:51:55.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.318+0000 7fa0857fa640 1 -- 172.21.15.18:0/2792875408 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa090028720 con 0x5564493688b0 2024-02-17T12:51:55.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.318+0000 7fa0aad97740 10 client.16338 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:55.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.318+0000 7fa0aad97740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:55.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.318+0000 7fa0aad97740 20 client.16338 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:55.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.318+0000 7fa0aad97740 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5564493d1180 0x5564493d35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.318+0000 7fa0aad97740 4 client.16338 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:51:55.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.318+0000 7fa0aad97740 1 -- 172.21.15.18:0/2792875408 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5564492ec900 con 0x5564493d1180 2024-02-17T12:51:55.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.318+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5564493d1180 0x5564493d35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.319+0000 7fa0a5e87640 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5564493d1180 0x5564493d35a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cc67a6b42067fdf server_cookie=438d82dd70e036c5 in_seq=0 out_seq=0 2024-02-17T12:51:55.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.319+0000 7fa0857fa640 10 client.16338 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:55.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0857fa640 1 -- 172.21.15.18:0/2792875408 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5564492ec900 con 0x5564493d1180 2024-02-17T12:51:55.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0857fa640 10 client.16338 handle_command_reply: tid=0 2024-02-17T12:51:55.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0aad97740 1 client.16338 shutdown 2024-02-17T12:51:55.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0aad97740 1 -- 172.21.15.18:0/2792875408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 msgr2=0x5564493c91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0aad97740 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 0x5564493c91a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa090000c00 tx=0x7fa090005c70 comp rx=0 tx=0).stop 2024-02-17T12:51:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0aad97740 1 -- 172.21.15.18:0/2792875408 shutdown_connections 2024-02-17T12:51:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0aad97740 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5564493d1180 0x5564493d35a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0aad97740 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5564493688b0 0x5564493c91a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0aad97740 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55644935fd30 0x55644936f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0857fa640 0 client.16338 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0aad97740 1 --2- 172.21.15.18:0/2792875408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55644935f380 0x55644936eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0aad97740 1 -- 172.21.15.18:0/2792875408 >> 172.21.15.18:0/2792875408 conn(0x556449359840 msgr2=0x5564493588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0aad97740 1 -- 172.21.15.18:0/2792875408 shutdown_connections 2024-02-17T12:51:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0aad97740 1 -- 172.21.15.18:0/2792875408 wait complete. 2024-02-17T12:51:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.320+0000 7fa0aad97740 20 client.16338 trim_cache size 0 max 16384 2024-02-17T12:51:55.318 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:51:55.319 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:51:55.319 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "080fc321-59ab-451b-bd89-dbd1da5f2bbd", 2024-02-17T12:51:55.319 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:51:55.319 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:51:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.321+0000 7fa0a80eb640 1 -- 172.21.15.18:0/2216281149 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0880a2230 msgr2=0x7fa0880a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.321+0000 7fa0a80eb640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0880a2230 0x7fa0880a46f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa094005fd0 tx=0x7fa094005850 comp rx=0 tx=0).stop 2024-02-17T12:51:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.321+0000 7fa0a80eb640 1 -- 172.21.15.18:0/2216281149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a01023c0 msgr2=0x7fa0a0100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.321+0000 7fa0a80eb640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a01023c0 0x7fa0a0100010 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa098005e30 tx=0x7fa098034dc0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.321+0000 7fa0a80eb640 1 -- 172.21.15.18:0/2216281149 shutdown_connections 2024-02-17T12:51:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.321+0000 7fa0a80eb640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0880a2230 0x7fa0880a46f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.321+0000 7fa0a80eb640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0a010b790 0x7fa0a0100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.321+0000 7fa0a80eb640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0a0102ce0 0x7fa0a0100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.321+0000 7fa0a80eb640 1 --2- 172.21.15.18:0/2216281149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a01023c0 0x7fa0a0100010 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.321+0000 7fa0a80eb640 1 -- 172.21.15.18:0/2216281149 >> 172.21.15.18:0/2216281149 conn(0x7fa0a0076400 msgr2=0x7fa0a00fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.321+0000 7fa0a80eb640 1 -- 172.21.15.18:0/2216281149 shutdown_connections 2024-02-17T12:51:55.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.321+0000 7fa0a80eb640 1 -- 172.21.15.18:0/2216281149 wait complete. 2024-02-17T12:51:55.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:51:55.325 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:51:55.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.444+0000 7f13b024e640 1 -- 172.21.15.18:0/3402180268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13a80fff20 msgr2=0x7f13a8100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.444+0000 7f13b024e640 1 --2- 172.21.15.18:0/3402180268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13a80fff20 0x7f13a8100300 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009900 tx=0x7f13a402bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:55.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 -- 172.21.15.18:0/3402180268 shutdown_connections 2024-02-17T12:51:55.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 --2- 172.21.15.18:0/3402180268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a810dad0 0x7f13a810fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 --2- 172.21.15.18:0/3402180268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a8100840 0x7f13a810d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 --2- 172.21.15.18:0/3402180268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13a80fff20 0x7f13a8100300 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 -- 172.21.15.18:0/3402180268 >> 172.21.15.18:0/3402180268 conn(0x7f13a8077230 msgr2=0x7f13a8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:55.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 -- 172.21.15.18:0/3402180268 shutdown_connections 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 -- 172.21.15.18:0/3402180268 wait complete. 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 Processor -- start 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 -- start start 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a80fff20 0x7f13a819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13a8100840 0x7f13a819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a810dad0 0x7f13a81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a8199f50 con 0x7f13a8100840 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a819a0c0 con 0x7f13a80fff20 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.445+0000 7f13b024e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a819a280 con 0x7f13a810dad0 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13ad7e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13a8100840 0x7f13a819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13ad7e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13a8100840 0x7f13a819fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36490/0 (socket says 172.21.15.18:36490) 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13ad7e9640 1 -- 172.21.15.18:0/25063877 learned_addr learned my addr 172.21.15.18:0/25063877 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13adfea640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a80fff20 0x7f13a819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13ae7eb640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a810dad0 0x7f13a81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13ad7e9640 1 -- 172.21.15.18:0/25063877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a810dad0 msgr2=0x7f13a81997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a810dad0 0x7f13a81997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13ad7e9640 1 -- 172.21.15.18:0/25063877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a80fff20 msgr2=0x7f13a819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a80fff20 0x7f13a819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13ad7e9640 1 -- 172.21.15.18:0/25063877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a0009840 con 0x7f13a8100840 2024-02-17T12:51:55.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13a8100840 0x7f13a819fbf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f139800e9f0 tx=0x7f139800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:55.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f139effd640 1 -- 172.21.15.18:0/25063877 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398009700 con 0x7f13a8100840 2024-02-17T12:51:55.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13b024e640 1 -- 172.21.15.18:0/25063877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a4009560 con 0x7f13a8100840 2024-02-17T12:51:55.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f13b024e640 1 -- 172.21.15.18:0/25063877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a81a6630 con 0x7f13a8100840 2024-02-17T12:51:55.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f139effd640 1 -- 172.21.15.18:0/25063877 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f139800cd80 con 0x7f13a8100840 2024-02-17T12:51:55.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.446+0000 7f139effd640 1 -- 172.21.15.18:0/25063877 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13980188e0 con 0x7f13a8100840 2024-02-17T12:51:55.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.448+0000 7f139effd640 1 -- 172.21.15.18:0/25063877 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1398018a40 con 0x7f13a8100840 2024-02-17T12:51:55.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13b2e28740 1 Processor -- start 2024-02-17T12:51:55.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13b2e28740 1 -- start start 2024-02-17T12:51:55.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13b2e28740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x557897907690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13b2e28740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557897907bd0 0x557897910270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13b2e28740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 0x557897912ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f139effd640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13940a2230 0x7f13940a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13b2e28740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897876280 con 0x5578979107b0 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13b2e28740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578978737b0 con 0x5578979072b0 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13b2e28740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789786ef90 con 0x557897907bd0 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13ad7e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 0x557897912ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13ae7eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x557897907690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13ad7e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 0x557897912ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36500/0 (socket says 172.21.15.18:36500) 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13ae7eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x557897907690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:58398/0 (socket says 172.21.15.18:58398) 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13ad7e9640 1 -- 172.21.15.18:0/2985440887 learned_addr learned my addr 172.21.15.18:0/2985440887 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13adfea640 1 --2- 172.21.15.18:0/2985440887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557897907bd0 0x557897910270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f13ae7eb640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13940a2230 0x7f13940a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.449+0000 7f139effd640 1 -- 172.21.15.18:0/25063877 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1398014070 con 0x7f13a8100840 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.450+0000 7f13ad7e9640 1 -- 172.21.15.18:0/2985440887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557897907bd0 msgr2=0x557897910270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.450+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/2985440887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557897907bd0 0x557897910270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.450+0000 7f13ad7e9640 1 -- 172.21.15.18:0/2985440887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 msgr2=0x557897907690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.450+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/2985440887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x557897907690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.450+0000 7f13ad7e9640 1 -- 172.21.15.18:0/2985440887 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55789782d570 con 0x5578979107b0 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.450+0000 7f13ae7eb640 1 --2- 172.21.15.18:0/2985440887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x557897907690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.450+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/2985440887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 0x557897912ba0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1398097750 tx=0x7f139809ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f667ada2c7d66901 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.450+0000 7f13ae7eb640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13940a2230 0x7f13940a46f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009fd0 tx=0x7f13a0009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.450+0000 7f1393fff640 1 -- 172.21.15.18:0/2985440887 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398064500 con 0x5578979107b0 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.450+0000 7f1393fff640 1 -- 172.21.15.18:0/2985440887 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f139806d0b0 con 0x5578979107b0 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.450+0000 7f1393fff640 1 -- 172.21.15.18:0/2985440887 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139809eed0 con 0x5578979107b0 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.451+0000 7f13b2e28740 1 -- 172.21.15.18:0/2985440887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 msgr2=0x557897912ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.451+0000 7f13b2e28740 1 --2- 172.21.15.18:0/2985440887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 0x557897912ba0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1398097750 tx=0x7f139809ab50 comp rx=0 tx=0).stop 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.451+0000 7f13b2e28740 1 -- 172.21.15.18:0/2985440887 shutdown_connections 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.451+0000 7f13b2e28740 1 --2- 172.21.15.18:0/2985440887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 0x557897912ba0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.451+0000 7f13b2e28740 1 --2- 172.21.15.18:0/2985440887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557897907bd0 0x557897910270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.451+0000 7f13b2e28740 1 --2- 172.21.15.18:0/2985440887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x557897907690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.451+0000 7f13b2e28740 1 -- 172.21.15.18:0/2985440887 >> 172.21.15.18:0/2985440887 conn(0x557897901990 msgr2=0x557897901d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.451+0000 7f13b2e28740 1 -- 172.21.15.18:0/2985440887 shutdown_connections 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.451+0000 7f13b2e28740 1 -- 172.21.15.18:0/2985440887 wait complete. 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.451+0000 7f13b2e28740 1 -- start start 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.451+0000 7f13b2e28740 1 Processor -- start 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13b2e28740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x5578979100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13b2e28740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 0x55789790d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13b2e28740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578979107b0 0x55789790d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13b2e28740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897542430 con 0x557897907bd0 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13b2e28740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578974fee00 con 0x5578979072b0 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13b2e28740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578977f3fd0 con 0x5578979107b0 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13b2e28740 10 client.0 _ll_register_callbacks cb 0x557897849fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13ad7e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 0x55789790d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13ad7e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 0x55789790d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36512/0 (socket says 172.21.15.18:36512) 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13ad7e9640 1 -- 172.21.15.18:0/4255842190 learned_addr learned my addr 172.21.15.18:0/4255842190 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13adfea640 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x5578979100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13ae7eb640 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578979107b0 0x55789790d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13ad7e9640 1 -- 172.21.15.18:0/4255842190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578979107b0 msgr2=0x55789790d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578979107b0 0x55789790d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13ad7e9640 1 -- 172.21.15.18:0/4255842190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 msgr2=0x5578979100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x5578979100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13ad7e9640 1 -- 172.21.15.18:0/4255842190 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139807ce70 con 0x557897907bd0 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13adfea640 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x5578979100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 0x55789790d470 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f139800e9c0 tx=0x7f139800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c808a6963ff615 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.452+0000 7f13917fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13917fa640 1 -- 172.21.15.18:0/4255842190 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13980120f0 con 0x557897907bd0 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13917fa640 1 -- 172.21.15.18:0/4255842190 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f139809b480 con 0x557897907bd0 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13b2e28740 1 -- 172.21.15.18:0/4255842190 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55789756e7e0 con 0x557897907bd0 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13917fa640 1 -- 172.21.15.18:0/4255842190 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13980a3520 con 0x557897907bd0 2024-02-17T12:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13b2e28740 10 client.16350 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:55.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13b2e28740 1 -- 172.21.15.18:0/4255842190 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55789786d610 con 0x557897907bd0 2024-02-17T12:51:55.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13917fa640 1 -- 172.21.15.18:0/4255842190 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55789756e7e0 con 0x557897907bd0 2024-02-17T12:51:55.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13917fa640 1 -- 172.21.15.18:0/4255842190 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f13980a3090 con 0x557897907bd0 2024-02-17T12:51:55.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13b2e28740 10 client.16350 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:55.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13b2e28740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:55.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13b2e28740 20 client.16350 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:55.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13b2e28740 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5578979206b0 0x557897922ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13b2e28740 4 client.16350 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:55.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.453+0000 7f13b2e28740 1 -- 172.21.15.18:0/4255842190 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578975ab720 con 0x5578979206b0 2024-02-17T12:51:55.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.454+0000 7f13adfea640 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5578979206b0 0x557897922ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.454+0000 7f13adfea640 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5578979206b0 0x557897922ad0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d344c77f445242 server_cookie=3755d4b6eaa9ddf5 in_seq=0 out_seq=0 2024-02-17T12:51:55.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.454+0000 7f13917fa640 10 client.16350 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:55.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.455+0000 7f13917fa640 1 -- 172.21.15.18:0/4255842190 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578975ab720 con 0x5578979206b0 2024-02-17T12:51:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.455+0000 7f13917fa640 10 client.16350 handle_command_reply: tid=0 2024-02-17T12:51:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.455+0000 7f13b2e28740 1 client.16350 shutdown 2024-02-17T12:51:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.455+0000 7f13b2e28740 1 -- 172.21.15.18:0/4255842190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 msgr2=0x55789790d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.455+0000 7f13b2e28740 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 0x55789790d470 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f139800e9c0 tx=0x7f139800c7d0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.455+0000 7f13b2e28740 1 -- 172.21.15.18:0/4255842190 shutdown_connections 2024-02-17T12:51:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.455+0000 7f13b2e28740 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5578979206b0 0x557897922ad0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.455+0000 7f13b2e28740 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578979107b0 0x55789790d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.455+0000 7f13b2e28740 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 0x55789790d470 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.455+0000 7f13917fa640 0 client.16350 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.455+0000 7f13b2e28740 1 --2- 172.21.15.18:0/4255842190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x5578979100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.455+0000 7f13b2e28740 1 -- 172.21.15.18:0/4255842190 >> 172.21.15.18:0/4255842190 conn(0x557897901990 msgr2=0x557897903730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.456+0000 7f13b2e28740 1 -- 172.21.15.18:0/4255842190 shutdown_connections 2024-02-17T12:51:55.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.456+0000 7f13b2e28740 1 -- 172.21.15.18:0/4255842190 wait complete. 2024-02-17T12:51:55.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.456+0000 7f13b2e28740 20 client.16350 trim_cache size 0 max 16384 2024-02-17T12:51:55.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.471+0000 7f13b2e28740 1 Processor -- start 2024-02-17T12:51:55.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.471+0000 7f13b2e28740 1 -- start start 2024-02-17T12:51:55.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.471+0000 7f13b2e28740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578979072b0 0x55789790fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.471+0000 7f13b2e28740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 0x5578979100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13b2e28740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979107b0 0x557897917680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13b2e28740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789790dfb0 con 0x557897907bd0 2024-02-17T12:51:55.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13b2e28740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789790e7c0 con 0x5578979107b0 2024-02-17T12:51:55.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13b2e28740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897906f10 con 0x5578979072b0 2024-02-17T12:51:55.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13ad7e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 0x5578979100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13ad7e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 0x5578979100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36526/0 (socket says 172.21.15.18:36526) 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13ad7e9640 1 -- 172.21.15.18:0/351638709 learned_addr learned my addr 172.21.15.18:0/351638709 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13ae7eb640 1 --2- 172.21.15.18:0/351638709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979107b0 0x557897917680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13adfea640 1 --2- 172.21.15.18:0/351638709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578979072b0 0x55789790fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13ad7e9640 1 -- 172.21.15.18:0/351638709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578979072b0 msgr2=0x55789790fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/351638709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578979072b0 0x55789790fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13ad7e9640 1 -- 172.21.15.18:0/351638709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979107b0 msgr2=0x557897917680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/351638709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979107b0 0x557897917680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13ad7e9640 1 -- 172.21.15.18:0/351638709 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557897809ac0 con 0x557897907bd0 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13ae7eb640 1 --2- 172.21.15.18:0/351638709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979107b0 0x557897917680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/351638709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 0x5578979100e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1398018530 tx=0x7f139800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bdea0883abbbb3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13937fe640 1 -- 172.21.15.18:0/351638709 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13980120f0 con 0x557897907bd0 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13937fe640 1 -- 172.21.15.18:0/351638709 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f139809b480 con 0x557897907bd0 2024-02-17T12:51:55.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.472+0000 7f13937fe640 1 -- 172.21.15.18:0/351638709 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13980cbd60 con 0x557897907bd0 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.473+0000 7f13b2e28740 1 -- 172.21.15.18:0/351638709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 msgr2=0x5578979100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.473+0000 7f13b2e28740 1 --2- 172.21.15.18:0/351638709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 0x5578979100e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1398018530 tx=0x7f139800c840 comp rx=0 tx=0).stop 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.473+0000 7f13b2e28740 1 -- 172.21.15.18:0/351638709 shutdown_connections 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.473+0000 7f13b2e28740 1 --2- 172.21.15.18:0/351638709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979107b0 0x557897917680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.473+0000 7f13b2e28740 1 --2- 172.21.15.18:0/351638709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557897907bd0 0x5578979100e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.473+0000 7f13b2e28740 1 --2- 172.21.15.18:0/351638709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578979072b0 0x55789790fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.473+0000 7f13b2e28740 1 -- 172.21.15.18:0/351638709 >> 172.21.15.18:0/351638709 conn(0x557897901990 msgr2=0x55789790bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.473+0000 7f13b2e28740 1 -- 172.21.15.18:0/351638709 shutdown_connections 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.473+0000 7f13b2e28740 1 -- 172.21.15.18:0/351638709 wait complete. 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.473+0000 7f13b2e28740 1 -- start start 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.473+0000 7f13b2e28740 1 Processor -- start 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13b2e28740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x55789790a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13b2e28740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557897907bd0 0x55789790a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13b2e28740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 0x55789790ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13b2e28740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897976c10 con 0x5578979107b0 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13b2e28740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897976d50 con 0x5578979072b0 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13b2e28740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897976e90 con 0x557897907bd0 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13b2e28740 10 client.0 _ll_register_callbacks cb 0x7f1398024eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13ae7eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 0x55789790ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13ae7eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 0x55789790ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36542/0 (socket says 172.21.15.18:36542) 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13adfea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x55789790a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13ae7eb640 1 -- 172.21.15.18:0/2549753176 learned_addr learned my addr 172.21.15.18:0/2549753176 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13ad7e9640 1 --2- 172.21.15.18:0/2549753176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557897907bd0 0x55789790a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13ae7eb640 1 -- 172.21.15.18:0/2549753176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557897907bd0 msgr2=0x55789790a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13ae7eb640 1 --2- 172.21.15.18:0/2549753176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557897907bd0 0x55789790a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13ae7eb640 1 -- 172.21.15.18:0/2549753176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 msgr2=0x55789790a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:55.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13ae7eb640 1 --2- 172.21.15.18:0/2549753176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x55789790a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13ae7eb640 1 -- 172.21.15.18:0/2549753176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55789786d610 con 0x5578979107b0 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13ae7eb640 1 --2- 172.21.15.18:0/2549753176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 0x55789790ac20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009370 tx=0x7f13a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df0b949f669f20 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.474+0000 7f13917fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.475+0000 7f13917fa640 1 -- 172.21.15.18:0/2549753176 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0012a50 con 0x5578979107b0 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.475+0000 7f13917fa640 1 -- 172.21.15.18:0/2549753176 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f13a0011b50 con 0x5578979107b0 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.475+0000 7f13b2e28740 1 -- 172.21.15.18:0/2549753176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f139806d0b0 con 0x5578979107b0 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.475+0000 7f13917fa640 1 -- 172.21.15.18:0/2549753176 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13a0011d10 con 0x5578979107b0 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.475+0000 7f13b2e28740 10 client.16356 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.475+0000 7f13b2e28740 1 -- 172.21.15.18:0/2549753176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55789782f6a0 con 0x5578979107b0 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.475+0000 7f13917fa640 1 -- 172.21.15.18:0/2549753176 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f139806d0b0 con 0x5578979107b0 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.475+0000 7f13917fa640 1 -- 172.21.15.18:0/2549753176 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f13a0004570 con 0x5578979107b0 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.476+0000 7f13b2e28740 10 client.16356 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.476+0000 7f13b2e28740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:55.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.476+0000 7f13b2e28740 20 client.16356 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.476+0000 7f13b2e28740 1 --2- 172.21.15.18:0/2549753176 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557897978750 0x55789797ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.476+0000 7f13b2e28740 4 client.16356 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.476+0000 7f13b2e28740 1 -- 172.21.15.18:0/2549753176 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5578975fa890 con 0x557897978750 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.476+0000 7f13adfea640 1 --2- 172.21.15.18:0/2549753176 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557897978750 0x55789797ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.476+0000 7f13917fa640 10 client.16356 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.476+0000 7f13adfea640 1 --2- 172.21.15.18:0/2549753176 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557897978750 0x55789797ab70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=581dda78bf6e72f6 server_cookie=8c04b6ece2fa82ad in_seq=0 out_seq=0 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13917fa640 1 -- 172.21.15.18:0/2549753176 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5578975fa890 con 0x557897978750 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13917fa640 10 client.16356 handle_command_reply: tid=0 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13b2e28740 1 client.16356 shutdown 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13b2e28740 1 -- 172.21.15.18:0/2549753176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 msgr2=0x55789790ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13b2e28740 1 --2- 172.21.15.18:0/2549753176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 0x55789790ac20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009370 tx=0x7f13a0004ab0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13b2e28740 1 -- 172.21.15.18:0/2549753176 shutdown_connections 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13b2e28740 1 --2- 172.21.15.18:0/2549753176 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557897978750 0x55789797ab70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13b2e28740 1 --2- 172.21.15.18:0/2549753176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578979107b0 0x55789790ac20 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13b2e28740 1 --2- 172.21.15.18:0/2549753176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557897907bd0 0x55789790a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13917fa640 0 client.16356 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:55.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13b2e28740 1 --2- 172.21.15.18:0/2549753176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578979072b0 0x55789790a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13b2e28740 1 -- 172.21.15.18:0/2549753176 >> 172.21.15.18:0/2549753176 conn(0x557897901990 msgr2=0x5578979001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:55.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13b2e28740 1 -- 172.21.15.18:0/2549753176 shutdown_connections 2024-02-17T12:51:55.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13b2e28740 1 -- 172.21.15.18:0/2549753176 wait complete. 2024-02-17T12:51:55.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.477+0000 7f13b2e28740 20 client.16356 trim_cache size 0 max 16384 2024-02-17T12:51:55.478 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:51:55.478 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:51:55.478 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:51:55.479 INFO:teuthology.orchestra.run.smithi018.stdout: "080fc321-59ab-451b-bd89-dbd1da5f2bbd": { 2024-02-17T12:51:55.479 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:51:55.479 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "080fc321-59ab-451b-bd89-dbd1da5f2bbd", 2024-02-17T12:51:55.479 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:51:55.479 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:51:55.479 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:51:55.479 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:51:55.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.478+0000 7f13b024e640 1 -- 172.21.15.18:0/25063877 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13940a2230 msgr2=0x7f13940a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.478+0000 7f13b024e640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13940a2230 0x7f13940a46f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009fd0 tx=0x7f13a0009500 comp rx=0 tx=0).stop 2024-02-17T12:51:55.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.478+0000 7f13b024e640 1 -- 172.21.15.18:0/25063877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13a8100840 msgr2=0x7f13a819fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:55.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.478+0000 7f13b024e640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13a8100840 0x7f13a819fbf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f139800e9f0 tx=0x7f139800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.478+0000 7f13b024e640 1 -- 172.21.15.18:0/25063877 shutdown_connections 2024-02-17T12:51:55.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.478+0000 7f13b024e640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13940a2230 0x7f13940a46f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.478+0000 7f13b024e640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a810dad0 0x7f13a81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.478+0000 7f13b024e640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13a8100840 0x7f13a819fbf0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.478+0000 7f13b024e640 1 --2- 172.21.15.18:0/25063877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a80fff20 0x7f13a819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:55.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.478+0000 7f13b024e640 1 -- 172.21.15.18:0/25063877 >> 172.21.15.18:0/25063877 conn(0x7f13a8077230 msgr2=0x7f13a810f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:55.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.478+0000 7f13b024e640 1 -- 172.21.15.18:0/25063877 shutdown_connections 2024-02-17T12:51:55.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:51:55.478+0000 7f13b024e640 1 -- 172.21.15.18:0/25063877 wait complete. 2024-02-17T12:51:55.481 INFO:tasks.cephfs.filesystem:scrub status for tag:080fc321-59ab-451b-bd89-dbd1da5f2bbd - {'path': '/', 'tag': '080fc321-59ab-451b-bd89-dbd1da5f2bbd', 'options': 'recursive,force'} 2024-02-17T12:51:56.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:55 smithi119 ceph-mon[36188]: pgmap v64: 97 pgs: 97 active+clean; 35 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 718 KiB/s wr, 142 op/s 2024-02-17T12:51:56.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:55 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:56.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:55 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:56.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:55 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:56.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:55 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:55 smithi146 ceph-mon[36454]: pgmap v64: 97 pgs: 97 active+clean; 35 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 718 KiB/s wr, 142 op/s 2024-02-17T12:51:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:55 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:55 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:56.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:55 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:56.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:55 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:56.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 67227 5.35 MB/sec warmup 71 sec latency 2755.522 ms 2024-02-17T12:51:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:55 smithi018 ceph-mon[32625]: pgmap v64: 97 pgs: 97 active+clean; 35 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 718 KiB/s wr, 142 op/s 2024-02-17T12:51:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:55 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:56.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:55 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:56.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:55 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:56.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:55 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:51:57.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:57.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:57.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:57.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:57.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:57.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:51:57.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:56 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:51:57.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:57.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:57.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:57.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:57.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:57.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:51:57.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:56 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 68087 5.34 MB/sec warmup 72 sec latency 356.212 ms 2024-02-17T12:51:57.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:57.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:56 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:51:58.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:57 smithi119 ceph-mon[36188]: pgmap v65: 97 pgs: 97 active+clean; 35 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 534 KiB/s wr, 98 op/s 2024-02-17T12:51:58.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:57 smithi146 ceph-mon[36454]: pgmap v65: 97 pgs: 97 active+clean; 35 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 534 KiB/s wr, 98 op/s 2024-02-17T12:51:58.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 68087 5.27 MB/sec warmup 73 sec latency 1356.282 ms 2024-02-17T12:51:58.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:57 smithi018 ceph-mon[32625]: pgmap v65: 97 pgs: 97 active+clean; 35 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 534 KiB/s wr, 98 op/s 2024-02-17T12:51:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.358+0000 7facda5bc640 1 -- 172.21.15.18:0/1147259663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41013a0 msgr2=0x7facd4101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.358+0000 7facda5bc640 1 --2- 172.21.15.18:0/1147259663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41013a0 0x7facd4101780 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7facc4009930 tx=0x7facc402bc30 comp rx=0 tx=0).stop 2024-02-17T12:51:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.358+0000 7facda5bc640 1 -- 172.21.15.18:0/1147259663 shutdown_connections 2024-02-17T12:51:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.358+0000 7facda5bc640 1 --2- 172.21.15.18:0/1147259663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facd4106470 0x7facd410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.358+0000 7facda5bc640 1 --2- 172.21.15.18:0/1147259663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd4101d50 0x7facd4105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.358+0000 7facda5bc640 1 --2- 172.21.15.18:0/1147259663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41013a0 0x7facd4101780 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.358+0000 7facda5bc640 1 -- 172.21.15.18:0/1147259663 >> 172.21.15.18:0/1147259663 conn(0x7facd40fcc40 msgr2=0x7facd40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.358+0000 7facda5bc640 1 -- 172.21.15.18:0/1147259663 shutdown_connections 2024-02-17T12:51:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.358+0000 7facda5bc640 1 -- 172.21.15.18:0/1147259663 wait complete. 2024-02-17T12:51:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.359+0000 7facda5bc640 1 Processor -- start 2024-02-17T12:51:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.359+0000 7facda5bc640 1 -- start start 2024-02-17T12:51:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.359+0000 7facda5bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facd41013a0 0x7facd419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.359+0000 7facda5bc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd4106470 0x7facd419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.359+0000 7facda5bc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41975b0 0x7facd4197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.359+0000 7facda5bc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd4197fa0 con 0x7facd41975b0 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.359+0000 7facda5bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd4198110 con 0x7facd41013a0 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.359+0000 7facda5bc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd41982d0 con 0x7facd4106470 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd8b59640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41975b0 0x7facd4197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facd41013a0 0x7facd419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd8b59640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41975b0 0x7facd4197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36556/0 (socket says 172.21.15.18:36556) 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facd41013a0 0x7facd419d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:58448/0 (socket says 172.21.15.18:58448) 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd8b59640 1 -- 172.21.15.18:0/1078106256 learned_addr learned my addr 172.21.15.18:0/1078106256 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd37fe640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd4106470 0x7facd419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd8b59640 1 -- 172.21.15.18:0/1078106256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd4106470 msgr2=0x7facd419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd8b59640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd4106470 0x7facd419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd8b59640 1 -- 172.21.15.18:0/1078106256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facd41013a0 msgr2=0x7facd419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd8b59640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facd41013a0 0x7facd419d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd8b59640 1 -- 172.21.15.18:0/1078106256 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc4009560 con 0x7facd41975b0 2024-02-17T12:51:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd8b59640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41975b0 0x7facd4197a60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faccc00eaa0 tx=0x7faccc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd17fa640 1 -- 172.21.15.18:0/1078106256 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc00cdb0 con 0x7facd41975b0 2024-02-17T12:51:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd17fa640 1 -- 172.21.15.18:0/1078106256 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faccc005020 con 0x7facd41975b0 2024-02-17T12:51:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facda5bc640 1 -- 172.21.15.18:0/1078106256 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd41a42c0 con 0x7facd41975b0 2024-02-17T12:51:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facd17fa640 1 -- 172.21.15.18:0/1078106256 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc0116b0 con 0x7facd41975b0 2024-02-17T12:51:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.360+0000 7facda5bc640 1 -- 172.21.15.18:0/1078106256 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd41a4780 con 0x7facd41975b0 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.363+0000 7facd17fa640 1 -- 172.21.15.18:0/1078106256 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7faccc005520 con 0x7facd41975b0 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.363+0000 7facdd191740 1 Processor -- start 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.363+0000 7facdd191740 1 -- start start 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.363+0000 7facdd191740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d38ad0 0x55aa63d37820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.363+0000 7facdd191740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.363+0000 7facdd191740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d412e0 0x55aa63d436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.363+0000 7facd17fa640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7facc00a22d0 0x7facc00a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.363+0000 7facdd191740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa63ca69f0 con 0x55aa63d37d60 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.363+0000 7facdd191740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa63ca3f20 con 0x55aa63d412e0 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.363+0000 7facdd191740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa63c9f700 con 0x55aa63d38ad0 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d381e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36564/0 (socket says 172.21.15.18:36564) 2024-02-17T12:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd3fff640 1 -- 172.21.15.18:0/2749755893 learned_addr learned my addr 172.21.15.18:0/2749755893 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd37fe640 1 --2- 172.21.15.18:0/2749755893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d38ad0 0x55aa63d37820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd8b59640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7facc00a22d0 0x7facc00a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd17fa640 1 -- 172.21.15.18:0/1078106256 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faccc0ce210 con 0x7facd41975b0 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd37fe640 1 --2- 172.21.15.18:0/2749755893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d412e0 0x55aa63d436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd3fff640 1 -- 172.21.15.18:0/2749755893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d38ad0 msgr2=0x55aa63d37820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd3fff640 1 --2- 172.21.15.18:0/2749755893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d38ad0 0x55aa63d37820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd3fff640 1 -- 172.21.15.18:0/2749755893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d412e0 msgr2=0x55aa63d436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd3fff640 1 --2- 172.21.15.18:0/2749755893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d412e0 0x55aa63d436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd3fff640 1 -- 172.21.15.18:0/2749755893 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa63c357b0 con 0x55aa63d37d60 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd37fe640 1 --2- 172.21.15.18:0/2749755893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d412e0 0x55aa63d436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd3fff640 1 --2- 172.21.15.18:0/2749755893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d381e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7facc4036040 tx=0x7facc4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9ca823ef567930 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facd8b59640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7facc00a22d0 0x7facc00a4790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faccc097670 tx=0x7faccc0153a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facca7fc640 1 -- 172.21.15.18:0/2749755893 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4004900 con 0x55aa63d37d60 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facca7fc640 1 -- 172.21.15.18:0/2749755893 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facc4004a60 con 0x55aa63d37d60 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.364+0000 7facca7fc640 1 -- 172.21.15.18:0/2749755893 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40051f0 con 0x55aa63d37d60 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.365+0000 7facdd191740 1 -- 172.21.15.18:0/2749755893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 msgr2=0x55aa63d381e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.365+0000 7facdd191740 1 --2- 172.21.15.18:0/2749755893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d381e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7facc4036040 tx=0x7facc4005c00 comp rx=0 tx=0).stop 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.365+0000 7facdd191740 1 -- 172.21.15.18:0/2749755893 shutdown_connections 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.365+0000 7facdd191740 1 --2- 172.21.15.18:0/2749755893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d412e0 0x55aa63d436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.365+0000 7facdd191740 1 --2- 172.21.15.18:0/2749755893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d381e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.365+0000 7facdd191740 1 --2- 172.21.15.18:0/2749755893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d38ad0 0x55aa63d37820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.365+0000 7facdd191740 1 -- 172.21.15.18:0/2749755893 >> 172.21.15.18:0/2749755893 conn(0x55aa63cff550 msgr2=0x55aa63d32a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.365+0000 7facdd191740 1 -- 172.21.15.18:0/2749755893 shutdown_connections 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.365+0000 7facdd191740 1 -- 172.21.15.18:0/2749755893 wait complete. 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facdd191740 1 -- start start 2024-02-17T12:51:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facdd191740 1 Processor -- start 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facdd191740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d3e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facdd191740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d3e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facdd191740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d412e0 0x55aa63d3ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facdd191740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa63c44bf0 con 0x55aa63d37d60 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facdd191740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa63973250 con 0x55aa63d38ad0 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facdd191740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa63c25200 con 0x55aa63d412e0 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facdd191740 10 client.0 _ll_register_callbacks cb 0x55aa63c93d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d3e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d3e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36572/0 (socket says 172.21.15.18:36572) 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facd37fe640 1 -- 172.21.15.18:0/2574229581 learned_addr learned my addr 172.21.15.18:0/2574229581 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facd37fe640 1 -- 172.21.15.18:0/2574229581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d412e0 msgr2=0x55aa63d3ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.366+0000 7facd3fff640 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d3e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facd37fe640 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d412e0 0x55aa63d3ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facd37fe640 1 -- 172.21.15.18:0/2574229581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 msgr2=0x55aa63d3e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facd37fe640 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d3e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facd37fe640 1 -- 172.21.15.18:0/2574229581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd410e850 con 0x55aa63d37d60 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facd3fff640 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d3e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facd37fe640 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d3e390 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7facd41986f0 tx=0x7facbc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e87500e1114f0a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facb37fe640 1 -- 172.21.15.18:0/2574229581 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc005220 con 0x55aa63d37d60 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facb37fe640 1 -- 172.21.15.18:0/2574229581 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7facbc005380 con 0x55aa63d37d60 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facdd191740 1 -- 172.21.15.18:0/2574229581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa6399f600 con 0x55aa63d37d60 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facb37fe640 1 -- 172.21.15.18:0/2574229581 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7facbc005540 con 0x55aa63d37d60 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facdd191740 10 client.16368 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facdd191740 1 -- 172.21.15.18:0/2574229581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55aa63c9dd80 con 0x55aa63d37d60 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facb37fe640 1 -- 172.21.15.18:0/2574229581 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aa6399f600 con 0x55aa63d37d60 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.367+0000 7facb37fe640 1 -- 172.21.15.18:0/2574229581 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7facbc0129c0 con 0x55aa63d37d60 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.368+0000 7facdd191740 10 client.16368 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.368+0000 7facdd191740 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.368+0000 7facdd191740 20 client.16368 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.368+0000 7facdd191740 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa63d517f0 0x55aa63d53c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.368+0000 7facdd191740 4 client.16368 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.368+0000 7facdd191740 1 -- 172.21.15.18:0/2574229581 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa639f01b0 con 0x55aa63d517f0 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.368+0000 7facd3fff640 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa63d517f0 0x55aa63d53c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.368+0000 7facd3fff640 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa63d517f0 0x55aa63d53c30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdc726ba446677dc server_cookie=faa09cdb22b57d2b in_seq=0 out_seq=0 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.368+0000 7facb37fe640 10 client.16368 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facb37fe640 1 -- 172.21.15.18:0/2574229581 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa639f01b0 con 0x55aa63d517f0 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facb37fe640 10 client.16368 handle_command_reply: tid=0 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facdd191740 1 client.16368 shutdown 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facdd191740 1 -- 172.21.15.18:0/2574229581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 msgr2=0x55aa63d3e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facdd191740 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d3e390 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7facd41986f0 tx=0x7facbc0039c0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facdd191740 1 -- 172.21.15.18:0/2574229581 shutdown_connections 2024-02-17T12:51:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facdd191740 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa63d517f0 0x55aa63d53c30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facdd191740 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d412e0 0x55aa63d3ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facdd191740 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d3e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facb37fe640 0 client.16368 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facdd191740 1 --2- 172.21.15.18:0/2574229581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d3e390 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facdd191740 1 -- 172.21.15.18:0/2574229581 >> 172.21.15.18:0/2574229581 conn(0x55aa63cff550 msgr2=0x55aa63d347a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facdd191740 1 -- 172.21.15.18:0/2574229581 shutdown_connections 2024-02-17T12:51:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facdd191740 1 -- 172.21.15.18:0/2574229581 wait complete. 2024-02-17T12:51:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.369+0000 7facdd191740 20 client.16368 trim_cache size 0 max 16384 2024-02-17T12:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facdd191740 1 Processor -- start 2024-02-17T12:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facdd191740 1 -- start start 2024-02-17T12:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facdd191740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d37d60 0x55aa63d40ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facdd191740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facdd191740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d412e0 0x55aa63d485e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facdd191740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa63d2e4e0 con 0x55aa63d412e0 2024-02-17T12:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facdd191740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa63d4fee0 con 0x55aa63d38ad0 2024-02-17T12:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facdd191740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa63d2e310 con 0x55aa63d37d60 2024-02-17T12:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d412e0 0x55aa63d485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d412e0 0x55aa63d485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36584/0 (socket says 172.21.15.18:36584) 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facd37fe640 1 -- 172.21.15.18:0/3005268429 learned_addr learned my addr 172.21.15.18:0/3005268429 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facd3fff640 1 --2- 172.21.15.18:0/3005268429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facd37fe640 1 --2- 172.21.15.18:0/3005268429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d37d60 0x55aa63d40ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facd37fe640 1 -- 172.21.15.18:0/3005268429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d37d60 msgr2=0x55aa63d40ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facd37fe640 1 --2- 172.21.15.18:0/3005268429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d37d60 0x55aa63d40ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facd37fe640 1 -- 172.21.15.18:0/3005268429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 msgr2=0x55aa63d480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facd37fe640 1 --2- 172.21.15.18:0/3005268429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.383+0000 7facd37fe640 1 -- 172.21.15.18:0/3005268429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa63c3a7b0 con 0x55aa63d412e0 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facd37fe640 1 --2- 172.21.15.18:0/3005268429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d412e0 0x55aa63d485e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7facbc001eb0 tx=0x7facbc00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eec8dbc7ceadc8c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facc9ffb640 1 -- 172.21.15.18:0/3005268429 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc005230 con 0x55aa63d412e0 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facc9ffb640 1 -- 172.21.15.18:0/3005268429 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facbc0129e0 con 0x55aa63d412e0 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facc9ffb640 1 -- 172.21.15.18:0/3005268429 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc004b20 con 0x55aa63d412e0 2024-02-17T12:51:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facdd191740 1 -- 172.21.15.18:0/3005268429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d412e0 msgr2=0x55aa63d485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facdd191740 1 --2- 172.21.15.18:0/3005268429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d412e0 0x55aa63d485e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7facbc001eb0 tx=0x7facbc00b420 comp rx=0 tx=0).stop 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facdd191740 1 -- 172.21.15.18:0/3005268429 shutdown_connections 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facdd191740 1 --2- 172.21.15.18:0/3005268429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d412e0 0x55aa63d485e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facdd191740 1 --2- 172.21.15.18:0/3005268429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facdd191740 1 --2- 172.21.15.18:0/3005268429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d37d60 0x55aa63d40ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facdd191740 1 -- 172.21.15.18:0/3005268429 >> 172.21.15.18:0/3005268429 conn(0x55aa63cff550 msgr2=0x55aa63d396a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facdd191740 1 -- 172.21.15.18:0/3005268429 shutdown_connections 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.384+0000 7facdd191740 1 -- 172.21.15.18:0/3005268429 wait complete. 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facdd191740 1 -- start start 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facdd191740 1 Processor -- start 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facdd191740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d44a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facdd191740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d44fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facdd191740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d412e0 0x55aa63d454f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facdd191740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa63d489e0 con 0x55aa63d37d60 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facdd191740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa63da7d10 con 0x55aa63d38ad0 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facdd191740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa63da7ea0 con 0x55aa63d412e0 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facdd191740 10 client.0 _ll_register_callbacks cb 0x7facc4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d44a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d44a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36592/0 (socket says 172.21.15.18:36592) 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facd37fe640 1 -- 172.21.15.18:0/618194149 learned_addr learned my addr 172.21.15.18:0/618194149 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.385+0000 7facd3fff640 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d44fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facd37fe640 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d412e0 0x55aa63d454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facd37fe640 1 -- 172.21.15.18:0/618194149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d412e0 msgr2=0x55aa63d454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facd37fe640 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d412e0 0x55aa63d454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facd37fe640 1 -- 172.21.15.18:0/618194149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 msgr2=0x55aa63d44fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facd37fe640 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d44fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facd37fe640 1 -- 172.21.15.18:0/618194149 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa63c9dd80 con 0x55aa63d37d60 2024-02-17T12:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facd37fe640 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d44a70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7facbc005c20 tx=0x7facbc004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed07fbb79a22f8c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facb37fe640 1 -- 172.21.15.18:0/618194149 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc027af0 con 0x55aa63d37d60 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facb37fe640 1 -- 172.21.15.18:0/618194149 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7facbc02e070 con 0x55aa63d37d60 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facdd191740 1 -- 172.21.15.18:0/618194149 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facc4004a60 con 0x55aa63d37d60 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facb37fe640 1 -- 172.21.15.18:0/618194149 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7facbc012b50 con 0x55aa63d37d60 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facdd191740 10 client.16374 fetch_fsmap learned FSMap version 28 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facdd191740 1 -- 172.21.15.18:0/618194149 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55aa63c60540 con 0x55aa63d37d60 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facb37fe640 1 -- 172.21.15.18:0/618194149 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facc4004a60 con 0x55aa63d37d60 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facb37fe640 1 -- 172.21.15.18:0/618194149 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7facbc043420 con 0x55aa63d37d60 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facdd191740 10 client.16374 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facdd191740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facdd191740 20 client.16374 populate_metadata read hostname 'smithi018' 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facdd191740 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa63da92a0 0x55aa63dab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facdd191740 4 client.16374 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.386+0000 7facdd191740 1 -- 172.21.15.18:0/618194149 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aa639dc550 con 0x55aa63da92a0 2024-02-17T12:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.387+0000 7facd3fff640 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa63da92a0 0x55aa63dab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.387+0000 7facd3fff640 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa63da92a0 0x55aa63dab6c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe0580ca3ea9ab5 server_cookie=ddc3406d846c1c1d in_seq=0 out_seq=0 2024-02-17T12:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.387+0000 7facb37fe640 10 client.16374 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facb37fe640 1 -- 172.21.15.18:0/618194149 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aa639dc550 con 0x55aa63da92a0 2024-02-17T12:51:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facb37fe640 10 client.16374 handle_command_reply: tid=0 2024-02-17T12:51:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facdd191740 1 client.16374 shutdown 2024-02-17T12:51:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facdd191740 1 -- 172.21.15.18:0/618194149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 msgr2=0x55aa63d44a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facdd191740 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d44a70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7facbc005c20 tx=0x7facbc004250 comp rx=0 tx=0).stop 2024-02-17T12:51:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facdd191740 1 -- 172.21.15.18:0/618194149 shutdown_connections 2024-02-17T12:51:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facdd191740 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aa63da92a0 0x55aa63dab6c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facdd191740 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa63d412e0 0x55aa63d454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facdd191740 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa63d38ad0 0x55aa63d44fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facb37fe640 0 client.16374 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:51:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facdd191740 1 --2- 172.21.15.18:0/618194149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa63d37d60 0x55aa63d44a70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facdd191740 1 -- 172.21.15.18:0/618194149 >> 172.21.15.18:0/618194149 conn(0x55aa63cff550 msgr2=0x55aa63d3cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facdd191740 1 -- 172.21.15.18:0/618194149 shutdown_connections 2024-02-17T12:51:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facdd191740 1 -- 172.21.15.18:0/618194149 wait complete. 2024-02-17T12:51:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.420+0000 7facdd191740 20 client.16374 trim_cache size 0 max 16384 2024-02-17T12:51:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:51:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:51:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:51:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.421+0000 7facda5bc640 1 -- 172.21.15.18:0/1078106256 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7facc00a22d0 msgr2=0x7facc00a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.421+0000 7facda5bc640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7facc00a22d0 0x7facc00a4790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faccc097670 tx=0x7faccc0153a0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.421+0000 7facda5bc640 1 -- 172.21.15.18:0/1078106256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41975b0 msgr2=0x7facd4197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.421+0000 7facda5bc640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41975b0 0x7facd4197a60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faccc00eaa0 tx=0x7faccc00ef60 comp rx=0 tx=0).stop 2024-02-17T12:51:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.421+0000 7facda5bc640 1 -- 172.21.15.18:0/1078106256 shutdown_connections 2024-02-17T12:51:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.421+0000 7facda5bc640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7facc00a22d0 0x7facc00a4790 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.421+0000 7facda5bc640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41975b0 0x7facd4197a60 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.421+0000 7facda5bc640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd4106470 0x7facd419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.421+0000 7facda5bc640 1 --2- 172.21.15.18:0/1078106256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facd41013a0 0x7facd419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:51:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.421+0000 7facda5bc640 1 -- 172.21.15.18:0/1078106256 >> 172.21.15.18:0/1078106256 conn(0x7facd40fcc40 msgr2=0x7facd40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:51:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.421+0000 7facda5bc640 1 -- 172.21.15.18:0/1078106256 shutdown_connections 2024-02-17T12:51:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:51:58.421+0000 7facda5bc640 1 -- 172.21.15.18:0/1078106256 wait complete. 2024-02-17T12:51:59.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:51:58 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:51:59.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:51:58 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:51:59.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 68087 5.19 MB/sec warmup 74 sec latency 2356.318 ms 2024-02-17T12:51:59.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:51:58 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:52:00.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 68087 5.13 MB/sec warmup 75 sec latency 3356.348 ms 2024-02-17T12:52:00.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:00 smithi018 ceph-mon[32625]: pgmap v66: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 763 KiB/s wr, 166 op/s 2024-02-17T12:52:00.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:00 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:00 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:00 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:00 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:00 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:52:00.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:00 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:52:00.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:00 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:00 smithi119 ceph-mon[36188]: pgmap v66: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 763 KiB/s wr, 166 op/s 2024-02-17T12:52:00.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:00 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:00 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:00 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:00 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:00 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:52:00.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:00 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:52:00.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:00 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:00 smithi146 ceph-mon[36454]: pgmap v66: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 763 KiB/s wr, 166 op/s 2024-02-17T12:52:00.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:00 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:00 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:00 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:00 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:00.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:00 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:52:00.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:00 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:52:00.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:00 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:52:01.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 69972 5.16 MB/sec warmup 76 sec latency 3404.480 ms 2024-02-17T12:52:02.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 71582 5.24 MB/sec warmup 77 sec latency 26.155 ms 2024-02-17T12:52:02.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:02 smithi018 ceph-mon[32625]: pgmap v67: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 427 KiB/s wr, 108 op/s 2024-02-17T12:52:02.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:02 smithi119 ceph-mon[36188]: pgmap v67: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 427 KiB/s wr, 108 op/s 2024-02-17T12:52:02.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:02 smithi146 ceph-mon[36454]: pgmap v67: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 427 KiB/s wr, 108 op/s 2024-02-17T12:52:03.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 71685 5.17 MB/sec warmup 78 sec latency 949.401 ms 2024-02-17T12:52:04.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 71685 5.11 MB/sec warmup 79 sec latency 1949.462 ms 2024-02-17T12:52:04.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:04 smithi018 ceph-mon[32625]: pgmap v68: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 427 KiB/s wr, 108 op/s 2024-02-17T12:52:04.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:04 smithi119 ceph-mon[36188]: pgmap v68: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 427 KiB/s wr, 108 op/s 2024-02-17T12:52:04.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:04 smithi146 ceph-mon[36454]: pgmap v68: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 427 KiB/s wr, 108 op/s 2024-02-17T12:52:05.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 71685 5.05 MB/sec warmup 80 sec latency 2949.531 ms 2024-02-17T12:52:05.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:05 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:52:05.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:05 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:52:05.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:05 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:52:06.263 INFO:tasks.workunit.client.0.smithi018.stdout: 1 71753 4.99 MB/sec warmup 81 sec latency 3514.897 ms 2024-02-17T12:52:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:06 smithi018 ceph-mon[32625]: pgmap v69: 97 pgs: 97 active+clean; 35 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1019 KiB/s wr, 197 op/s 2024-02-17T12:52:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:06 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:52:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:06 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:52:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:06 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:52:06.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:06 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:52:06.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:06 smithi119 ceph-mon[36188]: pgmap v69: 97 pgs: 97 active+clean; 35 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1019 KiB/s wr, 197 op/s 2024-02-17T12:52:06.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:06 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:52:06.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:06 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:52:06.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:06 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:52:06.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:06 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:52:06.602 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:06 smithi146 ceph-mon[36454]: pgmap v69: 97 pgs: 97 active+clean; 35 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1019 KiB/s wr, 197 op/s 2024-02-17T12:52:06.602 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:06 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:52:06.602 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:06 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:52:06.602 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:06 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:52:06.603 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:06 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:52:07.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 71753 4.92 MB/sec warmup 82 sec latency 1388.491 ms 2024-02-17T12:52:08.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 71753 4.87 MB/sec warmup 83 sec latency 2388.602 ms 2024-02-17T12:52:08.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:08 smithi119 ceph-mon[36188]: pgmap v70: 97 pgs: 97 active+clean; 35 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 822 KiB/s wr, 156 op/s 2024-02-17T12:52:08.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:08 smithi146 ceph-mon[36454]: pgmap v70: 97 pgs: 97 active+clean; 35 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 822 KiB/s wr, 156 op/s 2024-02-17T12:52:08.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:08 smithi018 ceph-mon[32625]: pgmap v70: 97 pgs: 97 active+clean; 35 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 822 KiB/s wr, 156 op/s 2024-02-17T12:52:09.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 71753 4.81 MB/sec warmup 84 sec latency 3388.747 ms 2024-02-17T12:52:10.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 71753 4.75 MB/sec warmup 85 sec latency 4388.836 ms 2024-02-17T12:52:10.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:10 smithi119 ceph-mon[36188]: pgmap v71: 97 pgs: 97 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 884 KiB/s wr, 168 op/s 2024-02-17T12:52:10.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:10 smithi146 ceph-mon[36454]: pgmap v71: 97 pgs: 97 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 884 KiB/s wr, 168 op/s 2024-02-17T12:52:10.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:10 smithi018 ceph-mon[32625]: pgmap v71: 97 pgs: 97 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 884 KiB/s wr, 168 op/s 2024-02-17T12:52:11.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 72943 4.75 MB/sec warmup 86 sec latency 4954.010 ms 2024-02-17T12:52:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.512+0000 7f0cb6e40640 1 -- 172.21.15.18:0/3899815169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cb01013c0 msgr2=0x7f0cb01017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.512+0000 7f0cb6e40640 1 --2- 172.21.15.18:0/3899815169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cb01013c0 0x7f0cb01017a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009930 tx=0x7f0cac02bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.512+0000 7f0cb6e40640 1 -- 172.21.15.18:0/3899815169 shutdown_connections 2024-02-17T12:52:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.512+0000 7f0cb6e40640 1 --2- 172.21.15.18:0/3899815169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cb0106490 0x7f0cb010e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.512+0000 7f0cb6e40640 1 --2- 172.21.15.18:0/3899815169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cb0101d70 0x7f0cb0105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.512+0000 7f0cb6e40640 1 --2- 172.21.15.18:0/3899815169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cb01013c0 0x7f0cb01017a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.512+0000 7f0cb6e40640 1 -- 172.21.15.18:0/3899815169 >> 172.21.15.18:0/3899815169 conn(0x7f0cb00fcc60 msgr2=0x7f0cb00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb6e40640 1 -- 172.21.15.18:0/3899815169 shutdown_connections 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb6e40640 1 -- 172.21.15.18:0/3899815169 wait complete. 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb6e40640 1 Processor -- start 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb6e40640 1 -- start start 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb6e40640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cb01013c0 0x7f0cb019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb6e40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cb0106490 0x7f0cb019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb6e40640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cb0197630 0x7f0cb0197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb6e40640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0198020 con 0x7f0cb01013c0 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb6e40640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0198190 con 0x7f0cb0106490 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb6e40640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0198350 con 0x7f0cb0197630 2024-02-17T12:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb4bdc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cb01013c0 0x7f0cb019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb4bdc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cb01013c0 0x7f0cb019d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56066/0 (socket says 172.21.15.18:56066) 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.513+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/4164492910 learned_addr learned my addr 172.21.15.18:0/4164492910 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0cb53dd640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cb0197630 0x7f0cb0197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0ca7fff640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cb0106490 0x7f0cb019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/4164492910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cb0197630 msgr2=0x7f0cb0197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cb0197630 0x7f0cb0197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/4164492910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cb0106490 msgr2=0x7f0cb019d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cb0106490 0x7f0cb019d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/4164492910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac009560 con 0x7f0cb01013c0 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cb01013c0 0x7f0cb019d3e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009230 tx=0x7f0cac02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0ca5ffb640 1 -- 172.21.15.18:0/4164492910 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac004350 con 0x7f0cb01013c0 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0ca5ffb640 1 -- 172.21.15.18:0/4164492910 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cac0044b0 con 0x7f0cb01013c0 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0cb6e40640 1 -- 172.21.15.18:0/4164492910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb01a42a0 con 0x7f0cb01013c0 2024-02-17T12:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0ca5ffb640 1 -- 172.21.15.18:0/4164492910 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac003d00 con 0x7f0cb01013c0 2024-02-17T12:52:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.514+0000 7f0cb6e40640 1 -- 172.21.15.18:0/4164492910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb01a4750 con 0x7f0cb01013c0 2024-02-17T12:52:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.516+0000 7f0ca5ffb640 1 -- 172.21.15.18:0/4164492910 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0cac004df0 con 0x7f0cb01013c0 2024-02-17T12:52:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0cb9a12740 1 Processor -- start 2024-02-17T12:52:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0cb9a12740 1 -- start start 2024-02-17T12:52:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0ca5ffb640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0c980a22d0 0x7f0c980a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0cb9a12740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a95ad0 0x5650d6a94820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0ca5ffb640 1 -- 172.21.15.18:0/4164492910 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0cac0efeb0 con 0x7f0cb01013c0 2024-02-17T12:52:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0cb9a12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a94d60 0x5650d6a951e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0cb53dd640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0c980a22d0 0x7f0c980a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0cb9a12740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a9e2e0 0x5650d6aa06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0cb9a12740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d6a039f0 con 0x5650d6a9e2e0 2024-02-17T12:52:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0cb9a12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d6a00f20 con 0x5650d6a94d60 2024-02-17T12:52:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0cb9a12740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d69fc700 con 0x5650d6a95ad0 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0cb4bdc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a95ad0 0x5650d6a94820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0cb4bdc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a95ad0 0x5650d6a94820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:47058/0 (socket says 172.21.15.18:47058) 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/1542688062 learned_addr learned my addr 172.21.15.18:0/1542688062 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.517+0000 7f0ca7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a94d60 0x5650d6a951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.518+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/1542688062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a94d60 msgr2=0x5650d6a951e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.518+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/1542688062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a94d60 0x5650d6a951e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.518+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/1542688062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a9e2e0 msgr2=0x5650d6aa06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.518+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/1542688062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a9e2e0 0x5650d6aa06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.518+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/1542688062 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650d69927b0 con 0x5650d6a95ad0 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.518+0000 7f0ca7fff640 1 --2- 172.21.15.18:0/1542688062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a94d60 0x5650d6a951e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.518+0000 7f0cb53dd640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0c980a22d0 0x7f0c980a4790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8005fd0 tx=0x7f0ca8009aa0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.518+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/1542688062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a95ad0 0x5650d6a94820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0bc990 tx=0x7f0cac034000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c6a9d9885ad34 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.518+0000 7f0c96ffd640 1 -- 172.21.15.18:0/1542688062 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0bca00 con 0x5650d6a95ad0 2024-02-17T12:52:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.518+0000 7f0c96ffd640 1 -- 172.21.15.18:0/1542688062 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cac0c0b50 con 0x5650d6a95ad0 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.518+0000 7f0c96ffd640 1 -- 172.21.15.18:0/1542688062 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0eda70 con 0x5650d6a95ad0 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.519+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1542688062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a95ad0 msgr2=0x5650d6a94820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.519+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1542688062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a95ad0 0x5650d6a94820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0bc990 tx=0x7f0cac034000 comp rx=0 tx=0).stop 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.519+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1542688062 shutdown_connections 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.519+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1542688062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a9e2e0 0x5650d6aa06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.519+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1542688062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a94d60 0x5650d6a951e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.519+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1542688062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a95ad0 0x5650d6a94820 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.519+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1542688062 >> 172.21.15.18:0/1542688062 conn(0x5650d6a5c550 msgr2=0x5650d6a8fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.519+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1542688062 shutdown_connections 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.519+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1542688062 wait complete. 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.519+0000 7f0cb9a12740 1 -- start start 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.519+0000 7f0cb9a12740 1 Processor -- start 2024-02-17T12:52:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb9a12740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a94d60 0x5650d6a9b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb9a12740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a95ad0 0x5650d6a9b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb9a12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a9e2e0 0x5650d6a9bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb9a12740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d69a1bf0 con 0x5650d6a94d60 2024-02-17T12:52:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb9a12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d66d0250 con 0x5650d6a9e2e0 2024-02-17T12:52:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb9a12740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d6982200 con 0x5650d6a95ad0 2024-02-17T12:52:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb9a12740 10 client.0 _ll_register_callbacks cb 0x5650d69f0d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb4bdc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a94d60 0x5650d6a9b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0ca7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a95ad0 0x5650d6a9b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb4bdc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a94d60 0x5650d6a9b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56080/0 (socket says 172.21.15.18:56080) 2024-02-17T12:52:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/1192856057 learned_addr learned my addr 172.21.15.18:0/1192856057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb53dd640 1 --2- 172.21.15.18:0/1192856057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a9e2e0 0x5650d6a9bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/1192856057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a95ad0 msgr2=0x5650d6a9b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/1192856057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a95ad0 0x5650d6a9b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/1192856057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a9e2e0 msgr2=0x5650d6a9bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/1192856057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a9e2e0 0x5650d6a9bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/1192856057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb01a4750 con 0x5650d6a94d60 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/1192856057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a94d60 0x5650d6a9b310 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0cac000c00 tx=0x7f0cac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6405231e8a767128 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0c8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.520+0000 7f0c8ffff640 1 -- 172.21.15.18:0/1192856057 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0c51a0 con 0x5650d6a94d60 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0c8ffff640 1 -- 172.21.15.18:0/1192856057 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0cac0bca00 con 0x5650d6a94d60 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1192856057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650d66fc600 con 0x5650d6a94d60 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0c8ffff640 1 -- 172.21.15.18:0/1192856057 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cac0c4d10 con 0x5650d6a94d60 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0cb9a12740 10 client.16383 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1192856057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5650d69fad80 con 0x5650d6a94d60 2024-02-17T12:52:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0c8ffff640 1 -- 172.21.15.18:0/1192856057 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5650d66fc600 con 0x5650d6a94d60 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0c8ffff640 1 -- 172.21.15.18:0/1192856057 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0cac03c070 con 0x5650d6a94d60 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0cb9a12740 10 client.16383 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0cb9a12740 10 client.16383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0cb9a12740 20 client.16383 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1192856057 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5650d6aae050 0x5650d6ab0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0cb9a12740 4 client.16383 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1192856057 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650d674d1b0 con 0x5650d6aae050 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.521+0000 7f0ca7fff640 1 --2- 172.21.15.18:0/1192856057 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5650d6aae050 0x5650d6ab0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.522+0000 7f0c8ffff640 10 client.16383 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.522+0000 7f0ca7fff640 1 --2- 172.21.15.18:0/1192856057 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5650d6aae050 0x5650d6ab0470 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cebe23180eeed17e server_cookie=9338b7fd948e4a55 in_seq=0 out_seq=0 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.522+0000 7f0c8ffff640 1 -- 172.21.15.18:0/1192856057 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650d674d1b0 con 0x5650d6aae050 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.522+0000 7f0c8ffff640 10 client.16383 handle_command_reply: tid=0 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.522+0000 7f0cb9a12740 1 client.16383 shutdown 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.523+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1192856057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a94d60 msgr2=0x5650d6a9b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.523+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1192856057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a94d60 0x5650d6a9b310 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0cac000c00 tx=0x7f0cac009260 comp rx=0 tx=0).stop 2024-02-17T12:52:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.523+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1192856057 shutdown_connections 2024-02-17T12:52:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.523+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1192856057 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5650d6aae050 0x5650d6ab0470 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.523+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1192856057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a9e2e0 0x5650d6a9bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.523+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1192856057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a95ad0 0x5650d6a9b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.523+0000 7f0c8ffff640 0 client.16383 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.523+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1192856057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a94d60 0x5650d6a9b310 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.523+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1192856057 >> 172.21.15.18:0/1192856057 conn(0x5650d6a5c550 msgr2=0x5650d6a917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.523+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1192856057 shutdown_connections 2024-02-17T12:52:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.523+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1192856057 wait complete. 2024-02-17T12:52:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.523+0000 7f0cb9a12740 20 client.16383 trim_cache size 0 max 16384 2024-02-17T12:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.537+0000 7f0cb9a12740 1 Processor -- start 2024-02-17T12:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.537+0000 7f0cb9a12740 1 -- start start 2024-02-17T12:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb9a12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a94d60 0x5650d6a9da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb9a12740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a95ad0 0x5650d6a9bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb9a12740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a9e2e0 0x5650d6a99be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb9a12740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d6a96d30 con 0x5650d6a95ad0 2024-02-17T12:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb9a12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d674d1b0 con 0x5650d6a94d60 2024-02-17T12:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb9a12740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d66fc600 con 0x5650d6a9e2e0 2024-02-17T12:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0ca7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a95ad0 0x5650d6a9bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0ca7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a95ad0 0x5650d6a9bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56090/0 (socket says 172.21.15.18:56090) 2024-02-17T12:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb4bdc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a94d60 0x5650d6a9da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0ca7fff640 1 -- 172.21.15.18:0/1606639869 learned_addr learned my addr 172.21.15.18:0/1606639869 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb53dd640 1 --2- 172.21.15.18:0/1606639869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a9e2e0 0x5650d6a99be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/1606639869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a9e2e0 msgr2=0x5650d6a99be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/1606639869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a9e2e0 0x5650d6a99be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/1606639869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a95ad0 msgr2=0x5650d6a9bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/1606639869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a95ad0 0x5650d6a9bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/1606639869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650d69977b0 con 0x5650d6a94d60 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0ca7fff640 1 --2- 172.21.15.18:0/1606639869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a95ad0 0x5650d6a9bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/1606639869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a94d60 0x5650d6a9da40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0b93d0 tx=0x7f0cac005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c89a06a3cdeef404 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.538+0000 7f0c967fc640 1 -- 172.21.15.18:0/1606639869 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0050a0 con 0x5650d6a94d60 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.539+0000 7f0c967fc640 1 -- 172.21.15.18:0/1606639869 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cac0ed810 con 0x5650d6a94d60 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.539+0000 7f0c967fc640 1 -- 172.21.15.18:0/1606639869 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0b8ad0 con 0x5650d6a94d60 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.539+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1606639869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a94d60 msgr2=0x5650d6a9da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.539+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1606639869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a94d60 0x5650d6a9da40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0b93d0 tx=0x7f0cac005fd0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.539+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1606639869 shutdown_connections 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.539+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1606639869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a9e2e0 0x5650d6a99be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.539+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1606639869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a95ad0 0x5650d6a9bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.539+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/1606639869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a94d60 0x5650d6a9da40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.539+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1606639869 >> 172.21.15.18:0/1606639869 conn(0x5650d6a5c550 msgr2=0x5650d6ab4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.539+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1606639869 shutdown_connections 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.539+0000 7f0cb9a12740 1 -- 172.21.15.18:0/1606639869 wait complete. 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb9a12740 1 -- start start 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb9a12740 1 Processor -- start 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb9a12740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a94d60 0x5650d6aa4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb9a12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a95ad0 0x5650d6aa5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb9a12740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a9e2e0 0x5650d6aa3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb9a12740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d6aa35f0 con 0x5650d6a9e2e0 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb9a12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d6aa3760 con 0x5650d6a95ad0 2024-02-17T12:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb9a12740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d6aa3920 con 0x5650d6a94d60 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb9a12740 10 client.0 _ll_register_callbacks cb 0x7f0cac0b8ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb53dd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a9e2e0 0x5650d6aa3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb53dd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a9e2e0 0x5650d6aa3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56098/0 (socket says 172.21.15.18:56098) 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb53dd640 1 -- 172.21.15.18:0/2863667400 learned_addr learned my addr 172.21.15.18:0/2863667400 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0ca7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a95ad0 0x5650d6aa5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.540+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a94d60 0x5650d6aa4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/2863667400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a95ad0 msgr2=0x5650d6aa5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a95ad0 0x5650d6aa5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/2863667400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a9e2e0 msgr2=0x5650d6aa3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a9e2e0 0x5650d6aa3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0cb4bdc640 1 -- 172.21.15.18:0/2863667400 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca0010430 con 0x5650d6a94d60 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0cb53dd640 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a9e2e0 0x5650d6aa3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0ca7fff640 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a95ad0 0x5650d6aa5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0cb4bdc640 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a94d60 0x5650d6aa4b00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0cac005d00 tx=0x7f0cac02c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3e0202a0f428605 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0c8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0c8ffff640 1 -- 172.21.15.18:0/2863667400 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0050a0 con 0x5650d6a94d60 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0c8ffff640 1 -- 172.21.15.18:0/2863667400 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0cac0bd100 con 0x5650d6a94d60 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0cb9a12740 1 -- 172.21.15.18:0/2863667400 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cac0c0b50 con 0x5650d6a94d60 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0c8ffff640 1 -- 172.21.15.18:0/2863667400 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cac0c4fc0 con 0x5650d6a94d60 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0cb9a12740 10 client.25807 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0cb9a12740 1 -- 172.21.15.18:0/2863667400 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5650d69fad80 con 0x5650d6a94d60 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.541+0000 7f0c8ffff640 1 -- 172.21.15.18:0/2863667400 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cac0c0b50 con 0x5650d6a94d60 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.542+0000 7f0c8ffff640 1 -- 172.21.15.18:0/2863667400 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0cac0a3020 con 0x5650d6a94d60 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.542+0000 7f0cb9a12740 10 client.25807 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.542+0000 7f0cb9a12740 10 client.25807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.542+0000 7f0cb9a12740 20 client.25807 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.542+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5650d6b051c0 0x5650d6b075b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.542+0000 7f0cb9a12740 4 client.25807 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.542+0000 7f0cb9a12740 1 -- 172.21.15.18:0/2863667400 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5650d6739550 con 0x5650d6b051c0 2024-02-17T12:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.542+0000 7f0ca7fff640 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5650d6b051c0 0x5650d6b075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.542+0000 7f0c8ffff640 10 client.25807 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.542+0000 7f0ca7fff640 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5650d6b051c0 0x5650d6b075b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=868559ca9f80e990 server_cookie=bea970b9d1ac9487 in_seq=0 out_seq=0 2024-02-17T12:52:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.572+0000 7f0c8ffff640 1 -- 172.21.15.18:0/2863667400 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5650d6739550 con 0x5650d6b051c0 2024-02-17T12:52:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.572+0000 7f0c8ffff640 10 client.25807 handle_command_reply: tid=0 2024-02-17T12:52:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0cb9a12740 1 client.25807 shutdown 2024-02-17T12:52:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0cb9a12740 1 -- 172.21.15.18:0/2863667400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a94d60 msgr2=0x5650d6aa4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a94d60 0x5650d6aa4b00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0cac005d00 tx=0x7f0cac02c740 comp rx=0 tx=0).stop 2024-02-17T12:52:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0cb9a12740 1 -- 172.21.15.18:0/2863667400 shutdown_connections 2024-02-17T12:52:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5650d6b051c0 0x5650d6b075b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5650d6a9e2e0 0x5650d6aa3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650d6a95ad0 0x5650d6aa5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0cb9a12740 1 --2- 172.21.15.18:0/2863667400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5650d6a94d60 0x5650d6aa4b00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0cb9a12740 1 -- 172.21.15.18:0/2863667400 >> 172.21.15.18:0/2863667400 conn(0x5650d6a5c550 msgr2=0x5650d6a99b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0c8ffff640 0 client.25807 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0cb9a12740 1 -- 172.21.15.18:0/2863667400 shutdown_connections 2024-02-17T12:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0cb9a12740 1 -- 172.21.15.18:0/2863667400 wait complete. 2024-02-17T12:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.573+0000 7f0cb9a12740 20 client.25807 trim_cache size 0 max 16384 2024-02-17T12:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:52:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:52:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.574+0000 7f0cb6e40640 1 -- 172.21.15.18:0/4164492910 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0c980a22d0 msgr2=0x7f0c980a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.574+0000 7f0cb6e40640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0c980a22d0 0x7f0c980a4790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8005fd0 tx=0x7f0ca8009aa0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.574+0000 7f0cb6e40640 1 -- 172.21.15.18:0/4164492910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cb01013c0 msgr2=0x7f0cb019d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.574+0000 7f0cb6e40640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cb01013c0 0x7f0cb019d3e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009230 tx=0x7f0cac02c650 comp rx=0 tx=0).stop 2024-02-17T12:52:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.574+0000 7f0cb6e40640 1 -- 172.21.15.18:0/4164492910 shutdown_connections 2024-02-17T12:52:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.574+0000 7f0cb6e40640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0c980a22d0 0x7f0c980a4790 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.574+0000 7f0cb6e40640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cb0197630 0x7f0cb0197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.574+0000 7f0cb6e40640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cb0106490 0x7f0cb019d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.574+0000 7f0cb6e40640 1 --2- 172.21.15.18:0/4164492910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cb01013c0 0x7f0cb019d3e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.574+0000 7f0cb6e40640 1 -- 172.21.15.18:0/4164492910 >> 172.21.15.18:0/4164492910 conn(0x7f0cb00fcc60 msgr2=0x7f0cb00fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.574+0000 7f0cb6e40640 1 -- 172.21.15.18:0/4164492910 shutdown_connections 2024-02-17T12:52:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:11.574+0000 7f0cb6e40640 1 -- 172.21.15.18:0/4164492910 wait complete. 2024-02-17T12:52:12.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 74971 4.86 MB/sec warmup 87 sec latency 22.073 ms 2024-02-17T12:52:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:12 smithi119 ceph-mon[36188]: pgmap v72: 97 pgs: 97 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 654 KiB/s wr, 100 op/s 2024-02-17T12:52:12.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:12 smithi146 ceph-mon[36454]: pgmap v72: 97 pgs: 97 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 654 KiB/s wr, 100 op/s 2024-02-17T12:52:12.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:12 smithi018 ceph-mon[32625]: pgmap v72: 97 pgs: 97 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 654 KiB/s wr, 100 op/s 2024-02-17T12:52:13.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 75266 4.80 MB/sec warmup 88 sec latency 732.404 ms 2024-02-17T12:52:14.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 75266 4.75 MB/sec warmup 89 sec latency 1732.462 ms 2024-02-17T12:52:14.576 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:14 smithi018 ceph-mon[32625]: pgmap v73: 97 pgs: 97 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 654 KiB/s wr, 100 op/s 2024-02-17T12:52:14.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:14 smithi119 ceph-mon[36188]: pgmap v73: 97 pgs: 97 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 654 KiB/s wr, 100 op/s 2024-02-17T12:52:14.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:14 smithi146 ceph-mon[36454]: pgmap v73: 97 pgs: 97 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 654 KiB/s wr, 100 op/s 2024-02-17T12:52:15.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 75266 4.70 MB/sec warmup 90 sec latency 2732.578 ms 2024-02-17T12:52:16.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 75334 4.65 MB/sec warmup 91 sec latency 3298.523 ms 2024-02-17T12:52:16.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:16 smithi119 ceph-mon[36188]: pgmap v74: 97 pgs: 97 active+clean; 37 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-02-17T12:52:16.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:16 smithi146 ceph-mon[36454]: pgmap v74: 97 pgs: 97 active+clean; 37 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-02-17T12:52:16.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:16 smithi018 ceph-mon[32625]: pgmap v74: 97 pgs: 97 active+clean; 37 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-02-17T12:52:17.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 75334 4.60 MB/sec warmup 92 sec latency 1388.435 ms 2024-02-17T12:52:18.264 INFO:tasks.workunit.client.0.smithi018.stdout: 1 75334 4.55 MB/sec warmup 93 sec latency 2388.518 ms 2024-02-17T12:52:18.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:18 smithi119 ceph-mon[36188]: pgmap v75: 97 pgs: 97 active+clean; 37 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 545 KiB/s wr, 75 op/s 2024-02-17T12:52:18.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:18 smithi146 ceph-mon[36454]: pgmap v75: 97 pgs: 97 active+clean; 37 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 545 KiB/s wr, 75 op/s 2024-02-17T12:52:18.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:18 smithi018 ceph-mon[32625]: pgmap v75: 97 pgs: 97 active+clean; 37 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 545 KiB/s wr, 75 op/s 2024-02-17T12:52:19.265 INFO:tasks.workunit.client.0.smithi018.stdout: 1 75334 4.50 MB/sec warmup 94 sec latency 3388.606 ms 2024-02-17T12:52:20.265 INFO:tasks.workunit.client.0.smithi018.stdout: 1 75334 4.45 MB/sec warmup 95 sec latency 4388.753 ms 2024-02-17T12:52:20.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:20 smithi119 ceph-mon[36188]: pgmap v76: 97 pgs: 97 active+clean; 41 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 897 KiB/s wr, 91 op/s 2024-02-17T12:52:20.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:20 smithi146 ceph-mon[36454]: pgmap v76: 97 pgs: 97 active+clean; 41 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 897 KiB/s wr, 91 op/s 2024-02-17T12:52:20.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:20 smithi018 ceph-mon[32625]: pgmap v76: 97 pgs: 97 active+clean; 41 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 897 KiB/s wr, 91 op/s 2024-02-17T12:52:21.265 INFO:tasks.workunit.client.0.smithi018.stdout: 1 76522 4.46 MB/sec warmup 96 sec latency 4955.090 ms 2024-02-17T12:52:22.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 78553 4.55 MB/sec warmup 97 sec latency 13.840 ms 2024-02-17T12:52:22.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:22 smithi119 ceph-mon[36188]: pgmap v77: 97 pgs: 97 active+clean; 41 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 835 KiB/s wr, 79 op/s 2024-02-17T12:52:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:22 smithi146 ceph-mon[36454]: pgmap v77: 97 pgs: 97 active+clean; 41 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 835 KiB/s wr, 79 op/s 2024-02-17T12:52:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:22 smithi018 ceph-mon[32625]: pgmap v77: 97 pgs: 97 active+clean; 41 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 835 KiB/s wr, 79 op/s 2024-02-17T12:52:23.265 INFO:tasks.workunit.client.0.smithi018.stdout: 1 78877 4.51 MB/sec warmup 98 sec latency 786.210 ms 2024-02-17T12:52:24.265 INFO:tasks.workunit.client.0.smithi018.stdout: 1 78877 4.47 MB/sec warmup 99 sec latency 1786.323 ms 2024-02-17T12:52:24.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:24 smithi119 ceph-mon[36188]: pgmap v78: 97 pgs: 97 active+clean; 41 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 835 KiB/s wr, 79 op/s 2024-02-17T12:52:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.671+0000 7f5a61215640 1 -- 172.21.15.18:0/3564854517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5c100040 msgr2=0x7f5a5c10e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.671+0000 7f5a61215640 1 --2- 172.21.15.18:0/3564854517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5c100040 0x7f5a5c10e010 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009cd0 tx=0x7f5a5402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.671+0000 7f5a61215640 1 -- 172.21.15.18:0/3564854517 shutdown_connections 2024-02-17T12:52:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.671+0000 7f5a61215640 1 --2- 172.21.15.18:0/3564854517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5c100040 0x7f5a5c10e010 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.671+0000 7f5a61215640 1 --2- 172.21.15.18:0/3564854517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a5c0ff680 0x7f5a5c0ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.671+0000 7f5a61215640 1 --2- 172.21.15.18:0/3564854517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c106160 0x7f5a5c106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.671+0000 7f5a61215640 1 -- 172.21.15.18:0/3564854517 >> 172.21.15.18:0/3564854517 conn(0x7f5a5c076610 msgr2=0x7f5a5c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.671+0000 7f5a61215640 1 -- 172.21.15.18:0/3564854517 shutdown_connections 2024-02-17T12:52:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.671+0000 7f5a61215640 1 -- 172.21.15.18:0/3564854517 wait complete. 2024-02-17T12:52:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.672+0000 7f5a61215640 1 Processor -- start 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.672+0000 7f5a61215640 1 -- start start 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.672+0000 7f5a61215640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c0ff680 0x7f5a5c197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.672+0000 7f5a61215640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a5c100040 0x7f5a5c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.672+0000 7f5a61215640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5c106160 0x7f5a5c19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.672+0000 7f5a61215640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c19c4d0 con 0x7f5a5c106160 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.672+0000 7f5a61215640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c19c640 con 0x7f5a5c0ff680 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.672+0000 7f5a61215640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c19c800 con 0x7f5a5c100040 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.672+0000 7f5a5b59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5c106160 0x7f5a5c19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.672+0000 7f5a5b59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5c106160 0x7f5a5c19bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34080/0 (socket says 172.21.15.18:34080) 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.672+0000 7f5a5b59e640 1 -- 172.21.15.18:0/3499085360 learned_addr learned my addr 172.21.15.18:0/3499085360 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a5c100040 0x7f5a5c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a5ad9d640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c0ff680 0x7f5a5c197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a5b59e640 1 -- 172.21.15.18:0/3499085360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a5c100040 msgr2=0x7f5a5c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a5b59e640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a5c100040 0x7f5a5c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a5b59e640 1 -- 172.21.15.18:0/3499085360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c0ff680 msgr2=0x7f5a5c197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a5b59e640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c0ff680 0x7f5a5c197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a5b59e640 1 -- 172.21.15.18:0/3499085360 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a54009900 con 0x7f5a5c106160 2024-02-17T12:52:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a5ad9d640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c0ff680 0x7f5a5c197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:52:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a5b59e640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5c106160 0x7f5a5c19bed0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5a54037040 tx=0x7f5a54034d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a43fff640 1 -- 172.21.15.18:0/3499085360 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54032ac0 con 0x7f5a5c106160 2024-02-17T12:52:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a61215640 1 -- 172.21.15.18:0/3499085360 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5c19ca80 con 0x7f5a5c106160 2024-02-17T12:52:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a43fff640 1 -- 172.21.15.18:0/3499085360 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a54003e50 con 0x7f5a5c106160 2024-02-17T12:52:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a43fff640 1 -- 172.21.15.18:0/3499085360 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a540406b0 con 0x7f5a5c106160 2024-02-17T12:52:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.673+0000 7f5a61215640 1 -- 172.21.15.18:0/3499085360 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c1a4360 con 0x7f5a5c106160 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a43fff640 1 -- 172.21.15.18:0/3499085360 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5a5400a040 con 0x7f5a5c106160 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a63de2740 1 Processor -- start 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a63de2740 1 -- start start 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a63de2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddecad0 0x55d54ddeb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a63de2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddebd60 0x55d54ddec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a63de2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddf76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a63de2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54dd5a9f0 con 0x55d54ddf52e0 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a63de2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54dd57f20 con 0x55d54ddebd60 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a63de2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54dd53700 con 0x55d54ddecad0 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a5a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddf76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a5a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddf76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34090/0 (socket says 172.21.15.18:34090) 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a5a59c640 1 -- 172.21.15.18:0/3134292224 learned_addr learned my addr 172.21.15.18:0/3134292224 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a43fff640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5a440a22d0 0x7f5a440a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/3134292224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddecad0 0x55d54ddeb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.676+0000 7f5a5ad9d640 1 --2- 172.21.15.18:0/3134292224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddebd60 0x55d54ddec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.677+0000 7f5a5b59e640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5a440a22d0 0x7f5a440a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.677+0000 7f5a43fff640 1 -- 172.21.15.18:0/3499085360 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5a540f0f20 con 0x7f5a5c106160 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.677+0000 7f5a5a59c640 1 -- 172.21.15.18:0/3134292224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddecad0 msgr2=0x55d54ddeb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.677+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/3134292224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddecad0 0x55d54ddeb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.677+0000 7f5a5a59c640 1 -- 172.21.15.18:0/3134292224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddebd60 msgr2=0x55d54ddec1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.677+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/3134292224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddebd60 0x55d54ddec1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.677+0000 7f5a5a59c640 1 -- 172.21.15.18:0/3134292224 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d54dce97b0 con 0x55d54ddf52e0 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.677+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/3134292224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddf76d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5a500117e0 tx=0x7f5a5000a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f853108e497f858c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.677+0000 7f5a5b59e640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5a440a22d0 0x7f5a440a4790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5a540eecc0 tx=0x7f5a5402c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.677+0000 7f5a417fa640 1 -- 172.21.15.18:0/3134292224 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50011e20 con 0x55d54ddf52e0 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.677+0000 7f5a417fa640 1 -- 172.21.15.18:0/3134292224 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a5000feb0 con 0x55d54ddf52e0 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.677+0000 7f5a417fa640 1 -- 172.21.15.18:0/3134292224 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50004a00 con 0x55d54ddf52e0 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.678+0000 7f5a63de2740 1 -- 172.21.15.18:0/3134292224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 msgr2=0x55d54ddf76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.678+0000 7f5a63de2740 1 --2- 172.21.15.18:0/3134292224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddf76d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5a500117e0 tx=0x7f5a5000a760 comp rx=0 tx=0).stop 2024-02-17T12:52:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.678+0000 7f5a63de2740 1 -- 172.21.15.18:0/3134292224 shutdown_connections 2024-02-17T12:52:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.678+0000 7f5a63de2740 1 --2- 172.21.15.18:0/3134292224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddf76d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.678+0000 7f5a63de2740 1 --2- 172.21.15.18:0/3134292224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddebd60 0x55d54ddec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.678+0000 7f5a63de2740 1 --2- 172.21.15.18:0/3134292224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddecad0 0x55d54ddeb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.678+0000 7f5a63de2740 1 -- 172.21.15.18:0/3134292224 >> 172.21.15.18:0/3134292224 conn(0x55d54ddb3550 msgr2=0x55d54dde6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.678+0000 7f5a63de2740 1 -- 172.21.15.18:0/3134292224 shutdown_connections 2024-02-17T12:52:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.678+0000 7f5a63de2740 1 -- 172.21.15.18:0/3134292224 wait complete. 2024-02-17T12:52:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.678+0000 7f5a63de2740 1 -- start start 2024-02-17T12:52:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.678+0000 7f5a63de2740 1 Processor -- start 2024-02-17T12:52:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a63de2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddebd60 0x55d54ddf23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a63de2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddecad0 0x55d54ddf28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a63de2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddf2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a63de2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54dcf8bf0 con 0x55d54ddf52e0 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a63de2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54da27250 con 0x55d54ddecad0 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a63de2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54dcd9200 con 0x55d54ddebd60 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a63de2740 10 client.0 _ll_register_callbacks cb 0x55d54dd47d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a5a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddf2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a5a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddf2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34092/0 (socket says 172.21.15.18:34092) 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a5a59c640 1 -- 172.21.15.18:0/1489394189 learned_addr learned my addr 172.21.15.18:0/1489394189 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a5ad9d640 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddecad0 0x55d54ddf28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddebd60 0x55d54ddf23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a5ad9d640 1 -- 172.21.15.18:0/1489394189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddebd60 msgr2=0x55d54ddf23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a5ad9d640 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddebd60 0x55d54ddf23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a5ad9d640 1 -- 172.21.15.18:0/1489394189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 msgr2=0x55d54ddf2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a5ad9d640 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddf2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a5ad9d640 1 -- 172.21.15.18:0/1489394189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a50019040 con 0x55d54ddecad0 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddf2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a5ad9d640 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddecad0 0x55d54ddf28f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c011790 tx=0x7f5a4c011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb4f9ec9c348c640 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.679+0000 7f5a3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.680+0000 7f5a3eff5640 1 -- 172.21.15.18:0/1489394189 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c011ea0 con 0x55d54ddecad0 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.680+0000 7f5a63de2740 1 -- 172.21.15.18:0/1489394189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d54da53600 con 0x55d54ddecad0 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.680+0000 7f5a3eff5640 1 -- 172.21.15.18:0/1489394189 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5a4c00f9a0 con 0x55d54ddecad0 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.680+0000 7f5a3eff5640 1 -- 172.21.15.18:0/1489394189 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a4c004520 con 0x55d54ddecad0 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.680+0000 7f5a63de2740 10 client.25790 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.680+0000 7f5a63de2740 1 -- 172.21.15.18:0/1489394189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d54dd51d80 con 0x55d54ddecad0 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.680+0000 7f5a3eff5640 1 -- 172.21.15.18:0/1489394189 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d54da53600 con 0x55d54ddecad0 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.680+0000 7f5a3eff5640 1 -- 172.21.15.18:0/1489394189 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5a4c022430 con 0x55d54ddecad0 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.680+0000 7f5a63de2740 10 client.25790 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.680+0000 7f5a63de2740 10 client.25790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.680+0000 7f5a63de2740 20 client.25790 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.680+0000 7f5a63de2740 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d54de058a0 0x55d54de07cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.681+0000 7f5a63de2740 4 client.25790 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.681+0000 7f5a63de2740 1 -- 172.21.15.18:0/1489394189 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d54daa41b0 con 0x55d54de058a0 2024-02-17T12:52:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.681+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d54de058a0 0x55d54de07cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.681+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d54de058a0 0x55d54de07cc0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631d5a86abd0ac4a server_cookie=5d42e550b0b022ad in_seq=0 out_seq=0 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.681+0000 7f5a3eff5640 10 client.25790 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a3eff5640 1 -- 172.21.15.18:0/1489394189 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d54daa41b0 con 0x55d54de058a0 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a3eff5640 10 client.25790 handle_command_reply: tid=0 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a63de2740 1 client.25790 shutdown 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a63de2740 1 -- 172.21.15.18:0/1489394189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddecad0 msgr2=0x55d54ddf28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a63de2740 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddecad0 0x55d54ddf28f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c011790 tx=0x7f5a4c011c50 comp rx=0 tx=0).stop 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a63de2740 1 -- 172.21.15.18:0/1489394189 shutdown_connections 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a63de2740 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d54de058a0 0x55d54de07cc0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a63de2740 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddf2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a63de2740 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddecad0 0x55d54ddf28f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a3eff5640 0 client.25790 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a63de2740 1 --2- 172.21.15.18:0/1489394189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddebd60 0x55d54ddf23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a63de2740 1 -- 172.21.15.18:0/1489394189 >> 172.21.15.18:0/1489394189 conn(0x55d54ddb3550 msgr2=0x55d54dde87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a63de2740 1 -- 172.21.15.18:0/1489394189 shutdown_connections 2024-02-17T12:52:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a63de2740 1 -- 172.21.15.18:0/1489394189 wait complete. 2024-02-17T12:52:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.682+0000 7f5a63de2740 20 client.25790 trim_cache size 0 max 16384 2024-02-17T12:52:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.695+0000 7f5a63de2740 1 Processor -- start 2024-02-17T12:52:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.695+0000 7f5a63de2740 1 -- start start 2024-02-17T12:52:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.695+0000 7f5a63de2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddebd60 0x55d54ddfc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.695+0000 7f5a63de2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddecad0 0x55d54ddfc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.695+0000 7f5a63de2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddf52e0 0x55d54ddf0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.695+0000 7f5a63de2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54dde2180 con 0x55d54ddebd60 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.695+0000 7f5a63de2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54ddf7b10 con 0x55d54ddecad0 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.695+0000 7f5a63de2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54dde7a50 con 0x55d54ddf52e0 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a5a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddebd60 0x55d54ddfc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a5a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddebd60 0x55d54ddfc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34100/0 (socket says 172.21.15.18:34100) 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a5a59c640 1 -- 172.21.15.18:0/3170288681 learned_addr learned my addr 172.21.15.18:0/3170288681 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a5ad9d640 1 --2- 172.21.15.18:0/3170288681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddecad0 0x55d54ddfc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a5a59c640 1 -- 172.21.15.18:0/3170288681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddf52e0 msgr2=0x55d54ddf0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/3170288681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddf52e0 0x55d54ddf0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a5a59c640 1 -- 172.21.15.18:0/3170288681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddecad0 msgr2=0x55d54ddfc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/3170288681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddecad0 0x55d54ddfc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a5a59c640 1 -- 172.21.15.18:0/3170288681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d54dcee7b0 con 0x55d54ddebd60 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/3170288681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddebd60 0x55d54ddfc100 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000f790 tx=0x7f5a50013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e524b7fc06c6e006 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a40ff9640 1 -- 172.21.15.18:0/3170288681 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50024040 con 0x55d54ddebd60 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a40ff9640 1 -- 172.21.15.18:0/3170288681 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a50005650 con 0x55d54ddebd60 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.696+0000 7f5a40ff9640 1 -- 172.21.15.18:0/3170288681 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50011200 con 0x55d54ddebd60 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.697+0000 7f5a63de2740 1 -- 172.21.15.18:0/3170288681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddebd60 msgr2=0x55d54ddfc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.697+0000 7f5a63de2740 1 --2- 172.21.15.18:0/3170288681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddebd60 0x55d54ddfc100 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000f790 tx=0x7f5a50013040 comp rx=0 tx=0).stop 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.697+0000 7f5a63de2740 1 -- 172.21.15.18:0/3170288681 shutdown_connections 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.697+0000 7f5a63de2740 1 --2- 172.21.15.18:0/3170288681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddf52e0 0x55d54ddf0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.697+0000 7f5a63de2740 1 --2- 172.21.15.18:0/3170288681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddecad0 0x55d54ddfc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.697+0000 7f5a63de2740 1 --2- 172.21.15.18:0/3170288681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddebd60 0x55d54ddfc100 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.697+0000 7f5a63de2740 1 -- 172.21.15.18:0/3170288681 >> 172.21.15.18:0/3170288681 conn(0x55d54ddb3550 msgr2=0x55d54dde6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.697+0000 7f5a63de2740 1 -- 172.21.15.18:0/3170288681 shutdown_connections 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.697+0000 7f5a63de2740 1 -- 172.21.15.18:0/3170288681 wait complete. 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.697+0000 7f5a63de2740 1 -- start start 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.697+0000 7f5a63de2740 1 Processor -- start 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a63de2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddebd60 0x55d54ddfc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a63de2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddecad0 0x55d54ddefb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a63de2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a63de2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54ddf0430 con 0x55d54ddf52e0 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a63de2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54ddeeb60 con 0x55d54ddebd60 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a63de2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54ddeed20 con 0x55d54ddecad0 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a63de2740 10 client.0 _ll_register_callbacks cb 0x7f5a5000aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a5a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a5a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddee620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34112/0 (socket says 172.21.15.18:34112) 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a5a59c640 1 -- 172.21.15.18:0/1968595019 learned_addr learned my addr 172.21.15.18:0/1968595019 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a5ad9d640 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddecad0 0x55d54ddefb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddebd60 0x55d54ddfc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a5a59c640 1 -- 172.21.15.18:0/1968595019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddecad0 msgr2=0x55d54ddefb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddecad0 0x55d54ddefb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a5a59c640 1 -- 172.21.15.18:0/1968595019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddebd60 msgr2=0x55d54ddfc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddebd60 0x55d54ddfc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a5a59c640 1 -- 172.21.15.18:0/1968595019 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a50027040 con 0x55d54ddf52e0 2024-02-17T12:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a5a59c640 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddee620 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5a500110a0 tx=0x7f5a500051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8fa6ca23e2f6b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a3eff5640 1 -- 172.21.15.18:0/1968595019 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50024040 con 0x55d54ddf52e0 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a3eff5640 1 -- 172.21.15.18:0/1968595019 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5a50004eb0 con 0x55d54ddf52e0 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.698+0000 7f5a63de2740 1 -- 172.21.15.18:0/1968595019 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a5000feb0 con 0x55d54ddf52e0 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.699+0000 7f5a3eff5640 1 -- 172.21.15.18:0/1968595019 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a500111a0 con 0x55d54ddf52e0 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.699+0000 7f5a63de2740 10 client.16407 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.699+0000 7f5a63de2740 1 -- 172.21.15.18:0/1968595019 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d54dd14540 con 0x55d54ddf52e0 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.699+0000 7f5a3eff5640 1 -- 172.21.15.18:0/1968595019 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a5000feb0 con 0x55d54ddf52e0 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.699+0000 7f5a3eff5640 1 -- 172.21.15.18:0/1968595019 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5a5002e4f0 con 0x55d54ddf52e0 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.699+0000 7f5a63de2740 10 client.16407 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.699+0000 7f5a63de2740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.699+0000 7f5a63de2740 20 client.16407 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.699+0000 7f5a63de2740 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d54de5c990 0x55d54de5edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.699+0000 7f5a63de2740 4 client.16407 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.699+0000 7f5a63de2740 1 -- 172.21.15.18:0/1968595019 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d54da90550 con 0x55d54de5c990 2024-02-17T12:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.699+0000 7f5a5ad9d640 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d54de5c990 0x55d54de5edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.700+0000 7f5a5ad9d640 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d54de5c990 0x55d54de5edb0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c9937e62be681b6 server_cookie=949526e8c608de4e in_seq=0 out_seq=0 2024-02-17T12:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.700+0000 7f5a3eff5640 10 client.16407 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:24.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:24 smithi146 ceph-mon[36454]: pgmap v78: 97 pgs: 97 active+clean; 41 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 835 KiB/s wr, 79 op/s 2024-02-17T12:52:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.730+0000 7f5a3eff5640 1 -- 172.21.15.18:0/1968595019 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d54da90550 con 0x55d54de5c990 2024-02-17T12:52:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.730+0000 7f5a3eff5640 10 client.16407 handle_command_reply: tid=0 2024-02-17T12:52:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.730+0000 7f5a63de2740 1 client.16407 shutdown 2024-02-17T12:52:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.731+0000 7f5a63de2740 1 -- 172.21.15.18:0/1968595019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 msgr2=0x55d54ddee620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.731+0000 7f5a63de2740 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddee620 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5a500110a0 tx=0x7f5a500051d0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.731+0000 7f5a63de2740 1 -- 172.21.15.18:0/1968595019 shutdown_connections 2024-02-17T12:52:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.731+0000 7f5a63de2740 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d54de5c990 0x55d54de5edb0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.731+0000 7f5a63de2740 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d54ddf52e0 0x55d54ddee620 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.731+0000 7f5a63de2740 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d54ddecad0 0x55d54ddefb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.731+0000 7f5a63de2740 1 --2- 172.21.15.18:0/1968595019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d54ddebd60 0x55d54ddfc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.731+0000 7f5a3eff5640 0 client.16407 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.731+0000 7f5a63de2740 1 -- 172.21.15.18:0/1968595019 >> 172.21.15.18:0/1968595019 conn(0x55d54ddb3550 msgr2=0x55d54ddf0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.731+0000 7f5a63de2740 1 -- 172.21.15.18:0/1968595019 shutdown_connections 2024-02-17T12:52:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.731+0000 7f5a63de2740 1 -- 172.21.15.18:0/1968595019 wait complete. 2024-02-17T12:52:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.731+0000 7f5a63de2740 20 client.16407 trim_cache size 0 max 16384 2024-02-17T12:52:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:52:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:52:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:52:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:52:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.732+0000 7f5a61215640 1 -- 172.21.15.18:0/3499085360 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5a440a22d0 msgr2=0x7f5a440a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.732+0000 7f5a61215640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5a440a22d0 0x7f5a440a4790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5a540eecc0 tx=0x7f5a5402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.732+0000 7f5a61215640 1 -- 172.21.15.18:0/3499085360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5c106160 msgr2=0x7f5a5c19bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.732+0000 7f5a61215640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5c106160 0x7f5a5c19bed0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5a54037040 tx=0x7f5a54034d30 comp rx=0 tx=0).stop 2024-02-17T12:52:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.732+0000 7f5a61215640 1 -- 172.21.15.18:0/3499085360 shutdown_connections 2024-02-17T12:52:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.732+0000 7f5a61215640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5a440a22d0 0x7f5a440a4790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.732+0000 7f5a61215640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a5c106160 0x7f5a5c19bed0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.732+0000 7f5a61215640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a5c100040 0x7f5a5c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.732+0000 7f5a61215640 1 --2- 172.21.15.18:0/3499085360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c0ff680 0x7f5a5c197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.732+0000 7f5a61215640 1 -- 172.21.15.18:0/3499085360 >> 172.21.15.18:0/3499085360 conn(0x7f5a5c076610 msgr2=0x7f5a5c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.732+0000 7f5a61215640 1 -- 172.21.15.18:0/3499085360 shutdown_connections 2024-02-17T12:52:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:24.732+0000 7f5a61215640 1 -- 172.21.15.18:0/3499085360 wait complete. 2024-02-17T12:52:24.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:24 smithi018 ceph-mon[32625]: pgmap v78: 97 pgs: 97 active+clean; 41 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 835 KiB/s wr, 79 op/s 2024-02-17T12:52:25.265 INFO:tasks.workunit.client.0.smithi018.stdout: 1 78877 4.42 MB/sec warmup 100 sec latency 2786.416 ms 2024-02-17T12:52:25.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:52:25.483 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:52:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.608+0000 7fccb7639640 1 -- 172.21.15.18:0/2756009000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb01060c0 msgr2=0x7fccb00697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.608+0000 7fccb7639640 1 --2- 172.21.15.18:0/2756009000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb01060c0 0x7fccb00697b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fccac009900 tx=0x7fccac02bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.608+0000 7fccb7639640 1 -- 172.21.15.18:0/2756009000 shutdown_connections 2024-02-17T12:52:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 --2- 172.21.15.18:0/2756009000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0103bc0 0x7fccb0104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 --2- 172.21.15.18:0/2756009000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb0069cf0 0x7fccb006a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 --2- 172.21.15.18:0/2756009000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb01060c0 0x7fccb00697b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 -- 172.21.15.18:0/2756009000 >> 172.21.15.18:0/2756009000 conn(0x7fccb0076e20 msgr2=0x7fccb0077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 -- 172.21.15.18:0/2756009000 shutdown_connections 2024-02-17T12:52:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 -- 172.21.15.18:0/2756009000 wait complete. 2024-02-17T12:52:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 Processor -- start 2024-02-17T12:52:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 -- start start 2024-02-17T12:52:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb0069cf0 0x7fccb01a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0103bc0 0x7fccb01a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb01060c0 0x7fccb019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb019dfc0 con 0x7fccb01060c0 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb019e130 con 0x7fccb0103bc0 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.609+0000 7fccb7639640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb019e2f0 con 0x7fccb0069cf0 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb5bd6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb01060c0 0x7fccb019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb5bd6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb01060c0 0x7fccb019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34126/0 (socket says 172.21.15.18:34126) 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb5bd6640 1 -- 172.21.15.18:0/1897901808 learned_addr learned my addr 172.21.15.18:0/1897901808 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb53d5640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb0069cf0 0x7fccb01a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0103bc0 0x7fccb01a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb5bd6640 1 -- 172.21.15.18:0/1897901808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb0069cf0 msgr2=0x7fccb01a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb5bd6640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb0069cf0 0x7fccb01a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb5bd6640 1 -- 172.21.15.18:0/1897901808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0103bc0 msgr2=0x7fccb01a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb5bd6640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0103bc0 0x7fccb01a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb5bd6640 1 -- 172.21.15.18:0/1897901808 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac009560 con 0x7fccb01060c0 2024-02-17T12:52:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0103bc0 0x7fccb01a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:52:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb5bd6640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb01060c0 0x7fccb019d970 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcca800cce0 tx=0x7fcca800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fcca67fc640 1 -- 172.21.15.18:0/1897901808 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca800f450 con 0x7fccb01060c0 2024-02-17T12:52:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fcca67fc640 1 -- 172.21.15.18:0/1897901808 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca800fa90 con 0x7fccb01060c0 2024-02-17T12:52:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb7639640 1 -- 172.21.15.18:0/1897901808 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb019e5d0 con 0x7fccb01060c0 2024-02-17T12:52:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fcca67fc640 1 -- 172.21.15.18:0/1897901808 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca8018b90 con 0x7fccb01060c0 2024-02-17T12:52:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.610+0000 7fccb7639640 1 -- 172.21.15.18:0/1897901808 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb01aa860 con 0x7fccb01060c0 2024-02-17T12:52:25.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.613+0000 7fcca67fc640 1 -- 172.21.15.18:0/1897901808 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcca800fc00 con 0x7fccb01060c0 2024-02-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.613+0000 7fccba20a740 1 Processor -- start 2024-02-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.613+0000 7fccba20a740 1 -- start start 2024-02-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.613+0000 7fccba20a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b10572b0 0x55f4b1057690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.613+0000 7fcca67fc640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc9c0a22b0 0x7fcc9c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.613+0000 7fccba20a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b1057bd0 0x55f4b1060270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.613+0000 7fccba20a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10607b0 0x55f4b1062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.613+0000 7fccba20a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b0fc6280 con 0x55f4b10607b0 2024-02-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.613+0000 7fccba20a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b0fc37b0 con 0x55f4b10572b0 2024-02-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccba20a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b0fbef90 con 0x55f4b1057bd0 2024-02-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb53d5640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc9c0a22b0 0x7fcc9c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb5bd6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10607b0 0x55f4b1062ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb5bd6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10607b0 0x55f4b1062ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34138/0 (socket says 172.21.15.18:34138) 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb5bd6640 1 -- 172.21.15.18:0/3071565782 learned_addr learned my addr 172.21.15.18:0/3071565782 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/3071565782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b10572b0 0x55f4b1057690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fcca67fc640 1 -- 172.21.15.18:0/1897901808 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcca80ce7a0 con 0x7fccb01060c0 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/3071565782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b1057bd0 0x55f4b1060270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb5bd6640 1 -- 172.21.15.18:0/3071565782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b1057bd0 msgr2=0x55f4b1060270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb5bd6640 1 --2- 172.21.15.18:0/3071565782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b1057bd0 0x55f4b1060270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb5bd6640 1 -- 172.21.15.18:0/3071565782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b10572b0 msgr2=0x55f4b1057690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb5bd6640 1 --2- 172.21.15.18:0/3071565782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b10572b0 0x55f4b1057690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb5bd6640 1 -- 172.21.15.18:0/3071565782 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4b0f7d570 con 0x55f4b10607b0 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/3071565782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b10572b0 0x55f4b1057690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb5bd6640 1 --2- 172.21.15.18:0/3071565782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10607b0 0x55f4b1062ba0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcca80cc540 tx=0x7fcca800af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134533b67b77abac server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:25.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fccb53d5640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc9c0a22b0 0x7fcc9c0a4770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fccac005ec0 tx=0x7fccac0058d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fcc9b7fe640 1 -- 172.21.15.18:0/3071565782 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca809ad20 con 0x55f4b10607b0 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fcc9b7fe640 1 -- 172.21.15.18:0/3071565782 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca809ab40 con 0x55f4b10607b0 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.614+0000 7fcc9b7fe640 1 -- 172.21.15.18:0/3071565782 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca809f330 con 0x55f4b10607b0 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.615+0000 7fccba20a740 1 -- 172.21.15.18:0/3071565782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10607b0 msgr2=0x55f4b1062ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.615+0000 7fccba20a740 1 --2- 172.21.15.18:0/3071565782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10607b0 0x55f4b1062ba0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcca80cc540 tx=0x7fcca800af30 comp rx=0 tx=0).stop 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.615+0000 7fccba20a740 1 -- 172.21.15.18:0/3071565782 shutdown_connections 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.615+0000 7fccba20a740 1 --2- 172.21.15.18:0/3071565782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10607b0 0x55f4b1062ba0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.615+0000 7fccba20a740 1 --2- 172.21.15.18:0/3071565782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b1057bd0 0x55f4b1060270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.615+0000 7fccba20a740 1 --2- 172.21.15.18:0/3071565782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b10572b0 0x55f4b1057690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.615+0000 7fccba20a740 1 -- 172.21.15.18:0/3071565782 >> 172.21.15.18:0/3071565782 conn(0x55f4b1051990 msgr2=0x55f4b1051d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.615+0000 7fccba20a740 1 -- 172.21.15.18:0/3071565782 shutdown_connections 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.615+0000 7fccba20a740 1 -- 172.21.15.18:0/3071565782 wait complete. 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccba20a740 1 -- start start 2024-02-17T12:52:25.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccba20a740 1 Processor -- start 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccba20a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 0x55f4b1060200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccba20a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b1057bd0 0x55f4b105d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccba20a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10607b0 0x55f4b105dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccba20a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b0c92430 con 0x55f4b10572b0 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccba20a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b0c4ee00 con 0x55f4b1057bd0 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccba20a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b0f43fd0 con 0x55f4b10607b0 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccba20a740 10 client.0 _ll_register_callbacks cb 0x55f4b0f99fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccb53d5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 0x55f4b1060200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccb53d5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 0x55f4b1060200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34140/0 (socket says 172.21.15.18:34140) 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccb53d5640 1 -- 172.21.15.18:0/133384367 learned_addr learned my addr 172.21.15.18:0/133384367 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccb5bd6640 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10607b0 0x55f4b105dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccb53d5640 1 -- 172.21.15.18:0/133384367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10607b0 msgr2=0x55f4b105dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:25.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccb53d5640 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10607b0 0x55f4b105dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccb53d5640 1 -- 172.21.15.18:0/133384367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b1057bd0 msgr2=0x55f4b105d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:25.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.616+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b1057bd0 0x55f4b105d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fccb53d5640 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b1057bd0 0x55f4b105d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fccb53d5640 1 -- 172.21.15.18:0/133384367 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb01aa860 con 0x55f4b10572b0 2024-02-17T12:52:25.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fccb5bd6640 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10607b0 0x55f4b105dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:52:25.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b1057bd0 0x55f4b105d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:52:25.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fccb53d5640 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 0x55f4b1060200 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fccac009230 tx=0x7fccac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46919e04ea2d72d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:25.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fcc98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:25.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fcc98ff9640 1 -- 172.21.15.18:0/133384367 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac004df0 con 0x55f4b10572b0 2024-02-17T12:52:25.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fcc98ff9640 1 -- 172.21.15.18:0/133384367 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fccac047050 con 0x55f4b10572b0 2024-02-17T12:52:25.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fccba20a740 1 -- 172.21.15.18:0/133384367 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4b0cbe7e0 con 0x55f4b10572b0 2024-02-17T12:52:25.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fcc98ff9640 1 -- 172.21.15.18:0/133384367 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccac004f50 con 0x55f4b10572b0 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fccba20a740 10 client.16419 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fccba20a740 1 -- 172.21.15.18:0/133384367 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f4b0fbd610 con 0x55f4b10572b0 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fcc98ff9640 1 -- 172.21.15.18:0/133384367 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f4b0cbe7e0 con 0x55f4b10572b0 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.617+0000 7fcc98ff9640 1 -- 172.21.15.18:0/133384367 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fccac003bf0 con 0x55f4b10572b0 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.618+0000 7fccba20a740 10 client.16419 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.618+0000 7fccba20a740 10 client.16419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.618+0000 7fccba20a740 20 client.16419 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.618+0000 7fccba20a740 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f4b1070850 0x55f4b1072c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.618+0000 7fccba20a740 4 client.16419 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.618+0000 7fccba20a740 1 -- 172.21.15.18:0/133384367 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4b0cfb720 con 0x55f4b1070850 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.618+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f4b1070850 0x55f4b1072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.618+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f4b1070850 0x55f4b1072c90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3204befb631a9dce server_cookie=5be9778a09a990c4 in_seq=0 out_seq=0 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.618+0000 7fcc98ff9640 10 client.16419 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:25.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fcc98ff9640 1 -- 172.21.15.18:0/133384367 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4b0cfb720 con 0x55f4b1070850 2024-02-17T12:52:25.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fcc98ff9640 10 client.16419 handle_command_reply: tid=0 2024-02-17T12:52:25.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fccba20a740 1 client.16419 shutdown 2024-02-17T12:52:25.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fccba20a740 1 -- 172.21.15.18:0/133384367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 msgr2=0x55f4b1060200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fccba20a740 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 0x55f4b1060200 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fccac009230 tx=0x7fccac009260 comp rx=0 tx=0).stop 2024-02-17T12:52:25.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fccba20a740 1 -- 172.21.15.18:0/133384367 shutdown_connections 2024-02-17T12:52:25.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fccba20a740 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f4b1070850 0x55f4b1072c90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fccba20a740 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10607b0 0x55f4b105dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fccba20a740 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b1057bd0 0x55f4b105d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fcc98ff9640 0 client.16419 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:25.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fccba20a740 1 --2- 172.21.15.18:0/133384367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 0x55f4b1060200 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fccba20a740 1 -- 172.21.15.18:0/133384367 >> 172.21.15.18:0/133384367 conn(0x55f4b1051990 msgr2=0x55f4b1053730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:25.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fccba20a740 1 -- 172.21.15.18:0/133384367 shutdown_connections 2024-02-17T12:52:25.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fccba20a740 1 -- 172.21.15.18:0/133384367 wait complete. 2024-02-17T12:52:25.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.619+0000 7fccba20a740 20 client.16419 trim_cache size 0 max 16384 2024-02-17T12:52:25.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.634+0000 7fccba20a740 1 Processor -- start 2024-02-17T12:52:25.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccba20a740 1 -- start start 2024-02-17T12:52:25.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccba20a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10572b0 0x55f4b105d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccba20a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b1057bd0 0x55f4b105fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccba20a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b10607b0 0x55f4b1060200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccba20a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b0cfb720 con 0x55f4b1057bd0 2024-02-17T12:52:25.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccba20a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b0cbe7e0 con 0x55f4b10607b0 2024-02-17T12:52:25.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccba20a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b1056f50 con 0x55f4b10572b0 2024-02-17T12:52:25.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccb4bd4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b1057bd0 0x55f4b105fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccb4bd4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b1057bd0 0x55f4b105fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34156/0 (socket says 172.21.15.18:34156) 2024-02-17T12:52:25.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccb4bd4640 1 -- 172.21.15.18:0/3966853780 learned_addr learned my addr 172.21.15.18:0/3966853780 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:25.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccb4bd4640 1 -- 172.21.15.18:0/3966853780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10572b0 msgr2=0x55f4b105d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:25.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/3966853780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10572b0 0x55f4b105d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccb4bd4640 1 -- 172.21.15.18:0/3966853780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b10607b0 msgr2=0x55f4b1060200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/3966853780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b10607b0 0x55f4b1060200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccb4bd4640 1 -- 172.21.15.18:0/3966853780 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4b0f59ac0 con 0x55f4b1057bd0 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/3966853780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b1057bd0 0x55f4b105fcc0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcca0005d60 tx=0x7fcca000b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136f1c277986af9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fcc9affd640 1 -- 172.21.15.18:0/3966853780 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0004020 con 0x55f4b1057bd0 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fcc9affd640 1 -- 172.21.15.18:0/3966853780 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca0020070 con 0x55f4b1057bd0 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.635+0000 7fcc9affd640 1 -- 172.21.15.18:0/3966853780 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca00053f0 con 0x55f4b1057bd0 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.636+0000 7fccba20a740 1 -- 172.21.15.18:0/3966853780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b1057bd0 msgr2=0x55f4b105fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.636+0000 7fccba20a740 1 --2- 172.21.15.18:0/3966853780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b1057bd0 0x55f4b105fcc0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcca0005d60 tx=0x7fcca000b460 comp rx=0 tx=0).stop 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.636+0000 7fccba20a740 1 -- 172.21.15.18:0/3966853780 shutdown_connections 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.636+0000 7fccba20a740 1 --2- 172.21.15.18:0/3966853780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b10607b0 0x55f4b1060200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.636+0000 7fccba20a740 1 --2- 172.21.15.18:0/3966853780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b1057bd0 0x55f4b105fcc0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.636+0000 7fccba20a740 1 --2- 172.21.15.18:0/3966853780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10572b0 0x55f4b105d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.636+0000 7fccba20a740 1 -- 172.21.15.18:0/3966853780 >> 172.21.15.18:0/3966853780 conn(0x55f4b1051990 msgr2=0x55f4b105caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.636+0000 7fccba20a740 1 -- 172.21.15.18:0/3966853780 shutdown_connections 2024-02-17T12:52:25.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.636+0000 7fccba20a740 1 -- 172.21.15.18:0/3966853780 wait complete. 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.636+0000 7fccba20a740 1 -- start start 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.636+0000 7fccba20a740 1 Processor -- start 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccba20a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 0x55f4b10647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccba20a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b1057bd0 0x55f4b1064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccba20a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10607b0 0x55f4b10c0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccba20a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b10c12c0 con 0x55f4b10572b0 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccba20a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b10c1400 con 0x55f4b1057bd0 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccba20a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b10c15c0 con 0x55f4b10607b0 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccba20a740 10 client.0 _ll_register_callbacks cb 0x7fcca8096d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb53d5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 0x55f4b10647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb53d5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 0x55f4b10647d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34172/0 (socket says 172.21.15.18:34172) 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb53d5640 1 -- 172.21.15.18:0/533108957 learned_addr learned my addr 172.21.15.18:0/533108957 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb5bd6640 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10607b0 0x55f4b10c0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b1057bd0 0x55f4b1064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb53d5640 1 -- 172.21.15.18:0/533108957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10607b0 msgr2=0x55f4b10c0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb53d5640 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10607b0 0x55f4b10c0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb53d5640 1 -- 172.21.15.18:0/533108957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b1057bd0 msgr2=0x55f4b1064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb53d5640 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b1057bd0 0x55f4b1064d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb53d5640 1 -- 172.21.15.18:0/533108957 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca000bb50 con 0x55f4b10572b0 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb5bd6640 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10607b0 0x55f4b10c0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b1057bd0 0x55f4b1064d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fccb53d5640 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 0x55f4b10647d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fccac02c170 tx=0x7fccac003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198de34afe53708b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.637+0000 7fcc98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fcc98ff9640 1 -- 172.21.15.18:0/533108957 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac004ff0 con 0x55f4b10572b0 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fcc98ff9640 1 -- 172.21.15.18:0/533108957 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fccac047070 con 0x55f4b10572b0 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fccba20a740 1 -- 172.21.15.18:0/533108957 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcca809ab40 con 0x55f4b10572b0 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fcc98ff9640 1 -- 172.21.15.18:0/533108957 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccac003d20 con 0x55f4b10572b0 2024-02-17T12:52:25.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fccba20a740 10 client.16425 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:25.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fccba20a740 1 -- 172.21.15.18:0/533108957 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f4b0f7f6a0 con 0x55f4b10572b0 2024-02-17T12:52:25.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fcc98ff9640 1 -- 172.21.15.18:0/533108957 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccac06c020 con 0x55f4b10572b0 2024-02-17T12:52:25.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fcc98ff9640 1 -- 172.21.15.18:0/533108957 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fccac03fbc0 con 0x55f4b10572b0 2024-02-17T12:52:25.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fccba20a740 10 client.16425 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:25.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fccba20a740 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:25.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fccba20a740 20 client.16425 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:25.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fccba20a740 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f4b10c8c00 0x55f4b10cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fccba20a740 4 client.16425 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:52:25.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.638+0000 7fccba20a740 1 -- 172.21.15.18:0/533108957 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f4b0d4a890 con 0x55f4b10c8c00 2024-02-17T12:52:25.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f4b10c8c00 0x55f4b10cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fccb4bd4640 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f4b10c8c00 0x55f4b10cb020 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f60f5e38ff8d09 server_cookie=ffbf63345d1a6325 in_seq=0 out_seq=0 2024-02-17T12:52:25.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fcc98ff9640 10 client.16425 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fcc98ff9640 1 -- 172.21.15.18:0/533108957 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f4b0d4a890 con 0x55f4b10c8c00 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fcc98ff9640 10 client.16425 handle_command_reply: tid=0 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fccba20a740 1 client.16425 shutdown 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fccba20a740 1 -- 172.21.15.18:0/533108957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 msgr2=0x55f4b10647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fccba20a740 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 0x55f4b10647d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fccac02c170 tx=0x7fccac003920 comp rx=0 tx=0).stop 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fccba20a740 1 -- 172.21.15.18:0/533108957 shutdown_connections 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fccba20a740 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f4b10c8c00 0x55f4b10cb020 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fccba20a740 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f4b10607b0 0x55f4b10c0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fccba20a740 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f4b1057bd0 0x55f4b1064d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fccba20a740 1 --2- 172.21.15.18:0/533108957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f4b10572b0 0x55f4b10647d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fccba20a740 1 -- 172.21.15.18:0/533108957 >> 172.21.15.18:0/533108957 conn(0x55f4b1051990 msgr2=0x55f4b105d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.639+0000 7fcc98ff9640 0 client.16425 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccba20a740 1 -- 172.21.15.18:0/533108957 shutdown_connections 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccba20a740 1 -- 172.21.15.18:0/533108957 wait complete. 2024-02-17T12:52:25.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccba20a740 20 client.16425 trim_cache size 0 max 16384 2024-02-17T12:52:25.638 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:52:25.638 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:52:25.638 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:52:25.638 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:52:25.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccb7639640 1 -- 172.21.15.18:0/1897901808 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc9c0a22b0 msgr2=0x7fcc9c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccb7639640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc9c0a22b0 0x7fcc9c0a4770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fccac005ec0 tx=0x7fccac0058d0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccb7639640 1 -- 172.21.15.18:0/1897901808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb01060c0 msgr2=0x7fccb019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccb7639640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb01060c0 0x7fccb019d970 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcca800cce0 tx=0x7fcca800a620 comp rx=0 tx=0).stop 2024-02-17T12:52:25.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccb7639640 1 -- 172.21.15.18:0/1897901808 shutdown_connections 2024-02-17T12:52:25.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccb7639640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc9c0a22b0 0x7fcc9c0a4770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccb7639640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb01060c0 0x7fccb019d970 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccb7639640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0103bc0 0x7fccb01a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccb7639640 1 --2- 172.21.15.18:0/1897901808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb0069cf0 0x7fccb01a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccb7639640 1 -- 172.21.15.18:0/1897901808 >> 172.21.15.18:0/1897901808 conn(0x7fccb0076e20 msgr2=0x7fccb0102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:25.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccb7639640 1 -- 172.21.15.18:0/1897901808 shutdown_connections 2024-02-17T12:52:25.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.640+0000 7fccb7639640 1 -- 172.21.15.18:0/1897901808 wait complete. 2024-02-17T12:52:25.642 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:52:25.643 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:52:25.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.754+0000 7f53a8c8d640 1 -- 172.21.15.18:0/1184448741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53a4102d10 msgr2=0x7f53a410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.754+0000 7f53a8c8d640 1 --2- 172.21.15.18:0/1184448741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53a4102d10 0x7f53a410b1f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f538c009900 tx=0x7f538c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:25.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.754+0000 7f53a8c8d640 1 -- 172.21.15.18:0/1184448741 shutdown_connections 2024-02-17T12:52:25.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.754+0000 7f53a8c8d640 1 --2- 172.21.15.18:0/1184448741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53a410b7c0 0x7f53a410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.754+0000 7f53a8c8d640 1 --2- 172.21.15.18:0/1184448741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53a4102d10 0x7f53a410b1f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.754+0000 7f53a8c8d640 1 --2- 172.21.15.18:0/1184448741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a41023f0 0x7f53a41027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.755+0000 7f53a8c8d640 1 -- 172.21.15.18:0/1184448741 >> 172.21.15.18:0/1184448741 conn(0x7f53a4077180 msgr2=0x7f53a4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:25.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.755+0000 7f53a8c8d640 1 -- 172.21.15.18:0/1184448741 shutdown_connections 2024-02-17T12:52:25.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.755+0000 7f53a8c8d640 1 -- 172.21.15.18:0/1184448741 wait complete. 2024-02-17T12:52:25.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.755+0000 7f53a8c8d640 1 Processor -- start 2024-02-17T12:52:25.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.755+0000 7f53a8c8d640 1 -- start start 2024-02-17T12:52:25.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.755+0000 7f53a8c8d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53a41023f0 0x7f53a419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.755+0000 7f53a8c8d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a4102d10 0x7f53a419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.755+0000 7f53a8c8d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53a410b7c0 0x7f53a4197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.755+0000 7f53a8c8d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4197df0 con 0x7f53a410b7c0 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.755+0000 7f53a8c8d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4197f60 con 0x7f53a4102d10 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.755+0000 7f53a8c8d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4198120 con 0x7f53a41023f0 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f53a2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53a410b7c0 0x7f53a4197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f53a2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53a410b7c0 0x7f53a4197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34194/0 (socket says 172.21.15.18:34194) 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f53a2d9d640 1 -- 172.21.15.18:0/4269910354 learned_addr learned my addr 172.21.15.18:0/4269910354 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f539bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a4102d10 0x7f53a419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f53a259c640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53a41023f0 0x7f53a419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f53a2d9d640 1 -- 172.21.15.18:0/4269910354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53a41023f0 msgr2=0x7f53a419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f53a2d9d640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53a41023f0 0x7f53a419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f53a2d9d640 1 -- 172.21.15.18:0/4269910354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a4102d10 msgr2=0x7f53a419dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f53a2d9d640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a4102d10 0x7f53a419dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f53a2d9d640 1 -- 172.21.15.18:0/4269910354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c009560 con 0x7f53a410b7c0 2024-02-17T12:52:25.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f53a2d9d640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53a410b7c0 0x7f53a4197660 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f539c00cad0 tx=0x7f539c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:25.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f539b7fe640 1 -- 172.21.15.18:0/4269910354 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c00ae20 con 0x7f53a410b7c0 2024-02-17T12:52:25.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f539b7fe640 1 -- 172.21.15.18:0/4269910354 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f539c005020 con 0x7f53a410b7c0 2024-02-17T12:52:25.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f53a8c8d640 1 -- 172.21.15.18:0/4269910354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a4198400 con 0x7f53a410b7c0 2024-02-17T12:52:25.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f539b7fe640 1 -- 172.21.15.18:0/4269910354 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c00f6b0 con 0x7f53a410b7c0 2024-02-17T12:52:25.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.756+0000 7f53a8c8d640 1 -- 172.21.15.18:0/4269910354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a41a4520 con 0x7f53a410b7c0 2024-02-17T12:52:25.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.757+0000 7f53a8c8d640 1 -- 172.21.15.18:0/4269910354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a4068b20 con 0x7f53a410b7c0 2024-02-17T12:52:25.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.758+0000 7f539b7fe640 1 -- 172.21.15.18:0/4269910354 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f539c005520 con 0x7f53a410b7c0 2024-02-17T12:52:25.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.759+0000 7f539b7fe640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f53900a2230 0x7f53900a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:25.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.759+0000 7f53a259c640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f53900a2230 0x7f53900a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:25.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.759+0000 7f539b7fe640 1 -- 172.21.15.18:0/4269910354 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f539c0ce320 con 0x7f53a410b7c0 2024-02-17T12:52:25.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.759+0000 7f53a259c640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f53900a2230 0x7f53900a46f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5394009bd0 tx=0x7f5394005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:25.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.763+0000 7f539b7fe640 1 -- 172.21.15.18:0/4269910354 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f539c0970b0 con 0x7f53a410b7c0 2024-02-17T12:52:25.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.988+0000 7f53a8c8d640 1 -- 172.21.15.18:0/4269910354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53a4106910 con 0x7f53a410b7c0 2024-02-17T12:52:25.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.989+0000 7f539b7fe640 1 -- 172.21.15.18:0/4269910354 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11399 (secure 0 0 0) 0x7f539c01d090 con 0x7f53a410b7c0 2024-02-17T12:52:25.980 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:52:25.981 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:51:04.933427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:52:25.981 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 28 2024-02-17T12:52:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.991+0000 7f53a8c8d640 1 -- 172.21.15.18:0/4269910354 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f53900a2230 msgr2=0x7f53900a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.991+0000 7f53a8c8d640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f53900a2230 0x7f53900a46f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5394009bd0 tx=0x7f5394005f90 comp rx=0 tx=0).stop 2024-02-17T12:52:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.991+0000 7f53a8c8d640 1 -- 172.21.15.18:0/4269910354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53a410b7c0 msgr2=0x7f53a4197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.991+0000 7f53a8c8d640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53a410b7c0 0x7f53a4197660 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f539c00cad0 tx=0x7f539c00cf90 comp rx=0 tx=0).stop 2024-02-17T12:52:25.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.991+0000 7f53a8c8d640 1 -- 172.21.15.18:0/4269910354 shutdown_connections 2024-02-17T12:52:25.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.991+0000 7f53a8c8d640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f53900a2230 0x7f53900a46f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.991+0000 7f53a8c8d640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53a410b7c0 0x7f53a4197660 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.991+0000 7f53a8c8d640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53a4102d10 0x7f53a419dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.991+0000 7f53a8c8d640 1 --2- 172.21.15.18:0/4269910354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53a41023f0 0x7f53a419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:25.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.991+0000 7f53a8c8d640 1 -- 172.21.15.18:0/4269910354 >> 172.21.15.18:0/4269910354 conn(0x7f53a4077180 msgr2=0x7f53a40fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:25.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.991+0000 7f53a8c8d640 1 -- 172.21.15.18:0/4269910354 shutdown_connections 2024-02-17T12:52:25.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:25.991+0000 7f53a8c8d640 1 -- 172.21.15.18:0/4269910354 wait complete. 2024-02-17T12:52:25.993 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:52:25.993 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:52:26.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.110+0000 7f448dde2640 1 -- 172.21.15.18:0/2374773256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4488101360 msgr2=0x7f4488101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.110+0000 7f448dde2640 1 --2- 172.21.15.18:0/2374773256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4488101360 0x7f4488101740 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4478009930 tx=0x7f447802bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:26.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.110+0000 7f448dde2640 1 -- 172.21.15.18:0/2374773256 shutdown_connections 2024-02-17T12:52:26.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.110+0000 7f448dde2640 1 --2- 172.21.15.18:0/2374773256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4488106430 0x7f448810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.110+0000 7f448dde2640 1 --2- 172.21.15.18:0/2374773256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4488101d10 0x7f4488105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.110+0000 7f448dde2640 1 --2- 172.21.15.18:0/2374773256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4488101360 0x7f4488101740 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.110+0000 7f448dde2640 1 -- 172.21.15.18:0/2374773256 >> 172.21.15.18:0/2374773256 conn(0x7f44880fcbe0 msgr2=0x7f44880ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.110+0000 7f448dde2640 1 -- 172.21.15.18:0/2374773256 shutdown_connections 2024-02-17T12:52:26.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.110+0000 7f448dde2640 1 -- 172.21.15.18:0/2374773256 wait complete. 2024-02-17T12:52:26.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f448dde2640 1 Processor -- start 2024-02-17T12:52:26.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f448dde2640 1 -- start start 2024-02-17T12:52:26.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f448dde2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4488101360 0x7f448819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f448dde2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4488106430 0x7f448819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f448dde2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44881976c0 0x7f4488197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f448dde2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4488198190 con 0x7f4488101360 2024-02-17T12:52:26.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f448dde2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4488198300 con 0x7f44881976c0 2024-02-17T12:52:26.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f448dde2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44881984c0 con 0x7f4488106430 2024-02-17T12:52:26.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f44877fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4488101360 0x7f448819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f44877fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4488101360 0x7f448819d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34218/0 (socket says 172.21.15.18:34218) 2024-02-17T12:52:26.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f44877fe640 1 -- 172.21.15.18:0/1192167647 learned_addr learned my addr 172.21.15.18:0/1192167647 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f4487fff640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44881976c0 0x7f4488197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f4486ffd640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4488106430 0x7f448819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f44877fe640 1 -- 172.21.15.18:0/1192167647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4488106430 msgr2=0x7f448819daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f44877fe640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4488106430 0x7f448819daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f44877fe640 1 -- 172.21.15.18:0/1192167647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44881976c0 msgr2=0x7f4488197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f44877fe640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44881976c0 0x7f4488197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.111+0000 7f44877fe640 1 -- 172.21.15.18:0/1192167647 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4478009560 con 0x7f4488101360 2024-02-17T12:52:26.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.112+0000 7f44877fe640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4488101360 0x7f448819d5b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4478000c00 tx=0x7f447802cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.112+0000 7f4484ff9640 1 -- 172.21.15.18:0/1192167647 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478004350 con 0x7f4488101360 2024-02-17T12:52:26.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.112+0000 7f4484ff9640 1 -- 172.21.15.18:0/1192167647 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44780044b0 con 0x7f4488101360 2024-02-17T12:52:26.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.112+0000 7f4484ff9640 1 -- 172.21.15.18:0/1192167647 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478003d00 con 0x7f4488101360 2024-02-17T12:52:26.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.112+0000 7f448dde2640 1 -- 172.21.15.18:0/1192167647 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44881a43b0 con 0x7f4488101360 2024-02-17T12:52:26.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.112+0000 7f448dde2640 1 -- 172.21.15.18:0/1192167647 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44881a4890 con 0x7f4488101360 2024-02-17T12:52:26.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.114+0000 7f4484ff9640 1 -- 172.21.15.18:0/1192167647 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4478004df0 con 0x7f4488101360 2024-02-17T12:52:26.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.114+0000 7f44909b5740 1 Processor -- start 2024-02-17T12:52:26.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.114+0000 7f44909b5740 1 -- start start 2024-02-17T12:52:26.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f4484ff9640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44700a2230 0x7f44700a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f4487fff640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44700a2230 0x7f44700a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44909b5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee0250 0x563c33ee0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44909b5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 0x563c33ee9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f4484ff9640 1 -- 172.21.15.18:0/1192167647 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f44780f06b0 con 0x7f4488101360 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44909b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee9750 0x563c33eebb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44909b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33e9b960 con 0x563c33ee9750 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44909b5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33e4f310 con 0x563c33ee0250 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44909b5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33e4c840 con 0x563c33ee0b70 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f4487fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee9750 0x563c33eebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44877fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee0250 0x563c33ee0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44877fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee0250 0x563c33ee0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:56984/0 (socket says 172.21.15.18:56984) 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44877fe640 1 -- 172.21.15.18:0/4054616770 learned_addr learned my addr 172.21.15.18:0/4054616770 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f4487fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee9750 0x563c33eebb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34224/0 (socket says 172.21.15.18:34224) 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f4486ffd640 1 --2- 172.21.15.18:0/4054616770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 0x563c33ee9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44877fe640 1 -- 172.21.15.18:0/4054616770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 msgr2=0x563c33ee9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44877fe640 1 --2- 172.21.15.18:0/4054616770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 0x563c33ee9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44877fe640 1 -- 172.21.15.18:0/4054616770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee9750 msgr2=0x563c33eebb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44877fe640 1 --2- 172.21.15.18:0/4054616770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee9750 0x563c33eebb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44877fe640 1 -- 172.21.15.18:0/4054616770 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c33e0a060 con 0x563c33ee0250 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f4487fff640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44700a2230 0x7f44700a46f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4480005fd0 tx=0x7f4480012ad0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.115+0000 7f44877fe640 1 --2- 172.21.15.18:0/4054616770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee0250 0x563c33ee0630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f44780bd340 tx=0x7f4478009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3019d50b1f7cead1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.116+0000 7f446dffb640 1 -- 172.21.15.18:0/4054616770 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44780bdab0 con 0x563c33ee0250 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.116+0000 7f446dffb640 1 -- 172.21.15.18:0/4054616770 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44780bd3b0 con 0x563c33ee0250 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.116+0000 7f446dffb640 1 -- 172.21.15.18:0/4054616770 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44780b9a20 con 0x563c33ee0250 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.116+0000 7f44909b5740 1 -- 172.21.15.18:0/4054616770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee0250 msgr2=0x563c33ee0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.116+0000 7f44909b5740 1 --2- 172.21.15.18:0/4054616770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee0250 0x563c33ee0630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f44780bd340 tx=0x7f4478009260 comp rx=0 tx=0).stop 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.116+0000 7f44909b5740 1 -- 172.21.15.18:0/4054616770 shutdown_connections 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.116+0000 7f44909b5740 1 --2- 172.21.15.18:0/4054616770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee9750 0x563c33eebb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.116+0000 7f44909b5740 1 --2- 172.21.15.18:0/4054616770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 0x563c33ee9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.116+0000 7f44909b5740 1 --2- 172.21.15.18:0/4054616770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee0250 0x563c33ee0630 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.116+0000 7f44909b5740 1 -- 172.21.15.18:0/4054616770 >> 172.21.15.18:0/4054616770 conn(0x563c33eda980 msgr2=0x563c33edad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.116+0000 7f44909b5740 1 -- 172.21.15.18:0/4054616770 shutdown_connections 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.116+0000 7f44909b5740 1 -- 172.21.15.18:0/4054616770 wait complete. 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f44909b5740 1 -- start start 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f44909b5740 1 Processor -- start 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f44909b5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0250 0x563c33ee66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f44909b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0b70 0x563c33ee6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f44909b5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33ee7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f44909b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33e48020 con 0x563c33ee0b70 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f44909b5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33b1b430 con 0x563c33ee9750 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f44909b5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33ad7e00 con 0x563c33ee0250 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f44909b5740 10 client.0 _ll_register_callbacks cb 0x563c33defcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f4486ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0b70 0x563c33ee6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f4486ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0b70 0x563c33ee6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34228/0 (socket says 172.21.15.18:34228) 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f4486ffd640 1 -- 172.21.15.18:0/1302274830 learned_addr learned my addr 172.21.15.18:0/1302274830 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f4487fff640 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33ee7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.117+0000 7f44877fe640 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0250 0x563c33ee66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f4486ffd640 1 -- 172.21.15.18:0/1302274830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0250 msgr2=0x563c33ee66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f4486ffd640 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0250 0x563c33ee66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f4486ffd640 1 -- 172.21.15.18:0/1302274830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 msgr2=0x563c33ee7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f4486ffd640 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33ee7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f4486ffd640 1 -- 172.21.15.18:0/1302274830 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44881a4890 con 0x563c33ee0b70 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f4486ffd640 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0b70 0x563c33ee6c00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f447c0134d0 tx=0x7f447c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4970152495391bfa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f4466ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f4466ffd640 1 -- 172.21.15.18:0/1302274830 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c009c10 con 0x563c33ee0b70 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f4466ffd640 1 -- 172.21.15.18:0/1302274830 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f447c009d70 con 0x563c33ee0b70 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f44909b5740 1 -- 172.21.15.18:0/1302274830 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c33b477e0 con 0x563c33ee0b70 2024-02-17T12:52:26.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f4466ffd640 1 -- 172.21.15.18:0/1302274830 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f447c011e60 con 0x563c33ee0b70 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f44909b5740 10 client.16443 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f44909b5740 1 -- 172.21.15.18:0/1302274830 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563c33e466a0 con 0x563c33ee0b70 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f4466ffd640 1 -- 172.21.15.18:0/1302274830 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563c33b477e0 con 0x563c33ee0b70 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f4466ffd640 1 -- 172.21.15.18:0/1302274830 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f447c003c50 con 0x563c33ee0b70 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f44909b5740 10 client.16443 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f44909b5740 10 client.16443 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f44909b5740 20 client.16443 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.118+0000 7f44909b5740 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x563c33ef9990 0x563c33efbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.119+0000 7f44909b5740 4 client.16443 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.119+0000 7f44909b5740 1 -- 172.21.15.18:0/1302274830 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c33b84720 con 0x563c33ef9990 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.119+0000 7f44877fe640 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x563c33ef9990 0x563c33efbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.119+0000 7f44877fe640 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x563c33ef9990 0x563c33efbdb0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9e823afff5d87c22 server_cookie=2e1a734ddca6f123 in_seq=0 out_seq=0 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.119+0000 7f4466ffd640 10 client.16443 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f4466ffd640 1 -- 172.21.15.18:0/1302274830 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c33b84720 con 0x563c33ef9990 2024-02-17T12:52:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f4466ffd640 10 client.16443 handle_command_reply: tid=0 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f44909b5740 1 client.16443 shutdown 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f44909b5740 1 -- 172.21.15.18:0/1302274830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0b70 msgr2=0x563c33ee6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f44909b5740 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0b70 0x563c33ee6c00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f447c0134d0 tx=0x7f447c011b20 comp rx=0 tx=0).stop 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f44909b5740 1 -- 172.21.15.18:0/1302274830 shutdown_connections 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f44909b5740 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x563c33ef9990 0x563c33efbdb0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f44909b5740 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33ee7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f44909b5740 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0b70 0x563c33ee6c00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f4466ffd640 0 client.16443 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f44909b5740 1 --2- 172.21.15.18:0/1302274830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0250 0x563c33ee66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f44909b5740 1 -- 172.21.15.18:0/1302274830 >> 172.21.15.18:0/1302274830 conn(0x563c33eda980 msgr2=0x563c33edc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f44909b5740 1 -- 172.21.15.18:0/1302274830 shutdown_connections 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f44909b5740 1 -- 172.21.15.18:0/1302274830 wait complete. 2024-02-17T12:52:26.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.120+0000 7f44909b5740 20 client.16443 trim_cache size 0 max 16384 2024-02-17T12:52:26.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.132+0000 7f44909b5740 1 Processor -- start 2024-02-17T12:52:26.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.132+0000 7f44909b5740 1 -- start start 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44909b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 0x563c33ee8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44909b5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 0x563c33ee6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44909b5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33ee3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44909b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33dcd130 con 0x563c33ee0250 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44909b5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33edfc90 con 0x563c33ee9750 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44909b5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33eebb40 con 0x563c33ee0b70 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44877fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 0x563c33ee8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44877fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 0x563c33ee8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34238/0 (socket says 172.21.15.18:34238) 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44877fe640 1 -- 172.21.15.18:0/374848194 learned_addr learned my addr 172.21.15.18:0/374848194 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f4487fff640 1 --2- 172.21.15.18:0/374848194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33ee3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f4486ffd640 1 --2- 172.21.15.18:0/374848194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 0x563c33ee6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44877fe640 1 -- 172.21.15.18:0/374848194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 msgr2=0x563c33ee6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44877fe640 1 --2- 172.21.15.18:0/374848194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 0x563c33ee6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44877fe640 1 -- 172.21.15.18:0/374848194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 msgr2=0x563c33ee3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44877fe640 1 --2- 172.21.15.18:0/374848194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33ee3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44877fe640 1 -- 172.21.15.18:0/374848194 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c33de2c20 con 0x563c33ee0250 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f44877fe640 1 --2- 172.21.15.18:0/374848194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 0x563c33ee8df0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f44780ee450 tx=0x7f44780b9780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c90fa8712432ba server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f446d7fa640 1 -- 172.21.15.18:0/374848194 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478082ea0 con 0x563c33ee0250 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f446d7fa640 1 -- 172.21.15.18:0/374848194 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44780c5ac0 con 0x563c33ee0250 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.133+0000 7f446d7fa640 1 -- 172.21.15.18:0/374848194 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44780b9a20 con 0x563c33ee0250 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.134+0000 7f44909b5740 1 -- 172.21.15.18:0/374848194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 msgr2=0x563c33ee8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.134+0000 7f44909b5740 1 --2- 172.21.15.18:0/374848194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 0x563c33ee8df0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f44780ee450 tx=0x7f44780b9780 comp rx=0 tx=0).stop 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.134+0000 7f44909b5740 1 -- 172.21.15.18:0/374848194 shutdown_connections 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.134+0000 7f44909b5740 1 --2- 172.21.15.18:0/374848194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33ee3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.134+0000 7f44909b5740 1 --2- 172.21.15.18:0/374848194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 0x563c33ee6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.134+0000 7f44909b5740 1 --2- 172.21.15.18:0/374848194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 0x563c33ee8df0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.134+0000 7f44909b5740 1 -- 172.21.15.18:0/374848194 >> 172.21.15.18:0/374848194 conn(0x563c33eda980 msgr2=0x563c33ee19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.134+0000 7f44909b5740 1 -- 172.21.15.18:0/374848194 shutdown_connections 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.134+0000 7f44909b5740 1 -- 172.21.15.18:0/374848194 wait complete. 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.134+0000 7f44909b5740 1 -- start start 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44909b5740 1 Processor -- start 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44909b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 0x563c33ef0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44909b5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 0x563c33eeea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44909b5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33eeef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44909b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33eef570 con 0x563c33ee0250 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44909b5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33eef6e0 con 0x563c33ee9750 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44909b5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33eef8a0 con 0x563c33ee0b70 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44909b5740 10 client.0 _ll_register_callbacks cb 0x7f447c0139e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44877fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 0x563c33ef0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44877fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 0x563c33ef0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34244/0 (socket says 172.21.15.18:34244) 2024-02-17T12:52:26.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f4487fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33eeef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44877fe640 1 -- 172.21.15.18:0/2454907838 learned_addr learned my addr 172.21.15.18:0/2454907838 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f4486ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 0x563c33eeea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44877fe640 1 -- 172.21.15.18:0/2454907838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 msgr2=0x563c33eeea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44877fe640 1 --2- 172.21.15.18:0/2454907838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 0x563c33eeea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44877fe640 1 -- 172.21.15.18:0/2454907838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 msgr2=0x563c33eeef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44877fe640 1 --2- 172.21.15.18:0/2454907838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33eeef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44877fe640 1 -- 172.21.15.18:0/2454907838 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c33e0a060 con 0x563c33ee0250 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f4487fff640 1 --2- 172.21.15.18:0/2454907838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33eeef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f44877fe640 1 --2- 172.21.15.18:0/2454907838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 0x563c33ef0650 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4478006010 tx=0x7f4478009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304c2fb1726e52a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.135+0000 7f4466ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f4466ffd640 1 -- 172.21.15.18:0/2454907838 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478082ea0 con 0x563c33ee0250 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f4466ffd640 1 -- 172.21.15.18:0/2454907838 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f44780b9a20 con 0x563c33ee0250 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f44909b5740 1 -- 172.21.15.18:0/2454907838 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44780bd3b0 con 0x563c33ee0250 2024-02-17T12:52:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f4466ffd640 1 -- 172.21.15.18:0/2454907838 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44780b9be0 con 0x563c33ee0250 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f44909b5740 10 client.16449 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f44909b5740 1 -- 172.21.15.18:0/2454907838 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563c33e088c0 con 0x563c33ee0250 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f4466ffd640 1 -- 172.21.15.18:0/2454907838 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44780bd3b0 con 0x563c33ee0250 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f4466ffd640 1 -- 172.21.15.18:0/2454907838 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f447803c070 con 0x563c33ee0250 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f44909b5740 10 client.16449 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f44909b5740 10 client.16449 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f44909b5740 20 client.16449 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f44909b5740 1 --2- 172.21.15.18:0/2454907838 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x563c33f51110 0x563c33f53500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f44909b5740 4 client.16449 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f44909b5740 1 -- 172.21.15.18:0/2454907838 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c33bd3890 con 0x563c33f51110 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.136+0000 7f4486ffd640 1 --2- 172.21.15.18:0/2454907838 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x563c33f51110 0x563c33f53500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f4466ffd640 10 client.16449 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f4486ffd640 1 --2- 172.21.15.18:0/2454907838 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x563c33f51110 0x563c33f53500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=568ec27892c93fb7 server_cookie=788afc95776f479b in_seq=0 out_seq=0 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f4466ffd640 1 -- 172.21.15.18:0/2454907838 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563c33bd3890 con 0x563c33f51110 2024-02-17T12:52:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f4466ffd640 10 client.16449 handle_command_reply: tid=0 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f44909b5740 1 client.16449 shutdown 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f44909b5740 1 -- 172.21.15.18:0/2454907838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 msgr2=0x563c33ef0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f44909b5740 1 --2- 172.21.15.18:0/2454907838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 0x563c33ef0650 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4478006010 tx=0x7f4478009260 comp rx=0 tx=0).stop 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f44909b5740 1 -- 172.21.15.18:0/2454907838 shutdown_connections 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f44909b5740 1 --2- 172.21.15.18:0/2454907838 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x563c33f51110 0x563c33f53500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f44909b5740 1 --2- 172.21.15.18:0/2454907838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c33ee9750 0x563c33eeef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f44909b5740 1 --2- 172.21.15.18:0/2454907838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c33ee0b70 0x563c33eeea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f4466ffd640 0 client.16449 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f44909b5740 1 --2- 172.21.15.18:0/2454907838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c33ee0250 0x563c33ef0650 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f44909b5740 1 -- 172.21.15.18:0/2454907838 >> 172.21.15.18:0/2454907838 conn(0x563c33eda980 msgr2=0x563c33edbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f44909b5740 1 -- 172.21.15.18:0/2454907838 shutdown_connections 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f44909b5740 1 -- 172.21.15.18:0/2454907838 wait complete. 2024-02-17T12:52:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.137+0000 7f44909b5740 20 client.16449 trim_cache size 0 max 16384 2024-02-17T12:52:26.136 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.138+0000 7f448dde2640 1 -- 172.21.15.18:0/1192167647 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44700a2230 msgr2=0x7f44700a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.138+0000 7f448dde2640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44700a2230 0x7f44700a46f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4480005fd0 tx=0x7f4480012ad0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.138+0000 7f448dde2640 1 -- 172.21.15.18:0/1192167647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4488101360 msgr2=0x7f448819d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.138+0000 7f448dde2640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4488101360 0x7f448819d5b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4478000c00 tx=0x7f447802cd80 comp rx=0 tx=0).stop 2024-02-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.138+0000 7f448dde2640 1 -- 172.21.15.18:0/1192167647 shutdown_connections 2024-02-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.138+0000 7f448dde2640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44700a2230 0x7f44700a46f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.138+0000 7f448dde2640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44881976c0 0x7f4488197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.138+0000 7f448dde2640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4488106430 0x7f448819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.138+0000 7f448dde2640 1 --2- 172.21.15.18:0/1192167647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4488101360 0x7f448819d5b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.138+0000 7f448dde2640 1 -- 172.21.15.18:0/1192167647 >> 172.21.15.18:0/1192167647 conn(0x7f44880fcbe0 msgr2=0x7f44880fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.138+0000 7f448dde2640 1 -- 172.21.15.18:0/1192167647 shutdown_connections 2024-02-17T12:52:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.138+0000 7f448dde2640 1 -- 172.21.15.18:0/1192167647 wait complete. 2024-02-17T12:52:26.140 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:52:26.140 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:52:26.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.258+0000 7f8303a0e640 1 -- 172.21.15.18:0/1076998964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82fc0685f0 msgr2=0x7f82fc0689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.258+0000 7f8303a0e640 1 --2- 172.21.15.18:0/1076998964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82fc0685f0 0x7f82fc0689d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009930 tx=0x7f82f402bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:26.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.258+0000 7f8303a0e640 1 -- 172.21.15.18:0/1076998964 shutdown_connections 2024-02-17T12:52:26.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.258+0000 7f8303a0e640 1 --2- 172.21.15.18:0/1076998964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82fc1064c0 0x7f82fc10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.258+0000 7f8303a0e640 1 --2- 172.21.15.18:0/1076998964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc068fa0 0x7f82fc105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.258+0000 7f8303a0e640 1 --2- 172.21.15.18:0/1076998964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82fc0685f0 0x7f82fc0689d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.258+0000 7f8303a0e640 1 -- 172.21.15.18:0/1076998964 >> 172.21.15.18:0/1076998964 conn(0x7f82fc077180 msgr2=0x7f82fc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.258+0000 7f8303a0e640 1 -- 172.21.15.18:0/1076998964 shutdown_connections 2024-02-17T12:52:26.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.258+0000 7f8303a0e640 1 -- 172.21.15.18:0/1076998964 wait complete. 2024-02-17T12:52:26.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.258+0000 7f8303a0e640 1 Processor -- start 2024-02-17T12:52:26.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.258+0000 7f8303a0e640 1 -- start start 2024-02-17T12:52:26.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8303a0e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82fc0685f0 0x7f82fc19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8303a0e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82fc1064c0 0x7f82fc19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8303a0e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc197610 0x7f82fc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8303a0e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc198000 con 0x7f82fc1064c0 2024-02-17T12:52:26.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8303a0e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc198170 con 0x7f82fc197610 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8303a0e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc198330 con 0x7f82fc0685f0 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8300fa9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82fc1064c0 0x7f82fc19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8300fa9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82fc1064c0 0x7f82fc19d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34262/0 (socket says 172.21.15.18:34262) 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8300fa9640 1 -- 172.21.15.18:0/4144903349 learned_addr learned my addr 172.21.15.18:0/4144903349 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f83017aa640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82fc0685f0 0x7f82fc19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8301fab640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc197610 0x7f82fc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8300fa9640 1 -- 172.21.15.18:0/4144903349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82fc0685f0 msgr2=0x7f82fc19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8300fa9640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82fc0685f0 0x7f82fc19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8300fa9640 1 -- 172.21.15.18:0/4144903349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc197610 msgr2=0x7f82fc197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8300fa9640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc197610 0x7f82fc197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8300fa9640 1 -- 172.21.15.18:0/4144903349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f0009490 con 0x7f82fc1064c0 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8300fa9640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82fc1064c0 0x7f82fc19d9a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f82f000e970 tx=0x7f82f000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f82fa7fc640 1 -- 172.21.15.18:0/4144903349 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f000cd80 con 0x7f82fc1064c0 2024-02-17T12:52:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f82fa7fc640 1 -- 172.21.15.18:0/4144903349 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82f00035f0 con 0x7f82fc1064c0 2024-02-17T12:52:26.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.259+0000 7f8303a0e640 1 -- 172.21.15.18:0/4144903349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f4009560 con 0x7f82fc1064c0 2024-02-17T12:52:26.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.260+0000 7f82fa7fc640 1 -- 172.21.15.18:0/4144903349 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0010690 con 0x7f82fc1064c0 2024-02-17T12:52:26.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.260+0000 7f8303a0e640 1 -- 172.21.15.18:0/4144903349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc1a46a0 con 0x7f82fc1064c0 2024-02-17T12:52:26.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.262+0000 7f82fa7fc640 1 -- 172.21.15.18:0/4144903349 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f82f0003760 con 0x7f82fc1064c0 2024-02-17T12:52:26.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.262+0000 7f83065dd740 1 Processor -- start 2024-02-17T12:52:26.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.262+0000 7f83065dd740 1 -- start start 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83065dd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d32250 0x556869d32630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83065dd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32b70 0x556869d3b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83065dd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d3b750 0x556869d3db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83065dd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556869ced960 con 0x556869d32b70 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83065dd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556869ca1310 con 0x556869d3b750 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83065dd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556869c9e840 con 0x556869d32250 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f82fa7fc640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82e40a2230 0x7f82e40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83017aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32b70 0x556869d3b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83017aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32b70 0x556869d3b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34272/0 (socket says 172.21.15.18:34272) 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83017aa640 1 -- 172.21.15.18:0/2682687454 learned_addr learned my addr 172.21.15.18:0/2682687454 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f8301fab640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82e40a2230 0x7f82e40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f8300fa9640 1 --2- 172.21.15.18:0/2682687454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d3b750 0x556869d3db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f8301fab640 1 --2- 172.21.15.18:0/2682687454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d32250 0x556869d32630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f82fa7fc640 1 -- 172.21.15.18:0/4144903349 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f82f0014070 con 0x7f82fc1064c0 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83017aa640 1 -- 172.21.15.18:0/2682687454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d32250 msgr2=0x556869d32630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83017aa640 1 --2- 172.21.15.18:0/2682687454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d32250 0x556869d32630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83017aa640 1 -- 172.21.15.18:0/2682687454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d3b750 msgr2=0x556869d3db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83017aa640 1 --2- 172.21.15.18:0/2682687454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d3b750 0x556869d3db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83017aa640 1 -- 172.21.15.18:0/2682687454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556869c5c060 con 0x556869d32b70 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f8301fab640 1 --2- 172.21.15.18:0/2682687454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d32250 0x556869d32630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f8300fa9640 1 --2- 172.21.15.18:0/2682687454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d3b750 0x556869d3db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f8301fab640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82e40a2230 0x7f82e40a46f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f82ec005fd0 tx=0x7f82ec009be0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f83017aa640 1 --2- 172.21.15.18:0/2682687454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32b70 0x556869d3b210 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f82f4036040 tx=0x7f82f402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b317b0d8e45aee9f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f82e37fe640 1 -- 172.21.15.18:0/2682687454 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4004800 con 0x556869d32b70 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f82e37fe640 1 -- 172.21.15.18:0/2682687454 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82f4004960 con 0x556869d32b70 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.263+0000 7f82e37fe640 1 -- 172.21.15.18:0/2682687454 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40050e0 con 0x556869d32b70 2024-02-17T12:52:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.264+0000 7f83065dd740 1 -- 172.21.15.18:0/2682687454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32b70 msgr2=0x556869d3b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.264+0000 7f83065dd740 1 --2- 172.21.15.18:0/2682687454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32b70 0x556869d3b210 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f82f4036040 tx=0x7f82f402c5d0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.264+0000 7f83065dd740 1 -- 172.21.15.18:0/2682687454 shutdown_connections 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.264+0000 7f83065dd740 1 --2- 172.21.15.18:0/2682687454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d3b750 0x556869d3db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.264+0000 7f83065dd740 1 --2- 172.21.15.18:0/2682687454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32b70 0x556869d3b210 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.264+0000 7f83065dd740 1 --2- 172.21.15.18:0/2682687454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d32250 0x556869d32630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.264+0000 7f83065dd740 1 -- 172.21.15.18:0/2682687454 >> 172.21.15.18:0/2682687454 conn(0x556869d2c980 msgr2=0x556869d2cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.264+0000 7f83065dd740 1 -- 172.21.15.18:0/2682687454 shutdown_connections 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.264+0000 7f83065dd740 1 -- 172.21.15.18:0/2682687454 wait complete. 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.264+0000 7f83065dd740 1 -- start start 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83065dd740 1 Processor -- start 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83065dd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d38770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83065dd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 0x556869d38cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83065dd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 0x556869d391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83065dd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556869c9a020 con 0x556869d32250 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83065dd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686996d430 con 0x556869d32b70 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83065dd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556869929e00 con 0x556869d3b750 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83065dd740 10 client.0 _ll_register_callbacks cb 0x556869c41cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83017aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d38770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83017aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d38770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34282/0 (socket says 172.21.15.18:34282) 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83017aa640 1 -- 172.21.15.18:0/3161504303 learned_addr learned my addr 172.21.15.18:0/3161504303 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f8300fa9640 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 0x556869d38cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f8301fab640 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 0x556869d391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83017aa640 1 -- 172.21.15.18:0/3161504303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 msgr2=0x556869d391f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83017aa640 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 0x556869d391f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83017aa640 1 -- 172.21.15.18:0/3161504303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 msgr2=0x556869d38cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83017aa640 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 0x556869d38cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.265+0000 7f83017aa640 1 -- 172.21.15.18:0/3161504303 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc10e880 con 0x556869d32250 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f83017aa640 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d38770 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f82f402c170 tx=0x7f82f402c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fea171543b8bb9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f82e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f82e0ff9640 1 -- 172.21.15.18:0/3161504303 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4004800 con 0x556869d32250 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f82e0ff9640 1 -- 172.21.15.18:0/3161504303 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f82f402cbe0 con 0x556869d32250 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f83065dd740 1 -- 172.21.15.18:0/3161504303 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568699997e0 con 0x556869d32250 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f82e0ff9640 1 -- 172.21.15.18:0/3161504303 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f4004ad0 con 0x556869d32250 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f83065dd740 10 client.16461 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f83065dd740 1 -- 172.21.15.18:0/3161504303 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556869c986a0 con 0x556869d32250 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f82e0ff9640 1 -- 172.21.15.18:0/3161504303 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5568699997e0 con 0x556869d32250 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f82e0ff9640 1 -- 172.21.15.18:0/3161504303 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f82f403c050 con 0x556869d32250 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f83065dd740 10 client.16461 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f83065dd740 10 client.16461 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f83065dd740 20 client.16461 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f83065dd740 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556869d4b6f0 0x556869d4dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f83065dd740 4 client.16461 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.266+0000 7f83065dd740 1 -- 172.21.15.18:0/3161504303 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568699d6720 con 0x556869d4b6f0 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.267+0000 7f8300fa9640 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556869d4b6f0 0x556869d4dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.267+0000 7f8300fa9640 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556869d4b6f0 0x556869d4dae0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ecae7014adcaa856 server_cookie=67ae4bfb29451a26 in_seq=0 out_seq=0 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.267+0000 7f82e0ff9640 10 client.16461 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.267+0000 7f82e0ff9640 1 -- 172.21.15.18:0/3161504303 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5568699d6720 con 0x556869d4b6f0 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f82e0ff9640 10 client.16461 handle_command_reply: tid=0 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f83065dd740 1 client.16461 shutdown 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f83065dd740 1 -- 172.21.15.18:0/3161504303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 msgr2=0x556869d38770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f83065dd740 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d38770 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f82f402c170 tx=0x7f82f402c740 comp rx=0 tx=0).stop 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f83065dd740 1 -- 172.21.15.18:0/3161504303 shutdown_connections 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f83065dd740 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556869d4b6f0 0x556869d4dae0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f83065dd740 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 0x556869d391f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f82e0ff9640 0 client.16461 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:52:26.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f83065dd740 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 0x556869d38cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f83065dd740 1 --2- 172.21.15.18:0/3161504303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d38770 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f83065dd740 1 -- 172.21.15.18:0/3161504303 >> 172.21.15.18:0/3161504303 conn(0x556869d2c980 msgr2=0x556869d2e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f83065dd740 1 -- 172.21.15.18:0/3161504303 shutdown_connections 2024-02-17T12:52:26.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f83065dd740 1 -- 172.21.15.18:0/3161504303 wait complete. 2024-02-17T12:52:26.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.268+0000 7f83065dd740 20 client.16461 trim_cache size 0 max 16384 2024-02-17T12:52:26.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 80383 4.44 MB/sec warmup 101 sec latency 3352.145 ms 2024-02-17T12:52:26.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83065dd740 1 Processor -- start 2024-02-17T12:52:26.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83065dd740 1 -- start start 2024-02-17T12:52:26.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83065dd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d3aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83065dd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 0x556869d38cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83065dd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 0x556869d39200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83065dd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556869c1f130 con 0x556869d32250 2024-02-17T12:52:26.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83065dd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556869d31660 con 0x556869d32b70 2024-02-17T12:52:26.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83065dd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556869d31c90 con 0x556869d3b750 2024-02-17T12:52:26.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83017aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d3aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83017aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d3aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34284/0 (socket says 172.21.15.18:34284) 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83017aa640 1 -- 172.21.15.18:0/2399285242 learned_addr learned my addr 172.21.15.18:0/2399285242 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f8301fab640 1 --2- 172.21.15.18:0/2399285242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 0x556869d39200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f8300fa9640 1 --2- 172.21.15.18:0/2399285242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 0x556869d38cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83017aa640 1 -- 172.21.15.18:0/2399285242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 msgr2=0x556869d39200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83017aa640 1 --2- 172.21.15.18:0/2399285242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 0x556869d39200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83017aa640 1 -- 172.21.15.18:0/2399285242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 msgr2=0x556869d38cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83017aa640 1 --2- 172.21.15.18:0/2399285242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 0x556869d38cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83017aa640 1 -- 172.21.15.18:0/2399285242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556869c34c20 con 0x556869d32250 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.281+0000 7f83017aa640 1 --2- 172.21.15.18:0/2399285242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d3aea0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f82f402c410 tx=0x7f82f402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a43f3d57da8f68 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.282+0000 7f82e2ffd640 1 -- 172.21.15.18:0/2399285242 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40046c0 con 0x556869d32250 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.282+0000 7f82e2ffd640 1 -- 172.21.15.18:0/2399285242 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82f4034e90 con 0x556869d32250 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.282+0000 7f82e2ffd640 1 -- 172.21.15.18:0/2399285242 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f405b480 con 0x556869d32250 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.282+0000 7f83065dd740 1 -- 172.21.15.18:0/2399285242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 msgr2=0x556869d3aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.282+0000 7f83065dd740 1 --2- 172.21.15.18:0/2399285242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d3aea0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f82f402c410 tx=0x7f82f402bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:26.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.282+0000 7f83065dd740 1 -- 172.21.15.18:0/2399285242 shutdown_connections 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.282+0000 7f83065dd740 1 --2- 172.21.15.18:0/2399285242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 0x556869d39200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.282+0000 7f83065dd740 1 --2- 172.21.15.18:0/2399285242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 0x556869d38cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.282+0000 7f83065dd740 1 --2- 172.21.15.18:0/2399285242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d3aea0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.282+0000 7f83065dd740 1 -- 172.21.15.18:0/2399285242 >> 172.21.15.18:0/2399285242 conn(0x556869d2c980 msgr2=0x556869d33a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.282+0000 7f83065dd740 1 -- 172.21.15.18:0/2399285242 shutdown_connections 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.282+0000 7f83065dd740 1 -- 172.21.15.18:0/2399285242 wait complete. 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.283+0000 7f83065dd740 1 -- start start 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.283+0000 7f83065dd740 1 Processor -- start 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.283+0000 7f83065dd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d42700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.283+0000 7f83065dd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 0x556869d40af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.283+0000 7f83065dd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 0x556869d41030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.283+0000 7f83065dd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556869d41570 con 0x556869d32250 2024-02-17T12:52:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.283+0000 7f83065dd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556869d416e0 con 0x556869d32b70 2024-02-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.283+0000 7f83065dd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556869d418a0 con 0x556869d3b750 2024-02-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.283+0000 7f83065dd740 10 client.0 _ll_register_callbacks cb 0x7f82ec0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.283+0000 7f83017aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d42700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f83017aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d42700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34288/0 (socket says 172.21.15.18:34288) 2024-02-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f83017aa640 1 -- 172.21.15.18:0/64340797 learned_addr learned my addr 172.21.15.18:0/64340797 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f8301fab640 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 0x556869d41030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f8300fa9640 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 0x556869d40af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f83017aa640 1 -- 172.21.15.18:0/64340797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 msgr2=0x556869d41030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f83017aa640 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 0x556869d41030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f83017aa640 1 -- 172.21.15.18:0/64340797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 msgr2=0x556869d40af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f83017aa640 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 0x556869d40af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f83017aa640 1 -- 172.21.15.18:0/64340797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556869c986a0 con 0x556869d32250 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f83017aa640 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d42700 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f82f402c3b0 tx=0x7f82f4005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c39d70501d708a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f82e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f82e0ff9640 1 -- 172.21.15.18:0/64340797 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f402caf0 con 0x556869d32250 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f82e0ff9640 1 -- 172.21.15.18:0/64340797 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f82f4000e40 con 0x556869d32250 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f83065dd740 1 -- 172.21.15.18:0/64340797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82f4004960 con 0x556869d32250 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f82e0ff9640 1 -- 172.21.15.18:0/64340797 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f4034930 con 0x556869d32250 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f83065dd740 10 client.16467 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f83065dd740 1 -- 172.21.15.18:0/64340797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556869c5a8c0 con 0x556869d32250 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.284+0000 7f82e0ff9640 1 -- 172.21.15.18:0/64340797 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82f4004960 con 0x556869d32250 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.285+0000 7f82e0ff9640 1 -- 172.21.15.18:0/64340797 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f82f403c040 con 0x556869d32250 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.285+0000 7f83065dd740 10 client.16467 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.285+0000 7f83065dd740 10 client.16467 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.285+0000 7f83065dd740 20 client.16467 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.285+0000 7f83065dd740 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556869da2be0 0x556869da4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.285+0000 7f83065dd740 4 client.16467 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.285+0000 7f83065dd740 1 -- 172.21.15.18:0/64340797 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556869a25890 con 0x556869da2be0 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.285+0000 7f8300fa9640 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556869da2be0 0x556869da4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.285+0000 7f82e0ff9640 10 client.16467 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.285+0000 7f8300fa9640 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556869da2be0 0x556869da4fd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5af2d9aa8f968eac server_cookie=2e5abc1154780468 in_seq=0 out_seq=0 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f82e0ff9640 1 -- 172.21.15.18:0/64340797 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556869a25890 con 0x556869da2be0 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f82e0ff9640 10 client.16467 handle_command_reply: tid=0 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f83065dd740 1 client.16467 shutdown 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f83065dd740 1 -- 172.21.15.18:0/64340797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 msgr2=0x556869d42700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f83065dd740 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d42700 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f82f402c3b0 tx=0x7f82f4005950 comp rx=0 tx=0).stop 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f83065dd740 1 -- 172.21.15.18:0/64340797 shutdown_connections 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f83065dd740 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556869da2be0 0x556869da4fd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f83065dd740 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556869d3b750 0x556869d41030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f83065dd740 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556869d32b70 0x556869d40af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f82e0ff9640 0 client.16467 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:52:26.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f83065dd740 1 --2- 172.21.15.18:0/64340797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556869d32250 0x556869d42700 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f83065dd740 1 -- 172.21.15.18:0/64340797 >> 172.21.15.18:0/64340797 conn(0x556869d2c980 msgr2=0x556869d2b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f83065dd740 1 -- 172.21.15.18:0/64340797 shutdown_connections 2024-02-17T12:52:26.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f83065dd740 1 -- 172.21.15.18:0/64340797 wait complete. 2024-02-17T12:52:26.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.286+0000 7f83065dd740 20 client.16467 trim_cache size 0 max 16384 2024-02-17T12:52:26.284 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:52:26.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.287+0000 7f8303a0e640 1 -- 172.21.15.18:0/4144903349 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82e40a2230 msgr2=0x7f82e40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.287+0000 7f8303a0e640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82e40a2230 0x7f82e40a46f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f82ec005fd0 tx=0x7f82ec009be0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.287+0000 7f8303a0e640 1 -- 172.21.15.18:0/4144903349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82fc1064c0 msgr2=0x7f82fc19d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.287+0000 7f8303a0e640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82fc1064c0 0x7f82fc19d9a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f82f000e970 tx=0x7f82f000ee30 comp rx=0 tx=0).stop 2024-02-17T12:52:26.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.287+0000 7f8303a0e640 1 -- 172.21.15.18:0/4144903349 shutdown_connections 2024-02-17T12:52:26.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.287+0000 7f8303a0e640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82e40a2230 0x7f82e40a46f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.287+0000 7f8303a0e640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc197610 0x7f82fc197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.287+0000 7f8303a0e640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82fc1064c0 0x7f82fc19d9a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.287+0000 7f8303a0e640 1 --2- 172.21.15.18:0/4144903349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82fc0685f0 0x7f82fc19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.287+0000 7f8303a0e640 1 -- 172.21.15.18:0/4144903349 >> 172.21.15.18:0/4144903349 conn(0x7f82fc077180 msgr2=0x7f82fc0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.287+0000 7f8303a0e640 1 -- 172.21.15.18:0/4144903349 shutdown_connections 2024-02-17T12:52:26.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.287+0000 7f8303a0e640 1 -- 172.21.15.18:0/4144903349 wait complete. 2024-02-17T12:52:26.289 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:52:26.289 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:52:26.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.407+0000 7f7532f8c640 1 -- 172.21.15.18:0/536593067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f752c100c10 msgr2=0x7f752c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.407+0000 7f7532f8c640 1 --2- 172.21.15.18:0/536593067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f752c100c10 0x7f752c104c50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7514009930 tx=0x7f751402bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.407+0000 7f7532f8c640 1 -- 172.21.15.18:0/536593067 shutdown_connections 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.407+0000 7f7532f8c640 1 --2- 172.21.15.18:0/536593067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f752c105330 0x7f752c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.407+0000 7f7532f8c640 1 --2- 172.21.15.18:0/536593067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f752c100c10 0x7f752c104c50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.408+0000 7f7532f8c640 1 --2- 172.21.15.18:0/536593067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f752c100260 0x7f752c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.408+0000 7f7532f8c640 1 -- 172.21.15.18:0/536593067 >> 172.21.15.18:0/536593067 conn(0x7f752c077180 msgr2=0x7f752c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.408+0000 7f7532f8c640 1 -- 172.21.15.18:0/536593067 shutdown_connections 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.408+0000 7f7532f8c640 1 -- 172.21.15.18:0/536593067 wait complete. 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.408+0000 7f7532f8c640 1 Processor -- start 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.408+0000 7f7532f8c640 1 -- start start 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.408+0000 7f7532f8c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f752c100260 0x7f752c1966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.408+0000 7f7532f8c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f752c105330 0x7f752c196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.408+0000 7f7532f8c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f752c19af70 0x7f752c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.408+0000 7f7532f8c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c19b960 con 0x7f752c19af70 2024-02-17T12:52:26.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.408+0000 7f7532f8c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c19bad0 con 0x7f752c100260 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.408+0000 7f7532f8c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c19bc40 con 0x7f752c105330 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7530d28640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f752c100260 0x7f752c1966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7531529640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f752c19af70 0x7f752c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7530d28640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f752c100260 0x7f752c1966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:57076/0 (socket says 172.21.15.18:57076) 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7530d28640 1 -- 172.21.15.18:0/1502554669 learned_addr learned my addr 172.21.15.18:0/1502554669 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f752bfff640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f752c105330 0x7f752c196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7530d28640 1 -- 172.21.15.18:0/1502554669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f752c105330 msgr2=0x7f752c196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7530d28640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f752c105330 0x7f752c196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7530d28640 1 -- 172.21.15.18:0/1502554669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f752c19af70 msgr2=0x7f752c19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7530d28640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f752c19af70 0x7f752c19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7530d28640 1 -- 172.21.15.18:0/1502554669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c009840 con 0x7f752c100260 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7531529640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f752c19af70 0x7f752c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7530d28640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f752c100260 0x7f752c1966b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f751c009d50 tx=0x7f751c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7529ffb640 1 -- 172.21.15.18:0/1502554669 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c003b70 con 0x7f752c100260 2024-02-17T12:52:26.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7529ffb640 1 -- 172.21.15.18:0/1502554669 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f751c003cd0 con 0x7f752c100260 2024-02-17T12:52:26.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7532f8c640 1 -- 172.21.15.18:0/1502554669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7514009560 con 0x7f752c100260 2024-02-17T12:52:26.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7529ffb640 1 -- 172.21.15.18:0/1502554669 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c010990 con 0x7f752c100260 2024-02-17T12:52:26.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.409+0000 7f7532f8c640 1 -- 172.21.15.18:0/1502554669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c1a3510 con 0x7f752c100260 2024-02-17T12:52:26.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.411+0000 7f7529ffb640 1 -- 172.21.15.18:0/1502554669 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f751c004000 con 0x7f752c100260 2024-02-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f75357fd740 1 Processor -- start 2024-02-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f75357fd740 1 -- start start 2024-02-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f7529ffb640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f75180a2230 0x7f75180a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f75357fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f2b250 0x55cda7f2b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f75357fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2bb70 0x55cda7f34210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f75357fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f34750 0x55cda7f36b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f7529ffb640 1 -- 172.21.15.18:0/1502554669 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f751c014070 con 0x7f752c100260 2024-02-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f7531529640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f75180a2230 0x7f75180a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f75357fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda7ee6960 con 0x55cda7f34750 2024-02-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f75357fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda7e9a310 con 0x55cda7f2bb70 2024-02-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f75357fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda7e97840 con 0x55cda7f2b250 2024-02-17T12:52:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f752bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2bb70 0x55cda7f34210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f7531529640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f34750 0x55cda7f36b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f752bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2bb70 0x55cda7f34210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:57078/0 (socket says 172.21.15.18:57078) 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f752bfff640 1 -- 172.21.15.18:0/3829336374 learned_addr learned my addr 172.21.15.18:0/3829336374 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.412+0000 7f7530d28640 1 --2- 172.21.15.18:0/3829336374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f2b250 0x55cda7f2b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.413+0000 7f752bfff640 1 -- 172.21.15.18:0/3829336374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f2b250 msgr2=0x55cda7f2b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.413+0000 7f752bfff640 1 --2- 172.21.15.18:0/3829336374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f2b250 0x55cda7f2b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.413+0000 7f752bfff640 1 -- 172.21.15.18:0/3829336374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f34750 msgr2=0x55cda7f36b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.413+0000 7f752bfff640 1 --2- 172.21.15.18:0/3829336374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f34750 0x55cda7f36b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.413+0000 7f752bfff640 1 -- 172.21.15.18:0/3829336374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cda7e55060 con 0x55cda7f2bb70 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.413+0000 7f7530d28640 1 --2- 172.21.15.18:0/3829336374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f2b250 0x55cda7f2b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.413+0000 7f7531529640 1 --2- 172.21.15.18:0/3829336374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f34750 0x55cda7f36b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.413+0000 7f7531529640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f75180a2230 0x7f75180a46f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7524005f60 tx=0x7f7524009ed0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.413+0000 7f752bfff640 1 --2- 172.21.15.18:0/3829336374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2bb70 0x55cda7f34210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7514005ab0 tx=0x7f751402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4514fa31b4fefb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.413+0000 7f7522ffd640 1 -- 172.21.15.18:0/3829336374 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7514004800 con 0x55cda7f2bb70 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.413+0000 7f7522ffd640 1 -- 172.21.15.18:0/3829336374 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7514004960 con 0x55cda7f2bb70 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.413+0000 7f7522ffd640 1 -- 172.21.15.18:0/3829336374 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7514005050 con 0x55cda7f2bb70 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.414+0000 7f75357fd740 1 -- 172.21.15.18:0/3829336374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2bb70 msgr2=0x55cda7f34210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.414+0000 7f75357fd740 1 --2- 172.21.15.18:0/3829336374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2bb70 0x55cda7f34210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7514005ab0 tx=0x7f751402c5d0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.414+0000 7f75357fd740 1 -- 172.21.15.18:0/3829336374 shutdown_connections 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.414+0000 7f75357fd740 1 --2- 172.21.15.18:0/3829336374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f34750 0x55cda7f36b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.414+0000 7f75357fd740 1 --2- 172.21.15.18:0/3829336374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2bb70 0x55cda7f34210 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.414+0000 7f75357fd740 1 --2- 172.21.15.18:0/3829336374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f2b250 0x55cda7f2b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.414+0000 7f75357fd740 1 -- 172.21.15.18:0/3829336374 >> 172.21.15.18:0/3829336374 conn(0x55cda7f25980 msgr2=0x55cda7f25d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.414+0000 7f75357fd740 1 -- 172.21.15.18:0/3829336374 shutdown_connections 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.414+0000 7f75357fd740 1 -- 172.21.15.18:0/3829336374 wait complete. 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.414+0000 7f75357fd740 1 -- start start 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.414+0000 7f75357fd740 1 Processor -- start 2024-02-17T12:52:26.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f75357fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 0x55cda7f316b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f75357fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2bb70 0x55cda7f31bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f75357fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f34750 0x55cda7f32130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f75357fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda7e93020 con 0x55cda7f2b250 2024-02-17T12:52:26.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f75357fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda7b66430 con 0x55cda7f2bb70 2024-02-17T12:52:26.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f75357fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda7b22e00 con 0x55cda7f34750 2024-02-17T12:52:26.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f75357fd740 10 client.0 _ll_register_callbacks cb 0x55cda7e3acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:26.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f7530d28640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 0x55cda7f316b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f7530d28640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 0x55cda7f316b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34320/0 (socket says 172.21.15.18:34320) 2024-02-17T12:52:26.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f7530d28640 1 -- 172.21.15.18:0/3012091263 learned_addr learned my addr 172.21.15.18:0/3012091263 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f752bfff640 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2bb70 0x55cda7f31bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f7531529640 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f34750 0x55cda7f32130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f7530d28640 1 -- 172.21.15.18:0/3012091263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f34750 msgr2=0x55cda7f32130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f7530d28640 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f34750 0x55cda7f32130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f7530d28640 1 -- 172.21.15.18:0/3012091263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2bb70 msgr2=0x55cda7f31bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f7530d28640 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2bb70 0x55cda7f31bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f7530d28640 1 -- 172.21.15.18:0/3012091263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c10d710 con 0x55cda7f2b250 2024-02-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f7530d28640 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 0x55cda7f316b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f751c00cba0 tx=0x7f751c003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06fa0e87f9e2e77 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f750bfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f750bfff640 1 -- 172.21.15.18:0/3012091263 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c09f400 con 0x55cda7f2b250 2024-02-17T12:52:26.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f750bfff640 1 -- 172.21.15.18:0/3012091263 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f751c0a3a50 con 0x55cda7f2b250 2024-02-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.415+0000 7f75357fd740 1 -- 172.21.15.18:0/3012091263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cda7b927e0 con 0x55cda7f2b250 2024-02-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.416+0000 7f750bfff640 1 -- 172.21.15.18:0/3012091263 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f751c0042b0 con 0x55cda7f2b250 2024-02-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.416+0000 7f75357fd740 10 client.16479 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.416+0000 7f75357fd740 1 -- 172.21.15.18:0/3012091263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cda7e916a0 con 0x55cda7f2b250 2024-02-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.416+0000 7f750bfff640 1 -- 172.21.15.18:0/3012091263 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cda7b927e0 con 0x55cda7f2b250 2024-02-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.416+0000 7f750bfff640 1 -- 172.21.15.18:0/3012091263 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f751c09a700 con 0x55cda7f2b250 2024-02-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.416+0000 7f75357fd740 10 client.16479 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.416+0000 7f75357fd740 10 client.16479 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.416+0000 7f75357fd740 20 client.16479 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.416+0000 7f75357fd740 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55cda7f44a10 0x55cda7f46e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.416+0000 7f75357fd740 4 client.16479 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.416+0000 7f75357fd740 1 -- 172.21.15.18:0/3012091263 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cda7bcf720 con 0x55cda7f44a10 2024-02-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.416+0000 7f752bfff640 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55cda7f44a10 0x55cda7f46e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f752bfff640 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55cda7f44a10 0x55cda7f46e00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e67dcae17f31c83f server_cookie=45bb16d5e4c52856 in_seq=0 out_seq=0 2024-02-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f750bfff640 10 client.16479 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f750bfff640 1 -- 172.21.15.18:0/3012091263 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cda7bcf720 con 0x55cda7f44a10 2024-02-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f750bfff640 10 client.16479 handle_command_reply: tid=0 2024-02-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f75357fd740 1 client.16479 shutdown 2024-02-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f75357fd740 1 -- 172.21.15.18:0/3012091263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 msgr2=0x55cda7f316b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f75357fd740 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 0x55cda7f316b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f751c00cba0 tx=0x7f751c003e40 comp rx=0 tx=0).stop 2024-02-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f75357fd740 1 -- 172.21.15.18:0/3012091263 shutdown_connections 2024-02-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f75357fd740 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55cda7f44a10 0x55cda7f46e00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f75357fd740 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f34750 0x55cda7f32130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f75357fd740 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2bb70 0x55cda7f31bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f75357fd740 1 --2- 172.21.15.18:0/3012091263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 0x55cda7f316b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f750bfff640 0 client.16479 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:52:26.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f75357fd740 1 -- 172.21.15.18:0/3012091263 >> 172.21.15.18:0/3012091263 conn(0x55cda7f25980 msgr2=0x55cda7f276d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f75357fd740 1 -- 172.21.15.18:0/3012091263 shutdown_connections 2024-02-17T12:52:26.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.417+0000 7f75357fd740 1 -- 172.21.15.18:0/3012091263 wait complete. 2024-02-17T12:52:26.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.418+0000 7f75357fd740 20 client.16479 trim_cache size 0 max 16384 2024-02-17T12:52:26.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f75357fd740 1 Processor -- start 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f75357fd740 1 -- start start 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f75357fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 0x55cda7f33de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f75357fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f2bb70 0x55cda7f31f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f75357fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f34750 0x55cda7f2e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f75357fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda7e18130 con 0x55cda7f2b250 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f75357fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda7f36b40 con 0x55cda7f34750 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f75357fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda7f2aeb0 con 0x55cda7f2bb70 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f7530d28640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 0x55cda7f33de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f7530d28640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 0x55cda7f33de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34324/0 (socket says 172.21.15.18:34324) 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f7530d28640 1 -- 172.21.15.18:0/4232093946 learned_addr learned my addr 172.21.15.18:0/4232093946 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f752bfff640 1 --2- 172.21.15.18:0/4232093946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f2bb70 0x55cda7f31f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.430+0000 7f7531529640 1 --2- 172.21.15.18:0/4232093946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f34750 0x55cda7f2e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.431+0000 7f7530d28640 1 -- 172.21.15.18:0/4232093946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f2bb70 msgr2=0x55cda7f31f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.431+0000 7f7530d28640 1 --2- 172.21.15.18:0/4232093946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f2bb70 0x55cda7f31f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.431+0000 7f7530d28640 1 -- 172.21.15.18:0/4232093946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f34750 msgr2=0x55cda7f2e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.431+0000 7f7530d28640 1 --2- 172.21.15.18:0/4232093946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f34750 0x55cda7f2e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.431+0000 7f7530d28640 1 -- 172.21.15.18:0/4232093946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cda7e2dc20 con 0x55cda7f2b250 2024-02-17T12:52:26.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.431+0000 7f7530d28640 1 --2- 172.21.15.18:0/4232093946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 0x55cda7f33de0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f751c009d20 tx=0x7f751c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0e7524cc223857 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.431+0000 7f75227fc640 1 -- 172.21.15.18:0/4232093946 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c0a3a50 con 0x55cda7f2b250 2024-02-17T12:52:26.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.431+0000 7f75227fc640 1 -- 172.21.15.18:0/4232093946 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f751c09f400 con 0x55cda7f2b250 2024-02-17T12:52:26.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.431+0000 7f75227fc640 1 -- 172.21.15.18:0/4232093946 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c0042b0 con 0x55cda7f2b250 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 -- 172.21.15.18:0/4232093946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 msgr2=0x55cda7f33de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 --2- 172.21.15.18:0/4232093946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 0x55cda7f33de0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f751c009d20 tx=0x7f751c009c00 comp rx=0 tx=0).stop 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 -- 172.21.15.18:0/4232093946 shutdown_connections 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 --2- 172.21.15.18:0/4232093946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f34750 0x55cda7f2e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 --2- 172.21.15.18:0/4232093946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f2bb70 0x55cda7f31f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 --2- 172.21.15.18:0/4232093946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2b250 0x55cda7f33de0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 -- 172.21.15.18:0/4232093946 >> 172.21.15.18:0/4232093946 conn(0x55cda7f25980 msgr2=0x55cda7f2c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 -- 172.21.15.18:0/4232093946 shutdown_connections 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 -- 172.21.15.18:0/4232093946 wait complete. 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 -- start start 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 Processor -- start 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2b250 0x55cda7f96990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.432+0000 7f75357fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2bb70 0x55cda7f96ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f75357fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f34750 0x55cda7f90e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f75357fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda7f91410 con 0x55cda7f2bb70 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f75357fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda7f91580 con 0x55cda7f2b250 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f75357fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda7f91740 con 0x55cda7f34750 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f75357fd740 10 client.0 _ll_register_callbacks cb 0x7f751c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f752bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2bb70 0x55cda7f96ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f7530d28640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2b250 0x55cda7f96990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f752bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2bb70 0x55cda7f96ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34336/0 (socket says 172.21.15.18:34336) 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f7530d28640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2b250 0x55cda7f96990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:57104/0 (socket says 172.21.15.18:57104) 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f752bfff640 1 -- 172.21.15.18:0/3349463932 learned_addr learned my addr 172.21.15.18:0/3349463932 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f7531529640 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f34750 0x55cda7f90e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f752bfff640 1 -- 172.21.15.18:0/3349463932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f34750 msgr2=0x55cda7f90e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f752bfff640 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f34750 0x55cda7f90e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f752bfff640 1 -- 172.21.15.18:0/3349463932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2b250 msgr2=0x55cda7f96990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f752bfff640 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2b250 0x55cda7f96990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f752bfff640 1 -- 172.21.15.18:0/3349463932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cda7e916a0 con 0x55cda7f2bb70 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f7530d28640 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2b250 0x55cda7f96990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f752bfff640 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2bb70 0x55cda7f96ed0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7514009930 tx=0x7f751402c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b03efb012cc1f4d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f750bfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f750bfff640 1 -- 172.21.15.18:0/3349463932 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751404f040 con 0x55cda7f2bb70 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f750bfff640 1 -- 172.21.15.18:0/3349463932 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f751402c860 con 0x55cda7f2bb70 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f75357fd740 1 -- 172.21.15.18:0/3349463932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7514004960 con 0x55cda7f2bb70 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f750bfff640 1 -- 172.21.15.18:0/3349463932 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f751402ca20 con 0x55cda7f2bb70 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f75357fd740 10 client.16485 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.433+0000 7f75357fd740 1 -- 172.21.15.18:0/3349463932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cda7e538c0 con 0x55cda7f2bb70 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f750bfff640 1 -- 172.21.15.18:0/3349463932 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7514004960 con 0x55cda7f2bb70 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f750bfff640 1 -- 172.21.15.18:0/3349463932 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f751403c050 con 0x55cda7f2bb70 2024-02-17T12:52:26.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f75357fd740 10 client.16485 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f75357fd740 10 client.16485 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f75357fd740 20 client.16485 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f75357fd740 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55cda7f9e580 0x55cda7fa09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f75357fd740 4 client.16485 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f75357fd740 1 -- 172.21.15.18:0/3349463932 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cda7c1e890 con 0x55cda7f9e580 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f7530d28640 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55cda7f9e580 0x55cda7fa09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f7530d28640 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55cda7f9e580 0x55cda7fa09a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52669761d1204373 server_cookie=a7ac076c7f89640b in_seq=0 out_seq=0 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f750bfff640 10 client.16485 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f750bfff640 1 -- 172.21.15.18:0/3349463932 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cda7c1e890 con 0x55cda7f9e580 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.434+0000 7f750bfff640 10 client.16485 handle_command_reply: tid=0 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f75357fd740 1 client.16485 shutdown 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f75357fd740 1 -- 172.21.15.18:0/3349463932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2bb70 msgr2=0x55cda7f96ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f75357fd740 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2bb70 0x55cda7f96ed0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7514009930 tx=0x7f751402c740 comp rx=0 tx=0).stop 2024-02-17T12:52:26.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f75357fd740 1 -- 172.21.15.18:0/3349463932 shutdown_connections 2024-02-17T12:52:26.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f75357fd740 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55cda7f9e580 0x55cda7fa09a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f75357fd740 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cda7f34750 0x55cda7f90e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f750bfff640 0 client.16485 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:52:26.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f75357fd740 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda7f2bb70 0x55cda7f96ed0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f75357fd740 1 --2- 172.21.15.18:0/3349463932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cda7f2b250 0x55cda7f96990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f75357fd740 1 -- 172.21.15.18:0/3349463932 >> 172.21.15.18:0/3349463932 conn(0x55cda7f25980 msgr2=0x55cda7f242f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f75357fd740 1 -- 172.21.15.18:0/3349463932 shutdown_connections 2024-02-17T12:52:26.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f75357fd740 1 -- 172.21.15.18:0/3349463932 wait complete. 2024-02-17T12:52:26.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f75357fd740 20 client.16485 trim_cache size 0 max 16384 2024-02-17T12:52:26.433 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:52:26.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f7532f8c640 1 -- 172.21.15.18:0/1502554669 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f75180a2230 msgr2=0x7f75180a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f7532f8c640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f75180a2230 0x7f75180a46f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7524005f60 tx=0x7f7524009ed0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f7532f8c640 1 -- 172.21.15.18:0/1502554669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f752c100260 msgr2=0x7f752c1966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f7532f8c640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f752c100260 0x7f752c1966b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f751c009d50 tx=0x7f751c00c6a0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f7532f8c640 1 -- 172.21.15.18:0/1502554669 shutdown_connections 2024-02-17T12:52:26.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f7532f8c640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f75180a2230 0x7f75180a46f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f7532f8c640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f752c19af70 0x7f752c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f7532f8c640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f752c105330 0x7f752c196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f7532f8c640 1 --2- 172.21.15.18:0/1502554669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f752c100260 0x7f752c1966b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.435+0000 7f7532f8c640 1 -- 172.21.15.18:0/1502554669 >> 172.21.15.18:0/1502554669 conn(0x7f752c077180 msgr2=0x7f752c0fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.436+0000 7f7532f8c640 1 -- 172.21.15.18:0/1502554669 shutdown_connections 2024-02-17T12:52:26.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.436+0000 7f7532f8c640 1 -- 172.21.15.18:0/1502554669 wait complete. 2024-02-17T12:52:26.438 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:52:26.438 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:52:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.554+0000 7f5670cd4640 1 -- 172.21.15.18:0/2413567992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f566c100b90 msgr2=0x7f566c101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.554+0000 7f5670cd4640 1 --2- 172.21.15.18:0/2413567992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f566c100b90 0x7f566c101010 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5654009930 tx=0x7f565402bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.554+0000 7f5670cd4640 1 -- 172.21.15.18:0/2413567992 shutdown_connections 2024-02-17T12:52:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.554+0000 7f5670cd4640 1 --2- 172.21.15.18:0/2413567992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f566c101550 0x7f566c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.554+0000 7f5670cd4640 1 --2- 172.21.15.18:0/2413567992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f566c100b90 0x7f566c101010 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.554+0000 7f5670cd4640 1 --2- 172.21.15.18:0/2413567992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f566c105020 0x7f566c105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.554+0000 7f5670cd4640 1 -- 172.21.15.18:0/2413567992 >> 172.21.15.18:0/2413567992 conn(0x7f566c0752e0 msgr2=0x7f566c0756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.554+0000 7f5670cd4640 1 -- 172.21.15.18:0/2413567992 shutdown_connections 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.554+0000 7f5670cd4640 1 -- 172.21.15.18:0/2413567992 wait complete. 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.555+0000 7f5670cd4640 1 Processor -- start 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.555+0000 7f5670cd4640 1 -- start start 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.555+0000 7f5670cd4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f566c100b90 0x7f566c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.555+0000 7f5670cd4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f566c101550 0x7f566c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.555+0000 7f5670cd4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f566c105020 0x7f566c19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.555+0000 7f5670cd4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c19c5b0 con 0x7f566c100b90 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.555+0000 7f5670cd4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c19c720 con 0x7f566c101550 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.555+0000 7f5670cd4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c19c8e0 con 0x7f566c105020 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f566a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f566c100b90 0x7f566c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f566a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f566c100b90 0x7f566c1976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34362/0 (socket says 172.21.15.18:34362) 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f5663fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f566c101550 0x7f566c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f566a59c640 1 -- 172.21.15.18:0/3556887621 learned_addr learned my addr 172.21.15.18:0/3556887621 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f566ad9d640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f566c105020 0x7f566c19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f566a59c640 1 -- 172.21.15.18:0/3556887621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f566c105020 msgr2=0x7f566c19bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f566a59c640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f566c105020 0x7f566c19bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f566a59c640 1 -- 172.21.15.18:0/3556887621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f566c101550 msgr2=0x7f566c197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f566a59c640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f566c101550 0x7f566c197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f566a59c640 1 -- 172.21.15.18:0/3556887621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5654009560 con 0x7f566c100b90 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f566ad9d640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f566c105020 0x7f566c19bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f5663fff640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f566c101550 0x7f566c197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f566a59c640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f566c100b90 0x7f566c1976f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f565c00e9c0 tx=0x7f565c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f56637fe640 1 -- 172.21.15.18:0/3556887621 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c009700 con 0x7f566c100b90 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f56637fe640 1 -- 172.21.15.18:0/3556887621 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f565c00cd80 con 0x7f566c100b90 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f5670cd4640 1 -- 172.21.15.18:0/3556887621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566c19cbc0 con 0x7f566c100b90 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f56637fe640 1 -- 172.21.15.18:0/3556887621 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c010430 con 0x7f566c100b90 2024-02-17T12:52:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.556+0000 7f5670cd4640 1 -- 172.21.15.18:0/3556887621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566c1a4440 con 0x7f566c100b90 2024-02-17T12:52:26.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.558+0000 7f56637fe640 1 -- 172.21.15.18:0/3556887621 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f565c003680 con 0x7f566c100b90 2024-02-17T12:52:26.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f5673980740 1 Processor -- start 2024-02-17T12:52:26.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f5673980740 1 -- start start 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f56637fe640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56580a2230 0x7f56580a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f5673980740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65250 0x55ff14f65630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f5673980740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 0x55ff14f6e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f5673980740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f70b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f5673980740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14f20960 con 0x55ff14f65b70 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f5673980740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14ed4310 con 0x55ff14f65250 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f5673980740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14ed1840 con 0x55ff14f6e750 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f56637fe640 1 -- 172.21.15.18:0/3556887621 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f565c014070 con 0x7f566c100b90 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f566ad9d640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56580a2230 0x7f56580a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f5663fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 0x55ff14f6e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f5663fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 0x55ff14f6e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34372/0 (socket says 172.21.15.18:34372) 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f5663fff640 1 -- 172.21.15.18:0/2973142531 learned_addr learned my addr 172.21.15.18:0/2973142531 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.559+0000 7f566a59c640 1 --2- 172.21.15.18:0/2973142531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65250 0x55ff14f65630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f566ad9d640 1 --2- 172.21.15.18:0/2973142531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f70b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f5663fff640 1 -- 172.21.15.18:0/2973142531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 msgr2=0x55ff14f70b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f5663fff640 1 --2- 172.21.15.18:0/2973142531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f70b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f5663fff640 1 -- 172.21.15.18:0/2973142531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65250 msgr2=0x55ff14f65630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f5663fff640 1 --2- 172.21.15.18:0/2973142531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65250 0x55ff14f65630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f5663fff640 1 -- 172.21.15.18:0/2973142531 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff14e8f060 con 0x55ff14f65b70 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f566ad9d640 1 --2- 172.21.15.18:0/2973142531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f70b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f566a59c640 1 --2- 172.21.15.18:0/2973142531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65250 0x55ff14f65630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f566ad9d640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56580a2230 0x7f56580a46f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f56640099e0 tx=0x7f566400fef0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f5663fff640 1 --2- 172.21.15.18:0/2973142531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 0x55ff14f6e210 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f565402c240 tx=0x7f565402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7d4cc99886b411 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f5660ff9640 1 -- 172.21.15.18:0/2973142531 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565402dea0 con 0x55ff14f65b70 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f5660ff9640 1 -- 172.21.15.18:0/2973142531 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5654004040 con 0x55ff14f65b70 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.560+0000 7f5660ff9640 1 -- 172.21.15.18:0/2973142531 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654005290 con 0x55ff14f65b70 2024-02-17T12:52:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.561+0000 7f5673980740 1 -- 172.21.15.18:0/2973142531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 msgr2=0x55ff14f6e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.561+0000 7f5673980740 1 --2- 172.21.15.18:0/2973142531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 0x55ff14f6e210 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f565402c240 tx=0x7f565402c660 comp rx=0 tx=0).stop 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.561+0000 7f5673980740 1 -- 172.21.15.18:0/2973142531 shutdown_connections 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.561+0000 7f5673980740 1 --2- 172.21.15.18:0/2973142531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f70b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.561+0000 7f5673980740 1 --2- 172.21.15.18:0/2973142531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 0x55ff14f6e210 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.561+0000 7f5673980740 1 --2- 172.21.15.18:0/2973142531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65250 0x55ff14f65630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.561+0000 7f5673980740 1 -- 172.21.15.18:0/2973142531 >> 172.21.15.18:0/2973142531 conn(0x55ff14f5f980 msgr2=0x55ff14f5fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.561+0000 7f5673980740 1 -- 172.21.15.18:0/2973142531 shutdown_connections 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.561+0000 7f5673980740 1 -- 172.21.15.18:0/2973142531 wait complete. 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.561+0000 7f5673980740 1 -- start start 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.561+0000 7f5673980740 1 Processor -- start 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5673980740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 0x55ff14f6b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5673980740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65b70 0x55ff14f6bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5673980740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f6c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5673980740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14ecd020 con 0x55ff14f65250 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5673980740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14ba0430 con 0x55ff14f65b70 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5673980740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14b5ce00 con 0x55ff14f6e750 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5673980740 10 client.0 _ll_register_callbacks cb 0x55ff14e74cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f566a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 0x55ff14f6b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f566a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 0x55ff14f6b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34388/0 (socket says 172.21.15.18:34388) 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f566a59c640 1 -- 172.21.15.18:0/2245388029 learned_addr learned my addr 172.21.15.18:0/2245388029 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f566ad9d640 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f6c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5663fff640 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65b70 0x55ff14f6bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f566a59c640 1 -- 172.21.15.18:0/2245388029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 msgr2=0x55ff14f6c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f566a59c640 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f6c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f566a59c640 1 -- 172.21.15.18:0/2245388029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65b70 msgr2=0x55ff14f6bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f566a59c640 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65b70 0x55ff14f6bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f566a59c640 1 -- 172.21.15.18:0/2245388029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566c10d800 con 0x55ff14f65250 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f566a59c640 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 0x55ff14f6b770 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f565c0cbc80 tx=0x7f565c09a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bccea4ba53edc3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5651ffb640 1 -- 172.21.15.18:0/2245388029 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c09ae70 con 0x55ff14f65250 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5651ffb640 1 -- 172.21.15.18:0/2245388029 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f565c09efc0 con 0x55ff14f65250 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5673980740 1 -- 172.21.15.18:0/2245388029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff14bcc7e0 con 0x55ff14f65250 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5651ffb640 1 -- 172.21.15.18:0/2245388029 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f565c0a3610 con 0x55ff14f65250 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5673980740 10 client.16497 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.562+0000 7f5673980740 1 -- 172.21.15.18:0/2245388029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ff14ecb6a0 con 0x55ff14f65250 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.563+0000 7f5651ffb640 1 -- 172.21.15.18:0/2245388029 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f565c07d020 con 0x55ff14f65250 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.563+0000 7f5651ffb640 1 -- 172.21.15.18:0/2245388029 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f565c01f060 con 0x55ff14f65250 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.563+0000 7f5673980740 10 client.16497 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.563+0000 7f5673980740 10 client.16497 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.563+0000 7f5673980740 20 client.16497 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.563+0000 7f5673980740 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ff14f7eb50 0x55ff14f80f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.563+0000 7f5673980740 4 client.16497 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.563+0000 7f5673980740 1 -- 172.21.15.18:0/2245388029 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff14c09720 con 0x55ff14f7eb50 2024-02-17T12:52:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.563+0000 7f5663fff640 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ff14f7eb50 0x55ff14f80f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.563+0000 7f5663fff640 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ff14f7eb50 0x55ff14f80f60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd915f9feff7f803 server_cookie=966bc6e7a1f25b41 in_seq=0 out_seq=0 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.563+0000 7f5651ffb640 10 client.16497 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5651ffb640 1 -- 172.21.15.18:0/2245388029 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff14c09720 con 0x55ff14f7eb50 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5651ffb640 10 client.16497 handle_command_reply: tid=0 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5673980740 1 client.16497 shutdown 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5673980740 1 -- 172.21.15.18:0/2245388029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 msgr2=0x55ff14f6b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5673980740 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 0x55ff14f6b770 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f565c0cbc80 tx=0x7f565c09a8c0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5673980740 1 -- 172.21.15.18:0/2245388029 shutdown_connections 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5673980740 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ff14f7eb50 0x55ff14f80f60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5673980740 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f6c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5673980740 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65b70 0x55ff14f6bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5651ffb640 0 client.16497 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5673980740 1 --2- 172.21.15.18:0/2245388029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 0x55ff14f6b770 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5673980740 1 -- 172.21.15.18:0/2245388029 >> 172.21.15.18:0/2245388029 conn(0x55ff14f5f980 msgr2=0x55ff14f616d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5673980740 1 -- 172.21.15.18:0/2245388029 shutdown_connections 2024-02-17T12:52:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5673980740 1 -- 172.21.15.18:0/2245388029 wait complete. 2024-02-17T12:52:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.564+0000 7f5673980740 20 client.16497 trim_cache size 0 max 16384 2024-02-17T12:52:26.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f5673980740 1 Processor -- start 2024-02-17T12:52:26.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f5673980740 1 -- start start 2024-02-17T12:52:26.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f5673980740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 0x55ff14f5ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f5673980740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f65b70 0x55ff14f755d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f5673980740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f6e750 0x55ff14f75b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f5673980740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14e52130 con 0x55ff14f65250 2024-02-17T12:52:26.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f5673980740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14f60550 con 0x55ff14f6e750 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f5673980740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14f86150 con 0x55ff14f65b70 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f566a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 0x55ff14f5ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f566a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 0x55ff14f5ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34392/0 (socket says 172.21.15.18:34392) 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f566a59c640 1 -- 172.21.15.18:0/3327230402 learned_addr learned my addr 172.21.15.18:0/3327230402 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f566ad9d640 1 --2- 172.21.15.18:0/3327230402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f6e750 0x55ff14f75b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f566a59c640 1 -- 172.21.15.18:0/3327230402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f65b70 msgr2=0x55ff14f755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f566a59c640 1 --2- 172.21.15.18:0/3327230402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f65b70 0x55ff14f755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f566a59c640 1 -- 172.21.15.18:0/3327230402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f6e750 msgr2=0x55ff14f75b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.576+0000 7f566a59c640 1 --2- 172.21.15.18:0/3327230402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f6e750 0x55ff14f75b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f566a59c640 1 -- 172.21.15.18:0/3327230402 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff14e67c20 con 0x55ff14f65250 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f566a59c640 1 --2- 172.21.15.18:0/3327230402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 0x55ff14f5ea00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f565c09ae00 tx=0x7f565c099380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a0cc909d2a73c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f5653fff640 1 -- 172.21.15.18:0/3327230402 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c04a8d0 con 0x55ff14f65250 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f5653fff640 1 -- 172.21.15.18:0/3327230402 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f565c07d030 con 0x55ff14f65250 2024-02-17T12:52:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f5653fff640 1 -- 172.21.15.18:0/3327230402 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c049f80 con 0x55ff14f65250 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f5673980740 1 -- 172.21.15.18:0/3327230402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 msgr2=0x55ff14f5ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f5673980740 1 --2- 172.21.15.18:0/3327230402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 0x55ff14f5ea00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f565c09ae00 tx=0x7f565c099380 comp rx=0 tx=0).stop 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f5673980740 1 -- 172.21.15.18:0/3327230402 shutdown_connections 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f5673980740 1 --2- 172.21.15.18:0/3327230402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f6e750 0x55ff14f75b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f5673980740 1 --2- 172.21.15.18:0/3327230402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f65b70 0x55ff14f755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f5673980740 1 --2- 172.21.15.18:0/3327230402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65250 0x55ff14f5ea00 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f5673980740 1 -- 172.21.15.18:0/3327230402 >> 172.21.15.18:0/3327230402 conn(0x55ff14f5f980 msgr2=0x55ff14f5e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f5673980740 1 -- 172.21.15.18:0/3327230402 shutdown_connections 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.577+0000 7f5673980740 1 -- 172.21.15.18:0/3327230402 wait complete. 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5673980740 1 -- start start 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5673980740 1 Processor -- start 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5673980740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65250 0x55ff14f740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5673980740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 0x55ff14f745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5673980740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f74b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5673980740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14fd5030 con 0x55ff14f65b70 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5673980740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14fd5170 con 0x55ff14f65250 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5673980740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14fd52e0 con 0x55ff14f6e750 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5673980740 10 client.0 _ll_register_callbacks cb 0x7f56640093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5663fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 0x55ff14f745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5663fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 0x55ff14f745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34406/0 (socket says 172.21.15.18:34406) 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5663fff640 1 -- 172.21.15.18:0/3502723376 learned_addr learned my addr 172.21.15.18:0/3502723376 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f566a59c640 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65250 0x55ff14f740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f566ad9d640 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f74b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5663fff640 1 -- 172.21.15.18:0/3502723376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 msgr2=0x55ff14f74b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5663fff640 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f74b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5663fff640 1 -- 172.21.15.18:0/3502723376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65250 msgr2=0x55ff14f740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5663fff640 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65250 0x55ff14f740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.578+0000 7f5663fff640 1 -- 172.21.15.18:0/3502723376 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff14ecb6a0 con 0x55ff14f65b70 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5663fff640 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 0x55ff14f745e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5654044000 tx=0x7f5654005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f66e559bb1bbbe1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5651ffb640 1 -- 172.21.15.18:0/3502723376 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565402dea0 con 0x55ff14f65b70 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5651ffb640 1 -- 172.21.15.18:0/3502723376 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f565404f040 con 0x55ff14f65b70 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5673980740 1 -- 172.21.15.18:0/3502723376 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5654004040 con 0x55ff14f65b70 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5651ffb640 1 -- 172.21.15.18:0/3502723376 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5654052de0 con 0x55ff14f65b70 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5673980740 10 client.16503 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5673980740 1 -- 172.21.15.18:0/3502723376 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ff14e8d8c0 con 0x55ff14f65b70 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5651ffb640 1 -- 172.21.15.18:0/3502723376 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5654004040 con 0x55ff14f65b70 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5651ffb640 1 -- 172.21.15.18:0/3502723376 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f565403c050 con 0x55ff14f65b70 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5673980740 10 client.16503 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5673980740 10 client.16503 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5673980740 20 client.16503 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5673980740 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ff14fd6810 0x55ff14fd8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5673980740 4 client.16503 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f5673980740 1 -- 172.21.15.18:0/3502723376 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ff14c58890 con 0x55ff14fd6810 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.579+0000 7f566a59c640 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ff14fd6810 0x55ff14fd8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5651ffb640 10 client.16503 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f566a59c640 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ff14fd6810 0x55ff14fd8c30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c873717e0475a686 server_cookie=6ea72f9709afc065 in_seq=0 out_seq=0 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5651ffb640 1 -- 172.21.15.18:0/3502723376 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ff14c58890 con 0x55ff14fd6810 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5651ffb640 10 client.16503 handle_command_reply: tid=0 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5673980740 1 client.16503 shutdown 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5673980740 1 -- 172.21.15.18:0/3502723376 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 msgr2=0x55ff14f745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5673980740 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 0x55ff14f745e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5654044000 tx=0x7f5654005bf0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5673980740 1 -- 172.21.15.18:0/3502723376 shutdown_connections 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5673980740 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55ff14fd6810 0x55ff14fd8c30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5673980740 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ff14f6e750 0x55ff14f74b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5651ffb640 0 client.16503 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5673980740 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff14f65b70 0x55ff14f745e0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5673980740 1 --2- 172.21.15.18:0/3502723376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff14f65250 0x55ff14f740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5673980740 1 -- 172.21.15.18:0/3502723376 >> 172.21.15.18:0/3502723376 conn(0x55ff14f5f980 msgr2=0x55ff14f5eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5673980740 1 -- 172.21.15.18:0/3502723376 shutdown_connections 2024-02-17T12:52:26.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5673980740 1 -- 172.21.15.18:0/3502723376 wait complete. 2024-02-17T12:52:26.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.580+0000 7f5673980740 20 client.16503 trim_cache size 0 max 16384 2024-02-17T12:52:26.579 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:52:26.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.581+0000 7f5670cd4640 1 -- 172.21.15.18:0/3556887621 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56580a2230 msgr2=0x7f56580a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.581+0000 7f5670cd4640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56580a2230 0x7f56580a46f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f56640099e0 tx=0x7f566400fef0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.581+0000 7f5670cd4640 1 -- 172.21.15.18:0/3556887621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f566c100b90 msgr2=0x7f566c1976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.581+0000 7f5670cd4640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f566c100b90 0x7f566c1976f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f565c00e9c0 tx=0x7f565c00ee80 comp rx=0 tx=0).stop 2024-02-17T12:52:26.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.581+0000 7f5670cd4640 1 -- 172.21.15.18:0/3556887621 shutdown_connections 2024-02-17T12:52:26.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.581+0000 7f5670cd4640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56580a2230 0x7f56580a46f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.581+0000 7f5670cd4640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f566c105020 0x7f566c19bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.581+0000 7f5670cd4640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f566c101550 0x7f566c197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.581+0000 7f5670cd4640 1 --2- 172.21.15.18:0/3556887621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f566c100b90 0x7f566c1976f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.581+0000 7f5670cd4640 1 -- 172.21.15.18:0/3556887621 >> 172.21.15.18:0/3556887621 conn(0x7f566c0752e0 msgr2=0x7f566c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.581+0000 7f5670cd4640 1 -- 172.21.15.18:0/3556887621 shutdown_connections 2024-02-17T12:52:26.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.581+0000 7f5670cd4640 1 -- 172.21.15.18:0/3556887621 wait complete. 2024-02-17T12:52:26.582 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:26 smithi018 ceph-mon[32625]: pgmap v79: 97 pgs: 97 active+clean; 43 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-02-17T12:52:26.582 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:26 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/4269910354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:52:26.583 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:52:26.583 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:52:26.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:26 smithi119 ceph-mon[36188]: pgmap v79: 97 pgs: 97 active+clean; 43 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-02-17T12:52:26.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:26 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/4269910354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:52:26.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.698+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/3828937565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c34100c50 msgr2=0x7f5c34104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.698+0000 7f5c3c3b5640 1 --2- 172.21.15.18:0/3828937565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c34100c50 0x7f5c34104c90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009930 tx=0x7f5c2402bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:26.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.698+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/3828937565 shutdown_connections 2024-02-17T12:52:26.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.698+0000 7f5c3c3b5640 1 --2- 172.21.15.18:0/3828937565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c34105370 0x7f5c3410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.698+0000 7f5c3c3b5640 1 --2- 172.21.15.18:0/3828937565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c34100c50 0x7f5c34104c90 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.698+0000 7f5c3c3b5640 1 --2- 172.21.15.18:0/3828937565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c341002a0 0x7f5c34100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.698+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/3828937565 >> 172.21.15.18:0/3828937565 conn(0x7f5c34076400 msgr2=0x7f5c34076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.699+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/3828937565 shutdown_connections 2024-02-17T12:52:26.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.699+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/3828937565 wait complete. 2024-02-17T12:52:26.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.699+0000 7f5c3c3b5640 1 Processor -- start 2024-02-17T12:52:26.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.699+0000 7f5c3c3b5640 1 -- start start 2024-02-17T12:52:26.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.699+0000 7f5c3c3b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341002a0 0x7f5c3419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.699+0000 7f5c3c3b5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c34105370 0x7f5c3419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.699+0000 7f5c3c3b5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c34197640 0x7f5c34197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.699+0000 7f5c3c3b5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34198110 con 0x7f5c34197640 2024-02-17T12:52:26.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.699+0000 7f5c3c3b5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34198280 con 0x7f5c341002a0 2024-02-17T12:52:26.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.699+0000 7f5c3c3b5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34198440 con 0x7f5c34105370 2024-02-17T12:52:26.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.699+0000 7f5c3a151640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341002a0 0x7f5c3419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.699+0000 7f5c39950640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c34105370 0x7f5c3419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c3a952640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c34197640 0x7f5c34197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c39950640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c34105370 0x7f5c3419dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:58924/0 (socket says 172.21.15.18:58924) 2024-02-17T12:52:26.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c3a151640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341002a0 0x7f5c3419d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:57164/0 (socket says 172.21.15.18:57164) 2024-02-17T12:52:26.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c39950640 1 -- 172.21.15.18:0/4102633856 learned_addr learned my addr 172.21.15.18:0/4102633856 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c3a952640 1 -- 172.21.15.18:0/4102633856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c34105370 msgr2=0x7f5c3419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c3a952640 1 --2- 172.21.15.18:0/4102633856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c34105370 0x7f5c3419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c3a952640 1 -- 172.21.15.18:0/4102633856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341002a0 msgr2=0x7f5c3419d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c3a952640 1 --2- 172.21.15.18:0/4102633856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341002a0 0x7f5c3419d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c3a952640 1 -- 172.21.15.18:0/4102633856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24009560 con 0x7f5c34197640 2024-02-17T12:52:26.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c3a952640 1 --2- 172.21.15.18:0/4102633856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c34197640 0x7f5c34197ac0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009fd0 tx=0x7f5c2c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c2b7fe640 1 -- 172.21.15.18:0/4102633856 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00ce20 con 0x7f5c34197640 2024-02-17T12:52:26.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c2b7fe640 1 -- 172.21.15.18:0/4102633856 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c2c005020 con 0x7f5c34197640 2024-02-17T12:52:26.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/4102633856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c341a43e0 con 0x7f5c34197640 2024-02-17T12:52:26.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c2b7fe640 1 -- 172.21.15.18:0/4102633856 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c010690 con 0x7f5c34197640 2024-02-17T12:52:26.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.700+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/4102633856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c341a4870 con 0x7f5c34197640 2024-02-17T12:52:26.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:26 smithi146 ceph-mon[36454]: pgmap v79: 97 pgs: 97 active+clean; 43 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-02-17T12:52:26.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:26 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/4269910354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:52:26.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.703+0000 7f5c2b7fe640 1 -- 172.21.15.18:0/4102633856 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5c2c005500 con 0x7f5c34197640 2024-02-17T12:52:26.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.703+0000 7f5c3ef88740 1 Processor -- start 2024-02-17T12:52:26.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.703+0000 7f5c3ef88740 1 -- start start 2024-02-17T12:52:26.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.703+0000 7f5c3ef88740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8250 0x55efd1fa8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.703+0000 7f5c3ef88740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fa8b70 0x55efd1fb1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.703+0000 7f5c3ef88740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fb1750 0x55efd1fb3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.703+0000 7f5c3ef88740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efd1f63960 con 0x55efd1fb1750 2024-02-17T12:52:26.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.703+0000 7f5c3ef88740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efd1f17310 con 0x55efd1fa8b70 2024-02-17T12:52:26.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.703+0000 7f5c3ef88740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efd1f14840 con 0x55efd1fa8250 2024-02-17T12:52:26.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.703+0000 7f5c2b7fe640 1 --2- 172.21.15.18:0/4102633856 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5c200a2230 0x7f5c200a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c39950640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fb1750 0x55efd1fb3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c3a151640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fa8b70 0x55efd1fb1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c3a151640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fa8b70 0x55efd1fb1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:57168/0 (socket says 172.21.15.18:57168) 2024-02-17T12:52:26.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c39950640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fb1750 0x55efd1fb3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34440/0 (socket says 172.21.15.18:34440) 2024-02-17T12:52:26.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c3a151640 1 -- 172.21.15.18:0/1634055555 learned_addr learned my addr 172.21.15.18:0/1634055555 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c3a952640 1 --2- 172.21.15.18:0/4102633856 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5c200a2230 0x7f5c200a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c39950640 1 --2- 172.21.15.18:0/1634055555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8250 0x55efd1fa8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c2b7fe640 1 -- 172.21.15.18:0/4102633856 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5c2c014070 con 0x7f5c34197640 2024-02-17T12:52:26.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c3a151640 1 -- 172.21.15.18:0/1634055555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8250 msgr2=0x55efd1fa8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c3a151640 1 --2- 172.21.15.18:0/1634055555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8250 0x55efd1fa8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c3a151640 1 -- 172.21.15.18:0/1634055555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fb1750 msgr2=0x55efd1fb3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c3a151640 1 --2- 172.21.15.18:0/1634055555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fb1750 0x55efd1fb3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c3a151640 1 -- 172.21.15.18:0/1634055555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efd1ed2060 con 0x55efd1fa8b70 2024-02-17T12:52:26.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c39950640 1 --2- 172.21.15.18:0/1634055555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fb1750 0x55efd1fb3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:26.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c3a952640 1 --2- 172.21.15.18:0/4102633856 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5c200a2230 0x7f5c200a46f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c09ae70 tx=0x7f5c2c06d130 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c3a151640 1 --2- 172.21.15.18:0/1634055555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fa8b70 0x55efd1fb1210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009cf0 tx=0x7f5c30011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0501e087689ca00 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c28bfe640 1 -- 172.21.15.18:0/1634055555 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30009ea0 con 0x55efd1fa8b70 2024-02-17T12:52:26.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c28bfe640 1 -- 172.21.15.18:0/1634055555 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c30004230 con 0x55efd1fa8b70 2024-02-17T12:52:26.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.704+0000 7f5c28bfe640 1 -- 172.21.15.18:0/1634055555 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30019800 con 0x55efd1fa8b70 2024-02-17T12:52:26.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.705+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1634055555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fa8b70 msgr2=0x55efd1fb1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.705+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1634055555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fa8b70 0x55efd1fb1210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009cf0 tx=0x7f5c30011810 comp rx=0 tx=0).stop 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.705+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1634055555 shutdown_connections 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.705+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1634055555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fb1750 0x55efd1fb3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.705+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1634055555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fa8b70 0x55efd1fb1210 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.705+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1634055555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8250 0x55efd1fa8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.705+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1634055555 >> 172.21.15.18:0/1634055555 conn(0x55efd1fa2980 msgr2=0x55efd1fa2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.705+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1634055555 shutdown_connections 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.705+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1634055555 wait complete. 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.705+0000 7f5c3ef88740 1 -- start start 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3ef88740 1 Processor -- start 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3ef88740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 0x55efd1fae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3ef88740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8b70 0x55efd1faec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3ef88740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fb1750 0x55efd1faf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3ef88740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efd1f10020 con 0x55efd1fa8250 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3ef88740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efd1be3430 con 0x55efd1fb1750 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3ef88740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efd1b9fe00 con 0x55efd1fa8b70 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3ef88740 10 client.0 _ll_register_callbacks cb 0x55efd1eb7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c39950640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 0x55efd1fae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c39950640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 0x55efd1fae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34452/0 (socket says 172.21.15.18:34452) 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c39950640 1 -- 172.21.15.18:0/1688521313 learned_addr learned my addr 172.21.15.18:0/1688521313 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3a151640 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8b70 0x55efd1faec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c39950640 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fb1750 0x55efd1faf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3a151640 1 -- 172.21.15.18:0/1688521313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fb1750 msgr2=0x55efd1faf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3a151640 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fb1750 0x55efd1faf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3a151640 1 -- 172.21.15.18:0/1688521313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 msgr2=0x55efd1fae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3a151640 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 0x55efd1fae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c3a151640 1 -- 172.21.15.18:0/1688521313 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c30009950 con 0x55efd1fa8b70 2024-02-17T12:52:26.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.706+0000 7f5c39950640 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 0x55efd1fae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c3a151640 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8b70 0x55efd1faec10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009cf0 tx=0x7f5c30005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c6c122ce48ca397 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c1dffb640 1 -- 172.21.15.18:0/1688521313 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30009ea0 con 0x55efd1fa8b70 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1688521313 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55efd1c0f7e0 con 0x55efd1fa8b70 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c1dffb640 1 -- 172.21.15.18:0/1688521313 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5c30005040 con 0x55efd1fa8b70 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c1dffb640 1 -- 172.21.15.18:0/1688521313 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c30005200 con 0x55efd1fa8b70 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c3ef88740 10 client.25912 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1688521313 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55efd1f0e6a0 con 0x55efd1fa8b70 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c1dffb640 1 -- 172.21.15.18:0/1688521313 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55efd1c0f7e0 con 0x55efd1fa8b70 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c1dffb640 1 -- 172.21.15.18:0/1688521313 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5c30021040 con 0x55efd1fa8b70 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c3ef88740 10 client.25912 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c3ef88740 10 client.25912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c3ef88740 20 client.25912 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55efd1fc1a50 0x55efd1fc3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c3ef88740 4 client.25912 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.707+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1688521313 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efd1c4c720 con 0x55efd1fc1a50 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.708+0000 7f5c39950640 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55efd1fc1a50 0x55efd1fc3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.708+0000 7f5c1dffb640 10 client.25912 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.708+0000 7f5c39950640 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55efd1fc1a50 0x55efd1fc3e70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c966c5284e9bd4a3 server_cookie=4f9a9b8f204c7c96 in_seq=0 out_seq=0 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c1dffb640 1 -- 172.21.15.18:0/1688521313 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f5c2404fa90 con 0x55efd1fc1a50 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c1dffb640 10 client.25912 handle_command_reply: tid=0 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c3ef88740 1 client.25912 shutdown 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1688521313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8b70 msgr2=0x55efd1faec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8b70 0x55efd1faec10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009cf0 tx=0x7f5c30005d30 comp rx=0 tx=0).stop 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1688521313 shutdown_connections 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55efd1fc1a50 0x55efd1fc3e70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fb1750 0x55efd1faf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8b70 0x55efd1faec10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c1dffb640 0 client.25912 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1688521313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 0x55efd1fae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1688521313 >> 172.21.15.18:0/1688521313 conn(0x55efd1fa2980 msgr2=0x55efd1fa46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.709+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1688521313 shutdown_connections 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.710+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1688521313 wait complete. 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.710+0000 7f5c3ef88740 20 client.25912 trim_cache size 0 max 16384 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.722+0000 7f5c3ef88740 1 Processor -- start 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.722+0000 7f5c3ef88740 1 -- start start 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.722+0000 7f5c3ef88740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fa8250 0x55efd1fb0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.722+0000 7f5c3ef88740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8b70 0x55efd1faef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.722+0000 7f5c3ef88740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fb1750 0x55efd1fab930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.722+0000 7f5c3ef88740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efd1e95130 con 0x55efd1fa8b70 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.722+0000 7f5c3ef88740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efd1fa3550 con 0x55efd1fa8250 2024-02-17T12:52:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.722+0000 7f5c3ef88740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efd1fa3980 con 0x55efd1fb1750 2024-02-17T12:52:26.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c3a151640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8b70 0x55efd1faef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c3a151640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8b70 0x55efd1faef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34466/0 (socket says 172.21.15.18:34466) 2024-02-17T12:52:26.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c3a151640 1 -- 172.21.15.18:0/3120955792 learned_addr learned my addr 172.21.15.18:0/3120955792 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c39950640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fa8250 0x55efd1fb0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c39950640 1 --2- 172.21.15.18:0/3120955792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fb1750 0x55efd1fab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c3a151640 1 -- 172.21.15.18:0/3120955792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fb1750 msgr2=0x55efd1fab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c3a151640 1 --2- 172.21.15.18:0/3120955792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fb1750 0x55efd1fab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c3a151640 1 -- 172.21.15.18:0/3120955792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fa8250 msgr2=0x55efd1fb0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c3a151640 1 --2- 172.21.15.18:0/3120955792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fa8250 0x55efd1fb0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c3a151640 1 -- 172.21.15.18:0/3120955792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efd1eaac20 con 0x55efd1fa8b70 2024-02-17T12:52:26.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c3a151640 1 --2- 172.21.15.18:0/3120955792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8b70 0x55efd1faef70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5c300056b0 tx=0x7f5c30003900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229ca2aea886f8be server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c1ffff640 1 -- 172.21.15.18:0/3120955792 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30009b70 con 0x55efd1fa8b70 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c1ffff640 1 -- 172.21.15.18:0/3120955792 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c300195f0 con 0x55efd1fa8b70 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.723+0000 7f5c1ffff640 1 -- 172.21.15.18:0/3120955792 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c300236c0 con 0x55efd1fa8b70 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 -- 172.21.15.18:0/3120955792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8b70 msgr2=0x55efd1faef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/3120955792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8b70 0x55efd1faef70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5c300056b0 tx=0x7f5c30003900 comp rx=0 tx=0).stop 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 -- 172.21.15.18:0/3120955792 shutdown_connections 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/3120955792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fb1750 0x55efd1fab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/3120955792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8b70 0x55efd1faef70 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/3120955792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fa8250 0x55efd1fb0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 -- 172.21.15.18:0/3120955792 >> 172.21.15.18:0/3120955792 conn(0x55efd1fa2980 msgr2=0x55efd1fa9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 -- 172.21.15.18:0/3120955792 shutdown_connections 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 -- 172.21.15.18:0/3120955792 wait complete. 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 -- start start 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 Processor -- start 2024-02-17T12:52:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 0x55efd1fb89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8b70 0x55efd1fb6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fb1750 0x55efd1fb7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efd1fb7a20 con 0x55efd1fa8250 2024-02-17T12:52:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efd1fb7b90 con 0x55efd1fb1750 2024-02-17T12:52:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efd2017f60 con 0x55efd1fa8b70 2024-02-17T12:52:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c3ef88740 10 client.0 _ll_register_callbacks cb 0x7f5c300097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.724+0000 7f5c39950640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 0x55efd1fb89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c39950640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 0x55efd1fb89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34470/0 (socket says 172.21.15.18:34470) 2024-02-17T12:52:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c39950640 1 -- 172.21.15.18:0/1979903834 learned_addr learned my addr 172.21.15.18:0/1979903834 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c3a151640 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8b70 0x55efd1fb6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c39950640 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fb1750 0x55efd1fb7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c39950640 1 -- 172.21.15.18:0/1979903834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8b70 msgr2=0x55efd1fb6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c39950640 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8b70 0x55efd1fb6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c39950640 1 -- 172.21.15.18:0/1979903834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fb1750 msgr2=0x55efd1fb7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c39950640 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fb1750 0x55efd1fb7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c39950640 1 -- 172.21.15.18:0/1979903834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efd1f0e6a0 con 0x55efd1fa8250 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c39950640 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 0x55efd1fb89f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5c2402c780 tx=0x7f5c24009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23165f14171f5f12 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c1dffb640 1 -- 172.21.15.18:0/1979903834 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24058070 con 0x55efd1fa8250 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c1dffb640 1 -- 172.21.15.18:0/1979903834 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5c240035f0 con 0x55efd1fa8250 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1979903834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55efd1c9b890 con 0x55efd1fa8250 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c1dffb640 1 -- 172.21.15.18:0/1979903834 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c240037b0 con 0x55efd1fa8250 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c3ef88740 10 client.16521 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.725+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1979903834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55efd1ed08c0 con 0x55efd1fa8250 2024-02-17T12:52:26.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c1dffb640 1 -- 172.21.15.18:0/1979903834 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55efd1c9b890 con 0x55efd1fa8250 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c1dffb640 1 -- 172.21.15.18:0/1979903834 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5c24005280 con 0x55efd1fa8250 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c3ef88740 10 client.16521 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c3ef88740 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c3ef88740 20 client.16521 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55efd2019750 0x55efd201bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c3ef88740 4 client.16521 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1979903834 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55efd1bbb700 con 0x55efd2019750 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c3a151640 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55efd2019750 0x55efd201bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c1dffb640 10 client.16521 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c3a151640 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55efd2019750 0x55efd201bb70 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b699aa2d6a834225 server_cookie=ab607919e7de0c7f in_seq=0 out_seq=0 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c1dffb640 1 -- 172.21.15.18:0/1979903834 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55efd1bbb700 con 0x55efd2019750 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c1dffb640 10 client.16521 handle_command_reply: tid=0 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.726+0000 7f5c3ef88740 1 client.16521 shutdown 2024-02-17T12:52:26.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1979903834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 msgr2=0x55efd1fb89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 0x55efd1fb89f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5c2402c780 tx=0x7f5c24009260 comp rx=0 tx=0).stop 2024-02-17T12:52:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1979903834 shutdown_connections 2024-02-17T12:52:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55efd2019750 0x55efd201bb70 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55efd1fb1750 0x55efd1fb7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efd1fa8b70 0x55efd1fb6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c1dffb640 0 client.16521 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3ef88740 1 --2- 172.21.15.18:0/1979903834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efd1fa8250 0x55efd1fb89f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1979903834 >> 172.21.15.18:0/1979903834 conn(0x55efd1fa2980 msgr2=0x55efd1fa12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1979903834 shutdown_connections 2024-02-17T12:52:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3ef88740 1 -- 172.21.15.18:0/1979903834 wait complete. 2024-02-17T12:52:26.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3ef88740 20 client.16521 trim_cache size 0 max 16384 2024-02-17T12:52:26.740 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:52:26.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/4102633856 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5c200a2230 msgr2=0x7f5c200a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3c3b5640 1 --2- 172.21.15.18:0/4102633856 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5c200a2230 0x7f5c200a46f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c09ae70 tx=0x7f5c2c06d130 comp rx=0 tx=0).stop 2024-02-17T12:52:26.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/4102633856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c34197640 msgr2=0x7f5c34197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:26.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3c3b5640 1 --2- 172.21.15.18:0/4102633856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c34197640 0x7f5c34197ac0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009fd0 tx=0x7f5c2c00ef10 comp rx=0 tx=0).stop 2024-02-17T12:52:26.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/4102633856 shutdown_connections 2024-02-17T12:52:26.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3c3b5640 1 --2- 172.21.15.18:0/4102633856 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5c200a2230 0x7f5c200a46f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3c3b5640 1 --2- 172.21.15.18:0/4102633856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c34197640 0x7f5c34197ac0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3c3b5640 1 --2- 172.21.15.18:0/4102633856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c34105370 0x7f5c3419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3c3b5640 1 --2- 172.21.15.18:0/4102633856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341002a0 0x7f5c3419d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:26.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/4102633856 >> 172.21.15.18:0/4102633856 conn(0x7f5c34076400 msgr2=0x7f5c34069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/4102633856 shutdown_connections 2024-02-17T12:52:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:26.727+0000 7f5c3c3b5640 1 -- 172.21.15.18:0/4102633856 wait complete. 2024-02-17T12:52:27.265 INFO:tasks.workunit.client.0.smithi018.stdout: 1 82180 4.52 MB/sec warmup 102 sec latency 17.191 ms 2024-02-17T12:52:27.743 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:52:27.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:52:27.744 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:52:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.864+0000 7fc609039640 1 -- 172.21.15.18:0/2244373654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc604105330 msgr2=0x7fc60410cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.864+0000 7fc609039640 1 --2- 172.21.15.18:0/2244373654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc604105330 0x7fc60410cf00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009cd0 tx=0x7fc5fc02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.864+0000 7fc609039640 1 -- 172.21.15.18:0/2244373654 shutdown_connections 2024-02-17T12:52:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.864+0000 7fc609039640 1 --2- 172.21.15.18:0/2244373654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc604105330 0x7fc60410cf00 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.864+0000 7fc609039640 1 --2- 172.21.15.18:0/2244373654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc604100c10 0x7fc604104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.864+0000 7fc609039640 1 --2- 172.21.15.18:0/2244373654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc604100260 0x7fc604100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.864+0000 7fc609039640 1 -- 172.21.15.18:0/2244373654 >> 172.21.15.18:0/2244373654 conn(0x7fc604077180 msgr2=0x7fc604077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.864+0000 7fc609039640 1 -- 172.21.15.18:0/2244373654 shutdown_connections 2024-02-17T12:52:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.864+0000 7fc609039640 1 -- 172.21.15.18:0/2244373654 wait complete. 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc609039640 1 Processor -- start 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc609039640 1 -- start start 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc609039640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc604100260 0x7fc60419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc609039640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc604105330 0x7fc60419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc609039640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc604197590 0x7fc604197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc609039640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc604197f80 con 0x7fc604105330 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc609039640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6041980f0 con 0x7fc604100260 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc609039640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6041982b0 con 0x7fc604197590 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc5fbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc604105330 0x7fc60419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc5fbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc604105330 0x7fc60419d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34486/0 (socket says 172.21.15.18:34486) 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc5fbfff640 1 -- 172.21.15.18:0/1433395462 learned_addr learned my addr 172.21.15.18:0/1433395462 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc602d9d640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc604100260 0x7fc60419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc60359e640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc604197590 0x7fc604197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc5fbfff640 1 -- 172.21.15.18:0/1433395462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc604197590 msgr2=0x7fc604197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc5fbfff640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc604197590 0x7fc604197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.865+0000 7fc5fbfff640 1 -- 172.21.15.18:0/1433395462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc604100260 msgr2=0x7fc60419d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.866+0000 7fc5fbfff640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc604100260 0x7fc60419d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.866+0000 7fc5fbfff640 1 -- 172.21.15.18:0/1433395462 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc009900 con 0x7fc604105330 2024-02-17T12:52:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.866+0000 7fc602d9d640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc604100260 0x7fc60419d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.866+0000 7fc5fbfff640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc604105330 0x7fc60419d920 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00d740 tx=0x7fc5ec00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.866+0000 7fc600d99640 1 -- 172.21.15.18:0/1433395462 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec009c30 con 0x7fc604105330 2024-02-17T12:52:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.866+0000 7fc600d99640 1 -- 172.21.15.18:0/1433395462 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5ec00de50 con 0x7fc604105330 2024-02-17T12:52:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.866+0000 7fc609039640 1 -- 172.21.15.18:0/1433395462 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6041a42a0 con 0x7fc604105330 2024-02-17T12:52:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.866+0000 7fc600d99640 1 -- 172.21.15.18:0/1433395462 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec010460 con 0x7fc604105330 2024-02-17T12:52:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.866+0000 7fc609039640 1 -- 172.21.15.18:0/1433395462 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6041a4760 con 0x7fc604105330 2024-02-17T12:52:27.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.868+0000 7fc600d99640 1 -- 172.21.15.18:0/1433395462 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc5ec004f10 con 0x7fc604105330 2024-02-17T12:52:27.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc60bc0a740 1 Processor -- start 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc60bc0a740 1 -- start start 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc60bc0a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5380 0x55b26efa5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc60bc0a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5d30 0x55b26efae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc60bc0a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efae8b0 0x55b26efb0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc60bc0a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b26ef13f90 con 0x55b26efa5380 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc60bc0a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b26ef114c0 con 0x55b26efa5d30 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc60bc0a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b26ef0cca0 con 0x55b26efae8b0 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc60359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5380 0x55b26efa5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc600d99640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5f00a2230 0x7fc5f00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc60359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5380 0x55b26efa5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34490/0 (socket says 172.21.15.18:34490) 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc60359e640 1 -- 172.21.15.18:0/4104251007 learned_addr learned my addr 172.21.15.18:0/4104251007 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc602d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5d30 0x55b26efae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc5fbfff640 1 --2- 172.21.15.18:0/4104251007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efae8b0 0x55b26efb0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.869+0000 7fc60359e640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5f00a2230 0x7fc5f00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.870+0000 7fc602d9d640 1 -- 172.21.15.18:0/4104251007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efae8b0 msgr2=0x55b26efb0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.870+0000 7fc602d9d640 1 --2- 172.21.15.18:0/4104251007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efae8b0 0x55b26efb0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.870+0000 7fc600d99640 1 -- 172.21.15.18:0/1433395462 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc5ec0cd910 con 0x7fc604105330 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.870+0000 7fc602d9d640 1 -- 172.21.15.18:0/4104251007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5380 msgr2=0x55b26efa5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.870+0000 7fc602d9d640 1 --2- 172.21.15.18:0/4104251007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5380 0x55b26efa5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.870+0000 7fc602d9d640 1 -- 172.21.15.18:0/4104251007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b26eecf0b0 con 0x55b26efa5d30 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.870+0000 7fc602d9d640 1 --2- 172.21.15.18:0/4104251007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5d30 0x55b26efae2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009fd0 tx=0x7fc5f40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3f3cc7fe621b4d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.870+0000 7fc60359e640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5f00a2230 0x7fc5f00a46f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc033000 tx=0x7fc5fc02c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.870+0000 7fc5f97fa640 1 -- 172.21.15.18:0/4104251007 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4004030 con 0x55b26efa5d30 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.870+0000 7fc5f97fa640 1 -- 172.21.15.18:0/4104251007 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5f4005500 con 0x55b26efa5d30 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.870+0000 7fc5f97fa640 1 -- 172.21.15.18:0/4104251007 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4004bf0 con 0x55b26efa5d30 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 -- 172.21.15.18:0/4104251007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5d30 msgr2=0x55b26efae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/4104251007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5d30 0x55b26efae2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009fd0 tx=0x7fc5f40118d0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 -- 172.21.15.18:0/4104251007 shutdown_connections 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/4104251007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efae8b0 0x55b26efb0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/4104251007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5d30 0x55b26efae2e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/4104251007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5380 0x55b26efa5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 -- 172.21.15.18:0/4104251007 >> 172.21.15.18:0/4104251007 conn(0x55b26ef9f840 msgr2=0x55b26ef9fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 -- 172.21.15.18:0/4104251007 shutdown_connections 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 -- 172.21.15.18:0/4104251007 wait complete. 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 -- start start 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 Processor -- start 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efa5380 0x55b26efab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 0x55b26efabd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efae8b0 0x55b26efac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.871+0000 7fc60bc0a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b26eeb1f40 con 0x55b26efa5d30 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc60bc0a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b26ebe04b0 con 0x55b26efae8b0 2024-02-17T12:52:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc60bc0a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b26ee92670 con 0x55b26efa5380 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc60bc0a740 10 client.0 _ll_register_callbacks cb 0x55b26eefff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5fbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 0x55b26efabd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5fbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 0x55b26efabd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34500/0 (socket says 172.21.15.18:34500) 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5fbfff640 1 -- 172.21.15.18:0/1054636278 learned_addr learned my addr 172.21.15.18:0/1054636278 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc602d9d640 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efa5380 0x55b26efab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc60359e640 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efae8b0 0x55b26efac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5fbfff640 1 -- 172.21.15.18:0/1054636278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efa5380 msgr2=0x55b26efab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5fbfff640 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efa5380 0x55b26efab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5fbfff640 1 -- 172.21.15.18:0/1054636278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efae8b0 msgr2=0x55b26efac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5fbfff640 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efae8b0 0x55b26efac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5fbfff640 1 -- 172.21.15.18:0/1054636278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f4009c00 con 0x55b26efa5d30 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5fbfff640 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 0x55b26efabd60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec096860 tx=0x7fc5ec004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdebe3f89bddb18 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5ea7fc640 1 -- 172.21.15.18:0/1054636278 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0a2af0 con 0x55b26efa5d30 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5ea7fc640 1 -- 172.21.15.18:0/1054636278 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc5ec098700 con 0x55b26efa5d30 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc60bc0a740 1 -- 172.21.15.18:0/1054636278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b26eee6f40 con 0x55b26efa5d30 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc5ea7fc640 1 -- 172.21.15.18:0/1054636278 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5ec09e120 con 0x55b26efa5d30 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc60bc0a740 10 client.16533 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.872+0000 7fc60bc0a740 1 -- 172.21.15.18:0/1054636278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b26ef0b320 con 0x55b26efa5d30 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.873+0000 7fc5ea7fc640 1 -- 172.21.15.18:0/1054636278 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b26eee6f40 con 0x55b26efa5d30 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.873+0000 7fc5ea7fc640 1 -- 172.21.15.18:0/1054636278 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc5ec0a2660 con 0x55b26efa5d30 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.873+0000 7fc60bc0a740 10 client.16533 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.873+0000 7fc60bc0a740 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.873+0000 7fc60bc0a740 20 client.16533 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.873+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b26efbee10 0x55b26efc1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.873+0000 7fc60bc0a740 4 client.16533 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.873+0000 7fc60bc0a740 1 -- 172.21.15.18:0/1054636278 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b26ef03310 con 0x55b26efbee10 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.873+0000 7fc602d9d640 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b26efbee10 0x55b26efc1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.873+0000 7fc602d9d640 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b26efbee10 0x55b26efc1230 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a36f9b4d5b3ebc server_cookie=ec17189790466699 in_seq=0 out_seq=0 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.873+0000 7fc5ea7fc640 10 client.16533 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc5ea7fc640 1 -- 172.21.15.18:0/1054636278 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b26ef03310 con 0x55b26efbee10 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc5ea7fc640 10 client.16533 handle_command_reply: tid=0 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc60bc0a740 1 client.16533 shutdown 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc60bc0a740 1 -- 172.21.15.18:0/1054636278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 msgr2=0x55b26efabd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 0x55b26efabd60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec096860 tx=0x7fc5ec004780 comp rx=0 tx=0).stop 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc60bc0a740 1 -- 172.21.15.18:0/1054636278 shutdown_connections 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b26efbee10 0x55b26efc1230 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efae8b0 0x55b26efac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 0x55b26efabd60 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc5ea7fc640 0 client.16533 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/1054636278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efa5380 0x55b26efab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc60bc0a740 1 -- 172.21.15.18:0/1054636278 >> 172.21.15.18:0/1054636278 conn(0x55b26ef9f840 msgr2=0x55b26efa17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc60bc0a740 1 -- 172.21.15.18:0/1054636278 shutdown_connections 2024-02-17T12:52:27.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc60bc0a740 1 -- 172.21.15.18:0/1054636278 wait complete. 2024-02-17T12:52:27.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.874+0000 7fc60bc0a740 20 client.16533 trim_cache size 0 max 16384 2024-02-17T12:52:27.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc60bc0a740 1 Processor -- start 2024-02-17T12:52:27.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc60bc0a740 1 -- start start 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc60bc0a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 0x55b26efac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc60bc0a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 0x55b26efb58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc60bc0a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efae8b0 0x55b26efb5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc60bc0a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b26efbb900 con 0x55b26efa5d30 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc60bc0a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b26ef9f370 con 0x55b26efa5380 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc60bc0a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b26ef99dc0 con 0x55b26efae8b0 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc5fbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 0x55b26efb58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc602d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 0x55b26efac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc60359e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efae8b0 0x55b26efb5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc602d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 0x55b26efac0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:57260/0 (socket says 172.21.15.18:57260) 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc5fbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 0x55b26efb58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34504/0 (socket says 172.21.15.18:34504) 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.890+0000 7fc602d9d640 1 -- 172.21.15.18:0/2368950858 learned_addr learned my addr 172.21.15.18:0/2368950858 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.891+0000 7fc602d9d640 1 -- 172.21.15.18:0/2368950858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efae8b0 msgr2=0x55b26efb5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:27.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.891+0000 7fc602d9d640 1 --2- 172.21.15.18:0/2368950858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efae8b0 0x55b26efb5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.891+0000 7fc602d9d640 1 -- 172.21.15.18:0/2368950858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 msgr2=0x55b26efb58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:27.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.891+0000 7fc602d9d640 1 --2- 172.21.15.18:0/2368950858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 0x55b26efb58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.891+0000 7fc602d9d640 1 -- 172.21.15.18:0/2368950858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b26eea7c00 con 0x55b26efa5380 2024-02-17T12:52:27.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.891+0000 7fc602d9d640 1 --2- 172.21.15.18:0/2368950858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 0x55b26efac0c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc5f401b040 tx=0x7fc5f40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=addc2b296bbd7606 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:27.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.891+0000 7fc5f8ff9640 1 -- 172.21.15.18:0/2368950858 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4005ab0 con 0x55b26efa5380 2024-02-17T12:52:27.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.891+0000 7fc5f8ff9640 1 -- 172.21.15.18:0/2368950858 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5f400f4e0 con 0x55b26efa5380 2024-02-17T12:52:27.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.891+0000 7fc5f8ff9640 1 -- 172.21.15.18:0/2368950858 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f402a520 con 0x55b26efa5380 2024-02-17T12:52:27.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.892+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2368950858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 msgr2=0x55b26efac0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:27.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.892+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/2368950858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 0x55b26efac0c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc5f401b040 tx=0x7fc5f40118d0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.892+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2368950858 shutdown_connections 2024-02-17T12:52:27.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.892+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/2368950858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efae8b0 0x55b26efb5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.892+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/2368950858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efa5d30 0x55b26efb58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.892+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/2368950858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 0x55b26efac0c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.892+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2368950858 >> 172.21.15.18:0/2368950858 conn(0x55b26ef9f840 msgr2=0x55b26ef9e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:27.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.892+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2368950858 shutdown_connections 2024-02-17T12:52:27.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.892+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2368950858 wait complete. 2024-02-17T12:52:27.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.892+0000 7fc60bc0a740 1 -- start start 2024-02-17T12:52:27.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.892+0000 7fc60bc0a740 1 Processor -- start 2024-02-17T12:52:27.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60bc0a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 0x55b26f00f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60bc0a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efa5d30 0x55b26f00fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60bc0a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efae8b0 0x55b26efb4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60bc0a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b26efb5200 con 0x55b26efae8b0 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60bc0a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b26efb5370 con 0x55b26efa5380 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60bc0a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b26efb5530 con 0x55b26efa5d30 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60bc0a740 10 client.0 _ll_register_callbacks cb 0x7fc5f4009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efae8b0 0x55b26efb4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc602d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 0x55b26f00f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efae8b0 0x55b26efb4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34506/0 (socket says 172.21.15.18:34506) 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60359e640 1 -- 172.21.15.18:0/2556234429 learned_addr learned my addr 172.21.15.18:0/2556234429 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc5fbfff640 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efa5d30 0x55b26f00fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60359e640 1 -- 172.21.15.18:0/2556234429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efa5d30 msgr2=0x55b26f00fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60359e640 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efa5d30 0x55b26f00fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60359e640 1 -- 172.21.15.18:0/2556234429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 msgr2=0x55b26f00f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60359e640 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 0x55b26f00f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60359e640 1 -- 172.21.15.18:0/2556234429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b26eecf0b0 con 0x55b26efae8b0 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc602d9d640 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 0x55b26f00f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60359e640 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efae8b0 0x55b26efb4c40 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc02c700 tx=0x7fc5fc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3bb14ee0fd6cde server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc5ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc5ea7fc640 1 -- 172.21.15.18:0/2556234429 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc032a40 con 0x55b26efae8b0 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc5ea7fc640 1 -- 172.21.15.18:0/2556234429 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc5fc003970 con 0x55b26efae8b0 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.893+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2556234429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b26ef32900 con 0x55b26efae8b0 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.894+0000 7fc5ea7fc640 1 -- 172.21.15.18:0/2556234429 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5fc003b30 con 0x55b26efae8b0 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.894+0000 7fc60bc0a740 10 client.16539 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.894+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2556234429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b26eecd910 con 0x55b26efae8b0 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.894+0000 7fc5ea7fc640 1 -- 172.21.15.18:0/2556234429 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b26ef32900 con 0x55b26efae8b0 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.894+0000 7fc5ea7fc640 1 -- 172.21.15.18:0/2556234429 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc5fc03e040 con 0x55b26efae8b0 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.894+0000 7fc60bc0a740 10 client.16539 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:27.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.894+0000 7fc60bc0a740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.894+0000 7fc60bc0a740 20 client.16539 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.894+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b26f017870 0x55b26f019c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.894+0000 7fc60bc0a740 4 client.16539 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.894+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2556234429 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b26ec0c860 con 0x55b26f017870 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.894+0000 7fc602d9d640 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b26f017870 0x55b26f019c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.895+0000 7fc602d9d640 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b26f017870 0x55b26f019c90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e8f37535ee4633 server_cookie=77502ffb05bd122f in_seq=0 out_seq=0 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.895+0000 7fc5ea7fc640 10 client.16539 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc5ea7fc640 1 -- 172.21.15.18:0/2556234429 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b26ec0c860 con 0x55b26f017870 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc5ea7fc640 10 client.16539 handle_command_reply: tid=0 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc60bc0a740 1 client.16539 shutdown 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2556234429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efae8b0 msgr2=0x55b26efb4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efae8b0 0x55b26efb4c40 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc02c700 tx=0x7fc5fc0095c0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2556234429 shutdown_connections 2024-02-17T12:52:27.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b26f017870 0x55b26f019c90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b26efae8b0 0x55b26efb4c40 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b26efa5d30 0x55b26f00fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc5ea7fc640 0 client.16539 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:27.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc60bc0a740 1 --2- 172.21.15.18:0/2556234429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b26efa5380 0x55b26f00f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2556234429 >> 172.21.15.18:0/2556234429 conn(0x55b26ef9f840 msgr2=0x55b26ef9e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:27.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2556234429 shutdown_connections 2024-02-17T12:52:27.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc60bc0a740 1 -- 172.21.15.18:0/2556234429 wait complete. 2024-02-17T12:52:27.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc60bc0a740 20 client.16539 trim_cache size 0 max 16384 2024-02-17T12:52:27.894 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:52:27.895 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:52:27.895 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "4ecd1416-1c94-4b1c-be5b-0cebd95fe661", 2024-02-17T12:52:27.895 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:52:27.895 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:52:27.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc609039640 1 -- 172.21.15.18:0/1433395462 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5f00a2230 msgr2=0x7fc5f00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:27.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc609039640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5f00a2230 0x7fc5f00a46f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc033000 tx=0x7fc5fc02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc609039640 1 -- 172.21.15.18:0/1433395462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc604105330 msgr2=0x7fc60419d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:27.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.896+0000 7fc609039640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc604105330 0x7fc60419d920 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00d740 tx=0x7fc5ec00dc00 comp rx=0 tx=0).stop 2024-02-17T12:52:27.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.897+0000 7fc609039640 1 -- 172.21.15.18:0/1433395462 shutdown_connections 2024-02-17T12:52:27.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.897+0000 7fc609039640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5f00a2230 0x7fc5f00a46f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.897+0000 7fc609039640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc604197590 0x7fc604197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.897+0000 7fc609039640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc604105330 0x7fc60419d920 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.897+0000 7fc609039640 1 --2- 172.21.15.18:0/1433395462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc604100260 0x7fc60419d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:27.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.897+0000 7fc609039640 1 -- 172.21.15.18:0/1433395462 >> 172.21.15.18:0/1433395462 conn(0x7fc604077180 msgr2=0x7fc6040fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:27.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.897+0000 7fc609039640 1 -- 172.21.15.18:0/1433395462 shutdown_connections 2024-02-17T12:52:27.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:27.897+0000 7fc609039640 1 -- 172.21.15.18:0/1433395462 wait complete. 2024-02-17T12:52:27.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:52:27.899 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:52:28.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.017+0000 7f3247b48640 1 -- 172.21.15.18:0/2863940076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100c20 msgr2=0x7f3240104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:28.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.017+0000 7f3247b48640 1 --2- 172.21.15.18:0/2863940076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100c20 0x7f3240104c60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3230009930 tx=0x7f323002bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:28.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.017+0000 7f3247b48640 1 -- 172.21.15.18:0/2863940076 shutdown_connections 2024-02-17T12:52:28.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.017+0000 7f3247b48640 1 --2- 172.21.15.18:0/2863940076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3240105340 0x7f324010cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.017+0000 7f3247b48640 1 --2- 172.21.15.18:0/2863940076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100c20 0x7f3240104c60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.017+0000 7f3247b48640 1 --2- 172.21.15.18:0/2863940076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3240100270 0x7f3240100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.017+0000 7f3247b48640 1 -- 172.21.15.18:0/2863940076 >> 172.21.15.18:0/2863940076 conn(0x7f3240077230 msgr2=0x7f3240077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.017+0000 7f3247b48640 1 -- 172.21.15.18:0/2863940076 shutdown_connections 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.017+0000 7f3247b48640 1 -- 172.21.15.18:0/2863940076 wait complete. 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.017+0000 7f3247b48640 1 Processor -- start 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.017+0000 7f3247b48640 1 -- start start 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f3247b48640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100270 0x7f3240196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f3247b48640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3240105340 0x7f3240196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f3247b48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324019af20 0x7f324019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f3247b48640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324019b910 con 0x7f3240100270 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f3247b48640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324019ba80 con 0x7f324019af20 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f3247b48640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324019bbf0 con 0x7f3240105340 2024-02-17T12:52:28.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f32458e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100270 0x7f3240196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f32458e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100270 0x7f3240196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34520/0 (socket says 172.21.15.18:34520) 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f32458e4640 1 -- 172.21.15.18:0/2593976728 learned_addr learned my addr 172.21.15.18:0/2593976728 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f32450e3640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3240105340 0x7f3240196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f32460e5640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324019af20 0x7f324019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f32458e4640 1 -- 172.21.15.18:0/2593976728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3240105340 msgr2=0x7f3240196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f32458e4640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3240105340 0x7f3240196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f32458e4640 1 -- 172.21.15.18:0/2593976728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324019af20 msgr2=0x7f324019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f32458e4640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324019af20 0x7f324019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f32458e4640 1 -- 172.21.15.18:0/2593976728 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3230009560 con 0x7f3240100270 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f32458e4640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100270 0x7f3240196660 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f323c00e9f0 tx=0x7f323c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f3236ffd640 1 -- 172.21.15.18:0/2593976728 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c009700 con 0x7f3240100270 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f3236ffd640 1 -- 172.21.15.18:0/2593976728 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f323c00cd80 con 0x7f3240100270 2024-02-17T12:52:28.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f3247b48640 1 -- 172.21.15.18:0/2593976728 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32401a31d0 con 0x7f3240100270 2024-02-17T12:52:28.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f3236ffd640 1 -- 172.21.15.18:0/2593976728 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c018920 con 0x7f3240100270 2024-02-17T12:52:28.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.018+0000 7f3247b48640 1 -- 172.21.15.18:0/2593976728 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32401a3650 con 0x7f3240100270 2024-02-17T12:52:28.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.021+0000 7f3236ffd640 1 -- 172.21.15.18:0/2593976728 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f323c018a80 con 0x7f3240100270 2024-02-17T12:52:28.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.021+0000 7f324a7f8740 1 Processor -- start 2024-02-17T12:52:28.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.021+0000 7f324a7f8740 1 -- start start 2024-02-17T12:52:28.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f324a7f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83882b0 0x5644f8388690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f324a7f8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f8388bd0 0x5644f8391270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f324a7f8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83917b0 0x5644f8393ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f324a7f8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f82f7280 con 0x5644f83917b0 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f324a7f8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f82f47b0 con 0x5644f8388bd0 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f324a7f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f82eff90 con 0x5644f83882b0 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32460e5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83882b0 0x5644f8388690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32458e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f8388bd0 0x5644f8391270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32450e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83917b0 0x5644f8393ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32458e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f8388bd0 0x5644f8391270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:57294/0 (socket says 172.21.15.18:57294) 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32458e4640 1 -- 172.21.15.18:0/1839395462 learned_addr learned my addr 172.21.15.18:0/1839395462 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f3236ffd640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f322c0a2230 0x7f322c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32460e5640 1 -- 172.21.15.18:0/1839395462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f8388bd0 msgr2=0x5644f8391270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32460e5640 1 --2- 172.21.15.18:0/1839395462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f8388bd0 0x5644f8391270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32460e5640 1 -- 172.21.15.18:0/1839395462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83917b0 msgr2=0x5644f8393ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32460e5640 1 --2- 172.21.15.18:0/1839395462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83917b0 0x5644f8393ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32460e5640 1 -- 172.21.15.18:0/1839395462 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644f82ae570 con 0x5644f83882b0 2024-02-17T12:52:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32450e3640 1 --2- 172.21.15.18:0/1839395462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83917b0 0x5644f8393ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32458e4640 1 --2- 172.21.15.18:0/1839395462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f8388bd0 0x5644f8391270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f3236ffd640 1 -- 172.21.15.18:0/2593976728 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f323c014070 con 0x7f3240100270 2024-02-17T12:52:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32460e5640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f322c0a2230 0x7f322c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.022+0000 7f32460e5640 1 --2- 172.21.15.18:0/1839395462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83882b0 0x5644f8388690 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3238009f00 tx=0x7f3238011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dba900f6fa5c1e34 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.023+0000 7f322bfff640 1 -- 172.21.15.18:0/1839395462 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238020030 con 0x5644f83882b0 2024-02-17T12:52:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.023+0000 7f322bfff640 1 -- 172.21.15.18:0/1839395462 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f323801be30 con 0x5644f83882b0 2024-02-17T12:52:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.023+0000 7f322bfff640 1 -- 172.21.15.18:0/1839395462 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238019970 con 0x5644f83882b0 2024-02-17T12:52:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.024+0000 7f324a7f8740 1 -- 172.21.15.18:0/1839395462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83882b0 msgr2=0x5644f8388690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.024+0000 7f32460e5640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f322c0a2230 0x7f322c0a46f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3238020000 tx=0x7f3238021000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.024+0000 7f324a7f8740 1 --2- 172.21.15.18:0/1839395462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83882b0 0x5644f8388690 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3238009f00 tx=0x7f3238011810 comp rx=0 tx=0).stop 2024-02-17T12:52:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.024+0000 7f324a7f8740 1 -- 172.21.15.18:0/1839395462 shutdown_connections 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.024+0000 7f324a7f8740 1 --2- 172.21.15.18:0/1839395462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83917b0 0x5644f8393ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.024+0000 7f324a7f8740 1 --2- 172.21.15.18:0/1839395462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f8388bd0 0x5644f8391270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.024+0000 7f324a7f8740 1 --2- 172.21.15.18:0/1839395462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83882b0 0x5644f8388690 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.024+0000 7f324a7f8740 1 -- 172.21.15.18:0/1839395462 >> 172.21.15.18:0/1839395462 conn(0x5644f8382990 msgr2=0x5644f8382d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.024+0000 7f324a7f8740 1 -- 172.21.15.18:0/1839395462 shutdown_connections 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.024+0000 7f324a7f8740 1 -- 172.21.15.18:0/1839395462 wait complete. 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.024+0000 7f324a7f8740 1 -- start start 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.024+0000 7f324a7f8740 1 Processor -- start 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f324a7f8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 0x5644f8391240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f324a7f8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f8388bd0 0x5644f838e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f324a7f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83917b0 0x5644f838ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f324a7f8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f7fc3430 con 0x5644f83882b0 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f324a7f8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f7f7fe00 con 0x5644f8388bd0 2024-02-17T12:52:28.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f324a7f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f8274fd0 con 0x5644f83917b0 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f324a7f8740 10 client.0 _ll_register_callbacks cb 0x5644f82cafd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32458e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 0x5644f8391240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32458e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 0x5644f8391240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34544/0 (socket says 172.21.15.18:34544) 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32458e4640 1 -- 172.21.15.18:0/2295337954 learned_addr learned my addr 172.21.15.18:0/2295337954 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32460e5640 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83917b0 0x5644f838ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32450e3640 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f8388bd0 0x5644f838e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32458e4640 1 -- 172.21.15.18:0/2295337954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83917b0 msgr2=0x5644f838ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32458e4640 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83917b0 0x5644f838ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32458e4640 1 -- 172.21.15.18:0/2295337954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f8388bd0 msgr2=0x5644f838e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32458e4640 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f8388bd0 0x5644f838e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32458e4640 1 -- 172.21.15.18:0/2295337954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3238009be0 con 0x5644f83882b0 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32458e4640 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 0x5644f8391240 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f323c09ae80 tx=0x7f323c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7289dae9aa5ef8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32297fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32297fa640 1 -- 172.21.15.18:0/2295337954 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c09aa30 con 0x5644f83882b0 2024-02-17T12:52:28.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f32297fa640 1 -- 172.21.15.18:0/2295337954 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f323c09aef0 con 0x5644f83882b0 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.025+0000 7f324a7f8740 1 -- 172.21.15.18:0/2295337954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644f7fef7e0 con 0x5644f83882b0 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.026+0000 7f32297fa640 1 -- 172.21.15.18:0/2295337954 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f323c09a850 con 0x5644f83882b0 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.026+0000 7f324a7f8740 10 client.16551 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.026+0000 7f324a7f8740 1 -- 172.21.15.18:0/2295337954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5644f82ee610 con 0x5644f83882b0 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.026+0000 7f32297fa640 1 -- 172.21.15.18:0/2295337954 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5644f7fef7e0 con 0x5644f83882b0 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.026+0000 7f32297fa640 1 -- 172.21.15.18:0/2295337954 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f323c099db0 con 0x5644f83882b0 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.026+0000 7f324a7f8740 10 client.16551 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.026+0000 7f324a7f8740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.026+0000 7f324a7f8740 20 client.16551 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.026+0000 7f324a7f8740 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5644f83a1a30 0x5644f83a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.026+0000 7f324a7f8740 4 client.16551 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.026+0000 7f324a7f8740 1 -- 172.21.15.18:0/2295337954 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644f802c720 con 0x5644f83a1a30 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.026+0000 7f32450e3640 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5644f83a1a30 0x5644f83a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f32450e3640 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5644f83a1a30 0x5644f83a3e50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd74d3530beba02a server_cookie=3b94d9a164654d99 in_seq=0 out_seq=0 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f32297fa640 10 client.16551 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f32297fa640 1 -- 172.21.15.18:0/2295337954 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644f802c720 con 0x5644f83a1a30 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f32297fa640 10 client.16551 handle_command_reply: tid=0 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f324a7f8740 1 client.16551 shutdown 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f324a7f8740 1 -- 172.21.15.18:0/2295337954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 msgr2=0x5644f8391240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f324a7f8740 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 0x5644f8391240 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f323c09ae80 tx=0x7f323c009310 comp rx=0 tx=0).stop 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f324a7f8740 1 -- 172.21.15.18:0/2295337954 shutdown_connections 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f324a7f8740 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5644f83a1a30 0x5644f83a3e50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f324a7f8740 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83917b0 0x5644f838ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f324a7f8740 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f8388bd0 0x5644f838e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f324a7f8740 1 --2- 172.21.15.18:0/2295337954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 0x5644f8391240 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f32297fa640 0 client.16551 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.027+0000 7f324a7f8740 1 -- 172.21.15.18:0/2295337954 >> 172.21.15.18:0/2295337954 conn(0x5644f8382990 msgr2=0x5644f8384730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:28.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.028+0000 7f324a7f8740 1 -- 172.21.15.18:0/2295337954 shutdown_connections 2024-02-17T12:52:28.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.028+0000 7f324a7f8740 1 -- 172.21.15.18:0/2295337954 wait complete. 2024-02-17T12:52:28.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.028+0000 7f324a7f8740 20 client.16551 trim_cache size 0 max 16384 2024-02-17T12:52:28.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.043+0000 7f324a7f8740 1 Processor -- start 2024-02-17T12:52:28.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.043+0000 7f324a7f8740 1 -- start start 2024-02-17T12:52:28.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f324a7f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83882b0 0x5644f8390d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f324a7f8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f8388bd0 0x5644f8391240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f324a7f8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f83917b0 0x5644f8398660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f324a7f8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f8393fa0 con 0x5644f8388bd0 2024-02-17T12:52:28.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f324a7f8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f837e120 con 0x5644f83917b0 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f324a7f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f838f590 con 0x5644f83882b0 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f32450e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f8388bd0 0x5644f8391240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f32450e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f8388bd0 0x5644f8391240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34546/0 (socket says 172.21.15.18:34546) 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f32450e3640 1 -- 172.21.15.18:0/3466782242 learned_addr learned my addr 172.21.15.18:0/3466782242 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f32460e5640 1 --2- 172.21.15.18:0/3466782242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f83917b0 0x5644f8398660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f32458e4640 1 --2- 172.21.15.18:0/3466782242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83882b0 0x5644f8390d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f32450e3640 1 -- 172.21.15.18:0/3466782242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83882b0 msgr2=0x5644f8390d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f32450e3640 1 --2- 172.21.15.18:0/3466782242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83882b0 0x5644f8390d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f32450e3640 1 -- 172.21.15.18:0/3466782242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f83917b0 msgr2=0x5644f8398660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f32450e3640 1 --2- 172.21.15.18:0/3466782242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f83917b0 0x5644f8398660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f32450e3640 1 -- 172.21.15.18:0/3466782242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644f828aac0 con 0x5644f8388bd0 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f32460e5640 1 --2- 172.21.15.18:0/3466782242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f83917b0 0x5644f8398660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:28.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f32450e3640 1 --2- 172.21.15.18:0/3466782242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f8388bd0 0x5644f8391240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3230000c00 tx=0x7f3230005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec652e101bed1e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f322b7fe640 1 -- 172.21.15.18:0/3466782242 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230004a60 con 0x5644f8388bd0 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f322b7fe640 1 -- 172.21.15.18:0/3466782242 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3230049be0 con 0x5644f8388bd0 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.044+0000 7f322b7fe640 1 -- 172.21.15.18:0/3466782242 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32300493a0 con 0x5644f8388bd0 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.045+0000 7f324a7f8740 1 -- 172.21.15.18:0/3466782242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f8388bd0 msgr2=0x5644f8391240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.045+0000 7f324a7f8740 1 --2- 172.21.15.18:0/3466782242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f8388bd0 0x5644f8391240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3230000c00 tx=0x7f3230005fd0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.045+0000 7f324a7f8740 1 -- 172.21.15.18:0/3466782242 shutdown_connections 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.045+0000 7f324a7f8740 1 --2- 172.21.15.18:0/3466782242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f83917b0 0x5644f8398660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.045+0000 7f324a7f8740 1 --2- 172.21.15.18:0/3466782242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f8388bd0 0x5644f8391240 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.045+0000 7f324a7f8740 1 --2- 172.21.15.18:0/3466782242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f83882b0 0x5644f8390d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.045+0000 7f324a7f8740 1 -- 172.21.15.18:0/3466782242 >> 172.21.15.18:0/3466782242 conn(0x5644f8382990 msgr2=0x5644f838cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.045+0000 7f324a7f8740 1 -- 172.21.15.18:0/3466782242 shutdown_connections 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.045+0000 7f324a7f8740 1 -- 172.21.15.18:0/3466782242 wait complete. 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.045+0000 7f324a7f8740 1 -- start start 2024-02-17T12:52:28.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f324a7f8740 1 Processor -- start 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f324a7f8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 0x5644f8397340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f324a7f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f8388bd0 0x5644f8397880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f324a7f8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f83917b0 0x5644f8397de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f324a7f8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f83fc4f0 con 0x5644f83882b0 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f324a7f8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f83fc660 con 0x5644f83917b0 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f324a7f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f83fc820 con 0x5644f8388bd0 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f32458e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 0x5644f8397340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f324a7f8740 10 client.0 _ll_register_callbacks cb 0x7f3238009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f32458e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 0x5644f8397340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34558/0 (socket says 172.21.15.18:34558) 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f32458e4640 1 -- 172.21.15.18:0/4083627157 learned_addr learned my addr 172.21.15.18:0/4083627157 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f32450e3640 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f8388bd0 0x5644f8397880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f32460e5640 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f83917b0 0x5644f8397de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f32458e4640 1 -- 172.21.15.18:0/4083627157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f8388bd0 msgr2=0x5644f8397880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:28.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f32458e4640 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f8388bd0 0x5644f8397880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f32458e4640 1 -- 172.21.15.18:0/4083627157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f83917b0 msgr2=0x5644f8397de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f32458e4640 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f83917b0 0x5644f8397de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.046+0000 7f32458e4640 1 -- 172.21.15.18:0/4083627157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644f82ae570 con 0x5644f83882b0 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f32458e4640 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 0x5644f8397340 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f323c00c6a0 tx=0x7f323c099fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b485c1b2a0e7927 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f32297fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f32297fa640 1 -- 172.21.15.18:0/4083627157 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c09a550 con 0x5644f83882b0 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f32297fa640 1 -- 172.21.15.18:0/4083627157 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f323c09aef0 con 0x5644f83882b0 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f324a7f8740 1 -- 172.21.15.18:0/4083627157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f323801be30 con 0x5644f83882b0 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f32297fa640 1 -- 172.21.15.18:0/4083627157 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f323c09a850 con 0x5644f83882b0 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f324a7f8740 10 client.16557 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f324a7f8740 1 -- 172.21.15.18:0/4083627157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f3230030070 con 0x5644f83882b0 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f32297fa640 1 -- 172.21.15.18:0/4083627157 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f323801be30 con 0x5644f83882b0 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f32297fa640 1 -- 172.21.15.18:0/4083627157 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f323c09feb0 con 0x5644f83882b0 2024-02-17T12:52:28.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f324a7f8740 10 client.16557 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f324a7f8740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f324a7f8740 20 client.16557 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f324a7f8740 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5644f83fdcb0 0x5644f84000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f324a7f8740 4 client.16557 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f324a7f8740 1 -- 172.21.15.18:0/4083627157 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5644f807b890 con 0x5644f83fdcb0 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.047+0000 7f32450e3640 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5644f83fdcb0 0x5644f84000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f32297fa640 10 client.16557 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f32450e3640 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5644f83fdcb0 0x5644f84000d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a215022f9a35f29 server_cookie=f3944b354b5de9f4 in_seq=0 out_seq=0 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f32297fa640 1 -- 172.21.15.18:0/4083627157 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5644f807b890 con 0x5644f83fdcb0 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f32297fa640 10 client.16557 handle_command_reply: tid=0 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f324a7f8740 1 client.16557 shutdown 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f324a7f8740 1 -- 172.21.15.18:0/4083627157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 msgr2=0x5644f8397340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f324a7f8740 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 0x5644f8397340 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f323c00c6a0 tx=0x7f323c099fc0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f324a7f8740 1 -- 172.21.15.18:0/4083627157 shutdown_connections 2024-02-17T12:52:28.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f324a7f8740 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5644f83fdcb0 0x5644f84000d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f324a7f8740 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644f83917b0 0x5644f8397de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f324a7f8740 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5644f8388bd0 0x5644f8397880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f324a7f8740 1 --2- 172.21.15.18:0/4083627157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644f83882b0 0x5644f8397340 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f32297fa640 0 client.16557 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:28.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f324a7f8740 1 -- 172.21.15.18:0/4083627157 >> 172.21.15.18:0/4083627157 conn(0x5644f8382990 msgr2=0x5644f838db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:28.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f324a7f8740 1 -- 172.21.15.18:0/4083627157 shutdown_connections 2024-02-17T12:52:28.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f324a7f8740 1 -- 172.21.15.18:0/4083627157 wait complete. 2024-02-17T12:52:28.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.048+0000 7f324a7f8740 20 client.16557 trim_cache size 0 max 16384 2024-02-17T12:52:28.046 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:52:28.046 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:52:28.047 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:52:28.047 INFO:teuthology.orchestra.run.smithi018.stdout: "4ecd1416-1c94-4b1c-be5b-0cebd95fe661": { 2024-02-17T12:52:28.047 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:52:28.047 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "4ecd1416-1c94-4b1c-be5b-0cebd95fe661", 2024-02-17T12:52:28.047 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:52:28.047 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:52:28.047 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:52:28.047 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:52:28.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.049+0000 7f3247b48640 1 -- 172.21.15.18:0/2593976728 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f322c0a2230 msgr2=0x7f322c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:28.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.049+0000 7f3247b48640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f322c0a2230 0x7f322c0a46f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3238020000 tx=0x7f3238021000 comp rx=0 tx=0).stop 2024-02-17T12:52:28.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.049+0000 7f3247b48640 1 -- 172.21.15.18:0/2593976728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100270 msgr2=0x7f3240196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:28.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.049+0000 7f3247b48640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100270 0x7f3240196660 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f323c00e9f0 tx=0x7f323c00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.049+0000 7f3247b48640 1 -- 172.21.15.18:0/2593976728 shutdown_connections 2024-02-17T12:52:28.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.049+0000 7f3247b48640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f322c0a2230 0x7f322c0a46f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.049+0000 7f3247b48640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324019af20 0x7f324019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.049+0000 7f3247b48640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3240105340 0x7f3240196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.049+0000 7f3247b48640 1 --2- 172.21.15.18:0/2593976728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100270 0x7f3240196660 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:28.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.049+0000 7f3247b48640 1 -- 172.21.15.18:0/2593976728 >> 172.21.15.18:0/2593976728 conn(0x7f3240077230 msgr2=0x7f32400fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:28.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.049+0000 7f3247b48640 1 -- 172.21.15.18:0/2593976728 shutdown_connections 2024-02-17T12:52:28.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:28.049+0000 7f3247b48640 1 -- 172.21.15.18:0/2593976728 wait complete. 2024-02-17T12:52:28.051 INFO:tasks.cephfs.filesystem:scrub status for tag:4ecd1416-1c94-4b1c-be5b-0cebd95fe661 - {'path': '/', 'tag': '4ecd1416-1c94-4b1c-be5b-0cebd95fe661', 'options': 'recursive,force'} 2024-02-17T12:52:28.265 INFO:tasks.workunit.client.0.smithi018.stdout: 1 84822 4.60 MB/sec warmup 103 sec latency 13.995 ms 2024-02-17T12:52:28.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:28 smithi119 ceph-mon[36188]: pgmap v80: 97 pgs: 97 active+clean; 43 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 844 KiB/s wr, 73 op/s 2024-02-17T12:52:28.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:28 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:52:28.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:28 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:28.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:28 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:52:28.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:28 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:52:28.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:28 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:28.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:28 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:52:28.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:28 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:28.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:28 smithi146 ceph-mon[36454]: pgmap v80: 97 pgs: 97 active+clean; 43 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 844 KiB/s wr, 73 op/s 2024-02-17T12:52:28.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:28 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:52:28.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:28 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:28.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:28 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:52:28.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:28 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:52:28.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:28 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:28.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:28 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:52:28.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:28 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:28.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:28 smithi018 ceph-mon[32625]: pgmap v80: 97 pgs: 97 active+clean; 43 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 844 KiB/s wr, 73 op/s 2024-02-17T12:52:28.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:28 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:52:28.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:28 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:28.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:28 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:52:28.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:28 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:52:28.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:28 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:28.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:28 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:52:28.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:28 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.265 INFO:tasks.workunit.client.0.smithi018.stdout: 1 85843 4.62 MB/sec warmup 104 sec latency 313.701 ms 2024-02-17T12:52:29.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:29 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:52:29.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:29 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:52:29.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:29 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:52:29.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:29 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:29 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:29 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:29 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:52:29.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:29 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:29 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:52:29.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:29 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:29 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:29 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:52:29.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:29 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:52:29.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:29 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:52:29.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:29 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:29 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:29 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:29 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:52:29.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:29 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:29 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:52:29.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:29 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:29 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:29 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:52:29.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:29 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:52:29.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:29 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:52:29.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:29 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:29 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:29 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:29 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:52:29.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:29 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:29 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:52:29.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:29 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:29.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:29 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:52:30.265 INFO:tasks.workunit.client.0.smithi018.stdout: 1 85843 4.57 MB/sec warmup 105 sec latency 1313.813 ms 2024-02-17T12:52:30.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:30 smithi119 ceph-mon[36188]: pgmap v81: 97 pgs: 97 active+clean; 42 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-02-17T12:52:30.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:30 smithi146 ceph-mon[36454]: pgmap v81: 97 pgs: 97 active+clean; 42 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-02-17T12:52:30.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:30 smithi018 ceph-mon[32625]: pgmap v81: 97 pgs: 97 active+clean; 42 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-02-17T12:52:31.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 85967 4.53 MB/sec warmup 106 sec latency 1876.584 ms 2024-02-17T12:52:32.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 85967 4.49 MB/sec warmup 107 sec latency 1349.808 ms 2024-02-17T12:52:32.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:32 smithi119 ceph-mon[36188]: pgmap v82: 97 pgs: 97 active+clean; 42 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-02-17T12:52:32.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:32 smithi146 ceph-mon[36454]: pgmap v82: 97 pgs: 97 active+clean; 42 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-02-17T12:52:32.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:32 smithi018 ceph-mon[32625]: pgmap v82: 97 pgs: 97 active+clean; 42 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-02-17T12:52:33.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 85967 4.45 MB/sec warmup 108 sec latency 2349.948 ms 2024-02-17T12:52:34.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 85967 4.41 MB/sec warmup 109 sec latency 3350.056 ms 2024-02-17T12:52:34.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:34 smithi119 ceph-mon[36188]: pgmap v83: 97 pgs: 97 active+clean; 42 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-02-17T12:52:34.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:34 smithi146 ceph-mon[36454]: pgmap v83: 97 pgs: 97 active+clean; 42 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-02-17T12:52:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:34 smithi018 ceph-mon[32625]: pgmap v83: 97 pgs: 97 active+clean; 42 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-02-17T12:52:35.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 85967 4.37 MB/sec warmup 110 sec latency 4350.169 ms 2024-02-17T12:52:35.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:35 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:52:35.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:35 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:52:35.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:35 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:52:36.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 87217 4.37 MB/sec warmup 111 sec latency 4918.103 ms 2024-02-17T12:52:36.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:36 smithi119 ceph-mon[36188]: pgmap v84: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-02-17T12:52:36.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:36 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:52:36.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:36 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:52:36.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:36 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:52:36.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:36 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:52:36.601 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:36 smithi146 ceph-mon[36454]: pgmap v84: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-02-17T12:52:36.601 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:36 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:52:36.601 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:36 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:52:36.601 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:36 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:52:36.601 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:36 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:52:36.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:36 smithi018 ceph-mon[32625]: pgmap v84: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-02-17T12:52:36.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:36 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:52:36.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:36 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:52:36.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:36 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:52:36.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:36 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:52:37.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 89165 4.46 MB/sec warmup 112 sec latency 9.209 ms 2024-02-17T12:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 -- 172.21.15.18:0/1326949022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa62c0735b0 msgr2=0x7fa62c073990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 --2- 172.21.15.18:0/1326949022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa62c0735b0 0x7fa62c073990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa628009900 tx=0x7fa62802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 -- 172.21.15.18:0/1326949022 shutdown_connections 2024-02-17T12:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 --2- 172.21.15.18:0/1326949022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa62c10d990 0x7fa62c10fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 --2- 172.21.15.18:0/1326949022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa62c073ed0 0x7fa62c10d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 --2- 172.21.15.18:0/1326949022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa62c0735b0 0x7fa62c073990 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 -- 172.21.15.18:0/1326949022 >> 172.21.15.18:0/1326949022 conn(0x7fa62c0fcc60 msgr2=0x7fa62c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 -- 172.21.15.18:0/1326949022 shutdown_connections 2024-02-17T12:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 -- 172.21.15.18:0/1326949022 wait complete. 2024-02-17T12:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 Processor -- start 2024-02-17T12:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 -- start start 2024-02-17T12:52:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa62c0735b0 0x7fa62c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa62c073ed0 0x7fa62c19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa62c10d990 0x7fa62c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c199f30 con 0x7fa62c0735b0 2024-02-17T12:52:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c19a0a0 con 0x7fa62c10d990 2024-02-17T12:52:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.826+0000 7fa633755640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c19a260 con 0x7fa62c073ed0 2024-02-17T12:52:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa6314f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa62c0735b0 0x7fa62c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa6314f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa62c0735b0 0x7fa62c19f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40912/0 (socket says 172.21.15.18:40912) 2024-02-17T12:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa6314f1640 1 -- 172.21.15.18:0/2228826609 learned_addr learned my addr 172.21.15.18:0/2228826609 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa631cf2640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa62c10d990 0x7fa62c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa630cf0640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa62c073ed0 0x7fa62c19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa6314f1640 1 -- 172.21.15.18:0/2228826609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa62c073ed0 msgr2=0x7fa62c19fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa6314f1640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa62c073ed0 0x7fa62c19fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa6314f1640 1 -- 172.21.15.18:0/2228826609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa62c10d990 msgr2=0x7fa62c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa6314f1640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa62c10d990 0x7fa62c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa6314f1640 1 -- 172.21.15.18:0/2228826609 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa628009560 con 0x7fa62c0735b0 2024-02-17T12:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa6314f1640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa62c0735b0 0x7fa62c19f690 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa628005940 tx=0x7fa62802c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa6227fc640 1 -- 172.21.15.18:0/2228826609 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628004350 con 0x7fa62c0735b0 2024-02-17T12:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa633755640 1 -- 172.21.15.18:0/2228826609 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62c19a4e0 con 0x7fa62c0735b0 2024-02-17T12:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa6227fc640 1 -- 172.21.15.18:0/2228826609 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6280044b0 con 0x7fa62c0735b0 2024-02-17T12:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa6227fc640 1 -- 172.21.15.18:0/2228826609 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628003d30 con 0x7fa62c0735b0 2024-02-17T12:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.827+0000 7fa633755640 1 -- 172.21.15.18:0/2228826609 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62c1a6610 con 0x7fa62c0735b0 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.830+0000 7fa6227fc640 1 -- 172.21.15.18:0/2228826609 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa628004df0 con 0x7fa62c0735b0 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.830+0000 7fa6363fd740 1 Processor -- start 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa6363fd740 1 -- start start 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa6227fc640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa6180a22d0 0x7fa6180a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa6363fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30ddad0 0x5587c30dc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa6363fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30dcd60 0x5587c30dd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa6363fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa6363fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c304b9f0 con 0x5587c30ddad0 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa6363fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c3048f20 con 0x5587c30dcd60 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa6363fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c3044700 con 0x5587c30eeaf0 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa6314f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30ddad0 0x5587c30dc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa631cf2640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa6180a22d0 0x7fa6180a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa630cf0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30dcd60 0x5587c30dd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa6314f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30ddad0 0x5587c30dc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40922/0 (socket says 172.21.15.18:40922) 2024-02-17T12:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa6314f1640 1 -- 172.21.15.18:0/2487536951 learned_addr learned my addr 172.21.15.18:0/2487536951 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa6227fc640 1 -- 172.21.15.18:0/2228826609 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa6280f0640 con 0x7fa62c0735b0 2024-02-17T12:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa631cf2640 1 --2- 172.21.15.18:0/2487536951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa630cf0640 1 -- 172.21.15.18:0/2487536951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 msgr2=0x5587c30f0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa630cf0640 1 --2- 172.21.15.18:0/2487536951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30f0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa630cf0640 1 -- 172.21.15.18:0/2487536951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30ddad0 msgr2=0x5587c30dc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa630cf0640 1 --2- 172.21.15.18:0/2487536951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30ddad0 0x5587c30dc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.831+0000 7fa630cf0640 1 -- 172.21.15.18:0/2487536951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587c2fda7b0 con 0x5587c30dcd60 2024-02-17T12:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.832+0000 7fa631cf2640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa6180a22d0 0x7fa6180a4790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa624009fd0 tx=0x7fa624009de0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.832+0000 7fa630cf0640 1 --2- 172.21.15.18:0/2487536951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30dcd60 0x5587c30dd1e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009bd0 tx=0x7fa61c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a83400213b245e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.832+0000 7fa6177fe640 1 -- 172.21.15.18:0/2487536951 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c004230 con 0x5587c30dcd60 2024-02-17T12:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.832+0000 7fa6177fe640 1 -- 172.21.15.18:0/2487536951 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa61c009e40 con 0x5587c30dcd60 2024-02-17T12:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.832+0000 7fa6177fe640 1 -- 172.21.15.18:0/2487536951 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c004c40 con 0x5587c30dcd60 2024-02-17T12:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.832+0000 7fa6363fd740 1 -- 172.21.15.18:0/2487536951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30dcd60 msgr2=0x5587c30dd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.832+0000 7fa6363fd740 1 --2- 172.21.15.18:0/2487536951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30dcd60 0x5587c30dd1e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009bd0 tx=0x7fa61c011810 comp rx=0 tx=0).stop 2024-02-17T12:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.832+0000 7fa6363fd740 1 -- 172.21.15.18:0/2487536951 shutdown_connections 2024-02-17T12:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.832+0000 7fa6363fd740 1 --2- 172.21.15.18:0/2487536951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.832+0000 7fa6363fd740 1 --2- 172.21.15.18:0/2487536951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30dcd60 0x5587c30dd1e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.832+0000 7fa6363fd740 1 --2- 172.21.15.18:0/2487536951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30ddad0 0x5587c30dc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.832+0000 7fa6363fd740 1 -- 172.21.15.18:0/2487536951 >> 172.21.15.18:0/2487536951 conn(0x5587c30a4550 msgr2=0x5587c30d7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.833+0000 7fa6363fd740 1 -- 172.21.15.18:0/2487536951 shutdown_connections 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.833+0000 7fa6363fd740 1 -- 172.21.15.18:0/2487536951 wait complete. 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.833+0000 7fa6363fd740 1 -- start start 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.833+0000 7fa6363fd740 1 Processor -- start 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.833+0000 7fa6363fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 0x5587c30ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.833+0000 7fa6363fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30ec030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.833+0000 7fa6363fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30ec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.833+0000 7fa6363fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c2fe9bf0 con 0x5587c30dcd60 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.833+0000 7fa6363fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c2d18250 con 0x5587c30ddad0 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.833+0000 7fa6363fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c2fca200 con 0x5587c30eeaf0 2024-02-17T12:52:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.833+0000 7fa6363fd740 10 client.0 _ll_register_callbacks cb 0x5587c3038d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.833+0000 7fa6314f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 0x5587c30ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa6314f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 0x5587c30ebaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40926/0 (socket says 172.21.15.18:40926) 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa6314f1640 1 -- 172.21.15.18:0/2813169860 learned_addr learned my addr 172.21.15.18:0/2813169860 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa630cf0640 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30ec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa631cf2640 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30ec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa6314f1640 1 -- 172.21.15.18:0/2813169860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 msgr2=0x5587c30ec570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa6314f1640 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30ec570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa6314f1640 1 -- 172.21.15.18:0/2813169860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 msgr2=0x5587c30ec030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa6314f1640 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30ec030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa6314f1640 1 -- 172.21.15.18:0/2813169860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c009830 con 0x5587c30dcd60 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa631cf2640 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30ec570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa630cf0640 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30ec030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa6314f1640 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 0x5587c30ebaf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa6280b9590 tx=0x7fa62802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193e2432987ca9a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa614ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa614ff9640 1 -- 172.21.15.18:0/2813169860 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6280bd830 con 0x5587c30dcd60 2024-02-17T12:52:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa614ff9640 1 -- 172.21.15.18:0/2813169860 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa62802ce40 con 0x5587c30dcd60 2024-02-17T12:52:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa6363fd740 1 -- 172.21.15.18:0/2813169860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587c2d44600 con 0x5587c30dcd60 2024-02-17T12:52:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa614ff9640 1 -- 172.21.15.18:0/2813169860 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6280bd130 con 0x5587c30dcd60 2024-02-17T12:52:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa6363fd740 10 client.16569 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.834+0000 7fa6363fd740 1 -- 172.21.15.18:0/2813169860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5587c3042d80 con 0x5587c30dcd60 2024-02-17T12:52:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.835+0000 7fa614ff9640 1 -- 172.21.15.18:0/2813169860 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa62809f070 con 0x5587c30dcd60 2024-02-17T12:52:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.835+0000 7fa614ff9640 1 -- 172.21.15.18:0/2813169860 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa62803c070 con 0x5587c30dcd60 2024-02-17T12:52:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.835+0000 7fa6363fd740 10 client.16569 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.835+0000 7fa6363fd740 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.835+0000 7fa6363fd740 20 client.16569 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.835+0000 7fa6363fd740 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5587c30fec50 0x5587c3101070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.835+0000 7fa6363fd740 4 client.16569 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.835+0000 7fa6363fd740 1 -- 172.21.15.18:0/2813169860 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587c2d951b0 con 0x5587c30fec50 2024-02-17T12:52:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.835+0000 7fa630cf0640 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5587c30fec50 0x5587c3101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.835+0000 7fa614ff9640 10 client.16569 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.835+0000 7fa630cf0640 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5587c30fec50 0x5587c3101070 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e1805d4bdf4dad server_cookie=ba79285ae6af07c5 in_seq=0 out_seq=0 2024-02-17T12:52:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa614ff9640 1 -- 172.21.15.18:0/2813169860 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587c2d951b0 con 0x5587c30fec50 2024-02-17T12:52:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa614ff9640 10 client.16569 handle_command_reply: tid=0 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa6363fd740 1 client.16569 shutdown 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa6363fd740 1 -- 172.21.15.18:0/2813169860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 msgr2=0x5587c30ebaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa6363fd740 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 0x5587c30ebaf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa6280b9590 tx=0x7fa62802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa6363fd740 1 -- 172.21.15.18:0/2813169860 shutdown_connections 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa6363fd740 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5587c30fec50 0x5587c3101070 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa6363fd740 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30ec570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa614ff9640 0 client.16569 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa6363fd740 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30ec030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa6363fd740 1 --2- 172.21.15.18:0/2813169860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 0x5587c30ebaf0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa6363fd740 1 -- 172.21.15.18:0/2813169860 >> 172.21.15.18:0/2813169860 conn(0x5587c30a4550 msgr2=0x5587c30d97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa6363fd740 1 -- 172.21.15.18:0/2813169860 shutdown_connections 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.836+0000 7fa6363fd740 1 -- 172.21.15.18:0/2813169860 wait complete. 2024-02-17T12:52:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.837+0000 7fa6363fd740 20 client.16569 trim_cache size 0 max 16384 2024-02-17T12:52:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6363fd740 1 Processor -- start 2024-02-17T12:52:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6363fd740 1 -- start start 2024-02-17T12:52:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6363fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 0x5587c30ee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6363fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30ec390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6363fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30f5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6363fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c30de0c0 con 0x5587c30dcd60 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6363fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c30d8850 con 0x5587c30ddad0 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6363fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c3106260 con 0x5587c30eeaf0 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6314f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 0x5587c30ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6314f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 0x5587c30ee220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40942/0 (socket says 172.21.15.18:40942) 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6314f1640 1 -- 172.21.15.18:0/4029070179 learned_addr learned my addr 172.21.15.18:0/4029070179 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa631cf2640 1 --2- 172.21.15.18:0/4029070179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa630cf0640 1 --2- 172.21.15.18:0/4029070179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30ec390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6314f1640 1 -- 172.21.15.18:0/4029070179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 msgr2=0x5587c30f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6314f1640 1 --2- 172.21.15.18:0/4029070179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6314f1640 1 -- 172.21.15.18:0/4029070179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 msgr2=0x5587c30ec390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6314f1640 1 --2- 172.21.15.18:0/4029070179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30ec390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa6314f1640 1 -- 172.21.15.18:0/4029070179 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587c2fdf7b0 con 0x5587c30dcd60 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa630cf0640 1 --2- 172.21.15.18:0/4029070179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30ec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.850+0000 7fa631cf2640 1 --2- 172.21.15.18:0/4029070179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.851+0000 7fa6314f1640 1 --2- 172.21.15.18:0/4029070179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 0x5587c30ee220 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa628005ec0 tx=0x7fa6280057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1618a7d181010f60 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.851+0000 7fa616ffd640 1 -- 172.21.15.18:0/4029070179 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6280c2920 con 0x5587c30dcd60 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.851+0000 7fa616ffd640 1 -- 172.21.15.18:0/4029070179 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa62809f070 con 0x5587c30dcd60 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.851+0000 7fa616ffd640 1 -- 172.21.15.18:0/4029070179 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6280bd5f0 con 0x5587c30dcd60 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.851+0000 7fa6363fd740 1 -- 172.21.15.18:0/4029070179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 msgr2=0x5587c30ee220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.851+0000 7fa6363fd740 1 --2- 172.21.15.18:0/4029070179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 0x5587c30ee220 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa628005ec0 tx=0x7fa6280057e0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.851+0000 7fa6363fd740 1 -- 172.21.15.18:0/4029070179 shutdown_connections 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.851+0000 7fa6363fd740 1 --2- 172.21.15.18:0/4029070179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30eeaf0 0x5587c30f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.851+0000 7fa6363fd740 1 --2- 172.21.15.18:0/4029070179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30ec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 1 --2- 172.21.15.18:0/4029070179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30dcd60 0x5587c30ee220 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 1 -- 172.21.15.18:0/4029070179 >> 172.21.15.18:0/4029070179 conn(0x5587c30a4550 msgr2=0x5587c30d7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 1 -- 172.21.15.18:0/4029070179 shutdown_connections 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 1 -- 172.21.15.18:0/4029070179 wait complete. 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 1 -- start start 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 1 Processor -- start 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30dcd60 0x5587c30f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30f2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30eeaf0 0x5587c3154120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c30f26f0 con 0x5587c30eeaf0 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c3154660 con 0x5587c30ddad0 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c31547a0 con 0x5587c30dcd60 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6363fd740 10 client.0 _ll_register_callbacks cb 0x7fa61c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa631cf2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30eeaf0 0x5587c3154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa631cf2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30eeaf0 0x5587c3154120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40944/0 (socket says 172.21.15.18:40944) 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa631cf2640 1 -- 172.21.15.18:0/3184458909 learned_addr learned my addr 172.21.15.18:0/3184458909 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa6314f1640 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30dcd60 0x5587c30f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.852+0000 7fa630cf0640 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa631cf2640 1 -- 172.21.15.18:0/3184458909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30dcd60 msgr2=0x5587c30f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa631cf2640 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30dcd60 0x5587c30f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa631cf2640 1 -- 172.21.15.18:0/3184458909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 msgr2=0x5587c30f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa631cf2640 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30f2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa631cf2640 1 -- 172.21.15.18:0/3184458909 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587c3042d80 con 0x5587c30eeaf0 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa6314f1640 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30dcd60 0x5587c30f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa631cf2640 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30eeaf0 0x5587c3154120 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa624005fd0 tx=0x7fa62401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3266348cde700ec1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa614ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa614ff9640 1 -- 172.21.15.18:0/3184458909 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624026040 con 0x5587c30eeaf0 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa614ff9640 1 -- 172.21.15.18:0/3184458909 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa6240047d0 con 0x5587c30eeaf0 2024-02-17T12:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa6363fd740 1 -- 172.21.15.18:0/3184458909 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa61c009e40 con 0x5587c30eeaf0 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa614ff9640 1 -- 172.21.15.18:0/3184458909 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa624036460 con 0x5587c30eeaf0 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa6363fd740 10 client.16575 fetch_fsmap learned FSMap version 28 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa6363fd740 1 -- 172.21.15.18:0/3184458909 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5587c3005540 con 0x5587c30eeaf0 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.853+0000 7fa614ff9640 1 -- 172.21.15.18:0/3184458909 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa61c009e40 con 0x5587c30eeaf0 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.854+0000 7fa614ff9640 1 -- 172.21.15.18:0/3184458909 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa624021410 con 0x5587c30eeaf0 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.854+0000 7fa6363fd740 10 client.16575 fetch_fsmap finished waiting for FSMap version 28 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.854+0000 7fa6363fd740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.854+0000 7fa6363fd740 20 client.16575 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.854+0000 7fa6363fd740 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5587c31560d0 0x5587c31584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.854+0000 7fa6363fd740 4 client.16575 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.854+0000 7fa6363fd740 1 -- 172.21.15.18:0/3184458909 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5587c2d81550 con 0x5587c31560d0 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.854+0000 7fa6314f1640 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5587c31560d0 0x5587c31584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.854+0000 7fa6314f1640 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5587c31560d0 0x5587c31584f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dee83b0d4565694 server_cookie=a27fd5d6637684 in_seq=0 out_seq=0 2024-02-17T12:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.854+0000 7fa614ff9640 10 client.16575 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.894+0000 7fa614ff9640 1 -- 172.21.15.18:0/3184458909 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5587c2d81550 con 0x5587c31560d0 2024-02-17T12:52:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.894+0000 7fa614ff9640 10 client.16575 handle_command_reply: tid=0 2024-02-17T12:52:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.894+0000 7fa6363fd740 1 client.16575 shutdown 2024-02-17T12:52:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa6363fd740 1 -- 172.21.15.18:0/3184458909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30eeaf0 msgr2=0x5587c3154120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa6363fd740 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30eeaf0 0x5587c3154120 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa624005fd0 tx=0x7fa62401d000 comp rx=0 tx=0).stop 2024-02-17T12:52:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa6363fd740 1 -- 172.21.15.18:0/3184458909 shutdown_connections 2024-02-17T12:52:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa6363fd740 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5587c31560d0 0x5587c31584f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa6363fd740 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587c30eeaf0 0x5587c3154120 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa614ff9640 0 client.16575 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa6363fd740 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5587c30ddad0 0x5587c30f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa6363fd740 1 --2- 172.21.15.18:0/3184458909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587c30dcd60 0x5587c30f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa6363fd740 1 -- 172.21.15.18:0/3184458909 >> 172.21.15.18:0/3184458909 conn(0x5587c30a4550 msgr2=0x5587c30ea3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa6363fd740 1 -- 172.21.15.18:0/3184458909 shutdown_connections 2024-02-17T12:52:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa6363fd740 1 -- 172.21.15.18:0/3184458909 wait complete. 2024-02-17T12:52:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa6363fd740 20 client.16575 trim_cache size 0 max 16384 2024-02-17T12:52:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:52:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:52:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:52:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:52:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa633755640 1 -- 172.21.15.18:0/2228826609 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa6180a22d0 msgr2=0x7fa6180a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa633755640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa6180a22d0 0x7fa6180a4790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa624009fd0 tx=0x7fa624009de0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa633755640 1 -- 172.21.15.18:0/2228826609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa62c0735b0 msgr2=0x7fa62c19f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa633755640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa62c0735b0 0x7fa62c19f690 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa628005940 tx=0x7fa62802c710 comp rx=0 tx=0).stop 2024-02-17T12:52:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa633755640 1 -- 172.21.15.18:0/2228826609 shutdown_connections 2024-02-17T12:52:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa633755640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa6180a22d0 0x7fa6180a4790 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa633755640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa62c10d990 0x7fa62c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa633755640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa62c073ed0 0x7fa62c19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa633755640 1 --2- 172.21.15.18:0/2228826609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa62c0735b0 0x7fa62c19f690 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.895+0000 7fa633755640 1 -- 172.21.15.18:0/2228826609 >> 172.21.15.18:0/2228826609 conn(0x7fa62c0fcc60 msgr2=0x7fa62c0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.896+0000 7fa633755640 1 -- 172.21.15.18:0/2228826609 shutdown_connections 2024-02-17T12:52:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:37.896+0000 7fa633755640 1 -- 172.21.15.18:0/2228826609 wait complete. 2024-02-17T12:52:38.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 91485 4.51 MB/sec warmup 113 sec latency 19.695 ms 2024-02-17T12:52:38.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:38 smithi119 ceph-mon[36188]: pgmap v85: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 975 KiB/s wr, 119 op/s 2024-02-17T12:52:38.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:38 smithi146 ceph-mon[36454]: pgmap v85: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 975 KiB/s wr, 119 op/s 2024-02-17T12:52:38.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:38 smithi018 ceph-mon[32625]: pgmap v85: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 975 KiB/s wr, 119 op/s 2024-02-17T12:52:39.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 92991 4.55 MB/sec warmup 114 sec latency 29.184 ms 2024-02-17T12:52:40.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 95822 4.66 MB/sec warmup 115 sec latency 10.995 ms 2024-02-17T12:52:40.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:40 smithi119 ceph-mon[36188]: pgmap v86: 97 pgs: 97 active+clean; 43 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-02-17T12:52:40.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:40 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:52:40.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:40 smithi146 ceph-mon[36454]: pgmap v86: 97 pgs: 97 active+clean; 43 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-02-17T12:52:40.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:40 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:52:40.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:40 smithi018 ceph-mon[32625]: pgmap v86: 97 pgs: 97 active+clean; 43 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-02-17T12:52:40.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:40 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:52:41.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 97272 4.68 MB/sec warmup 116 sec latency 9.748 ms 2024-02-17T12:52:42.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 99665 4.75 MB/sec warmup 117 sec latency 10.355 ms 2024-02-17T12:52:42.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:42 smithi119 ceph-mon[36188]: pgmap v87: 97 pgs: 97 active+clean; 43 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 728 KiB/s wr, 110 op/s 2024-02-17T12:52:42.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:42 smithi146 ceph-mon[36454]: pgmap v87: 97 pgs: 97 active+clean; 43 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 728 KiB/s wr, 110 op/s 2024-02-17T12:52:42.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:42 smithi018 ceph-mon[32625]: pgmap v87: 97 pgs: 97 active+clean; 43 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 728 KiB/s wr, 110 op/s 2024-02-17T12:52:43.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 101751 4.78 MB/sec warmup 118 sec latency 14.291 ms 2024-02-17T12:52:44.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 103362 4.84 MB/sec warmup 119 sec latency 26.252 ms 2024-02-17T12:52:44.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:44 smithi119 ceph-mon[36188]: pgmap v88: 97 pgs: 97 active+clean; 43 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 728 KiB/s wr, 110 op/s 2024-02-17T12:52:44.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:44 smithi146 ceph-mon[36454]: pgmap v88: 97 pgs: 97 active+clean; 43 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 728 KiB/s wr, 110 op/s 2024-02-17T12:52:44.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:44 smithi018 ceph-mon[32625]: pgmap v88: 97 pgs: 97 active+clean; 43 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 728 KiB/s wr, 110 op/s 2024-02-17T12:52:46.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 107394 7.37 MB/sec execute 1 sec latency 13.719 ms 2024-02-17T12:52:46.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:46 smithi119 ceph-mon[36188]: pgmap v89: 97 pgs: 97 active+clean; 53 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2024-02-17T12:52:46.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:46 smithi146 ceph-mon[36454]: pgmap v89: 97 pgs: 97 active+clean; 53 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2024-02-17T12:52:46.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:46 smithi018 ceph-mon[32625]: pgmap v89: 97 pgs: 97 active+clean; 53 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2024-02-17T12:52:47.266 INFO:tasks.workunit.client.0.smithi018.stdout: 1 110115 12.16 MB/sec execute 2 sec latency 11.362 ms 2024-02-17T12:52:48.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 111530 10.27 MB/sec execute 3 sec latency 11.281 ms 2024-02-17T12:52:48.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:48 smithi119 ceph-mon[36188]: pgmap v90: 97 pgs: 97 active+clean; 53 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2024-02-17T12:52:48.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:48 smithi146 ceph-mon[36454]: pgmap v90: 97 pgs: 97 active+clean; 53 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2024-02-17T12:52:48.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:48 smithi018 ceph-mon[32625]: pgmap v90: 97 pgs: 97 active+clean; 53 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2024-02-17T12:52:49.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 113881 11.19 MB/sec execute 4 sec latency 13.661 ms 2024-02-17T12:52:50.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 116114 10.56 MB/sec execute 5 sec latency 15.918 ms 2024-02-17T12:52:50.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:50 smithi119 ceph-mon[36188]: pgmap v91: 97 pgs: 97 active+clean; 56 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.9 MiB/s wr, 323 op/s 2024-02-17T12:52:50.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:50 smithi146 ceph-mon[36454]: pgmap v91: 97 pgs: 97 active+clean; 56 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.9 MiB/s wr, 323 op/s 2024-02-17T12:52:50.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:50 smithi018 ceph-mon[32625]: pgmap v91: 97 pgs: 97 active+clean; 56 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.9 MiB/s wr, 323 op/s 2024-02-17T12:52:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.983+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/2173811564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a18102360 msgr2=0x7f3a18102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.983+0000 7f3a1dc2e640 1 --2- 172.21.15.18:0/2173811564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a18102360 0x7f3a18102740 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009900 tx=0x7f3a0c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.983+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/2173811564 shutdown_connections 2024-02-17T12:52:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.983+0000 7f3a1dc2e640 1 --2- 172.21.15.18:0/2173811564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1810b7d0 0x7f3a1810dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.983+0000 7f3a1dc2e640 1 --2- 172.21.15.18:0/2173811564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a18102c80 0x7f3a1810b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.983+0000 7f3a1dc2e640 1 --2- 172.21.15.18:0/2173811564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a18102360 0x7f3a18102740 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.983+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/2173811564 >> 172.21.15.18:0/2173811564 conn(0x7f3a180771b0 msgr2=0x7f3a180775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.983+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/2173811564 shutdown_connections 2024-02-17T12:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.984+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/2173811564 wait complete. 2024-02-17T12:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.984+0000 7f3a1dc2e640 1 Processor -- start 2024-02-17T12:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.984+0000 7f3a1dc2e640 1 -- start start 2024-02-17T12:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.984+0000 7f3a1dc2e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a18102360 0x7f3a18197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.984+0000 7f3a1dc2e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a18102c80 0x7f3a18197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.984+0000 7f3a1dc2e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a1810b7d0 0x7f3a1819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.984+0000 7f3a1dc2e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1819c5f0 con 0x7f3a18102360 2024-02-17T12:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.984+0000 7f3a1dc2e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1819c760 con 0x7f3a18102c80 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.984+0000 7f3a1dc2e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1819c920 con 0x7f3a1810b7d0 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a18102360 0x7f3a18197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a18102360 0x7f3a18197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44232/0 (socket says 172.21.15.18:44232) 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a177fe640 1 -- 172.21.15.18:0/1162588403 learned_addr learned my addr 172.21.15.18:0/1162588403 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a16ffd640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a18102c80 0x7f3a18197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a17fff640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a1810b7d0 0x7f3a1819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a177fe640 1 -- 172.21.15.18:0/1162588403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a1810b7d0 msgr2=0x7f3a1819bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a177fe640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a1810b7d0 0x7f3a1819bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a177fe640 1 -- 172.21.15.18:0/1162588403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a18102c80 msgr2=0x7f3a18197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a177fe640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a18102c80 0x7f3a18197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a177fe640 1 -- 172.21.15.18:0/1162588403 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c009560 con 0x7f3a18102360 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a16ffd640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a18102c80 0x7f3a18197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a177fe640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a18102360 0x7f3a18197730 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c005940 tx=0x7f3a0c02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a14ff9640 1 -- 172.21.15.18:0/1162588403 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c004350 con 0x7f3a18102360 2024-02-17T12:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/1162588403 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1819cba0 con 0x7f3a18102360 2024-02-17T12:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a14ff9640 1 -- 172.21.15.18:0/1162588403 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c0044b0 con 0x7f3a18102360 2024-02-17T12:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a14ff9640 1 -- 172.21.15.18:0/1162588403 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c003d30 con 0x7f3a18102360 2024-02-17T12:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.985+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/1162588403 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a181a4480 con 0x7f3a18102360 2024-02-17T12:52:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.987+0000 7f3a14ff9640 1 -- 172.21.15.18:0/1162588403 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3a0c004df0 con 0x7f3a18102360 2024-02-17T12:52:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a20804740 1 Processor -- start 2024-02-17T12:52:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a20804740 1 -- start start 2024-02-17T12:52:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a14ff9640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3a000a22d0 0x7f3a000a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a20804740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16109820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a20804740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe16109d60 0x55fe1610a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a20804740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe161132e0 0x55fe161156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a20804740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe160789f0 con 0x55fe16109d60 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a20804740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe16075f20 con 0x55fe1610aad0 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a20804740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe16071700 con 0x55fe161132e0 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a17fff640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3a000a22d0 0x7f3a000a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a16ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe16109d60 0x55fe1610a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a16ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe16109d60 0x55fe1610a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44234/0 (socket says 172.21.15.18:44234) 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a16ffd640 1 -- 172.21.15.18:0/2311020711 learned_addr learned my addr 172.21.15.18:0/2311020711 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a14ff9640 1 -- 172.21.15.18:0/1162588403 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3a0c0efff0 con 0x7f3a18102360 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a177fe640 1 --2- 172.21.15.18:0/2311020711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16109820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.988+0000 7f3a17fff640 1 --2- 172.21.15.18:0/2311020711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe161132e0 0x55fe161156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.989+0000 7f3a16ffd640 1 -- 172.21.15.18:0/2311020711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe161132e0 msgr2=0x55fe161156d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.989+0000 7f3a16ffd640 1 --2- 172.21.15.18:0/2311020711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe161132e0 0x55fe161156d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.989+0000 7f3a16ffd640 1 -- 172.21.15.18:0/2311020711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 msgr2=0x55fe16109820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.989+0000 7f3a16ffd640 1 --2- 172.21.15.18:0/2311020711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16109820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.989+0000 7f3a16ffd640 1 -- 172.21.15.18:0/2311020711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe160077b0 con 0x55fe16109d60 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.989+0000 7f3a17fff640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3a000a22d0 0x7f3a000a4790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009fd0 tx=0x7f3a10013b80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.989+0000 7f3a16ffd640 1 --2- 172.21.15.18:0/2311020711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe16109d60 0x55fe1610a1e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3a080114f0 tx=0x7f3a080118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48cb0cc8f277d2f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.989+0000 7f39fdffb640 1 -- 172.21.15.18:0/2311020711 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08004020 con 0x55fe16109d60 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.989+0000 7f39fdffb640 1 -- 172.21.15.18:0/2311020711 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a08004180 con 0x55fe16109d60 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.989+0000 7f39fdffb640 1 -- 172.21.15.18:0/2311020711 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0800f7c0 con 0x55fe16109d60 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.990+0000 7f3a20804740 1 -- 172.21.15.18:0/2311020711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe16109d60 msgr2=0x55fe1610a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.990+0000 7f3a20804740 1 --2- 172.21.15.18:0/2311020711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe16109d60 0x55fe1610a1e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3a080114f0 tx=0x7f3a080118f0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.990+0000 7f3a20804740 1 -- 172.21.15.18:0/2311020711 shutdown_connections 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.990+0000 7f3a20804740 1 --2- 172.21.15.18:0/2311020711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe161132e0 0x55fe161156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.990+0000 7f3a20804740 1 --2- 172.21.15.18:0/2311020711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe16109d60 0x55fe1610a1e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.990+0000 7f3a20804740 1 --2- 172.21.15.18:0/2311020711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.990+0000 7f3a20804740 1 -- 172.21.15.18:0/2311020711 >> 172.21.15.18:0/2311020711 conn(0x55fe160d1550 msgr2=0x55fe16104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.990+0000 7f3a20804740 1 -- 172.21.15.18:0/2311020711 shutdown_connections 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.990+0000 7f3a20804740 1 -- 172.21.15.18:0/2311020711 wait complete. 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.990+0000 7f3a20804740 1 -- start start 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.990+0000 7f3a20804740 1 Processor -- start 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a20804740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe161102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a20804740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16110820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a20804740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 0x55fe16110d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a20804740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe16016bf0 con 0x55fe161132e0 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a20804740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe15d45250 con 0x55fe1610aad0 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a20804740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe15ff7200 con 0x55fe16109d60 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a20804740 10 client.0 _ll_register_callbacks cb 0x55fe16065d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a17fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 0x55fe16110d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe161102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a16ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16110820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a17fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 0x55fe16110d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44236/0 (socket says 172.21.15.18:44236) 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a16ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16110820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:49090/0 (socket says 172.21.15.18:49090) 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a17fff640 1 -- 172.21.15.18:0/3056597498 learned_addr learned my addr 172.21.15.18:0/3056597498 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a177fe640 1 -- 172.21.15.18:0/3056597498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 msgr2=0x55fe16110820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a177fe640 1 --2- 172.21.15.18:0/3056597498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16110820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a177fe640 1 -- 172.21.15.18:0/3056597498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 msgr2=0x55fe16110d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a177fe640 1 --2- 172.21.15.18:0/3056597498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 0x55fe16110d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a177fe640 1 -- 172.21.15.18:0/3056597498 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a08009cc0 con 0x55fe16109d60 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a17fff640 1 --2- 172.21.15.18:0/3056597498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 0x55fe16110d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a177fe640 1 --2- 172.21.15.18:0/3056597498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe161102e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0ed8a0 tx=0x7f3a0c0b8f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2021160037d30041 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f39f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f39f6ffd640 1 -- 172.21.15.18:0/3056597498 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0035f0 con 0x55fe16109d60 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f39f6ffd640 1 -- 172.21.15.18:0/3056597498 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3a0c0bd240 con 0x55fe16109d60 2024-02-17T12:52:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.991+0000 7f3a20804740 1 -- 172.21.15.18:0/3056597498 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe15d71600 con 0x55fe16109d60 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.992+0000 7f39f6ffd640 1 -- 172.21.15.18:0/3056597498 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a0c0cef80 con 0x55fe16109d60 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.992+0000 7f3a20804740 10 client.25984 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.992+0000 7f3a20804740 1 -- 172.21.15.18:0/3056597498 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fe1606fd80 con 0x55fe16109d60 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.992+0000 7f39f6ffd640 1 -- 172.21.15.18:0/3056597498 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fe15d71600 con 0x55fe16109d60 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.992+0000 7f39f6ffd640 1 -- 172.21.15.18:0/3056597498 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3a0c0dc040 con 0x55fe16109d60 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.992+0000 7f3a20804740 10 client.25984 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.992+0000 7f3a20804740 10 client.25984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.992+0000 7f3a20804740 20 client.25984 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.992+0000 7f3a20804740 1 --2- 172.21.15.18:0/3056597498 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fe16123360 0x55fe16125780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.992+0000 7f3a20804740 4 client.25984 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.992+0000 7f3a20804740 1 -- 172.21.15.18:0/3056597498 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe15dc21b0 con 0x55fe16123360 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.992+0000 7f3a16ffd640 1 --2- 172.21.15.18:0/3056597498 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fe16123360 0x55fe16125780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.993+0000 7f39f6ffd640 10 client.25984 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.993+0000 7f3a16ffd640 1 --2- 172.21.15.18:0/3056597498 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fe16123360 0x55fe16125780 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d04eb5a47deba8 server_cookie=da16ed6dc6235bfe in_seq=0 out_seq=0 2024-02-17T12:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.993+0000 7f39f6ffd640 1 -- 172.21.15.18:0/3056597498 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe15dc21b0 con 0x55fe16123360 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.993+0000 7f39f6ffd640 10 client.25984 handle_command_reply: tid=0 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.993+0000 7f3a20804740 1 client.25984 shutdown 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.993+0000 7f3a20804740 1 -- 172.21.15.18:0/3056597498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 msgr2=0x55fe161102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.993+0000 7f3a20804740 1 --2- 172.21.15.18:0/3056597498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe161102e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0ed8a0 tx=0x7f3a0c0b8f10 comp rx=0 tx=0).stop 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.994+0000 7f3a20804740 1 -- 172.21.15.18:0/3056597498 shutdown_connections 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.994+0000 7f3a20804740 1 --2- 172.21.15.18:0/3056597498 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fe16123360 0x55fe16125780 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.994+0000 7f3a20804740 1 --2- 172.21.15.18:0/3056597498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 0x55fe16110d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.994+0000 7f3a20804740 1 --2- 172.21.15.18:0/3056597498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.994+0000 7f39f6ffd640 0 client.25984 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.994+0000 7f3a20804740 1 --2- 172.21.15.18:0/3056597498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe161102e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.994+0000 7f3a20804740 1 -- 172.21.15.18:0/3056597498 >> 172.21.15.18:0/3056597498 conn(0x55fe160d1550 msgr2=0x55fe161067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.994+0000 7f3a20804740 1 -- 172.21.15.18:0/3056597498 shutdown_connections 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.994+0000 7f3a20804740 1 -- 172.21.15.18:0/3056597498 wait complete. 2024-02-17T12:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:50.994+0000 7f3a20804740 20 client.25984 trim_cache size 0 max 16384 2024-02-17T12:52:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a20804740 1 Processor -- start 2024-02-17T12:52:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a20804740 1 -- start start 2024-02-17T12:52:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a20804740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe1610b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a20804740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe1610aad0 0x55fe1610bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a20804740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe161132e0 0x55fe161146d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a20804740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe16105a80 con 0x55fe1610aad0 2024-02-17T12:52:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a20804740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe1611fb00 con 0x55fe161132e0 2024-02-17T12:52:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a20804740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe16109030 con 0x55fe16109d60 2024-02-17T12:52:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a16ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe1610aad0 0x55fe1610bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe1610b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a16ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe1610aad0 0x55fe1610bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44250/0 (socket says 172.21.15.18:44250) 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a16ffd640 1 -- 172.21.15.18:0/64438392 learned_addr learned my addr 172.21.15.18:0/64438392 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a17fff640 1 --2- 172.21.15.18:0/64438392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe161132e0 0x55fe161146d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a16ffd640 1 -- 172.21.15.18:0/64438392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 msgr2=0x55fe1610b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a16ffd640 1 --2- 172.21.15.18:0/64438392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe1610b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a16ffd640 1 -- 172.21.15.18:0/64438392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe161132e0 msgr2=0x55fe161146d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a16ffd640 1 --2- 172.21.15.18:0/64438392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe161132e0 0x55fe161146d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a16ffd640 1 -- 172.21.15.18:0/64438392 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe1600c7b0 con 0x55fe1610aad0 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.008+0000 7f3a16ffd640 1 --2- 172.21.15.18:0/64438392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe1610aad0 0x55fe1610bb00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3a08004430 tx=0x7f3a080118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f6843972ecf589 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.009+0000 7f39fd7fa640 1 -- 172.21.15.18:0/64438392 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08028040 con 0x55fe1610aad0 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.009+0000 7f39fd7fa640 1 -- 172.21.15.18:0/64438392 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a08003dd0 con 0x55fe1610aad0 2024-02-17T12:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.009+0000 7f39fd7fa640 1 -- 172.21.15.18:0/64438392 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08003a80 con 0x55fe1610aad0 2024-02-17T12:52:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.009+0000 7f3a20804740 1 -- 172.21.15.18:0/64438392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe1610aad0 msgr2=0x55fe1610bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.009+0000 7f3a20804740 1 --2- 172.21.15.18:0/64438392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe1610aad0 0x55fe1610bb00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3a08004430 tx=0x7f3a080118f0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.009+0000 7f3a20804740 1 -- 172.21.15.18:0/64438392 shutdown_connections 2024-02-17T12:52:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.009+0000 7f3a20804740 1 --2- 172.21.15.18:0/64438392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe161132e0 0x55fe161146d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.009+0000 7f3a20804740 1 --2- 172.21.15.18:0/64438392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe1610aad0 0x55fe1610bb00 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.009+0000 7f3a20804740 1 --2- 172.21.15.18:0/64438392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe1610b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.009+0000 7f3a20804740 1 -- 172.21.15.18:0/64438392 >> 172.21.15.18:0/64438392 conn(0x55fe160d1550 msgr2=0x55fe1612a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.009+0000 7f3a20804740 1 -- 172.21.15.18:0/64438392 shutdown_connections 2024-02-17T12:52:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.009+0000 7f3a20804740 1 -- 172.21.15.18:0/64438392 wait complete. 2024-02-17T12:52:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a20804740 1 -- start start 2024-02-17T12:52:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a20804740 1 Processor -- start 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a20804740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe1611a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a20804740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16118600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a20804740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 0x55fe16118b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a20804740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe16119080 con 0x55fe161132e0 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a20804740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe161191f0 con 0x55fe1610aad0 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a20804740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe161193b0 con 0x55fe16109d60 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a20804740 10 client.0 _ll_register_callbacks cb 0x7f3a08009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a17fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 0x55fe16118b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a177fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe1611a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a17fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 0x55fe16118b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44266/0 (socket says 172.21.15.18:44266) 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a17fff640 1 -- 172.21.15.18:0/3087392590 learned_addr learned my addr 172.21.15.18:0/3087392590 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a16ffd640 1 --2- 172.21.15.18:0/3087392590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16118600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a17fff640 1 -- 172.21.15.18:0/3087392590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 msgr2=0x55fe1611a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a17fff640 1 --2- 172.21.15.18:0/3087392590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe1611a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a17fff640 1 -- 172.21.15.18:0/3087392590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 msgr2=0x55fe16118600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a17fff640 1 --2- 172.21.15.18:0/3087392590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16118600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.010+0000 7f3a17fff640 1 -- 172.21.15.18:0/3087392590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe160077b0 con 0x55fe161132e0 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f3a17fff640 1 --2- 172.21.15.18:0/3087392590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 0x55fe16118b40 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3a10003ea0 tx=0x7f3a10004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4998edf68abcb6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f39f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f39f6ffd640 1 -- 172.21.15.18:0/3087392590 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000fea0 con 0x55fe161132e0 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f39f6ffd640 1 -- 172.21.15.18:0/3087392590 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3a10004530 con 0x55fe161132e0 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f3a20804740 1 -- 172.21.15.18:0/3087392590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a08004180 con 0x55fe161132e0 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f39f6ffd640 1 -- 172.21.15.18:0/3087392590 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a100046f0 con 0x55fe161132e0 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f3a20804740 10 client.16593 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f3a20804740 1 -- 172.21.15.18:0/3087392590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fe16032540 con 0x55fe161132e0 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f39f6ffd640 1 -- 172.21.15.18:0/3087392590 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a08004180 con 0x55fe161132e0 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f39f6ffd640 1 -- 172.21.15.18:0/3087392590 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3a10005730 con 0x55fe161132e0 2024-02-17T12:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f3a20804740 10 client.16593 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f3a20804740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f3a20804740 20 client.16593 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f3a20804740 1 --2- 172.21.15.18:0/3087392590 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fe1617ae80 0x55fe1617d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f3a20804740 4 client.16593 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.011+0000 7f3a20804740 1 -- 172.21.15.18:0/3087392590 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fe15dae550 con 0x55fe1617ae80 2024-02-17T12:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.012+0000 7f3a177fe640 1 --2- 172.21.15.18:0/3087392590 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fe1617ae80 0x55fe1617d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.012+0000 7f39f6ffd640 10 client.16593 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.012+0000 7f3a177fe640 1 --2- 172.21.15.18:0/3087392590 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fe1617ae80 0x55fe1617d2a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1dfefce62995c3 server_cookie=6f7afda4295abd0 in_seq=0 out_seq=0 2024-02-17T12:52:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.058+0000 7f39f6ffd640 1 -- 172.21.15.18:0/3087392590 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fe15dae550 con 0x55fe1617ae80 2024-02-17T12:52:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.058+0000 7f39f6ffd640 10 client.16593 handle_command_reply: tid=0 2024-02-17T12:52:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a20804740 1 client.16593 shutdown 2024-02-17T12:52:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a20804740 1 -- 172.21.15.18:0/3087392590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 msgr2=0x55fe16118b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a20804740 1 --2- 172.21.15.18:0/3087392590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 0x55fe16118b40 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3a10003ea0 tx=0x7f3a10004ab0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a20804740 1 -- 172.21.15.18:0/3087392590 shutdown_connections 2024-02-17T12:52:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a20804740 1 --2- 172.21.15.18:0/3087392590 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fe1617ae80 0x55fe1617d2a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a20804740 1 --2- 172.21.15.18:0/3087392590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe161132e0 0x55fe16118b40 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a20804740 1 --2- 172.21.15.18:0/3087392590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe1610aad0 0x55fe16118600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f39f6ffd640 0 client.16593 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a20804740 1 --2- 172.21.15.18:0/3087392590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fe16109d60 0x55fe1611a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a20804740 1 -- 172.21.15.18:0/3087392590 >> 172.21.15.18:0/3087392590 conn(0x55fe160d1550 msgr2=0x55fe1610b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a20804740 1 -- 172.21.15.18:0/3087392590 shutdown_connections 2024-02-17T12:52:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a20804740 1 -- 172.21.15.18:0/3087392590 wait complete. 2024-02-17T12:52:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a20804740 20 client.16593 trim_cache size 0 max 16384 2024-02-17T12:52:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:52:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:52:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:52:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:52:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.059+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/1162588403 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3a000a22d0 msgr2=0x7f3a000a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.060+0000 7f3a1dc2e640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3a000a22d0 0x7f3a000a4790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009fd0 tx=0x7f3a10013b80 comp rx=0 tx=0).stop 2024-02-17T12:52:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.060+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/1162588403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a18102360 msgr2=0x7f3a18197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.060+0000 7f3a1dc2e640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a18102360 0x7f3a18197730 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c005940 tx=0x7f3a0c02c670 comp rx=0 tx=0).stop 2024-02-17T12:52:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.060+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/1162588403 shutdown_connections 2024-02-17T12:52:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.060+0000 7f3a1dc2e640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3a000a22d0 0x7f3a000a4790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.060+0000 7f3a1dc2e640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a1810b7d0 0x7f3a1819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.060+0000 7f3a1dc2e640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a18102c80 0x7f3a18197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.060+0000 7f3a1dc2e640 1 --2- 172.21.15.18:0/1162588403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a18102360 0x7f3a18197730 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.060+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/1162588403 >> 172.21.15.18:0/1162588403 conn(0x7f3a180771b0 msgr2=0x7f3a180fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.060+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/1162588403 shutdown_connections 2024-02-17T12:52:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:52:51.060+0000 7f3a1dc2e640 1 -- 172.21.15.18:0/1162588403 wait complete. 2024-02-17T12:52:51.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 117657 10.65 MB/sec execute 6 sec latency 44.871 ms 2024-02-17T12:52:52.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 120265 11.01 MB/sec execute 7 sec latency 11.929 ms 2024-02-17T12:52:52.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:52 smithi119 ceph-mon[36188]: pgmap v92: 97 pgs: 97 active+clean; 56 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 234 op/s 2024-02-17T12:52:52.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:52 smithi146 ceph-mon[36454]: pgmap v92: 97 pgs: 97 active+clean; 56 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 234 op/s 2024-02-17T12:52:52.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:52 smithi018 ceph-mon[32625]: pgmap v92: 97 pgs: 97 active+clean; 56 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 234 op/s 2024-02-17T12:52:53.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 121633 10.50 MB/sec execute 8 sec latency 36.981 ms 2024-02-17T12:52:54.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 124336 11.21 MB/sec execute 9 sec latency 19.684 ms 2024-02-17T12:52:54.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:54 smithi119 ceph-mon[36188]: pgmap v93: 97 pgs: 97 active+clean; 56 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 234 op/s 2024-02-17T12:52:54.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:54 smithi146 ceph-mon[36454]: pgmap v93: 97 pgs: 97 active+clean; 56 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 234 op/s 2024-02-17T12:52:54.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:54 smithi018 ceph-mon[32625]: pgmap v93: 97 pgs: 97 active+clean; 56 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 234 op/s 2024-02-17T12:52:55.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 125795 10.74 MB/sec execute 10 sec latency 16.402 ms 2024-02-17T12:52:56.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 128031 11.00 MB/sec execute 11 sec latency 11.545 ms 2024-02-17T12:52:56.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:56 smithi119 ceph-mon[36188]: pgmap v94: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-02-17T12:52:56.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:56 smithi146 ceph-mon[36454]: pgmap v94: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-02-17T12:52:56.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:56 smithi018 ceph-mon[32625]: pgmap v94: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-02-17T12:52:57.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 130218 10.75 MB/sec execute 12 sec latency 9.848 ms 2024-02-17T12:52:58.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:52:58.052 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:52:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.170+0000 7f49b594f640 1 -- 172.21.15.18:0/2651154850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49b010b6e0 msgr2=0x7f49b010db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.170+0000 7f49b594f640 1 --2- 172.21.15.18:0/2651154850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49b010b6e0 0x7f49b010db80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009900 tx=0x7f49a802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.170+0000 7f49b594f640 1 -- 172.21.15.18:0/2651154850 shutdown_connections 2024-02-17T12:52:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.170+0000 7f49b594f640 1 --2- 172.21.15.18:0/2651154850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49b010b6e0 0x7f49b010db80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.170+0000 7f49b594f640 1 --2- 172.21.15.18:0/2651154850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49b0102cc0 0x7f49b010b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.170+0000 7f49b594f640 1 --2- 172.21.15.18:0/2651154850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49b01023a0 0x7f49b0102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.170+0000 7f49b594f640 1 -- 172.21.15.18:0/2651154850 >> 172.21.15.18:0/2651154850 conn(0x7f49b0077230 msgr2=0x7f49b0077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.170+0000 7f49b594f640 1 -- 172.21.15.18:0/2651154850 shutdown_connections 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.170+0000 7f49b594f640 1 -- 172.21.15.18:0/2651154850 wait complete. 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.170+0000 7f49b594f640 1 Processor -- start 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.170+0000 7f49b594f640 1 -- start start 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49b594f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49b01023a0 0x7f49b0197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49b594f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49b0102cc0 0x7f49b0197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49b594f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49b010b6e0 0x7f49b019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49b594f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b019c5c0 con 0x7f49b0102cc0 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49b594f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b019c730 con 0x7f49b010b6e0 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49b594f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b019c8f0 con 0x7f49b01023a0 2024-02-17T12:52:58.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49ae7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49b0102cc0 0x7f49b0197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49af7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49b010b6e0 0x7f49b019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49ae7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49b0102cc0 0x7f49b0197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44278/0 (socket says 172.21.15.18:44278) 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49ae7fc640 1 -- 172.21.15.18:0/2243477197 learned_addr learned my addr 172.21.15.18:0/2243477197 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49aeffd640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49b01023a0 0x7f49b0197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49ae7fc640 1 -- 172.21.15.18:0/2243477197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49b01023a0 msgr2=0x7f49b0197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49ae7fc640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49b01023a0 0x7f49b0197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49ae7fc640 1 -- 172.21.15.18:0/2243477197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49b010b6e0 msgr2=0x7f49b019bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49ae7fc640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49b010b6e0 0x7f49b019bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49ae7fc640 1 -- 172.21.15.18:0/2243477197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a8009560 con 0x7f49b0102cc0 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49af7fe640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49b010b6e0 0x7f49b019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.171+0000 7f49ae7fc640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49b0102cc0 0x7f49b0197c40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f49a400e9f0 tx=0x7f49a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.172+0000 7f49b494d640 1 -- 172.21.15.18:0/2243477197 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a4009700 con 0x7f49b0102cc0 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.172+0000 7f49b494d640 1 -- 172.21.15.18:0/2243477197 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49a400cd80 con 0x7f49b0102cc0 2024-02-17T12:52:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.172+0000 7f49b594f640 1 -- 172.21.15.18:0/2243477197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b019cbd0 con 0x7f49b0102cc0 2024-02-17T12:52:58.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.172+0000 7f49b494d640 1 -- 172.21.15.18:0/2243477197 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a4018920 con 0x7f49b0102cc0 2024-02-17T12:52:58.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.172+0000 7f49b594f640 1 -- 172.21.15.18:0/2243477197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b01a4450 con 0x7f49b0102cc0 2024-02-17T12:52:58.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.174+0000 7f49b494d640 1 -- 172.21.15.18:0/2243477197 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f49a4018a80 con 0x7f49b0102cc0 2024-02-17T12:52:58.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.174+0000 7f49b8601740 1 Processor -- start 2024-02-17T12:52:58.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.174+0000 7f49b8601740 1 -- start start 2024-02-17T12:52:58.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.174+0000 7f49b494d640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f49980a2230 0x7f49980a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.174+0000 7f49b8601740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d3184e6c0 0x560d3184eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.174+0000 7f49b8601740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 0x560d31857680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.174+0000 7f49b8601740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d31857bc0 0x560d31859fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.174+0000 7f49b8601740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d317b9280 con 0x560d3184efe0 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.174+0000 7f49b8601740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d317b67b0 con 0x560d31857bc0 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.174+0000 7f49b8601740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d317b1f90 con 0x560d3184e6c0 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49af7fe640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f49980a2230 0x7f49980a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49b494d640 1 -- 172.21.15.18:0/2243477197 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f49a4014070 con 0x7f49b0102cc0 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49ae7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 0x560d31857680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49aeffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d3184e6c0 0x560d3184eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49ae7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 0x560d31857680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44290/0 (socket says 172.21.15.18:44290) 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49ae7fc640 1 -- 172.21.15.18:0/400381456 learned_addr learned my addr 172.21.15.18:0/400381456 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49af7fe640 1 --2- 172.21.15.18:0/400381456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d31857bc0 0x560d31859fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49ae7fc640 1 -- 172.21.15.18:0/400381456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d3184e6c0 msgr2=0x560d3184eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49ae7fc640 1 --2- 172.21.15.18:0/400381456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d3184e6c0 0x560d3184eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49ae7fc640 1 -- 172.21.15.18:0/400381456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d31857bc0 msgr2=0x560d31859fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49ae7fc640 1 --2- 172.21.15.18:0/400381456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d31857bc0 0x560d31859fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49ae7fc640 1 -- 172.21.15.18:0/400381456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d31770570 con 0x560d3184efe0 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49af7fe640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f49980a2230 0x7f49980a46f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f49a8005ab0 tx=0x7f49a802c670 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f49ae7fc640 1 --2- 172.21.15.18:0/400381456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 0x560d31857680 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f49a40cbb40 tx=0x7f49a409aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f6e7f640138a67 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f4995ffb640 1 -- 172.21.15.18:0/400381456 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a409eed0 con 0x560d3184efe0 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f4995ffb640 1 -- 172.21.15.18:0/400381456 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49a40a3520 con 0x560d3184efe0 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.175+0000 7f4995ffb640 1 -- 172.21.15.18:0/400381456 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a40974a0 con 0x560d3184efe0 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.176+0000 7f49b8601740 1 -- 172.21.15.18:0/400381456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 msgr2=0x560d31857680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.176+0000 7f49b8601740 1 --2- 172.21.15.18:0/400381456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 0x560d31857680 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f49a40cbb40 tx=0x7f49a409aa30 comp rx=0 tx=0).stop 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.176+0000 7f49b8601740 1 -- 172.21.15.18:0/400381456 shutdown_connections 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.176+0000 7f49b8601740 1 --2- 172.21.15.18:0/400381456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d31857bc0 0x560d31859fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.176+0000 7f49b8601740 1 --2- 172.21.15.18:0/400381456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 0x560d31857680 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.176+0000 7f49b8601740 1 --2- 172.21.15.18:0/400381456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d3184e6c0 0x560d3184eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.176+0000 7f49b8601740 1 -- 172.21.15.18:0/400381456 >> 172.21.15.18:0/400381456 conn(0x560d31848da0 msgr2=0x560d31849180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.176+0000 7f49b8601740 1 -- 172.21.15.18:0/400381456 shutdown_connections 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.176+0000 7f49b8601740 1 -- 172.21.15.18:0/400381456 wait complete. 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49b8601740 1 -- start start 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49b8601740 1 Processor -- start 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49b8601740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184e6c0 0x560d31857660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49b8601740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 0x560d318549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49b8601740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d31857bc0 0x560d318550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49b8601740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d31485430 con 0x560d3184efe0 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49b8601740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d31441e00 con 0x560d3184e6c0 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49b8601740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d31736fd0 con 0x560d31857bc0 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49b8601740 10 client.0 _ll_register_callbacks cb 0x560d3178cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49ae7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 0x560d318549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49ae7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 0x560d318549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44304/0 (socket says 172.21.15.18:44304) 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49ae7fc640 1 -- 172.21.15.18:0/513092538 learned_addr learned my addr 172.21.15.18:0/513092538 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49aeffd640 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184e6c0 0x560d31857660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49af7fe640 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d31857bc0 0x560d318550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49ae7fc640 1 -- 172.21.15.18:0/513092538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d31857bc0 msgr2=0x560d318550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49ae7fc640 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d31857bc0 0x560d318550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49ae7fc640 1 -- 172.21.15.18:0/513092538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184e6c0 msgr2=0x560d31857660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49ae7fc640 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184e6c0 0x560d31857660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.177+0000 7f49ae7fc640 1 -- 172.21.15.18:0/513092538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b01a4450 con 0x560d3184efe0 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f49ae7fc640 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 0x560d318549f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f49a4010df0 tx=0x7f49a407de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fc2fd0ff618fc4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f498effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f498effd640 1 -- 172.21.15.18:0/513092538 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a4010b80 con 0x560d3184efe0 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f498effd640 1 -- 172.21.15.18:0/513092538 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f49a409eed0 con 0x560d3184efe0 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f49b8601740 1 -- 172.21.15.18:0/513092538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d314b17e0 con 0x560d3184efe0 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f498effd640 1 -- 172.21.15.18:0/513092538 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49a40974a0 con 0x560d3184efe0 2024-02-17T12:52:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f49b8601740 10 client.16605 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f49b8601740 1 -- 172.21.15.18:0/513092538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560d317b0610 con 0x560d3184efe0 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f498effd640 1 -- 172.21.15.18:0/513092538 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560d314b17e0 con 0x560d3184efe0 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f498effd640 1 -- 172.21.15.18:0/513092538 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f49a40a3190 con 0x560d3184efe0 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f49b8601740 10 client.16605 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f49b8601740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f49b8601740 20 client.16605 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f49b8601740 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560d31867c00 0x560d3186a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f49b8601740 4 client.16605 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f49b8601740 1 -- 172.21.15.18:0/513092538 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d314ee720 con 0x560d31867c00 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.178+0000 7f49aeffd640 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560d31867c00 0x560d3186a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.179+0000 7f49aeffd640 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560d31867c00 0x560d3186a040 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=843c2741017dba9 server_cookie=7c90219cc2ded666 in_seq=0 out_seq=0 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.179+0000 7f498effd640 10 client.16605 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f498effd640 1 -- 172.21.15.18:0/513092538 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d314ee720 con 0x560d31867c00 2024-02-17T12:52:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f498effd640 10 client.16605 handle_command_reply: tid=0 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f49b8601740 1 client.16605 shutdown 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f49b8601740 1 -- 172.21.15.18:0/513092538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 msgr2=0x560d318549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f49b8601740 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 0x560d318549f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f49a4010df0 tx=0x7f49a407de50 comp rx=0 tx=0).stop 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f49b8601740 1 -- 172.21.15.18:0/513092538 shutdown_connections 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f49b8601740 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560d31867c00 0x560d3186a040 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f49b8601740 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d31857bc0 0x560d318550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f49b8601740 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184efe0 0x560d318549f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f49b8601740 1 --2- 172.21.15.18:0/513092538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184e6c0 0x560d31857660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f498effd640 0 client.16605 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f49b8601740 1 -- 172.21.15.18:0/513092538 >> 172.21.15.18:0/513092538 conn(0x560d31848da0 msgr2=0x560d3184ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f49b8601740 1 -- 172.21.15.18:0/513092538 shutdown_connections 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f49b8601740 1 -- 172.21.15.18:0/513092538 wait complete. 2024-02-17T12:52:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.180+0000 7f49b8601740 20 client.16605 trim_cache size 0 max 16384 2024-02-17T12:52:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.195+0000 7f49b8601740 1 Processor -- start 2024-02-17T12:52:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.195+0000 7f49b8601740 1 -- start start 2024-02-17T12:52:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.195+0000 7f49b8601740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184e6c0 0x560d31857120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.195+0000 7f49b8601740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184efe0 0x560d31857660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.195+0000 7f49b8601740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d31857bc0 0x560d3185e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.195+0000 7f49b8601740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d318400f0 con 0x560d3184e6c0 2024-02-17T12:52:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.195+0000 7f49b8601740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d3184e100 con 0x560d3184efe0 2024-02-17T12:52:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.195+0000 7f49b8601740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d3184e320 con 0x560d31857bc0 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49aeffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184e6c0 0x560d31857120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49aeffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184e6c0 0x560d31857120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44312/0 (socket says 172.21.15.18:44312) 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49aeffd640 1 -- 172.21.15.18:0/2104848981 learned_addr learned my addr 172.21.15.18:0/2104848981 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49ae7fc640 1 --2- 172.21.15.18:0/2104848981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184efe0 0x560d31857660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49af7fe640 1 --2- 172.21.15.18:0/2104848981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d31857bc0 0x560d3185e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49aeffd640 1 -- 172.21.15.18:0/2104848981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d31857bc0 msgr2=0x560d3185e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49aeffd640 1 --2- 172.21.15.18:0/2104848981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d31857bc0 0x560d3185e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49aeffd640 1 -- 172.21.15.18:0/2104848981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184efe0 msgr2=0x560d31857660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49aeffd640 1 --2- 172.21.15.18:0/2104848981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184efe0 0x560d31857660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49aeffd640 1 -- 172.21.15.18:0/2104848981 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d3174cac0 con 0x560d3184e6c0 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49aeffd640 1 --2- 172.21.15.18:0/2104848981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184e6c0 0x560d31857120 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x560d3186a950 tx=0x7f49a0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe3d898efb4a23e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49957fa640 1 -- 172.21.15.18:0/2104848981 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0016040 con 0x560d3184e6c0 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49957fa640 1 -- 172.21.15.18:0/2104848981 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49a0017040 con 0x560d3184e6c0 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.196+0000 7f49957fa640 1 -- 172.21.15.18:0/2104848981 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a001c380 con 0x560d3184e6c0 2024-02-17T12:52:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.197+0000 7f49b8601740 1 -- 172.21.15.18:0/2104848981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184e6c0 msgr2=0x560d31857120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.197+0000 7f49b8601740 1 --2- 172.21.15.18:0/2104848981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184e6c0 0x560d31857120 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x560d3186a950 tx=0x7f49a0009500 comp rx=0 tx=0).stop 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.197+0000 7f49b8601740 1 -- 172.21.15.18:0/2104848981 shutdown_connections 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.197+0000 7f49b8601740 1 --2- 172.21.15.18:0/2104848981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d31857bc0 0x560d3185e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.197+0000 7f49b8601740 1 --2- 172.21.15.18:0/2104848981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184efe0 0x560d31857660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.197+0000 7f49b8601740 1 --2- 172.21.15.18:0/2104848981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d3184e6c0 0x560d31857120 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.197+0000 7f49b8601740 1 -- 172.21.15.18:0/2104848981 >> 172.21.15.18:0/2104848981 conn(0x560d31848da0 msgr2=0x560d3184fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.197+0000 7f49b8601740 1 -- 172.21.15.18:0/2104848981 shutdown_connections 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.197+0000 7f49b8601740 1 -- 172.21.15.18:0/2104848981 wait complete. 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.197+0000 7f49b8601740 1 -- start start 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.197+0000 7f49b8601740 1 Processor -- start 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49b8601740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d3184e6c0 0x560d3185d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49b8601740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184efe0 0x560d3185d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49b8601740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d31857bc0 0x560d3185de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49b8601740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d318be130 con 0x560d31857bc0 2024-02-17T12:52:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49b8601740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d318be270 con 0x560d3184efe0 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49b8601740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d318be3b0 con 0x560d3184e6c0 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49b8601740 10 client.0 _ll_register_callbacks cb 0x7f49a400cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49af7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d31857bc0 0x560d3185de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49af7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d31857bc0 0x560d3185de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44316/0 (socket says 172.21.15.18:44316) 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49af7fe640 1 -- 172.21.15.18:0/2563185825 learned_addr learned my addr 172.21.15.18:0/2563185825 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49aeffd640 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d3184e6c0 0x560d3185d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49ae7fc640 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184efe0 0x560d3185d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49af7fe640 1 -- 172.21.15.18:0/2563185825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d3184e6c0 msgr2=0x560d3185d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49af7fe640 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d3184e6c0 0x560d3185d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49af7fe640 1 -- 172.21.15.18:0/2563185825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184efe0 msgr2=0x560d3185d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49af7fe640 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184efe0 0x560d3185d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49af7fe640 1 -- 172.21.15.18:0/2563185825 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a0003970 con 0x560d31857bc0 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49aeffd640 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d3184e6c0 0x560d3185d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:52:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49af7fe640 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d31857bc0 0x560d3185de10 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009900 tx=0x7f49a802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408181d9eae31e22 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f498effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f498effd640 1 -- 172.21.15.18:0/2563185825 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8005280 con 0x560d31857bc0 2024-02-17T12:52:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f498effd640 1 -- 172.21.15.18:0/2563185825 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f49a80053e0 con 0x560d31857bc0 2024-02-17T12:52:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.198+0000 7f49b8601740 1 -- 172.21.15.18:0/2563185825 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49a40a3520 con 0x560d31857bc0 2024-02-17T12:52:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.199+0000 7f498effd640 1 -- 172.21.15.18:0/2563185825 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49a80055a0 con 0x560d31857bc0 2024-02-17T12:52:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.199+0000 7f49b8601740 10 client.16611 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.199+0000 7f49b8601740 1 -- 172.21.15.18:0/2563185825 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560d317726a0 con 0x560d31857bc0 2024-02-17T12:52:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.199+0000 7f498effd640 1 -- 172.21.15.18:0/2563185825 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49a40a3520 con 0x560d31857bc0 2024-02-17T12:52:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.199+0000 7f498effd640 1 -- 172.21.15.18:0/2563185825 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f49a8055020 con 0x560d31857bc0 2024-02-17T12:52:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.199+0000 7f49b8601740 10 client.16611 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.199+0000 7f49b8601740 10 client.16611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.199+0000 7f49b8601740 20 client.16611 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.199+0000 7f49b8601740 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560d318bfbd0 0x560d318c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.199+0000 7f49b8601740 4 client.16611 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.199+0000 7f49b8601740 1 -- 172.21.15.18:0/2563185825 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560d3153d890 con 0x560d318bfbd0 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.199+0000 7f49aeffd640 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560d318bfbd0 0x560d318c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f498effd640 10 client.16611 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49aeffd640 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560d318bfbd0 0x560d318c1ff0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695f0719161b8c8e server_cookie=63e701ad96a3f35f in_seq=0 out_seq=0 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f498effd640 1 -- 172.21.15.18:0/2563185825 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560d3153d890 con 0x560d318bfbd0 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f498effd640 10 client.16611 handle_command_reply: tid=0 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49b8601740 1 client.16611 shutdown 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49b8601740 1 -- 172.21.15.18:0/2563185825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d31857bc0 msgr2=0x560d3185de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49b8601740 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d31857bc0 0x560d3185de10 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009900 tx=0x7f49a802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49b8601740 1 -- 172.21.15.18:0/2563185825 shutdown_connections 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49b8601740 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560d318bfbd0 0x560d318c1ff0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49b8601740 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560d31857bc0 0x560d3185de10 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49b8601740 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d3184efe0 0x560d3185d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f498effd640 0 client.16611 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:58.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49b8601740 1 --2- 172.21.15.18:0/2563185825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560d3184e6c0 0x560d3185d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49b8601740 1 -- 172.21.15.18:0/2563185825 >> 172.21.15.18:0/2563185825 conn(0x560d31848da0 msgr2=0x560d31854280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49b8601740 1 -- 172.21.15.18:0/2563185825 shutdown_connections 2024-02-17T12:52:58.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49b8601740 1 -- 172.21.15.18:0/2563185825 wait complete. 2024-02-17T12:52:58.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.200+0000 7f49b8601740 20 client.16611 trim_cache size 0 max 16384 2024-02-17T12:52:58.198 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:52:58.198 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:52:58.198 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:52:58.198 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:52:58.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.201+0000 7f49b594f640 1 -- 172.21.15.18:0/2243477197 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f49980a2230 msgr2=0x7f49980a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.201+0000 7f49b594f640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f49980a2230 0x7f49980a46f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f49a8005ab0 tx=0x7f49a802c670 comp rx=0 tx=0).stop 2024-02-17T12:52:58.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.201+0000 7f49b594f640 1 -- 172.21.15.18:0/2243477197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49b0102cc0 msgr2=0x7f49b0197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.201+0000 7f49b594f640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49b0102cc0 0x7f49b0197c40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f49a400e9f0 tx=0x7f49a400eeb0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.201+0000 7f49b594f640 1 -- 172.21.15.18:0/2243477197 shutdown_connections 2024-02-17T12:52:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.201+0000 7f49b594f640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f49980a2230 0x7f49980a46f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.201+0000 7f49b594f640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49b010b6e0 0x7f49b019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.201+0000 7f49b594f640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49b0102cc0 0x7f49b0197c40 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.201+0000 7f49b594f640 1 --2- 172.21.15.18:0/2243477197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49b01023a0 0x7f49b0197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.201+0000 7f49b594f640 1 -- 172.21.15.18:0/2243477197 >> 172.21.15.18:0/2243477197 conn(0x7f49b0077230 msgr2=0x7f49b00fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.201+0000 7f49b594f640 1 -- 172.21.15.18:0/2243477197 shutdown_connections 2024-02-17T12:52:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.201+0000 7f49b594f640 1 -- 172.21.15.18:0/2243477197 wait complete. 2024-02-17T12:52:58.203 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:52:58.203 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:52:58.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 131944 10.81 MB/sec execute 13 sec latency 9.564 ms 2024-02-17T12:52:58.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.322+0000 7fc73ff77640 1 -- 172.21.15.18:0/690775182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7381002b0 msgr2=0x7fc738100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.322+0000 7fc73ff77640 1 --2- 172.21.15.18:0/690775182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7381002b0 0x7fc738100690 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc734009930 tx=0x7fc73402bc50 comp rx=0 tx=0).stop 2024-02-17T12:52:58.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.322+0000 7fc73ff77640 1 -- 172.21.15.18:0/690775182 shutdown_connections 2024-02-17T12:52:58.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.322+0000 7fc73ff77640 1 --2- 172.21.15.18:0/690775182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc738105380 0x7fc73810cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.322+0000 7fc73ff77640 1 --2- 172.21.15.18:0/690775182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc738100c60 0x7fc738104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.322+0000 7fc73ff77640 1 --2- 172.21.15.18:0/690775182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7381002b0 0x7fc738100690 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.322+0000 7fc73ff77640 1 -- 172.21.15.18:0/690775182 >> 172.21.15.18:0/690775182 conn(0x7fc738077120 msgr2=0x7fc738077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.322+0000 7fc73ff77640 1 -- 172.21.15.18:0/690775182 shutdown_connections 2024-02-17T12:52:58.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.322+0000 7fc73ff77640 1 -- 172.21.15.18:0/690775182 wait complete. 2024-02-17T12:52:58.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73ff77640 1 Processor -- start 2024-02-17T12:52:58.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73ff77640 1 -- start start 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73ff77640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7381002b0 0x7fc738101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73ff77640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc738105380 0x7fc738102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73ff77640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7381027c0 0x7fc7381a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73ff77640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7381a6060 con 0x7fc7381027c0 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73ff77640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7381a61d0 con 0x7fc7381002b0 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73ff77640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7381a6390 con 0x7fc738105380 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73e514640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7381027c0 0x7fc7381a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73dd13640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7381002b0 0x7fc738101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73e514640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7381027c0 0x7fc7381a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44330/0 (socket says 172.21.15.18:44330) 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73e514640 1 -- 172.21.15.18:0/4140964983 learned_addr learned my addr 172.21.15.18:0/4140964983 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.323+0000 7fc73d512640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc738105380 0x7fc738102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc73e514640 1 -- 172.21.15.18:0/4140964983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc738105380 msgr2=0x7fc738102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc73e514640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc738105380 0x7fc738102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc73e514640 1 -- 172.21.15.18:0/4140964983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7381002b0 msgr2=0x7fc738101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc73e514640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7381002b0 0x7fc738101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc73e514640 1 -- 172.21.15.18:0/4140964983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc734009560 con 0x7fc7381027c0 2024-02-17T12:52:58.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc73dd13640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7381002b0 0x7fc738101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:58.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc73d512640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc738105380 0x7fc738102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:58.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc73e514640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7381027c0 0x7fc7381a5a90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc73000eb10 tx=0x7fc73000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc72effd640 1 -- 172.21.15.18:0/4140964983 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730004b20 con 0x7fc7381027c0 2024-02-17T12:52:58.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc73ff77640 1 -- 172.21.15.18:0/4140964983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7381a6670 con 0x7fc7381027c0 2024-02-17T12:52:58.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc73ff77640 1 -- 172.21.15.18:0/4140964983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7381a6b80 con 0x7fc7381027c0 2024-02-17T12:52:58.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc72effd640 1 -- 172.21.15.18:0/4140964983 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc730004c80 con 0x7fc7381027c0 2024-02-17T12:52:58.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.324+0000 7fc72effd640 1 -- 172.21.15.18:0/4140964983 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730010650 con 0x7fc7381027c0 2024-02-17T12:52:58.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.325+0000 7fc73ff77640 1 -- 172.21.15.18:0/4140964983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7380683c0 con 0x7fc7381027c0 2024-02-17T12:52:58.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.326+0000 7fc72effd640 1 -- 172.21.15.18:0/4140964983 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc730005020 con 0x7fc7381027c0 2024-02-17T12:52:58.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.327+0000 7fc72effd640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc7240a2260 0x7fc7240a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.327+0000 7fc73dd13640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc7240a2260 0x7fc7240a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.327+0000 7fc72effd640 1 -- 172.21.15.18:0/4140964983 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc730014070 con 0x7fc7381027c0 2024-02-17T12:52:58.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.328+0000 7fc73dd13640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc7240a2260 0x7fc7240a4720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc734009900 tx=0x7fc734005a20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.332+0000 7fc72effd640 1 -- 172.21.15.18:0/4140964983 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc7300972c0 con 0x7fc7381027c0 2024-02-17T12:52:58.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.589+0000 7fc73ff77640 1 -- 172.21.15.18:0/4140964983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7381095c0 con 0x7fc7381027c0 2024-02-17T12:52:58.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.590+0000 7fc72effd640 1 -- 172.21.15.18:0/4140964983 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11400 (secure 0 0 0) 0x7fc73009a980 con 0x7fc7381027c0 2024-02-17T12:52:58.582 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:52:58.582 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:52:38.881824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:52:58.582 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-02-17T12:52:58.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.592+0000 7fc73ff77640 1 -- 172.21.15.18:0/4140964983 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc7240a2260 msgr2=0x7fc7240a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.592+0000 7fc73ff77640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc7240a2260 0x7fc7240a4720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc734009900 tx=0x7fc734005a20 comp rx=0 tx=0).stop 2024-02-17T12:52:58.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.592+0000 7fc73ff77640 1 -- 172.21.15.18:0/4140964983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7381027c0 msgr2=0x7fc7381a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.592+0000 7fc73ff77640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7381027c0 0x7fc7381a5a90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc73000eb10 tx=0x7fc73000c2d0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.592+0000 7fc73ff77640 1 -- 172.21.15.18:0/4140964983 shutdown_connections 2024-02-17T12:52:58.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.592+0000 7fc73ff77640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc7240a2260 0x7fc7240a4720 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.592+0000 7fc73ff77640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7381027c0 0x7fc7381a5a90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.592+0000 7fc73ff77640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc738105380 0x7fc738102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.592+0000 7fc73ff77640 1 --2- 172.21.15.18:0/4140964983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7381002b0 0x7fc738101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.592+0000 7fc73ff77640 1 -- 172.21.15.18:0/4140964983 >> 172.21.15.18:0/4140964983 conn(0x7fc738077120 msgr2=0x7fc738069b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.592+0000 7fc73ff77640 1 -- 172.21.15.18:0/4140964983 shutdown_connections 2024-02-17T12:52:58.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.593+0000 7fc73ff77640 1 -- 172.21.15.18:0/4140964983 wait complete. 2024-02-17T12:52:58.594 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:58 smithi018 ceph-mon[32625]: pgmap v95: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-02-17T12:52:58.595 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:52:58.595 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:52:58.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:58 smithi146 ceph-mon[36454]: pgmap v95: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-02-17T12:52:58.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.717+0000 7fe50ad7a640 1 -- 172.21.15.18:0/982809049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe504100310 msgr2=0x7fe5041006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.717+0000 7fe50ad7a640 1 --2- 172.21.15.18:0/982809049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe504100310 0x7fe5041006f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009930 tx=0x7fe4f402bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:58.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.717+0000 7fe50ad7a640 1 -- 172.21.15.18:0/982809049 shutdown_connections 2024-02-17T12:52:58.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.717+0000 7fe50ad7a640 1 --2- 172.21.15.18:0/982809049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5041053e0 0x7fe50410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.717+0000 7fe50ad7a640 1 --2- 172.21.15.18:0/982809049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe504100cc0 0x7fe504104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.717+0000 7fe50ad7a640 1 --2- 172.21.15.18:0/982809049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe504100310 0x7fe5041006f0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.717+0000 7fe50ad7a640 1 -- 172.21.15.18:0/982809049 >> 172.21.15.18:0/982809049 conn(0x7fe5040771e0 msgr2=0x7fe5040775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.717+0000 7fe50ad7a640 1 -- 172.21.15.18:0/982809049 shutdown_connections 2024-02-17T12:52:58.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.717+0000 7fe50ad7a640 1 -- 172.21.15.18:0/982809049 wait complete. 2024-02-17T12:52:58.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe50ad7a640 1 Processor -- start 2024-02-17T12:52:58.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe50ad7a640 1 -- start start 2024-02-17T12:52:58.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe50ad7a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe504100310 0x7fe50419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe50ad7a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5041053e0 0x7fe50419db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe50ad7a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5041976a0 0x7fe504197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe50ad7a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe504198170 con 0x7fe504100310 2024-02-17T12:52:58.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe50ad7a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5041982e0 con 0x7fe5041053e0 2024-02-17T12:52:58.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe50ad7a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5041984a0 con 0x7fe5041976a0 2024-02-17T12:52:58.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe508b16640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe504100310 0x7fe50419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe508b16640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe504100310 0x7fe50419d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44346/0 (socket says 172.21.15.18:44346) 2024-02-17T12:52:58.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe508b16640 1 -- 172.21.15.18:0/698772500 learned_addr learned my addr 172.21.15.18:0/698772500 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe503fff640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5041053e0 0x7fe50419db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.718+0000 7fe509317640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5041976a0 0x7fe504197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.719+0000 7fe508b16640 1 -- 172.21.15.18:0/698772500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5041976a0 msgr2=0x7fe504197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.719+0000 7fe508b16640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5041976a0 0x7fe504197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.719+0000 7fe508b16640 1 -- 172.21.15.18:0/698772500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5041053e0 msgr2=0x7fe50419db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.719+0000 7fe508b16640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5041053e0 0x7fe50419db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.719+0000 7fe508b16640 1 -- 172.21.15.18:0/698772500 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f4009560 con 0x7fe504100310 2024-02-17T12:52:58.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.719+0000 7fe503fff640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5041053e0 0x7fe50419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:58.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.719+0000 7fe508b16640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe504100310 0x7fe50419d5e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009230 tx=0x7fe4f402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.719+0000 7fe501ffb640 1 -- 172.21.15.18:0/698772500 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4004350 con 0x7fe504100310 2024-02-17T12:52:58.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.719+0000 7fe50ad7a640 1 -- 172.21.15.18:0/698772500 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5041a43e0 con 0x7fe504100310 2024-02-17T12:52:58.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.719+0000 7fe501ffb640 1 -- 172.21.15.18:0/698772500 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4f40044b0 con 0x7fe504100310 2024-02-17T12:52:58.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.719+0000 7fe501ffb640 1 -- 172.21.15.18:0/698772500 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4003d70 con 0x7fe504100310 2024-02-17T12:52:58.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.719+0000 7fe50ad7a640 1 -- 172.21.15.18:0/698772500 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5041a48c0 con 0x7fe504100310 2024-02-17T12:52:58.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.722+0000 7fe501ffb640 1 -- 172.21.15.18:0/698772500 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe4f4004df0 con 0x7fe504100310 2024-02-17T12:52:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.722+0000 7fe50da2e740 1 Processor -- start 2024-02-17T12:52:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.722+0000 7fe50da2e740 1 -- start start 2024-02-17T12:52:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe50da2e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 0x56339e3e4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe50da2e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e3f1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe50da2e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3f1b60 0x56339e3f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe501ffb640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe4ec0a2230 0x7fe4ec0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe50da2e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e39f960 con 0x56339e3e3f80 2024-02-17T12:52:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe50da2e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e353310 con 0x56339e3e48a0 2024-02-17T12:52:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe50da2e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e350840 con 0x56339e3f1b60 2024-02-17T12:52:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe509317640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 0x56339e3e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe508b16640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e3f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe509317640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 0x56339e3e4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44356/0 (socket says 172.21.15.18:44356) 2024-02-17T12:52:58.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe509317640 1 -- 172.21.15.18:0/3334644097 learned_addr learned my addr 172.21.15.18:0/3334644097 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe501ffb640 1 -- 172.21.15.18:0/698772500 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe4f40effe0 con 0x7fe504100310 2024-02-17T12:52:58.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe509317640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe4ec0a2230 0x7fe4ec0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe503fff640 1 --2- 172.21.15.18:0/3334644097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3f1b60 0x56339e3f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe508b16640 1 -- 172.21.15.18:0/3334644097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3f1b60 msgr2=0x56339e3f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe508b16640 1 --2- 172.21.15.18:0/3334644097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3f1b60 0x56339e3f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe508b16640 1 -- 172.21.15.18:0/3334644097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 msgr2=0x56339e3e4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe508b16640 1 --2- 172.21.15.18:0/3334644097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 0x56339e3e4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe508b16640 1 -- 172.21.15.18:0/3334644097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56339e30e060 con 0x56339e3e48a0 2024-02-17T12:52:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.723+0000 7fe509317640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe4ec0a2230 0x7fe4ec0a46f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0098b0 tx=0x7fe4fc009600 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.724+0000 7fe508b16640 1 --2- 172.21.15.18:0/3334644097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e3f1620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40ed890 tx=0x7fe4f403a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8245aa1cce3212e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.724+0000 7fe4eaffd640 1 -- 172.21.15.18:0/3334644097 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40bd230 con 0x56339e3e48a0 2024-02-17T12:52:58.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.724+0000 7fe4eaffd640 1 -- 172.21.15.18:0/3334644097 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4f40bcb30 con 0x56339e3e48a0 2024-02-17T12:52:58.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.724+0000 7fe4eaffd640 1 -- 172.21.15.18:0/3334644097 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40b91a0 con 0x56339e3e48a0 2024-02-17T12:52:58.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.724+0000 7fe50da2e740 1 -- 172.21.15.18:0/3334644097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 msgr2=0x56339e3f1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.724+0000 7fe50da2e740 1 --2- 172.21.15.18:0/3334644097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e3f1620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40ed890 tx=0x7fe4f403a430 comp rx=0 tx=0).stop 2024-02-17T12:52:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.724+0000 7fe50da2e740 1 -- 172.21.15.18:0/3334644097 shutdown_connections 2024-02-17T12:52:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.724+0000 7fe50da2e740 1 --2- 172.21.15.18:0/3334644097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3f1b60 0x56339e3f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.724+0000 7fe50da2e740 1 --2- 172.21.15.18:0/3334644097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e3f1620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.724+0000 7fe50da2e740 1 --2- 172.21.15.18:0/3334644097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 0x56339e3e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.724+0000 7fe50da2e740 1 -- 172.21.15.18:0/3334644097 >> 172.21.15.18:0/3334644097 conn(0x56339e3de980 msgr2=0x56339e3ded60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe50da2e740 1 -- 172.21.15.18:0/3334644097 shutdown_connections 2024-02-17T12:52:58.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe50da2e740 1 -- 172.21.15.18:0/3334644097 wait complete. 2024-02-17T12:52:58.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe50da2e740 1 -- start start 2024-02-17T12:52:58.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe50da2e740 1 Processor -- start 2024-02-17T12:52:58.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe50da2e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 0x56339e3eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe50da2e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e3eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe50da2e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3f1b60 0x56339e3ef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe50da2e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e34c020 con 0x56339e3e3f80 2024-02-17T12:52:58.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe50da2e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e01f430 con 0x56339e3e48a0 2024-02-17T12:52:58.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe50da2e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339dfdbe00 con 0x56339e3f1b60 2024-02-17T12:52:58.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe50da2e740 10 client.0 _ll_register_callbacks cb 0x56339e2f3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:58.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe508b16640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 0x56339e3eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe508b16640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 0x56339e3eea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44360/0 (socket says 172.21.15.18:44360) 2024-02-17T12:52:58.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.725+0000 7fe508b16640 1 -- 172.21.15.18:0/64882745 learned_addr learned my addr 172.21.15.18:0/64882745 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe503fff640 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e3eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe509317640 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3f1b60 0x56339e3ef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe508b16640 1 -- 172.21.15.18:0/64882745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3f1b60 msgr2=0x56339e3ef500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe508b16640 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3f1b60 0x56339e3ef500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe508b16640 1 -- 172.21.15.18:0/64882745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 msgr2=0x56339e3eefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe508b16640 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e3eefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe508b16640 1 -- 172.21.15.18:0/64882745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5041a48c0 con 0x56339e3e3f80 2024-02-17T12:52:58.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe508b16640 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 0x56339e3eea80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40edd80 tx=0x7fe4f40bc6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb29d4d4820416b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe4e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:58.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe4e3fff640 1 -- 172.21.15.18:0/64882745 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40baf30 con 0x56339e3e3f80 2024-02-17T12:52:58.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe4e3fff640 1 -- 172.21.15.18:0/64882745 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe4f40c52d0 con 0x56339e3e3f80 2024-02-17T12:52:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe50da2e740 1 -- 172.21.15.18:0/64882745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56339e04b7e0 con 0x56339e3e3f80 2024-02-17T12:52:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe4e3fff640 1 -- 172.21.15.18:0/64882745 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4f40c0c80 con 0x56339e3e3f80 2024-02-17T12:52:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe50da2e740 10 client.16629 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe50da2e740 1 -- 172.21.15.18:0/64882745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56339e34a6a0 con 0x56339e3e3f80 2024-02-17T12:52:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.726+0000 7fe4e3fff640 1 -- 172.21.15.18:0/64882745 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56339e04b7e0 con 0x56339e3e3f80 2024-02-17T12:52:58.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.727+0000 7fe4e3fff640 1 -- 172.21.15.18:0/64882745 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe4f403c070 con 0x56339e3e3f80 2024-02-17T12:52:58.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.727+0000 7fe50da2e740 10 client.16629 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:58.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.727+0000 7fe50da2e740 10 client.16629 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:52:58.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.727+0000 7fe50da2e740 20 client.16629 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:58.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.727+0000 7fe50da2e740 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56339e401750 0x56339e403b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.727+0000 7fe50da2e740 4 client.16629 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.727+0000 7fe50da2e740 1 -- 172.21.15.18:0/64882745 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56339e088720 con 0x56339e401750 2024-02-17T12:52:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.727+0000 7fe503fff640 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56339e401750 0x56339e403b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.727+0000 7fe503fff640 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56339e401750 0x56339e403b80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=edb4e1bf45d1c42c server_cookie=fb4373bf25be47ba in_seq=0 out_seq=0 2024-02-17T12:52:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.727+0000 7fe4e3fff640 10 client.16629 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:52:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe4e3fff640 1 -- 172.21.15.18:0/64882745 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56339e088720 con 0x56339e401750 2024-02-17T12:52:58.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe4e3fff640 10 client.16629 handle_command_reply: tid=0 2024-02-17T12:52:58.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe50da2e740 1 client.16629 shutdown 2024-02-17T12:52:58.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe50da2e740 1 -- 172.21.15.18:0/64882745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 msgr2=0x56339e3eea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe50da2e740 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 0x56339e3eea80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40edd80 tx=0x7fe4f40bc6f0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe50da2e740 1 -- 172.21.15.18:0/64882745 shutdown_connections 2024-02-17T12:52:58.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe50da2e740 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3f1b60 0x56339e3ef500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe50da2e740 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56339e401750 0x56339e403b80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe50da2e740 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e3eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe50da2e740 1 --2- 172.21.15.18:0/64882745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e3f80 0x56339e3eea80 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe4e3fff640 0 client.16629 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:52:58.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe50da2e740 1 -- 172.21.15.18:0/64882745 >> 172.21.15.18:0/64882745 conn(0x56339e3de980 msgr2=0x56339e3e06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe50da2e740 1 -- 172.21.15.18:0/64882745 shutdown_connections 2024-02-17T12:52:58.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe50da2e740 1 -- 172.21.15.18:0/64882745 wait complete. 2024-02-17T12:52:58.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.728+0000 7fe50da2e740 20 client.16629 trim_cache size 0 max 16384 2024-02-17T12:52:58.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe50da2e740 1 Processor -- start 2024-02-17T12:52:58.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe50da2e740 1 -- start start 2024-02-17T12:52:58.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe50da2e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3e3f80 0x56339e3f87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe50da2e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e48a0 0x56339e3f8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe50da2e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3f1b60 0x56339e3ebd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe50da2e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e2d1130 con 0x56339e3e48a0 2024-02-17T12:52:58.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe50da2e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e3e4da0 con 0x56339e3f1b60 2024-02-17T12:52:58.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe50da2e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e3f3f50 con 0x56339e3e3f80 2024-02-17T12:52:58.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe503fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e48a0 0x56339e3f8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe503fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e48a0 0x56339e3f8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44374/0 (socket says 172.21.15.18:44374) 2024-02-17T12:52:58.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe503fff640 1 -- 172.21.15.18:0/3522249598 learned_addr learned my addr 172.21.15.18:0/3522249598 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe509317640 1 --2- 172.21.15.18:0/3522249598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3f1b60 0x56339e3ebd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe508b16640 1 --2- 172.21.15.18:0/3522249598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3e3f80 0x56339e3f87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe503fff640 1 -- 172.21.15.18:0/3522249598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3e3f80 msgr2=0x56339e3f87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe503fff640 1 --2- 172.21.15.18:0/3522249598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3e3f80 0x56339e3f87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe503fff640 1 -- 172.21.15.18:0/3522249598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3f1b60 msgr2=0x56339e3ebd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe503fff640 1 --2- 172.21.15.18:0/3522249598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3f1b60 0x56339e3ebd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.741+0000 7fe503fff640 1 -- 172.21.15.18:0/3522249598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56339e2e6c20 con 0x56339e3e48a0 2024-02-17T12:52:58.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.742+0000 7fe503fff640 1 --2- 172.21.15.18:0/3522249598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e48a0 0x56339e3f8ce0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8012620 tx=0x7fe4f8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3902a600d988c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.742+0000 7fe4ea7fc640 1 -- 172.21.15.18:0/3522249598 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8017040 con 0x56339e3e48a0 2024-02-17T12:52:58.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.742+0000 7fe4ea7fc640 1 -- 172.21.15.18:0/3522249598 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4f8023070 con 0x56339e3e48a0 2024-02-17T12:52:58.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.742+0000 7fe4ea7fc640 1 -- 172.21.15.18:0/3522249598 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f801f2b0 con 0x56339e3e48a0 2024-02-17T12:52:58.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.742+0000 7fe50da2e740 1 -- 172.21.15.18:0/3522249598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e48a0 msgr2=0x56339e3f8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.742+0000 7fe50da2e740 1 --2- 172.21.15.18:0/3522249598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e48a0 0x56339e3f8ce0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8012620 tx=0x7fe4f8005f90 comp rx=0 tx=0).stop 2024-02-17T12:52:58.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.742+0000 7fe50da2e740 1 -- 172.21.15.18:0/3522249598 shutdown_connections 2024-02-17T12:52:58.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.742+0000 7fe50da2e740 1 --2- 172.21.15.18:0/3522249598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3f1b60 0x56339e3ebd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.742+0000 7fe50da2e740 1 --2- 172.21.15.18:0/3522249598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3e48a0 0x56339e3f8ce0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.742+0000 7fe50da2e740 1 --2- 172.21.15.18:0/3522249598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3e3f80 0x56339e3f87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.742+0000 7fe50da2e740 1 -- 172.21.15.18:0/3522249598 >> 172.21.15.18:0/3522249598 conn(0x56339e3de980 msgr2=0x56339e3ddb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe50da2e740 1 -- 172.21.15.18:0/3522249598 shutdown_connections 2024-02-17T12:52:58.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe50da2e740 1 -- 172.21.15.18:0/3522249598 wait complete. 2024-02-17T12:52:58.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe50da2e740 1 -- start start 2024-02-17T12:52:58.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe50da2e740 1 Processor -- start 2024-02-17T12:52:58.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe50da2e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3e3f80 0x56339e451a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe50da2e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e451f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe50da2e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3f1b60 0x56339e44be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe50da2e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e44c4f0 con 0x56339e3f1b60 2024-02-17T12:52:58.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe50da2e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e44c660 con 0x56339e3e48a0 2024-02-17T12:52:58.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe50da2e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e44c820 con 0x56339e3e3f80 2024-02-17T12:52:58.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe50da2e740 10 client.0 _ll_register_callbacks cb 0x7fe4fc0123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:58.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe509317640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3f1b60 0x56339e44be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe509317640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3f1b60 0x56339e44be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44386/0 (socket says 172.21.15.18:44386) 2024-02-17T12:52:58.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe509317640 1 -- 172.21.15.18:0/2322074704 learned_addr learned my addr 172.21.15.18:0/2322074704 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe508b16640 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3e3f80 0x56339e451a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.743+0000 7fe503fff640 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e451f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe509317640 1 -- 172.21.15.18:0/2322074704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3e3f80 msgr2=0x56339e451a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe509317640 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3e3f80 0x56339e451a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe509317640 1 -- 172.21.15.18:0/2322074704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 msgr2=0x56339e451f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe509317640 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e451f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe509317640 1 -- 172.21.15.18:0/2322074704 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f8012040 con 0x56339e3f1b60 2024-02-17T12:52:58.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe508b16640 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3e3f80 0x56339e451a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:58.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe4e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:58.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe509317640 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3f1b60 0x56339e44be50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00ab80 tx=0x7fe4fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183059a8bbdbc051 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe4e3fff640 1 -- 172.21.15.18:0/2322074704 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc023070 con 0x56339e3f1b60 2024-02-17T12:52:58.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe4e3fff640 1 -- 172.21.15.18:0/2322074704 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe4fc01ed00 con 0x56339e3f1b60 2024-02-17T12:52:58.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe50da2e740 1 -- 172.21.15.18:0/2322074704 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4f40bcb30 con 0x56339e3f1b60 2024-02-17T12:52:58.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe4e3fff640 1 -- 172.21.15.18:0/2322074704 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4fc029460 con 0x56339e3f1b60 2024-02-17T12:52:58.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe50da2e740 10 client.16635 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:58.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe50da2e740 1 -- 172.21.15.18:0/2322074704 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56339e30c8c0 con 0x56339e3f1b60 2024-02-17T12:52:58.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe4e3fff640 1 -- 172.21.15.18:0/2322074704 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4f40bcb30 con 0x56339e3f1b60 2024-02-17T12:52:58.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.744+0000 7fe4e3fff640 1 -- 172.21.15.18:0/2322074704 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe4fc028660 con 0x56339e3f1b60 2024-02-17T12:52:58.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 10 client.16635 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:58.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 10 client.16635 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:52:58.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 20 client.16635 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:58.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56339e459950 0x56339e45bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 4 client.16635 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:52:58.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 1 -- 172.21.15.18:0/2322074704 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56339e0d7890 con 0x56339e459950 2024-02-17T12:52:58.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe508b16640 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56339e459950 0x56339e45bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe4e3fff640 10 client.16635 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:52:58.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe508b16640 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56339e459950 0x56339e45bd70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5c462b85948f8bfc server_cookie=2da3a28ced7bd5df in_seq=0 out_seq=0 2024-02-17T12:52:58.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe4e3fff640 1 -- 172.21.15.18:0/2322074704 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56339e0d7890 con 0x56339e459950 2024-02-17T12:52:58.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe4e3fff640 10 client.16635 handle_command_reply: tid=0 2024-02-17T12:52:58.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 1 client.16635 shutdown 2024-02-17T12:52:58.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 1 -- 172.21.15.18:0/2322074704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3f1b60 msgr2=0x56339e44be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3f1b60 0x56339e44be50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00ab80 tx=0x7fe4fc004a40 comp rx=0 tx=0).stop 2024-02-17T12:52:58.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 1 -- 172.21.15.18:0/2322074704 shutdown_connections 2024-02-17T12:52:58.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56339e3f1b60 0x56339e44be50 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56339e459950 0x56339e45bd70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56339e3e48a0 0x56339e451f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 1 --2- 172.21.15.18:0/2322074704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56339e3e3f80 0x56339e451a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe4e3fff640 0 client.16635 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:52:58.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.745+0000 7fe50da2e740 1 -- 172.21.15.18:0/2322074704 >> 172.21.15.18:0/2322074704 conn(0x56339e3de980 msgr2=0x56339e3dfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50da2e740 1 -- 172.21.15.18:0/2322074704 shutdown_connections 2024-02-17T12:52:58.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50da2e740 1 -- 172.21.15.18:0/2322074704 wait complete. 2024-02-17T12:52:58.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50da2e740 20 client.16635 trim_cache size 0 max 16384 2024-02-17T12:52:58.775 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:52:58.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50ad7a640 1 -- 172.21.15.18:0/698772500 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe4ec0a2230 msgr2=0x7fe4ec0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50ad7a640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe4ec0a2230 0x7fe4ec0a46f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0098b0 tx=0x7fe4fc009600 comp rx=0 tx=0).stop 2024-02-17T12:52:58.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50ad7a640 1 -- 172.21.15.18:0/698772500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe504100310 msgr2=0x7fe50419d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50ad7a640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe504100310 0x7fe50419d5e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009230 tx=0x7fe4f402c6c0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50ad7a640 1 -- 172.21.15.18:0/698772500 shutdown_connections 2024-02-17T12:52:58.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50ad7a640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe4ec0a2230 0x7fe4ec0a46f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50ad7a640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5041976a0 0x7fe504197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50ad7a640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5041053e0 0x7fe50419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50ad7a640 1 --2- 172.21.15.18:0/698772500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe504100310 0x7fe50419d5e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50ad7a640 1 -- 172.21.15.18:0/698772500 >> 172.21.15.18:0/698772500 conn(0x7fe5040771e0 msgr2=0x7fe504069a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50ad7a640 1 -- 172.21.15.18:0/698772500 shutdown_connections 2024-02-17T12:52:58.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.746+0000 7fe50ad7a640 1 -- 172.21.15.18:0/698772500 wait complete. 2024-02-17T12:52:58.780 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:52:58.780 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:52:58.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:58 smithi119 ceph-mon[36188]: pgmap v95: 97 pgs: 97 active+clean; 57 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-02-17T12:52:58.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.912+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/558281110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5b8101c10 msgr2=0x7fc5b8105c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.912+0000 7fc5bdfa0640 1 --2- 172.21.15.18:0/558281110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5b8101c10 0x7fc5b8105c50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009930 tx=0x7fc5a002bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:58.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.912+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/558281110 shutdown_connections 2024-02-17T12:52:58.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.912+0000 7fc5bdfa0640 1 --2- 172.21.15.18:0/558281110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5b8106330 0x7fc5b810df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.912+0000 7fc5bdfa0640 1 --2- 172.21.15.18:0/558281110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5b8101c10 0x7fc5b8105c50 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.912+0000 7fc5bdfa0640 1 --2- 172.21.15.18:0/558281110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8101260 0x7fc5b8101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.912+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/558281110 >> 172.21.15.18:0/558281110 conn(0x7fc5b80fcae0 msgr2=0x7fc5b80fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.912+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/558281110 shutdown_connections 2024-02-17T12:52:58.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.912+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/558281110 wait complete. 2024-02-17T12:52:58.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5bdfa0640 1 Processor -- start 2024-02-17T12:52:58.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5bdfa0640 1 -- start start 2024-02-17T12:52:58.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5bdfa0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8101260 0x7fc5b8197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5bdfa0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5b8106330 0x7fc5b8197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5bdfa0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5b819bfa0 0x7fc5b819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5bdfa0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b819c990 con 0x7fc5b819bfa0 2024-02-17T12:52:58.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5bdfa0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b819cb00 con 0x7fc5b8101260 2024-02-17T12:52:58.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5bdfa0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b819cc70 con 0x7fc5b8106330 2024-02-17T12:52:58.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5bd79f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5b819bfa0 0x7fc5b819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5bd79f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5b819bfa0 0x7fc5b819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44402/0 (socket says 172.21.15.18:44402) 2024-02-17T12:52:58.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5bd79f640 1 -- 172.21.15.18:0/470136092 learned_addr learned my addr 172.21.15.18:0/470136092 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5bcf9e640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8101260 0x7fc5b8197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.913+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5b8106330 0x7fc5b8197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.914+0000 7fc5bd79f640 1 -- 172.21.15.18:0/470136092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5b8106330 msgr2=0x7fc5b8197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.914+0000 7fc5bd79f640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5b8106330 0x7fc5b8197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.914+0000 7fc5bd79f640 1 -- 172.21.15.18:0/470136092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8101260 msgr2=0x7fc5b8197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.914+0000 7fc5bd79f640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8101260 0x7fc5b8197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.914+0000 7fc5bd79f640 1 -- 172.21.15.18:0/470136092 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a0009560 con 0x7fc5b819bfa0 2024-02-17T12:52:58.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.914+0000 7fc5bd79f640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5b819bfa0 0x7fc5b819c450 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000e970 tx=0x7fc5b000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.914+0000 7fc5b5ffb640 1 -- 172.21.15.18:0/470136092 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0009720 con 0x7fc5b819bfa0 2024-02-17T12:52:58.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.914+0000 7fc5b5ffb640 1 -- 172.21.15.18:0/470136092 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5b0005500 con 0x7fc5b819bfa0 2024-02-17T12:52:58.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.914+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/470136092 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b81a4250 con 0x7fc5b819bfa0 2024-02-17T12:52:58.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.914+0000 7fc5b5ffb640 1 -- 172.21.15.18:0/470136092 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0010430 con 0x7fc5b819bfa0 2024-02-17T12:52:58.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.914+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/470136092 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b81a4720 con 0x7fc5b819bfa0 2024-02-17T12:52:58.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.916+0000 7fc5b5ffb640 1 -- 172.21.15.18:0/470136092 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc5b0004ff0 con 0x7fc5b819bfa0 2024-02-17T12:52:58.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5c1dd3740 1 Processor -- start 2024-02-17T12:52:58.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5c1dd3740 1 -- start start 2024-02-17T12:52:58.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5b5ffb640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5a40a22b0 0x7fc5a40a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5c1dd3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 0x5579210cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5c1dd3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210ccd60 0x5579210d5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5c1dd3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210d5940 0x5579210d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5a40a22b0 0x7fc5a40a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5c1dd3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557921087960 con 0x5579210cc440 2024-02-17T12:52:58.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5c1dd3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792103b310 con 0x5579210d5940 2024-02-17T12:52:58.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5c1dd3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557921038840 con 0x5579210ccd60 2024-02-17T12:52:58.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5b5ffb640 1 -- 172.21.15.18:0/470136092 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc5b0014070 con 0x7fc5b819bfa0 2024-02-17T12:52:58.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5bcf9e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 0x5579210cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5bcf9e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 0x5579210cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44416/0 (socket says 172.21.15.18:44416) 2024-02-17T12:52:58.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5bcf9e640 1 -- 172.21.15.18:0/2791138731 learned_addr learned my addr 172.21.15.18:0/2791138731 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5bd79f640 1 --2- 172.21.15.18:0/2791138731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210d5940 0x5579210d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.917+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/2791138731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210ccd60 0x5579210d5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.918+0000 7fc5bcf9e640 1 -- 172.21.15.18:0/2791138731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210ccd60 msgr2=0x5579210d5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.918+0000 7fc5bcf9e640 1 --2- 172.21.15.18:0/2791138731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210ccd60 0x5579210d5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.918+0000 7fc5bcf9e640 1 -- 172.21.15.18:0/2791138731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210d5940 msgr2=0x5579210d7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.918+0000 7fc5bcf9e640 1 --2- 172.21.15.18:0/2791138731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210d5940 0x5579210d7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.918+0000 7fc5bcf9e640 1 -- 172.21.15.18:0/2791138731 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557920ff6060 con 0x5579210cc440 2024-02-17T12:52:58.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.918+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5a40a22b0 0x7fc5a40a4770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0000c00 tx=0x7fc5a0004a80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.918+0000 7fc5bcf9e640 1 --2- 172.21.15.18:0/2791138731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 0x5579210cc820 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8003a20 tx=0x7fc5a8003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d482ff2576715bd0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.918+0000 7fc5aeffd640 1 -- 172.21.15.18:0/2791138731 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8011640 con 0x5579210cc440 2024-02-17T12:52:58.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.918+0000 7fc5aeffd640 1 -- 172.21.15.18:0/2791138731 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5a8004860 con 0x5579210cc440 2024-02-17T12:52:58.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.918+0000 7fc5aeffd640 1 -- 172.21.15.18:0/2791138731 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8010b20 con 0x5579210cc440 2024-02-17T12:52:58.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.919+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/2791138731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 msgr2=0x5579210cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.919+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/2791138731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 0x5579210cc820 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8003a20 tx=0x7fc5a8003ee0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.919+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/2791138731 shutdown_connections 2024-02-17T12:52:58.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.919+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/2791138731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210d5940 0x5579210d7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.919+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/2791138731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210ccd60 0x5579210d5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.919+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/2791138731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 0x5579210cc820 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.919+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/2791138731 >> 172.21.15.18:0/2791138731 conn(0x5579210c6b70 msgr2=0x5579210c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.919+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/2791138731 shutdown_connections 2024-02-17T12:52:58.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.919+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/2791138731 wait complete. 2024-02-17T12:52:58.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.919+0000 7fc5c1dd3740 1 -- start start 2024-02-17T12:52:58.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.919+0000 7fc5c1dd3740 1 Processor -- start 2024-02-17T12:52:58.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5c1dd3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 0x5579210d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5c1dd3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210ccd60 0x5579210d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5c1dd3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210d5940 0x5579210d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5c1dd3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557921034020 con 0x5579210cc440 2024-02-17T12:52:58.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5c1dd3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557920d07430 con 0x5579210ccd60 2024-02-17T12:52:58.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5c1dd3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557920cc3e00 con 0x5579210d5940 2024-02-17T12:52:58.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5c1dd3740 10 client.0 _ll_register_callbacks cb 0x557920fdbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:58.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5bcf9e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 0x5579210d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5bcf9e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 0x5579210d2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44418/0 (socket says 172.21.15.18:44418) 2024-02-17T12:52:58.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5bcf9e640 1 -- 172.21.15.18:0/3901188882 learned_addr learned my addr 172.21.15.18:0/3901188882 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5bd79f640 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210d5940 0x5579210d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210ccd60 0x5579210d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5bcf9e640 1 -- 172.21.15.18:0/3901188882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210d5940 msgr2=0x5579210d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5bcf9e640 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210d5940 0x5579210d33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5bcf9e640 1 -- 172.21.15.18:0/3901188882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210ccd60 msgr2=0x5579210d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5bcf9e640 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210ccd60 0x5579210d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5bcf9e640 1 -- 172.21.15.18:0/3901188882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a8009cd0 con 0x5579210cc440 2024-02-17T12:52:58.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5bcf9e640 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 0x5579210d2960 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8000f80 tx=0x7fc5a80105a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a328935fed272ca7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc597fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:58.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc597fff640 1 -- 172.21.15.18:0/3901188882 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8010d30 con 0x5579210cc440 2024-02-17T12:52:58.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc597fff640 1 -- 172.21.15.18:0/3901188882 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc5a8011440 con 0x5579210cc440 2024-02-17T12:52:58.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.920+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/3901188882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557920d337e0 con 0x5579210cc440 2024-02-17T12:52:58.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.921+0000 7fc597fff640 1 -- 172.21.15.18:0/3901188882 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5a8010e90 con 0x5579210cc440 2024-02-17T12:52:58.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.921+0000 7fc5c1dd3740 10 client.16647 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:58.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.921+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/3901188882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5579210326a0 con 0x5579210cc440 2024-02-17T12:52:58.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.921+0000 7fc597fff640 1 -- 172.21.15.18:0/3901188882 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5a8032020 con 0x5579210cc440 2024-02-17T12:52:58.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.921+0000 7fc597fff640 1 -- 172.21.15.18:0/3901188882 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fc5a802e950 con 0x5579210cc440 2024-02-17T12:52:58.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.921+0000 7fc5c1dd3740 10 client.16647 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:58.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.921+0000 7fc5c1dd3740 10 client.16647 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:52:58.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.921+0000 7fc5c1dd3740 20 client.16647 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:58.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.921+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5579210e5c70 0x5579210e8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.921+0000 7fc5c1dd3740 4 client.16647 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:58.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.921+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/3901188882 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557920d70720 con 0x5579210e5c70 2024-02-17T12:52:58.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.921+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5579210e5c70 0x5579210e8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.922+0000 7fc597fff640 10 client.16647 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:52:58.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.922+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5579210e5c70 0x5579210e8060 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b79130ca5bfe910 server_cookie=4585eb0b9ffe21bc in_seq=0 out_seq=0 2024-02-17T12:52:58.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc597fff640 1 -- 172.21.15.18:0/3901188882 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557920d70720 con 0x5579210e5c70 2024-02-17T12:52:58.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc597fff640 10 client.16647 handle_command_reply: tid=0 2024-02-17T12:52:58.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc5c1dd3740 1 client.16647 shutdown 2024-02-17T12:52:58.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/3901188882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 msgr2=0x5579210d2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 0x5579210d2960 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8000f80 tx=0x7fc5a80105a0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/3901188882 shutdown_connections 2024-02-17T12:52:58.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5579210e5c70 0x5579210e8060 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210d5940 0x5579210d33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210ccd60 0x5579210d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/3901188882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210cc440 0x5579210d2960 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc597fff640 0 client.16647 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:52:58.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/3901188882 >> 172.21.15.18:0/3901188882 conn(0x5579210c6b70 msgr2=0x5579210c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/3901188882 shutdown_connections 2024-02-17T12:52:58.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/3901188882 wait complete. 2024-02-17T12:52:58.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.924+0000 7fc5c1dd3740 20 client.16647 trim_cache size 0 max 16384 2024-02-17T12:52:58.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5c1dd3740 1 Processor -- start 2024-02-17T12:52:58.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5c1dd3740 1 -- start start 2024-02-17T12:52:58.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5c1dd3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210cc440 0x5579210c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5c1dd3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 0x5579210dc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5c1dd3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210d5940 0x5579210dcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5c1dd3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557920fb9130 con 0x5579210ccd60 2024-02-17T12:52:58.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5c1dd3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579210d0f10 con 0x5579210cc440 2024-02-17T12:52:58.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5c1dd3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579210c7520 con 0x5579210d5940 2024-02-17T12:52:58.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5b7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 0x5579210dc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5bcf9e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210cc440 0x5579210c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5bcf9e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210cc440 0x5579210c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:49292/0 (socket says 172.21.15.18:49292) 2024-02-17T12:52:58.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5bcf9e640 1 -- 172.21.15.18:0/2276391933 learned_addr learned my addr 172.21.15.18:0/2276391933 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5bd79f640 1 --2- 172.21.15.18:0/2276391933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210d5940 0x5579210dcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5b7fff640 1 -- 172.21.15.18:0/2276391933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210d5940 msgr2=0x5579210dcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/2276391933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210d5940 0x5579210dcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5b7fff640 1 -- 172.21.15.18:0/2276391933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210cc440 msgr2=0x5579210c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/2276391933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210cc440 0x5579210c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5b7fff640 1 -- 172.21.15.18:0/2276391933 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557920fcec20 con 0x5579210ccd60 2024-02-17T12:52:58.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.937+0000 7fc5bcf9e640 1 --2- 172.21.15.18:0/2276391933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210cc440 0x5579210c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:58.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.938+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/2276391933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 0x5579210dc7f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc5a002c170 tx=0x7fc5a0046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb229f14cf722c8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.938+0000 7fc5ae7fc640 1 -- 172.21.15.18:0/2276391933 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0004410 con 0x5579210ccd60 2024-02-17T12:52:58.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.938+0000 7fc5ae7fc640 1 -- 172.21.15.18:0/2276391933 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5a0052630 con 0x5579210ccd60 2024-02-17T12:52:58.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.938+0000 7fc5ae7fc640 1 -- 172.21.15.18:0/2276391933 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0005330 con 0x5579210ccd60 2024-02-17T12:52:58.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.939+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/2276391933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 msgr2=0x5579210dc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.939+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/2276391933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 0x5579210dc7f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc5a002c170 tx=0x7fc5a0046000 comp rx=0 tx=0).stop 2024-02-17T12:52:58.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.939+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/2276391933 shutdown_connections 2024-02-17T12:52:58.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.939+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/2276391933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210d5940 0x5579210dcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.939+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/2276391933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 0x5579210dc7f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.939+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/2276391933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210cc440 0x5579210c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.939+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/2276391933 >> 172.21.15.18:0/2276391933 conn(0x5579210c6b70 msgr2=0x5579210c5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.939+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/2276391933 shutdown_connections 2024-02-17T12:52:58.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.939+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/2276391933 wait complete. 2024-02-17T12:52:58.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.939+0000 7fc5c1dd3740 1 -- start start 2024-02-17T12:52:58.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.939+0000 7fc5c1dd3740 1 Processor -- start 2024-02-17T12:52:58.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5c1dd3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210cc440 0x557921136670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5c1dd3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 0x557921136bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5c1dd3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210d5940 0x5579210dbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5c1dd3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579210dc1f0 con 0x5579210ccd60 2024-02-17T12:52:58.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5c1dd3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579210dc360 con 0x5579210d5940 2024-02-17T12:52:58.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5c1dd3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579210dc520 con 0x5579210cc440 2024-02-17T12:52:58.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5c1dd3740 10 client.0 _ll_register_callbacks cb 0x7fc5a8009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:58.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5b7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 0x557921136bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5b7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 0x557921136bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44438/0 (socket says 172.21.15.18:44438) 2024-02-17T12:52:58.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5b7fff640 1 -- 172.21.15.18:0/1476185074 learned_addr learned my addr 172.21.15.18:0/1476185074 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:58.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5bcf9e640 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210cc440 0x557921136670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5bd79f640 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210d5940 0x5579210dbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5b7fff640 1 -- 172.21.15.18:0/1476185074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210cc440 msgr2=0x557921136670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210cc440 0x557921136670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5b7fff640 1 -- 172.21.15.18:0/1476185074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210d5940 msgr2=0x5579210dbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:58.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210d5940 0x5579210dbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5b7fff640 1 -- 172.21.15.18:0/1476185074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579210326a0 con 0x5579210ccd60 2024-02-17T12:52:58.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5b7fff640 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 0x557921136bb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009900 tx=0x7fc5a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd63046c89e63007 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:58.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc597fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:58.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc597fff640 1 -- 172.21.15.18:0/1476185074 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0004df0 con 0x5579210ccd60 2024-02-17T12:52:58.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc597fff640 1 -- 172.21.15.18:0/1476185074 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc5a00527a0 con 0x5579210ccd60 2024-02-17T12:52:58.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/1476185074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5a8004860 con 0x5579210ccd60 2024-02-17T12:52:58.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc597fff640 1 -- 172.21.15.18:0/1476185074 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5a0052150 con 0x5579210ccd60 2024-02-17T12:52:58.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5c1dd3740 10 client.16653 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:58.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.940+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/1476185074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fc5a0049050 con 0x5579210ccd60 2024-02-17T12:52:58.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.941+0000 7fc597fff640 1 -- 172.21.15.18:0/1476185074 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5a8004860 con 0x5579210ccd60 2024-02-17T12:52:58.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.941+0000 7fc597fff640 1 -- 172.21.15.18:0/1476185074 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fc5a00038f0 con 0x5579210ccd60 2024-02-17T12:52:58.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.941+0000 7fc5c1dd3740 10 client.16653 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:58.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.941+0000 7fc5c1dd3740 10 client.16653 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:52:58.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.941+0000 7fc5c1dd3740 20 client.16653 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:58.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.941+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55792113e200 0x557921140620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:58.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.941+0000 7fc5c1dd3740 4 client.16653 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:52:58.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.941+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/1476185074 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557920dbf890 con 0x55792113e200 2024-02-17T12:52:58.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.941+0000 7fc5bcf9e640 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55792113e200 0x557921140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:58.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.941+0000 7fc5bcf9e640 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55792113e200 0x557921140620 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=29f595d74cab037c server_cookie=f3a0040c4e575d26 in_seq=0 out_seq=0 2024-02-17T12:52:58.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.941+0000 7fc597fff640 10 client.16653 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:52:58.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc597fff640 1 -- 172.21.15.18:0/1476185074 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557920dbf890 con 0x55792113e200 2024-02-17T12:52:58.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc597fff640 10 client.16653 handle_command_reply: tid=0 2024-02-17T12:52:58.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc5c1dd3740 1 client.16653 shutdown 2024-02-17T12:52:58.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/1476185074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 msgr2=0x557921136bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 0x557921136bb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009900 tx=0x7fc5a0005fd0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/1476185074 shutdown_connections 2024-02-17T12:52:58.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55792113e200 0x557921140620 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579210d5940 0x5579210dbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5579210ccd60 0x557921136bb0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc597fff640 0 client.16653 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:52:58.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc5c1dd3740 1 --2- 172.21.15.18:0/1476185074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5579210cc440 0x557921136670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/1476185074 >> 172.21.15.18:0/1476185074 conn(0x5579210c6b70 msgr2=0x5579210c84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/1476185074 shutdown_connections 2024-02-17T12:52:58.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc5c1dd3740 1 -- 172.21.15.18:0/1476185074 wait complete. 2024-02-17T12:52:58.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.943+0000 7fc5c1dd3740 20 client.16653 trim_cache size 0 max 16384 2024-02-17T12:52:58.969 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:52:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.944+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/470136092 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5a40a22b0 msgr2=0x7fc5a40a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.944+0000 7fc5bdfa0640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5a40a22b0 0x7fc5a40a4770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0000c00 tx=0x7fc5a0004a80 comp rx=0 tx=0).stop 2024-02-17T12:52:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.944+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/470136092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5b819bfa0 msgr2=0x7fc5b819c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.944+0000 7fc5bdfa0640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5b819bfa0 0x7fc5b819c450 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000e970 tx=0x7fc5b000ed70 comp rx=0 tx=0).stop 2024-02-17T12:52:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.944+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/470136092 shutdown_connections 2024-02-17T12:52:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.944+0000 7fc5bdfa0640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5a40a22b0 0x7fc5a40a4770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.944+0000 7fc5bdfa0640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5b819bfa0 0x7fc5b819c450 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.944+0000 7fc5bdfa0640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5b8106330 0x7fc5b8197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.944+0000 7fc5bdfa0640 1 --2- 172.21.15.18:0/470136092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8101260 0x7fc5b8197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.944+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/470136092 >> 172.21.15.18:0/470136092 conn(0x7fc5b80fcae0 msgr2=0x7fc5b80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.944+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/470136092 shutdown_connections 2024-02-17T12:52:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:58.944+0000 7fc5bdfa0640 1 -- 172.21.15.18:0/470136092 wait complete. 2024-02-17T12:52:58.972 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:52:58.972 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:52:59.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.089+0000 7f2d575c7640 1 -- 172.21.15.18:0/826332776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d5010db20 msgr2=0x7f2d5010ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.089+0000 7f2d575c7640 1 --2- 172.21.15.18:0/826332776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d5010db20 0x7f2d5010ff10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009ca0 tx=0x7f2d4802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.089+0000 7f2d575c7640 1 -- 172.21.15.18:0/826332776 shutdown_connections 2024-02-17T12:52:59.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.089+0000 7f2d575c7640 1 --2- 172.21.15.18:0/826332776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d5010db20 0x7f2d5010ff10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.089+0000 7f2d575c7640 1 --2- 172.21.15.18:0/826332776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d50068c80 0x7f2d5010d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.089+0000 7f2d575c7640 1 --2- 172.21.15.18:0/826332776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d50068360 0x7f2d50068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.089+0000 7f2d575c7640 1 -- 172.21.15.18:0/826332776 >> 172.21.15.18:0/826332776 conn(0x7f2d50077180 msgr2=0x7f2d50077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.089+0000 7f2d575c7640 1 -- 172.21.15.18:0/826332776 shutdown_connections 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.089+0000 7f2d575c7640 1 -- 172.21.15.18:0/826332776 wait complete. 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.089+0000 7f2d575c7640 1 Processor -- start 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.089+0000 7f2d575c7640 1 -- start start 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d575c7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d50068360 0x7f2d5019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d575c7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d50068c80 0x7f2d5019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d575c7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d5010db20 0x7f2d501997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d575c7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d50199e60 con 0x7f2d50068360 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d575c7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d50199fd0 con 0x7f2d50068c80 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d575c7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5019a190 con 0x7f2d5010db20 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d55363640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d50068360 0x7f2d5019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d55363640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d50068360 0x7f2d5019f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44468/0 (socket says 172.21.15.18:44468) 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d55363640 1 -- 172.21.15.18:0/3493799202 learned_addr learned my addr 172.21.15.18:0/3493799202 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d54b62640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d50068c80 0x7f2d5019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d55b64640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d5010db20 0x7f2d501997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d54b62640 1 -- 172.21.15.18:0/3493799202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d5010db20 msgr2=0x7f2d501997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d54b62640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d5010db20 0x7f2d501997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d54b62640 1 -- 172.21.15.18:0/3493799202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d50068360 msgr2=0x7f2d5019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d54b62640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d50068360 0x7f2d5019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d54b62640 1 -- 172.21.15.18:0/3493799202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c009490 con 0x7f2d50068c80 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d55363640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d50068360 0x7f2d5019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d54b62640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d50068c80 0x7f2d5019fb90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000e9f0 tx=0x7f2d4000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.090+0000 7f2d467fc640 1 -- 172.21.15.18:0/3493799202 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4000ccc0 con 0x7f2d50068c80 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.091+0000 7f2d575c7640 1 -- 172.21.15.18:0/3493799202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d48009900 con 0x7f2d50068c80 2024-02-17T12:52:59.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.091+0000 7f2d467fc640 1 -- 172.21.15.18:0/3493799202 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d4000ce20 con 0x7f2d50068c80 2024-02-17T12:52:59.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.091+0000 7f2d467fc640 1 -- 172.21.15.18:0/3493799202 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40004a00 con 0x7f2d50068c80 2024-02-17T12:52:59.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.091+0000 7f2d575c7640 1 -- 172.21.15.18:0/3493799202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d501a64a0 con 0x7f2d50068c80 2024-02-17T12:52:59.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.093+0000 7f2d467fc640 1 -- 172.21.15.18:0/3493799202 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2d400050b0 con 0x7f2d50068c80 2024-02-17T12:52:59.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.093+0000 7f2d5a198740 1 Processor -- start 2024-02-17T12:52:59.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.093+0000 7f2d5a198740 1 -- start start 2024-02-17T12:52:59.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d5a198740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46250 0x563cbca46630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d5a198740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46b70 0x563cbca4f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d5a198740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbca51b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d5a198740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbca01960 con 0x563cbca46b70 2024-02-17T12:52:59.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d5a198740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbc9b5310 con 0x563cbca46250 2024-02-17T12:52:59.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d467fc640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2d3c0a2230 0x7f2d3c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d5a198740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbc9b2840 con 0x563cbca4f750 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d55363640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46b70 0x563cbca4f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d55363640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46b70 0x563cbca4f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44474/0 (socket says 172.21.15.18:44474) 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d55b64640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46250 0x563cbca46630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d55363640 1 -- 172.21.15.18:0/1053112912 learned_addr learned my addr 172.21.15.18:0/1053112912 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d55b64640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2d3c0a2230 0x7f2d3c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d54b62640 1 --2- 172.21.15.18:0/1053112912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbca51b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d467fc640 1 -- 172.21.15.18:0/3493799202 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2d40014070 con 0x7f2d50068c80 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d55b64640 1 -- 172.21.15.18:0/1053112912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 msgr2=0x563cbca51b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d55b64640 1 --2- 172.21.15.18:0/1053112912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbca51b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d55b64640 1 -- 172.21.15.18:0/1053112912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46b70 msgr2=0x563cbca4f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d55b64640 1 --2- 172.21.15.18:0/1053112912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46b70 0x563cbca4f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d55b64640 1 -- 172.21.15.18:0/1053112912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cbc970060 con 0x563cbca46250 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d55363640 1 --2- 172.21.15.18:0/1053112912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46b70 0x563cbca4f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.094+0000 7f2d55b64640 1 --2- 172.21.15.18:0/1053112912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46250 0x563cbca46630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2d4802c630 tx=0x7f2d480347d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=258a290a0cf0a4b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.095+0000 7f2d55b64640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2d3c0a2230 0x7f2d3c0a46f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d48037040 tx=0x7f2d4803c000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.095+0000 7f2d3b7fe640 1 -- 172.21.15.18:0/1053112912 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48048070 con 0x563cbca46250 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.095+0000 7f2d3b7fe640 1 -- 172.21.15.18:0/1053112912 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d48032b40 con 0x563cbca46250 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.095+0000 7f2d3b7fe640 1 -- 172.21.15.18:0/1053112912 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d480042a0 con 0x563cbca46250 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.095+0000 7f2d5a198740 1 -- 172.21.15.18:0/1053112912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46250 msgr2=0x563cbca46630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.095+0000 7f2d5a198740 1 --2- 172.21.15.18:0/1053112912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46250 0x563cbca46630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2d4802c630 tx=0x7f2d480347d0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.095+0000 7f2d5a198740 1 -- 172.21.15.18:0/1053112912 shutdown_connections 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.095+0000 7f2d5a198740 1 --2- 172.21.15.18:0/1053112912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbca51b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.095+0000 7f2d5a198740 1 --2- 172.21.15.18:0/1053112912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46b70 0x563cbca4f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.095+0000 7f2d5a198740 1 --2- 172.21.15.18:0/1053112912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46250 0x563cbca46630 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.096+0000 7f2d5a198740 1 -- 172.21.15.18:0/1053112912 >> 172.21.15.18:0/1053112912 conn(0x563cbca40980 msgr2=0x563cbca40d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.096+0000 7f2d5a198740 1 -- 172.21.15.18:0/1053112912 shutdown_connections 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.096+0000 7f2d5a198740 1 -- 172.21.15.18:0/1053112912 wait complete. 2024-02-17T12:52:59.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.096+0000 7f2d5a198740 1 -- start start 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.096+0000 7f2d5a198740 1 Processor -- start 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.096+0000 7f2d5a198740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca46250 0x563cbca4c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.096+0000 7f2d5a198740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca4cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.096+0000 7f2d5a198740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca4f750 0x563cbca4d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.096+0000 7f2d5a198740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbc9ae020 con 0x563cbca4f750 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.096+0000 7f2d5a198740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbc681430 con 0x563cbca46b70 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.096+0000 7f2d5a198740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbc63de00 con 0x563cbca46250 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.096+0000 7f2d5a198740 10 client.0 _ll_register_callbacks cb 0x563cbc955cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d55b64640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca4f750 0x563cbca4d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d54b62640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca4cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d54b62640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca4cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:49336/0 (socket says 172.21.15.18:49336) 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d54b62640 1 -- 172.21.15.18:0/1034340877 learned_addr learned my addr 172.21.15.18:0/1034340877 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d55363640 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca46250 0x563cbca4c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d54b62640 1 -- 172.21.15.18:0/1034340877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca46250 msgr2=0x563cbca4c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d54b62640 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca46250 0x563cbca4c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d54b62640 1 -- 172.21.15.18:0/1034340877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca4f750 msgr2=0x563cbca4d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d54b62640 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca4f750 0x563cbca4d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d54b62640 1 -- 172.21.15.18:0/1034340877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d50110a00 con 0x563cbca46b70 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d55b64640 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca4f750 0x563cbca4d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d54b62640 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca4cd50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2d40096e00 tx=0x7f2d4001ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f4dbbb033996e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d38ff9640 1 -- 172.21.15.18:0/1034340877 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4009ae00 con 0x563cbca46b70 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d5a198740 1 -- 172.21.15.18:0/1034340877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cbc6ad7e0 con 0x563cbca46b70 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d38ff9640 1 -- 172.21.15.18:0/1034340877 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2d4009a9a0 con 0x563cbca46b70 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d38ff9640 1 -- 172.21.15.18:0/1034340877 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d400cbb90 con 0x563cbca46b70 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.097+0000 7f2d5a198740 10 client.26036 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.098+0000 7f2d5a198740 1 -- 172.21.15.18:0/1034340877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563cbc9ac6a0 con 0x563cbca46b70 2024-02-17T12:52:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.098+0000 7f2d38ff9640 1 -- 172.21.15.18:0/1034340877 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563cbc6ad7e0 con 0x563cbca46b70 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.098+0000 7f2d38ff9640 1 -- 172.21.15.18:0/1034340877 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2d40095030 con 0x563cbca46b70 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.098+0000 7f2d5a198740 10 client.26036 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.098+0000 7f2d5a198740 10 client.26036 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.098+0000 7f2d5a198740 20 client.26036 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.098+0000 7f2d5a198740 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563cbca5fa60 0x563cbca61e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.098+0000 7f2d5a198740 4 client.26036 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.098+0000 7f2d5a198740 1 -- 172.21.15.18:0/1034340877 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cbc6ea720 con 0x563cbca5fa60 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.098+0000 7f2d55363640 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563cbca5fa60 0x563cbca61e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.098+0000 7f2d38ff9640 10 client.26036 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.098+0000 7f2d55363640 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563cbca5fa60 0x563cbca61e50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=95c20a8c957f5745 server_cookie=f5ef0448664c24dc in_seq=0 out_seq=0 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d38ff9640 1 -- 172.21.15.18:0/1034340877 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563cbc6ea720 con 0x563cbca5fa60 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d38ff9640 10 client.26036 handle_command_reply: tid=0 2024-02-17T12:52:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d5a198740 1 client.26036 shutdown 2024-02-17T12:52:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d5a198740 1 -- 172.21.15.18:0/1034340877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 msgr2=0x563cbca4cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d5a198740 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca4cd50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2d40096e00 tx=0x7f2d4001ce60 comp rx=0 tx=0).stop 2024-02-17T12:52:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d5a198740 1 -- 172.21.15.18:0/1034340877 shutdown_connections 2024-02-17T12:52:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d5a198740 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563cbca5fa60 0x563cbca61e50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d5a198740 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca4f750 0x563cbca4d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d5a198740 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca4cd50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d38ff9640 0 client.26036 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:52:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d5a198740 1 --2- 172.21.15.18:0/1034340877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca46250 0x563cbca4c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d5a198740 1 -- 172.21.15.18:0/1034340877 >> 172.21.15.18:0/1034340877 conn(0x563cbca40980 msgr2=0x563cbca426d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d5a198740 1 -- 172.21.15.18:0/1034340877 shutdown_connections 2024-02-17T12:52:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d5a198740 1 -- 172.21.15.18:0/1034340877 wait complete. 2024-02-17T12:52:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.099+0000 7f2d5a198740 20 client.26036 trim_cache size 0 max 16384 2024-02-17T12:52:59.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d5a198740 1 Processor -- start 2024-02-17T12:52:59.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d5a198740 1 -- start start 2024-02-17T12:52:59.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d5a198740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 0x563cbca4ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d5a198740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca4d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d5a198740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbca49ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d5a198740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbc933130 con 0x563cbca46250 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d5a198740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbca3c5c0 con 0x563cbca46b70 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d5a198740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbca51e70 con 0x563cbca4f750 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d55363640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 0x563cbca4ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d55363640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 0x563cbca4ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44502/0 (socket says 172.21.15.18:44502) 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d55363640 1 -- 172.21.15.18:0/2247564394 learned_addr learned my addr 172.21.15.18:0/2247564394 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d54b62640 1 --2- 172.21.15.18:0/2247564394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca4d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d55b64640 1 --2- 172.21.15.18:0/2247564394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbca49ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d55363640 1 -- 172.21.15.18:0/2247564394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 msgr2=0x563cbca49ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d55363640 1 --2- 172.21.15.18:0/2247564394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbca49ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d55363640 1 -- 172.21.15.18:0/2247564394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 msgr2=0x563cbca4d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d55363640 1 --2- 172.21.15.18:0/2247564394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca4d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.113+0000 7f2d55363640 1 -- 172.21.15.18:0/2247564394 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cbc948c20 con 0x563cbca46250 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.114+0000 7f2d55363640 1 --2- 172.21.15.18:0/2247564394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 0x563cbca4ef40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0127c0 tx=0x7f2d4c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc157ddbfdb65ea server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.114+0000 7f2d3affd640 1 -- 172.21.15.18:0/2247564394 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c012ea0 con 0x563cbca46250 2024-02-17T12:52:59.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.114+0000 7f2d3affd640 1 -- 172.21.15.18:0/2247564394 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d4c018040 con 0x563cbca46250 2024-02-17T12:52:59.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.114+0000 7f2d3affd640 1 -- 172.21.15.18:0/2247564394 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c017040 con 0x563cbca46250 2024-02-17T12:52:59.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.114+0000 7f2d5a198740 1 -- 172.21.15.18:0/2247564394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 msgr2=0x563cbca4ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.114+0000 7f2d5a198740 1 --2- 172.21.15.18:0/2247564394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 0x563cbca4ef40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0127c0 tx=0x7f2d4c005f90 comp rx=0 tx=0).stop 2024-02-17T12:52:59.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.114+0000 7f2d5a198740 1 -- 172.21.15.18:0/2247564394 shutdown_connections 2024-02-17T12:52:59.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.114+0000 7f2d5a198740 1 --2- 172.21.15.18:0/2247564394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbca49ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.114+0000 7f2d5a198740 1 --2- 172.21.15.18:0/2247564394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca4d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 1 --2- 172.21.15.18:0/2247564394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 0x563cbca4ef40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 1 -- 172.21.15.18:0/2247564394 >> 172.21.15.18:0/2247564394 conn(0x563cbca40980 msgr2=0x563cbca4b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 1 -- 172.21.15.18:0/2247564394 shutdown_connections 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 1 -- 172.21.15.18:0/2247564394 wait complete. 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 1 -- start start 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 1 Processor -- start 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 0x563cbca55310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca55850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbcab5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbca56060 con 0x563cbca46250 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbca561d0 con 0x563cbca46b70 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbcab64a0 con 0x563cbca4f750 2024-02-17T12:52:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d5a198740 10 client.0 _ll_register_callbacks cb 0x7f2d4c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:59.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d55363640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 0x563cbca55310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d55363640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 0x563cbca55310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44518/0 (socket says 172.21.15.18:44518) 2024-02-17T12:52:59.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d55363640 1 -- 172.21.15.18:0/2374746873 learned_addr learned my addr 172.21.15.18:0/2374746873 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d54b62640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca55850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.115+0000 7f2d55b64640 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbcab5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d55363640 1 -- 172.21.15.18:0/2374746873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 msgr2=0x563cbcab5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d55363640 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbcab5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d55363640 1 -- 172.21.15.18:0/2374746873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 msgr2=0x563cbca55850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d55363640 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca55850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d55363640 1 -- 172.21.15.18:0/2374746873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c012040 con 0x563cbca46250 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d55b64640 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbcab5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d54b62640 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca55850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d55363640 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 0x563cbca55310 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c000c00 tx=0x7f2d4c013f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f70a9cffd785117 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d38ff9640 1 -- 172.21.15.18:0/2374746873 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0043f0 con 0x563cbca46250 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d5a198740 1 -- 172.21.15.18:0/2374746873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d48032b40 con 0x563cbca46250 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d38ff9640 1 -- 172.21.15.18:0/2374746873 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2d4c0052c0 con 0x563cbca46250 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d38ff9640 1 -- 172.21.15.18:0/2374746873 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d4c005480 con 0x563cbca46250 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d5a198740 10 client.16671 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.116+0000 7f2d5a198740 1 -- 172.21.15.18:0/2374746873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563cbc96e8c0 con 0x563cbca46250 2024-02-17T12:52:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.117+0000 7f2d38ff9640 1 -- 172.21.15.18:0/2374746873 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d48032b40 con 0x563cbca46250 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.117+0000 7f2d38ff9640 1 -- 172.21.15.18:0/2374746873 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2d4c004b90 con 0x563cbca46250 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.117+0000 7f2d5a198740 10 client.16671 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.117+0000 7f2d5a198740 10 client.16671 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.117+0000 7f2d5a198740 20 client.16671 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.117+0000 7f2d5a198740 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563cbcab7c50 0x563cbcaba070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.117+0000 7f2d5a198740 4 client.16671 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.117+0000 7f2d5a198740 1 -- 172.21.15.18:0/2374746873 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563cbc739890 con 0x563cbcab7c50 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.117+0000 7f2d54b62640 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563cbcab7c50 0x563cbcaba070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.117+0000 7f2d38ff9640 10 client.16671 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.117+0000 7f2d54b62640 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563cbcab7c50 0x563cbcaba070 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb41bac563966e92 server_cookie=1c15392b7a642788 in_seq=0 out_seq=0 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d38ff9640 1 -- 172.21.15.18:0/2374746873 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563cbc739890 con 0x563cbcab7c50 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d38ff9640 10 client.16671 handle_command_reply: tid=0 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d5a198740 1 client.16671 shutdown 2024-02-17T12:52:59.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d5a198740 1 -- 172.21.15.18:0/2374746873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 msgr2=0x563cbca55310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d5a198740 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 0x563cbca55310 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c000c00 tx=0x7f2d4c013f40 comp rx=0 tx=0).stop 2024-02-17T12:52:59.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d5a198740 1 -- 172.21.15.18:0/2374746873 shutdown_connections 2024-02-17T12:52:59.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d5a198740 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563cbcab7c50 0x563cbcaba070 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d5a198740 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563cbca4f750 0x563cbcab5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d5a198740 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563cbca46b70 0x563cbca55850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d38ff9640 0 client.16671 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:52:59.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d5a198740 1 --2- 172.21.15.18:0/2374746873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563cbca46250 0x563cbca55310 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d5a198740 1 -- 172.21.15.18:0/2374746873 >> 172.21.15.18:0/2374746873 conn(0x563cbca40980 msgr2=0x563cbca3f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d5a198740 1 -- 172.21.15.18:0/2374746873 shutdown_connections 2024-02-17T12:52:59.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d5a198740 1 -- 172.21.15.18:0/2374746873 wait complete. 2024-02-17T12:52:59.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.118+0000 7f2d5a198740 20 client.16671 trim_cache size 0 max 16384 2024-02-17T12:52:59.118 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:52:59.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.119+0000 7f2d575c7640 1 -- 172.21.15.18:0/3493799202 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2d3c0a2230 msgr2=0x7f2d3c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.119+0000 7f2d575c7640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2d3c0a2230 0x7f2d3c0a46f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d48037040 tx=0x7f2d4803c000 comp rx=0 tx=0).stop 2024-02-17T12:52:59.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.119+0000 7f2d575c7640 1 -- 172.21.15.18:0/3493799202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d50068c80 msgr2=0x7f2d5019fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.119+0000 7f2d575c7640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d50068c80 0x7f2d5019fb90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000e9f0 tx=0x7f2d4000eeb0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.119+0000 7f2d575c7640 1 -- 172.21.15.18:0/3493799202 shutdown_connections 2024-02-17T12:52:59.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.119+0000 7f2d575c7640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2d3c0a2230 0x7f2d3c0a46f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.119+0000 7f2d575c7640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d5010db20 0x7f2d501997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.119+0000 7f2d575c7640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d50068c80 0x7f2d5019fb90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.119+0000 7f2d575c7640 1 --2- 172.21.15.18:0/3493799202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d50068360 0x7f2d5019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.119+0000 7f2d575c7640 1 -- 172.21.15.18:0/3493799202 >> 172.21.15.18:0/3493799202 conn(0x7f2d50077180 msgr2=0x7f2d5010f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.119+0000 7f2d575c7640 1 -- 172.21.15.18:0/3493799202 shutdown_connections 2024-02-17T12:52:59.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.119+0000 7f2d575c7640 1 -- 172.21.15.18:0/3493799202 wait complete. 2024-02-17T12:52:59.121 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:52:59.121 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:52:59.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.242+0000 7fa49c302640 1 -- 172.21.15.18:0/1014395072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa494100c10 msgr2=0x7fa494104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.242+0000 7fa49c302640 1 --2- 172.21.15.18:0/1014395072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa494100c10 0x7fa494104c50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa484009930 tx=0x7fa48402bc30 comp rx=0 tx=0).stop 2024-02-17T12:52:59.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.242+0000 7fa49c302640 1 -- 172.21.15.18:0/1014395072 shutdown_connections 2024-02-17T12:52:59.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.242+0000 7fa49c302640 1 --2- 172.21.15.18:0/1014395072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494105330 0x7fa49410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.242+0000 7fa49c302640 1 --2- 172.21.15.18:0/1014395072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa494100c10 0x7fa494104c50 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.242+0000 7fa49c302640 1 --2- 172.21.15.18:0/1014395072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa494100260 0x7fa494100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.242+0000 7fa49c302640 1 -- 172.21.15.18:0/1014395072 >> 172.21.15.18:0/1014395072 conn(0x7fa494077180 msgr2=0x7fa494077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.242+0000 7fa49c302640 1 -- 172.21.15.18:0/1014395072 shutdown_connections 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.242+0000 7fa49c302640 1 -- 172.21.15.18:0/1014395072 wait complete. 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.242+0000 7fa49c302640 1 Processor -- start 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.242+0000 7fa49c302640 1 -- start start 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49c302640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494100260 0x7fa494198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49c302640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa494105330 0x7fa494198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49c302640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4941990d0 0x7fa4941a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49c302640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941a3d50 con 0x7fa494105330 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49c302640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941a3ec0 con 0x7fa4941990d0 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49c302640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941a4080 con 0x7fa494100260 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49989d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa494105330 0x7fa494198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49989d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa494105330 0x7fa494198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44540/0 (socket says 172.21.15.18:44540) 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49a09e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494100260 0x7fa494198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49989d640 1 -- 172.21.15.18:0/3078605336 learned_addr learned my addr 172.21.15.18:0/3078605336 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49a89f640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4941990d0 0x7fa4941a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49989d640 1 -- 172.21.15.18:0/3078605336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494100260 msgr2=0x7fa494198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49989d640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494100260 0x7fa494198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49989d640 1 -- 172.21.15.18:0/3078605336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4941990d0 msgr2=0x7fa4941a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49989d640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4941990d0 0x7fa4941a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49989d640 1 -- 172.21.15.18:0/3078605336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa484009560 con 0x7fa494105330 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49989d640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa494105330 0x7fa494198b40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa484005ab0 tx=0x7fa48402cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa48b7fe640 1 -- 172.21.15.18:0/3078605336 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484004350 con 0x7fa494105330 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa48b7fe640 1 -- 172.21.15.18:0/3078605336 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4840044b0 con 0x7fa494105330 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.243+0000 7fa49c302640 1 -- 172.21.15.18:0/3078605336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4941a4300 con 0x7fa494105330 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.244+0000 7fa48b7fe640 1 -- 172.21.15.18:0/3078605336 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484003cc0 con 0x7fa494105330 2024-02-17T12:52:59.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.244+0000 7fa49c302640 1 -- 172.21.15.18:0/3078605336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4941a47e0 con 0x7fa494105330 2024-02-17T12:52:59.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.245+0000 7fa48b7fe640 1 -- 172.21.15.18:0/3078605336 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa484004df0 con 0x7fa494105330 2024-02-17T12:52:59.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.246+0000 7fa48b7fe640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa4800a2230 0x7fa4800a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.246+0000 7fa49efb2740 1 Processor -- start 2024-02-17T12:52:59.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.246+0000 7fa48b7fe640 1 -- 172.21.15.18:0/3078605336 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa4840f0630 con 0x7fa494105330 2024-02-17T12:52:59.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.246+0000 7fa49a09e640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa4800a2230 0x7fa4800a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49efb2740 1 -- start start 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49efb2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49efb2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658aeb70 0x558c658b7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49efb2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c658b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49efb2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c65869960 con 0x558c658ae250 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49efb2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c6581d310 con 0x558c658b7750 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49efb2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c6581a840 con 0x558c658aeb70 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49a09e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49a09e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44554/0 (socket says 172.21.15.18:44554) 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49a09e640 1 -- 172.21.15.18:0/2519223018 learned_addr learned my addr 172.21.15.18:0/2519223018 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49a89f640 1 --2- 172.21.15.18:0/2519223018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c658b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49989d640 1 --2- 172.21.15.18:0/2519223018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658aeb70 0x558c658b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49a09e640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa4800a2230 0x7fa4800a46f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa490005d00 tx=0x7fa49000b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49a09e640 1 -- 172.21.15.18:0/2519223018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658aeb70 msgr2=0x558c658b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49a09e640 1 --2- 172.21.15.18:0/2519223018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658aeb70 0x558c658b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49a09e640 1 -- 172.21.15.18:0/2519223018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 msgr2=0x558c658b9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49a09e640 1 --2- 172.21.15.18:0/2519223018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c658b9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49a09e640 1 -- 172.21.15.18:0/2519223018 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c657d8060 con 0x558c658ae250 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49a89f640 1 --2- 172.21.15.18:0/2519223018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c658b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.247+0000 7fa49989d640 1 --2- 172.21.15.18:0/2519223018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658aeb70 0x558c658b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.248+0000 7fa49a09e640 1 --2- 172.21.15.18:0/2519223018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658ae630 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa49000bff0 tx=0x7fa490013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d972d1079c2af3da server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.248+0000 7fa488bfe640 1 -- 172.21.15.18:0/2519223018 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49001e620 con 0x558c658ae250 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.248+0000 7fa488bfe640 1 -- 172.21.15.18:0/2519223018 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa490015070 con 0x558c658ae250 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.248+0000 7fa488bfe640 1 -- 172.21.15.18:0/2519223018 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49001db90 con 0x558c658ae250 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 -- 172.21.15.18:0/2519223018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 msgr2=0x558c658ae630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 --2- 172.21.15.18:0/2519223018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658ae630 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa49000bff0 tx=0x7fa490013000 comp rx=0 tx=0).stop 2024-02-17T12:52:59.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 -- 172.21.15.18:0/2519223018 shutdown_connections 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 --2- 172.21.15.18:0/2519223018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c658b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 --2- 172.21.15.18:0/2519223018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658aeb70 0x558c658b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 --2- 172.21.15.18:0/2519223018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658ae630 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 -- 172.21.15.18:0/2519223018 >> 172.21.15.18:0/2519223018 conn(0x558c658a8980 msgr2=0x558c658a8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 -- 172.21.15.18:0/2519223018 shutdown_connections 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 -- 172.21.15.18:0/2519223018 wait complete. 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 -- start start 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 Processor -- start 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658aeb70 0x558c658b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658b7750 0x558c658b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c65816020 con 0x558c658ae250 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c654e9430 con 0x558c658aeb70 2024-02-17T12:52:59.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c654a5e00 con 0x558c658b7750 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.249+0000 7fa49efb2740 10 client.0 _ll_register_callbacks cb 0x558c657bdcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa49a09e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa49a09e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44564/0 (socket says 172.21.15.18:44564) 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa49a09e640 1 -- 172.21.15.18:0/4000663597 learned_addr learned my addr 172.21.15.18:0/4000663597 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa49a89f640 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658b7750 0x558c658b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa49989d640 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658aeb70 0x558c658b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa49a89f640 1 -- 172.21.15.18:0/4000663597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658aeb70 msgr2=0x558c658b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa49a89f640 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658aeb70 0x558c658b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa49a89f640 1 -- 172.21.15.18:0/4000663597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 msgr2=0x558c658b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa49a89f640 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa49a89f640 1 -- 172.21.15.18:0/4000663597 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49000bca0 con 0x558c658b7750 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa49a09e640 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa49a89f640 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658b7750 0x558c658b51a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa48c005fd0 tx=0x7fa48c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86a24e3978f45ec4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa47dffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:52:59.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.250+0000 7fa47dffb640 1 -- 172.21.15.18:0/4000663597 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c016070 con 0x558c658b7750 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.251+0000 7fa49efb2740 1 -- 172.21.15.18:0/4000663597 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c655157e0 con 0x558c658b7750 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.251+0000 7fa47dffb640 1 -- 172.21.15.18:0/4000663597 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa48c009e30 con 0x558c658b7750 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.251+0000 7fa47dffb640 1 -- 172.21.15.18:0/4000663597 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa48c004b20 con 0x558c658b7750 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.251+0000 7fa49efb2740 10 client.26080 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.251+0000 7fa49efb2740 1 -- 172.21.15.18:0/4000663597 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558c658146a0 con 0x558c658b7750 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.251+0000 7fa47dffb640 1 -- 172.21.15.18:0/4000663597 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa48c027080 con 0x558c658b7750 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.251+0000 7fa47dffb640 1 -- 172.21.15.18:0/4000663597 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa48c0044c0 con 0x558c658b7750 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.251+0000 7fa49efb2740 10 client.26080 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.251+0000 7fa49efb2740 10 client.26080 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.252+0000 7fa49efb2740 20 client.26080 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.252+0000 7fa49efb2740 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x558c658c7b20 0x558c658c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.252+0000 7fa49efb2740 4 client.26080 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:59.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.252+0000 7fa49efb2740 1 -- 172.21.15.18:0/4000663597 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c65552720 con 0x558c658c7b20 2024-02-17T12:52:59.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.252+0000 7fa49a09e640 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x558c658c7b20 0x558c658c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.253+0000 7fa49a09e640 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x558c658c7b20 0x558c658c9f40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9a326e906163c24 server_cookie=9142325bdcea0323 in_seq=0 out_seq=0 2024-02-17T12:52:59.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.253+0000 7fa47dffb640 10 client.26080 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:52:59.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.254+0000 7fa47dffb640 1 -- 172.21.15.18:0/4000663597 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c65552720 con 0x558c658c7b20 2024-02-17T12:52:59.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.254+0000 7fa47dffb640 10 client.26080 handle_command_reply: tid=0 2024-02-17T12:52:59.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.254+0000 7fa49efb2740 1 client.26080 shutdown 2024-02-17T12:52:59.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.254+0000 7fa49efb2740 1 -- 172.21.15.18:0/4000663597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658b7750 msgr2=0x558c658b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.254+0000 7fa49efb2740 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658b7750 0x558c658b51a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa48c005fd0 tx=0x7fa48c00f790 comp rx=0 tx=0).stop 2024-02-17T12:52:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.254+0000 7fa49efb2740 1 -- 172.21.15.18:0/4000663597 shutdown_connections 2024-02-17T12:52:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.254+0000 7fa49efb2740 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x558c658c7b20 0x558c658c9f40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.254+0000 7fa49efb2740 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658b7750 0x558c658b51a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.254+0000 7fa49efb2740 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658aeb70 0x558c658b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.254+0000 7fa47dffb640 0 client.26080 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:52:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.254+0000 7fa49efb2740 1 --2- 172.21.15.18:0/4000663597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.254+0000 7fa49efb2740 1 -- 172.21.15.18:0/4000663597 >> 172.21.15.18:0/4000663597 conn(0x558c658a8980 msgr2=0x558c658aa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.255+0000 7fa49efb2740 1 -- 172.21.15.18:0/4000663597 shutdown_connections 2024-02-17T12:52:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.255+0000 7fa49efb2740 1 -- 172.21.15.18:0/4000663597 wait complete. 2024-02-17T12:52:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.255+0000 7fa49efb2740 20 client.26080 trim_cache size 0 max 16384 2024-02-17T12:52:59.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 134483 10.98 MB/sec execute 14 sec latency 27.015 ms 2024-02-17T12:52:59.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.279+0000 7fa49efb2740 1 Processor -- start 2024-02-17T12:52:59.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.279+0000 7fa49efb2740 1 -- start start 2024-02-17T12:52:59.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.279+0000 7fa49efb2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.279+0000 7fa49efb2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658aeb70 0x558c658b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.279+0000 7fa49efb2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c658b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.279+0000 7fa49efb2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c6579b130 con 0x558c658ae250 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.279+0000 7fa49efb2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c658c78f0 con 0x558c658b7750 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.279+0000 7fa49efb2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c658cf150 con 0x558c658aeb70 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.280+0000 7fa49a09e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.280+0000 7fa49a09e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44566/0 (socket says 172.21.15.18:44566) 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.280+0000 7fa49a09e640 1 -- 172.21.15.18:0/4114338832 learned_addr learned my addr 172.21.15.18:0/4114338832 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.280+0000 7fa49a89f640 1 --2- 172.21.15.18:0/4114338832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c658b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.280+0000 7fa49989d640 1 --2- 172.21.15.18:0/4114338832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658aeb70 0x558c658b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.280+0000 7fa49a89f640 1 -- 172.21.15.18:0/4114338832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658aeb70 msgr2=0x558c658b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.280+0000 7fa49a89f640 1 --2- 172.21.15.18:0/4114338832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658aeb70 0x558c658b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.280+0000 7fa49a89f640 1 -- 172.21.15.18:0/4114338832 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 msgr2=0x558c658b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.280+0000 7fa49a89f640 1 --2- 172.21.15.18:0/4114338832 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658b6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.280+0000 7fa49a89f640 1 -- 172.21.15.18:0/4114338832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c657b0c20 con 0x558c658b7750 2024-02-17T12:52:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.280+0000 7fa49a89f640 1 --2- 172.21.15.18:0/4114338832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c658b19e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00fd10 tx=0x7fa48c009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8455c87ba665797c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.281+0000 7fa47ffff640 1 -- 172.21.15.18:0/4114338832 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c016040 con 0x558c658b7750 2024-02-17T12:52:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.281+0000 7fa47ffff640 1 -- 172.21.15.18:0/4114338832 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558c655157e0 con 0x558c658b7750 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.281+0000 7fa47ffff640 1 -- 172.21.15.18:0/4114338832 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c0057d0 con 0x558c658b7750 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.282+0000 7fa49efb2740 1 -- 172.21.15.18:0/4114338832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 msgr2=0x558c658b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.282+0000 7fa49efb2740 1 --2- 172.21.15.18:0/4114338832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c658b19e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00fd10 tx=0x7fa48c009570 comp rx=0 tx=0).stop 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.282+0000 7fa49efb2740 1 -- 172.21.15.18:0/4114338832 shutdown_connections 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.282+0000 7fa49efb2740 1 --2- 172.21.15.18:0/4114338832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c658b19e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.282+0000 7fa49efb2740 1 --2- 172.21.15.18:0/4114338832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658aeb70 0x558c658b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.282+0000 7fa49efb2740 1 --2- 172.21.15.18:0/4114338832 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658ae250 0x558c658b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.282+0000 7fa49efb2740 1 -- 172.21.15.18:0/4114338832 >> 172.21.15.18:0/4114338832 conn(0x558c658a8980 msgr2=0x558c658afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.282+0000 7fa49efb2740 1 -- 172.21.15.18:0/4114338832 shutdown_connections 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.282+0000 7fa49efb2740 1 -- 172.21.15.18:0/4114338832 wait complete. 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.282+0000 7fa49efb2740 1 -- start start 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.282+0000 7fa49efb2740 1 Processor -- start 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49efb2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658ae250 0x558c658bd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49efb2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658aeb70 0x558c658bddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49efb2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c65918000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49efb2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c65918540 con 0x558c658aeb70 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49efb2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c659186b0 con 0x558c658b7750 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49efb2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c65918870 con 0x558c658ae250 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49989d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658aeb70 0x558c658bddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49a09e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658ae250 0x558c658bd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49efb2740 10 client.0 _ll_register_callbacks cb 0x7fa49000b890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49a09e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658ae250 0x558c658bd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:58168/0 (socket says 172.21.15.18:58168) 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49a09e640 1 -- 172.21.15.18:0/3089835663 learned_addr learned my addr 172.21.15.18:0/3089835663 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49a89f640 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c65918000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49989d640 1 -- 172.21.15.18:0/3089835663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658ae250 msgr2=0x558c658bd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49989d640 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658ae250 0x558c658bd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49989d640 1 -- 172.21.15.18:0/3089835663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 msgr2=0x558c65918000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49989d640 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c65918000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49989d640 1 -- 172.21.15.18:0/3089835663 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c658146a0 con 0x558c658aeb70 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49a89f640 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c65918000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49a09e640 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658ae250 0x558c658bd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa49989d640 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658aeb70 0x558c658bddc0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa4840edee0 tx=0x7fa4840bcc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8cd6ebe0b110e44 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.283+0000 7fa47dffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa47dffb640 1 -- 172.21.15.18:0/3089835663 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4840bd210 con 0x558c658aeb70 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa47dffb640 1 -- 172.21.15.18:0/3089835663 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa4840c1360 con 0x558c658aeb70 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa49efb2740 1 -- 172.21.15.18:0/3089835663 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa490015070 con 0x558c658aeb70 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa47dffb640 1 -- 172.21.15.18:0/3089835663 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4840c5920 con 0x558c658aeb70 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa49efb2740 10 client.16689 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa49efb2740 1 -- 172.21.15.18:0/3089835663 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558c657d68c0 con 0x558c658aeb70 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa47dffb640 1 -- 172.21.15.18:0/3089835663 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4840db020 con 0x558c658aeb70 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa47dffb640 1 -- 172.21.15.18:0/3089835663 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa48403c070 con 0x558c658aeb70 2024-02-17T12:52:59.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa49efb2740 10 client.16689 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa49efb2740 10 client.16689 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa49efb2740 20 client.16689 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa49efb2740 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x558c658c7b20 0x558c65921fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa49efb2740 4 client.16689 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.284+0000 7fa49efb2740 1 -- 172.21.15.18:0/3089835663 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558c655a1890 con 0x558c658c7b20 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49a09e640 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x558c658c7b20 0x558c65921fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49a09e640 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x558c658c7b20 0x558c65921fe0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5bf8e0c6f122c52 server_cookie=9afb738278cd95bd in_seq=0 out_seq=0 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa47dffb640 10 client.16689 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa47dffb640 1 -- 172.21.15.18:0/3089835663 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558c655a1890 con 0x558c658c7b20 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa47dffb640 10 client.16689 handle_command_reply: tid=0 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49efb2740 1 client.16689 shutdown 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49efb2740 1 -- 172.21.15.18:0/3089835663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658aeb70 msgr2=0x558c658bddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49efb2740 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658aeb70 0x558c658bddc0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa4840edee0 tx=0x7fa4840bcc40 comp rx=0 tx=0).stop 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49efb2740 1 -- 172.21.15.18:0/3089835663 shutdown_connections 2024-02-17T12:52:59.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49efb2740 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x558c658c7b20 0x558c65921fe0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49efb2740 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c658b7750 0x558c65918000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa47dffb640 0 client.16689 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:52:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49efb2740 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c658aeb70 0x558c658bddc0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49efb2740 1 --2- 172.21.15.18:0/3089835663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c658ae250 0x558c658bd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49efb2740 1 -- 172.21.15.18:0/3089835663 >> 172.21.15.18:0/3089835663 conn(0x558c658a8980 msgr2=0x558c658a72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49efb2740 1 -- 172.21.15.18:0/3089835663 shutdown_connections 2024-02-17T12:52:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49efb2740 1 -- 172.21.15.18:0/3089835663 wait complete. 2024-02-17T12:52:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.285+0000 7fa49efb2740 20 client.16689 trim_cache size 0 max 16384 2024-02-17T12:52:59.282 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:52:59.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.286+0000 7fa49c302640 1 -- 172.21.15.18:0/3078605336 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa4800a2230 msgr2=0x7fa4800a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.286+0000 7fa49c302640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa4800a2230 0x7fa4800a46f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa490005d00 tx=0x7fa49000b410 comp rx=0 tx=0).stop 2024-02-17T12:52:59.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.286+0000 7fa49c302640 1 -- 172.21.15.18:0/3078605336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa494105330 msgr2=0x7fa494198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.286+0000 7fa49c302640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa494105330 0x7fa494198b40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa484005ab0 tx=0x7fa48402cd80 comp rx=0 tx=0).stop 2024-02-17T12:52:59.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.286+0000 7fa49c302640 1 -- 172.21.15.18:0/3078605336 shutdown_connections 2024-02-17T12:52:59.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.286+0000 7fa49c302640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa4800a2230 0x7fa4800a46f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.286+0000 7fa49c302640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4941990d0 0x7fa4941a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.286+0000 7fa49c302640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa494105330 0x7fa494198b40 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.286+0000 7fa49c302640 1 --2- 172.21.15.18:0/3078605336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494100260 0x7fa494198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.286+0000 7fa49c302640 1 -- 172.21.15.18:0/3078605336 >> 172.21.15.18:0/3078605336 conn(0x7fa494077180 msgr2=0x7fa4940fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.286+0000 7fa49c302640 1 -- 172.21.15.18:0/3078605336 shutdown_connections 2024-02-17T12:52:59.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.286+0000 7fa49c302640 1 -- 172.21.15.18:0/3078605336 wait complete. 2024-02-17T12:52:59.288 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:52:59.288 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:52:59.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.405+0000 7f8236a0b640 1 -- 172.21.15.18:0/3710514817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8230106450 msgr2=0x7f823010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.405+0000 7f8236a0b640 1 --2- 172.21.15.18:0/3710514817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8230106450 0x7f823010e020 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8228009cd0 tx=0x7f822802c0e0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.405+0000 7f8236a0b640 1 -- 172.21.15.18:0/3710514817 shutdown_connections 2024-02-17T12:52:59.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.405+0000 7f8236a0b640 1 --2- 172.21.15.18:0/3710514817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8230106450 0x7f823010e020 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.405+0000 7f8236a0b640 1 --2- 172.21.15.18:0/3710514817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8230101d30 0x7f8230105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.405+0000 7f8236a0b640 1 --2- 172.21.15.18:0/3710514817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230101380 0x7f8230101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.405+0000 7f8236a0b640 1 -- 172.21.15.18:0/3710514817 >> 172.21.15.18:0/3710514817 conn(0x7f8230078580 msgr2=0x7f8230078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.405+0000 7f8236a0b640 1 -- 172.21.15.18:0/3710514817 shutdown_connections 2024-02-17T12:52:59.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.405+0000 7f8236a0b640 1 -- 172.21.15.18:0/3710514817 wait complete. 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.406+0000 7f8236a0b640 1 Processor -- start 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.406+0000 7f8236a0b640 1 -- start start 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.406+0000 7f8236a0b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8230101380 0x7f823019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.406+0000 7f8236a0b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230106450 0x7f823019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.406+0000 7f8236a0b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82301975a0 0x7f8230197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.406+0000 7f8236a0b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8230197f90 con 0x7f82301975a0 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.406+0000 7f8236a0b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8230198100 con 0x7f8230101380 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.406+0000 7f8236a0b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82301982c0 con 0x7f8230106450 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.406+0000 7f8234fa8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82301975a0 0x7f8230197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.406+0000 7f822ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8230101380 0x7f823019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.406+0000 7f8234fa8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82301975a0 0x7f8230197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44584/0 (socket says 172.21.15.18:44584) 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f8234fa8640 1 -- 172.21.15.18:0/360503944 learned_addr learned my addr 172.21.15.18:0/360503944 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f822f7fe640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230106450 0x7f823019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f8234fa8640 1 -- 172.21.15.18:0/360503944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230106450 msgr2=0x7f823019d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f8234fa8640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230106450 0x7f823019d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f8234fa8640 1 -- 172.21.15.18:0/360503944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8230101380 msgr2=0x7f823019d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f8234fa8640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8230101380 0x7f823019d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f8234fa8640 1 -- 172.21.15.18:0/360503944 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8220009490 con 0x7f82301975a0 2024-02-17T12:52:59.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f822ffff640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8230101380 0x7f823019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f8234fa8640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82301975a0 0x7f8230197a50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8228004af0 tx=0x7f8228032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f822d7fa640 1 -- 172.21.15.18:0/360503944 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280328b0 con 0x7f82301975a0 2024-02-17T12:52:59.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f8236a0b640 1 -- 172.21.15.18:0/360503944 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8228009900 con 0x7f82301975a0 2024-02-17T12:52:59.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f8236a0b640 1 -- 172.21.15.18:0/360503944 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82301a45d0 con 0x7f82301975a0 2024-02-17T12:52:59.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f822d7fa640 1 -- 172.21.15.18:0/360503944 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8228003e50 con 0x7f82301975a0 2024-02-17T12:52:59.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.407+0000 7f822d7fa640 1 -- 172.21.15.18:0/360503944 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228004760 con 0x7f82301975a0 2024-02-17T12:52:59.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.409+0000 7f822d7fa640 1 -- 172.21.15.18:0/360503944 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f822800a040 con 0x7f82301975a0 2024-02-17T12:52:59.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f822d7fa640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82180a2230 0x7f82180a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f82395d7740 1 Processor -- start 2024-02-17T12:52:59.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f82395d7740 1 -- start start 2024-02-17T12:52:59.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f822ffff640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82180a2230 0x7f82180a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f822d7fa640 1 -- 172.21.15.18:0/360503944 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f82280f03f0 con 0x7f82301975a0 2024-02-17T12:52:59.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f82395d7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f82395d7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 0x561c1d3d7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f82395d7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3d7750 0x561c1d3d9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f82395d7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1d389960 con 0x561c1d3ceb70 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f82395d7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1d33d310 con 0x561c1d3ce250 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f82395d7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1d33a840 con 0x561c1d3d7750 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f822f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 0x561c1d3d7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f822f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 0x561c1d3d7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44594/0 (socket says 172.21.15.18:44594) 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.410+0000 7f822f7fe640 1 -- 172.21.15.18:0/3130083783 learned_addr learned my addr 172.21.15.18:0/3130083783 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.411+0000 7f822ffff640 1 --2- 172.21.15.18:0/3130083783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.411+0000 7f8234fa8640 1 --2- 172.21.15.18:0/3130083783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3d7750 0x561c1d3d9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.411+0000 7f822ffff640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82180a2230 0x7f82180a46f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8224011490 tx=0x7f8224011890 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.411+0000 7f822f7fe640 1 -- 172.21.15.18:0/3130083783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3d7750 msgr2=0x561c1d3d9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.411+0000 7f822f7fe640 1 --2- 172.21.15.18:0/3130083783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3d7750 0x561c1d3d9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.411+0000 7f822f7fe640 1 -- 172.21.15.18:0/3130083783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 msgr2=0x561c1d3ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.411+0000 7f822f7fe640 1 --2- 172.21.15.18:0/3130083783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.411+0000 7f822f7fe640 1 -- 172.21.15.18:0/3130083783 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c1d2f8060 con 0x561c1d3ceb70 2024-02-17T12:52:59.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.411+0000 7f822f7fe640 1 --2- 172.21.15.18:0/3130083783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 0x561c1d3d7210 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8220009d30 tx=0x7f8220011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b95c1511b36f402 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.411+0000 7f82167fc640 1 -- 172.21.15.18:0/3130083783 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220004230 con 0x561c1d3ceb70 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.411+0000 7f82167fc640 1 -- 172.21.15.18:0/3130083783 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8220004390 con 0x561c1d3ceb70 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.411+0000 7f82167fc640 1 -- 172.21.15.18:0/3130083783 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82200035f0 con 0x561c1d3ceb70 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.412+0000 7f82395d7740 1 -- 172.21.15.18:0/3130083783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 msgr2=0x561c1d3d7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.412+0000 7f82395d7740 1 --2- 172.21.15.18:0/3130083783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 0x561c1d3d7210 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8220009d30 tx=0x7f8220011810 comp rx=0 tx=0).stop 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.412+0000 7f82395d7740 1 -- 172.21.15.18:0/3130083783 shutdown_connections 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.412+0000 7f82395d7740 1 --2- 172.21.15.18:0/3130083783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3d7750 0x561c1d3d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.412+0000 7f82395d7740 1 --2- 172.21.15.18:0/3130083783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 0x561c1d3d7210 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.412+0000 7f82395d7740 1 --2- 172.21.15.18:0/3130083783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.412+0000 7f82395d7740 1 -- 172.21.15.18:0/3130083783 >> 172.21.15.18:0/3130083783 conn(0x561c1d3c8980 msgr2=0x561c1d3c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.412+0000 7f82395d7740 1 -- 172.21.15.18:0/3130083783 shutdown_connections 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.412+0000 7f82395d7740 1 -- 172.21.15.18:0/3130083783 wait complete. 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.412+0000 7f82395d7740 1 -- start start 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.412+0000 7f82395d7740 1 Processor -- start 2024-02-17T12:52:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f82395d7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3d4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f82395d7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3ceb70 0x561c1d3d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f82395d7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 0x561c1d3d50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f82395d7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1d336020 con 0x561c1d3d7750 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f82395d7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1d009430 con 0x561c1d3ce250 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f82395d7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1cfc5e00 con 0x561c1d3ceb70 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f82395d7740 10 client.0 _ll_register_callbacks cb 0x561c1d2ddcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f8234fa8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 0x561c1d3d50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f8234fa8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 0x561c1d3d50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44596/0 (socket says 172.21.15.18:44596) 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f8234fa8640 1 -- 172.21.15.18:0/2280469186 learned_addr learned my addr 172.21.15.18:0/2280469186 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f822ffff640 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3d4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f822f7fe640 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3ceb70 0x561c1d3d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f8234fa8640 1 -- 172.21.15.18:0/2280469186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3ceb70 msgr2=0x561c1d3d4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f8234fa8640 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3ceb70 0x561c1d3d4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f8234fa8640 1 -- 172.21.15.18:0/2280469186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 msgr2=0x561c1d3d4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f8234fa8640 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3d4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f8234fa8640 1 -- 172.21.15.18:0/2280469186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82200099e0 con 0x561c1d3d7750 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f8234fa8640 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 0x561c1d3d50c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f82280348f0 tx=0x7f82280e16b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358c96f66be480fb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f820f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f820f7fe640 1 -- 172.21.15.18:0/2280469186 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280bcf40 con 0x561c1d3d7750 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f820f7fe640 1 -- 172.21.15.18:0/2280469186 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f82280c1090 con 0x561c1d3d7750 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.413+0000 7f82395d7740 1 -- 172.21.15.18:0/2280469186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c1d0357e0 con 0x561c1d3d7750 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.414+0000 7f820f7fe640 1 -- 172.21.15.18:0/2280469186 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82280c56e0 con 0x561c1d3d7750 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.414+0000 7f82395d7740 10 client.16701 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.414+0000 7f82395d7740 1 -- 172.21.15.18:0/2280469186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561c1d3346a0 con 0x561c1d3d7750 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.414+0000 7f820f7fe640 1 -- 172.21.15.18:0/2280469186 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82280d9020 con 0x561c1d3d7750 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.414+0000 7f820f7fe640 1 -- 172.21.15.18:0/2280469186 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f82280c5250 con 0x561c1d3d7750 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.414+0000 7f82395d7740 10 client.16701 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:59.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.414+0000 7f82395d7740 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.414+0000 7f82395d7740 20 client.16701 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.414+0000 7f82395d7740 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561c1d3e7880 0x561c1d3e9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.414+0000 7f82395d7740 4 client.16701 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.414+0000 7f82395d7740 1 -- 172.21.15.18:0/2280469186 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c1d072720 con 0x561c1d3e7880 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.414+0000 7f822ffff640 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561c1d3e7880 0x561c1d3e9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f820f7fe640 10 client.16701 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f822ffff640 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561c1d3e7880 0x561c1d3e9ca0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e5900b43123f62 server_cookie=643f4a9ddd7bd919 in_seq=0 out_seq=0 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f820f7fe640 1 -- 172.21.15.18:0/2280469186 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561c1d072720 con 0x561c1d3e7880 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f820f7fe640 10 client.16701 handle_command_reply: tid=0 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f82395d7740 1 client.16701 shutdown 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f82395d7740 1 -- 172.21.15.18:0/2280469186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 msgr2=0x561c1d3d50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f82395d7740 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 0x561c1d3d50c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f82280348f0 tx=0x7f82280e16b0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f82395d7740 1 -- 172.21.15.18:0/2280469186 shutdown_connections 2024-02-17T12:52:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f82395d7740 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561c1d3e7880 0x561c1d3e9ca0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f82395d7740 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 0x561c1d3d50c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f82395d7740 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3ceb70 0x561c1d3d4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f82395d7740 1 --2- 172.21.15.18:0/2280469186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3d4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f820f7fe640 0 client.16701 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:59.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.415+0000 7f82395d7740 1 -- 172.21.15.18:0/2280469186 >> 172.21.15.18:0/2280469186 conn(0x561c1d3c8980 msgr2=0x561c1d3ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.416+0000 7f82395d7740 1 -- 172.21.15.18:0/2280469186 shutdown_connections 2024-02-17T12:52:59.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.416+0000 7f82395d7740 1 -- 172.21.15.18:0/2280469186 wait complete. 2024-02-17T12:52:59.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.416+0000 7f82395d7740 20 client.16701 trim_cache size 0 max 16384 2024-02-17T12:52:59.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.428+0000 7f82395d7740 1 Processor -- start 2024-02-17T12:52:59.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.428+0000 7f82395d7740 1 -- start start 2024-02-17T12:52:59.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f82395d7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3ce250 0x561c1d3d6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f82395d7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ceb70 0x561c1d3d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f82395d7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 0x561c1d3d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f82395d7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1d2bb130 con 0x561c1d3d7750 2024-02-17T12:52:59.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f82395d7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1d3c45e0 con 0x561c1d3ceb70 2024-02-17T12:52:59.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f82395d7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1d3cdeb0 con 0x561c1d3ce250 2024-02-17T12:52:59.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f8234fa8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 0x561c1d3d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f822ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3ce250 0x561c1d3d6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f8234fa8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 0x561c1d3d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44608/0 (socket says 172.21.15.18:44608) 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f822ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3ce250 0x561c1d3d6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:58212/0 (socket says 172.21.15.18:58212) 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f8234fa8640 1 -- 172.21.15.18:0/1571349466 learned_addr learned my addr 172.21.15.18:0/1571349466 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f822f7fe640 1 --2- 172.21.15.18:0/1571349466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ceb70 0x561c1d3d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f822ffff640 1 -- 172.21.15.18:0/1571349466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ceb70 msgr2=0x561c1d3d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f822ffff640 1 --2- 172.21.15.18:0/1571349466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ceb70 0x561c1d3d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f822ffff640 1 -- 172.21.15.18:0/1571349466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 msgr2=0x561c1d3d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f822ffff640 1 --2- 172.21.15.18:0/1571349466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 0x561c1d3d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f822ffff640 1 -- 172.21.15.18:0/1571349466 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c1d2d0c20 con 0x561c1d3ce250 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f8234fa8640 1 --2- 172.21.15.18:0/1571349466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 0x561c1d3d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.429+0000 7f822ffff640 1 --2- 172.21.15.18:0/1571349466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3ce250 0x561c1d3d6d70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x561c1d3e7c60 tx=0x7f822401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fecc080a0f373c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.430+0000 7f8215ffb640 1 -- 172.21.15.18:0/1571349466 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224022070 con 0x561c1d3ce250 2024-02-17T12:52:59.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.430+0000 7f8215ffb640 1 -- 172.21.15.18:0/1571349466 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8224023040 con 0x561c1d3ce250 2024-02-17T12:52:59.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.430+0000 7f8215ffb640 1 -- 172.21.15.18:0/1571349466 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822400f840 con 0x561c1d3ce250 2024-02-17T12:52:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.430+0000 7f82395d7740 1 -- 172.21.15.18:0/1571349466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3ce250 msgr2=0x561c1d3d6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.430+0000 7f82395d7740 1 --2- 172.21.15.18:0/1571349466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3ce250 0x561c1d3d6d70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x561c1d3e7c60 tx=0x7f822401d000 comp rx=0 tx=0).stop 2024-02-17T12:52:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.430+0000 7f82395d7740 1 -- 172.21.15.18:0/1571349466 shutdown_connections 2024-02-17T12:52:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.430+0000 7f82395d7740 1 --2- 172.21.15.18:0/1571349466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3d7750 0x561c1d3d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.430+0000 7f82395d7740 1 --2- 172.21.15.18:0/1571349466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ceb70 0x561c1d3d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.430+0000 7f82395d7740 1 --2- 172.21.15.18:0/1571349466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3ce250 0x561c1d3d6d70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.430+0000 7f82395d7740 1 -- 172.21.15.18:0/1571349466 >> 172.21.15.18:0/1571349466 conn(0x561c1d3c8980 msgr2=0x561c1d3cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.430+0000 7f82395d7740 1 -- 172.21.15.18:0/1571349466 shutdown_connections 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.430+0000 7f82395d7740 1 -- 172.21.15.18:0/1571349466 wait complete. 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f82395d7740 1 -- start start 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f82395d7740 1 Processor -- start 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f82395d7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3ddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f82395d7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 0x561c1d3de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f82395d7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3d7750 0x561c1d436d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f82395d7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1d437290 con 0x561c1d3ceb70 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f82395d7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1d437400 con 0x561c1d3ce250 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f82395d7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1d4375c0 con 0x561c1d3d7750 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f82395d7740 10 client.0 _ll_register_callbacks cb 0x7f8220009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f822f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 0x561c1d3de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f822f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 0x561c1d3de090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44614/0 (socket says 172.21.15.18:44614) 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f822ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3ddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f822f7fe640 1 -- 172.21.15.18:0/3894040480 learned_addr learned my addr 172.21.15.18:0/3894040480 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.431+0000 7f8234fa8640 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3d7750 0x561c1d436d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f822ffff640 1 -- 172.21.15.18:0/3894040480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3d7750 msgr2=0x561c1d436d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f822ffff640 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3d7750 0x561c1d436d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f822ffff640 1 -- 172.21.15.18:0/3894040480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 msgr2=0x561c1d3de090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f822ffff640 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 0x561c1d3de090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f822ffff640 1 -- 172.21.15.18:0/3894040480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c1d3346a0 con 0x561c1d3ce250 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f8234fa8640 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3d7750 0x561c1d436d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f822f7fe640 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 0x561c1d3de090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f822ffff640 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3ddb50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8224009560 tx=0x7f82240042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9251ad98179da79 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f820f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f820f7fe640 1 -- 172.21.15.18:0/3894040480 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224022040 con 0x561c1d3ce250 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f820f7fe640 1 -- 172.21.15.18:0/3894040480 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8224005040 con 0x561c1d3ce250 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f82395d7740 1 -- 172.21.15.18:0/3894040480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8220004390 con 0x561c1d3ce250 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f820f7fe640 1 -- 172.21.15.18:0/3894040480 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8224005200 con 0x561c1d3ce250 2024-02-17T12:52:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f82395d7740 10 client.26078 fetch_fsmap learned FSMap version 29 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.432+0000 7f82395d7740 1 -- 172.21.15.18:0/3894040480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561c1d2f68c0 con 0x561c1d3ce250 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f820f7fe640 1 -- 172.21.15.18:0/3894040480 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8224041080 con 0x561c1d3ce250 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f820f7fe640 1 -- 172.21.15.18:0/3894040480 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f822402a190 con 0x561c1d3ce250 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f82395d7740 10 client.26078 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f82395d7740 10 client.26078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f82395d7740 20 client.26078 populate_metadata read hostname 'smithi018' 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f82395d7740 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561c1d440230 0x561c1d442650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f82395d7740 4 client.26078 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f82395d7740 1 -- 172.21.15.18:0/3894040480 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561c1d0c1890 con 0x561c1d440230 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f822f7fe640 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561c1d440230 0x561c1d442650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f822f7fe640 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561c1d440230 0x561c1d442650 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d642accf0db74d1a server_cookie=a3e9136f26542068 in_seq=0 out_seq=0 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f820f7fe640 10 client.26078 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f820f7fe640 1 -- 172.21.15.18:0/3894040480 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561c1d0c1890 con 0x561c1d440230 2024-02-17T12:52:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f820f7fe640 10 client.26078 handle_command_reply: tid=0 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.433+0000 7f82395d7740 1 client.26078 shutdown 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f82395d7740 1 -- 172.21.15.18:0/3894040480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 msgr2=0x561c1d3ddb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f82395d7740 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3ddb50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8224009560 tx=0x7f82240042f0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f82395d7740 1 -- 172.21.15.18:0/3894040480 shutdown_connections 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f82395d7740 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561c1d440230 0x561c1d442650 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f82395d7740 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561c1d3d7750 0x561c1d436d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f82395d7740 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561c1d3ceb70 0x561c1d3de090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f82395d7740 1 --2- 172.21.15.18:0/3894040480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c1d3ce250 0x561c1d3ddb50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f820f7fe640 0 client.26078 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f82395d7740 1 -- 172.21.15.18:0/3894040480 >> 172.21.15.18:0/3894040480 conn(0x561c1d3c8980 msgr2=0x561c1d3c72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f82395d7740 1 -- 172.21.15.18:0/3894040480 shutdown_connections 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f82395d7740 1 -- 172.21.15.18:0/3894040480 wait complete. 2024-02-17T12:52:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f82395d7740 20 client.26078 trim_cache size 0 max 16384 2024-02-17T12:52:59.432 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:52:59.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f8236a0b640 1 -- 172.21.15.18:0/360503944 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82180a2230 msgr2=0x7f82180a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f8236a0b640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82180a2230 0x7f82180a46f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8224011490 tx=0x7f8224011890 comp rx=0 tx=0).stop 2024-02-17T12:52:59.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f8236a0b640 1 -- 172.21.15.18:0/360503944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82301975a0 msgr2=0x7f8230197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:52:59.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f8236a0b640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82301975a0 0x7f8230197a50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8228004af0 tx=0x7f8228032570 comp rx=0 tx=0).stop 2024-02-17T12:52:59.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f8236a0b640 1 -- 172.21.15.18:0/360503944 shutdown_connections 2024-02-17T12:52:59.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f8236a0b640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82180a2230 0x7f82180a46f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f8236a0b640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82301975a0 0x7f8230197a50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f8236a0b640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8230106450 0x7f823019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f8236a0b640 1 --2- 172.21.15.18:0/360503944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8230101380 0x7f823019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:52:59.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f8236a0b640 1 -- 172.21.15.18:0/360503944 >> 172.21.15.18:0/360503944 conn(0x7f8230078580 msgr2=0x7f82300ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:52:59.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.434+0000 7f8236a0b640 1 -- 172.21.15.18:0/360503944 shutdown_connections 2024-02-17T12:52:59.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:52:59.435+0000 7f8236a0b640 1 -- 172.21.15.18:0/360503944 wait complete. 2024-02-17T12:52:59.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:52:59 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/4140964983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:52:59.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:52:59 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/4140964983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:52:59.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:52:59 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/4140964983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:53:00.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 135946 10.77 MB/sec execute 15 sec latency 14.549 ms 2024-02-17T12:53:00.436 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:53:00.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:53:00.437 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:53:00.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.563+0000 7f07ce13a640 1 -- 172.21.15.18:0/812451521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07c810b750 msgr2=0x7f07c810dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.563+0000 7f07ce13a640 1 --2- 172.21.15.18:0/812451521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07c810b750 0x7f07c810dbc0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009ca0 tx=0x7f07c002c180 comp rx=0 tx=0).stop 2024-02-17T12:53:00.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.563+0000 7f07ce13a640 1 -- 172.21.15.18:0/812451521 shutdown_connections 2024-02-17T12:53:00.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.563+0000 7f07ce13a640 1 --2- 172.21.15.18:0/812451521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07c810b750 0x7f07c810dbc0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.563+0000 7f07ce13a640 1 --2- 172.21.15.18:0/812451521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c8102d30 0x7f07c810b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.563+0000 7f07ce13a640 1 --2- 172.21.15.18:0/812451521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07c8102410 0x7f07c81027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.563+0000 7f07ce13a640 1 -- 172.21.15.18:0/812451521 >> 172.21.15.18:0/812451521 conn(0x7f07c80771e0 msgr2=0x7f07c80775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:00.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.563+0000 7f07ce13a640 1 -- 172.21.15.18:0/812451521 shutdown_connections 2024-02-17T12:53:00.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.563+0000 7f07ce13a640 1 -- 172.21.15.18:0/812451521 wait complete. 2024-02-17T12:53:00.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.563+0000 7f07ce13a640 1 Processor -- start 2024-02-17T12:53:00.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.563+0000 7f07ce13a640 1 -- start start 2024-02-17T12:53:00.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07ce13a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c8102410 0x7f07c8100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07ce13a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07c8102d30 0x7f07c81005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07ce13a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07c810b750 0x7f07c8100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07ce13a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c81a5af0 con 0x7f07c8102d30 2024-02-17T12:53:00.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07ce13a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c81a5c60 con 0x7f07c8102410 2024-02-17T12:53:00.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07ce13a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c81a5e20 con 0x7f07c810b750 2024-02-17T12:53:00.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07c6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07c8102d30 0x7f07c81005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07c6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07c8102d30 0x7f07c81005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42660/0 (socket says 172.21.15.18:42660) 2024-02-17T12:53:00.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07c6ffd640 1 -- 172.21.15.18:0/1630837461 learned_addr learned my addr 172.21.15.18:0/1630837461 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:00.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07c77fe640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c8102410 0x7f07c8100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07c7fff640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07c810b750 0x7f07c8100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07c6ffd640 1 -- 172.21.15.18:0/1630837461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07c810b750 msgr2=0x7f07c8100ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07c6ffd640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07c810b750 0x7f07c8100ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07c6ffd640 1 -- 172.21.15.18:0/1630837461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c8102410 msgr2=0x7f07c8100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07c6ffd640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c8102410 0x7f07c8100060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07c6ffd640 1 -- 172.21.15.18:0/1630837461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c0009900 con 0x7f07c8102d30 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07c7fff640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07c810b750 0x7f07c8100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.564+0000 7f07c6ffd640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07c8102d30 0x7f07c81005a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e9f0 tx=0x7f07b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.565+0000 7f07c4ff9640 1 -- 172.21.15.18:0/1630837461 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8009700 con 0x7f07c8102d30 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.565+0000 7f07ce13a640 1 -- 172.21.15.18:0/1630837461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c81a6100 con 0x7f07c8102d30 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.565+0000 7f07c4ff9640 1 -- 172.21.15.18:0/1630837461 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07b800cd80 con 0x7f07c8102d30 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.565+0000 7f07c4ff9640 1 -- 172.21.15.18:0/1630837461 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8018920 con 0x7f07c8102d30 2024-02-17T12:53:00.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.565+0000 7f07ce13a640 1 -- 172.21.15.18:0/1630837461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c81a6660 con 0x7f07c8102d30 2024-02-17T12:53:00.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.567+0000 7f07c4ff9640 1 -- 172.21.15.18:0/1630837461 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f07b8018a80 con 0x7f07c8102d30 2024-02-17T12:53:00.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.567+0000 7f07c4ff9640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07b00a22b0 0x7f07b00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.567+0000 7f07c77fe640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07b00a22b0 0x7f07b00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.568+0000 7f07c4ff9640 1 -- 172.21.15.18:0/1630837461 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f07b8014070 con 0x7f07c8102d30 2024-02-17T12:53:00.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.568+0000 7f07c77fe640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07b00a22b0 0x7f07b00a4770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00bbe0 tx=0x7f07bc00b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:00.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.568+0000 7f07d0de3740 1 Processor -- start 2024-02-17T12:53:00.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.568+0000 7f07d0de3740 1 -- start start 2024-02-17T12:53:00.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.568+0000 7f07d0de3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bccd1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.568+0000 7f07d0de3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1d30 0x55d5bccda2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.568+0000 7f07d0de3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccda8b0 0x55d5bccdcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.568+0000 7f07d0de3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcc3ff90 con 0x55d5bccd1380 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07d0de3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcc3d4c0 con 0x55d5bccda8b0 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07d0de3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcc38ca0 con 0x55d5bccd1d30 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bccd1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bccd1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42672/0 (socket says 172.21.15.18:42672) 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07c77fe640 1 -- 172.21.15.18:0/4238463874 learned_addr learned my addr 172.21.15.18:0/4238463874 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07c6ffd640 1 --2- 172.21.15.18:0/4238463874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1d30 0x55d5bccda2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07c7fff640 1 --2- 172.21.15.18:0/4238463874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccda8b0 0x55d5bccdcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07c77fe640 1 -- 172.21.15.18:0/4238463874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1d30 msgr2=0x55d5bccda2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07c77fe640 1 --2- 172.21.15.18:0/4238463874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1d30 0x55d5bccda2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07c77fe640 1 -- 172.21.15.18:0/4238463874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccda8b0 msgr2=0x55d5bccdcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07c77fe640 1 --2- 172.21.15.18:0/4238463874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccda8b0 0x55d5bccdcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07c77fe640 1 -- 172.21.15.18:0/4238463874 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5bcbfb0b0 con 0x55d5bccd1380 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07c7fff640 1 --2- 172.21.15.18:0/4238463874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccda8b0 0x55d5bccdcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:53:00.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07c77fe640 1 --2- 172.21.15.18:0/4238463874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bccd1760 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f07bc005d00 tx=0x7f07bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efc6d1800cd2f26 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:00.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07adffb640 1 -- 172.21.15.18:0/4238463874 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc005220 con 0x55d5bccd1380 2024-02-17T12:53:00.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07adffb640 1 -- 172.21.15.18:0/4238463874 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07bc015070 con 0x55d5bccd1380 2024-02-17T12:53:00.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.569+0000 7f07adffb640 1 -- 172.21.15.18:0/4238463874 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc01e8c0 con 0x55d5bccd1380 2024-02-17T12:53:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.570+0000 7f07d0de3740 1 -- 172.21.15.18:0/4238463874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 msgr2=0x55d5bccd1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.570+0000 7f07d0de3740 1 --2- 172.21.15.18:0/4238463874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bccd1760 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f07bc005d00 tx=0x7f07bc013000 comp rx=0 tx=0).stop 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.570+0000 7f07d0de3740 1 -- 172.21.15.18:0/4238463874 shutdown_connections 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.570+0000 7f07d0de3740 1 --2- 172.21.15.18:0/4238463874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccda8b0 0x55d5bccdcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.570+0000 7f07d0de3740 1 --2- 172.21.15.18:0/4238463874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1d30 0x55d5bccda2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.570+0000 7f07d0de3740 1 --2- 172.21.15.18:0/4238463874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bccd1760 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.570+0000 7f07d0de3740 1 -- 172.21.15.18:0/4238463874 >> 172.21.15.18:0/4238463874 conn(0x55d5bcccb840 msgr2=0x55d5bcccbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.570+0000 7f07d0de3740 1 -- 172.21.15.18:0/4238463874 shutdown_connections 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.570+0000 7f07d0de3740 1 -- 172.21.15.18:0/4238463874 wait complete. 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07d0de3740 1 -- start start 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07d0de3740 1 Processor -- start 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07d0de3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bccd7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07d0de3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccd1d30 0x55d5bccd7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07d0de3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccda8b0 0x55d5bccd8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07d0de3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcbddf40 con 0x55d5bccd1380 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07d0de3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bc90c4b0 con 0x55d5bccd1d30 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07d0de3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcbbe670 con 0x55d5bccda8b0 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bccd7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07d0de3740 10 client.0 _ll_register_callbacks cb 0x55d5bcc2bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bccd7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42674/0 (socket says 172.21.15.18:42674) 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07c77fe640 1 -- 172.21.15.18:0/3847644059 learned_addr learned my addr 172.21.15.18:0/3847644059 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07c7fff640 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccda8b0 0x55d5bccd8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.571+0000 7f07c6ffd640 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccd1d30 0x55d5bccd7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07c77fe640 1 -- 172.21.15.18:0/3847644059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccda8b0 msgr2=0x55d5bccd8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07c77fe640 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccda8b0 0x55d5bccd8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07c77fe640 1 -- 172.21.15.18:0/3847644059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccd1d30 msgr2=0x55d5bccd7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07c77fe640 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccd1d30 0x55d5bccd7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07c77fe640 1 -- 172.21.15.18:0/3847644059 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07bc010d10 con 0x55d5bccd1380 2024-02-17T12:53:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07c7fff640 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccda8b0 0x55d5bccd8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07c77fe640 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bccd7800 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f07bc01f5f0 tx=0x7f07bc01f620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a128894053f932d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07a6ffd640 1 -- 172.21.15.18:0/3847644059 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc02d070 con 0x55d5bccd1380 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07a6ffd640 1 -- 172.21.15.18:0/3847644059 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f07bc017070 con 0x55d5bccd1380 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07d0de3740 1 -- 172.21.15.18:0/3847644059 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5bcc12f40 con 0x55d5bccd1380 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07a6ffd640 1 -- 172.21.15.18:0/3847644059 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07bc01fa90 con 0x55d5bccd1380 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07d0de3740 10 client.16719 fetch_fsmap learned FSMap version 29 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07d0de3740 1 -- 172.21.15.18:0/3847644059 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d5bcc37320 con 0x55d5bccd1380 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.572+0000 7f07a6ffd640 1 -- 172.21.15.18:0/3847644059 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d5bcc12f40 con 0x55d5bccd1380 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.573+0000 7f07a6ffd640 1 -- 172.21.15.18:0/3847644059 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f07bc024030 con 0x55d5bccd1380 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.573+0000 7f07d0de3740 10 client.16719 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.573+0000 7f07d0de3740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.573+0000 7f07d0de3740 20 client.16719 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.573+0000 7f07d0de3740 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d5bcceadc0 0x55d5bcced1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.573+0000 7f07d0de3740 4 client.16719 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.573+0000 7f07d0de3740 1 -- 172.21.15.18:0/3847644059 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5bcc2f310 con 0x55d5bcceadc0 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.573+0000 7f07c6ffd640 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d5bcceadc0 0x55d5bcced1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.573+0000 7f07a6ffd640 10 client.16719 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.573+0000 7f07c6ffd640 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d5bcceadc0 0x55d5bcced1e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d8e49d59af1473 server_cookie=8cb1c39ea27f032d in_seq=0 out_seq=0 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07a6ffd640 1 -- 172.21.15.18:0/3847644059 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5bcc2f310 con 0x55d5bcceadc0 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07a6ffd640 10 client.16719 handle_command_reply: tid=0 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07d0de3740 1 client.16719 shutdown 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07d0de3740 1 -- 172.21.15.18:0/3847644059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 msgr2=0x55d5bccd7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07d0de3740 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bccd7800 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f07bc01f5f0 tx=0x7f07bc01f620 comp rx=0 tx=0).stop 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07d0de3740 1 -- 172.21.15.18:0/3847644059 shutdown_connections 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07d0de3740 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d5bcceadc0 0x55d5bcced1e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07d0de3740 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccda8b0 0x55d5bccd8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07d0de3740 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccd1d30 0x55d5bccd7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07a6ffd640 0 client.16719 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:00.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07d0de3740 1 --2- 172.21.15.18:0/3847644059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bccd7800 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07d0de3740 1 -- 172.21.15.18:0/3847644059 >> 172.21.15.18:0/3847644059 conn(0x55d5bcccb840 msgr2=0x55d5bcccd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:00.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.574+0000 7f07d0de3740 1 -- 172.21.15.18:0/3847644059 shutdown_connections 2024-02-17T12:53:00.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.575+0000 7f07d0de3740 1 -- 172.21.15.18:0/3847644059 wait complete. 2024-02-17T12:53:00.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.575+0000 7f07d0de3740 20 client.16719 trim_cache size 0 max 16384 2024-02-17T12:53:00.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07d0de3740 1 Processor -- start 2024-02-17T12:53:00.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07d0de3740 1 -- start start 2024-02-17T12:53:00.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07d0de3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bcccd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07d0de3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1d30 0x55d5bcce18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07d0de3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccda8b0 0x55d5bcce1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07d0de3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcccb6b0 con 0x55d5bccd1380 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07d0de3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcce78f0 con 0x55d5bccda8b0 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07d0de3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bccc7110 con 0x55d5bccd1d30 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bcccd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bcccd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42684/0 (socket says 172.21.15.18:42684) 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07c77fe640 1 -- 172.21.15.18:0/3967815039 learned_addr learned my addr 172.21.15.18:0/3967815039 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07c6ffd640 1 --2- 172.21.15.18:0/3967815039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1d30 0x55d5bcce18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07c7fff640 1 --2- 172.21.15.18:0/3967815039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccda8b0 0x55d5bcce1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07c77fe640 1 -- 172.21.15.18:0/3967815039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1d30 msgr2=0x55d5bcce18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07c77fe640 1 --2- 172.21.15.18:0/3967815039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1d30 0x55d5bcce18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07c77fe640 1 -- 172.21.15.18:0/3967815039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccda8b0 msgr2=0x55d5bcce1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07c77fe640 1 --2- 172.21.15.18:0/3967815039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccda8b0 0x55d5bcce1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07c77fe640 1 -- 172.21.15.18:0/3967815039 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5bcbd3c00 con 0x55d5bccd1380 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07c7fff640 1 --2- 172.21.15.18:0/3967815039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccda8b0 0x55d5bcce1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.591+0000 7f07c77fe640 1 --2- 172.21.15.18:0/3967815039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bcccd560 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f07bc006010 tx=0x7f07bc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ba56c50c47a2c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.592+0000 7f07ad7fa640 1 -- 172.21.15.18:0/3967815039 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc02d070 con 0x55d5bccd1380 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.592+0000 7f07ad7fa640 1 -- 172.21.15.18:0/3967815039 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07bc015040 con 0x55d5bccd1380 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.592+0000 7f07ad7fa640 1 -- 172.21.15.18:0/3967815039 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc01a8c0 con 0x55d5bccd1380 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.592+0000 7f07d0de3740 1 -- 172.21.15.18:0/3967815039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 msgr2=0x55d5bcccd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.592+0000 7f07d0de3740 1 --2- 172.21.15.18:0/3967815039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bcccd560 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f07bc006010 tx=0x7f07bc005e50 comp rx=0 tx=0).stop 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.592+0000 7f07d0de3740 1 -- 172.21.15.18:0/3967815039 shutdown_connections 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.592+0000 7f07d0de3740 1 --2- 172.21.15.18:0/3967815039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccda8b0 0x55d5bcce1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.592+0000 7f07d0de3740 1 --2- 172.21.15.18:0/3967815039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1d30 0x55d5bcce18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.592+0000 7f07d0de3740 1 --2- 172.21.15.18:0/3967815039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccd1380 0x55d5bcccd560 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.592+0000 7f07d0de3740 1 -- 172.21.15.18:0/3967815039 >> 172.21.15.18:0/3967815039 conn(0x55d5bcccb840 msgr2=0x55d5bcccce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.592+0000 7f07d0de3740 1 -- 172.21.15.18:0/3967815039 shutdown_connections 2024-02-17T12:53:00.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.592+0000 7f07d0de3740 1 -- 172.21.15.18:0/3967815039 wait complete. 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07d0de3740 1 -- start start 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07d0de3740 1 Processor -- start 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07d0de3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1380 0x55d5bcd3b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07d0de3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccd1d30 0x55d5bcd3bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07d0de3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccda8b0 0x55d5bcd35a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07d0de3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcd35fc0 con 0x55d5bccda8b0 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07d0de3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcd36130 con 0x55d5bccd1d30 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07d0de3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcd362f0 con 0x55d5bccd1380 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07d0de3740 10 client.0 _ll_register_callbacks cb 0x7f07bc00b8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccda8b0 0x55d5bcd35a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccda8b0 0x55d5bcd35a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42690/0 (socket says 172.21.15.18:42690) 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07c77fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1380 0x55d5bcd3b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07c7fff640 1 -- 172.21.15.18:0/627466298 learned_addr learned my addr 172.21.15.18:0/627466298 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.593+0000 7f07c6ffd640 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccd1d30 0x55d5bcd3bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07c77fe640 1 -- 172.21.15.18:0/627466298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccd1d30 msgr2=0x55d5bcd3bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07c77fe640 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccd1d30 0x55d5bcd3bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07c77fe640 1 -- 172.21.15.18:0/627466298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccda8b0 msgr2=0x55d5bcd35a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07c77fe640 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccda8b0 0x55d5bcd35a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07c77fe640 1 -- 172.21.15.18:0/627466298 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5bcc37320 con 0x55d5bccd1380 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07c7fff640 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccda8b0 0x55d5bcd35a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07c77fe640 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1380 0x55d5bcd3b5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00b9c0 tx=0x7f07bc01f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10c05d49c758283a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07a6ffd640 1 -- 172.21.15.18:0/627466298 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc02d040 con 0x55d5bccd1380 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07a6ffd640 1 -- 172.21.15.18:0/627466298 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f07bc017070 con 0x55d5bccd1380 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07d0de3740 1 -- 172.21.15.18:0/627466298 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5bcc5e900 con 0x55d5bccd1380 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07a6ffd640 1 -- 172.21.15.18:0/627466298 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07bc01fea0 con 0x55d5bccd1380 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07d0de3740 10 client.26122 fetch_fsmap learned FSMap version 29 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.594+0000 7f07d0de3740 1 -- 172.21.15.18:0/627466298 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d5bcbf9910 con 0x55d5bccd1380 2024-02-17T12:53:00.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.595+0000 7f07a6ffd640 1 -- 172.21.15.18:0/627466298 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07bc052020 con 0x55d5bccd1380 2024-02-17T12:53:00.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.595+0000 7f07a6ffd640 1 -- 172.21.15.18:0/627466298 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f07bc024040 con 0x55d5bccd1380 2024-02-17T12:53:00.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.595+0000 7f07d0de3740 10 client.26122 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:53:00.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.595+0000 7f07d0de3740 10 client.26122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:00.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.595+0000 7f07d0de3740 20 client.26122 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:00.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.595+0000 7f07d0de3740 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d5bcd430f0 0x55d5bcd45510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.595+0000 7f07d0de3740 4 client.26122 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:53:00.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.595+0000 7f07d0de3740 1 -- 172.21.15.18:0/627466298 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d5bc938860 con 0x55d5bcd430f0 2024-02-17T12:53:00.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.595+0000 7f07c6ffd640 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d5bcd430f0 0x55d5bcd45510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.595+0000 7f07c6ffd640 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d5bcd430f0 0x55d5bcd45510 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a210abede0ceee94 server_cookie=ca153bee47d17c92 in_seq=0 out_seq=0 2024-02-17T12:53:00.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.595+0000 7f07a6ffd640 10 client.26122 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:00.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07a6ffd640 1 -- 172.21.15.18:0/627466298 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d5bc938860 con 0x55d5bcd430f0 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07a6ffd640 10 client.26122 handle_command_reply: tid=0 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07d0de3740 1 client.26122 shutdown 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07d0de3740 1 -- 172.21.15.18:0/627466298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1380 msgr2=0x55d5bcd3b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07d0de3740 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1380 0x55d5bcd3b5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00b9c0 tx=0x7f07bc01f510 comp rx=0 tx=0).stop 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07d0de3740 1 -- 172.21.15.18:0/627466298 shutdown_connections 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07d0de3740 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d5bcd430f0 0x55d5bcd45510 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07d0de3740 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5bccda8b0 0x55d5bcd35a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07a6ffd640 0 client.26122 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07d0de3740 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5bccd1d30 0x55d5bcd3bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07d0de3740 1 --2- 172.21.15.18:0/627466298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5bccd1380 0x55d5bcd3b5f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07d0de3740 1 -- 172.21.15.18:0/627466298 >> 172.21.15.18:0/627466298 conn(0x55d5bcccb840 msgr2=0x55d5bccca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07d0de3740 1 -- 172.21.15.18:0/627466298 shutdown_connections 2024-02-17T12:53:00.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07d0de3740 1 -- 172.21.15.18:0/627466298 wait complete. 2024-02-17T12:53:00.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.597+0000 7f07d0de3740 20 client.26122 trim_cache size 0 max 16384 2024-02-17T12:53:00.594 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:53:00.594 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:53:00.594 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "3474ed19-9909-400b-965a-f342bdd57246", 2024-02-17T12:53:00.594 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:53:00.594 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:53:00.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.598+0000 7f07ce13a640 1 -- 172.21.15.18:0/1630837461 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07b00a22b0 msgr2=0x7f07b00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.598+0000 7f07ce13a640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07b00a22b0 0x7f07b00a4770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00bbe0 tx=0x7f07bc00b390 comp rx=0 tx=0).stop 2024-02-17T12:53:00.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.598+0000 7f07ce13a640 1 -- 172.21.15.18:0/1630837461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07c8102d30 msgr2=0x7f07c81005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.598+0000 7f07ce13a640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07c8102d30 0x7f07c81005a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e9f0 tx=0x7f07b800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.598+0000 7f07ce13a640 1 -- 172.21.15.18:0/1630837461 shutdown_connections 2024-02-17T12:53:00.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.598+0000 7f07ce13a640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07b00a22b0 0x7f07b00a4770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.598+0000 7f07ce13a640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07c810b750 0x7f07c8100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.598+0000 7f07ce13a640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07c8102d30 0x7f07c81005a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.598+0000 7f07ce13a640 1 --2- 172.21.15.18:0/1630837461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c8102410 0x7f07c8100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.598+0000 7f07ce13a640 1 -- 172.21.15.18:0/1630837461 >> 172.21.15.18:0/1630837461 conn(0x7f07c80771e0 msgr2=0x7f07c80fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:00.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.598+0000 7f07ce13a640 1 -- 172.21.15.18:0/1630837461 shutdown_connections 2024-02-17T12:53:00.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.598+0000 7f07ce13a640 1 -- 172.21.15.18:0/1630837461 wait complete. 2024-02-17T12:53:00.600 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:00 smithi018 ceph-mon[32625]: pgmap v96: 97 pgs: 97 active+clean; 65 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 385 op/s 2024-02-17T12:53:00.600 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:00 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:53:00.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:53:00.601 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:53:00.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:00 smithi146 ceph-mon[36454]: pgmap v96: 97 pgs: 97 active+clean; 65 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 385 op/s 2024-02-17T12:53:00.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:00 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:53:00.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.726+0000 7f37e850d640 1 -- 172.21.15.18:0/4212136063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37e006a370 msgr2=0x7f37e0105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.726+0000 7f37e850d640 1 --2- 172.21.15.18:0/4212136063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37e006a370 0x7f37e0105d90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009930 tx=0x7f37dc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:53:00.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.726+0000 7f37e850d640 1 -- 172.21.15.18:0/4212136063 shutdown_connections 2024-02-17T12:53:00.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.726+0000 7f37e850d640 1 --2- 172.21.15.18:0/4212136063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37e0106470 0x7f37e010e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.726+0000 7f37e850d640 1 --2- 172.21.15.18:0/4212136063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37e006a370 0x7f37e0105d90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.726+0000 7f37e850d640 1 --2- 172.21.15.18:0/4212136063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e00699c0 0x7f37e0069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.726+0000 7f37e850d640 1 -- 172.21.15.18:0/4212136063 >> 172.21.15.18:0/4212136063 conn(0x7f37e0077060 msgr2=0x7f37e0077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:00.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.726+0000 7f37e850d640 1 -- 172.21.15.18:0/4212136063 shutdown_connections 2024-02-17T12:53:00.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.726+0000 7f37e850d640 1 -- 172.21.15.18:0/4212136063 wait complete. 2024-02-17T12:53:00.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e850d640 1 Processor -- start 2024-02-17T12:53:00.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e850d640 1 -- start start 2024-02-17T12:53:00.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e850d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e00699c0 0x7f37e019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e850d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37e0106470 0x7f37e019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e850d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37e0197610 0x7f37e0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e850d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e0198000 con 0x7f37e0106470 2024-02-17T12:53:00.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e850d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e0198170 con 0x7f37e00699c0 2024-02-17T12:53:00.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e850d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e0198330 con 0x7f37e0197610 2024-02-17T12:53:00.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e5aa8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37e0106470 0x7f37e019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e5aa8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37e0106470 0x7f37e019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42714/0 (socket says 172.21.15.18:42714) 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e62a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e00699c0 0x7f37e019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e5aa8640 1 -- 172.21.15.18:0/2077886286 learned_addr learned my addr 172.21.15.18:0/2077886286 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e6aaa640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37e0197610 0x7f37e0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e5aa8640 1 -- 172.21.15.18:0/2077886286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37e0197610 msgr2=0x7f37e0197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e5aa8640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37e0197610 0x7f37e0197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e5aa8640 1 -- 172.21.15.18:0/2077886286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e00699c0 msgr2=0x7f37e019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e5aa8640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e00699c0 0x7f37e019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e5aa8640 1 -- 172.21.15.18:0/2077886286 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37dc009560 con 0x7f37e0106470 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.727+0000 7f37e62a9640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e00699c0 0x7f37e019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.728+0000 7f37e5aa8640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37e0106470 0x7f37e019d9a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f37dc000c00 tx=0x7f37dc02c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.728+0000 7f37d37fe640 1 -- 172.21.15.18:0/2077886286 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0042d0 con 0x7f37e0106470 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.728+0000 7f37d37fe640 1 -- 172.21.15.18:0/2077886286 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37dc004430 con 0x7f37e0106470 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.728+0000 7f37e850d640 1 -- 172.21.15.18:0/2077886286 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e0100360 con 0x7f37e0106470 2024-02-17T12:53:00.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.728+0000 7f37d37fe640 1 -- 172.21.15.18:0/2077886286 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc003c90 con 0x7f37e0106470 2024-02-17T12:53:00.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.728+0000 7f37e850d640 1 -- 172.21.15.18:0/2077886286 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e0100870 con 0x7f37e0106470 2024-02-17T12:53:00.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.731+0000 7f37d37fe640 1 -- 172.21.15.18:0/2077886286 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f37dc005580 con 0x7f37e0106470 2024-02-17T12:53:00.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.731+0000 7f37eb1b9740 1 Processor -- start 2024-02-17T12:53:00.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.731+0000 7f37eb1b9740 1 -- start start 2024-02-17T12:53:00.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.731+0000 7f37eb1b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bb8b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.731+0000 7f37eb1b9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 0x55bc2bb9ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.731+0000 7f37eb1b9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37eb1b9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2bafa280 con 0x55bc2bb8ba70 2024-02-17T12:53:00.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37eb1b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2baf77b0 con 0x55bc2bb8b0c0 2024-02-17T12:53:00.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37eb1b9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2baf2f90 con 0x55bc2bb9cfc0 2024-02-17T12:53:00.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e62a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 0x55bc2bb9ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e6aaa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bb8b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e5aa8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e62a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 0x55bc2bb9ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42718/0 (socket says 172.21.15.18:42718) 2024-02-17T12:53:00.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e62a9640 1 -- 172.21.15.18:0/1094042776 learned_addr learned my addr 172.21.15.18:0/1094042776 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:00.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37d37fe640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37cc0a2230 0x7f37cc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e62a9640 1 -- 172.21.15.18:0/1094042776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb9cfc0 msgr2=0x55bc2bb9f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e62a9640 1 --2- 172.21.15.18:0/1094042776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e62a9640 1 -- 172.21.15.18:0/1094042776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb8b0c0 msgr2=0x55bc2bb8b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e62a9640 1 --2- 172.21.15.18:0/1094042776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bb8b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e62a9640 1 -- 172.21.15.18:0/1094042776 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc2bab1570 con 0x55bc2bb8ba70 2024-02-17T12:53:00.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e5aa8640 1 --2- 172.21.15.18:0/1094042776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:00.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e6aaa640 1 --2- 172.21.15.18:0/1094042776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bb8b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:00.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e6aaa640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37cc0a2230 0x7f37cc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37d37fe640 1 -- 172.21.15.18:0/2077886286 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f37dc0eff10 con 0x7f37e0106470 2024-02-17T12:53:00.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37e62a9640 1 --2- 172.21.15.18:0/1094042776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 0x55bc2bb9ca80 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f37d40115f0 tx=0x7f37d4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2202f5f1d8b6527e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:00.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.732+0000 7f37d0bfe640 1 -- 172.21.15.18:0/1094042776 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d400f640 con 0x55bc2bb8ba70 2024-02-17T12:53:00.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.733+0000 7f37d0bfe640 1 -- 172.21.15.18:0/1094042776 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37d400fc80 con 0x55bc2bb8ba70 2024-02-17T12:53:00.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.733+0000 7f37d0bfe640 1 -- 172.21.15.18:0/1094042776 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4019ce0 con 0x55bc2bb8ba70 2024-02-17T12:53:00.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.733+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1094042776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 msgr2=0x55bc2bb9ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.733+0000 7f37e6aaa640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37cc0a2230 0x7f37cc0a46f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009fd0 tx=0x7f37d8004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:00.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.733+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1094042776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 0x55bc2bb9ca80 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f37d40115f0 tx=0x7f37d4011ab0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1094042776 shutdown_connections 2024-02-17T12:53:00.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1094042776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1094042776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 0x55bc2bb9ca80 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1094042776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bb8b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1094042776 >> 172.21.15.18:0/1094042776 conn(0x55bc2bb85990 msgr2=0x55bc2bb85d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:00.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1094042776 shutdown_connections 2024-02-17T12:53:00.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1094042776 wait complete. 2024-02-17T12:53:00.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 -- start start 2024-02-17T12:53:00.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 Processor -- start 2024-02-17T12:53:00.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bb9ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8ba70 0x55bc2bb99d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2b7c6430 con 0x55bc2bb8b0c0 2024-02-17T12:53:00.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2b782e00 con 0x55bc2bb9cfc0 2024-02-17T12:53:00.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2ba77fd0 con 0x55bc2bb8ba70 2024-02-17T12:53:00.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37eb1b9740 10 client.0 _ll_register_callbacks cb 0x55bc2bacdfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:00.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37e62a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bb9ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37e62a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bb9ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42730/0 (socket says 172.21.15.18:42730) 2024-02-17T12:53:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37e62a9640 1 -- 172.21.15.18:0/927196207 learned_addr learned my addr 172.21.15.18:0/927196207 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37e5aa8640 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8ba70 0x55bc2bb99d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.734+0000 7f37e6aaa640 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37e62a9640 1 -- 172.21.15.18:0/927196207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8ba70 msgr2=0x55bc2bb99d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37e62a9640 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8ba70 0x55bc2bb99d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37e62a9640 1 -- 172.21.15.18:0/927196207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb9cfc0 msgr2=0x55bc2bb9a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37e62a9640 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37e62a9640 1 -- 172.21.15.18:0/927196207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d4009d40 con 0x55bc2bb8b0c0 2024-02-17T12:53:00.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37e62a9640 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bb9ca00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f37d40197b0 tx=0x7f37d40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6998e64ac40c6598 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:00.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:00.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37c9ffb640 1 -- 172.21.15.18:0/927196207 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d401ae40 con 0x55bc2bb8b0c0 2024-02-17T12:53:00.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37c9ffb640 1 -- 172.21.15.18:0/927196207 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f37d4004040 con 0x55bc2bb8b0c0 2024-02-17T12:53:00.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37eb1b9740 1 -- 172.21.15.18:0/927196207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc2b7f27e0 con 0x55bc2bb8b0c0 2024-02-17T12:53:00.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37c9ffb640 1 -- 172.21.15.18:0/927196207 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37d4004200 con 0x55bc2bb8b0c0 2024-02-17T12:53:00.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37eb1b9740 10 client.16737 fetch_fsmap learned FSMap version 29 2024-02-17T12:53:00.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37eb1b9740 1 -- 172.21.15.18:0/927196207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bc2baf1610 con 0x55bc2bb8b0c0 2024-02-17T12:53:00.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.735+0000 7f37c9ffb640 1 -- 172.21.15.18:0/927196207 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bc2b7f27e0 con 0x55bc2bb8b0c0 2024-02-17T12:53:00.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.736+0000 7f37c9ffb640 1 -- 172.21.15.18:0/927196207 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f37d4003b60 con 0x55bc2bb8b0c0 2024-02-17T12:53:00.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.736+0000 7f37eb1b9740 10 client.16737 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:53:00.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.736+0000 7f37eb1b9740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:00.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.736+0000 7f37eb1b9740 20 client.16737 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:00.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.736+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bc2bbad140 0x55bc2bbaf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.736+0000 7f37eb1b9740 4 client.16737 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:00.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.736+0000 7f37eb1b9740 1 -- 172.21.15.18:0/927196207 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc2b82f720 con 0x55bc2bbad140 2024-02-17T12:53:00.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.736+0000 7f37e5aa8640 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bc2bbad140 0x55bc2bbaf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.736+0000 7f37e5aa8640 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bc2bbad140 0x55bc2bbaf560 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee2fb96745b0ded server_cookie=b06c242ad5c73796 in_seq=0 out_seq=0 2024-02-17T12:53:00.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.736+0000 7f37c9ffb640 10 client.16737 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:00.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37c9ffb640 1 -- 172.21.15.18:0/927196207 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc2b82f720 con 0x55bc2bbad140 2024-02-17T12:53:00.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37c9ffb640 10 client.16737 handle_command_reply: tid=0 2024-02-17T12:53:00.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37eb1b9740 1 client.16737 shutdown 2024-02-17T12:53:00.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37eb1b9740 1 -- 172.21.15.18:0/927196207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8b0c0 msgr2=0x55bc2bb9ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bb9ca00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f37d40197b0 tx=0x7f37d40056d0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37eb1b9740 1 -- 172.21.15.18:0/927196207 shutdown_connections 2024-02-17T12:53:00.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bc2bbad140 0x55bc2bbaf560 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8ba70 0x55bc2bb99d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37c9ffb640 0 client.16737 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:00.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/927196207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bb9ca00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37eb1b9740 1 -- 172.21.15.18:0/927196207 >> 172.21.15.18:0/927196207 conn(0x55bc2bb85990 msgr2=0x55bc2bb87730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:00.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37eb1b9740 1 -- 172.21.15.18:0/927196207 shutdown_connections 2024-02-17T12:53:00.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37eb1b9740 1 -- 172.21.15.18:0/927196207 wait complete. 2024-02-17T12:53:00.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.737+0000 7f37eb1b9740 20 client.16737 trim_cache size 0 max 16384 2024-02-17T12:53:00.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37eb1b9740 1 Processor -- start 2024-02-17T12:53:00.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37eb1b9740 1 -- start start 2024-02-17T12:53:00.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37eb1b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bba3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37eb1b9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8ba70 0x55bc2bba43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37eb1b9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37eb1b9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2b82f720 con 0x55bc2bb9cfc0 2024-02-17T12:53:00.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37eb1b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2b7f27e0 con 0x55bc2bb8b0c0 2024-02-17T12:53:00.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37eb1b9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2bb94c40 con 0x55bc2bb8ba70 2024-02-17T12:53:00.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37e6aaa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37e6aaa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42736/0 (socket says 172.21.15.18:42736) 2024-02-17T12:53:00.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37e62a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bba3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37e6aaa640 1 -- 172.21.15.18:0/1172157510 learned_addr learned my addr 172.21.15.18:0/1172157510 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:00.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.753+0000 7f37e5aa8640 1 --2- 172.21.15.18:0/1172157510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8ba70 0x55bc2bba43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.754+0000 7f37e6aaa640 1 -- 172.21.15.18:0/1172157510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8ba70 msgr2=0x55bc2bba43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.754+0000 7f37e6aaa640 1 --2- 172.21.15.18:0/1172157510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8ba70 0x55bc2bba43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.754+0000 7f37e6aaa640 1 -- 172.21.15.18:0/1172157510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb8b0c0 msgr2=0x55bc2bba3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.754+0000 7f37e6aaa640 1 --2- 172.21.15.18:0/1172157510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bba3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.754+0000 7f37e6aaa640 1 -- 172.21.15.18:0/1172157510 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc2ba8dac0 con 0x55bc2bb9cfc0 2024-02-17T12:53:00.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.754+0000 7f37e6aaa640 1 --2- 172.21.15.18:0/1172157510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9c4c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f37d800f9d0 tx=0x7f37d8013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118898f2f7c0e2e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:00.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.754+0000 7f37cbfff640 1 -- 172.21.15.18:0/1172157510 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8022050 con 0x55bc2bb9cfc0 2024-02-17T12:53:00.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.754+0000 7f37cbfff640 1 -- 172.21.15.18:0/1172157510 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37d8023040 con 0x55bc2bb9cfc0 2024-02-17T12:53:00.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.754+0000 7f37cbfff640 1 -- 172.21.15.18:0/1172157510 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d801b9b0 con 0x55bc2bb9cfc0 2024-02-17T12:53:00.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.755+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1172157510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb9cfc0 msgr2=0x55bc2bb9c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.755+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1172157510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9c4c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f37d800f9d0 tx=0x7f37d8013030 comp rx=0 tx=0).stop 2024-02-17T12:53:00.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.755+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1172157510 shutdown_connections 2024-02-17T12:53:00.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.755+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1172157510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bb9c4c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.755+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1172157510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8ba70 0x55bc2bba43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.755+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1172157510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bba3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.755+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1172157510 >> 172.21.15.18:0/1172157510 conn(0x55bc2bb85990 msgr2=0x55bc2bb98660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:00.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.755+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1172157510 shutdown_connections 2024-02-17T12:53:00.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.755+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1172157510 wait complete. 2024-02-17T12:53:00.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.755+0000 7f37eb1b9740 1 -- start start 2024-02-17T12:53:00.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.755+0000 7f37eb1b9740 1 Processor -- start 2024-02-17T12:53:00.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37eb1b9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bba2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37eb1b9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 0x55bc2bba3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37eb1b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bc03650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37eb1b9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2bba38c0 con 0x55bc2bb8ba70 2024-02-17T12:53:00.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37eb1b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2bba3a30 con 0x55bc2bb9cfc0 2024-02-17T12:53:00.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37eb1b9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2bc03b90 con 0x55bc2bb8b0c0 2024-02-17T12:53:00.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37eb1b9740 10 client.0 _ll_register_callbacks cb 0x7f37d4009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:00.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37e5aa8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 0x55bc2bba3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37e5aa8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 0x55bc2bba3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42742/0 (socket says 172.21.15.18:42742) 2024-02-17T12:53:00.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37e62a9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bba2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37e5aa8640 1 -- 172.21.15.18:0/1016287946 learned_addr learned my addr 172.21.15.18:0/1016287946 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:00.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37e6aaa640 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bc03650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37e5aa8640 1 -- 172.21.15.18:0/1016287946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8b0c0 msgr2=0x55bc2bba2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37e5aa8640 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bba2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37e5aa8640 1 -- 172.21.15.18:0/1016287946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb9cfc0 msgr2=0x55bc2bc03650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:00.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37e5aa8640 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bc03650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37e5aa8640 1 -- 172.21.15.18:0/1016287946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d800f6e0 con 0x55bc2bb8ba70 2024-02-17T12:53:00.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37e62a9640 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bba2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:00.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37e5aa8640 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 0x55bc2bba3130 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0ed870 tx=0x7f37dc0f23e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660a5af44b8ad771 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:00.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.756+0000 7f37c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:00.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37c9ffb640 1 -- 172.21.15.18:0/1016287946 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0bd160 con 0x55bc2bb8ba70 2024-02-17T12:53:00.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37c9ffb640 1 -- 172.21.15.18:0/1016287946 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f37dc0bca60 con 0x55bc2bb8ba70 2024-02-17T12:53:00.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1016287946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37d400fc80 con 0x55bc2bb8ba70 2024-02-17T12:53:00.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37c9ffb640 1 -- 172.21.15.18:0/1016287946 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37dc0c0bb0 con 0x55bc2bb8ba70 2024-02-17T12:53:00.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37eb1b9740 10 client.16743 fetch_fsmap learned FSMap version 29 2024-02-17T12:53:00.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1016287946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bc2bab36a0 con 0x55bc2bb8ba70 2024-02-17T12:53:00.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37c9ffb640 1 -- 172.21.15.18:0/1016287946 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37dc0d9080 con 0x55bc2bb8ba70 2024-02-17T12:53:00.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37c9ffb640 1 -- 172.21.15.18:0/1016287946 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f37dc03c070 con 0x55bc2bb8ba70 2024-02-17T12:53:00.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37eb1b9740 10 client.16743 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:53:00.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37eb1b9740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:00.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37eb1b9740 20 client.16743 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:00.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bc2bc052b0 0x55bc2bc076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:00.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37eb1b9740 4 client.16743 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:53:00.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1016287946 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bc2b87e890 con 0x55bc2bc052b0 2024-02-17T12:53:00.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.757+0000 7f37e62a9640 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bc2bc052b0 0x55bc2bc076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:00.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37c9ffb640 10 client.16743 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:00.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37e62a9640 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bc2bc052b0 0x55bc2bc076d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=761ec32262c0d0e1 server_cookie=b6e183cdc3a54c54 in_seq=0 out_seq=0 2024-02-17T12:53:00.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37c9ffb640 1 -- 172.21.15.18:0/1016287946 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bc2b87e890 con 0x55bc2bc052b0 2024-02-17T12:53:00.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37c9ffb640 10 client.16743 handle_command_reply: tid=0 2024-02-17T12:53:00.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37eb1b9740 1 client.16743 shutdown 2024-02-17T12:53:00.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1016287946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 msgr2=0x55bc2bba3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 0x55bc2bba3130 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0ed870 tx=0x7f37dc0f23e0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1016287946 shutdown_connections 2024-02-17T12:53:00.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bc2bc052b0 0x55bc2bc076d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc2bb9cfc0 0x55bc2bc03650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bc2bb8ba70 0x55bc2bba3130 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37eb1b9740 1 --2- 172.21.15.18:0/1016287946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc2bb8b0c0 0x55bc2bba2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.758+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1016287946 >> 172.21.15.18:0/1016287946 conn(0x55bc2bb85990 msgr2=0x55bc2bb995c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:00.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37c9ffb640 0 client.16743 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:00.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1016287946 shutdown_connections 2024-02-17T12:53:00.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37eb1b9740 1 -- 172.21.15.18:0/1016287946 wait complete. 2024-02-17T12:53:00.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37eb1b9740 20 client.16743 trim_cache size 0 max 16384 2024-02-17T12:53:00.776 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:53:00.776 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:53:00.777 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:53:00.777 INFO:teuthology.orchestra.run.smithi018.stdout: "3474ed19-9909-400b-965a-f342bdd57246": { 2024-02-17T12:53:00.777 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:53:00.777 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "3474ed19-9909-400b-965a-f342bdd57246", 2024-02-17T12:53:00.777 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:53:00.777 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:53:00.778 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:53:00.778 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:53:00.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37e850d640 1 -- 172.21.15.18:0/2077886286 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37cc0a2230 msgr2=0x7f37cc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37e850d640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37cc0a2230 0x7f37cc0a46f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009fd0 tx=0x7f37d8004a40 comp rx=0 tx=0).stop 2024-02-17T12:53:00.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37e850d640 1 -- 172.21.15.18:0/2077886286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37e0106470 msgr2=0x7f37e019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:00.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37e850d640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37e0106470 0x7f37e019d9a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f37dc000c00 tx=0x7f37dc02c5d0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37e850d640 1 -- 172.21.15.18:0/2077886286 shutdown_connections 2024-02-17T12:53:00.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37e850d640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37cc0a2230 0x7f37cc0a46f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37e850d640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37e0197610 0x7f37e0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37e850d640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37e0106470 0x7f37e019d9a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37e850d640 1 --2- 172.21.15.18:0/2077886286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e00699c0 0x7f37e019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:00.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37e850d640 1 -- 172.21.15.18:0/2077886286 >> 172.21.15.18:0/2077886286 conn(0x7f37e0077060 msgr2=0x7f37e00fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:00.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37e850d640 1 -- 172.21.15.18:0/2077886286 shutdown_connections 2024-02-17T12:53:00.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:00.759+0000 7f37e850d640 1 -- 172.21.15.18:0/2077886286 wait complete. 2024-02-17T12:53:00.783 INFO:tasks.cephfs.filesystem:scrub status for tag:3474ed19-9909-400b-965a-f342bdd57246 - {'path': '/', 'tag': '3474ed19-9909-400b-965a-f342bdd57246', 'options': 'recursive,force'} 2024-02-17T12:53:00.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:00 smithi119 ceph-mon[36188]: pgmap v96: 97 pgs: 97 active+clean; 65 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 385 op/s 2024-02-17T12:53:00.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:00 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:53:01.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 137601 10.66 MB/sec execute 16 sec latency 69.535 ms 2024-02-17T12:53:01.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:53:01.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:01.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:01.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:53:01.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:01 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:53:01.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:53:01.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:01.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:01.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:01.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:01.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:01.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:01.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:01.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:53:01.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:53:01.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:01 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:53:01.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:01.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:01.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:53:01.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:53:01.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:01 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:53:02.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 139281 10.55 MB/sec execute 17 sec latency 11.271 ms 2024-02-17T12:53:02.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:02 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:02.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:02 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:02.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:02 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:02.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:02 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:02.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:02 smithi146 ceph-mon[36454]: pgmap v97: 97 pgs: 97 active+clean; 65 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-02-17T12:53:02.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:02 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:02.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:02 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:02.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:02 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:02.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:02 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:02.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:02 smithi018 ceph-mon[32625]: pgmap v97: 97 pgs: 97 active+clean; 65 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-02-17T12:53:02.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:02 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:02.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:02 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:02.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:02 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:02.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:02 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:02.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:02 smithi119 ceph-mon[36188]: pgmap v97: 97 pgs: 97 active+clean; 65 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-02-17T12:53:03.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 142001 10.91 MB/sec execute 18 sec latency 11.809 ms 2024-02-17T12:53:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.150+0000 7f08fba1c640 1 -- 172.21.15.18:0/1406918714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f41002d0 msgr2=0x7f08f4100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.150+0000 7f08fba1c640 1 --2- 172.21.15.18:0/1406918714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f41002d0 0x7f08f4100750 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009900 tx=0x7f08e402bc30 comp rx=0 tx=0).stop 2024-02-17T12:53:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.150+0000 7f08fba1c640 1 -- 172.21.15.18:0/1406918714 shutdown_connections 2024-02-17T12:53:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.150+0000 7f08fba1c640 1 --2- 172.21.15.18:0/1406918714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f4100c90 0x7f08f410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.150+0000 7f08fba1c640 1 --2- 172.21.15.18:0/1406918714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f41002d0 0x7f08f4100750 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.150+0000 7f08fba1c640 1 --2- 172.21.15.18:0/1406918714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f4105020 0x7f08f4105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.150+0000 7f08fba1c640 1 -- 172.21.15.18:0/1406918714 >> 172.21.15.18:0/1406918714 conn(0x7f08f4076ed0 msgr2=0x7f08f40772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.150+0000 7f08fba1c640 1 -- 172.21.15.18:0/1406918714 shutdown_connections 2024-02-17T12:53:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.150+0000 7f08fba1c640 1 -- 172.21.15.18:0/1406918714 wait complete. 2024-02-17T12:53:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08fba1c640 1 Processor -- start 2024-02-17T12:53:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08fba1c640 1 -- start start 2024-02-17T12:53:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08fba1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f41002d0 0x7f08f419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08fba1c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f4100c90 0x7f08f419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08fba1c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f4105020 0x7f08f4197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08fba1c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4197dd0 con 0x7f08f4100c90 2024-02-17T12:53:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08fba1c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4197f40 con 0x7f08f41002d0 2024-02-17T12:53:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08fba1c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4198100 con 0x7f08f4105020 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08f8fb7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f4100c90 0x7f08f419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08f8fb7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f4100c90 0x7f08f419dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42754/0 (socket says 172.21.15.18:42754) 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08f8fb7640 1 -- 172.21.15.18:0/3041191604 learned_addr learned my addr 172.21.15.18:0/3041191604 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08f97b8640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f41002d0 0x7f08f419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.151+0000 7f08f9fb9640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f4105020 0x7f08f4197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.152+0000 7f08f8fb7640 1 -- 172.21.15.18:0/3041191604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f4105020 msgr2=0x7f08f4197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.152+0000 7f08f8fb7640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f4105020 0x7f08f4197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.152+0000 7f08f8fb7640 1 -- 172.21.15.18:0/3041191604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f41002d0 msgr2=0x7f08f419d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.152+0000 7f08f8fb7640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f41002d0 0x7f08f419d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.152+0000 7f08f8fb7640 1 -- 172.21.15.18:0/3041191604 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4009560 con 0x7f08f4100c90 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.152+0000 7f08f8fb7640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f4100c90 0x7f08f419dac0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f08e4005940 tx=0x7f08e402c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.152+0000 7f08ea7fc640 1 -- 172.21.15.18:0/3041191604 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e40042d0 con 0x7f08f4100c90 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.152+0000 7f08ea7fc640 1 -- 172.21.15.18:0/3041191604 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08e4004430 con 0x7f08f4100c90 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.152+0000 7f08fba1c640 1 -- 172.21.15.18:0/3041191604 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f4198380 con 0x7f08f4100c90 2024-02-17T12:53:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.152+0000 7f08ea7fc640 1 -- 172.21.15.18:0/3041191604 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4003c50 con 0x7f08f4100c90 2024-02-17T12:53:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.152+0000 7f08fba1c640 1 -- 172.21.15.18:0/3041191604 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f41a4500 con 0x7f08f4100c90 2024-02-17T12:53:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.154+0000 7f08ea7fc640 1 -- 172.21.15.18:0/3041191604 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f08e40054f0 con 0x7f08f4100c90 2024-02-17T12:53:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08fe5f3740 1 Processor -- start 2024-02-17T12:53:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08fe5f3740 1 -- start start 2024-02-17T12:53:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08fe5f3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bcad0 0x563ce98bb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08ea7fc640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f08e00a22d0 0x7f08e00a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08fe5f3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bbd60 0x563ce98bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08fe5f3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08fe5f3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce982a9f0 con 0x563ce98c52e0 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08fe5f3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce9827f20 con 0x563ce98bcad0 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08fe5f3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce9823700 con 0x563ce98bbd60 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08f9fb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08f97b8640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f08e00a22d0 0x7f08e00a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08f9fb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42766/0 (socket says 172.21.15.18:42766) 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08f9fb9640 1 -- 172.21.15.18:0/1164034825 learned_addr learned my addr 172.21.15.18:0/1164034825 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08f9fb9640 1 --2- 172.21.15.18:0/1164034825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bcad0 0x563ce98bb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08ea7fc640 1 -- 172.21.15.18:0/3041191604 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f08e40f0000 con 0x7f08f4100c90 2024-02-17T12:53:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08f8fb7640 1 --2- 172.21.15.18:0/1164034825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bbd60 0x563ce98bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08f9fb9640 1 -- 172.21.15.18:0/1164034825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bbd60 msgr2=0x563ce98bc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08f9fb9640 1 --2- 172.21.15.18:0/1164034825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bbd60 0x563ce98bc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08f9fb9640 1 -- 172.21.15.18:0/1164034825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bcad0 msgr2=0x563ce98bb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08f9fb9640 1 --2- 172.21.15.18:0/1164034825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bcad0 0x563ce98bb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08f9fb9640 1 -- 172.21.15.18:0/1164034825 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ce97b97b0 con 0x563ce98c52e0 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.155+0000 7f08f8fb7640 1 --2- 172.21.15.18:0/1164034825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bbd60 0x563ce98bc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.156+0000 7f08f97b8640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f08e00a22d0 0x7f08e00a4790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009b50 tx=0x7f08f0005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.156+0000 7f08f9fb9640 1 --2- 172.21.15.18:0/1164034825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c76d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f08ec011b20 tx=0x7f08ec009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b59e2e00714e484 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.156+0000 7f08df7fe640 1 -- 172.21.15.18:0/1164034825 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec011050 con 0x563ce98c52e0 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.156+0000 7f08df7fe640 1 -- 172.21.15.18:0/1164034825 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08ec00fd60 con 0x563ce98c52e0 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.156+0000 7f08df7fe640 1 -- 172.21.15.18:0/1164034825 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec010a00 con 0x563ce98c52e0 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.156+0000 7f08fe5f3740 1 -- 172.21.15.18:0/1164034825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 msgr2=0x563ce98c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.156+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/1164034825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c76d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f08ec011b20 tx=0x7f08ec009b70 comp rx=0 tx=0).stop 2024-02-17T12:53:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 -- 172.21.15.18:0/1164034825 shutdown_connections 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/1164034825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c76d0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/1164034825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bbd60 0x563ce98bc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/1164034825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bcad0 0x563ce98bb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 -- 172.21.15.18:0/1164034825 >> 172.21.15.18:0/1164034825 conn(0x563ce9883550 msgr2=0x563ce98b6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 -- 172.21.15.18:0/1164034825 shutdown_connections 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 -- 172.21.15.18:0/1164034825 wait complete. 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 -- start start 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 Processor -- start 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bbd60 0x563ce98c2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bcad0 0x563ce98c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce97c8bf0 con 0x563ce98c52e0 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce94f7250 con 0x563ce98bbd60 2024-02-17T12:53:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce97a9200 con 0x563ce98bcad0 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.157+0000 7f08fe5f3740 10 client.0 _ll_register_callbacks cb 0x563ce9817d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f9fb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f97b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bbd60 0x563ce98c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f8fb7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bcad0 0x563ce98c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f9fb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42780/0 (socket says 172.21.15.18:42780) 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f97b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bbd60 0x563ce98c2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:41104/0 (socket says 172.21.15.18:41104) 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f9fb9640 1 -- 172.21.15.18:0/3080159207 learned_addr learned my addr 172.21.15.18:0/3080159207 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f9fb9640 1 -- 172.21.15.18:0/3080159207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bcad0 msgr2=0x563ce98c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f9fb9640 1 --2- 172.21.15.18:0/3080159207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bcad0 0x563ce98c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f9fb9640 1 -- 172.21.15.18:0/3080159207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bbd60 msgr2=0x563ce98c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f9fb9640 1 --2- 172.21.15.18:0/3080159207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bbd60 0x563ce98c2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f9fb9640 1 -- 172.21.15.18:0/3080159207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec019040 con 0x563ce98c52e0 2024-02-17T12:53:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f97b8640 1 --2- 172.21.15.18:0/3080159207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bbd60 0x563ce98c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08f9fb9640 1 --2- 172.21.15.18:0/3080159207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c2da0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f08ec011af0 tx=0x7f08ec005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6030d5b5cb7b51e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08dcff9640 1 -- 172.21.15.18:0/3080159207 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec024070 con 0x563ce98c52e0 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08fe5f3740 1 -- 172.21.15.18:0/3080159207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ce9523600 con 0x563ce98c52e0 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08dcff9640 1 -- 172.21.15.18:0/3080159207 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f08ec010430 con 0x563ce98c52e0 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08dcff9640 1 -- 172.21.15.18:0/3080159207 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08ec00fed0 con 0x563ce98c52e0 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08fe5f3740 10 client.16755 fetch_fsmap learned FSMap version 29 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.158+0000 7f08fe5f3740 1 -- 172.21.15.18:0/3080159207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563ce9821d80 con 0x563ce98c52e0 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.159+0000 7f08dcff9640 1 -- 172.21.15.18:0/3080159207 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563ce9523600 con 0x563ce98c52e0 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.159+0000 7f08dcff9640 1 -- 172.21.15.18:0/3080159207 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f08ec01e830 con 0x563ce98c52e0 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.159+0000 7f08fe5f3740 10 client.16755 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.159+0000 7f08fe5f3740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.159+0000 7f08fe5f3740 20 client.16755 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.159+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/3080159207 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ce98d5650 0x563ce98d7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.159+0000 7f08fe5f3740 4 client.16755 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.159+0000 7f08fe5f3740 1 -- 172.21.15.18:0/3080159207 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ce95741b0 con 0x563ce98d5650 2024-02-17T12:53:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.159+0000 7f08f97b8640 1 --2- 172.21.15.18:0/3080159207 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ce98d5650 0x563ce98d7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.159+0000 7f08dcff9640 10 client.16755 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.159+0000 7f08f97b8640 1 --2- 172.21.15.18:0/3080159207 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ce98d5650 0x563ce98d7a70 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4b980b66aaa52f server_cookie=cf076539a0f313f8 in_seq=0 out_seq=0 2024-02-17T12:53:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08dcff9640 1 -- 172.21.15.18:0/3080159207 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ce95741b0 con 0x563ce98d5650 2024-02-17T12:53:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08dcff9640 10 client.16755 handle_command_reply: tid=0 2024-02-17T12:53:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08fe5f3740 1 client.16755 shutdown 2024-02-17T12:53:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08fe5f3740 1 -- 172.21.15.18:0/3080159207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 msgr2=0x563ce98c2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/3080159207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c2da0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f08ec011af0 tx=0x7f08ec005bd0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08fe5f3740 1 -- 172.21.15.18:0/3080159207 shutdown_connections 2024-02-17T12:53:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/3080159207 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ce98d5650 0x563ce98d7a70 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/3080159207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c2da0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/3080159207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bcad0 0x563ce98c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08dcff9640 0 client.16755 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/3080159207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bbd60 0x563ce98c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08fe5f3740 1 -- 172.21.15.18:0/3080159207 >> 172.21.15.18:0/3080159207 conn(0x563ce9883550 msgr2=0x563ce98b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08fe5f3740 1 -- 172.21.15.18:0/3080159207 shutdown_connections 2024-02-17T12:53:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08fe5f3740 1 -- 172.21.15.18:0/3080159207 wait complete. 2024-02-17T12:53:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.164+0000 7f08fe5f3740 20 client.16755 trim_cache size 0 max 16384 2024-02-17T12:53:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08fe5f3740 1 Processor -- start 2024-02-17T12:53:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08fe5f3740 1 -- start start 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08fe5f3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bbd60 0x563ce98b6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08fe5f3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bcad0 0x563ce98b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08fe5f3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08fe5f3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce9523600 con 0x563ce98c52e0 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08fe5f3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce98b2140 con 0x563ce98bbd60 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08fe5f3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce98bd2b0 con 0x563ce98bcad0 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08f9fb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08f9fb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42790/0 (socket says 172.21.15.18:42790) 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08f9fb9640 1 -- 172.21.15.18:0/4187526929 learned_addr learned my addr 172.21.15.18:0/4187526929 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08f8fb7640 1 --2- 172.21.15.18:0/4187526929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bcad0 0x563ce98b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.179+0000 7f08f97b8640 1 --2- 172.21.15.18:0/4187526929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bbd60 0x563ce98b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.180+0000 7f08f9fb9640 1 -- 172.21.15.18:0/4187526929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bcad0 msgr2=0x563ce98b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.180+0000 7f08f9fb9640 1 --2- 172.21.15.18:0/4187526929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bcad0 0x563ce98b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.180+0000 7f08f9fb9640 1 -- 172.21.15.18:0/4187526929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bbd60 msgr2=0x563ce98b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.180+0000 7f08f9fb9640 1 --2- 172.21.15.18:0/4187526929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bbd60 0x563ce98b6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.180+0000 7f08f9fb9640 1 -- 172.21.15.18:0/4187526929 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ce97be7b0 con 0x563ce98c52e0 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.180+0000 7f08f8fb7640 1 --2- 172.21.15.18:0/4187526929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bcad0 0x563ce98b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.180+0000 7f08f9fb9640 1 --2- 172.21.15.18:0/4187526929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c66d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009560 tx=0x7f08ec01ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dc73078a96c1d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.180+0000 7f08deffd640 1 -- 172.21.15.18:0/4187526929 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec024040 con 0x563ce98c52e0 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.180+0000 7f08deffd640 1 -- 172.21.15.18:0/4187526929 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08ec003e50 con 0x563ce98c52e0 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.180+0000 7f08deffd640 1 -- 172.21.15.18:0/4187526929 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec01e800 con 0x563ce98c52e0 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.181+0000 7f08fe5f3740 1 -- 172.21.15.18:0/4187526929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 msgr2=0x563ce98c66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.181+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/4187526929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c66d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009560 tx=0x7f08ec01ef00 comp rx=0 tx=0).stop 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.181+0000 7f08fe5f3740 1 -- 172.21.15.18:0/4187526929 shutdown_connections 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.181+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/4187526929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98c52e0 0x563ce98c66d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.181+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/4187526929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98bcad0 0x563ce98b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.181+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/4187526929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bbd60 0x563ce98b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.181+0000 7f08fe5f3740 1 -- 172.21.15.18:0/4187526929 >> 172.21.15.18:0/4187526929 conn(0x563ce9883550 msgr2=0x563ce98dc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.181+0000 7f08fe5f3740 1 -- 172.21.15.18:0/4187526929 shutdown_connections 2024-02-17T12:53:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.181+0000 7f08fe5f3740 1 -- 172.21.15.18:0/4187526929 wait complete. 2024-02-17T12:53:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.181+0000 7f08fe5f3740 1 -- start start 2024-02-17T12:53:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.181+0000 7f08fe5f3740 1 Processor -- start 2024-02-17T12:53:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08fe5f3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98bbd60 0x563ce98ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08fe5f3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bcad0 0x563ce98ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08fe5f3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98c52e0 0x563ce98cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08fe5f3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce992ef40 con 0x563ce98bbd60 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08fe5f3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce992f080 con 0x563ce98bcad0 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08fe5f3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce992f240 con 0x563ce98c52e0 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08fe5f3740 10 client.0 _ll_register_callbacks cb 0x7f08ec009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08f97b8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98bbd60 0x563ce98ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08f97b8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98bbd60 0x563ce98ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42794/0 (socket says 172.21.15.18:42794) 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08f97b8640 1 -- 172.21.15.18:0/2358335993 learned_addr learned my addr 172.21.15.18:0/2358335993 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08f8fb7640 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bcad0 0x563ce98ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08f9fb9640 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98c52e0 0x563ce98cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08f97b8640 1 -- 172.21.15.18:0/2358335993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98c52e0 msgr2=0x563ce98cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08f97b8640 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98c52e0 0x563ce98cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08f97b8640 1 -- 172.21.15.18:0/2358335993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bcad0 msgr2=0x563ce98ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08f97b8640 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bcad0 0x563ce98ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08f97b8640 1 -- 172.21.15.18:0/2358335993 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f41a4500 con 0x563ce98bbd60 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08f97b8640 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98bbd60 0x563ce98ca0e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f08f00037e0 tx=0x7f08f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d898d80c52c047c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.182+0000 7f08dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08dcff9640 1 -- 172.21.15.18:0/2358335993 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f0003ee0 con 0x563ce98bbd60 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08fe5f3740 1 -- 172.21.15.18:0/2358335993 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08ec00fd60 con 0x563ce98bbd60 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08dcff9640 1 -- 172.21.15.18:0/2358335993 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f08f0026340 con 0x563ce98bbd60 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08dcff9640 1 -- 172.21.15.18:0/2358335993 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08f0026500 con 0x563ce98bbd60 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08fe5f3740 10 client.16761 fetch_fsmap learned FSMap version 29 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08fe5f3740 1 -- 172.21.15.18:0/2358335993 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563ce9821d80 con 0x563ce98bbd60 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08dcff9640 1 -- 172.21.15.18:0/2358335993 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08ec00fd60 con 0x563ce98bbd60 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08dcff9640 1 -- 172.21.15.18:0/2358335993 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f08f001a040 con 0x563ce98bbd60 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08fe5f3740 10 client.16761 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:53:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08fe5f3740 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08fe5f3740 20 client.16761 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ce9930b50 0x563ce9932f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08fe5f3740 4 client.16761 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:53:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08fe5f3740 1 -- 172.21.15.18:0/2358335993 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563ce9560550 con 0x563ce9930b50 2024-02-17T12:53:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.183+0000 7f08f8fb7640 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ce9930b50 0x563ce9932f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.184+0000 7f08f8fb7640 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ce9930b50 0x563ce9932f70 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7be8d8b87836d62d server_cookie=d463eea4080451c1 in_seq=0 out_seq=0 2024-02-17T12:53:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.184+0000 7f08dcff9640 10 client.16761 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08dcff9640 1 -- 172.21.15.18:0/2358335993 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563ce9560550 con 0x563ce9930b50 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08dcff9640 10 client.16761 handle_command_reply: tid=0 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08fe5f3740 1 client.16761 shutdown 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08fe5f3740 1 -- 172.21.15.18:0/2358335993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98bbd60 msgr2=0x563ce98ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98bbd60 0x563ce98ca0e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f08f00037e0 tx=0x7f08f0014000 comp rx=0 tx=0).stop 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08fe5f3740 1 -- 172.21.15.18:0/2358335993 shutdown_connections 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ce9930b50 0x563ce9932f70 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ce98c52e0 0x563ce98cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ce98bcad0 0x563ce98ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08dcff9640 0 client.16761 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08fe5f3740 1 --2- 172.21.15.18:0/2358335993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce98bbd60 0x563ce98ca0e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08fe5f3740 1 -- 172.21.15.18:0/2358335993 >> 172.21.15.18:0/2358335993 conn(0x563ce9883550 msgr2=0x563ce98b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08fe5f3740 1 -- 172.21.15.18:0/2358335993 shutdown_connections 2024-02-17T12:53:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08fe5f3740 1 -- 172.21.15.18:0/2358335993 wait complete. 2024-02-17T12:53:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.238+0000 7f08fe5f3740 20 client.16761 trim_cache size 0 max 16384 2024-02-17T12:53:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:53:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:53:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:53:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:53:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.239+0000 7f08fba1c640 1 -- 172.21.15.18:0/3041191604 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f08e00a22d0 msgr2=0x7f08e00a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.239+0000 7f08fba1c640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f08e00a22d0 0x7f08e00a4790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009b50 tx=0x7f08f0005f90 comp rx=0 tx=0).stop 2024-02-17T12:53:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.239+0000 7f08fba1c640 1 -- 172.21.15.18:0/3041191604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f4100c90 msgr2=0x7f08f419dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.239+0000 7f08fba1c640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f4100c90 0x7f08f419dac0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f08e4005940 tx=0x7f08e402c670 comp rx=0 tx=0).stop 2024-02-17T12:53:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.239+0000 7f08fba1c640 1 -- 172.21.15.18:0/3041191604 shutdown_connections 2024-02-17T12:53:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.239+0000 7f08fba1c640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f08e00a22d0 0x7f08e00a4790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.239+0000 7f08fba1c640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08f4105020 0x7f08f4197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.239+0000 7f08fba1c640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08f4100c90 0x7f08f419dac0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.239+0000 7f08fba1c640 1 --2- 172.21.15.18:0/3041191604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f41002d0 0x7f08f419d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.239+0000 7f08fba1c640 1 -- 172.21.15.18:0/3041191604 >> 172.21.15.18:0/3041191604 conn(0x7f08f4076ed0 msgr2=0x7f08f40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.239+0000 7f08fba1c640 1 -- 172.21.15.18:0/3041191604 shutdown_connections 2024-02-17T12:53:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:04.239+0000 7f08fba1c640 1 -- 172.21.15.18:0/3041191604 wait complete. 2024-02-17T12:53:04.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 143551 10.68 MB/sec execute 19 sec latency 31.067 ms 2024-02-17T12:53:04.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:04 smithi146 ceph-mon[36454]: pgmap v98: 97 pgs: 97 active+clean; 65 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-02-17T12:53:04.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:04 smithi018 ceph-mon[32625]: pgmap v98: 97 pgs: 97 active+clean; 65 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-02-17T12:53:04.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:04 smithi119 ceph-mon[36188]: pgmap v98: 97 pgs: 97 active+clean; 65 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-02-17T12:53:05.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 145846 10.84 MB/sec execute 20 sec latency 10.756 ms 2024-02-17T12:53:06.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 148182 10.85 MB/sec execute 21 sec latency 27.146 ms 2024-02-17T12:53:06.644 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:06 smithi119 ceph-mon[36188]: pgmap v99: 97 pgs: 97 active+clean; 66 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 428 op/s 2024-02-17T12:53:06.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:06 smithi146 ceph-mon[36454]: pgmap v99: 97 pgs: 97 active+clean; 66 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 428 op/s 2024-02-17T12:53:06.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:06 smithi018 ceph-mon[32625]: pgmap v99: 97 pgs: 97 active+clean; 66 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 428 op/s 2024-02-17T12:53:07.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 149875 10.76 MB/sec execute 22 sec latency 13.421 ms 2024-02-17T12:53:08.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 152764 11.08 MB/sec execute 23 sec latency 9.409 ms 2024-02-17T12:53:08.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:08 smithi146 ceph-mon[36454]: pgmap v100: 97 pgs: 97 active+clean; 66 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 282 op/s 2024-02-17T12:53:08.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:08 smithi018 ceph-mon[32625]: pgmap v100: 97 pgs: 97 active+clean; 66 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 282 op/s 2024-02-17T12:53:08.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:08 smithi119 ceph-mon[36188]: pgmap v100: 97 pgs: 97 active+clean; 66 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 282 op/s 2024-02-17T12:53:09.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 154421 10.86 MB/sec execute 24 sec latency 33.246 ms 2024-02-17T12:53:10.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 156478 10.97 MB/sec execute 25 sec latency 26.900 ms 2024-02-17T12:53:10.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:10 smithi146 ceph-mon[36454]: pgmap v101: 97 pgs: 97 active+clean; 64 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 424 op/s 2024-02-17T12:53:10.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:10 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:53:10.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:10 smithi018 ceph-mon[32625]: pgmap v101: 97 pgs: 97 active+clean; 64 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 424 op/s 2024-02-17T12:53:10.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:10 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:53:10.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:10 smithi119 ceph-mon[36188]: pgmap v101: 97 pgs: 97 active+clean; 64 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 424 op/s 2024-02-17T12:53:10.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:10 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:53:11.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 158701 10.86 MB/sec execute 26 sec latency 9.817 ms 2024-02-17T12:53:11.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:11 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:53:11.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:11 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:53:11.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:11 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:53:11.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:11 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:53:11.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:11 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:53:11.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:11 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:53:11.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:11 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:53:11.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:11 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:53:11.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:11 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:53:11.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:11 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:53:11.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:11 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:53:11.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:11 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:53:12.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 160429 10.88 MB/sec execute 27 sec latency 13.641 ms 2024-02-17T12:53:12.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:12 smithi146 ceph-mon[36454]: pgmap v102: 97 pgs: 97 active+clean; 64 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-02-17T12:53:12.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:12 smithi018 ceph-mon[32625]: pgmap v102: 97 pgs: 97 active+clean; 64 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-02-17T12:53:12.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:12 smithi119 ceph-mon[36188]: pgmap v102: 97 pgs: 97 active+clean; 64 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-02-17T12:53:13.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 163149 11.10 MB/sec execute 28 sec latency 9.980 ms 2024-02-17T12:53:14.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 164367 10.85 MB/sec execute 29 sec latency 29.440 ms 2024-02-17T12:53:14.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:14 smithi146 ceph-mon[36454]: pgmap v103: 97 pgs: 97 active+clean; 64 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-02-17T12:53:14.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:14 smithi018 ceph-mon[32625]: pgmap v103: 97 pgs: 97 active+clean; 64 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-02-17T12:53:14.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:14 smithi119 ceph-mon[36188]: pgmap v103: 97 pgs: 97 active+clean; 64 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-02-17T12:53:15.267 INFO:tasks.workunit.client.0.smithi018.stdout: 1 166963 11.04 MB/sec execute 30 sec latency 21.966 ms 2024-02-17T12:53:16.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 169079 10.92 MB/sec execute 31 sec latency 11.423 ms 2024-02-17T12:53:16.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:16 smithi146 ceph-mon[36454]: pgmap v104: 97 pgs: 97 active+clean; 71 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 431 op/s 2024-02-17T12:53:16.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:16 smithi018 ceph-mon[32625]: pgmap v104: 97 pgs: 97 active+clean; 71 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 431 op/s 2024-02-17T12:53:16.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:16 smithi119 ceph-mon[36188]: pgmap v104: 97 pgs: 97 active+clean; 71 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 431 op/s 2024-02-17T12:53:17.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 170753 10.96 MB/sec execute 32 sec latency 14.792 ms 2024-02-17T12:53:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.333+0000 7fa51fb25640 1 -- 172.21.15.18:0/1492382808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa518107270 msgr2=0x7fa518107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.333+0000 7fa51fb25640 1 --2- 172.21.15.18:0/1492382808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa518107270 0x7fa518107650 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa510009900 tx=0x7fa51002bcd0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.333+0000 7fa51fb25640 1 -- 172.21.15.18:0/1492382808 shutdown_connections 2024-02-17T12:53:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.333+0000 7fa51fb25640 1 --2- 172.21.15.18:0/1492382808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5180691a0 0x7fa51810e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.333+0000 7fa51fb25640 1 --2- 172.21.15.18:0/1492382808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa518068800 0x7fa518068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.333+0000 7fa51fb25640 1 --2- 172.21.15.18:0/1492382808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa518107270 0x7fa518107650 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.333+0000 7fa51fb25640 1 -- 172.21.15.18:0/1492382808 >> 172.21.15.18:0/1492382808 conn(0x7fa518076ed0 msgr2=0x7fa5180772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.333+0000 7fa51fb25640 1 -- 172.21.15.18:0/1492382808 shutdown_connections 2024-02-17T12:53:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.333+0000 7fa51fb25640 1 -- 172.21.15.18:0/1492382808 wait complete. 2024-02-17T12:53:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.333+0000 7fa51fb25640 1 Processor -- start 2024-02-17T12:53:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.333+0000 7fa51fb25640 1 -- start start 2024-02-17T12:53:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51fb25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa518068800 0x7fa518197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51fb25640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5180691a0 0x7fa518197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51fb25640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa518107270 0x7fa51819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51fb25640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51819c550 con 0x7fa518068800 2024-02-17T12:53:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51fb25640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51819c6c0 con 0x7fa5180691a0 2024-02-17T12:53:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51fb25640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51819c880 con 0x7fa518107270 2024-02-17T12:53:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51d8c1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa518068800 0x7fa518197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51d8c1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa518068800 0x7fa518197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60994/0 (socket says 172.21.15.18:60994) 2024-02-17T12:53:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51d8c1640 1 -- 172.21.15.18:0/403579094 learned_addr learned my addr 172.21.15.18:0/403579094 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51d0c0640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5180691a0 0x7fa518197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51e0c2640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa518107270 0x7fa51819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51d8c1640 1 -- 172.21.15.18:0/403579094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa518107270 msgr2=0x7fa51819bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa518107270 0x7fa51819bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51d8c1640 1 -- 172.21.15.18:0/403579094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5180691a0 msgr2=0x7fa518197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5180691a0 0x7fa518197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51d8c1640 1 -- 172.21.15.18:0/403579094 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c009490 con 0x7fa518068800 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51e0c2640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa518107270 0x7fa51819bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51d0c0640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5180691a0 0x7fa518197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa518068800 0x7fa518197690 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa510005940 tx=0x7fa51002c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa516ffd640 1 -- 172.21.15.18:0/403579094 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510004350 con 0x7fa518068800 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa516ffd640 1 -- 172.21.15.18:0/403579094 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5100044b0 con 0x7fa518068800 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51fb25640 1 -- 172.21.15.18:0/403579094 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa510009560 con 0x7fa518068800 2024-02-17T12:53:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa516ffd640 1 -- 172.21.15.18:0/403579094 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510003d30 con 0x7fa518068800 2024-02-17T12:53:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.334+0000 7fa51fb25640 1 -- 172.21.15.18:0/403579094 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5181a4200 con 0x7fa518068800 2024-02-17T12:53:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.336+0000 7fa516ffd640 1 -- 172.21.15.18:0/403579094 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa510004df0 con 0x7fa518068800 2024-02-17T12:53:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.337+0000 7fa5227d5740 1 Processor -- start 2024-02-17T12:53:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.337+0000 7fa5227d5740 1 -- start start 2024-02-17T12:53:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.337+0000 7fa5227d5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359dad0 0x5555b359c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.337+0000 7fa5227d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359cd60 0x5555b359d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.337+0000 7fa5227d5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 0x5555b35a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.337+0000 7fa5227d5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b350b9f0 con 0x5555b35a62e0 2024-02-17T12:53:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.337+0000 7fa5227d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b3508f20 con 0x5555b359cd60 2024-02-17T12:53:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.337+0000 7fa5227d5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b3504700 con 0x5555b359dad0 2024-02-17T12:53:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.337+0000 7fa516ffd640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa5000a22d0 0x7fa5000a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51d8c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359cd60 0x5555b359d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51d0c0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 0x5555b35a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51e0c2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359dad0 0x5555b359c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51d0c0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 0x5555b35a86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32772/0 (socket says 172.21.15.18:32772) 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51d0c0640 1 -- 172.21.15.18:0/2414611862 learned_addr learned my addr 172.21.15.18:0/2414611862 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51e0c2640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa5000a22d0 0x7fa5000a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa516ffd640 1 -- 172.21.15.18:0/403579094 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa5100f0750 con 0x7fa518068800 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51d8c1640 1 -- 172.21.15.18:0/2414611862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359dad0 msgr2=0x5555b359c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/2414611862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359dad0 0x5555b359c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51d8c1640 1 -- 172.21.15.18:0/2414611862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 msgr2=0x5555b35a86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/2414611862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 0x5555b35a86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51d8c1640 1 -- 172.21.15.18:0/2414611862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555b349a7b0 con 0x5555b359cd60 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51e0c2640 1 --2- 172.21.15.18:0/2414611862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359dad0 0x5555b359c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51d0c0640 1 --2- 172.21.15.18:0/2414611862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 0x5555b35a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/2414611862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359cd60 0x5555b359d1e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa5100ee000 tx=0x7fa510090430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1eb7c724ec3023b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa51e0c2640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa5000a22d0 0x7fa5000a4790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa508005fd0 tx=0x7fa508009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa4fffff640 1 -- 172.21.15.18:0/2414611862 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510088480 con 0x5555b359cd60 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa4fffff640 1 -- 172.21.15.18:0/2414611862 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5100bd9a0 con 0x5555b359cd60 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.338+0000 7fa4fffff640 1 -- 172.21.15.18:0/2414611862 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100055d0 con 0x5555b359cd60 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.339+0000 7fa5227d5740 1 -- 172.21.15.18:0/2414611862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359cd60 msgr2=0x5555b359d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.339+0000 7fa5227d5740 1 --2- 172.21.15.18:0/2414611862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359cd60 0x5555b359d1e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa5100ee000 tx=0x7fa510090430 comp rx=0 tx=0).stop 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.339+0000 7fa5227d5740 1 -- 172.21.15.18:0/2414611862 shutdown_connections 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.339+0000 7fa5227d5740 1 --2- 172.21.15.18:0/2414611862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 0x5555b35a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.339+0000 7fa5227d5740 1 --2- 172.21.15.18:0/2414611862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359cd60 0x5555b359d1e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.339+0000 7fa5227d5740 1 --2- 172.21.15.18:0/2414611862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359dad0 0x5555b359c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.339+0000 7fa5227d5740 1 -- 172.21.15.18:0/2414611862 >> 172.21.15.18:0/2414611862 conn(0x5555b3564550 msgr2=0x5555b3597a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.339+0000 7fa5227d5740 1 -- 172.21.15.18:0/2414611862 shutdown_connections 2024-02-17T12:53:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.339+0000 7fa5227d5740 1 -- 172.21.15.18:0/2414611862 wait complete. 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa5227d5740 1 -- start start 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa5227d5740 1 Processor -- start 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa5227d5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359cd60 0x5555b35a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa5227d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359dad0 0x5555b35a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa5227d5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 0x5555b35a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa5227d5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b34a9bf0 con 0x5555b35a62e0 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa5227d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b31d8250 con 0x5555b359dad0 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa5227d5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b348a200 con 0x5555b359cd60 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa5227d5740 10 client.0 _ll_register_callbacks cb 0x5555b34f8d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa51e0c2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 0x5555b35a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa51d0c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359dad0 0x5555b35a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa51e0c2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 0x5555b35a3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32780/0 (socket says 172.21.15.18:32780) 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa51e0c2640 1 -- 172.21.15.18:0/601197448 learned_addr learned my addr 172.21.15.18:0/601197448 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.340+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359cd60 0x5555b35a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa51e0c2640 1 -- 172.21.15.18:0/601197448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359cd60 msgr2=0x5555b35a3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa51e0c2640 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359cd60 0x5555b35a3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa51e0c2640 1 -- 172.21.15.18:0/601197448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359dad0 msgr2=0x5555b35a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa51e0c2640 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359dad0 0x5555b35a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa51e0c2640 1 -- 172.21.15.18:0/601197448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5181a4200 con 0x5555b35a62e0 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359cd60 0x5555b35a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa51d0c0640 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359dad0 0x5555b35a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa51e0c2640 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 0x5555b35a3dc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa5080122f0 tx=0x7fa508004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a222adc09d1613d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa4fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa4fd7fa640 1 -- 172.21.15.18:0/601197448 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508012e90 con 0x5555b35a62e0 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa4fd7fa640 1 -- 172.21.15.18:0/601197448 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa508011b50 con 0x5555b35a62e0 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa5227d5740 1 -- 172.21.15.18:0/601197448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555b3204600 con 0x5555b35a62e0 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa4fd7fa640 1 -- 172.21.15.18:0/601197448 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa508011d10 con 0x5555b35a62e0 2024-02-17T12:53:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa5227d5740 10 client.16773 fetch_fsmap learned FSMap version 29 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa5227d5740 1 -- 172.21.15.18:0/601197448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5555b3502d80 con 0x5555b35a62e0 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa4fd7fa640 1 -- 172.21.15.18:0/601197448 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5555b3204600 con 0x5555b35a62e0 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.341+0000 7fa4fd7fa640 1 -- 172.21.15.18:0/601197448 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa508005550 con 0x5555b35a62e0 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.342+0000 7fa5227d5740 10 client.16773 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.342+0000 7fa5227d5740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.342+0000 7fa5227d5740 20 client.16773 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.342+0000 7fa5227d5740 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555b35b64b0 0x5555b35b88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.342+0000 7fa5227d5740 4 client.16773 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.342+0000 7fa5227d5740 1 -- 172.21.15.18:0/601197448 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555b32551b0 con 0x5555b35b64b0 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.342+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555b35b64b0 0x5555b35b88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.342+0000 7fa4fd7fa640 10 client.16773 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.342+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555b35b64b0 0x5555b35b88d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aeebeb114986180 server_cookie=85991ee3f914303 in_seq=0 out_seq=0 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa4fd7fa640 1 -- 172.21.15.18:0/601197448 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5555b32551b0 con 0x5555b35b64b0 2024-02-17T12:53:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa4fd7fa640 10 client.16773 handle_command_reply: tid=0 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa5227d5740 1 client.16773 shutdown 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa5227d5740 1 -- 172.21.15.18:0/601197448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 msgr2=0x5555b35a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa5227d5740 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 0x5555b35a3dc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa5080122f0 tx=0x7fa508004ab0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa5227d5740 1 -- 172.21.15.18:0/601197448 shutdown_connections 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa5227d5740 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555b35b64b0 0x5555b35b88d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa5227d5740 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b35a62e0 0x5555b35a3dc0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa5227d5740 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359dad0 0x5555b35a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa5227d5740 1 --2- 172.21.15.18:0/601197448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359cd60 0x5555b35a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa4fd7fa640 0 client.16773 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa5227d5740 1 -- 172.21.15.18:0/601197448 >> 172.21.15.18:0/601197448 conn(0x5555b3564550 msgr2=0x5555b35997a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa5227d5740 1 -- 172.21.15.18:0/601197448 shutdown_connections 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa5227d5740 1 -- 172.21.15.18:0/601197448 wait complete. 2024-02-17T12:53:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.343+0000 7fa5227d5740 20 client.16773 trim_cache size 0 max 16384 2024-02-17T12:53:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.357+0000 7fa5227d5740 1 Processor -- start 2024-02-17T12:53:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.357+0000 7fa5227d5740 1 -- start start 2024-02-17T12:53:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.357+0000 7fa5227d5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 0x5555b35a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.357+0000 7fa5227d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359dad0 0x5555b35a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.357+0000 7fa5227d5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b35a62e0 0x5555b35a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.357+0000 7fa5227d5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b3598850 con 0x5555b359cd60 2024-02-17T12:53:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.357+0000 7fa5227d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b3597310 con 0x5555b359dad0 2024-02-17T12:53:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.357+0000 7fa5227d5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b3598aa0 con 0x5555b35a62e0 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.357+0000 7fa51d8c1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 0x5555b35a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.357+0000 7fa51d8c1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 0x5555b35a5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32790/0 (socket says 172.21.15.18:32790) 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.357+0000 7fa51d8c1640 1 -- 172.21.15.18:0/1236956061 learned_addr learned my addr 172.21.15.18:0/1236956061 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.358+0000 7fa51d0c0640 1 --2- 172.21.15.18:0/1236956061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359dad0 0x5555b35a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.358+0000 7fa51e0c2640 1 --2- 172.21.15.18:0/1236956061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b35a62e0 0x5555b35a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.358+0000 7fa51d8c1640 1 -- 172.21.15.18:0/1236956061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b35a62e0 msgr2=0x5555b35a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.358+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/1236956061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b35a62e0 0x5555b35a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.358+0000 7fa51d8c1640 1 -- 172.21.15.18:0/1236956061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359dad0 msgr2=0x5555b35a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.358+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/1236956061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359dad0 0x5555b35a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.358+0000 7fa51d8c1640 1 -- 172.21.15.18:0/1236956061 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555b359cb10 con 0x5555b359cd60 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.358+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/1236956061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 0x5555b35a5a70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa5100b95c0 tx=0x7fa5100bdb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527b1556d3810b5a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.358+0000 7fa4ff7fe640 1 -- 172.21.15.18:0/1236956061 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100c5ad0 con 0x5555b359cd60 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.358+0000 7fa4ff7fe640 1 -- 172.21.15.18:0/1236956061 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5100888b0 con 0x5555b359cd60 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.358+0000 7fa4ff7fe640 1 -- 172.21.15.18:0/1236956061 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100f0a10 con 0x5555b359cd60 2024-02-17T12:53:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 -- 172.21.15.18:0/1236956061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 msgr2=0x5555b35a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 --2- 172.21.15.18:0/1236956061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 0x5555b35a5a70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa5100b95c0 tx=0x7fa5100bdb10 comp rx=0 tx=0).stop 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 -- 172.21.15.18:0/1236956061 shutdown_connections 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 --2- 172.21.15.18:0/1236956061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b35a62e0 0x5555b35a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 --2- 172.21.15.18:0/1236956061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b359dad0 0x5555b35a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 --2- 172.21.15.18:0/1236956061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 0x5555b35a5a70 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 -- 172.21.15.18:0/1236956061 >> 172.21.15.18:0/1236956061 conn(0x5555b3564550 msgr2=0x5555b35a1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 -- 172.21.15.18:0/1236956061 shutdown_connections 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 -- 172.21.15.18:0/1236956061 wait complete. 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 -- start start 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 Processor -- start 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 0x5555b35ab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.359+0000 7fa5227d5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359dad0 0x5555b35ab940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa5227d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b35a62e0 0x5555b35ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa5227d5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b35ace30 con 0x5555b359cd60 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa5227d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b35acfa0 con 0x5555b35a62e0 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa5227d5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b35ad160 con 0x5555b359dad0 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa5227d5740 10 client.0 _ll_register_callbacks cb 0x7fa5100bcde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa51d8c1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 0x5555b35ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa51d8c1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 0x5555b35ab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32792/0 (socket says 172.21.15.18:32792) 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa51d8c1640 1 -- 172.21.15.18:0/2555795424 learned_addr learned my addr 172.21.15.18:0/2555795424 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa51d0c0640 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359dad0 0x5555b35ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa51e0c2640 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b35a62e0 0x5555b35ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa51d8c1640 1 -- 172.21.15.18:0/2555795424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359dad0 msgr2=0x5555b35ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359dad0 0x5555b35ab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa51d8c1640 1 -- 172.21.15.18:0/2555795424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b35a62e0 msgr2=0x5555b35ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b35a62e0 0x5555b35ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa51d8c1640 1 -- 172.21.15.18:0/2555795424 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555b349a7b0 con 0x5555b359cd60 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa51d8c1640 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 0x5555b35ab400 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa510036040 tx=0x7fa5100c1550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5476ee144636acaa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa4fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa4fd7fa640 1 -- 172.21.15.18:0/2555795424 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51008c070 con 0x5555b359cd60 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa4fd7fa640 1 -- 172.21.15.18:0/2555795424 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa51002ce40 con 0x5555b359cd60 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa5227d5740 1 -- 172.21.15.18:0/2555795424 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5100bd9a0 con 0x5555b359cd60 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa4fd7fa640 1 -- 172.21.15.18:0/2555795424 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa510084ea0 con 0x5555b359cd60 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa5227d5740 10 client.16779 fetch_fsmap learned FSMap version 29 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.360+0000 7fa5227d5740 1 -- 172.21.15.18:0/2555795424 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5555b349f7b0 con 0x5555b359cd60 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.361+0000 7fa4fd7fa640 1 -- 172.21.15.18:0/2555795424 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5100d9080 con 0x5555b359cd60 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.361+0000 7fa4fd7fa640 1 -- 172.21.15.18:0/2555795424 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa51003c070 con 0x5555b359cd60 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.361+0000 7fa5227d5740 10 client.16779 fetch_fsmap finished waiting for FSMap version 29 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.361+0000 7fa5227d5740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.361+0000 7fa5227d5740 20 client.16779 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.361+0000 7fa5227d5740 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555b360d810 0x5555b360fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.361+0000 7fa5227d5740 4 client.16779 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:53:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.361+0000 7fa5227d5740 1 -- 172.21.15.18:0/2555795424 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5555b3241550 con 0x5555b360d810 2024-02-17T12:53:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.361+0000 7fa51d0c0640 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555b360d810 0x5555b360fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.361+0000 7fa4fd7fa640 10 client.16779 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.361+0000 7fa51d0c0640 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555b360d810 0x5555b360fc30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e639df662a7e8e server_cookie=49c599eff0e5a1a5 in_seq=0 out_seq=0 2024-02-17T12:53:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.410+0000 7fa4fd7fa640 1 -- 172.21.15.18:0/2555795424 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5555b3241550 con 0x5555b360d810 2024-02-17T12:53:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.410+0000 7fa4fd7fa640 10 client.16779 handle_command_reply: tid=0 2024-02-17T12:53:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.410+0000 7fa5227d5740 1 client.16779 shutdown 2024-02-17T12:53:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.410+0000 7fa5227d5740 1 -- 172.21.15.18:0/2555795424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 msgr2=0x5555b35ab400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.410+0000 7fa5227d5740 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 0x5555b35ab400 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa510036040 tx=0x7fa5100c1550 comp rx=0 tx=0).stop 2024-02-17T12:53:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.410+0000 7fa5227d5740 1 -- 172.21.15.18:0/2555795424 shutdown_connections 2024-02-17T12:53:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.410+0000 7fa5227d5740 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555b360d810 0x5555b360fc30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.410+0000 7fa5227d5740 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555b35a62e0 0x5555b35ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.410+0000 7fa5227d5740 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555b359dad0 0x5555b35ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.410+0000 7fa5227d5740 1 --2- 172.21.15.18:0/2555795424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555b359cd60 0x5555b35ab400 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.410+0000 7fa4fd7fa640 0 client.16779 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.410+0000 7fa5227d5740 1 -- 172.21.15.18:0/2555795424 >> 172.21.15.18:0/2555795424 conn(0x5555b3564550 msgr2=0x5555b3596300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa5227d5740 1 -- 172.21.15.18:0/2555795424 shutdown_connections 2024-02-17T12:53:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa5227d5740 1 -- 172.21.15.18:0/2555795424 wait complete. 2024-02-17T12:53:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa5227d5740 20 client.16779 trim_cache size 0 max 16384 2024-02-17T12:53:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:53:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:53:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:53:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:53:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa51fb25640 1 -- 172.21.15.18:0/403579094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa5000a22d0 msgr2=0x7fa5000a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa51fb25640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa5000a22d0 0x7fa5000a4790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa508005fd0 tx=0x7fa508009500 comp rx=0 tx=0).stop 2024-02-17T12:53:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa51fb25640 1 -- 172.21.15.18:0/403579094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa518068800 msgr2=0x7fa518197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa51fb25640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa518068800 0x7fa518197690 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa510005940 tx=0x7fa51002c710 comp rx=0 tx=0).stop 2024-02-17T12:53:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa51fb25640 1 -- 172.21.15.18:0/403579094 shutdown_connections 2024-02-17T12:53:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa51fb25640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa5000a22d0 0x7fa5000a4790 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa51fb25640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa518107270 0x7fa51819bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa51fb25640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5180691a0 0x7fa518197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa51fb25640 1 --2- 172.21.15.18:0/403579094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa518068800 0x7fa518197690 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa51fb25640 1 -- 172.21.15.18:0/403579094 >> 172.21.15.18:0/403579094 conn(0x7fa518076ed0 msgr2=0x7fa5180fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa51fb25640 1 -- 172.21.15.18:0/403579094 shutdown_connections 2024-02-17T12:53:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:17.411+0000 7fa51fb25640 1 -- 172.21.15.18:0/403579094 wait complete. 2024-02-17T12:53:18.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 173262 11.01 MB/sec execute 33 sec latency 29.291 ms 2024-02-17T12:53:18.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:18 smithi146 ceph-mon[36454]: pgmap v105: 97 pgs: 97 active+clean; 71 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2024-02-17T12:53:18.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:18 smithi018 ceph-mon[32625]: pgmap v105: 97 pgs: 97 active+clean; 71 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2024-02-17T12:53:18.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:18 smithi119 ceph-mon[36188]: pgmap v105: 97 pgs: 97 active+clean; 71 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2024-02-17T12:53:19.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 174746 10.90 MB/sec execute 34 sec latency 35.426 ms 2024-02-17T12:53:20.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 177448 11.07 MB/sec execute 35 sec latency 32.006 ms 2024-02-17T12:53:20.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:20 smithi146 ceph-mon[36454]: pgmap v106: 97 pgs: 97 active+clean; 68 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 441 op/s 2024-02-17T12:53:20.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:20 smithi018 ceph-mon[32625]: pgmap v106: 97 pgs: 97 active+clean; 68 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 441 op/s 2024-02-17T12:53:20.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:20 smithi119 ceph-mon[36188]: pgmap v106: 97 pgs: 97 active+clean; 68 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 441 op/s 2024-02-17T12:53:21.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 178975 10.95 MB/sec execute 36 sec latency 10.739 ms 2024-02-17T12:53:21.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:21 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:53:21.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:21 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:53:21.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:21 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:53:22.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 181281 11.03 MB/sec execute 37 sec latency 13.185 ms 2024-02-17T12:53:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:22 smithi146 ceph-mon[36454]: pgmap v107: 97 pgs: 97 active+clean; 68 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 299 op/s 2024-02-17T12:53:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:22 smithi018 ceph-mon[32625]: pgmap v107: 97 pgs: 97 active+clean; 68 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 299 op/s 2024-02-17T12:53:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:22 smithi119 ceph-mon[36188]: pgmap v107: 97 pgs: 97 active+clean; 68 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 299 op/s 2024-02-17T12:53:23.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 183240 10.92 MB/sec execute 38 sec latency 64.539 ms 2024-02-17T12:53:24.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 184878 10.95 MB/sec execute 39 sec latency 40.717 ms 2024-02-17T12:53:24.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:24 smithi146 ceph-mon[36454]: pgmap v108: 97 pgs: 97 active+clean; 68 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 299 op/s 2024-02-17T12:53:24.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:24 smithi018 ceph-mon[32625]: pgmap v108: 97 pgs: 97 active+clean; 68 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 299 op/s 2024-02-17T12:53:24.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:24 smithi119 ceph-mon[36188]: pgmap v108: 97 pgs: 97 active+clean; 68 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 299 op/s 2024-02-17T12:53:25.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 187214 10.95 MB/sec execute 40 sec latency 17.141 ms 2024-02-17T12:53:26.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 188795 10.90 MB/sec execute 41 sec latency 12.309 ms 2024-02-17T12:53:26.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:26 smithi146 ceph-mon[36454]: pgmap v109: 97 pgs: 97 active+clean; 73 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-02-17T12:53:26.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:26 smithi018 ceph-mon[32625]: pgmap v109: 97 pgs: 97 active+clean; 73 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-02-17T12:53:26.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:26 smithi119 ceph-mon[36188]: pgmap v109: 97 pgs: 97 active+clean; 73 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-02-17T12:53:27.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 191513 11.04 MB/sec execute 42 sec latency 9.060 ms 2024-02-17T12:53:28.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 193000 10.94 MB/sec execute 43 sec latency 24.957 ms 2024-02-17T12:53:28.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:28 smithi146 ceph-mon[36454]: pgmap v110: 97 pgs: 97 active+clean; 73 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 304 op/s 2024-02-17T12:53:28.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:28 smithi018 ceph-mon[32625]: pgmap v110: 97 pgs: 97 active+clean; 73 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 304 op/s 2024-02-17T12:53:28.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:28 smithi119 ceph-mon[36188]: pgmap v110: 97 pgs: 97 active+clean; 73 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 304 op/s 2024-02-17T12:53:29.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 195408 11.01 MB/sec execute 44 sec latency 28.695 ms 2024-02-17T12:53:30.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 197586 10.94 MB/sec execute 45 sec latency 16.080 ms 2024-02-17T12:53:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.504+0000 7fa688dac640 1 -- 172.21.15.18:0/979132934 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa684101d70 msgr2=0x7fa684105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.504+0000 7fa688dac640 1 --2- 172.21.15.18:0/979132934 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa684101d70 0x7fa684105db0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa678009930 tx=0x7fa67802bc30 comp rx=0 tx=0).stop 2024-02-17T12:53:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.504+0000 7fa688dac640 1 -- 172.21.15.18:0/979132934 shutdown_connections 2024-02-17T12:53:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.504+0000 7fa688dac640 1 --2- 172.21.15.18:0/979132934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa684106490 0x7fa68410e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.504+0000 7fa688dac640 1 --2- 172.21.15.18:0/979132934 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa684101d70 0x7fa684105db0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.504+0000 7fa688dac640 1 --2- 172.21.15.18:0/979132934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6841013c0 0x7fa6841017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.504+0000 7fa688dac640 1 -- 172.21.15.18:0/979132934 >> 172.21.15.18:0/979132934 conn(0x7fa6840fcc60 msgr2=0x7fa6840ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.504+0000 7fa688dac640 1 -- 172.21.15.18:0/979132934 shutdown_connections 2024-02-17T12:53:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.504+0000 7fa688dac640 1 -- 172.21.15.18:0/979132934 wait complete. 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa688dac640 1 Processor -- start 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa688dac640 1 -- start start 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa688dac640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6841013c0 0x7fa68419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa688dac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa684106490 0x7fa68419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa688dac640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa684197590 0x7fa684197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa688dac640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa684197f80 con 0x7fa6841013c0 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa688dac640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6841980f0 con 0x7fa684106490 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa688dac640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6841982b0 con 0x7fa684197590 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa68259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6841013c0 0x7fa68419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa68259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6841013c0 0x7fa68419d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60354/0 (socket says 172.21.15.18:60354) 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa681d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa684106490 0x7fa68419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa68259c640 1 -- 172.21.15.18:0/3769219625 learned_addr learned my addr 172.21.15.18:0/3769219625 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa682d9d640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa684197590 0x7fa684197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa68259c640 1 -- 172.21.15.18:0/3769219625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa684197590 msgr2=0x7fa684197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa68259c640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa684197590 0x7fa684197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa68259c640 1 -- 172.21.15.18:0/3769219625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa684106490 msgr2=0x7fa68419d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa68259c640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa684106490 0x7fa68419d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa68259c640 1 -- 172.21.15.18:0/3769219625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678009560 con 0x7fa6841013c0 2024-02-17T12:53:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa681d9b640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa684106490 0x7fa68419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.505+0000 7fa68259c640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6841013c0 0x7fa68419d3e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa67400d770 tx=0x7fa67400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.506+0000 7fa6737fe640 1 -- 172.21.15.18:0/3769219625 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674009c30 con 0x7fa6841013c0 2024-02-17T12:53:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.506+0000 7fa688dac640 1 -- 172.21.15.18:0/3769219625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6841a42a0 con 0x7fa6841013c0 2024-02-17T12:53:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.506+0000 7fa6737fe640 1 -- 172.21.15.18:0/3769219625 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa67400de80 con 0x7fa6841013c0 2024-02-17T12:53:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.506+0000 7fa6737fe640 1 -- 172.21.15.18:0/3769219625 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674004e80 con 0x7fa6841013c0 2024-02-17T12:53:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.506+0000 7fa688dac640 1 -- 172.21.15.18:0/3769219625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6841a4730 con 0x7fa6841013c0 2024-02-17T12:53:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.508+0000 7fa6737fe640 1 -- 172.21.15.18:0/3769219625 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa674005020 con 0x7fa6841013c0 2024-02-17T12:53:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.508+0000 7fa68b97d740 1 Processor -- start 2024-02-17T12:53:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.508+0000 7fa68b97d740 1 -- start start 2024-02-17T12:53:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa68b97d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe18820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa6737fe640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa66c0a22d0 0x7fa66c0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa68b97d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 0x5555fbe191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa68b97d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe266f0 0x5555fbe28ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa68b97d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555fbd879f0 con 0x5555fbe18d60 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa68b97d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555fbd84f20 con 0x5555fbe19ad0 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa68b97d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555fbd80700 con 0x5555fbe266f0 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa68259c640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa66c0a22d0 0x7fa66c0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa681d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 0x5555fbe191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa682d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe18820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa681d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 0x5555fbe191e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60356/0 (socket says 172.21.15.18:60356) 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1347566624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe18820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:37282/0 (socket says 172.21.15.18:37282) 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa681d9b640 1 -- 172.21.15.18:0/1347566624 learned_addr learned my addr 172.21.15.18:0/1347566624 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa6737fe640 1 -- 172.21.15.18:0/3769219625 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa6740cda00 con 0x7fa6841013c0 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1347566624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe266f0 0x5555fbe28ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa682d9d640 1 -- 172.21.15.18:0/1347566624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe266f0 msgr2=0x5555fbe28ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1347566624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe266f0 0x5555fbe28ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa682d9d640 1 -- 172.21.15.18:0/1347566624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 msgr2=0x5555fbe191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1347566624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 0x5555fbe191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa682d9d640 1 -- 172.21.15.18:0/1347566624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555fbd167b0 con 0x5555fbe19ad0 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa68259c640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa66c0a22d0 0x7fa66c0a4790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa6740cb250 tx=0x7fa6740ba040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.509+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1347566624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe18820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa67c005fd0 tx=0x7fa67c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d2cf2378f2779b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.510+0000 7fa670ff9640 1 -- 172.21.15.18:0/1347566624 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c011030 con 0x5555fbe19ad0 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.510+0000 7fa670ff9640 1 -- 172.21.15.18:0/1347566624 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa67c017070 con 0x5555fbe19ad0 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.510+0000 7fa670ff9640 1 -- 172.21.15.18:0/1347566624 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00fd90 con 0x5555fbe19ad0 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.510+0000 7fa68b97d740 1 -- 172.21.15.18:0/1347566624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 msgr2=0x5555fbe18820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.510+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1347566624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe18820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa67c005fd0 tx=0x7fa67c004ab0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.510+0000 7fa68b97d740 1 -- 172.21.15.18:0/1347566624 shutdown_connections 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.510+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1347566624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe266f0 0x5555fbe28ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.510+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1347566624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 0x5555fbe191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.510+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1347566624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe18820 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.510+0000 7fa68b97d740 1 -- 172.21.15.18:0/1347566624 >> 172.21.15.18:0/1347566624 conn(0x5555fbde0550 msgr2=0x5555fbe13a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.510+0000 7fa68b97d740 1 -- 172.21.15.18:0/1347566624 shutdown_connections 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa68b97d740 1 -- 172.21.15.18:0/1347566624 wait complete. 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa68b97d740 1 -- start start 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa68b97d740 1 Processor -- start 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa68b97d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 0x5555fbe236f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa68b97d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe23c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa68b97d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe266f0 0x5555fbe24170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa68b97d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555fbd25bf0 con 0x5555fbe18d60 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa68b97d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555fba54250 con 0x5555fbe19ad0 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa68b97d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555fbd06200 con 0x5555fbe266f0 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa682d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 0x5555fbe236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa68b97d740 10 client.0 _ll_register_callbacks cb 0x5555fbd74d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa682d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 0x5555fbe236f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60358/0 (socket says 172.21.15.18:60358) 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa682d9d640 1 -- 172.21.15.18:0/1134555750 learned_addr learned my addr 172.21.15.18:0/1134555750 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa681d9b640 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe23c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.511+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe266f0 0x5555fbe24170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa682d9d640 1 -- 172.21.15.18:0/1134555750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe266f0 msgr2=0x5555fbe24170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe266f0 0x5555fbe24170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa682d9d640 1 -- 172.21.15.18:0/1134555750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 msgr2=0x5555fbe23c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe23c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa682d9d640 1 -- 172.21.15.18:0/1134555750 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67c019040 con 0x5555fbe18d60 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 0x5555fbe236f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009a70 tx=0x7fa67c00fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9116f3012efe21a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa669ffb640 1 -- 172.21.15.18:0/1134555750 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00fb10 con 0x5555fbe18d60 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa669ffb640 1 -- 172.21.15.18:0/1134555750 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa67c00fc70 con 0x5555fbe18d60 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa68b97d740 1 -- 172.21.15.18:0/1134555750 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555fba80600 con 0x5555fbe18d60 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa669ffb640 1 -- 172.21.15.18:0/1134555750 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa67c00fe30 con 0x5555fbe18d60 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa68b97d740 10 client.16791 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa68b97d740 1 -- 172.21.15.18:0/1134555750 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5555fbd7ed80 con 0x5555fbe18d60 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa669ffb640 1 -- 172.21.15.18:0/1134555750 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5555fba80600 con 0x5555fbe18d60 2024-02-17T12:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa669ffb640 1 -- 172.21.15.18:0/1134555750 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa67c01e4b0 con 0x5555fbe18d60 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa68b97d740 10 client.16791 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa68b97d740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa68b97d740 20 client.16791 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555fbe36930 0x5555fbe38d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa68b97d740 4 client.16791 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.512+0000 7fa68b97d740 1 -- 172.21.15.18:0/1134555750 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555fbad11b0 con 0x5555fbe36930 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.513+0000 7fa681d9b640 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555fbe36930 0x5555fbe38d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.513+0000 7fa681d9b640 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555fbe36930 0x5555fbe38d50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9cb2beb6c37a477 server_cookie=4faa25b6187c60ec in_seq=0 out_seq=0 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.513+0000 7fa669ffb640 10 client.16791 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa669ffb640 1 -- 172.21.15.18:0/1134555750 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5555fbad11b0 con 0x5555fbe36930 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa669ffb640 10 client.16791 handle_command_reply: tid=0 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa68b97d740 1 client.16791 shutdown 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa68b97d740 1 -- 172.21.15.18:0/1134555750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 msgr2=0x5555fbe236f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 0x5555fbe236f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009a70 tx=0x7fa67c00fa10 comp rx=0 tx=0).stop 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa68b97d740 1 -- 172.21.15.18:0/1134555750 shutdown_connections 2024-02-17T12:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555fbe36930 0x5555fbe38d50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe266f0 0x5555fbe24170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa669ffb640 0 client.16791 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe23c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1134555750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe18d60 0x5555fbe236f0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa68b97d740 1 -- 172.21.15.18:0/1134555750 >> 172.21.15.18:0/1134555750 conn(0x5555fbde0550 msgr2=0x5555fbe157a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa68b97d740 1 -- 172.21.15.18:0/1134555750 shutdown_connections 2024-02-17T12:53:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa68b97d740 1 -- 172.21.15.18:0/1134555750 wait complete. 2024-02-17T12:53:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.514+0000 7fa68b97d740 20 client.16791 trim_cache size 0 max 16384 2024-02-17T12:53:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa68b97d740 1 Processor -- start 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa68b97d740 1 -- start start 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa68b97d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe18d60 0x5555fbe25e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa68b97d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe19ad0 0x5555fbe21fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa68b97d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 0x5555fbe22520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa68b97d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555fbe246c0 con 0x5555fbe266f0 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa68b97d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555fbe33540 con 0x5555fbe18d60 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa68b97d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555fbe36730 con 0x5555fbe19ad0 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa682d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 0x5555fbe22520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa682d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 0x5555fbe22520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60374/0 (socket says 172.21.15.18:60374) 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa682d9d640 1 -- 172.21.15.18:0/3110636646 learned_addr learned my addr 172.21.15.18:0/3110636646 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa681d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe19ad0 0x5555fbe21fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.528+0000 7fa682d9d640 1 --2- 172.21.15.18:0/3110636646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe18d60 0x5555fbe25e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.529+0000 7fa682d9d640 1 -- 172.21.15.18:0/3110636646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe19ad0 msgr2=0x5555fbe21fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.529+0000 7fa682d9d640 1 --2- 172.21.15.18:0/3110636646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe19ad0 0x5555fbe21fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.529+0000 7fa682d9d640 1 -- 172.21.15.18:0/3110636646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe18d60 msgr2=0x5555fbe25e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.529+0000 7fa682d9d640 1 --2- 172.21.15.18:0/3110636646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe18d60 0x5555fbe25e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.529+0000 7fa682d9d640 1 -- 172.21.15.18:0/3110636646 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555fbd1b7b0 con 0x5555fbe266f0 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.529+0000 7fa682d9d640 1 --2- 172.21.15.18:0/3110636646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 0x5555fbe22520 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00f740 tx=0x7fa67c011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27400189194d4c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.529+0000 7fa66bfff640 1 -- 172.21.15.18:0/3110636646 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00f7e0 con 0x5555fbe266f0 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.529+0000 7fa66bfff640 1 -- 172.21.15.18:0/3110636646 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa67c031070 con 0x5555fbe266f0 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.529+0000 7fa66bfff640 1 -- 172.21.15.18:0/3110636646 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c010500 con 0x5555fbe266f0 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 -- 172.21.15.18:0/3110636646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 msgr2=0x5555fbe22520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 --2- 172.21.15.18:0/3110636646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 0x5555fbe22520 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00f740 tx=0x7fa67c011f90 comp rx=0 tx=0).stop 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 -- 172.21.15.18:0/3110636646 shutdown_connections 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 --2- 172.21.15.18:0/3110636646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 0x5555fbe22520 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 --2- 172.21.15.18:0/3110636646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe19ad0 0x5555fbe21fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 --2- 172.21.15.18:0/3110636646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe18d60 0x5555fbe25e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 -- 172.21.15.18:0/3110636646 >> 172.21.15.18:0/3110636646 conn(0x5555fbde0550 msgr2=0x5555fbe1ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 -- 172.21.15.18:0/3110636646 shutdown_connections 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 -- 172.21.15.18:0/3110636646 wait complete. 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 -- start start 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 Processor -- start 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe18d60 0x5555fbe2d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe2bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 0x5555fbe2c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555fbe2c550 con 0x5555fbe266f0 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555fbe2c6c0 con 0x5555fbe19ad0 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555fbe2c880 con 0x5555fbe18d60 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa68b97d740 10 client.0 _ll_register_callbacks cb 0x7fa67c009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa682d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 0x5555fbe2c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa682d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 0x5555fbe2c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60376/0 (socket says 172.21.15.18:60376) 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa682d9d640 1 -- 172.21.15.18:0/1714610399 learned_addr learned my addr 172.21.15.18:0/1714610399 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe18d60 0x5555fbe2d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.530+0000 7fa681d9b640 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe2bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa682d9d640 1 -- 172.21.15.18:0/1714610399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe18d60 msgr2=0x5555fbe2d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe18d60 0x5555fbe2d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa682d9d640 1 -- 172.21.15.18:0/1714610399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 msgr2=0x5555fbe2bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe2bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa682d9d640 1 -- 172.21.15.18:0/1714610399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555fbd7ed80 con 0x5555fbe266f0 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa682d9d640 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 0x5555fbe2c010 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa67c01e5d0 tx=0x7fa67c02f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72515864abc0b80e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa669ffb640 1 -- 172.21.15.18:0/1714610399 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00f770 con 0x5555fbe266f0 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa669ffb640 1 -- 172.21.15.18:0/1714610399 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa67c00f8d0 con 0x5555fbe266f0 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa68b97d740 1 -- 172.21.15.18:0/1714610399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa67c017070 con 0x5555fbe266f0 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa669ffb640 1 -- 172.21.15.18:0/1714610399 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa67c00fa90 con 0x5555fbe266f0 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa68b97d740 10 client.16797 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa68b97d740 1 -- 172.21.15.18:0/1714610399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5555fbd41540 con 0x5555fbe266f0 2024-02-17T12:53:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa669ffb640 1 -- 172.21.15.18:0/1714610399 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa67c017070 con 0x5555fbe266f0 2024-02-17T12:53:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa669ffb640 1 -- 172.21.15.18:0/1714610399 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa67c024070 con 0x5555fbe266f0 2024-02-17T12:53:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa68b97d740 10 client.16797 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa68b97d740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa68b97d740 20 client.16797 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555fbe8dfc0 0x5555fbe903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa68b97d740 4 client.16797 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:53:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.531+0000 7fa68b97d740 1 -- 172.21.15.18:0/1714610399 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5555fbabd550 con 0x5555fbe8dfc0 2024-02-17T12:53:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.532+0000 7fa681d9b640 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555fbe8dfc0 0x5555fbe903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.532+0000 7fa681d9b640 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555fbe8dfc0 0x5555fbe903e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e179711151fa1b54 server_cookie=8b4b861369731690 in_seq=0 out_seq=0 2024-02-17T12:53:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.532+0000 7fa669ffb640 10 client.16797 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.583+0000 7fa669ffb640 1 -- 172.21.15.18:0/1714610399 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5555fbabd550 con 0x5555fbe8dfc0 2024-02-17T12:53:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.583+0000 7fa669ffb640 10 client.16797 handle_command_reply: tid=0 2024-02-17T12:53:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.583+0000 7fa68b97d740 1 client.16797 shutdown 2024-02-17T12:53:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.583+0000 7fa68b97d740 1 -- 172.21.15.18:0/1714610399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 msgr2=0x5555fbe2c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.583+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 0x5555fbe2c010 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa67c01e5d0 tx=0x7fa67c02f040 comp rx=0 tx=0).stop 2024-02-17T12:53:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.583+0000 7fa68b97d740 1 -- 172.21.15.18:0/1714610399 shutdown_connections 2024-02-17T12:53:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.583+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5555fbe8dfc0 0x5555fbe903e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.583+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555fbe266f0 0x5555fbe2c010 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.583+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555fbe19ad0 0x5555fbe2bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.583+0000 7fa669ffb640 0 client.16797 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.583+0000 7fa68b97d740 1 --2- 172.21.15.18:0/1714610399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555fbe18d60 0x5555fbe2d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.583+0000 7fa68b97d740 1 -- 172.21.15.18:0/1714610399 >> 172.21.15.18:0/1714610399 conn(0x5555fbde0550 msgr2=0x5555fbe1ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa68b97d740 1 -- 172.21.15.18:0/1714610399 shutdown_connections 2024-02-17T12:53:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa68b97d740 1 -- 172.21.15.18:0/1714610399 wait complete. 2024-02-17T12:53:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa68b97d740 20 client.16797 trim_cache size 0 max 16384 2024-02-17T12:53:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:53:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:53:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:53:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:53:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa688dac640 1 -- 172.21.15.18:0/3769219625 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa66c0a22d0 msgr2=0x7fa66c0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa688dac640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa66c0a22d0 0x7fa66c0a4790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa6740cb250 tx=0x7fa6740ba040 comp rx=0 tx=0).stop 2024-02-17T12:53:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa688dac640 1 -- 172.21.15.18:0/3769219625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6841013c0 msgr2=0x7fa68419d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa688dac640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6841013c0 0x7fa68419d3e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa67400d770 tx=0x7fa67400dc30 comp rx=0 tx=0).stop 2024-02-17T12:53:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa688dac640 1 -- 172.21.15.18:0/3769219625 shutdown_connections 2024-02-17T12:53:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa688dac640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa66c0a22d0 0x7fa66c0a4790 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa688dac640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa684197590 0x7fa684197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa688dac640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa684106490 0x7fa68419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa688dac640 1 --2- 172.21.15.18:0/3769219625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6841013c0 0x7fa68419d3e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa688dac640 1 -- 172.21.15.18:0/3769219625 >> 172.21.15.18:0/3769219625 conn(0x7fa6840fcc60 msgr2=0x7fa6840fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa688dac640 1 -- 172.21.15.18:0/3769219625 shutdown_connections 2024-02-17T12:53:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:30.584+0000 7fa688dac640 1 -- 172.21.15.18:0/3769219625 wait complete. 2024-02-17T12:53:30.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:30 smithi146 ceph-mon[36454]: pgmap v111: 97 pgs: 97 active+clean; 80 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 441 op/s 2024-02-17T12:53:30.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:30 smithi018 ceph-mon[32625]: pgmap v111: 97 pgs: 97 active+clean; 80 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 441 op/s 2024-02-17T12:53:30.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:53:30.785 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:53:30.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:30 smithi119 ceph-mon[36188]: pgmap v111: 97 pgs: 97 active+clean; 80 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 441 op/s 2024-02-17T12:53:30.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.906+0000 7fcc47a76640 1 -- 172.21.15.18:0/1866440456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc40105340 msgr2=0x7fcc4010cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.906+0000 7fcc47a76640 1 --2- 172.21.15.18:0/1866440456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc40105340 0x7fcc4010cf10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009cd0 tx=0x7fcc3802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.906+0000 7fcc47a76640 1 -- 172.21.15.18:0/1866440456 shutdown_connections 2024-02-17T12:53:30.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.906+0000 7fcc47a76640 1 --2- 172.21.15.18:0/1866440456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc40105340 0x7fcc4010cf10 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.906+0000 7fcc47a76640 1 --2- 172.21.15.18:0/1866440456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc40100c20 0x7fcc40104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.906+0000 7fcc47a76640 1 --2- 172.21.15.18:0/1866440456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc40100270 0x7fcc40100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.906+0000 7fcc47a76640 1 -- 172.21.15.18:0/1866440456 >> 172.21.15.18:0/1866440456 conn(0x7fcc40077230 msgr2=0x7fcc40077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:30.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.906+0000 7fcc47a76640 1 -- 172.21.15.18:0/1866440456 shutdown_connections 2024-02-17T12:53:30.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.906+0000 7fcc47a76640 1 -- 172.21.15.18:0/1866440456 wait complete. 2024-02-17T12:53:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.906+0000 7fcc47a76640 1 Processor -- start 2024-02-17T12:53:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.906+0000 7fcc47a76640 1 -- start start 2024-02-17T12:53:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc47a76640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc40100270 0x7fcc40101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc47a76640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc40105340 0x7fcc40102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc47a76640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc40102780 0x7fcc401a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc47a76640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc401a5ec0 con 0x7fcc40105340 2024-02-17T12:53:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc47a76640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc401a6030 con 0x7fcc40102780 2024-02-17T12:53:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc47a76640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc401a61f0 con 0x7fcc40100270 2024-02-17T12:53:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc45011640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc40105340 0x7fcc40102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc45812640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc40100270 0x7fcc40101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc45011640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc40105340 0x7fcc40102240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60388/0 (socket says 172.21.15.18:60388) 2024-02-17T12:53:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc45011640 1 -- 172.21.15.18:0/1770573395 learned_addr learned my addr 172.21.15.18:0/1770573395 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc46013640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc40102780 0x7fcc401a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc45011640 1 -- 172.21.15.18:0/1770573395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc40100270 msgr2=0x7fcc40101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc45011640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc40100270 0x7fcc40101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc45011640 1 -- 172.21.15.18:0/1770573395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc40102780 msgr2=0x7fcc401a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc45011640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc40102780 0x7fcc401a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc45011640 1 -- 172.21.15.18:0/1770573395 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc38009900 con 0x7fcc40105340 2024-02-17T12:53:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc45011640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc40105340 0x7fcc40102240 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000d770 tx=0x7fcc3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc36ffd640 1 -- 172.21.15.18:0/1770573395 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc30009c30 con 0x7fcc40105340 2024-02-17T12:53:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc36ffd640 1 -- 172.21.15.18:0/1770573395 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc3000de80 con 0x7fcc40105340 2024-02-17T12:53:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.907+0000 7fcc47a76640 1 -- 172.21.15.18:0/1770573395 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc401a64d0 con 0x7fcc40105340 2024-02-17T12:53:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.908+0000 7fcc36ffd640 1 -- 172.21.15.18:0/1770573395 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3000f6b0 con 0x7fcc40105340 2024-02-17T12:53:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.908+0000 7fcc47a76640 1 -- 172.21.15.18:0/1770573395 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc401a6a30 con 0x7fcc40105340 2024-02-17T12:53:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.910+0000 7fcc4a64a740 1 Processor -- start 2024-02-17T12:53:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.910+0000 7fcc36ffd640 1 -- 172.21.15.18:0/1770573395 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcc30010460 con 0x7fcc40105340 2024-02-17T12:53:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.910+0000 7fcc4a64a740 1 -- start start 2024-02-17T12:53:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.910+0000 7fcc4a64a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 0x562f1318a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.910+0000 7fcc4a64a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318abd0 0x562f13193270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.910+0000 7fcc4a64a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f13195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.910+0000 7fcc4a64a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f130f9280 con 0x562f1318a2b0 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.910+0000 7fcc4a64a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f130f67b0 con 0x562f131937b0 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.910+0000 7fcc4a64a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f130f1f90 con 0x562f1318abd0 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc46013640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 0x562f1318a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc46013640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 0x562f1318a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60390/0 (socket says 172.21.15.18:60390) 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc45011640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f13195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc46013640 1 -- 172.21.15.18:0/3363648158 learned_addr learned my addr 172.21.15.18:0/3363648158 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc45812640 1 --2- 172.21.15.18:0/3363648158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318abd0 0x562f13193270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc46013640 1 -- 172.21.15.18:0/3363648158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318abd0 msgr2=0x562f13193270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc46013640 1 --2- 172.21.15.18:0/3363648158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318abd0 0x562f13193270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc46013640 1 -- 172.21.15.18:0/3363648158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 msgr2=0x562f13195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc46013640 1 --2- 172.21.15.18:0/3363648158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f13195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc36ffd640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc2c0a2360 0x7fcc2c0a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc46013640 1 -- 172.21.15.18:0/3363648158 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f130b0570 con 0x562f1318a2b0 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc45011640 1 --2- 172.21.15.18:0/3363648158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f13195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc36ffd640 1 -- 172.21.15.18:0/1770573395 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcc300cdc80 con 0x7fcc40105340 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc46013640 1 --2- 172.21.15.18:0/3363648158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 0x562f1318a690 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fcc3802c700 tx=0x7fcc38032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b8e4a79a7425d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc46013640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc2c0a2360 0x7fcc2c0a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.911+0000 7fcc2bfff640 1 -- 172.21.15.18:0/3363648158 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38034640 con 0x562f1318a2b0 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.912+0000 7fcc2bfff640 1 -- 172.21.15.18:0/3363648158 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc38003970 con 0x562f1318a2b0 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.912+0000 7fcc2bfff640 1 -- 172.21.15.18:0/3363648158 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38004400 con 0x562f1318a2b0 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.912+0000 7fcc4a64a740 1 -- 172.21.15.18:0/3363648158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 msgr2=0x562f1318a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.912+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/3363648158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 0x562f1318a690 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fcc3802c700 tx=0x7fcc38032d70 comp rx=0 tx=0).stop 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.912+0000 7fcc46013640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc2c0a2360 0x7fcc2c0a4820 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcc3804b000 tx=0x7fcc3804e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.912+0000 7fcc4a64a740 1 -- 172.21.15.18:0/3363648158 shutdown_connections 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.912+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/3363648158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f13195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.912+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/3363648158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318abd0 0x562f13193270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.912+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/3363648158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 0x562f1318a690 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.912+0000 7fcc4a64a740 1 -- 172.21.15.18:0/3363648158 >> 172.21.15.18:0/3363648158 conn(0x562f13184990 msgr2=0x562f13184d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.912+0000 7fcc4a64a740 1 -- 172.21.15.18:0/3363648158 shutdown_connections 2024-02-17T12:53:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.912+0000 7fcc4a64a740 1 -- 172.21.15.18:0/3363648158 wait complete. 2024-02-17T12:53:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.913+0000 7fcc4a64a740 1 -- start start 2024-02-17T12:53:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.913+0000 7fcc4a64a740 1 Processor -- start 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.913+0000 7fcc4a64a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f1318a2b0 0x562f13193220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.913+0000 7fcc4a64a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 0x562f131905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.913+0000 7fcc4a64a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f131937b0 0x562f13190cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.913+0000 7fcc4a64a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f12dc5430 con 0x562f1318abd0 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.913+0000 7fcc4a64a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f12d81e00 con 0x562f1318a2b0 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.913+0000 7fcc4a64a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f13076fd0 con 0x562f131937b0 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc4a64a740 10 client.0 _ll_register_callbacks cb 0x562f130ccfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc45011640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 0x562f131905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc45011640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 0x562f131905b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60396/0 (socket says 172.21.15.18:60396) 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc45011640 1 -- 172.21.15.18:0/1428311534 learned_addr learned my addr 172.21.15.18:0/1428311534 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc45812640 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f1318a2b0 0x562f13193220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc46013640 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f131937b0 0x562f13190cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc45812640 1 -- 172.21.15.18:0/1428311534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f131937b0 msgr2=0x562f13190cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc45812640 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f131937b0 0x562f13190cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc45812640 1 -- 172.21.15.18:0/1428311534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 msgr2=0x562f131905b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc45812640 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 0x562f131905b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc45812640 1 -- 172.21.15.18:0/1428311534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4010d720 con 0x562f1318a2b0 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc45011640 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 0x562f131905b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc45812640 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f1318a2b0 0x562f13193220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0126d0 tx=0x7fcc3c012ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f52150ade7ee29e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc297fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc297fa640 1 -- 172.21.15.18:0/1428311534 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c017070 con 0x562f1318a2b0 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc297fa640 1 -- 172.21.15.18:0/1428311534 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcc3c004050 con 0x562f1318a2b0 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.914+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1428311534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f12df17e0 con 0x562f1318a2b0 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.915+0000 7fcc297fa640 1 -- 172.21.15.18:0/1428311534 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc3c004210 con 0x562f1318a2b0 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.915+0000 7fcc4a64a740 10 client.26177 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.915+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1428311534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562f130f0610 con 0x562f1318a2b0 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.915+0000 7fcc297fa640 1 -- 172.21.15.18:0/1428311534 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562f12df17e0 con 0x562f1318a2b0 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.915+0000 7fcc297fa640 1 -- 172.21.15.18:0/1428311534 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fcc3c00f570 con 0x562f1318a2b0 2024-02-17T12:53:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.915+0000 7fcc4a64a740 10 client.26177 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.915+0000 7fcc4a64a740 10 client.26177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.915+0000 7fcc4a64a740 20 client.26177 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.915+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562f131b49e0 0x562f131b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.915+0000 7fcc4a64a740 4 client.26177 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.915+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1428311534 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f12e2e720 con 0x562f131b49e0 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.915+0000 7fcc45011640 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562f131b49e0 0x562f131b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.916+0000 7fcc297fa640 10 client.26177 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.916+0000 7fcc45011640 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562f131b49e0 0x562f131b6e00 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acec5928edf47f6b server_cookie=74cda420fe701498 in_seq=0 out_seq=0 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc297fa640 1 -- 172.21.15.18:0/1428311534 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f12e2e720 con 0x562f131b49e0 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc297fa640 10 client.26177 handle_command_reply: tid=0 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc4a64a740 1 client.26177 shutdown 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1428311534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f1318a2b0 msgr2=0x562f13193220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f1318a2b0 0x562f13193220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0126d0 tx=0x7fcc3c012ad0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1428311534 shutdown_connections 2024-02-17T12:53:30.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562f131b49e0 0x562f131b6e00 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f131937b0 0x562f13190cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 0x562f131905b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/1428311534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f1318a2b0 0x562f13193220 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc297fa640 0 client.26177 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:30.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1428311534 >> 172.21.15.18:0/1428311534 conn(0x562f13184990 msgr2=0x562f13186730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:30.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1428311534 shutdown_connections 2024-02-17T12:53:30.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1428311534 wait complete. 2024-02-17T12:53:30.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.917+0000 7fcc4a64a740 20 client.26177 trim_cache size 0 max 16384 2024-02-17T12:53:30.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc4a64a740 1 Processor -- start 2024-02-17T12:53:30.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc4a64a740 1 -- start start 2024-02-17T12:53:30.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc4a64a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318a2b0 0x562f13192ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc4a64a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 0x562f13193220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc4a64a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f1318fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc4a64a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f13189570 con 0x562f1318abd0 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc4a64a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f131b4760 con 0x562f131937b0 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc4a64a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f13189f10 con 0x562f1318a2b0 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc45011640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 0x562f13193220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc45011640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 0x562f13193220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60402/0 (socket says 172.21.15.18:60402) 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc45011640 1 -- 172.21.15.18:0/2292617339 learned_addr learned my addr 172.21.15.18:0/2292617339 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc45812640 1 --2- 172.21.15.18:0/2292617339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318a2b0 0x562f13192ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.934+0000 7fcc46013640 1 --2- 172.21.15.18:0/2292617339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f1318fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.935+0000 7fcc45011640 1 -- 172.21.15.18:0/2292617339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318a2b0 msgr2=0x562f13192ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.935+0000 7fcc45011640 1 --2- 172.21.15.18:0/2292617339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318a2b0 0x562f13192ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.935+0000 7fcc45011640 1 -- 172.21.15.18:0/2292617339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 msgr2=0x562f1318fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.935+0000 7fcc45011640 1 --2- 172.21.15.18:0/2292617339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f1318fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.935+0000 7fcc45011640 1 -- 172.21.15.18:0/2292617339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f1308cac0 con 0x562f1318abd0 2024-02-17T12:53:30.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.935+0000 7fcc45812640 1 --2- 172.21.15.18:0/2292617339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318a2b0 0x562f13192ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:30.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.935+0000 7fcc45011640 1 --2- 172.21.15.18:0/2292617339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 0x562f13193220 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcc30096b70 tx=0x7fcc300ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8580f34a93c7c785 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:30.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.935+0000 7fcc2b7fe640 1 -- 172.21.15.18:0/2292617339 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3009a4e0 con 0x562f1318abd0 2024-02-17T12:53:30.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.935+0000 7fcc2b7fe640 1 -- 172.21.15.18:0/2292617339 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc3009a250 con 0x562f1318abd0 2024-02-17T12:53:30.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.935+0000 7fcc2b7fe640 1 -- 172.21.15.18:0/2292617339 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc30098a10 con 0x562f1318abd0 2024-02-17T12:53:30.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.936+0000 7fcc4a64a740 1 -- 172.21.15.18:0/2292617339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 msgr2=0x562f13193220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.936+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/2292617339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 0x562f13193220 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcc30096b70 tx=0x7fcc300ba040 comp rx=0 tx=0).stop 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.936+0000 7fcc4a64a740 1 -- 172.21.15.18:0/2292617339 shutdown_connections 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.936+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/2292617339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f1318fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.936+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/2292617339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318abd0 0x562f13193220 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.936+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/2292617339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318a2b0 0x562f13192ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.936+0000 7fcc4a64a740 1 -- 172.21.15.18:0/2292617339 >> 172.21.15.18:0/2292617339 conn(0x562f13184990 msgr2=0x562f1318b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.936+0000 7fcc4a64a740 1 -- 172.21.15.18:0/2292617339 shutdown_connections 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.936+0000 7fcc4a64a740 1 -- 172.21.15.18:0/2292617339 wait complete. 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.936+0000 7fcc4a64a740 1 -- start start 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.936+0000 7fcc4a64a740 1 Processor -- start 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc4a64a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 0x562f1319a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc4a64a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318abd0 0x562f13196c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc4a64a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f13197170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc4a64a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f131976b0 con 0x562f1318a2b0 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc4a64a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f13197820 con 0x562f131937b0 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc4a64a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f131979e0 con 0x562f1318abd0 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc4a64a740 10 client.0 _ll_register_callbacks cb 0x7fcc3c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc45812640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 0x562f1319a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc45812640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 0x562f1319a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60406/0 (socket says 172.21.15.18:60406) 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc45812640 1 -- 172.21.15.18:0/1655570061 learned_addr learned my addr 172.21.15.18:0/1655570061 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc46013640 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f13197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc45011640 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318abd0 0x562f13196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc45812640 1 -- 172.21.15.18:0/1655570061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318abd0 msgr2=0x562f13196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc45812640 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318abd0 0x562f13196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc45812640 1 -- 172.21.15.18:0/1655570061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 msgr2=0x562f13197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc45812640 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f13197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc45812640 1 -- 172.21.15.18:0/1655570061 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f130f0610 con 0x562f1318a2b0 2024-02-17T12:53:30.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc46013640 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f13197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc45812640 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 0x562f1319a810 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00f980 tx=0x7fcc3c009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136fad75440b9a0e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc297fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc297fa640 1 -- 172.21.15.18:0/1655570061 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c017040 con 0x562f1318a2b0 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc297fa640 1 -- 172.21.15.18:0/1655570061 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcc3c000e40 con 0x562f1318a2b0 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.937+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1655570061 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc38003970 con 0x562f1318a2b0 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.938+0000 7fcc297fa640 1 -- 172.21.15.18:0/1655570061 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc3c0052b0 con 0x562f1318a2b0 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.938+0000 7fcc4a64a740 10 client.16815 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.938+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1655570061 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fcc401a64d0 con 0x562f1318a2b0 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.938+0000 7fcc297fa640 1 -- 172.21.15.18:0/1655570061 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc38003970 con 0x562f1318a2b0 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.938+0000 7fcc297fa640 1 -- 172.21.15.18:0/1655570061 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fcc3c026480 con 0x562f1318a2b0 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.938+0000 7fcc4a64a740 10 client.16815 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.938+0000 7fcc4a64a740 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.938+0000 7fcc4a64a740 20 client.16815 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.938+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562f1320c510 0x562f1320e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.938+0000 7fcc4a64a740 4 client.16815 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.938+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1655570061 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562f12e7d890 con 0x562f1320c510 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.938+0000 7fcc45011640 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562f1320c510 0x562f1320e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc297fa640 10 client.16815 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc45011640 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562f1320c510 0x562f1320e930 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e6abfaf6afde0a server_cookie=2a3a52ec8d4bac7a in_seq=0 out_seq=0 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc297fa640 1 -- 172.21.15.18:0/1655570061 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562f12e7d890 con 0x562f1320c510 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc297fa640 10 client.16815 handle_command_reply: tid=0 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc4a64a740 1 client.16815 shutdown 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1655570061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 msgr2=0x562f1319a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 0x562f1319a810 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00f980 tx=0x7fcc3c009eb0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1655570061 shutdown_connections 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562f1320c510 0x562f1320e930 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f131937b0 0x562f13197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562f1318abd0 0x562f13196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc4a64a740 1 --2- 172.21.15.18:0/1655570061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f1318a2b0 0x562f1319a810 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc297fa640 0 client.16815 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1655570061 >> 172.21.15.18:0/1655570061 conn(0x562f13184990 msgr2=0x562f13185cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1655570061 shutdown_connections 2024-02-17T12:53:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc4a64a740 1 -- 172.21.15.18:0/1655570061 wait complete. 2024-02-17T12:53:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc4a64a740 20 client.16815 trim_cache size 0 max 16384 2024-02-17T12:53:30.937 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:53:30.937 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:53:30.937 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:53:30.937 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:53:30.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.939+0000 7fcc47a76640 1 -- 172.21.15.18:0/1770573395 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc2c0a2360 msgr2=0x7fcc2c0a4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.940+0000 7fcc47a76640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc2c0a2360 0x7fcc2c0a4820 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcc3804b000 tx=0x7fcc3804e000 comp rx=0 tx=0).stop 2024-02-17T12:53:30.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.940+0000 7fcc47a76640 1 -- 172.21.15.18:0/1770573395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc40105340 msgr2=0x7fcc40102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:30.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.940+0000 7fcc47a76640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc40105340 0x7fcc40102240 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000d770 tx=0x7fcc3000dc30 comp rx=0 tx=0).stop 2024-02-17T12:53:30.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.940+0000 7fcc47a76640 1 -- 172.21.15.18:0/1770573395 shutdown_connections 2024-02-17T12:53:30.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.940+0000 7fcc47a76640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc2c0a2360 0x7fcc2c0a4820 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.940+0000 7fcc47a76640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc40102780 0x7fcc401a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.940+0000 7fcc47a76640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc40105340 0x7fcc40102240 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.940+0000 7fcc47a76640 1 --2- 172.21.15.18:0/1770573395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc40100270 0x7fcc40101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:30.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.940+0000 7fcc47a76640 1 -- 172.21.15.18:0/1770573395 >> 172.21.15.18:0/1770573395 conn(0x7fcc40077230 msgr2=0x7fcc40068630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:30.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.940+0000 7fcc47a76640 1 -- 172.21.15.18:0/1770573395 shutdown_connections 2024-02-17T12:53:30.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:30.940+0000 7fcc47a76640 1 -- 172.21.15.18:0/1770573395 wait complete. 2024-02-17T12:53:30.942 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:53:30.942 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:53:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.057+0000 7f7853d6b640 1 -- 172.21.15.18:0/378797667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f784c102380 msgr2=0x7f784c102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.057+0000 7f7853d6b640 1 --2- 172.21.15.18:0/378797667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f784c102380 0x7f784c102760 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7848009900 tx=0x7f784802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.057+0000 7f7853d6b640 1 -- 172.21.15.18:0/378797667 shutdown_connections 2024-02-17T12:53:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.057+0000 7f7853d6b640 1 --2- 172.21.15.18:0/378797667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f784c10b750 0x7f784c10dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.057+0000 7f7853d6b640 1 --2- 172.21.15.18:0/378797667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f784c102ca0 0x7f784c10b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.057+0000 7f7853d6b640 1 --2- 172.21.15.18:0/378797667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f784c102380 0x7f784c102760 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.057+0000 7f7853d6b640 1 -- 172.21.15.18:0/378797667 >> 172.21.15.18:0/378797667 conn(0x7f784c076e30 msgr2=0x7f784c0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.057+0000 7f7853d6b640 1 -- 172.21.15.18:0/378797667 shutdown_connections 2024-02-17T12:53:31.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.057+0000 7f7853d6b640 1 -- 172.21.15.18:0/378797667 wait complete. 2024-02-17T12:53:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7853d6b640 1 Processor -- start 2024-02-17T12:53:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7853d6b640 1 -- start start 2024-02-17T12:53:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7853d6b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f784c102380 0x7f784c19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7853d6b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f784c102ca0 0x7f784c19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7853d6b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f784c10b750 0x7f784c1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7853d6b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c197bf0 con 0x7f784c102380 2024-02-17T12:53:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7853d6b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c197d60 con 0x7f784c102ca0 2024-02-17T12:53:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7853d6b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c197f20 con 0x7f784c10b750 2024-02-17T12:53:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7851b07640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f784c102380 0x7f784c19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7851b07640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f784c102380 0x7f784c19d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60414/0 (socket says 172.21.15.18:60414) 2024-02-17T12:53:31.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7851b07640 1 -- 172.21.15.18:0/1474526647 learned_addr learned my addr 172.21.15.18:0/1474526647 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7852308640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f784c10b750 0x7f784c1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.058+0000 7f7851306640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f784c102ca0 0x7f784c19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.059+0000 7f7851b07640 1 -- 172.21.15.18:0/1474526647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f784c10b750 msgr2=0x7f784c1975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.059+0000 7f7851b07640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f784c10b750 0x7f784c1975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.059+0000 7f7851b07640 1 -- 172.21.15.18:0/1474526647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f784c102ca0 msgr2=0x7f784c19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.059+0000 7f7851b07640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f784c102ca0 0x7f784c19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.059+0000 7f7851b07640 1 -- 172.21.15.18:0/1474526647 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7848009560 con 0x7f784c102380 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.059+0000 7f7851b07640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f784c102380 0x7f784c19d3f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7848005940 tx=0x7f784802c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.059+0000 7f7842ffd640 1 -- 172.21.15.18:0/1474526647 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848004350 con 0x7f784c102380 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.059+0000 7f7842ffd640 1 -- 172.21.15.18:0/1474526647 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78480044b0 con 0x7f784c102380 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.059+0000 7f7853d6b640 1 -- 172.21.15.18:0/1474526647 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784c1981a0 con 0x7f784c102380 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.059+0000 7f7842ffd640 1 -- 172.21.15.18:0/1474526647 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848003d30 con 0x7f784c102380 2024-02-17T12:53:31.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.059+0000 7f7853d6b640 1 -- 172.21.15.18:0/1474526647 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784c1a4490 con 0x7f784c102380 2024-02-17T12:53:31.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.061+0000 7f7853d6b640 1 -- 172.21.15.18:0/1474526647 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7830005290 con 0x7f784c102380 2024-02-17T12:53:31.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.062+0000 7f7842ffd640 1 -- 172.21.15.18:0/1474526647 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7848004df0 con 0x7f784c102380 2024-02-17T12:53:31.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.062+0000 7f7842ffd640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f78380a22b0 0x7f78380a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.062+0000 7f7842ffd640 1 -- 172.21.15.18:0/1474526647 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f78480f0aa0 con 0x7f784c102380 2024-02-17T12:53:31.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.063+0000 7f7851306640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f78380a22b0 0x7f78380a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.063+0000 7f7851306640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f78380a22b0 0x7f78380a4770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f784c0ff6b0 tx=0x7f783c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.066+0000 7f7842ffd640 1 -- 172.21.15.18:0/1474526647 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f78480b9890 con 0x7f784c102380 2024-02-17T12:53:31.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 199118 10.95 MB/sec execute 46 sec latency 78.239 ms 2024-02-17T12:53:31.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.301+0000 7f7853d6b640 1 -- 172.21.15.18:0/1474526647 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78300058f0 con 0x7f784c102380 2024-02-17T12:53:31.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.302+0000 7f7842ffd640 1 -- 172.21.15.18:0/1474526647 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11396 (secure 0 0 0) 0x7f78300058f0 con 0x7f784c102380 2024-02-17T12:53:31.294 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:53:31.294 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:53:29.937758+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:53:31.294 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-02-17T12:53:31.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.305+0000 7f7853d6b640 1 -- 172.21.15.18:0/1474526647 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f78380a22b0 msgr2=0x7f78380a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.305+0000 7f7853d6b640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f78380a22b0 0x7f78380a4770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f784c0ff6b0 tx=0x7f783c005f90 comp rx=0 tx=0).stop 2024-02-17T12:53:31.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.305+0000 7f7853d6b640 1 -- 172.21.15.18:0/1474526647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f784c102380 msgr2=0x7f784c19d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.305+0000 7f7853d6b640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f784c102380 0x7f784c19d3f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7848005940 tx=0x7f784802c710 comp rx=0 tx=0).stop 2024-02-17T12:53:31.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.305+0000 7f7853d6b640 1 -- 172.21.15.18:0/1474526647 shutdown_connections 2024-02-17T12:53:31.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.305+0000 7f7853d6b640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f78380a22b0 0x7f78380a4770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.305+0000 7f7853d6b640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f784c10b750 0x7f784c1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.305+0000 7f7853d6b640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f784c102ca0 0x7f784c19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.305+0000 7f7853d6b640 1 --2- 172.21.15.18:0/1474526647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f784c102380 0x7f784c19d3f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.305+0000 7f7853d6b640 1 -- 172.21.15.18:0/1474526647 >> 172.21.15.18:0/1474526647 conn(0x7f784c076e30 msgr2=0x7f784c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.305+0000 7f7853d6b640 1 -- 172.21.15.18:0/1474526647 shutdown_connections 2024-02-17T12:53:31.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.305+0000 7f7853d6b640 1 -- 172.21.15.18:0/1474526647 wait complete. 2024-02-17T12:53:31.310 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:53:31.310 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:53:31.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.428+0000 7fbf599bc640 1 -- 172.21.15.18:0/3028805555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf54100260 msgr2=0x7fbf54100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.428+0000 7fbf599bc640 1 --2- 172.21.15.18:0/3028805555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf54100260 0x7fbf54100640 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009930 tx=0x7fbf4402bc30 comp rx=0 tx=0).stop 2024-02-17T12:53:31.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.428+0000 7fbf599bc640 1 -- 172.21.15.18:0/3028805555 shutdown_connections 2024-02-17T12:53:31.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.428+0000 7fbf599bc640 1 --2- 172.21.15.18:0/3028805555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54105330 0x7fbf5410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.428+0000 7fbf599bc640 1 --2- 172.21.15.18:0/3028805555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf54100c10 0x7fbf54104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.428+0000 7fbf599bc640 1 --2- 172.21.15.18:0/3028805555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf54100260 0x7fbf54100640 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.428+0000 7fbf599bc640 1 -- 172.21.15.18:0/3028805555 >> 172.21.15.18:0/3028805555 conn(0x7fbf54077180 msgr2=0x7fbf54077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.428+0000 7fbf599bc640 1 -- 172.21.15.18:0/3028805555 shutdown_connections 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.428+0000 7fbf599bc640 1 -- 172.21.15.18:0/3028805555 wait complete. 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.428+0000 7fbf599bc640 1 Processor -- start 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf599bc640 1 -- start start 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf599bc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf54100260 0x7fbf54198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf599bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54105330 0x7fbf54198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf599bc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf54199130 0x7fbf541a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf599bc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf541a3d80 con 0x7fbf54199130 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf599bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf541a3ef0 con 0x7fbf54105330 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf599bc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf541a40b0 con 0x7fbf54100260 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf537fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf54199130 0x7fbf541a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf527fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54105330 0x7fbf54198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf537fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf54199130 0x7fbf541a36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60434/0 (socket says 172.21.15.18:60434) 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf537fe640 1 -- 172.21.15.18:0/4166596938 learned_addr learned my addr 172.21.15.18:0/4166596938 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf52ffd640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf54100260 0x7fbf54198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf537fe640 1 -- 172.21.15.18:0/4166596938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf54100260 msgr2=0x7fbf54198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf537fe640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf54100260 0x7fbf54198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf537fe640 1 -- 172.21.15.18:0/4166596938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54105330 msgr2=0x7fbf54198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf537fe640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54105330 0x7fbf54198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf537fe640 1 -- 172.21.15.18:0/4166596938 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c009840 con 0x7fbf54199130 2024-02-17T12:53:31.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf537fe640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf54199130 0x7fbf541a36e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00efd0 tx=0x7fbf4c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf589ba640 1 -- 172.21.15.18:0/4166596938 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c009b40 con 0x7fbf54199130 2024-02-17T12:53:31.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf589ba640 1 -- 172.21.15.18:0/4166596938 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf4c003b70 con 0x7fbf54199130 2024-02-17T12:53:31.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.429+0000 7fbf599bc640 1 -- 172.21.15.18:0/4166596938 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf44009560 con 0x7fbf54199130 2024-02-17T12:53:31.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.430+0000 7fbf589ba640 1 -- 172.21.15.18:0/4166596938 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0108f0 con 0x7fbf54199130 2024-02-17T12:53:31.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.430+0000 7fbf599bc640 1 -- 172.21.15.18:0/4166596938 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf541a46c0 con 0x7fbf54199130 2024-02-17T12:53:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf5c594740 1 Processor -- start 2024-02-17T12:53:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf589ba640 1 -- 172.21.15.18:0/4166596938 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fbf4c003ce0 con 0x7fbf54199130 2024-02-17T12:53:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf5c594740 1 -- start start 2024-02-17T12:53:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf5c594740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 0x55d761d8c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf5c594740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d8cb70 0x55d761d95210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf5c594740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d95750 0x55d761d97b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf5c594740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761d47960 con 0x55d761d8c250 2024-02-17T12:53:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf5c594740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761cfb310 con 0x55d761d8cb70 2024-02-17T12:53:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf5c594740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761cf8840 con 0x55d761d95750 2024-02-17T12:53:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf527fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 0x55d761d8c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf52ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d8cb70 0x55d761d95210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf527fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 0x55d761d8c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60436/0 (socket says 172.21.15.18:60436) 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.432+0000 7fbf527fc640 1 -- 172.21.15.18:0/1803360497 learned_addr learned my addr 172.21.15.18:0/1803360497 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf527fc640 1 --2- 172.21.15.18:0/1803360497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d95750 0x55d761d97b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf589ba640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fbf3c0a2230 0x7fbf3c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf52ffd640 1 -- 172.21.15.18:0/1803360497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d95750 msgr2=0x55d761d97b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf52ffd640 1 --2- 172.21.15.18:0/1803360497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d95750 0x55d761d97b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf537fe640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fbf3c0a2230 0x7fbf3c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf52ffd640 1 -- 172.21.15.18:0/1803360497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 msgr2=0x55d761d8c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf52ffd640 1 --2- 172.21.15.18:0/1803360497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 0x55d761d8c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf52ffd640 1 -- 172.21.15.18:0/1803360497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d761cb6060 con 0x55d761d8cb70 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf527fc640 1 --2- 172.21.15.18:0/1803360497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 0x55d761d8c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf589ba640 1 -- 172.21.15.18:0/4166596938 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbf4c014070 con 0x7fbf54199130 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf52ffd640 1 --2- 172.21.15.18:0/1803360497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d8cb70 0x55d761d95210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf44036040 tx=0x7fbf4402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3076dd54d59f2e95 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf537fe640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fbf3c0a2230 0x7fbf3c0a46f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c097980 tx=0x7fbf4c06a150 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf39ffb640 1 -- 172.21.15.18:0/1803360497 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44004880 con 0x55d761d8cb70 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf39ffb640 1 -- 172.21.15.18:0/1803360497 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf440049e0 con 0x55d761d8cb70 2024-02-17T12:53:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.433+0000 7fbf39ffb640 1 -- 172.21.15.18:0/1803360497 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44005010 con 0x55d761d8cb70 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.434+0000 7fbf5c594740 1 -- 172.21.15.18:0/1803360497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d8cb70 msgr2=0x55d761d95210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.434+0000 7fbf5c594740 1 --2- 172.21.15.18:0/1803360497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d8cb70 0x55d761d95210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf44036040 tx=0x7fbf4402c650 comp rx=0 tx=0).stop 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.434+0000 7fbf5c594740 1 -- 172.21.15.18:0/1803360497 shutdown_connections 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.434+0000 7fbf5c594740 1 --2- 172.21.15.18:0/1803360497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d95750 0x55d761d97b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.434+0000 7fbf5c594740 1 --2- 172.21.15.18:0/1803360497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d8cb70 0x55d761d95210 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.434+0000 7fbf5c594740 1 --2- 172.21.15.18:0/1803360497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 0x55d761d8c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.434+0000 7fbf5c594740 1 -- 172.21.15.18:0/1803360497 >> 172.21.15.18:0/1803360497 conn(0x55d761d86980 msgr2=0x55d761d86d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.434+0000 7fbf5c594740 1 -- 172.21.15.18:0/1803360497 shutdown_connections 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.434+0000 7fbf5c594740 1 -- 172.21.15.18:0/1803360497 wait complete. 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.434+0000 7fbf5c594740 1 -- start start 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf5c594740 1 Processor -- start 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf5c594740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 0x55d761d92780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf5c594740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8cb70 0x55d761d92cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf5c594740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d95750 0x55d761d93200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf5c594740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761cf4020 con 0x55d761d8c250 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf5c594740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7619c7430 con 0x55d761d95750 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf5c594740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761983e00 con 0x55d761d8cb70 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf5c594740 10 client.0 _ll_register_callbacks cb 0x55d761c9bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf527fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 0x55d761d92780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf527fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 0x55d761d92780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60452/0 (socket says 172.21.15.18:60452) 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf527fc640 1 -- 172.21.15.18:0/172320531 learned_addr learned my addr 172.21.15.18:0/172320531 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf52ffd640 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8cb70 0x55d761d92cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf527fc640 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d95750 0x55d761d93200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf527fc640 1 -- 172.21.15.18:0/172320531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8cb70 msgr2=0x55d761d92cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf527fc640 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8cb70 0x55d761d92cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf527fc640 1 -- 172.21.15.18:0/172320531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d95750 msgr2=0x55d761d93200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf527fc640 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d95750 0x55d761d93200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.435+0000 7fbf527fc640 1 -- 172.21.15.18:0/172320531 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5410d710 con 0x55d761d8c250 2024-02-17T12:53:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf527fc640 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 0x55d761d92780 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbf480132d0 tx=0x7fbf48005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187ae26875614678 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf32ffd640 1 -- 172.21.15.18:0/172320531 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48004230 con 0x55d761d8c250 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf32ffd640 1 -- 172.21.15.18:0/172320531 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbf48004390 con 0x55d761d8c250 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf5c594740 1 -- 172.21.15.18:0/172320531 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7619f37e0 con 0x55d761d8c250 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf32ffd640 1 -- 172.21.15.18:0/172320531 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf480035f0 con 0x55d761d8c250 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf5c594740 10 client.16833 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf5c594740 1 -- 172.21.15.18:0/172320531 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d761cf26a0 con 0x55d761d8c250 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf32ffd640 1 -- 172.21.15.18:0/172320531 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf48029020 con 0x55d761d8c250 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf32ffd640 1 -- 172.21.15.18:0/172320531 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fbf48023070 con 0x55d761d8c250 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf5c594740 10 client.16833 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf5c594740 10 client.16833 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf5c594740 20 client.16833 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf5c594740 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d761da5b50 0x55d761da7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf5c594740 4 client.16833 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.436+0000 7fbf5c594740 1 -- 172.21.15.18:0/172320531 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d761a30720 con 0x55d761da5b50 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.437+0000 7fbf52ffd640 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d761da5b50 0x55d761da7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.437+0000 7fbf52ffd640 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d761da5b50 0x55d761da7f70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c68c3674246875ae server_cookie=76ea69d8d3ee65f3 in_seq=0 out_seq=0 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.437+0000 7fbf32ffd640 10 client.16833 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.437+0000 7fbf32ffd640 1 -- 172.21.15.18:0/172320531 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d761a30720 con 0x55d761da5b50 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf32ffd640 10 client.16833 handle_command_reply: tid=0 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf5c594740 1 client.16833 shutdown 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf5c594740 1 -- 172.21.15.18:0/172320531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 msgr2=0x55d761d92780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf5c594740 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 0x55d761d92780 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbf480132d0 tx=0x7fbf48005f90 comp rx=0 tx=0).stop 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf5c594740 1 -- 172.21.15.18:0/172320531 shutdown_connections 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf5c594740 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d761da5b50 0x55d761da7f70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf5c594740 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d95750 0x55d761d93200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf32ffd640 0 client.16833 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf5c594740 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8cb70 0x55d761d92cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf5c594740 1 --2- 172.21.15.18:0/172320531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8c250 0x55d761d92780 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf5c594740 1 -- 172.21.15.18:0/172320531 >> 172.21.15.18:0/172320531 conn(0x55d761d86980 msgr2=0x55d761d886d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf5c594740 1 -- 172.21.15.18:0/172320531 shutdown_connections 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf5c594740 1 -- 172.21.15.18:0/172320531 wait complete. 2024-02-17T12:53:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.438+0000 7fbf5c594740 20 client.16833 trim_cache size 0 max 16384 2024-02-17T12:53:31.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf5c594740 1 Processor -- start 2024-02-17T12:53:31.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf5c594740 1 -- start start 2024-02-17T12:53:31.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf5c594740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8c250 0x55d761d94eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf5c594740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8cb70 0x55d761dac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf5c594740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d95750 0x55d761d8fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf5c594740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761c79130 con 0x55d761d8cb70 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf5c594740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761d93be0 con 0x55d761d95750 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf5c594740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761da2600 con 0x55d761d8c250 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf52ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8cb70 0x55d761dac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf52ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8cb70 0x55d761dac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60460/0 (socket says 172.21.15.18:60460) 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf527fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8c250 0x55d761d94eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf52ffd640 1 -- 172.21.15.18:0/1173871067 learned_addr learned my addr 172.21.15.18:0/1173871067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.454+0000 7fbf527fc640 1 --2- 172.21.15.18:0/1173871067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d95750 0x55d761d8fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.455+0000 7fbf527fc640 1 -- 172.21.15.18:0/1173871067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d95750 msgr2=0x55d761d8fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.455+0000 7fbf527fc640 1 --2- 172.21.15.18:0/1173871067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d95750 0x55d761d8fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.455+0000 7fbf527fc640 1 -- 172.21.15.18:0/1173871067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8cb70 msgr2=0x55d761dac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.455+0000 7fbf527fc640 1 --2- 172.21.15.18:0/1173871067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8cb70 0x55d761dac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.455+0000 7fbf527fc640 1 -- 172.21.15.18:0/1173871067 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d761c8ec20 con 0x55d761d8c250 2024-02-17T12:53:31.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.455+0000 7fbf52ffd640 1 --2- 172.21.15.18:0/1173871067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8cb70 0x55d761dac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:31.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.455+0000 7fbf527fc640 1 --2- 172.21.15.18:0/1173871067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8c250 0x55d761d94eb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbf480092e0 tx=0x7fbf48001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee94d1811e6a39a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.455+0000 7fbf397fa640 1 -- 172.21.15.18:0/1173871067 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480139e0 con 0x55d761d8c250 2024-02-17T12:53:31.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.455+0000 7fbf397fa640 1 -- 172.21.15.18:0/1173871067 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf48029070 con 0x55d761d8c250 2024-02-17T12:53:31.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.455+0000 7fbf397fa640 1 -- 172.21.15.18:0/1173871067 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480052d0 con 0x55d761d8c250 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.456+0000 7fbf5c594740 1 -- 172.21.15.18:0/1173871067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8c250 msgr2=0x55d761d94eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.456+0000 7fbf5c594740 1 --2- 172.21.15.18:0/1173871067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8c250 0x55d761d94eb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbf480092e0 tx=0x7fbf48001e70 comp rx=0 tx=0).stop 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.456+0000 7fbf5c594740 1 -- 172.21.15.18:0/1173871067 shutdown_connections 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.456+0000 7fbf5c594740 1 --2- 172.21.15.18:0/1173871067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d95750 0x55d761d8fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.456+0000 7fbf5c594740 1 --2- 172.21.15.18:0/1173871067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d8cb70 0x55d761dac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.456+0000 7fbf5c594740 1 --2- 172.21.15.18:0/1173871067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8c250 0x55d761d94eb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.456+0000 7fbf5c594740 1 -- 172.21.15.18:0/1173871067 >> 172.21.15.18:0/1173871067 conn(0x55d761d86980 msgr2=0x55d761d81a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.456+0000 7fbf5c594740 1 -- 172.21.15.18:0/1173871067 shutdown_connections 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.456+0000 7fbf5c594740 1 -- 172.21.15.18:0/1173871067 wait complete. 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.456+0000 7fbf5c594740 1 -- start start 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.456+0000 7fbf5c594740 1 Processor -- start 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf5c594740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8c250 0x55d761d9ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf5c594740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d8cb70 0x55d761d9bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf5c594740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d95750 0x55d761df5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf5c594740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761df5560 con 0x55d761d95750 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf5c594740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761df56d0 con 0x55d761d8cb70 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf5c594740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761df5890 con 0x55d761d8c250 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf5c594740 10 client.0 _ll_register_callbacks cb 0x7fbf48005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf527fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d95750 0x55d761df5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf52ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d8cb70 0x55d761d9bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf527fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d95750 0x55d761df5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60470/0 (socket says 172.21.15.18:60470) 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf527fc640 1 -- 172.21.15.18:0/611499819 learned_addr learned my addr 172.21.15.18:0/611499819 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf527fc640 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8c250 0x55d761d9ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf52ffd640 1 -- 172.21.15.18:0/611499819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8c250 msgr2=0x55d761d9ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf52ffd640 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8c250 0x55d761d9ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf52ffd640 1 -- 172.21.15.18:0/611499819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d95750 msgr2=0x55d761df5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf52ffd640 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d95750 0x55d761df5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf52ffd640 1 -- 172.21.15.18:0/611499819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d761cb6060 con 0x55d761d8cb70 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf527fc640 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d95750 0x55d761df5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:31.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf52ffd640 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d8cb70 0x55d761d9bfd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbf44006010 tx=0x7fbf44000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=566ce4c1348a83af server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.457+0000 7fbf32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf32ffd640 1 -- 172.21.15.18:0/611499819 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44053be0 con 0x55d761d8cb70 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf32ffd640 1 -- 172.21.15.18:0/611499819 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbf44050070 con 0x55d761d8cb70 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf5c594740 1 -- 172.21.15.18:0/611499819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf440049e0 con 0x55d761d8cb70 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf32ffd640 1 -- 172.21.15.18:0/611499819 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf44053d40 con 0x55d761d8cb70 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf5c594740 10 client.26207 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf5c594740 1 -- 172.21.15.18:0/611499819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d761cf26a0 con 0x55d761d8cb70 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf32ffd640 1 -- 172.21.15.18:0/611499819 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf440049e0 con 0x55d761d8cb70 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf32ffd640 1 -- 172.21.15.18:0/611499819 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fbf4403c050 con 0x55d761d8cb70 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf5c594740 10 client.26207 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf5c594740 10 client.26207 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf5c594740 20 client.26207 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf5c594740 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d761dfdff0 0x55d761e00410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf5c594740 4 client.26207 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf5c594740 1 -- 172.21.15.18:0/611499819 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d761a7f890 con 0x55d761dfdff0 2024-02-17T12:53:31.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.458+0000 7fbf527fc640 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d761dfdff0 0x55d761e00410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf32ffd640 10 client.26207 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf527fc640 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d761dfdff0 0x55d761e00410 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2f45f250756d892 server_cookie=619628298bf684d7 in_seq=0 out_seq=0 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf32ffd640 1 -- 172.21.15.18:0/611499819 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d761a7f890 con 0x55d761dfdff0 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf32ffd640 10 client.26207 handle_command_reply: tid=0 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf5c594740 1 client.26207 shutdown 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf5c594740 1 -- 172.21.15.18:0/611499819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d8cb70 msgr2=0x55d761d9bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf5c594740 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d8cb70 0x55d761d9bfd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbf44006010 tx=0x7fbf44000bc0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf5c594740 1 -- 172.21.15.18:0/611499819 shutdown_connections 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf5c594740 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d761d95750 0x55d761df5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf5c594740 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55d761dfdff0 0x55d761e00410 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf5c594740 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d761d8cb70 0x55d761d9bfd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf5c594740 1 --2- 172.21.15.18:0/611499819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d761d8c250 0x55d761d9ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf32ffd640 0 client.26207 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf5c594740 1 -- 172.21.15.18:0/611499819 >> 172.21.15.18:0/611499819 conn(0x55d761d86980 msgr2=0x55d761dac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf5c594740 1 -- 172.21.15.18:0/611499819 shutdown_connections 2024-02-17T12:53:31.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf5c594740 1 -- 172.21.15.18:0/611499819 wait complete. 2024-02-17T12:53:31.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.459+0000 7fbf5c594740 20 client.26207 trim_cache size 0 max 16384 2024-02-17T12:53:31.457 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:53:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.460+0000 7fbf599bc640 1 -- 172.21.15.18:0/4166596938 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fbf3c0a2230 msgr2=0x7fbf3c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.460+0000 7fbf599bc640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fbf3c0a2230 0x7fbf3c0a46f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c097980 tx=0x7fbf4c06a150 comp rx=0 tx=0).stop 2024-02-17T12:53:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.460+0000 7fbf599bc640 1 -- 172.21.15.18:0/4166596938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf54199130 msgr2=0x7fbf541a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.460+0000 7fbf599bc640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf54199130 0x7fbf541a36e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00efd0 tx=0x7fbf4c00c6a0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.460+0000 7fbf599bc640 1 -- 172.21.15.18:0/4166596938 shutdown_connections 2024-02-17T12:53:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.460+0000 7fbf599bc640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fbf3c0a2230 0x7fbf3c0a46f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.460+0000 7fbf599bc640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf54199130 0x7fbf541a36e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.460+0000 7fbf599bc640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf54105330 0x7fbf54198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.460+0000 7fbf599bc640 1 --2- 172.21.15.18:0/4166596938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf54100260 0x7fbf54198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.460+0000 7fbf599bc640 1 -- 172.21.15.18:0/4166596938 >> 172.21.15.18:0/4166596938 conn(0x7fbf54077180 msgr2=0x7fbf540fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.460+0000 7fbf599bc640 1 -- 172.21.15.18:0/4166596938 shutdown_connections 2024-02-17T12:53:31.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.460+0000 7fbf599bc640 1 -- 172.21.15.18:0/4166596938 wait complete. 2024-02-17T12:53:31.462 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:53:31.462 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:53:31.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.580+0000 7fa10f5d0640 1 -- 172.21.15.18:0/3787589806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1080685f0 msgr2=0x7fa1080689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.580+0000 7fa10f5d0640 1 --2- 172.21.15.18:0/3787589806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1080685f0 0x7fa1080689d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa104009930 tx=0x7fa10402bc30 comp rx=0 tx=0).stop 2024-02-17T12:53:31.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.580+0000 7fa10f5d0640 1 -- 172.21.15.18:0/3787589806 shutdown_connections 2024-02-17T12:53:31.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.580+0000 7fa10f5d0640 1 --2- 172.21.15.18:0/3787589806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1081064c0 0x7fa10810e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.580+0000 7fa10f5d0640 1 --2- 172.21.15.18:0/3787589806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa108068fa0 0x7fa108105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.580+0000 7fa10f5d0640 1 --2- 172.21.15.18:0/3787589806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1080685f0 0x7fa1080689d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.580+0000 7fa10f5d0640 1 -- 172.21.15.18:0/3787589806 >> 172.21.15.18:0/3787589806 conn(0x7fa108077180 msgr2=0x7fa108077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.580+0000 7fa10f5d0640 1 -- 172.21.15.18:0/3787589806 shutdown_connections 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.580+0000 7fa10f5d0640 1 -- 172.21.15.18:0/3787589806 wait complete. 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10f5d0640 1 Processor -- start 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10f5d0640 1 -- start start 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10f5d0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1080685f0 0x7fa10819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10f5d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1081064c0 0x7fa10819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10f5d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1081975e0 0x7fa108197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10f5d0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa108197fd0 con 0x7fa1080685f0 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10f5d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa108198140 con 0x7fa1081975e0 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10f5d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa108198300 con 0x7fa1081064c0 2024-02-17T12:53:31.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10d36c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1080685f0 0x7fa10819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10d36c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1080685f0 0x7fa10819d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60486/0 (socket says 172.21.15.18:60486) 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10d36c640 1 -- 172.21.15.18:0/893387867 learned_addr learned my addr 172.21.15.18:0/893387867 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10db6d640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1081975e0 0x7fa108197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10cb6b640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1081064c0 0x7fa10819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10d36c640 1 -- 172.21.15.18:0/893387867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1081064c0 msgr2=0x7fa10819d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10d36c640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1081064c0 0x7fa10819d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10d36c640 1 -- 172.21.15.18:0/893387867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1081975e0 msgr2=0x7fa108197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10d36c640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1081975e0 0x7fa108197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.581+0000 7fa10d36c640 1 -- 172.21.15.18:0/893387867 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa104009560 con 0x7fa1080685f0 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.582+0000 7fa10d36c640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1080685f0 0x7fa10819d430 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa104009230 tx=0x7fa10402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.582+0000 7fa0fe7fc640 1 -- 172.21.15.18:0/893387867 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104004350 con 0x7fa1080685f0 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.582+0000 7fa0fe7fc640 1 -- 172.21.15.18:0/893387867 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1040044b0 con 0x7fa1080685f0 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.582+0000 7fa10f5d0640 1 -- 172.21.15.18:0/893387867 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1081a42f0 con 0x7fa1080685f0 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.582+0000 7fa0fe7fc640 1 -- 172.21.15.18:0/893387867 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104003d70 con 0x7fa1080685f0 2024-02-17T12:53:31.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.582+0000 7fa10f5d0640 1 -- 172.21.15.18:0/893387867 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1081a47a0 con 0x7fa1080685f0 2024-02-17T12:53:31.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.584+0000 7fa0fe7fc640 1 -- 172.21.15.18:0/893387867 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa104004df0 con 0x7fa1080685f0 2024-02-17T12:53:31.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa11219b740 1 Processor -- start 2024-02-17T12:53:31.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa11219b740 1 -- start start 2024-02-17T12:53:31.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa11219b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0f250 0x5568d5d0f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa11219b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d0fb70 0x5568d5d18210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa11219b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d18750 0x5568d5d1ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa11219b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5cca960 con 0x5568d5d0f250 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa11219b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5c7e310 con 0x5568d5d18750 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa11219b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5c7b840 con 0x5568d5d0fb70 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa10db6d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0f250 0x5568d5d0f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa10d36c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d0fb70 0x5568d5d18210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa10db6d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0f250 0x5568d5d0f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60500/0 (socket says 172.21.15.18:60500) 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa10cb6b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d18750 0x5568d5d1ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa10db6d640 1 -- 172.21.15.18:0/568951638 learned_addr learned my addr 172.21.15.18:0/568951638 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa0fe7fc640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0f40a2230 0x7fa0f40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa10cb6b640 1 -- 172.21.15.18:0/568951638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d0fb70 msgr2=0x5568d5d18210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa10cb6b640 1 --2- 172.21.15.18:0/568951638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d0fb70 0x5568d5d18210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa10cb6b640 1 -- 172.21.15.18:0/568951638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0f250 msgr2=0x5568d5d0f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa10cb6b640 1 --2- 172.21.15.18:0/568951638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0f250 0x5568d5d0f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa10cb6b640 1 -- 172.21.15.18:0/568951638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568d5c39060 con 0x5568d5d18750 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa10db6d640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0f40a2230 0x7fa0f40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.585+0000 7fa0fe7fc640 1 -- 172.21.15.18:0/893387867 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa1040f0030 con 0x7fa1080685f0 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.586+0000 7fa10cb6b640 1 --2- 172.21.15.18:0/568951638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d18750 0x5568d5d1ab40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa108198720 tx=0x7fa0f8010370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4675bb3d1cd9a581 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.586+0000 7fa0f37fe640 1 -- 172.21.15.18:0/568951638 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8010d70 con 0x5568d5d18750 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.586+0000 7fa10db6d640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0f40a2230 0x7fa0f40a46f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa100009fd0 tx=0x7fa100004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.586+0000 7fa0f37fe640 1 -- 172.21.15.18:0/568951638 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0f8010ed0 con 0x5568d5d18750 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.586+0000 7fa0f37fe640 1 -- 172.21.15.18:0/568951638 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8012450 con 0x5568d5d18750 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.587+0000 7fa11219b740 1 -- 172.21.15.18:0/568951638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d18750 msgr2=0x5568d5d1ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.587+0000 7fa11219b740 1 --2- 172.21.15.18:0/568951638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d18750 0x5568d5d1ab40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa108198720 tx=0x7fa0f8010370 comp rx=0 tx=0).stop 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.587+0000 7fa11219b740 1 -- 172.21.15.18:0/568951638 shutdown_connections 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.587+0000 7fa11219b740 1 --2- 172.21.15.18:0/568951638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d18750 0x5568d5d1ab40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.587+0000 7fa11219b740 1 --2- 172.21.15.18:0/568951638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d0fb70 0x5568d5d18210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.587+0000 7fa11219b740 1 --2- 172.21.15.18:0/568951638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0f250 0x5568d5d0f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.587+0000 7fa11219b740 1 -- 172.21.15.18:0/568951638 >> 172.21.15.18:0/568951638 conn(0x5568d5d09980 msgr2=0x5568d5d09d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.587+0000 7fa11219b740 1 -- 172.21.15.18:0/568951638 shutdown_connections 2024-02-17T12:53:31.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.587+0000 7fa11219b740 1 -- 172.21.15.18:0/568951638 wait complete. 2024-02-17T12:53:31.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.587+0000 7fa11219b740 1 -- start start 2024-02-17T12:53:31.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.587+0000 7fa11219b740 1 Processor -- start 2024-02-17T12:53:31.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa11219b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0f250 0x5568d5d15750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa11219b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 0x5568d5d15c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa11219b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d18750 0x5568d5d161d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa11219b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5c77020 con 0x5568d5d0fb70 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa11219b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d594a430 con 0x5568d5d0f250 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa11219b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5906e00 con 0x5568d5d18750 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa11219b740 10 client.0 _ll_register_callbacks cb 0x5568d5c1ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa10cb6b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 0x5568d5d15c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa10cb6b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 0x5568d5d15c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60514/0 (socket says 172.21.15.18:60514) 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa10cb6b640 1 -- 172.21.15.18:0/2112838472 learned_addr learned my addr 172.21.15.18:0/2112838472 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa10db6d640 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d18750 0x5568d5d161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa10d36c640 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0f250 0x5568d5d15750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa10cb6b640 1 -- 172.21.15.18:0/2112838472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d18750 msgr2=0x5568d5d161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa10cb6b640 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d18750 0x5568d5d161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa10cb6b640 1 -- 172.21.15.18:0/2112838472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0f250 msgr2=0x5568d5d15750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa10cb6b640 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0f250 0x5568d5d15750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa10cb6b640 1 -- 172.21.15.18:0/2112838472 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f80097c0 con 0x5568d5d0fb70 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa10cb6b640 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 0x5568d5d15c90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8003ba0 tx=0x7fa0f8003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e455761a3c272ddd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa0f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2112838472 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8021070 con 0x5568d5d0fb70 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2112838472 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa0f8016070 con 0x5568d5d0fb70 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.588+0000 7fa11219b740 1 -- 172.21.15.18:0/2112838472 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568d59767e0 con 0x5568d5d0fb70 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.589+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2112838472 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0f8004270 con 0x5568d5d0fb70 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.589+0000 7fa11219b740 10 client.16851 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.589+0000 7fa11219b740 1 -- 172.21.15.18:0/2112838472 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5568d5c756a0 con 0x5568d5d0fb70 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.589+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2112838472 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5568d59767e0 con 0x5568d5d0fb70 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.589+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2112838472 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa0f801a430 con 0x5568d5d0fb70 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.589+0000 7fa11219b740 10 client.16851 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.589+0000 7fa11219b740 10 client.16851 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.589+0000 7fa11219b740 20 client.16851 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.589+0000 7fa11219b740 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5568d5d28ad0 0x5568d5d2aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.589+0000 7fa11219b740 4 client.16851 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.589+0000 7fa11219b740 1 -- 172.21.15.18:0/2112838472 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568d59b3720 con 0x5568d5d28ad0 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.589+0000 7fa10d36c640 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5568d5d28ad0 0x5568d5d2aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.590+0000 7fa0f0ff9640 10 client.16851 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.590+0000 7fa10d36c640 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5568d5d28ad0 0x5568d5d2aef0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dfa1f7d606a7dd63 server_cookie=26532611a084a02b in_seq=0 out_seq=0 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.590+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2112838472 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5568d59b3720 con 0x5568d5d28ad0 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.590+0000 7fa0f0ff9640 10 client.16851 handle_command_reply: tid=0 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.590+0000 7fa11219b740 1 client.16851 shutdown 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.591+0000 7fa11219b740 1 -- 172.21.15.18:0/2112838472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 msgr2=0x5568d5d15c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.591+0000 7fa11219b740 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 0x5568d5d15c90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8003ba0 tx=0x7fa0f8003890 comp rx=0 tx=0).stop 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.591+0000 7fa11219b740 1 -- 172.21.15.18:0/2112838472 shutdown_connections 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.591+0000 7fa11219b740 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5568d5d28ad0 0x5568d5d2aef0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.591+0000 7fa11219b740 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d18750 0x5568d5d161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.591+0000 7fa11219b740 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 0x5568d5d15c90 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.591+0000 7fa0f0ff9640 0 client.16851 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.591+0000 7fa11219b740 1 --2- 172.21.15.18:0/2112838472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0f250 0x5568d5d15750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.591+0000 7fa11219b740 1 -- 172.21.15.18:0/2112838472 >> 172.21.15.18:0/2112838472 conn(0x5568d5d09980 msgr2=0x5568d5d0b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.591+0000 7fa11219b740 1 -- 172.21.15.18:0/2112838472 shutdown_connections 2024-02-17T12:53:31.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.591+0000 7fa11219b740 1 -- 172.21.15.18:0/2112838472 wait complete. 2024-02-17T12:53:31.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.591+0000 7fa11219b740 20 client.16851 trim_cache size 0 max 16384 2024-02-17T12:53:31.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.604+0000 7fa11219b740 1 Processor -- start 2024-02-17T12:53:31.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.604+0000 7fa11219b740 1 -- start start 2024-02-17T12:53:31.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.604+0000 7fa11219b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0f250 0x5568d5d17e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.604+0000 7fa11219b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 0x5568d5d2f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.604+0000 7fa11219b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d18750 0x5568d5d129b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.604+0000 7fa11219b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5bfc130 con 0x5568d5d0fb70 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.604+0000 7fa11219b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5d0a780 con 0x5568d5d0f250 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.604+0000 7fa11219b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5d11380 con 0x5568d5d18750 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa10cb6b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 0x5568d5d2f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa10cb6b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 0x5568d5d2f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60526/0 (socket says 172.21.15.18:60526) 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa10cb6b640 1 -- 172.21.15.18:0/3920378446 learned_addr learned my addr 172.21.15.18:0/3920378446 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa10d36c640 1 --2- 172.21.15.18:0/3920378446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0f250 0x5568d5d17e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa10db6d640 1 --2- 172.21.15.18:0/3920378446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d18750 0x5568d5d129b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa10cb6b640 1 -- 172.21.15.18:0/3920378446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d18750 msgr2=0x5568d5d129b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa10cb6b640 1 --2- 172.21.15.18:0/3920378446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d18750 0x5568d5d129b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa10cb6b640 1 -- 172.21.15.18:0/3920378446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0f250 msgr2=0x5568d5d17e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa10cb6b640 1 --2- 172.21.15.18:0/3920378446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0f250 0x5568d5d17e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa10cb6b640 1 -- 172.21.15.18:0/3920378446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568d5c11c20 con 0x5568d5d0fb70 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa10cb6b640 1 --2- 172.21.15.18:0/3920378446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 0x5568d5d2f890 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8010ea0 tx=0x7fa0f8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a821ccd26597a600 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa0f2ffd640 1 -- 172.21.15.18:0/3920378446 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8021070 con 0x5568d5d0fb70 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa0f2ffd640 1 -- 172.21.15.18:0/3920378446 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0f8005510 con 0x5568d5d0fb70 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.605+0000 7fa0f2ffd640 1 -- 172.21.15.18:0/3920378446 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8004660 con 0x5568d5d0fb70 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.606+0000 7fa11219b740 1 -- 172.21.15.18:0/3920378446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 msgr2=0x5568d5d2f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.606+0000 7fa11219b740 1 --2- 172.21.15.18:0/3920378446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 0x5568d5d2f890 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8010ea0 tx=0x7fa0f8005850 comp rx=0 tx=0).stop 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.606+0000 7fa11219b740 1 -- 172.21.15.18:0/3920378446 shutdown_connections 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.606+0000 7fa11219b740 1 --2- 172.21.15.18:0/3920378446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d18750 0x5568d5d129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.606+0000 7fa11219b740 1 --2- 172.21.15.18:0/3920378446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d0fb70 0x5568d5d2f890 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.606+0000 7fa11219b740 1 --2- 172.21.15.18:0/3920378446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0f250 0x5568d5d17e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.606+0000 7fa11219b740 1 -- 172.21.15.18:0/3920378446 >> 172.21.15.18:0/3920378446 conn(0x5568d5d09980 msgr2=0x5568d5d04a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.606+0000 7fa11219b740 1 -- 172.21.15.18:0/3920378446 shutdown_connections 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.606+0000 7fa11219b740 1 -- 172.21.15.18:0/3920378446 wait complete. 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.606+0000 7fa11219b740 1 -- start start 2024-02-17T12:53:31.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.606+0000 7fa11219b740 1 Processor -- start 2024-02-17T12:53:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa11219b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d0f250 0x5568d5d1e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa11219b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0fb70 0x5568d5d1ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa11219b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d18750 0x5568d5d7f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa11219b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5d7f560 con 0x5568d5d18750 2024-02-17T12:53:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa11219b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5d7f6a0 con 0x5568d5d0fb70 2024-02-17T12:53:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa11219b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d5d7f7e0 con 0x5568d5d0f250 2024-02-17T12:53:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa11219b740 10 client.0 _ll_register_callbacks cb 0x7fa0f80095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa10db6d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d18750 0x5568d5d7f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa10db6d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d18750 0x5568d5d7f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60538/0 (socket says 172.21.15.18:60538) 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa10db6d640 1 -- 172.21.15.18:0/2779111146 learned_addr learned my addr 172.21.15.18:0/2779111146 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa10cb6b640 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0fb70 0x5568d5d1ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa10d36c640 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d0f250 0x5568d5d1e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa10db6d640 1 -- 172.21.15.18:0/2779111146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d0f250 msgr2=0x5568d5d1e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa10db6d640 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d0f250 0x5568d5d1e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa10db6d640 1 -- 172.21.15.18:0/2779111146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0fb70 msgr2=0x5568d5d1ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa10db6d640 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0fb70 0x5568d5d1ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa10db6d640 1 -- 172.21.15.18:0/2779111146 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568d5c756a0 con 0x5568d5d18750 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa10db6d640 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d18750 0x5568d5d7f020 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa100009940 tx=0x7fa100004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4f21d901339d88 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.607+0000 7fa0f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2779111146 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100011bc0 con 0x5568d5d18750 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2779111146 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa100011d20 con 0x5568d5d18750 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa11219b740 1 -- 172.21.15.18:0/2779111146 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0f8010ed0 con 0x5568d5d18750 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2779111146 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa100010430 con 0x5568d5d18750 2024-02-17T12:53:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa11219b740 10 client.16857 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa11219b740 1 -- 172.21.15.18:0/2779111146 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5568d5c378c0 con 0x5568d5d18750 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2779111146 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0f8010ed0 con 0x5568d5d18750 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2779111146 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa100023510 con 0x5568d5d18750 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa11219b740 10 client.16857 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa11219b740 10 client.16857 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa11219b740 20 client.16857 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa11219b740 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5568d5d81030 0x5568d5d83450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa11219b740 4 client.16857 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa11219b740 1 -- 172.21.15.18:0/2779111146 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5568d5a02890 con 0x5568d5d81030 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.608+0000 7fa10d36c640 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5568d5d81030 0x5568d5d83450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa0f0ff9640 10 client.16857 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa10d36c640 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5568d5d81030 0x5568d5d83450 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=65b6f686070b8550 server_cookie=de3e8ed6c130c39b in_seq=0 out_seq=0 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2779111146 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5568d5a02890 con 0x5568d5d81030 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa0f0ff9640 10 client.16857 handle_command_reply: tid=0 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa11219b740 1 client.16857 shutdown 2024-02-17T12:53:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa11219b740 1 -- 172.21.15.18:0/2779111146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d18750 msgr2=0x5568d5d7f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa11219b740 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d18750 0x5568d5d7f020 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa100009940 tx=0x7fa100004a40 comp rx=0 tx=0).stop 2024-02-17T12:53:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa11219b740 1 -- 172.21.15.18:0/2779111146 shutdown_connections 2024-02-17T12:53:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa11219b740 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5568d5d81030 0x5568d5d83450 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa11219b740 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568d5d18750 0x5568d5d7f020 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa11219b740 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568d5d0fb70 0x5568d5d1ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa11219b740 1 --2- 172.21.15.18:0/2779111146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568d5d0f250 0x5568d5d1e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa11219b740 1 -- 172.21.15.18:0/2779111146 >> 172.21.15.18:0/2779111146 conn(0x5568d5d09980 msgr2=0x5568d5d13be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa0f0ff9640 0 client.16857 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:53:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa11219b740 1 -- 172.21.15.18:0/2779111146 shutdown_connections 2024-02-17T12:53:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa11219b740 1 -- 172.21.15.18:0/2779111146 wait complete. 2024-02-17T12:53:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.609+0000 7fa11219b740 20 client.16857 trim_cache size 0 max 16384 2024-02-17T12:53:31.606 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:53:31.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.610+0000 7fa10f5d0640 1 -- 172.21.15.18:0/893387867 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0f40a2230 msgr2=0x7fa0f40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.610+0000 7fa10f5d0640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0f40a2230 0x7fa0f40a46f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa100009fd0 tx=0x7fa100004ab0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.610+0000 7fa10f5d0640 1 -- 172.21.15.18:0/893387867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1080685f0 msgr2=0x7fa10819d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.610+0000 7fa10f5d0640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1080685f0 0x7fa10819d430 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa104009230 tx=0x7fa10402c6c0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.610+0000 7fa10f5d0640 1 -- 172.21.15.18:0/893387867 shutdown_connections 2024-02-17T12:53:31.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.610+0000 7fa10f5d0640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0f40a2230 0x7fa0f40a46f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.610+0000 7fa10f5d0640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1081975e0 0x7fa108197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.610+0000 7fa10f5d0640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1081064c0 0x7fa10819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.610+0000 7fa10f5d0640 1 --2- 172.21.15.18:0/893387867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1080685f0 0x7fa10819d430 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.610+0000 7fa10f5d0640 1 -- 172.21.15.18:0/893387867 >> 172.21.15.18:0/893387867 conn(0x7fa108077180 msgr2=0x7fa1080fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.610+0000 7fa10f5d0640 1 -- 172.21.15.18:0/893387867 shutdown_connections 2024-02-17T12:53:31.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.610+0000 7fa10f5d0640 1 -- 172.21.15.18:0/893387867 wait complete. 2024-02-17T12:53:31.611 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:31 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:53:31.611 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:31 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1474526647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:53:31.612 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:53:31.612 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:53:31.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:31 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:53:31.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:31 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1474526647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:53:31.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.729+0000 7f08c4127640 1 -- 172.21.15.18:0/2460723884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08bc105330 msgr2=0x7f08bc10cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.729+0000 7f08c4127640 1 --2- 172.21.15.18:0/2460723884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08bc105330 0x7f08bc10cf00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009cd0 tx=0x7f08b402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.729+0000 7f08c4127640 1 -- 172.21.15.18:0/2460723884 shutdown_connections 2024-02-17T12:53:31.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.729+0000 7f08c4127640 1 --2- 172.21.15.18:0/2460723884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08bc105330 0x7f08bc10cf00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.729+0000 7f08c4127640 1 --2- 172.21.15.18:0/2460723884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc100c10 0x7f08bc104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.729+0000 7f08c4127640 1 --2- 172.21.15.18:0/2460723884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08bc100260 0x7f08bc100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.729+0000 7f08c4127640 1 -- 172.21.15.18:0/2460723884 >> 172.21.15.18:0/2460723884 conn(0x7f08bc077180 msgr2=0x7f08bc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.729+0000 7f08c4127640 1 -- 172.21.15.18:0/2460723884 shutdown_connections 2024-02-17T12:53:31.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.729+0000 7f08c4127640 1 -- 172.21.15.18:0/2460723884 wait complete. 2024-02-17T12:53:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c4127640 1 Processor -- start 2024-02-17T12:53:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c4127640 1 -- start start 2024-02-17T12:53:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c4127640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08bc100260 0x7f08bc101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c4127640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08bc105330 0x7f08bc102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c4127640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc102770 0x7f08bc1a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c4127640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1a5f30 con 0x7f08bc105330 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c4127640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1a60a0 con 0x7f08bc100260 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c4127640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1a6260 con 0x7f08bc102770 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c16c2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08bc105330 0x7f08bc102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c16c2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08bc105330 0x7f08bc102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60556/0 (socket says 172.21.15.18:60556) 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c1ec3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08bc100260 0x7f08bc101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c16c2640 1 -- 172.21.15.18:0/1388950220 learned_addr learned my addr 172.21.15.18:0/1388950220 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.730+0000 7f08c26c4640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc102770 0x7f08bc1a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.731+0000 7f08c1ec3640 1 -- 172.21.15.18:0/1388950220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc102770 msgr2=0x7f08bc1a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.731+0000 7f08c1ec3640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc102770 0x7f08bc1a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.731+0000 7f08c1ec3640 1 -- 172.21.15.18:0/1388950220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08bc105330 msgr2=0x7f08bc102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.731+0000 7f08c1ec3640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08bc105330 0x7f08bc102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.731+0000 7f08c1ec3640 1 -- 172.21.15.18:0/1388950220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b4009900 con 0x7f08bc100260 2024-02-17T12:53:31.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.731+0000 7f08c16c2640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08bc105330 0x7f08bc102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:31.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.731+0000 7f08c1ec3640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08bc100260 0x7f08bc101cf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f08b800e9f0 tx=0x7f08b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.731+0000 7f08b2ffd640 1 -- 172.21.15.18:0/1388950220 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b800ccc0 con 0x7f08bc100260 2024-02-17T12:53:31.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.731+0000 7f08c4127640 1 -- 172.21.15.18:0/1388950220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc1a6540 con 0x7f08bc100260 2024-02-17T12:53:31.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.731+0000 7f08b2ffd640 1 -- 172.21.15.18:0/1388950220 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08b800ce20 con 0x7f08bc100260 2024-02-17T12:53:31.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.731+0000 7f08b2ffd640 1 -- 172.21.15.18:0/1388950220 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8018920 con 0x7f08bc100260 2024-02-17T12:53:31.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.731+0000 7f08c4127640 1 -- 172.21.15.18:0/1388950220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc1a6aa0 con 0x7f08bc100260 2024-02-17T12:53:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.733+0000 7f08b2ffd640 1 -- 172.21.15.18:0/1388950220 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f08b8018a80 con 0x7f08bc100260 2024-02-17T12:53:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c6dd0740 1 Processor -- start 2024-02-17T12:53:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c6dd0740 1 -- start start 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c6dd0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbf250 0x55f132cbf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c6dd0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 0x55f132cc8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c6dd0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cc8750 0x55f132ccab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c6dd0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f132c7a960 con 0x55f132cbfb70 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c6dd0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f132c2e310 con 0x55f132cc8750 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c6dd0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f132c2b840 con 0x55f132cbf250 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c1ec3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 0x55f132cc8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c1ec3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 0x55f132cc8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60560/0 (socket says 172.21.15.18:60560) 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c1ec3640 1 -- 172.21.15.18:0/1290063508 learned_addr learned my addr 172.21.15.18:0/1290063508 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c16c2640 1 --2- 172.21.15.18:0/1290063508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cc8750 0x55f132ccab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c26c4640 1 --2- 172.21.15.18:0/1290063508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbf250 0x55f132cbf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08b2ffd640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f08a80a22e0 0x7f08a80a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c1ec3640 1 -- 172.21.15.18:0/1290063508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbf250 msgr2=0x55f132cbf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c1ec3640 1 --2- 172.21.15.18:0/1290063508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbf250 0x55f132cbf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c1ec3640 1 -- 172.21.15.18:0/1290063508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cc8750 msgr2=0x55f132ccab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c1ec3640 1 --2- 172.21.15.18:0/1290063508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cc8750 0x55f132ccab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c1ec3640 1 -- 172.21.15.18:0/1290063508 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f132be9060 con 0x55f132cbfb70 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.734+0000 7f08c26c4640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f08a80a22e0 0x7f08a80a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.735+0000 7f08b2ffd640 1 -- 172.21.15.18:0/1388950220 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f08b8014070 con 0x7f08bc100260 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.735+0000 7f08c1ec3640 1 --2- 172.21.15.18:0/1290063508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 0x55f132cc8210 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f08b80cb960 tx=0x7f08b8090410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b710569c48a0c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.735+0000 7f08c26c4640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f08a80a22e0 0x7f08a80a47a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f08b4005cc0 tx=0x7f08b40095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.735+0000 7f08a7fff640 1 -- 172.21.15.18:0/1290063508 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b809ab10 con 0x55f132cbfb70 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.735+0000 7f08a7fff640 1 -- 172.21.15.18:0/1290063508 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08b809a880 con 0x55f132cbfb70 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.735+0000 7f08a7fff640 1 -- 172.21.15.18:0/1290063508 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8010b30 con 0x55f132cbfb70 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 -- 172.21.15.18:0/1290063508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 msgr2=0x55f132cc8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/1290063508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 0x55f132cc8210 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f08b80cb960 tx=0x7f08b8090410 comp rx=0 tx=0).stop 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 -- 172.21.15.18:0/1290063508 shutdown_connections 2024-02-17T12:53:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/1290063508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cc8750 0x55f132ccab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/1290063508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 0x55f132cc8210 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/1290063508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbf250 0x55f132cbf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 -- 172.21.15.18:0/1290063508 >> 172.21.15.18:0/1290063508 conn(0x55f132cb9980 msgr2=0x55f132cb9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 -- 172.21.15.18:0/1290063508 shutdown_connections 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 -- 172.21.15.18:0/1290063508 wait complete. 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 -- start start 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 Processor -- start 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132cc57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.736+0000 7f08c6dd0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 0x55f132cc5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c6dd0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cc8750 0x55f132cc6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c6dd0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f132c27020 con 0x55f132cbfb70 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c6dd0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1328fa430 con 0x55f132cbf250 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c6dd0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1328b6e00 con 0x55f132cc8750 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c6dd0740 10 client.0 _ll_register_callbacks cb 0x55f132bcecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c16c2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 0x55f132cc5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c16c2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 0x55f132cc5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60574/0 (socket says 172.21.15.18:60574) 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c1ec3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132cc57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c16c2640 1 -- 172.21.15.18:0/2749455525 learned_addr learned my addr 172.21.15.18:0/2749455525 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c26c4640 1 --2- 172.21.15.18:0/2749455525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cc8750 0x55f132cc6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c16c2640 1 -- 172.21.15.18:0/2749455525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cc8750 msgr2=0x55f132cc6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c16c2640 1 --2- 172.21.15.18:0/2749455525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cc8750 0x55f132cc6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c16c2640 1 -- 172.21.15.18:0/2749455525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 msgr2=0x55f132cc57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c16c2640 1 --2- 172.21.15.18:0/2749455525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132cc57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c16c2640 1 -- 172.21.15.18:0/2749455525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc1a6aa0 con 0x55f132cbfb70 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c16c2640 1 --2- 172.21.15.18:0/2749455525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 0x55f132cc5ce0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f08ac012d50 tx=0x7f08ac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ef9c2d6031d789 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08a57fa640 1 -- 172.21.15.18:0/2749455525 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac017070 con 0x55f132cbfb70 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08a57fa640 1 -- 172.21.15.18:0/2749455525 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f08ac012110 con 0x55f132cbfb70 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c6dd0740 1 -- 172.21.15.18:0/2749455525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1329267e0 con 0x55f132cbfb70 2024-02-17T12:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08a57fa640 1 -- 172.21.15.18:0/2749455525 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08ac0122d0 con 0x55f132cbfb70 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c6dd0740 10 client.16869 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.737+0000 7f08c6dd0740 1 -- 172.21.15.18:0/2749455525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f132c256a0 con 0x55f132cbfb70 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.738+0000 7f08a57fa640 1 -- 172.21.15.18:0/2749455525 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f1329267e0 con 0x55f132cbfb70 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.738+0000 7f08a57fa640 1 -- 172.21.15.18:0/2749455525 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f08ac025430 con 0x55f132cbfb70 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.738+0000 7f08c6dd0740 10 client.16869 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.738+0000 7f08c6dd0740 10 client.16869 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.738+0000 7f08c6dd0740 20 client.16869 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.738+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/2749455525 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55f132cd8b10 0x55f132cdaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.738+0000 7f08c6dd0740 4 client.16869 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.738+0000 7f08c6dd0740 1 -- 172.21.15.18:0/2749455525 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f132963720 con 0x55f132cd8b10 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.738+0000 7f08c1ec3640 1 --2- 172.21.15.18:0/2749455525 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55f132cd8b10 0x55f132cdaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.738+0000 7f08c1ec3640 1 --2- 172.21.15.18:0/2749455525 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55f132cd8b10 0x55f132cdaf50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5b9e0c725c49cb6 server_cookie=44133d6e860099de in_seq=0 out_seq=0 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.738+0000 7f08a57fa640 10 client.16869 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08a57fa640 1 -- 172.21.15.18:0/2749455525 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f132963720 con 0x55f132cd8b10 2024-02-17T12:53:31.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08a57fa640 10 client.16869 handle_command_reply: tid=0 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08c6dd0740 1 client.16869 shutdown 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08c6dd0740 1 -- 172.21.15.18:0/2749455525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 msgr2=0x55f132cc5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/2749455525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 0x55f132cc5ce0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f08ac012d50 tx=0x7f08ac00f790 comp rx=0 tx=0).stop 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08c6dd0740 1 -- 172.21.15.18:0/2749455525 shutdown_connections 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/2749455525 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55f132cd8b10 0x55f132cdaf50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/2749455525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cc8750 0x55f132cc6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/2749455525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cbfb70 0x55f132cc5ce0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/2749455525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132cc57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08a57fa640 0 client.16869 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08c6dd0740 1 -- 172.21.15.18:0/2749455525 >> 172.21.15.18:0/2749455525 conn(0x55f132cb9980 msgr2=0x55f132cbb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08c6dd0740 1 -- 172.21.15.18:0/2749455525 shutdown_connections 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08c6dd0740 1 -- 172.21.15.18:0/2749455525 wait complete. 2024-02-17T12:53:31.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.739+0000 7f08c6dd0740 20 client.16869 trim_cache size 0 max 16384 2024-02-17T12:53:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.751+0000 7f08c6dd0740 1 Processor -- start 2024-02-17T12:53:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.751+0000 7f08c6dd0740 1 -- start start 2024-02-17T12:53:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.751+0000 7f08c6dd0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132cc7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.751+0000 7f08c6dd0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbfb70 0x55f132cc6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.751+0000 7f08c6dd0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 0x55f132cc2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.751+0000 7f08c6dd0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f132bac130 con 0x55f132cc8750 2024-02-17T12:53:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.751+0000 7f08c6dd0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f132cbeeb0 con 0x55f132cbf250 2024-02-17T12:53:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.751+0000 7f08c6dd0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f132ce01a0 con 0x55f132cbfb70 2024-02-17T12:53:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08c26c4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 0x55f132cc2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08c26c4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 0x55f132cc2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60582/0 (socket says 172.21.15.18:60582) 2024-02-17T12:53:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08c26c4640 1 -- 172.21.15.18:0/1586677059 learned_addr learned my addr 172.21.15.18:0/1586677059 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08c1ec3640 1 --2- 172.21.15.18:0/1586677059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132cc7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08c16c2640 1 --2- 172.21.15.18:0/1586677059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbfb70 0x55f132cc6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08c26c4640 1 -- 172.21.15.18:0/1586677059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbfb70 msgr2=0x55f132cc6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08c26c4640 1 --2- 172.21.15.18:0/1586677059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbfb70 0x55f132cc6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08c26c4640 1 -- 172.21.15.18:0/1586677059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 msgr2=0x55f132cc7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08c26c4640 1 --2- 172.21.15.18:0/1586677059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132cc7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08c26c4640 1 -- 172.21.15.18:0/1586677059 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f132bc1c20 con 0x55f132cc8750 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08c26c4640 1 --2- 172.21.15.18:0/1586677059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 0x55f132cc2a00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009cd0 tx=0x7f08b4042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fea2e1a6f4a3bbb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08a77fe640 1 -- 172.21.15.18:0/1586677059 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4034e90 con 0x55f132cc8750 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08a77fe640 1 -- 172.21.15.18:0/1586677059 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08b4003970 con 0x55f132cc8750 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.752+0000 7f08a77fe640 1 -- 172.21.15.18:0/1586677059 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40045d0 con 0x55f132cc8750 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.753+0000 7f08c6dd0740 1 -- 172.21.15.18:0/1586677059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 msgr2=0x55f132cc2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.753+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/1586677059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 0x55f132cc2a00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009cd0 tx=0x7f08b4042000 comp rx=0 tx=0).stop 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.753+0000 7f08c6dd0740 1 -- 172.21.15.18:0/1586677059 shutdown_connections 2024-02-17T12:53:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.753+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/1586677059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 0x55f132cc2a00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.753+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/1586677059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbfb70 0x55f132cc6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.753+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/1586677059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132cc7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.753+0000 7f08c6dd0740 1 -- 172.21.15.18:0/1586677059 >> 172.21.15.18:0/1586677059 conn(0x55f132cb9980 msgr2=0x55f132cc0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.753+0000 7f08c6dd0740 1 -- 172.21.15.18:0/1586677059 shutdown_connections 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.753+0000 7f08c6dd0740 1 -- 172.21.15.18:0/1586677059 wait complete. 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.753+0000 7f08c6dd0740 1 -- start start 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.753+0000 7f08c6dd0740 1 Processor -- start 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c6dd0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132ccdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c6dd0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbfb70 0x55f132cce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c6dd0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 0x55f132ccea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c6dd0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f132d2f080 con 0x55f132cc8750 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c6dd0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f132d2f1c0 con 0x55f132cbf250 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c6dd0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f132d2f300 con 0x55f132cbfb70 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c6dd0740 10 client.0 _ll_register_callbacks cb 0x7f08b80d1a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:31.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c1ec3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132ccdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c26c4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 0x55f132ccea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c1ec3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132ccdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:37550/0 (socket says 172.21.15.18:37550) 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c1ec3640 1 -- 172.21.15.18:0/205998424 learned_addr learned my addr 172.21.15.18:0/205998424 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c16c2640 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbfb70 0x55f132cce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c1ec3640 1 -- 172.21.15.18:0/205998424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbfb70 msgr2=0x55f132cce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c1ec3640 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbfb70 0x55f132cce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c1ec3640 1 -- 172.21.15.18:0/205998424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 msgr2=0x55f132ccea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c1ec3640 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 0x55f132ccea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c1ec3640 1 -- 172.21.15.18:0/205998424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc10d710 con 0x55f132cbf250 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c26c4640 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 0x55f132ccea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c1ec3640 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132ccdff0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f08b8098ff0 tx=0x7f08b8010f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bf49daa6164832c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08a57fa640 1 -- 172.21.15.18:0/205998424 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b809ab10 con 0x55f132cbf250 2024-02-17T12:53:31.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.754+0000 7f08c6dd0740 1 -- 172.21.15.18:0/205998424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08b809a880 con 0x55f132cbf250 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08a57fa640 1 -- 172.21.15.18:0/205998424 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f08b809ad40 con 0x55f132cbf250 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08a57fa640 1 -- 172.21.15.18:0/205998424 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08b809a6a0 con 0x55f132cbf250 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08c6dd0740 10 client.26243 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08c6dd0740 1 -- 172.21.15.18:0/205998424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f132be78c0 con 0x55f132cbf250 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08a57fa640 1 -- 172.21.15.18:0/205998424 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08b809a880 con 0x55f132cbf250 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08a57fa640 1 -- 172.21.15.18:0/205998424 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f08b80973b0 con 0x55f132cbf250 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08c6dd0740 10 client.26243 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08c6dd0740 10 client.26243 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08c6dd0740 20 client.26243 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55f132d30860 0x55f132d32c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08c6dd0740 4 client.26243 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08c6dd0740 1 -- 172.21.15.18:0/205998424 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f1329b2890 con 0x55f132d30860 2024-02-17T12:53:31.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.755+0000 7f08c16c2640 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55f132d30860 0x55f132d32c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c16c2640 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55f132d30860 0x55f132d32c80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fbcbcc12125a4944 server_cookie=b07247dafbd0987c in_seq=0 out_seq=0 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08a57fa640 10 client.26243 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08a57fa640 1 -- 172.21.15.18:0/205998424 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f1329b2890 con 0x55f132d30860 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08a57fa640 10 client.26243 handle_command_reply: tid=0 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c6dd0740 1 client.26243 shutdown 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c6dd0740 1 -- 172.21.15.18:0/205998424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 msgr2=0x55f132ccdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132ccdff0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f08b8098ff0 tx=0x7f08b8010f00 comp rx=0 tx=0).stop 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c6dd0740 1 -- 172.21.15.18:0/205998424 shutdown_connections 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55f132d30860 0x55f132d32c80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f132cc8750 0x55f132ccea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08a57fa640 0 client.26243 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f132cbfb70 0x55f132cce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c6dd0740 1 --2- 172.21.15.18:0/205998424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f132cbf250 0x55f132ccdff0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c6dd0740 1 -- 172.21.15.18:0/205998424 >> 172.21.15.18:0/205998424 conn(0x55f132cb9980 msgr2=0x55f132cb82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c6dd0740 1 -- 172.21.15.18:0/205998424 shutdown_connections 2024-02-17T12:53:31.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c6dd0740 1 -- 172.21.15.18:0/205998424 wait complete. 2024-02-17T12:53:31.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.756+0000 7f08c6dd0740 20 client.26243 trim_cache size 0 max 16384 2024-02-17T12:53:31.754 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:53:31.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.757+0000 7f08c4127640 1 -- 172.21.15.18:0/1388950220 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f08a80a22e0 msgr2=0x7f08a80a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.757+0000 7f08c4127640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f08a80a22e0 0x7f08a80a47a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f08b4005cc0 tx=0x7f08b40095c0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.757+0000 7f08c4127640 1 -- 172.21.15.18:0/1388950220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08bc100260 msgr2=0x7f08bc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.757+0000 7f08c4127640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08bc100260 0x7f08bc101cf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f08b800e9f0 tx=0x7f08b800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.757+0000 7f08c4127640 1 -- 172.21.15.18:0/1388950220 shutdown_connections 2024-02-17T12:53:31.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.757+0000 7f08c4127640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f08a80a22e0 0x7f08a80a47a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.757+0000 7f08c4127640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08bc102770 0x7f08bc1a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.757+0000 7f08c4127640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08bc105330 0x7f08bc102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.757+0000 7f08c4127640 1 --2- 172.21.15.18:0/1388950220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08bc100260 0x7f08bc101cf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.757+0000 7f08c4127640 1 -- 172.21.15.18:0/1388950220 >> 172.21.15.18:0/1388950220 conn(0x7f08bc077180 msgr2=0x7f08bc103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.757+0000 7f08c4127640 1 -- 172.21.15.18:0/1388950220 shutdown_connections 2024-02-17T12:53:31.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.757+0000 7f08c4127640 1 -- 172.21.15.18:0/1388950220 wait complete. 2024-02-17T12:53:31.759 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:53:31.759 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:53:31.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:31 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:53:31.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:31 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1474526647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:53:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.876+0000 7fe0a319f640 1 -- 172.21.15.18:0/227309767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c100b00 msgr2=0x7fe09c104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.876+0000 7fe0a319f640 1 --2- 172.21.15.18:0/227309767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c100b00 0x7fe09c104b40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009900 tx=0x7fe08c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:53:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.877+0000 7fe0a319f640 1 -- 172.21.15.18:0/227309767 shutdown_connections 2024-02-17T12:53:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.877+0000 7fe0a319f640 1 --2- 172.21.15.18:0/227309767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c105220 0x7fe09c10cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.877+0000 7fe0a319f640 1 --2- 172.21.15.18:0/227309767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c100b00 0x7fe09c104b40 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.877+0000 7fe0a319f640 1 --2- 172.21.15.18:0/227309767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c100150 0x7fe09c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.877+0000 7fe0a319f640 1 -- 172.21.15.18:0/227309767 >> 172.21.15.18:0/227309767 conn(0x7fe09c0fb9d0 msgr2=0x7fe09c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.877+0000 7fe0a319f640 1 -- 172.21.15.18:0/227309767 shutdown_connections 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.877+0000 7fe0a319f640 1 -- 172.21.15.18:0/227309767 wait complete. 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.877+0000 7fe0a319f640 1 Processor -- start 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.877+0000 7fe0a319f640 1 -- start start 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a319f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c100150 0x7fe09c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a319f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c105220 0x7fe09c197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a319f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c19bfc0 0x7fe09c19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a319f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c19c9b0 con 0x7fe09c105220 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a319f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c19cb20 con 0x7fe09c19bfc0 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a319f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c19cc90 con 0x7fe09c100150 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a199c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c105220 0x7fe09c197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a199c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c105220 0x7fe09c197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60612/0 (socket says 172.21.15.18:60612) 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a199c640 1 -- 172.21.15.18:0/2048884781 learned_addr learned my addr 172.21.15.18:0/2048884781 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a219d640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c100150 0x7fe09c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a299e640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c19bfc0 0x7fe09c19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a199c640 1 -- 172.21.15.18:0/2048884781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c100150 msgr2=0x7fe09c197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a199c640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c100150 0x7fe09c197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a199c640 1 -- 172.21.15.18:0/2048884781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c19bfc0 msgr2=0x7fe09c19c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a199c640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c19bfc0 0x7fe09c19c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a199c640 1 -- 172.21.15.18:0/2048884781 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c009560 con 0x7fe09c105220 2024-02-17T12:53:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a219d640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c100150 0x7fe09c197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a199c640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c105220 0x7fe09c197c90 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe08c036040 tx=0x7fe08c02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0937fe640 1 -- 172.21.15.18:0/2048884781 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c004350 con 0x7fe09c105220 2024-02-17T12:53:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0a319f640 1 -- 172.21.15.18:0/2048884781 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09c10d470 con 0x7fe09c105220 2024-02-17T12:53:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0937fe640 1 -- 172.21.15.18:0/2048884781 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe08c0044b0 con 0x7fe09c105220 2024-02-17T12:53:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.878+0000 7fe0937fe640 1 -- 172.21.15.18:0/2048884781 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c003da0 con 0x7fe09c105220 2024-02-17T12:53:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.879+0000 7fe0a319f640 1 -- 172.21.15.18:0/2048884781 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c10d980 con 0x7fe09c105220 2024-02-17T12:53:31.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.881+0000 7fe0937fe640 1 -- 172.21.15.18:0/2048884781 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe08c004df0 con 0x7fe09c105220 2024-02-17T12:53:31.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.881+0000 7fe0a6fcb740 1 Processor -- start 2024-02-17T12:53:31.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.881+0000 7fe0937fe640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0880a2230 0x7fe0880a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.881+0000 7fe0937fe640 1 -- 172.21.15.18:0/2048884781 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe08c0effd0 con 0x7fe09c105220 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.881+0000 7fe0a299e640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0880a2230 0x7fe0880a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.881+0000 7fe0a6fcb740 1 -- start start 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a6fcb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 0x55dcafd86820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a6fcb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd8f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a6fcb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd8f940 0x55dcafd91d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a6fcb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcafd41960 con 0x55dcafd86440 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a6fcb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcafcf5310 con 0x55dcafd8f940 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a6fcb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcafcf2840 con 0x55dcafd86d60 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a219d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 0x55dcafd86820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a219d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 0x55dcafd86820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60620/0 (socket says 172.21.15.18:60620) 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a219d640 1 -- 172.21.15.18:0/262003852 learned_addr learned my addr 172.21.15.18:0/262003852 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a199c640 1 --2- 172.21.15.18:0/262003852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd8f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a299e640 1 --2- 172.21.15.18:0/262003852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd8f940 0x55dcafd91d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a299e640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0880a2230 0x7fe0880a46f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe094005fd0 tx=0x7fe094012040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a219d640 1 -- 172.21.15.18:0/262003852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 msgr2=0x55dcafd8f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a219d640 1 --2- 172.21.15.18:0/262003852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd8f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a219d640 1 -- 172.21.15.18:0/262003852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd8f940 msgr2=0x55dcafd91d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a219d640 1 --2- 172.21.15.18:0/262003852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd8f940 0x55dcafd91d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a219d640 1 -- 172.21.15.18:0/262003852 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcafcb0060 con 0x55dcafd86440 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a199c640 1 --2- 172.21.15.18:0/262003852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd8f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe0a219d640 1 --2- 172.21.15.18:0/262003852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 0x55dcafd86820 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe0980115f0 tx=0x7fe098011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac046e39fdb24867 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe090bfe640 1 -- 172.21.15.18:0/262003852 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09800f640 con 0x55dcafd86440 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe090bfe640 1 -- 172.21.15.18:0/262003852 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe09800fc80 con 0x55dcafd86440 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.882+0000 7fe090bfe640 1 -- 172.21.15.18:0/262003852 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098019cb0 con 0x55dcafd86440 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.883+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/262003852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 msgr2=0x55dcafd86820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.883+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/262003852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 0x55dcafd86820 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe0980115f0 tx=0x7fe098011ab0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.883+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/262003852 shutdown_connections 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.883+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/262003852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd8f940 0x55dcafd91d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.883+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/262003852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd8f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.883+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/262003852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 0x55dcafd86820 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.883+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/262003852 >> 172.21.15.18:0/262003852 conn(0x55dcafd80b70 msgr2=0x55dcafd80f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.883+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/262003852 shutdown_connections 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.883+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/262003852 wait complete. 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a6fcb740 1 -- start start 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a6fcb740 1 Processor -- start 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a6fcb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd86440 0x55dcafd8c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a6fcb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd8cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a6fcb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd8f940 0x55dcafd8d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a6fcb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcafcee020 con 0x55dcafd8f940 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a6fcb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcaf9c1430 con 0x55dcafd86440 2024-02-17T12:53:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a6fcb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcaf97de00 con 0x55dcafd86d60 2024-02-17T12:53:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a299e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd8f940 0x55dcafd8d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a6fcb740 10 client.0 _ll_register_callbacks cb 0x55dcafc95cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a219d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd86440 0x55dcafd8c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a299e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd8f940 0x55dcafd8d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60622/0 (socket says 172.21.15.18:60622) 2024-02-17T12:53:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a199c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd8cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a299e640 1 -- 172.21.15.18:0/3067305213 learned_addr learned my addr 172.21.15.18:0/3067305213 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a299e640 1 -- 172.21.15.18:0/3067305213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 msgr2=0x55dcafd8cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a299e640 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd8cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a299e640 1 -- 172.21.15.18:0/3067305213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd86440 msgr2=0x55dcafd8c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a299e640 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd86440 0x55dcafd8c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a299e640 1 -- 172.21.15.18:0/3067305213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe098009d40 con 0x55dcafd8f940 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a199c640 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd8cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.884+0000 7fe0a219d640 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd86440 0x55dcafd8c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe0a299e640 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd8f940 0x55dcafd8d440 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe094004240 tx=0x7fe094004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5c518b646b4bcd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe085ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe085ffb640 1 -- 172.21.15.18:0/3067305213 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09401d9f0 con 0x55dcafd8f940 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe085ffb640 1 -- 172.21.15.18:0/3067305213 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe09401db50 con 0x55dcafd8f940 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/3067305213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcaf9ed7e0 con 0x55dcafd8f940 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe085ffb640 1 -- 172.21.15.18:0/3067305213 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe09401dd10 con 0x55dcafd8f940 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe0a6fcb740 10 client.16887 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/3067305213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55dcafcec6a0 con 0x55dcafd8f940 2024-02-17T12:53:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe085ffb640 1 -- 172.21.15.18:0/3067305213 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dcaf9ed7e0 con 0x55dcafd8f940 2024-02-17T12:53:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe085ffb640 1 -- 172.21.15.18:0/3067305213 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe09400b4a0 con 0x55dcafd8f940 2024-02-17T12:53:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe0a6fcb740 10 client.16887 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe0a6fcb740 10 client.16887 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:53:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe0a6fcb740 20 client.16887 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55dcafd9feb0 0x55dcafda22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe0a6fcb740 4 client.16887 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.885+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/3067305213 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcafa2a720 con 0x55dcafd9feb0 2024-02-17T12:53:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.886+0000 7fe0a219d640 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55dcafd9feb0 0x55dcafda22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.886+0000 7fe0a219d640 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55dcafd9feb0 0x55dcafda22d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec751ac0468dc02e server_cookie=9aa7effe245d0540 in_seq=0 out_seq=0 2024-02-17T12:53:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.886+0000 7fe085ffb640 10 client.16887 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe085ffb640 1 -- 172.21.15.18:0/3067305213 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dcafa2a720 con 0x55dcafd9feb0 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe085ffb640 10 client.16887 handle_command_reply: tid=0 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe0a6fcb740 1 client.16887 shutdown 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/3067305213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd8f940 msgr2=0x55dcafd8d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd8f940 0x55dcafd8d440 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe094004240 tx=0x7fe094004a40 comp rx=0 tx=0).stop 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/3067305213 shutdown_connections 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55dcafd9feb0 0x55dcafda22d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd8f940 0x55dcafd8d440 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd8cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe085ffb640 0 client.16887 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/3067305213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd86440 0x55dcafd8c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/3067305213 >> 172.21.15.18:0/3067305213 conn(0x55dcafd80b70 msgr2=0x55dcafd828c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/3067305213 shutdown_connections 2024-02-17T12:53:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/3067305213 wait complete. 2024-02-17T12:53:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.887+0000 7fe0a6fcb740 20 client.16887 trim_cache size 0 max 16384 2024-02-17T12:53:31.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.900+0000 7fe0a6fcb740 1 Processor -- start 2024-02-17T12:53:31.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.900+0000 7fe0a6fcb740 1 -- start start 2024-02-17T12:53:31.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.900+0000 7fe0a6fcb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd86440 0x55dcafd8f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.900+0000 7fe0a6fcb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86d60 0x55dcafd8d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.900+0000 7fe0a6fcb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd8f940 0x55dcafd89c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.900+0000 7fe0a6fcb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcafc73130 con 0x55dcafd86d60 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.900+0000 7fe0a6fcb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcafd81b70 con 0x55dcafd86440 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.900+0000 7fe0a6fcb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcafd85590 con 0x55dcafd8f940 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe0a199c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86d60 0x55dcafd8d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe0a199c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86d60 0x55dcafd8d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60628/0 (socket says 172.21.15.18:60628) 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe0a299e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd8f940 0x55dcafd89c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe0a199c640 1 -- 172.21.15.18:0/2541153488 learned_addr learned my addr 172.21.15.18:0/2541153488 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe0a219d640 1 --2- 172.21.15.18:0/2541153488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd86440 0x55dcafd8f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe0a199c640 1 -- 172.21.15.18:0/2541153488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd8f940 msgr2=0x55dcafd89c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe0a199c640 1 --2- 172.21.15.18:0/2541153488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd8f940 0x55dcafd89c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe0a199c640 1 -- 172.21.15.18:0/2541153488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd86440 msgr2=0x55dcafd8f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe0a199c640 1 --2- 172.21.15.18:0/2541153488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd86440 0x55dcafd8f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe0a199c640 1 -- 172.21.15.18:0/2541153488 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcafc88c20 con 0x55dcafd86d60 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe0a299e640 1 --2- 172.21.15.18:0/2541153488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd8f940 0x55dcafd89c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe0a199c640 1 --2- 172.21.15.18:0/2541153488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86d60 0x55dcafd8d260 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0b9010 tx=0x7fe08c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6bdb79687e0305 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe087fff640 1 -- 172.21.15.18:0/2541153488 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0bcbb0 con 0x55dcafd86d60 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe087fff640 1 -- 172.21.15.18:0/2541153488 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe08c0c0d00 con 0x55dcafd86d60 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.901+0000 7fe087fff640 1 -- 172.21.15.18:0/2541153488 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0c4e30 con 0x55dcafd86d60 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.902+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/2541153488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86d60 msgr2=0x55dcafd8d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.902+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/2541153488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86d60 0x55dcafd8d260 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0b9010 tx=0x7fe08c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.902+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/2541153488 shutdown_connections 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.902+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/2541153488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd8f940 0x55dcafd89c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.902+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/2541153488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86d60 0x55dcafd8d260 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.902+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/2541153488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd86440 0x55dcafd8f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.902+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/2541153488 >> 172.21.15.18:0/2541153488 conn(0x55dcafd80b70 msgr2=0x55dcafd8b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.902+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/2541153488 shutdown_connections 2024-02-17T12:53:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.902+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/2541153488 wait complete. 2024-02-17T12:53:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.902+0000 7fe0a6fcb740 1 -- start start 2024-02-17T12:53:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.902+0000 7fe0a6fcb740 1 Processor -- start 2024-02-17T12:53:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a6fcb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 0x55dcafd968e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a6fcb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd94d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a6fcb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd8f940 0x55dcafd952b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a6fcb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcafd957f0 con 0x55dcafd86440 2024-02-17T12:53:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a6fcb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcafd95960 con 0x55dcafd8f940 2024-02-17T12:53:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a6fcb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcafd95b20 con 0x55dcafd86d60 2024-02-17T12:53:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a219d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 0x55dcafd968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a219d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 0x55dcafd968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60632/0 (socket says 172.21.15.18:60632) 2024-02-17T12:53:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a219d640 1 -- 172.21.15.18:0/393669374 learned_addr learned my addr 172.21.15.18:0/393669374 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a6fcb740 10 client.0 _ll_register_callbacks cb 0x7fe098009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a199c640 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd94d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a299e640 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd8f940 0x55dcafd952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a219d640 1 -- 172.21.15.18:0/393669374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 msgr2=0x55dcafd94d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a219d640 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd94d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a219d640 1 -- 172.21.15.18:0/393669374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd8f940 msgr2=0x55dcafd952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a219d640 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd8f940 0x55dcafd952b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a219d640 1 -- 172.21.15.18:0/393669374 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dcafcec6a0 con 0x55dcafd86440 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a219d640 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 0x55dcafd968e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe098005fd0 tx=0x7fe09800ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2294bb879af53270 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe085ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe085ffb640 1 -- 172.21.15.18:0/393669374 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09801ae10 con 0x55dcafd86440 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe085ffb640 1 -- 172.21.15.18:0/393669374 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe0980199c0 con 0x55dcafd86440 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.903+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/393669374 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe09800fc80 con 0x55dcafd86440 2024-02-17T12:53:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe085ffb640 1 -- 172.21.15.18:0/393669374 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe098019b80 con 0x55dcafd86440 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe0a6fcb740 10 client.16893 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/393669374 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fe09c10d470 con 0x55dcafd86440 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe085ffb640 1 -- 172.21.15.18:0/393669374 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe09800fc80 con 0x55dcafd86440 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe085ffb640 1 -- 172.21.15.18:0/393669374 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe0980053b0 con 0x55dcafd86440 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe0a6fcb740 10 client.16893 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe0a6fcb740 10 client.16893 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe0a6fcb740 20 client.16893 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55dcafd9feb0 0x55dcafdf9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe0a6fcb740 4 client.16893 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/393669374 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dcafa79890 con 0x55dcafd9feb0 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe0a199c640 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55dcafd9feb0 0x55dcafdf9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe0a199c640 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55dcafd9feb0 0x55dcafdf9820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84c2e84e1d3cb9c server_cookie=e0a098819c7a127b in_seq=0 out_seq=0 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.904+0000 7fe085ffb640 10 client.16893 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:53:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe085ffb640 1 -- 172.21.15.18:0/393669374 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dcafa79890 con 0x55dcafd9feb0 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe085ffb640 10 client.16893 handle_command_reply: tid=0 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a6fcb740 1 client.16893 shutdown 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/393669374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 msgr2=0x55dcafd968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 0x55dcafd968e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe098005fd0 tx=0x7fe09800ffc0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/393669374 shutdown_connections 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55dcafd9feb0 0x55dcafdf9820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcafd8f940 0x55dcafd952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe085ffb640 0 client.16893 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dcafd86d60 0x55dcafd94d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a6fcb740 1 --2- 172.21.15.18:0/393669374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcafd86440 0x55dcafd968e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/393669374 >> 172.21.15.18:0/393669374 conn(0x55dcafd80b70 msgr2=0x55dcafd7fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/393669374 shutdown_connections 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a6fcb740 1 -- 172.21.15.18:0/393669374 wait complete. 2024-02-17T12:53:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a6fcb740 20 client.16893 trim_cache size 0 max 16384 2024-02-17T12:53:31.904 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:53:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a319f640 1 -- 172.21.15.18:0/2048884781 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0880a2230 msgr2=0x7fe0880a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a319f640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0880a2230 0x7fe0880a46f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe094005fd0 tx=0x7fe094012040 comp rx=0 tx=0).stop 2024-02-17T12:53:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a319f640 1 -- 172.21.15.18:0/2048884781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c105220 msgr2=0x7fe09c197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a319f640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c105220 0x7fe09c197c90 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe08c036040 tx=0x7fe08c02c680 comp rx=0 tx=0).stop 2024-02-17T12:53:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a319f640 1 -- 172.21.15.18:0/2048884781 shutdown_connections 2024-02-17T12:53:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a319f640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0880a2230 0x7fe0880a46f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a319f640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c19bfc0 0x7fe09c19c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a319f640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c105220 0x7fe09c197c90 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a319f640 1 --2- 172.21.15.18:0/2048884781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c100150 0x7fe09c197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.905+0000 7fe0a319f640 1 -- 172.21.15.18:0/2048884781 >> 172.21.15.18:0/2048884781 conn(0x7fe09c0fb9d0 msgr2=0x7fe09c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.906+0000 7fe0a319f640 1 -- 172.21.15.18:0/2048884781 shutdown_connections 2024-02-17T12:53:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:31.906+0000 7fe0a319f640 1 -- 172.21.15.18:0/2048884781 wait complete. 2024-02-17T12:53:31.907 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:53:31.908 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:53:32.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.026+0000 7f570410d640 1 -- 172.21.15.18:0/215546567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56fc100c10 msgr2=0x7f56fc104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.026+0000 7f570410d640 1 --2- 172.21.15.18:0/215546567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56fc100c10 0x7f56fc104c50 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009930 tx=0x7f56ec02bc30 comp rx=0 tx=0).stop 2024-02-17T12:53:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.026+0000 7f570410d640 1 -- 172.21.15.18:0/215546567 shutdown_connections 2024-02-17T12:53:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.026+0000 7f570410d640 1 --2- 172.21.15.18:0/215546567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56fc105330 0x7f56fc10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.026+0000 7f570410d640 1 --2- 172.21.15.18:0/215546567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56fc100c10 0x7f56fc104c50 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.026+0000 7f570410d640 1 --2- 172.21.15.18:0/215546567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56fc100260 0x7f56fc100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.026+0000 7f570410d640 1 -- 172.21.15.18:0/215546567 >> 172.21.15.18:0/215546567 conn(0x7f56fc077180 msgr2=0x7f56fc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.026+0000 7f570410d640 1 -- 172.21.15.18:0/215546567 shutdown_connections 2024-02-17T12:53:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.026+0000 7f570410d640 1 -- 172.21.15.18:0/215546567 wait complete. 2024-02-17T12:53:32.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f570410d640 1 Processor -- start 2024-02-17T12:53:32.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f570410d640 1 -- start start 2024-02-17T12:53:32.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f570410d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56fc100260 0x7f56fc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f570410d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56fc105330 0x7f56fc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f570410d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56fc199100 0x7f56fc1a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f570410d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc1a3d50 con 0x7f56fc105330 2024-02-17T12:53:32.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f570410d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc1a3ec0 con 0x7f56fc100260 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f570410d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc1a4080 con 0x7f56fc199100 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f57016a8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56fc105330 0x7f56fc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f57016a8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56fc105330 0x7f56fc198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60654/0 (socket says 172.21.15.18:60654) 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f5701ea9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56fc100260 0x7f56fc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f57016a8640 1 -- 172.21.15.18:0/871063222 learned_addr learned my addr 172.21.15.18:0/871063222 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.027+0000 7f57026aa640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56fc199100 0x7f56fc1a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.028+0000 7f57016a8640 1 -- 172.21.15.18:0/871063222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56fc199100 msgr2=0x7f56fc1a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.028+0000 7f57016a8640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56fc199100 0x7f56fc1a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.028+0000 7f57016a8640 1 -- 172.21.15.18:0/871063222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56fc100260 msgr2=0x7f56fc1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.028+0000 7f57016a8640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56fc100260 0x7f56fc1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.028+0000 7f57016a8640 1 -- 172.21.15.18:0/871063222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec009560 con 0x7f56fc105330 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.028+0000 7f5701ea9640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56fc100260 0x7f56fc1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.028+0000 7f57016a8640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56fc105330 0x7f56fc198b20 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f56ec005ab0 tx=0x7f56ec02c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.028+0000 7f56f2ffd640 1 -- 172.21.15.18:0/871063222 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec02ca60 con 0x7f56fc105330 2024-02-17T12:53:32.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.028+0000 7f56f2ffd640 1 -- 172.21.15.18:0/871063222 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56ec02cbc0 con 0x7f56fc105330 2024-02-17T12:53:32.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.028+0000 7f56f2ffd640 1 -- 172.21.15.18:0/871063222 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0035f0 con 0x7f56fc105330 2024-02-17T12:53:32.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.028+0000 7f570410d640 1 -- 172.21.15.18:0/871063222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56fc1963d0 con 0x7f56fc105330 2024-02-17T12:53:32.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.028+0000 7f570410d640 1 -- 172.21.15.18:0/871063222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc196860 con 0x7f56fc105330 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.031+0000 7f56f2ffd640 1 -- 172.21.15.18:0/871063222 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f56ec049070 con 0x7f56fc105330 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.031+0000 7f5706db9740 1 Processor -- start 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.031+0000 7f5706db9740 1 -- start start 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.031+0000 7f5706db9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720e250 0x5568c720e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.031+0000 7f5706db9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 0x5568c7217210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.031+0000 7f5706db9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 0x5568c7219b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.031+0000 7f5706db9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c71c9960 con 0x5568c720eb70 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.031+0000 7f5706db9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c717d310 con 0x5568c720e250 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.031+0000 7f5706db9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c717a840 con 0x5568c7217750 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f56f2ffd640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56e80a2230 0x7f56e80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f5701ea9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 0x5568c7217210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f5701ea9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 0x5568c7217210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60660/0 (socket says 172.21.15.18:60660) 2024-02-17T12:53:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f57026aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720e250 0x5568c720e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f57016a8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 0x5568c7219b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f5701ea9640 1 -- 172.21.15.18:0/3497929745 learned_addr learned my addr 172.21.15.18:0/3497929745 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f57026aa640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56e80a2230 0x7f56e80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f56f2ffd640 1 -- 172.21.15.18:0/871063222 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f56ec0f7bf0 con 0x7f56fc105330 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f5701ea9640 1 -- 172.21.15.18:0/3497929745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 msgr2=0x5568c7219b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f5701ea9640 1 --2- 172.21.15.18:0/3497929745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 0x5568c7219b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f5701ea9640 1 -- 172.21.15.18:0/3497929745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720e250 msgr2=0x5568c720e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f5701ea9640 1 --2- 172.21.15.18:0/3497929745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720e250 0x5568c720e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f5701ea9640 1 -- 172.21.15.18:0/3497929745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568c7138060 con 0x5568c720eb70 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f5701ea9640 1 --2- 172.21.15.18:0/3497929745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 0x5568c7217210 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f56f80106f0 tx=0x7f56f8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3e4b0520669239 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f57026aa640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56e80a2230 0x7f56e80a46f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56f4005fd0 tx=0x7f56f4009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f56e7fff640 1 -- 172.21.15.18:0/3497929745 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8018620 con 0x5568c720eb70 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f56e7fff640 1 -- 172.21.15.18:0/3497929745 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56f8018c60 con 0x5568c720eb70 2024-02-17T12:53:32.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.032+0000 7f56e7fff640 1 -- 172.21.15.18:0/3497929745 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f800ed70 con 0x5568c720eb70 2024-02-17T12:53:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.033+0000 7f5706db9740 1 -- 172.21.15.18:0/3497929745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 msgr2=0x5568c7217210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.033+0000 7f5706db9740 1 --2- 172.21.15.18:0/3497929745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 0x5568c7217210 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f56f80106f0 tx=0x7f56f8005f90 comp rx=0 tx=0).stop 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.033+0000 7f5706db9740 1 -- 172.21.15.18:0/3497929745 shutdown_connections 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.033+0000 7f5706db9740 1 --2- 172.21.15.18:0/3497929745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 0x5568c7219b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.033+0000 7f5706db9740 1 --2- 172.21.15.18:0/3497929745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 0x5568c7217210 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.033+0000 7f5706db9740 1 --2- 172.21.15.18:0/3497929745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720e250 0x5568c720e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.033+0000 7f5706db9740 1 -- 172.21.15.18:0/3497929745 >> 172.21.15.18:0/3497929745 conn(0x5568c7208980 msgr2=0x5568c7208d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.033+0000 7f5706db9740 1 -- 172.21.15.18:0/3497929745 shutdown_connections 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.033+0000 7f5706db9740 1 -- 172.21.15.18:0/3497929745 wait complete. 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f5706db9740 1 -- start start 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f5706db9740 1 Processor -- start 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f5706db9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720e250 0x5568c7214710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f5706db9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 0x5568c7214c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f5706db9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 0x5568c7215190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f5706db9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c7176020 con 0x5568c720eb70 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f5706db9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c6e49430 con 0x5568c720e250 2024-02-17T12:53:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f5706db9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c6e05e00 con 0x5568c7217750 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f5706db9740 10 client.0 _ll_register_callbacks cb 0x5568c711dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f57016a8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 0x5568c7214c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f57016a8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 0x5568c7214c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60674/0 (socket says 172.21.15.18:60674) 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f57016a8640 1 -- 172.21.15.18:0/2296622112 learned_addr learned my addr 172.21.15.18:0/2296622112 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f57026aa640 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 0x5568c7215190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f5701ea9640 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720e250 0x5568c7214710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f57016a8640 1 -- 172.21.15.18:0/2296622112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 msgr2=0x5568c7215190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f57016a8640 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 0x5568c7215190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f57016a8640 1 -- 172.21.15.18:0/2296622112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720e250 msgr2=0x5568c7214710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f57016a8640 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720e250 0x5568c7214710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.034+0000 7f57016a8640 1 -- 172.21.15.18:0/2296622112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f80103d0 con 0x5568c720eb70 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.035+0000 7f57016a8640 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 0x5568c7214c50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0f5990 tx=0x7f56ec0967b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9716c79a54e94ad server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.035+0000 7f56e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.035+0000 7f56e57fa640 1 -- 172.21.15.18:0/2296622112 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0c4150 con 0x5568c720eb70 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.035+0000 7f56e57fa640 1 -- 172.21.15.18:0/2296622112 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f56ec0c47f0 con 0x5568c720eb70 2024-02-17T12:53:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.035+0000 7f5706db9740 1 -- 172.21.15.18:0/2296622112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568c6e757e0 con 0x5568c720eb70 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.035+0000 7f56e57fa640 1 -- 172.21.15.18:0/2296622112 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56ec0c8890 con 0x5568c720eb70 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.035+0000 7f5706db9740 10 client.16905 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.035+0000 7f5706db9740 1 -- 172.21.15.18:0/2296622112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5568c71746a0 con 0x5568c720eb70 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.035+0000 7f56e57fa640 1 -- 172.21.15.18:0/2296622112 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5568c6e757e0 con 0x5568c720eb70 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.035+0000 7f56e57fa640 1 -- 172.21.15.18:0/2296622112 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f56ec0e40b0 con 0x5568c720eb70 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.036+0000 7f5706db9740 10 client.16905 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.036+0000 7f5706db9740 10 client.16905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.036+0000 7f5706db9740 20 client.16905 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.036+0000 7f5706db9740 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5568c7227720 0x5568c7229b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.036+0000 7f5706db9740 4 client.16905 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.036+0000 7f5706db9740 1 -- 172.21.15.18:0/2296622112 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568c6eb2720 con 0x5568c7227720 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.036+0000 7f5701ea9640 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5568c7227720 0x5568c7229b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.036+0000 7f56e57fa640 10 client.16905 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.036+0000 7f5701ea9640 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5568c7227720 0x5568c7229b40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab323b63b0ebd20e server_cookie=f2d6b3e51021fa6c in_seq=0 out_seq=0 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f56e57fa640 1 -- 172.21.15.18:0/2296622112 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5568c6eb2720 con 0x5568c7227720 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f56e57fa640 10 client.16905 handle_command_reply: tid=0 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f5706db9740 1 client.16905 shutdown 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f5706db9740 1 -- 172.21.15.18:0/2296622112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 msgr2=0x5568c7214c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f5706db9740 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 0x5568c7214c50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0f5990 tx=0x7f56ec0967b0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f5706db9740 1 -- 172.21.15.18:0/2296622112 shutdown_connections 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f5706db9740 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5568c7227720 0x5568c7229b40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f5706db9740 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 0x5568c7215190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f5706db9740 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720eb70 0x5568c7214c50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f5706db9740 1 --2- 172.21.15.18:0/2296622112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720e250 0x5568c7214710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f56e57fa640 0 client.16905 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f5706db9740 1 -- 172.21.15.18:0/2296622112 >> 172.21.15.18:0/2296622112 conn(0x5568c7208980 msgr2=0x5568c720a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f5706db9740 1 -- 172.21.15.18:0/2296622112 shutdown_connections 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f5706db9740 1 -- 172.21.15.18:0/2296622112 wait complete. 2024-02-17T12:53:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.037+0000 7f5706db9740 20 client.16905 trim_cache size 0 max 16384 2024-02-17T12:53:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5706db9740 1 Processor -- start 2024-02-17T12:53:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5706db9740 1 -- start start 2024-02-17T12:53:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5706db9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720e250 0x5568c7216e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5706db9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720eb70 0x5568c7214fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5706db9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 0x5568c7211970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5706db9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c70fb130 con 0x5568c720e250 2024-02-17T12:53:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5706db9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c720deb0 con 0x5568c720eb70 2024-02-17T12:53:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5706db9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c7209bb0 con 0x5568c7217750 2024-02-17T12:53:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5701ea9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720e250 0x5568c7216e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5701ea9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720e250 0x5568c7216e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60688/0 (socket says 172.21.15.18:60688) 2024-02-17T12:53:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5701ea9640 1 -- 172.21.15.18:0/1307303081 learned_addr learned my addr 172.21.15.18:0/1307303081 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f57026aa640 1 --2- 172.21.15.18:0/1307303081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 0x5568c7211970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f57016a8640 1 --2- 172.21.15.18:0/1307303081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720eb70 0x5568c7214fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5701ea9640 1 -- 172.21.15.18:0/1307303081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 msgr2=0x5568c7211970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5701ea9640 1 --2- 172.21.15.18:0/1307303081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 0x5568c7211970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5701ea9640 1 -- 172.21.15.18:0/1307303081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720eb70 msgr2=0x5568c7214fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5701ea9640 1 --2- 172.21.15.18:0/1307303081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720eb70 0x5568c7214fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.050+0000 7f5701ea9640 1 -- 172.21.15.18:0/1307303081 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568c7110c20 con 0x5568c720e250 2024-02-17T12:53:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f5701ea9640 1 --2- 172.21.15.18:0/1307303081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720e250 0x5568c7216e40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f56f80106f0 tx=0x7f56f8010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bdb3cfd3dad4fb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f56e77fe640 1 -- 172.21.15.18:0/1307303081 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8027040 con 0x5568c720e250 2024-02-17T12:53:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f56e77fe640 1 -- 172.21.15.18:0/1307303081 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56f800e830 con 0x5568c720e250 2024-02-17T12:53:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f56e77fe640 1 -- 172.21.15.18:0/1307303081 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f802ba90 con 0x5568c720e250 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f5706db9740 1 -- 172.21.15.18:0/1307303081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720e250 msgr2=0x5568c7216e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f5706db9740 1 --2- 172.21.15.18:0/1307303081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720e250 0x5568c7216e40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f56f80106f0 tx=0x7f56f8010df0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f5706db9740 1 -- 172.21.15.18:0/1307303081 shutdown_connections 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f5706db9740 1 --2- 172.21.15.18:0/1307303081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c7217750 0x5568c7211970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f5706db9740 1 --2- 172.21.15.18:0/1307303081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720eb70 0x5568c7214fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f5706db9740 1 --2- 172.21.15.18:0/1307303081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c720e250 0x5568c7216e40 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f5706db9740 1 -- 172.21.15.18:0/1307303081 >> 172.21.15.18:0/1307303081 conn(0x5568c7208980 msgr2=0x5568c722e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f5706db9740 1 -- 172.21.15.18:0/1307303081 shutdown_connections 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.051+0000 7f5706db9740 1 -- 172.21.15.18:0/1307303081 wait complete. 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f5706db9740 1 -- start start 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f5706db9740 1 Processor -- start 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f5706db9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c720e250 0x5568c721d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f5706db9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720eb70 0x5568c721d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f5706db9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c7217750 0x5568c721dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f5706db9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c727ddc0 con 0x5568c7217750 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f5706db9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c727df30 con 0x5568c720eb70 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f5706db9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c727e0f0 con 0x5568c720e250 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f5706db9740 10 client.0 _ll_register_callbacks cb 0x7f56f8010d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f57026aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c7217750 0x5568c721dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f57026aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c7217750 0x5568c721dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60696/0 (socket says 172.21.15.18:60696) 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f57026aa640 1 -- 172.21.15.18:0/2285669242 learned_addr learned my addr 172.21.15.18:0/2285669242 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f5701ea9640 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c720e250 0x5568c721d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.052+0000 7f57016a8640 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720eb70 0x5568c721d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f57026aa640 1 -- 172.21.15.18:0/2285669242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c720e250 msgr2=0x5568c721d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f57026aa640 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c720e250 0x5568c721d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f57026aa640 1 -- 172.21.15.18:0/2285669242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720eb70 msgr2=0x5568c721d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f57026aa640 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720eb70 0x5568c721d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f57026aa640 1 -- 172.21.15.18:0/2285669242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568c7138060 con 0x5568c7217750 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f57016a8640 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720eb70 0x5568c721d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f57026aa640 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c7217750 0x5568c721dcc0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f56f400fe70 tx=0x7f56f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e83823d5db5ef3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f56e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f56e57fa640 1 -- 172.21.15.18:0/2285669242 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4025040 con 0x5568c7217750 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f5706db9740 1 -- 172.21.15.18:0/2285669242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56f8018c60 con 0x5568c7217750 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f56e57fa640 1 -- 172.21.15.18:0/2285669242 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f56f4010e00 con 0x5568c7217750 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f56e57fa640 1 -- 172.21.15.18:0/2285669242 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56f4004b20 con 0x5568c7217750 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f5706db9740 10 client.16911 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f5706db9740 1 -- 172.21.15.18:0/2285669242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5568c71368c0 con 0x5568c7217750 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.053+0000 7f56e57fa640 1 -- 172.21.15.18:0/2285669242 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56f8018c60 con 0x5568c7217750 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.054+0000 7f56e57fa640 1 -- 172.21.15.18:0/2285669242 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f56f402a510 con 0x5568c7217750 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.054+0000 7f5706db9740 10 client.16911 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.054+0000 7f5706db9740 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.054+0000 7f5706db9740 20 client.16911 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.054+0000 7f5706db9740 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5568c727f9f0 0x5568c7281e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.054+0000 7f5706db9740 4 client.16911 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.054+0000 7f5706db9740 1 -- 172.21.15.18:0/2285669242 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5568c6f01890 con 0x5568c727f9f0 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.054+0000 7f5701ea9640 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5568c727f9f0 0x5568c7281e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.054+0000 7f56e57fa640 10 client.16911 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.054+0000 7f5701ea9640 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5568c727f9f0 0x5568c7281e10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ebf262e13ccf143 server_cookie=dcff0d9265af5706 in_seq=0 out_seq=0 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f56e57fa640 1 -- 172.21.15.18:0/2285669242 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5568c6f01890 con 0x5568c727f9f0 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f56e57fa640 10 client.16911 handle_command_reply: tid=0 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f5706db9740 1 client.16911 shutdown 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f5706db9740 1 -- 172.21.15.18:0/2285669242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c7217750 msgr2=0x5568c721dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f5706db9740 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c7217750 0x5568c721dcc0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f56f400fe70 tx=0x7f56f4004ab0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f5706db9740 1 -- 172.21.15.18:0/2285669242 shutdown_connections 2024-02-17T12:53:32.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f5706db9740 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5568c727f9f0 0x5568c7281e10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f5706db9740 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568c7217750 0x5568c721dcc0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f5706db9740 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5568c720eb70 0x5568c721d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f5706db9740 1 --2- 172.21.15.18:0/2285669242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5568c720e250 0x5568c721d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f56e57fa640 0 client.16911 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:32.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f5706db9740 1 -- 172.21.15.18:0/2285669242 >> 172.21.15.18:0/2285669242 conn(0x5568c7208980 msgr2=0x5568c72072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:32.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f5706db9740 1 -- 172.21.15.18:0/2285669242 shutdown_connections 2024-02-17T12:53:32.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f5706db9740 1 -- 172.21.15.18:0/2285669242 wait complete. 2024-02-17T12:53:32.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f5706db9740 20 client.16911 trim_cache size 0 max 16384 2024-02-17T12:53:32.052 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:53:32.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f570410d640 1 -- 172.21.15.18:0/871063222 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56e80a2230 msgr2=0x7f56e80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:32.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f570410d640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56e80a2230 0x7f56e80a46f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56f4005fd0 tx=0x7f56f4009500 comp rx=0 tx=0).stop 2024-02-17T12:53:32.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f570410d640 1 -- 172.21.15.18:0/871063222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56fc105330 msgr2=0x7f56fc198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:32.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.055+0000 7f570410d640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56fc105330 0x7f56fc198b20 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f56ec005ab0 tx=0x7f56ec02c5f0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.056+0000 7f570410d640 1 -- 172.21.15.18:0/871063222 shutdown_connections 2024-02-17T12:53:32.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.056+0000 7f570410d640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f56e80a2230 0x7f56e80a46f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.056+0000 7f570410d640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56fc199100 0x7f56fc1a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.056+0000 7f570410d640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56fc105330 0x7f56fc198b20 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.056+0000 7f570410d640 1 --2- 172.21.15.18:0/871063222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56fc100260 0x7f56fc1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:32.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.056+0000 7f570410d640 1 -- 172.21.15.18:0/871063222 >> 172.21.15.18:0/871063222 conn(0x7f56fc077180 msgr2=0x7f56fc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:32.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.056+0000 7f570410d640 1 -- 172.21.15.18:0/871063222 shutdown_connections 2024-02-17T12:53:32.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:32.056+0000 7f570410d640 1 -- 172.21.15.18:0/871063222 wait complete. 2024-02-17T12:53:32.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 201620 10.98 MB/sec execute 47 sec latency 10.313 ms 2024-02-17T12:53:32.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:32 smithi146 ceph-mon[36454]: pgmap v112: 97 pgs: 97 active+clean; 80 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2024-02-17T12:53:32.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:32 smithi018 ceph-mon[32625]: pgmap v112: 97 pgs: 97 active+clean; 80 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2024-02-17T12:53:32.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:32 smithi119 ceph-mon[36188]: pgmap v112: 97 pgs: 97 active+clean; 80 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2024-02-17T12:53:33.058 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:53:33.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:53:33.059 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:53:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.180+0000 7f462f064640 1 -- 172.21.15.18:0/2398174223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46281007d0 msgr2=0x7f4628100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.180+0000 7f462f064640 1 --2- 172.21.15.18:0/2398174223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46281007d0 0x7f4628100c30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f461c009900 tx=0x7f461c02bc80 comp rx=0 tx=0).stop 2024-02-17T12:53:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.180+0000 7f462f064640 1 -- 172.21.15.18:0/2398174223 shutdown_connections 2024-02-17T12:53:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.180+0000 7f462f064640 1 --2- 172.21.15.18:0/2398174223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4628101170 0x7f462810cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.180+0000 7f462f064640 1 --2- 172.21.15.18:0/2398174223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46281007d0 0x7f4628100c30 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.180+0000 7f462f064640 1 --2- 172.21.15.18:0/2398174223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4628105030 0x7f4628105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.180+0000 7f462f064640 1 -- 172.21.15.18:0/2398174223 >> 172.21.15.18:0/2398174223 conn(0x7f4628076640 msgr2=0x7f46280fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.180+0000 7f462f064640 1 -- 172.21.15.18:0/2398174223 shutdown_connections 2024-02-17T12:53:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.180+0000 7f462f064640 1 -- 172.21.15.18:0/2398174223 wait complete. 2024-02-17T12:53:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f462f064640 1 Processor -- start 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f462f064640 1 -- start start 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f462f064640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46281007d0 0x7f462819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f462f064640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4628101170 0x7f462819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f462f064640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4628105030 0x7f46281976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f462f064640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628197e50 con 0x7f4628101170 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f462f064640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628197fc0 con 0x7f46281007d0 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f462f064640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628198180 con 0x7f4628105030 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f4627fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4628101170 0x7f462819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f462ce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46281007d0 0x7f462819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f462ce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46281007d0 0x7f462819d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:37698/0 (socket says 172.21.15.18:37698) 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f4627fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4628101170 0x7f462819daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60708/0 (socket says 172.21.15.18:60708) 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.181+0000 7f462ce00640 1 -- 172.21.15.18:0/2395858677 learned_addr learned my addr 172.21.15.18:0/2395858677 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f462d601640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4628105030 0x7f46281976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f4627fff640 1 -- 172.21.15.18:0/2395858677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4628105030 msgr2=0x7f46281976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f4627fff640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4628105030 0x7f46281976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f4627fff640 1 -- 172.21.15.18:0/2395858677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46281007d0 msgr2=0x7f462819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f4627fff640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46281007d0 0x7f462819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f4627fff640 1 -- 172.21.15.18:0/2395858677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c009560 con 0x7f4628101170 2024-02-17T12:53:33.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f462d601640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4628105030 0x7f46281976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:53:33.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f4627fff640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4628101170 0x7f462819daf0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f461c036040 tx=0x7f461c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:33.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f4625ffb640 1 -- 172.21.15.18:0/2395858677 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c004440 con 0x7f4628101170 2024-02-17T12:53:33.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f4625ffb640 1 -- 172.21.15.18:0/2395858677 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f461c0052d0 con 0x7f4628101170 2024-02-17T12:53:33.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f462f064640 1 -- 172.21.15.18:0/2395858677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4628198400 con 0x7f4628101170 2024-02-17T12:53:33.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f4625ffb640 1 -- 172.21.15.18:0/2395858677 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c003da0 con 0x7f4628101170 2024-02-17T12:53:33.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.182+0000 7f462f064640 1 -- 172.21.15.18:0/2395858677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46281a4530 con 0x7f4628101170 2024-02-17T12:53:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f4625ffb640 1 -- 172.21.15.18:0/2395858677 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f461c005440 con 0x7f4628101170 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f4631c3e740 1 Processor -- start 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f4631c3e740 1 -- start start 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f4631c3e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4260 0x55c03d1b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f4631c3e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4c10 0x55c03d1bd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f4631c3e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1bdd10 0x55c03d1be170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f4631c3e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03d122f90 con 0x55c03d1bdd10 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f4631c3e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03d1204c0 con 0x55c03d1b4c10 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f4631c3e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03d11bca0 con 0x55c03d1b4260 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f4625ffb640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46100a2230 0x7f46100a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f4627fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1bdd10 0x55c03d1be170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f462ce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4c10 0x55c03d1bd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f462d601640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4260 0x55c03d1b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f462ce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4c10 0x55c03d1bd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:37702/0 (socket says 172.21.15.18:37702) 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f462ce00640 1 -- 172.21.15.18:0/1926614763 learned_addr learned my addr 172.21.15.18:0/1926614763 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.185+0000 7f462d601640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46100a2230 0x7f46100a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.186+0000 7f4625ffb640 1 -- 172.21.15.18:0/2395858677 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f461c0efff0 con 0x7f4628101170 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.186+0000 7f462ce00640 1 -- 172.21.15.18:0/1926614763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4260 msgr2=0x55c03d1b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.186+0000 7f462ce00640 1 --2- 172.21.15.18:0/1926614763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4260 0x55c03d1b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.186+0000 7f462ce00640 1 -- 172.21.15.18:0/1926614763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1bdd10 msgr2=0x55c03d1be170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.186+0000 7f462ce00640 1 --2- 172.21.15.18:0/1926614763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1bdd10 0x55c03d1be170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.186+0000 7f462ce00640 1 -- 172.21.15.18:0/1926614763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c03d0de0b0 con 0x55c03d1b4c10 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.186+0000 7f462d601640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46100a2230 0x7f46100a46f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4620009fd0 tx=0x7f46200094a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.186+0000 7f462ce00640 1 --2- 172.21.15.18:0/1926614763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4c10 0x55c03d1bd740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f461800b7e0 tx=0x7f461800bca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8767dbc1c97be0de server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.186+0000 7f460effd640 1 -- 172.21.15.18:0/1926614763 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618004130 con 0x55c03d1b4c10 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.186+0000 7f460effd640 1 -- 172.21.15.18:0/1926614763 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4618003dd0 con 0x55c03d1b4c10 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.186+0000 7f460effd640 1 -- 172.21.15.18:0/1926614763 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180108c0 con 0x55c03d1b4c10 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.187+0000 7f4631c3e740 1 -- 172.21.15.18:0/1926614763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4c10 msgr2=0x55c03d1bd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.187+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1926614763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4c10 0x55c03d1bd740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f461800b7e0 tx=0x7f461800bca0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.187+0000 7f4631c3e740 1 -- 172.21.15.18:0/1926614763 shutdown_connections 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.187+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1926614763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1bdd10 0x55c03d1be170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.187+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1926614763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4c10 0x55c03d1bd740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.187+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1926614763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4260 0x55c03d1b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.187+0000 7f4631c3e740 1 -- 172.21.15.18:0/1926614763 >> 172.21.15.18:0/1926614763 conn(0x55c03d1ae840 msgr2=0x55c03d1aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.187+0000 7f4631c3e740 1 -- 172.21.15.18:0/1926614763 shutdown_connections 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.187+0000 7f4631c3e740 1 -- 172.21.15.18:0/1926614763 wait complete. 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.187+0000 7f4631c3e740 1 -- start start 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.187+0000 7f4631c3e740 1 Processor -- start 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f4631c3e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4260 0x55c03d1c1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f4631c3e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4c10 0x55c03d1c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f4631c3e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1bdd10 0x55c03d1c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f4631c3e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03d0c0f40 con 0x55c03d1b4260 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f4631c3e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03cdef4b0 con 0x55c03d1bdd10 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f4631c3e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03d0a1670 con 0x55c03d1b4c10 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f4631c3e740 10 client.0 _ll_register_callbacks cb 0x55c03d10ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f462ce00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4260 0x55c03d1c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f462ce00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4260 0x55c03d1c1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60736/0 (socket says 172.21.15.18:60736) 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f462d601640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1bdd10 0x55c03d1c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f462ce00640 1 -- 172.21.15.18:0/448159758 learned_addr learned my addr 172.21.15.18:0/448159758 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f4627fff640 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4c10 0x55c03d1c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f462d601640 1 -- 172.21.15.18:0/448159758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4c10 msgr2=0x55c03d1c1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f462d601640 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4c10 0x55c03d1c1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f462d601640 1 -- 172.21.15.18:0/448159758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4260 msgr2=0x55c03d1c1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f462d601640 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4260 0x55c03d1c1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f462d601640 1 -- 172.21.15.18:0/448159758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4618004290 con 0x55c03d1bdd10 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.188+0000 7f462ce00640 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4260 0x55c03d1c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f462d601640 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1bdd10 0x55c03d1c21d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4620005fd0 tx=0x7f4620004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d15c893edf2130b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4607fff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4607fff640 1 -- 172.21.15.18:0/448159758 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620010ce0 con 0x55c03d1bdd10 2024-02-17T12:53:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4631c3e740 1 -- 172.21.15.18:0/448159758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c03d0f5f40 con 0x55c03d1bdd10 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4607fff640 1 -- 172.21.15.18:0/448159758 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4620010e40 con 0x55c03d1bdd10 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4607fff640 1 -- 172.21.15.18:0/448159758 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4620004b20 con 0x55c03d1bdd10 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4631c3e740 10 client.26291 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4631c3e740 1 -- 172.21.15.18:0/448159758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c03d11a320 con 0x55c03d1bdd10 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4607fff640 1 -- 172.21.15.18:0/448159758 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c03d0f5f40 con 0x55c03d1bdd10 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4607fff640 1 -- 172.21.15.18:0/448159758 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4620030050 con 0x55c03d1bdd10 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4631c3e740 10 client.26291 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4631c3e740 10 client.26291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4631c3e740 20 client.26291 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4631c3e740 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c03d1d22f0 0x55c03d1d4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4631c3e740 4 client.26291 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.189+0000 7f4631c3e740 1 -- 172.21.15.18:0/448159758 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c03d112310 con 0x55c03d1d22f0 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.190+0000 7f462ce00640 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c03d1d22f0 0x55c03d1d4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.190+0000 7f462ce00640 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c03d1d22f0 0x55c03d1d4710 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca8d6c436b4b3645 server_cookie=1f9d38b01775169a in_seq=0 out_seq=0 2024-02-17T12:53:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.190+0000 7f4607fff640 10 client.26291 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4607fff640 1 -- 172.21.15.18:0/448159758 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c03d112310 con 0x55c03d1d22f0 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4607fff640 10 client.26291 handle_command_reply: tid=0 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4631c3e740 1 client.26291 shutdown 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4631c3e740 1 -- 172.21.15.18:0/448159758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1bdd10 msgr2=0x55c03d1c21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4631c3e740 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1bdd10 0x55c03d1c21d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4620005fd0 tx=0x7f4620004a40 comp rx=0 tx=0).stop 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4631c3e740 1 -- 172.21.15.18:0/448159758 shutdown_connections 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4631c3e740 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c03d1d22f0 0x55c03d1d4710 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4631c3e740 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1bdd10 0x55c03d1c21d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4607fff640 0 client.26291 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4631c3e740 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4c10 0x55c03d1c1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4631c3e740 1 --2- 172.21.15.18:0/448159758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4260 0x55c03d1c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4631c3e740 1 -- 172.21.15.18:0/448159758 >> 172.21.15.18:0/448159758 conn(0x55c03d1ae840 msgr2=0x55c03d1b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4631c3e740 1 -- 172.21.15.18:0/448159758 shutdown_connections 2024-02-17T12:53:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4631c3e740 1 -- 172.21.15.18:0/448159758 wait complete. 2024-02-17T12:53:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.191+0000 7f4631c3e740 20 client.26291 trim_cache size 0 max 16384 2024-02-17T12:53:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f4631c3e740 1 Processor -- start 2024-02-17T12:53:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f4631c3e740 1 -- start start 2024-02-17T12:53:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f4631c3e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4260 0x55c03d1bf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f4631c3e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4c10 0x55c03d1c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f4631c3e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1bdd10 0x55c03d1c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f4631c3e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03d1af410 con 0x55c03d1b4c10 2024-02-17T12:53:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f4631c3e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03d1a8da0 con 0x55c03d1b4260 2024-02-17T12:53:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f4631c3e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03d1c41b0 con 0x55c03d1bdd10 2024-02-17T12:53:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f4627fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4c10 0x55c03d1c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f4627fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4c10 0x55c03d1c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60750/0 (socket says 172.21.15.18:60750) 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f4627fff640 1 -- 172.21.15.18:0/1203474561 learned_addr learned my addr 172.21.15.18:0/1203474561 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f462d601640 1 --2- 172.21.15.18:0/1203474561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1bdd10 0x55c03d1c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.209+0000 7f462ce00640 1 --2- 172.21.15.18:0/1203474561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4260 0x55c03d1bf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.210+0000 7f4627fff640 1 -- 172.21.15.18:0/1203474561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1bdd10 msgr2=0x55c03d1c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.210+0000 7f4627fff640 1 --2- 172.21.15.18:0/1203474561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1bdd10 0x55c03d1c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.210+0000 7f4627fff640 1 -- 172.21.15.18:0/1203474561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4260 msgr2=0x55c03d1bf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.210+0000 7f4627fff640 1 --2- 172.21.15.18:0/1203474561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4260 0x55c03d1bf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.210+0000 7f4627fff640 1 -- 172.21.15.18:0/1203474561 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c03d0b6c00 con 0x55c03d1b4c10 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.210+0000 7f4627fff640 1 --2- 172.21.15.18:0/1203474561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4c10 0x55c03d1c8db0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f461c02bcc0 tx=0x7f461c0e7440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfeb79bd996988c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.210+0000 7f460e7fc640 1 -- 172.21.15.18:0/1203474561 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c0bcb40 con 0x55c03d1b4c10 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.210+0000 7f460e7fc640 1 -- 172.21.15.18:0/1203474561 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f461c0c0c90 con 0x55c03d1b4c10 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.210+0000 7f460e7fc640 1 -- 172.21.15.18:0/1203474561 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c0edba0 con 0x55c03d1b4c10 2024-02-17T12:53:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.211+0000 7f4631c3e740 1 -- 172.21.15.18:0/1203474561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4c10 msgr2=0x55c03d1c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.211+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1203474561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4c10 0x55c03d1c8db0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f461c02bcc0 tx=0x7f461c0e7440 comp rx=0 tx=0).stop 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.211+0000 7f4631c3e740 1 -- 172.21.15.18:0/1203474561 shutdown_connections 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.211+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1203474561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1bdd10 0x55c03d1c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.211+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1203474561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1b4c10 0x55c03d1c8db0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.211+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1203474561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4260 0x55c03d1bf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.211+0000 7f4631c3e740 1 -- 172.21.15.18:0/1203474561 >> 172.21.15.18:0/1203474561 conn(0x55c03d1ae840 msgr2=0x55c03d1bba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.211+0000 7f4631c3e740 1 -- 172.21.15.18:0/1203474561 shutdown_connections 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.211+0000 7f4631c3e740 1 -- 172.21.15.18:0/1203474561 wait complete. 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.211+0000 7f4631c3e740 1 -- start start 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.211+0000 7f4631c3e740 1 Processor -- start 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f4631c3e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4260 0x55c03d221dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f4631c3e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4c10 0x55c03d222300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f4631c3e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1bdd10 0x55c03d21c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f4631c3e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03d21cd90 con 0x55c03d1bdd10 2024-02-17T12:53:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f4631c3e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03d21cf00 con 0x55c03d1b4c10 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f4631c3e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c03d21d0c0 con 0x55c03d1b4260 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f4631c3e740 10 client.0 _ll_register_callbacks cb 0x7f4618004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f462d601640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1bdd10 0x55c03d21c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f462d601640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1bdd10 0x55c03d21c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60766/0 (socket says 172.21.15.18:60766) 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f462ce00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4260 0x55c03d221dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f462d601640 1 -- 172.21.15.18:0/1850868804 learned_addr learned my addr 172.21.15.18:0/1850868804 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f4627fff640 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4c10 0x55c03d222300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f462ce00640 1 -- 172.21.15.18:0/1850868804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4c10 msgr2=0x55c03d222300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f462ce00640 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4c10 0x55c03d222300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f462ce00640 1 -- 172.21.15.18:0/1850868804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1bdd10 msgr2=0x55c03d21c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f462ce00640 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1bdd10 0x55c03d21c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f462ce00640 1 -- 172.21.15.18:0/1850868804 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46281a4530 con 0x55c03d1b4260 2024-02-17T12:53:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f462d601640 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1bdd10 0x55c03d21c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f462ce00640 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4260 0x55c03d221dc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46180039c0 tx=0x7f4618005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e74dfe79c5c77b04 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f4607fff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.212+0000 7f4607fff640 1 -- 172.21.15.18:0/1850868804 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618021040 con 0x55c03d1b4260 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4631c3e740 1 -- 172.21.15.18:0/1850868804 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c03d141900 con 0x55c03d1b4260 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4607fff640 1 -- 172.21.15.18:0/1850868804 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f461800b4a0 con 0x55c03d1b4260 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4607fff640 1 -- 172.21.15.18:0/1850868804 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4618029400 con 0x55c03d1b4260 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4631c3e740 10 client.26323 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4631c3e740 1 -- 172.21.15.18:0/1850868804 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c03d0dc910 con 0x55c03d1b4260 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4607fff640 1 -- 172.21.15.18:0/1850868804 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c03d141900 con 0x55c03d1b4260 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4607fff640 1 -- 172.21.15.18:0/1850868804 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f46180047a0 con 0x55c03d1b4260 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4631c3e740 10 client.26323 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4631c3e740 10 client.26323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4631c3e740 20 client.26323 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c03d229ce0 0x55c03d22c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4631c3e740 4 client.26323 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:53:33.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4631c3e740 1 -- 172.21.15.18:0/1850868804 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c03ce1b860 con 0x55c03d229ce0 2024-02-17T12:53:33.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.213+0000 7f4627fff640 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c03d229ce0 0x55c03d22c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.214+0000 7f4607fff640 10 client.26323 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:33.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.214+0000 7f4627fff640 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c03d229ce0 0x55c03d22c100 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e345a64258b7d8 server_cookie=acc642006518140f in_seq=0 out_seq=0 2024-02-17T12:53:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4607fff640 1 -- 172.21.15.18:0/1850868804 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c03ce1b860 con 0x55c03d229ce0 2024-02-17T12:53:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4607fff640 10 client.26323 handle_command_reply: tid=0 2024-02-17T12:53:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4631c3e740 1 client.26323 shutdown 2024-02-17T12:53:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4631c3e740 1 -- 172.21.15.18:0/1850868804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4260 msgr2=0x55c03d221dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4260 0x55c03d221dc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46180039c0 tx=0x7f4618005fb0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4631c3e740 1 -- 172.21.15.18:0/1850868804 shutdown_connections 2024-02-17T12:53:33.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c03d229ce0 0x55c03d22c100 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c03d1bdd10 0x55c03d21c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c03d1b4c10 0x55c03d222300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4631c3e740 1 --2- 172.21.15.18:0/1850868804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c03d1b4260 0x55c03d221dc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4631c3e740 1 -- 172.21.15.18:0/1850868804 >> 172.21.15.18:0/1850868804 conn(0x55c03d1ae840 msgr2=0x55c03d1bc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:33.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4607fff640 0 client.26323 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:33.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4631c3e740 1 -- 172.21.15.18:0/1850868804 shutdown_connections 2024-02-17T12:53:33.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4631c3e740 1 -- 172.21.15.18:0/1850868804 wait complete. 2024-02-17T12:53:33.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.215+0000 7f4631c3e740 20 client.26323 trim_cache size 0 max 16384 2024-02-17T12:53:33.212 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:53:33.212 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:53:33.213 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "35948641-239e-419c-8bdd-d5767dd49fd1", 2024-02-17T12:53:33.213 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:53:33.213 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:53:33.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.216+0000 7f462f064640 1 -- 172.21.15.18:0/2395858677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46100a2230 msgr2=0x7f46100a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.216+0000 7f462f064640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46100a2230 0x7f46100a46f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4620009fd0 tx=0x7f46200094a0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.216+0000 7f462f064640 1 -- 172.21.15.18:0/2395858677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4628101170 msgr2=0x7f462819daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.216+0000 7f462f064640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4628101170 0x7f462819daf0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f461c036040 tx=0x7f461c004040 comp rx=0 tx=0).stop 2024-02-17T12:53:33.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.216+0000 7f462f064640 1 -- 172.21.15.18:0/2395858677 shutdown_connections 2024-02-17T12:53:33.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.216+0000 7f462f064640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46100a2230 0x7f46100a46f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.216+0000 7f462f064640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4628105030 0x7f46281976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.216+0000 7f462f064640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4628101170 0x7f462819daf0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.216+0000 7f462f064640 1 --2- 172.21.15.18:0/2395858677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46281007d0 0x7f462819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.216+0000 7f462f064640 1 -- 172.21.15.18:0/2395858677 >> 172.21.15.18:0/2395858677 conn(0x7f4628076640 msgr2=0x7f46280ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:33.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.216+0000 7f462f064640 1 -- 172.21.15.18:0/2395858677 shutdown_connections 2024-02-17T12:53:33.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.216+0000 7f462f064640 1 -- 172.21.15.18:0/2395858677 wait complete. 2024-02-17T12:53:33.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:53:33.219 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:53:33.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 203059 10.90 MB/sec execute 48 sec latency 38.607 ms 2024-02-17T12:53:33.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.344+0000 7fe67ea64640 1 -- 172.21.15.18:0/166409385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe678100c20 msgr2=0x7fe678104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.344+0000 7fe67ea64640 1 --2- 172.21.15.18:0/166409385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe678100c20 0x7fe678104c60 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009930 tx=0x7fe66c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:53:33.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.344+0000 7fe67ea64640 1 -- 172.21.15.18:0/166409385 shutdown_connections 2024-02-17T12:53:33.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.344+0000 7fe67ea64640 1 --2- 172.21.15.18:0/166409385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe678105340 0x7fe67810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.344+0000 7fe67ea64640 1 --2- 172.21.15.18:0/166409385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe678100c20 0x7fe678104c60 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.344+0000 7fe67ea64640 1 --2- 172.21.15.18:0/166409385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678100270 0x7fe678100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.344+0000 7fe67ea64640 1 -- 172.21.15.18:0/166409385 >> 172.21.15.18:0/166409385 conn(0x7fe678077230 msgr2=0x7fe678077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.344+0000 7fe67ea64640 1 -- 172.21.15.18:0/166409385 shutdown_connections 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.344+0000 7fe67ea64640 1 -- 172.21.15.18:0/166409385 wait complete. 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67ea64640 1 Processor -- start 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67ea64640 1 -- start start 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67ea64640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe678100270 0x7fe67819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67ea64640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678105340 0x7fe67819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67ea64640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe678199810 0x7fe678199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67ea64640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67819a2c0 con 0x7fe678100270 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67ea64640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67819a430 con 0x7fe678105340 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67ea64640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67819a5a0 con 0x7fe678199810 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67c800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe678100270 0x7fe67819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67c800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe678100270 0x7fe67819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60778/0 (socket says 172.21.15.18:60778) 2024-02-17T12:53:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67c800640 1 -- 172.21.15.18:0/2509881035 learned_addr learned my addr 172.21.15.18:0/2509881035 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67d001640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe678199810 0x7fe678199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe677fff640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678105340 0x7fe67819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67c800640 1 -- 172.21.15.18:0/2509881035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe678199810 msgr2=0x7fe678199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67c800640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe678199810 0x7fe678199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67c800640 1 -- 172.21.15.18:0/2509881035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678105340 msgr2=0x7fe67819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67c800640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678105340 0x7fe67819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.345+0000 7fe67c800640 1 -- 172.21.15.18:0/2509881035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c009560 con 0x7fe678100270 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.346+0000 7fe67c800640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe678100270 0x7fe67819f6a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe66800ebc0 tx=0x7fe66800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.346+0000 7fe675ffb640 1 -- 172.21.15.18:0/2509881035 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6680099b0 con 0x7fe678100270 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.346+0000 7fe67ea64640 1 -- 172.21.15.18:0/2509881035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6781a6550 con 0x7fe678100270 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.346+0000 7fe675ffb640 1 -- 172.21.15.18:0/2509881035 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe66800eec0 con 0x7fe678100270 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.346+0000 7fe675ffb640 1 -- 172.21.15.18:0/2509881035 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668018b60 con 0x7fe678100270 2024-02-17T12:53:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.346+0000 7fe67ea64640 1 -- 172.21.15.18:0/2509881035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6781a6a30 con 0x7fe678100270 2024-02-17T12:53:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe68163b740 1 Processor -- start 2024-02-17T12:53:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe675ffb640 1 -- 172.21.15.18:0/2509881035 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe668018cc0 con 0x7fe678100270 2024-02-17T12:53:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe68163b740 1 -- start start 2024-02-17T12:53:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe68163b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70f2b0 0x559d3b70f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe68163b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 0x559d3b718270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe68163b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b71aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe68163b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3b67e280 con 0x559d3b70fbd0 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe68163b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3b67b7b0 con 0x559d3b7187b0 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe68163b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3b676f90 con 0x559d3b70f2b0 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe67c800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 0x559d3b718270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe67d001640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70f2b0 0x559d3b70f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe67c800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 0x559d3b718270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60786/0 (socket says 172.21.15.18:60786) 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe677fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b71aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.349+0000 7fe67c800640 1 -- 172.21.15.18:0/517254710 learned_addr learned my addr 172.21.15.18:0/517254710 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe675ffb640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe6600a2230 0x7fe6600a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe677fff640 1 -- 172.21.15.18:0/517254710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70f2b0 msgr2=0x559d3b70f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe677fff640 1 --2- 172.21.15.18:0/517254710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70f2b0 0x559d3b70f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe677fff640 1 -- 172.21.15.18:0/517254710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 msgr2=0x559d3b718270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe677fff640 1 --2- 172.21.15.18:0/517254710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 0x559d3b718270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe677fff640 1 -- 172.21.15.18:0/517254710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3b635570 con 0x559d3b7187b0 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe67d001640 1 --2- 172.21.15.18:0/517254710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70f2b0 0x559d3b70f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe67d001640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe6600a2230 0x7fe6600a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe675ffb640 1 -- 172.21.15.18:0/2509881035 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe668014070 con 0x7fe678100270 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe677fff640 1 --2- 172.21.15.18:0/517254710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b71aba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009230 tx=0x7fe66c005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed772caea053fba8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe65effd640 1 -- 172.21.15.18:0/517254710 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c004880 con 0x559d3b7187b0 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe67d001640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe6600a2230 0x7fe6600a46f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe670004650 tx=0x7fe6700046c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe65effd640 1 -- 172.21.15.18:0/517254710 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe66c0049e0 con 0x559d3b7187b0 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.350+0000 7fe65effd640 1 -- 172.21.15.18:0/517254710 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c005190 con 0x559d3b7187b0 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.351+0000 7fe68163b740 1 -- 172.21.15.18:0/517254710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 msgr2=0x559d3b71aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.351+0000 7fe68163b740 1 --2- 172.21.15.18:0/517254710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b71aba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009230 tx=0x7fe66c005be0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.351+0000 7fe68163b740 1 -- 172.21.15.18:0/517254710 shutdown_connections 2024-02-17T12:53:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.351+0000 7fe68163b740 1 --2- 172.21.15.18:0/517254710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b71aba0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.351+0000 7fe68163b740 1 --2- 172.21.15.18:0/517254710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 0x559d3b718270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.351+0000 7fe68163b740 1 --2- 172.21.15.18:0/517254710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70f2b0 0x559d3b70f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.351+0000 7fe68163b740 1 -- 172.21.15.18:0/517254710 >> 172.21.15.18:0/517254710 conn(0x559d3b709990 msgr2=0x559d3b709d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.351+0000 7fe68163b740 1 -- 172.21.15.18:0/517254710 shutdown_connections 2024-02-17T12:53:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.351+0000 7fe68163b740 1 -- 172.21.15.18:0/517254710 wait complete. 2024-02-17T12:53:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.351+0000 7fe68163b740 1 -- start start 2024-02-17T12:53:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.351+0000 7fe68163b740 1 Processor -- start 2024-02-17T12:53:33.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe68163b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70f2b0 0x559d3b7181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe68163b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 0x559d3b715580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe68163b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b715cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe68163b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3b34a430 con 0x559d3b70fbd0 2024-02-17T12:53:33.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe68163b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3b306e00 con 0x559d3b7187b0 2024-02-17T12:53:33.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe68163b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3b5fbfd0 con 0x559d3b70f2b0 2024-02-17T12:53:33.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe68163b740 10 client.0 _ll_register_callbacks cb 0x559d3b651fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:33.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe677fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 0x559d3b715580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe677fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 0x559d3b715580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60802/0 (socket says 172.21.15.18:60802) 2024-02-17T12:53:33.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe677fff640 1 -- 172.21.15.18:0/2637054015 learned_addr learned my addr 172.21.15.18:0/2637054015 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:33.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe67d001640 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b715cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe67c800640 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70f2b0 0x559d3b7181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe677fff640 1 -- 172.21.15.18:0/2637054015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70f2b0 msgr2=0x559d3b7181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe677fff640 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70f2b0 0x559d3b7181f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe677fff640 1 -- 172.21.15.18:0/2637054015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 msgr2=0x559d3b715cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe677fff640 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b715cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe677fff640 1 -- 172.21.15.18:0/2637054015 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67810d720 con 0x559d3b70fbd0 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.352+0000 7fe67d001640 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b715cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe677fff640 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 0x559d3b715580 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe66c036040 tx=0x7fe66c02cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b162fdda6e86f3ce server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe657fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe657fff640 1 -- 172.21.15.18:0/2637054015 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0052e0 con 0x559d3b70fbd0 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe657fff640 1 -- 172.21.15.18:0/2637054015 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe66c005440 con 0x559d3b70fbd0 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe68163b740 1 -- 172.21.15.18:0/2637054015 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d3b3767e0 con 0x559d3b70fbd0 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe657fff640 1 -- 172.21.15.18:0/2637054015 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe66c005600 con 0x559d3b70fbd0 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe68163b740 10 client.16941 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:33.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe68163b740 1 -- 172.21.15.18:0/2637054015 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559d3b675610 con 0x559d3b70fbd0 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe657fff640 1 -- 172.21.15.18:0/2637054015 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe66c04e070 con 0x559d3b70fbd0 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe657fff640 1 -- 172.21.15.18:0/2637054015 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe66c052070 con 0x559d3b70fbd0 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe68163b740 10 client.16941 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe68163b740 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.353+0000 7fe68163b740 20 client.16941 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.354+0000 7fe68163b740 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559d3b7284b0 0x559d3b72a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.354+0000 7fe68163b740 4 client.16941 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.354+0000 7fe68163b740 1 -- 172.21.15.18:0/2637054015 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d3b3767e0 con 0x559d3b7284b0 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.354+0000 7fe67c800640 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559d3b7284b0 0x559d3b72a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.354+0000 7fe657fff640 10 client.16941 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.354+0000 7fe67c800640 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559d3b7284b0 0x559d3b72a8e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc8b598b0f1d75f server_cookie=8c2d1a1219631a88 in_seq=0 out_seq=0 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe657fff640 1 -- 172.21.15.18:0/2637054015 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d3b3767e0 con 0x559d3b7284b0 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe657fff640 10 client.16941 handle_command_reply: tid=0 2024-02-17T12:53:33.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe68163b740 1 client.16941 shutdown 2024-02-17T12:53:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe68163b740 1 -- 172.21.15.18:0/2637054015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 msgr2=0x559d3b715580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe68163b740 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 0x559d3b715580 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe66c036040 tx=0x7fe66c02cd80 comp rx=0 tx=0).stop 2024-02-17T12:53:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe68163b740 1 -- 172.21.15.18:0/2637054015 shutdown_connections 2024-02-17T12:53:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe68163b740 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559d3b7284b0 0x559d3b72a8e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe68163b740 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b715cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe68163b740 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70fbd0 0x559d3b715580 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe657fff640 0 client.16941 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe68163b740 1 --2- 172.21.15.18:0/2637054015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70f2b0 0x559d3b7181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe68163b740 1 -- 172.21.15.18:0/2637054015 >> 172.21.15.18:0/2637054015 conn(0x559d3b709990 msgr2=0x559d3b70b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe68163b740 1 -- 172.21.15.18:0/2637054015 shutdown_connections 2024-02-17T12:53:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe68163b740 1 -- 172.21.15.18:0/2637054015 wait complete. 2024-02-17T12:53:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.355+0000 7fe68163b740 20 client.16941 trim_cache size 0 max 16384 2024-02-17T12:53:33.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.371+0000 7fe68163b740 1 Processor -- start 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.371+0000 7fe68163b740 1 -- start start 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.371+0000 7fe68163b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b70f2b0 0x559d3b717cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.371+0000 7fe68163b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70fbd0 0x559d3b7181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.371+0000 7fe68163b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b7187b0 0x559d3b7127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.371+0000 7fe68163b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3b70a3e0 con 0x559d3b7187b0 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.371+0000 7fe68163b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3b70bd60 con 0x559d3b70f2b0 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.371+0000 7fe68163b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3b3767e0 con 0x559d3b70fbd0 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.371+0000 7fe67d001640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b7187b0 0x559d3b7127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe67d001640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b7187b0 0x559d3b7127e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60810/0 (socket says 172.21.15.18:60810) 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe67d001640 1 -- 172.21.15.18:0/774261777 learned_addr learned my addr 172.21.15.18:0/774261777 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe67c800640 1 --2- 172.21.15.18:0/774261777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b70f2b0 0x559d3b717cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe677fff640 1 --2- 172.21.15.18:0/774261777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70fbd0 0x559d3b7181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe67d001640 1 -- 172.21.15.18:0/774261777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70fbd0 msgr2=0x559d3b7181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe67d001640 1 --2- 172.21.15.18:0/774261777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70fbd0 0x559d3b7181f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe67d001640 1 -- 172.21.15.18:0/774261777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b70f2b0 msgr2=0x559d3b717cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe67d001640 1 --2- 172.21.15.18:0/774261777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b70f2b0 0x559d3b717cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe67d001640 1 -- 172.21.15.18:0/774261777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3b611ac0 con 0x559d3b7187b0 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe67d001640 1 --2- 172.21.15.18:0/774261777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b7187b0 0x559d3b7127e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe670003fc0 tx=0x7fe670016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d817bbbc9eb33 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe65e7fc640 1 -- 172.21.15.18:0/774261777 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670024070 con 0x559d3b7187b0 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe65e7fc640 1 -- 172.21.15.18:0/774261777 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe670011dd0 con 0x559d3b7187b0 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.372+0000 7fe65e7fc640 1 -- 172.21.15.18:0/774261777 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67001f820 con 0x559d3b7187b0 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.373+0000 7fe68163b740 1 -- 172.21.15.18:0/774261777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b7187b0 msgr2=0x559d3b7127e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.373+0000 7fe68163b740 1 --2- 172.21.15.18:0/774261777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b7187b0 0x559d3b7127e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe670003fc0 tx=0x7fe670016000 comp rx=0 tx=0).stop 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.373+0000 7fe68163b740 1 -- 172.21.15.18:0/774261777 shutdown_connections 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.373+0000 7fe68163b740 1 --2- 172.21.15.18:0/774261777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b7187b0 0x559d3b7127e0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.373+0000 7fe68163b740 1 --2- 172.21.15.18:0/774261777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70fbd0 0x559d3b7181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.373+0000 7fe68163b740 1 --2- 172.21.15.18:0/774261777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b70f2b0 0x559d3b717cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.373+0000 7fe68163b740 1 -- 172.21.15.18:0/774261777 >> 172.21.15.18:0/774261777 conn(0x559d3b709990 msgr2=0x559d3b710810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.373+0000 7fe68163b740 1 -- 172.21.15.18:0/774261777 shutdown_connections 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.373+0000 7fe68163b740 1 -- 172.21.15.18:0/774261777 wait complete. 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.373+0000 7fe68163b740 1 -- start start 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.373+0000 7fe68163b740 1 Processor -- start 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe68163b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70f2b0 0x559d3b71c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe68163b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70fbd0 0x559d3b71c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe68163b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b77ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe68163b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3b71d1f0 con 0x559d3b70f2b0 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe68163b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3b77ef40 con 0x559d3b7187b0 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe68163b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3b77f080 con 0x559d3b70fbd0 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe68163b740 10 client.0 _ll_register_callbacks cb 0x7fe66c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe67c800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70f2b0 0x559d3b71c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe67c800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70f2b0 0x559d3b71c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60814/0 (socket says 172.21.15.18:60814) 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe67c800640 1 -- 172.21.15.18:0/4069287073 learned_addr learned my addr 172.21.15.18:0/4069287073 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe677fff640 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70fbd0 0x559d3b71c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe67d001640 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b77ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe67c800640 1 -- 172.21.15.18:0/4069287073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70fbd0 msgr2=0x559d3b71c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe67c800640 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70fbd0 0x559d3b71c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe67c800640 1 -- 172.21.15.18:0/4069287073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 msgr2=0x559d3b77ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe67c800640 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b77ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe67c800640 1 -- 172.21.15.18:0/4069287073 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe670003c70 con 0x559d3b70f2b0 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe67c800640 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70f2b0 0x559d3b71c3e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe66809b130 tx=0x7fe6680ce8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8476a829025bea45 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe657fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe657fff640 1 -- 172.21.15.18:0/4069287073 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66809ab00 con 0x559d3b70f2b0 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe657fff640 1 -- 172.21.15.18:0/4069287073 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe668010dc0 con 0x559d3b70f2b0 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.374+0000 7fe68163b740 1 -- 172.21.15.18:0/4069287073 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe66c0049e0 con 0x559d3b70f2b0 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.375+0000 7fe657fff640 1 -- 172.21.15.18:0/4069287073 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe66809b1a0 con 0x559d3b70f2b0 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.375+0000 7fe68163b740 10 client.16947 fetch_fsmap learned FSMap version 31 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.375+0000 7fe68163b740 1 -- 172.21.15.18:0/4069287073 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559d3b6376a0 con 0x559d3b70f2b0 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.375+0000 7fe657fff640 1 -- 172.21.15.18:0/4069287073 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe66c0049e0 con 0x559d3b70f2b0 2024-02-17T12:53:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.375+0000 7fe657fff640 1 -- 172.21.15.18:0/4069287073 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe66809a5f0 con 0x559d3b70f2b0 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.375+0000 7fe68163b740 10 client.16947 fetch_fsmap finished waiting for FSMap version 31 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.375+0000 7fe68163b740 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.375+0000 7fe68163b740 20 client.16947 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.375+0000 7fe68163b740 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559d3b7284b0 0x559d3b782850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.375+0000 7fe68163b740 4 client.16947 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.375+0000 7fe68163b740 1 -- 172.21.15.18:0/4069287073 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559d3b3b3720 con 0x559d3b7284b0 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.375+0000 7fe677fff640 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559d3b7284b0 0x559d3b782850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe677fff640 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559d3b7284b0 0x559d3b782850 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d193f62bcbc1142f server_cookie=ff0b07885e0edeb4 in_seq=0 out_seq=0 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe657fff640 10 client.16947 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe657fff640 1 -- 172.21.15.18:0/4069287073 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559d3b3b3720 con 0x559d3b7284b0 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe657fff640 10 client.16947 handle_command_reply: tid=0 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe68163b740 1 client.16947 shutdown 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe68163b740 1 -- 172.21.15.18:0/4069287073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70f2b0 msgr2=0x559d3b71c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe68163b740 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70f2b0 0x559d3b71c3e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe66809b130 tx=0x7fe6680ce8a0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe68163b740 1 -- 172.21.15.18:0/4069287073 shutdown_connections 2024-02-17T12:53:33.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe68163b740 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559d3b7284b0 0x559d3b782850 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe68163b740 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d3b7187b0 0x559d3b77ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe68163b740 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559d3b70fbd0 0x559d3b71c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe657fff640 0 client.16947 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:33.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe68163b740 1 --2- 172.21.15.18:0/4069287073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d3b70f2b0 0x559d3b71c3e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe68163b740 1 -- 172.21.15.18:0/4069287073 >> 172.21.15.18:0/4069287073 conn(0x559d3b709990 msgr2=0x559d3b713e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:33.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe68163b740 1 -- 172.21.15.18:0/4069287073 shutdown_connections 2024-02-17T12:53:33.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe68163b740 1 -- 172.21.15.18:0/4069287073 wait complete. 2024-02-17T12:53:33.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.376+0000 7fe68163b740 20 client.16947 trim_cache size 0 max 16384 2024-02-17T12:53:33.373 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:53:33.374 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:53:33.374 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:53:33.374 INFO:teuthology.orchestra.run.smithi018.stdout: "35948641-239e-419c-8bdd-d5767dd49fd1": { 2024-02-17T12:53:33.374 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:53:33.374 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "35948641-239e-419c-8bdd-d5767dd49fd1", 2024-02-17T12:53:33.374 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:53:33.374 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:53:33.374 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:53:33.374 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:53:33.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.377+0000 7fe67ea64640 1 -- 172.21.15.18:0/2509881035 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe6600a2230 msgr2=0x7fe6600a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.377+0000 7fe67ea64640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe6600a2230 0x7fe6600a46f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe670004650 tx=0x7fe6700046c0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.377+0000 7fe67ea64640 1 -- 172.21.15.18:0/2509881035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe678100270 msgr2=0x7fe67819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:33.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.377+0000 7fe67ea64640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe678100270 0x7fe67819f6a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe66800ebc0 tx=0x7fe66800c6a0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.377+0000 7fe67ea64640 1 -- 172.21.15.18:0/2509881035 shutdown_connections 2024-02-17T12:53:33.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.377+0000 7fe67ea64640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe6600a2230 0x7fe6600a46f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.377+0000 7fe67ea64640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe678199810 0x7fe678199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.377+0000 7fe67ea64640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678105340 0x7fe67819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.377+0000 7fe67ea64640 1 --2- 172.21.15.18:0/2509881035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe678100270 0x7fe67819f6a0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:33.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.377+0000 7fe67ea64640 1 -- 172.21.15.18:0/2509881035 >> 172.21.15.18:0/2509881035 conn(0x7fe678077230 msgr2=0x7fe6780fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:33.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.377+0000 7fe67ea64640 1 -- 172.21.15.18:0/2509881035 shutdown_connections 2024-02-17T12:53:33.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:53:33.377+0000 7fe67ea64640 1 -- 172.21.15.18:0/2509881035 wait complete. 2024-02-17T12:53:33.379 INFO:tasks.cephfs.filesystem:scrub status for tag:35948641-239e-419c-8bdd-d5767dd49fd1 - {'path': '/', 'tag': '35948641-239e-419c-8bdd-d5767dd49fd1', 'options': 'recursive,force'} 2024-02-17T12:53:34.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 205876 11.03 MB/sec execute 49 sec latency 9.379 ms 2024-02-17T12:53:34.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: pgmap v113: 97 pgs: 97 active+clean; 80 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2024-02-17T12:53:34.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:53:34.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:34.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: pgmap v113: 97 pgs: 97 active+clean; 80 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:34.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:34.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:53:34.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: pgmap v113: 97 pgs: 97 active+clean; 80 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2024-02-17T12:53:34.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:53:34.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:34.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:34.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:34.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:34.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:34.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:34.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:34.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:53:34.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:34.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:53:35.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 207378 10.94 MB/sec execute 50 sec latency 24.849 ms 2024-02-17T12:53:36.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 209666 11.00 MB/sec execute 51 sec latency 11.183 ms 2024-02-17T12:53:36.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:36 smithi146 ceph-mon[36454]: pgmap v114: 97 pgs: 97 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 450 op/s 2024-02-17T12:53:36.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:36 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:53:36.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:36 smithi018 ceph-mon[32625]: pgmap v114: 97 pgs: 97 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 450 op/s 2024-02-17T12:53:36.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:36 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:53:36.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:36 smithi119 ceph-mon[36188]: pgmap v114: 97 pgs: 97 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 450 op/s 2024-02-17T12:53:36.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:36 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:53:37.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 211996 11.00 MB/sec execute 52 sec latency 13.518 ms 2024-02-17T12:53:38.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 213599 10.95 MB/sec execute 53 sec latency 11.031 ms 2024-02-17T12:53:38.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:38 smithi146 ceph-mon[36454]: pgmap v115: 97 pgs: 97 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 313 op/s 2024-02-17T12:53:38.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:38 smithi018 ceph-mon[32625]: pgmap v115: 97 pgs: 97 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 313 op/s 2024-02-17T12:53:38.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:38 smithi119 ceph-mon[36188]: pgmap v115: 97 pgs: 97 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 313 op/s 2024-02-17T12:53:39.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 216250 11.01 MB/sec execute 54 sec latency 12.483 ms 2024-02-17T12:53:40.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 217606 10.93 MB/sec execute 55 sec latency 50.649 ms 2024-02-17T12:53:40.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:40 smithi146 ceph-mon[36454]: pgmap v116: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 450 op/s 2024-02-17T12:53:40.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:40 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:53:40.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:40 smithi018 ceph-mon[32625]: pgmap v116: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 450 op/s 2024-02-17T12:53:40.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:40 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:53:40.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:40 smithi119 ceph-mon[36188]: pgmap v116: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 450 op/s 2024-02-17T12:53:40.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:40 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:53:41.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 220209 11.04 MB/sec execute 56 sec latency 14.324 ms 2024-02-17T12:53:41.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:41 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:53:41.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:41 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:53:41.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:41 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:53:41.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:41 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:53:41.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:41 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:53:41.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:41 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:53:41.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:41 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:53:41.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:41 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:53:41.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:41 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:53:41.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:41 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:53:41.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:41 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:53:41.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:41 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:53:42.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 222057 10.95 MB/sec execute 57 sec latency 9.773 ms 2024-02-17T12:53:42.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:42 smithi146 ceph-mon[36454]: pgmap v117: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 313 op/s 2024-02-17T12:53:42.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:42 smithi018 ceph-mon[32625]: pgmap v117: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 313 op/s 2024-02-17T12:53:42.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:42 smithi119 ceph-mon[36188]: pgmap v117: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 313 op/s 2024-02-17T12:53:43.268 INFO:tasks.workunit.client.0.smithi018.stdout: 1 223941 10.99 MB/sec execute 58 sec latency 9.256 ms 2024-02-17T12:53:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.676+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/2467549383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3ac100360 msgr2=0x7fd3ac1007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.676+0000 7fd3b0ec0640 1 --2- 172.21.15.18:0/2467549383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3ac100360 0x7fd3ac1007c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009930 tx=0x7fd3a002bc30 comp rx=0 tx=0).stop 2024-02-17T12:53:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.676+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/2467549383 shutdown_connections 2024-02-17T12:53:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.676+0000 7fd3b0ec0640 1 --2- 172.21.15.18:0/2467549383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3ac100d00 0x7fd3ac10cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.676+0000 7fd3b0ec0640 1 --2- 172.21.15.18:0/2467549383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3ac100360 0x7fd3ac1007c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.676+0000 7fd3b0ec0640 1 --2- 172.21.15.18:0/2467549383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3ac1050a0 0x7fd3ac105480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.676+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/2467549383 >> 172.21.15.18:0/2467549383 conn(0x7fd3ac077060 msgr2=0x7fd3ac077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.676+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/2467549383 shutdown_connections 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.676+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/2467549383 wait complete. 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3b0ec0640 1 Processor -- start 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3b0ec0640 1 -- start start 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3b0ec0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3ac100360 0x7fd3ac0fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3b0ec0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3ac100d00 0x7fd3ac0fe170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3b0ec0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3ac1050a0 0x7fd3ac0fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3b0ec0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac1a5ab0 con 0x7fd3ac100360 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3b0ec0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac1a5c20 con 0x7fd3ac100d00 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3b0ec0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac1a5de0 con 0x7fd3ac1050a0 2024-02-17T12:53:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3aa59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3ac100360 0x7fd3ac0fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3aa59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3ac100360 0x7fd3ac0fdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40092/0 (socket says 172.21.15.18:40092) 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3aa59c640 1 -- 172.21.15.18:0/134560903 learned_addr learned my addr 172.21.15.18:0/134560903 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3aad9d640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3ac1050a0 0x7fd3ac0fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3ac100d00 0x7fd3ac0fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3aa59c640 1 -- 172.21.15.18:0/134560903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3ac1050a0 msgr2=0x7fd3ac0fe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3aa59c640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3ac1050a0 0x7fd3ac0fe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3aa59c640 1 -- 172.21.15.18:0/134560903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3ac100d00 msgr2=0x7fd3ac0fe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3aa59c640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3ac100d00 0x7fd3ac0fe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3aa59c640 1 -- 172.21.15.18:0/134560903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a0009560 con 0x7fd3ac100360 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.677+0000 7fd3aa59c640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3ac100360 0x7fd3ac0fdc30 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00c920 tx=0x7fd39c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.678+0000 7fd39b7fe640 1 -- 172.21.15.18:0/134560903 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c007c70 con 0x7fd3ac100360 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.678+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/134560903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ac1a60c0 con 0x7fd3ac100360 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.678+0000 7fd39b7fe640 1 -- 172.21.15.18:0/134560903 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd39c007dd0 con 0x7fd3ac100360 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.678+0000 7fd39b7fe640 1 -- 172.21.15.18:0/134560903 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c018860 con 0x7fd3ac100360 2024-02-17T12:53:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.678+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/134560903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ac1a65f0 con 0x7fd3ac100360 2024-02-17T12:53:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.680+0000 7fd39b7fe640 1 -- 172.21.15.18:0/134560903 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd39c00fac0 con 0x7fd3ac100360 2024-02-17T12:53:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd39b7fe640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3940a22d0 0x7fd3940a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3b3b6f740 1 Processor -- start 2024-02-17T12:53:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3940a22d0 0x7fd3940a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd39b7fe640 1 -- 172.21.15.18:0/134560903 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd39c0ce100 con 0x7fd3ac100360 2024-02-17T12:53:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3b3b6f740 1 -- start start 2024-02-17T12:53:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3b3b6f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2436ad0 0x5596d2435820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3b3b6f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d2435d60 0x5596d24361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3b3b6f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d243f2e0 0x5596d24416d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3b3b6f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d23a49f0 con 0x5596d243f2e0 2024-02-17T12:53:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3b3b6f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d23a1f20 con 0x5596d2436ad0 2024-02-17T12:53:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3b3b6f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d239d700 con 0x5596d2435d60 2024-02-17T12:53:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3aad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d243f2e0 0x5596d24416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3aa59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2436ad0 0x5596d2435820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3aad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d243f2e0 0x5596d24416d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40108/0 (socket says 172.21.15.18:40108) 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3aad9d640 1 -- 172.21.15.18:0/504835342 learned_addr learned my addr 172.21.15.18:0/504835342 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3940a22d0 0x7fd3940a4790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0005ab0 tx=0x7fd3a0005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.681+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/504835342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d2435d60 0x5596d24361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.682+0000 7fd3aad9d640 1 -- 172.21.15.18:0/504835342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d2435d60 msgr2=0x5596d24361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.682+0000 7fd3aad9d640 1 --2- 172.21.15.18:0/504835342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d2435d60 0x5596d24361e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.682+0000 7fd3aad9d640 1 -- 172.21.15.18:0/504835342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2436ad0 msgr2=0x5596d2435820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.682+0000 7fd3aad9d640 1 --2- 172.21.15.18:0/504835342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2436ad0 0x5596d2435820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.682+0000 7fd3aad9d640 1 -- 172.21.15.18:0/504835342 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596d23337b0 con 0x5596d243f2e0 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.682+0000 7fd3aad9d640 1 --2- 172.21.15.18:0/504835342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d243f2e0 0x5596d24416d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009fd0 tx=0x7fd3a40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a567fc4c062c88d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.682+0000 7fd398ff9640 1 -- 172.21.15.18:0/504835342 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4004d20 con 0x5596d243f2e0 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.682+0000 7fd398ff9640 1 -- 172.21.15.18:0/504835342 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3a4004e80 con 0x5596d243f2e0 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.682+0000 7fd398ff9640 1 -- 172.21.15.18:0/504835342 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4005280 con 0x5596d243f2e0 2024-02-17T12:53:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.683+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/504835342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d243f2e0 msgr2=0x5596d24416d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.683+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/504835342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d243f2e0 0x5596d24416d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009fd0 tx=0x7fd3a40118d0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.683+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/504835342 shutdown_connections 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.683+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/504835342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d243f2e0 0x5596d24416d0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.683+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/504835342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d2435d60 0x5596d24361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.683+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/504835342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2436ad0 0x5596d2435820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.683+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/504835342 >> 172.21.15.18:0/504835342 conn(0x5596d23fd550 msgr2=0x5596d2430a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.683+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/504835342 shutdown_connections 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.683+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/504835342 wait complete. 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.683+0000 7fd3b3b6f740 1 -- start start 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.683+0000 7fd3b3b6f740 1 Processor -- start 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3b3b6f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 0x5596d243c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3b3b6f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2436ad0 0x5596d243c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3b3b6f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d243f2e0 0x5596d243cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3b3b6f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d2342bf0 con 0x5596d2435d60 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3b3b6f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d2071250 con 0x5596d2436ad0 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3b3b6f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d2323200 con 0x5596d243f2e0 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3b3b6f740 10 client.0 _ll_register_callbacks cb 0x5596d2391d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3aa59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 0x5596d243c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3aa59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 0x5596d243c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40122/0 (socket says 172.21.15.18:40122) 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3aa59c640 1 -- 172.21.15.18:0/1532697439 learned_addr learned my addr 172.21.15.18:0/1532697439 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2436ad0 0x5596d243c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3aad9d640 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d243f2e0 0x5596d243cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3aa59c640 1 -- 172.21.15.18:0/1532697439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d243f2e0 msgr2=0x5596d243cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3aa59c640 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d243f2e0 0x5596d243cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3aa59c640 1 -- 172.21.15.18:0/1532697439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2436ad0 msgr2=0x5596d243c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3aa59c640 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2436ad0 0x5596d243c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3aa59c640 1 -- 172.21.15.18:0/1532697439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a4009c80 con 0x5596d2435d60 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2436ad0 0x5596d243c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3aa59c640 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 0x5596d243c300 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd39c007590 tx=0x7fd39c09b2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e3e6125c9e6f94 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd391ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd391ffb640 1 -- 172.21.15.18:0/1532697439 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c0a3280 con 0x5596d2435d60 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd391ffb640 1 -- 172.21.15.18:0/1532697439 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd39c0a2df0 con 0x5596d2435d60 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.684+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/1532697439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596d20ee1b0 con 0x5596d2435d60 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.685+0000 7fd391ffb640 1 -- 172.21.15.18:0/1532697439 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd39c0a2fb0 con 0x5596d2435d60 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.685+0000 7fd3b3b6f740 10 client.16959 fetch_fsmap learned FSMap version 32 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.685+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/1532697439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5596d239bd80 con 0x5596d2435d60 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.685+0000 7fd391ffb640 1 -- 172.21.15.18:0/1532697439 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5596d20ee1b0 con 0x5596d2435d60 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.685+0000 7fd391ffb640 1 -- 172.21.15.18:0/1532697439 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd39c016020 con 0x5596d2435d60 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.685+0000 7fd3b3b6f740 10 client.16959 fetch_fsmap finished waiting for FSMap version 32 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.685+0000 7fd3b3b6f740 10 client.16959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.685+0000 7fd3b3b6f740 20 client.16959 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.685+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5596d244f600 0x5596d24519f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.685+0000 7fd3b3b6f740 4 client.16959 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.685+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/1532697439 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596d20da550 con 0x5596d244f600 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.685+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5596d244f600 0x5596d24519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd391ffb640 10 client.16959 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5596d244f600 0x5596d24519f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4f3782864e03c02 server_cookie=c0a50f2fa67f0853 in_seq=0 out_seq=0 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd391ffb640 1 -- 172.21.15.18:0/1532697439 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5596d20da550 con 0x5596d244f600 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd391ffb640 10 client.16959 handle_command_reply: tid=0 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd3b3b6f740 1 client.16959 shutdown 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/1532697439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 msgr2=0x5596d243c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 0x5596d243c300 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd39c007590 tx=0x7fd39c09b2a0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/1532697439 shutdown_connections 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5596d244f600 0x5596d24519f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d243f2e0 0x5596d243cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2436ad0 0x5596d243c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd391ffb640 0 client.16959 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/1532697439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 0x5596d243c300 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.686+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/1532697439 >> 172.21.15.18:0/1532697439 conn(0x5596d23fd550 msgr2=0x5596d24327a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.687+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/1532697439 shutdown_connections 2024-02-17T12:53:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.687+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/1532697439 wait complete. 2024-02-17T12:53:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.687+0000 7fd3b3b6f740 20 client.16959 trim_cache size 0 max 16384 2024-02-17T12:53:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.700+0000 7fd3b3b6f740 1 Processor -- start 2024-02-17T12:53:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.700+0000 7fd3b3b6f740 1 -- start start 2024-02-17T12:53:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.700+0000 7fd3b3b6f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 0x5596d243ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.700+0000 7fd3b3b6f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d2436ad0 0x5596d243cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.700+0000 7fd3b3b6f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d243f2e0 0x5596d243abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.700+0000 7fd3b3b6f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d244f400 con 0x5596d2435d60 2024-02-17T12:53:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.700+0000 7fd3b3b6f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d2431620 con 0x5596d243f2e0 2024-02-17T12:53:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.700+0000 7fd3b3b6f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d2435030 con 0x5596d2436ad0 2024-02-17T12:53:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.700+0000 7fd3aa59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 0x5596d243ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.700+0000 7fd3aa59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 0x5596d243ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40130/0 (socket says 172.21.15.18:40130) 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.700+0000 7fd3aa59c640 1 -- 172.21.15.18:0/3006966944 learned_addr learned my addr 172.21.15.18:0/3006966944 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.700+0000 7fd3aad9d640 1 --2- 172.21.15.18:0/3006966944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d243f2e0 0x5596d243abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.701+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/3006966944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d2436ad0 0x5596d243cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.701+0000 7fd3aa59c640 1 -- 172.21.15.18:0/3006966944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d2436ad0 msgr2=0x5596d243cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.701+0000 7fd3aa59c640 1 --2- 172.21.15.18:0/3006966944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d2436ad0 0x5596d243cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.701+0000 7fd3aa59c640 1 -- 172.21.15.18:0/3006966944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d243f2e0 msgr2=0x5596d243abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.701+0000 7fd3aa59c640 1 --2- 172.21.15.18:0/3006966944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d243f2e0 0x5596d243abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.701+0000 7fd3aa59c640 1 -- 172.21.15.18:0/3006966944 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596d23387b0 con 0x5596d2435d60 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.701+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/3006966944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d2436ad0 0x5596d243cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.701+0000 7fd3aa59c640 1 --2- 172.21.15.18:0/3006966944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 0x5596d243ea30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0a2d50 tx=0x7fd39c00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a509476f743f6b35 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.701+0000 7fd393fff640 1 -- 172.21.15.18:0/3006966944 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c099f30 con 0x5596d2435d60 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.701+0000 7fd393fff640 1 -- 172.21.15.18:0/3006966944 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd39c0a3280 con 0x5596d2435d60 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.701+0000 7fd393fff640 1 -- 172.21.15.18:0/3006966944 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c0aed90 con 0x5596d2435d60 2024-02-17T12:53:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/3006966944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 msgr2=0x5596d243ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/3006966944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 0x5596d243ea30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0a2d50 tx=0x7fd39c00fec0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/3006966944 shutdown_connections 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/3006966944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d243f2e0 0x5596d243abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/3006966944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d2436ad0 0x5596d243cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/3006966944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2435d60 0x5596d243ea30 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/3006966944 >> 172.21.15.18:0/3006966944 conn(0x5596d23fd550 msgr2=0x5596d2456320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/3006966944 shutdown_connections 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/3006966944 wait complete. 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 -- start start 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 Processor -- start 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2435d60 0x5596d2446180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2436ad0 0x5596d2444710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d243f2e0 0x5596d2444c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d2445190 con 0x5596d2436ad0 2024-02-17T12:53:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d2445300 con 0x5596d2435d60 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d24454c0 con 0x5596d243f2e0 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3b3b6f740 10 client.0 _ll_register_callbacks cb 0x7fd3a4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3a9d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2436ad0 0x5596d2444710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3aa59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2435d60 0x5596d2446180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3a9d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2436ad0 0x5596d2444710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40134/0 (socket says 172.21.15.18:40134) 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3a9d9b640 1 -- 172.21.15.18:0/610139775 learned_addr learned my addr 172.21.15.18:0/610139775 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.702+0000 7fd3aad9d640 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d243f2e0 0x5596d2444c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd3a9d9b640 1 -- 172.21.15.18:0/610139775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d243f2e0 msgr2=0x5596d2444c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d243f2e0 0x5596d2444c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd3a9d9b640 1 -- 172.21.15.18:0/610139775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2435d60 msgr2=0x5596d2446180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2435d60 0x5596d2446180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd3a9d9b640 1 -- 172.21.15.18:0/610139775 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596d239bd80 con 0x5596d2436ad0 2024-02-17T12:53:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd3aa59c640 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2435d60 0x5596d2446180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd3a9d9b640 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2436ad0 0x5596d2444710 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd3a002cd80 tx=0x7fd3a002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8935babca818c846 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd391ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd391ffb640 1 -- 172.21.15.18:0/610139775 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0004200 con 0x5596d2436ad0 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd391ffb640 1 -- 172.21.15.18:0/610139775 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd3a0053050 con 0x5596d2436ad0 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/610139775 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596d209d600 con 0x5596d2436ad0 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd391ffb640 1 -- 172.21.15.18:0/610139775 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3a0004360 con 0x5596d2436ad0 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd3b3b6f740 10 client.16965 fetch_fsmap learned FSMap version 32 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/610139775 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5596d235e540 con 0x5596d2436ad0 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.703+0000 7fd391ffb640 1 -- 172.21.15.18:0/610139775 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5596d209d600 con 0x5596d2436ad0 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.704+0000 7fd391ffb640 1 -- 172.21.15.18:0/610139775 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd3a0003a20 con 0x5596d2436ad0 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.704+0000 7fd3b3b6f740 10 client.16965 fetch_fsmap finished waiting for FSMap version 32 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.704+0000 7fd3b3b6f740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.704+0000 7fd3b3b6f740 20 client.16965 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.704+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5596d24a6e80 0x5596d24a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.704+0000 7fd3b3b6f740 4 client.16965 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:53:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.704+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/610139775 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5596d2049520 con 0x5596d24a6e80 2024-02-17T12:53:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.704+0000 7fd3aa59c640 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5596d24a6e80 0x5596d24a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.704+0000 7fd391ffb640 10 client.16965 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.704+0000 7fd3aa59c640 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5596d24a6e80 0x5596d24a92a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f00b5deb36bb2e server_cookie=5238e50661a5e251 in_seq=0 out_seq=0 2024-02-17T12:53:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd391ffb640 1 -- 172.21.15.18:0/610139775 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5596d2049520 con 0x5596d24a6e80 2024-02-17T12:53:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd391ffb640 10 client.16965 handle_command_reply: tid=0 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd3b3b6f740 1 client.16965 shutdown 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/610139775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2436ad0 msgr2=0x5596d2444710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2436ad0 0x5596d2444710 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd3a002cd80 tx=0x7fd3a002bc30 comp rx=0 tx=0).stop 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/610139775 shutdown_connections 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5596d24a6e80 0x5596d24a92a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5596d243f2e0 0x5596d2444c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5596d2436ad0 0x5596d2444710 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd3b3b6f740 1 --2- 172.21.15.18:0/610139775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d2435d60 0x5596d2446180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/610139775 >> 172.21.15.18:0/610139775 conn(0x5596d23fd550 msgr2=0x5596d243ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd391ffb640 0 client.16965 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/610139775 shutdown_connections 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd3b3b6f740 1 -- 172.21.15.18:0/610139775 wait complete. 2024-02-17T12:53:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.753+0000 7fd3b3b6f740 20 client.16965 trim_cache size 0 max 16384 2024-02-17T12:53:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:53:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:53:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:53:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:53:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.754+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/134560903 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3940a22d0 msgr2=0x7fd3940a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.754+0000 7fd3b0ec0640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3940a22d0 0x7fd3940a4790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0005ab0 tx=0x7fd3a0005fd0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.754+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/134560903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3ac100360 msgr2=0x7fd3ac0fdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.754+0000 7fd3b0ec0640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3ac100360 0x7fd3ac0fdc30 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00c920 tx=0x7fd39c00cde0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.754+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/134560903 shutdown_connections 2024-02-17T12:53:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.754+0000 7fd3b0ec0640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3940a22d0 0x7fd3940a4790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.754+0000 7fd3b0ec0640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3ac1050a0 0x7fd3ac0fe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.754+0000 7fd3b0ec0640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3ac100d00 0x7fd3ac0fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.754+0000 7fd3b0ec0640 1 --2- 172.21.15.18:0/134560903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3ac100360 0x7fd3ac0fdc30 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.754+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/134560903 >> 172.21.15.18:0/134560903 conn(0x7fd3ac077060 msgr2=0x7fd3ac069b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.754+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/134560903 shutdown_connections 2024-02-17T12:53:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:43.754+0000 7fd3b0ec0640 1 -- 172.21.15.18:0/134560903 wait complete. 2024-02-17T12:53:44.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 226110 10.94 MB/sec execute 59 sec latency 33.195 ms 2024-02-17T12:53:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:44 smithi146 ceph-mon[36454]: pgmap v118: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 313 op/s 2024-02-17T12:53:44.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:44 smithi018 ceph-mon[32625]: pgmap v118: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 313 op/s 2024-02-17T12:53:44.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:44 smithi119 ceph-mon[36188]: pgmap v118: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 313 op/s 2024-02-17T12:53:45.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 227485 10.94 MB/sec execute 60 sec latency 68.275 ms 2024-02-17T12:53:46.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 229815 10.95 MB/sec execute 61 sec latency 25.935 ms 2024-02-17T12:53:46.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:46 smithi146 ceph-mon[36454]: pgmap v119: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 453 op/s 2024-02-17T12:53:46.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:46 smithi018 ceph-mon[32625]: pgmap v119: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 453 op/s 2024-02-17T12:53:46.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:46 smithi119 ceph-mon[36188]: pgmap v119: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 453 op/s 2024-02-17T12:53:47.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 231346 10.91 MB/sec execute 62 sec latency 15.486 ms 2024-02-17T12:53:48.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 233961 10.96 MB/sec execute 63 sec latency 13.083 ms 2024-02-17T12:53:48.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:48 smithi146 ceph-mon[36454]: pgmap v120: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 276 op/s 2024-02-17T12:53:48.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:48 smithi018 ceph-mon[32625]: pgmap v120: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 276 op/s 2024-02-17T12:53:48.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:48 smithi119 ceph-mon[36188]: pgmap v120: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 276 op/s 2024-02-17T12:53:49.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 235355 10.89 MB/sec execute 64 sec latency 9.665 ms 2024-02-17T12:53:50.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 237755 10.96 MB/sec execute 65 sec latency 55.868 ms 2024-02-17T12:53:50.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:50 smithi146 ceph-mon[36454]: pgmap v121: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-02-17T12:53:50.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:50 smithi018 ceph-mon[32625]: pgmap v121: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-02-17T12:53:50.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:50 smithi119 ceph-mon[36188]: pgmap v121: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-02-17T12:53:51.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 239215 10.90 MB/sec execute 66 sec latency 21.893 ms 2024-02-17T12:53:52.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 241601 10.94 MB/sec execute 67 sec latency 10.148 ms 2024-02-17T12:53:52.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:52 smithi146 ceph-mon[36454]: pgmap v122: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 299 op/s 2024-02-17T12:53:52.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:52 smithi018 ceph-mon[32625]: pgmap v122: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 299 op/s 2024-02-17T12:53:52.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:52 smithi119 ceph-mon[36188]: pgmap v122: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 299 op/s 2024-02-17T12:53:53.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 243737 10.90 MB/sec execute 68 sec latency 15.511 ms 2024-02-17T12:53:54.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 245353 10.90 MB/sec execute 69 sec latency 13.458 ms 2024-02-17T12:53:54.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:54 smithi146 ceph-mon[36454]: pgmap v123: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 299 op/s 2024-02-17T12:53:54.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:54 smithi018 ceph-mon[32625]: pgmap v123: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 299 op/s 2024-02-17T12:53:54.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:54 smithi119 ceph-mon[36188]: pgmap v123: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 299 op/s 2024-02-17T12:53:55.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 247778 10.93 MB/sec execute 70 sec latency 41.173 ms 2024-02-17T12:53:56.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 248275 10.84 MB/sec execute 71 sec latency 847.261 ms 2024-02-17T12:53:56.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:56 smithi146 ceph-mon[36454]: pgmap v124: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 435 op/s 2024-02-17T12:53:56.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:56 smithi018 ceph-mon[32625]: pgmap v124: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 435 op/s 2024-02-17T12:53:56.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:56 smithi119 ceph-mon[36188]: pgmap v124: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 435 op/s 2024-02-17T12:53:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.853+0000 7f884a393640 1 -- 172.21.15.18:0/214426958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88441013a0 msgr2=0x7f8844101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.853+0000 7f884a393640 1 --2- 172.21.15.18:0/214426958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88441013a0 0x7f8844101780 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8834009930 tx=0x7f883402bc30 comp rx=0 tx=0).stop 2024-02-17T12:53:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.853+0000 7f884a393640 1 -- 172.21.15.18:0/214426958 shutdown_connections 2024-02-17T12:53:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.853+0000 7f884a393640 1 --2- 172.21.15.18:0/214426958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8844106470 0x7f884410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.853+0000 7f884a393640 1 --2- 172.21.15.18:0/214426958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8844101d50 0x7f8844105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.853+0000 7f884a393640 1 --2- 172.21.15.18:0/214426958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88441013a0 0x7f8844101780 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.853+0000 7f884a393640 1 -- 172.21.15.18:0/214426958 >> 172.21.15.18:0/214426958 conn(0x7f88440fcc40 msgr2=0x7f88440ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.853+0000 7f884a393640 1 -- 172.21.15.18:0/214426958 shutdown_connections 2024-02-17T12:53:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.853+0000 7f884a393640 1 -- 172.21.15.18:0/214426958 wait complete. 2024-02-17T12:53:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.853+0000 7f884a393640 1 Processor -- start 2024-02-17T12:53:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.853+0000 7f884a393640 1 -- start start 2024-02-17T12:53:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f884a393640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88441013a0 0x7f8844197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f884a393640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8844106470 0x7f8844197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f884a393640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f884419c020 0x7f884419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f884a393640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884419ca10 con 0x7f88441013a0 2024-02-17T12:53:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f884a393640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884419cb80 con 0x7f884419c020 2024-02-17T12:53:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f884a393640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884419ccf0 con 0x7f8844106470 2024-02-17T12:53:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f8843fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88441013a0 0x7f8844197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f8843fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88441013a0 0x7f8844197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38098/0 (socket says 172.21.15.18:38098) 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f8843fff640 1 -- 172.21.15.18:0/4099254766 learned_addr learned my addr 172.21.15.18:0/4099254766 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f88437fe640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8844106470 0x7f8844197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f8848930640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f884419c020 0x7f884419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f8843fff640 1 -- 172.21.15.18:0/4099254766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8844106470 msgr2=0x7f8844197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f8843fff640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8844106470 0x7f8844197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f8843fff640 1 -- 172.21.15.18:0/4099254766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f884419c020 msgr2=0x7f884419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f8843fff640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f884419c020 0x7f884419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f8843fff640 1 -- 172.21.15.18:0/4099254766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8834009560 con 0x7f88441013a0 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.854+0000 7f8848930640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f884419c020 0x7f884419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.855+0000 7f8843fff640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88441013a0 0x7f8844197760 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8834009230 tx=0x7f883402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.855+0000 7f88417fa640 1 -- 172.21.15.18:0/4099254766 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834004350 con 0x7f88441013a0 2024-02-17T12:53:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.855+0000 7f884a393640 1 -- 172.21.15.18:0/4099254766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88441a42d0 con 0x7f88441013a0 2024-02-17T12:53:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.855+0000 7f88417fa640 1 -- 172.21.15.18:0/4099254766 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f88340044b0 con 0x7f88441013a0 2024-02-17T12:53:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.855+0000 7f88417fa640 1 -- 172.21.15.18:0/4099254766 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834003d70 con 0x7f88441013a0 2024-02-17T12:53:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.855+0000 7f884a393640 1 -- 172.21.15.18:0/4099254766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88441a4780 con 0x7f88441013a0 2024-02-17T12:53:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.857+0000 7f88417fa640 1 -- 172.21.15.18:0/4099254766 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8834004df0 con 0x7f88441013a0 2024-02-17T12:53:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.857+0000 7f884cf6b740 1 Processor -- start 2024-02-17T12:53:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f884cf6b740 1 -- start start 2024-02-17T12:53:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f884cf6b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 0x5582ce69b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f884cf6b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce69bd60 0x5582ce69c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f884cf6b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce6a96f0 0x5582ce6abae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f884cf6b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ce60a9f0 con 0x5582ce69cad0 2024-02-17T12:53:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f884cf6b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ce607f20 con 0x5582ce6a96f0 2024-02-17T12:53:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f88417fa640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f882c0a2350 0x7f882c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f884cf6b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ce603700 con 0x5582ce69bd60 2024-02-17T12:53:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f8848930640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 0x5582ce69b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f8848930640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 0x5582ce69b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38114/0 (socket says 172.21.15.18:38114) 2024-02-17T12:53:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f8848930640 1 -- 172.21.15.18:0/2039457542 learned_addr learned my addr 172.21.15.18:0/2039457542 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f8848930640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f882c0a2350 0x7f882c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f88417fa640 1 -- 172.21.15.18:0/4099254766 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f88340f01f0 con 0x7f88441013a0 2024-02-17T12:53:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f8843fff640 1 --2- 172.21.15.18:0/2039457542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce69bd60 0x5582ce69c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f88437fe640 1 --2- 172.21.15.18:0/2039457542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce6a96f0 0x5582ce6abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f8848930640 1 -- 172.21.15.18:0/2039457542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce69bd60 msgr2=0x5582ce69c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f8848930640 1 --2- 172.21.15.18:0/2039457542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce69bd60 0x5582ce69c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f8848930640 1 -- 172.21.15.18:0/2039457542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce6a96f0 msgr2=0x5582ce6abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f8848930640 1 --2- 172.21.15.18:0/2039457542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce6a96f0 0x5582ce6abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.858+0000 7f8848930640 1 -- 172.21.15.18:0/2039457542 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582ce5997b0 con 0x5582ce69cad0 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.859+0000 7f8848930640 1 --2- 172.21.15.18:0/2039457542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 0x5582ce69b820 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f883c012d90 tx=0x7f883c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4e8542c8e63831 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.859+0000 7f8848930640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f882c0a2350 0x7f882c0a4810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f883c010ca0 tx=0x7f883c01d000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.859+0000 7f882a7fc640 1 -- 172.21.15.18:0/2039457542 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c012580 con 0x5582ce69cad0 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.859+0000 7f882a7fc640 1 -- 172.21.15.18:0/2039457542 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f883c009d00 con 0x5582ce69cad0 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.859+0000 7f882a7fc640 1 -- 172.21.15.18:0/2039457542 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c005390 con 0x5582ce69cad0 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.859+0000 7f884cf6b740 1 -- 172.21.15.18:0/2039457542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 msgr2=0x5582ce69b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.860+0000 7f884cf6b740 1 --2- 172.21.15.18:0/2039457542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 0x5582ce69b820 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f883c012d90 tx=0x7f883c0107a0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.860+0000 7f884cf6b740 1 -- 172.21.15.18:0/2039457542 shutdown_connections 2024-02-17T12:53:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.860+0000 7f884cf6b740 1 --2- 172.21.15.18:0/2039457542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce6a96f0 0x5582ce6abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.860+0000 7f884cf6b740 1 --2- 172.21.15.18:0/2039457542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce69bd60 0x5582ce69c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.860+0000 7f884cf6b740 1 --2- 172.21.15.18:0/2039457542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 0x5582ce69b820 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.860+0000 7f884cf6b740 1 -- 172.21.15.18:0/2039457542 >> 172.21.15.18:0/2039457542 conn(0x5582ce663550 msgr2=0x5582ce696a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.860+0000 7f884cf6b740 1 -- 172.21.15.18:0/2039457542 shutdown_connections 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.860+0000 7f884cf6b740 1 -- 172.21.15.18:0/2039457542 wait complete. 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.860+0000 7f884cf6b740 1 -- start start 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.860+0000 7f884cf6b740 1 Processor -- start 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.860+0000 7f884cf6b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69bd60 0x5582ce6a6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f884cf6b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 0x5582ce6a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f884cf6b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce6a96f0 0x5582ce6a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f884cf6b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ce5a8bf0 con 0x5582ce69cad0 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f884cf6b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ce2d7250 con 0x5582ce69bd60 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f884cf6b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ce589200 con 0x5582ce6a96f0 2024-02-17T12:53:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f884cf6b740 10 client.0 _ll_register_callbacks cb 0x5582ce5f7d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f88437fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 0x5582ce6a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f88437fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 0x5582ce6a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38116/0 (socket says 172.21.15.18:38116) 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f88437fe640 1 -- 172.21.15.18:0/1748786065 learned_addr learned my addr 172.21.15.18:0/1748786065 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f8843fff640 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69bd60 0x5582ce6a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f8848930640 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce6a96f0 0x5582ce6a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f88437fe640 1 -- 172.21.15.18:0/1748786065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce6a96f0 msgr2=0x5582ce6a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f88437fe640 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce6a96f0 0x5582ce6a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f88437fe640 1 -- 172.21.15.18:0/1748786065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69bd60 msgr2=0x5582ce6a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f88437fe640 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69bd60 0x5582ce6a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f88437fe640 1 -- 172.21.15.18:0/1748786065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883c01a040 con 0x5582ce69cad0 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f8843fff640 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69bd60 0x5582ce6a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f88437fe640 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 0x5582ce6a6cc0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f883800d9e0 tx=0x7f883800dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69173d6fc52984c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f88237fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f88237fe640 1 -- 172.21.15.18:0/1748786065 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838005020 con 0x5582ce69cad0 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f88237fe640 1 -- 172.21.15.18:0/1748786065 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8838005180 con 0x5582ce69cad0 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.861+0000 7f884cf6b740 1 -- 172.21.15.18:0/1748786065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582ce303600 con 0x5582ce69cad0 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.862+0000 7f88237fe640 1 -- 172.21.15.18:0/1748786065 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8838005340 con 0x5582ce69cad0 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.862+0000 7f884cf6b740 10 client.16977 fetch_fsmap learned FSMap version 32 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.862+0000 7f884cf6b740 1 -- 172.21.15.18:0/1748786065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5582ce601d80 con 0x5582ce69cad0 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.862+0000 7f88237fe640 1 -- 172.21.15.18:0/1748786065 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5582ce303600 con 0x5582ce69cad0 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.862+0000 7f88237fe640 1 -- 172.21.15.18:0/1748786065 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f88380246e0 con 0x5582ce69cad0 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.862+0000 7f884cf6b740 10 client.16977 fetch_fsmap finished waiting for FSMap version 32 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.862+0000 7f884cf6b740 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.862+0000 7f884cf6b740 20 client.16977 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.862+0000 7f884cf6b740 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5582ce6b9a70 0x5582ce6bbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.862+0000 7f884cf6b740 4 client.16977 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.862+0000 7f884cf6b740 1 -- 172.21.15.18:0/1748786065 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582ce3541b0 con 0x5582ce6b9a70 2024-02-17T12:53:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.862+0000 7f8843fff640 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5582ce6b9a70 0x5582ce6bbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.863+0000 7f8843fff640 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5582ce6b9a70 0x5582ce6bbe90 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22fc8b8c5869a1a7 server_cookie=535002580655117e in_seq=0 out_seq=0 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.863+0000 7f88237fe640 10 client.16977 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f88237fe640 1 -- 172.21.15.18:0/1748786065 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582ce3541b0 con 0x5582ce6b9a70 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f88237fe640 10 client.16977 handle_command_reply: tid=0 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f884cf6b740 1 client.16977 shutdown 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f884cf6b740 1 -- 172.21.15.18:0/1748786065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 msgr2=0x5582ce6a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f884cf6b740 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 0x5582ce6a6cc0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f883800d9e0 tx=0x7f883800dea0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f884cf6b740 1 -- 172.21.15.18:0/1748786065 shutdown_connections 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f884cf6b740 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5582ce6b9a70 0x5582ce6bbe90 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f884cf6b740 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce6a96f0 0x5582ce6a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f884cf6b740 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69cad0 0x5582ce6a6cc0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f88237fe640 0 client.16977 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f884cf6b740 1 --2- 172.21.15.18:0/1748786065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69bd60 0x5582ce6a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f884cf6b740 1 -- 172.21.15.18:0/1748786065 >> 172.21.15.18:0/1748786065 conn(0x5582ce663550 msgr2=0x5582ce6987a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f884cf6b740 1 -- 172.21.15.18:0/1748786065 shutdown_connections 2024-02-17T12:53:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f884cf6b740 1 -- 172.21.15.18:0/1748786065 wait complete. 2024-02-17T12:53:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.864+0000 7f884cf6b740 20 client.16977 trim_cache size 0 max 16384 2024-02-17T12:53:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.877+0000 7f884cf6b740 1 Processor -- start 2024-02-17T12:53:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f884cf6b740 1 -- start start 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f884cf6b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69bd60 0x5582ce6a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f884cf6b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69cad0 0x5582ce6a7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f884cf6b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce6a96f0 0x5582ce6a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f884cf6b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ce6ac070 con 0x5582ce69bd60 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f884cf6b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ce697c80 con 0x5582ce69cad0 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f884cf6b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ce6b6520 con 0x5582ce6a96f0 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f8843fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69bd60 0x5582ce6a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f8843fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69bd60 0x5582ce6a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38122/0 (socket says 172.21.15.18:38122) 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f8843fff640 1 -- 172.21.15.18:0/3303809316 learned_addr learned my addr 172.21.15.18:0/3303809316 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f8848930640 1 --2- 172.21.15.18:0/3303809316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce6a96f0 0x5582ce6a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f88437fe640 1 --2- 172.21.15.18:0/3303809316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69cad0 0x5582ce6a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f8843fff640 1 -- 172.21.15.18:0/3303809316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce6a96f0 msgr2=0x5582ce6a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f8843fff640 1 --2- 172.21.15.18:0/3303809316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce6a96f0 0x5582ce6a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f8843fff640 1 -- 172.21.15.18:0/3303809316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69cad0 msgr2=0x5582ce6a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f8843fff640 1 --2- 172.21.15.18:0/3303809316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69cad0 0x5582ce6a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f8843fff640 1 -- 172.21.15.18:0/3303809316 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582ce59e7b0 con 0x5582ce69bd60 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f88437fe640 1 --2- 172.21.15.18:0/3303809316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69cad0 0x5582ce6a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.878+0000 7f8843fff640 1 --2- 172.21.15.18:0/3303809316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69bd60 0x5582ce6a8eb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f88340f0160 tx=0x7f8834009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1a3651535fc790 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.879+0000 7f8829ffb640 1 -- 172.21.15.18:0/3303809316 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88340bd3e0 con 0x5582ce69bd60 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.879+0000 7f8829ffb640 1 -- 172.21.15.18:0/3303809316 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f883403d3b0 con 0x5582ce69bd60 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.879+0000 7f8829ffb640 1 -- 172.21.15.18:0/3303809316 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88340c1de0 con 0x5582ce69bd60 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.879+0000 7f884cf6b740 1 -- 172.21.15.18:0/3303809316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69bd60 msgr2=0x5582ce6a8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.879+0000 7f884cf6b740 1 --2- 172.21.15.18:0/3303809316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69bd60 0x5582ce6a8eb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f88340f0160 tx=0x7f8834009260 comp rx=0 tx=0).stop 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.879+0000 7f884cf6b740 1 -- 172.21.15.18:0/3303809316 shutdown_connections 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.879+0000 7f884cf6b740 1 --2- 172.21.15.18:0/3303809316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce6a96f0 0x5582ce6a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.879+0000 7f884cf6b740 1 --2- 172.21.15.18:0/3303809316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69cad0 0x5582ce6a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.879+0000 7f884cf6b740 1 --2- 172.21.15.18:0/3303809316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce69bd60 0x5582ce6a8eb0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.879+0000 7f884cf6b740 1 -- 172.21.15.18:0/3303809316 >> 172.21.15.18:0/3303809316 conn(0x5582ce663550 msgr2=0x5582ce6a5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.879+0000 7f884cf6b740 1 -- 172.21.15.18:0/3303809316 shutdown_connections 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.879+0000 7f884cf6b740 1 -- 172.21.15.18:0/3303809316 wait complete. 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f884cf6b740 1 -- start start 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f884cf6b740 1 Processor -- start 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f884cf6b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce69bd60 0x5582ce6b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f884cf6b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69cad0 0x5582ce6aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f884cf6b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce6a96f0 0x5582ce6af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f884cf6b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ce6af7a0 con 0x5582ce6a96f0 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f884cf6b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ce6af910 con 0x5582ce69cad0 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f884cf6b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ce70ffc0 con 0x5582ce69bd60 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f884cf6b740 10 client.0 _ll_register_callbacks cb 0x7f883c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f8848930640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce6a96f0 0x5582ce6af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f8848930640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce6a96f0 0x5582ce6af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38130/0 (socket says 172.21.15.18:38130) 2024-02-17T12:53:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f8848930640 1 -- 172.21.15.18:0/3574939786 learned_addr learned my addr 172.21.15.18:0/3574939786 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.880+0000 7f8843fff640 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce69bd60 0x5582ce6b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f88437fe640 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69cad0 0x5582ce6aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f8848930640 1 -- 172.21.15.18:0/3574939786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce69bd60 msgr2=0x5582ce6b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f8848930640 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce69bd60 0x5582ce6b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f8848930640 1 -- 172.21.15.18:0/3574939786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69cad0 msgr2=0x5582ce6aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f8848930640 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69cad0 0x5582ce6aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f8848930640 1 -- 172.21.15.18:0/3574939786 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88441a4780 con 0x5582ce6a96f0 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f88437fe640 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69cad0 0x5582ce6aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f8843fff640 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce69bd60 0x5582ce6b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f8848930640 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce6a96f0 0x5582ce6af0d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f883c028040 tx=0x7f883c012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c425914fec3b092a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f88237fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f88237fe640 1 -- 172.21.15.18:0/3574939786 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00de70 con 0x5582ce6a96f0 2024-02-17T12:53:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f88237fe640 1 -- 172.21.15.18:0/3574939786 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f883c0049c0 con 0x5582ce6a96f0 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f884cf6b740 1 -- 172.21.15.18:0/3574939786 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f883c009d00 con 0x5582ce6a96f0 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f88237fe640 1 -- 172.21.15.18:0/3574939786 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f883c004b80 con 0x5582ce6a96f0 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f884cf6b740 10 client.16983 fetch_fsmap learned FSMap version 32 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f884cf6b740 1 -- 172.21.15.18:0/3574939786 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5582ce5c4540 con 0x5582ce6a96f0 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.881+0000 7f88237fe640 1 -- 172.21.15.18:0/3574939786 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f883c009d00 con 0x5582ce6a96f0 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.882+0000 7f88237fe640 1 -- 172.21.15.18:0/3574939786 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f883c02e040 con 0x5582ce6a96f0 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.882+0000 7f884cf6b740 10 client.16983 fetch_fsmap finished waiting for FSMap version 32 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.882+0000 7f884cf6b740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.882+0000 7f884cf6b740 20 client.16983 populate_metadata read hostname 'smithi018' 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.882+0000 7f884cf6b740 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5582ce711750 0x5582ce713b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.882+0000 7f884cf6b740 4 client.16983 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.882+0000 7f884cf6b740 1 -- 172.21.15.18:0/3574939786 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5582ce340550 con 0x5582ce711750 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.882+0000 7f8843fff640 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5582ce711750 0x5582ce713b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.882+0000 7f88237fe640 10 client.16983 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.882+0000 7f8843fff640 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5582ce711750 0x5582ce713b70 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13078ee9de5162d0 server_cookie=cdb004eb8769520c in_seq=0 out_seq=0 2024-02-17T12:53:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f88237fe640 1 -- 172.21.15.18:0/3574939786 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5582ce340550 con 0x5582ce711750 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f88237fe640 10 client.16983 handle_command_reply: tid=0 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f884cf6b740 1 client.16983 shutdown 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f884cf6b740 1 -- 172.21.15.18:0/3574939786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce6a96f0 msgr2=0x5582ce6af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f884cf6b740 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce6a96f0 0x5582ce6af0d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f883c028040 tx=0x7f883c012670 comp rx=0 tx=0).stop 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f884cf6b740 1 -- 172.21.15.18:0/3574939786 shutdown_connections 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f884cf6b740 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5582ce711750 0x5582ce713b70 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f884cf6b740 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ce6a96f0 0x5582ce6af0d0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f884cf6b740 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ce69cad0 0x5582ce6aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f884cf6b740 1 --2- 172.21.15.18:0/3574939786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582ce69bd60 0x5582ce6b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f88237fe640 0 client.16983 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f884cf6b740 1 -- 172.21.15.18:0/3574939786 >> 172.21.15.18:0/3574939786 conn(0x5582ce663550 msgr2=0x5582ce697e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f884cf6b740 1 -- 172.21.15.18:0/3574939786 shutdown_connections 2024-02-17T12:53:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f884cf6b740 1 -- 172.21.15.18:0/3574939786 wait complete. 2024-02-17T12:53:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.930+0000 7f884cf6b740 20 client.16983 trim_cache size 0 max 16384 2024-02-17T12:53:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:53:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:53:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:53:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:53:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.931+0000 7f884a393640 1 -- 172.21.15.18:0/4099254766 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f882c0a2350 msgr2=0x7f882c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.931+0000 7f884a393640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f882c0a2350 0x7f882c0a4810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f883c010ca0 tx=0x7f883c01d000 comp rx=0 tx=0).stop 2024-02-17T12:53:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.931+0000 7f884a393640 1 -- 172.21.15.18:0/4099254766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88441013a0 msgr2=0x7f8844197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:53:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.931+0000 7f884a393640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88441013a0 0x7f8844197760 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8834009230 tx=0x7f883402c6c0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.931+0000 7f884a393640 1 -- 172.21.15.18:0/4099254766 shutdown_connections 2024-02-17T12:53:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.931+0000 7f884a393640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f882c0a2350 0x7f882c0a4810 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.931+0000 7f884a393640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f884419c020 0x7f884419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.931+0000 7f884a393640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8844106470 0x7f8844197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.931+0000 7f884a393640 1 --2- 172.21.15.18:0/4099254766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88441013a0 0x7f8844197760 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:53:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.931+0000 7f884a393640 1 -- 172.21.15.18:0/4099254766 >> 172.21.15.18:0/4099254766 conn(0x7f88440fcc40 msgr2=0x7f88440feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:53:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.931+0000 7f884a393640 1 -- 172.21.15.18:0/4099254766 shutdown_connections 2024-02-17T12:53:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:53:56.931+0000 7f884a393640 1 -- 172.21.15.18:0/4099254766 wait complete. 2024-02-17T12:53:57.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 248275 10.69 MB/sec execute 72 sec latency 1847.354 ms 2024-02-17T12:53:58.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 248275 10.54 MB/sec execute 73 sec latency 2847.448 ms 2024-02-17T12:53:58.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:53:58 smithi146 ceph-mon[36454]: pgmap v125: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-02-17T12:53:58.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:53:58 smithi018 ceph-mon[32625]: pgmap v125: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-02-17T12:53:58.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:53:58 smithi119 ceph-mon[36188]: pgmap v125: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-02-17T12:53:59.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 248313 10.40 MB/sec execute 74 sec latency 3494.025 ms 2024-02-17T12:54:00.269 INFO:tasks.workunit.client.0.smithi018.stdout: 1 248313 10.26 MB/sec execute 75 sec latency 1333.313 ms 2024-02-17T12:54:00.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:00 smithi146 ceph-mon[36454]: pgmap v126: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 389 op/s 2024-02-17T12:54:00.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:00 smithi018 ceph-mon[32625]: pgmap v126: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 389 op/s 2024-02-17T12:54:00.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:00 smithi119 ceph-mon[36188]: pgmap v126: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 389 op/s 2024-02-17T12:54:01.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 248338 10.13 MB/sec execute 76 sec latency 1523.777 ms 2024-02-17T12:54:01.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:01 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:54:01.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:01 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:54:01.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:01 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:54:02.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 248338 10.00 MB/sec execute 77 sec latency 1792.906 ms 2024-02-17T12:54:02.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:02 smithi146 ceph-mon[36454]: pgmap v127: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 229 op/s 2024-02-17T12:54:02.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:02 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:54:02.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:02 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:54:02.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:02 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:54:02.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:02 smithi018 ceph-mon[32625]: pgmap v127: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 229 op/s 2024-02-17T12:54:02.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:02 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:54:02.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:02 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:54:02.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:02 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:54:02.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:02 smithi119 ceph-mon[36188]: pgmap v127: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 229 op/s 2024-02-17T12:54:02.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:02 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:54:02.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:02 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:54:02.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:02 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:54:03.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 248549 9.89 MB/sec execute 78 sec latency 2368.042 ms 2024-02-17T12:54:03.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:54:03.380 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:54:03.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.502+0000 7fb48bfd0640 1 -- 172.21.15.18:0/2195995356 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb484105340 msgr2=0x7fb48410cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.502+0000 7fb48bfd0640 1 --2- 172.21.15.18:0/2195995356 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb484105340 0x7fb48410cf10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb474009cd0 tx=0x7fb47402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.502+0000 7fb48bfd0640 1 -- 172.21.15.18:0/2195995356 shutdown_connections 2024-02-17T12:54:03.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.502+0000 7fb48bfd0640 1 --2- 172.21.15.18:0/2195995356 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb484105340 0x7fb48410cf10 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.502+0000 7fb48bfd0640 1 --2- 172.21.15.18:0/2195995356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb484100c20 0x7fb484104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.502+0000 7fb48bfd0640 1 --2- 172.21.15.18:0/2195995356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb484100270 0x7fb484100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.502+0000 7fb48bfd0640 1 -- 172.21.15.18:0/2195995356 >> 172.21.15.18:0/2195995356 conn(0x7fb484077230 msgr2=0x7fb484077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:03.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.502+0000 7fb48bfd0640 1 -- 172.21.15.18:0/2195995356 shutdown_connections 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.502+0000 7fb48bfd0640 1 -- 172.21.15.18:0/2195995356 wait complete. 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.502+0000 7fb48bfd0640 1 Processor -- start 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.502+0000 7fb48bfd0640 1 -- start start 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48bfd0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb484100270 0x7fb484196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48bfd0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb484105340 0x7fb484196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48bfd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb48419af00 0x7fb48419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48bfd0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48419b8f0 con 0x7fb484105340 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48bfd0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48419ba60 con 0x7fb48419af00 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48bfd0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48419bbd0 con 0x7fb484100270 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48956b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb484105340 0x7fb484196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48956b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb484105340 0x7fb484196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40140/0 (socket says 172.21.15.18:40140) 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48956b640 1 -- 172.21.15.18:0/376017645 learned_addr learned my addr 172.21.15.18:0/376017645 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:03.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48a56d640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb48419af00 0x7fb48419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb489d6c640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb484100270 0x7fb484196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48956b640 1 -- 172.21.15.18:0/376017645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb484100270 msgr2=0x7fb484196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48956b640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb484100270 0x7fb484196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48956b640 1 -- 172.21.15.18:0/376017645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb48419af00 msgr2=0x7fb48419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48956b640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb48419af00 0x7fb48419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48956b640 1 -- 172.21.15.18:0/376017645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb474009900 con 0x7fb484105340 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb489d6c640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb484100270 0x7fb484196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.503+0000 7fb48956b640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb484105340 0x7fb484196bd0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb47800e9f0 tx=0x7fb47800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.504+0000 7fb482ffd640 1 -- 172.21.15.18:0/376017645 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4780096c0 con 0x7fb484105340 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.504+0000 7fb48bfd0640 1 -- 172.21.15.18:0/376017645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4840682e0 con 0x7fb484105340 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.504+0000 7fb482ffd640 1 -- 172.21.15.18:0/376017645 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb478005500 con 0x7fb484105340 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.504+0000 7fb482ffd640 1 -- 172.21.15.18:0/376017645 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478010430 con 0x7fb484105340 2024-02-17T12:54:03.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.504+0000 7fb48bfd0640 1 -- 172.21.15.18:0/376017645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb484068790 con 0x7fb484105340 2024-02-17T12:54:03.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.506+0000 7fb482ffd640 1 -- 172.21.15.18:0/376017645 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb478005020 con 0x7fb484105340 2024-02-17T12:54:03.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.506+0000 7fb48eba3740 1 Processor -- start 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.506+0000 7fb48eba3740 1 -- start start 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb48eba3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 0x557942c9d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb48eba3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9dbd0 0x557942ca6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb48eba3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942ca67b0 0x557942ca8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb48eba3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942c0c280 con 0x557942c9d2b0 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb48eba3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942c097b0 con 0x557942c9dbd0 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb48eba3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942c04f90 con 0x557942ca67b0 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb482ffd640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb46c0a2230 0x7fb46c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb48a56d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 0x557942c9d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb489d6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9dbd0 0x557942ca6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb48a56d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 0x557942c9d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40152/0 (socket says 172.21.15.18:40152) 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb489d6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9dbd0 0x557942ca6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:59952/0 (socket says 172.21.15.18:59952) 2024-02-17T12:54:03.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb48a56d640 1 -- 172.21.15.18:0/1742833432 learned_addr learned my addr 172.21.15.18:0/1742833432 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb48956b640 1 --2- 172.21.15.18:0/1742833432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942ca67b0 0x557942ca8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb48a56d640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb46c0a2230 0x7fb46c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb489d6c640 1 -- 172.21.15.18:0/1742833432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942ca67b0 msgr2=0x557942ca8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb489d6c640 1 --2- 172.21.15.18:0/1742833432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942ca67b0 0x557942ca8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb482ffd640 1 -- 172.21.15.18:0/376017645 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb478014070 con 0x7fb484105340 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb489d6c640 1 -- 172.21.15.18:0/1742833432 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 msgr2=0x557942c9d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb489d6c640 1 --2- 172.21.15.18:0/1742833432 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 0x557942c9d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb489d6c640 1 -- 172.21.15.18:0/1742833432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557942bc3570 con 0x557942c9dbd0 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.507+0000 7fb489d6c640 1 --2- 172.21.15.18:0/1742833432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9dbd0 0x557942ca6270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009c30 tx=0x7fb47c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10dd879c3c6e5a0b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.508+0000 7fb48a56d640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb46c0a2230 0x7fb46c0a46f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb474006010 tx=0x7fb47402c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.508+0000 7fb46bfff640 1 -- 172.21.15.18:0/1742833432 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c009e60 con 0x557942c9dbd0 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.508+0000 7fb46bfff640 1 -- 172.21.15.18:0/1742833432 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb47c012930 con 0x557942c9dbd0 2024-02-17T12:54:03.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.508+0000 7fb46bfff640 1 -- 172.21.15.18:0/1742833432 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c00f910 con 0x557942c9dbd0 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.508+0000 7fb48eba3740 1 -- 172.21.15.18:0/1742833432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9dbd0 msgr2=0x557942ca6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.508+0000 7fb48eba3740 1 --2- 172.21.15.18:0/1742833432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9dbd0 0x557942ca6270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009c30 tx=0x7fb47c0117b0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.508+0000 7fb48eba3740 1 -- 172.21.15.18:0/1742833432 shutdown_connections 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.508+0000 7fb48eba3740 1 --2- 172.21.15.18:0/1742833432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942ca67b0 0x557942ca8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.508+0000 7fb48eba3740 1 --2- 172.21.15.18:0/1742833432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9dbd0 0x557942ca6270 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.508+0000 7fb48eba3740 1 --2- 172.21.15.18:0/1742833432 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 0x557942c9d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.508+0000 7fb48eba3740 1 -- 172.21.15.18:0/1742833432 >> 172.21.15.18:0/1742833432 conn(0x557942c97990 msgr2=0x557942c97d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.509+0000 7fb48eba3740 1 -- 172.21.15.18:0/1742833432 shutdown_connections 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.509+0000 7fb48eba3740 1 -- 172.21.15.18:0/1742833432 wait complete. 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.509+0000 7fb48eba3740 1 -- start start 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.509+0000 7fb48eba3740 1 Processor -- start 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.509+0000 7fb48eba3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9d2b0 0x557942ca6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.509+0000 7fb48eba3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9dbd0 0x557942ca3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.509+0000 7fb48eba3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942ca67b0 0x557942ca3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.509+0000 7fb48eba3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579428d8430 con 0x557942ca67b0 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.509+0000 7fb48eba3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942894e00 con 0x557942c9d2b0 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.509+0000 7fb48eba3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942b89fd0 con 0x557942c9dbd0 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.509+0000 7fb48eba3740 10 client.0 _ll_register_callbacks cb 0x557942bdffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.509+0000 7fb48a56d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942ca67b0 0x557942ca3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb48a56d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942ca67b0 0x557942ca3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40164/0 (socket says 172.21.15.18:40164) 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb489d6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9d2b0 0x557942ca6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb48a56d640 1 -- 172.21.15.18:0/874938290 learned_addr learned my addr 172.21.15.18:0/874938290 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb48956b640 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9dbd0 0x557942ca3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb48a56d640 1 -- 172.21.15.18:0/874938290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9dbd0 msgr2=0x557942ca3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb48a56d640 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9dbd0 0x557942ca3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb48a56d640 1 -- 172.21.15.18:0/874938290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9d2b0 msgr2=0x557942ca6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb48a56d640 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9d2b0 0x557942ca6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb48a56d640 1 -- 172.21.15.18:0/874938290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47c009860 con 0x557942ca67b0 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb489d6c640 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9d2b0 0x557942ca6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb48a56d640 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942ca67b0 0x557942ca3c00 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb47402c630 tx=0x7fb4740095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ff4ad7011affa8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb4697fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb4697fa640 1 -- 172.21.15.18:0/874938290 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474032ac0 con 0x557942ca67b0 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb4697fa640 1 -- 172.21.15.18:0/874938290 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb474003970 con 0x557942ca67b0 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb48eba3740 1 -- 172.21.15.18:0/874938290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579429047e0 con 0x557942ca67b0 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb4697fa640 1 -- 172.21.15.18:0/874938290 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb474003b30 con 0x557942ca67b0 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb48eba3740 10 client.16995 fetch_fsmap learned FSMap version 32 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb48eba3740 1 -- 172.21.15.18:0/874938290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557942c03610 con 0x557942ca67b0 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.510+0000 7fb4697fa640 1 -- 172.21.15.18:0/874938290 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5579429047e0 con 0x557942ca67b0 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.511+0000 7fb4697fa640 1 -- 172.21.15.18:0/874938290 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb47404c070 con 0x557942ca67b0 2024-02-17T12:54:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.511+0000 7fb48eba3740 10 client.16995 fetch_fsmap finished waiting for FSMap version 32 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.511+0000 7fb48eba3740 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.511+0000 7fb48eba3740 20 client.16995 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.511+0000 7fb48eba3740 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557942cb6960 0x557942cb8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.511+0000 7fb48eba3740 4 client.16995 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.511+0000 7fb48eba3740 1 -- 172.21.15.18:0/874938290 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557942941720 con 0x557942cb6960 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.511+0000 7fb489d6c640 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557942cb6960 0x557942cb8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.511+0000 7fb4697fa640 10 client.16995 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.511+0000 7fb489d6c640 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557942cb6960 0x557942cb8d80 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f30292828fbaeca0 server_cookie=30dc70f8ed88c865 in_seq=0 out_seq=0 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb4697fa640 1 -- 172.21.15.18:0/874938290 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557942941720 con 0x557942cb6960 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb4697fa640 10 client.16995 handle_command_reply: tid=0 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb48eba3740 1 client.16995 shutdown 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb48eba3740 1 -- 172.21.15.18:0/874938290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942ca67b0 msgr2=0x557942ca3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb48eba3740 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942ca67b0 0x557942ca3c00 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb47402c630 tx=0x7fb4740095c0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb48eba3740 1 -- 172.21.15.18:0/874938290 shutdown_connections 2024-02-17T12:54:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb48eba3740 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557942cb6960 0x557942cb8d80 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb48eba3740 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942ca67b0 0x557942ca3c00 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb48eba3740 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9dbd0 0x557942ca3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb4697fa640 0 client.16995 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb48eba3740 1 --2- 172.21.15.18:0/874938290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942c9d2b0 0x557942ca6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb48eba3740 1 -- 172.21.15.18:0/874938290 >> 172.21.15.18:0/874938290 conn(0x557942c97990 msgr2=0x557942c99730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb48eba3740 1 -- 172.21.15.18:0/874938290 shutdown_connections 2024-02-17T12:54:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb48eba3740 1 -- 172.21.15.18:0/874938290 wait complete. 2024-02-17T12:54:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.512+0000 7fb48eba3740 20 client.16995 trim_cache size 0 max 16384 2024-02-17T12:54:03.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb48eba3740 1 Processor -- start 2024-02-17T12:54:03.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb48eba3740 1 -- start start 2024-02-17T12:54:03.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb48eba3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9d2b0 0x557942ca1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb48eba3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9dbd0 0x557942ca2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb48eba3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942ca67b0 0x557942ca07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb48eba3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942c97220 con 0x557942c9dbd0 2024-02-17T12:54:03.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb48eba3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942c9c5d0 con 0x557942ca67b0 2024-02-17T12:54:03.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb48eba3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942c977f0 con 0x557942c9d2b0 2024-02-17T12:54:03.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb48956b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9dbd0 0x557942ca2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb48956b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9dbd0 0x557942ca2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40166/0 (socket says 172.21.15.18:40166) 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb48956b640 1 -- 172.21.15.18:0/1585261290 learned_addr learned my addr 172.21.15.18:0/1585261290 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb48a56d640 1 --2- 172.21.15.18:0/1585261290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942ca67b0 0x557942ca07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.528+0000 7fb489d6c640 1 --2- 172.21.15.18:0/1585261290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9d2b0 0x557942ca1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.529+0000 7fb48956b640 1 -- 172.21.15.18:0/1585261290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9d2b0 msgr2=0x557942ca1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.529+0000 7fb48956b640 1 --2- 172.21.15.18:0/1585261290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9d2b0 0x557942ca1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.529+0000 7fb48956b640 1 -- 172.21.15.18:0/1585261290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942ca67b0 msgr2=0x557942ca07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.529+0000 7fb48956b640 1 --2- 172.21.15.18:0/1585261290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942ca67b0 0x557942ca07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.529+0000 7fb48956b640 1 -- 172.21.15.18:0/1585261290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557942b9fac0 con 0x557942c9dbd0 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.529+0000 7fb48956b640 1 --2- 172.21.15.18:0/1585261290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9dbd0 0x557942ca2390 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb478097840 tx=0x7fb47801b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd7b1c93654e694 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.529+0000 7fb46b7fe640 1 -- 172.21.15.18:0/1585261290 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478005670 con 0x557942c9dbd0 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.529+0000 7fb46b7fe640 1 -- 172.21.15.18:0/1585261290 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb47809b570 con 0x557942c9dbd0 2024-02-17T12:54:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.529+0000 7fb46b7fe640 1 -- 172.21.15.18:0/1585261290 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47800acf0 con 0x557942c9dbd0 2024-02-17T12:54:03.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 -- 172.21.15.18:0/1585261290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9dbd0 msgr2=0x557942ca2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 --2- 172.21.15.18:0/1585261290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9dbd0 0x557942ca2390 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb478097840 tx=0x7fb47801b6d0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 -- 172.21.15.18:0/1585261290 shutdown_connections 2024-02-17T12:54:03.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 --2- 172.21.15.18:0/1585261290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942ca67b0 0x557942ca07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 --2- 172.21.15.18:0/1585261290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9dbd0 0x557942ca2390 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 --2- 172.21.15.18:0/1585261290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9d2b0 0x557942ca1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 -- 172.21.15.18:0/1585261290 >> 172.21.15.18:0/1585261290 conn(0x557942c97990 msgr2=0x557942cbd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:03.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 -- 172.21.15.18:0/1585261290 shutdown_connections 2024-02-17T12:54:03.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 -- 172.21.15.18:0/1585261290 wait complete. 2024-02-17T12:54:03.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 -- start start 2024-02-17T12:54:03.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 Processor -- start 2024-02-17T12:54:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 0x557942d0a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9dbd0 0x557942d0abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942ca67b0 0x557942cac910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942cacfc0 con 0x557942c9d2b0 2024-02-17T12:54:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942cad130 con 0x557942ca67b0 2024-02-17T12:54:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557942cad2f0 con 0x557942c9dbd0 2024-02-17T12:54:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.530+0000 7fb48eba3740 10 client.0 _ll_register_callbacks cb 0x7fb47c0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb489d6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 0x557942d0a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb489d6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 0x557942d0a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40170/0 (socket says 172.21.15.18:40170) 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb489d6c640 1 -- 172.21.15.18:0/2084980756 learned_addr learned my addr 172.21.15.18:0/2084980756 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb48956b640 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9dbd0 0x557942d0abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb48a56d640 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942ca67b0 0x557942cac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb489d6c640 1 -- 172.21.15.18:0/2084980756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9dbd0 msgr2=0x557942d0abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb489d6c640 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9dbd0 0x557942d0abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb489d6c640 1 -- 172.21.15.18:0/2084980756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942ca67b0 msgr2=0x557942cac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb489d6c640 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942ca67b0 0x557942cac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb489d6c640 1 -- 172.21.15.18:0/2084980756 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb484068790 con 0x557942c9d2b0 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb48956b640 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9dbd0 0x557942d0abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb489d6c640 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 0x557942d0a670 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb47c005860 tx=0x7fb47c012570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d69a2f8eef0e393 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb4697fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb4697fa640 1 -- 172.21.15.18:0/2084980756 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c02d040 con 0x557942c9d2b0 2024-02-17T12:54:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb4697fa640 1 -- 172.21.15.18:0/2084980756 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb47c012d70 con 0x557942c9d2b0 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb48eba3740 1 -- 172.21.15.18:0/2084980756 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb47c012930 con 0x557942c9d2b0 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb4697fa640 1 -- 172.21.15.18:0/2084980756 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb47c005040 con 0x557942c9d2b0 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb48eba3740 10 client.17001 fetch_fsmap learned FSMap version 32 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.531+0000 7fb48eba3740 1 -- 172.21.15.18:0/2084980756 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557942bc56a0 con 0x557942c9d2b0 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.532+0000 7fb4697fa640 1 -- 172.21.15.18:0/2084980756 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb47c03f020 con 0x557942c9d2b0 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.532+0000 7fb4697fa640 1 -- 172.21.15.18:0/2084980756 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb47c039070 con 0x557942c9d2b0 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.532+0000 7fb48eba3740 10 client.17001 fetch_fsmap finished waiting for FSMap version 32 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.532+0000 7fb48eba3740 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.532+0000 7fb48eba3740 20 client.17001 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.532+0000 7fb48eba3740 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557942d13520 0x557942d15940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.532+0000 7fb48eba3740 4 client.17001 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.532+0000 7fb48eba3740 1 -- 172.21.15.18:0/2084980756 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557942990890 con 0x557942d13520 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.532+0000 7fb48956b640 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557942d13520 0x557942d15940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.532+0000 7fb4697fa640 10 client.17001 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.532+0000 7fb48956b640 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557942d13520 0x557942d15940 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b70359a81ddbab7a server_cookie=2e652dc92fadc9fd in_seq=0 out_seq=0 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb4697fa640 1 -- 172.21.15.18:0/2084980756 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557942990890 con 0x557942d13520 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb4697fa640 10 client.17001 handle_command_reply: tid=0 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48eba3740 1 client.17001 shutdown 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48eba3740 1 -- 172.21.15.18:0/2084980756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 msgr2=0x557942d0a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48eba3740 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 0x557942d0a670 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb47c005860 tx=0x7fb47c012570 comp rx=0 tx=0).stop 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48eba3740 1 -- 172.21.15.18:0/2084980756 shutdown_connections 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48eba3740 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557942d13520 0x557942d15940 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48eba3740 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557942ca67b0 0x557942cac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb4697fa640 0 client.17001 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48eba3740 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557942c9dbd0 0x557942d0abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48eba3740 1 --2- 172.21.15.18:0/2084980756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557942c9d2b0 0x557942d0a670 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48eba3740 1 -- 172.21.15.18:0/2084980756 >> 172.21.15.18:0/2084980756 conn(0x557942c97990 msgr2=0x557942ca1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48eba3740 1 -- 172.21.15.18:0/2084980756 shutdown_connections 2024-02-17T12:54:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48eba3740 1 -- 172.21.15.18:0/2084980756 wait complete. 2024-02-17T12:54:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48eba3740 20 client.17001 trim_cache size 0 max 16384 2024-02-17T12:54:03.531 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:54:03.531 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:54:03.531 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:54:03.531 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:54:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48bfd0640 1 -- 172.21.15.18:0/376017645 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb46c0a2230 msgr2=0x7fb46c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48bfd0640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb46c0a2230 0x7fb46c0a46f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb474006010 tx=0x7fb47402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48bfd0640 1 -- 172.21.15.18:0/376017645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb484105340 msgr2=0x7fb484196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48bfd0640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb484105340 0x7fb484196bd0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb47800e9f0 tx=0x7fb47800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48bfd0640 1 -- 172.21.15.18:0/376017645 shutdown_connections 2024-02-17T12:54:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.533+0000 7fb48bfd0640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb46c0a2230 0x7fb46c0a46f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.534+0000 7fb48bfd0640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb48419af00 0x7fb48419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.534+0000 7fb48bfd0640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb484105340 0x7fb484196bd0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.534+0000 7fb48bfd0640 1 --2- 172.21.15.18:0/376017645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb484100270 0x7fb484196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.534+0000 7fb48bfd0640 1 -- 172.21.15.18:0/376017645 >> 172.21.15.18:0/376017645 conn(0x7fb484077230 msgr2=0x7fb4840fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:03.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.534+0000 7fb48bfd0640 1 -- 172.21.15.18:0/376017645 shutdown_connections 2024-02-17T12:54:03.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.534+0000 7fb48bfd0640 1 -- 172.21.15.18:0/376017645 wait complete. 2024-02-17T12:54:03.535 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:54:03.536 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:54:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.653+0000 7fcb980a3640 1 -- 172.21.15.18:0/3403799305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb90101d30 msgr2=0x7fcb90105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.653+0000 7fcb980a3640 1 --2- 172.21.15.18:0/3403799305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb90101d30 0x7fcb90105d70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009890 tx=0x7fcb8002bb70 comp rx=0 tx=0).stop 2024-02-17T12:54:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.653+0000 7fcb980a3640 1 -- 172.21.15.18:0/3403799305 shutdown_connections 2024-02-17T12:54:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.653+0000 7fcb980a3640 1 --2- 172.21.15.18:0/3403799305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb90106450 0x7fcb9010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.653+0000 7fcb980a3640 1 --2- 172.21.15.18:0/3403799305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb90101d30 0x7fcb90105d70 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.653+0000 7fcb980a3640 1 --2- 172.21.15.18:0/3403799305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb90101380 0x7fcb90101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.653+0000 7fcb980a3640 1 -- 172.21.15.18:0/3403799305 >> 172.21.15.18:0/3403799305 conn(0x7fcb900fcc20 msgr2=0x7fcb900ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.653+0000 7fcb980a3640 1 -- 172.21.15.18:0/3403799305 shutdown_connections 2024-02-17T12:54:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.653+0000 7fcb980a3640 1 -- 172.21.15.18:0/3403799305 wait complete. 2024-02-17T12:54:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb980a3640 1 Processor -- start 2024-02-17T12:54:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb980a3640 1 -- start start 2024-02-17T12:54:03.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb980a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb90101380 0x7fcb90073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb980a3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb90106450 0x7fcb900739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb980a3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb90073f00 0x7fcb901a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb980a3640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb901a5f90 con 0x7fcb90073f00 2024-02-17T12:54:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb980a3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb901a6100 con 0x7fcb90101380 2024-02-17T12:54:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb980a3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb901a62c0 con 0x7fcb90106450 2024-02-17T12:54:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb95e3f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb90101380 0x7fcb90073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb96640640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb90073f00 0x7fcb901a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb95e3f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb90101380 0x7fcb90073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:59996/0 (socket says 172.21.15.18:59996) 2024-02-17T12:54:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb95e3f640 1 -- 172.21.15.18:0/3937896094 learned_addr learned my addr 172.21.15.18:0/3937896094 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.654+0000 7fcb9563e640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb90106450 0x7fcb900739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.655+0000 7fcb95e3f640 1 -- 172.21.15.18:0/3937896094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb90106450 msgr2=0x7fcb900739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.655+0000 7fcb95e3f640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb90106450 0x7fcb900739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.655+0000 7fcb95e3f640 1 -- 172.21.15.18:0/3937896094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb90073f00 msgr2=0x7fcb901a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.655+0000 7fcb95e3f640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb90073f00 0x7fcb901a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.655+0000 7fcb95e3f640 1 -- 172.21.15.18:0/3937896094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c009840 con 0x7fcb90101380 2024-02-17T12:54:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.655+0000 7fcb96640640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb90073f00 0x7fcb901a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.655+0000 7fcb95e3f640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb90101380 0x7fcb90073480 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009d50 tx=0x7fcb8c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.655+0000 7fcb86ffd640 1 -- 172.21.15.18:0/3937896094 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c003b70 con 0x7fcb90101380 2024-02-17T12:54:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.655+0000 7fcb86ffd640 1 -- 172.21.15.18:0/3937896094 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb8c003cd0 con 0x7fcb90101380 2024-02-17T12:54:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.655+0000 7fcb980a3640 1 -- 172.21.15.18:0/3937896094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb88009560 con 0x7fcb90101380 2024-02-17T12:54:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.655+0000 7fcb86ffd640 1 -- 172.21.15.18:0/3937896094 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c010990 con 0x7fcb90101380 2024-02-17T12:54:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.655+0000 7fcb980a3640 1 -- 172.21.15.18:0/3937896094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb80009490 con 0x7fcb90101380 2024-02-17T12:54:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.656+0000 7fcb980a3640 1 -- 172.21.15.18:0/3937896094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb900683c0 con 0x7fcb90101380 2024-02-17T12:54:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.658+0000 7fcb86ffd640 1 -- 172.21.15.18:0/3937896094 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcb8c010af0 con 0x7fcb90101380 2024-02-17T12:54:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.658+0000 7fcb86ffd640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb7c0a2230 0x7fcb7c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.658+0000 7fcb9563e640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb7c0a2230 0x7fcb7c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.658+0000 7fcb86ffd640 1 -- 172.21.15.18:0/3937896094 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb8c014070 con 0x7fcb90101380 2024-02-17T12:54:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.659+0000 7fcb9563e640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb7c0a2230 0x7fcb7c0a46f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcb80005940 tx=0x7fcb800058d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:03.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.662+0000 7fcb86ffd640 1 -- 172.21.15.18:0/3937896094 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb8c0976a0 con 0x7fcb90101380 2024-02-17T12:54:03.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.913+0000 7fcb980a3640 1 -- 172.21.15.18:0/3937896094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb90074960 con 0x7fcb90101380 2024-02-17T12:54:03.906 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:54:03.906 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:53:34.937535+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:54:03.906 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 32 2024-02-17T12:54:03.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.915+0000 7fcb86ffd640 1 -- 172.21.15.18:0/3937896094 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11393 (secure 0 0 0) 0x7fcb8c09ad60 con 0x7fcb90101380 2024-02-17T12:54:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.918+0000 7fcb980a3640 1 -- 172.21.15.18:0/3937896094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb7c0a2230 msgr2=0x7fcb7c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.918+0000 7fcb980a3640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb7c0a2230 0x7fcb7c0a46f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcb80005940 tx=0x7fcb800058d0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.918+0000 7fcb980a3640 1 -- 172.21.15.18:0/3937896094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb90101380 msgr2=0x7fcb90073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.918+0000 7fcb980a3640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb90101380 0x7fcb90073480 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009d50 tx=0x7fcb8c00c6a0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.918+0000 7fcb980a3640 1 -- 172.21.15.18:0/3937896094 shutdown_connections 2024-02-17T12:54:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.918+0000 7fcb980a3640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb7c0a2230 0x7fcb7c0a46f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.918+0000 7fcb980a3640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb90073f00 0x7fcb901a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.918+0000 7fcb980a3640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb90106450 0x7fcb900739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.918+0000 7fcb980a3640 1 --2- 172.21.15.18:0/3937896094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb90101380 0x7fcb90073480 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009d50 tx=0x7fcb8c00c6a0 comp rx=0 tx=0).stop 2024-02-17T12:54:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.918+0000 7fcb980a3640 1 -- 172.21.15.18:0/3937896094 >> 172.21.15.18:0/3937896094 conn(0x7fcb900fcc20 msgr2=0x7fcb900fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.919+0000 7fcb980a3640 1 -- 172.21.15.18:0/3937896094 shutdown_connections 2024-02-17T12:54:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:03.919+0000 7fcb980a3640 1 -- 172.21.15.18:0/3937896094 wait complete. 2024-02-17T12:54:03.922 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:54:03.922 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:54:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.044+0000 7fd41b902640 1 -- 172.21.15.18:0/1784757286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd414100c50 msgr2=0x7fd414104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.044+0000 7fd41b902640 1 --2- 172.21.15.18:0/1784757286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd414100c50 0x7fd414104c90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd404009930 tx=0x7fd40402bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.044+0000 7fd41b902640 1 -- 172.21.15.18:0/1784757286 shutdown_connections 2024-02-17T12:54:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.044+0000 7fd41b902640 1 --2- 172.21.15.18:0/1784757286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd414105370 0x7fd41410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.044+0000 7fd41b902640 1 --2- 172.21.15.18:0/1784757286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd414100c50 0x7fd414104c90 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.044+0000 7fd41b902640 1 --2- 172.21.15.18:0/1784757286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4141002a0 0x7fd414100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.044+0000 7fd41b902640 1 -- 172.21.15.18:0/1784757286 >> 172.21.15.18:0/1784757286 conn(0x7fd414076400 msgr2=0x7fd414076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.044+0000 7fd41b902640 1 -- 172.21.15.18:0/1784757286 shutdown_connections 2024-02-17T12:54:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.044+0000 7fd41b902640 1 -- 172.21.15.18:0/1784757286 wait complete. 2024-02-17T12:54:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.045+0000 7fd41b902640 1 Processor -- start 2024-02-17T12:54:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.045+0000 7fd41b902640 1 -- start start 2024-02-17T12:54:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.045+0000 7fd41b902640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4141002a0 0x7fd414101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.045+0000 7fd41b902640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd414105370 0x7fd414102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.045+0000 7fd41b902640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4141027b0 0x7fd4141a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.045+0000 7fd41b902640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4141a5f80 con 0x7fd4141002a0 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.045+0000 7fd41b902640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4141a60f0 con 0x7fd4141027b0 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.045+0000 7fd41b902640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4141a62b0 con 0x7fd414105370 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.045+0000 7fd41969e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4141002a0 0x7fd414101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.045+0000 7fd41969e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4141002a0 0x7fd414101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40194/0 (socket says 172.21.15.18:40194) 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.045+0000 7fd41969e640 1 -- 172.21.15.18:0/3327501347 learned_addr learned my addr 172.21.15.18:0/3327501347 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.045+0000 7fd419e9f640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4141027b0 0x7fd4141a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd418e9d640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd414105370 0x7fd414102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd41969e640 1 -- 172.21.15.18:0/3327501347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd414105370 msgr2=0x7fd414102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd41969e640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd414105370 0x7fd414102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd41969e640 1 -- 172.21.15.18:0/3327501347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4141027b0 msgr2=0x7fd4141a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd41969e640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4141027b0 0x7fd4141a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd41969e640 1 -- 172.21.15.18:0/3327501347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd404009560 con 0x7fd4141002a0 2024-02-17T12:54:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd419e9f640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4141027b0 0x7fd4141a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd41969e640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4141002a0 0x7fd414101d30 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e9f0 tx=0x7fd41000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd40a7fc640 1 -- 172.21.15.18:0/3327501347 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410009700 con 0x7fd4141002a0 2024-02-17T12:54:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd40a7fc640 1 -- 172.21.15.18:0/3327501347 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd41000cd80 con 0x7fd4141002a0 2024-02-17T12:54:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd41b902640 1 -- 172.21.15.18:0/3327501347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4141a6590 con 0x7fd4141002a0 2024-02-17T12:54:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd40a7fc640 1 -- 172.21.15.18:0/3327501347 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410018920 con 0x7fd4141002a0 2024-02-17T12:54:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.046+0000 7fd41b902640 1 -- 172.21.15.18:0/3327501347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4141a6a70 con 0x7fd4141002a0 2024-02-17T12:54:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.049+0000 7fd40a7fc640 1 -- 172.21.15.18:0/3327501347 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd410018a80 con 0x7fd4141002a0 2024-02-17T12:54:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd41e5b3740 1 Processor -- start 2024-02-17T12:54:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd41e5b3740 1 -- start start 2024-02-17T12:54:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd41e5b3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977250 0x56443f977630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd41e5b3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 0x56443f980210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd41e5b3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f980750 0x56443f982b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd41e5b3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56443f932960 con 0x56443f977b70 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd41e5b3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56443f8e6310 con 0x56443f977250 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd41e5b3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56443f8e3840 con 0x56443f980750 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd41969e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 0x56443f980210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd419e9f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977250 0x56443f977630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd419e9f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977250 0x56443f977630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:60022/0 (socket says 172.21.15.18:60022) 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd40a7fc640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd4000a23b0 0x7fd4000a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd419e9f640 1 -- 172.21.15.18:0/4002475361 learned_addr learned my addr 172.21.15.18:0/4002475361 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd41969e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 0x56443f980210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40208/0 (socket says 172.21.15.18:40208) 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd418e9d640 1 --2- 172.21.15.18:0/4002475361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f980750 0x56443f982b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd419e9f640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd4000a23b0 0x7fd4000a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd419e9f640 1 -- 172.21.15.18:0/4002475361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f980750 msgr2=0x56443f982b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd419e9f640 1 --2- 172.21.15.18:0/4002475361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f980750 0x56443f982b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd419e9f640 1 -- 172.21.15.18:0/4002475361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 msgr2=0x56443f980210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd419e9f640 1 --2- 172.21.15.18:0/4002475361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 0x56443f980210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd40a7fc640 1 -- 172.21.15.18:0/3327501347 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd410014070 con 0x7fd4141002a0 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.050+0000 7fd419e9f640 1 -- 172.21.15.18:0/4002475361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56443f8a1060 con 0x56443f977250 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.051+0000 7fd419e9f640 1 --2- 172.21.15.18:0/4002475361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977250 0x56443f977630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009fd0 tx=0x7fd40c00f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ebd98d36802c8d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.051+0000 7fd419e9f640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd4000a23b0 0x7fd4000a4870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd40c01e000 tx=0x7fd40c020000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.051+0000 7fd3ff7fe640 1 -- 172.21.15.18:0/4002475361 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c013860 con 0x56443f977250 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.051+0000 7fd3ff7fe640 1 -- 172.21.15.18:0/4002475361 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd40c00ed70 con 0x56443f977250 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.051+0000 7fd3ff7fe640 1 -- 172.21.15.18:0/4002475361 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c01fe00 con 0x56443f977250 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.052+0000 7fd41e5b3740 1 -- 172.21.15.18:0/4002475361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977250 msgr2=0x56443f977630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.052+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/4002475361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977250 0x56443f977630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009fd0 tx=0x7fd40c00f7d0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.052+0000 7fd41e5b3740 1 -- 172.21.15.18:0/4002475361 shutdown_connections 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.052+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/4002475361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f980750 0x56443f982b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.052+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/4002475361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 0x56443f980210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.052+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/4002475361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977250 0x56443f977630 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.052+0000 7fd41e5b3740 1 -- 172.21.15.18:0/4002475361 >> 172.21.15.18:0/4002475361 conn(0x56443f971980 msgr2=0x56443f971d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.052+0000 7fd41e5b3740 1 -- 172.21.15.18:0/4002475361 shutdown_connections 2024-02-17T12:54:04.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.052+0000 7fd41e5b3740 1 -- 172.21.15.18:0/4002475361 wait complete. 2024-02-17T12:54:04.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.052+0000 7fd41e5b3740 1 -- start start 2024-02-17T12:54:04.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.052+0000 7fd41e5b3740 1 Processor -- start 2024-02-17T12:54:04.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41e5b3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 0x56443f97d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41e5b3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f977b70 0x56443f97dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41e5b3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f980750 0x56443f97e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41e5b3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56443f8df020 con 0x56443f977250 2024-02-17T12:54:04.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41e5b3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56443f5b2430 con 0x56443f980750 2024-02-17T12:54:04.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41e5b3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56443f56ee00 con 0x56443f977b70 2024-02-17T12:54:04.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41e5b3740 10 client.0 _ll_register_callbacks cb 0x56443f886cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:04.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41969e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 0x56443f97d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41969e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 0x56443f97d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40214/0 (socket says 172.21.15.18:40214) 2024-02-17T12:54:04.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41969e640 1 -- 172.21.15.18:0/3037461836 learned_addr learned my addr 172.21.15.18:0/3037461836 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd418e9d640 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f977b70 0x56443f97dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd419e9f640 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f980750 0x56443f97e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41969e640 1 -- 172.21.15.18:0/3037461836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f977b70 msgr2=0x56443f97dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41969e640 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f977b70 0x56443f97dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41969e640 1 -- 172.21.15.18:0/3037461836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f980750 msgr2=0x56443f97e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41969e640 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f980750 0x56443f97e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41969e640 1 -- 172.21.15.18:0/3037461836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c013a60 con 0x56443f977250 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd41969e640 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 0x56443f97d7c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e9c0 tx=0x7fd410010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6420df3d43be6d95 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.053+0000 7fd3fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd3fcff9640 1 -- 172.21.15.18:0/3037461836 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41009ee10 con 0x56443f977250 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd3fcff9640 1 -- 172.21.15.18:0/3037461836 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd4100cbca0 con 0x56443f977250 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd41e5b3740 1 -- 172.21.15.18:0/3037461836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56443f5de7e0 con 0x56443f977250 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd3fcff9640 1 -- 172.21.15.18:0/3037461836 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4100cbe60 con 0x56443f977250 2024-02-17T12:54:04.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd41e5b3740 10 client.17019 fetch_fsmap learned FSMap version 32 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd41e5b3740 1 -- 172.21.15.18:0/3037461836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56443f8dd6a0 con 0x56443f977250 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd3fcff9640 1 -- 172.21.15.18:0/3037461836 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56443f5de7e0 con 0x56443f977250 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd3fcff9640 1 -- 172.21.15.18:0/3037461836 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd410099c30 con 0x56443f977250 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd41e5b3740 10 client.17019 fetch_fsmap finished waiting for FSMap version 32 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd41e5b3740 10 client.17019 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd41e5b3740 20 client.17019 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56443f990c50 0x56443f993040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd41e5b3740 4 client.17019 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.054+0000 7fd41e5b3740 1 -- 172.21.15.18:0/3037461836 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56443f61b720 con 0x56443f990c50 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.055+0000 7fd418e9d640 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56443f990c50 0x56443f993040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.055+0000 7fd418e9d640 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56443f990c50 0x56443f993040 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=76f2ace1109ede4b server_cookie=cbedfbe8bab62c60 in_seq=0 out_seq=0 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.055+0000 7fd3fcff9640 10 client.17019 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd3fcff9640 1 -- 172.21.15.18:0/3037461836 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56443f61b720 con 0x56443f990c50 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd3fcff9640 10 client.17019 handle_command_reply: tid=0 2024-02-17T12:54:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd41e5b3740 1 client.17019 shutdown 2024-02-17T12:54:04.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd41e5b3740 1 -- 172.21.15.18:0/3037461836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 msgr2=0x56443f97d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 0x56443f97d7c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e9c0 tx=0x7fd410010f50 comp rx=0 tx=0).stop 2024-02-17T12:54:04.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd41e5b3740 1 -- 172.21.15.18:0/3037461836 shutdown_connections 2024-02-17T12:54:04.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56443f990c50 0x56443f993040 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f980750 0x56443f97e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f977b70 0x56443f97dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd3fcff9640 0 client.17019 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:54:04.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/3037461836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 0x56443f97d7c0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd41e5b3740 1 -- 172.21.15.18:0/3037461836 >> 172.21.15.18:0/3037461836 conn(0x56443f971980 msgr2=0x56443f9736d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd41e5b3740 1 -- 172.21.15.18:0/3037461836 shutdown_connections 2024-02-17T12:54:04.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd41e5b3740 1 -- 172.21.15.18:0/3037461836 wait complete. 2024-02-17T12:54:04.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.056+0000 7fd41e5b3740 20 client.17019 trim_cache size 0 max 16384 2024-02-17T12:54:04.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd41e5b3740 1 Processor -- start 2024-02-17T12:54:04.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd41e5b3740 1 -- start start 2024-02-17T12:54:04.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd41e5b3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f977250 0x56443f97fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd41e5b3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 0x56443f97e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd41e5b3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f980750 0x56443f97aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd41e5b3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56443f864130 con 0x56443f977b70 2024-02-17T12:54:04.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd41e5b3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56443f96d080 con 0x56443f980750 2024-02-17T12:54:04.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd41e5b3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56443f990a00 con 0x56443f977250 2024-02-17T12:54:04.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd418e9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 0x56443f97e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd418e9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 0x56443f97e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40224/0 (socket says 172.21.15.18:40224) 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd418e9d640 1 -- 172.21.15.18:0/2006262196 learned_addr learned my addr 172.21.15.18:0/2006262196 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd419e9f640 1 --2- 172.21.15.18:0/2006262196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f980750 0x56443f97aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd41969e640 1 --2- 172.21.15.18:0/2006262196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f977250 0x56443f97fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd418e9d640 1 -- 172.21.15.18:0/2006262196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f977250 msgr2=0x56443f97fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd418e9d640 1 --2- 172.21.15.18:0/2006262196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f977250 0x56443f97fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd418e9d640 1 -- 172.21.15.18:0/2006262196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f980750 msgr2=0x56443f97aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd418e9d640 1 --2- 172.21.15.18:0/2006262196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f980750 0x56443f97aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd418e9d640 1 -- 172.21.15.18:0/2006262196 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56443f879c20 con 0x56443f977b70 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.069+0000 7fd418e9d640 1 --2- 172.21.15.18:0/2006262196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 0x56443f97e060 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd414069560 tx=0x7fd404005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3141386e9f31a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.070+0000 7fd3feffd640 1 -- 172.21.15.18:0/2006262196 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404046040 con 0x56443f977b70 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.070+0000 7fd3feffd640 1 -- 172.21.15.18:0/2006262196 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd404004a60 con 0x56443f977b70 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.070+0000 7fd3feffd640 1 -- 172.21.15.18:0/2006262196 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40402cb40 con 0x56443f977b70 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.070+0000 7fd41e5b3740 1 -- 172.21.15.18:0/2006262196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 msgr2=0x56443f97e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.070+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/2006262196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 0x56443f97e060 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd414069560 tx=0x7fd404005fd0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.070+0000 7fd41e5b3740 1 -- 172.21.15.18:0/2006262196 shutdown_connections 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.070+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/2006262196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f980750 0x56443f97aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.070+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/2006262196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977b70 0x56443f97e060 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.070+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/2006262196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f977250 0x56443f97fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.070+0000 7fd41e5b3740 1 -- 172.21.15.18:0/2006262196 >> 172.21.15.18:0/2006262196 conn(0x56443f971980 msgr2=0x56443f997980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.071+0000 7fd41e5b3740 1 -- 172.21.15.18:0/2006262196 shutdown_connections 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.071+0000 7fd41e5b3740 1 -- 172.21.15.18:0/2006262196 wait complete. 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.071+0000 7fd41e5b3740 1 -- start start 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.071+0000 7fd41e5b3740 1 Processor -- start 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.071+0000 7fd41e5b3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 0x56443f987a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.071+0000 7fd41e5b3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977b70 0x56443f985e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.071+0000 7fd41e5b3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f980750 0x56443f986550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.071+0000 7fd41e5b3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56443f986c20 con 0x56443f977250 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.071+0000 7fd41e5b3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56443f9e7110 con 0x56443f977b70 2024-02-17T12:54:04.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.071+0000 7fd41e5b3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56443f9e7250 con 0x56443f980750 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.071+0000 7fd41e5b3740 10 client.0 _ll_register_callbacks cb 0x7fd40c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd41969e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 0x56443f987a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd418e9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977b70 0x56443f985e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd41969e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 0x56443f987a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40232/0 (socket says 172.21.15.18:40232) 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd41969e640 1 -- 172.21.15.18:0/1513195067 learned_addr learned my addr 172.21.15.18:0/1513195067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd418e9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977b70 0x56443f985e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:60048/0 (socket says 172.21.15.18:60048) 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd419e9f640 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f980750 0x56443f986550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd41969e640 1 -- 172.21.15.18:0/1513195067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f980750 msgr2=0x56443f986550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd41969e640 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f980750 0x56443f986550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd41969e640 1 -- 172.21.15.18:0/1513195067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977b70 msgr2=0x56443f985e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd41969e640 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977b70 0x56443f985e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd41969e640 1 -- 172.21.15.18:0/1513195067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41410d750 con 0x56443f977250 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd418e9d640 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977b70 0x56443f985e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd41969e640 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 0x56443f987a70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd410097740 tx=0x7fd41000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe1e77a10c26455 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd3fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd3fcff9640 1 -- 172.21.15.18:0/1513195067 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41009a3c0 con 0x56443f977250 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd3fcff9640 1 -- 172.21.15.18:0/1513195067 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd41009ee10 con 0x56443f977250 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd41e5b3740 1 -- 172.21.15.18:0/1513195067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd40c00ed70 con 0x56443f977250 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd3fcff9640 1 -- 172.21.15.18:0/1513195067 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd41009a520 con 0x56443f977250 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd41e5b3740 10 client.17025 fetch_fsmap learned FSMap version 32 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.072+0000 7fd41e5b3740 1 -- 172.21.15.18:0/1513195067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56443f89f8c0 con 0x56443f977250 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd3fcff9640 1 -- 172.21.15.18:0/1513195067 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd40c00ed70 con 0x56443f977250 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd3fcff9640 1 -- 172.21.15.18:0/1513195067 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd4100d0100 con 0x56443f977250 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd41e5b3740 10 client.17025 fetch_fsmap finished waiting for FSMap version 32 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd41e5b3740 10 client.17025 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd41e5b3740 20 client.17025 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:04.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56443f9e8740 0x56443f9eab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd41e5b3740 4 client.17025 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd41e5b3740 1 -- 172.21.15.18:0/1513195067 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56443f66a890 con 0x56443f9e8740 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd418e9d640 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56443f9e8740 0x56443f9eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd3fcff9640 10 client.17025 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd418e9d640 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56443f9e8740 0x56443f9eab60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2134dd6fa69c8ee server_cookie=c2dbbcde452d8a89 in_seq=0 out_seq=0 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd3fcff9640 1 -- 172.21.15.18:0/1513195067 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56443f66a890 con 0x56443f9e8740 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd3fcff9640 10 client.17025 handle_command_reply: tid=0 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.073+0000 7fd41e5b3740 1 client.17025 shutdown 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41e5b3740 1 -- 172.21.15.18:0/1513195067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 msgr2=0x56443f987a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 0x56443f987a70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd410097740 tx=0x7fd41000c840 comp rx=0 tx=0).stop 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41e5b3740 1 -- 172.21.15.18:0/1513195067 shutdown_connections 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56443f980750 0x56443f986550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56443f9e8740 0x56443f9eab60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56443f977b70 0x56443f985e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41e5b3740 1 --2- 172.21.15.18:0/1513195067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56443f977250 0x56443f987a70 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd3fcff9640 0 client.17025 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:54:04.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41e5b3740 1 -- 172.21.15.18:0/1513195067 >> 172.21.15.18:0/1513195067 conn(0x56443f971980 msgr2=0x56443f9702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41e5b3740 1 -- 172.21.15.18:0/1513195067 shutdown_connections 2024-02-17T12:54:04.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41e5b3740 1 -- 172.21.15.18:0/1513195067 wait complete. 2024-02-17T12:54:04.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41e5b3740 20 client.17025 trim_cache size 0 max 16384 2024-02-17T12:54:04.072 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:54:04.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41b902640 1 -- 172.21.15.18:0/3327501347 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd4000a23b0 msgr2=0x7fd4000a4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41b902640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd4000a23b0 0x7fd4000a4870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd40c01e000 tx=0x7fd40c020000 comp rx=0 tx=0).stop 2024-02-17T12:54:04.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41b902640 1 -- 172.21.15.18:0/3327501347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4141002a0 msgr2=0x7fd414101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41b902640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4141002a0 0x7fd414101d30 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e9f0 tx=0x7fd41000eeb0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41b902640 1 -- 172.21.15.18:0/3327501347 shutdown_connections 2024-02-17T12:54:04.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41b902640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd4000a23b0 0x7fd4000a4870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41b902640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4141027b0 0x7fd4141a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41b902640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd414105370 0x7fd414102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41b902640 1 --2- 172.21.15.18:0/3327501347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4141002a0 0x7fd414101d30 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.074+0000 7fd41b902640 1 -- 172.21.15.18:0/3327501347 >> 172.21.15.18:0/3327501347 conn(0x7fd414076400 msgr2=0x7fd414069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.075+0000 7fd41b902640 1 -- 172.21.15.18:0/3327501347 shutdown_connections 2024-02-17T12:54:04.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.075+0000 7fd41b902640 1 -- 172.21.15.18:0/3327501347 wait complete. 2024-02-17T12:54:04.076 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:54:04.077 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:54:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.198+0000 7f471b96b640 1 -- 172.21.15.18:0/392133864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4714100c10 msgr2=0x7f4714104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.198+0000 7f471b96b640 1 --2- 172.21.15.18:0/392133864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4714100c10 0x7f4714104c50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4704009930 tx=0x7f470402bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.198+0000 7f471b96b640 1 -- 172.21.15.18:0/392133864 shutdown_connections 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.198+0000 7f471b96b640 1 --2- 172.21.15.18:0/392133864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4714105330 0x7f471410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.198+0000 7f471b96b640 1 --2- 172.21.15.18:0/392133864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4714100c10 0x7f4714104c50 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.198+0000 7f471b96b640 1 --2- 172.21.15.18:0/392133864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4714100260 0x7f4714100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.198+0000 7f471b96b640 1 -- 172.21.15.18:0/392133864 >> 172.21.15.18:0/392133864 conn(0x7f4714077180 msgr2=0x7f4714077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.198+0000 7f471b96b640 1 -- 172.21.15.18:0/392133864 shutdown_connections 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.198+0000 7f471b96b640 1 -- 172.21.15.18:0/392133864 wait complete. 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f471b96b640 1 Processor -- start 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f471b96b640 1 -- start start 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f471b96b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4714100260 0x7f471419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f471b96b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4714105330 0x7f471419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f471b96b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4714197600 0x7f4714197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f471b96b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4714197ff0 con 0x7f4714197600 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f471b96b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4714198160 con 0x7f4714105330 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f471b96b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4714198320 con 0x7f4714100260 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f4719f08640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4714197600 0x7f4714197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f4718f06640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4714105330 0x7f471419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f4719f08640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4714197600 0x7f4714197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40252/0 (socket says 172.21.15.18:40252) 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f4719f08640 1 -- 172.21.15.18:0/2352031732 learned_addr learned my addr 172.21.15.18:0/2352031732 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.199+0000 7f4719707640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4714100260 0x7f471419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.200+0000 7f4719f08640 1 -- 172.21.15.18:0/2352031732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4714100260 msgr2=0x7f471419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.200+0000 7f4719f08640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4714100260 0x7f471419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.200+0000 7f4719f08640 1 -- 172.21.15.18:0/2352031732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4714105330 msgr2=0x7f471419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.200+0000 7f4719f08640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4714105330 0x7f471419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.200+0000 7f4719f08640 1 -- 172.21.15.18:0/2352031732 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4704009560 con 0x7f4714197600 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.200+0000 7f4718f06640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4714105330 0x7f471419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.200+0000 7f4719f08640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4714197600 0x7f4714197ab0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f470c00eb10 tx=0x7f470c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.200+0000 7f470a7fc640 1 -- 172.21.15.18:0/2352031732 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c004b20 con 0x7f4714197600 2024-02-17T12:54:04.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.200+0000 7f471b96b640 1 -- 172.21.15.18:0/2352031732 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47141a4310 con 0x7f4714197600 2024-02-17T12:54:04.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.200+0000 7f470a7fc640 1 -- 172.21.15.18:0/2352031732 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f470c004c80 con 0x7f4714197600 2024-02-17T12:54:04.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.200+0000 7f470a7fc640 1 -- 172.21.15.18:0/2352031732 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c010650 con 0x7f4714197600 2024-02-17T12:54:04.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.200+0000 7f471b96b640 1 -- 172.21.15.18:0/2352031732 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47141a47a0 con 0x7f4714197600 2024-02-17T12:54:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.203+0000 7f470a7fc640 1 -- 172.21.15.18:0/2352031732 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f470c005020 con 0x7f4714197600 2024-02-17T12:54:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.203+0000 7f471e60e740 1 Processor -- start 2024-02-17T12:54:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.203+0000 7f471e60e740 1 -- start start 2024-02-17T12:54:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f471e60e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649250 0x5593bd649630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f471e60e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd652210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f471e60e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd654b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f471e60e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593bd604960 con 0x5593bd652750 2024-02-17T12:54:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f471e60e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593bd5b8310 con 0x5593bd649b70 2024-02-17T12:54:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f471e60e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593bd5b5840 con 0x5593bd649250 2024-02-17T12:54:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f470a7fc640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f47000a2230 0x7f47000a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4719707640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd652210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4718f06640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd654b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4719707640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd652210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:60090/0 (socket says 172.21.15.18:60090) 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4719707640 1 -- 172.21.15.18:0/2577548336 learned_addr learned my addr 172.21.15.18:0/2577548336 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4719f08640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f47000a2230 0x7f47000a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4718f06640 1 --2- 172.21.15.18:0/2577548336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649250 0x5593bd649630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f470a7fc640 1 -- 172.21.15.18:0/2352031732 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f470c014070 con 0x7f4714197600 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4719707640 1 -- 172.21.15.18:0/2577548336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649250 msgr2=0x5593bd649630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4719707640 1 --2- 172.21.15.18:0/2577548336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649250 0x5593bd649630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4719707640 1 -- 172.21.15.18:0/2577548336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 msgr2=0x5593bd654b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4719707640 1 --2- 172.21.15.18:0/2577548336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd654b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4719707640 1 -- 172.21.15.18:0/2577548336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593bd573060 con 0x5593bd649b70 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4719f08640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f47000a2230 0x7f47000a46f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f470c09acf0 tx=0x7f470c06e060 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f4719707640 1 --2- 172.21.15.18:0/2577548336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd652210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f471000b760 tx=0x7f471000bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7674ee6993cea8bf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.204+0000 7f46ff7fe640 1 -- 172.21.15.18:0/2577548336 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710003dd0 con 0x5593bd649b70 2024-02-17T12:54:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.205+0000 7f46ff7fe640 1 -- 172.21.15.18:0/2577548336 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f471000be70 con 0x5593bd649b70 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.205+0000 7f46ff7fe640 1 -- 172.21.15.18:0/2577548336 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710011440 con 0x5593bd649b70 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.205+0000 7f471e60e740 1 -- 172.21.15.18:0/2577548336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 msgr2=0x5593bd652210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.205+0000 7f471e60e740 1 --2- 172.21.15.18:0/2577548336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd652210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f471000b760 tx=0x7f471000bc20 comp rx=0 tx=0).stop 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.205+0000 7f471e60e740 1 -- 172.21.15.18:0/2577548336 shutdown_connections 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.205+0000 7f471e60e740 1 --2- 172.21.15.18:0/2577548336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd654b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.205+0000 7f471e60e740 1 --2- 172.21.15.18:0/2577548336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd652210 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.205+0000 7f471e60e740 1 --2- 172.21.15.18:0/2577548336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649250 0x5593bd649630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.205+0000 7f471e60e740 1 -- 172.21.15.18:0/2577548336 >> 172.21.15.18:0/2577548336 conn(0x5593bd643980 msgr2=0x5593bd643d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.205+0000 7f471e60e740 1 -- 172.21.15.18:0/2577548336 shutdown_connections 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.205+0000 7f471e60e740 1 -- 172.21.15.18:0/2577548336 wait complete. 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f471e60e740 1 -- start start 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f471e60e740 1 Processor -- start 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f471e60e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649250 0x5593bd64f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f471e60e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649b70 0x5593bd64fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f471e60e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd650180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f471e60e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593bd5b1020 con 0x5593bd652750 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f471e60e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593bd284430 con 0x5593bd649250 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f471e60e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593bd240e00 con 0x5593bd649b70 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f471e60e740 10 client.0 _ll_register_callbacks cb 0x5593bd558cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f4718f06640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd650180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f4718f06640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd650180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40266/0 (socket says 172.21.15.18:40266) 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f4718f06640 1 -- 172.21.15.18:0/3451681626 learned_addr learned my addr 172.21.15.18:0/3451681626 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.206+0000 7f4718f06640 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649250 0x5593bd64f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f4719707640 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649b70 0x5593bd64fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f4718f06640 1 -- 172.21.15.18:0/3451681626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649b70 msgr2=0x5593bd64fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f4718f06640 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649b70 0x5593bd64fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f4718f06640 1 -- 172.21.15.18:0/3451681626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649250 msgr2=0x5593bd64f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f4718f06640 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649250 0x5593bd64f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f4718f06640 1 -- 172.21.15.18:0/3451681626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4710004370 con 0x5593bd652750 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f4719707640 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649b70 0x5593bd64fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f4718f06640 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd650180 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4704009930 tx=0x7f470402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c608c39fbaa851d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f46fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f46fcff9640 1 -- 172.21.15.18:0/3451681626 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704004df0 con 0x5593bd652750 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f46fcff9640 1 -- 172.21.15.18:0/3451681626 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4704004f50 con 0x5593bd652750 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f471e60e740 1 -- 172.21.15.18:0/3451681626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593bd2b07e0 con 0x5593bd652750 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f46fcff9640 1 -- 172.21.15.18:0/3451681626 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4704005110 con 0x5593bd652750 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f471e60e740 10 client.17037 fetch_fsmap learned FSMap version 32 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f471e60e740 1 -- 172.21.15.18:0/3451681626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5593bd5af6a0 con 0x5593bd652750 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.207+0000 7f46fcff9640 1 -- 172.21.15.18:0/3451681626 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5593bd2b07e0 con 0x5593bd652750 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.208+0000 7f46fcff9640 1 -- 172.21.15.18:0/3451681626 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f470403c040 con 0x5593bd652750 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.208+0000 7f471e60e740 10 client.17037 fetch_fsmap finished waiting for FSMap version 32 2024-02-17T12:54:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.208+0000 7f471e60e740 10 client.17037 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.208+0000 7f471e60e740 20 client.17037 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.208+0000 7f471e60e740 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5593bd662a60 0x5593bd664e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.208+0000 7f471e60e740 4 client.17037 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.208+0000 7f471e60e740 1 -- 172.21.15.18:0/3451681626 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593bd2ed720 con 0x5593bd662a60 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.208+0000 7f4719707640 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5593bd662a60 0x5593bd664e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.208+0000 7f4719707640 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5593bd662a60 0x5593bd664e80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8204577c4e080ec0 server_cookie=f234bd28000df024 in_seq=0 out_seq=0 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.208+0000 7f46fcff9640 10 client.17037 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f46fcff9640 1 -- 172.21.15.18:0/3451681626 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5593bd2ed720 con 0x5593bd662a60 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f46fcff9640 10 client.17037 handle_command_reply: tid=0 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f471e60e740 1 client.17037 shutdown 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f471e60e740 1 -- 172.21.15.18:0/3451681626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 msgr2=0x5593bd650180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f471e60e740 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd650180 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4704009930 tx=0x7f470402bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f471e60e740 1 -- 172.21.15.18:0/3451681626 shutdown_connections 2024-02-17T12:54:04.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f471e60e740 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5593bd662a60 0x5593bd664e80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f471e60e740 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd650180 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f471e60e740 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649b70 0x5593bd64fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f46fcff9640 0 client.17037 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:54:04.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f471e60e740 1 --2- 172.21.15.18:0/3451681626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649250 0x5593bd64f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f471e60e740 1 -- 172.21.15.18:0/3451681626 >> 172.21.15.18:0/3451681626 conn(0x5593bd643980 msgr2=0x5593bd6456d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f471e60e740 1 -- 172.21.15.18:0/3451681626 shutdown_connections 2024-02-17T12:54:04.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f471e60e740 1 -- 172.21.15.18:0/3451681626 wait complete. 2024-02-17T12:54:04.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.209+0000 7f471e60e740 20 client.17037 trim_cache size 0 max 16384 2024-02-17T12:54:04.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.222+0000 7f471e60e740 1 Processor -- start 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.222+0000 7f471e60e740 1 -- start start 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f471e60e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649250 0x5593bd651e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f471e60e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd64c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f471e60e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd64cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f471e60e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593bd536130 con 0x5593bd652750 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f471e60e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593bd2ed720 con 0x5593bd649b70 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f471e60e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593bd2b07e0 con 0x5593bd649250 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f4718f06640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd64cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f4718f06640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd64cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40274/0 (socket says 172.21.15.18:40274) 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f4718f06640 1 -- 172.21.15.18:0/3395157959 learned_addr learned my addr 172.21.15.18:0/3395157959 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f4719707640 1 --2- 172.21.15.18:0/3395157959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd64c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f4718f06640 1 --2- 172.21.15.18:0/3395157959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649250 0x5593bd651e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f4719707640 1 -- 172.21.15.18:0/3395157959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649250 msgr2=0x5593bd651e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f4719707640 1 --2- 172.21.15.18:0/3395157959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649250 0x5593bd651e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f4719707640 1 -- 172.21.15.18:0/3395157959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 msgr2=0x5593bd64cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f4719707640 1 --2- 172.21.15.18:0/3395157959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd64cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f4719707640 1 -- 172.21.15.18:0/3395157959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593bd54bc20 con 0x5593bd649b70 2024-02-17T12:54:04.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f4718f06640 1 --2- 172.21.15.18:0/3395157959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649250 0x5593bd651e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:04.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.223+0000 7f4719707640 1 --2- 172.21.15.18:0/3395157959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd64c980 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5593bd650cd0 tx=0x7f4710004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=215bd82da42c6b2b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.224+0000 7f46feffd640 1 -- 172.21.15.18:0/3395157959 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471001e040 con 0x5593bd649b70 2024-02-17T12:54:04.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.224+0000 7f46feffd640 1 -- 172.21.15.18:0/3395157959 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f471001f040 con 0x5593bd649b70 2024-02-17T12:54:04.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.224+0000 7f46feffd640 1 -- 172.21.15.18:0/3395157959 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100116a0 con 0x5593bd649b70 2024-02-17T12:54:04.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.225+0000 7f471e60e740 1 -- 172.21.15.18:0/3395157959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 msgr2=0x5593bd64c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.225+0000 7f471e60e740 1 --2- 172.21.15.18:0/3395157959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd64c980 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5593bd650cd0 tx=0x7f4710004ad0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.225+0000 7f471e60e740 1 -- 172.21.15.18:0/3395157959 shutdown_connections 2024-02-17T12:54:04.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.225+0000 7f471e60e740 1 --2- 172.21.15.18:0/3395157959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd652750 0x5593bd64cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.225+0000 7f471e60e740 1 --2- 172.21.15.18:0/3395157959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd64c980 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.225+0000 7f471e60e740 1 --2- 172.21.15.18:0/3395157959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd649250 0x5593bd651e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.225+0000 7f471e60e740 1 -- 172.21.15.18:0/3395157959 >> 172.21.15.18:0/3395157959 conn(0x5593bd643980 msgr2=0x5593bd669820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.225+0000 7f471e60e740 1 -- 172.21.15.18:0/3395157959 shutdown_connections 2024-02-17T12:54:04.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.225+0000 7f471e60e740 1 -- 172.21.15.18:0/3395157959 wait complete. 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.225+0000 7f471e60e740 1 -- start start 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.225+0000 7f471e60e740 1 Processor -- start 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f471e60e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd649250 0x5593bd6b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f471e60e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd6b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f471e60e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd652750 0x5593bd6589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f471e60e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593bd658f90 con 0x5593bd649250 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f471e60e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593bd659100 con 0x5593bd649b70 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f471e60e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593bd6592c0 con 0x5593bd652750 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f4718f06640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd649250 0x5593bd6b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f4719707640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd6b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f4718f06640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd649250 0x5593bd6b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40276/0 (socket says 172.21.15.18:40276) 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f4718f06640 1 -- 172.21.15.18:0/2151812484 learned_addr learned my addr 172.21.15.18:0/2151812484 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f471e60e740 10 client.0 _ll_register_callbacks cb 0x7f4710004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f4718f06640 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd652750 0x5593bd6589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f4718f06640 1 -- 172.21.15.18:0/2151812484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd652750 msgr2=0x5593bd6589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f4718f06640 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd652750 0x5593bd6589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f4718f06640 1 -- 172.21.15.18:0/2151812484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 msgr2=0x5593bd6b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f4718f06640 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd6b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f4718f06640 1 -- 172.21.15.18:0/2151812484 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593bd573060 con 0x5593bd649250 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f4719707640 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd6b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f4718f06640 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd649250 0x5593bd6b34c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x5593bd659f50 tx=0x7f4704005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184d623f1fa53f70 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.226+0000 7f46fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.227+0000 7f46fcff9640 1 -- 172.21.15.18:0/2151812484 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704003a20 con 0x5593bd649250 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.227+0000 7f471e60e740 1 -- 172.21.15.18:0/2151812484 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f471000be70 con 0x5593bd649250 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.227+0000 7f46fcff9640 1 -- 172.21.15.18:0/2151812484 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4704048070 con 0x5593bd649250 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.227+0000 7f46fcff9640 1 -- 172.21.15.18:0/2151812484 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4704003b80 con 0x5593bd649250 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.227+0000 7f471e60e740 10 client.17043 fetch_fsmap learned FSMap version 32 2024-02-17T12:54:04.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.227+0000 7f471e60e740 1 -- 172.21.15.18:0/2151812484 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5593bd5718c0 con 0x5593bd649250 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.227+0000 7f46fcff9640 1 -- 172.21.15.18:0/2151812484 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f471000be70 con 0x5593bd649250 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f46fcff9640 1 -- 172.21.15.18:0/2151812484 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f4704064050 con 0x5593bd649250 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f471e60e740 10 client.17043 fetch_fsmap finished waiting for FSMap version 32 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f471e60e740 10 client.17043 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f471e60e740 20 client.17043 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f471e60e740 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5593bd6bb0f0 0x5593bd6bd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f471e60e740 4 client.17043 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f471e60e740 1 -- 172.21.15.18:0/2151812484 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5593bd33c890 con 0x5593bd6bb0f0 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f4719707640 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5593bd6bb0f0 0x5593bd6bd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f4719707640 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5593bd6bb0f0 0x5593bd6bd510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b43d5deb80cf6ea5 server_cookie=405889eae82c25e2 in_seq=0 out_seq=0 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f46fcff9640 10 client.17043 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f46fcff9640 1 -- 172.21.15.18:0/2151812484 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5593bd33c890 con 0x5593bd6bb0f0 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f46fcff9640 10 client.17043 handle_command_reply: tid=0 2024-02-17T12:54:04.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.228+0000 7f471e60e740 1 client.17043 shutdown 2024-02-17T12:54:04.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471e60e740 1 -- 172.21.15.18:0/2151812484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd649250 msgr2=0x5593bd6b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471e60e740 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd649250 0x5593bd6b34c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x5593bd659f50 tx=0x7f4704005850 comp rx=0 tx=0).stop 2024-02-17T12:54:04.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471e60e740 1 -- 172.21.15.18:0/2151812484 shutdown_connections 2024-02-17T12:54:04.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471e60e740 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5593bd6bb0f0 0x5593bd6bd510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471e60e740 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593bd652750 0x5593bd6589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471e60e740 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593bd649b70 0x5593bd6b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471e60e740 1 --2- 172.21.15.18:0/2151812484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593bd649250 0x5593bd6b34c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471e60e740 1 -- 172.21.15.18:0/2151812484 >> 172.21.15.18:0/2151812484 conn(0x5593bd643980 msgr2=0x5593bd6422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f46fcff9640 0 client.17043 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:54:04.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471e60e740 1 -- 172.21.15.18:0/2151812484 shutdown_connections 2024-02-17T12:54:04.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471e60e740 1 -- 172.21.15.18:0/2151812484 wait complete. 2024-02-17T12:54:04.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471e60e740 20 client.17043 trim_cache size 0 max 16384 2024-02-17T12:54:04.226 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:54:04.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471b96b640 1 -- 172.21.15.18:0/2352031732 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f47000a2230 msgr2=0x7f47000a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471b96b640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f47000a2230 0x7f47000a46f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f470c09acf0 tx=0x7f470c06e060 comp rx=0 tx=0).stop 2024-02-17T12:54:04.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471b96b640 1 -- 172.21.15.18:0/2352031732 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4714197600 msgr2=0x7f4714197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471b96b640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4714197600 0x7f4714197ab0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f470c00eb10 tx=0x7f470c00c2d0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471b96b640 1 -- 172.21.15.18:0/2352031732 shutdown_connections 2024-02-17T12:54:04.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.229+0000 7f471b96b640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f47000a2230 0x7f47000a46f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.230+0000 7f471b96b640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4714197600 0x7f4714197ab0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.230+0000 7f471b96b640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4714105330 0x7f471419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.230+0000 7f471b96b640 1 --2- 172.21.15.18:0/2352031732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4714100260 0x7f471419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.230+0000 7f471b96b640 1 -- 172.21.15.18:0/2352031732 >> 172.21.15.18:0/2352031732 conn(0x7f4714077180 msgr2=0x7f4714068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.230+0000 7f471b96b640 1 -- 172.21.15.18:0/2352031732 shutdown_connections 2024-02-17T12:54:04.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.230+0000 7f471b96b640 1 -- 172.21.15.18:0/2352031732 wait complete. 2024-02-17T12:54:04.232 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:54:04.232 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:54:04.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 250625 9.85 MB/sec execute 79 sec latency 21.245 ms 2024-02-17T12:54:04.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.351+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/1459816762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c4100260 msgr2=0x7fd8c4100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.351+0000 7fd8cb8d4640 1 --2- 172.21.15.18:0/1459816762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c4100260 0x7fd8c4100640 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009930 tx=0x7fd8c002bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.351+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/1459816762 shutdown_connections 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.351+0000 7fd8cb8d4640 1 --2- 172.21.15.18:0/1459816762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8c4105330 0x7fd8c410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.351+0000 7fd8cb8d4640 1 --2- 172.21.15.18:0/1459816762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8c4100c10 0x7fd8c4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.351+0000 7fd8cb8d4640 1 --2- 172.21.15.18:0/1459816762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c4100260 0x7fd8c4100640 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.351+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/1459816762 >> 172.21.15.18:0/1459816762 conn(0x7fd8c4077180 msgr2=0x7fd8c4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.351+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/1459816762 shutdown_connections 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.351+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/1459816762 wait complete. 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8cb8d4640 1 Processor -- start 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8cb8d4640 1 -- start start 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8cb8d4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8c4100260 0x7fd8c4101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8cb8d4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c4105330 0x7fd8c4102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8cb8d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8c4102770 0x7fd8c41a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8cb8d4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c41a5fd0 con 0x7fd8c4105330 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8cb8d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c41a6140 con 0x7fd8c4102770 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8cb8d4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c41a6300 con 0x7fd8c4100260 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8c8e6f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c4105330 0x7fd8c4102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8c8e6f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c4105330 0x7fd8c4102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40302/0 (socket says 172.21.15.18:40302) 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/468678922 learned_addr learned my addr 172.21.15.18:0/468678922 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8c9670640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8c4100260 0x7fd8c4101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.352+0000 7fd8c9e71640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8c4102770 0x7fd8c41a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.353+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/468678922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8c4100260 msgr2=0x7fd8c4101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.353+0000 7fd8c8e6f640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8c4100260 0x7fd8c4101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.353+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/468678922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8c4102770 msgr2=0x7fd8c41a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.353+0000 7fd8c8e6f640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8c4102770 0x7fd8c41a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.353+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/468678922 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c0009560 con 0x7fd8c4105330 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.353+0000 7fd8c9e71640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8c4102770 0x7fd8c41a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.353+0000 7fd8c8e6f640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c4105330 0x7fd8c4102230 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400e9c0 tx=0x7fd8b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.353+0000 7fd8ba7fc640 1 -- 172.21.15.18:0/468678922 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4009700 con 0x7fd8c4105330 2024-02-17T12:54:04.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.353+0000 7fd8ba7fc640 1 -- 172.21.15.18:0/468678922 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8b400cd80 con 0x7fd8c4105330 2024-02-17T12:54:04.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.353+0000 7fd8ba7fc640 1 -- 172.21.15.18:0/468678922 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4010430 con 0x7fd8c4105330 2024-02-17T12:54:04.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.353+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/468678922 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c41a65e0 con 0x7fd8c4105330 2024-02-17T12:54:04.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.353+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/468678922 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c41a6af0 con 0x7fd8c4105330 2024-02-17T12:54:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.356+0000 7fd8ba7fc640 1 -- 172.21.15.18:0/468678922 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd8b4005000 con 0x7fd8c4105330 2024-02-17T12:54:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.356+0000 7fd8ce57c740 1 Processor -- start 2024-02-17T12:54:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.356+0000 7fd8ce57c740 1 -- start start 2024-02-17T12:54:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8ce57c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542250 0x563723542630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8ce57c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563723542b70 0x56372354b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8ce57c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56372354b750 0x56372354db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8ce57c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637234fd960 con 0x56372354b750 2024-02-17T12:54:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8ce57c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637234b1310 con 0x563723542b70 2024-02-17T12:54:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8ce57c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637234ae840 con 0x563723542250 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c8e6f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56372354b750 0x56372354db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c8e6f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56372354b750 0x56372354db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40314/0 (socket says 172.21.15.18:40314) 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/1582623269 learned_addr learned my addr 172.21.15.18:0/1582623269 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c9670640 1 --2- 172.21.15.18:0/1582623269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563723542b70 0x56372354b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c9e71640 1 --2- 172.21.15.18:0/1582623269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542250 0x563723542630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8ba7fc640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8b00a2260 0x7fd8b00a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c9e71640 1 -- 172.21.15.18:0/1582623269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563723542b70 msgr2=0x56372354b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c9e71640 1 --2- 172.21.15.18:0/1582623269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563723542b70 0x56372354b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c9e71640 1 -- 172.21.15.18:0/1582623269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56372354b750 msgr2=0x56372354db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c9e71640 1 --2- 172.21.15.18:0/1582623269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56372354b750 0x56372354db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c9e71640 1 -- 172.21.15.18:0/1582623269 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56372346c060 con 0x563723542250 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c8e6f640 1 --2- 172.21.15.18:0/1582623269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56372354b750 0x56372354db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8ba7fc640 1 -- 172.21.15.18:0/468678922 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd8b4014070 con 0x7fd8c4105330 2024-02-17T12:54:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c9e71640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8b00a2260 0x7fd8b00a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.357+0000 7fd8c9e71640 1 --2- 172.21.15.18:0/1582623269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542250 0x563723542630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc005fd0 tx=0x7fd8bc011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee8e6d88d73c8b70 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.358+0000 7fd8af7fe640 1 -- 172.21.15.18:0/1582623269 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc021070 con 0x563723542250 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.358+0000 7fd8af7fe640 1 -- 172.21.15.18:0/1582623269 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8bc003d50 con 0x563723542250 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.358+0000 7fd8af7fe640 1 -- 172.21.15.18:0/1582623269 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc01c8c0 con 0x563723542250 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.358+0000 7fd8c9e71640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8b00a2260 0x7fd8b00a4720 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc020000 tx=0x7fd8bc021000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.358+0000 7fd8ce57c740 1 -- 172.21.15.18:0/1582623269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542250 msgr2=0x563723542630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.358+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/1582623269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542250 0x563723542630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc005fd0 tx=0x7fd8bc011bf0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.359+0000 7fd8ce57c740 1 -- 172.21.15.18:0/1582623269 shutdown_connections 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.359+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/1582623269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56372354b750 0x56372354db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.359+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/1582623269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563723542b70 0x56372354b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.359+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/1582623269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542250 0x563723542630 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.359+0000 7fd8ce57c740 1 -- 172.21.15.18:0/1582623269 >> 172.21.15.18:0/1582623269 conn(0x56372353c980 msgr2=0x56372353cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.359+0000 7fd8ce57c740 1 -- 172.21.15.18:0/1582623269 shutdown_connections 2024-02-17T12:54:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.359+0000 7fd8ce57c740 1 -- 172.21.15.18:0/1582623269 wait complete. 2024-02-17T12:54:04.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.359+0000 7fd8ce57c740 1 -- start start 2024-02-17T12:54:04.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.359+0000 7fd8ce57c740 1 Processor -- start 2024-02-17T12:54:04.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.359+0000 7fd8ce57c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542250 0x5637235486d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.359+0000 7fd8ce57c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563723542b70 0x563723548c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.359+0000 7fd8ce57c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56372354b750 0x563723549150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8ce57c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637234aa020 con 0x563723542250 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8ce57c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372317d430 con 0x563723542b70 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8ce57c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563723139e00 con 0x56372354b750 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8c8e6f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563723542b70 0x563723548c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8c9670640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542250 0x5637235486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8c8e6f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563723542b70 0x563723548c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:60132/0 (socket says 172.21.15.18:60132) 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/3333903537 learned_addr learned my addr 172.21.15.18:0/3333903537 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8c9e71640 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56372354b750 0x563723549150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8ce57c740 10 client.0 _ll_register_callbacks cb 0x563723451cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/3333903537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56372354b750 msgr2=0x563723549150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8c8e6f640 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56372354b750 0x563723549150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/3333903537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542250 msgr2=0x5637235486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8c8e6f640 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542250 0x5637235486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/3333903537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8bc011440 con 0x563723542b70 2024-02-17T12:54:04.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8c9e71640 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56372354b750 0x563723549150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8ce57c740 1 -- 172.21.15.18:0/3333903537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637231e6720 con 0x563723542b70 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8c8e6f640 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563723542b70 0x563723548c10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40cbc70 tx=0x7fd8b40d1f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e34e2d07b5a795f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8acff9640 10 client.26420 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8acff9640 1 -- 172.21.15.18:0/3333903537 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b409a9a0 con 0x563723542b70 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.360+0000 7fd8acff9640 1 -- 172.21.15.18:0/3333903537 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd8b409d070 con 0x563723542b70 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.361+0000 7fd8acff9640 1 -- 172.21.15.18:0/3333903537 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8b409a7c0 con 0x563723542b70 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.361+0000 7fd8ce57c740 10 client.26420 fetch_fsmap learned FSMap version 32 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.361+0000 7fd8ce57c740 1 -- 172.21.15.18:0/3333903537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x5637234a86a0 con 0x563723542b70 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.361+0000 7fd8acff9640 1 -- 172.21.15.18:0/3333903537 <== mon.2 v2:172.21.15.119:3300/0 4 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd8b401f060 con 0x563723542b70 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.361+0000 7fd8ce57c740 10 client.26420 fetch_fsmap finished waiting for FSMap version 32 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.361+0000 7fd8ce57c740 10 client.26420 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.361+0000 7fd8ce57c740 20 client.26420 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.361+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x56372355b850 0x56372355dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.361+0000 7fd8ce57c740 4 client.26420 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.361+0000 7fd8ce57c740 1 -- 172.21.15.18:0/3333903537 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563723235890 con 0x56372355b850 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.361+0000 7fd8c9670640 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x56372355b850 0x56372355dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.362+0000 7fd8c9670640 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x56372355b850 0x56372355dc60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70e106a3477351cb server_cookie=9ee4a791efda94bb in_seq=0 out_seq=0 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.362+0000 7fd8acff9640 10 client.26420 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.362+0000 7fd8acff9640 1 -- 172.21.15.18:0/3333903537 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563723235890 con 0x56372355b850 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.362+0000 7fd8acff9640 10 client.26420 handle_command_reply: tid=0 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.362+0000 7fd8ce57c740 1 client.26420 shutdown 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.362+0000 7fd8ce57c740 1 -- 172.21.15.18:0/3333903537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563723542b70 msgr2=0x563723548c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.362+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563723542b70 0x563723548c10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40cbc70 tx=0x7fd8b40d1f20 comp rx=0 tx=0).stop 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.363+0000 7fd8ce57c740 1 -- 172.21.15.18:0/3333903537 shutdown_connections 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.363+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x56372355b850 0x56372355dc60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.363+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56372354b750 0x563723549150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.363+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563723542b70 0x563723548c10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.363+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/3333903537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542250 0x5637235486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.363+0000 7fd8acff9640 0 client.26420 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:54:04.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.363+0000 7fd8ce57c740 1 -- 172.21.15.18:0/3333903537 >> 172.21.15.18:0/3333903537 conn(0x56372353c980 msgr2=0x56372353e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.363+0000 7fd8ce57c740 1 -- 172.21.15.18:0/3333903537 shutdown_connections 2024-02-17T12:54:04.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.363+0000 7fd8ce57c740 1 -- 172.21.15.18:0/3333903537 wait complete. 2024-02-17T12:54:04.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.363+0000 7fd8ce57c740 20 client.26420 trim_cache size 0 max 16384 2024-02-17T12:54:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8ce57c740 1 Processor -- start 2024-02-17T12:54:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8ce57c740 1 -- start start 2024-02-17T12:54:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8ce57c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542250 0x56372354ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8ce57c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542b70 0x563723545930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8ce57c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372354b750 0x563723545e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8ce57c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372342f130 con 0x563723542250 2024-02-17T12:54:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8ce57c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563723541540 con 0x56372354b750 2024-02-17T12:54:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8ce57c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563723541c20 con 0x563723542b70 2024-02-17T12:54:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8c9670640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542250 0x56372354ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8c9670640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542250 0x56372354ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40328/0 (socket says 172.21.15.18:40328) 2024-02-17T12:54:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8c9670640 1 -- 172.21.15.18:0/742406756 learned_addr learned my addr 172.21.15.18:0/742406756 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8c8e6f640 1 --2- 172.21.15.18:0/742406756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542b70 0x563723545930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8c9e71640 1 --2- 172.21.15.18:0/742406756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372354b750 0x563723545e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8c9670640 1 -- 172.21.15.18:0/742406756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542b70 msgr2=0x563723545930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8c9670640 1 --2- 172.21.15.18:0/742406756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542b70 0x563723545930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8c9670640 1 -- 172.21.15.18:0/742406756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372354b750 msgr2=0x563723545e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8c9670640 1 --2- 172.21.15.18:0/742406756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372354b750 0x563723545e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8c9670640 1 -- 172.21.15.18:0/742406756 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563723444c20 con 0x563723542250 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.377+0000 7fd8c9670640 1 --2- 172.21.15.18:0/742406756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542250 0x56372354ad90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009900 tx=0x7fd8c0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9f9a8cc83b4f58 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.378+0000 7fd8aeffd640 1 -- 172.21.15.18:0/742406756 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00049e0 con 0x563723542250 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.378+0000 7fd8aeffd640 1 -- 172.21.15.18:0/742406756 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8c0049be0 con 0x563723542250 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.378+0000 7fd8aeffd640 1 -- 172.21.15.18:0/742406756 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00493a0 con 0x563723542250 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.378+0000 7fd8ce57c740 1 -- 172.21.15.18:0/742406756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542250 msgr2=0x56372354ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.378+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/742406756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542250 0x56372354ad90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009900 tx=0x7fd8c0005fd0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 -- 172.21.15.18:0/742406756 shutdown_connections 2024-02-17T12:54:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/742406756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372354b750 0x563723545e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/742406756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542b70 0x563723545930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/742406756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542250 0x56372354ad90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 -- 172.21.15.18:0/742406756 >> 172.21.15.18:0/742406756 conn(0x56372353c980 msgr2=0x563723543990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 -- 172.21.15.18:0/742406756 shutdown_connections 2024-02-17T12:54:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 -- 172.21.15.18:0/742406756 wait complete. 2024-02-17T12:54:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 -- start start 2024-02-17T12:54:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 Processor -- start 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542250 0x563723551950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542b70 0x563723551e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372354b750 0x5637235abe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637235ac380 con 0x563723542b70 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637235ac4f0 con 0x56372354b750 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.379+0000 7fd8ce57c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637235ac6b0 con 0x563723542250 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8ce57c740 10 client.0 _ll_register_callbacks cb 0x7fd8bc009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8c8e6f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542b70 0x563723551e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8c9670640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542250 0x563723551950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8c8e6f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542b70 0x563723551e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40344/0 (socket says 172.21.15.18:40344) 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8c9e71640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372354b750 0x5637235abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/2672071450 learned_addr learned my addr 172.21.15.18:0/2672071450 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/2672071450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542250 msgr2=0x563723551950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8c8e6f640 1 --2- 172.21.15.18:0/2672071450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542250 0x563723551950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/2672071450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372354b750 msgr2=0x5637235abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8c8e6f640 1 --2- 172.21.15.18:0/2672071450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372354b750 0x5637235abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8c8e6f640 1 -- 172.21.15.18:0/2672071450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c410d710 con 0x563723542b70 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8c8e6f640 1 --2- 172.21.15.18:0/2672071450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542b70 0x563723551e90 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd8b409ab60 tx=0x7fd8b40d0730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137a8e0358192d25 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8acff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8acff9640 1 -- 172.21.15.18:0/2672071450 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b40a3370 con 0x563723542b70 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8acff9640 1 -- 172.21.15.18:0/2672071450 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd8b409d040 con 0x563723542b70 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8ce57c740 1 -- 172.21.15.18:0/2672071450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56372353d300 con 0x563723542b70 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8acff9640 1 -- 172.21.15.18:0/2672071450 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8b409a9a0 con 0x563723542b70 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8ce57c740 10 client.17061 fetch_fsmap learned FSMap version 32 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.380+0000 7fd8ce57c740 1 -- 172.21.15.18:0/2672071450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56372346a8c0 con 0x563723542b70 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8acff9640 1 -- 172.21.15.18:0/2672071450 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56372353d300 con 0x563723542b70 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8acff9640 1 -- 172.21.15.18:0/2672071450 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 32) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd8b40b3b90 con 0x563723542b70 2024-02-17T12:54:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8ce57c740 10 client.17061 fetch_fsmap finished waiting for FSMap version 32 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8ce57c740 10 client.17061 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8ce57c740 20 client.17061 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/2672071450 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5637235b3ad0 0x5637235b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8ce57c740 4 client.17061 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8ce57c740 1 -- 172.21.15.18:0/2672071450 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5637231a97e0 con 0x5637235b3ad0 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8c9670640 1 --2- 172.21.15.18:0/2672071450 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5637235b3ad0 0x5637235b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8acff9640 10 client.17061 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8c9670640 1 --2- 172.21.15.18:0/2672071450 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5637235b3ad0 0x5637235b5ec0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5146e884477c5dc5 server_cookie=e4587edbca82ba4f in_seq=0 out_seq=0 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8acff9640 1 -- 172.21.15.18:0/2672071450 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5637231a97e0 con 0x5637235b3ad0 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8acff9640 10 client.17061 handle_command_reply: tid=0 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.381+0000 7fd8ce57c740 1 client.17061 shutdown 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8ce57c740 1 -- 172.21.15.18:0/2672071450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542b70 msgr2=0x563723551e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/2672071450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542b70 0x563723551e90 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd8b409ab60 tx=0x7fd8b40d0730 comp rx=0 tx=0).stop 2024-02-17T12:54:04.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8ce57c740 1 -- 172.21.15.18:0/2672071450 shutdown_connections 2024-02-17T12:54:04.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/2672071450 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5637235b3ad0 0x5637235b5ec0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8acff9640 0 client.17061 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:54:04.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/2672071450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372354b750 0x5637235abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/2672071450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563723542b70 0x563723551e90 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8ce57c740 1 --2- 172.21.15.18:0/2672071450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563723542250 0x563723551950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8ce57c740 1 -- 172.21.15.18:0/2672071450 >> 172.21.15.18:0/2672071450 conn(0x56372353c980 msgr2=0x56372353ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8ce57c740 1 -- 172.21.15.18:0/2672071450 shutdown_connections 2024-02-17T12:54:04.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8ce57c740 1 -- 172.21.15.18:0/2672071450 wait complete. 2024-02-17T12:54:04.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8ce57c740 20 client.17061 trim_cache size 0 max 16384 2024-02-17T12:54:04.379 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:54:04.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/468678922 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8b00a2260 msgr2=0x7fd8b00a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8cb8d4640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8b00a2260 0x7fd8b00a4720 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc020000 tx=0x7fd8bc021000 comp rx=0 tx=0).stop 2024-02-17T12:54:04.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/468678922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c4105330 msgr2=0x7fd8c4102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8cb8d4640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c4105330 0x7fd8c4102230 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400e9c0 tx=0x7fd8b400ee80 comp rx=0 tx=0).stop 2024-02-17T12:54:04.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/468678922 shutdown_connections 2024-02-17T12:54:04.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8cb8d4640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8b00a2260 0x7fd8b00a4720 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8cb8d4640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8c4102770 0x7fd8c41a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8cb8d4640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c4105330 0x7fd8c4102230 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8cb8d4640 1 --2- 172.21.15.18:0/468678922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8c4100260 0x7fd8c4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.382+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/468678922 >> 172.21.15.18:0/468678922 conn(0x7fd8c4077180 msgr2=0x7fd8c40686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.383+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/468678922 shutdown_connections 2024-02-17T12:54:04.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.383+0000 7fd8cb8d4640 1 -- 172.21.15.18:0/468678922 wait complete. 2024-02-17T12:54:04.385 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:54:04.385 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:54:04.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:04 smithi018 ceph-mon[32625]: pgmap v128: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 229 op/s 2024-02-17T12:54:04.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:04 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3937896094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:54:04.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.550+0000 7efecf5f4640 1 -- 172.21.15.18:0/435222660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efec8100c10 msgr2=0x7efec8104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.550+0000 7efecf5f4640 1 --2- 172.21.15.18:0/435222660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efec8100c10 0x7efec8104c50 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7efec4009cd0 tx=0x7efec402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.550+0000 7efecf5f4640 1 -- 172.21.15.18:0/435222660 shutdown_connections 2024-02-17T12:54:04.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.550+0000 7efecf5f4640 1 --2- 172.21.15.18:0/435222660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efec8105330 0x7efec810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.550+0000 7efecf5f4640 1 --2- 172.21.15.18:0/435222660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efec8100c10 0x7efec8104c50 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.550+0000 7efecf5f4640 1 --2- 172.21.15.18:0/435222660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec8100260 0x7efec8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.550+0000 7efecf5f4640 1 -- 172.21.15.18:0/435222660 >> 172.21.15.18:0/435222660 conn(0x7efec8077180 msgr2=0x7efec8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.550+0000 7efecf5f4640 1 -- 172.21.15.18:0/435222660 shutdown_connections 2024-02-17T12:54:04.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.550+0000 7efecf5f4640 1 -- 172.21.15.18:0/435222660 wait complete. 2024-02-17T12:54:04.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efecf5f4640 1 Processor -- start 2024-02-17T12:54:04.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efecf5f4640 1 -- start start 2024-02-17T12:54:04.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efecf5f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efec8100260 0x7efec819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efecf5f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec8105330 0x7efec819db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efecf5f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efec8197690 0x7efec8197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efecf5f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec8198160 con 0x7efec8100260 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efecf5f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec81982d0 con 0x7efec8105330 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efecf5f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec8198490 con 0x7efec8197690 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efecd390640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efec8100260 0x7efec819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efecd390640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efec8100260 0x7efec819d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40362/0 (socket says 172.21.15.18:40362) 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efecd390640 1 -- 172.21.15.18:0/3178864382 learned_addr learned my addr 172.21.15.18:0/3178864382 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efeccb8f640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec8105330 0x7efec819db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.551+0000 7efecdb91640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efec8197690 0x7efec8197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.552+0000 7efecd390640 1 -- 172.21.15.18:0/3178864382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efec8197690 msgr2=0x7efec8197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.552+0000 7efecd390640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efec8197690 0x7efec8197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.552+0000 7efecd390640 1 -- 172.21.15.18:0/3178864382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec8105330 msgr2=0x7efec819db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.552+0000 7efecd390640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec8105330 0x7efec819db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.552+0000 7efecd390640 1 -- 172.21.15.18:0/3178864382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec0009490 con 0x7efec8100260 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.552+0000 7efeccb8f640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec8105330 0x7efec819db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.552+0000 7efecd390640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efec8100260 0x7efec819d5d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efec000ea60 tx=0x7efec000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.552+0000 7efebe7fc640 1 -- 172.21.15.18:0/3178864382 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000cde0 con 0x7efec8100260 2024-02-17T12:54:04.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.552+0000 7efebe7fc640 1 -- 172.21.15.18:0/3178864382 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efec00035f0 con 0x7efec8100260 2024-02-17T12:54:04.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.552+0000 7efecf5f4640 1 -- 172.21.15.18:0/3178864382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec4009900 con 0x7efec8100260 2024-02-17T12:54:04.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.552+0000 7efebe7fc640 1 -- 172.21.15.18:0/3178864382 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0010690 con 0x7efec8100260 2024-02-17T12:54:04.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.552+0000 7efecf5f4640 1 -- 172.21.15.18:0/3178864382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec81a4780 con 0x7efec8100260 2024-02-17T12:54:04.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.554+0000 7efebe7fc640 1 -- 172.21.15.18:0/3178864382 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7efec0003af0 con 0x7efec8100260 2024-02-17T12:54:04.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.555+0000 7efed21c6740 1 Processor -- start 2024-02-17T12:54:04.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.555+0000 7efed21c6740 1 -- start start 2024-02-17T12:54:04.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.555+0000 7efebe7fc640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efeb40a2230 0x7efeb40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.555+0000 7efed21c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 0x565443864630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.555+0000 7efed21c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864b70 0x56544386d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.555+0000 7efed21c6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 0x56544386fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efed21c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544381f960 con 0x565443864250 2024-02-17T12:54:04.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efed21c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654437d3310 con 0x565443864b70 2024-02-17T12:54:04.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efed21c6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654437d0840 con 0x56544386d750 2024-02-17T12:54:04.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efecd390640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 0x565443864630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efecdb91640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efeb40a2230 0x7efeb40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efecd390640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 0x565443864630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40366/0 (socket says 172.21.15.18:40366) 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efecd390640 1 -- 172.21.15.18:0/287373623 learned_addr learned my addr 172.21.15.18:0/287373623 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efeccb8f640 1 --2- 172.21.15.18:0/287373623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864b70 0x56544386d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efebe7fc640 1 -- 172.21.15.18:0/3178864382 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7efec0014070 con 0x7efec8100260 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efecdb91640 1 --2- 172.21.15.18:0/287373623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 0x56544386fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efecd390640 1 -- 172.21.15.18:0/287373623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 msgr2=0x56544386fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efecd390640 1 --2- 172.21.15.18:0/287373623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 0x56544386fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efecd390640 1 -- 172.21.15.18:0/287373623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864b70 msgr2=0x56544386d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efecd390640 1 --2- 172.21.15.18:0/287373623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864b70 0x56544386d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efecd390640 1 -- 172.21.15.18:0/287373623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56544378e060 con 0x565443864250 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efeccb8f640 1 --2- 172.21.15.18:0/287373623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864b70 0x56544386d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efecdb91640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efeb40a2230 0x7efeb40a46f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efeb800b590 tx=0x7efeb800ba20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efecd390640 1 --2- 172.21.15.18:0/287373623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 0x565443864630 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7efec0004610 tx=0x7efec0013fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f38632f210ebe61 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efeb37fe640 1 -- 172.21.15.18:0/287373623 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00a0560 con 0x565443864250 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efeb37fe640 1 -- 172.21.15.18:0/287373623 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efec009ff60 con 0x565443864250 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.556+0000 7efeb37fe640 1 -- 172.21.15.18:0/287373623 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec009c4e0 con 0x565443864250 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.557+0000 7efed21c6740 1 -- 172.21.15.18:0/287373623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 msgr2=0x565443864630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.557+0000 7efed21c6740 1 --2- 172.21.15.18:0/287373623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 0x565443864630 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7efec0004610 tx=0x7efec0013fa0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.557+0000 7efed21c6740 1 -- 172.21.15.18:0/287373623 shutdown_connections 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.557+0000 7efed21c6740 1 --2- 172.21.15.18:0/287373623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 0x56544386fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.557+0000 7efed21c6740 1 --2- 172.21.15.18:0/287373623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864b70 0x56544386d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.557+0000 7efed21c6740 1 --2- 172.21.15.18:0/287373623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 0x565443864630 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.557+0000 7efed21c6740 1 -- 172.21.15.18:0/287373623 >> 172.21.15.18:0/287373623 conn(0x56544385e980 msgr2=0x56544385ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.557+0000 7efed21c6740 1 -- 172.21.15.18:0/287373623 shutdown_connections 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.557+0000 7efed21c6740 1 -- 172.21.15.18:0/287373623 wait complete. 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efed21c6740 1 -- start start 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efed21c6740 1 Processor -- start 2024-02-17T12:54:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efed21c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864250 0x56544386a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efed21c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 0x56544386abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efed21c6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 0x56544386b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efed21c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654437cc020 con 0x565443864b70 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efed21c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544349f430 con 0x565443864250 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efed21c6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544345be00 con 0x56544386d750 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efed21c6740 10 client.0 _ll_register_callbacks cb 0x565443773cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efeccb8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 0x56544386abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efeccb8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 0x56544386abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40374/0 (socket says 172.21.15.18:40374) 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efeccb8f640 1 -- 172.21.15.18:0/3046838003 learned_addr learned my addr 172.21.15.18:0/3046838003 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efecd390640 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864250 0x56544386a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efecdb91640 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 0x56544386b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.558+0000 7efeccb8f640 1 -- 172.21.15.18:0/3046838003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 msgr2=0x56544386b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efeccb8f640 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 0x56544386b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efeccb8f640 1 -- 172.21.15.18:0/3046838003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864250 msgr2=0x56544386a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efeccb8f640 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864250 0x56544386a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efeccb8f640 1 -- 172.21.15.18:0/3046838003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec81a4780 con 0x565443864b70 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efecd390640 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864250 0x56544386a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efeccb8f640 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 0x56544386abf0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efec806a8e0 tx=0x7efec4034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683397e585a60bf8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efeb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efeb0ff9640 1 -- 172.21.15.18:0/3046838003 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4032ac0 con 0x565443864b70 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efeb0ff9640 1 -- 172.21.15.18:0/3046838003 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7efec4046070 con 0x565443864b70 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efed21c6740 1 -- 172.21.15.18:0/3046838003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654434cb7e0 con 0x565443864b70 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efeb0ff9640 1 -- 172.21.15.18:0/3046838003 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efec4032eb0 con 0x565443864b70 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efed21c6740 10 client.17073 fetch_fsmap learned FSMap version 33 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efed21c6740 1 -- 172.21.15.18:0/3046838003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5654437ca6a0 con 0x565443864b70 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efeb0ff9640 1 -- 172.21.15.18:0/3046838003 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5654434cb7e0 con 0x565443864b70 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efeb0ff9640 1 -- 172.21.15.18:0/3046838003 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efec4053430 con 0x565443864b70 2024-02-17T12:54:04.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efed21c6740 10 client.17073 fetch_fsmap finished waiting for FSMap version 33 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efed21c6740 10 client.17073 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efed21c6740 20 client.17073 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efed21c6740 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56544387d9c0 0x56544387fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efed21c6740 4 client.17073 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.559+0000 7efed21c6740 1 -- 172.21.15.18:0/3046838003 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565443508720 con 0x56544387d9c0 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.560+0000 7efecd390640 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56544387d9c0 0x56544387fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.560+0000 7efecd390640 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56544387d9c0 0x56544387fde0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d77f83438b0ae98c server_cookie=e0d7c5bd68929866 in_seq=0 out_seq=0 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.560+0000 7efeb0ff9640 10 client.17073 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efeb0ff9640 1 -- 172.21.15.18:0/3046838003 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565443508720 con 0x56544387d9c0 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efeb0ff9640 10 client.17073 handle_command_reply: tid=0 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efed21c6740 1 client.17073 shutdown 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efed21c6740 1 -- 172.21.15.18:0/3046838003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 msgr2=0x56544386abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efed21c6740 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 0x56544386abf0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efec806a8e0 tx=0x7efec4034cc0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efed21c6740 1 -- 172.21.15.18:0/3046838003 shutdown_connections 2024-02-17T12:54:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efed21c6740 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56544387d9c0 0x56544387fde0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efed21c6740 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 0x56544386b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efed21c6740 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 0x56544386abf0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efed21c6740 1 --2- 172.21.15.18:0/3046838003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864250 0x56544386a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efed21c6740 1 -- 172.21.15.18:0/3046838003 >> 172.21.15.18:0/3046838003 conn(0x56544385e980 msgr2=0x5654438606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efeb0ff9640 0 client.17073 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:54:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efed21c6740 1 -- 172.21.15.18:0/3046838003 shutdown_connections 2024-02-17T12:54:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efed21c6740 1 -- 172.21.15.18:0/3046838003 wait complete. 2024-02-17T12:54:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.561+0000 7efed21c6740 20 client.17073 trim_cache size 0 max 16384 2024-02-17T12:54:04.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.574+0000 7efed21c6740 1 Processor -- start 2024-02-17T12:54:04.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.574+0000 7efed21c6740 1 -- start start 2024-02-17T12:54:04.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.574+0000 7efed21c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 0x56544386cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.574+0000 7efed21c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864b70 0x56544386af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.574+0000 7efed21c6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 0x565443867910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.574+0000 7efed21c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x565443751130 con 0x565443864250 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.574+0000 7efed21c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544386ff00 con 0x565443864b70 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.574+0000 7efed21c6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544385f980 con 0x56544386d750 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.574+0000 7efecd390640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 0x56544386cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efeccb8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864b70 0x56544386af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efecd390640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 0x56544386cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40380/0 (socket says 172.21.15.18:40380) 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efecd390640 1 -- 172.21.15.18:0/1285927781 learned_addr learned my addr 172.21.15.18:0/1285927781 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efecdb91640 1 --2- 172.21.15.18:0/1285927781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 0x565443867910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efecd390640 1 -- 172.21.15.18:0/1285927781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 msgr2=0x565443867910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efecd390640 1 --2- 172.21.15.18:0/1285927781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 0x565443867910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efecd390640 1 -- 172.21.15.18:0/1285927781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864b70 msgr2=0x56544386af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efecd390640 1 --2- 172.21.15.18:0/1285927781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864b70 0x56544386af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efecd390640 1 -- 172.21.15.18:0/1285927781 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565443766c20 con 0x565443864250 2024-02-17T12:54:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efeccb8f640 1 --2- 172.21.15.18:0/1285927781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864b70 0x56544386af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efecd390640 1 --2- 172.21.15.18:0/1285927781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 0x56544386cde0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7efec00978c0 tx=0x7efec0003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24cbca52ebab0f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efeb2ffd640 1 -- 172.21.15.18:0/1285927781 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00b9070 con 0x565443864250 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efeb2ffd640 1 -- 172.21.15.18:0/1285927781 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efec0097190 con 0x565443864250 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.575+0000 7efeb2ffd640 1 -- 172.21.15.18:0/1285927781 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00992c0 con 0x565443864250 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.576+0000 7efed21c6740 1 -- 172.21.15.18:0/1285927781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 msgr2=0x56544386cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.576+0000 7efed21c6740 1 --2- 172.21.15.18:0/1285927781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 0x56544386cde0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7efec00978c0 tx=0x7efec0003ef0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.576+0000 7efed21c6740 1 -- 172.21.15.18:0/1285927781 shutdown_connections 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.576+0000 7efed21c6740 1 --2- 172.21.15.18:0/1285927781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56544386d750 0x565443867910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.576+0000 7efed21c6740 1 --2- 172.21.15.18:0/1285927781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565443864b70 0x56544386af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.576+0000 7efed21c6740 1 --2- 172.21.15.18:0/1285927781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864250 0x56544386cde0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.576+0000 7efed21c6740 1 -- 172.21.15.18:0/1285927781 >> 172.21.15.18:0/1285927781 conn(0x56544385e980 msgr2=0x565443865990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.576+0000 7efed21c6740 1 -- 172.21.15.18:0/1285927781 shutdown_connections 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.576+0000 7efed21c6740 1 -- 172.21.15.18:0/1285927781 wait complete. 2024-02-17T12:54:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.576+0000 7efed21c6740 1 -- start start 2024-02-17T12:54:04.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.576+0000 7efed21c6740 1 Processor -- start 2024-02-17T12:54:04.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efed21c6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565443864250 0x565443874890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efed21c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 0x565443872c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efed21c6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56544386d750 0x565443873170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efed21c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x565443873840 con 0x565443864b70 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efed21c6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654438739b0 con 0x56544386d750 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efed21c6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654438d3ef0 con 0x565443864250 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efed21c6740 10 client.0 _ll_register_callbacks cb 0x7efec4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efeccb8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 0x565443872c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efeccb8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 0x565443872c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40394/0 (socket says 172.21.15.18:40394) 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efeccb8f640 1 -- 172.21.15.18:0/3706547996 learned_addr learned my addr 172.21.15.18:0/3706547996 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efecdb91640 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56544386d750 0x565443873170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efecd390640 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565443864250 0x565443874890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efeccb8f640 1 -- 172.21.15.18:0/3706547996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565443864250 msgr2=0x565443874890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efeccb8f640 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565443864250 0x565443874890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efeccb8f640 1 -- 172.21.15.18:0/3706547996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56544386d750 msgr2=0x565443873170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efeccb8f640 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56544386d750 0x565443873170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efeccb8f640 1 -- 172.21.15.18:0/3706547996 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654437ca6a0 con 0x565443864b70 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efecdb91640 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56544386d750 0x565443873170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efeccb8f640 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 0x565443872c30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7efec4002230 tx=0x7efec40095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c3e140e8e41729 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efeb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efeb0ff9640 1 -- 172.21.15.18:0/3706547996 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4034d30 con 0x565443864b70 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efeb0ff9640 1 -- 172.21.15.18:0/3706547996 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7efec4046040 con 0x565443864b70 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.577+0000 7efed21c6740 1 -- 172.21.15.18:0/3706547996 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efec009ff60 con 0x565443864b70 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.578+0000 7efeb0ff9640 1 -- 172.21.15.18:0/3706547996 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efec40328c0 con 0x565443864b70 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.578+0000 7efed21c6740 10 client.17079 fetch_fsmap learned FSMap version 33 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.578+0000 7efed21c6740 1 -- 172.21.15.18:0/3706547996 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7efec00b2040 con 0x565443864b70 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.578+0000 7efeb0ff9640 1 -- 172.21.15.18:0/3706547996 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efec009ff60 con 0x565443864b70 2024-02-17T12:54:04.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.578+0000 7efeb0ff9640 1 -- 172.21.15.18:0/3706547996 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efec4004330 con 0x565443864b70 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.578+0000 7efed21c6740 10 client.17079 fetch_fsmap finished waiting for FSMap version 33 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.578+0000 7efed21c6740 10 client.17079 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.578+0000 7efed21c6740 20 client.17079 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.578+0000 7efed21c6740 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5654438d5340 0x5654438d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.578+0000 7efed21c6740 4 client.17079 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.578+0000 7efed21c6740 1 -- 172.21.15.18:0/3706547996 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565443557890 con 0x5654438d5340 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.578+0000 7efecd390640 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5654438d5340 0x5654438d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efeb0ff9640 10 client.17079 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efecd390640 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5654438d5340 0x5654438d7760 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=daae434d62d38db4 server_cookie=87b34848f7d296d7 in_seq=0 out_seq=0 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efeb0ff9640 1 -- 172.21.15.18:0/3706547996 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565443557890 con 0x5654438d5340 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efeb0ff9640 10 client.17079 handle_command_reply: tid=0 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efed21c6740 1 client.17079 shutdown 2024-02-17T12:54:04.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efed21c6740 1 -- 172.21.15.18:0/3706547996 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 msgr2=0x565443872c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efed21c6740 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 0x565443872c30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7efec4002230 tx=0x7efec40095a0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efed21c6740 1 -- 172.21.15.18:0/3706547996 shutdown_connections 2024-02-17T12:54:04.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efed21c6740 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5654438d5340 0x5654438d7760 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efed21c6740 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56544386d750 0x565443873170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efed21c6740 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565443864b70 0x565443872c30 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efed21c6740 1 --2- 172.21.15.18:0/3706547996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565443864250 0x565443874890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efed21c6740 1 -- 172.21.15.18:0/3706547996 >> 172.21.15.18:0/3706547996 conn(0x56544385e980 msgr2=0x56544385fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efeb0ff9640 0 client.17079 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:54:04.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efed21c6740 1 -- 172.21.15.18:0/3706547996 shutdown_connections 2024-02-17T12:54:04.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efed21c6740 1 -- 172.21.15.18:0/3706547996 wait complete. 2024-02-17T12:54:04.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.579+0000 7efed21c6740 20 client.17079 trim_cache size 0 max 16384 2024-02-17T12:54:04.577 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:54:04.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.580+0000 7efecf5f4640 1 -- 172.21.15.18:0/3178864382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efeb40a2230 msgr2=0x7efeb40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.580+0000 7efecf5f4640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efeb40a2230 0x7efeb40a46f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efeb800b590 tx=0x7efeb800ba20 comp rx=0 tx=0).stop 2024-02-17T12:54:04.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.580+0000 7efecf5f4640 1 -- 172.21.15.18:0/3178864382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efec8100260 msgr2=0x7efec819d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.580+0000 7efecf5f4640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efec8100260 0x7efec819d5d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efec000ea60 tx=0x7efec000ef20 comp rx=0 tx=0).stop 2024-02-17T12:54:04.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.580+0000 7efecf5f4640 1 -- 172.21.15.18:0/3178864382 shutdown_connections 2024-02-17T12:54:04.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.580+0000 7efecf5f4640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efeb40a2230 0x7efeb40a46f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.580+0000 7efecf5f4640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efec8197690 0x7efec8197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.580+0000 7efecf5f4640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec8105330 0x7efec819db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.580+0000 7efecf5f4640 1 --2- 172.21.15.18:0/3178864382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efec8100260 0x7efec819d5d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.580+0000 7efecf5f4640 1 -- 172.21.15.18:0/3178864382 >> 172.21.15.18:0/3178864382 conn(0x7efec8077180 msgr2=0x7efec8068760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.580+0000 7efecf5f4640 1 -- 172.21.15.18:0/3178864382 shutdown_connections 2024-02-17T12:54:04.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.580+0000 7efecf5f4640 1 -- 172.21.15.18:0/3178864382 wait complete. 2024-02-17T12:54:04.582 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:54:04.582 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:54:04.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.703+0000 7f895bfe2640 1 -- 172.21.15.18:0/2162029493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8954101390 msgr2=0x7f8954101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.703+0000 7f895bfe2640 1 --2- 172.21.15.18:0/2162029493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8954101390 0x7f8954101770 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8950009cd0 tx=0x7f895002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.703+0000 7f895bfe2640 1 -- 172.21.15.18:0/2162029493 shutdown_connections 2024-02-17T12:54:04.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.703+0000 7f895bfe2640 1 --2- 172.21.15.18:0/2162029493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8954106460 0x7f895410e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.703+0000 7f895bfe2640 1 --2- 172.21.15.18:0/2162029493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8954101d40 0x7f8954105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.703+0000 7f895bfe2640 1 --2- 172.21.15.18:0/2162029493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8954101390 0x7f8954101770 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.703+0000 7f895bfe2640 1 -- 172.21.15.18:0/2162029493 >> 172.21.15.18:0/2162029493 conn(0x7f89540fcbb0 msgr2=0x7f89540fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.704+0000 7f895bfe2640 1 -- 172.21.15.18:0/2162029493 shutdown_connections 2024-02-17T12:54:04.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.704+0000 7f895bfe2640 1 -- 172.21.15.18:0/2162029493 wait complete. 2024-02-17T12:54:04.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.704+0000 7f895bfe2640 1 Processor -- start 2024-02-17T12:54:04.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.704+0000 7f895bfe2640 1 -- start start 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.704+0000 7f895bfe2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8954101390 0x7f895419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.704+0000 7f895bfe2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8954106460 0x7f895419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f895bfe2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8954197660 0x7f8954197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f895bfe2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954198130 con 0x7f8954197660 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f895bfe2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89541982a0 con 0x7f8954101390 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f895bfe2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954198460 con 0x7f8954106460 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f895a57f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8954197660 0x7f8954197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f8959d7e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8954101390 0x7f895419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f8959d7e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8954101390 0x7f895419d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:60190/0 (socket says 172.21.15.18:60190) 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f8959d7e640 1 -- 172.21.15.18:0/79276037 learned_addr learned my addr 172.21.15.18:0/79276037 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f895a57f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8954197660 0x7f8954197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40408/0 (socket says 172.21.15.18:40408) 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f895957d640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8954106460 0x7f895419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f8959d7e640 1 -- 172.21.15.18:0/79276037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8954106460 msgr2=0x7f895419dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f8959d7e640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8954106460 0x7f895419dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f8959d7e640 1 -- 172.21.15.18:0/79276037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8954197660 msgr2=0x7f8954197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f8959d7e640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8954197660 0x7f8954197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f8959d7e640 1 -- 172.21.15.18:0/79276037 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950009900 con 0x7f8954101390 2024-02-17T12:54:04.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f8959d7e640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8954101390 0x7f895419d5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8950004af0 tx=0x7f89500325c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.705+0000 7f894affd640 1 -- 172.21.15.18:0/79276037 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950034e20 con 0x7f8954101390 2024-02-17T12:54:04.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.706+0000 7f895bfe2640 1 -- 172.21.15.18:0/79276037 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89541a43a0 con 0x7f8954101390 2024-02-17T12:54:04.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.706+0000 7f894affd640 1 -- 172.21.15.18:0/79276037 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8950003e50 con 0x7f8954101390 2024-02-17T12:54:04.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.706+0000 7f894affd640 1 -- 172.21.15.18:0/79276037 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950004880 con 0x7f8954101390 2024-02-17T12:54:04.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.706+0000 7f895bfe2640 1 -- 172.21.15.18:0/79276037 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89541a4870 con 0x7f8954101390 2024-02-17T12:54:04.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.708+0000 7f894affd640 1 -- 172.21.15.18:0/79276037 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f895000a040 con 0x7f8954101390 2024-02-17T12:54:04.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.708+0000 7f895ebb4740 1 Processor -- start 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.708+0000 7f895ebb4740 1 -- start start 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.708+0000 7f895ebb4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 0x55e2e8de7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.708+0000 7f895ebb4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8df0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.708+0000 7f895ebb4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8df0750 0x55e2e8df2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.708+0000 7f895ebb4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e8da2960 con 0x55e2e8de7250 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.708+0000 7f895ebb4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e8d56310 con 0x55e2e8de7b70 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.708+0000 7f895ebb4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e8d53840 con 0x55e2e8df0750 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f8959d7e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8df0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f895a57f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 0x55e2e8de7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f895957d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8df0750 0x55e2e8df2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f895a57f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 0x55e2e8de7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40416/0 (socket says 172.21.15.18:40416) 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f8959d7e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8df0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:60192/0 (socket says 172.21.15.18:60192) 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f895a57f640 1 -- 172.21.15.18:0/3211755784 learned_addr learned my addr 172.21.15.18:0/3211755784 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f894affd640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f89400a2230 0x7f89400a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f895a57f640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f89400a2230 0x7f89400a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f895957d640 1 -- 172.21.15.18:0/3211755784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 msgr2=0x55e2e8df0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f895957d640 1 --2- 172.21.15.18:0/3211755784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8df0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f895957d640 1 -- 172.21.15.18:0/3211755784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 msgr2=0x55e2e8de7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f895957d640 1 --2- 172.21.15.18:0/3211755784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 0x55e2e8de7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f895957d640 1 -- 172.21.15.18:0/3211755784 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2e8d11060 con 0x55e2e8df0750 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f894affd640 1 -- 172.21.15.18:0/79276037 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f89500f0aa0 con 0x7f8954101390 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f895957d640 1 --2- 172.21.15.18:0/3211755784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8df0750 0x55e2e8df2b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8954067f10 tx=0x7f8944011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6291051db38b4060 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f895a57f640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f89400a2230 0x7f89400a46f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f894c0058c0 tx=0x7f894c004730 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f893ffff640 1 -- 172.21.15.18:0/3211755784 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944004230 con 0x55e2e8df0750 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f893ffff640 1 -- 172.21.15.18:0/3211755784 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8944009e10 con 0x55e2e8df0750 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.709+0000 7f893ffff640 1 -- 172.21.15.18:0/3211755784 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944004c70 con 0x55e2e8df0750 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.710+0000 7f895ebb4740 1 -- 172.21.15.18:0/3211755784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8df0750 msgr2=0x55e2e8df2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.710+0000 7f895ebb4740 1 --2- 172.21.15.18:0/3211755784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8df0750 0x55e2e8df2b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8954067f10 tx=0x7f8944011810 comp rx=0 tx=0).stop 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.710+0000 7f895ebb4740 1 -- 172.21.15.18:0/3211755784 shutdown_connections 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.710+0000 7f895ebb4740 1 --2- 172.21.15.18:0/3211755784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8df0750 0x55e2e8df2b40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.710+0000 7f895ebb4740 1 --2- 172.21.15.18:0/3211755784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8df0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.710+0000 7f895ebb4740 1 --2- 172.21.15.18:0/3211755784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 0x55e2e8de7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.710+0000 7f895ebb4740 1 -- 172.21.15.18:0/3211755784 >> 172.21.15.18:0/3211755784 conn(0x55e2e8de1980 msgr2=0x55e2e8de1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.710+0000 7f895ebb4740 1 -- 172.21.15.18:0/3211755784 shutdown_connections 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.710+0000 7f895ebb4740 1 -- 172.21.15.18:0/3211755784 wait complete. 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895ebb4740 1 -- start start 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895ebb4740 1 Processor -- start 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895ebb4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8de7250 0x55e2e8ded760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895ebb4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8dedca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895ebb4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 0x55e2e8dee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895ebb4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e8d4f020 con 0x55e2e8df0750 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895ebb4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e8a22430 con 0x55e2e8de7b70 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895ebb4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e89dee00 con 0x55e2e8de7250 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895ebb4740 10 client.0 _ll_register_callbacks cb 0x55e2e8cf6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f8959d7e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8de7250 0x55e2e8ded760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895a57f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 0x55e2e8dee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895a57f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 0x55e2e8dee1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40418/0 (socket says 172.21.15.18:40418) 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f8959d7e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8de7250 0x55e2e8ded760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:59856/0 (socket says 172.21.15.18:59856) 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895957d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8dedca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.711+0000 7f895a57f640 1 -- 172.21.15.18:0/910095141 learned_addr learned my addr 172.21.15.18:0/910095141 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f895a57f640 1 -- 172.21.15.18:0/910095141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8de7250 msgr2=0x55e2e8ded760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f895a57f640 1 --2- 172.21.15.18:0/910095141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8de7250 0x55e2e8ded760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f895a57f640 1 -- 172.21.15.18:0/910095141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 msgr2=0x55e2e8dedca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f895a57f640 1 --2- 172.21.15.18:0/910095141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8dedca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f895a57f640 1 -- 172.21.15.18:0/910095141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8944009830 con 0x55e2e8df0750 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f8959d7e640 1 --2- 172.21.15.18:0/910095141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8de7250 0x55e2e8ded760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f895a57f640 1 --2- 172.21.15.18:0/910095141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 0x55e2e8dee1e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f894c012930 tx=0x7f894c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcacb6922b79246 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f893d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f893d7fa640 1 -- 172.21.15.18:0/910095141 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c011cd0 con 0x55e2e8df0750 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f893d7fa640 1 -- 172.21.15.18:0/910095141 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f894c011e30 con 0x55e2e8df0750 2024-02-17T12:54:04.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f895ebb4740 1 -- 172.21.15.18:0/910095141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2e8a4e7e0 con 0x55e2e8df0750 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f893d7fa640 1 -- 172.21.15.18:0/910095141 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f894c021410 con 0x55e2e8df0750 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f895ebb4740 10 client.17091 fetch_fsmap learned FSMap version 33 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f895ebb4740 1 -- 172.21.15.18:0/910095141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e2e8d4d6a0 con 0x55e2e8df0750 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.712+0000 7f893d7fa640 1 -- 172.21.15.18:0/910095141 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e2e8a4e7e0 con 0x55e2e8df0750 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.713+0000 7f893d7fa640 1 -- 172.21.15.18:0/910095141 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f894c029710 con 0x55e2e8df0750 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.713+0000 7f895ebb4740 10 client.17091 fetch_fsmap finished waiting for FSMap version 33 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.713+0000 7f895ebb4740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.713+0000 7f895ebb4740 20 client.17091 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.713+0000 7f895ebb4740 1 --2- 172.21.15.18:0/910095141 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e2e8e00b80 0x55e2e8e02fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.713+0000 7f895ebb4740 4 client.17091 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.713+0000 7f895ebb4740 1 -- 172.21.15.18:0/910095141 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2e8a8b720 con 0x55e2e8e00b80 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.713+0000 7f8959d7e640 1 --2- 172.21.15.18:0/910095141 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e2e8e00b80 0x55e2e8e02fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.713+0000 7f893d7fa640 10 client.17091 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:04.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.713+0000 7f8959d7e640 1 --2- 172.21.15.18:0/910095141 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e2e8e00b80 0x55e2e8e02fa0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f3c9556082b554 server_cookie=2cfbfdde73c0ef42 in_seq=0 out_seq=0 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f893d7fa640 1 -- 172.21.15.18:0/910095141 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e2e8a8b720 con 0x55e2e8e00b80 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f893d7fa640 10 client.17091 handle_command_reply: tid=0 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f895ebb4740 1 client.17091 shutdown 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f895ebb4740 1 -- 172.21.15.18:0/910095141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 msgr2=0x55e2e8dee1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f895ebb4740 1 --2- 172.21.15.18:0/910095141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 0x55e2e8dee1e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f894c012930 tx=0x7f894c0046c0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f895ebb4740 1 -- 172.21.15.18:0/910095141 shutdown_connections 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f895ebb4740 1 --2- 172.21.15.18:0/910095141 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e2e8e00b80 0x55e2e8e02fa0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f893d7fa640 0 client.17091 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f895ebb4740 1 --2- 172.21.15.18:0/910095141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 0x55e2e8dee1e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f895ebb4740 1 --2- 172.21.15.18:0/910095141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8dedca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f895ebb4740 1 --2- 172.21.15.18:0/910095141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8de7250 0x55e2e8ded760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f895ebb4740 1 -- 172.21.15.18:0/910095141 >> 172.21.15.18:0/910095141 conn(0x55e2e8de1980 msgr2=0x55e2e8de36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f895ebb4740 1 -- 172.21.15.18:0/910095141 shutdown_connections 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.714+0000 7f895ebb4740 1 -- 172.21.15.18:0/910095141 wait complete. 2024-02-17T12:54:04.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.715+0000 7f895ebb4740 20 client.17091 trim_cache size 0 max 16384 2024-02-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.728+0000 7f895ebb4740 1 Processor -- start 2024-02-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.728+0000 7f895ebb4740 1 -- start start 2024-02-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.728+0000 7f895ebb4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 0x55e2e8defe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895ebb4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8dee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895ebb4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8df0750 0x55e2e8dea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895ebb4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e8cd4130 con 0x55e2e8de7250 2024-02-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895ebb4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e8de6c90 con 0x55e2e8de7b70 2024-02-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895ebb4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e8e081f0 con 0x55e2e8df0750 2024-02-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f8959d7e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 0x55e2e8defe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895957d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8dee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f8959d7e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 0x55e2e8defe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40424/0 (socket says 172.21.15.18:40424) 2024-02-17T12:54:04.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f8959d7e640 1 -- 172.21.15.18:0/1047615276 learned_addr learned my addr 172.21.15.18:0/1047615276 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895a57f640 1 --2- 172.21.15.18:0/1047615276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8df0750 0x55e2e8dea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895957d640 1 -- 172.21.15.18:0/1047615276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8df0750 msgr2=0x55e2e8dea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895957d640 1 --2- 172.21.15.18:0/1047615276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8df0750 0x55e2e8dea9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895957d640 1 -- 172.21.15.18:0/1047615276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 msgr2=0x55e2e8defe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895957d640 1 --2- 172.21.15.18:0/1047615276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 0x55e2e8defe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895957d640 1 -- 172.21.15.18:0/1047615276 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2e8ce9c20 con 0x55e2e8de7b70 2024-02-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f8959d7e640 1 --2- 172.21.15.18:0/1047615276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 0x55e2e8defe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895a57f640 1 --2- 172.21.15.18:0/1047615276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8df0750 0x55e2e8dea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f895957d640 1 --2- 172.21.15.18:0/1047615276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8dee000 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8944004620 tx=0x7f89440048b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8832d1605fd73ea5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.729+0000 7f893f7fe640 1 -- 172.21.15.18:0/1047615276 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944024680 con 0x55e2e8de7b70 2024-02-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.730+0000 7f893f7fe640 1 -- 172.21.15.18:0/1047615276 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89440046d0 con 0x55e2e8de7b70 2024-02-17T12:54:04.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.730+0000 7f893f7fe640 1 -- 172.21.15.18:0/1047615276 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944019c30 con 0x55e2e8de7b70 2024-02-17T12:54:04.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.730+0000 7f895ebb4740 1 -- 172.21.15.18:0/1047615276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 msgr2=0x55e2e8dee000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.730+0000 7f895ebb4740 1 --2- 172.21.15.18:0/1047615276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8dee000 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8944004620 tx=0x7f89440048b0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.730+0000 7f895ebb4740 1 -- 172.21.15.18:0/1047615276 shutdown_connections 2024-02-17T12:54:04.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.730+0000 7f895ebb4740 1 --2- 172.21.15.18:0/1047615276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8df0750 0x55e2e8dea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.730+0000 7f895ebb4740 1 --2- 172.21.15.18:0/1047615276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7b70 0x55e2e8dee000 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.730+0000 7f895ebb4740 1 --2- 172.21.15.18:0/1047615276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8de7250 0x55e2e8defe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.730+0000 7f895ebb4740 1 -- 172.21.15.18:0/1047615276 >> 172.21.15.18:0/1047615276 conn(0x55e2e8de1980 msgr2=0x55e2e8de8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.730+0000 7f895ebb4740 1 -- 172.21.15.18:0/1047615276 shutdown_connections 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.730+0000 7f895ebb4740 1 -- 172.21.15.18:0/1047615276 wait complete. 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f895ebb4740 1 -- start start 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f895ebb4740 1 Processor -- start 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f895ebb4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7250 0x55e2e8df6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f895ebb4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8de7b70 0x55e2e8df6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f895ebb4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 0x55e2e8e570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f895ebb4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e8e57610 con 0x55e2e8df0750 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f895ebb4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e8e57750 con 0x55e2e8de7250 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f895ebb4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e8e57890 con 0x55e2e8de7b70 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f895ebb4740 10 client.0 _ll_register_callbacks cb 0x7f8944009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f895a57f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 0x55e2e8e570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f8959d7e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7250 0x55e2e8df6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f895957d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8de7b70 0x55e2e8df6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f8959d7e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7250 0x55e2e8df6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:60228/0 (socket says 172.21.15.18:60228) 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f895a57f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 0x55e2e8e570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40438/0 (socket says 172.21.15.18:40438) 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.731+0000 7f8959d7e640 1 -- 172.21.15.18:0/720587013 learned_addr learned my addr 172.21.15.18:0/720587013 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f895a57f640 1 -- 172.21.15.18:0/720587013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8de7b70 msgr2=0x55e2e8df6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f895a57f640 1 --2- 172.21.15.18:0/720587013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8de7b70 0x55e2e8df6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f895a57f640 1 -- 172.21.15.18:0/720587013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7250 msgr2=0x55e2e8df6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f895a57f640 1 --2- 172.21.15.18:0/720587013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7250 0x55e2e8df6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f895a57f640 1 -- 172.21.15.18:0/720587013 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89541a4870 con 0x55e2e8df0750 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f8959d7e640 1 --2- 172.21.15.18:0/720587013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7250 0x55e2e8df6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f895a57f640 1 --2- 172.21.15.18:0/720587013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 0x55e2e8e570d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f894c029af0 tx=0x7f894c011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f83f114e0b997a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f893d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f893d7fa640 1 -- 172.21.15.18:0/720587013 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c009ee0 con 0x55e2e8df0750 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f893d7fa640 1 -- 172.21.15.18:0/720587013 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f894c0297c0 con 0x55e2e8df0750 2024-02-17T12:54:04.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f895ebb4740 1 -- 172.21.15.18:0/720587013 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8944009e10 con 0x55e2e8df0750 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f893d7fa640 1 -- 172.21.15.18:0/720587013 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f894c011ef0 con 0x55e2e8df0750 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f895ebb4740 10 client.17097 fetch_fsmap learned FSMap version 33 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f895ebb4740 1 -- 172.21.15.18:0/720587013 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e2e8d4d6a0 con 0x55e2e8df0750 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f893d7fa640 1 -- 172.21.15.18:0/720587013 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8944009e10 con 0x55e2e8df0750 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f893d7fa640 1 -- 172.21.15.18:0/720587013 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f894c003780 con 0x55e2e8df0750 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f895ebb4740 10 client.17097 fetch_fsmap finished waiting for FSMap version 33 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f895ebb4740 10 client.17097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.732+0000 7f895ebb4740 20 client.17097 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f895ebb4740 1 --2- 172.21.15.18:0/720587013 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e2e8e00b80 0x55e2e8e5af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f895ebb4740 4 client.17097 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f895ebb4740 1 -- 172.21.15.18:0/720587013 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e2e8ada890 con 0x55e2e8e00b80 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f8959d7e640 1 --2- 172.21.15.18:0/720587013 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e2e8e00b80 0x55e2e8e5af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f893d7fa640 10 client.17097 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:04.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f8959d7e640 1 --2- 172.21.15.18:0/720587013 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e2e8e00b80 0x55e2e8e5af00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12ae4cea981a5f server_cookie=dcee67d85c114131 in_seq=0 out_seq=0 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f893d7fa640 1 -- 172.21.15.18:0/720587013 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e2e8ada890 con 0x55e2e8e00b80 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f893d7fa640 10 client.17097 handle_command_reply: tid=0 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f895ebb4740 1 client.17097 shutdown 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f895ebb4740 1 -- 172.21.15.18:0/720587013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 msgr2=0x55e2e8e570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f895ebb4740 1 --2- 172.21.15.18:0/720587013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 0x55e2e8e570d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f894c029af0 tx=0x7f894c011d00 comp rx=0 tx=0).stop 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f895ebb4740 1 -- 172.21.15.18:0/720587013 shutdown_connections 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f895ebb4740 1 --2- 172.21.15.18:0/720587013 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e2e8e00b80 0x55e2e8e5af00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f895ebb4740 1 --2- 172.21.15.18:0/720587013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2e8df0750 0x55e2e8e570d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f895ebb4740 1 --2- 172.21.15.18:0/720587013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e2e8de7b70 0x55e2e8df6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f893d7fa640 0 client.17097 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f895ebb4740 1 --2- 172.21.15.18:0/720587013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2e8de7250 0x55e2e8df6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.733+0000 7f895ebb4740 1 -- 172.21.15.18:0/720587013 >> 172.21.15.18:0/720587013 conn(0x55e2e8de1980 msgr2=0x55e2e8de02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895ebb4740 1 -- 172.21.15.18:0/720587013 shutdown_connections 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895ebb4740 1 -- 172.21.15.18:0/720587013 wait complete. 2024-02-17T12:54:04.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895ebb4740 20 client.17097 trim_cache size 0 max 16384 2024-02-17T12:54:04.732 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895bfe2640 1 -- 172.21.15.18:0/79276037 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f89400a2230 msgr2=0x7f89400a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895bfe2640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f89400a2230 0x7f89400a46f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f894c0058c0 tx=0x7f894c004730 comp rx=0 tx=0).stop 2024-02-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895bfe2640 1 -- 172.21.15.18:0/79276037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8954101390 msgr2=0x7f895419d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895bfe2640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8954101390 0x7f895419d5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8950004af0 tx=0x7f89500325c0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895bfe2640 1 -- 172.21.15.18:0/79276037 shutdown_connections 2024-02-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895bfe2640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f89400a2230 0x7f89400a46f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895bfe2640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8954197660 0x7f8954197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895bfe2640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8954106460 0x7f895419dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895bfe2640 1 --2- 172.21.15.18:0/79276037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8954101390 0x7f895419d5a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895bfe2640 1 -- 172.21.15.18:0/79276037 >> 172.21.15.18:0/79276037 conn(0x7f89540fcbb0 msgr2=0x7f89540fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895bfe2640 1 -- 172.21.15.18:0/79276037 shutdown_connections 2024-02-17T12:54:04.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:04.734+0000 7f895bfe2640 1 -- 172.21.15.18:0/79276037 wait complete. 2024-02-17T12:54:04.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:04 smithi119 ceph-mon[36188]: pgmap v128: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 229 op/s 2024-02-17T12:54:04.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:04 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3937896094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:54:04.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:04 smithi146 ceph-mon[36454]: pgmap v128: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 229 op/s 2024-02-17T12:54:04.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:04 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3937896094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:54:05.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 251963 9.86 MB/sec execute 80 sec latency 422.917 ms 2024-02-17T12:54:05.737 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:54:05.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:54:05.738 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:54:05.766 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:05 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:54:05.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:05 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:54:05.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.867+0000 7f923c8ff640 1 -- 172.21.15.18:0/1013115232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9238102410 msgr2=0x7f92381027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:05.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.867+0000 7f923c8ff640 1 --2- 172.21.15.18:0/1013115232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9238102410 0x7f92381027f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9228009900 tx=0x7f922802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.867+0000 7f923c8ff640 1 -- 172.21.15.18:0/1013115232 shutdown_connections 2024-02-17T12:54:05.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.867+0000 7f923c8ff640 1 --2- 172.21.15.18:0/1013115232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f923810b750 0x7f923810dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.867+0000 7f923c8ff640 1 --2- 172.21.15.18:0/1013115232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9238102d30 0x7f923810b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.867+0000 7f923c8ff640 1 --2- 172.21.15.18:0/1013115232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9238102410 0x7f92381027f0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.867+0000 7f923c8ff640 1 -- 172.21.15.18:0/1013115232 >> 172.21.15.18:0/1013115232 conn(0x7f92380771e0 msgr2=0x7f92380775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:05.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.868+0000 7f923c8ff640 1 -- 172.21.15.18:0/1013115232 shutdown_connections 2024-02-17T12:54:05.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.868+0000 7f923c8ff640 1 -- 172.21.15.18:0/1013115232 wait complete. 2024-02-17T12:54:05.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.868+0000 7f923c8ff640 1 Processor -- start 2024-02-17T12:54:05.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.868+0000 7f923c8ff640 1 -- start start 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f923c8ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9238102410 0x7f923819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f923c8ff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9238102d30 0x7f923819db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f923c8ff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923810b750 0x7f92381976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f923c8ff640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9238197e30 con 0x7f9238102d30 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f923c8ff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9238197fa0 con 0x7f9238102410 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f923c8ff640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9238198160 con 0x7f923810b750 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f9235d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9238102d30 0x7f923819db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f9235d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9238102d30 0x7f923819db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40452/0 (socket says 172.21.15.18:40452) 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f9235d9b640 1 -- 172.21.15.18:0/3784776528 learned_addr learned my addr 172.21.15.18:0/3784776528 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f9236d9d640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923810b750 0x7f92381976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f923659c640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9238102410 0x7f923819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f9235d9b640 1 -- 172.21.15.18:0/3784776528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923810b750 msgr2=0x7f92381976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f9235d9b640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923810b750 0x7f92381976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f9235d9b640 1 -- 172.21.15.18:0/3784776528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9238102410 msgr2=0x7f923819d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f9235d9b640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9238102410 0x7f923819d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f9235d9b640 1 -- 172.21.15.18:0/3784776528 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9228009560 con 0x7f9238102d30 2024-02-17T12:54:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f9236d9d640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923810b750 0x7f92381976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f9235d9b640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9238102d30 0x7f923819db20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f922c00d770 tx=0x7f922c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f92277fe640 1 -- 172.21.15.18:0/3784776528 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c009c30 con 0x7f9238102d30 2024-02-17T12:54:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f923c8ff640 1 -- 172.21.15.18:0/3784776528 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9238198440 con 0x7f9238102d30 2024-02-17T12:54:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f92277fe640 1 -- 172.21.15.18:0/3784776528 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f922c00de80 con 0x7f9238102d30 2024-02-17T12:54:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f92277fe640 1 -- 172.21.15.18:0/3784776528 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c004e80 con 0x7f9238102d30 2024-02-17T12:54:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.869+0000 7f923c8ff640 1 -- 172.21.15.18:0/3784776528 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92381a4560 con 0x7f9238102d30 2024-02-17T12:54:05.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.871+0000 7f92277fe640 1 -- 172.21.15.18:0/3784776528 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f922c005020 con 0x7f9238102d30 2024-02-17T12:54:05.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.872+0000 7f923f5aa740 1 Processor -- start 2024-02-17T12:54:05.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.872+0000 7f923f5aa740 1 -- start start 2024-02-17T12:54:05.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.872+0000 7f92277fe640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f92200a2230 0x7f92200a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.872+0000 7f923f5aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ff380 0x5638ad3ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.872+0000 7f923f5aa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ffd30 0x5638ad4082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.872+0000 7f923f5aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad4088b0 0x5638ad40ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.872+0000 7f923f5aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ad36df90 con 0x5638ad4088b0 2024-02-17T12:54:05.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.872+0000 7f923f5aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ad36b4c0 con 0x5638ad3ff380 2024-02-17T12:54:05.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.872+0000 7f923f5aa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ad366ca0 con 0x5638ad3ffd30 2024-02-17T12:54:05.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9236d9d640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f92200a2230 0x7f92200a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9235d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ffd30 0x5638ad4082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9235d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ffd30 0x5638ad4082e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:59916/0 (socket says 172.21.15.18:59916) 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f92277fe640 1 -- 172.21.15.18:0/3784776528 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f922c0cdb50 con 0x7f9238102d30 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9235d9b640 1 -- 172.21.15.18:0/116052917 learned_addr learned my addr 172.21.15.18:0/116052917 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9236d9d640 1 --2- 172.21.15.18:0/116052917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad4088b0 0x5638ad40ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f923659c640 1 --2- 172.21.15.18:0/116052917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ff380 0x5638ad3ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9235d9b640 1 -- 172.21.15.18:0/116052917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ff380 msgr2=0x5638ad3ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9235d9b640 1 --2- 172.21.15.18:0/116052917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ff380 0x5638ad3ff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9235d9b640 1 -- 172.21.15.18:0/116052917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad4088b0 msgr2=0x5638ad40ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9235d9b640 1 --2- 172.21.15.18:0/116052917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad4088b0 0x5638ad40ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9235d9b640 1 -- 172.21.15.18:0/116052917 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638ad3290b0 con 0x5638ad3ffd30 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f923659c640 1 --2- 172.21.15.18:0/116052917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ff380 0x5638ad3ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9236d9d640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f92200a2230 0x7f92200a46f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f923000afd0 tx=0x7f923000ad20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9235d9b640 1 --2- 172.21.15.18:0/116052917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ffd30 0x5638ad4082e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f922c09a520 tx=0x7f922c01e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60090d1d64554c22 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9224ff9640 1 -- 172.21.15.18:0/116052917 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c0b32c0 con 0x5638ad3ffd30 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9224ff9640 1 -- 172.21.15.18:0/116052917 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f922c09ac90 con 0x5638ad3ffd30 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.873+0000 7f9224ff9640 1 -- 172.21.15.18:0/116052917 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c0cb570 con 0x5638ad3ffd30 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.874+0000 7f923f5aa740 1 -- 172.21.15.18:0/116052917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ffd30 msgr2=0x5638ad4082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.874+0000 7f923f5aa740 1 --2- 172.21.15.18:0/116052917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ffd30 0x5638ad4082e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f922c09a520 tx=0x7f922c01e000 comp rx=0 tx=0).stop 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.874+0000 7f923f5aa740 1 -- 172.21.15.18:0/116052917 shutdown_connections 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.874+0000 7f923f5aa740 1 --2- 172.21.15.18:0/116052917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad4088b0 0x5638ad40ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.874+0000 7f923f5aa740 1 --2- 172.21.15.18:0/116052917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ffd30 0x5638ad4082e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.874+0000 7f923f5aa740 1 --2- 172.21.15.18:0/116052917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ff380 0x5638ad3ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.874+0000 7f923f5aa740 1 -- 172.21.15.18:0/116052917 >> 172.21.15.18:0/116052917 conn(0x5638ad3f9840 msgr2=0x5638ad3f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.874+0000 7f923f5aa740 1 -- 172.21.15.18:0/116052917 shutdown_connections 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.874+0000 7f923f5aa740 1 -- 172.21.15.18:0/116052917 wait complete. 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923f5aa740 1 -- start start 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923f5aa740 1 Processor -- start 2024-02-17T12:54:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923f5aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ff380 0x5638ad405880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923f5aa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ffd30 0x5638ad405dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923f5aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad4088b0 0x5638ad406300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923f5aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ad30bf40 con 0x5638ad3ff380 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923f5aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ad03a4b0 con 0x5638ad4088b0 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923f5aa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ad2ec670 con 0x5638ad3ffd30 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923f5aa740 10 client.0 _ll_register_callbacks cb 0x5638ad359f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ff380 0x5638ad405880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ff380 0x5638ad405880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40470/0 (socket says 172.21.15.18:40470) 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923659c640 1 -- 172.21.15.18:0/2497488999 learned_addr learned my addr 172.21.15.18:0/2497488999 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f9235d9b640 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ffd30 0x5638ad405dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f9236d9d640 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad4088b0 0x5638ad406300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923659c640 1 -- 172.21.15.18:0/2497488999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ffd30 msgr2=0x5638ad405dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923659c640 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ffd30 0x5638ad405dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923659c640 1 -- 172.21.15.18:0/2497488999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad4088b0 msgr2=0x5638ad406300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923659c640 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad4088b0 0x5638ad406300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f923659c640 1 -- 172.21.15.18:0/2497488999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92381a4560 con 0x5638ad3ff380 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.875+0000 7f9236d9d640 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad4088b0 0x5638ad406300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f923659c640 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ff380 0x5638ad405880 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f922802c440 tx=0x7f922802cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca15716be3406cff server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f921dffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f921dffb640 1 -- 172.21.15.18:0/2497488999 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228004450 con 0x5638ad3ff380 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f921dffb640 1 -- 172.21.15.18:0/2497488999 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9228044070 con 0x5638ad3ff380 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f923f5aa740 1 -- 172.21.15.18:0/2497488999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638ad340f40 con 0x5638ad3ff380 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f921dffb640 1 -- 172.21.15.18:0/2497488999 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9228004df0 con 0x5638ad3ff380 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f923f5aa740 10 client.17109 fetch_fsmap learned FSMap version 33 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f923f5aa740 1 -- 172.21.15.18:0/2497488999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5638ad365320 con 0x5638ad3ff380 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f921dffb640 1 -- 172.21.15.18:0/2497488999 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5638ad340f40 con 0x5638ad3ff380 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f921dffb640 1 -- 172.21.15.18:0/2497488999 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9228003930 con 0x5638ad3ff380 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f923f5aa740 10 client.17109 fetch_fsmap finished waiting for FSMap version 33 2024-02-17T12:54:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f923f5aa740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f923f5aa740 20 client.17109 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f923f5aa740 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638ad418d60 0x5638ad41b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f923f5aa740 4 client.17109 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.876+0000 7f923f5aa740 1 -- 172.21.15.18:0/2497488999 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638ad35d310 con 0x5638ad418d60 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.877+0000 7f9235d9b640 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638ad418d60 0x5638ad41b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.877+0000 7f9235d9b640 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638ad418d60 0x5638ad41b150 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5203330a580c7b42 server_cookie=2a2531d737421bad in_seq=0 out_seq=0 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.877+0000 7f921dffb640 10 client.17109 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f921dffb640 1 -- 172.21.15.18:0/2497488999 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5638ad35d310 con 0x5638ad418d60 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f921dffb640 10 client.17109 handle_command_reply: tid=0 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f923f5aa740 1 client.17109 shutdown 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f923f5aa740 1 -- 172.21.15.18:0/2497488999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ff380 msgr2=0x5638ad405880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f923f5aa740 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ff380 0x5638ad405880 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f922802c440 tx=0x7f922802cb30 comp rx=0 tx=0).stop 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f923f5aa740 1 -- 172.21.15.18:0/2497488999 shutdown_connections 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f923f5aa740 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638ad418d60 0x5638ad41b150 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f923f5aa740 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad4088b0 0x5638ad406300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f921dffb640 0 client.17109 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f923f5aa740 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ffd30 0x5638ad405dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f923f5aa740 1 --2- 172.21.15.18:0/2497488999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ff380 0x5638ad405880 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f923f5aa740 1 -- 172.21.15.18:0/2497488999 >> 172.21.15.18:0/2497488999 conn(0x5638ad3f9840 msgr2=0x5638ad3fb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f923f5aa740 1 -- 172.21.15.18:0/2497488999 shutdown_connections 2024-02-17T12:54:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f923f5aa740 1 -- 172.21.15.18:0/2497488999 wait complete. 2024-02-17T12:54:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.878+0000 7f923f5aa740 20 client.17109 trim_cache size 0 max 16384 2024-02-17T12:54:05.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.893+0000 7f923f5aa740 1 Processor -- start 2024-02-17T12:54:05.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.893+0000 7f923f5aa740 1 -- start start 2024-02-17T12:54:05.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.893+0000 7f923f5aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ff380 0x5638ad40fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.893+0000 7f923f5aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ffd30 0x5638ad3f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.893+0000 7f923f5aa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad4088b0 0x5638ad405dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.893+0000 7f923f5aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ad415910 con 0x5638ad3ffd30 2024-02-17T12:54:05.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.893+0000 7f923f5aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ad3fe410 con 0x5638ad3ff380 2024-02-17T12:54:05.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.893+0000 7f923f5aa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ad418b90 con 0x5638ad4088b0 2024-02-17T12:54:05.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f9235d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ffd30 0x5638ad3f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f9235d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ffd30 0x5638ad3f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40482/0 (socket says 172.21.15.18:40482) 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f9235d9b640 1 -- 172.21.15.18:0/2602519975 learned_addr learned my addr 172.21.15.18:0/2602519975 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f923659c640 1 --2- 172.21.15.18:0/2602519975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ff380 0x5638ad40fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f9236d9d640 1 --2- 172.21.15.18:0/2602519975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad4088b0 0x5638ad405dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f9235d9b640 1 -- 172.21.15.18:0/2602519975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad4088b0 msgr2=0x5638ad405dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f9235d9b640 1 --2- 172.21.15.18:0/2602519975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad4088b0 0x5638ad405dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f9235d9b640 1 -- 172.21.15.18:0/2602519975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ff380 msgr2=0x5638ad40fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f9235d9b640 1 --2- 172.21.15.18:0/2602519975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ff380 0x5638ad40fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f9235d9b640 1 -- 172.21.15.18:0/2602519975 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638ad301c00 con 0x5638ad3ffd30 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f9235d9b640 1 --2- 172.21.15.18:0/2602519975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ffd30 0x5638ad3f8760 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f922c0cb8f0 tx=0x7f922c01d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee389e3b5559f631 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f921ffff640 1 -- 172.21.15.18:0/2602519975 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c09ac90 con 0x5638ad3ffd30 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f921ffff640 1 -- 172.21.15.18:0/2602519975 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f922c014070 con 0x5638ad3ffd30 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.894+0000 7f921ffff640 1 -- 172.21.15.18:0/2602519975 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c098920 con 0x5638ad3ffd30 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 -- 172.21.15.18:0/2602519975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ffd30 msgr2=0x5638ad3f8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:05.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 --2- 172.21.15.18:0/2602519975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ffd30 0x5638ad3f8760 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f922c0cb8f0 tx=0x7f922c01d040 comp rx=0 tx=0).stop 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 -- 172.21.15.18:0/2602519975 shutdown_connections 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 --2- 172.21.15.18:0/2602519975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad4088b0 0x5638ad405dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 --2- 172.21.15.18:0/2602519975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad3ffd30 0x5638ad3f8760 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 --2- 172.21.15.18:0/2602519975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ff380 0x5638ad40fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 -- 172.21.15.18:0/2602519975 >> 172.21.15.18:0/2602519975 conn(0x5638ad3f9840 msgr2=0x5638ad3f8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 -- 172.21.15.18:0/2602519975 shutdown_connections 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 -- 172.21.15.18:0/2602519975 wait complete. 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 -- start start 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 Processor -- start 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ff380 0x5638ad46d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ffd30 0x5638ad46dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad4088b0 0x5638ad467b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ad4680e0 con 0x5638ad4088b0 2024-02-17T12:54:05.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ad468250 con 0x5638ad3ffd30 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ad468410 con 0x5638ad3ff380 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923f5aa740 10 client.0 _ll_register_callbacks cb 0x7f922c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f9236d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad4088b0 0x5638ad467b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f9236d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad4088b0 0x5638ad467b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40496/0 (socket says 172.21.15.18:40496) 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f923659c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ff380 0x5638ad46d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.895+0000 7f9236d9d640 1 -- 172.21.15.18:0/1921579090 learned_addr learned my addr 172.21.15.18:0/1921579090 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f9235d9b640 1 --2- 172.21.15.18:0/1921579090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ffd30 0x5638ad46dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f9236d9d640 1 -- 172.21.15.18:0/1921579090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ff380 msgr2=0x5638ad46d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f9236d9d640 1 --2- 172.21.15.18:0/1921579090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ff380 0x5638ad46d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f9236d9d640 1 -- 172.21.15.18:0/1921579090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ffd30 msgr2=0x5638ad46dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f9236d9d640 1 --2- 172.21.15.18:0/1921579090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ffd30 0x5638ad46dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f9236d9d640 1 -- 172.21.15.18:0/1921579090 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638ad3290b0 con 0x5638ad4088b0 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f9236d9d640 1 --2- 172.21.15.18:0/1921579090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad4088b0 0x5638ad467b20 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9230010fc0 tx=0x7f9230005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7265ba4f1fc0056c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:05.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f921dffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f921dffb640 1 -- 172.21.15.18:0/1921579090 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230025070 con 0x5638ad4088b0 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f921dffb640 1 -- 172.21.15.18:0/1921579090 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9230003970 con 0x5638ad4088b0 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f923f5aa740 1 -- 172.21.15.18:0/1921579090 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638ad38c900 con 0x5638ad4088b0 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f921dffb640 1 -- 172.21.15.18:0/1921579090 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9230003b70 con 0x5638ad4088b0 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f923f5aa740 10 client.17115 fetch_fsmap learned FSMap version 33 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f923f5aa740 1 -- 172.21.15.18:0/1921579090 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5638ad327910 con 0x5638ad4088b0 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.896+0000 7f921dffb640 1 -- 172.21.15.18:0/1921579090 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9230036020 con 0x5638ad4088b0 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.897+0000 7f921dffb640 1 -- 172.21.15.18:0/1921579090 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9230020720 con 0x5638ad4088b0 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.897+0000 7f923f5aa740 10 client.17115 fetch_fsmap finished waiting for FSMap version 33 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.897+0000 7f923f5aa740 10 client.17115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.897+0000 7f923f5aa740 20 client.17115 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.897+0000 7f923f5aa740 1 --2- 172.21.15.18:0/1921579090 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638ad475730 0x5638ad477b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.897+0000 7f923f5aa740 4 client.17115 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.897+0000 7f923f5aa740 1 -- 172.21.15.18:0/1921579090 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5638ad066860 con 0x5638ad475730 2024-02-17T12:54:05.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.897+0000 7f923659c640 1 --2- 172.21.15.18:0/1921579090 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638ad475730 0x5638ad477b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.897+0000 7f921dffb640 10 client.17115 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.897+0000 7f923659c640 1 --2- 172.21.15.18:0/1921579090 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638ad475730 0x5638ad477b50 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=664c551ba19f5da0 server_cookie=3844bf8f71e68aff in_seq=0 out_seq=0 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.898+0000 7f921dffb640 1 -- 172.21.15.18:0/1921579090 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5638ad066860 con 0x5638ad475730 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.898+0000 7f921dffb640 10 client.17115 handle_command_reply: tid=0 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.898+0000 7f923f5aa740 1 client.17115 shutdown 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.898+0000 7f923f5aa740 1 -- 172.21.15.18:0/1921579090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad4088b0 msgr2=0x5638ad467b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.898+0000 7f923f5aa740 1 --2- 172.21.15.18:0/1921579090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad4088b0 0x5638ad467b20 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9230010fc0 tx=0x7f9230005c60 comp rx=0 tx=0).stop 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923f5aa740 1 -- 172.21.15.18:0/1921579090 shutdown_connections 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923f5aa740 1 --2- 172.21.15.18:0/1921579090 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638ad475730 0x5638ad477b50 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923f5aa740 1 --2- 172.21.15.18:0/1921579090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638ad4088b0 0x5638ad467b20 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923f5aa740 1 --2- 172.21.15.18:0/1921579090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638ad3ffd30 0x5638ad46dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923f5aa740 1 --2- 172.21.15.18:0/1921579090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638ad3ff380 0x5638ad46d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923f5aa740 1 -- 172.21.15.18:0/1921579090 >> 172.21.15.18:0/1921579090 conn(0x5638ad3f9840 msgr2=0x5638ad3f88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f921dffb640 0 client.17115 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:05.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923f5aa740 1 -- 172.21.15.18:0/1921579090 shutdown_connections 2024-02-17T12:54:05.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923f5aa740 1 -- 172.21.15.18:0/1921579090 wait complete. 2024-02-17T12:54:05.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923f5aa740 20 client.17115 trim_cache size 0 max 16384 2024-02-17T12:54:05.896 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:54:05.896 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:54:05.896 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "94ea47af-bd39-4c64-b78b-f4db6d4aa6a5", 2024-02-17T12:54:05.896 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:54:05.896 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:54:05.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923c8ff640 1 -- 172.21.15.18:0/3784776528 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f92200a2230 msgr2=0x7f92200a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:05.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923c8ff640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f92200a2230 0x7f92200a46f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f923000afd0 tx=0x7f923000ad20 comp rx=0 tx=0).stop 2024-02-17T12:54:05.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923c8ff640 1 -- 172.21.15.18:0/3784776528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9238102d30 msgr2=0x7f923819db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:05.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923c8ff640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9238102d30 0x7f923819db20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f922c00d770 tx=0x7f922c00dc30 comp rx=0 tx=0).stop 2024-02-17T12:54:05.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923c8ff640 1 -- 172.21.15.18:0/3784776528 shutdown_connections 2024-02-17T12:54:05.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923c8ff640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f92200a2230 0x7f92200a46f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923c8ff640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923810b750 0x7f92381976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923c8ff640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9238102d30 0x7f923819db20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923c8ff640 1 --2- 172.21.15.18:0/3784776528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9238102410 0x7f923819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:05.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923c8ff640 1 -- 172.21.15.18:0/3784776528 >> 172.21.15.18:0/3784776528 conn(0x7f92380771e0 msgr2=0x7f92380fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:05.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923c8ff640 1 -- 172.21.15.18:0/3784776528 shutdown_connections 2024-02-17T12:54:05.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:05.899+0000 7f923c8ff640 1 -- 172.21.15.18:0/3784776528 wait complete. 2024-02-17T12:54:05.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:54:05.903 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:54:05.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:05 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:54:06.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.066+0000 7f3b533e9640 1 -- 172.21.15.18:0/4121946653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4c102cc0 msgr2=0x7f3b4c10b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:06.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.066+0000 7f3b533e9640 1 --2- 172.21.15.18:0/4121946653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4c102cc0 0x7f3b4c10b1a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009900 tx=0x7f3b3c02bcd0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.066+0000 7f3b533e9640 1 -- 172.21.15.18:0/4121946653 shutdown_connections 2024-02-17T12:54:06.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.066+0000 7f3b533e9640 1 --2- 172.21.15.18:0/4121946653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b4c10b6e0 0x7f3b4c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.066+0000 7f3b533e9640 1 --2- 172.21.15.18:0/4121946653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4c102cc0 0x7f3b4c10b1a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.066+0000 7f3b533e9640 1 --2- 172.21.15.18:0/4121946653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b4c1023a0 0x7f3b4c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.066+0000 7f3b533e9640 1 -- 172.21.15.18:0/4121946653 >> 172.21.15.18:0/4121946653 conn(0x7f3b4c077230 msgr2=0x7f3b4c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:06.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.066+0000 7f3b533e9640 1 -- 172.21.15.18:0/4121946653 shutdown_connections 2024-02-17T12:54:06.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.066+0000 7f3b533e9640 1 -- 172.21.15.18:0/4121946653 wait complete. 2024-02-17T12:54:06.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.066+0000 7f3b533e9640 1 Processor -- start 2024-02-17T12:54:06.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.066+0000 7f3b533e9640 1 -- start start 2024-02-17T12:54:06.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b533e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b4c1023a0 0x7f3b4c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b533e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4c102cc0 0x7f3b4c197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b533e9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b4c10b6e0 0x7f3b4c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b533e9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c19c5f0 con 0x7f3b4c102cc0 2024-02-17T12:54:06.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b533e9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c19c760 con 0x7f3b4c1023a0 2024-02-17T12:54:06.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b533e9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c19c920 con 0x7f3b4c10b6e0 2024-02-17T12:54:06.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b50984640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4c102cc0 0x7f3b4c197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b50984640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4c102cc0 0x7f3b4c197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40504/0 (socket says 172.21.15.18:40504) 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b50984640 1 -- 172.21.15.18:0/3267539335 learned_addr learned my addr 172.21.15.18:0/3267539335 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b51185640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b4c1023a0 0x7f3b4c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b51986640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b4c10b6e0 0x7f3b4c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b50984640 1 -- 172.21.15.18:0/3267539335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b4c10b6e0 msgr2=0x7f3b4c19bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b50984640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b4c10b6e0 0x7f3b4c19bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b50984640 1 -- 172.21.15.18:0/3267539335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b4c1023a0 msgr2=0x7f3b4c197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b50984640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b4c1023a0 0x7f3b4c197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b50984640 1 -- 172.21.15.18:0/3267539335 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c009560 con 0x7f3b4c102cc0 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b50984640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4c102cc0 0x7f3b4c197cc0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c005940 tx=0x7f3b3c02c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b427fc640 1 -- 172.21.15.18:0/3267539335 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0043d0 con 0x7f3b4c102cc0 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b427fc640 1 -- 172.21.15.18:0/3267539335 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b3c0052d0 con 0x7f3b4c102cc0 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b533e9640 1 -- 172.21.15.18:0/3267539335 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4c19cba0 con 0x7f3b4c102cc0 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b427fc640 1 -- 172.21.15.18:0/3267539335 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c003d30 con 0x7f3b4c102cc0 2024-02-17T12:54:06.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.067+0000 7f3b533e9640 1 -- 172.21.15.18:0/3267539335 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4c1a4480 con 0x7f3b4c102cc0 2024-02-17T12:54:06.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.070+0000 7f3b427fc640 1 -- 172.21.15.18:0/3267539335 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3b3c005440 con 0x7f3b4c102cc0 2024-02-17T12:54:06.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.070+0000 7f3b55fc1740 1 Processor -- start 2024-02-17T12:54:06.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.070+0000 7f3b55fc1740 1 -- start start 2024-02-17T12:54:06.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b427fc640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3b380a22b0 0x7f3b380a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b55fc1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa1c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b55fc1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1cbd0 0x55692aa25270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b55fc1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa257b0 0x55692aa27ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b55fc1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692a98b280 con 0x55692aa1c2b0 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b55fc1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692a9887b0 con 0x55692aa257b0 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b55fc1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692a983f90 con 0x55692aa1cbd0 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b51986640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3b380a22b0 0x7f3b380a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b51185640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa1c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b427fc640 1 -- 172.21.15.18:0/3267539335 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3b3c0f0630 con 0x7f3b4c102cc0 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b51185640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa1c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40520/0 (socket says 172.21.15.18:40520) 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b51185640 1 -- 172.21.15.18:0/1695170834 learned_addr learned my addr 172.21.15.18:0/1695170834 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b51986640 1 --2- 172.21.15.18:0/1695170834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa257b0 0x55692aa27ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b50984640 1 --2- 172.21.15.18:0/1695170834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1cbd0 0x55692aa25270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b51185640 1 -- 172.21.15.18:0/1695170834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1cbd0 msgr2=0x55692aa25270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b51185640 1 --2- 172.21.15.18:0/1695170834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1cbd0 0x55692aa25270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b51185640 1 -- 172.21.15.18:0/1695170834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa257b0 msgr2=0x55692aa27ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:06.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b51185640 1 --2- 172.21.15.18:0/1695170834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa257b0 0x55692aa27ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b51185640 1 -- 172.21.15.18:0/1695170834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55692a942570 con 0x55692aa1c2b0 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b51185640 1 --2- 172.21.15.18:0/1695170834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa1c690 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800bd10 tx=0x7f3b480107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1266ea4dfbe3431 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.071+0000 7f3b51986640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3b380a22b0 0x7f3b380a4770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009fd0 tx=0x7f3b44009d80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.072+0000 7f3b377fe640 1 -- 172.21.15.18:0/1695170834 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48004030 con 0x55692aa1c2b0 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.072+0000 7f3b377fe640 1 -- 172.21.15.18:0/1695170834 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b48015070 con 0x55692aa1c2b0 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.072+0000 7f3b377fe640 1 -- 172.21.15.18:0/1695170834 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b480116a0 con 0x55692aa1c2b0 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.072+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1695170834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 msgr2=0x55692aa1c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.072+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1695170834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa1c690 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800bd10 tx=0x7f3b480107a0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.072+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1695170834 shutdown_connections 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.072+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1695170834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa257b0 0x55692aa27ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.072+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1695170834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1cbd0 0x55692aa25270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.072+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1695170834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa1c690 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.072+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1695170834 >> 172.21.15.18:0/1695170834 conn(0x55692aa16990 msgr2=0x55692aa16d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.072+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1695170834 shutdown_connections 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.072+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1695170834 wait complete. 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b55fc1740 1 -- start start 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b55fc1740 1 Processor -- start 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b55fc1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa25190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b55fc1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1cbd0 0x55692aa22590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b55fc1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa257b0 0x55692aa22c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b55fc1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692a657430 con 0x55692aa1c2b0 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b55fc1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692a613e00 con 0x55692aa257b0 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b55fc1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692a908fd0 con 0x55692aa1cbd0 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b55fc1740 10 client.0 _ll_register_callbacks cb 0x55692a95efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b51185640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa25190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b51185640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa25190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40536/0 (socket says 172.21.15.18:40536) 2024-02-17T12:54:06.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b50984640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1cbd0 0x55692aa22590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b51185640 1 -- 172.21.15.18:0/2375822203 learned_addr learned my addr 172.21.15.18:0/2375822203 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.073+0000 7f3b51986640 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa257b0 0x55692aa22c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b51185640 1 -- 172.21.15.18:0/2375822203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1cbd0 msgr2=0x55692aa22590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b51185640 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1cbd0 0x55692aa22590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b51185640 1 -- 172.21.15.18:0/2375822203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa257b0 msgr2=0x55692aa22c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b51185640 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa257b0 0x55692aa22c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b51185640 1 -- 172.21.15.18:0/2375822203 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4800b940 con 0x55692aa1c2b0 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b50984640 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1cbd0 0x55692aa22590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b51986640 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa257b0 0x55692aa22c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b51185640 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa25190 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f3b48005940 tx=0x7f3b4800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30ff85eaeb98fcd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b34ff9640 1 -- 172.21.15.18:0/2375822203 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48020070 con 0x55692aa1c2b0 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b34ff9640 1 -- 172.21.15.18:0/2375822203 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3b48017070 con 0x55692aa1c2b0 2024-02-17T12:54:06.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b55fc1740 1 -- 172.21.15.18:0/2375822203 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55692a6837e0 con 0x55692aa1c2b0 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b34ff9640 1 -- 172.21.15.18:0/2375822203 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b48003880 con 0x55692aa1c2b0 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b55fc1740 10 client.17127 fetch_fsmap learned FSMap version 33 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b55fc1740 1 -- 172.21.15.18:0/2375822203 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55692a982610 con 0x55692aa1c2b0 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b34ff9640 1 -- 172.21.15.18:0/2375822203 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b48032020 con 0x55692aa1c2b0 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.074+0000 7f3b34ff9640 1 -- 172.21.15.18:0/2375822203 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3b480046c0 con 0x55692aa1c2b0 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.075+0000 7f3b55fc1740 10 client.17127 fetch_fsmap finished waiting for FSMap version 33 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.075+0000 7f3b55fc1740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.075+0000 7f3b55fc1740 20 client.17127 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.075+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55692aa358d0 0x55692aa37cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.075+0000 7f3b55fc1740 4 client.17127 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.075+0000 7f3b55fc1740 1 -- 172.21.15.18:0/2375822203 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55692a6c0720 con 0x55692aa358d0 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.075+0000 7f3b50984640 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55692aa358d0 0x55692aa37cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.075+0000 7f3b50984640 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55692aa358d0 0x55692aa37cc0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f62c8729720d1d25 server_cookie=7fbbf35d99df3e71 in_seq=0 out_seq=0 2024-02-17T12:54:06.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.075+0000 7f3b34ff9640 10 client.17127 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b34ff9640 1 -- 172.21.15.18:0/2375822203 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55692a6c0720 con 0x55692aa358d0 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b34ff9640 10 client.17127 handle_command_reply: tid=0 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b55fc1740 1 client.17127 shutdown 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b55fc1740 1 -- 172.21.15.18:0/2375822203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 msgr2=0x55692aa25190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa25190 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f3b48005940 tx=0x7f3b4800bf50 comp rx=0 tx=0).stop 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b55fc1740 1 -- 172.21.15.18:0/2375822203 shutdown_connections 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55692aa358d0 0x55692aa37cc0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa257b0 0x55692aa22c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b34ff9640 0 client.17127 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1cbd0 0x55692aa22590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/2375822203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa25190 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b55fc1740 1 -- 172.21.15.18:0/2375822203 >> 172.21.15.18:0/2375822203 conn(0x55692aa16990 msgr2=0x55692aa18730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b55fc1740 1 -- 172.21.15.18:0/2375822203 shutdown_connections 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b55fc1740 1 -- 172.21.15.18:0/2375822203 wait complete. 2024-02-17T12:54:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.076+0000 7f3b55fc1740 20 client.17127 trim_cache size 0 max 16384 2024-02-17T12:54:06.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b55fc1740 1 Processor -- start 2024-02-17T12:54:06.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b55fc1740 1 -- start start 2024-02-17T12:54:06.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b55fc1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1c2b0 0x55692aa158d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b55fc1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa1cbd0 0x55692aa2c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b55fc1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa257b0 0x55692aa2cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b55fc1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692aa10e20 con 0x55692aa257b0 2024-02-17T12:54:06.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b55fc1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692aa238e0 con 0x55692aa1cbd0 2024-02-17T12:54:06.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b55fc1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692aa31dd0 con 0x55692aa1c2b0 2024-02-17T12:54:06.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b51986640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa257b0 0x55692aa2cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b51986640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa257b0 0x55692aa2cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40550/0 (socket says 172.21.15.18:40550) 2024-02-17T12:54:06.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b51986640 1 -- 172.21.15.18:0/1952304601 learned_addr learned my addr 172.21.15.18:0/1952304601 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:06.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b50984640 1 --2- 172.21.15.18:0/1952304601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa1cbd0 0x55692aa2c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b51185640 1 --2- 172.21.15.18:0/1952304601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1c2b0 0x55692aa158d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b51986640 1 -- 172.21.15.18:0/1952304601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1c2b0 msgr2=0x55692aa158d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b51986640 1 --2- 172.21.15.18:0/1952304601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1c2b0 0x55692aa158d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b51986640 1 -- 172.21.15.18:0/1952304601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa1cbd0 msgr2=0x55692aa2c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b51986640 1 --2- 172.21.15.18:0/1952304601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa1cbd0 0x55692aa2c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b51986640 1 -- 172.21.15.18:0/1952304601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55692a91eac0 con 0x55692aa257b0 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.093+0000 7f3b51986640 1 --2- 172.21.15.18:0/1952304601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa257b0 0x55692aa2cba0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3b44013590 tx=0x7f3b44009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23977b964e08232a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.094+0000 7f3b36ffd640 1 -- 172.21.15.18:0/1952304601 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b440135c0 con 0x55692aa257b0 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.094+0000 7f3b36ffd640 1 -- 172.21.15.18:0/1952304601 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b44013120 con 0x55692aa257b0 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.094+0000 7f3b36ffd640 1 -- 172.21.15.18:0/1952304601 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4402a7d0 con 0x55692aa257b0 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.094+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1952304601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa257b0 msgr2=0x55692aa2cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.094+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1952304601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa257b0 0x55692aa2cba0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3b44013590 tx=0x7f3b44009500 comp rx=0 tx=0).stop 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.094+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1952304601 shutdown_connections 2024-02-17T12:54:06.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.094+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1952304601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa257b0 0x55692aa2cba0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.094+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1952304601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa1cbd0 0x55692aa2c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.094+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1952304601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa1c2b0 0x55692aa158d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.094+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1952304601 >> 172.21.15.18:0/1952304601 conn(0x55692aa16990 msgr2=0x55692aa151c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.094+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1952304601 shutdown_connections 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.094+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1952304601 wait complete. 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b55fc1740 1 -- start start 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b55fc1740 1 Processor -- start 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b55fc1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa89d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b55fc1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa1cbd0 0x55692aa8a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b55fc1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa257b0 0x55692aa84170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b55fc1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692aa84730 con 0x55692aa1c2b0 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b55fc1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692aa848a0 con 0x55692aa1cbd0 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b55fc1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692aa84a60 con 0x55692aa257b0 2024-02-17T12:54:06.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b55fc1740 10 client.0 _ll_register_callbacks cb 0x7f3b4800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b51185640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa89d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b51185640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa89d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40558/0 (socket says 172.21.15.18:40558) 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b51185640 1 -- 172.21.15.18:0/1658315937 learned_addr learned my addr 172.21.15.18:0/1658315937 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b50984640 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa1cbd0 0x55692aa8a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b51986640 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa257b0 0x55692aa84170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b51185640 1 -- 172.21.15.18:0/1658315937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa257b0 msgr2=0x55692aa84170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b51185640 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa257b0 0x55692aa84170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b51185640 1 -- 172.21.15.18:0/1658315937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa1cbd0 msgr2=0x55692aa8a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.095+0000 7f3b51185640 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa1cbd0 0x55692aa8a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b51185640 1 -- 172.21.15.18:0/1658315937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4401b040 con 0x55692aa1c2b0 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b51185640 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa89d60 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3b48001eb0 tx=0x7f3b48005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9a7fa707ff2228 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:06.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b34ff9640 1 -- 172.21.15.18:0/1658315937 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48020040 con 0x55692aa1c2b0 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b34ff9640 1 -- 172.21.15.18:0/1658315937 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3b48000e40 con 0x55692aa1c2b0 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1658315937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b48015070 con 0x55692aa1c2b0 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b34ff9640 1 -- 172.21.15.18:0/1658315937 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b48005c50 con 0x55692aa1c2b0 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b55fc1740 10 client.17133 fetch_fsmap learned FSMap version 33 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1658315937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55692a9446a0 con 0x55692aa1c2b0 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b34ff9640 1 -- 172.21.15.18:0/1658315937 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b48015070 con 0x55692aa1c2b0 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b34ff9640 1 -- 172.21.15.18:0/1658315937 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3b4802e460 con 0x55692aa1c2b0 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b55fc1740 10 client.17133 fetch_fsmap finished waiting for FSMap version 33 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b55fc1740 10 client.17133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b55fc1740 20 client.17133 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55692aa919a0 0x55692aa93dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b55fc1740 4 client.17133 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.096+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1658315937 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55692a70f890 con 0x55692aa919a0 2024-02-17T12:54:06.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b50984640 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55692aa919a0 0x55692aa93dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b50984640 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55692aa919a0 0x55692aa93dc0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5626acc63198e3d9 server_cookie=9855e483fcc601f5 in_seq=0 out_seq=0 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b34ff9640 10 client.17133 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b34ff9640 1 -- 172.21.15.18:0/1658315937 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55692a70f890 con 0x55692aa919a0 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b34ff9640 10 client.17133 handle_command_reply: tid=0 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b55fc1740 1 client.17133 shutdown 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1658315937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 msgr2=0x55692aa89d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa89d60 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3b48001eb0 tx=0x7f3b48005160 comp rx=0 tx=0).stop 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1658315937 shutdown_connections 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55692aa919a0 0x55692aa93dc0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55692aa257b0 0x55692aa84170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b34ff9640 0 client.17133 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55692aa1cbd0 0x55692aa8a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b55fc1740 1 --2- 172.21.15.18:0/1658315937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55692aa1c2b0 0x55692aa89d60 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.097+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1658315937 >> 172.21.15.18:0/1658315937 conn(0x55692aa16990 msgr2=0x55692aa15a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:06.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1658315937 shutdown_connections 2024-02-17T12:54:06.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b55fc1740 1 -- 172.21.15.18:0/1658315937 wait complete. 2024-02-17T12:54:06.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b55fc1740 20 client.17133 trim_cache size 0 max 16384 2024-02-17T12:54:06.096 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:54:06.096 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:54:06.096 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:54:06.096 INFO:teuthology.orchestra.run.smithi018.stdout: "94ea47af-bd39-4c64-b78b-f4db6d4aa6a5": { 2024-02-17T12:54:06.096 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:54:06.096 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "94ea47af-bd39-4c64-b78b-f4db6d4aa6a5", 2024-02-17T12:54:06.096 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:54:06.097 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:54:06.097 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:54:06.097 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:54:06.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b533e9640 1 -- 172.21.15.18:0/3267539335 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3b380a22b0 msgr2=0x7f3b380a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:06.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b533e9640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3b380a22b0 0x7f3b380a4770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009fd0 tx=0x7f3b44009d80 comp rx=0 tx=0).stop 2024-02-17T12:54:06.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b533e9640 1 -- 172.21.15.18:0/3267539335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4c102cc0 msgr2=0x7f3b4c197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:06.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b533e9640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4c102cc0 0x7f3b4c197cc0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c005940 tx=0x7f3b3c02c780 comp rx=0 tx=0).stop 2024-02-17T12:54:06.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b533e9640 1 -- 172.21.15.18:0/3267539335 shutdown_connections 2024-02-17T12:54:06.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b533e9640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3b380a22b0 0x7f3b380a4770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b533e9640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b4c10b6e0 0x7f3b4c19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b533e9640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4c102cc0 0x7f3b4c197cc0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b533e9640 1 --2- 172.21.15.18:0/3267539335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b4c1023a0 0x7f3b4c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:06.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b533e9640 1 -- 172.21.15.18:0/3267539335 >> 172.21.15.18:0/3267539335 conn(0x7f3b4c077230 msgr2=0x7f3b4c0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:06.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b533e9640 1 -- 172.21.15.18:0/3267539335 shutdown_connections 2024-02-17T12:54:06.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:06.098+0000 7f3b533e9640 1 -- 172.21.15.18:0/3267539335 wait complete. 2024-02-17T12:54:06.100 INFO:tasks.cephfs.filesystem:scrub status for tag:94ea47af-bd39-4c64-b78b-f4db6d4aa6a5 - {'path': '/', 'tag': '94ea47af-bd39-4c64-b78b-f4db6d4aa6a5', 'options': 'recursive,force'} 2024-02-17T12:54:06.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 252271 9.76 MB/sec execute 81 sec latency 1078.366 ms 2024-02-17T12:54:06.743 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:06 smithi119 ceph-mon[36188]: pgmap v129: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 307 op/s 2024-02-17T12:54:06.743 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:06 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:54:06.743 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:06 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:06.743 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:06 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:06.743 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:06 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:06.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:06 smithi018 ceph-mon[32625]: pgmap v129: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 307 op/s 2024-02-17T12:54:06.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:06 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:54:06.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:06 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:06.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:06 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:06.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:06 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:06.851 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:06 smithi146 ceph-mon[36454]: pgmap v129: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 307 op/s 2024-02-17T12:54:06.851 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:06 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:54:06.851 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:06 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:06.851 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:06 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:06.851 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:06 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 254488 9.74 MB/sec execute 82 sec latency 15.224 ms 2024-02-17T12:54:07.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:07.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:07.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:07.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:07.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:07.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:07.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:07.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:07 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:07.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:07.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:07.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:07.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:07.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:07.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:07.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:07 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:07.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:07.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:07.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:07.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:07.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:07.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:07.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:07.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:07 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:08.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 255952 9.76 MB/sec execute 83 sec latency 195.803 ms 2024-02-17T12:54:08.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:08 smithi018 ceph-mon[32625]: pgmap v130: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-02-17T12:54:08.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:08 smithi119 ceph-mon[36188]: pgmap v130: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-02-17T12:54:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:08 smithi146 ceph-mon[36454]: pgmap v130: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-02-17T12:54:09.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 258443 9.79 MB/sec execute 84 sec latency 10.895 ms 2024-02-17T12:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.027+0000 7fab973e2640 1 -- 172.21.15.18:0/469570344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab90068800 msgr2=0x7fab90068c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.027+0000 7fab973e2640 1 --2- 172.21.15.18:0/469570344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab90068800 0x7fab90068c60 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fab80009900 tx=0x7fab8002bcd0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.027+0000 7fab973e2640 1 -- 172.21.15.18:0/469570344 shutdown_connections 2024-02-17T12:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.027+0000 7fab973e2640 1 --2- 172.21.15.18:0/469570344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab900691a0 0x7fab9010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.027+0000 7fab973e2640 1 --2- 172.21.15.18:0/469570344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab90068800 0x7fab90068c60 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.027+0000 7fab973e2640 1 --2- 172.21.15.18:0/469570344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab90107270 0x7fab90107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.027+0000 7fab973e2640 1 -- 172.21.15.18:0/469570344 >> 172.21.15.18:0/469570344 conn(0x7fab90076ed0 msgr2=0x7fab900772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.027+0000 7fab973e2640 1 -- 172.21.15.18:0/469570344 shutdown_connections 2024-02-17T12:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.027+0000 7fab973e2640 1 -- 172.21.15.18:0/469570344 wait complete. 2024-02-17T12:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.027+0000 7fab973e2640 1 Processor -- start 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.027+0000 7fab973e2640 1 -- start start 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab973e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab90068800 0x7fab901003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab973e2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab900691a0 0x7fab901008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab973e2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab90107270 0x7fab90100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab973e2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab901a5a40 con 0x7fab900691a0 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab973e2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab901a5bb0 con 0x7fab90068800 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab973e2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab901a5d70 con 0x7fab90107270 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab9497d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab900691a0 0x7fab901008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab9497d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab900691a0 0x7fab901008e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58360/0 (socket says 172.21.15.18:58360) 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab9497d640 1 -- 172.21.15.18:0/2748190195 learned_addr learned my addr 172.21.15.18:0/2748190195 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab9517e640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab90068800 0x7fab901003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab9597f640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab90107270 0x7fab90100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab9497d640 1 -- 172.21.15.18:0/2748190195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab90107270 msgr2=0x7fab90100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab9497d640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab90107270 0x7fab90100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab9497d640 1 -- 172.21.15.18:0/2748190195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab90068800 msgr2=0x7fab901003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab9497d640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab90068800 0x7fab901003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab9497d640 1 -- 172.21.15.18:0/2748190195 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab80009560 con 0x7fab900691a0 2024-02-17T12:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.028+0000 7fab9497d640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab900691a0 0x7fab901008e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fab80005940 tx=0x7fab8002c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.029+0000 7fab867fc640 1 -- 172.21.15.18:0/2748190195 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab80004350 con 0x7fab900691a0 2024-02-17T12:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.029+0000 7fab867fc640 1 -- 172.21.15.18:0/2748190195 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab800044b0 con 0x7fab900691a0 2024-02-17T12:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.029+0000 7fab973e2640 1 -- 172.21.15.18:0/2748190195 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab901a5ff0 con 0x7fab900691a0 2024-02-17T12:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.029+0000 7fab867fc640 1 -- 172.21.15.18:0/2748190195 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab80003d30 con 0x7fab900691a0 2024-02-17T12:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.029+0000 7fab973e2640 1 -- 172.21.15.18:0/2748190195 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab901a64c0 con 0x7fab900691a0 2024-02-17T12:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.031+0000 7fab867fc640 1 -- 172.21.15.18:0/2748190195 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fab80004df0 con 0x7fab900691a0 2024-02-17T12:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.031+0000 7fab99fbc740 1 Processor -- start 2024-02-17T12:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.031+0000 7fab99fbc740 1 -- start start 2024-02-17T12:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab99fbc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce62ad0 0x55fabce61820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab99fbc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce61d60 0x55fabce621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab99fbc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 0x55fabce6d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab99fbc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabcdd09f0 con 0x55fabce6b2e0 2024-02-17T12:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab99fbc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabcdcdf20 con 0x55fabce61d60 2024-02-17T12:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab99fbc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabcdc9700 con 0x55fabce62ad0 2024-02-17T12:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab867fc640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab7c0a2350 0x7fab7c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab9497d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 0x55fabce6d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab9597f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce62ad0 0x55fabce61820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab9497d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 0x55fabce6d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58364/0 (socket says 172.21.15.18:58364) 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab9497d640 1 -- 172.21.15.18:0/2894593891 learned_addr learned my addr 172.21.15.18:0/2894593891 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab9597f640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab7c0a2350 0x7fab7c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab9517e640 1 --2- 172.21.15.18:0/2894593891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce61d60 0x55fabce621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab867fc640 1 -- 172.21.15.18:0/2748190195 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fab800f05a0 con 0x7fab900691a0 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab9597f640 1 -- 172.21.15.18:0/2894593891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce61d60 msgr2=0x55fabce621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab9597f640 1 --2- 172.21.15.18:0/2894593891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce61d60 0x55fabce621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab9597f640 1 -- 172.21.15.18:0/2894593891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 msgr2=0x55fabce6d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab9597f640 1 --2- 172.21.15.18:0/2894593891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 0x55fabce6d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.032+0000 7fab9597f640 1 -- 172.21.15.18:0/2894593891 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fabcd5f7b0 con 0x55fabce62ad0 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.033+0000 7fab9597f640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab7c0a2350 0x7fab7c0a4810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fab8800fc90 tx=0x7fab8801e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.033+0000 7fab9597f640 1 --2- 172.21.15.18:0/2894593891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce62ad0 0x55fabce61820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab88005fd0 tx=0x7fab8800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7911a81b084a701 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.033+0000 7fab7b7fe640 1 -- 172.21.15.18:0/2894593891 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88011550 con 0x55fabce62ad0 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.033+0000 7fab7b7fe640 1 -- 172.21.15.18:0/2894593891 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab88009d90 con 0x55fabce62ad0 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.033+0000 7fab7b7fe640 1 -- 172.21.15.18:0/2894593891 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88004050 con 0x55fabce62ad0 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 -- 172.21.15.18:0/2894593891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce62ad0 msgr2=0x55fabce61820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2894593891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce62ad0 0x55fabce61820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab88005fd0 tx=0x7fab8800f790 comp rx=0 tx=0).stop 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 -- 172.21.15.18:0/2894593891 shutdown_connections 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2894593891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 0x55fabce6d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2894593891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce61d60 0x55fabce621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2894593891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce62ad0 0x55fabce61820 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 -- 172.21.15.18:0/2894593891 >> 172.21.15.18:0/2894593891 conn(0x55fabce29550 msgr2=0x55fabce5ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 -- 172.21.15.18:0/2894593891 shutdown_connections 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 -- 172.21.15.18:0/2894593891 wait complete. 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 -- start start 2024-02-17T12:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 Processor -- start 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce61d60 0x55fabce68350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce62ad0 0x55fabce68890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce6b2e0 0x55fabce68dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabcd6ebf0 con 0x55fabce62ad0 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabca9d250 con 0x55fabce61d60 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.034+0000 7fab99fbc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabcd4f200 con 0x55fabce6b2e0 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab99fbc740 10 client.0 _ll_register_callbacks cb 0x55fabcdbdd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab9497d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce62ad0 0x55fabce68890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab9497d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce62ad0 0x55fabce68890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58370/0 (socket says 172.21.15.18:58370) 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab9497d640 1 -- 172.21.15.18:0/2264210945 learned_addr learned my addr 172.21.15.18:0/2264210945 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab9517e640 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce61d60 0x55fabce68350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab9597f640 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce6b2e0 0x55fabce68dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab9517e640 1 -- 172.21.15.18:0/2264210945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce6b2e0 msgr2=0x55fabce68dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab9517e640 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce6b2e0 0x55fabce68dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab9517e640 1 -- 172.21.15.18:0/2264210945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce62ad0 msgr2=0x55fabce68890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab9517e640 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce62ad0 0x55fabce68890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab9517e640 1 -- 172.21.15.18:0/2264210945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab88019040 con 0x55fabce61d60 2024-02-17T12:54:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab9497d640 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce62ad0 0x55fabce68890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab9517e640 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce61d60 0x55fabce68350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab8c012700 tx=0x7fab8c012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894bfd16927c0461 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab78ff9640 1 -- 172.21.15.18:0/2264210945 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c017070 con 0x55fabce61d60 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab99fbc740 1 -- 172.21.15.18:0/2264210945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fabcac9600 con 0x55fabce61d60 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.035+0000 7fab78ff9640 1 -- 172.21.15.18:0/2264210945 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fab8c0047a0 con 0x55fabce61d60 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.036+0000 7fab78ff9640 1 -- 172.21.15.18:0/2264210945 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab8c004960 con 0x55fabce61d60 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.036+0000 7fab99fbc740 10 client.26507 fetch_fsmap learned FSMap version 33 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.036+0000 7fab99fbc740 1 -- 172.21.15.18:0/2264210945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55fabcdc7d80 con 0x55fabce61d60 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.036+0000 7fab78ff9640 1 -- 172.21.15.18:0/2264210945 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fabcac9600 con 0x55fabce61d60 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.036+0000 7fab78ff9640 1 -- 172.21.15.18:0/2264210945 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fab8c00f6e0 con 0x55fabce61d60 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.036+0000 7fab99fbc740 10 client.26507 fetch_fsmap finished waiting for FSMap version 33 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.036+0000 7fab99fbc740 10 client.26507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.036+0000 7fab99fbc740 20 client.26507 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.036+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fabce7b6a0 0x55fabce7dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.036+0000 7fab99fbc740 4 client.26507 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.036+0000 7fab99fbc740 1 -- 172.21.15.18:0/2264210945 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fabcb1a1b0 con 0x55fabce7b6a0 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.036+0000 7fab9497d640 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fabce7b6a0 0x55fabce7dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab78ff9640 10 client.26507 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab9497d640 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fabce7b6a0 0x55fabce7dac0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f216d937abc5fb00 server_cookie=4f1335fc302ee68d in_seq=0 out_seq=0 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab78ff9640 1 -- 172.21.15.18:0/2264210945 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fabcb1a1b0 con 0x55fabce7b6a0 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab78ff9640 10 client.26507 handle_command_reply: tid=0 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab99fbc740 1 client.26507 shutdown 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab99fbc740 1 -- 172.21.15.18:0/2264210945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce61d60 msgr2=0x55fabce68350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce61d60 0x55fabce68350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab8c012700 tx=0x7fab8c012b00 comp rx=0 tx=0).stop 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab99fbc740 1 -- 172.21.15.18:0/2264210945 shutdown_connections 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fabce7b6a0 0x55fabce7dac0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce6b2e0 0x55fabce68dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce62ad0 0x55fabce68890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab78ff9640 0 client.26507 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2264210945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce61d60 0x55fabce68350 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.037+0000 7fab99fbc740 1 -- 172.21.15.18:0/2264210945 >> 172.21.15.18:0/2264210945 conn(0x55fabce29550 msgr2=0x55fabce5e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.038+0000 7fab99fbc740 1 -- 172.21.15.18:0/2264210945 shutdown_connections 2024-02-17T12:54:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.038+0000 7fab99fbc740 1 -- 172.21.15.18:0/2264210945 wait complete. 2024-02-17T12:54:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.038+0000 7fab99fbc740 20 client.26507 trim_cache size 0 max 16384 2024-02-17T12:54:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab99fbc740 1 Processor -- start 2024-02-17T12:54:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab99fbc740 1 -- start start 2024-02-17T12:54:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab99fbc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce61d60 0x55fabce6aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab99fbc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce62ad0 0x55fabce68bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab99fbc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce6b2e0 0x55fabce720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab99fbc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabce63f90 con 0x55fabce61d60 2024-02-17T12:54:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab99fbc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabce61030 con 0x55fabce62ad0 2024-02-17T12:54:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab99fbc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabce668c0 con 0x55fabce6b2e0 2024-02-17T12:54:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab9517e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce61d60 0x55fabce6aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab9517e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce61d60 0x55fabce6aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58378/0 (socket says 172.21.15.18:58378) 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab9517e640 1 -- 172.21.15.18:0/124967318 learned_addr learned my addr 172.21.15.18:0/124967318 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab9597f640 1 --2- 172.21.15.18:0/124967318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce6b2e0 0x55fabce720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab9517e640 1 -- 172.21.15.18:0/124967318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce6b2e0 msgr2=0x55fabce720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab9517e640 1 --2- 172.21.15.18:0/124967318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce6b2e0 0x55fabce720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab9517e640 1 -- 172.21.15.18:0/124967318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce62ad0 msgr2=0x55fabce68bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab9517e640 1 --2- 172.21.15.18:0/124967318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce62ad0 0x55fabce68bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.051+0000 7fab9517e640 1 -- 172.21.15.18:0/124967318 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fabcd647b0 con 0x55fabce61d60 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab9517e640 1 --2- 172.21.15.18:0/124967318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce61d60 0x55fabce6aa80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fab8c000c00 tx=0x7fab8c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c1c481db92c394 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab7affd640 1 -- 172.21.15.18:0/124967318 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c017070 con 0x55fabce61d60 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab7affd640 1 -- 172.21.15.18:0/124967318 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab8c018040 con 0x55fabce61d60 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab7affd640 1 -- 172.21.15.18:0/124967318 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c003850 con 0x55fabce61d60 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab99fbc740 1 -- 172.21.15.18:0/124967318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce61d60 msgr2=0x55fabce6aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab99fbc740 1 --2- 172.21.15.18:0/124967318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce61d60 0x55fabce6aa80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fab8c000c00 tx=0x7fab8c0096a0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab99fbc740 1 -- 172.21.15.18:0/124967318 shutdown_connections 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab99fbc740 1 --2- 172.21.15.18:0/124967318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce6b2e0 0x55fabce720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab99fbc740 1 --2- 172.21.15.18:0/124967318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce62ad0 0x55fabce68bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab99fbc740 1 --2- 172.21.15.18:0/124967318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce61d60 0x55fabce6aa80 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab99fbc740 1 -- 172.21.15.18:0/124967318 >> 172.21.15.18:0/124967318 conn(0x55fabce29550 msgr2=0x55fabce5c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab99fbc740 1 -- 172.21.15.18:0/124967318 shutdown_connections 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.052+0000 7fab99fbc740 1 -- 172.21.15.18:0/124967318 wait complete. 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab99fbc740 1 -- start start 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab99fbc740 1 Processor -- start 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab99fbc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce61d60 0x55fabce65000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab99fbc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce62ad0 0x55fabce65540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab99fbc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 0x55fabce65a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab99fbc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabced1be0 con 0x55fabce6b2e0 2024-02-17T12:54:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab99fbc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabced1d20 con 0x55fabce62ad0 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab99fbc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabced1e60 con 0x55fabce61d60 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab99fbc740 10 client.0 _ll_register_callbacks cb 0x7fab88009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab9597f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 0x55fabce65a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab9597f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 0x55fabce65a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58388/0 (socket says 172.21.15.18:58388) 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab9597f640 1 -- 172.21.15.18:0/2153420730 learned_addr learned my addr 172.21.15.18:0/2153420730 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.053+0000 7fab9517e640 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce61d60 0x55fabce65000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab9497d640 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce62ad0 0x55fabce65540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab9597f640 1 -- 172.21.15.18:0/2153420730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce61d60 msgr2=0x55fabce65000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab9597f640 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce61d60 0x55fabce65000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab9597f640 1 -- 172.21.15.18:0/2153420730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce62ad0 msgr2=0x55fabce65540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab9597f640 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce62ad0 0x55fabce65540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab9597f640 1 -- 172.21.15.18:0/2153420730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fabcdc7d80 con 0x55fabce6b2e0 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab9497d640 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce62ad0 0x55fabce65540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab9597f640 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 0x55fabce65a80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fab88005fd0 tx=0x7fab88003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b23241b93a48aa6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab78ff9640 1 -- 172.21.15.18:0/2153420730 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8800fcc0 con 0x55fabce6b2e0 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab78ff9640 1 -- 172.21.15.18:0/2153420730 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fab880114c0 con 0x55fabce6b2e0 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab99fbc740 1 -- 172.21.15.18:0/2153420730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab88009d90 con 0x55fabce6b2e0 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab78ff9640 1 -- 172.21.15.18:0/2153420730 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab8801dbc0 con 0x55fabce6b2e0 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab99fbc740 10 client.17151 fetch_fsmap learned FSMap version 33 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab99fbc740 1 -- 172.21.15.18:0/2153420730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55fabcd8a540 con 0x55fabce6b2e0 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.054+0000 7fab78ff9640 1 -- 172.21.15.18:0/2153420730 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab88009d90 con 0x55fabce6b2e0 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.055+0000 7fab78ff9640 1 -- 172.21.15.18:0/2153420730 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fab88024070 con 0x55fabce6b2e0 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.055+0000 7fab99fbc740 10 client.17151 fetch_fsmap finished waiting for FSMap version 33 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.055+0000 7fab99fbc740 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.055+0000 7fab99fbc740 20 client.17151 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.055+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fabced3600 0x55fabced5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.055+0000 7fab99fbc740 4 client.17151 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:54:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.055+0000 7fab99fbc740 1 -- 172.21.15.18:0/2153420730 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fabcb06550 con 0x55fabced3600 2024-02-17T12:54:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.055+0000 7fab9517e640 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fabced3600 0x55fabced5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.055+0000 7fab9517e640 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fabced3600 0x55fabced5a20 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff5fc1a7d1ece5f server_cookie=43e14d09e98d6d78 in_seq=0 out_seq=0 2024-02-17T12:54:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.055+0000 7fab78ff9640 10 client.17151 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.088+0000 7fab78ff9640 1 -- 172.21.15.18:0/2153420730 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fabcb06550 con 0x55fabced3600 2024-02-17T12:54:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.088+0000 7fab78ff9640 10 client.17151 handle_command_reply: tid=0 2024-02-17T12:54:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.088+0000 7fab99fbc740 1 client.17151 shutdown 2024-02-17T12:54:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.088+0000 7fab99fbc740 1 -- 172.21.15.18:0/2153420730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 msgr2=0x55fabce65a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.088+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 0x55fabce65a80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fab88005fd0 tx=0x7fab88003a80 comp rx=0 tx=0).stop 2024-02-17T12:54:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.088+0000 7fab99fbc740 1 -- 172.21.15.18:0/2153420730 shutdown_connections 2024-02-17T12:54:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.088+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fabced3600 0x55fabced5a20 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.088+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fabce6b2e0 0x55fabce65a80 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.088+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fabce62ad0 0x55fabce65540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.088+0000 7fab99fbc740 1 --2- 172.21.15.18:0/2153420730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fabce61d60 0x55fabce65000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.088+0000 7fab99fbc740 1 -- 172.21.15.18:0/2153420730 >> 172.21.15.18:0/2153420730 conn(0x55fabce29550 msgr2=0x55fabce66b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.088+0000 7fab78ff9640 0 client.17151 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.089+0000 7fab99fbc740 1 -- 172.21.15.18:0/2153420730 shutdown_connections 2024-02-17T12:54:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.089+0000 7fab99fbc740 1 -- 172.21.15.18:0/2153420730 wait complete. 2024-02-17T12:54:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.089+0000 7fab99fbc740 20 client.17151 trim_cache size 0 max 16384 2024-02-17T12:54:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:54:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:54:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:54:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:54:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.089+0000 7fab973e2640 1 -- 172.21.15.18:0/2748190195 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab7c0a2350 msgr2=0x7fab7c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.089+0000 7fab973e2640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab7c0a2350 0x7fab7c0a4810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fab8800fc90 tx=0x7fab8801e000 comp rx=0 tx=0).stop 2024-02-17T12:54:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.089+0000 7fab973e2640 1 -- 172.21.15.18:0/2748190195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab900691a0 msgr2=0x7fab901008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.089+0000 7fab973e2640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab900691a0 0x7fab901008e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fab80005940 tx=0x7fab8002c710 comp rx=0 tx=0).stop 2024-02-17T12:54:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.090+0000 7fab973e2640 1 -- 172.21.15.18:0/2748190195 shutdown_connections 2024-02-17T12:54:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.090+0000 7fab973e2640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab7c0a2350 0x7fab7c0a4810 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.090+0000 7fab973e2640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab90107270 0x7fab90100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.090+0000 7fab973e2640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab900691a0 0x7fab901008e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.090+0000 7fab973e2640 1 --2- 172.21.15.18:0/2748190195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab90068800 0x7fab901003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.090+0000 7fab973e2640 1 -- 172.21.15.18:0/2748190195 >> 172.21.15.18:0/2748190195 conn(0x7fab90076ed0 msgr2=0x7fab900feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.090+0000 7fab973e2640 1 -- 172.21.15.18:0/2748190195 shutdown_connections 2024-02-17T12:54:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:10.090+0000 7fab973e2640 1 -- 172.21.15.18:0/2748190195 wait complete. 2024-02-17T12:54:10.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 259055 9.73 MB/sec execute 85 sec latency 790.945 ms 2024-02-17T12:54:10.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:10 smithi018 ceph-mon[32625]: pgmap v131: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 301 op/s 2024-02-17T12:54:10.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:10 smithi119 ceph-mon[36188]: pgmap v131: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 301 op/s 2024-02-17T12:54:10.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:10 smithi146 ceph-mon[36454]: pgmap v131: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 301 op/s 2024-02-17T12:54:11.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 259055 9.62 MB/sec execute 86 sec latency 1790.979 ms 2024-02-17T12:54:12.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 259055 9.51 MB/sec execute 87 sec latency 2791.069 ms 2024-02-17T12:54:12.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:12 smithi018 ceph-mon[32625]: pgmap v132: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 2.2 MiB/s wr, 207 op/s 2024-02-17T12:54:12.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:12 smithi119 ceph-mon[36188]: pgmap v132: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 2.2 MiB/s wr, 207 op/s 2024-02-17T12:54:12.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:12 smithi146 ceph-mon[36454]: pgmap v132: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 2.2 MiB/s wr, 207 op/s 2024-02-17T12:54:13.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 259401 9.42 MB/sec execute 88 sec latency 3389.065 ms 2024-02-17T12:54:14.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 259563 9.31 MB/sec execute 89 sec latency 845.970 ms 2024-02-17T12:54:14.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:14 smithi018 ceph-mon[32625]: pgmap v133: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 2.2 MiB/s wr, 207 op/s 2024-02-17T12:54:14.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:14 smithi119 ceph-mon[36188]: pgmap v133: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 2.2 MiB/s wr, 207 op/s 2024-02-17T12:54:14.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:14 smithi146 ceph-mon[36454]: pgmap v133: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 2.2 MiB/s wr, 207 op/s 2024-02-17T12:54:15.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 259563 9.21 MB/sec execute 90 sec latency 1846.007 ms 2024-02-17T12:54:15.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:15 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:54:15.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:15 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:54:15.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:15 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:54:16.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 260394 9.16 MB/sec execute 91 sec latency 2434.577 ms 2024-02-17T12:54:16.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:16 smithi018 ceph-mon[32625]: pgmap v134: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 281 op/s 2024-02-17T12:54:16.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:16 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:54:16.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:16 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:54:16.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:16 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:54:16.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:16 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:54:16.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:16 smithi119 ceph-mon[36188]: pgmap v134: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 281 op/s 2024-02-17T12:54:16.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:16 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:54:16.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:16 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:54:16.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:16 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:54:16.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:16 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:54:16.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:16 smithi146 ceph-mon[36454]: pgmap v134: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 281 op/s 2024-02-17T12:54:16.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:16 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:54:16.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:16 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:54:16.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:16 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:54:16.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:16 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:54:17.270 INFO:tasks.workunit.client.0.smithi018.stdout: 1 262520 9.20 MB/sec execute 92 sec latency 58.791 ms 2024-02-17T12:54:18.271 INFO:tasks.workunit.client.0.smithi018.stdout: 1 262671 9.10 MB/sec execute 93 sec latency 633.045 ms 2024-02-17T12:54:18.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:18 smithi018 ceph-mon[32625]: pgmap v135: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-02-17T12:54:18.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:18 smithi119 ceph-mon[36188]: pgmap v135: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-02-17T12:54:18.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:18 smithi146 ceph-mon[36454]: pgmap v135: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-02-17T12:54:19.271 INFO:tasks.workunit.client.0.smithi018.stdout: 1 262671 9.01 MB/sec execute 94 sec latency 1315.374 ms 2024-02-17T12:54:20.271 INFO:tasks.workunit.client.0.smithi018.stdout: 1 262671 8.91 MB/sec execute 95 sec latency 2315.480 ms 2024-02-17T12:54:20.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:20 smithi018 ceph-mon[32625]: pgmap v136: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-02-17T12:54:20.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:20 smithi119 ceph-mon[36188]: pgmap v136: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-02-17T12:54:20.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:20 smithi146 ceph-mon[36454]: pgmap v136: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-02-17T12:54:21.271 INFO:tasks.workunit.client.0.smithi018.stdout: 1 262671 8.82 MB/sec execute 96 sec latency 3315.550 ms 2024-02-17T12:54:22.271 INFO:tasks.workunit.client.0.smithi018.stdout: 1 262671 8.73 MB/sec execute 97 sec latency 4315.672 ms 2024-02-17T12:54:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:22 smithi018 ceph-mon[32625]: pgmap v137: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-02-17T12:54:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:22 smithi119 ceph-mon[36188]: pgmap v137: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-02-17T12:54:22.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:22 smithi146 ceph-mon[36454]: pgmap v137: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-02-17T12:54:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.190+0000 7fa7ece6f640 1 -- 172.21.15.18:0/2068700305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101d40 msgr2=0x7fa7e8105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.190+0000 7fa7ece6f640 1 --2- 172.21.15.18:0/2068700305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101d40 0x7fa7e8105d80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009930 tx=0x7fa7dc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.190+0000 7fa7ece6f640 1 -- 172.21.15.18:0/2068700305 shutdown_connections 2024-02-17T12:54:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.190+0000 7fa7ece6f640 1 --2- 172.21.15.18:0/2068700305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8106460 0x7fa7e810e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.190+0000 7fa7ece6f640 1 --2- 172.21.15.18:0/2068700305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101d40 0x7fa7e8105d80 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.190+0000 7fa7ece6f640 1 --2- 172.21.15.18:0/2068700305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7e8101390 0x7fa7e8101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.190+0000 7fa7ece6f640 1 -- 172.21.15.18:0/2068700305 >> 172.21.15.18:0/2068700305 conn(0x7fa7e80fcc10 msgr2=0x7fa7e80ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.190+0000 7fa7ece6f640 1 -- 172.21.15.18:0/2068700305 shutdown_connections 2024-02-17T12:54:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.190+0000 7fa7ece6f640 1 -- 172.21.15.18:0/2068700305 wait complete. 2024-02-17T12:54:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7ece6f640 1 Processor -- start 2024-02-17T12:54:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7ece6f640 1 -- start start 2024-02-17T12:54:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7ece6f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101390 0x7fa7e8073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7ece6f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7e8106460 0x7fa7e80739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7ece6f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8073f00 0x7fa7e81a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7ece6f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e81a5f60 con 0x7fa7e8101390 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7ece6f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e81a60d0 con 0x7fa7e8106460 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7ece6f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e81a6290 con 0x7fa7e8073f00 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101390 0x7fa7e8073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101390 0x7fa7e8073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49214/0 (socket says 172.21.15.18:49214) 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7e659c640 1 -- 172.21.15.18:0/467448226 learned_addr learned my addr 172.21.15.18:0/467448226 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7e5d9b640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7e8106460 0x7fa7e80739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.191+0000 7fa7e6d9d640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8073f00 0x7fa7e81a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.192+0000 7fa7e659c640 1 -- 172.21.15.18:0/467448226 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8073f00 msgr2=0x7fa7e81a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.192+0000 7fa7e659c640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8073f00 0x7fa7e81a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.192+0000 7fa7e659c640 1 -- 172.21.15.18:0/467448226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7e8106460 msgr2=0x7fa7e80739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.192+0000 7fa7e659c640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7e8106460 0x7fa7e80739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.192+0000 7fa7e659c640 1 -- 172.21.15.18:0/467448226 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc009560 con 0x7fa7e8101390 2024-02-17T12:54:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.192+0000 7fa7e659c640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101390 0x7fa7e8073480 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800e9f0 tx=0x7fa7d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.192+0000 7fa7d77fe640 1 -- 172.21.15.18:0/467448226 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8009700 con 0x7fa7e8101390 2024-02-17T12:54:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.192+0000 7fa7d77fe640 1 -- 172.21.15.18:0/467448226 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7d800cd80 con 0x7fa7e8101390 2024-02-17T12:54:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.192+0000 7fa7ece6f640 1 -- 172.21.15.18:0/467448226 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e81a6570 con 0x7fa7e8101390 2024-02-17T12:54:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.192+0000 7fa7d77fe640 1 -- 172.21.15.18:0/467448226 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8018920 con 0x7fa7e8101390 2024-02-17T12:54:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.192+0000 7fa7ece6f640 1 -- 172.21.15.18:0/467448226 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e81a6aa0 con 0x7fa7e8101390 2024-02-17T12:54:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.194+0000 7fa7d77fe640 1 -- 172.21.15.18:0/467448226 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa7d8018a80 con 0x7fa7e8101390 2024-02-17T12:54:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7efa41740 1 Processor -- start 2024-02-17T12:54:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7efa41740 1 -- start start 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7efa41740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 0x55eecaef8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7efa41740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaef8d60 0x55eecaef91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7efa41740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 0x55eecaf046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7efa41740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecae679f0 con 0x55eecaef8d60 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7efa41740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecae64f20 con 0x55eecaf022e0 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7efa41740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecae60700 con 0x55eecaef9ad0 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7d77fe640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa7d00a2350 0x7fa7d00a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaef8d60 0x55eecaef91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7e5d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 0x55eecaf046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7e5d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 0x55eecaf046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:59608/0 (socket says 172.21.15.18:59608) 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7e5d9b640 1 -- 172.21.15.18:0/4085017200 learned_addr learned my addr 172.21.15.18:0/4085017200 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7e6d9d640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa7d00a2350 0x7fa7d00a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7e6d9d640 1 --2- 172.21.15.18:0/4085017200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 0x55eecaef8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.195+0000 7fa7d77fe640 1 -- 172.21.15.18:0/467448226 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa7d8014070 con 0x7fa7e8101390 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.196+0000 7fa7e5d9b640 1 -- 172.21.15.18:0/4085017200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 msgr2=0x55eecaef8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.196+0000 7fa7e5d9b640 1 --2- 172.21.15.18:0/4085017200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 0x55eecaef8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.196+0000 7fa7e5d9b640 1 -- 172.21.15.18:0/4085017200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaef8d60 msgr2=0x55eecaef91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.196+0000 7fa7e5d9b640 1 --2- 172.21.15.18:0/4085017200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaef8d60 0x55eecaef91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.196+0000 7fa7e5d9b640 1 -- 172.21.15.18:0/4085017200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eecadf67b0 con 0x55eecaf022e0 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.196+0000 7fa7e6d9d640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa7d00a2350 0x7fa7d00a4810 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0005fd0 tx=0x7fa7e0009e80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.196+0000 7fa7e5d9b640 1 --2- 172.21.15.18:0/4085017200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 0x55eecaf046d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc005ab0 tx=0x7fa7dc02c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2838c9e62fbbbb0a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.196+0000 7fa7d4ff9640 1 -- 172.21.15.18:0/4085017200 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc004800 con 0x55eecaf022e0 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.196+0000 7fa7d4ff9640 1 -- 172.21.15.18:0/4085017200 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7dc004960 con 0x55eecaf022e0 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.196+0000 7fa7d4ff9640 1 -- 172.21.15.18:0/4085017200 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc005030 con 0x55eecaf022e0 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.197+0000 7fa7efa41740 1 -- 172.21.15.18:0/4085017200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 msgr2=0x55eecaf046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.197+0000 7fa7efa41740 1 --2- 172.21.15.18:0/4085017200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 0x55eecaf046d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc005ab0 tx=0x7fa7dc02c5d0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.197+0000 7fa7efa41740 1 -- 172.21.15.18:0/4085017200 shutdown_connections 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.197+0000 7fa7efa41740 1 --2- 172.21.15.18:0/4085017200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 0x55eecaf046d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.197+0000 7fa7efa41740 1 --2- 172.21.15.18:0/4085017200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaef8d60 0x55eecaef91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.197+0000 7fa7efa41740 1 --2- 172.21.15.18:0/4085017200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 0x55eecaef8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.197+0000 7fa7efa41740 1 -- 172.21.15.18:0/4085017200 >> 172.21.15.18:0/4085017200 conn(0x55eecaec0550 msgr2=0x55eecaef3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.197+0000 7fa7efa41740 1 -- 172.21.15.18:0/4085017200 shutdown_connections 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.197+0000 7fa7efa41740 1 -- 172.21.15.18:0/4085017200 wait complete. 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.197+0000 7fa7efa41740 1 -- start start 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.197+0000 7fa7efa41740 1 Processor -- start 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7efa41740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaef8d60 0x55eecaeff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7efa41740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 0x55eecaeff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7efa41740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 0x55eecaeffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7efa41740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecae05bf0 con 0x55eecaef8d60 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7efa41740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecab34250 con 0x55eecaf022e0 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7efa41740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecade6200 con 0x55eecaef9ad0 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7efa41740 10 client.0 _ll_register_callbacks cb 0x55eecae54d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaef8d60 0x55eecaeff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaef8d60 0x55eecaeff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49228/0 (socket says 172.21.15.18:49228) 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7e659c640 1 -- 172.21.15.18:0/4050334410 learned_addr learned my addr 172.21.15.18:0/4050334410 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7e6d9d640 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 0x55eecaeffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7e5d9b640 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 0x55eecaeff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7e659c640 1 -- 172.21.15.18:0/4050334410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 msgr2=0x55eecaeff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7e659c640 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 0x55eecaeff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7e659c640 1 -- 172.21.15.18:0/4050334410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 msgr2=0x55eecaeffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7e659c640 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 0x55eecaeffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7e659c640 1 -- 172.21.15.18:0/4050334410 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e810e840 con 0x55eecaef8d60 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7e6d9d640 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 0x55eecaeffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7e659c640 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaef8d60 0x55eecaeff2a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80cbb60 tx=0x7fa7d80a2ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4732f64128bfc16b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.198+0000 7fa7cdffb640 1 -- 172.21.15.18:0/4050334410 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d809eef0 con 0x55eecaef8d60 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7cdffb640 1 -- 172.21.15.18:0/4050334410 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa7d80a2000 con 0x55eecaef8d60 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7efa41740 1 -- 172.21.15.18:0/4050334410 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eecab60600 con 0x55eecaef8d60 2024-02-17T12:54:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7cdffb640 1 -- 172.21.15.18:0/4050334410 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7d80a2430 con 0x55eecaef8d60 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7efa41740 10 client.17163 fetch_fsmap learned FSMap version 33 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7efa41740 1 -- 172.21.15.18:0/4050334410 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55eecae5ed80 con 0x55eecaef8d60 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7cdffb640 1 -- 172.21.15.18:0/4050334410 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7d80b9030 con 0x55eecaef8d60 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7cdffb640 1 -- 172.21.15.18:0/4050334410 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa7d80b91a0 con 0x55eecaef8d60 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7efa41740 10 client.17163 fetch_fsmap finished waiting for FSMap version 33 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7efa41740 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7efa41740 20 client.17163 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7efa41740 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eecaf12650 0x55eecaf14a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7efa41740 4 client.17163 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7efa41740 1 -- 172.21.15.18:0/4050334410 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eecabb11b0 con 0x55eecaf12650 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.199+0000 7fa7e5d9b640 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eecaf12650 0x55eecaf14a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.200+0000 7fa7e5d9b640 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eecaf12650 0x55eecaf14a40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78629292bd815b39 server_cookie=8040ea4a4aa9e67d in_seq=0 out_seq=0 2024-02-17T12:54:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.200+0000 7fa7cdffb640 10 client.17163 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7cdffb640 1 -- 172.21.15.18:0/4050334410 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eecabb11b0 con 0x55eecaf12650 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7cdffb640 10 client.17163 handle_command_reply: tid=0 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7efa41740 1 client.17163 shutdown 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7efa41740 1 -- 172.21.15.18:0/4050334410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaef8d60 msgr2=0x55eecaeff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7efa41740 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaef8d60 0x55eecaeff2a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80cbb60 tx=0x7fa7d80a2ea0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7efa41740 1 -- 172.21.15.18:0/4050334410 shutdown_connections 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7efa41740 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eecaf12650 0x55eecaf14a40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7efa41740 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaf022e0 0x55eecaeffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7efa41740 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 0x55eecaeff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7cdffb640 0 client.17163 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7efa41740 1 --2- 172.21.15.18:0/4050334410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaef8d60 0x55eecaeff2a0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7efa41740 1 -- 172.21.15.18:0/4050334410 >> 172.21.15.18:0/4050334410 conn(0x55eecaec0550 msgr2=0x55eecaef57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7efa41740 1 -- 172.21.15.18:0/4050334410 shutdown_connections 2024-02-17T12:54:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7efa41740 1 -- 172.21.15.18:0/4050334410 wait complete. 2024-02-17T12:54:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.201+0000 7fa7efa41740 20 client.17163 trim_cache size 0 max 16384 2024-02-17T12:54:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7efa41740 1 Processor -- start 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7efa41740 1 -- start start 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7efa41740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaef8d60 0x55eecaf019d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7efa41740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 0x55eecaeffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7efa41740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 0x55eecaefc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7efa41740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecaf0f0b0 con 0x55eecaf022e0 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7efa41740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecaf12450 con 0x55eecaef8d60 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7efa41740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecaf19c70 con 0x55eecaef9ad0 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7e6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 0x55eecaefc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7e6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 0x55eecaefc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49234/0 (socket says 172.21.15.18:49234) 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7e6d9d640 1 -- 172.21.15.18:0/761216746 learned_addr learned my addr 172.21.15.18:0/761216746 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7e659c640 1 --2- 172.21.15.18:0/761216746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaef8d60 0x55eecaf019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7e5d9b640 1 --2- 172.21.15.18:0/761216746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 0x55eecaeffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7e6d9d640 1 -- 172.21.15.18:0/761216746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 msgr2=0x55eecaeffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7e6d9d640 1 --2- 172.21.15.18:0/761216746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 0x55eecaeffb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7e6d9d640 1 -- 172.21.15.18:0/761216746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaef8d60 msgr2=0x55eecaf019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7e6d9d640 1 --2- 172.21.15.18:0/761216746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaef8d60 0x55eecaf019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7e6d9d640 1 -- 172.21.15.18:0/761216746 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eecadfb7b0 con 0x55eecaf022e0 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.215+0000 7fa7e659c640 1 --2- 172.21.15.18:0/761216746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaef8d60 0x55eecaf019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7e6d9d640 1 --2- 172.21.15.18:0/761216746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 0x55eecaefc500 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00095a0 tx=0x7fa7e0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1445751ae7010140 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7cffff640 1 -- 172.21.15.18:0/761216746 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0016050 con 0x55eecaf022e0 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7cffff640 1 -- 172.21.15.18:0/761216746 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7e0003e00 con 0x55eecaf022e0 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7cffff640 1 -- 172.21.15.18:0/761216746 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e001fe50 con 0x55eecaf022e0 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7efa41740 1 -- 172.21.15.18:0/761216746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 msgr2=0x55eecaefc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7efa41740 1 --2- 172.21.15.18:0/761216746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 0x55eecaefc500 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00095a0 tx=0x7fa7e0021000 comp rx=0 tx=0).stop 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7efa41740 1 -- 172.21.15.18:0/761216746 shutdown_connections 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7efa41740 1 --2- 172.21.15.18:0/761216746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 0x55eecaefc500 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7efa41740 1 --2- 172.21.15.18:0/761216746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef9ad0 0x55eecaeffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7efa41740 1 --2- 172.21.15.18:0/761216746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaef8d60 0x55eecaf019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7efa41740 1 -- 172.21.15.18:0/761216746 >> 172.21.15.18:0/761216746 conn(0x55eecaec0550 msgr2=0x55eecaefdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7efa41740 1 -- 172.21.15.18:0/761216746 shutdown_connections 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.216+0000 7fa7efa41740 1 -- 172.21.15.18:0/761216746 wait complete. 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7efa41740 1 -- start start 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7efa41740 1 Processor -- start 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7efa41740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef8d60 0x55eecaf07cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7efa41740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaef9ad0 0x55eecaf08200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7efa41740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 0x55eecaf08740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7efa41740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecaf68c40 con 0x55eecaf022e0 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7efa41740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecaf68db0 con 0x55eecaef9ad0 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7efa41740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecaf68f70 con 0x55eecaef8d60 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7efa41740 10 client.0 _ll_register_callbacks cb 0x7fa7e00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7e6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 0x55eecaf08740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7e5d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaef9ad0 0x55eecaf08200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7e6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 0x55eecaf08740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49248/0 (socket says 172.21.15.18:49248) 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7e6d9d640 1 -- 172.21.15.18:0/713710868 learned_addr learned my addr 172.21.15.18:0/713710868 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7e659c640 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef8d60 0x55eecaf07cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7e6d9d640 1 -- 172.21.15.18:0/713710868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef8d60 msgr2=0x55eecaf07cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7e6d9d640 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef8d60 0x55eecaf07cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7e6d9d640 1 -- 172.21.15.18:0/713710868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaef9ad0 msgr2=0x55eecaf08200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7e6d9d640 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaef9ad0 0x55eecaf08200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7e6d9d640 1 -- 172.21.15.18:0/713710868 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eecae5ed80 con 0x55eecaf022e0 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.217+0000 7fa7e5d9b640 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaef9ad0 0x55eecaf08200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7e6d9d640 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 0x55eecaf08740 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0004400 tx=0x7fa7e0009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a64994d1b885c39 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7cdffb640 1 -- 172.21.15.18:0/713710868 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e001fde0 con 0x55eecaf022e0 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7cdffb640 1 -- 172.21.15.18:0/713710868 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa7e0004b20 con 0x55eecaf022e0 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7efa41740 1 -- 172.21.15.18:0/713710868 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7dc004960 con 0x55eecaf022e0 2024-02-17T12:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7cdffb640 1 -- 172.21.15.18:0/713710868 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7e0016020 con 0x55eecaf022e0 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7efa41740 10 client.17169 fetch_fsmap learned FSMap version 33 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7efa41740 1 -- 172.21.15.18:0/713710868 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fa7e0019040 con 0x55eecaf022e0 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7cdffb640 1 -- 172.21.15.18:0/713710868 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7dc004960 con 0x55eecaf022e0 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7cdffb640 1 -- 172.21.15.18:0/713710868 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa7e0034a90 con 0x55eecaf022e0 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7efa41740 10 client.17169 fetch_fsmap finished waiting for FSMap version 33 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7efa41740 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7efa41740 20 client.17169 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7efa41740 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eecaf6a6d0 0x55eecaf6caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.218+0000 7fa7efa41740 4 client.17169 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.219+0000 7fa7efa41740 1 -- 172.21.15.18:0/713710868 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eecab9d550 con 0x55eecaf6a6d0 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.219+0000 7fa7e659c640 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eecaf6a6d0 0x55eecaf6caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.219+0000 7fa7e659c640 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eecaf6a6d0 0x55eecaf6caf0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f18343007b3e6983 server_cookie=3b9e2f0badcd7dec in_seq=0 out_seq=0 2024-02-17T12:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.219+0000 7fa7cdffb640 10 client.17169 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7cdffb640 1 -- 172.21.15.18:0/713710868 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eecab9d550 con 0x55eecaf6a6d0 2024-02-17T12:54:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7cdffb640 10 client.17169 handle_command_reply: tid=0 2024-02-17T12:54:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7efa41740 1 client.17169 shutdown 2024-02-17T12:54:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7efa41740 1 -- 172.21.15.18:0/713710868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 msgr2=0x55eecaf08740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7efa41740 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 0x55eecaf08740 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0004400 tx=0x7fa7e0009480 comp rx=0 tx=0).stop 2024-02-17T12:54:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7efa41740 1 -- 172.21.15.18:0/713710868 shutdown_connections 2024-02-17T12:54:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7efa41740 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55eecaf6a6d0 0x55eecaf6caf0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7efa41740 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eecaf022e0 0x55eecaf08740 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7efa41740 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eecaef9ad0 0x55eecaf08200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7efa41740 1 --2- 172.21.15.18:0/713710868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eecaef8d60 0x55eecaf07cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7cdffb640 0 client.17169 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7efa41740 1 -- 172.21.15.18:0/713710868 >> 172.21.15.18:0/713710868 conn(0x55eecaec0550 msgr2=0x55eecaef2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7efa41740 1 -- 172.21.15.18:0/713710868 shutdown_connections 2024-02-17T12:54:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.257+0000 7fa7efa41740 1 -- 172.21.15.18:0/713710868 wait complete. 2024-02-17T12:54:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7efa41740 20 client.17169 trim_cache size 0 max 16384 2024-02-17T12:54:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:54:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:54:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:54:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:54:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7ece6f640 1 -- 172.21.15.18:0/467448226 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa7d00a2350 msgr2=0x7fa7d00a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7ece6f640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa7d00a2350 0x7fa7d00a4810 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0005fd0 tx=0x7fa7e0009e80 comp rx=0 tx=0).stop 2024-02-17T12:54:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7ece6f640 1 -- 172.21.15.18:0/467448226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101390 msgr2=0x7fa7e8073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7ece6f640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101390 0x7fa7e8073480 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800e9f0 tx=0x7fa7d800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7ece6f640 1 -- 172.21.15.18:0/467448226 shutdown_connections 2024-02-17T12:54:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7ece6f640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa7d00a2350 0x7fa7d00a4810 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7ece6f640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7e8073f00 0x7fa7e81a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7ece6f640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7e8106460 0x7fa7e80739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7ece6f640 1 --2- 172.21.15.18:0/467448226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8101390 0x7fa7e8073480 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7ece6f640 1 -- 172.21.15.18:0/467448226 >> 172.21.15.18:0/467448226 conn(0x7fa7e80fcc10 msgr2=0x7fa7e80fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7ece6f640 1 -- 172.21.15.18:0/467448226 shutdown_connections 2024-02-17T12:54:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:23.258+0000 7fa7ece6f640 1 -- 172.21.15.18:0/467448226 wait complete. 2024-02-17T12:54:23.271 INFO:tasks.workunit.client.0.smithi018.stdout: 1 262938 8.65 MB/sec execute 98 sec latency 4897.845 ms 2024-02-17T12:54:24.271 INFO:tasks.workunit.client.0.smithi018.stdout: 1 264890 8.64 MB/sec execute 99 sec latency 18.108 ms 2024-02-17T12:54:24.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:24 smithi018 ceph-mon[32625]: pgmap v138: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-02-17T12:54:24.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:24 smithi119 ceph-mon[36188]: pgmap v138: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-02-17T12:54:24.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:24 smithi146 ceph-mon[36454]: pgmap v138: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-02-17T12:54:25.271 INFO:tasks.workunit.client.0.smithi018.stdout: 1 266145 8.66 MB/sec execute 100 sec latency 467.547 ms 2024-02-17T12:54:26.271 INFO:tasks.workunit.client.0.smithi018.stdout: 1 266145 8.57 MB/sec execute 101 sec latency 1467.649 ms 2024-02-17T12:54:26.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:26 smithi018 ceph-mon[32625]: pgmap v139: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-02-17T12:54:26.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:26 smithi119 ceph-mon[36188]: pgmap v139: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-02-17T12:54:26.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:26 smithi146 ceph-mon[36454]: pgmap v139: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-02-17T12:54:27.271 INFO:tasks.workunit.client.0.smithi018.stdout: 1 266145 8.49 MB/sec execute 102 sec latency 2467.741 ms 2024-02-17T12:54:28.271 INFO:tasks.workunit.client.0.smithi018.stdout: 1 266215 8.41 MB/sec execute 103 sec latency 3049.485 ms 2024-02-17T12:54:28.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:28 smithi018 ceph-mon[32625]: pgmap v140: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-02-17T12:54:28.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:28 smithi119 ceph-mon[36188]: pgmap v140: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-02-17T12:54:28.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:28 smithi146 ceph-mon[36454]: pgmap v140: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-02-17T12:54:29.271 INFO:tasks.workunit.client.0.smithi018.stdout: 1 266215 8.32 MB/sec execute 104 sec latency 1354.318 ms 2024-02-17T12:54:30.272 INFO:tasks.workunit.client.0.smithi018.stdout: 1 266215 8.24 MB/sec execute 105 sec latency 2354.466 ms 2024-02-17T12:54:30.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:30 smithi018 ceph-mon[32625]: pgmap v141: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-02-17T12:54:30.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:30 smithi119 ceph-mon[36188]: pgmap v141: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-02-17T12:54:30.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:30 smithi146 ceph-mon[36454]: pgmap v141: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-02-17T12:54:31.272 INFO:tasks.workunit.client.0.smithi018.stdout: 1 266215 8.17 MB/sec execute 106 sec latency 3354.520 ms 2024-02-17T12:54:32.272 INFO:tasks.workunit.client.0.smithi018.stdout: 1 266215 8.09 MB/sec execute 107 sec latency 4354.592 ms 2024-02-17T12:54:32.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:32 smithi018 ceph-mon[32625]: pgmap v142: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-02-17T12:54:32.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:32 smithi119 ceph-mon[36188]: pgmap v142: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-02-17T12:54:32.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:32 smithi146 ceph-mon[36454]: pgmap v142: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-02-17T12:54:33.272 INFO:tasks.workunit.client.0.smithi018.stdout: 1 266534 8.03 MB/sec execute 108 sec latency 4928.925 ms 2024-02-17T12:54:34.272 INFO:tasks.workunit.client.0.smithi018.stdout: 1 268551 8.03 MB/sec execute 109 sec latency 24.942 ms 2024-02-17T12:54:34.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:34 smithi018 ceph-mon[32625]: pgmap v143: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-02-17T12:54:34.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:34 smithi119 ceph-mon[36188]: pgmap v143: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-02-17T12:54:34.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:34 smithi146 ceph-mon[36454]: pgmap v143: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-02-17T12:54:35.272 INFO:tasks.workunit.client.0.smithi018.stdout: 1 269692 8.04 MB/sec execute 110 sec latency 556.712 ms 2024-02-17T12:54:36.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:54:36.102 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.230+0000 7fa59a664640 1 -- 172.21.15.18:0/1603259235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa594102cc0 msgr2=0x7fa59410b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.230+0000 7fa59a664640 1 --2- 172.21.15.18:0/1603259235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa594102cc0 0x7fa59410b1a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa588009900 tx=0x7fa58802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.230+0000 7fa59a664640 1 -- 172.21.15.18:0/1603259235 shutdown_connections 2024-02-17T12:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.230+0000 7fa59a664640 1 --2- 172.21.15.18:0/1603259235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa59410b6e0 0x7fa59410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.230+0000 7fa59a664640 1 --2- 172.21.15.18:0/1603259235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa594102cc0 0x7fa59410b1a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.230+0000 7fa59a664640 1 --2- 172.21.15.18:0/1603259235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5941023a0 0x7fa594102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.230+0000 7fa59a664640 1 -- 172.21.15.18:0/1603259235 >> 172.21.15.18:0/1603259235 conn(0x7fa594077230 msgr2=0x7fa594077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.230+0000 7fa59a664640 1 -- 172.21.15.18:0/1603259235 shutdown_connections 2024-02-17T12:54:36.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.230+0000 7fa59a664640 1 -- 172.21.15.18:0/1603259235 wait complete. 2024-02-17T12:54:36.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.230+0000 7fa59a664640 1 Processor -- start 2024-02-17T12:54:36.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.230+0000 7fa59a664640 1 -- start start 2024-02-17T12:54:36.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa59a664640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5941023a0 0x7fa59419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa59a664640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa594102cc0 0x7fa59419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa59a664640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa59410b6e0 0x7fa5941975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa59a664640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594197c30 con 0x7fa594102cc0 2024-02-17T12:54:36.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa59a664640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594197da0 con 0x7fa5941023a0 2024-02-17T12:54:36.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa59a664640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594197f60 con 0x7fa59410b6e0 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa5937fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa594102cc0 0x7fa59419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa5937fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa594102cc0 0x7fa59419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42634/0 (socket says 172.21.15.18:42634) 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa5937fe640 1 -- 172.21.15.18:0/2295084865 learned_addr learned my addr 172.21.15.18:0/2295084865 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa593fff640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5941023a0 0x7fa59419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa598c01640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa59410b6e0 0x7fa5941975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa5937fe640 1 -- 172.21.15.18:0/2295084865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa59410b6e0 msgr2=0x7fa5941975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa5937fe640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa59410b6e0 0x7fa5941975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa5937fe640 1 -- 172.21.15.18:0/2295084865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5941023a0 msgr2=0x7fa59419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa5937fe640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5941023a0 0x7fa59419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa5937fe640 1 -- 172.21.15.18:0/2295084865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa588009560 con 0x7fa594102cc0 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa598c01640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa59410b6e0 0x7fa5941975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa5937fe640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa594102cc0 0x7fa59419d970 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa588005940 tx=0x7fa58802c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.231+0000 7fa5917fa640 1 -- 172.21.15.18:0/2295084865 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5880043d0 con 0x7fa594102cc0 2024-02-17T12:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.232+0000 7fa59a664640 1 -- 172.21.15.18:0/2295084865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5941981e0 con 0x7fa594102cc0 2024-02-17T12:54:36.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.232+0000 7fa5917fa640 1 -- 172.21.15.18:0/2295084865 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5880052d0 con 0x7fa594102cc0 2024-02-17T12:54:36.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.232+0000 7fa5917fa640 1 -- 172.21.15.18:0/2295084865 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588003d30 con 0x7fa594102cc0 2024-02-17T12:54:36.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.232+0000 7fa59a664640 1 -- 172.21.15.18:0/2295084865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5941a44d0 con 0x7fa594102cc0 2024-02-17T12:54:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.234+0000 7fa5917fa640 1 -- 172.21.15.18:0/2295084865 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa588005440 con 0x7fa594102cc0 2024-02-17T12:54:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.234+0000 7fa59d236740 1 Processor -- start 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.234+0000 7fa59d236740 1 -- start start 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.234+0000 7fa5917fa640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa57c0a2230 0x7fa57c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.234+0000 7fa59d236740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ed43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa59d236740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 0x563ff4ee1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa59d236740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ee1bc0 0x563ff4ee3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa59d236740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4e43280 con 0x563ff4ed4900 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa59d236740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4e407b0 con 0x563ff4ed3fe0 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa59d236740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4e3bf90 con 0x563ff4ee1bc0 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa5917fa640 1 -- 172.21.15.18:0/2295084865 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa5880f0760 con 0x7fa594102cc0 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa598c01640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa57c0a2230 0x7fa57c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa5937fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 0x563ff4ee1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa5937fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 0x563ff4ee1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42642/0 (socket says 172.21.15.18:42642) 2024-02-17T12:54:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa5937fe640 1 -- 172.21.15.18:0/2025962697 learned_addr learned my addr 172.21.15.18:0/2025962697 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa593fff640 1 --2- 172.21.15.18:0/2025962697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ed43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa598c01640 1 --2- 172.21.15.18:0/2025962697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ee1bc0 0x563ff4ee3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa5937fe640 1 -- 172.21.15.18:0/2025962697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ee1bc0 msgr2=0x563ff4ee3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa5937fe640 1 --2- 172.21.15.18:0/2025962697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ee1bc0 0x563ff4ee3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa5937fe640 1 -- 172.21.15.18:0/2025962697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 msgr2=0x563ff4ed43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa5937fe640 1 --2- 172.21.15.18:0/2025962697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ed43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa5937fe640 1 -- 172.21.15.18:0/2025962697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ff4dfa570 con 0x563ff4ed4900 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa598c01640 1 --2- 172.21.15.18:0/2025962697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ee1bc0 0x563ff4ee3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa598c01640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa57c0a2230 0x7fa57c0a46f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005c40 tx=0x7fa58c013a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa5937fe640 1 --2- 172.21.15.18:0/2025962697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 0x563ff4ee1680 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa5880c5640 tx=0x7fa5880bd220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476bb2214149b065 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa57a7fc640 1 -- 172.21.15.18:0/2025962697 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5880bdad0 con 0x563ff4ed4900 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa57a7fc640 1 -- 172.21.15.18:0/2025962697 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5880bd3d0 con 0x563ff4ed4900 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.235+0000 7fa57a7fc640 1 -- 172.21.15.18:0/2025962697 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5880c1170 con 0x563ff4ed4900 2024-02-17T12:54:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.236+0000 7fa59d236740 1 -- 172.21.15.18:0/2025962697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 msgr2=0x563ff4ee1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.236+0000 7fa59d236740 1 --2- 172.21.15.18:0/2025962697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 0x563ff4ee1680 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa5880c5640 tx=0x7fa5880bd220 comp rx=0 tx=0).stop 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.236+0000 7fa59d236740 1 -- 172.21.15.18:0/2025962697 shutdown_connections 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.236+0000 7fa59d236740 1 --2- 172.21.15.18:0/2025962697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ee1bc0 0x563ff4ee3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.236+0000 7fa59d236740 1 --2- 172.21.15.18:0/2025962697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 0x563ff4ee1680 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.236+0000 7fa59d236740 1 --2- 172.21.15.18:0/2025962697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ed43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.236+0000 7fa59d236740 1 -- 172.21.15.18:0/2025962697 >> 172.21.15.18:0/2025962697 conn(0x563ff4ece990 msgr2=0x563ff4eced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.236+0000 7fa59d236740 1 -- 172.21.15.18:0/2025962697 shutdown_connections 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.236+0000 7fa59d236740 1 -- 172.21.15.18:0/2025962697 wait complete. 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.236+0000 7fa59d236740 1 -- start start 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa59d236740 1 Processor -- start 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa59d236740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa59d236740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 0x563ff4ede980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa59d236740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ee1bc0 0x563ff4edf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa59d236740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4b0f430 con 0x563ff4ed4900 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa59d236740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4acbe00 con 0x563ff4ed3fe0 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa59d236740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4dc0fd0 con 0x563ff4ee1bc0 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa59d236740 10 client.0 _ll_register_callbacks cb 0x563ff4e16fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa5937fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 0x563ff4ede980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa5937fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 0x563ff4ede980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42650/0 (socket says 172.21.15.18:42650) 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa5937fe640 1 -- 172.21.15.18:0/406495234 learned_addr learned my addr 172.21.15.18:0/406495234 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa593fff640 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa598c01640 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ee1bc0 0x563ff4edf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa5937fe640 1 -- 172.21.15.18:0/406495234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ee1bc0 msgr2=0x563ff4edf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa5937fe640 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ee1bc0 0x563ff4edf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa5937fe640 1 -- 172.21.15.18:0/406495234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 msgr2=0x563ff4ee15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa5937fe640 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa5937fe640 1 -- 172.21.15.18:0/406495234 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5941a44d0 con 0x563ff4ed4900 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa598c01640 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ee1bc0 0x563ff4edf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.237+0000 7fa593fff640 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa5937fe640 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 0x563ff4ede980 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa5880b9830 tx=0x7fa5880045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124a5b6e56484ddb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa5737fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa5737fe640 1 -- 172.21.15.18:0/406495234 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5880bdad0 con 0x563ff4ed4900 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa5737fe640 1 -- 172.21.15.18:0/406495234 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa5880c1520 con 0x563ff4ed4900 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa59d236740 1 -- 172.21.15.18:0/406495234 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ff4b3b7e0 con 0x563ff4ed4900 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa5737fe640 1 -- 172.21.15.18:0/406495234 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5880ee180 con 0x563ff4ed4900 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa59d236740 10 client.17181 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa59d236740 1 -- 172.21.15.18:0/406495234 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563ff4e3a610 con 0x563ff4ed4900 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa5737fe640 1 -- 172.21.15.18:0/406495234 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563ff4b3b7e0 con 0x563ff4ed4900 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa5737fe640 1 -- 172.21.15.18:0/406495234 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa58803c070 con 0x563ff4ed4900 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa59d236740 10 client.17181 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa59d236740 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa59d236740 20 client.17181 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa59d236740 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ff4ef1630 0x563ff4ef3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa59d236740 4 client.17181 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.238+0000 7fa59d236740 1 -- 172.21.15.18:0/406495234 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ff4b78720 con 0x563ff4ef1630 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.239+0000 7fa593fff640 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ff4ef1630 0x563ff4ef3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.239+0000 7fa5737fe640 10 client.17181 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.239+0000 7fa593fff640 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ff4ef1630 0x563ff4ef3a90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b5c03cbdec95c8 server_cookie=8009cdf3873b71c4 in_seq=0 out_seq=0 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa5737fe640 1 -- 172.21.15.18:0/406495234 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ff4b78720 con 0x563ff4ef1630 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa5737fe640 10 client.17181 handle_command_reply: tid=0 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa59d236740 1 client.17181 shutdown 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa59d236740 1 -- 172.21.15.18:0/406495234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 msgr2=0x563ff4ede980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa59d236740 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 0x563ff4ede980 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa5880b9830 tx=0x7fa5880045a0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa59d236740 1 -- 172.21.15.18:0/406495234 shutdown_connections 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa59d236740 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ff4ef1630 0x563ff4ef3a90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa59d236740 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ee1bc0 0x563ff4edf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa59d236740 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed4900 0x563ff4ede980 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa59d236740 1 --2- 172.21.15.18:0/406495234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa5737fe640 0 client.17181 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa59d236740 1 -- 172.21.15.18:0/406495234 >> 172.21.15.18:0/406495234 conn(0x563ff4ece990 msgr2=0x563ff4ed0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa59d236740 1 -- 172.21.15.18:0/406495234 shutdown_connections 2024-02-17T12:54:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa59d236740 1 -- 172.21.15.18:0/406495234 wait complete. 2024-02-17T12:54:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.240+0000 7fa59d236740 20 client.17181 trim_cache size 0 max 16384 2024-02-17T12:54:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.255+0000 7fa59d236740 1 Processor -- start 2024-02-17T12:54:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.255+0000 7fa59d236740 1 -- start start 2024-02-17T12:54:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa59d236740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa59d236740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ed4900 0x563ff4ee15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa59d236740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ee1bc0 0x563ff4edd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa59d236740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4eed5f0 con 0x563ff4ed3fe0 2024-02-17T12:54:36.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa59d236740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4ee4330 con 0x563ff4ee1bc0 2024-02-17T12:54:36.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa59d236740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4ed3c10 con 0x563ff4ed4900 2024-02-17T12:54:36.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa593fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa593fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42666/0 (socket says 172.21.15.18:42666) 2024-02-17T12:54:36.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa593fff640 1 -- 172.21.15.18:0/3224897896 learned_addr learned my addr 172.21.15.18:0/3224897896 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa5937fe640 1 --2- 172.21.15.18:0/3224897896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ed4900 0x563ff4ee15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa598c01640 1 --2- 172.21.15.18:0/3224897896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ee1bc0 0x563ff4edd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa593fff640 1 -- 172.21.15.18:0/3224897896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ed4900 msgr2=0x563ff4ee15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa593fff640 1 --2- 172.21.15.18:0/3224897896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ed4900 0x563ff4ee15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa593fff640 1 -- 172.21.15.18:0/3224897896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ee1bc0 msgr2=0x563ff4edd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa593fff640 1 --2- 172.21.15.18:0/3224897896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ee1bc0 0x563ff4edd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa593fff640 1 -- 172.21.15.18:0/3224897896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ff4dd6ac0 con 0x563ff4ed3fe0 2024-02-17T12:54:36.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa598c01640 1 --2- 172.21.15.18:0/3224897896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ee1bc0 0x563ff4edd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:36.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa593fff640 1 --2- 172.21.15.18:0/3224897896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee10b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x563ff4ef4370 tx=0x7fa584012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0dab1c48832b97e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa579ffb640 1 -- 172.21.15.18:0/3224897896 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840122d0 con 0x563ff4ed3fe0 2024-02-17T12:54:36.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa579ffb640 1 -- 172.21.15.18:0/3224897896 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa584005500 con 0x563ff4ed3fe0 2024-02-17T12:54:36.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.256+0000 7fa579ffb640 1 -- 172.21.15.18:0/3224897896 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58401e2d0 con 0x563ff4ed3fe0 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.257+0000 7fa59d236740 1 -- 172.21.15.18:0/3224897896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed3fe0 msgr2=0x563ff4ee10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.257+0000 7fa59d236740 1 --2- 172.21.15.18:0/3224897896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee10b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x563ff4ef4370 tx=0x7fa584012820 comp rx=0 tx=0).stop 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.257+0000 7fa59d236740 1 -- 172.21.15.18:0/3224897896 shutdown_connections 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.257+0000 7fa59d236740 1 --2- 172.21.15.18:0/3224897896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ee1bc0 0x563ff4edd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.257+0000 7fa59d236740 1 --2- 172.21.15.18:0/3224897896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ed4900 0x563ff4ee15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.257+0000 7fa59d236740 1 --2- 172.21.15.18:0/3224897896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee10b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.257+0000 7fa59d236740 1 -- 172.21.15.18:0/3224897896 >> 172.21.15.18:0/3224897896 conn(0x563ff4ece990 msgr2=0x563ff4ed9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.257+0000 7fa59d236740 1 -- 172.21.15.18:0/3224897896 shutdown_connections 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.257+0000 7fa59d236740 1 -- 172.21.15.18:0/3224897896 wait complete. 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.257+0000 7fa59d236740 1 -- start start 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa59d236740 1 Processor -- start 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa59d236740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa59d236740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ed4900 0x563ff4ee4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa59d236740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ee1bc0 0x563ff4ee53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa59d236740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4ee5a80 con 0x563ff4ee1bc0 2024-02-17T12:54:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa59d236740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4ee5bf0 con 0x563ff4ed3fe0 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa59d236740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4f47b70 con 0x563ff4ed4900 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa59d236740 10 client.0 _ll_register_callbacks cb 0x7fa588005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa598c01640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ee1bc0 0x563ff4ee53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa598c01640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ee1bc0 0x563ff4ee53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42668/0 (socket says 172.21.15.18:42668) 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa598c01640 1 -- 172.21.15.18:0/3143760379 learned_addr learned my addr 172.21.15.18:0/3143760379 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa593fff640 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa5937fe640 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ed4900 0x563ff4ee4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa598c01640 1 -- 172.21.15.18:0/3143760379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ed4900 msgr2=0x563ff4ee4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa598c01640 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ed4900 0x563ff4ee4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa598c01640 1 -- 172.21.15.18:0/3143760379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 msgr2=0x563ff4ee8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa598c01640 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa598c01640 1 -- 172.21.15.18:0/3143760379 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ff4dfc6a0 con 0x563ff4ee1bc0 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa593fff640 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa598c01640 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ee1bc0 0x563ff4ee53d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005fd0 tx=0x7fa58c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb0b3c17711c294 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.258+0000 7fa5737fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa5737fe640 1 -- 172.21.15.18:0/3143760379 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c004d20 con 0x563ff4ee1bc0 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa5737fe640 1 -- 172.21.15.18:0/3143760379 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa58c021610 con 0x563ff4ee1bc0 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa59d236740 1 -- 172.21.15.18:0/3143760379 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5880bd3d0 con 0x563ff4ee1bc0 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa5737fe640 1 -- 172.21.15.18:0/3143760379 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa58c004e80 con 0x563ff4ee1bc0 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa59d236740 10 client.17187 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa59d236740 1 -- 172.21.15.18:0/3143760379 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563ff4de2e30 con 0x563ff4ee1bc0 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa5737fe640 1 -- 172.21.15.18:0/3143760379 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5880bd3d0 con 0x563ff4ee1bc0 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa5737fe640 1 -- 172.21.15.18:0/3143760379 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa58c027030 con 0x563ff4ee1bc0 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa59d236740 10 client.17187 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa59d236740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa59d236740 20 client.17187 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa59d236740 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ff4f49260 0x563ff4f4b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa59d236740 4 client.17187 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.259+0000 7fa59d236740 1 -- 172.21.15.18:0/3143760379 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563ff4bc7890 con 0x563ff4f49260 2024-02-17T12:54:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa593fff640 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ff4f49260 0x563ff4f4b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa5737fe640 10 client.17187 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa593fff640 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ff4f49260 0x563ff4f4b680 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6380f743c2091d0 server_cookie=db019fbca33954c5 in_seq=0 out_seq=0 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa5737fe640 1 -- 172.21.15.18:0/3143760379 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563ff4bc7890 con 0x563ff4f49260 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa5737fe640 10 client.17187 handle_command_reply: tid=0 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa59d236740 1 client.17187 shutdown 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa59d236740 1 -- 172.21.15.18:0/3143760379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ee1bc0 msgr2=0x563ff4ee53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa59d236740 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ee1bc0 0x563ff4ee53d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005fd0 tx=0x7fa58c019000 comp rx=0 tx=0).stop 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa59d236740 1 -- 172.21.15.18:0/3143760379 shutdown_connections 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa59d236740 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563ff4f49260 0x563ff4f4b680 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa59d236740 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ff4ee1bc0 0x563ff4ee53d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa5737fe640 0 client.17187 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa59d236740 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ff4ed4900 0x563ff4ee4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa59d236740 1 --2- 172.21.15.18:0/3143760379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563ff4ed3fe0 0x563ff4ee8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa59d236740 1 -- 172.21.15.18:0/3143760379 >> 172.21.15.18:0/3143760379 conn(0x563ff4ece990 msgr2=0x563ff4edd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa59d236740 1 -- 172.21.15.18:0/3143760379 shutdown_connections 2024-02-17T12:54:36.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.260+0000 7fa59d236740 1 -- 172.21.15.18:0/3143760379 wait complete. 2024-02-17T12:54:36.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59d236740 20 client.17187 trim_cache size 0 max 16384 2024-02-17T12:54:36.262 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:54:36.262 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:54:36.262 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:54:36.262 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:54:36.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59a664640 1 -- 172.21.15.18:0/2295084865 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa57c0a2230 msgr2=0x7fa57c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59a664640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa57c0a2230 0x7fa57c0a46f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005c40 tx=0x7fa58c013a40 comp rx=0 tx=0).stop 2024-02-17T12:54:36.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59a664640 1 -- 172.21.15.18:0/2295084865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa594102cc0 msgr2=0x7fa59419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59a664640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa594102cc0 0x7fa59419d970 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa588005940 tx=0x7fa58802c780 comp rx=0 tx=0).stop 2024-02-17T12:54:36.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59a664640 1 -- 172.21.15.18:0/2295084865 shutdown_connections 2024-02-17T12:54:36.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59a664640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa57c0a2230 0x7fa57c0a46f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59a664640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa59410b6e0 0x7fa5941975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59a664640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa594102cc0 0x7fa59419d970 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59a664640 1 --2- 172.21.15.18:0/2295084865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5941023a0 0x7fa59419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59a664640 1 -- 172.21.15.18:0/2295084865 >> 172.21.15.18:0/2295084865 conn(0x7fa594077230 msgr2=0x7fa5940fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59a664640 1 -- 172.21.15.18:0/2295084865 shutdown_connections 2024-02-17T12:54:36.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.261+0000 7fa59a664640 1 -- 172.21.15.18:0/2295084865 wait complete. 2024-02-17T12:54:36.265 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:54:36.265 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:54:36.272 INFO:tasks.workunit.client.0.smithi018.stdout: 1 269692 7.97 MB/sec execute 111 sec latency 1556.756 ms 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.352+0000 7f00c33d7640 1 -- 172.21.15.18:0/2704940339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc1002d0 msgr2=0x7f00bc100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.352+0000 7f00c33d7640 1 --2- 172.21.15.18:0/2704940339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc1002d0 0x7f00bc100750 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009900 tx=0x7f00ac02bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.352+0000 7f00c33d7640 1 -- 172.21.15.18:0/2704940339 shutdown_connections 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.352+0000 7f00c33d7640 1 --2- 172.21.15.18:0/2704940339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00bc100c90 0x7f00bc10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.352+0000 7f00c33d7640 1 --2- 172.21.15.18:0/2704940339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc1002d0 0x7f00bc100750 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.352+0000 7f00c33d7640 1 --2- 172.21.15.18:0/2704940339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00bc105020 0x7f00bc105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.352+0000 7f00c33d7640 1 -- 172.21.15.18:0/2704940339 >> 172.21.15.18:0/2704940339 conn(0x7f00bc076ed0 msgr2=0x7f00bc0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.352+0000 7f00c33d7640 1 -- 172.21.15.18:0/2704940339 shutdown_connections 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.352+0000 7f00c33d7640 1 -- 172.21.15.18:0/2704940339 wait complete. 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.352+0000 7f00c33d7640 1 Processor -- start 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.352+0000 7f00c33d7640 1 -- start start 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c33d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00bc1002d0 0x7f00bc198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c33d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00bc100c90 0x7f00bc198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c33d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc105020 0x7f00bc1a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c33d7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc1993e0 con 0x7f00bc105020 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c33d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc1a3be0 con 0x7f00bc1002d0 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c33d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc1a3d20 con 0x7f00bc100c90 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c1974640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc105020 0x7f00bc1a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c1974640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc105020 0x7f00bc1a36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42674/0 (socket says 172.21.15.18:42674) 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c1974640 1 -- 172.21.15.18:0/218230371 learned_addr learned my addr 172.21.15.18:0/218230371 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c1173640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00bc1002d0 0x7f00bc198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c0972640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00bc100c90 0x7f00bc198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c1974640 1 -- 172.21.15.18:0/218230371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00bc100c90 msgr2=0x7f00bc198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c1974640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00bc100c90 0x7f00bc198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c1974640 1 -- 172.21.15.18:0/218230371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00bc1002d0 msgr2=0x7f00bc198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c1974640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00bc1002d0 0x7f00bc198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c1974640 1 -- 172.21.15.18:0/218230371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac009560 con 0x7f00bc105020 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c1974640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc105020 0x7f00bc1a36a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f00b400e990 tx=0x7f00b400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00b27fc640 1 -- 172.21.15.18:0/218230371 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4004d20 con 0x7f00bc105020 2024-02-17T12:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00b27fc640 1 -- 172.21.15.18:0/218230371 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00b4004e80 con 0x7f00bc105020 2024-02-17T12:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c33d7640 1 -- 172.21.15.18:0/218230371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00bc1a3f80 con 0x7f00bc105020 2024-02-17T12:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00b27fc640 1 -- 172.21.15.18:0/218230371 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b40116d0 con 0x7f00bc105020 2024-02-17T12:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.353+0000 7f00c33d7640 1 -- 172.21.15.18:0/218230371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc1a4410 con 0x7f00bc105020 2024-02-17T12:54:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.356+0000 7f00c5fad740 1 Processor -- start 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.356+0000 7f00b27fc640 1 -- 172.21.15.18:0/218230371 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f00b40052d0 con 0x7f00bc105020 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.356+0000 7f00c5fad740 1 -- start start 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.356+0000 7f00c5fad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f50820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.356+0000 7f00c5fad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f50d60 0x55c5f1f511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.356+0000 7f00c5fad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f60ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.356+0000 7f00c5fad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1ebf9f0 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.356+0000 7f00c5fad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1ebcf20 con 0x55c5f1f50d60 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.356+0000 7f00c5fad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1eb8700 con 0x55c5f1f51ad0 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.356+0000 7f00c0972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f60ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.356+0000 7f00c0972640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f50820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00c0972640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f50820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:59176/0 (socket says 172.21.15.18:59176) 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00c0972640 1 -- 172.21.15.18:0/964598084 learned_addr learned my addr 172.21.15.18:0/964598084 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00b27fc640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f00a80a22d0 0x7f00a80a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00c1173640 1 --2- 172.21.15.18:0/964598084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f50d60 0x55c5f1f511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00c1974640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f00a80a22d0 0x7f00a80a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00b27fc640 1 -- 172.21.15.18:0/218230371 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f00b40ce270 con 0x7f00bc105020 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00c0972640 1 -- 172.21.15.18:0/964598084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f51ad0 msgr2=0x55c5f1f50820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00c0972640 1 --2- 172.21.15.18:0/964598084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f50820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00c0972640 1 -- 172.21.15.18:0/964598084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f50d60 msgr2=0x55c5f1f511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00c0972640 1 --2- 172.21.15.18:0/964598084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f50d60 0x55c5f1f511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00c0972640 1 -- 172.21.15.18:0/964598084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5f1e4e7b0 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00c0972640 1 --2- 172.21.15.18:0/964598084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f60ae0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f00ac02c170 tx=0x7f00ac02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0da0fcab520b7a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00c1974640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f00a80a22d0 0x7f00a80a4790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f00b40a2f20 tx=0x7f00b4059040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00a77fe640 1 -- 172.21.15.18:0/964598084 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac004970 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00a77fe640 1 -- 172.21.15.18:0/964598084 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00ac02cbb0 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.357+0000 7f00a77fe640 1 -- 172.21.15.18:0/964598084 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0055b0 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.358+0000 7f00c5fad740 1 -- 172.21.15.18:0/964598084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 msgr2=0x55c5f1f60ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.358+0000 7f00c5fad740 1 --2- 172.21.15.18:0/964598084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f60ae0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f00ac02c170 tx=0x7f00ac02bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.358+0000 7f00c5fad740 1 -- 172.21.15.18:0/964598084 shutdown_connections 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.358+0000 7f00c5fad740 1 --2- 172.21.15.18:0/964598084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f60ae0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.358+0000 7f00c5fad740 1 --2- 172.21.15.18:0/964598084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f50d60 0x55c5f1f511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.358+0000 7f00c5fad740 1 --2- 172.21.15.18:0/964598084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f50820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.358+0000 7f00c5fad740 1 -- 172.21.15.18:0/964598084 >> 172.21.15.18:0/964598084 conn(0x55c5f1f18550 msgr2=0x55c5f1f4ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.358+0000 7f00c5fad740 1 -- 172.21.15.18:0/964598084 shutdown_connections 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.358+0000 7f00c5fad740 1 -- 172.21.15.18:0/964598084 wait complete. 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.358+0000 7f00c5fad740 1 -- start start 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c5fad740 1 Processor -- start 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c5fad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f50d60 0x55c5f1f5b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c5fad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f5bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c5fad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f5c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c5fad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1e5dbf0 con 0x55c5f1f51ad0 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c5fad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1b8c250 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c5fad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1e3e200 con 0x55c5f1f50d60 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c5fad740 10 client.0 _ll_register_callbacks cb 0x55c5f1eacd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c1173640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f5bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c1173640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f5bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42682/0 (socket says 172.21.15.18:42682) 2024-02-17T12:54:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c1173640 1 -- 172.21.15.18:0/110442164 learned_addr learned my addr 172.21.15.18:0/110442164 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c0972640 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f5c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c0972640 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f50d60 0x55c5f1f5b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c1173640 1 -- 172.21.15.18:0/110442164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f50d60 msgr2=0x55c5f1f5b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c1173640 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f50d60 0x55c5f1f5b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c1173640 1 -- 172.21.15.18:0/110442164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f5e6f0 msgr2=0x55c5f1f5c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.359+0000 7f00c1173640 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f5c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00c1173640 1 -- 172.21.15.18:0/110442164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc10d860 con 0x55c5f1f51ad0 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00c0972640 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f50d60 0x55c5f1f5b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00c1173640 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f5bc50 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f00b800bf40 tx=0x7f00b80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bef3761fc86c20 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00a4ff9640 1 -- 172.21.15.18:0/110442164 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8016070 con 0x55c5f1f51ad0 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00c5fad740 1 -- 172.21.15.18:0/110442164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5f1bb8600 con 0x55c5f1f51ad0 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00a4ff9640 1 -- 172.21.15.18:0/110442164 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f00b8004020 con 0x55c5f1f51ad0 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00a4ff9640 1 -- 172.21.15.18:0/110442164 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00b8004220 con 0x55c5f1f51ad0 2024-02-17T12:54:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00c5fad740 10 client.17199 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00c5fad740 1 -- 172.21.15.18:0/110442164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c5f1eb6d80 con 0x55c5f1f51ad0 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00a4ff9640 1 -- 172.21.15.18:0/110442164 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c5f1bb8600 con 0x55c5f1f51ad0 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00a4ff9640 1 -- 172.21.15.18:0/110442164 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f00b8004c10 con 0x55c5f1f51ad0 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00c5fad740 10 client.17199 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00c5fad740 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.360+0000 7f00c5fad740 20 client.17199 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.361+0000 7f00c5fad740 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c5f1f6eba0 0x55c5f1f70fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.361+0000 7f00c5fad740 4 client.17199 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.361+0000 7f00c5fad740 1 -- 172.21.15.18:0/110442164 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5f1c091b0 con 0x55c5f1f6eba0 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.361+0000 7f00c0972640 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c5f1f6eba0 0x55c5f1f70fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.361+0000 7f00a4ff9640 10 client.17199 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.361+0000 7f00c0972640 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c5f1f6eba0 0x55c5f1f70fc0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ec8797abc40b28 server_cookie=54e03f51c93aa15d in_seq=0 out_seq=0 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00a4ff9640 1 -- 172.21.15.18:0/110442164 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5f1c091b0 con 0x55c5f1f6eba0 2024-02-17T12:54:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00a4ff9640 10 client.17199 handle_command_reply: tid=0 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00c5fad740 1 client.17199 shutdown 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00c5fad740 1 -- 172.21.15.18:0/110442164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f51ad0 msgr2=0x55c5f1f5bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00c5fad740 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f5bc50 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f00b800bf40 tx=0x7f00b80118d0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00c5fad740 1 -- 172.21.15.18:0/110442164 shutdown_connections 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00c5fad740 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c5f1f6eba0 0x55c5f1f70fc0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00c5fad740 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f5c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00c5fad740 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f5bc50 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00a4ff9640 0 client.17199 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00c5fad740 1 --2- 172.21.15.18:0/110442164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f50d60 0x55c5f1f5b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00c5fad740 1 -- 172.21.15.18:0/110442164 >> 172.21.15.18:0/110442164 conn(0x55c5f1f18550 msgr2=0x55c5f1f4d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00c5fad740 1 -- 172.21.15.18:0/110442164 shutdown_connections 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00c5fad740 1 -- 172.21.15.18:0/110442164 wait complete. 2024-02-17T12:54:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.362+0000 7f00c5fad740 20 client.17199 trim_cache size 0 max 16384 2024-02-17T12:54:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.375+0000 7f00c5fad740 1 Processor -- start 2024-02-17T12:54:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.375+0000 7f00c5fad740 1 -- start start 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c5fad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f50d60 0x55c5f1f59fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c5fad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f5a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c5fad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f5bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c5fad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1f6b560 con 0x55c5f1f50d60 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c5fad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1f5e170 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c5fad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1f5aa60 con 0x55c5f1f51ad0 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c0972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f50d60 0x55c5f1f59fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c0972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f50d60 0x55c5f1f59fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42690/0 (socket says 172.21.15.18:42690) 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c0972640 1 -- 172.21.15.18:0/863129644 learned_addr learned my addr 172.21.15.18:0/863129644 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c1173640 1 --2- 172.21.15.18:0/863129644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f5a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c0972640 1 --2- 172.21.15.18:0/863129644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f5bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c0972640 1 -- 172.21.15.18:0/863129644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f51ad0 msgr2=0x55c5f1f5a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c0972640 1 --2- 172.21.15.18:0/863129644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f5a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c0972640 1 -- 172.21.15.18:0/863129644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f5e6f0 msgr2=0x55c5f1f5bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c0972640 1 --2- 172.21.15.18:0/863129644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f5bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c0972640 1 -- 172.21.15.18:0/863129644 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5f1e537b0 con 0x55c5f1f50d60 2024-02-17T12:54:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00c0972640 1 --2- 172.21.15.18:0/863129644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f50d60 0x55c5f1f59fe0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f00ac02ce10 tx=0x7f00ac005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea8f95a74731d6a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00a6ffd640 1 -- 172.21.15.18:0/863129644 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac05a070 con 0x55c5f1f50d60 2024-02-17T12:54:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.376+0000 7f00a6ffd640 1 -- 172.21.15.18:0/863129644 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00ac02c5d0 con 0x55c5f1f50d60 2024-02-17T12:54:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.377+0000 7f00a6ffd640 1 -- 172.21.15.18:0/863129644 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac005580 con 0x55c5f1f50d60 2024-02-17T12:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.377+0000 7f00c5fad740 1 -- 172.21.15.18:0/863129644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f50d60 msgr2=0x55c5f1f59fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.377+0000 7f00c5fad740 1 --2- 172.21.15.18:0/863129644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f50d60 0x55c5f1f59fe0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f00ac02ce10 tx=0x7f00ac005150 comp rx=0 tx=0).stop 2024-02-17T12:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.377+0000 7f00c5fad740 1 -- 172.21.15.18:0/863129644 shutdown_connections 2024-02-17T12:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.377+0000 7f00c5fad740 1 --2- 172.21.15.18:0/863129644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f5bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.377+0000 7f00c5fad740 1 --2- 172.21.15.18:0/863129644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f5a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.377+0000 7f00c5fad740 1 --2- 172.21.15.18:0/863129644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f50d60 0x55c5f1f59fe0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.377+0000 7f00c5fad740 1 -- 172.21.15.18:0/863129644 >> 172.21.15.18:0/863129644 conn(0x55c5f1f18550 msgr2=0x55c5f1f56a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.377+0000 7f00c5fad740 1 -- 172.21.15.18:0/863129644 shutdown_connections 2024-02-17T12:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.377+0000 7f00c5fad740 1 -- 172.21.15.18:0/863129644 wait complete. 2024-02-17T12:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c5fad740 1 -- start start 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c5fad740 1 Processor -- start 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c5fad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f50d60 0x55c5f1f651e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c5fad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f65720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c5fad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f63850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c5fad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1f63e10 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c5fad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1f63f80 con 0x55c5f1f51ad0 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c5fad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1f64140 con 0x55c5f1f50d60 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c5fad740 10 client.0 _ll_register_callbacks cb 0x7f00ac02ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c0972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f63850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c0972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f63850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42694/0 (socket says 172.21.15.18:42694) 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c0972640 1 -- 172.21.15.18:0/1992510792 learned_addr learned my addr 172.21.15.18:0/1992510792 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c1173640 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f65720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c0972640 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f50d60 0x55c5f1f651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c0972640 1 -- 172.21.15.18:0/1992510792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f50d60 msgr2=0x55c5f1f651e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c0972640 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f50d60 0x55c5f1f651e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c0972640 1 -- 172.21.15.18:0/1992510792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f51ad0 msgr2=0x55c5f1f65720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c0972640 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f65720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c0972640 1 -- 172.21.15.18:0/1992510792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5f1e4e7b0 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.378+0000 7f00c1173640 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f65720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00c0972640 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f63850 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f00ac054b10 tx=0x7f00ac056040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c4e63f82bfd378 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00a4ff9640 1 -- 172.21.15.18:0/1992510792 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac05a070 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00a4ff9640 1 -- 172.21.15.18:0/1992510792 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f00ac000e40 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00c5fad740 1 -- 172.21.15.18:0/1992510792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00ac02cbb0 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00a4ff9640 1 -- 172.21.15.18:0/1992510792 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00ac005320 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00c5fad740 10 client.17205 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00c5fad740 1 -- 172.21.15.18:0/1992510792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c5f1e79540 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00a4ff9640 1 -- 172.21.15.18:0/1992510792 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00ac02cbb0 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00a4ff9640 1 -- 172.21.15.18:0/1992510792 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f00ac03c050 con 0x55c5f1f5e6f0 2024-02-17T12:54:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00c5fad740 10 client.17205 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00c5fad740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00c5fad740 20 client.17205 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00c5fad740 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c5f1fc5ea0 0x55c5f1fc8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00c5fad740 4 client.17205 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:54:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.379+0000 7f00c5fad740 1 -- 172.21.15.18:0/1992510792 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c5f1bf5550 con 0x55c5f1fc5ea0 2024-02-17T12:54:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.380+0000 7f00c1173640 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c5f1fc5ea0 0x55c5f1fc8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.380+0000 7f00c1173640 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c5f1fc5ea0 0x55c5f1fc8290 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1106b4705a460d23 server_cookie=e1d018e442e797ab in_seq=0 out_seq=0 2024-02-17T12:54:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.380+0000 7f00a4ff9640 10 client.17205 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:36.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.384+0000 7f4babed4640 1 -- 172.21.15.18:0/2817297590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba4102d10 msgr2=0x7f4ba410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.384+0000 7f4babed4640 1 --2- 172.21.15.18:0/2817297590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba4102d10 0x7f4ba410b1f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009900 tx=0x7f4ba002bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:36.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.384+0000 7f4babed4640 1 -- 172.21.15.18:0/2817297590 shutdown_connections 2024-02-17T12:54:36.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.384+0000 7f4babed4640 1 --2- 172.21.15.18:0/2817297590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ba410b7c0 0x7f4ba410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.384+0000 7f4babed4640 1 --2- 172.21.15.18:0/2817297590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba4102d10 0x7f4ba410b1f0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.384+0000 7f4babed4640 1 --2- 172.21.15.18:0/2817297590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ba41023f0 0x7f4ba41027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.384+0000 7f4babed4640 1 -- 172.21.15.18:0/2817297590 >> 172.21.15.18:0/2817297590 conn(0x7f4ba4077180 msgr2=0x7f4ba4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.384+0000 7f4babed4640 1 -- 172.21.15.18:0/2817297590 shutdown_connections 2024-02-17T12:54:36.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.384+0000 7f4babed4640 1 -- 172.21.15.18:0/2817297590 wait complete. 2024-02-17T12:54:36.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.384+0000 7f4babed4640 1 Processor -- start 2024-02-17T12:54:36.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4babed4640 1 -- start start 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4babed4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ba41023f0 0x7f4ba419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4babed4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba4102d10 0x7f4ba419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4babed4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ba410b7c0 0x7f4ba4197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4babed4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4197e10 con 0x7f4ba4102d10 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4babed4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4197f80 con 0x7f4ba41023f0 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4babed4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4198140 con 0x7f4ba410b7c0 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4ba946f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba4102d10 0x7f4ba419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4ba946f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba4102d10 0x7f4ba419db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42712/0 (socket says 172.21.15.18:42712) 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4ba946f640 1 -- 172.21.15.18:0/1358854877 learned_addr learned my addr 172.21.15.18:0/1358854877 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4ba9c70640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ba41023f0 0x7f4ba419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4baa471640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ba410b7c0 0x7f4ba4197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4ba946f640 1 -- 172.21.15.18:0/1358854877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ba410b7c0 msgr2=0x7f4ba4197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4ba946f640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ba410b7c0 0x7f4ba4197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4ba946f640 1 -- 172.21.15.18:0/1358854877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ba41023f0 msgr2=0x7f4ba419d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4ba946f640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ba41023f0 0x7f4ba419d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4ba946f640 1 -- 172.21.15.18:0/1358854877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba0009560 con 0x7f4ba4102d10 2024-02-17T12:54:36.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.385+0000 7f4ba946f640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba4102d10 0x7f4ba419db00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0005940 tx=0x7f4ba002c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.386+0000 7f4b9affd640 1 -- 172.21.15.18:0/1358854877 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00043d0 con 0x7f4ba4102d10 2024-02-17T12:54:36.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.386+0000 7f4babed4640 1 -- 172.21.15.18:0/1358854877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba41983c0 con 0x7f4ba4102d10 2024-02-17T12:54:36.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.386+0000 7f4b9affd640 1 -- 172.21.15.18:0/1358854877 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ba00052d0 con 0x7f4ba4102d10 2024-02-17T12:54:36.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.386+0000 7f4b9affd640 1 -- 172.21.15.18:0/1358854877 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0003d30 con 0x7f4ba4102d10 2024-02-17T12:54:36.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.386+0000 7f4babed4640 1 -- 172.21.15.18:0/1358854877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba40fff70 con 0x7f4ba4102d10 2024-02-17T12:54:36.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.387+0000 7f4babed4640 1 -- 172.21.15.18:0/1358854877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba4068b20 con 0x7f4ba4102d10 2024-02-17T12:54:36.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.389+0000 7f4b9affd640 1 -- 172.21.15.18:0/1358854877 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4ba0005440 con 0x7f4ba4102d10 2024-02-17T12:54:36.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.389+0000 7f4b9affd640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f4b900a6640 0x7f4b900a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.389+0000 7f4b9affd640 1 -- 172.21.15.18:0/1358854877 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4ba00f0670 con 0x7f4ba4102d10 2024-02-17T12:54:36.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.389+0000 7f4ba9c70640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f4b900a6640 0x7f4b900a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.390+0000 7f4ba9c70640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f4b900a6640 0x7f4b900a8b00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b94009bd0 tx=0x7f4b94005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.393+0000 7f4b9affd640 1 -- 172.21.15.18:0/1358854877 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ba00b9460 con 0x7f4ba4102d10 2024-02-17T12:54:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00a4ff9640 1 -- 172.21.15.18:0/1992510792 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c5f1bf5550 con 0x55c5f1fc5ea0 2024-02-17T12:54:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00a4ff9640 10 client.17205 handle_command_reply: tid=0 2024-02-17T12:54:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00c5fad740 1 client.17205 shutdown 2024-02-17T12:54:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00c5fad740 1 -- 172.21.15.18:0/1992510792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 msgr2=0x55c5f1f63850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00c5fad740 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f63850 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f00ac054b10 tx=0x7f00ac056040 comp rx=0 tx=0).stop 2024-02-17T12:54:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00c5fad740 1 -- 172.21.15.18:0/1992510792 shutdown_connections 2024-02-17T12:54:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00c5fad740 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c5f1fc5ea0 0x55c5f1fc8290 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00c5fad740 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5f1f5e6f0 0x55c5f1f63850 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00c5fad740 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5f1f51ad0 0x55c5f1f65720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00c5fad740 1 --2- 172.21.15.18:0/1992510792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5f1f50d60 0x55c5f1f651e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00a4ff9640 0 client.17205 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00c5fad740 1 -- 172.21.15.18:0/1992510792 >> 172.21.15.18:0/1992510792 conn(0x55c5f1f18550 msgr2=0x55c5f1f59f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00c5fad740 1 -- 172.21.15.18:0/1992510792 shutdown_connections 2024-02-17T12:54:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00c5fad740 1 -- 172.21.15.18:0/1992510792 wait complete. 2024-02-17T12:54:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.420+0000 7f00c5fad740 20 client.17205 trim_cache size 0 max 16384 2024-02-17T12:54:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:54:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:54:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:54:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:54:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.421+0000 7f00c33d7640 1 -- 172.21.15.18:0/218230371 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f00a80a22d0 msgr2=0x7f00a80a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.421+0000 7f00c33d7640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f00a80a22d0 0x7f00a80a4790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f00b40a2f20 tx=0x7f00b4059040 comp rx=0 tx=0).stop 2024-02-17T12:54:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.421+0000 7f00c33d7640 1 -- 172.21.15.18:0/218230371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc105020 msgr2=0x7f00bc1a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.421+0000 7f00c33d7640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc105020 0x7f00bc1a36a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f00b400e990 tx=0x7f00b400ee50 comp rx=0 tx=0).stop 2024-02-17T12:54:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.421+0000 7f00c33d7640 1 -- 172.21.15.18:0/218230371 shutdown_connections 2024-02-17T12:54:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.421+0000 7f00c33d7640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f00a80a22d0 0x7f00a80a4790 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.421+0000 7f00c33d7640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc105020 0x7f00bc1a36a0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.421+0000 7f00c33d7640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00bc100c90 0x7f00bc198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.421+0000 7f00c33d7640 1 --2- 172.21.15.18:0/218230371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00bc1002d0 0x7f00bc198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.421+0000 7f00c33d7640 1 -- 172.21.15.18:0/218230371 >> 172.21.15.18:0/218230371 conn(0x7f00bc076ed0 msgr2=0x7f00bc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.421+0000 7f00c33d7640 1 -- 172.21.15.18:0/218230371 shutdown_connections 2024-02-17T12:54:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:36.421+0000 7f00c33d7640 1 -- 172.21.15.18:0/218230371 wait complete. 2024-02-17T12:54:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.617+0000 7f4babed4640 1 -- 172.21.15.18:0/1358854877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ba41985b0 con 0x7f4ba4102d10 2024-02-17T12:54:36.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.618+0000 7f4b9affd640 1 -- 172.21.15.18:0/1358854877 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+11398 (secure 0 0 0) 0x7f4ba00bcb20 con 0x7f4ba4102d10 2024-02-17T12:54:36.610 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:54:36.610 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:54:34.939120+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:54:36.610 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 34 2024-02-17T12:54:36.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.620+0000 7f4babed4640 1 -- 172.21.15.18:0/1358854877 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f4b900a6640 msgr2=0x7f4b900a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.620+0000 7f4babed4640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f4b900a6640 0x7f4b900a8b00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b94009bd0 tx=0x7f4b94005f90 comp rx=0 tx=0).stop 2024-02-17T12:54:36.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.620+0000 7f4babed4640 1 -- 172.21.15.18:0/1358854877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba4102d10 msgr2=0x7f4ba419db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.620+0000 7f4babed4640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba4102d10 0x7f4ba419db00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0005940 tx=0x7f4ba002c6e0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.620+0000 7f4babed4640 1 -- 172.21.15.18:0/1358854877 shutdown_connections 2024-02-17T12:54:36.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.620+0000 7f4babed4640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f4b900a6640 0x7f4b900a8b00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.620+0000 7f4babed4640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ba410b7c0 0x7f4ba4197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.620+0000 7f4babed4640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba4102d10 0x7f4ba419db00 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.620+0000 7f4babed4640 1 --2- 172.21.15.18:0/1358854877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ba41023f0 0x7f4ba419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.620+0000 7f4babed4640 1 -- 172.21.15.18:0/1358854877 >> 172.21.15.18:0/1358854877 conn(0x7f4ba4077180 msgr2=0x7f4ba40fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.620+0000 7f4babed4640 1 -- 172.21.15.18:0/1358854877 shutdown_connections 2024-02-17T12:54:36.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.620+0000 7f4babed4640 1 -- 172.21.15.18:0/1358854877 wait complete. 2024-02-17T12:54:36.622 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:54:36.622 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:54:36.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.775+0000 7f7035e23640 1 -- 172.21.15.18:0/3693401080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7030100c10 msgr2=0x7f7030104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.775+0000 7f7035e23640 1 --2- 172.21.15.18:0/3693401080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7030100c10 0x7f7030104c50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f7024009930 tx=0x7f702402bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:36.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.775+0000 7f7035e23640 1 -- 172.21.15.18:0/3693401080 shutdown_connections 2024-02-17T12:54:36.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.775+0000 7f7035e23640 1 --2- 172.21.15.18:0/3693401080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7030105330 0x7f703010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.775+0000 7f7035e23640 1 --2- 172.21.15.18:0/3693401080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7030100c10 0x7f7030104c50 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.775+0000 7f7035e23640 1 --2- 172.21.15.18:0/3693401080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7030100260 0x7f7030100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.775+0000 7f7035e23640 1 -- 172.21.15.18:0/3693401080 >> 172.21.15.18:0/3693401080 conn(0x7f7030077180 msgr2=0x7f7030077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.775+0000 7f7035e23640 1 -- 172.21.15.18:0/3693401080 shutdown_connections 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.776+0000 7f7035e23640 1 -- 172.21.15.18:0/3693401080 wait complete. 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.776+0000 7f7035e23640 1 Processor -- start 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.776+0000 7f7035e23640 1 -- start start 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.776+0000 7f7035e23640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7030100260 0x7f703019d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.776+0000 7f7035e23640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7030105330 0x7f703019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.776+0000 7f7035e23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7030197650 0x7f7030197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.776+0000 7f7035e23640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7030198120 con 0x7f7030100260 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.776+0000 7f7035e23640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7030198290 con 0x7f7030197650 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.776+0000 7f7035e23640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7030198450 con 0x7f7030105330 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7030100260 0x7f703019d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7030100260 0x7f703019d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42738/0 (socket says 172.21.15.18:42738) 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702f7fe640 1 -- 172.21.15.18:0/56827587 learned_addr learned my addr 172.21.15.18:0/56827587 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702effd640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7030105330 0x7f703019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702ffff640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7030197650 0x7f7030197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702f7fe640 1 -- 172.21.15.18:0/56827587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7030105330 msgr2=0x7f703019dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702f7fe640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7030105330 0x7f703019dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702f7fe640 1 -- 172.21.15.18:0/56827587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7030197650 msgr2=0x7f7030197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702f7fe640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7030197650 0x7f7030197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702f7fe640 1 -- 172.21.15.18:0/56827587 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7024009560 con 0x7f7030100260 2024-02-17T12:54:36.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702effd640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7030105330 0x7f703019dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:36.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702ffff640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7030197650 0x7f7030197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:36.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702f7fe640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7030100260 0x7f703019d590 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f702000d770 tx=0x7f702000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702cff9640 1 -- 172.21.15.18:0/56827587 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020009c30 con 0x7f7030100260 2024-02-17T12:54:36.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702cff9640 1 -- 172.21.15.18:0/56827587 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f702000de80 con 0x7f7030100260 2024-02-17T12:54:36.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f7035e23640 1 -- 172.21.15.18:0/56827587 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70301a43f0 con 0x7f7030100260 2024-02-17T12:54:36.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f702cff9640 1 -- 172.21.15.18:0/56827587 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020004e80 con 0x7f7030100260 2024-02-17T12:54:36.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.777+0000 7f7035e23640 1 -- 172.21.15.18:0/56827587 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70301a4880 con 0x7f7030100260 2024-02-17T12:54:36.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.780+0000 7f702cff9640 1 -- 172.21.15.18:0/56827587 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7020005020 con 0x7f7030100260 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.780+0000 7f70389fc740 1 Processor -- start 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.780+0000 7f70389fc740 1 -- start start 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.780+0000 7f70389fc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53eef80 0x55fed53ef360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.780+0000 7f70389fc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 0x55fed53fc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.780+0000 7f70389fc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53fcb60 0x55fed53fef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.780+0000 7f70389fc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed53aa960 con 0x55fed53ef8a0 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.780+0000 7f70389fc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed535e310 con 0x55fed53eef80 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.780+0000 7f70389fc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed535b840 con 0x55fed53fcb60 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.780+0000 7f702f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 0x55fed53fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.780+0000 7f702f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 0x55fed53fc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42742/0 (socket says 172.21.15.18:42742) 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.780+0000 7f702f7fe640 1 -- 172.21.15.18:0/1144354360 learned_addr learned my addr 172.21.15.18:0/1144354360 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702cff9640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f70180a2230 0x7f70180a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702effd640 1 --2- 172.21.15.18:0/1144354360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53fcb60 0x55fed53fef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702ffff640 1 --2- 172.21.15.18:0/1144354360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53eef80 0x55fed53ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702ffff640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f70180a2230 0x7f70180a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702cff9640 1 -- 172.21.15.18:0/56827587 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f70200cdb50 con 0x7f7030100260 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702f7fe640 1 -- 172.21.15.18:0/1144354360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53fcb60 msgr2=0x55fed53fef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702f7fe640 1 --2- 172.21.15.18:0/1144354360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53fcb60 0x55fed53fef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702f7fe640 1 -- 172.21.15.18:0/1144354360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53eef80 msgr2=0x55fed53ef360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702f7fe640 1 --2- 172.21.15.18:0/1144354360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53eef80 0x55fed53ef360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702f7fe640 1 -- 172.21.15.18:0/1144354360 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fed5319060 con 0x55fed53ef8a0 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702ffff640 1 --2- 172.21.15.18:0/1144354360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53eef80 0x55fed53ef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702f7fe640 1 --2- 172.21.15.18:0/1144354360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 0x55fed53fc620 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f702009a290 tx=0x7f7020004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678a562f3818a909 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f702ffff640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f70180a2230 0x7f70180a46f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7028005c40 tx=0x7f7028009200 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f7015ffb640 1 -- 172.21.15.18:0/1144354360 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702009a590 con 0x55fed53ef8a0 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f7015ffb640 1 -- 172.21.15.18:0/1144354360 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f702009e6e0 con 0x55fed53ef8a0 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.781+0000 7f7015ffb640 1 -- 172.21.15.18:0/1144354360 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70200cb570 con 0x55fed53ef8a0 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.782+0000 7f70389fc740 1 -- 172.21.15.18:0/1144354360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 msgr2=0x55fed53fc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.782+0000 7f70389fc740 1 --2- 172.21.15.18:0/1144354360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 0x55fed53fc620 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f702009a290 tx=0x7f7020004900 comp rx=0 tx=0).stop 2024-02-17T12:54:36.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.782+0000 7f70389fc740 1 -- 172.21.15.18:0/1144354360 shutdown_connections 2024-02-17T12:54:36.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.782+0000 7f70389fc740 1 --2- 172.21.15.18:0/1144354360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53fcb60 0x55fed53fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.782+0000 7f70389fc740 1 --2- 172.21.15.18:0/1144354360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 0x55fed53fc620 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.782+0000 7f70389fc740 1 --2- 172.21.15.18:0/1144354360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53eef80 0x55fed53ef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.782+0000 7f70389fc740 1 -- 172.21.15.18:0/1144354360 >> 172.21.15.18:0/1144354360 conn(0x55fed53e9980 msgr2=0x55fed53e9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.782+0000 7f70389fc740 1 -- 172.21.15.18:0/1144354360 shutdown_connections 2024-02-17T12:54:36.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.782+0000 7f70389fc740 1 -- 172.21.15.18:0/1144354360 wait complete. 2024-02-17T12:54:36.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:36 smithi018 ceph-mon[32625]: pgmap v144: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.7 MiB/s wr, 141 op/s 2024-02-17T12:54:36.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:36 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:54:36.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.782+0000 7f70389fc740 1 -- start start 2024-02-17T12:54:36.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f70389fc740 1 Processor -- start 2024-02-17T12:54:36.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f70389fc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53eef80 0x55fed53f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f70389fc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53ef8a0 0x55fed53fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f70389fc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53fcb60 0x55fed53fa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f70389fc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed5357020 con 0x55fed53fcb60 2024-02-17T12:54:36.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f70389fc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed502a430 con 0x55fed53ef8a0 2024-02-17T12:54:36.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f70389fc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed4fe6e00 con 0x55fed53eef80 2024-02-17T12:54:36.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f70389fc740 10 client.0 _ll_register_callbacks cb 0x55fed52fecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:36.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f702ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53fcb60 0x55fed53fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f702ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53fcb60 0x55fed53fa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42744/0 (socket says 172.21.15.18:42744) 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f702ffff640 1 -- 172.21.15.18:0/820081914 learned_addr learned my addr 172.21.15.18:0/820081914 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f702f7fe640 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53eef80 0x55fed53f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f702effd640 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53ef8a0 0x55fed53fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f702ffff640 1 -- 172.21.15.18:0/820081914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53eef80 msgr2=0x55fed53f9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f702ffff640 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53eef80 0x55fed53f9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f702ffff640 1 -- 172.21.15.18:0/820081914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53ef8a0 msgr2=0x55fed53fa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f702ffff640 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53ef8a0 0x55fed53fa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f702ffff640 1 -- 172.21.15.18:0/820081914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70301a4880 con 0x55fed53fcb60 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f702ffff640 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53fcb60 0x55fed53fa5e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7028011920 tx=0x7f7028004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6cbad01909d561 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.783+0000 7f700effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f700effd640 1 -- 172.21.15.18:0/820081914 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702800fb70 con 0x55fed53fcb60 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f700effd640 1 -- 172.21.15.18:0/820081914 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f702801b040 con 0x55fed53fcb60 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f70389fc740 1 -- 172.21.15.18:0/820081914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fed50567e0 con 0x55fed53fcb60 2024-02-17T12:54:36.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f700effd640 1 -- 172.21.15.18:0/820081914 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f702800fcd0 con 0x55fed53fcb60 2024-02-17T12:54:36.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f70389fc740 10 client.17223 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:36.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f70389fc740 1 -- 172.21.15.18:0/820081914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55fed53556a0 con 0x55fed53fcb60 2024-02-17T12:54:36.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f700effd640 1 -- 172.21.15.18:0/820081914 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fed50567e0 con 0x55fed53fcb60 2024-02-17T12:54:36.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f700effd640 1 -- 172.21.15.18:0/820081914 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7028028440 con 0x55fed53fcb60 2024-02-17T12:54:36.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f70389fc740 10 client.17223 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:36.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f70389fc740 10 client.17223 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:54:36.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f70389fc740 20 client.17223 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:36.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f70389fc740 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55fed540cec0 0x55fed540f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f70389fc740 4 client.17223 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f70389fc740 1 -- 172.21.15.18:0/820081914 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fed5093720 con 0x55fed540cec0 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.784+0000 7f702f7fe640 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55fed540cec0 0x55fed540f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.785+0000 7f700effd640 10 client.17223 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.785+0000 7f702f7fe640 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55fed540cec0 0x55fed540f2e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=774e4bf90d842399 server_cookie=c8ad0b598ed772b4 in_seq=0 out_seq=0 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.785+0000 7f700effd640 1 -- 172.21.15.18:0/820081914 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fed5093720 con 0x55fed540cec0 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.785+0000 7f700effd640 10 client.17223 handle_command_reply: tid=0 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.785+0000 7f70389fc740 1 client.17223 shutdown 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.786+0000 7f70389fc740 1 -- 172.21.15.18:0/820081914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53fcb60 msgr2=0x55fed53fa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.786+0000 7f70389fc740 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53fcb60 0x55fed53fa5e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7028011920 tx=0x7f7028004a40 comp rx=0 tx=0).stop 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.786+0000 7f70389fc740 1 -- 172.21.15.18:0/820081914 shutdown_connections 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.786+0000 7f70389fc740 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53fcb60 0x55fed53fa5e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.786+0000 7f70389fc740 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55fed540cec0 0x55fed540f2e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.786+0000 7f70389fc740 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53ef8a0 0x55fed53fa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.786+0000 7f70389fc740 1 --2- 172.21.15.18:0/820081914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53eef80 0x55fed53f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.786+0000 7f700effd640 0 client.17223 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:54:36.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.786+0000 7f70389fc740 1 -- 172.21.15.18:0/820081914 >> 172.21.15.18:0/820081914 conn(0x55fed53e9980 msgr2=0x55fed53eb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.786+0000 7f70389fc740 1 -- 172.21.15.18:0/820081914 shutdown_connections 2024-02-17T12:54:36.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.786+0000 7f70389fc740 1 -- 172.21.15.18:0/820081914 wait complete. 2024-02-17T12:54:36.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.786+0000 7f70389fc740 20 client.17223 trim_cache size 0 max 16384 2024-02-17T12:54:36.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.797+0000 7f70389fc740 1 Processor -- start 2024-02-17T12:54:36.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.797+0000 7f70389fc740 1 -- start start 2024-02-17T12:54:36.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f70389fc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53eef80 0x55fed53fa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f70389fc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 0x55fed53e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f70389fc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53fcb60 0x55fed5403960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f70389fc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed52dc130 con 0x55fed53ef8a0 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f70389fc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed54144f0 con 0x55fed53fcb60 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f70389fc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed53ee340 con 0x55fed53eef80 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f702effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 0x55fed53e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f702effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 0x55fed53e8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42758/0 (socket says 172.21.15.18:42758) 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f702effd640 1 -- 172.21.15.18:0/2359406372 learned_addr learned my addr 172.21.15.18:0/2359406372 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f702f7fe640 1 --2- 172.21.15.18:0/2359406372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53eef80 0x55fed53fa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f702ffff640 1 --2- 172.21.15.18:0/2359406372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53fcb60 0x55fed5403960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f702effd640 1 -- 172.21.15.18:0/2359406372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53eef80 msgr2=0x55fed53fa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f702effd640 1 --2- 172.21.15.18:0/2359406372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53eef80 0x55fed53fa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f702effd640 1 -- 172.21.15.18:0/2359406372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53fcb60 msgr2=0x55fed5403960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f702effd640 1 --2- 172.21.15.18:0/2359406372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53fcb60 0x55fed5403960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f702effd640 1 -- 172.21.15.18:0/2359406372 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fed52f1c20 con 0x55fed53ef8a0 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f702effd640 1 --2- 172.21.15.18:0/2359406372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 0x55fed53e8a00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f702402c780 tx=0x7f7024004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb4f6f884b3a0d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f70157fa640 1 -- 172.21.15.18:0/2359406372 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024004a30 con 0x55fed53ef8a0 2024-02-17T12:54:36.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f70157fa640 1 -- 172.21.15.18:0/2359406372 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7024004030 con 0x55fed53ef8a0 2024-02-17T12:54:36.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.798+0000 7f70157fa640 1 -- 172.21.15.18:0/2359406372 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70240052f0 con 0x55fed53ef8a0 2024-02-17T12:54:36.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.799+0000 7f70389fc740 1 -- 172.21.15.18:0/2359406372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 msgr2=0x55fed53e8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.799+0000 7f70389fc740 1 --2- 172.21.15.18:0/2359406372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 0x55fed53e8a00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f702402c780 tx=0x7f7024004800 comp rx=0 tx=0).stop 2024-02-17T12:54:36.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.799+0000 7f70389fc740 1 -- 172.21.15.18:0/2359406372 shutdown_connections 2024-02-17T12:54:36.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.799+0000 7f70389fc740 1 --2- 172.21.15.18:0/2359406372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53fcb60 0x55fed5403960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.799+0000 7f70389fc740 1 --2- 172.21.15.18:0/2359406372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53ef8a0 0x55fed53e8a00 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.799+0000 7f70389fc740 1 --2- 172.21.15.18:0/2359406372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53eef80 0x55fed53fa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.799+0000 7f70389fc740 1 -- 172.21.15.18:0/2359406372 >> 172.21.15.18:0/2359406372 conn(0x55fed53e9980 msgr2=0x55fed53e82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.799+0000 7f70389fc740 1 -- 172.21.15.18:0/2359406372 shutdown_connections 2024-02-17T12:54:36.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.799+0000 7f70389fc740 1 -- 172.21.15.18:0/2359406372 wait complete. 2024-02-17T12:54:36.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.799+0000 7f70389fc740 1 -- start start 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.799+0000 7f70389fc740 1 Processor -- start 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f70389fc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53eef80 0x55fed545d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f70389fc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53ef8a0 0x55fed545dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f70389fc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53fcb60 0x55fed5402de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f70389fc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed54033a0 con 0x55fed53eef80 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f70389fc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed5403510 con 0x55fed53ef8a0 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f70389fc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed54036d0 con 0x55fed53fcb60 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f70389fc740 10 client.0 _ll_register_callbacks cb 0x7f70200968b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53eef80 0x55fed545d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53eef80 0x55fed545d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42772/0 (socket says 172.21.15.18:42772) 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702f7fe640 1 -- 172.21.15.18:0/843668547 learned_addr learned my addr 172.21.15.18:0/843668547 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702effd640 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53ef8a0 0x55fed545dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702ffff640 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53fcb60 0x55fed5402de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702f7fe640 1 -- 172.21.15.18:0/843668547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53fcb60 msgr2=0x55fed5402de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702f7fe640 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53fcb60 0x55fed5402de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702f7fe640 1 -- 172.21.15.18:0/843668547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53ef8a0 msgr2=0x55fed545dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702f7fe640 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53ef8a0 0x55fed545dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702f7fe640 1 -- 172.21.15.18:0/843668547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70300682e0 con 0x55fed53eef80 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702ffff640 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53fcb60 0x55fed5402de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702effd640 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53ef8a0 0x55fed545dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f702f7fe640 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53eef80 0x55fed545d8a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f70200cb3a0 tx=0x7f702000d6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e40bac65e7bf31 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f700effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f700effd640 1 -- 172.21.15.18:0/843668547 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020009d90 con 0x55fed53eef80 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f700effd640 1 -- 172.21.15.18:0/843668547 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7020098890 con 0x55fed53eef80 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f70389fc740 1 -- 172.21.15.18:0/843668547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f702009e6e0 con 0x55fed53eef80 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f700effd640 1 -- 172.21.15.18:0/843668547 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70200cb570 con 0x55fed53eef80 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f70389fc740 10 client.17229 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.800+0000 7f70389fc740 1 -- 172.21.15.18:0/843668547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55fed53178c0 con 0x55fed53eef80 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.801+0000 7f700effd640 1 -- 172.21.15.18:0/843668547 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f702009e6e0 con 0x55fed53eef80 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.801+0000 7f700effd640 1 -- 172.21.15.18:0/843668547 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f70200962e0 con 0x55fed53eef80 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.801+0000 7f70389fc740 10 client.17229 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:36.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.801+0000 7f70389fc740 10 client.17229 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.801+0000 7f70389fc740 20 client.17229 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.801+0000 7f70389fc740 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55fed5465460 0x55fed5467880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.801+0000 7f70389fc740 4 client.17229 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.801+0000 7f70389fc740 1 -- 172.21.15.18:0/843668547 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fed50e2890 con 0x55fed5465460 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.801+0000 7f702effd640 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55fed5465460 0x55fed5467880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.801+0000 7f702effd640 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55fed5465460 0x55fed5467880 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=960d29489bf06dea server_cookie=86816001158898cd in_seq=0 out_seq=0 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.801+0000 7f700effd640 10 client.17229 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f700effd640 1 -- 172.21.15.18:0/843668547 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fed50e2890 con 0x55fed5465460 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f700effd640 10 client.17229 handle_command_reply: tid=0 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f70389fc740 1 client.17229 shutdown 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f70389fc740 1 -- 172.21.15.18:0/843668547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53eef80 msgr2=0x55fed545d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f70389fc740 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53eef80 0x55fed545d8a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f70200cb3a0 tx=0x7f702000d6c0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f70389fc740 1 -- 172.21.15.18:0/843668547 shutdown_connections 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f70389fc740 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fed53fcb60 0x55fed5402de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f70389fc740 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55fed5465460 0x55fed5467880 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f70389fc740 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fed53ef8a0 0x55fed545dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f70389fc740 1 --2- 172.21.15.18:0/843668547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fed53eef80 0x55fed545d8a0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f700effd640 0 client.17229 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:54:36.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f70389fc740 1 -- 172.21.15.18:0/843668547 >> 172.21.15.18:0/843668547 conn(0x55fed53e9980 msgr2=0x55fed53e8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f70389fc740 1 -- 172.21.15.18:0/843668547 shutdown_connections 2024-02-17T12:54:36.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f70389fc740 1 -- 172.21.15.18:0/843668547 wait complete. 2024-02-17T12:54:36.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f70389fc740 20 client.17229 trim_cache size 0 max 16384 2024-02-17T12:54:36.800 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:54:36.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f7035e23640 1 -- 172.21.15.18:0/56827587 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f70180a2230 msgr2=0x7f70180a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f7035e23640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f70180a2230 0x7f70180a46f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7028005c40 tx=0x7f7028009200 comp rx=0 tx=0).stop 2024-02-17T12:54:36.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f7035e23640 1 -- 172.21.15.18:0/56827587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7030100260 msgr2=0x7f703019d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f7035e23640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7030100260 0x7f703019d590 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f702000d770 tx=0x7f702000dc30 comp rx=0 tx=0).stop 2024-02-17T12:54:36.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f7035e23640 1 -- 172.21.15.18:0/56827587 shutdown_connections 2024-02-17T12:54:36.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f7035e23640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f70180a2230 0x7f70180a46f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f7035e23640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7030197650 0x7f7030197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f7035e23640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7030105330 0x7f703019dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f7035e23640 1 --2- 172.21.15.18:0/56827587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7030100260 0x7f703019d590 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.802+0000 7f7035e23640 1 -- 172.21.15.18:0/56827587 >> 172.21.15.18:0/56827587 conn(0x7f7030077180 msgr2=0x7f70300fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.803+0000 7f7035e23640 1 -- 172.21.15.18:0/56827587 shutdown_connections 2024-02-17T12:54:36.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.803+0000 7f7035e23640 1 -- 172.21.15.18:0/56827587 wait complete. 2024-02-17T12:54:36.804 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:54:36.804 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:54:36.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:36 smithi119 ceph-mon[36188]: pgmap v144: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.7 MiB/s wr, 141 op/s 2024-02-17T12:54:36.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:36 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:54:36.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.917+0000 7fabb8f01640 1 -- 172.21.15.18:0/4115500756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb4106570 msgr2=0x7fabb410e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.917+0000 7fabb8f01640 1 --2- 172.21.15.18:0/4115500756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb4106570 0x7fabb410e0f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fabac009cd0 tx=0x7fabac02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.917+0000 7fabb8f01640 1 -- 172.21.15.18:0/4115500756 shutdown_connections 2024-02-17T12:54:36.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.917+0000 7fabb8f01640 1 --2- 172.21.15.18:0/4115500756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb4106570 0x7fabb410e0f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.917+0000 7fabb8f01640 1 --2- 172.21.15.18:0/4115500756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb4100310 0x7fabb4105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.917+0000 7fabb8f01640 1 --2- 172.21.15.18:0/4115500756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb40ff960 0x7fabb40ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.917+0000 7fabb8f01640 1 -- 172.21.15.18:0/4115500756 >> 172.21.15.18:0/4115500756 conn(0x7fabb40766b0 msgr2=0x7fabb40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.917+0000 7fabb8f01640 1 -- 172.21.15.18:0/4115500756 shutdown_connections 2024-02-17T12:54:36.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.917+0000 7fabb8f01640 1 -- 172.21.15.18:0/4115500756 wait complete. 2024-02-17T12:54:36.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fabb8f01640 1 Processor -- start 2024-02-17T12:54:36.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fabb8f01640 1 -- start start 2024-02-17T12:54:36.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fabb8f01640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb40ff960 0x7fabb419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fabb8f01640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb4106570 0x7fabb419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fabb8f01640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb41998f0 0x7fabb4199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fabb8f01640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb419a2e0 con 0x7fabb4106570 2024-02-17T12:54:36.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fabb8f01640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb419a450 con 0x7fabb40ff960 2024-02-17T12:54:36.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fabb8f01640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb419a610 con 0x7fabb41998f0 2024-02-17T12:54:36.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fababfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb4106570 0x7fabb419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fababfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb4106570 0x7fabb419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42792/0 (socket says 172.21.15.18:42792) 2024-02-17T12:54:36.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fababfff640 1 -- 172.21.15.18:0/3669752208 learned_addr learned my addr 172.21.15.18:0/3669752208 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fabb259c640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb40ff960 0x7fabb419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.918+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb41998f0 0x7fabb4199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.919+0000 7fababfff640 1 -- 172.21.15.18:0/3669752208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb41998f0 msgr2=0x7fabb4199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.919+0000 7fababfff640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb41998f0 0x7fabb4199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.919+0000 7fababfff640 1 -- 172.21.15.18:0/3669752208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb40ff960 msgr2=0x7fabb419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.919+0000 7fababfff640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb40ff960 0x7fabb419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.919+0000 7fababfff640 1 -- 172.21.15.18:0/3669752208 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac009900 con 0x7fabb4106570 2024-02-17T12:54:36.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.919+0000 7fababfff640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb4106570 0x7fabb419fc30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00e9f0 tx=0x7fab9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.919+0000 7fabab7fe640 1 -- 172.21.15.18:0/3669752208 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0096a0 con 0x7fabb4106570 2024-02-17T12:54:36.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.919+0000 7fabab7fe640 1 -- 172.21.15.18:0/3669752208 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab9c00ccc0 con 0x7fabb4106570 2024-02-17T12:54:36.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.919+0000 7fabab7fe640 1 -- 172.21.15.18:0/3669752208 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c018860 con 0x7fabb4106570 2024-02-17T12:54:36.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.919+0000 7fabb8f01640 1 -- 172.21.15.18:0/3669752208 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb41a65b0 con 0x7fabb4106570 2024-02-17T12:54:36.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.919+0000 7fabb8f01640 1 -- 172.21.15.18:0/3669752208 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb41a6ac0 con 0x7fabb4106570 2024-02-17T12:54:36.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.922+0000 7fabab7fe640 1 -- 172.21.15.18:0/3669752208 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fab9c010aa0 con 0x7fabb4106570 2024-02-17T12:54:36.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.922+0000 7fabbbba8740 1 Processor -- start 2024-02-17T12:54:36.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.922+0000 7fabbbba8740 1 -- start start 2024-02-17T12:54:36.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.922+0000 7fabbbba8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3a250 0x557accf3a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.922+0000 7fabbbba8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3ab70 0x557accf43210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.922+0000 7fabbbba8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf43750 0x557accf45b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.922+0000 7fabbbba8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accef5960 con 0x557accf3a250 2024-02-17T12:54:36.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.922+0000 7fabbbba8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accea9310 con 0x557accf43750 2024-02-17T12:54:36.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.922+0000 7fabbbba8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accea6840 con 0x557accf3ab70 2024-02-17T12:54:36.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.922+0000 7fabab7fe640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7faba00a2230 0x7faba00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabb2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3a250 0x557accf3a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabb2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3a250 0x557accf3a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42804/0 (socket says 172.21.15.18:42804) 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabb2d9d640 1 -- 172.21.15.18:0/1320144822 learned_addr learned my addr 172.21.15.18:0/1320144822 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabb259c640 1 --2- 172.21.15.18:0/1320144822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3ab70 0x557accf43210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fababfff640 1 --2- 172.21.15.18:0/1320144822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf43750 0x557accf45b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7faba00a2230 0x7faba00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabab7fe640 1 -- 172.21.15.18:0/3669752208 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fab9c014070 con 0x7fabb4106570 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabb2d9d640 1 -- 172.21.15.18:0/1320144822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3ab70 msgr2=0x557accf43210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/1320144822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3ab70 0x557accf43210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabb2d9d640 1 -- 172.21.15.18:0/1320144822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf43750 msgr2=0x557accf45b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/1320144822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf43750 0x557accf45b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabb2d9d640 1 -- 172.21.15.18:0/1320144822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557acce64060 con 0x557accf3a250 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/1320144822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3a250 0x557accf3a630 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fabac004af0 tx=0x7fabac034810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7baf0542508341d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7faba00a2230 0x7faba00a46f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fabac02c700 tx=0x7fabac03c000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7faba8ff9640 1 -- 172.21.15.18:0/1320144822 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac048070 con 0x557accf3a250 2024-02-17T12:54:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7faba8ff9640 1 -- 172.21.15.18:0/1320144822 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabac032df0 con 0x557accf3a250 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.923+0000 7faba8ff9640 1 -- 172.21.15.18:0/1320144822 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac0042a0 con 0x557accf3a250 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.924+0000 7fabbbba8740 1 -- 172.21.15.18:0/1320144822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3a250 msgr2=0x557accf3a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.924+0000 7fabbbba8740 1 --2- 172.21.15.18:0/1320144822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3a250 0x557accf3a630 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fabac004af0 tx=0x7fabac034810 comp rx=0 tx=0).stop 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.924+0000 7fabbbba8740 1 -- 172.21.15.18:0/1320144822 shutdown_connections 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.924+0000 7fabbbba8740 1 --2- 172.21.15.18:0/1320144822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf43750 0x557accf45b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.924+0000 7fabbbba8740 1 --2- 172.21.15.18:0/1320144822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3ab70 0x557accf43210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.924+0000 7fabbbba8740 1 --2- 172.21.15.18:0/1320144822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3a250 0x557accf3a630 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.924+0000 7fabbbba8740 1 -- 172.21.15.18:0/1320144822 >> 172.21.15.18:0/1320144822 conn(0x557accf34980 msgr2=0x557accf34d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.924+0000 7fabbbba8740 1 -- 172.21.15.18:0/1320144822 shutdown_connections 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.924+0000 7fabbbba8740 1 -- 172.21.15.18:0/1320144822 wait complete. 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.924+0000 7fabbbba8740 1 -- start start 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabbbba8740 1 Processor -- start 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabbbba8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf407d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabbbba8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf3ab70 0x557accf40d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabbbba8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 0x557accf41250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabbbba8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accea2020 con 0x557accf43750 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabbbba8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accb75430 con 0x557accf3ab70 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabbbba8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accb31e00 con 0x557accf3a250 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabbbba8740 10 client.0 _ll_register_callbacks cb 0x557acce49cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabb2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 0x557accf41250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabb2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 0x557accf41250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42810/0 (socket says 172.21.15.18:42810) 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabb2d9d640 1 -- 172.21.15.18:0/2132323012 learned_addr learned my addr 172.21.15.18:0/2132323012 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabb259c640 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fababfff640 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf3ab70 0x557accf40d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabb2d9d640 1 -- 172.21.15.18:0/2132323012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 msgr2=0x557accf407d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf407d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabb2d9d640 1 -- 172.21.15.18:0/2132323012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf3ab70 msgr2=0x557accf40d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf3ab70 0x557accf40d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabb2d9d640 1 -- 172.21.15.18:0/2132323012 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb410e900 con 0x557accf43750 2024-02-17T12:54:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 0x557accf41250 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fabac0093a0 tx=0x7fabac02c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598e4a54b32c5e08 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.925+0000 7fab99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fab99ffb640 1 -- 172.21.15.18:0/2132323012 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac048040 con 0x557accf43750 2024-02-17T12:54:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fab99ffb640 1 -- 172.21.15.18:0/2132323012 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fabac004450 con 0x557accf43750 2024-02-17T12:54:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fabbbba8740 1 -- 172.21.15.18:0/2132323012 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557accbde720 con 0x557accf43750 2024-02-17T12:54:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fab99ffb640 1 -- 172.21.15.18:0/2132323012 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabac004b40 con 0x557accf43750 2024-02-17T12:54:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fabbbba8740 10 client.17241 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fabbbba8740 1 -- 172.21.15.18:0/2132323012 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557accea06a0 con 0x557accf43750 2024-02-17T12:54:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fab99ffb640 1 -- 172.21.15.18:0/2132323012 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557accbde720 con 0x557accf43750 2024-02-17T12:54:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fab99ffb640 1 -- 172.21.15.18:0/2132323012 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fabac057b80 con 0x557accf43750 2024-02-17T12:54:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fabbbba8740 10 client.17241 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fabbbba8740 10 client.17241 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:54:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fabbbba8740 20 client.17241 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fabbbba8740 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557accf53930 0x557accf55d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fabbbba8740 4 client.17241 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fabbbba8740 1 -- 172.21.15.18:0/2132323012 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557accc2d890 con 0x557accf53930 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.926+0000 7fabb259c640 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557accf53930 0x557accf55d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.927+0000 7fab99ffb640 10 client.17241 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.927+0000 7fabb259c640 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557accf53930 0x557accf55d20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d5245cc9382812d server_cookie=d21aa0fcdc176823 in_seq=0 out_seq=0 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fab99ffb640 1 -- 172.21.15.18:0/2132323012 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557accc2d890 con 0x557accf53930 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fab99ffb640 10 client.17241 handle_command_reply: tid=0 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fabbbba8740 1 client.17241 shutdown 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fabbbba8740 1 -- 172.21.15.18:0/2132323012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 msgr2=0x557accf41250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fabbbba8740 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 0x557accf41250 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fabac0093a0 tx=0x7fabac02c770 comp rx=0 tx=0).stop 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fabbbba8740 1 -- 172.21.15.18:0/2132323012 shutdown_connections 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fabbbba8740 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557accf53930 0x557accf55d20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fabbbba8740 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 0x557accf41250 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fab99ffb640 0 client.17241 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:54:36.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fabbbba8740 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf3ab70 0x557accf40d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fabbbba8740 1 --2- 172.21.15.18:0/2132323012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fabbbba8740 1 -- 172.21.15.18:0/2132323012 >> 172.21.15.18:0/2132323012 conn(0x557accf34980 msgr2=0x557accf366d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fabbbba8740 1 -- 172.21.15.18:0/2132323012 shutdown_connections 2024-02-17T12:54:36.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fabbbba8740 1 -- 172.21.15.18:0/2132323012 wait complete. 2024-02-17T12:54:36.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.928+0000 7fabbbba8740 20 client.17241 trim_cache size 0 max 16384 2024-02-17T12:54:36.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.939+0000 7fabbbba8740 1 Processor -- start 2024-02-17T12:54:36.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.939+0000 7fabbbba8740 1 -- start start 2024-02-17T12:54:36.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabbbba8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf42f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabbbba8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf3ab70 0x557accf41070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabbbba8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 0x557accf3da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabbbba8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acce27130 con 0x557accf43750 2024-02-17T12:54:36.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabbbba8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accf3aa30 con 0x557accf3ab70 2024-02-17T12:54:36.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabbbba8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accf305c0 con 0x557accf3a250 2024-02-17T12:54:36.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabb2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 0x557accf3da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabb2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 0x557accf3da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42826/0 (socket says 172.21.15.18:42826) 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabb2d9d640 1 -- 172.21.15.18:0/4284086123 learned_addr learned my addr 172.21.15.18:0/4284086123 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabb259c640 1 --2- 172.21.15.18:0/4284086123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf42f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabb2d9d640 1 -- 172.21.15.18:0/4284086123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 msgr2=0x557accf42f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/4284086123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf42f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabb2d9d640 1 -- 172.21.15.18:0/4284086123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf3ab70 msgr2=0x557accf41070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/4284086123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf3ab70 0x557accf41070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabb2d9d640 1 -- 172.21.15.18:0/4284086123 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557acce3cc20 con 0x557accf43750 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabb259c640 1 --2- 172.21.15.18:0/4284086123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf42f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/4284086123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 0x557accf3da70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fabac02c630 tx=0x7fabac009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d66c29775491c49 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fab9bfff640 1 -- 172.21.15.18:0/4284086123 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac06d070 con 0x557accf43750 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fab9bfff640 1 -- 172.21.15.18:0/4284086123 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabac032b40 con 0x557accf43750 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.940+0000 7fab9bfff640 1 -- 172.21.15.18:0/4284086123 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac048070 con 0x557accf43750 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 -- 172.21.15.18:0/4284086123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 msgr2=0x557accf3da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 --2- 172.21.15.18:0/4284086123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 0x557accf3da70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fabac02c630 tx=0x7fabac009680 comp rx=0 tx=0).stop 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 -- 172.21.15.18:0/4284086123 shutdown_connections 2024-02-17T12:54:36.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 --2- 172.21.15.18:0/4284086123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf43750 0x557accf3da70 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 --2- 172.21.15.18:0/4284086123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf3ab70 0x557accf41070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 --2- 172.21.15.18:0/4284086123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf42f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 -- 172.21.15.18:0/4284086123 >> 172.21.15.18:0/4284086123 conn(0x557accf34980 msgr2=0x557accf3f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 -- 172.21.15.18:0/4284086123 shutdown_connections 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 -- 172.21.15.18:0/4284086123 wait complete. 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 -- start start 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 Processor -- start 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf4a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3ab70 0x557accf48b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf43750 0x557accf49090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accf49620 con 0x557accf3ab70 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accf49790 con 0x557accf43750 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accf49950 con 0x557accf3a250 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fabbbba8740 10 client.0 _ll_register_callbacks cb 0x7fab9c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.941+0000 7fababfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3ab70 0x557accf48b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fababfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3ab70 0x557accf48b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42836/0 (socket says 172.21.15.18:42836) 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fababfff640 1 -- 172.21.15.18:0/2702102353 learned_addr learned my addr 172.21.15.18:0/2702102353 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fabb259c640 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf4a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fabb2d9d640 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf43750 0x557accf49090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fababfff640 1 -- 172.21.15.18:0/2702102353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 msgr2=0x557accf4a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fababfff640 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf4a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fababfff640 1 -- 172.21.15.18:0/2702102353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf43750 msgr2=0x557accf49090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fababfff640 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf43750 0x557accf49090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fababfff640 1 -- 172.21.15.18:0/2702102353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557accea06a0 con 0x557accf3ab70 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fababfff640 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3ab70 0x557accf48b50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00e9c0 tx=0x7fab9c0663e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c9a3288ec94a72 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fab99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fab99ffb640 1 -- 172.21.15.18:0/2702102353 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c09a490 con 0x557accf3ab70 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fab99ffb640 1 -- 172.21.15.18:0/2702102353 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fab9c09ae40 con 0x557accf3ab70 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fabbbba8740 1 -- 172.21.15.18:0/2702102353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557accba17e0 con 0x557accf3ab70 2024-02-17T12:54:36.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fab99ffb640 1 -- 172.21.15.18:0/2702102353 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab9c09a5f0 con 0x557accf3ab70 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fabbbba8740 10 client.17247 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fabbbba8740 1 -- 172.21.15.18:0/2702102353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557acce628c0 con 0x557accf3ab70 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fab99ffb640 1 -- 172.21.15.18:0/2702102353 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557accba17e0 con 0x557accf3ab70 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.942+0000 7fab99ffb640 1 -- 172.21.15.18:0/2702102353 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fab9c0ce040 con 0x557accf3ab70 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.943+0000 7fabbbba8740 10 client.17247 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.943+0000 7fabbbba8740 10 client.17247 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.943+0000 7fabbbba8740 20 client.17247 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.943+0000 7fabbbba8740 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557accfab3d0 0x557accfad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.943+0000 7fabbbba8740 4 client.17247 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.943+0000 7fabbbba8740 1 -- 172.21.15.18:0/2702102353 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557accb4d700 con 0x557accfab3d0 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.943+0000 7fabb259c640 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557accfab3d0 0x557accfad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.943+0000 7fabb259c640 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557accfab3d0 0x557accfad7f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e564e1b96eaee98f server_cookie=f3ef493775a65ce5 in_seq=0 out_seq=0 2024-02-17T12:54:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.943+0000 7fab99ffb640 10 client.17247 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fab99ffb640 1 -- 172.21.15.18:0/2702102353 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557accb4d700 con 0x557accfab3d0 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fab99ffb640 10 client.17247 handle_command_reply: tid=0 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabbbba8740 1 client.17247 shutdown 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabbbba8740 1 -- 172.21.15.18:0/2702102353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3ab70 msgr2=0x557accf48b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabbbba8740 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3ab70 0x557accf48b50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00e9c0 tx=0x7fab9c0663e0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabbbba8740 1 -- 172.21.15.18:0/2702102353 shutdown_connections 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabbbba8740 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x557accfab3d0 0x557accfad7f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabbbba8740 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557accf43750 0x557accf49090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabbbba8740 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557accf3ab70 0x557accf48b50 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabbbba8740 1 --2- 172.21.15.18:0/2702102353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557accf3a250 0x557accf4a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fab99ffb640 0 client.17247 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabbbba8740 1 -- 172.21.15.18:0/2702102353 >> 172.21.15.18:0/2702102353 conn(0x557accf34980 msgr2=0x557accf3f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabbbba8740 1 -- 172.21.15.18:0/2702102353 shutdown_connections 2024-02-17T12:54:36.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabbbba8740 1 -- 172.21.15.18:0/2702102353 wait complete. 2024-02-17T12:54:36.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabbbba8740 20 client.17247 trim_cache size 0 max 16384 2024-02-17T12:54:36.942 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:54:36.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabb8f01640 1 -- 172.21.15.18:0/3669752208 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7faba00a2230 msgr2=0x7faba00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabb8f01640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7faba00a2230 0x7faba00a46f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fabac02c700 tx=0x7fabac03c000 comp rx=0 tx=0).stop 2024-02-17T12:54:36.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabb8f01640 1 -- 172.21.15.18:0/3669752208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb4106570 msgr2=0x7fabb419fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:36.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabb8f01640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb4106570 0x7fabb419fc30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00e9f0 tx=0x7fab9c00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabb8f01640 1 -- 172.21.15.18:0/3669752208 shutdown_connections 2024-02-17T12:54:36.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabb8f01640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7faba00a2230 0x7faba00a46f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabb8f01640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabb41998f0 0x7fabb4199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabb8f01640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb4106570 0x7fabb419fc30 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabb8f01640 1 --2- 172.21.15.18:0/3669752208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb40ff960 0x7fabb419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:36.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabb8f01640 1 -- 172.21.15.18:0/3669752208 >> 172.21.15.18:0/3669752208 conn(0x7fabb40766b0 msgr2=0x7fabb4104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:36.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabb8f01640 1 -- 172.21.15.18:0/3669752208 shutdown_connections 2024-02-17T12:54:36.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:36.944+0000 7fabb8f01640 1 -- 172.21.15.18:0/3669752208 wait complete. 2024-02-17T12:54:36.946 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:54:36.946 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:54:36.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:36 smithi146 ceph-mon[36454]: pgmap v144: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.7 MiB/s wr, 141 op/s 2024-02-17T12:54:36.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:36 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:54:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.059+0000 7f465df72640 1 -- 172.21.15.18:0/2059724785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4658106450 msgr2=0x7f465810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.059+0000 7f465df72640 1 --2- 172.21.15.18:0/2059724785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4658106450 0x7f465810e020 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f4650009cd0 tx=0x7f465002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.059+0000 7f465df72640 1 -- 172.21.15.18:0/2059724785 shutdown_connections 2024-02-17T12:54:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.059+0000 7f465df72640 1 --2- 172.21.15.18:0/2059724785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4658106450 0x7f465810e020 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.059+0000 7f465df72640 1 --2- 172.21.15.18:0/2059724785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658101d30 0x7f4658105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.059+0000 7f465df72640 1 --2- 172.21.15.18:0/2059724785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4658101380 0x7f4658101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.059+0000 7f465df72640 1 -- 172.21.15.18:0/2059724785 >> 172.21.15.18:0/2059724785 conn(0x7f4658078580 msgr2=0x7f4658078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.059+0000 7f465df72640 1 -- 172.21.15.18:0/2059724785 shutdown_connections 2024-02-17T12:54:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.059+0000 7f465df72640 1 -- 172.21.15.18:0/2059724785 wait complete. 2024-02-17T12:54:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.059+0000 7f465df72640 1 Processor -- start 2024-02-17T12:54:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f465df72640 1 -- start start 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f465df72640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658101380 0x7f4658073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f465df72640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4658106450 0x7f46580739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f465df72640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4658073f00 0x7f46581a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f465df72640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46581a5fd0 con 0x7f4658106450 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f465df72640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46581a6140 con 0x7f4658101380 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f465df72640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46581a6300 con 0x7f4658073f00 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f4656ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4658106450 0x7f46580739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f4656ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4658106450 0x7f46580739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42852/0 (socket says 172.21.15.18:42852) 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f4656ffd640 1 -- 172.21.15.18:0/1055191869 learned_addr learned my addr 172.21.15.18:0/1055191869 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f46577fe640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658101380 0x7f4658073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f4656ffd640 1 -- 172.21.15.18:0/1055191869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4658073f00 msgr2=0x7f46581a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f4656ffd640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4658073f00 0x7f46581a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f4656ffd640 1 -- 172.21.15.18:0/1055191869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658101380 msgr2=0x7f4658073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f4656ffd640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658101380 0x7f4658073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f4656ffd640 1 -- 172.21.15.18:0/1055191869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4650009900 con 0x7f4658106450 2024-02-17T12:54:37.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f4656ffd640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4658106450 0x7f46580739c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f464c00b450 tx=0x7f464c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.060+0000 7f4654ff9640 1 -- 172.21.15.18:0/1055191869 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c011890 con 0x7f4658106450 2024-02-17T12:54:37.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.061+0000 7f4654ff9640 1 -- 172.21.15.18:0/1055191869 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f464c0103e0 con 0x7f4658106450 2024-02-17T12:54:37.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.061+0000 7f465df72640 1 -- 172.21.15.18:0/1055191869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46581a65e0 con 0x7f4658106450 2024-02-17T12:54:37.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.061+0000 7f4654ff9640 1 -- 172.21.15.18:0/1055191869 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c00f580 con 0x7f4658106450 2024-02-17T12:54:37.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.061+0000 7f465df72640 1 -- 172.21.15.18:0/1055191869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46581a6af0 con 0x7f4658106450 2024-02-17T12:54:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.063+0000 7f4654ff9640 1 -- 172.21.15.18:0/1055191869 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f464c0108e0 con 0x7f4658106450 2024-02-17T12:54:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.063+0000 7f4660c16740 1 Processor -- start 2024-02-17T12:54:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.063+0000 7f4660c16740 1 -- start start 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.063+0000 7f4654ff9640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46400a2260 0x7f46400a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.063+0000 7f4660c16740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790a8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.063+0000 7f4660c16740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790b1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.063+0000 7f4660c16740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.063+0000 7f4660c16740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e79063960 con 0x561e790b1750 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.063+0000 7f4660c16740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e79017310 con 0x561e790a8250 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.063+0000 7f4660c16740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e79014840 con 0x561e790a8b70 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.063+0000 7f4654ff9640 1 -- 172.21.15.18:0/1055191869 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f464c0cdd60 con 0x7f4658106450 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f4657fff640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46400a2260 0x7f46400a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f46577fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f46577fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790a8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:42938/0 (socket says 172.21.15.18:42938) 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f46577fe640 1 -- 172.21.15.18:0/802260939 learned_addr learned my addr 172.21.15.18:0/802260939 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f4657fff640 1 --2- 172.21.15.18:0/802260939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f4656ffd640 1 --2- 172.21.15.18:0/802260939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f46577fe640 1 -- 172.21.15.18:0/802260939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 msgr2=0x561e790b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f46577fe640 1 --2- 172.21.15.18:0/802260939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f46577fe640 1 -- 172.21.15.18:0/802260939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 msgr2=0x561e790b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f46577fe640 1 --2- 172.21.15.18:0/802260939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f46577fe640 1 -- 172.21.15.18:0/802260939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e78fd2060 con 0x561e790a8250 2024-02-17T12:54:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f4657fff640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46400a2260 0x7f46400a4720 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4650006010 tx=0x7f4650032530 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f46577fe640 1 --2- 172.21.15.18:0/802260939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790a8630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4648009c10 tx=0x7f46480117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f59ee90d5135147c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f463dffb640 1 -- 172.21.15.18:0/802260939 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4648009760 con 0x561e790a8250 2024-02-17T12:54:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f463dffb640 1 -- 172.21.15.18:0/802260939 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4648009e40 con 0x561e790a8250 2024-02-17T12:54:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.064+0000 7f463dffb640 1 -- 172.21.15.18:0/802260939 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464800fa70 con 0x561e790a8250 2024-02-17T12:54:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.065+0000 7f4660c16740 1 -- 172.21.15.18:0/802260939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 msgr2=0x561e790a8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.065+0000 7f4660c16740 1 --2- 172.21.15.18:0/802260939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790a8630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4648009c10 tx=0x7f46480117b0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.065+0000 7f4660c16740 1 -- 172.21.15.18:0/802260939 shutdown_connections 2024-02-17T12:54:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.065+0000 7f4660c16740 1 --2- 172.21.15.18:0/802260939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.065+0000 7f4660c16740 1 --2- 172.21.15.18:0/802260939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.065+0000 7f4660c16740 1 --2- 172.21.15.18:0/802260939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790a8630 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.065+0000 7f4660c16740 1 -- 172.21.15.18:0/802260939 >> 172.21.15.18:0/802260939 conn(0x561e790a2980 msgr2=0x561e790a2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.065+0000 7f4660c16740 1 -- 172.21.15.18:0/802260939 shutdown_connections 2024-02-17T12:54:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.065+0000 7f4660c16740 1 -- 172.21.15.18:0/802260939 wait complete. 2024-02-17T12:54:37.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.065+0000 7f4660c16740 1 -- start start 2024-02-17T12:54:37.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4660c16740 1 Processor -- start 2024-02-17T12:54:37.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4660c16740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4660c16740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4660c16740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790af110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4660c16740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e79010020 con 0x561e790b1750 2024-02-17T12:54:37.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4660c16740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e78ce3430 con 0x561e790a8250 2024-02-17T12:54:37.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4660c16740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e78c9fe00 con 0x561e790a8b70 2024-02-17T12:54:37.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4660c16740 10 client.0 _ll_register_callbacks cb 0x561e78fb7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:37.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4657fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4657fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42870/0 (socket says 172.21.15.18:42870) 2024-02-17T12:54:37.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4657fff640 1 -- 172.21.15.18:0/3642092512 learned_addr learned my addr 172.21.15.18:0/3642092512 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f46577fe640 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4656ffd640 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4657fff640 1 -- 172.21.15.18:0/3642092512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 msgr2=0x561e790aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4657fff640 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4657fff640 1 -- 172.21.15.18:0/3642092512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 msgr2=0x561e790ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4657fff640 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4657fff640 1 -- 172.21.15.18:0/3642092512 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46480098c0 con 0x561e790b1750 2024-02-17T12:54:37.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4657fff640 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790af110 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4650009cd0 tx=0x7f46500095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3858b965b0b7467 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.066+0000 7f4636ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:37.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4636ffd640 1 -- 172.21.15.18:0/3642092512 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650003b70 con 0x561e790b1750 2024-02-17T12:54:37.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4636ffd640 1 -- 172.21.15.18:0/3642092512 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4650003cd0 con 0x561e790b1750 2024-02-17T12:54:37.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4660c16740 1 -- 172.21.15.18:0/3642092512 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e78d0f7e0 con 0x561e790b1750 2024-02-17T12:54:37.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4636ffd640 1 -- 172.21.15.18:0/3642092512 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4650003e90 con 0x561e790b1750 2024-02-17T12:54:37.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4660c16740 10 client.17259 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:37.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4660c16740 1 -- 172.21.15.18:0/3642092512 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561e7900e6a0 con 0x561e790b1750 2024-02-17T12:54:37.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4636ffd640 1 -- 172.21.15.18:0/3642092512 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561e78d0f7e0 con 0x561e790b1750 2024-02-17T12:54:37.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4636ffd640 1 -- 172.21.15.18:0/3642092512 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f465003e040 con 0x561e790b1750 2024-02-17T12:54:37.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4660c16740 10 client.17259 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:37.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4660c16740 10 client.17259 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:54:37.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4660c16740 20 client.17259 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:37.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4660c16740 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561e790c1ab0 0x561e790c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4660c16740 4 client.17259 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:37.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f4660c16740 1 -- 172.21.15.18:0/3642092512 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e78d4c720 con 0x561e790c1ab0 2024-02-17T12:54:37.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.067+0000 7f46577fe640 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561e790c1ab0 0x561e790c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.068+0000 7f46577fe640 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561e790c1ab0 0x561e790c3ed0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae78f8c456473675 server_cookie=3a3c2c259ffd776 in_seq=0 out_seq=0 2024-02-17T12:54:37.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.068+0000 7f4636ffd640 10 client.17259 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:54:37.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4636ffd640 1 -- 172.21.15.18:0/3642092512 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561e78d4c720 con 0x561e790c1ab0 2024-02-17T12:54:37.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4636ffd640 10 client.17259 handle_command_reply: tid=0 2024-02-17T12:54:37.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4660c16740 1 client.17259 shutdown 2024-02-17T12:54:37.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4660c16740 1 -- 172.21.15.18:0/3642092512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 msgr2=0x561e790af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4660c16740 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790af110 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4650009cd0 tx=0x7f46500095c0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4660c16740 1 -- 172.21.15.18:0/3642092512 shutdown_connections 2024-02-17T12:54:37.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4660c16740 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561e790c1ab0 0x561e790c3ed0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4660c16740 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790af110 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4660c16740 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4636ffd640 0 client.17259 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:54:37.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4660c16740 1 --2- 172.21.15.18:0/3642092512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4660c16740 1 -- 172.21.15.18:0/3642092512 >> 172.21.15.18:0/3642092512 conn(0x561e790a2980 msgr2=0x561e790a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4660c16740 1 -- 172.21.15.18:0/3642092512 shutdown_connections 2024-02-17T12:54:37.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4660c16740 1 -- 172.21.15.18:0/3642092512 wait complete. 2024-02-17T12:54:37.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.069+0000 7f4660c16740 20 client.17259 trim_cache size 0 max 16384 2024-02-17T12:54:37.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.080+0000 7f4660c16740 1 Processor -- start 2024-02-17T12:54:37.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f4660c16740 1 -- start start 2024-02-17T12:54:37.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f4660c16740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790a8250 0x561e790ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f4660c16740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f4660c16740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790b1750 0x561e790a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f4660c16740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e78f95130 con 0x561e790a8250 2024-02-17T12:54:37.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f4660c16740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e790a7a20 con 0x561e790b1750 2024-02-17T12:54:37.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f4660c16740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e790a3980 con 0x561e790a8b70 2024-02-17T12:54:37.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f46577fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790a8250 0x561e790ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f46577fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790a8250 0x561e790ad7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42884/0 (socket says 172.21.15.18:42884) 2024-02-17T12:54:37.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f46577fe640 1 -- 172.21.15.18:0/1513204312 learned_addr learned my addr 172.21.15.18:0/1513204312 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f4656ffd640 1 --2- 172.21.15.18:0/1513204312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f4657fff640 1 --2- 172.21.15.18:0/1513204312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790b1750 0x561e790a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f46577fe640 1 -- 172.21.15.18:0/1513204312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 msgr2=0x561e790aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f46577fe640 1 --2- 172.21.15.18:0/1513204312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f46577fe640 1 -- 172.21.15.18:0/1513204312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790b1750 msgr2=0x561e790a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f46577fe640 1 --2- 172.21.15.18:0/1513204312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790b1750 0x561e790a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f46577fe640 1 -- 172.21.15.18:0/1513204312 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e78faac20 con 0x561e790a8250 2024-02-17T12:54:37.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f4657fff640 1 --2- 172.21.15.18:0/1513204312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790b1750 0x561e790a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:37.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f46577fe640 1 --2- 172.21.15.18:0/1513204312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790a8250 0x561e790ad7f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4648000c00 tx=0x7f464800f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f991dc9ac810e7f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f463d7fa640 1 -- 172.21.15.18:0/1513204312 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4648005390 con 0x561e790a8250 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f463d7fa640 1 -- 172.21.15.18:0/1513204312 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f464800f440 con 0x561e790a8250 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.081+0000 7f463d7fa640 1 -- 172.21.15.18:0/1513204312 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4648003850 con 0x561e790a8250 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.082+0000 7f4660c16740 1 -- 172.21.15.18:0/1513204312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790a8250 msgr2=0x561e790ad7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.082+0000 7f4660c16740 1 --2- 172.21.15.18:0/1513204312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790a8250 0x561e790ad7f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4648000c00 tx=0x7f464800f5e0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.082+0000 7f4660c16740 1 -- 172.21.15.18:0/1513204312 shutdown_connections 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.082+0000 7f4660c16740 1 --2- 172.21.15.18:0/1513204312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790b1750 0x561e790a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.082+0000 7f4660c16740 1 --2- 172.21.15.18:0/1513204312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.082+0000 7f4660c16740 1 --2- 172.21.15.18:0/1513204312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790a8250 0x561e790ad7f0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.082+0000 7f4660c16740 1 -- 172.21.15.18:0/1513204312 >> 172.21.15.18:0/1513204312 conn(0x561e790a2980 msgr2=0x561e790a12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.082+0000 7f4660c16740 1 -- 172.21.15.18:0/1513204312 shutdown_connections 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.082+0000 7f4660c16740 1 -- 172.21.15.18:0/1513204312 wait complete. 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.082+0000 7f4660c16740 1 -- start start 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.082+0000 7f4660c16740 1 Processor -- start 2024-02-17T12:54:37.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4660c16740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790b7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4660c16740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790b7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4660c16740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4660c16740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e790b8f40 con 0x561e790b1750 2024-02-17T12:54:37.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4660c16740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e79117fc0 con 0x561e790a8250 2024-02-17T12:54:37.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4660c16740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e79118130 con 0x561e790a8b70 2024-02-17T12:54:37.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4660c16740 10 client.0 _ll_register_callbacks cb 0x7f46480096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:37.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4657fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4657fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790b7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42894/0 (socket says 172.21.15.18:42894) 2024-02-17T12:54:37.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4657fff640 1 -- 172.21.15.18:0/915765045 learned_addr learned my addr 172.21.15.18:0/915765045 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4656ffd640 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4657fff640 1 -- 172.21.15.18:0/915765045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 msgr2=0x561e790b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4657fff640 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790b7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4657fff640 1 -- 172.21.15.18:0/915765045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 msgr2=0x561e790b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f46577fe640 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4657fff640 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790b7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4657fff640 1 -- 172.21.15.18:0/915765045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e78fd2060 con 0x561e790b1750 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4657fff640 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790b7ac0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f465002c700 tx=0x7f465002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da8be12ec0c1129 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4636ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4636ffd640 1 -- 172.21.15.18:0/915765045 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650003b70 con 0x561e790b1750 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4636ffd640 1 -- 172.21.15.18:0/915765045 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4650003cd0 con 0x561e790b1750 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.083+0000 7f4660c16740 1 -- 172.21.15.18:0/915765045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4648009e40 con 0x561e790b1750 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.084+0000 7f4636ffd640 1 -- 172.21.15.18:0/915765045 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4650003e90 con 0x561e790b1750 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.084+0000 7f4660c16740 10 client.17265 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.084+0000 7f4660c16740 1 -- 172.21.15.18:0/915765045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561e78fd08c0 con 0x561e790b1750 2024-02-17T12:54:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.084+0000 7f4636ffd640 1 -- 172.21.15.18:0/915765045 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4648009e40 con 0x561e790b1750 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.084+0000 7f4636ffd640 1 -- 172.21.15.18:0/915765045 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f465006a680 con 0x561e790b1750 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.084+0000 7f4660c16740 10 client.17265 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.084+0000 7f4660c16740 10 client.17265 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.084+0000 7f4660c16740 20 client.17265 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.084+0000 7f4660c16740 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561e79119630 0x561e7911ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.084+0000 7f4660c16740 4 client.17265 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.084+0000 7f4660c16740 1 -- 172.21.15.18:0/915765045 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e78d9b890 con 0x561e79119630 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.084+0000 7f46577fe640 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561e79119630 0x561e7911ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f46577fe640 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561e79119630 0x561e7911ba50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=702edf0f69b3244c server_cookie=1c8cd9a3d4163458 in_seq=0 out_seq=0 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4636ffd640 10 client.17265 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4636ffd640 1 -- 172.21.15.18:0/915765045 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e78d9b890 con 0x561e79119630 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4636ffd640 10 client.17265 handle_command_reply: tid=0 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4660c16740 1 client.17265 shutdown 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4660c16740 1 -- 172.21.15.18:0/915765045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 msgr2=0x561e790b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4660c16740 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790b7ac0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f465002c700 tx=0x7f465002c7e0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4660c16740 1 -- 172.21.15.18:0/915765045 shutdown_connections 2024-02-17T12:54:37.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4660c16740 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561e79119630 0x561e7911ba50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4660c16740 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561e790b1750 0x561e790b7ac0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4636ffd640 0 client.17265 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:54:37.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4660c16740 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e790a8b70 0x561e790b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4660c16740 1 --2- 172.21.15.18:0/915765045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e790a8250 0x561e790b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4660c16740 1 -- 172.21.15.18:0/915765045 >> 172.21.15.18:0/915765045 conn(0x561e790a2980 msgr2=0x561e790a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4660c16740 1 -- 172.21.15.18:0/915765045 shutdown_connections 2024-02-17T12:54:37.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4660c16740 1 -- 172.21.15.18:0/915765045 wait complete. 2024-02-17T12:54:37.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.085+0000 7f4660c16740 20 client.17265 trim_cache size 0 max 16384 2024-02-17T12:54:37.086 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:54:37.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.086+0000 7f465df72640 1 -- 172.21.15.18:0/1055191869 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46400a2260 msgr2=0x7f46400a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.086+0000 7f465df72640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46400a2260 0x7f46400a4720 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4650006010 tx=0x7f4650032530 comp rx=0 tx=0).stop 2024-02-17T12:54:37.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.086+0000 7f465df72640 1 -- 172.21.15.18:0/1055191869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4658106450 msgr2=0x7f46580739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.086+0000 7f465df72640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4658106450 0x7f46580739c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f464c00b450 tx=0x7f464c00b910 comp rx=0 tx=0).stop 2024-02-17T12:54:37.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.086+0000 7f465df72640 1 -- 172.21.15.18:0/1055191869 shutdown_connections 2024-02-17T12:54:37.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.086+0000 7f465df72640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46400a2260 0x7f46400a4720 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.086+0000 7f465df72640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4658073f00 0x7f46581a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.086+0000 7f465df72640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4658106450 0x7f46580739c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.086+0000 7f465df72640 1 --2- 172.21.15.18:0/1055191869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658101380 0x7f4658073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.086+0000 7f465df72640 1 -- 172.21.15.18:0/1055191869 >> 172.21.15.18:0/1055191869 conn(0x7f4658078580 msgr2=0x7f46580ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.086+0000 7f465df72640 1 -- 172.21.15.18:0/1055191869 shutdown_connections 2024-02-17T12:54:37.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.086+0000 7f465df72640 1 -- 172.21.15.18:0/1055191869 wait complete. 2024-02-17T12:54:37.088 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:54:37.088 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:54:37.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.201+0000 7f52f3748640 1 -- 172.21.15.18:0/2290599241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52ec100c10 msgr2=0x7f52ec104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.201+0000 7f52f3748640 1 --2- 172.21.15.18:0/2290599241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52ec100c10 0x7f52ec104c50 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009930 tx=0x7f52dc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:37.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.201+0000 7f52f3748640 1 -- 172.21.15.18:0/2290599241 shutdown_connections 2024-02-17T12:54:37.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.201+0000 7f52f3748640 1 --2- 172.21.15.18:0/2290599241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52ec105330 0x7f52ec10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 --2- 172.21.15.18:0/2290599241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52ec100c10 0x7f52ec104c50 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 --2- 172.21.15.18:0/2290599241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52ec100260 0x7f52ec100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 -- 172.21.15.18:0/2290599241 >> 172.21.15.18:0/2290599241 conn(0x7f52ec077180 msgr2=0x7f52ec077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 -- 172.21.15.18:0/2290599241 shutdown_connections 2024-02-17T12:54:37.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 -- 172.21.15.18:0/2290599241 wait complete. 2024-02-17T12:54:37.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 Processor -- start 2024-02-17T12:54:37.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 -- start start 2024-02-17T12:54:37.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52ec100260 0x7f52ec1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52ec105330 0x7f52ec198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52ec1990c0 0x7f52ec1a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec1a3d40 con 0x7f52ec1990c0 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec1a3eb0 con 0x7f52ec105330 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f3748640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec1a4070 con 0x7f52ec100260 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.202+0000 7f52f1ce5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52ec1990c0 0x7f52ec1a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52f14e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52ec100260 0x7f52ec1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52f1ce5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52ec1990c0 0x7f52ec1a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42904/0 (socket says 172.21.15.18:42904) 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52f1ce5640 1 -- 172.21.15.18:0/2537088145 learned_addr learned my addr 172.21.15.18:0/2537088145 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52ec105330 0x7f52ec198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52f1ce5640 1 -- 172.21.15.18:0/2537088145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52ec100260 msgr2=0x7f52ec1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52f1ce5640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52ec100260 0x7f52ec1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52f1ce5640 1 -- 172.21.15.18:0/2537088145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52ec105330 msgr2=0x7f52ec198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52f1ce5640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52ec105330 0x7f52ec198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52f1ce5640 1 -- 172.21.15.18:0/2537088145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc009560 con 0x7f52ec1990c0 2024-02-17T12:54:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52f1ce5640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52ec1990c0 0x7f52ec1a3670 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f52e800eb10 tx=0x7f52e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52e27fc640 1 -- 172.21.15.18:0/2537088145 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8004b20 con 0x7f52ec1990c0 2024-02-17T12:54:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52e27fc640 1 -- 172.21.15.18:0/2537088145 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52e8004c80 con 0x7f52ec1990c0 2024-02-17T12:54:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52f3748640 1 -- 172.21.15.18:0/2537088145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec1a4350 con 0x7f52ec1990c0 2024-02-17T12:54:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52e27fc640 1 -- 172.21.15.18:0/2537088145 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8010650 con 0x7f52ec1990c0 2024-02-17T12:54:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.203+0000 7f52f3748640 1 -- 172.21.15.18:0/2537088145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec1a47e0 con 0x7f52ec1990c0 2024-02-17T12:54:37.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.205+0000 7f52e27fc640 1 -- 172.21.15.18:0/2537088145 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f52e8005020 con 0x7f52ec1990c0 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f63f2740 1 Processor -- start 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f63f2740 1 -- start start 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52e27fc640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52d80a2230 0x7f52d80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f63f2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 0x564919cfd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f63f2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d06210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f63f2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d08b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f63f2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564919cb8960 con 0x564919cfdb70 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f63f2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564919c6c310 con 0x564919d06750 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f63f2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564919c69840 con 0x564919cfd250 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52d80a2230 0x7f52d80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f0ce3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d06210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52e27fc640 1 -- 172.21.15.18:0/2537088145 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f52e8014070 con 0x7f52ec1990c0 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f14e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 0x564919cfd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f0ce3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d06210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42918/0 (socket says 172.21.15.18:42918) 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.206+0000 7f52f0ce3640 1 -- 172.21.15.18:0/2251343880 learned_addr learned my addr 172.21.15.18:0/2251343880 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.207+0000 7f52f1ce5640 1 --2- 172.21.15.18:0/2251343880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d08b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.207+0000 7f52f0ce3640 1 -- 172.21.15.18:0/2251343880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 msgr2=0x564919cfd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.207+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/2251343880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 0x564919cfd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.207+0000 7f52f0ce3640 1 -- 172.21.15.18:0/2251343880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 msgr2=0x564919d08b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.207+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/2251343880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d08b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.207+0000 7f52f0ce3640 1 -- 172.21.15.18:0/2251343880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564919c27060 con 0x564919cfdb70 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.207+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/2251343880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d06210 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f52ec06a8e0 tx=0x7f52dc02cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293c2bf1b5e1f5e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.207+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52d80a2230 0x7f52d80a46f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009930 tx=0x7f52dc03b000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.207+0000 7f52d77fe640 1 -- 172.21.15.18:0/2251343880 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc048070 con 0x564919cfdb70 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.207+0000 7f52d77fe640 1 -- 172.21.15.18:0/2251343880 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52dc004410 con 0x564919cfdb70 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.207+0000 7f52d77fe640 1 -- 172.21.15.18:0/2251343880 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc003850 con 0x564919cfdb70 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.208+0000 7f52f63f2740 1 -- 172.21.15.18:0/2251343880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 msgr2=0x564919d06210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.208+0000 7f52f63f2740 1 --2- 172.21.15.18:0/2251343880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d06210 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f52ec06a8e0 tx=0x7f52dc02cae0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.208+0000 7f52f63f2740 1 -- 172.21.15.18:0/2251343880 shutdown_connections 2024-02-17T12:54:37.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.208+0000 7f52f63f2740 1 --2- 172.21.15.18:0/2251343880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d08b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.208+0000 7f52f63f2740 1 --2- 172.21.15.18:0/2251343880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d06210 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.208+0000 7f52f63f2740 1 --2- 172.21.15.18:0/2251343880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 0x564919cfd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.208+0000 7f52f63f2740 1 -- 172.21.15.18:0/2251343880 >> 172.21.15.18:0/2251343880 conn(0x564919cf7980 msgr2=0x564919cf7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.208+0000 7f52f63f2740 1 -- 172.21.15.18:0/2251343880 shutdown_connections 2024-02-17T12:54:37.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.208+0000 7f52f63f2740 1 -- 172.21.15.18:0/2251343880 wait complete. 2024-02-17T12:54:37.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.208+0000 7f52f63f2740 1 -- start start 2024-02-17T12:54:37.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.208+0000 7f52f63f2740 1 Processor -- start 2024-02-17T12:54:37.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f63f2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 0x564919d037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f63f2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d03ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f63f2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d04220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f63f2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564919c65020 con 0x564919cfdb70 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f63f2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564919938430 con 0x564919d06750 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f63f2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649198f4e00 con 0x564919cfd250 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f63f2740 10 client.0 _ll_register_callbacks cb 0x564919c0ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f0ce3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d03ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f0ce3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d03ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42932/0 (socket says 172.21.15.18:42932) 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f0ce3640 1 -- 172.21.15.18:0/1138121180 learned_addr learned my addr 172.21.15.18:0/1138121180 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f1ce5640 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d04220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f14e4640 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 0x564919d037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f0ce3640 1 -- 172.21.15.18:0/1138121180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 msgr2=0x564919d037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 0x564919d037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f0ce3640 1 -- 172.21.15.18:0/1138121180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 msgr2=0x564919d04220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d04220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f0ce3640 1 -- 172.21.15.18:0/1138121180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec10d710 con 0x564919cfdb70 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d03ce0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f52dc005ab0 tx=0x7f52dc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b253559c31092a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52d4ff9640 1 -- 172.21.15.18:0/1138121180 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc048040 con 0x564919cfdb70 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52d4ff9640 1 -- 172.21.15.18:0/1138121180 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f52dc004df0 con 0x564919cfdb70 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.209+0000 7f52f63f2740 1 -- 172.21.15.18:0/1138121180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649199647e0 con 0x564919cfdb70 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.210+0000 7f52d4ff9640 1 -- 172.21.15.18:0/1138121180 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52dc004fb0 con 0x564919cfdb70 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.210+0000 7f52f63f2740 10 client.17277 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.210+0000 7f52f63f2740 1 -- 172.21.15.18:0/1138121180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564919c636a0 con 0x564919cfdb70 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.210+0000 7f52d4ff9640 1 -- 172.21.15.18:0/1138121180 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5649199647e0 con 0x564919cfdb70 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.210+0000 7f52d4ff9640 1 -- 172.21.15.18:0/1138121180 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f52dc04d810 con 0x564919cfdb70 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.210+0000 7f52f63f2740 10 client.17277 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.210+0000 7f52f63f2740 10 client.17277 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.210+0000 7f52f63f2740 20 client.17277 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.210+0000 7f52f63f2740 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x564919d16920 0x564919d18d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.210+0000 7f52f63f2740 4 client.17277 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:37.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.210+0000 7f52f63f2740 1 -- 172.21.15.18:0/1138121180 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649199a1720 con 0x564919d16920 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.210+0000 7f52f14e4640 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x564919d16920 0x564919d18d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.211+0000 7f52d4ff9640 10 client.17277 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.211+0000 7f52f14e4640 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x564919d16920 0x564919d18d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd606c7535629bf9 server_cookie=c0c45e9cb530c4f5 in_seq=0 out_seq=0 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.211+0000 7f52d4ff9640 1 -- 172.21.15.18:0/1138121180 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5649199a1720 con 0x564919d16920 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.211+0000 7f52d4ff9640 10 client.17277 handle_command_reply: tid=0 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.211+0000 7f52f63f2740 1 client.17277 shutdown 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.212+0000 7f52f63f2740 1 -- 172.21.15.18:0/1138121180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 msgr2=0x564919d03ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.212+0000 7f52f63f2740 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d03ce0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f52dc005ab0 tx=0x7f52dc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.212+0000 7f52f63f2740 1 -- 172.21.15.18:0/1138121180 shutdown_connections 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.212+0000 7f52f63f2740 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x564919d16920 0x564919d18d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.212+0000 7f52f63f2740 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d04220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.212+0000 7f52f63f2740 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d03ce0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.212+0000 7f52d4ff9640 0 client.17277 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.212+0000 7f52f63f2740 1 --2- 172.21.15.18:0/1138121180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 0x564919d037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.212+0000 7f52f63f2740 1 -- 172.21.15.18:0/1138121180 >> 172.21.15.18:0/1138121180 conn(0x564919cf7980 msgr2=0x564919cf96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.212+0000 7f52f63f2740 1 -- 172.21.15.18:0/1138121180 shutdown_connections 2024-02-17T12:54:37.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.212+0000 7f52f63f2740 1 -- 172.21.15.18:0/1138121180 wait complete. 2024-02-17T12:54:37.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.212+0000 7f52f63f2740 20 client.17277 trim_cache size 0 max 16384 2024-02-17T12:54:37.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f63f2740 1 Processor -- start 2024-02-17T12:54:37.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f63f2740 1 -- start start 2024-02-17T12:54:37.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f63f2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfd250 0x564919d05ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f63f2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919cfdb70 0x564919d04040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f63f2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919d06750 0x564919d00a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f63f2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564919bea130 con 0x564919cfd250 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f63f2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564919d166d0 con 0x564919cfdb70 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f63f2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564919d1df90 con 0x564919d06750 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f14e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfd250 0x564919d05ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f14e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfd250 0x564919d05ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42946/0 (socket says 172.21.15.18:42946) 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f14e4640 1 -- 172.21.15.18:0/4117519905 learned_addr learned my addr 172.21.15.18:0/4117519905 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f1ce5640 1 --2- 172.21.15.18:0/4117519905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919d06750 0x564919d00a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/4117519905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919cfdb70 0x564919d04040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f14e4640 1 -- 172.21.15.18:0/4117519905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919d06750 msgr2=0x564919d00a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f14e4640 1 --2- 172.21.15.18:0/4117519905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919d06750 0x564919d00a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f14e4640 1 -- 172.21.15.18:0/4117519905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919cfdb70 msgr2=0x564919d04040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f14e4640 1 --2- 172.21.15.18:0/4117519905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919cfdb70 0x564919d04040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f14e4640 1 -- 172.21.15.18:0/4117519905 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564919bffc20 con 0x564919cfd250 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52f14e4640 1 --2- 172.21.15.18:0/4117519905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfd250 0x564919d05ed0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f52e400bdd0 tx=0x7f52e400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f88ba9615025b99 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52d6ffd640 1 -- 172.21.15.18:0/4117519905 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4004220 con 0x564919cfd250 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52d6ffd640 1 -- 172.21.15.18:0/4117519905 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52e4022070 con 0x564919cfd250 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.224+0000 7f52d6ffd640 1 -- 172.21.15.18:0/4117519905 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4004e80 con 0x564919cfd250 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.225+0000 7f52f63f2740 1 -- 172.21.15.18:0/4117519905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfd250 msgr2=0x564919d05ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.225+0000 7f52f63f2740 1 --2- 172.21.15.18:0/4117519905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfd250 0x564919d05ed0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f52e400bdd0 tx=0x7f52e400b410 comp rx=0 tx=0).stop 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.225+0000 7f52f63f2740 1 -- 172.21.15.18:0/4117519905 shutdown_connections 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.225+0000 7f52f63f2740 1 --2- 172.21.15.18:0/4117519905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919d06750 0x564919d00a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.225+0000 7f52f63f2740 1 --2- 172.21.15.18:0/4117519905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919cfdb70 0x564919d04040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.225+0000 7f52f63f2740 1 --2- 172.21.15.18:0/4117519905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfd250 0x564919d05ed0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.225+0000 7f52f63f2740 1 -- 172.21.15.18:0/4117519905 >> 172.21.15.18:0/4117519905 conn(0x564919cf7980 msgr2=0x564919d02090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.225+0000 7f52f63f2740 1 -- 172.21.15.18:0/4117519905 shutdown_connections 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.225+0000 7f52f63f2740 1 -- 172.21.15.18:0/4117519905 wait complete. 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.225+0000 7f52f63f2740 1 -- start start 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.225+0000 7f52f63f2740 1 Processor -- start 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f63f2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 0x564919d0c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f63f2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d0c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f63f2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d0cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f63f2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564919d6ce10 con 0x564919cfdb70 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f63f2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564919d6cf50 con 0x564919d06750 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f63f2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564919d6d110 con 0x564919cfd250 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f63f2740 10 client.0 _ll_register_callbacks cb 0x7f52dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f0ce3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d0c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f0ce3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d0c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42952/0 (socket says 172.21.15.18:42952) 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f0ce3640 1 -- 172.21.15.18:0/943961486 learned_addr learned my addr 172.21.15.18:0/943961486 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f1ce5640 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d0cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f0ce3640 1 -- 172.21.15.18:0/943961486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 msgr2=0x564919d0c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f14e4640 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 0x564919d0c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 0x564919d0c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f0ce3640 1 -- 172.21.15.18:0/943961486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 msgr2=0x564919d0cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d0cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f0ce3640 1 -- 172.21.15.18:0/943961486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e400bb00 con 0x564919cfdb70 2024-02-17T12:54:37.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f1ce5640 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d0cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f0ce3640 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d0c630 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f52dc04dba0 tx=0x7f52dc02cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3672328dace20232 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52d4ff9640 1 -- 172.21.15.18:0/943961486 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc048040 con 0x564919cfdb70 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52d4ff9640 1 -- 172.21.15.18:0/943961486 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f52dc04d670 con 0x564919cfdb70 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.226+0000 7f52f63f2740 1 -- 172.21.15.18:0/943961486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52dc004410 con 0x564919cfdb70 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.227+0000 7f52d4ff9640 1 -- 172.21.15.18:0/943961486 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52dc003a40 con 0x564919cfdb70 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.227+0000 7f52f63f2740 10 client.17283 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.227+0000 7f52f63f2740 1 -- 172.21.15.18:0/943961486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564919c258c0 con 0x564919cfdb70 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.227+0000 7f52d4ff9640 1 -- 172.21.15.18:0/943961486 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52dc004410 con 0x564919cfdb70 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.227+0000 7f52d4ff9640 1 -- 172.21.15.18:0/943961486 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f52dc056450 con 0x564919cfdb70 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.227+0000 7f52f63f2740 10 client.17283 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.227+0000 7f52f63f2740 10 client.17283 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.227+0000 7f52f63f2740 20 client.17283 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.227+0000 7f52f63f2740 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x564919d6e630 0x564919d70a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.227+0000 7f52f63f2740 4 client.17283 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:54:37.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.227+0000 7f52f63f2740 1 -- 172.21.15.18:0/943961486 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5649199f0890 con 0x564919d6e630 2024-02-17T12:54:37.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.227+0000 7f52f14e4640 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x564919d6e630 0x564919d70a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52d4ff9640 10 client.17283 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:54:37.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f14e4640 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x564919d6e630 0x564919d70a50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86378b5a385fab3b server_cookie=1d231cece3f4030c in_seq=0 out_seq=0 2024-02-17T12:54:37.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52d4ff9640 1 -- 172.21.15.18:0/943961486 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5649199f0890 con 0x564919d6e630 2024-02-17T12:54:37.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52d4ff9640 10 client.17283 handle_command_reply: tid=0 2024-02-17T12:54:37.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f63f2740 1 client.17283 shutdown 2024-02-17T12:54:37.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f63f2740 1 -- 172.21.15.18:0/943961486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 msgr2=0x564919d0c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f63f2740 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d0c630 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f52dc04dba0 tx=0x7f52dc02cb60 comp rx=0 tx=0).stop 2024-02-17T12:54:37.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f63f2740 1 -- 172.21.15.18:0/943961486 shutdown_connections 2024-02-17T12:54:37.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f63f2740 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x564919d6e630 0x564919d70a50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f63f2740 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564919d06750 0x564919d0cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f63f2740 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564919cfdb70 0x564919d0c630 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52d4ff9640 0 client.17283 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:54:37.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f63f2740 1 --2- 172.21.15.18:0/943961486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564919cfd250 0x564919d0c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f63f2740 1 -- 172.21.15.18:0/943961486 >> 172.21.15.18:0/943961486 conn(0x564919cf7980 msgr2=0x564919cf62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f63f2740 1 -- 172.21.15.18:0/943961486 shutdown_connections 2024-02-17T12:54:37.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f63f2740 1 -- 172.21.15.18:0/943961486 wait complete. 2024-02-17T12:54:37.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.228+0000 7f52f63f2740 20 client.17283 trim_cache size 0 max 16384 2024-02-17T12:54:37.227 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:54:37.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.229+0000 7f52f3748640 1 -- 172.21.15.18:0/2537088145 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52d80a2230 msgr2=0x7f52d80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.229+0000 7f52f3748640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52d80a2230 0x7f52d80a46f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009930 tx=0x7f52dc03b000 comp rx=0 tx=0).stop 2024-02-17T12:54:37.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.229+0000 7f52f3748640 1 -- 172.21.15.18:0/2537088145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52ec1990c0 msgr2=0x7f52ec1a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.229+0000 7f52f3748640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52ec1990c0 0x7f52ec1a3670 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f52e800eb10 tx=0x7f52e800c2d0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.229+0000 7f52f3748640 1 -- 172.21.15.18:0/2537088145 shutdown_connections 2024-02-17T12:54:37.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.229+0000 7f52f3748640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52d80a2230 0x7f52d80a46f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.229+0000 7f52f3748640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52ec1990c0 0x7f52ec1a3670 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.229+0000 7f52f3748640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52ec105330 0x7f52ec198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.229+0000 7f52f3748640 1 --2- 172.21.15.18:0/2537088145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52ec100260 0x7f52ec1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.229+0000 7f52f3748640 1 -- 172.21.15.18:0/2537088145 >> 172.21.15.18:0/2537088145 conn(0x7f52ec077180 msgr2=0x7f52ec0fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.229+0000 7f52f3748640 1 -- 172.21.15.18:0/2537088145 shutdown_connections 2024-02-17T12:54:37.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.229+0000 7f52f3748640 1 -- 172.21.15.18:0/2537088145 wait complete. 2024-02-17T12:54:37.230 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:54:37.230 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:54:37.273 INFO:tasks.workunit.client.0.smithi018.stdout: 1 269692 7.90 MB/sec execute 112 sec latency 2556.798 ms 2024-02-17T12:54:37.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.344+0000 7ff119b7a640 1 -- 172.21.15.18:0/372365604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff114101380 msgr2=0x7ff114101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.344+0000 7ff119b7a640 1 --2- 172.21.15.18:0/372365604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff114101380 0x7ff114101760 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff104009930 tx=0x7ff10402bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:37.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.344+0000 7ff119b7a640 1 -- 172.21.15.18:0/372365604 shutdown_connections 2024-02-17T12:54:37.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.344+0000 7ff119b7a640 1 --2- 172.21.15.18:0/372365604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff114106450 0x7ff11410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.344+0000 7ff119b7a640 1 --2- 172.21.15.18:0/372365604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff114101d30 0x7ff114105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.344+0000 7ff119b7a640 1 --2- 172.21.15.18:0/372365604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff114101380 0x7ff114101760 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.344+0000 7ff119b7a640 1 -- 172.21.15.18:0/372365604 >> 172.21.15.18:0/372365604 conn(0x7ff114078580 msgr2=0x7ff114078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.344+0000 7ff119b7a640 1 -- 172.21.15.18:0/372365604 shutdown_connections 2024-02-17T12:54:37.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.344+0000 7ff119b7a640 1 -- 172.21.15.18:0/372365604 wait complete. 2024-02-17T12:54:37.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.344+0000 7ff119b7a640 1 Processor -- start 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff119b7a640 1 -- start start 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff119b7a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff114101380 0x7ff11419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff119b7a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff114106450 0x7ff11419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff119b7a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff114197650 0x7ff114197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff119b7a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff114198120 con 0x7ff114106450 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff119b7a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff114198290 con 0x7ff114101380 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff119b7a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff114198450 con 0x7ff114197650 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff112ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff114106450 0x7ff11419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff112ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff114106450 0x7ff11419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42974/0 (socket says 172.21.15.18:42974) 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff112ffd640 1 -- 172.21.15.18:0/3545788035 learned_addr learned my addr 172.21.15.18:0/3545788035 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff1137fe640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff114101380 0x7ff11419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff112ffd640 1 -- 172.21.15.18:0/3545788035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff114197650 msgr2=0x7ff114197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff113fff640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff114197650 0x7ff114197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff112ffd640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff114197650 0x7ff114197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff112ffd640 1 -- 172.21.15.18:0/3545788035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff114101380 msgr2=0x7ff11419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff112ffd640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff114101380 0x7ff11419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff112ffd640 1 -- 172.21.15.18:0/3545788035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff104009560 con 0x7ff114106450 2024-02-17T12:54:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff1137fe640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff114101380 0x7ff11419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.345+0000 7ff113fff640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff114197650 0x7ff114197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.346+0000 7ff112ffd640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff114106450 0x7ff11419dad0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff10800e9f0 tx=0x7ff10800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.346+0000 7ff110ff9640 1 -- 172.21.15.18:0/3545788035 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1080096c0 con 0x7ff114106450 2024-02-17T12:54:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.346+0000 7ff119b7a640 1 -- 172.21.15.18:0/3545788035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1141a43f0 con 0x7ff114106450 2024-02-17T12:54:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.346+0000 7ff110ff9640 1 -- 172.21.15.18:0/3545788035 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff108005500 con 0x7ff114106450 2024-02-17T12:54:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.346+0000 7ff110ff9640 1 -- 172.21.15.18:0/3545788035 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108010430 con 0x7ff114106450 2024-02-17T12:54:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.346+0000 7ff119b7a640 1 -- 172.21.15.18:0/3545788035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1141a4880 con 0x7ff114106450 2024-02-17T12:54:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.348+0000 7ff110ff9640 1 -- 172.21.15.18:0/3545788035 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff108005020 con 0x7ff114106450 2024-02-17T12:54:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.348+0000 7ff11c82f740 1 Processor -- start 2024-02-17T12:54:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.348+0000 7ff11c82f740 1 -- start start 2024-02-17T12:54:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.348+0000 7ff110ff9640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff0fc0a2230 0x7ff0fc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.348+0000 7ff11c82f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2250 0x5583fb3c2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.348+0000 7ff11c82f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.348+0000 7ff11c82f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.348+0000 7ff11c82f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fb37d960 con 0x5583fb3cb750 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.348+0000 7ff11c82f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fb331310 con 0x5583fb3c2250 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.348+0000 7ff11c82f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fb32e840 con 0x5583fb3c2b70 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.348+0000 7ff113fff640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff0fc0a2230 0x7ff0fc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff110ff9640 1 -- 172.21.15.18:0/3545788035 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff108014070 con 0x7ff114106450 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff113fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff113fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42988/0 (socket says 172.21.15.18:42988) 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff113fff640 1 -- 172.21.15.18:0/4045876798 learned_addr learned my addr 172.21.15.18:0/4045876798 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff1137fe640 1 --2- 172.21.15.18:0/4045876798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2250 0x5583fb3c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff112ffd640 1 --2- 172.21.15.18:0/4045876798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff113fff640 1 -- 172.21.15.18:0/4045876798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 msgr2=0x5583fb3cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff113fff640 1 --2- 172.21.15.18:0/4045876798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff113fff640 1 -- 172.21.15.18:0/4045876798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2250 msgr2=0x5583fb3c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff113fff640 1 --2- 172.21.15.18:0/4045876798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2250 0x5583fb3c2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff113fff640 1 -- 172.21.15.18:0/4045876798 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583fb2ec060 con 0x5583fb3cb750 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff113fff640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff0fc0a2230 0x7ff0fc0a46f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff10c005c40 tx=0x7ff10c009d70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff113fff640 1 --2- 172.21.15.18:0/4045876798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3cdb40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff10c011ce0 tx=0x7ff10c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f29bae4bce0ea3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff0f9ffb640 1 -- 172.21.15.18:0/4045876798 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0144f0 con 0x5583fb3cb750 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff0f9ffb640 1 -- 172.21.15.18:0/4045876798 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff10c01a040 con 0x5583fb3cb750 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.349+0000 7ff0f9ffb640 1 -- 172.21.15.18:0/4045876798 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0052f0 con 0x5583fb3cb750 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.350+0000 7ff11c82f740 1 -- 172.21.15.18:0/4045876798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 msgr2=0x5583fb3cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.350+0000 7ff11c82f740 1 --2- 172.21.15.18:0/4045876798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3cdb40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff10c011ce0 tx=0x7ff10c01b000 comp rx=0 tx=0).stop 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.350+0000 7ff11c82f740 1 -- 172.21.15.18:0/4045876798 shutdown_connections 2024-02-17T12:54:37.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.350+0000 7ff11c82f740 1 --2- 172.21.15.18:0/4045876798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3cdb40 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.350+0000 7ff11c82f740 1 --2- 172.21.15.18:0/4045876798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.350+0000 7ff11c82f740 1 --2- 172.21.15.18:0/4045876798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2250 0x5583fb3c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.350+0000 7ff11c82f740 1 -- 172.21.15.18:0/4045876798 >> 172.21.15.18:0/4045876798 conn(0x5583fb3bc980 msgr2=0x5583fb3bcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.350+0000 7ff11c82f740 1 -- 172.21.15.18:0/4045876798 shutdown_connections 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.350+0000 7ff11c82f740 1 -- 172.21.15.18:0/4045876798 wait complete. 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.350+0000 7ff11c82f740 1 -- start start 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.350+0000 7ff11c82f740 1 Processor -- start 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff11c82f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3c2250 0x5583fb3c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff11c82f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff11c82f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3cb750 0x5583fb3c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff11c82f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fb32a020 con 0x5583fb3c2250 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff11c82f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583faffd430 con 0x5583fb3cb750 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff11c82f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fafb9e00 con 0x5583fb3c2b70 2024-02-17T12:54:37.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff11c82f740 10 client.0 _ll_register_callbacks cb 0x5583fb2d1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff1137fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3c2250 0x5583fb3c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff1137fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3c2250 0x5583fb3c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43002/0 (socket says 172.21.15.18:43002) 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff1137fe640 1 -- 172.21.15.18:0/959786684 learned_addr learned my addr 172.21.15.18:0/959786684 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff113fff640 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3cb750 0x5583fb3c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff112ffd640 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff1137fe640 1 -- 172.21.15.18:0/959786684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 msgr2=0x5583fb3c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff1137fe640 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff1137fe640 1 -- 172.21.15.18:0/959786684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3cb750 msgr2=0x5583fb3c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff1137fe640 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3cb750 0x5583fb3c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff1137fe640 1 -- 172.21.15.18:0/959786684 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10c011aa0 con 0x5583fb3c2250 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff112ffd640 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff1137fe640 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3c2250 0x5583fb3c8760 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff104009900 tx=0x7ff10402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e69ea981ddf602 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff0f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:37.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff0f2ffd640 1 -- 172.21.15.18:0/959786684 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10402c5d0 con 0x5583fb3c2250 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff0f2ffd640 1 -- 172.21.15.18:0/959786684 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff104004250 con 0x5583fb3c2250 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.351+0000 7ff11c82f740 1 -- 172.21.15.18:0/959786684 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583fb0297e0 con 0x5583fb3c2250 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.352+0000 7ff0f2ffd640 1 -- 172.21.15.18:0/959786684 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff104004410 con 0x5583fb3c2250 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.352+0000 7ff11c82f740 10 client.17295 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.352+0000 7ff11c82f740 1 -- 172.21.15.18:0/959786684 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5583fb3286a0 con 0x5583fb3c2250 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.352+0000 7ff0f2ffd640 1 -- 172.21.15.18:0/959786684 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5583fb0297e0 con 0x5583fb3c2250 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.352+0000 7ff0f2ffd640 1 -- 172.21.15.18:0/959786684 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff10403c050 con 0x5583fb3c2250 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.352+0000 7ff11c82f740 10 client.17295 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.352+0000 7ff11c82f740 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.352+0000 7ff11c82f740 20 client.17295 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.352+0000 7ff11c82f740 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5583fb3dbbc0 0x5583fb3ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.352+0000 7ff11c82f740 4 client.17295 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.352+0000 7ff11c82f740 1 -- 172.21.15.18:0/959786684 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583fb066720 con 0x5583fb3dbbc0 2024-02-17T12:54:37.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.352+0000 7ff112ffd640 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5583fb3dbbc0 0x5583fb3ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff112ffd640 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5583fb3dbbc0 0x5583fb3ddfe0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85fc8f14783b2e68 server_cookie=3ac9b0fe33e0666 in_seq=0 out_seq=0 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff0f2ffd640 10 client.17295 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff0f2ffd640 1 -- 172.21.15.18:0/959786684 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5583fb066720 con 0x5583fb3dbbc0 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff0f2ffd640 10 client.17295 handle_command_reply: tid=0 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff11c82f740 1 client.17295 shutdown 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff11c82f740 1 -- 172.21.15.18:0/959786684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3c2250 msgr2=0x5583fb3c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff11c82f740 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3c2250 0x5583fb3c8760 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff104009900 tx=0x7ff10402ca10 comp rx=0 tx=0).stop 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff11c82f740 1 -- 172.21.15.18:0/959786684 shutdown_connections 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff11c82f740 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5583fb3dbbc0 0x5583fb3ddfe0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff11c82f740 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3cb750 0x5583fb3c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff11c82f740 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff11c82f740 1 --2- 172.21.15.18:0/959786684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3c2250 0x5583fb3c8760 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff0f2ffd640 0 client.17295 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:37.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.353+0000 7ff11c82f740 1 -- 172.21.15.18:0/959786684 >> 172.21.15.18:0/959786684 conn(0x5583fb3bc980 msgr2=0x5583fb3be6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.354+0000 7ff11c82f740 1 -- 172.21.15.18:0/959786684 shutdown_connections 2024-02-17T12:54:37.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.354+0000 7ff11c82f740 1 -- 172.21.15.18:0/959786684 wait complete. 2024-02-17T12:54:37.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.354+0000 7ff11c82f740 20 client.17295 trim_cache size 0 max 16384 2024-02-17T12:54:37.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.365+0000 7ff11c82f740 1 Processor -- start 2024-02-17T12:54:37.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.365+0000 7ff11c82f740 1 -- start start 2024-02-17T12:54:37.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.365+0000 7ff11c82f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2250 0x5583fb3cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.365+0000 7ff11c82f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.365+0000 7ff11c82f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.365+0000 7ff11c82f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fb2af130 con 0x5583fb3cb750 2024-02-17T12:54:37.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.365+0000 7ff11c82f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fb3aea40 con 0x5583fb3c2250 2024-02-17T12:54:37.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.365+0000 7ff11c82f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fb3d86b0 con 0x5583fb3c2b70 2024-02-17T12:54:37.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.365+0000 7ff113fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff113fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3c59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43016/0 (socket says 172.21.15.18:43016) 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff113fff640 1 -- 172.21.15.18:0/2246846258 learned_addr learned my addr 172.21.15.18:0/2246846258 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff1137fe640 1 --2- 172.21.15.18:0/2246846258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2250 0x5583fb3cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff112ffd640 1 --2- 172.21.15.18:0/2246846258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff113fff640 1 -- 172.21.15.18:0/2246846258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 msgr2=0x5583fb3c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff113fff640 1 --2- 172.21.15.18:0/2246846258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff113fff640 1 -- 172.21.15.18:0/2246846258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2250 msgr2=0x5583fb3cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff113fff640 1 --2- 172.21.15.18:0/2246846258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2250 0x5583fb3cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff113fff640 1 -- 172.21.15.18:0/2246846258 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583fb2c4c20 con 0x5583fb3cb750 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff1137fe640 1 --2- 172.21.15.18:0/2246846258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2250 0x5583fb3cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff112ffd640 1 --2- 172.21.15.18:0/2246846258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff113fff640 1 --2- 172.21.15.18:0/2246846258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3c59e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff10c0094d0 tx=0x7ff10c014370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b733baf26f0186 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff0f97fa640 1 -- 172.21.15.18:0/2246846258 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0144f0 con 0x5583fb3cb750 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff0f97fa640 1 -- 172.21.15.18:0/2246846258 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff10c011dd0 con 0x5583fb3cb750 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.366+0000 7ff0f97fa640 1 -- 172.21.15.18:0/2246846258 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c005330 con 0x5583fb3cb750 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 -- 172.21.15.18:0/2246846258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 msgr2=0x5583fb3c59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 --2- 172.21.15.18:0/2246846258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3c59e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff10c0094d0 tx=0x7ff10c014370 comp rx=0 tx=0).stop 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 -- 172.21.15.18:0/2246846258 shutdown_connections 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 --2- 172.21.15.18:0/2246846258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3c59e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 --2- 172.21.15.18:0/2246846258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2b70 0x5583fb3c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 --2- 172.21.15.18:0/2246846258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2250 0x5583fb3cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 -- 172.21.15.18:0/2246846258 >> 172.21.15.18:0/2246846258 conn(0x5583fb3bc980 msgr2=0x5583fb3c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 -- 172.21.15.18:0/2246846258 shutdown_connections 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 -- 172.21.15.18:0/2246846258 wait complete. 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 -- start start 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 Processor -- start 2024-02-17T12:54:37.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2250 0x5583fb3d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2b70 0x5583fb3d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fb3d1af0 con 0x5583fb3cb750 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fb3d1c60 con 0x5583fb3c2b70 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.367+0000 7ff11c82f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583fb432140 con 0x5583fb3c2250 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff11c82f740 10 client.0 _ll_register_callbacks cb 0x7ff10c014670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff113fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff113fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43026/0 (socket says 172.21.15.18:43026) 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff113fff640 1 -- 172.21.15.18:0/2550317466 learned_addr learned my addr 172.21.15.18:0/2550317466 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff112ffd640 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2b70 0x5583fb3d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff1137fe640 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2250 0x5583fb3d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff113fff640 1 -- 172.21.15.18:0/2550317466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2250 msgr2=0x5583fb3d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff113fff640 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2250 0x5583fb3d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff113fff640 1 -- 172.21.15.18:0/2550317466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2b70 msgr2=0x5583fb3d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff113fff640 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2b70 0x5583fb3d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff113fff640 1 -- 172.21.15.18:0/2550317466 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583fb2ec060 con 0x5583fb3cb750 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff1137fe640 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2250 0x5583fb3d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff113fff640 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3d1420 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff10c031040 tx=0x7ff10c0142e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df1eca5f407ba5a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff0f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff0f2ffd640 1 -- 172.21.15.18:0/2550317466 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c004bd0 con 0x5583fb3cb750 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff11c82f740 1 -- 172.21.15.18:0/2550317466 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff10c01a040 con 0x5583fb3cb750 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff0f2ffd640 1 -- 172.21.15.18:0/2550317466 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff10c004d30 con 0x5583fb3cb750 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff0f2ffd640 1 -- 172.21.15.18:0/2550317466 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff10c0050b0 con 0x5583fb3cb750 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff11c82f740 10 client.17301 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.368+0000 7ff11c82f740 1 -- 172.21.15.18:0/2550317466 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5583fb2ea8c0 con 0x5583fb3cb750 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.369+0000 7ff0f2ffd640 1 -- 172.21.15.18:0/2550317466 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff10c01a040 con 0x5583fb3cb750 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.369+0000 7ff0f2ffd640 1 -- 172.21.15.18:0/2550317466 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff10c037070 con 0x5583fb3cb750 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.369+0000 7ff11c82f740 10 client.17301 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:37.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.369+0000 7ff11c82f740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.369+0000 7ff11c82f740 20 client.17301 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.369+0000 7ff11c82f740 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5583fb433620 0x5583fb435a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.369+0000 7ff11c82f740 4 client.17301 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.369+0000 7ff11c82f740 1 -- 172.21.15.18:0/2550317466 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5583fb0b5890 con 0x5583fb433620 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.369+0000 7ff1137fe640 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5583fb433620 0x5583fb435a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.369+0000 7ff1137fe640 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5583fb433620 0x5583fb435a40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c121a136093a9234 server_cookie=f0b4ea90a857884f in_seq=0 out_seq=0 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.369+0000 7ff0f2ffd640 10 client.17301 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff0f2ffd640 1 -- 172.21.15.18:0/2550317466 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5583fb0b5890 con 0x5583fb433620 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff0f2ffd640 10 client.17301 handle_command_reply: tid=0 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff11c82f740 1 client.17301 shutdown 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff11c82f740 1 -- 172.21.15.18:0/2550317466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 msgr2=0x5583fb3d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff11c82f740 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3d1420 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff10c031040 tx=0x7ff10c0142e0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff11c82f740 1 -- 172.21.15.18:0/2550317466 shutdown_connections 2024-02-17T12:54:37.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff11c82f740 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5583fb433620 0x5583fb435a40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff11c82f740 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583fb3cb750 0x5583fb3d1420 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff11c82f740 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583fb3c2b70 0x5583fb3d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff0f2ffd640 0 client.17301 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:37.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff11c82f740 1 --2- 172.21.15.18:0/2550317466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5583fb3c2250 0x5583fb3d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff11c82f740 1 -- 172.21.15.18:0/2550317466 >> 172.21.15.18:0/2550317466 conn(0x5583fb3bc980 msgr2=0x5583fb3bddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff11c82f740 1 -- 172.21.15.18:0/2550317466 shutdown_connections 2024-02-17T12:54:37.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff11c82f740 1 -- 172.21.15.18:0/2550317466 wait complete. 2024-02-17T12:54:37.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff11c82f740 20 client.17301 trim_cache size 0 max 16384 2024-02-17T12:54:37.368 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:54:37.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff119b7a640 1 -- 172.21.15.18:0/3545788035 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff0fc0a2230 msgr2=0x7ff0fc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff119b7a640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff0fc0a2230 0x7ff0fc0a46f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff10c005c40 tx=0x7ff10c009d70 comp rx=0 tx=0).stop 2024-02-17T12:54:37.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff119b7a640 1 -- 172.21.15.18:0/3545788035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff114106450 msgr2=0x7ff11419dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:37.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff119b7a640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff114106450 0x7ff11419dad0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff10800e9f0 tx=0x7ff10800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff119b7a640 1 -- 172.21.15.18:0/3545788035 shutdown_connections 2024-02-17T12:54:37.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff119b7a640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff0fc0a2230 0x7ff0fc0a46f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff119b7a640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff114197650 0x7ff114197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff119b7a640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff114106450 0x7ff11419dad0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff119b7a640 1 --2- 172.21.15.18:0/3545788035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff114101380 0x7ff11419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:37.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.370+0000 7ff119b7a640 1 -- 172.21.15.18:0/3545788035 >> 172.21.15.18:0/3545788035 conn(0x7ff114078580 msgr2=0x7ff1140ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:37.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.371+0000 7ff119b7a640 1 -- 172.21.15.18:0/3545788035 shutdown_connections 2024-02-17T12:54:37.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:37.371+0000 7ff119b7a640 1 -- 172.21.15.18:0/3545788035 wait complete. 2024-02-17T12:54:37.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:37 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1358854877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:54:37.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:37 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1358854877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:54:37.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:37 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1358854877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:54:38.272 INFO:tasks.workunit.client.0.smithi018.stdout: 1 269743 7.83 MB/sec execute 113 sec latency 3133.035 ms 2024-02-17T12:54:38.372 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:54:38.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:54:38.373 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:54:38.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.502+0000 7fb2f358a640 1 -- 172.21.15.18:0/3187163952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ec105330 msgr2=0x7fb2ec10cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.502+0000 7fb2f358a640 1 --2- 172.21.15.18:0/3187163952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ec105330 0x7fb2ec10cf00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009cd0 tx=0x7fb2e402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.502+0000 7fb2f358a640 1 -- 172.21.15.18:0/3187163952 shutdown_connections 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.502+0000 7fb2f358a640 1 --2- 172.21.15.18:0/3187163952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ec105330 0x7fb2ec10cf00 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.502+0000 7fb2f358a640 1 --2- 172.21.15.18:0/3187163952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2ec100c10 0x7fb2ec104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.502+0000 7fb2f358a640 1 --2- 172.21.15.18:0/3187163952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2ec100260 0x7fb2ec100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.502+0000 7fb2f358a640 1 -- 172.21.15.18:0/3187163952 >> 172.21.15.18:0/3187163952 conn(0x7fb2ec077180 msgr2=0x7fb2ec077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.502+0000 7fb2f358a640 1 -- 172.21.15.18:0/3187163952 shutdown_connections 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.502+0000 7fb2f358a640 1 -- 172.21.15.18:0/3187163952 wait complete. 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.502+0000 7fb2f358a640 1 Processor -- start 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f358a640 1 -- start start 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f358a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2ec100260 0x7fb2ec198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f358a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ec105330 0x7fb2ec198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f358a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2ec198ff0 0x7fb2ec1a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f358a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec1a3c70 con 0x7fb2ec105330 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f358a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec1a3de0 con 0x7fb2ec198ff0 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f358a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec1a3fa0 con 0x7fb2ec100260 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f0b25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ec105330 0x7fb2ec198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f0b25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ec105330 0x7fb2ec198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43044/0 (socket says 172.21.15.18:43044) 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f0b25640 1 -- 172.21.15.18:0/3328853413 learned_addr learned my addr 172.21.15.18:0/3328853413 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f1b27640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2ec198ff0 0x7fb2ec1a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f1326640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2ec100260 0x7fb2ec198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f0b25640 1 -- 172.21.15.18:0/3328853413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2ec100260 msgr2=0x7fb2ec198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2ec100260 0x7fb2ec198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f0b25640 1 -- 172.21.15.18:0/3328853413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2ec198ff0 msgr2=0x7fb2ec1a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2ec198ff0 0x7fb2ec1a35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f0b25640 1 -- 172.21.15.18:0/3328853413 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e4009900 con 0x7fb2ec105330 2024-02-17T12:54:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f1326640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2ec100260 0x7fb2ec198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:38.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.503+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ec105330 0x7fb2ec198a60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00d740 tx=0x7fb2dc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:38.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.504+0000 7fb2e27fc640 1 -- 172.21.15.18:0/3328853413 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc009c30 con 0x7fb2ec105330 2024-02-17T12:54:38.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.504+0000 7fb2e27fc640 1 -- 172.21.15.18:0/3328853413 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2dc00de50 con 0x7fb2ec105330 2024-02-17T12:54:38.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.504+0000 7fb2f358a640 1 -- 172.21.15.18:0/3328853413 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec1a4280 con 0x7fb2ec105330 2024-02-17T12:54:38.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.504+0000 7fb2e27fc640 1 -- 172.21.15.18:0/3328853413 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc010460 con 0x7fb2ec105330 2024-02-17T12:54:38.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.504+0000 7fb2f358a640 1 -- 172.21.15.18:0/3328853413 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec1a4740 con 0x7fb2ec105330 2024-02-17T12:54:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.506+0000 7fb2e27fc640 1 -- 172.21.15.18:0/3328853413 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb2dc004f10 con 0x7fb2ec105330 2024-02-17T12:54:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.506+0000 7fb2f6235740 1 Processor -- start 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.506+0000 7fb2f6235740 1 -- start start 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.506+0000 7fb2e27fc640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb2d80a2230 0x7fb2d80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.506+0000 7fb2f6235740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261380 0x563a03261760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f6235740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261d30 0x563a0326a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f6235740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 0x563a0326cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f6235740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a031cff90 con 0x563a0326a8b0 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f6235740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a031cd4c0 con 0x563a03261d30 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f6235740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a031c8ca0 con 0x563a03261380 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f1b27640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb2d80a2230 0x7fb2d80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2e27fc640 1 -- 172.21.15.18:0/3328853413 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb2dc0cd9c0 con 0x7fb2ec105330 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f1b27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 0x563a0326cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f1326640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261380 0x563a03261760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f1b27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 0x563a0326cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43056/0 (socket says 172.21.15.18:43056) 2024-02-17T12:54:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f1b27640 1 -- 172.21.15.18:0/1863588526 learned_addr learned my addr 172.21.15.18:0/1863588526 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/1863588526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261d30 0x563a0326a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f0b25640 1 -- 172.21.15.18:0/1863588526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261380 msgr2=0x563a03261760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/1863588526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261380 0x563a03261760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f0b25640 1 -- 172.21.15.18:0/1863588526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 msgr2=0x563a0326cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/1863588526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 0x563a0326cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f0b25640 1 -- 172.21.15.18:0/1863588526 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a0318b0b0 con 0x563a03261d30 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f1b27640 1 --2- 172.21.15.18:0/1863588526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 0x563a0326cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f1b27640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb2d80a2230 0x7fb2d80a46f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4032650 tx=0x7fb2e4032340 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.507+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/1863588526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261d30 0x563a0326a2e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0cb160 tx=0x7fb2dc099e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d7ba4f6ab2e083 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.508+0000 7fb2d77fe640 1 -- 172.21.15.18:0/1863588526 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0636b0 con 0x563a03261d30 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.508+0000 7fb2d77fe640 1 -- 172.21.15.18:0/1863588526 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2dc0046c0 con 0x563a03261d30 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.508+0000 7fb2d77fe640 1 -- 172.21.15.18:0/1863588526 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc009d90 con 0x563a03261d30 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.508+0000 7fb2f6235740 1 -- 172.21.15.18:0/1863588526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261d30 msgr2=0x563a0326a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.508+0000 7fb2f6235740 1 --2- 172.21.15.18:0/1863588526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261d30 0x563a0326a2e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0cb160 tx=0x7fb2dc099e20 comp rx=0 tx=0).stop 2024-02-17T12:54:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.508+0000 7fb2f6235740 1 -- 172.21.15.18:0/1863588526 shutdown_connections 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.508+0000 7fb2f6235740 1 --2- 172.21.15.18:0/1863588526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 0x563a0326cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.508+0000 7fb2f6235740 1 --2- 172.21.15.18:0/1863588526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261d30 0x563a0326a2e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.508+0000 7fb2f6235740 1 --2- 172.21.15.18:0/1863588526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261380 0x563a03261760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.508+0000 7fb2f6235740 1 -- 172.21.15.18:0/1863588526 >> 172.21.15.18:0/1863588526 conn(0x563a0325b840 msgr2=0x563a0325bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.508+0000 7fb2f6235740 1 -- 172.21.15.18:0/1863588526 shutdown_connections 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f6235740 1 -- 172.21.15.18:0/1863588526 wait complete. 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f6235740 1 -- start start 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f6235740 1 Processor -- start 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f6235740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261380 0x563a03267870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f6235740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261d30 0x563a03267db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f6235740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a0326a8b0 0x563a032682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f6235740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0316df40 con 0x563a03261380 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f6235740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a02e9c4b0 con 0x563a0326a8b0 2024-02-17T12:54:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f6235740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0314e670 con 0x563a03261d30 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f6235740 10 client.0 _ll_register_callbacks cb 0x563a031bbf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f1326640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261380 0x563a03267870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f1326640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261380 0x563a03267870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43068/0 (socket says 172.21.15.18:43068) 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f1326640 1 -- 172.21.15.18:0/4167548146 learned_addr learned my addr 172.21.15.18:0/4167548146 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.509+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261d30 0x563a03267db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2f1b27640 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a0326a8b0 0x563a032682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2f1326640 1 -- 172.21.15.18:0/4167548146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261d30 msgr2=0x563a03267db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2f1326640 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261d30 0x563a03267db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2f1326640 1 -- 172.21.15.18:0/4167548146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a0326a8b0 msgr2=0x563a032682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2f1326640 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a0326a8b0 0x563a032682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2f1326640 1 -- 172.21.15.18:0/4167548146 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec1a4740 con 0x563a03261380 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2f1326640 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261380 0x563a03267870 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8011a60 tx=0x7fb2e8011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad8a5c05f75c1f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2d4ff9640 1 -- 172.21.15.18:0/4167548146 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8004460 con 0x563a03261380 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2d4ff9640 1 -- 172.21.15.18:0/4167548146 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb2e800f9a0 con 0x563a03261380 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2f6235740 1 -- 172.21.15.18:0/4167548146 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a031a2f40 con 0x563a03261380 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2d4ff9640 1 -- 172.21.15.18:0/4167548146 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2e800fb60 con 0x563a03261380 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2f6235740 10 client.17313 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2f6235740 1 -- 172.21.15.18:0/4167548146 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563a031c7320 con 0x563a03261380 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.510+0000 7fb2d4ff9640 1 -- 172.21.15.18:0/4167548146 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563a031a2f40 con 0x563a03261380 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.511+0000 7fb2d4ff9640 1 -- 172.21.15.18:0/4167548146 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb2e80195b0 con 0x563a03261380 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.511+0000 7fb2f6235740 10 client.17313 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.511+0000 7fb2f6235740 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.511+0000 7fb2f6235740 20 client.17313 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.511+0000 7fb2f6235740 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563a0327aee0 0x563a0327d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.511+0000 7fb2f6235740 4 client.17313 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.511+0000 7fb2f6235740 1 -- 172.21.15.18:0/4167548146 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a031bf310 con 0x563a0327aee0 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.511+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563a0327aee0 0x563a0327d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.511+0000 7fb2d4ff9640 10 client.17313 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.511+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563a0327aee0 0x563a0327d300 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fa1fcfd09493be1 server_cookie=fe12b8c305f9081e in_seq=0 out_seq=0 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2d4ff9640 1 -- 172.21.15.18:0/4167548146 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a031bf310 con 0x563a0327aee0 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2d4ff9640 10 client.17313 handle_command_reply: tid=0 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2f6235740 1 client.17313 shutdown 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2f6235740 1 -- 172.21.15.18:0/4167548146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261380 msgr2=0x563a03267870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2f6235740 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261380 0x563a03267870 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8011a60 tx=0x7fb2e8011f20 comp rx=0 tx=0).stop 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2f6235740 1 -- 172.21.15.18:0/4167548146 shutdown_connections 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2f6235740 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563a0327aee0 0x563a0327d300 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2f6235740 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a0326a8b0 0x563a032682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2f6235740 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261d30 0x563a03267db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2f6235740 1 --2- 172.21.15.18:0/4167548146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261380 0x563a03267870 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2d4ff9640 0 client.17313 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2f6235740 1 -- 172.21.15.18:0/4167548146 >> 172.21.15.18:0/4167548146 conn(0x563a0325b840 msgr2=0x563a0325d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2f6235740 1 -- 172.21.15.18:0/4167548146 shutdown_connections 2024-02-17T12:54:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2f6235740 1 -- 172.21.15.18:0/4167548146 wait complete. 2024-02-17T12:54:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.512+0000 7fb2f6235740 20 client.17313 trim_cache size 0 max 16384 2024-02-17T12:54:38.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.528+0000 7fb2f6235740 1 Processor -- start 2024-02-17T12:54:38.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.528+0000 7fb2f6235740 1 -- start start 2024-02-17T12:54:38.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.528+0000 7fb2f6235740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261380 0x563a03268110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.528+0000 7fb2f6235740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261d30 0x563a03271880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.528+0000 7fb2f6235740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a0326a8b0 0x563a03271dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.528+0000 7fb2f6235740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a03268bc0 con 0x563a03261d30 2024-02-17T12:54:38.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.528+0000 7fb2f6235740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0326d180 con 0x563a03261380 2024-02-17T12:54:38.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.528+0000 7fb2f6235740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a03265dc0 con 0x563a0326a8b0 2024-02-17T12:54:38.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.528+0000 7fb2f0b25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261d30 0x563a03271880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.528+0000 7fb2f0b25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261d30 0x563a03271880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43082/0 (socket says 172.21.15.18:43082) 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.528+0000 7fb2f0b25640 1 -- 172.21.15.18:0/3363973897 learned_addr learned my addr 172.21.15.18:0/3363973897 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.529+0000 7fb2f1b27640 1 --2- 172.21.15.18:0/3363973897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a0326a8b0 0x563a03271dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.529+0000 7fb2f1326640 1 --2- 172.21.15.18:0/3363973897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261380 0x563a03268110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.529+0000 7fb2f0b25640 1 -- 172.21.15.18:0/3363973897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a0326a8b0 msgr2=0x563a03271dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.529+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/3363973897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a0326a8b0 0x563a03271dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.529+0000 7fb2f0b25640 1 -- 172.21.15.18:0/3363973897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261380 msgr2=0x563a03268110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.529+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/3363973897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261380 0x563a03268110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.529+0000 7fb2f0b25640 1 -- 172.21.15.18:0/3363973897 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a03163c00 con 0x563a03261d30 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.529+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/3363973897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261d30 0x563a03271880 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009610 tx=0x7fb2dc099e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64df136dcf6e55dd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.529+0000 7fb2d6ffd640 1 -- 172.21.15.18:0/3363973897 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0046c0 con 0x563a03261d30 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.529+0000 7fb2d6ffd640 1 -- 172.21.15.18:0/3363973897 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2dc09ab00 con 0x563a03261d30 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.529+0000 7fb2d6ffd640 1 -- 172.21.15.18:0/3363973897 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc098570 con 0x563a03261d30 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.530+0000 7fb2f6235740 1 -- 172.21.15.18:0/3363973897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261d30 msgr2=0x563a03271880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.530+0000 7fb2f6235740 1 --2- 172.21.15.18:0/3363973897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261d30 0x563a03271880 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009610 tx=0x7fb2dc099e20 comp rx=0 tx=0).stop 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.530+0000 7fb2f6235740 1 -- 172.21.15.18:0/3363973897 shutdown_connections 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.530+0000 7fb2f6235740 1 --2- 172.21.15.18:0/3363973897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a0326a8b0 0x563a03271dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.530+0000 7fb2f6235740 1 --2- 172.21.15.18:0/3363973897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a03261d30 0x563a03271880 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.530+0000 7fb2f6235740 1 --2- 172.21.15.18:0/3363973897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261380 0x563a03268110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.530+0000 7fb2f6235740 1 -- 172.21.15.18:0/3363973897 >> 172.21.15.18:0/3363973897 conn(0x563a0325b840 msgr2=0x563a0325a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.530+0000 7fb2f6235740 1 -- 172.21.15.18:0/3363973897 shutdown_connections 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.530+0000 7fb2f6235740 1 -- 172.21.15.18:0/3363973897 wait complete. 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.530+0000 7fb2f6235740 1 -- start start 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.530+0000 7fb2f6235740 1 Processor -- start 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f6235740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261380 0x563a03270c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f6235740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261d30 0x563a03271170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f6235740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 0x563a032cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f6235740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a032cba10 con 0x563a0326a8b0 2024-02-17T12:54:38.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f6235740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a032cbb50 con 0x563a03261380 2024-02-17T12:54:38.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f6235740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a032cbd10 con 0x563a03261d30 2024-02-17T12:54:38.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f6235740 10 client.0 _ll_register_callbacks cb 0x7fb2dc0966a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:38.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f1b27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 0x563a032cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f1b27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 0x563a032cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43086/0 (socket says 172.21.15.18:43086) 2024-02-17T12:54:38.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f1b27640 1 -- 172.21.15.18:0/592587879 learned_addr learned my addr 172.21.15.18:0/592587879 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:38.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f0b25640 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261d30 0x563a03271170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f1326640 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261380 0x563a03270c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f1b27640 1 -- 172.21.15.18:0/592587879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261d30 msgr2=0x563a03271170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f1b27640 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261d30 0x563a03271170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f1b27640 1 -- 172.21.15.18:0/592587879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261380 msgr2=0x563a03270c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f1b27640 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261380 0x563a03270c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f1b27640 1 -- 172.21.15.18:0/592587879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a0318b0b0 con 0x563a0326a8b0 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2f1b27640 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 0x563a032cb4d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb2e402c700 tx=0x7fb2e4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bb4eaac368a07c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.531+0000 7fb2d4ff9640 1 -- 172.21.15.18:0/592587879 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4048070 con 0x563a0326a8b0 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2d4ff9640 1 -- 172.21.15.18:0/592587879 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb2e4003b80 con 0x563a0326a8b0 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2f6235740 1 -- 172.21.15.18:0/592587879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a031ee900 con 0x563a0326a8b0 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2d4ff9640 1 -- 172.21.15.18:0/592587879 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2e4032b60 con 0x563a0326a8b0 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2f6235740 10 client.17319 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2f6235740 1 -- 172.21.15.18:0/592587879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563a03189910 con 0x563a0326a8b0 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2d4ff9640 1 -- 172.21.15.18:0/592587879 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563a031ee900 con 0x563a0326a8b0 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2d4ff9640 1 -- 172.21.15.18:0/592587879 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb2e4005190 con 0x563a0326a8b0 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2f6235740 10 client.17319 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:38.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2f6235740 10 client.17319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:38.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2f6235740 20 client.17319 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:38.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2f6235740 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563a032d3610 0x563a032d5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2f6235740 4 client.17319 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:54:38.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2f6235740 1 -- 172.21.15.18:0/592587879 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563a02ec8860 con 0x563a032d3610 2024-02-17T12:54:38.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.532+0000 7fb2f1326640 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563a032d3610 0x563a032d5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.533+0000 7fb2d4ff9640 10 client.17319 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:38.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.533+0000 7fb2f1326640 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563a032d3610 0x563a032d5a30 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f495dcd6e357a82b server_cookie=79537864c8be40e4 in_seq=0 out_seq=0 2024-02-17T12:54:38.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2d4ff9640 1 -- 172.21.15.18:0/592587879 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563a02ec8860 con 0x563a032d3610 2024-02-17T12:54:38.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2d4ff9640 10 client.17319 handle_command_reply: tid=0 2024-02-17T12:54:38.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2f6235740 1 client.17319 shutdown 2024-02-17T12:54:38.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2f6235740 1 -- 172.21.15.18:0/592587879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 msgr2=0x563a032cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2f6235740 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 0x563a032cb4d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb2e402c700 tx=0x7fb2e4005c50 comp rx=0 tx=0).stop 2024-02-17T12:54:38.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2f6235740 1 -- 172.21.15.18:0/592587879 shutdown_connections 2024-02-17T12:54:38.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2f6235740 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563a032d3610 0x563a032d5a30 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2f6235740 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a0326a8b0 0x563a032cb4d0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2f6235740 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563a03261d30 0x563a03271170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2d4ff9640 0 client.17319 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:38.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2f6235740 1 --2- 172.21.15.18:0/592587879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a03261380 0x563a03270c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2f6235740 1 -- 172.21.15.18:0/592587879 >> 172.21.15.18:0/592587879 conn(0x563a0325b840 msgr2=0x563a0325a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:38.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2f6235740 1 -- 172.21.15.18:0/592587879 shutdown_connections 2024-02-17T12:54:38.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2f6235740 1 -- 172.21.15.18:0/592587879 wait complete. 2024-02-17T12:54:38.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.534+0000 7fb2f6235740 20 client.17319 trim_cache size 0 max 16384 2024-02-17T12:54:38.531 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:54:38.532 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:54:38.532 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "4d044d9b-7b89-4337-a5f8-acc800413929", 2024-02-17T12:54:38.532 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:54:38.532 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:54:38.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.535+0000 7fb2f358a640 1 -- 172.21.15.18:0/3328853413 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb2d80a2230 msgr2=0x7fb2d80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.535+0000 7fb2f358a640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb2d80a2230 0x7fb2d80a46f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4032650 tx=0x7fb2e4032340 comp rx=0 tx=0).stop 2024-02-17T12:54:38.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.535+0000 7fb2f358a640 1 -- 172.21.15.18:0/3328853413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ec105330 msgr2=0x7fb2ec198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.535+0000 7fb2f358a640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ec105330 0x7fb2ec198a60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00d740 tx=0x7fb2dc00dc00 comp rx=0 tx=0).stop 2024-02-17T12:54:38.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.535+0000 7fb2f358a640 1 -- 172.21.15.18:0/3328853413 shutdown_connections 2024-02-17T12:54:38.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.535+0000 7fb2f358a640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb2d80a2230 0x7fb2d80a46f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.535+0000 7fb2f358a640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2ec198ff0 0x7fb2ec1a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.535+0000 7fb2f358a640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ec105330 0x7fb2ec198a60 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.535+0000 7fb2f358a640 1 --2- 172.21.15.18:0/3328853413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2ec100260 0x7fb2ec198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.535+0000 7fb2f358a640 1 -- 172.21.15.18:0/3328853413 >> 172.21.15.18:0/3328853413 conn(0x7fb2ec077180 msgr2=0x7fb2ec0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:38.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.535+0000 7fb2f358a640 1 -- 172.21.15.18:0/3328853413 shutdown_connections 2024-02-17T12:54:38.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.535+0000 7fb2f358a640 1 -- 172.21.15.18:0/3328853413 wait complete. 2024-02-17T12:54:38.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:54:38.538 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:54:38.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.653+0000 7fa11334b640 1 -- 172.21.15.18:0/4064664361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c100c20 msgr2=0x7fa10c104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.653+0000 7fa11334b640 1 --2- 172.21.15.18:0/4064664361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c100c20 0x7fa10c104c60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa100009930 tx=0x7fa10002bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:38.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.653+0000 7fa11334b640 1 -- 172.21.15.18:0/4064664361 shutdown_connections 2024-02-17T12:54:38.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.653+0000 7fa11334b640 1 --2- 172.21.15.18:0/4064664361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa10c105340 0x7fa10c10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.653+0000 7fa11334b640 1 --2- 172.21.15.18:0/4064664361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c100c20 0x7fa10c104c60 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.653+0000 7fa11334b640 1 --2- 172.21.15.18:0/4064664361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa10c100270 0x7fa10c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.653+0000 7fa11334b640 1 -- 172.21.15.18:0/4064664361 >> 172.21.15.18:0/4064664361 conn(0x7fa10c077230 msgr2=0x7fa10c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:38.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.653+0000 7fa11334b640 1 -- 172.21.15.18:0/4064664361 shutdown_connections 2024-02-17T12:54:38.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.653+0000 7fa11334b640 1 -- 172.21.15.18:0/4064664361 wait complete. 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.653+0000 7fa11334b640 1 Processor -- start 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.653+0000 7fa11334b640 1 -- start start 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa11334b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c100270 0x7fa10c19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa11334b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa10c105340 0x7fa10c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa11334b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa10c197610 0x7fa10c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa11334b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c198000 con 0x7fa10c100270 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa11334b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c198170 con 0x7fa10c197610 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa11334b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c198330 con 0x7fa10c105340 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa1110e7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c100270 0x7fa10c19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa1110e7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c100270 0x7fa10c19d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43098/0 (socket says 172.21.15.18:43098) 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa1110e7640 1 -- 172.21.15.18:0/2186973079 learned_addr learned my addr 172.21.15.18:0/2186973079 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa1118e8640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa10c197610 0x7fa10c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa1108e6640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa10c105340 0x7fa10c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa1110e7640 1 -- 172.21.15.18:0/2186973079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa10c105340 msgr2=0x7fa10c19d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa1110e7640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa10c105340 0x7fa10c19d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa1110e7640 1 -- 172.21.15.18:0/2186973079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa10c197610 msgr2=0x7fa10c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa1110e7640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa10c197610 0x7fa10c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa1110e7640 1 -- 172.21.15.18:0/2186973079 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa100009560 con 0x7fa10c100270 2024-02-17T12:54:38.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa1110e7640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c100270 0x7fa10c19d410 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e9f0 tx=0x7fa10400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:38.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa10a7fc640 1 -- 172.21.15.18:0/2186973079 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104009720 con 0x7fa10c100270 2024-02-17T12:54:38.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa11334b640 1 -- 172.21.15.18:0/2186973079 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10c1a42d0 con 0x7fa10c100270 2024-02-17T12:54:38.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa10a7fc640 1 -- 172.21.15.18:0/2186973079 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa10400cd80 con 0x7fa10c100270 2024-02-17T12:54:38.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa10a7fc640 1 -- 172.21.15.18:0/2186973079 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104010690 con 0x7fa10c100270 2024-02-17T12:54:38.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.654+0000 7fa11334b640 1 -- 172.21.15.18:0/2186973079 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c1a47b0 con 0x7fa10c100270 2024-02-17T12:54:38.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.657+0000 7fa10a7fc640 1 -- 172.21.15.18:0/2186973079 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa1040046c0 con 0x7fa10c100270 2024-02-17T12:54:38.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.657+0000 7fa115ff5740 1 Processor -- start 2024-02-17T12:54:38.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.657+0000 7fa115ff5740 1 -- start start 2024-02-17T12:54:38.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa115ff5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a842b0 0x557774a84690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa115ff5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 0x557774a8d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa115ff5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 0x557774a8fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa115ff5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577749f3280 con 0x557774a8d7b0 2024-02-17T12:54:38.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa115ff5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577749f07b0 con 0x557774a842b0 2024-02-17T12:54:38.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa115ff5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577749ebf90 con 0x557774a84bd0 2024-02-17T12:54:38.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1108e6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 0x557774a8fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1108e6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 0x557774a8fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43112/0 (socket says 172.21.15.18:43112) 2024-02-17T12:54:38.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1118e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a842b0 0x557774a84690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1108e6640 1 -- 172.21.15.18:0/3963480930 learned_addr learned my addr 172.21.15.18:0/3963480930 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1110e7640 1 --2- 172.21.15.18:0/3963480930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 0x557774a8d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa10a7fc640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0f40a2230 0x7fa0f40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1118e8640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0f40a2230 0x7fa0f40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1108e6640 1 -- 172.21.15.18:0/3963480930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 msgr2=0x557774a8d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1108e6640 1 --2- 172.21.15.18:0/3963480930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 0x557774a8d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1108e6640 1 -- 172.21.15.18:0/3963480930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a842b0 msgr2=0x557774a84690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1108e6640 1 --2- 172.21.15.18:0/3963480930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a842b0 0x557774a84690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa10a7fc640 1 -- 172.21.15.18:0/2186973079 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa104014070 con 0x7fa10c100270 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1108e6640 1 -- 172.21.15.18:0/3963480930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577749aa570 con 0x557774a8d7b0 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1118e8640 1 --2- 172.21.15.18:0/3963480930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a842b0 0x557774a84690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1108e6640 1 --2- 172.21.15.18:0/3963480930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 0x557774a8fba0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa100000c00 tx=0x7fa10002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd1dd6c58fc4beb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.658+0000 7fa1118e8640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0f40a2230 0x7fa0f40a46f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009fd0 tx=0x7fa0fc004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:38.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.659+0000 7fa0f37fe640 1 -- 172.21.15.18:0/3963480930 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100004a00 con 0x557774a8d7b0 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.659+0000 7fa0f37fe640 1 -- 172.21.15.18:0/3963480930 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa100004030 con 0x557774a8d7b0 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.659+0000 7fa0f37fe640 1 -- 172.21.15.18:0/3963480930 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1000052f0 con 0x557774a8d7b0 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.659+0000 7fa115ff5740 1 -- 172.21.15.18:0/3963480930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 msgr2=0x557774a8fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.659+0000 7fa115ff5740 1 --2- 172.21.15.18:0/3963480930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 0x557774a8fba0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa100000c00 tx=0x7fa10002c5d0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.659+0000 7fa115ff5740 1 -- 172.21.15.18:0/3963480930 shutdown_connections 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.659+0000 7fa115ff5740 1 --2- 172.21.15.18:0/3963480930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 0x557774a8fba0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.659+0000 7fa115ff5740 1 --2- 172.21.15.18:0/3963480930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 0x557774a8d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.659+0000 7fa115ff5740 1 --2- 172.21.15.18:0/3963480930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a842b0 0x557774a84690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.659+0000 7fa115ff5740 1 -- 172.21.15.18:0/3963480930 >> 172.21.15.18:0/3963480930 conn(0x557774a7e990 msgr2=0x557774a7ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.659+0000 7fa115ff5740 1 -- 172.21.15.18:0/3963480930 shutdown_connections 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.659+0000 7fa115ff5740 1 -- 172.21.15.18:0/3963480930 wait complete. 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa115ff5740 1 -- start start 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa115ff5740 1 Processor -- start 2024-02-17T12:54:38.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa115ff5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a842b0 0x557774a8d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa115ff5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a84bd0 0x557774a8a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa115ff5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a8d7b0 0x557774a8aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa115ff5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577746bf430 con 0x557774a84bd0 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa115ff5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55777467be00 con 0x557774a8d7b0 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa115ff5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774970fd0 con 0x557774a842b0 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa115ff5740 10 client.0 _ll_register_callbacks cb 0x5577749c6fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa1108e6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a84bd0 0x557774a8a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa1108e6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a84bd0 0x557774a8a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43124/0 (socket says 172.21.15.18:43124) 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa1108e6640 1 -- 172.21.15.18:0/3849235823 learned_addr learned my addr 172.21.15.18:0/3849235823 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa1118e8640 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a8d7b0 0x557774a8aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.660+0000 7fa1110e7640 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a842b0 0x557774a8d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa1108e6640 1 -- 172.21.15.18:0/3849235823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a842b0 msgr2=0x557774a8d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa1108e6640 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a842b0 0x557774a8d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa1108e6640 1 -- 172.21.15.18:0/3849235823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a8d7b0 msgr2=0x557774a8aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa1108e6640 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a8d7b0 0x557774a8aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa1108e6640 1 -- 172.21.15.18:0/3849235823 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c10d720 con 0x557774a84bd0 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa1110e7640 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a842b0 0x557774a8d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa1108e6640 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a84bd0 0x557774a8a4c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa100004b70 tx=0x7fa100004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70eac182c51b5ab server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa0f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/3849235823 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10002c9e0 con 0x557774a84bd0 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa115ff5740 1 -- 172.21.15.18:0/3849235823 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577746eb7e0 con 0x557774a84bd0 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/3849235823 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa10002cb40 con 0x557774a84bd0 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/3849235823 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa100005070 con 0x557774a84bd0 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa115ff5740 10 client.17331 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa115ff5740 1 -- 172.21.15.18:0/3849235823 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5577749ea610 con 0x557774a84bd0 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/3849235823 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5577746eb7e0 con 0x557774a84bd0 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/3849235823 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa10003c040 con 0x557774a84bd0 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa115ff5740 10 client.17331 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:38.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa115ff5740 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa115ff5740 20 client.17331 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa115ff5740 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557774a9d510 0x557774a9f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa115ff5740 4 client.17331 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.661+0000 7fa115ff5740 1 -- 172.21.15.18:0/3849235823 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557774728720 con 0x557774a9d510 2024-02-17T12:54:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.662+0000 7fa1110e7640 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557774a9d510 0x557774a9f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.662+0000 7fa0f0ff9640 10 client.17331 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.662+0000 7fa1110e7640 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557774a9d510 0x557774a9f930 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7164d336df3bef75 server_cookie=6a4fcd329113dd64 in_seq=0 out_seq=0 2024-02-17T12:54:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/3849235823 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557774728720 con 0x557774a9d510 2024-02-17T12:54:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa0f0ff9640 10 client.17331 handle_command_reply: tid=0 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa115ff5740 1 client.17331 shutdown 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa115ff5740 1 -- 172.21.15.18:0/3849235823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a84bd0 msgr2=0x557774a8a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa115ff5740 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a84bd0 0x557774a8a4c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa100004b70 tx=0x7fa100004ba0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa115ff5740 1 -- 172.21.15.18:0/3849235823 shutdown_connections 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa115ff5740 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557774a9d510 0x557774a9f930 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa115ff5740 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a8d7b0 0x557774a8aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa0f0ff9640 0 client.17331 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa115ff5740 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a84bd0 0x557774a8a4c0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa115ff5740 1 --2- 172.21.15.18:0/3849235823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a842b0 0x557774a8d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa115ff5740 1 -- 172.21.15.18:0/3849235823 >> 172.21.15.18:0/3849235823 conn(0x557774a7e990 msgr2=0x557774a80730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa115ff5740 1 -- 172.21.15.18:0/3849235823 shutdown_connections 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa115ff5740 1 -- 172.21.15.18:0/3849235823 wait complete. 2024-02-17T12:54:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.664+0000 7fa115ff5740 20 client.17331 trim_cache size 0 max 16384 2024-02-17T12:54:38.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.679+0000 7fa115ff5740 1 Processor -- start 2024-02-17T12:54:38.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.679+0000 7fa115ff5740 1 -- start start 2024-02-17T12:54:38.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.679+0000 7fa115ff5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a842b0 0x557774a8cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.679+0000 7fa115ff5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 0x557774a8d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.679+0000 7fa115ff5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 0x557774a945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.679+0000 7fa115ff5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774a7f9e0 con 0x557774a8d7b0 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.679+0000 7fa115ff5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774a7f790 con 0x557774a842b0 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.679+0000 7fa115ff5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774a83c80 con 0x557774a84bd0 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa1118e8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 0x557774a945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa1118e8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 0x557774a945f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43128/0 (socket says 172.21.15.18:43128) 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa1118e8640 1 -- 172.21.15.18:0/376486161 learned_addr learned my addr 172.21.15.18:0/376486161 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa1108e6640 1 --2- 172.21.15.18:0/376486161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 0x557774a8d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa1110e7640 1 --2- 172.21.15.18:0/376486161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a842b0 0x557774a8cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa1118e8640 1 -- 172.21.15.18:0/376486161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 msgr2=0x557774a8d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa1118e8640 1 --2- 172.21.15.18:0/376486161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 0x557774a8d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa1118e8640 1 -- 172.21.15.18:0/376486161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a842b0 msgr2=0x557774a8cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa1118e8640 1 --2- 172.21.15.18:0/376486161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a842b0 0x557774a8cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa1118e8640 1 -- 172.21.15.18:0/376486161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557774986ac0 con 0x557774a8d7b0 2024-02-17T12:54:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa1110e7640 1 --2- 172.21.15.18:0/376486161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a842b0 0x557774a8cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:54:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa1118e8640 1 --2- 172.21.15.18:0/376486161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 0x557774a945f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc013270 tx=0x7fa0fc009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5578e63994fdec server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa0f2ffd640 1 -- 172.21.15.18:0/376486161 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0229d0 con 0x557774a8d7b0 2024-02-17T12:54:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa0f2ffd640 1 -- 172.21.15.18:0/376486161 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0fc022b30 con 0x557774a8d7b0 2024-02-17T12:54:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.680+0000 7fa0f2ffd640 1 -- 172.21.15.18:0/376486161 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc02a3e0 con 0x557774a8d7b0 2024-02-17T12:54:38.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.681+0000 7fa115ff5740 1 -- 172.21.15.18:0/376486161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 msgr2=0x557774a945f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.681+0000 7fa115ff5740 1 --2- 172.21.15.18:0/376486161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 0x557774a945f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc013270 tx=0x7fa0fc009c80 comp rx=0 tx=0).stop 2024-02-17T12:54:38.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.681+0000 7fa115ff5740 1 -- 172.21.15.18:0/376486161 shutdown_connections 2024-02-17T12:54:38.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.681+0000 7fa115ff5740 1 --2- 172.21.15.18:0/376486161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a8d7b0 0x557774a945f0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.681+0000 7fa115ff5740 1 --2- 172.21.15.18:0/376486161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 0x557774a8d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.681+0000 7fa115ff5740 1 --2- 172.21.15.18:0/376486161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a842b0 0x557774a8cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.681+0000 7fa115ff5740 1 -- 172.21.15.18:0/376486161 >> 172.21.15.18:0/376486161 conn(0x557774a7e990 msgr2=0x557774a88d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:38.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.681+0000 7fa115ff5740 1 -- 172.21.15.18:0/376486161 shutdown_connections 2024-02-17T12:54:38.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.681+0000 7fa115ff5740 1 -- 172.21.15.18:0/376486161 wait complete. 2024-02-17T12:54:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.681+0000 7fa115ff5740 1 -- start start 2024-02-17T12:54:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.681+0000 7fa115ff5740 1 Processor -- start 2024-02-17T12:54:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa115ff5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a842b0 0x557774a91280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa115ff5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 0x557774a917c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa115ff5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a8d7b0 0x557774a91d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa115ff5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774af3b80 con 0x557774a842b0 2024-02-17T12:54:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa115ff5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774af3cf0 con 0x557774a8d7b0 2024-02-17T12:54:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa115ff5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774af3eb0 con 0x557774a84bd0 2024-02-17T12:54:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa115ff5740 10 client.0 _ll_register_callbacks cb 0x7fa0fc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa1110e7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a842b0 0x557774a91280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa1110e7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a842b0 0x557774a91280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43130/0 (socket says 172.21.15.18:43130) 2024-02-17T12:54:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa1110e7640 1 -- 172.21.15.18:0/2494317215 learned_addr learned my addr 172.21.15.18:0/2494317215 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:38.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa1108e6640 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 0x557774a917c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa1118e8640 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a8d7b0 0x557774a91d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa1110e7640 1 -- 172.21.15.18:0/2494317215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 msgr2=0x557774a917c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa1110e7640 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 0x557774a917c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa1110e7640 1 -- 172.21.15.18:0/2494317215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a8d7b0 msgr2=0x557774a91d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:38.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa1110e7640 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a8d7b0 0x557774a91d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa1110e7640 1 -- 172.21.15.18:0/2494317215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c1a47b0 con 0x557774a842b0 2024-02-17T12:54:38.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa1110e7640 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a842b0 0x557774a91280 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa1040978a0 tx=0x7fa10409af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc683ba90cce061 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:38.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa0f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:38.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2494317215 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10409f020 con 0x557774a842b0 2024-02-17T12:54:38.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2494317215 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa104097540 con 0x557774a842b0 2024-02-17T12:54:38.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.682+0000 7fa115ff5740 1 -- 172.21.15.18:0/2494317215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa100004030 con 0x557774a842b0 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.683+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2494317215 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa104097700 con 0x557774a842b0 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.683+0000 7fa115ff5740 10 client.17337 fetch_fsmap learned FSMap version 34 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.683+0000 7fa115ff5740 1 -- 172.21.15.18:0/2494317215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fa0fc009a50 con 0x557774a842b0 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.683+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2494317215 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1040b7080 con 0x557774a842b0 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.683+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2494317215 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa10401f060 con 0x557774a842b0 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.683+0000 7fa115ff5740 10 client.17337 fetch_fsmap finished waiting for FSMap version 34 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.683+0000 7fa115ff5740 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.683+0000 7fa115ff5740 20 client.17337 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.683+0000 7fa115ff5740 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557774af5710 0x557774af7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.683+0000 7fa115ff5740 4 client.17337 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.683+0000 7fa115ff5740 1 -- 172.21.15.18:0/2494317215 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557774777890 con 0x557774af5710 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.683+0000 7fa1108e6640 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557774af5710 0x557774af7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa1108e6640 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557774af5710 0x557774af7b00 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5587671d82e9455b server_cookie=5cbbf5797cf38757 in_seq=0 out_seq=0 2024-02-17T12:54:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa0f0ff9640 10 client.17337 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa0f0ff9640 1 -- 172.21.15.18:0/2494317215 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557774777890 con 0x557774af5710 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa0f0ff9640 10 client.17337 handle_command_reply: tid=0 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa115ff5740 1 client.17337 shutdown 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa115ff5740 1 -- 172.21.15.18:0/2494317215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a842b0 msgr2=0x557774a91280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa115ff5740 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a842b0 0x557774a91280 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa1040978a0 tx=0x7fa10409af90 comp rx=0 tx=0).stop 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa115ff5740 1 -- 172.21.15.18:0/2494317215 shutdown_connections 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa115ff5740 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557774af5710 0x557774af7b00 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa115ff5740 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557774a8d7b0 0x557774a91d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa115ff5740 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557774a84bd0 0x557774a917c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa0f0ff9640 0 client.17337 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa115ff5740 1 --2- 172.21.15.18:0/2494317215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557774a842b0 0x557774a91280 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa115ff5740 1 -- 172.21.15.18:0/2494317215 >> 172.21.15.18:0/2494317215 conn(0x557774a7e990 msgr2=0x557774a7d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa115ff5740 1 -- 172.21.15.18:0/2494317215 shutdown_connections 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa115ff5740 1 -- 172.21.15.18:0/2494317215 wait complete. 2024-02-17T12:54:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.684+0000 7fa115ff5740 20 client.17337 trim_cache size 0 max 16384 2024-02-17T12:54:38.683 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:54:38.683 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:54:38.683 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:54:38.683 INFO:teuthology.orchestra.run.smithi018.stdout: "4d044d9b-7b89-4337-a5f8-acc800413929": { 2024-02-17T12:54:38.683 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:54:38.683 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "4d044d9b-7b89-4337-a5f8-acc800413929", 2024-02-17T12:54:38.683 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:54:38.683 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:54:38.683 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:54:38.683 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:54:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.685+0000 7fa11334b640 1 -- 172.21.15.18:0/2186973079 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0f40a2230 msgr2=0x7fa0f40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.685+0000 7fa11334b640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0f40a2230 0x7fa0f40a46f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009fd0 tx=0x7fa0fc004ab0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.685+0000 7fa11334b640 1 -- 172.21.15.18:0/2186973079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c100270 msgr2=0x7fa10c19d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.685+0000 7fa11334b640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c100270 0x7fa10c19d410 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e9f0 tx=0x7fa10400edf0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.685+0000 7fa11334b640 1 -- 172.21.15.18:0/2186973079 shutdown_connections 2024-02-17T12:54:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.685+0000 7fa11334b640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0f40a2230 0x7fa0f40a46f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.685+0000 7fa11334b640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa10c197610 0x7fa10c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.685+0000 7fa11334b640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa10c105340 0x7fa10c19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.685+0000 7fa11334b640 1 --2- 172.21.15.18:0/2186973079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c100270 0x7fa10c19d410 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.685+0000 7fa11334b640 1 -- 172.21.15.18:0/2186973079 >> 172.21.15.18:0/2186973079 conn(0x7fa10c077230 msgr2=0x7fa10c104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.685+0000 7fa11334b640 1 -- 172.21.15.18:0/2186973079 shutdown_connections 2024-02-17T12:54:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:54:38.685+0000 7fa11334b640 1 -- 172.21.15.18:0/2186973079 wait complete. 2024-02-17T12:54:38.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:38 smithi018 ceph-mon[32625]: pgmap v145: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 902 KiB/s wr, 79 op/s 2024-02-17T12:54:38.687 INFO:tasks.cephfs.filesystem:scrub status for tag:4d044d9b-7b89-4337-a5f8-acc800413929 - {'path': '/', 'tag': '4d044d9b-7b89-4337-a5f8-acc800413929', 'options': 'recursive,force'} 2024-02-17T12:54:38.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:38 smithi119 ceph-mon[36188]: pgmap v145: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 902 KiB/s wr, 79 op/s 2024-02-17T12:54:38.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:38 smithi146 ceph-mon[36454]: pgmap v145: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 902 KiB/s wr, 79 op/s 2024-02-17T12:54:39.272 INFO:tasks.workunit.client.0.smithi018.stdout: 1 270662 7.79 MB/sec execute 114 sec latency 659.368 ms 2024-02-17T12:54:39.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:39.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:39.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:54:39.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:54:39.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:39.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:39.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:54:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:54:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:39.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:54:40.272 INFO:tasks.workunit.client.0.smithi018.stdout: 1 272979 7.83 MB/sec execute 115 sec latency 13.436 ms 2024-02-17T12:54:40.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:40 smithi018 ceph-mon[32625]: pgmap v146: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-02-17T12:54:40.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:40 smithi119 ceph-mon[36188]: pgmap v146: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-02-17T12:54:40.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:40 smithi146 ceph-mon[36454]: pgmap v146: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-02-17T12:54:41.272 INFO:tasks.workunit.client.0.smithi018.stdout: 1 273236 7.79 MB/sec execute 116 sec latency 687.281 ms 2024-02-17T12:54:41.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:41 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:54:41.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:41 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:54:42.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:41 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:54:42.273 INFO:tasks.workunit.client.0.smithi018.stdout: 1 273236 7.72 MB/sec execute 117 sec latency 1687.383 ms 2024-02-17T12:54:42.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:42 smithi119 ceph-mon[36188]: pgmap v147: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 623 KiB/s wr, 92 op/s 2024-02-17T12:54:42.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:42 smithi146 ceph-mon[36454]: pgmap v147: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 623 KiB/s wr, 92 op/s 2024-02-17T12:54:43.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:42 smithi018 ceph-mon[32625]: pgmap v147: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 623 KiB/s wr, 92 op/s 2024-02-17T12:54:43.272 INFO:tasks.workunit.client.0.smithi018.stdout: 1 273287 7.66 MB/sec execute 118 sec latency 2272.175 ms 2024-02-17T12:54:44.273 INFO:tasks.workunit.client.0.smithi018.stdout: 1 273287 7.60 MB/sec execute 119 sec latency 1368.252 ms 2024-02-17T12:54:44.825 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:44 smithi018 ceph-mon[32625]: pgmap v148: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 623 KiB/s wr, 92 op/s 2024-02-17T12:54:44.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:44 smithi119 ceph-mon[36188]: pgmap v148: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 623 KiB/s wr, 92 op/s 2024-02-17T12:54:44.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:44 smithi146 ceph-mon[36454]: pgmap v148: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 623 KiB/s wr, 92 op/s 2024-02-17T12:54:45.273 INFO:tasks.workunit.client.0.smithi018.stdout: 1 273287 7.53 MB/sec execute 120 sec latency 2368.369 ms 2024-02-17T12:54:45.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:45 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:54:45.875 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:45 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:54:45.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:45 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:54:45.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:45 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:54:46.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:45 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:54:46.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:45 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:54:46.273 INFO:tasks.workunit.client.0.smithi018.stdout: 1 273287 7.47 MB/sec execute 121 sec latency 3368.472 ms 2024-02-17T12:54:46.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:46 smithi119 ceph-mon[36188]: pgmap v149: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-02-17T12:54:46.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:46 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:54:46.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:46 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:54:46.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:46 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:54:46.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:46 smithi146 ceph-mon[36454]: pgmap v149: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-02-17T12:54:46.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:46 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:54:46.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:46 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:54:46.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:46 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:54:47.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:46 smithi018 ceph-mon[32625]: pgmap v149: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-02-17T12:54:47.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:46 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:54:47.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:46 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:54:47.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:46 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:54:47.273 INFO:tasks.workunit.client.0.smithi018.stdout: 1 273287 7.41 MB/sec execute 122 sec latency 4368.525 ms 2024-02-17T12:54:48.273 INFO:tasks.workunit.client.0.smithi018.stdout: 1 273598 7.36 MB/sec execute 123 sec latency 4948.278 ms 2024-02-17T12:54:48.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:48 smithi119 ceph-mon[36188]: pgmap v150: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T12:54:48.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:48 smithi146 ceph-mon[36454]: pgmap v150: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T12:54:49.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:48 smithi018 ceph-mon[32625]: pgmap v150: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T12:54:49.273 INFO:tasks.workunit.client.0.smithi018.stdout: 1 274843 7.34 MB/sec execute 124 sec latency 246.944 ms 2024-02-17T12:54:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.518+0000 7f371f98c640 1 -- 172.21.15.18:0/1862992435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37181013b0 msgr2=0x7f3718101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.518+0000 7f371f98c640 1 --2- 172.21.15.18:0/1862992435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37181013b0 0x7f3718101790 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3714009930 tx=0x7f371402bc30 comp rx=0 tx=0).stop 2024-02-17T12:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.518+0000 7f371f98c640 1 -- 172.21.15.18:0/1862992435 shutdown_connections 2024-02-17T12:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.518+0000 7f371f98c640 1 --2- 172.21.15.18:0/1862992435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3718106480 0x7f371810e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.518+0000 7f371f98c640 1 --2- 172.21.15.18:0/1862992435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3718101d60 0x7f3718105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.518+0000 7f371f98c640 1 --2- 172.21.15.18:0/1862992435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37181013b0 0x7f3718101790 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.518+0000 7f371f98c640 1 -- 172.21.15.18:0/1862992435 >> 172.21.15.18:0/1862992435 conn(0x7f37180fcc30 msgr2=0x7f37180ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.518+0000 7f371f98c640 1 -- 172.21.15.18:0/1862992435 shutdown_connections 2024-02-17T12:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.518+0000 7f371f98c640 1 -- 172.21.15.18:0/1862992435 wait complete. 2024-02-17T12:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.519+0000 7f371f98c640 1 Processor -- start 2024-02-17T12:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.519+0000 7f371f98c640 1 -- start start 2024-02-17T12:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.519+0000 7f371f98c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37181013b0 0x7f3718197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.519+0000 7f371f98c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3718106480 0x7f3718197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.519+0000 7f371f98c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f371819c020 0x7f371819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.519+0000 7f371f98c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371819ca10 con 0x7f37181013b0 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.519+0000 7f371f98c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371819cb80 con 0x7f3718106480 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.519+0000 7f371f98c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371819ccf0 con 0x7f371819c020 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.519+0000 7f371d728640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37181013b0 0x7f3718197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f371d728640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37181013b0 0x7f3718197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50576/0 (socket says 172.21.15.18:50576) 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f371d728640 1 -- 172.21.15.18:0/2780210928 learned_addr learned my addr 172.21.15.18:0/2780210928 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f371cf27640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3718106480 0x7f3718197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f371df29640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f371819c020 0x7f371819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f371d728640 1 -- 172.21.15.18:0/2780210928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f371819c020 msgr2=0x7f371819c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f371d728640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f371819c020 0x7f371819c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f371d728640 1 -- 172.21.15.18:0/2780210928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3718106480 msgr2=0x7f3718197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f371d728640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3718106480 0x7f3718197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f371d728640 1 -- 172.21.15.18:0/2780210928 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3714009560 con 0x7f37181013b0 2024-02-17T12:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f371d728640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37181013b0 0x7f3718197760 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3714009230 tx=0x7f371402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f370e7fc640 1 -- 172.21.15.18:0/2780210928 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3714004350 con 0x7f37181013b0 2024-02-17T12:54:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f371f98c640 1 -- 172.21.15.18:0/2780210928 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37181a42d0 con 0x7f37181013b0 2024-02-17T12:54:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f370e7fc640 1 -- 172.21.15.18:0/2780210928 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37140044b0 con 0x7f37181013b0 2024-02-17T12:54:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f370e7fc640 1 -- 172.21.15.18:0/2780210928 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3714003d70 con 0x7f37181013b0 2024-02-17T12:54:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.520+0000 7f371f98c640 1 -- 172.21.15.18:0/2780210928 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37181a4750 con 0x7f37181013b0 2024-02-17T12:54:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.523+0000 7f370e7fc640 1 -- 172.21.15.18:0/2780210928 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3714004df0 con 0x7f37181013b0 2024-02-17T12:54:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f3722567740 1 Processor -- start 2024-02-17T12:54:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f3722567740 1 -- start start 2024-02-17T12:54:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f3722567740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec388820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f3722567740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec3891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f3722567740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec398ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f3722567740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ec2f79f0 con 0x5610ec389ad0 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f3722567740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ec2f4f20 con 0x5610ec3966f0 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f3722567740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ec2f0700 con 0x5610ec388d60 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f371df29640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec388820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f371df29640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec388820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50586/0 (socket says 172.21.15.18:50586) 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f371df29640 1 -- 172.21.15.18:0/2780758928 learned_addr learned my addr 172.21.15.18:0/2780758928 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f371d728640 1 --2- 172.21.15.18:0/2780758928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec3891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f370e7fc640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37040a22d0 0x7f37040a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f371cf27640 1 --2- 172.21.15.18:0/2780758928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec398ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f371df29640 1 -- 172.21.15.18:0/2780758928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 msgr2=0x5610ec3891e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f371df29640 1 --2- 172.21.15.18:0/2780758928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec3891e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f371df29640 1 -- 172.21.15.18:0/2780758928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 msgr2=0x5610ec398ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f371df29640 1 --2- 172.21.15.18:0/2780758928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec398ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f371df29640 1 -- 172.21.15.18:0/2780758928 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610ec2867b0 con 0x5610ec389ad0 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f371d728640 1 --2- 172.21.15.18:0/2780758928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec3891e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f370e7fc640 1 -- 172.21.15.18:0/2780210928 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f37140effe0 con 0x7f37181013b0 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.524+0000 7f371df29640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37040a22d0 0x7f37040a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.525+0000 7f371df29640 1 --2- 172.21.15.18:0/2780758928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec388820 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3710010440 tx=0x7f3710010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9167a547918bbddc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.525+0000 7f37037fe640 1 -- 172.21.15.18:0/2780758928 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710022070 con 0x5610ec389ad0 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.525+0000 7f37037fe640 1 -- 172.21.15.18:0/2780758928 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3710013520 con 0x5610ec389ad0 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.525+0000 7f37037fe640 1 -- 172.21.15.18:0/2780758928 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710011760 con 0x5610ec389ad0 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.525+0000 7f371df29640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37040a22d0 0x7f37040a4790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3710021000 tx=0x7f371001e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.526+0000 7f3722567740 1 -- 172.21.15.18:0/2780758928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 msgr2=0x5610ec388820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.526+0000 7f3722567740 1 --2- 172.21.15.18:0/2780758928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec388820 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3710010440 tx=0x7f3710010900 comp rx=0 tx=0).stop 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.526+0000 7f3722567740 1 -- 172.21.15.18:0/2780758928 shutdown_connections 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.526+0000 7f3722567740 1 --2- 172.21.15.18:0/2780758928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec398ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.526+0000 7f3722567740 1 --2- 172.21.15.18:0/2780758928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec3891e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.526+0000 7f3722567740 1 --2- 172.21.15.18:0/2780758928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec388820 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.526+0000 7f3722567740 1 -- 172.21.15.18:0/2780758928 >> 172.21.15.18:0/2780758928 conn(0x5610ec350550 msgr2=0x5610ec383a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.526+0000 7f3722567740 1 -- 172.21.15.18:0/2780758928 shutdown_connections 2024-02-17T12:54:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.526+0000 7f3722567740 1 -- 172.21.15.18:0/2780758928 wait complete. 2024-02-17T12:54:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.526+0000 7f3722567740 1 -- start start 2024-02-17T12:54:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.526+0000 7f3722567740 1 Processor -- start 2024-02-17T12:54:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f3722567740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec3936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f3722567740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec393c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f3722567740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec394150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f3722567740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ec295bf0 con 0x5610ec389ad0 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f3722567740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ebfc4250 con 0x5610ec3966f0 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f3722567740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ec276200 con 0x5610ec388d60 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f3722567740 10 client.0 _ll_register_callbacks cb 0x5610ec2e4d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f371cf27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec393c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f371cf27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec393c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50596/0 (socket says 172.21.15.18:50596) 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f371cf27640 1 -- 172.21.15.18:0/867745254 learned_addr learned my addr 172.21.15.18:0/867745254 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f371df29640 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec394150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f371d728640 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec3936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f371cf27640 1 -- 172.21.15.18:0/867745254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 msgr2=0x5610ec3936d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f371cf27640 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec3936d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f371cf27640 1 -- 172.21.15.18:0/867745254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 msgr2=0x5610ec394150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f371cf27640 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec394150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f371cf27640 1 -- 172.21.15.18:0/867745254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3710009c20 con 0x5610ec389ad0 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f371cf27640 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec393c10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3708012700 tx=0x7f3708012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c7c1985bcf5379 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f3700ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f3700ff9640 1 -- 172.21.15.18:0/867745254 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708017070 con 0x5610ec389ad0 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f3700ff9640 1 -- 172.21.15.18:0/867745254 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3708004050 con 0x5610ec389ad0 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.527+0000 7f3722567740 1 -- 172.21.15.18:0/867745254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610ebff0600 con 0x5610ec389ad0 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.528+0000 7f3700ff9640 1 -- 172.21.15.18:0/867745254 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3708004210 con 0x5610ec389ad0 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.528+0000 7f3722567740 10 client.17349 fetch_fsmap learned FSMap version 35 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.528+0000 7f3722567740 1 -- 172.21.15.18:0/867745254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5610ec2eed80 con 0x5610ec389ad0 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.528+0000 7f3700ff9640 1 -- 172.21.15.18:0/867745254 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5610ebff0600 con 0x5610ec389ad0 2024-02-17T12:54:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.528+0000 7f3700ff9640 1 -- 172.21.15.18:0/867745254 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f37080039e0 con 0x5610ec389ad0 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.528+0000 7f3722567740 10 client.17349 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.528+0000 7f3722567740 10 client.17349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.528+0000 7f3722567740 20 client.17349 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.528+0000 7f3722567740 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5610ec3a6b80 0x5610ec3a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.528+0000 7f3722567740 4 client.17349 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.528+0000 7f3722567740 1 -- 172.21.15.18:0/867745254 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610ec0411b0 con 0x5610ec3a6b80 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.528+0000 7f371d728640 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5610ec3a6b80 0x5610ec3a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f3700ff9640 10 client.17349 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f371d728640 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5610ec3a6b80 0x5610ec3a8fa0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b50fee017fcbf4 server_cookie=b59c99ba21f1653e in_seq=0 out_seq=0 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f3700ff9640 1 -- 172.21.15.18:0/867745254 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5610ec0411b0 con 0x5610ec3a6b80 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f3700ff9640 10 client.17349 handle_command_reply: tid=0 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f3722567740 1 client.17349 shutdown 2024-02-17T12:54:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f3722567740 1 -- 172.21.15.18:0/867745254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 msgr2=0x5610ec393c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f3722567740 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec393c10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3708012700 tx=0x7f3708012bc0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f3722567740 1 -- 172.21.15.18:0/867745254 shutdown_connections 2024-02-17T12:54:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f3722567740 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5610ec3a6b80 0x5610ec3a8fa0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f3722567740 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec394150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f3722567740 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec393c10 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f3700ff9640 0 client.17349 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.529+0000 7f3722567740 1 --2- 172.21.15.18:0/867745254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec3936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.530+0000 7f3722567740 1 -- 172.21.15.18:0/867745254 >> 172.21.15.18:0/867745254 conn(0x5610ec350550 msgr2=0x5610ec3857a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.530+0000 7f3722567740 1 -- 172.21.15.18:0/867745254 shutdown_connections 2024-02-17T12:54:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.530+0000 7f3722567740 1 -- 172.21.15.18:0/867745254 wait complete. 2024-02-17T12:54:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.530+0000 7f3722567740 20 client.17349 trim_cache size 0 max 16384 2024-02-17T12:54:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.543+0000 7f3722567740 1 Processor -- start 2024-02-17T12:54:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.543+0000 7f3722567740 1 -- start start 2024-02-17T12:54:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.543+0000 7f3722567740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec393c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.543+0000 7f3722567740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec394160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.543+0000 7f3722567740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec390950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.543+0000 7f3722567740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ec3a5260 con 0x5610ec389ad0 2024-02-17T12:54:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.543+0000 7f3722567740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ec38a0c0 con 0x5610ec3966f0 2024-02-17T12:54:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.543+0000 7f3722567740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ec384a50 con 0x5610ec388d60 2024-02-17T12:54:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f371cf27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec394160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f371cf27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec394160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50612/0 (socket says 172.21.15.18:50612) 2024-02-17T12:54:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f371cf27640 1 -- 172.21.15.18:0/2475617809 learned_addr learned my addr 172.21.15.18:0/2475617809 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f371df29640 1 --2- 172.21.15.18:0/2475617809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec390950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f371d728640 1 --2- 172.21.15.18:0/2475617809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec393c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f371cf27640 1 -- 172.21.15.18:0/2475617809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 msgr2=0x5610ec393c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f371cf27640 1 --2- 172.21.15.18:0/2475617809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec393c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f371cf27640 1 -- 172.21.15.18:0/2475617809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 msgr2=0x5610ec390950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f371cf27640 1 --2- 172.21.15.18:0/2475617809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec390950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f371cf27640 1 -- 172.21.15.18:0/2475617809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610ec28b7b0 con 0x5610ec389ad0 2024-02-17T12:54:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f371df29640 1 --2- 172.21.15.18:0/2475617809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec390950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:54:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f371cf27640 1 --2- 172.21.15.18:0/2475617809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec394160 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f370800a010 tx=0x7f37080096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48662fa037e9f51c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f3702ffd640 1 -- 172.21.15.18:0/2475617809 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708017040 con 0x5610ec389ad0 2024-02-17T12:54:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f3702ffd640 1 -- 172.21.15.18:0/2475617809 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37080055a0 con 0x5610ec389ad0 2024-02-17T12:54:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.544+0000 7f3702ffd640 1 -- 172.21.15.18:0/2475617809 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708003850 con 0x5610ec389ad0 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.545+0000 7f3722567740 1 -- 172.21.15.18:0/2475617809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 msgr2=0x5610ec394160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.545+0000 7f3722567740 1 --2- 172.21.15.18:0/2475617809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec394160 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f370800a010 tx=0x7f37080096d0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.545+0000 7f3722567740 1 -- 172.21.15.18:0/2475617809 shutdown_connections 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.545+0000 7f3722567740 1 --2- 172.21.15.18:0/2475617809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec390950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.545+0000 7f3722567740 1 --2- 172.21.15.18:0/2475617809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec389ad0 0x5610ec394160 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.545+0000 7f3722567740 1 --2- 172.21.15.18:0/2475617809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec388d60 0x5610ec393c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.545+0000 7f3722567740 1 -- 172.21.15.18:0/2475617809 >> 172.21.15.18:0/2475617809 conn(0x5610ec350550 msgr2=0x5610ec391fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.545+0000 7f3722567740 1 -- 172.21.15.18:0/2475617809 shutdown_connections 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.545+0000 7f3722567740 1 -- 172.21.15.18:0/2475617809 wait complete. 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.545+0000 7f3722567740 1 -- start start 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.545+0000 7f3722567740 1 Processor -- start 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f3722567740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec388d60 0x5610ec39bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f3722567740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec389ad0 0x5610ec39c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f3722567740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec39c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f3722567740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ec3fd050 con 0x5610ec388d60 2024-02-17T12:54:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f3722567740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ec3fd190 con 0x5610ec3966f0 2024-02-17T12:54:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f3722567740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ec3fd2d0 con 0x5610ec389ad0 2024-02-17T12:54:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f3722567740 10 client.0 _ll_register_callbacks cb 0x7f3710009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:54:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f371d728640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec388d60 0x5610ec39bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f371d728640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec388d60 0x5610ec39bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50614/0 (socket says 172.21.15.18:50614) 2024-02-17T12:54:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f371d728640 1 -- 172.21.15.18:0/287641182 learned_addr learned my addr 172.21.15.18:0/287641182 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:54:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f371df29640 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec39c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f371cf27640 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec389ad0 0x5610ec39c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f371d728640 1 -- 172.21.15.18:0/287641182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec389ad0 msgr2=0x5610ec39c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f371d728640 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec389ad0 0x5610ec39c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f371d728640 1 -- 172.21.15.18:0/287641182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 msgr2=0x5610ec39c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:54:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f371d728640 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec39c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f371d728640 1 -- 172.21.15.18:0/287641182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610ec2eed80 con 0x5610ec388d60 2024-02-17T12:54:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f371cf27640 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec389ad0 0x5610ec39c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:54:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f371d728640 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec388d60 0x5610ec39bf00 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f37140b8f90 tx=0x7f3714003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee39eb93004e17fd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:54:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.546+0000 7f3700ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:54:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3700ff9640 1 -- 172.21.15.18:0/287641182 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37140c0c80 con 0x5610ec388d60 2024-02-17T12:54:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3700ff9640 1 -- 172.21.15.18:0/287641182 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f37140c52d0 con 0x5610ec388d60 2024-02-17T12:54:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3722567740 1 -- 172.21.15.18:0/287641182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3710013520 con 0x5610ec388d60 2024-02-17T12:54:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3700ff9640 1 -- 172.21.15.18:0/287641182 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37140bae30 con 0x5610ec388d60 2024-02-17T12:54:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3722567740 10 client.17355 fetch_fsmap learned FSMap version 35 2024-02-17T12:54:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3722567740 1 -- 172.21.15.18:0/287641182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5610ec2b1540 con 0x5610ec388d60 2024-02-17T12:54:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3700ff9640 1 -- 172.21.15.18:0/287641182 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3710013520 con 0x5610ec388d60 2024-02-17T12:54:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3700ff9640 1 -- 172.21.15.18:0/287641182 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f371403c070 con 0x5610ec388d60 2024-02-17T12:54:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3722567740 10 client.17355 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:54:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3722567740 10 client.17355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:54:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3722567740 20 client.17355 populate_metadata read hostname 'smithi018' 2024-02-17T12:54:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3722567740 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5610ec3fe830 0x5610ec400c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:54:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3722567740 4 client.17355 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:54:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f3722567740 1 -- 172.21.15.18:0/287641182 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5610ec02d550 con 0x5610ec3fe830 2024-02-17T12:54:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.547+0000 7f371cf27640 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5610ec3fe830 0x5610ec400c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:54:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.548+0000 7f3700ff9640 10 client.17355 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.548+0000 7f371cf27640 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5610ec3fe830 0x5610ec400c50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a21bfe7b2f53b76 server_cookie=d916087906b7cd24 in_seq=0 out_seq=0 2024-02-17T12:54:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.970+0000 7f3700ff9640 1 -- 172.21.15.18:0/287641182 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5610ec02d550 con 0x5610ec3fe830 2024-02-17T12:54:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.970+0000 7f3700ff9640 10 client.17355 handle_command_reply: tid=0 2024-02-17T12:54:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.970+0000 7f3722567740 1 client.17355 shutdown 2024-02-17T12:54:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.971+0000 7f3722567740 1 -- 172.21.15.18:0/287641182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec388d60 msgr2=0x5610ec39bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.971+0000 7f3722567740 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec388d60 0x5610ec39bf00 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f37140b8f90 tx=0x7f3714003840 comp rx=0 tx=0).stop 2024-02-17T12:54:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.971+0000 7f3722567740 1 -- 172.21.15.18:0/287641182 shutdown_connections 2024-02-17T12:54:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.971+0000 7f3722567740 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5610ec3fe830 0x5610ec400c50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.971+0000 7f3722567740 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610ec3966f0 0x5610ec39c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.971+0000 7f3722567740 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610ec389ad0 0x5610ec39c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.971+0000 7f3700ff9640 0 client.17355 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:54:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.971+0000 7f3722567740 1 --2- 172.21.15.18:0/287641182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610ec388d60 0x5610ec39bf00 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.971+0000 7f3722567740 1 -- 172.21.15.18:0/287641182 >> 172.21.15.18:0/287641182 conn(0x5610ec350550 msgr2=0x5610ec382300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.971+0000 7f3722567740 1 -- 172.21.15.18:0/287641182 shutdown_connections 2024-02-17T12:54:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.971+0000 7f3722567740 1 -- 172.21.15.18:0/287641182 wait complete. 2024-02-17T12:54:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.971+0000 7f3722567740 20 client.17355 trim_cache size 0 max 16384 2024-02-17T12:54:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:54:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:54:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:54:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:54:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.972+0000 7f371f98c640 1 -- 172.21.15.18:0/2780210928 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37040a22d0 msgr2=0x7f37040a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.972+0000 7f371f98c640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37040a22d0 0x7f37040a4790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3710021000 tx=0x7f371001e000 comp rx=0 tx=0).stop 2024-02-17T12:54:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.972+0000 7f371f98c640 1 -- 172.21.15.18:0/2780210928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37181013b0 msgr2=0x7f3718197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:54:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.972+0000 7f371f98c640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37181013b0 0x7f3718197760 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3714009230 tx=0x7f371402c6c0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.972+0000 7f371f98c640 1 -- 172.21.15.18:0/2780210928 shutdown_connections 2024-02-17T12:54:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.972+0000 7f371f98c640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37040a22d0 0x7f37040a4790 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.972+0000 7f371f98c640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f371819c020 0x7f371819c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.972+0000 7f371f98c640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3718106480 0x7f3718197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.972+0000 7f371f98c640 1 --2- 172.21.15.18:0/2780210928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37181013b0 0x7f3718197760 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:54:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.972+0000 7f371f98c640 1 -- 172.21.15.18:0/2780210928 >> 172.21.15.18:0/2780210928 conn(0x7f37180fcc30 msgr2=0x7f37180fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:54:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.972+0000 7f371f98c640 1 -- 172.21.15.18:0/2780210928 shutdown_connections 2024-02-17T12:54:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:54:49.972+0000 7f371f98c640 1 -- 172.21.15.18:0/2780210928 wait complete. 2024-02-17T12:54:50.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 274843 7.28 MB/sec execute 125 sec latency 1247.031 ms 2024-02-17T12:54:50.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:50 smithi119 ceph-mon[36188]: pgmap v151: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-02-17T12:54:50.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:50 smithi146 ceph-mon[36454]: pgmap v151: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-02-17T12:54:51.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:50 smithi018 ceph-mon[32625]: pgmap v151: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-02-17T12:54:51.273 INFO:tasks.workunit.client.0.smithi018.stdout: 1 274843 7.23 MB/sec execute 126 sec latency 2247.172 ms 2024-02-17T12:54:52.273 INFO:tasks.workunit.client.0.smithi018.stdout: 1 274843 7.17 MB/sec execute 127 sec latency 3247.297 ms 2024-02-17T12:54:52.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:52 smithi119 ceph-mon[36188]: pgmap v152: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 934 KiB/s wr, 91 op/s 2024-02-17T12:54:52.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:52 smithi146 ceph-mon[36454]: pgmap v152: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 934 KiB/s wr, 91 op/s 2024-02-17T12:54:53.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:52 smithi018 ceph-mon[32625]: pgmap v152: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 934 KiB/s wr, 91 op/s 2024-02-17T12:54:53.273 INFO:tasks.workunit.client.0.smithi018.stdout: 1 274843 7.11 MB/sec execute 128 sec latency 4247.422 ms 2024-02-17T12:54:54.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 275722 7.08 MB/sec execute 129 sec latency 4860.873 ms 2024-02-17T12:54:54.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:54 smithi119 ceph-mon[36188]: pgmap v153: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 934 KiB/s wr, 91 op/s 2024-02-17T12:54:54.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:54 smithi146 ceph-mon[36454]: pgmap v153: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 934 KiB/s wr, 91 op/s 2024-02-17T12:54:55.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:54 smithi018 ceph-mon[32625]: pgmap v153: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 934 KiB/s wr, 91 op/s 2024-02-17T12:54:55.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 277295 7.11 MB/sec execute 130 sec latency 19.557 ms 2024-02-17T12:54:56.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 278385 7.10 MB/sec execute 131 sec latency 242.735 ms 2024-02-17T12:54:56.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:56 smithi119 ceph-mon[36188]: pgmap v154: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T12:54:56.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:56 smithi146 ceph-mon[36454]: pgmap v154: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T12:54:57.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:56 smithi018 ceph-mon[32625]: pgmap v154: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T12:54:57.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 278385 7.04 MB/sec execute 132 sec latency 1242.805 ms 2024-02-17T12:54:58.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 278385 6.99 MB/sec execute 133 sec latency 2242.877 ms 2024-02-17T12:54:58.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:54:58 smithi119 ceph-mon[36188]: pgmap v155: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 490 KiB/s wr, 64 op/s 2024-02-17T12:54:58.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:54:58 smithi146 ceph-mon[36454]: pgmap v155: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 490 KiB/s wr, 64 op/s 2024-02-17T12:54:59.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:54:58 smithi018 ceph-mon[32625]: pgmap v155: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 490 KiB/s wr, 64 op/s 2024-02-17T12:54:59.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 279289 6.96 MB/sec execute 134 sec latency 2857.893 ms 2024-02-17T12:55:00.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 280844 6.99 MB/sec execute 135 sec latency 11.240 ms 2024-02-17T12:55:00.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:00 smithi119 ceph-mon[36188]: pgmap v156: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1014 KiB/s wr, 123 op/s 2024-02-17T12:55:00.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:00 smithi146 ceph-mon[36454]: pgmap v156: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1014 KiB/s wr, 123 op/s 2024-02-17T12:55:01.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:00 smithi018 ceph-mon[32625]: pgmap v156: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1014 KiB/s wr, 123 op/s 2024-02-17T12:55:01.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 281934 6.98 MB/sec execute 136 sec latency 415.190 ms 2024-02-17T12:55:02.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 281934 6.93 MB/sec execute 137 sec latency 1415.234 ms 2024-02-17T12:55:02.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:02 smithi119 ceph-mon[36188]: pgmap v157: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 823 KiB/s wr, 92 op/s 2024-02-17T12:55:02.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:02 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:55:02.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:02 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:55:02.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:02 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:55:02.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:02 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:55:02.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:02 smithi146 ceph-mon[36454]: pgmap v157: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 823 KiB/s wr, 92 op/s 2024-02-17T12:55:02.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:02 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:55:02.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:02 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:55:02.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:02 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:55:02.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:02 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:55:03.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:02 smithi018 ceph-mon[32625]: pgmap v157: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 823 KiB/s wr, 92 op/s 2024-02-17T12:55:03.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:02 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:55:03.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:02 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:55:03.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:02 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:55:03.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:02 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:55:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.065+0000 7f48aed93640 1 -- 172.21.15.18:0/112838799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48a81002d0 msgr2=0x7f48a8100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.065+0000 7f48aed93640 1 --2- 172.21.15.18:0/112838799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48a81002d0 0x7f48a8100750 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f4894009900 tx=0x7f489402bc30 comp rx=0 tx=0).stop 2024-02-17T12:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 -- 172.21.15.18:0/112838799 shutdown_connections 2024-02-17T12:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 --2- 172.21.15.18:0/112838799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48a8100c90 0x7f48a810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 --2- 172.21.15.18:0/112838799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48a81002d0 0x7f48a8100750 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 --2- 172.21.15.18:0/112838799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a8105020 0x7f48a8105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 -- 172.21.15.18:0/112838799 >> 172.21.15.18:0/112838799 conn(0x7f48a8076ed0 msgr2=0x7f48a80772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 -- 172.21.15.18:0/112838799 shutdown_connections 2024-02-17T12:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 -- 172.21.15.18:0/112838799 wait complete. 2024-02-17T12:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 Processor -- start 2024-02-17T12:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 -- start start 2024-02-17T12:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a81002d0 0x7f48a819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48a8100c90 0x7f48a819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48a8105020 0x7f48a81976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a8197e60 con 0x7f48a8100c90 2024-02-17T12:55:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a8197fd0 con 0x7f48a81002d0 2024-02-17T12:55:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.066+0000 7f48aed93640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a8198190 con 0x7f48a8105020 2024-02-17T12:55:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48a8100c90 0x7f48a819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48acb2f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a81002d0 0x7f48a819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48acb2f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a81002d0 0x7f48a819d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:58910/0 (socket says 172.21.15.18:58910) 2024-02-17T12:55:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48a8100c90 0x7f48a819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35560/0 (socket says 172.21.15.18:35560) 2024-02-17T12:55:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48acb2f640 1 -- 172.21.15.18:0/1161495870 learned_addr learned my addr 172.21.15.18:0/1161495870 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48ad330640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48a8105020 0x7f48a81976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48acb2f640 1 -- 172.21.15.18:0/1161495870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48a8105020 msgr2=0x7f48a81976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48acb2f640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48a8105020 0x7f48a81976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48acb2f640 1 -- 172.21.15.18:0/1161495870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48a8100c90 msgr2=0x7f48a819db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48acb2f640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48a8100c90 0x7f48a819db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48acb2f640 1 -- 172.21.15.18:0/1161495870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4894009560 con 0x7f48a81002d0 2024-02-17T12:55:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48acb2f640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a81002d0 0x7f48a819d610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f489c00e9f0 tx=0x7f489c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48a5ffb640 1 -- 172.21.15.18:0/1161495870 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c009700 con 0x7f48a81002d0 2024-02-17T12:55:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48a5ffb640 1 -- 172.21.15.18:0/1161495870 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f489c00cd80 con 0x7f48a81002d0 2024-02-17T12:55:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48aed93640 1 -- 172.21.15.18:0/1161495870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a8198470 con 0x7f48a81002d0 2024-02-17T12:55:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.067+0000 7f48a5ffb640 1 -- 172.21.15.18:0/1161495870 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c018920 con 0x7f48a81002d0 2024-02-17T12:55:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.068+0000 7f48aed93640 1 -- 172.21.15.18:0/1161495870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a81a4590 con 0x7f48a81002d0 2024-02-17T12:55:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48a5ffb640 1 -- 172.21.15.18:0/1161495870 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f489c018a80 con 0x7f48a81002d0 2024-02-17T12:55:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48b196d740 1 Processor -- start 2024-02-17T12:55:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48b196d740 1 -- start start 2024-02-17T12:55:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48b196d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48b196d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890d51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48b196d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890e06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48b196d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d890439f0 con 0x563d890d5ad0 2024-02-17T12:55:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48b196d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d89040f20 con 0x563d890d4d60 2024-02-17T12:55:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48b196d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d8903c700 con 0x563d890de2e0 2024-02-17T12:55:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48ad330640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48acb2f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890d51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48ad330640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890d4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35562/0 (socket says 172.21.15.18:35562) 2024-02-17T12:55:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48ad330640 1 -- 172.21.15.18:0/2844177909 learned_addr learned my addr 172.21.15.18:0/2844177909 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48a5ffb640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f48980a22d0 0x7f48980a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.071+0000 7f48a7fff640 1 --2- 172.21.15.18:0/2844177909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890e06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.072+0000 7f48ad330640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f48980a22d0 0x7f48980a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.072+0000 7f48acb2f640 1 -- 172.21.15.18:0/2844177909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 msgr2=0x563d890e06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.072+0000 7f48acb2f640 1 --2- 172.21.15.18:0/2844177909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890e06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.072+0000 7f48acb2f640 1 -- 172.21.15.18:0/2844177909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 msgr2=0x563d890d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.072+0000 7f48acb2f640 1 --2- 172.21.15.18:0/2844177909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.072+0000 7f48acb2f640 1 -- 172.21.15.18:0/2844177909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d88fd27b0 con 0x563d890d4d60 2024-02-17T12:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.072+0000 7f48a5ffb640 1 -- 172.21.15.18:0/1161495870 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f489c014070 con 0x7f48a81002d0 2024-02-17T12:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.072+0000 7f48acb2f640 1 --2- 172.21.15.18:0/2844177909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890d51e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f489c0cbae0 tx=0x7f489c010f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1a45548f128cc1a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.072+0000 7f4892ffd640 1 -- 172.21.15.18:0/2844177909 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0d17a0 con 0x563d890d4d60 2024-02-17T12:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.072+0000 7f48ad330640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f48980a22d0 0x7f48980a4790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48a0005f40 tx=0x7f48a0004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.072+0000 7f4892ffd640 1 -- 172.21.15.18:0/2844177909 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f489c09a790 con 0x563d890d4d60 2024-02-17T12:55:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.072+0000 7f4892ffd640 1 -- 172.21.15.18:0/2844177909 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0974a0 con 0x563d890d4d60 2024-02-17T12:55:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.073+0000 7f48b196d740 1 -- 172.21.15.18:0/2844177909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 msgr2=0x563d890d51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.073+0000 7f48b196d740 1 --2- 172.21.15.18:0/2844177909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890d51e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f489c0cbae0 tx=0x7f489c010f50 comp rx=0 tx=0).stop 2024-02-17T12:55:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.073+0000 7f48b196d740 1 -- 172.21.15.18:0/2844177909 shutdown_connections 2024-02-17T12:55:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.073+0000 7f48b196d740 1 --2- 172.21.15.18:0/2844177909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890e06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.073+0000 7f48b196d740 1 --2- 172.21.15.18:0/2844177909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890d51e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.073+0000 7f48b196d740 1 --2- 172.21.15.18:0/2844177909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.073+0000 7f48b196d740 1 -- 172.21.15.18:0/2844177909 >> 172.21.15.18:0/2844177909 conn(0x563d8909c550 msgr2=0x563d890cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.073+0000 7f48b196d740 1 -- 172.21.15.18:0/2844177909 shutdown_connections 2024-02-17T12:55:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.073+0000 7f48b196d740 1 -- 172.21.15.18:0/2844177909 wait complete. 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48b196d740 1 -- start start 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48b196d740 1 Processor -- start 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48b196d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890db340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48b196d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890db880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48b196d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890dbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48b196d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d88fe1bf0 con 0x563d890d5ad0 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48b196d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d88d10250 con 0x563d890d4d60 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48b196d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d88fc2200 con 0x563d890de2e0 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48b196d740 10 client.0 _ll_register_callbacks cb 0x563d89030d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890db880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890db880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35564/0 (socket says 172.21.15.18:35564) 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48a7fff640 1 -- 172.21.15.18:0/334107822 learned_addr learned my addr 172.21.15.18:0/334107822 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48acb2f640 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890db340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.074+0000 7f48ad330640 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890dbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f48a7fff640 1 -- 172.21.15.18:0/334107822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 msgr2=0x563d890dbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f48a7fff640 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890dbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f48a7fff640 1 -- 172.21.15.18:0/334107822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 msgr2=0x563d890db340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f48a7fff640 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890db340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f48a7fff640 1 -- 172.21.15.18:0/334107822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a81a4590 con 0x563d890d5ad0 2024-02-17T12:55:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f48acb2f640 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890db340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f48a7fff640 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890db880 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4894009900 tx=0x7f48940048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c418abbf4f10f73 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f4887fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f4887fff640 1 -- 172.21.15.18:0/334107822 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894004040 con 0x563d890d5ad0 2024-02-17T12:55:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f4887fff640 1 -- 172.21.15.18:0/334107822 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f48940041a0 con 0x563d890d5ad0 2024-02-17T12:55:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f48b196d740 1 -- 172.21.15.18:0/334107822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d88d3c600 con 0x563d890d5ad0 2024-02-17T12:55:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f4887fff640 1 -- 172.21.15.18:0/334107822 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4894004360 con 0x563d890d5ad0 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f48b196d740 10 client.17367 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f48b196d740 1 -- 172.21.15.18:0/334107822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563d8903ad80 con 0x563d890d5ad0 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.075+0000 7f4887fff640 1 -- 172.21.15.18:0/334107822 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563d88d3c600 con 0x563d890d5ad0 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.076+0000 7f4887fff640 1 -- 172.21.15.18:0/334107822 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f489403c040 con 0x563d890d5ad0 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.076+0000 7f48b196d740 10 client.17367 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.076+0000 7f48b196d740 10 client.17367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.076+0000 7f48b196d740 20 client.17367 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.076+0000 7f48b196d740 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563d890ee750 0x563d890f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.076+0000 7f48b196d740 4 client.17367 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.076+0000 7f48b196d740 1 -- 172.21.15.18:0/334107822 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d88d8d1b0 con 0x563d890ee750 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.076+0000 7f48acb2f640 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563d890ee750 0x563d890f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.076+0000 7f4887fff640 10 client.17367 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.076+0000 7f48acb2f640 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563d890ee750 0x563d890f0b90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c761b29ea2e7d0 server_cookie=c77be3cd0f651d5e in_seq=0 out_seq=0 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f4887fff640 1 -- 172.21.15.18:0/334107822 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d88d8d1b0 con 0x563d890ee750 2024-02-17T12:55:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f4887fff640 10 client.17367 handle_command_reply: tid=0 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f48b196d740 1 client.17367 shutdown 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f48b196d740 1 -- 172.21.15.18:0/334107822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 msgr2=0x563d890db880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f48b196d740 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890db880 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4894009900 tx=0x7f48940048f0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f48b196d740 1 -- 172.21.15.18:0/334107822 shutdown_connections 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f48b196d740 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563d890ee750 0x563d890f0b90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f48b196d740 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890dbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f48b196d740 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890db880 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f4887fff640 0 client.17367 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f48b196d740 1 --2- 172.21.15.18:0/334107822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890db340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f48b196d740 1 -- 172.21.15.18:0/334107822 >> 172.21.15.18:0/334107822 conn(0x563d8909c550 msgr2=0x563d890d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f48b196d740 1 -- 172.21.15.18:0/334107822 shutdown_connections 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f48b196d740 1 -- 172.21.15.18:0/334107822 wait complete. 2024-02-17T12:55:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.077+0000 7f48b196d740 20 client.17367 trim_cache size 0 max 16384 2024-02-17T12:55:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48b196d740 1 Processor -- start 2024-02-17T12:55:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48b196d740 1 -- start start 2024-02-17T12:55:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48b196d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48b196d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890dbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48b196d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890e50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48b196d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d890d6260 con 0x563d890d5ad0 2024-02-17T12:55:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48b196d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d890d4b80 con 0x563d890d4d60 2024-02-17T12:55:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48b196d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d890f5dc0 con 0x563d890de2e0 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890dbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48acb2f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890dbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35568/0 (socket says 172.21.15.18:35568) 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48a7fff640 1 -- 172.21.15.18:0/1278539004 learned_addr learned my addr 172.21.15.18:0/1278539004 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.091+0000 7f48ad330640 1 --2- 172.21.15.18:0/1278539004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890e50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.092+0000 7f48acb2f640 1 -- 172.21.15.18:0/1278539004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 msgr2=0x563d890e50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.092+0000 7f48acb2f640 1 --2- 172.21.15.18:0/1278539004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890e50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.092+0000 7f48acb2f640 1 -- 172.21.15.18:0/1278539004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 msgr2=0x563d890dbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.092+0000 7f48acb2f640 1 --2- 172.21.15.18:0/1278539004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890dbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.092+0000 7f48acb2f640 1 -- 172.21.15.18:0/1278539004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d88fd77b0 con 0x563d890d4d60 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.092+0000 7f48a7fff640 1 --2- 172.21.15.18:0/1278539004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890dbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.092+0000 7f48acb2f640 1 --2- 172.21.15.18:0/1278539004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890dda70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f489c09aff0 tx=0x7f489c097200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c17fe25c5fcf565 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.092+0000 7f48927fc640 1 -- 172.21.15.18:0/1278539004 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0ce440 con 0x563d890d4d60 2024-02-17T12:55:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.092+0000 7f48927fc640 1 -- 172.21.15.18:0/1278539004 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f489c09ae30 con 0x563d890d4d60 2024-02-17T12:55:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.092+0000 7f48927fc640 1 -- 172.21.15.18:0/1278539004 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c024d40 con 0x563d890d4d60 2024-02-17T12:55:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.093+0000 7f48b196d740 1 -- 172.21.15.18:0/1278539004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 msgr2=0x563d890dda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.093+0000 7f48b196d740 1 --2- 172.21.15.18:0/1278539004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890dda70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f489c09aff0 tx=0x7f489c097200 comp rx=0 tx=0).stop 2024-02-17T12:55:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.093+0000 7f48b196d740 1 -- 172.21.15.18:0/1278539004 shutdown_connections 2024-02-17T12:55:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.093+0000 7f48b196d740 1 --2- 172.21.15.18:0/1278539004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.093+0000 7f48b196d740 1 --2- 172.21.15.18:0/1278539004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d5ad0 0x563d890dbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.093+0000 7f48b196d740 1 --2- 172.21.15.18:0/1278539004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d4d60 0x563d890dda70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.093+0000 7f48b196d740 1 -- 172.21.15.18:0/1278539004 >> 172.21.15.18:0/1278539004 conn(0x563d8909c550 msgr2=0x563d890cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.093+0000 7f48b196d740 1 -- 172.21.15.18:0/1278539004 shutdown_connections 2024-02-17T12:55:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.093+0000 7f48b196d740 1 -- 172.21.15.18:0/1278539004 wait complete. 2024-02-17T12:55:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.093+0000 7f48b196d740 1 -- start start 2024-02-17T12:55:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.093+0000 7f48b196d740 1 Processor -- start 2024-02-17T12:55:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48b196d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d4d60 0x563d890e50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48b196d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d5ad0 0x563d890e55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48b196d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890d7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48b196d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d890d82c0 con 0x563d890d4d60 2024-02-17T12:55:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48b196d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d890d8430 con 0x563d890d5ad0 2024-02-17T12:55:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48b196d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d890d85f0 con 0x563d890de2e0 2024-02-17T12:55:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48b196d740 10 client.0 _ll_register_callbacks cb 0x7f489c097150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48acb2f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d4d60 0x563d890e50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48acb2f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d4d60 0x563d890e50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35584/0 (socket says 172.21.15.18:35584) 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48acb2f640 1 -- 172.21.15.18:0/501368054 learned_addr learned my addr 172.21.15.18:0/501368054 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48a7fff640 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d5ad0 0x563d890e55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48ad330640 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890d7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48acb2f640 1 -- 172.21.15.18:0/501368054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 msgr2=0x563d890d7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48acb2f640 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890d7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48acb2f640 1 -- 172.21.15.18:0/501368054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d5ad0 msgr2=0x563d890e55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48acb2f640 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d5ad0 0x563d890e55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48acb2f640 1 -- 172.21.15.18:0/501368054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d8903ad80 con 0x563d890d4d60 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f48acb2f640 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d4d60 0x563d890e50b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f489c09afc0 tx=0x7f489c097530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b4cf11ba3b1887 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.094+0000 7f4887fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f4887fff640 1 -- 172.21.15.18:0/501368054 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0a3520 con 0x563d890d4d60 2024-02-17T12:55:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f4887fff640 1 -- 172.21.15.18:0/501368054 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f489c0b92b0 con 0x563d890d4d60 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f48b196d740 1 -- 172.21.15.18:0/501368054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f489c09a790 con 0x563d890d4d60 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f4887fff640 1 -- 172.21.15.18:0/501368054 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f489c024ae0 con 0x563d890d4d60 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f48b196d740 10 client.17373 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f48b196d740 1 -- 172.21.15.18:0/501368054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563d88fd27b0 con 0x563d890d4d60 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f4887fff640 1 -- 172.21.15.18:0/501368054 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f489c09a790 con 0x563d890d4d60 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f4887fff640 1 -- 172.21.15.18:0/501368054 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f489c09a440 con 0x563d890d4d60 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f48b196d740 10 client.17373 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f48b196d740 10 client.17373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f48b196d740 20 client.17373 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f48b196d740 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563d89145c80 0x563d891480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f48b196d740 4 client.17373 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f48b196d740 1 -- 172.21.15.18:0/501368054 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563d88d79550 con 0x563d89145c80 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.095+0000 7f48a7fff640 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563d89145c80 0x563d891480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.096+0000 7f4887fff640 10 client.17373 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.096+0000 7f48a7fff640 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563d89145c80 0x563d891480a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f586581cb98e44 server_cookie=6ee60607c398628 in_seq=0 out_seq=0 2024-02-17T12:55:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.121+0000 7f4887fff640 1 -- 172.21.15.18:0/501368054 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563d88d79550 con 0x563d89145c80 2024-02-17T12:55:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.121+0000 7f4887fff640 10 client.17373 handle_command_reply: tid=0 2024-02-17T12:55:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.121+0000 7f48b196d740 1 client.17373 shutdown 2024-02-17T12:55:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.122+0000 7f48b196d740 1 -- 172.21.15.18:0/501368054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d4d60 msgr2=0x563d890e50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.122+0000 7f48b196d740 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d4d60 0x563d890e50b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f489c09afc0 tx=0x7f489c097530 comp rx=0 tx=0).stop 2024-02-17T12:55:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.122+0000 7f48b196d740 1 -- 172.21.15.18:0/501368054 shutdown_connections 2024-02-17T12:55:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.122+0000 7f48b196d740 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563d89145c80 0x563d891480a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.122+0000 7f48b196d740 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d890de2e0 0x563d890d7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.122+0000 7f48b196d740 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d890d5ad0 0x563d890e55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.122+0000 7f4887fff640 0 client.17373 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.122+0000 7f48b196d740 1 --2- 172.21.15.18:0/501368054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d890d4d60 0x563d890e50b0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.122+0000 7f48b196d740 1 -- 172.21.15.18:0/501368054 >> 172.21.15.18:0/501368054 conn(0x563d8909c550 msgr2=0x563d890d9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.122+0000 7f48b196d740 1 -- 172.21.15.18:0/501368054 shutdown_connections 2024-02-17T12:55:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.122+0000 7f48b196d740 1 -- 172.21.15.18:0/501368054 wait complete. 2024-02-17T12:55:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.122+0000 7f48b196d740 20 client.17373 trim_cache size 0 max 16384 2024-02-17T12:55:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:55:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:55:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:55:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:55:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.123+0000 7f48aed93640 1 -- 172.21.15.18:0/1161495870 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f48980a22d0 msgr2=0x7f48980a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.123+0000 7f48aed93640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f48980a22d0 0x7f48980a4790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48a0005f40 tx=0x7f48a0004ab0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.123+0000 7f48aed93640 1 -- 172.21.15.18:0/1161495870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a81002d0 msgr2=0x7f48a819d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.123+0000 7f48aed93640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a81002d0 0x7f48a819d610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f489c00e9f0 tx=0x7f489c00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.123+0000 7f48aed93640 1 -- 172.21.15.18:0/1161495870 shutdown_connections 2024-02-17T12:55:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.123+0000 7f48aed93640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f48980a22d0 0x7f48980a4790 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.123+0000 7f48aed93640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48a8105020 0x7f48a81976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.123+0000 7f48aed93640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48a8100c90 0x7f48a819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.123+0000 7f48aed93640 1 --2- 172.21.15.18:0/1161495870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a81002d0 0x7f48a819d610 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.123+0000 7f48aed93640 1 -- 172.21.15.18:0/1161495870 >> 172.21.15.18:0/1161495870 conn(0x7f48a8076ed0 msgr2=0x7f48a8068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.123+0000 7f48aed93640 1 -- 172.21.15.18:0/1161495870 shutdown_connections 2024-02-17T12:55:03.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:03.123+0000 7f48aed93640 1 -- 172.21.15.18:0/1161495870 wait complete. 2024-02-17T12:55:03.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 282532 6.89 MB/sec execute 138 sec latency 1989.198 ms 2024-02-17T12:55:04.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 284193 6.93 MB/sec execute 139 sec latency 19.587 ms 2024-02-17T12:55:04.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:04 smithi119 ceph-mon[36188]: pgmap v158: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 823 KiB/s wr, 92 op/s 2024-02-17T12:55:04.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:04 smithi146 ceph-mon[36454]: pgmap v158: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 823 KiB/s wr, 92 op/s 2024-02-17T12:55:05.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:04 smithi018 ceph-mon[32625]: pgmap v158: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 823 KiB/s wr, 92 op/s 2024-02-17T12:55:05.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 285476 6.91 MB/sec execute 140 sec latency 46.814 ms 2024-02-17T12:55:06.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 285476 6.86 MB/sec execute 141 sec latency 1046.927 ms 2024-02-17T12:55:06.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:06 smithi119 ceph-mon[36188]: pgmap v159: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2024-02-17T12:55:06.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:06 smithi146 ceph-mon[36454]: pgmap v159: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2024-02-17T12:55:07.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:06 smithi018 ceph-mon[32625]: pgmap v159: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2024-02-17T12:55:07.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 285476 6.82 MB/sec execute 142 sec latency 2046.994 ms 2024-02-17T12:55:08.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 285476 6.77 MB/sec execute 143 sec latency 3047.057 ms 2024-02-17T12:55:08.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:55:08.688 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:55:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.814+0000 7f7ba1f27640 1 -- 172.21.15.18:0/1486665029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b9c1023a0 msgr2=0x7f7b9c102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.814+0000 7f7ba1f27640 1 --2- 172.21.15.18:0/1486665029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c102780 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009ca0 tx=0x7f7b8402c180 comp rx=0 tx=0).stop 2024-02-17T12:55:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.814+0000 7f7ba1f27640 1 -- 172.21.15.18:0/1486665029 shutdown_connections 2024-02-17T12:55:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.814+0000 7f7ba1f27640 1 --2- 172.21.15.18:0/1486665029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b9c10b6e0 0x7f7b9c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.814+0000 7f7ba1f27640 1 --2- 172.21.15.18:0/1486665029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9c102cc0 0x7f7b9c10b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.814+0000 7f7ba1f27640 1 --2- 172.21.15.18:0/1486665029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c102780 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.814+0000 7f7ba1f27640 1 -- 172.21.15.18:0/1486665029 >> 172.21.15.18:0/1486665029 conn(0x7f7b9c077230 msgr2=0x7f7b9c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:08.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.814+0000 7f7ba1f27640 1 -- 172.21.15.18:0/1486665029 shutdown_connections 2024-02-17T12:55:08.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.814+0000 7f7ba1f27640 1 -- 172.21.15.18:0/1486665029 wait complete. 2024-02-17T12:55:08.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.814+0000 7f7ba1f27640 1 Processor -- start 2024-02-17T12:55:08.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7ba1f27640 1 -- start start 2024-02-17T12:55:08.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7ba1f27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7ba1f27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9c102cc0 0x7f7b9c197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7ba1f27640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b9c10b6e0 0x7f7b9c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7ba1f27640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c19c5f0 con 0x7f7b9c1023a0 2024-02-17T12:55:08.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7ba1f27640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c19c760 con 0x7f7b9c102cc0 2024-02-17T12:55:08.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7ba1f27640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c19c920 con 0x7f7b9c10b6e0 2024-02-17T12:55:08.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7b9b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7b9b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35604/0 (socket says 172.21.15.18:35604) 2024-02-17T12:55:08.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7b9b7fe640 1 -- 172.21.15.18:0/4223708313 learned_addr learned my addr 172.21.15.18:0/4223708313 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b9c10b6e0 0x7f7b9c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7b9affd640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9c102cc0 0x7f7b9c197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7b9b7fe640 1 -- 172.21.15.18:0/4223708313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b9c10b6e0 msgr2=0x7f7b9c19bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7b9b7fe640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b9c10b6e0 0x7f7b9c19bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7b9b7fe640 1 -- 172.21.15.18:0/4223708313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9c102cc0 msgr2=0x7f7b9c197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7b9b7fe640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9c102cc0 0x7f7b9c197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7b9b7fe640 1 -- 172.21.15.18:0/4223708313 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b84009900 con 0x7f7b9c1023a0 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.815+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b9c10b6e0 0x7f7b9c19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.816+0000 7f7b9b7fe640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c197780 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7b84004af0 tx=0x7f7b840346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.816+0000 7f7b98ff9640 1 -- 172.21.15.18:0/4223708313 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84003d80 con 0x7f7b9c1023a0 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.816+0000 7f7b98ff9640 1 -- 172.21.15.18:0/4223708313 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b840348e0 con 0x7f7b9c1023a0 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.816+0000 7f7ba1f27640 1 -- 172.21.15.18:0/4223708313 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c19cba0 con 0x7f7b9c1023a0 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.816+0000 7f7b98ff9640 1 -- 172.21.15.18:0/4223708313 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84004790 con 0x7f7b9c1023a0 2024-02-17T12:55:08.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.816+0000 7f7ba1f27640 1 -- 172.21.15.18:0/4223708313 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c19ce70 con 0x7f7b9c1023a0 2024-02-17T12:55:08.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.818+0000 7f7b98ff9640 1 -- 172.21.15.18:0/4223708313 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7b8400a040 con 0x7f7b9c1023a0 2024-02-17T12:55:08.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.818+0000 7f7ba4bcf740 1 Processor -- start 2024-02-17T12:55:08.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.818+0000 7f7ba4bcf740 1 -- start start 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7ba4bcf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d509e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b98ff9640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b7c0a2230 0x7f7b7c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7ba4bcf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 0x5560d50a7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7ba4bcf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d50a77b0 0x5560d50a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7ba4bcf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d500d280 con 0x5560d509ebd0 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7ba4bcf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d500a7b0 con 0x5560d509e2b0 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7ba4bcf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d5005f90 con 0x5560d50a77b0 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 0x5560d50a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9b7fe640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b7c0a2230 0x7f7b7c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d509e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 0x5560d50a7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35608/0 (socket says 172.21.15.18:35608) 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9affd640 1 -- 172.21.15.18:0/2862709740 learned_addr learned my addr 172.21.15.18:0/2862709740 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b98ff9640 1 -- 172.21.15.18:0/4223708313 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7b840f0a70 con 0x7f7b9c1023a0 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/2862709740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d50a77b0 0x5560d50a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9affd640 1 -- 172.21.15.18:0/2862709740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d50a77b0 msgr2=0x5560d50a9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9affd640 1 --2- 172.21.15.18:0/2862709740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d50a77b0 0x5560d50a9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9affd640 1 -- 172.21.15.18:0/2862709740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 msgr2=0x5560d509e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9affd640 1 --2- 172.21.15.18:0/2862709740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d509e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9affd640 1 -- 172.21.15.18:0/2862709740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560d4fc4570 con 0x5560d509ebd0 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/2862709740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d509e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9affd640 1 --2- 172.21.15.18:0/2862709740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 0x5560d50a7270 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c013c40 tx=0x7f7b8c009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6be9254f7fe3bc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b9b7fe640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b7c0a2230 0x7f7b7c0a46f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b840ee3d0 tx=0x7f7b84000f40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.819+0000 7f7b91ffb640 1 -- 172.21.15.18:0/2862709740 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c018070 con 0x5560d509ebd0 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.820+0000 7f7b91ffb640 1 -- 172.21.15.18:0/2862709740 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c00ee70 con 0x5560d509ebd0 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.820+0000 7f7b91ffb640 1 -- 172.21.15.18:0/2862709740 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c01cca0 con 0x5560d509ebd0 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.820+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2862709740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 msgr2=0x5560d50a7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:08.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.820+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2862709740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 0x5560d50a7270 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c013c40 tx=0x7f7b8c009b70 comp rx=0 tx=0).stop 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.820+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2862709740 shutdown_connections 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.820+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2862709740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d50a77b0 0x5560d50a9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.820+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2862709740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 0x5560d50a7270 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.820+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2862709740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d509e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.820+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2862709740 >> 172.21.15.18:0/2862709740 conn(0x5560d5098990 msgr2=0x5560d5098d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.820+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2862709740 shutdown_connections 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.820+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2862709740 wait complete. 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7ba4bcf740 1 -- start start 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7ba4bcf740 1 Processor -- start 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7ba4bcf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d509e2b0 0x5560d50a7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7ba4bcf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509ebd0 0x5560d50a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7ba4bcf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d50a77b0 0x5560d50a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7ba4bcf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d4cd9430 con 0x5560d50a77b0 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7ba4bcf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d4c95e00 con 0x5560d509ebd0 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7ba4bcf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d4f8afd0 con 0x5560d509e2b0 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7ba4bcf740 10 client.0 _ll_register_callbacks cb 0x5560d4fe0fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7b9bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d50a77b0 0x5560d50a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7b9bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d50a77b0 0x5560d50a4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35610/0 (socket says 172.21.15.18:35610) 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7b9affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509ebd0 0x5560d50a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7b9bfff640 1 -- 172.21.15.18:0/2002204573 learned_addr learned my addr 172.21.15.18:0/2002204573 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.821+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d509e2b0 0x5560d50a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b9bfff640 1 -- 172.21.15.18:0/2002204573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d509e2b0 msgr2=0x5560d50a7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d509e2b0 0x5560d50a7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b9bfff640 1 -- 172.21.15.18:0/2002204573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509ebd0 msgr2=0x5560d50a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509ebd0 0x5560d50a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b9bfff640 1 -- 172.21.15.18:0/2002204573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c013920 con 0x5560d50a77b0 2024-02-17T12:55:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b9affd640 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509ebd0 0x5560d50a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d50a77b0 0x5560d50a4cc0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400f740 tx=0x7f7b94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b185c15142995958 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b8affd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b8affd640 1 -- 172.21.15.18:0/2002204573 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94016070 con 0x5560d50a77b0 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b8affd640 1 -- 172.21.15.18:0/2002204573 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7b94013310 con 0x5560d50a77b0 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2002204573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560d4d057e0 con 0x5560d50a77b0 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b8affd640 1 -- 172.21.15.18:0/2002204573 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b940134d0 con 0x5560d50a77b0 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7ba4bcf740 10 client.17385 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2002204573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5560d5004610 con 0x5560d50a77b0 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b8affd640 1 -- 172.21.15.18:0/2002204573 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b9402e080 con 0x5560d50a77b0 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7b8affd640 1 -- 172.21.15.18:0/2002204573 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f7b94011ac0 con 0x5560d50a77b0 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7ba4bcf740 10 client.17385 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7ba4bcf740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.822+0000 7f7ba4bcf740 20 client.17385 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.823+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5560d50b7a80 0x5560d50b9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.823+0000 7f7ba4bcf740 4 client.17385 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.823+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2002204573 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560d4d42720 con 0x5560d50b7a80 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.823+0000 7f7b9affd640 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5560d50b7a80 0x5560d50b9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.823+0000 7f7b9affd640 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5560d50b7a80 0x5560d50b9ea0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa74c6f2a793ae49 server_cookie=d36498a51533f805 in_seq=0 out_seq=0 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.823+0000 7f7b8affd640 10 client.17385 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7b8affd640 1 -- 172.21.15.18:0/2002204573 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5560d4d42720 con 0x5560d50b7a80 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7b8affd640 10 client.17385 handle_command_reply: tid=0 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7ba4bcf740 1 client.17385 shutdown 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2002204573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d50a77b0 msgr2=0x5560d50a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d50a77b0 0x5560d50a4cc0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400f740 tx=0x7f7b94004730 comp rx=0 tx=0).stop 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2002204573 shutdown_connections 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5560d50b7a80 0x5560d50b9ea0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d50a77b0 0x5560d50a4cc0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509ebd0 0x5560d50a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2002204573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d509e2b0 0x5560d50a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2002204573 >> 172.21.15.18:0/2002204573 conn(0x5560d5098990 msgr2=0x5560d509a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7b8affd640 0 client.17385 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2002204573 shutdown_connections 2024-02-17T12:55:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.824+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2002204573 wait complete. 2024-02-17T12:55:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.825+0000 7f7ba4bcf740 20 client.17385 trim_cache size 0 max 16384 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.841+0000 7f7ba4bcf740 1 Processor -- start 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.841+0000 7f7ba4bcf740 1 -- start start 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.841+0000 7f7ba4bcf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d50a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.841+0000 7f7ba4bcf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 0x5560d50a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.841+0000 7f7ba4bcf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d50a77b0 0x5560d50a7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.841+0000 7f7ba4bcf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d50a43c0 con 0x5560d509ebd0 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.841+0000 7f7ba4bcf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d50a52c0 con 0x5560d509e2b0 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.841+0000 7f7ba4bcf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d50999e0 con 0x5560d50a77b0 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.841+0000 7f7b9affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 0x5560d50a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.841+0000 7f7b9affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 0x5560d50a6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35618/0 (socket says 172.21.15.18:35618) 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.841+0000 7f7b9affd640 1 -- 172.21.15.18:0/4136012072 learned_addr learned my addr 172.21.15.18:0/4136012072 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.842+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/4136012072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d50a77b0 0x5560d50a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.842+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/4136012072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d50a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.842+0000 7f7b9affd640 1 -- 172.21.15.18:0/4136012072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d50a77b0 msgr2=0x5560d50a7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.842+0000 7f7b9affd640 1 --2- 172.21.15.18:0/4136012072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d50a77b0 0x5560d50a7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.842+0000 7f7b9affd640 1 -- 172.21.15.18:0/4136012072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 msgr2=0x5560d50a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.842+0000 7f7b9affd640 1 --2- 172.21.15.18:0/4136012072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d50a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.842+0000 7f7b9affd640 1 -- 172.21.15.18:0/4136012072 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560d4fa0ac0 con 0x5560d509ebd0 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.842+0000 7f7b9affd640 1 --2- 172.21.15.18:0/4136012072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 0x5560d50a6cd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c015010 tx=0x7f7b8c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0739c3f6a64dac server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.842+0000 7f7b917fa640 1 -- 172.21.15.18:0/4136012072 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c018070 con 0x5560d509ebd0 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.842+0000 7f7b917fa640 1 -- 172.21.15.18:0/4136012072 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c019040 con 0x5560d509ebd0 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.842+0000 7f7b917fa640 1 -- 172.21.15.18:0/4136012072 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c02c070 con 0x5560d509ebd0 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.843+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/4136012072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 msgr2=0x5560d50a6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.843+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/4136012072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 0x5560d50a6cd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c015010 tx=0x7f7b8c01c760 comp rx=0 tx=0).stop 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.843+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/4136012072 shutdown_connections 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.843+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/4136012072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d50a77b0 0x5560d50a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.843+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/4136012072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d509ebd0 0x5560d50a6cd0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.843+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/4136012072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d50a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.843+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/4136012072 >> 172.21.15.18:0/4136012072 conn(0x5560d5098990 msgr2=0x5560d50be7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.843+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/4136012072 shutdown_connections 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.843+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/4136012072 wait complete. 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.843+0000 7f7ba4bcf740 1 -- start start 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.843+0000 7f7ba4bcf740 1 Processor -- start 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7ba4bcf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d50ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7ba4bcf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d509ebd0 0x5560d50acc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7ba4bcf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d50a77b0 0x5560d50ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7ba4bcf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d50ad6d0 con 0x5560d50a77b0 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7ba4bcf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d50ad840 con 0x5560d509e2b0 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7ba4bcf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560d50ada00 con 0x5560d509ebd0 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7ba4bcf740 10 client.0 _ll_register_callbacks cb 0x7f7b8c0137d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b9bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d50ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b9affd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d509ebd0 0x5560d50acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b9bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d50ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:58998/0 (socket says 172.21.15.18:58998) 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b9bfff640 1 -- 172.21.15.18:0/2097811735 learned_addr learned my addr 172.21.15.18:0/2097811735 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/2097811735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d50a77b0 0x5560d50ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b9bfff640 1 -- 172.21.15.18:0/2097811735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d509ebd0 msgr2=0x5560d50acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/2097811735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d509ebd0 0x5560d50acc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b9bfff640 1 -- 172.21.15.18:0/2097811735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d50a77b0 msgr2=0x5560d50ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/2097811735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d50a77b0 0x5560d50ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b9bfff640 1 -- 172.21.15.18:0/2097811735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560d4fc4570 con 0x5560d509e2b0 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b9bfff640 1 --2- 172.21.15.18:0/2097811735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d50ae850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7b94013920 tx=0x7f7b940096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48a21d48f3da2334 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b8affd640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.844+0000 7f7b8affd640 1 -- 172.21.15.18:0/2097811735 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94016040 con 0x5560d509e2b0 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7b8affd640 1 -- 172.21.15.18:0/2097811735 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7b94000e40 con 0x5560d509e2b0 2024-02-17T12:55:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2097811735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b8c00ee70 con 0x5560d509e2b0 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7b8affd640 1 -- 172.21.15.18:0/2097811735 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b94009c00 con 0x5560d509e2b0 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7ba4bcf740 10 client.26723 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2097811735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5560d4fc66a0 con 0x5560d509e2b0 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7b8affd640 1 -- 172.21.15.18:0/2097811735 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c00ee70 con 0x5560d509e2b0 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7b8affd640 1 -- 172.21.15.18:0/2097811735 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f7b94011970 con 0x5560d509e2b0 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7ba4bcf740 10 client.26723 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7ba4bcf740 10 client.26723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7ba4bcf740 20 client.26723 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2097811735 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5560d510f390 0x5560d51117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7ba4bcf740 4 client.26723 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2097811735 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5560d4d91890 con 0x5560d510f390 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.845+0000 7f7b9affd640 1 --2- 172.21.15.18:0/2097811735 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5560d510f390 0x5560d51117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7b8affd640 10 client.26723 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7b9affd640 1 --2- 172.21.15.18:0/2097811735 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5560d510f390 0x5560d51117b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ada2db7c0766a524 server_cookie=881869de553d8b50 in_seq=0 out_seq=0 2024-02-17T12:55:08.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7b8affd640 1 -- 172.21.15.18:0/2097811735 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5560d4d91890 con 0x5560d510f390 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7b8affd640 10 client.26723 handle_command_reply: tid=0 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7ba4bcf740 1 client.26723 shutdown 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2097811735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 msgr2=0x5560d50ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2097811735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d50ae850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7b94013920 tx=0x7f7b940096a0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2097811735 shutdown_connections 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2097811735 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5560d510f390 0x5560d51117b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2097811735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560d50a77b0 0x5560d50ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2097811735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560d509ebd0 0x5560d50acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7b8affd640 0 client.26723 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7ba4bcf740 1 --2- 172.21.15.18:0/2097811735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5560d509e2b0 0x5560d50ae850 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2097811735 >> 172.21.15.18:0/2097811735 conn(0x5560d5098990 msgr2=0x5560d50971c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2097811735 shutdown_connections 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7ba4bcf740 1 -- 172.21.15.18:0/2097811735 wait complete. 2024-02-17T12:55:08.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.846+0000 7f7ba4bcf740 20 client.26723 trim_cache size 0 max 16384 2024-02-17T12:55:08.844 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:55:08.844 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:55:08.844 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:55:08.844 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:55:08.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.847+0000 7f7ba1f27640 1 -- 172.21.15.18:0/4223708313 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b7c0a2230 msgr2=0x7f7b7c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:08.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.847+0000 7f7ba1f27640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b7c0a2230 0x7f7b7c0a46f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b840ee3d0 tx=0x7f7b84000f40 comp rx=0 tx=0).stop 2024-02-17T12:55:08.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.847+0000 7f7ba1f27640 1 -- 172.21.15.18:0/4223708313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b9c1023a0 msgr2=0x7f7b9c197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:08.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.847+0000 7f7ba1f27640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c197780 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7b84004af0 tx=0x7f7b840346f0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.847+0000 7f7ba1f27640 1 -- 172.21.15.18:0/4223708313 shutdown_connections 2024-02-17T12:55:08.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.847+0000 7f7ba1f27640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b7c0a2230 0x7f7b7c0a46f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.847+0000 7f7ba1f27640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b9c10b6e0 0x7f7b9c19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.847+0000 7f7ba1f27640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9c102cc0 0x7f7b9c197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.847+0000 7f7ba1f27640 1 --2- 172.21.15.18:0/4223708313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c197780 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.847+0000 7f7ba1f27640 1 -- 172.21.15.18:0/4223708313 >> 172.21.15.18:0/4223708313 conn(0x7f7b9c077230 msgr2=0x7f7b9c0fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:08.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.847+0000 7f7ba1f27640 1 -- 172.21.15.18:0/4223708313 shutdown_connections 2024-02-17T12:55:08.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.847+0000 7f7ba1f27640 1 -- 172.21.15.18:0/4223708313 wait complete. 2024-02-17T12:55:08.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:08 smithi119 ceph-mon[36188]: pgmap v160: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-02-17T12:55:08.847 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:08 smithi018 ceph-mon[32625]: pgmap v160: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-02-17T12:55:08.848 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:55:08.849 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:55:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:08 smithi146 ceph-mon[36454]: pgmap v160: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-02-17T12:55:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.967+0000 7f17f9ceb640 1 -- 172.21.15.18:0/3586687398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17f41002f0 msgr2=0x7f17f41006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.967+0000 7f17f9ceb640 1 --2- 172.21.15.18:0/3586687398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17f41002f0 0x7f17f41006d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009900 tx=0x7f17e402bc30 comp rx=0 tx=0).stop 2024-02-17T12:55:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.967+0000 7f17f9ceb640 1 -- 172.21.15.18:0/3586687398 shutdown_connections 2024-02-17T12:55:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.967+0000 7f17f9ceb640 1 --2- 172.21.15.18:0/3586687398 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17f41053c0 0x7f17f410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.967+0000 7f17f9ceb640 1 --2- 172.21.15.18:0/3586687398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17f4100ca0 0x7f17f4104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.967+0000 7f17f9ceb640 1 --2- 172.21.15.18:0/3586687398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17f41002f0 0x7f17f41006d0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.967+0000 7f17f9ceb640 1 -- 172.21.15.18:0/3586687398 >> 172.21.15.18:0/3586687398 conn(0x7f17f4077180 msgr2=0x7f17f4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.967+0000 7f17f9ceb640 1 -- 172.21.15.18:0/3586687398 shutdown_connections 2024-02-17T12:55:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.967+0000 7f17f9ceb640 1 -- 172.21.15.18:0/3586687398 wait complete. 2024-02-17T12:55:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.967+0000 7f17f9ceb640 1 Processor -- start 2024-02-17T12:55:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.967+0000 7f17f9ceb640 1 -- start start 2024-02-17T12:55:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f9ceb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17f41002f0 0x7f17f419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f9ceb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17f41053c0 0x7f17f419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f9ceb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17f4197670 0x7f17f4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f9ceb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f4198140 con 0x7f17f4197670 2024-02-17T12:55:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f9ceb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f41982b0 con 0x7f17f41053c0 2024-02-17T12:55:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f9ceb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f4198470 con 0x7f17f41002f0 2024-02-17T12:55:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17f4197670 0x7f17f4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17f4197670 0x7f17f4197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35662/0 (socket says 172.21.15.18:35662) 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f37fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17f41002f0 0x7f17f419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f3fff640 1 -- 172.21.15.18:0/2005347712 learned_addr learned my addr 172.21.15.18:0/2005347712 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f2ffd640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17f41053c0 0x7f17f419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f3fff640 1 -- 172.21.15.18:0/2005347712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17f41002f0 msgr2=0x7f17f419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f3fff640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17f41002f0 0x7f17f419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f3fff640 1 -- 172.21.15.18:0/2005347712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17f41053c0 msgr2=0x7f17f419daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f3fff640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17f41053c0 0x7f17f419daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f3fff640 1 -- 172.21.15.18:0/2005347712 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e4009560 con 0x7f17f4197670 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f3fff640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17f4197670 0x7f17f4197af0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00eb10 tx=0x7f17ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f0ff9640 1 -- 172.21.15.18:0/2005347712 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec004b20 con 0x7f17f4197670 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f0ff9640 1 -- 172.21.15.18:0/2005347712 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17ec004c80 con 0x7f17f4197670 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f9ceb640 1 -- 172.21.15.18:0/2005347712 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f41a4410 con 0x7f17f4197670 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f0ff9640 1 -- 172.21.15.18:0/2005347712 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec010650 con 0x7f17f4197670 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.968+0000 7f17f9ceb640 1 -- 172.21.15.18:0/2005347712 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f41a48f0 con 0x7f17f4197670 2024-02-17T12:55:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.969+0000 7f17f9ceb640 1 -- 172.21.15.18:0/2005347712 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f41017c0 con 0x7f17f4197670 2024-02-17T12:55:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.970+0000 7f17f0ff9640 1 -- 172.21.15.18:0/2005347712 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f17ec005020 con 0x7f17f4197670 2024-02-17T12:55:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.971+0000 7f17f0ff9640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f17dc0a2230 0x7f17dc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.971+0000 7f17f37fe640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f17dc0a2230 0x7f17dc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.971+0000 7f17f0ff9640 1 -- 172.21.15.18:0/2005347712 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f17ec014070 con 0x7f17f4197670 2024-02-17T12:55:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.972+0000 7f17f37fe640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f17dc0a2230 0x7f17dc0a46f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f17e4006010 tx=0x7f17e4005e40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:08.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:08.976+0000 7f17f0ff9640 1 -- 172.21.15.18:0/2005347712 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17ec0971f0 con 0x7f17f4197670 2024-02-17T12:55:09.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.211+0000 7f17f9ceb640 1 -- 172.21.15.18:0/2005347712 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17f4108690 con 0x7f17f4197670 2024-02-17T12:55:09.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.211+0000 7f17f0ff9640 1 -- 172.21.15.18:0/2005347712 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11402 (secure 0 0 0) 0x7f17ec09a8b0 con 0x7f17f4197670 2024-02-17T12:55:09.202 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:55:09.202 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:54:39.938860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3061839650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3061839650},{"type":"v1","addr":"172.21.15.146:6839","nonce":3061839650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:55:09.203 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 35 2024-02-17T12:55:09.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.214+0000 7f17f9ceb640 1 -- 172.21.15.18:0/2005347712 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f17dc0a2230 msgr2=0x7f17dc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.214+0000 7f17f9ceb640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f17dc0a2230 0x7f17dc0a46f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f17e4006010 tx=0x7f17e4005e40 comp rx=0 tx=0).stop 2024-02-17T12:55:09.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.214+0000 7f17f9ceb640 1 -- 172.21.15.18:0/2005347712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17f4197670 msgr2=0x7f17f4197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.214+0000 7f17f9ceb640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17f4197670 0x7f17f4197af0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00eb10 tx=0x7f17ec00c2d0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.214+0000 7f17f9ceb640 1 -- 172.21.15.18:0/2005347712 shutdown_connections 2024-02-17T12:55:09.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.214+0000 7f17f9ceb640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f17dc0a2230 0x7f17dc0a46f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.214+0000 7f17f9ceb640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17f4197670 0x7f17f4197af0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.214+0000 7f17f9ceb640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17f41053c0 0x7f17f419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.214+0000 7f17f9ceb640 1 --2- 172.21.15.18:0/2005347712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17f41002f0 0x7f17f419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.214+0000 7f17f9ceb640 1 -- 172.21.15.18:0/2005347712 >> 172.21.15.18:0/2005347712 conn(0x7f17f4077180 msgr2=0x7f17f40fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.214+0000 7f17f9ceb640 1 -- 172.21.15.18:0/2005347712 shutdown_connections 2024-02-17T12:55:09.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.214+0000 7f17f9ceb640 1 -- 172.21.15.18:0/2005347712 wait complete. 2024-02-17T12:55:09.216 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:55:09.216 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:55:09.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 287437 6.81 MB/sec execute 144 sec latency 3248.206 ms 2024-02-17T12:55:09.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.334+0000 7fe085264640 1 -- 172.21.15.18:0/2820920289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe080105350 msgr2=0x7fe08010cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.334+0000 7fe085264640 1 --2- 172.21.15.18:0/2820920289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe080105350 0x7fe08010cf20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe078009cd0 tx=0x7fe07802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.334+0000 7fe085264640 1 -- 172.21.15.18:0/2820920289 shutdown_connections 2024-02-17T12:55:09.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.334+0000 7fe085264640 1 --2- 172.21.15.18:0/2820920289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe080105350 0x7fe08010cf20 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.334+0000 7fe085264640 1 --2- 172.21.15.18:0/2820920289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe080100c30 0x7fe080104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.334+0000 7fe085264640 1 --2- 172.21.15.18:0/2820920289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe080100280 0x7fe080100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.334+0000 7fe085264640 1 -- 172.21.15.18:0/2820920289 >> 172.21.15.18:0/2820920289 conn(0x7fe080077150 msgr2=0x7fe080077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.334+0000 7fe085264640 1 -- 172.21.15.18:0/2820920289 shutdown_connections 2024-02-17T12:55:09.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.334+0000 7fe085264640 1 -- 172.21.15.18:0/2820920289 wait complete. 2024-02-17T12:55:09.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.334+0000 7fe085264640 1 Processor -- start 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe085264640 1 -- start start 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe085264640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe080100280 0x7fe08019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe085264640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe080105350 0x7fe08019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe085264640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0801975a0 0x7fe080197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe085264640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe080197f90 con 0x7fe0801975a0 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe085264640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe080198100 con 0x7fe080105350 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe085264640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0801982c0 con 0x7fe080100280 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe07f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0801975a0 0x7fe080197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe07effd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe080100280 0x7fe08019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe07e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe080105350 0x7fe08019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe07f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0801975a0 0x7fe080197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35674/0 (socket says 172.21.15.18:35674) 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe07f7fe640 1 -- 172.21.15.18:0/1217868022 learned_addr learned my addr 172.21.15.18:0/1217868022 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe07e7fc640 1 -- 172.21.15.18:0/1217868022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe080100280 msgr2=0x7fe08019d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe080100280 0x7fe08019d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe07e7fc640 1 -- 172.21.15.18:0/1217868022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0801975a0 msgr2=0x7fe080197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0801975a0 0x7fe080197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe07e7fc640 1 -- 172.21.15.18:0/1217868022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe078009900 con 0x7fe080105350 2024-02-17T12:55:09.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe07f7fe640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0801975a0 0x7fe080197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.335+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe080105350 0x7fe08019d930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c940 tx=0x7fe07400ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.336+0000 7fe067fff640 1 -- 172.21.15.18:0/1217868022 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074007bb0 con 0x7fe080105350 2024-02-17T12:55:09.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.336+0000 7fe085264640 1 -- 172.21.15.18:0/1217868022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0801a42b0 con 0x7fe080105350 2024-02-17T12:55:09.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.336+0000 7fe067fff640 1 -- 172.21.15.18:0/1217868022 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe074007d10 con 0x7fe080105350 2024-02-17T12:55:09.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.336+0000 7fe067fff640 1 -- 172.21.15.18:0/1217868022 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074004e60 con 0x7fe080105350 2024-02-17T12:55:09.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.336+0000 7fe085264640 1 -- 172.21.15.18:0/1217868022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0801a4770 con 0x7fe080105350 2024-02-17T12:55:09.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.338+0000 7fe067fff640 1 -- 172.21.15.18:0/1217868022 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe074005020 con 0x7fe080105350 2024-02-17T12:55:09.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.338+0000 7fe087e37740 1 Processor -- start 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.338+0000 7fe087e37740 1 -- start start 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.338+0000 7fe087e37740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca94250 0x56251ca94630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.338+0000 7fe087e37740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 0x56251ca9d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.338+0000 7fe087e37740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca9d750 0x56251ca9fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.338+0000 7fe087e37740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56251ca4f960 con 0x56251ca94b70 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.338+0000 7fe087e37740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56251ca03310 con 0x56251ca9d750 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.338+0000 7fe087e37740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56251ca00840 con 0x56251ca94250 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.338+0000 7fe067fff640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0680a2230 0x7fe0680a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 0x56251ca9d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 0x56251ca9d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35690/0 (socket says 172.21.15.18:35690) 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07effd640 1 -- 172.21.15.18:0/687486910 learned_addr learned my addr 172.21.15.18:0/687486910 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/687486910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca9d750 0x56251ca9fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07f7fe640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0680a2230 0x7fe0680a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07f7fe640 1 --2- 172.21.15.18:0/687486910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca94250 0x56251ca94630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07effd640 1 -- 172.21.15.18:0/687486910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca94250 msgr2=0x56251ca94630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07effd640 1 --2- 172.21.15.18:0/687486910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca94250 0x56251ca94630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07effd640 1 -- 172.21.15.18:0/687486910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca9d750 msgr2=0x56251ca9fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07effd640 1 --2- 172.21.15.18:0/687486910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca9d750 0x56251ca9fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07effd640 1 -- 172.21.15.18:0/687486910 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56251c9be060 con 0x56251ca94b70 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe067fff640 1 -- 172.21.15.18:0/1217868022 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe0740cdbc0 con 0x7fe080105350 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07effd640 1 --2- 172.21.15.18:0/687486910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 0x56251ca9d210 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe070009cf0 tx=0x7fe070011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649073507cc5362b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe0657fa640 1 -- 172.21.15.18:0/687486910 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe070009ea0 con 0x56251ca94b70 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe07f7fe640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0680a2230 0x7fe0680a46f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe078006010 tx=0x7fe0780347f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe0657fa640 1 -- 172.21.15.18:0/687486910 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe070004230 con 0x56251ca94b70 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.339+0000 7fe0657fa640 1 -- 172.21.15.18:0/687486910 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe070019880 con 0x56251ca94b70 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.340+0000 7fe087e37740 1 -- 172.21.15.18:0/687486910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 msgr2=0x56251ca9d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.340+0000 7fe087e37740 1 --2- 172.21.15.18:0/687486910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 0x56251ca9d210 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe070009cf0 tx=0x7fe070011810 comp rx=0 tx=0).stop 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.340+0000 7fe087e37740 1 -- 172.21.15.18:0/687486910 shutdown_connections 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.340+0000 7fe087e37740 1 --2- 172.21.15.18:0/687486910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca9d750 0x56251ca9fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.340+0000 7fe087e37740 1 --2- 172.21.15.18:0/687486910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 0x56251ca9d210 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.340+0000 7fe087e37740 1 --2- 172.21.15.18:0/687486910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca94250 0x56251ca94630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.340+0000 7fe087e37740 1 -- 172.21.15.18:0/687486910 >> 172.21.15.18:0/687486910 conn(0x56251ca8e980 msgr2=0x56251ca8ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.340+0000 7fe087e37740 1 -- 172.21.15.18:0/687486910 shutdown_connections 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.340+0000 7fe087e37740 1 -- 172.21.15.18:0/687486910 wait complete. 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.340+0000 7fe087e37740 1 -- start start 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.340+0000 7fe087e37740 1 Processor -- start 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe087e37740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94250 0x56251ca9a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe087e37740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca94b70 0x56251ca9ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe087e37740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca9d750 0x56251ca9b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe087e37740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56251c9fc020 con 0x56251ca94250 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe087e37740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56251c6cf430 con 0x56251ca94b70 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe087e37740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56251c68be00 con 0x56251ca9d750 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe07effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94250 0x56251ca9a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe07effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94250 0x56251ca9a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35694/0 (socket says 172.21.15.18:35694) 2024-02-17T12:55:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe07effd640 1 -- 172.21.15.18:0/3391659635 learned_addr learned my addr 172.21.15.18:0/3391659635 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe07f7fe640 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca9d750 0x56251ca9b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe087e37740 10 client.0 _ll_register_callbacks cb 0x56251c9a3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca94b70 0x56251ca9ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe07effd640 1 -- 172.21.15.18:0/3391659635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca9d750 msgr2=0x56251ca9b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe07effd640 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca9d750 0x56251ca9b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe07effd640 1 -- 172.21.15.18:0/3391659635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca94b70 msgr2=0x56251ca9ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe07effd640 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca94b70 0x56251ca9ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe07effd640 1 -- 172.21.15.18:0/3391659635 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe070009950 con 0x56251ca94250 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca94b70 0x56251ca9ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe087e37740 1 -- 172.21.15.18:0/3391659635 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56251c6fb7e0 con 0x56251ca94250 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe07effd640 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94250 0x56251ca9a7a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe070005750 tx=0x7fe070005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f105c2b39e4febc0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe062ff5640 10 client.17409 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe062ff5640 1 -- 172.21.15.18:0/3391659635 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe070009ea0 con 0x56251ca94250 2024-02-17T12:55:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.341+0000 7fe062ff5640 1 -- 172.21.15.18:0/3391659635 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe0700035f0 con 0x56251ca94250 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.342+0000 7fe062ff5640 1 -- 172.21.15.18:0/3391659635 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe070003830 con 0x56251ca94250 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.342+0000 7fe087e37740 10 client.17409 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.342+0000 7fe087e37740 1 -- 172.21.15.18:0/3391659635 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({fsmap=35}) v3 -- 0x56251c9fa6a0 con 0x56251ca94250 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.342+0000 7fe062ff5640 1 -- 172.21.15.18:0/3391659635 <== mon.0 v2:172.21.15.18:3300/0 4 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe070021040 con 0x56251ca94250 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.342+0000 7fe087e37740 10 client.17409 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.342+0000 7fe087e37740 10 client.17409 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.342+0000 7fe087e37740 20 client.17409 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.342+0000 7fe087e37740 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56251caad9b0 0x56251caafde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.342+0000 7fe087e37740 4 client.17409 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.342+0000 7fe087e37740 1 -- 172.21.15.18:0/3391659635 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56251c738720 con 0x56251caad9b0 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.342+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56251caad9b0 0x56251caafde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56251caad9b0 0x56251caafde0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fe3d652b688857ff server_cookie=6906f46a6b16da43 in_seq=0 out_seq=0 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe062ff5640 10 client.17409 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe062ff5640 1 -- 172.21.15.18:0/3391659635 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56251c738720 con 0x56251caad9b0 2024-02-17T12:55:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe062ff5640 10 client.17409 handle_command_reply: tid=0 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe087e37740 1 client.17409 shutdown 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe087e37740 1 -- 172.21.15.18:0/3391659635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94250 msgr2=0x56251ca9a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe087e37740 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94250 0x56251ca9a7a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe070005750 tx=0x7fe070005780 comp rx=0 tx=0).stop 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe087e37740 1 -- 172.21.15.18:0/3391659635 shutdown_connections 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe087e37740 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca9d750 0x56251ca9b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe087e37740 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56251caad9b0 0x56251caafde0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe087e37740 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca94b70 0x56251ca9ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe087e37740 1 --2- 172.21.15.18:0/3391659635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94250 0x56251ca9a7a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe087e37740 1 -- 172.21.15.18:0/3391659635 >> 172.21.15.18:0/3391659635 conn(0x56251ca8e980 msgr2=0x56251ca906d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.343+0000 7fe062ff5640 0 client.17409 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.344+0000 7fe087e37740 1 -- 172.21.15.18:0/3391659635 shutdown_connections 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.344+0000 7fe087e37740 1 -- 172.21.15.18:0/3391659635 wait complete. 2024-02-17T12:55:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.344+0000 7fe087e37740 20 client.17409 trim_cache size 0 max 16384 2024-02-17T12:55:09.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe087e37740 1 Processor -- start 2024-02-17T12:55:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe087e37740 1 -- start start 2024-02-17T12:55:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe087e37740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca94b70 0x56251ca9ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe087e37740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca9d750 0x56251ca9b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe087e37740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca95a90 0x56251ca95f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe087e37740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56251c981130 con 0x56251ca9d750 2024-02-17T12:55:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe087e37740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56251ca8e560 con 0x56251ca95a90 2024-02-17T12:55:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe087e37740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56251ca9b5b0 con 0x56251ca94b70 2024-02-17T12:55:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe07e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca9d750 0x56251ca9b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe07e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca9d750 0x56251ca9b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35704/0 (socket says 172.21.15.18:35704) 2024-02-17T12:55:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe07e7fc640 1 -- 172.21.15.18:0/505133721 learned_addr learned my addr 172.21.15.18:0/505133721 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe07effd640 1 --2- 172.21.15.18:0/505133721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca94b70 0x56251ca9ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe07f7fe640 1 --2- 172.21.15.18:0/505133721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca95a90 0x56251ca95f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe07e7fc640 1 -- 172.21.15.18:0/505133721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca94b70 msgr2=0x56251ca9ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/505133721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca94b70 0x56251ca9ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe07e7fc640 1 -- 172.21.15.18:0/505133721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca95a90 msgr2=0x56251ca95f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/505133721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca95a90 0x56251ca95f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe07e7fc640 1 -- 172.21.15.18:0/505133721 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56251c996c20 con 0x56251ca9d750 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/505133721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca9d750 0x56251ca9b070 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe0740cb960 tx=0x7fe074085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf3d3d30fe3a512 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.356+0000 7fe064ff9640 1 -- 172.21.15.18:0/505133721 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07409e750 con 0x56251ca9d750 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.357+0000 7fe064ff9640 1 -- 172.21.15.18:0/505133721 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0740a2da0 con 0x56251ca9d750 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.357+0000 7fe064ff9640 1 -- 172.21.15.18:0/505133721 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0740ba210 con 0x56251ca9d750 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.357+0000 7fe087e37740 1 -- 172.21.15.18:0/505133721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca9d750 msgr2=0x56251ca9b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.357+0000 7fe087e37740 1 --2- 172.21.15.18:0/505133721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca9d750 0x56251ca9b070 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe0740cb960 tx=0x7fe074085040 comp rx=0 tx=0).stop 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.357+0000 7fe087e37740 1 -- 172.21.15.18:0/505133721 shutdown_connections 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.357+0000 7fe087e37740 1 --2- 172.21.15.18:0/505133721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca95a90 0x56251ca95f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.357+0000 7fe087e37740 1 --2- 172.21.15.18:0/505133721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca9d750 0x56251ca9b070 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.357+0000 7fe087e37740 1 --2- 172.21.15.18:0/505133721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca94b70 0x56251ca9ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.357+0000 7fe087e37740 1 -- 172.21.15.18:0/505133721 >> 172.21.15.18:0/505133721 conn(0x56251ca8e980 msgr2=0x56251cab4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.357+0000 7fe087e37740 1 -- 172.21.15.18:0/505133721 shutdown_connections 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.357+0000 7fe087e37740 1 -- 172.21.15.18:0/505133721 wait complete. 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe087e37740 1 -- start start 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe087e37740 1 Processor -- start 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe087e37740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 0x56251cafe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe087e37740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca95a90 0x56251cafed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe087e37740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca9d750 0x56251caa3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe087e37740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56251caa42f0 con 0x56251ca94b70 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe087e37740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56251caa4460 con 0x56251ca95a90 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe087e37740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56251caa4620 con 0x56251ca9d750 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe087e37740 10 client.0 _ll_register_callbacks cb 0x7fe0700097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe07effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 0x56251cafe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe07effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 0x56251cafe7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35706/0 (socket says 172.21.15.18:35706) 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe07effd640 1 -- 172.21.15.18:0/852686865 learned_addr learned my addr 172.21.15.18:0/852686865 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe07f7fe640 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca9d750 0x56251caa3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca95a90 0x56251cafed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe07effd640 1 -- 172.21.15.18:0/852686865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca9d750 msgr2=0x56251caa3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe07effd640 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca9d750 0x56251caa3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe07effd640 1 -- 172.21.15.18:0/852686865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca95a90 msgr2=0x56251cafed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe07effd640 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca95a90 0x56251cafed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe07effd640 1 -- 172.21.15.18:0/852686865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0801a4770 con 0x56251ca94b70 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe07effd640 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 0x56251cafe7e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x56251ca9b8b0 tx=0x7fe070005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4296b62c5eca779e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.358+0000 7fe062ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe062ff5640 1 -- 172.21.15.18:0/852686865 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0700185b0 con 0x56251ca94b70 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe062ff5640 1 -- 172.21.15.18:0/852686865 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe070000e40 con 0x56251ca94b70 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe087e37740 1 -- 172.21.15.18:0/852686865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe070004230 con 0x56251ca94b70 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe062ff5640 1 -- 172.21.15.18:0/852686865 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe070018710 con 0x56251ca94b70 2024-02-17T12:55:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe087e37740 10 client.17415 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe087e37740 1 -- 172.21.15.18:0/852686865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56251c9bc8c0 con 0x56251ca94b70 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe062ff5640 1 -- 172.21.15.18:0/852686865 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe070004230 con 0x56251ca94b70 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe062ff5640 1 -- 172.21.15.18:0/852686865 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe070021030 con 0x56251ca94b70 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe087e37740 10 client.17415 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe087e37740 10 client.17415 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe087e37740 20 client.17415 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe087e37740 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56251cb06200 0x56251cb085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe087e37740 4 client.17415 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe087e37740 1 -- 172.21.15.18:0/852686865 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56251c787890 con 0x56251cb06200 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.359+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56251cb06200 0x56251cb085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe062ff5640 10 client.17415 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe07e7fc640 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56251cb06200 0x56251cb085f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b7e480bd95f0f918 server_cookie=9c65597712166f03 in_seq=0 out_seq=0 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe062ff5640 1 -- 172.21.15.18:0/852686865 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56251c787890 con 0x56251cb06200 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe062ff5640 10 client.17415 handle_command_reply: tid=0 2024-02-17T12:55:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe087e37740 1 client.17415 shutdown 2024-02-17T12:55:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe087e37740 1 -- 172.21.15.18:0/852686865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 msgr2=0x56251cafe7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe087e37740 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 0x56251cafe7e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x56251ca9b8b0 tx=0x7fe070005f90 comp rx=0 tx=0).stop 2024-02-17T12:55:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe087e37740 1 -- 172.21.15.18:0/852686865 shutdown_connections 2024-02-17T12:55:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe087e37740 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56251ca9d750 0x56251caa3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe087e37740 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56251cb06200 0x56251cb085f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe087e37740 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56251ca95a90 0x56251cafed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe087e37740 1 --2- 172.21.15.18:0/852686865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56251ca94b70 0x56251cafe7e0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe087e37740 1 -- 172.21.15.18:0/852686865 >> 172.21.15.18:0/852686865 conn(0x56251ca8e980 msgr2=0x56251ca8d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe062ff5640 0 client.17415 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:55:09.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe087e37740 1 -- 172.21.15.18:0/852686865 shutdown_connections 2024-02-17T12:55:09.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe087e37740 1 -- 172.21.15.18:0/852686865 wait complete. 2024-02-17T12:55:09.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.360+0000 7fe087e37740 20 client.17415 trim_cache size 0 max 16384 2024-02-17T12:55:09.361 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:55:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.361+0000 7fe085264640 1 -- 172.21.15.18:0/1217868022 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0680a2230 msgr2=0x7fe0680a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.361+0000 7fe085264640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0680a2230 0x7fe0680a46f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe078006010 tx=0x7fe0780347f0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.361+0000 7fe085264640 1 -- 172.21.15.18:0/1217868022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe080105350 msgr2=0x7fe08019d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.361+0000 7fe085264640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe080105350 0x7fe08019d930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c940 tx=0x7fe07400ce00 comp rx=0 tx=0).stop 2024-02-17T12:55:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.361+0000 7fe085264640 1 -- 172.21.15.18:0/1217868022 shutdown_connections 2024-02-17T12:55:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.361+0000 7fe085264640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0680a2230 0x7fe0680a46f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.361+0000 7fe085264640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0801975a0 0x7fe080197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.361+0000 7fe085264640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe080105350 0x7fe08019d930 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.361+0000 7fe085264640 1 --2- 172.21.15.18:0/1217868022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe080100280 0x7fe08019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.361+0000 7fe085264640 1 -- 172.21.15.18:0/1217868022 >> 172.21.15.18:0/1217868022 conn(0x7fe080077150 msgr2=0x7fe0800fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.361+0000 7fe085264640 1 -- 172.21.15.18:0/1217868022 shutdown_connections 2024-02-17T12:55:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.361+0000 7fe085264640 1 -- 172.21.15.18:0/1217868022 wait complete. 2024-02-17T12:55:09.364 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:55:09.364 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:55:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.482+0000 7f5bdec22640 1 -- 172.21.15.18:0/3867701521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd8101770 msgr2=0x7f5bd8104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.482+0000 7f5bdec22640 1 --2- 172.21.15.18:0/3867701521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd8101770 0x7f5bd8104d10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009930 tx=0x7f5bc402bc30 comp rx=0 tx=0).stop 2024-02-17T12:55:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.482+0000 7f5bdec22640 1 -- 172.21.15.18:0/3867701521 shutdown_connections 2024-02-17T12:55:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.482+0000 7f5bdec22640 1 --2- 172.21.15.18:0/3867701521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bd8105440 0x7f5bd810cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.482+0000 7f5bdec22640 1 --2- 172.21.15.18:0/3867701521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd8101770 0x7f5bd8104d10 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.482+0000 7f5bdec22640 1 --2- 172.21.15.18:0/3867701521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bd8100dc0 0x7f5bd81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.482+0000 7f5bdec22640 1 -- 172.21.15.18:0/3867701521 >> 172.21.15.18:0/3867701521 conn(0x7f5bd80753c0 msgr2=0x7f5bd80757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.482+0000 7f5bdec22640 1 -- 172.21.15.18:0/3867701521 shutdown_connections 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.482+0000 7f5bdec22640 1 -- 172.21.15.18:0/3867701521 wait complete. 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdec22640 1 Processor -- start 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdec22640 1 -- start start 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdec22640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd8100dc0 0x7f5bd810d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdec22640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bd8105440 0x7f5bd810dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdec22640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bd810e2e0 0x7f5bd81a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdec22640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81a5f70 con 0x7f5bd8100dc0 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdec22640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81a60e0 con 0x7f5bd810e2e0 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdec22640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81a62a0 con 0x7f5bd8105440 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdc9be640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd8100dc0 0x7f5bd810d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdc9be640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd8100dc0 0x7f5bd810d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45620/0 (socket says 172.21.15.18:45620) 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdc9be640 1 -- 172.21.15.18:0/4047110097 learned_addr learned my addr 172.21.15.18:0/4047110097 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bd7fff640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bd8105440 0x7f5bd810dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdd1bf640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bd810e2e0 0x7f5bd81a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdc9be640 1 -- 172.21.15.18:0/4047110097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bd8105440 msgr2=0x7f5bd810dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bd8105440 0x7f5bd810dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdc9be640 1 -- 172.21.15.18:0/4047110097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bd810e2e0 msgr2=0x7f5bd81a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bd810e2e0 0x7f5bd81a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdc9be640 1 -- 172.21.15.18:0/4047110097 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc4009560 con 0x7f5bd8100dc0 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.483+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd8100dc0 0x7f5bd810d860 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00e9f0 tx=0x7f5bcc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.484+0000 7f5bd5ffb640 1 -- 172.21.15.18:0/4047110097 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc009700 con 0x7f5bd8100dc0 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.484+0000 7f5bdec22640 1 -- 172.21.15.18:0/4047110097 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd81a6580 con 0x7f5bd8100dc0 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.484+0000 7f5bd5ffb640 1 -- 172.21.15.18:0/4047110097 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bcc00cd80 con 0x7f5bd8100dc0 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.484+0000 7f5bd5ffb640 1 -- 172.21.15.18:0/4047110097 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc018920 con 0x7f5bd8100dc0 2024-02-17T12:55:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.484+0000 7f5bdec22640 1 -- 172.21.15.18:0/4047110097 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd81a6a60 con 0x7f5bd8100dc0 2024-02-17T12:55:09.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.486+0000 7f5bd5ffb640 1 -- 172.21.15.18:0/4047110097 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5bcc018a80 con 0x7f5bd8100dc0 2024-02-17T12:55:09.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.486+0000 7f5be17f1740 1 Processor -- start 2024-02-17T12:55:09.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.486+0000 7f5be17f1740 1 -- start start 2024-02-17T12:55:09.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.486+0000 7f5bd5ffb640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5bc80a2260 0x7f5bc80a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.486+0000 7f5be17f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5be17f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5be17f1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 0x555db14cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5be17f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db147b960 con 0x555db14c0250 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5be17f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db142f310 con 0x555db14c0b70 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5be17f1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db142c840 con 0x555db14c9750 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bdd1bf640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5bc80a2260 0x7f5bc80a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bdc9be640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bd7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bdc9be640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45636/0 (socket says 172.21.15.18:45636) 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bd5ffb640 1 -- 172.21.15.18:0/4047110097 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5bcc014070 con 0x7f5bd8100dc0 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bdc9be640 1 -- 172.21.15.18:0/542717948 learned_addr learned my addr 172.21.15.18:0/542717948 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bdd1bf640 1 --2- 172.21.15.18:0/542717948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 0x555db14cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bdc9be640 1 -- 172.21.15.18:0/542717948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 msgr2=0x555db14cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/542717948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 0x555db14cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bdc9be640 1 -- 172.21.15.18:0/542717948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 msgr2=0x555db14c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/542717948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bdc9be640 1 -- 172.21.15.18:0/542717948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db13ea060 con 0x555db14c0250 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bd7fff640 1 --2- 172.21.15.18:0/542717948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bdd1bf640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5bc80a2260 0x7f5bc80a4720 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009fd0 tx=0x7f5bd0009da0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/542717948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c0630 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc097310 tx=0x7f5bcc09a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e34e2f30b74eb2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bc2ffd640 1 -- 172.21.15.18:0/542717948 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc09aeb0 con 0x555db14c0250 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bc2ffd640 1 -- 172.21.15.18:0/542717948 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bcc09ef50 con 0x555db14c0250 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.487+0000 7f5bc2ffd640 1 -- 172.21.15.18:0/542717948 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0cbde0 con 0x555db14c0250 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.488+0000 7f5be17f1740 1 -- 172.21.15.18:0/542717948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 msgr2=0x555db14c0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.488+0000 7f5be17f1740 1 --2- 172.21.15.18:0/542717948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c0630 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc097310 tx=0x7f5bcc09a9f0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.488+0000 7f5be17f1740 1 -- 172.21.15.18:0/542717948 shutdown_connections 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.488+0000 7f5be17f1740 1 --2- 172.21.15.18:0/542717948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 0x555db14cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.488+0000 7f5be17f1740 1 --2- 172.21.15.18:0/542717948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.488+0000 7f5be17f1740 1 --2- 172.21.15.18:0/542717948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c0630 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.488+0000 7f5be17f1740 1 -- 172.21.15.18:0/542717948 >> 172.21.15.18:0/542717948 conn(0x555db14ba980 msgr2=0x555db14bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.488+0000 7f5be17f1740 1 -- 172.21.15.18:0/542717948 shutdown_connections 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.488+0000 7f5be17f1740 1 -- 172.21.15.18:0/542717948 wait complete. 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5be17f1740 1 -- start start 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5be17f1740 1 Processor -- start 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5be17f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5be17f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5be17f1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 0x555db14c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5be17f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db1428020 con 0x555db14c0250 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5be17f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db10fb430 con 0x555db14c0b70 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5be17f1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db10b7e00 con 0x555db14c9750 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5be17f1740 10 client.0 _ll_register_callbacks cb 0x555db13cfcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5bdc9be640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5bdc9be640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45638/0 (socket says 172.21.15.18:45638) 2024-02-17T12:55:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5bdc9be640 1 -- 172.21.15.18:0/2441040584 learned_addr learned my addr 172.21.15.18:0/2441040584 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5bd7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.489+0000 7f5bdd1bf640 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 0x555db14c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bdc9be640 1 -- 172.21.15.18:0/2441040584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 msgr2=0x555db14c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 0x555db14c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bdc9be640 1 -- 172.21.15.18:0/2441040584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 msgr2=0x555db14c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bdc9be640 1 -- 172.21.15.18:0/2441040584 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd81a6a60 con 0x555db14c0250 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bd7fff640 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c67e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0a3070 tx=0x7f5bcc0a35a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37259a97aefb33ce server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bb7fff640 1 -- 172.21.15.18:0/2441040584 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc09d070 con 0x555db14c0250 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bb7fff640 1 -- 172.21.15.18:0/2441040584 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5bcc010b80 con 0x555db14c0250 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5be17f1740 1 -- 172.21.15.18:0/2441040584 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555db11277e0 con 0x555db14c0250 2024-02-17T12:55:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bb7fff640 1 -- 172.21.15.18:0/2441040584 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bcc0b9310 con 0x555db14c0250 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5be17f1740 10 client.17427 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5be17f1740 1 -- 172.21.15.18:0/2441040584 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x555db14266a0 con 0x555db14c0250 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bb7fff640 1 -- 172.21.15.18:0/2441040584 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555db11277e0 con 0x555db14c0250 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5bb7fff640 1 -- 172.21.15.18:0/2441040584 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5bcc0cd070 con 0x555db14c0250 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5be17f1740 10 client.17427 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5be17f1740 10 client.17427 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5be17f1740 20 client.17427 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5be17f1740 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x555db14d98f0 0x555db14dbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5be17f1740 4 client.17427 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.490+0000 7f5be17f1740 1 -- 172.21.15.18:0/2441040584 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555db1164720 con 0x555db14d98f0 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.491+0000 7f5bd7fff640 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x555db14d98f0 0x555db14dbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.491+0000 7f5bb7fff640 10 client.17427 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.491+0000 7f5bd7fff640 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x555db14d98f0 0x555db14dbce0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d9c8bbe72c31374d server_cookie=3df69124158c64e9 in_seq=0 out_seq=0 2024-02-17T12:55:09.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5bb7fff640 1 -- 172.21.15.18:0/2441040584 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555db1164720 con 0x555db14d98f0 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5bb7fff640 10 client.17427 handle_command_reply: tid=0 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5be17f1740 1 client.17427 shutdown 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5be17f1740 1 -- 172.21.15.18:0/2441040584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 msgr2=0x555db14c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5be17f1740 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c67e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0a3070 tx=0x7f5bcc0a35a0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5be17f1740 1 -- 172.21.15.18:0/2441040584 shutdown_connections 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5be17f1740 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x555db14d98f0 0x555db14dbce0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5be17f1740 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 0x555db14c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5be17f1740 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5be17f1740 1 --2- 172.21.15.18:0/2441040584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c67e0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5be17f1740 1 -- 172.21.15.18:0/2441040584 >> 172.21.15.18:0/2441040584 conn(0x555db14ba980 msgr2=0x555db14bc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5bb7fff640 0 client.17427 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5be17f1740 1 -- 172.21.15.18:0/2441040584 shutdown_connections 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5be17f1740 1 -- 172.21.15.18:0/2441040584 wait complete. 2024-02-17T12:55:09.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.492+0000 7f5be17f1740 20 client.17427 trim_cache size 0 max 16384 2024-02-17T12:55:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.504+0000 7f5be17f1740 1 Processor -- start 2024-02-17T12:55:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.504+0000 7f5be17f1740 1 -- start start 2024-02-17T12:55:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.504+0000 7f5be17f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.504+0000 7f5be17f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.504+0000 7f5be17f1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 0x555db14c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.504+0000 7f5be17f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db13ad130 con 0x555db14c0250 2024-02-17T12:55:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.504+0000 7f5be17f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db14bb9b0 con 0x555db14c0b70 2024-02-17T12:55:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.504+0000 7f5be17f1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db14b6100 con 0x555db14c9750 2024-02-17T12:55:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.504+0000 7f5bdc9be640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.504+0000 7f5bdc9be640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45640/0 (socket says 172.21.15.18:45640) 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.504+0000 7f5bdc9be640 1 -- 172.21.15.18:0/3120855155 learned_addr learned my addr 172.21.15.18:0/3120855155 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.505+0000 7f5bd7fff640 1 --2- 172.21.15.18:0/3120855155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.505+0000 7f5bdd1bf640 1 --2- 172.21.15.18:0/3120855155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 0x555db14c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.505+0000 7f5bdc9be640 1 -- 172.21.15.18:0/3120855155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 msgr2=0x555db14c1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.505+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/3120855155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 0x555db14c1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.505+0000 7f5bdc9be640 1 -- 172.21.15.18:0/3120855155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 msgr2=0x555db14c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.505+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/3120855155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.505+0000 7f5bdc9be640 1 -- 172.21.15.18:0/3120855155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db13c2c20 con 0x555db14c0250 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.505+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/3120855155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c8f10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0cc160 tx=0x7f5bcc010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200bb640c1a65450 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.505+0000 7f5bc27fc640 1 -- 172.21.15.18:0/3120855155 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc09a810 con 0x555db14c0250 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.505+0000 7f5bc27fc640 1 -- 172.21.15.18:0/3120855155 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bcc010b80 con 0x555db14c0250 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.505+0000 7f5bc27fc640 1 -- 172.21.15.18:0/3120855155 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0b3b30 con 0x555db14c0250 2024-02-17T12:55:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.506+0000 7f5be17f1740 1 -- 172.21.15.18:0/3120855155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 msgr2=0x555db14c8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.506+0000 7f5be17f1740 1 --2- 172.21.15.18:0/3120855155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c8f10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0cc160 tx=0x7f5bcc010f50 comp rx=0 tx=0).stop 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.506+0000 7f5be17f1740 1 -- 172.21.15.18:0/3120855155 shutdown_connections 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.506+0000 7f5be17f1740 1 --2- 172.21.15.18:0/3120855155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c9750 0x555db14c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.506+0000 7f5be17f1740 1 --2- 172.21.15.18:0/3120855155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.506+0000 7f5be17f1740 1 --2- 172.21.15.18:0/3120855155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c0250 0x555db14c8f10 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.506+0000 7f5be17f1740 1 -- 172.21.15.18:0/3120855155 >> 172.21.15.18:0/3120855155 conn(0x555db14ba980 msgr2=0x555db14e0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.506+0000 7f5be17f1740 1 -- 172.21.15.18:0/3120855155 shutdown_connections 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.506+0000 7f5be17f1740 1 -- 172.21.15.18:0/3120855155 wait complete. 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.506+0000 7f5be17f1740 1 -- start start 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.506+0000 7f5be17f1740 1 Processor -- start 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5be17f1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c0250 0x555db14d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5be17f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5be17f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c9750 0x555db14cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5be17f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db14cf820 con 0x555db14c9750 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5be17f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db14cf990 con 0x555db14c0b70 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5be17f1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db152fdb0 con 0x555db14c0250 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5be17f1740 10 client.0 _ll_register_callbacks cb 0x7f5bcc00cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bdd1bf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c9750 0x555db14cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bdc9be640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c0250 0x555db14d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bdd1bf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c9750 0x555db14cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45648/0 (socket says 172.21.15.18:45648) 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bd7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bdd1bf640 1 -- 172.21.15.18:0/3357604665 learned_addr learned my addr 172.21.15.18:0/3357604665 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bdd1bf640 1 -- 172.21.15.18:0/3357604665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c0250 msgr2=0x555db14d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bdd1bf640 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c0250 0x555db14d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bdd1bf640 1 -- 172.21.15.18:0/3357604665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 msgr2=0x555db14cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bdd1bf640 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bdd1bf640 1 -- 172.21.15.18:0/3357604665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555db14266a0 con 0x555db14c9750 2024-02-17T12:55:09.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c0250 0x555db14d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bd7fff640 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bdd1bf640 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c9750 0x555db14cf150 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0010d40 tx=0x7f5bd001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154059099229f0b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bb7fff640 1 -- 172.21.15.18:0/3357604665 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0026070 con 0x555db14c9750 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5bb7fff640 1 -- 172.21.15.18:0/3357604665 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5bd00044b0 con 0x555db14c9750 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.507+0000 7f5be17f1740 1 -- 172.21.15.18:0/3357604665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bcc09ef50 con 0x555db14c9750 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.508+0000 7f5bb7fff640 1 -- 172.21.15.18:0/3357604665 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bd0004670 con 0x555db14c9750 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.508+0000 7f5be17f1740 10 client.17433 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.508+0000 7f5be17f1740 1 -- 172.21.15.18:0/3357604665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x555db13e88c0 con 0x555db14c9750 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.508+0000 7f5bb7fff640 1 -- 172.21.15.18:0/3357604665 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bd0038020 con 0x555db14c9750 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.508+0000 7f5bb7fff640 1 -- 172.21.15.18:0/3357604665 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5bd002a410 con 0x555db14c9750 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.508+0000 7f5be17f1740 10 client.17433 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.508+0000 7f5be17f1740 10 client.17433 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.508+0000 7f5be17f1740 20 client.17433 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.508+0000 7f5be17f1740 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x555db15314e0 0x555db1533900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.508+0000 7f5be17f1740 4 client.17433 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.508+0000 7f5be17f1740 1 -- 172.21.15.18:0/3357604665 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555db11b3890 con 0x555db15314e0 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.508+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x555db15314e0 0x555db1533900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5bb7fff640 10 client.17433 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5bdc9be640 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x555db15314e0 0x555db1533900 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=945fd860ef7092c7 server_cookie=f02691cea7c699a9 in_seq=0 out_seq=0 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5bb7fff640 1 -- 172.21.15.18:0/3357604665 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555db11b3890 con 0x555db15314e0 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5bb7fff640 10 client.17433 handle_command_reply: tid=0 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5be17f1740 1 client.17433 shutdown 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5be17f1740 1 -- 172.21.15.18:0/3357604665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c9750 msgr2=0x555db14cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5be17f1740 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c9750 0x555db14cf150 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0010d40 tx=0x7f5bd001c000 comp rx=0 tx=0).stop 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5be17f1740 1 -- 172.21.15.18:0/3357604665 shutdown_connections 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5be17f1740 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x555db15314e0 0x555db1533900 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5be17f1740 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555db14c9750 0x555db14cf150 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5be17f1740 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555db14c0b70 0x555db14cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5be17f1740 1 --2- 172.21.15.18:0/3357604665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db14c0250 0x555db14d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5bb7fff640 0 client.17433 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:55:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5be17f1740 1 -- 172.21.15.18:0/3357604665 >> 172.21.15.18:0/3357604665 conn(0x555db14ba980 msgr2=0x555db14c1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5be17f1740 1 -- 172.21.15.18:0/3357604665 shutdown_connections 2024-02-17T12:55:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5be17f1740 1 -- 172.21.15.18:0/3357604665 wait complete. 2024-02-17T12:55:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.509+0000 7f5be17f1740 20 client.17433 trim_cache size 0 max 16384 2024-02-17T12:55:09.507 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:55:09.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.510+0000 7f5bdec22640 1 -- 172.21.15.18:0/4047110097 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5bc80a2260 msgr2=0x7f5bc80a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.510+0000 7f5bdec22640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5bc80a2260 0x7f5bc80a4720 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009fd0 tx=0x7f5bd0009da0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.510+0000 7f5bdec22640 1 -- 172.21.15.18:0/4047110097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd8100dc0 msgr2=0x7f5bd810d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.510+0000 7f5bdec22640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd8100dc0 0x7f5bd810d860 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00e9f0 tx=0x7f5bcc00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.510+0000 7f5bdec22640 1 -- 172.21.15.18:0/4047110097 shutdown_connections 2024-02-17T12:55:09.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.510+0000 7f5bdec22640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f5bc80a2260 0x7f5bc80a4720 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.510+0000 7f5bdec22640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bd810e2e0 0x7f5bd81a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.510+0000 7f5bdec22640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bd8105440 0x7f5bd810dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.510+0000 7f5bdec22640 1 --2- 172.21.15.18:0/4047110097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd8100dc0 0x7f5bd810d860 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.510+0000 7f5bdec22640 1 -- 172.21.15.18:0/4047110097 >> 172.21.15.18:0/4047110097 conn(0x7f5bd80753c0 msgr2=0x7f5bd80fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.510+0000 7f5bdec22640 1 -- 172.21.15.18:0/4047110097 shutdown_connections 2024-02-17T12:55:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.510+0000 7f5bdec22640 1 -- 172.21.15.18:0/4047110097 wait complete. 2024-02-17T12:55:09.511 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:55:09.512 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:55:09.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.627+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1519631679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b94106450 msgr2=0x7f7b9410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.627+0000 7f7b9a04d640 1 --2- 172.21.15.18:0/1519631679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b94106450 0x7f7b9410e020 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009cd0 tx=0x7f7b8c02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.627+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1519631679 shutdown_connections 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.627+0000 7f7b9a04d640 1 --2- 172.21.15.18:0/1519631679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b94106450 0x7f7b9410e020 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.627+0000 7f7b9a04d640 1 --2- 172.21.15.18:0/1519631679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b94101d30 0x7f7b94105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.627+0000 7f7b9a04d640 1 --2- 172.21.15.18:0/1519631679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b94101380 0x7f7b94101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.627+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1519631679 >> 172.21.15.18:0/1519631679 conn(0x7f7b94078580 msgr2=0x7f7b94078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.627+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1519631679 shutdown_connections 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.627+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1519631679 wait complete. 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b9a04d640 1 Processor -- start 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b9a04d640 1 -- start start 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b9a04d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b94101380 0x7f7b9419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b9a04d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b94106450 0x7f7b9419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b9a04d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b941975e0 0x7f7b94197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b9a04d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b94197fd0 con 0x7f7b94106450 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b9a04d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b94198140 con 0x7f7b94101380 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b9a04d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b94198300 con 0x7f7b941975e0 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b92ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b94106450 0x7f7b9419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b92ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b94106450 0x7f7b9419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45672/0 (socket says 172.21.15.18:45672) 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b92ffd640 1 -- 172.21.15.18:0/1970023482 learned_addr learned my addr 172.21.15.18:0/1970023482 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b937fe640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b94101380 0x7f7b9419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b93fff640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b941975e0 0x7f7b94197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.628+0000 7f7b92ffd640 1 -- 172.21.15.18:0/1970023482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b941975e0 msgr2=0x7f7b94197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.629+0000 7f7b92ffd640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b941975e0 0x7f7b94197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.629+0000 7f7b92ffd640 1 -- 172.21.15.18:0/1970023482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b94101380 msgr2=0x7f7b9419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.629+0000 7f7b92ffd640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b94101380 0x7f7b9419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.629+0000 7f7b92ffd640 1 -- 172.21.15.18:0/1970023482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c009900 con 0x7f7b94106450 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.629+0000 7f7b937fe640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b94101380 0x7f7b9419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.629+0000 7f7b92ffd640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b94106450 0x7f7b9419d970 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400e9f0 tx=0x7f7b8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.629+0000 7f7b90ff9640 1 -- 172.21.15.18:0/1970023482 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b840096c0 con 0x7f7b94106450 2024-02-17T12:55:09.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.629+0000 7f7b90ff9640 1 -- 172.21.15.18:0/1970023482 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b84005500 con 0x7f7b94106450 2024-02-17T12:55:09.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.629+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1970023482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b941a42f0 con 0x7f7b94106450 2024-02-17T12:55:09.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.629+0000 7f7b90ff9640 1 -- 172.21.15.18:0/1970023482 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84010430 con 0x7f7b94106450 2024-02-17T12:55:09.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.629+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1970023482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b941a47b0 con 0x7f7b94106450 2024-02-17T12:55:09.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.631+0000 7f7b90ff9640 1 -- 172.21.15.18:0/1970023482 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7b84005020 con 0x7f7b94106450 2024-02-17T12:55:09.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.631+0000 7f7b9cc1c740 1 Processor -- start 2024-02-17T12:55:09.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.631+0000 7f7b9cc1c740 1 -- start start 2024-02-17T12:55:09.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b9cc1c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49250 0x5612bbb49630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b9cc1c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb49b70 0x5612bbb52210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b9cc1c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb52750 0x5612bbb54b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b9cc1c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bbb04960 con 0x5612bbb49b70 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b9cc1c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bbab8310 con 0x5612bbb52750 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b9cc1c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bbab5840 con 0x5612bbb49250 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b937fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb49b70 0x5612bbb52210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b937fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb49b70 0x5612bbb52210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45678/0 (socket says 172.21.15.18:45678) 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49250 0x5612bbb49630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b937fe640 1 -- 172.21.15.18:0/109018336 learned_addr learned my addr 172.21.15.18:0/109018336 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b90ff9640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b7c0a2230 0x7f7b7c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b92ffd640 1 --2- 172.21.15.18:0/109018336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb52750 0x5612bbb54b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b93fff640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b7c0a2230 0x7f7b7c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b90ff9640 1 -- 172.21.15.18:0/1970023482 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7b84014070 con 0x7f7b94106450 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b937fe640 1 -- 172.21.15.18:0/109018336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49250 msgr2=0x5612bbb49630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b937fe640 1 --2- 172.21.15.18:0/109018336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49250 0x5612bbb49630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b937fe640 1 -- 172.21.15.18:0/109018336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb52750 msgr2=0x5612bbb54b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b937fe640 1 --2- 172.21.15.18:0/109018336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb52750 0x5612bbb54b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b937fe640 1 -- 172.21.15.18:0/109018336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612bba73060 con 0x5612bbb49b70 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b93fff640 1 --2- 172.21.15.18:0/109018336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49250 0x5612bbb49630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.632+0000 7f7b937fe640 1 --2- 172.21.15.18:0/109018336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb49b70 0x5612bbb52210 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f7b88013900 tx=0x7f7b88013dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ead5745d1c4cfd3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.633+0000 7f7b93fff640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b7c0a2230 0x7f7b7c0a46f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005cc0 tx=0x7f7b8c02c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.633+0000 7f7b79ffb640 1 -- 172.21.15.18:0/109018336 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88018070 con 0x5612bbb49b70 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.633+0000 7f7b79ffb640 1 -- 172.21.15.18:0/109018336 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b8800ee70 con 0x5612bbb49b70 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.633+0000 7f7b79ffb640 1 -- 172.21.15.18:0/109018336 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8801b370 con 0x5612bbb49b70 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.633+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/109018336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb49b70 msgr2=0x5612bbb52210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.633+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/109018336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb49b70 0x5612bbb52210 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f7b88013900 tx=0x7f7b88013dc0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.633+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/109018336 shutdown_connections 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.633+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/109018336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb52750 0x5612bbb54b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.633+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/109018336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb49b70 0x5612bbb52210 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.633+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/109018336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49250 0x5612bbb49630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.633+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/109018336 >> 172.21.15.18:0/109018336 conn(0x5612bbb43980 msgr2=0x5612bbb43d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.633+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/109018336 shutdown_connections 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/109018336 wait complete. 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b9cc1c740 1 -- start start 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b9cc1c740 1 Processor -- start 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b9cc1c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49250 0x5612bbb4f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b9cc1c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49b70 0x5612bbb4fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b9cc1c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb501f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b9cc1c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bbab1020 con 0x5612bbb52750 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b9cc1c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bb784430 con 0x5612bbb49250 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b9cc1c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bb740e00 con 0x5612bbb49b70 2024-02-17T12:55:09.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b9cc1c740 10 client.0 _ll_register_callbacks cb 0x5612bba58cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b92ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49b70 0x5612bbb4fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.634+0000 7f7b937fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49250 0x5612bbb4f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b92ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49b70 0x5612bbb4fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:41022/0 (socket says 172.21.15.18:41022) 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b92ffd640 1 -- 172.21.15.18:0/831582446 learned_addr learned my addr 172.21.15.18:0/831582446 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b937fe640 1 -- 172.21.15.18:0/831582446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49b70 msgr2=0x5612bbb4fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b937fe640 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49b70 0x5612bbb4fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b937fe640 1 -- 172.21.15.18:0/831582446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 msgr2=0x5612bbb501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b937fe640 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b937fe640 1 -- 172.21.15.18:0/831582446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b88009d40 con 0x5612bbb49250 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b92ffd640 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49b70 0x5612bbb4fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b93fff640 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b937fe640 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49250 0x5612bbb4f770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7b88013fd0 tx=0x7f7b8801c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df1db511b242028e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b72ffd640 1 -- 172.21.15.18:0/831582446 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88018070 con 0x5612bbb49250 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/831582446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612bb7b07e0 con 0x5612bbb49250 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b72ffd640 1 -- 172.21.15.18:0/831582446 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7b88004030 con 0x5612bbb49250 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b72ffd640 1 -- 172.21.15.18:0/831582446 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b880041f0 con 0x5612bbb49250 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b9cc1c740 10 client.26777 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.635+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/831582446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5612bbaaf6a0 con 0x5612bbb49250 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.636+0000 7f7b72ffd640 1 -- 172.21.15.18:0/831582446 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b88033020 con 0x5612bbb49250 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.636+0000 7f7b72ffd640 1 -- 172.21.15.18:0/831582446 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f7b88004930 con 0x5612bbb49250 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.636+0000 7f7b9cc1c740 10 client.26777 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.636+0000 7f7b9cc1c740 10 client.26777 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.636+0000 7f7b9cc1c740 20 client.26777 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.636+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5612bbb629e0 0x5612bbb64e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.636+0000 7f7b9cc1c740 4 client.26777 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:09.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.636+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/831582446 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612bb7ed720 con 0x5612bbb629e0 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.636+0000 7f7b92ffd640 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5612bbb629e0 0x5612bbb64e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.636+0000 7f7b92ffd640 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5612bbb629e0 0x5612bbb64e00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5711f39b5ada5d5c server_cookie=2fc91d97ba47c89b in_seq=0 out_seq=0 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.636+0000 7f7b72ffd640 10 client.26777 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b72ffd640 1 -- 172.21.15.18:0/831582446 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612bb7ed720 con 0x5612bbb629e0 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b72ffd640 10 client.26777 handle_command_reply: tid=0 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b9cc1c740 1 client.26777 shutdown 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/831582446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49250 msgr2=0x5612bbb4f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49250 0x5612bbb4f770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7b88013fd0 tx=0x7f7b8801c6c0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/831582446 shutdown_connections 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5612bbb629e0 0x5612bbb64e00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b72ffd640 0 client.26777 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49b70 0x5612bbb4fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/831582446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49250 0x5612bbb4f770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/831582446 >> 172.21.15.18:0/831582446 conn(0x5612bbb43980 msgr2=0x5612bbb456d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.637+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/831582446 shutdown_connections 2024-02-17T12:55:09.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.638+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/831582446 wait complete. 2024-02-17T12:55:09.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.638+0000 7f7b9cc1c740 20 client.26777 trim_cache size 0 max 16384 2024-02-17T12:55:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.649+0000 7f7b9cc1c740 1 Processor -- start 2024-02-17T12:55:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.649+0000 7f7b9cc1c740 1 -- start start 2024-02-17T12:55:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b9cc1c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49250 0x5612bbb51ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b9cc1c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49b70 0x5612bbb4e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b9cc1c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb50010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b9cc1c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bba36130 con 0x5612bbb52750 2024-02-17T12:55:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b9cc1c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bbb62810 con 0x5612bbb49250 2024-02-17T12:55:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b9cc1c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bbb48ca0 con 0x5612bbb49b70 2024-02-17T12:55:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb50010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb50010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45700/0 (socket says 172.21.15.18:45700) 2024-02-17T12:55:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b93fff640 1 -- 172.21.15.18:0/3666793947 learned_addr learned my addr 172.21.15.18:0/3666793947 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b937fe640 1 --2- 172.21.15.18:0/3666793947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49250 0x5612bbb51ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b92ffd640 1 --2- 172.21.15.18:0/3666793947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49b70 0x5612bbb4e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b93fff640 1 -- 172.21.15.18:0/3666793947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49b70 msgr2=0x5612bbb4e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b93fff640 1 --2- 172.21.15.18:0/3666793947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49b70 0x5612bbb4e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b93fff640 1 -- 172.21.15.18:0/3666793947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49250 msgr2=0x5612bbb51ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b93fff640 1 --2- 172.21.15.18:0/3666793947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49250 0x5612bbb51ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b93fff640 1 -- 172.21.15.18:0/3666793947 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612bba4bc20 con 0x5612bbb52750 2024-02-17T12:55:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b93fff640 1 --2- 172.21.15.18:0/3666793947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb50010 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c036000 tx=0x7f7b8c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a01bba4c786bd8f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b797fa640 1 -- 172.21.15.18:0/3666793947 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c032ab0 con 0x5612bbb52750 2024-02-17T12:55:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b797fa640 1 -- 172.21.15.18:0/3666793947 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c003e50 con 0x5612bbb52750 2024-02-17T12:55:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.650+0000 7f7b797fa640 1 -- 172.21.15.18:0/3666793947 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0506d0 con 0x5612bbb52750 2024-02-17T12:55:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.651+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/3666793947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 msgr2=0x5612bbb50010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.651+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/3666793947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb50010 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c036000 tx=0x7f7b8c005f70 comp rx=0 tx=0).stop 2024-02-17T12:55:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.651+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/3666793947 shutdown_connections 2024-02-17T12:55:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.651+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/3666793947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb50010 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.651+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/3666793947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49b70 0x5612bbb4e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.651+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/3666793947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49250 0x5612bbb51ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.651+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/3666793947 >> 172.21.15.18:0/3666793947 conn(0x5612bbb43980 msgr2=0x5612bbb4e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.651+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/3666793947 shutdown_connections 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.651+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/3666793947 wait complete. 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.651+0000 7f7b9cc1c740 1 -- start start 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b9cc1c740 1 Processor -- start 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b9cc1c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49250 0x5612bbb58040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b9cc1c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49b70 0x5612bbb58580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b9cc1c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb58ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b9cc1c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bbbb8ef0 con 0x5612bbb52750 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b9cc1c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bbbb9030 con 0x5612bbb49b70 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b9cc1c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bbbb9170 con 0x5612bbb49250 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b9cc1c740 10 client.0 _ll_register_callbacks cb 0x7f7b88009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb58ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb58ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45714/0 (socket says 172.21.15.18:45714) 2024-02-17T12:55:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b93fff640 1 -- 172.21.15.18:0/980316283 learned_addr learned my addr 172.21.15.18:0/980316283 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b92ffd640 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49b70 0x5612bbb58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b937fe640 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49250 0x5612bbb58040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b93fff640 1 -- 172.21.15.18:0/980316283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49250 msgr2=0x5612bbb58040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b93fff640 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49250 0x5612bbb58040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b93fff640 1 -- 172.21.15.18:0/980316283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49b70 msgr2=0x5612bbb58580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b93fff640 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49b70 0x5612bbb58580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b93fff640 1 -- 172.21.15.18:0/980316283 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c030070 con 0x5612bbb52750 2024-02-17T12:55:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b93fff640 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb58ac0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c037040 tx=0x7f7b8c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b741baae9b8a0b1b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.652+0000 7f7b72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b72ffd640 1 -- 172.21.15.18:0/980316283 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0041c0 con 0x5612bbb52750 2024-02-17T12:55:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b72ffd640 1 -- 172.21.15.18:0/980316283 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7b8c003970 con 0x5612bbb52750 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/980316283 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b8800ee70 con 0x5612bbb52750 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b72ffd640 1 -- 172.21.15.18:0/980316283 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b8c003b30 con 0x5612bbb52750 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b9cc1c740 10 client.17451 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/980316283 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5612bba718c0 con 0x5612bbb52750 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b72ffd640 1 -- 172.21.15.18:0/980316283 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b8800ee70 con 0x5612bbb52750 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b72ffd640 1 -- 172.21.15.18:0/980316283 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f7b8c03e040 con 0x5612bbb52750 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b9cc1c740 10 client.17451 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b9cc1c740 10 client.17451 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b9cc1c740 20 client.17451 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5612bbbba660 0x5612bbbbca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b9cc1c740 4 client.17451 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/980316283 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5612bb83c890 con 0x5612bbbba660 2024-02-17T12:55:09.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.653+0000 7f7b937fe640 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5612bbbba660 0x5612bbbbca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b72ffd640 10 client.17451 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b937fe640 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5612bbbba660 0x5612bbbbca80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69b3f00fa1dd07a6 server_cookie=5d7b7483b250a0a5 in_seq=0 out_seq=0 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b72ffd640 1 -- 172.21.15.18:0/980316283 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5612bb83c890 con 0x5612bbbba660 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b72ffd640 10 client.17451 handle_command_reply: tid=0 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b9cc1c740 1 client.17451 shutdown 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/980316283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 msgr2=0x5612bbb58ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb58ac0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c037040 tx=0x7f7b8c005bd0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/980316283 shutdown_connections 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5612bbbba660 0x5612bbbbca80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612bbb52750 0x5612bbb58ac0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612bbb49b70 0x5612bbb58580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b9cc1c740 1 --2- 172.21.15.18:0/980316283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612bbb49250 0x5612bbb58040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/980316283 >> 172.21.15.18:0/980316283 conn(0x5612bbb43980 msgr2=0x5612bbb42b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b72ffd640 0 client.17451 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:55:09.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/980316283 shutdown_connections 2024-02-17T12:55:09.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b9cc1c740 1 -- 172.21.15.18:0/980316283 wait complete. 2024-02-17T12:55:09.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.654+0000 7f7b9cc1c740 20 client.17451 trim_cache size 0 max 16384 2024-02-17T12:55:09.653 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:55:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.655+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1970023482 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b7c0a2230 msgr2=0x7f7b7c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.655+0000 7f7b9a04d640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b7c0a2230 0x7f7b7c0a46f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005cc0 tx=0x7f7b8c02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.655+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1970023482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b94106450 msgr2=0x7f7b9419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.655+0000 7f7b9a04d640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b94106450 0x7f7b9419d970 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400e9f0 tx=0x7f7b8400eeb0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.655+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1970023482 shutdown_connections 2024-02-17T12:55:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.655+0000 7f7b9a04d640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b7c0a2230 0x7f7b7c0a46f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.655+0000 7f7b9a04d640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b941975e0 0x7f7b94197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.655+0000 7f7b9a04d640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b94106450 0x7f7b9419d970 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.655+0000 7f7b9a04d640 1 --2- 172.21.15.18:0/1970023482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b94101380 0x7f7b9419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.655+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1970023482 >> 172.21.15.18:0/1970023482 conn(0x7f7b94078580 msgr2=0x7f7b940ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.655+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1970023482 shutdown_connections 2024-02-17T12:55:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.655+0000 7f7b9a04d640 1 -- 172.21.15.18:0/1970023482 wait complete. 2024-02-17T12:55:09.657 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:55:09.657 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:55:09.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.772+0000 7fe9a268d640 1 -- 172.21.15.18:0/1141351671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c1064e0 msgr2=0x7fe99c10e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.772+0000 7fe9a268d640 1 --2- 172.21.15.18:0/1141351671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c1064e0 0x7fe99c10e060 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe994009cd0 tx=0x7fe99402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.772+0000 7fe9a268d640 1 -- 172.21.15.18:0/1141351671 shutdown_connections 2024-02-17T12:55:09.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.772+0000 7fe9a268d640 1 --2- 172.21.15.18:0/1141351671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c1064e0 0x7fe99c10e060 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.772+0000 7fe9a268d640 1 --2- 172.21.15.18:0/1141351671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe99c100280 0x7fe99c105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.772+0000 7fe9a268d640 1 --2- 172.21.15.18:0/1141351671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c0ff8d0 0x7fe99c0ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.772+0000 7fe9a268d640 1 -- 172.21.15.18:0/1141351671 >> 172.21.15.18:0/1141351671 conn(0x7fe99c0765a0 msgr2=0x7fe99c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.772+0000 7fe9a268d640 1 -- 172.21.15.18:0/1141351671 shutdown_connections 2024-02-17T12:55:09.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.772+0000 7fe9a268d640 1 -- 172.21.15.18:0/1141351671 wait complete. 2024-02-17T12:55:09.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe9a268d640 1 Processor -- start 2024-02-17T12:55:09.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe9a268d640 1 -- start start 2024-02-17T12:55:09.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe9a268d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe99c0ff8d0 0x7fe99c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe9a268d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1064e0 0x7fe99c19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe9a268d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c1976c0 0x7fe99c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe9a268d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c198190 con 0x7fe99c1976c0 2024-02-17T12:55:09.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe9a268d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c198300 con 0x7fe99c0ff8d0 2024-02-17T12:55:09.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe9a268d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c1984c0 con 0x7fe99c1064e0 2024-02-17T12:55:09.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe99bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe99c0ff8d0 0x7fe99c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe99bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe99c0ff8d0 0x7fe99c19d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:45656/0 (socket says 172.21.15.18:45656) 2024-02-17T12:55:09.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe99b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1064e0 0x7fe99c19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe99bfff640 1 -- 172.21.15.18:0/1307599748 learned_addr learned my addr 172.21.15.18:0/1307599748 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.773+0000 7fe9a0c2a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c1976c0 0x7fe99c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.774+0000 7fe9a0c2a640 1 -- 172.21.15.18:0/1307599748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1064e0 msgr2=0x7fe99c19db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.774+0000 7fe9a0c2a640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1064e0 0x7fe99c19db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.774+0000 7fe9a0c2a640 1 -- 172.21.15.18:0/1307599748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe99c0ff8d0 msgr2=0x7fe99c19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.774+0000 7fe9a0c2a640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe99c0ff8d0 0x7fe99c19d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.774+0000 7fe9a0c2a640 1 -- 172.21.15.18:0/1307599748 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe994009900 con 0x7fe99c1976c0 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.774+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1064e0 0x7fe99c19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.774+0000 7fe9a0c2a640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c1976c0 0x7fe99c197b40 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe99402c630 tx=0x7fe994034b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.774+0000 7fe9997fa640 1 -- 172.21.15.18:0/1307599748 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940328b0 con 0x7fe99c1976c0 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.774+0000 7fe9a268d640 1 -- 172.21.15.18:0/1307599748 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99c1a4400 con 0x7fe99c1976c0 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.774+0000 7fe9997fa640 1 -- 172.21.15.18:0/1307599748 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe994003e50 con 0x7fe99c1976c0 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.774+0000 7fe9997fa640 1 -- 172.21.15.18:0/1307599748 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994040450 con 0x7fe99c1976c0 2024-02-17T12:55:09.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.774+0000 7fe9a268d640 1 -- 172.21.15.18:0/1307599748 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99c1a48d0 con 0x7fe99c1976c0 2024-02-17T12:55:09.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.776+0000 7fe9997fa640 1 -- 172.21.15.18:0/1307599748 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe99400a040 con 0x7fe99c1976c0 2024-02-17T12:55:09.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe9a525a740 1 Processor -- start 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe9a525a740 1 -- start start 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe9a525a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172c250 0x55bd9172c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe9a525a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172cb70 0x55bd91735210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe9a525a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 0x55bd91737b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe9a525a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd916e7960 con 0x55bd9172cb70 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe9a525a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9169b310 con 0x55bd9172c250 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe9a525a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd91698840 con 0x55bd91735750 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe99bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172cb70 0x55bd91735210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe9997fa640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe9840a2230 0x7fe9840a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe99b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172c250 0x55bd9172c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe99b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172c250 0x55bd9172c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:45664/0 (socket says 172.21.15.18:45664) 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe99bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172cb70 0x55bd91735210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45754/0 (socket says 172.21.15.18:45754) 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe99b7fe640 1 -- 172.21.15.18:0/3220748546 learned_addr learned my addr 172.21.15.18:0/3220748546 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/3220748546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 0x55bd91737b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe9997fa640 1 -- 172.21.15.18:0/1307599748 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe9940f0b70 con 0x7fe99c1976c0 2024-02-17T12:55:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe9a0c2a640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe9840a2230 0x7fe9840a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe99bfff640 1 -- 172.21.15.18:0/3220748546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 msgr2=0x55bd91737b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe99bfff640 1 --2- 172.21.15.18:0/3220748546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 0x55bd91737b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe99bfff640 1 -- 172.21.15.18:0/3220748546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172c250 msgr2=0x55bd9172c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe99bfff640 1 --2- 172.21.15.18:0/3220748546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172c250 0x55bd9172c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe99bfff640 1 -- 172.21.15.18:0/3220748546 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd91656060 con 0x55bd9172cb70 2024-02-17T12:55:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.777+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/3220748546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172c250 0x55bd9172c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.778+0000 7fe99bfff640 1 --2- 172.21.15.18:0/3220748546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172cb70 0x55bd91735210 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009cf0 tx=0x7fe98c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c54e63c825dde7a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.778+0000 7fe9a0c2a640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe9840a2230 0x7fe9840a46f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe9940bd3c0 tx=0x7fe9940095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.778+0000 7fe9827fc640 1 -- 172.21.15.18:0/3220748546 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c005220 con 0x55bd9172cb70 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.778+0000 7fe9827fc640 1 -- 172.21.15.18:0/3220748546 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe98c004c80 con 0x55bd9172cb70 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.778+0000 7fe9827fc640 1 -- 172.21.15.18:0/3220748546 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c019870 con 0x55bd9172cb70 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 -- 172.21.15.18:0/3220748546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172cb70 msgr2=0x55bd91735210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 --2- 172.21.15.18:0/3220748546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172cb70 0x55bd91735210 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009cf0 tx=0x7fe98c011810 comp rx=0 tx=0).stop 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 -- 172.21.15.18:0/3220748546 shutdown_connections 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 --2- 172.21.15.18:0/3220748546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 0x55bd91737b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 --2- 172.21.15.18:0/3220748546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172cb70 0x55bd91735210 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 --2- 172.21.15.18:0/3220748546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172c250 0x55bd9172c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 -- 172.21.15.18:0/3220748546 >> 172.21.15.18:0/3220748546 conn(0x55bd91726980 msgr2=0x55bd91726d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 -- 172.21.15.18:0/3220748546 shutdown_connections 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 -- 172.21.15.18:0/3220748546 wait complete. 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 -- start start 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 Processor -- start 2024-02-17T12:55:09.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd917327c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172cb70 0x55bd91732d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 0x55bd91733240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd91694020 con 0x55bd9172c250 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd91367430 con 0x55bd9172cb70 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.779+0000 7fe9a525a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd91323e00 con 0x55bd91735750 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe9a525a740 10 client.0 _ll_register_callbacks cb 0x55bd9163bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe99b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd917327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe99b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd917327c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45756/0 (socket says 172.21.15.18:45756) 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe99b7fe640 1 -- 172.21.15.18:0/1117287510 learned_addr learned my addr 172.21.15.18:0/1117287510 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 0x55bd91733240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe99bfff640 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172cb70 0x55bd91732d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe99b7fe640 1 -- 172.21.15.18:0/1117287510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 msgr2=0x55bd91733240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 0x55bd91733240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe99b7fe640 1 -- 172.21.15.18:0/1117287510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172cb70 msgr2=0x55bd91732d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172cb70 0x55bd91732d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe99b7fe640 1 -- 172.21.15.18:0/1117287510 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c009920 con 0x55bd9172c250 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd917327c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe990010420 tx=0x7fe990005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93966c6ca330a3e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe97b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe97b7fe640 1 -- 172.21.15.18:0/1117287510 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990004020 con 0x55bd9172c250 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe97b7fe640 1 -- 172.21.15.18:0/1117287510 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe990004180 con 0x55bd9172c250 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe9a525a740 1 -- 172.21.15.18:0/1117287510 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd913937e0 con 0x55bd9172c250 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe97b7fe640 1 -- 172.21.15.18:0/1117287510 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe990004340 con 0x55bd9172c250 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe9a525a740 10 client.17463 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.780+0000 7fe9a525a740 1 -- 172.21.15.18:0/1117287510 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55bd916926a0 con 0x55bd9172c250 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.781+0000 7fe97b7fe640 1 -- 172.21.15.18:0/1117287510 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bd913937e0 con 0x55bd9172c250 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.781+0000 7fe97b7fe640 1 -- 172.21.15.18:0/1117287510 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe99000e560 con 0x55bd9172c250 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.781+0000 7fe9a525a740 10 client.17463 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.781+0000 7fe9a525a740 10 client.17463 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.781+0000 7fe9a525a740 20 client.17463 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.781+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55bd91745bf0 0x55bd91748010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.781+0000 7fe9a525a740 4 client.17463 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:09.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.781+0000 7fe9a525a740 1 -- 172.21.15.18:0/1117287510 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd913d0720 con 0x55bd91745bf0 2024-02-17T12:55:09.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.781+0000 7fe99bfff640 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55bd91745bf0 0x55bd91748010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.781+0000 7fe97b7fe640 10 client.17463 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:55:09.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.781+0000 7fe99bfff640 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55bd91745bf0 0x55bd91748010 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72a078313fe5d0a2 server_cookie=ddfdbc8902d60ecc in_seq=0 out_seq=0 2024-02-17T12:55:09.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.782+0000 7fe97b7fe640 1 -- 172.21.15.18:0/1117287510 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd913d0720 con 0x55bd91745bf0 2024-02-17T12:55:09.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.782+0000 7fe97b7fe640 10 client.17463 handle_command_reply: tid=0 2024-02-17T12:55:09.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.782+0000 7fe9a525a740 1 client.17463 shutdown 2024-02-17T12:55:09.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:09 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2005347712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:55:09.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.782+0000 7fe9a525a740 1 -- 172.21.15.18:0/1117287510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 msgr2=0x55bd917327c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.782+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd917327c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe990010420 tx=0x7fe990005d90 comp rx=0 tx=0).stop 2024-02-17T12:55:09.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.782+0000 7fe9a525a740 1 -- 172.21.15.18:0/1117287510 shutdown_connections 2024-02-17T12:55:09.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.782+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55bd91745bf0 0x55bd91748010 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.782+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 0x55bd91733240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.782+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172cb70 0x55bd91732d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.782+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1117287510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd917327c0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.782+0000 7fe9a525a740 1 -- 172.21.15.18:0/1117287510 >> 172.21.15.18:0/1117287510 conn(0x55bd91726980 msgr2=0x55bd917286d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.782+0000 7fe97b7fe640 0 client.17463 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:55:09.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.783+0000 7fe9a525a740 1 -- 172.21.15.18:0/1117287510 shutdown_connections 2024-02-17T12:55:09.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.783+0000 7fe9a525a740 1 -- 172.21.15.18:0/1117287510 wait complete. 2024-02-17T12:55:09.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.783+0000 7fe9a525a740 20 client.17463 trim_cache size 0 max 16384 2024-02-17T12:55:09.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.794+0000 7fe9a525a740 1 Processor -- start 2024-02-17T12:55:09.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.794+0000 7fe9a525a740 1 -- start start 2024-02-17T12:55:09.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe9a525a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd91734ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe9a525a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172cb70 0x55bd9172daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe9a525a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 0x55bd9172e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe9a525a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd91619130 con 0x55bd9172c250 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe9a525a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd917426b0 con 0x55bd9172cb70 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe9a525a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd917443f0 con 0x55bd91735750 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe99b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd91734ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe99b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd91734ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45760/0 (socket says 172.21.15.18:45760) 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe99b7fe640 1 -- 172.21.15.18:0/1002500705 learned_addr learned my addr 172.21.15.18:0/1002500705 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe99bfff640 1 --2- 172.21.15.18:0/1002500705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172cb70 0x55bd9172daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe99b7fe640 1 -- 172.21.15.18:0/1002500705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 msgr2=0x55bd9172e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/1002500705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 0x55bd9172e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe99b7fe640 1 -- 172.21.15.18:0/1002500705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172cb70 msgr2=0x55bd9172daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/1002500705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172cb70 0x55bd9172daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe99b7fe640 1 -- 172.21.15.18:0/1002500705 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd9162ec20 con 0x55bd9172c250 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/1002500705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd91734ef0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe990006010 tx=0x7fe99000b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423b132ddfe51e2f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe981ffb640 1 -- 172.21.15.18:0/1002500705 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990012c20 con 0x55bd9172c250 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe981ffb640 1 -- 172.21.15.18:0/1002500705 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe99001f040 con 0x55bd9172c250 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.795+0000 7fe981ffb640 1 -- 172.21.15.18:0/1002500705 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99002d910 con 0x55bd9172c250 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.796+0000 7fe9a525a740 1 -- 172.21.15.18:0/1002500705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 msgr2=0x55bd91734ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.796+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1002500705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd91734ef0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe990006010 tx=0x7fe99000b490 comp rx=0 tx=0).stop 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.796+0000 7fe9a525a740 1 -- 172.21.15.18:0/1002500705 shutdown_connections 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.796+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1002500705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd91735750 0x55bd9172e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.796+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1002500705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd9172cb70 0x55bd9172daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.796+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1002500705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd91734ef0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.796+0000 7fe9a525a740 1 -- 172.21.15.18:0/1002500705 >> 172.21.15.18:0/1002500705 conn(0x55bd91726980 msgr2=0x55bd9174c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.796+0000 7fe9a525a740 1 -- 172.21.15.18:0/1002500705 shutdown_connections 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.796+0000 7fe9a525a740 1 -- 172.21.15.18:0/1002500705 wait complete. 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.796+0000 7fe9a525a740 1 -- start start 2024-02-17T12:55:09.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe9a525a740 1 Processor -- start 2024-02-17T12:55:09.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe9a525a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd9173af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe9a525a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd9172cb70 0x55bd9173b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe9a525a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd91735750 0x55bd9173b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe9a525a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9173ceb0 con 0x55bd9172c250 2024-02-17T12:55:09.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe9a525a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9179c0c0 con 0x55bd91735750 2024-02-17T12:55:09.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe9a525a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9179c230 con 0x55bd9172cb70 2024-02-17T12:55:09.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe9a525a740 10 client.0 _ll_register_callbacks cb 0x7fe98c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe99b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd9173af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe99b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd9173af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45766/0 (socket says 172.21.15.18:45766) 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe99b7fe640 1 -- 172.21.15.18:0/1387746325 learned_addr learned my addr 172.21.15.18:0/1387746325 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe99bfff640 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd9172cb70 0x55bd9173b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd91735750 0x55bd9173b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe99b7fe640 1 -- 172.21.15.18:0/1387746325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd9172cb70 msgr2=0x55bd9173b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd9172cb70 0x55bd9173b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe99b7fe640 1 -- 172.21.15.18:0/1387746325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd91735750 msgr2=0x55bd9173b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd91735750 0x55bd9173b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.797+0000 7fe99b7fe640 1 -- 172.21.15.18:0/1387746325 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd916926a0 con 0x55bd9172c250 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe99b7fe640 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd9173af10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fe990004700 tx=0x7fe99000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460fc36b518f8c64 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe97b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe97b7fe640 1 -- 172.21.15.18:0/1387746325 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9900129f0 con 0x55bd9172c250 2024-02-17T12:55:09.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe97b7fe640 1 -- 172.21.15.18:0/1387746325 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe990000e40 con 0x55bd9172c250 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe9a525a740 1 -- 172.21.15.18:0/1387746325 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe98c004c80 con 0x55bd9172c250 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe97b7fe640 1 -- 172.21.15.18:0/1387746325 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe99000ee50 con 0x55bd9172c250 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe9a525a740 10 client.17469 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe9a525a740 1 -- 172.21.15.18:0/1387746325 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55bd916548c0 con 0x55bd9172c250 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe97b7fe640 1 -- 172.21.15.18:0/1387746325 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe98c004c80 con 0x55bd9172c250 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe97b7fe640 1 -- 172.21.15.18:0/1387746325 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe9900445b0 con 0x55bd9172c250 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe9a525a740 10 client.17469 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe9a525a740 10 client.17469 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe9a525a740 20 client.17469 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55bd9179d6b0 0x55bd9179faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe9a525a740 4 client.17469 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe9a525a740 1 -- 172.21.15.18:0/1387746325 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bd9141f890 con 0x55bd9179d6b0 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.798+0000 7fe99bfff640 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55bd9179d6b0 0x55bd9179faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe97b7fe640 10 client.17469 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:55:09.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe99bfff640 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55bd9179d6b0 0x55bd9179faa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b13e91ce40c8cfe server_cookie=580316500550c400 in_seq=0 out_seq=0 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe97b7fe640 1 -- 172.21.15.18:0/1387746325 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bd9141f890 con 0x55bd9179d6b0 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe97b7fe640 10 client.17469 handle_command_reply: tid=0 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe9a525a740 1 client.17469 shutdown 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe9a525a740 1 -- 172.21.15.18:0/1387746325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 msgr2=0x55bd9173af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd9173af10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fe990004700 tx=0x7fe99000ed00 comp rx=0 tx=0).stop 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe9a525a740 1 -- 172.21.15.18:0/1387746325 shutdown_connections 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55bd9179d6b0 0x55bd9179faa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd91735750 0x55bd9173b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bd9172cb70 0x55bd9173b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe9a525a740 1 --2- 172.21.15.18:0/1387746325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bd9172c250 0x55bd9173af10 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe9a525a740 1 -- 172.21.15.18:0/1387746325 >> 172.21.15.18:0/1387746325 conn(0x55bd91726980 msgr2=0x55bd9172daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe97b7fe640 0 client.17469 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe9a525a740 1 -- 172.21.15.18:0/1387746325 shutdown_connections 2024-02-17T12:55:09.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe9a525a740 1 -- 172.21.15.18:0/1387746325 wait complete. 2024-02-17T12:55:09.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.799+0000 7fe9a525a740 20 client.17469 trim_cache size 0 max 16384 2024-02-17T12:55:09.797 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:55:09.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.800+0000 7fe9a268d640 1 -- 172.21.15.18:0/1307599748 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe9840a2230 msgr2=0x7fe9840a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.800+0000 7fe9a268d640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe9840a2230 0x7fe9840a46f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe9940bd3c0 tx=0x7fe9940095c0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.800+0000 7fe9a268d640 1 -- 172.21.15.18:0/1307599748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c1976c0 msgr2=0x7fe99c197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.800+0000 7fe9a268d640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c1976c0 0x7fe99c197b40 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe99402c630 tx=0x7fe994034b10 comp rx=0 tx=0).stop 2024-02-17T12:55:09.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.800+0000 7fe9a268d640 1 -- 172.21.15.18:0/1307599748 shutdown_connections 2024-02-17T12:55:09.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.800+0000 7fe9a268d640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe9840a2230 0x7fe9840a46f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.800+0000 7fe9a268d640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c1976c0 0x7fe99c197b40 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.800+0000 7fe9a268d640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe99c1064e0 0x7fe99c19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.800+0000 7fe9a268d640 1 --2- 172.21.15.18:0/1307599748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe99c0ff8d0 0x7fe99c19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.800+0000 7fe9a268d640 1 -- 172.21.15.18:0/1307599748 >> 172.21.15.18:0/1307599748 conn(0x7fe99c0765a0 msgr2=0x7fe99c104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.800+0000 7fe9a268d640 1 -- 172.21.15.18:0/1307599748 shutdown_connections 2024-02-17T12:55:09.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.800+0000 7fe9a268d640 1 -- 172.21.15.18:0/1307599748 wait complete. 2024-02-17T12:55:09.801 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:55:09.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:55:09.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:09 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2005347712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:55:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.920+0000 7f18e99ac640 1 -- 172.21.15.18:0/4272412854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18e4101170 msgr2=0x7f18e410cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.920+0000 7f18e99ac640 1 --2- 172.21.15.18:0/4272412854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18e4101170 0x7f18e410cea0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009ca0 tx=0x7f18dc02c140 comp rx=0 tx=0).stop 2024-02-17T12:55:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.920+0000 7f18e99ac640 1 -- 172.21.15.18:0/4272412854 shutdown_connections 2024-02-17T12:55:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.920+0000 7f18e99ac640 1 --2- 172.21.15.18:0/4272412854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18e4101170 0x7f18e410cea0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.920+0000 7f18e99ac640 1 --2- 172.21.15.18:0/4272412854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18e41007d0 0x7f18e4100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.920+0000 7f18e99ac640 1 --2- 172.21.15.18:0/4272412854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18e4105030 0x7f18e4105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.920+0000 7f18e99ac640 1 -- 172.21.15.18:0/4272412854 >> 172.21.15.18:0/4272412854 conn(0x7f18e4076640 msgr2=0x7f18e40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.920+0000 7f18e99ac640 1 -- 172.21.15.18:0/4272412854 shutdown_connections 2024-02-17T12:55:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.920+0000 7f18e99ac640 1 -- 172.21.15.18:0/4272412854 wait complete. 2024-02-17T12:55:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.920+0000 7f18e99ac640 1 Processor -- start 2024-02-17T12:55:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e99ac640 1 -- start start 2024-02-17T12:55:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e99ac640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18e41007d0 0x7f18e419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e99ac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18e4101170 0x7f18e419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e99ac640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18e4105030 0x7f18e41976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e99ac640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e4197e60 con 0x7f18e41007d0 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e99ac640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e4197fd0 con 0x7f18e4101170 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e99ac640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e4198190 con 0x7f18e4105030 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18e41007d0 0x7f18e419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18e41007d0 0x7f18e419d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45770/0 (socket says 172.21.15.18:45770) 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e2ffd640 1 -- 172.21.15.18:0/3930014240 learned_addr learned my addr 172.21.15.18:0/3930014240 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e37fe640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18e4105030 0x7f18e41976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e27fc640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18e4101170 0x7f18e419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e2ffd640 1 -- 172.21.15.18:0/3930014240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18e4105030 msgr2=0x7f18e41976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e2ffd640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18e4105030 0x7f18e41976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e2ffd640 1 -- 172.21.15.18:0/3930014240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18e4101170 msgr2=0x7f18e419db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e2ffd640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18e4101170 0x7f18e419db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e2ffd640 1 -- 172.21.15.18:0/3930014240 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc009900 con 0x7f18e41007d0 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e2ffd640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18e41007d0 0x7f18e419d5c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f18d400e9f0 tx=0x7f18d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.921+0000 7f18e89aa640 1 -- 172.21.15.18:0/3930014240 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4009700 con 0x7f18e41007d0 2024-02-17T12:55:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.922+0000 7f18e89aa640 1 -- 172.21.15.18:0/3930014240 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18d400cd80 con 0x7f18e41007d0 2024-02-17T12:55:09.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.922+0000 7f18e99ac640 1 -- 172.21.15.18:0/3930014240 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e4198470 con 0x7f18e41007d0 2024-02-17T12:55:09.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.922+0000 7f18e89aa640 1 -- 172.21.15.18:0/3930014240 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4018920 con 0x7f18e41007d0 2024-02-17T12:55:09.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.922+0000 7f18e99ac640 1 -- 172.21.15.18:0/3930014240 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e41a4540 con 0x7f18e41007d0 2024-02-17T12:55:09.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.924+0000 7f18ec580740 1 Processor -- start 2024-02-17T12:55:09.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.924+0000 7f18e89aa640 1 -- 172.21.15.18:0/3930014240 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f18d4018a80 con 0x7f18e41007d0 2024-02-17T12:55:09.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.924+0000 7f18ec580740 1 -- start start 2024-02-17T12:55:09.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18ec580740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220f4250 0x55d0220f4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18ec580740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d0220fd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18ec580740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220fd750 0x55d0220ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18ec580740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0220af960 con 0x55d0220f4250 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18ec580740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d022063310 con 0x55d0220f4b70 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18ec580740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d022060840 con 0x55d0220fd750 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220f4250 0x55d0220f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220f4250 0x55d0220f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45786/0 (socket says 172.21.15.18:45786) 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e37fe640 1 -- 172.21.15.18:0/748502081 learned_addr learned my addr 172.21.15.18:0/748502081 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e2ffd640 1 --2- 172.21.15.18:0/748502081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d0220fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e27fc640 1 --2- 172.21.15.18:0/748502081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220fd750 0x55d0220ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e37fe640 1 -- 172.21.15.18:0/748502081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220fd750 msgr2=0x55d0220ffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e37fe640 1 --2- 172.21.15.18:0/748502081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220fd750 0x55d0220ffb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e37fe640 1 -- 172.21.15.18:0/748502081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 msgr2=0x55d0220fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e37fe640 1 --2- 172.21.15.18:0/748502081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d0220fd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e37fe640 1 -- 172.21.15.18:0/748502081 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d02201e060 con 0x55d0220f4250 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e89aa640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f18cc0a2230 0x7f18cc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e2ffd640 1 --2- 172.21.15.18:0/748502081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d0220fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e2ffd640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f18cc0a2230 0x7f18cc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e37fe640 1 --2- 172.21.15.18:0/748502081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220f4250 0x55d0220f4630 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009ca0 tx=0x7f18dc032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263f41586cb9d79d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.925+0000 7f18e89aa640 1 -- 172.21.15.18:0/3930014240 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f18d4014070 con 0x7f18e41007d0 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18c9ffb640 1 -- 172.21.15.18:0/748502081 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc032340 con 0x55d0220f4250 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18c9ffb640 1 -- 172.21.15.18:0/748502081 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18dc0324a0 con 0x55d0220f4250 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18c9ffb640 1 -- 172.21.15.18:0/748502081 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc003970 con 0x55d0220f4250 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18ec580740 1 -- 172.21.15.18:0/748502081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220f4250 msgr2=0x55d0220f4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18e2ffd640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f18cc0a2230 0x7f18cc0a46f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18d40cbb90 tx=0x7f18d4010f50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18ec580740 1 --2- 172.21.15.18:0/748502081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220f4250 0x55d0220f4630 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009ca0 tx=0x7f18dc032d60 comp rx=0 tx=0).stop 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18ec580740 1 -- 172.21.15.18:0/748502081 shutdown_connections 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18ec580740 1 --2- 172.21.15.18:0/748502081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220fd750 0x55d0220ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18ec580740 1 --2- 172.21.15.18:0/748502081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d0220fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18ec580740 1 --2- 172.21.15.18:0/748502081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220f4250 0x55d0220f4630 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18ec580740 1 -- 172.21.15.18:0/748502081 >> 172.21.15.18:0/748502081 conn(0x55d0220ee980 msgr2=0x55d0220eed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18ec580740 1 -- 172.21.15.18:0/748502081 shutdown_connections 2024-02-17T12:55:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.926+0000 7f18ec580740 1 -- 172.21.15.18:0/748502081 wait complete. 2024-02-17T12:55:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18ec580740 1 -- start start 2024-02-17T12:55:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18ec580740 1 Processor -- start 2024-02-17T12:55:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18ec580740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4250 0x55d0220fa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18ec580740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d0220facb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18ec580740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d0220fb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18ec580740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02205c020 con 0x55d0220fd750 2024-02-17T12:55:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18ec580740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d021d2f430 con 0x55d0220f4b70 2024-02-17T12:55:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18ec580740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d021cebe00 con 0x55d0220f4250 2024-02-17T12:55:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18ec580740 10 client.0 _ll_register_callbacks cb 0x55d022003cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18e37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d0220fb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18e37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d0220fb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45792/0 (socket says 172.21.15.18:45792) 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18e37fe640 1 -- 172.21.15.18:0/2694638160 learned_addr learned my addr 172.21.15.18:0/2694638160 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18e27fc640 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d0220facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18e37fe640 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4250 0x55d0220fa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18e37fe640 1 -- 172.21.15.18:0/2694638160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4250 msgr2=0x55d0220fa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.927+0000 7f18e37fe640 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4250 0x55d0220fa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18e37fe640 1 -- 172.21.15.18:0/2694638160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 msgr2=0x55d0220facb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18e37fe640 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d0220facb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18e37fe640 1 -- 172.21.15.18:0/2694638160 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e40ff6c0 con 0x55d0220fd750 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18e27fc640 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d0220facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18e37fe640 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d0220fb1f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f18dc005cc0 tx=0x7f18dc036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127d3309694ef1dd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18c2ffd640 1 -- 172.21.15.18:0/2694638160 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc034440 con 0x55d0220fd750 2024-02-17T12:55:09.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18c2ffd640 1 -- 172.21.15.18:0/2694638160 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f18dc0345a0 con 0x55d0220fd750 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18ec580740 1 -- 172.21.15.18:0/2694638160 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d021d5b7e0 con 0x55d0220fd750 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18c2ffd640 1 -- 172.21.15.18:0/2694638160 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18dc034760 con 0x55d0220fd750 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18ec580740 10 client.17481 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18ec580740 1 -- 172.21.15.18:0/2694638160 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d02205a6a0 con 0x55d0220fd750 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18c2ffd640 1 -- 172.21.15.18:0/2694638160 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d021d5b7e0 con 0x55d0220fd750 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18c2ffd640 1 -- 172.21.15.18:0/2694638160 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f18dc03f480 con 0x55d0220fd750 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18ec580740 10 client.17481 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18ec580740 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18ec580740 20 client.17481 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18ec580740 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d02210d880 0x55d02210fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18ec580740 4 client.17481 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.928+0000 7f18ec580740 1 -- 172.21.15.18:0/2694638160 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d021d98720 con 0x55d02210d880 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.929+0000 7f18e27fc640 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d02210d880 0x55d02210fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.929+0000 7f18e27fc640 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d02210d880 0x55d02210fca0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d93c8d2ecb67b6 server_cookie=6add5c3b2a308fea in_seq=0 out_seq=0 2024-02-17T12:55:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.929+0000 7f18c2ffd640 10 client.17481 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18c2ffd640 1 -- 172.21.15.18:0/2694638160 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d021d98720 con 0x55d02210d880 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18c2ffd640 10 client.17481 handle_command_reply: tid=0 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18ec580740 1 client.17481 shutdown 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18ec580740 1 -- 172.21.15.18:0/2694638160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 msgr2=0x55d0220fb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18ec580740 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d0220fb1f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f18dc005cc0 tx=0x7f18dc036040 comp rx=0 tx=0).stop 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18ec580740 1 -- 172.21.15.18:0/2694638160 shutdown_connections 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18ec580740 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d02210d880 0x55d02210fca0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18ec580740 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d0220fb1f0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18ec580740 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d0220facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18c2ffd640 0 client.17481 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18ec580740 1 --2- 172.21.15.18:0/2694638160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4250 0x55d0220fa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18ec580740 1 -- 172.21.15.18:0/2694638160 >> 172.21.15.18:0/2694638160 conn(0x55d0220ee980 msgr2=0x55d0220f06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18ec580740 1 -- 172.21.15.18:0/2694638160 shutdown_connections 2024-02-17T12:55:09.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18ec580740 1 -- 172.21.15.18:0/2694638160 wait complete. 2024-02-17T12:55:09.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.930+0000 7f18ec580740 20 client.17481 trim_cache size 0 max 16384 2024-02-17T12:55:09.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.942+0000 7f18ec580740 1 Processor -- start 2024-02-17T12:55:09.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.942+0000 7f18ec580740 1 -- start start 2024-02-17T12:55:09.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.942+0000 7f18ec580740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4250 0x55d0220fcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.942+0000 7f18ec580740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4b70 0x55d0220f9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.942+0000 7f18ec580740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d0220f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.942+0000 7f18ec580740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d021fe1130 con 0x55d0220fd750 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.942+0000 7f18ec580740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0220ef980 con 0x55d0220f4250 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.942+0000 7f18ec580740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0220f3650 con 0x55d0220f4b70 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4250 0x55d0220fcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e27fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4b70 0x55d0220f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4250 0x55d0220fcea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:45724/0 (socket says 172.21.15.18:45724) 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e27fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4b70 0x55d0220f9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:41138/0 (socket says 172.21.15.18:41138) 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e37fe640 1 -- 172.21.15.18:0/1662523762 learned_addr learned my addr 172.21.15.18:0/1662523762 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e37fe640 1 --2- 172.21.15.18:0/1662523762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d0220f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e27fc640 1 -- 172.21.15.18:0/1662523762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4250 msgr2=0x55d0220fcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e27fc640 1 --2- 172.21.15.18:0/1662523762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4250 0x55d0220fcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e27fc640 1 -- 172.21.15.18:0/1662523762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 msgr2=0x55d0220f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e27fc640 1 --2- 172.21.15.18:0/1662523762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d0220f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e27fc640 1 -- 172.21.15.18:0/1662523762 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d021ff6c20 con 0x55d0220f4b70 2024-02-17T12:55:09.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e37fe640 1 --2- 172.21.15.18:0/1662523762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4250 0x55d0220fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18e27fc640 1 --2- 172.21.15.18:0/1662523762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4b70 0x55d0220f9750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55d022110570 tx=0x7f18d8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37f032b015630ab server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.943+0000 7f18c97fa640 1 -- 172.21.15.18:0/1662523762 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8017040 con 0x55d0220f4b70 2024-02-17T12:55:09.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.944+0000 7f18c97fa640 1 -- 172.21.15.18:0/1662523762 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18d8018040 con 0x55d0220f4b70 2024-02-17T12:55:09.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.944+0000 7f18c97fa640 1 -- 172.21.15.18:0/1662523762 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8021890 con 0x55d0220f4b70 2024-02-17T12:55:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.944+0000 7f18ec580740 1 -- 172.21.15.18:0/1662523762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4b70 msgr2=0x55d0220f9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.944+0000 7f18ec580740 1 --2- 172.21.15.18:0/1662523762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4b70 0x55d0220f9750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55d022110570 tx=0x7f18d8005f90 comp rx=0 tx=0).stop 2024-02-17T12:55:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.944+0000 7f18ec580740 1 -- 172.21.15.18:0/1662523762 shutdown_connections 2024-02-17T12:55:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.944+0000 7f18ec580740 1 --2- 172.21.15.18:0/1662523762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d0220f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.944+0000 7f18ec580740 1 --2- 172.21.15.18:0/1662523762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4b70 0x55d0220f9750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.944+0000 7f18ec580740 1 --2- 172.21.15.18:0/1662523762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4250 0x55d0220fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.944+0000 7f18ec580740 1 -- 172.21.15.18:0/1662523762 >> 172.21.15.18:0/1662523762 conn(0x55d0220ee980 msgr2=0x55d0220f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.944+0000 7f18ec580740 1 -- 172.21.15.18:0/1662523762 shutdown_connections 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.944+0000 7f18ec580740 1 -- 172.21.15.18:0/1662523762 wait complete. 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18ec580740 1 -- start start 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18ec580740 1 Processor -- start 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18ec580740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4250 0x55d022103120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18ec580740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d022103660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18ec580740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d022103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18ec580740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d022163d50 con 0x55d0220fd750 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18ec580740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d022163e90 con 0x55d0220f4b70 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18ec580740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d022164050 con 0x55d0220f4250 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18ec580740 10 client.0 _ll_register_callbacks cb 0x7f18d4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18e37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d022103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18e27fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d022103660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18e37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d022103ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45814/0 (socket says 172.21.15.18:45814) 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18e37fe640 1 -- 172.21.15.18:0/2326704167 learned_addr learned my addr 172.21.15.18:0/2326704167 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.945+0000 7f18e37fe640 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4250 0x55d022103120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18e37fe640 1 -- 172.21.15.18:0/2326704167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4250 msgr2=0x55d022103120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18e37fe640 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4250 0x55d022103120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18e37fe640 1 -- 172.21.15.18:0/2326704167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 msgr2=0x55d022103660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18e37fe640 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d022103660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18e37fe640 1 -- 172.21.15.18:0/2326704167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d02205a6a0 con 0x55d0220fd750 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18e27fc640 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d022103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18e37fe640 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d022103ba0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f18dc005cc0 tx=0x7f18dc032880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6320361c232e868 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18c2ffd640 1 -- 172.21.15.18:0/2326704167 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc032ea0 con 0x55d0220fd750 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18ec580740 1 -- 172.21.15.18:0/2326704167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18dc0324a0 con 0x55d0220fd750 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18c2ffd640 1 -- 172.21.15.18:0/2326704167 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f18dc03fc10 con 0x55d0220fd750 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18c2ffd640 1 -- 172.21.15.18:0/2326704167 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18dc034440 con 0x55d0220fd750 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18ec580740 10 client.17487 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18ec580740 1 -- 172.21.15.18:0/2326704167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f18d8012690 con 0x55d0220fd750 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.946+0000 7f18c2ffd640 1 -- 172.21.15.18:0/2326704167 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18dc0324a0 con 0x55d0220fd750 2024-02-17T12:55:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.947+0000 7f18c2ffd640 1 -- 172.21.15.18:0/2326704167 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f18dc0041d0 con 0x55d0220fd750 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.947+0000 7f18ec580740 10 client.17487 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.947+0000 7f18ec580740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.947+0000 7f18ec580740 20 client.17487 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.947+0000 7f18ec580740 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d0221654d0 0x55d0221678f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.947+0000 7f18ec580740 4 client.17487 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.947+0000 7f18ec580740 1 -- 172.21.15.18:0/2326704167 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d021de7890 con 0x55d0221654d0 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.947+0000 7f18e27fc640 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d0221654d0 0x55d0221678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18e27fc640 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d0221654d0 0x55d0221678f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce331dd7c331e96 server_cookie=205260177efc71bf in_seq=0 out_seq=0 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18c2ffd640 10 client.17487 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18c2ffd640 1 -- 172.21.15.18:0/2326704167 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d021de7890 con 0x55d0221654d0 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18c2ffd640 10 client.17487 handle_command_reply: tid=0 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18ec580740 1 client.17487 shutdown 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18ec580740 1 -- 172.21.15.18:0/2326704167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 msgr2=0x55d022103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18ec580740 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d022103ba0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f18dc005cc0 tx=0x7f18dc032880 comp rx=0 tx=0).stop 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18ec580740 1 -- 172.21.15.18:0/2326704167 shutdown_connections 2024-02-17T12:55:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18ec580740 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d0221654d0 0x55d0221678f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18ec580740 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0220fd750 0x55d022103ba0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18ec580740 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0220f4b70 0x55d022103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18ec580740 1 --2- 172.21.15.18:0/2326704167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0220f4250 0x55d022103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18ec580740 1 -- 172.21.15.18:0/2326704167 >> 172.21.15.18:0/2326704167 conn(0x55d0220ee980 msgr2=0x55d0220f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18c2ffd640 0 client.17487 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:09.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18ec580740 1 -- 172.21.15.18:0/2326704167 shutdown_connections 2024-02-17T12:55:09.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18ec580740 1 -- 172.21.15.18:0/2326704167 wait complete. 2024-02-17T12:55:09.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.948+0000 7f18ec580740 20 client.17487 trim_cache size 0 max 16384 2024-02-17T12:55:09.944 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:55:09.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.949+0000 7f18e99ac640 1 -- 172.21.15.18:0/3930014240 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f18cc0a2230 msgr2=0x7f18cc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.949+0000 7f18e99ac640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f18cc0a2230 0x7f18cc0a46f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18d40cbb90 tx=0x7f18d4010f50 comp rx=0 tx=0).stop 2024-02-17T12:55:09.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.949+0000 7f18e99ac640 1 -- 172.21.15.18:0/3930014240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18e41007d0 msgr2=0x7f18e419d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:09.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.949+0000 7f18e99ac640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18e41007d0 0x7f18e419d5c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f18d400e9f0 tx=0x7f18d400eeb0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.949+0000 7f18e99ac640 1 -- 172.21.15.18:0/3930014240 shutdown_connections 2024-02-17T12:55:09.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.949+0000 7f18e99ac640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f18cc0a2230 0x7f18cc0a46f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.949+0000 7f18e99ac640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18e4105030 0x7f18e41976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.949+0000 7f18e99ac640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18e4101170 0x7f18e419db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.949+0000 7f18e99ac640 1 --2- 172.21.15.18:0/3930014240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18e41007d0 0x7f18e419d5c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:09.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.949+0000 7f18e99ac640 1 -- 172.21.15.18:0/3930014240 >> 172.21.15.18:0/3930014240 conn(0x7f18e4076640 msgr2=0x7f18e4103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:09.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.949+0000 7f18e99ac640 1 -- 172.21.15.18:0/3930014240 shutdown_connections 2024-02-17T12:55:09.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:09.949+0000 7f18e99ac640 1 -- 172.21.15.18:0/3930014240 wait complete. 2024-02-17T12:55:09.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:09 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2005347712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:55:10.274 INFO:tasks.workunit.client.0.smithi018.stdout: 1 288767 6.81 MB/sec execute 145 sec latency 14.307 ms 2024-02-17T12:55:10.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:10 smithi119 ceph-mon[36188]: pgmap v161: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-02-17T12:55:10.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:10 smithi146 ceph-mon[36454]: pgmap v161: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-02-17T12:55:10.951 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:55:10.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:55:10.952 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:55:10.976 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:10 smithi018 ceph-mon[32625]: pgmap v161: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-02-17T12:55:11.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.069+0000 7ff94e6e1640 1 -- 172.21.15.18:0/3071916335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff948101390 msgr2=0x7ff948101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.069+0000 7ff94e6e1640 1 --2- 172.21.15.18:0/3071916335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff948101390 0x7ff948101770 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff938009930 tx=0x7ff93802bc30 comp rx=0 tx=0).stop 2024-02-17T12:55:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.069+0000 7ff94e6e1640 1 -- 172.21.15.18:0/3071916335 shutdown_connections 2024-02-17T12:55:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.069+0000 7ff94e6e1640 1 --2- 172.21.15.18:0/3071916335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff948106460 0x7ff94810e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.069+0000 7ff94e6e1640 1 --2- 172.21.15.18:0/3071916335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff948101d40 0x7ff948105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.069+0000 7ff94e6e1640 1 --2- 172.21.15.18:0/3071916335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff948101390 0x7ff948101770 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.069+0000 7ff94e6e1640 1 -- 172.21.15.18:0/3071916335 >> 172.21.15.18:0/3071916335 conn(0x7ff9480fcbb0 msgr2=0x7ff9480fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.069+0000 7ff94e6e1640 1 -- 172.21.15.18:0/3071916335 shutdown_connections 2024-02-17T12:55:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.069+0000 7ff94e6e1640 1 -- 172.21.15.18:0/3071916335 wait complete. 2024-02-17T12:55:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.070+0000 7ff94e6e1640 1 Processor -- start 2024-02-17T12:55:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.070+0000 7ff94e6e1640 1 -- start start 2024-02-17T12:55:11.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.070+0000 7ff94e6e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff948101390 0x7ff948073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.070+0000 7ff94e6e1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff948106460 0x7ff9480739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.070+0000 7ff94e6e1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff948073f00 0x7ff9481a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.070+0000 7ff94e6e1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9481a5fd0 con 0x7ff948073f00 2024-02-17T12:55:11.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.070+0000 7ff94e6e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9481a6140 con 0x7ff948101390 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.070+0000 7ff94e6e1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9481a6300 con 0x7ff948106460 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.070+0000 7ff94cc7e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff948073f00 0x7ff9481a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.070+0000 7ff94cc7e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff948073f00 0x7ff9481a5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45826/0 (socket says 172.21.15.18:45826) 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.070+0000 7ff94cc7e640 1 -- 172.21.15.18:0/2188324598 learned_addr learned my addr 172.21.15.18:0/2188324598 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff9477fe640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff948106460 0x7ff9480739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff947fff640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff948101390 0x7ff948073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff94cc7e640 1 -- 172.21.15.18:0/2188324598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff948106460 msgr2=0x7ff9480739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff94cc7e640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff948106460 0x7ff9480739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff94cc7e640 1 -- 172.21.15.18:0/2188324598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff948101390 msgr2=0x7ff948073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff94cc7e640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff948101390 0x7ff948073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff94cc7e640 1 -- 172.21.15.18:0/2188324598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938009560 con 0x7ff948073f00 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff94cc7e640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff948073f00 0x7ff9481a5a00 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff94000eb10 tx=0x7ff94000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff9457fa640 1 -- 172.21.15.18:0/2188324598 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940004b20 con 0x7ff948073f00 2024-02-17T12:55:11.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff94e6e1640 1 -- 172.21.15.18:0/2188324598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9481a65e0 con 0x7ff948073f00 2024-02-17T12:55:11.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff9457fa640 1 -- 172.21.15.18:0/2188324598 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff940004c80 con 0x7ff948073f00 2024-02-17T12:55:11.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff9457fa640 1 -- 172.21.15.18:0/2188324598 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940010650 con 0x7ff948073f00 2024-02-17T12:55:11.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.071+0000 7ff94e6e1640 1 -- 172.21.15.18:0/2188324598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9481a6af0 con 0x7ff948073f00 2024-02-17T12:55:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.073+0000 7ff9457fa640 1 -- 172.21.15.18:0/2188324598 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff940005020 con 0x7ff948073f00 2024-02-17T12:55:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.073+0000 7ff95138a740 1 Processor -- start 2024-02-17T12:55:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff95138a740 1 -- start start 2024-02-17T12:55:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff9457fa640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff9300a2260 0x7ff9300a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff95138a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff95138a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0d30 0x557ceedb92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff95138a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb98b0 0x557ceedbbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff95138a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ceed1ef90 con 0x557ceedb0d30 2024-02-17T12:55:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff95138a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ceed1c4c0 con 0x557ceedb98b0 2024-02-17T12:55:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff95138a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ceed17ca0 con 0x557ceedb0380 2024-02-17T12:55:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff9477fe640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff9300a2260 0x7ff9300a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff9457fa640 1 -- 172.21.15.18:0/2188324598 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff940014070 con 0x7ff948073f00 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff9477fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0d30 0x557ceedb92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff9477fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0d30 0x557ceedb92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45832/0 (socket says 172.21.15.18:45832) 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff9477fe640 1 -- 172.21.15.18:0/2201512558 learned_addr learned my addr 172.21.15.18:0/2201512558 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff947fff640 1 --2- 172.21.15.18:0/2201512558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff94cc7e640 1 --2- 172.21.15.18:0/2201512558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb98b0 0x557ceedbbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff9477fe640 1 -- 172.21.15.18:0/2201512558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 msgr2=0x557ceedb0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff9477fe640 1 --2- 172.21.15.18:0/2201512558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff9477fe640 1 -- 172.21.15.18:0/2201512558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb98b0 msgr2=0x557ceedbbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff9477fe640 1 --2- 172.21.15.18:0/2201512558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb98b0 0x557ceedbbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.074+0000 7ff9477fe640 1 -- 172.21.15.18:0/2201512558 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ceecda0b0 con 0x557ceedb0d30 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff9477fe640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff9300a2260 0x7ff9300a4720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff93c012900 tx=0x7ff93c009ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff9477fe640 1 --2- 172.21.15.18:0/2201512558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0d30 0x557ceedb92e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009d60 tx=0x7ff93c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f1e4e9341315fe server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff92e7fc640 1 -- 172.21.15.18:0/2201512558 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c010e60 con 0x557ceedb0d30 2024-02-17T12:55:11.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff92e7fc640 1 -- 172.21.15.18:0/2201512558 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff93c009850 con 0x557ceedb0d30 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff92e7fc640 1 -- 172.21.15.18:0/2201512558 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c020a80 con 0x557ceedb0d30 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff95138a740 1 -- 172.21.15.18:0/2201512558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0d30 msgr2=0x557ceedb92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff95138a740 1 --2- 172.21.15.18:0/2201512558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0d30 0x557ceedb92e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009d60 tx=0x7ff93c01c000 comp rx=0 tx=0).stop 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff95138a740 1 -- 172.21.15.18:0/2201512558 shutdown_connections 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff95138a740 1 --2- 172.21.15.18:0/2201512558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb98b0 0x557ceedbbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff95138a740 1 --2- 172.21.15.18:0/2201512558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0d30 0x557ceedb92e0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff95138a740 1 --2- 172.21.15.18:0/2201512558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff95138a740 1 -- 172.21.15.18:0/2201512558 >> 172.21.15.18:0/2201512558 conn(0x557ceedaa840 msgr2=0x557ceedaac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff95138a740 1 -- 172.21.15.18:0/2201512558 shutdown_connections 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.075+0000 7ff95138a740 1 -- 172.21.15.18:0/2201512558 wait complete. 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff95138a740 1 -- start start 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff95138a740 1 Processor -- start 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff95138a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff95138a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceedb6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff95138a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 0x557ceedb72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff95138a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ceecbcf40 con 0x557ceedb98b0 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff95138a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cee9eb4b0 con 0x557ceedb0d30 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff95138a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ceec9d670 con 0x557ceedb0380 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff95138a740 10 client.0 _ll_register_callbacks cb 0x557ceed0af70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff94cc7e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 0x557ceedb72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff94cc7e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 0x557ceedb72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45844/0 (socket says 172.21.15.18:45844) 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff94cc7e640 1 -- 172.21.15.18:0/2941884971 learned_addr learned my addr 172.21.15.18:0/2941884971 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.076+0000 7ff947fff640 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff94cc7e640 1 -- 172.21.15.18:0/2941884971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 msgr2=0x557ceedb6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff94cc7e640 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff94cc7e640 1 -- 172.21.15.18:0/2941884971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 msgr2=0x557ceedb6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff9477fe640 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceedb6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff94cc7e640 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceedb6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff94cc7e640 1 -- 172.21.15.18:0/2941884971 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93c01a040 con 0x557ceedb98b0 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff947fff640 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff9477fe640 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceedb6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff94cc7e640 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 0x557ceedb72f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff9400cc470 tx=0x7ff94009ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff2a4a4c66665e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff9277fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff9277fe640 1 -- 172.21.15.18:0/2941884971 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94009f260 con 0x557ceedb98b0 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff9277fe640 1 -- 172.21.15.18:0/2941884971 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff9400a38b0 con 0x557ceedb98b0 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff95138a740 1 -- 172.21.15.18:0/2941884971 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ceecf1f40 con 0x557ceedb98b0 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff9277fe640 1 -- 172.21.15.18:0/2941884971 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff940099410 con 0x557ceedb98b0 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff95138a740 10 client.17499 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff95138a740 1 -- 172.21.15.18:0/2941884971 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557ceed16320 con 0x557ceedb98b0 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff9277fe640 1 -- 172.21.15.18:0/2941884971 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557ceecf1f40 con 0x557ceedb98b0 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff9277fe640 1 -- 172.21.15.18:0/2941884971 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff94001f060 con 0x557ceedb98b0 2024-02-17T12:55:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff95138a740 10 client.17499 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff95138a740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff95138a740 20 client.17499 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff95138a740 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557ceedc9f40 0x557ceedcc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff95138a740 4 client.17499 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.077+0000 7ff95138a740 1 -- 172.21.15.18:0/2941884971 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ceed0e310 con 0x557ceedc9f40 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.078+0000 7ff947fff640 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557ceedc9f40 0x557ceedcc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.078+0000 7ff9277fe640 10 client.17499 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.078+0000 7ff947fff640 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557ceedc9f40 0x557ceedcc360 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965ed3ae7dbde1ba server_cookie=abce6f71ccb608ba in_seq=0 out_seq=0 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff9277fe640 1 -- 172.21.15.18:0/2941884971 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ceed0e310 con 0x557ceedc9f40 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff9277fe640 10 client.17499 handle_command_reply: tid=0 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff95138a740 1 client.17499 shutdown 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff95138a740 1 -- 172.21.15.18:0/2941884971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 msgr2=0x557ceedb72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff95138a740 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 0x557ceedb72f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff9400cc470 tx=0x7ff94009ab30 comp rx=0 tx=0).stop 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff95138a740 1 -- 172.21.15.18:0/2941884971 shutdown_connections 2024-02-17T12:55:11.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff95138a740 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557ceedc9f40 0x557ceedcc360 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff95138a740 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 0x557ceedb72f0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff95138a740 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceedb6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff9277fe640 0 client.17499 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:11.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff95138a740 1 --2- 172.21.15.18:0/2941884971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff95138a740 1 -- 172.21.15.18:0/2941884971 >> 172.21.15.18:0/2941884971 conn(0x557ceedaa840 msgr2=0x557ceedac7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:11.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff95138a740 1 -- 172.21.15.18:0/2941884971 shutdown_connections 2024-02-17T12:55:11.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff95138a740 1 -- 172.21.15.18:0/2941884971 wait complete. 2024-02-17T12:55:11.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.079+0000 7ff95138a740 20 client.17499 trim_cache size 0 max 16384 2024-02-17T12:55:11.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.094+0000 7ff95138a740 1 Processor -- start 2024-02-17T12:55:11.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.094+0000 7ff95138a740 1 -- start start 2024-02-17T12:55:11.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff95138a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff95138a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceeda9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff95138a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 0x557ceedc0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff95138a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ceedc5910 con 0x557ceedb98b0 2024-02-17T12:55:11.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff95138a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ceedc6980 con 0x557ceedb0d30 2024-02-17T12:55:11.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff95138a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ceedaa370 con 0x557ceedb0380 2024-02-17T12:55:11.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff94cc7e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 0x557ceedc0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff94cc7e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 0x557ceedc0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45856/0 (socket says 172.21.15.18:45856) 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff94cc7e640 1 -- 172.21.15.18:0/228417144 learned_addr learned my addr 172.21.15.18:0/228417144 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff947fff640 1 --2- 172.21.15.18:0/228417144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff9477fe640 1 --2- 172.21.15.18:0/228417144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceeda9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff94cc7e640 1 -- 172.21.15.18:0/228417144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 msgr2=0x557ceedb7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff94cc7e640 1 --2- 172.21.15.18:0/228417144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff94cc7e640 1 -- 172.21.15.18:0/228417144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 msgr2=0x557ceeda9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff94cc7e640 1 --2- 172.21.15.18:0/228417144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceeda9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff94cc7e640 1 -- 172.21.15.18:0/228417144 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ceecb2c00 con 0x557ceedb98b0 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff9477fe640 1 --2- 172.21.15.18:0/228417144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceeda9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff947fff640 1 --2- 172.21.15.18:0/228417144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff94cc7e640 1 --2- 172.21.15.18:0/228417144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 0x557ceedc0900 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff9400049d0 tx=0x7ff94009b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8c317c1dd972d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff92dffb640 1 -- 172.21.15.18:0/228417144 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94009aee0 con 0x557ceedb98b0 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff92dffb640 1 -- 172.21.15.18:0/228417144 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9400a38b0 con 0x557ceedb98b0 2024-02-17T12:55:11.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.095+0000 7ff92dffb640 1 -- 172.21.15.18:0/228417144 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400a3680 con 0x557ceedb98b0 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.096+0000 7ff95138a740 1 -- 172.21.15.18:0/228417144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 msgr2=0x557ceedc0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.096+0000 7ff95138a740 1 --2- 172.21.15.18:0/228417144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 0x557ceedc0900 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff9400049d0 tx=0x7ff94009b290 comp rx=0 tx=0).stop 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.096+0000 7ff95138a740 1 -- 172.21.15.18:0/228417144 shutdown_connections 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.096+0000 7ff95138a740 1 --2- 172.21.15.18:0/228417144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb98b0 0x557ceedc0900 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.096+0000 7ff95138a740 1 --2- 172.21.15.18:0/228417144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceeda9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.096+0000 7ff95138a740 1 --2- 172.21.15.18:0/228417144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb0380 0x557ceedb7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.096+0000 7ff95138a740 1 -- 172.21.15.18:0/228417144 >> 172.21.15.18:0/228417144 conn(0x557ceedaa840 msgr2=0x557ceeda9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.096+0000 7ff95138a740 1 -- 172.21.15.18:0/228417144 shutdown_connections 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.096+0000 7ff95138a740 1 -- 172.21.15.18:0/228417144 wait complete. 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.096+0000 7ff95138a740 1 -- start start 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff95138a740 1 Processor -- start 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff95138a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0380 0x557ceee196e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff95138a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceee19c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff95138a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb98b0 0x557ceedbfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff95138a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ceedc0120 con 0x557ceedb0380 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff95138a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ceedc0290 con 0x557ceedb0d30 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff95138a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ceedc0450 con 0x557ceedb98b0 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff95138a740 10 client.0 _ll_register_callbacks cb 0x7ff93c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff947fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0380 0x557ceee196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff947fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0380 0x557ceee196e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45864/0 (socket says 172.21.15.18:45864) 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff947fff640 1 -- 172.21.15.18:0/4003167473 learned_addr learned my addr 172.21.15.18:0/4003167473 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff94cc7e640 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb98b0 0x557ceedbfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff9477fe640 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceee19c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff947fff640 1 -- 172.21.15.18:0/4003167473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb98b0 msgr2=0x557ceedbfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff947fff640 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb98b0 0x557ceedbfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff947fff640 1 -- 172.21.15.18:0/4003167473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 msgr2=0x557ceee19c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff947fff640 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceee19c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff947fff640 1 -- 172.21.15.18:0/4003167473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ceed16320 con 0x557ceedb0380 2024-02-17T12:55:11.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff9477fe640 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceee19c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff947fff640 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0380 0x557ceee196e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff93802c240 tx=0x7ff938000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d61439703ca5d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.097+0000 7ff9277fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff9277fe640 1 -- 172.21.15.18:0/4003167473 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93802c9e0 con 0x557ceedb0380 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff95138a740 1 -- 172.21.15.18:0/4003167473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ceed3d900 con 0x557ceedb0380 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff9277fe640 1 -- 172.21.15.18:0/4003167473 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff938044040 con 0x557ceedb0380 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff9277fe640 1 -- 172.21.15.18:0/4003167473 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff93802cb40 con 0x557ceedb0380 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff95138a740 10 client.17505 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff95138a740 1 -- 172.21.15.18:0/4003167473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557ceecd8910 con 0x557ceedb0380 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff9277fe640 1 -- 172.21.15.18:0/4003167473 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557ceed3d900 con 0x557ceedb0380 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff9277fe640 1 -- 172.21.15.18:0/4003167473 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff93803c040 con 0x557ceedb0380 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff95138a740 10 client.17505 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff95138a740 10 client.17505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff95138a740 20 client.17505 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff95138a740 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557ceee227a0 0x557ceee24bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff95138a740 4 client.17505 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:55:11.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff95138a740 1 -- 172.21.15.18:0/4003167473 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557ceea17860 con 0x557ceee227a0 2024-02-17T12:55:11.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.098+0000 7ff9477fe640 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557ceee227a0 0x557ceee24bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.099+0000 7ff9477fe640 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557ceee227a0 0x557ceee24bc0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=722fa069e0dc5a03 server_cookie=93af4951e3328abc in_seq=0 out_seq=0 2024-02-17T12:55:11.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.099+0000 7ff9277fe640 10 client.17505 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:11.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff9277fe640 1 -- 172.21.15.18:0/4003167473 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557ceea17860 con 0x557ceee227a0 2024-02-17T12:55:11.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff9277fe640 10 client.17505 handle_command_reply: tid=0 2024-02-17T12:55:11.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff95138a740 1 client.17505 shutdown 2024-02-17T12:55:11.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff95138a740 1 -- 172.21.15.18:0/4003167473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0380 msgr2=0x557ceee196e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff95138a740 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0380 0x557ceee196e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff93802c240 tx=0x7ff938000bc0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff95138a740 1 -- 172.21.15.18:0/4003167473 shutdown_connections 2024-02-17T12:55:11.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff95138a740 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557ceee227a0 0x557ceee24bc0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff95138a740 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ceedb98b0 0x557ceedbfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff95138a740 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ceedb0d30 0x557ceee19c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff9277fe640 0 client.17505 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:11.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff95138a740 1 --2- 172.21.15.18:0/4003167473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ceedb0380 0x557ceee196e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff95138a740 1 -- 172.21.15.18:0/4003167473 >> 172.21.15.18:0/4003167473 conn(0x557ceedaa840 msgr2=0x557ceeda98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:11.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff95138a740 1 -- 172.21.15.18:0/4003167473 shutdown_connections 2024-02-17T12:55:11.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff95138a740 1 -- 172.21.15.18:0/4003167473 wait complete. 2024-02-17T12:55:11.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.100+0000 7ff95138a740 20 client.17505 trim_cache size 0 max 16384 2024-02-17T12:55:11.098 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:55:11.098 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:55:11.098 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "d7c4a596-7b81-4e32-8df2-d5f598a70c20", 2024-02-17T12:55:11.098 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:55:11.098 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:55:11.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.101+0000 7ff94e6e1640 1 -- 172.21.15.18:0/2188324598 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff9300a2260 msgr2=0x7ff9300a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.101+0000 7ff94e6e1640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff9300a2260 0x7ff9300a4720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff93c012900 tx=0x7ff93c009ab0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.101+0000 7ff94e6e1640 1 -- 172.21.15.18:0/2188324598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff948073f00 msgr2=0x7ff9481a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.101+0000 7ff94e6e1640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff948073f00 0x7ff9481a5a00 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff94000eb10 tx=0x7ff94000c2d0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.101+0000 7ff94e6e1640 1 -- 172.21.15.18:0/2188324598 shutdown_connections 2024-02-17T12:55:11.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.101+0000 7ff94e6e1640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff9300a2260 0x7ff9300a4720 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.101+0000 7ff94e6e1640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff948073f00 0x7ff9481a5a00 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.101+0000 7ff94e6e1640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff948106460 0x7ff9480739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.101+0000 7ff94e6e1640 1 --2- 172.21.15.18:0/2188324598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff948101390 0x7ff948073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.101+0000 7ff94e6e1640 1 -- 172.21.15.18:0/2188324598 >> 172.21.15.18:0/2188324598 conn(0x7ff9480fcbb0 msgr2=0x7ff9480fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:11.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.101+0000 7ff94e6e1640 1 -- 172.21.15.18:0/2188324598 shutdown_connections 2024-02-17T12:55:11.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.101+0000 7ff94e6e1640 1 -- 172.21.15.18:0/2188324598 wait complete. 2024-02-17T12:55:11.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:55:11.103 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:55:11.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.263+0000 7f00de32e640 1 -- 172.21.15.18:0/2975711966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d8105340 msgr2=0x7f00d810cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.263+0000 7f00de32e640 1 --2- 172.21.15.18:0/2975711966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d8105340 0x7f00d810cf10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009cd0 tx=0x7f00d002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.263+0000 7f00de32e640 1 -- 172.21.15.18:0/2975711966 shutdown_connections 2024-02-17T12:55:11.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.263+0000 7f00de32e640 1 --2- 172.21.15.18:0/2975711966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d8105340 0x7f00d810cf10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.263+0000 7f00de32e640 1 --2- 172.21.15.18:0/2975711966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d8100c20 0x7f00d8104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.263+0000 7f00de32e640 1 --2- 172.21.15.18:0/2975711966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d8100270 0x7f00d8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.263+0000 7f00de32e640 1 -- 172.21.15.18:0/2975711966 >> 172.21.15.18:0/2975711966 conn(0x7f00d8077230 msgr2=0x7f00d8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:11.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.263+0000 7f00de32e640 1 -- 172.21.15.18:0/2975711966 shutdown_connections 2024-02-17T12:55:11.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.263+0000 7f00de32e640 1 -- 172.21.15.18:0/2975711966 wait complete. 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00de32e640 1 Processor -- start 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00de32e640 1 -- start start 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00de32e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d8100270 0x7f00d819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00de32e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d8105340 0x7f00d819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00de32e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d81975a0 0x7f00d8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00de32e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d8197f90 con 0x7f00d81975a0 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00de32e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d8198100 con 0x7f00d8105340 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00de32e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d81982c0 con 0x7f00d8100270 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d8100270 0x7f00d819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00dc8cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d81975a0 0x7f00d8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d8100270 0x7f00d819d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:41212/0 (socket says 172.21.15.18:41212) 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00d7fff640 1 -- 172.21.15.18:0/3210414884 learned_addr learned my addr 172.21.15.18:0/3210414884 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.264+0000 7f00d77fe640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d8105340 0x7f00d819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.265+0000 7f00d7fff640 1 -- 172.21.15.18:0/3210414884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d8105340 msgr2=0x7f00d819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.265+0000 7f00d7fff640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d8105340 0x7f00d819d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.265+0000 7f00d7fff640 1 -- 172.21.15.18:0/3210414884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d81975a0 msgr2=0x7f00d8197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.265+0000 7f00d7fff640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d81975a0 0x7f00d8197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.265+0000 7f00d7fff640 1 -- 172.21.15.18:0/3210414884 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d0009900 con 0x7f00d8100270 2024-02-17T12:55:11.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.265+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d81975a0 0x7f00d8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:11.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.265+0000 7f00d7fff640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d8100270 0x7f00d819d3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f00c800e9f0 tx=0x7f00c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:11.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.265+0000 7f00d57fa640 1 -- 172.21.15.18:0/3210414884 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8009700 con 0x7f00d8100270 2024-02-17T12:55:11.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.265+0000 7f00de32e640 1 -- 172.21.15.18:0/3210414884 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d81a42b0 con 0x7f00d8100270 2024-02-17T12:55:11.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.265+0000 7f00d57fa640 1 -- 172.21.15.18:0/3210414884 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00c800cd80 con 0x7f00d8100270 2024-02-17T12:55:11.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.265+0000 7f00d57fa640 1 -- 172.21.15.18:0/3210414884 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8018920 con 0x7f00d8100270 2024-02-17T12:55:11.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.265+0000 7f00de32e640 1 -- 172.21.15.18:0/3210414884 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d81a47c0 con 0x7f00d8100270 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00e0fd7740 1 Processor -- start 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00d57fa640 1 -- 172.21.15.18:0/3210414884 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f00c8018a80 con 0x7f00d8100270 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00e0fd7740 1 -- start start 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00e0fd7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d3562b0 0x55be1d356690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00e0fd7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d356bd0 0x55be1d35f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00e0fd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d35f7b0 0x55be1d361ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00e0fd7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1d2c5280 con 0x55be1d3562b0 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00e0fd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1d2c27b0 con 0x55be1d35f7b0 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00e0fd7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1d2bdf90 con 0x55be1d356bd0 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00dc8cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d3562b0 0x55be1d356690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00dc8cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d3562b0 0x55be1d356690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45888/0 (socket says 172.21.15.18:45888) 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00dc8cb640 1 -- 172.21.15.18:0/1197539000 learned_addr learned my addr 172.21.15.18:0/1197539000 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:11.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00d7fff640 1 --2- 172.21.15.18:0/1197539000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d356bd0 0x55be1d35f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.268+0000 7f00d77fe640 1 --2- 172.21.15.18:0/1197539000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d35f7b0 0x55be1d361ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00d57fa640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f00c00a2230 0x7f00c00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00dc8cb640 1 -- 172.21.15.18:0/1197539000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d356bd0 msgr2=0x55be1d35f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/1197539000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d356bd0 0x55be1d35f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00dc8cb640 1 -- 172.21.15.18:0/1197539000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d35f7b0 msgr2=0x55be1d361ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/1197539000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d35f7b0 0x55be1d361ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00dc8cb640 1 -- 172.21.15.18:0/1197539000 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be1d27c570 con 0x55be1d3562b0 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00d7fff640 1 --2- 172.21.15.18:0/1197539000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d356bd0 0x55be1d35f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00d57fa640 1 -- 172.21.15.18:0/3210414884 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f00c8014070 con 0x7f00d8100270 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/1197539000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d3562b0 0x55be1d356690 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f00d002c700 tx=0x7f00d0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b406511f45595775 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f00c00a2230 0x7f00c00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00be7fc640 1 -- 172.21.15.18:0/1197539000 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0034c90 con 0x55be1d3562b0 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00be7fc640 1 -- 172.21.15.18:0/1197539000 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00d0032eb0 con 0x55be1d3562b0 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.269+0000 7f00be7fc640 1 -- 172.21.15.18:0/1197539000 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0050070 con 0x55be1d3562b0 2024-02-17T12:55:11.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.270+0000 7f00e0fd7740 1 -- 172.21.15.18:0/1197539000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d3562b0 msgr2=0x55be1d356690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.270+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f00c00a2230 0x7f00c00a46f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f00d004b000 tx=0x7f00d004e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.270+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/1197539000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d3562b0 0x55be1d356690 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f00d002c700 tx=0x7f00d0005ef0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.270+0000 7f00e0fd7740 1 -- 172.21.15.18:0/1197539000 shutdown_connections 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.270+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/1197539000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d35f7b0 0x55be1d361ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.270+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/1197539000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d356bd0 0x55be1d35f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.270+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/1197539000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d3562b0 0x55be1d356690 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.270+0000 7f00e0fd7740 1 -- 172.21.15.18:0/1197539000 >> 172.21.15.18:0/1197539000 conn(0x55be1d350990 msgr2=0x55be1d350d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.270+0000 7f00e0fd7740 1 -- 172.21.15.18:0/1197539000 shutdown_connections 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.270+0000 7f00e0fd7740 1 -- 172.21.15.18:0/1197539000 wait complete. 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.270+0000 7f00e0fd7740 1 -- start start 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00e0fd7740 1 Processor -- start 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00e0fd7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d3562b0 0x55be1d35f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00e0fd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d356bd0 0x55be1d35c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00e0fd7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 0x55be1d35ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00e0fd7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1cf91430 con 0x55be1d35f7b0 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00e0fd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1cf4de00 con 0x55be1d356bd0 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00e0fd7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1d242fd0 con 0x55be1d3562b0 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00e0fd7740 10 client.0 _ll_register_callbacks cb 0x55be1d298fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00dc8cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 0x55be1d35ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00d77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d356bd0 0x55be1d35c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d3562b0 0x55be1d35f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00d77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d356bd0 0x55be1d35c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:45804/0 (socket says 172.21.15.18:45804) 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00d77fe640 1 -- 172.21.15.18:0/2915516984 learned_addr learned my addr 172.21.15.18:0/2915516984 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00dc8cb640 1 -- 172.21.15.18:0/2915516984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d3562b0 msgr2=0x55be1d35f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/2915516984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d3562b0 0x55be1d35f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00dc8cb640 1 -- 172.21.15.18:0/2915516984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d356bd0 msgr2=0x55be1d35c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/2915516984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d356bd0 0x55be1d35c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00dc8cb640 1 -- 172.21.15.18:0/2915516984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d810d720 con 0x55be1d35f7b0 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/2915516984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 0x55be1d35ccd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f00d002c630 tx=0x7f00d00045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3647a47a59e0f5e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.271+0000 7f00b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00b77fe640 1 -- 172.21.15.18:0/2915516984 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0050040 con 0x55be1d35f7b0 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2915516984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be1cffa720 con 0x55be1d35f7b0 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00b77fe640 1 -- 172.21.15.18:0/2915516984 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f00d0032b00 con 0x55be1d35f7b0 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00b77fe640 1 -- 172.21.15.18:0/2915516984 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00d0003970 con 0x55be1d35f7b0 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00e0fd7740 10 client.17517 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2915516984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55be1d2bc610 con 0x55be1d35f7b0 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00b77fe640 1 -- 172.21.15.18:0/2915516984 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55be1cffa720 con 0x55be1d35f7b0 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00b77fe640 1 -- 172.21.15.18:0/2915516984 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f00d004cc60 con 0x55be1d35f7b0 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00e0fd7740 10 client.17517 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00e0fd7740 10 client.17517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00e0fd7740 20 client.17517 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:11.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/2915516984 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55be1d36f590 0x55be1d3719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00e0fd7740 4 client.17517 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.272+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2915516984 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be1d049890 con 0x55be1d36f590 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.273+0000 7f00d7fff640 1 --2- 172.21.15.18:0/2915516984 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55be1d36f590 0x55be1d3719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.273+0000 7f00d7fff640 1 --2- 172.21.15.18:0/2915516984 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55be1d36f590 0x55be1d3719b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9651388f036dec11 server_cookie=da7b57aec589562f in_seq=0 out_seq=0 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.273+0000 7f00b77fe640 10 client.17517 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.273+0000 7f00b77fe640 1 -- 172.21.15.18:0/2915516984 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be1d049890 con 0x55be1d36f590 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.273+0000 7f00b77fe640 10 client.17517 handle_command_reply: tid=0 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.273+0000 7f00e0fd7740 1 client.17517 shutdown 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.274+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2915516984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 msgr2=0x55be1d35ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.274+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/2915516984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 0x55be1d35ccd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f00d002c630 tx=0x7f00d00045d0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.274+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2915516984 shutdown_connections 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.274+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/2915516984 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55be1d36f590 0x55be1d3719b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.274+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/2915516984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 0x55be1d35ccd0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.274+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/2915516984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d356bd0 0x55be1d35c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.274+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/2915516984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d3562b0 0x55be1d35f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.274+0000 7f00b77fe640 0 client.17517 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:11.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.274+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2915516984 >> 172.21.15.18:0/2915516984 conn(0x55be1d350990 msgr2=0x55be1d352730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:11.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.274+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2915516984 shutdown_connections 2024-02-17T12:55:11.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.274+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2915516984 wait complete. 2024-02-17T12:55:11.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.274+0000 7f00e0fd7740 20 client.17517 trim_cache size 0 max 16384 2024-02-17T12:55:11.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 289532 6.76 MB/sec execute 146 sec latency 577.136 ms 2024-02-17T12:55:11.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00e0fd7740 1 Processor -- start 2024-02-17T12:55:11.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00e0fd7740 1 -- start start 2024-02-17T12:55:11.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00e0fd7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d3562b0 0x55be1d35c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00e0fd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d356bd0 0x55be1d35ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00e0fd7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 0x55be1d35f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00e0fd7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1d361fa0 con 0x55be1d35f7b0 2024-02-17T12:55:11.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00e0fd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1d36c4b0 con 0x55be1d356bd0 2024-02-17T12:55:11.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00e0fd7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1d34c2b0 con 0x55be1d3562b0 2024-02-17T12:55:11.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00dc8cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 0x55be1d35f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00dc8cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 0x55be1d35f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45900/0 (socket says 172.21.15.18:45900) 2024-02-17T12:55:11.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00dc8cb640 1 -- 172.21.15.18:0/2084619035 learned_addr learned my addr 172.21.15.18:0/2084619035 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:11.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00d7fff640 1 --2- 172.21.15.18:0/2084619035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d3562b0 0x55be1d35c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.290+0000 7f00d77fe640 1 --2- 172.21.15.18:0/2084619035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d356bd0 0x55be1d35ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.291+0000 7f00dc8cb640 1 -- 172.21.15.18:0/2084619035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d3562b0 msgr2=0x55be1d35c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.291+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/2084619035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d3562b0 0x55be1d35c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.291+0000 7f00dc8cb640 1 -- 172.21.15.18:0/2084619035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d356bd0 msgr2=0x55be1d35ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.291+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/2084619035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d356bd0 0x55be1d35ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.291+0000 7f00dc8cb640 1 -- 172.21.15.18:0/2084619035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be1d258ac0 con 0x55be1d35f7b0 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.291+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/2084619035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 0x55be1d35f200 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f00d0002230 tx=0x7f00d0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d13f930408023a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.291+0000 7f00bdffb640 1 -- 172.21.15.18:0/2084619035 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0050070 con 0x55be1d35f7b0 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.291+0000 7f00bdffb640 1 -- 172.21.15.18:0/2084619035 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00d0003970 con 0x55be1d35f7b0 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.291+0000 7f00bdffb640 1 -- 172.21.15.18:0/2084619035 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0004120 con 0x55be1d35f7b0 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2084619035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 msgr2=0x55be1d35f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/2084619035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 0x55be1d35f200 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f00d0002230 tx=0x7f00d0000f40 comp rx=0 tx=0).stop 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2084619035 shutdown_connections 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/2084619035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d35f7b0 0x55be1d35f200 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/2084619035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d356bd0 0x55be1d35ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/2084619035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d3562b0 0x55be1d35c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2084619035 >> 172.21.15.18:0/2084619035 conn(0x55be1d350990 msgr2=0x55be1d35be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:11.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2084619035 shutdown_connections 2024-02-17T12:55:11.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 -- 172.21.15.18:0/2084619035 wait complete. 2024-02-17T12:55:11.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 -- start start 2024-02-17T12:55:11.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 Processor -- start 2024-02-17T12:55:11.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d3562b0 0x55be1d366ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d356bd0 0x55be1d364e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d35f7b0 0x55be1d3654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1d365ba0 con 0x55be1d356bd0 2024-02-17T12:55:11.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1d365d10 con 0x55be1d3562b0 2024-02-17T12:55:11.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1d3c5ad0 con 0x55be1d35f7b0 2024-02-17T12:55:11.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.292+0000 7f00e0fd7740 10 client.0 _ll_register_callbacks cb 0x7f00c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:11.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00d77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d356bd0 0x55be1d364e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00d77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d356bd0 0x55be1d364e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45902/0 (socket says 172.21.15.18:45902) 2024-02-17T12:55:11.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d3562b0 0x55be1d366ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00d77fe640 1 -- 172.21.15.18:0/13099062 learned_addr learned my addr 172.21.15.18:0/13099062 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:11.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00dc8cb640 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d35f7b0 0x55be1d3654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00d77fe640 1 -- 172.21.15.18:0/13099062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d35f7b0 msgr2=0x55be1d3654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00d77fe640 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d35f7b0 0x55be1d3654d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00d77fe640 1 -- 172.21.15.18:0/13099062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d3562b0 msgr2=0x55be1d366ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:11.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00d77fe640 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d3562b0 0x55be1d366ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00d77fe640 1 -- 172.21.15.18:0/13099062 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be1d2bc610 con 0x55be1d356bd0 2024-02-17T12:55:11.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00d7fff640 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d3562b0 0x55be1d366ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:11.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00d77fe640 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d356bd0 0x55be1d364e50 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f00cc012730 tx=0x7f00cc012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2bbc1fc3eb1fa7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00b77fe640 1 -- 172.21.15.18:0/13099062 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc017040 con 0x55be1d356bd0 2024-02-17T12:55:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00b77fe640 1 -- 172.21.15.18:0/13099062 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f00cc004230 con 0x55be1d356bd0 2024-02-17T12:55:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00e0fd7740 1 -- 172.21.15.18:0/13099062 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be1cfbd7e0 con 0x55be1d356bd0 2024-02-17T12:55:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00b77fe640 1 -- 172.21.15.18:0/13099062 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00cc0043f0 con 0x55be1d356bd0 2024-02-17T12:55:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00e0fd7740 10 client.17523 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00e0fd7740 1 -- 172.21.15.18:0/13099062 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55be1d27e6a0 con 0x55be1d356bd0 2024-02-17T12:55:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.293+0000 7f00b77fe640 1 -- 172.21.15.18:0/13099062 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00cc02c020 con 0x55be1d356bd0 2024-02-17T12:55:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.294+0000 7f00b77fe640 1 -- 172.21.15.18:0/13099062 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f00cc003bc0 con 0x55be1d356bd0 2024-02-17T12:55:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.294+0000 7f00e0fd7740 10 client.17523 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.294+0000 7f00e0fd7740 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.294+0000 7f00e0fd7740 20 client.17523 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.294+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55be1d3c74b0 0x55be1d3c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.294+0000 7f00e0fd7740 4 client.17523 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.294+0000 7f00e0fd7740 1 -- 172.21.15.18:0/13099062 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55be1cfac4e0 con 0x55be1d3c74b0 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.294+0000 7f00d7fff640 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55be1d3c74b0 0x55be1d3c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.294+0000 7f00d7fff640 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55be1d3c74b0 0x55be1d3c98d0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8d33c86e52fb88 server_cookie=683c4cb87cb249ff in_seq=0 out_seq=0 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.294+0000 7f00b77fe640 10 client.17523 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00b77fe640 1 -- 172.21.15.18:0/13099062 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55be1cfac4e0 con 0x55be1d3c74b0 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00b77fe640 10 client.17523 handle_command_reply: tid=0 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00e0fd7740 1 client.17523 shutdown 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00e0fd7740 1 -- 172.21.15.18:0/13099062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d356bd0 msgr2=0x55be1d364e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d356bd0 0x55be1d364e50 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f00cc012730 tx=0x7f00cc012b30 comp rx=0 tx=0).stop 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00e0fd7740 1 -- 172.21.15.18:0/13099062 shutdown_connections 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55be1d3c74b0 0x55be1d3c98d0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be1d35f7b0 0x55be1d3654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be1d356bd0 0x55be1d364e50 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00b77fe640 0 client.17523 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00e0fd7740 1 --2- 172.21.15.18:0/13099062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be1d3562b0 0x55be1d366ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00e0fd7740 1 -- 172.21.15.18:0/13099062 >> 172.21.15.18:0/13099062 conn(0x55be1d350990 msgr2=0x55be1d35ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00e0fd7740 1 -- 172.21.15.18:0/13099062 shutdown_connections 2024-02-17T12:55:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00e0fd7740 1 -- 172.21.15.18:0/13099062 wait complete. 2024-02-17T12:55:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00e0fd7740 20 client.17523 trim_cache size 0 max 16384 2024-02-17T12:55:11.295 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:55:11.295 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:55:11.296 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:55:11.296 INFO:teuthology.orchestra.run.smithi018.stdout: "d7c4a596-7b81-4e32-8df2-d5f598a70c20": { 2024-02-17T12:55:11.296 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:55:11.296 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "d7c4a596-7b81-4e32-8df2-d5f598a70c20", 2024-02-17T12:55:11.296 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:55:11.296 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:55:11.296 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:55:11.296 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:55:11.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00de32e640 1 -- 172.21.15.18:0/3210414884 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f00c00a2230 msgr2=0x7f00c00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00de32e640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f00c00a2230 0x7f00c00a46f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f00d004b000 tx=0x7f00d004e000 comp rx=0 tx=0).stop 2024-02-17T12:55:11.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00de32e640 1 -- 172.21.15.18:0/3210414884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d8100270 msgr2=0x7f00d819d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:11.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00de32e640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d8100270 0x7f00d819d3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f00c800e9f0 tx=0x7f00c800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00de32e640 1 -- 172.21.15.18:0/3210414884 shutdown_connections 2024-02-17T12:55:11.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.295+0000 7f00de32e640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f00c00a2230 0x7f00c00a46f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.296+0000 7f00de32e640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00d81975a0 0x7f00d8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.296+0000 7f00de32e640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d8105340 0x7f00d819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.296+0000 7f00de32e640 1 --2- 172.21.15.18:0/3210414884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00d8100270 0x7f00d819d3f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:11.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.296+0000 7f00de32e640 1 -- 172.21.15.18:0/3210414884 >> 172.21.15.18:0/3210414884 conn(0x7f00d8077230 msgr2=0x7f00d8068560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:11.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.296+0000 7f00de32e640 1 -- 172.21.15.18:0/3210414884 shutdown_connections 2024-02-17T12:55:11.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:11.296+0000 7f00de32e640 1 -- 172.21.15.18:0/3210414884 wait complete. 2024-02-17T12:55:11.299 INFO:tasks.cephfs.filesystem:scrub status for tag:d7c4a596-7b81-4e32-8df2-d5f598a70c20 - {'path': '/', 'tag': 'd7c4a596-7b81-4e32-8df2-d5f598a70c20', 'options': 'recursive,force'} 2024-02-17T12:55:11.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:11 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:11.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:11 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:11.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:11 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:11.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:11 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:11.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:11 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:11.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:11 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:11.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:11 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:11.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:11 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:11.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:11 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:11.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:11 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:11.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:11 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:11.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:11 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:11 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:12.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:11 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:11 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:12.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:11 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:11 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:12.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:11 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 291274 6.81 MB/sec execute 147 sec latency 14.962 ms 2024-02-17T12:55:12.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: pgmap v162: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-02-17T12:55:12.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:55:12.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:12.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:12.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:12.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:12.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:12.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:12.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: pgmap v162: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:12.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:13.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: pgmap v162: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-02-17T12:55:13.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:55:13.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:13.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:13.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:13.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:13.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:13.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:13.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:13.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:13.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:13.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:13.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:13.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:13.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:13.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 292567 6.80 MB/sec execute 148 sec latency 210.325 ms 2024-02-17T12:55:14.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 293482 6.77 MB/sec execute 149 sec latency 817.950 ms 2024-02-17T12:55:14.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:14 smithi119 ceph-mon[36188]: pgmap v163: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-02-17T12:55:14.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:14 smithi146 ceph-mon[36454]: pgmap v163: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-02-17T12:55:15.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:14 smithi018 ceph-mon[32625]: pgmap v163: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-02-17T12:55:15.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 295105 6.80 MB/sec execute 150 sec latency 14.528 ms 2024-02-17T12:55:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.212+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1166403746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed81007f0 msgr2=0x7f8ed8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.212+0000 7f8ee06c3640 1 --2- 172.21.15.18:0/1166403746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed81007f0 0x7f8ed8100c50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009900 tx=0x7f8ec802bc80 comp rx=0 tx=0).stop 2024-02-17T12:55:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.212+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1166403746 shutdown_connections 2024-02-17T12:55:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.212+0000 7f8ee06c3640 1 --2- 172.21.15.18:0/1166403746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ed8101190 0x7f8ed810cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.212+0000 7f8ee06c3640 1 --2- 172.21.15.18:0/1166403746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed81007f0 0x7f8ed8100c50 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.212+0000 7f8ee06c3640 1 --2- 172.21.15.18:0/1166403746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8105050 0x7f8ed8105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.212+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1166403746 >> 172.21.15.18:0/1166403746 conn(0x7f8ed8076610 msgr2=0x7f8ed80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.212+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1166403746 shutdown_connections 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.212+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1166403746 wait complete. 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ee06c3640 1 Processor -- start 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ee06c3640 1 -- start start 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ee06c3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed81007f0 0x7f8ed810d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ee06c3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ed8101190 0x7f8ed810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ee06c3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8105050 0x7f8ed810e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ee06c3640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed81a5ae0 con 0x7f8ed81007f0 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ee06c3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed81a5c50 con 0x7f8ed8101190 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ee06c3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed81a5e10 con 0x7f8ed8105050 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ede45f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed81007f0 0x7f8ed810d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ede45f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed81007f0 0x7f8ed810d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45922/0 (socket says 172.21.15.18:45922) 2024-02-17T12:55:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ede45f640 1 -- 172.21.15.18:0/1706418555 learned_addr learned my addr 172.21.15.18:0/1706418555 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8edec60640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8105050 0x7f8ed810e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ede45f640 1 -- 172.21.15.18:0/1706418555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8105050 msgr2=0x7f8ed810e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ede45f640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8105050 0x7f8ed810e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ede45f640 1 -- 172.21.15.18:0/1706418555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ed8101190 msgr2=0x7f8ed810de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8eddc5e640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ed8101190 0x7f8ed810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ede45f640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ed8101190 0x7f8ed810de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ede45f640 1 -- 172.21.15.18:0/1706418555 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec8009560 con 0x7f8ed81007f0 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ede45f640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed81007f0 0x7f8ed810d940 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400d770 tx=0x7f8ed400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.213+0000 7f8ecf7fe640 1 -- 172.21.15.18:0/1706418555 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4009c30 con 0x7f8ed81007f0 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.214+0000 7f8ecf7fe640 1 -- 172.21.15.18:0/1706418555 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ed400de80 con 0x7f8ed81007f0 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.214+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1706418555 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed81a60f0 con 0x7f8ed81007f0 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.214+0000 7f8ecf7fe640 1 -- 172.21.15.18:0/1706418555 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4004e80 con 0x7f8ed81007f0 2024-02-17T12:55:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.214+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1706418555 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed81a65b0 con 0x7f8ed81007f0 2024-02-17T12:55:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.216+0000 7f8ecf7fe640 1 -- 172.21.15.18:0/1706418555 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8ed4005020 con 0x7f8ed81007f0 2024-02-17T12:55:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.216+0000 7f8ee3371740 1 Processor -- start 2024-02-17T12:55:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.216+0000 7f8ee3371740 1 -- start start 2024-02-17T12:55:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.216+0000 7f8ecf7fe640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8ec40a2300 0x7f8ec40a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.216+0000 7f8ee3371740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb305ad0 0x560cbb304820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.216+0000 7f8ee3371740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 0x560cbb3051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.216+0000 7f8ee3371740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb30e2e0 0x560cbb3106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.216+0000 7f8ee3371740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbb2739f0 con 0x560cbb304d60 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.216+0000 7f8ee3371740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbb270f20 con 0x560cbb30e2e0 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.216+0000 7f8ee3371740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbb26c700 con 0x560cbb305ad0 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.216+0000 7f8ecf7fe640 1 -- 172.21.15.18:0/1706418555 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8ed40cda30 con 0x7f8ed81007f0 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8edec60640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8ec40a2300 0x7f8ec40a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8eddc5e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 0x560cbb3051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8eddc5e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 0x560cbb3051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45930/0 (socket says 172.21.15.18:45930) 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8ede45f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb305ad0 0x560cbb304820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8eddc5e640 1 -- 172.21.15.18:0/1044898045 learned_addr learned my addr 172.21.15.18:0/1044898045 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8edec60640 1 --2- 172.21.15.18:0/1044898045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb30e2e0 0x560cbb3106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8ede45f640 1 -- 172.21.15.18:0/1044898045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb30e2e0 msgr2=0x560cbb3106d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8ede45f640 1 --2- 172.21.15.18:0/1044898045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb30e2e0 0x560cbb3106d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8ede45f640 1 -- 172.21.15.18:0/1044898045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 msgr2=0x560cbb3051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8ede45f640 1 --2- 172.21.15.18:0/1044898045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 0x560cbb3051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8ede45f640 1 -- 172.21.15.18:0/1044898045 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cbb2027b0 con 0x560cbb305ad0 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8edec60640 1 --2- 172.21.15.18:0/1044898045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb30e2e0 0x560cbb3106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8edec60640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8ec40a2300 0x7f8ec40a47c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0013750 tx=0x7f8ed0013b50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8ede45f640 1 --2- 172.21.15.18:0/1044898045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb305ad0 0x560cbb304820 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40cb1d0 tx=0x7f8ed4099dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de674872a3f13edd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8eccff9640 1 -- 172.21.15.18:0/1044898045 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed404ba50 con 0x560cbb305ad0 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8eccff9640 1 -- 172.21.15.18:0/1044898045 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ed409ab70 con 0x560cbb305ad0 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.217+0000 7f8eccff9640 1 -- 172.21.15.18:0/1044898045 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed40cb450 con 0x560cbb305ad0 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.218+0000 7f8ee3371740 1 -- 172.21.15.18:0/1044898045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb305ad0 msgr2=0x560cbb304820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.218+0000 7f8ee3371740 1 --2- 172.21.15.18:0/1044898045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb305ad0 0x560cbb304820 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40cb1d0 tx=0x7f8ed4099dd0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.218+0000 7f8ee3371740 1 -- 172.21.15.18:0/1044898045 shutdown_connections 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.218+0000 7f8ee3371740 1 --2- 172.21.15.18:0/1044898045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb30e2e0 0x560cbb3106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.218+0000 7f8ee3371740 1 --2- 172.21.15.18:0/1044898045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 0x560cbb3051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.218+0000 7f8ee3371740 1 --2- 172.21.15.18:0/1044898045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb305ad0 0x560cbb304820 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.218+0000 7f8ee3371740 1 -- 172.21.15.18:0/1044898045 >> 172.21.15.18:0/1044898045 conn(0x560cbb2cc550 msgr2=0x560cbb2ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.218+0000 7f8ee3371740 1 -- 172.21.15.18:0/1044898045 shutdown_connections 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.218+0000 7f8ee3371740 1 -- 172.21.15.18:0/1044898045 wait complete. 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8ee3371740 1 -- start start 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8ee3371740 1 Processor -- start 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8ee3371740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb304d60 0x560cbb30b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8ee3371740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb305ad0 0x560cbb30b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8ee3371740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb30e2e0 0x560cbb30be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8ee3371740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbb211bf0 con 0x560cbb30e2e0 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8ee3371740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbaf40250 con 0x560cbb305ad0 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8ee3371740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbb1f2200 con 0x560cbb304d60 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8ee3371740 10 client.0 _ll_register_callbacks cb 0x560cbb260d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8edec60640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb30e2e0 0x560cbb30be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8ede45f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb304d60 0x560cbb30b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8edec60640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb30e2e0 0x560cbb30be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45934/0 (socket says 172.21.15.18:45934) 2024-02-17T12:55:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8edec60640 1 -- 172.21.15.18:0/2571783481 learned_addr learned my addr 172.21.15.18:0/2571783481 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8eddc5e640 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb305ad0 0x560cbb30b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8edec60640 1 -- 172.21.15.18:0/2571783481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb304d60 msgr2=0x560cbb30b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8edec60640 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb304d60 0x560cbb30b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8edec60640 1 -- 172.21.15.18:0/2571783481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb305ad0 msgr2=0x560cbb30b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8edec60640 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb305ad0 0x560cbb30b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8edec60640 1 -- 172.21.15.18:0/2571783481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed81a65b0 con 0x560cbb30e2e0 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8eddc5e640 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb305ad0 0x560cbb30b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8edec60640 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb30e2e0 0x560cbb30be00 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0011d70 tx=0x7f8ed0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d156a2ea586c7e81 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.219+0000 7f8ec1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ec1ffb640 1 -- 172.21.15.18:0/2571783481 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0020620 con 0x560cbb30e2e0 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ec1ffb640 1 -- 172.21.15.18:0/2571783481 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8ed0020780 con 0x560cbb30e2e0 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ee3371740 1 -- 172.21.15.18:0/2571783481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cbaf6c600 con 0x560cbb30e2e0 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ec1ffb640 1 -- 172.21.15.18:0/2571783481 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ed0020940 con 0x560cbb30e2e0 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ee3371740 10 client.17535 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ee3371740 1 -- 172.21.15.18:0/2571783481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560cbb26ad80 con 0x560cbb30e2e0 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ec1ffb640 1 -- 172.21.15.18:0/2571783481 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560cbaf6c600 con 0x560cbb30e2e0 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ec1ffb640 1 -- 172.21.15.18:0/2571783481 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8ed000ebc0 con 0x560cbb30e2e0 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ee3371740 10 client.17535 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ee3371740 10 client.17535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ee3371740 20 client.17535 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ee3371740 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560cbb31e7c0 0x560cbb320be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ee3371740 4 client.17535 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.220+0000 7f8ee3371740 1 -- 172.21.15.18:0/2571783481 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cbafbd1b0 con 0x560cbb31e7c0 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.221+0000 7f8ede45f640 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560cbb31e7c0 0x560cbb320be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.221+0000 7f8ec1ffb640 10 client.17535 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.221+0000 7f8ede45f640 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560cbb31e7c0 0x560cbb320be0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f3ad42b41b72b1 server_cookie=e71a7b005a9c4d30 in_seq=0 out_seq=0 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ec1ffb640 1 -- 172.21.15.18:0/2571783481 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560cbafbd1b0 con 0x560cbb31e7c0 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ec1ffb640 10 client.17535 handle_command_reply: tid=0 2024-02-17T12:55:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ee3371740 1 client.17535 shutdown 2024-02-17T12:55:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ee3371740 1 -- 172.21.15.18:0/2571783481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb30e2e0 msgr2=0x560cbb30be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ee3371740 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb30e2e0 0x560cbb30be00 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0011d70 tx=0x7f8ed0004ab0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ee3371740 1 -- 172.21.15.18:0/2571783481 shutdown_connections 2024-02-17T12:55:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ee3371740 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560cbb31e7c0 0x560cbb320be0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ee3371740 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb30e2e0 0x560cbb30be00 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ee3371740 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb305ad0 0x560cbb30b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ee3371740 1 --2- 172.21.15.18:0/2571783481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb304d60 0x560cbb30b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ee3371740 1 -- 172.21.15.18:0/2571783481 >> 172.21.15.18:0/2571783481 conn(0x560cbb2cc550 msgr2=0x560cbb3017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ec1ffb640 0 client.17535 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ee3371740 1 -- 172.21.15.18:0/2571783481 shutdown_connections 2024-02-17T12:55:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ee3371740 1 -- 172.21.15.18:0/2571783481 wait complete. 2024-02-17T12:55:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.222+0000 7f8ee3371740 20 client.17535 trim_cache size 0 max 16384 2024-02-17T12:55:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ee3371740 1 Processor -- start 2024-02-17T12:55:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ee3371740 1 -- start start 2024-02-17T12:55:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ee3371740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 0x560cbb30dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ee3371740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb305ad0 0x560cbb30bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ee3371740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb30e2e0 0x560cbb309c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ee3371740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbb31b1d0 con 0x560cbb304d60 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ee3371740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbb31e5b0 con 0x560cbb305ad0 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ee3371740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbb304030 con 0x560cbb30e2e0 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ede45f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 0x560cbb30dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ede45f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 0x560cbb30dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45942/0 (socket says 172.21.15.18:45942) 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ede45f640 1 -- 172.21.15.18:0/3242814455 learned_addr learned my addr 172.21.15.18:0/3242814455 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8eddc5e640 1 --2- 172.21.15.18:0/3242814455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb305ad0 0x560cbb30bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8edec60640 1 --2- 172.21.15.18:0/3242814455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb30e2e0 0x560cbb309c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ede45f640 1 -- 172.21.15.18:0/3242814455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb30e2e0 msgr2=0x560cbb309c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ede45f640 1 --2- 172.21.15.18:0/3242814455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb30e2e0 0x560cbb309c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ede45f640 1 -- 172.21.15.18:0/3242814455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb305ad0 msgr2=0x560cbb30bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ede45f640 1 --2- 172.21.15.18:0/3242814455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb305ad0 0x560cbb30bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ede45f640 1 -- 172.21.15.18:0/3242814455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cbb2077b0 con 0x560cbb304d60 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8edec60640 1 --2- 172.21.15.18:0/3242814455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb30e2e0 0x560cbb309c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.236+0000 7f8ede45f640 1 --2- 172.21.15.18:0/3242814455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 0x560cbb30dab0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40968d0 tx=0x7f8ed4096040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d82ad146693e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.237+0000 7f8ec3fff640 1 -- 172.21.15.18:0/3242814455 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed409a470 con 0x560cbb304d60 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.237+0000 7f8ec3fff640 1 -- 172.21.15.18:0/3242814455 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ed409e5c0 con 0x560cbb304d60 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.237+0000 7f8ec3fff640 1 -- 172.21.15.18:0/3242814455 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4096ae0 con 0x560cbb304d60 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.237+0000 7f8ee3371740 1 -- 172.21.15.18:0/3242814455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 msgr2=0x560cbb30dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.237+0000 7f8ee3371740 1 --2- 172.21.15.18:0/3242814455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 0x560cbb30dab0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40968d0 tx=0x7f8ed4096040 comp rx=0 tx=0).stop 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.237+0000 7f8ee3371740 1 -- 172.21.15.18:0/3242814455 shutdown_connections 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.237+0000 7f8ee3371740 1 --2- 172.21.15.18:0/3242814455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb30e2e0 0x560cbb309c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.237+0000 7f8ee3371740 1 --2- 172.21.15.18:0/3242814455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb305ad0 0x560cbb30bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.237+0000 7f8ee3371740 1 --2- 172.21.15.18:0/3242814455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb304d60 0x560cbb30dab0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.237+0000 7f8ee3371740 1 -- 172.21.15.18:0/3242814455 >> 172.21.15.18:0/3242814455 conn(0x560cbb2cc550 msgr2=0x560cbb306640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.237+0000 7f8ee3371740 1 -- 172.21.15.18:0/3242814455 shutdown_connections 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.237+0000 7f8ee3371740 1 -- 172.21.15.18:0/3242814455 wait complete. 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8ee3371740 1 -- start start 2024-02-17T12:55:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8ee3371740 1 Processor -- start 2024-02-17T12:55:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8ee3371740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb304d60 0x560cbb312df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8ee3371740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb305ad0 0x560cbb313330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8ee3371740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb30e2e0 0x560cbb3138b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8ee3371740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbb373e80 con 0x560cbb305ad0 2024-02-17T12:55:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8ee3371740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbb373ff0 con 0x560cbb304d60 2024-02-17T12:55:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8ee3371740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbb3741b0 con 0x560cbb30e2e0 2024-02-17T12:55:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8ee3371740 10 client.0 _ll_register_callbacks cb 0x7f8ed4004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8eddc5e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb305ad0 0x560cbb313330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8eddc5e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb305ad0 0x560cbb313330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45954/0 (socket says 172.21.15.18:45954) 2024-02-17T12:55:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8eddc5e640 1 -- 172.21.15.18:0/1621148818 learned_addr learned my addr 172.21.15.18:0/1621148818 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8ede45f640 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb304d60 0x560cbb312df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8edec60640 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb30e2e0 0x560cbb3138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8eddc5e640 1 -- 172.21.15.18:0/1621148818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb30e2e0 msgr2=0x560cbb3138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8eddc5e640 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb30e2e0 0x560cbb3138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8eddc5e640 1 -- 172.21.15.18:0/1621148818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb304d60 msgr2=0x560cbb312df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8eddc5e640 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb304d60 0x560cbb312df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8eddc5e640 1 -- 172.21.15.18:0/1621148818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cbb2027b0 con 0x560cbb305ad0 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8eddc5e640 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb305ad0 0x560cbb313330 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8ec802c7a0 tx=0x7f8ec80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e371fb9712ece4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.238+0000 7f8ec1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ec1ffb640 1 -- 172.21.15.18:0/1621148818 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8004b20 con 0x560cbb305ad0 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ec1ffb640 1 -- 172.21.15.18:0/1621148818 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8ec8004450 con 0x560cbb305ad0 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ee3371740 1 -- 172.21.15.18:0/1621148818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ed409ab70 con 0x560cbb305ad0 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ec1ffb640 1 -- 172.21.15.18:0/1621148818 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ec8004df0 con 0x560cbb305ad0 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ee3371740 10 client.17541 fetch_fsmap learned FSMap version 35 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ee3371740 1 -- 172.21.15.18:0/1621148818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560cbb22d540 con 0x560cbb305ad0 2024-02-17T12:55:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ec1ffb640 1 -- 172.21.15.18:0/1621148818 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ed409ab70 con 0x560cbb305ad0 2024-02-17T12:55:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ec1ffb640 1 -- 172.21.15.18:0/1621148818 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8ec8046020 con 0x560cbb305ad0 2024-02-17T12:55:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ee3371740 10 client.17541 fetch_fsmap finished waiting for FSMap version 35 2024-02-17T12:55:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ee3371740 10 client.17541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ee3371740 20 client.17541 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ee3371740 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560cbb375a20 0x560cbb377e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ee3371740 4 client.17541 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:55:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ee3371740 1 -- 172.21.15.18:0/1621148818 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560cbafa9550 con 0x560cbb375a20 2024-02-17T12:55:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.239+0000 7f8ede45f640 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560cbb375a20 0x560cbb377e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.240+0000 7f8ec1ffb640 10 client.17541 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.240+0000 7f8ede45f640 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560cbb375a20 0x560cbb377e40 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b85913eedf0efbf server_cookie=9fa7f5d9195dccaf in_seq=0 out_seq=0 2024-02-17T12:55:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ec1ffb640 1 -- 172.21.15.18:0/1621148818 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560cbafa9550 con 0x560cbb375a20 2024-02-17T12:55:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ec1ffb640 10 client.17541 handle_command_reply: tid=0 2024-02-17T12:55:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ee3371740 1 client.17541 shutdown 2024-02-17T12:55:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ee3371740 1 -- 172.21.15.18:0/1621148818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb305ad0 msgr2=0x560cbb313330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ee3371740 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb305ad0 0x560cbb313330 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8ec802c7a0 tx=0x7f8ec80048f0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ee3371740 1 -- 172.21.15.18:0/1621148818 shutdown_connections 2024-02-17T12:55:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ee3371740 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560cbb375a20 0x560cbb377e40 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ee3371740 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560cbb30e2e0 0x560cbb3138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ec1ffb640 0 client.17541 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ee3371740 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbb305ad0 0x560cbb313330 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ee3371740 1 --2- 172.21.15.18:0/1621148818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560cbb304d60 0x560cbb312df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ee3371740 1 -- 172.21.15.18:0/1621148818 >> 172.21.15.18:0/1621148818 conn(0x560cbb2cc550 msgr2=0x560cbb309c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ee3371740 1 -- 172.21.15.18:0/1621148818 shutdown_connections 2024-02-17T12:55:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.272+0000 7f8ee3371740 1 -- 172.21.15.18:0/1621148818 wait complete. 2024-02-17T12:55:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee3371740 20 client.17541 trim_cache size 0 max 16384 2024-02-17T12:55:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:55:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:55:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:55:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:55:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1706418555 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8ec40a2300 msgr2=0x7f8ec40a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee06c3640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8ec40a2300 0x7f8ec40a47c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0013750 tx=0x7f8ed0013b50 comp rx=0 tx=0).stop 2024-02-17T12:55:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1706418555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed81007f0 msgr2=0x7f8ed810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee06c3640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed81007f0 0x7f8ed810d940 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400d770 tx=0x7f8ed400dc30 comp rx=0 tx=0).stop 2024-02-17T12:55:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1706418555 shutdown_connections 2024-02-17T12:55:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee06c3640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8ec40a2300 0x7f8ec40a47c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee06c3640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8105050 0x7f8ed810e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee06c3640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ed8101190 0x7f8ed810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee06c3640 1 --2- 172.21.15.18:0/1706418555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed81007f0 0x7f8ed810d940 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1706418555 >> 172.21.15.18:0/1706418555 conn(0x7f8ed8076610 msgr2=0x7f8ed8102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1706418555 shutdown_connections 2024-02-17T12:55:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:16.273+0000 7f8ee06c3640 1 -- 172.21.15.18:0/1706418555 wait complete. 2024-02-17T12:55:16.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 296117 6.79 MB/sec execute 151 sec latency 320.035 ms 2024-02-17T12:55:16.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:16 smithi146 ceph-mon[36454]: pgmap v164: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 262 op/s 2024-02-17T12:55:16.950 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 12:55:16 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i[67657]: ignoring --setuser ceph since I am not root 2024-02-17T12:55:16.950 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 12:55:16 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i[67657]: ignoring --setgroup ceph since I am not root 2024-02-17T12:55:16.950 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 12:55:16 smithi146 ceph-mds[67661]: -- 172.21.15.146:0/549673641 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b855d1e1a0 con 0x55b855cf4c00 2024-02-17T12:55:16.950 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 12:55:16 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i[67657]: starting mds.i at 2024-02-17T12:55:17.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:16 smithi018 ceph-mon[32625]: pgmap v164: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 262 op/s 2024-02-17T12:55:17.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:16 smithi119 ceph-mon[36188]: pgmap v164: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 262 op/s 2024-02-17T12:55:17.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 296117 6.74 MB/sec execute 152 sec latency 1320.156 ms 2024-02-17T12:55:17.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:17 smithi146 ceph-mon[36454]: daemon mds.i restarted 2024-02-17T12:55:18.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:17 smithi018 ceph-mon[32625]: daemon mds.i restarted 2024-02-17T12:55:18.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:17 smithi119 ceph-mon[36188]: daemon mds.i restarted 2024-02-17T12:55:18.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 297191 6.74 MB/sec execute 153 sec latency 1897.344 ms 2024-02-17T12:55:18.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:18 smithi146 ceph-mon[36454]: pgmap v165: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-02-17T12:55:18.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:18 smithi146 ceph-mon[36454]: osdmap e90: 12 total, 12 up, 12 in 2024-02-17T12:55:18.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:18 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.146:6838/1419682628,v1:172.21.15.146:6839/1419682628] up:boot 2024-02-17T12:55:18.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:18 smithi146 ceph-mon[36454]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-02-17T12:55:18.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:18 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T12:55:18.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:18 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:55:19.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:18 smithi018 ceph-mon[32625]: pgmap v165: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-02-17T12:55:19.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:18 smithi018 ceph-mon[32625]: osdmap e90: 12 total, 12 up, 12 in 2024-02-17T12:55:19.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:18 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.146:6838/1419682628,v1:172.21.15.146:6839/1419682628] up:boot 2024-02-17T12:55:19.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:18 smithi018 ceph-mon[32625]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-02-17T12:55:19.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:18 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T12:55:19.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:18 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:55:19.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:18 smithi119 ceph-mon[36188]: pgmap v165: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-02-17T12:55:19.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:18 smithi119 ceph-mon[36188]: osdmap e90: 12 total, 12 up, 12 in 2024-02-17T12:55:19.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:18 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.146:6838/1419682628,v1:172.21.15.146:6839/1419682628] up:boot 2024-02-17T12:55:19.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:18 smithi119 ceph-mon[36188]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-02-17T12:55:19.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:18 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T12:55:19.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:18 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:55:19.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 298725 6.75 MB/sec execute 154 sec latency 16.515 ms 2024-02-17T12:55:19.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:19 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:55:20.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:19 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:55:20.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:19 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:55:20.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 299665 6.74 MB/sec execute 155 sec latency 378.397 ms 2024-02-17T12:55:20.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:20 smithi146 ceph-mon[36454]: pgmap v167: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2024-02-17T12:55:21.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:20 smithi018 ceph-mon[32625]: pgmap v167: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2024-02-17T12:55:21.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:20 smithi119 ceph-mon[36188]: pgmap v167: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2024-02-17T12:55:21.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 299665 6.69 MB/sec execute 156 sec latency 1378.520 ms 2024-02-17T12:55:22.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 299665 6.65 MB/sec execute 157 sec latency 2378.618 ms 2024-02-17T12:55:22.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:22 smithi146 ceph-mon[36454]: pgmap v168: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2024-02-17T12:55:23.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:22 smithi018 ceph-mon[32625]: pgmap v168: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2024-02-17T12:55:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:22 smithi119 ceph-mon[36188]: pgmap v168: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2024-02-17T12:55:23.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 299665 6.61 MB/sec execute 158 sec latency 3378.691 ms 2024-02-17T12:55:24.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 301658 6.65 MB/sec execute 159 sec latency 3579.535 ms 2024-02-17T12:55:24.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:24 smithi146 ceph-mon[36454]: pgmap v169: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2024-02-17T12:55:25.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:24 smithi018 ceph-mon[32625]: pgmap v169: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2024-02-17T12:55:25.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:24 smithi119 ceph-mon[36188]: pgmap v169: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2024-02-17T12:55:25.275 INFO:tasks.workunit.client.0.smithi018.stdout: 1 302993 6.65 MB/sec execute 160 sec latency 16.983 ms 2024-02-17T12:55:25.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:25 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:55:25.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:25 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:55:26.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:25 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:55:26.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:25 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:55:26.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:25 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:55:26.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:25 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:55:26.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 303214 6.61 MB/sec execute 161 sec latency 761.635 ms 2024-02-17T12:55:26.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:26 smithi146 ceph-mon[36454]: pgmap v170: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 834 KiB/s wr, 139 op/s 2024-02-17T12:55:26.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:26 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:55:26.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:26 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:55:26.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:26 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:55:27.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:26 smithi018 ceph-mon[32625]: pgmap v170: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 834 KiB/s wr, 139 op/s 2024-02-17T12:55:27.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:26 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:55:27.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:26 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:55:27.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:26 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:55:27.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:26 smithi119 ceph-mon[36188]: pgmap v170: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 834 KiB/s wr, 139 op/s 2024-02-17T12:55:27.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:26 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:55:27.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:26 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:55:27.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:26 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:55:27.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 303214 6.56 MB/sec execute 162 sec latency 1761.735 ms 2024-02-17T12:55:28.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 303214 6.52 MB/sec execute 163 sec latency 2761.850 ms 2024-02-17T12:55:28.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:28 smithi146 ceph-mon[36454]: pgmap v171: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 834 KiB/s wr, 139 op/s 2024-02-17T12:55:29.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:28 smithi018 ceph-mon[32625]: pgmap v171: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 834 KiB/s wr, 139 op/s 2024-02-17T12:55:29.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:28 smithi119 ceph-mon[36188]: pgmap v171: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 834 KiB/s wr, 139 op/s 2024-02-17T12:55:29.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 304129 6.50 MB/sec execute 164 sec latency 3374.497 ms 2024-02-17T12:55:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.365+0000 7f26318c0640 1 -- 172.21.15.18:0/708798531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f262c1002d0 msgr2=0x7f262c100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.365+0000 7f26318c0640 1 --2- 172.21.15.18:0/708798531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f262c1002d0 0x7f262c100750 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2614009900 tx=0x7f261402bc30 comp rx=0 tx=0).stop 2024-02-17T12:55:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.365+0000 7f26318c0640 1 -- 172.21.15.18:0/708798531 shutdown_connections 2024-02-17T12:55:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.365+0000 7f26318c0640 1 --2- 172.21.15.18:0/708798531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f262c100c90 0x7f262c10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.365+0000 7f26318c0640 1 --2- 172.21.15.18:0/708798531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f262c1002d0 0x7f262c100750 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.365+0000 7f26318c0640 1 --2- 172.21.15.18:0/708798531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f262c105020 0x7f262c105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.365+0000 7f26318c0640 1 -- 172.21.15.18:0/708798531 >> 172.21.15.18:0/708798531 conn(0x7f262c076ed0 msgr2=0x7f262c0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.365+0000 7f26318c0640 1 -- 172.21.15.18:0/708798531 shutdown_connections 2024-02-17T12:55:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.365+0000 7f26318c0640 1 -- 172.21.15.18:0/708798531 wait complete. 2024-02-17T12:55:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f26318c0640 1 Processor -- start 2024-02-17T12:55:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f26318c0640 1 -- start start 2024-02-17T12:55:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f26318c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f262c1002d0 0x7f262c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f26318c0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f262c100c90 0x7f262c19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f26318c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f262c105020 0x7f262c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f26318c0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c197c30 con 0x7f262c100c90 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f26318c0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c197da0 con 0x7f262c1002d0 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f26318c0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c197f60 con 0x7f262c105020 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f2623fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f262c100c90 0x7f262c19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f262affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f262c1002d0 0x7f262c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f2623fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f262c100c90 0x7f262c19d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39622/0 (socket says 172.21.15.18:39622) 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f2623fff640 1 -- 172.21.15.18:0/1785778749 learned_addr learned my addr 172.21.15.18:0/1785778749 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f262affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f262c1002d0 0x7f262c19d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:39992/0 (socket says 172.21.15.18:39992) 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.366+0000 7f262b7fe640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f262c105020 0x7f262c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.367+0000 7f2623fff640 1 -- 172.21.15.18:0/1785778749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f262c105020 msgr2=0x7f262c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.367+0000 7f2623fff640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f262c105020 0x7f262c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.367+0000 7f2623fff640 1 -- 172.21.15.18:0/1785778749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f262c1002d0 msgr2=0x7f262c19d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.367+0000 7f2623fff640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f262c1002d0 0x7f262c19d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.367+0000 7f2623fff640 1 -- 172.21.15.18:0/1785778749 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2614009560 con 0x7f262c100c90 2024-02-17T12:55:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.367+0000 7f2623fff640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f262c100c90 0x7f262c19d920 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2614000c00 tx=0x7f261402c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.367+0000 7f26308be640 1 -- 172.21.15.18:0/1785778749 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261402dea0 con 0x7f262c100c90 2024-02-17T12:55:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.367+0000 7f26318c0640 1 -- 172.21.15.18:0/1785778749 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262c1981e0 con 0x7f262c100c90 2024-02-17T12:55:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.367+0000 7f26308be640 1 -- 172.21.15.18:0/1785778749 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2614004410 con 0x7f262c100c90 2024-02-17T12:55:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.367+0000 7f26308be640 1 -- 172.21.15.18:0/1785778749 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614003bb0 con 0x7f262c100c90 2024-02-17T12:55:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.367+0000 7f26318c0640 1 -- 172.21.15.18:0/1785778749 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c1a4480 con 0x7f262c100c90 2024-02-17T12:55:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.369+0000 7f26308be640 1 -- 172.21.15.18:0/1785778749 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2614003d10 con 0x7f262c100c90 2024-02-17T12:55:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.369+0000 7f2634569740 1 Processor -- start 2024-02-17T12:55:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.369+0000 7f2634569740 1 -- start start 2024-02-17T12:55:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f26308be640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f26180a22d0 0x7f26180a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f2634569740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f2385ad0 0x55c0f2384820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f2634569740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f23851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f2634569740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f238e2e0 0x55c0f23906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f2634569740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f22f39f0 con 0x55c0f2385ad0 2024-02-17T12:55:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f2634569740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f22f0f20 con 0x55c0f2384d60 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f2634569740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f22ec700 con 0x55c0f238e2e0 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f262b7fe640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f26180a22d0 0x7f26180a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f262affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f2385ad0 0x55c0f2384820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f262affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f2385ad0 0x55c0f2384820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39636/0 (socket says 172.21.15.18:39636) 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f262affd640 1 -- 172.21.15.18:0/461128987 learned_addr learned my addr 172.21.15.18:0/461128987 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f26308be640 1 -- 172.21.15.18:0/1785778749 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f26140f0520 con 0x7f262c100c90 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f2623fff640 1 --2- 172.21.15.18:0/461128987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f23851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f262b7fe640 1 --2- 172.21.15.18:0/461128987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f238e2e0 0x55c0f23906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f262affd640 1 -- 172.21.15.18:0/461128987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f238e2e0 msgr2=0x55c0f23906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f262affd640 1 --2- 172.21.15.18:0/461128987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f238e2e0 0x55c0f23906d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f262affd640 1 -- 172.21.15.18:0/461128987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 msgr2=0x55c0f23851e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f262affd640 1 --2- 172.21.15.18:0/461128987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f23851e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f262affd640 1 -- 172.21.15.18:0/461128987 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0f22827b0 con 0x55c0f2385ad0 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f2623fff640 1 --2- 172.21.15.18:0/461128987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f23851e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f262b7fe640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f26180a22d0 0x7f26180a4790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2624005fd0 tx=0x7f2624013b60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.370+0000 7f262affd640 1 --2- 172.21.15.18:0/461128987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f2385ad0 0x55c0f2384820 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f261c009d20 tx=0x7f261c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc972b2ddeceda87 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.371+0000 7f2621ffb640 1 -- 172.21.15.18:0/461128987 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c019620 con 0x55c0f2385ad0 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.371+0000 7f2621ffb640 1 -- 172.21.15.18:0/461128987 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f261c019c60 con 0x55c0f2385ad0 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.371+0000 7f2621ffb640 1 -- 172.21.15.18:0/461128987 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c018c30 con 0x55c0f2385ad0 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.371+0000 7f2634569740 1 -- 172.21.15.18:0/461128987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f2385ad0 msgr2=0x55c0f2384820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.371+0000 7f2634569740 1 --2- 172.21.15.18:0/461128987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f2385ad0 0x55c0f2384820 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f261c009d20 tx=0x7f261c011810 comp rx=0 tx=0).stop 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.371+0000 7f2634569740 1 -- 172.21.15.18:0/461128987 shutdown_connections 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.371+0000 7f2634569740 1 --2- 172.21.15.18:0/461128987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f238e2e0 0x55c0f23906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.371+0000 7f2634569740 1 --2- 172.21.15.18:0/461128987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f23851e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.371+0000 7f2634569740 1 --2- 172.21.15.18:0/461128987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f2385ad0 0x55c0f2384820 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.371+0000 7f2634569740 1 -- 172.21.15.18:0/461128987 >> 172.21.15.18:0/461128987 conn(0x55c0f234c550 msgr2=0x55c0f237fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.371+0000 7f2634569740 1 -- 172.21.15.18:0/461128987 shutdown_connections 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.371+0000 7f2634569740 1 -- 172.21.15.18:0/461128987 wait complete. 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f2634569740 1 -- start start 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f2634569740 1 Processor -- start 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f2634569740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f238b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f2634569740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 0x55c0f238b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f2634569740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f238bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f2634569740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f2291bf0 con 0x55c0f238e2e0 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f2634569740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f1fc0250 con 0x55c0f2384d60 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f2634569740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f2272200 con 0x55c0f2385ad0 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f2634569740 10 client.0 _ll_register_callbacks cb 0x55c0f22e0d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f262b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f238bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f262b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f238bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39644/0 (socket says 172.21.15.18:39644) 2024-02-17T12:55:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f262affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f238b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f262b7fe640 1 -- 172.21.15.18:0/2613766296 learned_addr learned my addr 172.21.15.18:0/2613766296 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.372+0000 7f2623fff640 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 0x55c0f238b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f262b7fe640 1 -- 172.21.15.18:0/2613766296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 msgr2=0x55c0f238b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f262b7fe640 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 0x55c0f238b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f262b7fe640 1 -- 172.21.15.18:0/2613766296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 msgr2=0x55c0f238b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f262b7fe640 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f238b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f262b7fe640 1 -- 172.21.15.18:0/2613766296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261c009950 con 0x55c0f238e2e0 2024-02-17T12:55:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f262affd640 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f238b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f262b7fe640 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f238bd10 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f26240095a0 tx=0x7f2624004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30dc1fbcc41d24b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f260affd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f260affd640 1 -- 172.21.15.18:0/2613766296 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624004740 con 0x55c0f238e2e0 2024-02-17T12:55:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f260affd640 1 -- 172.21.15.18:0/2613766296 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2624020430 con 0x55c0f238e2e0 2024-02-17T12:55:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f2634569740 1 -- 172.21.15.18:0/2613766296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0f1fec600 con 0x55c0f238e2e0 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f260affd640 1 -- 172.21.15.18:0/2613766296 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26240048a0 con 0x55c0f238e2e0 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f2634569740 10 client.17556 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f2634569740 1 -- 172.21.15.18:0/2613766296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c0f22ead80 con 0x55c0f238e2e0 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f260affd640 1 -- 172.21.15.18:0/2613766296 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c0f1fec600 con 0x55c0f238e2e0 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f260affd640 1 -- 172.21.15.18:0/2613766296 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f2624010700 con 0x55c0f238e2e0 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f2634569740 10 client.17556 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f2634569740 10 client.17556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f2634569740 20 client.17556 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f2634569740 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c0f239e730 0x55c0f23a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f2634569740 4 client.17556 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.373+0000 7f2634569740 1 -- 172.21.15.18:0/2613766296 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0f203d1b0 con 0x55c0f239e730 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.374+0000 7f262affd640 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c0f239e730 0x55c0f23a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.374+0000 7f262affd640 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c0f239e730 0x55c0f23a0b50 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d67337e6f305688 server_cookie=82df674e26766dd in_seq=0 out_seq=0 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.374+0000 7f260affd640 10 client.17556 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f260affd640 1 -- 172.21.15.18:0/2613766296 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c0f203d1b0 con 0x55c0f239e730 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f260affd640 10 client.17556 handle_command_reply: tid=0 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f2634569740 1 client.17556 shutdown 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f2634569740 1 -- 172.21.15.18:0/2613766296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 msgr2=0x55c0f238bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f2634569740 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f238bd10 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f26240095a0 tx=0x7f2624004ab0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f2634569740 1 -- 172.21.15.18:0/2613766296 shutdown_connections 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f2634569740 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c0f239e730 0x55c0f23a0b50 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f2634569740 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f238bd10 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f2634569740 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 0x55c0f238b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f260affd640 0 client.17556 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f2634569740 1 --2- 172.21.15.18:0/2613766296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f238b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f2634569740 1 -- 172.21.15.18:0/2613766296 >> 172.21.15.18:0/2613766296 conn(0x55c0f234c550 msgr2=0x55c0f23817a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f2634569740 1 -- 172.21.15.18:0/2613766296 shutdown_connections 2024-02-17T12:55:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f2634569740 1 -- 172.21.15.18:0/2613766296 wait complete. 2024-02-17T12:55:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.375+0000 7f2634569740 20 client.17556 trim_cache size 0 max 16384 2024-02-17T12:55:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f2634569740 1 Processor -- start 2024-02-17T12:55:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f2634569740 1 -- start start 2024-02-17T12:55:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f2634569740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f238d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f2634569740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 0x55c0f238bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f2634569740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f23884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f2634569740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f239ce30 con 0x55c0f238e2e0 2024-02-17T12:55:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f2634569740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f2380620 con 0x55c0f2384d60 2024-02-17T12:55:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f2634569740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f239b0f0 con 0x55c0f2385ad0 2024-02-17T12:55:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f262b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f23884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f262b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f23884f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39646/0 (socket says 172.21.15.18:39646) 2024-02-17T12:55:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f262b7fe640 1 -- 172.21.15.18:0/1547021342 learned_addr learned my addr 172.21.15.18:0/1547021342 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f2623fff640 1 --2- 172.21.15.18:0/1547021342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 0x55c0f238bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f262affd640 1 --2- 172.21.15.18:0/1547021342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f238d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f262b7fe640 1 -- 172.21.15.18:0/1547021342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 msgr2=0x55c0f238bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.389+0000 7f262b7fe640 1 --2- 172.21.15.18:0/1547021342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 0x55c0f238bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.390+0000 7f262b7fe640 1 -- 172.21.15.18:0/1547021342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 msgr2=0x55c0f238d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.390+0000 7f262b7fe640 1 --2- 172.21.15.18:0/1547021342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f238d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.390+0000 7f262b7fe640 1 -- 172.21.15.18:0/1547021342 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0f22877b0 con 0x55c0f238e2e0 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.390+0000 7f262b7fe640 1 --2- 172.21.15.18:0/1547021342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f23884f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2624009570 tx=0x7f2624009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535deb09286f784 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.390+0000 7f26217fa640 1 -- 172.21.15.18:0/1547021342 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624010c50 con 0x55c0f238e2e0 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.390+0000 7f26217fa640 1 -- 172.21.15.18:0/1547021342 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2624005b70 con 0x55c0f238e2e0 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.390+0000 7f26217fa640 1 -- 172.21.15.18:0/1547021342 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26240296b0 con 0x55c0f238e2e0 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.390+0000 7f2634569740 1 -- 172.21.15.18:0/1547021342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 msgr2=0x55c0f23884f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.390+0000 7f2634569740 1 --2- 172.21.15.18:0/1547021342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f23884f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2624009570 tx=0x7f2624009630 comp rx=0 tx=0).stop 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.390+0000 7f2634569740 1 -- 172.21.15.18:0/1547021342 shutdown_connections 2024-02-17T12:55:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 --2- 172.21.15.18:0/1547021342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f23884f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 --2- 172.21.15.18:0/1547021342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 0x55c0f238bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 --2- 172.21.15.18:0/1547021342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f238d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 -- 172.21.15.18:0/1547021342 >> 172.21.15.18:0/1547021342 conn(0x55c0f234c550 msgr2=0x55c0f2389b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 -- 172.21.15.18:0/1547021342 shutdown_connections 2024-02-17T12:55:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 -- 172.21.15.18:0/1547021342 wait complete. 2024-02-17T12:55:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 -- start start 2024-02-17T12:55:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 Processor -- start 2024-02-17T12:55:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f23951f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 0x55c0f23935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f2393b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f2394060 con 0x55c0f238e2e0 2024-02-17T12:55:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f23941d0 con 0x55c0f2384d60 2024-02-17T12:55:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f2394390 con 0x55c0f2385ad0 2024-02-17T12:55:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.391+0000 7f2634569740 10 client.0 _ll_register_callbacks cb 0x7f261c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f262b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f2393b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f262b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f2393b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39650/0 (socket says 172.21.15.18:39650) 2024-02-17T12:55:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f262b7fe640 1 -- 172.21.15.18:0/4264963047 learned_addr learned my addr 172.21.15.18:0/4264963047 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f262affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f23951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f2623fff640 1 --2- 172.21.15.18:0/4264963047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 0x55c0f23935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f262b7fe640 1 -- 172.21.15.18:0/4264963047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 msgr2=0x55c0f23935e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f262b7fe640 1 --2- 172.21.15.18:0/4264963047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 0x55c0f23935e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f262b7fe640 1 -- 172.21.15.18:0/4264963047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 msgr2=0x55c0f23951f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f262b7fe640 1 --2- 172.21.15.18:0/4264963047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f23951f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f262b7fe640 1 -- 172.21.15.18:0/4264963047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0f22ead80 con 0x55c0f238e2e0 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f262b7fe640 1 --2- 172.21.15.18:0/4264963047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f2393b20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2624009ab0 tx=0x7f2624010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4702c46a715095b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f260affd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f260affd640 1 -- 172.21.15.18:0/4264963047 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262400fd50 con 0x55c0f238e2e0 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f260affd640 1 -- 172.21.15.18:0/4264963047 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f262400f810 con 0x55c0f238e2e0 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f2634569740 1 -- 172.21.15.18:0/4264963047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0f2029550 con 0x55c0f238e2e0 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f260affd640 1 -- 172.21.15.18:0/4264963047 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2624010640 con 0x55c0f238e2e0 2024-02-17T12:55:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f2634569740 10 client.17562 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f2634569740 1 -- 172.21.15.18:0/4264963047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c0f22ad540 con 0x55c0f238e2e0 2024-02-17T12:55:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.392+0000 7f260affd640 1 -- 172.21.15.18:0/4264963047 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c0f2029550 con 0x55c0f238e2e0 2024-02-17T12:55:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.393+0000 7f260affd640 1 -- 172.21.15.18:0/4264963047 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f262403e9c0 con 0x55c0f238e2e0 2024-02-17T12:55:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.393+0000 7f2634569740 10 client.17562 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.393+0000 7f2634569740 10 client.17562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.393+0000 7f2634569740 20 client.17562 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.393+0000 7f2634569740 1 --2- 172.21.15.18:0/4264963047 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c0f23f5cb0 0x55c0f23f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.393+0000 7f2634569740 4 client.17562 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:55:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.393+0000 7f2634569740 1 -- 172.21.15.18:0/4264963047 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c0f1f98520 con 0x55c0f23f5cb0 2024-02-17T12:55:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.393+0000 7f262affd640 1 --2- 172.21.15.18:0/4264963047 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c0f23f5cb0 0x55c0f23f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.393+0000 7f262affd640 1 --2- 172.21.15.18:0/4264963047 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c0f23f5cb0 0x55c0f23f80d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983b5aaa608fdc7b server_cookie=a95e3e66af2f380c in_seq=0 out_seq=0 2024-02-17T12:55:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.393+0000 7f260affd640 10 client.17562 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f260affd640 1 -- 172.21.15.18:0/4264963047 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c0f1f98520 con 0x55c0f23f5cb0 2024-02-17T12:55:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f260affd640 10 client.17562 handle_command_reply: tid=0 2024-02-17T12:55:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f2634569740 1 client.17562 shutdown 2024-02-17T12:55:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f2634569740 1 -- 172.21.15.18:0/4264963047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 msgr2=0x55c0f2393b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f2634569740 1 --2- 172.21.15.18:0/4264963047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f2393b20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2624009ab0 tx=0x7f2624010d90 comp rx=0 tx=0).stop 2024-02-17T12:55:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f2634569740 1 -- 172.21.15.18:0/4264963047 shutdown_connections 2024-02-17T12:55:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f2634569740 1 --2- 172.21.15.18:0/4264963047 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c0f23f5cb0 0x55c0f23f80d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f2634569740 1 --2- 172.21.15.18:0/4264963047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c0f238e2e0 0x55c0f2393b20 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f2634569740 1 --2- 172.21.15.18:0/4264963047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0f2385ad0 0x55c0f23935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f2634569740 1 --2- 172.21.15.18:0/4264963047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c0f2384d60 0x55c0f23951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f260affd640 0 client.17562 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f2634569740 1 -- 172.21.15.18:0/4264963047 >> 172.21.15.18:0/4264963047 conn(0x55c0f234c550 msgr2=0x55c0f237e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f2634569740 1 -- 172.21.15.18:0/4264963047 shutdown_connections 2024-02-17T12:55:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f2634569740 1 -- 172.21.15.18:0/4264963047 wait complete. 2024-02-17T12:55:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.424+0000 7f2634569740 20 client.17562 trim_cache size 0 max 16384 2024-02-17T12:55:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:55:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:55:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:55:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:55:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.425+0000 7f26318c0640 1 -- 172.21.15.18:0/1785778749 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f26180a22d0 msgr2=0x7f26180a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.425+0000 7f26318c0640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f26180a22d0 0x7f26180a4790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2624005fd0 tx=0x7f2624013b60 comp rx=0 tx=0).stop 2024-02-17T12:55:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.425+0000 7f26318c0640 1 -- 172.21.15.18:0/1785778749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f262c100c90 msgr2=0x7f262c19d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.425+0000 7f26318c0640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f262c100c90 0x7f262c19d920 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2614000c00 tx=0x7f261402c6b0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.425+0000 7f26318c0640 1 -- 172.21.15.18:0/1785778749 shutdown_connections 2024-02-17T12:55:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.425+0000 7f26318c0640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f26180a22d0 0x7f26180a4790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.425+0000 7f26318c0640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f262c105020 0x7f262c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.425+0000 7f26318c0640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f262c100c90 0x7f262c19d920 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.425+0000 7f26318c0640 1 --2- 172.21.15.18:0/1785778749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f262c1002d0 0x7f262c19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.425+0000 7f26318c0640 1 -- 172.21.15.18:0/1785778749 >> 172.21.15.18:0/1785778749 conn(0x7f262c076ed0 msgr2=0x7f262c068a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.425+0000 7f26318c0640 1 -- 172.21.15.18:0/1785778749 shutdown_connections 2024-02-17T12:55:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:29.425+0000 7f26318c0640 1 -- 172.21.15.18:0/1785778749 wait complete. 2024-02-17T12:55:30.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 305376 6.53 MB/sec execute 165 sec latency 381.631 ms 2024-02-17T12:55:30.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:30 smithi146 ceph-mon[36454]: pgmap v172: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-02-17T12:55:31.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:30 smithi018 ceph-mon[32625]: pgmap v172: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-02-17T12:55:31.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:30 smithi119 ceph-mon[36188]: pgmap v172: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-02-17T12:55:31.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 306637 6.52 MB/sec execute 166 sec latency 15.196 ms 2024-02-17T12:55:32.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 306757 6.48 MB/sec execute 167 sec latency 922.612 ms 2024-02-17T12:55:32.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:32 smithi146 ceph-mon[36454]: pgmap v173: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 764 KiB/s wr, 94 op/s 2024-02-17T12:55:33.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:32 smithi018 ceph-mon[32625]: pgmap v173: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 764 KiB/s wr, 94 op/s 2024-02-17T12:55:33.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:32 smithi119 ceph-mon[36188]: pgmap v173: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 764 KiB/s wr, 94 op/s 2024-02-17T12:55:33.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 306757 6.44 MB/sec execute 168 sec latency 1922.697 ms 2024-02-17T12:55:34.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 307661 6.42 MB/sec execute 169 sec latency 2537.752 ms 2024-02-17T12:55:34.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:34 smithi146 ceph-mon[36454]: pgmap v174: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 764 KiB/s wr, 94 op/s 2024-02-17T12:55:35.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:34 smithi018 ceph-mon[32625]: pgmap v174: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 764 KiB/s wr, 94 op/s 2024-02-17T12:55:35.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:34 smithi119 ceph-mon[36188]: pgmap v174: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 764 KiB/s wr, 94 op/s 2024-02-17T12:55:35.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 309289 6.45 MB/sec execute 170 sec latency 14.982 ms 2024-02-17T12:55:36.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 310305 6.44 MB/sec execute 171 sec latency 312.969 ms 2024-02-17T12:55:36.894 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:36 smithi119 ceph-mon[36188]: pgmap v175: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-02-17T12:55:36.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:36 smithi146 ceph-mon[36454]: pgmap v175: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-02-17T12:55:37.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:36 smithi018 ceph-mon[32625]: pgmap v175: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-02-17T12:55:37.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 310305 6.41 MB/sec execute 172 sec latency 1313.085 ms 2024-02-17T12:55:38.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 311373 6.40 MB/sec execute 173 sec latency 1887.329 ms 2024-02-17T12:55:38.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:38 smithi146 ceph-mon[36454]: pgmap v176: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-02-17T12:55:39.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:38 smithi018 ceph-mon[32625]: pgmap v176: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-02-17T12:55:39.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:38 smithi119 ceph-mon[36188]: pgmap v176: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-02-17T12:55:39.276 INFO:tasks.workunit.client.0.smithi018.stdout: 1 312775 6.41 MB/sec execute 174 sec latency 23.539 ms 2024-02-17T12:55:40.277 INFO:tasks.workunit.client.0.smithi018.stdout: 1 313847 6.40 MB/sec execute 175 sec latency 440.207 ms 2024-02-17T12:55:40.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:40 smithi146 ceph-mon[36454]: pgmap v177: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-02-17T12:55:41.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:40 smithi018 ceph-mon[32625]: pgmap v177: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-02-17T12:55:41.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:40 smithi119 ceph-mon[36188]: pgmap v177: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-02-17T12:55:41.277 INFO:tasks.workunit.client.0.smithi018.stdout: 1 313847 6.37 MB/sec execute 176 sec latency 1440.323 ms 2024-02-17T12:55:41.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:55:41.300 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:55:41.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.425+0000 7f7002cc9640 1 -- 172.21.15.18:0/1595861538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ffc101380 msgr2=0x7f6ffc101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.425+0000 7f7002cc9640 1 --2- 172.21.15.18:0/1595861538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ffc101380 0x7f6ffc101760 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009cd0 tx=0x7f6ff402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.425+0000 7f7002cc9640 1 -- 172.21.15.18:0/1595861538 shutdown_connections 2024-02-17T12:55:41.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.425+0000 7f7002cc9640 1 --2- 172.21.15.18:0/1595861538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc106450 0x7f6ffc10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.425+0000 7f7002cc9640 1 --2- 172.21.15.18:0/1595861538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ffc101d30 0x7f6ffc105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.425+0000 7f7002cc9640 1 --2- 172.21.15.18:0/1595861538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ffc101380 0x7f6ffc101760 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.425+0000 7f7002cc9640 1 -- 172.21.15.18:0/1595861538 >> 172.21.15.18:0/1595861538 conn(0x7f6ffc0fcc20 msgr2=0x7f6ffc0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.425+0000 7f7002cc9640 1 -- 172.21.15.18:0/1595861538 shutdown_connections 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.425+0000 7f7002cc9640 1 -- 172.21.15.18:0/1595861538 wait complete. 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7002cc9640 1 Processor -- start 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7002cc9640 1 -- start start 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7002cc9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ffc101380 0x7f6ffc19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7002cc9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ffc106450 0x7f6ffc19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7002cc9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc197610 0x7f6ffc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7002cc9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc198000 con 0x7f6ffc101380 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7002cc9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc198170 con 0x7f6ffc197610 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7002cc9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc198330 con 0x7f6ffc106450 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7000a65640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ffc101380 0x7f6ffc19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7000a65640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ffc101380 0x7f6ffc19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50338/0 (socket says 172.21.15.18:50338) 2024-02-17T12:55:41.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7000a65640 1 -- 172.21.15.18:0/2813018911 learned_addr learned my addr 172.21.15.18:0/2813018911 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7001266640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc197610 0x7f6ffc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f6ffbfff640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ffc106450 0x7f6ffc19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7000a65640 1 -- 172.21.15.18:0/2813018911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ffc106450 msgr2=0x7f6ffc19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7000a65640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ffc106450 0x7f6ffc19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7000a65640 1 -- 172.21.15.18:0/2813018911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc197610 msgr2=0x7f6ffc197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7000a65640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc197610 0x7f6ffc197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f7000a65640 1 -- 172.21.15.18:0/2813018911 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff4009900 con 0x7f6ffc101380 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.426+0000 7f6ffbfff640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ffc106450 0x7f6ffc19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.427+0000 7f7000a65640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ffc101380 0x7f6ffc19d460 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4004af0 tx=0x7f6ff4034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.427+0000 7f6ff9ffb640 1 -- 172.21.15.18:0/2813018911 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4032ac0 con 0x7f6ffc101380 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.427+0000 7f6ff9ffb640 1 -- 172.21.15.18:0/2813018911 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ff4003e50 con 0x7f6ffc101380 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.427+0000 7f7002cc9640 1 -- 172.21.15.18:0/2813018911 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc1a4320 con 0x7f6ffc101380 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.427+0000 7f6ff9ffb640 1 -- 172.21.15.18:0/2813018911 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4040450 con 0x7f6ffc101380 2024-02-17T12:55:41.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.427+0000 7f7002cc9640 1 -- 172.21.15.18:0/2813018911 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc1a47a0 con 0x7f6ffc101380 2024-02-17T12:55:41.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.430+0000 7f6ff9ffb640 1 -- 172.21.15.18:0/2813018911 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6ff400a040 con 0x7f6ffc101380 2024-02-17T12:55:41.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.430+0000 7f700597a740 1 Processor -- start 2024-02-17T12:55:41.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.430+0000 7f700597a740 1 -- start start 2024-02-17T12:55:41.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.430+0000 7f700597a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec02b0 0x557770ec0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.430+0000 7f700597a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec0bd0 0x557770ec9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.430+0000 7f700597a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 0x557770ecbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.430+0000 7f700597a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557770e2f280 con 0x557770ec97b0 2024-02-17T12:55:41.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.430+0000 7f700597a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557770e2c7b0 con 0x557770ec02b0 2024-02-17T12:55:41.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.430+0000 7f700597a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557770e27f90 con 0x557770ec0bd0 2024-02-17T12:55:41.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.430+0000 7f6ff9ffb640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6fe80a2230 0x7f6fe80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.430+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 0x557770ecbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.430+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 0x557770ecbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50340/0 (socket says 172.21.15.18:50340) 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f6ffbfff640 1 -- 172.21.15.18:0/2054032858 learned_addr learned my addr 172.21.15.18:0/2054032858 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f7000a65640 1 --2- 172.21.15.18:0/2054032858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec0bd0 0x557770ec9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f7001266640 1 --2- 172.21.15.18:0/2054032858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec02b0 0x557770ec0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f7001266640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6fe80a2230 0x7f6fe80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f6ff9ffb640 1 -- 172.21.15.18:0/2813018911 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6ff40f0e10 con 0x7f6ffc101380 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f6ffbfff640 1 -- 172.21.15.18:0/2054032858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec0bd0 msgr2=0x557770ec9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f6ffbfff640 1 --2- 172.21.15.18:0/2054032858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec0bd0 0x557770ec9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f6ffbfff640 1 -- 172.21.15.18:0/2054032858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec02b0 msgr2=0x557770ec0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f6ffbfff640 1 --2- 172.21.15.18:0/2054032858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec02b0 0x557770ec0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f6ffbfff640 1 -- 172.21.15.18:0/2054032858 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557770de6570 con 0x557770ec97b0 2024-02-17T12:55:41.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f6ffbfff640 1 --2- 172.21.15.18:0/2054032858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 0x557770ecbba0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009bc0 tx=0x7f6fec00e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d42d326a75f6489 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f7001266640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6fe80a2230 0x7f6fe80a46f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4012870 tx=0x7f6fe400b2e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f6ff2ffd640 1 -- 172.21.15.18:0/2054032858 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec00ed20 con 0x557770ec97b0 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f6ff2ffd640 1 -- 172.21.15.18:0/2054032858 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fec00ee80 con 0x557770ec97b0 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.431+0000 7f6ff2ffd640 1 -- 172.21.15.18:0/2054032858 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec010900 con 0x557770ec97b0 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.432+0000 7f700597a740 1 -- 172.21.15.18:0/2054032858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 msgr2=0x557770ecbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.432+0000 7f700597a740 1 --2- 172.21.15.18:0/2054032858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 0x557770ecbba0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009bc0 tx=0x7f6fec00e2f0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.432+0000 7f700597a740 1 -- 172.21.15.18:0/2054032858 shutdown_connections 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.432+0000 7f700597a740 1 --2- 172.21.15.18:0/2054032858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 0x557770ecbba0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.432+0000 7f700597a740 1 --2- 172.21.15.18:0/2054032858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec0bd0 0x557770ec9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.432+0000 7f700597a740 1 --2- 172.21.15.18:0/2054032858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec02b0 0x557770ec0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.432+0000 7f700597a740 1 -- 172.21.15.18:0/2054032858 >> 172.21.15.18:0/2054032858 conn(0x557770eba990 msgr2=0x557770ebad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.432+0000 7f700597a740 1 -- 172.21.15.18:0/2054032858 shutdown_connections 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.432+0000 7f700597a740 1 -- 172.21.15.18:0/2054032858 wait complete. 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.432+0000 7f700597a740 1 -- start start 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.432+0000 7f700597a740 1 Processor -- start 2024-02-17T12:55:41.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f700597a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec02b0 0x557770ec91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f700597a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec0bd0 0x557770ec6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f700597a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 0x557770ec6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f700597a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557770afb430 con 0x557770ec97b0 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f700597a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557770ab7e00 con 0x557770ec0bd0 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f700597a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557770dacfd0 con 0x557770ec02b0 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f700597a740 10 client.0 _ll_register_callbacks cb 0x557770e02fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f7001266640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 0x557770ec6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f7001266640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 0x557770ec6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50346/0 (socket says 172.21.15.18:50346) 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f7001266640 1 -- 172.21.15.18:0/2531964146 learned_addr learned my addr 172.21.15.18:0/2531964146 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f6ffbfff640 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec0bd0 0x557770ec6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f7000a65640 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec02b0 0x557770ec91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f7001266640 1 -- 172.21.15.18:0/2531964146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec02b0 msgr2=0x557770ec91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f7001266640 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec02b0 0x557770ec91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f7001266640 1 -- 172.21.15.18:0/2531964146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec0bd0 msgr2=0x557770ec6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f7001266640 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec0bd0 0x557770ec6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f7001266640 1 -- 172.21.15.18:0/2531964146 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec009820 con 0x557770ec97b0 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f6ffbfff640 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec0bd0 0x557770ec6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f7001266640 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 0x557770ec6b40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4012df0 tx=0x7f6fe4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83541b1f00330637 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.433+0000 7f6fdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f6fdbfff640 1 -- 172.21.15.18:0/2531964146 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4009cc0 con 0x557770ec97b0 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f6fdbfff640 1 -- 172.21.15.18:0/2531964146 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6fe4009e20 con 0x557770ec97b0 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f700597a740 1 -- 172.21.15.18:0/2531964146 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557770b277e0 con 0x557770ec97b0 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f6fdbfff640 1 -- 172.21.15.18:0/2531964146 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fe40035f0 con 0x557770ec97b0 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f700597a740 10 client.17574 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f700597a740 1 -- 172.21.15.18:0/2531964146 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x557770e26610 con 0x557770ec97b0 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f6fdbfff640 1 -- 172.21.15.18:0/2531964146 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557770b277e0 con 0x557770ec97b0 2024-02-17T12:55:41.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f6fdbfff640 1 -- 172.21.15.18:0/2531964146 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f6fe4021b70 con 0x557770ec97b0 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f700597a740 10 client.17574 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f700597a740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f700597a740 20 client.17574 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f700597a740 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557770ed9a30 0x557770edbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f700597a740 4 client.17574 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.434+0000 7f700597a740 1 -- 172.21.15.18:0/2531964146 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557770b64720 con 0x557770ed9a30 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.435+0000 7f7000a65640 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557770ed9a30 0x557770edbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.435+0000 7f6fdbfff640 10 client.17574 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.435+0000 7f7000a65640 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557770ed9a30 0x557770edbe50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77d6d0e010fcc3eb server_cookie=37182da14c5790ff in_seq=0 out_seq=0 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f6fdbfff640 1 -- 172.21.15.18:0/2531964146 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557770b64720 con 0x557770ed9a30 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f6fdbfff640 10 client.17574 handle_command_reply: tid=0 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f700597a740 1 client.17574 shutdown 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f700597a740 1 -- 172.21.15.18:0/2531964146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 msgr2=0x557770ec6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f700597a740 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 0x557770ec6b40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4012df0 tx=0x7f6fe4004730 comp rx=0 tx=0).stop 2024-02-17T12:55:41.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f700597a740 1 -- 172.21.15.18:0/2531964146 shutdown_connections 2024-02-17T12:55:41.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f700597a740 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557770ed9a30 0x557770edbe50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f700597a740 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec97b0 0x557770ec6b40 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f700597a740 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec0bd0 0x557770ec6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f6fdbfff640 0 client.17574 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:41.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f700597a740 1 --2- 172.21.15.18:0/2531964146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec02b0 0x557770ec91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f700597a740 1 -- 172.21.15.18:0/2531964146 >> 172.21.15.18:0/2531964146 conn(0x557770eba990 msgr2=0x557770ebc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f700597a740 1 -- 172.21.15.18:0/2531964146 shutdown_connections 2024-02-17T12:55:41.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f700597a740 1 -- 172.21.15.18:0/2531964146 wait complete. 2024-02-17T12:55:41.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.436+0000 7f700597a740 20 client.17574 trim_cache size 0 max 16384 2024-02-17T12:55:41.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.451+0000 7f700597a740 1 Processor -- start 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.451+0000 7f700597a740 1 -- start start 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.451+0000 7f700597a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec02b0 0x557770eb98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.451+0000 7f700597a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec0bd0 0x557770ec8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.451+0000 7f700597a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec97b0 0x557770ec91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.451+0000 7f700597a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557770ebfd40 con 0x557770ec02b0 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.451+0000 7f700597a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557770ec1350 con 0x557770ec0bd0 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.451+0000 7f700597a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557770ebb770 con 0x557770ec97b0 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.451+0000 7f7000a65640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec02b0 0x557770eb98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.451+0000 7f7000a65640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec02b0 0x557770eb98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50356/0 (socket says 172.21.15.18:50356) 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.451+0000 7f7000a65640 1 -- 172.21.15.18:0/1602296918 learned_addr learned my addr 172.21.15.18:0/1602296918 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.452+0000 7f7001266640 1 --2- 172.21.15.18:0/1602296918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec97b0 0x557770ec91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.452+0000 7f7000a65640 1 -- 172.21.15.18:0/1602296918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec97b0 msgr2=0x557770ec91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.452+0000 7f7000a65640 1 --2- 172.21.15.18:0/1602296918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec97b0 0x557770ec91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.452+0000 7f6ffbfff640 1 --2- 172.21.15.18:0/1602296918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec0bd0 0x557770ec8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.452+0000 7f7000a65640 1 -- 172.21.15.18:0/1602296918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec0bd0 msgr2=0x557770ec8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.452+0000 7f7000a65640 1 --2- 172.21.15.18:0/1602296918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec0bd0 0x557770ec8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.452+0000 7f7000a65640 1 -- 172.21.15.18:0/1602296918 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557770dc2ac0 con 0x557770ec02b0 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.452+0000 7f7000a65640 1 --2- 172.21.15.18:0/1602296918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec02b0 0x557770eb98d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009ca0 tx=0x7f6ff408f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53407a8a0f5965e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.452+0000 7f6ff27fc640 1 -- 172.21.15.18:0/1602296918 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff40be000 con 0x557770ec02b0 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.452+0000 7f6ff27fc640 1 -- 172.21.15.18:0/1602296918 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ff40c1a50 con 0x557770ec02b0 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.452+0000 7f6ff27fc640 1 -- 172.21.15.18:0/1602296918 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff40bbd20 con 0x557770ec02b0 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 -- 172.21.15.18:0/1602296918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec02b0 msgr2=0x557770eb98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 --2- 172.21.15.18:0/1602296918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec02b0 0x557770eb98d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009ca0 tx=0x7f6ff408f8c0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 -- 172.21.15.18:0/1602296918 shutdown_connections 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 --2- 172.21.15.18:0/1602296918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec97b0 0x557770ec91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 --2- 172.21.15.18:0/1602296918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec0bd0 0x557770ec8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 --2- 172.21.15.18:0/1602296918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec02b0 0x557770eb98d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 -- 172.21.15.18:0/1602296918 >> 172.21.15.18:0/1602296918 conn(0x557770eba990 msgr2=0x557770eb91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 -- 172.21.15.18:0/1602296918 shutdown_connections 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 -- 172.21.15.18:0/1602296918 wait complete. 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 -- start start 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 Processor -- start 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec02b0 0x557770ecd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec0bd0 0x557770ecde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec97b0 0x557770f29f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557770f2a480 con 0x557770ec0bd0 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557770f2a5f0 con 0x557770ec97b0 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.453+0000 7f700597a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557770f2a7b0 con 0x557770ec02b0 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec0bd0 0x557770ecde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec0bd0 0x557770ecde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50368/0 (socket says 172.21.15.18:50368) 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6ffbfff640 1 -- 172.21.15.18:0/2479548071 learned_addr learned my addr 172.21.15.18:0/2479548071 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:41.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f700597a740 10 client.0 _ll_register_callbacks cb 0x7f6fec009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f7001266640 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec97b0 0x557770f29f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f7000a65640 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec02b0 0x557770ecd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6ffbfff640 1 -- 172.21.15.18:0/2479548071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec02b0 msgr2=0x557770ecd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6ffbfff640 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec02b0 0x557770ecd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6ffbfff640 1 -- 172.21.15.18:0/2479548071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec97b0 msgr2=0x557770f29f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6ffbfff640 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec97b0 0x557770f29f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6ffbfff640 1 -- 172.21.15.18:0/2479548071 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc1a47a0 con 0x557770ec0bd0 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6ffbfff640 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec0bd0 0x557770ecde00 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6fec006010 tx=0x7f6fec005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87677ebe19c8335b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6fdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6fdbfff640 1 -- 172.21.15.18:0/2479548071 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec022070 con 0x557770ec0bd0 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6fdbfff640 1 -- 172.21.15.18:0/2479548071 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6fec004620 con 0x557770ec0bd0 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f700597a740 1 -- 172.21.15.18:0/2479548071 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fec00ee80 con 0x557770ec0bd0 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f6fdbfff640 1 -- 172.21.15.18:0/2479548071 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fec005630 con 0x557770ec0bd0 2024-02-17T12:55:41.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f700597a740 10 client.17580 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.454+0000 7f700597a740 1 -- 172.21.15.18:0/2479548071 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x557770de86a0 con 0x557770ec0bd0 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.455+0000 7f6fdbfff640 1 -- 172.21.15.18:0/2479548071 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fec00ee80 con 0x557770ec0bd0 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.455+0000 7f6fdbfff640 1 -- 172.21.15.18:0/2479548071 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f6fec00c070 con 0x557770ec0bd0 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.455+0000 7f700597a740 10 client.17580 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.455+0000 7f700597a740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.455+0000 7f700597a740 20 client.17580 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.455+0000 7f700597a740 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557770f31ff0 0x557770f34410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.455+0000 7f700597a740 4 client.17580 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.455+0000 7f700597a740 1 -- 172.21.15.18:0/2479548071 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557770bb3890 con 0x557770f31ff0 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.455+0000 7f7000a65640 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557770f31ff0 0x557770f34410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.455+0000 7f7000a65640 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557770f31ff0 0x557770f34410 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe08cd11ef7f53ea server_cookie=eaf28de888bec1d7 in_seq=0 out_seq=0 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.455+0000 7f6fdbfff640 10 client.17580 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f6fdbfff640 1 -- 172.21.15.18:0/2479548071 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557770bb3890 con 0x557770f31ff0 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f6fdbfff640 10 client.17580 handle_command_reply: tid=0 2024-02-17T12:55:41.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f700597a740 1 client.17580 shutdown 2024-02-17T12:55:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f700597a740 1 -- 172.21.15.18:0/2479548071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec0bd0 msgr2=0x557770ecde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f700597a740 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec0bd0 0x557770ecde00 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6fec006010 tx=0x7f6fec005270 comp rx=0 tx=0).stop 2024-02-17T12:55:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f700597a740 1 -- 172.21.15.18:0/2479548071 shutdown_connections 2024-02-17T12:55:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f700597a740 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557770f31ff0 0x557770f34410 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f700597a740 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557770ec97b0 0x557770f29f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f700597a740 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557770ec0bd0 0x557770ecde00 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f700597a740 1 --2- 172.21.15.18:0/2479548071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557770ec02b0 0x557770ecd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f6fdbfff640 0 client.17580 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f700597a740 1 -- 172.21.15.18:0/2479548071 >> 172.21.15.18:0/2479548071 conn(0x557770eba990 msgr2=0x557770eb9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f700597a740 1 -- 172.21.15.18:0/2479548071 shutdown_connections 2024-02-17T12:55:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f700597a740 1 -- 172.21.15.18:0/2479548071 wait complete. 2024-02-17T12:55:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.456+0000 7f700597a740 20 client.17580 trim_cache size 0 max 16384 2024-02-17T12:55:41.455 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:55:41.455 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:55:41.455 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:55:41.455 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:55:41.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.457+0000 7f7002cc9640 1 -- 172.21.15.18:0/2813018911 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6fe80a2230 msgr2=0x7f6fe80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.457+0000 7f7002cc9640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6fe80a2230 0x7f6fe80a46f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4012870 tx=0x7f6fe400b2e0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.457+0000 7f7002cc9640 1 -- 172.21.15.18:0/2813018911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ffc101380 msgr2=0x7f6ffc19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.457+0000 7f7002cc9640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ffc101380 0x7f6ffc19d460 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4004af0 tx=0x7f6ff4034cc0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.457+0000 7f7002cc9640 1 -- 172.21.15.18:0/2813018911 shutdown_connections 2024-02-17T12:55:41.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.457+0000 7f7002cc9640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6fe80a2230 0x7f6fe80a46f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.457+0000 7f7002cc9640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc197610 0x7f6ffc197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.457+0000 7f7002cc9640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ffc106450 0x7f6ffc19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.457+0000 7f7002cc9640 1 --2- 172.21.15.18:0/2813018911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ffc101380 0x7f6ffc19d460 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.457+0000 7f7002cc9640 1 -- 172.21.15.18:0/2813018911 >> 172.21.15.18:0/2813018911 conn(0x7f6ffc0fcc20 msgr2=0x7f6ffc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.457+0000 7f7002cc9640 1 -- 172.21.15.18:0/2813018911 shutdown_connections 2024-02-17T12:55:41.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.457+0000 7f7002cc9640 1 -- 172.21.15.18:0/2813018911 wait complete. 2024-02-17T12:55:41.459 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:55:41.459 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:55:41.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.570+0000 7f0408b5e640 1 -- 172.21.15.18:0/3409619290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404105060 msgr2=0x7f0404105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.570+0000 7f0408b5e640 1 --2- 172.21.15.18:0/3409619290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404105060 0x7f0404105440 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009900 tx=0x7f03f402bc30 comp rx=0 tx=0).stop 2024-02-17T12:55:41.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.571+0000 7f0408b5e640 1 -- 172.21.15.18:0/3409619290 shutdown_connections 2024-02-17T12:55:41.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.571+0000 7f0408b5e640 1 --2- 172.21.15.18:0/3409619290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04041011d0 0x7f040410cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.571+0000 7f0408b5e640 1 --2- 172.21.15.18:0/3409619290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0404100810 0x7f0404100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.571+0000 7f0408b5e640 1 --2- 172.21.15.18:0/3409619290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404105060 0x7f0404105440 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.571+0000 7f0408b5e640 1 -- 172.21.15.18:0/3409619290 >> 172.21.15.18:0/3409619290 conn(0x7f0404076610 msgr2=0x7f04040fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.571+0000 7f0408b5e640 1 -- 172.21.15.18:0/3409619290 shutdown_connections 2024-02-17T12:55:41.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.571+0000 7f0408b5e640 1 -- 172.21.15.18:0/3409619290 wait complete. 2024-02-17T12:55:41.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.571+0000 7f0408b5e640 1 Processor -- start 2024-02-17T12:55:41.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.571+0000 7f0408b5e640 1 -- start start 2024-02-17T12:55:41.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.571+0000 7f0408b5e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0404100810 0x7f04040fba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0408b5e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04041011d0 0x7f04040fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0408b5e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404105060 0x7f04040fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0408b5e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04041a5b40 con 0x7f0404105060 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0408b5e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04041a5cb0 con 0x7f0404100810 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0408b5e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04041a5e70 con 0x7f04041011d0 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0402d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404105060 0x7f04040fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0402d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404105060 0x7f04040fc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50374/0 (socket says 172.21.15.18:50374) 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0402d9d640 1 -- 172.21.15.18:0/2443853272 learned_addr learned my addr 172.21.15.18:0/2443853272 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f040259c640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0404100810 0x7f04040fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f03fbfff640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04041011d0 0x7f04040fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0402d9d640 1 -- 172.21.15.18:0/2443853272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04041011d0 msgr2=0x7f04040fbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0402d9d640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04041011d0 0x7f04040fbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0402d9d640 1 -- 172.21.15.18:0/2443853272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0404100810 msgr2=0x7f04040fba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0402d9d640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0404100810 0x7f04040fba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0402d9d640 1 -- 172.21.15.18:0/2443853272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f4009560 con 0x7f0404105060 2024-02-17T12:55:41.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f03fbfff640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04041011d0 0x7f04040fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:41.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0402d9d640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404105060 0x7f04040fc4d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00eb10 tx=0x7f03fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f03fb7fe640 1 -- 172.21.15.18:0/2443853272 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc004b20 con 0x7f0404105060 2024-02-17T12:55:41.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0408b5e640 1 -- 172.21.15.18:0/2443853272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04041a6150 con 0x7f0404105060 2024-02-17T12:55:41.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f03fb7fe640 1 -- 172.21.15.18:0/2443853272 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03fc004c80 con 0x7f0404105060 2024-02-17T12:55:41.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f03fb7fe640 1 -- 172.21.15.18:0/2443853272 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc010650 con 0x7f0404105060 2024-02-17T12:55:41.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.572+0000 7f0408b5e640 1 -- 172.21.15.18:0/2443853272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04041a6660 con 0x7f0404105060 2024-02-17T12:55:41.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.573+0000 7f0408b5e640 1 -- 172.21.15.18:0/2443853272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04040683c0 con 0x7f0404105060 2024-02-17T12:55:41.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.574+0000 7f03fb7fe640 1 -- 172.21.15.18:0/2443853272 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f03fc005020 con 0x7f0404105060 2024-02-17T12:55:41.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.575+0000 7f03fb7fe640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03f00a2230 0x7f03f00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.575+0000 7f03fb7fe640 1 -- 172.21.15.18:0/2443853272 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f03fc014070 con 0x7f0404105060 2024-02-17T12:55:41.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.575+0000 7f040259c640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03f00a2230 0x7f03f00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.575+0000 7f040259c640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03f00a2230 0x7f03f00a46f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f03f402c7b0 tx=0x7f03f4000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.580+0000 7f03fb7fe640 1 -- 172.21.15.18:0/2443853272 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f03fc0973b0 con 0x7f0404105060 2024-02-17T12:55:41.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.808+0000 7f0408b5e640 1 -- 172.21.15.18:0/2443853272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04041095c0 con 0x7f0404105060 2024-02-17T12:55:41.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.809+0000 7f03fb7fe640 1 -- 172.21.15.18:0/2443853272 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+11403 (secure 0 0 0) 0x7f03fc09aa70 con 0x7f0404105060 2024-02-17T12:55:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:55:41.801 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:55:18.471834+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:55:41.801 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 38 2024-02-17T12:55:41.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.811+0000 7f0408b5e640 1 -- 172.21.15.18:0/2443853272 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03f00a2230 msgr2=0x7f03f00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.811+0000 7f0408b5e640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03f00a2230 0x7f03f00a46f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f03f402c7b0 tx=0x7f03f4000bc0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.811+0000 7f0408b5e640 1 -- 172.21.15.18:0/2443853272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404105060 msgr2=0x7f04040fc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.811+0000 7f0408b5e640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404105060 0x7f04040fc4d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00eb10 tx=0x7f03fc00c2d0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.811+0000 7f0408b5e640 1 -- 172.21.15.18:0/2443853272 shutdown_connections 2024-02-17T12:55:41.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.811+0000 7f0408b5e640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03f00a2230 0x7f03f00a46f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.811+0000 7f0408b5e640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404105060 0x7f04040fc4d0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.811+0000 7f0408b5e640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04041011d0 0x7f04040fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.811+0000 7f0408b5e640 1 --2- 172.21.15.18:0/2443853272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0404100810 0x7f04040fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.811+0000 7f0408b5e640 1 -- 172.21.15.18:0/2443853272 >> 172.21.15.18:0/2443853272 conn(0x7f0404076610 msgr2=0x7f04040ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.811+0000 7f0408b5e640 1 -- 172.21.15.18:0/2443853272 shutdown_connections 2024-02-17T12:55:41.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.811+0000 7f0408b5e640 1 -- 172.21.15.18:0/2443853272 wait complete. 2024-02-17T12:55:41.814 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:55:41.814 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:55:41.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.927+0000 7fd39d04d640 1 -- 172.21.15.18:0/3077447963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd398105030 msgr2=0x7fd398105410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.927+0000 7fd39d04d640 1 --2- 172.21.15.18:0/3077447963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd398105030 0x7fd398105410 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd388009900 tx=0x7fd38802bc80 comp rx=0 tx=0).stop 2024-02-17T12:55:41.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.927+0000 7fd39d04d640 1 -- 172.21.15.18:0/3077447963 shutdown_connections 2024-02-17T12:55:41.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.927+0000 7fd39d04d640 1 --2- 172.21.15.18:0/3077447963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd398101170 0x7fd39810cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.927+0000 7fd39d04d640 1 --2- 172.21.15.18:0/3077447963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3981007d0 0x7fd398100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.927+0000 7fd39d04d640 1 --2- 172.21.15.18:0/3077447963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd398105030 0x7fd398105410 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.927+0000 7fd39d04d640 1 -- 172.21.15.18:0/3077447963 >> 172.21.15.18:0/3077447963 conn(0x7fd398076640 msgr2=0x7fd3980fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.927+0000 7fd39d04d640 1 -- 172.21.15.18:0/3077447963 shutdown_connections 2024-02-17T12:55:41.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.927+0000 7fd39d04d640 1 -- 172.21.15.18:0/3077447963 wait complete. 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.927+0000 7fd39d04d640 1 Processor -- start 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39d04d640 1 -- start start 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39d04d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3981007d0 0x7fd398198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39d04d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd398101170 0x7fd398198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39d04d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd398105030 0x7fd3981a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39d04d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd398199450 con 0x7fd398105030 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39d04d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3981a3c50 con 0x7fd3981007d0 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39d04d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3981a3d90 con 0x7fd398101170 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39759e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd398105030 0x7fd3981a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39759e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd398105030 0x7fd3981a3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50400/0 (socket says 172.21.15.18:50400) 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39759e640 1 -- 172.21.15.18:0/187758285 learned_addr learned my addr 172.21.15.18:0/187758285 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd396d9d640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3981007d0 0x7fd398198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39659c640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd398101170 0x7fd398198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39759e640 1 -- 172.21.15.18:0/187758285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd398101170 msgr2=0x7fd398198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39759e640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd398101170 0x7fd398198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39759e640 1 -- 172.21.15.18:0/187758285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3981007d0 msgr2=0x7fd398198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39759e640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3981007d0 0x7fd398198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39759e640 1 -- 172.21.15.18:0/187758285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd388009560 con 0x7fd398105030 2024-02-17T12:55:41.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.928+0000 7fd39659c640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd398101170 0x7fd398198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:41.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.929+0000 7fd39759e640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd398105030 0x7fd3981a3710 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd39000eb40 tx=0x7fd39000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.929+0000 7fd37ffff640 1 -- 172.21.15.18:0/187758285 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39000ceb0 con 0x7fd398105030 2024-02-17T12:55:41.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.929+0000 7fd37ffff640 1 -- 172.21.15.18:0/187758285 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd390005020 con 0x7fd398105030 2024-02-17T12:55:41.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.929+0000 7fd39d04d640 1 -- 172.21.15.18:0/187758285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3981a3ff0 con 0x7fd398105030 2024-02-17T12:55:41.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.929+0000 7fd37ffff640 1 -- 172.21.15.18:0/187758285 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390010690 con 0x7fd398105030 2024-02-17T12:55:41.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.929+0000 7fd39d04d640 1 -- 172.21.15.18:0/187758285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3981a4480 con 0x7fd398105030 2024-02-17T12:55:41.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.931+0000 7fd37ffff640 1 -- 172.21.15.18:0/187758285 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd390005190 con 0x7fd398105030 2024-02-17T12:55:41.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.931+0000 7fd39fc1b740 1 Processor -- start 2024-02-17T12:55:41.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.931+0000 7fd39fc1b740 1 -- start start 2024-02-17T12:55:41.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd39fc1b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0ef80 0x561801a0f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd39fc1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a1c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd39fc1b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 0x561801a1ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd39fc1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618019ca960 con 0x561801a0f8a0 2024-02-17T12:55:41.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd39fc1b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180197e310 con 0x561801a1cb60 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd39fc1b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180197b840 con 0x561801a0ef80 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd396d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a1c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd396d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a1c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50408/0 (socket says 172.21.15.18:50408) 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd396d9d640 1 -- 172.21.15.18:0/2299293603 learned_addr learned my addr 172.21.15.18:0/2299293603 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd37ffff640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3800a2230 0x7fd3800a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd39659c640 1 --2- 172.21.15.18:0/2299293603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 0x561801a1ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd39759e640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3800a2230 0x7fd3800a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd39659c640 1 --2- 172.21.15.18:0/2299293603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0ef80 0x561801a0f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd396d9d640 1 -- 172.21.15.18:0/2299293603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0ef80 msgr2=0x561801a0f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd396d9d640 1 --2- 172.21.15.18:0/2299293603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0ef80 0x561801a0f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd396d9d640 1 -- 172.21.15.18:0/2299293603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 msgr2=0x561801a1ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd396d9d640 1 --2- 172.21.15.18:0/2299293603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 0x561801a1ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd396d9d640 1 -- 172.21.15.18:0/2299293603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561801939060 con 0x561801a0f8a0 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd39659c640 1 --2- 172.21.15.18:0/2299293603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 0x561801a1ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:41.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd37ffff640 1 -- 172.21.15.18:0/187758285 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd390014070 con 0x7fd398105030 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd396d9d640 1 --2- 172.21.15.18:0/2299293603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a1c620 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fd388009900 tx=0x7fd38802c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1770aca16f12f34 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd37d7fa640 1 -- 172.21.15.18:0/2299293603 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38802c900 con 0x561801a0f8a0 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd37d7fa640 1 -- 172.21.15.18:0/2299293603 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd38802ca60 con 0x561801a0f8a0 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd37d7fa640 1 -- 172.21.15.18:0/2299293603 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388005170 con 0x561801a0f8a0 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.932+0000 7fd39759e640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3800a2230 0x7fd3800a46f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd39009ade0 tx=0x7fd39006e150 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.933+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2299293603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 msgr2=0x561801a1c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.933+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2299293603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a1c620 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fd388009900 tx=0x7fd38802c690 comp rx=0 tx=0).stop 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.933+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2299293603 shutdown_connections 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.933+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2299293603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 0x561801a1ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.933+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2299293603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a1c620 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.933+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2299293603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0ef80 0x561801a0f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.933+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2299293603 >> 172.21.15.18:0/2299293603 conn(0x561801a09980 msgr2=0x561801a09d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.933+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2299293603 shutdown_connections 2024-02-17T12:55:41.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.933+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2299293603 wait complete. 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.933+0000 7fd39fc1b740 1 -- start start 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39fc1b740 1 Processor -- start 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39fc1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0ef80 0x561801a19b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39fc1b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0f8a0 0x561801a1a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39fc1b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 0x561801a1a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39fc1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561801977020 con 0x561801a0ef80 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39fc1b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180164a430 con 0x561801a1cb60 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39fc1b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561801606e00 con 0x561801a0f8a0 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39fc1b740 10 client.0 _ll_register_callbacks cb 0x56180191ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0ef80 0x561801a19b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0ef80 0x561801a19b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50410/0 (socket says 172.21.15.18:50410) 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39659c640 1 -- 172.21.15.18:0/2442374628 learned_addr learned my addr 172.21.15.18:0/2442374628 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39659c640 1 -- 172.21.15.18:0/2442374628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0f8a0 msgr2=0x561801a1a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39659c640 1 --2- 172.21.15.18:0/2442374628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0f8a0 0x561801a1a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39659c640 1 -- 172.21.15.18:0/2442374628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 msgr2=0x561801a1a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39659c640 1 --2- 172.21.15.18:0/2442374628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 0x561801a1a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39659c640 1 -- 172.21.15.18:0/2442374628 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39810d850 con 0x561801a0ef80 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39659c640 1 --2- 172.21.15.18:0/2442374628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0ef80 0x561801a19b40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0128b0 tx=0x7fd38c012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98055f81316f476a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd37aff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd37aff5640 1 -- 172.21.15.18:0/2442374628 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c011e10 con 0x561801a0ef80 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd37aff5640 1 -- 172.21.15.18:0/2442374628 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd38c005020 con 0x561801a0ef80 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.934+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2442374628 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618016767e0 con 0x561801a0ef80 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.935+0000 7fd37aff5640 1 -- 172.21.15.18:0/2442374628 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd38c0051e0 con 0x561801a0ef80 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.935+0000 7fd39fc1b740 10 client.17598 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.935+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2442374628 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5618019756a0 con 0x561801a0ef80 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.935+0000 7fd37aff5640 1 -- 172.21.15.18:0/2442374628 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5618016767e0 con 0x561801a0ef80 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.935+0000 7fd37aff5640 1 -- 172.21.15.18:0/2442374628 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fd38c026480 con 0x561801a0ef80 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.935+0000 7fd39fc1b740 10 client.17598 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:41.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.935+0000 7fd39fc1b740 10 client.17598 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.935+0000 7fd39fc1b740 20 client.17598 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.935+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2442374628 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x561801a2cef0 0x561801a2f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.935+0000 7fd39fc1b740 4 client.17598 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.935+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2442374628 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618016b3720 con 0x561801a2cef0 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.935+0000 7fd396d9d640 1 --2- 172.21.15.18:0/2442374628 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x561801a2cef0 0x561801a2f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.936+0000 7fd37aff5640 10 client.17598 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.936+0000 7fd396d9d640 1 --2- 172.21.15.18:0/2442374628 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x561801a2cef0 0x561801a2f310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f6a0b36106424c27 server_cookie=3720ec75c88577f5 in_seq=0 out_seq=0 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.936+0000 7fd37aff5640 1 -- 172.21.15.18:0/2442374628 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5618016b3720 con 0x561801a2cef0 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.936+0000 7fd37aff5640 10 client.17598 handle_command_reply: tid=0 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.936+0000 7fd39fc1b740 1 client.17598 shutdown 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.937+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2442374628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0ef80 msgr2=0x561801a19b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.937+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2442374628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0ef80 0x561801a19b40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0128b0 tx=0x7fd38c012d70 comp rx=0 tx=0).stop 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.937+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2442374628 shutdown_connections 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.937+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2442374628 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x561801a2cef0 0x561801a2f310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.937+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2442374628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 0x561801a1a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.937+0000 7fd37aff5640 0 client.17598 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:55:41.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.937+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2442374628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0f8a0 0x561801a1a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.937+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2442374628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0ef80 0x561801a19b40 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.937+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2442374628 >> 172.21.15.18:0/2442374628 conn(0x561801a09980 msgr2=0x561801a0b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.937+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2442374628 shutdown_connections 2024-02-17T12:55:41.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.937+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2442374628 wait complete. 2024-02-17T12:55:41.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.937+0000 7fd39fc1b740 20 client.17598 trim_cache size 0 max 16384 2024-02-17T12:55:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.948+0000 7fd39fc1b740 1 Processor -- start 2024-02-17T12:55:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.948+0000 7fd39fc1b740 1 -- start start 2024-02-17T12:55:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd39fc1b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a0ef80 0x561801a1c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd39fc1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a08a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd39fc1b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a1cb60 0x561801a239f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd39fc1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618018fc130 con 0x561801a0f8a0 2024-02-17T12:55:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd39fc1b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618016767e0 con 0x561801a0ef80 2024-02-17T12:55:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd39fc1b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561801a0fda0 con 0x561801a1cb60 2024-02-17T12:55:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd396d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a08a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd396d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a08a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50412/0 (socket says 172.21.15.18:50412) 2024-02-17T12:55:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd396d9d640 1 -- 172.21.15.18:0/809246124 learned_addr learned my addr 172.21.15.18:0/809246124 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd39659c640 1 --2- 172.21.15.18:0/809246124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a0ef80 0x561801a1c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd39659c640 1 --2- 172.21.15.18:0/809246124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a1cb60 0x561801a239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd396d9d640 1 -- 172.21.15.18:0/809246124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a1cb60 msgr2=0x561801a239f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd396d9d640 1 --2- 172.21.15.18:0/809246124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a1cb60 0x561801a239f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd396d9d640 1 -- 172.21.15.18:0/809246124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a0ef80 msgr2=0x561801a1c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd396d9d640 1 --2- 172.21.15.18:0/809246124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a0ef80 0x561801a1c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd396d9d640 1 -- 172.21.15.18:0/809246124 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561801911c20 con 0x561801a0f8a0 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd39659c640 1 --2- 172.21.15.18:0/809246124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a1cb60 0x561801a239f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd396d9d640 1 --2- 172.21.15.18:0/809246124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a08a00 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd388004650 tx=0x7fd388004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b8cd5f4286b942 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd37cff9640 1 -- 172.21.15.18:0/809246124 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38802c900 con 0x561801a0f8a0 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd37cff9640 1 -- 172.21.15.18:0/809246124 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd38804d070 con 0x561801a0f8a0 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.949+0000 7fd37cff9640 1 -- 172.21.15.18:0/809246124 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38803f6a0 con 0x561801a0f8a0 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 -- 172.21.15.18:0/809246124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 msgr2=0x561801a08a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/809246124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a08a00 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd388004650 tx=0x7fd388004df0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 -- 172.21.15.18:0/809246124 shutdown_connections 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/809246124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a1cb60 0x561801a239f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/809246124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a08a00 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/809246124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a0ef80 0x561801a1c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 -- 172.21.15.18:0/809246124 >> 172.21.15.18:0/809246124 conn(0x561801a09980 msgr2=0x561801a082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 -- 172.21.15.18:0/809246124 shutdown_connections 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 -- 172.21.15.18:0/809246124 wait complete. 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 -- start start 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 Processor -- start 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0ef80 0x561801a23bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a21f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 0x561801a224c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561801a22a30 con 0x561801a0f8a0 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561801a22ba0 con 0x561801a1cb60 2024-02-17T12:55:41.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561801a22d60 con 0x561801a0ef80 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd39fc1b740 10 client.0 _ll_register_callbacks cb 0x7fd38c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd396d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a21f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd396d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a21f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50428/0 (socket says 172.21.15.18:50428) 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.950+0000 7fd396d9d640 1 -- 172.21.15.18:0/2084915767 learned_addr learned my addr 172.21.15.18:0/2084915767 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd39659c640 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0ef80 0x561801a23bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd396d9d640 1 -- 172.21.15.18:0/2084915767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0ef80 msgr2=0x561801a23bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd396d9d640 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0ef80 0x561801a23bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd396d9d640 1 -- 172.21.15.18:0/2084915767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 msgr2=0x561801a224c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd396d9d640 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 0x561801a224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd396d9d640 1 -- 172.21.15.18:0/2084915767 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561801939060 con 0x561801a0f8a0 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd39659c640 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0ef80 0x561801a23bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd396d9d640 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a21f80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fd38802c1c0 tx=0x7fd38802bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2027d3762cb92664 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd37aff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:41.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd37aff5640 1 -- 172.21.15.18:0/2084915767 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3880048c0 con 0x561801a0f8a0 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd37aff5640 1 -- 172.21.15.18:0/2084915767 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd38804f070 con 0x561801a0f8a0 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2084915767 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd38802ca60 con 0x561801a0f8a0 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd37aff5640 1 -- 172.21.15.18:0/2084915767 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd388004a20 con 0x561801a0f8a0 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd39fc1b740 10 client.17604 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2084915767 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5618019378c0 con 0x561801a0f8a0 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd37aff5640 1 -- 172.21.15.18:0/2084915767 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd38802ca60 con 0x561801a0f8a0 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd37aff5640 1 -- 172.21.15.18:0/2084915767 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fd38803c050 con 0x561801a0f8a0 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd39fc1b740 10 client.17604 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd39fc1b740 10 client.17604 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd39fc1b740 20 client.17604 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.951+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x561801a2cef0 0x561801a86560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 4 client.17604 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2084915767 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561801702890 con 0x561801a2cef0 2024-02-17T12:55:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39659c640 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x561801a2cef0 0x561801a86560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39659c640 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x561801a2cef0 0x561801a86560 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=53ac8056758ab8ee server_cookie=43e70caedc484230 in_seq=0 out_seq=0 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd37aff5640 10 client.17604 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd37aff5640 1 -- 172.21.15.18:0/2084915767 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561801702890 con 0x561801a2cef0 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd37aff5640 10 client.17604 handle_command_reply: tid=0 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 1 client.17604 shutdown 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2084915767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 msgr2=0x561801a21f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a21f80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fd38802c1c0 tx=0x7fd38802bc80 comp rx=0 tx=0).stop 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2084915767 shutdown_connections 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x561801a2cef0 0x561801a86560 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561801a1cb60 0x561801a224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561801a0f8a0 0x561801a21f80 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 1 --2- 172.21.15.18:0/2084915767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561801a0ef80 0x561801a23bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd37aff5640 0 client.17604 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2084915767 >> 172.21.15.18:0/2084915767 conn(0x561801a09980 msgr2=0x561801a082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2084915767 shutdown_connections 2024-02-17T12:55:41.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.952+0000 7fd39fc1b740 1 -- 172.21.15.18:0/2084915767 wait complete. 2024-02-17T12:55:41.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39fc1b740 20 client.17604 trim_cache size 0 max 16384 2024-02-17T12:55:41.952 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:55:41.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39d04d640 1 -- 172.21.15.18:0/187758285 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3800a2230 msgr2=0x7fd3800a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39d04d640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3800a2230 0x7fd3800a46f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd39009ade0 tx=0x7fd39006e150 comp rx=0 tx=0).stop 2024-02-17T12:55:41.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39d04d640 1 -- 172.21.15.18:0/187758285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd398105030 msgr2=0x7fd3981a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:41.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39d04d640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd398105030 0x7fd3981a3710 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd39000eb40 tx=0x7fd39000c2d0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39d04d640 1 -- 172.21.15.18:0/187758285 shutdown_connections 2024-02-17T12:55:41.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39d04d640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3800a2230 0x7fd3800a46f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39d04d640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd398105030 0x7fd3981a3710 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39d04d640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd398101170 0x7fd398198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39d04d640 1 --2- 172.21.15.18:0/187758285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3981007d0 0x7fd398198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:41.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39d04d640 1 -- 172.21.15.18:0/187758285 >> 172.21.15.18:0/187758285 conn(0x7fd398076640 msgr2=0x7fd398103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:41.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39d04d640 1 -- 172.21.15.18:0/187758285 shutdown_connections 2024-02-17T12:55:41.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:41.953+0000 7fd39d04d640 1 -- 172.21.15.18:0/187758285 wait complete. 2024-02-17T12:55:41.955 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:55:41.955 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:55:42.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.068+0000 7f779e00e640 1 -- 172.21.15.18:0/2264731653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77980ffcf0 msgr2=0x7f77981000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.068+0000 7f779e00e640 1 --2- 172.21.15.18:0/2264731653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77980ffcf0 0x7f77981000d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7788009cd0 tx=0x7f778802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.068+0000 7f779e00e640 1 -- 172.21.15.18:0/2264731653 shutdown_connections 2024-02-17T12:55:42.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.068+0000 7f779e00e640 1 --2- 172.21.15.18:0/2264731653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77981064b0 0x7f779810e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.068+0000 7f779e00e640 1 --2- 172.21.15.18:0/2264731653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77981006a0 0x7f7798105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.068+0000 7f779e00e640 1 --2- 172.21.15.18:0/2264731653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77980ffcf0 0x7f77981000d0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.068+0000 7f779e00e640 1 -- 172.21.15.18:0/2264731653 >> 172.21.15.18:0/2264731653 conn(0x7f77980752e0 msgr2=0x7f77980756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.068+0000 7f779e00e640 1 -- 172.21.15.18:0/2264731653 shutdown_connections 2024-02-17T12:55:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.068+0000 7f779e00e640 1 -- 172.21.15.18:0/2264731653 wait complete. 2024-02-17T12:55:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.068+0000 7f779e00e640 1 Processor -- start 2024-02-17T12:55:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.068+0000 7f779e00e640 1 -- start start 2024-02-17T12:55:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f779e00e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77980ffcf0 0x7f779819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f779e00e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77981064b0 0x7f779819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f779e00e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77981975f0 0x7f7798197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f779e00e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798197fe0 con 0x7f77980ffcf0 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f779e00e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798198150 con 0x7f77981975f0 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f779e00e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798198310 con 0x7f77981064b0 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f77977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77980ffcf0 0x7f779819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f77977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77980ffcf0 0x7f779819d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50456/0 (socket says 172.21.15.18:50456) 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f77977fe640 1 -- 172.21.15.18:0/1657601144 learned_addr learned my addr 172.21.15.18:0/1657601144 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f7797fff640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77981975f0 0x7f7798197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f7796ffd640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77981064b0 0x7f779819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f77977fe640 1 -- 172.21.15.18:0/1657601144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77981064b0 msgr2=0x7f779819d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f77977fe640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77981064b0 0x7f779819d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f77977fe640 1 -- 172.21.15.18:0/1657601144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77981975f0 msgr2=0x7f7798197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f77977fe640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77981975f0 0x7f7798197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f77977fe640 1 -- 172.21.15.18:0/1657601144 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7788009900 con 0x7f77980ffcf0 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f7797fff640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77981975f0 0x7f7798197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f7796ffd640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77981064b0 0x7f779819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:42.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f77977fe640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77980ffcf0 0x7f779819d440 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7788004af0 tx=0x7f7788034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f7794ff9640 1 -- 172.21.15.18:0/1657601144 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788032ac0 con 0x7f77980ffcf0 2024-02-17T12:55:42.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f7794ff9640 1 -- 172.21.15.18:0/1657601144 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7788003e50 con 0x7f77980ffcf0 2024-02-17T12:55:42.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f779e00e640 1 -- 172.21.15.18:0/1657601144 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77981a4300 con 0x7f77980ffcf0 2024-02-17T12:55:42.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f7794ff9640 1 -- 172.21.15.18:0/1657601144 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788040450 con 0x7f77980ffcf0 2024-02-17T12:55:42.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.069+0000 7f779e00e640 1 -- 172.21.15.18:0/1657601144 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77981a4780 con 0x7f77980ffcf0 2024-02-17T12:55:42.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.072+0000 7f7794ff9640 1 -- 172.21.15.18:0/1657601144 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f778800a040 con 0x7f77980ffcf0 2024-02-17T12:55:42.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.072+0000 7f77a0bdd740 1 Processor -- start 2024-02-17T12:55:42.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.072+0000 7f77a0bdd740 1 -- start start 2024-02-17T12:55:42.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.072+0000 7f77a0bdd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee78b250 0x55c7ee78b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.072+0000 7f77a0bdd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78bb70 0x55c7ee794210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.072+0000 7f77a0bdd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 0x55c7ee796b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.072+0000 7f77a0bdd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ee746960 con 0x55c7ee794750 2024-02-17T12:55:42.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.072+0000 7f77a0bdd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ee6fa310 con 0x55c7ee78bb70 2024-02-17T12:55:42.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.072+0000 7f77a0bdd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ee6f7840 con 0x55c7ee78b250 2024-02-17T12:55:42.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7794ff9640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f77800a2230 0x7f77800a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7796ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 0x55c7ee796b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7796ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 0x55c7ee796b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50464/0 (socket says 172.21.15.18:50464) 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7796ffd640 1 -- 172.21.15.18:0/3505174659 learned_addr learned my addr 172.21.15.18:0/3505174659 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7797fff640 1 --2- 172.21.15.18:0/3505174659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee78b250 0x55c7ee78b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7797fff640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f77800a2230 0x7f77800a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f77977fe640 1 --2- 172.21.15.18:0/3505174659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78bb70 0x55c7ee794210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7794ff9640 1 -- 172.21.15.18:0/1657601144 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f77880f0ee0 con 0x7f77980ffcf0 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7796ffd640 1 -- 172.21.15.18:0/3505174659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee78b250 msgr2=0x55c7ee78b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7796ffd640 1 --2- 172.21.15.18:0/3505174659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee78b250 0x55c7ee78b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7796ffd640 1 -- 172.21.15.18:0/3505174659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78bb70 msgr2=0x55c7ee794210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7796ffd640 1 --2- 172.21.15.18:0/3505174659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78bb70 0x55c7ee794210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7796ffd640 1 -- 172.21.15.18:0/3505174659 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7ee6b5060 con 0x55c7ee794750 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7796ffd640 1 --2- 172.21.15.18:0/3505174659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 0x55c7ee796b40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f778c00bb80 tx=0x7f778c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396c1c116b9dcfc2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f7797fff640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f77800a2230 0x7f77800a46f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7790005fd0 tx=0x7f7790005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f777dffb640 1 -- 172.21.15.18:0/3505174659 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c017620 con 0x55c7ee794750 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f777dffb640 1 -- 172.21.15.18:0/3505174659 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f778c00be80 con 0x55c7ee794750 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.073+0000 7f777dffb640 1 -- 172.21.15.18:0/3505174659 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c00fbd0 con 0x55c7ee794750 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.074+0000 7f77a0bdd740 1 -- 172.21.15.18:0/3505174659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 msgr2=0x55c7ee796b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.074+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/3505174659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 0x55c7ee796b40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f778c00bb80 tx=0x7f778c00d710 comp rx=0 tx=0).stop 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.074+0000 7f77a0bdd740 1 -- 172.21.15.18:0/3505174659 shutdown_connections 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.074+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/3505174659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 0x55c7ee796b40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.074+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/3505174659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78bb70 0x55c7ee794210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.074+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/3505174659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee78b250 0x55c7ee78b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.074+0000 7f77a0bdd740 1 -- 172.21.15.18:0/3505174659 >> 172.21.15.18:0/3505174659 conn(0x55c7ee785980 msgr2=0x55c7ee785d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.074+0000 7f77a0bdd740 1 -- 172.21.15.18:0/3505174659 shutdown_connections 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.074+0000 7f77a0bdd740 1 -- 172.21.15.18:0/3505174659 wait complete. 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.074+0000 7f77a0bdd740 1 -- start start 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.074+0000 7f77a0bdd740 1 Processor -- start 2024-02-17T12:55:42.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77a0bdd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78b250 0x55c7ee791700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77a0bdd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78bb70 0x55c7ee791c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77a0bdd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee794750 0x55c7ee792180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77a0bdd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ee6f3020 con 0x55c7ee78b250 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77a0bdd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ee3c6430 con 0x55c7ee78bb70 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77a0bdd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ee382e00 con 0x55c7ee794750 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77a0bdd740 10 client.0 _ll_register_callbacks cb 0x55c7ee69acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78b250 0x55c7ee791700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78b250 0x55c7ee791700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50478/0 (socket says 172.21.15.18:50478) 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77977fe640 1 -- 172.21.15.18:0/637254520 learned_addr learned my addr 172.21.15.18:0/637254520 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f7796ffd640 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78bb70 0x55c7ee791c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f7797fff640 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee794750 0x55c7ee792180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77977fe640 1 -- 172.21.15.18:0/637254520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee794750 msgr2=0x55c7ee792180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77977fe640 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee794750 0x55c7ee792180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77977fe640 1 -- 172.21.15.18:0/637254520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78bb70 msgr2=0x55c7ee791c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77977fe640 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78bb70 0x55c7ee791c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77977fe640 1 -- 172.21.15.18:0/637254520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778c00b7b0 con 0x55c7ee78b250 2024-02-17T12:55:42.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f7796ffd640 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78bb70 0x55c7ee791c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:42.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77977fe640 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78b250 0x55c7ee791700 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f77880ba3a0 tx=0x7f778803cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a09da65919c74cb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f7776ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:42.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f7776ffd640 1 -- 172.21.15.18:0/637254520 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788004360 con 0x55c7ee78b250 2024-02-17T12:55:42.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f7776ffd640 1 -- 172.21.15.18:0/637254520 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f77880c1b20 con 0x55c7ee78b250 2024-02-17T12:55:42.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.075+0000 7f77a0bdd740 1 -- 172.21.15.18:0/637254520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7ee42f720 con 0x55c7ee78b250 2024-02-17T12:55:42.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.076+0000 7f7776ffd640 1 -- 172.21.15.18:0/637254520 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77880c6170 con 0x55c7ee78b250 2024-02-17T12:55:42.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.076+0000 7f77a0bdd740 10 client.17616 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:42.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.076+0000 7f77a0bdd740 1 -- 172.21.15.18:0/637254520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c7ee6f16a0 con 0x55c7ee78b250 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.076+0000 7f7776ffd640 1 -- 172.21.15.18:0/637254520 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77880b7070 con 0x55c7ee78b250 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.076+0000 7f7776ffd640 1 -- 172.21.15.18:0/637254520 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f7788050300 con 0x55c7ee78b250 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.076+0000 7f77a0bdd740 10 client.17616 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.076+0000 7f77a0bdd740 10 client.17616 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.076+0000 7f77a0bdd740 20 client.17616 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.076+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c7ee7a4920 0x55c7ee7a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.076+0000 7f77a0bdd740 4 client.17616 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.076+0000 7f77a0bdd740 1 -- 172.21.15.18:0/637254520 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7ee47e890 con 0x55c7ee7a4920 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.076+0000 7f7796ffd640 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c7ee7a4920 0x55c7ee7a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.077+0000 7f7776ffd640 10 client.17616 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.077+0000 7f7796ffd640 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c7ee7a4920 0x55c7ee7a6d30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9919b77fea57f396 server_cookie=6ead958051afb26b in_seq=0 out_seq=0 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.077+0000 7f7776ffd640 1 -- 172.21.15.18:0/637254520 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7ee47e890 con 0x55c7ee7a4920 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.077+0000 7f7776ffd640 10 client.17616 handle_command_reply: tid=0 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.077+0000 7f77a0bdd740 1 client.17616 shutdown 2024-02-17T12:55:42.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.078+0000 7f77a0bdd740 1 -- 172.21.15.18:0/637254520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78b250 msgr2=0x55c7ee791700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.078+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78b250 0x55c7ee791700 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f77880ba3a0 tx=0x7f778803cab0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.078+0000 7f77a0bdd740 1 -- 172.21.15.18:0/637254520 shutdown_connections 2024-02-17T12:55:42.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.078+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c7ee7a4920 0x55c7ee7a6d30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.078+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee794750 0x55c7ee792180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.078+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78bb70 0x55c7ee791c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.078+0000 7f7776ffd640 0 client.17616 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:55:42.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.078+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/637254520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78b250 0x55c7ee791700 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.078+0000 7f77a0bdd740 1 -- 172.21.15.18:0/637254520 >> 172.21.15.18:0/637254520 conn(0x55c7ee785980 msgr2=0x55c7ee7876d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.078+0000 7f77a0bdd740 1 -- 172.21.15.18:0/637254520 shutdown_connections 2024-02-17T12:55:42.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.078+0000 7f77a0bdd740 1 -- 172.21.15.18:0/637254520 wait complete. 2024-02-17T12:55:42.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.078+0000 7f77a0bdd740 20 client.17616 trim_cache size 0 max 16384 2024-02-17T12:55:42.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f77a0bdd740 1 Processor -- start 2024-02-17T12:55:42.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f77a0bdd740 1 -- start start 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f77a0bdd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78b250 0x55c7ee784a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f77a0bdd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee78bb70 0x55c7ee79b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f77a0bdd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 0x55c7ee79bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f77a0bdd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ee678130 con 0x55c7ee794750 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f77a0bdd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ee47e8c0 con 0x55c7ee78b250 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f77a0bdd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ee7857f0 con 0x55c7ee78bb70 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f7797fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 0x55c7ee79bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f7797fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 0x55c7ee79bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50484/0 (socket says 172.21.15.18:50484) 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f7797fff640 1 -- 172.21.15.18:0/1321999511 learned_addr learned my addr 172.21.15.18:0/1321999511 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f7796ffd640 1 --2- 172.21.15.18:0/1321999511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee78bb70 0x55c7ee79b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f77977fe640 1 --2- 172.21.15.18:0/1321999511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78b250 0x55c7ee784a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f7797fff640 1 -- 172.21.15.18:0/1321999511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee78bb70 msgr2=0x55c7ee79b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f7797fff640 1 --2- 172.21.15.18:0/1321999511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee78bb70 0x55c7ee79b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f7797fff640 1 -- 172.21.15.18:0/1321999511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78b250 msgr2=0x55c7ee784a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f7797fff640 1 --2- 172.21.15.18:0/1321999511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78b250 0x55c7ee784a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f7797fff640 1 -- 172.21.15.18:0/1321999511 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7ee68dc20 con 0x55c7ee794750 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f7797fff640 1 --2- 172.21.15.18:0/1321999511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 0x55c7ee79bb40 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f7790010a70 tx=0x7f779000e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6258c18658b12fdb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f777d7fa640 1 -- 172.21.15.18:0/1321999511 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779001eb60 con 0x55c7ee794750 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f777d7fa640 1 -- 172.21.15.18:0/1321999511 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7790023070 con 0x55c7ee794750 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.090+0000 7f777d7fa640 1 -- 172.21.15.18:0/1321999511 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779001fe00 con 0x55c7ee794750 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1321999511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 msgr2=0x55c7ee79bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/1321999511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 0x55c7ee79bb40 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f7790010a70 tx=0x7f779000e3d0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1321999511 shutdown_connections 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/1321999511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee794750 0x55c7ee79bb40 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/1321999511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee78bb70 0x55c7ee79b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/1321999511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78b250 0x55c7ee784a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1321999511 >> 172.21.15.18:0/1321999511 conn(0x55c7ee785980 msgr2=0x55c7ee7842f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1321999511 shutdown_connections 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1321999511 wait complete. 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 -- start start 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 Processor -- start 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78b250 0x55c7ee79a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78bb70 0x55c7ee79a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee794750 0x55c7ee7fae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ee79b0c0 con 0x55c7ee78bb70 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ee79b230 con 0x55c7ee78b250 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.091+0000 7f77a0bdd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ee7fb3e0 con 0x55c7ee794750 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f77a0bdd740 10 client.0 _ll_register_callbacks cb 0x7f778c00b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7796ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78bb70 0x55c7ee79a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7796ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78bb70 0x55c7ee79a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50496/0 (socket says 172.21.15.18:50496) 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7796ffd640 1 -- 172.21.15.18:0/1161366632 learned_addr learned my addr 172.21.15.18:0/1161366632 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f77977fe640 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78b250 0x55c7ee79a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7797fff640 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee794750 0x55c7ee7fae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7796ffd640 1 -- 172.21.15.18:0/1161366632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee794750 msgr2=0x55c7ee7fae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7796ffd640 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee794750 0x55c7ee7fae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7796ffd640 1 -- 172.21.15.18:0/1161366632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78b250 msgr2=0x55c7ee79a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7796ffd640 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78b250 0x55c7ee79a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7796ffd640 1 -- 172.21.15.18:0/1161366632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77900107e0 con 0x55c7ee78bb70 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7796ffd640 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78bb70 0x55c7ee79a8b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f778c00b4d0 tx=0x7f778c00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113ed29121e96441 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7776ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7776ffd640 1 -- 172.21.15.18:0/1161366632 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c032070 con 0x55c7ee78bb70 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1161366632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7ee3f27e0 con 0x55c7ee78bb70 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7776ffd640 1 -- 172.21.15.18:0/1161366632 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f778c028040 con 0x55c7ee78bb70 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7776ffd640 1 -- 172.21.15.18:0/1161366632 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f778c00b630 con 0x55c7ee78bb70 2024-02-17T12:55:42.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f77a0bdd740 10 client.17622 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1161366632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c7ee6b38c0 con 0x55c7ee78bb70 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.092+0000 7f7776ffd640 1 -- 172.21.15.18:0/1161366632 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f778c03b080 con 0x55c7ee78bb70 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.093+0000 7f7776ffd640 1 -- 172.21.15.18:0/1161366632 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f778c035030 con 0x55c7ee78bb70 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.093+0000 7f77a0bdd740 10 client.17622 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.093+0000 7f77a0bdd740 10 client.17622 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.093+0000 7f77a0bdd740 20 client.17622 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.093+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c7ee7fcc80 0x55c7ee7ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.093+0000 7f77a0bdd740 4 client.17622 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.093+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1161366632 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c7ee39e700 con 0x55c7ee7fcc80 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.093+0000 7f77977fe640 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c7ee7fcc80 0x55c7ee7ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.093+0000 7f77977fe640 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c7ee7fcc80 0x55c7ee7ff0a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8f299192a7846a3d server_cookie=af10c60eb0356c74 in_seq=0 out_seq=0 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.093+0000 7f7776ffd640 10 client.17622 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f7776ffd640 1 -- 172.21.15.18:0/1161366632 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c7ee39e700 con 0x55c7ee7fcc80 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f7776ffd640 10 client.17622 handle_command_reply: tid=0 2024-02-17T12:55:42.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f77a0bdd740 1 client.17622 shutdown 2024-02-17T12:55:42.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1161366632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78bb70 msgr2=0x55c7ee79a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78bb70 0x55c7ee79a8b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f778c00b4d0 tx=0x7f778c00fbe0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1161366632 shutdown_connections 2024-02-17T12:55:42.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c7ee7fcc80 0x55c7ee7ff0a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c7ee794750 0x55c7ee7fae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c7ee78bb70 0x55c7ee79a8b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f7776ffd640 0 client.17622 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:55:42.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f77a0bdd740 1 --2- 172.21.15.18:0/1161366632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c7ee78b250 0x55c7ee79a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1161366632 >> 172.21.15.18:0/1161366632 conn(0x55c7ee785980 msgr2=0x55c7ee784b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1161366632 shutdown_connections 2024-02-17T12:55:42.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f77a0bdd740 1 -- 172.21.15.18:0/1161366632 wait complete. 2024-02-17T12:55:42.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f77a0bdd740 20 client.17622 trim_cache size 0 max 16384 2024-02-17T12:55:42.093 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:55:42.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f779e00e640 1 -- 172.21.15.18:0/1657601144 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f77800a2230 msgr2=0x7f77800a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f779e00e640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f77800a2230 0x7f77800a46f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7790005fd0 tx=0x7f7790005850 comp rx=0 tx=0).stop 2024-02-17T12:55:42.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f779e00e640 1 -- 172.21.15.18:0/1657601144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77980ffcf0 msgr2=0x7f779819d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f779e00e640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77980ffcf0 0x7f779819d440 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7788004af0 tx=0x7f7788034cc0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f779e00e640 1 -- 172.21.15.18:0/1657601144 shutdown_connections 2024-02-17T12:55:42.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f779e00e640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f77800a2230 0x7f77800a46f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f779e00e640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77981975f0 0x7f7798197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f779e00e640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77981064b0 0x7f779819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f779e00e640 1 --2- 172.21.15.18:0/1657601144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77980ffcf0 0x7f779819d440 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.094+0000 7f779e00e640 1 -- 172.21.15.18:0/1657601144 >> 172.21.15.18:0/1657601144 conn(0x7f77980752e0 msgr2=0x7f77980fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.095+0000 7f779e00e640 1 -- 172.21.15.18:0/1657601144 shutdown_connections 2024-02-17T12:55:42.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.095+0000 7f779e00e640 1 -- 172.21.15.18:0/1657601144 wait complete. 2024-02-17T12:55:42.097 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:55:42.097 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:55:42.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.210+0000 7f41ec935640 1 -- 172.21.15.18:0/3101243105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41e8101360 msgr2=0x7f41e8101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.210+0000 7f41ec935640 1 --2- 172.21.15.18:0/3101243105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41e8101360 0x7f41e8101740 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009930 tx=0x7f41e002bc30 comp rx=0 tx=0).stop 2024-02-17T12:55:42.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.210+0000 7f41ec935640 1 -- 172.21.15.18:0/3101243105 shutdown_connections 2024-02-17T12:55:42.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.210+0000 7f41ec935640 1 --2- 172.21.15.18:0/3101243105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e8106430 0x7f41e810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.210+0000 7f41ec935640 1 --2- 172.21.15.18:0/3101243105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e8101d10 0x7f41e8105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.210+0000 7f41ec935640 1 --2- 172.21.15.18:0/3101243105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41e8101360 0x7f41e8101740 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.210+0000 7f41ec935640 1 -- 172.21.15.18:0/3101243105 >> 172.21.15.18:0/3101243105 conn(0x7f41e80fcbe0 msgr2=0x7f41e80ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.210+0000 7f41ec935640 1 -- 172.21.15.18:0/3101243105 shutdown_connections 2024-02-17T12:55:42.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.210+0000 7f41ec935640 1 -- 172.21.15.18:0/3101243105 wait complete. 2024-02-17T12:55:42.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41ec935640 1 Processor -- start 2024-02-17T12:55:42.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41ec935640 1 -- start start 2024-02-17T12:55:42.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41ec935640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e8101360 0x7f41e8073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41ec935640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41e8106430 0x7f41e80739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41ec935640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e8073f00 0x7f41e81a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41ec935640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e81a5f50 con 0x7f41e8106430 2024-02-17T12:55:42.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41ec935640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e81a60c0 con 0x7f41e8101360 2024-02-17T12:55:42.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41ec935640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e81a6280 con 0x7f41e8073f00 2024-02-17T12:55:42.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41e5d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41e8106430 0x7f41e80739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41e5d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41e8106430 0x7f41e80739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50522/0 (socket says 172.21.15.18:50522) 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41e5d9b640 1 -- 172.21.15.18:0/2677894330 learned_addr learned my addr 172.21.15.18:0/2677894330 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41e659c640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e8101360 0x7f41e8073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41e6d9d640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e8073f00 0x7f41e81a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41e5d9b640 1 -- 172.21.15.18:0/2677894330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e8073f00 msgr2=0x7f41e81a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41e5d9b640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e8073f00 0x7f41e81a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41e5d9b640 1 -- 172.21.15.18:0/2677894330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e8101360 msgr2=0x7f41e8073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41e5d9b640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e8101360 0x7f41e8073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41e5d9b640 1 -- 172.21.15.18:0/2677894330 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e0009560 con 0x7f41e8106430 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.211+0000 7f41e6d9d640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e8073f00 0x7f41e81a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.212+0000 7f41e5d9b640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41e8106430 0x7f41e80739c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00c970 tx=0x7f41dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.212+0000 7f41d77fe640 1 -- 172.21.15.18:0/2677894330 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc004440 con 0x7f41e8106430 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.212+0000 7f41d77fe640 1 -- 172.21.15.18:0/2677894330 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41dc007d40 con 0x7f41e8106430 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.212+0000 7f41ec935640 1 -- 172.21.15.18:0/2677894330 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e81a6560 con 0x7f41e8106430 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.212+0000 7f41d77fe640 1 -- 172.21.15.18:0/2677894330 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00f450 con 0x7f41e8106430 2024-02-17T12:55:42.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.212+0000 7f41ec935640 1 -- 172.21.15.18:0/2677894330 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e81a6ac0 con 0x7f41e8106430 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.214+0000 7f41d77fe640 1 -- 172.21.15.18:0/2677894330 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f41dc007860 con 0x7f41e8106430 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.214+0000 7f41d77fe640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f41d00a22b0 0x7f41d00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.214+0000 7f41ef5e0740 1 Processor -- start 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.214+0000 7f41d77fe640 1 -- 172.21.15.18:0/2677894330 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f41dc0ce0a0 con 0x7f41e8106430 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e659c640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f41d00a22b0 0x7f41d00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41ef5e0740 1 -- start start 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41ef5e0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b983572250 0x55b983572630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41ef5e0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572b70 0x55b98357b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41ef5e0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b98357b750 0x55b98357db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41ef5e0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98352d960 con 0x55b983572b70 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41ef5e0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9834e1310 con 0x55b983572250 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41ef5e0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9834de840 con 0x55b98357b750 2024-02-17T12:55:42.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e5d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572b70 0x55b98357b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e5d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572b70 0x55b98357b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50534/0 (socket says 172.21.15.18:50534) 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e5d9b640 1 -- 172.21.15.18:0/3611025823 learned_addr learned my addr 172.21.15.18:0/3611025823 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e659c640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f41d00a22b0 0x7f41d00a4770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41e0005ab0 tx=0x7f41e0005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e6d9d640 1 --2- 172.21.15.18:0/3611025823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b98357b750 0x55b98357db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e659c640 1 --2- 172.21.15.18:0/3611025823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b983572250 0x55b983572630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e5d9b640 1 -- 172.21.15.18:0/3611025823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b98357b750 msgr2=0x55b98357db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e5d9b640 1 --2- 172.21.15.18:0/3611025823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b98357b750 0x55b98357db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e5d9b640 1 -- 172.21.15.18:0/3611025823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b983572250 msgr2=0x55b983572630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e5d9b640 1 --2- 172.21.15.18:0/3611025823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b983572250 0x55b983572630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e5d9b640 1 -- 172.21.15.18:0/3611025823 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b98349c060 con 0x55b983572b70 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e659c640 1 --2- 172.21.15.18:0/3611025823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b983572250 0x55b983572630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.215+0000 7f41e5d9b640 1 --2- 172.21.15.18:0/3611025823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572b70 0x55b98357b210 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f41dc096fe0 tx=0x7f41dc096f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46316c6594ff5b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.216+0000 7f41d4ff9640 1 -- 172.21.15.18:0/3611025823 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc09ab80 con 0x55b983572b70 2024-02-17T12:55:42.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.216+0000 7f41d4ff9640 1 -- 172.21.15.18:0/3611025823 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41dc09ecd0 con 0x55b983572b70 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.216+0000 7f41d4ff9640 1 -- 172.21.15.18:0/3611025823 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0971f0 con 0x55b983572b70 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.216+0000 7f41ef5e0740 1 -- 172.21.15.18:0/3611025823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572b70 msgr2=0x55b98357b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.216+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/3611025823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572b70 0x55b98357b210 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f41dc096fe0 tx=0x7f41dc096f50 comp rx=0 tx=0).stop 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.216+0000 7f41ef5e0740 1 -- 172.21.15.18:0/3611025823 shutdown_connections 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.216+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/3611025823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b98357b750 0x55b98357db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.216+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/3611025823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572b70 0x55b98357b210 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.216+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/3611025823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b983572250 0x55b983572630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.216+0000 7f41ef5e0740 1 -- 172.21.15.18:0/3611025823 >> 172.21.15.18:0/3611025823 conn(0x55b98356c980 msgr2=0x55b98356cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.216+0000 7f41ef5e0740 1 -- 172.21.15.18:0/3611025823 shutdown_connections 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.216+0000 7f41ef5e0740 1 -- 172.21.15.18:0/3611025823 wait complete. 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41ef5e0740 1 -- start start 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41ef5e0740 1 Processor -- start 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41ef5e0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b983578730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41ef5e0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 0x55b983578c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41ef5e0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 0x55b9835791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41ef5e0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9834da020 con 0x55b983572250 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41ef5e0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9831ad430 con 0x55b98357b750 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41ef5e0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b983169e00 con 0x55b983572b70 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41ef5e0740 10 client.0 _ll_register_callbacks cb 0x55b983481cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b983578730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b983578730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50540/0 (socket says 172.21.15.18:50540) 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41e659c640 1 -- 172.21.15.18:0/2953297173 learned_addr learned my addr 172.21.15.18:0/2953297173 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41e5d9b640 1 --2- 172.21.15.18:0/2953297173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 0x55b983578c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41e659c640 1 -- 172.21.15.18:0/2953297173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 msgr2=0x55b983578c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41e659c640 1 --2- 172.21.15.18:0/2953297173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 0x55b983578c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41e659c640 1 -- 172.21.15.18:0/2953297173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 msgr2=0x55b9835791b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41e659c640 1 --2- 172.21.15.18:0/2953297173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 0x55b9835791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.217+0000 7f41e659c640 1 -- 172.21.15.18:0/2953297173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e81a6ac0 con 0x55b983572250 2024-02-17T12:55:42.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41e659c640 1 --2- 172.21.15.18:0/2953297173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b983578730 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009930 tx=0x7f41e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b364819a4989045 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41cdffb640 1 -- 172.21.15.18:0/2953297173 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0004df0 con 0x55b983572250 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41cdffb640 1 -- 172.21.15.18:0/2953297173 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f41e0004f50 con 0x55b983572250 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2953297173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9831d97e0 con 0x55b983572250 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41cdffb640 1 -- 172.21.15.18:0/2953297173 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41e0005110 con 0x55b983572250 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41ef5e0740 10 client.17634 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2953297173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55b9834d86a0 con 0x55b983572250 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41cdffb640 1 -- 172.21.15.18:0/2953297173 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b9831d97e0 con 0x55b983572250 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41cdffb640 1 -- 172.21.15.18:0/2953297173 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f41e003f440 con 0x55b983572250 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41ef5e0740 10 client.17634 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41ef5e0740 10 client.17634 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41ef5e0740 20 client.17634 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/2953297173 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55b98358ba50 0x55b98358de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41ef5e0740 4 client.17634 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:42.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.218+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2953297173 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b983216720 con 0x55b98358ba50 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.219+0000 7f41e5d9b640 1 --2- 172.21.15.18:0/2953297173 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55b98358ba50 0x55b98358de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.219+0000 7f41e5d9b640 1 --2- 172.21.15.18:0/2953297173 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55b98358ba50 0x55b98358de70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c881aa0ecf83f1d9 server_cookie=425c89b58938f9a2 in_seq=0 out_seq=0 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.219+0000 7f41cdffb640 10 client.17634 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41cdffb640 1 -- 172.21.15.18:0/2953297173 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b983216720 con 0x55b98358ba50 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41cdffb640 10 client.17634 handle_command_reply: tid=0 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41ef5e0740 1 client.17634 shutdown 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2953297173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 msgr2=0x55b983578730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/2953297173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b983578730 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009930 tx=0x7f41e0009260 comp rx=0 tx=0).stop 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2953297173 shutdown_connections 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/2953297173 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55b98358ba50 0x55b98358de70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/2953297173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 0x55b9835791b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/2953297173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 0x55b983578c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41cdffb640 0 client.17634 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/2953297173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b983578730 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2953297173 >> 172.21.15.18:0/2953297173 conn(0x55b98356c980 msgr2=0x55b98356e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2953297173 shutdown_connections 2024-02-17T12:55:42.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2953297173 wait complete. 2024-02-17T12:55:42.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.220+0000 7f41ef5e0740 20 client.17634 trim_cache size 0 max 16384 2024-02-17T12:55:42.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.231+0000 7f41ef5e0740 1 Processor -- start 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41ef5e0740 1 -- start start 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41ef5e0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b98357ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41ef5e0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 0x55b983578fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41ef5e0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 0x55b9835759b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41ef5e0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b98345f130 con 0x55b983572250 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41ef5e0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9831d97e0 con 0x55b98357b750 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41ef5e0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b983579b90 con 0x55b983572b70 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b98357ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b98357ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50554/0 (socket says 172.21.15.18:50554) 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41e659c640 1 -- 172.21.15.18:0/2977040614 learned_addr learned my addr 172.21.15.18:0/2977040614 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41e659c640 1 -- 172.21.15.18:0/2977040614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 msgr2=0x55b983578fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41e659c640 1 --2- 172.21.15.18:0/2977040614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 0x55b983578fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41e659c640 1 -- 172.21.15.18:0/2977040614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 msgr2=0x55b9835759b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41e659c640 1 --2- 172.21.15.18:0/2977040614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 0x55b9835759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41e659c640 1 -- 172.21.15.18:0/2977040614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b983474c20 con 0x55b983572250 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41e659c640 1 --2- 172.21.15.18:0/2977040614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b98357ae60 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f41e003f7f0 tx=0x7f41e0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487ea5a6c3616696 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41cffff640 1 -- 172.21.15.18:0/2977040614 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e003fe40 con 0x55b983572250 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41cffff640 1 -- 172.21.15.18:0/2977040614 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41e003f440 con 0x55b983572250 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.232+0000 7f41cffff640 1 -- 172.21.15.18:0/2977040614 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0003e90 con 0x55b983572250 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2977040614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 msgr2=0x55b98357ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/2977040614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b98357ae60 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f41e003f7f0 tx=0x7f41e0004610 comp rx=0 tx=0).stop 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2977040614 shutdown_connections 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/2977040614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 0x55b9835759b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/2977040614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 0x55b983578fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/2977040614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b98357ae60 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2977040614 >> 172.21.15.18:0/2977040614 conn(0x55b98356c980 msgr2=0x55b983573a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2977040614 shutdown_connections 2024-02-17T12:55:42.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 -- 172.21.15.18:0/2977040614 wait complete. 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 -- start start 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 Processor -- start 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b983582980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 0x55b983580d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 0x55b983581350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b983581a20 con 0x55b983572250 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.233+0000 7f41ef5e0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b983581b90 con 0x55b98357b750 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41ef5e0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9835e1fd0 con 0x55b983572b70 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41ef5e0740 10 client.0 _ll_register_callbacks cb 0x7f41e002cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b983582980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b983582980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50570/0 (socket says 172.21.15.18:50570) 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41e659c640 1 -- 172.21.15.18:0/1708404026 learned_addr learned my addr 172.21.15.18:0/1708404026 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41e6d9d640 1 --2- 172.21.15.18:0/1708404026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 0x55b983581350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41e659c640 1 -- 172.21.15.18:0/1708404026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 msgr2=0x55b983580d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41e659c640 1 --2- 172.21.15.18:0/1708404026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 0x55b983580d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41e659c640 1 -- 172.21.15.18:0/1708404026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 msgr2=0x55b983581350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41e659c640 1 --2- 172.21.15.18:0/1708404026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 0x55b983581350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41e659c640 1 -- 172.21.15.18:0/1708404026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9834d86a0 con 0x55b983572250 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41e659c640 1 --2- 172.21.15.18:0/1708404026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b983582980 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009850 tx=0x7f41e0003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6c993493ccd558 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41cdffb640 1 -- 172.21.15.18:0/1708404026 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e003f5b0 con 0x55b983572250 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41cdffb640 1 -- 172.21.15.18:0/1708404026 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f41e0003800 con 0x55b983572250 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41ef5e0740 1 -- 172.21.15.18:0/1708404026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41dc09ecd0 con 0x55b983572250 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41cdffb640 1 -- 172.21.15.18:0/1708404026 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41e00039c0 con 0x55b983572250 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41ef5e0740 10 client.17640 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41ef5e0740 1 -- 172.21.15.18:0/1708404026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55b98349a8c0 con 0x55b983572250 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.234+0000 7f41cdffb640 1 -- 172.21.15.18:0/1708404026 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41dc09ecd0 con 0x55b983572250 2024-02-17T12:55:42.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41cdffb640 1 -- 172.21.15.18:0/1708404026 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f41e0034670 con 0x55b983572250 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41ef5e0740 10 client.17640 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41ef5e0740 10 client.17640 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41ef5e0740 20 client.17640 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/1708404026 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55b9835e3310 0x55b9835e5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41ef5e0740 4 client.17640 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41ef5e0740 1 -- 172.21.15.18:0/1708404026 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b983265890 con 0x55b9835e3310 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41e5d9b640 1 --2- 172.21.15.18:0/1708404026 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55b9835e3310 0x55b9835e5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41e5d9b640 1 --2- 172.21.15.18:0/1708404026 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55b9835e3310 0x55b9835e5700 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f9fbc36417e6879 server_cookie=614927bc0e7c5c61 in_seq=0 out_seq=0 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41cdffb640 10 client.17640 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41cdffb640 1 -- 172.21.15.18:0/1708404026 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b983265890 con 0x55b9835e3310 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41cdffb640 10 client.17640 handle_command_reply: tid=0 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.235+0000 7f41ef5e0740 1 client.17640 shutdown 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ef5e0740 1 -- 172.21.15.18:0/1708404026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 msgr2=0x55b983582980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/1708404026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b983582980 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009850 tx=0x7f41e0003b70 comp rx=0 tx=0).stop 2024-02-17T12:55:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ef5e0740 1 -- 172.21.15.18:0/1708404026 shutdown_connections 2024-02-17T12:55:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/1708404026 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55b9835e3310 0x55b9835e5700 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/1708404026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b98357b750 0x55b983581350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/1708404026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b983572b70 0x55b983580d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41cdffb640 0 client.17640 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:55:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ef5e0740 1 --2- 172.21.15.18:0/1708404026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b983572250 0x55b983582980 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ef5e0740 1 -- 172.21.15.18:0/1708404026 >> 172.21.15.18:0/1708404026 conn(0x55b98356c980 msgr2=0x55b98356b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ef5e0740 1 -- 172.21.15.18:0/1708404026 shutdown_connections 2024-02-17T12:55:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ef5e0740 1 -- 172.21.15.18:0/1708404026 wait complete. 2024-02-17T12:55:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ef5e0740 20 client.17640 trim_cache size 0 max 16384 2024-02-17T12:55:42.234 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:55:42.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ec935640 1 -- 172.21.15.18:0/2677894330 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f41d00a22b0 msgr2=0x7f41d00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ec935640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f41d00a22b0 0x7f41d00a4770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41e0005ab0 tx=0x7f41e0005fd0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ec935640 1 -- 172.21.15.18:0/2677894330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41e8106430 msgr2=0x7f41e80739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ec935640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41e8106430 0x7f41e80739c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00c970 tx=0x7f41dc00ce30 comp rx=0 tx=0).stop 2024-02-17T12:55:42.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ec935640 1 -- 172.21.15.18:0/2677894330 shutdown_connections 2024-02-17T12:55:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ec935640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f41d00a22b0 0x7f41d00a4770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ec935640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e8073f00 0x7f41e81a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ec935640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41e8106430 0x7f41e80739c0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ec935640 1 --2- 172.21.15.18:0/2677894330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e8101360 0x7f41e8073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ec935640 1 -- 172.21.15.18:0/2677894330 >> 172.21.15.18:0/2677894330 conn(0x7f41e80fcbe0 msgr2=0x7f41e80fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ec935640 1 -- 172.21.15.18:0/2677894330 shutdown_connections 2024-02-17T12:55:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.236+0000 7f41ec935640 1 -- 172.21.15.18:0/2677894330 wait complete. 2024-02-17T12:55:42.238 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:55:42.238 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:55:42.277 INFO:tasks.workunit.client.0.smithi018.stdout: 1 313847 6.33 MB/sec execute 177 sec latency 2440.383 ms 2024-02-17T12:55:42.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.365+0000 7f1be9083640 1 -- 172.21.15.18:0/580569731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1be4101380 msgr2=0x7f1be4101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.365+0000 7f1be9083640 1 --2- 172.21.15.18:0/580569731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1be4101380 0x7f1be4101760 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009930 tx=0x7f1bdc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:55:42.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.365+0000 7f1be9083640 1 -- 172.21.15.18:0/580569731 shutdown_connections 2024-02-17T12:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.365+0000 7f1be9083640 1 --2- 172.21.15.18:0/580569731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1be4106450 0x7f1be410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.365+0000 7f1be9083640 1 --2- 172.21.15.18:0/580569731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1be4101d30 0x7f1be4105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.365+0000 7f1be9083640 1 --2- 172.21.15.18:0/580569731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1be4101380 0x7f1be4101760 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.365+0000 7f1be9083640 1 -- 172.21.15.18:0/580569731 >> 172.21.15.18:0/580569731 conn(0x7f1be4078580 msgr2=0x7f1be4078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.365+0000 7f1be9083640 1 -- 172.21.15.18:0/580569731 shutdown_connections 2024-02-17T12:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.365+0000 7f1be9083640 1 -- 172.21.15.18:0/580569731 wait complete. 2024-02-17T12:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.366+0000 7f1be9083640 1 Processor -- start 2024-02-17T12:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.366+0000 7f1be9083640 1 -- start start 2024-02-17T12:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.366+0000 7f1be9083640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1be4101380 0x7f1be419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.366+0000 7f1be9083640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1be4106450 0x7f1be419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.366+0000 7f1be9083640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1be4197770 0x7f1be4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.366+0000 7f1be9083640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4198240 con 0x7f1be4101380 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.366+0000 7f1be9083640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be41983b0 con 0x7f1be4106450 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.366+0000 7f1be9083640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4198570 con 0x7f1be4197770 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.366+0000 7f1be2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1be4101380 0x7f1be419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.366+0000 7f1be2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1be4101380 0x7f1be419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50584/0 (socket says 172.21.15.18:50584) 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.366+0000 7f1be2d9d640 1 -- 172.21.15.18:0/247619552 learned_addr learned my addr 172.21.15.18:0/247619552 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.366+0000 7f1be359e640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1be4197770 0x7f1be4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.367+0000 7f1be259c640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1be4106450 0x7f1be419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.367+0000 7f1be2d9d640 1 -- 172.21.15.18:0/247619552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1be4197770 msgr2=0x7f1be4197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.367+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1be4197770 0x7f1be4197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.367+0000 7f1be2d9d640 1 -- 172.21.15.18:0/247619552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1be4106450 msgr2=0x7f1be419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.367+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1be4106450 0x7f1be419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.367+0000 7f1be2d9d640 1 -- 172.21.15.18:0/247619552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc009560 con 0x7f1be4101380 2024-02-17T12:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.367+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1be4101380 0x7f1be419d6b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc005940 tx=0x7f1bdc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.367+0000 7f1bcbfff640 1 -- 172.21.15.18:0/247619552 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0044e0 con 0x7f1be4101380 2024-02-17T12:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.367+0000 7f1be9083640 1 -- 172.21.15.18:0/247619552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be41a44b0 con 0x7f1be4101380 2024-02-17T12:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.367+0000 7f1bcbfff640 1 -- 172.21.15.18:0/247619552 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bdc02cbc0 con 0x7f1be4101380 2024-02-17T12:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.367+0000 7f1bcbfff640 1 -- 172.21.15.18:0/247619552 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc003e20 con 0x7f1be4101380 2024-02-17T12:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.367+0000 7f1be9083640 1 -- 172.21.15.18:0/247619552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be41a4990 con 0x7f1be4101380 2024-02-17T12:55:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.369+0000 7f1bcbfff640 1 -- 172.21.15.18:0/247619552 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1bdc0035f0 con 0x7f1be4101380 2024-02-17T12:55:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1bebc4b740 1 Processor -- start 2024-02-17T12:55:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1bebc4b740 1 -- start start 2024-02-17T12:55:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1bebc4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aab250 0x559975aab630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1bebc4b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 0x559975ab4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1bcbfff640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1bcc0a2230 0x7f1bcc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1bebc4b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 0x559975ab6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1bebc4b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559975a66960 con 0x559975ab4750 2024-02-17T12:55:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1bebc4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559975a1a310 con 0x559975aab250 2024-02-17T12:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1bebc4b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559975a17840 con 0x559975aabb70 2024-02-17T12:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1be359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 0x559975ab6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1be259c640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1bcc0a2230 0x7f1bcc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1be359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 0x559975ab6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50600/0 (socket says 172.21.15.18:50600) 2024-02-17T12:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1be359e640 1 -- 172.21.15.18:0/79057647 learned_addr learned my addr 172.21.15.18:0/79057647 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1bcbfff640 1 -- 172.21.15.18:0/247619552 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1bdc0f0280 con 0x7f1be4101380 2024-02-17T12:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/79057647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 0x559975ab4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.370+0000 7f1be359e640 1 --2- 172.21.15.18:0/79057647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aab250 0x559975aab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.371+0000 7f1be359e640 1 -- 172.21.15.18:0/79057647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 msgr2=0x559975ab4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.371+0000 7f1be359e640 1 --2- 172.21.15.18:0/79057647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 0x559975ab4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.371+0000 7f1be359e640 1 -- 172.21.15.18:0/79057647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aab250 msgr2=0x559975aab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.371+0000 7f1be359e640 1 --2- 172.21.15.18:0/79057647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aab250 0x559975aab630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.371+0000 7f1be359e640 1 -- 172.21.15.18:0/79057647 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599759d5060 con 0x559975ab4750 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.371+0000 7f1be259c640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1bcc0a2230 0x7f1bcc0a46f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1be4198fc0 tx=0x7f1bd8008040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.371+0000 7f1be359e640 1 --2- 172.21.15.18:0/79057647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 0x559975ab6b40 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400b830 tx=0x7f1bd4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf012b10cf9be50 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.371+0000 7f1bc97fa640 1 -- 172.21.15.18:0/79057647 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd400be70 con 0x559975ab4750 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.371+0000 7f1bc97fa640 1 -- 172.21.15.18:0/79057647 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bd400b260 con 0x559975ab4750 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.371+0000 7f1bc97fa640 1 -- 172.21.15.18:0/79057647 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd400ab70 con 0x559975ab4750 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 -- 172.21.15.18:0/79057647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 msgr2=0x559975ab6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/79057647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 0x559975ab6b40 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400b830 tx=0x7f1bd4009c30 comp rx=0 tx=0).stop 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 -- 172.21.15.18:0/79057647 shutdown_connections 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/79057647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 0x559975ab6b40 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/79057647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 0x559975ab4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/79057647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aab250 0x559975aab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 -- 172.21.15.18:0/79057647 >> 172.21.15.18:0/79057647 conn(0x559975aa5980 msgr2=0x559975aa5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 -- 172.21.15.18:0/79057647 shutdown_connections 2024-02-17T12:55:42.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 -- 172.21.15.18:0/79057647 wait complete. 2024-02-17T12:55:42.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 -- start start 2024-02-17T12:55:42.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 Processor -- start 2024-02-17T12:55:42.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aab250 0x559975ab1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 0x559975ab1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 0x559975ab2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559975a13020 con 0x559975ab4750 2024-02-17T12:55:42.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599756e6430 con 0x559975aab250 2024-02-17T12:55:42.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599756a2e00 con 0x559975aabb70 2024-02-17T12:55:42.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1bebc4b740 10 client.0 _ll_register_callbacks cb 0x5599759bacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:42.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1be359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 0x559975ab2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1be359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 0x559975ab2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50616/0 (socket says 172.21.15.18:50616) 2024-02-17T12:55:42.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1be359e640 1 -- 172.21.15.18:0/2839414824 learned_addr learned my addr 172.21.15.18:0/2839414824 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.372+0000 7f1be259c640 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 0x559975ab1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aab250 0x559975ab1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1be359e640 1 -- 172.21.15.18:0/2839414824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 msgr2=0x559975ab1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1be359e640 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 0x559975ab1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1be359e640 1 -- 172.21.15.18:0/2839414824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aab250 msgr2=0x559975ab1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1be359e640 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aab250 0x559975ab1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1be359e640 1 -- 172.21.15.18:0/2839414824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd4013040 con 0x559975ab4750 2024-02-17T12:55:42.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aab250 0x559975ab1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:42.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1be359e640 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 0x559975ab2200 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4002230 tx=0x7f1bd4007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f546f60bb66f1133 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1bc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:42.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1bc6ff5640 1 -- 172.21.15.18:0/2839414824 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd400a510 con 0x559975ab4750 2024-02-17T12:55:42.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1bc6ff5640 1 -- 172.21.15.18:0/2839414824 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1bd40087c0 con 0x559975ab4750 2024-02-17T12:55:42.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1bebc4b740 1 -- 172.21.15.18:0/2839414824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599757127e0 con 0x559975ab4750 2024-02-17T12:55:42.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1bc6ff5640 1 -- 172.21.15.18:0/2839414824 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bd400b3d0 con 0x559975ab4750 2024-02-17T12:55:42.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1bebc4b740 10 client.17652 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:42.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1bebc4b740 1 -- 172.21.15.18:0/2839414824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x559975a116a0 con 0x559975ab4750 2024-02-17T12:55:42.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1bc6ff5640 1 -- 172.21.15.18:0/2839414824 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5599757127e0 con 0x559975ab4750 2024-02-17T12:55:42.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.373+0000 7f1bc6ff5640 1 -- 172.21.15.18:0/2839414824 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1bd402a060 con 0x559975ab4750 2024-02-17T12:55:42.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.374+0000 7f1bebc4b740 10 client.17652 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:42.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.374+0000 7f1bebc4b740 10 client.17652 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:55:42.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.374+0000 7f1bebc4b740 20 client.17652 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:42.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.374+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x559975ac4b30 0x559975ac6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.374+0000 7f1bebc4b740 4 client.17652 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:42.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.374+0000 7f1bebc4b740 1 -- 172.21.15.18:0/2839414824 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55997574f720 con 0x559975ac4b30 2024-02-17T12:55:42.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.374+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x559975ac4b30 0x559975ac6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.374+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x559975ac4b30 0x559975ac6f50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2f7d5830de94aa server_cookie=a4bb9f7e20dc4a96 in_seq=0 out_seq=0 2024-02-17T12:55:42.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.374+0000 7f1bc6ff5640 10 client.17652 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:55:42.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bc6ff5640 1 -- 172.21.15.18:0/2839414824 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55997574f720 con 0x559975ac4b30 2024-02-17T12:55:42.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bc6ff5640 10 client.17652 handle_command_reply: tid=0 2024-02-17T12:55:42.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bebc4b740 1 client.17652 shutdown 2024-02-17T12:55:42.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bebc4b740 1 -- 172.21.15.18:0/2839414824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 msgr2=0x559975ab2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 0x559975ab2200 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4002230 tx=0x7f1bd4007000 comp rx=0 tx=0).stop 2024-02-17T12:55:42.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bebc4b740 1 -- 172.21.15.18:0/2839414824 shutdown_connections 2024-02-17T12:55:42.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x559975ac4b30 0x559975ac6f50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bc6ff5640 0 client.17652 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:55:42.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975ab4750 0x559975ab2200 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 0x559975ab1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/2839414824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aab250 0x559975ab1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bebc4b740 1 -- 172.21.15.18:0/2839414824 >> 172.21.15.18:0/2839414824 conn(0x559975aa5980 msgr2=0x559975aa76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bebc4b740 1 -- 172.21.15.18:0/2839414824 shutdown_connections 2024-02-17T12:55:42.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bebc4b740 1 -- 172.21.15.18:0/2839414824 wait complete. 2024-02-17T12:55:42.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.375+0000 7f1bebc4b740 20 client.17652 trim_cache size 0 max 16384 2024-02-17T12:55:42.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1bebc4b740 1 Processor -- start 2024-02-17T12:55:42.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1bebc4b740 1 -- start start 2024-02-17T12:55:42.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1bebc4b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 0x559975ab3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1bebc4b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 0x559975ab2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1bebc4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975ab4750 0x559975acb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1bebc4b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559975998130 con 0x559975aab250 2024-02-17T12:55:42.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1bebc4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559975aaac90 con 0x559975ab4750 2024-02-17T12:55:42.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1bebc4b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559975ab0760 con 0x559975aabb70 2024-02-17T12:55:42.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1be2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 0x559975ab3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1be2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 0x559975ab3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50618/0 (socket says 172.21.15.18:50618) 2024-02-17T12:55:42.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1be2d9d640 1 -- 172.21.15.18:0/3483680231 learned_addr learned my addr 172.21.15.18:0/3483680231 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1be259c640 1 --2- 172.21.15.18:0/3483680231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 0x559975ab2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1be2d9d640 1 -- 172.21.15.18:0/3483680231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 msgr2=0x559975ab2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/3483680231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 0x559975ab2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1be2d9d640 1 -- 172.21.15.18:0/3483680231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975ab4750 msgr2=0x559975acb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/3483680231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975ab4750 0x559975acb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.387+0000 7f1be2d9d640 1 -- 172.21.15.18:0/3483680231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599759adc20 con 0x559975aab250 2024-02-17T12:55:42.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/3483680231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 0x559975ab3eb0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0b91d0 tx=0x7f1bdc003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3d7db69f1e81ca server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bc8ff9640 1 -- 172.21.15.18:0/3483680231 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0bb520 con 0x559975aab250 2024-02-17T12:55:42.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bc8ff9640 1 -- 172.21.15.18:0/3483680231 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bdc0bb330 con 0x559975aab250 2024-02-17T12:55:42.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bc8ff9640 1 -- 172.21.15.18:0/3483680231 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0bb100 con 0x559975aab250 2024-02-17T12:55:42.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bebc4b740 1 -- 172.21.15.18:0/3483680231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 msgr2=0x559975ab3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/3483680231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 0x559975ab3eb0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0b91d0 tx=0x7f1bdc003920 comp rx=0 tx=0).stop 2024-02-17T12:55:42.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bebc4b740 1 -- 172.21.15.18:0/3483680231 shutdown_connections 2024-02-17T12:55:42.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/3483680231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975ab4750 0x559975acb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/3483680231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975aabb70 0x559975ab2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/3483680231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 0x559975ab3eb0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bebc4b740 1 -- 172.21.15.18:0/3483680231 >> 172.21.15.18:0/3483680231 conn(0x559975aa5980 msgr2=0x559975ab0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bebc4b740 1 -- 172.21.15.18:0/3483680231 shutdown_connections 2024-02-17T12:55:42.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bebc4b740 1 -- 172.21.15.18:0/3483680231 wait complete. 2024-02-17T12:55:42.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.388+0000 7f1bebc4b740 1 -- start start 2024-02-17T12:55:42.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1bebc4b740 1 Processor -- start 2024-02-17T12:55:42.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1bebc4b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 0x559975abb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1bebc4b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aabb70 0x559975abbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1bebc4b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975ab4750 0x559975ab9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1bebc4b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559975aba210 con 0x559975aab250 2024-02-17T12:55:42.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1bebc4b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559975aba380 con 0x559975aabb70 2024-02-17T12:55:42.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1bebc4b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559975aba540 con 0x559975ab4750 2024-02-17T12:55:42.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1bebc4b740 10 client.0 _ll_register_callbacks cb 0x7f1bd4009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:42.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1be2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 0x559975abb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1be2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 0x559975abb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50626/0 (socket says 172.21.15.18:50626) 2024-02-17T12:55:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1be2d9d640 1 -- 172.21.15.18:0/212505969 learned_addr learned my addr 172.21.15.18:0/212505969 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1be259c640 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aabb70 0x559975abbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1be2d9d640 1 -- 172.21.15.18:0/212505969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975ab4750 msgr2=0x559975ab9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975ab4750 0x559975ab9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1be2d9d640 1 -- 172.21.15.18:0/212505969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aabb70 msgr2=0x559975abbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aabb70 0x559975abbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1be2d9d640 1 -- 172.21.15.18:0/212505969 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be41a4990 con 0x559975aab250 2024-02-17T12:55:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1be259c640 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aabb70 0x559975abbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1be2d9d640 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 0x559975abb570 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0bca70 tx=0x7f1bdc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe696264f80ef27 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1bc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1bc6ff5640 1 -- 172.21.15.18:0/212505969 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0bd470 con 0x559975aab250 2024-02-17T12:55:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1bc6ff5640 1 -- 172.21.15.18:0/212505969 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1bdc0b93e0 con 0x559975aab250 2024-02-17T12:55:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.389+0000 7f1bebc4b740 1 -- 172.21.15.18:0/212505969 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bd400b260 con 0x559975aab250 2024-02-17T12:55:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.390+0000 7f1bc6ff5640 1 -- 172.21.15.18:0/212505969 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bdc0b95a0 con 0x559975aab250 2024-02-17T12:55:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.390+0000 7f1bebc4b740 10 client.17658 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.390+0000 7f1bebc4b740 1 -- 172.21.15.18:0/212505969 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5599759d38c0 con 0x559975aab250 2024-02-17T12:55:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.390+0000 7f1bc6ff5640 1 -- 172.21.15.18:0/212505969 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bd400b260 con 0x559975aab250 2024-02-17T12:55:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.390+0000 7f1bc6ff5640 1 -- 172.21.15.18:0/212505969 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1bdc0a7040 con 0x559975aab250 2024-02-17T12:55:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.390+0000 7f1bebc4b740 10 client.17658 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.390+0000 7f1bebc4b740 10 client.17658 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:55:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.390+0000 7f1bebc4b740 20 client.17658 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.390+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x559975ac4b30 0x559975b1e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.390+0000 7f1bebc4b740 4 client.17658 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:55:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.390+0000 7f1bebc4b740 1 -- 172.21.15.18:0/212505969 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55997579e890 con 0x559975ac4b30 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.390+0000 7f1be259c640 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x559975ac4b30 0x559975b1e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1be259c640 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x559975ac4b30 0x559975b1e0c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9460583db1039faa server_cookie=b49ca7805f01f319 in_seq=0 out_seq=0 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bc6ff5640 10 client.17658 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bc6ff5640 1 -- 172.21.15.18:0/212505969 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55997579e890 con 0x559975ac4b30 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bc6ff5640 10 client.17658 handle_command_reply: tid=0 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bebc4b740 1 client.17658 shutdown 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bebc4b740 1 -- 172.21.15.18:0/212505969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 msgr2=0x559975abb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 0x559975abb570 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0bca70 tx=0x7f1bdc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bebc4b740 1 -- 172.21.15.18:0/212505969 shutdown_connections 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x559975ac4b30 0x559975b1e0c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559975ab4750 0x559975ab9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559975aabb70 0x559975abbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bebc4b740 1 --2- 172.21.15.18:0/212505969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559975aab250 0x559975abb570 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bc6ff5640 0 client.17658 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:55:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bebc4b740 1 -- 172.21.15.18:0/212505969 >> 172.21.15.18:0/212505969 conn(0x559975aa5980 msgr2=0x559975acb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bebc4b740 1 -- 172.21.15.18:0/212505969 shutdown_connections 2024-02-17T12:55:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bebc4b740 1 -- 172.21.15.18:0/212505969 wait complete. 2024-02-17T12:55:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.391+0000 7f1bebc4b740 20 client.17658 trim_cache size 0 max 16384 2024-02-17T12:55:42.392 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:55:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.392+0000 7f1be9083640 1 -- 172.21.15.18:0/247619552 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1bcc0a2230 msgr2=0x7f1bcc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.392+0000 7f1be9083640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1bcc0a2230 0x7f1bcc0a46f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1be4198fc0 tx=0x7f1bd8008040 comp rx=0 tx=0).stop 2024-02-17T12:55:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.392+0000 7f1be9083640 1 -- 172.21.15.18:0/247619552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1be4101380 msgr2=0x7f1be419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.392+0000 7f1be9083640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1be4101380 0x7f1be419d6b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc005940 tx=0x7f1bdc0042f0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.392+0000 7f1be9083640 1 -- 172.21.15.18:0/247619552 shutdown_connections 2024-02-17T12:55:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.392+0000 7f1be9083640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1bcc0a2230 0x7f1bcc0a46f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.392+0000 7f1be9083640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1be4197770 0x7f1be4197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.392+0000 7f1be9083640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1be4106450 0x7f1be419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.392+0000 7f1be9083640 1 --2- 172.21.15.18:0/247619552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1be4101380 0x7f1be419d6b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.392+0000 7f1be9083640 1 -- 172.21.15.18:0/247619552 >> 172.21.15.18:0/247619552 conn(0x7f1be4078580 msgr2=0x7f1be40ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.392+0000 7f1be9083640 1 -- 172.21.15.18:0/247619552 shutdown_connections 2024-02-17T12:55:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.392+0000 7f1be9083640 1 -- 172.21.15.18:0/247619552 wait complete. 2024-02-17T12:55:42.395 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:55:42.395 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:55:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.512+0000 7f80a9cae640 1 -- 172.21.15.18:0/3049563464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a4068800 msgr2=0x7f80a4068c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.512+0000 7f80a9cae640 1 --2- 172.21.15.18:0/3049563464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a4068800 0x7f80a4068c60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f8098009900 tx=0x7f809802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.512+0000 7f80a9cae640 1 -- 172.21.15.18:0/3049563464 shutdown_connections 2024-02-17T12:55:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.512+0000 7f80a9cae640 1 --2- 172.21.15.18:0/3049563464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a40691a0 0x7f80a410e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.512+0000 7f80a9cae640 1 --2- 172.21.15.18:0/3049563464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a4068800 0x7f80a4068c60 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.512+0000 7f80a9cae640 1 --2- 172.21.15.18:0/3049563464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80a4107270 0x7f80a4107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.512+0000 7f80a9cae640 1 -- 172.21.15.18:0/3049563464 >> 172.21.15.18:0/3049563464 conn(0x7f80a4076ed0 msgr2=0x7f80a40772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.512+0000 7f80a9cae640 1 -- 172.21.15.18:0/3049563464 shutdown_connections 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.512+0000 7f80a9cae640 1 -- 172.21.15.18:0/3049563464 wait complete. 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a9cae640 1 Processor -- start 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a9cae640 1 -- start start 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a9cae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80a4068800 0x7f80a4197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a9cae640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40691a0 0x7f80a4197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a9cae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a4107270 0x7f80a419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a9cae640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a419c510 con 0x7f80a40691a0 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a9cae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a419c680 con 0x7f80a4068800 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a9cae640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a419c840 con 0x7f80a4107270 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40691a0 0x7f80a4197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40691a0 0x7f80a4197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50654/0 (socket says 172.21.15.18:50654) 2024-02-17T12:55:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a2ffd640 1 -- 172.21.15.18:0/3933678385 learned_addr learned my addr 172.21.15.18:0/3933678385 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a37fe640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80a4068800 0x7f80a4197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.513+0000 7f80a3fff640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a4107270 0x7f80a419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.514+0000 7f80a2ffd640 1 -- 172.21.15.18:0/3933678385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a4107270 msgr2=0x7f80a419bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.514+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a4107270 0x7f80a419bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.514+0000 7f80a2ffd640 1 -- 172.21.15.18:0/3933678385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80a4068800 msgr2=0x7f80a4197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.514+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80a4068800 0x7f80a4197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.514+0000 7f80a2ffd640 1 -- 172.21.15.18:0/3933678385 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8098009560 con 0x7f80a40691a0 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.514+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40691a0 0x7f80a4197b90 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8098005940 tx=0x7f8098004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.514+0000 7f80a0ff9640 1 -- 172.21.15.18:0/3933678385 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098004440 con 0x7f80a40691a0 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.514+0000 7f80a0ff9640 1 -- 172.21.15.18:0/3933678385 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80980052d0 con 0x7f80a40691a0 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.514+0000 7f80a9cae640 1 -- 172.21.15.18:0/3933678385 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a419cac0 con 0x7f80a40691a0 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.514+0000 7f80a0ff9640 1 -- 172.21.15.18:0/3933678385 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098003da0 con 0x7f80a40691a0 2024-02-17T12:55:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.514+0000 7f80a9cae640 1 -- 172.21.15.18:0/3933678385 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a41a43a0 con 0x7f80a40691a0 2024-02-17T12:55:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.516+0000 7f80a0ff9640 1 -- 172.21.15.18:0/3933678385 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f809803e450 con 0x7f80a40691a0 2024-02-17T12:55:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.516+0000 7f80ac522740 1 Processor -- start 2024-02-17T12:55:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80ac522740 1 -- start start 2024-02-17T12:55:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a0ff9640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f808c0a22d0 0x7f808c0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80ac522740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37afad0 0x5611e37ae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80ac522740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37aed60 0x5611e37af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80ac522740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37bc6f0 0x5611e37beae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80ac522740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e371d9f0 con 0x5611e37bc6f0 2024-02-17T12:55:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80ac522740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e371af20 con 0x5611e37aed60 2024-02-17T12:55:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80ac522740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e3716700 con 0x5611e37afad0 2024-02-17T12:55:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a0ff9640 1 -- 172.21.15.18:0/3933678385 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f80980f0700 con 0x7f80a40691a0 2024-02-17T12:55:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a3fff640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f808c0a22d0 0x7f808c0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37bc6f0 0x5611e37beae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37bc6f0 0x5611e37beae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50670/0 (socket says 172.21.15.18:50670) 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a3fff640 1 -- 172.21.15.18:0/1939387540 learned_addr learned my addr 172.21.15.18:0/1939387540 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/1939387540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37aed60 0x5611e37af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a37fe640 1 --2- 172.21.15.18:0/1939387540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37afad0 0x5611e37ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a2ffd640 1 -- 172.21.15.18:0/1939387540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37afad0 msgr2=0x5611e37ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/1939387540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37afad0 0x5611e37ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a2ffd640 1 -- 172.21.15.18:0/1939387540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37bc6f0 msgr2=0x5611e37beae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/1939387540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37bc6f0 0x5611e37beae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.517+0000 7f80a2ffd640 1 -- 172.21.15.18:0/1939387540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611e36ac7b0 con 0x5611e37aed60 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.518+0000 7f80a3fff640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f808c0a22d0 0x7f808c0a4790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f809c012c60 tx=0x7f809c009e70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.518+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/1939387540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37aed60 0x5611e37af1e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80980edfb0 tx=0x7f80980bd0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=419b995616729ec1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.518+0000 7f8089ffb640 1 -- 172.21.15.18:0/1939387540 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980c1160 con 0x5611e37aed60 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.518+0000 7f8089ffb640 1 -- 172.21.15.18:0/1939387540 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80980de040 con 0x5611e37aed60 2024-02-17T12:55:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.518+0000 7f8089ffb640 1 -- 172.21.15.18:0/1939387540 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980de1b0 con 0x5611e37aed60 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.518+0000 7f80ac522740 1 -- 172.21.15.18:0/1939387540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37aed60 msgr2=0x5611e37af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.518+0000 7f80ac522740 1 --2- 172.21.15.18:0/1939387540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37aed60 0x5611e37af1e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80980edfb0 tx=0x7f80980bd0e0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 -- 172.21.15.18:0/1939387540 shutdown_connections 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 --2- 172.21.15.18:0/1939387540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37bc6f0 0x5611e37beae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 --2- 172.21.15.18:0/1939387540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37aed60 0x5611e37af1e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 --2- 172.21.15.18:0/1939387540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37afad0 0x5611e37ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 -- 172.21.15.18:0/1939387540 >> 172.21.15.18:0/1939387540 conn(0x5611e3776550 msgr2=0x5611e37a9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 -- 172.21.15.18:0/1939387540 shutdown_connections 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 -- 172.21.15.18:0/1939387540 wait complete. 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 -- start start 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 Processor -- start 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 0x5611e37b96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37afad0 0x5611e37b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37bc6f0 0x5611e37ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e36bbbf0 con 0x5611e37aed60 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e33ea250 con 0x5611e37bc6f0 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e369c200 con 0x5611e37afad0 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.519+0000 7f80ac522740 10 client.0 _ll_register_callbacks cb 0x5611e370ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80a37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 0x5611e37b96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80a37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 0x5611e37b96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50676/0 (socket says 172.21.15.18:50676) 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80a37fe640 1 -- 172.21.15.18:0/3197819966 learned_addr learned my addr 172.21.15.18:0/3197819966 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37afad0 0x5611e37b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80a3fff640 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37bc6f0 0x5611e37ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80a37fe640 1 -- 172.21.15.18:0/3197819966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37afad0 msgr2=0x5611e37b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80a37fe640 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37afad0 0x5611e37b9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80a37fe640 1 -- 172.21.15.18:0/3197819966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37bc6f0 msgr2=0x5611e37ba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80a37fe640 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37bc6f0 0x5611e37ba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80a37fe640 1 -- 172.21.15.18:0/3197819966 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a41a43a0 con 0x5611e37aed60 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80a37fe640 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 0x5611e37b96c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f809400bab0 tx=0x7f809400bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd66eebd2bef6edf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f8082ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f8082ffd640 1 -- 172.21.15.18:0/3197819966 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094016070 con 0x5611e37aed60 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80ac522740 1 -- 172.21.15.18:0/3197819966 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611e3416600 con 0x5611e37aed60 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f8082ffd640 1 -- 172.21.15.18:0/3197819966 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8094011440 con 0x5611e37aed60 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f8082ffd640 1 -- 172.21.15.18:0/3197819966 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80940041f0 con 0x5611e37aed60 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80ac522740 10 client.17670 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.520+0000 7f80ac522740 1 -- 172.21.15.18:0/3197819966 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5611e3714d80 con 0x5611e37aed60 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.521+0000 7f8082ffd640 1 -- 172.21.15.18:0/3197819966 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5611e3416600 con 0x5611e37aed60 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.521+0000 7f8082ffd640 1 -- 172.21.15.18:0/3197819966 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f809401b7f0 con 0x5611e37aed60 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.521+0000 7f80ac522740 10 client.17670 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.521+0000 7f80ac522740 10 client.17670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.521+0000 7f80ac522740 20 client.17670 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.521+0000 7f80ac522740 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5611e37cc7d0 0x5611e37cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.521+0000 7f80ac522740 4 client.17670 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.521+0000 7f80ac522740 1 -- 172.21.15.18:0/3197819966 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611e34671b0 con 0x5611e37cc7d0 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.521+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5611e37cc7d0 0x5611e37cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f8082ffd640 10 client.17670 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5611e37cc7d0 0x5611e37cebf0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be53998bcbbfd6f server_cookie=59f453ca0784068 in_seq=0 out_seq=0 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f8082ffd640 1 -- 172.21.15.18:0/3197819966 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5611e34671b0 con 0x5611e37cc7d0 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f8082ffd640 10 client.17670 handle_command_reply: tid=0 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f80ac522740 1 client.17670 shutdown 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f80ac522740 1 -- 172.21.15.18:0/3197819966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 msgr2=0x5611e37b96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f80ac522740 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 0x5611e37b96c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f809400bab0 tx=0x7f809400bf70 comp rx=0 tx=0).stop 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f80ac522740 1 -- 172.21.15.18:0/3197819966 shutdown_connections 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f80ac522740 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5611e37cc7d0 0x5611e37cebf0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f80ac522740 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37bc6f0 0x5611e37ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f80ac522740 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37afad0 0x5611e37b9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f8082ffd640 0 client.17670 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f80ac522740 1 --2- 172.21.15.18:0/3197819966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 0x5611e37b96c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f80ac522740 1 -- 172.21.15.18:0/3197819966 >> 172.21.15.18:0/3197819966 conn(0x5611e3776550 msgr2=0x5611e37ab7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f80ac522740 1 -- 172.21.15.18:0/3197819966 shutdown_connections 2024-02-17T12:55:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.522+0000 7f80ac522740 1 -- 172.21.15.18:0/3197819966 wait complete. 2024-02-17T12:55:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.523+0000 7f80ac522740 20 client.17670 trim_cache size 0 max 16384 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80ac522740 1 Processor -- start 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80ac522740 1 -- start start 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80ac522740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 0x5611e37bbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80ac522740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37afad0 0x5611e37b9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80ac522740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37bc6f0 0x5611e37b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80ac522740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e37cc5c0 con 0x5611e37aed60 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80ac522740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e37aaa50 con 0x5611e37afad0 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80ac522740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e37caed0 con 0x5611e37bc6f0 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80a37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 0x5611e37bbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80a37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 0x5611e37bbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50678/0 (socket says 172.21.15.18:50678) 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80a37fe640 1 -- 172.21.15.18:0/3048313647 learned_addr learned my addr 172.21.15.18:0/3048313647 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80a3fff640 1 --2- 172.21.15.18:0/3048313647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37bc6f0 0x5611e37b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/3048313647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37afad0 0x5611e37b9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80a37fe640 1 -- 172.21.15.18:0/3048313647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37bc6f0 msgr2=0x5611e37b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80a37fe640 1 --2- 172.21.15.18:0/3048313647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37bc6f0 0x5611e37b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80a37fe640 1 -- 172.21.15.18:0/3048313647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37afad0 msgr2=0x5611e37b9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80a37fe640 1 --2- 172.21.15.18:0/3048313647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37afad0 0x5611e37b9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80a37fe640 1 -- 172.21.15.18:0/3048313647 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611e36b17b0 con 0x5611e37aed60 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/3048313647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37afad0 0x5611e37b9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.536+0000 7f80a3fff640 1 --2- 172.21.15.18:0/3048313647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37bc6f0 0x5611e37b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.537+0000 7f80a37fe640 1 --2- 172.21.15.18:0/3048313647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 0x5611e37bbdf0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8094000c00 tx=0x7f80940058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec5886c19ea6d16 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.537+0000 7f80897fa640 1 -- 172.21.15.18:0/3048313647 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094016030 con 0x5611e37aed60 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.537+0000 7f80897fa640 1 -- 172.21.15.18:0/3048313647 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8094017040 con 0x5611e37aed60 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.537+0000 7f80897fa640 1 -- 172.21.15.18:0/3048313647 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094004760 con 0x5611e37aed60 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.537+0000 7f80ac522740 1 -- 172.21.15.18:0/3048313647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 msgr2=0x5611e37bbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.537+0000 7f80ac522740 1 --2- 172.21.15.18:0/3048313647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 0x5611e37bbdf0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8094000c00 tx=0x7f80940058d0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.537+0000 7f80ac522740 1 -- 172.21.15.18:0/3048313647 shutdown_connections 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.537+0000 7f80ac522740 1 --2- 172.21.15.18:0/3048313647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37bc6f0 0x5611e37b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.537+0000 7f80ac522740 1 --2- 172.21.15.18:0/3048313647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37afad0 0x5611e37b9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.537+0000 7f80ac522740 1 --2- 172.21.15.18:0/3048313647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37aed60 0x5611e37bbdf0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.537+0000 7f80ac522740 1 -- 172.21.15.18:0/3048313647 >> 172.21.15.18:0/3048313647 conn(0x5611e3776550 msgr2=0x5611e37b7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80ac522740 1 -- 172.21.15.18:0/3048313647 shutdown_connections 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80ac522740 1 -- 172.21.15.18:0/3048313647 wait complete. 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80ac522740 1 -- start start 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80ac522740 1 Processor -- start 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80ac522740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37aed60 0x5611e37c2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80ac522740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37afad0 0x5611e37c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80ac522740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37bc6f0 0x5611e3822d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80ac522740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e37c2eb0 con 0x5611e37afad0 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80ac522740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e3823260 con 0x5611e37aed60 2024-02-17T12:55:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80ac522740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e38233a0 con 0x5611e37bc6f0 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80ac522740 10 client.0 _ll_register_callbacks cb 0x7f80980045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80a2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37afad0 0x5611e37c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80a2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37afad0 0x5611e37c27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50694/0 (socket says 172.21.15.18:50694) 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80a37fe640 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37aed60 0x5611e37c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.538+0000 7f80a2ffd640 1 -- 172.21.15.18:0/2943191829 learned_addr learned my addr 172.21.15.18:0/2943191829 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f80a3fff640 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37bc6f0 0x5611e3822d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f80a2ffd640 1 -- 172.21.15.18:0/2943191829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37bc6f0 msgr2=0x5611e3822d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37bc6f0 0x5611e3822d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f80a2ffd640 1 -- 172.21.15.18:0/2943191829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37aed60 msgr2=0x5611e37c2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37aed60 0x5611e37c2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f80a2ffd640 1 -- 172.21.15.18:0/2943191829 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611e3714d80 con 0x5611e37afad0 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f80a37fe640 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37aed60 0x5611e37c2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f80a2ffd640 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37afad0 0x5611e37c27d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f80980ee4a0 tx=0x7f80980b95c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5652113272872f3d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f8082ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f8082ffd640 1 -- 172.21.15.18:0/2943191829 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980bd250 con 0x5611e37afad0 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f8082ffd640 1 -- 172.21.15.18:0/2943191829 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f80980c13a0 con 0x5611e37afad0 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f80ac522740 1 -- 172.21.15.18:0/2943191829 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80980de040 con 0x5611e37afad0 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f8082ffd640 1 -- 172.21.15.18:0/2943191829 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80980c59f0 con 0x5611e37afad0 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f80ac522740 10 client.17676 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f80ac522740 1 -- 172.21.15.18:0/2943191829 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5611e36d7540 con 0x5611e37afad0 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.539+0000 7f8082ffd640 1 -- 172.21.15.18:0/2943191829 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80980de040 con 0x5611e37afad0 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.540+0000 7f8082ffd640 1 -- 172.21.15.18:0/2943191829 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f809803c070 con 0x5611e37afad0 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.540+0000 7f80ac522740 10 client.17676 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.540+0000 7f80ac522740 10 client.17676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.540+0000 7f80ac522740 20 client.17676 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.540+0000 7f80ac522740 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5611e3824910 0x5611e3826d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.540+0000 7f80ac522740 4 client.17676 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:55:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.540+0000 7f80ac522740 1 -- 172.21.15.18:0/2943191829 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5611e3453550 con 0x5611e3824910 2024-02-17T12:55:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.540+0000 7f80a37fe640 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5611e3824910 0x5611e3826d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.540+0000 7f8082ffd640 10 client.17676 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.540+0000 7f80a37fe640 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5611e3824910 0x5611e3826d30 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e158d3bd180aa4eb server_cookie=696131ec32a5e68e in_seq=0 out_seq=0 2024-02-17T12:55:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.559+0000 7f2926c4d640 1 -- 172.21.15.18:0/3380456175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29201060d0 msgr2=0x7f2920068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.559+0000 7f2926c4d640 1 --2- 172.21.15.18:0/3380456175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29201060d0 0x7f2920068380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2910009930 tx=0x7f291002bc50 comp rx=0 tx=0).stop 2024-02-17T12:55:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.559+0000 7f2926c4d640 1 -- 172.21.15.18:0/3380456175 shutdown_connections 2024-02-17T12:55:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.559+0000 7f2926c4d640 1 --2- 172.21.15.18:0/3380456175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2920103c40 0x7f29201040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.559+0000 7f2926c4d640 1 --2- 172.21.15.18:0/3380456175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29200688c0 0x7f2920068d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.559+0000 7f2926c4d640 1 --2- 172.21.15.18:0/3380456175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29201060d0 0x7f2920068380 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.559+0000 7f2926c4d640 1 -- 172.21.15.18:0/3380456175 >> 172.21.15.18:0/3380456175 conn(0x7f2920077180 msgr2=0x7f2920077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.559+0000 7f2926c4d640 1 -- 172.21.15.18:0/3380456175 shutdown_connections 2024-02-17T12:55:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.559+0000 7f2926c4d640 1 -- 172.21.15.18:0/3380456175 wait complete. 2024-02-17T12:55:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.560+0000 7f2926c4d640 1 Processor -- start 2024-02-17T12:55:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.560+0000 7f2926c4d640 1 -- start start 2024-02-17T12:55:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.560+0000 7f2926c4d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29200688c0 0x7f292019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.560+0000 7f2926c4d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2920103c40 0x7f292019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.560+0000 7f2926c4d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29201060d0 0x7f29201a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.560+0000 7f2926c4d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29201a2be0 con 0x7f2920103c40 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.560+0000 7f2926c4d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29201a2d50 con 0x7f29200688c0 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.560+0000 7f2926c4d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29201a2f10 con 0x7f29201060d0 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f291ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2920103c40 0x7f292019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f291ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2920103c40 0x7f292019e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50724/0 (socket says 172.21.15.18:50724) 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f291ffff640 1 -- 172.21.15.18:0/2025044427 learned_addr learned my addr 172.21.15.18:0/2025044427 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f29249e9640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29200688c0 0x7f292019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f29251ea640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29201060d0 0x7f29201a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f291ffff640 1 -- 172.21.15.18:0/2025044427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29201060d0 msgr2=0x7f29201a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f291ffff640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29201060d0 0x7f29201a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f291ffff640 1 -- 172.21.15.18:0/2025044427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29200688c0 msgr2=0x7f292019dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f291ffff640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29200688c0 0x7f292019dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f291ffff640 1 -- 172.21.15.18:0/2025044427 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2914009490 con 0x7f2920103c40 2024-02-17T12:55:42.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f291ffff640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2920103c40 0x7f292019e2b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f291800ebc0 tx=0x7f291800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f291dffb640 1 -- 172.21.15.18:0/2025044427 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29180099b0 con 0x7f2920103c40 2024-02-17T12:55:42.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f291dffb640 1 -- 172.21.15.18:0/2025044427 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f291800eec0 con 0x7f2920103c40 2024-02-17T12:55:42.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f2926c4d640 1 -- 172.21.15.18:0/2025044427 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2910009560 con 0x7f2920103c40 2024-02-17T12:55:42.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.561+0000 7f291dffb640 1 -- 172.21.15.18:0/2025044427 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918018b60 con 0x7f2920103c40 2024-02-17T12:55:42.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.562+0000 7f2926c4d640 1 -- 172.21.15.18:0/2025044427 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29201aa890 con 0x7f2920103c40 2024-02-17T12:55:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.564+0000 7f291dffb640 1 -- 172.21.15.18:0/2025044427 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2918018cc0 con 0x7f2920103c40 2024-02-17T12:55:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.564+0000 7f292981b740 1 Processor -- start 2024-02-17T12:55:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.564+0000 7f292981b740 1 -- start start 2024-02-17T12:55:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.564+0000 7f292981b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54a250 0x55d9ae54a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.564+0000 7f292981b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae54ab70 0x55d9ae553210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.564+0000 7f292981b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae553750 0x55d9ae555b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.564+0000 7f292981b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ae505960 con 0x55d9ae553750 2024-02-17T12:55:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.564+0000 7f292981b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ae4b9310 con 0x55d9ae54a250 2024-02-17T12:55:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f292981b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ae4b6840 con 0x55d9ae54ab70 2024-02-17T12:55:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f291ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae553750 0x55d9ae555b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f29251ea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54a250 0x55d9ae54a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f291ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae553750 0x55d9ae555b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50736/0 (socket says 172.21.15.18:50736) 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f291ffff640 1 -- 172.21.15.18:0/746446755 learned_addr learned my addr 172.21.15.18:0/746446755 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f29249e9640 1 --2- 172.21.15.18:0/746446755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae54ab70 0x55d9ae553210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f291dffb640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f29080a22f0 0x7f29080a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f291ffff640 1 -- 172.21.15.18:0/746446755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae54ab70 msgr2=0x55d9ae553210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f291ffff640 1 --2- 172.21.15.18:0/746446755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae54ab70 0x55d9ae553210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f291ffff640 1 -- 172.21.15.18:0/746446755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54a250 msgr2=0x55d9ae54a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f291ffff640 1 --2- 172.21.15.18:0/746446755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54a250 0x55d9ae54a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f291ffff640 1 -- 172.21.15.18:0/746446755 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9ae474060 con 0x55d9ae553750 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f29249e9640 1 --2- 172.21.15.18:0/746446755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae54ab70 0x55d9ae553210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f29251ea640 1 --2- 172.21.15.18:0/746446755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54a250 0x55d9ae54a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f29251ea640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f29080a22f0 0x7f29080a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f291dffb640 1 -- 172.21.15.18:0/2025044427 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2918014070 con 0x7f2920103c40 2024-02-17T12:55:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f291ffff640 1 --2- 172.21.15.18:0/746446755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae553750 0x55d9ae555b40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f29180cbdd0 tx=0x7f2918024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f90d1fddb53681b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.565+0000 7f2906ffd640 1 -- 172.21.15.18:0/746446755 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291809ac70 con 0x55d9ae553750 2024-02-17T12:55:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.566+0000 7f29251ea640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f29080a22f0 0x7f29080a47b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2914014530 tx=0x7f29140046c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.566+0000 7f2906ffd640 1 -- 172.21.15.18:0/746446755 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f291809aa90 con 0x55d9ae553750 2024-02-17T12:55:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.566+0000 7f2906ffd640 1 -- 172.21.15.18:0/746446755 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291809f280 con 0x55d9ae553750 2024-02-17T12:55:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.566+0000 7f292981b740 1 -- 172.21.15.18:0/746446755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae553750 msgr2=0x55d9ae555b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.566+0000 7f292981b740 1 --2- 172.21.15.18:0/746446755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae553750 0x55d9ae555b40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f29180cbdd0 tx=0x7f2918024000 comp rx=0 tx=0).stop 2024-02-17T12:55:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.566+0000 7f292981b740 1 -- 172.21.15.18:0/746446755 shutdown_connections 2024-02-17T12:55:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.566+0000 7f292981b740 1 --2- 172.21.15.18:0/746446755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae553750 0x55d9ae555b40 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.566+0000 7f292981b740 1 --2- 172.21.15.18:0/746446755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae54ab70 0x55d9ae553210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.566+0000 7f292981b740 1 --2- 172.21.15.18:0/746446755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54a250 0x55d9ae54a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.566+0000 7f292981b740 1 -- 172.21.15.18:0/746446755 >> 172.21.15.18:0/746446755 conn(0x55d9ae544980 msgr2=0x55d9ae544d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.566+0000 7f292981b740 1 -- 172.21.15.18:0/746446755 shutdown_connections 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.566+0000 7f292981b740 1 -- 172.21.15.18:0/746446755 wait complete. 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f292981b740 1 -- start start 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f292981b740 1 Processor -- start 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f292981b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 0x55d9ae550750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f292981b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae54ab70 0x55d9ae550c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f292981b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae5511d0 0x55d9ae55f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f292981b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ae4b2020 con 0x55d9ae54a250 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f292981b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ae185430 con 0x55d9ae5511d0 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f292981b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ae141e00 con 0x55d9ae54ab70 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f292981b740 10 client.0 _ll_register_callbacks cb 0x55d9ae459cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f29249e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 0x55d9ae550750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f29249e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 0x55d9ae550750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50740/0 (socket says 172.21.15.18:50740) 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f29249e9640 1 -- 172.21.15.18:0/2651578641 learned_addr learned my addr 172.21.15.18:0/2651578641 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f29251ea640 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae5511d0 0x55d9ae55f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.567+0000 7f291ffff640 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae54ab70 0x55d9ae550c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f29249e9640 1 -- 172.21.15.18:0/2651578641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae54ab70 msgr2=0x55d9ae550c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f29249e9640 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae54ab70 0x55d9ae550c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f29249e9640 1 -- 172.21.15.18:0/2651578641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae5511d0 msgr2=0x55d9ae55f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f29249e9640 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae5511d0 0x55d9ae55f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f29249e9640 1 -- 172.21.15.18:0/2651578641 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29201aa890 con 0x55d9ae54a250 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f291ffff640 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae54ab70 0x55d9ae550c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f29249e9640 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 0x55d9ae550750 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f2910005ab0 tx=0x7f291002c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9f0d7f15e2826 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f28fffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f28fffff640 1 -- 172.21.15.18:0/2651578641 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291002cc30 con 0x55d9ae54a250 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f28fffff640 1 -- 172.21.15.18:0/2651578641 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2910045070 con 0x55d9ae54a250 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f292981b740 1 -- 172.21.15.18:0/2651578641 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9ae1b17e0 con 0x55d9ae54a250 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f28fffff640 1 -- 172.21.15.18:0/2651578641 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2910004040 con 0x55d9ae54a250 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f292981b740 10 client.17688 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f292981b740 1 -- 172.21.15.18:0/2651578641 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55d9ae4b06a0 con 0x55d9ae54a250 2024-02-17T12:55:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f28fffff640 1 -- 172.21.15.18:0/2651578641 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d9ae1b17e0 con 0x55d9ae54a250 2024-02-17T12:55:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.568+0000 7f28fffff640 1 -- 172.21.15.18:0/2651578641 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f291003c040 con 0x55d9ae54a250 2024-02-17T12:55:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.569+0000 7f292981b740 10 client.17688 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.569+0000 7f292981b740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.569+0000 7f292981b740 20 client.17688 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.569+0000 7f292981b740 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ae563ef0 0x55d9ae566310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.569+0000 7f292981b740 4 client.17688 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.569+0000 7f292981b740 1 -- 172.21.15.18:0/2651578641 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9ae1ee720 con 0x55d9ae563ef0 2024-02-17T12:55:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.569+0000 7f291ffff640 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ae563ef0 0x55d9ae566310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.569+0000 7f28fffff640 10 client.17688 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:42.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.569+0000 7f291ffff640 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ae563ef0 0x55d9ae566310 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd31c4aea80b36d8 server_cookie=d20d2dd9f64936f7 in_seq=0 out_seq=0 2024-02-17T12:55:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f8082ffd640 1 -- 172.21.15.18:0/2943191829 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5611e3453550 con 0x5611e3824910 2024-02-17T12:55:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f8082ffd640 10 client.17676 handle_command_reply: tid=0 2024-02-17T12:55:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f80ac522740 1 client.17676 shutdown 2024-02-17T12:55:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f28fffff640 1 -- 172.21.15.18:0/2651578641 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d9ae1ee720 con 0x55d9ae563ef0 2024-02-17T12:55:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f28fffff640 10 client.17688 handle_command_reply: tid=0 2024-02-17T12:55:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f292981b740 1 client.17688 shutdown 2024-02-17T12:55:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f80ac522740 1 -- 172.21.15.18:0/2943191829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37afad0 msgr2=0x5611e37c27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f80ac522740 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37afad0 0x5611e37c27d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f80980ee4a0 tx=0x7f80980b95c0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f80ac522740 1 -- 172.21.15.18:0/2943191829 shutdown_connections 2024-02-17T12:55:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f80ac522740 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5611e3824910 0x5611e3826d30 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f80ac522740 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611e37bc6f0 0x5611e3822d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f80ac522740 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5611e37afad0 0x5611e37c27d0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f80ac522740 1 --2- 172.21.15.18:0/2943191829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e37aed60 0x5611e37c2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f8082ffd640 0 client.17676 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f80ac522740 1 -- 172.21.15.18:0/2943191829 >> 172.21.15.18:0/2943191829 conn(0x5611e3776550 msgr2=0x5611e37aae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f292981b740 1 -- 172.21.15.18:0/2651578641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 msgr2=0x55d9ae550750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f80ac522740 1 -- 172.21.15.18:0/2943191829 shutdown_connections 2024-02-17T12:55:42.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f292981b740 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 0x55d9ae550750 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f2910005ab0 tx=0x7f291002c830 comp rx=0 tx=0).stop 2024-02-17T12:55:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f80ac522740 1 -- 172.21.15.18:0/2943191829 wait complete. 2024-02-17T12:55:42.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f292981b740 1 -- 172.21.15.18:0/2651578641 shutdown_connections 2024-02-17T12:55:42.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f292981b740 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ae563ef0 0x55d9ae566310 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f28fffff640 0 client.17688 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:42.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f292981b740 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae5511d0 0x55d9ae55f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f80ac522740 20 client.17676 trim_cache size 0 max 16384 2024-02-17T12:55:42.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f292981b740 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae54ab70 0x55d9ae550c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f292981b740 1 --2- 172.21.15.18:0/2651578641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 0x55d9ae550750 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f292981b740 1 -- 172.21.15.18:0/2651578641 >> 172.21.15.18:0/2651578641 conn(0x55d9ae544980 msgr2=0x55d9ae5466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:55:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:55:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:55:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:55:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f292981b740 1 -- 172.21.15.18:0/2651578641 shutdown_connections 2024-02-17T12:55:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.574+0000 7f292981b740 1 -- 172.21.15.18:0/2651578641 wait complete. 2024-02-17T12:55:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f292981b740 20 client.17688 trim_cache size 0 max 16384 2024-02-17T12:55:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f80a9cae640 1 -- 172.21.15.18:0/3933678385 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f808c0a22d0 msgr2=0x7f808c0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f80a9cae640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f808c0a22d0 0x7f808c0a4790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f809c012c60 tx=0x7f809c009e70 comp rx=0 tx=0).stop 2024-02-17T12:55:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f80a9cae640 1 -- 172.21.15.18:0/3933678385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40691a0 msgr2=0x7f80a4197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f80a9cae640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40691a0 0x7f80a4197b90 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8098005940 tx=0x7f8098004040 comp rx=0 tx=0).stop 2024-02-17T12:55:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f80a9cae640 1 -- 172.21.15.18:0/3933678385 shutdown_connections 2024-02-17T12:55:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f80a9cae640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f808c0a22d0 0x7f808c0a4790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f80a9cae640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a4107270 0x7f80a419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f80a9cae640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40691a0 0x7f80a4197b90 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f80a9cae640 1 --2- 172.21.15.18:0/3933678385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80a4068800 0x7f80a4197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f80a9cae640 1 -- 172.21.15.18:0/3933678385 >> 172.21.15.18:0/3933678385 conn(0x7f80a4076ed0 msgr2=0x7f80a40feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f80a9cae640 1 -- 172.21.15.18:0/3933678385 shutdown_connections 2024-02-17T12:55:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:42.575+0000 7f80a9cae640 1 -- 172.21.15.18:0/3933678385 wait complete. 2024-02-17T12:55:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.586+0000 7f292981b740 1 Processor -- start 2024-02-17T12:55:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f292981b740 1 -- start start 2024-02-17T12:55:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f292981b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 0x55d9ae54f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f292981b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54ab70 0x55d9ae54f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f292981b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae5511d0 0x55d9ae54d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f292981b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ae437130 con 0x55d9ae54a250 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f292981b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ae545bb0 con 0x55d9ae54ab70 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f292981b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ae553500 con 0x55d9ae5511d0 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f29249e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 0x55d9ae54f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f29249e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 0x55d9ae54f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50744/0 (socket says 172.21.15.18:50744) 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f29249e9640 1 -- 172.21.15.18:0/100104524 learned_addr learned my addr 172.21.15.18:0/100104524 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f291ffff640 1 --2- 172.21.15.18:0/100104524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54ab70 0x55d9ae54f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f29251ea640 1 --2- 172.21.15.18:0/100104524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae5511d0 0x55d9ae54d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f29249e9640 1 -- 172.21.15.18:0/100104524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae5511d0 msgr2=0x55d9ae54d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f29249e9640 1 --2- 172.21.15.18:0/100104524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae5511d0 0x55d9ae54d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f29249e9640 1 -- 172.21.15.18:0/100104524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54ab70 msgr2=0x55d9ae54f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f29249e9640 1 --2- 172.21.15.18:0/100104524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54ab70 0x55d9ae54f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f29249e9640 1 -- 172.21.15.18:0/100104524 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9ae44cc20 con 0x55d9ae54a250 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.587+0000 7f29249e9640 1 --2- 172.21.15.18:0/100104524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 0x55d9ae54f020 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f2910009430 tx=0x7f2910009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49495c3c3968af1d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.588+0000 7f29067fc640 1 -- 172.21.15.18:0/100104524 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291002c8a0 con 0x55d9ae54a250 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.588+0000 7f29067fc640 1 -- 172.21.15.18:0/100104524 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2910043070 con 0x55d9ae54a250 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.588+0000 7f29067fc640 1 -- 172.21.15.18:0/100104524 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910003e90 con 0x55d9ae54a250 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.588+0000 7f292981b740 1 -- 172.21.15.18:0/100104524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 msgr2=0x55d9ae54f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.588+0000 7f292981b740 1 --2- 172.21.15.18:0/100104524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 0x55d9ae54f020 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f2910009430 tx=0x7f2910009800 comp rx=0 tx=0).stop 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.588+0000 7f292981b740 1 -- 172.21.15.18:0/100104524 shutdown_connections 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.588+0000 7f292981b740 1 --2- 172.21.15.18:0/100104524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae5511d0 0x55d9ae54d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.588+0000 7f292981b740 1 --2- 172.21.15.18:0/100104524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54ab70 0x55d9ae54f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.588+0000 7f292981b740 1 --2- 172.21.15.18:0/100104524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54a250 0x55d9ae54f020 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.588+0000 7f292981b740 1 -- 172.21.15.18:0/100104524 >> 172.21.15.18:0/100104524 conn(0x55d9ae544980 msgr2=0x55d9ae55f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.588+0000 7f292981b740 1 -- 172.21.15.18:0/100104524 shutdown_connections 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.588+0000 7f292981b740 1 -- 172.21.15.18:0/100104524 wait complete. 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f292981b740 1 -- start start 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f292981b740 1 Processor -- start 2024-02-17T12:55:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f292981b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54a250 0x55d9ae55a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f292981b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54ab70 0x55d9ae558ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f292981b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae5511d0 0x55d9ae558ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f292981b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ae559530 con 0x55d9ae54ab70 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f292981b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ae5596a0 con 0x55d9ae54a250 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f292981b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ae559810 con 0x55d9ae5511d0 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f292981b740 10 client.0 _ll_register_callbacks cb 0x7f29180d1ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f291ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54ab70 0x55d9ae558ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f291ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54ab70 0x55d9ae558ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50748/0 (socket says 172.21.15.18:50748) 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f291ffff640 1 -- 172.21.15.18:0/2812672170 learned_addr learned my addr 172.21.15.18:0/2812672170 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f29249e9640 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54a250 0x55d9ae55a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f29251ea640 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae5511d0 0x55d9ae558ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f291ffff640 1 -- 172.21.15.18:0/2812672170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae5511d0 msgr2=0x55d9ae558ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f291ffff640 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae5511d0 0x55d9ae558ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f291ffff640 1 -- 172.21.15.18:0/2812672170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54a250 msgr2=0x55d9ae55a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f291ffff640 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54a250 0x55d9ae55a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f291ffff640 1 -- 172.21.15.18:0/2812672170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9ae4b06a0 con 0x55d9ae54ab70 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.589+0000 7f29251ea640 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae5511d0 0x55d9ae558ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f291ffff640 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54ab70 0x55d9ae558ab0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f2918097b00 tx=0x7f29180ce7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554785abc01505cb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f28fffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f28fffff640 1 -- 172.21.15.18:0/2812672170 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291809b830 con 0x55d9ae54ab70 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f28fffff640 1 -- 172.21.15.18:0/2812672170 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f29180a38d0 con 0x55d9ae54ab70 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f292981b740 1 -- 172.21.15.18:0/2812672170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f291809aa90 con 0x55d9ae54ab70 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f28fffff640 1 -- 172.21.15.18:0/2812672170 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f291809f280 con 0x55d9ae54ab70 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f292981b740 10 client.17694 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f292981b740 1 -- 172.21.15.18:0/2812672170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55d9ae4728c0 con 0x55d9ae54ab70 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f28fffff640 1 -- 172.21.15.18:0/2812672170 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29180b7080 con 0x55d9ae54ab70 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f28fffff640 1 -- 172.21.15.18:0/2812672170 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f2918075340 con 0x55d9ae54ab70 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f292981b740 10 client.17694 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f292981b740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f292981b740 20 client.17694 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f292981b740 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ae563ef0 0x55d9ae5bd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f292981b740 4 client.17694 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.590+0000 7f292981b740 1 -- 172.21.15.18:0/2812672170 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d9ae23d890 con 0x55d9ae563ef0 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f29249e9640 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ae563ef0 0x55d9ae5bd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f29249e9640 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ae563ef0 0x55d9ae5bd860 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb5cd549a9fe595d server_cookie=b11e02e78e2c1049 in_seq=0 out_seq=0 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f28fffff640 10 client.17694 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f28fffff640 1 -- 172.21.15.18:0/2812672170 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d9ae23d890 con 0x55d9ae563ef0 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f28fffff640 10 client.17694 handle_command_reply: tid=0 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f292981b740 1 client.17694 shutdown 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f292981b740 1 -- 172.21.15.18:0/2812672170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54ab70 msgr2=0x55d9ae558ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f292981b740 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54ab70 0x55d9ae558ab0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f2918097b00 tx=0x7f29180ce7d0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f292981b740 1 -- 172.21.15.18:0/2812672170 shutdown_connections 2024-02-17T12:55:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f292981b740 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d9ae563ef0 0x55d9ae5bd860 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f292981b740 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9ae5511d0 0x55d9ae558ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f28fffff640 0 client.17694 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f292981b740 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9ae54ab70 0x55d9ae558ab0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f292981b740 1 --2- 172.21.15.18:0/2812672170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9ae54a250 0x55d9ae55a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f292981b740 1 -- 172.21.15.18:0/2812672170 >> 172.21.15.18:0/2812672170 conn(0x55d9ae544980 msgr2=0x55d9ae54efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f292981b740 1 -- 172.21.15.18:0/2812672170 shutdown_connections 2024-02-17T12:55:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f292981b740 1 -- 172.21.15.18:0/2812672170 wait complete. 2024-02-17T12:55:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.591+0000 7f292981b740 20 client.17694 trim_cache size 0 max 16384 2024-02-17T12:55:42.590 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:55:42.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.592+0000 7f2926c4d640 1 -- 172.21.15.18:0/2025044427 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f29080a22f0 msgr2=0x7f29080a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.592+0000 7f2926c4d640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f29080a22f0 0x7f29080a47b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2914014530 tx=0x7f29140046c0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.592+0000 7f2926c4d640 1 -- 172.21.15.18:0/2025044427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2920103c40 msgr2=0x7f292019e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:42.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.592+0000 7f2926c4d640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2920103c40 0x7f292019e2b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f291800ebc0 tx=0x7f291800c6a0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.592+0000 7f2926c4d640 1 -- 172.21.15.18:0/2025044427 shutdown_connections 2024-02-17T12:55:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.592+0000 7f2926c4d640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f29080a22f0 0x7f29080a47b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.592+0000 7f2926c4d640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29201060d0 0x7f29201a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.592+0000 7f2926c4d640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2920103c40 0x7f292019e2b0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.592+0000 7f2926c4d640 1 --2- 172.21.15.18:0/2025044427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29200688c0 0x7f292019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.592+0000 7f2926c4d640 1 -- 172.21.15.18:0/2025044427 >> 172.21.15.18:0/2025044427 conn(0x7f2920077180 msgr2=0x7f2920101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.592+0000 7f2926c4d640 1 -- 172.21.15.18:0/2025044427 shutdown_connections 2024-02-17T12:55:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:42.592+0000 7f2926c4d640 1 -- 172.21.15.18:0/2025044427 wait complete. 2024-02-17T12:55:42.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:42 smithi146 ceph-mon[36454]: pgmap v178: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-02-17T12:55:42.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:42 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2443853272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:55:43.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:42 smithi018 ceph-mon[32625]: pgmap v178: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-02-17T12:55:43.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:42 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2443853272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:55:43.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:42 smithi119 ceph-mon[36188]: pgmap v178: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-02-17T12:55:43.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:42 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2443853272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:55:43.277 INFO:tasks.workunit.client.0.smithi018.stdout: 1 313847 6.30 MB/sec execute 178 sec latency 3440.497 ms 2024-02-17T12:55:43.595 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:55:43.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:55:43.595 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:55:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.726+0000 7f64393bc640 1 -- 172.21.15.18:0/707517909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6434100260 msgr2=0x7f6434100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.726+0000 7f64393bc640 1 --2- 172.21.15.18:0/707517909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6434100260 0x7f6434100640 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6424009930 tx=0x7f642402bc30 comp rx=0 tx=0).stop 2024-02-17T12:55:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.726+0000 7f64393bc640 1 -- 172.21.15.18:0/707517909 shutdown_connections 2024-02-17T12:55:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.726+0000 7f64393bc640 1 --2- 172.21.15.18:0/707517909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6434105330 0x7f643410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.726+0000 7f64393bc640 1 --2- 172.21.15.18:0/707517909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6434100c10 0x7f6434104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.726+0000 7f64393bc640 1 --2- 172.21.15.18:0/707517909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6434100260 0x7f6434100640 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.726+0000 7f64393bc640 1 -- 172.21.15.18:0/707517909 >> 172.21.15.18:0/707517909 conn(0x7f6434077180 msgr2=0x7f6434077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:43.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.726+0000 7f64393bc640 1 -- 172.21.15.18:0/707517909 shutdown_connections 2024-02-17T12:55:43.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.726+0000 7f64393bc640 1 -- 172.21.15.18:0/707517909 wait complete. 2024-02-17T12:55:43.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.726+0000 7f64393bc640 1 Processor -- start 2024-02-17T12:55:43.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.726+0000 7f64393bc640 1 -- start start 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f64393bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6434100260 0x7f643419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f64393bc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6434105330 0x7f643419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f64393bc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6434197630 0x7f6434197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f64393bc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6434198100 con 0x7f6434105330 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f64393bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6434198270 con 0x7f6434100260 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f64393bc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6434198430 con 0x7f6434197630 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f642bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6434105330 0x7f643419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f642bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6434105330 0x7f643419dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50760/0 (socket says 172.21.15.18:50760) 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f642bfff640 1 -- 172.21.15.18:0/2313893352 learned_addr learned my addr 172.21.15.18:0/2313893352 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f64337fe640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6434197630 0x7f6434197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f6432ffd640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6434100260 0x7f643419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f642bfff640 1 -- 172.21.15.18:0/2313893352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6434197630 msgr2=0x7f6434197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f642bfff640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6434197630 0x7f6434197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f642bfff640 1 -- 172.21.15.18:0/2313893352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6434100260 msgr2=0x7f643419d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f642bfff640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6434100260 0x7f643419d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f642bfff640 1 -- 172.21.15.18:0/2313893352 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6424009560 con 0x7f6434105330 2024-02-17T12:55:43.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f642bfff640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6434105330 0x7f643419dab0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f641c00e9c0 tx=0x7f641c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:43.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f6430ff9640 1 -- 172.21.15.18:0/2313893352 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c009700 con 0x7f6434105330 2024-02-17T12:55:43.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f6430ff9640 1 -- 172.21.15.18:0/2313893352 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f641c00cd80 con 0x7f6434105330 2024-02-17T12:55:43.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f64393bc640 1 -- 172.21.15.18:0/2313893352 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64341a43d0 con 0x7f6434105330 2024-02-17T12:55:43.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f6430ff9640 1 -- 172.21.15.18:0/2313893352 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c010430 con 0x7f6434105330 2024-02-17T12:55:43.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.727+0000 7f64393bc640 1 -- 172.21.15.18:0/2313893352 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64341a48b0 con 0x7f6434105330 2024-02-17T12:55:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.730+0000 7f6430ff9640 1 -- 172.21.15.18:0/2313893352 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f641c005000 con 0x7f6434105330 2024-02-17T12:55:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.730+0000 7f643bf93740 1 Processor -- start 2024-02-17T12:55:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.730+0000 7f6430ff9640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64200a2230 0x7f64200a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.730+0000 7f643bf93740 1 -- start start 2024-02-17T12:55:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.730+0000 7f64337fe640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64200a2230 0x7f64200a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.730+0000 7f6430ff9640 1 -- 172.21.15.18:0/2313893352 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f641c014070 con 0x7f6434105330 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f643bf93740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066260 0x561841066640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f643bf93740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561841066c10 0x56184106f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f643bf93740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56184106fd10 0x561841070170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f643bf93740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561840fd4f90 con 0x561841066c10 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f643bf93740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561840fd24c0 con 0x561841066260 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f643bf93740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561840fcdca0 con 0x56184106fd10 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f642bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561841066c10 0x56184106f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f642bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561841066c10 0x56184106f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50776/0 (socket says 172.21.15.18:50776) 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f642bfff640 1 -- 172.21.15.18:0/3934318876 learned_addr learned my addr 172.21.15.18:0/3934318876 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f64337fe640 1 --2- 172.21.15.18:0/3934318876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56184106fd10 0x561841070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f6432ffd640 1 --2- 172.21.15.18:0/3934318876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066260 0x561841066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f64337fe640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64200a2230 0x7f64200a46f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f642c005fd0 tx=0x7f642c009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f642bfff640 1 -- 172.21.15.18:0/3934318876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56184106fd10 msgr2=0x561841070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f642bfff640 1 --2- 172.21.15.18:0/3934318876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56184106fd10 0x561841070170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f642bfff640 1 -- 172.21.15.18:0/3934318876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066260 msgr2=0x561841066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f642bfff640 1 --2- 172.21.15.18:0/3934318876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066260 0x561841066640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f642bfff640 1 -- 172.21.15.18:0/3934318876 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561840f900b0 con 0x561841066c10 2024-02-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f642bfff640 1 --2- 172.21.15.18:0/3934318876 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561841066c10 0x56184106f740 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f641c097300 tx=0x7f641c09ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce6084850bc4039 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.731+0000 7f64297fa640 1 -- 172.21.15.18:0/3934318876 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c0a3640 con 0x561841066c10 2024-02-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.732+0000 7f64297fa640 1 -- 172.21.15.18:0/3934318876 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f641c0046c0 con 0x561841066c10 2024-02-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.732+0000 7f64297fa640 1 -- 172.21.15.18:0/3934318876 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c0052b0 con 0x561841066c10 2024-02-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.732+0000 7f643bf93740 1 -- 172.21.15.18:0/3934318876 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561841066c10 msgr2=0x56184106f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.732+0000 7f643bf93740 1 --2- 172.21.15.18:0/3934318876 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561841066c10 0x56184106f740 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f641c097300 tx=0x7f641c09ada0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.732+0000 7f643bf93740 1 -- 172.21.15.18:0/3934318876 shutdown_connections 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.732+0000 7f643bf93740 1 --2- 172.21.15.18:0/3934318876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56184106fd10 0x561841070170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.732+0000 7f643bf93740 1 --2- 172.21.15.18:0/3934318876 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561841066c10 0x56184106f740 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.732+0000 7f643bf93740 1 --2- 172.21.15.18:0/3934318876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066260 0x561841066640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.732+0000 7f643bf93740 1 -- 172.21.15.18:0/3934318876 >> 172.21.15.18:0/3934318876 conn(0x561841060840 msgr2=0x561841060c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.732+0000 7f643bf93740 1 -- 172.21.15.18:0/3934318876 shutdown_connections 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.732+0000 7f643bf93740 1 -- 172.21.15.18:0/3934318876 wait complete. 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f643bf93740 1 -- start start 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f643bf93740 1 Processor -- start 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f643bf93740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066260 0x5618410737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f643bf93740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066c10 0x561841073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f643bf93740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x561841074220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f643bf93740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561840f72f40 con 0x56184106fd10 2024-02-17T12:55:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f643bf93740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561840ca14b0 con 0x561841066260 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f643bf93740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561840f53670 con 0x561841066c10 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f643bf93740 10 client.0 _ll_register_callbacks cb 0x561840fc0f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f64337fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x561841074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f64337fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x561841074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50780/0 (socket says 172.21.15.18:50780) 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f642bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066c10 0x561841073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f64337fe640 1 -- 172.21.15.18:0/3449551172 learned_addr learned my addr 172.21.15.18:0/3449551172 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f6432ffd640 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066260 0x5618410737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f64337fe640 1 -- 172.21.15.18:0/3449551172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066c10 msgr2=0x561841073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f64337fe640 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066c10 0x561841073ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.733+0000 7f64337fe640 1 -- 172.21.15.18:0/3449551172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066260 msgr2=0x5618410737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f64337fe640 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066260 0x5618410737a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f64337fe640 1 -- 172.21.15.18:0/3449551172 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64341a48b0 con 0x56184106fd10 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f642bfff640 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066c10 0x561841073ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f6432ffd640 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066260 0x5618410737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f64337fe640 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x561841074220 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f642c0043b0 tx=0x7f642c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c207ac59836cf5e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f641a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f641a7fc640 1 -- 172.21.15.18:0/3449551172 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c021070 con 0x56184106fd10 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f641a7fc640 1 -- 172.21.15.18:0/3449551172 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f642c004b20 con 0x56184106fd10 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f643bf93740 1 -- 172.21.15.18:0/3449551172 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561840fa7f40 con 0x56184106fd10 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f641a7fc640 1 -- 172.21.15.18:0/3449551172 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f642c01c7b0 con 0x56184106fd10 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f643bf93740 10 client.17706 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f643bf93740 1 -- 172.21.15.18:0/3449551172 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561840fcc320 con 0x56184106fd10 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f641a7fc640 1 -- 172.21.15.18:0/3449551172 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f642c032080 con 0x56184106fd10 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.734+0000 7f641a7fc640 1 -- 172.21.15.18:0/3449551172 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f642c025a40 con 0x56184106fd10 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.735+0000 7f643bf93740 10 client.17706 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.735+0000 7f643bf93740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.735+0000 7f643bf93740 20 client.17706 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.735+0000 7f643bf93740 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561841084370 0x561841086790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.735+0000 7f643bf93740 4 client.17706 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.735+0000 7f643bf93740 1 -- 172.21.15.18:0/3449551172 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561840fc4310 con 0x561841084370 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.735+0000 7f6432ffd640 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561841084370 0x561841086790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.735+0000 7f641a7fc640 10 client.17706 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.735+0000 7f6432ffd640 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561841084370 0x561841086790 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26905e3c05f49c33 server_cookie=3893dde4854acee2 in_seq=0 out_seq=0 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f641a7fc640 1 -- 172.21.15.18:0/3449551172 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561840fc4310 con 0x561841084370 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f641a7fc640 10 client.17706 handle_command_reply: tid=0 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f643bf93740 1 client.17706 shutdown 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f643bf93740 1 -- 172.21.15.18:0/3449551172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 msgr2=0x561841074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f643bf93740 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x561841074220 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f642c0043b0 tx=0x7f642c004ab0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f643bf93740 1 -- 172.21.15.18:0/3449551172 shutdown_connections 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f643bf93740 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561841084370 0x561841086790 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f643bf93740 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x561841074220 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f643bf93740 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066c10 0x561841073ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f643bf93740 1 --2- 172.21.15.18:0/3449551172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066260 0x5618410737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f641a7fc640 0 client.17706 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f643bf93740 1 -- 172.21.15.18:0/3449551172 >> 172.21.15.18:0/3449551172 conn(0x561841060840 msgr2=0x5618410627a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f643bf93740 1 -- 172.21.15.18:0/3449551172 shutdown_connections 2024-02-17T12:55:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f643bf93740 1 -- 172.21.15.18:0/3449551172 wait complete. 2024-02-17T12:55:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.736+0000 7f643bf93740 20 client.17706 trim_cache size 0 max 16384 2024-02-17T12:55:43.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.752+0000 7f643bf93740 1 Processor -- start 2024-02-17T12:55:43.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.752+0000 7f643bf93740 1 -- start start 2024-02-17T12:55:43.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f643bf93740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066260 0x561841075ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f643bf93740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066c10 0x561841071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f643bf93740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x561841073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f643bf93740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561841080d30 con 0x56184106fd10 2024-02-17T12:55:43.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f643bf93740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561841065690 con 0x561841066c10 2024-02-17T12:55:43.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f643bf93740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561841065ca0 con 0x561841066260 2024-02-17T12:55:43.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f64337fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x561841073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f64337fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x561841073cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50782/0 (socket says 172.21.15.18:50782) 2024-02-17T12:55:43.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f64337fe640 1 -- 172.21.15.18:0/935564002 learned_addr learned my addr 172.21.15.18:0/935564002 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f6432ffd640 1 --2- 172.21.15.18:0/935564002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066260 0x561841075ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f642bfff640 1 --2- 172.21.15.18:0/935564002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066c10 0x561841071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f64337fe640 1 -- 172.21.15.18:0/935564002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066260 msgr2=0x561841075ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f64337fe640 1 --2- 172.21.15.18:0/935564002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066260 0x561841075ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f64337fe640 1 -- 172.21.15.18:0/935564002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066c10 msgr2=0x561841071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f64337fe640 1 --2- 172.21.15.18:0/935564002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066c10 0x561841071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f64337fe640 1 -- 172.21.15.18:0/935564002 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561840f68c00 con 0x56184106fd10 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f64337fe640 1 --2- 172.21.15.18:0/935564002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x561841073cf0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f642c00a010 tx=0x7f642c003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bed30efb29cbcbd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f6428ff9640 1 -- 172.21.15.18:0/935564002 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c021040 con 0x56184106fd10 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f6428ff9640 1 -- 172.21.15.18:0/935564002 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561840fa7f40 con 0x56184106fd10 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.753+0000 7f6428ff9640 1 -- 172.21.15.18:0/935564002 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c025670 con 0x56184106fd10 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.754+0000 7f643bf93740 1 -- 172.21.15.18:0/935564002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 msgr2=0x561841073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.754+0000 7f643bf93740 1 --2- 172.21.15.18:0/935564002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x561841073cf0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f642c00a010 tx=0x7f642c003ee0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.754+0000 7f643bf93740 1 -- 172.21.15.18:0/935564002 shutdown_connections 2024-02-17T12:55:43.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.754+0000 7f643bf93740 1 --2- 172.21.15.18:0/935564002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x561841073cf0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.754+0000 7f643bf93740 1 --2- 172.21.15.18:0/935564002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066c10 0x561841071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.754+0000 7f643bf93740 1 --2- 172.21.15.18:0/935564002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066260 0x561841075ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.754+0000 7f643bf93740 1 -- 172.21.15.18:0/935564002 >> 172.21.15.18:0/935564002 conn(0x561841060840 msgr2=0x561841072070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:43.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.754+0000 7f643bf93740 1 -- 172.21.15.18:0/935564002 shutdown_connections 2024-02-17T12:55:43.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.754+0000 7f643bf93740 1 -- 172.21.15.18:0/935564002 wait complete. 2024-02-17T12:55:43.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f643bf93740 1 -- start start 2024-02-17T12:55:43.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f643bf93740 1 Processor -- start 2024-02-17T12:55:43.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f643bf93740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066260 0x561841079f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f643bf93740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066c10 0x56184107a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f643bf93740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x5618410da840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f643bf93740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618410dad80 con 0x56184106fd10 2024-02-17T12:55:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f643bf93740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618410daec0 con 0x561841066c10 2024-02-17T12:55:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f643bf93740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618410db030 con 0x561841066260 2024-02-17T12:55:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f643bf93740 10 client.0 _ll_register_callbacks cb 0x7f641c00cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f64337fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x5618410da840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f642bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066c10 0x56184107a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f64337fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x5618410da840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50794/0 (socket says 172.21.15.18:50794) 2024-02-17T12:55:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f642bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066c10 0x56184107a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:53212/0 (socket says 172.21.15.18:53212) 2024-02-17T12:55:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f64337fe640 1 -- 172.21.15.18:0/3027263445 learned_addr learned my addr 172.21.15.18:0/3027263445 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f6432ffd640 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066260 0x561841079f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f642bfff640 1 -- 172.21.15.18:0/3027263445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066260 msgr2=0x561841079f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f642bfff640 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066260 0x561841079f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f642bfff640 1 -- 172.21.15.18:0/3027263445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 msgr2=0x5618410da840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f642bfff640 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x5618410da840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f642bfff640 1 -- 172.21.15.18:0/3027263445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561840fcc320 con 0x561841066c10 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.755+0000 7f64337fe640 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x5618410da840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f642bfff640 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066c10 0x56184107a490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f641c0cbcb0 tx=0x7f641c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cac5eee7e69e8b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f641a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f641a7fc640 1 -- 172.21.15.18:0/3027263445 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c09eff0 con 0x561841066c10 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f641a7fc640 1 -- 172.21.15.18:0/3027263445 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f641c07f070 con 0x561841066c10 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f643bf93740 1 -- 172.21.15.18:0/3027263445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561840ff3900 con 0x561841066c10 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f641a7fc640 1 -- 172.21.15.18:0/3027263445 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f641c0991a0 con 0x561841066c10 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f643bf93740 10 client.26984 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f643bf93740 1 -- 172.21.15.18:0/3027263445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561840f8e910 con 0x561841066c10 2024-02-17T12:55:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f641a7fc640 1 -- 172.21.15.18:0/3027263445 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561840ff3900 con 0x561841066c10 2024-02-17T12:55:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f641a7fc640 1 -- 172.21.15.18:0/3027263445 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f641c01f060 con 0x561841066c10 2024-02-17T12:55:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f643bf93740 10 client.26984 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f643bf93740 10 client.26984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f643bf93740 20 client.26984 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f643bf93740 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5618410dc720 0x5618410deb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f643bf93740 4 client.26984 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:55:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.756+0000 7f643bf93740 1 -- 172.21.15.18:0/3027263445 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561840ccd860 con 0x5618410dc720 2024-02-17T12:55:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.757+0000 7f6432ffd640 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5618410dc720 0x5618410deb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.757+0000 7f641a7fc640 10 client.26984 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.757+0000 7f6432ffd640 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5618410dc720 0x5618410deb40 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc86bde9723b035 server_cookie=a83bdfbec4cfe252 in_seq=0 out_seq=0 2024-02-17T12:55:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f641a7fc640 1 -- 172.21.15.18:0/3027263445 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561840ccd860 con 0x5618410dc720 2024-02-17T12:55:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f641a7fc640 10 client.26984 handle_command_reply: tid=0 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f643bf93740 1 client.26984 shutdown 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f643bf93740 1 -- 172.21.15.18:0/3027263445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066c10 msgr2=0x56184107a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f643bf93740 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066c10 0x56184107a490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f641c0cbcb0 tx=0x7f641c005880 comp rx=0 tx=0).stop 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f643bf93740 1 -- 172.21.15.18:0/3027263445 shutdown_connections 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f643bf93740 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5618410dc720 0x5618410deb40 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f643bf93740 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56184106fd10 0x5618410da840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f643bf93740 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561841066c10 0x56184107a490 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f641a7fc640 0 client.26984 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f643bf93740 1 --2- 172.21.15.18:0/3027263445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561841066260 0x561841079f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f643bf93740 1 -- 172.21.15.18:0/3027263445 >> 172.21.15.18:0/3027263445 conn(0x561841060840 msgr2=0x561841061d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f643bf93740 1 -- 172.21.15.18:0/3027263445 shutdown_connections 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f643bf93740 1 -- 172.21.15.18:0/3027263445 wait complete. 2024-02-17T12:55:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.758+0000 7f643bf93740 20 client.26984 trim_cache size 0 max 16384 2024-02-17T12:55:43.756 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:55:43.756 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:55:43.756 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "9d1b4b6d-63ea-412e-9343-7ebee4133304", 2024-02-17T12:55:43.756 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:55:43.756 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:55:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.759+0000 7f64393bc640 1 -- 172.21.15.18:0/2313893352 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64200a2230 msgr2=0x7f64200a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.759+0000 7f64393bc640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64200a2230 0x7f64200a46f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f642c005fd0 tx=0x7f642c009500 comp rx=0 tx=0).stop 2024-02-17T12:55:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.759+0000 7f64393bc640 1 -- 172.21.15.18:0/2313893352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6434105330 msgr2=0x7f643419dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.759+0000 7f64393bc640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6434105330 0x7f643419dab0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f641c00e9c0 tx=0x7f641c00ee80 comp rx=0 tx=0).stop 2024-02-17T12:55:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.759+0000 7f64393bc640 1 -- 172.21.15.18:0/2313893352 shutdown_connections 2024-02-17T12:55:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.759+0000 7f64393bc640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64200a2230 0x7f64200a46f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.759+0000 7f64393bc640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6434197630 0x7f6434197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.759+0000 7f64393bc640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6434105330 0x7f643419dab0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.759+0000 7f64393bc640 1 --2- 172.21.15.18:0/2313893352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6434100260 0x7f643419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.759+0000 7f64393bc640 1 -- 172.21.15.18:0/2313893352 >> 172.21.15.18:0/2313893352 conn(0x7f6434077180 msgr2=0x7f64340fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.759+0000 7f64393bc640 1 -- 172.21.15.18:0/2313893352 shutdown_connections 2024-02-17T12:55:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.759+0000 7f64393bc640 1 -- 172.21.15.18:0/2313893352 wait complete. 2024-02-17T12:55:43.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:55:43.762 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:55:43.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.880+0000 7f1371a0e640 1 -- 172.21.15.18:0/3236752025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f136c101360 msgr2=0x7f136c101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.880+0000 7f1371a0e640 1 --2- 172.21.15.18:0/3236752025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f136c101360 0x7f136c101740 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f135c009930 tx=0x7f135c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:55:43.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.880+0000 7f1371a0e640 1 -- 172.21.15.18:0/3236752025 shutdown_connections 2024-02-17T12:55:43.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.880+0000 7f1371a0e640 1 --2- 172.21.15.18:0/3236752025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c106430 0x7f136c10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.880+0000 7f1371a0e640 1 --2- 172.21.15.18:0/3236752025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f136c101d10 0x7f136c105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.880+0000 7f1371a0e640 1 --2- 172.21.15.18:0/3236752025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f136c101360 0x7f136c101740 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.880+0000 7f1371a0e640 1 -- 172.21.15.18:0/3236752025 >> 172.21.15.18:0/3236752025 conn(0x7f136c0fcc20 msgr2=0x7f136c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:43.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.880+0000 7f1371a0e640 1 -- 172.21.15.18:0/3236752025 shutdown_connections 2024-02-17T12:55:43.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.881+0000 7f1371a0e640 1 -- 172.21.15.18:0/3236752025 wait complete. 2024-02-17T12:55:43.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.881+0000 7f1371a0e640 1 Processor -- start 2024-02-17T12:55:43.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.881+0000 7f1371a0e640 1 -- start start 2024-02-17T12:55:43.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.881+0000 7f1371a0e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c106430 0x7f136c19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.881+0000 7f1371a0e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f136c19fe20 0x7f136c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.881+0000 7f1371a0e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f136c199fd0 0x7f136c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.881+0000 7f1371a0e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c1a04c0 con 0x7f136c19fe20 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.881+0000 7f1371a0e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c1a67a0 con 0x7f136c106430 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.881+0000 7f1371a0e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c1a68e0 con 0x7f136c199fd0 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.881+0000 7f1363fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f136c19fe20 0x7f136c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.881+0000 7f1363fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f136c19fe20 0x7f136c199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50816/0 (socket says 172.21.15.18:50816) 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.881+0000 7f1363fff640 1 -- 172.21.15.18:0/963651681 learned_addr learned my addr 172.21.15.18:0/963651681 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f136b7fe640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f136c199fd0 0x7f136c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f136affd640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c106430 0x7f136c19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f1363fff640 1 -- 172.21.15.18:0/963651681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f136c199fd0 msgr2=0x7f136c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f1363fff640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f136c199fd0 0x7f136c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f1363fff640 1 -- 172.21.15.18:0/963651681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c106430 msgr2=0x7f136c19f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f1363fff640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c106430 0x7f136c19f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f1363fff640 1 -- 172.21.15.18:0/963651681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1364009840 con 0x7f136c19fe20 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f136affd640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c106430 0x7f136c19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f1363fff640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f136c19fe20 0x7f136c199a90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f135400ea20 tx=0x7f135400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f1370a0c640 1 -- 172.21.15.18:0/963651681 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13540096c0 con 0x7f136c19fe20 2024-02-17T12:55:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f1370a0c640 1 -- 172.21.15.18:0/963651681 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1354005500 con 0x7f136c19fe20 2024-02-17T12:55:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f1371a0e640 1 -- 172.21.15.18:0/963651681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135c009560 con 0x7f136c19fe20 2024-02-17T12:55:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f1370a0c640 1 -- 172.21.15.18:0/963651681 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354010430 con 0x7f136c19fe20 2024-02-17T12:55:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.882+0000 7f1371a0e640 1 -- 172.21.15.18:0/963651681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c1a6f10 con 0x7f136c19fe20 2024-02-17T12:55:43.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.884+0000 7f1370a0c640 1 -- 172.21.15.18:0/963651681 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1354004f80 con 0x7f136c19fe20 2024-02-17T12:55:43.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f13745e4740 1 Processor -- start 2024-02-17T12:55:43.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f13745e4740 1 -- start start 2024-02-17T12:55:43.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f1370a0c640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13580a22b0 0x7f13580a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f13745e4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7ee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f13745e4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7f7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f13745e4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 0x55aadb7f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f13745e4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aadb75d280 con 0x55aadb7ee2b0 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f13745e4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aadb75a7b0 con 0x55aadb7f77b0 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f13745e4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aadb755f90 con 0x55aadb7eebd0 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f136b7fe640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13580a22b0 0x7f13580a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f136affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7ee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f136affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7ee690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50826/0 (socket says 172.21.15.18:50826) 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f136affd640 1 -- 172.21.15.18:0/891870447 learned_addr learned my addr 172.21.15.18:0/891870447 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f136b7fe640 1 --2- 172.21.15.18:0/891870447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 0x55aadb7f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f1370a0c640 1 -- 172.21.15.18:0/963651681 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1354014070 con 0x7f136c19fe20 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f1363fff640 1 --2- 172.21.15.18:0/891870447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f136affd640 1 -- 172.21.15.18:0/891870447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 msgr2=0x55aadb7f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f136affd640 1 --2- 172.21.15.18:0/891870447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7f7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f136affd640 1 -- 172.21.15.18:0/891870447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 msgr2=0x55aadb7f9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f136affd640 1 --2- 172.21.15.18:0/891870447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 0x55aadb7f9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f136affd640 1 -- 172.21.15.18:0/891870447 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aadb714570 con 0x55aadb7ee2b0 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.885+0000 7f1363fff640 1 --2- 172.21.15.18:0/891870447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.886+0000 7f136b7fe640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13580a22b0 0x7f13580a4770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1364009fd0 tx=0x7f1364009ca0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.886+0000 7f136affd640 1 --2- 172.21.15.18:0/891870447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7ee690 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f136c101020 tx=0x7f135c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100abb4fa0f42596 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.886+0000 7f1361ffb640 1 -- 172.21.15.18:0/891870447 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c004af0 con 0x55aadb7ee2b0 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.886+0000 7f1361ffb640 1 -- 172.21.15.18:0/891870447 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f135c004030 con 0x55aadb7ee2b0 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.886+0000 7f1361ffb640 1 -- 172.21.15.18:0/891870447 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c005210 con 0x55aadb7ee2b0 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 -- 172.21.15.18:0/891870447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 msgr2=0x55aadb7ee690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 --2- 172.21.15.18:0/891870447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7ee690 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f136c101020 tx=0x7f135c000bc0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 -- 172.21.15.18:0/891870447 shutdown_connections 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 --2- 172.21.15.18:0/891870447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 0x55aadb7f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 --2- 172.21.15.18:0/891870447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 --2- 172.21.15.18:0/891870447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7ee690 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 -- 172.21.15.18:0/891870447 >> 172.21.15.18:0/891870447 conn(0x55aadb7e8990 msgr2=0x55aadb7e8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 -- 172.21.15.18:0/891870447 shutdown_connections 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 -- 172.21.15.18:0/891870447 wait complete. 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 -- start start 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 Processor -- start 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7f7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 0x55aadb7f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aadb429430 con 0x55aadb7ee2b0 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aadb3e5e00 con 0x55aadb7f77b0 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aadb6dafd0 con 0x55aadb7eebd0 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.887+0000 7f13745e4740 10 client.0 _ll_register_callbacks cb 0x55aadb730fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f136affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f136affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7f7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50834/0 (socket says 172.21.15.18:50834) 2024-02-17T12:55:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f136affd640 1 -- 172.21.15.18:0/2421467083 learned_addr learned my addr 172.21.15.18:0/2421467083 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f136b7fe640 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 0x55aadb7f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f1363fff640 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f136affd640 1 -- 172.21.15.18:0/2421467083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 msgr2=0x55aadb7f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f136affd640 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f136affd640 1 -- 172.21.15.18:0/2421467083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 msgr2=0x55aadb7f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f136affd640 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 0x55aadb7f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f136affd640 1 -- 172.21.15.18:0/2421467083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c10e810 con 0x55aadb7ee2b0 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f136affd640 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7f7130 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f135c001eb0 tx=0x7f135c02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af93ad33527d77 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f134affd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f134affd640 1 -- 172.21.15.18:0/2421467083 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c004af0 con 0x55aadb7ee2b0 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f134affd640 1 -- 172.21.15.18:0/2421467083 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f135c000e40 con 0x55aadb7ee2b0 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f13745e4740 1 -- 172.21.15.18:0/2421467083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aadb4557e0 con 0x55aadb7ee2b0 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f134affd640 1 -- 172.21.15.18:0/2421467083 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f135c02cc00 con 0x55aadb7ee2b0 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f13745e4740 10 client.17724 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f13745e4740 1 -- 172.21.15.18:0/2421467083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55aadb754610 con 0x55aadb7ee2b0 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.888+0000 7f134affd640 1 -- 172.21.15.18:0/2421467083 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aadb4557e0 con 0x55aadb7ee2b0 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.889+0000 7f134affd640 1 -- 172.21.15.18:0/2421467083 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f135c03c050 con 0x55aadb7ee2b0 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.889+0000 7f13745e4740 10 client.17724 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.889+0000 7f13745e4740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.889+0000 7f13745e4740 20 client.17724 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.889+0000 7f13745e4740 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aadb807540 0x55aadb809960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.889+0000 7f13745e4740 4 client.17724 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.889+0000 7f13745e4740 1 -- 172.21.15.18:0/2421467083 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aadb492720 con 0x55aadb807540 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.889+0000 7f1363fff640 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aadb807540 0x55aadb809960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.889+0000 7f134affd640 10 client.17724 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.889+0000 7f1363fff640 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aadb807540 0x55aadb809960 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a6b1eb4d3af9733 server_cookie=5e594bab778a0973 in_seq=0 out_seq=0 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f134affd640 1 -- 172.21.15.18:0/2421467083 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aadb492720 con 0x55aadb807540 2024-02-17T12:55:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f134affd640 10 client.17724 handle_command_reply: tid=0 2024-02-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f13745e4740 1 client.17724 shutdown 2024-02-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f13745e4740 1 -- 172.21.15.18:0/2421467083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 msgr2=0x55aadb7f7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f13745e4740 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7f7130 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f135c001eb0 tx=0x7f135c02ca10 comp rx=0 tx=0).stop 2024-02-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f13745e4740 1 -- 172.21.15.18:0/2421467083 shutdown_connections 2024-02-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f13745e4740 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aadb807540 0x55aadb809960 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f13745e4740 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 0x55aadb7f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f13745e4740 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f134affd640 0 client.17724 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f13745e4740 1 --2- 172.21.15.18:0/2421467083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7f7130 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f13745e4740 1 -- 172.21.15.18:0/2421467083 >> 172.21.15.18:0/2421467083 conn(0x55aadb7e8990 msgr2=0x55aadb7ea730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f13745e4740 1 -- 172.21.15.18:0/2421467083 shutdown_connections 2024-02-17T12:55:43.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f13745e4740 1 -- 172.21.15.18:0/2421467083 wait complete. 2024-02-17T12:55:43.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.890+0000 7f13745e4740 20 client.17724 trim_cache size 0 max 16384 2024-02-17T12:55:43.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.906+0000 7f13745e4740 1 Processor -- start 2024-02-17T12:55:43.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.906+0000 7f13745e4740 1 -- start start 2024-02-17T12:55:43.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.906+0000 7f13745e4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7ee2b0 0x55aadb7f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.906+0000 7f13745e4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7eebd0 0x55aadb7f7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.906+0000 7f13745e4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7f77b0 0x55aadb7f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.906+0000 7f13745e4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aadb492750 con 0x55aadb7eebd0 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.906+0000 7f13745e4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aadb7e9a10 con 0x55aadb7ee2b0 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.906+0000 7f13745e4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aadb7ef320 con 0x55aadb7f77b0 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f1363fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7eebd0 0x55aadb7f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f1363fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7eebd0 0x55aadb7f7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50838/0 (socket says 172.21.15.18:50838) 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f1363fff640 1 -- 172.21.15.18:0/216471582 learned_addr learned my addr 172.21.15.18:0/216471582 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f136affd640 1 --2- 172.21.15.18:0/216471582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7ee2b0 0x55aadb7f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f136b7fe640 1 --2- 172.21.15.18:0/216471582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7f77b0 0x55aadb7f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f1363fff640 1 -- 172.21.15.18:0/216471582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7f77b0 msgr2=0x55aadb7f2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f1363fff640 1 --2- 172.21.15.18:0/216471582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7f77b0 0x55aadb7f2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f1363fff640 1 -- 172.21.15.18:0/216471582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7ee2b0 msgr2=0x55aadb7f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f1363fff640 1 --2- 172.21.15.18:0/216471582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7ee2b0 0x55aadb7f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f1363fff640 1 -- 172.21.15.18:0/216471582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aadb6f0ac0 con 0x55aadb7eebd0 2024-02-17T12:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f136affd640 1 --2- 172.21.15.18:0/216471582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7ee2b0 0x55aadb7f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f1363fff640 1 --2- 172.21.15.18:0/216471582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7eebd0 0x55aadb7f7130 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f13540cbb30 tx=0x7f1354005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d277880e012fe66c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f13617fa640 1 -- 172.21.15.18:0/216471582 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135409ad70 con 0x55aadb7eebd0 2024-02-17T12:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f13617fa640 1 -- 172.21.15.18:0/216471582 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f135409e020 con 0x55aadb7eebd0 2024-02-17T12:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.907+0000 7f13617fa640 1 -- 172.21.15.18:0/216471582 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13540046c0 con 0x55aadb7eebd0 2024-02-17T12:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.908+0000 7f13745e4740 1 -- 172.21.15.18:0/216471582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7eebd0 msgr2=0x55aadb7f7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.908+0000 7f13745e4740 1 --2- 172.21.15.18:0/216471582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7eebd0 0x55aadb7f7130 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f13540cbb30 tx=0x7f1354005670 comp rx=0 tx=0).stop 2024-02-17T12:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.908+0000 7f13745e4740 1 -- 172.21.15.18:0/216471582 shutdown_connections 2024-02-17T12:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.908+0000 7f13745e4740 1 --2- 172.21.15.18:0/216471582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7f77b0 0x55aadb7f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.908+0000 7f13745e4740 1 --2- 172.21.15.18:0/216471582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7eebd0 0x55aadb7f7130 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.908+0000 7f13745e4740 1 --2- 172.21.15.18:0/216471582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7ee2b0 0x55aadb7f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.908+0000 7f13745e4740 1 -- 172.21.15.18:0/216471582 >> 172.21.15.18:0/216471582 conn(0x55aadb7e8990 msgr2=0x55aadb7ef7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.908+0000 7f13745e4740 1 -- 172.21.15.18:0/216471582 shutdown_connections 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.908+0000 7f13745e4740 1 -- 172.21.15.18:0/216471582 wait complete. 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.908+0000 7f13745e4740 1 -- start start 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.908+0000 7f13745e4740 1 Processor -- start 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f13745e4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7faf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f13745e4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7fb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f13745e4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 0x55aadb7fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f13745e4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aadb7feeb0 con 0x55aadb7ee2b0 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f13745e4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aadb85da80 con 0x55aadb7f77b0 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f13745e4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aadb85dbc0 con 0x55aadb7eebd0 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f13745e4740 10 client.0 _ll_register_callbacks cb 0x7f1354005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:43.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f136affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7faf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f136affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7faf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50854/0 (socket says 172.21.15.18:50854) 2024-02-17T12:55:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f136affd640 1 -- 172.21.15.18:0/1124805630 learned_addr learned my addr 172.21.15.18:0/1124805630 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f1363fff640 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f136b7fe640 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 0x55aadb7fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f136affd640 1 -- 172.21.15.18:0/1124805630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 msgr2=0x55aadb7fb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f136affd640 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7fb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f136affd640 1 -- 172.21.15.18:0/1124805630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 msgr2=0x55aadb7fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f136affd640 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 0x55aadb7fb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f136affd640 1 -- 172.21.15.18:0/1124805630 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c1a6f10 con 0x55aadb7ee2b0 2024-02-17T12:55:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f136affd640 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7faf40 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f135c02cce0 tx=0x7f135c02cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed255d912bb2044d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f134affd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f134affd640 1 -- 172.21.15.18:0/1124805630 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c004520 con 0x55aadb7ee2b0 2024-02-17T12:55:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f134affd640 1 -- 172.21.15.18:0/1124805630 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f135c000e40 con 0x55aadb7ee2b0 2024-02-17T12:55:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f13745e4740 1 -- 172.21.15.18:0/1124805630 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f135c004030 con 0x55aadb7ee2b0 2024-02-17T12:55:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.909+0000 7f134affd640 1 -- 172.21.15.18:0/1124805630 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f135c004df0 con 0x55aadb7ee2b0 2024-02-17T12:55:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f13745e4740 10 client.17730 fetch_fsmap learned FSMap version 38 2024-02-17T12:55:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f13745e4740 1 -- 172.21.15.18:0/1124805630 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55aadb7166a0 con 0x55aadb7ee2b0 2024-02-17T12:55:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f134affd640 1 -- 172.21.15.18:0/1124805630 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f135c067080 con 0x55aadb7ee2b0 2024-02-17T12:55:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f134affd640 1 -- 172.21.15.18:0/1124805630 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 38) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f135c03c040 con 0x55aadb7ee2b0 2024-02-17T12:55:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f13745e4740 10 client.17730 fetch_fsmap finished waiting for FSMap version 38 2024-02-17T12:55:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f13745e4740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f13745e4740 20 client.17730 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f13745e4740 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aadb85f140 0x55aadb861560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f13745e4740 4 client.17730 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:55:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f13745e4740 1 -- 172.21.15.18:0/1124805630 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aadb4e1890 con 0x55aadb85f140 2024-02-17T12:55:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f1363fff640 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aadb85f140 0x55aadb861560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f134affd640 10 client.17730 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.910+0000 7f1363fff640 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aadb85f140 0x55aadb861560 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c9e7ede60e9026 server_cookie=82eb0a3d9e30c01c in_seq=0 out_seq=0 2024-02-17T12:55:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.911+0000 7f134affd640 1 -- 172.21.15.18:0/1124805630 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55aadb4e1890 con 0x55aadb85f140 2024-02-17T12:55:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.911+0000 7f134affd640 10 client.17730 handle_command_reply: tid=0 2024-02-17T12:55:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.911+0000 7f13745e4740 1 client.17730 shutdown 2024-02-17T12:55:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.911+0000 7f13745e4740 1 -- 172.21.15.18:0/1124805630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 msgr2=0x55aadb7faf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.911+0000 7f13745e4740 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7faf40 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f135c02cce0 tx=0x7f135c02cd10 comp rx=0 tx=0).stop 2024-02-17T12:55:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.911+0000 7f13745e4740 1 -- 172.21.15.18:0/1124805630 shutdown_connections 2024-02-17T12:55:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.911+0000 7f13745e4740 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55aadb85f140 0x55aadb861560 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.911+0000 7f13745e4740 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aadb7f77b0 0x55aadb7fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.911+0000 7f13745e4740 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aadb7eebd0 0x55aadb7fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.911+0000 7f13745e4740 1 --2- 172.21.15.18:0/1124805630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aadb7ee2b0 0x55aadb7faf40 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.911+0000 7f134affd640 0 client.17730 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.911+0000 7f13745e4740 1 -- 172.21.15.18:0/1124805630 >> 172.21.15.18:0/1124805630 conn(0x55aadb7e8990 msgr2=0x55aadb7f2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f13745e4740 1 -- 172.21.15.18:0/1124805630 shutdown_connections 2024-02-17T12:55:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f13745e4740 1 -- 172.21.15.18:0/1124805630 wait complete. 2024-02-17T12:55:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f13745e4740 20 client.17730 trim_cache size 0 max 16384 2024-02-17T12:55:43.911 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:55:43.911 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:55:43.911 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:55:43.912 INFO:teuthology.orchestra.run.smithi018.stdout: "9d1b4b6d-63ea-412e-9343-7ebee4133304": { 2024-02-17T12:55:43.912 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:55:43.912 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "9d1b4b6d-63ea-412e-9343-7ebee4133304", 2024-02-17T12:55:43.912 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:55:43.912 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:55:43.912 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:55:43.912 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:55:43.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f1371a0e640 1 -- 172.21.15.18:0/963651681 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13580a22b0 msgr2=0x7f13580a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f1371a0e640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13580a22b0 0x7f13580a4770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1364009fd0 tx=0x7f1364009ca0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f1371a0e640 1 -- 172.21.15.18:0/963651681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f136c19fe20 msgr2=0x7f136c199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:43.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f1371a0e640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f136c19fe20 0x7f136c199a90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f135400ea20 tx=0x7f135400ee20 comp rx=0 tx=0).stop 2024-02-17T12:55:43.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f1371a0e640 1 -- 172.21.15.18:0/963651681 shutdown_connections 2024-02-17T12:55:43.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f1371a0e640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13580a22b0 0x7f13580a4770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f1371a0e640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f136c199fd0 0x7f136c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f1371a0e640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f136c19fe20 0x7f136c199a90 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f1371a0e640 1 --2- 172.21.15.18:0/963651681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c106430 0x7f136c19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.912+0000 7f1371a0e640 1 -- 172.21.15.18:0/963651681 >> 172.21.15.18:0/963651681 conn(0x7f136c0fcc20 msgr2=0x7f136c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.913+0000 7f1371a0e640 1 -- 172.21.15.18:0/963651681 shutdown_connections 2024-02-17T12:55:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:55:43.913+0000 7f1371a0e640 1 -- 172.21.15.18:0/963651681 wait complete. 2024-02-17T12:55:43.915 INFO:tasks.cephfs.filesystem:scrub status for tag:9d1b4b6d-63ea-412e-9343-7ebee4133304 - {'path': '/', 'tag': '9d1b4b6d-63ea-412e-9343-7ebee4133304', 'options': 'recursive,force'} 2024-02-17T12:55:44.277 INFO:tasks.workunit.client.0.smithi018.stdout: 1 315818 6.33 MB/sec execute 179 sec latency 3641.149 ms 2024-02-17T12:55:44.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: pgmap v179: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-02-17T12:55:44.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:55:44.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:44.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:44.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:44.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:44.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:44.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:44.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:44.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:55:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:44.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: pgmap v179: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-02-17T12:55:45.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:55:45.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:45.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:45.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:45.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:45.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: pgmap v179: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-02-17T12:55:45.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:55:45.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:45.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:45.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:45.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:45.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:55:45.277 INFO:tasks.workunit.client.0.smithi018.stdout: 1 317234 6.33 MB/sec execute 180 sec latency 14.424 ms 2024-02-17T12:55:46.277 INFO:tasks.workunit.client.0.smithi018.stdout: 1 317396 6.30 MB/sec execute 181 sec latency 894.758 ms 2024-02-17T12:55:46.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:46 smithi146 ceph-mon[36454]: pgmap v180: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 202 op/s 2024-02-17T12:55:47.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:46 smithi018 ceph-mon[32625]: pgmap v180: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 202 op/s 2024-02-17T12:55:47.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:46 smithi119 ceph-mon[36188]: pgmap v180: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 202 op/s 2024-02-17T12:55:47.277 INFO:tasks.workunit.client.0.smithi018.stdout: 1 317396 6.26 MB/sec execute 182 sec latency 1894.877 ms 2024-02-17T12:55:48.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 317396 6.23 MB/sec execute 183 sec latency 2894.987 ms 2024-02-17T12:55:48.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:48 smithi146 ceph-mon[36454]: pgmap v181: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-02-17T12:55:49.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:48 smithi018 ceph-mon[32625]: pgmap v181: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-02-17T12:55:49.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:48 smithi119 ceph-mon[36188]: pgmap v181: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-02-17T12:55:49.277 INFO:tasks.workunit.client.0.smithi018.stdout: 1 318281 6.21 MB/sec execute 184 sec latency 3510.043 ms 2024-02-17T12:55:50.277 INFO:tasks.workunit.client.0.smithi018.stdout: 1 319841 6.24 MB/sec execute 185 sec latency 15.568 ms 2024-02-17T12:55:50.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:50 smithi146 ceph-mon[36454]: pgmap v182: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-02-17T12:55:50.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:50 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:55:50.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:50 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:55:50.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:50 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:55:51.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:50 smithi018 ceph-mon[32625]: pgmap v182: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-02-17T12:55:51.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:50 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:55:51.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:50 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:55:51.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:50 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:55:51.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:50 smithi119 ceph-mon[36188]: pgmap v182: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-02-17T12:55:51.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:50 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:55:51.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:50 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:55:51.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:50 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:55:51.277 INFO:tasks.workunit.client.0.smithi018.stdout: 1 320938 6.23 MB/sec execute 186 sec latency 355.498 ms 2024-02-17T12:55:51.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:51 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:55:51.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:51 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:55:51.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:51 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:55:52.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:51 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:55:52.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:51 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:55:52.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:51 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:55:52.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:51 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:55:52.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:51 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:55:52.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:51 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:55:52.277 INFO:tasks.workunit.client.0.smithi018.stdout: 1 320938 6.20 MB/sec execute 187 sec latency 1355.599 ms 2024-02-17T12:55:52.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:52 smithi146 ceph-mon[36454]: pgmap v183: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-02-17T12:55:53.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:52 smithi018 ceph-mon[32625]: pgmap v183: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-02-17T12:55:53.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:52 smithi119 ceph-mon[36188]: pgmap v183: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-02-17T12:55:53.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 321961 6.20 MB/sec execute 188 sec latency 1928.746 ms 2024-02-17T12:55:53.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:53 smithi146 ceph-mon[36454]: pgmap v184: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-02-17T12:55:54.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:53 smithi018 ceph-mon[32625]: pgmap v184: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-02-17T12:55:54.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:53 smithi119 ceph-mon[36188]: pgmap v184: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-02-17T12:55:54.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 323516 6.21 MB/sec execute 189 sec latency 12.838 ms 2024-02-17T12:55:55.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324487 6.20 MB/sec execute 190 sec latency 341.679 ms 2024-02-17T12:55:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.670+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1464668506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a041013a0 msgr2=0x7f3a04101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.670+0000 7f3a0aa0e640 1 --2- 172.21.15.18:0/1464668506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a041013a0 0x7f3a04101780 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009930 tx=0x7f39f402bc30 comp rx=0 tx=0).stop 2024-02-17T12:55:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.670+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1464668506 shutdown_connections 2024-02-17T12:55:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.670+0000 7f3a0aa0e640 1 --2- 172.21.15.18:0/1464668506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a04106470 0x7f3a0410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.670+0000 7f3a0aa0e640 1 --2- 172.21.15.18:0/1464668506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a04101d50 0x7f3a04105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.670+0000 7f3a0aa0e640 1 --2- 172.21.15.18:0/1464668506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a041013a0 0x7f3a04101780 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.670+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1464668506 >> 172.21.15.18:0/1464668506 conn(0x7f3a040fcc40 msgr2=0x7f3a040ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.670+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1464668506 shutdown_connections 2024-02-17T12:55:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.670+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1464668506 wait complete. 2024-02-17T12:55:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.671+0000 7f3a0aa0e640 1 Processor -- start 2024-02-17T12:55:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.671+0000 7f3a0aa0e640 1 -- start start 2024-02-17T12:55:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.671+0000 7f3a0aa0e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a041013a0 0x7f3a04197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.671+0000 7f3a0aa0e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a04106470 0x7f3a04197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.671+0000 7f3a0aa0e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0419c020 0x7f3a0419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.671+0000 7f3a0aa0e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0419ca10 con 0x7f3a041013a0 2024-02-17T12:55:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.671+0000 7f3a0aa0e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0419cb80 con 0x7f3a0419c020 2024-02-17T12:55:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.671+0000 7f3a0aa0e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0419ccf0 con 0x7f3a04106470 2024-02-17T12:55:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.671+0000 7f3a03fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a041013a0 0x7f3a04197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.671+0000 7f3a03fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a041013a0 0x7f3a04197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43786/0 (socket says 172.21.15.18:43786) 2024-02-17T12:55:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.671+0000 7f3a03fff640 1 -- 172.21.15.18:0/1732103293 learned_addr learned my addr 172.21.15.18:0/1732103293 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.671+0000 7f3a037fe640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a04106470 0x7f3a04197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a08fab640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0419c020 0x7f3a0419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a03fff640 1 -- 172.21.15.18:0/1732103293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a04106470 msgr2=0x7f3a04197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a03fff640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a04106470 0x7f3a04197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a03fff640 1 -- 172.21.15.18:0/1732103293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0419c020 msgr2=0x7f3a0419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a03fff640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0419c020 0x7f3a0419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a03fff640 1 -- 172.21.15.18:0/1732103293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f4009560 con 0x7f3a041013a0 2024-02-17T12:55:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a08fab640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0419c020 0x7f3a0419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a03fff640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a041013a0 0x7f3a04197760 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009230 tx=0x7f39f4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a017fa640 1 -- 172.21.15.18:0/1732103293 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4004440 con 0x7f3a041013a0 2024-02-17T12:55:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a017fa640 1 -- 172.21.15.18:0/1732103293 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39f40052d0 con 0x7f3a041013a0 2024-02-17T12:55:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1732103293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a041a42d0 con 0x7f3a041013a0 2024-02-17T12:55:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a017fa640 1 -- 172.21.15.18:0/1732103293 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4003d70 con 0x7f3a041013a0 2024-02-17T12:55:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.672+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1732103293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a041a4750 con 0x7f3a041013a0 2024-02-17T12:55:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a017fa640 1 -- 172.21.15.18:0/1732103293 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f39f4005440 con 0x7f3a041013a0 2024-02-17T12:55:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a0d5de740 1 Processor -- start 2024-02-17T12:55:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a0d5de740 1 -- start start 2024-02-17T12:55:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a0d5de740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c8f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a0d5de740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c8fd60 0x561392c901e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a0d5de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c992e0 0x561392c9b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a0d5de740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561392bfe9f0 con 0x561392c90ad0 2024-02-17T12:55:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a0d5de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561392bfbf20 con 0x561392c992e0 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a0d5de740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561392bf7700 con 0x561392c8fd60 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a017fa640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f39ec0a22d0 0x7f39ec0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a08fab640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c8f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a08fab640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c8f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43790/0 (socket says 172.21.15.18:43790) 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a08fab640 1 -- 172.21.15.18:0/3218204621 learned_addr learned my addr 172.21.15.18:0/3218204621 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a03fff640 1 --2- 172.21.15.18:0/3218204621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c8fd60 0x561392c901e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a037fe640 1 --2- 172.21.15.18:0/3218204621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c992e0 0x561392c9b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.675+0000 7f3a017fa640 1 -- 172.21.15.18:0/1732103293 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f39f40f0200 con 0x7f3a041013a0 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.676+0000 7f3a08fab640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f39ec0a22d0 0x7f39ec0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.676+0000 7f3a03fff640 1 -- 172.21.15.18:0/3218204621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c992e0 msgr2=0x561392c9b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.676+0000 7f3a03fff640 1 --2- 172.21.15.18:0/3218204621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c992e0 0x561392c9b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.676+0000 7f3a03fff640 1 -- 172.21.15.18:0/3218204621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 msgr2=0x561392c8f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.676+0000 7f3a03fff640 1 --2- 172.21.15.18:0/3218204621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c8f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.676+0000 7f3a03fff640 1 -- 172.21.15.18:0/3218204621 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561392b8d7b0 con 0x561392c8fd60 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.676+0000 7f3a08fab640 1 --2- 172.21.15.18:0/3218204621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c8f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.676+0000 7f3a03fff640 1 --2- 172.21.15.18:0/3218204621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c8fd60 0x561392c901e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39f40eda50 tx=0x7f39f408fd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=239904539772a273 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.676+0000 7f3a08fab640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f39ec0a22d0 0x7f39ec0a4790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0092f0 tx=0x7f39fc004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.676+0000 7f39ea7fc640 1 -- 172.21.15.18:0/3218204621 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f40f0020 con 0x561392c8fd60 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.676+0000 7f39ea7fc640 1 -- 172.21.15.18:0/3218204621 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39f4088250 con 0x561392c8fd60 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.676+0000 7f39ea7fc640 1 -- 172.21.15.18:0/3218204621 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f40b9360 con 0x561392c8fd60 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.677+0000 7f3a0d5de740 1 -- 172.21.15.18:0/3218204621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c8fd60 msgr2=0x561392c901e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.677+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/3218204621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c8fd60 0x561392c901e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39f40eda50 tx=0x7f39f408fd60 comp rx=0 tx=0).stop 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.677+0000 7f3a0d5de740 1 -- 172.21.15.18:0/3218204621 shutdown_connections 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.677+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/3218204621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c992e0 0x561392c9b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.677+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/3218204621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c8fd60 0x561392c901e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.677+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/3218204621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c8f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.677+0000 7f3a0d5de740 1 -- 172.21.15.18:0/3218204621 >> 172.21.15.18:0/3218204621 conn(0x561392c57550 msgr2=0x561392c8aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.677+0000 7f3a0d5de740 1 -- 172.21.15.18:0/3218204621 shutdown_connections 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.677+0000 7f3a0d5de740 1 -- 172.21.15.18:0/3218204621 wait complete. 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.677+0000 7f3a0d5de740 1 -- start start 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a0d5de740 1 Processor -- start 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a0d5de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392c96260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a0d5de740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a0d5de740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c96ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a0d5de740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561392b9cbf0 con 0x561392c90ad0 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a0d5de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613928cb250 con 0x561392c8fd60 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a0d5de740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561392b7d200 con 0x561392c992e0 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a0d5de740 10 client.0 _ll_register_callbacks cb 0x561392bebd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a037fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a037fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43794/0 (socket says 172.21.15.18:43794) 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a037fe640 1 -- 172.21.15.18:0/1629822744 learned_addr learned my addr 172.21.15.18:0/1629822744 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a08fab640 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c96ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a03fff640 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392c96260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a037fe640 1 -- 172.21.15.18:0/1629822744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 msgr2=0x561392c96ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a037fe640 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c96ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a037fe640 1 -- 172.21.15.18:0/1629822744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 msgr2=0x561392c96260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a037fe640 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392c96260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a037fe640 1 -- 172.21.15.18:0/1629822744 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a041a4750 con 0x561392c90ad0 2024-02-17T12:55:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a03fff640 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392c96260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a08fab640 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c96ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:55:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f3a037fe640 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c967a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f3a0406a830 tx=0x7f39f80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4c812e3e3ba6d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.678+0000 7f39e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f39e37fe640 1 -- 172.21.15.18:0/1629822744 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8016070 con 0x561392c90ad0 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f39e37fe640 1 -- 172.21.15.18:0/1629822744 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f39f8004020 con 0x561392c90ad0 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1629822744 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613928f7600 con 0x561392c90ad0 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f39e37fe640 1 -- 172.21.15.18:0/1629822744 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f80041e0 con 0x561392c90ad0 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f3a0d5de740 10 client.17742 fetch_fsmap learned FSMap version 39 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1629822744 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561392bf5d80 con 0x561392c90ad0 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f39e37fe640 1 -- 172.21.15.18:0/1629822744 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39f8027080 con 0x561392c90ad0 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f39e37fe640 1 -- 172.21.15.18:0/1629822744 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 39) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f39f8003830 con 0x561392c90ad0 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f3a0d5de740 10 client.17742 fetch_fsmap finished waiting for FSMap version 39 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f3a0d5de740 10 client.17742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f3a0d5de740 20 client.17742 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561392ca9430 0x561392cab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f3a0d5de740 4 client.17742 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1629822744 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613929481b0 con 0x561392ca9430 2024-02-17T12:55:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.679+0000 7f3a03fff640 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561392ca9430 0x561392cab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.680+0000 7f39e37fe640 10 client.17742 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.680+0000 7f3a03fff640 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561392ca9430 0x561392cab850 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eeac9b315169059 server_cookie=fb8281957c0cc9d6 in_seq=0 out_seq=0 2024-02-17T12:55:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.680+0000 7f39e37fe640 1 -- 172.21.15.18:0/1629822744 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5613929481b0 con 0x561392ca9430 2024-02-17T12:55:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.680+0000 7f39e37fe640 10 client.17742 handle_command_reply: tid=0 2024-02-17T12:55:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.680+0000 7f3a0d5de740 1 client.17742 shutdown 2024-02-17T12:55:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.681+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1629822744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 msgr2=0x561392c967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.681+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c967a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f3a0406a830 tx=0x7f39f80118d0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.681+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1629822744 shutdown_connections 2024-02-17T12:55:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.681+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561392ca9430 0x561392cab850 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.681+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c96ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.681+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c967a0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.681+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/1629822744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392c96260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.681+0000 7f39e37fe640 0 client.17742 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.681+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1629822744 >> 172.21.15.18:0/1629822744 conn(0x561392c57550 msgr2=0x561392c8c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.681+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1629822744 shutdown_connections 2024-02-17T12:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.681+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1629822744 wait complete. 2024-02-17T12:55:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.681+0000 7f3a0d5de740 20 client.17742 trim_cache size 0 max 16384 2024-02-17T12:55:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.694+0000 7f3a0d5de740 1 Processor -- start 2024-02-17T12:55:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.694+0000 7f3a0d5de740 1 -- start start 2024-02-17T12:55:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.694+0000 7f3a0d5de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392c98990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.694+0000 7f3a0d5de740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c96b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.694+0000 7f3a0d5de740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c94b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.694+0000 7f3a0d5de740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561392c8bc80 con 0x561392c90ad0 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.694+0000 7f3a0d5de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561392c8ba50 con 0x561392c8fd60 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.694+0000 7f3a0d5de740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561392ca91f0 con 0x561392c992e0 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.694+0000 7f3a037fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c96b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.694+0000 7f3a037fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c96b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43804/0 (socket says 172.21.15.18:43804) 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.694+0000 7f3a037fe640 1 -- 172.21.15.18:0/3914619052 learned_addr learned my addr 172.21.15.18:0/3914619052 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.695+0000 7f3a08fab640 1 --2- 172.21.15.18:0/3914619052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c94b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.695+0000 7f3a03fff640 1 --2- 172.21.15.18:0/3914619052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392c98990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.695+0000 7f3a037fe640 1 -- 172.21.15.18:0/3914619052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 msgr2=0x561392c94b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.695+0000 7f3a037fe640 1 --2- 172.21.15.18:0/3914619052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c94b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.695+0000 7f3a037fe640 1 -- 172.21.15.18:0/3914619052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 msgr2=0x561392c98990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.695+0000 7f3a037fe640 1 --2- 172.21.15.18:0/3914619052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392c98990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.695+0000 7f3a037fe640 1 -- 172.21.15.18:0/3914619052 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561392c8ac60 con 0x561392c90ad0 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.695+0000 7f3a03fff640 1 --2- 172.21.15.18:0/3914619052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392c98990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.695+0000 7f3a037fe640 1 --2- 172.21.15.18:0/3914619052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c96b00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f39f8001eb0 tx=0x7f39f800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f53cabc0f0fded server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.695+0000 7f39e9ffb640 1 -- 172.21.15.18:0/3914619052 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8016040 con 0x561392c90ad0 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.695+0000 7f39e9ffb640 1 -- 172.21.15.18:0/3914619052 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5613928f7600 con 0x561392c90ad0 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.695+0000 7f39e9ffb640 1 -- 172.21.15.18:0/3914619052 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80041e0 con 0x561392c90ad0 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.696+0000 7f3a0d5de740 1 -- 172.21.15.18:0/3914619052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 msgr2=0x561392c96b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.696+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/3914619052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c96b00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f39f8001eb0 tx=0x7f39f800bfd0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.696+0000 7f3a0d5de740 1 -- 172.21.15.18:0/3914619052 shutdown_connections 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.696+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/3914619052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c94b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.696+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/3914619052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c96b00 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.696+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/3914619052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392c98990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.696+0000 7f3a0d5de740 1 -- 172.21.15.18:0/3914619052 >> 172.21.15.18:0/3914619052 conn(0x561392c57550 msgr2=0x561392cb01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.696+0000 7f3a0d5de740 1 -- 172.21.15.18:0/3914619052 shutdown_connections 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.696+0000 7f3a0d5de740 1 -- 172.21.15.18:0/3914619052 wait complete. 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.696+0000 7f3a0d5de740 1 -- start start 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.696+0000 7f3a0d5de740 1 Processor -- start 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.696+0000 7f3a0d5de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392ca0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a0d5de740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c9e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a0d5de740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c9e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a0d5de740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561392c9eef0 con 0x561392c90ad0 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a0d5de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561392c9f060 con 0x561392c8fd60 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a0d5de740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561392c9f220 con 0x561392c992e0 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a0d5de740 10 client.0 _ll_register_callbacks cb 0x7f39f800bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a037fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c9e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a037fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c9e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43820/0 (socket says 172.21.15.18:43820) 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a037fe640 1 -- 172.21.15.18:0/1697498990 learned_addr learned my addr 172.21.15.18:0/1697498990 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a03fff640 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392ca0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a08fab640 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c9e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a037fe640 1 -- 172.21.15.18:0/1697498990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 msgr2=0x561392c9e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a037fe640 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c9e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a037fe640 1 -- 172.21.15.18:0/1697498990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 msgr2=0x561392ca0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a037fe640 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392ca0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a037fe640 1 -- 172.21.15.18:0/1697498990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561392bf5d80 con 0x561392c90ad0 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a037fe640 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c9e470 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f39f8003d50 tx=0x7f39f800bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe2bcab1e186990 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f39e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f39e37fe640 1 -- 172.21.15.18:0/1697498990 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8016050 con 0x561392c90ad0 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f39e37fe640 1 -- 172.21.15.18:0/1697498990 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f39f8005850 con 0x561392c90ad0 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1697498990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39f4088250 con 0x561392c90ad0 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f39e37fe640 1 -- 172.21.15.18:0/1697498990 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f8004020 con 0x561392c90ad0 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a0d5de740 10 client.17748 fetch_fsmap learned FSMap version 39 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.697+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1697498990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561392b927b0 con 0x561392c90ad0 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.698+0000 7f39e37fe640 1 -- 172.21.15.18:0/1697498990 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39f4088250 con 0x561392c90ad0 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.698+0000 7f39e37fe640 1 -- 172.21.15.18:0/1697498990 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 39) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f39f8004660 con 0x561392c90ad0 2024-02-17T12:55:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.698+0000 7f3a0d5de740 10 client.17748 fetch_fsmap finished waiting for FSMap version 39 2024-02-17T12:55:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.698+0000 7f3a0d5de740 10 client.17748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:55:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.698+0000 7f3a0d5de740 20 client.17748 populate_metadata read hostname 'smithi018' 2024-02-17T12:55:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.698+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561392d05000 0x561392d07420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:55:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.698+0000 7f3a0d5de740 4 client.17748 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:55:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.698+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1697498990 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561392934550 con 0x561392d05000 2024-02-17T12:55:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.698+0000 7f3a03fff640 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561392d05000 0x561392d07420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:55:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.698+0000 7f39e37fe640 10 client.17748 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.698+0000 7f3a03fff640 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561392d05000 0x561392d07420 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403b96270f10a4c2 server_cookie=4395a3189bb7fa81 in_seq=0 out_seq=0 2024-02-17T12:55:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f39e37fe640 1 -- 172.21.15.18:0/1697498990 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561392934550 con 0x561392d05000 2024-02-17T12:55:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f39e37fe640 10 client.17748 handle_command_reply: tid=0 2024-02-17T12:55:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f3a0d5de740 1 client.17748 shutdown 2024-02-17T12:55:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1697498990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 msgr2=0x561392c9e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c9e470 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f39f8003d50 tx=0x7f39f800bb00 comp rx=0 tx=0).stop 2024-02-17T12:55:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1697498990 shutdown_connections 2024-02-17T12:55:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561392d05000 0x561392d07420 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561392c992e0 0x561392c9e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561392c90ad0 0x561392c9e470 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f3a0d5de740 1 --2- 172.21.15.18:0/1697498990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561392c8fd60 0x561392ca0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f39e37fe640 0 client.17748 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:55:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1697498990 >> 172.21.15.18:0/1697498990 conn(0x561392c57550 msgr2=0x561392c94af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1697498990 shutdown_connections 2024-02-17T12:55:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.734+0000 7f3a0d5de740 1 -- 172.21.15.18:0/1697498990 wait complete. 2024-02-17T12:55:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0d5de740 20 client.17748 trim_cache size 0 max 16384 2024-02-17T12:55:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:55:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:55:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:55:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:55:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1732103293 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f39ec0a22d0 msgr2=0x7f39ec0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0aa0e640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f39ec0a22d0 0x7f39ec0a4790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0092f0 tx=0x7f39fc004ab0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1732103293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a041013a0 msgr2=0x7f3a04197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:55:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0aa0e640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a041013a0 0x7f3a04197760 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009230 tx=0x7f39f4004040 comp rx=0 tx=0).stop 2024-02-17T12:55:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1732103293 shutdown_connections 2024-02-17T12:55:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0aa0e640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f39ec0a22d0 0x7f39ec0a4790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0aa0e640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0419c020 0x7f3a0419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0aa0e640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a04106470 0x7f3a04197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0aa0e640 1 --2- 172.21.15.18:0/1732103293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a041013a0 0x7f3a04197760 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:55:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1732103293 >> 172.21.15.18:0/1732103293 conn(0x7f3a040fcc40 msgr2=0x7f3a040feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:55:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1732103293 shutdown_connections 2024-02-17T12:55:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:55:55.735+0000 7f3a0aa0e640 1 -- 172.21.15.18:0/1732103293 wait complete. 2024-02-17T12:55:56.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:55 smithi119 ceph-mon[36188]: pgmap v185: 97 pgs: 97 active+clean; 116 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-02-17T12:55:56.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:55 smithi146 ceph-mon[36454]: pgmap v185: 97 pgs: 97 active+clean; 116 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-02-17T12:55:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:55 smithi018 ceph-mon[32625]: pgmap v185: 97 pgs: 97 active+clean; 116 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-02-17T12:55:56.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324487 6.17 MB/sec execute 191 sec latency 1341.759 ms 2024-02-17T12:55:57.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324487 6.14 MB/sec execute 192 sec latency 2341.879 ms 2024-02-17T12:55:58.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:57 smithi119 ceph-mon[36188]: pgmap v186: 97 pgs: 97 active+clean; 116 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-02-17T12:55:58.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:57 smithi146 ceph-mon[36454]: pgmap v186: 97 pgs: 97 active+clean; 116 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-02-17T12:55:58.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:57 smithi018 ceph-mon[32625]: pgmap v186: 97 pgs: 97 active+clean; 116 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-02-17T12:55:58.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 325785 6.14 MB/sec execute 193 sec latency 2738.109 ms 2024-02-17T12:55:59.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 327337 6.15 MB/sec execute 194 sec latency 14.007 ms 2024-02-17T12:56:00.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:55:59 smithi119 ceph-mon[36188]: pgmap v187: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-02-17T12:56:00.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:55:59 smithi146 ceph-mon[36454]: pgmap v187: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-02-17T12:56:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:55:59 smithi018 ceph-mon[32625]: pgmap v187: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-02-17T12:56:00.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 328029 6.14 MB/sec execute 195 sec latency 513.702 ms 2024-02-17T12:56:01.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 328029 6.11 MB/sec execute 196 sec latency 1513.865 ms 2024-02-17T12:56:02.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:01 smithi119 ceph-mon[36188]: pgmap v188: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-02-17T12:56:02.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:01 smithi146 ceph-mon[36454]: pgmap v188: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-02-17T12:56:02.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:01 smithi018 ceph-mon[32625]: pgmap v188: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-02-17T12:56:02.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 328029 6.08 MB/sec execute 197 sec latency 2513.914 ms 2024-02-17T12:56:03.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:02 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:56:03.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:02 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:56:03.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:02 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:56:03.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 328029 6.05 MB/sec execute 198 sec latency 3513.954 ms 2024-02-17T12:56:04.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:03 smithi119 ceph-mon[36188]: pgmap v189: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-02-17T12:56:04.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:03 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:56:04.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:03 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:56:04.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:03 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:56:04.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:03 smithi146 ceph-mon[36454]: pgmap v189: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-02-17T12:56:04.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:03 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:56:04.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:03 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:56:04.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:03 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:56:04.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:03 smithi018 ceph-mon[32625]: pgmap v189: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-02-17T12:56:04.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:03 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:56:04.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:03 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:56:04.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:03 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:56:04.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 328626 6.02 MB/sec execute 199 sec latency 4123.369 ms 2024-02-17T12:56:05.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 330285 6.06 MB/sec execute 200 sec latency 20.476 ms 2024-02-17T12:56:06.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:05 smithi119 ceph-mon[36188]: pgmap v190: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-02-17T12:56:06.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:05 smithi146 ceph-mon[36454]: pgmap v190: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-02-17T12:56:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:05 smithi018 ceph-mon[32625]: pgmap v190: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-02-17T12:56:06.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 331479 6.05 MB/sec execute 201 sec latency 29.706 ms 2024-02-17T12:56:07.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:06 smithi119 ceph-mon[36188]: osdmap e91: 12 total, 12 up, 12 in 2024-02-17T12:56:07.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:06 smithi146 ceph-mon[36454]: osdmap e91: 12 total, 12 up, 12 in 2024-02-17T12:56:07.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:06 smithi018 ceph-mon[32625]: osdmap e91: 12 total, 12 up, 12 in 2024-02-17T12:56:07.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 331577 6.02 MB/sec execute 202 sec latency 940.405 ms 2024-02-17T12:56:08.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:07 smithi119 ceph-mon[36188]: pgmap v192: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-02-17T12:56:08.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:07 smithi146 ceph-mon[36454]: pgmap v192: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-02-17T12:56:08.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:07 smithi018 ceph-mon[32625]: pgmap v192: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-02-17T12:56:08.278 INFO:tasks.workunit.client.0.smithi018.stdout: 1 332493 6.01 MB/sec execute 203 sec latency 1514.791 ms 2024-02-17T12:56:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.830+0000 7fcbaa57a640 1 -- 172.21.15.18:0/1009006285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba41002d0 msgr2=0x7fcba4100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.830+0000 7fcbaa57a640 1 --2- 172.21.15.18:0/1009006285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba41002d0 0x7fcba4100750 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009900 tx=0x7fcb9802bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.830+0000 7fcbaa57a640 1 -- 172.21.15.18:0/1009006285 shutdown_connections 2024-02-17T12:56:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.830+0000 7fcbaa57a640 1 --2- 172.21.15.18:0/1009006285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4100c90 0x7fcba410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.830+0000 7fcbaa57a640 1 --2- 172.21.15.18:0/1009006285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba41002d0 0x7fcba4100750 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.830+0000 7fcbaa57a640 1 --2- 172.21.15.18:0/1009006285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba4105020 0x7fcba4105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.830+0000 7fcbaa57a640 1 -- 172.21.15.18:0/1009006285 >> 172.21.15.18:0/1009006285 conn(0x7fcba4076ed0 msgr2=0x7fcba40772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.830+0000 7fcbaa57a640 1 -- 172.21.15.18:0/1009006285 shutdown_connections 2024-02-17T12:56:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.830+0000 7fcbaa57a640 1 -- 172.21.15.18:0/1009006285 wait complete. 2024-02-17T12:56:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.830+0000 7fcbaa57a640 1 Processor -- start 2024-02-17T12:56:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.830+0000 7fcbaa57a640 1 -- start start 2024-02-17T12:56:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcbaa57a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba41002d0 0x7fcba419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcbaa57a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4100c90 0x7fcba419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcbaa57a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba4105020 0x7fcba410bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcbaa57a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba410c640 con 0x7fcba41002d0 2024-02-17T12:56:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcbaa57a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba410c7b0 con 0x7fcba4105020 2024-02-17T12:56:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcbaa57a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba410c970 con 0x7fcba4100c90 2024-02-17T12:56:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcba3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba41002d0 0x7fcba419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcba3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba41002d0 0x7fcba419f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38916/0 (socket says 172.21.15.18:38916) 2024-02-17T12:56:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcba8b17640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba4105020 0x7fcba410bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcba3fff640 1 -- 172.21.15.18:0/2687295190 learned_addr learned my addr 172.21.15.18:0/2687295190 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcba37fe640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4100c90 0x7fcba419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcba3fff640 1 -- 172.21.15.18:0/2687295190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4100c90 msgr2=0x7fcba419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcba3fff640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4100c90 0x7fcba419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcba3fff640 1 -- 172.21.15.18:0/2687295190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba4105020 msgr2=0x7fcba410bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcba3fff640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba4105020 0x7fcba410bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcba3fff640 1 -- 172.21.15.18:0/2687295190 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb98009560 con 0x7fcba41002d0 2024-02-17T12:56:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.831+0000 7fcba3fff640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba41002d0 0x7fcba419f630 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400d950 tx=0x7fcb9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.832+0000 7fcba17fa640 1 -- 172.21.15.18:0/2687295190 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94009a30 con 0x7fcba41002d0 2024-02-17T12:56:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.832+0000 7fcbaa57a640 1 -- 172.21.15.18:0/2687295190 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba410cc50 con 0x7fcba41002d0 2024-02-17T12:56:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.832+0000 7fcba17fa640 1 -- 172.21.15.18:0/2687295190 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb94009b90 con 0x7fcba41002d0 2024-02-17T12:56:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.832+0000 7fcba17fa640 1 -- 172.21.15.18:0/2687295190 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94004a00 con 0x7fcba41002d0 2024-02-17T12:56:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.832+0000 7fcbaa57a640 1 -- 172.21.15.18:0/2687295190 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba41a6660 con 0x7fcba41002d0 2024-02-17T12:56:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.834+0000 7fcba17fa640 1 -- 172.21.15.18:0/2687295190 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcb94009d00 con 0x7fcba41002d0 2024-02-17T12:56:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.834+0000 7fcbad22e740 1 Processor -- start 2024-02-17T12:56:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.834+0000 7fcbad22e740 1 -- start start 2024-02-17T12:56:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcbad22e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e9ad0 0x55a9f06e8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcbad22e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 0x55a9f06e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcbad22e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcbad22e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f06579f0 con 0x55a9f06e8d60 2024-02-17T12:56:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcbad22e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f0654f20 con 0x55a9f06e9ad0 2024-02-17T12:56:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcbad22e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f0650700 con 0x55a9f06f22e0 2024-02-17T12:56:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 0x55a9f06e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba17fa640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb8c0a22d0 0x7fcb8c0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 0x55a9f06e91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38918/0 (socket says 172.21.15.18:38918) 2024-02-17T12:56:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba3fff640 1 -- 172.21.15.18:0/4131135403 learned_addr learned my addr 172.21.15.18:0/4131135403 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba8b17640 1 --2- 172.21.15.18:0/4131135403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e9ad0 0x55a9f06e8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba8b17640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb8c0a22d0 0x7fcb8c0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba37fe640 1 --2- 172.21.15.18:0/4131135403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba17fa640 1 -- 172.21.15.18:0/2687295190 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fcb940cda40 con 0x7fcba41002d0 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba3fff640 1 -- 172.21.15.18:0/4131135403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 msgr2=0x55a9f06f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba3fff640 1 --2- 172.21.15.18:0/4131135403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba3fff640 1 -- 172.21.15.18:0/4131135403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e9ad0 msgr2=0x55a9f06e8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba3fff640 1 --2- 172.21.15.18:0/4131135403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e9ad0 0x55a9f06e8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba3fff640 1 -- 172.21.15.18:0/4131135403 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f05e67b0 con 0x55a9f06e8d60 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.835+0000 7fcba3fff640 1 --2- 172.21.15.18:0/4131135403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 0x55a9f06e91e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fcb940cb1f0 tx=0x7fcb9409a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e07a427f762158 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcba8b17640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb8c0a22d0 0x7fcb8c0a4790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c005fd0 tx=0x7fcb9c009b00 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcb8a7fc640 1 -- 172.21.15.18:0/4131135403 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9409a430 con 0x55a9f06e8d60 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcb8a7fc640 1 -- 172.21.15.18:0/4131135403 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb9409ab30 con 0x55a9f06e8d60 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcb8a7fc640 1 -- 172.21.15.18:0/4131135403 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9409a690 con 0x55a9f06e8d60 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcbad22e740 1 -- 172.21.15.18:0/4131135403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 msgr2=0x55a9f06e91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcbad22e740 1 --2- 172.21.15.18:0/4131135403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 0x55a9f06e91e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fcb940cb1f0 tx=0x7fcb9409a330 comp rx=0 tx=0).stop 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcbad22e740 1 -- 172.21.15.18:0/4131135403 shutdown_connections 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcbad22e740 1 --2- 172.21.15.18:0/4131135403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcbad22e740 1 --2- 172.21.15.18:0/4131135403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 0x55a9f06e91e0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcbad22e740 1 --2- 172.21.15.18:0/4131135403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e9ad0 0x55a9f06e8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcbad22e740 1 -- 172.21.15.18:0/4131135403 >> 172.21.15.18:0/4131135403 conn(0x55a9f06b0550 msgr2=0x55a9f06e3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcbad22e740 1 -- 172.21.15.18:0/4131135403 shutdown_connections 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.836+0000 7fcbad22e740 1 -- 172.21.15.18:0/4131135403 wait complete. 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcbad22e740 1 -- start start 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcbad22e740 1 Processor -- start 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcbad22e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 0x55a9f06ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcbad22e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e9ad0 0x55a9f06ef910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcbad22e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06efe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcbad22e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f05f5bf0 con 0x55a9f06e8d60 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcbad22e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f0324250 con 0x55a9f06e9ad0 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcbad22e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f05d6200 con 0x55a9f06f22e0 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcbad22e740 10 client.0 _ll_register_callbacks cb 0x55a9f0644d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcba3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 0x55a9f06ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcba3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 0x55a9f06ef3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38934/0 (socket says 172.21.15.18:38934) 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcba3fff640 1 -- 172.21.15.18:0/1940176082 learned_addr learned my addr 172.21.15.18:0/1940176082 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcba37fe640 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e9ad0 0x55a9f06ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.837+0000 7fcba8b17640 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcba3fff640 1 -- 172.21.15.18:0/1940176082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 msgr2=0x55a9f06efe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcba3fff640 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06efe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcba3fff640 1 -- 172.21.15.18:0/1940176082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e9ad0 msgr2=0x55a9f06ef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcba3fff640 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e9ad0 0x55a9f06ef910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcba3fff640 1 -- 172.21.15.18:0/1940176082 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba41a6660 con 0x55a9f06e8d60 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcba3fff640 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 0x55a9f06ef3d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fcb940052f0 tx=0x7fcb94000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e123ab60556611 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcb837fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcb837fe640 1 -- 172.21.15.18:0/1940176082 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94096c80 con 0x55a9f06e8d60 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcb837fe640 1 -- 172.21.15.18:0/1940176082 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fcb9409a430 con 0x55a9f06e8d60 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcbad22e740 1 -- 172.21.15.18:0/1940176082 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9f0350600 con 0x55a9f06e8d60 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcb837fe640 1 -- 172.21.15.18:0/1940176082 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb94096aa0 con 0x55a9f06e8d60 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcbad22e740 10 client.17760 fetch_fsmap learned FSMap version 39 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcbad22e740 1 -- 172.21.15.18:0/1940176082 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a9f064ed80 con 0x55a9f06e8d60 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcb837fe640 1 -- 172.21.15.18:0/1940176082 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a9f0350600 con 0x55a9f06e8d60 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.838+0000 7fcb837fe640 1 -- 172.21.15.18:0/1940176082 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 39) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fcb94099880 con 0x55a9f06e8d60 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.839+0000 7fcbad22e740 10 client.17760 fetch_fsmap finished waiting for FSMap version 39 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.839+0000 7fcbad22e740 10 client.17760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.839+0000 7fcbad22e740 20 client.17760 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.839+0000 7fcbad22e740 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9f07023b0 0x55a9f07047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.839+0000 7fcbad22e740 4 client.17760 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.839+0000 7fcbad22e740 1 -- 172.21.15.18:0/1940176082 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9f03a11b0 con 0x55a9f07023b0 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.839+0000 7fcba37fe640 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9f07023b0 0x55a9f07047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.839+0000 7fcb837fe640 10 client.17760 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.839+0000 7fcba37fe640 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9f07023b0 0x55a9f07047d0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e8802636ca83ec server_cookie=4e6fc4fade1cb89c in_seq=0 out_seq=0 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcb837fe640 1 -- 172.21.15.18:0/1940176082 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9f03a11b0 con 0x55a9f07023b0 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcb837fe640 10 client.17760 handle_command_reply: tid=0 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcbad22e740 1 client.17760 shutdown 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcbad22e740 1 -- 172.21.15.18:0/1940176082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 msgr2=0x55a9f06ef3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcbad22e740 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 0x55a9f06ef3d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fcb940052f0 tx=0x7fcb94000bc0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcbad22e740 1 -- 172.21.15.18:0/1940176082 shutdown_connections 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcbad22e740 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9f07023b0 0x55a9f07047d0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcbad22e740 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06efe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcbad22e740 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e9ad0 0x55a9f06ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcbad22e740 1 --2- 172.21.15.18:0/1940176082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e8d60 0x55a9f06ef3d0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcbad22e740 1 -- 172.21.15.18:0/1940176082 >> 172.21.15.18:0/1940176082 conn(0x55a9f06b0550 msgr2=0x55a9f06e57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcb837fe640 0 client.17760 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcbad22e740 1 -- 172.21.15.18:0/1940176082 shutdown_connections 2024-02-17T12:56:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcbad22e740 1 -- 172.21.15.18:0/1940176082 wait complete. 2024-02-17T12:56:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.840+0000 7fcbad22e740 20 client.17760 trim_cache size 0 max 16384 2024-02-17T12:56:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.854+0000 7fcbad22e740 1 Processor -- start 2024-02-17T12:56:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.854+0000 7fcbad22e740 1 -- start start 2024-02-17T12:56:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.854+0000 7fcbad22e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e8d60 0x55a9f06f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.854+0000 7fcbad22e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 0x55a9f06f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.854+0000 7fcbad22e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.854+0000 7fcbad22e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f06e46a0 con 0x55a9f06e9ad0 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.854+0000 7fcbad22e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f06e8060 con 0x55a9f06e8d60 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.854+0000 7fcbad22e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f07099f0 con 0x55a9f06f22e0 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcba37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 0x55a9f06f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcba37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 0x55a9f06f9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38950/0 (socket says 172.21.15.18:38950) 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcba37fe640 1 -- 172.21.15.18:0/93408162 learned_addr learned my addr 172.21.15.18:0/93408162 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcba3fff640 1 --2- 172.21.15.18:0/93408162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e8d60 0x55a9f06f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcba8b17640 1 --2- 172.21.15.18:0/93408162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcba37fe640 1 -- 172.21.15.18:0/93408162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 msgr2=0x55a9f06f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcba37fe640 1 --2- 172.21.15.18:0/93408162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06f9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcba37fe640 1 -- 172.21.15.18:0/93408162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e8d60 msgr2=0x55a9f06f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcba37fe640 1 --2- 172.21.15.18:0/93408162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e8d60 0x55a9f06f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcba37fe640 1 -- 172.21.15.18:0/93408162 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f05eb7b0 con 0x55a9f06e9ad0 2024-02-17T12:56:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcba8b17640 1 --2- 172.21.15.18:0/93408162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcba37fe640 1 --2- 172.21.15.18:0/93408162 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 0x55a9f06f9040 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005ab0 tx=0x7fcb98005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9ff340b3b7fb73 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcb89ffb640 1 -- 172.21.15.18:0/93408162 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98045040 con 0x55a9f06e9ad0 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcb89ffb640 1 -- 172.21.15.18:0/93408162 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb98047040 con 0x55a9f06e9ad0 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.855+0000 7fcb89ffb640 1 -- 172.21.15.18:0/93408162 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9802dea0 con 0x55a9f06e9ad0 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.856+0000 7fcbad22e740 1 -- 172.21.15.18:0/93408162 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 msgr2=0x55a9f06f9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.856+0000 7fcbad22e740 1 --2- 172.21.15.18:0/93408162 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 0x55a9f06f9040 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005ab0 tx=0x7fcb98005a20 comp rx=0 tx=0).stop 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.856+0000 7fcbad22e740 1 -- 172.21.15.18:0/93408162 shutdown_connections 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.856+0000 7fcbad22e740 1 --2- 172.21.15.18:0/93408162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.856+0000 7fcbad22e740 1 --2- 172.21.15.18:0/93408162 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 0x55a9f06f9040 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.856+0000 7fcbad22e740 1 --2- 172.21.15.18:0/93408162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e8d60 0x55a9f06f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.856+0000 7fcbad22e740 1 -- 172.21.15.18:0/93408162 >> 172.21.15.18:0/93408162 conn(0x55a9f06b0550 msgr2=0x55a9f06e8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.856+0000 7fcbad22e740 1 -- 172.21.15.18:0/93408162 shutdown_connections 2024-02-17T12:56:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.856+0000 7fcbad22e740 1 -- 172.21.15.18:0/93408162 wait complete. 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.856+0000 7fcbad22e740 1 -- start start 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.856+0000 7fcbad22e740 1 Processor -- start 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcbad22e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e8d60 0x55a9f06f9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcbad22e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 0x55a9f06eba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcbad22e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06ebfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcbad22e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f06f97b0 con 0x55a9f06e9ad0 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcbad22e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f06f9920 con 0x55a9f06e8d60 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcbad22e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f06ec540 con 0x55a9f06f22e0 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcbad22e740 10 client.0 _ll_register_callbacks cb 0x7fcb940966d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcba37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 0x55a9f06eba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcba37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 0x55a9f06eba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38956/0 (socket says 172.21.15.18:38956) 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcba37fe640 1 -- 172.21.15.18:0/2628272895 learned_addr learned my addr 172.21.15.18:0/2628272895 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcba3fff640 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e8d60 0x55a9f06f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcba8b17640 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06ebfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcba37fe640 1 -- 172.21.15.18:0/2628272895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 msgr2=0x55a9f06ebfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcba37fe640 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06ebfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcba37fe640 1 -- 172.21.15.18:0/2628272895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e8d60 msgr2=0x55a9f06f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcba37fe640 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e8d60 0x55a9f06f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcba37fe640 1 -- 172.21.15.18:0/2628272895 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba410d860 con 0x55a9f06e9ad0 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcba3fff640 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e8d60 0x55a9f06f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcba37fe640 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 0x55a9f06eba90 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009230 tx=0x7fcb980049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6728c2d3b16875 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcb837fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.857+0000 7fcb837fe640 1 -- 172.21.15.18:0/2628272895 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98045050 con 0x55a9f06e9ad0 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcb837fe640 1 -- 172.21.15.18:0/2628272895 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fcb98004040 con 0x55a9f06e9ad0 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcbad22e740 1 -- 172.21.15.18:0/2628272895 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb9409ab30 con 0x55a9f06e9ad0 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcb837fe640 1 -- 172.21.15.18:0/2628272895 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb9802cc50 con 0x55a9f06e9ad0 2024-02-17T12:56:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcbad22e740 10 client.17766 fetch_fsmap learned FSMap version 39 2024-02-17T12:56:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcbad22e740 1 -- 172.21.15.18:0/2628272895 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a9f0611540 con 0x55a9f06e9ad0 2024-02-17T12:56:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcb837fe640 1 -- 172.21.15.18:0/2628272895 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb9409ab30 con 0x55a9f06e9ad0 2024-02-17T12:56:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcb837fe640 1 -- 172.21.15.18:0/2628272895 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 39) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fcb9803c040 con 0x55a9f06e9ad0 2024-02-17T12:56:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcbad22e740 10 client.17766 fetch_fsmap finished waiting for FSMap version 39 2024-02-17T12:56:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcbad22e740 10 client.17766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcbad22e740 20 client.17766 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcbad22e740 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9f0759a30 0x55a9f075be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcbad22e740 4 client.17766 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:56:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcbad22e740 1 -- 172.21.15.18:0/2628272895 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a9f038d550 con 0x55a9f0759a30 2024-02-17T12:56:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.858+0000 7fcba3fff640 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9f0759a30 0x55a9f075be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.859+0000 7fcb837fe640 10 client.17766 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.859+0000 7fcba3fff640 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9f0759a30 0x55a9f075be50 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e225f9b8d0adf9 server_cookie=c13757e3abfc28f1 in_seq=0 out_seq=0 2024-02-17T12:56:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcb837fe640 1 -- 172.21.15.18:0/2628272895 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a9f038d550 con 0x55a9f0759a30 2024-02-17T12:56:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcb837fe640 10 client.17766 handle_command_reply: tid=0 2024-02-17T12:56:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcbad22e740 1 client.17766 shutdown 2024-02-17T12:56:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcbad22e740 1 -- 172.21.15.18:0/2628272895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 msgr2=0x55a9f06eba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcbad22e740 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 0x55a9f06eba90 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009230 tx=0x7fcb980049a0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcbad22e740 1 -- 172.21.15.18:0/2628272895 shutdown_connections 2024-02-17T12:56:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcbad22e740 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9f0759a30 0x55a9f075be50 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcbad22e740 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9f06f22e0 0x55a9f06ebfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcb837fe640 0 client.17766 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcbad22e740 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9f06e9ad0 0x55a9f06eba90 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcbad22e740 1 --2- 172.21.15.18:0/2628272895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9f06e8d60 0x55a9f06f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcbad22e740 1 -- 172.21.15.18:0/2628272895 >> 172.21.15.18:0/2628272895 conn(0x55a9f06b0550 msgr2=0x55a9f06e3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcbad22e740 1 -- 172.21.15.18:0/2628272895 shutdown_connections 2024-02-17T12:56:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcbad22e740 1 -- 172.21.15.18:0/2628272895 wait complete. 2024-02-17T12:56:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.896+0000 7fcbad22e740 20 client.17766 trim_cache size 0 max 16384 2024-02-17T12:56:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:56:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:56:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:56:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:56:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.897+0000 7fcbaa57a640 1 -- 172.21.15.18:0/2687295190 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb8c0a22d0 msgr2=0x7fcb8c0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.897+0000 7fcbaa57a640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb8c0a22d0 0x7fcb8c0a4790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c005fd0 tx=0x7fcb9c009b00 comp rx=0 tx=0).stop 2024-02-17T12:56:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.897+0000 7fcbaa57a640 1 -- 172.21.15.18:0/2687295190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba41002d0 msgr2=0x7fcba419f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.897+0000 7fcbaa57a640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba41002d0 0x7fcba419f630 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400d950 tx=0x7fcb9400de10 comp rx=0 tx=0).stop 2024-02-17T12:56:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.897+0000 7fcbaa57a640 1 -- 172.21.15.18:0/2687295190 shutdown_connections 2024-02-17T12:56:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.897+0000 7fcbaa57a640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb8c0a22d0 0x7fcb8c0a4790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.897+0000 7fcbaa57a640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba4105020 0x7fcba410bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.897+0000 7fcbaa57a640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcba4100c90 0x7fcba419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.897+0000 7fcbaa57a640 1 --2- 172.21.15.18:0/2687295190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba41002d0 0x7fcba419f630 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.897+0000 7fcbaa57a640 1 -- 172.21.15.18:0/2687295190 >> 172.21.15.18:0/2687295190 conn(0x7fcba4076ed0 msgr2=0x7fcba4110a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.897+0000 7fcbaa57a640 1 -- 172.21.15.18:0/2687295190 shutdown_connections 2024-02-17T12:56:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:08.897+0000 7fcbaa57a640 1 -- 172.21.15.18:0/2687295190 wait complete. 2024-02-17T12:56:09.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 334095 6.03 MB/sec execute 204 sec latency 22.204 ms 2024-02-17T12:56:10.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:09 smithi119 ceph-mon[36188]: pgmap v193: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-02-17T12:56:10.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:09 smithi146 ceph-mon[36454]: pgmap v193: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-02-17T12:56:10.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:09 smithi018 ceph-mon[32625]: pgmap v193: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-02-17T12:56:10.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 335120 6.03 MB/sec execute 205 sec latency 323.736 ms 2024-02-17T12:56:11.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:10 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:56:11.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:10 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:56:11.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:10 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:56:11.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 335120 6.00 MB/sec execute 206 sec latency 1323.838 ms 2024-02-17T12:56:12.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:11 smithi119 ceph-mon[36188]: pgmap v194: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-02-17T12:56:12.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:11 smithi146 ceph-mon[36454]: pgmap v194: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-02-17T12:56:12.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:11 smithi018 ceph-mon[32625]: pgmap v194: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-02-17T12:56:12.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 335120 5.97 MB/sec execute 207 sec latency 2323.928 ms 2024-02-17T12:56:13.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 335120 5.94 MB/sec execute 208 sec latency 3324.025 ms 2024-02-17T12:56:13.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:56:13.916 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:56:14.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.044+0000 7ff4e109d640 1 -- 172.21.15.18:0/3425489125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc101d40 msgr2=0x7ff4dc105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.044+0000 7ff4e109d640 1 --2- 172.21.15.18:0/3425489125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc101d40 0x7ff4dc105d80 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009cd0 tx=0x7ff4cc02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.044+0000 7ff4e109d640 1 -- 172.21.15.18:0/3425489125 shutdown_connections 2024-02-17T12:56:14.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.044+0000 7ff4e109d640 1 --2- 172.21.15.18:0/3425489125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc106460 0x7ff4dc10e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.044+0000 7ff4e109d640 1 --2- 172.21.15.18:0/3425489125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc101d40 0x7ff4dc105d80 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.044+0000 7ff4e109d640 1 --2- 172.21.15.18:0/3425489125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4dc101390 0x7ff4dc101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.044+0000 7ff4e109d640 1 -- 172.21.15.18:0/3425489125 >> 172.21.15.18:0/3425489125 conn(0x7ff4dc0fcc30 msgr2=0x7ff4dc0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.044+0000 7ff4e109d640 1 -- 172.21.15.18:0/3425489125 shutdown_connections 2024-02-17T12:56:14.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.044+0000 7ff4e109d640 1 -- 172.21.15.18:0/3425489125 wait complete. 2024-02-17T12:56:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.045+0000 7ff4e109d640 1 Processor -- start 2024-02-17T12:56:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.045+0000 7ff4e109d640 1 -- start start 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.045+0000 7ff4e109d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc101390 0x7ff4dc19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.045+0000 7ff4e109d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc106460 0x7ff4dc19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.045+0000 7ff4e109d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4dc199810 0x7ff4dc199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.045+0000 7ff4e109d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc19a2c0 con 0x7ff4dc106460 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.045+0000 7ff4e109d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc19a430 con 0x7ff4dc101390 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.045+0000 7ff4e109d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc19a5a0 con 0x7ff4dc199810 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.046+0000 7ff4da59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc106460 0x7ff4dc19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.046+0000 7ff4da59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc106460 0x7ff4dc19fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58164/0 (socket says 172.21.15.18:58164) 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.046+0000 7ff4da59c640 1 -- 172.21.15.18:0/3087169925 learned_addr learned my addr 172.21.15.18:0/3087169925 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.046+0000 7ff4db59e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4dc199810 0x7ff4dc199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.046+0000 7ff4dad9d640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc101390 0x7ff4dc19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.046+0000 7ff4da59c640 1 -- 172.21.15.18:0/3087169925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4dc199810 msgr2=0x7ff4dc199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.046+0000 7ff4da59c640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4dc199810 0x7ff4dc199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.046+0000 7ff4da59c640 1 -- 172.21.15.18:0/3087169925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc101390 msgr2=0x7ff4dc19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.046+0000 7ff4da59c640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc101390 0x7ff4dc19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.046+0000 7ff4da59c640 1 -- 172.21.15.18:0/3087169925 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc009900 con 0x7ff4dc106460 2024-02-17T12:56:14.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.046+0000 7ff4da59c640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc106460 0x7ff4dc19fbe0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc000f80 tx=0x7ff4cc0325e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.046+0000 7ff4d3fff640 1 -- 172.21.15.18:0/3087169925 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc034e20 con 0x7ff4dc106460 2024-02-17T12:56:14.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.047+0000 7ff4d3fff640 1 -- 172.21.15.18:0/3087169925 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc003e50 con 0x7ff4dc106460 2024-02-17T12:56:14.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.047+0000 7ff4e109d640 1 -- 172.21.15.18:0/3087169925 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4dc1a64f0 con 0x7ff4dc106460 2024-02-17T12:56:14.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.047+0000 7ff4e109d640 1 -- 172.21.15.18:0/3087169925 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc1a6920 con 0x7ff4dc106460 2024-02-17T12:56:14.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.047+0000 7ff4d3fff640 1 -- 172.21.15.18:0/3087169925 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc004760 con 0x7ff4dc106460 2024-02-17T12:56:14.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.050+0000 7ff4d3fff640 1 -- 172.21.15.18:0/3087169925 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff4cc00a040 con 0x7ff4dc106460 2024-02-17T12:56:14.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.050+0000 7ff4e3c75740 1 Processor -- start 2024-02-17T12:56:14.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4e3c75740 1 -- start start 2024-02-17T12:56:14.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4e3c75740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 0x56046c70f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4e3c75740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c718270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4e3c75740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c7187b0 0x56046c71aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4e3c75740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c67e280 con 0x56046c7187b0 2024-02-17T12:56:14.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4e3c75740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c67b7b0 con 0x56046c70f2b0 2024-02-17T12:56:14.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4e3c75740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c676f90 con 0x56046c70fbd0 2024-02-17T12:56:14.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4d3fff640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4c80a22b0 0x7ff4c80a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4dad9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c718270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4db59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 0x56046c70f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4dad9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c718270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40394/0 (socket says 172.21.15.18:40394) 2024-02-17T12:56:14.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4db59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 0x56046c70f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:56166/0 (socket says 172.21.15.18:56166) 2024-02-17T12:56:14.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4dad9d640 1 -- 172.21.15.18:0/2934397211 learned_addr learned my addr 172.21.15.18:0/2934397211 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4db59e640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4c80a22b0 0x7ff4c80a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.051+0000 7ff4da59c640 1 --2- 172.21.15.18:0/2934397211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c7187b0 0x56046c71aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4d3fff640 1 -- 172.21.15.18:0/3087169925 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff4cc0f0b70 con 0x7ff4dc106460 2024-02-17T12:56:14.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4db59e640 1 -- 172.21.15.18:0/2934397211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 msgr2=0x56046c718270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4db59e640 1 --2- 172.21.15.18:0/2934397211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c718270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4db59e640 1 -- 172.21.15.18:0/2934397211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c7187b0 msgr2=0x56046c71aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4db59e640 1 --2- 172.21.15.18:0/2934397211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c7187b0 0x56046c71aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4db59e640 1 -- 172.21.15.18:0/2934397211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56046c635570 con 0x56046c70f2b0 2024-02-17T12:56:14.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4dad9d640 1 --2- 172.21.15.18:0/2934397211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c718270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4da59c640 1 --2- 172.21.15.18:0/2934397211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c7187b0 0x56046c71aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4db59e640 1 --2- 172.21.15.18:0/2934397211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 0x56046c70f690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4011910 tx=0x7ff4c4011dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2867be3fc4582999 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4db59e640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4c80a22b0 0x7ff4c80a4770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400f790 tx=0x7ff4c401e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4d17fa640 1 -- 172.21.15.18:0/2934397211 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4011270 con 0x56046c70f2b0 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4d17fa640 1 -- 172.21.15.18:0/2934397211 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4c40113d0 con 0x56046c70f2b0 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.052+0000 7ff4d17fa640 1 -- 172.21.15.18:0/2934397211 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c401de70 con 0x56046c70f2b0 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.053+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2934397211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 msgr2=0x56046c70f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.053+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2934397211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 0x56046c70f690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4011910 tx=0x7ff4c4011dd0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.053+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2934397211 shutdown_connections 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.053+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2934397211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c7187b0 0x56046c71aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.053+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2934397211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c718270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.053+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2934397211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 0x56046c70f690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.053+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2934397211 >> 172.21.15.18:0/2934397211 conn(0x56046c709990 msgr2=0x56046c709d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.053+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2934397211 shutdown_connections 2024-02-17T12:56:14.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.053+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2934397211 wait complete. 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.053+0000 7ff4e3c75740 1 -- start start 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4e3c75740 1 Processor -- start 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4e3c75740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 0x56046c718270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4e3c75740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c715600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4e3c75740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c7187b0 0x56046c715d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4e3c75740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c34a430 con 0x56046c7187b0 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4e3c75740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c306e00 con 0x56046c70f2b0 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4e3c75740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c5fbfd0 con 0x56046c70fbd0 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4e3c75740 10 client.0 _ll_register_callbacks cb 0x56046c651fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4dad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 0x56046c718270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4da59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c715600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4dad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 0x56046c718270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:56172/0 (socket says 172.21.15.18:56172) 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4dad9d640 1 -- 172.21.15.18:0/3767123023 learned_addr learned my addr 172.21.15.18:0/3767123023 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4da59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c715600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40408/0 (socket says 172.21.15.18:40408) 2024-02-17T12:56:14.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4db59e640 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c7187b0 0x56046c715d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4dad9d640 1 -- 172.21.15.18:0/3767123023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 msgr2=0x56046c715600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4dad9d640 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c715600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4dad9d640 1 -- 172.21.15.18:0/3767123023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c7187b0 msgr2=0x56046c715d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4dad9d640 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c7187b0 0x56046c715d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.054+0000 7ff4dad9d640 1 -- 172.21.15.18:0/3767123023 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c4019040 con 0x56046c70f2b0 2024-02-17T12:56:14.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.055+0000 7ff4da59c640 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c715600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.055+0000 7ff4dad9d640 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 0x56046c718270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4012720 tx=0x7ff4d4012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb826a09d04092 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.055+0000 7ff4c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:56:14.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.055+0000 7ff4c27fc640 1 -- 172.21.15.18:0/3767123023 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4017070 con 0x56046c70f2b0 2024-02-17T12:56:14.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.055+0000 7ff4c27fc640 1 -- 172.21.15.18:0/3767123023 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff4d4004050 con 0x56046c70f2b0 2024-02-17T12:56:14.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.055+0000 7ff4e3c75740 1 -- 172.21.15.18:0/3767123023 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56046c3767e0 con 0x56046c70f2b0 2024-02-17T12:56:14.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.055+0000 7ff4c27fc640 1 -- 172.21.15.18:0/3767123023 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4d4004210 con 0x56046c70f2b0 2024-02-17T12:56:14.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.055+0000 7ff4e3c75740 10 client.27041 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:14.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.055+0000 7ff4e3c75740 1 -- 172.21.15.18:0/3767123023 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56046c675610 con 0x56046c70f2b0 2024-02-17T12:56:14.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.055+0000 7ff4c27fc640 1 -- 172.21.15.18:0/3767123023 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56046c3767e0 con 0x56046c70f2b0 2024-02-17T12:56:14.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.056+0000 7ff4c27fc640 1 -- 172.21.15.18:0/3767123023 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff4d4003a60 con 0x56046c70f2b0 2024-02-17T12:56:14.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.056+0000 7ff4e3c75740 10 client.27041 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:14.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.056+0000 7ff4e3c75740 10 client.27041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:14.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.056+0000 7ff4e3c75740 20 client.27041 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:14.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.056+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56046c7289d0 0x56046c72adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.056+0000 7ff4e3c75740 4 client.27041 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.056+0000 7ff4e3c75740 1 -- 172.21.15.18:0/3767123023 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56046c3b3720 con 0x56046c7289d0 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.056+0000 7ff4da59c640 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56046c7289d0 0x56046c72adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.056+0000 7ff4da59c640 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56046c7289d0 0x56046c72adf0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29de0446b65dfe1f server_cookie=102bfe0d2fb868ff in_seq=0 out_seq=0 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.056+0000 7ff4c27fc640 10 client.27041 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4c27fc640 1 -- 172.21.15.18:0/3767123023 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56046c3b3720 con 0x56046c7289d0 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4c27fc640 10 client.27041 handle_command_reply: tid=0 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4e3c75740 1 client.27041 shutdown 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4e3c75740 1 -- 172.21.15.18:0/3767123023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 msgr2=0x56046c718270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 0x56046c718270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4012720 tx=0x7ff4d4012be0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4e3c75740 1 -- 172.21.15.18:0/3767123023 shutdown_connections 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56046c7289d0 0x56046c72adf0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c7187b0 0x56046c715d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c715600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4c27fc640 0 client.27041 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/3767123023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70f2b0 0x56046c718270 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4e3c75740 1 -- 172.21.15.18:0/3767123023 >> 172.21.15.18:0/3767123023 conn(0x56046c709990 msgr2=0x56046c70b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4e3c75740 1 -- 172.21.15.18:0/3767123023 shutdown_connections 2024-02-17T12:56:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4e3c75740 1 -- 172.21.15.18:0/3767123023 wait complete. 2024-02-17T12:56:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.057+0000 7ff4e3c75740 20 client.27041 trim_cache size 0 max 16384 2024-02-17T12:56:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.073+0000 7ff4e3c75740 1 Processor -- start 2024-02-17T12:56:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.073+0000 7ff4e3c75740 1 -- start start 2024-02-17T12:56:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4e3c75740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 0x56046c717d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4e3c75740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70fbd0 0x56046c718270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4e3c75740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c7187b0 0x56046c71f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4e3c75740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c727120 con 0x56046c70f2b0 2024-02-17T12:56:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4e3c75740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c716950 con 0x56046c70fbd0 2024-02-17T12:56:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4e3c75740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c70ef10 con 0x56046c7187b0 2024-02-17T12:56:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4dad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 0x56046c717d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4dad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 0x56046c717d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58206/0 (socket says 172.21.15.18:58206) 2024-02-17T12:56:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4dad9d640 1 -- 172.21.15.18:0/2738957782 learned_addr learned my addr 172.21.15.18:0/2738957782 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4da59c640 1 --2- 172.21.15.18:0/2738957782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70fbd0 0x56046c718270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4db59e640 1 --2- 172.21.15.18:0/2738957782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c7187b0 0x56046c71f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4dad9d640 1 -- 172.21.15.18:0/2738957782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c7187b0 msgr2=0x56046c71f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4dad9d640 1 --2- 172.21.15.18:0/2738957782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c7187b0 0x56046c71f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4dad9d640 1 -- 172.21.15.18:0/2738957782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70fbd0 msgr2=0x56046c718270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4dad9d640 1 --2- 172.21.15.18:0/2738957782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70fbd0 0x56046c718270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4dad9d640 1 -- 172.21.15.18:0/2738957782 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56046c611ac0 con 0x56046c70f2b0 2024-02-17T12:56:14.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4db59e640 1 --2- 172.21.15.18:0/2738957782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c7187b0 0x56046c71f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:14.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4dad9d640 1 --2- 172.21.15.18:0/2738957782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 0x56046c717d30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4003e90 tx=0x7ff4d40123a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f8c831b65e99a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4d0ff9640 1 -- 172.21.15.18:0/2738957782 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4017040 con 0x56046c70f2b0 2024-02-17T12:56:14.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.074+0000 7ff4d0ff9640 1 -- 172.21.15.18:0/2738957782 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4d400fe00 con 0x56046c70f2b0 2024-02-17T12:56:14.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.075+0000 7ff4d0ff9640 1 -- 172.21.15.18:0/2738957782 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40052d0 con 0x56046c70f2b0 2024-02-17T12:56:14.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.075+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2738957782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 msgr2=0x56046c717d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.075+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2738957782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 0x56046c717d30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4003e90 tx=0x7ff4d40123a0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.075+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2738957782 shutdown_connections 2024-02-17T12:56:14.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.075+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2738957782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c7187b0 0x56046c71f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.075+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2738957782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c70fbd0 0x56046c718270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.075+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2738957782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 0x56046c717d30 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.075+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2738957782 >> 172.21.15.18:0/2738957782 conn(0x56046c709990 msgr2=0x56046c72f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.075+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2738957782 shutdown_connections 2024-02-17T12:56:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.075+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2738957782 wait complete. 2024-02-17T12:56:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4e3c75740 1 -- start start 2024-02-17T12:56:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4e3c75740 1 Processor -- start 2024-02-17T12:56:14.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4e3c75740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 0x56046c71f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4e3c75740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c71fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4e3c75740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c7187b0 0x56046c71db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4e3c75740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c71e1f0 con 0x56046c70f2b0 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4e3c75740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c71e360 con 0x56046c7187b0 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4e3c75740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c71e520 con 0x56046c70fbd0 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4e3c75740 10 client.0 _ll_register_callbacks cb 0x7ff4c4009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4dad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 0x56046c71f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4dad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 0x56046c71f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58214/0 (socket says 172.21.15.18:58214) 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4dad9d640 1 -- 172.21.15.18:0/2699426521 learned_addr learned my addr 172.21.15.18:0/2699426521 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4db59e640 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c7187b0 0x56046c71db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4da59c640 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c71fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4dad9d640 1 -- 172.21.15.18:0/2699426521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 msgr2=0x56046c71fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4dad9d640 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c71fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4dad9d640 1 -- 172.21.15.18:0/2699426521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c7187b0 msgr2=0x56046c71db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.076+0000 7ff4dad9d640 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c7187b0 0x56046c71db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.077+0000 7ff4dad9d640 1 -- 172.21.15.18:0/2699426521 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56046c675610 con 0x56046c70f2b0 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.077+0000 7ff4dad9d640 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 0x56046c71f580 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4000c00 tx=0x7ff4d4009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b2224e150c308e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.077+0000 7ff4c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.077+0000 7ff4c27fc640 1 -- 172.21.15.18:0/2699426521 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4017030 con 0x56046c70f2b0 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.077+0000 7ff4c27fc640 1 -- 172.21.15.18:0/2699426521 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff4d4000e40 con 0x56046c70f2b0 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.077+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2699426521 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4c40113d0 con 0x56046c70f2b0 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.077+0000 7ff4c27fc640 1 -- 172.21.15.18:0/2699426521 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4d40039a0 con 0x56046c70f2b0 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.077+0000 7ff4e3c75740 10 client.17784 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.077+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2699426521 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56046c6376a0 con 0x56046c70f2b0 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.077+0000 7ff4c27fc640 1 -- 172.21.15.18:0/2699426521 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4d40054e0 con 0x56046c70f2b0 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4c27fc640 1 -- 172.21.15.18:0/2699426521 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff4d4010430 con 0x56046c70f2b0 2024-02-17T12:56:14.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4e3c75740 10 client.17784 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4e3c75740 10 client.17784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4e3c75740 20 client.17784 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56046c77ff20 0x56046c782340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4e3c75740 4 client.17784 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2699426521 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7ff4c40113d0 con 0x56046c77ff20 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4da59c640 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56046c77ff20 0x56046c782340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4c27fc640 10 client.17784 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4da59c640 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56046c77ff20 0x56046c782340 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693375053dc31699 server_cookie=fbda1eafeac4c359 in_seq=0 out_seq=0 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4c27fc640 1 -- 172.21.15.18:0/2699426521 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7ff4c40113d0 con 0x56046c77ff20 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4c27fc640 10 client.17784 handle_command_reply: tid=0 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.078+0000 7ff4e3c75740 1 client.17784 shutdown 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2699426521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 msgr2=0x56046c71f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 0x56046c71f580 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4000c00 tx=0x7ff4d4009e50 comp rx=0 tx=0).stop 2024-02-17T12:56:14.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2699426521 shutdown_connections 2024-02-17T12:56:14.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56046c77ff20 0x56046c782340 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56046c7187b0 0x56046c71db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56046c70fbd0 0x56046c71fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4c27fc640 0 client.17784 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:14.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e3c75740 1 --2- 172.21.15.18:0/2699426521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56046c70f2b0 0x56046c71f580 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2699426521 >> 172.21.15.18:0/2699426521 conn(0x56046c709990 msgr2=0x56046c713de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2699426521 shutdown_connections 2024-02-17T12:56:14.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e3c75740 1 -- 172.21.15.18:0/2699426521 wait complete. 2024-02-17T12:56:14.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e3c75740 20 client.17784 trim_cache size 0 max 16384 2024-02-17T12:56:14.076 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:56:14.077 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:56:14.077 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:56:14.077 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:56:14.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e109d640 1 -- 172.21.15.18:0/3087169925 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4c80a22b0 msgr2=0x7ff4c80a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e109d640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4c80a22b0 0x7ff4c80a4770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400f790 tx=0x7ff4c401e000 comp rx=0 tx=0).stop 2024-02-17T12:56:14.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e109d640 1 -- 172.21.15.18:0/3087169925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc106460 msgr2=0x7ff4dc19fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e109d640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc106460 0x7ff4dc19fbe0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc000f80 tx=0x7ff4cc0325e0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e109d640 1 -- 172.21.15.18:0/3087169925 shutdown_connections 2024-02-17T12:56:14.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e109d640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4c80a22b0 0x7ff4c80a4770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e109d640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4dc199810 0x7ff4dc199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e109d640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc106460 0x7ff4dc19fbe0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e109d640 1 --2- 172.21.15.18:0/3087169925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc101390 0x7ff4dc19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e109d640 1 -- 172.21.15.18:0/3087169925 >> 172.21.15.18:0/3087169925 conn(0x7ff4dc0fcc30 msgr2=0x7ff4dc0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.079+0000 7ff4e109d640 1 -- 172.21.15.18:0/3087169925 shutdown_connections 2024-02-17T12:56:14.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.080+0000 7ff4e109d640 1 -- 172.21.15.18:0/3087169925 wait complete. 2024-02-17T12:56:14.081 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:56:14.081 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:56:14.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:13 smithi119 ceph-mon[36188]: pgmap v195: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-02-17T12:56:14.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:13 smithi119 ceph-mon[36188]: osdmap e92: 12 total, 12 up, 12 in 2024-02-17T12:56:14.106 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:13 smithi018 ceph-mon[32625]: pgmap v195: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-02-17T12:56:14.107 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:13 smithi018 ceph-mon[32625]: osdmap e92: 12 total, 12 up, 12 in 2024-02-17T12:56:14.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.198+0000 7f053df56640 1 -- 172.21.15.18:0/3993689204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05381002f0 msgr2=0x7f05381006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.198+0000 7f053df56640 1 --2- 172.21.15.18:0/3993689204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05381002f0 0x7f05381006d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0528009930 tx=0x7f052802bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:14.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.198+0000 7f053df56640 1 -- 172.21.15.18:0/3993689204 shutdown_connections 2024-02-17T12:56:14.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.198+0000 7f053df56640 1 --2- 172.21.15.18:0/3993689204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05381053c0 0x7f053810cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.198+0000 7f053df56640 1 --2- 172.21.15.18:0/3993689204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0538100ca0 0x7f0538104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.198+0000 7f053df56640 1 --2- 172.21.15.18:0/3993689204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05381002f0 0x7f05381006d0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.198+0000 7f053df56640 1 -- 172.21.15.18:0/3993689204 >> 172.21.15.18:0/3993689204 conn(0x7f0538077180 msgr2=0x7f0538077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.198+0000 7f053df56640 1 -- 172.21.15.18:0/3993689204 shutdown_connections 2024-02-17T12:56:14.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.199+0000 7f053df56640 1 -- 172.21.15.18:0/3993689204 wait complete. 2024-02-17T12:56:14.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.199+0000 7f053df56640 1 Processor -- start 2024-02-17T12:56:14.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.199+0000 7f053df56640 1 -- start start 2024-02-17T12:56:14.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.199+0000 7f053df56640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05381002f0 0x7f053819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.199+0000 7f053df56640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05381053c0 0x7f053819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.199+0000 7f053df56640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0538197670 0x7f0538197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.199+0000 7f053df56640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0538198140 con 0x7f05381053c0 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.199+0000 7f053df56640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05381982b0 con 0x7f05381002f0 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.199+0000 7f053df56640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0538198470 con 0x7f0538197670 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0536ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05381053c0 0x7f053819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0536ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05381053c0 0x7f053819daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58234/0 (socket says 172.21.15.18:58234) 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0536ffd640 1 -- 172.21.15.18:0/861497329 learned_addr learned my addr 172.21.15.18:0/861497329 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f05377fe640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05381002f0 0x7f053819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0537fff640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0538197670 0x7f0538197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0536ffd640 1 -- 172.21.15.18:0/861497329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0538197670 msgr2=0x7f0538197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0536ffd640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0538197670 0x7f0538197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0536ffd640 1 -- 172.21.15.18:0/861497329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05381002f0 msgr2=0x7f053819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0536ffd640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05381002f0 0x7f053819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0536ffd640 1 -- 172.21.15.18:0/861497329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0528009560 con 0x7f05381053c0 2024-02-17T12:56:14.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f05377fe640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05381002f0 0x7f053819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0536ffd640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05381053c0 0x7f053819daf0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f052c00e9f0 tx=0x7f052c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0534ff9640 1 -- 172.21.15.18:0/861497329 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0096c0 con 0x7f05381053c0 2024-02-17T12:56:14.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f053df56640 1 -- 172.21.15.18:0/861497329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05381a4410 con 0x7f05381053c0 2024-02-17T12:56:14.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0534ff9640 1 -- 172.21.15.18:0/861497329 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f052c005500 con 0x7f05381053c0 2024-02-17T12:56:14.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f0534ff9640 1 -- 172.21.15.18:0/861497329 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c010430 con 0x7f05381053c0 2024-02-17T12:56:14.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.200+0000 7f053df56640 1 -- 172.21.15.18:0/861497329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05381a48f0 con 0x7f05381053c0 2024-02-17T12:56:14.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.201+0000 7f053df56640 1 -- 172.21.15.18:0/861497329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05381017c0 con 0x7f05381053c0 2024-02-17T12:56:14.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.203+0000 7f0534ff9640 1 -- 172.21.15.18:0/861497329 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f052c005000 con 0x7f05381053c0 2024-02-17T12:56:14.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.203+0000 7f0534ff9640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f05200a2230 0x7f05200a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.203+0000 7f05377fe640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f05200a2230 0x7f05200a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.203+0000 7f0534ff9640 1 -- 172.21.15.18:0/861497329 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f052c014070 con 0x7f05381053c0 2024-02-17T12:56:14.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.204+0000 7f05377fe640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f05200a2230 0x7f05200a46f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0528005ab0 tx=0x7f0528005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.207+0000 7f0534ff9640 1 -- 172.21.15.18:0/861497329 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f052c097260 con 0x7f05381053c0 2024-02-17T12:56:14.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:13 smithi146 ceph-mon[36454]: pgmap v195: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-02-17T12:56:14.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:13 smithi146 ceph-mon[36454]: osdmap e92: 12 total, 12 up, 12 in 2024-02-17T12:56:14.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 337061 5.97 MB/sec execute 209 sec latency 3523.573 ms 2024-02-17T12:56:14.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.438+0000 7f053df56640 1 -- 172.21.15.18:0/861497329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0538109600 con 0x7f05381053c0 2024-02-17T12:56:14.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.438+0000 7f0534ff9640 1 -- 172.21.15.18:0/861497329 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11400 (secure 0 0 0) 0x7f052c09a920 con 0x7f05381053c0 2024-02-17T12:56:14.429 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:56:14.429 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:56:08.885723+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:56:14.431 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 40 2024-02-17T12:56:14.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.441+0000 7f053df56640 1 -- 172.21.15.18:0/861497329 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f05200a2230 msgr2=0x7f05200a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.441+0000 7f053df56640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f05200a2230 0x7f05200a46f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0528005ab0 tx=0x7f0528005fd0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.441+0000 7f053df56640 1 -- 172.21.15.18:0/861497329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05381053c0 msgr2=0x7f053819daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.441+0000 7f053df56640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05381053c0 0x7f053819daf0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f052c00e9f0 tx=0x7f052c00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.441+0000 7f053df56640 1 -- 172.21.15.18:0/861497329 shutdown_connections 2024-02-17T12:56:14.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.441+0000 7f053df56640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f05200a2230 0x7f05200a46f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.441+0000 7f053df56640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0538197670 0x7f0538197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.441+0000 7f053df56640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05381053c0 0x7f053819daf0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.441+0000 7f053df56640 1 --2- 172.21.15.18:0/861497329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05381002f0 0x7f053819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.441+0000 7f053df56640 1 -- 172.21.15.18:0/861497329 >> 172.21.15.18:0/861497329 conn(0x7f0538077180 msgr2=0x7f05380686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.441+0000 7f053df56640 1 -- 172.21.15.18:0/861497329 shutdown_connections 2024-02-17T12:56:14.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.441+0000 7f053df56640 1 -- 172.21.15.18:0/861497329 wait complete. 2024-02-17T12:56:14.444 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:56:14.444 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:56:14.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.600+0000 7f537f8f2640 1 -- 172.21.15.18:0/1416889572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537810b7c0 msgr2=0x7f537810dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.600+0000 7f537f8f2640 1 --2- 172.21.15.18:0/1416889572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537810b7c0 0x7f537810dc30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5368009ca0 tx=0x7f536802c180 comp rx=0 tx=0).stop 2024-02-17T12:56:14.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.600+0000 7f537f8f2640 1 -- 172.21.15.18:0/1416889572 shutdown_connections 2024-02-17T12:56:14.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.600+0000 7f537f8f2640 1 --2- 172.21.15.18:0/1416889572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537810b7c0 0x7f537810dc30 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.600+0000 7f537f8f2640 1 --2- 172.21.15.18:0/1416889572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5378102d10 0x7f537810b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.600+0000 7f537f8f2640 1 --2- 172.21.15.18:0/1416889572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53781023f0 0x7f53781027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.600+0000 7f537f8f2640 1 -- 172.21.15.18:0/1416889572 >> 172.21.15.18:0/1416889572 conn(0x7f53780771e0 msgr2=0x7f53780775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.600+0000 7f537f8f2640 1 -- 172.21.15.18:0/1416889572 shutdown_connections 2024-02-17T12:56:14.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.600+0000 7f537f8f2640 1 -- 172.21.15.18:0/1416889572 wait complete. 2024-02-17T12:56:14.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537f8f2640 1 Processor -- start 2024-02-17T12:56:14.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537f8f2640 1 -- start start 2024-02-17T12:56:14.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537f8f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53781023f0 0x7f537819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537f8f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5378102d10 0x7f537819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537f8f2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537810b7c0 0x7f53781975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537f8f2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5378197c00 con 0x7f5378102d10 2024-02-17T12:56:14.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537f8f2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5378197d70 con 0x7f53781023f0 2024-02-17T12:56:14.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537f8f2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5378197f30 con 0x7f537810b7c0 2024-02-17T12:56:14.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537ce8d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5378102d10 0x7f537819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537ce8d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5378102d10 0x7f537819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58248/0 (socket says 172.21.15.18:58248) 2024-02-17T12:56:14.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537ce8d640 1 -- 172.21.15.18:0/2120357119 learned_addr learned my addr 172.21.15.18:0/2120357119 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537d68e640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53781023f0 0x7f537819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.601+0000 7f537de8f640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537810b7c0 0x7f53781975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.602+0000 7f537ce8d640 1 -- 172.21.15.18:0/2120357119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537810b7c0 msgr2=0x7f53781975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.602+0000 7f537ce8d640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537810b7c0 0x7f53781975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.602+0000 7f537ce8d640 1 -- 172.21.15.18:0/2120357119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53781023f0 msgr2=0x7f537819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.602+0000 7f537ce8d640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53781023f0 0x7f537819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.602+0000 7f537ce8d640 1 -- 172.21.15.18:0/2120357119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368009900 con 0x7f5378102d10 2024-02-17T12:56:14.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.602+0000 7f537ce8d640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5378102d10 0x7f537819d940 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f536c00e9f0 tx=0x7f536c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.602+0000 7f53767fc640 1 -- 172.21.15.18:0/2120357119 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c009700 con 0x7f5378102d10 2024-02-17T12:56:14.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.602+0000 7f53767fc640 1 -- 172.21.15.18:0/2120357119 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f536c00cd80 con 0x7f5378102d10 2024-02-17T12:56:14.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.602+0000 7f537f8f2640 1 -- 172.21.15.18:0/2120357119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5378198210 con 0x7f5378102d10 2024-02-17T12:56:14.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.602+0000 7f53767fc640 1 -- 172.21.15.18:0/2120357119 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c018920 con 0x7f5378102d10 2024-02-17T12:56:14.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.602+0000 7f537f8f2640 1 -- 172.21.15.18:0/2120357119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53781a44a0 con 0x7f5378102d10 2024-02-17T12:56:14.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.604+0000 7f53767fc640 1 -- 172.21.15.18:0/2120357119 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f536c018a80 con 0x7f5378102d10 2024-02-17T12:56:14.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f538259d740 1 Processor -- start 2024-02-17T12:56:14.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f538259d740 1 -- start start 2024-02-17T12:56:14.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f538259d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97af00 0x55839e97b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f538259d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97b820 0x55839e988620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f538259d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e988b60 0x55839e98af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f538259d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55839e936960 con 0x55839e97af00 2024-02-17T12:56:14.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f538259d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55839e8ea310 con 0x55839e988b60 2024-02-17T12:56:14.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f538259d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55839e8e7840 con 0x55839e97b820 2024-02-17T12:56:14.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f53767fc640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f53600a2230 0x7f53600a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f537de8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97af00 0x55839e97b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f537de8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97af00 0x55839e97b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58256/0 (socket says 172.21.15.18:58256) 2024-02-17T12:56:14.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f537de8f640 1 -- 172.21.15.18:0/383745335 learned_addr learned my addr 172.21.15.18:0/383745335 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f537ce8d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e988b60 0x55839e98af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f537d68e640 1 --2- 172.21.15.18:0/383745335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97b820 0x55839e988620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f537de8f640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f53600a2230 0x7f53600a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f53767fc640 1 -- 172.21.15.18:0/2120357119 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f536c014070 con 0x7f5378102d10 2024-02-17T12:56:14.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f537ce8d640 1 -- 172.21.15.18:0/383745335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97b820 msgr2=0x55839e988620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f537ce8d640 1 --2- 172.21.15.18:0/383745335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97b820 0x55839e988620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f537ce8d640 1 -- 172.21.15.18:0/383745335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97af00 msgr2=0x55839e97b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f537ce8d640 1 --2- 172.21.15.18:0/383745335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97af00 0x55839e97b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f537ce8d640 1 -- 172.21.15.18:0/383745335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55839e8a5060 con 0x55839e988b60 2024-02-17T12:56:14.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.605+0000 7f537d68e640 1 --2- 172.21.15.18:0/383745335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97b820 0x55839e988620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.606+0000 7f537de8f640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f53600a2230 0x7f53600a46f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5368032730 tx=0x7f5368005c50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.606+0000 7f537ce8d640 1 --2- 172.21.15.18:0/383745335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e988b60 0x55839e98af50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f536c0cbae0 tx=0x7f536c090500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b3868345ba1ac4f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.606+0000 7f535f7fe640 1 -- 172.21.15.18:0/383745335 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c09eed0 con 0x55839e988b60 2024-02-17T12:56:14.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.606+0000 7f535f7fe640 1 -- 172.21.15.18:0/383745335 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f536c0a3520 con 0x55839e988b60 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.606+0000 7f535f7fe640 1 -- 172.21.15.18:0/383745335 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c09b090 con 0x55839e988b60 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.606+0000 7f538259d740 1 -- 172.21.15.18:0/383745335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e988b60 msgr2=0x55839e98af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.606+0000 7f538259d740 1 --2- 172.21.15.18:0/383745335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e988b60 0x55839e98af50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f536c0cbae0 tx=0x7f536c090500 comp rx=0 tx=0).stop 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 -- 172.21.15.18:0/383745335 shutdown_connections 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 --2- 172.21.15.18:0/383745335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e988b60 0x55839e98af50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 --2- 172.21.15.18:0/383745335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97b820 0x55839e988620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 --2- 172.21.15.18:0/383745335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97af00 0x55839e97b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 -- 172.21.15.18:0/383745335 >> 172.21.15.18:0/383745335 conn(0x55839e975980 msgr2=0x55839e975d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 -- 172.21.15.18:0/383745335 shutdown_connections 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 -- 172.21.15.18:0/383745335 wait complete. 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 -- start start 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 Processor -- start 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e97af00 0x55839e985ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97b820 0x55839e985ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e988b60 0x55839e986530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55839e8e3020 con 0x55839e988b60 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55839e5b6430 con 0x55839e97af00 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55839e572e00 con 0x55839e97b820 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f538259d740 10 client.0 _ll_register_callbacks cb 0x55839e88acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.607+0000 7f537de8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e988b60 0x55839e986530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f537de8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e988b60 0x55839e986530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58270/0 (socket says 172.21.15.18:58270) 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f537d68e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e97af00 0x55839e985ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f537de8f640 1 -- 172.21.15.18:0/486834540 learned_addr learned my addr 172.21.15.18:0/486834540 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f537ce8d640 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97b820 0x55839e985ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f537de8f640 1 -- 172.21.15.18:0/486834540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97b820 msgr2=0x55839e985ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f537de8f640 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97b820 0x55839e985ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f537de8f640 1 -- 172.21.15.18:0/486834540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e97af00 msgr2=0x55839e985ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f537de8f640 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e97af00 0x55839e985ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f537de8f640 1 -- 172.21.15.18:0/486834540 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53781a44a0 con 0x55839e988b60 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f537ce8d640 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97b820 0x55839e985ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f537d68e640 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e97af00 0x55839e985ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f537de8f640 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e988b60 0x55839e986530 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f5368004af0 tx=0x7f53680095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5528d89084cbb5f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f535cff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f535cff9640 1 -- 172.21.15.18:0/486834540 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368048050 con 0x55839e988b60 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f538259d740 1 -- 172.21.15.18:0/486834540 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55839e5e27e0 con 0x55839e988b60 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f535cff9640 1 -- 172.21.15.18:0/486834540 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5368042620 con 0x55839e988b60 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f535cff9640 1 -- 172.21.15.18:0/486834540 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53680427e0 con 0x55839e988b60 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f538259d740 10 client.17802 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.608+0000 7f538259d740 1 -- 172.21.15.18:0/486834540 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55839e8e16a0 con 0x55839e988b60 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.609+0000 7f535cff9640 1 -- 172.21.15.18:0/486834540 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55839e5e27e0 con 0x55839e988b60 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.609+0000 7f535cff9640 1 -- 172.21.15.18:0/486834540 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f536804fc50 con 0x55839e988b60 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.609+0000 7f538259d740 10 client.17802 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.609+0000 7f538259d740 10 client.17802 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.609+0000 7f538259d740 20 client.17802 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:14.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.609+0000 7f538259d740 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55839e97da60 0x55839e97fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.609+0000 7f538259d740 4 client.17802 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.609+0000 7f538259d740 1 -- 172.21.15.18:0/486834540 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55839e61f720 con 0x55839e97da60 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.609+0000 7f537d68e640 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55839e97da60 0x55839e97fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.609+0000 7f537d68e640 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55839e97da60 0x55839e97fe80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ecc495a00967e540 server_cookie=c0da91209774eb34 in_seq=0 out_seq=0 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.609+0000 7f535cff9640 10 client.17802 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f535cff9640 1 -- 172.21.15.18:0/486834540 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55839e61f720 con 0x55839e97da60 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f535cff9640 10 client.17802 handle_command_reply: tid=0 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f538259d740 1 client.17802 shutdown 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f538259d740 1 -- 172.21.15.18:0/486834540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e988b60 msgr2=0x55839e986530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f538259d740 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e988b60 0x55839e986530 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f5368004af0 tx=0x7f53680095c0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f538259d740 1 -- 172.21.15.18:0/486834540 shutdown_connections 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f538259d740 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e988b60 0x55839e986530 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f538259d740 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97b820 0x55839e985ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f538259d740 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55839e97da60 0x55839e97fe80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f538259d740 1 --2- 172.21.15.18:0/486834540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e97af00 0x55839e985ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f535cff9640 0 client.17802 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:56:14.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f538259d740 1 -- 172.21.15.18:0/486834540 >> 172.21.15.18:0/486834540 conn(0x55839e975980 msgr2=0x55839e9776d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f538259d740 1 -- 172.21.15.18:0/486834540 shutdown_connections 2024-02-17T12:56:14.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f538259d740 1 -- 172.21.15.18:0/486834540 wait complete. 2024-02-17T12:56:14.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.610+0000 7f538259d740 20 client.17802 trim_cache size 0 max 16384 2024-02-17T12:56:14.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f538259d740 1 Processor -- start 2024-02-17T12:56:14.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f538259d740 1 -- start start 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f538259d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e97af00 0x55839e984c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f538259d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 0x55839e986350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f538259d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e988b60 0x55839e974a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f538259d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55839e868130 con 0x55839e97b820 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f538259d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55839e61f720 con 0x55839e97af00 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f538259d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55839e5e27e0 con 0x55839e988b60 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f537ce8d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 0x55839e986350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f537ce8d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 0x55839e986350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58286/0 (socket says 172.21.15.18:58286) 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f537ce8d640 1 -- 172.21.15.18:0/1399993764 learned_addr learned my addr 172.21.15.18:0/1399993764 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f537d68e640 1 --2- 172.21.15.18:0/1399993764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e97af00 0x55839e984c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.622+0000 7f537de8f640 1 --2- 172.21.15.18:0/1399993764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e988b60 0x55839e974a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.623+0000 7f537ce8d640 1 -- 172.21.15.18:0/1399993764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e988b60 msgr2=0x55839e974a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.623+0000 7f537ce8d640 1 --2- 172.21.15.18:0/1399993764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e988b60 0x55839e974a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.623+0000 7f537ce8d640 1 -- 172.21.15.18:0/1399993764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e97af00 msgr2=0x55839e984c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.623+0000 7f537ce8d640 1 --2- 172.21.15.18:0/1399993764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e97af00 0x55839e984c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.623+0000 7f537ce8d640 1 -- 172.21.15.18:0/1399993764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55839e87dc20 con 0x55839e97b820 2024-02-17T12:56:14.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.623+0000 7f537ce8d640 1 --2- 172.21.15.18:0/1399993764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 0x55839e986350 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f536c0a2ff0 tx=0x7f536c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2186a01b14d8be7e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.623+0000 7f535effd640 1 -- 172.21.15.18:0/1399993764 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c010b80 con 0x55839e97b820 2024-02-17T12:56:14.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.623+0000 7f535effd640 1 -- 172.21.15.18:0/1399993764 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f536c0974a0 con 0x55839e97b820 2024-02-17T12:56:14.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.623+0000 7f535effd640 1 -- 172.21.15.18:0/1399993764 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c0a3090 con 0x55839e97b820 2024-02-17T12:56:14.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.623+0000 7f538259d740 1 -- 172.21.15.18:0/1399993764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 msgr2=0x55839e986350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 --2- 172.21.15.18:0/1399993764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 0x55839e986350 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f536c0a2ff0 tx=0x7f536c000bc0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 -- 172.21.15.18:0/1399993764 shutdown_connections 2024-02-17T12:56:14.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 --2- 172.21.15.18:0/1399993764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e988b60 0x55839e974a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 --2- 172.21.15.18:0/1399993764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 0x55839e986350 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 --2- 172.21.15.18:0/1399993764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e97af00 0x55839e984c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 -- 172.21.15.18:0/1399993764 >> 172.21.15.18:0/1399993764 conn(0x55839e975980 msgr2=0x55839e9742f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 -- 172.21.15.18:0/1399993764 shutdown_connections 2024-02-17T12:56:14.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 -- 172.21.15.18:0/1399993764 wait complete. 2024-02-17T12:56:14.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 -- start start 2024-02-17T12:56:14.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 Processor -- start 2024-02-17T12:56:14.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97af00 0x55839e98faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 0x55839e98ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e988b60 0x55839e98e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55839e98e710 con 0x55839e97b820 2024-02-17T12:56:14.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55839e98e880 con 0x55839e988b60 2024-02-17T12:56:14.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55839e98ea40 con 0x55839e97af00 2024-02-17T12:56:14.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f538259d740 10 client.0 _ll_register_callbacks cb 0x7f536c012f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:14.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f537ce8d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 0x55839e98ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.624+0000 7f537ce8d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 0x55839e98ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58292/0 (socket says 172.21.15.18:58292) 2024-02-17T12:56:14.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f537ce8d640 1 -- 172.21.15.18:0/1684008310 learned_addr learned my addr 172.21.15.18:0/1684008310 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f537de8f640 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e988b60 0x55839e98e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f537d68e640 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97af00 0x55839e98faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f537ce8d640 1 -- 172.21.15.18:0/1684008310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97af00 msgr2=0x55839e98faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f537ce8d640 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97af00 0x55839e98faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f537ce8d640 1 -- 172.21.15.18:0/1684008310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e988b60 msgr2=0x55839e98e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f537ce8d640 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e988b60 0x55839e98e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f537ce8d640 1 -- 172.21.15.18:0/1684008310 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55839e8a5060 con 0x55839e97b820 2024-02-17T12:56:14.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f537ce8d640 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 0x55839e98ffe0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f536c09ad70 tx=0x7f536c09f010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed842c7bcafc159d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f535cff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:14.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f535cff9640 1 -- 172.21.15.18:0/1684008310 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c09b090 con 0x55839e97b820 2024-02-17T12:56:14.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f535cff9640 1 -- 172.21.15.18:0/1684008310 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f536c09b1f0 con 0x55839e97b820 2024-02-17T12:56:14.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f538259d740 1 -- 172.21.15.18:0/1684008310 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f536c0a3520 con 0x55839e97b820 2024-02-17T12:56:14.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f535cff9640 1 -- 172.21.15.18:0/1684008310 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f536c0a3090 con 0x55839e97b820 2024-02-17T12:56:14.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f538259d740 10 client.17808 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f538259d740 1 -- 172.21.15.18:0/1684008310 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55839e8a38c0 con 0x55839e97b820 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f535cff9640 1 -- 172.21.15.18:0/1684008310 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f536c0a3520 con 0x55839e97b820 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f535cff9640 1 -- 172.21.15.18:0/1684008310 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f536c0b3ab0 con 0x55839e97b820 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f538259d740 10 client.17808 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f538259d740 10 client.17808 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.625+0000 7f538259d740 20 client.17808 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f538259d740 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55839e9f4830 0x55839e9f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f538259d740 4 client.17808 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f538259d740 1 -- 172.21.15.18:0/1684008310 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55839e66e890 con 0x55839e9f4830 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f537d68e640 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55839e9f4830 0x55839e9f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f535cff9640 10 client.17808 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f537d68e640 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55839e9f4830 0x55839e9f6c50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=76e5ee90878ca979 server_cookie=7651648b59fc346f in_seq=0 out_seq=0 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f535cff9640 1 -- 172.21.15.18:0/1684008310 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55839e66e890 con 0x55839e9f4830 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f535cff9640 10 client.17808 handle_command_reply: tid=0 2024-02-17T12:56:14.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f538259d740 1 client.17808 shutdown 2024-02-17T12:56:14.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f538259d740 1 -- 172.21.15.18:0/1684008310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 msgr2=0x55839e98ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f538259d740 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 0x55839e98ffe0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f536c09ad70 tx=0x7f536c09f010 comp rx=0 tx=0).stop 2024-02-17T12:56:14.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f538259d740 1 -- 172.21.15.18:0/1684008310 shutdown_connections 2024-02-17T12:56:14.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f538259d740 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55839e9f4830 0x55839e9f6c50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f538259d740 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55839e988b60 0x55839e98e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f538259d740 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839e97b820 0x55839e98ffe0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f535cff9640 0 client.17808 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:56:14.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f538259d740 1 --2- 172.21.15.18:0/1684008310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55839e97af00 0x55839e98faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.626+0000 7f538259d740 1 -- 172.21.15.18:0/1684008310 >> 172.21.15.18:0/1684008310 conn(0x55839e975980 msgr2=0x55839e9742f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f538259d740 1 -- 172.21.15.18:0/1684008310 shutdown_connections 2024-02-17T12:56:14.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f538259d740 1 -- 172.21.15.18:0/1684008310 wait complete. 2024-02-17T12:56:14.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f538259d740 20 client.17808 trim_cache size 0 max 16384 2024-02-17T12:56:14.627 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:56:14.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f537f8f2640 1 -- 172.21.15.18:0/2120357119 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f53600a2230 msgr2=0x7f53600a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f537f8f2640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f53600a2230 0x7f53600a46f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5368032730 tx=0x7f5368005c50 comp rx=0 tx=0).stop 2024-02-17T12:56:14.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f537f8f2640 1 -- 172.21.15.18:0/2120357119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5378102d10 msgr2=0x7f537819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f537f8f2640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5378102d10 0x7f537819d940 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f536c00e9f0 tx=0x7f536c00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f537f8f2640 1 -- 172.21.15.18:0/2120357119 shutdown_connections 2024-02-17T12:56:14.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f537f8f2640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f53600a2230 0x7f53600a46f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f537f8f2640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537810b7c0 0x7f53781975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f537f8f2640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5378102d10 0x7f537819d940 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f537f8f2640 1 --2- 172.21.15.18:0/2120357119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53781023f0 0x7f537819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f537f8f2640 1 -- 172.21.15.18:0/2120357119 >> 172.21.15.18:0/2120357119 conn(0x7f53780771e0 msgr2=0x7f53780fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f537f8f2640 1 -- 172.21.15.18:0/2120357119 shutdown_connections 2024-02-17T12:56:14.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.627+0000 7f537f8f2640 1 -- 172.21.15.18:0/2120357119 wait complete. 2024-02-17T12:56:14.629 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:56:14.629 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:56:14.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.746+0000 7fb190765640 1 -- 172.21.15.18:0/737860078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb188106430 msgr2=0x7fb18810e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.746+0000 7fb190765640 1 --2- 172.21.15.18:0/737860078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb188106430 0x7fb18810e000 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb178009930 tx=0x7fb17802bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:14.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.746+0000 7fb190765640 1 -- 172.21.15.18:0/737860078 shutdown_connections 2024-02-17T12:56:14.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.746+0000 7fb190765640 1 --2- 172.21.15.18:0/737860078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb188106430 0x7fb18810e000 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.746+0000 7fb190765640 1 --2- 172.21.15.18:0/737860078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb188068f80 0x7fb188105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.746+0000 7fb190765640 1 --2- 172.21.15.18:0/737860078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1880685d0 0x7fb1880689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.746+0000 7fb190765640 1 -- 172.21.15.18:0/737860078 >> 172.21.15.18:0/737860078 conn(0x7fb188077180 msgr2=0x7fb188077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.746+0000 7fb190765640 1 -- 172.21.15.18:0/737860078 shutdown_connections 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.746+0000 7fb190765640 1 -- 172.21.15.18:0/737860078 wait complete. 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.746+0000 7fb190765640 1 Processor -- start 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.746+0000 7fb190765640 1 -- start start 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb190765640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1880685d0 0x7fb18819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb190765640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb188106430 0x7fb18819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb190765640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb188197620 0x7fb188197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb190765640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb188198010 con 0x7fb1880685d0 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb190765640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb188198180 con 0x7fb188197620 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb190765640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb188198340 con 0x7fb188106430 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb18e501640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1880685d0 0x7fb18819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb18e501640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1880685d0 0x7fb18819d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58314/0 (socket says 172.21.15.18:58314) 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb18e501640 1 -- 172.21.15.18:0/941479210 learned_addr learned my addr 172.21.15.18:0/941479210 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb18ed02640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb188197620 0x7fb188197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb18dd00640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb188106430 0x7fb18819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb18e501640 1 -- 172.21.15.18:0/941479210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb188106430 msgr2=0x7fb18819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb18e501640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb188106430 0x7fb18819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb18e501640 1 -- 172.21.15.18:0/941479210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb188197620 msgr2=0x7fb188197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb18e501640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb188197620 0x7fb188197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb18e501640 1 -- 172.21.15.18:0/941479210 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c009490 con 0x7fb1880685d0 2024-02-17T12:56:14.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb18e501640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1880685d0 0x7fb18819d420 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb18000eb90 tx=0x7fb18000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb1877fe640 1 -- 172.21.15.18:0/941479210 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1800099b0 con 0x7fb1880685d0 2024-02-17T12:56:14.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb190765640 1 -- 172.21.15.18:0/941479210 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb178009560 con 0x7fb1880685d0 2024-02-17T12:56:14.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.747+0000 7fb1877fe640 1 -- 172.21.15.18:0/941479210 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb18000ee90 con 0x7fb1880685d0 2024-02-17T12:56:14.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.748+0000 7fb1877fe640 1 -- 172.21.15.18:0/941479210 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1800107a0 con 0x7fb1880685d0 2024-02-17T12:56:14.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.748+0000 7fb190765640 1 -- 172.21.15.18:0/941479210 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1881a4660 con 0x7fb1880685d0 2024-02-17T12:56:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb1877fe640 1 -- 172.21.15.18:0/941479210 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb180005020 con 0x7fb1880685d0 2024-02-17T12:56:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb193413740 1 Processor -- start 2024-02-17T12:56:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb193413740 1 -- start start 2024-02-17T12:56:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb193413740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 0x56356b9fb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb193413740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fbb70 0x56356ba04210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb193413740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356ba04750 0x56356ba06b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb193413740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356b9b6960 con 0x56356b9fb250 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb193413740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356b96a310 con 0x56356ba04750 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb193413740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356b967840 con 0x56356b9fbb70 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb18ed02640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 0x56356b9fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb18ed02640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 0x56356b9fb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58324/0 (socket says 172.21.15.18:58324) 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb18ed02640 1 -- 172.21.15.18:0/3723193218 learned_addr learned my addr 172.21.15.18:0/3723193218 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb18dd00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356ba04750 0x56356ba06b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.750+0000 7fb18e501640 1 --2- 172.21.15.18:0/3723193218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fbb70 0x56356ba04210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb1877fe640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb1700a2230 0x7fb1700a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb18dd00640 1 -- 172.21.15.18:0/3723193218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fbb70 msgr2=0x56356ba04210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb18dd00640 1 --2- 172.21.15.18:0/3723193218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fbb70 0x56356ba04210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb18dd00640 1 -- 172.21.15.18:0/3723193218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 msgr2=0x56356b9fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb18dd00640 1 --2- 172.21.15.18:0/3723193218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 0x56356b9fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb18dd00640 1 -- 172.21.15.18:0/3723193218 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56356b925060 con 0x56356ba04750 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb18ed02640 1 --2- 172.21.15.18:0/3723193218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 0x56356b9fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb18ed02640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb1700a2230 0x7fb1700a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb1877fe640 1 -- 172.21.15.18:0/941479210 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb180014070 con 0x7fb1880685d0 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb18dd00640 1 --2- 172.21.15.18:0/3723193218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356ba04750 0x56356ba06b40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb188105fc0 tx=0x7fb17c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d40d7b80321bdea6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb184ff9640 1 -- 172.21.15.18:0/3723193218 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c004250 con 0x56356ba04750 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb184ff9640 1 -- 172.21.15.18:0/3723193218 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb17c0043b0 con 0x56356ba04750 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.751+0000 7fb184ff9640 1 -- 172.21.15.18:0/3723193218 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0198a0 con 0x56356ba04750 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.752+0000 7fb193413740 1 -- 172.21.15.18:0/3723193218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356ba04750 msgr2=0x56356ba06b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.752+0000 7fb193413740 1 --2- 172.21.15.18:0/3723193218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356ba04750 0x56356ba06b40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb188105fc0 tx=0x7fb17c011810 comp rx=0 tx=0).stop 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.752+0000 7fb18ed02640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb1700a2230 0x7fb1700a46f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb178006010 tx=0x7fb178009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.752+0000 7fb193413740 1 -- 172.21.15.18:0/3723193218 shutdown_connections 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.752+0000 7fb193413740 1 --2- 172.21.15.18:0/3723193218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356ba04750 0x56356ba06b40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.752+0000 7fb193413740 1 --2- 172.21.15.18:0/3723193218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fbb70 0x56356ba04210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.752+0000 7fb193413740 1 --2- 172.21.15.18:0/3723193218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 0x56356b9fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.752+0000 7fb193413740 1 -- 172.21.15.18:0/3723193218 >> 172.21.15.18:0/3723193218 conn(0x56356b9f5980 msgr2=0x56356b9f5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.752+0000 7fb193413740 1 -- 172.21.15.18:0/3723193218 shutdown_connections 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.752+0000 7fb193413740 1 -- 172.21.15.18:0/3723193218 wait complete. 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.752+0000 7fb193413740 1 -- start start 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.752+0000 7fb193413740 1 Processor -- start 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb193413740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fb250 0x56356ba01760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb193413740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fbb70 0x56356ba01ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb193413740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 0x56356ba021e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb193413740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356b963020 con 0x56356ba04750 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb193413740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356b636430 con 0x56356b9fb250 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb193413740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356b5f2e00 con 0x56356b9fbb70 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb193413740 10 client.0 _ll_register_callbacks cb 0x56356b90acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18ed02640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 0x56356ba021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18dd00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fbb70 0x56356ba01ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18e501640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fb250 0x56356ba01760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18ed02640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 0x56356ba021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58334/0 (socket says 172.21.15.18:58334) 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18ed02640 1 -- 172.21.15.18:0/4276858494 learned_addr learned my addr 172.21.15.18:0/4276858494 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18ed02640 1 -- 172.21.15.18:0/4276858494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fbb70 msgr2=0x56356ba01ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18ed02640 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fbb70 0x56356ba01ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18ed02640 1 -- 172.21.15.18:0/4276858494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fb250 msgr2=0x56356ba01760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18ed02640 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fb250 0x56356ba01760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18ed02640 1 -- 172.21.15.18:0/4276858494 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c0099e0 con 0x56356ba04750 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18e501640 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fb250 0x56356ba01760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18dd00640 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fbb70 0x56356ba01ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb18ed02640 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 0x56356ba021e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb178036040 tx=0x7fb1780058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2338e96f4de7ee83 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.753+0000 7fb16dffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.754+0000 7fb16dffb640 1 -- 172.21.15.18:0/4276858494 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb178005000 con 0x56356ba04750 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.754+0000 7fb193413740 1 -- 172.21.15.18:0/4276858494 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56356b6627e0 con 0x56356ba04750 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.754+0000 7fb16dffb640 1 -- 172.21.15.18:0/4276858494 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb178005160 con 0x56356ba04750 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.754+0000 7fb16dffb640 1 -- 172.21.15.18:0/4276858494 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb178005320 con 0x56356ba04750 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.754+0000 7fb193413740 10 client.17820 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.754+0000 7fb193413740 1 -- 172.21.15.18:0/4276858494 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56356b9616a0 con 0x56356ba04750 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.754+0000 7fb16dffb640 1 -- 172.21.15.18:0/4276858494 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56356b6627e0 con 0x56356ba04750 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.754+0000 7fb16dffb640 1 -- 172.21.15.18:0/4276858494 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb178049050 con 0x56356ba04750 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.755+0000 7fb193413740 10 client.17820 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.755+0000 7fb193413740 10 client.17820 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.755+0000 7fb193413740 20 client.17820 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.755+0000 7fb193413740 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56356ba14a30 0x56356ba16e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.755+0000 7fb193413740 4 client.17820 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.755+0000 7fb193413740 1 -- 172.21.15.18:0/4276858494 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56356b69f720 con 0x56356ba14a30 2024-02-17T12:56:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.755+0000 7fb18e501640 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56356ba14a30 0x56356ba16e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.755+0000 7fb18e501640 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56356ba14a30 0x56356ba16e50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6eb024ed21d1ad4a server_cookie=f5cd9e98459bafda in_seq=0 out_seq=0 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.755+0000 7fb16dffb640 10 client.17820 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb16dffb640 1 -- 172.21.15.18:0/4276858494 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56356b69f720 con 0x56356ba14a30 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb16dffb640 10 client.17820 handle_command_reply: tid=0 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb193413740 1 client.17820 shutdown 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb193413740 1 -- 172.21.15.18:0/4276858494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 msgr2=0x56356ba021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb193413740 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 0x56356ba021e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb178036040 tx=0x7fb1780058d0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb193413740 1 -- 172.21.15.18:0/4276858494 shutdown_connections 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb193413740 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56356ba14a30 0x56356ba16e50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb193413740 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 0x56356ba021e0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb193413740 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fbb70 0x56356ba01ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb16dffb640 0 client.17820 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb193413740 1 --2- 172.21.15.18:0/4276858494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fb250 0x56356ba01760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb193413740 1 -- 172.21.15.18:0/4276858494 >> 172.21.15.18:0/4276858494 conn(0x56356b9f5980 msgr2=0x56356b9f76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb193413740 1 -- 172.21.15.18:0/4276858494 shutdown_connections 2024-02-17T12:56:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb193413740 1 -- 172.21.15.18:0/4276858494 wait complete. 2024-02-17T12:56:14.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.756+0000 7fb193413740 20 client.17820 trim_cache size 0 max 16384 2024-02-17T12:56:14.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.768+0000 7fb193413740 1 Processor -- start 2024-02-17T12:56:14.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.768+0000 7fb193413740 1 -- start start 2024-02-17T12:56:14.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb193413740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 0x56356ba03e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb193413740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fbb70 0x56356ba02000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb193413740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356ba04750 0x56356b9fe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb193413740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356b8e8130 con 0x56356b9fb250 2024-02-17T12:56:14.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb193413740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356b9f6bb0 con 0x56356b9fbb70 2024-02-17T12:56:14.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb193413740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356b9f6980 con 0x56356ba04750 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb18e501640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 0x56356ba03e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb18e501640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 0x56356ba03e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58336/0 (socket says 172.21.15.18:58336) 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb18e501640 1 -- 172.21.15.18:0/112092908 learned_addr learned my addr 172.21.15.18:0/112092908 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb18dd00640 1 --2- 172.21.15.18:0/112092908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fbb70 0x56356ba02000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb18ed02640 1 --2- 172.21.15.18:0/112092908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356ba04750 0x56356b9fe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb18e501640 1 -- 172.21.15.18:0/112092908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356ba04750 msgr2=0x56356b9fe9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb18e501640 1 --2- 172.21.15.18:0/112092908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356ba04750 0x56356b9fe9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb18e501640 1 -- 172.21.15.18:0/112092908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fbb70 msgr2=0x56356ba02000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb18e501640 1 --2- 172.21.15.18:0/112092908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fbb70 0x56356ba02000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb18e501640 1 -- 172.21.15.18:0/112092908 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56356b8fdc20 con 0x56356b9fb250 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb18e501640 1 --2- 172.21.15.18:0/112092908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 0x56356ba03e90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb18000cfd0 tx=0x7fb1800d4550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ea43a2645417e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb16ffff640 1 -- 172.21.15.18:0/112092908 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18009abd0 con 0x56356b9fb250 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb16ffff640 1 -- 172.21.15.18:0/112092908 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb18009b970 con 0x56356b9fb250 2024-02-17T12:56:14.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.769+0000 7fb16ffff640 1 -- 172.21.15.18:0/112092908 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180099570 con 0x56356b9fb250 2024-02-17T12:56:14.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.770+0000 7fb193413740 1 -- 172.21.15.18:0/112092908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 msgr2=0x56356ba03e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.770+0000 7fb193413740 1 --2- 172.21.15.18:0/112092908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 0x56356ba03e90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb18000cfd0 tx=0x7fb1800d4550 comp rx=0 tx=0).stop 2024-02-17T12:56:14.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.770+0000 7fb193413740 1 -- 172.21.15.18:0/112092908 shutdown_connections 2024-02-17T12:56:14.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.770+0000 7fb193413740 1 --2- 172.21.15.18:0/112092908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356ba04750 0x56356b9fe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.770+0000 7fb193413740 1 --2- 172.21.15.18:0/112092908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fbb70 0x56356ba02000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.770+0000 7fb193413740 1 --2- 172.21.15.18:0/112092908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356b9fb250 0x56356ba03e90 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.770+0000 7fb193413740 1 -- 172.21.15.18:0/112092908 >> 172.21.15.18:0/112092908 conn(0x56356b9f5980 msgr2=0x56356b9fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.770+0000 7fb193413740 1 -- 172.21.15.18:0/112092908 shutdown_connections 2024-02-17T12:56:14.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.770+0000 7fb193413740 1 -- 172.21.15.18:0/112092908 wait complete. 2024-02-17T12:56:14.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb193413740 1 -- start start 2024-02-17T12:56:14.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb193413740 1 Processor -- start 2024-02-17T12:56:14.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb193413740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fb250 0x56356ba0a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb193413740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fbb70 0x56356ba0acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb193413740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 0x56356ba6af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb193413740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356ba6b4a0 con 0x56356ba04750 2024-02-17T12:56:14.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb193413740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356ba6b5e0 con 0x56356b9fbb70 2024-02-17T12:56:14.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb193413740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356ba6b770 con 0x56356b9fb250 2024-02-17T12:56:14.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb193413740 10 client.0 _ll_register_callbacks cb 0x7fb17c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:14.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb18ed02640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 0x56356ba6af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb18ed02640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 0x56356ba6af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58342/0 (socket says 172.21.15.18:58342) 2024-02-17T12:56:14.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb18ed02640 1 -- 172.21.15.18:0/760010737 learned_addr learned my addr 172.21.15.18:0/760010737 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb18dd00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fbb70 0x56356ba0acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb18e501640 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fb250 0x56356ba0a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb18ed02640 1 -- 172.21.15.18:0/760010737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fb250 msgr2=0x56356ba0a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb18ed02640 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fb250 0x56356ba0a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.771+0000 7fb18ed02640 1 -- 172.21.15.18:0/760010737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fbb70 msgr2=0x56356ba0acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb18ed02640 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fbb70 0x56356ba0acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb18ed02640 1 -- 172.21.15.18:0/760010737 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1881a4660 con 0x56356ba04750 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb18dd00640 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fbb70 0x56356ba0acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb18ed02640 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 0x56356ba6af60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb178036040 tx=0x7fb17804c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d274a49685f64e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb16dffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb16dffb640 1 -- 172.21.15.18:0/760010737 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17804cd20 con 0x56356ba04750 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb16dffb640 1 -- 172.21.15.18:0/760010737 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb178009780 con 0x56356ba04750 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb193413740 1 -- 172.21.15.18:0/760010737 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb17c0043b0 con 0x56356ba04750 2024-02-17T12:56:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb16dffb640 1 -- 172.21.15.18:0/760010737 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb17804ce80 con 0x56356ba04750 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb193413740 10 client.17826 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb193413740 1 -- 172.21.15.18:0/760010737 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56356b9238c0 con 0x56356ba04750 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb16dffb640 1 -- 172.21.15.18:0/760010737 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb17806d020 con 0x56356ba04750 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb16dffb640 1 -- 172.21.15.18:0/760010737 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb178049040 con 0x56356ba04750 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb193413740 10 client.17826 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb193413740 10 client.17826 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb193413740 20 client.17826 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb193413740 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56356ba6cba0 0x56356ba6efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb193413740 4 client.17826 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.772+0000 7fb193413740 1 -- 172.21.15.18:0/760010737 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56356b6ee890 con 0x56356ba6cba0 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb18e501640 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56356ba6cba0 0x56356ba6efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb16dffb640 10 client.17826 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb18e501640 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56356ba6cba0 0x56356ba6efc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bdf721d4dfc2969e server_cookie=5e8a8ac8b91260ad in_seq=0 out_seq=0 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb16dffb640 1 -- 172.21.15.18:0/760010737 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56356b6ee890 con 0x56356ba6cba0 2024-02-17T12:56:14.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb16dffb640 10 client.17826 handle_command_reply: tid=0 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb193413740 1 client.17826 shutdown 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb193413740 1 -- 172.21.15.18:0/760010737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 msgr2=0x56356ba6af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb193413740 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 0x56356ba6af60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb178036040 tx=0x7fb17804c710 comp rx=0 tx=0).stop 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb193413740 1 -- 172.21.15.18:0/760010737 shutdown_connections 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb193413740 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56356ba6cba0 0x56356ba6efc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb193413740 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56356ba04750 0x56356ba6af60 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb193413740 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56356b9fbb70 0x56356ba0acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb16dffb640 0 client.17826 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb193413740 1 --2- 172.21.15.18:0/760010737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56356b9fb250 0x56356ba0a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb193413740 1 -- 172.21.15.18:0/760010737 >> 172.21.15.18:0/760010737 conn(0x56356b9f5980 msgr2=0x56356b9f42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb193413740 1 -- 172.21.15.18:0/760010737 shutdown_connections 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb193413740 1 -- 172.21.15.18:0/760010737 wait complete. 2024-02-17T12:56:14.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.773+0000 7fb193413740 20 client.17826 trim_cache size 0 max 16384 2024-02-17T12:56:14.771 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:56:14.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.774+0000 7fb190765640 1 -- 172.21.15.18:0/941479210 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb1700a2230 msgr2=0x7fb1700a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.774+0000 7fb190765640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb1700a2230 0x7fb1700a46f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb178006010 tx=0x7fb178009260 comp rx=0 tx=0).stop 2024-02-17T12:56:14.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.774+0000 7fb190765640 1 -- 172.21.15.18:0/941479210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1880685d0 msgr2=0x7fb18819d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.774+0000 7fb190765640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1880685d0 0x7fb18819d420 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb18000eb90 tx=0x7fb18000c6a0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.774+0000 7fb190765640 1 -- 172.21.15.18:0/941479210 shutdown_connections 2024-02-17T12:56:14.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.774+0000 7fb190765640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb1700a2230 0x7fb1700a46f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.774+0000 7fb190765640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb188197620 0x7fb188197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.774+0000 7fb190765640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb188106430 0x7fb18819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.774+0000 7fb190765640 1 --2- 172.21.15.18:0/941479210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1880685d0 0x7fb18819d420 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.774+0000 7fb190765640 1 -- 172.21.15.18:0/941479210 >> 172.21.15.18:0/941479210 conn(0x7fb188077180 msgr2=0x7fb1880fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.774+0000 7fb190765640 1 -- 172.21.15.18:0/941479210 shutdown_connections 2024-02-17T12:56:14.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.774+0000 7fb190765640 1 -- 172.21.15.18:0/941479210 wait complete. 2024-02-17T12:56:14.775 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:56:14.775 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:56:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.898+0000 7ff525681640 1 -- 172.21.15.18:0/3388695727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5201064c0 msgr2=0x7ff52010e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.898+0000 7ff525681640 1 --2- 172.21.15.18:0/3388695727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5201064c0 0x7ff52010e070 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7ff518009cd0 tx=0x7ff51802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.898+0000 7ff525681640 1 -- 172.21.15.18:0/3388695727 shutdown_connections 2024-02-17T12:56:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.898+0000 7ff525681640 1 --2- 172.21.15.18:0/3388695727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5201064c0 0x7ff52010e070 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.898+0000 7ff525681640 1 --2- 172.21.15.18:0/3388695727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff520068fa0 0x7ff520105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.898+0000 7ff525681640 1 --2- 172.21.15.18:0/3388695727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5200685f0 0x7ff5200689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.898+0000 7ff525681640 1 -- 172.21.15.18:0/3388695727 >> 172.21.15.18:0/3388695727 conn(0x7ff520077180 msgr2=0x7ff520077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.898+0000 7ff525681640 1 -- 172.21.15.18:0/3388695727 shutdown_connections 2024-02-17T12:56:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.898+0000 7ff525681640 1 -- 172.21.15.18:0/3388695727 wait complete. 2024-02-17T12:56:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.898+0000 7ff525681640 1 Processor -- start 2024-02-17T12:56:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff525681640 1 -- start start 2024-02-17T12:56:14.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff525681640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5200685f0 0x7ff52019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff525681640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5201064c0 0x7ff52019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff525681640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff520197600 0x7ff520197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff525681640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff520197ff0 con 0x7ff520197600 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff525681640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff520198160 con 0x7ff5201064c0 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff525681640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff520198320 con 0x7ff5200685f0 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff51f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff520197600 0x7ff520197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff51e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5201064c0 0x7ff52019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff51f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff520197600 0x7ff520197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58362/0 (socket says 172.21.15.18:58362) 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff51f7fe640 1 -- 172.21.15.18:0/663806136 learned_addr learned my addr 172.21.15.18:0/663806136 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff51effd640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5200685f0 0x7ff52019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff51e7fc640 1 -- 172.21.15.18:0/663806136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5200685f0 msgr2=0x7ff52019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff51e7fc640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5200685f0 0x7ff52019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff51e7fc640 1 -- 172.21.15.18:0/663806136 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff520197600 msgr2=0x7ff520197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff51e7fc640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff520197600 0x7ff520197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff51e7fc640 1 -- 172.21.15.18:0/663806136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff518009900 con 0x7ff5201064c0 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff51f7fe640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff520197600 0x7ff520197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.899+0000 7ff51e7fc640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5201064c0 0x7ff52019d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff520102290 tx=0x7ff51400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.900+0000 7ff507fff640 1 -- 172.21.15.18:0/663806136 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51400cc40 con 0x7ff5201064c0 2024-02-17T12:56:14.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.900+0000 7ff525681640 1 -- 172.21.15.18:0/663806136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5200fff70 con 0x7ff5201064c0 2024-02-17T12:56:14.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.900+0000 7ff507fff640 1 -- 172.21.15.18:0/663806136 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff51400cda0 con 0x7ff5201064c0 2024-02-17T12:56:14.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.900+0000 7ff507fff640 1 -- 172.21.15.18:0/663806136 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514004a00 con 0x7ff5201064c0 2024-02-17T12:56:14.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.900+0000 7ff525681640 1 -- 172.21.15.18:0/663806136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff520100430 con 0x7ff5201064c0 2024-02-17T12:56:14.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff507fff640 1 -- 172.21.15.18:0/663806136 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff514004b60 con 0x7ff5201064c0 2024-02-17T12:56:14.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff52824e740 1 Processor -- start 2024-02-17T12:56:14.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff52824e740 1 -- start start 2024-02-17T12:56:14.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff52824e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758250 0x561693758630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff52824e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758b70 0x561693761210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff52824e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x561693763b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff52824e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561693713960 con 0x561693761750 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff52824e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616936c7310 con 0x561693758b70 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff52824e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616936c4840 con 0x561693758250 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff51e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x561693763b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff51f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758250 0x561693758630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff51f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758250 0x561693758630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40550/0 (socket says 172.21.15.18:40550) 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff51f7fe640 1 -- 172.21.15.18:0/672661726 learned_addr learned my addr 172.21.15.18:0/672661726 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff51effd640 1 --2- 172.21.15.18:0/672661726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758b70 0x561693761210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.903+0000 7ff507fff640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff5080a2230 0x7ff5080a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff51f7fe640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff5080a2230 0x7ff5080a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff507fff640 1 -- 172.21.15.18:0/663806136 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff514014070 con 0x7ff5201064c0 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff51e7fc640 1 -- 172.21.15.18:0/672661726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758250 msgr2=0x561693758630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff51e7fc640 1 --2- 172.21.15.18:0/672661726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758250 0x561693758630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff51e7fc640 1 -- 172.21.15.18:0/672661726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758b70 msgr2=0x561693761210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff51e7fc640 1 --2- 172.21.15.18:0/672661726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758b70 0x561693761210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff51e7fc640 1 -- 172.21.15.18:0/672661726 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561693682060 con 0x561693761750 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff51effd640 1 --2- 172.21.15.18:0/672661726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758b70 0x561693761210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff51f7fe640 1 --2- 172.21.15.18:0/672661726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758250 0x561693758630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff51e7fc640 1 --2- 172.21.15.18:0/672661726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x561693763b40 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7ff5140cb7c0 tx=0x7ff51400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc68642e0173e175 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff5057fa640 1 -- 172.21.15.18:0/672661726 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51409a5a0 con 0x561693761750 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff51f7fe640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff5080a2230 0x7ff5080a46f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff5180348f0 tx=0x7ff51802c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff5057fa640 1 -- 172.21.15.18:0/672661726 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff51409a3c0 con 0x561693761750 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.904+0000 7ff5057fa640 1 -- 172.21.15.18:0/672661726 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514004e10 con 0x561693761750 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.905+0000 7ff52824e740 1 -- 172.21.15.18:0/672661726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 msgr2=0x561693763b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.905+0000 7ff52824e740 1 --2- 172.21.15.18:0/672661726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x561693763b40 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7ff5140cb7c0 tx=0x7ff51400cf10 comp rx=0 tx=0).stop 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.905+0000 7ff52824e740 1 -- 172.21.15.18:0/672661726 shutdown_connections 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.905+0000 7ff52824e740 1 --2- 172.21.15.18:0/672661726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x561693763b40 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.905+0000 7ff52824e740 1 --2- 172.21.15.18:0/672661726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758b70 0x561693761210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.905+0000 7ff52824e740 1 --2- 172.21.15.18:0/672661726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758250 0x561693758630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.905+0000 7ff52824e740 1 -- 172.21.15.18:0/672661726 >> 172.21.15.18:0/672661726 conn(0x561693752980 msgr2=0x561693752d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.905+0000 7ff52824e740 1 -- 172.21.15.18:0/672661726 shutdown_connections 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.905+0000 7ff52824e740 1 -- 172.21.15.18:0/672661726 wait complete. 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.905+0000 7ff52824e740 1 -- start start 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.905+0000 7ff52824e740 1 Processor -- start 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff52824e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x56169375e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff52824e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758b70 0x56169375ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff52824e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x56169375f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff52824e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616936c0020 con 0x561693761750 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff52824e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561693393430 con 0x561693758250 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff52824e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169334fe00 con 0x561693758b70 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff52824e740 10 client.0 _ll_register_callbacks cb 0x561693667cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff51f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x56169375f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff51f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x56169375f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58376/0 (socket says 172.21.15.18:58376) 2024-02-17T12:56:14.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff51effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x56169375e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff51f7fe640 1 -- 172.21.15.18:0/1484526493 learned_addr learned my addr 172.21.15.18:0/1484526493 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff51e7fc640 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758b70 0x56169375ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff51f7fe640 1 -- 172.21.15.18:0/1484526493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758b70 msgr2=0x56169375ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff51f7fe640 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758b70 0x56169375ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff51f7fe640 1 -- 172.21.15.18:0/1484526493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 msgr2=0x56169375e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff51f7fe640 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x56169375e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff51f7fe640 1 -- 172.21.15.18:0/1484526493 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff520100430 con 0x561693761750 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff51effd640 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x56169375e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff51f7fe640 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x56169375f250 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff518032930 tx=0x7ff5180095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d707976f0ab30024 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff502ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.906+0000 7ff502ff5640 1 -- 172.21.15.18:0/1484526493 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff518004040 con 0x561693761750 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff52824e740 1 -- 172.21.15.18:0/1484526493 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616933bf7e0 con 0x561693761750 2024-02-17T12:56:14.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff502ff5640 1 -- 172.21.15.18:0/1484526493 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff5180041a0 con 0x561693761750 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff502ff5640 1 -- 172.21.15.18:0/1484526493 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff518004360 con 0x561693761750 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff52824e740 10 client.17838 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff52824e740 1 -- 172.21.15.18:0/1484526493 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5616936be6a0 con 0x561693761750 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff502ff5640 1 -- 172.21.15.18:0/1484526493 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5616933bf7e0 con 0x561693761750 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff502ff5640 1 -- 172.21.15.18:0/1484526493 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff518055060 con 0x561693761750 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff52824e740 10 client.17838 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff52824e740 10 client.17838 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff52824e740 20 client.17838 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff52824e740 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561693771ae0 0x561693773f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff52824e740 4 client.17838 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff52824e740 1 -- 172.21.15.18:0/1484526493 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616933fc720 con 0x561693771ae0 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.907+0000 7ff51effd640 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561693771ae0 0x561693773f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff51effd640 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561693771ae0 0x561693773f00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29eaff8f5eb4fc47 server_cookie=4066a63da4beceef in_seq=0 out_seq=0 2024-02-17T12:56:14.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff502ff5640 10 client.17838 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff502ff5640 1 -- 172.21.15.18:0/1484526493 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5616933fc720 con 0x561693771ae0 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff502ff5640 10 client.17838 handle_command_reply: tid=0 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff52824e740 1 client.17838 shutdown 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff52824e740 1 -- 172.21.15.18:0/1484526493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 msgr2=0x56169375f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff52824e740 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x56169375f250 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff518032930 tx=0x7ff5180095c0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff52824e740 1 -- 172.21.15.18:0/1484526493 shutdown_connections 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff52824e740 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561693771ae0 0x561693773f00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff52824e740 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x56169375f250 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff52824e740 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758b70 0x56169375ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff502ff5640 0 client.17838 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff52824e740 1 --2- 172.21.15.18:0/1484526493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x56169375e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff52824e740 1 -- 172.21.15.18:0/1484526493 >> 172.21.15.18:0/1484526493 conn(0x561693752980 msgr2=0x5616937546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff52824e740 1 -- 172.21.15.18:0/1484526493 shutdown_connections 2024-02-17T12:56:14.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.908+0000 7ff52824e740 1 -- 172.21.15.18:0/1484526493 wait complete. 2024-02-17T12:56:14.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.909+0000 7ff52824e740 20 client.17838 trim_cache size 0 max 16384 2024-02-17T12:56:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.925+0000 7ff52824e740 1 Processor -- start 2024-02-17T12:56:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.925+0000 7ff52824e740 1 -- start start 2024-02-17T12:56:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.925+0000 7ff52824e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x561693760f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.925+0000 7ff52824e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693758b70 0x56169375f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.925+0000 7ff52824e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693761750 0x56169375ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.925+0000 7ff52824e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561693645130 con 0x561693758b70 2024-02-17T12:56:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.925+0000 7ff52824e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616937791b0 con 0x561693758250 2024-02-17T12:56:14.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.925+0000 7ff52824e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169374e100 con 0x561693761750 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.925+0000 7ff51e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693758b70 0x56169375f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x561693760f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693761750 0x56169375ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693758b70 0x56169375f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58386/0 (socket says 172.21.15.18:58386) 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51e7fc640 1 -- 172.21.15.18:0/3781913025 learned_addr learned my addr 172.21.15.18:0/3781913025 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51effd640 1 --2- 172.21.15.18:0/3781913025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x561693760f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:56368/0 (socket says 172.21.15.18:56368) 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51e7fc640 1 -- 172.21.15.18:0/3781913025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693761750 msgr2=0x56169375ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51e7fc640 1 --2- 172.21.15.18:0/3781913025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693761750 0x56169375ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51e7fc640 1 -- 172.21.15.18:0/3781913025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 msgr2=0x561693760f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51e7fc640 1 --2- 172.21.15.18:0/3781913025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x561693760f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51e7fc640 1 -- 172.21.15.18:0/3781913025 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56169365ac20 con 0x561693758b70 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51f7fe640 1 --2- 172.21.15.18:0/3781913025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693761750 0x56169375ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51effd640 1 --2- 172.21.15.18:0/3781913025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x561693760f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:14.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff51e7fc640 1 --2- 172.21.15.18:0/3781913025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693758b70 0x56169375f070 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff51409a530 tx=0x7ff5140053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d57e05dd353bb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff504ff9640 1 -- 172.21.15.18:0/3781913025 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51409ebb0 con 0x561693758b70 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff504ff9640 1 -- 172.21.15.18:0/3781913025 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff51409b160 con 0x561693758b70 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.926+0000 7ff504ff9640 1 -- 172.21.15.18:0/3781913025 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5140cba40 con 0x561693758b70 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.927+0000 7ff52824e740 1 -- 172.21.15.18:0/3781913025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693758b70 msgr2=0x56169375f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.927+0000 7ff52824e740 1 --2- 172.21.15.18:0/3781913025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693758b70 0x56169375f070 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff51409a530 tx=0x7ff5140053f0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.927+0000 7ff52824e740 1 -- 172.21.15.18:0/3781913025 shutdown_connections 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.927+0000 7ff52824e740 1 --2- 172.21.15.18:0/3781913025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693761750 0x56169375ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.927+0000 7ff52824e740 1 --2- 172.21.15.18:0/3781913025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693758b70 0x56169375f070 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.927+0000 7ff52824e740 1 --2- 172.21.15.18:0/3781913025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x561693760f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.927+0000 7ff52824e740 1 -- 172.21.15.18:0/3781913025 >> 172.21.15.18:0/3781913025 conn(0x561693752980 msgr2=0x561693759a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.927+0000 7ff52824e740 1 -- 172.21.15.18:0/3781913025 shutdown_connections 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.927+0000 7ff52824e740 1 -- 172.21.15.18:0/3781913025 wait complete. 2024-02-17T12:56:14.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.927+0000 7ff52824e740 1 -- start start 2024-02-17T12:56:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff52824e740 1 Processor -- start 2024-02-17T12:56:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff52824e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x561693766eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff52824e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758b70 0x5616937673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff52824e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x561693767930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff52824e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561693768e00 con 0x561693761750 2024-02-17T12:56:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff52824e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616937c8030 con 0x561693758250 2024-02-17T12:56:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff52824e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616937c8170 con 0x561693758b70 2024-02-17T12:56:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff52824e740 10 client.0 _ll_register_callbacks cb 0x7ff514096d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff51f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x561693767930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff51f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x561693767930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58396/0 (socket says 172.21.15.18:58396) 2024-02-17T12:56:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff51f7fe640 1 -- 172.21.15.18:0/981527176 learned_addr learned my addr 172.21.15.18:0/981527176 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:14.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff51effd640 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x561693766eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff51e7fc640 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758b70 0x5616937673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff51f7fe640 1 -- 172.21.15.18:0/981527176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758b70 msgr2=0x5616937673f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff51f7fe640 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758b70 0x5616937673f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff51f7fe640 1 -- 172.21.15.18:0/981527176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 msgr2=0x561693766eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff51f7fe640 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x561693766eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff51f7fe640 1 -- 172.21.15.18:0/981527176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff510012690 con 0x561693761750 2024-02-17T12:56:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff51f7fe640 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x561693767930 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff518009ca0 tx=0x7ff518032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f890e82a03a7a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.928+0000 7ff502ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff502ff5640 1 -- 172.21.15.18:0/981527176 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff518043a80 con 0x561693761750 2024-02-17T12:56:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff52824e740 1 -- 172.21.15.18:0/981527176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff51409a3c0 con 0x561693761750 2024-02-17T12:56:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff502ff5640 1 -- 172.21.15.18:0/981527176 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff518044a20 con 0x561693761750 2024-02-17T12:56:14.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff502ff5640 1 -- 172.21.15.18:0/981527176 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff518044dd0 con 0x561693761750 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff52824e740 10 client.17844 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff52824e740 1 -- 172.21.15.18:0/981527176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5616936be6a0 con 0x561693761750 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff502ff5640 1 -- 172.21.15.18:0/981527176 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff51409a3c0 con 0x561693761750 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff502ff5640 1 -- 172.21.15.18:0/981527176 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff518055040 con 0x561693761750 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff52824e740 10 client.17844 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff52824e740 10 client.17844 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff52824e740 20 client.17844 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff52824e740 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5616937c9620 0x5616937cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff52824e740 4 client.17844 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.929+0000 7ff52824e740 1 -- 172.21.15.18:0/981527176 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56169344b890 con 0x5616937c9620 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff51effd640 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5616937c9620 0x5616937cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff502ff5640 10 client.17844 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff51effd640 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5616937c9620 0x5616937cba40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b0f719d399cdaa0 server_cookie=a6c37c373d14c525 in_seq=0 out_seq=0 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff502ff5640 1 -- 172.21.15.18:0/981527176 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56169344b890 con 0x5616937c9620 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff502ff5640 10 client.17844 handle_command_reply: tid=0 2024-02-17T12:56:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff52824e740 1 client.17844 shutdown 2024-02-17T12:56:14.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff52824e740 1 -- 172.21.15.18:0/981527176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 msgr2=0x561693767930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff52824e740 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x561693767930 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff518009ca0 tx=0x7ff518032c50 comp rx=0 tx=0).stop 2024-02-17T12:56:14.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff52824e740 1 -- 172.21.15.18:0/981527176 shutdown_connections 2024-02-17T12:56:14.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff52824e740 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5616937c9620 0x5616937cba40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff52824e740 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561693761750 0x561693767930 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff52824e740 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561693758b70 0x5616937673f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff52824e740 1 --2- 172.21.15.18:0/981527176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561693758250 0x561693766eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff502ff5640 0 client.17844 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:56:14.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.930+0000 7ff52824e740 1 -- 172.21.15.18:0/981527176 >> 172.21.15.18:0/981527176 conn(0x561693752980 msgr2=0x5616937512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff52824e740 1 -- 172.21.15.18:0/981527176 shutdown_connections 2024-02-17T12:56:14.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff52824e740 1 -- 172.21.15.18:0/981527176 wait complete. 2024-02-17T12:56:14.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff52824e740 20 client.17844 trim_cache size 0 max 16384 2024-02-17T12:56:14.929 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:56:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff525681640 1 -- 172.21.15.18:0/663806136 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff5080a2230 msgr2=0x7ff5080a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff525681640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff5080a2230 0x7ff5080a46f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff5180348f0 tx=0x7ff51802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff525681640 1 -- 172.21.15.18:0/663806136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5201064c0 msgr2=0x7ff52019d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff525681640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5201064c0 0x7ff52019d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff520102290 tx=0x7ff51400ee50 comp rx=0 tx=0).stop 2024-02-17T12:56:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff525681640 1 -- 172.21.15.18:0/663806136 shutdown_connections 2024-02-17T12:56:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff525681640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff5080a2230 0x7ff5080a46f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff525681640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff520197600 0x7ff520197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff525681640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5201064c0 0x7ff52019d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff525681640 1 --2- 172.21.15.18:0/663806136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5200685f0 0x7ff52019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:14.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff525681640 1 -- 172.21.15.18:0/663806136 >> 172.21.15.18:0/663806136 conn(0x7ff520077180 msgr2=0x7ff5200fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:14.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff525681640 1 -- 172.21.15.18:0/663806136 shutdown_connections 2024-02-17T12:56:14.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:14.931+0000 7ff525681640 1 -- 172.21.15.18:0/663806136 wait complete. 2024-02-17T12:56:14.933 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:56:14.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:56:15.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.058+0000 7fe354a64640 1 -- 172.21.15.18:0/1677669016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe350101390 msgr2=0x7fe350101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.058+0000 7fe354a64640 1 --2- 172.21.15.18:0/1677669016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe350101390 0x7fe350101770 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fe340009930 tx=0x7fe34002bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:15.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.058+0000 7fe354a64640 1 -- 172.21.15.18:0/1677669016 shutdown_connections 2024-02-17T12:56:15.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.058+0000 7fe354a64640 1 --2- 172.21.15.18:0/1677669016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe350106460 0x7fe35010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.058+0000 7fe354a64640 1 --2- 172.21.15.18:0/1677669016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe350101d40 0x7fe350105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.058+0000 7fe354a64640 1 --2- 172.21.15.18:0/1677669016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe350101390 0x7fe350101770 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.058+0000 7fe354a64640 1 -- 172.21.15.18:0/1677669016 >> 172.21.15.18:0/1677669016 conn(0x7fe3500fcbb0 msgr2=0x7fe3500fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:15.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.058+0000 7fe354a64640 1 -- 172.21.15.18:0/1677669016 shutdown_connections 2024-02-17T12:56:15.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.058+0000 7fe354a64640 1 -- 172.21.15.18:0/1677669016 wait complete. 2024-02-17T12:56:15.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe354a64640 1 Processor -- start 2024-02-17T12:56:15.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe354a64640 1 -- start start 2024-02-17T12:56:15.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe354a64640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe350101390 0x7fe35019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe354a64640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe350106460 0x7fe35019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe354a64640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe350197750 0x7fe350197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe354a64640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe350198220 con 0x7fe350106460 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe354a64640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe350198390 con 0x7fe350197750 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe354a64640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe350198550 con 0x7fe350101390 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe347fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe350106460 0x7fe35019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe347fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe350106460 0x7fe35019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58414/0 (socket says 172.21.15.18:58414) 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe347fff640 1 -- 172.21.15.18:0/2754096535 learned_addr learned my addr 172.21.15.18:0/2754096535 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe34e59c640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe350101390 0x7fe35019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe34ed9d640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe350197750 0x7fe350197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe347fff640 1 -- 172.21.15.18:0/2754096535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe350101390 msgr2=0x7fe35019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe347fff640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe350101390 0x7fe35019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe347fff640 1 -- 172.21.15.18:0/2754096535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe350197750 msgr2=0x7fe350197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe347fff640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe350197750 0x7fe350197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe347fff640 1 -- 172.21.15.18:0/2754096535 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe340009560 con 0x7fe350106460 2024-02-17T12:56:15.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.059+0000 7fe347fff640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe350106460 0x7fe35019db80 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe33800e9c0 tx=0x7fe33800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:15.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.060+0000 7fe3477fe640 1 -- 172.21.15.18:0/2754096535 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338009700 con 0x7fe350106460 2024-02-17T12:56:15.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.060+0000 7fe3477fe640 1 -- 172.21.15.18:0/2754096535 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe33800cd80 con 0x7fe350106460 2024-02-17T12:56:15.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.060+0000 7fe354a64640 1 -- 172.21.15.18:0/2754096535 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3501a44a0 con 0x7fe350106460 2024-02-17T12:56:15.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.060+0000 7fe3477fe640 1 -- 172.21.15.18:0/2754096535 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338010430 con 0x7fe350106460 2024-02-17T12:56:15.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.060+0000 7fe354a64640 1 -- 172.21.15.18:0/2754096535 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3501a4980 con 0x7fe350106460 2024-02-17T12:56:15.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.062+0000 7fe3477fe640 1 -- 172.21.15.18:0/2754096535 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe338005000 con 0x7fe350106460 2024-02-17T12:56:15.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.063+0000 7fe35763c740 1 Processor -- start 2024-02-17T12:56:15.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.063+0000 7fe35763c740 1 -- start start 2024-02-17T12:56:15.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.063+0000 7fe35763c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c152e8f0 0x5610c152ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.063+0000 7fe35763c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152f2a0 0x5610c151b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.063+0000 7fe35763c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c151ba50 0x5610c151beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.063+0000 7fe35763c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c14dc960 con 0x5610c152e8f0 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.063+0000 7fe35763c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c1490310 con 0x5610c152f2a0 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.063+0000 7fe3477fe640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe33c0a2230 0x7fe33c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.063+0000 7fe35763c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c148d840 con 0x5610c151ba50 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe34e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152f2a0 0x5610c151b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe34ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c152e8f0 0x5610c152ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe34e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152f2a0 0x5610c151b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:56416/0 (socket says 172.21.15.18:56416) 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe34e59c640 1 -- 172.21.15.18:0/3181367599 learned_addr learned my addr 172.21.15.18:0/3181367599 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe347fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c151ba50 0x5610c151beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe34ed9d640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe33c0a2230 0x7fe33c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe3477fe640 1 -- 172.21.15.18:0/2754096535 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe338014070 con 0x7fe350106460 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe34e59c640 1 -- 172.21.15.18:0/3181367599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c151ba50 msgr2=0x5610c151beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe34e59c640 1 --2- 172.21.15.18:0/3181367599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c151ba50 0x5610c151beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe34e59c640 1 -- 172.21.15.18:0/3181367599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c152e8f0 msgr2=0x5610c152ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe34e59c640 1 --2- 172.21.15.18:0/3181367599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c152e8f0 0x5610c152ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe34e59c640 1 -- 172.21.15.18:0/3181367599 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610c144b060 con 0x5610c152f2a0 2024-02-17T12:56:15.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe347fff640 1 --2- 172.21.15.18:0/3181367599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c151ba50 0x5610c151beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:15.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe34ed9d640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe33c0a2230 0x7fe33c0a46f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x5610c151c570 tx=0x7fe348009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:15.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe34e59c640 1 --2- 172.21.15.18:0/3181367599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152f2a0 0x5610c151b430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe340036040 tx=0x7fe34002c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f49f1d7c2218020 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:15.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe344ff9640 1 -- 172.21.15.18:0/3181367599 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340004800 con 0x5610c152f2a0 2024-02-17T12:56:15.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe344ff9640 1 -- 172.21.15.18:0/3181367599 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe340004960 con 0x5610c152f2a0 2024-02-17T12:56:15.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.064+0000 7fe344ff9640 1 -- 172.21.15.18:0/3181367599 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340005050 con 0x5610c152f2a0 2024-02-17T12:56:15.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.065+0000 7fe35763c740 1 -- 172.21.15.18:0/3181367599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152f2a0 msgr2=0x5610c151b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.065+0000 7fe35763c740 1 --2- 172.21.15.18:0/3181367599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152f2a0 0x5610c151b430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe340036040 tx=0x7fe34002c5d0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.065+0000 7fe35763c740 1 -- 172.21.15.18:0/3181367599 shutdown_connections 2024-02-17T12:56:15.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.065+0000 7fe35763c740 1 --2- 172.21.15.18:0/3181367599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c151ba50 0x5610c151beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.065+0000 7fe35763c740 1 --2- 172.21.15.18:0/3181367599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152f2a0 0x5610c151b430 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.065+0000 7fe35763c740 1 --2- 172.21.15.18:0/3181367599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c152e8f0 0x5610c152ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.065+0000 7fe35763c740 1 -- 172.21.15.18:0/3181367599 >> 172.21.15.18:0/3181367599 conn(0x5610c151def0 msgr2=0x5610c151e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:15.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.065+0000 7fe35763c740 1 -- 172.21.15.18:0/3181367599 shutdown_connections 2024-02-17T12:56:15.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.065+0000 7fe35763c740 1 -- 172.21.15.18:0/3181367599 wait complete. 2024-02-17T12:56:15.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe35763c740 1 -- start start 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe35763c740 1 Processor -- start 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe35763c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c151ba50 0x5610c1532850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe35763c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c152e8f0 0x5610c1532d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe35763c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152f2a0 0x5610c15332d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe35763c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c1489020 con 0x5610c152e8f0 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe35763c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c115c430 con 0x5610c151ba50 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe35763c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c1118e00 con 0x5610c152f2a0 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe35763c740 10 client.0 _ll_register_callbacks cb 0x5610c1430cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe347fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c152e8f0 0x5610c1532d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe34e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c151ba50 0x5610c1532850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe34e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c151ba50 0x5610c1532850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:56428/0 (socket says 172.21.15.18:56428) 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe34e59c640 1 -- 172.21.15.18:0/182020194 learned_addr learned my addr 172.21.15.18:0/182020194 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe34ed9d640 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152f2a0 0x5610c15332d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe34e59c640 1 -- 172.21.15.18:0/182020194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152f2a0 msgr2=0x5610c15332d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe34e59c640 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152f2a0 0x5610c15332d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe34e59c640 1 -- 172.21.15.18:0/182020194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c152e8f0 msgr2=0x5610c1532d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe34e59c640 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c152e8f0 0x5610c1532d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe34e59c640 1 -- 172.21.15.18:0/182020194 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35010e840 con 0x5610c151ba50 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.066+0000 7fe347fff640 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c152e8f0 0x5610c1532d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe34e59c640 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c151ba50 0x5610c1532850 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe3400045e0 tx=0x7fe340004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=781fdf3d08748218 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe335ffb640 1 -- 172.21.15.18:0/182020194 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340004800 con 0x5610c151ba50 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe35763c740 1 -- 172.21.15.18:0/182020194 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610c11887e0 con 0x5610c151ba50 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe335ffb640 1 -- 172.21.15.18:0/182020194 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe34004d070 con 0x5610c151ba50 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe335ffb640 1 -- 172.21.15.18:0/182020194 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe340004ad0 con 0x5610c151ba50 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe35763c740 10 client.27119 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe35763c740 1 -- 172.21.15.18:0/182020194 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5610c14876a0 con 0x5610c151ba50 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe335ffb640 1 -- 172.21.15.18:0/182020194 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5610c11887e0 con 0x5610c151ba50 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe335ffb640 1 -- 172.21.15.18:0/182020194 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe34003c050 con 0x5610c151ba50 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe35763c740 10 client.27119 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe35763c740 10 client.27119 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe35763c740 20 client.27119 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.067+0000 7fe35763c740 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5610c1542f00 0x5610c1545310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.068+0000 7fe35763c740 4 client.27119 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.068+0000 7fe35763c740 1 -- 172.21.15.18:0/182020194 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610c11c5720 con 0x5610c1542f00 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.068+0000 7fe347fff640 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5610c1542f00 0x5610c1545310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.068+0000 7fe347fff640 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5610c1542f00 0x5610c1545310 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1982eea95ffa1551 server_cookie=d1872eca3e088d73 in_seq=0 out_seq=0 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.068+0000 7fe335ffb640 10 client.27119 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe335ffb640 1 -- 172.21.15.18:0/182020194 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5610c11c5720 con 0x5610c1542f00 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe335ffb640 10 client.27119 handle_command_reply: tid=0 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe35763c740 1 client.27119 shutdown 2024-02-17T12:56:15.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe35763c740 1 -- 172.21.15.18:0/182020194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c151ba50 msgr2=0x5610c1532850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe35763c740 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c151ba50 0x5610c1532850 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe3400045e0 tx=0x7fe340004610 comp rx=0 tx=0).stop 2024-02-17T12:56:15.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe35763c740 1 -- 172.21.15.18:0/182020194 shutdown_connections 2024-02-17T12:56:15.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe35763c740 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5610c1542f00 0x5610c1545310 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe35763c740 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152f2a0 0x5610c15332d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe35763c740 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c152e8f0 0x5610c1532d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe335ffb640 0 client.27119 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:56:15.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe35763c740 1 --2- 172.21.15.18:0/182020194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c151ba50 0x5610c1532850 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe35763c740 1 -- 172.21.15.18:0/182020194 >> 172.21.15.18:0/182020194 conn(0x5610c151def0 msgr2=0x5610c151cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:15.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe35763c740 1 -- 172.21.15.18:0/182020194 shutdown_connections 2024-02-17T12:56:15.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe35763c740 1 -- 172.21.15.18:0/182020194 wait complete. 2024-02-17T12:56:15.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.069+0000 7fe35763c740 20 client.27119 trim_cache size 0 max 16384 2024-02-17T12:56:15.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.082+0000 7fe35763c740 1 Processor -- start 2024-02-17T12:56:15.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.082+0000 7fe35763c740 1 -- start start 2024-02-17T12:56:15.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.082+0000 7fe35763c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 0x5610c1531120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.082+0000 7fe35763c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152e8f0 0x5610c1531660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.082+0000 7fe35763c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152f2a0 0x5610c152d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.082+0000 7fe35763c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c140e130 con 0x5610c151ba50 2024-02-17T12:56:15.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.082+0000 7fe35763c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c11887e0 con 0x5610c152f2a0 2024-02-17T12:56:15.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.082+0000 7fe35763c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c1533820 con 0x5610c152e8f0 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe34e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 0x5610c1531120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe34e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 0x5610c1531120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58428/0 (socket says 172.21.15.18:58428) 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe34e59c640 1 -- 172.21.15.18:0/163052172 learned_addr learned my addr 172.21.15.18:0/163052172 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe34ed9d640 1 --2- 172.21.15.18:0/163052172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152f2a0 0x5610c152d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe347fff640 1 --2- 172.21.15.18:0/163052172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152e8f0 0x5610c1531660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe34e59c640 1 -- 172.21.15.18:0/163052172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152e8f0 msgr2=0x5610c1531660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe34e59c640 1 --2- 172.21.15.18:0/163052172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152e8f0 0x5610c1531660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe34e59c640 1 -- 172.21.15.18:0/163052172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152f2a0 msgr2=0x5610c152d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe34e59c640 1 --2- 172.21.15.18:0/163052172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152f2a0 0x5610c152d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe34e59c640 1 -- 172.21.15.18:0/163052172 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610c1423c20 con 0x5610c151ba50 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe34e59c640 1 --2- 172.21.15.18:0/163052172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 0x5610c1531120 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe340000c00 tx=0x7fe34002cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d298df111e0e6a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe337fff640 1 -- 172.21.15.18:0/163052172 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340004ad0 con 0x5610c151ba50 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe337fff640 1 -- 172.21.15.18:0/163052172 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3400046c0 con 0x5610c151ba50 2024-02-17T12:56:15.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.083+0000 7fe337fff640 1 -- 172.21.15.18:0/163052172 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34003faa0 con 0x5610c151ba50 2024-02-17T12:56:15.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.084+0000 7fe35763c740 1 -- 172.21.15.18:0/163052172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 msgr2=0x5610c1531120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.084+0000 7fe35763c740 1 --2- 172.21.15.18:0/163052172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 0x5610c1531120 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe340000c00 tx=0x7fe34002cb10 comp rx=0 tx=0).stop 2024-02-17T12:56:15.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.084+0000 7fe35763c740 1 -- 172.21.15.18:0/163052172 shutdown_connections 2024-02-17T12:56:15.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.084+0000 7fe35763c740 1 --2- 172.21.15.18:0/163052172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152f2a0 0x5610c152d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.084+0000 7fe35763c740 1 --2- 172.21.15.18:0/163052172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152e8f0 0x5610c1531660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.084+0000 7fe35763c740 1 --2- 172.21.15.18:0/163052172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 0x5610c1531120 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.084+0000 7fe35763c740 1 -- 172.21.15.18:0/163052172 >> 172.21.15.18:0/163052172 conn(0x5610c151def0 msgr2=0x5610c1549c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:15.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.084+0000 7fe35763c740 1 -- 172.21.15.18:0/163052172 shutdown_connections 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.084+0000 7fe35763c740 1 -- 172.21.15.18:0/163052172 wait complete. 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.084+0000 7fe35763c740 1 -- start start 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.084+0000 7fe35763c740 1 Processor -- start 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe35763c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 0x5610c1539dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe35763c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152e8f0 0x5610c153a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe35763c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152f2a0 0x5610c15383f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe35763c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c1538ac0 con 0x5610c151ba50 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe35763c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c1538c30 con 0x5610c152e8f0 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe35763c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c1538df0 con 0x5610c152f2a0 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe35763c740 10 client.0 _ll_register_callbacks cb 0x7fe3480093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe34e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 0x5610c1539dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe34e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 0x5610c1539dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58438/0 (socket says 172.21.15.18:58438) 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe34e59c640 1 -- 172.21.15.18:0/816442077 learned_addr learned my addr 172.21.15.18:0/816442077 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe347fff640 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152e8f0 0x5610c153a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe34ed9d640 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152f2a0 0x5610c15383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe34e59c640 1 -- 172.21.15.18:0/816442077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152f2a0 msgr2=0x5610c15383f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe34e59c640 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152f2a0 0x5610c15383f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe34e59c640 1 -- 172.21.15.18:0/816442077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152e8f0 msgr2=0x5610c153a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe34e59c640 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152e8f0 0x5610c153a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe34e59c640 1 -- 172.21.15.18:0/816442077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610c14876a0 con 0x5610c151ba50 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe34ed9d640 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152f2a0 0x5610c15383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe34e59c640 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 0x5610c1539dd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe340009400 tx=0x7fe34003f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc854df7f594415 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe335ffb640 1 -- 172.21.15.18:0/816442077 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34003f440 con 0x5610c151ba50 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe335ffb640 1 -- 172.21.15.18:0/816442077 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe34004d070 con 0x5610c151ba50 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.085+0000 7fe35763c740 1 -- 172.21.15.18:0/816442077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe340004960 con 0x5610c151ba50 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.086+0000 7fe335ffb640 1 -- 172.21.15.18:0/816442077 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe340004ad0 con 0x5610c151ba50 2024-02-17T12:56:15.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.086+0000 7fe35763c740 10 client.17862 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.086+0000 7fe35763c740 1 -- 172.21.15.18:0/816442077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5610c14498c0 con 0x5610c151ba50 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.086+0000 7fe335ffb640 1 -- 172.21.15.18:0/816442077 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe340004960 con 0x5610c151ba50 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.086+0000 7fe335ffb640 1 -- 172.21.15.18:0/816442077 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe34003c040 con 0x5610c151ba50 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.086+0000 7fe35763c740 10 client.17862 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.086+0000 7fe35763c740 10 client.17862 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.086+0000 7fe35763c740 20 client.17862 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.086+0000 7fe35763c740 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5610c1542f00 0x5610c159c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.086+0000 7fe35763c740 4 client.17862 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.086+0000 7fe35763c740 1 -- 172.21.15.18:0/816442077 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5610c1214890 con 0x5610c1542f00 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.086+0000 7fe347fff640 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5610c1542f00 0x5610c159c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe347fff640 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5610c1542f00 0x5610c159c2e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3193abc0e5ce96d server_cookie=53b0a533959a77d4 in_seq=0 out_seq=0 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe335ffb640 10 client.17862 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:56:15.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe335ffb640 1 -- 172.21.15.18:0/816442077 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5610c1214890 con 0x5610c1542f00 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe335ffb640 10 client.17862 handle_command_reply: tid=0 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe35763c740 1 client.17862 shutdown 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe35763c740 1 -- 172.21.15.18:0/816442077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 msgr2=0x5610c1539dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe35763c740 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 0x5610c1539dd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe340009400 tx=0x7fe34003f5e0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe35763c740 1 -- 172.21.15.18:0/816442077 shutdown_connections 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe35763c740 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5610c1542f00 0x5610c159c2e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe335ffb640 0 client.17862 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe35763c740 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5610c152f2a0 0x5610c15383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe35763c740 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5610c152e8f0 0x5610c153a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe35763c740 1 --2- 172.21.15.18:0/816442077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5610c151ba50 0x5610c1539dd0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe35763c740 1 -- 172.21.15.18:0/816442077 >> 172.21.15.18:0/816442077 conn(0x5610c151def0 msgr2=0x5610c1531060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe35763c740 1 -- 172.21.15.18:0/816442077 shutdown_connections 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe35763c740 1 -- 172.21.15.18:0/816442077 wait complete. 2024-02-17T12:56:15.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.087+0000 7fe35763c740 20 client.17862 trim_cache size 0 max 16384 2024-02-17T12:56:15.085 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:56:15.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.088+0000 7fe354a64640 1 -- 172.21.15.18:0/2754096535 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe33c0a2230 msgr2=0x7fe33c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.088+0000 7fe354a64640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe33c0a2230 0x7fe33c0a46f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x5610c151c570 tx=0x7fe348009500 comp rx=0 tx=0).stop 2024-02-17T12:56:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.088+0000 7fe354a64640 1 -- 172.21.15.18:0/2754096535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe350106460 msgr2=0x7fe35019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.088+0000 7fe354a64640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe350106460 0x7fe35019db80 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe33800e9c0 tx=0x7fe33800ee80 comp rx=0 tx=0).stop 2024-02-17T12:56:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.088+0000 7fe354a64640 1 -- 172.21.15.18:0/2754096535 shutdown_connections 2024-02-17T12:56:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.088+0000 7fe354a64640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe33c0a2230 0x7fe33c0a46f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.088+0000 7fe354a64640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe350197750 0x7fe350197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.088+0000 7fe354a64640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe350106460 0x7fe35019db80 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.088+0000 7fe354a64640 1 --2- 172.21.15.18:0/2754096535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe350101390 0x7fe35019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.088+0000 7fe354a64640 1 -- 172.21.15.18:0/2754096535 >> 172.21.15.18:0/2754096535 conn(0x7fe3500fcbb0 msgr2=0x7fe3500fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.088+0000 7fe354a64640 1 -- 172.21.15.18:0/2754096535 shutdown_connections 2024-02-17T12:56:15.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.088+0000 7fe354a64640 1 -- 172.21.15.18:0/2754096535 wait complete. 2024-02-17T12:56:15.089 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:14 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/861497329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:56:15.090 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:56:15.090 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:56:15.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:14 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/861497329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:56:15.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:14 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/861497329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:56:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.212+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1870360516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5f8101390 msgr2=0x7fb5f8101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.212+0000 7fb5fdf4f640 1 --2- 172.21.15.18:0/1870360516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5f8101390 0x7fb5f8101770 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009930 tx=0x7fb5e802bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.212+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1870360516 shutdown_connections 2024-02-17T12:56:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.212+0000 7fb5fdf4f640 1 --2- 172.21.15.18:0/1870360516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5f8106460 0x7fb5f810e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.212+0000 7fb5fdf4f640 1 --2- 172.21.15.18:0/1870360516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f8101d40 0x7fb5f8105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.212+0000 7fb5fdf4f640 1 --2- 172.21.15.18:0/1870360516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5f8101390 0x7fb5f8101770 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.212+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1870360516 >> 172.21.15.18:0/1870360516 conn(0x7fb5f80fcbb0 msgr2=0x7fb5f80fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.212+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1870360516 shutdown_connections 2024-02-17T12:56:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.212+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1870360516 wait complete. 2024-02-17T12:56:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5fdf4f640 1 Processor -- start 2024-02-17T12:56:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5fdf4f640 1 -- start start 2024-02-17T12:56:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5fdf4f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f8101390 0x7fb5f819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5fdf4f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5f8106460 0x7fb5f819db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5fdf4f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5f81976f0 0x7fb5f8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5fdf4f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f81981c0 con 0x7fb5f8106460 2024-02-17T12:56:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5fdf4f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8198330 con 0x7fb5f8101390 2024-02-17T12:56:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5fdf4f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f81984f0 con 0x7fb5f81976f0 2024-02-17T12:56:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5f6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5f8106460 0x7fb5f819db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5f6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5f8106460 0x7fb5f819db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58454/0 (socket says 172.21.15.18:58454) 2024-02-17T12:56:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5f6ffd640 1 -- 172.21.15.18:0/1577485853 learned_addr learned my addr 172.21.15.18:0/1577485853 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f8101390 0x7fb5f819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5f7fff640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5f81976f0 0x7fb5f8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5f6ffd640 1 -- 172.21.15.18:0/1577485853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5f81976f0 msgr2=0x7fb5f8197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5f6ffd640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5f81976f0 0x7fb5f8197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5f6ffd640 1 -- 172.21.15.18:0/1577485853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f8101390 msgr2=0x7fb5f819d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.213+0000 7fb5f6ffd640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f8101390 0x7fb5f819d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.214+0000 7fb5f6ffd640 1 -- 172.21.15.18:0/1577485853 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e8009560 con 0x7fb5f8106460 2024-02-17T12:56:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.214+0000 7fb5f6ffd640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5f8106460 0x7fb5f819db20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00e9c0 tx=0x7fb5ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.214+0000 7fb5f4ff9640 1 -- 172.21.15.18:0/1577485853 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec009700 con 0x7fb5f8106460 2024-02-17T12:56:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.214+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1577485853 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f81a4440 con 0x7fb5f8106460 2024-02-17T12:56:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.214+0000 7fb5f4ff9640 1 -- 172.21.15.18:0/1577485853 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5ec00cd80 con 0x7fb5f8106460 2024-02-17T12:56:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.214+0000 7fb5f4ff9640 1 -- 172.21.15.18:0/1577485853 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec010430 con 0x7fb5f8106460 2024-02-17T12:56:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.214+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1577485853 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f81a4920 con 0x7fb5f8106460 2024-02-17T12:56:15.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.216+0000 7fb5f4ff9640 1 -- 172.21.15.18:0/1577485853 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb5ec005000 con 0x7fb5f8106460 2024-02-17T12:56:15.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.216+0000 7fb600c00740 1 Processor -- start 2024-02-17T12:56:15.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb600c00740 1 -- start start 2024-02-17T12:56:15.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f4ff9640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb5e00a2230 0x7fb5e00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb600c00740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769c250 0x55753769c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f4ff9640 1 -- 172.21.15.18:0/1577485853 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb5ec014070 con 0x7fb5f8106460 2024-02-17T12:56:15.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb600c00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55753769cb70 0x5575376a5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f7fff640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb5e00a2230 0x7fb5e00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb600c00740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575376a5750 0x5575376a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb600c00740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557537657960 con 0x5575376a5750 2024-02-17T12:56:15.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb600c00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753760b310 con 0x55753769cb70 2024-02-17T12:56:15.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb600c00740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557537608840 con 0x55753769c250 2024-02-17T12:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575376a5750 0x5575376a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55753769cb70 0x5575376a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575376a5750 0x5575376a7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58470/0 (socket says 172.21.15.18:58470) 2024-02-17T12:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f77fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769c250 0x55753769c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f7fff640 1 -- 172.21.15.18:0/1007378112 learned_addr learned my addr 172.21.15.18:0/1007378112 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f77fe640 1 -- 172.21.15.18:0/1007378112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55753769cb70 msgr2=0x5575376a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/1007378112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55753769cb70 0x5575376a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f77fe640 1 -- 172.21.15.18:0/1007378112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575376a5750 msgr2=0x5575376a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f7fff640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb5e00a2230 0x7fb5e00a46f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0014820 tx=0x7fb5f0014c20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/1007378112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575376a5750 0x5575376a7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f77fe640 1 -- 172.21.15.18:0/1007378112 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575375c6060 con 0x55753769c250 2024-02-17T12:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.217+0000 7fb5f6ffd640 1 --2- 172.21.15.18:0/1007378112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55753769cb70 0x5575376a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.218+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/1007378112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769c250 0x55753769c630 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009230 tx=0x7fb5e802c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56aa5a3b396be732 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.218+0000 7fb5ddffb640 1 -- 172.21.15.18:0/1007378112 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8004800 con 0x55753769c250 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.218+0000 7fb5ddffb640 1 -- 172.21.15.18:0/1007378112 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5e8004960 con 0x55753769c250 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.218+0000 7fb5ddffb640 1 -- 172.21.15.18:0/1007378112 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8005030 con 0x55753769c250 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.219+0000 7fb600c00740 1 -- 172.21.15.18:0/1007378112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769c250 msgr2=0x55753769c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.219+0000 7fb600c00740 1 --2- 172.21.15.18:0/1007378112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769c250 0x55753769c630 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009230 tx=0x7fb5e802c680 comp rx=0 tx=0).stop 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.219+0000 7fb600c00740 1 -- 172.21.15.18:0/1007378112 shutdown_connections 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.219+0000 7fb600c00740 1 --2- 172.21.15.18:0/1007378112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575376a5750 0x5575376a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.219+0000 7fb600c00740 1 --2- 172.21.15.18:0/1007378112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55753769cb70 0x5575376a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.219+0000 7fb600c00740 1 --2- 172.21.15.18:0/1007378112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769c250 0x55753769c630 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.219+0000 7fb600c00740 1 -- 172.21.15.18:0/1007378112 >> 172.21.15.18:0/1007378112 conn(0x557537696980 msgr2=0x557537696d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.219+0000 7fb600c00740 1 -- 172.21.15.18:0/1007378112 shutdown_connections 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.219+0000 7fb600c00740 1 -- 172.21.15.18:0/1007378112 wait complete. 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.219+0000 7fb600c00740 1 -- start start 2024-02-17T12:56:15.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.219+0000 7fb600c00740 1 Processor -- start 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb600c00740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb600c00740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769cb70 0x5575376a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb600c00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575376a5750 0x5575376a3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb600c00740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557537604020 con 0x55753769c250 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb600c00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575372d7430 con 0x5575376a5750 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb600c00740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557537293e00 con 0x55753769cb70 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb600c00740 10 client.0 _ll_register_callbacks cb 0x5575375abcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376a26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58482/0 (socket says 172.21.15.18:58482) 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5f77fe640 1 -- 172.21.15.18:0/3063794617 learned_addr learned my addr 172.21.15.18:0/3063794617 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5f6ffd640 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769cb70 0x5575376a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5f7fff640 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575376a5750 0x5575376a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5f77fe640 1 -- 172.21.15.18:0/3063794617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769cb70 msgr2=0x5575376a2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769cb70 0x5575376a2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5f77fe640 1 -- 172.21.15.18:0/3063794617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575376a5750 msgr2=0x5575376a3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575376a5750 0x5575376a3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5f77fe640 1 -- 172.21.15.18:0/3063794617 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f810e840 con 0x55753769c250 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5f6ffd640 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769cb70 0x5575376a2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376a26b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb5e802c170 tx=0x7fb5e80044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7059c6e3eabbe565 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5d6ffd640 1 -- 172.21.15.18:0/3063794617 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8005200 con 0x55753769c250 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb5d6ffd640 1 -- 172.21.15.18:0/3063794617 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb5e8005360 con 0x55753769c250 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.220+0000 7fb600c00740 1 -- 172.21.15.18:0/3063794617 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575373037e0 con 0x55753769c250 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.221+0000 7fb5d6ffd640 1 -- 172.21.15.18:0/3063794617 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5e8004ad0 con 0x55753769c250 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.221+0000 7fb600c00740 10 client.17874 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.221+0000 7fb600c00740 1 -- 172.21.15.18:0/3063794617 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5575376026a0 con 0x55753769c250 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.221+0000 7fb5d6ffd640 1 -- 172.21.15.18:0/3063794617 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5575373037e0 con 0x55753769c250 2024-02-17T12:56:15.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.221+0000 7fb5d6ffd640 1 -- 172.21.15.18:0/3063794617 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb5e803c050 con 0x55753769c250 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.221+0000 7fb600c00740 10 client.17874 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.221+0000 7fb600c00740 10 client.17874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.221+0000 7fb600c00740 20 client.17874 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.221+0000 7fb600c00740 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5575376b56d0 0x5575376b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.221+0000 7fb600c00740 4 client.17874 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.221+0000 7fb600c00740 1 -- 172.21.15.18:0/3063794617 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557537340720 con 0x5575376b56d0 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.221+0000 7fb5f6ffd640 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5575376b56d0 0x5575376b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.222+0000 7fb5f6ffd640 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5575376b56d0 0x5575376b7af0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739118bd279b9c6b server_cookie=ca9176cc7dda5ce3 in_seq=0 out_seq=0 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.222+0000 7fb5d6ffd640 10 client.17874 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.222+0000 7fb5d6ffd640 1 -- 172.21.15.18:0/3063794617 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557537340720 con 0x5575376b56d0 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.222+0000 7fb5d6ffd640 10 client.17874 handle_command_reply: tid=0 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.222+0000 7fb600c00740 1 client.17874 shutdown 2024-02-17T12:56:15.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.223+0000 7fb600c00740 1 -- 172.21.15.18:0/3063794617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 msgr2=0x5575376a26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.223+0000 7fb600c00740 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376a26b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb5e802c170 tx=0x7fb5e80044a0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.223+0000 7fb600c00740 1 -- 172.21.15.18:0/3063794617 shutdown_connections 2024-02-17T12:56:15.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.223+0000 7fb600c00740 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5575376b56d0 0x5575376b7af0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.223+0000 7fb600c00740 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575376a5750 0x5575376a3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.223+0000 7fb600c00740 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769cb70 0x5575376a2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.223+0000 7fb5d6ffd640 0 client.17874 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:15.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.223+0000 7fb600c00740 1 --2- 172.21.15.18:0/3063794617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376a26b0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.223+0000 7fb600c00740 1 -- 172.21.15.18:0/3063794617 >> 172.21.15.18:0/3063794617 conn(0x557537696980 msgr2=0x5575376986d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:15.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.223+0000 7fb600c00740 1 -- 172.21.15.18:0/3063794617 shutdown_connections 2024-02-17T12:56:15.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.223+0000 7fb600c00740 1 -- 172.21.15.18:0/3063794617 wait complete. 2024-02-17T12:56:15.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.223+0000 7fb600c00740 20 client.17874 trim_cache size 0 max 16384 2024-02-17T12:56:15.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb600c00740 1 Processor -- start 2024-02-17T12:56:15.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb600c00740 1 -- start start 2024-02-17T12:56:15.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb600c00740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb600c00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55753769cb70 0x557537698490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb600c00740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575376a5750 0x5575376ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb600c00740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557537589130 con 0x55753769c250 2024-02-17T12:56:15.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb600c00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557537697980 con 0x55753769cb70 2024-02-17T12:56:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb600c00740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575376b2530 con 0x5575376a5750 2024-02-17T12:56:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb5f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb5f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376a2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58494/0 (socket says 172.21.15.18:58494) 2024-02-17T12:56:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb5f77fe640 1 -- 172.21.15.18:0/298970348 learned_addr learned my addr 172.21.15.18:0/298970348 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb5f6ffd640 1 --2- 172.21.15.18:0/298970348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55753769cb70 0x557537698490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb5f7fff640 1 --2- 172.21.15.18:0/298970348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575376a5750 0x5575376ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb5f77fe640 1 -- 172.21.15.18:0/298970348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575376a5750 msgr2=0x5575376ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/298970348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575376a5750 0x5575376ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb5f77fe640 1 -- 172.21.15.18:0/298970348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55753769cb70 msgr2=0x557537698490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/298970348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55753769cb70 0x557537698490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb5f77fe640 1 -- 172.21.15.18:0/298970348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55753759ec20 con 0x55753769c250 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.235+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/298970348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376a2f20 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009230 tx=0x7fb5e8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318454d307a93415 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.236+0000 7fb5dd7fa640 1 -- 172.21.15.18:0/298970348 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e803fb60 con 0x55753769c250 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.236+0000 7fb5dd7fa640 1 -- 172.21.15.18:0/298970348 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5e8004ad0 con 0x55753769c250 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.236+0000 7fb5dd7fa640 1 -- 172.21.15.18:0/298970348 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e805c460 con 0x55753769c250 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.236+0000 7fb600c00740 1 -- 172.21.15.18:0/298970348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 msgr2=0x5575376a2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.236+0000 7fb600c00740 1 --2- 172.21.15.18:0/298970348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376a2f20 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009230 tx=0x7fb5e8005fd0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.236+0000 7fb600c00740 1 -- 172.21.15.18:0/298970348 shutdown_connections 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.236+0000 7fb600c00740 1 --2- 172.21.15.18:0/298970348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575376a5750 0x5575376ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.236+0000 7fb600c00740 1 --2- 172.21.15.18:0/298970348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55753769cb70 0x557537698490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.236+0000 7fb600c00740 1 --2- 172.21.15.18:0/298970348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376a2f20 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.236+0000 7fb600c00740 1 -- 172.21.15.18:0/298970348 >> 172.21.15.18:0/298970348 conn(0x557537696980 msgr2=0x557537697d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:15.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.236+0000 7fb600c00740 1 -- 172.21.15.18:0/298970348 shutdown_connections 2024-02-17T12:56:15.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.236+0000 7fb600c00740 1 -- 172.21.15.18:0/298970348 wait complete. 2024-02-17T12:56:15.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb600c00740 1 -- start start 2024-02-17T12:56:15.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb600c00740 1 Processor -- start 2024-02-17T12:56:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb600c00740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb600c00740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769cb70 0x5575376ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb600c00740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575376a5750 0x5575376aba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb600c00740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753770bbe0 con 0x55753769c250 2024-02-17T12:56:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb600c00740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753770bd20 con 0x5575376a5750 2024-02-17T12:56:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb600c00740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753770be60 con 0x55753769cb70 2024-02-17T12:56:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb600c00740 10 client.0 _ll_register_callbacks cb 0x7fb5e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb5f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb5f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376aaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58510/0 (socket says 172.21.15.18:58510) 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb5f77fe640 1 -- 172.21.15.18:0/750061500 learned_addr learned my addr 172.21.15.18:0/750061500 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb5f6ffd640 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769cb70 0x5575376ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb5f7fff640 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575376a5750 0x5575376aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb5f77fe640 1 -- 172.21.15.18:0/750061500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769cb70 msgr2=0x5575376ab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769cb70 0x5575376ab4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb5f77fe640 1 -- 172.21.15.18:0/750061500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575376a5750 msgr2=0x5575376aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.237+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575376a5750 0x5575376aba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb5f77fe640 1 -- 172.21.15.18:0/750061500 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575376026a0 con 0x55753769c250 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb5f7fff640 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575376a5750 0x5575376aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb5f77fe640 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376aaf80 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb5e803f770 tx=0x7fb5e803f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976e670044a16e85 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb5d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb5d6ffd640 1 -- 172.21.15.18:0/750061500 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8005740 con 0x55753769c250 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb5d6ffd640 1 -- 172.21.15.18:0/750061500 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb5e8000e40 con 0x55753769c250 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb600c00740 1 -- 172.21.15.18:0/750061500 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5e8004960 con 0x55753769c250 2024-02-17T12:56:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb5d6ffd640 1 -- 172.21.15.18:0/750061500 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5e802c5d0 con 0x55753769c250 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb600c00740 10 client.17880 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb600c00740 1 -- 172.21.15.18:0/750061500 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5575375c48c0 con 0x55753769c250 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb5d6ffd640 1 -- 172.21.15.18:0/750061500 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5e8004960 con 0x55753769c250 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb5d6ffd640 1 -- 172.21.15.18:0/750061500 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb5e803c070 con 0x55753769c250 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb600c00740 10 client.17880 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb600c00740 10 client.17880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb600c00740 20 client.17880 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb600c00740 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55753770d210 0x55753770f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb600c00740 4 client.17880 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.238+0000 7fb600c00740 1 -- 172.21.15.18:0/750061500 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55753738f890 con 0x55753770d210 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.239+0000 7fb5f6ffd640 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55753770d210 0x55753770f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.239+0000 7fb5d6ffd640 10 client.17880 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.239+0000 7fb5f6ffd640 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55753770d210 0x55753770f630 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cc2f601544174c server_cookie=c9f4c0301f1b7d85 in_seq=0 out_seq=0 2024-02-17T12:56:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb5d6ffd640 1 -- 172.21.15.18:0/750061500 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55753738f890 con 0x55753770d210 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb5d6ffd640 10 client.17880 handle_command_reply: tid=0 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb600c00740 1 client.17880 shutdown 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb600c00740 1 -- 172.21.15.18:0/750061500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 msgr2=0x5575376aaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb600c00740 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376aaf80 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb5e803f770 tx=0x7fb5e803f6b0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb600c00740 1 -- 172.21.15.18:0/750061500 shutdown_connections 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb600c00740 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55753770d210 0x55753770f630 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb600c00740 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575376a5750 0x5575376aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb600c00740 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55753769cb70 0x5575376ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb5d6ffd640 0 client.17880 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb600c00740 1 --2- 172.21.15.18:0/750061500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753769c250 0x5575376aaf80 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb600c00740 1 -- 172.21.15.18:0/750061500 >> 172.21.15.18:0/750061500 conn(0x557537696980 msgr2=0x557537695b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb600c00740 1 -- 172.21.15.18:0/750061500 shutdown_connections 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb600c00740 1 -- 172.21.15.18:0/750061500 wait complete. 2024-02-17T12:56:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.240+0000 7fb600c00740 20 client.17880 trim_cache size 0 max 16384 2024-02-17T12:56:15.238 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:56:15.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.241+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1577485853 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb5e00a2230 msgr2=0x7fb5e00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.241+0000 7fb5fdf4f640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb5e00a2230 0x7fb5e00a46f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0014820 tx=0x7fb5f0014c20 comp rx=0 tx=0).stop 2024-02-17T12:56:15.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.241+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1577485853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5f8106460 msgr2=0x7fb5f819db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:15.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.241+0000 7fb5fdf4f640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5f8106460 0x7fb5f819db20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00e9c0 tx=0x7fb5ec00ee80 comp rx=0 tx=0).stop 2024-02-17T12:56:15.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.241+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1577485853 shutdown_connections 2024-02-17T12:56:15.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.241+0000 7fb5fdf4f640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb5e00a2230 0x7fb5e00a46f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.241+0000 7fb5fdf4f640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5f81976f0 0x7fb5f8197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.241+0000 7fb5fdf4f640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5f8106460 0x7fb5f819db20 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.241+0000 7fb5fdf4f640 1 --2- 172.21.15.18:0/1577485853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5f8101390 0x7fb5f819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:15.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.241+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1577485853 >> 172.21.15.18:0/1577485853 conn(0x7fb5f80fcbb0 msgr2=0x7fb5f80fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:15.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.241+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1577485853 shutdown_connections 2024-02-17T12:56:15.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:15.241+0000 7fb5fdf4f640 1 -- 172.21.15.18:0/1577485853 wait complete. 2024-02-17T12:56:15.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 338403 5.97 MB/sec execute 210 sec latency 19.143 ms 2024-02-17T12:56:16.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:15 smithi119 ceph-mon[36188]: pgmap v197: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 222 op/s 2024-02-17T12:56:16.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:15 smithi146 ceph-mon[36454]: pgmap v197: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 222 op/s 2024-02-17T12:56:16.243 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:56:16.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:56:16.243 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:56:16.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:15 smithi018 ceph-mon[32625]: pgmap v197: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 222 op/s 2024-02-17T12:56:16.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 338670 5.95 MB/sec execute 211 sec latency 725.903 ms 2024-02-17T12:56:16.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.369+0000 7f65221f4640 1 -- 172.21.15.18:0/363423129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f651c101390 msgr2=0x7f651c101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.369+0000 7f65221f4640 1 --2- 172.21.15.18:0/363423129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f651c101390 0x7f651c101770 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f650c009930 tx=0x7f650c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.369+0000 7f65221f4640 1 -- 172.21.15.18:0/363423129 shutdown_connections 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.369+0000 7f65221f4640 1 --2- 172.21.15.18:0/363423129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c106460 0x7f651c10e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.369+0000 7f65221f4640 1 --2- 172.21.15.18:0/363423129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f651c101d40 0x7f651c105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.369+0000 7f65221f4640 1 --2- 172.21.15.18:0/363423129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f651c101390 0x7f651c101770 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.369+0000 7f65221f4640 1 -- 172.21.15.18:0/363423129 >> 172.21.15.18:0/363423129 conn(0x7f651c0fcbb0 msgr2=0x7f651c0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.369+0000 7f65221f4640 1 -- 172.21.15.18:0/363423129 shutdown_connections 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.369+0000 7f65221f4640 1 -- 172.21.15.18:0/363423129 wait complete. 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.370+0000 7f65221f4640 1 Processor -- start 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.370+0000 7f65221f4640 1 -- start start 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.370+0000 7f65221f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f651c101390 0x7f651c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.370+0000 7f65221f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c106460 0x7f651c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.370+0000 7f65221f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f651c1976a0 0x7f651c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.370+0000 7f65221f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c198170 con 0x7f651c101390 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.370+0000 7f65221f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c1982e0 con 0x7f651c106460 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.370+0000 7f65221f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c1984a0 con 0x7f651c1976a0 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f651b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f651c101390 0x7f651c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f651b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f651c101390 0x7f651c19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58530/0 (socket says 172.21.15.18:58530) 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f651b7fe640 1 -- 172.21.15.18:0/823521172 learned_addr learned my addr 172.21.15.18:0/823521172 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f651affd640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c106460 0x7f651c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f651bfff640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f651c1976a0 0x7f651c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f651b7fe640 1 -- 172.21.15.18:0/823521172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f651c1976a0 msgr2=0x7f651c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f651b7fe640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f651c1976a0 0x7f651c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f651b7fe640 1 -- 172.21.15.18:0/823521172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c106460 msgr2=0x7f651c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f651b7fe640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c106460 0x7f651c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f651b7fe640 1 -- 172.21.15.18:0/823521172 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c009560 con 0x7f651c101390 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f651b7fe640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f651c101390 0x7f651c19d590 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f650c009230 tx=0x7f650c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:16.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f6518ff9640 1 -- 172.21.15.18:0/823521172 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c004440 con 0x7f651c101390 2024-02-17T12:56:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f65221f4640 1 -- 172.21.15.18:0/823521172 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c1a4390 con 0x7f651c101390 2024-02-17T12:56:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f6518ff9640 1 -- 172.21.15.18:0/823521172 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f650c0052d0 con 0x7f651c101390 2024-02-17T12:56:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f6518ff9640 1 -- 172.21.15.18:0/823521172 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c003d70 con 0x7f651c101390 2024-02-17T12:56:16.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.371+0000 7f65221f4640 1 -- 172.21.15.18:0/823521172 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c1a48c0 con 0x7f651c101390 2024-02-17T12:56:16.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.374+0000 7f6518ff9640 1 -- 172.21.15.18:0/823521172 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f650c005440 con 0x7f651c101390 2024-02-17T12:56:16.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.374+0000 7f6524dc6740 1 Processor -- start 2024-02-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.374+0000 7f6524dc6740 1 -- start start 2024-02-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.374+0000 7f6524dc6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8ee8b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.374+0000 7f6524dc6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8bd30 0x559a8ee942e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f6524dc6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee948b0 0x559a8ee96d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f6524dc6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8edf9f90 con 0x559a8ee8b380 2024-02-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f6524dc6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8edf74c0 con 0x559a8ee8bd30 2024-02-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f6524dc6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8edf2ca0 con 0x559a8ee948b0 2024-02-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f6518ff9640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f65040a2230 0x7f65040a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8ee8b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8ee8b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58538/0 (socket says 172.21.15.18:58538) 2024-02-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651bfff640 1 -- 172.21.15.18:0/1744598716 learned_addr learned my addr 172.21.15.18:0/1744598716 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:16.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651bfff640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f65040a2230 0x7f65040a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f6518ff9640 1 -- 172.21.15.18:0/823521172 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f650c0f0120 con 0x7f651c101390 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651affd640 1 --2- 172.21.15.18:0/1744598716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee948b0 0x559a8ee96d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651b7fe640 1 --2- 172.21.15.18:0/1744598716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8bd30 0x559a8ee942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651bfff640 1 -- 172.21.15.18:0/1744598716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee948b0 msgr2=0x559a8ee96d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651bfff640 1 --2- 172.21.15.18:0/1744598716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee948b0 0x559a8ee96d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651bfff640 1 -- 172.21.15.18:0/1744598716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8bd30 msgr2=0x559a8ee942e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651bfff640 1 --2- 172.21.15.18:0/1744598716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8bd30 0x559a8ee942e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651bfff640 1 -- 172.21.15.18:0/1744598716 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a8edb50b0 con 0x559a8ee8b380 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651bfff640 1 --2- 172.21.15.18:0/1744598716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8ee8b760 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6514012d00 tx=0x7f6514010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20976044e243535 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.375+0000 7f651bfff640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f65040a2230 0x7f65040a46f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6514010c40 tx=0x7f651400c000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.376+0000 7f6501ffb640 1 -- 172.21.15.18:0/1744598716 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514012eb0 con 0x559a8ee8b380 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.376+0000 7f6501ffb640 1 -- 172.21.15.18:0/1744598716 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6514012520 con 0x559a8ee8b380 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.376+0000 7f6501ffb640 1 -- 172.21.15.18:0/1744598716 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514005240 con 0x559a8ee8b380 2024-02-17T12:56:16.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.376+0000 7f6524dc6740 1 -- 172.21.15.18:0/1744598716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 msgr2=0x559a8ee8b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.376+0000 7f6524dc6740 1 --2- 172.21.15.18:0/1744598716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8ee8b760 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6514012d00 tx=0x7f6514010740 comp rx=0 tx=0).stop 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.376+0000 7f6524dc6740 1 -- 172.21.15.18:0/1744598716 shutdown_connections 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.376+0000 7f6524dc6740 1 --2- 172.21.15.18:0/1744598716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee948b0 0x559a8ee96d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.376+0000 7f6524dc6740 1 --2- 172.21.15.18:0/1744598716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8bd30 0x559a8ee942e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.376+0000 7f6524dc6740 1 --2- 172.21.15.18:0/1744598716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8ee8b760 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.376+0000 7f6524dc6740 1 -- 172.21.15.18:0/1744598716 >> 172.21.15.18:0/1744598716 conn(0x559a8ee85840 msgr2=0x559a8ee85c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.376+0000 7f6524dc6740 1 -- 172.21.15.18:0/1744598716 shutdown_connections 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.376+0000 7f6524dc6740 1 -- 172.21.15.18:0/1744598716 wait complete. 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f6524dc6740 1 -- start start 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f6524dc6740 1 Processor -- start 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f6524dc6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8ee91890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f6524dc6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee8bd30 0x559a8ee91dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f6524dc6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee948b0 0x559a8ee92310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f6524dc6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8ed97f40 con 0x559a8ee8b380 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f6524dc6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8eac64b0 con 0x559a8ee948b0 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f6524dc6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8ed78670 con 0x559a8ee8bd30 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f6524dc6740 10 client.0 _ll_register_callbacks cb 0x559a8ede5f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f651b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8ee91890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f651b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8ee91890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58548/0 (socket says 172.21.15.18:58548) 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f651b7fe640 1 -- 172.21.15.18:0/2847019493 learned_addr learned my addr 172.21.15.18:0/2847019493 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f651affd640 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee8bd30 0x559a8ee91dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f651bfff640 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee948b0 0x559a8ee92310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f651b7fe640 1 -- 172.21.15.18:0/2847019493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee8bd30 msgr2=0x559a8ee91dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.377+0000 7f651b7fe640 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee8bd30 0x559a8ee91dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f651b7fe640 1 -- 172.21.15.18:0/2847019493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee948b0 msgr2=0x559a8ee92310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f651b7fe640 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee948b0 0x559a8ee92310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f651b7fe640 1 -- 172.21.15.18:0/2847019493 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651401a040 con 0x559a8ee8b380 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f651affd640 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee8bd30 0x559a8ee91dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:16.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f651bfff640 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee948b0 0x559a8ee92310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f651b7fe640 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8ee91890 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f650c0ede90 tx=0x7f650c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f335f055679704d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f64faffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f64faffd640 1 -- 172.21.15.18:0/2847019493 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c0d4070 con 0x559a8ee8b380 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f64faffd640 1 -- 172.21.15.18:0/2847019493 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f650c0c0d90 con 0x559a8ee8b380 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f6524dc6740 1 -- 172.21.15.18:0/2847019493 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a8edccf40 con 0x559a8ee8b380 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f64faffd640 1 -- 172.21.15.18:0/2847019493 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f650c0c53e0 con 0x559a8ee8b380 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f6524dc6740 10 client.17892 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f6524dc6740 1 -- 172.21.15.18:0/2847019493 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559a8edf1320 con 0x559a8ee8b380 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f64faffd640 1 -- 172.21.15.18:0/2847019493 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559a8edccf40 con 0x559a8ee8b380 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f64faffd640 1 -- 172.21.15.18:0/2847019493 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f650c03c070 con 0x559a8ee8b380 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f6524dc6740 10 client.17892 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f6524dc6740 10 client.17892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f6524dc6740 20 client.17892 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:16.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f6524dc6740 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559a8eea4d50 0x559a8eea7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f6524dc6740 4 client.17892 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.378+0000 7f6524dc6740 1 -- 172.21.15.18:0/2847019493 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a8ede9310 con 0x559a8eea4d50 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.379+0000 7f651affd640 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559a8eea4d50 0x559a8eea7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.379+0000 7f64faffd640 10 client.17892 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.379+0000 7f651affd640 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559a8eea4d50 0x559a8eea7190 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b06b784bfda78d2 server_cookie=c016e5b1be2b8f55 in_seq=0 out_seq=0 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f64faffd640 1 -- 172.21.15.18:0/2847019493 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559a8ede9310 con 0x559a8eea4d50 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f64faffd640 10 client.17892 handle_command_reply: tid=0 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f6524dc6740 1 client.17892 shutdown 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f6524dc6740 1 -- 172.21.15.18:0/2847019493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 msgr2=0x559a8ee91890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f6524dc6740 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8ee91890 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f650c0ede90 tx=0x7f650c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f6524dc6740 1 -- 172.21.15.18:0/2847019493 shutdown_connections 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f6524dc6740 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559a8eea4d50 0x559a8eea7190 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f6524dc6740 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee948b0 0x559a8ee92310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f6524dc6740 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee8bd30 0x559a8ee91dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f6524dc6740 1 --2- 172.21.15.18:0/2847019493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8ee91890 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f64faffd640 0 client.17892 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:16.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f6524dc6740 1 -- 172.21.15.18:0/2847019493 >> 172.21.15.18:0/2847019493 conn(0x559a8ee85840 msgr2=0x559a8ee877a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:16.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f6524dc6740 1 -- 172.21.15.18:0/2847019493 shutdown_connections 2024-02-17T12:56:16.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f6524dc6740 1 -- 172.21.15.18:0/2847019493 wait complete. 2024-02-17T12:56:16.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.380+0000 7f6524dc6740 20 client.17892 trim_cache size 0 max 16384 2024-02-17T12:56:16.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.395+0000 7f6524dc6740 1 Processor -- start 2024-02-17T12:56:16.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.395+0000 7f6524dc6740 1 -- start start 2024-02-17T12:56:16.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.395+0000 7f6524dc6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8b380 0x559a8ee92130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.395+0000 7f6524dc6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee8bd30 0x559a8ee9b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.395+0000 7f6524dc6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee948b0 0x559a8ee9bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.395+0000 7f6524dc6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8ee8ad70 con 0x559a8ee948b0 2024-02-17T12:56:16.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.395+0000 7f6524dc6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8ee97120 con 0x559a8ee8b380 2024-02-17T12:56:16.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.395+0000 7f6524dc6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8ee8fe20 con 0x559a8ee8bd30 2024-02-17T12:56:16.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.395+0000 7f651bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee948b0 0x559a8ee9bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.395+0000 7f651bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee948b0 0x559a8ee9bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58558/0 (socket says 172.21.15.18:58558) 2024-02-17T12:56:16.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.395+0000 7f651bfff640 1 -- 172.21.15.18:0/2703601538 learned_addr learned my addr 172.21.15.18:0/2703601538 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:16.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f651affd640 1 --2- 172.21.15.18:0/2703601538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee8bd30 0x559a8ee9b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f651bfff640 1 -- 172.21.15.18:0/2703601538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee8bd30 msgr2=0x559a8ee9b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f651b7fe640 1 --2- 172.21.15.18:0/2703601538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8b380 0x559a8ee92130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f651bfff640 1 --2- 172.21.15.18:0/2703601538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee8bd30 0x559a8ee9b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f651bfff640 1 -- 172.21.15.18:0/2703601538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8b380 msgr2=0x559a8ee92130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f651bfff640 1 --2- 172.21.15.18:0/2703601538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8b380 0x559a8ee92130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f651bfff640 1 -- 172.21.15.18:0/2703601538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a8ed8dc00 con 0x559a8ee948b0 2024-02-17T12:56:16.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f651affd640 1 --2- 172.21.15.18:0/2703601538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee8bd30 0x559a8ee9b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:16.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f651b7fe640 1 --2- 172.21.15.18:0/2703601538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8b380 0x559a8ee92130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:16.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f651bfff640 1 --2- 172.21.15.18:0/2703601538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee948b0 0x559a8ee9bdc0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f6514010fd0 tx=0x7f6514012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f723c3bebd8521da server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:16.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f65017fa640 1 -- 172.21.15.18:0/2703601538 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514009920 con 0x559a8ee948b0 2024-02-17T12:56:16.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f65017fa640 1 -- 172.21.15.18:0/2703601538 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6514012520 con 0x559a8ee948b0 2024-02-17T12:56:16.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.396+0000 7f65017fa640 1 -- 172.21.15.18:0/2703601538 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400cdd0 con 0x559a8ee948b0 2024-02-17T12:56:16.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.397+0000 7f6524dc6740 1 -- 172.21.15.18:0/2703601538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee948b0 msgr2=0x559a8ee9bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.397+0000 7f6524dc6740 1 --2- 172.21.15.18:0/2703601538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee948b0 0x559a8ee9bdc0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f6514010fd0 tx=0x7f6514012bb0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.397+0000 7f6524dc6740 1 -- 172.21.15.18:0/2703601538 shutdown_connections 2024-02-17T12:56:16.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.397+0000 7f6524dc6740 1 --2- 172.21.15.18:0/2703601538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee948b0 0x559a8ee9bdc0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.397+0000 7f6524dc6740 1 --2- 172.21.15.18:0/2703601538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee8bd30 0x559a8ee9b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.397+0000 7f6524dc6740 1 --2- 172.21.15.18:0/2703601538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8b380 0x559a8ee92130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.397+0000 7f6524dc6740 1 -- 172.21.15.18:0/2703601538 >> 172.21.15.18:0/2703601538 conn(0x559a8ee85840 msgr2=0x559a8ee86d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.397+0000 7f6524dc6740 1 -- 172.21.15.18:0/2703601538 shutdown_connections 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.397+0000 7f6524dc6740 1 -- 172.21.15.18:0/2703601538 wait complete. 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.397+0000 7f6524dc6740 1 -- start start 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.397+0000 7f6524dc6740 1 Processor -- start 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f6524dc6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8eef5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f6524dc6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8bd30 0x559a8eef60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f6524dc6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee948b0 0x559a8ee9b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f6524dc6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8ee9b610 con 0x559a8ee8b380 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f6524dc6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8ee9b780 con 0x559a8ee8bd30 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f6524dc6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8ee9b940 con 0x559a8ee948b0 2024-02-17T12:56:16.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f6524dc6740 10 client.0 _ll_register_callbacks cb 0x7f65140123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:16.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f651b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8eef5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f651b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8eef5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58568/0 (socket says 172.21.15.18:58568) 2024-02-17T12:56:16.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f651b7fe640 1 -- 172.21.15.18:0/1295173672 learned_addr learned my addr 172.21.15.18:0/1295173672 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:16.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f651bfff640 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee948b0 0x559a8ee9b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f651affd640 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8bd30 0x559a8eef60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f651b7fe640 1 -- 172.21.15.18:0/1295173672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee948b0 msgr2=0x559a8ee9b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f651b7fe640 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee948b0 0x559a8ee9b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f651b7fe640 1 -- 172.21.15.18:0/1295173672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8bd30 msgr2=0x559a8eef60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f651b7fe640 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8bd30 0x559a8eef60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f651b7fe640 1 -- 172.21.15.18:0/1295173672 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a8edb50b0 con 0x559a8ee8b380 2024-02-17T12:56:16.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f651affd640 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8bd30 0x559a8eef60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:16.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f651b7fe640 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8eef5b90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f650c0eda50 tx=0x7f650c0f2210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7a1c43a1c367f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:16.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f64faffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:16.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f64faffd640 1 -- 172.21.15.18:0/1295173672 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c0d4070 con 0x559a8ee8b380 2024-02-17T12:56:16.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f64faffd640 1 -- 172.21.15.18:0/1295173672 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f650c0f22d0 con 0x559a8ee8b380 2024-02-17T12:56:16.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f6524dc6740 1 -- 172.21.15.18:0/1295173672 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a8ee18900 con 0x559a8ee8b380 2024-02-17T12:56:16.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f64faffd640 1 -- 172.21.15.18:0/1295173672 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f650c0bca10 con 0x559a8ee8b380 2024-02-17T12:56:16.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f6524dc6740 10 client.17898 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:16.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.398+0000 7f6524dc6740 1 -- 172.21.15.18:0/1295173672 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559a8edb3910 con 0x559a8ee8b380 2024-02-17T12:56:16.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.399+0000 7f64faffd640 1 -- 172.21.15.18:0/1295173672 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559a8ee18900 con 0x559a8ee8b380 2024-02-17T12:56:16.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.399+0000 7f64faffd640 1 -- 172.21.15.18:0/1295173672 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f650c03c050 con 0x559a8ee8b380 2024-02-17T12:56:16.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.399+0000 7f6524dc6740 10 client.17898 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.399+0000 7f6524dc6740 10 client.17898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.399+0000 7f6524dc6740 20 client.17898 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.399+0000 7f6524dc6740 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559a8eefd4e0 0x559a8eeff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.399+0000 7f6524dc6740 4 client.17898 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.399+0000 7f6524dc6740 1 -- 172.21.15.18:0/1295173672 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559a8eaf2860 con 0x559a8eefd4e0 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.399+0000 7f651affd640 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559a8eefd4e0 0x559a8eeff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.399+0000 7f651affd640 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559a8eefd4e0 0x559a8eeff8d0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8d3119d6f12f7d server_cookie=6c59a61a31eb08e6 in_seq=0 out_seq=0 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.399+0000 7f64faffd640 10 client.17898 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.400+0000 7f64faffd640 1 -- 172.21.15.18:0/1295173672 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559a8eaf2860 con 0x559a8eefd4e0 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f64faffd640 10 client.17898 handle_command_reply: tid=0 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f6524dc6740 1 client.17898 shutdown 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f6524dc6740 1 -- 172.21.15.18:0/1295173672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 msgr2=0x559a8eef5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f6524dc6740 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8eef5b90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f650c0eda50 tx=0x7f650c0f2210 comp rx=0 tx=0).stop 2024-02-17T12:56:16.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f6524dc6740 1 -- 172.21.15.18:0/1295173672 shutdown_connections 2024-02-17T12:56:16.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f6524dc6740 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559a8eefd4e0 0x559a8eeff8d0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f6524dc6740 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a8ee948b0 0x559a8ee9b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f6524dc6740 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a8ee8bd30 0x559a8eef60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f64faffd640 0 client.17898 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:16.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f6524dc6740 1 --2- 172.21.15.18:0/1295173672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a8ee8b380 0x559a8eef5b90 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f6524dc6740 1 -- 172.21.15.18:0/1295173672 >> 172.21.15.18:0/1295173672 conn(0x559a8ee85840 msgr2=0x559a8ee848e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:16.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f6524dc6740 1 -- 172.21.15.18:0/1295173672 shutdown_connections 2024-02-17T12:56:16.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f6524dc6740 1 -- 172.21.15.18:0/1295173672 wait complete. 2024-02-17T12:56:16.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f6524dc6740 20 client.17898 trim_cache size 0 max 16384 2024-02-17T12:56:16.399 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:56:16.400 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:56:16.400 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "b3421776-00b0-466d-8c50-5e5dc0b90153", 2024-02-17T12:56:16.400 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:56:16.400 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:56:16.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f65221f4640 1 -- 172.21.15.18:0/823521172 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f65040a2230 msgr2=0x7f65040a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f65221f4640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f65040a2230 0x7f65040a46f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6514010c40 tx=0x7f651400c000 comp rx=0 tx=0).stop 2024-02-17T12:56:16.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f65221f4640 1 -- 172.21.15.18:0/823521172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f651c101390 msgr2=0x7f651c19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.401+0000 7f65221f4640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f651c101390 0x7f651c19d590 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f650c009230 tx=0x7f650c004040 comp rx=0 tx=0).stop 2024-02-17T12:56:16.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.402+0000 7f65221f4640 1 -- 172.21.15.18:0/823521172 shutdown_connections 2024-02-17T12:56:16.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.402+0000 7f65221f4640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f65040a2230 0x7f65040a46f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.402+0000 7f65221f4640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f651c1976a0 0x7f651c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.402+0000 7f65221f4640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c106460 0x7f651c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.402+0000 7f65221f4640 1 --2- 172.21.15.18:0/823521172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f651c101390 0x7f651c19d590 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.402+0000 7f65221f4640 1 -- 172.21.15.18:0/823521172 >> 172.21.15.18:0/823521172 conn(0x7f651c0fcbb0 msgr2=0x7f651c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:16.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.402+0000 7f65221f4640 1 -- 172.21.15.18:0/823521172 shutdown_connections 2024-02-17T12:56:16.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.402+0000 7f65221f4640 1 -- 172.21.15.18:0/823521172 wait complete. 2024-02-17T12:56:16.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:56:16.404 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:56:16.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.562+0000 7fa537c22640 1 -- 172.21.15.18:0/2076704838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5301009a0 msgr2=0x7fa530100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.562+0000 7fa537c22640 1 --2- 172.21.15.18:0/2076704838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5301009a0 0x7fa530100d80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009930 tx=0x7fa52c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:16.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.562+0000 7fa537c22640 1 -- 172.21.15.18:0/2076704838 shutdown_connections 2024-02-17T12:56:16.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.562+0000 7fa537c22640 1 --2- 172.21.15.18:0/2076704838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530105550 0x7fa53010d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.562+0000 7fa537c22640 1 --2- 172.21.15.18:0/2076704838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa530101350 0x7fa530104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.562+0000 7fa537c22640 1 --2- 172.21.15.18:0/2076704838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5301009a0 0x7fa530100d80 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.562+0000 7fa537c22640 1 -- 172.21.15.18:0/2076704838 >> 172.21.15.18:0/2076704838 conn(0x7fa5300765b0 msgr2=0x7fa5300fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:16.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.562+0000 7fa537c22640 1 -- 172.21.15.18:0/2076704838 shutdown_connections 2024-02-17T12:56:16.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.562+0000 7fa537c22640 1 -- 172.21.15.18:0/2076704838 wait complete. 2024-02-17T12:56:16.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa537c22640 1 Processor -- start 2024-02-17T12:56:16.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa537c22640 1 -- start start 2024-02-17T12:56:16.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa537c22640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5301009a0 0x7fa53019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa537c22640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa530105550 0x7fa53019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa537c22640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5301977d0 0x7fa530197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa537c22640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa530198280 con 0x7fa530105550 2024-02-17T12:56:16.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa537c22640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5301983f0 con 0x7fa5301009a0 2024-02-17T12:56:16.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa537c22640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa530198560 con 0x7fa5301977d0 2024-02-17T12:56:16.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa5351bd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa530105550 0x7fa53019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa5359be640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5301009a0 0x7fa53019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa5359be640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5301009a0 0x7fa53019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:56574/0 (socket says 172.21.15.18:56574) 2024-02-17T12:56:16.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa5359be640 1 -- 172.21.15.18:0/4052074986 learned_addr learned my addr 172.21.15.18:0/4052074986 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:16.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa5361bf640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5301977d0 0x7fa530197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.563+0000 7fa5351bd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa530105550 0x7fa53019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58590/0 (socket says 172.21.15.18:58590) 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.564+0000 7fa5359be640 1 -- 172.21.15.18:0/4052074986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5301977d0 msgr2=0x7fa530197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.564+0000 7fa5359be640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5301977d0 0x7fa530197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.564+0000 7fa5359be640 1 -- 172.21.15.18:0/4052074986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa530105550 msgr2=0x7fa53019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.564+0000 7fa5359be640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa530105550 0x7fa53019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.564+0000 7fa5359be640 1 -- 172.21.15.18:0/4052074986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c009560 con 0x7fa5301009a0 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.564+0000 7fa5351bd640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa530105550 0x7fa53019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.564+0000 7fa5359be640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5301009a0 0x7fa53019d660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa52c000c00 tx=0x7fa52c02c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.564+0000 7fa526ffd640 1 -- 172.21.15.18:0/4052074986 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0042d0 con 0x7fa5301009a0 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.564+0000 7fa526ffd640 1 -- 172.21.15.18:0/4052074986 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa52c004430 con 0x7fa5301009a0 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.564+0000 7fa537c22640 1 -- 172.21.15.18:0/4052074986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5301a44b0 con 0x7fa5301009a0 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.564+0000 7fa526ffd640 1 -- 172.21.15.18:0/4052074986 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c003c50 con 0x7fa5301009a0 2024-02-17T12:56:16.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.564+0000 7fa537c22640 1 -- 172.21.15.18:0/4052074986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5301a4990 con 0x7fa5301009a0 2024-02-17T12:56:16.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.566+0000 7fa526ffd640 1 -- 172.21.15.18:0/4052074986 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa52c004df0 con 0x7fa5301009a0 2024-02-17T12:56:16.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.567+0000 7fa53a7f1740 1 Processor -- start 2024-02-17T12:56:16.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.567+0000 7fa53a7f1740 1 -- start start 2024-02-17T12:56:16.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.567+0000 7fa53a7f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 0x55c50ad2c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.567+0000 7fa53a7f1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2cbd0 0x55c50ad35270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.567+0000 7fa53a7f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad357b0 0x55c50ad37ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.567+0000 7fa53a7f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ac9b280 con 0x55c50ad2c2b0 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.567+0000 7fa53a7f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ac987b0 con 0x55c50ad357b0 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.567+0000 7fa53a7f1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ac93f90 con 0x55c50ad2cbd0 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.567+0000 7fa526ffd640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa51c0a2230 0x7fa51c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.567+0000 7fa5361bf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 0x55c50ad2c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa5351bd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad357b0 0x55c50ad37ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.567+0000 7fa5361bf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 0x55c50ad2c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58606/0 (socket says 172.21.15.18:58606) 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa5361bf640 1 -- 172.21.15.18:0/1412509009 learned_addr learned my addr 172.21.15.18:0/1412509009 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa5359be640 1 --2- 172.21.15.18:0/1412509009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2cbd0 0x55c50ad35270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa5361bf640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa51c0a2230 0x7fa51c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa526ffd640 1 -- 172.21.15.18:0/4052074986 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa52c0f06e0 con 0x7fa5301009a0 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa5361bf640 1 -- 172.21.15.18:0/1412509009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2cbd0 msgr2=0x55c50ad35270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa5361bf640 1 --2- 172.21.15.18:0/1412509009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2cbd0 0x55c50ad35270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa5361bf640 1 -- 172.21.15.18:0/1412509009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad357b0 msgr2=0x55c50ad37ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa5361bf640 1 --2- 172.21.15.18:0/1412509009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad357b0 0x55c50ad37ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa5361bf640 1 -- 172.21.15.18:0/1412509009 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c50ac52570 con 0x55c50ad2c2b0 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa5351bd640 1 --2- 172.21.15.18:0/1412509009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad357b0 0x55c50ad37ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa5361bf640 1 --2- 172.21.15.18:0/1412509009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 0x55c50ad2c690 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fa528013d90 tx=0x7fa5280117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37944742f1c27 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa5361bf640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa51c0a2230 0x7fa51c0a46f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa528011cb0 tx=0x7fa528018000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa51bfff640 1 -- 172.21.15.18:0/1412509009 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528009e40 con 0x55c50ad2c2b0 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa51bfff640 1 -- 172.21.15.18:0/1412509009 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa528003e00 con 0x55c50ad2c2b0 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.568+0000 7fa51bfff640 1 -- 172.21.15.18:0/1412509009 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528005240 con 0x55c50ad2c2b0 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.569+0000 7fa53a7f1740 1 -- 172.21.15.18:0/1412509009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 msgr2=0x55c50ad2c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.569+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/1412509009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 0x55c50ad2c690 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fa528013d90 tx=0x7fa5280117b0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.569+0000 7fa53a7f1740 1 -- 172.21.15.18:0/1412509009 shutdown_connections 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.569+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/1412509009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad357b0 0x55c50ad37ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.569+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/1412509009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2cbd0 0x55c50ad35270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.569+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/1412509009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 0x55c50ad2c690 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.569+0000 7fa53a7f1740 1 -- 172.21.15.18:0/1412509009 >> 172.21.15.18:0/1412509009 conn(0x55c50ad26990 msgr2=0x55c50ad26d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.569+0000 7fa53a7f1740 1 -- 172.21.15.18:0/1412509009 shutdown_connections 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.569+0000 7fa53a7f1740 1 -- 172.21.15.18:0/1412509009 wait complete. 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.569+0000 7fa53a7f1740 1 -- start start 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa53a7f1740 1 Processor -- start 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa53a7f1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2c2b0 0x55c50ad35260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa53a7f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2cbd0 0x55c50ad325f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa53a7f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad357b0 0x55c50ad32d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa53a7f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50a967430 con 0x55c50ad2cbd0 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa53a7f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50a923e00 con 0x55c50ad357b0 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa53a7f1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ac18fd0 con 0x55c50ad2c2b0 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa53a7f1740 10 client.0 _ll_register_callbacks cb 0x55c50ac6efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa5351bd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2cbd0 0x55c50ad325f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa5351bd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2cbd0 0x55c50ad325f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58612/0 (socket says 172.21.15.18:58612) 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa5351bd640 1 -- 172.21.15.18:0/2613613232 learned_addr learned my addr 172.21.15.18:0/2613613232 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa5361bf640 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad357b0 0x55c50ad32d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa5359be640 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2c2b0 0x55c50ad35260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa5351bd640 1 -- 172.21.15.18:0/2613613232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2c2b0 msgr2=0x55c50ad35260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa5351bd640 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2c2b0 0x55c50ad35260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa5351bd640 1 -- 172.21.15.18:0/2613613232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad357b0 msgr2=0x55c50ad32d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa5351bd640 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad357b0 0x55c50ad32d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa5351bd640 1 -- 172.21.15.18:0/2613613232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52801b040 con 0x55c50ad2cbd0 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.570+0000 7fa5361bf640 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad357b0 0x55c50ad32d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa5351bd640 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2cbd0 0x55c50ad325f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fa520012d20 tx=0x7fa52000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1bf37b59acf0a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa5197fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa5197fa640 1 -- 172.21.15.18:0/2613613232 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520017070 con 0x55c50ad2cbd0 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa5197fa640 1 -- 172.21.15.18:0/2613613232 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa520012110 con 0x55c50ad2cbd0 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa53a7f1740 1 -- 172.21.15.18:0/2613613232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c50a9937e0 con 0x55c50ad2cbd0 2024-02-17T12:56:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa5197fa640 1 -- 172.21.15.18:0/2613613232 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5200122d0 con 0x55c50ad2cbd0 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa53a7f1740 10 client.17910 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa53a7f1740 1 -- 172.21.15.18:0/2613613232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c50ac92610 con 0x55c50ad2cbd0 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa5197fa640 1 -- 172.21.15.18:0/2613613232 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c50a9937e0 con 0x55c50ad2cbd0 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa5197fa640 1 -- 172.21.15.18:0/2613613232 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa520003c90 con 0x55c50ad2cbd0 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa53a7f1740 10 client.17910 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa53a7f1740 10 client.17910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa53a7f1740 20 client.17910 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c50ad45a60 0x55c50ad47e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa53a7f1740 4 client.17910 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.571+0000 7fa53a7f1740 1 -- 172.21.15.18:0/2613613232 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c50a9d0720 con 0x55c50ad45a60 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.572+0000 7fa5359be640 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c50ad45a60 0x55c50ad47e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.572+0000 7fa5359be640 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c50ad45a60 0x55c50ad47e80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a89c6020c289c3a server_cookie=13d72b3957514120 in_seq=0 out_seq=0 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.572+0000 7fa5197fa640 10 client.17910 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa5197fa640 1 -- 172.21.15.18:0/2613613232 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c50a9d0720 con 0x55c50ad45a60 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa5197fa640 10 client.17910 handle_command_reply: tid=0 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa53a7f1740 1 client.17910 shutdown 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa53a7f1740 1 -- 172.21.15.18:0/2613613232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2cbd0 msgr2=0x55c50ad325f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2cbd0 0x55c50ad325f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fa520012d20 tx=0x7fa52000f790 comp rx=0 tx=0).stop 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa53a7f1740 1 -- 172.21.15.18:0/2613613232 shutdown_connections 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c50ad45a60 0x55c50ad47e80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad357b0 0x55c50ad32d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2cbd0 0x55c50ad325f0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa5197fa640 0 client.17910 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/2613613232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2c2b0 0x55c50ad35260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa53a7f1740 1 -- 172.21.15.18:0/2613613232 >> 172.21.15.18:0/2613613232 conn(0x55c50ad26990 msgr2=0x55c50ad28730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa53a7f1740 1 -- 172.21.15.18:0/2613613232 shutdown_connections 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa53a7f1740 1 -- 172.21.15.18:0/2613613232 wait complete. 2024-02-17T12:56:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.573+0000 7fa53a7f1740 20 client.17910 trim_cache size 0 max 16384 2024-02-17T12:56:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.588+0000 7fa53a7f1740 1 Processor -- start 2024-02-17T12:56:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.588+0000 7fa53a7f1740 1 -- start start 2024-02-17T12:56:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.588+0000 7fa53a7f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 0x55c50ad4c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.588+0000 7fa53a7f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad2cbd0 0x55c50ad3c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.588+0000 7fa53a7f1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad357b0 0x55c50ad3cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.588+0000 7fa53a7f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ad27790 con 0x55c50ad2c2b0 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.588+0000 7fa53a7f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ad441b0 con 0x55c50ad2cbd0 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.588+0000 7fa53a7f1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ad41540 con 0x55c50ad357b0 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa5359be640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 0x55c50ad4c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa5359be640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 0x55c50ad4c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58620/0 (socket says 172.21.15.18:58620) 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa5359be640 1 -- 172.21.15.18:0/3903673419 learned_addr learned my addr 172.21.15.18:0/3903673419 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa5351bd640 1 --2- 172.21.15.18:0/3903673419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad2cbd0 0x55c50ad3c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa5361bf640 1 --2- 172.21.15.18:0/3903673419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad357b0 0x55c50ad3cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa5359be640 1 -- 172.21.15.18:0/3903673419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad357b0 msgr2=0x55c50ad3cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa5359be640 1 --2- 172.21.15.18:0/3903673419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad357b0 0x55c50ad3cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa5359be640 1 -- 172.21.15.18:0/3903673419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad2cbd0 msgr2=0x55c50ad3c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa5359be640 1 --2- 172.21.15.18:0/3903673419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad2cbd0 0x55c50ad3c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa5359be640 1 -- 172.21.15.18:0/3903673419 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c50ac2eac0 con 0x55c50ad2c2b0 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa5359be640 1 --2- 172.21.15.18:0/3903673419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 0x55c50ad4c7e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fa52c005ab0 tx=0x7fa52c041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a0a14b5cf9b231 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa51b7fe640 1 -- 172.21.15.18:0/3903673419 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0bd1a0 con 0x55c50ad2c2b0 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa51b7fe640 1 -- 172.21.15.18:0/3903673419 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa52c0c12f0 con 0x55c50ad2c2b0 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.589+0000 7fa51b7fe640 1 -- 172.21.15.18:0/3903673419 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0b9810 con 0x55c50ad2c2b0 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.590+0000 7fa53a7f1740 1 -- 172.21.15.18:0/3903673419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 msgr2=0x55c50ad4c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.590+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/3903673419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 0x55c50ad4c7e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fa52c005ab0 tx=0x7fa52c041040 comp rx=0 tx=0).stop 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.590+0000 7fa53a7f1740 1 -- 172.21.15.18:0/3903673419 shutdown_connections 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.590+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/3903673419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad357b0 0x55c50ad3cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.590+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/3903673419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad2cbd0 0x55c50ad3c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.590+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/3903673419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad2c2b0 0x55c50ad4c7e0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.590+0000 7fa53a7f1740 1 -- 172.21.15.18:0/3903673419 >> 172.21.15.18:0/3903673419 conn(0x55c50ad26990 msgr2=0x55c50ad26d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.590+0000 7fa53a7f1740 1 -- 172.21.15.18:0/3903673419 shutdown_connections 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.590+0000 7fa53a7f1740 1 -- 172.21.15.18:0/3903673419 wait complete. 2024-02-17T12:56:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.590+0000 7fa53a7f1740 1 -- start start 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.590+0000 7fa53a7f1740 1 Processor -- start 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa53a7f1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2c2b0 0x55c50ad3b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa53a7f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad2cbd0 0x55c50ad3b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa53a7f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad357b0 0x55c50ad3bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa53a7f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ad9bf70 con 0x55c50ad357b0 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa53a7f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ad9c0b0 con 0x55c50ad2cbd0 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa53a7f1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ad9c270 con 0x55c50ad2c2b0 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa53a7f1740 10 client.0 _ll_register_callbacks cb 0x7fa528013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5361bf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad357b0 0x55c50ad3bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5351bd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad2cbd0 0x55c50ad3b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5361bf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad357b0 0x55c50ad3bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58628/0 (socket says 172.21.15.18:58628) 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5361bf640 1 -- 172.21.15.18:0/517255746 learned_addr learned my addr 172.21.15.18:0/517255746 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5359be640 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2c2b0 0x55c50ad3b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5351bd640 1 -- 172.21.15.18:0/517255746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2c2b0 msgr2=0x55c50ad3b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5351bd640 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2c2b0 0x55c50ad3b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5351bd640 1 -- 172.21.15.18:0/517255746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad357b0 msgr2=0x55c50ad3bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5351bd640 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad357b0 0x55c50ad3bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5351bd640 1 -- 172.21.15.18:0/517255746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5301a4990 con 0x55c50ad2cbd0 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5361bf640 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad357b0 0x55c50ad3bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5359be640 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2c2b0 0x55c50ad3b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5351bd640 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad2cbd0 0x55c50ad3b720 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa5200104b0 tx=0x7fa520009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e02601833dc630ec server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.591+0000 7fa5197fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa5197fa640 1 -- 172.21.15.18:0/517255746 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520017040 con 0x55c50ad2cbd0 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa5197fa640 1 -- 172.21.15.18:0/517255746 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa520010820 con 0x55c50ad2cbd0 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa53a7f1740 1 -- 172.21.15.18:0/517255746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa528003e00 con 0x55c50ad2cbd0 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa5197fa640 1 -- 172.21.15.18:0/517255746 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa520010a00 con 0x55c50ad2cbd0 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa53a7f1740 10 client.27170 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa53a7f1740 1 -- 172.21.15.18:0/517255746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c50ac546a0 con 0x55c50ad2cbd0 2024-02-17T12:56:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa5197fa640 1 -- 172.21.15.18:0/517255746 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa520039080 con 0x55c50ad2cbd0 2024-02-17T12:56:16.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa5197fa640 1 -- 172.21.15.18:0/517255746 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa5200035f0 con 0x55c50ad2cbd0 2024-02-17T12:56:16.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa53a7f1740 10 client.27170 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:16.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa53a7f1740 10 client.27170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:16.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa53a7f1740 20 client.27170 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:16.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c50ad9dbb0 0x55c50ad9ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:16.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa53a7f1740 4 client.27170 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:56:16.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa53a7f1740 1 -- 172.21.15.18:0/517255746 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c50aa1f890 con 0x55c50ad9dbb0 2024-02-17T12:56:16.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.592+0000 7fa5359be640 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c50ad9dbb0 0x55c50ad9ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:16.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.593+0000 7fa5359be640 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c50ad9dbb0 0x55c50ad9ffd0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d20ad42241b2616a server_cookie=d7e274f94954ea8e in_seq=0 out_seq=0 2024-02-17T12:56:16.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.593+0000 7fa5197fa640 10 client.27170 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.593+0000 7fa5197fa640 1 -- 172.21.15.18:0/517255746 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c50aa1f890 con 0x55c50ad9dbb0 2024-02-17T12:56:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.593+0000 7fa5197fa640 10 client.27170 handle_command_reply: tid=0 2024-02-17T12:56:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.593+0000 7fa53a7f1740 1 client.27170 shutdown 2024-02-17T12:56:16.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.593+0000 7fa53a7f1740 1 -- 172.21.15.18:0/517255746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad2cbd0 msgr2=0x55c50ad3b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.593+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad2cbd0 0x55c50ad3b720 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa5200104b0 tx=0x7fa520009c50 comp rx=0 tx=0).stop 2024-02-17T12:56:16.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa53a7f1740 1 -- 172.21.15.18:0/517255746 shutdown_connections 2024-02-17T12:56:16.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c50ad9dbb0 0x55c50ad9ffd0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c50ad357b0 0x55c50ad3bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c50ad2cbd0 0x55c50ad3b720 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa53a7f1740 1 --2- 172.21.15.18:0/517255746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c50ad2c2b0 0x55c50ad3b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa5197fa640 0 client.27170 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:16.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa53a7f1740 1 -- 172.21.15.18:0/517255746 >> 172.21.15.18:0/517255746 conn(0x55c50ad26990 msgr2=0x55c50ad4c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:16.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa53a7f1740 1 -- 172.21.15.18:0/517255746 shutdown_connections 2024-02-17T12:56:16.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa53a7f1740 1 -- 172.21.15.18:0/517255746 wait complete. 2024-02-17T12:56:16.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa53a7f1740 20 client.27170 trim_cache size 0 max 16384 2024-02-17T12:56:16.591 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:56:16.591 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:56:16.591 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:56:16.591 INFO:teuthology.orchestra.run.smithi018.stdout: "b3421776-00b0-466d-8c50-5e5dc0b90153": { 2024-02-17T12:56:16.591 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:56:16.591 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "b3421776-00b0-466d-8c50-5e5dc0b90153", 2024-02-17T12:56:16.591 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:56:16.591 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:56:16.591 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:56:16.591 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:56:16.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa537c22640 1 -- 172.21.15.18:0/4052074986 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa51c0a2230 msgr2=0x7fa51c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa537c22640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa51c0a2230 0x7fa51c0a46f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa528011cb0 tx=0x7fa528018000 comp rx=0 tx=0).stop 2024-02-17T12:56:16.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa537c22640 1 -- 172.21.15.18:0/4052074986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5301009a0 msgr2=0x7fa53019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:16.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa537c22640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5301009a0 0x7fa53019d660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa52c000c00 tx=0x7fa52c02c5f0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa537c22640 1 -- 172.21.15.18:0/4052074986 shutdown_connections 2024-02-17T12:56:16.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa537c22640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa51c0a2230 0x7fa51c0a46f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa537c22640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5301977d0 0x7fa530197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa537c22640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa530105550 0x7fa53019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa537c22640 1 --2- 172.21.15.18:0/4052074986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5301009a0 0x7fa53019d660 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:16.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa537c22640 1 -- 172.21.15.18:0/4052074986 >> 172.21.15.18:0/4052074986 conn(0x7fa5300765b0 msgr2=0x7fa5300ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:16.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa537c22640 1 -- 172.21.15.18:0/4052074986 shutdown_connections 2024-02-17T12:56:16.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:16.594+0000 7fa537c22640 1 -- 172.21.15.18:0/4052074986 wait complete. 2024-02-17T12:56:16.595 INFO:tasks.cephfs.filesystem:scrub status for tag:b3421776-00b0-466d-8c50-5e5dc0b90153 - {'path': '/', 'tag': 'b3421776-00b0-466d-8c50-5e5dc0b90153', 'options': 'recursive,force'} 2024-02-17T12:56:17.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:16 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:17.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:16 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:17.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:16 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:17.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:16 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:17.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:16 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:17.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:16 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:17.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:16 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:17.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:16 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:17.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 340759 5.98 MB/sec execute 212 sec latency 876.554 ms 2024-02-17T12:56:17.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:16 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:17.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:16 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:17.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:16 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:17.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:16 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:56:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:18.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:18.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:18.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.202 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:17 smithi146 ceph-mon[36454]: pgmap v198: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-02-17T12:56:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:56:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:18.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:18.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:17 smithi018 ceph-mon[32625]: pgmap v198: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-02-17T12:56:18.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 342167 5.98 MB/sec execute 213 sec latency 21.976 ms 2024-02-17T12:56:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:56:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:18.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:18.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:18.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:17 smithi119 ceph-mon[36188]: pgmap v198: 97 pgs: 97 active+clean; 121 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-02-17T12:56:19.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 343127 5.97 MB/sec execute 214 sec latency 570.989 ms 2024-02-17T12:56:20.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:19 smithi146 ceph-mon[36454]: pgmap v199: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-02-17T12:56:20.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:19 smithi018 ceph-mon[32625]: pgmap v199: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-02-17T12:56:20.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 344822 5.99 MB/sec execute 215 sec latency 12.559 ms 2024-02-17T12:56:20.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:19 smithi119 ceph-mon[36188]: pgmap v199: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-02-17T12:56:21.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 345759 5.99 MB/sec execute 216 sec latency 380.861 ms 2024-02-17T12:56:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.993+0000 7f1b00a23640 1 -- 172.21.15.18:0/3073409341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1afc100c70 msgr2=0x7f1afc10ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.993+0000 7f1b00a23640 1 --2- 172.21.15.18:0/3073409341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1afc100c70 0x7f1afc10ceb0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009ca0 tx=0x7f1af402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.993+0000 7f1b00a23640 1 -- 172.21.15.18:0/3073409341 shutdown_connections 2024-02-17T12:56:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.993+0000 7f1b00a23640 1 --2- 172.21.15.18:0/3073409341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1afc100c70 0x7f1afc10ceb0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.993+0000 7f1b00a23640 1 --2- 172.21.15.18:0/3073409341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1afc1002d0 0x7f1afc100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.993+0000 7f1b00a23640 1 --2- 172.21.15.18:0/3073409341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1afc105000 0x7f1afc1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.993+0000 7f1b00a23640 1 -- 172.21.15.18:0/3073409341 >> 172.21.15.18:0/3073409341 conn(0x7f1afc076e50 msgr2=0x7f1afc077260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.993+0000 7f1b00a23640 1 -- 172.21.15.18:0/3073409341 shutdown_connections 2024-02-17T12:56:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.993+0000 7f1b00a23640 1 -- 172.21.15.18:0/3073409341 wait complete. 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.994+0000 7f1b00a23640 1 Processor -- start 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.994+0000 7f1b00a23640 1 -- start start 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.994+0000 7f1b00a23640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1afc1002d0 0x7f1afc196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.994+0000 7f1b00a23640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1afc100c70 0x7f1afc196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.994+0000 7f1b00a23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1afc105000 0x7f1afc19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.994+0000 7f1b00a23640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc19b4f0 con 0x7f1afc100c70 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.994+0000 7f1b00a23640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc19b660 con 0x7f1afc105000 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.994+0000 7f1b00a23640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc19b820 con 0x7f1afc1002d0 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1af1d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1afc100c70 0x7f1afc196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1af1d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1afc100c70 0x7f1afc196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46264/0 (socket says 172.21.15.18:46264) 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1af1d9b640 1 -- 172.21.15.18:0/2167777188 learned_addr learned my addr 172.21.15.18:0/2167777188 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1afad9d640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1afc105000 0x7f1afc19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1afa59c640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1afc1002d0 0x7f1afc196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1af1d9b640 1 -- 172.21.15.18:0/2167777188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1afc1002d0 msgr2=0x7f1afc196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1af1d9b640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1afc1002d0 0x7f1afc196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1af1d9b640 1 -- 172.21.15.18:0/2167777188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1afc105000 msgr2=0x7f1afc19aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1af1d9b640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1afc105000 0x7f1afc19aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1af1d9b640 1 -- 172.21.15.18:0/2167777188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af4009900 con 0x7f1afc100c70 2024-02-17T12:56:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1af1d9b640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1afc100c70 0x7f1afc196b70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400c920 tx=0x7f1ae400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1af3fff640 1 -- 172.21.15.18:0/2167777188 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4007c70 con 0x7f1afc100c70 2024-02-17T12:56:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1b00a23640 1 -- 172.21.15.18:0/2167777188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1afc19bb00 con 0x7f1afc100c70 2024-02-17T12:56:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1af3fff640 1 -- 172.21.15.18:0/2167777188 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ae4007dd0 con 0x7f1afc100c70 2024-02-17T12:56:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.995+0000 7f1af3fff640 1 -- 172.21.15.18:0/2167777188 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4018860 con 0x7f1afc100c70 2024-02-17T12:56:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.996+0000 7f1b00a23640 1 -- 172.21.15.18:0/2167777188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1afc1a3380 con 0x7f1afc100c70 2024-02-17T12:56:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.998+0000 7f1af3fff640 1 -- 172.21.15.18:0/2167777188 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1ae400fac0 con 0x7f1afc100c70 2024-02-17T12:56:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.998+0000 7f1b035fa740 1 Processor -- start 2024-02-17T12:56:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.998+0000 7f1b035fa740 1 -- start start 2024-02-17T12:56:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.998+0000 7f1af3fff640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1ae80a22d0 0x7f1ae80a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1b035fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6e98ad0 0x5607b6e97820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1b035fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 0x5607b6e981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1b035fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6ea12e0 0x5607b6ea36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1b035fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b6e069f0 con 0x5607b6e97d60 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1b035fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b6e03f20 con 0x5607b6e98ad0 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1b035fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b6dff700 con 0x5607b6ea12e0 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1afad9d640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1ae80a22d0 0x7f1ae80a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1af3fff640 1 -- 172.21.15.18:0/2167777188 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1ae40ce120 con 0x7f1afc100c70 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1af1d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 0x5607b6e981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1af1d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 0x5607b6e981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46272/0 (socket says 172.21.15.18:46272) 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1af1d9b640 1 -- 172.21.15.18:0/3374095937 learned_addr learned my addr 172.21.15.18:0/3374095937 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1afa59c640 1 --2- 172.21.15.18:0/3374095937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6e98ad0 0x5607b6e97820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1afad9d640 1 --2- 172.21.15.18:0/3374095937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6ea12e0 0x5607b6ea36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1af1d9b640 1 -- 172.21.15.18:0/3374095937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6ea12e0 msgr2=0x5607b6ea36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1af1d9b640 1 --2- 172.21.15.18:0/3374095937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6ea12e0 0x5607b6ea36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1af1d9b640 1 -- 172.21.15.18:0/3374095937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6e98ad0 msgr2=0x5607b6e97820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1af1d9b640 1 --2- 172.21.15.18:0/3374095937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6e98ad0 0x5607b6e97820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1af1d9b640 1 -- 172.21.15.18:0/3374095937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607b6d957b0 con 0x5607b6e97d60 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1afad9d640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1ae80a22d0 0x7f1ae80a4790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1af4004af0 tx=0x7f1af4034520 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1af1d9b640 1 --2- 172.21.15.18:0/3374095937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 0x5607b6e981e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4096f90 tx=0x7f1ae409a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236a24c90d8224ba server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:21.999+0000 7f1af0d99640 1 -- 172.21.15.18:0/3374095937 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae401b870 con 0x5607b6e97d60 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.000+0000 7f1af0d99640 1 -- 172.21.15.18:0/3374095937 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ae400dd70 con 0x5607b6e97d60 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.000+0000 7f1af0d99640 1 -- 172.21.15.18:0/3374095937 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40cbb10 con 0x5607b6e97d60 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.000+0000 7f1b035fa740 1 -- 172.21.15.18:0/3374095937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 msgr2=0x5607b6e981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.000+0000 7f1b035fa740 1 --2- 172.21.15.18:0/3374095937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 0x5607b6e981e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4096f90 tx=0x7f1ae409a550 comp rx=0 tx=0).stop 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.000+0000 7f1b035fa740 1 -- 172.21.15.18:0/3374095937 shutdown_connections 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.000+0000 7f1b035fa740 1 --2- 172.21.15.18:0/3374095937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6ea12e0 0x5607b6ea36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.000+0000 7f1b035fa740 1 --2- 172.21.15.18:0/3374095937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 0x5607b6e981e0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.000+0000 7f1b035fa740 1 --2- 172.21.15.18:0/3374095937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6e98ad0 0x5607b6e97820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.000+0000 7f1b035fa740 1 -- 172.21.15.18:0/3374095937 >> 172.21.15.18:0/3374095937 conn(0x5607b6e5f550 msgr2=0x5607b6e92a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.000+0000 7f1b035fa740 1 -- 172.21.15.18:0/3374095937 shutdown_connections 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.000+0000 7f1b035fa740 1 -- 172.21.15.18:0/3374095937 wait complete. 2024-02-17T12:56:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1b035fa740 1 -- start start 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1b035fa740 1 Processor -- start 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1b035fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e97d60 0x5607b6e9e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1b035fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e98ad0 0x5607b6e9e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1b035fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6ea12e0 0x5607b6e9eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1b035fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b6da4bf0 con 0x5607b6e98ad0 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1b035fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b6ad3250 con 0x5607b6ea12e0 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1b035fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b6d85200 con 0x5607b6e97d60 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1b035fa740 10 client.0 _ll_register_callbacks cb 0x5607b6df3d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1af1d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e98ad0 0x5607b6e9e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1af1d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e98ad0 0x5607b6e9e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46288/0 (socket says 172.21.15.18:46288) 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1af1d9b640 1 -- 172.21.15.18:0/2801607766 learned_addr learned my addr 172.21.15.18:0/2801607766 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1afa59c640 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e97d60 0x5607b6e9e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1afad9d640 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6ea12e0 0x5607b6e9eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1af1d9b640 1 -- 172.21.15.18:0/2801607766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e97d60 msgr2=0x5607b6e9e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1af1d9b640 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e97d60 0x5607b6e9e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1af1d9b640 1 -- 172.21.15.18:0/2801607766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6ea12e0 msgr2=0x5607b6e9eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1af1d9b640 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6ea12e0 0x5607b6e9eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.001+0000 7f1af1d9b640 1 -- 172.21.15.18:0/2801607766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1afc1a3380 con 0x5607b6e98ad0 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1af1d9b640 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e98ad0 0x5607b6e9e860 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40cbe90 tx=0x7f1ae40a32d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abddf59c42fb63a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1ae1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1ae1ffb640 1 -- 172.21.15.18:0/2801607766 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae409ab30 con 0x5607b6e98ad0 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1ae1ffb640 1 -- 172.21.15.18:0/2801607766 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1ae401b870 con 0x5607b6e98ad0 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1b035fa740 1 -- 172.21.15.18:0/2801607766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607b6aff600 con 0x5607b6e98ad0 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1ae1ffb640 1 -- 172.21.15.18:0/2801607766 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ae40971a0 con 0x5607b6e98ad0 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1b035fa740 10 client.17928 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1b035fa740 1 -- 172.21.15.18:0/2801607766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5607b6dfdd80 con 0x5607b6e98ad0 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1ae1ffb640 1 -- 172.21.15.18:0/2801607766 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5607b6aff600 con 0x5607b6e98ad0 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1ae1ffb640 1 -- 172.21.15.18:0/2801607766 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1ae4016020 con 0x5607b6e98ad0 2024-02-17T12:56:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1b035fa740 10 client.17928 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1b035fa740 10 client.17928 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1b035fa740 20 client.17928 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.002+0000 7f1b035fa740 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607b6eb12a0 0x5607b6eb3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.003+0000 7f1b035fa740 4 client.17928 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.003+0000 7f1b035fa740 1 -- 172.21.15.18:0/2801607766 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607b6b501b0 con 0x5607b6eb12a0 2024-02-17T12:56:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.003+0000 7f1afa59c640 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607b6eb12a0 0x5607b6eb3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.003+0000 7f1afa59c640 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607b6eb12a0 0x5607b6eb3690 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e72292b18123e7e server_cookie=9a9d4abf2eba03ff in_seq=0 out_seq=0 2024-02-17T12:56:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.003+0000 7f1ae1ffb640 10 client.17928 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1ae1ffb640 1 -- 172.21.15.18:0/2801607766 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5607b6b501b0 con 0x5607b6eb12a0 2024-02-17T12:56:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1ae1ffb640 10 client.17928 handle_command_reply: tid=0 2024-02-17T12:56:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1b035fa740 1 client.17928 shutdown 2024-02-17T12:56:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1b035fa740 1 -- 172.21.15.18:0/2801607766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e98ad0 msgr2=0x5607b6e9e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1b035fa740 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e98ad0 0x5607b6e9e860 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40cbe90 tx=0x7f1ae40a32d0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1b035fa740 1 -- 172.21.15.18:0/2801607766 shutdown_connections 2024-02-17T12:56:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1b035fa740 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607b6eb12a0 0x5607b6eb3690 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1b035fa740 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6ea12e0 0x5607b6e9eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1b035fa740 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e98ad0 0x5607b6e9e860 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1ae1ffb640 0 client.17928 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1b035fa740 1 --2- 172.21.15.18:0/2801607766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e97d60 0x5607b6e9e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1b035fa740 1 -- 172.21.15.18:0/2801607766 >> 172.21.15.18:0/2801607766 conn(0x5607b6e5f550 msgr2=0x5607b6e947a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1b035fa740 1 -- 172.21.15.18:0/2801607766 shutdown_connections 2024-02-17T12:56:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1b035fa740 1 -- 172.21.15.18:0/2801607766 wait complete. 2024-02-17T12:56:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.004+0000 7f1b035fa740 20 client.17928 trim_cache size 0 max 16384 2024-02-17T12:56:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.017+0000 7f1b035fa740 1 Processor -- start 2024-02-17T12:56:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.017+0000 7f1b035fa740 1 -- start start 2024-02-17T12:56:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1b035fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 0x5607b6ea0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1b035fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e98ad0 0x5607b6e9ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1b035fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6ea12e0 0x5607b6e9d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1b035fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b6e93a50 con 0x5607b6e97d60 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1b035fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b6eb8860 con 0x5607b6ea12e0 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1b035fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b6ea3b30 con 0x5607b6e98ad0 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1afa59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 0x5607b6ea0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1afa59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 0x5607b6ea0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46304/0 (socket says 172.21.15.18:46304) 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1afa59c640 1 -- 172.21.15.18:0/3061713160 learned_addr learned my addr 172.21.15.18:0/3061713160 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1afa59c640 1 -- 172.21.15.18:0/3061713160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e98ad0 msgr2=0x5607b6e9ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1afa59c640 1 --2- 172.21.15.18:0/3061713160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e98ad0 0x5607b6e9ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1afad9d640 1 --2- 172.21.15.18:0/3061713160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6ea12e0 0x5607b6e9d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1afa59c640 1 -- 172.21.15.18:0/3061713160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6ea12e0 msgr2=0x5607b6e9d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1afa59c640 1 --2- 172.21.15.18:0/3061713160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6ea12e0 0x5607b6e9d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1afa59c640 1 -- 172.21.15.18:0/3061713160 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607b6d9a7b0 con 0x5607b6e97d60 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1afa59c640 1 --2- 172.21.15.18:0/3061713160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 0x5607b6ea0a50 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x5607b6eb3f60 tx=0x7f1aec00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813981a9c7fb8797 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1ae3fff640 1 -- 172.21.15.18:0/3061713160 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec016030 con 0x5607b6e97d60 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1ae3fff640 1 -- 172.21.15.18:0/3061713160 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1aec023070 con 0x5607b6e97d60 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.018+0000 7f1ae3fff640 1 -- 172.21.15.18:0/3061713160 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec004d40 con 0x5607b6e97d60 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.019+0000 7f1b035fa740 1 -- 172.21.15.18:0/3061713160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 msgr2=0x5607b6ea0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.019+0000 7f1b035fa740 1 --2- 172.21.15.18:0/3061713160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 0x5607b6ea0a50 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x5607b6eb3f60 tx=0x7f1aec00b390 comp rx=0 tx=0).stop 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.019+0000 7f1b035fa740 1 -- 172.21.15.18:0/3061713160 shutdown_connections 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.019+0000 7f1b035fa740 1 --2- 172.21.15.18:0/3061713160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6ea12e0 0x5607b6e9d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.019+0000 7f1b035fa740 1 --2- 172.21.15.18:0/3061713160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e98ad0 0x5607b6e9ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.019+0000 7f1b035fa740 1 --2- 172.21.15.18:0/3061713160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6e97d60 0x5607b6ea0a50 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.019+0000 7f1b035fa740 1 -- 172.21.15.18:0/3061713160 >> 172.21.15.18:0/3061713160 conn(0x5607b6e5f550 msgr2=0x5607b6e9cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.019+0000 7f1b035fa740 1 -- 172.21.15.18:0/3061713160 shutdown_connections 2024-02-17T12:56:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.019+0000 7f1b035fa740 1 -- 172.21.15.18:0/3061713160 wait complete. 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.019+0000 7f1b035fa740 1 -- start start 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.019+0000 7f1b035fa740 1 Processor -- start 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1b035fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6e97d60 0x5607b6f03530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1b035fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e98ad0 0x5607b6f03a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1b035fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6ea12e0 0x5607b6efd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1b035fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b6efdfa0 con 0x5607b6ea12e0 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1b035fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b6efe110 con 0x5607b6e97d60 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1b035fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b6efe2d0 con 0x5607b6e98ad0 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1b035fa740 10 client.0 _ll_register_callbacks cb 0x7f1ae400a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1afad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6ea12e0 0x5607b6efd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1afad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6ea12e0 0x5607b6efd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46306/0 (socket says 172.21.15.18:46306) 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1afad9d640 1 -- 172.21.15.18:0/422928410 learned_addr learned my addr 172.21.15.18:0/422928410 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1afa59c640 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6e97d60 0x5607b6f03530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1af1d9b640 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e98ad0 0x5607b6f03a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1afad9d640 1 -- 172.21.15.18:0/422928410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e98ad0 msgr2=0x5607b6f03a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1afad9d640 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e98ad0 0x5607b6f03a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1afad9d640 1 -- 172.21.15.18:0/422928410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6e97d60 msgr2=0x5607b6f03530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1afad9d640 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6e97d60 0x5607b6f03530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1afad9d640 1 -- 172.21.15.18:0/422928410 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aec00b930 con 0x5607b6ea12e0 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1afad9d640 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6ea12e0 0x5607b6efd900 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009ca0 tx=0x7f1af402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281d9ccabfcb9e7b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1ae1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1ae1ffb640 1 -- 172.21.15.18:0/422928410 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4003d80 con 0x5607b6ea12e0 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1ae1ffb640 1 -- 172.21.15.18:0/422928410 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1af4004040 con 0x5607b6ea12e0 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1b035fa740 1 -- 172.21.15.18:0/422928410 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ae400dd70 con 0x5607b6ea12e0 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1ae1ffb640 1 -- 172.21.15.18:0/422928410 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1af4004200 con 0x5607b6ea12e0 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1b035fa740 10 client.17934 fetch_fsmap learned FSMap version 40 2024-02-17T12:56:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.020+0000 7f1b035fa740 1 -- 172.21.15.18:0/422928410 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5607b6dc0540 con 0x5607b6ea12e0 2024-02-17T12:56:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.021+0000 7f1ae1ffb640 1 -- 172.21.15.18:0/422928410 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ae400dd70 con 0x5607b6ea12e0 2024-02-17T12:56:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.021+0000 7f1ae1ffb640 1 -- 172.21.15.18:0/422928410 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1af40449e0 con 0x5607b6ea12e0 2024-02-17T12:56:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.021+0000 7f1b035fa740 10 client.17934 fetch_fsmap finished waiting for FSMap version 40 2024-02-17T12:56:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.021+0000 7f1b035fa740 10 client.17934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.021+0000 7f1b035fa740 20 client.17934 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.021+0000 7f1b035fa740 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607b6f0b4a0 0x5607b6f0d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.021+0000 7f1b035fa740 4 client.17934 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:56:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.021+0000 7f1b035fa740 1 -- 172.21.15.18:0/422928410 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5607b6b3c550 con 0x5607b6f0b4a0 2024-02-17T12:56:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.021+0000 7f1afa59c640 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607b6f0b4a0 0x5607b6f0d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.022+0000 7f1ae1ffb640 10 client.17934 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.022+0000 7f1afa59c640 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607b6f0b4a0 0x5607b6f0d8c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32189cbe13533f9b server_cookie=6203f99bdde2663b in_seq=0 out_seq=0 2024-02-17T12:56:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.053+0000 7f1ae1ffb640 1 -- 172.21.15.18:0/422928410 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5607b6b3c550 con 0x5607b6f0b4a0 2024-02-17T12:56:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.053+0000 7f1ae1ffb640 10 client.17934 handle_command_reply: tid=0 2024-02-17T12:56:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1b035fa740 1 client.17934 shutdown 2024-02-17T12:56:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1b035fa740 1 -- 172.21.15.18:0/422928410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6ea12e0 msgr2=0x5607b6efd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1b035fa740 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6ea12e0 0x5607b6efd900 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009ca0 tx=0x7f1af402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1b035fa740 1 -- 172.21.15.18:0/422928410 shutdown_connections 2024-02-17T12:56:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1b035fa740 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607b6f0b4a0 0x5607b6f0d8c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1b035fa740 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607b6ea12e0 0x5607b6efd900 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1ae1ffb640 0 client.17934 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1b035fa740 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607b6e98ad0 0x5607b6f03a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1b035fa740 1 --2- 172.21.15.18:0/422928410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607b6e97d60 0x5607b6f03530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1b035fa740 1 -- 172.21.15.18:0/422928410 >> 172.21.15.18:0/422928410 conn(0x5607b6e5f550 msgr2=0x5607b6e9cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1b035fa740 1 -- 172.21.15.18:0/422928410 shutdown_connections 2024-02-17T12:56:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1b035fa740 1 -- 172.21.15.18:0/422928410 wait complete. 2024-02-17T12:56:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.054+0000 7f1b035fa740 20 client.17934 trim_cache size 0 max 16384 2024-02-17T12:56:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:56:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:56:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:56:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:56:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.055+0000 7f1b00a23640 1 -- 172.21.15.18:0/2167777188 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1ae80a22d0 msgr2=0x7f1ae80a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.055+0000 7f1b00a23640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1ae80a22d0 0x7f1ae80a4790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1af4004af0 tx=0x7f1af4034520 comp rx=0 tx=0).stop 2024-02-17T12:56:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.055+0000 7f1b00a23640 1 -- 172.21.15.18:0/2167777188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1afc100c70 msgr2=0x7f1afc196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.055+0000 7f1b00a23640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1afc100c70 0x7f1afc196b70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400c920 tx=0x7f1ae400cde0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.055+0000 7f1b00a23640 1 -- 172.21.15.18:0/2167777188 shutdown_connections 2024-02-17T12:56:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.055+0000 7f1b00a23640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1ae80a22d0 0x7f1ae80a4790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.055+0000 7f1b00a23640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1afc105000 0x7f1afc19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.055+0000 7f1b00a23640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1afc100c70 0x7f1afc196b70 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.055+0000 7f1b00a23640 1 --2- 172.21.15.18:0/2167777188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1afc1002d0 0x7f1afc196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.055+0000 7f1b00a23640 1 -- 172.21.15.18:0/2167777188 >> 172.21.15.18:0/2167777188 conn(0x7f1afc076e50 msgr2=0x7f1afc0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.055+0000 7f1b00a23640 1 -- 172.21.15.18:0/2167777188 shutdown_connections 2024-02-17T12:56:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:22.055+0000 7f1b00a23640 1 -- 172.21.15.18:0/2167777188 wait complete. 2024-02-17T12:56:22.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:21 smithi146 ceph-mon[36454]: pgmap v200: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-02-17T12:56:22.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:21 smithi018 ceph-mon[32625]: pgmap v200: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-02-17T12:56:22.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 345759 5.96 MB/sec execute 217 sec latency 1380.916 ms 2024-02-17T12:56:22.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:21 smithi119 ceph-mon[36188]: pgmap v200: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-02-17T12:56:23.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 346673 5.94 MB/sec execute 218 sec latency 1949.607 ms 2024-02-17T12:56:24.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:23 smithi146 ceph-mon[36454]: pgmap v201: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-02-17T12:56:24.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:23 smithi018 ceph-mon[32625]: pgmap v201: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-02-17T12:56:24.279 INFO:tasks.workunit.client.0.smithi018.stdout: 1 348321 5.97 MB/sec execute 219 sec latency 16.935 ms 2024-02-17T12:56:24.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:23 smithi119 ceph-mon[36188]: pgmap v201: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-02-17T12:56:25.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 349302 5.96 MB/sec execute 220 sec latency 486.360 ms 2024-02-17T12:56:26.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:25 smithi146 ceph-mon[36454]: pgmap v202: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-02-17T12:56:26.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:25 smithi018 ceph-mon[32625]: pgmap v202: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-02-17T12:56:26.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 349302 5.94 MB/sec execute 221 sec latency 1486.455 ms 2024-02-17T12:56:26.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:25 smithi119 ceph-mon[36188]: pgmap v202: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-02-17T12:56:27.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 349302 5.91 MB/sec execute 222 sec latency 2486.543 ms 2024-02-17T12:56:28.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:27 smithi146 ceph-mon[36454]: pgmap v203: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-02-17T12:56:28.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:27 smithi018 ceph-mon[32625]: pgmap v203: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-02-17T12:56:28.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 349302 5.88 MB/sec execute 223 sec latency 3486.614 ms 2024-02-17T12:56:28.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:27 smithi119 ceph-mon[36188]: pgmap v203: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-02-17T12:56:29.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 351137 5.91 MB/sec execute 224 sec latency 3682.328 ms 2024-02-17T12:56:30.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:29 smithi146 ceph-mon[36454]: pgmap v204: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 239 op/s 2024-02-17T12:56:30.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:29 smithi018 ceph-mon[32625]: pgmap v204: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 239 op/s 2024-02-17T12:56:30.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 352543 5.92 MB/sec execute 225 sec latency 11.579 ms 2024-02-17T12:56:30.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:29 smithi119 ceph-mon[36188]: pgmap v204: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 239 op/s 2024-02-17T12:56:31.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:30 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:56:31.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:30 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:56:31.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:30 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:56:31.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:30 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:56:31.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 352850 5.89 MB/sec execute 226 sec latency 795.778 ms 2024-02-17T12:56:31.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:30 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:56:31.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:30 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:56:32.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:31 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:56:32.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:31 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:56:32.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:31 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:56:32.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:31 smithi146 ceph-mon[36454]: pgmap v205: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-02-17T12:56:32.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:31 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:56:32.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:31 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:56:32.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:31 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:56:32.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:31 smithi018 ceph-mon[32625]: pgmap v205: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-02-17T12:56:32.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 352850 5.86 MB/sec execute 227 sec latency 1795.833 ms 2024-02-17T12:56:32.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:31 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:56:32.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:31 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:56:32.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:31 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:56:32.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:31 smithi119 ceph-mon[36188]: pgmap v205: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-02-17T12:56:33.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 352850 5.84 MB/sec execute 228 sec latency 2795.921 ms 2024-02-17T12:56:34.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:33 smithi146 ceph-mon[36454]: pgmap v206: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-02-17T12:56:34.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:33 smithi018 ceph-mon[32625]: pgmap v206: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-02-17T12:56:34.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 353740 5.82 MB/sec execute 229 sec latency 3403.399 ms 2024-02-17T12:56:34.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:33 smithi119 ceph-mon[36188]: pgmap v206: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-02-17T12:56:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.148+0000 7fd58c039640 1 -- 172.21.15.18:0/2425862501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd584069980 msgr2=0x7fd584069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.148+0000 7fd58c039640 1 --2- 172.21.15.18:0/2425862501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd584069980 0x7fd584069e00 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd578009930 tx=0x7fd57802bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.148+0000 7fd58c039640 1 -- 172.21.15.18:0/2425862501 shutdown_connections 2024-02-17T12:56:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.148+0000 7fd58c039640 1 --2- 172.21.15.18:0/2425862501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd58406a340 0x7fd58410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.148+0000 7fd58c039640 1 --2- 172.21.15.18:0/2425862501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd584069980 0x7fd584069e00 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.148+0000 7fd58c039640 1 --2- 172.21.15.18:0/2425862501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd584106130 0x7fd584106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.148+0000 7fd58c039640 1 -- 172.21.15.18:0/2425862501 >> 172.21.15.18:0/2425862501 conn(0x7fd584076e50 msgr2=0x7fd584077260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.148+0000 7fd58c039640 1 -- 172.21.15.18:0/2425862501 shutdown_connections 2024-02-17T12:56:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.148+0000 7fd58c039640 1 -- 172.21.15.18:0/2425862501 wait complete. 2024-02-17T12:56:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.148+0000 7fd58c039640 1 Processor -- start 2024-02-17T12:56:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd58c039640 1 -- start start 2024-02-17T12:56:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd58c039640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd584069980 0x7fd58419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd58c039640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd58406a340 0x7fd58419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd58c039640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd584106130 0x7fd584199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd58c039640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd584199e00 con 0x7fd58406a340 2024-02-17T12:56:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd58c039640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd584199f70 con 0x7fd584106130 2024-02-17T12:56:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd58c039640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58419a130 con 0x7fd584069980 2024-02-17T12:56:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd5895d4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd58406a340 0x7fd58419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd589dd5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd584069980 0x7fd58419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd589dd5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd584069980 0x7fd58419f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:57620/0 (socket says 172.21.15.18:57620) 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd5895d4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd58406a340 0x7fd58419faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35896/0 (socket says 172.21.15.18:35896) 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd589dd5640 1 -- 172.21.15.18:0/2694166917 learned_addr learned my addr 172.21.15.18:0/2694166917 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd58a5d6640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd584106130 0x7fd584199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd5895d4640 1 -- 172.21.15.18:0/2694166917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd584069980 msgr2=0x7fd58419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd5895d4640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd584069980 0x7fd58419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd5895d4640 1 -- 172.21.15.18:0/2694166917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd584106130 msgr2=0x7fd584199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd5895d4640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd584106130 0x7fd584199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd5895d4640 1 -- 172.21.15.18:0/2694166917 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd578009560 con 0x7fd58406a340 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd589dd5640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd584069980 0x7fd58419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.149+0000 7fd5895d4640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd58406a340 0x7fd58419faf0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fd578009230 tx=0x7fd57802c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.150+0000 7fd582ffd640 1 -- 172.21.15.18:0/2694166917 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5780042d0 con 0x7fd58406a340 2024-02-17T12:56:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.150+0000 7fd58c039640 1 -- 172.21.15.18:0/2694166917 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58419a3b0 con 0x7fd58406a340 2024-02-17T12:56:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.150+0000 7fd582ffd640 1 -- 172.21.15.18:0/2694166917 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd578004430 con 0x7fd58406a340 2024-02-17T12:56:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.150+0000 7fd582ffd640 1 -- 172.21.15.18:0/2694166917 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578003bd0 con 0x7fd58406a340 2024-02-17T12:56:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.150+0000 7fd58c039640 1 -- 172.21.15.18:0/2694166917 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5841a6530 con 0x7fd58406a340 2024-02-17T12:56:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.152+0000 7fd58ec0a740 1 Processor -- start 2024-02-17T12:56:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.152+0000 7fd582ffd640 1 -- 172.21.15.18:0/2694166917 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd578003d30 con 0x7fd58406a340 2024-02-17T12:56:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd58ec0a740 1 -- start start 2024-02-17T12:56:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd58ec0a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b206f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd58ec0a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 0x5629b20701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd58ec0a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b20792e0 0x5629b207b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd58ec0a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b1fde9f0 con 0x5629b206fd60 2024-02-17T12:56:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd58ec0a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b1fdbf20 con 0x5629b2070ad0 2024-02-17T12:56:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd58ec0a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b1fd7700 con 0x5629b20792e0 2024-02-17T12:56:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd589dd5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 0x5629b20701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd58a5d6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b206f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd589dd5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 0x5629b20701e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35910/0 (socket says 172.21.15.18:35910) 2024-02-17T12:56:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd589dd5640 1 -- 172.21.15.18:0/3858784075 learned_addr learned my addr 172.21.15.18:0/3858784075 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd5895d4640 1 --2- 172.21.15.18:0/3858784075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b20792e0 0x5629b207b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd582ffd640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd56c0a22d0 0x7fd56c0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd58a5d6640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd56c0a22d0 0x7fd56c0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd589dd5640 1 -- 172.21.15.18:0/3858784075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b20792e0 msgr2=0x5629b207b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd589dd5640 1 --2- 172.21.15.18:0/3858784075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b20792e0 0x5629b207b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd589dd5640 1 -- 172.21.15.18:0/3858784075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 msgr2=0x5629b206f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd589dd5640 1 --2- 172.21.15.18:0/3858784075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b206f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd589dd5640 1 -- 172.21.15.18:0/3858784075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629b1f6d7b0 con 0x5629b206fd60 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd58a5d6640 1 --2- 172.21.15.18:0/3858784075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b206f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.153+0000 7fd582ffd640 1 -- 172.21.15.18:0/2694166917 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd5780efe80 con 0x7fd58406a340 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.154+0000 7fd589dd5640 1 --2- 172.21.15.18:0/3858784075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 0x5629b20701e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fd57c011650 tx=0x7fd57c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4d23867b7adb95 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.154+0000 7fd58a5d6640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd56c0a22d0 0x7fd56c0a4790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd574005c40 tx=0x7fd574004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.154+0000 7fd56bfff640 1 -- 172.21.15.18:0/3858784075 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00f6c0 con 0x5629b206fd60 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.154+0000 7fd56bfff640 1 -- 172.21.15.18:0/3858784075 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd57c00fd00 con 0x5629b206fd60 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.154+0000 7fd56bfff640 1 -- 172.21.15.18:0/3858784075 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c019da0 con 0x5629b206fd60 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.155+0000 7fd58ec0a740 1 -- 172.21.15.18:0/3858784075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 msgr2=0x5629b20701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.155+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/3858784075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 0x5629b20701e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fd57c011650 tx=0x7fd57c009f90 comp rx=0 tx=0).stop 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.155+0000 7fd58ec0a740 1 -- 172.21.15.18:0/3858784075 shutdown_connections 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.155+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/3858784075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b20792e0 0x5629b207b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.155+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/3858784075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 0x5629b20701e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.155+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/3858784075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b206f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.155+0000 7fd58ec0a740 1 -- 172.21.15.18:0/3858784075 >> 172.21.15.18:0/3858784075 conn(0x5629b2037550 msgr2=0x5629b206aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.155+0000 7fd58ec0a740 1 -- 172.21.15.18:0/3858784075 shutdown_connections 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.155+0000 7fd58ec0a740 1 -- 172.21.15.18:0/3858784075 wait complete. 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.155+0000 7fd58ec0a740 1 -- start start 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.155+0000 7fd58ec0a740 1 Processor -- start 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.155+0000 7fd58ec0a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 0x5629b2076320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd58ec0a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b2076860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd58ec0a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b20792e0 0x5629b2076da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd58ec0a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b1f7cbf0 con 0x5629b206fd60 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd58ec0a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b1cab250 con 0x5629b2070ad0 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd58ec0a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b1f5d200 con 0x5629b20792e0 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd58ec0a740 10 client.0 _ll_register_callbacks cb 0x5629b1fcbd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd589dd5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 0x5629b2076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd589dd5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 0x5629b2076320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35922/0 (socket says 172.21.15.18:35922) 2024-02-17T12:56:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd589dd5640 1 -- 172.21.15.18:0/2460339591 learned_addr learned my addr 172.21.15.18:0/2460339591 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd5895d4640 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b2076860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd58a5d6640 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b20792e0 0x5629b2076da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd589dd5640 1 -- 172.21.15.18:0/2460339591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b20792e0 msgr2=0x5629b2076da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd589dd5640 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b20792e0 0x5629b2076da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd589dd5640 1 -- 172.21.15.18:0/2460339591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 msgr2=0x5629b2076860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd589dd5640 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b2076860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd589dd5640 1 -- 172.21.15.18:0/2460339591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c009e00 con 0x5629b206fd60 2024-02-17T12:56:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd589dd5640 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 0x5629b2076320 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fd57c019640 tx=0x7fd57c019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4b59b212b0c059 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd5697fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd5697fa640 1 -- 172.21.15.18:0/2460339591 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c004250 con 0x5629b206fd60 2024-02-17T12:56:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd5697fa640 1 -- 172.21.15.18:0/2460339591 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd57c0043b0 con 0x5629b206fd60 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2460339591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629b1cd7600 con 0x5629b206fd60 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd5697fa640 1 -- 172.21.15.18:0/2460339591 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd57c00f6c0 con 0x5629b206fd60 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd58ec0a740 10 client.17946 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.156+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2460339591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5629b1fd5d80 con 0x5629b206fd60 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.157+0000 7fd5697fa640 1 -- 172.21.15.18:0/2460339591 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5629b1cd7600 con 0x5629b206fd60 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.157+0000 7fd5697fa640 1 -- 172.21.15.18:0/2460339591 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd57c0046c0 con 0x5629b206fd60 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.157+0000 7fd58ec0a740 10 client.17946 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.157+0000 7fd58ec0a740 10 client.17946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.157+0000 7fd58ec0a740 20 client.17946 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.157+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5629b20896b0 0x5629b208bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.157+0000 7fd58ec0a740 4 client.17946 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.157+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2460339591 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629b1d281b0 con 0x5629b20896b0 2024-02-17T12:56:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.157+0000 7fd5895d4640 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5629b20896b0 0x5629b208bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.157+0000 7fd5697fa640 10 client.17946 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.157+0000 7fd5895d4640 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5629b20896b0 0x5629b208bad0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca9248ae697dae38 server_cookie=3b16602e07b01734 in_seq=0 out_seq=0 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd5697fa640 1 -- 172.21.15.18:0/2460339591 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629b1d281b0 con 0x5629b20896b0 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd5697fa640 10 client.17946 handle_command_reply: tid=0 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd58ec0a740 1 client.17946 shutdown 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2460339591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 msgr2=0x5629b2076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 0x5629b2076320 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fd57c019640 tx=0x7fd57c019670 comp rx=0 tx=0).stop 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2460339591 shutdown_connections 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5629b20896b0 0x5629b208bad0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b20792e0 0x5629b2076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b2076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd5697fa640 0 client.17946 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/2460339591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b206fd60 0x5629b2076320 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2460339591 >> 172.21.15.18:0/2460339591 conn(0x5629b2037550 msgr2=0x5629b206c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2460339591 shutdown_connections 2024-02-17T12:56:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2460339591 wait complete. 2024-02-17T12:56:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.158+0000 7fd58ec0a740 20 client.17946 trim_cache size 0 max 16384 2024-02-17T12:56:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.171+0000 7fd58ec0a740 1 Processor -- start 2024-02-17T12:56:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.171+0000 7fd58ec0a740 1 -- start start 2024-02-17T12:56:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd58ec0a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b206fd60 0x5629b2078a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd58ec0a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b2070ad0 0x5629b2076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd58ec0a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b20792e0 0x5629b2074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd58ec0a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b2090d20 con 0x5629b2070ad0 2024-02-17T12:56:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd58ec0a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b2089420 con 0x5629b20792e0 2024-02-17T12:56:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd58ec0a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b206bbc0 con 0x5629b206fd60 2024-02-17T12:56:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd5895d4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b2070ad0 0x5629b2076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd5895d4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b2070ad0 0x5629b2076bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35934/0 (socket says 172.21.15.18:35934) 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd5895d4640 1 -- 172.21.15.18:0/214609449 learned_addr learned my addr 172.21.15.18:0/214609449 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd58a5d6640 1 --2- 172.21.15.18:0/214609449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b20792e0 0x5629b2074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd589dd5640 1 --2- 172.21.15.18:0/214609449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b206fd60 0x5629b2078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd5895d4640 1 -- 172.21.15.18:0/214609449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b206fd60 msgr2=0x5629b2078a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd5895d4640 1 --2- 172.21.15.18:0/214609449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b206fd60 0x5629b2078a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd5895d4640 1 -- 172.21.15.18:0/214609449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b20792e0 msgr2=0x5629b2074bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd5895d4640 1 --2- 172.21.15.18:0/214609449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b20792e0 0x5629b2074bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd5895d4640 1 -- 172.21.15.18:0/214609449 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629b1f727b0 con 0x5629b2070ad0 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd58a5d6640 1 --2- 172.21.15.18:0/214609449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b20792e0 0x5629b2074bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd589dd5640 1 --2- 172.21.15.18:0/214609449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b206fd60 0x5629b2078a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd5895d4640 1 --2- 172.21.15.18:0/214609449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b2070ad0 0x5629b2076bc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd5780bc930 tx=0x7fd5780f2350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c640472033a0e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd56b7fe640 1 -- 172.21.15.18:0/214609449 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5780bc770 con 0x5629b2070ad0 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd56b7fe640 1 -- 172.21.15.18:0/214609449 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5780bc4e0 con 0x5629b2070ad0 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.172+0000 7fd56b7fe640 1 -- 172.21.15.18:0/214609449 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5780c0af0 con 0x5629b2070ad0 2024-02-17T12:56:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.173+0000 7fd58ec0a740 1 -- 172.21.15.18:0/214609449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b2070ad0 msgr2=0x5629b2076bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.173+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/214609449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b2070ad0 0x5629b2076bc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd5780bc930 tx=0x7fd5780f2350 comp rx=0 tx=0).stop 2024-02-17T12:56:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.173+0000 7fd58ec0a740 1 -- 172.21.15.18:0/214609449 shutdown_connections 2024-02-17T12:56:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.173+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/214609449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b20792e0 0x5629b2074bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.173+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/214609449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b2070ad0 0x5629b2076bc0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.173+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/214609449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b206fd60 0x5629b2078a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.173+0000 7fd58ec0a740 1 -- 172.21.15.18:0/214609449 >> 172.21.15.18:0/214609449 conn(0x5629b2037550 msgr2=0x5629b206a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.173+0000 7fd58ec0a740 1 -- 172.21.15.18:0/214609449 shutdown_connections 2024-02-17T12:56:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.173+0000 7fd58ec0a740 1 -- 172.21.15.18:0/214609449 wait complete. 2024-02-17T12:56:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.173+0000 7fd58ec0a740 1 -- start start 2024-02-17T12:56:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.173+0000 7fd58ec0a740 1 Processor -- start 2024-02-17T12:56:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58ec0a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b206fd60 0x5629b207e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58ec0a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b207e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58ec0a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b20792e0 0x5629b20debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58ec0a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b20df130 con 0x5629b20792e0 2024-02-17T12:56:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58ec0a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b20df270 con 0x5629b2070ad0 2024-02-17T12:56:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58ec0a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b20df3b0 con 0x5629b206fd60 2024-02-17T12:56:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58ec0a740 10 client.0 _ll_register_callbacks cb 0x7fd57c009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd5895d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b207e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd589dd5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b206fd60 0x5629b207e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58a5d6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b20792e0 0x5629b20debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd5895d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b207e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:35564/0 (socket says 172.21.15.18:35564) 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd589dd5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b206fd60 0x5629b207e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:57648/0 (socket says 172.21.15.18:57648) 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd5895d4640 1 -- 172.21.15.18:0/2747209348 learned_addr learned my addr 172.21.15.18:0/2747209348 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58a5d6640 1 -- 172.21.15.18:0/2747209348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b206fd60 msgr2=0x5629b207e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58a5d6640 1 --2- 172.21.15.18:0/2747209348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b206fd60 0x5629b207e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58a5d6640 1 -- 172.21.15.18:0/2747209348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 msgr2=0x5629b207e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58a5d6640 1 --2- 172.21.15.18:0/2747209348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b207e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58a5d6640 1 -- 172.21.15.18:0/2747209348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5841a6530 con 0x5629b20792e0 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd58a5d6640 1 --2- 172.21.15.18:0/2747209348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b20792e0 0x5629b20debf0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd574005fd0 tx=0x7fd574004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dc3f3d0606d6b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.174+0000 7fd5697fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd5697fa640 1 -- 172.21.15.18:0/2747209348 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574004460 con 0x5629b20792e0 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd5697fa640 1 -- 172.21.15.18:0/2747209348 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd5740045c0 con 0x5629b20792e0 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2747209348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd57c00fd00 con 0x5629b20792e0 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd5697fa640 1 -- 172.21.15.18:0/2747209348 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd574003ec0 con 0x5629b20792e0 2024-02-17T12:56:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd58ec0a740 10 client.17952 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2747209348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5629b1f98540 con 0x5629b20792e0 2024-02-17T12:56:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd5697fa640 1 -- 172.21.15.18:0/2747209348 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd57c00fd00 con 0x5629b20792e0 2024-02-17T12:56:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd5697fa640 1 -- 172.21.15.18:0/2747209348 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd574022450 con 0x5629b20792e0 2024-02-17T12:56:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd58ec0a740 10 client.17952 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd58ec0a740 10 client.17952 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd58ec0a740 20 client.17952 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/2747209348 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5629b20e0d30 0x5629b20e3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd58ec0a740 4 client.17952 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:56:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2747209348 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629b1d14550 con 0x5629b20e0d30 2024-02-17T12:56:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.175+0000 7fd589dd5640 1 --2- 172.21.15.18:0/2747209348 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5629b20e0d30 0x5629b20e3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.176+0000 7fd5697fa640 10 client.17952 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.176+0000 7fd589dd5640 1 --2- 172.21.15.18:0/2747209348 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5629b20e0d30 0x5629b20e3150 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3018b038897be51 server_cookie=6cd5adbecdaeae18 in_seq=0 out_seq=0 2024-02-17T12:56:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd5697fa640 1 -- 172.21.15.18:0/2747209348 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629b1d14550 con 0x5629b20e0d30 2024-02-17T12:56:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd5697fa640 10 client.17952 handle_command_reply: tid=0 2024-02-17T12:56:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd58ec0a740 1 client.17952 shutdown 2024-02-17T12:56:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2747209348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b20792e0 msgr2=0x5629b20debf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/2747209348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b20792e0 0x5629b20debf0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd574005fd0 tx=0x7fd574004a40 comp rx=0 tx=0).stop 2024-02-17T12:56:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2747209348 shutdown_connections 2024-02-17T12:56:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/2747209348 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5629b20e0d30 0x5629b20e3150 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd5697fa640 0 client.17952 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/2747209348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b20792e0 0x5629b20debf0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/2747209348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b2070ad0 0x5629b207e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd58ec0a740 1 --2- 172.21.15.18:0/2747209348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629b206fd60 0x5629b207e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2747209348 >> 172.21.15.18:0/2747209348 conn(0x5629b2037550 msgr2=0x5629b20747b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2747209348 shutdown_connections 2024-02-17T12:56:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd58ec0a740 1 -- 172.21.15.18:0/2747209348 wait complete. 2024-02-17T12:56:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.210+0000 7fd58ec0a740 20 client.17952 trim_cache size 0 max 16384 2024-02-17T12:56:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:56:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:56:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:56:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:56:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.211+0000 7fd58c039640 1 -- 172.21.15.18:0/2694166917 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd56c0a22d0 msgr2=0x7fd56c0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.211+0000 7fd58c039640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd56c0a22d0 0x7fd56c0a4790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd574005c40 tx=0x7fd574004ab0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.211+0000 7fd58c039640 1 -- 172.21.15.18:0/2694166917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd58406a340 msgr2=0x7fd58419faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.211+0000 7fd58c039640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd58406a340 0x7fd58419faf0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fd578009230 tx=0x7fd57802c5d0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.211+0000 7fd58c039640 1 -- 172.21.15.18:0/2694166917 shutdown_connections 2024-02-17T12:56:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.211+0000 7fd58c039640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd56c0a22d0 0x7fd56c0a4790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.211+0000 7fd58c039640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd584106130 0x7fd584199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.211+0000 7fd58c039640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd58406a340 0x7fd58419faf0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.211+0000 7fd58c039640 1 --2- 172.21.15.18:0/2694166917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd584069980 0x7fd58419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.211+0000 7fd58c039640 1 -- 172.21.15.18:0/2694166917 >> 172.21.15.18:0/2694166917 conn(0x7fd584076e50 msgr2=0x7fd58410c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.211+0000 7fd58c039640 1 -- 172.21.15.18:0/2694166917 shutdown_connections 2024-02-17T12:56:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:35.212+0000 7fd58c039640 1 -- 172.21.15.18:0/2694166917 wait complete. 2024-02-17T12:56:35.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 355294 5.85 MB/sec execute 230 sec latency 28.257 ms 2024-02-17T12:56:36.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:35 smithi146 ceph-mon[36454]: pgmap v207: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-02-17T12:56:36.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:35 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:56:36.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:35 smithi018 ceph-mon[32625]: pgmap v207: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-02-17T12:56:36.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:35 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:56:36.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 356392 5.85 MB/sec execute 231 sec latency 411.207 ms 2024-02-17T12:56:36.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:35 smithi119 ceph-mon[36188]: pgmap v207: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-02-17T12:56:36.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:35 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:56:37.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 356392 5.82 MB/sec execute 232 sec latency 1411.299 ms 2024-02-17T12:56:38.144 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:37 smithi119 ceph-mon[36188]: pgmap v208: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1012 KiB/s wr, 98 op/s 2024-02-17T12:56:38.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:37 smithi146 ceph-mon[36454]: pgmap v208: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1012 KiB/s wr, 98 op/s 2024-02-17T12:56:38.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:37 smithi018 ceph-mon[32625]: pgmap v208: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1012 KiB/s wr, 98 op/s 2024-02-17T12:56:38.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 356392 5.80 MB/sec execute 233 sec latency 2411.365 ms 2024-02-17T12:56:39.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 357262 5.78 MB/sec execute 234 sec latency 3029.047 ms 2024-02-17T12:56:40.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:39 smithi146 ceph-mon[36454]: pgmap v209: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-02-17T12:56:40.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:39 smithi018 ceph-mon[32625]: pgmap v209: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-02-17T12:56:40.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 358824 5.80 MB/sec execute 235 sec latency 21.635 ms 2024-02-17T12:56:40.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:39 smithi119 ceph-mon[36188]: pgmap v209: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-02-17T12:56:41.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 359941 5.80 MB/sec execute 236 sec latency 372.009 ms 2024-02-17T12:56:42.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:41 smithi146 ceph-mon[36454]: pgmap v210: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-17T12:56:42.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:41 smithi018 ceph-mon[32625]: pgmap v210: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-17T12:56:42.280 INFO:tasks.workunit.client.0.smithi018.stdout: 1 359941 5.78 MB/sec execute 237 sec latency 1372.089 ms 2024-02-17T12:56:42.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:41 smithi119 ceph-mon[36188]: pgmap v210: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-17T12:56:43.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 361011 5.78 MB/sec execute 238 sec latency 1941.839 ms 2024-02-17T12:56:44.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:43 smithi146 ceph-mon[36454]: pgmap v211: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-17T12:56:44.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:43 smithi018 ceph-mon[32625]: pgmap v211: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-17T12:56:44.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 362513 5.79 MB/sec execute 239 sec latency 15.559 ms 2024-02-17T12:56:44.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:43 smithi119 ceph-mon[36188]: pgmap v211: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-17T12:56:45.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 363483 5.79 MB/sec execute 240 sec latency 472.227 ms 2024-02-17T12:56:46.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:45 smithi146 ceph-mon[36454]: pgmap v212: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 194 op/s 2024-02-17T12:56:46.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:45 smithi018 ceph-mon[32625]: pgmap v212: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 194 op/s 2024-02-17T12:56:46.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 363483 5.76 MB/sec execute 241 sec latency 1472.283 ms 2024-02-17T12:56:46.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:45 smithi119 ceph-mon[36188]: pgmap v212: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 194 op/s 2024-02-17T12:56:46.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:56:46.597 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:56:46.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.724+0000 7fb0415c7640 1 -- 172.21.15.18:0/2155234711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb03c100270 msgr2=0x7fb03c100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:46.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.724+0000 7fb0415c7640 1 --2- 172.21.15.18:0/2155234711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb03c100270 0x7fb03c100650 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009930 tx=0x7fb02c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:46.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.724+0000 7fb0415c7640 1 -- 172.21.15.18:0/2155234711 shutdown_connections 2024-02-17T12:56:46.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.724+0000 7fb0415c7640 1 --2- 172.21.15.18:0/2155234711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c105340 0x7fb03c10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.724+0000 7fb0415c7640 1 --2- 172.21.15.18:0/2155234711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb03c100c20 0x7fb03c104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.724+0000 7fb0415c7640 1 --2- 172.21.15.18:0/2155234711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb03c100270 0x7fb03c100650 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.724+0000 7fb0415c7640 1 -- 172.21.15.18:0/2155234711 >> 172.21.15.18:0/2155234711 conn(0x7fb03c077230 msgr2=0x7fb03c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.724+0000 7fb0415c7640 1 -- 172.21.15.18:0/2155234711 shutdown_connections 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.724+0000 7fb0415c7640 1 -- 172.21.15.18:0/2155234711 wait complete. 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb0415c7640 1 Processor -- start 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb0415c7640 1 -- start start 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb0415c7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c100270 0x7fb03c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb0415c7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb03c105340 0x7fb03c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb0415c7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb03c1975e0 0x7fb03c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb0415c7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c197fd0 con 0x7fb03c105340 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb0415c7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c198140 con 0x7fb03c100270 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb0415c7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c198300 con 0x7fb03c1975e0 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb033fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb03c105340 0x7fb03c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb033fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb03c105340 0x7fb03c19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41496/0 (socket says 172.21.15.18:41496) 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb033fff640 1 -- 172.21.15.18:0/3505981414 learned_addr learned my addr 172.21.15.18:0/3505981414 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:46.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb03affd640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c100270 0x7fb03c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb03b7fe640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb03c1975e0 0x7fb03c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb033fff640 1 -- 172.21.15.18:0/3505981414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb03c1975e0 msgr2=0x7fb03c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.725+0000 7fb033fff640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb03c1975e0 0x7fb03c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.726+0000 7fb033fff640 1 -- 172.21.15.18:0/3505981414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c100270 msgr2=0x7fb03c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.726+0000 7fb033fff640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c100270 0x7fb03c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.726+0000 7fb033fff640 1 -- 172.21.15.18:0/3505981414 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c009560 con 0x7fb03c105340 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.726+0000 7fb03affd640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c100270 0x7fb03c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.726+0000 7fb033fff640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb03c105340 0x7fb03c19d970 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb02400d740 tx=0x7fb02400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.726+0000 7fb038ff9640 1 -- 172.21.15.18:0/3505981414 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024009c30 con 0x7fb03c105340 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.726+0000 7fb0415c7640 1 -- 172.21.15.18:0/3505981414 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c1a42f0 con 0x7fb03c105340 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.726+0000 7fb038ff9640 1 -- 172.21.15.18:0/3505981414 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb02400de50 con 0x7fb03c105340 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.726+0000 7fb038ff9640 1 -- 172.21.15.18:0/3505981414 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024010460 con 0x7fb03c105340 2024-02-17T12:56:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.726+0000 7fb0415c7640 1 -- 172.21.15.18:0/3505981414 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c1a4800 con 0x7fb03c105340 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.728+0000 7fb038ff9640 1 -- 172.21.15.18:0/3505981414 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb024004f10 con 0x7fb03c105340 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.728+0000 7fb044197740 1 Processor -- start 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.728+0000 7fb044197740 1 -- start start 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb044197740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d612b0 0x55b8d9d61690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb044197740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d6a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb044197740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d6cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb044197740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d9cd0280 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb044197740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d9ccd7b0 con 0x55b8d9d61bd0 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb044197740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d9cc8f90 con 0x55b8d9d612b0 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb038ff9640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb0280a2230 0x7fb0280a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb03b7fe640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb0280a2230 0x7fb0280a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb033fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d6cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb033fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d6cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41504/0 (socket says 172.21.15.18:41504) 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb033fff640 1 -- 172.21.15.18:0/3913928655 learned_addr learned my addr 172.21.15.18:0/3913928655 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb03b7fe640 1 --2- 172.21.15.18:0/3913928655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d612b0 0x55b8d9d61690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb038ff9640 1 -- 172.21.15.18:0/3505981414 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb0240cda50 con 0x7fb03c105340 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb03affd640 1 --2- 172.21.15.18:0/3913928655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d6a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb033fff640 1 -- 172.21.15.18:0/3913928655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d612b0 msgr2=0x55b8d9d61690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb033fff640 1 --2- 172.21.15.18:0/3913928655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d612b0 0x55b8d9d61690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb033fff640 1 -- 172.21.15.18:0/3913928655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d61bd0 msgr2=0x55b8d9d6a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb033fff640 1 --2- 172.21.15.18:0/3913928655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d6a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.729+0000 7fb033fff640 1 -- 172.21.15.18:0/3913928655 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8d9c87570 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.730+0000 7fb03b7fe640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb0280a2230 0x7fb0280a46f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb034005fd0 tx=0x7fb034009e80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.730+0000 7fb033fff640 1 --2- 172.21.15.18:0/3913928655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d6cba0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb0240cb7c0 tx=0x7fb02406d4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8fd8239131651 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.730+0000 7fb0317fa640 1 -- 172.21.15.18:0/3913928655 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0240ba070 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.730+0000 7fb0317fa640 1 -- 172.21.15.18:0/3913928655 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb02409ab60 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.730+0000 7fb0317fa640 1 -- 172.21.15.18:0/3913928655 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0240cb440 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 -- 172.21.15.18:0/3913928655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 msgr2=0x55b8d9d6cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:46.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 --2- 172.21.15.18:0/3913928655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d6cba0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb0240cb7c0 tx=0x7fb02406d4d0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 -- 172.21.15.18:0/3913928655 shutdown_connections 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 --2- 172.21.15.18:0/3913928655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d6cba0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 --2- 172.21.15.18:0/3913928655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d6a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 --2- 172.21.15.18:0/3913928655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d612b0 0x55b8d9d61690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 -- 172.21.15.18:0/3913928655 >> 172.21.15.18:0/3913928655 conn(0x55b8d9d5b990 msgr2=0x55b8d9d5bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 -- 172.21.15.18:0/3913928655 shutdown_connections 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 -- 172.21.15.18:0/3913928655 wait complete. 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 -- start start 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 Processor -- start 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d612b0 0x55b8d9d6a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d67560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d67b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d999c430 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d9958e00 con 0x55b8d9d612b0 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d9c4dfd0 con 0x55b8d9d61bd0 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb044197740 10 client.0 _ll_register_callbacks cb 0x55b8d9ca3fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.731+0000 7fb03b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d67b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb03b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d67b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41520/0 (socket says 172.21.15.18:41520) 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb03b7fe640 1 -- 172.21.15.18:0/210430859 learned_addr learned my addr 172.21.15.18:0/210430859 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb03affd640 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d612b0 0x55b8d9d6a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb033fff640 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d67560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb03b7fe640 1 -- 172.21.15.18:0/210430859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d61bd0 msgr2=0x55b8d9d67560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb03b7fe640 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d67560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb03b7fe640 1 -- 172.21.15.18:0/210430859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d612b0 msgr2=0x55b8d9d6a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb03b7fe640 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d612b0 0x55b8d9d6a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb03b7fe640 1 -- 172.21.15.18:0/210430859 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c1a4800 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb03b7fe640 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d67b70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb034009dc0 tx=0x7fb03401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24507b1e2e449938 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:46.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb0227fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb0227fc640 1 -- 172.21.15.18:0/210430859 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034016040 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb0227fc640 1 -- 172.21.15.18:0/210430859 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb034004740 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb044197740 1 -- 172.21.15.18:0/210430859 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8d99c87e0 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb0227fc640 1 -- 172.21.15.18:0/210430859 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb03401f450 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb044197740 10 client.17964 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.732+0000 7fb044197740 1 -- 172.21.15.18:0/210430859 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55b8d9cc7610 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.733+0000 7fb0227fc640 1 -- 172.21.15.18:0/210430859 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b8d99c87e0 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.733+0000 7fb0227fc640 1 -- 172.21.15.18:0/210430859 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb034026030 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.733+0000 7fb044197740 10 client.17964 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.733+0000 7fb044197740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.733+0000 7fb044197740 20 client.17964 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.733+0000 7fb044197740 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b8d9d7a9f0 0x55b8d9d7ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.733+0000 7fb044197740 4 client.17964 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.733+0000 7fb044197740 1 -- 172.21.15.18:0/210430859 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8d9a05720 con 0x55b8d9d7a9f0 2024-02-17T12:56:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.733+0000 7fb03affd640 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b8d9d7a9f0 0x55b8d9d7ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.733+0000 7fb0227fc640 10 client.17964 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.733+0000 7fb03affd640 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b8d9d7a9f0 0x55b8d9d7ce10 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac432f962cd2833 server_cookie=296d39cbde26c6c3 in_seq=0 out_seq=0 2024-02-17T12:56:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb0227fc640 1 -- 172.21.15.18:0/210430859 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b8d9a05720 con 0x55b8d9d7a9f0 2024-02-17T12:56:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb0227fc640 10 client.17964 handle_command_reply: tid=0 2024-02-17T12:56:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb044197740 1 client.17964 shutdown 2024-02-17T12:56:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb044197740 1 -- 172.21.15.18:0/210430859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 msgr2=0x55b8d9d67b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb044197740 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d67b70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb034009dc0 tx=0x7fb03401d000 comp rx=0 tx=0).stop 2024-02-17T12:56:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb044197740 1 -- 172.21.15.18:0/210430859 shutdown_connections 2024-02-17T12:56:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb044197740 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b8d9d7a9f0 0x55b8d9d7ce10 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb044197740 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d67b70 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb0227fc640 0 client.17964 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:46.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb044197740 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d67560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb044197740 1 --2- 172.21.15.18:0/210430859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d612b0 0x55b8d9d6a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb044197740 1 -- 172.21.15.18:0/210430859 >> 172.21.15.18:0/210430859 conn(0x55b8d9d5b990 msgr2=0x55b8d9d5d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:46.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb044197740 1 -- 172.21.15.18:0/210430859 shutdown_connections 2024-02-17T12:56:46.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb044197740 1 -- 172.21.15.18:0/210430859 wait complete. 2024-02-17T12:56:46.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.734+0000 7fb044197740 20 client.17964 trim_cache size 0 max 16384 2024-02-17T12:56:46.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb044197740 1 Processor -- start 2024-02-17T12:56:46.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb044197740 1 -- start start 2024-02-17T12:56:46.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb044197740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d612b0 0x55b8d9d71980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb044197740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d5a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb044197740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d69c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb044197740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d9d77390 con 0x55b8d9d61bd0 2024-02-17T12:56:46.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb044197740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d9d68170 con 0x55b8d9d612b0 2024-02-17T12:56:46.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb044197740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d9d572b0 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb033fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d5a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb033fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d5a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41536/0 (socket says 172.21.15.18:41536) 2024-02-17T12:56:46.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb033fff640 1 -- 172.21.15.18:0/507763922 learned_addr learned my addr 172.21.15.18:0/507763922 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:46.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb03b7fe640 1 --2- 172.21.15.18:0/507763922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d69c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb033fff640 1 -- 172.21.15.18:0/507763922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d6a7b0 msgr2=0x55b8d9d69c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb03affd640 1 --2- 172.21.15.18:0/507763922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d612b0 0x55b8d9d71980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb033fff640 1 --2- 172.21.15.18:0/507763922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d69c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb033fff640 1 -- 172.21.15.18:0/507763922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d612b0 msgr2=0x55b8d9d71980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb033fff640 1 --2- 172.21.15.18:0/507763922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d612b0 0x55b8d9d71980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb033fff640 1 -- 172.21.15.18:0/507763922 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8d9c63ac0 con 0x55b8d9d61bd0 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.750+0000 7fb03b7fe640 1 --2- 172.21.15.18:0/507763922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d69c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb033fff640 1 --2- 172.21.15.18:0/507763922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d5a8d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb0240968c0 tx=0x7fb0240049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41679d2ced99a08 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb030ff9640 1 -- 172.21.15.18:0/507763922 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0240a2c00 con 0x55b8d9d61bd0 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb030ff9640 1 -- 172.21.15.18:0/507763922 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb02409e5b0 con 0x55b8d9d61bd0 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb030ff9640 1 -- 172.21.15.18:0/507763922 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024096ad0 con 0x55b8d9d61bd0 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb044197740 1 -- 172.21.15.18:0/507763922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d61bd0 msgr2=0x55b8d9d5a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb044197740 1 --2- 172.21.15.18:0/507763922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d5a8d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb0240968c0 tx=0x7fb0240049b0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb044197740 1 -- 172.21.15.18:0/507763922 shutdown_connections 2024-02-17T12:56:46.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb044197740 1 --2- 172.21.15.18:0/507763922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9d69c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb044197740 1 --2- 172.21.15.18:0/507763922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d61bd0 0x55b8d9d5a8d0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb044197740 1 --2- 172.21.15.18:0/507763922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d612b0 0x55b8d9d71980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb044197740 1 -- 172.21.15.18:0/507763922 >> 172.21.15.18:0/507763922 conn(0x55b8d9d5b990 msgr2=0x55b8d9d5a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb044197740 1 -- 172.21.15.18:0/507763922 shutdown_connections 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.751+0000 7fb044197740 1 -- 172.21.15.18:0/507763922 wait complete. 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb044197740 1 -- start start 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb044197740 1 Processor -- start 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb044197740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d612b0 0x55b8d9dcf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb044197740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d61bd0 0x55b8d9dcf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb044197740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9dc96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb044197740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d9dc9c60 con 0x55b8d9d612b0 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb044197740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d9dc9dd0 con 0x55b8d9d61bd0 2024-02-17T12:56:46.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb044197740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8d9dc9f90 con 0x55b8d9d6a7b0 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb044197740 10 client.0 _ll_register_callbacks cb 0x7fb024096740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb03affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d612b0 0x55b8d9dcf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb03affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d612b0 0x55b8d9dcf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41548/0 (socket says 172.21.15.18:41548) 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb03affd640 1 -- 172.21.15.18:0/3371904046 learned_addr learned my addr 172.21.15.18:0/3371904046 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb033fff640 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d61bd0 0x55b8d9dcf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb03b7fe640 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9dc96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb03affd640 1 -- 172.21.15.18:0/3371904046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d6a7b0 msgr2=0x55b8d9dc96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb03affd640 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9dc96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb03affd640 1 -- 172.21.15.18:0/3371904046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d61bd0 msgr2=0x55b8d9dcf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb03affd640 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d61bd0 0x55b8d9dcf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.752+0000 7fb03affd640 1 -- 172.21.15.18:0/3371904046 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8d9c87570 con 0x55b8d9d612b0 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb03affd640 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d612b0 0x55b8d9dcf290 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb02c005940 tx=0x7fb02c004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216e357e0f3b9603 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb0227fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:46.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb0227fc640 1 -- 172.21.15.18:0/3371904046 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c004240 con 0x55b8d9d612b0 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb0227fc640 1 -- 172.21.15.18:0/3371904046 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb02c045040 con 0x55b8d9d612b0 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb044197740 1 -- 172.21.15.18:0/3371904046 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb02409ab60 con 0x55b8d9d612b0 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb0227fc640 1 -- 172.21.15.18:0/3371904046 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb02c0043a0 con 0x55b8d9d612b0 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb044197740 10 client.17970 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb044197740 1 -- 172.21.15.18:0/3371904046 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55b8d9c896a0 con 0x55b8d9d612b0 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb0227fc640 1 -- 172.21.15.18:0/3371904046 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb02409ab60 con 0x55b8d9d612b0 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb0227fc640 1 -- 172.21.15.18:0/3371904046 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb02c03c040 con 0x55b8d9d612b0 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb044197740 10 client.17970 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb044197740 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb044197740 20 client.17970 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb044197740 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b8d9dd71e0 0x55b8d9dd9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb044197740 4 client.17970 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:56:46.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.753+0000 7fb044197740 1 -- 172.21.15.18:0/3371904046 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b8d9a54890 con 0x55b8d9dd71e0 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb033fff640 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b8d9dd71e0 0x55b8d9dd9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb033fff640 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b8d9dd71e0 0x55b8d9dd9600 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c816edf994d101 server_cookie=91441833d613fb95 in_seq=0 out_seq=0 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb0227fc640 10 client.17970 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb0227fc640 1 -- 172.21.15.18:0/3371904046 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b8d9a54890 con 0x55b8d9dd71e0 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb0227fc640 10 client.17970 handle_command_reply: tid=0 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb044197740 1 client.17970 shutdown 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb044197740 1 -- 172.21.15.18:0/3371904046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d612b0 msgr2=0x55b8d9dcf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb044197740 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d612b0 0x55b8d9dcf290 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb02c005940 tx=0x7fb02c004a80 comp rx=0 tx=0).stop 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb044197740 1 -- 172.21.15.18:0/3371904046 shutdown_connections 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb044197740 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b8d9dd71e0 0x55b8d9dd9600 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb044197740 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8d9d6a7b0 0x55b8d9dc96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb044197740 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8d9d61bd0 0x55b8d9dcf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb0227fc640 0 client.17970 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:46.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb044197740 1 --2- 172.21.15.18:0/3371904046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8d9d612b0 0x55b8d9dcf290 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb044197740 1 -- 172.21.15.18:0/3371904046 >> 172.21.15.18:0/3371904046 conn(0x55b8d9d5b990 msgr2=0x55b8d9d5a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:46.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb044197740 1 -- 172.21.15.18:0/3371904046 shutdown_connections 2024-02-17T12:56:46.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb044197740 1 -- 172.21.15.18:0/3371904046 wait complete. 2024-02-17T12:56:46.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.754+0000 7fb044197740 20 client.17970 trim_cache size 0 max 16384 2024-02-17T12:56:46.752 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:56:46.752 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:56:46.752 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:56:46.753 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:56:46.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.755+0000 7fb0415c7640 1 -- 172.21.15.18:0/3505981414 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb0280a2230 msgr2=0x7fb0280a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:46.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.755+0000 7fb0415c7640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb0280a2230 0x7fb0280a46f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb034005fd0 tx=0x7fb034009e80 comp rx=0 tx=0).stop 2024-02-17T12:56:46.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.755+0000 7fb0415c7640 1 -- 172.21.15.18:0/3505981414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb03c105340 msgr2=0x7fb03c19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:46.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.755+0000 7fb0415c7640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb03c105340 0x7fb03c19d970 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb02400d740 tx=0x7fb02400dc00 comp rx=0 tx=0).stop 2024-02-17T12:56:46.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.755+0000 7fb0415c7640 1 -- 172.21.15.18:0/3505981414 shutdown_connections 2024-02-17T12:56:46.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.755+0000 7fb0415c7640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb0280a2230 0x7fb0280a46f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.755+0000 7fb0415c7640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb03c1975e0 0x7fb03c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.755+0000 7fb0415c7640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb03c105340 0x7fb03c19d970 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.755+0000 7fb0415c7640 1 --2- 172.21.15.18:0/3505981414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c100270 0x7fb03c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.755+0000 7fb0415c7640 1 -- 172.21.15.18:0/3505981414 >> 172.21.15.18:0/3505981414 conn(0x7fb03c077230 msgr2=0x7fb03c0fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:46.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.755+0000 7fb0415c7640 1 -- 172.21.15.18:0/3505981414 shutdown_connections 2024-02-17T12:56:46.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.755+0000 7fb0415c7640 1 -- 172.21.15.18:0/3505981414 wait complete. 2024-02-17T12:56:46.756 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:56:46.757 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:56:46.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.869+0000 7f2b9508a640 1 -- 172.21.15.18:0/2833931659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b90101380 msgr2=0x7f2b90101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:46.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.869+0000 7f2b9508a640 1 --2- 172.21.15.18:0/2833931659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b90101380 0x7f2b90101760 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009930 tx=0x7f2b8802bc20 comp rx=0 tx=0).stop 2024-02-17T12:56:46.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.869+0000 7f2b9508a640 1 -- 172.21.15.18:0/2833931659 shutdown_connections 2024-02-17T12:56:46.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.869+0000 7f2b9508a640 1 --2- 172.21.15.18:0/2833931659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b90106450 0x7f2b9010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.869+0000 7f2b9508a640 1 --2- 172.21.15.18:0/2833931659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b90101d30 0x7f2b90105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.869+0000 7f2b9508a640 1 --2- 172.21.15.18:0/2833931659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b90101380 0x7f2b90101760 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.869+0000 7f2b9508a640 1 -- 172.21.15.18:0/2833931659 >> 172.21.15.18:0/2833931659 conn(0x7f2b900fcc20 msgr2=0x7f2b900ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:46.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.869+0000 7f2b9508a640 1 -- 172.21.15.18:0/2833931659 shutdown_connections 2024-02-17T12:56:46.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.869+0000 7f2b9508a640 1 -- 172.21.15.18:0/2833931659 wait complete. 2024-02-17T12:56:46.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b9508a640 1 Processor -- start 2024-02-17T12:56:46.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b9508a640 1 -- start start 2024-02-17T12:56:46.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b9508a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b90101380 0x7f2b9019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b9508a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b90106450 0x7f2b9019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b9508a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b90197630 0x7f2b90197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b9508a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90198020 con 0x7f2b90197630 2024-02-17T12:56:46.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b9508a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90198190 con 0x7f2b90101380 2024-02-17T12:56:46.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b9508a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90198350 con 0x7f2b90106450 2024-02-17T12:56:46.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b8f59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b90197630 0x7f2b90197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b8f59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b90197630 0x7f2b90197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41556/0 (socket says 172.21.15.18:41556) 2024-02-17T12:56:46.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b8f59e640 1 -- 172.21.15.18:0/3696564147 learned_addr learned my addr 172.21.15.18:0/3696564147 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:46.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b8e59c640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b90106450 0x7f2b9019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b8ed9d640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b90101380 0x7f2b9019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b8f59e640 1 -- 172.21.15.18:0/3696564147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b90106450 msgr2=0x7f2b9019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:46.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b8f59e640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b90106450 0x7f2b9019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b8f59e640 1 -- 172.21.15.18:0/3696564147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b90101380 msgr2=0x7f2b9019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:46.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b8f59e640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b90101380 0x7f2b9019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:46.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.870+0000 7f2b8f59e640 1 -- 172.21.15.18:0/3696564147 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b88009560 con 0x7f2b90197630 2024-02-17T12:56:46.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.871+0000 7f2b8f59e640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b90197630 0x7f2b90197ae0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000cce0 tx=0x7f2b8000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:46.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.871+0000 7f2b87fff640 1 -- 172.21.15.18:0/3696564147 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8000f450 con 0x7f2b90197630 2024-02-17T12:56:46.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.871+0000 7f2b9508a640 1 -- 172.21.15.18:0/3696564147 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b901a42f0 con 0x7f2b90197630 2024-02-17T12:56:46.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.871+0000 7f2b87fff640 1 -- 172.21.15.18:0/3696564147 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b8000fa90 con 0x7f2b90197630 2024-02-17T12:56:46.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.871+0000 7f2b87fff640 1 -- 172.21.15.18:0/3696564147 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b80018b90 con 0x7f2b90197630 2024-02-17T12:56:46.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.871+0000 7f2b9508a640 1 -- 172.21.15.18:0/3696564147 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b901a47b0 con 0x7f2b90197630 2024-02-17T12:56:46.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.872+0000 7f2b9508a640 1 -- 172.21.15.18:0/3696564147 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b90198b10 con 0x7f2b90197630 2024-02-17T12:56:46.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.873+0000 7f2b87fff640 1 -- 172.21.15.18:0/3696564147 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2b8000fc00 con 0x7f2b90197630 2024-02-17T12:56:46.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.873+0000 7f2b87fff640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b7c0a2230 0x7f2b7c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:46.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.874+0000 7f2b8ed9d640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b7c0a2230 0x7f2b7c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:46.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.874+0000 7f2b87fff640 1 -- 172.21.15.18:0/3696564147 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2b800ce880 con 0x7f2b90197630 2024-02-17T12:56:46.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.874+0000 7f2b8ed9d640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b7c0a2230 0x7f2b7c0a46f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b88006010 tx=0x7f2b88000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:46.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:46.877+0000 7f2b87fff640 1 -- 172.21.15.18:0/3696564147 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b800975c0 con 0x7f2b90197630 2024-02-17T12:56:47.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.099+0000 7f2b9508a640 1 -- 172.21.15.18:0/3696564147 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b901096b0 con 0x7f2b90197630 2024-02-17T12:56:47.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.100+0000 7f2b87fff640 1 -- 172.21.15.18:0/3696564147 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11401 (secure 0 0 0) 0x7f2b8009ac80 con 0x7f2b90197630 2024-02-17T12:56:47.091 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:56:47.091 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:56:33.886411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:56:47.091 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 41 2024-02-17T12:56:47.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.102+0000 7f2b9508a640 1 -- 172.21.15.18:0/3696564147 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b7c0a2230 msgr2=0x7f2b7c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.102+0000 7f2b9508a640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b7c0a2230 0x7f2b7c0a46f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b88006010 tx=0x7f2b88000bc0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.102+0000 7f2b9508a640 1 -- 172.21.15.18:0/3696564147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b90197630 msgr2=0x7f2b90197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.102+0000 7f2b9508a640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b90197630 0x7f2b90197ae0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000cce0 tx=0x7f2b8000a620 comp rx=0 tx=0).stop 2024-02-17T12:56:47.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.102+0000 7f2b9508a640 1 -- 172.21.15.18:0/3696564147 shutdown_connections 2024-02-17T12:56:47.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.102+0000 7f2b9508a640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b7c0a2230 0x7f2b7c0a46f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.102+0000 7f2b9508a640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b90197630 0x7f2b90197ae0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.102+0000 7f2b9508a640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b90106450 0x7f2b9019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.102+0000 7f2b9508a640 1 --2- 172.21.15.18:0/3696564147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b90101380 0x7f2b9019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.102+0000 7f2b9508a640 1 -- 172.21.15.18:0/3696564147 >> 172.21.15.18:0/3696564147 conn(0x7f2b900fcc20 msgr2=0x7f2b900fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.102+0000 7f2b9508a640 1 -- 172.21.15.18:0/3696564147 shutdown_connections 2024-02-17T12:56:47.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.102+0000 7f2b9508a640 1 -- 172.21.15.18:0/3696564147 wait complete. 2024-02-17T12:56:47.103 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:56:47.104 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:56:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.217+0000 7f58983d7640 1 -- 172.21.15.18:0/2359813811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5890100260 msgr2=0x7f5890100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.217+0000 7f58983d7640 1 --2- 172.21.15.18:0/2359813811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5890100260 0x7f5890100640 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5888009930 tx=0x7f588802bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 -- 172.21.15.18:0/2359813811 shutdown_connections 2024-02-17T12:56:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 --2- 172.21.15.18:0/2359813811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5890105330 0x7f589010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 --2- 172.21.15.18:0/2359813811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5890100c10 0x7f5890104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 --2- 172.21.15.18:0/2359813811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5890100260 0x7f5890100640 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 -- 172.21.15.18:0/2359813811 >> 172.21.15.18:0/2359813811 conn(0x7f5890077180 msgr2=0x7f5890077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 -- 172.21.15.18:0/2359813811 shutdown_connections 2024-02-17T12:56:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 -- 172.21.15.18:0/2359813811 wait complete. 2024-02-17T12:56:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 Processor -- start 2024-02-17T12:56:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 -- start start 2024-02-17T12:56:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5890100260 0x7f5890101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5890105330 0x7f5890102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5890102770 0x7f58901a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58901a6080 con 0x7f5890102770 2024-02-17T12:56:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58901a61f0 con 0x7f5890105330 2024-02-17T12:56:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f58983d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58901a63b0 con 0x7f5890100260 2024-02-17T12:56:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f5896974640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5890102770 0x7f58901a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f5896974640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5890102770 0x7f58901a5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41574/0 (socket says 172.21.15.18:41574) 2024-02-17T12:56:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.218+0000 7f5896974640 1 -- 172.21.15.18:0/864297175 learned_addr learned my addr 172.21.15.18:0/864297175 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f5896173640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5890100260 0x7f5890101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f5895972640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5890105330 0x7f5890102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f5896974640 1 -- 172.21.15.18:0/864297175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5890100260 msgr2=0x7f5890101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f5896974640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5890100260 0x7f5890101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f5896974640 1 -- 172.21.15.18:0/864297175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5890105330 msgr2=0x7f5890102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f5896974640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5890105330 0x7f5890102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f5896974640 1 -- 172.21.15.18:0/864297175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5888009560 con 0x7f5890102770 2024-02-17T12:56:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f5896974640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5890102770 0x7f58901a5ab0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f588c00eaa0 tx=0x7f588c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f58877fe640 1 -- 172.21.15.18:0/864297175 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c00cdb0 con 0x7f5890102770 2024-02-17T12:56:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f58983d7640 1 -- 172.21.15.18:0/864297175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58901a6690 con 0x7f5890102770 2024-02-17T12:56:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f58877fe640 1 -- 172.21.15.18:0/864297175 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f588c005020 con 0x7f5890102770 2024-02-17T12:56:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f58877fe640 1 -- 172.21.15.18:0/864297175 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c0116b0 con 0x7f5890102770 2024-02-17T12:56:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.219+0000 7f58983d7640 1 -- 172.21.15.18:0/864297175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58901a6b50 con 0x7f5890102770 2024-02-17T12:56:47.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.221+0000 7f58877fe640 1 -- 172.21.15.18:0/864297175 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f588c005520 con 0x7f5890102770 2024-02-17T12:56:47.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f589afae740 1 Processor -- start 2024-02-17T12:56:47.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f589afae740 1 -- start start 2024-02-17T12:56:47.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f589afae740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512250 0x55dbc2512630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f589afae740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc251b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f589afae740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 0x55dbc251db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f589afae740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbc24cd960 con 0x55dbc251b750 2024-02-17T12:56:47.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f589afae740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbc2481310 con 0x55dbc2512b70 2024-02-17T12:56:47.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f589afae740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbc247e840 con 0x55dbc2512250 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f58877fe640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f587c0a2260 0x7f587c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f5895972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 0x55dbc251db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f5895972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 0x55dbc251db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41582/0 (socket says 172.21.15.18:41582) 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f5895972640 1 -- 172.21.15.18:0/4193590797 learned_addr learned my addr 172.21.15.18:0/4193590797 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f5896974640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f587c0a2260 0x7f587c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f5896173640 1 --2- 172.21.15.18:0/4193590797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc251b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f58877fe640 1 -- 172.21.15.18:0/864297175 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f588c0ce320 con 0x7f5890102770 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.222+0000 7f5895972640 1 --2- 172.21.15.18:0/4193590797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512250 0x55dbc2512630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.223+0000 7f5895972640 1 -- 172.21.15.18:0/4193590797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512250 msgr2=0x55dbc2512630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.223+0000 7f5895972640 1 --2- 172.21.15.18:0/4193590797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512250 0x55dbc2512630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.223+0000 7f5895972640 1 -- 172.21.15.18:0/4193590797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 msgr2=0x55dbc251b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.223+0000 7f5895972640 1 --2- 172.21.15.18:0/4193590797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc251b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.223+0000 7f5895972640 1 -- 172.21.15.18:0/4193590797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbc243c060 con 0x55dbc251b750 2024-02-17T12:56:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.223+0000 7f5896173640 1 --2- 172.21.15.18:0/4193590797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc251b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.223+0000 7f5896974640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f587c0a2260 0x7f587c0a4720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f588c097130 tx=0x7f588c084040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.223+0000 7f5895972640 1 --2- 172.21.15.18:0/4193590797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 0x55dbc251db40 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f588000ad50 tx=0x7f588000a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6582669b125ced0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.223+0000 7f5884bfe640 1 -- 172.21.15.18:0/4193590797 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588000fdf0 con 0x55dbc251b750 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.223+0000 7f5884bfe640 1 -- 172.21.15.18:0/4193590797 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f588000c070 con 0x55dbc251b750 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.223+0000 7f5884bfe640 1 -- 172.21.15.18:0/4193590797 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58800146e0 con 0x55dbc251b750 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 -- 172.21.15.18:0/4193590797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 msgr2=0x55dbc251db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 --2- 172.21.15.18:0/4193590797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 0x55dbc251db40 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f588000ad50 tx=0x7f588000a470 comp rx=0 tx=0).stop 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 -- 172.21.15.18:0/4193590797 shutdown_connections 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 --2- 172.21.15.18:0/4193590797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 0x55dbc251db40 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 --2- 172.21.15.18:0/4193590797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc251b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 --2- 172.21.15.18:0/4193590797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512250 0x55dbc2512630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 -- 172.21.15.18:0/4193590797 >> 172.21.15.18:0/4193590797 conn(0x55dbc250c980 msgr2=0x55dbc250cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 -- 172.21.15.18:0/4193590797 shutdown_connections 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 -- 172.21.15.18:0/4193590797 wait complete. 2024-02-17T12:56:47.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 -- start start 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 Processor -- start 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 0x55dbc2518810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc2518d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc251b750 0x55dbc2519290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbc247a020 con 0x55dbc2512250 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbc214d430 con 0x55dbc2512b70 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbc2109e00 con 0x55dbc251b750 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.224+0000 7f589afae740 10 client.0 _ll_register_callbacks cb 0x55dbc2421cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5895972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 0x55dbc2518810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5895972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 0x55dbc2518810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41588/0 (socket says 172.21.15.18:41588) 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5895972640 1 -- 172.21.15.18:0/1339424198 learned_addr learned my addr 172.21.15.18:0/1339424198 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5896173640 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc2518d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5895972640 1 -- 172.21.15.18:0/1339424198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc251b750 msgr2=0x55dbc2519290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5895972640 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc251b750 0x55dbc2519290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5895972640 1 -- 172.21.15.18:0/1339424198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 msgr2=0x55dbc2518d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5895972640 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc2518d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5895972640 1 -- 172.21.15.18:0/1339424198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588000e040 con 0x55dbc2512250 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5896173640 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc2518d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5895972640 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 0x55dbc2518810 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5880004810 tx=0x7f588000a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94756786fe32ec server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5879ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5879ffb640 1 -- 172.21.15.18:0/1339424198 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880017870 con 0x55dbc2512250 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5879ffb640 1 -- 172.21.15.18:0/1339424198 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5880000e40 con 0x55dbc2512250 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f589afae740 1 -- 172.21.15.18:0/1339424198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbc21797e0 con 0x55dbc2512250 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f5879ffb640 1 -- 172.21.15.18:0/1339424198 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5880017ca0 con 0x55dbc2512250 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f589afae740 10 client.17988 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.225+0000 7f589afae740 1 -- 172.21.15.18:0/1339424198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55dbc24786a0 con 0x55dbc2512250 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.226+0000 7f5879ffb640 1 -- 172.21.15.18:0/1339424198 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dbc21797e0 con 0x55dbc2512250 2024-02-17T12:56:47.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.226+0000 7f5879ffb640 1 -- 172.21.15.18:0/1339424198 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5880014460 con 0x55dbc2512250 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.226+0000 7f589afae740 10 client.17988 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.226+0000 7f589afae740 10 client.17988 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.226+0000 7f589afae740 20 client.17988 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.226+0000 7f589afae740 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55dbc252ba80 0x55dbc252de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.226+0000 7f589afae740 4 client.17988 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.226+0000 7f589afae740 1 -- 172.21.15.18:0/1339424198 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbc21b6720 con 0x55dbc252ba80 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.226+0000 7f5896173640 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55dbc252ba80 0x55dbc252de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.226+0000 7f5896173640 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55dbc252ba80 0x55dbc252de70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5abb793e293d6b3c server_cookie=757677e9939ce6ba in_seq=0 out_seq=0 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.226+0000 7f5879ffb640 10 client.17988 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f5879ffb640 1 -- 172.21.15.18:0/1339424198 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dbc21b6720 con 0x55dbc252ba80 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f5879ffb640 10 client.17988 handle_command_reply: tid=0 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f589afae740 1 client.17988 shutdown 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f589afae740 1 -- 172.21.15.18:0/1339424198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 msgr2=0x55dbc2518810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f589afae740 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 0x55dbc2518810 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5880004810 tx=0x7f588000a6d0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f589afae740 1 -- 172.21.15.18:0/1339424198 shutdown_connections 2024-02-17T12:56:47.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f589afae740 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc251b750 0x55dbc2519290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f589afae740 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55dbc252ba80 0x55dbc252de70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f589afae740 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc2518d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f5879ffb640 0 client.17988 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:56:47.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f589afae740 1 --2- 172.21.15.18:0/1339424198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 0x55dbc2518810 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f589afae740 1 -- 172.21.15.18:0/1339424198 >> 172.21.15.18:0/1339424198 conn(0x55dbc250c980 msgr2=0x55dbc250e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f589afae740 1 -- 172.21.15.18:0/1339424198 shutdown_connections 2024-02-17T12:56:47.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f589afae740 1 -- 172.21.15.18:0/1339424198 wait complete. 2024-02-17T12:56:47.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.227+0000 7f589afae740 20 client.17988 trim_cache size 0 max 16384 2024-02-17T12:56:47.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.239+0000 7f589afae740 1 Processor -- start 2024-02-17T12:56:47.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.239+0000 7f589afae740 1 -- start start 2024-02-17T12:56:47.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.239+0000 7f589afae740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512250 0x55dbc251af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.239+0000 7f589afae740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc25170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.239+0000 7f589afae740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 0x55dbc2517620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.239+0000 7f589afae740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbc23ff130 con 0x55dbc251b750 2024-02-17T12:56:47.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.239+0000 7f589afae740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbc252b880 con 0x55dbc2512b70 2024-02-17T12:56:47.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.239+0000 7f589afae740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbc2511eb0 con 0x55dbc2512250 2024-02-17T12:56:47.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.239+0000 7f5895972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 0x55dbc2517620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.239+0000 7f5895972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 0x55dbc2517620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41602/0 (socket says 172.21.15.18:41602) 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.239+0000 7f5895972640 1 -- 172.21.15.18:0/754913839 learned_addr learned my addr 172.21.15.18:0/754913839 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f5895972640 1 --2- 172.21.15.18:0/754913839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512250 0x55dbc251af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f5895972640 1 -- 172.21.15.18:0/754913839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512250 msgr2=0x55dbc251af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f5895972640 1 --2- 172.21.15.18:0/754913839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512250 0x55dbc251af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f5896173640 1 --2- 172.21.15.18:0/754913839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc25170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f5895972640 1 -- 172.21.15.18:0/754913839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 msgr2=0x55dbc25170e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f5895972640 1 --2- 172.21.15.18:0/754913839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc25170e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f5895972640 1 -- 172.21.15.18:0/754913839 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbc2414c20 con 0x55dbc251b750 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f5895972640 1 --2- 172.21.15.18:0/754913839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 0x55dbc2517620 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f588000a9a0 tx=0x7f588000a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a3d27e0c120bb7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f587bfff640 1 -- 172.21.15.18:0/754913839 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880000e40 con 0x55dbc251b750 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f587bfff640 1 -- 172.21.15.18:0/754913839 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f588000aba0 con 0x55dbc251b750 2024-02-17T12:56:47.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f587bfff640 1 -- 172.21.15.18:0/754913839 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880017960 con 0x55dbc251b750 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f589afae740 1 -- 172.21.15.18:0/754913839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 msgr2=0x55dbc2517620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.240+0000 7f589afae740 1 --2- 172.21.15.18:0/754913839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 0x55dbc2517620 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f588000a9a0 tx=0x7f588000a8f0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 -- 172.21.15.18:0/754913839 shutdown_connections 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 --2- 172.21.15.18:0/754913839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc251b750 0x55dbc2517620 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 --2- 172.21.15.18:0/754913839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc2512b70 0x55dbc25170e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 --2- 172.21.15.18:0/754913839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512250 0x55dbc251af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 -- 172.21.15.18:0/754913839 >> 172.21.15.18:0/754913839 conn(0x55dbc250c980 msgr2=0x55dbc2513ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 -- 172.21.15.18:0/754913839 shutdown_connections 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 -- 172.21.15.18:0/754913839 wait complete. 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 -- start start 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 Processor -- start 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 0x55dbc2522660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512b70 0x55dbc2520a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc251b750 0x55dbc2520f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbc2522f30 con 0x55dbc2512250 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbc2521480 con 0x55dbc251b750 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbc25215f0 con 0x55dbc2512b70 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f589afae740 10 client.0 _ll_register_callbacks cb 0x7f588000a310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f5895972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 0x55dbc2522660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f5895972640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 0x55dbc2522660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41604/0 (socket says 172.21.15.18:41604) 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f5895972640 1 -- 172.21.15.18:0/553722626 learned_addr learned my addr 172.21.15.18:0/553722626 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f5896173640 1 --2- 172.21.15.18:0/553722626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512b70 0x55dbc2520a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f5895972640 1 -- 172.21.15.18:0/553722626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512b70 msgr2=0x55dbc2520a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f5895972640 1 --2- 172.21.15.18:0/553722626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512b70 0x55dbc2520a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f5895972640 1 -- 172.21.15.18:0/553722626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc251b750 msgr2=0x55dbc2520f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.241+0000 7f5895972640 1 --2- 172.21.15.18:0/553722626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc251b750 0x55dbc2520f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f5895972640 1 -- 172.21.15.18:0/553722626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbc24786a0 con 0x55dbc2512250 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f5895972640 1 --2- 172.21.15.18:0/553722626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 0x55dbc2522660 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f5880006f40 tx=0x7f58800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9648939d9486a9b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f5879ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:47.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f5879ffb640 1 -- 172.21.15.18:0/553722626 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880014600 con 0x55dbc2512250 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f5879ffb640 1 -- 172.21.15.18:0/553722626 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5880000e40 con 0x55dbc2512250 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f589afae740 1 -- 172.21.15.18:0/553722626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f588000c070 con 0x55dbc2512250 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f5879ffb640 1 -- 172.21.15.18:0/553722626 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5880022b30 con 0x55dbc2512250 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f589afae740 10 client.17994 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f589afae740 1 -- 172.21.15.18:0/553722626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55dbc243a8c0 con 0x55dbc2512250 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f5879ffb640 1 -- 172.21.15.18:0/553722626 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f588000c070 con 0x55dbc2512250 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f5879ffb640 1 -- 172.21.15.18:0/553722626 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5880017700 con 0x55dbc2512250 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f589afae740 10 client.17994 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f589afae740 10 client.17994 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f589afae740 20 client.17994 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f589afae740 1 --2- 172.21.15.18:0/553722626 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55dbc2582e50 0x55dbc2585240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f589afae740 4 client.17994 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.242+0000 7f589afae740 1 -- 172.21.15.18:0/553722626 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dbc2205890 con 0x55dbc2582e50 2024-02-17T12:56:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f5896173640 1 --2- 172.21.15.18:0/553722626 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55dbc2582e50 0x55dbc2585240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f5896173640 1 --2- 172.21.15.18:0/553722626 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55dbc2582e50 0x55dbc2585240 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=400b6b285b23b9da server_cookie=38a3294cdd4d14a7 in_seq=0 out_seq=0 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f5879ffb640 10 client.17994 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f5879ffb640 1 -- 172.21.15.18:0/553722626 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dbc2205890 con 0x55dbc2582e50 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f5879ffb640 10 client.17994 handle_command_reply: tid=0 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f589afae740 1 client.17994 shutdown 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f589afae740 1 -- 172.21.15.18:0/553722626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 msgr2=0x55dbc2522660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f589afae740 1 --2- 172.21.15.18:0/553722626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 0x55dbc2522660 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f5880006f40 tx=0x7f58800046b0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f589afae740 1 -- 172.21.15.18:0/553722626 shutdown_connections 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f589afae740 1 --2- 172.21.15.18:0/553722626 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55dbc2582e50 0x55dbc2585240 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f589afae740 1 --2- 172.21.15.18:0/553722626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dbc251b750 0x55dbc2520f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f589afae740 1 --2- 172.21.15.18:0/553722626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dbc2512b70 0x55dbc2520a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f589afae740 1 --2- 172.21.15.18:0/553722626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbc2512250 0x55dbc2522660 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f5879ffb640 0 client.17994 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f589afae740 1 -- 172.21.15.18:0/553722626 >> 172.21.15.18:0/553722626 conn(0x55dbc250c980 msgr2=0x55dbc2517010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f589afae740 1 -- 172.21.15.18:0/553722626 shutdown_connections 2024-02-17T12:56:47.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.243+0000 7f589afae740 1 -- 172.21.15.18:0/553722626 wait complete. 2024-02-17T12:56:47.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f589afae740 20 client.17994 trim_cache size 0 max 16384 2024-02-17T12:56:47.241 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:56:47.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f58983d7640 1 -- 172.21.15.18:0/864297175 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f587c0a2260 msgr2=0x7f587c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f58983d7640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f587c0a2260 0x7f587c0a4720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f588c097130 tx=0x7f588c084040 comp rx=0 tx=0).stop 2024-02-17T12:56:47.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f58983d7640 1 -- 172.21.15.18:0/864297175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5890102770 msgr2=0x7f58901a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f58983d7640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5890102770 0x7f58901a5ab0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f588c00eaa0 tx=0x7f588c00ef60 comp rx=0 tx=0).stop 2024-02-17T12:56:47.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f58983d7640 1 -- 172.21.15.18:0/864297175 shutdown_connections 2024-02-17T12:56:47.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f58983d7640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f587c0a2260 0x7f587c0a4720 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f58983d7640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5890102770 0x7f58901a5ab0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f58983d7640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5890105330 0x7f5890102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f58983d7640 1 --2- 172.21.15.18:0/864297175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5890100260 0x7f5890101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f58983d7640 1 -- 172.21.15.18:0/864297175 >> 172.21.15.18:0/864297175 conn(0x7f5890077180 msgr2=0x7f5890104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f58983d7640 1 -- 172.21.15.18:0/864297175 shutdown_connections 2024-02-17T12:56:47.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.244+0000 7f58983d7640 1 -- 172.21.15.18:0/864297175 wait complete. 2024-02-17T12:56:47.245 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:56:47.245 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:56:47.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 363483 5.74 MB/sec execute 242 sec latency 2472.330 ms 2024-02-17T12:56:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.359+0000 7f7505864640 1 -- 172.21.15.18:0/552798099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75001002a0 msgr2=0x7f7500105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.359+0000 7f7505864640 1 --2- 172.21.15.18:0/552798099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75001002a0 0x7f7500105dd0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009930 tx=0x7f74f402bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.359+0000 7f7505864640 1 -- 172.21.15.18:0/552798099 shutdown_connections 2024-02-17T12:56:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.359+0000 7f7505864640 1 --2- 172.21.15.18:0/552798099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7500106500 0x7f750010e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.359+0000 7f7505864640 1 --2- 172.21.15.18:0/552798099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75001002a0 0x7f7500105dd0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.359+0000 7f7505864640 1 --2- 172.21.15.18:0/552798099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75000ff8f0 0x7f75000ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.359+0000 7f7505864640 1 -- 172.21.15.18:0/552798099 >> 172.21.15.18:0/552798099 conn(0x7f7500076640 msgr2=0x7f75000fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.359+0000 7f7505864640 1 -- 172.21.15.18:0/552798099 shutdown_connections 2024-02-17T12:56:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.359+0000 7f7505864640 1 -- 172.21.15.18:0/552798099 wait complete. 2024-02-17T12:56:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.359+0000 7f7505864640 1 Processor -- start 2024-02-17T12:56:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f7505864640 1 -- start start 2024-02-17T12:56:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f7505864640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75000ff8f0 0x7f750019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f7505864640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7500106500 0x7f750019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f7505864640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7500197690 0x7f7500197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f7505864640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7500198160 con 0x7f7500197690 2024-02-17T12:56:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f7505864640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75001982d0 con 0x7f7500106500 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f7505864640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7500198490 con 0x7f75000ff8f0 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f74ff7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7500197690 0x7f7500197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f74ff7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7500197690 0x7f7500197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41618/0 (socket says 172.21.15.18:41618) 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f74ff7fe640 1 -- 172.21.15.18:0/317062332 learned_addr learned my addr 172.21.15.18:0/317062332 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f74feffd640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75000ff8f0 0x7f750019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7500106500 0x7f750019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f74ff7fe640 1 -- 172.21.15.18:0/317062332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75000ff8f0 msgr2=0x7f750019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f74ff7fe640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75000ff8f0 0x7f750019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f74ff7fe640 1 -- 172.21.15.18:0/317062332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7500106500 msgr2=0x7f750019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f74ff7fe640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7500106500 0x7f750019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f74ff7fe640 1 -- 172.21.15.18:0/317062332 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f4009560 con 0x7f7500197690 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f74ff7fe640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7500197690 0x7f7500197b10 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f74f800eb10 tx=0x7f74f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.360+0000 7f7504862640 1 -- 172.21.15.18:0/317062332 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8004b20 con 0x7f7500197690 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.361+0000 7f7504862640 1 -- 172.21.15.18:0/317062332 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74f8004c80 con 0x7f7500197690 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.361+0000 7f7505864640 1 -- 172.21.15.18:0/317062332 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75001a4430 con 0x7f7500197690 2024-02-17T12:56:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.361+0000 7f7504862640 1 -- 172.21.15.18:0/317062332 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8010650 con 0x7f7500197690 2024-02-17T12:56:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.361+0000 7f7505864640 1 -- 172.21.15.18:0/317062332 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75001a4960 con 0x7f7500197690 2024-02-17T12:56:47.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f7504862640 1 -- 172.21.15.18:0/317062332 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f74f8005020 con 0x7f7500197690 2024-02-17T12:56:47.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f7508437740 1 Processor -- start 2024-02-17T12:56:47.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f7508437740 1 -- start start 2024-02-17T12:56:47.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f7508437740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91f250 0x558daf91f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f7508437740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf928210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f7508437740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 0x558daf92ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f7508437740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daf8da960 con 0x558daf928750 2024-02-17T12:56:47.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f7508437740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daf88e310 con 0x558daf91f250 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f7508437740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daf88b840 con 0x558daf91fb70 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f7504862640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74e80a22b0 0x7f74e80a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f74fe7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 0x558daf92ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f74fe7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 0x558daf92ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41626/0 (socket says 172.21.15.18:41626) 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f74fe7fc640 1 -- 172.21.15.18:0/1901105088 learned_addr learned my addr 172.21.15.18:0/1901105088 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/1901105088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91f250 0x558daf91f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.363+0000 7f74ff7fe640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74e80a22b0 0x7f74e80a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f74feffd640 1 --2- 172.21.15.18:0/1901105088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf928210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f7504862640 1 -- 172.21.15.18:0/317062332 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f74f8014070 con 0x7f7500197690 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f74fe7fc640 1 -- 172.21.15.18:0/1901105088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 msgr2=0x558daf928210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/1901105088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf928210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f74fe7fc640 1 -- 172.21.15.18:0/1901105088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91f250 msgr2=0x558daf91f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/1901105088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91f250 0x558daf91f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f74fe7fc640 1 -- 172.21.15.18:0/1901105088 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558daf849060 con 0x558daf928750 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f74feffd640 1 --2- 172.21.15.18:0/1901105088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf928210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f74ff7fe640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74e80a22b0 0x7f74e80a4770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f74f80cc250 tx=0x7f74f800cee0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/1901105088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 0x558daf92ab40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f75000690e0 tx=0x7f74f4005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74085277ccd83ac server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f74e5ffb640 1 -- 172.21.15.18:0/1901105088 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f402c890 con 0x558daf928750 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f74e5ffb640 1 -- 172.21.15.18:0/1901105088 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74f4004510 con 0x558daf928750 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.364+0000 7f74e5ffb640 1 -- 172.21.15.18:0/1901105088 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4003bf0 con 0x558daf928750 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.365+0000 7f7508437740 1 -- 172.21.15.18:0/1901105088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 msgr2=0x558daf92ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.365+0000 7f7508437740 1 --2- 172.21.15.18:0/1901105088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 0x558daf92ab40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f75000690e0 tx=0x7f74f4005d70 comp rx=0 tx=0).stop 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.365+0000 7f7508437740 1 -- 172.21.15.18:0/1901105088 shutdown_connections 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.365+0000 7f7508437740 1 --2- 172.21.15.18:0/1901105088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 0x558daf92ab40 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.365+0000 7f7508437740 1 --2- 172.21.15.18:0/1901105088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf928210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.365+0000 7f7508437740 1 --2- 172.21.15.18:0/1901105088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91f250 0x558daf91f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.365+0000 7f7508437740 1 -- 172.21.15.18:0/1901105088 >> 172.21.15.18:0/1901105088 conn(0x558daf919980 msgr2=0x558daf919d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.365+0000 7f7508437740 1 -- 172.21.15.18:0/1901105088 shutdown_connections 2024-02-17T12:56:47.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.365+0000 7f7508437740 1 -- 172.21.15.18:0/1901105088 wait complete. 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.365+0000 7f7508437740 1 -- start start 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.365+0000 7f7508437740 1 Processor -- start 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f7508437740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 0x558daf925800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f7508437740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91fb70 0x558daf925d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f7508437740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf928750 0x558daf926280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f7508437740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daf887020 con 0x558daf91f250 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f7508437740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daf55a430 con 0x558daf91fb70 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f7508437740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daf516e00 con 0x558daf928750 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f7508437740 10 client.0 _ll_register_callbacks cb 0x558daf82ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74fe7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 0x558daf925800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74fe7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 0x558daf925800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41640/0 (socket says 172.21.15.18:41640) 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74fe7fc640 1 -- 172.21.15.18:0/3697940674 learned_addr learned my addr 172.21.15.18:0/3697940674 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74feffd640 1 --2- 172.21.15.18:0/3697940674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91fb70 0x558daf925d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74fe7fc640 1 -- 172.21.15.18:0/3697940674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf928750 msgr2=0x558daf926280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/3697940674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf928750 0x558daf926280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74fe7fc640 1 -- 172.21.15.18:0/3697940674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91fb70 msgr2=0x558daf925d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/3697940674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91fb70 0x558daf925d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74fe7fc640 1 -- 172.21.15.18:0/3697940674 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750010e890 con 0x558daf91f250 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/3697940674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 0x558daf925800 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f74f4036040 tx=0x7f74f402c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d01b7e7c39d4da8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74deffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74deffd640 1 -- 172.21.15.18:0/3697940674 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4004ff0 con 0x558daf91f250 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74deffd640 1 -- 172.21.15.18:0/3697940674 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f74f4000e40 con 0x558daf91f250 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f7508437740 1 -- 172.21.15.18:0/3697940674 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558daf5867e0 con 0x558daf91f250 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.366+0000 7f74deffd640 1 -- 172.21.15.18:0/3697940674 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74f4005150 con 0x558daf91f250 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.367+0000 7f7508437740 10 client.18006 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.367+0000 7f7508437740 1 -- 172.21.15.18:0/3697940674 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558daf8856a0 con 0x558daf91f250 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.367+0000 7f74deffd640 1 -- 172.21.15.18:0/3697940674 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558daf5867e0 con 0x558daf91f250 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.367+0000 7f74deffd640 1 -- 172.21.15.18:0/3697940674 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f74f404b070 con 0x558daf91f250 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.367+0000 7f7508437740 10 client.18006 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.367+0000 7f7508437740 10 client.18006 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:56:47.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.367+0000 7f7508437740 20 client.18006 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.367+0000 7f7508437740 1 --2- 172.21.15.18:0/3697940674 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558daf9387f0 0x558daf93ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.367+0000 7f7508437740 4 client.18006 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.367+0000 7f7508437740 1 -- 172.21.15.18:0/3697940674 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558daf5c3720 con 0x558daf9387f0 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.367+0000 7f74feffd640 1 --2- 172.21.15.18:0/3697940674 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558daf9387f0 0x558daf93ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f74feffd640 1 --2- 172.21.15.18:0/3697940674 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558daf9387f0 0x558daf93ac10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=548a3f7f2e75f6ac server_cookie=973c9be3bb4d2295 in_seq=0 out_seq=0 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f74deffd640 10 client.18006 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f74deffd640 1 -- 172.21.15.18:0/3697940674 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558daf5c3720 con 0x558daf9387f0 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f74deffd640 10 client.18006 handle_command_reply: tid=0 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f7508437740 1 client.18006 shutdown 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f7508437740 1 -- 172.21.15.18:0/3697940674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 msgr2=0x558daf925800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f7508437740 1 --2- 172.21.15.18:0/3697940674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 0x558daf925800 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f74f4036040 tx=0x7f74f402c3b0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f7508437740 1 -- 172.21.15.18:0/3697940674 shutdown_connections 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f7508437740 1 --2- 172.21.15.18:0/3697940674 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558daf9387f0 0x558daf93ac10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f7508437740 1 --2- 172.21.15.18:0/3697940674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf928750 0x558daf926280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f7508437740 1 --2- 172.21.15.18:0/3697940674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91fb70 0x558daf925d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f74deffd640 0 client.18006 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:56:47.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f7508437740 1 --2- 172.21.15.18:0/3697940674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 0x558daf925800 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.368+0000 7f7508437740 1 -- 172.21.15.18:0/3697940674 >> 172.21.15.18:0/3697940674 conn(0x558daf919980 msgr2=0x558daf91b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.369+0000 7f7508437740 1 -- 172.21.15.18:0/3697940674 shutdown_connections 2024-02-17T12:56:47.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.369+0000 7f7508437740 1 -- 172.21.15.18:0/3697940674 wait complete. 2024-02-17T12:56:47.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.369+0000 7f7508437740 20 client.18006 trim_cache size 0 max 16384 2024-02-17T12:56:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.380+0000 7f7508437740 1 Processor -- start 2024-02-17T12:56:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.380+0000 7f7508437740 1 -- start start 2024-02-17T12:56:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.380+0000 7f7508437740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 0x558daf927f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.380+0000 7f7508437740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf922a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.380+0000 7f7508437740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf928750 0x558daf922fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.380+0000 7f7508437740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daf80c130 con 0x558daf91f250 2024-02-17T12:56:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.380+0000 7f7508437740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daf91e650 con 0x558daf928750 2024-02-17T12:56:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.380+0000 7f7508437740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daf91eeb0 con 0x558daf91fb70 2024-02-17T12:56:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74fe7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 0x558daf927f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74fe7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 0x558daf927f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41644/0 (socket says 172.21.15.18:41644) 2024-02-17T12:56:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74fe7fc640 1 -- 172.21.15.18:0/2890949314 learned_addr learned my addr 172.21.15.18:0/2890949314 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74feffd640 1 --2- 172.21.15.18:0/2890949314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf922a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74fe7fc640 1 -- 172.21.15.18:0/2890949314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 msgr2=0x558daf922a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/2890949314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf922a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74fe7fc640 1 -- 172.21.15.18:0/2890949314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf928750 msgr2=0x558daf922fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/2890949314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf928750 0x558daf922fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74fe7fc640 1 -- 172.21.15.18:0/2890949314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558daf821c20 con 0x558daf91f250 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74feffd640 1 --2- 172.21.15.18:0/2890949314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf922a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/2890949314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 0x558daf927f30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f74f402c780 tx=0x7f74f4005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646b5c0d772533a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74e57fa640 1 -- 172.21.15.18:0/2890949314 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f402ccd0 con 0x558daf91f250 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74e57fa640 1 -- 172.21.15.18:0/2890949314 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74f4003730 con 0x558daf91f250 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.381+0000 7f74e57fa640 1 -- 172.21.15.18:0/2890949314 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4003f70 con 0x558daf91f250 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 -- 172.21.15.18:0/2890949314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 msgr2=0x558daf927f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 --2- 172.21.15.18:0/2890949314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 0x558daf927f30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f74f402c780 tx=0x7f74f4005e30 comp rx=0 tx=0).stop 2024-02-17T12:56:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 -- 172.21.15.18:0/2890949314 shutdown_connections 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 --2- 172.21.15.18:0/2890949314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf928750 0x558daf922fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 --2- 172.21.15.18:0/2890949314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf922a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 --2- 172.21.15.18:0/2890949314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf91f250 0x558daf927f30 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 -- 172.21.15.18:0/2890949314 >> 172.21.15.18:0/2890949314 conn(0x558daf919980 msgr2=0x558daf9240d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 -- 172.21.15.18:0/2890949314 shutdown_connections 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 -- 172.21.15.18:0/2890949314 wait complete. 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 -- start start 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 Processor -- start 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91f250 0x558daf92fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf92de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 0x558daf92e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.382+0000 7f7508437740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daf92ebc0 con 0x558daf928750 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f7508437740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daf9930d0 con 0x558daf91f250 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f7508437740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daf993210 con 0x558daf91fb70 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f7508437740 10 client.0 _ll_register_callbacks cb 0x7f74f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74fe7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 0x558daf92e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74fe7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 0x558daf92e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41648/0 (socket says 172.21.15.18:41648) 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74fe7fc640 1 -- 172.21.15.18:0/3361333818 learned_addr learned my addr 172.21.15.18:0/3361333818 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91f250 0x558daf92fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74feffd640 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf92de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74fe7fc640 1 -- 172.21.15.18:0/3361333818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 msgr2=0x558daf92de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf92de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74fe7fc640 1 -- 172.21.15.18:0/3361333818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91f250 msgr2=0x558daf92fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91f250 0x558daf92fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74fe7fc640 1 -- 172.21.15.18:0/3361333818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558daf8856a0 con 0x558daf928750 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74fe7fc640 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 0x558daf92e4c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f74f402ce20 tx=0x7f74f4046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c566f2e9be130ef8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74deffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74deffd640 1 -- 172.21.15.18:0/3361333818 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4004360 con 0x558daf928750 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74deffd640 1 -- 172.21.15.18:0/3361333818 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f74f4000e40 con 0x558daf928750 2024-02-17T12:56:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f7508437740 1 -- 172.21.15.18:0/3361333818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74f4004510 con 0x558daf928750 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f74deffd640 1 -- 172.21.15.18:0/3361333818 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74f4043480 con 0x558daf928750 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f7508437740 10 client.18012 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.383+0000 7f7508437740 1 -- 172.21.15.18:0/3361333818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558daf8478c0 con 0x558daf928750 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.384+0000 7f74deffd640 1 -- 172.21.15.18:0/3361333818 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74f4004510 con 0x558daf928750 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.384+0000 7f74deffd640 1 -- 172.21.15.18:0/3361333818 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f74f406c040 con 0x558daf928750 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.384+0000 7f7508437740 10 client.18012 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.384+0000 7f7508437740 10 client.18012 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.384+0000 7f7508437740 20 client.18012 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.384+0000 7f7508437740 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558daf9944e0 0x558daf9968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.384+0000 7f7508437740 4 client.18012 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.384+0000 7f7508437740 1 -- 172.21.15.18:0/3361333818 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558daf612890 con 0x558daf9944e0 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.384+0000 7f74feffd640 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558daf9944e0 0x558daf9968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f74feffd640 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558daf9944e0 0x558daf9968d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=814e5632f6c31246 server_cookie=861c18e424de3303 in_seq=0 out_seq=0 2024-02-17T12:56:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f74deffd640 10 client.18012 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f74deffd640 1 -- 172.21.15.18:0/3361333818 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558daf612890 con 0x558daf9944e0 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f74deffd640 10 client.18012 handle_command_reply: tid=0 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7508437740 1 client.18012 shutdown 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7508437740 1 -- 172.21.15.18:0/3361333818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 msgr2=0x558daf92e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7508437740 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 0x558daf92e4c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f74f402ce20 tx=0x7f74f4046040 comp rx=0 tx=0).stop 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7508437740 1 -- 172.21.15.18:0/3361333818 shutdown_connections 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7508437740 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558daf9944e0 0x558daf9968d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7508437740 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558daf928750 0x558daf92e4c0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f74deffd640 0 client.18012 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7508437740 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558daf91fb70 0x558daf92de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7508437740 1 --2- 172.21.15.18:0/3361333818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558daf91f250 0x558daf92fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7508437740 1 -- 172.21.15.18:0/3361333818 >> 172.21.15.18:0/3361333818 conn(0x558daf919980 msgr2=0x558daf918b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7508437740 1 -- 172.21.15.18:0/3361333818 shutdown_connections 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7508437740 1 -- 172.21.15.18:0/3361333818 wait complete. 2024-02-17T12:56:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7508437740 20 client.18012 trim_cache size 0 max 16384 2024-02-17T12:56:47.382 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:56:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.385+0000 7f7505864640 1 -- 172.21.15.18:0/317062332 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74e80a22b0 msgr2=0x7f74e80a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.386+0000 7f7505864640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74e80a22b0 0x7f74e80a4770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f74f80cc250 tx=0x7f74f800cee0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.386+0000 7f7505864640 1 -- 172.21.15.18:0/317062332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7500197690 msgr2=0x7f7500197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.386+0000 7f7505864640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7500197690 0x7f7500197b10 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f74f800eb10 tx=0x7f74f800c2d0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.386+0000 7f7505864640 1 -- 172.21.15.18:0/317062332 shutdown_connections 2024-02-17T12:56:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.386+0000 7f7505864640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74e80a22b0 0x7f74e80a4770 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.386+0000 7f7505864640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7500197690 0x7f7500197b10 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.386+0000 7f7505864640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7500106500 0x7f750019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.386+0000 7f7505864640 1 --2- 172.21.15.18:0/317062332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75000ff8f0 0x7f750019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.386+0000 7f7505864640 1 -- 172.21.15.18:0/317062332 >> 172.21.15.18:0/317062332 conn(0x7f7500076640 msgr2=0x7f7500104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.386+0000 7f7505864640 1 -- 172.21.15.18:0/317062332 shutdown_connections 2024-02-17T12:56:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.386+0000 7f7505864640 1 -- 172.21.15.18:0/317062332 wait complete. 2024-02-17T12:56:47.387 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:56:47.387 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:56:47.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.541+0000 7fabb336b640 1 -- 172.21.15.18:0/658854430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabac101380 msgr2=0x7fabac101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.541+0000 7fabb336b640 1 --2- 172.21.15.18:0/658854430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabac101380 0x7fabac101760 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7faba8009930 tx=0x7faba802bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:47.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.541+0000 7fabb336b640 1 -- 172.21.15.18:0/658854430 shutdown_connections 2024-02-17T12:56:47.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.541+0000 7fabb336b640 1 --2- 172.21.15.18:0/658854430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac106450 0x7fabac10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.541+0000 7fabb336b640 1 --2- 172.21.15.18:0/658854430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabac101d30 0x7fabac105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.541+0000 7fabb336b640 1 --2- 172.21.15.18:0/658854430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabac101380 0x7fabac101760 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.541+0000 7fabb336b640 1 -- 172.21.15.18:0/658854430 >> 172.21.15.18:0/658854430 conn(0x7fabac078580 msgr2=0x7fabac078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.541+0000 7fabb336b640 1 -- 172.21.15.18:0/658854430 shutdown_connections 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.541+0000 7fabb336b640 1 -- 172.21.15.18:0/658854430 wait complete. 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb336b640 1 Processor -- start 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb336b640 1 -- start start 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb336b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabac101380 0x7fabac197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb336b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabac106450 0x7fabac197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb336b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac19bff0 0x7fabac19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb336b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac19c9e0 con 0x7fabac106450 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb336b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac19cb50 con 0x7fabac101380 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb336b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac19ccc0 con 0x7fabac19bff0 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb0906640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabac106450 0x7fabac197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb0906640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabac106450 0x7fabac197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41662/0 (socket says 172.21.15.18:41662) 2024-02-17T12:56:47.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb0906640 1 -- 172.21.15.18:0/2371186263 learned_addr learned my addr 172.21.15.18:0/2371186263 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb1107640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabac101380 0x7fabac197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb1908640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac19bff0 0x7fabac19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb0906640 1 -- 172.21.15.18:0/2371186263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac19bff0 msgr2=0x7fabac19c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb0906640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac19bff0 0x7fabac19c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb0906640 1 -- 172.21.15.18:0/2371186263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabac101380 msgr2=0x7fabac197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb0906640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabac101380 0x7fabac197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb0906640 1 -- 172.21.15.18:0/2371186263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8009560 con 0x7fabac106450 2024-02-17T12:56:47.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.542+0000 7fabb1107640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabac101380 0x7fabac197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:47.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.543+0000 7fabb0906640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabac106450 0x7fabac197c70 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00c970 tx=0x7fab9c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.543+0000 7faba27fc640 1 -- 172.21.15.18:0/2371186263 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c004440 con 0x7fabac106450 2024-02-17T12:56:47.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.543+0000 7faba27fc640 1 -- 172.21.15.18:0/2371186263 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab9c007d40 con 0x7fabac106450 2024-02-17T12:56:47.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.543+0000 7fabb336b640 1 -- 172.21.15.18:0/2371186263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac1a42a0 con 0x7fabac106450 2024-02-17T12:56:47.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.543+0000 7faba27fc640 1 -- 172.21.15.18:0/2371186263 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00f450 con 0x7fabac106450 2024-02-17T12:56:47.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.543+0000 7fabb336b640 1 -- 172.21.15.18:0/2371186263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac1a4750 con 0x7fabac106450 2024-02-17T12:56:47.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.545+0000 7faba27fc640 1 -- 172.21.15.18:0/2371186263 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fab9c007860 con 0x7fabac106450 2024-02-17T12:56:47.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.545+0000 7fabb600e740 1 Processor -- start 2024-02-17T12:56:47.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.545+0000 7fabb600e740 1 -- start start 2024-02-17T12:56:47.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.545+0000 7faba27fc640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab980a2230 0x7fab980a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.545+0000 7fabb600e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x560037657630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.545+0000 7fabb600e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657b70 0x560037660210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb600e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037660750 0x560037662b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb600e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560037612960 con 0x560037657250 2024-02-17T12:56:47.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb600e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600375c6310 con 0x560037657b70 2024-02-17T12:56:47.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb600e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600375c3840 con 0x560037660750 2024-02-17T12:56:47.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb1107640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x560037657630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb1107640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x560037657630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41666/0 (socket says 172.21.15.18:41666) 2024-02-17T12:56:47.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb1908640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab980a2230 0x7fab980a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb1107640 1 -- 172.21.15.18:0/1078910684 learned_addr learned my addr 172.21.15.18:0/1078910684 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7faba27fc640 1 -- 172.21.15.18:0/2371186263 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fab9c0ce2a0 con 0x7fabac106450 2024-02-17T12:56:47.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb0906640 1 --2- 172.21.15.18:0/1078910684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657b70 0x560037660210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb1107640 1 -- 172.21.15.18:0/1078910684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037660750 msgr2=0x560037662b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb1107640 1 --2- 172.21.15.18:0/1078910684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037660750 0x560037662b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb1107640 1 -- 172.21.15.18:0/1078910684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657b70 msgr2=0x560037660210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb1107640 1 --2- 172.21.15.18:0/1078910684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657b70 0x560037660210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb1107640 1 -- 172.21.15.18:0/1078910684 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560037581060 con 0x560037657250 2024-02-17T12:56:47.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb1908640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab980a2230 0x7fab980a46f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faba40092f0 tx=0x7faba4010740 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fabb1107640 1 --2- 172.21.15.18:0/1078910684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x560037657630 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7faba8009930 tx=0x7faba8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a24ba1419d8905 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fab977fe640 1 -- 172.21.15.18:0/1078910684 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba802c7e0 con 0x560037657250 2024-02-17T12:56:47.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fab977fe640 1 -- 172.21.15.18:0/1078910684 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faba8004510 con 0x560037657250 2024-02-17T12:56:47.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.546+0000 7fab977fe640 1 -- 172.21.15.18:0/1078910684 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80053c0 con 0x560037657250 2024-02-17T12:56:47.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.547+0000 7fabb600e740 1 -- 172.21.15.18:0/1078910684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 msgr2=0x560037657630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.547+0000 7fabb600e740 1 --2- 172.21.15.18:0/1078910684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x560037657630 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7faba8009930 tx=0x7faba8004740 comp rx=0 tx=0).stop 2024-02-17T12:56:47.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.547+0000 7fabb600e740 1 -- 172.21.15.18:0/1078910684 shutdown_connections 2024-02-17T12:56:47.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.547+0000 7fabb600e740 1 --2- 172.21.15.18:0/1078910684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037660750 0x560037662b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.547+0000 7fabb600e740 1 --2- 172.21.15.18:0/1078910684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657b70 0x560037660210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.547+0000 7fabb600e740 1 --2- 172.21.15.18:0/1078910684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x560037657630 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.547+0000 7fabb600e740 1 -- 172.21.15.18:0/1078910684 >> 172.21.15.18:0/1078910684 conn(0x560037651980 msgr2=0x560037651d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.547+0000 7fabb600e740 1 -- 172.21.15.18:0/1078910684 shutdown_connections 2024-02-17T12:56:47.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.547+0000 7fabb600e740 1 -- 172.21.15.18:0/1078910684 wait complete. 2024-02-17T12:56:47.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.547+0000 7fabb600e740 1 -- start start 2024-02-17T12:56:47.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.547+0000 7fabb600e740 1 Processor -- start 2024-02-17T12:56:47.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb600e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x56003765d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb600e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657b70 0x56003765dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb600e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037660750 0x56003765e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb600e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600375bf020 con 0x560037657250 2024-02-17T12:56:47.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb600e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560037292430 con 0x560037657b70 2024-02-17T12:56:47.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb600e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003724ee00 con 0x560037660750 2024-02-17T12:56:47.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb600e740 10 client.0 _ll_register_callbacks cb 0x560037566cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:47.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb1107640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x56003765d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb1107640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x56003765d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41674/0 (socket says 172.21.15.18:41674) 2024-02-17T12:56:47.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb1107640 1 -- 172.21.15.18:0/4146473471 learned_addr learned my addr 172.21.15.18:0/4146473471 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb0906640 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657b70 0x56003765dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb1908640 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037660750 0x56003765e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb1107640 1 -- 172.21.15.18:0/4146473471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037660750 msgr2=0x56003765e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb1107640 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037660750 0x56003765e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb1107640 1 -- 172.21.15.18:0/4146473471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657b70 msgr2=0x56003765dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb1107640 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657b70 0x56003765dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb1107640 1 -- 172.21.15.18:0/4146473471 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac10e830 con 0x560037657250 2024-02-17T12:56:47.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb1908640 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037660750 0x56003765e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:47.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fabb1107640 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x56003765d740 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7faba8005940 tx=0x7faba8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca7865e822f91dc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.548+0000 7fab94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:47.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fab94ff9640 1 -- 172.21.15.18:0/4146473471 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8005570 con 0x560037657250 2024-02-17T12:56:47.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fab94ff9640 1 -- 172.21.15.18:0/4146473471 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7faba802cbe0 con 0x560037657250 2024-02-17T12:56:47.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fabb600e740 1 -- 172.21.15.18:0/4146473471 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600372be7e0 con 0x560037657250 2024-02-17T12:56:47.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fab94ff9640 1 -- 172.21.15.18:0/4146473471 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7faba8034440 con 0x560037657250 2024-02-17T12:56:47.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fabb600e740 10 client.18024 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:47.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fabb600e740 1 -- 172.21.15.18:0/4146473471 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5600375bd6a0 con 0x560037657250 2024-02-17T12:56:47.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fab94ff9640 1 -- 172.21.15.18:0/4146473471 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5600372be7e0 con 0x560037657250 2024-02-17T12:56:47.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fab94ff9640 1 -- 172.21.15.18:0/4146473471 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7faba803c040 con 0x560037657250 2024-02-17T12:56:47.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fabb600e740 10 client.18024 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:47.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fabb600e740 10 client.18024 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:56:47.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fabb600e740 20 client.18024 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:47.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fabb600e740 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560037670710 0x560037672b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fabb600e740 4 client.18024 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:47.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.549+0000 7fabb600e740 1 -- 172.21.15.18:0/4146473471 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600372fb720 con 0x560037670710 2024-02-17T12:56:47.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.550+0000 7fabb0906640 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560037670710 0x560037672b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.550+0000 7fabb0906640 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560037670710 0x560037672b00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df499703338fe70a server_cookie=df5adbd60ec5c51a in_seq=0 out_seq=0 2024-02-17T12:56:47.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.550+0000 7fab94ff9640 10 client.18024 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:56:47.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fab94ff9640 1 -- 172.21.15.18:0/4146473471 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600372fb720 con 0x560037670710 2024-02-17T12:56:47.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fab94ff9640 10 client.18024 handle_command_reply: tid=0 2024-02-17T12:56:47.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fabb600e740 1 client.18024 shutdown 2024-02-17T12:56:47.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fabb600e740 1 -- 172.21.15.18:0/4146473471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 msgr2=0x56003765d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fabb600e740 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x56003765d740 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7faba8005940 tx=0x7faba8005d50 comp rx=0 tx=0).stop 2024-02-17T12:56:47.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fabb600e740 1 -- 172.21.15.18:0/4146473471 shutdown_connections 2024-02-17T12:56:47.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fabb600e740 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560037670710 0x560037672b00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fabb600e740 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037660750 0x56003765e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fabb600e740 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657b70 0x56003765dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fab94ff9640 0 client.18024 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:56:47.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fabb600e740 1 --2- 172.21.15.18:0/4146473471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x56003765d740 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fabb600e740 1 -- 172.21.15.18:0/4146473471 >> 172.21.15.18:0/4146473471 conn(0x560037651980 msgr2=0x5600376536d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fabb600e740 1 -- 172.21.15.18:0/4146473471 shutdown_connections 2024-02-17T12:56:47.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fabb600e740 1 -- 172.21.15.18:0/4146473471 wait complete. 2024-02-17T12:56:47.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.551+0000 7fabb600e740 20 client.18024 trim_cache size 0 max 16384 2024-02-17T12:56:47.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.563+0000 7fabb600e740 1 Processor -- start 2024-02-17T12:56:47.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.563+0000 7fabb600e740 1 -- start start 2024-02-17T12:56:47.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.563+0000 7fabb600e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x560037650a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.563+0000 7fabb600e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037657b70 0x5600376675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.563+0000 7fabb600e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037660750 0x560037667ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.563+0000 7fabb600e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560037544130 con 0x560037657250 2024-02-17T12:56:47.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.563+0000 7fabb600e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560037652980 con 0x560037660750 2024-02-17T12:56:47.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.563+0000 7fabb600e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560037658620 con 0x560037657b70 2024-02-17T12:56:47.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.563+0000 7fabb1107640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x560037650a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.563+0000 7fabb1107640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x560037650a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41686/0 (socket says 172.21.15.18:41686) 2024-02-17T12:56:47.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.563+0000 7fabb1107640 1 -- 172.21.15.18:0/2111556661 learned_addr learned my addr 172.21.15.18:0/2111556661 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fabb1107640 1 -- 172.21.15.18:0/2111556661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037657b70 msgr2=0x5600376675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fabb0906640 1 --2- 172.21.15.18:0/2111556661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037657b70 0x5600376675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fabb1908640 1 --2- 172.21.15.18:0/2111556661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037660750 0x560037667ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fabb1107640 1 --2- 172.21.15.18:0/2111556661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037657b70 0x5600376675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fabb1107640 1 -- 172.21.15.18:0/2111556661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037660750 msgr2=0x560037667ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fabb1107640 1 --2- 172.21.15.18:0/2111556661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037660750 0x560037667ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fabb1107640 1 -- 172.21.15.18:0/2111556661 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560037559c20 con 0x560037657250 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fabb0906640 1 --2- 172.21.15.18:0/2111556661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037657b70 0x5600376675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fabb1908640 1 --2- 172.21.15.18:0/2111556661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037660750 0x560037667ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fabb1107640 1 --2- 172.21.15.18:0/2111556661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x560037650a00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7faba8005910 tx=0x7faba802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15215d192e494d33 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fab96ffd640 1 -- 172.21.15.18:0/2111556661 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8004df0 con 0x560037657250 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fab96ffd640 1 -- 172.21.15.18:0/2111556661 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faba803e430 con 0x560037657250 2024-02-17T12:56:47.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fab96ffd640 1 -- 172.21.15.18:0/2111556661 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80596c0 con 0x560037657250 2024-02-17T12:56:47.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fabb600e740 1 -- 172.21.15.18:0/2111556661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 msgr2=0x560037650a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.564+0000 7fabb600e740 1 --2- 172.21.15.18:0/2111556661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x560037650a00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7faba8005910 tx=0x7faba802bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:47.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 -- 172.21.15.18:0/2111556661 shutdown_connections 2024-02-17T12:56:47.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 --2- 172.21.15.18:0/2111556661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037660750 0x560037667ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 --2- 172.21.15.18:0/2111556661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037657b70 0x5600376675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 --2- 172.21.15.18:0/2111556661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037657250 0x560037650a00 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 -- 172.21.15.18:0/2111556661 >> 172.21.15.18:0/2111556661 conn(0x560037651980 msgr2=0x5600376502f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 -- 172.21.15.18:0/2111556661 shutdown_connections 2024-02-17T12:56:47.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 -- 172.21.15.18:0/2111556661 wait complete. 2024-02-17T12:56:47.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 -- start start 2024-02-17T12:56:47.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 Processor -- start 2024-02-17T12:56:47.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657250 0x560037666140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037657b70 0x560037666680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037660750 0x560037666bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600376c6ca0 con 0x560037660750 2024-02-17T12:56:47.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600376c6de0 con 0x560037657250 2024-02-17T12:56:47.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600376c6fa0 con 0x560037657b70 2024-02-17T12:56:47.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb600e740 10 client.0 _ll_register_callbacks cb 0x7faba8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:47.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb1908640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037660750 0x560037666bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb1908640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037660750 0x560037666bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41688/0 (socket says 172.21.15.18:41688) 2024-02-17T12:56:47.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.565+0000 7fabb1908640 1 -- 172.21.15.18:0/3413423210 learned_addr learned my addr 172.21.15.18:0/3413423210 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fabb0906640 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037657b70 0x560037666680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fabb1107640 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657250 0x560037666140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fabb1908640 1 -- 172.21.15.18:0/3413423210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037657b70 msgr2=0x560037666680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fabb1908640 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037657b70 0x560037666680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fabb1908640 1 -- 172.21.15.18:0/3413423210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657250 msgr2=0x560037666140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fabb1908640 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657250 0x560037666140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fabb1908640 1 -- 172.21.15.18:0/3413423210 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600375bd6a0 con 0x560037660750 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fabb1908640 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037660750 0x560037666bc0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7faba4012740 tx=0x7faba4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651084b22aaf215d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fab94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fab94ff9640 1 -- 172.21.15.18:0/3413423210 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4011930 con 0x560037660750 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fab94ff9640 1 -- 172.21.15.18:0/3413423210 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7faba4019040 con 0x560037660750 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fabb600e740 1 -- 172.21.15.18:0/3413423210 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faba8004510 con 0x560037660750 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fab94ff9640 1 -- 172.21.15.18:0/3413423210 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7faba4011a90 con 0x560037660750 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fabb600e740 10 client.18030 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:47.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fabb600e740 1 -- 172.21.15.18:0/3413423210 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56003757f8c0 con 0x560037660750 2024-02-17T12:56:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.566+0000 7fab94ff9640 1 -- 172.21.15.18:0/3413423210 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faba8004510 con 0x560037660750 2024-02-17T12:56:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.567+0000 7fab94ff9640 1 -- 172.21.15.18:0/3413423210 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7faba401f3e0 con 0x560037660750 2024-02-17T12:56:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.567+0000 7fabb600e740 10 client.18030 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.567+0000 7fabb600e740 10 client.18030 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:56:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.567+0000 7fabb600e740 20 client.18030 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.567+0000 7fabb600e740 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5600376c8770 0x5600376cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.567+0000 7fabb600e740 4 client.18030 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:56:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.567+0000 7fabb600e740 1 -- 172.21.15.18:0/3413423210 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56003734a890 con 0x5600376c8770 2024-02-17T12:56:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.567+0000 7fabb1107640 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5600376c8770 0x5600376cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.567+0000 7fabb1107640 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5600376c8770 0x5600376cab90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6954e562e41b0fb server_cookie=6938bb9d72698834 in_seq=0 out_seq=0 2024-02-17T12:56:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.567+0000 7fab94ff9640 10 client.18030 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:56:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fab94ff9640 1 -- 172.21.15.18:0/3413423210 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56003734a890 con 0x5600376c8770 2024-02-17T12:56:47.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fab94ff9640 10 client.18030 handle_command_reply: tid=0 2024-02-17T12:56:47.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb600e740 1 client.18030 shutdown 2024-02-17T12:56:47.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb600e740 1 -- 172.21.15.18:0/3413423210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037660750 msgr2=0x560037666bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb600e740 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037660750 0x560037666bc0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7faba4012740 tx=0x7faba4004ab0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb600e740 1 -- 172.21.15.18:0/3413423210 shutdown_connections 2024-02-17T12:56:47.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb600e740 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5600376c8770 0x5600376cab90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb600e740 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560037660750 0x560037666bc0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb600e740 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560037657b70 0x560037666680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fab94ff9640 0 client.18030 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:56:47.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb600e740 1 --2- 172.21.15.18:0/3413423210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560037657250 0x560037666140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb600e740 1 -- 172.21.15.18:0/3413423210 >> 172.21.15.18:0/3413423210 conn(0x560037651980 msgr2=0x560037650b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb600e740 1 -- 172.21.15.18:0/3413423210 shutdown_connections 2024-02-17T12:56:47.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb600e740 1 -- 172.21.15.18:0/3413423210 wait complete. 2024-02-17T12:56:47.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb600e740 20 client.18030 trim_cache size 0 max 16384 2024-02-17T12:56:47.568 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:56:47.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb336b640 1 -- 172.21.15.18:0/2371186263 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab980a2230 msgr2=0x7fab980a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb336b640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab980a2230 0x7fab980a46f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faba40092f0 tx=0x7faba4010740 comp rx=0 tx=0).stop 2024-02-17T12:56:47.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb336b640 1 -- 172.21.15.18:0/2371186263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabac106450 msgr2=0x7fabac197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb336b640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabac106450 0x7fabac197c70 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00c970 tx=0x7fab9c00ce30 comp rx=0 tx=0).stop 2024-02-17T12:56:47.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb336b640 1 -- 172.21.15.18:0/2371186263 shutdown_connections 2024-02-17T12:56:47.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb336b640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab980a2230 0x7fab980a46f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb336b640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabac19bff0 0x7fabac19c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb336b640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabac106450 0x7fabac197c70 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb336b640 1 --2- 172.21.15.18:0/2371186263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabac101380 0x7fabac197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb336b640 1 -- 172.21.15.18:0/2371186263 >> 172.21.15.18:0/2371186263 conn(0x7fabac078580 msgr2=0x7fabac0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.568+0000 7fabb336b640 1 -- 172.21.15.18:0/2371186263 shutdown_connections 2024-02-17T12:56:47.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.569+0000 7fabb336b640 1 -- 172.21.15.18:0/2371186263 wait complete. 2024-02-17T12:56:47.571 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:56:47.571 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:56:47.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.685+0000 7f679edbc640 1 -- 172.21.15.18:0/3623735173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67981023c0 msgr2=0x7f67981027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.685+0000 7f679edbc640 1 --2- 172.21.15.18:0/3623735173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67981023c0 0x7f67981027a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f678c009900 tx=0x7f678c02bcd0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.685+0000 7f679edbc640 1 -- 172.21.15.18:0/3623735173 shutdown_connections 2024-02-17T12:56:47.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.685+0000 7f679edbc640 1 --2- 172.21.15.18:0/3623735173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f679810b790 0x7f679810dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.685+0000 7f679edbc640 1 --2- 172.21.15.18:0/3623735173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6798102ce0 0x7f679810b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.685+0000 7f679edbc640 1 --2- 172.21.15.18:0/3623735173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67981023c0 0x7f67981027a0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.685+0000 7f679edbc640 1 -- 172.21.15.18:0/3623735173 >> 172.21.15.18:0/3623735173 conn(0x7f6798076400 msgr2=0x7f6798076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.685+0000 7f679edbc640 1 -- 172.21.15.18:0/3623735173 shutdown_connections 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.685+0000 7f679edbc640 1 -- 172.21.15.18:0/3623735173 wait complete. 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f679edbc640 1 Processor -- start 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f679edbc640 1 -- start start 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f679edbc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67981023c0 0x7f6798197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f679edbc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6798102ce0 0x7f6798197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f679edbc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679810b790 0x7f679819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f679edbc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679819c5f0 con 0x7f6798102ce0 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f679edbc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679819c760 con 0x7f679810b790 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f679edbc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679819c920 con 0x7f67981023c0 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f6797fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6798102ce0 0x7f6798197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f6797fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6798102ce0 0x7f6798197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41710/0 (socket says 172.21.15.18:41710) 2024-02-17T12:56:47.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f6797fff640 1 -- 172.21.15.18:0/1454541161 learned_addr learned my addr 172.21.15.18:0/1454541161 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f679cb58640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67981023c0 0x7f6798197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f679d359640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679810b790 0x7f679819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f6797fff640 1 -- 172.21.15.18:0/1454541161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67981023c0 msgr2=0x7f6798197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f6797fff640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67981023c0 0x7f6798197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f6797fff640 1 -- 172.21.15.18:0/1454541161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679810b790 msgr2=0x7f679819bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f6797fff640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679810b790 0x7f679819bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.686+0000 7f6797fff640 1 -- 172.21.15.18:0/1454541161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c009560 con 0x7f6798102ce0 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.687+0000 7f6797fff640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6798102ce0 0x7f6798197c70 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f678400c920 tx=0x7f678400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.687+0000 7f6795ffb640 1 -- 172.21.15.18:0/1454541161 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784007c70 con 0x7f6798102ce0 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.687+0000 7f6795ffb640 1 -- 172.21.15.18:0/1454541161 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6784007dd0 con 0x7f6798102ce0 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.687+0000 7f679edbc640 1 -- 172.21.15.18:0/1454541161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679819cc00 con 0x7f6798102ce0 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.687+0000 7f6795ffb640 1 -- 172.21.15.18:0/1454541161 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784018860 con 0x7f6798102ce0 2024-02-17T12:56:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.687+0000 7f679edbc640 1 -- 172.21.15.18:0/1454541161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67981a4480 con 0x7f6798102ce0 2024-02-17T12:56:47.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.689+0000 7f6795ffb640 1 -- 172.21.15.18:0/1454541161 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f678400fac0 con 0x7f6798102ce0 2024-02-17T12:56:47.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.689+0000 7f67a1994740 1 Processor -- start 2024-02-17T12:56:47.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.689+0000 7f6795ffb640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f67880a22b0 0x7f67880a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.689+0000 7f67a1994740 1 -- start start 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f679d359640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f67880a22b0 0x7f67880a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f6795ffb640 1 -- 172.21.15.18:0/1454541161 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f67840ce280 con 0x7f6798102ce0 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f67a1994740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 0x5612f246d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f67a1994740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246db70 0x5612f2476210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f67a1994740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f2478b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f67a1994740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f2428960 con 0x5612f2476750 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f67a1994740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f23dc310 con 0x5612f246db70 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f67a1994740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f23d9840 con 0x5612f246d250 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f679d359640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f2478b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f679d359640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f2478b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41722/0 (socket says 172.21.15.18:41722) 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f679d359640 1 -- 172.21.15.18:0/3477361944 learned_addr learned my addr 172.21.15.18:0/3477361944 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f6797fff640 1 --2- 172.21.15.18:0/3477361944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246db70 0x5612f2476210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f679cb58640 1 --2- 172.21.15.18:0/3477361944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 0x5612f246d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f679d359640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f67880a22b0 0x7f67880a4770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6790012700 tx=0x7f6790012b00 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f6797fff640 1 -- 172.21.15.18:0/3477361944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 msgr2=0x5612f246d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f6797fff640 1 --2- 172.21.15.18:0/3477361944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 0x5612f246d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f6797fff640 1 -- 172.21.15.18:0/3477361944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 msgr2=0x5612f2478b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f6797fff640 1 --2- 172.21.15.18:0/3477361944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f2478b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f6797fff640 1 -- 172.21.15.18:0/3477361944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612f2397060 con 0x5612f246db70 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f679d359640 1 --2- 172.21.15.18:0/3477361944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f2478b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.690+0000 7f6797fff640 1 --2- 172.21.15.18:0/3477361944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246db70 0x5612f2476210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67840cb990 tx=0x7f678409b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf732202e1b2c326 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.691+0000 7f6782ffd640 1 -- 172.21.15.18:0/3477361944 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840b2070 con 0x5612f246db70 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.691+0000 7f6782ffd640 1 -- 172.21.15.18:0/3477361944 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67840a33d0 con 0x5612f246db70 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.691+0000 7f6782ffd640 1 -- 172.21.15.18:0/3477361944 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840a2f40 con 0x5612f246db70 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.691+0000 7f67a1994740 1 -- 172.21.15.18:0/3477361944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246db70 msgr2=0x5612f2476210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.691+0000 7f67a1994740 1 --2- 172.21.15.18:0/3477361944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246db70 0x5612f2476210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67840cb990 tx=0x7f678409b330 comp rx=0 tx=0).stop 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.691+0000 7f67a1994740 1 -- 172.21.15.18:0/3477361944 shutdown_connections 2024-02-17T12:56:47.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.691+0000 7f67a1994740 1 --2- 172.21.15.18:0/3477361944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f2478b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.691+0000 7f67a1994740 1 --2- 172.21.15.18:0/3477361944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246db70 0x5612f2476210 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.691+0000 7f67a1994740 1 --2- 172.21.15.18:0/3477361944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 0x5612f246d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.691+0000 7f67a1994740 1 -- 172.21.15.18:0/3477361944 >> 172.21.15.18:0/3477361944 conn(0x5612f2467980 msgr2=0x5612f2467d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.691+0000 7f67a1994740 1 -- 172.21.15.18:0/3477361944 shutdown_connections 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.691+0000 7f67a1994740 1 -- 172.21.15.18:0/3477361944 wait complete. 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f67a1994740 1 -- start start 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f67a1994740 1 Processor -- start 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f67a1994740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246d250 0x5612f2473810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f67a1994740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246db70 0x5612f2473d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f67a1994740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f2474290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f67a1994740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f23d5020 con 0x5612f2476750 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f67a1994740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f20a8430 con 0x5612f246d250 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f67a1994740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f2064e00 con 0x5612f246db70 2024-02-17T12:56:47.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f67a1994740 10 client.0 _ll_register_callbacks cb 0x5612f237ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f679d359640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f2474290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f679d359640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f2474290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41732/0 (socket says 172.21.15.18:41732) 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f679d359640 1 -- 172.21.15.18:0/3807936363 learned_addr learned my addr 172.21.15.18:0/3807936363 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f679cb58640 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246d250 0x5612f2473810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.692+0000 7f6797fff640 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246db70 0x5612f2473d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f679d359640 1 -- 172.21.15.18:0/3807936363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246db70 msgr2=0x5612f2473d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f679d359640 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246db70 0x5612f2473d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f679d359640 1 -- 172.21.15.18:0/3807936363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246d250 msgr2=0x5612f2473810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f679d359640 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246d250 0x5612f2473810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f679d359640 1 -- 172.21.15.18:0/3807936363 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67981a4480 con 0x5612f2476750 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f679d359640 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f2474290 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f679000eb80 tx=0x7f6790004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81adc07e65bc6efb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f6777fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f6777fff640 1 -- 172.21.15.18:0/3807936363 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790017040 con 0x5612f2476750 2024-02-17T12:56:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f67a1994740 1 -- 172.21.15.18:0/3807936363 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612f20d47e0 con 0x5612f2476750 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f6777fff640 1 -- 172.21.15.18:0/3807936363 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f679001f430 con 0x5612f2476750 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f6777fff640 1 -- 172.21.15.18:0/3807936363 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6790004850 con 0x5612f2476750 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f67a1994740 10 client.18042 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f67a1994740 1 -- 172.21.15.18:0/3807936363 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5612f23d36a0 con 0x5612f2476750 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f6777fff640 1 -- 172.21.15.18:0/3807936363 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5612f20d47e0 con 0x5612f2476750 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f6777fff640 1 -- 172.21.15.18:0/3807936363 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6790027650 con 0x5612f2476750 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f67a1994740 10 client.18042 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f67a1994740 10 client.18042 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f67a1994740 20 client.18042 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f67a1994740 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5612f2486bb0 0x5612f2488fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f67a1994740 4 client.18042 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.693+0000 7f67a1994740 1 -- 172.21.15.18:0/3807936363 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612f2111720 con 0x5612f2486bb0 2024-02-17T12:56:47.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.694+0000 7f679cb58640 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5612f2486bb0 0x5612f2488fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.694+0000 7f6777fff640 10 client.18042 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.694+0000 7f679cb58640 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5612f2486bb0 0x5612f2488fd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec92fae718e48093 server_cookie=8370fd10d02cf7e in_seq=0 out_seq=0 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f6777fff640 1 -- 172.21.15.18:0/3807936363 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612f2111720 con 0x5612f2486bb0 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f6777fff640 10 client.18042 handle_command_reply: tid=0 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f67a1994740 1 client.18042 shutdown 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f67a1994740 1 -- 172.21.15.18:0/3807936363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 msgr2=0x5612f2474290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f67a1994740 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f2474290 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f679000eb80 tx=0x7f6790004ab0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f67a1994740 1 -- 172.21.15.18:0/3807936363 shutdown_connections 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f67a1994740 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5612f2486bb0 0x5612f2488fd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f67a1994740 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f2474290 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f67a1994740 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246db70 0x5612f2473d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f67a1994740 1 --2- 172.21.15.18:0/3807936363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246d250 0x5612f2473810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f6777fff640 0 client.18042 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:56:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f67a1994740 1 -- 172.21.15.18:0/3807936363 >> 172.21.15.18:0/3807936363 conn(0x5612f2467980 msgr2=0x5612f24696d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f67a1994740 1 -- 172.21.15.18:0/3807936363 shutdown_connections 2024-02-17T12:56:47.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f67a1994740 1 -- 172.21.15.18:0/3807936363 wait complete. 2024-02-17T12:56:47.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.695+0000 7f67a1994740 20 client.18042 trim_cache size 0 max 16384 2024-02-17T12:56:47.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f67a1994740 1 Processor -- start 2024-02-17T12:56:47.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f67a1994740 1 -- start start 2024-02-17T12:56:47.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f67a1994740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 0x5612f2474080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f67a1994740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f246db70 0x5612f2466a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f67a1994740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f2476750 0x5612f247d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f67a1994740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f235a130 con 0x5612f246db70 2024-02-17T12:56:47.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f67a1994740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f2486980 con 0x5612f2476750 2024-02-17T12:56:47.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f67a1994740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f248e200 con 0x5612f246d250 2024-02-17T12:56:47.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f6797fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f246db70 0x5612f2466a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f6797fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f246db70 0x5612f2466a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41744/0 (socket says 172.21.15.18:41744) 2024-02-17T12:56:47.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f6797fff640 1 -- 172.21.15.18:0/3045036101 learned_addr learned my addr 172.21.15.18:0/3045036101 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f6797fff640 1 -- 172.21.15.18:0/3045036101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 msgr2=0x5612f2474080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f6797fff640 1 --2- 172.21.15.18:0/3045036101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 0x5612f2474080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f6797fff640 1 -- 172.21.15.18:0/3045036101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f2476750 msgr2=0x5612f247d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f6797fff640 1 --2- 172.21.15.18:0/3045036101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f2476750 0x5612f247d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.707+0000 7f6797fff640 1 -- 172.21.15.18:0/3045036101 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612f236fc20 con 0x5612f246db70 2024-02-17T12:56:47.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f6797fff640 1 --2- 172.21.15.18:0/3045036101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f246db70 0x5612f2466a00 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6784098ee0 tx=0x7f6784007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a9afab81f4c59 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f67827fc640 1 -- 172.21.15.18:0/3045036101 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678407d070 con 0x5612f246db70 2024-02-17T12:56:47.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f67827fc640 1 -- 172.21.15.18:0/3045036101 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6784098f30 con 0x5612f246db70 2024-02-17T12:56:47.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f67827fc640 1 -- 172.21.15.18:0/3045036101 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840b2040 con 0x5612f246db70 2024-02-17T12:56:47.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f67a1994740 1 -- 172.21.15.18:0/3045036101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f246db70 msgr2=0x5612f2466a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f67a1994740 1 --2- 172.21.15.18:0/3045036101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f246db70 0x5612f2466a00 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6784098ee0 tx=0x7f6784007f60 comp rx=0 tx=0).stop 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f67a1994740 1 -- 172.21.15.18:0/3045036101 shutdown_connections 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f67a1994740 1 --2- 172.21.15.18:0/3045036101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f2476750 0x5612f247d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f67a1994740 1 --2- 172.21.15.18:0/3045036101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f246db70 0x5612f2466a00 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f67a1994740 1 --2- 172.21.15.18:0/3045036101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 0x5612f2474080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f67a1994740 1 -- 172.21.15.18:0/3045036101 >> 172.21.15.18:0/3045036101 conn(0x5612f2467980 msgr2=0x5612f24662f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f67a1994740 1 -- 172.21.15.18:0/3045036101 shutdown_connections 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.708+0000 7f67a1994740 1 -- 172.21.15.18:0/3045036101 wait complete. 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f67a1994740 1 -- start start 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f67a1994740 1 Processor -- start 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f67a1994740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 0x5612f247c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f67a1994740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246db70 0x5612f247c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f67a1994740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f247cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f67a1994740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f24dd0e0 con 0x5612f2476750 2024-02-17T12:56:47.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f67a1994740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f24dd220 con 0x5612f246db70 2024-02-17T12:56:47.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f67a1994740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f24dd360 con 0x5612f246d250 2024-02-17T12:56:47.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f67a1994740 10 client.0 _ll_register_callbacks cb 0x7f6784096ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:47.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f679d359640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f247cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f679d359640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f247cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41750/0 (socket says 172.21.15.18:41750) 2024-02-17T12:56:47.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f679d359640 1 -- 172.21.15.18:0/4037170907 learned_addr learned my addr 172.21.15.18:0/4037170907 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f679cb58640 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 0x5612f247c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f6797fff640 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246db70 0x5612f247c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f679d359640 1 -- 172.21.15.18:0/4037170907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 msgr2=0x5612f247c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f679d359640 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 0x5612f247c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f679d359640 1 -- 172.21.15.18:0/4037170907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246db70 msgr2=0x5612f247c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f679d359640 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246db70 0x5612f247c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f679d359640 1 -- 172.21.15.18:0/4037170907 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612f2397060 con 0x5612f2476750 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.709+0000 7f6797fff640 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246db70 0x5612f247c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f679d359640 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f247cb30 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6790027ab0 tx=0x7f67900276c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1aa0117d9c914c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f6777fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f6777fff640 1 -- 172.21.15.18:0/4037170907 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790017070 con 0x5612f2476750 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f6777fff640 1 -- 172.21.15.18:0/4037170907 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6790003c80 con 0x5612f2476750 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f67a1994740 1 -- 172.21.15.18:0/4037170907 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67840a33d0 con 0x5612f2476750 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f6777fff640 1 -- 172.21.15.18:0/4037170907 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6790009ea0 con 0x5612f2476750 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f67a1994740 10 client.18048 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f67a1994740 1 -- 172.21.15.18:0/4037170907 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5612f23958c0 con 0x5612f2476750 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f6777fff640 1 -- 172.21.15.18:0/4037170907 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67840a33d0 con 0x5612f2476750 2024-02-17T12:56:47.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f6777fff640 1 -- 172.21.15.18:0/4037170907 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6790005450 con 0x5612f2476750 2024-02-17T12:56:47.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f67a1994740 10 client.18048 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:47.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f67a1994740 10 client.18048 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:56:47.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f67a1994740 20 client.18048 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:47.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f67a1994740 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5612f24deb80 0x5612f24e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f67a1994740 4 client.18048 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:56:47.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.710+0000 7f67a1994740 1 -- 172.21.15.18:0/4037170907 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5612f2160890 con 0x5612f24deb80 2024-02-17T12:56:47.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f679cb58640 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5612f24deb80 0x5612f24e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f6777fff640 10 client.18048 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:56:47.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f679cb58640 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5612f24deb80 0x5612f24e0fa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cafece3e1134b8a2 server_cookie=1a44c0424641f752 in_seq=0 out_seq=0 2024-02-17T12:56:47.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f6777fff640 1 -- 172.21.15.18:0/4037170907 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5612f2160890 con 0x5612f24deb80 2024-02-17T12:56:47.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f6777fff640 10 client.18048 handle_command_reply: tid=0 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f67a1994740 1 client.18048 shutdown 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f67a1994740 1 -- 172.21.15.18:0/4037170907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 msgr2=0x5612f247cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f67a1994740 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f247cb30 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6790027ab0 tx=0x7f67900276c0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f67a1994740 1 -- 172.21.15.18:0/4037170907 shutdown_connections 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f67a1994740 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5612f24deb80 0x5612f24e0fa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f67a1994740 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5612f2476750 0x5612f247cb30 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f67a1994740 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5612f246db70 0x5612f247c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f6777fff640 0 client.18048 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f67a1994740 1 --2- 172.21.15.18:0/4037170907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5612f246d250 0x5612f247c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f67a1994740 1 -- 172.21.15.18:0/4037170907 >> 172.21.15.18:0/4037170907 conn(0x5612f2467980 msgr2=0x5612f2466b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f67a1994740 1 -- 172.21.15.18:0/4037170907 shutdown_connections 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.711+0000 7f67a1994740 1 -- 172.21.15.18:0/4037170907 wait complete. 2024-02-17T12:56:47.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f67a1994740 20 client.18048 trim_cache size 0 max 16384 2024-02-17T12:56:47.710 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:56:47.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f679edbc640 1 -- 172.21.15.18:0/1454541161 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f67880a22b0 msgr2=0x7f67880a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f679edbc640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f67880a22b0 0x7f67880a4770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6790012700 tx=0x7f6790012b00 comp rx=0 tx=0).stop 2024-02-17T12:56:47.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f679edbc640 1 -- 172.21.15.18:0/1454541161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6798102ce0 msgr2=0x7f6798197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f679edbc640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6798102ce0 0x7f6798197c70 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f678400c920 tx=0x7f678400cde0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f679edbc640 1 -- 172.21.15.18:0/1454541161 shutdown_connections 2024-02-17T12:56:47.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f679edbc640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f67880a22b0 0x7f67880a4770 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f679edbc640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679810b790 0x7f679819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f679edbc640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6798102ce0 0x7f6798197c70 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f679edbc640 1 --2- 172.21.15.18:0/1454541161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67981023c0 0x7f6798197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f679edbc640 1 -- 172.21.15.18:0/1454541161 >> 172.21.15.18:0/1454541161 conn(0x7f6798076400 msgr2=0x7f67980fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f679edbc640 1 -- 172.21.15.18:0/1454541161 shutdown_connections 2024-02-17T12:56:47.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.712+0000 7f679edbc640 1 -- 172.21.15.18:0/1454541161 wait complete. 2024-02-17T12:56:47.713 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:56:47.713 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:56:47.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.828+0000 7f82a0d6c640 1 -- 172.21.15.18:0/2136414740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f829c100260 msgr2=0x7f829c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.828+0000 7f82a0d6c640 1 --2- 172.21.15.18:0/2136414740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f829c100260 0x7f829c100640 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f828c009930 tx=0x7f828c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:47.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.828+0000 7f82a0d6c640 1 -- 172.21.15.18:0/2136414740 shutdown_connections 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.828+0000 7f82a0d6c640 1 --2- 172.21.15.18:0/2136414740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f829c105330 0x7f829c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.828+0000 7f82a0d6c640 1 --2- 172.21.15.18:0/2136414740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f829c100c10 0x7f829c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.828+0000 7f82a0d6c640 1 --2- 172.21.15.18:0/2136414740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f829c100260 0x7f829c100640 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.828+0000 7f82a0d6c640 1 -- 172.21.15.18:0/2136414740 >> 172.21.15.18:0/2136414740 conn(0x7f829c077180 msgr2=0x7f829c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.828+0000 7f82a0d6c640 1 -- 172.21.15.18:0/2136414740 shutdown_connections 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.828+0000 7f82a0d6c640 1 -- 172.21.15.18:0/2136414740 wait complete. 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f82a0d6c640 1 Processor -- start 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f82a0d6c640 1 -- start start 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f82a0d6c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f829c100260 0x7f829c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f82a0d6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f829c105330 0x7f829c102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f82a0d6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f829c102770 0x7f829c1a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f82a0d6c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c1a5fc0 con 0x7f829c102770 2024-02-17T12:56:47.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f82a0d6c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c1a6130 con 0x7f829c105330 2024-02-17T12:56:47.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f82a0d6c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c1a62f0 con 0x7f829c100260 2024-02-17T12:56:47.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f829ad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f829c102770 0x7f829c1a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f829ad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f829c102770 0x7f829c1a59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41770/0 (socket says 172.21.15.18:41770) 2024-02-17T12:56:47.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f829ad9d640 1 -- 172.21.15.18:0/1440276241 learned_addr learned my addr 172.21.15.18:0/1440276241 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f829a59c640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f829c100260 0x7f829c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.829+0000 7f8293fff640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f829c105330 0x7f829c102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.830+0000 7f829ad9d640 1 -- 172.21.15.18:0/1440276241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f829c100260 msgr2=0x7f829c101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.830+0000 7f829ad9d640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f829c100260 0x7f829c101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.830+0000 7f829ad9d640 1 -- 172.21.15.18:0/1440276241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f829c105330 msgr2=0x7f829c102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.830+0000 7f829ad9d640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f829c105330 0x7f829c102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.830+0000 7f829ad9d640 1 -- 172.21.15.18:0/1440276241 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c009560 con 0x7f829c102770 2024-02-17T12:56:47.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.830+0000 7f8293fff640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f829c105330 0x7f829c102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:47.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.830+0000 7f829ad9d640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f829c102770 0x7f829c1a59f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f829400cad0 tx=0x7f829400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.830+0000 7f82937fe640 1 -- 172.21.15.18:0/1440276241 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829400ae20 con 0x7f829c102770 2024-02-17T12:56:47.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.830+0000 7f82937fe640 1 -- 172.21.15.18:0/1440276241 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8294005020 con 0x7f829c102770 2024-02-17T12:56:47.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.830+0000 7f82937fe640 1 -- 172.21.15.18:0/1440276241 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829400f6b0 con 0x7f829c102770 2024-02-17T12:56:47.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.830+0000 7f82a0d6c640 1 -- 172.21.15.18:0/1440276241 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829c1a65d0 con 0x7f829c102770 2024-02-17T12:56:47.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.830+0000 7f82a0d6c640 1 -- 172.21.15.18:0/1440276241 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c1a6ae0 con 0x7f829c102770 2024-02-17T12:56:47.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.832+0000 7f82937fe640 1 -- 172.21.15.18:0/1440276241 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8294005520 con 0x7f829c102770 2024-02-17T12:56:47.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.832+0000 7f82a3a1d740 1 Processor -- start 2024-02-17T12:56:47.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f82a3a1d740 1 -- start start 2024-02-17T12:56:47.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f82a3a1d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77f250 0x55d78b77f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f82a3a1d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77fb70 0x55d78b788210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f82a3a1d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b788750 0x55d78b78ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f82a3a1d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78b73a960 con 0x55d78b77f250 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f82a3a1d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78b6ee310 con 0x55d78b77fb70 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f82a3a1d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78b6eb840 con 0x55d78b788750 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f8293fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77f250 0x55d78b77f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f8293fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77f250 0x55d78b77f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41774/0 (socket says 172.21.15.18:41774) 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f8293fff640 1 -- 172.21.15.18:0/1136590643 learned_addr learned my addr 172.21.15.18:0/1136590643 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f82937fe640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82880a2260 0x7f82880a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f8293fff640 1 --2- 172.21.15.18:0/1136590643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b788750 0x55d78b78ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f829a59c640 1 --2- 172.21.15.18:0/1136590643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77fb70 0x55d78b788210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f8293fff640 1 -- 172.21.15.18:0/1136590643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b788750 msgr2=0x55d78b78ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f8293fff640 1 --2- 172.21.15.18:0/1136590643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b788750 0x55d78b78ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f8293fff640 1 -- 172.21.15.18:0/1136590643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77fb70 msgr2=0x55d78b788210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f8293fff640 1 --2- 172.21.15.18:0/1136590643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77fb70 0x55d78b788210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f829ad9d640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82880a2260 0x7f82880a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f8293fff640 1 -- 172.21.15.18:0/1136590643 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d78b6a9060 con 0x55d78b77f250 2024-02-17T12:56:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f829a59c640 1 --2- 172.21.15.18:0/1136590643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77fb70 0x55d78b788210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f82937fe640 1 -- 172.21.15.18:0/1440276241 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f82940ce330 con 0x7f829c102770 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.833+0000 7f8293fff640 1 --2- 172.21.15.18:0/1136590643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77f250 0x55d78b77f630 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8284005e00 tx=0x7f8284009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9d43dd1331c0a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f8290ff9640 1 -- 172.21.15.18:0/1136590643 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828401a070 con 0x55d78b77f250 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f8290ff9640 1 -- 172.21.15.18:0/1136590643 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82840150f0 con 0x55d78b77f250 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f8290ff9640 1 -- 172.21.15.18:0/1136590643 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284003d20 con 0x55d78b77f250 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1136590643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77f250 msgr2=0x55d78b77f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f829ad9d640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82880a2260 0x7f82880a4720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f829409aa40 tx=0x7f829406ddb0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/1136590643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77f250 0x55d78b77f630 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8284005e00 tx=0x7f8284009670 comp rx=0 tx=0).stop 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1136590643 shutdown_connections 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/1136590643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b788750 0x55d78b78ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/1136590643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77fb70 0x55d78b788210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/1136590643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77f250 0x55d78b77f630 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1136590643 >> 172.21.15.18:0/1136590643 conn(0x55d78b779980 msgr2=0x55d78b779d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1136590643 shutdown_connections 2024-02-17T12:56:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.834+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1136590643 wait complete. 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f82a3a1d740 1 -- start start 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f82a3a1d740 1 Processor -- start 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f82a3a1d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77f250 0x55d78b7857b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f82a3a1d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 0x55d78b785cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f82a3a1d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b788750 0x55d78b786230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f82a3a1d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78b6e7020 con 0x55d78b77fb70 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f82a3a1d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78b3ba430 con 0x55d78b77f250 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f82a3a1d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78b376e00 con 0x55d78b788750 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f82a3a1d740 10 client.0 _ll_register_callbacks cb 0x55d78b68ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f829a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 0x55d78b785cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f829a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 0x55d78b785cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41790/0 (socket says 172.21.15.18:41790) 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f829a59c640 1 -- 172.21.15.18:0/922130510 learned_addr learned my addr 172.21.15.18:0/922130510 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f8293fff640 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77f250 0x55d78b7857b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f8293fff640 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b788750 0x55d78b786230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f829a59c640 1 -- 172.21.15.18:0/922130510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b788750 msgr2=0x55d78b786230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f829a59c640 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b788750 0x55d78b786230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.835+0000 7f829a59c640 1 -- 172.21.15.18:0/922130510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77f250 msgr2=0x55d78b7857b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f829a59c640 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77f250 0x55d78b7857b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f829a59c640 1 -- 172.21.15.18:0/922130510 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8284009da0 con 0x55d78b77fb70 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f829a59c640 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 0x55d78b785cf0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f828c036040 tx=0x7f828c02caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3222802426289b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f8281ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f8281ffb640 1 -- 172.21.15.18:0/922130510 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c004250 con 0x55d78b77fb70 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f8281ffb640 1 -- 172.21.15.18:0/922130510 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f828c0043b0 con 0x55d78b77fb70 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f82a3a1d740 1 -- 172.21.15.18:0/922130510 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d78b3e67e0 con 0x55d78b77fb70 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f8281ffb640 1 -- 172.21.15.18:0/922130510 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f828c004df0 con 0x55d78b77fb70 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f82a3a1d740 10 client.18060 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:47.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f82a3a1d740 1 -- 172.21.15.18:0/922130510 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d78b6e56a0 con 0x55d78b77fb70 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f8281ffb640 1 -- 172.21.15.18:0/922130510 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d78b3e67e0 con 0x55d78b77fb70 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f8281ffb640 1 -- 172.21.15.18:0/922130510 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f828c03c050 con 0x55d78b77fb70 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f82a3a1d740 10 client.18060 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f82a3a1d740 10 client.18060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f82a3a1d740 20 client.18060 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d78b798b30 0x55d78b79af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f82a3a1d740 4 client.18060 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.836+0000 7f82a3a1d740 1 -- 172.21.15.18:0/922130510 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d78b423720 con 0x55d78b798b30 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.837+0000 7f8293fff640 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d78b798b30 0x55d78b79af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.837+0000 7f8293fff640 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d78b798b30 0x55d78b79af50 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4c07111398070c0 server_cookie=9aca0b1622162253 in_seq=0 out_seq=0 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.837+0000 7f8281ffb640 10 client.18060 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f8281ffb640 1 -- 172.21.15.18:0/922130510 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d78b423720 con 0x55d78b798b30 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f8281ffb640 10 client.18060 handle_command_reply: tid=0 2024-02-17T12:56:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f82a3a1d740 1 client.18060 shutdown 2024-02-17T12:56:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f82a3a1d740 1 -- 172.21.15.18:0/922130510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 msgr2=0x55d78b785cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 0x55d78b785cf0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f828c036040 tx=0x7f828c02caa0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f82a3a1d740 1 -- 172.21.15.18:0/922130510 shutdown_connections 2024-02-17T12:56:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d78b798b30 0x55d78b79af50 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b788750 0x55d78b786230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 0x55d78b785cf0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f8281ffb640 0 client.18060 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/922130510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77f250 0x55d78b7857b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f82a3a1d740 1 -- 172.21.15.18:0/922130510 >> 172.21.15.18:0/922130510 conn(0x55d78b779980 msgr2=0x55d78b77b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f82a3a1d740 1 -- 172.21.15.18:0/922130510 shutdown_connections 2024-02-17T12:56:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f82a3a1d740 1 -- 172.21.15.18:0/922130510 wait complete. 2024-02-17T12:56:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.838+0000 7f82a3a1d740 20 client.18060 trim_cache size 0 max 16384 2024-02-17T12:56:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f82a3a1d740 1 Processor -- start 2024-02-17T12:56:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f82a3a1d740 1 -- start start 2024-02-17T12:56:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f82a3a1d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b77f250 0x55d78b787ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f82a3a1d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 0x55d78b786050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f82a3a1d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b788750 0x55d78b782a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f82a3a1d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78b66c130 con 0x55d78b77fb70 2024-02-17T12:56:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f82a3a1d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78b7813e0 con 0x55d78b788750 2024-02-17T12:56:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f82a3a1d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78b775610 con 0x55d78b77f250 2024-02-17T12:56:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f829a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 0x55d78b786050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f829a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 0x55d78b786050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41804/0 (socket says 172.21.15.18:41804) 2024-02-17T12:56:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f829a59c640 1 -- 172.21.15.18:0/3331325999 learned_addr learned my addr 172.21.15.18:0/3331325999 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f8293fff640 1 --2- 172.21.15.18:0/3331325999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b77f250 0x55d78b787ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f8293fff640 1 --2- 172.21.15.18:0/3331325999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b788750 0x55d78b782a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f829a59c640 1 -- 172.21.15.18:0/3331325999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b77f250 msgr2=0x55d78b787ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f829a59c640 1 --2- 172.21.15.18:0/3331325999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b77f250 0x55d78b787ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f829a59c640 1 -- 172.21.15.18:0/3331325999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b788750 msgr2=0x55d78b782a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f829a59c640 1 --2- 172.21.15.18:0/3331325999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b788750 0x55d78b782a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.851+0000 7f829a59c640 1 -- 172.21.15.18:0/3331325999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d78b681c20 con 0x55d78b77fb70 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.852+0000 7f829a59c640 1 --2- 172.21.15.18:0/3331325999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 0x55d78b786050 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f828c009850 tx=0x7f828c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a777885e95dd72 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.852+0000 7f8283fff640 1 -- 172.21.15.18:0/3331325999 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c0037f0 con 0x55d78b77fb70 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.852+0000 7f8283fff640 1 -- 172.21.15.18:0/3331325999 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f828c02c5d0 con 0x55d78b77fb70 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.852+0000 7f8283fff640 1 -- 172.21.15.18:0/3331325999 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c005530 con 0x55d78b77fb70 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.852+0000 7f82a3a1d740 1 -- 172.21.15.18:0/3331325999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 msgr2=0x55d78b786050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.852+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/3331325999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 0x55d78b786050 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f828c009850 tx=0x7f828c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.852+0000 7f82a3a1d740 1 -- 172.21.15.18:0/3331325999 shutdown_connections 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.852+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/3331325999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b788750 0x55d78b782a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.852+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/3331325999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b77fb70 0x55d78b786050 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.852+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/3331325999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b77f250 0x55d78b787ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.852+0000 7f82a3a1d740 1 -- 172.21.15.18:0/3331325999 >> 172.21.15.18:0/3331325999 conn(0x55d78b779980 msgr2=0x55d78b79f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.852+0000 7f82a3a1d740 1 -- 172.21.15.18:0/3331325999 shutdown_connections 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f82a3a1d740 1 -- 172.21.15.18:0/3331325999 wait complete. 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f82a3a1d740 1 -- start start 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f82a3a1d740 1 Processor -- start 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f82a3a1d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b77f250 0x55d78b78dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f82a3a1d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77fb70 0x55d78b78e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f82a3a1d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b788750 0x55d78b78ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f82a3a1d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78b7ef020 con 0x55d78b788750 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f82a3a1d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78b7ef160 con 0x55d78b77fb70 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f82a3a1d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78b7ef2a0 con 0x55d78b77f250 2024-02-17T12:56:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f82a3a1d740 10 client.0 _ll_register_callbacks cb 0x7f82840094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f8293fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b788750 0x55d78b78ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f8293fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b788750 0x55d78b78ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41810/0 (socket says 172.21.15.18:41810) 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f8293fff640 1 -- 172.21.15.18:0/1024362821 learned_addr learned my addr 172.21.15.18:0/1024362821 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f8293fff640 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b77f250 0x55d78b78dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f829a59c640 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77fb70 0x55d78b78e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f8293fff640 1 -- 172.21.15.18:0/1024362821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b77f250 msgr2=0x55d78b78dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f8293fff640 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b77f250 0x55d78b78dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f8293fff640 1 -- 172.21.15.18:0/1024362821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77fb70 msgr2=0x55d78b78e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.853+0000 7f8293fff640 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77fb70 0x55d78b78e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f8293fff640 1 -- 172.21.15.18:0/1024362821 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d78b6e56a0 con 0x55d78b788750 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f8293fff640 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b788750 0x55d78b78ea30 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8284003990 tx=0x7f828c004b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c381db25f6d58e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f8281ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f8281ffb640 1 -- 172.21.15.18:0/1024362821 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284027040 con 0x55d78b788750 2024-02-17T12:56:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1024362821 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82840150f0 con 0x55d78b788750 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f8281ffb640 1 -- 172.21.15.18:0/1024362821 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8284029490 con 0x55d78b788750 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f8281ffb640 1 -- 172.21.15.18:0/1024362821 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f828401a040 con 0x55d78b788750 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f82a3a1d740 10 client.18066 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1024362821 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d78b6a78c0 con 0x55d78b788750 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f8281ffb640 1 -- 172.21.15.18:0/1024362821 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8284039080 con 0x55d78b788750 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f8281ffb640 1 -- 172.21.15.18:0/1024362821 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8284013d10 con 0x55d78b788750 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f82a3a1d740 10 client.18066 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f82a3a1d740 10 client.18066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f82a3a1d740 20 client.18066 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d78b798b30 0x55d78b7f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f82a3a1d740 4 client.18066 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.854+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1024362821 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d78b472890 con 0x55d78b798b30 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f829a59c640 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d78b798b30 0x55d78b7f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f8281ffb640 10 client.18066 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f829a59c640 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d78b798b30 0x55d78b7f2ab0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb4723c909e30064 server_cookie=a2faae57e3afe028 in_seq=0 out_seq=0 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f8281ffb640 1 -- 172.21.15.18:0/1024362821 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d78b472890 con 0x55d78b798b30 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f8281ffb640 10 client.18066 handle_command_reply: tid=0 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f82a3a1d740 1 client.18066 shutdown 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1024362821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b788750 msgr2=0x55d78b78ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b788750 0x55d78b78ea30 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8284003990 tx=0x7f828c004b30 comp rx=0 tx=0).stop 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1024362821 shutdown_connections 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d78b798b30 0x55d78b7f2ab0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d78b788750 0x55d78b78ea30 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d78b77fb70 0x55d78b78e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f8281ffb640 0 client.18066 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f82a3a1d740 1 --2- 172.21.15.18:0/1024362821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d78b77f250 0x55d78b78dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1024362821 >> 172.21.15.18:0/1024362821 conn(0x55d78b779980 msgr2=0x55d78b7782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1024362821 shutdown_connections 2024-02-17T12:56:47.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f82a3a1d740 1 -- 172.21.15.18:0/1024362821 wait complete. 2024-02-17T12:56:47.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.855+0000 7f82a3a1d740 20 client.18066 trim_cache size 0 max 16384 2024-02-17T12:56:47.853 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:56:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.856+0000 7f82a0d6c640 1 -- 172.21.15.18:0/1440276241 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82880a2260 msgr2=0x7f82880a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.856+0000 7f82a0d6c640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82880a2260 0x7f82880a4720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f829409aa40 tx=0x7f829406ddb0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.856+0000 7f82a0d6c640 1 -- 172.21.15.18:0/1440276241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f829c102770 msgr2=0x7f829c1a59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.856+0000 7f82a0d6c640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f829c102770 0x7f829c1a59f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f829400cad0 tx=0x7f829400cf90 comp rx=0 tx=0).stop 2024-02-17T12:56:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.856+0000 7f82a0d6c640 1 -- 172.21.15.18:0/1440276241 shutdown_connections 2024-02-17T12:56:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.856+0000 7f82a0d6c640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f82880a2260 0x7f82880a4720 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.856+0000 7f82a0d6c640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f829c102770 0x7f829c1a59f0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.856+0000 7f82a0d6c640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f829c105330 0x7f829c102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.856+0000 7f82a0d6c640 1 --2- 172.21.15.18:0/1440276241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f829c100260 0x7f829c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:47.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.856+0000 7f82a0d6c640 1 -- 172.21.15.18:0/1440276241 >> 172.21.15.18:0/1440276241 conn(0x7f829c077180 msgr2=0x7f829c0686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:47.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.856+0000 7f82a0d6c640 1 -- 172.21.15.18:0/1440276241 shutdown_connections 2024-02-17T12:56:47.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:47.856+0000 7f82a0d6c640 1 -- 172.21.15.18:0/1440276241 wait complete. 2024-02-17T12:56:48.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:47 smithi018 ceph-mon[32625]: pgmap v213: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-02-17T12:56:48.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:47 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3696564147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:56:48.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:47 smithi146 ceph-mon[36454]: pgmap v213: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-02-17T12:56:48.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:47 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3696564147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:56:48.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 363483 5.71 MB/sec execute 243 sec latency 3472.364 ms 2024-02-17T12:56:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.300+0000 7fc33fcdf640 1 -- 172.21.15.18:0/3791895797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3380ff7b0 msgr2=0x7fc3380ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.300+0000 7fc33fcdf640 1 --2- 172.21.15.18:0/3791895797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3380ff7b0 0x7fc3380ffc30 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc328009930 tx=0x7fc32802bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.300+0000 7fc33fcdf640 1 -- 172.21.15.18:0/3791895797 shutdown_connections 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.300+0000 7fc33fcdf640 1 --2- 172.21.15.18:0/3791895797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc338100170 0x7fc33810c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.300+0000 7fc33fcdf640 1 --2- 172.21.15.18:0/3791895797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3380ff7b0 0x7fc3380ffc30 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.300+0000 7fc33fcdf640 1 --2- 172.21.15.18:0/3791895797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3381057c0 0x7fc338105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.300+0000 7fc33fcdf640 1 -- 172.21.15.18:0/3791895797 >> 172.21.15.18:0/3791895797 conn(0x7fc3380765a0 msgr2=0x7fc3380fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.300+0000 7fc33fcdf640 1 -- 172.21.15.18:0/3791895797 shutdown_connections 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.300+0000 7fc33fcdf640 1 -- 172.21.15.18:0/3791895797 wait complete. 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.300+0000 7fc33fcdf640 1 Processor -- start 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.300+0000 7fc33fcdf640 1 -- start start 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33fcdf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3380ff7b0 0x7fc338197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33fcdf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc338100170 0x7fc338197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33fcdf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3381057c0 0x7fc33819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33fcdf640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33819c5c0 con 0x7fc3380ff7b0 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33fcdf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33819c730 con 0x7fc338100170 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33fcdf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33819c8f0 con 0x7fc3381057c0 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33da7b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3380ff7b0 0x7fc338197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33da7b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3380ff7b0 0x7fc338197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41816/0 (socket says 172.21.15.18:41816) 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33da7b640 1 -- 172.21.15.18:0/4143137582 learned_addr learned my addr 172.21.15.18:0/4143137582 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33d27a640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc338100170 0x7fc338197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33da7b640 1 -- 172.21.15.18:0/4143137582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3381057c0 msgr2=0x7fc33819bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33da7b640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3381057c0 0x7fc33819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33da7b640 1 -- 172.21.15.18:0/4143137582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc338100170 msgr2=0x7fc338197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33da7b640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc338100170 0x7fc338197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33da7b640 1 -- 172.21.15.18:0/4143137582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc328009560 con 0x7fc3380ff7b0 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33d27a640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc338100170 0x7fc338197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33da7b640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3380ff7b0 0x7fc338197700 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc33400c920 tx=0x7fc33400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc32effd640 1 -- 172.21.15.18:0/4143137582 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334007c70 con 0x7fc3380ff7b0 2024-02-17T12:56:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33fcdf640 1 -- 172.21.15.18:0/4143137582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33819cbd0 con 0x7fc3380ff7b0 2024-02-17T12:56:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc32effd640 1 -- 172.21.15.18:0/4143137582 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc334007dd0 con 0x7fc3380ff7b0 2024-02-17T12:56:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc32effd640 1 -- 172.21.15.18:0/4143137582 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334018860 con 0x7fc3380ff7b0 2024-02-17T12:56:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.301+0000 7fc33fcdf640 1 -- 172.21.15.18:0/4143137582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3381a4450 con 0x7fc3380ff7b0 2024-02-17T12:56:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.304+0000 7fc32effd640 1 -- 172.21.15.18:0/4143137582 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc33400fac0 con 0x7fc3380ff7b0 2024-02-17T12:56:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.304+0000 7fc342994740 1 Processor -- start 2024-02-17T12:56:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc342994740 1 -- start start 2024-02-17T12:56:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc342994740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7eaad0 0x562abb7e9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc342994740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7e9d60 0x562abb7ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc342994740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7f32e0 0x562abb7f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc342994740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abb7589f0 con 0x562abb7f32e0 2024-02-17T12:56:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc342994740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abb755f20 con 0x562abb7e9d60 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc342994740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abb751700 con 0x562abb7eaad0 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc33d27a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7f32e0 0x562abb7f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc33d27a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7f32e0 0x562abb7f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41826/0 (socket says 172.21.15.18:41826) 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc33d27a640 1 -- 172.21.15.18:0/2067443016 learned_addr learned my addr 172.21.15.18:0/2067443016 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc32effd640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc3240a22d0 0x7fc3240a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc33da7b640 1 --2- 172.21.15.18:0/2067443016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7e9d60 0x562abb7ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc33e27c640 1 --2- 172.21.15.18:0/2067443016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7eaad0 0x562abb7e9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc33e27c640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc3240a22d0 0x7fc3240a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc33d27a640 1 -- 172.21.15.18:0/2067443016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7eaad0 msgr2=0x562abb7e9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc33d27a640 1 --2- 172.21.15.18:0/2067443016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7eaad0 0x562abb7e9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc33d27a640 1 -- 172.21.15.18:0/2067443016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7e9d60 msgr2=0x562abb7ea1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc33d27a640 1 --2- 172.21.15.18:0/2067443016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7e9d60 0x562abb7ea1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc33d27a640 1 -- 172.21.15.18:0/2067443016 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562abb6e77b0 con 0x562abb7f32e0 2024-02-17T12:56:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.305+0000 7fc32effd640 1 -- 172.21.15.18:0/4143137582 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc3340ce1c0 con 0x7fc3380ff7b0 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.306+0000 7fc33d27a640 1 --2- 172.21.15.18:0/2067443016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7f32e0 0x562abb7f56d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc328009930 tx=0x7fc3280047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76f1b66c1271ea8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.306+0000 7fc33e27c640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc3240a22d0 0x7fc3240a4790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc330013a00 tx=0x7fc330005e00 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.306+0000 7fc323fff640 1 -- 172.21.15.18:0/2067443016 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32802c9e0 con 0x562abb7f32e0 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.306+0000 7fc323fff640 1 -- 172.21.15.18:0/2067443016 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc328004510 con 0x562abb7f32e0 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.306+0000 7fc323fff640 1 -- 172.21.15.18:0/2067443016 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3280054a0 con 0x562abb7f32e0 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.307+0000 7fc342994740 1 -- 172.21.15.18:0/2067443016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7f32e0 msgr2=0x562abb7f56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.307+0000 7fc342994740 1 --2- 172.21.15.18:0/2067443016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7f32e0 0x562abb7f56d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc328009930 tx=0x7fc3280047f0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.307+0000 7fc342994740 1 -- 172.21.15.18:0/2067443016 shutdown_connections 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.307+0000 7fc342994740 1 --2- 172.21.15.18:0/2067443016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7f32e0 0x562abb7f56d0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.307+0000 7fc342994740 1 --2- 172.21.15.18:0/2067443016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7e9d60 0x562abb7ea1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.307+0000 7fc342994740 1 --2- 172.21.15.18:0/2067443016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7eaad0 0x562abb7e9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.307+0000 7fc342994740 1 -- 172.21.15.18:0/2067443016 >> 172.21.15.18:0/2067443016 conn(0x562abb7b1550 msgr2=0x562abb7e4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.307+0000 7fc342994740 1 -- 172.21.15.18:0/2067443016 shutdown_connections 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.307+0000 7fc342994740 1 -- 172.21.15.18:0/2067443016 wait complete. 2024-02-17T12:56:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.307+0000 7fc342994740 1 -- start start 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.307+0000 7fc342994740 1 Processor -- start 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc342994740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7e9d60 0x562abb7f02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc342994740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 0x562abb7f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc342994740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc342994740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abb6f6bf0 con 0x562abb7eaad0 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc342994740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abb425250 con 0x562abb7f32e0 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc342994740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abb6d7200 con 0x562abb7e9d60 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc342994740 10 client.0 _ll_register_callbacks cb 0x562abb745d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc33d27a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 0x562abb7f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc33d27a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 0x562abb7f0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41842/0 (socket says 172.21.15.18:41842) 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc33d27a640 1 -- 172.21.15.18:0/3154537338 learned_addr learned my addr 172.21.15.18:0/3154537338 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc33e27c640 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc33d27a640 1 -- 172.21.15.18:0/3154537338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7e9d60 msgr2=0x562abb7f02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc33da7b640 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7e9d60 0x562abb7f02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc33d27a640 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7e9d60 0x562abb7f02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc33d27a640 1 -- 172.21.15.18:0/3154537338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 msgr2=0x562abb7f0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc33d27a640 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7f0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc33d27a640 1 -- 172.21.15.18:0/3154537338 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33810cf80 con 0x562abb7eaad0 2024-02-17T12:56:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc33e27c640 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc33d27a640 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 0x562abb7f0810 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc328006010 tx=0x7fc328005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935c172338783d76 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc3217fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc3217fa640 1 -- 172.21.15.18:0/3154537338 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32802cbf0 con 0x562abb7eaad0 2024-02-17T12:56:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc3217fa640 1 -- 172.21.15.18:0/3154537338 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc328004aa0 con 0x562abb7eaad0 2024-02-17T12:56:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc342994740 1 -- 172.21.15.18:0/3154537338 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562abb451600 con 0x562abb7eaad0 2024-02-17T12:56:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc3217fa640 1 -- 172.21.15.18:0/3154537338 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc328034440 con 0x562abb7eaad0 2024-02-17T12:56:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc342994740 10 client.18078 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.308+0000 7fc342994740 1 -- 172.21.15.18:0/3154537338 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562abb74fd80 con 0x562abb7eaad0 2024-02-17T12:56:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.309+0000 7fc3217fa640 1 -- 172.21.15.18:0/3154537338 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562abb451600 con 0x562abb7eaad0 2024-02-17T12:56:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.309+0000 7fc3217fa640 1 -- 172.21.15.18:0/3154537338 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc32803c040 con 0x562abb7eaad0 2024-02-17T12:56:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.309+0000 7fc342994740 10 client.18078 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.309+0000 7fc342994740 10 client.18078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.309+0000 7fc342994740 20 client.18078 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.309+0000 7fc342994740 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562abb803350 0x562abb805770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.309+0000 7fc342994740 4 client.18078 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.309+0000 7fc342994740 1 -- 172.21.15.18:0/3154537338 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562abb4a21b0 con 0x562abb803350 2024-02-17T12:56:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.309+0000 7fc33da7b640 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562abb803350 0x562abb805770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.309+0000 7fc3217fa640 10 client.18078 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.309+0000 7fc33da7b640 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562abb803350 0x562abb805770 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d51ba9ec41f68ef3 server_cookie=231a6cf0ab668bd in_seq=0 out_seq=0 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc3217fa640 1 -- 172.21.15.18:0/3154537338 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562abb4a21b0 con 0x562abb803350 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc3217fa640 10 client.18078 handle_command_reply: tid=0 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc342994740 1 client.18078 shutdown 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc342994740 1 -- 172.21.15.18:0/3154537338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 msgr2=0x562abb7f0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc342994740 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 0x562abb7f0810 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc328006010 tx=0x7fc328005020 comp rx=0 tx=0).stop 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc342994740 1 -- 172.21.15.18:0/3154537338 shutdown_connections 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc342994740 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562abb803350 0x562abb805770 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc342994740 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc342994740 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 0x562abb7f0810 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc342994740 1 --2- 172.21.15.18:0/3154537338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7e9d60 0x562abb7f02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc3217fa640 0 client.18078 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc342994740 1 -- 172.21.15.18:0/3154537338 >> 172.21.15.18:0/3154537338 conn(0x562abb7b1550 msgr2=0x562abb7e67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc342994740 1 -- 172.21.15.18:0/3154537338 shutdown_connections 2024-02-17T12:56:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.310+0000 7fc342994740 1 -- 172.21.15.18:0/3154537338 wait complete. 2024-02-17T12:56:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.311+0000 7fc342994740 20 client.18078 trim_cache size 0 max 16384 2024-02-17T12:56:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc342994740 1 Processor -- start 2024-02-17T12:56:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc342994740 1 -- start start 2024-02-17T12:56:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc342994740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7e9d60 0x562abb7f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc342994740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 0x562abb7f0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc342994740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7ed550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc342994740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abb451600 con 0x562abb7eaad0 2024-02-17T12:56:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc342994740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abb7ea5a0 con 0x562abb7f32e0 2024-02-17T12:56:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc342994740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abb7e5a50 con 0x562abb7e9d60 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33d27a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 0x562abb7f0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33d27a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 0x562abb7f0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41844/0 (socket says 172.21.15.18:41844) 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33d27a640 1 -- 172.21.15.18:0/2189056121 learned_addr learned my addr 172.21.15.18:0/2189056121 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33d27a640 1 -- 172.21.15.18:0/2189056121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7e9d60 msgr2=0x562abb7f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33da7b640 1 --2- 172.21.15.18:0/2189056121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7e9d60 0x562abb7f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33e27c640 1 --2- 172.21.15.18:0/2189056121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7ed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33d27a640 1 --2- 172.21.15.18:0/2189056121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7e9d60 0x562abb7f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33d27a640 1 -- 172.21.15.18:0/2189056121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 msgr2=0x562abb7ed550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33d27a640 1 --2- 172.21.15.18:0/2189056121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7ed550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33d27a640 1 -- 172.21.15.18:0/2189056121 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562abb6ec7b0 con 0x562abb7eaad0 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33da7b640 1 --2- 172.21.15.18:0/2189056121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7e9d60 0x562abb7f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33e27c640 1 --2- 172.21.15.18:0/2189056121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7ed550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:56:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc33d27a640 1 --2- 172.21.15.18:0/2189056121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 0x562abb7f0b70 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc32802c170 tx=0x7fc328005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546f568c617a4752 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.324+0000 7fc3237fe640 1 -- 172.21.15.18:0/2189056121 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328005110 con 0x562abb7eaad0 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc3237fe640 1 -- 172.21.15.18:0/2189056121 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3280048e0 con 0x562abb7eaad0 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc3237fe640 1 -- 172.21.15.18:0/2189056121 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3280597e0 con 0x562abb7eaad0 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc342994740 1 -- 172.21.15.18:0/2189056121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 msgr2=0x562abb7f0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc342994740 1 --2- 172.21.15.18:0/2189056121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 0x562abb7f0b70 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc32802c170 tx=0x7fc328005620 comp rx=0 tx=0).stop 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc342994740 1 -- 172.21.15.18:0/2189056121 shutdown_connections 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc342994740 1 --2- 172.21.15.18:0/2189056121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7ed550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc342994740 1 --2- 172.21.15.18:0/2189056121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7eaad0 0x562abb7f0b70 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc342994740 1 --2- 172.21.15.18:0/2189056121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7e9d60 0x562abb7f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc342994740 1 -- 172.21.15.18:0/2189056121 >> 172.21.15.18:0/2189056121 conn(0x562abb7b1550 msgr2=0x562abb7eeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc342994740 1 -- 172.21.15.18:0/2189056121 shutdown_connections 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc342994740 1 -- 172.21.15.18:0/2189056121 wait complete. 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc342994740 1 -- start start 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.325+0000 7fc342994740 1 Processor -- start 2024-02-17T12:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc342994740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7e9d60 0x562abb7f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc342994740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7eaad0 0x562abb7f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc342994740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc342994740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abb7fa9c0 con 0x562abb7e9d60 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc342994740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abb859820 con 0x562abb7f32e0 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc342994740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abb859960 con 0x562abb7eaad0 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc342994740 10 client.0 _ll_register_callbacks cb 0x7fc328009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc33da7b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7e9d60 0x562abb7f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc33da7b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7e9d60 0x562abb7f8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41848/0 (socket says 172.21.15.18:41848) 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc33da7b640 1 -- 172.21.15.18:0/739590536 learned_addr learned my addr 172.21.15.18:0/739590536 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc33d27a640 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7eaad0 0x562abb7f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc33da7b640 1 -- 172.21.15.18:0/739590536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7eaad0 msgr2=0x562abb7f8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc33da7b640 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7eaad0 0x562abb7f8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc33da7b640 1 -- 172.21.15.18:0/739590536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 msgr2=0x562abb7f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc33da7b640 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc33da7b640 1 -- 172.21.15.18:0/739590536 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562abb74fd80 con 0x562abb7e9d60 2024-02-17T12:56:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc33d27a640 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7eaad0 0x562abb7f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:56:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc33da7b640 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7e9d60 0x562abb7f8a20 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc334007fd0 tx=0x7fc334007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a6e260561741ba server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc3217fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.326+0000 7fc3217fa640 1 -- 172.21.15.18:0/739590536 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3340a3370 con 0x562abb7e9d60 2024-02-17T12:56:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc3217fa640 1 -- 172.21.15.18:0/739590536 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc3340cbbb0 con 0x562abb7e9d60 2024-02-17T12:56:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc342994740 1 -- 172.21.15.18:0/739590536 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc328004510 con 0x562abb7e9d60 2024-02-17T12:56:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc3217fa640 1 -- 172.21.15.18:0/739590536 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3340cbd70 con 0x562abb7e9d60 2024-02-17T12:56:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc342994740 10 client.18084 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc342994740 1 -- 172.21.15.18:0/739590536 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562abb712540 con 0x562abb7e9d60 2024-02-17T12:56:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc3217fa640 1 -- 172.21.15.18:0/739590536 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3340b9030 con 0x562abb7e9d60 2024-02-17T12:56:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc3217fa640 1 -- 172.21.15.18:0/739590536 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc334016020 con 0x562abb7e9d60 2024-02-17T12:56:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc342994740 10 client.18084 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc342994740 10 client.18084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc342994740 20 client.18084 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc342994740 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562abb85b100 0x562abb85d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc342994740 4 client.18084 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:56:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc342994740 1 -- 172.21.15.18:0/739590536 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562abb48e550 con 0x562abb85b100 2024-02-17T12:56:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.327+0000 7fc33d27a640 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562abb85b100 0x562abb85d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.328+0000 7fc3217fa640 10 client.18084 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.328+0000 7fc33d27a640 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562abb85b100 0x562abb85d520 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303246e728d16882 server_cookie=a857c49f12feecd4 in_seq=0 out_seq=0 2024-02-17T12:56:48.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:47 smithi119 ceph-mon[36188]: pgmap v213: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-02-17T12:56:48.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:47 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3696564147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:56:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.362+0000 7fc3217fa640 1 -- 172.21.15.18:0/739590536 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562abb48e550 con 0x562abb85b100 2024-02-17T12:56:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.362+0000 7fc3217fa640 10 client.18084 handle_command_reply: tid=0 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.362+0000 7fc342994740 1 client.18084 shutdown 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.363+0000 7fc342994740 1 -- 172.21.15.18:0/739590536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7e9d60 msgr2=0x562abb7f8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.363+0000 7fc342994740 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7e9d60 0x562abb7f8a20 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc334007fd0 tx=0x7fc334007f40 comp rx=0 tx=0).stop 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.363+0000 7fc342994740 1 -- 172.21.15.18:0/739590536 shutdown_connections 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.363+0000 7fc342994740 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562abb85b100 0x562abb85d520 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.363+0000 7fc342994740 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abb7f32e0 0x562abb7f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.363+0000 7fc342994740 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562abb7eaad0 0x562abb7f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.363+0000 7fc3217fa640 0 client.18084 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.363+0000 7fc342994740 1 --2- 172.21.15.18:0/739590536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562abb7e9d60 0x562abb7f8a20 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.363+0000 7fc342994740 1 -- 172.21.15.18:0/739590536 >> 172.21.15.18:0/739590536 conn(0x562abb7b1550 msgr2=0x562abb7e5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.363+0000 7fc342994740 1 -- 172.21.15.18:0/739590536 shutdown_connections 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.363+0000 7fc342994740 1 -- 172.21.15.18:0/739590536 wait complete. 2024-02-17T12:56:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.363+0000 7fc342994740 20 client.18084 trim_cache size 0 max 16384 2024-02-17T12:56:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:56:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:56:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:56:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:56:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.364+0000 7fc33fcdf640 1 -- 172.21.15.18:0/4143137582 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc3240a22d0 msgr2=0x7fc3240a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.364+0000 7fc33fcdf640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc3240a22d0 0x7fc3240a4790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc330013a00 tx=0x7fc330005e00 comp rx=0 tx=0).stop 2024-02-17T12:56:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.364+0000 7fc33fcdf640 1 -- 172.21.15.18:0/4143137582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3380ff7b0 msgr2=0x7fc338197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.364+0000 7fc33fcdf640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3380ff7b0 0x7fc338197700 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc33400c920 tx=0x7fc33400cde0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.364+0000 7fc33fcdf640 1 -- 172.21.15.18:0/4143137582 shutdown_connections 2024-02-17T12:56:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.364+0000 7fc33fcdf640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc3240a22d0 0x7fc3240a4790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.364+0000 7fc33fcdf640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3381057c0 0x7fc33819bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.364+0000 7fc33fcdf640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc338100170 0x7fc338197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.364+0000 7fc33fcdf640 1 --2- 172.21.15.18:0/4143137582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3380ff7b0 0x7fc338197700 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.364+0000 7fc33fcdf640 1 -- 172.21.15.18:0/4143137582 >> 172.21.15.18:0/4143137582 conn(0x7fc3380765a0 msgr2=0x7fc338106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.364+0000 7fc33fcdf640 1 -- 172.21.15.18:0/4143137582 shutdown_connections 2024-02-17T12:56:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:56:48.364+0000 7fc33fcdf640 1 -- 172.21.15.18:0/4143137582 wait complete. 2024-02-17T12:56:48.857 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:56:48.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:56:48.858 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:56:48.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.977+0000 7f76c8055640 1 -- 172.21.15.18:0/3194349205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c01002a0 msgr2=0x7f76c0100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.977+0000 7f76c8055640 1 --2- 172.21.15.18:0/3194349205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c01002a0 0x7f76c0100680 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009930 tx=0x7f76bc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:48.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.977+0000 7f76c8055640 1 -- 172.21.15.18:0/3194349205 shutdown_connections 2024-02-17T12:56:48.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.977+0000 7f76c8055640 1 --2- 172.21.15.18:0/3194349205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c0105370 0x7f76c010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.977+0000 7f76c8055640 1 --2- 172.21.15.18:0/3194349205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76c0100c50 0x7f76c0104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.977+0000 7f76c8055640 1 --2- 172.21.15.18:0/3194349205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c01002a0 0x7f76c0100680 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.977+0000 7f76c8055640 1 -- 172.21.15.18:0/3194349205 >> 172.21.15.18:0/3194349205 conn(0x7f76c0076400 msgr2=0x7f76c0076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:48.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c8055640 1 -- 172.21.15.18:0/3194349205 shutdown_connections 2024-02-17T12:56:48.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c8055640 1 -- 172.21.15.18:0/3194349205 wait complete. 2024-02-17T12:56:48.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c8055640 1 Processor -- start 2024-02-17T12:56:48.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c8055640 1 -- start start 2024-02-17T12:56:48.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c8055640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76c01002a0 0x7f76c019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c8055640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0105370 0x7f76c019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c8055640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c0197670 0x7f76c0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c8055640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c0198140 con 0x7f76c0105370 2024-02-17T12:56:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c8055640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c01982b0 con 0x7f76c0197670 2024-02-17T12:56:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c8055640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c0198470 con 0x7f76c01002a0 2024-02-17T12:56:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c55f0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0105370 0x7f76c019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c55f0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0105370 0x7f76c019daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41874/0 (socket says 172.21.15.18:41874) 2024-02-17T12:56:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c5df1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76c01002a0 0x7f76c019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c55f0640 1 -- 172.21.15.18:0/2534480354 learned_addr learned my addr 172.21.15.18:0/2534480354 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.978+0000 7f76c65f2640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c0197670 0x7f76c0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.979+0000 7f76c55f0640 1 -- 172.21.15.18:0/2534480354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76c01002a0 msgr2=0x7f76c019d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.979+0000 7f76c55f0640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76c01002a0 0x7f76c019d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.979+0000 7f76c55f0640 1 -- 172.21.15.18:0/2534480354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c0197670 msgr2=0x7f76c0197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.979+0000 7f76c55f0640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c0197670 0x7f76c0197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.979+0000 7f76c55f0640 1 -- 172.21.15.18:0/2534480354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc009560 con 0x7f76c0105370 2024-02-17T12:56:48.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.979+0000 7f76c55f0640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0105370 0x7f76c019daf0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f76b400e9f0 tx=0x7f76b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:48.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.979+0000 7f76b2ffd640 1 -- 172.21.15.18:0/2534480354 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b40096c0 con 0x7f76c0105370 2024-02-17T12:56:48.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.979+0000 7f76b2ffd640 1 -- 172.21.15.18:0/2534480354 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76b4005500 con 0x7f76c0105370 2024-02-17T12:56:48.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.979+0000 7f76c8055640 1 -- 172.21.15.18:0/2534480354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c01a4410 con 0x7f76c0105370 2024-02-17T12:56:48.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.979+0000 7f76b2ffd640 1 -- 172.21.15.18:0/2534480354 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4010430 con 0x7f76c0105370 2024-02-17T12:56:48.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.979+0000 7f76c8055640 1 -- 172.21.15.18:0/2534480354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c01a48a0 con 0x7f76c0105370 2024-02-17T12:56:48.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.981+0000 7f76b2ffd640 1 -- 172.21.15.18:0/2534480354 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f76b4005020 con 0x7f76c0105370 2024-02-17T12:56:48.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76cac2e740 1 Processor -- start 2024-02-17T12:56:48.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76b2ffd640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f76ac0a2230 0x7f76ac0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76c65f2640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f76ac0a2230 0x7f76ac0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76b2ffd640 1 -- 172.21.15.18:0/2534480354 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f76b4014070 con 0x7f76c0105370 2024-02-17T12:56:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76cac2e740 1 -- start start 2024-02-17T12:56:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76cac2e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4380 0x562467ca4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76cac2e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4d30 0x562467cad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76cac2e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467cad8b0 0x562467cafd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76cac2e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562467c12f90 con 0x562467cad8b0 2024-02-17T12:56:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76cac2e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562467c104c0 con 0x562467ca4380 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76cac2e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562467c0bca0 con 0x562467ca4d30 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76c5df1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4380 0x562467ca4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76c55f0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4d30 0x562467cad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76c5df1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4380 0x562467ca4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:35970/0 (socket says 172.21.15.18:35970) 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76c5df1640 1 -- 172.21.15.18:0/1672112421 learned_addr learned my addr 172.21.15.18:0/1672112421 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76c65f2640 1 --2- 172.21.15.18:0/1672112421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467cad8b0 0x562467cafd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76c65f2640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f76ac0a2230 0x7f76ac0a46f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f76b8005fd0 tx=0x7f76b8009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76c5df1640 1 -- 172.21.15.18:0/1672112421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4d30 msgr2=0x562467cad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76c5df1640 1 --2- 172.21.15.18:0/1672112421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4d30 0x562467cad2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76c5df1640 1 -- 172.21.15.18:0/1672112421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467cad8b0 msgr2=0x562467cafd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76c5df1640 1 --2- 172.21.15.18:0/1672112421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467cad8b0 0x562467cafd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.982+0000 7f76c5df1640 1 -- 172.21.15.18:0/1672112421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562467bce0b0 con 0x562467ca4380 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.983+0000 7f76c5df1640 1 --2- 172.21.15.18:0/1672112421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4380 0x562467ca4760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f76bc005ab0 tx=0x7f76bc02c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d4b119001b17bfa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.983+0000 7f76abfff640 1 -- 172.21.15.18:0/1672112421 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc004840 con 0x562467ca4380 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.983+0000 7f76abfff640 1 -- 172.21.15.18:0/1672112421 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76bc0049a0 con 0x562467ca4380 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.983+0000 7f76abfff640 1 -- 172.21.15.18:0/1672112421 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc005170 con 0x562467ca4380 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.984+0000 7f76cac2e740 1 -- 172.21.15.18:0/1672112421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4380 msgr2=0x562467ca4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.984+0000 7f76cac2e740 1 --2- 172.21.15.18:0/1672112421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4380 0x562467ca4760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f76bc005ab0 tx=0x7f76bc02c6e0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.984+0000 7f76cac2e740 1 -- 172.21.15.18:0/1672112421 shutdown_connections 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.984+0000 7f76cac2e740 1 --2- 172.21.15.18:0/1672112421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467cad8b0 0x562467cafd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.984+0000 7f76cac2e740 1 --2- 172.21.15.18:0/1672112421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4d30 0x562467cad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.984+0000 7f76cac2e740 1 --2- 172.21.15.18:0/1672112421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4380 0x562467ca4760 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.984+0000 7f76cac2e740 1 -- 172.21.15.18:0/1672112421 >> 172.21.15.18:0/1672112421 conn(0x562467c9e840 msgr2=0x562467c9ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.984+0000 7f76cac2e740 1 -- 172.21.15.18:0/1672112421 shutdown_connections 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.984+0000 7f76cac2e740 1 -- 172.21.15.18:0/1672112421 wait complete. 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.984+0000 7f76cac2e740 1 -- start start 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.984+0000 7f76cac2e740 1 Processor -- start 2024-02-17T12:56:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76cac2e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 0x562467caa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76cac2e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4d30 0x562467caac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76cac2e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467cad8b0 0x562467cab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76cac2e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562467bb0f40 con 0x562467ca4380 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76cac2e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624678df4b0 con 0x562467cad8b0 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76cac2e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562467b91670 con 0x562467ca4d30 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76cac2e740 10 client.0 _ll_register_callbacks cb 0x562467bfef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76c5df1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 0x562467caa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76c5df1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 0x562467caa740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41892/0 (socket says 172.21.15.18:41892) 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76c5df1640 1 -- 172.21.15.18:0/80632718 learned_addr learned my addr 172.21.15.18:0/80632718 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76c65f2640 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467cad8b0 0x562467cab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76c55f0640 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4d30 0x562467caac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76c5df1640 1 -- 172.21.15.18:0/80632718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4d30 msgr2=0x562467caac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76c5df1640 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4d30 0x562467caac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76c5df1640 1 -- 172.21.15.18:0/80632718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467cad8b0 msgr2=0x562467cab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76c5df1640 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467cad8b0 0x562467cab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76c5df1640 1 -- 172.21.15.18:0/80632718 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c010d750 con 0x562467ca4380 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76c5df1640 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 0x562467caa740 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f76bc004650 tx=0x7f76bc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f9c198f246ae3a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.985+0000 7f76a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76a97fa640 1 -- 172.21.15.18:0/80632718 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0050f0 con 0x562467ca4380 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76a97fa640 1 -- 172.21.15.18:0/80632718 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f76bc005250 con 0x562467ca4380 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76cac2e740 1 -- 172.21.15.18:0/80632718 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562467be5f40 con 0x562467ca4380 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76a97fa640 1 -- 172.21.15.18:0/80632718 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76bc005410 con 0x562467ca4380 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76cac2e740 10 client.18096 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76cac2e740 1 -- 172.21.15.18:0/80632718 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562467c0a320 con 0x562467ca4380 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76a97fa640 1 -- 172.21.15.18:0/80632718 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562467be5f40 con 0x562467ca4380 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76a97fa640 1 -- 172.21.15.18:0/80632718 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f76bc03c050 con 0x562467ca4380 2024-02-17T12:56:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76cac2e740 10 client.18096 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76cac2e740 10 client.18096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76cac2e740 20 client.18096 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76cac2e740 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562467cbda70 0x562467cbfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76cac2e740 4 client.18096 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76cac2e740 1 -- 172.21.15.18:0/80632718 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562467c02310 con 0x562467cbda70 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.986+0000 7f76c55f0640 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562467cbda70 0x562467cbfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76c55f0640 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562467cbda70 0x562467cbfe90 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec00940c83f0f554 server_cookie=1a3b967413373617 in_seq=0 out_seq=0 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76a97fa640 10 client.18096 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76a97fa640 1 -- 172.21.15.18:0/80632718 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562467c02310 con 0x562467cbda70 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76a97fa640 10 client.18096 handle_command_reply: tid=0 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76cac2e740 1 client.18096 shutdown 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76cac2e740 1 -- 172.21.15.18:0/80632718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 msgr2=0x562467caa740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76cac2e740 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 0x562467caa740 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f76bc004650 tx=0x7f76bc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76cac2e740 1 -- 172.21.15.18:0/80632718 shutdown_connections 2024-02-17T12:56:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76cac2e740 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562467cbda70 0x562467cbfe90 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76cac2e740 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467cad8b0 0x562467cab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76cac2e740 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4d30 0x562467caac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76a97fa640 0 client.18096 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76cac2e740 1 --2- 172.21.15.18:0/80632718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 0x562467caa740 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.987+0000 7f76cac2e740 1 -- 172.21.15.18:0/80632718 >> 172.21.15.18:0/80632718 conn(0x562467c9e840 msgr2=0x562467ca07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.988+0000 7f76cac2e740 1 -- 172.21.15.18:0/80632718 shutdown_connections 2024-02-17T12:56:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.988+0000 7f76cac2e740 1 -- 172.21.15.18:0/80632718 wait complete. 2024-02-17T12:56:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:48.988+0000 7f76cac2e740 20 client.18096 trim_cache size 0 max 16384 2024-02-17T12:56:48.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.003+0000 7f76cac2e740 1 Processor -- start 2024-02-17T12:56:48.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.003+0000 7f76cac2e740 1 -- start start 2024-02-17T12:56:48.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76cac2e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4380 0x562467cace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76cac2e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4d30 0x562467ca9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76cac2e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467cad8b0 0x562467caafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76cac2e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562467c9e6f0 con 0x562467cad8b0 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76cac2e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562467c9a140 con 0x562467ca4d30 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76cac2e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562467ca8cb0 con 0x562467ca4380 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76c65f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467cad8b0 0x562467caafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76c65f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467cad8b0 0x562467caafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41900/0 (socket says 172.21.15.18:41900) 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76c65f2640 1 -- 172.21.15.18:0/184307034 learned_addr learned my addr 172.21.15.18:0/184307034 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76c55f0640 1 --2- 172.21.15.18:0/184307034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4d30 0x562467ca9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76c5df1640 1 --2- 172.21.15.18:0/184307034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4380 0x562467cace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76c65f2640 1 -- 172.21.15.18:0/184307034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4380 msgr2=0x562467cace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76c65f2640 1 --2- 172.21.15.18:0/184307034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4380 0x562467cace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76c65f2640 1 -- 172.21.15.18:0/184307034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4d30 msgr2=0x562467ca9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76c65f2640 1 --2- 172.21.15.18:0/184307034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4d30 0x562467ca9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76c65f2640 1 -- 172.21.15.18:0/184307034 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562467ba6c00 con 0x562467cad8b0 2024-02-17T12:56:48.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76c55f0640 1 --2- 172.21.15.18:0/184307034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4d30 0x562467ca9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76c65f2640 1 --2- 172.21.15.18:0/184307034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467cad8b0 0x562467caafe0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f76b8003ca0 tx=0x7f76b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b08e2d69ca75d99 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76ab7fe640 1 -- 172.21.15.18:0/184307034 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8024070 con 0x562467cad8b0 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76ab7fe640 1 -- 172.21.15.18:0/184307034 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76b8003cd0 con 0x562467cad8b0 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.004+0000 7f76ab7fe640 1 -- 172.21.15.18:0/184307034 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b801f800 con 0x562467cad8b0 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.005+0000 7f76cac2e740 1 -- 172.21.15.18:0/184307034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467cad8b0 msgr2=0x562467caafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.005+0000 7f76cac2e740 1 --2- 172.21.15.18:0/184307034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467cad8b0 0x562467caafe0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f76b8003ca0 tx=0x7f76b8014000 comp rx=0 tx=0).stop 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.005+0000 7f76cac2e740 1 -- 172.21.15.18:0/184307034 shutdown_connections 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.005+0000 7f76cac2e740 1 --2- 172.21.15.18:0/184307034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467cad8b0 0x562467caafe0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.005+0000 7f76cac2e740 1 --2- 172.21.15.18:0/184307034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4d30 0x562467ca9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.005+0000 7f76cac2e740 1 --2- 172.21.15.18:0/184307034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467ca4380 0x562467cace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.005+0000 7f76cac2e740 1 -- 172.21.15.18:0/184307034 >> 172.21.15.18:0/184307034 conn(0x562467c9e840 msgr2=0x562467ca8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.005+0000 7f76cac2e740 1 -- 172.21.15.18:0/184307034 shutdown_connections 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.005+0000 7f76cac2e740 1 -- 172.21.15.18:0/184307034 wait complete. 2024-02-17T12:56:49.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.005+0000 7f76cac2e740 1 -- start start 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.005+0000 7f76cac2e740 1 Processor -- start 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76cac2e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 0x562467d0d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76cac2e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4d30 0x562467d0d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76cac2e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467cad8b0 0x562467cb3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76cac2e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562467cb4220 con 0x562467ca4380 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76cac2e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562467cb4390 con 0x562467ca4d30 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76cac2e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562467cb4550 con 0x562467cad8b0 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76c5df1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 0x562467d0d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76cac2e740 10 client.0 _ll_register_callbacks cb 0x7f76b4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76c5df1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 0x562467d0d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41908/0 (socket says 172.21.15.18:41908) 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76c5df1640 1 -- 172.21.15.18:0/493720315 learned_addr learned my addr 172.21.15.18:0/493720315 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76c55f0640 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4d30 0x562467d0d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76c65f2640 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467cad8b0 0x562467cb3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76c5df1640 1 -- 172.21.15.18:0/493720315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467cad8b0 msgr2=0x562467cb3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76c5df1640 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467cad8b0 0x562467cb3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76c5df1640 1 -- 172.21.15.18:0/493720315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4d30 msgr2=0x562467d0d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76c5df1640 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4d30 0x562467d0d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76c5df1640 1 -- 172.21.15.18:0/493720315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c01a48a0 con 0x562467ca4380 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76c55f0640 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4d30 0x562467d0d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76c5df1640 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 0x562467d0d210 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f76bc005d00 tx=0x7f76bc0055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c5f4ea59904463 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.006+0000 7f76a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76a97fa640 1 -- 172.21.15.18:0/493720315 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc02c9f0 con 0x562467ca4380 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76a97fa640 1 -- 172.21.15.18:0/493720315 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f76bc000e40 con 0x562467ca4380 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76cac2e740 1 -- 172.21.15.18:0/493720315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562467c31900 con 0x562467ca4380 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76a97fa640 1 -- 172.21.15.18:0/493720315 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76bc0057b0 con 0x562467ca4380 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76cac2e740 10 client.18102 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76cac2e740 1 -- 172.21.15.18:0/493720315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f76b8017070 con 0x562467ca4380 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76a97fa640 1 -- 172.21.15.18:0/493720315 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562467c31900 con 0x562467ca4380 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76a97fa640 1 -- 172.21.15.18:0/493720315 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f76bc03c070 con 0x562467ca4380 2024-02-17T12:56:49.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76cac2e740 10 client.18102 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:49.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76cac2e740 10 client.18102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:49.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76cac2e740 20 client.18102 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:49.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76cac2e740 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562467d15f50 0x562467d18370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76cac2e740 4 client.18102 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:56:49.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76cac2e740 1 -- 172.21.15.18:0/493720315 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56246790b860 con 0x562467d15f50 2024-02-17T12:56:49.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.007+0000 7f76c55f0640 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562467d15f50 0x562467d18370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.008+0000 7f76a97fa640 10 client.18102 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:49.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.008+0000 7f76c55f0640 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562467d15f50 0x562467d18370 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58bcd26d46443dc6 server_cookie=16b2a434f9aaef41 in_seq=0 out_seq=0 2024-02-17T12:56:49.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76a97fa640 1 -- 172.21.15.18:0/493720315 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56246790b860 con 0x562467d15f50 2024-02-17T12:56:49.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76a97fa640 10 client.18102 handle_command_reply: tid=0 2024-02-17T12:56:49.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76cac2e740 1 client.18102 shutdown 2024-02-17T12:56:49.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76cac2e740 1 -- 172.21.15.18:0/493720315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 msgr2=0x562467d0d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76cac2e740 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 0x562467d0d210 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f76bc005d00 tx=0x7f76bc0055c0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76cac2e740 1 -- 172.21.15.18:0/493720315 shutdown_connections 2024-02-17T12:56:49.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76cac2e740 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562467d15f50 0x562467d18370 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76cac2e740 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562467cad8b0 0x562467cb3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76cac2e740 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562467ca4d30 0x562467d0d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76a97fa640 0 client.18102 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:49.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76cac2e740 1 --2- 172.21.15.18:0/493720315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562467ca4380 0x562467d0d210 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76cac2e740 1 -- 172.21.15.18:0/493720315 >> 172.21.15.18:0/493720315 conn(0x562467c9e840 msgr2=0x562467c9d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:49.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76cac2e740 1 -- 172.21.15.18:0/493720315 shutdown_connections 2024-02-17T12:56:49.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.009+0000 7f76cac2e740 1 -- 172.21.15.18:0/493720315 wait complete. 2024-02-17T12:56:49.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76cac2e740 20 client.18102 trim_cache size 0 max 16384 2024-02-17T12:56:49.005 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:56:49.006 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:56:49.006 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "2daabae1-b53f-4dad-af54-f261f12172ee", 2024-02-17T12:56:49.006 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:56:49.006 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76c8055640 1 -- 172.21.15.18:0/2534480354 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f76ac0a2230 msgr2=0x7f76ac0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76c8055640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f76ac0a2230 0x7f76ac0a46f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f76b8005fd0 tx=0x7f76b8009500 comp rx=0 tx=0).stop 2024-02-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76c8055640 1 -- 172.21.15.18:0/2534480354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0105370 msgr2=0x7f76c019daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76c8055640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0105370 0x7f76c019daf0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f76b400e9f0 tx=0x7f76b400eeb0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76c8055640 1 -- 172.21.15.18:0/2534480354 shutdown_connections 2024-02-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76c8055640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f76ac0a2230 0x7f76ac0a46f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76c8055640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76c0197670 0x7f76c0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76c8055640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0105370 0x7f76c019daf0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76c8055640 1 --2- 172.21.15.18:0/2534480354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76c01002a0 0x7f76c019d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76c8055640 1 -- 172.21.15.18:0/2534480354 >> 172.21.15.18:0/2534480354 conn(0x7f76c0076400 msgr2=0x7f76c0069a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76c8055640 1 -- 172.21.15.18:0/2534480354 shutdown_connections 2024-02-17T12:56:49.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.010+0000 7f76c8055640 1 -- 172.21.15.18:0/2534480354 wait complete. 2024-02-17T12:56:49.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:56:49.014 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:56:49.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.135+0000 7f0393748640 1 -- 172.21.15.18:0/361207037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f038c0ff680 msgr2=0x7f038c0ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.135+0000 7f0393748640 1 --2- 172.21.15.18:0/361207037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f038c0ff680 0x7f038c0ffb00 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f0380009930 tx=0x7f038002bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.135+0000 7f0393748640 1 -- 172.21.15.18:0/361207037 shutdown_connections 2024-02-17T12:56:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.135+0000 7f0393748640 1 --2- 172.21.15.18:0/361207037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f038c100040 0x7f038c10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.135+0000 7f0393748640 1 --2- 172.21.15.18:0/361207037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f038c0ff680 0x7f038c0ffb00 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.135+0000 7f0393748640 1 --2- 172.21.15.18:0/361207037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f038c106160 0x7f038c106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.135+0000 7f0393748640 1 -- 172.21.15.18:0/361207037 >> 172.21.15.18:0/361207037 conn(0x7f038c0765a0 msgr2=0x7f038c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.135+0000 7f0393748640 1 -- 172.21.15.18:0/361207037 shutdown_connections 2024-02-17T12:56:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.135+0000 7f0393748640 1 -- 172.21.15.18:0/361207037 wait complete. 2024-02-17T12:56:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0393748640 1 Processor -- start 2024-02-17T12:56:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0393748640 1 -- start start 2024-02-17T12:56:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0393748640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f038c0ff680 0x7f038c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0393748640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f038c100040 0x7f038c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0393748640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f038c106160 0x7f038c1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0393748640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c197e40 con 0x7f038c100040 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0393748640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c197fb0 con 0x7f038c106160 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0393748640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c198170 con 0x7f038c0ff680 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0390ce3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f038c100040 0x7f038c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0390ce3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f038c100040 0x7f038c19dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41932/0 (socket says 172.21.15.18:41932) 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0390ce3640 1 -- 172.21.15.18:0/2276950382 learned_addr learned my addr 172.21.15.18:0/2276950382 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0391ce5640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f038c106160 0x7f038c1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f03914e4640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f038c0ff680 0x7f038c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0390ce3640 1 -- 172.21.15.18:0/2276950382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f038c0ff680 msgr2=0x7f038c19d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0390ce3640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f038c0ff680 0x7f038c19d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0390ce3640 1 -- 172.21.15.18:0/2276950382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f038c106160 msgr2=0x7f038c1976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0390ce3640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f038c106160 0x7f038c1976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.136+0000 7f0390ce3640 1 -- 172.21.15.18:0/2276950382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0380009560 con 0x7f038c100040 2024-02-17T12:56:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.137+0000 7f0390ce3640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f038c100040 0x7f038c19dae0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0380009230 tx=0x7f038002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:49.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.137+0000 7f038a7fc640 1 -- 172.21.15.18:0/2276950382 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380004350 con 0x7f038c100040 2024-02-17T12:56:49.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.137+0000 7f038a7fc640 1 -- 172.21.15.18:0/2276950382 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03800044b0 con 0x7f038c100040 2024-02-17T12:56:49.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.137+0000 7f0393748640 1 -- 172.21.15.18:0/2276950382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c1983f0 con 0x7f038c100040 2024-02-17T12:56:49.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.137+0000 7f038a7fc640 1 -- 172.21.15.18:0/2276950382 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380003d00 con 0x7f038c100040 2024-02-17T12:56:49.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.137+0000 7f0393748640 1 -- 172.21.15.18:0/2276950382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038c1a4520 con 0x7f038c100040 2024-02-17T12:56:49.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f038a7fc640 1 -- 172.21.15.18:0/2276950382 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0380004df0 con 0x7f038c100040 2024-02-17T12:56:49.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f03963f6740 1 Processor -- start 2024-02-17T12:56:49.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f03963f6740 1 -- start start 2024-02-17T12:56:49.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f03963f6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f652b0 0x561809f65690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f03963f6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 0x561809f6e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f03963f6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f6e7b0 0x561809f70ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f03963f6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809ed4280 con 0x561809f652b0 2024-02-17T12:56:49.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f03963f6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809ed17b0 con 0x561809f6e7b0 2024-02-17T12:56:49.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f03963f6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809eccf90 con 0x561809f65bd0 2024-02-17T12:56:49.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f038a7fc640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03740a2230 0x7f03740a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f03914e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 0x561809f6e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f0391ce5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f652b0 0x561809f65690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f03914e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 0x561809f6e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:57810/0 (socket says 172.21.15.18:57810) 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f03914e4640 1 -- 172.21.15.18:0/752701151 learned_addr learned my addr 172.21.15.18:0/752701151 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.140+0000 7f0390ce3640 1 --2- 172.21.15.18:0/752701151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f6e7b0 0x561809f70ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.141+0000 7f0391ce5640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03740a2230 0x7f03740a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.141+0000 7f038a7fc640 1 -- 172.21.15.18:0/2276950382 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f03800eff50 con 0x7f038c100040 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.141+0000 7f03914e4640 1 -- 172.21.15.18:0/752701151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f6e7b0 msgr2=0x561809f70ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.141+0000 7f03914e4640 1 --2- 172.21.15.18:0/752701151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f6e7b0 0x561809f70ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.141+0000 7f03914e4640 1 -- 172.21.15.18:0/752701151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f652b0 msgr2=0x561809f65690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.141+0000 7f03914e4640 1 --2- 172.21.15.18:0/752701151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f652b0 0x561809f65690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.141+0000 7f03914e4640 1 -- 172.21.15.18:0/752701151 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561809e8b570 con 0x561809f65bd0 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.141+0000 7f03914e4640 1 --2- 172.21.15.18:0/752701151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 0x561809f6e270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f037c0114f0 tx=0x7f037c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff86bf0e50bb7def server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.141+0000 7f0391ce5640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03740a2230 0x7f03740a46f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03840058c0 tx=0x7f0384005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.141+0000 7f03737fe640 1 -- 172.21.15.18:0/752701151 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c004d20 con 0x561809f65bd0 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.141+0000 7f03737fe640 1 -- 172.21.15.18:0/752701151 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f037c004e80 con 0x561809f65bd0 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.141+0000 7f03737fe640 1 -- 172.21.15.18:0/752701151 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00f7c0 con 0x561809f65bd0 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.142+0000 7f03963f6740 1 -- 172.21.15.18:0/752701151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 msgr2=0x561809f6e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.142+0000 7f03963f6740 1 --2- 172.21.15.18:0/752701151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 0x561809f6e270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f037c0114f0 tx=0x7f037c0119b0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.142+0000 7f03963f6740 1 -- 172.21.15.18:0/752701151 shutdown_connections 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.142+0000 7f03963f6740 1 --2- 172.21.15.18:0/752701151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f6e7b0 0x561809f70ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.142+0000 7f03963f6740 1 --2- 172.21.15.18:0/752701151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 0x561809f6e270 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.142+0000 7f03963f6740 1 --2- 172.21.15.18:0/752701151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f652b0 0x561809f65690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.142+0000 7f03963f6740 1 -- 172.21.15.18:0/752701151 >> 172.21.15.18:0/752701151 conn(0x561809f5f990 msgr2=0x561809f5fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.142+0000 7f03963f6740 1 -- 172.21.15.18:0/752701151 shutdown_connections 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.142+0000 7f03963f6740 1 -- 172.21.15.18:0/752701151 wait complete. 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.142+0000 7f03963f6740 1 -- start start 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.142+0000 7f03963f6740 1 Processor -- start 2024-02-17T12:56:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f03963f6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f652b0 0x561809f6e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f03963f6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 0x561809f6b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f03963f6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 0x561809f6bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f03963f6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809ba0430 con 0x561809f6e7b0 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f03963f6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809b5ce00 con 0x561809f652b0 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f03963f6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809e51fd0 con 0x561809f65bd0 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0391ce5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 0x561809f6bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f03963f6740 10 client.0 _ll_register_callbacks cb 0x561809ea7fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f03914e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f652b0 0x561809f6e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0390ce3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 0x561809f6b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f03914e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f652b0 0x561809f6e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:36026/0 (socket says 172.21.15.18:36026) 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0391ce5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 0x561809f6bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41952/0 (socket says 172.21.15.18:41952) 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f03914e4640 1 -- 172.21.15.18:0/2591404416 learned_addr learned my addr 172.21.15.18:0/2591404416 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0390ce3640 1 -- 172.21.15.18:0/2591404416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f652b0 msgr2=0x561809f6e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:49.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0390ce3640 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f652b0 0x561809f6e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0390ce3640 1 -- 172.21.15.18:0/2591404416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 msgr2=0x561809f6bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0390ce3640 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 0x561809f6bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0390ce3640 1 -- 172.21.15.18:0/2591404416 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037c009c80 con 0x561809f65bd0 2024-02-17T12:56:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0391ce5640 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 0x561809f6bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f03914e4640 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f652b0 0x561809f6e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0390ce3640 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 0x561809f6b580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f03800ed7d0 tx=0x7f038008c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d718139b7f07908 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0370ff9640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:56:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0370ff9640 1 -- 172.21.15.18:0/2591404416 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03800035f0 con 0x561809f65bd0 2024-02-17T12:56:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f03963f6740 1 -- 172.21.15.18:0/2591404416 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561809c09720 con 0x561809f65bd0 2024-02-17T12:56:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.143+0000 7f0370ff9640 1 -- 172.21.15.18:0/2591404416 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f03800bd170 con 0x561809f65bd0 2024-02-17T12:56:49.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.144+0000 7f0370ff9640 1 -- 172.21.15.18:0/2591404416 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03800c0bc0 con 0x561809f65bd0 2024-02-17T12:56:49.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.144+0000 7f03963f6740 10 client.27334 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:49.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.144+0000 7f03963f6740 1 -- 172.21.15.18:0/2591404416 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561809ecb610 con 0x561809f65bd0 2024-02-17T12:56:49.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.144+0000 7f0370ff9640 1 -- 172.21.15.18:0/2591404416 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03800d4080 con 0x561809f65bd0 2024-02-17T12:56:49.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.144+0000 7f0370ff9640 1 -- 172.21.15.18:0/2591404416 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f038003c070 con 0x561809f65bd0 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.144+0000 7f03963f6740 10 client.27334 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.144+0000 7f03963f6740 10 client.27334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.144+0000 7f03963f6740 20 client.27334 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.144+0000 7f03963f6740 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561809f7e6c0 0x561809f80ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.144+0000 7f03963f6740 4 client.27334 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.144+0000 7f03963f6740 1 -- 172.21.15.18:0/2591404416 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561809c58890 con 0x561809f7e6c0 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.144+0000 7f03914e4640 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561809f7e6c0 0x561809f80ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.145+0000 7f0370ff9640 10 client.27334 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.145+0000 7f03914e4640 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561809f7e6c0 0x561809f80ae0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c50d3c6abd2897 server_cookie=aa6ed9cf7d82e07e in_seq=0 out_seq=0 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.145+0000 7f0370ff9640 1 -- 172.21.15.18:0/2591404416 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561809c58890 con 0x561809f7e6c0 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.145+0000 7f0370ff9640 10 client.27334 handle_command_reply: tid=0 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.145+0000 7f03963f6740 1 client.27334 shutdown 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.146+0000 7f03963f6740 1 -- 172.21.15.18:0/2591404416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 msgr2=0x561809f6b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.146+0000 7f03963f6740 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 0x561809f6b580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f03800ed7d0 tx=0x7f038008c8d0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.146+0000 7f03963f6740 1 -- 172.21.15.18:0/2591404416 shutdown_connections 2024-02-17T12:56:49.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.146+0000 7f03963f6740 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561809f7e6c0 0x561809f80ae0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.146+0000 7f03963f6740 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 0x561809f6bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.146+0000 7f0370ff9640 0 client.27334 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:49.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.146+0000 7f03963f6740 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f65bd0 0x561809f6b580 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.146+0000 7f03963f6740 1 --2- 172.21.15.18:0/2591404416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f652b0 0x561809f6e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.146+0000 7f03963f6740 1 -- 172.21.15.18:0/2591404416 >> 172.21.15.18:0/2591404416 conn(0x561809f5f990 msgr2=0x561809f61730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:49.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.146+0000 7f03963f6740 1 -- 172.21.15.18:0/2591404416 shutdown_connections 2024-02-17T12:56:49.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.146+0000 7f03963f6740 1 -- 172.21.15.18:0/2591404416 wait complete. 2024-02-17T12:56:49.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.146+0000 7f03963f6740 20 client.27334 trim_cache size 0 max 16384 2024-02-17T12:56:49.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.161+0000 7f03963f6740 1 Processor -- start 2024-02-17T12:56:49.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03963f6740 1 -- start start 2024-02-17T12:56:49.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03963f6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f652b0 0x561809f85480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03963f6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f65bd0 0x561809f69e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03963f6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 0x561809f6a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03963f6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809f6ca10 con 0x561809f6e7b0 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03963f6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809f5b0d0 con 0x561809f65bd0 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03963f6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809f5f7f0 con 0x561809f652b0 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f0391ce5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 0x561809f6a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03914e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f652b0 0x561809f85480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f0391ce5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 0x561809f6a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41968/0 (socket says 172.21.15.18:41968) 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f0390ce3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f65bd0 0x561809f69e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f0391ce5640 1 -- 172.21.15.18:0/2433654520 learned_addr learned my addr 172.21.15.18:0/2433654520 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03914e4640 1 -- 172.21.15.18:0/2433654520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f65bd0 msgr2=0x561809f69e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03914e4640 1 --2- 172.21.15.18:0/2433654520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f65bd0 0x561809f69e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03914e4640 1 -- 172.21.15.18:0/2433654520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 msgr2=0x561809f6a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03914e4640 1 --2- 172.21.15.18:0/2433654520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 0x561809f6a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03914e4640 1 -- 172.21.15.18:0/2433654520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561809e67ac0 con 0x561809f652b0 2024-02-17T12:56:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f0391ce5640 1 --2- 172.21.15.18:0/2433654520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 0x561809f6a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:56:49.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.162+0000 7f03914e4640 1 --2- 172.21.15.18:0/2433654520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f652b0 0x561809f85480 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f037c0047a0 tx=0x7f037c013f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17ca7eb87f6164e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:49.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.163+0000 7f0372ffd640 1 -- 172.21.15.18:0/2433654520 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c029070 con 0x561809f652b0 2024-02-17T12:56:49.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.163+0000 7f0372ffd640 1 -- 172.21.15.18:0/2433654520 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f037c003ca0 con 0x561809f652b0 2024-02-17T12:56:49.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.163+0000 7f0372ffd640 1 -- 172.21.15.18:0/2433654520 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00fa20 con 0x561809f652b0 2024-02-17T12:56:49.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.163+0000 7f03963f6740 1 -- 172.21.15.18:0/2433654520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f652b0 msgr2=0x561809f85480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.163+0000 7f03963f6740 1 --2- 172.21.15.18:0/2433654520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f652b0 0x561809f85480 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f037c0047a0 tx=0x7f037c013f80 comp rx=0 tx=0).stop 2024-02-17T12:56:49.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.163+0000 7f03963f6740 1 -- 172.21.15.18:0/2433654520 shutdown_connections 2024-02-17T12:56:49.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.163+0000 7f03963f6740 1 --2- 172.21.15.18:0/2433654520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f6e7b0 0x561809f6a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.163+0000 7f03963f6740 1 --2- 172.21.15.18:0/2433654520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f65bd0 0x561809f69e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.163+0000 7f03963f6740 1 --2- 172.21.15.18:0/2433654520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f652b0 0x561809f85480 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.163+0000 7f03963f6740 1 -- 172.21.15.18:0/2433654520 >> 172.21.15.18:0/2433654520 conn(0x561809f5f990 msgr2=0x561809f668d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.163+0000 7f03963f6740 1 -- 172.21.15.18:0/2433654520 shutdown_connections 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.163+0000 7f03963f6740 1 -- 172.21.15.18:0/2433654520 wait complete. 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f03963f6740 1 -- start start 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f03963f6740 1 Processor -- start 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f03963f6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f652b0 0x561809f752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f03963f6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f65bd0 0x561809f75800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f03963f6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f6e7b0 0x561809f73920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f03963f6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809f73e90 con 0x561809f65bd0 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f03963f6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809f74000 con 0x561809f652b0 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f03963f6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809f741c0 con 0x561809f6e7b0 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f03963f6740 10 client.0 _ll_register_callbacks cb 0x7f037c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f0390ce3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f65bd0 0x561809f75800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f0390ce3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f65bd0 0x561809f75800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41976/0 (socket says 172.21.15.18:41976) 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f0390ce3640 1 -- 172.21.15.18:0/396829841 learned_addr learned my addr 172.21.15.18:0/396829841 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f0391ce5640 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f6e7b0 0x561809f73920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.164+0000 7f03914e4640 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f652b0 0x561809f752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f0390ce3640 1 -- 172.21.15.18:0/396829841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f6e7b0 msgr2=0x561809f73920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f0390ce3640 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f6e7b0 0x561809f73920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f0390ce3640 1 -- 172.21.15.18:0/396829841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f652b0 msgr2=0x561809f752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f0390ce3640 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f652b0 0x561809f752c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f0390ce3640 1 -- 172.21.15.18:0/396829841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0384010e70 con 0x561809f65bd0 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f0390ce3640 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f65bd0 0x561809f75800 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f03800edcc0 tx=0x7f038002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef14019858d5b1a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f0370ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f0370ff9640 1 -- 172.21.15.18:0/396829841 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03800b8d10 con 0x561809f65bd0 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f0370ff9640 1 -- 172.21.15.18:0/396829841 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f03800035f0 con 0x561809f65bd0 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f03963f6740 1 -- 172.21.15.18:0/396829841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561809bcc7e0 con 0x561809f65bd0 2024-02-17T12:56:49.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f0370ff9640 1 -- 172.21.15.18:0/396829841 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03800bc3d0 con 0x561809f65bd0 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f03963f6740 10 client.18120 fetch_fsmap learned FSMap version 41 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f03963f6740 1 -- 172.21.15.18:0/396829841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561809e8b570 con 0x561809f65bd0 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f0370ff9640 1 -- 172.21.15.18:0/396829841 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561809bcc7e0 con 0x561809f65bd0 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f0370ff9640 1 -- 172.21.15.18:0/396829841 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f038003c040 con 0x561809f65bd0 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f03963f6740 10 client.18120 fetch_fsmap finished waiting for FSMap version 41 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f03963f6740 10 client.18120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f03963f6740 20 client.18120 populate_metadata read hostname 'smithi018' 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f03963f6740 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561809fd5a10 0x561809fd7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f03963f6740 4 client.18120 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.165+0000 7f03963f6740 1 -- 172.21.15.18:0/396829841 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561809bbb4e0 con 0x561809fd5a10 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.166+0000 7f03914e4640 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561809fd5a10 0x561809fd7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.166+0000 7f03914e4640 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561809fd5a10 0x561809fd7e00 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa81c66ed21af5e server_cookie=84496462cb97bf35 in_seq=0 out_seq=0 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.166+0000 7f0370ff9640 10 client.18120 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:49.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.166+0000 7f0370ff9640 1 -- 172.21.15.18:0/396829841 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561809bbb4e0 con 0x561809fd5a10 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.166+0000 7f0370ff9640 10 client.18120 handle_command_reply: tid=0 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.166+0000 7f03963f6740 1 client.18120 shutdown 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.166+0000 7f03963f6740 1 -- 172.21.15.18:0/396829841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f65bd0 msgr2=0x561809f75800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.166+0000 7f03963f6740 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f65bd0 0x561809f75800 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f03800edcc0 tx=0x7f038002bc30 comp rx=0 tx=0).stop 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f03963f6740 1 -- 172.21.15.18:0/396829841 shutdown_connections 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f03963f6740 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561809fd5a10 0x561809fd7e00 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f03963f6740 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561809f6e7b0 0x561809f73920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0370ff9640 0 client.18120 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f03963f6740 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561809f65bd0 0x561809f75800 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f03963f6740 1 --2- 172.21.15.18:0/396829841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809f652b0 0x561809f752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f03963f6740 1 -- 172.21.15.18:0/396829841 >> 172.21.15.18:0/396829841 conn(0x561809f5f990 msgr2=0x561809f85480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f03963f6740 1 -- 172.21.15.18:0/396829841 shutdown_connections 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f03963f6740 1 -- 172.21.15.18:0/396829841 wait complete. 2024-02-17T12:56:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f03963f6740 20 client.18120 trim_cache size 0 max 16384 2024-02-17T12:56:49.164 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:56:49.164 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:56:49.164 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:56:49.164 INFO:teuthology.orchestra.run.smithi018.stdout: "2daabae1-b53f-4dad-af54-f261f12172ee": { 2024-02-17T12:56:49.164 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:56:49.164 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "2daabae1-b53f-4dad-af54-f261f12172ee", 2024-02-17T12:56:49.164 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:56:49.164 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:56:49.164 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:56:49.164 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:56:49.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0393748640 1 -- 172.21.15.18:0/2276950382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03740a2230 msgr2=0x7f03740a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0393748640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03740a2230 0x7f03740a46f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03840058c0 tx=0x7f0384005f90 comp rx=0 tx=0).stop 2024-02-17T12:56:49.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0393748640 1 -- 172.21.15.18:0/2276950382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f038c100040 msgr2=0x7f038c19dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:56:49.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0393748640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f038c100040 0x7f038c19dae0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0380009230 tx=0x7f038002c650 comp rx=0 tx=0).stop 2024-02-17T12:56:49.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0393748640 1 -- 172.21.15.18:0/2276950382 shutdown_connections 2024-02-17T12:56:49.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0393748640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03740a2230 0x7f03740a46f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0393748640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f038c106160 0x7f038c1976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0393748640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f038c100040 0x7f038c19dae0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0393748640 1 --2- 172.21.15.18:0/2276950382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f038c0ff680 0x7f038c19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:56:49.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0393748640 1 -- 172.21.15.18:0/2276950382 >> 172.21.15.18:0/2276950382 conn(0x7f038c0765a0 msgr2=0x7f038c104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:56:49.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0393748640 1 -- 172.21.15.18:0/2276950382 shutdown_connections 2024-02-17T12:56:49.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:56:49.167+0000 7f0393748640 1 -- 172.21.15.18:0/2276950382 wait complete. 2024-02-17T12:56:49.169 INFO:tasks.cephfs.filesystem:scrub status for tag:2daabae1-b53f-4dad-af54-f261f12172ee - {'path': '/', 'tag': '2daabae1-b53f-4dad-af54-f261f12172ee', 'options': 'recursive,force'} 2024-02-17T12:56:49.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 365409 5.74 MB/sec execute 244 sec latency 3666.997 ms 2024-02-17T12:56:50.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:49 smithi146 ceph-mon[36454]: pgmap v214: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-02-17T12:56:50.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:49 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:50.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:49 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:50.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:49 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:50.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:49 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:50.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:49 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:50.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:49 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:50.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:49 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:50.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:49 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:50.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:49 smithi018 ceph-mon[32625]: pgmap v214: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-02-17T12:56:50.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:49 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:50.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:49 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:50.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:49 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:50.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:49 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:50.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:49 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:50.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:49 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:50.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:49 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:50.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:49 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:50.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 366725 5.74 MB/sec execute 245 sec latency 52.993 ms 2024-02-17T12:56:50.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:49 smithi119 ceph-mon[36188]: pgmap v214: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-02-17T12:56:50.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:49 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:50.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:49 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:50.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:49 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:50.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:49 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:50.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:49 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:50.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:49 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:50.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:49 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:50.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:49 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:50 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:56:51.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:51.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:51.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:51.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:51.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:56:51.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:50 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:56:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:56:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 367032 5.72 MB/sec execute 246 sec latency 784.114 ms 2024-02-17T12:56:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:50 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:56:51.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:51.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:51.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:51.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:51.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:56:51.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:51.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:56:52.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:51 smithi146 ceph-mon[36454]: pgmap v215: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 800 KiB/s wr, 129 op/s 2024-02-17T12:56:52.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:51 smithi018 ceph-mon[32625]: pgmap v215: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 800 KiB/s wr, 129 op/s 2024-02-17T12:56:52.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 367032 5.70 MB/sec execute 247 sec latency 1784.167 ms 2024-02-17T12:56:52.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:51 smithi119 ceph-mon[36188]: pgmap v215: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 800 KiB/s wr, 129 op/s 2024-02-17T12:56:53.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 367032 5.68 MB/sec execute 248 sec latency 2784.300 ms 2024-02-17T12:56:54.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:53 smithi146 ceph-mon[36454]: pgmap v216: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 800 KiB/s wr, 129 op/s 2024-02-17T12:56:54.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:53 smithi018 ceph-mon[32625]: pgmap v216: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 800 KiB/s wr, 129 op/s 2024-02-17T12:56:54.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 367839 5.66 MB/sec execute 249 sec latency 3402.681 ms 2024-02-17T12:56:54.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:53 smithi119 ceph-mon[36188]: pgmap v216: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 800 KiB/s wr, 129 op/s 2024-02-17T12:56:55.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 369455 5.69 MB/sec execute 250 sec latency 20.733 ms 2024-02-17T12:56:56.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:55 smithi146 ceph-mon[36454]: pgmap v217: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-02-17T12:56:56.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:55 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:56:56.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:55 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:56:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:55 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:56:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:55 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:56:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:55 smithi018 ceph-mon[32625]: pgmap v217: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-02-17T12:56:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:55 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:56:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:55 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:56:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:55 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:56:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:55 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:56:56.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 370575 5.68 MB/sec execute 251 sec latency 260.103 ms 2024-02-17T12:56:56.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:55 smithi119 ceph-mon[36188]: pgmap v217: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-02-17T12:56:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:55 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:56:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:55 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:56:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:55 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:56:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:55 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:56:57.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:56 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:56:57.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:56 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:56:57.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:56 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:56:57.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:56 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:56:57.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 370575 5.66 MB/sec execute 252 sec latency 1260.205 ms 2024-02-17T12:56:57.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:56 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:56:57.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:56 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:56:58.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:57 smithi146 ceph-mon[36454]: pgmap v218: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-02-17T12:56:58.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:57 smithi018 ceph-mon[32625]: pgmap v218: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-02-17T12:56:58.281 INFO:tasks.workunit.client.0.smithi018.stdout: 1 371274 5.65 MB/sec execute 253 sec latency 1831.700 ms 2024-02-17T12:56:58.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:57 smithi119 ceph-mon[36188]: pgmap v218: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-02-17T12:56:59.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 372871 5.67 MB/sec execute 254 sec latency 12.244 ms 2024-02-17T12:57:00.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:56:59 smithi146 ceph-mon[36454]: pgmap v219: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-02-17T12:57:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:56:59 smithi018 ceph-mon[32625]: pgmap v219: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-02-17T12:57:00.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 374123 5.67 MB/sec execute 255 sec latency 233.242 ms 2024-02-17T12:57:00.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:56:59 smithi119 ceph-mon[36188]: pgmap v219: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-02-17T12:57:01.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 374123 5.65 MB/sec execute 256 sec latency 1233.326 ms 2024-02-17T12:57:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.460+0000 7fd3fe127640 1 -- 172.21.15.18:0/3806813835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f80ff680 msgr2=0x7fd3f80ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.460+0000 7fd3fe127640 1 --2- 172.21.15.18:0/3806813835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f80ff680 0x7fd3f80ffb00 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009930 tx=0x7fd3ec02bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.460+0000 7fd3fe127640 1 -- 172.21.15.18:0/3806813835 shutdown_connections 2024-02-17T12:57:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.460+0000 7fd3fe127640 1 --2- 172.21.15.18:0/3806813835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3f8100040 0x7fd3f810e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.460+0000 7fd3fe127640 1 --2- 172.21.15.18:0/3806813835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f80ff680 0x7fd3f80ffb00 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.460+0000 7fd3fe127640 1 --2- 172.21.15.18:0/3806813835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3f8106160 0x7fd3f8106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.460+0000 7fd3fe127640 1 -- 172.21.15.18:0/3806813835 >> 172.21.15.18:0/3806813835 conn(0x7fd3f8076640 msgr2=0x7fd3f80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.460+0000 7fd3fe127640 1 -- 172.21.15.18:0/3806813835 shutdown_connections 2024-02-17T12:57:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.460+0000 7fd3fe127640 1 -- 172.21.15.18:0/3806813835 wait complete. 2024-02-17T12:57:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3fe127640 1 Processor -- start 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3fe127640 1 -- start start 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3fe127640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3f80ff680 0x7fd3f819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3fe127640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3f8100040 0x7fd3f819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3fe127640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f8106160 0x7fd3f81975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3fe127640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8197c40 con 0x7fd3f8106160 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3fe127640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8197db0 con 0x7fd3f80ff680 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3fe127640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8197f70 con 0x7fd3f8100040 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3f7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f8106160 0x7fd3f81975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3f7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f8106160 0x7fd3f81975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60682/0 (socket says 172.21.15.18:60682) 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3f7fff640 1 -- 172.21.15.18:0/2715323841 learned_addr learned my addr 172.21.15.18:0/2715323841 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3f6ffd640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3f8100040 0x7fd3f819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.461+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3f80ff680 0x7fd3f819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.462+0000 7fd3f7fff640 1 -- 172.21.15.18:0/2715323841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3f8100040 msgr2=0x7fd3f819d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.462+0000 7fd3f7fff640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3f8100040 0x7fd3f819d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.462+0000 7fd3f7fff640 1 -- 172.21.15.18:0/2715323841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3f80ff680 msgr2=0x7fd3f819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.462+0000 7fd3f7fff640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3f80ff680 0x7fd3f819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.462+0000 7fd3f7fff640 1 -- 172.21.15.18:0/2715323841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec009560 con 0x7fd3f8106160 2024-02-17T12:57:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.462+0000 7fd3f7fff640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f8106160 0x7fd3f81975f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000eaa0 tx=0x7fd3f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.462+0000 7fd3f4ff9640 1 -- 172.21.15.18:0/2715323841 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f000cdb0 con 0x7fd3f8106160 2024-02-17T12:57:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.462+0000 7fd3f4ff9640 1 -- 172.21.15.18:0/2715323841 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3f0005020 con 0x7fd3f8106160 2024-02-17T12:57:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.462+0000 7fd3fe127640 1 -- 172.21.15.18:0/2715323841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f8198250 con 0x7fd3f8106160 2024-02-17T12:57:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.462+0000 7fd3f4ff9640 1 -- 172.21.15.18:0/2715323841 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f00116b0 con 0x7fd3f8106160 2024-02-17T12:57:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.462+0000 7fd3fe127640 1 -- 172.21.15.18:0/2715323841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f81a44e0 con 0x7fd3f8106160 2024-02-17T12:57:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.464+0000 7fd3f4ff9640 1 -- 172.21.15.18:0/2715323841 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd3f0005520 con 0x7fd3f8106160 2024-02-17T12:57:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd400dce740 1 Processor -- start 2024-02-17T12:57:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd400dce740 1 -- start start 2024-02-17T12:57:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd3f4ff9640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3e00a22d0 0x7fd3e00a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd400dce740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5cad0 0x560acab5b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd400dce740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab5bd60 0x560acab5c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd400dce740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab696f0 0x560acab6bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd400dce740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acaaca9f0 con 0x560acab5cad0 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd400dce740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acaac7f20 con 0x560acab696f0 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd400dce740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acaac3700 con 0x560acab5bd60 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd3f6ffd640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3e00a22d0 0x7fd3e00a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd3f4ff9640 1 -- 172.21.15.18:0/2715323841 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd3f00ce280 con 0x7fd3f8106160 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd3f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5cad0 0x560acab5b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd3f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5cad0 0x560acab5b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60692/0 (socket says 172.21.15.18:60692) 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd3f77fe640 1 -- 172.21.15.18:0/3058993224 learned_addr learned my addr 172.21.15.18:0/3058993224 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.465+0000 7fd3f6ffd640 1 --2- 172.21.15.18:0/3058993224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab5bd60 0x560acab5c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.466+0000 7fd3f77fe640 1 -- 172.21.15.18:0/3058993224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab5bd60 msgr2=0x560acab5c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.466+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/3058993224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab5bd60 0x560acab5c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.466+0000 7fd3f77fe640 1 -- 172.21.15.18:0/3058993224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab696f0 msgr2=0x560acab6bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.466+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/3058993224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab696f0 0x560acab6bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.466+0000 7fd3f77fe640 1 -- 172.21.15.18:0/3058993224 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560acaa597b0 con 0x560acab5cad0 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.466+0000 7fd3f6ffd640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3e00a22d0 0x7fd3e00a4790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec005ab0 tx=0x7fd3ec004b90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.466+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/3058993224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5cad0 0x560acab5b820 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800ac20 tx=0x7fd3e800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33e959836a8c7d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.466+0000 7fd3ddffb640 1 -- 172.21.15.18:0/3058993224 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800a720 con 0x560acab5cad0 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.466+0000 7fd3ddffb640 1 -- 172.21.15.18:0/3058993224 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3e8018b20 con 0x560acab5cad0 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.466+0000 7fd3ddffb640 1 -- 172.21.15.18:0/3058993224 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8010bf0 con 0x560acab5cad0 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.467+0000 7fd400dce740 1 -- 172.21.15.18:0/3058993224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5cad0 msgr2=0x560acab5b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.467+0000 7fd400dce740 1 --2- 172.21.15.18:0/3058993224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5cad0 0x560acab5b820 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800ac20 tx=0x7fd3e800f790 comp rx=0 tx=0).stop 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.467+0000 7fd400dce740 1 -- 172.21.15.18:0/3058993224 shutdown_connections 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.467+0000 7fd400dce740 1 --2- 172.21.15.18:0/3058993224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab696f0 0x560acab6bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.467+0000 7fd400dce740 1 --2- 172.21.15.18:0/3058993224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab5bd60 0x560acab5c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.467+0000 7fd400dce740 1 --2- 172.21.15.18:0/3058993224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5cad0 0x560acab5b820 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.467+0000 7fd400dce740 1 -- 172.21.15.18:0/3058993224 >> 172.21.15.18:0/3058993224 conn(0x560acab23550 msgr2=0x560acab56a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.467+0000 7fd400dce740 1 -- 172.21.15.18:0/3058993224 shutdown_connections 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.467+0000 7fd400dce740 1 -- 172.21.15.18:0/3058993224 wait complete. 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.467+0000 7fd400dce740 1 -- start start 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.467+0000 7fd400dce740 1 Processor -- start 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd400dce740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 0x560acab667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd400dce740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5cad0 0x560acab66d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd400dce740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab696f0 0x560acab67250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd400dce740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acaa68bf0 con 0x560acab5bd60 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd400dce740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aca797250 con 0x560acab5cad0 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd400dce740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acaa49200 con 0x560acab696f0 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd400dce740 10 client.0 _ll_register_callbacks cb 0x560acaab7d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 0x560acab667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 0x560acab667d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60702/0 (socket says 172.21.15.18:60702) 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3f77fe640 1 -- 172.21.15.18:0/218469539 learned_addr learned my addr 172.21.15.18:0/218469539 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3f7fff640 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab696f0 0x560acab67250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3f6ffd640 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5cad0 0x560acab66d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3f77fe640 1 -- 172.21.15.18:0/218469539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab696f0 msgr2=0x560acab67250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab696f0 0x560acab67250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3f77fe640 1 -- 172.21.15.18:0/218469539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5cad0 msgr2=0x560acab66d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5cad0 0x560acab66d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3f77fe640 1 -- 172.21.15.18:0/218469539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e800a880 con 0x560acab5bd60 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3f6ffd640 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5cad0 0x560acab66d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 0x560acab667d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8005c50 tx=0x7fd3e8010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9cd935267ffbb4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.468+0000 7fd3d6ffd640 1 -- 172.21.15.18:0/218469539 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800a720 con 0x560acab5bd60 2024-02-17T12:57:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd400dce740 1 -- 172.21.15.18:0/218469539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560aca7c3600 con 0x560acab5bd60 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd3d6ffd640 1 -- 172.21.15.18:0/218469539 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd3e8010e10 con 0x560acab5bd60 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd3d6ffd640 1 -- 172.21.15.18:0/218469539 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3e8018850 con 0x560acab5bd60 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd400dce740 10 client.18132 fetch_fsmap learned FSMap version 42 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd400dce740 1 -- 172.21.15.18:0/218469539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x560acaac1d80 con 0x560acab5bd60 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd3d6ffd640 1 -- 172.21.15.18:0/218469539 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560aca7c3600 con 0x560acab5bd60 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd3d6ffd640 1 -- 172.21.15.18:0/218469539 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 42) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd3e8004990 con 0x560acab5bd60 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd400dce740 10 client.18132 fetch_fsmap finished waiting for FSMap version 42 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd400dce740 10 client.18132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd400dce740 20 client.18132 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd400dce740 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560acab79ab0 0x560acab7bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd400dce740 4 client.18132 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.469+0000 7fd400dce740 1 -- 172.21.15.18:0/218469539 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560aca8141b0 con 0x560acab79ab0 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.470+0000 7fd3f6ffd640 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560acab79ab0 0x560acab7bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.470+0000 7fd3f6ffd640 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560acab79ab0 0x560acab7bed0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b40f087dbdb3b44f server_cookie=1a89815308ed37dd in_seq=0 out_seq=0 2024-02-17T12:57:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.470+0000 7fd3d6ffd640 10 client.18132 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd3d6ffd640 1 -- 172.21.15.18:0/218469539 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560aca8141b0 con 0x560acab79ab0 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd3d6ffd640 10 client.18132 handle_command_reply: tid=0 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd400dce740 1 client.18132 shutdown 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd400dce740 1 -- 172.21.15.18:0/218469539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 msgr2=0x560acab667d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd400dce740 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 0x560acab667d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8005c50 tx=0x7fd3e8010c20 comp rx=0 tx=0).stop 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd400dce740 1 -- 172.21.15.18:0/218469539 shutdown_connections 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd400dce740 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560acab79ab0 0x560acab7bed0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd400dce740 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab696f0 0x560acab67250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd400dce740 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5cad0 0x560acab66d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd3d6ffd640 0 client.18132 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd400dce740 1 --2- 172.21.15.18:0/218469539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 0x560acab667d0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd400dce740 1 -- 172.21.15.18:0/218469539 >> 172.21.15.18:0/218469539 conn(0x560acab23550 msgr2=0x560acab587a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.471+0000 7fd400dce740 1 -- 172.21.15.18:0/218469539 shutdown_connections 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.472+0000 7fd400dce740 1 -- 172.21.15.18:0/218469539 wait complete. 2024-02-17T12:57:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.472+0000 7fd400dce740 20 client.18132 trim_cache size 0 max 16384 2024-02-17T12:57:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd400dce740 1 Processor -- start 2024-02-17T12:57:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd400dce740 1 -- start start 2024-02-17T12:57:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd400dce740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 0x560acab66d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd400dce740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5cad0 0x560acab67260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd400dce740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab696f0 0x560acab650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd400dce740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aca7c3600 con 0x560acab5bd60 2024-02-17T12:57:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd400dce740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acab5d0c0 con 0x560acab5cad0 2024-02-17T12:57:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd400dce740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acab76630 con 0x560acab696f0 2024-02-17T12:57:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd3f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 0x560acab66d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd3f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 0x560acab66d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60706/0 (socket says 172.21.15.18:60706) 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd3f77fe640 1 -- 172.21.15.18:0/4020655141 learned_addr learned my addr 172.21.15.18:0/4020655141 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd3f77fe640 1 -- 172.21.15.18:0/4020655141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab696f0 msgr2=0x560acab650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/4020655141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab696f0 0x560acab650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd3f77fe640 1 -- 172.21.15.18:0/4020655141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5cad0 msgr2=0x560acab67260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/4020655141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5cad0 0x560acab67260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.485+0000 7fd3f77fe640 1 -- 172.21.15.18:0/4020655141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560acaa5e7b0 con 0x560acab5bd60 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.486+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/4020655141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 0x560acab66d20 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800a310 tx=0x7fd3e800ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced23e46dea7ba29 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.486+0000 7fd3dd7fa640 1 -- 172.21.15.18:0/4020655141 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800a720 con 0x560acab5bd60 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.486+0000 7fd3dd7fa640 1 -- 172.21.15.18:0/4020655141 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3e80244f0 con 0x560acab5bd60 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.486+0000 7fd3dd7fa640 1 -- 172.21.15.18:0/4020655141 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8005040 con 0x560acab5bd60 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.486+0000 7fd400dce740 1 -- 172.21.15.18:0/4020655141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 msgr2=0x560acab66d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.486+0000 7fd400dce740 1 --2- 172.21.15.18:0/4020655141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 0x560acab66d20 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800a310 tx=0x7fd3e800ffc0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.486+0000 7fd400dce740 1 -- 172.21.15.18:0/4020655141 shutdown_connections 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.486+0000 7fd400dce740 1 --2- 172.21.15.18:0/4020655141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab696f0 0x560acab650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.486+0000 7fd400dce740 1 --2- 172.21.15.18:0/4020655141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5cad0 0x560acab67260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.486+0000 7fd400dce740 1 --2- 172.21.15.18:0/4020655141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab5bd60 0x560acab66d20 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.486+0000 7fd400dce740 1 -- 172.21.15.18:0/4020655141 >> 172.21.15.18:0/4020655141 conn(0x560acab23550 msgr2=0x560acab61ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.486+0000 7fd400dce740 1 -- 172.21.15.18:0/4020655141 shutdown_connections 2024-02-17T12:57:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd400dce740 1 -- 172.21.15.18:0/4020655141 wait complete. 2024-02-17T12:57:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd400dce740 1 -- start start 2024-02-17T12:57:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd400dce740 1 Processor -- start 2024-02-17T12:57:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd400dce740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5bd60 0x560acab70210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd400dce740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab5cad0 0x560acab70750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd400dce740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab696f0 0x560acab6e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd400dce740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acab6ed70 con 0x560acab696f0 2024-02-17T12:57:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd400dce740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acab6eee0 con 0x560acab5bd60 2024-02-17T12:57:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd400dce740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acab6f0a0 con 0x560acab5cad0 2024-02-17T12:57:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd400dce740 10 client.0 _ll_register_callbacks cb 0x7fd3e800a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd3f7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab696f0 0x560acab6e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd3f7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab696f0 0x560acab6e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60710/0 (socket says 172.21.15.18:60710) 2024-02-17T12:57:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.487+0000 7fd3f7fff640 1 -- 172.21.15.18:0/3574269358 learned_addr learned my addr 172.21.15.18:0/3574269358 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3f6ffd640 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab5cad0 0x560acab70750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5bd60 0x560acab70210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3f7fff640 1 -- 172.21.15.18:0/3574269358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab5cad0 msgr2=0x560acab70750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3f7fff640 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab5cad0 0x560acab70750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3f7fff640 1 -- 172.21.15.18:0/3574269358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5bd60 msgr2=0x560acab70210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3f7fff640 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5bd60 0x560acab70210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3f7fff640 1 -- 172.21.15.18:0/3574269358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560acaac1d80 con 0x560acab696f0 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5bd60 0x560acab70210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3f7fff640 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab696f0 0x560acab6e7b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0097140 tx=0x7fd3f009ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fa7ef4dd60ce8a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3d6ffd640 1 -- 172.21.15.18:0/3574269358 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f00b2070 con 0x560acab696f0 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3d6ffd640 1 -- 172.21.15.18:0/3574269358 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd3f00a33d0 con 0x560acab696f0 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd400dce740 1 -- 172.21.15.18:0/3574269358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3e8018b20 con 0x560acab696f0 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3d6ffd640 1 -- 172.21.15.18:0/3574269358 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3f00b91a0 con 0x560acab696f0 2024-02-17T12:57:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd400dce740 10 client.18138 fetch_fsmap learned FSMap version 42 2024-02-17T12:57:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd400dce740 1 -- 172.21.15.18:0/3574269358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x560acaa84540 con 0x560acab696f0 2024-02-17T12:57:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3d6ffd640 1 -- 172.21.15.18:0/3574269358 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3e8018b20 con 0x560acab696f0 2024-02-17T12:57:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.488+0000 7fd3d6ffd640 1 -- 172.21.15.18:0/3574269358 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 42) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd3f0018030 con 0x560acab696f0 2024-02-17T12:57:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.489+0000 7fd400dce740 10 client.18138 fetch_fsmap finished waiting for FSMap version 42 2024-02-17T12:57:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.489+0000 7fd400dce740 10 client.18138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.489+0000 7fd400dce740 20 client.18138 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.489+0000 7fd400dce740 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560acabd1210 0x560acabd3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.489+0000 7fd400dce740 4 client.18138 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:57:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.489+0000 7fd400dce740 1 -- 172.21.15.18:0/3574269358 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560aca800550 con 0x560acabd1210 2024-02-17T12:57:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.489+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560acabd1210 0x560acabd3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.489+0000 7fd3d6ffd640 10 client.18138 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.489+0000 7fd3f77fe640 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560acabd1210 0x560acabd3630 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36ac5502fd51d73f server_cookie=26a2887f56d51ca3 in_seq=0 out_seq=0 2024-02-17T12:57:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd3d6ffd640 1 -- 172.21.15.18:0/3574269358 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560aca800550 con 0x560acabd1210 2024-02-17T12:57:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd3d6ffd640 10 client.18138 handle_command_reply: tid=0 2024-02-17T12:57:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd400dce740 1 client.18138 shutdown 2024-02-17T12:57:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd400dce740 1 -- 172.21.15.18:0/3574269358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab696f0 msgr2=0x560acab6e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd400dce740 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab696f0 0x560acab6e7b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0097140 tx=0x7fd3f009ad60 comp rx=0 tx=0).stop 2024-02-17T12:57:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd400dce740 1 -- 172.21.15.18:0/3574269358 shutdown_connections 2024-02-17T12:57:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd400dce740 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560acabd1210 0x560acabd3630 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd400dce740 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560acab696f0 0x560acab6e7b0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd400dce740 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560acab5cad0 0x560acab70750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd400dce740 1 --2- 172.21.15.18:0/3574269358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560acab5bd60 0x560acab70210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd3d6ffd640 0 client.18138 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd400dce740 1 -- 172.21.15.18:0/3574269358 >> 172.21.15.18:0/3574269358 conn(0x560acab23550 msgr2=0x560acab65000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd400dce740 1 -- 172.21.15.18:0/3574269358 shutdown_connections 2024-02-17T12:57:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd400dce740 1 -- 172.21.15.18:0/3574269358 wait complete. 2024-02-17T12:57:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd400dce740 20 client.18138 trim_cache size 0 max 16384 2024-02-17T12:57:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:57:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:57:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:57:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:57:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd3fe127640 1 -- 172.21.15.18:0/2715323841 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3e00a22d0 msgr2=0x7fd3e00a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd3fe127640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3e00a22d0 0x7fd3e00a4790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec005ab0 tx=0x7fd3ec004b90 comp rx=0 tx=0).stop 2024-02-17T12:57:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd3fe127640 1 -- 172.21.15.18:0/2715323841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f8106160 msgr2=0x7fd3f81975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.533+0000 7fd3fe127640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f8106160 0x7fd3f81975f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000eaa0 tx=0x7fd3f000ef60 comp rx=0 tx=0).stop 2024-02-17T12:57:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.534+0000 7fd3fe127640 1 -- 172.21.15.18:0/2715323841 shutdown_connections 2024-02-17T12:57:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.534+0000 7fd3fe127640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd3e00a22d0 0x7fd3e00a4790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.534+0000 7fd3fe127640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f8106160 0x7fd3f81975f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.534+0000 7fd3fe127640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3f8100040 0x7fd3f819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.534+0000 7fd3fe127640 1 --2- 172.21.15.18:0/2715323841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3f80ff680 0x7fd3f819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.534+0000 7fd3fe127640 1 -- 172.21.15.18:0/2715323841 >> 172.21.15.18:0/2715323841 conn(0x7fd3f8076640 msgr2=0x7fd3f8104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.534+0000 7fd3fe127640 1 -- 172.21.15.18:0/2715323841 shutdown_connections 2024-02-17T12:57:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:01.534+0000 7fd3fe127640 1 -- 172.21.15.18:0/2715323841 wait complete. 2024-02-17T12:57:02.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:01 smithi146 ceph-mon[36454]: pgmap v220: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 146 op/s 2024-02-17T12:57:02.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:01 smithi018 ceph-mon[32625]: pgmap v220: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 146 op/s 2024-02-17T12:57:02.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 374123 5.63 MB/sec execute 257 sec latency 2233.450 ms 2024-02-17T12:57:02.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:01 smithi119 ceph-mon[36188]: pgmap v220: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 146 op/s 2024-02-17T12:57:03.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 374123 5.60 MB/sec execute 258 sec latency 3233.486 ms 2024-02-17T12:57:04.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:03 smithi146 ceph-mon[36454]: pgmap v221: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 146 op/s 2024-02-17T12:57:04.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:03 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:57:04.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:03 smithi018 ceph-mon[32625]: pgmap v221: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 146 op/s 2024-02-17T12:57:04.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:03 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:57:04.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 374292 5.58 MB/sec execute 259 sec latency 3426.732 ms 2024-02-17T12:57:04.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:03 smithi119 ceph-mon[36188]: pgmap v221: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 146 op/s 2024-02-17T12:57:04.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:03 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:57:05.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:04 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:57:05.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:04 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:57:05.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:04 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:57:05.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:04 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:57:05.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:04 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:57:05.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:04 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:57:05.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 374718 5.57 MB/sec execute 260 sec latency 1394.308 ms 2024-02-17T12:57:05.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:04 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:57:05.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:04 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:57:05.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:04 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:57:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:05 smithi018 ceph-mon[32625]: pgmap v222: 97 pgs: 97 active+clean; 130 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 221 op/s 2024-02-17T12:57:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:05 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:57:06.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 376357 5.59 MB/sec execute 261 sec latency 20.786 ms 2024-02-17T12:57:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:05 smithi119 ceph-mon[36188]: pgmap v222: 97 pgs: 97 active+clean; 130 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 221 op/s 2024-02-17T12:57:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:05 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:57:06.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:05 smithi146 ceph-mon[36454]: pgmap v222: 97 pgs: 97 active+clean; 130 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 221 op/s 2024-02-17T12:57:06.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:05 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:57:07.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 377666 5.59 MB/sec execute 262 sec latency 150.361 ms 2024-02-17T12:57:08.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:07 smithi018 ceph-mon[32625]: pgmap v223: 97 pgs: 97 active+clean; 130 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-02-17T12:57:08.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 377666 5.57 MB/sec execute 263 sec latency 1150.454 ms 2024-02-17T12:57:08.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:07 smithi119 ceph-mon[36188]: pgmap v223: 97 pgs: 97 active+clean; 130 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-02-17T12:57:08.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:07 smithi146 ceph-mon[36454]: pgmap v223: 97 pgs: 97 active+clean; 130 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-02-17T12:57:09.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 378621 5.56 MB/sec execute 264 sec latency 1756.369 ms 2024-02-17T12:57:10.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:09 smithi018 ceph-mon[32625]: pgmap v224: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 231 op/s 2024-02-17T12:57:10.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 380117 5.58 MB/sec execute 265 sec latency 19.274 ms 2024-02-17T12:57:10.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:09 smithi119 ceph-mon[36188]: pgmap v224: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 231 op/s 2024-02-17T12:57:10.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:09 smithi146 ceph-mon[36454]: pgmap v224: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 231 op/s 2024-02-17T12:57:11.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:11 smithi018 ceph-mon[32625]: osdmap e93: 12 total, 12 up, 12 in 2024-02-17T12:57:11.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 381255 5.58 MB/sec execute 266 sec latency 255.942 ms 2024-02-17T12:57:11.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:11 smithi119 ceph-mon[36188]: osdmap e93: 12 total, 12 up, 12 in 2024-02-17T12:57:11.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:11 smithi146 ceph-mon[36454]: osdmap e93: 12 total, 12 up, 12 in 2024-02-17T12:57:12.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:12 smithi018 ceph-mon[32625]: pgmap v226: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-02-17T12:57:12.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 381255 5.56 MB/sec execute 267 sec latency 1256.035 ms 2024-02-17T12:57:12.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:12 smithi119 ceph-mon[36188]: pgmap v226: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-02-17T12:57:12.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:12 smithi146 ceph-mon[36454]: pgmap v226: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-02-17T12:57:13.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 381255 5.54 MB/sec execute 268 sec latency 2256.129 ms 2024-02-17T12:57:14.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:14 smithi018 ceph-mon[32625]: pgmap v227: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-02-17T12:57:14.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 382165 5.53 MB/sec execute 269 sec latency 2865.180 ms 2024-02-17T12:57:14.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:14 smithi119 ceph-mon[36188]: pgmap v227: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-02-17T12:57:14.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:14 smithi146 ceph-mon[36454]: pgmap v227: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-02-17T12:57:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.625+0000 7f76fb46f640 1 -- 172.21.15.18:0/1085995236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4068800 msgr2=0x7f76f4068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.625+0000 7f76fb46f640 1 --2- 172.21.15.18:0/1085995236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4068800 0x7f76f4068c40 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009930 tx=0x7f76e402bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.625+0000 7f76fb46f640 1 -- 172.21.15.18:0/1085995236 shutdown_connections 2024-02-17T12:57:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.625+0000 7f76fb46f640 1 --2- 172.21.15.18:0/1085995236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76f4069180 0x7f76f410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.625+0000 7f76fb46f640 1 --2- 172.21.15.18:0/1085995236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4068800 0x7f76f4068c40 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.625+0000 7f76fb46f640 1 --2- 172.21.15.18:0/1085995236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f41071f0 0x7f76f41075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.625+0000 7f76fb46f640 1 -- 172.21.15.18:0/1085995236 >> 172.21.15.18:0/1085995236 conn(0x7f76f4076ed0 msgr2=0x7f76f40772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.625+0000 7f76fb46f640 1 -- 172.21.15.18:0/1085995236 shutdown_connections 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.625+0000 7f76fb46f640 1 -- 172.21.15.18:0/1085995236 wait complete. 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.625+0000 7f76fb46f640 1 Processor -- start 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.625+0000 7f76fb46f640 1 -- start start 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76fb46f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4068800 0x7f76f419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76fb46f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4069180 0x7f76f419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76fb46f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76f41071f0 0x7f76f41975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76fb46f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4197c40 con 0x7f76f4069180 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76fb46f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4197db0 con 0x7f76f4068800 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76fb46f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4197f70 con 0x7f76f41071f0 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76f8a0a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4069180 0x7f76f419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76f8a0a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4069180 0x7f76f419d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43002/0 (socket says 172.21.15.18:43002) 2024-02-17T12:57:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76f920b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4068800 0x7f76f419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76f8a0a640 1 -- 172.21.15.18:0/3005625094 learned_addr learned my addr 172.21.15.18:0/3005625094 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76f9a0c640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76f41071f0 0x7f76f41975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76f8a0a640 1 -- 172.21.15.18:0/3005625094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76f41071f0 msgr2=0x7f76f41975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76f41071f0 0x7f76f41975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76f8a0a640 1 -- 172.21.15.18:0/3005625094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4068800 msgr2=0x7f76f419d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4068800 0x7f76f419d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76f8a0a640 1 -- 172.21.15.18:0/3005625094 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e4009560 con 0x7f76f4069180 2024-02-17T12:57:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76f920b640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4068800 0x7f76f419d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4069180 0x7f76f419d980 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009230 tx=0x7f76e4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76ea7fc640 1 -- 172.21.15.18:0/3005625094 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4004440 con 0x7f76f4069180 2024-02-17T12:57:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76ea7fc640 1 -- 172.21.15.18:0/3005625094 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76e40052d0 con 0x7f76f4069180 2024-02-17T12:57:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76fb46f640 1 -- 172.21.15.18:0/3005625094 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f41981f0 con 0x7f76f4069180 2024-02-17T12:57:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76ea7fc640 1 -- 172.21.15.18:0/3005625094 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4003d70 con 0x7f76f4069180 2024-02-17T12:57:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.626+0000 7f76fb46f640 1 -- 172.21.15.18:0/3005625094 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f41a44e0 con 0x7f76f4069180 2024-02-17T12:57:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.629+0000 7f76ea7fc640 1 -- 172.21.15.18:0/3005625094 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f76e4005440 con 0x7f76f4069180 2024-02-17T12:57:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.629+0000 7f76fe040740 1 Processor -- start 2024-02-17T12:57:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.629+0000 7f76ea7fc640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f76e00a22d0 0x7f76e00a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.629+0000 7f76fe040740 1 -- start start 2024-02-17T12:57:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f9a0c640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f76e00a22d0 0x7f76e00a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76ea7fc640 1 -- 172.21.15.18:0/3005625094 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f76e40f0140 con 0x7f76f4069180 2024-02-17T12:57:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76fe040740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d4ad0 0x5622b50d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76fe040740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50d3d60 0x5622b50d41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76fe040740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50dd2e0 0x5622b50df6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76fe040740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b50429f0 con 0x5622b50dd2e0 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76fe040740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b503ff20 con 0x5622b50d3d60 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76fe040740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b503b700 con 0x5622b50d4ad0 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f920b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d4ad0 0x5622b50d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f920b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d4ad0 0x5622b50d3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:41046/0 (socket says 172.21.15.18:41046) 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f920b640 1 -- 172.21.15.18:0/3622778802 learned_addr learned my addr 172.21.15.18:0/3622778802 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/3622778802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50d3d60 0x5622b50d41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f9a0c640 1 --2- 172.21.15.18:0/3622778802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50dd2e0 0x5622b50df6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f920b640 1 -- 172.21.15.18:0/3622778802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50d3d60 msgr2=0x5622b50d41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f920b640 1 --2- 172.21.15.18:0/3622778802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50d3d60 0x5622b50d41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f9a0c640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f76e00a22d0 0x7f76e00a4790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f76ec005fd0 tx=0x7f76ec009f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f920b640 1 -- 172.21.15.18:0/3622778802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50dd2e0 msgr2=0x5622b50df6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f920b640 1 --2- 172.21.15.18:0/3622778802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50dd2e0 0x5622b50df6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f920b640 1 -- 172.21.15.18:0/3622778802 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622b4fd17b0 con 0x5622b50d4ad0 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/3622778802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50d3d60 0x5622b50d41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.630+0000 7f76f920b640 1 --2- 172.21.15.18:0/3622778802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d4ad0 0x5622b50d3820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f76f0010450 tx=0x7f76f0010910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dad0db5420b4e10 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.631+0000 7f76df7fe640 1 -- 172.21.15.18:0/3622778802 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0011760 con 0x5622b50d4ad0 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.631+0000 7f76df7fe640 1 -- 172.21.15.18:0/3622778802 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76f0015070 con 0x5622b50d4ad0 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.631+0000 7f76df7fe640 1 -- 172.21.15.18:0/3622778802 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f001ce20 con 0x5622b50d4ad0 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.631+0000 7f76fe040740 1 -- 172.21.15.18:0/3622778802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d4ad0 msgr2=0x5622b50d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.631+0000 7f76fe040740 1 --2- 172.21.15.18:0/3622778802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d4ad0 0x5622b50d3820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f76f0010450 tx=0x7f76f0010910 comp rx=0 tx=0).stop 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.631+0000 7f76fe040740 1 -- 172.21.15.18:0/3622778802 shutdown_connections 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.631+0000 7f76fe040740 1 --2- 172.21.15.18:0/3622778802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50dd2e0 0x5622b50df6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.631+0000 7f76fe040740 1 --2- 172.21.15.18:0/3622778802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50d3d60 0x5622b50d41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.631+0000 7f76fe040740 1 --2- 172.21.15.18:0/3622778802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d4ad0 0x5622b50d3820 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.631+0000 7f76fe040740 1 -- 172.21.15.18:0/3622778802 >> 172.21.15.18:0/3622778802 conn(0x5622b509b550 msgr2=0x5622b50cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.631+0000 7f76fe040740 1 -- 172.21.15.18:0/3622778802 shutdown_connections 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.631+0000 7f76fe040740 1 -- 172.21.15.18:0/3622778802 wait complete. 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76fe040740 1 -- start start 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76fe040740 1 Processor -- start 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76fe040740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50da320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76fe040740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50d4ad0 0x5622b50da860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76fe040740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50dd2e0 0x5622b50dada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76fe040740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b4fe0bf0 con 0x5622b50dd2e0 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76fe040740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b4d0f250 con 0x5622b50d4ad0 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76fe040740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b4fc1200 con 0x5622b50d3d60 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76fe040740 10 client.0 _ll_register_callbacks cb 0x5622b502fd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76f9a0c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50dd2e0 0x5622b50dada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76f9a0c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50dd2e0 0x5622b50dada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43024/0 (socket says 172.21.15.18:43024) 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76f920b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50da320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76f9a0c640 1 -- 172.21.15.18:0/1320461800 learned_addr learned my addr 172.21.15.18:0/1320461800 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.632+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50d4ad0 0x5622b50da860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76f9a0c640 1 -- 172.21.15.18:0/1320461800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 msgr2=0x5622b50da320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76f9a0c640 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50da320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76f9a0c640 1 -- 172.21.15.18:0/1320461800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50d4ad0 msgr2=0x5622b50da860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76f9a0c640 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50d4ad0 0x5622b50da860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76f9a0c640 1 -- 172.21.15.18:0/1320461800 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f000bcb0 con 0x5622b50dd2e0 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50d4ad0 0x5622b50da860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76f920b640 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50da320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76f9a0c640 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50dd2e0 0x5622b50dada0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0137a0 tx=0x7f76ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f28aa8ed14081b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76dcff9640 1 -- 172.21.15.18:0/1320461800 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec0132d0 con 0x5622b50dd2e0 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76dcff9640 1 -- 172.21.15.18:0/1320461800 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f76ec019040 con 0x5622b50dd2e0 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76fe040740 1 -- 172.21.15.18:0/1320461800 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622b4d3b600 con 0x5622b50dd2e0 2024-02-17T12:57:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76dcff9640 1 -- 172.21.15.18:0/1320461800 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76ec01f9b0 con 0x5622b50dd2e0 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76fe040740 10 client.18150 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76fe040740 1 -- 172.21.15.18:0/1320461800 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5622b5039d80 con 0x5622b50dd2e0 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76dcff9640 1 -- 172.21.15.18:0/1320461800 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76ec034080 con 0x5622b50dd2e0 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.633+0000 7f76dcff9640 1 -- 172.21.15.18:0/1320461800 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f76ec02e030 con 0x5622b50dd2e0 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.634+0000 7f76fe040740 10 client.18150 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.634+0000 7f76fe040740 10 client.18150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.634+0000 7f76fe040740 20 client.18150 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.634+0000 7f76fe040740 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5622b50ed7f0 0x5622b50efc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.634+0000 7f76fe040740 4 client.18150 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.634+0000 7f76fe040740 1 -- 172.21.15.18:0/1320461800 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622b4d8c1b0 con 0x5622b50ed7f0 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.634+0000 7f76f920b640 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5622b50ed7f0 0x5622b50efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.634+0000 7f76dcff9640 10 client.18150 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.634+0000 7f76f920b640 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5622b50ed7f0 0x5622b50efc10 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc914ee6dd883c8 server_cookie=642d0b6801dd3310 in_seq=0 out_seq=0 2024-02-17T12:57:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76dcff9640 1 -- 172.21.15.18:0/1320461800 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5622b4d8c1b0 con 0x5622b50ed7f0 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76dcff9640 10 client.18150 handle_command_reply: tid=0 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76fe040740 1 client.18150 shutdown 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76fe040740 1 -- 172.21.15.18:0/1320461800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50dd2e0 msgr2=0x5622b50dada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76fe040740 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50dd2e0 0x5622b50dada0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0137a0 tx=0x7f76ec004ab0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76fe040740 1 -- 172.21.15.18:0/1320461800 shutdown_connections 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76fe040740 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5622b50ed7f0 0x5622b50efc10 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76fe040740 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50dd2e0 0x5622b50dada0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76fe040740 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50d4ad0 0x5622b50da860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76fe040740 1 --2- 172.21.15.18:0/1320461800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50da320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76dcff9640 0 client.18150 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76fe040740 1 -- 172.21.15.18:0/1320461800 >> 172.21.15.18:0/1320461800 conn(0x5622b509b550 msgr2=0x5622b50d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76fe040740 1 -- 172.21.15.18:0/1320461800 shutdown_connections 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76fe040740 1 -- 172.21.15.18:0/1320461800 wait complete. 2024-02-17T12:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.635+0000 7f76fe040740 20 client.18150 trim_cache size 0 max 16384 2024-02-17T12:57:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76fe040740 1 Processor -- start 2024-02-17T12:57:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76fe040740 1 -- start start 2024-02-17T12:57:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76fe040740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50dca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76fe040740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 0x5622b50dabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76fe040740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50dd2e0 0x5622b50d8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76fe040740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b4d8c1b0 con 0x5622b50d4ad0 2024-02-17T12:57:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76fe040740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b50ea190 con 0x5622b50dd2e0 2024-02-17T12:57:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76fe040740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b50d3030 con 0x5622b50d3d60 2024-02-17T12:57:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76f8a0a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 0x5622b50dabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76f8a0a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 0x5622b50dabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43038/0 (socket says 172.21.15.18:43038) 2024-02-17T12:57:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76f8a0a640 1 -- 172.21.15.18:0/2843317498 learned_addr learned my addr 172.21.15.18:0/2843317498 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76f920b640 1 --2- 172.21.15.18:0/2843317498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50dca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76f9a0c640 1 --2- 172.21.15.18:0/2843317498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50dd2e0 0x5622b50d8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76f8a0a640 1 -- 172.21.15.18:0/2843317498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 msgr2=0x5622b50dca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/2843317498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50dca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76f8a0a640 1 -- 172.21.15.18:0/2843317498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50dd2e0 msgr2=0x5622b50d8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/2843317498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50dd2e0 0x5622b50d8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.649+0000 7f76f8a0a640 1 -- 172.21.15.18:0/2843317498 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622b4fd67b0 con 0x5622b50d4ad0 2024-02-17T12:57:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/2843317498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 0x5622b50dabc0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f76e40bcbd0 tx=0x7f76e404a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ea11d4e6cb8cea server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76deffd640 1 -- 172.21.15.18:0/2843317498 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e40bcc40 con 0x5622b50d4ad0 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76deffd640 1 -- 172.21.15.18:0/2843317498 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76e40c0d90 con 0x5622b50d4ad0 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76deffd640 1 -- 172.21.15.18:0/2843317498 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e40c5220 con 0x5622b50d4ad0 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76fe040740 1 -- 172.21.15.18:0/2843317498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 msgr2=0x5622b50dabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76fe040740 1 --2- 172.21.15.18:0/2843317498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 0x5622b50dabc0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f76e40bcbd0 tx=0x7f76e404a040 comp rx=0 tx=0).stop 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76fe040740 1 -- 172.21.15.18:0/2843317498 shutdown_connections 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76fe040740 1 --2- 172.21.15.18:0/2843317498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50dd2e0 0x5622b50d8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76fe040740 1 --2- 172.21.15.18:0/2843317498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 0x5622b50dabc0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76fe040740 1 --2- 172.21.15.18:0/2843317498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50dca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76fe040740 1 -- 172.21.15.18:0/2843317498 >> 172.21.15.18:0/2843317498 conn(0x5622b509b550 msgr2=0x5622b50d5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76fe040740 1 -- 172.21.15.18:0/2843317498 shutdown_connections 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.650+0000 7f76fe040740 1 -- 172.21.15.18:0/2843317498 wait complete. 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76fe040740 1 -- start start 2024-02-17T12:57:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76fe040740 1 Processor -- start 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76fe040740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50e4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76fe040740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 0x5622b50e2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76fe040740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50dd2e0 0x5622b50e2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76fe040740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b50e3220 con 0x5622b50d4ad0 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76fe040740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b50e3390 con 0x5622b50dd2e0 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76fe040740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b50e3550 con 0x5622b50d3d60 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76fe040740 10 client.0 _ll_register_callbacks cb 0x7f76f000baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76f920b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76f8a0a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 0x5622b50e2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76f9a0c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50dd2e0 0x5622b50e2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76f920b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50e4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:41082/0 (socket says 172.21.15.18:41082) 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.651+0000 7f76f920b640 1 -- 172.21.15.18:0/1862742778 learned_addr learned my addr 172.21.15.18:0/1862742778 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76f8a0a640 1 -- 172.21.15.18:0/1862742778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 msgr2=0x5622b50e4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/1862742778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50e4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76f8a0a640 1 -- 172.21.15.18:0/1862742778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50dd2e0 msgr2=0x5622b50e2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/1862742778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50dd2e0 0x5622b50e2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76f8a0a640 1 -- 172.21.15.18:0/1862742778 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f41981f0 con 0x5622b50d4ad0 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76f920b640 1 --2- 172.21.15.18:0/1862742778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76f8a0a640 1 --2- 172.21.15.18:0/1862742778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 0x5622b50e2750 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f76e40bcbd0 tx=0x7f76e4003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42e96ef6cb7fc56 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76dcff9640 1 -- 172.21.15.18:0/1862742778 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e40c51c0 con 0x5622b50d4ad0 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76dcff9640 1 -- 172.21.15.18:0/1862742778 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f76e40bcc40 con 0x5622b50d4ad0 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76fe040740 1 -- 172.21.15.18:0/1862742778 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76f0015070 con 0x5622b50d4ad0 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76dcff9640 1 -- 172.21.15.18:0/1862742778 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76e40bc090 con 0x5622b50d4ad0 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76fe040740 10 client.18156 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76fe040740 1 -- 172.21.15.18:0/1862742778 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5622b4ffc540 con 0x5622b50d4ad0 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76dcff9640 1 -- 172.21.15.18:0/1862742778 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76f0015070 con 0x5622b50d4ad0 2024-02-17T12:57:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76dcff9640 1 -- 172.21.15.18:0/1862742778 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f76e40db020 con 0x5622b50d4ad0 2024-02-17T12:57:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76fe040740 10 client.18156 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76fe040740 10 client.18156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76fe040740 20 client.18156 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76fe040740 1 --2- 172.21.15.18:0/1862742778 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5622b51493b0 0x5622b514b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.652+0000 7f76fe040740 4 client.18156 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:57:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.653+0000 7f76fe040740 1 -- 172.21.15.18:0/1862742778 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5622b4d78550 con 0x5622b51493b0 2024-02-17T12:57:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.653+0000 7f76f920b640 1 --2- 172.21.15.18:0/1862742778 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5622b51493b0 0x5622b514b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.653+0000 7f76dcff9640 10 client.18156 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.653+0000 7f76f920b640 1 --2- 172.21.15.18:0/1862742778 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5622b51493b0 0x5622b514b7d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe8428c9b316ff99 server_cookie=12ad43830fa1e1ad in_seq=0 out_seq=0 2024-02-17T12:57:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76dcff9640 1 -- 172.21.15.18:0/1862742778 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5622b4d78550 con 0x5622b51493b0 2024-02-17T12:57:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76dcff9640 10 client.18156 handle_command_reply: tid=0 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76fe040740 1 client.18156 shutdown 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76fe040740 1 -- 172.21.15.18:0/1862742778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 msgr2=0x5622b50e2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76fe040740 1 --2- 172.21.15.18:0/1862742778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 0x5622b50e2750 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f76e40bcbd0 tx=0x7f76e4003840 comp rx=0 tx=0).stop 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76fe040740 1 -- 172.21.15.18:0/1862742778 shutdown_connections 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76fe040740 1 --2- 172.21.15.18:0/1862742778 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5622b51493b0 0x5622b514b7d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76fe040740 1 --2- 172.21.15.18:0/1862742778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622b50dd2e0 0x5622b50e2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76fe040740 1 --2- 172.21.15.18:0/1862742778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622b50d4ad0 0x5622b50e2750 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76dcff9640 0 client.18156 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76fe040740 1 --2- 172.21.15.18:0/1862742778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5622b50d3d60 0x5622b50e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76fe040740 1 -- 172.21.15.18:0/1862742778 >> 172.21.15.18:0/1862742778 conn(0x5622b509b550 msgr2=0x5622b50d8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.692+0000 7f76fe040740 1 -- 172.21.15.18:0/1862742778 shutdown_connections 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fe040740 1 -- 172.21.15.18:0/1862742778 wait complete. 2024-02-17T12:57:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fe040740 20 client.18156 trim_cache size 0 max 16384 2024-02-17T12:57:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:57:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:57:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:57:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:57:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fb46f640 1 -- 172.21.15.18:0/3005625094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f76e00a22d0 msgr2=0x7f76e00a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fb46f640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f76e00a22d0 0x7f76e00a4790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f76ec005fd0 tx=0x7f76ec009f90 comp rx=0 tx=0).stop 2024-02-17T12:57:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fb46f640 1 -- 172.21.15.18:0/3005625094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4069180 msgr2=0x7f76f419d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fb46f640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4069180 0x7f76f419d980 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009230 tx=0x7f76e4004040 comp rx=0 tx=0).stop 2024-02-17T12:57:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fb46f640 1 -- 172.21.15.18:0/3005625094 shutdown_connections 2024-02-17T12:57:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fb46f640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f76e00a22d0 0x7f76e00a4790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fb46f640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76f41071f0 0x7f76f41975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fb46f640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4069180 0x7f76f419d980 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fb46f640 1 --2- 172.21.15.18:0/3005625094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4068800 0x7f76f419d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fb46f640 1 -- 172.21.15.18:0/3005625094 >> 172.21.15.18:0/3005625094 conn(0x7f76f4076ed0 msgr2=0x7f76f40fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fb46f640 1 -- 172.21.15.18:0/3005625094 shutdown_connections 2024-02-17T12:57:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:14.693+0000 7f76fb46f640 1 -- 172.21.15.18:0/3005625094 wait complete. 2024-02-17T12:57:15.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 383680 5.55 MB/sec execute 270 sec latency 32.410 ms 2024-02-17T12:57:16.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:16 smithi018 ceph-mon[32625]: pgmap v228: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-02-17T12:57:16.282 INFO:tasks.workunit.client.0.smithi018.stdout: 1 384796 5.55 MB/sec execute 271 sec latency 373.535 ms 2024-02-17T12:57:16.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:16 smithi119 ceph-mon[36188]: pgmap v228: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-02-17T12:57:16.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:16 smithi146 ceph-mon[36454]: pgmap v228: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-02-17T12:57:17.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 384796 5.53 MB/sec execute 272 sec latency 1373.635 ms 2024-02-17T12:57:18.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:18 smithi018 ceph-mon[32625]: pgmap v229: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-02-17T12:57:18.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 385596 5.52 MB/sec execute 273 sec latency 1940.437 ms 2024-02-17T12:57:18.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:18 smithi119 ceph-mon[36188]: pgmap v229: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-02-17T12:57:18.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:18 smithi146 ceph-mon[36454]: pgmap v229: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-02-17T12:57:19.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:57:19.170 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:57:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.289+0000 7fdad661c640 1 -- 172.21.15.18:0/723699733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdad010b6e0 msgr2=0x7fdad010db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.289+0000 7fdad661c640 1 --2- 172.21.15.18:0/723699733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdad010b6e0 0x7fdad010db80 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009ca0 tx=0x7fdac802c180 comp rx=0 tx=0).stop 2024-02-17T12:57:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.289+0000 7fdad661c640 1 -- 172.21.15.18:0/723699733 shutdown_connections 2024-02-17T12:57:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.289+0000 7fdad661c640 1 --2- 172.21.15.18:0/723699733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdad010b6e0 0x7fdad010db80 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.289+0000 7fdad661c640 1 --2- 172.21.15.18:0/723699733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdad0102cc0 0x7fdad010b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.289+0000 7fdad661c640 1 --2- 172.21.15.18:0/723699733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad01023a0 0x7fdad0102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.289+0000 7fdad661c640 1 -- 172.21.15.18:0/723699733 >> 172.21.15.18:0/723699733 conn(0x7fdad0077230 msgr2=0x7fdad0077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.289+0000 7fdad661c640 1 -- 172.21.15.18:0/723699733 shutdown_connections 2024-02-17T12:57:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.289+0000 7fdad661c640 1 -- 172.21.15.18:0/723699733 wait complete. 2024-02-17T12:57:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.289+0000 7fdad661c640 1 Processor -- start 2024-02-17T12:57:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.289+0000 7fdad661c640 1 -- start start 2024-02-17T12:57:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad661c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad01023a0 0x7fdad0197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad661c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdad0102cc0 0x7fdad0197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad661c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdad010b6e0 0x7fdad019bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad661c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad019c5b0 con 0x7fdad010b6e0 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad661c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad019c720 con 0x7fdad01023a0 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad661c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad019c8e0 con 0x7fdad0102cc0 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad4bb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdad010b6e0 0x7fdad019bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdacffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad01023a0 0x7fdad0197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad4bb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdad010b6e0 0x7fdad019bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43056/0 (socket says 172.21.15.18:43056) 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad4bb9640 1 -- 172.21.15.18:0/269458992 learned_addr learned my addr 172.21.15.18:0/269458992 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdac7fff640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdad0102cc0 0x7fdad0197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad4bb9640 1 -- 172.21.15.18:0/269458992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdad0102cc0 msgr2=0x7fdad0197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdad0102cc0 0x7fdad0197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad4bb9640 1 -- 172.21.15.18:0/269458992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad01023a0 msgr2=0x7fdad0197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad01023a0 0x7fdad0197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad4bb9640 1 -- 172.21.15.18:0/269458992 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac8009900 con 0x7fdad010b6e0 2024-02-17T12:57:19.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdad010b6e0 0x7fdad019bfb0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fdac802c790 tx=0x7fdac8034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdacdffb640 1 -- 172.21.15.18:0/269458992 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8003b70 con 0x7fdad010b6e0 2024-02-17T12:57:19.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdacdffb640 1 -- 172.21.15.18:0/269458992 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdac8003cd0 con 0x7fdad010b6e0 2024-02-17T12:57:19.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdacdffb640 1 -- 172.21.15.18:0/269458992 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80046c0 con 0x7fdad010b6e0 2024-02-17T12:57:19.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad661c640 1 -- 172.21.15.18:0/269458992 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad019cb60 con 0x7fdad010b6e0 2024-02-17T12:57:19.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.290+0000 7fdad661c640 1 -- 172.21.15.18:0/269458992 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad019ce30 con 0x7fdad010b6e0 2024-02-17T12:57:19.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.292+0000 7fdacdffb640 1 -- 172.21.15.18:0/269458992 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fdac800a040 con 0x7fdad010b6e0 2024-02-17T12:57:19.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 387139 5.54 MB/sec execute 274 sec latency 26.370 ms 2024-02-17T12:57:19.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdad91f3740 1 Processor -- start 2024-02-17T12:57:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdad91f3740 1 -- start start 2024-02-17T12:57:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdacdffb640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdabc0a2230 0x7fdabc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdad91f3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cf2b0 0x562d275cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdad91f3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cfbd0 0x562d275d8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdac7fff640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdabc0a2230 0x7fdabc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdacdffb640 1 -- 172.21.15.18:0/269458992 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdac80f0b60 con 0x7fdad010b6e0 2024-02-17T12:57:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdad91f3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275daba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdad91f3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d2753e280 con 0x562d275d87b0 2024-02-17T12:57:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdad91f3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d2753b7b0 con 0x562d275cfbd0 2024-02-17T12:57:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdad91f3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d27536f90 con 0x562d275cf2b0 2024-02-17T12:57:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdad4bb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdad4bb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275daba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43060/0 (socket says 172.21.15.18:43060) 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdad4bb9640 1 -- 172.21.15.18:0/2715891920 learned_addr learned my addr 172.21.15.18:0/2715891920 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.293+0000 7fdacffff640 1 --2- 172.21.15.18:0/2715891920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cf2b0 0x562d275cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.294+0000 7fdac7fff640 1 --2- 172.21.15.18:0/2715891920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cfbd0 0x562d275d8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.294+0000 7fdad4bb9640 1 -- 172.21.15.18:0/2715891920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cf2b0 msgr2=0x562d275cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.294+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/2715891920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cf2b0 0x562d275cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.294+0000 7fdad4bb9640 1 -- 172.21.15.18:0/2715891920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cfbd0 msgr2=0x562d275d8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.294+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/2715891920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cfbd0 0x562d275d8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.294+0000 7fdad4bb9640 1 -- 172.21.15.18:0/2715891920 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d274f5570 con 0x562d275d87b0 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.294+0000 7fdac7fff640 1 --2- 172.21.15.18:0/2715891920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cfbd0 0x562d275d8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.294+0000 7fdac7fff640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdabc0a2230 0x7fdabc0a46f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdad00ff9e0 tx=0x7fdab8009ee0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.294+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/2715891920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275daba0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fdac8037040 tx=0x7fdac80bcfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d94290a7dd77f2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.294+0000 7fdac67fc640 1 -- 172.21.15.18:0/2715891920 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80bdd80 con 0x562d275d87b0 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.294+0000 7fdac67fc640 1 -- 172.21.15.18:0/2715891920 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdac80bd680 con 0x562d275d87b0 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.294+0000 7fdac67fc640 1 -- 172.21.15.18:0/2715891920 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8032e50 con 0x562d275d87b0 2024-02-17T12:57:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.295+0000 7fdad91f3740 1 -- 172.21.15.18:0/2715891920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 msgr2=0x562d275daba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.295+0000 7fdad91f3740 1 --2- 172.21.15.18:0/2715891920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275daba0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fdac8037040 tx=0x7fdac80bcfe0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.295+0000 7fdad91f3740 1 -- 172.21.15.18:0/2715891920 shutdown_connections 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.295+0000 7fdad91f3740 1 --2- 172.21.15.18:0/2715891920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275daba0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.295+0000 7fdad91f3740 1 --2- 172.21.15.18:0/2715891920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cfbd0 0x562d275d8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.295+0000 7fdad91f3740 1 --2- 172.21.15.18:0/2715891920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cf2b0 0x562d275cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.295+0000 7fdad91f3740 1 -- 172.21.15.18:0/2715891920 >> 172.21.15.18:0/2715891920 conn(0x562d275c9990 msgr2=0x562d275c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.295+0000 7fdad91f3740 1 -- 172.21.15.18:0/2715891920 shutdown_connections 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.295+0000 7fdad91f3740 1 -- 172.21.15.18:0/2715891920 wait complete. 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.295+0000 7fdad91f3740 1 -- start start 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.295+0000 7fdad91f3740 1 Processor -- start 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad91f3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cf2b0 0x562d275d8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad91f3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cfbd0 0x562d275d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad91f3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275d5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad4bb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275d5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad91f3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d2720a430 con 0x562d275d87b0 2024-02-17T12:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad91f3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d271c6e00 con 0x562d275cf2b0 2024-02-17T12:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad91f3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d274bbfd0 con 0x562d275cfbd0 2024-02-17T12:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdac7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cfbd0 0x562d275d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad4bb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275d5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43074/0 (socket says 172.21.15.18:43074) 2024-02-17T12:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad4bb9640 1 -- 172.21.15.18:0/4180783156 learned_addr learned my addr 172.21.15.18:0/4180783156 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad91f3740 10 client.0 _ll_register_callbacks cb 0x562d27511fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdacffff640 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cf2b0 0x562d275d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad4bb9640 1 -- 172.21.15.18:0/4180783156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cfbd0 msgr2=0x562d275d55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cfbd0 0x562d275d55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad4bb9640 1 -- 172.21.15.18:0/4180783156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cf2b0 msgr2=0x562d275d8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cf2b0 0x562d275d8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad4bb9640 1 -- 172.21.15.18:0/4180783156 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad019ce30 con 0x562d275d87b0 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdac7fff640 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cfbd0 0x562d275d55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275d5cf0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdac80ba050 tx=0x7fdac80bd110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49515fb95582a847 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdaaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.296+0000 7fdaaf7fe640 1 -- 172.21.15.18:0/4180783156 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80c17d0 con 0x562d275d87b0 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdaaf7fe640 1 -- 172.21.15.18:0/4180783156 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdac80c5e20 con 0x562d275d87b0 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdad91f3740 1 -- 172.21.15.18:0/4180783156 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d272367e0 con 0x562d275d87b0 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdaaf7fe640 1 -- 172.21.15.18:0/4180783156 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdac80b9cf0 con 0x562d275d87b0 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdad91f3740 10 client.18168 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdad91f3740 1 -- 172.21.15.18:0/4180783156 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x562d27535610 con 0x562d275d87b0 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdaaf7fe640 1 -- 172.21.15.18:0/4180783156 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562d272367e0 con 0x562d275d87b0 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdaaf7fe640 1 -- 172.21.15.18:0/4180783156 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdac80c5990 con 0x562d275d87b0 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdad91f3740 10 client.18168 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdad91f3740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdad91f3740 20 client.18168 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdad91f3740 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d275e8340 0x562d275ea730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdad91f3740 4 client.18168 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdad91f3740 1 -- 172.21.15.18:0/4180783156 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d27273720 con 0x562d275e8340 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.297+0000 7fdacffff640 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d275e8340 0x562d275ea730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdaaf7fe640 10 client.18168 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdacffff640 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d275e8340 0x562d275ea730 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9458cbb25216bef server_cookie=1f57d40880e405c2 in_seq=0 out_seq=0 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdaaf7fe640 1 -- 172.21.15.18:0/4180783156 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d27273720 con 0x562d275e8340 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdaaf7fe640 10 client.18168 handle_command_reply: tid=0 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdad91f3740 1 client.18168 shutdown 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdad91f3740 1 -- 172.21.15.18:0/4180783156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 msgr2=0x562d275d5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdad91f3740 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275d5cf0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdac80ba050 tx=0x7fdac80bd110 comp rx=0 tx=0).stop 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdad91f3740 1 -- 172.21.15.18:0/4180783156 shutdown_connections 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdad91f3740 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d275e8340 0x562d275ea730 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdad91f3740 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275d5cf0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdad91f3740 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cfbd0 0x562d275d55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdaaf7fe640 0 client.18168 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdad91f3740 1 --2- 172.21.15.18:0/4180783156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cf2b0 0x562d275d8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.298+0000 7fdad91f3740 1 -- 172.21.15.18:0/4180783156 >> 172.21.15.18:0/4180783156 conn(0x562d275c9990 msgr2=0x562d275cb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.299+0000 7fdad91f3740 1 -- 172.21.15.18:0/4180783156 shutdown_connections 2024-02-17T12:57:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.299+0000 7fdad91f3740 1 -- 172.21.15.18:0/4180783156 wait complete. 2024-02-17T12:57:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.299+0000 7fdad91f3740 20 client.18168 trim_cache size 0 max 16384 2024-02-17T12:57:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.314+0000 7fdad91f3740 1 Processor -- start 2024-02-17T12:57:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.314+0000 7fdad91f3740 1 -- start start 2024-02-17T12:57:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.314+0000 7fdad91f3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cf2b0 0x562d275d7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.314+0000 7fdad91f3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cfbd0 0x562d275d8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.314+0000 7fdad91f3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275d4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.314+0000 7fdad91f3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d272367e0 con 0x562d275d87b0 2024-02-17T12:57:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.314+0000 7fdad91f3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d275c50f0 con 0x562d275cfbd0 2024-02-17T12:57:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.314+0000 7fdad91f3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d275cef10 con 0x562d275cf2b0 2024-02-17T12:57:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.314+0000 7fdad4bb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275d4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.314+0000 7fdacffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cf2b0 0x562d275d7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.314+0000 7fdad4bb9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275d4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43078/0 (socket says 172.21.15.18:43078) 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.314+0000 7fdad4bb9640 1 -- 172.21.15.18:0/3327921057 learned_addr learned my addr 172.21.15.18:0/3327921057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.315+0000 7fdac7fff640 1 --2- 172.21.15.18:0/3327921057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cfbd0 0x562d275d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.315+0000 7fdad4bb9640 1 -- 172.21.15.18:0/3327921057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cf2b0 msgr2=0x562d275d7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.315+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/3327921057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cf2b0 0x562d275d7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.315+0000 7fdad4bb9640 1 -- 172.21.15.18:0/3327921057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cfbd0 msgr2=0x562d275d8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.315+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/3327921057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cfbd0 0x562d275d8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.315+0000 7fdad4bb9640 1 -- 172.21.15.18:0/3327921057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d274d1ac0 con 0x562d275d87b0 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.315+0000 7fdacffff640 1 --2- 172.21.15.18:0/3327921057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cf2b0 0x562d275d7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.315+0000 7fdac7fff640 1 --2- 172.21.15.18:0/3327921057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cfbd0 0x562d275d8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.315+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/3327921057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275d4e40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fdac80bb930 tx=0x7fdac80bbb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1276c2a01bf04fa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.315+0000 7fdac5ffb640 1 -- 172.21.15.18:0/3327921057 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8032de0 con 0x562d275d87b0 2024-02-17T12:57:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.315+0000 7fdac5ffb640 1 -- 172.21.15.18:0/3327921057 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdac80bd450 con 0x562d275d87b0 2024-02-17T12:57:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.315+0000 7fdac5ffb640 1 -- 172.21.15.18:0/3327921057 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80b9cf0 con 0x562d275d87b0 2024-02-17T12:57:19.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.316+0000 7fdad91f3740 1 -- 172.21.15.18:0/3327921057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 msgr2=0x562d275d4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.316+0000 7fdad91f3740 1 --2- 172.21.15.18:0/3327921057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275d4e40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fdac80bb930 tx=0x7fdac80bbb90 comp rx=0 tx=0).stop 2024-02-17T12:57:19.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.316+0000 7fdad91f3740 1 -- 172.21.15.18:0/3327921057 shutdown_connections 2024-02-17T12:57:19.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.316+0000 7fdad91f3740 1 --2- 172.21.15.18:0/3327921057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275d87b0 0x562d275d4e40 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.316+0000 7fdad91f3740 1 --2- 172.21.15.18:0/3327921057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cfbd0 0x562d275d8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.316+0000 7fdad91f3740 1 --2- 172.21.15.18:0/3327921057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275cf2b0 0x562d275d7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.316+0000 7fdad91f3740 1 -- 172.21.15.18:0/3327921057 >> 172.21.15.18:0/3327921057 conn(0x562d275c9990 msgr2=0x562d275d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.316+0000 7fdad91f3740 1 -- 172.21.15.18:0/3327921057 shutdown_connections 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.316+0000 7fdad91f3740 1 -- 172.21.15.18:0/3327921057 wait complete. 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.316+0000 7fdad91f3740 1 -- start start 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.316+0000 7fdad91f3740 1 Processor -- start 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdad91f3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cf2b0 0x562d275df420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdad91f3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275cfbd0 0x562d275df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdad91f3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275d87b0 0x562d275dba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdad91f3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d275dc0d0 con 0x562d275cfbd0 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdad91f3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d275dc240 con 0x562d275cf2b0 2024-02-17T12:57:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdad91f3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d275dc400 con 0x562d275d87b0 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdad91f3740 10 client.0 _ll_register_callbacks cb 0x7fdac8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdac7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275cfbd0 0x562d275df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdac7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275cfbd0 0x562d275df960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43088/0 (socket says 172.21.15.18:43088) 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdac7fff640 1 -- 172.21.15.18:0/3058951692 learned_addr learned my addr 172.21.15.18:0/3058951692 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdacffff640 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cf2b0 0x562d275df420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdad4bb9640 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275d87b0 0x562d275dba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdacffff640 1 -- 172.21.15.18:0/3058951692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275d87b0 msgr2=0x562d275dba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdacffff640 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275d87b0 0x562d275dba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdacffff640 1 -- 172.21.15.18:0/3058951692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275cfbd0 msgr2=0x562d275df960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdacffff640 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275cfbd0 0x562d275df960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdacffff640 1 -- 172.21.15.18:0/3058951692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac0009ad0 con 0x562d275cf2b0 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdac7fff640 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275cfbd0 0x562d275df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdacffff640 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cf2b0 0x562d275df420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x562d275ead10 tx=0x7fdac001f960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3df9a622aa63025 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdaaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.317+0000 7fdaaf7fe640 1 -- 172.21.15.18:0/3058951692 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0004030 con 0x562d275cf2b0 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdaaf7fe640 1 -- 172.21.15.18:0/3058951692 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdac0004190 con 0x562d275cf2b0 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdad91f3740 1 -- 172.21.15.18:0/3058951692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdac80bd680 con 0x562d275cf2b0 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdaaf7fe640 1 -- 172.21.15.18:0/3058951692 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdac0004370 con 0x562d275cf2b0 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdad91f3740 10 client.27365 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdad91f3740 1 -- 172.21.15.18:0/3058951692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fdab8014470 con 0x562d275cf2b0 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdaaf7fe640 1 -- 172.21.15.18:0/3058951692 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdac80bd680 con 0x562d275cf2b0 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdaaf7fe640 1 -- 172.21.15.18:0/3058951692 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdac00275f0 con 0x562d275cf2b0 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdad91f3740 10 client.27365 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdad91f3740 10 client.27365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdad91f3740 20 client.27365 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdad91f3740 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d2763fc40 0x562d27642060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdad91f3740 4 client.27365 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdad91f3740 1 -- 172.21.15.18:0/3058951692 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562d272c2890 con 0x562d2763fc40 2024-02-17T12:57:19.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.318+0000 7fdac7fff640 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d2763fc40 0x562d27642060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.319+0000 7fdaaf7fe640 10 client.27365 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.319+0000 7fdac7fff640 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d2763fc40 0x562d27642060 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31acb8503e3f2f06 server_cookie=3d97f0c38e20a723 in_seq=0 out_seq=0 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.319+0000 7fdaaf7fe640 1 -- 172.21.15.18:0/3058951692 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562d272c2890 con 0x562d2763fc40 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.319+0000 7fdaaf7fe640 10 client.27365 handle_command_reply: tid=0 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad91f3740 1 client.27365 shutdown 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad91f3740 1 -- 172.21.15.18:0/3058951692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cf2b0 msgr2=0x562d275df420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad91f3740 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cf2b0 0x562d275df420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x562d275ead10 tx=0x7fdac001f960 comp rx=0 tx=0).stop 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad91f3740 1 -- 172.21.15.18:0/3058951692 shutdown_connections 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad91f3740 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d2763fc40 0x562d27642060 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad91f3740 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d275d87b0 0x562d275dba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad91f3740 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d275cfbd0 0x562d275df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdaaf7fe640 0 client.27365 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:19.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad91f3740 1 --2- 172.21.15.18:0/3058951692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d275cf2b0 0x562d275df420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad91f3740 1 -- 172.21.15.18:0/3058951692 >> 172.21.15.18:0/3058951692 conn(0x562d275c9990 msgr2=0x562d275d4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad91f3740 1 -- 172.21.15.18:0/3058951692 shutdown_connections 2024-02-17T12:57:19.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad91f3740 1 -- 172.21.15.18:0/3058951692 wait complete. 2024-02-17T12:57:19.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad91f3740 20 client.27365 trim_cache size 0 max 16384 2024-02-17T12:57:19.317 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:57:19.317 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:57:19.318 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:57:19.318 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:57:19.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad661c640 1 -- 172.21.15.18:0/269458992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdabc0a2230 msgr2=0x7fdabc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad661c640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdabc0a2230 0x7fdabc0a46f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdad00ff9e0 tx=0x7fdab8009ee0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad661c640 1 -- 172.21.15.18:0/269458992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdad010b6e0 msgr2=0x7fdad019bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad661c640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdad010b6e0 0x7fdad019bfb0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fdac802c790 tx=0x7fdac8034670 comp rx=0 tx=0).stop 2024-02-17T12:57:19.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad661c640 1 -- 172.21.15.18:0/269458992 shutdown_connections 2024-02-17T12:57:19.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad661c640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdabc0a2230 0x7fdabc0a46f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad661c640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdad010b6e0 0x7fdad019bfb0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad661c640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdad0102cc0 0x7fdad0197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad661c640 1 --2- 172.21.15.18:0/269458992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad01023a0 0x7fdad0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.320+0000 7fdad661c640 1 -- 172.21.15.18:0/269458992 >> 172.21.15.18:0/269458992 conn(0x7fdad0077230 msgr2=0x7fdad00fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.321+0000 7fdad661c640 1 -- 172.21.15.18:0/269458992 shutdown_connections 2024-02-17T12:57:19.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.321+0000 7fdad661c640 1 -- 172.21.15.18:0/269458992 wait complete. 2024-02-17T12:57:19.321 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:57:19.322 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:57:19.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.436+0000 7fab3d82e640 1 -- 172.21.15.18:0/192382714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab381053c0 msgr2=0x7fab3810cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.436+0000 7fab3d82e640 1 --2- 172.21.15.18:0/192382714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab381053c0 0x7fab3810cf90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fab30009cd0 tx=0x7fab3002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.436+0000 7fab3d82e640 1 -- 172.21.15.18:0/192382714 shutdown_connections 2024-02-17T12:57:19.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.436+0000 7fab3d82e640 1 --2- 172.21.15.18:0/192382714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab381053c0 0x7fab3810cf90 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.436+0000 7fab3d82e640 1 --2- 172.21.15.18:0/192382714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab38100ca0 0x7fab38104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.436+0000 7fab3d82e640 1 --2- 172.21.15.18:0/192382714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab381002f0 0x7fab381006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.436+0000 7fab3d82e640 1 -- 172.21.15.18:0/192382714 >> 172.21.15.18:0/192382714 conn(0x7fab38077180 msgr2=0x7fab38077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.436+0000 7fab3d82e640 1 -- 172.21.15.18:0/192382714 shutdown_connections 2024-02-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.436+0000 7fab3d82e640 1 -- 172.21.15.18:0/192382714 wait complete. 2024-02-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab3d82e640 1 Processor -- start 2024-02-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab3d82e640 1 -- start start 2024-02-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab3d82e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab381002f0 0x7fab3819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab3d82e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab381053c0 0x7fab3819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab3d82e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab38197600 0x7fab38197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab3d82e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38197ff0 con 0x7fab381002f0 2024-02-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab3d82e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38198160 con 0x7fab381053c0 2024-02-17T12:57:19.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab3d82e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38198320 con 0x7fab38197600 2024-02-17T12:57:19.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab36ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab381002f0 0x7fab3819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab36ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab381002f0 0x7fab3819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43092/0 (socket says 172.21.15.18:43092) 2024-02-17T12:57:19.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab36ffd640 1 -- 172.21.15.18:0/2186676137 learned_addr learned my addr 172.21.15.18:0/2186676137 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab377fe640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab38197600 0x7fab38197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab367fc640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab381053c0 0x7fab3819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab36ffd640 1 -- 172.21.15.18:0/2186676137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab38197600 msgr2=0x7fab38197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab36ffd640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab38197600 0x7fab38197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab36ffd640 1 -- 172.21.15.18:0/2186676137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab381053c0 msgr2=0x7fab3819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab36ffd640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab381053c0 0x7fab3819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.437+0000 7fab36ffd640 1 -- 172.21.15.18:0/2186676137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30009900 con 0x7fab381002f0 2024-02-17T12:57:19.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.438+0000 7fab36ffd640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab381002f0 0x7fab3819d450 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fab2800e9f0 tx=0x7fab2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.438+0000 7fab3c82c640 1 -- 172.21.15.18:0/2186676137 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28009700 con 0x7fab381002f0 2024-02-17T12:57:19.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.438+0000 7fab3c82c640 1 -- 172.21.15.18:0/2186676137 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab2800cd80 con 0x7fab381002f0 2024-02-17T12:57:19.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.438+0000 7fab3d82e640 1 -- 172.21.15.18:0/2186676137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab381a4310 con 0x7fab381002f0 2024-02-17T12:57:19.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.438+0000 7fab3c82c640 1 -- 172.21.15.18:0/2186676137 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28018920 con 0x7fab381002f0 2024-02-17T12:57:19.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.438+0000 7fab3d82e640 1 -- 172.21.15.18:0/2186676137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab381a4820 con 0x7fab381002f0 2024-02-17T12:57:19.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.439+0000 7fab3d82e640 1 -- 172.21.15.18:0/2186676137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab38067f70 con 0x7fab381002f0 2024-02-17T12:57:19.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.440+0000 7fab3c82c640 1 -- 172.21.15.18:0/2186676137 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fab28018a80 con 0x7fab381002f0 2024-02-17T12:57:19.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.441+0000 7fab3c82c640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab200a2230 0x7fab200a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.441+0000 7fab367fc640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab200a2230 0x7fab200a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.441+0000 7fab3c82c640 1 -- 172.21.15.18:0/2186676137 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fab28014070 con 0x7fab381002f0 2024-02-17T12:57:19.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.441+0000 7fab367fc640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab200a2230 0x7fab200a46f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fab380ffb40 tx=0x7fab2c00b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.445+0000 7fab3c82c640 1 -- 172.21.15.18:0/2186676137 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab28097600 con 0x7fab381002f0 2024-02-17T12:57:19.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.683+0000 7fab3d82e640 1 -- 172.21.15.18:0/2186676137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab38198b30 con 0x7fab381002f0 2024-02-17T12:57:19.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.683+0000 7fab3c82c640 1 -- 172.21.15.18:0/2186676137 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+11394 (secure 0 0 0) 0x7fab28026090 con 0x7fab381002f0 2024-02-17T12:57:19.674 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:57:19.674 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:57:04.942943+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:57:19.674 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 43 2024-02-17T12:57:19.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.685+0000 7fab3d82e640 1 -- 172.21.15.18:0/2186676137 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab200a2230 msgr2=0x7fab200a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.685+0000 7fab3d82e640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab200a2230 0x7fab200a46f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fab380ffb40 tx=0x7fab2c00b410 comp rx=0 tx=0).stop 2024-02-17T12:57:19.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.685+0000 7fab3d82e640 1 -- 172.21.15.18:0/2186676137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab381002f0 msgr2=0x7fab3819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.685+0000 7fab3d82e640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab381002f0 0x7fab3819d450 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fab2800e9f0 tx=0x7fab2800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.686+0000 7fab3d82e640 1 -- 172.21.15.18:0/2186676137 shutdown_connections 2024-02-17T12:57:19.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.686+0000 7fab3d82e640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fab200a2230 0x7fab200a46f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.686+0000 7fab3d82e640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab38197600 0x7fab38197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.686+0000 7fab3d82e640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab381053c0 0x7fab3819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.686+0000 7fab3d82e640 1 --2- 172.21.15.18:0/2186676137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab381002f0 0x7fab3819d450 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.686+0000 7fab3d82e640 1 -- 172.21.15.18:0/2186676137 >> 172.21.15.18:0/2186676137 conn(0x7fab38077180 msgr2=0x7fab380fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.686+0000 7fab3d82e640 1 -- 172.21.15.18:0/2186676137 shutdown_connections 2024-02-17T12:57:19.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.686+0000 7fab3d82e640 1 -- 172.21.15.18:0/2186676137 wait complete. 2024-02-17T12:57:19.689 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:57:19.689 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:57:19.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.809+0000 7f97df042640 1 -- 172.21.15.18:0/524630262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d81002e0 msgr2=0x7f97d81006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.809+0000 7f97df042640 1 --2- 172.21.15.18:0/524630262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d81002e0 0x7f97d81006c0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009930 tx=0x7f97cc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:19.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.809+0000 7f97df042640 1 -- 172.21.15.18:0/524630262 shutdown_connections 2024-02-17T12:57:19.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.809+0000 7f97df042640 1 --2- 172.21.15.18:0/524630262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97d8105360 0x7f97d810cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.809+0000 7f97df042640 1 --2- 172.21.15.18:0/524630262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d8100c90 0x7f97d8104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.809+0000 7f97df042640 1 --2- 172.21.15.18:0/524630262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d81002e0 0x7f97d81006c0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.809+0000 7f97df042640 1 -- 172.21.15.18:0/524630262 >> 172.21.15.18:0/524630262 conn(0x7f97d80771e0 msgr2=0x7f97d80775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97df042640 1 -- 172.21.15.18:0/524630262 shutdown_connections 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97df042640 1 -- 172.21.15.18:0/524630262 wait complete. 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97df042640 1 Processor -- start 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97df042640 1 -- start start 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97df042640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d81002e0 0x7f97d8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97df042640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d8105360 0x7f97d8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97df042640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97d81990d0 0x7f97d81a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97df042640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d81a3d20 con 0x7f97d8105360 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97df042640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d81a3e90 con 0x7f97d81002e0 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97df042640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d81a4050 con 0x7f97d81990d0 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d8105360 0x7f97d8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97dcdde640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d81002e0 0x7f97d8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d8105360 0x7f97d8198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43944/0 (socket says 172.21.15.18:43944) 2024-02-17T12:57:19.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.810+0000 7f97d7fff640 1 -- 172.21.15.18:0/3523084490 learned_addr learned my addr 172.21.15.18:0/3523084490 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97dd5df640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97d81990d0 0x7f97d81a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97d7fff640 1 -- 172.21.15.18:0/3523084490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97d81990d0 msgr2=0x7f97d81a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97d7fff640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97d81990d0 0x7f97d81a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97d7fff640 1 -- 172.21.15.18:0/3523084490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d81002e0 msgr2=0x7f97d8198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97d7fff640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d81002e0 0x7f97d8198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97d7fff640 1 -- 172.21.15.18:0/3523084490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc009560 con 0x7f97d8105360 2024-02-17T12:57:19.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97dcdde640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d81002e0 0x7f97d8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:19.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97d7fff640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d8105360 0x7f97d8198b90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f97c400e9c0 tx=0x7f97c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97d5ffb640 1 -- 172.21.15.18:0/3523084490 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4009700 con 0x7f97d8105360 2024-02-17T12:57:19.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97d5ffb640 1 -- 172.21.15.18:0/3523084490 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97c400cd80 con 0x7f97d8105360 2024-02-17T12:57:19.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97df042640 1 -- 172.21.15.18:0/3523084490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d81a4330 con 0x7f97d8105360 2024-02-17T12:57:19.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97d5ffb640 1 -- 172.21.15.18:0/3523084490 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4010430 con 0x7f97d8105360 2024-02-17T12:57:19.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.811+0000 7f97df042640 1 -- 172.21.15.18:0/3523084490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d81a4840 con 0x7f97d8105360 2024-02-17T12:57:19.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.814+0000 7f97d5ffb640 1 -- 172.21.15.18:0/3523084490 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f97c4005000 con 0x7f97d8105360 2024-02-17T12:57:19.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.814+0000 7f97e1c0c740 1 Processor -- start 2024-02-17T12:57:19.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.814+0000 7f97e1c0c740 1 -- start start 2024-02-17T12:57:19.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.814+0000 7f97d5ffb640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f97c80a2230 0x7f97c80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97e1c0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fb95630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97e1c0c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fb9e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97e1c0c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fba0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97d5ffb640 1 -- 172.21.15.18:0/3523084490 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f97c4014070 con 0x7f97d8105360 2024-02-17T12:57:19.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97e1c0c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609fb50960 con 0x55609fb9e750 2024-02-17T12:57:19.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97dd5df640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f97c80a2230 0x7f97c80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97e1c0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609fb04310 con 0x55609fb95250 2024-02-17T12:57:19.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97e1c0c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609fb01840 con 0x55609fb95b70 2024-02-17T12:57:19.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97dd5df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fba0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97dcdde640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fb95630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fb9e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97dd5df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fba0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43956/0 (socket says 172.21.15.18:43956) 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fb9e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:35068/0 (socket says 172.21.15.18:35068) 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97dd5df640 1 -- 172.21.15.18:0/3436907423 learned_addr learned my addr 172.21.15.18:0/3436907423 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97dd5df640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f97c80a2230 0x7f97c80a46f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f97d0012780 tx=0x7f97d0012b80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97d7fff640 1 -- 172.21.15.18:0/3436907423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 msgr2=0x55609fb95630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97d7fff640 1 --2- 172.21.15.18:0/3436907423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fb95630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97d7fff640 1 -- 172.21.15.18:0/3436907423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 msgr2=0x55609fba0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97d7fff640 1 --2- 172.21.15.18:0/3436907423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fba0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97d7fff640 1 -- 172.21.15.18:0/3436907423 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55609fabf060 con 0x55609fb95b70 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97dcdde640 1 --2- 172.21.15.18:0/3436907423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fb95630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.815+0000 7f97d7fff640 1 --2- 172.21.15.18:0/3436907423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fb9e210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f97c40cbbe0 tx=0x7f97c409ad80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ebcf07651abf000 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.817+0000 7f97c2ffd640 1 -- 172.21.15.18:0/3436907423 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c409efd0 con 0x55609fb95b70 2024-02-17T12:57:19.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.817+0000 7f97c2ffd640 1 -- 172.21.15.18:0/3436907423 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97c40a3620 con 0x55609fb95b70 2024-02-17T12:57:19.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.817+0000 7f97c2ffd640 1 -- 172.21.15.18:0/3436907423 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4005510 con 0x55609fb95b70 2024-02-17T12:57:19.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.818+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3436907423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 msgr2=0x55609fb9e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.818+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/3436907423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fb9e210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f97c40cbbe0 tx=0x7f97c409ad80 comp rx=0 tx=0).stop 2024-02-17T12:57:19.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.818+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3436907423 shutdown_connections 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.818+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/3436907423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fba0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.818+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/3436907423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fb9e210 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.818+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/3436907423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fb95630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.818+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3436907423 >> 172.21.15.18:0/3436907423 conn(0x55609fb8f980 msgr2=0x55609fb8fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.818+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3436907423 shutdown_connections 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.818+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3436907423 wait complete. 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.818+0000 7f97e1c0c740 1 -- start start 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.818+0000 7f97e1c0c740 1 Processor -- start 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97e1c0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fb9b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97e1c0c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fb9bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97e1c0c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fb9c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97e1c0c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609fafd020 con 0x55609fb9e750 2024-02-17T12:57:19.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97e1c0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609f7d0430 con 0x55609fb95250 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97e1c0c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609f78ce00 con 0x55609fb95b70 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97e1c0c740 10 client.0 _ll_register_callbacks cb 0x55609faa4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97dd5df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fb9c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97dcdde640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fb9b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fb9bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97dcdde640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fb9b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:33180/0 (socket says 172.21.15.18:33180) 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fb9bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:35072/0 (socket says 172.21.15.18:35072) 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97dd5df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fb9c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43968/0 (socket says 172.21.15.18:43968) 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97dcdde640 1 -- 172.21.15.18:0/2527125307 learned_addr learned my addr 172.21.15.18:0/2527125307 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97dcdde640 1 -- 172.21.15.18:0/2527125307 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 msgr2=0x55609fb9bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97dcdde640 1 --2- 172.21.15.18:0/2527125307 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fb9bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97dcdde640 1 -- 172.21.15.18:0/2527125307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 msgr2=0x55609fb9c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97dcdde640 1 --2- 172.21.15.18:0/2527125307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fb9c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97dcdde640 1 -- 172.21.15.18:0/2527125307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d81a4840 con 0x55609fb95250 2024-02-17T12:57:19.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97dd5df640 1 --2- 172.21.15.18:0/2527125307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fb9c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:19.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97dcdde640 1 --2- 172.21.15.18:0/2527125307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fb9b7b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009930 tx=0x7f97cc02c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5107861645cb5e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.819+0000 7f97b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:57:19.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97b7fff640 1 -- 172.21.15.18:0/2527125307 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc004490 con 0x55609fb95250 2024-02-17T12:57:19.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97b7fff640 1 -- 172.21.15.18:0/2527125307 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f97cc046070 con 0x55609fb95250 2024-02-17T12:57:19.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97e1c0c740 1 -- 172.21.15.18:0/2527125307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55609f7fc7e0 con 0x55609fb95250 2024-02-17T12:57:19.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97b7fff640 1 -- 172.21.15.18:0/2527125307 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97cc004df0 con 0x55609fb95250 2024-02-17T12:57:19.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97e1c0c740 10 client.27383 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:19.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97e1c0c740 1 -- 172.21.15.18:0/2527125307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55609fafb6a0 con 0x55609fb95250 2024-02-17T12:57:19.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97b7fff640 1 -- 172.21.15.18:0/2527125307 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55609f7fc7e0 con 0x55609fb95250 2024-02-17T12:57:19.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97b7fff640 1 -- 172.21.15.18:0/2527125307 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f97cc03c040 con 0x55609fb95250 2024-02-17T12:57:19.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97e1c0c740 10 client.27383 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:19.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97e1c0c740 10 client.27383 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:57:19.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97e1c0c740 20 client.27383 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:19.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/2527125307 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55609fbaeb90 0x55609fbb0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97e1c0c740 4 client.27383 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:19.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97e1c0c740 1 -- 172.21.15.18:0/2527125307 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55609f839720 con 0x55609fbaeb90 2024-02-17T12:57:19.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.820+0000 7f97d7fff640 1 --2- 172.21.15.18:0/2527125307 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55609fbaeb90 0x55609fbb0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.821+0000 7f97b7fff640 10 client.27383 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:57:19.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.821+0000 7f97d7fff640 1 --2- 172.21.15.18:0/2527125307 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55609fbaeb90 0x55609fbb0f80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a2f906988a5ea5f1 server_cookie=e37c692f6bcd3010 in_seq=0 out_seq=0 2024-02-17T12:57:19.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97b7fff640 1 -- 172.21.15.18:0/2527125307 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55609f839720 con 0x55609fbaeb90 2024-02-17T12:57:19.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97b7fff640 10 client.27383 handle_command_reply: tid=0 2024-02-17T12:57:19.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97e1c0c740 1 client.27383 shutdown 2024-02-17T12:57:19.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97e1c0c740 1 -- 172.21.15.18:0/2527125307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 msgr2=0x55609fb9b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/2527125307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fb9b7b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009930 tx=0x7f97cc02c710 comp rx=0 tx=0).stop 2024-02-17T12:57:19.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97e1c0c740 1 -- 172.21.15.18:0/2527125307 shutdown_connections 2024-02-17T12:57:19.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/2527125307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fb9c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/2527125307 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fb9bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/2527125307 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55609fbaeb90 0x55609fbb0f80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/2527125307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fb9b7b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97b7fff640 0 client.27383 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:57:19.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97e1c0c740 1 -- 172.21.15.18:0/2527125307 >> 172.21.15.18:0/2527125307 conn(0x55609fb8f980 msgr2=0x55609fb916d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97e1c0c740 1 -- 172.21.15.18:0/2527125307 shutdown_connections 2024-02-17T12:57:19.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97e1c0c740 1 -- 172.21.15.18:0/2527125307 wait complete. 2024-02-17T12:57:19.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.822+0000 7f97e1c0c740 20 client.27383 trim_cache size 0 max 16384 2024-02-17T12:57:19.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97e1c0c740 1 Processor -- start 2024-02-17T12:57:19.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97e1c0c740 1 -- start start 2024-02-17T12:57:19.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97e1c0c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95250 0x55609fb9c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97e1c0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95b70 0x55609fb914b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97e1c0c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fba5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97e1c0c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609fa82130 con 0x55609fb9e750 2024-02-17T12:57:19.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97e1c0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609fbb6190 con 0x55609fb95b70 2024-02-17T12:57:19.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97e1c0c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609fb8b2b0 con 0x55609fb95250 2024-02-17T12:57:19.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97dd5df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fba5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97dd5df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fba5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43984/0 (socket says 172.21.15.18:43984) 2024-02-17T12:57:19.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95b70 0x55609fb914b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97dd5df640 1 -- 172.21.15.18:0/1098005640 learned_addr learned my addr 172.21.15.18:0/1098005640 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.834+0000 7f97dcdde640 1 --2- 172.21.15.18:0/1098005640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95250 0x55609fb9c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.835+0000 7f97dd5df640 1 -- 172.21.15.18:0/1098005640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95250 msgr2=0x55609fb9c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.835+0000 7f97dd5df640 1 --2- 172.21.15.18:0/1098005640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95250 0x55609fb9c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.835+0000 7f97dd5df640 1 -- 172.21.15.18:0/1098005640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95b70 msgr2=0x55609fb914b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.835+0000 7f97dd5df640 1 --2- 172.21.15.18:0/1098005640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95b70 0x55609fb914b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.835+0000 7f97dd5df640 1 -- 172.21.15.18:0/1098005640 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55609fa97c20 con 0x55609fb9e750 2024-02-17T12:57:19.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.835+0000 7f97dcdde640 1 --2- 172.21.15.18:0/1098005640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95250 0x55609fb9c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:19.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.835+0000 7f97d7fff640 1 --2- 172.21.15.18:0/1098005640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95b70 0x55609fb914b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:19.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.835+0000 7f97dd5df640 1 --2- 172.21.15.18:0/1098005640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fba5600 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f97d000ec10 tx=0x7f97d001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c3c1ed8ad4fe5e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.835+0000 7f97c27fc640 1 -- 172.21.15.18:0/1098005640 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0024070 con 0x55609fb9e750 2024-02-17T12:57:19.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.835+0000 7f97c27fc640 1 -- 172.21.15.18:0/1098005640 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97d0003e50 con 0x55609fb9e750 2024-02-17T12:57:19.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.835+0000 7f97c27fc640 1 -- 172.21.15.18:0/1098005640 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0017040 con 0x55609fb9e750 2024-02-17T12:57:19.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.836+0000 7f97e1c0c740 1 -- 172.21.15.18:0/1098005640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 msgr2=0x55609fba5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.836+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/1098005640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fba5600 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f97d000ec10 tx=0x7f97d001d000 comp rx=0 tx=0).stop 2024-02-17T12:57:19.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.836+0000 7f97e1c0c740 1 -- 172.21.15.18:0/1098005640 shutdown_connections 2024-02-17T12:57:19.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.836+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/1098005640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fba5600 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.836+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/1098005640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95b70 0x55609fb914b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.836+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/1098005640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95250 0x55609fb9c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.836+0000 7f97e1c0c740 1 -- 172.21.15.18:0/1098005640 >> 172.21.15.18:0/1098005640 conn(0x55609fb8f980 msgr2=0x55609fb90da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.836+0000 7f97e1c0c740 1 -- 172.21.15.18:0/1098005640 shutdown_connections 2024-02-17T12:57:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.836+0000 7f97e1c0c740 1 -- 172.21.15.18:0/1098005640 wait complete. 2024-02-17T12:57:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.836+0000 7f97e1c0c740 1 -- start start 2024-02-17T12:57:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.836+0000 7f97e1c0c740 1 Processor -- start 2024-02-17T12:57:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97e1c0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fba48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97e1c0c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fba4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97e1c0c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fbff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97e1c0c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609fbff5f0 con 0x55609fb9e750 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97e1c0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609fbff730 con 0x55609fb95250 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97e1c0c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55609fbff8f0 con 0x55609fb95b70 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97e1c0c740 10 client.0 _ll_register_callbacks cb 0x7f97c400cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97dd5df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fbff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97dcdde640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fba48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fba4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97dd5df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fbff0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43990/0 (socket says 172.21.15.18:43990) 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fba4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:35092/0 (socket says 172.21.15.18:35092) 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97dd5df640 1 -- 172.21.15.18:0/3352854360 learned_addr learned my addr 172.21.15.18:0/3352854360 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97dcdde640 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fba48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:33194/0 (socket says 172.21.15.18:33194) 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97dd5df640 1 -- 172.21.15.18:0/3352854360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 msgr2=0x55609fba4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97dd5df640 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fba4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97dd5df640 1 -- 172.21.15.18:0/3352854360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 msgr2=0x55609fba48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97dd5df640 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fba48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97dd5df640 1 -- 172.21.15.18:0/3352854360 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c40b7040 con 0x55609fb9e750 2024-02-17T12:57:19.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97dcdde640 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fba48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:19.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97dd5df640 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fbff0b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009a90 tx=0x7f97d0005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa7fc7bfcc3c826 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:19.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97b7fff640 1 -- 172.21.15.18:0/3352854360 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0017070 con 0x55609fb9e750 2024-02-17T12:57:19.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97b7fff640 1 -- 172.21.15.18:0/3352854360 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f97d0003b70 con 0x55609fb9e750 2024-02-17T12:57:19.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.837+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3352854360 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97c40a3620 con 0x55609fb9e750 2024-02-17T12:57:19.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.838+0000 7f97b7fff640 1 -- 172.21.15.18:0/3352854360 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97d0024050 con 0x55609fb9e750 2024-02-17T12:57:19.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.838+0000 7f97e1c0c740 10 client.18198 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:19.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.838+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3352854360 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f97d000e9d0 con 0x55609fb9e750 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.838+0000 7f97b7fff640 1 -- 172.21.15.18:0/3352854360 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97c40a3620 con 0x55609fb9e750 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.838+0000 7f97b7fff640 1 -- 172.21.15.18:0/3352854360 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f97d0029530 con 0x55609fb9e750 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.838+0000 7f97e1c0c740 10 client.18198 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.838+0000 7f97e1c0c740 10 client.18198 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.838+0000 7f97e1c0c740 20 client.18198 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.838+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55609fc07130 0x55609fc09550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.838+0000 7f97e1c0c740 4 client.18198 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.838+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3352854360 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55609f888890 con 0x55609fc07130 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.838+0000 7f97dcdde640 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55609fc07130 0x55609fc09550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97dcdde640 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55609fc07130 0x55609fc09550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3a7efa8be33428 server_cookie=fb8fb170bf59f1d7 in_seq=0 out_seq=0 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97b7fff640 10 client.18198 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97b7fff640 1 -- 172.21.15.18:0/3352854360 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55609f888890 con 0x55609fc07130 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97b7fff640 10 client.18198 handle_command_reply: tid=0 2024-02-17T12:57:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97e1c0c740 1 client.18198 shutdown 2024-02-17T12:57:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3352854360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 msgr2=0x55609fbff0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fbff0b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009a90 tx=0x7f97d0005b70 comp rx=0 tx=0).stop 2024-02-17T12:57:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3352854360 shutdown_connections 2024-02-17T12:57:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55609fb9e750 0x55609fbff0b0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55609fb95b70 0x55609fba4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55609fc07130 0x55609fc09550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97e1c0c740 1 --2- 172.21.15.18:0/3352854360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55609fb95250 0x55609fba48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97b7fff640 0 client.18198 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:57:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3352854360 >> 172.21.15.18:0/3352854360 conn(0x55609fb8f980 msgr2=0x55609fb91630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3352854360 shutdown_connections 2024-02-17T12:57:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97e1c0c740 1 -- 172.21.15.18:0/3352854360 wait complete. 2024-02-17T12:57:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.839+0000 7f97e1c0c740 20 client.18198 trim_cache size 0 max 16384 2024-02-17T12:57:19.839 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:57:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.840+0000 7f97df042640 1 -- 172.21.15.18:0/3523084490 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f97c80a2230 msgr2=0x7f97c80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.840+0000 7f97df042640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f97c80a2230 0x7f97c80a46f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f97d0012780 tx=0x7f97d0012b80 comp rx=0 tx=0).stop 2024-02-17T12:57:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.840+0000 7f97df042640 1 -- 172.21.15.18:0/3523084490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d8105360 msgr2=0x7f97d8198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.840+0000 7f97df042640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d8105360 0x7f97d8198b90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f97c400e9c0 tx=0x7f97c400ee80 comp rx=0 tx=0).stop 2024-02-17T12:57:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.840+0000 7f97df042640 1 -- 172.21.15.18:0/3523084490 shutdown_connections 2024-02-17T12:57:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.840+0000 7f97df042640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f97c80a2230 0x7f97c80a46f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.840+0000 7f97df042640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97d81990d0 0x7f97d81a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.840+0000 7f97df042640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d8105360 0x7f97d8198b90 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.840+0000 7f97df042640 1 --2- 172.21.15.18:0/3523084490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d81002e0 0x7f97d8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.840+0000 7f97df042640 1 -- 172.21.15.18:0/3523084490 >> 172.21.15.18:0/3523084490 conn(0x7f97d80771e0 msgr2=0x7f97d80fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.840+0000 7f97df042640 1 -- 172.21.15.18:0/3523084490 shutdown_connections 2024-02-17T12:57:19.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.840+0000 7f97df042640 1 -- 172.21.15.18:0/3523084490 wait complete. 2024-02-17T12:57:19.842 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:57:19.842 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:57:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.965+0000 7f641f52c640 1 -- 172.21.15.18:0/2094030619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6418106430 msgr2=0x7f641810e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.965+0000 7f641f52c640 1 --2- 172.21.15.18:0/2094030619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6418106430 0x7f641810e000 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6410009930 tx=0x7f641002bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.965+0000 7f641f52c640 1 -- 172.21.15.18:0/2094030619 shutdown_connections 2024-02-17T12:57:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.965+0000 7f641f52c640 1 --2- 172.21.15.18:0/2094030619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6418106430 0x7f641810e000 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.965+0000 7f641f52c640 1 --2- 172.21.15.18:0/2094030619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6418068f80 0x7f6418105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.965+0000 7f641f52c640 1 --2- 172.21.15.18:0/2094030619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64180685d0 0x7f64180689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.965+0000 7f641f52c640 1 -- 172.21.15.18:0/2094030619 >> 172.21.15.18:0/2094030619 conn(0x7f6418077180 msgr2=0x7f6418077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.965+0000 7f641f52c640 1 -- 172.21.15.18:0/2094030619 shutdown_connections 2024-02-17T12:57:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.965+0000 7f641f52c640 1 -- 172.21.15.18:0/2094030619 wait complete. 2024-02-17T12:57:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.965+0000 7f641f52c640 1 Processor -- start 2024-02-17T12:57:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641f52c640 1 -- start start 2024-02-17T12:57:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641f52c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64180685d0 0x7f641819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641f52c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6418106430 0x7f641819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641f52c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6418197620 0x7f6418197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641f52c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6418198010 con 0x7f6418197620 2024-02-17T12:57:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641f52c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6418198180 con 0x7f6418106430 2024-02-17T12:57:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641f52c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6418198340 con 0x7f64180685d0 2024-02-17T12:57:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641d2c8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64180685d0 0x7f641819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641dac9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6418197620 0x7f6418197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641cac7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6418106430 0x7f641819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641d2c8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64180685d0 0x7f641819d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:35118/0 (socket says 172.21.15.18:35118) 2024-02-17T12:57:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641d2c8640 1 -- 172.21.15.18:0/1598847213 learned_addr learned my addr 172.21.15.18:0/1598847213 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641cac7640 1 -- 172.21.15.18:0/1598847213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64180685d0 msgr2=0x7f641819d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641cac7640 1 --2- 172.21.15.18:0/1598847213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64180685d0 0x7f641819d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641cac7640 1 -- 172.21.15.18:0/1598847213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6418197620 msgr2=0x7f6418197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641cac7640 1 --2- 172.21.15.18:0/1598847213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6418197620 0x7f6418197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641cac7640 1 -- 172.21.15.18:0/1598847213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6408009490 con 0x7f6418106430 2024-02-17T12:57:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.966+0000 7f641cac7640 1 --2- 172.21.15.18:0/1598847213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6418106430 0x7f641819d960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f640800e9c0 tx=0x7f640800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.967+0000 7f640e7fc640 1 -- 172.21.15.18:0/1598847213 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640800cd80 con 0x7f6418106430 2024-02-17T12:57:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.967+0000 7f641f52c640 1 -- 172.21.15.18:0/1598847213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6410009560 con 0x7f6418106430 2024-02-17T12:57:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.967+0000 7f641f52c640 1 -- 172.21.15.18:0/1598847213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64181a46b0 con 0x7f6418106430 2024-02-17T12:57:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.967+0000 7f640e7fc640 1 -- 172.21.15.18:0/1598847213 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64080035f0 con 0x7f6418106430 2024-02-17T12:57:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.967+0000 7f640e7fc640 1 -- 172.21.15.18:0/1598847213 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6408010690 con 0x7f6418106430 2024-02-17T12:57:19.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.969+0000 7f640e7fc640 1 -- 172.21.15.18:0/1598847213 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6408003af0 con 0x7f6418106430 2024-02-17T12:57:19.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.969+0000 7f64221de740 1 Processor -- start 2024-02-17T12:57:19.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.969+0000 7f64221de740 1 -- start start 2024-02-17T12:57:19.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.969+0000 7f64221de740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748cf80 0x56458748d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.969+0000 7f64221de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x56458749a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f64221de740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458749ab60 0x56458749cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f64221de740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564587448960 con 0x56458749ab60 2024-02-17T12:57:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f64221de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645873fc310 con 0x56458748d8a0 2024-02-17T12:57:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f64221de740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645873f9840 con 0x56458748cf80 2024-02-17T12:57:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f640e7fc640 1 --2- 172.21.15.18:0/1598847213 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64040a2230 0x7f64040a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641d2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x56458749a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641dac9640 1 --2- 172.21.15.18:0/1598847213 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64040a2230 0x7f64040a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641cac7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458749ab60 0x56458749cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641d2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x56458749a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:33216/0 (socket says 172.21.15.18:33216) 2024-02-17T12:57:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641d2c8640 1 -- 172.21.15.18:0/60765151 learned_addr learned my addr 172.21.15.18:0/60765151 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641dac9640 1 --2- 172.21.15.18:0/60765151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748cf80 0x56458748d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f640e7fc640 1 -- 172.21.15.18:0/1598847213 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6408014070 con 0x7f6418106430 2024-02-17T12:57:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641d2c8640 1 -- 172.21.15.18:0/60765151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748cf80 msgr2=0x56458748d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641d2c8640 1 --2- 172.21.15.18:0/60765151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748cf80 0x56458748d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641d2c8640 1 -- 172.21.15.18:0/60765151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458749ab60 msgr2=0x56458749cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641d2c8640 1 --2- 172.21.15.18:0/60765151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458749ab60 0x56458749cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641d2c8640 1 -- 172.21.15.18:0/60765151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645873b7060 con 0x56458748d8a0 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641dac9640 1 --2- 172.21.15.18:0/1598847213 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64040a2230 0x7f64040a46f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6410005940 tx=0x7f64100046c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f641d2c8640 1 --2- 172.21.15.18:0/60765151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x56458749a620 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f64140115f0 tx=0x7f6414011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a14f86fa5323151 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f64037fe640 1 -- 172.21.15.18:0/60765151 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414009bf0 con 0x56458748d8a0 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.970+0000 7f64037fe640 1 -- 172.21.15.18:0/60765151 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f641400f640 con 0x56458748d8a0 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.971+0000 7f64037fe640 1 -- 172.21.15.18:0/60765151 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414005390 con 0x56458748d8a0 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.971+0000 7f64221de740 1 -- 172.21.15.18:0/60765151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 msgr2=0x56458749a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.971+0000 7f64221de740 1 --2- 172.21.15.18:0/60765151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x56458749a620 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f64140115f0 tx=0x7f6414011ab0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.971+0000 7f64221de740 1 -- 172.21.15.18:0/60765151 shutdown_connections 2024-02-17T12:57:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.971+0000 7f64221de740 1 --2- 172.21.15.18:0/60765151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458749ab60 0x56458749cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.971+0000 7f64221de740 1 --2- 172.21.15.18:0/60765151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x56458749a620 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.971+0000 7f64221de740 1 --2- 172.21.15.18:0/60765151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748cf80 0x56458748d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.971+0000 7f64221de740 1 -- 172.21.15.18:0/60765151 >> 172.21.15.18:0/60765151 conn(0x564587487980 msgr2=0x564587487d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.971+0000 7f64221de740 1 -- 172.21.15.18:0/60765151 shutdown_connections 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.971+0000 7f64221de740 1 -- 172.21.15.18:0/60765151 wait complete. 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f64221de740 1 -- start start 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f64221de740 1 Processor -- start 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f64221de740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748cf80 0x564587497a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f64221de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x564587497f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f64221de740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458749ab60 0x5645874984b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f64221de740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645873f5020 con 0x56458749ab60 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f64221de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645870c8430 con 0x56458748d8a0 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f64221de740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564587084e00 con 0x56458748cf80 2024-02-17T12:57:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f64221de740 10 client.0 _ll_register_callbacks cb 0x56458739ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f641dac9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458749ab60 0x5645874984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f641dac9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458749ab60 0x5645874984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44010/0 (socket says 172.21.15.18:44010) 2024-02-17T12:57:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f641dac9640 1 -- 172.21.15.18:0/1791641703 learned_addr learned my addr 172.21.15.18:0/1791641703 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f641d2c8640 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748cf80 0x564587497a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f641cac7640 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x564587497f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f641dac9640 1 -- 172.21.15.18:0/1791641703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748cf80 msgr2=0x564587497a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f641dac9640 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748cf80 0x564587497a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f641dac9640 1 -- 172.21.15.18:0/1791641703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 msgr2=0x564587497f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.972+0000 7f641dac9640 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x564587497f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f641dac9640 1 -- 172.21.15.18:0/1791641703 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6414009d50 con 0x56458749ab60 2024-02-17T12:57:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f641dac9640 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458749ab60 0x5645874984b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6410035000 tx=0x7f64100058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01244a49e6ab66f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f6400ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f6400ff9640 1 -- 172.21.15.18:0/1791641703 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410048070 con 0x56458749ab60 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f64221de740 1 -- 172.21.15.18:0/1791641703 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645870f47e0 con 0x56458749ab60 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f6400ff9640 1 -- 172.21.15.18:0/1791641703 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6410005000 con 0x56458749ab60 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f6400ff9640 1 -- 172.21.15.18:0/1791641703 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64100051c0 con 0x56458749ab60 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f64221de740 10 client.18210 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f64221de740 1 -- 172.21.15.18:0/1791641703 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5645873f36a0 con 0x56458749ab60 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f6400ff9640 1 -- 172.21.15.18:0/1791641703 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5645870f47e0 con 0x56458749ab60 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f6400ff9640 1 -- 172.21.15.18:0/1791641703 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6410003c60 con 0x56458749ab60 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f64221de740 10 client.18210 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f64221de740 10 client.18210 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f64221de740 20 client.18210 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f64221de740 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5645874aaea0 0x5645874ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f64221de740 4 client.18210 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.973+0000 7f64221de740 1 -- 172.21.15.18:0/1791641703 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564587131720 con 0x5645874aaea0 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.974+0000 7f641d2c8640 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5645874aaea0 0x5645874ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.974+0000 7f641d2c8640 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5645874aaea0 0x5645874ad2c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce5ca4500edfd34a server_cookie=b9d4a64b08933e7c in_seq=0 out_seq=0 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.974+0000 7f6400ff9640 10 client.18210 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f6400ff9640 1 -- 172.21.15.18:0/1791641703 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564587131720 con 0x5645874aaea0 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f6400ff9640 10 client.18210 handle_command_reply: tid=0 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f64221de740 1 client.18210 shutdown 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f64221de740 1 -- 172.21.15.18:0/1791641703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458749ab60 msgr2=0x5645874984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f64221de740 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458749ab60 0x5645874984b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6410035000 tx=0x7f64100058d0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f64221de740 1 -- 172.21.15.18:0/1791641703 shutdown_connections 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f64221de740 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5645874aaea0 0x5645874ad2c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f64221de740 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458749ab60 0x5645874984b0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f64221de740 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x564587497f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f6400ff9640 0 client.18210 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f64221de740 1 --2- 172.21.15.18:0/1791641703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748cf80 0x564587497a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f64221de740 1 -- 172.21.15.18:0/1791641703 >> 172.21.15.18:0/1791641703 conn(0x564587487980 msgr2=0x5645874896d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f64221de740 1 -- 172.21.15.18:0/1791641703 shutdown_connections 2024-02-17T12:57:19.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f64221de740 1 -- 172.21.15.18:0/1791641703 wait complete. 2024-02-17T12:57:19.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.975+0000 7f64221de740 20 client.18210 trim_cache size 0 max 16384 2024-02-17T12:57:19.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f64221de740 1 Processor -- start 2024-02-17T12:57:19.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f64221de740 1 -- start start 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f64221de740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 0x56458749a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f64221de740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748d8a0 0x564587492d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f64221de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458749ab60 0x5645874932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f64221de740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458737a130 con 0x56458748cf80 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f64221de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645874928f0 con 0x56458749ab60 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f64221de740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458748c5c0 con 0x56458748d8a0 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f641d2c8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 0x56458749a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f641d2c8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 0x56458749a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44014/0 (socket says 172.21.15.18:44014) 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f641d2c8640 1 -- 172.21.15.18:0/1974167120 learned_addr learned my addr 172.21.15.18:0/1974167120 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f641cac7640 1 --2- 172.21.15.18:0/1974167120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748d8a0 0x564587492d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f641dac9640 1 --2- 172.21.15.18:0/1974167120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458749ab60 0x5645874932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f641d2c8640 1 -- 172.21.15.18:0/1974167120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748d8a0 msgr2=0x564587492d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f641d2c8640 1 --2- 172.21.15.18:0/1974167120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748d8a0 0x564587492d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f641d2c8640 1 -- 172.21.15.18:0/1974167120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458749ab60 msgr2=0x5645874932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f641d2c8640 1 --2- 172.21.15.18:0/1974167120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458749ab60 0x5645874932a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.991+0000 7f641d2c8640 1 -- 172.21.15.18:0/1974167120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56458738fc20 con 0x56458748cf80 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.992+0000 7f641d2c8640 1 --2- 172.21.15.18:0/1974167120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 0x56458749a160 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6414004f00 tx=0x7f64140092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec54ff6bcf69ebe server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.992+0000 7f6402ffd640 1 -- 172.21.15.18:0/1974167120 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414004cc0 con 0x56458748cf80 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.992+0000 7f6402ffd640 1 -- 172.21.15.18:0/1974167120 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f641402c070 con 0x56458748cf80 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.992+0000 7f6402ffd640 1 -- 172.21.15.18:0/1974167120 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414028440 con 0x56458748cf80 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 -- 172.21.15.18:0/1974167120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 msgr2=0x56458749a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 --2- 172.21.15.18:0/1974167120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 0x56458749a160 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6414004f00 tx=0x7f64140092e0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 -- 172.21.15.18:0/1974167120 shutdown_connections 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 --2- 172.21.15.18:0/1974167120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458749ab60 0x5645874932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 --2- 172.21.15.18:0/1974167120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458748d8a0 0x564587492d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 --2- 172.21.15.18:0/1974167120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 0x56458749a160 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 -- 172.21.15.18:0/1974167120 >> 172.21.15.18:0/1974167120 conn(0x564587487980 msgr2=0x564587482a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 -- 172.21.15.18:0/1974167120 shutdown_connections 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 -- 172.21.15.18:0/1974167120 wait complete. 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 -- start start 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 Processor -- start 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 0x5645874a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x5645874a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458749ab60 0x5645874a1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645875014c0 con 0x56458748cf80 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564587501630 con 0x56458748d8a0 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645875017f0 con 0x56458749ab60 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.993+0000 7f64221de740 10 client.0 _ll_register_callbacks cb 0x7f6410009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f641d2c8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 0x5645874a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f641d2c8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 0x5645874a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44020/0 (socket says 172.21.15.18:44020) 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f641d2c8640 1 -- 172.21.15.18:0/544595505 learned_addr learned my addr 172.21.15.18:0/544595505 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f641cac7640 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x5645874a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f641dac9640 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458749ab60 0x5645874a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f641d2c8640 1 -- 172.21.15.18:0/544595505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458749ab60 msgr2=0x5645874a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f641d2c8640 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458749ab60 0x5645874a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f641d2c8640 1 -- 172.21.15.18:0/544595505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 msgr2=0x5645874a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f641d2c8640 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x5645874a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f641d2c8640 1 -- 172.21.15.18:0/544595505 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645873b7060 con 0x56458748cf80 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f641cac7640 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x5645874a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f641d2c8640 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 0x5645874a06c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f64140115b0 tx=0x7f6414004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67602e1299627b21 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f6400ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f6400ff9640 1 -- 172.21.15.18:0/544595505 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641400fe50 con 0x56458748cf80 2024-02-17T12:57:19.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f6400ff9640 1 -- 172.21.15.18:0/544595505 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f641400f830 con 0x56458748cf80 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f64221de740 1 -- 172.21.15.18:0/544595505 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564587497860 con 0x56458748cf80 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f6400ff9640 1 -- 172.21.15.18:0/544595505 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f641400f9f0 con 0x56458748cf80 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f64221de740 10 client.18216 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.994+0000 7f64221de740 1 -- 172.21.15.18:0/544595505 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5645873b58c0 con 0x56458748cf80 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.995+0000 7f6400ff9640 1 -- 172.21.15.18:0/544595505 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564587497860 con 0x56458748cf80 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.995+0000 7f6400ff9640 1 -- 172.21.15.18:0/544595505 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6414005020 con 0x56458748cf80 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.995+0000 7f64221de740 10 client.18216 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.995+0000 7f64221de740 10 client.18216 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.995+0000 7f64221de740 20 client.18216 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.995+0000 7f64221de740 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x564587502c40 0x564587505060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.995+0000 7f64221de740 4 client.18216 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.995+0000 7f64221de740 1 -- 172.21.15.18:0/544595505 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f641400f640 con 0x564587502c40 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.995+0000 7f641cac7640 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x564587502c40 0x564587505060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:19.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.995+0000 7f6400ff9640 10 client.18216 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.995+0000 7f641cac7640 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x564587502c40 0x564587505060 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=67cd967c3570a690 server_cookie=f3e132a23c0596c5 in_seq=0 out_seq=0 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f6400ff9640 1 -- 172.21.15.18:0/544595505 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f641400f640 con 0x564587502c40 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f6400ff9640 10 client.18216 handle_command_reply: tid=0 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f64221de740 1 client.18216 shutdown 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f64221de740 1 -- 172.21.15.18:0/544595505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 msgr2=0x5645874a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f64221de740 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 0x5645874a06c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f64140115b0 tx=0x7f6414004cc0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f64221de740 1 -- 172.21.15.18:0/544595505 shutdown_connections 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f64221de740 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x564587502c40 0x564587505060 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f64221de740 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56458749ab60 0x5645874a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f6400ff9640 0 client.18216 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f64221de740 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56458748d8a0 0x5645874a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f64221de740 1 --2- 172.21.15.18:0/544595505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56458748cf80 0x5645874a06c0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f64221de740 1 -- 172.21.15.18:0/544595505 >> 172.21.15.18:0/544595505 conn(0x564587487980 msgr2=0x564587495ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f64221de740 1 -- 172.21.15.18:0/544595505 shutdown_connections 2024-02-17T12:57:19.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f64221de740 1 -- 172.21.15.18:0/544595505 wait complete. 2024-02-17T12:57:19.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f64221de740 20 client.18216 trim_cache size 0 max 16384 2024-02-17T12:57:19.994 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:57:19.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f641f52c640 1 -- 172.21.15.18:0/1598847213 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64040a2230 msgr2=0x7f64040a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f641f52c640 1 --2- 172.21.15.18:0/1598847213 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64040a2230 0x7f64040a46f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6410005940 tx=0x7f64100046c0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f641f52c640 1 -- 172.21.15.18:0/1598847213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6418106430 msgr2=0x7f641819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:19.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f641f52c640 1 --2- 172.21.15.18:0/1598847213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6418106430 0x7f641819d960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f640800e9c0 tx=0x7f640800ee80 comp rx=0 tx=0).stop 2024-02-17T12:57:19.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f641f52c640 1 -- 172.21.15.18:0/1598847213 shutdown_connections 2024-02-17T12:57:19.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f641f52c640 1 --2- 172.21.15.18:0/1598847213 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64040a2230 0x7f64040a46f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f641f52c640 1 --2- 172.21.15.18:0/1598847213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6418197620 0x7f6418197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f641f52c640 1 --2- 172.21.15.18:0/1598847213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6418106430 0x7f641819d960 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f641f52c640 1 --2- 172.21.15.18:0/1598847213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64180685d0 0x7f641819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:19.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.996+0000 7f641f52c640 1 -- 172.21.15.18:0/1598847213 >> 172.21.15.18:0/1598847213 conn(0x7f6418077180 msgr2=0x7f64180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:19.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.997+0000 7f641f52c640 1 -- 172.21.15.18:0/1598847213 shutdown_connections 2024-02-17T12:57:19.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:19.997+0000 7f641f52c640 1 -- 172.21.15.18:0/1598847213 wait complete. 2024-02-17T12:57:19.996 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:19 smithi018 ceph-mon[32625]: pgmap v230: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-02-17T12:57:19.996 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:19 smithi018 ceph-mon[32625]: osdmap e94: 12 total, 12 up, 12 in 2024-02-17T12:57:19.996 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:19 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2186676137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:57:19.997 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:57:19.998 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:57:20.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:19 smithi119 ceph-mon[36188]: pgmap v230: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-02-17T12:57:20.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:19 smithi119 ceph-mon[36188]: osdmap e94: 12 total, 12 up, 12 in 2024-02-17T12:57:20.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:19 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2186676137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:57:20.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.117+0000 7f5540f77640 1 -- 172.21.15.18:0/3784611026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f553c100cc0 msgr2=0x7f553c104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.117+0000 7f5540f77640 1 --2- 172.21.15.18:0/3784611026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f553c100cc0 0x7f553c104d00 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f5524009930 tx=0x7f552402bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:20.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.117+0000 7f5540f77640 1 -- 172.21.15.18:0/3784611026 shutdown_connections 2024-02-17T12:57:20.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.117+0000 7f5540f77640 1 --2- 172.21.15.18:0/3784611026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f553c1053e0 0x7f553c10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.117+0000 7f5540f77640 1 --2- 172.21.15.18:0/3784611026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f553c100cc0 0x7f553c104d00 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.117+0000 7f5540f77640 1 --2- 172.21.15.18:0/3784611026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f553c100310 0x7f553c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.117+0000 7f5540f77640 1 -- 172.21.15.18:0/3784611026 >> 172.21.15.18:0/3784611026 conn(0x7f553c0771e0 msgr2=0x7f553c0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.117+0000 7f5540f77640 1 -- 172.21.15.18:0/3784611026 shutdown_connections 2024-02-17T12:57:20.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.117+0000 7f5540f77640 1 -- 172.21.15.18:0/3784611026 wait complete. 2024-02-17T12:57:20.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f5540f77640 1 Processor -- start 2024-02-17T12:57:20.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f5540f77640 1 -- start start 2024-02-17T12:57:20.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f5540f77640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f553c100310 0x7f553c101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f5540f77640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f553c1053e0 0x7f553c1022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f5540f77640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f553c102820 0x7f553c1a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f5540f77640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1a6020 con 0x7f553c100310 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f5540f77640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1a6190 con 0x7f553c1053e0 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f5540f77640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1a6350 con 0x7f553c102820 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f553a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f553c100310 0x7f553c101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f553a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f553c100310 0x7f553c101da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44046/0 (socket says 172.21.15.18:44046) 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f553a59c640 1 -- 172.21.15.18:0/1954146259 learned_addr learned my addr 172.21.15.18:0/1954146259 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f5531d9b640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f553c1053e0 0x7f553c1022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.118+0000 7f553ad9d640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f553c102820 0x7f553c1a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.119+0000 7f553a59c640 1 -- 172.21.15.18:0/1954146259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f553c102820 msgr2=0x7f553c1a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.119+0000 7f553a59c640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f553c102820 0x7f553c1a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.119+0000 7f553a59c640 1 -- 172.21.15.18:0/1954146259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f553c1053e0 msgr2=0x7f553c1022e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.119+0000 7f553a59c640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f553c1053e0 0x7f553c1022e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.119+0000 7f553a59c640 1 -- 172.21.15.18:0/1954146259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5524009560 con 0x7f553c100310 2024-02-17T12:57:20.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.119+0000 7f553a59c640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f553c100310 0x7f553c101da0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f552c00d770 tx=0x7f552c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.119+0000 7f5533fff640 1 -- 172.21.15.18:0/1954146259 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c009c30 con 0x7f553c100310 2024-02-17T12:57:20.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.119+0000 7f5540f77640 1 -- 172.21.15.18:0/1954146259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553c1a6630 con 0x7f553c100310 2024-02-17T12:57:20.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.119+0000 7f5533fff640 1 -- 172.21.15.18:0/1954146259 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f552c00de80 con 0x7f553c100310 2024-02-17T12:57:20.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.119+0000 7f5533fff640 1 -- 172.21.15.18:0/1954146259 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c004e80 con 0x7f553c100310 2024-02-17T12:57:20.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.119+0000 7f5540f77640 1 -- 172.21.15.18:0/1954146259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553c1a6ac0 con 0x7f553c100310 2024-02-17T12:57:20.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.122+0000 7f5533fff640 1 -- 172.21.15.18:0/1954146259 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f552c005020 con 0x7f553c100310 2024-02-17T12:57:20.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.122+0000 7f5543c22740 1 Processor -- start 2024-02-17T12:57:20.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.122+0000 7f5543c22740 1 -- start start 2024-02-17T12:57:20.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.122+0000 7f5543c22740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcef5250 0x561fbcef5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.122+0000 7f5543c22740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 0x561fbcefe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.122+0000 7f5543c22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcefe750 0x561fbcf00b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f5543c22740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbceb0960 con 0x561fbcef5b70 2024-02-17T12:57:20.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f5543c22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbce64310 con 0x561fbcefe750 2024-02-17T12:57:20.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f5543c22740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbce61840 con 0x561fbcef5250 2024-02-17T12:57:20.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 0x561fbcefe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553ad9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcef5250 0x561fbcef5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 0x561fbcefe210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44060/0 (socket says 172.21.15.18:44060) 2024-02-17T12:57:20.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553a59c640 1 -- 172.21.15.18:0/90391171 learned_addr learned my addr 172.21.15.18:0/90391171 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f5531d9b640 1 --2- 172.21.15.18:0/90391171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcefe750 0x561fbcf00b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f5533fff640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f55280a2260 0x7f55280a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553ad9d640 1 -- 172.21.15.18:0/90391171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcefe750 msgr2=0x561fbcf00b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553ad9d640 1 --2- 172.21.15.18:0/90391171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcefe750 0x561fbcf00b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553ad9d640 1 -- 172.21.15.18:0/90391171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 msgr2=0x561fbcefe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553ad9d640 1 --2- 172.21.15.18:0/90391171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 0x561fbcefe210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553ad9d640 1 -- 172.21.15.18:0/90391171 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fbce1f060 con 0x561fbcef5250 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553a59c640 1 --2- 172.21.15.18:0/90391171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 0x561fbcefe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f5533fff640 1 -- 172.21.15.18:0/1954146259 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f552c0cda00 con 0x7f553c100310 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553ad9d640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f55280a2260 0x7f55280a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553ad9d640 1 --2- 172.21.15.18:0/90391171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcef5250 0x561fbcef5630 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5534005f10 tx=0x7f5534010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e02410c53f19a24 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f5530d99640 1 -- 172.21.15.18:0/90391171 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534022070 con 0x561fbcef5250 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.123+0000 7f553ad9d640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f55280a2260 0x7f55280a4720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5534021000 tx=0x7f553401e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.124+0000 7f5530d99640 1 -- 172.21.15.18:0/90391171 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f553401d910 con 0x561fbcef5250 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.124+0000 7f5530d99640 1 -- 172.21.15.18:0/90391171 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534011aa0 con 0x561fbcef5250 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.124+0000 7f5543c22740 1 -- 172.21.15.18:0/90391171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcef5250 msgr2=0x561fbcef5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.124+0000 7f5543c22740 1 --2- 172.21.15.18:0/90391171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcef5250 0x561fbcef5630 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5534005f10 tx=0x7f5534010860 comp rx=0 tx=0).stop 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.124+0000 7f5543c22740 1 -- 172.21.15.18:0/90391171 shutdown_connections 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.124+0000 7f5543c22740 1 --2- 172.21.15.18:0/90391171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcefe750 0x561fbcf00b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.124+0000 7f5543c22740 1 --2- 172.21.15.18:0/90391171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 0x561fbcefe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.124+0000 7f5543c22740 1 --2- 172.21.15.18:0/90391171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcef5250 0x561fbcef5630 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.124+0000 7f5543c22740 1 -- 172.21.15.18:0/90391171 >> 172.21.15.18:0/90391171 conn(0x561fbceef980 msgr2=0x561fbceefd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.124+0000 7f5543c22740 1 -- 172.21.15.18:0/90391171 shutdown_connections 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.124+0000 7f5543c22740 1 -- 172.21.15.18:0/90391171 wait complete. 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f5543c22740 1 -- start start 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f5543c22740 1 Processor -- start 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f5543c22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5250 0x561fbcefb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f5543c22740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcef5b70 0x561fbcefbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f5543c22740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcefe750 0x561fbcefc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f5543c22740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbce5d020 con 0x561fbcefe750 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f5543c22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbcb30430 con 0x561fbcef5250 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f5543c22740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbcaece00 con 0x561fbcef5b70 2024-02-17T12:57:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f5543c22740 10 client.0 _ll_register_callbacks cb 0x561fbce04cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f553ad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcefe750 0x561fbcefc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f553ad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcefe750 0x561fbcefc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44074/0 (socket says 172.21.15.18:44074) 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f553ad9d640 1 -- 172.21.15.18:0/2285292308 learned_addr learned my addr 172.21.15.18:0/2285292308 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f553a59c640 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5250 0x561fbcefb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f5531d9b640 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcef5b70 0x561fbcefbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f553ad9d640 1 -- 172.21.15.18:0/2285292308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcef5b70 msgr2=0x561fbcefbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f553ad9d640 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcef5b70 0x561fbcefbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f553ad9d640 1 -- 172.21.15.18:0/2285292308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5250 msgr2=0x561fbcefb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f553ad9d640 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5250 0x561fbcefb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.125+0000 7f553ad9d640 1 -- 172.21.15.18:0/2285292308 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5534009b60 con 0x561fbcefe750 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f553ad9d640 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcefe750 0x561fbcefc1c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5534013280 tx=0x7f55340132b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e15b356ad0c7132 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5521ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5521ffb640 1 -- 172.21.15.18:0/2285292308 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534022030 con 0x561fbcefe750 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5521ffb640 1 -- 172.21.15.18:0/2285292308 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5534023070 con 0x561fbcefe750 2024-02-17T12:57:20.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5543c22740 1 -- 172.21.15.18:0/2285292308 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fbcb5c7e0 con 0x561fbcefe750 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5521ffb640 1 -- 172.21.15.18:0/2285292308 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f553402b070 con 0x561fbcefe750 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5543c22740 10 client.18228 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5543c22740 1 -- 172.21.15.18:0/2285292308 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x561fbce5b6a0 con 0x561fbcefe750 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5521ffb640 1 -- 172.21.15.18:0/2285292308 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f553403c020 con 0x561fbcefe750 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5521ffb640 1 -- 172.21.15.18:0/2285292308 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5534011c20 con 0x561fbcefe750 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5543c22740 10 client.18228 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5543c22740 10 client.18228 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5543c22740 20 client.18228 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5543c22740 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561fbcf0eab0 0x561fbcf10ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5543c22740 4 client.18228 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.126+0000 7f5543c22740 1 -- 172.21.15.18:0/2285292308 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fbcb99720 con 0x561fbcf0eab0 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.127+0000 7f553a59c640 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561fbcf0eab0 0x561fbcf10ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.127+0000 7f5521ffb640 10 client.18228 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:57:20.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.127+0000 7f553a59c640 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561fbcf0eab0 0x561fbcf10ed0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9ee67d03e0d8646 server_cookie=7e08148790d950d6 in_seq=0 out_seq=0 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.127+0000 7f5521ffb640 1 -- 172.21.15.18:0/2285292308 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561fbcb99720 con 0x561fbcf0eab0 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.127+0000 7f5521ffb640 10 client.18228 handle_command_reply: tid=0 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.127+0000 7f5543c22740 1 client.18228 shutdown 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.128+0000 7f5543c22740 1 -- 172.21.15.18:0/2285292308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcefe750 msgr2=0x561fbcefc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.128+0000 7f5543c22740 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcefe750 0x561fbcefc1c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5534013280 tx=0x7f55340132b0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.128+0000 7f5543c22740 1 -- 172.21.15.18:0/2285292308 shutdown_connections 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.128+0000 7f5543c22740 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561fbcf0eab0 0x561fbcf10ed0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.128+0000 7f5543c22740 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcefe750 0x561fbcefc1c0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.128+0000 7f5543c22740 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcef5b70 0x561fbcefbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.128+0000 7f5521ffb640 0 client.18228 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.128+0000 7f5543c22740 1 --2- 172.21.15.18:0/2285292308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5250 0x561fbcefb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.128+0000 7f5543c22740 1 -- 172.21.15.18:0/2285292308 >> 172.21.15.18:0/2285292308 conn(0x561fbceef980 msgr2=0x561fbcef16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.128+0000 7f5543c22740 1 -- 172.21.15.18:0/2285292308 shutdown_connections 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.128+0000 7f5543c22740 1 -- 172.21.15.18:0/2285292308 wait complete. 2024-02-17T12:57:20.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.128+0000 7f5543c22740 20 client.18228 trim_cache size 0 max 16384 2024-02-17T12:57:20.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.139+0000 7f5543c22740 1 Processor -- start 2024-02-17T12:57:20.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f5543c22740 1 -- start start 2024-02-17T12:57:20.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f5543c22740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5250 0x561fbcefde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f5543c22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5b70 0x561fbcefbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f5543c22740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcefe750 0x561fbcef6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f5543c22740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbcde2130 con 0x561fbcef5250 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f5543c22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbcef0bb0 con 0x561fbcef5b70 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f5543c22740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbcf160a0 con 0x561fbcefe750 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f553a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5250 0x561fbcefde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f553a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5250 0x561fbcefde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44084/0 (socket says 172.21.15.18:44084) 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f553a59c640 1 -- 172.21.15.18:0/873883801 learned_addr learned my addr 172.21.15.18:0/873883801 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f5531d9b640 1 --2- 172.21.15.18:0/873883801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5b70 0x561fbcefbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f553a59c640 1 -- 172.21.15.18:0/873883801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcefe750 msgr2=0x561fbcef6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f553ad9d640 1 --2- 172.21.15.18:0/873883801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcefe750 0x561fbcef6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f553a59c640 1 --2- 172.21.15.18:0/873883801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcefe750 0x561fbcef6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f553a59c640 1 -- 172.21.15.18:0/873883801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5b70 msgr2=0x561fbcefbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f553a59c640 1 --2- 172.21.15.18:0/873883801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5b70 0x561fbcefbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f553a59c640 1 -- 172.21.15.18:0/873883801 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fbcdf7c20 con 0x561fbcef5250 2024-02-17T12:57:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f553ad9d640 1 --2- 172.21.15.18:0/873883801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcefe750 0x561fbcef6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f553a59c640 1 --2- 172.21.15.18:0/873883801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5250 0x561fbcefde70 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f552c00d740 tx=0x7f552c0ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546ae44219b51e44 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f5523fff640 1 -- 172.21.15.18:0/873883801 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c0d7040 con 0x561fbcef5250 2024-02-17T12:57:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f5523fff640 1 -- 172.21.15.18:0/873883801 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f552c096760 con 0x561fbcef5250 2024-02-17T12:57:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.140+0000 7f5523fff640 1 -- 172.21.15.18:0/873883801 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c09a4c0 con 0x561fbcef5250 2024-02-17T12:57:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.141+0000 7f5543c22740 1 -- 172.21.15.18:0/873883801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5250 msgr2=0x561fbcefde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.141+0000 7f5543c22740 1 --2- 172.21.15.18:0/873883801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5250 0x561fbcefde70 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f552c00d740 tx=0x7f552c0ba040 comp rx=0 tx=0).stop 2024-02-17T12:57:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.141+0000 7f5543c22740 1 -- 172.21.15.18:0/873883801 shutdown_connections 2024-02-17T12:57:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.141+0000 7f5543c22740 1 --2- 172.21.15.18:0/873883801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcefe750 0x561fbcef6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.141+0000 7f5543c22740 1 --2- 172.21.15.18:0/873883801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5b70 0x561fbcefbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.141+0000 7f5543c22740 1 --2- 172.21.15.18:0/873883801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5250 0x561fbcefde70 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.141+0000 7f5543c22740 1 -- 172.21.15.18:0/873883801 >> 172.21.15.18:0/873883801 conn(0x561fbceef980 msgr2=0x561fbceeaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.141+0000 7f5543c22740 1 -- 172.21.15.18:0/873883801 shutdown_connections 2024-02-17T12:57:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.141+0000 7f5543c22740 1 -- 172.21.15.18:0/873883801 wait complete. 2024-02-17T12:57:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.141+0000 7f5543c22740 1 -- start start 2024-02-17T12:57:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.141+0000 7f5543c22740 1 Processor -- start 2024-02-17T12:57:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5543c22740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5250 0x561fbcf04420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5543c22740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 0x561fbcf04960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5543c22740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcefe750 0x561fbcf64f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5543c22740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbcf05230 con 0x561fbcef5b70 2024-02-17T12:57:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5543c22740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbcf654c0 con 0x561fbcef5250 2024-02-17T12:57:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5543c22740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fbcf65600 con 0x561fbcefe750 2024-02-17T12:57:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5543c22740 10 client.0 _ll_register_callbacks cb 0x7f5534009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5531d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 0x561fbcf04960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5531d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 0x561fbcf04960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44100/0 (socket says 172.21.15.18:44100) 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f553a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5250 0x561fbcf04420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5531d9b640 1 -- 172.21.15.18:0/3755539457 learned_addr learned my addr 172.21.15.18:0/3755539457 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f553ad9d640 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcefe750 0x561fbcf64f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5531d9b640 1 -- 172.21.15.18:0/3755539457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcefe750 msgr2=0x561fbcf64f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5531d9b640 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcefe750 0x561fbcf64f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5531d9b640 1 -- 172.21.15.18:0/3755539457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5250 msgr2=0x561fbcf04420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5531d9b640 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5250 0x561fbcf04420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5531d9b640 1 -- 172.21.15.18:0/3755539457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552c0cde70 con 0x561fbcef5b70 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f553ad9d640 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcefe750 0x561fbcf64f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f553a59c640 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5250 0x561fbcf04420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5531d9b640 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 0x561fbcf04960 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f553c069620 tx=0x7f552402ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db13b661ea0a4f89 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5521ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5521ffb640 1 -- 172.21.15.18:0/3755539457 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524045070 con 0x561fbcef5b70 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5521ffb640 1 -- 172.21.15.18:0/3755539457 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5524004260 con 0x561fbcef5b70 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5543c22740 1 -- 172.21.15.18:0/3755539457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f553401d910 con 0x561fbcef5b70 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5521ffb640 1 -- 172.21.15.18:0/3755539457 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5524004420 con 0x561fbcef5b70 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5543c22740 10 client.18234 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.142+0000 7f5543c22740 1 -- 172.21.15.18:0/3755539457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x561fbce1d8c0 con 0x561fbcef5b70 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.143+0000 7f5521ffb640 1 -- 172.21.15.18:0/3755539457 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5524056020 con 0x561fbcef5b70 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.143+0000 7f5521ffb640 1 -- 172.21.15.18:0/3755539457 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f552403c040 con 0x561fbcef5b70 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.143+0000 7f5543c22740 10 client.18234 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.143+0000 7f5543c22740 10 client.18234 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.143+0000 7f5543c22740 20 client.18234 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.143+0000 7f5543c22740 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561fbcf66e20 0x561fbcf69240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.143+0000 7f5543c22740 4 client.18234 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:57:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.143+0000 7f5543c22740 1 -- 172.21.15.18:0/3755539457 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561fbcbe8890 con 0x561fbcf66e20 2024-02-17T12:57:20.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.143+0000 7f553a59c640 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561fbcf66e20 0x561fbcf69240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.143+0000 7f553a59c640 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561fbcf66e20 0x561fbcf69240 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4637bea752addfe5 server_cookie=80e2e088da7d10b4 in_seq=0 out_seq=0 2024-02-17T12:57:20.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.143+0000 7f5521ffb640 10 client.18234 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:57:20.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5521ffb640 1 -- 172.21.15.18:0/3755539457 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561fbcbe8890 con 0x561fbcf66e20 2024-02-17T12:57:20.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5521ffb640 10 client.18234 handle_command_reply: tid=0 2024-02-17T12:57:20.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5543c22740 1 client.18234 shutdown 2024-02-17T12:57:20.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5543c22740 1 -- 172.21.15.18:0/3755539457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 msgr2=0x561fbcf04960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5543c22740 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 0x561fbcf04960 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f553c069620 tx=0x7f552402ccb0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5543c22740 1 -- 172.21.15.18:0/3755539457 shutdown_connections 2024-02-17T12:57:20.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5543c22740 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x561fbcf66e20 0x561fbcf69240 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5543c22740 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561fbcefe750 0x561fbcf64f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5543c22740 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561fbcef5b70 0x561fbcf04960 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5521ffb640 0 client.18234 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:57:20.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5543c22740 1 --2- 172.21.15.18:0/3755539457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561fbcef5250 0x561fbcf04420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5543c22740 1 -- 172.21.15.18:0/3755539457 >> 172.21.15.18:0/3755539457 conn(0x561fbceef980 msgr2=0x561fbcef9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5543c22740 1 -- 172.21.15.18:0/3755539457 shutdown_connections 2024-02-17T12:57:20.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5543c22740 1 -- 172.21.15.18:0/3755539457 wait complete. 2024-02-17T12:57:20.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5543c22740 20 client.18234 trim_cache size 0 max 16384 2024-02-17T12:57:20.143 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:57:20.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5540f77640 1 -- 172.21.15.18:0/1954146259 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f55280a2260 msgr2=0x7f55280a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5540f77640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f55280a2260 0x7f55280a4720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5534021000 tx=0x7f553401e000 comp rx=0 tx=0).stop 2024-02-17T12:57:20.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5540f77640 1 -- 172.21.15.18:0/1954146259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f553c100310 msgr2=0x7f553c101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5540f77640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f553c100310 0x7f553c101da0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f552c00d770 tx=0x7f552c00dc30 comp rx=0 tx=0).stop 2024-02-17T12:57:20.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5540f77640 1 -- 172.21.15.18:0/1954146259 shutdown_connections 2024-02-17T12:57:20.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5540f77640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f55280a2260 0x7f55280a4720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5540f77640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f553c102820 0x7f553c1a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5540f77640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f553c1053e0 0x7f553c1022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5540f77640 1 --2- 172.21.15.18:0/1954146259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f553c100310 0x7f553c101da0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5540f77640 1 -- 172.21.15.18:0/1954146259 >> 172.21.15.18:0/1954146259 conn(0x7f553c0771e0 msgr2=0x7f553c069a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.144+0000 7f5540f77640 1 -- 172.21.15.18:0/1954146259 shutdown_connections 2024-02-17T12:57:20.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.145+0000 7f5540f77640 1 -- 172.21.15.18:0/1954146259 wait complete. 2024-02-17T12:57:20.146 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:57:20.146 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:57:20.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:19 smithi146 ceph-mon[36454]: pgmap v230: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-02-17T12:57:20.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:19 smithi146 ceph-mon[36454]: osdmap e94: 12 total, 12 up, 12 in 2024-02-17T12:57:20.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:19 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2186676137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:57:20.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.260+0000 7f6976b4f640 1 -- 172.21.15.18:0/4005617442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6970102d10 msgr2=0x7f697010b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.260+0000 7f6976b4f640 1 --2- 172.21.15.18:0/4005617442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6970102d10 0x7f697010b1f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f6960009900 tx=0x7f696002bcd0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.260+0000 7f6976b4f640 1 -- 172.21.15.18:0/4005617442 shutdown_connections 2024-02-17T12:57:20.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.260+0000 7f6976b4f640 1 --2- 172.21.15.18:0/4005617442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f697010b7c0 0x7f697010dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.260+0000 7f6976b4f640 1 --2- 172.21.15.18:0/4005617442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6970102d10 0x7f697010b1f0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.260+0000 7f6976b4f640 1 --2- 172.21.15.18:0/4005617442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69701023f0 0x7f69701027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.260+0000 7f6976b4f640 1 -- 172.21.15.18:0/4005617442 >> 172.21.15.18:0/4005617442 conn(0x7f69700771e0 msgr2=0x7f69700775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.260+0000 7f6976b4f640 1 -- 172.21.15.18:0/4005617442 shutdown_connections 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.260+0000 7f6976b4f640 1 -- 172.21.15.18:0/4005617442 wait complete. 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f6976b4f640 1 Processor -- start 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f6976b4f640 1 -- start start 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f6976b4f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69701023f0 0x7f6970100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f6976b4f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6970102d10 0x7f6970100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f6976b4f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f697010b7c0 0x7f6970100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f6976b4f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69701a5c00 con 0x7f69701023f0 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f6976b4f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69701a5d70 con 0x7f697010b7c0 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f6976b4f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69701a5f30 con 0x7f6970102d10 2024-02-17T12:57:20.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f69748eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69701023f0 0x7f6970100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f69748eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69701023f0 0x7f6970100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44120/0 (socket says 172.21.15.18:44120) 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f69748eb640 1 -- 172.21.15.18:0/1113201815 learned_addr learned my addr 172.21.15.18:0/1113201815 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f69750ec640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f697010b7c0 0x7f6970100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.261+0000 7f6967fff640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6970102d10 0x7f6970100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.262+0000 7f69748eb640 1 -- 172.21.15.18:0/1113201815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6970102d10 msgr2=0x7f6970100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.262+0000 7f69748eb640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6970102d10 0x7f6970100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.262+0000 7f69748eb640 1 -- 172.21.15.18:0/1113201815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f697010b7c0 msgr2=0x7f6970100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.262+0000 7f69748eb640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f697010b7c0 0x7f6970100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.262+0000 7f69748eb640 1 -- 172.21.15.18:0/1113201815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960009560 con 0x7f69701023f0 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.262+0000 7f69748eb640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69701023f0 0x7f6970100040 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f696c00e9f0 tx=0x7f696c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.262+0000 7f6965ffb640 1 -- 172.21.15.18:0/1113201815 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c009720 con 0x7f69701023f0 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.262+0000 7f6965ffb640 1 -- 172.21.15.18:0/1113201815 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f696c00cd80 con 0x7f69701023f0 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.262+0000 7f6976b4f640 1 -- 172.21.15.18:0/1113201815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69701a6210 con 0x7f69701023f0 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.262+0000 7f6965ffb640 1 -- 172.21.15.18:0/1113201815 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c010690 con 0x7f69701023f0 2024-02-17T12:57:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.262+0000 7f6976b4f640 1 -- 172.21.15.18:0/1113201815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69701a66d0 con 0x7f69701023f0 2024-02-17T12:57:20.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.264+0000 7f6965ffb640 1 -- 172.21.15.18:0/1113201815 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f696c0046c0 con 0x7f69701023f0 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f6979802740 1 Processor -- start 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f6979802740 1 -- start start 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f6965ffb640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f69580a2230 0x7f69580a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f6979802740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f127630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f6979802740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 0x55786f130210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f6979802740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f130750 0x55786f132b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f6979802740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f0e2960 con 0x55786f127250 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f6979802740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f096310 con 0x55786f130750 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f6979802740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f093840 con 0x55786f127b70 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f69750ec640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f69580a2230 0x7f69580a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f69748eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f127630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f6965ffb640 1 -- 172.21.15.18:0/1113201815 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f696c014070 con 0x7f69701023f0 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f69748eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f127630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44132/0 (socket says 172.21.15.18:44132) 2024-02-17T12:57:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.265+0000 7f69748eb640 1 -- 172.21.15.18:0/1971874360 learned_addr learned my addr 172.21.15.18:0/1971874360 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.266+0000 7f6967fff640 1 --2- 172.21.15.18:0/1971874360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 0x55786f130210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.266+0000 7f69748eb640 1 -- 172.21.15.18:0/1971874360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 msgr2=0x55786f130210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.266+0000 7f69748eb640 1 --2- 172.21.15.18:0/1971874360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 0x55786f130210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.266+0000 7f69748eb640 1 -- 172.21.15.18:0/1971874360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f130750 msgr2=0x55786f132b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.266+0000 7f69748eb640 1 --2- 172.21.15.18:0/1971874360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f130750 0x55786f132b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.266+0000 7f69748eb640 1 -- 172.21.15.18:0/1971874360 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55786f051060 con 0x55786f127250 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.266+0000 7f69750ec640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f69580a2230 0x7f69580a46f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6968005fd0 tx=0x7f6968011810 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.266+0000 7f69748eb640 1 --2- 172.21.15.18:0/1971874360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f127630 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f696c0a3140 tx=0x7f696c09ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e43f808a7cdb9e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.266+0000 7f6956ffd640 1 -- 172.21.15.18:0/1971874360 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c086d50 con 0x55786f127250 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.266+0000 7f6956ffd640 1 -- 172.21.15.18:0/1971874360 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f696c081120 con 0x55786f127250 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.266+0000 7f6956ffd640 1 -- 172.21.15.18:0/1971874360 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c097540 con 0x55786f127250 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 -- 172.21.15.18:0/1971874360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 msgr2=0x55786f127630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 --2- 172.21.15.18:0/1971874360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f127630 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f696c0a3140 tx=0x7f696c09ad60 comp rx=0 tx=0).stop 2024-02-17T12:57:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 -- 172.21.15.18:0/1971874360 shutdown_connections 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 --2- 172.21.15.18:0/1971874360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f130750 0x55786f132b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 --2- 172.21.15.18:0/1971874360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 0x55786f130210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 --2- 172.21.15.18:0/1971874360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f127630 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 -- 172.21.15.18:0/1971874360 >> 172.21.15.18:0/1971874360 conn(0x55786f121980 msgr2=0x55786f121d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 -- 172.21.15.18:0/1971874360 shutdown_connections 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 -- 172.21.15.18:0/1971874360 wait complete. 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 -- start start 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 Processor -- start 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f12d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f127b70 0x55786f12dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f130750 0x55786f12e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f08f020 con 0x55786f127250 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786ed62430 con 0x55786f127b70 2024-02-17T12:57:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.267+0000 7f6979802740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786ed1ee00 con 0x55786f130750 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f69748eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f12d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f6979802740 10 client.0 _ll_register_callbacks cb 0x55786f036cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f69748eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f12d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44146/0 (socket says 172.21.15.18:44146) 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f69748eb640 1 -- 172.21.15.18:0/3349799017 learned_addr learned my addr 172.21.15.18:0/3349799017 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f69750ec640 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f130750 0x55786f12e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f6967fff640 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f127b70 0x55786f12dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f69748eb640 1 -- 172.21.15.18:0/3349799017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f130750 msgr2=0x55786f12e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f69748eb640 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f130750 0x55786f12e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f69748eb640 1 -- 172.21.15.18:0/3349799017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f127b70 msgr2=0x55786f12dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f69748eb640 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f127b70 0x55786f12dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f69748eb640 1 -- 172.21.15.18:0/3349799017 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69701a66d0 con 0x55786f127250 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f69750ec640 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f130750 0x55786f12e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f6967fff640 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f127b70 0x55786f12dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f69748eb640 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f12d7d0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f696c09abd0 tx=0x7f696c0cc160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981c07e950292d91 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f694ffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f694ffff640 1 -- 172.21.15.18:0/3349799017 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c09f020 con 0x55786f127250 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f694ffff640 1 -- 172.21.15.18:0/3349799017 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f696c09b5d0 con 0x55786f127250 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f6979802740 1 -- 172.21.15.18:0/3349799017 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55786ed8e7e0 con 0x55786f127250 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f694ffff640 1 -- 172.21.15.18:0/3349799017 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f696c0b92f0 con 0x55786f127250 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f6979802740 10 client.18246 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f6979802740 1 -- 172.21.15.18:0/3349799017 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55786f08d6a0 con 0x55786f127250 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.268+0000 7f694ffff640 1 -- 172.21.15.18:0/3349799017 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f696c064020 con 0x55786f127250 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.269+0000 7f694ffff640 1 -- 172.21.15.18:0/3349799017 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f696c01f060 con 0x55786f127250 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.269+0000 7f6979802740 10 client.18246 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.269+0000 7f6979802740 10 client.18246 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.269+0000 7f6979802740 20 client.18246 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.269+0000 7f6979802740 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55786f1408b0 0x55786f142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.269+0000 7f6979802740 4 client.18246 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.269+0000 7f6979802740 1 -- 172.21.15.18:0/3349799017 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55786edcb720 con 0x55786f1408b0 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.269+0000 7f6967fff640 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55786f1408b0 0x55786f142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.269+0000 7f6967fff640 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55786f1408b0 0x55786f142ca0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5781a3c9224ad51b server_cookie=e32b411d4cd66489 in_seq=0 out_seq=0 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.269+0000 7f694ffff640 10 client.18246 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f694ffff640 1 -- 172.21.15.18:0/3349799017 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55786edcb720 con 0x55786f1408b0 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f694ffff640 10 client.18246 handle_command_reply: tid=0 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f6979802740 1 client.18246 shutdown 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f6979802740 1 -- 172.21.15.18:0/3349799017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 msgr2=0x55786f12d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f6979802740 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f12d7d0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f696c09abd0 tx=0x7f696c0cc160 comp rx=0 tx=0).stop 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f6979802740 1 -- 172.21.15.18:0/3349799017 shutdown_connections 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f6979802740 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55786f1408b0 0x55786f142ca0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f6979802740 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f130750 0x55786f12e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f694ffff640 0 client.18246 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f6979802740 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f127b70 0x55786f12dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f6979802740 1 --2- 172.21.15.18:0/3349799017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f12d7d0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f6979802740 1 -- 172.21.15.18:0/3349799017 >> 172.21.15.18:0/3349799017 conn(0x55786f121980 msgr2=0x55786f1236d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f6979802740 1 -- 172.21.15.18:0/3349799017 shutdown_connections 2024-02-17T12:57:20.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f6979802740 1 -- 172.21.15.18:0/3349799017 wait complete. 2024-02-17T12:57:20.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.270+0000 7f6979802740 20 client.18246 trim_cache size 0 max 16384 2024-02-17T12:57:20.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f6979802740 1 Processor -- start 2024-02-17T12:57:20.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f6979802740 1 -- start start 2024-02-17T12:57:20.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f6979802740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f12ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f6979802740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 0x55786f12e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f6979802740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f130750 0x55786f12aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f6979802740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f014130 con 0x55786f127250 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f6979802740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f126eb0 con 0x55786f130750 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f6979802740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f132eb0 con 0x55786f127b70 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f69748eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f12ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f69748eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f12ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44160/0 (socket says 172.21.15.18:44160) 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f69748eb640 1 -- 172.21.15.18:0/466502265 learned_addr learned my addr 172.21.15.18:0/466502265 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f69750ec640 1 --2- 172.21.15.18:0/466502265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f130750 0x55786f12aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.282+0000 7f6967fff640 1 --2- 172.21.15.18:0/466502265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 0x55786f12e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.283+0000 7f69748eb640 1 -- 172.21.15.18:0/466502265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 msgr2=0x55786f12e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.283+0000 7f69748eb640 1 --2- 172.21.15.18:0/466502265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 0x55786f12e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.283+0000 7f69748eb640 1 -- 172.21.15.18:0/466502265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f130750 msgr2=0x55786f12aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.283+0000 7f69748eb640 1 --2- 172.21.15.18:0/466502265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f130750 0x55786f12aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.283+0000 7f69748eb640 1 -- 172.21.15.18:0/466502265 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55786f029c20 con 0x55786f127250 2024-02-17T12:57:20.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.283+0000 7f69750ec640 1 --2- 172.21.15.18:0/466502265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f130750 0x55786f12aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.283+0000 7f69748eb640 1 --2- 172.21.15.18:0/466502265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f12ff00 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f696c0cbce0 tx=0x7f696c084320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71824a2e7ba0f99e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.283+0000 7f69567fc640 1 -- 172.21.15.18:0/466502265 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c097170 con 0x55786f127250 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.283+0000 7f69567fc640 1 -- 172.21.15.18:0/466502265 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f696c064030 con 0x55786f127250 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.283+0000 7f69567fc640 1 -- 172.21.15.18:0/466502265 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c0b92f0 con 0x55786f127250 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 -- 172.21.15.18:0/466502265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 msgr2=0x55786f12ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 --2- 172.21.15.18:0/466502265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f12ff00 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f696c0cbce0 tx=0x7f696c084320 comp rx=0 tx=0).stop 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 -- 172.21.15.18:0/466502265 shutdown_connections 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 --2- 172.21.15.18:0/466502265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f130750 0x55786f12aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 --2- 172.21.15.18:0/466502265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 0x55786f12e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 --2- 172.21.15.18:0/466502265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f127250 0x55786f12ff00 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 -- 172.21.15.18:0/466502265 >> 172.21.15.18:0/466502265 conn(0x55786f121980 msgr2=0x55786f12c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 -- 172.21.15.18:0/466502265 shutdown_connections 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 -- 172.21.15.18:0/466502265 wait complete. 2024-02-17T12:57:20.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 -- start start 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 Processor -- start 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f127250 0x55786f1378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 0x55786f135cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f130750 0x55786f136200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f1367c0 con 0x55786f130750 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f136930 con 0x55786f127250 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f136af0 con 0x55786f127b70 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f6979802740 10 client.0 _ll_register_callbacks cb 0x7f696c004a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f69750ec640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f130750 0x55786f136200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f69750ec640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f130750 0x55786f136200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44172/0 (socket says 172.21.15.18:44172) 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.284+0000 7f69750ec640 1 -- 172.21.15.18:0/1538469826 learned_addr learned my addr 172.21.15.18:0/1538469826 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f69748eb640 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f127250 0x55786f1378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f6967fff640 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 0x55786f135cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f69750ec640 1 -- 172.21.15.18:0/1538469826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 msgr2=0x55786f135cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f69750ec640 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 0x55786f135cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f69750ec640 1 -- 172.21.15.18:0/1538469826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f127250 msgr2=0x55786f1378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f69750ec640 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f127250 0x55786f1378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f69750ec640 1 -- 172.21.15.18:0/1538469826 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55786f08d6a0 con 0x55786f130750 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f69750ec640 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f130750 0x55786f136200 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f6968011d10 tx=0x7f6968004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279f7d72114ed57 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f694ffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f694ffff640 1 -- 172.21.15.18:0/1538469826 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696800fc10 con 0x55786f130750 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f694ffff640 1 -- 172.21.15.18:0/1538469826 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f696800fd70 con 0x55786f130750 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f6979802740 1 -- 172.21.15.18:0/1538469826 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f696c081120 con 0x55786f130750 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f694ffff640 1 -- 172.21.15.18:0/1538469826 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f696802a470 con 0x55786f130750 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f6979802740 10 client.18252 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f6979802740 1 -- 172.21.15.18:0/1538469826 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55786f04f8c0 con 0x55786f130750 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.285+0000 7f694ffff640 1 -- 172.21.15.18:0/1538469826 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f696c081120 con 0x55786f130750 2024-02-17T12:57:20.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f694ffff640 1 -- 172.21.15.18:0/1538469826 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f696801d040 con 0x55786f130750 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f6979802740 10 client.18252 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f6979802740 10 client.18252 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f6979802740 20 client.18252 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f6979802740 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55786f198480 0x55786f19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f6979802740 4 client.18252 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f6979802740 1 -- 172.21.15.18:0/1538469826 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55786ee1a890 con 0x55786f198480 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f69748eb640 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55786f198480 0x55786f19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f694ffff640 10 client.18252 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f69748eb640 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55786f198480 0x55786f19a8a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4c4fb65cd994839 server_cookie=8abd4faf7a823116 in_seq=0 out_seq=0 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f694ffff640 1 -- 172.21.15.18:0/1538469826 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55786ee1a890 con 0x55786f198480 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f694ffff640 10 client.18252 handle_command_reply: tid=0 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.286+0000 7f6979802740 1 client.18252 shutdown 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6979802740 1 -- 172.21.15.18:0/1538469826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f130750 msgr2=0x55786f136200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6979802740 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f130750 0x55786f136200 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f6968011d10 tx=0x7f6968004ab0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6979802740 1 -- 172.21.15.18:0/1538469826 shutdown_connections 2024-02-17T12:57:20.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6979802740 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55786f198480 0x55786f19a8a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6979802740 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55786f130750 0x55786f136200 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6979802740 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55786f127b70 0x55786f135cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f694ffff640 0 client.18252 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:57:20.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6979802740 1 --2- 172.21.15.18:0/1538469826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55786f127250 0x55786f1378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6979802740 1 -- 172.21.15.18:0/1538469826 >> 172.21.15.18:0/1538469826 conn(0x55786f121980 msgr2=0x55786f120b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6979802740 1 -- 172.21.15.18:0/1538469826 shutdown_connections 2024-02-17T12:57:20.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6979802740 1 -- 172.21.15.18:0/1538469826 wait complete. 2024-02-17T12:57:20.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6979802740 20 client.18252 trim_cache size 0 max 16384 2024-02-17T12:57:20.284 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:57:20.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6976b4f640 1 -- 172.21.15.18:0/1113201815 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f69580a2230 msgr2=0x7f69580a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6976b4f640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f69580a2230 0x7f69580a46f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6968005fd0 tx=0x7f6968011810 comp rx=0 tx=0).stop 2024-02-17T12:57:20.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6976b4f640 1 -- 172.21.15.18:0/1113201815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69701023f0 msgr2=0x7f6970100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6976b4f640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69701023f0 0x7f6970100040 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f696c00e9f0 tx=0x7f696c00edf0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6976b4f640 1 -- 172.21.15.18:0/1113201815 shutdown_connections 2024-02-17T12:57:20.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6976b4f640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f69580a2230 0x7f69580a46f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6976b4f640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f697010b7c0 0x7f6970100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6976b4f640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6970102d10 0x7f6970100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6976b4f640 1 --2- 172.21.15.18:0/1113201815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69701023f0 0x7f6970100040 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6976b4f640 1 -- 172.21.15.18:0/1113201815 >> 172.21.15.18:0/1113201815 conn(0x7f69700771e0 msgr2=0x7f69700fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6976b4f640 1 -- 172.21.15.18:0/1113201815 shutdown_connections 2024-02-17T12:57:20.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.287+0000 7f6976b4f640 1 -- 172.21.15.18:0/1113201815 wait complete. 2024-02-17T12:57:20.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 388368 5.54 MB/sec execute 275 sec latency 263.235 ms 2024-02-17T12:57:20.288 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:57:20.288 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:57:20.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.402+0000 7f20ee42e640 1 -- 172.21.15.18:0/3642813838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e8106450 msgr2=0x7f20e810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.402+0000 7f20ee42e640 1 --2- 172.21.15.18:0/3642813838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e8106450 0x7f20e810e020 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009cd0 tx=0x7f20e002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.402+0000 7f20ee42e640 1 -- 172.21.15.18:0/3642813838 shutdown_connections 2024-02-17T12:57:20.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.402+0000 7f20ee42e640 1 --2- 172.21.15.18:0/3642813838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e8106450 0x7f20e810e020 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.402+0000 7f20ee42e640 1 --2- 172.21.15.18:0/3642813838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20e8101d30 0x7f20e8105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.402+0000 7f20ee42e640 1 --2- 172.21.15.18:0/3642813838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20e8101380 0x7f20e8101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.402+0000 7f20ee42e640 1 -- 172.21.15.18:0/3642813838 >> 172.21.15.18:0/3642813838 conn(0x7f20e8078580 msgr2=0x7f20e8078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.402+0000 7f20ee42e640 1 -- 172.21.15.18:0/3642813838 shutdown_connections 2024-02-17T12:57:20.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.402+0000 7f20ee42e640 1 -- 172.21.15.18:0/3642813838 wait complete. 2024-02-17T12:57:20.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.403+0000 7f20ee42e640 1 Processor -- start 2024-02-17T12:57:20.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.403+0000 7f20ee42e640 1 -- start start 2024-02-17T12:57:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.403+0000 7f20ee42e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e8101380 0x7f20e819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.403+0000 7f20ee42e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20e8106450 0x7f20e819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.403+0000 7f20ee42e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20e81975c0 0x7f20e8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.403+0000 7f20ee42e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e8197fb0 con 0x7f20e8101380 2024-02-17T12:57:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.403+0000 7f20ee42e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e8198120 con 0x7f20e8106450 2024-02-17T12:57:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.403+0000 7f20ee42e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81982e0 con 0x7f20e81975c0 2024-02-17T12:57:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.403+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e8101380 0x7f20e819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.403+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e8101380 0x7f20e819d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44192/0 (socket says 172.21.15.18:44192) 2024-02-17T12:57:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.403+0000 7f20e7fff640 1 -- 172.21.15.18:0/3580416900 learned_addr learned my addr 172.21.15.18:0/3580416900 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.403+0000 7f20ec9cb640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20e81975c0 0x7f20e8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.404+0000 7f20e77fe640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20e8106450 0x7f20e819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.404+0000 7f20e7fff640 1 -- 172.21.15.18:0/3580416900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20e81975c0 msgr2=0x7f20e8197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.404+0000 7f20e7fff640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20e81975c0 0x7f20e8197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.404+0000 7f20e7fff640 1 -- 172.21.15.18:0/3580416900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20e8106450 msgr2=0x7f20e819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.404+0000 7f20e7fff640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20e8106450 0x7f20e819d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.404+0000 7f20e7fff640 1 -- 172.21.15.18:0/3580416900 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0009900 con 0x7f20e8101380 2024-02-17T12:57:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.404+0000 7f20e7fff640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e8101380 0x7f20e819d410 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f20d000d950 tx=0x7f20d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.404+0000 7f20e57fa640 1 -- 172.21.15.18:0/3580416900 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0009a30 con 0x7f20e8101380 2024-02-17T12:57:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.404+0000 7f20ee42e640 1 -- 172.21.15.18:0/3580416900 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e81a42d0 con 0x7f20e8101380 2024-02-17T12:57:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.404+0000 7f20e57fa640 1 -- 172.21.15.18:0/3580416900 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20d0009b90 con 0x7f20e8101380 2024-02-17T12:57:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.404+0000 7f20e57fa640 1 -- 172.21.15.18:0/3580416900 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0004a00 con 0x7f20e8101380 2024-02-17T12:57:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.404+0000 7f20ee42e640 1 -- 172.21.15.18:0/3580416900 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e81a47e0 con 0x7f20e8101380 2024-02-17T12:57:20.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.406+0000 7f20e57fa640 1 -- 172.21.15.18:0/3580416900 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f20d0009d00 con 0x7f20e8101380 2024-02-17T12:57:20.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20f1000740 1 Processor -- start 2024-02-17T12:57:20.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20f1000740 1 -- start start 2024-02-17T12:57:20.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20e57fa640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f20d40a2230 0x7f20d40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20f1000740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 0x55d22cf9f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20f1000740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9fb70 0x55d22cfa8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20f1000740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 0x55d22cfaab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20f1000740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d22cf5a960 con 0x55d22cf9f250 2024-02-17T12:57:20.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20f1000740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d22cf0e310 con 0x55d22cfa8750 2024-02-17T12:57:20.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20e57fa640 1 -- 172.21.15.18:0/3580416900 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f20d00cdad0 con 0x7f20e8101380 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20f1000740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d22cf0b840 con 0x55d22cf9fb70 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20ec9cb640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f20d40a2230 0x7f20d40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 0x55d22cf9f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 0x55d22cf9f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44194/0 (socket says 172.21.15.18:44194) 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20e7fff640 1 -- 172.21.15.18:0/1406637384 learned_addr learned my addr 172.21.15.18:0/1406637384 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20e77fe640 1 --2- 172.21.15.18:0/1406637384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9fb70 0x55d22cfa8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20e7fff640 1 -- 172.21.15.18:0/1406637384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9fb70 msgr2=0x55d22cfa8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20e7fff640 1 --2- 172.21.15.18:0/1406637384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9fb70 0x55d22cfa8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20e7fff640 1 -- 172.21.15.18:0/1406637384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 msgr2=0x55d22cfaab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20e7fff640 1 --2- 172.21.15.18:0/1406637384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 0x55d22cfaab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20e7fff640 1 -- 172.21.15.18:0/1406637384 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d22cec9060 con 0x55d22cf9f250 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20ec9cb640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f20d40a2230 0x7f20d40a46f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f20e0005cc0 tx=0x7f20e0032ef0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.407+0000 7f20e7fff640 1 --2- 172.21.15.18:0/1406637384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 0x55d22cf9f630 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f20d0096940 tx=0x7f20d0099f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2600f4e1f997199c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.408+0000 7f20de7fc640 1 -- 172.21.15.18:0/1406637384 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d009e630 con 0x55d22cf9f250 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.408+0000 7f20de7fc640 1 -- 172.21.15.18:0/1406637384 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20d00a2c80 con 0x55d22cf9f250 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.408+0000 7f20de7fc640 1 -- 172.21.15.18:0/1406637384 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d00cb650 con 0x55d22cf9f250 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.408+0000 7f20f1000740 1 -- 172.21.15.18:0/1406637384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 msgr2=0x55d22cf9f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.408+0000 7f20f1000740 1 --2- 172.21.15.18:0/1406637384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 0x55d22cf9f630 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f20d0096940 tx=0x7f20d0099f00 comp rx=0 tx=0).stop 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.408+0000 7f20f1000740 1 -- 172.21.15.18:0/1406637384 shutdown_connections 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.408+0000 7f20f1000740 1 --2- 172.21.15.18:0/1406637384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 0x55d22cfaab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.408+0000 7f20f1000740 1 --2- 172.21.15.18:0/1406637384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9fb70 0x55d22cfa8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.408+0000 7f20f1000740 1 --2- 172.21.15.18:0/1406637384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 0x55d22cf9f630 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.408+0000 7f20f1000740 1 -- 172.21.15.18:0/1406637384 >> 172.21.15.18:0/1406637384 conn(0x55d22cf99980 msgr2=0x55d22cf99d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.408+0000 7f20f1000740 1 -- 172.21.15.18:0/1406637384 shutdown_connections 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.408+0000 7f20f1000740 1 -- 172.21.15.18:0/1406637384 wait complete. 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20f1000740 1 -- start start 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20f1000740 1 Processor -- start 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20f1000740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9f250 0x55d22cfa5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20f1000740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9fb70 0x55d22cfa5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20f1000740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 0x55d22cfa62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20f1000740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d22cf07020 con 0x55d22cf9fb70 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20f1000740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d22cbda430 con 0x55d22cfa8750 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20f1000740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d22cb96e00 con 0x55d22cf9f250 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20f1000740 10 client.0 _ll_register_callbacks cb 0x55d22ceaecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20e77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9fb70 0x55d22cfa5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20e77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9fb70 0x55d22cfa5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44200/0 (socket says 172.21.15.18:44200) 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20e77fe640 1 -- 172.21.15.18:0/3836883854 learned_addr learned my addr 172.21.15.18:0/3836883854 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20e7fff640 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9f250 0x55d22cfa5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20e77fe640 1 -- 172.21.15.18:0/3836883854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9f250 msgr2=0x55d22cfa5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20e77fe640 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9f250 0x55d22cfa5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20ec9cb640 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 0x55d22cfa62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20e77fe640 1 -- 172.21.15.18:0/3836883854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 msgr2=0x55d22cfa62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20e77fe640 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 0x55d22cfa62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20e77fe640 1 -- 172.21.15.18:0/3836883854 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e81a47e0 con 0x55d22cf9fb70 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.409+0000 7f20e7fff640 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9f250 0x55d22cfa5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20e77fe640 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9fb70 0x55d22cfa5d80 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f20d80134e0 tx=0x7f20d80139a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638fa2df47dedfd4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20c77fe640 1 -- 172.21.15.18:0/3836883854 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8004460 con 0x55d22cf9fb70 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20c77fe640 1 -- 172.21.15.18:0/3836883854 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f20d80035f0 con 0x55d22cf9fb70 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20f1000740 1 -- 172.21.15.18:0/3836883854 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d22cc067e0 con 0x55d22cf9fb70 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20c77fe640 1 -- 172.21.15.18:0/3836883854 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20d80037b0 con 0x55d22cf9fb70 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20f1000740 10 client.18264 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20f1000740 1 -- 172.21.15.18:0/3836883854 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55d22cf056a0 con 0x55d22cf9fb70 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20c77fe640 1 -- 172.21.15.18:0/3836883854 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20d8028020 con 0x55d22cf9fb70 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20c77fe640 1 -- 172.21.15.18:0/3836883854 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f20d800f450 con 0x55d22cf9fb70 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20f1000740 10 client.18264 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:20.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20f1000740 10 client.18264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20f1000740 20 client.18264 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20f1000740 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d22cfb8c70 0x55d22cfbb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20f1000740 4 client.18264 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.410+0000 7f20f1000740 1 -- 172.21.15.18:0/3836883854 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d22cc43720 con 0x55d22cfb8c70 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.411+0000 7f20e7fff640 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d22cfb8c70 0x55d22cfbb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.411+0000 7f20c77fe640 10 client.18264 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.411+0000 7f20e7fff640 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d22cfb8c70 0x55d22cfbb0b0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e65458be3dce9f51 server_cookie=36ecc9f68bcc4064 in_seq=0 out_seq=0 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20c77fe640 1 -- 172.21.15.18:0/3836883854 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d22cc43720 con 0x55d22cfb8c70 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20c77fe640 10 client.18264 handle_command_reply: tid=0 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20f1000740 1 client.18264 shutdown 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20f1000740 1 -- 172.21.15.18:0/3836883854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9fb70 msgr2=0x55d22cfa5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20f1000740 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9fb70 0x55d22cfa5d80 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f20d80134e0 tx=0x7f20d80139a0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20f1000740 1 -- 172.21.15.18:0/3836883854 shutdown_connections 2024-02-17T12:57:20.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20f1000740 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d22cfb8c70 0x55d22cfbb0b0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20f1000740 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 0x55d22cfa62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20c77fe640 0 client.18264 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:20.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20f1000740 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9fb70 0x55d22cfa5d80 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20f1000740 1 --2- 172.21.15.18:0/3836883854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9f250 0x55d22cfa5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20f1000740 1 -- 172.21.15.18:0/3836883854 >> 172.21.15.18:0/3836883854 conn(0x55d22cf99980 msgr2=0x55d22cf9b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20f1000740 1 -- 172.21.15.18:0/3836883854 shutdown_connections 2024-02-17T12:57:20.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20f1000740 1 -- 172.21.15.18:0/3836883854 wait complete. 2024-02-17T12:57:20.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.412+0000 7f20f1000740 20 client.18264 trim_cache size 0 max 16384 2024-02-17T12:57:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20f1000740 1 Processor -- start 2024-02-17T12:57:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20f1000740 1 -- start start 2024-02-17T12:57:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20f1000740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9f250 0x55d22cfa7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20f1000740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cf9fb70 0x55d22cfa60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20f1000740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cfa8750 0x55d22cfa2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20f1000740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d22ce8c130 con 0x55d22cfa8750 2024-02-17T12:57:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20f1000740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d22cfc0300 con 0x55d22cf9fb70 2024-02-17T12:57:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20f1000740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d22cf95080 con 0x55d22cf9f250 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20ec9cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cfa8750 0x55d22cfa2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20ec9cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cfa8750 0x55d22cfa2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44206/0 (socket says 172.21.15.18:44206) 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20ec9cb640 1 -- 172.21.15.18:0/3515228171 learned_addr learned my addr 172.21.15.18:0/3515228171 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20e7fff640 1 --2- 172.21.15.18:0/3515228171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9f250 0x55d22cfa7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20e77fe640 1 --2- 172.21.15.18:0/3515228171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cf9fb70 0x55d22cfa60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20ec9cb640 1 -- 172.21.15.18:0/3515228171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9f250 msgr2=0x55d22cfa7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20ec9cb640 1 --2- 172.21.15.18:0/3515228171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9f250 0x55d22cfa7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20ec9cb640 1 -- 172.21.15.18:0/3515228171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cf9fb70 msgr2=0x55d22cfa60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20ec9cb640 1 --2- 172.21.15.18:0/3515228171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cf9fb70 0x55d22cfa60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.428+0000 7f20ec9cb640 1 -- 172.21.15.18:0/3515228171 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d22cea1c20 con 0x55d22cfa8750 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.429+0000 7f20ec9cb640 1 --2- 172.21.15.18:0/3515228171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cfa8750 0x55d22cfa2aa0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f20e002c700 tx=0x7f20e0041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4db19fa65a3789d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.429+0000 7f20ddffb640 1 -- 172.21.15.18:0/3515228171 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0034580 con 0x55d22cfa8750 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.429+0000 7f20ddffb640 1 -- 172.21.15.18:0/3515228171 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20e00346e0 con 0x55d22cfa8750 2024-02-17T12:57:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.429+0000 7f20ddffb640 1 -- 172.21.15.18:0/3515228171 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0003e10 con 0x55d22cfa8750 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 -- 172.21.15.18:0/3515228171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cfa8750 msgr2=0x55d22cfa2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 --2- 172.21.15.18:0/3515228171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cfa8750 0x55d22cfa2aa0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f20e002c700 tx=0x7f20e0041000 comp rx=0 tx=0).stop 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 -- 172.21.15.18:0/3515228171 shutdown_connections 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 --2- 172.21.15.18:0/3515228171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cfa8750 0x55d22cfa2aa0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 --2- 172.21.15.18:0/3515228171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cf9fb70 0x55d22cfa60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 --2- 172.21.15.18:0/3515228171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9f250 0x55d22cfa7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 -- 172.21.15.18:0/3515228171 >> 172.21.15.18:0/3515228171 conn(0x55d22cf99980 msgr2=0x55d22cfa0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 -- 172.21.15.18:0/3515228171 shutdown_connections 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 -- 172.21.15.18:0/3515228171 wait complete. 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 -- start start 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 Processor -- start 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 0x55d22cfae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9fb70 0x55d22cfaec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 0x55d22d00f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d22cfaf350 con 0x55d22cf9f250 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d22d00f720 con 0x55d22cfa8750 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d22d00f860 con 0x55d22cf9fb70 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20f1000740 10 client.0 _ll_register_callbacks cb 0x7f20d00960c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 0x55d22cfae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.430+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 0x55d22cfae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44212/0 (socket says 172.21.15.18:44212) 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20e7fff640 1 -- 172.21.15.18:0/2585313533 learned_addr learned my addr 172.21.15.18:0/2585313533 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20e77fe640 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9fb70 0x55d22cfaec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20ec9cb640 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 0x55d22d00f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20e7fff640 1 -- 172.21.15.18:0/2585313533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9fb70 msgr2=0x55d22cfaec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20e7fff640 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9fb70 0x55d22cfaec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20e7fff640 1 -- 172.21.15.18:0/2585313533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 msgr2=0x55d22d00f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20e7fff640 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 0x55d22d00f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20e7fff640 1 -- 172.21.15.18:0/2585313533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e810e830 con 0x55d22cf9f250 2024-02-17T12:57:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20e7fff640 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 0x55d22cfae730 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f20d009a470 tx=0x7f20d009ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c91e10b687c91a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20c77fe640 1 -- 172.21.15.18:0/2585313533 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d00cb4c0 con 0x55d22cf9f250 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20c77fe640 1 -- 172.21.15.18:0/2585313533 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f20d009e630 con 0x55d22cf9f250 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20f1000740 1 -- 172.21.15.18:0/2585313533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20d00a2c80 con 0x55d22cf9f250 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20c77fe640 1 -- 172.21.15.18:0/2585313533 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20d0005680 con 0x55d22cf9f250 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20f1000740 10 client.18270 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20f1000740 1 -- 172.21.15.18:0/2585313533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55d22cec78c0 con 0x55d22cf9f250 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.431+0000 7f20c77fe640 1 -- 172.21.15.18:0/2585313533 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20d0064080 con 0x55d22cf9f250 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20c77fe640 1 -- 172.21.15.18:0/2585313533 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f20d0099930 con 0x55d22cf9f250 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20f1000740 10 client.18270 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20f1000740 10 client.18270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20f1000740 20 client.18270 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20f1000740 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d22d010da0 0x55d22d0131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20f1000740 4 client.18270 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20f1000740 1 -- 172.21.15.18:0/2585313533 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d22cc92890 con 0x55d22d010da0 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20e77fe640 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d22d010da0 0x55d22d0131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20c77fe640 10 client.18270 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20e77fe640 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d22d010da0 0x55d22d0131c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709dab18bc22ba57 server_cookie=ffd664f1a9a4dc84 in_seq=0 out_seq=0 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20c77fe640 1 -- 172.21.15.18:0/2585313533 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d22cc92890 con 0x55d22d010da0 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20c77fe640 10 client.18270 handle_command_reply: tid=0 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.432+0000 7f20f1000740 1 client.18270 shutdown 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20f1000740 1 -- 172.21.15.18:0/2585313533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 msgr2=0x55d22cfae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20f1000740 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 0x55d22cfae730 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f20d009a470 tx=0x7f20d009ad20 comp rx=0 tx=0).stop 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20f1000740 1 -- 172.21.15.18:0/2585313533 shutdown_connections 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20f1000740 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d22d010da0 0x55d22d0131c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20f1000740 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22cfa8750 0x55d22d00f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20f1000740 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d22cf9fb70 0x55d22cfaec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20c77fe640 0 client.18270 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:20.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20f1000740 1 --2- 172.21.15.18:0/2585313533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d22cf9f250 0x55d22cfae730 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20f1000740 1 -- 172.21.15.18:0/2585313533 >> 172.21.15.18:0/2585313533 conn(0x55d22cf99980 msgr2=0x55d22cf982f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20f1000740 1 -- 172.21.15.18:0/2585313533 shutdown_connections 2024-02-17T12:57:20.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20f1000740 1 -- 172.21.15.18:0/2585313533 wait complete. 2024-02-17T12:57:20.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20f1000740 20 client.18270 trim_cache size 0 max 16384 2024-02-17T12:57:20.430 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:57:20.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20ee42e640 1 -- 172.21.15.18:0/3580416900 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f20d40a2230 msgr2=0x7f20d40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20ee42e640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f20d40a2230 0x7f20d40a46f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f20e0005cc0 tx=0x7f20e0032ef0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20ee42e640 1 -- 172.21.15.18:0/3580416900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e8101380 msgr2=0x7f20e819d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:20.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20ee42e640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e8101380 0x7f20e819d410 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f20d000d950 tx=0x7f20d000de10 comp rx=0 tx=0).stop 2024-02-17T12:57:20.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20ee42e640 1 -- 172.21.15.18:0/3580416900 shutdown_connections 2024-02-17T12:57:20.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20ee42e640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f20d40a2230 0x7f20d40a46f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20ee42e640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20e81975c0 0x7f20e8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20ee42e640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20e8106450 0x7f20e819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20ee42e640 1 --2- 172.21.15.18:0/3580416900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e8101380 0x7f20e819d410 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:20.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20ee42e640 1 -- 172.21.15.18:0/3580416900 >> 172.21.15.18:0/3580416900 conn(0x7f20e8078580 msgr2=0x7f20e80ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:20.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20ee42e640 1 -- 172.21.15.18:0/3580416900 shutdown_connections 2024-02-17T12:57:20.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:20.433+0000 7f20ee42e640 1 -- 172.21.15.18:0/3580416900 wait complete. 2024-02-17T12:57:21.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 388368 5.52 MB/sec execute 276 sec latency 1263.324 ms 2024-02-17T12:57:21.435 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:57:21.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:57:21.435 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:57:21.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.564+0000 7f29632ec640 1 -- 172.21.15.18:0/2384621937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f295c101390 msgr2=0x7f295c101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.564+0000 7f29632ec640 1 --2- 172.21.15.18:0/2384621937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f295c101390 0x7f295c101770 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f2958009930 tx=0x7f295802bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:21.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.564+0000 7f29632ec640 1 -- 172.21.15.18:0/2384621937 shutdown_connections 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.564+0000 7f29632ec640 1 --2- 172.21.15.18:0/2384621937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f295c106460 0x7f295c10e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.564+0000 7f29632ec640 1 --2- 172.21.15.18:0/2384621937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c101d40 0x7f295c105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.564+0000 7f29632ec640 1 --2- 172.21.15.18:0/2384621937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f295c101390 0x7f295c101770 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.564+0000 7f29632ec640 1 -- 172.21.15.18:0/2384621937 >> 172.21.15.18:0/2384621937 conn(0x7f295c0fcbb0 msgr2=0x7f295c0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.564+0000 7f29632ec640 1 -- 172.21.15.18:0/2384621937 shutdown_connections 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.564+0000 7f29632ec640 1 -- 172.21.15.18:0/2384621937 wait complete. 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f29632ec640 1 Processor -- start 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f29632ec640 1 -- start start 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f29632ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c101390 0x7f295c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f29632ec640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f295c106460 0x7f295c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f29632ec640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f295c073f00 0x7f295c1a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f29632ec640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1a5fd0 con 0x7f295c106460 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f29632ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1a6140 con 0x7f295c101390 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f29632ec640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1a6300 con 0x7f295c073f00 2024-02-17T12:57:21.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f2960887640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f295c106460 0x7f295c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f2960887640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f295c106460 0x7f295c0739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44220/0 (socket says 172.21.15.18:44220) 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f2960887640 1 -- 172.21.15.18:0/1617942655 learned_addr learned my addr 172.21.15.18:0/1617942655 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f2961889640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f295c073f00 0x7f295c1a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.565+0000 7f2961088640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c101390 0x7f295c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.566+0000 7f2960887640 1 -- 172.21.15.18:0/1617942655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f295c073f00 msgr2=0x7f295c1a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.566+0000 7f2960887640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f295c073f00 0x7f295c1a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.566+0000 7f2960887640 1 -- 172.21.15.18:0/1617942655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c101390 msgr2=0x7f295c073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.566+0000 7f2960887640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c101390 0x7f295c073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.566+0000 7f2960887640 1 -- 172.21.15.18:0/1617942655 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2958009560 con 0x7f295c106460 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.566+0000 7f2961088640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c101390 0x7f295c073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.566+0000 7f2960887640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f295c106460 0x7f295c0739c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f294c005fd0 tx=0x7f294c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.566+0000 7f29527fc640 1 -- 172.21.15.18:0/1617942655 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c009c30 con 0x7f295c106460 2024-02-17T12:57:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.566+0000 7f29527fc640 1 -- 172.21.15.18:0/1617942655 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f294c009d90 con 0x7f295c106460 2024-02-17T12:57:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.566+0000 7f29632ec640 1 -- 172.21.15.18:0/1617942655 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295c1a65e0 con 0x7f295c106460 2024-02-17T12:57:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.566+0000 7f29527fc640 1 -- 172.21.15.18:0/1617942655 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c010460 con 0x7f295c106460 2024-02-17T12:57:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.566+0000 7f29632ec640 1 -- 172.21.15.18:0/1617942655 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c1a6af0 con 0x7f295c106460 2024-02-17T12:57:21.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.568+0000 7f29527fc640 1 -- 172.21.15.18:0/1617942655 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f294c003f20 con 0x7f295c106460 2024-02-17T12:57:21.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2965f95740 1 Processor -- start 2024-02-17T12:57:21.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f29527fc640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f29480a2260 0x7f29480a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2965f95740 1 -- start start 2024-02-17T12:57:21.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2965f95740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80380 0x55abeac80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f29527fc640 1 -- 172.21.15.18:0/1617942655 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f294c0cdd00 con 0x7f295c106460 2024-02-17T12:57:21.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2961889640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f29480a2260 0x7f29480a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2965f95740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 0x55abeac892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2965f95740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 0x55abeac8bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2965f95740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeabeef90 con 0x55abeac80d30 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2965f95740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeabec4c0 con 0x55abeac80380 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2965f95740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeabe7ca0 con 0x55abeac898b0 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2960887640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 0x55abeac892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2960887640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 0x55abeac892e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44234/0 (socket says 172.21.15.18:44234) 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2960887640 1 -- 172.21.15.18:0/2186035750 learned_addr learned my addr 172.21.15.18:0/2186035750 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2961889640 1 --2- 172.21.15.18:0/2186035750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 0x55abeac8bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2961088640 1 --2- 172.21.15.18:0/2186035750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80380 0x55abeac80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2960887640 1 -- 172.21.15.18:0/2186035750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 msgr2=0x55abeac8bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2960887640 1 --2- 172.21.15.18:0/2186035750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 0x55abeac8bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2960887640 1 -- 172.21.15.18:0/2186035750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80380 msgr2=0x55abeac80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2960887640 1 --2- 172.21.15.18:0/2186035750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80380 0x55abeac80760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2960887640 1 -- 172.21.15.18:0/2186035750 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abeabaa0b0 con 0x55abeac80d30 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.569+0000 7f2961088640 1 --2- 172.21.15.18:0/2186035750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80380 0x55abeac80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.570+0000 7f2961889640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f29480a2260 0x7f29480a4720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2954005fd0 tx=0x7f2954011810 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.570+0000 7f2960887640 1 --2- 172.21.15.18:0/2186035750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 0x55abeac892e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f294c005c20 tx=0x7f294c09a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5435d82fe5357eb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.570+0000 7f29477fe640 1 -- 172.21.15.18:0/2186035750 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c09a710 con 0x55abeac80d30 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.570+0000 7f29477fe640 1 -- 172.21.15.18:0/2186035750 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f294c09e860 con 0x55abeac80d30 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.570+0000 7f29477fe640 1 -- 172.21.15.18:0/2186035750 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c096d80 con 0x55abeac80d30 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.570+0000 7f2965f95740 1 -- 172.21.15.18:0/2186035750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 msgr2=0x55abeac892e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.570+0000 7f2965f95740 1 --2- 172.21.15.18:0/2186035750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 0x55abeac892e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f294c005c20 tx=0x7f294c09a610 comp rx=0 tx=0).stop 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 -- 172.21.15.18:0/2186035750 shutdown_connections 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 --2- 172.21.15.18:0/2186035750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 0x55abeac8bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 --2- 172.21.15.18:0/2186035750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 0x55abeac892e0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 --2- 172.21.15.18:0/2186035750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80380 0x55abeac80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 -- 172.21.15.18:0/2186035750 >> 172.21.15.18:0/2186035750 conn(0x55abeac7a840 msgr2=0x55abeac7ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 -- 172.21.15.18:0/2186035750 shutdown_connections 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 -- 172.21.15.18:0/2186035750 wait complete. 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 -- start start 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 Processor -- start 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 0x55abeac86800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80d30 0x55abeac86d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 0x55abeac87280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeab8cf40 con 0x55abeac80380 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abea8bb4b0 con 0x55abeac80d30 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeab6d670 con 0x55abeac898b0 2024-02-17T12:57:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.571+0000 7f2965f95740 10 client.0 _ll_register_callbacks cb 0x55abeabdaf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2961088640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 0x55abeac86800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2961088640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 0x55abeac86800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44244/0 (socket says 172.21.15.18:44244) 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2961088640 1 -- 172.21.15.18:0/1926190972 learned_addr learned my addr 172.21.15.18:0/1926190972 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2960887640 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80d30 0x55abeac86d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2961889640 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 0x55abeac87280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2961088640 1 -- 172.21.15.18:0/1926190972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 msgr2=0x55abeac87280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2961088640 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 0x55abeac87280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2961088640 1 -- 172.21.15.18:0/1926190972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80d30 msgr2=0x55abeac86d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2961088640 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80d30 0x55abeac86d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2961088640 1 -- 172.21.15.18:0/1926190972 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c1a6af0 con 0x55abeac80380 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2961088640 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 0x55abeac86800 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f2958005ab0 tx=0x7f295802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5038ab7c30235eba server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2944ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2944ff9640 1 -- 172.21.15.18:0/1926190972 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958004460 con 0x55abeac80380 2024-02-17T12:57:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2944ff9640 1 -- 172.21.15.18:0/1926190972 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f295802cc00 con 0x55abeac80380 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2965f95740 1 -- 172.21.15.18:0/1926190972 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abeabc1f40 con 0x55abeac80380 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2944ff9640 1 -- 172.21.15.18:0/1926190972 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2958004df0 con 0x55abeac80380 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2965f95740 10 client.18282 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.572+0000 7f2965f95740 1 -- 172.21.15.18:0/1926190972 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55abeabe6320 con 0x55abeac80380 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.573+0000 7f2944ff9640 1 -- 172.21.15.18:0/1926190972 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55abeabc1f40 con 0x55abeac80380 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.573+0000 7f2944ff9640 1 -- 172.21.15.18:0/1926190972 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f295803c040 con 0x55abeac80380 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.573+0000 7f2965f95740 10 client.18282 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.573+0000 7f2965f95740 10 client.18282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.573+0000 7f2965f95740 20 client.18282 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.573+0000 7f2965f95740 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55abeac99e60 0x55abeac9c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.573+0000 7f2965f95740 4 client.18282 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.573+0000 7f2965f95740 1 -- 172.21.15.18:0/1926190972 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abeabde310 con 0x55abeac99e60 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.573+0000 7f2960887640 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55abeac99e60 0x55abeac9c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2944ff9640 10 client.18282 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2960887640 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55abeac99e60 0x55abeac9c280 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7f0e51b5113c5c server_cookie=e6ce497dc74035a9 in_seq=0 out_seq=0 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2944ff9640 1 -- 172.21.15.18:0/1926190972 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55abeabde310 con 0x55abeac99e60 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2944ff9640 10 client.18282 handle_command_reply: tid=0 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2965f95740 1 client.18282 shutdown 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2965f95740 1 -- 172.21.15.18:0/1926190972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 msgr2=0x55abeac86800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2965f95740 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 0x55abeac86800 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f2958005ab0 tx=0x7f295802ca10 comp rx=0 tx=0).stop 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2965f95740 1 -- 172.21.15.18:0/1926190972 shutdown_connections 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2965f95740 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55abeac99e60 0x55abeac9c280 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2965f95740 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 0x55abeac87280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2965f95740 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80d30 0x55abeac86d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2944ff9640 0 client.18282 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2965f95740 1 --2- 172.21.15.18:0/1926190972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 0x55abeac86800 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.574+0000 7f2965f95740 1 -- 172.21.15.18:0/1926190972 >> 172.21.15.18:0/1926190972 conn(0x55abeac7a840 msgr2=0x55abeac7c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.575+0000 7f2965f95740 1 -- 172.21.15.18:0/1926190972 shutdown_connections 2024-02-17T12:57:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.575+0000 7f2965f95740 1 -- 172.21.15.18:0/1926190972 wait complete. 2024-02-17T12:57:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.575+0000 7f2965f95740 20 client.18282 trim_cache size 0 max 16384 2024-02-17T12:57:21.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2965f95740 1 Processor -- start 2024-02-17T12:57:21.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2965f95740 1 -- start start 2024-02-17T12:57:21.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2965f95740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac80380 0x55abeac88f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2965f95740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 0x55abeac870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2965f95740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac898b0 0x55abeac83a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2965f95740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeabc1f40 con 0x55abeac80d30 2024-02-17T12:57:21.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2965f95740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeac81670 con 0x55abeac898b0 2024-02-17T12:57:21.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2965f95740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeac74da0 con 0x55abeac80380 2024-02-17T12:57:21.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2960887640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 0x55abeac870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2960887640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 0x55abeac870a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44258/0 (socket says 172.21.15.18:44258) 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2960887640 1 -- 172.21.15.18:0/3641199589 learned_addr learned my addr 172.21.15.18:0/3641199589 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2961088640 1 --2- 172.21.15.18:0/3641199589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac80380 0x55abeac88f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2960887640 1 -- 172.21.15.18:0/3641199589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac80380 msgr2=0x55abeac88f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2960887640 1 --2- 172.21.15.18:0/3641199589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac80380 0x55abeac88f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2960887640 1 -- 172.21.15.18:0/3641199589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac898b0 msgr2=0x55abeac83a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2960887640 1 --2- 172.21.15.18:0/3641199589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac898b0 0x55abeac83a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.590+0000 7f2960887640 1 -- 172.21.15.18:0/3641199589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abeab82c00 con 0x55abeac80d30 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2960887640 1 --2- 172.21.15.18:0/3641199589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 0x55abeac870a0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f294c009610 tx=0x7f294c01d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb6b69b204425ac server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2946ffd640 1 -- 172.21.15.18:0/3641199589 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c062070 con 0x55abeac80d30 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2946ffd640 1 -- 172.21.15.18:0/3641199589 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f294c09a710 con 0x55abeac80d30 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2946ffd640 1 -- 172.21.15.18:0/3641199589 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c0cb6f0 con 0x55abeac80d30 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2965f95740 1 -- 172.21.15.18:0/3641199589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 msgr2=0x55abeac870a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2965f95740 1 --2- 172.21.15.18:0/3641199589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 0x55abeac870a0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f294c009610 tx=0x7f294c01d040 comp rx=0 tx=0).stop 2024-02-17T12:57:21.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2965f95740 1 -- 172.21.15.18:0/3641199589 shutdown_connections 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2965f95740 1 --2- 172.21.15.18:0/3641199589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac898b0 0x55abeac83a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2965f95740 1 --2- 172.21.15.18:0/3641199589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80d30 0x55abeac870a0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2965f95740 1 --2- 172.21.15.18:0/3641199589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac80380 0x55abeac88f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2965f95740 1 -- 172.21.15.18:0/3641199589 >> 172.21.15.18:0/3641199589 conn(0x55abeac7a840 msgr2=0x55abeac850d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2965f95740 1 -- 172.21.15.18:0/3641199589 shutdown_connections 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.591+0000 7f2965f95740 1 -- 172.21.15.18:0/3641199589 wait complete. 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2965f95740 1 -- start start 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2965f95740 1 Processor -- start 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2965f95740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 0x55abeac8f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2965f95740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80d30 0x55abeac8fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2965f95740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 0x55abeac900b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2965f95740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeacf0520 con 0x55abeac80380 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2965f95740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeacf0690 con 0x55abeac80d30 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2965f95740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeacf0850 con 0x55abeac898b0 2024-02-17T12:57:21.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2965f95740 10 client.0 _ll_register_callbacks cb 0x7f294c009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2961088640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 0x55abeac8f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2961088640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 0x55abeac8f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44266/0 (socket says 172.21.15.18:44266) 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2961088640 1 -- 172.21.15.18:0/3292269414 learned_addr learned my addr 172.21.15.18:0/3292269414 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2961889640 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 0x55abeac900b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2960887640 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80d30 0x55abeac8fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2961088640 1 -- 172.21.15.18:0/3292269414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 msgr2=0x55abeac900b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2961088640 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 0x55abeac900b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.592+0000 7f2961088640 1 -- 172.21.15.18:0/3292269414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80d30 msgr2=0x55abeac8fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2961088640 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80d30 0x55abeac8fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2961088640 1 -- 172.21.15.18:0/3292269414 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abeabaa0b0 con 0x55abeac80380 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2960887640 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80d30 0x55abeac8fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2961088640 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 0x55abeac8f630 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f295803e480 tx=0x7f2958009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86d33dcd11e3150 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:21.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2944ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2944ff9640 1 -- 172.21.15.18:0/3292269414 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958004460 con 0x55abeac80380 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2944ff9640 1 -- 172.21.15.18:0/3292269414 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2958000e40 con 0x55abeac80380 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2965f95740 1 -- 172.21.15.18:0/3292269414 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abeac0d900 con 0x55abeac80380 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2944ff9640 1 -- 172.21.15.18:0/3292269414 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2958003ee0 con 0x55abeac80380 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2965f95740 10 client.18288 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2965f95740 1 -- 172.21.15.18:0/3292269414 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55abeaba8910 con 0x55abeac80380 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2944ff9640 1 -- 172.21.15.18:0/3292269414 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55abeac0d900 con 0x55abeac80380 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2944ff9640 1 -- 172.21.15.18:0/3292269414 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f295803c070 con 0x55abeac80380 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2965f95740 10 client.18288 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2965f95740 10 client.18288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2965f95740 20 client.18288 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2965f95740 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55abeacf1ed0 0x55abeacf42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2965f95740 4 client.18288 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:57:21.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.593+0000 7f2965f95740 1 -- 172.21.15.18:0/3292269414 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55abea8e7860 con 0x55abeacf1ed0 2024-02-17T12:57:21.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.594+0000 7f2960887640 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55abeacf1ed0 0x55abeacf42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.594+0000 7f2944ff9640 10 client.18288 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:21.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.594+0000 7f2960887640 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55abeacf1ed0 0x55abeacf42f0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1643d13a23ab8141 server_cookie=194b3dc7505d4dc4 in_seq=0 out_seq=0 2024-02-17T12:57:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.595+0000 7f2944ff9640 1 -- 172.21.15.18:0/3292269414 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55abea8e7860 con 0x55abeacf1ed0 2024-02-17T12:57:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.595+0000 7f2944ff9640 10 client.18288 handle_command_reply: tid=0 2024-02-17T12:57:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.595+0000 7f2965f95740 1 client.18288 shutdown 2024-02-17T12:57:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.595+0000 7f2965f95740 1 -- 172.21.15.18:0/3292269414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 msgr2=0x55abeac8f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.595+0000 7f2965f95740 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 0x55abeac8f630 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f295803e480 tx=0x7f2958009780 comp rx=0 tx=0).stop 2024-02-17T12:57:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.595+0000 7f2965f95740 1 -- 172.21.15.18:0/3292269414 shutdown_connections 2024-02-17T12:57:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.595+0000 7f2965f95740 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55abeacf1ed0 0x55abeacf42f0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.595+0000 7f2965f95740 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abeac898b0 0x55abeac900b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.595+0000 7f2944ff9640 0 client.18288 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.595+0000 7f2965f95740 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abeac80d30 0x55abeac8fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.595+0000 7f2965f95740 1 --2- 172.21.15.18:0/3292269414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abeac80380 0x55abeac8f630 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.595+0000 7f2965f95740 1 -- 172.21.15.18:0/3292269414 >> 172.21.15.18:0/3292269414 conn(0x55abeac7a840 msgr2=0x55abeac7be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f2965f95740 1 -- 172.21.15.18:0/3292269414 shutdown_connections 2024-02-17T12:57:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f2965f95740 1 -- 172.21.15.18:0/3292269414 wait complete. 2024-02-17T12:57:21.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f2965f95740 20 client.18288 trim_cache size 0 max 16384 2024-02-17T12:57:21.592 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:57:21.592 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:57:21.593 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "ab9802c6-87ee-48b3-8c7d-0f3d706999d2", 2024-02-17T12:57:21.593 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:57:21.593 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:57:21.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f29632ec640 1 -- 172.21.15.18:0/1617942655 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f29480a2260 msgr2=0x7f29480a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f29632ec640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f29480a2260 0x7f29480a4720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2954005fd0 tx=0x7f2954011810 comp rx=0 tx=0).stop 2024-02-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f29632ec640 1 -- 172.21.15.18:0/1617942655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f295c106460 msgr2=0x7f295c0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f29632ec640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f295c106460 0x7f295c0739c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f294c005fd0 tx=0x7f294c00dde0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f29632ec640 1 -- 172.21.15.18:0/1617942655 shutdown_connections 2024-02-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f29632ec640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f29480a2260 0x7f29480a4720 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f29632ec640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f295c073f00 0x7f295c1a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f29632ec640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f295c106460 0x7f295c0739c0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f29632ec640 1 --2- 172.21.15.18:0/1617942655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c101390 0x7f295c073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f29632ec640 1 -- 172.21.15.18:0/1617942655 >> 172.21.15.18:0/1617942655 conn(0x7f295c0fcbb0 msgr2=0x7f295c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f29632ec640 1 -- 172.21.15.18:0/1617942655 shutdown_connections 2024-02-17T12:57:21.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.596+0000 7f29632ec640 1 -- 172.21.15.18:0/1617942655 wait complete. 2024-02-17T12:57:21.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:57:21.598 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:57:21.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.717+0000 7f3f06765640 1 -- 172.21.15.18:0/640981241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f001061a0 msgr2=0x7f3f00106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.717+0000 7f3f06765640 1 --2- 172.21.15.18:0/640981241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f001061a0 0x7f3f00106580 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009930 tx=0x7f3ef002bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:21.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.717+0000 7f3f06765640 1 -- 172.21.15.18:0/640981241 shutdown_connections 2024-02-17T12:57:21.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.717+0000 7f3f06765640 1 --2- 172.21.15.18:0/640981241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f00100c90 0x7f3f0010ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.717+0000 7f3f06765640 1 --2- 172.21.15.18:0/640981241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f001002d0 0x7f3f00100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.717+0000 7f3f06765640 1 --2- 172.21.15.18:0/640981241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f001061a0 0x7f3f00106580 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.717+0000 7f3f06765640 1 -- 172.21.15.18:0/640981241 >> 172.21.15.18:0/640981241 conn(0x7f3f00076e10 msgr2=0x7f3f00077220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:21.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.717+0000 7f3f06765640 1 -- 172.21.15.18:0/640981241 shutdown_connections 2024-02-17T12:57:21.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.717+0000 7f3f06765640 1 -- 172.21.15.18:0/640981241 wait complete. 2024-02-17T12:57:21.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.718+0000 7f3f06765640 1 Processor -- start 2024-02-17T12:57:21.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.718+0000 7f3f06765640 1 -- start start 2024-02-17T12:57:21.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.718+0000 7f3f06765640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f001002d0 0x7f3f00196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.718+0000 7f3f06765640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f00100c90 0x7f3f00196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.718+0000 7f3f06765640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f001061a0 0x7f3f0019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.718+0000 7f3f06765640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0019b500 con 0x7f3f00100c90 2024-02-17T12:57:21.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.718+0000 7f3f06765640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0019b670 con 0x7f3f001061a0 2024-02-17T12:57:21.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.718+0000 7f3f06765640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0019b830 con 0x7f3f001002d0 2024-02-17T12:57:21.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3eff7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f00100c90 0x7f3f00196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3eff7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f00100c90 0x7f3f00196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44282/0 (socket says 172.21.15.18:44282) 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3eff7fe640 1 -- 172.21.15.18:0/1225620502 learned_addr learned my addr 172.21.15.18:0/1225620502 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3efffff640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f001002d0 0x7f3f00196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3f04d02640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f001061a0 0x7f3f0019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3eff7fe640 1 -- 172.21.15.18:0/1225620502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f001002d0 msgr2=0x7f3f00196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3eff7fe640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f001002d0 0x7f3f00196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3eff7fe640 1 -- 172.21.15.18:0/1225620502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f001061a0 msgr2=0x7f3f0019af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3eff7fe640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f001061a0 0x7f3f0019af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3eff7fe640 1 -- 172.21.15.18:0/1225620502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef0009560 con 0x7f3f00100c90 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3eff7fe640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f00100c90 0x7f3f00196bd0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400e9f0 tx=0x7f3ef400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3efd7fa640 1 -- 172.21.15.18:0/1225620502 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4009700 con 0x7f3f00100c90 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3efd7fa640 1 -- 172.21.15.18:0/1225620502 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ef400cd80 con 0x7f3f00100c90 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3f06765640 1 -- 172.21.15.18:0/1225620502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0019bb10 con 0x7f3f00100c90 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3efd7fa640 1 -- 172.21.15.18:0/1225620502 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4018920 con 0x7f3f00100c90 2024-02-17T12:57:21.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.719+0000 7f3f06765640 1 -- 172.21.15.18:0/1225620502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f001a3390 con 0x7f3f00100c90 2024-02-17T12:57:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3efd7fa640 1 -- 172.21.15.18:0/1225620502 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3ef4018a80 con 0x7f3f00100c90 2024-02-17T12:57:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3f09411740 1 Processor -- start 2024-02-17T12:57:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3f09411740 1 -- start start 2024-02-17T12:57:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3efd7fa640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ee80a2230 0x7f3ee80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3f09411740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 0x5597c549c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3f09411740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549c900 0x5597c54a7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3f09411740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c54a79c0 0x5597c54a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3f09411740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597c540b280 con 0x5597c549bfe0 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3f09411740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597c54087b0 con 0x5597c549c900 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3f09411740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597c5403f90 con 0x5597c54a79c0 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3efffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 0x5597c549c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3efffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 0x5597c549c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44294/0 (socket says 172.21.15.18:44294) 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3efffff640 1 -- 172.21.15.18:0/3117158283 learned_addr learned my addr 172.21.15.18:0/3117158283 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3f04d02640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ee80a2230 0x7f3ee80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.722+0000 7f3efd7fa640 1 -- 172.21.15.18:0/1225620502 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3ef4014070 con 0x7f3f00100c90 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3eff7fe640 1 --2- 172.21.15.18:0/3117158283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549c900 0x5597c54a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3f04d02640 1 --2- 172.21.15.18:0/3117158283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c54a79c0 0x5597c54a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3efffff640 1 -- 172.21.15.18:0/3117158283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c54a79c0 msgr2=0x5597c54a9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3efffff640 1 --2- 172.21.15.18:0/3117158283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c54a79c0 0x5597c54a9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3efffff640 1 -- 172.21.15.18:0/3117158283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549c900 msgr2=0x5597c54a7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3efffff640 1 --2- 172.21.15.18:0/3117158283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549c900 0x5597c54a7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3efffff640 1 -- 172.21.15.18:0/3117158283 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597c53c2570 con 0x5597c549bfe0 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3eff7fe640 1 --2- 172.21.15.18:0/3117158283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549c900 0x5597c54a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3efffff640 1 --2- 172.21.15.18:0/3117158283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 0x5597c549c3c0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0036040 tx=0x7f3ef002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbccc0c9cfda8a57 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3f04d02640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ee80a2230 0x7f3ee80a46f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x5597c5497b70 tx=0x7f3ef8009da0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3ee67fc640 1 -- 172.21.15.18:0/3117158283 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0004800 con 0x5597c549bfe0 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3ee67fc640 1 -- 172.21.15.18:0/3117158283 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ef0004960 con 0x5597c549bfe0 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.723+0000 7f3ee67fc640 1 -- 172.21.15.18:0/3117158283 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0005120 con 0x5597c549bfe0 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.724+0000 7f3f09411740 1 -- 172.21.15.18:0/3117158283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 msgr2=0x5597c549c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.724+0000 7f3f09411740 1 --2- 172.21.15.18:0/3117158283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 0x5597c549c3c0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0036040 tx=0x7f3ef002c5d0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.724+0000 7f3f09411740 1 -- 172.21.15.18:0/3117158283 shutdown_connections 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.724+0000 7f3f09411740 1 --2- 172.21.15.18:0/3117158283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c54a79c0 0x5597c54a9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.724+0000 7f3f09411740 1 --2- 172.21.15.18:0/3117158283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549c900 0x5597c54a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.724+0000 7f3f09411740 1 --2- 172.21.15.18:0/3117158283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 0x5597c549c3c0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.724+0000 7f3f09411740 1 -- 172.21.15.18:0/3117158283 >> 172.21.15.18:0/3117158283 conn(0x5597c5496990 msgr2=0x5597c5496d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.724+0000 7f3f09411740 1 -- 172.21.15.18:0/3117158283 shutdown_connections 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.724+0000 7f3f09411740 1 -- 172.21.15.18:0/3117158283 wait complete. 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3f09411740 1 -- start start 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3f09411740 1 Processor -- start 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3f09411740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 0x5597c54a2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3f09411740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 0x5597c54a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3f09411740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 0x5597c54a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3f09411740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597c50d7430 con 0x5597c54a79c0 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3f09411740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597c5093e00 con 0x5597c549bfe0 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3f09411740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597c5388fd0 con 0x5597c549c900 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3f09411740 10 client.0 _ll_register_callbacks cb 0x5597c53defd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3efffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 0x5597c54a2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3f04d02640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 0x5597c54a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3eff7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 0x5597c54a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3efffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 0x5597c54a2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:33526/0 (socket says 172.21.15.18:33526) 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3f04d02640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 0x5597c54a32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44310/0 (socket says 172.21.15.18:44310) 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.725+0000 7f3efffff640 1 -- 172.21.15.18:0/1533923931 learned_addr learned my addr 172.21.15.18:0/1533923931 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3efffff640 1 -- 172.21.15.18:0/1533923931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 msgr2=0x5597c54a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3efffff640 1 --2- 172.21.15.18:0/1533923931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 0x5597c54a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3efffff640 1 -- 172.21.15.18:0/1533923931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 msgr2=0x5597c54a32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3efffff640 1 --2- 172.21.15.18:0/1533923931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 0x5597c54a32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3efffff640 1 -- 172.21.15.18:0/1533923931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0010d760 con 0x5597c549bfe0 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3f04d02640 1 --2- 172.21.15.18:0/1533923931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 0x5597c54a32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3efffff640 1 --2- 172.21.15.18:0/1533923931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 0x5597c54a2860 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ef002c170 tx=0x7f3ef00044a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d21772baba96d42 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3edf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3edf7fe640 1 -- 172.21.15.18:0/1533923931 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef004f070 con 0x5597c549bfe0 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3f09411740 1 -- 172.21.15.18:0/1533923931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597c51037e0 con 0x5597c549bfe0 2024-02-17T12:57:21.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3edf7fe640 1 -- 172.21.15.18:0/1533923931 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3ef002cbe0 con 0x5597c549bfe0 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3edf7fe640 1 -- 172.21.15.18:0/1533923931 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ef0004800 con 0x5597c549bfe0 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3f09411740 10 client.27470 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3f09411740 1 -- 172.21.15.18:0/1533923931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5597c5402610 con 0x5597c549bfe0 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.726+0000 7f3edf7fe640 1 -- 172.21.15.18:0/1533923931 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5597c51037e0 con 0x5597c549bfe0 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.727+0000 7f3edf7fe640 1 -- 172.21.15.18:0/1533923931 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3ef0052020 con 0x5597c549bfe0 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.727+0000 7f3f09411740 10 client.27470 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.727+0000 7f3f09411740 10 client.27470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.727+0000 7f3f09411740 20 client.27470 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.727+0000 7f3f09411740 1 --2- 172.21.15.18:0/1533923931 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5597c54b5610 0x5597c54b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.727+0000 7f3f09411740 4 client.27470 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.727+0000 7f3f09411740 1 -- 172.21.15.18:0/1533923931 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597c5140720 con 0x5597c54b5610 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.727+0000 7f3eff7fe640 1 --2- 172.21.15.18:0/1533923931 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5597c54b5610 0x5597c54b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.727+0000 7f3edf7fe640 10 client.27470 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.727+0000 7f3eff7fe640 1 --2- 172.21.15.18:0/1533923931 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5597c54b5610 0x5597c54b7a00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=647068c08e1a786a server_cookie=3b5e78970104e0d8 in_seq=0 out_seq=0 2024-02-17T12:57:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3edf7fe640 1 -- 172.21.15.18:0/1533923931 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5597c5140720 con 0x5597c54b5610 2024-02-17T12:57:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3edf7fe640 10 client.27470 handle_command_reply: tid=0 2024-02-17T12:57:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3f09411740 1 client.27470 shutdown 2024-02-17T12:57:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3f09411740 1 -- 172.21.15.18:0/1533923931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 msgr2=0x5597c54a2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3f09411740 1 --2- 172.21.15.18:0/1533923931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 0x5597c54a2860 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ef002c170 tx=0x7f3ef00044a0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3f09411740 1 -- 172.21.15.18:0/1533923931 shutdown_connections 2024-02-17T12:57:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3f09411740 1 --2- 172.21.15.18:0/1533923931 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5597c54b5610 0x5597c54b7a00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3f09411740 1 --2- 172.21.15.18:0/1533923931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 0x5597c54a32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3f09411740 1 --2- 172.21.15.18:0/1533923931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 0x5597c54a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3edf7fe640 0 client.27470 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:21.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3f09411740 1 --2- 172.21.15.18:0/1533923931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 0x5597c54a2860 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3f09411740 1 -- 172.21.15.18:0/1533923931 >> 172.21.15.18:0/1533923931 conn(0x5597c5496990 msgr2=0x5597c5498730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:21.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3f09411740 1 -- 172.21.15.18:0/1533923931 shutdown_connections 2024-02-17T12:57:21.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3f09411740 1 -- 172.21.15.18:0/1533923931 wait complete. 2024-02-17T12:57:21.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.729+0000 7f3f09411740 20 client.27470 trim_cache size 0 max 16384 2024-02-17T12:57:21.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f09411740 1 Processor -- start 2024-02-17T12:57:21.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f09411740 1 -- start start 2024-02-17T12:57:21.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f09411740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 0x5597c54a3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f09411740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 0x5597c54984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f09411740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 0x5597c54ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f09411740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597c54967f0 con 0x5597c54a79c0 2024-02-17T12:57:21.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f09411740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597c5497790 con 0x5597c549bfe0 2024-02-17T12:57:21.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f09411740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597c549d030 con 0x5597c549c900 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f04d02640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 0x5597c54ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f04d02640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 0x5597c54ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44314/0 (socket says 172.21.15.18:44314) 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f04d02640 1 -- 172.21.15.18:0/1699974918 learned_addr learned my addr 172.21.15.18:0/1699974918 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3efffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 0x5597c54a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3eff7fe640 1 --2- 172.21.15.18:0/1699974918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 0x5597c54984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f04d02640 1 -- 172.21.15.18:0/1699974918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 msgr2=0x5597c54984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f04d02640 1 --2- 172.21.15.18:0/1699974918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 0x5597c54984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f04d02640 1 -- 172.21.15.18:0/1699974918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 msgr2=0x5597c54a3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f04d02640 1 --2- 172.21.15.18:0/1699974918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 0x5597c54a3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f04d02640 1 -- 172.21.15.18:0/1699974918 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597c539eac0 con 0x5597c54a79c0 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3efffff640 1 --2- 172.21.15.18:0/1699974918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 0x5597c54a3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.746+0000 7f3f04d02640 1 --2- 172.21.15.18:0/1699974918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 0x5597c54ae3b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800ed70 tx=0x7f3ef8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429a159d199ae14c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.747+0000 7f3ee5ffb640 1 -- 172.21.15.18:0/1699974918 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8018070 con 0x5597c54a79c0 2024-02-17T12:57:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.747+0000 7f3ee5ffb640 1 -- 172.21.15.18:0/1699974918 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ef8025e90 con 0x5597c54a79c0 2024-02-17T12:57:21.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.747+0000 7f3ee5ffb640 1 -- 172.21.15.18:0/1699974918 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8020520 con 0x5597c54a79c0 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.747+0000 7f3f09411740 1 -- 172.21.15.18:0/1699974918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 msgr2=0x5597c54ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.747+0000 7f3f09411740 1 --2- 172.21.15.18:0/1699974918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 0x5597c54ae3b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800ed70 tx=0x7f3ef8018000 comp rx=0 tx=0).stop 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.747+0000 7f3f09411740 1 -- 172.21.15.18:0/1699974918 shutdown_connections 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.747+0000 7f3f09411740 1 --2- 172.21.15.18:0/1699974918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c54a79c0 0x5597c54ae3b0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.747+0000 7f3f09411740 1 --2- 172.21.15.18:0/1699974918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 0x5597c54984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.747+0000 7f3f09411740 1 --2- 172.21.15.18:0/1699974918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c549bfe0 0x5597c54a3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.747+0000 7f3f09411740 1 -- 172.21.15.18:0/1699974918 >> 172.21.15.18:0/1699974918 conn(0x5597c5496990 msgr2=0x5597c5497dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.747+0000 7f3f09411740 1 -- 172.21.15.18:0/1699974918 shutdown_connections 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3f09411740 1 -- 172.21.15.18:0/1699974918 wait complete. 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3f09411740 1 -- start start 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3f09411740 1 Processor -- start 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3f09411740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 0x5597c54ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3f09411740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 0x5597c54ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3f09411740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c54a79c0 0x5597c550baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3f09411740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597c54adfd0 con 0x5597c549bfe0 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3f09411740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597c550c030 con 0x5597c54a79c0 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3f09411740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597c550c170 con 0x5597c549c900 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3f09411740 10 client.0 _ll_register_callbacks cb 0x7f3ef80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3efffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 0x5597c54ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3efffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 0x5597c54ad190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44324/0 (socket says 172.21.15.18:44324) 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3efffff640 1 -- 172.21.15.18:0/2850154571 learned_addr learned my addr 172.21.15.18:0/2850154571 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3eff7fe640 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 0x5597c54ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.748+0000 7f3f04d02640 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c54a79c0 0x5597c550baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3efffff640 1 -- 172.21.15.18:0/2850154571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 msgr2=0x5597c54ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3efffff640 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 0x5597c54ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3efffff640 1 -- 172.21.15.18:0/2850154571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c54a79c0 msgr2=0x5597c550baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3efffff640 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c54a79c0 0x5597c550baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3efffff640 1 -- 172.21.15.18:0/2850154571 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597c5402610 con 0x5597c549bfe0 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3efffff640 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 0x5597c54ad190 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0004510 tx=0x7f3ef002c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536f74eea70f349f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3edf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3edf7fe640 1 -- 172.21.15.18:0/2850154571 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0043070 con 0x5597c549bfe0 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3edf7fe640 1 -- 172.21.15.18:0/2850154571 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3ef0000e40 con 0x5597c549bfe0 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3f09411740 1 -- 172.21.15.18:0/2850154571 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ef0004960 con 0x5597c549bfe0 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3edf7fe640 1 -- 172.21.15.18:0/2850154571 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ef004f070 con 0x5597c549bfe0 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3f09411740 10 client.18306 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3f09411740 1 -- 172.21.15.18:0/2850154571 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f3ef801b030 con 0x5597c549bfe0 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.749+0000 7f3edf7fe640 1 -- 172.21.15.18:0/2850154571 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ef0004960 con 0x5597c549bfe0 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3edf7fe640 1 -- 172.21.15.18:0/2850154571 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3ef0052040 con 0x5597c549bfe0 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3f09411740 10 client.18306 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3f09411740 10 client.18306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:21.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3f09411740 20 client.18306 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3f09411740 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5597c550d4b0 0x5597c550f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3f09411740 4 client.18306 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3f09411740 1 -- 172.21.15.18:0/2850154571 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5597c518f890 con 0x5597c550d4b0 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3eff7fe640 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5597c550d4b0 0x5597c550f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3edf7fe640 10 client.18306 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3eff7fe640 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5597c550d4b0 0x5597c550f8d0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63fa90e61e954348 server_cookie=ad9e64813d39cf91 in_seq=0 out_seq=0 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3edf7fe640 1 -- 172.21.15.18:0/2850154571 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5597c518f890 con 0x5597c550d4b0 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3edf7fe640 10 client.18306 handle_command_reply: tid=0 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.750+0000 7f3f09411740 1 client.18306 shutdown 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f09411740 1 -- 172.21.15.18:0/2850154571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 msgr2=0x5597c54ad190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f09411740 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 0x5597c54ad190 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0004510 tx=0x7f3ef002c330 comp rx=0 tx=0).stop 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f09411740 1 -- 172.21.15.18:0/2850154571 shutdown_connections 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f09411740 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5597c550d4b0 0x5597c550f8d0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f09411740 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597c54a79c0 0x5597c550baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f09411740 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5597c549c900 0x5597c54ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3edf7fe640 0 client.18306 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:21.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f09411740 1 --2- 172.21.15.18:0/2850154571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5597c549bfe0 0x5597c54ad190 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f09411740 1 -- 172.21.15.18:0/2850154571 >> 172.21.15.18:0/2850154571 conn(0x5597c5496990 msgr2=0x5597c5495a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:21.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f09411740 1 -- 172.21.15.18:0/2850154571 shutdown_connections 2024-02-17T12:57:21.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f09411740 1 -- 172.21.15.18:0/2850154571 wait complete. 2024-02-17T12:57:21.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f09411740 20 client.18306 trim_cache size 0 max 16384 2024-02-17T12:57:21.747 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:57:21.747 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:57:21.747 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:57:21.747 INFO:teuthology.orchestra.run.smithi018.stdout: "ab9802c6-87ee-48b3-8c7d-0f3d706999d2": { 2024-02-17T12:57:21.748 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:57:21.748 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "ab9802c6-87ee-48b3-8c7d-0f3d706999d2", 2024-02-17T12:57:21.748 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:57:21.748 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:57:21.748 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:57:21.748 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:57:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f06765640 1 -- 172.21.15.18:0/1225620502 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ee80a2230 msgr2=0x7f3ee80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f06765640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ee80a2230 0x7f3ee80a46f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x5597c5497b70 tx=0x7f3ef8009da0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f06765640 1 -- 172.21.15.18:0/1225620502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f00100c90 msgr2=0x7f3f00196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f06765640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f00100c90 0x7f3f00196bd0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400e9f0 tx=0x7f3ef400eeb0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f06765640 1 -- 172.21.15.18:0/1225620502 shutdown_connections 2024-02-17T12:57:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f06765640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ee80a2230 0x7f3ee80a46f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f06765640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f001061a0 0x7f3f0019af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f06765640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f00100c90 0x7f3f00196bd0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f06765640 1 --2- 172.21.15.18:0/1225620502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f001002d0 0x7f3f00196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:21.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f06765640 1 -- 172.21.15.18:0/1225620502 >> 172.21.15.18:0/1225620502 conn(0x7f3f00076e10 msgr2=0x7f3f000ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:21.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f06765640 1 -- 172.21.15.18:0/1225620502 shutdown_connections 2024-02-17T12:57:21.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:21.751+0000 7f3f06765640 1 -- 172.21.15.18:0/1225620502 wait complete. 2024-02-17T12:57:21.753 INFO:tasks.cephfs.filesystem:scrub status for tag:ab9802c6-87ee-48b3-8c7d-0f3d706999d2 - {'path': '/', 'tag': 'ab9802c6-87ee-48b3-8c7d-0f3d706999d2', 'options': 'recursive,force'} 2024-02-17T12:57:22.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:21 smithi018 ceph-mon[32625]: pgmap v232: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-02-17T12:57:22.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:21 smithi119 ceph-mon[36188]: pgmap v232: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-02-17T12:57:22.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:21 smithi146 ceph-mon[36454]: pgmap v232: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-02-17T12:57:22.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 390038 5.53 MB/sec execute 277 sec latency 1602.565 ms 2024-02-17T12:57:23.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:57:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:23.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:23.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:23.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:23.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:57:23.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:23.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:23.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:23.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:23.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:23.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:23.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:23.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:23.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:23.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:57:23.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:23.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:23.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:23.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:23.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:23.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:23.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:23.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:23.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:23.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:23.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 391458 5.53 MB/sec execute 278 sec latency 17.627 ms 2024-02-17T12:57:24.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:23 smithi119 ceph-mon[36188]: pgmap v233: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-02-17T12:57:24.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:23 smithi146 ceph-mon[36454]: pgmap v233: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-02-17T12:57:24.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:23 smithi018 ceph-mon[32625]: pgmap v233: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-02-17T12:57:24.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 392978 5.55 MB/sec execute 279 sec latency 170.255 ms 2024-02-17T12:57:25.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 394547 5.56 MB/sec execute 280 sec latency 17.473 ms 2024-02-17T12:57:26.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:25 smithi119 ceph-mon[36188]: pgmap v234: 97 pgs: 97 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-02-17T12:57:26.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:25 smithi146 ceph-mon[36454]: pgmap v234: 97 pgs: 97 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-02-17T12:57:26.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:25 smithi018 ceph-mon[32625]: pgmap v234: 97 pgs: 97 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-02-17T12:57:26.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 395459 5.55 MB/sec execute 281 sec latency 532.146 ms 2024-02-17T12:57:27.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 395459 5.53 MB/sec execute 282 sec latency 1532.234 ms 2024-02-17T12:57:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.791+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/2043132558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4101d70 msgr2=0x7fd6b4105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.791+0000 7fd6b8c4e640 1 --2- 172.21.15.18:0/2043132558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4101d70 0x7fd6b4105db0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009930 tx=0x7fd69c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.791+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/2043132558 shutdown_connections 2024-02-17T12:57:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.791+0000 7fd6b8c4e640 1 --2- 172.21.15.18:0/2043132558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b4106490 0x7fd6b410e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.791+0000 7fd6b8c4e640 1 --2- 172.21.15.18:0/2043132558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4101d70 0x7fd6b4105db0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.791+0000 7fd6b8c4e640 1 --2- 172.21.15.18:0/2043132558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6b41013c0 0x7fd6b41017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.791+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/2043132558 >> 172.21.15.18:0/2043132558 conn(0x7fd6b40fcc60 msgr2=0x7fd6b40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.791+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/2043132558 shutdown_connections 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.791+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/2043132558 wait complete. 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.792+0000 7fd6b8c4e640 1 Processor -- start 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.792+0000 7fd6b8c4e640 1 -- start start 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b8c4e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b41013c0 0x7fd6b4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b8c4e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6b4106490 0x7fd6b40739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b8c4e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4073f00 0x7fd6b41a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b8c4e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b41a5f50 con 0x7fd6b4073f00 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b8c4e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b41a60c0 con 0x7fd6b41013c0 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b8c4e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b41a6280 con 0x7fd6b4106490 2024-02-17T12:57:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4073f00 0x7fd6b41a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4073f00 0x7fd6b41a5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44350/0 (socket says 172.21.15.18:44350) 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b2d9d640 1 -- 172.21.15.18:0/1572127658 learned_addr learned my addr 172.21.15.18:0/1572127658 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b259c640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b41013c0 0x7fd6b4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6abfff640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6b4106490 0x7fd6b40739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b2d9d640 1 -- 172.21.15.18:0/1572127658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6b4106490 msgr2=0x7fd6b40739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b2d9d640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6b4106490 0x7fd6b40739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b2d9d640 1 -- 172.21.15.18:0/1572127658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b41013c0 msgr2=0x7fd6b4073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b2d9d640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b41013c0 0x7fd6b4073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b2d9d640 1 -- 172.21.15.18:0/1572127658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c009560 con 0x7fd6b4073f00 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6b2d9d640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4073f00 0x7fd6b41a5980 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00eaa0 tx=0x7fd6ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.793+0000 7fd6ab7fe640 1 -- 172.21.15.18:0/1572127658 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac00cdb0 con 0x7fd6b4073f00 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.794+0000 7fd6ab7fe640 1 -- 172.21.15.18:0/1572127658 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6ac005020 con 0x7fd6b4073f00 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.794+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/1572127658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b41a6560 con 0x7fd6b4073f00 2024-02-17T12:57:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.794+0000 7fd6ab7fe640 1 -- 172.21.15.18:0/1572127658 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac0116b0 con 0x7fd6b4073f00 2024-02-17T12:57:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.794+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/1572127658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b41a69f0 con 0x7fd6b4073f00 2024-02-17T12:57:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.796+0000 7fd6ab7fe640 1 -- 172.21.15.18:0/1572127658 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd6ac005520 con 0x7fd6b4073f00 2024-02-17T12:57:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.796+0000 7fd6bb81e740 1 Processor -- start 2024-02-17T12:57:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.796+0000 7fd6bb81e740 1 -- start start 2024-02-17T12:57:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.796+0000 7fd6bb81e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a2015ad0 0x5556a2014820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.796+0000 7fd6bb81e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2014d60 0x5556a20151e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6bb81e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a201e2e0 0x5556a20206d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6bb81e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a1f839f0 con 0x5556a2014d60 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6bb81e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a1f80f20 con 0x5556a2015ad0 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6bb81e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a1f7c700 con 0x5556a201e2e0 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6ab7fe640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd6a00a2300 0x7fd6a00a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2014d60 0x5556a20151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2014d60 0x5556a20151e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44362/0 (socket says 172.21.15.18:44362) 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6b259c640 1 -- 172.21.15.18:0/1902094551 learned_addr learned my addr 172.21.15.18:0/1902094551 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6b2d9d640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd6a00a2300 0x7fd6a00a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6abfff640 1 --2- 172.21.15.18:0/1902094551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a2015ad0 0x5556a2014820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6ab7fe640 1 -- 172.21.15.18:0/1572127658 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd6ac0ce0e0 con 0x7fd6b4073f00 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6abfff640 1 --2- 172.21.15.18:0/1902094551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a201e2e0 0x5556a20206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6b259c640 1 -- 172.21.15.18:0/1902094551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a201e2e0 msgr2=0x5556a20206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6b259c640 1 --2- 172.21.15.18:0/1902094551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a201e2e0 0x5556a20206d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6b259c640 1 -- 172.21.15.18:0/1902094551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a2015ad0 msgr2=0x5556a2014820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6b259c640 1 --2- 172.21.15.18:0/1902094551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a2015ad0 0x5556a2014820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6b259c640 1 -- 172.21.15.18:0/1902094551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556a1f127b0 con 0x5556a2014d60 2024-02-17T12:57:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6abfff640 1 --2- 172.21.15.18:0/1902094551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a2015ad0 0x5556a2014820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6b259c640 1 --2- 172.21.15.18:0/1902094551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2014d60 0x5556a20151e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400b760 tx=0x7fd6a400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d55d3be462bcb70 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6b2d9d640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd6a00a2300 0x7fd6a00a47c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0cb9a0 tx=0x7fd6ac009270 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6a8ff9640 1 -- 172.21.15.18:0/1902094551 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4003dd0 con 0x5556a2014d60 2024-02-17T12:57:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6a8ff9640 1 -- 172.21.15.18:0/1902094551 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a400be70 con 0x5556a2014d60 2024-02-17T12:57:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.797+0000 7fd6a8ff9640 1 -- 172.21.15.18:0/1902094551 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4011440 con 0x5556a2014d60 2024-02-17T12:57:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.798+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1902094551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2014d60 msgr2=0x5556a20151e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.798+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1902094551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2014d60 0x5556a20151e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400b760 tx=0x7fd6a400bc20 comp rx=0 tx=0).stop 2024-02-17T12:57:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.798+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1902094551 shutdown_connections 2024-02-17T12:57:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.798+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1902094551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a201e2e0 0x5556a20206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.798+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1902094551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2014d60 0x5556a20151e0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.798+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1902094551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a2015ad0 0x5556a2014820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.798+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1902094551 >> 172.21.15.18:0/1902094551 conn(0x5556a1fdc550 msgr2=0x5556a200fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.798+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1902094551 shutdown_connections 2024-02-17T12:57:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.798+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1902094551 wait complete. 2024-02-17T12:57:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6bb81e740 1 -- start start 2024-02-17T12:57:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6bb81e740 1 Processor -- start 2024-02-17T12:57:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6bb81e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a2014d60 0x5556a201b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6bb81e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a201b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6bb81e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a201e2e0 0x5556a201bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6bb81e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a1f21bf0 con 0x5556a2015ad0 2024-02-17T12:57:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6bb81e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a1c50250 con 0x5556a2014d60 2024-02-17T12:57:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6bb81e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a1f02200 con 0x5556a201e2e0 2024-02-17T12:57:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6bb81e740 10 client.0 _ll_register_callbacks cb 0x5556a1f70d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a201b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a201b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44370/0 (socket says 172.21.15.18:44370) 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6b259c640 1 -- 172.21.15.18:0/1689975349 learned_addr learned my addr 172.21.15.18:0/1689975349 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6abfff640 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a2014d60 0x5556a201b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.799+0000 7fd6abfff640 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a201e2e0 0x5556a201bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd6b259c640 1 -- 172.21.15.18:0/1689975349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a201e2e0 msgr2=0x5556a201bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd6b259c640 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a201e2e0 0x5556a201bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd6b259c640 1 -- 172.21.15.18:0/1689975349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a2014d60 msgr2=0x5556a201b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd6b259c640 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a2014d60 0x5556a201b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd6b259c640 1 -- 172.21.15.18:0/1689975349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a4004370 con 0x5556a2015ad0 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd6b259c640 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a201b890 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40039c0 tx=0x7fd6a4003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40db8e15192d1a32 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd699ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd699ffb640 1 -- 172.21.15.18:0/1689975349 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4005150 con 0x5556a2015ad0 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd699ffb640 1 -- 172.21.15.18:0/1689975349 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd6a40052b0 con 0x5556a2015ad0 2024-02-17T12:57:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1689975349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556a1c7c600 con 0x5556a2015ad0 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd699ffb640 1 -- 172.21.15.18:0/1689975349 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6a4004960 con 0x5556a2015ad0 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd6bb81e740 10 client.18318 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1689975349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5556a1f7ad80 con 0x5556a2015ad0 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd699ffb640 1 -- 172.21.15.18:0/1689975349 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a4030020 con 0x5556a2015ad0 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd699ffb640 1 -- 172.21.15.18:0/1689975349 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd6a4019420 con 0x5556a2015ad0 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.800+0000 7fd6bb81e740 10 client.18318 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.801+0000 7fd6bb81e740 10 client.18318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.801+0000 7fd6bb81e740 20 client.18318 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.801+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5556a202e540 0x5556a2030960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.801+0000 7fd6bb81e740 4 client.18318 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.801+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1689975349 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556a1ccd1b0 con 0x5556a202e540 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.801+0000 7fd6abfff640 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5556a202e540 0x5556a2030960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.801+0000 7fd6abfff640 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5556a202e540 0x5556a2030960 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d36d5c969401305d server_cookie=150f7f08047c2791 in_seq=0 out_seq=0 2024-02-17T12:57:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.801+0000 7fd699ffb640 10 client.18318 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd699ffb640 1 -- 172.21.15.18:0/1689975349 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fd69c04ea90 con 0x5556a202e540 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd699ffb640 10 client.18318 handle_command_reply: tid=0 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd6bb81e740 1 client.18318 shutdown 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1689975349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 msgr2=0x5556a201b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a201b890 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40039c0 tx=0x7fd6a4003e00 comp rx=0 tx=0).stop 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1689975349 shutdown_connections 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5556a202e540 0x5556a2030960 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a201e2e0 0x5556a201bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd699ffb640 0 client.18318 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a201b890 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1689975349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a2014d60 0x5556a201b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1689975349 >> 172.21.15.18:0/1689975349 conn(0x5556a1fdc550 msgr2=0x5556a20117a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1689975349 shutdown_connections 2024-02-17T12:57:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1689975349 wait complete. 2024-02-17T12:57:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.802+0000 7fd6bb81e740 20 client.18318 trim_cache size 0 max 16384 2024-02-17T12:57:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.815+0000 7fd6bb81e740 1 Processor -- start 2024-02-17T12:57:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.815+0000 7fd6bb81e740 1 -- start start 2024-02-17T12:57:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6bb81e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a2014d60 0x5556a201da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6bb81e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a201bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6bb81e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a201e2e0 0x5556a20185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6bb81e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a2014030 con 0x5556a2015ad0 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6bb81e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a20198a0 con 0x5556a201e2e0 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6bb81e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a201c130 con 0x5556a2014d60 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a201bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a201bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44384/0 (socket says 172.21.15.18:44384) 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6b259c640 1 -- 172.21.15.18:0/1179348181 learned_addr learned my addr 172.21.15.18:0/1179348181 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6abfff640 1 --2- 172.21.15.18:0/1179348181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a2014d60 0x5556a201da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6b259c640 1 -- 172.21.15.18:0/1179348181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a2014d60 msgr2=0x5556a201da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6b259c640 1 --2- 172.21.15.18:0/1179348181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a2014d60 0x5556a201da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6b259c640 1 -- 172.21.15.18:0/1179348181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a201e2e0 msgr2=0x5556a20185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6b259c640 1 --2- 172.21.15.18:0/1179348181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a201e2e0 0x5556a20185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6b259c640 1 -- 172.21.15.18:0/1179348181 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556a1f177b0 con 0x5556a2015ad0 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd6b259c640 1 --2- 172.21.15.18:0/1179348181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a201bbf0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400b5f0 tx=0x7fd6a400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4944183086112145 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd69bfff640 1 -- 172.21.15.18:0/1179348181 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4004210 con 0x5556a2015ad0 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd69bfff640 1 -- 172.21.15.18:0/1179348181 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a4030070 con 0x5556a2015ad0 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.816+0000 7fd69bfff640 1 -- 172.21.15.18:0/1179348181 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4004800 con 0x5556a2015ad0 2024-02-17T12:57:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.817+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1179348181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 msgr2=0x5556a201bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.817+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1179348181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a201bbf0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400b5f0 tx=0x7fd6a400b4a0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.817+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1179348181 shutdown_connections 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.817+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1179348181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a201e2e0 0x5556a20185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.817+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1179348181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a201bbf0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.817+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/1179348181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a2014d60 0x5556a201da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.817+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1179348181 >> 172.21.15.18:0/1179348181 conn(0x5556a1fdc550 msgr2=0x5556a2019c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.817+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1179348181 shutdown_connections 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.817+0000 7fd6bb81e740 1 -- 172.21.15.18:0/1179348181 wait complete. 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.817+0000 7fd6bb81e740 1 -- start start 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.817+0000 7fd6bb81e740 1 Processor -- start 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6bb81e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a2014d60 0x5556a2023c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6bb81e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a2024190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6bb81e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a201e2e0 0x5556a20246d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6bb81e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a2084a90 con 0x5556a2015ad0 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6bb81e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a2084bd0 con 0x5556a201e2e0 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6bb81e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a2084d90 con 0x5556a2014d60 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6bb81e740 10 client.0 _ll_register_callbacks cb 0x7fd6a4004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a2024190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a2024190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44390/0 (socket says 172.21.15.18:44390) 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6b259c640 1 -- 172.21.15.18:0/2185741935 learned_addr learned my addr 172.21.15.18:0/2185741935 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6abfff640 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a2014d60 0x5556a2023c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6abfff640 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a201e2e0 0x5556a20246d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6b259c640 1 -- 172.21.15.18:0/2185741935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a2014d60 msgr2=0x5556a2023c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6b259c640 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a2014d60 0x5556a2023c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6b259c640 1 -- 172.21.15.18:0/2185741935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a201e2e0 msgr2=0x5556a20246d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6b259c640 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a201e2e0 0x5556a20246d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6b259c640 1 -- 172.21.15.18:0/2185741935 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556a1f7ad80 con 0x5556a2015ad0 2024-02-17T12:57:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6abfff640 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a201e2e0 0x5556a20246d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6b259c640 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a2024190 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40059f0 tx=0x7fd6a4019d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6107ba27807fb1b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd699ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd699ffb640 1 -- 172.21.15.18:0/2185741935 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4004a10 con 0x5556a2015ad0 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd699ffb640 1 -- 172.21.15.18:0/2185741935 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd6a4000e40 con 0x5556a2015ad0 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.818+0000 7fd6bb81e740 1 -- 172.21.15.18:0/2185741935 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6a400be70 con 0x5556a2015ad0 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.819+0000 7fd699ffb640 1 -- 172.21.15.18:0/2185741935 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6a4004b70 con 0x5556a2015ad0 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.819+0000 7fd6bb81e740 10 client.18324 fetch_fsmap learned FSMap version 43 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.819+0000 7fd6bb81e740 1 -- 172.21.15.18:0/2185741935 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5556a1f3d540 con 0x5556a2015ad0 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.819+0000 7fd699ffb640 1 -- 172.21.15.18:0/2185741935 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a400be70 con 0x5556a2015ad0 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.819+0000 7fd699ffb640 1 -- 172.21.15.18:0/2185741935 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd6a4033070 con 0x5556a2015ad0 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.819+0000 7fd6bb81e740 10 client.18324 fetch_fsmap finished waiting for FSMap version 43 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.819+0000 7fd6bb81e740 10 client.18324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.819+0000 7fd6bb81e740 20 client.18324 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.819+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5556a20861a0 0x5556a2088590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.819+0000 7fd6bb81e740 4 client.18324 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:57:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.819+0000 7fd6bb81e740 1 -- 172.21.15.18:0/2185741935 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5556a1cb9550 con 0x5556a20861a0 2024-02-17T12:57:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.819+0000 7fd6abfff640 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5556a20861a0 0x5556a2088590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.820+0000 7fd699ffb640 10 client.18324 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.820+0000 7fd6abfff640 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5556a20861a0 0x5556a2088590 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3d033dee4c588c server_cookie=5279bc10a26e2977 in_seq=0 out_seq=0 2024-02-17T12:57:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.863+0000 7fd699ffb640 1 -- 172.21.15.18:0/2185741935 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5556a1cb9550 con 0x5556a20861a0 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.863+0000 7fd699ffb640 10 client.18324 handle_command_reply: tid=0 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.863+0000 7fd6bb81e740 1 client.18324 shutdown 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6bb81e740 1 -- 172.21.15.18:0/2185741935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 msgr2=0x5556a2024190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a2024190 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40059f0 tx=0x7fd6a4019d10 comp rx=0 tx=0).stop 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6bb81e740 1 -- 172.21.15.18:0/2185741935 shutdown_connections 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5556a20861a0 0x5556a2088590 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556a201e2e0 0x5556a20246d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5556a2015ad0 0x5556a2024190 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6bb81e740 1 --2- 172.21.15.18:0/2185741935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5556a2014d60 0x5556a2023c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6bb81e740 1 -- 172.21.15.18:0/2185741935 >> 172.21.15.18:0/2185741935 conn(0x5556a1fdc550 msgr2=0x5556a200e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd699ffb640 0 client.18324 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6bb81e740 1 -- 172.21.15.18:0/2185741935 shutdown_connections 2024-02-17T12:57:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6bb81e740 1 -- 172.21.15.18:0/2185741935 wait complete. 2024-02-17T12:57:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6bb81e740 20 client.18324 trim_cache size 0 max 16384 2024-02-17T12:57:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:57:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:57:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:57:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:57:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/1572127658 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd6a00a2300 msgr2=0x7fd6a00a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6b8c4e640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd6a00a2300 0x7fd6a00a47c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0cb9a0 tx=0x7fd6ac009270 comp rx=0 tx=0).stop 2024-02-17T12:57:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/1572127658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4073f00 msgr2=0x7fd6b41a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6b8c4e640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4073f00 0x7fd6b41a5980 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00eaa0 tx=0x7fd6ac00ef60 comp rx=0 tx=0).stop 2024-02-17T12:57:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/1572127658 shutdown_connections 2024-02-17T12:57:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6b8c4e640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd6a00a2300 0x7fd6a00a47c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6b8c4e640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4073f00 0x7fd6b41a5980 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6b8c4e640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6b4106490 0x7fd6b40739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6b8c4e640 1 --2- 172.21.15.18:0/1572127658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b41013c0 0x7fd6b4073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.864+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/1572127658 >> 172.21.15.18:0/1572127658 conn(0x7fd6b40fcc60 msgr2=0x7fd6b40fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.865+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/1572127658 shutdown_connections 2024-02-17T12:57:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:27.865+0000 7fd6b8c4e640 1 -- 172.21.15.18:0/1572127658 wait complete. 2024-02-17T12:57:28.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:27 smithi119 ceph-mon[36188]: pgmap v235: 97 pgs: 97 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-02-17T12:57:28.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:27 smithi146 ceph-mon[36454]: pgmap v235: 97 pgs: 97 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-02-17T12:57:28.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:27 smithi018 ceph-mon[32625]: pgmap v235: 97 pgs: 97 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-02-17T12:57:28.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 396563 5.54 MB/sec execute 283 sec latency 2101.832 ms 2024-02-17T12:57:29.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 398129 5.55 MB/sec execute 284 sec latency 20.159 ms 2024-02-17T12:57:30.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:29 smithi119 ceph-mon[36188]: pgmap v236: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-02-17T12:57:30.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:29 smithi146 ceph-mon[36454]: pgmap v236: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-02-17T12:57:30.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:29 smithi018 ceph-mon[32625]: pgmap v236: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-02-17T12:57:30.283 INFO:tasks.workunit.client.0.smithi018.stdout: 1 399001 5.54 MB/sec execute 285 sec latency 564.660 ms 2024-02-17T12:57:31.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 399001 5.52 MB/sec execute 286 sec latency 1564.705 ms 2024-02-17T12:57:32.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:31 smithi119 ceph-mon[36188]: pgmap v237: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-02-17T12:57:32.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:31 smithi146 ceph-mon[36454]: pgmap v237: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-02-17T12:57:32.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:31 smithi018 ceph-mon[32625]: pgmap v237: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-02-17T12:57:32.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 399001 5.50 MB/sec execute 287 sec latency 2564.797 ms 2024-02-17T12:57:33.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 399001 5.49 MB/sec execute 288 sec latency 3564.886 ms 2024-02-17T12:57:34.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:33 smithi119 ceph-mon[36188]: pgmap v238: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-02-17T12:57:34.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:33 smithi146 ceph-mon[36454]: pgmap v238: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-02-17T12:57:34.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:33 smithi018 ceph-mon[32625]: pgmap v238: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-02-17T12:57:34.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 401051 5.51 MB/sec execute 289 sec latency 3756.048 ms 2024-02-17T12:57:35.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:34 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:57:35.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:34 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:57:35.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:34 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:57:35.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 402430 5.51 MB/sec execute 290 sec latency 17.017 ms 2024-02-17T12:57:36.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:35 smithi119 ceph-mon[36188]: pgmap v239: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-02-17T12:57:36.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:35 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:57:36.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:35 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:57:36.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:35 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:57:36.100 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:35 smithi146 ceph-mon[36454]: pgmap v239: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-02-17T12:57:36.101 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:35 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:57:36.101 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:35 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:57:36.101 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:35 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:57:36.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:35 smithi018 ceph-mon[32625]: pgmap v239: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-02-17T12:57:36.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:35 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:57:36.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:35 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:57:36.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:35 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:57:36.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 402551 5.49 MB/sec execute 291 sec latency 921.490 ms 2024-02-17T12:57:37.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:36 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:57:37.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:36 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:57:37.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:36 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:57:37.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:36 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:57:37.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:36 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:57:37.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:36 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:57:37.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 402551 5.48 MB/sec execute 292 sec latency 1921.535 ms 2024-02-17T12:57:38.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:37 smithi119 ceph-mon[36188]: pgmap v240: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-02-17T12:57:38.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:37 smithi146 ceph-mon[36454]: pgmap v240: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-02-17T12:57:38.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:37 smithi018 ceph-mon[32625]: pgmap v240: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-02-17T12:57:38.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 402551 5.46 MB/sec execute 293 sec latency 2921.584 ms 2024-02-17T12:57:39.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 403466 5.45 MB/sec execute 294 sec latency 3526.592 ms 2024-02-17T12:57:40.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:39 smithi146 ceph-mon[36454]: pgmap v241: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 189 op/s 2024-02-17T12:57:40.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:39 smithi018 ceph-mon[32625]: pgmap v241: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 189 op/s 2024-02-17T12:57:40.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 405145 5.47 MB/sec execute 295 sec latency 23.317 ms 2024-02-17T12:57:40.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:39 smithi119 ceph-mon[36188]: pgmap v241: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 189 op/s 2024-02-17T12:57:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.953+0000 7f96e798a640 1 -- 172.21.15.18:0/4000680141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96e0105000 msgr2=0x7f96e01053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.953+0000 7f96e798a640 1 --2- 172.21.15.18:0/4000680141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96e0105000 0x7f96e01053e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009900 tx=0x7f96dc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.953+0000 7f96e798a640 1 -- 172.21.15.18:0/4000680141 shutdown_connections 2024-02-17T12:57:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.953+0000 7f96e798a640 1 --2- 172.21.15.18:0/4000680141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96e0100c70 0x7f96e010ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.953+0000 7f96e798a640 1 --2- 172.21.15.18:0/4000680141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96e01002d0 0x7f96e0100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.953+0000 7f96e798a640 1 --2- 172.21.15.18:0/4000680141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96e0105000 0x7f96e01053e0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.953+0000 7f96e798a640 1 -- 172.21.15.18:0/4000680141 >> 172.21.15.18:0/4000680141 conn(0x7f96e0076e50 msgr2=0x7f96e0077260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.953+0000 7f96e798a640 1 -- 172.21.15.18:0/4000680141 shutdown_connections 2024-02-17T12:57:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.953+0000 7f96e798a640 1 -- 172.21.15.18:0/4000680141 wait complete. 2024-02-17T12:57:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.953+0000 7f96e798a640 1 Processor -- start 2024-02-17T12:57:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.953+0000 7f96e798a640 1 -- start start 2024-02-17T12:57:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.953+0000 7f96e798a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96e01002d0 0x7f96e01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e798a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96e0100c70 0x7f96e0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e798a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96e0105000 0x7f96e01a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e798a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e0199390 con 0x7f96e0100c70 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e798a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e01a3b90 con 0x7f96e0105000 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e798a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e01a3cd0 con 0x7f96e01002d0 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e4f25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96e0100c70 0x7f96e0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e4f25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96e0100c70 0x7f96e0198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46752/0 (socket says 172.21.15.18:46752) 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e4f25640 1 -- 172.21.15.18:0/3447776687 learned_addr learned my addr 172.21.15.18:0/3447776687 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e5f27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96e0105000 0x7f96e01a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e5726640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96e01002d0 0x7f96e01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e4f25640 1 -- 172.21.15.18:0/3447776687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96e01002d0 msgr2=0x7f96e01985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e4f25640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96e01002d0 0x7f96e01985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e4f25640 1 -- 172.21.15.18:0/3447776687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96e0105000 msgr2=0x7f96e01a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e4f25640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96e0105000 0x7f96e01a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e4f25640 1 -- 172.21.15.18:0/3447776687 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc009560 con 0x7f96e0100c70 2024-02-17T12:57:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e5f27640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96e0105000 0x7f96e01a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e4f25640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96e0100c70 0x7f96e0198b10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f96d000c950 tx=0x7f96d000ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96d67fc640 1 -- 172.21.15.18:0/3447776687 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0007c10 con 0x7f96e0100c70 2024-02-17T12:57:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96d67fc640 1 -- 172.21.15.18:0/3447776687 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96d0007d70 con 0x7f96e0100c70 2024-02-17T12:57:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e798a640 1 -- 172.21.15.18:0/3447776687 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e01a3f30 con 0x7f96e0100c70 2024-02-17T12:57:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96d67fc640 1 -- 172.21.15.18:0/3447776687 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0018800 con 0x7f96e0100c70 2024-02-17T12:57:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.954+0000 7f96e798a640 1 -- 172.21.15.18:0/3447776687 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e01a4400 con 0x7f96e0100c70 2024-02-17T12:57:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.956+0000 7f96d67fc640 1 -- 172.21.15.18:0/3447776687 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f96d000faa0 con 0x7f96e0100c70 2024-02-17T12:57:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96ea633740 1 Processor -- start 2024-02-17T12:57:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96ea633740 1 -- start start 2024-02-17T12:57:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96ea633740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d5cad0 0x55b678d5b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96ea633740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 0x55b678d5c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96ea633740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 0x55b678d6bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96ea633740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b678cca9f0 con 0x55b678d696f0 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96ea633740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b678cc7f20 con 0x55b678d5cad0 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96ea633740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b678cc3700 con 0x55b678d5bd60 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96d67fc640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96cc0a2350 0x7f96cc0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96e4f25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 0x55b678d6bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96e4f25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 0x55b678d6bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46756/0 (socket says 172.21.15.18:46756) 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96e4f25640 1 -- 172.21.15.18:0/975479726 learned_addr learned my addr 172.21.15.18:0/975479726 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96e5f27640 1 --2- 172.21.15.18:0/975479726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d5cad0 0x55b678d5b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96e5726640 1 --2- 172.21.15.18:0/975479726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 0x55b678d5c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96e5f27640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96cc0a2350 0x7f96cc0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.957+0000 7f96d67fc640 1 -- 172.21.15.18:0/3447776687 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f96d00ce080 con 0x7f96e0100c70 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.958+0000 7f96e5f27640 1 -- 172.21.15.18:0/975479726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 msgr2=0x55b678d5c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.958+0000 7f96e5f27640 1 --2- 172.21.15.18:0/975479726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 0x55b678d5c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.958+0000 7f96e5f27640 1 -- 172.21.15.18:0/975479726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 msgr2=0x55b678d6bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.958+0000 7f96e5f27640 1 --2- 172.21.15.18:0/975479726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 0x55b678d6bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.958+0000 7f96e5f27640 1 -- 172.21.15.18:0/975479726 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b678c597b0 con 0x55b678d5cad0 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.958+0000 7f96e4f25640 1 --2- 172.21.15.18:0/975479726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 0x55b678d6bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.958+0000 7f96e5f27640 1 --2- 172.21.15.18:0/975479726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d5cad0 0x55b678d5b820 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f96d800f440 tx=0x7f96d800f900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11aa0ca81fa1f455 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.958+0000 7f96e5f27640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96cc0a2350 0x7f96cc0a4810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f96d800fe00 tx=0x7f96d801e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.958+0000 7f96cb7fe640 1 -- 172.21.15.18:0/975479726 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d800ee20 con 0x55b678d5cad0 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.958+0000 7f96cb7fe640 1 -- 172.21.15.18:0/975479726 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96d80114e0 con 0x55b678d5cad0 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.958+0000 7f96cb7fe640 1 -- 172.21.15.18:0/975479726 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8026e70 con 0x55b678d5cad0 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.959+0000 7f96ea633740 1 -- 172.21.15.18:0/975479726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d5cad0 msgr2=0x55b678d5b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.959+0000 7f96ea633740 1 --2- 172.21.15.18:0/975479726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d5cad0 0x55b678d5b820 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f96d800f440 tx=0x7f96d800f900 comp rx=0 tx=0).stop 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.959+0000 7f96ea633740 1 -- 172.21.15.18:0/975479726 shutdown_connections 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.959+0000 7f96ea633740 1 --2- 172.21.15.18:0/975479726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 0x55b678d6bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.959+0000 7f96ea633740 1 --2- 172.21.15.18:0/975479726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 0x55b678d5c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.959+0000 7f96ea633740 1 --2- 172.21.15.18:0/975479726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d5cad0 0x55b678d5b820 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.959+0000 7f96ea633740 1 -- 172.21.15.18:0/975479726 >> 172.21.15.18:0/975479726 conn(0x55b678d23550 msgr2=0x55b678d56a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.959+0000 7f96ea633740 1 -- 172.21.15.18:0/975479726 shutdown_connections 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.959+0000 7f96ea633740 1 -- 172.21.15.18:0/975479726 wait complete. 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.959+0000 7f96ea633740 1 -- start start 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.959+0000 7f96ea633740 1 Processor -- start 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96ea633740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 0x55b678d66730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96ea633740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 0x55b678d66c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96ea633740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d696f0 0x55b678d671b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96ea633740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b678c68bf0 con 0x55b678d5cad0 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96ea633740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b678997250 con 0x55b678d696f0 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96ea633740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b678c49200 con 0x55b678d5bd60 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96ea633740 10 client.0 _ll_register_callbacks cb 0x55b678cb7d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96e4f25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 0x55b678d66c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96e4f25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 0x55b678d66c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46764/0 (socket says 172.21.15.18:46764) 2024-02-17T12:57:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96e4f25640 1 -- 172.21.15.18:0/2464757369 learned_addr learned my addr 172.21.15.18:0/2464757369 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96e5f27640 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d696f0 0x55b678d671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96e5726640 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 0x55b678d66730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96e4f25640 1 -- 172.21.15.18:0/2464757369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 msgr2=0x55b678d66730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96e4f25640 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 0x55b678d66730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96e4f25640 1 -- 172.21.15.18:0/2464757369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d696f0 msgr2=0x55b678d671b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96e4f25640 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d696f0 0x55b678d671b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96e4f25640 1 -- 172.21.15.18:0/2464757369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d8019040 con 0x55b678d5cad0 2024-02-17T12:57:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96e5f27640 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d696f0 0x55b678d671b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96e4f25640 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 0x55b678d66c70 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f96d0083fd0 tx=0x7f96d000fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39141de7cb54f52 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96c8ff9640 1 -- 172.21.15.18:0/2464757369 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00d02f0 con 0x55b678d5cad0 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96c8ff9640 1 -- 172.21.15.18:0/2464757369 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f96d009aa90 con 0x55b678d5cad0 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96ea633740 1 -- 172.21.15.18:0/2464757369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6789c3600 con 0x55b678d5cad0 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96c8ff9640 1 -- 172.21.15.18:0/2464757369 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96d009ebe0 con 0x55b678d5cad0 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96ea633740 10 client.18336 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.960+0000 7f96ea633740 1 -- 172.21.15.18:0/2464757369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55b678cc1d80 con 0x55b678d5cad0 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.961+0000 7f96c8ff9640 1 -- 172.21.15.18:0/2464757369 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b6789c3600 con 0x55b678d5cad0 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.961+0000 7f96c8ff9640 1 -- 172.21.15.18:0/2464757369 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f96d0016020 con 0x55b678d5cad0 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.961+0000 7f96ea633740 10 client.18336 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.961+0000 7f96ea633740 10 client.18336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.961+0000 7f96ea633740 20 client.18336 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.961+0000 7f96ea633740 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b678d79a80 0x55b678d7bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.961+0000 7f96ea633740 4 client.18336 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.961+0000 7f96ea633740 1 -- 172.21.15.18:0/2464757369 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b678a141b0 con 0x55b678d79a80 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.961+0000 7f96e5726640 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b678d79a80 0x55b678d7bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.961+0000 7f96c8ff9640 10 client.18336 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.961+0000 7f96e5726640 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b678d79a80 0x55b678d7bea0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c4712648b1da1b server_cookie=931800224d0413e6 in_seq=0 out_seq=0 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96c8ff9640 1 -- 172.21.15.18:0/2464757369 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b678a141b0 con 0x55b678d79a80 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96c8ff9640 10 client.18336 handle_command_reply: tid=0 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96ea633740 1 client.18336 shutdown 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96ea633740 1 -- 172.21.15.18:0/2464757369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 msgr2=0x55b678d66c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96ea633740 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 0x55b678d66c70 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f96d0083fd0 tx=0x7f96d000fea0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96ea633740 1 -- 172.21.15.18:0/2464757369 shutdown_connections 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96ea633740 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b678d79a80 0x55b678d7bea0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96ea633740 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d696f0 0x55b678d671b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96c8ff9640 0 client.18336 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96ea633740 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 0x55b678d66c70 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96ea633740 1 --2- 172.21.15.18:0/2464757369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 0x55b678d66730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96ea633740 1 -- 172.21.15.18:0/2464757369 >> 172.21.15.18:0/2464757369 conn(0x55b678d23550 msgr2=0x55b678d587a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96ea633740 1 -- 172.21.15.18:0/2464757369 shutdown_connections 2024-02-17T12:57:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96ea633740 1 -- 172.21.15.18:0/2464757369 wait complete. 2024-02-17T12:57:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.962+0000 7f96ea633740 20 client.18336 trim_cache size 0 max 16384 2024-02-17T12:57:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.975+0000 7f96ea633740 1 Processor -- start 2024-02-17T12:57:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.975+0000 7f96ea633740 1 -- start start 2024-02-17T12:57:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96ea633740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 0x55b678d68e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96ea633740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 0x55b678d66fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96ea633740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d696f0 0x55b678d63990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96ea633740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6789c3600 con 0x55b678d5cad0 2024-02-17T12:57:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96ea633740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b678d62360 con 0x55b678d696f0 2024-02-17T12:57:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96ea633740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b678d57a50 con 0x55b678d5bd60 2024-02-17T12:57:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96e4f25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 0x55b678d66fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96e4f25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 0x55b678d66fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46770/0 (socket says 172.21.15.18:46770) 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96e4f25640 1 -- 172.21.15.18:0/3530561733 learned_addr learned my addr 172.21.15.18:0/3530561733 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96e5f27640 1 --2- 172.21.15.18:0/3530561733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d696f0 0x55b678d63990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96e5726640 1 --2- 172.21.15.18:0/3530561733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 0x55b678d68e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96e4f25640 1 -- 172.21.15.18:0/3530561733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 msgr2=0x55b678d68e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96e4f25640 1 --2- 172.21.15.18:0/3530561733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 0x55b678d68e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96e4f25640 1 -- 172.21.15.18:0/3530561733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d696f0 msgr2=0x55b678d63990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96e4f25640 1 --2- 172.21.15.18:0/3530561733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d696f0 0x55b678d63990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96e4f25640 1 -- 172.21.15.18:0/3530561733 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b678c5e7b0 con 0x55b678d5cad0 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96e5f27640 1 --2- 172.21.15.18:0/3530561733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d696f0 0x55b678d63990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96e4f25640 1 --2- 172.21.15.18:0/3530561733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 0x55b678d66fd0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f96d0098d40 tx=0x7f96d000fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b683e5b8423f5d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96caffd640 1 -- 172.21.15.18:0/3530561733 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0096d30 con 0x55b678d5cad0 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96caffd640 1 -- 172.21.15.18:0/3530561733 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96d009a3f0 con 0x55b678d5cad0 2024-02-17T12:57:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.976+0000 7f96caffd640 1 -- 172.21.15.18:0/3530561733 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d009a140 con 0x55b678d5cad0 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.977+0000 7f96ea633740 1 -- 172.21.15.18:0/3530561733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 msgr2=0x55b678d66fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.977+0000 7f96ea633740 1 --2- 172.21.15.18:0/3530561733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 0x55b678d66fd0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f96d0098d40 tx=0x7f96d000fda0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.977+0000 7f96ea633740 1 -- 172.21.15.18:0/3530561733 shutdown_connections 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.977+0000 7f96ea633740 1 --2- 172.21.15.18:0/3530561733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d696f0 0x55b678d63990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.977+0000 7f96ea633740 1 --2- 172.21.15.18:0/3530561733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d5cad0 0x55b678d66fd0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.977+0000 7f96ea633740 1 --2- 172.21.15.18:0/3530561733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5bd60 0x55b678d68e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.977+0000 7f96ea633740 1 -- 172.21.15.18:0/3530561733 >> 172.21.15.18:0/3530561733 conn(0x55b678d23550 msgr2=0x55b678d65000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.977+0000 7f96ea633740 1 -- 172.21.15.18:0/3530561733 shutdown_connections 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.977+0000 7f96ea633740 1 -- 172.21.15.18:0/3530561733 wait complete. 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.977+0000 7f96ea633740 1 -- start start 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.977+0000 7f96ea633740 1 Processor -- start 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96ea633740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d5bd60 0x55b678d708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96ea633740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5cad0 0x55b678d6ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96ea633740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 0x55b678d6f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96ea633740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b678d6f910 con 0x55b678d696f0 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96ea633740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b678d6fa80 con 0x55b678d5bd60 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96ea633740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b678dcff50 con 0x55b678d5cad0 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96ea633740 10 client.0 _ll_register_callbacks cb 0x7f96d80116c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96e5f27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 0x55b678d6f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96e5f27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 0x55b678d6f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46778/0 (socket says 172.21.15.18:46778) 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96e5f27640 1 -- 172.21.15.18:0/1510185062 learned_addr learned my addr 172.21.15.18:0/1510185062 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96e4f25640 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5cad0 0x55b678d6ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96e5726640 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d5bd60 0x55b678d708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96e5f27640 1 -- 172.21.15.18:0/1510185062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5cad0 msgr2=0x55b678d6ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96e5f27640 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5cad0 0x55b678d6ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96e5f27640 1 -- 172.21.15.18:0/1510185062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d5bd60 msgr2=0x55b678d708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96e5f27640 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d5bd60 0x55b678d708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96e5f27640 1 -- 172.21.15.18:0/1510185062 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b678cc1d80 con 0x55b678d696f0 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96e5726640 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d5bd60 0x55b678d708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96e5f27640 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 0x55b678d6f210 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f96d8011690 tx=0x7f96d8009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae2d7c419ffbfc9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.978+0000 7f96c8ff9640 1 -- 172.21.15.18:0/1510185062 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8011250 con 0x55b678d696f0 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96c8ff9640 1 -- 172.21.15.18:0/1510185062 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f96d8026450 con 0x55b678d696f0 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96ea633740 1 -- 172.21.15.18:0/1510185062 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96d80114e0 con 0x55b678d696f0 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96c8ff9640 1 -- 172.21.15.18:0/1510185062 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96d8026610 con 0x55b678d696f0 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96ea633740 10 client.18342 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96ea633740 1 -- 172.21.15.18:0/1510185062 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55b678c84540 con 0x55b678d696f0 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96c8ff9640 1 -- 172.21.15.18:0/1510185062 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96d80114e0 con 0x55b678d696f0 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96c8ff9640 1 -- 172.21.15.18:0/1510185062 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f96d8024070 con 0x55b678d696f0 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96ea633740 10 client.18342 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96ea633740 10 client.18342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96ea633740 20 client.18342 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96ea633740 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b678dd1790 0x55b678dd3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96ea633740 4 client.18342 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:57:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96ea633740 1 -- 172.21.15.18:0/1510185062 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b678a00550 con 0x55b678dd1790 2024-02-17T12:57:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.979+0000 7f96e5726640 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b678dd1790 0x55b678dd3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.980+0000 7f96e5726640 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b678dd1790 0x55b678dd3bb0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bacd9ac0b5a24b98 server_cookie=97feb723c75ac3dc in_seq=0 out_seq=0 2024-02-17T12:57:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:40.980+0000 7f96c8ff9640 10 client.18342 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96c8ff9640 1 -- 172.21.15.18:0/1510185062 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b678a00550 con 0x55b678dd1790 2024-02-17T12:57:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96c8ff9640 10 client.18342 handle_command_reply: tid=0 2024-02-17T12:57:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96ea633740 1 client.18342 shutdown 2024-02-17T12:57:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96ea633740 1 -- 172.21.15.18:0/1510185062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 msgr2=0x55b678d6f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96ea633740 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 0x55b678d6f210 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f96d8011690 tx=0x7f96d8009590 comp rx=0 tx=0).stop 2024-02-17T12:57:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96ea633740 1 -- 172.21.15.18:0/1510185062 shutdown_connections 2024-02-17T12:57:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96ea633740 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b678dd1790 0x55b678dd3bb0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96ea633740 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b678d696f0 0x55b678d6f210 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96ea633740 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b678d5cad0 0x55b678d6ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96ea633740 1 --2- 172.21.15.18:0/1510185062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b678d5bd60 0x55b678d708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96c8ff9640 0 client.18342 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96ea633740 1 -- 172.21.15.18:0/1510185062 >> 172.21.15.18:0/1510185062 conn(0x55b678d23550 msgr2=0x55b678d57d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96ea633740 1 -- 172.21.15.18:0/1510185062 shutdown_connections 2024-02-17T12:57:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.013+0000 7f96ea633740 1 -- 172.21.15.18:0/1510185062 wait complete. 2024-02-17T12:57:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96ea633740 20 client.18342 trim_cache size 0 max 16384 2024-02-17T12:57:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:57:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:57:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:57:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:57:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96e798a640 1 -- 172.21.15.18:0/3447776687 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96cc0a2350 msgr2=0x7f96cc0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96e798a640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96cc0a2350 0x7f96cc0a4810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f96d800fe00 tx=0x7f96d801e000 comp rx=0 tx=0).stop 2024-02-17T12:57:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96e798a640 1 -- 172.21.15.18:0/3447776687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96e0100c70 msgr2=0x7f96e0198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96e798a640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96e0100c70 0x7f96e0198b10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f96d000c950 tx=0x7f96d000ce10 comp rx=0 tx=0).stop 2024-02-17T12:57:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96e798a640 1 -- 172.21.15.18:0/3447776687 shutdown_connections 2024-02-17T12:57:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96e798a640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96cc0a2350 0x7f96cc0a4810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96e798a640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96e0105000 0x7f96e01a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96e798a640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96e0100c70 0x7f96e0198b10 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96e798a640 1 --2- 172.21.15.18:0/3447776687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96e01002d0 0x7f96e01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96e798a640 1 -- 172.21.15.18:0/3447776687 >> 172.21.15.18:0/3447776687 conn(0x7f96e0076e50 msgr2=0x7f96e00fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96e798a640 1 -- 172.21.15.18:0/3447776687 shutdown_connections 2024-02-17T12:57:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:41.014+0000 7f96e798a640 1 -- 172.21.15.18:0/3447776687 wait complete. 2024-02-17T12:57:41.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 406092 5.47 MB/sec execute 296 sec latency 510.729 ms 2024-02-17T12:57:42.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:41 smithi146 ceph-mon[36454]: pgmap v242: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 893 KiB/s wr, 98 op/s 2024-02-17T12:57:42.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:41 smithi018 ceph-mon[32625]: pgmap v242: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 893 KiB/s wr, 98 op/s 2024-02-17T12:57:42.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 406092 5.45 MB/sec execute 297 sec latency 1510.792 ms 2024-02-17T12:57:42.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:41 smithi119 ceph-mon[36188]: pgmap v242: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 893 KiB/s wr, 98 op/s 2024-02-17T12:57:43.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 406092 5.43 MB/sec execute 298 sec latency 2510.899 ms 2024-02-17T12:57:44.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:43 smithi146 ceph-mon[36454]: pgmap v243: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 893 KiB/s wr, 98 op/s 2024-02-17T12:57:44.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:43 smithi018 ceph-mon[32625]: pgmap v243: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 893 KiB/s wr, 98 op/s 2024-02-17T12:57:44.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 407007 5.42 MB/sec execute 299 sec latency 3115.226 ms 2024-02-17T12:57:44.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:43 smithi119 ceph-mon[36188]: pgmap v243: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 893 KiB/s wr, 98 op/s 2024-02-17T12:57:45.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 408671 5.44 MB/sec execute 300 sec latency 12.365 ms 2024-02-17T12:57:46.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:45 smithi146 ceph-mon[36454]: pgmap v244: 97 pgs: 97 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-02-17T12:57:46.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:45 smithi018 ceph-mon[32625]: pgmap v244: 97 pgs: 97 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-02-17T12:57:46.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 409641 5.44 MB/sec execute 301 sec latency 420.199 ms 2024-02-17T12:57:46.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:45 smithi119 ceph-mon[36188]: pgmap v244: 97 pgs: 97 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-02-17T12:57:47.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 409641 5.42 MB/sec execute 302 sec latency 1420.251 ms 2024-02-17T12:57:48.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:47 smithi146 ceph-mon[36454]: pgmap v245: 97 pgs: 97 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T12:57:48.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:47 smithi018 ceph-mon[32625]: pgmap v245: 97 pgs: 97 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T12:57:48.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 410715 5.42 MB/sec execute 303 sec latency 1986.480 ms 2024-02-17T12:57:48.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:47 smithi119 ceph-mon[36188]: pgmap v245: 97 pgs: 97 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T12:57:49.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 412214 5.43 MB/sec execute 304 sec latency 14.449 ms 2024-02-17T12:57:50.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:49 smithi146 ceph-mon[36454]: pgmap v246: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 196 op/s 2024-02-17T12:57:50.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:49 smithi018 ceph-mon[32625]: pgmap v246: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 196 op/s 2024-02-17T12:57:50.284 INFO:tasks.workunit.client.0.smithi018.stdout: 1 413183 5.43 MB/sec execute 305 sec latency 325.591 ms 2024-02-17T12:57:50.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:49 smithi119 ceph-mon[36188]: pgmap v246: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 196 op/s 2024-02-17T12:57:51.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 413183 5.41 MB/sec execute 306 sec latency 1325.704 ms 2024-02-17T12:57:51.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:57:51.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:57:51.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.881+0000 7f075cb8a640 1 -- 172.21.15.18:0/1867511127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0758106500 msgr2=0x7f075810e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.881+0000 7f075cb8a640 1 --2- 172.21.15.18:0/1867511127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0758106500 0x7f075810e060 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f0750009ca0 tx=0x7f075002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.881+0000 7f075cb8a640 1 -- 172.21.15.18:0/1867511127 shutdown_connections 2024-02-17T12:57:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.881+0000 7f075cb8a640 1 --2- 172.21.15.18:0/1867511127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0758106500 0x7f075810e060 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.881+0000 7f075cb8a640 1 --2- 172.21.15.18:0/1867511127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07581002a0 0x7f0758105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.881+0000 7f075cb8a640 1 --2- 172.21.15.18:0/1867511127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07580ff8f0 0x7f07580ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.881+0000 7f075cb8a640 1 -- 172.21.15.18:0/1867511127 >> 172.21.15.18:0/1867511127 conn(0x7f07580765b0 msgr2=0x7f07580fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.881+0000 7f075cb8a640 1 -- 172.21.15.18:0/1867511127 shutdown_connections 2024-02-17T12:57:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.881+0000 7f075cb8a640 1 -- 172.21.15.18:0/1867511127 wait complete. 2024-02-17T12:57:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.882+0000 7f075cb8a640 1 Processor -- start 2024-02-17T12:57:51.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.882+0000 7f075cb8a640 1 -- start start 2024-02-17T12:57:51.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.882+0000 7f075cb8a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07580ff8f0 0x7f07581977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.882+0000 7f075cb8a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0758106500 0x7f0758197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.882+0000 7f075cb8a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f075819bfe0 0x7f075819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.882+0000 7f075cb8a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075819c9d0 con 0x7f07580ff8f0 2024-02-17T12:57:51.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.882+0000 7f075cb8a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075819cb40 con 0x7f075819bfe0 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.882+0000 7f075cb8a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075819ccb0 con 0x7f0758106500 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.882+0000 7f075659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07580ff8f0 0x7f07581977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.882+0000 7f075659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07580ff8f0 0x7f07581977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35468/0 (socket says 172.21.15.18:35468) 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.882+0000 7f075659c640 1 -- 172.21.15.18:0/935249461 learned_addr learned my addr 172.21.15.18:0/935249461 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f0755d9b640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0758106500 0x7f0758197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f0756d9d640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f075819bfe0 0x7f075819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f075659c640 1 -- 172.21.15.18:0/935249461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0758106500 msgr2=0x7f0758197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f075659c640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0758106500 0x7f0758197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f075659c640 1 -- 172.21.15.18:0/935249461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f075819bfe0 msgr2=0x7f075819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f075659c640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f075819bfe0 0x7f075819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f075659c640 1 -- 172.21.15.18:0/935249461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0750009900 con 0x7f07580ff8f0 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f0756d9d640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f075819bfe0 0x7f075819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f075659c640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07580ff8f0 0x7f07581977c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f074800d770 tx=0x7f074800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f07477fe640 1 -- 172.21.15.18:0/935249461 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748009c30 con 0x7f07580ff8f0 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f07477fe640 1 -- 172.21.15.18:0/935249461 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f074800de80 con 0x7f07580ff8f0 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f075cb8a640 1 -- 172.21.15.18:0/935249461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07581a4290 con 0x7f07580ff8f0 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f07477fe640 1 -- 172.21.15.18:0/935249461 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748004e80 con 0x7f07580ff8f0 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.883+0000 7f075cb8a640 1 -- 172.21.15.18:0/935249461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07581a46f0 con 0x7f07580ff8f0 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.885+0000 7f07477fe640 1 -- 172.21.15.18:0/935249461 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0748005020 con 0x7f07580ff8f0 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.886+0000 7f075f836740 1 Processor -- start 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.886+0000 7f075f836740 1 -- start start 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.886+0000 7f07477fe640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07400a2230 0x7f07400a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.886+0000 7f075f836740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05daa7fa0 0x55a05daa8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.886+0000 7f075f836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05dab5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.886+0000 7f075f836740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05dab5bc0 0x55a05dab7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.886+0000 7f075f836740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05da17280 con 0x55a05daa7fa0 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.886+0000 7f075f836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05da147b0 con 0x55a05daa88c0 2024-02-17T12:57:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.886+0000 7f075f836740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05da0ff90 con 0x55a05dab5bc0 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.886+0000 7f0756d9d640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07400a2230 0x7f07400a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.886+0000 7f07477fe640 1 -- 172.21.15.18:0/935249461 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f07480cdb40 con 0x7f07580ff8f0 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f075659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05daa7fa0 0x55a05daa8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f075659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05daa7fa0 0x55a05daa8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35484/0 (socket says 172.21.15.18:35484) 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f075659c640 1 -- 172.21.15.18:0/1680275304 learned_addr learned my addr 172.21.15.18:0/1680275304 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f0755d9b640 1 --2- 172.21.15.18:0/1680275304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05dab5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f075659c640 1 -- 172.21.15.18:0/1680275304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05dab5bc0 msgr2=0x55a05dab7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f075659c640 1 --2- 172.21.15.18:0/1680275304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05dab5bc0 0x55a05dab7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f075659c640 1 -- 172.21.15.18:0/1680275304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 msgr2=0x55a05dab5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f075659c640 1 --2- 172.21.15.18:0/1680275304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05dab5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f075659c640 1 -- 172.21.15.18:0/1680275304 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a05d9ce570 con 0x55a05daa7fa0 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f0756d9d640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07400a2230 0x7f07400a46f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0750005e30 tx=0x7f0750032f50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f075659c640 1 --2- 172.21.15.18:0/1680275304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05daa7fa0 0x55a05daa8380 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f074809a480 tx=0x7f0748096190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840ef33152c6b22a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f0744ff9640 1 -- 172.21.15.18:0/1680275304 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07480196e0 con 0x55a05daa7fa0 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f0744ff9640 1 -- 172.21.15.18:0/1680275304 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f074801a360 con 0x55a05daa7fa0 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.887+0000 7f0744ff9640 1 -- 172.21.15.18:0/1680275304 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07480a2d00 con 0x55a05daa7fa0 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.888+0000 7f075f836740 1 -- 172.21.15.18:0/1680275304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05daa7fa0 msgr2=0x55a05daa8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.888+0000 7f075f836740 1 --2- 172.21.15.18:0/1680275304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05daa7fa0 0x55a05daa8380 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f074809a480 tx=0x7f0748096190 comp rx=0 tx=0).stop 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.888+0000 7f075f836740 1 -- 172.21.15.18:0/1680275304 shutdown_connections 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.888+0000 7f075f836740 1 --2- 172.21.15.18:0/1680275304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05dab5bc0 0x55a05dab7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.888+0000 7f075f836740 1 --2- 172.21.15.18:0/1680275304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05dab5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.888+0000 7f075f836740 1 --2- 172.21.15.18:0/1680275304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05daa7fa0 0x55a05daa8380 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.888+0000 7f075f836740 1 -- 172.21.15.18:0/1680275304 >> 172.21.15.18:0/1680275304 conn(0x55a05daa2990 msgr2=0x55a05daa2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.888+0000 7f075f836740 1 -- 172.21.15.18:0/1680275304 shutdown_connections 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.888+0000 7f075f836740 1 -- 172.21.15.18:0/1680275304 wait complete. 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.888+0000 7f075f836740 1 -- start start 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f075f836740 1 Processor -- start 2024-02-17T12:57:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f075f836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa7fa0 0x55a05dab55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f075f836740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa88c0 0x55a05dab2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f075f836740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dab2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f075f836740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05d6e3430 con 0x55a05dab5bc0 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f075f836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05d69fe00 con 0x55a05daa7fa0 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f075f836740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05d994fd0 con 0x55a05daa88c0 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f075f836740 10 client.0 _ll_register_callbacks cb 0x55a05d9eafd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f0756d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dab2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f0756d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dab2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35490/0 (socket says 172.21.15.18:35490) 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f0756d9d640 1 -- 172.21.15.18:0/1502534358 learned_addr learned my addr 172.21.15.18:0/1502534358 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f075659c640 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa7fa0 0x55a05dab55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f0755d9b640 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa88c0 0x55a05dab2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f0756d9d640 1 -- 172.21.15.18:0/1502534358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa88c0 msgr2=0x55a05dab2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f0756d9d640 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa88c0 0x55a05dab2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f0756d9d640 1 -- 172.21.15.18:0/1502534358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa7fa0 msgr2=0x55a05dab55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f0756d9d640 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa7fa0 0x55a05dab55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f0756d9d640 1 -- 172.21.15.18:0/1502534358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07581a46f0 con 0x55a05dab5bc0 2024-02-17T12:57:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f0755d9b640 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa88c0 0x55a05dab2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f0756d9d640 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dab2f70 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0750009ca0 tx=0x7f07500095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51172373c7d5f54f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.889+0000 7f073dffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f073dffb640 1 -- 172.21.15.18:0/1502534358 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0750034e40 con 0x55a05dab5bc0 2024-02-17T12:57:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f073dffb640 1 -- 172.21.15.18:0/1502534358 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0750003b80 con 0x55a05dab5bc0 2024-02-17T12:57:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f075f836740 1 -- 172.21.15.18:0/1502534358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a05d70f7e0 con 0x55a05dab5bc0 2024-02-17T12:57:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f073dffb640 1 -- 172.21.15.18:0/1502534358 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0750003d40 con 0x55a05dab5bc0 2024-02-17T12:57:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f075f836740 10 client.18354 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f075f836740 1 -- 172.21.15.18:0/1502534358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55a05da0e610 con 0x55a05dab5bc0 2024-02-17T12:57:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f073dffb640 1 -- 172.21.15.18:0/1502534358 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a05d70f7e0 con 0x55a05dab5bc0 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f073dffb640 1 -- 172.21.15.18:0/1502534358 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f0750040700 con 0x55a05dab5bc0 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f075f836740 10 client.18354 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f075f836740 10 client.18354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f075f836740 20 client.18354 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f075f836740 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a05dac5e30 0x55a05dac8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f075f836740 4 client.18354 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.890+0000 7f075f836740 1 -- 172.21.15.18:0/1502534358 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a05d74c720 con 0x55a05dac5e30 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.891+0000 7f075659c640 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a05dac5e30 0x55a05dac8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.891+0000 7f073dffb640 10 client.18354 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.891+0000 7f075659c640 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a05dac5e30 0x55a05dac8250 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34ee0da19dca4ec6 server_cookie=61d019444316c63b in_seq=0 out_seq=0 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.891+0000 7f073dffb640 1 -- 172.21.15.18:0/1502534358 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a05d74c720 con 0x55a05dac5e30 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.891+0000 7f073dffb640 10 client.18354 handle_command_reply: tid=0 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f075f836740 1 client.18354 shutdown 2024-02-17T12:57:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f075f836740 1 -- 172.21.15.18:0/1502534358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 msgr2=0x55a05dab2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:51.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f075f836740 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dab2f70 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0750009ca0 tx=0x7f07500095c0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f075f836740 1 -- 172.21.15.18:0/1502534358 shutdown_connections 2024-02-17T12:57:51.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f075f836740 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a05dac5e30 0x55a05dac8250 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f075f836740 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dab2f70 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f075f836740 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa88c0 0x55a05dab2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f075f836740 1 --2- 172.21.15.18:0/1502534358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa7fa0 0x55a05dab55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f073dffb640 0 client.18354 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f075f836740 1 -- 172.21.15.18:0/1502534358 >> 172.21.15.18:0/1502534358 conn(0x55a05daa2990 msgr2=0x55a05daa4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f075f836740 1 -- 172.21.15.18:0/1502534358 shutdown_connections 2024-02-17T12:57:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f075f836740 1 -- 172.21.15.18:0/1502534358 wait complete. 2024-02-17T12:57:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.892+0000 7f075f836740 20 client.18354 trim_cache size 0 max 16384 2024-02-17T12:57:51.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.907+0000 7f075f836740 1 Processor -- start 2024-02-17T12:57:51.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.907+0000 7f075f836740 1 -- start start 2024-02-17T12:57:51.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.907+0000 7f075f836740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa7fa0 0x55a05dab5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.907+0000 7f075f836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05dab55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.907+0000 7f075f836740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dabcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.907+0000 7f075f836740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05d70f810 con 0x55a05dab5bc0 2024-02-17T12:57:51.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.907+0000 7f075f836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05da9e120 con 0x55a05daa88c0 2024-02-17T12:57:51.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.907+0000 7f075f836740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05daa27f0 con 0x55a05daa7fa0 2024-02-17T12:57:51.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.907+0000 7f0756d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dabcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.907+0000 7f0756d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dabcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35504/0 (socket says 172.21.15.18:35504) 2024-02-17T12:57:51.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.907+0000 7f0756d9d640 1 -- 172.21.15.18:0/326558115 learned_addr learned my addr 172.21.15.18:0/326558115 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:51.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.908+0000 7f075659c640 1 --2- 172.21.15.18:0/326558115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa7fa0 0x55a05dab5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.908+0000 7f0755d9b640 1 --2- 172.21.15.18:0/326558115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05dab55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.908+0000 7f0756d9d640 1 -- 172.21.15.18:0/326558115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa7fa0 msgr2=0x55a05dab5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:51.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.908+0000 7f0756d9d640 1 --2- 172.21.15.18:0/326558115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa7fa0 0x55a05dab5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.908+0000 7f0756d9d640 1 -- 172.21.15.18:0/326558115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 msgr2=0x55a05dab55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:51.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.908+0000 7f0756d9d640 1 --2- 172.21.15.18:0/326558115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05dab55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.908+0000 7f0756d9d640 1 -- 172.21.15.18:0/326558115 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a05d9aaac0 con 0x55a05dab5bc0 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.908+0000 7f0755d9b640 1 --2- 172.21.15.18:0/326558115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05dab55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.908+0000 7f0756d9d640 1 --2- 172.21.15.18:0/326558115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dabcaa0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f0750032f20 tx=0x7f07500347e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccb97a1017196ba server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.908+0000 7f073ffff640 1 -- 172.21.15.18:0/326558115 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0750005c90 con 0x55a05dab5bc0 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.908+0000 7f073ffff640 1 -- 172.21.15.18:0/326558115 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0750032a80 con 0x55a05dab5bc0 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.908+0000 7f073ffff640 1 -- 172.21.15.18:0/326558115 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07500046b0 con 0x55a05dab5bc0 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 -- 172.21.15.18:0/326558115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 msgr2=0x55a05dabcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 --2- 172.21.15.18:0/326558115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dabcaa0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f0750032f20 tx=0x7f07500347e0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 -- 172.21.15.18:0/326558115 shutdown_connections 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 --2- 172.21.15.18:0/326558115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dabcaa0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 --2- 172.21.15.18:0/326558115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05dab55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 --2- 172.21.15.18:0/326558115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa7fa0 0x55a05dab5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 -- 172.21.15.18:0/326558115 >> 172.21.15.18:0/326558115 conn(0x55a05daa2990 msgr2=0x55a05dab1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 -- 172.21.15.18:0/326558115 shutdown_connections 2024-02-17T12:57:51.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 -- 172.21.15.18:0/326558115 wait complete. 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 -- start start 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 Processor -- start 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa7fa0 0x55a05daaf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05daafc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dab0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05db1c330 con 0x55a05dab5bc0 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05db1c470 con 0x55a05daa88c0 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05db1c600 con 0x55a05daa7fa0 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.909+0000 7f075f836740 10 client.0 _ll_register_callbacks cb 0x7f07480048e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f0756d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dab0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f0756d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dab0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35514/0 (socket says 172.21.15.18:35514) 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f0756d9d640 1 -- 172.21.15.18:0/309075504 learned_addr learned my addr 172.21.15.18:0/309075504 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f0755d9b640 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05daafc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f075659c640 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa7fa0 0x55a05daaf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f0756d9d640 1 -- 172.21.15.18:0/309075504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa7fa0 msgr2=0x55a05daaf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f0756d9d640 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa7fa0 0x55a05daaf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f0756d9d640 1 -- 172.21.15.18:0/309075504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 msgr2=0x55a05daafc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f0756d9d640 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05daafc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f0756d9d640 1 -- 172.21.15.18:0/309075504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a05da0e610 con 0x55a05dab5bc0 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f0756d9d640 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dab0150 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f07500348f0 tx=0x7f0750004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c45111583b75c8d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f073dffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f073dffb640 1 -- 172.21.15.18:0/309075504 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0750003d90 con 0x55a05dab5bc0 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f073dffb640 1 -- 172.21.15.18:0/309075504 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0750034e40 con 0x55a05dab5bc0 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f075f836740 1 -- 172.21.15.18:0/309075504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a05d79b890 con 0x55a05dab5bc0 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f073dffb640 1 -- 172.21.15.18:0/309075504 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0750040430 con 0x55a05dab5bc0 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f075f836740 10 client.18360 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.910+0000 7f075f836740 1 -- 172.21.15.18:0/309075504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55a05d9d06a0 con 0x55a05dab5bc0 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.911+0000 7f073dffb640 1 -- 172.21.15.18:0/309075504 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a05d79b890 con 0x55a05dab5bc0 2024-02-17T12:57:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.911+0000 7f073dffb640 1 -- 172.21.15.18:0/309075504 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f0750067800 con 0x55a05dab5bc0 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.911+0000 7f075f836740 10 client.18360 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.911+0000 7f075f836740 10 client.18360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.911+0000 7f075f836740 20 client.18360 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.911+0000 7f075f836740 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a05db1d900 0x55a05db1fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.911+0000 7f075f836740 4 client.18360 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.911+0000 7f075f836740 1 -- 172.21.15.18:0/309075504 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a05d6fe4e0 con 0x55a05db1d900 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.911+0000 7f075659c640 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a05db1d900 0x55a05db1fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.911+0000 7f073dffb640 10 client.18360 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.911+0000 7f075659c640 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a05db1d900 0x55a05db1fcf0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b0267a649bdf00d server_cookie=45aeb94f0d564be9 in_seq=0 out_seq=0 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f073dffb640 1 -- 172.21.15.18:0/309075504 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a05d6fe4e0 con 0x55a05db1d900 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f073dffb640 10 client.18360 handle_command_reply: tid=0 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075f836740 1 client.18360 shutdown 2024-02-17T12:57:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075f836740 1 -- 172.21.15.18:0/309075504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 msgr2=0x55a05dab0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075f836740 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dab0150 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f07500348f0 tx=0x7f0750004260 comp rx=0 tx=0).stop 2024-02-17T12:57:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075f836740 1 -- 172.21.15.18:0/309075504 shutdown_connections 2024-02-17T12:57:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075f836740 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a05db1d900 0x55a05db1fcf0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075f836740 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a05dab5bc0 0x55a05dab0150 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f073dffb640 0 client.18360 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075f836740 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a05daa88c0 0x55a05daafc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075f836740 1 --2- 172.21.15.18:0/309075504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a05daa7fa0 0x55a05daaf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075f836740 1 -- 172.21.15.18:0/309075504 >> 172.21.15.18:0/309075504 conn(0x55a05daa2990 msgr2=0x55a05daa11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075f836740 1 -- 172.21.15.18:0/309075504 shutdown_connections 2024-02-17T12:57:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075f836740 1 -- 172.21.15.18:0/309075504 wait complete. 2024-02-17T12:57:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075f836740 20 client.18360 trim_cache size 0 max 16384 2024-02-17T12:57:51.914 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:57:51.914 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:57:51.914 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:57:51.914 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:57:51.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075cb8a640 1 -- 172.21.15.18:0/935249461 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07400a2230 msgr2=0x7f07400a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:51.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075cb8a640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07400a2230 0x7f07400a46f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0750005e30 tx=0x7f0750032f50 comp rx=0 tx=0).stop 2024-02-17T12:57:51.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075cb8a640 1 -- 172.21.15.18:0/935249461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07580ff8f0 msgr2=0x7f07581977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:51.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075cb8a640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07580ff8f0 0x7f07581977c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f074800d770 tx=0x7f074800dc30 comp rx=0 tx=0).stop 2024-02-17T12:57:51.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075cb8a640 1 -- 172.21.15.18:0/935249461 shutdown_connections 2024-02-17T12:57:51.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075cb8a640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f07400a2230 0x7f07400a46f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075cb8a640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f075819bfe0 0x7f075819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075cb8a640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0758106500 0x7f0758197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075cb8a640 1 --2- 172.21.15.18:0/935249461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07580ff8f0 0x7f07581977c0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:51.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075cb8a640 1 -- 172.21.15.18:0/935249461 >> 172.21.15.18:0/935249461 conn(0x7f07580765b0 msgr2=0x7f07580fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:51.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075cb8a640 1 -- 172.21.15.18:0/935249461 shutdown_connections 2024-02-17T12:57:51.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:51.912+0000 7f075cb8a640 1 -- 172.21.15.18:0/935249461 wait complete. 2024-02-17T12:57:51.916 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:57:51.916 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:57:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.029+0000 7efec2c0e640 1 -- 172.21.15.18:0/1931074115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efebc1053c0 msgr2=0x7efebc10cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.029+0000 7efec2c0e640 1 --2- 172.21.15.18:0/1931074115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efebc1053c0 0x7efebc10cf90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7efeb4009cd0 tx=0x7efeb402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 -- 172.21.15.18:0/1931074115 shutdown_connections 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 --2- 172.21.15.18:0/1931074115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efebc1053c0 0x7efebc10cf90 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 --2- 172.21.15.18:0/1931074115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efebc100ca0 0x7efebc104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 --2- 172.21.15.18:0/1931074115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efebc1002f0 0x7efebc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 -- 172.21.15.18:0/1931074115 >> 172.21.15.18:0/1931074115 conn(0x7efebc077180 msgr2=0x7efebc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 -- 172.21.15.18:0/1931074115 shutdown_connections 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 -- 172.21.15.18:0/1931074115 wait complete. 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 Processor -- start 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 -- start start 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efebc1002f0 0x7efebc19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efebc1053c0 0x7efebc19d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efebc197620 0x7efebc197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc198010 con 0x7efebc1053c0 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc198180 con 0x7efebc1002f0 2024-02-17T12:57:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efec2c0e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc198340 con 0x7efebc197620 2024-02-17T12:57:52.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efebbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efebc1053c0 0x7efebc19d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efebbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efebc1053c0 0x7efebc19d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35536/0 (socket says 172.21.15.18:35536) 2024-02-17T12:57:52.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.030+0000 7efebbfff640 1 -- 172.21.15.18:0/3345406710 learned_addr learned my addr 172.21.15.18:0/3345406710 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efec11ab640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efebc197620 0x7efebc197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efec09aa640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efebc1002f0 0x7efebc19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efebbfff640 1 -- 172.21.15.18:0/3345406710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efebc197620 msgr2=0x7efebc197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efebbfff640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efebc197620 0x7efebc197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efebbfff640 1 -- 172.21.15.18:0/3345406710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efebc1002f0 msgr2=0x7efebc19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efebbfff640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efebc1002f0 0x7efebc19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efebbfff640 1 -- 172.21.15.18:0/3345406710 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb4009900 con 0x7efebc1053c0 2024-02-17T12:57:52.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efec11ab640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efebc197620 0x7efebc197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:52.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efebbfff640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efebc1053c0 0x7efebc19d960 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7efea800c970 tx=0x7efea800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efeb9ffb640 1 -- 172.21.15.18:0/3345406710 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8007bb0 con 0x7efebc1053c0 2024-02-17T12:57:52.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efec2c0e640 1 -- 172.21.15.18:0/3345406710 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efebc1a42e0 con 0x7efebc1053c0 2024-02-17T12:57:52.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efeb9ffb640 1 -- 172.21.15.18:0/3345406710 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efea8007d10 con 0x7efebc1053c0 2024-02-17T12:57:52.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efeb9ffb640 1 -- 172.21.15.18:0/3345406710 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea800f450 con 0x7efebc1053c0 2024-02-17T12:57:52.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.031+0000 7efec2c0e640 1 -- 172.21.15.18:0/3345406710 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efebc1a47f0 con 0x7efebc1053c0 2024-02-17T12:57:52.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.032+0000 7efec2c0e640 1 -- 172.21.15.18:0/3345406710 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efebc067f70 con 0x7efebc1053c0 2024-02-17T12:57:52.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.033+0000 7efeb9ffb640 1 -- 172.21.15.18:0/3345406710 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7efea8004f20 con 0x7efebc1053c0 2024-02-17T12:57:52.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.034+0000 7efeb9ffb640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efeac0a2230 0x7efeac0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.034+0000 7efec09aa640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efeac0a2230 0x7efeac0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.034+0000 7efeb9ffb640 1 -- 172.21.15.18:0/3345406710 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efea80ce160 con 0x7efebc1053c0 2024-02-17T12:57:52.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.035+0000 7efec09aa640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efeac0a2230 0x7efeac0a46f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efeb0005e10 tx=0x7efeb0005da0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.039+0000 7efeb9ffb640 1 -- 172.21.15.18:0/3345406710 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efea8096db0 con 0x7efebc1053c0 2024-02-17T12:57:52.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:51 smithi146 ceph-mon[36454]: pgmap v247: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-02-17T12:57:52.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.261+0000 7efec2c0e640 1 -- 172.21.15.18:0/3345406710 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efebc198b00 con 0x7efebc1053c0 2024-02-17T12:57:52.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.262+0000 7efeb9ffb640 1 -- 172.21.15.18:0/3345406710 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+11393 (secure 0 0 0) 0x7efea801d090 con 0x7efebc1053c0 2024-02-17T12:57:52.252 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:57:52.252 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:57:33.474835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:57:52.252 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 44 2024-02-17T12:57:52.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.264+0000 7efec2c0e640 1 -- 172.21.15.18:0/3345406710 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efeac0a2230 msgr2=0x7efeac0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.264+0000 7efec2c0e640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efeac0a2230 0x7efeac0a46f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efeb0005e10 tx=0x7efeb0005da0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.264+0000 7efec2c0e640 1 -- 172.21.15.18:0/3345406710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efebc1053c0 msgr2=0x7efebc19d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.264+0000 7efec2c0e640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efebc1053c0 0x7efebc19d960 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7efea800c970 tx=0x7efea800ce30 comp rx=0 tx=0).stop 2024-02-17T12:57:52.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.264+0000 7efec2c0e640 1 -- 172.21.15.18:0/3345406710 shutdown_connections 2024-02-17T12:57:52.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.264+0000 7efec2c0e640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efeac0a2230 0x7efeac0a46f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.264+0000 7efec2c0e640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efebc197620 0x7efebc197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.264+0000 7efec2c0e640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efebc1053c0 0x7efebc19d960 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.264+0000 7efec2c0e640 1 --2- 172.21.15.18:0/3345406710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efebc1002f0 0x7efebc19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.264+0000 7efec2c0e640 1 -- 172.21.15.18:0/3345406710 >> 172.21.15.18:0/3345406710 conn(0x7efebc077180 msgr2=0x7efebc0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.264+0000 7efec2c0e640 1 -- 172.21.15.18:0/3345406710 shutdown_connections 2024-02-17T12:57:52.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.264+0000 7efec2c0e640 1 -- 172.21.15.18:0/3345406710 wait complete. 2024-02-17T12:57:52.263 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:51 smithi018 ceph-mon[32625]: pgmap v247: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-02-17T12:57:52.265 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:57:52.265 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:57:52.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 413183 5.39 MB/sec execute 307 sec latency 2325.736 ms 2024-02-17T12:57:52.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:51 smithi119 ceph-mon[36188]: pgmap v247: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-02-17T12:57:52.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.379+0000 7ff18b163640 1 -- 172.21.15.18:0/1251170651 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002a0 msgr2=0x7ff184100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.379+0000 7ff18b163640 1 --2- 172.21.15.18:0/1251170651 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002a0 0x7ff184100680 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff178009930 tx=0x7ff17802bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:52.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.379+0000 7ff18b163640 1 -- 172.21.15.18:0/1251170651 shutdown_connections 2024-02-17T12:57:52.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.379+0000 7ff18b163640 1 --2- 172.21.15.18:0/1251170651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184105370 0x7ff18410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.379+0000 7ff18b163640 1 --2- 172.21.15.18:0/1251170651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff184100c50 0x7ff184104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.379+0000 7ff18b163640 1 --2- 172.21.15.18:0/1251170651 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002a0 0x7ff184100680 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.379+0000 7ff18b163640 1 -- 172.21.15.18:0/1251170651 >> 172.21.15.18:0/1251170651 conn(0x7ff184076400 msgr2=0x7ff184076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.379+0000 7ff18b163640 1 -- 172.21.15.18:0/1251170651 shutdown_connections 2024-02-17T12:57:52.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.379+0000 7ff18b163640 1 -- 172.21.15.18:0/1251170651 wait complete. 2024-02-17T12:57:52.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.379+0000 7ff18b163640 1 Processor -- start 2024-02-17T12:57:52.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.379+0000 7ff18b163640 1 -- start start 2024-02-17T12:57:52.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff18b163640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1841002a0 0x7ff18419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff18b163640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff184105370 0x7ff18419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff18b163640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff184197630 0x7ff184197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff18b163640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff184198100 con 0x7ff184197630 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff18b163640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff184198270 con 0x7ff1841002a0 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff18b163640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff184198430 con 0x7ff184105370 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff189700640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff184197630 0x7ff184197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff189700640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff184197630 0x7ff184197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35550/0 (socket says 172.21.15.18:35550) 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff189700640 1 -- 172.21.15.18:0/3578037631 learned_addr learned my addr 172.21.15.18:0/3578037631 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff188eff640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1841002a0 0x7ff18419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff183fff640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff184105370 0x7ff18419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff189700640 1 -- 172.21.15.18:0/3578037631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff184105370 msgr2=0x7ff18419dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff189700640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff184105370 0x7ff18419dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff189700640 1 -- 172.21.15.18:0/3578037631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1841002a0 msgr2=0x7ff18419d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff189700640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1841002a0 0x7ff18419d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff189700640 1 -- 172.21.15.18:0/3578037631 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff178009560 con 0x7ff184197630 2024-02-17T12:57:52.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff183fff640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff184105370 0x7ff18419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:52.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff189700640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff184197630 0x7ff184197ab0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00eaa0 tx=0x7ff17c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff181ffb640 1 -- 172.21.15.18:0/3578037631 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00cdb0 con 0x7ff184197630 2024-02-17T12:57:52.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff18b163640 1 -- 172.21.15.18:0/3578037631 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1841a43d0 con 0x7ff184197630 2024-02-17T12:57:52.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff181ffb640 1 -- 172.21.15.18:0/3578037631 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff17c005020 con 0x7ff184197630 2024-02-17T12:57:52.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.380+0000 7ff181ffb640 1 -- 172.21.15.18:0/3578037631 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0116b0 con 0x7ff184197630 2024-02-17T12:57:52.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.381+0000 7ff18b163640 1 -- 172.21.15.18:0/3578037631 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1841a4860 con 0x7ff184197630 2024-02-17T12:57:52.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.382+0000 7ff181ffb640 1 -- 172.21.15.18:0/3578037631 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff17c005520 con 0x7ff184197630 2024-02-17T12:57:52.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.383+0000 7ff18de0c740 1 Processor -- start 2024-02-17T12:57:52.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.383+0000 7ff18de0c740 1 -- start start 2024-02-17T12:57:52.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.383+0000 7ff18de0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd61585630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.383+0000 7ff181ffb640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff1740a2230 0x7ff1740a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.383+0000 7ff18de0c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585b70 0x55cd6158e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.383+0000 7ff18de0c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd6158e750 0x55cd61590b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.383+0000 7ff18de0c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd61540960 con 0x55cd61585b70 2024-02-17T12:57:52.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.383+0000 7ff18de0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd614f4310 con 0x55cd61585250 2024-02-17T12:57:52.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.383+0000 7ff18de0c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd614f1840 con 0x55cd6158e750 2024-02-17T12:57:52.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.383+0000 7ff183fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585b70 0x55cd6158e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.383+0000 7ff188eff640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff1740a2230 0x7ff1740a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff181ffb640 1 -- 172.21.15.18:0/3578037631 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff17c0ce2c0 con 0x7ff184197630 2024-02-17T12:57:52.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff183fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585b70 0x55cd6158e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35564/0 (socket says 172.21.15.18:35564) 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff183fff640 1 -- 172.21.15.18:0/2797980579 learned_addr learned my addr 172.21.15.18:0/2797980579 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff183fff640 1 --2- 172.21.15.18:0/2797980579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd61585630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff189700640 1 --2- 172.21.15.18:0/2797980579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd6158e750 0x55cd61590b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff183fff640 1 -- 172.21.15.18:0/2797980579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd6158e750 msgr2=0x55cd61590b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff183fff640 1 --2- 172.21.15.18:0/2797980579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd6158e750 0x55cd61590b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff183fff640 1 -- 172.21.15.18:0/2797980579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 msgr2=0x55cd61585630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff183fff640 1 --2- 172.21.15.18:0/2797980579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd61585630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff183fff640 1 -- 172.21.15.18:0/2797980579 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd614af060 con 0x55cd61585b70 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff188eff640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff1740a2230 0x7ff1740a46f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff178005ab0 tx=0x7ff178005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff183fff640 1 --2- 172.21.15.18:0/2797980579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585b70 0x55cd6158e210 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff1700155c0 tx=0x7ff170009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810c3f3f91ea16c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff16effd640 1 -- 172.21.15.18:0/2797980579 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170019030 con 0x55cd61585b70 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff16effd640 1 -- 172.21.15.18:0/2797980579 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1700139a0 con 0x55cd61585b70 2024-02-17T12:57:52.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.384+0000 7ff16effd640 1 -- 172.21.15.18:0/2797980579 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17000d8c0 con 0x55cd61585b70 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.385+0000 7ff18de0c740 1 -- 172.21.15.18:0/2797980579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585b70 msgr2=0x55cd6158e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.385+0000 7ff18de0c740 1 --2- 172.21.15.18:0/2797980579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585b70 0x55cd6158e210 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff1700155c0 tx=0x7ff170009640 comp rx=0 tx=0).stop 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.385+0000 7ff18de0c740 1 -- 172.21.15.18:0/2797980579 shutdown_connections 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.385+0000 7ff18de0c740 1 --2- 172.21.15.18:0/2797980579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd6158e750 0x55cd61590b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.385+0000 7ff18de0c740 1 --2- 172.21.15.18:0/2797980579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585b70 0x55cd6158e210 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.385+0000 7ff18de0c740 1 --2- 172.21.15.18:0/2797980579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd61585630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.385+0000 7ff18de0c740 1 -- 172.21.15.18:0/2797980579 >> 172.21.15.18:0/2797980579 conn(0x55cd6157f980 msgr2=0x55cd6157fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.385+0000 7ff18de0c740 1 -- 172.21.15.18:0/2797980579 shutdown_connections 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.385+0000 7ff18de0c740 1 -- 172.21.15.18:0/2797980579 wait complete. 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.385+0000 7ff18de0c740 1 -- start start 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.385+0000 7ff18de0c740 1 Processor -- start 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff18de0c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585250 0x55cd6158b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff18de0c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 0x55cd6158bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff18de0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd6158e750 0x55cd6158c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff18de0c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd614ed020 con 0x55cd61585250 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff18de0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd611c0430 con 0x55cd6158e750 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff18de0c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6117ce00 con 0x55cd61585b70 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff18de0c740 10 client.0 _ll_register_callbacks cb 0x55cd61494cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff188eff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585250 0x55cd6158b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff188eff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585250 0x55cd6158b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35580/0 (socket says 172.21.15.18:35580) 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff188eff640 1 -- 172.21.15.18:0/4183277488 learned_addr learned my addr 172.21.15.18:0/4183277488 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff183fff640 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 0x55cd6158bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff189700640 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd6158e750 0x55cd6158c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff188eff640 1 -- 172.21.15.18:0/4183277488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 msgr2=0x55cd6158bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff188eff640 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 0x55cd6158bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff188eff640 1 -- 172.21.15.18:0/4183277488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd6158e750 msgr2=0x55cd6158c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff188eff640 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd6158e750 0x55cd6158c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff188eff640 1 -- 172.21.15.18:0/4183277488 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff170009d50 con 0x55cd61585250 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff188eff640 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585250 0x55cd6158b750 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff178009900 tx=0x7ff178009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cb2d12accc7f13 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff163fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff163fff640 1 -- 172.21.15.18:0/4183277488 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178004410 con 0x55cd61585250 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff163fff640 1 -- 172.21.15.18:0/4183277488 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff178047050 con 0x55cd61585250 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.386+0000 7ff18de0c740 1 -- 172.21.15.18:0/4183277488 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd611ec7e0 con 0x55cd61585250 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.387+0000 7ff163fff640 1 -- 172.21.15.18:0/4183277488 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff178004df0 con 0x55cd61585250 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.387+0000 7ff18de0c740 10 client.18378 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.387+0000 7ff18de0c740 1 -- 172.21.15.18:0/4183277488 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55cd614eb6a0 con 0x55cd61585250 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.387+0000 7ff163fff640 1 -- 172.21.15.18:0/4183277488 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cd611ec7e0 con 0x55cd61585250 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.387+0000 7ff163fff640 1 -- 172.21.15.18:0/4183277488 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ff178003a80 con 0x55cd61585250 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.387+0000 7ff18de0c740 10 client.18378 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.387+0000 7ff18de0c740 10 client.18378 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.387+0000 7ff18de0c740 20 client.18378 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.387+0000 7ff18de0c740 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55cd6159eac0 0x55cd615a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.387+0000 7ff18de0c740 4 client.18378 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.387+0000 7ff18de0c740 1 -- 172.21.15.18:0/4183277488 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd61229720 con 0x55cd6159eac0 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.387+0000 7ff183fff640 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55cd6159eac0 0x55cd615a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff163fff640 10 client.18378 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff183fff640 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55cd6159eac0 0x55cd615a0eb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=984a00fee3e92e77 server_cookie=11d34baf46cf032b in_seq=0 out_seq=0 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff163fff640 1 -- 172.21.15.18:0/4183277488 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd61229720 con 0x55cd6159eac0 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff163fff640 10 client.18378 handle_command_reply: tid=0 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff18de0c740 1 client.18378 shutdown 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff18de0c740 1 -- 172.21.15.18:0/4183277488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585250 msgr2=0x55cd6158b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff18de0c740 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585250 0x55cd6158b750 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff178009900 tx=0x7ff178009260 comp rx=0 tx=0).stop 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff18de0c740 1 -- 172.21.15.18:0/4183277488 shutdown_connections 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff18de0c740 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55cd6159eac0 0x55cd615a0eb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff18de0c740 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd6158e750 0x55cd6158c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff163fff640 0 client.18378 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff18de0c740 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 0x55cd6158bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff18de0c740 1 --2- 172.21.15.18:0/4183277488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd61585250 0x55cd6158b750 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.388+0000 7ff18de0c740 1 -- 172.21.15.18:0/4183277488 >> 172.21.15.18:0/4183277488 conn(0x55cd6157f980 msgr2=0x55cd615816d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.389+0000 7ff18de0c740 1 -- 172.21.15.18:0/4183277488 shutdown_connections 2024-02-17T12:57:52.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.389+0000 7ff18de0c740 1 -- 172.21.15.18:0/4183277488 wait complete. 2024-02-17T12:57:52.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.389+0000 7ff18de0c740 20 client.18378 trim_cache size 0 max 16384 2024-02-17T12:57:52.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.400+0000 7ff18de0c740 1 Processor -- start 2024-02-17T12:57:52.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.400+0000 7ff18de0c740 1 -- start start 2024-02-17T12:57:52.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.400+0000 7ff18de0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd615814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.400+0000 7ff18de0c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 0x55cd61595600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.400+0000 7ff18de0c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 0x55cd61595b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.400+0000 7ff18de0c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd61472130 con 0x55cd6158e750 2024-02-17T12:57:52.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.400+0000 7ff18de0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd61584650 con 0x55cd61585250 2024-02-17T12:57:52.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.400+0000 7ff18de0c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd61580bb0 con 0x55cd61585b70 2024-02-17T12:57:52.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff189700640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 0x55cd61595b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff189700640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 0x55cd61595b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35588/0 (socket says 172.21.15.18:35588) 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff189700640 1 -- 172.21.15.18:0/132680325 learned_addr learned my addr 172.21.15.18:0/132680325 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff183fff640 1 --2- 172.21.15.18:0/132680325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 0x55cd61595600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff189700640 1 -- 172.21.15.18:0/132680325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 msgr2=0x55cd61595600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff189700640 1 --2- 172.21.15.18:0/132680325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 0x55cd61595600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff189700640 1 -- 172.21.15.18:0/132680325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 msgr2=0x55cd615814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff188eff640 1 --2- 172.21.15.18:0/132680325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd615814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff189700640 1 --2- 172.21.15.18:0/132680325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd615814c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff189700640 1 -- 172.21.15.18:0/132680325 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd61487c20 con 0x55cd6158e750 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff188eff640 1 --2- 172.21.15.18:0/132680325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd615814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff189700640 1 --2- 172.21.15.18:0/132680325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 0x55cd61595b40 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ff17c09a970 tx=0x7ff17c09a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2cf0ea1a487079 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff16e7fc640 1 -- 172.21.15.18:0/132680325 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0a3410 con 0x55cd6158e750 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff16e7fc640 1 -- 172.21.15.18:0/132680325 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff17c098d10 con 0x55cd6158e750 2024-02-17T12:57:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.401+0000 7ff16e7fc640 1 -- 172.21.15.18:0/132680325 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c005310 con 0x55cd6158e750 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 -- 172.21.15.18:0/132680325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 msgr2=0x55cd61595b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 --2- 172.21.15.18:0/132680325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 0x55cd61595b40 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ff17c09a970 tx=0x7ff17c09a8e0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 -- 172.21.15.18:0/132680325 shutdown_connections 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 --2- 172.21.15.18:0/132680325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 0x55cd61595b40 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 --2- 172.21.15.18:0/132680325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 0x55cd61595600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 --2- 172.21.15.18:0/132680325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd615814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 -- 172.21.15.18:0/132680325 >> 172.21.15.18:0/132680325 conn(0x55cd6157f980 msgr2=0x55cd61580db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 -- 172.21.15.18:0/132680325 shutdown_connections 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 -- 172.21.15.18:0/132680325 wait complete. 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 -- start start 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 Processor -- start 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd615946a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 0x55cd61594be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 0x55cd615f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd615952e0 con 0x55cd6158e750 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd615f5520 con 0x55cd61585250 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.402+0000 7ff18de0c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd615f5660 con 0x55cd61585b70 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff18de0c740 10 client.0 _ll_register_callbacks cb 0x7ff1700094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff189700640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 0x55cd615f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff189700640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 0x55cd615f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35592/0 (socket says 172.21.15.18:35592) 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff189700640 1 -- 172.21.15.18:0/974422336 learned_addr learned my addr 172.21.15.18:0/974422336 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff188eff640 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd615946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff183fff640 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 0x55cd61594be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff189700640 1 -- 172.21.15.18:0/974422336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 msgr2=0x55cd61594be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff189700640 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 0x55cd61594be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff189700640 1 -- 172.21.15.18:0/974422336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 msgr2=0x55cd615946a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff189700640 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd615946a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff189700640 1 -- 172.21.15.18:0/974422336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1841a4860 con 0x55cd6158e750 2024-02-17T12:57:52.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff188eff640 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd615946a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:52.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff189700640 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 0x55cd615f4fe0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff17c09a740 tx=0x7ff17c09adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24da2e0fcca3e814 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff163fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:52.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff163fff640 1 -- 172.21.15.18:0/974422336 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c09a2c0 con 0x55cd6158e750 2024-02-17T12:57:52.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff163fff640 1 -- 172.21.15.18:0/974422336 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff17c0ce040 con 0x55cd6158e750 2024-02-17T12:57:52.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff18de0c740 1 -- 172.21.15.18:0/974422336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1700139a0 con 0x55cd6158e750 2024-02-17T12:57:52.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff163fff640 1 -- 172.21.15.18:0/974422336 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff17c09a420 con 0x55cd6158e750 2024-02-17T12:57:52.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff18de0c740 10 client.18384 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:52.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff18de0c740 1 -- 172.21.15.18:0/974422336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55cd614ad8c0 con 0x55cd6158e750 2024-02-17T12:57:52.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.403+0000 7ff163fff640 1 -- 172.21.15.18:0/974422336 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1700139a0 con 0x55cd6158e750 2024-02-17T12:57:52.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff163fff640 1 -- 172.21.15.18:0/974422336 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ff17c018030 con 0x55cd6158e750 2024-02-17T12:57:52.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff18de0c740 10 client.18384 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:52.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff18de0c740 10 client.18384 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:57:52.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff18de0c740 20 client.18384 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:52.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff18de0c740 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55cd615f6950 0x55cd615f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff18de0c740 4 client.18384 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:57:52.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff18de0c740 1 -- 172.21.15.18:0/974422336 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cd61278890 con 0x55cd615f6950 2024-02-17T12:57:52.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff188eff640 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55cd615f6950 0x55cd615f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff163fff640 10 client.18384 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:57:52.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff188eff640 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55cd615f6950 0x55cd615f8d40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e479f64174486fee server_cookie=fb3d751512b16aea in_seq=0 out_seq=0 2024-02-17T12:57:52.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff163fff640 1 -- 172.21.15.18:0/974422336 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cd61278890 con 0x55cd615f6950 2024-02-17T12:57:52.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff163fff640 10 client.18384 handle_command_reply: tid=0 2024-02-17T12:57:52.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.404+0000 7ff18de0c740 1 client.18384 shutdown 2024-02-17T12:57:52.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18de0c740 1 -- 172.21.15.18:0/974422336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 msgr2=0x55cd615f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18de0c740 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 0x55cd615f4fe0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff17c09a740 tx=0x7ff17c09adf0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18de0c740 1 -- 172.21.15.18:0/974422336 shutdown_connections 2024-02-17T12:57:52.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18de0c740 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd6158e750 0x55cd615f4fe0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18de0c740 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd61585b70 0x55cd61594be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18de0c740 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55cd615f6950 0x55cd615f8d40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18de0c740 1 --2- 172.21.15.18:0/974422336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd61585250 0x55cd615946a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff163fff640 0 client.18384 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:57:52.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18de0c740 1 -- 172.21.15.18:0/974422336 >> 172.21.15.18:0/974422336 conn(0x55cd6157f980 msgr2=0x55cd61581640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18de0c740 1 -- 172.21.15.18:0/974422336 shutdown_connections 2024-02-17T12:57:52.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18de0c740 1 -- 172.21.15.18:0/974422336 wait complete. 2024-02-17T12:57:52.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18de0c740 20 client.18384 trim_cache size 0 max 16384 2024-02-17T12:57:52.404 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:57:52.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18b163640 1 -- 172.21.15.18:0/3578037631 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff1740a2230 msgr2=0x7ff1740a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18b163640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff1740a2230 0x7ff1740a46f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff178005ab0 tx=0x7ff178005fd0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18b163640 1 -- 172.21.15.18:0/3578037631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff184197630 msgr2=0x7ff184197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18b163640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff184197630 0x7ff184197ab0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00eaa0 tx=0x7ff17c00ef60 comp rx=0 tx=0).stop 2024-02-17T12:57:52.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18b163640 1 -- 172.21.15.18:0/3578037631 shutdown_connections 2024-02-17T12:57:52.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18b163640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff1740a2230 0x7ff1740a46f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18b163640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff184197630 0x7ff184197ab0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18b163640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff184105370 0x7ff18419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18b163640 1 --2- 172.21.15.18:0/3578037631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1841002a0 0x7ff18419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18b163640 1 -- 172.21.15.18:0/3578037631 >> 172.21.15.18:0/3578037631 conn(0x7ff184076400 msgr2=0x7ff1840fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18b163640 1 -- 172.21.15.18:0/3578037631 shutdown_connections 2024-02-17T12:57:52.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.405+0000 7ff18b163640 1 -- 172.21.15.18:0/3578037631 wait complete. 2024-02-17T12:57:52.407 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:57:52.407 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:57:52.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.561+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2926391581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f8105330 msgr2=0x7f57f810cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.561+0000 7f57ffaf1640 1 --2- 172.21.15.18:0/2926391581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f8105330 0x7f57f810cf00 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009cd0 tx=0x7f57f002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2926391581 shutdown_connections 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 --2- 172.21.15.18:0/2926391581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f8105330 0x7f57f810cf00 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 --2- 172.21.15.18:0/2926391581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f8100c10 0x7f57f8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 --2- 172.21.15.18:0/2926391581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57f8100260 0x7f57f8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2926391581 >> 172.21.15.18:0/2926391581 conn(0x7f57f8077180 msgr2=0x7f57f8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2926391581 shutdown_connections 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2926391581 wait complete. 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 Processor -- start 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 -- start start 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f8100260 0x7f57f819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57f8105330 0x7f57f819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f81975b0 0x7f57f8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f8197fa0 con 0x7f57f81975b0 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f8198110 con 0x7f57f8100260 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57ffaf1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f81982d0 con 0x7f57f8105330 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.562+0000 7f57fe08e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f81975b0 0x7f57f8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57fe08e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f81975b0 0x7f57f8197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35608/0 (socket says 172.21.15.18:35608) 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57fe08e640 1 -- 172.21.15.18:0/2975328073 learned_addr learned my addr 172.21.15.18:0/2975328073 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57fd08c640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57f8105330 0x7f57f819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57fd88d640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f8100260 0x7f57f819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57fe08e640 1 -- 172.21.15.18:0/2975328073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57f8105330 msgr2=0x7f57f819d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57fe08e640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57f8105330 0x7f57f819d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57fe08e640 1 -- 172.21.15.18:0/2975328073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f8100260 msgr2=0x7f57f819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57fe08e640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f8100260 0x7f57f819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57fe08e640 1 -- 172.21.15.18:0/2975328073 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f0009900 con 0x7f57f81975b0 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57fd88d640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f8100260 0x7f57f819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57fe08e640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f81975b0 0x7f57f8197a60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005e30 tx=0x7f57f0034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57eeffd640 1 -- 172.21.15.18:0/2975328073 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0032ac0 con 0x7f57f81975b0 2024-02-17T12:57:52.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57eeffd640 1 -- 172.21.15.18:0/2975328073 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57f0032eb0 con 0x7f57f81975b0 2024-02-17T12:57:52.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2975328073 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f81a42c0 con 0x7f57f81975b0 2024-02-17T12:57:52.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57eeffd640 1 -- 172.21.15.18:0/2975328073 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0040450 con 0x7f57f81975b0 2024-02-17T12:57:52.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.563+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2975328073 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f81a4740 con 0x7f57f81975b0 2024-02-17T12:57:52.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.566+0000 7f57eeffd640 1 -- 172.21.15.18:0/2975328073 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f57f000a040 con 0x7f57f81975b0 2024-02-17T12:57:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.566+0000 7f580279f740 1 Processor -- start 2024-02-17T12:57:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.566+0000 7f580279f740 1 -- start start 2024-02-17T12:57:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.566+0000 7f580279f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 0x55682e9c0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.566+0000 7f580279f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0b70 0x55682e9c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.566+0000 7f580279f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c9750 0x55682e9cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.566+0000 7f580279f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682e97b960 con 0x55682e9c0b70 2024-02-17T12:57:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.566+0000 7f580279f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682e92f310 con 0x55682e9c9750 2024-02-17T12:57:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.566+0000 7f580279f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682e92c840 con 0x55682e9c0250 2024-02-17T12:57:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.566+0000 7f57eeffd640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f57e40a2230 0x7f57e40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fd88d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0b70 0x55682e9c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fe08e640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f57e40a2230 0x7f57e40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fd88d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0b70 0x55682e9c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35624/0 (socket says 172.21.15.18:35624) 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fd88d640 1 -- 172.21.15.18:0/52572184 learned_addr learned my addr 172.21.15.18:0/52572184 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fd08c640 1 --2- 172.21.15.18:0/52572184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 0x55682e9c0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fd08c640 1 --2- 172.21.15.18:0/52572184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c9750 0x55682e9cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57eeffd640 1 -- 172.21.15.18:0/2975328073 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f57f00f0b10 con 0x7f57f81975b0 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fd88d640 1 -- 172.21.15.18:0/52572184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 msgr2=0x55682e9c0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fd88d640 1 --2- 172.21.15.18:0/52572184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 0x55682e9c0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fd88d640 1 -- 172.21.15.18:0/52572184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c9750 msgr2=0x55682e9cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fd88d640 1 --2- 172.21.15.18:0/52572184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c9750 0x55682e9cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fd88d640 1 -- 172.21.15.18:0/52572184 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55682e8ea060 con 0x55682e9c0b70 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fe08e640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f57e40a2230 0x7f57e40a46f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f57f00bd2d0 tx=0x7f57f00af040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57fd88d640 1 --2- 172.21.15.18:0/52572184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0b70 0x55682e9c9210 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f57f4005fd0 tx=0x7f57f400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee556d129b85c574 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57e3fff640 1 -- 172.21.15.18:0/52572184 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4005020 con 0x55682e9c0b70 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57e3fff640 1 -- 172.21.15.18:0/52572184 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57f4005660 con 0x55682e9c0b70 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.567+0000 7f57e3fff640 1 -- 172.21.15.18:0/52572184 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4004ea0 con 0x55682e9c0b70 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.568+0000 7f580279f740 1 -- 172.21.15.18:0/52572184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0b70 msgr2=0x55682e9c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.568+0000 7f580279f740 1 --2- 172.21.15.18:0/52572184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0b70 0x55682e9c9210 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f57f4005fd0 tx=0x7f57f400e6c0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.568+0000 7f580279f740 1 -- 172.21.15.18:0/52572184 shutdown_connections 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.568+0000 7f580279f740 1 --2- 172.21.15.18:0/52572184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c9750 0x55682e9cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.568+0000 7f580279f740 1 --2- 172.21.15.18:0/52572184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0b70 0x55682e9c9210 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.568+0000 7f580279f740 1 --2- 172.21.15.18:0/52572184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 0x55682e9c0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.568+0000 7f580279f740 1 -- 172.21.15.18:0/52572184 >> 172.21.15.18:0/52572184 conn(0x55682e9ba980 msgr2=0x55682e9bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.568+0000 7f580279f740 1 -- 172.21.15.18:0/52572184 shutdown_connections 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.568+0000 7f580279f740 1 -- 172.21.15.18:0/52572184 wait complete. 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.568+0000 7f580279f740 1 -- start start 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f580279f740 1 Processor -- start 2024-02-17T12:57:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f580279f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 0x55682e9c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f580279f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c0b70 0x55682e9c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f580279f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 0x55682e9c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f580279f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682e928020 con 0x55682e9c9750 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f580279f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682e5fb430 con 0x55682e9c0b70 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f580279f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682e5b7e00 con 0x55682e9c0250 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f580279f740 10 client.0 _ll_register_callbacks cb 0x55682e8cfcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f57fd08c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 0x55682e9c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f57fd08c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 0x55682e9c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35638/0 (socket says 172.21.15.18:35638) 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f57fd08c640 1 -- 172.21.15.18:0/3164882697 learned_addr learned my addr 172.21.15.18:0/3164882697 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f57fd08c640 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 0x55682e9c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f57fd08c640 1 -- 172.21.15.18:0/3164882697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 msgr2=0x55682e9c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f57fd08c640 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 0x55682e9c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f57fd08c640 1 -- 172.21.15.18:0/3164882697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c0b70 msgr2=0x55682e9c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f57fd88d640 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c0b70 0x55682e9c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f57fd08c640 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c0b70 0x55682e9c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f57fd08c640 1 -- 172.21.15.18:0/3164882697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f4009900 con 0x55682e9c9750 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.569+0000 7f57fd88d640 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c0b70 0x55682e9c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f57fd08c640 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 0x55682e9c71e0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f57e80127d0 tx=0x7f57e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac274a71dec17b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f57e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f57e17fa640 1 -- 172.21.15.18:0/3164882697 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8010ce0 con 0x55682e9c9750 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f57e17fa640 1 -- 172.21.15.18:0/3164882697 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f57e8010e40 con 0x55682e9c9750 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f580279f740 1 -- 172.21.15.18:0/3164882697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55682e6277e0 con 0x55682e9c9750 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f57e17fa640 1 -- 172.21.15.18:0/3164882697 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57e8004030 con 0x55682e9c9750 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f580279f740 10 client.18396 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f580279f740 1 -- 172.21.15.18:0/3164882697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55682e9266a0 con 0x55682e9c9750 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f57e17fa640 1 -- 172.21.15.18:0/3164882697 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55682e6277e0 con 0x55682e9c9750 2024-02-17T12:57:52.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f57e17fa640 1 -- 172.21.15.18:0/3164882697 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f57e801e070 con 0x55682e9c9750 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f580279f740 10 client.18396 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f580279f740 10 client.18396 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f580279f740 20 client.18396 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f580279f740 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55682e9d9c30 0x55682e9dc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f580279f740 4 client.18396 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.570+0000 7f580279f740 1 -- 172.21.15.18:0/3164882697 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55682e664720 con 0x55682e9d9c30 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.571+0000 7f57fd88d640 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55682e9d9c30 0x55682e9dc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.571+0000 7f57fd88d640 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55682e9d9c30 0x55682e9dc050 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1bf3ab2345030116 server_cookie=764babae6abbad87 in_seq=0 out_seq=0 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.571+0000 7f57e17fa640 10 client.18396 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f57e17fa640 1 -- 172.21.15.18:0/3164882697 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55682e664720 con 0x55682e9d9c30 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f57e17fa640 10 client.18396 handle_command_reply: tid=0 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f580279f740 1 client.18396 shutdown 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f580279f740 1 -- 172.21.15.18:0/3164882697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 msgr2=0x55682e9c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f580279f740 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 0x55682e9c71e0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f57e80127d0 tx=0x7f57e80046c0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f580279f740 1 -- 172.21.15.18:0/3164882697 shutdown_connections 2024-02-17T12:57:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f580279f740 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55682e9d9c30 0x55682e9dc050 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f580279f740 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 0x55682e9c71e0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f580279f740 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c0b70 0x55682e9c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f580279f740 1 --2- 172.21.15.18:0/3164882697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 0x55682e9c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f57e17fa640 0 client.18396 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:57:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f580279f740 1 -- 172.21.15.18:0/3164882697 >> 172.21.15.18:0/3164882697 conn(0x55682e9ba980 msgr2=0x55682e9bc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f580279f740 1 -- 172.21.15.18:0/3164882697 shutdown_connections 2024-02-17T12:57:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f580279f740 1 -- 172.21.15.18:0/3164882697 wait complete. 2024-02-17T12:57:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.572+0000 7f580279f740 20 client.18396 trim_cache size 0 max 16384 2024-02-17T12:57:52.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f580279f740 1 Processor -- start 2024-02-17T12:57:52.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f580279f740 1 -- start start 2024-02-17T12:57:52.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f580279f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0250 0x55682e9c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f580279f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0b70 0x55682e9c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f580279f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c9750 0x55682e9c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f580279f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682e8ad130 con 0x55682e9c0250 2024-02-17T12:57:52.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f580279f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682e9b62b0 con 0x55682e9c9750 2024-02-17T12:57:52.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f580279f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682e664750 con 0x55682e9c0b70 2024-02-17T12:57:52.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f57fd08c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0250 0x55682e9c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f57fd08c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0250 0x55682e9c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35646/0 (socket says 172.21.15.18:35646) 2024-02-17T12:57:52.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f57fd08c640 1 -- 172.21.15.18:0/3354294481 learned_addr learned my addr 172.21.15.18:0/3354294481 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f57fd08c640 1 -- 172.21.15.18:0/3354294481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0b70 msgr2=0x55682e9c1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f57fd08c640 1 --2- 172.21.15.18:0/3354294481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0b70 0x55682e9c1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f57fd08c640 1 -- 172.21.15.18:0/3354294481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c9750 msgr2=0x55682e9c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f57fd08c640 1 --2- 172.21.15.18:0/3354294481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c9750 0x55682e9c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f57fd08c640 1 -- 172.21.15.18:0/3354294481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55682e8c2c20 con 0x55682e9c0250 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.584+0000 7f57fd08c640 1 --2- 172.21.15.18:0/3354294481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0250 0x55682e9c8e90 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f57e8000c00 tx=0x7f57e800b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bc7939e886292f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f57e37fe640 1 -- 172.21.15.18:0/3354294481 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e800b860 con 0x55682e9c0250 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f57e37fe640 1 -- 172.21.15.18:0/3354294481 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57e800b9c0 con 0x55682e9c0250 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f57e37fe640 1 -- 172.21.15.18:0/3354294481 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8004320 con 0x55682e9c0250 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f580279f740 1 -- 172.21.15.18:0/3354294481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0250 msgr2=0x55682e9c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f580279f740 1 --2- 172.21.15.18:0/3354294481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0250 0x55682e9c8e90 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f57e8000c00 tx=0x7f57e800b5e0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f580279f740 1 -- 172.21.15.18:0/3354294481 shutdown_connections 2024-02-17T12:57:52.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f580279f740 1 --2- 172.21.15.18:0/3354294481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c9750 0x55682e9c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f580279f740 1 --2- 172.21.15.18:0/3354294481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0b70 0x55682e9c1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f580279f740 1 --2- 172.21.15.18:0/3354294481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c0250 0x55682e9c8e90 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f580279f740 1 -- 172.21.15.18:0/3354294481 >> 172.21.15.18:0/3354294481 conn(0x55682e9ba980 msgr2=0x55682e9b5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f580279f740 1 -- 172.21.15.18:0/3354294481 shutdown_connections 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f580279f740 1 -- 172.21.15.18:0/3354294481 wait complete. 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.585+0000 7f580279f740 1 -- start start 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f580279f740 1 Processor -- start 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f580279f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 0x55682e9cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f580279f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c0b70 0x55682e9cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f580279f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 0x55682ea30140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f580279f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682ea30680 con 0x55682e9c9750 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f580279f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682ea307c0 con 0x55682e9c0b70 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f580279f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682ea30900 con 0x55682e9c0250 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f580279f740 10 client.0 _ll_register_callbacks cb 0x7f57f4009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:52.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f57fd08c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 0x55682ea30140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f57fd08c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 0x55682ea30140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35650/0 (socket says 172.21.15.18:35650) 2024-02-17T12:57:52.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f57fd08c640 1 -- 172.21.15.18:0/1640225476 learned_addr learned my addr 172.21.15.18:0/1640225476 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f57fd08c640 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 0x55682e9cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f57fd88d640 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c0b70 0x55682e9cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f57fd08c640 1 -- 172.21.15.18:0/1640225476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 msgr2=0x55682e9cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f57fd08c640 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 0x55682e9cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f57fd08c640 1 -- 172.21.15.18:0/1640225476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c0b70 msgr2=0x55682e9cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f57fd08c640 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c0b70 0x55682e9cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.586+0000 7f57fd08c640 1 -- 172.21.15.18:0/1640225476 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55682e9266a0 con 0x55682e9c9750 2024-02-17T12:57:52.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f57fd08c640 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 0x55682ea30140 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f57e8004000 tx=0x7f57e8012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256533b718b8418e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f57e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:52.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f57e17fa640 1 -- 172.21.15.18:0/1640225476 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8005240 con 0x55682e9c9750 2024-02-17T12:57:52.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f57e17fa640 1 -- 172.21.15.18:0/1640225476 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f57e8000e40 con 0x55682e9c9750 2024-02-17T12:57:52.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f580279f740 1 -- 172.21.15.18:0/1640225476 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55682e9c6590 con 0x55682e9c9750 2024-02-17T12:57:52.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f57e17fa640 1 -- 172.21.15.18:0/1640225476 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57e8020dc0 con 0x55682e9c9750 2024-02-17T12:57:52.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f580279f740 10 client.18402 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:52.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f580279f740 1 -- 172.21.15.18:0/1640225476 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55682e8e88c0 con 0x55682e9c9750 2024-02-17T12:57:52.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f57e17fa640 1 -- 172.21.15.18:0/1640225476 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55682e9c6590 con 0x55682e9c9750 2024-02-17T12:57:52.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f57e17fa640 1 -- 172.21.15.18:0/1640225476 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f57e802f070 con 0x55682e9c9750 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f580279f740 10 client.18402 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f580279f740 10 client.18402 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f580279f740 20 client.18402 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f580279f740 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55682ea31cd0 0x55682ea340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f580279f740 4 client.18402 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f580279f740 1 -- 172.21.15.18:0/1640225476 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f57f4005660 con 0x55682ea31cd0 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.587+0000 7f57fd88d640 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55682ea31cd0 0x55682ea340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f57fd88d640 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55682ea31cd0 0x55682ea340c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f1128c222d15f2f server_cookie=d268f5b588164f48 in_seq=0 out_seq=0 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f57e17fa640 10 client.18402 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f57e17fa640 1 -- 172.21.15.18:0/1640225476 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f57f4005660 con 0x55682ea31cd0 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f57e17fa640 10 client.18402 handle_command_reply: tid=0 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f580279f740 1 client.18402 shutdown 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f580279f740 1 -- 172.21.15.18:0/1640225476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 msgr2=0x55682ea30140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f580279f740 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 0x55682ea30140 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f57e8004000 tx=0x7f57e8012730 comp rx=0 tx=0).stop 2024-02-17T12:57:52.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f580279f740 1 -- 172.21.15.18:0/1640225476 shutdown_connections 2024-02-17T12:57:52.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f580279f740 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55682ea31cd0 0x55682ea340c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f580279f740 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682e9c9750 0x55682ea30140 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f580279f740 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55682e9c0b70 0x55682e9cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f57e17fa640 0 client.18402 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:57:52.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f580279f740 1 --2- 172.21.15.18:0/1640225476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55682e9c0250 0x55682e9cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f580279f740 1 -- 172.21.15.18:0/1640225476 >> 172.21.15.18:0/1640225476 conn(0x55682e9ba980 msgr2=0x55682e9c1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f580279f740 1 -- 172.21.15.18:0/1640225476 shutdown_connections 2024-02-17T12:57:52.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f580279f740 1 -- 172.21.15.18:0/1640225476 wait complete. 2024-02-17T12:57:52.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.588+0000 7f580279f740 20 client.18402 trim_cache size 0 max 16384 2024-02-17T12:57:52.587 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:57:52.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.589+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2975328073 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f57e40a2230 msgr2=0x7f57e40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.589+0000 7f57ffaf1640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f57e40a2230 0x7f57e40a46f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f57f00bd2d0 tx=0x7f57f00af040 comp rx=0 tx=0).stop 2024-02-17T12:57:52.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.589+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2975328073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f81975b0 msgr2=0x7f57f8197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.589+0000 7f57ffaf1640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f81975b0 0x7f57f8197a60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005e30 tx=0x7f57f0034cc0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.589+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2975328073 shutdown_connections 2024-02-17T12:57:52.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.589+0000 7f57ffaf1640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f57e40a2230 0x7f57e40a46f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.589+0000 7f57ffaf1640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f81975b0 0x7f57f8197a60 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.589+0000 7f57ffaf1640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57f8105330 0x7f57f819d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.589+0000 7f57ffaf1640 1 --2- 172.21.15.18:0/2975328073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f8100260 0x7f57f819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.589+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2975328073 >> 172.21.15.18:0/2975328073 conn(0x7f57f8077180 msgr2=0x7f57f8068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.589+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2975328073 shutdown_connections 2024-02-17T12:57:52.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.589+0000 7f57ffaf1640 1 -- 172.21.15.18:0/2975328073 wait complete. 2024-02-17T12:57:52.590 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:57:52.590 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:57:52.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.704+0000 7ffaea44e640 1 -- 172.21.15.18:0/2522499737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffae4100260 msgr2=0x7ffae4100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.704+0000 7ffaea44e640 1 --2- 172.21.15.18:0/2522499737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffae4100260 0x7ffae4100640 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009930 tx=0x7ffad402bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:52.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.704+0000 7ffaea44e640 1 -- 172.21.15.18:0/2522499737 shutdown_connections 2024-02-17T12:57:52.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.704+0000 7ffaea44e640 1 --2- 172.21.15.18:0/2522499737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffae4105330 0x7ffae410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.704+0000 7ffaea44e640 1 --2- 172.21.15.18:0/2522499737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffae4100c10 0x7ffae4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.704+0000 7ffaea44e640 1 --2- 172.21.15.18:0/2522499737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffae4100260 0x7ffae4100640 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.704+0000 7ffaea44e640 1 -- 172.21.15.18:0/2522499737 >> 172.21.15.18:0/2522499737 conn(0x7ffae4077180 msgr2=0x7ffae4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.704+0000 7ffaea44e640 1 -- 172.21.15.18:0/2522499737 shutdown_connections 2024-02-17T12:57:52.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.704+0000 7ffaea44e640 1 -- 172.21.15.18:0/2522499737 wait complete. 2024-02-17T12:57:52.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.705+0000 7ffaea44e640 1 Processor -- start 2024-02-17T12:57:52.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.705+0000 7ffaea44e640 1 -- start start 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.705+0000 7ffaea44e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffae4100260 0x7ffae419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.705+0000 7ffaea44e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffae4105330 0x7ffae419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.705+0000 7ffaea44e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffae4197600 0x7ffae4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.705+0000 7ffaea44e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae4197ff0 con 0x7ffae4197600 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.705+0000 7ffaea44e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae4198160 con 0x7ffae4100260 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.705+0000 7ffaea44e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae4198320 con 0x7ffae4105330 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.705+0000 7ffae89eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffae4197600 0x7ffae4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.705+0000 7ffae89eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffae4197600 0x7ffae4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35664/0 (socket says 172.21.15.18:35664) 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.705+0000 7ffae89eb640 1 -- 172.21.15.18:0/528185664 learned_addr learned my addr 172.21.15.18:0/528185664 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffae37fe640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffae4105330 0x7ffae419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffae3fff640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffae4100260 0x7ffae419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffae89eb640 1 -- 172.21.15.18:0/528185664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffae4105330 msgr2=0x7ffae419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffae89eb640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffae4105330 0x7ffae419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffae89eb640 1 -- 172.21.15.18:0/528185664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffae4100260 msgr2=0x7ffae419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffae89eb640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffae4100260 0x7ffae419d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffae89eb640 1 -- 172.21.15.18:0/528185664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad4009560 con 0x7ffae4197600 2024-02-17T12:57:52.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffae89eb640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffae4197600 0x7ffae4197ab0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009fd0 tx=0x7ffadc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffae17fa640 1 -- 172.21.15.18:0/528185664 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc00bea0 con 0x7ffae4197600 2024-02-17T12:57:52.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffae17fa640 1 -- 172.21.15.18:0/528185664 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffadc009940 con 0x7ffae4197600 2024-02-17T12:57:52.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffaea44e640 1 -- 172.21.15.18:0/528185664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae41a4310 con 0x7ffae4197600 2024-02-17T12:57:52.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffae17fa640 1 -- 172.21.15.18:0/528185664 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc0106c0 con 0x7ffae4197600 2024-02-17T12:57:52.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.706+0000 7ffaea44e640 1 -- 172.21.15.18:0/528185664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae41a47d0 con 0x7ffae4197600 2024-02-17T12:57:52.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.708+0000 7ffae17fa640 1 -- 172.21.15.18:0/528185664 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ffadc004240 con 0x7ffae4197600 2024-02-17T12:57:52.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.708+0000 7ffaed028740 1 Processor -- start 2024-02-17T12:57:52.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffaed028740 1 -- start start 2024-02-17T12:57:52.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae17fa640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffacc0a2230 0x7ffacc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffaed028740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a15736e250 0x55a15736e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffaed028740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 0x55a157377210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffaed028740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 0x55a157379b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffaed028740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a157329960 con 0x55a157377750 2024-02-17T12:57:52.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffaed028740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1572dd310 con 0x55a15736e250 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffaed028740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1572da840 con 0x55a15736eb70 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae17fa640 1 -- 172.21.15.18:0/528185664 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ffadc0cddc0 con 0x7ffae4197600 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae37fe640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffacc0a2230 0x7ffacc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae89eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 0x55a157379b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae89eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 0x55a157379b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35680/0 (socket says 172.21.15.18:35680) 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae89eb640 1 -- 172.21.15.18:0/1499532771 learned_addr learned my addr 172.21.15.18:0/1499532771 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae37fe640 1 --2- 172.21.15.18:0/1499532771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 0x55a157377210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae89eb640 1 -- 172.21.15.18:0/1499532771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 msgr2=0x55a157377210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae89eb640 1 --2- 172.21.15.18:0/1499532771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 0x55a157377210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae3fff640 1 --2- 172.21.15.18:0/1499532771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a15736e250 0x55a15736e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae89eb640 1 -- 172.21.15.18:0/1499532771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a15736e250 msgr2=0x55a15736e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae89eb640 1 --2- 172.21.15.18:0/1499532771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a15736e250 0x55a15736e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae89eb640 1 -- 172.21.15.18:0/1499532771 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a157298060 con 0x55a157377750 2024-02-17T12:57:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae3fff640 1 --2- 172.21.15.18:0/1499532771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a15736e250 0x55a15736e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae37fe640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffacc0a2230 0x7ffacc0a46f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffae40fff10 tx=0x7ffad800be70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.709+0000 7ffae89eb640 1 --2- 172.21.15.18:0/1499532771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 0x55a157379b40 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00dfd0 tx=0x7ffadc09a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e52fcef3b92008 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.710+0000 7ffaca7fc640 1 -- 172.21.15.18:0/1499532771 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc09e8c0 con 0x55a157377750 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.710+0000 7ffaca7fc640 1 -- 172.21.15.18:0/1499532771 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffadc0a2f10 con 0x55a157377750 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.710+0000 7ffaca7fc640 1 -- 172.21.15.18:0/1499532771 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc0cb920 con 0x55a157377750 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.710+0000 7ffaed028740 1 -- 172.21.15.18:0/1499532771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 msgr2=0x55a157379b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.710+0000 7ffaed028740 1 --2- 172.21.15.18:0/1499532771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 0x55a157379b40 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00dfd0 tx=0x7ffadc09a1e0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.710+0000 7ffaed028740 1 -- 172.21.15.18:0/1499532771 shutdown_connections 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.710+0000 7ffaed028740 1 --2- 172.21.15.18:0/1499532771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 0x55a157379b40 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.710+0000 7ffaed028740 1 --2- 172.21.15.18:0/1499532771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 0x55a157377210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.710+0000 7ffaed028740 1 --2- 172.21.15.18:0/1499532771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a15736e250 0x55a15736e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.710+0000 7ffaed028740 1 -- 172.21.15.18:0/1499532771 >> 172.21.15.18:0/1499532771 conn(0x55a157368980 msgr2=0x55a157368d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.710+0000 7ffaed028740 1 -- 172.21.15.18:0/1499532771 shutdown_connections 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.710+0000 7ffaed028740 1 -- 172.21.15.18:0/1499532771 wait complete. 2024-02-17T12:57:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffaed028740 1 -- start start 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffaed028740 1 Processor -- start 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffaed028740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736e250 0x55a157374810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffaed028740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736eb70 0x55a157374d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffaed028740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a157377750 0x55a157375290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffaed028740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1572d6020 con 0x55a15736eb70 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffaed028740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a156fa9430 con 0x55a157377750 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffaed028740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a156f65e00 con 0x55a15736e250 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffaed028740 10 client.0 _ll_register_callbacks cb 0x55a15727dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffae37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736eb70 0x55a157374d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffae37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736eb70 0x55a157374d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35694/0 (socket says 172.21.15.18:35694) 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffae37fe640 1 -- 172.21.15.18:0/1756342389 learned_addr learned my addr 172.21.15.18:0/1756342389 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffae3fff640 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736e250 0x55a157374810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffae89eb640 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a157377750 0x55a157375290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffae37fe640 1 -- 172.21.15.18:0/1756342389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736e250 msgr2=0x55a157374810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffae37fe640 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736e250 0x55a157374810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffae37fe640 1 -- 172.21.15.18:0/1756342389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a157377750 msgr2=0x55a157375290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffae37fe640 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a157377750 0x55a157375290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffae37fe640 1 -- 172.21.15.18:0/1756342389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae41a47d0 con 0x55a15736eb70 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.711+0000 7ffae89eb640 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a157377750 0x55a157375290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffae37fe640 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736eb70 0x55a157374d50 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ffad8012490 tx=0x7ffad8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c8feef2491bf00 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffac37fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffac37fe640 1 -- 172.21.15.18:0/1756342389 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8005220 con 0x55a15736eb70 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffac37fe640 1 -- 172.21.15.18:0/1756342389 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ffad8005380 con 0x55a15736eb70 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffaed028740 1 -- 172.21.15.18:0/1756342389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a156fd57e0 con 0x55a15736eb70 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffac37fe640 1 -- 172.21.15.18:0/1756342389 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffad8005540 con 0x55a15736eb70 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffaed028740 10 client.18414 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffaed028740 1 -- 172.21.15.18:0/1756342389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55a1572d46a0 con 0x55a15736eb70 2024-02-17T12:57:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffac37fe640 1 -- 172.21.15.18:0/1756342389 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a156fd57e0 con 0x55a15736eb70 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffac37fe640 1 -- 172.21.15.18:0/1756342389 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ffad8025030 con 0x55a15736eb70 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffaed028740 10 client.18414 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffaed028740 10 client.18414 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffaed028740 20 client.18414 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffaed028740 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a157387ad0 0x55a157389ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffaed028740 4 client.18414 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.712+0000 7ffaed028740 1 -- 172.21.15.18:0/1756342389 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a157012720 con 0x55a157387ad0 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.713+0000 7ffae3fff640 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a157387ad0 0x55a157389ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.713+0000 7ffac37fe640 10 client.18414 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.713+0000 7ffae3fff640 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a157387ad0 0x55a157389ef0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d62fcde0e02f123b server_cookie=14a5817da91afd9b in_seq=0 out_seq=0 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffac37fe640 1 -- 172.21.15.18:0/1756342389 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a157012720 con 0x55a157387ad0 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffac37fe640 10 client.18414 handle_command_reply: tid=0 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffaed028740 1 client.18414 shutdown 2024-02-17T12:57:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffaed028740 1 -- 172.21.15.18:0/1756342389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736eb70 msgr2=0x55a157374d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffaed028740 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736eb70 0x55a157374d50 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ffad8012490 tx=0x7ffad8004730 comp rx=0 tx=0).stop 2024-02-17T12:57:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffaed028740 1 -- 172.21.15.18:0/1756342389 shutdown_connections 2024-02-17T12:57:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffaed028740 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a157387ad0 0x55a157389ef0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffaed028740 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a157377750 0x55a157375290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffaed028740 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736eb70 0x55a157374d50 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffac37fe640 0 client.18414 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:57:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffaed028740 1 --2- 172.21.15.18:0/1756342389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736e250 0x55a157374810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffaed028740 1 -- 172.21.15.18:0/1756342389 >> 172.21.15.18:0/1756342389 conn(0x55a157368980 msgr2=0x55a15736a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffaed028740 1 -- 172.21.15.18:0/1756342389 shutdown_connections 2024-02-17T12:57:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffaed028740 1 -- 172.21.15.18:0/1756342389 wait complete. 2024-02-17T12:57:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.714+0000 7ffaed028740 20 client.18414 trim_cache size 0 max 16384 2024-02-17T12:57:52.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffaed028740 1 Processor -- start 2024-02-17T12:57:52.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffaed028740 1 -- start start 2024-02-17T12:57:52.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffaed028740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a15736e250 0x55a157376f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffaed028740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 0x55a1573750b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffaed028740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 0x55a157371ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffaed028740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15725b130 con 0x55a157377750 2024-02-17T12:57:52.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffaed028740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a15736def0 con 0x55a15736e250 2024-02-17T12:57:52.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffaed028740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a157369980 con 0x55a15736eb70 2024-02-17T12:57:52.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffae89eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 0x55a157371ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffae89eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 0x55a157371ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35700/0 (socket says 172.21.15.18:35700) 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffae89eb640 1 -- 172.21.15.18:0/2558742769 learned_addr learned my addr 172.21.15.18:0/2558742769 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffae89eb640 1 -- 172.21.15.18:0/2558742769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 msgr2=0x55a1573750b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffae89eb640 1 --2- 172.21.15.18:0/2558742769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 0x55a1573750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffae89eb640 1 -- 172.21.15.18:0/2558742769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a15736e250 msgr2=0x55a157376f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffae3fff640 1 --2- 172.21.15.18:0/2558742769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a15736e250 0x55a157376f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffae89eb640 1 --2- 172.21.15.18:0/2558742769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a15736e250 0x55a157376f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.726+0000 7ffae89eb640 1 -- 172.21.15.18:0/2558742769 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a157270c20 con 0x55a157377750 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffae89eb640 1 --2- 172.21.15.18:0/2558742769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 0x55a157371ab0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ffadc096bd0 tx=0x7ffadc0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16b64e2bdae15cc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffac9ffb640 1 -- 172.21.15.18:0/2558742769 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc09e8c0 con 0x55a157377750 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffac9ffb640 1 -- 172.21.15.18:0/2558742769 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffadc09a770 con 0x55a157377750 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffac9ffb640 1 -- 172.21.15.18:0/2558742769 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc099bc0 con 0x55a157377750 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffaed028740 1 -- 172.21.15.18:0/2558742769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 msgr2=0x55a157371ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffaed028740 1 --2- 172.21.15.18:0/2558742769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 0x55a157371ab0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ffadc096bd0 tx=0x7ffadc0092a0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffaed028740 1 -- 172.21.15.18:0/2558742769 shutdown_connections 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffaed028740 1 --2- 172.21.15.18:0/2558742769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a157377750 0x55a157371ab0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffaed028740 1 --2- 172.21.15.18:0/2558742769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 0x55a1573750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffaed028740 1 --2- 172.21.15.18:0/2558742769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a15736e250 0x55a157376f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffaed028740 1 -- 172.21.15.18:0/2558742769 >> 172.21.15.18:0/2558742769 conn(0x55a157368980 msgr2=0x55a157373100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffaed028740 1 -- 172.21.15.18:0/2558742769 shutdown_connections 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.727+0000 7ffaed028740 1 -- 172.21.15.18:0/2558742769 wait complete. 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffaed028740 1 -- start start 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffaed028740 1 Processor -- start 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffaed028740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736e250 0x55a15737d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffaed028740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 0x55a15737d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffaed028740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a157377750 0x55a15737db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffaed028740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1573de020 con 0x55a15736e250 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffaed028740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1573de190 con 0x55a157377750 2024-02-17T12:57:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffaed028740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1573de350 con 0x55a15736eb70 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffaed028740 10 client.0 _ll_register_callbacks cb 0x7ffadc096350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffae3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736e250 0x55a15737d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffae3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736e250 0x55a15737d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35710/0 (socket says 172.21.15.18:35710) 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffae3fff640 1 -- 172.21.15.18:0/3348192446 learned_addr learned my addr 172.21.15.18:0/3348192446 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffae3fff640 1 -- 172.21.15.18:0/3348192446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 msgr2=0x55a15737d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffae37fe640 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 0x55a15737d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffae3fff640 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 0x55a15737d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffae3fff640 1 -- 172.21.15.18:0/3348192446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a157377750 msgr2=0x55a15737db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffae3fff640 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a157377750 0x55a15737db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffae3fff640 1 -- 172.21.15.18:0/3348192446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a157298060 con 0x55a15736e250 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.728+0000 7ffae37fe640 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 0x55a15737d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffae3fff640 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736e250 0x55a15737d0a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009900 tx=0x7ffad4004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f8a50b0c571d00 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffac37fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffac37fe640 1 -- 172.21.15.18:0/3348192446 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad4004450 con 0x55a15736e250 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffac37fe640 1 -- 172.21.15.18:0/3348192446 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ffad402c7e0 con 0x55a15736e250 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffaed028740 1 -- 172.21.15.18:0/3348192446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffadc0a2f10 con 0x55a15736e250 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffac37fe640 1 -- 172.21.15.18:0/3348192446 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffad4004af0 con 0x55a15736e250 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffaed028740 10 client.18420 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffaed028740 1 -- 172.21.15.18:0/3348192446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55a1572968c0 con 0x55a15736e250 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffac37fe640 1 -- 172.21.15.18:0/3348192446 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffadc0a2f10 con 0x55a15736e250 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffac37fe640 1 -- 172.21.15.18:0/3348192446 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ffad403c040 con 0x55a15736e250 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffaed028740 10 client.18420 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffaed028740 10 client.18420 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffaed028740 20 client.18420 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffaed028740 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a1573dfb50 0x55a1573e1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffaed028740 4 client.18420 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.729+0000 7ffaed028740 1 -- 172.21.15.18:0/3348192446 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a157061890 con 0x55a1573dfb50 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffae37fe640 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a1573dfb50 0x55a1573e1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffac37fe640 10 client.18420 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffae37fe640 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a1573dfb50 0x55a1573e1f40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df23f1c2651ddf19 server_cookie=53f72d6a41b1fd03 in_seq=0 out_seq=0 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffac37fe640 1 -- 172.21.15.18:0/3348192446 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a157061890 con 0x55a1573dfb50 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffac37fe640 10 client.18420 handle_command_reply: tid=0 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffaed028740 1 client.18420 shutdown 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffaed028740 1 -- 172.21.15.18:0/3348192446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736e250 msgr2=0x55a15737d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffaed028740 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736e250 0x55a15737d0a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009900 tx=0x7ffad4004980 comp rx=0 tx=0).stop 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffaed028740 1 -- 172.21.15.18:0/3348192446 shutdown_connections 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffaed028740 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a1573dfb50 0x55a1573e1f40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffaed028740 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a157377750 0x55a15737db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffac37fe640 0 client.18420 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffaed028740 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a15736eb70 0x55a15737d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffaed028740 1 --2- 172.21.15.18:0/3348192446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a15736e250 0x55a15737d0a0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffaed028740 1 -- 172.21.15.18:0/3348192446 >> 172.21.15.18:0/3348192446 conn(0x55a157368980 msgr2=0x55a1573672f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffaed028740 1 -- 172.21.15.18:0/3348192446 shutdown_connections 2024-02-17T12:57:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffaed028740 1 -- 172.21.15.18:0/3348192446 wait complete. 2024-02-17T12:57:52.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.730+0000 7ffaed028740 20 client.18420 trim_cache size 0 max 16384 2024-02-17T12:57:52.728 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:57:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.731+0000 7ffaea44e640 1 -- 172.21.15.18:0/528185664 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffacc0a2230 msgr2=0x7ffacc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.731+0000 7ffaea44e640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffacc0a2230 0x7ffacc0a46f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffae40fff10 tx=0x7ffad800be70 comp rx=0 tx=0).stop 2024-02-17T12:57:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.731+0000 7ffaea44e640 1 -- 172.21.15.18:0/528185664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffae4197600 msgr2=0x7ffae4197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.731+0000 7ffaea44e640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffae4197600 0x7ffae4197ab0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009fd0 tx=0x7ffadc00b260 comp rx=0 tx=0).stop 2024-02-17T12:57:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.731+0000 7ffaea44e640 1 -- 172.21.15.18:0/528185664 shutdown_connections 2024-02-17T12:57:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.731+0000 7ffaea44e640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffacc0a2230 0x7ffacc0a46f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.731+0000 7ffaea44e640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffae4197600 0x7ffae4197ab0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.731+0000 7ffaea44e640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffae4105330 0x7ffae419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.731+0000 7ffaea44e640 1 --2- 172.21.15.18:0/528185664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffae4100260 0x7ffae419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.731+0000 7ffaea44e640 1 -- 172.21.15.18:0/528185664 >> 172.21.15.18:0/528185664 conn(0x7ffae4077180 msgr2=0x7ffae4068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.731+0000 7ffaea44e640 1 -- 172.21.15.18:0/528185664 shutdown_connections 2024-02-17T12:57:52.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.731+0000 7ffaea44e640 1 -- 172.21.15.18:0/528185664 wait complete. 2024-02-17T12:57:52.732 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:57:52.732 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:57:52.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.847+0000 7f2013881640 1 -- 172.21.15.18:0/1657273842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f200c1002a0 msgr2=0x7f200c100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.847+0000 7f2013881640 1 --2- 172.21.15.18:0/1657273842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f200c1002a0 0x7f200c100680 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2008009930 tx=0x7f200802bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:52.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.847+0000 7f2013881640 1 -- 172.21.15.18:0/1657273842 shutdown_connections 2024-02-17T12:57:52.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.847+0000 7f2013881640 1 --2- 172.21.15.18:0/1657273842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f200c105370 0x7f200c10cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.847+0000 7f2013881640 1 --2- 172.21.15.18:0/1657273842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f200c100c50 0x7f200c104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.847+0000 7f2013881640 1 --2- 172.21.15.18:0/1657273842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f200c1002a0 0x7f200c100680 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.847+0000 7f2013881640 1 -- 172.21.15.18:0/1657273842 >> 172.21.15.18:0/1657273842 conn(0x7f200c076400 msgr2=0x7f200c076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.847+0000 7f2013881640 1 -- 172.21.15.18:0/1657273842 shutdown_connections 2024-02-17T12:57:52.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.847+0000 7f2013881640 1 -- 172.21.15.18:0/1657273842 wait complete. 2024-02-17T12:57:52.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2013881640 1 Processor -- start 2024-02-17T12:57:52.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2013881640 1 -- start start 2024-02-17T12:57:52.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2013881640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f200c1002a0 0x7f200c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2013881640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f200c105370 0x7f200c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2013881640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f200c197660 0x7f200c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2013881640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c198130 con 0x7f200c197660 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2013881640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c1982a0 con 0x7f200c1002a0 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2013881640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c198460 con 0x7f200c105370 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2011e1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f200c197660 0x7f200c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2011e1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f200c197660 0x7f200c197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35738/0 (socket says 172.21.15.18:35738) 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2011e1e640 1 -- 172.21.15.18:0/1800879683 learned_addr learned my addr 172.21.15.18:0/1800879683 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f201161d640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f200c1002a0 0x7f200c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2010e1c640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f200c105370 0x7f200c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2011e1e640 1 -- 172.21.15.18:0/1800879683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f200c105370 msgr2=0x7f200c19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2011e1e640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f200c105370 0x7f200c19dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2011e1e640 1 -- 172.21.15.18:0/1800879683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f200c1002a0 msgr2=0x7f200c19d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2011e1e640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f200c1002a0 0x7f200c19d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f2011e1e640 1 -- 172.21.15.18:0/1800879683 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2008009560 con 0x7f200c197660 2024-02-17T12:57:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.848+0000 7f201161d640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f200c1002a0 0x7f200c19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:52.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.849+0000 7f2011e1e640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f200c197660 0x7f200c197ae0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f200400b970 tx=0x7f200400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.849+0000 7f20027fc640 1 -- 172.21.15.18:0/1800879683 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200400cb40 con 0x7f200c197660 2024-02-17T12:57:52.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.849+0000 7f20027fc640 1 -- 172.21.15.18:0/1800879683 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f200400cca0 con 0x7f200c197660 2024-02-17T12:57:52.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.849+0000 7f2013881640 1 -- 172.21.15.18:0/1800879683 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200c1a4400 con 0x7f200c197660 2024-02-17T12:57:52.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.849+0000 7f20027fc640 1 -- 172.21.15.18:0/1800879683 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20040077d0 con 0x7f200c197660 2024-02-17T12:57:52.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.849+0000 7f2013881640 1 -- 172.21.15.18:0/1800879683 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200c1a48e0 con 0x7f200c197660 2024-02-17T12:57:52.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.851+0000 7f20027fc640 1 -- 172.21.15.18:0/1800879683 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2004014440 con 0x7f200c197660 2024-02-17T12:57:52.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2016453740 1 Processor -- start 2024-02-17T12:57:52.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2016453740 1 -- start start 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f20027fc640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1ff80a22b0 0x7f1ff80a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2016453740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8b250 0x5585c5f8b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2016453740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8bb70 0x5585c5f94210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2016453740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 0x5585c5f96b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2016453740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c5f46960 con 0x5585c5f94750 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2016453740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c5efa310 con 0x5585c5f8b250 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2016453740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c5ef7840 con 0x5585c5f8bb70 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2011e1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 0x5585c5f96b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2010e1c640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1ff80a22b0 0x7f1ff80a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2011e1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 0x5585c5f96b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35748/0 (socket says 172.21.15.18:35748) 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2011e1e640 1 -- 172.21.15.18:0/1233278608 learned_addr learned my addr 172.21.15.18:0/1233278608 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f20027fc640 1 -- 172.21.15.18:0/1800879683 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f20040cd790 con 0x7f200c197660 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f201161d640 1 --2- 172.21.15.18:0/1233278608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8b250 0x5585c5f8b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2010e1c640 1 --2- 172.21.15.18:0/1233278608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8bb70 0x5585c5f94210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2011e1e640 1 -- 172.21.15.18:0/1233278608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8bb70 msgr2=0x5585c5f94210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2011e1e640 1 --2- 172.21.15.18:0/1233278608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8bb70 0x5585c5f94210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2011e1e640 1 -- 172.21.15.18:0/1233278608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8b250 msgr2=0x5585c5f8b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2011e1e640 1 --2- 172.21.15.18:0/1233278608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8b250 0x5585c5f8b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2011e1e640 1 -- 172.21.15.18:0/1233278608 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585c5eb5060 con 0x5585c5f94750 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.852+0000 7f2010e1c640 1 --2- 172.21.15.18:0/1233278608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8bb70 0x5585c5f94210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.853+0000 7f2011e1e640 1 --2- 172.21.15.18:0/1233278608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 0x5585c5f96b40 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f20040967c0 tx=0x7f2004014ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69fba8746a2ae4f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.853+0000 7f2010e1c640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1ff80a22b0 0x7f1ff80a4770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5585c5f8ad20 tx=0x7f1ffc0041e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.853+0000 7f1ff77fe640 1 -- 172.21.15.18:0/1233278608 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004022170 con 0x5585c5f94750 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.853+0000 7f1ff77fe640 1 -- 172.21.15.18:0/1233278608 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f200409a360 con 0x5585c5f94750 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.853+0000 7f1ff77fe640 1 -- 172.21.15.18:0/1233278608 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004098660 con 0x5585c5f94750 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 -- 172.21.15.18:0/1233278608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 msgr2=0x5585c5f96b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 --2- 172.21.15.18:0/1233278608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 0x5585c5f96b40 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f20040967c0 tx=0x7f2004014ce0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 -- 172.21.15.18:0/1233278608 shutdown_connections 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 --2- 172.21.15.18:0/1233278608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 0x5585c5f96b40 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 --2- 172.21.15.18:0/1233278608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8bb70 0x5585c5f94210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 --2- 172.21.15.18:0/1233278608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8b250 0x5585c5f8b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 -- 172.21.15.18:0/1233278608 >> 172.21.15.18:0/1233278608 conn(0x5585c5f85980 msgr2=0x5585c5f85d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 -- 172.21.15.18:0/1233278608 shutdown_connections 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 -- 172.21.15.18:0/1233278608 wait complete. 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 -- start start 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 Processor -- start 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 0x5585c5f91800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8bb70 0x5585c5f91d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f94750 0x5585c5f92280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c5ef3020 con 0x5585c5f8b250 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c5bc6430 con 0x5585c5f8bb70 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c5b82e00 con 0x5585c5f94750 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.854+0000 7f2016453740 10 client.0 _ll_register_callbacks cb 0x5585c5e9acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f201161d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 0x5585c5f91800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f201161d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 0x5585c5f91800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35750/0 (socket says 172.21.15.18:35750) 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f201161d640 1 -- 172.21.15.18:0/3050393471 learned_addr learned my addr 172.21.15.18:0/3050393471 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f2010e1c640 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8bb70 0x5585c5f91d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f2011e1e640 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f94750 0x5585c5f92280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f201161d640 1 -- 172.21.15.18:0/3050393471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f94750 msgr2=0x5585c5f92280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f201161d640 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f94750 0x5585c5f92280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f201161d640 1 -- 172.21.15.18:0/3050393471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8bb70 msgr2=0x5585c5f91d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f201161d640 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8bb70 0x5585c5f91d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f201161d640 1 -- 172.21.15.18:0/3050393471 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200c1a48e0 con 0x5585c5f8b250 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f201161d640 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 0x5585c5f91800 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f2008006010 tx=0x7f2008004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a918e90ea0788455 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f1ff4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f1ff4ff9640 1 -- 172.21.15.18:0/3050393471 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080044a0 con 0x5585c5f8b250 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f1ff4ff9640 1 -- 172.21.15.18:0/3050393471 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2008005000 con 0x5585c5f8b250 2024-02-17T12:57:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f2016453740 1 -- 172.21.15.18:0/3050393471 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585c5bf27e0 con 0x5585c5f8b250 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f1ff4ff9640 1 -- 172.21.15.18:0/3050393471 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20080051c0 con 0x5585c5f8b250 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f2016453740 10 client.18432 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.855+0000 7f2016453740 1 -- 172.21.15.18:0/3050393471 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5585c5ef16a0 con 0x5585c5f8b250 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.856+0000 7f1ff4ff9640 1 -- 172.21.15.18:0/3050393471 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5585c5bf27e0 con 0x5585c5f8b250 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.856+0000 7f1ff4ff9640 1 -- 172.21.15.18:0/3050393471 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f200803c040 con 0x5585c5f8b250 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.856+0000 7f2016453740 10 client.18432 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.856+0000 7f2016453740 10 client.18432 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.856+0000 7f2016453740 20 client.18432 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.856+0000 7f2016453740 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5585c5fa4b20 0x5585c5fa6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.856+0000 7f2016453740 4 client.18432 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.856+0000 7f2016453740 1 -- 172.21.15.18:0/3050393471 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585c5c2f720 con 0x5585c5fa4b20 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.856+0000 7f2010e1c640 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5585c5fa4b20 0x5585c5fa6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.856+0000 7f1ff4ff9640 10 client.18432 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:57:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.856+0000 7f2010e1c640 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5585c5fa4b20 0x5585c5fa6f60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23c4a8bbf931a255 server_cookie=8dcee47edcbd66b7 in_seq=0 out_seq=0 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f1ff4ff9640 1 -- 172.21.15.18:0/3050393471 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5585c5c2f720 con 0x5585c5fa4b20 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f1ff4ff9640 10 client.18432 handle_command_reply: tid=0 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f2016453740 1 client.18432 shutdown 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f2016453740 1 -- 172.21.15.18:0/3050393471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 msgr2=0x5585c5f91800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f2016453740 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 0x5585c5f91800 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f2008006010 tx=0x7f2008004270 comp rx=0 tx=0).stop 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f2016453740 1 -- 172.21.15.18:0/3050393471 shutdown_connections 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f2016453740 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5585c5fa4b20 0x5585c5fa6f60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f2016453740 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f94750 0x5585c5f92280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f2016453740 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8bb70 0x5585c5f91d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f1ff4ff9640 0 client.18432 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f2016453740 1 --2- 172.21.15.18:0/3050393471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 0x5585c5f91800 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f2016453740 1 -- 172.21.15.18:0/3050393471 >> 172.21.15.18:0/3050393471 conn(0x5585c5f85980 msgr2=0x5585c5f876d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f2016453740 1 -- 172.21.15.18:0/3050393471 shutdown_connections 2024-02-17T12:57:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f2016453740 1 -- 172.21.15.18:0/3050393471 wait complete. 2024-02-17T12:57:52.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.857+0000 7f2016453740 20 client.18432 trim_cache size 0 max 16384 2024-02-17T12:57:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f2016453740 1 Processor -- start 2024-02-17T12:57:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f2016453740 1 -- start start 2024-02-17T12:57:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f2016453740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 0x5585c5f93f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f2016453740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8bb70 0x5585c5f920c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f2016453740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f94750 0x5585c5f8eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f2016453740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c5e78130 con 0x5585c5f8b250 2024-02-17T12:57:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f2016453740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c5f86780 con 0x5585c5f94750 2024-02-17T12:57:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f2016453740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c5f8b9d0 con 0x5585c5f8bb70 2024-02-17T12:57:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f201161d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 0x5585c5f93f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f201161d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 0x5585c5f93f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35756/0 (socket says 172.21.15.18:35756) 2024-02-17T12:57:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f201161d640 1 -- 172.21.15.18:0/450466437 learned_addr learned my addr 172.21.15.18:0/450466437 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f2010e1c640 1 --2- 172.21.15.18:0/450466437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8bb70 0x5585c5f920c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f2011e1e640 1 --2- 172.21.15.18:0/450466437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f94750 0x5585c5f8eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f201161d640 1 -- 172.21.15.18:0/450466437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8bb70 msgr2=0x5585c5f920c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f201161d640 1 --2- 172.21.15.18:0/450466437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8bb70 0x5585c5f920c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f201161d640 1 -- 172.21.15.18:0/450466437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f94750 msgr2=0x5585c5f8eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.871+0000 7f201161d640 1 --2- 172.21.15.18:0/450466437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f94750 0x5585c5f8eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.872+0000 7f201161d640 1 -- 172.21.15.18:0/450466437 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585c5e8dc20 con 0x5585c5f8b250 2024-02-17T12:57:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.872+0000 7f201161d640 1 --2- 172.21.15.18:0/450466437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 0x5585c5f93f30 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f200802c170 tx=0x7f2008005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7365589e37a677ee server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.874+0000 7f1ff6ffd640 1 -- 172.21.15.18:0/450466437 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080535e0 con 0x5585c5f8b250 2024-02-17T12:57:52.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.874+0000 7f1ff6ffd640 1 -- 172.21.15.18:0/450466437 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20080044a0 con 0x5585c5f8b250 2024-02-17T12:57:52.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.874+0000 7f1ff6ffd640 1 -- 172.21.15.18:0/450466437 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008003b90 con 0x5585c5f8b250 2024-02-17T12:57:52.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.875+0000 7f2016453740 1 -- 172.21.15.18:0/450466437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 msgr2=0x5585c5f93f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.875+0000 7f2016453740 1 --2- 172.21.15.18:0/450466437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 0x5585c5f93f30 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f200802c170 tx=0x7f2008005a20 comp rx=0 tx=0).stop 2024-02-17T12:57:52.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.875+0000 7f2016453740 1 -- 172.21.15.18:0/450466437 shutdown_connections 2024-02-17T12:57:52.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.875+0000 7f2016453740 1 --2- 172.21.15.18:0/450466437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f94750 0x5585c5f8eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.875+0000 7f2016453740 1 --2- 172.21.15.18:0/450466437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8bb70 0x5585c5f920c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.875+0000 7f2016453740 1 --2- 172.21.15.18:0/450466437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f8b250 0x5585c5f93f30 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.875+0000 7f2016453740 1 -- 172.21.15.18:0/450466437 >> 172.21.15.18:0/450466437 conn(0x5585c5f85980 msgr2=0x5585c5f90130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.875+0000 7f2016453740 1 -- 172.21.15.18:0/450466437 shutdown_connections 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.875+0000 7f2016453740 1 -- 172.21.15.18:0/450466437 wait complete. 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.875+0000 7f2016453740 1 -- start start 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.875+0000 7f2016453740 1 Processor -- start 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2016453740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8b250 0x5585c5f9bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2016453740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8bb70 0x5585c5f99e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2016453740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 0x5585c5f9a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2016453740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c5f9ac60 con 0x5585c5f94750 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2016453740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c5ffb090 con 0x5585c5f8bb70 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2016453740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c5ffb1d0 con 0x5585c5f8b250 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2016453740 10 client.0 _ll_register_callbacks cb 0x7f1ffc00b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2011e1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 0x5585c5f9a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2011e1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 0x5585c5f9a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35760/0 (socket says 172.21.15.18:35760) 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2011e1e640 1 -- 172.21.15.18:0/1575827173 learned_addr learned my addr 172.21.15.18:0/1575827173 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f201161d640 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8b250 0x5585c5f9bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2010e1c640 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8bb70 0x5585c5f99e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2011e1e640 1 -- 172.21.15.18:0/1575827173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8b250 msgr2=0x5585c5f9bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2011e1e640 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8b250 0x5585c5f9bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2011e1e640 1 -- 172.21.15.18:0/1575827173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8bb70 msgr2=0x5585c5f99e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2011e1e640 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8bb70 0x5585c5f99e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2011e1e640 1 -- 172.21.15.18:0/1575827173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585c5ef16a0 con 0x5585c5f94750 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2011e1e640 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 0x5585c5f9a560 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f200409a2f0 tx=0x7f200409e400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47c3820dca434ed server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f1ff4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f1ff4ff9640 1 -- 172.21.15.18:0/1575827173 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200400ce10 con 0x5585c5f94750 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f1ff4ff9640 1 -- 172.21.15.18:0/1575827173 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f20040a2a50 con 0x5585c5f94750 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.876+0000 7f2016453740 1 -- 172.21.15.18:0/1575827173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f200409a360 con 0x5585c5f94750 2024-02-17T12:57:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.877+0000 7f1ff4ff9640 1 -- 172.21.15.18:0/1575827173 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20040bf3f0 con 0x5585c5f94750 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.877+0000 7f2016453740 10 client.18438 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.877+0000 7f2016453740 1 -- 172.21.15.18:0/1575827173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5585c5eb38c0 con 0x5585c5f94750 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.877+0000 7f1ff4ff9640 1 -- 172.21.15.18:0/1575827173 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f200409a360 con 0x5585c5f94750 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.877+0000 7f1ff4ff9640 1 -- 172.21.15.18:0/1575827173 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f20040b8020 con 0x5585c5f94750 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.877+0000 7f2016453740 10 client.18438 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.877+0000 7f2016453740 10 client.18438 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.877+0000 7f2016453740 20 client.18438 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.877+0000 7f2016453740 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5585c5ffc660 0x5585c5ffea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.877+0000 7f2016453740 4 client.18438 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.877+0000 7f2016453740 1 -- 172.21.15.18:0/1575827173 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5585c5c7e890 con 0x5585c5ffc660 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.877+0000 7f201161d640 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5585c5ffc660 0x5585c5ffea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f1ff4ff9640 10 client.18438 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f201161d640 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5585c5ffc660 0x5585c5ffea80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3db0b126bfbaecda server_cookie=aa578945088eb8dd in_seq=0 out_seq=0 2024-02-17T12:57:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f1ff4ff9640 1 -- 172.21.15.18:0/1575827173 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5585c5c7e890 con 0x5585c5ffc660 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f1ff4ff9640 10 client.18438 handle_command_reply: tid=0 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f2016453740 1 client.18438 shutdown 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f2016453740 1 -- 172.21.15.18:0/1575827173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 msgr2=0x5585c5f9a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f2016453740 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 0x5585c5f9a560 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f200409a2f0 tx=0x7f200409e400 comp rx=0 tx=0).stop 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f2016453740 1 -- 172.21.15.18:0/1575827173 shutdown_connections 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f2016453740 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5585c5ffc660 0x5585c5ffea80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f2016453740 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585c5f94750 0x5585c5f9a560 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f2016453740 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585c5f8bb70 0x5585c5f99e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f1ff4ff9640 0 client.18438 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f2016453740 1 --2- 172.21.15.18:0/1575827173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c5f8b250 0x5585c5f9bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f2016453740 1 -- 172.21.15.18:0/1575827173 >> 172.21.15.18:0/1575827173 conn(0x5585c5f85980 msgr2=0x5585c5f84b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f2016453740 1 -- 172.21.15.18:0/1575827173 shutdown_connections 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f2016453740 1 -- 172.21.15.18:0/1575827173 wait complete. 2024-02-17T12:57:52.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.878+0000 7f2016453740 20 client.18438 trim_cache size 0 max 16384 2024-02-17T12:57:52.874 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:57:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.879+0000 7f2013881640 1 -- 172.21.15.18:0/1800879683 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1ff80a22b0 msgr2=0x7f1ff80a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.879+0000 7f2013881640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1ff80a22b0 0x7f1ff80a4770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5585c5f8ad20 tx=0x7f1ffc0041e0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.879+0000 7f2013881640 1 -- 172.21.15.18:0/1800879683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f200c197660 msgr2=0x7f200c197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.879+0000 7f2013881640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f200c197660 0x7f200c197ae0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f200400b970 tx=0x7f200400be30 comp rx=0 tx=0).stop 2024-02-17T12:57:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.879+0000 7f2013881640 1 -- 172.21.15.18:0/1800879683 shutdown_connections 2024-02-17T12:57:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.879+0000 7f2013881640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1ff80a22b0 0x7f1ff80a4770 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.879+0000 7f2013881640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f200c197660 0x7f200c197ae0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.879+0000 7f2013881640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f200c105370 0x7f200c19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.879+0000 7f2013881640 1 --2- 172.21.15.18:0/1800879683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f200c1002a0 0x7f200c19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.879+0000 7f2013881640 1 -- 172.21.15.18:0/1800879683 >> 172.21.15.18:0/1800879683 conn(0x7f200c076400 msgr2=0x7f200c069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.879+0000 7f2013881640 1 -- 172.21.15.18:0/1800879683 shutdown_connections 2024-02-17T12:57:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.879+0000 7f2013881640 1 -- 172.21.15.18:0/1800879683 wait complete. 2024-02-17T12:57:52.879 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:57:52.880 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:57:52.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.994+0000 7fa16a33a640 1 -- 172.21.15.18:0/160069627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1640ff8f0 msgr2=0x7fa1640ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.994+0000 7fa16a33a640 1 --2- 172.21.15.18:0/160069627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1640ff8f0 0x7fa1640ffcd0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009930 tx=0x7fa14c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:57:52.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.994+0000 7fa16a33a640 1 -- 172.21.15.18:0/160069627 shutdown_connections 2024-02-17T12:57:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.994+0000 7fa16a33a640 1 --2- 172.21.15.18:0/160069627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa164106500 0x7fa16410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.994+0000 7fa16a33a640 1 --2- 172.21.15.18:0/160069627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1641002a0 0x7fa164105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.994+0000 7fa16a33a640 1 --2- 172.21.15.18:0/160069627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1640ff8f0 0x7fa1640ffcd0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.994+0000 7fa16a33a640 1 -- 172.21.15.18:0/160069627 >> 172.21.15.18:0/160069627 conn(0x7fa164076640 msgr2=0x7fa1640fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.994+0000 7fa16a33a640 1 -- 172.21.15.18:0/160069627 shutdown_connections 2024-02-17T12:57:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.994+0000 7fa16a33a640 1 -- 172.21.15.18:0/160069627 wait complete. 2024-02-17T12:57:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.994+0000 7fa16a33a640 1 Processor -- start 2024-02-17T12:57:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.994+0000 7fa16a33a640 1 -- start start 2024-02-17T12:57:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa16a33a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1640ff8f0 0x7fa16419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa16a33a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa164106500 0x7fa16419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa16a33a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa164197650 0x7fa164197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa16a33a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa164198040 con 0x7fa164197650 2024-02-17T12:57:52.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa16a33a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641981b0 con 0x7fa1640ff8f0 2024-02-17T12:57:52.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa16a33a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa164198370 con 0x7fa164106500 2024-02-17T12:57:52.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1688d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa164197650 0x7fa164197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1688d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa164197650 0x7fa164197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35784/0 (socket says 172.21.15.18:35784) 2024-02-17T12:57:52.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1688d7640 1 -- 172.21.15.18:0/3327186315 learned_addr learned my addr 172.21.15.18:0/3327186315 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa163fff640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1640ff8f0 0x7fa16419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1637fe640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa164106500 0x7fa16419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1688d7640 1 -- 172.21.15.18:0/3327186315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa164106500 msgr2=0x7fa16419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1688d7640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa164106500 0x7fa16419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1688d7640 1 -- 172.21.15.18:0/3327186315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1640ff8f0 msgr2=0x7fa16419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:52.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1688d7640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1640ff8f0 0x7fa16419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1688d7640 1 -- 172.21.15.18:0/3327186315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c009560 con 0x7fa164197650 2024-02-17T12:57:52.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1688d7640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa164197650 0x7fa164197b00 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00cce0 tx=0x7fa15c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:52.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1617fa640 1 -- 172.21.15.18:0/3327186315 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00f450 con 0x7fa164197650 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1617fa640 1 -- 172.21.15.18:0/3327186315 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa15c00fa90 con 0x7fa164197650 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa16a33a640 1 -- 172.21.15.18:0/3327186315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1641a4310 con 0x7fa164197650 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa1617fa640 1 -- 172.21.15.18:0/3327186315 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c018b90 con 0x7fa164197650 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.995+0000 7fa16a33a640 1 -- 172.21.15.18:0/3327186315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1641a47d0 con 0x7fa164197650 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.997+0000 7fa1617fa640 1 -- 172.21.15.18:0/3327186315 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa15c00fc00 con 0x7fa164197650 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa16cfe5740 1 Processor -- start 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa16cfe5740 1 -- start start 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa16cfe5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba78c250 0x55a3ba78c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa16cfe5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 0x55a3ba795210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa16cfe5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba795750 0x55a3ba797b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa16cfe5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ba747960 con 0x55a3ba78cb70 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa1617fa640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa1500a2230 0x7fa1500a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa16cfe5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ba6fb310 con 0x55a3ba78c250 2024-02-17T12:57:52.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa16cfe5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ba6f8840 con 0x55a3ba795750 2024-02-17T12:57:52.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa163fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 0x55a3ba795210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa163fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 0x55a3ba795210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35800/0 (socket says 172.21.15.18:35800) 2024-02-17T12:57:52.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa1688d7640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa1500a2230 0x7fa1500a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa163fff640 1 -- 172.21.15.18:0/3940967617 learned_addr learned my addr 172.21.15.18:0/3940967617 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:52.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa1637fe640 1 --2- 172.21.15.18:0/3940967617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba78c250 0x55a3ba78c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa1617fa640 1 -- 172.21.15.18:0/3327186315 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa15c0ce7f0 con 0x7fa164197650 2024-02-17T12:57:52.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa1637fe640 1 --2- 172.21.15.18:0/3940967617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba795750 0x55a3ba797b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:52.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa163fff640 1 -- 172.21.15.18:0/3940967617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba795750 msgr2=0x55a3ba797b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:52.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa163fff640 1 --2- 172.21.15.18:0/3940967617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba795750 0x55a3ba797b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:52.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa163fff640 1 -- 172.21.15.18:0/3940967617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba78c250 msgr2=0x55a3ba78c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.998+0000 7fa163fff640 1 --2- 172.21.15.18:0/3940967617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba78c250 0x55a3ba78c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.999+0000 7fa163fff640 1 -- 172.21.15.18:0/3940967617 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3ba6b6060 con 0x55a3ba78cb70 2024-02-17T12:57:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.999+0000 7fa1688d7640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa1500a2230 0x7fa1500a46f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0cc010 tx=0x7fa15c0908d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.999+0000 7fa163fff640 1 --2- 172.21.15.18:0/3940967617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 0x55a3ba795210 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fa14c036040 tx=0x7fa14c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a442988592b430 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.999+0000 7fa15a7fc640 1 -- 172.21.15.18:0/3940967617 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c0048a0 con 0x55a3ba78cb70 2024-02-17T12:57:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.999+0000 7fa15a7fc640 1 -- 172.21.15.18:0/3940967617 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa14c004a00 con 0x55a3ba78cb70 2024-02-17T12:57:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:52.999+0000 7fa15a7fc640 1 -- 172.21.15.18:0/3940967617 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c0050e0 con 0x55a3ba78cb70 2024-02-17T12:57:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.000+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3940967617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 msgr2=0x55a3ba795210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.000+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/3940967617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 0x55a3ba795210 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fa14c036040 tx=0x7fa14c004770 comp rx=0 tx=0).stop 2024-02-17T12:57:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.000+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3940967617 shutdown_connections 2024-02-17T12:57:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.000+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/3940967617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba795750 0x55a3ba797b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.000+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/3940967617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 0x55a3ba795210 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.000+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/3940967617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba78c250 0x55a3ba78c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.000+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3940967617 >> 172.21.15.18:0/3940967617 conn(0x55a3ba786980 msgr2=0x55a3ba786d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.000+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3940967617 shutdown_connections 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.000+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3940967617 wait complete. 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.000+0000 7fa16cfe5740 1 -- start start 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa16cfe5740 1 Processor -- start 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa16cfe5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 0x55a3ba795190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa16cfe5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba78cb70 0x55a3ba792780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa16cfe5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba795750 0x55a3ba792cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa16cfe5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ba6f4020 con 0x55a3ba78c250 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa16cfe5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ba3c7430 con 0x55a3ba795750 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa16cfe5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ba383e00 con 0x55a3ba78cb70 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa1637fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 0x55a3ba795190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa1637fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 0x55a3ba795190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35814/0 (socket says 172.21.15.18:35814) 2024-02-17T12:57:53.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa1637fe640 1 -- 172.21.15.18:0/3108319555 learned_addr learned my addr 172.21.15.18:0/3108319555 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa16cfe5740 10 client.0 _ll_register_callbacks cb 0x55a3ba69bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa163fff640 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba78cb70 0x55a3ba792780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.001+0000 7fa1637fe640 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba795750 0x55a3ba792cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.002+0000 7fa1637fe640 1 -- 172.21.15.18:0/3108319555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba78cb70 msgr2=0x55a3ba792780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.002+0000 7fa1637fe640 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba78cb70 0x55a3ba792780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.002+0000 7fa1637fe640 1 -- 172.21.15.18:0/3108319555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba795750 msgr2=0x55a3ba792cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.002+0000 7fa1637fe640 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba795750 0x55a3ba792cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.002+0000 7fa1637fe640 1 -- 172.21.15.18:0/3108319555 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16410e890 con 0x55a3ba78c250 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.002+0000 7fa1637fe640 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 0x55a3ba795190 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa1640690e0 tx=0x7fa154016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78034795de9636b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.002+0000 7fa1437fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.002+0000 7fa1437fe640 1 -- 172.21.15.18:0/3108319555 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154012ce0 con 0x55a3ba78c250 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.002+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3108319555 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3ba3f37e0 con 0x55a3ba78c250 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.002+0000 7fa1437fe640 1 -- 172.21.15.18:0/3108319555 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa154012e40 con 0x55a3ba78c250 2024-02-17T12:57:53.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.002+0000 7fa1437fe640 1 -- 172.21.15.18:0/3108319555 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa15400f440 con 0x55a3ba78c250 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.003+0000 7fa16cfe5740 10 client.18450 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.003+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3108319555 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55a3ba6f26a0 con 0x55a3ba78c250 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.003+0000 7fa1437fe640 1 -- 172.21.15.18:0/3108319555 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a3ba3f37e0 con 0x55a3ba78c250 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.003+0000 7fa1437fe640 1 -- 172.21.15.18:0/3108319555 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fa154011600 con 0x55a3ba78c250 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.003+0000 7fa16cfe5740 10 client.18450 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.003+0000 7fa16cfe5740 10 client.18450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.003+0000 7fa16cfe5740 20 client.18450 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.003+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a3ba7ae2c0 0x55a3ba7b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.003+0000 7fa16cfe5740 4 client.18450 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.003+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3108319555 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3ba430720 con 0x55a3ba7ae2c0 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.003+0000 7fa163fff640 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a3ba7ae2c0 0x55a3ba7b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.004+0000 7fa1437fe640 10 client.18450 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.004+0000 7fa163fff640 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a3ba7ae2c0 0x55a3ba7b0700 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94e7bbe3a06126e server_cookie=c32b8a85b27bb3c6 in_seq=0 out_seq=0 2024-02-17T12:57:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa1437fe640 1 -- 172.21.15.18:0/3108319555 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a3ba430720 con 0x55a3ba7ae2c0 2024-02-17T12:57:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa1437fe640 10 client.18450 handle_command_reply: tid=0 2024-02-17T12:57:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa16cfe5740 1 client.18450 shutdown 2024-02-17T12:57:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3108319555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 msgr2=0x55a3ba795190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 0x55a3ba795190 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa1640690e0 tx=0x7fa154016040 comp rx=0 tx=0).stop 2024-02-17T12:57:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3108319555 shutdown_connections 2024-02-17T12:57:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a3ba7ae2c0 0x55a3ba7b0700 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba795750 0x55a3ba792cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba78cb70 0x55a3ba792780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa1437fe640 0 client.18450 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/3108319555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 0x55a3ba795190 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3108319555 >> 172.21.15.18:0/3108319555 conn(0x55a3ba786980 msgr2=0x55a3ba7886d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3108319555 shutdown_connections 2024-02-17T12:57:53.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa16cfe5740 1 -- 172.21.15.18:0/3108319555 wait complete. 2024-02-17T12:57:53.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.005+0000 7fa16cfe5740 20 client.18450 trim_cache size 0 max 16384 2024-02-17T12:57:53.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.017+0000 7fa16cfe5740 1 Processor -- start 2024-02-17T12:57:53.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.017+0000 7fa16cfe5740 1 -- start start 2024-02-17T12:57:53.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.017+0000 7fa16cfe5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 0x55a3ba794eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:53.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.017+0000 7fa16cfe5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba78cb70 0x55a3ba79ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:53.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.017+0000 7fa16cfe5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba795750 0x55a3ba78f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:53.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.017+0000 7fa16cfe5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ba679130 con 0x55a3ba78c250 2024-02-17T12:57:53.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.017+0000 7fa16cfe5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ba78e380 con 0x55a3ba78cb70 2024-02-17T12:57:53.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.017+0000 7fa16cfe5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ba7b5930 con 0x55a3ba795750 2024-02-17T12:57:53.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.017+0000 7fa1637fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 0x55a3ba794eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:53.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.017+0000 7fa1637fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 0x55a3ba794eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35828/0 (socket says 172.21.15.18:35828) 2024-02-17T12:57:53.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.017+0000 7fa1637fe640 1 -- 172.21.15.18:0/1232830299 learned_addr learned my addr 172.21.15.18:0/1232830299 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:53.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.018+0000 7fa163fff640 1 --2- 172.21.15.18:0/1232830299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba78cb70 0x55a3ba79ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:53.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.018+0000 7fa1637fe640 1 -- 172.21.15.18:0/1232830299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba795750 msgr2=0x55a3ba78f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:53.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.018+0000 7fa1637fe640 1 --2- 172.21.15.18:0/1232830299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba795750 0x55a3ba78f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.018+0000 7fa1637fe640 1 -- 172.21.15.18:0/1232830299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba78cb70 msgr2=0x55a3ba79ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:53.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.018+0000 7fa1637fe640 1 --2- 172.21.15.18:0/1232830299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba78cb70 0x55a3ba79ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.018+0000 7fa1637fe640 1 -- 172.21.15.18:0/1232830299 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3ba68ec20 con 0x55a3ba78c250 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.018+0000 7fa163fff640 1 --2- 172.21.15.18:0/1232830299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba78cb70 0x55a3ba79ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.018+0000 7fa1637fe640 1 --2- 172.21.15.18:0/1232830299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 0x55a3ba794eb0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fa15400c450 tx=0x7fa1540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a8d923e7ed8189 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.018+0000 7fa159ffb640 1 -- 172.21.15.18:0/1232830299 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154014680 con 0x55a3ba78c250 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.018+0000 7fa159ffb640 1 -- 172.21.15.18:0/1232830299 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa15401a070 con 0x55a3ba78c250 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.018+0000 7fa159ffb640 1 -- 172.21.15.18:0/1232830299 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154004030 con 0x55a3ba78c250 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 -- 172.21.15.18:0/1232830299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 msgr2=0x55a3ba794eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/1232830299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 0x55a3ba794eb0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fa15400c450 tx=0x7fa1540057f0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 -- 172.21.15.18:0/1232830299 shutdown_connections 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/1232830299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba795750 0x55a3ba78f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/1232830299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba78cb70 0x55a3ba79ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/1232830299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78c250 0x55a3ba794eb0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 -- 172.21.15.18:0/1232830299 >> 172.21.15.18:0/1232830299 conn(0x55a3ba786980 msgr2=0x55a3ba79a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 -- 172.21.15.18:0/1232830299 shutdown_connections 2024-02-17T12:57:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 -- 172.21.15.18:0/1232830299 wait complete. 2024-02-17T12:57:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 -- start start 2024-02-17T12:57:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 Processor -- start 2024-02-17T12:57:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba78c250 0x55a3ba7aa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.019+0000 7fa16cfe5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 0x55a3ba7aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa16cfe5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba795750 0x55a3ba7a7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa16cfe5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ba7a7f70 con 0x55a3ba78cb70 2024-02-17T12:57:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa16cfe5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ba7a80e0 con 0x55a3ba795750 2024-02-17T12:57:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa16cfe5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ba7a82a0 con 0x55a3ba78c250 2024-02-17T12:57:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa16cfe5740 10 client.0 _ll_register_callbacks cb 0x7fa14c02c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa163fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 0x55a3ba7aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa163fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 0x55a3ba7aa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35842/0 (socket says 172.21.15.18:35842) 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa163fff640 1 -- 172.21.15.18:0/2047871162 learned_addr learned my addr 172.21.15.18:0/2047871162 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa1637fe640 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba795750 0x55a3ba7a7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa1637fe640 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba78c250 0x55a3ba7aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa163fff640 1 -- 172.21.15.18:0/2047871162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba78c250 msgr2=0x55a3ba7aa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa163fff640 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba78c250 0x55a3ba7aa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa163fff640 1 -- 172.21.15.18:0/2047871162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba795750 msgr2=0x55a3ba7a7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa163fff640 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba795750 0x55a3ba7a7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa163fff640 1 -- 172.21.15.18:0/2047871162 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3ba6f26a0 con 0x55a3ba78cb70 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa1637fe640 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba78c250 0x55a3ba7aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa163fff640 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 0x55a3ba7aa9c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fa14c004470 tx=0x7fa14c004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cb8579f2256f0e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa1437fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa1437fe640 1 -- 172.21.15.18:0/2047871162 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c054070 con 0x55a3ba78cb70 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa1437fe640 1 -- 172.21.15.18:0/2047871162 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa14c04e070 con 0x55a3ba78cb70 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa16cfe5740 1 -- 172.21.15.18:0/2047871162 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3ba7820f0 con 0x55a3ba78cb70 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa1437fe640 1 -- 172.21.15.18:0/2047871162 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa14c005850 con 0x55a3ba78cb70 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa16cfe5740 10 client.18456 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.020+0000 7fa16cfe5740 1 -- 172.21.15.18:0/2047871162 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55a3ba6b48c0 con 0x55a3ba78cb70 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.021+0000 7fa1437fe640 1 -- 172.21.15.18:0/2047871162 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a3ba7820f0 con 0x55a3ba78cb70 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.021+0000 7fa1437fe640 1 -- 172.21.15.18:0/2047871162 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fa14c057020 con 0x55a3ba78cb70 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.021+0000 7fa16cfe5740 10 client.18456 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.021+0000 7fa16cfe5740 10 client.18456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.021+0000 7fa16cfe5740 20 client.18456 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.021+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a3ba806590 0x55a3ba8089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.021+0000 7fa16cfe5740 4 client.18456 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:57:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.021+0000 7fa16cfe5740 1 -- 172.21.15.18:0/2047871162 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a3ba79bd20 con 0x55a3ba806590 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.021+0000 7fa1637fe640 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a3ba806590 0x55a3ba8089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.021+0000 7fa1437fe640 10 client.18456 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.021+0000 7fa1637fe640 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a3ba806590 0x55a3ba8089b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dba0d544c9a366b8 server_cookie=fd77b51586482ed8 in_seq=0 out_seq=0 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa1437fe640 1 -- 172.21.15.18:0/2047871162 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a3ba79bd20 con 0x55a3ba806590 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa1437fe640 10 client.18456 handle_command_reply: tid=0 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16cfe5740 1 client.18456 shutdown 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16cfe5740 1 -- 172.21.15.18:0/2047871162 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 msgr2=0x55a3ba7aa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 0x55a3ba7aa9c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fa14c004470 tx=0x7fa14c004540 comp rx=0 tx=0).stop 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16cfe5740 1 -- 172.21.15.18:0/2047871162 shutdown_connections 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a3ba806590 0x55a3ba8089b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3ba795750 0x55a3ba7a7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa1437fe640 0 client.18456 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3ba78cb70 0x55a3ba7aa9c0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16cfe5740 1 --2- 172.21.15.18:0/2047871162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3ba78c250 0x55a3ba7aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16cfe5740 1 -- 172.21.15.18:0/2047871162 >> 172.21.15.18:0/2047871162 conn(0x55a3ba786980 msgr2=0x55a3ba79ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16cfe5740 1 -- 172.21.15.18:0/2047871162 shutdown_connections 2024-02-17T12:57:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16cfe5740 1 -- 172.21.15.18:0/2047871162 wait complete. 2024-02-17T12:57:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16cfe5740 20 client.18456 trim_cache size 0 max 16384 2024-02-17T12:57:53.020 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:57:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16a33a640 1 -- 172.21.15.18:0/3327186315 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa1500a2230 msgr2=0x7fa1500a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16a33a640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa1500a2230 0x7fa1500a46f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0cc010 tx=0x7fa15c0908d0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16a33a640 1 -- 172.21.15.18:0/3327186315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa164197650 msgr2=0x7fa164197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.022+0000 7fa16a33a640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa164197650 0x7fa164197b00 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00cce0 tx=0x7fa15c00a620 comp rx=0 tx=0).stop 2024-02-17T12:57:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.023+0000 7fa16a33a640 1 -- 172.21.15.18:0/3327186315 shutdown_connections 2024-02-17T12:57:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.023+0000 7fa16a33a640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa1500a2230 0x7fa1500a46f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.023+0000 7fa16a33a640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa164197650 0x7fa164197b00 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.023+0000 7fa16a33a640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa164106500 0x7fa16419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.023+0000 7fa16a33a640 1 --2- 172.21.15.18:0/3327186315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1640ff8f0 0x7fa16419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.023+0000 7fa16a33a640 1 -- 172.21.15.18:0/3327186315 >> 172.21.15.18:0/3327186315 conn(0x7fa164076640 msgr2=0x7fa164104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.023+0000 7fa16a33a640 1 -- 172.21.15.18:0/3327186315 shutdown_connections 2024-02-17T12:57:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:53.023+0000 7fa16a33a640 1 -- 172.21.15.18:0/3327186315 wait complete. 2024-02-17T12:57:53.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:52 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3345406710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:57:53.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:52 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3345406710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:57:53.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 413183 5.38 MB/sec execute 308 sec latency 3325.773 ms 2024-02-17T12:57:53.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:52 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3345406710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:57:54.025 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:57:54.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:57:54.025 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:57:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.112+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1837043760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c105380 msgr2=0x7fe09c10cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.112+0000 7fe0a0e15640 1 --2- 172.21.15.18:0/1837043760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c105380 0x7fe09c10cf50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe094009cd0 tx=0x7fe09402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.112+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1837043760 shutdown_connections 2024-02-17T12:57:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.112+0000 7fe0a0e15640 1 --2- 172.21.15.18:0/1837043760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c105380 0x7fe09c10cf50 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.112+0000 7fe0a0e15640 1 --2- 172.21.15.18:0/1837043760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c100c60 0x7fe09c104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.112+0000 7fe0a0e15640 1 --2- 172.21.15.18:0/1837043760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c1002b0 0x7fe09c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.112+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1837043760 >> 172.21.15.18:0/1837043760 conn(0x7fe09c0771b0 msgr2=0x7fe09c0775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.112+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1837043760 shutdown_connections 2024-02-17T12:57:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.112+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1837043760 wait complete. 2024-02-17T12:57:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.112+0000 7fe0a0e15640 1 Processor -- start 2024-02-17T12:57:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe0a0e15640 1 -- start start 2024-02-17T12:57:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe0a0e15640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c1002b0 0x7fe09c101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe0a0e15640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c105380 0x7fe09c102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe0a0e15640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c1027c0 0x7fe09c1a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe0a0e15640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c1a5e80 con 0x7fe09c1002b0 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe0a0e15640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c1a5ff0 con 0x7fe09c105380 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe0a0e15640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c1a61b0 con 0x7fe09c1027c0 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe09a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c1002b0 0x7fe09c101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe09a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c1002b0 0x7fe09c101d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35852/0 (socket says 172.21.15.18:35852) 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe09a59c640 1 -- 172.21.15.18:0/1920622041 learned_addr learned my addr 172.21.15.18:0/1920622041 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe09ad9d640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c1027c0 0x7fe09c1a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe091d9b640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c105380 0x7fe09c102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe09a59c640 1 -- 172.21.15.18:0/1920622041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c1027c0 msgr2=0x7fe09c1a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe09a59c640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c1027c0 0x7fe09c1a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe09a59c640 1 -- 172.21.15.18:0/1920622041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c105380 msgr2=0x7fe09c102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe09a59c640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c105380 0x7fe09c102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe09a59c640 1 -- 172.21.15.18:0/1920622041 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe094009900 con 0x7fe09c1002b0 2024-02-17T12:57:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe09ad9d640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c1027c0 0x7fe09c1a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.113+0000 7fe091d9b640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c105380 0x7fe09c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.114+0000 7fe09a59c640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c1002b0 0x7fe09c101d40 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00e9f0 tx=0x7fe08c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.114+0000 7fe093fff640 1 -- 172.21.15.18:0/1920622041 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c009700 con 0x7fe09c1002b0 2024-02-17T12:57:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.114+0000 7fe093fff640 1 -- 172.21.15.18:0/1920622041 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe08c00cd80 con 0x7fe09c1002b0 2024-02-17T12:57:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.114+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1920622041 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09c1a6490 con 0x7fe09c1002b0 2024-02-17T12:57:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.114+0000 7fe093fff640 1 -- 172.21.15.18:0/1920622041 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c018920 con 0x7fe09c1002b0 2024-02-17T12:57:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.114+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1920622041 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c1a69a0 con 0x7fe09c1002b0 2024-02-17T12:57:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.116+0000 7fe093fff640 1 -- 172.21.15.18:0/1920622041 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe08c018a80 con 0x7fe09c1002b0 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.116+0000 7fe0a39eb740 1 Processor -- start 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe0a39eb740 1 -- start start 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe093fff640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0880a22d0 0x7fe0880a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe0a39eb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 0x564256365820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe0a39eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564256365d60 0x5642563661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe0a39eb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56425636f2e0 0x5642563716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe0a39eb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642562d49f0 con 0x564256366ad0 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe0a39eb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642562d1f20 con 0x56425636f2e0 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe0a39eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642562cd700 con 0x564256365d60 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe09ad9d640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0880a22d0 0x7fe0880a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe093fff640 1 -- 172.21.15.18:0/1920622041 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe08c014070 con 0x7fe09c1002b0 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe09a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 0x564256365820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe09a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 0x564256365820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35860/0 (socket says 172.21.15.18:35860) 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe09a59c640 1 -- 172.21.15.18:0/3041084196 learned_addr learned my addr 172.21.15.18:0/3041084196 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe09ad9d640 1 --2- 172.21.15.18:0/3041084196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56425636f2e0 0x5642563716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe091d9b640 1 --2- 172.21.15.18:0/3041084196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564256365d60 0x5642563661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe09a59c640 1 -- 172.21.15.18:0/3041084196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564256365d60 msgr2=0x5642563661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe09a59c640 1 --2- 172.21.15.18:0/3041084196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564256365d60 0x5642563661e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe09a59c640 1 -- 172.21.15.18:0/3041084196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56425636f2e0 msgr2=0x5642563716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe09a59c640 1 --2- 172.21.15.18:0/3041084196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56425636f2e0 0x5642563716d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.117+0000 7fe09a59c640 1 -- 172.21.15.18:0/3041084196 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642562637b0 con 0x564256366ad0 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.118+0000 7fe09ad9d640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0880a22d0 0x7fe0880a4790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe09402c630 tx=0x7fe094032490 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.118+0000 7fe09a59c640 1 --2- 172.21.15.18:0/3041084196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 0x564256365820 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0978a0 tx=0x7fe08c09ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760aaf7b16e46be1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.118+0000 7fe090d99640 1 -- 172.21.15.18:0/3041084196 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c09aed0 con 0x564256366ad0 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.118+0000 7fe090d99640 1 -- 172.21.15.18:0/3041084196 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe08c09ef70 con 0x564256366ad0 2024-02-17T12:57:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.118+0000 7fe090d99640 1 -- 172.21.15.18:0/3041084196 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c097540 con 0x564256366ad0 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 -- 172.21.15.18:0/3041084196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 msgr2=0x564256365820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/3041084196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 0x564256365820 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0978a0 tx=0x7fe08c09ab20 comp rx=0 tx=0).stop 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 -- 172.21.15.18:0/3041084196 shutdown_connections 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/3041084196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56425636f2e0 0x5642563716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/3041084196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564256365d60 0x5642563661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/3041084196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 0x564256365820 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 -- 172.21.15.18:0/3041084196 >> 172.21.15.18:0/3041084196 conn(0x56425632d550 msgr2=0x564256360a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 -- 172.21.15.18:0/3041084196 shutdown_connections 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 -- 172.21.15.18:0/3041084196 wait complete. 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 -- start start 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 Processor -- start 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 0x56425636c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564256366ad0 0x56425636c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.119+0000 7fe0a39eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56425636f2e0 0x56425636cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe0a39eb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564256272bf0 con 0x564256365d60 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe0a39eb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564255fa1250 con 0x564256366ad0 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe0a39eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564256253200 con 0x56425636f2e0 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe0a39eb740 10 client.0 _ll_register_callbacks cb 0x5642562c1d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe09a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 0x56425636c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe09a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 0x56425636c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35864/0 (socket says 172.21.15.18:35864) 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe09a59c640 1 -- 172.21.15.18:0/493047074 learned_addr learned my addr 172.21.15.18:0/493047074 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe091d9b640 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564256366ad0 0x56425636c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe09ad9d640 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56425636f2e0 0x56425636cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe091d9b640 1 -- 172.21.15.18:0/493047074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56425636f2e0 msgr2=0x56425636cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe091d9b640 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56425636f2e0 0x56425636cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe091d9b640 1 -- 172.21.15.18:0/493047074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 msgr2=0x56425636c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe091d9b640 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 0x56425636c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe091d9b640 1 -- 172.21.15.18:0/493047074 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c1a69a0 con 0x564256366ad0 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe09ad9d640 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56425636f2e0 0x56425636cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe09a59c640 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 0x56425636c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe091d9b640 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564256366ad0 0x56425636c860 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe09c103780 tx=0x7fe084012e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35a54c4b17d5364 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe081ffb640 1 -- 172.21.15.18:0/493047074 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084017070 con 0x564256366ad0 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe081ffb640 1 -- 172.21.15.18:0/493047074 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe084004050 con 0x564256366ad0 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.120+0000 7fe0a39eb740 1 -- 172.21.15.18:0/493047074 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564255fcd600 con 0x564256366ad0 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.121+0000 7fe081ffb640 1 -- 172.21.15.18:0/493047074 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe084004210 con 0x564256366ad0 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.121+0000 7fe0a39eb740 10 client.27593 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.121+0000 7fe0a39eb740 1 -- 172.21.15.18:0/493047074 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5642562cbd80 con 0x564256366ad0 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.121+0000 7fe081ffb640 1 -- 172.21.15.18:0/493047074 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564255fcd600 con 0x564256366ad0 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.121+0000 7fe081ffb640 1 -- 172.21.15.18:0/493047074 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe084003a60 con 0x564256366ad0 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.121+0000 7fe0a39eb740 10 client.27593 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.121+0000 7fe0a39eb740 10 client.27593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.121+0000 7fe0a39eb740 20 client.27593 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.121+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56425637f6d0 0x564256381af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.121+0000 7fe0a39eb740 4 client.27593 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.121+0000 7fe0a39eb740 1 -- 172.21.15.18:0/493047074 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56425601e1b0 con 0x56425637f6d0 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.121+0000 7fe09a59c640 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56425637f6d0 0x564256381af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe09a59c640 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56425637f6d0 0x564256381af0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293a1b24a6b17bf server_cookie=ddf9271d290cdb35 in_seq=0 out_seq=0 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe081ffb640 10 client.27593 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe081ffb640 1 -- 172.21.15.18:0/493047074 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56425601e1b0 con 0x56425637f6d0 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe081ffb640 10 client.27593 handle_command_reply: tid=0 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe0a39eb740 1 client.27593 shutdown 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe0a39eb740 1 -- 172.21.15.18:0/493047074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564256366ad0 msgr2=0x56425636c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564256366ad0 0x56425636c860 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe09c103780 tx=0x7fe084012e70 comp rx=0 tx=0).stop 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe0a39eb740 1 -- 172.21.15.18:0/493047074 shutdown_connections 2024-02-17T12:57:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56425637f6d0 0x564256381af0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56425636f2e0 0x56425636cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564256366ad0 0x56425636c860 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe081ffb640 0 client.27593 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/493047074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 0x56425636c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe0a39eb740 1 -- 172.21.15.18:0/493047074 >> 172.21.15.18:0/493047074 conn(0x56425632d550 msgr2=0x5642563627a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe0a39eb740 1 -- 172.21.15.18:0/493047074 shutdown_connections 2024-02-17T12:57:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.122+0000 7fe0a39eb740 1 -- 172.21.15.18:0/493047074 wait complete. 2024-02-17T12:57:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.123+0000 7fe0a39eb740 20 client.27593 trim_cache size 0 max 16384 2024-02-17T12:57:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe0a39eb740 1 Processor -- start 2024-02-17T12:57:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe0a39eb740 1 -- start start 2024-02-17T12:57:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe0a39eb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564256365d60 0x56425636ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe0a39eb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 0x56425636cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe0a39eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56425636f2e0 0x56425636abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe0a39eb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425635c380 con 0x564256366ad0 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe0a39eb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564256367f30 con 0x564256365d60 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe0a39eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425636d6a0 con 0x56425636f2e0 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe091d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 0x56425636cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe091d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 0x56425636cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35870/0 (socket says 172.21.15.18:35870) 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe09a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564256365d60 0x56425636ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe091d9b640 1 -- 172.21.15.18:0/1446236742 learned_addr learned my addr 172.21.15.18:0/1446236742 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe09ad9d640 1 --2- 172.21.15.18:0/1446236742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56425636f2e0 0x56425636abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe091d9b640 1 -- 172.21.15.18:0/1446236742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56425636f2e0 msgr2=0x56425636abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe091d9b640 1 --2- 172.21.15.18:0/1446236742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56425636f2e0 0x56425636abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe091d9b640 1 -- 172.21.15.18:0/1446236742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564256365d60 msgr2=0x56425636ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe091d9b640 1 --2- 172.21.15.18:0/1446236742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564256365d60 0x56425636ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.137+0000 7fe091d9b640 1 -- 172.21.15.18:0/1446236742 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642562687b0 con 0x564256366ad0 2024-02-17T12:57:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.138+0000 7fe091d9b640 1 --2- 172.21.15.18:0/1446236742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 0x56425636cbc0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe084003fb0 tx=0x7fe0840092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541bec679a0ca94a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.138+0000 7fe083fff640 1 -- 172.21.15.18:0/1446236742 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084017040 con 0x564256366ad0 2024-02-17T12:57:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.138+0000 7fe083fff640 1 -- 172.21.15.18:0/1446236742 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe08400fe30 con 0x564256366ad0 2024-02-17T12:57:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.138+0000 7fe083fff640 1 -- 172.21.15.18:0/1446236742 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084005300 con 0x564256366ad0 2024-02-17T12:57:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.139+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1446236742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 msgr2=0x56425636cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.139+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/1446236742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 0x56425636cbc0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe084003fb0 tx=0x7fe0840092f0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.139+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1446236742 shutdown_connections 2024-02-17T12:57:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.139+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/1446236742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56425636f2e0 0x56425636abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.139+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/1446236742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256366ad0 0x56425636cbc0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.139+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/1446236742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564256365d60 0x56425636ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.139+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1446236742 >> 172.21.15.18:0/1446236742 conn(0x56425632d550 msgr2=0x564256386470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.139+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1446236742 shutdown_connections 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.139+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1446236742 wait complete. 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.139+0000 7fe0a39eb740 1 -- start start 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.139+0000 7fe0a39eb740 1 Processor -- start 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe0a39eb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 0x564256376300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe0a39eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564256366ad0 0x5642563746a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe0a39eb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56425636f2e0 0x564256374be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe0a39eb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642563751f0 con 0x564256365d60 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe0a39eb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564256375360 con 0x56425636f2e0 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe09a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 0x564256376300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe0a39eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564256375520 con 0x564256366ad0 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe09a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 0x564256376300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35880/0 (socket says 172.21.15.18:35880) 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe09a59c640 1 -- 172.21.15.18:0/1783980463 learned_addr learned my addr 172.21.15.18:0/1783980463 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe091d9b640 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564256366ad0 0x5642563746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe09ad9d640 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56425636f2e0 0x564256374be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe0a39eb740 10 client.0 _ll_register_callbacks cb 0x7fe08c096b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe09a59c640 1 -- 172.21.15.18:0/1783980463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564256366ad0 msgr2=0x5642563746a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe09a59c640 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564256366ad0 0x5642563746a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe09a59c640 1 -- 172.21.15.18:0/1783980463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56425636f2e0 msgr2=0x564256374be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe09a59c640 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56425636f2e0 0x564256374be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.140+0000 7fe09a59c640 1 -- 172.21.15.18:0/1783980463 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642562cbd80 con 0x564256365d60 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe09a59c640 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 0x564256376300 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fe08c01c3c0 tx=0x7fe08c010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07df64e1976727c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe081ffb640 1 -- 172.21.15.18:0/1783980463 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0cc000 con 0x564256365d60 2024-02-17T12:57:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1783980463 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe08c09ef70 con 0x564256365d60 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe081ffb640 1 -- 172.21.15.18:0/1783980463 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe08c0b3040 con 0x564256365d60 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe081ffb640 1 -- 172.21.15.18:0/1783980463 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe08c098f70 con 0x564256365d60 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe0a39eb740 10 client.18474 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1783980463 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56425628e540 con 0x564256365d60 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe081ffb640 1 -- 172.21.15.18:0/1783980463 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe08c09ef70 con 0x564256365d60 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe081ffb640 1 -- 172.21.15.18:0/1783980463 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe08c0b3b50 con 0x564256365d60 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe0a39eb740 10 client.18474 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe0a39eb740 10 client.18474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe0a39eb740 20 client.18474 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5642563d6f80 0x5642563d93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe0a39eb740 4 client.18474 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1783980463 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56425600a550 con 0x5642563d6f80 2024-02-17T12:57:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.141+0000 7fe091d9b640 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5642563d6f80 0x5642563d93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.142+0000 7fe091d9b640 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5642563d6f80 0x5642563d93a0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a40626300f544f50 server_cookie=dfe03d87f651d771 in_seq=0 out_seq=0 2024-02-17T12:57:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.142+0000 7fe081ffb640 10 client.18474 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:54.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.160+0000 7f7f3dffb640 1 -- 172.21.15.18:0/341467397 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3400e8c0 con 0x7f7f4010a440 2024-02-17T12:57:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.160+0000 7f7f458f4640 1 -- 172.21.15.18:0/341467397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f4010a440 msgr2=0x7f7f40112b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.160+0000 7f7f458f4640 1 --2- 172.21.15.18:0/341467397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f4010a440 0x7f7f40112b90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400fb10 tx=0x7f7f3402fe80 comp rx=0 tx=0).stop 2024-02-17T12:57:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.160+0000 7f7f458f4640 1 -- 172.21.15.18:0/341467397 shutdown_connections 2024-02-17T12:57:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.160+0000 7f7f458f4640 1 --2- 172.21.15.18:0/341467397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f401130d0 0x7f7f401154c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.160+0000 7f7f458f4640 1 --2- 172.21.15.18:0/341467397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f4010a440 0x7f7f40112b90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.160+0000 7f7f458f4640 1 --2- 172.21.15.18:0/341467397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f40109b20 0x7f7f40109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.160+0000 7f7f458f4640 1 -- 172.21.15.18:0/341467397 >> 172.21.15.18:0/341467397 conn(0x7f7f4006c6c0 msgr2=0x7f7f4006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.161+0000 7f7f458f4640 1 -- 172.21.15.18:0/341467397 shutdown_connections 2024-02-17T12:57:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.161+0000 7f7f458f4640 1 -- 172.21.15.18:0/341467397 wait complete. 2024-02-17T12:57:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.161+0000 7f7f458f4640 1 Processor -- start 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f458f4640 1 -- start start 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f458f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f40109b20 0x7f7f4010c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f458f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4010a440 0x7f7f4010ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f458f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f401130d0 0x7f7f401120a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f458f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40112830 con 0x7f7f4010a440 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f458f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f401129a0 con 0x7f7f401130d0 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f458f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40112b60 con 0x7f7f40109b20 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f3e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4010a440 0x7f7f4010ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f3effd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f40109b20 0x7f7f4010c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f3e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4010a440 0x7f7f4010ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35902/0 (socket says 172.21.15.18:35902) 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f3e7fc640 1 -- 172.21.15.18:0/3664498606 learned_addr learned my addr 172.21.15.18:0/3664498606 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f3f7fe640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f401130d0 0x7f7f401120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f3e7fc640 1 -- 172.21.15.18:0/3664498606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f40109b20 msgr2=0x7f7f4010c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f3e7fc640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f40109b20 0x7f7f4010c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f3e7fc640 1 -- 172.21.15.18:0/3664498606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f401130d0 msgr2=0x7f7f401120a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f3e7fc640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f401130d0 0x7f7f401120a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.162+0000 7f7f3e7fc640 1 -- 172.21.15.18:0/3664498606 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3400f770 con 0x7f7f4010a440 2024-02-17T12:57:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.163+0000 7f7f3e7fc640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4010a440 0x7f7f4010ce00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f7f34030410 tx=0x7f7f34005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.163+0000 7f7f448f2640 1 -- 172.21.15.18:0/3664498606 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f340048c0 con 0x7f7f4010a440 2024-02-17T12:57:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.163+0000 7f7f448f2640 1 -- 172.21.15.18:0/3664498606 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f34005670 con 0x7f7f4010a440 2024-02-17T12:57:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.163+0000 7f7f458f4640 1 -- 172.21.15.18:0/3664498606 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f40112de0 con 0x7f7f4010a440 2024-02-17T12:57:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.163+0000 7f7f448f2640 1 -- 172.21.15.18:0/3664498606 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3400ee70 con 0x7f7f4010a440 2024-02-17T12:57:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.163+0000 7f7f458f4640 1 -- 172.21.15.18:0/3664498606 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f401bbbf0 con 0x7f7f4010a440 2024-02-17T12:57:54.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f448f2640 1 -- 172.21.15.18:0/3664498606 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7f3400e470 con 0x7f7f4010a440 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f485a5740 1 Processor -- start 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f485a5740 1 -- start start 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f485a5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c0502c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f485a5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c050352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f485a5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 0x556c05037d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f485a5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c04f9af90 con 0x556c0502c380 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f485a5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c04f984c0 con 0x556c050358b0 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f485a5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c04f93ca0 con 0x556c0502cd30 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f3f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c0502c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f3f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c0502c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35910/0 (socket says 172.21.15.18:35910) 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f448f2640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7f280a22f0 0x7f7f280a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f3f7fe640 1 -- 172.21.15.18:0/2537027008 learned_addr learned my addr 172.21.15.18:0/2537027008 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f3effd640 1 --2- 172.21.15.18:0/2537027008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c050352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.166+0000 7f7f3e7fc640 1 --2- 172.21.15.18:0/2537027008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 0x556c05037d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f3f7fe640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7f280a22f0 0x7f7f280a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f3effd640 1 -- 172.21.15.18:0/2537027008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 msgr2=0x556c05037d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f3effd640 1 --2- 172.21.15.18:0/2537027008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 0x556c05037d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f3effd640 1 -- 172.21.15.18:0/2537027008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 msgr2=0x556c0502c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f3effd640 1 --2- 172.21.15.18:0/2537027008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c0502c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f3effd640 1 -- 172.21.15.18:0/2537027008 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c04f560b0 con 0x556c0502cd30 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f3f7fe640 1 --2- 172.21.15.18:0/2537027008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c0502c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f448f2640 1 -- 172.21.15.18:0/3664498606 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7f34031e40 con 0x7f7f4010a440 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f3effd640 1 --2- 172.21.15.18:0/2537027008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c050352e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000bb60 tx=0x7f7f30005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e2fd450537a4a0a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f3f7fe640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7f280a22f0 0x7f7f280a47b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7f380070b0 tx=0x7f7f38004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f25ffb640 1 -- 172.21.15.18:0/2537027008 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30010040 con 0x556c0502cd30 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f25ffb640 1 -- 172.21.15.18:0/2537027008 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f3000be50 con 0x556c0502cd30 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.167+0000 7f7f25ffb640 1 -- 172.21.15.18:0/2537027008 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30004a90 con 0x556c0502cd30 2024-02-17T12:57:54.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.168+0000 7f7f485a5740 1 -- 172.21.15.18:0/2537027008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 msgr2=0x556c050352e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.168+0000 7f7f485a5740 1 --2- 172.21.15.18:0/2537027008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c050352e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000bb60 tx=0x7f7f30005f90 comp rx=0 tx=0).stop 2024-02-17T12:57:54.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.168+0000 7f7f485a5740 1 -- 172.21.15.18:0/2537027008 shutdown_connections 2024-02-17T12:57:54.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.168+0000 7f7f485a5740 1 --2- 172.21.15.18:0/2537027008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 0x556c05037d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.168+0000 7f7f485a5740 1 --2- 172.21.15.18:0/2537027008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c050352e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.168+0000 7f7f485a5740 1 --2- 172.21.15.18:0/2537027008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c0502c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.168+0000 7f7f485a5740 1 -- 172.21.15.18:0/2537027008 >> 172.21.15.18:0/2537027008 conn(0x556c05026840 msgr2=0x556c05026c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.168+0000 7f7f485a5740 1 -- 172.21.15.18:0/2537027008 shutdown_connections 2024-02-17T12:57:54.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.168+0000 7f7f485a5740 1 -- 172.21.15.18:0/2537027008 wait complete. 2024-02-17T12:57:54.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.168+0000 7f7f485a5740 1 -- start start 2024-02-17T12:57:54.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f485a5740 1 Processor -- start 2024-02-17T12:57:54.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f485a5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c0502c380 0x556c050327e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f485a5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c05032d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f485a5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c050358b0 0x556c05033260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f485a5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c04f38f40 con 0x556c050358b0 2024-02-17T12:57:54.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f485a5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c04c674b0 con 0x556c0502c380 2024-02-17T12:57:54.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f485a5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c04f19670 con 0x556c0502cd30 2024-02-17T12:57:54.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f485a5740 10 client.0 _ll_register_callbacks cb 0x556c04f86f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f3f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c050358b0 0x556c05033260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f3f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c050358b0 0x556c05033260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35914/0 (socket says 172.21.15.18:35914) 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f3effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c0502c380 0x556c050327e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f3e7fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c05032d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f3f7fe640 1 -- 172.21.15.18:0/1091144792 learned_addr learned my addr 172.21.15.18:0/1091144792 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f3f7fe640 1 -- 172.21.15.18:0/1091144792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 msgr2=0x556c05032d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f3f7fe640 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c05032d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f3f7fe640 1 -- 172.21.15.18:0/1091144792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c0502c380 msgr2=0x556c050327e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f3f7fe640 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c0502c380 0x556c050327e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f3f7fe640 1 -- 172.21.15.18:0/1091144792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3000b790 con 0x556c050358b0 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f3e7fc640 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c05032d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.169+0000 7f7f3effd640 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c0502c380 0x556c050327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f3f7fe640 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c050358b0 0x556c05033260 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f7f38019720 tx=0x7f7f38004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab737829a5f88ee server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f1effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:54.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f1effd640 1 -- 172.21.15.18:0/1091144792 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38024600 con 0x556c050358b0 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f1effd640 1 -- 172.21.15.18:0/1091144792 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7f38024760 con 0x556c050358b0 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f485a5740 1 -- 172.21.15.18:0/1091144792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c04f6df40 con 0x556c050358b0 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f1effd640 1 -- 172.21.15.18:0/1091144792 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f38019e60 con 0x556c050358b0 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f485a5740 10 client.18483 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f485a5740 1 -- 172.21.15.18:0/1091144792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x556c04f92320 con 0x556c050358b0 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f1effd640 1 -- 172.21.15.18:0/1091144792 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f38031020 con 0x556c050358b0 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f1effd640 1 -- 172.21.15.18:0/1091144792 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7f38008040 con 0x556c050358b0 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f485a5740 10 client.18483 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f485a5740 10 client.18483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f485a5740 20 client.18483 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f485a5740 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556c05045ed0 0x556c050482c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f485a5740 4 client.18483 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.170+0000 7f7f485a5740 1 -- 172.21.15.18:0/1091144792 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c04f8a310 con 0x556c05045ed0 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.171+0000 7f7f3effd640 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556c05045ed0 0x556c050482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.171+0000 7f7f1effd640 10 client.18483 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:54.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.171+0000 7f7f3effd640 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556c05045ed0 0x556c050482c0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763eebda60e2be32 server_cookie=f7635791588a9474 in_seq=0 out_seq=0 2024-02-17T12:57:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe081ffb640 1 -- 172.21.15.18:0/1783980463 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56425600a550 con 0x5642563d6f80 2024-02-17T12:57:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe081ffb640 10 client.18474 handle_command_reply: tid=0 2024-02-17T12:57:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe0a39eb740 1 client.18474 shutdown 2024-02-17T12:57:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1783980463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 msgr2=0x564256376300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 0x564256376300 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fe08c01c3c0 tx=0x7fe08c010f50 comp rx=0 tx=0).stop 2024-02-17T12:57:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1783980463 shutdown_connections 2024-02-17T12:57:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5642563d6f80 0x5642563d93a0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56425636f2e0 0x564256374be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe081ffb640 0 client.18474 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564256366ad0 0x5642563746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe0a39eb740 1 --2- 172.21.15.18:0/1783980463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564256365d60 0x564256376300 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1783980463 >> 172.21.15.18:0/1783980463 conn(0x56425632d550 msgr2=0x56425636ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1783980463 shutdown_connections 2024-02-17T12:57:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.176+0000 7fe0a39eb740 1 -- 172.21.15.18:0/1783980463 wait complete. 2024-02-17T12:57:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.177+0000 7fe0a39eb740 20 client.18474 trim_cache size 0 max 16384 2024-02-17T12:57:54.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f1effd640 1 -- 172.21.15.18:0/1091144792 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c04f8a310 con 0x556c05045ed0 2024-02-17T12:57:54.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f1effd640 10 client.18483 handle_command_reply: tid=0 2024-02-17T12:57:54.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f485a5740 1 client.18483 shutdown 2024-02-17T12:57:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:57:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:57:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:57:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:57:54.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f485a5740 1 -- 172.21.15.18:0/1091144792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c050358b0 msgr2=0x556c05033260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f485a5740 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c050358b0 0x556c05033260 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f7f38019720 tx=0x7f7f38004b50 comp rx=0 tx=0).stop 2024-02-17T12:57:54.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f485a5740 1 -- 172.21.15.18:0/1091144792 shutdown_connections 2024-02-17T12:57:54.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f485a5740 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556c05045ed0 0x556c050482c0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f485a5740 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c050358b0 0x556c05033260 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f1effd640 0 client.18483 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:54.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f485a5740 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c05032d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f485a5740 1 --2- 172.21.15.18:0/1091144792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c0502c380 0x556c050327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f485a5740 1 -- 172.21.15.18:0/1091144792 >> 172.21.15.18:0/1091144792 conn(0x556c05026840 msgr2=0x556c050287a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f485a5740 1 -- 172.21.15.18:0/1091144792 shutdown_connections 2024-02-17T12:57:54.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f485a5740 1 -- 172.21.15.18:0/1091144792 wait complete. 2024-02-17T12:57:54.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.177+0000 7f7f485a5740 20 client.18483 trim_cache size 0 max 16384 2024-02-17T12:57:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.177+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1920622041 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0880a22d0 msgr2=0x7fe0880a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.177+0000 7fe0a0e15640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0880a22d0 0x7fe0880a4790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe09402c630 tx=0x7fe094032490 comp rx=0 tx=0).stop 2024-02-17T12:57:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.177+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1920622041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c1002b0 msgr2=0x7fe09c101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.177+0000 7fe0a0e15640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c1002b0 0x7fe09c101d40 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00e9f0 tx=0x7fe08c00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.177+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1920622041 shutdown_connections 2024-02-17T12:57:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.177+0000 7fe0a0e15640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe0880a22d0 0x7fe0880a4790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.177+0000 7fe0a0e15640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09c1027c0 0x7fe09c1a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.177+0000 7fe0a0e15640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe09c105380 0x7fe09c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.177+0000 7fe0a0e15640 1 --2- 172.21.15.18:0/1920622041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe09c1002b0 0x7fe09c101d40 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.177+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1920622041 >> 172.21.15.18:0/1920622041 conn(0x7fe09c0771b0 msgr2=0x7fe09c069670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.178+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1920622041 shutdown_connections 2024-02-17T12:57:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:57:54.178+0000 7fe0a0e15640 1 -- 172.21.15.18:0/1920622041 wait complete. 2024-02-17T12:57:54.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.194+0000 7f7f485a5740 1 Processor -- start 2024-02-17T12:57:54.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f485a5740 1 -- start start 2024-02-17T12:57:54.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f485a5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c05034f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f485a5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c05033080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f485a5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 0x556c050310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f485a5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c05020da0 con 0x556c0502c380 2024-02-17T12:57:54.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f485a5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c05037ca0 con 0x556c050358b0 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f485a5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0504d4d0 con 0x556c0502cd30 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f3effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c05034f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f3effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c05034f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35928/0 (socket says 172.21.15.18:35928) 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f3effd640 1 -- 172.21.15.18:0/3932805478 learned_addr learned my addr 172.21.15.18:0/3932805478 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f3e7fc640 1 --2- 172.21.15.18:0/3932805478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c05033080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f3f7fe640 1 --2- 172.21.15.18:0/3932805478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 0x556c050310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f3effd640 1 -- 172.21.15.18:0/3932805478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 msgr2=0x556c05033080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f3effd640 1 --2- 172.21.15.18:0/3932805478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c05033080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f3effd640 1 -- 172.21.15.18:0/3932805478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 msgr2=0x556c050310b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f3effd640 1 --2- 172.21.15.18:0/3932805478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 0x556c050310b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f3effd640 1 -- 172.21.15.18:0/3932805478 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c04f2ec00 con 0x556c0502c380 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.195+0000 7f7f3effd640 1 --2- 172.21.15.18:0/3932805478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c05034f10 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000bb60 tx=0x7f7f30021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83148f4b69471ead server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.196+0000 7f7f257fa640 1 -- 172.21.15.18:0/3932805478 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3002a630 con 0x556c0502c380 2024-02-17T12:57:54.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.196+0000 7f7f257fa640 1 -- 172.21.15.18:0/3932805478 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f30025040 con 0x556c0502c380 2024-02-17T12:57:54.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.196+0000 7f7f257fa640 1 -- 172.21.15.18:0/3932805478 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30004b10 con 0x556c0502c380 2024-02-17T12:57:54.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.196+0000 7f7f485a5740 1 -- 172.21.15.18:0/3932805478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 msgr2=0x556c05034f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.196+0000 7f7f485a5740 1 --2- 172.21.15.18:0/3932805478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c05034f10 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000bb60 tx=0x7f7f30021040 comp rx=0 tx=0).stop 2024-02-17T12:57:54.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.196+0000 7f7f485a5740 1 -- 172.21.15.18:0/3932805478 shutdown_connections 2024-02-17T12:57:54.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.196+0000 7f7f485a5740 1 --2- 172.21.15.18:0/3932805478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 0x556c050310b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.196+0000 7f7f485a5740 1 --2- 172.21.15.18:0/3932805478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c05033080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.196+0000 7f7f485a5740 1 --2- 172.21.15.18:0/3932805478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c05034f10 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.196+0000 7f7f485a5740 1 -- 172.21.15.18:0/3932805478 >> 172.21.15.18:0/3932805478 conn(0x556c05026840 msgr2=0x556c0502dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f485a5740 1 -- 172.21.15.18:0/3932805478 shutdown_connections 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f485a5740 1 -- 172.21.15.18:0/3932805478 wait complete. 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f485a5740 1 -- start start 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f485a5740 1 Processor -- start 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f485a5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c0503aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f485a5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c0503afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f485a5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 0x556c0509b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f485a5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0509b900 con 0x556c0502c380 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f485a5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0509ba40 con 0x556c050358b0 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f485a5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0509bb80 con 0x556c0502cd30 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f485a5740 10 client.0 _ll_register_callbacks cb 0x7f7f3000b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f3effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c0503aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f3effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c0503aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35932/0 (socket says 172.21.15.18:35932) 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f3effd640 1 -- 172.21.15.18:0/622209947 learned_addr learned my addr 172.21.15.18:0/622209947 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f3e7fc640 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c0503afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.197+0000 7f7f3f7fe640 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 0x556c0509b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f3effd640 1 -- 172.21.15.18:0/622209947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 msgr2=0x556c0503afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f3effd640 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c0503afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f3effd640 1 -- 172.21.15.18:0/622209947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 msgr2=0x556c0509b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f3effd640 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 0x556c0509b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f3effd640 1 -- 172.21.15.18:0/622209947 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c04f560b0 con 0x556c0502c380 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f3e7fc640 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c0503afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f3effd640 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c0503aa60 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f7f30004570 tx=0x7f7f3000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a352ac7285ef5a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f1effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f1effd640 1 -- 172.21.15.18:0/622209947 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000b320 con 0x556c0502c380 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f1effd640 1 -- 172.21.15.18:0/622209947 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7f30010040 con 0x556c0502c380 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f485a5740 1 -- 172.21.15.18:0/622209947 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c04fb9900 con 0x556c0502c380 2024-02-17T12:57:54.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f1effd640 1 -- 172.21.15.18:0/622209947 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f3002a640 con 0x556c0502c380 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f485a5740 10 client.18489 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f485a5740 1 -- 172.21.15.18:0/622209947 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x556c04f54910 con 0x556c0502c380 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.198+0000 7f7f1effd640 1 -- 172.21.15.18:0/622209947 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556c04fb9900 con 0x556c0502c380 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.199+0000 7f7f1effd640 1 -- 172.21.15.18:0/622209947 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7f30003670 con 0x556c0502c380 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.199+0000 7f7f485a5740 10 client.18489 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.199+0000 7f7f485a5740 10 client.18489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.199+0000 7f7f485a5740 20 client.18489 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.199+0000 7f7f485a5740 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556c0509ced0 0x556c0509f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.199+0000 7f7f485a5740 4 client.18489 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.199+0000 7f7f485a5740 1 -- 172.21.15.18:0/622209947 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556c04c93860 con 0x556c0509ced0 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.199+0000 7f7f3e7fc640 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556c0509ced0 0x556c0509f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.199+0000 7f7f1effd640 10 client.18489 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.199+0000 7f7f3e7fc640 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556c0509ced0 0x556c0509f2c0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f4a8d0951e46a0 server_cookie=e9ed32c27f8df5cf in_seq=0 out_seq=0 2024-02-17T12:57:54.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f1effd640 1 -- 172.21.15.18:0/622209947 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556c04c93860 con 0x556c0509ced0 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f1effd640 10 client.18489 handle_command_reply: tid=0 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f485a5740 1 client.18489 shutdown 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f485a5740 1 -- 172.21.15.18:0/622209947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 msgr2=0x556c0503aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f485a5740 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c0503aa60 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f7f30004570 tx=0x7f7f3000b040 comp rx=0 tx=0).stop 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f485a5740 1 -- 172.21.15.18:0/622209947 shutdown_connections 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f485a5740 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556c0509ced0 0x556c0509f2c0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f485a5740 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c050358b0 0x556c0509b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f485a5740 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556c0502cd30 0x556c0503afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f1effd640 0 client.18489 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f485a5740 1 --2- 172.21.15.18:0/622209947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c0502c380 0x556c0503aa60 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f485a5740 1 -- 172.21.15.18:0/622209947 >> 172.21.15.18:0/622209947 conn(0x556c05026840 msgr2=0x556c05025050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f485a5740 1 -- 172.21.15.18:0/622209947 shutdown_connections 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f485a5740 1 -- 172.21.15.18:0/622209947 wait complete. 2024-02-17T12:57:54.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.201+0000 7f7f485a5740 20 client.18489 trim_cache size 0 max 16384 2024-02-17T12:57:54.197 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:57:54.197 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:57:54.197 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "09112172-dd48-4e61-8c93-9efc4a5cd0a3", 2024-02-17T12:57:54.197 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:57:54.197 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:57:54.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.202+0000 7f7f458f4640 1 -- 172.21.15.18:0/3664498606 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7f280a22f0 msgr2=0x7f7f280a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.202+0000 7f7f458f4640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7f280a22f0 0x7f7f280a47b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7f380070b0 tx=0x7f7f38004a40 comp rx=0 tx=0).stop 2024-02-17T12:57:54.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.202+0000 7f7f458f4640 1 -- 172.21.15.18:0/3664498606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4010a440 msgr2=0x7f7f4010ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.202+0000 7f7f458f4640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4010a440 0x7f7f4010ce00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f7f34030410 tx=0x7f7f34005270 comp rx=0 tx=0).stop 2024-02-17T12:57:54.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.202+0000 7f7f458f4640 1 -- 172.21.15.18:0/3664498606 shutdown_connections 2024-02-17T12:57:54.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.202+0000 7f7f458f4640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7f280a22f0 0x7f7f280a47b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.202+0000 7f7f458f4640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f401130d0 0x7f7f401120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.202+0000 7f7f458f4640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4010a440 0x7f7f4010ce00 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.202+0000 7f7f458f4640 1 --2- 172.21.15.18:0/3664498606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f40109b20 0x7f7f4010c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.202+0000 7f7f458f4640 1 -- 172.21.15.18:0/3664498606 >> 172.21.15.18:0/3664498606 conn(0x7f7f4006c6c0 msgr2=0x7f7f4006fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.202+0000 7f7f458f4640 1 -- 172.21.15.18:0/3664498606 shutdown_connections 2024-02-17T12:57:54.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.202+0000 7f7f458f4640 1 -- 172.21.15.18:0/3664498606 wait complete. 2024-02-17T12:57:54.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:53 smithi146 ceph-mon[36454]: pgmap v248: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-02-17T12:57:54.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:53 smithi018 ceph-mon[32625]: pgmap v248: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-02-17T12:57:54.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:57:54.203 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:57:54.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 415027 5.40 MB/sec execute 309 sec latency 3525.534 ms 2024-02-17T12:57:54.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.321+0000 7fdda876c640 1 -- 172.21.15.18:0/1225009458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdda0105330 msgr2=0x7fdda010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.321+0000 7fdda876c640 1 --2- 172.21.15.18:0/1225009458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdda0105330 0x7fdda010cf00 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdd98009cd0 tx=0x7fdd9802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.321+0000 7fdda876c640 1 -- 172.21.15.18:0/1225009458 shutdown_connections 2024-02-17T12:57:54.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.321+0000 7fdda876c640 1 --2- 172.21.15.18:0/1225009458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdda0105330 0x7fdda010cf00 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.321+0000 7fdda876c640 1 --2- 172.21.15.18:0/1225009458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdda0100c10 0x7fdda0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.321+0000 7fdda876c640 1 --2- 172.21.15.18:0/1225009458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0100260 0x7fdda0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.321+0000 7fdda876c640 1 -- 172.21.15.18:0/1225009458 >> 172.21.15.18:0/1225009458 conn(0x7fdda00771b0 msgr2=0x7fdda00775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.321+0000 7fdda876c640 1 -- 172.21.15.18:0/1225009458 shutdown_connections 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.321+0000 7fdda876c640 1 -- 172.21.15.18:0/1225009458 wait complete. 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.321+0000 7fdda876c640 1 Processor -- start 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.321+0000 7fdda876c640 1 -- start start 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda876c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0100260 0x7fdda0197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda876c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdda0105330 0x7fdda0197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda876c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdda019bff0 0x7fdda019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda876c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda019c9e0 con 0x7fdda0105330 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda876c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda019cb50 con 0x7fdda0100260 2024-02-17T12:57:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda876c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda019ccc0 con 0x7fdda019bff0 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda5d07640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdda0105330 0x7fdda0197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda5d07640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdda0105330 0x7fdda0197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35946/0 (socket says 172.21.15.18:35946) 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda5d07640 1 -- 172.21.15.18:0/3490903288 learned_addr learned my addr 172.21.15.18:0/3490903288 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda6508640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0100260 0x7fdda0197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda6d09640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdda019bff0 0x7fdda019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda5d07640 1 -- 172.21.15.18:0/3490903288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdda019bff0 msgr2=0x7fdda019c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda5d07640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdda019bff0 0x7fdda019c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda5d07640 1 -- 172.21.15.18:0/3490903288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0100260 msgr2=0x7fdda0197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda5d07640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0100260 0x7fdda0197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda5d07640 1 -- 172.21.15.18:0/3490903288 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c009490 con 0x7fdda0105330 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda5d07640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdda0105330 0x7fdda0197c70 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000e9c0 tx=0x7fdd9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdd977fe640 1 -- 172.21.15.18:0/3490903288 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90009700 con 0x7fdda0105330 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdd977fe640 1 -- 172.21.15.18:0/3490903288 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd9000cd80 con 0x7fdda0105330 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.322+0000 7fdda876c640 1 -- 172.21.15.18:0/3490903288 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd98009900 con 0x7fdda0105330 2024-02-17T12:57:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.323+0000 7fdd977fe640 1 -- 172.21.15.18:0/3490903288 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90010430 con 0x7fdda0105330 2024-02-17T12:57:54.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.323+0000 7fdda876c640 1 -- 172.21.15.18:0/3490903288 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda01a4610 con 0x7fdda0105330 2024-02-17T12:57:54.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.325+0000 7fdd977fe640 1 -- 172.21.15.18:0/3490903288 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fdd90005000 con 0x7fdda0105330 2024-02-17T12:57:54.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fddab418740 1 Processor -- start 2024-02-17T12:57:54.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fddab418740 1 -- start start 2024-02-17T12:57:54.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fddab418740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb552b0 0x560b8bb55690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fddab418740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb55bd0 0x560b8bb5e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fddab418740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb5e7b0 0x560b8bb60ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fddab418740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8bac4280 con 0x560b8bb5e7b0 2024-02-17T12:57:54.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fddab418740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8bac17b0 con 0x560b8bb55bd0 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fddab418740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8babcf90 con 0x560b8bb552b0 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fdd977fe640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdd8c0a22b0 0x7fdd8c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fdda5d07640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb5e7b0 0x560b8bb60ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fdda6508640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb55bd0 0x560b8bb5e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fdda5d07640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb5e7b0 0x560b8bb60ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35960/0 (socket says 172.21.15.18:35960) 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fdda5d07640 1 -- 172.21.15.18:0/4098403474 learned_addr learned my addr 172.21.15.18:0/4098403474 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fdda6d09640 1 --2- 172.21.15.18:0/4098403474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb552b0 0x560b8bb55690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fdda6d09640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdd8c0a22b0 0x7fdd8c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.326+0000 7fdd977fe640 1 -- 172.21.15.18:0/3490903288 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdd90014070 con 0x7fdda0105330 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.327+0000 7fdda6508640 1 -- 172.21.15.18:0/4098403474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb552b0 msgr2=0x560b8bb55690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.327+0000 7fdda6508640 1 --2- 172.21.15.18:0/4098403474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb552b0 0x560b8bb55690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.327+0000 7fdda6508640 1 -- 172.21.15.18:0/4098403474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb5e7b0 msgr2=0x560b8bb60ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.327+0000 7fdda6508640 1 --2- 172.21.15.18:0/4098403474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb5e7b0 0x560b8bb60ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.327+0000 7fdda6508640 1 -- 172.21.15.18:0/4098403474 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b8ba7b570 con 0x560b8bb55bd0 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.327+0000 7fdda5d07640 1 --2- 172.21.15.18:0/4098403474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb5e7b0 0x560b8bb60ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.327+0000 7fdda6d09640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdd8c0a22b0 0x7fdd8c0a4770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdd98005cc0 tx=0x7fdd98005c50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.327+0000 7fdda6508640 1 --2- 172.21.15.18:0/4098403474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb55bd0 0x560b8bb5e270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009db0 tx=0x7fdd9c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ae2fde1cd8c709 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.327+0000 7fdd94ff9640 1 -- 172.21.15.18:0/4098403474 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c019410 con 0x560b8bb55bd0 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.327+0000 7fdd94ff9640 1 -- 172.21.15.18:0/4098403474 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd9c019a50 con 0x560b8bb55bd0 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.327+0000 7fdd94ff9640 1 -- 172.21.15.18:0/4098403474 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c018a90 con 0x560b8bb55bd0 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.328+0000 7fddab418740 1 -- 172.21.15.18:0/4098403474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb55bd0 msgr2=0x560b8bb5e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.328+0000 7fddab418740 1 --2- 172.21.15.18:0/4098403474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb55bd0 0x560b8bb5e270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009db0 tx=0x7fdd9c011810 comp rx=0 tx=0).stop 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.328+0000 7fddab418740 1 -- 172.21.15.18:0/4098403474 shutdown_connections 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.328+0000 7fddab418740 1 --2- 172.21.15.18:0/4098403474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb5e7b0 0x560b8bb60ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.328+0000 7fddab418740 1 --2- 172.21.15.18:0/4098403474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb55bd0 0x560b8bb5e270 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.328+0000 7fddab418740 1 --2- 172.21.15.18:0/4098403474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb552b0 0x560b8bb55690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.328+0000 7fddab418740 1 -- 172.21.15.18:0/4098403474 >> 172.21.15.18:0/4098403474 conn(0x560b8bb4f990 msgr2=0x560b8bb4fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.328+0000 7fddab418740 1 -- 172.21.15.18:0/4098403474 shutdown_connections 2024-02-17T12:57:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.328+0000 7fddab418740 1 -- 172.21.15.18:0/4098403474 wait complete. 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.328+0000 7fddab418740 1 -- start start 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.328+0000 7fddab418740 1 Processor -- start 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fddab418740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb552b0 0x560b8bb5e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fddab418740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 0x560b8bb5b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fddab418740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb5e7b0 0x560b8bb5bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fddab418740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8b790430 con 0x560b8bb55bd0 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fddab418740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8b74ce00 con 0x560b8bb552b0 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fddab418740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ba41fd0 con 0x560b8bb5e7b0 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fddab418740 10 client.0 _ll_register_callbacks cb 0x560b8ba97fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdda5d07640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 0x560b8bb5b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdda6508640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb552b0 0x560b8bb5e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdda5d07640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 0x560b8bb5b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35974/0 (socket says 172.21.15.18:35974) 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdda5d07640 1 -- 172.21.15.18:0/3547446588 learned_addr learned my addr 172.21.15.18:0/3547446588 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdda6d09640 1 --2- 172.21.15.18:0/3547446588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb5e7b0 0x560b8bb5bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdda5d07640 1 -- 172.21.15.18:0/3547446588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb5e7b0 msgr2=0x560b8bb5bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdda5d07640 1 --2- 172.21.15.18:0/3547446588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb5e7b0 0x560b8bb5bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdda5d07640 1 -- 172.21.15.18:0/3547446588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb552b0 msgr2=0x560b8bb5e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdda5d07640 1 --2- 172.21.15.18:0/3547446588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb552b0 0x560b8bb5e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdda5d07640 1 -- 172.21.15.18:0/3547446588 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9c0099e0 con 0x560b8bb55bd0 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdda5d07640 1 --2- 172.21.15.18:0/3547446588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 0x560b8bb5b550 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fdd90097220 tx=0x7fdd90021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab60b28c05853a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdd89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdd89ffb640 1 -- 172.21.15.18:0/3547446588 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9009b020 con 0x560b8bb55bd0 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdd89ffb640 1 -- 172.21.15.18:0/3547446588 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdd9009e030 con 0x560b8bb55bd0 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fddab418740 1 -- 172.21.15.18:0/3547446588 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b8b7bc7e0 con 0x560b8bb55bd0 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fdd89ffb640 1 -- 172.21.15.18:0/3547446588 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd9009b4c0 con 0x560b8bb55bd0 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fddab418740 10 client.18501 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.329+0000 7fddab418740 1 -- 172.21.15.18:0/3547446588 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x560b8babb610 con 0x560b8bb55bd0 2024-02-17T12:57:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.330+0000 7fdd89ffb640 1 -- 172.21.15.18:0/3547446588 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd900b7080 con 0x560b8bb55bd0 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.330+0000 7fdd89ffb640 1 -- 172.21.15.18:0/3547446588 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fdd9001f060 con 0x560b8bb55bd0 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.330+0000 7fddab418740 10 client.18501 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.330+0000 7fddab418740 10 client.18501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.330+0000 7fddab418740 20 client.18501 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.330+0000 7fddab418740 1 --2- 172.21.15.18:0/3547446588 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560b8bb6e8f0 0x560b8bb70ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.330+0000 7fddab418740 4 client.18501 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.330+0000 7fddab418740 1 -- 172.21.15.18:0/3547446588 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b8b7f9720 con 0x560b8bb6e8f0 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.330+0000 7fdda6508640 1 --2- 172.21.15.18:0/3547446588 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560b8bb6e8f0 0x560b8bb70ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.330+0000 7fdda6508640 1 --2- 172.21.15.18:0/3547446588 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560b8bb6e8f0 0x560b8bb70ce0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=877de2729546631f server_cookie=25e7fea02c5c246e in_seq=0 out_seq=0 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.330+0000 7fdd89ffb640 10 client.18501 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fdd89ffb640 1 -- 172.21.15.18:0/3547446588 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560b8b7f9720 con 0x560b8bb6e8f0 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fdd89ffb640 10 client.18501 handle_command_reply: tid=0 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fddab418740 1 client.18501 shutdown 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fddab418740 1 -- 172.21.15.18:0/3547446588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 msgr2=0x560b8bb5b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fddab418740 1 --2- 172.21.15.18:0/3547446588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 0x560b8bb5b550 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fdd90097220 tx=0x7fdd90021000 comp rx=0 tx=0).stop 2024-02-17T12:57:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fddab418740 1 -- 172.21.15.18:0/3547446588 shutdown_connections 2024-02-17T12:57:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fddab418740 1 --2- 172.21.15.18:0/3547446588 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560b8bb6e8f0 0x560b8bb70ce0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fddab418740 1 --2- 172.21.15.18:0/3547446588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb5e7b0 0x560b8bb5bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fddab418740 1 --2- 172.21.15.18:0/3547446588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 0x560b8bb5b550 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fdd89ffb640 0 client.18501 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fddab418740 1 --2- 172.21.15.18:0/3547446588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb552b0 0x560b8bb5e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fddab418740 1 -- 172.21.15.18:0/3547446588 >> 172.21.15.18:0/3547446588 conn(0x560b8bb4f990 msgr2=0x560b8bb51730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fddab418740 1 -- 172.21.15.18:0/3547446588 shutdown_connections 2024-02-17T12:57:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fddab418740 1 -- 172.21.15.18:0/3547446588 wait complete. 2024-02-17T12:57:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.331+0000 7fddab418740 20 client.18501 trim_cache size 0 max 16384 2024-02-17T12:57:54.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.346+0000 7fddab418740 1 Processor -- start 2024-02-17T12:57:54.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.346+0000 7fddab418740 1 -- start start 2024-02-17T12:57:54.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.346+0000 7fddab418740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb552b0 0x560b8bb5dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.346+0000 7fddab418740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 0x560b8bb5e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.346+0000 7fddab418740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb5e7b0 0x560b8bb59e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.346+0000 7fddab418740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8bb509e0 con 0x560b8bb55bd0 2024-02-17T12:57:54.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.346+0000 7fddab418740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8bb75f10 con 0x560b8bb5e7b0 2024-02-17T12:57:54.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.346+0000 7fddab418740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8bb49e20 con 0x560b8bb552b0 2024-02-17T12:57:54.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.346+0000 7fdda5d07640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 0x560b8bb5e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdda5d07640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 0x560b8bb5e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35988/0 (socket says 172.21.15.18:35988) 2024-02-17T12:57:54.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdda5d07640 1 -- 172.21.15.18:0/3878729220 learned_addr learned my addr 172.21.15.18:0/3878729220 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdda6508640 1 --2- 172.21.15.18:0/3878729220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb552b0 0x560b8bb5dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdda6d09640 1 --2- 172.21.15.18:0/3878729220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb5e7b0 0x560b8bb59e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdda5d07640 1 -- 172.21.15.18:0/3878729220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb552b0 msgr2=0x560b8bb5dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdda5d07640 1 --2- 172.21.15.18:0/3878729220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb552b0 0x560b8bb5dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdda5d07640 1 -- 172.21.15.18:0/3878729220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb5e7b0 msgr2=0x560b8bb59e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdda5d07640 1 --2- 172.21.15.18:0/3878729220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb5e7b0 0x560b8bb59e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdda5d07640 1 -- 172.21.15.18:0/3878729220 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b8ba57ac0 con 0x560b8bb55bd0 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdda5d07640 1 --2- 172.21.15.18:0/3878729220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 0x560b8bb5e1c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fdd90000c00 tx=0x7fdd90004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595439a8ebdb8814 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdd8bfff640 1 -- 172.21.15.18:0/3878729220 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9009a720 con 0x560b8bb55bd0 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdd8bfff640 1 -- 172.21.15.18:0/3878729220 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560b8b7bc7e0 con 0x560b8bb55bd0 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.347+0000 7fdd8bfff640 1 -- 172.21.15.18:0/3878729220 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd900a3560 con 0x560b8bb55bd0 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 -- 172.21.15.18:0/3878729220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 msgr2=0x560b8bb5e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 --2- 172.21.15.18:0/3878729220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 0x560b8bb5e1c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fdd90000c00 tx=0x7fdd90004a50 comp rx=0 tx=0).stop 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 -- 172.21.15.18:0/3878729220 shutdown_connections 2024-02-17T12:57:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 --2- 172.21.15.18:0/3878729220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb5e7b0 0x560b8bb59e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 --2- 172.21.15.18:0/3878729220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb55bd0 0x560b8bb5e1c0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 --2- 172.21.15.18:0/3878729220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb552b0 0x560b8bb5dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 -- 172.21.15.18:0/3878729220 >> 172.21.15.18:0/3878729220 conn(0x560b8bb4f990 msgr2=0x560b8bb56810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 -- 172.21.15.18:0/3878729220 shutdown_connections 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 -- 172.21.15.18:0/3878729220 wait complete. 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 -- start start 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 Processor -- start 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb552b0 0x560b8bb65860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb55bd0 0x560b8bb61c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb5e7b0 0x560b8bb62170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8bb626b0 con 0x560b8bb552b0 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8bb62820 con 0x560b8bb5e7b0 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8bb629e0 con 0x560b8bb55bd0 2024-02-17T12:57:54.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fddab418740 10 client.0 _ll_register_callbacks cb 0x7fdd9c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:57:54.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fdda6508640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb552b0 0x560b8bb65860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.348+0000 7fdda6508640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb552b0 0x560b8bb65860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35996/0 (socket says 172.21.15.18:35996) 2024-02-17T12:57:54.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdda6508640 1 -- 172.21.15.18:0/4119144334 learned_addr learned my addr 172.21.15.18:0/4119144334 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:57:54.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdda5d07640 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb55bd0 0x560b8bb61c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdda6d09640 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb5e7b0 0x560b8bb62170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdda6508640 1 -- 172.21.15.18:0/4119144334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb55bd0 msgr2=0x560b8bb61c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdda6508640 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb55bd0 0x560b8bb61c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdda6508640 1 -- 172.21.15.18:0/4119144334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb5e7b0 msgr2=0x560b8bb62170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:57:54.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdda6508640 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb5e7b0 0x560b8bb62170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdda6508640 1 -- 172.21.15.18:0/4119144334 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b8babb610 con 0x560b8bb552b0 2024-02-17T12:57:54.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdda6508640 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb552b0 0x560b8bb65860 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x560b8bb54d80 tx=0x7fdd9c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c891ee06a6ff7f9a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:57:54.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdd89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdd89ffb640 1 -- 172.21.15.18:0/4119144334 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c029ea0 con 0x560b8bb552b0 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdd89ffb640 1 -- 172.21.15.18:0/4119144334 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdd9c0050b0 con 0x560b8bb552b0 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fddab418740 1 -- 172.21.15.18:0/4119144334 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd9c019a50 con 0x560b8bb552b0 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdd89ffb640 1 -- 172.21.15.18:0/4119144334 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd9c005a70 con 0x560b8bb552b0 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fddab418740 10 client.18507 fetch_fsmap learned FSMap version 44 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fddab418740 1 -- 172.21.15.18:0/4119144334 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x560b8ba7d6a0 con 0x560b8bb552b0 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.349+0000 7fdd89ffb640 1 -- 172.21.15.18:0/4119144334 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd9c019a50 con 0x560b8bb552b0 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.350+0000 7fdd89ffb640 1 -- 172.21.15.18:0/4119144334 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fdd9c018b50 con 0x560b8bb552b0 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.350+0000 7fddab418740 10 client.18507 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.350+0000 7fddab418740 10 client.18507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.350+0000 7fddab418740 20 client.18507 populate_metadata read hostname 'smithi018' 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.350+0000 7fddab418740 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560b8bbc63e0 0x560b8bbc8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.350+0000 7fddab418740 4 client.18507 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.350+0000 7fddab418740 1 -- 172.21.15.18:0/4119144334 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b8b848890 con 0x560b8bbc63e0 2024-02-17T12:57:54.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.350+0000 7fdda5d07640 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560b8bbc63e0 0x560b8bbc8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:57:54.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.350+0000 7fdda5d07640 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560b8bbc63e0 0x560b8bbc8800 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3146c8f5774c6d4f server_cookie=c2766649cc9485b4 in_seq=0 out_seq=0 2024-02-17T12:57:54.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.350+0000 7fdd89ffb640 10 client.18507 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:54.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:53 smithi119 ceph-mon[36188]: pgmap v248: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-02-17T12:57:54.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fdd89ffb640 1 -- 172.21.15.18:0/4119144334 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560b8b848890 con 0x560b8bbc63e0 2024-02-17T12:57:54.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fdd89ffb640 10 client.18507 handle_command_reply: tid=0 2024-02-17T12:57:54.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fddab418740 1 client.18507 shutdown 2024-02-17T12:57:54.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fddab418740 1 -- 172.21.15.18:0/4119144334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb552b0 msgr2=0x560b8bb65860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fddab418740 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb552b0 0x560b8bb65860 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x560b8bb54d80 tx=0x7fdd9c009f60 comp rx=0 tx=0).stop 2024-02-17T12:57:54.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fddab418740 1 -- 172.21.15.18:0/4119144334 shutdown_connections 2024-02-17T12:57:54.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fddab418740 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560b8bbc63e0 0x560b8bbc8800 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fddab418740 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b8bb5e7b0 0x560b8bb62170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fddab418740 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b8bb55bd0 0x560b8bb61c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fdd89ffb640 0 client.18507 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:57:54.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fddab418740 1 --2- 172.21.15.18:0/4119144334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b8bb552b0 0x560b8bb65860 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fddab418740 1 -- 172.21.15.18:0/4119144334 >> 172.21.15.18:0/4119144334 conn(0x560b8bb4f990 msgr2=0x560b8bb5ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fddab418740 1 -- 172.21.15.18:0/4119144334 shutdown_connections 2024-02-17T12:57:54.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.351+0000 7fddab418740 1 -- 172.21.15.18:0/4119144334 wait complete. 2024-02-17T12:57:54.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fddab418740 20 client.18507 trim_cache size 0 max 16384 2024-02-17T12:57:54.349 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:57:54.349 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:57:54.349 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:57:54.349 INFO:teuthology.orchestra.run.smithi018.stdout: "09112172-dd48-4e61-8c93-9efc4a5cd0a3": { 2024-02-17T12:57:54.349 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:57:54.349 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "09112172-dd48-4e61-8c93-9efc4a5cd0a3", 2024-02-17T12:57:54.349 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:57:54.349 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:57:54.349 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:57:54.349 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:57:54.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fdda876c640 1 -- 172.21.15.18:0/3490903288 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdd8c0a22b0 msgr2=0x7fdd8c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fdda876c640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdd8c0a22b0 0x7fdd8c0a4770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdd98005cc0 tx=0x7fdd98005c50 comp rx=0 tx=0).stop 2024-02-17T12:57:54.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fdda876c640 1 -- 172.21.15.18:0/3490903288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdda0105330 msgr2=0x7fdda0197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:57:54.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fdda876c640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdda0105330 0x7fdda0197c70 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000e9c0 tx=0x7fdd9000ee80 comp rx=0 tx=0).stop 2024-02-17T12:57:54.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fdda876c640 1 -- 172.21.15.18:0/3490903288 shutdown_connections 2024-02-17T12:57:54.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fdda876c640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdd8c0a22b0 0x7fdd8c0a4770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fdda876c640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdda019bff0 0x7fdda019c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fdda876c640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdda0105330 0x7fdda0197c70 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fdda876c640 1 --2- 172.21.15.18:0/3490903288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0100260 0x7fdda0197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:57:54.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fdda876c640 1 -- 172.21.15.18:0/3490903288 >> 172.21.15.18:0/3490903288 conn(0x7fdda00771b0 msgr2=0x7fdda00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:57:54.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fdda876c640 1 -- 172.21.15.18:0/3490903288 shutdown_connections 2024-02-17T12:57:54.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:57:54.352+0000 7fdda876c640 1 -- 172.21.15.18:0/3490903288 wait complete. 2024-02-17T12:57:54.353 INFO:tasks.cephfs.filesystem:scrub status for tag:09112172-dd48-4e61-8c93-9efc4a5cd0a3 - {'path': '/', 'tag': '09112172-dd48-4e61-8c93-9efc4a5cd0a3', 'options': 'recursive,force'} 2024-02-17T12:57:55.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:54 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:55.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:54 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:55.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:54 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:55.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:54 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:55.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:54 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:55.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:54 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:55.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:54 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:55.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:54 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:55.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 416425 5.40 MB/sec execute 310 sec latency 12.354 ms 2024-02-17T12:57:55.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:54 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:55.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:54 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:55.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:54 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:55.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:54 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:57:56.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:56.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:56.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:56.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.201 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:55 smithi146 ceph-mon[36454]: pgmap v249: 97 pgs: 97 active+clean; 146 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-02-17T12:57:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:57:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:56.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:56.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:56.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:56.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:56.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:56.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:56.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:57:56.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:55 smithi018 ceph-mon[32625]: pgmap v249: 97 pgs: 97 active+clean; 146 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-02-17T12:57:56.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 416732 5.39 MB/sec execute 311 sec latency 666.777 ms 2024-02-17T12:57:56.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:56.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:57:56.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:57:56.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:55 smithi119 ceph-mon[36188]: pgmap v249: 97 pgs: 97 active+clean; 146 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-02-17T12:57:57.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 416732 5.37 MB/sec execute 312 sec latency 1666.833 ms 2024-02-17T12:57:58.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:57 smithi146 ceph-mon[36454]: pgmap v250: 97 pgs: 97 active+clean; 146 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-17T12:57:58.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:57 smithi018 ceph-mon[32625]: pgmap v250: 97 pgs: 97 active+clean; 146 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-17T12:57:58.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 416732 5.35 MB/sec execute 313 sec latency 2666.882 ms 2024-02-17T12:57:58.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:57 smithi119 ceph-mon[36188]: pgmap v250: 97 pgs: 97 active+clean; 146 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-17T12:57:59.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 417611 5.34 MB/sec execute 314 sec latency 3273.678 ms 2024-02-17T12:58:00.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:57:59 smithi146 ceph-mon[36454]: pgmap v251: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 202 op/s 2024-02-17T12:58:00.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:57:59 smithi018 ceph-mon[32625]: pgmap v251: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 202 op/s 2024-02-17T12:58:00.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 419192 5.36 MB/sec execute 315 sec latency 11.930 ms 2024-02-17T12:58:00.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:57:59 smithi119 ceph-mon[36188]: pgmap v251: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 202 op/s 2024-02-17T12:58:01.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:00 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:58:01.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:00 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:58:01.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:00 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:58:01.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:00 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:58:01.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:00 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:58:01.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:00 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:58:01.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 420275 5.36 MB/sec execute 316 sec latency 397.988 ms 2024-02-17T12:58:01.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:00 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:58:01.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:00 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:58:01.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:00 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:58:02.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:01 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:58:02.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:01 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:58:02.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:01 smithi146 ceph-mon[36454]: pgmap v252: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-02-17T12:58:02.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:01 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:58:02.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:01 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:58:02.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:01 smithi018 ceph-mon[32625]: pgmap v252: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-02-17T12:58:02.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 420275 5.35 MB/sec execute 317 sec latency 1398.062 ms 2024-02-17T12:58:02.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:01 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:58:02.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:01 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:58:02.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:01 smithi119 ceph-mon[36188]: pgmap v252: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-02-17T12:58:03.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 420854 5.33 MB/sec execute 318 sec latency 1969.268 ms 2024-02-17T12:58:04.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:03 smithi146 ceph-mon[36454]: pgmap v253: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-02-17T12:58:04.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:03 smithi018 ceph-mon[32625]: pgmap v253: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-02-17T12:58:04.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 422356 5.35 MB/sec execute 319 sec latency 22.259 ms 2024-02-17T12:58:04.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:03 smithi119 ceph-mon[36188]: pgmap v253: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-02-17T12:58:05.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:04 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:58:05.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:04 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:58:05.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:04 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:58:05.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:04 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:58:05.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:04 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:58:05.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:04 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:58:05.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:04 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:58:05.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:04 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:58:05.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 423601 5.35 MB/sec execute 320 sec latency 17.368 ms 2024-02-17T12:58:05.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:04 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:58:05.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:04 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:58:05.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:04 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:58:05.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:04 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:58:06.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:05 smithi146 ceph-mon[36454]: pgmap v254: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-02-17T12:58:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:05 smithi018 ceph-mon[32625]: pgmap v254: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-02-17T12:58:06.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 423824 5.34 MB/sec execute 321 sec latency 755.647 ms 2024-02-17T12:58:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:05 smithi119 ceph-mon[36188]: pgmap v254: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-02-17T12:58:07.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:06 smithi146 ceph-mon[36454]: osdmap e95: 12 total, 12 up, 12 in 2024-02-17T12:58:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.276+0000 7fe6b6548640 1 -- 172.21.15.18:0/3243705959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6b0068800 msgr2=0x7fe6b0068c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.276+0000 7fe6b6548640 1 --2- 172.21.15.18:0/3243705959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6b0068800 0x7fe6b0068c60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe698009900 tx=0x7fe69802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.276+0000 7fe6b6548640 1 -- 172.21.15.18:0/3243705959 shutdown_connections 2024-02-17T12:58:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.276+0000 7fe6b6548640 1 --2- 172.21.15.18:0/3243705959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6b00691a0 0x7fe6b010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.276+0000 7fe6b6548640 1 --2- 172.21.15.18:0/3243705959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6b0068800 0x7fe6b0068c60 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.276+0000 7fe6b6548640 1 --2- 172.21.15.18:0/3243705959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6b0107270 0x7fe6b0107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.276+0000 7fe6b6548640 1 -- 172.21.15.18:0/3243705959 >> 172.21.15.18:0/3243705959 conn(0x7fe6b0076ed0 msgr2=0x7fe6b00772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.276+0000 7fe6b6548640 1 -- 172.21.15.18:0/3243705959 shutdown_connections 2024-02-17T12:58:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.276+0000 7fe6b6548640 1 -- 172.21.15.18:0/3243705959 wait complete. 2024-02-17T12:58:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.276+0000 7fe6b6548640 1 Processor -- start 2024-02-17T12:58:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.276+0000 7fe6b6548640 1 -- start start 2024-02-17T12:58:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b6548640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6b0068800 0x7fe6b01976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b6548640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6b00691a0 0x7fe6b0197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b6548640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6b0107270 0x7fe6b019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b6548640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b019c540 con 0x7fe6b0107270 2024-02-17T12:58:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b6548640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b019c6b0 con 0x7fe6b00691a0 2024-02-17T12:58:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b6548640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b019c870 con 0x7fe6b0068800 2024-02-17T12:58:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b4ae5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6b0107270 0x7fe6b019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b4ae5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6b0107270 0x7fe6b019bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44410/0 (socket says 172.21.15.18:44410) 2024-02-17T12:58:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b4ae5640 1 -- 172.21.15.18:0/4127601897 learned_addr learned my addr 172.21.15.18:0/4127601897 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6affff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6b0068800 0x7fe6b01976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6af7fe640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6b00691a0 0x7fe6b0197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b4ae5640 1 -- 172.21.15.18:0/4127601897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6b0068800 msgr2=0x7fe6b01976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b4ae5640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6b0068800 0x7fe6b01976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b4ae5640 1 -- 172.21.15.18:0/4127601897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6b00691a0 msgr2=0x7fe6b0197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b4ae5640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6b00691a0 0x7fe6b0197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b4ae5640 1 -- 172.21.15.18:0/4127601897 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe698009560 con 0x7fe6b0107270 2024-02-17T12:58:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.277+0000 7fe6b4ae5640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6b0107270 0x7fe6b019bf40 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800ea60 tx=0x7fe6a800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.278+0000 7fe6ad7fa640 1 -- 172.21.15.18:0/4127601897 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a800ce40 con 0x7fe6b0107270 2024-02-17T12:58:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.278+0000 7fe6ad7fa640 1 -- 172.21.15.18:0/4127601897 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6a8005020 con 0x7fe6b0107270 2024-02-17T12:58:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.278+0000 7fe6b6548640 1 -- 172.21.15.18:0/4127601897 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b019cb50 con 0x7fe6b0107270 2024-02-17T12:58:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.278+0000 7fe6ad7fa640 1 -- 172.21.15.18:0/4127601897 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8010690 con 0x7fe6b0107270 2024-02-17T12:58:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.278+0000 7fe6b6548640 1 -- 172.21.15.18:0/4127601897 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b01a43d0 con 0x7fe6b0107270 2024-02-17T12:58:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.280+0000 7fe6ad7fa640 1 -- 172.21.15.18:0/4127601897 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe6a8005500 con 0x7fe6b0107270 2024-02-17T12:58:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.280+0000 7fe6b91f7740 1 Processor -- start 2024-02-17T12:58:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.280+0000 7fe6b91f7740 1 -- start start 2024-02-17T12:58:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.280+0000 7fe6b91f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac178ad0 0x55cbac177820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.280+0000 7fe6b91f7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac177d60 0x55cbac1781e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.280+0000 7fe6b91f7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 0x55cbac1836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.280+0000 7fe6b91f7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac0e69f0 con 0x55cbac1812e0 2024-02-17T12:58:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.280+0000 7fe6b91f7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac0e3f20 con 0x55cbac177d60 2024-02-17T12:58:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.280+0000 7fe6b91f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac0df700 con 0x55cbac178ad0 2024-02-17T12:58:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6ad7fa640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe69c0a22d0 0x7fe69c0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6af7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 0x55cbac1836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6af7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 0x55cbac1836d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44420/0 (socket says 172.21.15.18:44420) 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6af7fe640 1 -- 172.21.15.18:0/1620844896 learned_addr learned my addr 172.21.15.18:0/1620844896 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6affff640 1 --2- 172.21.15.18:0/1620844896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac177d60 0x55cbac1781e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6b4ae5640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe69c0a22d0 0x7fe69c0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6af7fe640 1 --2- 172.21.15.18:0/1620844896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac178ad0 0x55cbac177820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6ad7fa640 1 -- 172.21.15.18:0/4127601897 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fe6a8014070 con 0x7fe6b0107270 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6affff640 1 -- 172.21.15.18:0/1620844896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac178ad0 msgr2=0x55cbac177820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6affff640 1 --2- 172.21.15.18:0/1620844896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac178ad0 0x55cbac177820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6affff640 1 -- 172.21.15.18:0/1620844896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 msgr2=0x55cbac1836d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6affff640 1 --2- 172.21.15.18:0/1620844896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 0x55cbac1836d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6affff640 1 -- 172.21.15.18:0/1620844896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbac0757b0 con 0x55cbac177d60 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6affff640 1 --2- 172.21.15.18:0/1620844896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac177d60 0x55cbac1781e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009c20 tx=0x7fe6a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52433d3e2fc7c8e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6b4ae5640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe69c0a22d0 0x7fe69c0a4790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe6a809b050 tx=0x7fe6a800ef90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.281+0000 7fe6a67fc640 1 -- 172.21.15.18:0/1620844896 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0004050 con 0x55cbac177d60 2024-02-17T12:58:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.282+0000 7fe6a67fc640 1 -- 172.21.15.18:0/1620844896 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6a0005500 con 0x55cbac177d60 2024-02-17T12:58:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.282+0000 7fe6a67fc640 1 -- 172.21.15.18:0/1620844896 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0004d30 con 0x55cbac177d60 2024-02-17T12:58:07.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:06 smithi018 ceph-mon[32625]: osdmap e95: 12 total, 12 up, 12 in 2024-02-17T12:58:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.282+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1620844896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac177d60 msgr2=0x55cbac1781e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.282+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/1620844896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac177d60 0x55cbac1781e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009c20 tx=0x7fe6a0011810 comp rx=0 tx=0).stop 2024-02-17T12:58:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.282+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1620844896 shutdown_connections 2024-02-17T12:58:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.282+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/1620844896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 0x55cbac1836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.282+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/1620844896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac177d60 0x55cbac1781e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.282+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/1620844896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac178ad0 0x55cbac177820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.282+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1620844896 >> 172.21.15.18:0/1620844896 conn(0x55cbac13f550 msgr2=0x55cbac172a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.282+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1620844896 shutdown_connections 2024-02-17T12:58:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.282+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1620844896 wait complete. 2024-02-17T12:58:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6b91f7740 1 -- start start 2024-02-17T12:58:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6b91f7740 1 Processor -- start 2024-02-17T12:58:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6b91f7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 0x55cbac17e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6b91f7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 0x55cbac17e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6b91f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac1812e0 0x55cbac17ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6b91f7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac084bf0 con 0x55cbac177d60 2024-02-17T12:58:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6b91f7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbabdb3250 con 0x55cbac178ad0 2024-02-17T12:58:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6b91f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac065200 con 0x55cbac1812e0 2024-02-17T12:58:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6b91f7740 10 client.0 _ll_register_callbacks cb 0x55cbac0d3d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6af7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 0x55cbac17e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6af7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 0x55cbac17e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44432/0 (socket says 172.21.15.18:44432) 2024-02-17T12:58:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6af7fe640 1 -- 172.21.15.18:0/2774077754 learned_addr learned my addr 172.21.15.18:0/2774077754 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.283+0000 7fe6affff640 1 --2- 172.21.15.18:0/2774077754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 0x55cbac17e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe6af7fe640 1 -- 172.21.15.18:0/2774077754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac1812e0 msgr2=0x55cbac17ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe6af7fe640 1 --2- 172.21.15.18:0/2774077754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac1812e0 0x55cbac17ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe6af7fe640 1 -- 172.21.15.18:0/2774077754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 msgr2=0x55cbac17e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe6af7fe640 1 --2- 172.21.15.18:0/2774077754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 0x55cbac17e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe6af7fe640 1 -- 172.21.15.18:0/2774077754 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a0009850 con 0x55cbac177d60 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe6af7fe640 1 --2- 172.21.15.18:0/2774077754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 0x55cbac17e280 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe698036040 tx=0x7fe69802c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7140a7bad2b44d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe68f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe68f7fe640 1 -- 172.21.15.18:0/2774077754 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698004410 con 0x55cbac177d60 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe68f7fe640 1 -- 172.21.15.18:0/2774077754 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fe69802c880 con 0x55cbac177d60 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe6b91f7740 1 -- 172.21.15.18:0/2774077754 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbabddf600 con 0x55cbac177d60 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe68f7fe640 1 -- 172.21.15.18:0/2774077754 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe69802ca40 con 0x55cbac177d60 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe6b91f7740 10 client.18519 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe6b91f7740 1 -- 172.21.15.18:0/2774077754 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55cbac0ddd80 con 0x55cbac177d60 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.284+0000 7fe68f7fe640 1 -- 172.21.15.18:0/2774077754 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cbabddf600 con 0x55cbac177d60 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.285+0000 7fe68f7fe640 1 -- 172.21.15.18:0/2774077754 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe69803c070 con 0x55cbac177d60 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.285+0000 7fe6b91f7740 10 client.18519 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.285+0000 7fe6b91f7740 10 client.18519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.285+0000 7fe6b91f7740 20 client.18519 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.285+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/2774077754 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cbac191580 0x55cbac193970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.285+0000 7fe6b91f7740 4 client.18519 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.285+0000 7fe6b91f7740 1 -- 172.21.15.18:0/2774077754 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbabe301b0 con 0x55cbac191580 2024-02-17T12:58:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.285+0000 7fe6affff640 1 --2- 172.21.15.18:0/2774077754 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cbac191580 0x55cbac193970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.285+0000 7fe6affff640 1 --2- 172.21.15.18:0/2774077754 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cbac191580 0x55cbac193970 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef0e6460778a2cbe server_cookie=c826cf1f373819c1 in_seq=0 out_seq=0 2024-02-17T12:58:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.285+0000 7fe68f7fe640 10 client.18519 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe68f7fe640 1 -- 172.21.15.18:0/2774077754 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cbabe301b0 con 0x55cbac191580 2024-02-17T12:58:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe68f7fe640 10 client.18519 handle_command_reply: tid=0 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe6b91f7740 1 client.18519 shutdown 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe6b91f7740 1 -- 172.21.15.18:0/2774077754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 msgr2=0x55cbac17e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/2774077754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 0x55cbac17e280 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe698036040 tx=0x7fe69802c770 comp rx=0 tx=0).stop 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe6b91f7740 1 -- 172.21.15.18:0/2774077754 shutdown_connections 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/2774077754 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cbac191580 0x55cbac193970 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/2774077754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac1812e0 0x55cbac17ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/2774077754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 0x55cbac17e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe68f7fe640 0 client.18519 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/2774077754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 0x55cbac17e280 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe6b91f7740 1 -- 172.21.15.18:0/2774077754 >> 172.21.15.18:0/2774077754 conn(0x55cbac13f550 msgr2=0x55cbac1747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe6b91f7740 1 -- 172.21.15.18:0/2774077754 shutdown_connections 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe6b91f7740 1 -- 172.21.15.18:0/2774077754 wait complete. 2024-02-17T12:58:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.286+0000 7fe6b91f7740 20 client.18519 trim_cache size 0 max 16384 2024-02-17T12:58:07.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 423824 5.32 MB/sec execute 322 sec latency 1755.677 ms 2024-02-17T12:58:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.300+0000 7fe6b91f7740 1 Processor -- start 2024-02-17T12:58:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.300+0000 7fe6b91f7740 1 -- start start 2024-02-17T12:58:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.300+0000 7fe6b91f7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 0x55cbac1809b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.300+0000 7fe6b91f7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 0x55cbac17cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.300+0000 7fe6b91f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac1812e0 0x55cbac17d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.300+0000 7fe6b91f7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac18d020 con 0x55cbac177d60 2024-02-17T12:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.300+0000 7fe6b91f7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac191380 con 0x55cbac178ad0 2024-02-17T12:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.300+0000 7fe6b91f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac17d5d0 con 0x55cbac1812e0 2024-02-17T12:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.300+0000 7fe6af7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 0x55cbac1809b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.300+0000 7fe6af7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 0x55cbac1809b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44434/0 (socket says 172.21.15.18:44434) 2024-02-17T12:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.300+0000 7fe6af7fe640 1 -- 172.21.15.18:0/3278712794 learned_addr learned my addr 172.21.15.18:0/3278712794 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.301+0000 7fe6affff640 1 --2- 172.21.15.18:0/3278712794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 0x55cbac17cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.300+0000 7fe6af7fe640 1 -- 172.21.15.18:0/3278712794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac1812e0 msgr2=0x55cbac17d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.301+0000 7fe6af7fe640 1 --2- 172.21.15.18:0/3278712794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac1812e0 0x55cbac17d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.301+0000 7fe6af7fe640 1 -- 172.21.15.18:0/3278712794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 msgr2=0x55cbac17cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.301+0000 7fe6af7fe640 1 --2- 172.21.15.18:0/3278712794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 0x55cbac17cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.301+0000 7fe6af7fe640 1 -- 172.21.15.18:0/3278712794 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbac07a7b0 con 0x55cbac177d60 2024-02-17T12:58:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.301+0000 7fe6af7fe640 1 --2- 172.21.15.18:0/3278712794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 0x55cbac1809b0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe698001eb0 tx=0x7fe698005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffaf642128f3ee97 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.301+0000 7fe6a5ffb640 1 -- 172.21.15.18:0/3278712794 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698004900 con 0x55cbac177d60 2024-02-17T12:58:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.301+0000 7fe6a5ffb640 1 -- 172.21.15.18:0/3278712794 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe698047070 con 0x55cbac177d60 2024-02-17T12:58:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.301+0000 7fe6a5ffb640 1 -- 172.21.15.18:0/3278712794 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698004df0 con 0x55cbac177d60 2024-02-17T12:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.301+0000 7fe6b91f7740 1 -- 172.21.15.18:0/3278712794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 msgr2=0x55cbac1809b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.301+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/3278712794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 0x55cbac1809b0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe698001eb0 tx=0x7fe698005a20 comp rx=0 tx=0).stop 2024-02-17T12:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 -- 172.21.15.18:0/3278712794 shutdown_connections 2024-02-17T12:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/3278712794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac1812e0 0x55cbac17d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/3278712794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 0x55cbac17cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/3278712794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac177d60 0x55cbac1809b0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 -- 172.21.15.18:0/3278712794 >> 172.21.15.18:0/3278712794 conn(0x55cbac13f550 msgr2=0x55cbac172580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 -- 172.21.15.18:0/3278712794 shutdown_connections 2024-02-17T12:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 -- 172.21.15.18:0/3278712794 wait complete. 2024-02-17T12:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 -- start start 2024-02-17T12:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 Processor -- start 2024-02-17T12:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac177d60 0x55cbac1883b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 0x55cbac186750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 0x55cbac186c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac1872d0 con 0x55cbac1812e0 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac187440 con 0x55cbac178ad0 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac187600 con 0x55cbac177d60 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.302+0000 7fe6b91f7740 10 client.0 _ll_register_callbacks cb 0x7fe6a00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe6af7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 0x55cbac186c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe6af7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 0x55cbac186c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44442/0 (socket says 172.21.15.18:44442) 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe6af7fe640 1 -- 172.21.15.18:0/1634810098 learned_addr learned my addr 172.21.15.18:0/1634810098 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe6affff640 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 0x55cbac186750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe6af7fe640 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac177d60 0x55cbac1883b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe6af7fe640 1 -- 172.21.15.18:0/1634810098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac177d60 msgr2=0x55cbac1883b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe6af7fe640 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac177d60 0x55cbac1883b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe6af7fe640 1 -- 172.21.15.18:0/1634810098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 msgr2=0x55cbac186750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe6af7fe640 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 0x55cbac186750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe6af7fe640 1 -- 172.21.15.18:0/1634810098 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbac0ddd80 con 0x55cbac1812e0 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe6af7fe640 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 0x55cbac186c90 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe698005db0 tx=0x7fe69802cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90118b509f994df server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe68f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe68f7fe640 1 -- 172.21.15.18:0/1634810098 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69805b070 con 0x55cbac1812e0 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe68f7fe640 1 -- 172.21.15.18:0/1634810098 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fe698000e40 con 0x55cbac1812e0 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.303+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1634810098 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbac17e0b0 con 0x55cbac1812e0 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.304+0000 7fe68f7fe640 1 -- 172.21.15.18:0/1634810098 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe698003a90 con 0x55cbac1812e0 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.304+0000 7fe6b91f7740 10 client.18525 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.304+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1634810098 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55cbac0a0540 con 0x55cbac1812e0 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.304+0000 7fe68f7fe640 1 -- 172.21.15.18:0/1634810098 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cbac17e0b0 con 0x55cbac1812e0 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.304+0000 7fe68f7fe640 1 -- 172.21.15.18:0/1634810098 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe698064020 con 0x55cbac1812e0 2024-02-17T12:58:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.304+0000 7fe6b91f7740 10 client.18525 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.304+0000 7fe6b91f7740 10 client.18525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.304+0000 7fe6b91f7740 20 client.18525 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.304+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cbac1e8d00 0x55cbac1eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.304+0000 7fe6b91f7740 4 client.18525 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:58:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.304+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1634810098 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fe6a0005500 con 0x55cbac1e8d00 2024-02-17T12:58:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.304+0000 7fe6affff640 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cbac1e8d00 0x55cbac1eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.305+0000 7fe68f7fe640 10 client.18525 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.305+0000 7fe6affff640 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cbac1e8d00 0x55cbac1eb0f0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6756a10fb3772f39 server_cookie=824e481db98a423b in_seq=0 out_seq=0 2024-02-17T12:58:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.344+0000 7fe68f7fe640 1 -- 172.21.15.18:0/1634810098 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fe6a0005500 con 0x55cbac1e8d00 2024-02-17T12:58:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.344+0000 7fe68f7fe640 10 client.18525 handle_command_reply: tid=0 2024-02-17T12:58:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.344+0000 7fe6b91f7740 1 client.18525 shutdown 2024-02-17T12:58:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1634810098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 msgr2=0x55cbac186c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 0x55cbac186c90 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe698005db0 tx=0x7fe69802cac0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1634810098 shutdown_connections 2024-02-17T12:58:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cbac1e8d00 0x55cbac1eb0f0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cbac1812e0 0x55cbac186c90 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cbac178ad0 0x55cbac186750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b91f7740 1 --2- 172.21.15.18:0/1634810098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbac177d60 0x55cbac1883b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe68f7fe640 0 client.18525 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1634810098 >> 172.21.15.18:0/1634810098 conn(0x55cbac13f550 msgr2=0x55cbac172580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1634810098 shutdown_connections 2024-02-17T12:58:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b91f7740 1 -- 172.21.15.18:0/1634810098 wait complete. 2024-02-17T12:58:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b91f7740 20 client.18525 trim_cache size 0 max 16384 2024-02-17T12:58:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:58:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:58:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:58:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:58:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b6548640 1 -- 172.21.15.18:0/4127601897 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe69c0a22d0 msgr2=0x7fe69c0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b6548640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe69c0a22d0 0x7fe69c0a4790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe6a809b050 tx=0x7fe6a800ef90 comp rx=0 tx=0).stop 2024-02-17T12:58:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b6548640 1 -- 172.21.15.18:0/4127601897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6b0107270 msgr2=0x7fe6b019bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.345+0000 7fe6b6548640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6b0107270 0x7fe6b019bf40 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800ea60 tx=0x7fe6a800ef20 comp rx=0 tx=0).stop 2024-02-17T12:58:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.346+0000 7fe6b6548640 1 -- 172.21.15.18:0/4127601897 shutdown_connections 2024-02-17T12:58:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.346+0000 7fe6b6548640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe69c0a22d0 0x7fe69c0a4790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.346+0000 7fe6b6548640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6b0107270 0x7fe6b019bf40 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.346+0000 7fe6b6548640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6b00691a0 0x7fe6b0197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.346+0000 7fe6b6548640 1 --2- 172.21.15.18:0/4127601897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6b0068800 0x7fe6b01976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.346+0000 7fe6b6548640 1 -- 172.21.15.18:0/4127601897 >> 172.21.15.18:0/4127601897 conn(0x7fe6b0076ed0 msgr2=0x7fe6b00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.346+0000 7fe6b6548640 1 -- 172.21.15.18:0/4127601897 shutdown_connections 2024-02-17T12:58:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:07.346+0000 7fe6b6548640 1 -- 172.21.15.18:0/4127601897 wait complete. 2024-02-17T12:58:07.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:06 smithi119 ceph-mon[36188]: osdmap e95: 12 total, 12 up, 12 in 2024-02-17T12:58:08.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:07 smithi146 ceph-mon[36454]: pgmap v256: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-02-17T12:58:08.242 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:07 smithi119 ceph-mon[36188]: pgmap v256: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-02-17T12:58:08.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:07 smithi018 ceph-mon[32625]: pgmap v256: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-02-17T12:58:08.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 423824 5.31 MB/sec execute 323 sec latency 2755.710 ms 2024-02-17T12:58:09.285 INFO:tasks.workunit.client.0.smithi018.stdout: 1 425763 5.33 MB/sec execute 324 sec latency 2946.310 ms 2024-02-17T12:58:10.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:09 smithi146 ceph-mon[36454]: pgmap v257: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-02-17T12:58:10.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:09 smithi018 ceph-mon[32625]: pgmap v257: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-02-17T12:58:10.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 427126 5.33 MB/sec execute 325 sec latency 15.589 ms 2024-02-17T12:58:10.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:09 smithi119 ceph-mon[36188]: pgmap v257: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-02-17T12:58:11.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 427365 5.32 MB/sec execute 326 sec latency 731.014 ms 2024-02-17T12:58:12.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:11 smithi146 ceph-mon[36454]: pgmap v258: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-02-17T12:58:12.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:11 smithi018 ceph-mon[32625]: pgmap v258: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-02-17T12:58:12.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 427365 5.30 MB/sec execute 327 sec latency 1731.059 ms 2024-02-17T12:58:12.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:11 smithi119 ceph-mon[36188]: pgmap v258: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-02-17T12:58:13.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 427365 5.28 MB/sec execute 328 sec latency 2731.178 ms 2024-02-17T12:58:14.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:13 smithi146 ceph-mon[36454]: pgmap v259: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-02-17T12:58:14.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:13 smithi146 ceph-mon[36454]: osdmap e96: 12 total, 12 up, 12 in 2024-02-17T12:58:14.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:13 smithi018 ceph-mon[32625]: pgmap v259: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-02-17T12:58:14.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:13 smithi018 ceph-mon[32625]: osdmap e96: 12 total, 12 up, 12 in 2024-02-17T12:58:14.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 428270 5.28 MB/sec execute 329 sec latency 3342.589 ms 2024-02-17T12:58:14.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:13 smithi119 ceph-mon[36188]: pgmap v259: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-02-17T12:58:14.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:13 smithi119 ceph-mon[36188]: osdmap e96: 12 total, 12 up, 12 in 2024-02-17T12:58:15.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 429944 5.29 MB/sec execute 330 sec latency 23.404 ms 2024-02-17T12:58:16.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:15 smithi146 ceph-mon[36454]: pgmap v261: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-02-17T12:58:16.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:15 smithi018 ceph-mon[32625]: pgmap v261: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-02-17T12:58:16.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 430914 5.29 MB/sec execute 331 sec latency 339.135 ms 2024-02-17T12:58:16.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:15 smithi119 ceph-mon[36188]: pgmap v261: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-02-17T12:58:17.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 430914 5.28 MB/sec execute 332 sec latency 1339.206 ms 2024-02-17T12:58:18.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:17 smithi146 ceph-mon[36454]: pgmap v262: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-02-17T12:58:18.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:17 smithi018 ceph-mon[32625]: pgmap v262: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-02-17T12:58:18.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 430914 5.26 MB/sec execute 333 sec latency 2339.305 ms 2024-02-17T12:58:18.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:17 smithi119 ceph-mon[36188]: pgmap v262: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-02-17T12:58:19.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 431582 5.25 MB/sec execute 334 sec latency 2945.184 ms 2024-02-17T12:58:20.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:19 smithi146 ceph-mon[36454]: pgmap v263: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-02-17T12:58:20.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:19 smithi018 ceph-mon[32625]: pgmap v263: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-02-17T12:58:20.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 433239 5.27 MB/sec execute 335 sec latency 11.877 ms 2024-02-17T12:58:20.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:19 smithi119 ceph-mon[36188]: pgmap v263: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-02-17T12:58:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.436+0000 7fcbe67d7640 1 -- 172.21.15.18:0/45493301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbe0105380 msgr2=0x7fcbe010cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.436+0000 7fcbe67d7640 1 --2- 172.21.15.18:0/45493301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbe0105380 0x7fcbe010cf50 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009cd0 tx=0x7fcbd802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.436+0000 7fcbe67d7640 1 -- 172.21.15.18:0/45493301 shutdown_connections 2024-02-17T12:58:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.436+0000 7fcbe67d7640 1 --2- 172.21.15.18:0/45493301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbe0105380 0x7fcbe010cf50 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.436+0000 7fcbe67d7640 1 --2- 172.21.15.18:0/45493301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe0100c60 0x7fcbe0104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.436+0000 7fcbe67d7640 1 --2- 172.21.15.18:0/45493301 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe01002b0 0x7fcbe0100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.436+0000 7fcbe67d7640 1 -- 172.21.15.18:0/45493301 >> 172.21.15.18:0/45493301 conn(0x7fcbe00771a0 msgr2=0x7fcbe00775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.436+0000 7fcbe67d7640 1 -- 172.21.15.18:0/45493301 shutdown_connections 2024-02-17T12:58:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.436+0000 7fcbe67d7640 1 -- 172.21.15.18:0/45493301 wait complete. 2024-02-17T12:58:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbe67d7640 1 Processor -- start 2024-02-17T12:58:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbe67d7640 1 -- start start 2024-02-17T12:58:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbe67d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe01002b0 0x7fcbe019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbe67d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe0105380 0x7fcbe019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbe67d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbe01975e0 0x7fcbe0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbe67d7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe0197fd0 con 0x7fcbe01975e0 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbe67d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe0198140 con 0x7fcbe0105380 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbe67d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe0198300 con 0x7fcbe01002b0 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbe4d74640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbe01975e0 0x7fcbe0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbdffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe01002b0 0x7fcbe019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbe4d74640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbe01975e0 0x7fcbe0197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42398/0 (socket says 172.21.15.18:42398) 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe0105380 0x7fcbe019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.437+0000 7fcbe4d74640 1 -- 172.21.15.18:0/1822376075 learned_addr learned my addr 172.21.15.18:0/1822376075 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.438+0000 7fcbdffff640 1 -- 172.21.15.18:0/1822376075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe0105380 msgr2=0x7fcbe019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.438+0000 7fcbdffff640 1 --2- 172.21.15.18:0/1822376075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe0105380 0x7fcbe019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.438+0000 7fcbdffff640 1 -- 172.21.15.18:0/1822376075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbe01975e0 msgr2=0x7fcbe0197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.438+0000 7fcbdffff640 1 --2- 172.21.15.18:0/1822376075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbe01975e0 0x7fcbe0197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.438+0000 7fcbdffff640 1 -- 172.21.15.18:0/1822376075 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd8009900 con 0x7fcbe01002b0 2024-02-17T12:58:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.438+0000 7fcbdffff640 1 --2- 172.21.15.18:0/1822376075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe01002b0 0x7fcbe019d430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000e9f0 tx=0x7fcbd000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.438+0000 7fcbdd7fa640 1 -- 172.21.15.18:0/1822376075 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000ccc0 con 0x7fcbe01002b0 2024-02-17T12:58:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.438+0000 7fcbdd7fa640 1 -- 172.21.15.18:0/1822376075 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbd000ce20 con 0x7fcbe01002b0 2024-02-17T12:58:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.438+0000 7fcbe67d7640 1 -- 172.21.15.18:0/1822376075 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe01a42f0 con 0x7fcbe01002b0 2024-02-17T12:58:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.438+0000 7fcbdd7fa640 1 -- 172.21.15.18:0/1822376075 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0018920 con 0x7fcbe01002b0 2024-02-17T12:58:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.438+0000 7fcbe67d7640 1 -- 172.21.15.18:0/1822376075 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe01a4800 con 0x7fcbe01002b0 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbe93ad740 1 Processor -- start 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbdd7fa640 1 -- 172.21.15.18:0/1822376075 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcbd0018a80 con 0x7fcbe01002b0 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbe93ad740 1 -- start start 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbe93ad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e13717a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbe93ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e13717ad60 0x55e13717b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbe93ad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1371842e0 0x55e1371866d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbe93ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1370e99f0 con 0x55e13717ad60 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbe93ad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1370e6f20 con 0x55e1371842e0 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbe93ad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1370e2700 con 0x55e13717bad0 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbe4d74640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e13717a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbe4d74640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e13717a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40734/0 (socket says 172.21.15.18:40734) 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbe4d74640 1 -- 172.21.15.18:0/885116626 learned_addr learned my addr 172.21.15.18:0/885116626 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbdffff640 1 --2- 172.21.15.18:0/885116626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e13717ad60 0x55e13717b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.441+0000 7fcbdf7fe640 1 --2- 172.21.15.18:0/885116626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1371842e0 0x55e1371866d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.442+0000 7fcbe4d74640 1 -- 172.21.15.18:0/885116626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1371842e0 msgr2=0x55e1371866d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.442+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/885116626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1371842e0 0x55e1371866d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.442+0000 7fcbe4d74640 1 -- 172.21.15.18:0/885116626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e13717ad60 msgr2=0x55e13717b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.442+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/885116626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e13717ad60 0x55e13717b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.442+0000 7fcbe4d74640 1 -- 172.21.15.18:0/885116626 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1370787b0 con 0x55e13717bad0 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.442+0000 7fcbdd7fa640 1 --2- 172.21.15.18:0/1822376075 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcbc80a22d0 0x7fcbc80a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.442+0000 7fcbdffff640 1 --2- 172.21.15.18:0/1822376075 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcbc80a22d0 0x7fcbc80a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.442+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/885116626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e13717a820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8005fb0 tx=0x7fcbd8032d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff329cf72af7ddf3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.442+0000 7fcbdd7fa640 1 -- 172.21.15.18:0/1822376075 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fcbd0014070 con 0x7fcbe01002b0 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.442+0000 7fcbc67fc640 1 -- 172.21.15.18:0/885116626 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8034440 con 0x55e13717bad0 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.442+0000 7fcbc67fc640 1 -- 172.21.15.18:0/885116626 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbd80345a0 con 0x55e13717bad0 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.442+0000 7fcbc67fc640 1 -- 172.21.15.18:0/885116626 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8003bd0 con 0x55e13717bad0 2024-02-17T12:58:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.443+0000 7fcbe93ad740 1 -- 172.21.15.18:0/885116626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 msgr2=0x55e13717a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.443+0000 7fcbdffff640 1 --2- 172.21.15.18:0/1822376075 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcbc80a22d0 0x7fcbc80a4790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00cbb40 tx=0x7fcbd00d1c70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.443+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/885116626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e13717a820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8005fb0 tx=0x7fcbd8032d70 comp rx=0 tx=0).stop 2024-02-17T12:58:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.443+0000 7fcbe93ad740 1 -- 172.21.15.18:0/885116626 shutdown_connections 2024-02-17T12:58:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.443+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/885116626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1371842e0 0x55e1371866d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.443+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/885116626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e13717ad60 0x55e13717b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.443+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/885116626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e13717a820 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.443+0000 7fcbe93ad740 1 -- 172.21.15.18:0/885116626 >> 172.21.15.18:0/885116626 conn(0x55e137142550 msgr2=0x55e137175a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.443+0000 7fcbe93ad740 1 -- 172.21.15.18:0/885116626 shutdown_connections 2024-02-17T12:58:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.443+0000 7fcbe93ad740 1 -- 172.21.15.18:0/885116626 wait complete. 2024-02-17T12:58:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.443+0000 7fcbe93ad740 1 -- start start 2024-02-17T12:58:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbe93ad740 1 Processor -- start 2024-02-17T12:58:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbe93ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e13717ad60 0x55e137181340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbe93ad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e137181880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbe93ad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1371842e0 0x55e137181dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbe93ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e137087bf0 con 0x55e13717ad60 2024-02-17T12:58:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbe93ad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e136db6250 con 0x55e1371842e0 2024-02-17T12:58:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbe93ad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e137068200 con 0x55e13717bad0 2024-02-17T12:58:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbe93ad740 10 client.0 _ll_register_callbacks cb 0x55e1370d6d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbe4d74640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e13717ad60 0x55e137181340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e137181880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e137181880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40742/0 (socket says 172.21.15.18:40742) 2024-02-17T12:58:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbdf7fe640 1 -- 172.21.15.18:0/3893081848 learned_addr learned my addr 172.21.15.18:0/3893081848 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.444+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/3893081848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1371842e0 0x55e137181dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbdf7fe640 1 -- 172.21.15.18:0/3893081848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1371842e0 msgr2=0x55e137181dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbdf7fe640 1 --2- 172.21.15.18:0/3893081848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1371842e0 0x55e137181dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbdf7fe640 1 -- 172.21.15.18:0/3893081848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e13717ad60 msgr2=0x55e137181340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbdf7fe640 1 --2- 172.21.15.18:0/3893081848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e13717ad60 0x55e137181340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbdf7fe640 1 -- 172.21.15.18:0/3893081848 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe010d760 con 0x55e13717bad0 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbdf7fe640 1 --2- 172.21.15.18:0/3893081848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e137181880 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4012700 tx=0x7fcbd4012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=648418a2a6cbf5d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbbf7fe640 1 -- 172.21.15.18:0/3893081848 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4017070 con 0x55e13717bad0 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbe93ad740 1 -- 172.21.15.18:0/3893081848 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e136de2600 con 0x55e13717bad0 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbbf7fe640 1 -- 172.21.15.18:0/3893081848 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fcbd4004030 con 0x55e13717bad0 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbbf7fe640 1 -- 172.21.15.18:0/3893081848 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbd40041f0 con 0x55e13717bad0 2024-02-17T12:58:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbe93ad740 10 client.27706 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.445+0000 7fcbe93ad740 1 -- 172.21.15.18:0/3893081848 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55e1370e0d80 con 0x55e13717bad0 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.446+0000 7fcbbf7fe640 1 -- 172.21.15.18:0/3893081848 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e136de2600 con 0x55e13717bad0 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.446+0000 7fcbbf7fe640 1 -- 172.21.15.18:0/3893081848 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fcbd40038f0 con 0x55e13717bad0 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.446+0000 7fcbe93ad740 10 client.27706 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.446+0000 7fcbe93ad740 10 client.27706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.446+0000 7fcbe93ad740 20 client.27706 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.446+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/3893081848 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e137198a50 0x55e13719ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.446+0000 7fcbe93ad740 4 client.27706 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.446+0000 7fcbe93ad740 1 -- 172.21.15.18:0/3893081848 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e136e331b0 con 0x55e137198a50 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.446+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/3893081848 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e137198a50 0x55e13719ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.447+0000 7fcbbf7fe640 10 client.27706 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.447+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/3893081848 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e137198a50 0x55e13719ae70 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=449363003e44eaf0 server_cookie=5ede01f0872c4a93 in_seq=0 out_seq=0 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbbf7fe640 1 -- 172.21.15.18:0/3893081848 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e136e331b0 con 0x55e137198a50 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbbf7fe640 10 client.27706 handle_command_reply: tid=0 2024-02-17T12:58:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbe93ad740 1 client.27706 shutdown 2024-02-17T12:58:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbe93ad740 1 -- 172.21.15.18:0/3893081848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 msgr2=0x55e137181880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/3893081848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e137181880 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4012700 tx=0x7fcbd4012bc0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbe93ad740 1 -- 172.21.15.18:0/3893081848 shutdown_connections 2024-02-17T12:58:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/3893081848 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e137198a50 0x55e13719ae70 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/3893081848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1371842e0 0x55e137181dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/3893081848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e137181880 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/3893081848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e13717ad60 0x55e137181340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbbf7fe640 0 client.27706 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbe93ad740 1 -- 172.21.15.18:0/3893081848 >> 172.21.15.18:0/3893081848 conn(0x55e137142550 msgr2=0x55e1371777a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbe93ad740 1 -- 172.21.15.18:0/3893081848 shutdown_connections 2024-02-17T12:58:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbe93ad740 1 -- 172.21.15.18:0/3893081848 wait complete. 2024-02-17T12:58:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.448+0000 7fcbe93ad740 20 client.27706 trim_cache size 0 max 16384 2024-02-17T12:58:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.462+0000 7fcbe93ad740 1 Processor -- start 2024-02-17T12:58:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.462+0000 7fcbe93ad740 1 -- start start 2024-02-17T12:58:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe93ad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717ad60 0x55e137183a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe93ad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e13717bad0 0x55e137181be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe93ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 0x55e13717fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe93ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e137176a50 con 0x55e1371842e0 2024-02-17T12:58:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe93ad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1371a00c0 con 0x55e13717bad0 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe93ad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13717a320 con 0x55e13717ad60 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe4d74640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 0x55e13717fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e13717bad0 0x55e137181be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe4d74640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 0x55e13717fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42428/0 (socket says 172.21.15.18:42428) 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e13717bad0 0x55e137181be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:35850/0 (socket says 172.21.15.18:35850) 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe4d74640 1 -- 172.21.15.18:0/1967706178 learned_addr learned my addr 172.21.15.18:0/1967706178 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/1967706178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717ad60 0x55e137183a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe4d74640 1 -- 172.21.15.18:0/1967706178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717ad60 msgr2=0x55e137183a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/1967706178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717ad60 0x55e137183a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe4d74640 1 -- 172.21.15.18:0/1967706178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e13717bad0 msgr2=0x55e137181be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/1967706178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e13717bad0 0x55e137181be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbe4d74640 1 -- 172.21.15.18:0/1967706178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e13707d7b0 con 0x55e1371842e0 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.463+0000 7fcbdf7fe640 1 --2- 172.21.15.18:0/1967706178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e13717bad0 0x55e137181be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.464+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/1967706178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 0x55e13717fc10 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80329f0 tx=0x7fcbd8032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973b9726b5950cf2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.464+0000 7fcbc5ffb640 1 -- 172.21.15.18:0/1967706178 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8032a90 con 0x55e1371842e0 2024-02-17T12:58:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.464+0000 7fcbc5ffb640 1 -- 172.21.15.18:0/1967706178 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbd8032340 con 0x55e1371842e0 2024-02-17T12:58:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.464+0000 7fcbc5ffb640 1 -- 172.21.15.18:0/1967706178 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd80583c0 con 0x55e1371842e0 2024-02-17T12:58:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.464+0000 7fcbe93ad740 1 -- 172.21.15.18:0/1967706178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 msgr2=0x55e13717fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.464+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/1967706178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 0x55e13717fc10 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80329f0 tx=0x7fcbd8032980 comp rx=0 tx=0).stop 2024-02-17T12:58:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.464+0000 7fcbe93ad740 1 -- 172.21.15.18:0/1967706178 shutdown_connections 2024-02-17T12:58:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.464+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/1967706178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 0x55e13717fc10 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.464+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/1967706178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e13717bad0 0x55e137181be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.464+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/1967706178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717ad60 0x55e137183a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe93ad740 1 -- 172.21.15.18:0/1967706178 >> 172.21.15.18:0/1967706178 conn(0x55e137142550 msgr2=0x55e137175580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe93ad740 1 -- 172.21.15.18:0/1967706178 shutdown_connections 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe93ad740 1 -- 172.21.15.18:0/1967706178 wait complete. 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe93ad740 1 -- start start 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe93ad740 1 Processor -- start 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe93ad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e13717ad60 0x55e13718f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe93ad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e13718f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe93ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 0x55e13718d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe93ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13718dc50 con 0x55e1371842e0 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe93ad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13718ddc0 con 0x55e13717ad60 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe93ad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13718df80 con 0x55e13717bad0 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe93ad740 10 client.0 _ll_register_callbacks cb 0x7fcbd4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbe4d74640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 0x55e13718d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.465+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e13718f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbe4d74640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 0x55e13718d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42432/0 (socket says 172.21.15.18:42432) 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbe4d74640 1 -- 172.21.15.18:0/121964587 learned_addr learned my addr 172.21.15.18:0/121964587 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e13717ad60 0x55e13718f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbdf7fe640 1 -- 172.21.15.18:0/121964587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e13717ad60 msgr2=0x55e13718f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbdf7fe640 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e13717ad60 0x55e13718f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbdf7fe640 1 -- 172.21.15.18:0/121964587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 msgr2=0x55e13718d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbdf7fe640 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 0x55e13718d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbdf7fe640 1 -- 172.21.15.18:0/121964587 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1370e0d80 con 0x55e13717bad0 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 0x55e13718d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbdf7fe640 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e13718f5f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4000c00 tx=0x7fcbd4009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6cae9e82c5d65fe server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbbf7fe640 1 -- 172.21.15.18:0/121964587 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4017040 con 0x55e13717bad0 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbbf7fe640 1 -- 172.21.15.18:0/121964587 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fcbd4000e40 con 0x55e13717bad0 2024-02-17T12:58:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbe93ad740 1 -- 172.21.15.18:0/121964587 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbd80345a0 con 0x55e13717bad0 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbbf7fe640 1 -- 172.21.15.18:0/121964587 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbd4003a20 con 0x55e13717bad0 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbe93ad740 10 client.27712 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.466+0000 7fcbe93ad740 1 -- 172.21.15.18:0/121964587 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7fcbd8052040 con 0x55e13717bad0 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.467+0000 7fcbbf7fe640 1 -- 172.21.15.18:0/121964587 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbd80345a0 con 0x55e13717bad0 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.467+0000 7fcbbf7fe640 1 -- 172.21.15.18:0/121964587 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fcbd402d070 con 0x55e13717bad0 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.467+0000 7fcbe93ad740 10 client.27712 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.467+0000 7fcbe93ad740 10 client.27712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.467+0000 7fcbe93ad740 20 client.27712 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.467+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e1371f00e0 0x55e1371f2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.467+0000 7fcbe93ad740 4 client.27712 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.467+0000 7fcbe93ad740 1 -- 172.21.15.18:0/121964587 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e136e1f550 con 0x55e1371f00e0 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.467+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e1371f00e0 0x55e1371f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.467+0000 7fcbbf7fe640 10 client.27712 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.467+0000 7fcbe4d74640 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e1371f00e0 0x55e1371f2500 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47497fa0b78f559d server_cookie=7e2f7c6a46663ff7 in_seq=0 out_seq=0 2024-02-17T12:58:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.501+0000 7fcbbf7fe640 1 -- 172.21.15.18:0/121964587 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e136e1f550 con 0x55e1371f00e0 2024-02-17T12:58:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.501+0000 7fcbbf7fe640 10 client.27712 handle_command_reply: tid=0 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbe93ad740 1 client.27712 shutdown 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbe93ad740 1 -- 172.21.15.18:0/121964587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 msgr2=0x55e13718f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e13718f5f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4000c00 tx=0x7fcbd4009640 comp rx=0 tx=0).stop 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbe93ad740 1 -- 172.21.15.18:0/121964587 shutdown_connections 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e1371f00e0 0x55e1371f2500 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1371842e0 0x55e13718d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e13717bad0 0x55e13718f5f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbbf7fe640 0 client.27712 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbe93ad740 1 --2- 172.21.15.18:0/121964587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e13717ad60 0x55e13718f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbe93ad740 1 -- 172.21.15.18:0/121964587 >> 172.21.15.18:0/121964587 conn(0x55e137142550 msgr2=0x55e13717fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbe93ad740 1 -- 172.21.15.18:0/121964587 shutdown_connections 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbe93ad740 1 -- 172.21.15.18:0/121964587 wait complete. 2024-02-17T12:58:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.502+0000 7fcbe93ad740 20 client.27712 trim_cache size 0 max 16384 2024-02-17T12:58:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:58:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:58:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:58:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:58:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.503+0000 7fcbe67d7640 1 -- 172.21.15.18:0/1822376075 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcbc80a22d0 msgr2=0x7fcbc80a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.503+0000 7fcbe67d7640 1 --2- 172.21.15.18:0/1822376075 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcbc80a22d0 0x7fcbc80a4790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00cbb40 tx=0x7fcbd00d1c70 comp rx=0 tx=0).stop 2024-02-17T12:58:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.503+0000 7fcbe67d7640 1 -- 172.21.15.18:0/1822376075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe01002b0 msgr2=0x7fcbe019d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.503+0000 7fcbe67d7640 1 --2- 172.21.15.18:0/1822376075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe01002b0 0x7fcbe019d430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000e9f0 tx=0x7fcbd000eeb0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.503+0000 7fcbe67d7640 1 -- 172.21.15.18:0/1822376075 shutdown_connections 2024-02-17T12:58:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.503+0000 7fcbe67d7640 1 --2- 172.21.15.18:0/1822376075 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcbc80a22d0 0x7fcbc80a4790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.503+0000 7fcbe67d7640 1 --2- 172.21.15.18:0/1822376075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcbe01975e0 0x7fcbe0197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.503+0000 7fcbe67d7640 1 --2- 172.21.15.18:0/1822376075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe0105380 0x7fcbe019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.503+0000 7fcbe67d7640 1 --2- 172.21.15.18:0/1822376075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbe01002b0 0x7fcbe019d430 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.503+0000 7fcbe67d7640 1 -- 172.21.15.18:0/1822376075 >> 172.21.15.18:0/1822376075 conn(0x7fcbe00771a0 msgr2=0x7fcbe00695f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.503+0000 7fcbe67d7640 1 -- 172.21.15.18:0/1822376075 shutdown_connections 2024-02-17T12:58:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:20.503+0000 7fcbe67d7640 1 -- 172.21.15.18:0/1822376075 wait complete. 2024-02-17T12:58:21.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 434458 5.27 MB/sec execute 336 sec latency 134.634 ms 2024-02-17T12:58:22.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:21 smithi146 ceph-mon[36454]: pgmap v264: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-02-17T12:58:22.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:21 smithi018 ceph-mon[32625]: pgmap v264: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-02-17T12:58:22.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 434458 5.26 MB/sec execute 337 sec latency 1134.747 ms 2024-02-17T12:58:22.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:21 smithi119 ceph-mon[36188]: pgmap v264: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-02-17T12:58:23.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 435527 5.26 MB/sec execute 338 sec latency 1698.311 ms 2024-02-17T12:58:24.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:23 smithi146 ceph-mon[36454]: pgmap v265: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-02-17T12:58:24.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:23 smithi018 ceph-mon[32625]: pgmap v265: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-02-17T12:58:24.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 436917 5.26 MB/sec execute 339 sec latency 81.376 ms 2024-02-17T12:58:24.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:23 smithi119 ceph-mon[36188]: pgmap v265: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-02-17T12:58:24.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:58:24.354 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:58:24.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.482+0000 7f84ae12c640 1 -- 172.21.15.18:0/1844894129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84a8105340 msgr2=0x7f84a810cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.482+0000 7f84ae12c640 1 --2- 172.21.15.18:0/1844894129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84a8105340 0x7f84a810cf10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009c10 tx=0x7f84a002bef0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.482+0000 7f84ae12c640 1 -- 172.21.15.18:0/1844894129 shutdown_connections 2024-02-17T12:58:24.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.482+0000 7f84ae12c640 1 --2- 172.21.15.18:0/1844894129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84a8105340 0x7f84a810cf10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.482+0000 7f84ae12c640 1 --2- 172.21.15.18:0/1844894129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84a8100c20 0x7f84a8104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.482+0000 7f84ae12c640 1 --2- 172.21.15.18:0/1844894129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84a8100270 0x7f84a8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.482+0000 7f84ae12c640 1 -- 172.21.15.18:0/1844894129 >> 172.21.15.18:0/1844894129 conn(0x7f84a8077230 msgr2=0x7f84a8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:24.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.482+0000 7f84ae12c640 1 -- 172.21.15.18:0/1844894129 shutdown_connections 2024-02-17T12:58:24.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.482+0000 7f84ae12c640 1 -- 172.21.15.18:0/1844894129 wait complete. 2024-02-17T12:58:24.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.483+0000 7f84ae12c640 1 Processor -- start 2024-02-17T12:58:24.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.483+0000 7f84ae12c640 1 -- start start 2024-02-17T12:58:24.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84ae12c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84a8100270 0x7f84a81a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84ae12c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84a8105340 0x7f84a81a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84ae12c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84a819d9b0 0x7f84a819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84ae12c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a819e3a0 con 0x7f84a8105340 2024-02-17T12:58:24.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84ae12c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a819e510 con 0x7f84a819d9b0 2024-02-17T12:58:24.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84ae12c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a819e6d0 con 0x7f84a8100270 2024-02-17T12:58:24.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84a6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84a8105340 0x7f84a81a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84a6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84a8105340 0x7f84a81a3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42446/0 (socket says 172.21.15.18:42446) 2024-02-17T12:58:24.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84a6ffd640 1 -- 172.21.15.18:0/2534665680 learned_addr learned my addr 172.21.15.18:0/2534665680 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:24.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84a7fff640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84a819d9b0 0x7f84a819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84a77fe640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84a8100270 0x7f84a81a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84a7fff640 1 -- 172.21.15.18:0/2534665680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84a8100270 msgr2=0x7f84a81a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84a7fff640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84a8100270 0x7f84a81a3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84a7fff640 1 -- 172.21.15.18:0/2534665680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84a8105340 msgr2=0x7f84a81a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84a7fff640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84a8105340 0x7f84a81a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84a7fff640 1 -- 172.21.15.18:0/2534665680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a0009840 con 0x7f84a819d9b0 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84a8105340 0x7f84a81a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.484+0000 7f84a77fe640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84a8100270 0x7f84a81a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.485+0000 7f84a7fff640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84a819d9b0 0x7f84a819de60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009b30 tx=0x7f84a0033700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.485+0000 7f84a4ff9640 1 -- 172.21.15.18:0/2534665680 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a0003d80 con 0x7f84a819d9b0 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.485+0000 7f84a4ff9640 1 -- 172.21.15.18:0/2534665680 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84a0004520 con 0x7f84a819d9b0 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.485+0000 7f84a4ff9640 1 -- 172.21.15.18:0/2534665680 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a003db50 con 0x7f84a819d9b0 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.485+0000 7f84ae12c640 1 -- 172.21.15.18:0/2534665680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a81aa6c0 con 0x7f84a819d9b0 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.485+0000 7f84ae12c640 1 -- 172.21.15.18:0/2534665680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a81aaa90 con 0x7f84a819d9b0 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.488+0000 7f84a4ff9640 1 -- 172.21.15.18:0/2534665680 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f84a0004690 con 0x7f84a819d9b0 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.488+0000 7f84b0dd8740 1 Processor -- start 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.488+0000 7f84b0dd8740 1 -- start start 2024-02-17T12:58:24.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.488+0000 7f84b0dd8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea595e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.488+0000 7f84b0dd8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea5967270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.488+0000 7f84b0dd8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5969ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.488+0000 7f84b0dd8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea58cd280 con 0x556ea595ebd0 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.488+0000 7f84b0dd8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea58ca7b0 con 0x556ea595e2b0 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.488+0000 7f84b0dd8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea58c5f90 con 0x556ea59677b0 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a6ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5969ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a4ff9640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f84900a22b0 0x7f84900a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a6ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5969ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40800/0 (socket says 172.21.15.18:40800) 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a6ffd640 1 -- 172.21.15.18:0/3003248538 learned_addr learned my addr 172.21.15.18:0/3003248538 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a77fe640 1 --2- 172.21.15.18:0/3003248538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea5967270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/3003248538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea595e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a7fff640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f84900a22b0 0x7f84900a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a4ff9640 1 -- 172.21.15.18:0/2534665680 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f84a00f07b0 con 0x7f84a819d9b0 2024-02-17T12:58:24.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a6ffd640 1 -- 172.21.15.18:0/3003248538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 msgr2=0x556ea595e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:24.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/3003248538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea595e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a6ffd640 1 -- 172.21.15.18:0/3003248538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 msgr2=0x556ea5967270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:24.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/3003248538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea5967270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a6ffd640 1 -- 172.21.15.18:0/3003248538 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ea5884570 con 0x556ea59677b0 2024-02-17T12:58:24.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.489+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/3003248538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5969ba0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f849c016bf0 tx=0x7f849c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35379dce57d50314 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:24.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.490+0000 7f848dffb640 1 -- 172.21.15.18:0/3003248538 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c010070 con 0x556ea59677b0 2024-02-17T12:58:24.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.490+0000 7f848dffb640 1 -- 172.21.15.18:0/3003248538 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f849c016ce0 con 0x556ea59677b0 2024-02-17T12:58:24.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.490+0000 7f848dffb640 1 -- 172.21.15.18:0/3003248538 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c01cc80 con 0x556ea59677b0 2024-02-17T12:58:24.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.490+0000 7f84a7fff640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f84900a22b0 0x7f84900a4770 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f84a00c5540 tx=0x7f84a004d000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:24.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.490+0000 7f84b0dd8740 1 -- 172.21.15.18:0/3003248538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 msgr2=0x556ea5969ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.490+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/3003248538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5969ba0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f849c016bf0 tx=0x7f849c005f90 comp rx=0 tx=0).stop 2024-02-17T12:58:24.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.490+0000 7f84b0dd8740 1 -- 172.21.15.18:0/3003248538 shutdown_connections 2024-02-17T12:58:24.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.490+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/3003248538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5969ba0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.490+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/3003248538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea5967270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.490+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/3003248538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea595e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.490+0000 7f84b0dd8740 1 -- 172.21.15.18:0/3003248538 >> 172.21.15.18:0/3003248538 conn(0x556ea5958990 msgr2=0x556ea5958d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:24.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.491+0000 7f84b0dd8740 1 -- 172.21.15.18:0/3003248538 shutdown_connections 2024-02-17T12:58:24.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.491+0000 7f84b0dd8740 1 -- 172.21.15.18:0/3003248538 wait complete. 2024-02-17T12:58:24.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.491+0000 7f84b0dd8740 1 -- start start 2024-02-17T12:58:24.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.491+0000 7f84b0dd8740 1 Processor -- start 2024-02-17T12:58:24.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.491+0000 7f84b0dd8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea5967220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.491+0000 7f84b0dd8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea59645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.491+0000 7f84b0dd8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5964cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.491+0000 7f84b0dd8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea5599430 con 0x556ea595ebd0 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.491+0000 7f84b0dd8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea5555e00 con 0x556ea595e2b0 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.491+0000 7f84b0dd8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea584afd0 con 0x556ea59677b0 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.491+0000 7f84b0dd8740 10 client.0 _ll_register_callbacks cb 0x556ea58a0fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.491+0000 7f84a6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea5967220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f84a6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea5967220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:35892/0 (socket says 172.21.15.18:35892) 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f84a6ffd640 1 -- 172.21.15.18:0/2453306616 learned_addr learned my addr 172.21.15.18:0/2453306616 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f84a77fe640 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea59645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5964cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f84a6ffd640 1 -- 172.21.15.18:0/2453306616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 msgr2=0x556ea5964cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5964cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f84a6ffd640 1 -- 172.21.15.18:0/2453306616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 msgr2=0x556ea59645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea59645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f84a6ffd640 1 -- 172.21.15.18:0/2453306616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849c0168a0 con 0x556ea595e2b0 2024-02-17T12:58:24.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea5967220 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f849c00b460 tx=0x7f849c01cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb00550c79c7a1e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f8486ffd640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f8486ffd640 1 -- 172.21.15.18:0/2453306616 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c010070 con 0x556ea595e2b0 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f8486ffd640 1 -- 172.21.15.18:0/2453306616 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f849c000e40 con 0x556ea595e2b0 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.492+0000 7f84b0dd8740 1 -- 172.21.15.18:0/2453306616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ea55c57e0 con 0x556ea595e2b0 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.493+0000 7f8486ffd640 1 -- 172.21.15.18:0/2453306616 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f849c01c470 con 0x556ea595e2b0 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.493+0000 7f84b0dd8740 10 client.27680 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.493+0000 7f84b0dd8740 1 -- 172.21.15.18:0/2453306616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x556ea58c4610 con 0x556ea595e2b0 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.493+0000 7f8486ffd640 1 -- 172.21.15.18:0/2453306616 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556ea55c57e0 con 0x556ea595e2b0 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.493+0000 7f8486ffd640 1 -- 172.21.15.18:0/2453306616 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f849c0046c0 con 0x556ea595e2b0 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.493+0000 7f84b0dd8740 10 client.27680 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.493+0000 7f84b0dd8740 10 client.27680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.493+0000 7f84b0dd8740 20 client.27680 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.493+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556ea59778f0 0x556ea5979d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.493+0000 7f84b0dd8740 4 client.27680 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.493+0000 7f84b0dd8740 1 -- 172.21.15.18:0/2453306616 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ea5602720 con 0x556ea59778f0 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.494+0000 7f84a77fe640 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556ea59778f0 0x556ea5979d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.494+0000 7f8486ffd640 10 client.27680 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.494+0000 7f84a77fe640 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556ea59778f0 0x556ea5979d10 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=350ae980eaa53bff server_cookie=cf1d63fa68ab7ad9 in_seq=0 out_seq=0 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f8486ffd640 1 -- 172.21.15.18:0/2453306616 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556ea5602720 con 0x556ea59778f0 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f8486ffd640 10 client.27680 handle_command_reply: tid=0 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f84b0dd8740 1 client.27680 shutdown 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f84b0dd8740 1 -- 172.21.15.18:0/2453306616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 msgr2=0x556ea5967220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea5967220 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f849c00b460 tx=0x7f849c01cba0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f84b0dd8740 1 -- 172.21.15.18:0/2453306616 shutdown_connections 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556ea59778f0 0x556ea5979d10 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5964cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea59645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f8486ffd640 0 client.27680 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:24.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/2453306616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea5967220 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f84b0dd8740 1 -- 172.21.15.18:0/2453306616 >> 172.21.15.18:0/2453306616 conn(0x556ea5958990 msgr2=0x556ea595a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:24.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f84b0dd8740 1 -- 172.21.15.18:0/2453306616 shutdown_connections 2024-02-17T12:58:24.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f84b0dd8740 1 -- 172.21.15.18:0/2453306616 wait complete. 2024-02-17T12:58:24.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.495+0000 7f84b0dd8740 20 client.27680 trim_cache size 0 max 16384 2024-02-17T12:58:24.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.513+0000 7f84b0dd8740 1 Processor -- start 2024-02-17T12:58:24.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.513+0000 7f84b0dd8740 1 -- start start 2024-02-17T12:58:24.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.513+0000 7f84b0dd8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea5966ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.513+0000 7f84b0dd8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea5967220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.513+0000 7f84b0dd8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5961830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.513+0000 7f84b0dd8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea5602720 con 0x556ea595ebd0 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.513+0000 7f84b0dd8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea55c57e0 con 0x556ea595e2b0 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.513+0000 7f84b0dd8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea59540f0 con 0x556ea59677b0 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f84a77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea5967220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f84a6ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5961830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f84a77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea5967220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42474/0 (socket says 172.21.15.18:42474) 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f84a77fe640 1 -- 172.21.15.18:0/1579975724 learned_addr learned my addr 172.21.15.18:0/1579975724 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/1579975724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea5966ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f84a6ffd640 1 -- 172.21.15.18:0/1579975724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 msgr2=0x556ea5966ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/1579975724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea5966ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f84a6ffd640 1 -- 172.21.15.18:0/1579975724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 msgr2=0x556ea5967220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/1579975724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea5967220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f84a6ffd640 1 -- 172.21.15.18:0/1579975724 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ea5860ac0 con 0x556ea59677b0 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/1579975724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5961830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f849c001eb0 tx=0x7f849c005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99a96aaa9c90bf98 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f848d7fa640 1 -- 172.21.15.18:0/1579975724 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c010040 con 0x556ea59677b0 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f848d7fa640 1 -- 172.21.15.18:0/1579975724 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f849c004d70 con 0x556ea59677b0 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.514+0000 7f848d7fa640 1 -- 172.21.15.18:0/1579975724 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c005000 con 0x556ea59677b0 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.515+0000 7f84b0dd8740 1 -- 172.21.15.18:0/1579975724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 msgr2=0x556ea5961830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.515+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/1579975724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5961830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f849c001eb0 tx=0x7f849c005970 comp rx=0 tx=0).stop 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.515+0000 7f84b0dd8740 1 -- 172.21.15.18:0/1579975724 shutdown_connections 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.515+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/1579975724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea59677b0 0x556ea5961830 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.515+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/1579975724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea5967220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.515+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/1579975724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea595e2b0 0x556ea5966ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.515+0000 7f84b0dd8740 1 -- 172.21.15.18:0/1579975724 >> 172.21.15.18:0/1579975724 conn(0x556ea5958990 msgr2=0x556ea595f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.515+0000 7f84b0dd8740 1 -- 172.21.15.18:0/1579975724 shutdown_connections 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.515+0000 7f84b0dd8740 1 -- 172.21.15.18:0/1579975724 wait complete. 2024-02-17T12:58:24.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.515+0000 7f84b0dd8740 1 -- start start 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84b0dd8740 1 Processor -- start 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84b0dd8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea595e2b0 0x556ea596eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84b0dd8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea596af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84b0dd8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea59677b0 0x556ea596b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84b0dd8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea596bcf0 con 0x556ea595ebd0 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84b0dd8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea59cddf0 con 0x556ea59677b0 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84b0dd8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea59cdf30 con 0x556ea595e2b0 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84b0dd8740 10 client.0 _ll_register_callbacks cb 0x7f849c00bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84a77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea596af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84a77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea596af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42490/0 (socket says 172.21.15.18:42490) 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84a77fe640 1 -- 172.21.15.18:0/414256028 learned_addr learned my addr 172.21.15.18:0/414256028 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea59677b0 0x556ea596b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea595e2b0 0x556ea596eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84a77fe640 1 -- 172.21.15.18:0/414256028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea595e2b0 msgr2=0x556ea596eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84a77fe640 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea595e2b0 0x556ea596eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84a77fe640 1 -- 172.21.15.18:0/414256028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea59677b0 msgr2=0x556ea596b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84a77fe640 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea59677b0 0x556ea596b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84a77fe640 1 -- 172.21.15.18:0/414256028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ea58c4610 con 0x556ea595ebd0 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.516+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea59677b0 0x556ea596b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f84a77fe640 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea596af00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x556ea595dd80 tx=0x7f849801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a35fadc930b98d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f8486ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f8486ffd640 1 -- 172.21.15.18:0/414256028 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498017040 con 0x556ea595ebd0 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f8486ffd640 1 -- 172.21.15.18:0/414256028 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8498003cc0 con 0x556ea595ebd0 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f84b0dd8740 1 -- 172.21.15.18:0/414256028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ea59643e0 con 0x556ea595ebd0 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f8486ffd640 1 -- 172.21.15.18:0/414256028 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8498004500 con 0x556ea595ebd0 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f84b0dd8740 10 client.18561 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f84b0dd8740 1 -- 172.21.15.18:0/414256028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x556ea58866a0 con 0x556ea595ebd0 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f8486ffd640 1 -- 172.21.15.18:0/414256028 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556ea59643e0 con 0x556ea595ebd0 2024-02-17T12:58:24.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f8486ffd640 1 -- 172.21.15.18:0/414256028 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f84980114d0 con 0x556ea595ebd0 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f84b0dd8740 10 client.18561 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f84b0dd8740 10 client.18561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f84b0dd8740 20 client.18561 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556ea59cf7b0 0x556ea59d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f84b0dd8740 4 client.18561 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.517+0000 7f84b0dd8740 1 -- 172.21.15.18:0/414256028 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f849c016ce0 con 0x556ea59cf7b0 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556ea59cf7b0 0x556ea59d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f8486ffd640 10 client.18561 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f84a6ffd640 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556ea59cf7b0 0x556ea59d1bd0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a05beebc65bfe805 server_cookie=773d2c4445579bdb in_seq=0 out_seq=0 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f8486ffd640 1 -- 172.21.15.18:0/414256028 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f849c016ce0 con 0x556ea59cf7b0 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f8486ffd640 10 client.18561 handle_command_reply: tid=0 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f84b0dd8740 1 client.18561 shutdown 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f84b0dd8740 1 -- 172.21.15.18:0/414256028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 msgr2=0x556ea596af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea596af00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x556ea595dd80 tx=0x7f849801c680 comp rx=0 tx=0).stop 2024-02-17T12:58:24.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f84b0dd8740 1 -- 172.21.15.18:0/414256028 shutdown_connections 2024-02-17T12:58:24.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556ea59cf7b0 0x556ea59d1bd0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ea59677b0 0x556ea596b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea595ebd0 0x556ea596af00 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f8486ffd640 0 client.18561 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:24.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f84b0dd8740 1 --2- 172.21.15.18:0/414256028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556ea595e2b0 0x556ea596eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f84b0dd8740 1 -- 172.21.15.18:0/414256028 >> 172.21.15.18:0/414256028 conn(0x556ea5958990 msgr2=0x556ea5962e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:24.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.518+0000 7f84b0dd8740 1 -- 172.21.15.18:0/414256028 shutdown_connections 2024-02-17T12:58:24.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84b0dd8740 1 -- 172.21.15.18:0/414256028 wait complete. 2024-02-17T12:58:24.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84b0dd8740 20 client.18561 trim_cache size 0 max 16384 2024-02-17T12:58:24.516 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:58:24.517 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:58:24.517 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:58:24.517 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:58:24.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84ae12c640 1 -- 172.21.15.18:0/2534665680 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f84900a22b0 msgr2=0x7f84900a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84ae12c640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f84900a22b0 0x7f84900a4770 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f84a00c5540 tx=0x7f84a004d000 comp rx=0 tx=0).stop 2024-02-17T12:58:24.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84ae12c640 1 -- 172.21.15.18:0/2534665680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84a819d9b0 msgr2=0x7f84a819de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84ae12c640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84a819d9b0 0x7f84a819de60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009b30 tx=0x7f84a0033700 comp rx=0 tx=0).stop 2024-02-17T12:58:24.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84ae12c640 1 -- 172.21.15.18:0/2534665680 shutdown_connections 2024-02-17T12:58:24.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84ae12c640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f84900a22b0 0x7f84900a4770 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84ae12c640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84a819d9b0 0x7f84a819de60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84ae12c640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84a8105340 0x7f84a81a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84ae12c640 1 --2- 172.21.15.18:0/2534665680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84a8100270 0x7f84a81a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84ae12c640 1 -- 172.21.15.18:0/2534665680 >> 172.21.15.18:0/2534665680 conn(0x7f84a8077230 msgr2=0x7f84a80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:24.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84ae12c640 1 -- 172.21.15.18:0/2534665680 shutdown_connections 2024-02-17T12:58:24.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.519+0000 7f84ae12c640 1 -- 172.21.15.18:0/2534665680 wait complete. 2024-02-17T12:58:24.521 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:58:24.522 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:58:24.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.642+0000 7ff18aea3640 1 -- 172.21.15.18:0/250365939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002f0 msgr2=0x7ff1841006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.642+0000 7ff18aea3640 1 --2- 172.21.15.18:0/250365939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002f0 0x7ff1841006d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff17800d940 tx=0x7ff17802dbb0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.642+0000 7ff18aea3640 1 -- 172.21.15.18:0/250365939 shutdown_connections 2024-02-17T12:58:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.642+0000 7ff18aea3640 1 --2- 172.21.15.18:0/250365939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1841053c0 0x7ff18410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.642+0000 7ff18aea3640 1 --2- 172.21.15.18:0/250365939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184100ca0 0x7ff184104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.642+0000 7ff18aea3640 1 --2- 172.21.15.18:0/250365939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002f0 0x7ff1841006d0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.642+0000 7ff18aea3640 1 -- 172.21.15.18:0/250365939 >> 172.21.15.18:0/250365939 conn(0x7ff184077180 msgr2=0x7ff184077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.642+0000 7ff18aea3640 1 -- 172.21.15.18:0/250365939 shutdown_connections 2024-02-17T12:58:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.642+0000 7ff18aea3640 1 -- 172.21.15.18:0/250365939 wait complete. 2024-02-17T12:58:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.643+0000 7ff18aea3640 1 Processor -- start 2024-02-17T12:58:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.643+0000 7ff18aea3640 1 -- start start 2024-02-17T12:58:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.643+0000 7ff18aea3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002f0 0x7ff184195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.643+0000 7ff18aea3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1841053c0 0x7ff184195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.643+0000 7ff18aea3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184199f00 0x7ff18419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.643+0000 7ff18aea3640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18419a8f0 con 0x7ff1841002f0 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.643+0000 7ff18aea3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18419aa60 con 0x7ff184199f00 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.643+0000 7ff18aea3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18419abd0 con 0x7ff1841053c0 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.643+0000 7ff188c3f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002f0 0x7ff184195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.643+0000 7ff188c3f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002f0 0x7ff184195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42502/0 (socket says 172.21.15.18:42502) 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.643+0000 7ff188c3f640 1 -- 172.21.15.18:0/963584111 learned_addr learned my addr 172.21.15.18:0/963584111 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.643+0000 7ff189440640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184199f00 0x7ff18419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.644+0000 7ff183fff640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1841053c0 0x7ff184195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.644+0000 7ff188c3f640 1 -- 172.21.15.18:0/963584111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1841053c0 msgr2=0x7ff184195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.644+0000 7ff188c3f640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1841053c0 0x7ff184195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.644+0000 7ff188c3f640 1 -- 172.21.15.18:0/963584111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184199f00 msgr2=0x7ff18419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.644+0000 7ff188c3f640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184199f00 0x7ff18419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.644+0000 7ff188c3f640 1 -- 172.21.15.18:0/963584111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c009840 con 0x7ff1841002f0 2024-02-17T12:58:24.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.644+0000 7ff189440640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184199f00 0x7ff18419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:24.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.644+0000 7ff188c3f640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002f0 0x7ff184195640 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff178005ab0 tx=0x7ff17800d500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:24.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.644+0000 7ff181ffb640 1 -- 172.21.15.18:0/963584111 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178030d90 con 0x7ff1841002f0 2024-02-17T12:58:24.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.644+0000 7ff181ffb640 1 -- 172.21.15.18:0/963584111 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff178030ef0 con 0x7ff1841002f0 2024-02-17T12:58:24.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.644+0000 7ff18aea3640 1 -- 172.21.15.18:0/963584111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17800d570 con 0x7ff1841002f0 2024-02-17T12:58:24.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.644+0000 7ff181ffb640 1 -- 172.21.15.18:0/963584111 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178005170 con 0x7ff1841002f0 2024-02-17T12:58:24.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.645+0000 7ff18aea3640 1 -- 172.21.15.18:0/963584111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1841a2440 con 0x7ff1841002f0 2024-02-17T12:58:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.646+0000 7ff18aea3640 1 -- 172.21.15.18:0/963584111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff184067f70 con 0x7ff1841002f0 2024-02-17T12:58:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.647+0000 7ff181ffb640 1 -- 172.21.15.18:0/963584111 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff1780047e0 con 0x7ff1841002f0 2024-02-17T12:58:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.647+0000 7ff181ffb640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff1740a2230 0x7ff1740a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.647+0000 7ff183fff640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff1740a2230 0x7ff1740a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.647+0000 7ff181ffb640 1 -- 172.21.15.18:0/963584111 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff1780f1c20 con 0x7ff1841002f0 2024-02-17T12:58:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.648+0000 7ff183fff640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff1740a2230 0x7ff1740a46f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff1840ffc30 tx=0x7ff170005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:24.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.652+0000 7ff181ffb640 1 -- 172.21.15.18:0/963584111 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff1780ba9e0 con 0x7ff1841002f0 2024-02-17T12:58:24.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.888+0000 7ff18aea3640 1 -- 172.21.15.18:0/963584111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff184196990 con 0x7ff1841002f0 2024-02-17T12:58:24.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.888+0000 7ff181ffb640 1 -- 172.21.15.18:0/963584111 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+11393 (secure 0 0 0) 0x7ff1780be0a0 con 0x7ff1841002f0 2024-02-17T12:58:24.879 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:58:24.880 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:57:33.474835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:58:24.880 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 44 2024-02-17T12:58:24.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.890+0000 7ff18aea3640 1 -- 172.21.15.18:0/963584111 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff1740a2230 msgr2=0x7ff1740a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.890+0000 7ff18aea3640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff1740a2230 0x7ff1740a46f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff1840ffc30 tx=0x7ff170005f90 comp rx=0 tx=0).stop 2024-02-17T12:58:24.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.890+0000 7ff18aea3640 1 -- 172.21.15.18:0/963584111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002f0 msgr2=0x7ff184195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:24.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.890+0000 7ff18aea3640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002f0 0x7ff184195640 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff178005ab0 tx=0x7ff17800d500 comp rx=0 tx=0).stop 2024-02-17T12:58:24.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.890+0000 7ff18aea3640 1 -- 172.21.15.18:0/963584111 shutdown_connections 2024-02-17T12:58:24.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.890+0000 7ff18aea3640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff1740a2230 0x7ff1740a46f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.890+0000 7ff18aea3640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184199f00 0x7ff18419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.890+0000 7ff18aea3640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1841053c0 0x7ff184195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.890+0000 7ff18aea3640 1 --2- 172.21.15.18:0/963584111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1841002f0 0x7ff184195640 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:24.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.890+0000 7ff18aea3640 1 -- 172.21.15.18:0/963584111 >> 172.21.15.18:0/963584111 conn(0x7ff184077180 msgr2=0x7ff1840fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:24.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.890+0000 7ff18aea3640 1 -- 172.21.15.18:0/963584111 shutdown_connections 2024-02-17T12:58:24.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:24.890+0000 7ff18aea3640 1 -- 172.21.15.18:0/963584111 wait complete. 2024-02-17T12:58:24.893 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:58:24.893 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:58:25.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.008+0000 7f6d690b5640 1 -- 172.21.15.18:0/3387323753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d641002c0 msgr2=0x7f6d64105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.008+0000 7f6d690b5640 1 --2- 172.21.15.18:0/3387323753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d641002c0 0x7f6d64105df0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009900 tx=0x7f6d4c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:25.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.008+0000 7f6d690b5640 1 -- 172.21.15.18:0/3387323753 shutdown_connections 2024-02-17T12:58:25.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.008+0000 7f6d690b5640 1 --2- 172.21.15.18:0/3387323753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d64106520 0x7f6d6410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.008+0000 7f6d690b5640 1 --2- 172.21.15.18:0/3387323753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d641002c0 0x7f6d64105df0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.008+0000 7f6d690b5640 1 --2- 172.21.15.18:0/3387323753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d640ff910 0x7f6d640ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.008+0000 7f6d690b5640 1 -- 172.21.15.18:0/3387323753 >> 172.21.15.18:0/3387323753 conn(0x7f6d64076640 msgr2=0x7f6d640fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.008+0000 7f6d690b5640 1 -- 172.21.15.18:0/3387323753 shutdown_connections 2024-02-17T12:58:25.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.008+0000 7f6d690b5640 1 -- 172.21.15.18:0/3387323753 wait complete. 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d690b5640 1 Processor -- start 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d690b5640 1 -- start start 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d690b5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d640ff910 0x7f6d6419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d690b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d64106520 0x7f6d6419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d690b5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d64197590 0x7f6d64197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d690b5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d64197f80 con 0x7f6d64197590 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d690b5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d641980f0 con 0x7f6d64106520 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d690b5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d641982b0 con 0x7f6d640ff910 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d6359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d64197590 0x7f6d64197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d6359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d64197590 0x7f6d64197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42522/0 (socket says 172.21.15.18:42522) 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d6359e640 1 -- 172.21.15.18:0/2595021232 learned_addr learned my addr 172.21.15.18:0/2595021232 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d5bfff640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d64106520 0x7f6d6419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.009+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d640ff910 0x7f6d6419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.010+0000 7f6d6359e640 1 -- 172.21.15.18:0/2595021232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d640ff910 msgr2=0x7f6d6419d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.010+0000 7f6d6359e640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d640ff910 0x7f6d6419d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.010+0000 7f6d6359e640 1 -- 172.21.15.18:0/2595021232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d64106520 msgr2=0x7f6d6419d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.010+0000 7f6d6359e640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d64106520 0x7f6d6419d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.010+0000 7f6d6359e640 1 -- 172.21.15.18:0/2595021232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c009560 con 0x7f6d64197590 2024-02-17T12:58:25.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.010+0000 7f6d6359e640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d64197590 0x7f6d64197a40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00eaa0 tx=0x7f6d5c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.010+0000 7f6d60d99640 1 -- 172.21.15.18:0/2595021232 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00cdb0 con 0x7f6d64197590 2024-02-17T12:58:25.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.010+0000 7f6d60d99640 1 -- 172.21.15.18:0/2595021232 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d5c005020 con 0x7f6d64197590 2024-02-17T12:58:25.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.010+0000 7f6d690b5640 1 -- 172.21.15.18:0/2595021232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d641a42a0 con 0x7f6d64197590 2024-02-17T12:58:25.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.010+0000 7f6d60d99640 1 -- 172.21.15.18:0/2595021232 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0116b0 con 0x7f6d64197590 2024-02-17T12:58:25.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.010+0000 7f6d690b5640 1 -- 172.21.15.18:0/2595021232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d641a4730 con 0x7f6d64197590 2024-02-17T12:58:25.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.012+0000 7f6d60d99640 1 -- 172.21.15.18:0/2595021232 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6d5c005520 con 0x7f6d64197590 2024-02-17T12:58:25.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.012+0000 7f6d6bd66740 1 Processor -- start 2024-02-17T12:58:25.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.012+0000 7f6d6bd66740 1 -- start start 2024-02-17T12:58:25.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d60d99640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6d500a2230 0x7f6d500a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d6bd66740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 0x55c8dbe38630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d6bd66740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38b70 0x55c8dbe41210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d6bd66740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe41750 0x55c8dbe43b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d6bd66740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8dbdf3960 con 0x55c8dbe38250 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d6bd66740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8dbda7310 con 0x55c8dbe38b70 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d6bd66740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8dbda4840 con 0x55c8dbe41750 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d62d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 0x55c8dbe38630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d60d99640 1 -- 172.21.15.18:0/2595021232 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6d5c0ce2d0 con 0x7f6d64197590 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d62d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 0x55c8dbe38630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42532/0 (socket says 172.21.15.18:42532) 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d62d9d640 1 -- 172.21.15.18:0/48427148 learned_addr learned my addr 172.21.15.18:0/48427148 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d5bfff640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6d500a2230 0x7f6d500a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d5bfff640 1 --2- 172.21.15.18:0/48427148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38b70 0x55c8dbe41210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d6359e640 1 --2- 172.21.15.18:0/48427148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe41750 0x55c8dbe43b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d62d9d640 1 -- 172.21.15.18:0/48427148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe41750 msgr2=0x55c8dbe43b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/48427148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe41750 0x55c8dbe43b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d62d9d640 1 -- 172.21.15.18:0/48427148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38b70 msgr2=0x55c8dbe41210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/48427148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38b70 0x55c8dbe41210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d62d9d640 1 -- 172.21.15.18:0/48427148 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8dbd62060 con 0x55c8dbe38250 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/48427148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 0x55c8dbe38630 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400bcf0 tx=0x7f6d5400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d4b9ca2292c594 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d5bfff640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6d500a2230 0x7f6d500a46f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009230 tx=0x7f6d4c02c680 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.013+0000 7f6d597fa640 1 -- 172.21.15.18:0/48427148 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5401a460 con 0x55c8dbe38250 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.014+0000 7f6d597fa640 1 -- 172.21.15.18:0/48427148 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d5401aaa0 con 0x55c8dbe38250 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.014+0000 7f6d597fa640 1 -- 172.21.15.18:0/48427148 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54011a80 con 0x55c8dbe38250 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.014+0000 7f6d6bd66740 1 -- 172.21.15.18:0/48427148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 msgr2=0x55c8dbe38630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.014+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/48427148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 0x55c8dbe38630 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400bcf0 tx=0x7f6d5400f730 comp rx=0 tx=0).stop 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.014+0000 7f6d6bd66740 1 -- 172.21.15.18:0/48427148 shutdown_connections 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.014+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/48427148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe41750 0x55c8dbe43b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.014+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/48427148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38b70 0x55c8dbe41210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.014+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/48427148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 0x55c8dbe38630 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.014+0000 7f6d6bd66740 1 -- 172.21.15.18:0/48427148 >> 172.21.15.18:0/48427148 conn(0x55c8dbe32980 msgr2=0x55c8dbe32d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.014+0000 7f6d6bd66740 1 -- 172.21.15.18:0/48427148 shutdown_connections 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.014+0000 7f6d6bd66740 1 -- 172.21.15.18:0/48427148 wait complete. 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6bd66740 1 -- start start 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6bd66740 1 Processor -- start 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6bd66740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38250 0x55c8dbe3e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6bd66740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38b70 0x55c8dbe3ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6bd66740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 0x55c8dbe3f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6bd66740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8dbda0020 con 0x55c8dbe41750 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6bd66740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8dba73430 con 0x55c8dbe38b70 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6bd66740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8dba2fe00 con 0x55c8dbe38250 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6bd66740 10 client.0 _ll_register_callbacks cb 0x55c8dbd47cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 0x55c8dbe3f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 0x55c8dbe3f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42546/0 (socket says 172.21.15.18:42546) 2024-02-17T12:58:25.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6359e640 1 -- 172.21.15.18:0/3281436369 learned_addr learned my addr 172.21.15.18:0/3281436369 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38250 0x55c8dbe3e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d5bfff640 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38b70 0x55c8dbe3ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6359e640 1 -- 172.21.15.18:0/3281436369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38250 msgr2=0x55c8dbe3e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6359e640 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38250 0x55c8dbe3e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6359e640 1 -- 172.21.15.18:0/3281436369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38b70 msgr2=0x55c8dbe3ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6359e640 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38b70 0x55c8dbe3ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.015+0000 7f6d6359e640 1 -- 172.21.15.18:0/3281436369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5400b920 con 0x55c8dbe41750 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d6359e640 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 0x55c8dbe3f220 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c09ac70 tx=0x7f6d5c09b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1575563ebf0d82e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d4a7fc640 1 -- 172.21.15.18:0/3281436369 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c095070 con 0x55c8dbe41750 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d4a7fc640 1 -- 172.21.15.18:0/3281436369 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6d5c0cd030 con 0x55c8dbe41750 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3281436369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8dba9f7e0 con 0x55c8dbe41750 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d4a7fc640 1 -- 172.21.15.18:0/3281436369 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d5c005390 con 0x55c8dbe41750 2024-02-17T12:58:25.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d6bd66740 10 client.18579 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3281436369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55c8dbd9e6a0 con 0x55c8dbe41750 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d4a7fc640 1 -- 172.21.15.18:0/3281436369 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c8dba9f7e0 con 0x55c8dbe41750 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d4a7fc640 1 -- 172.21.15.18:0/3281436369 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f6d5c018030 con 0x55c8dbe41750 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d6bd66740 10 client.18579 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d6bd66740 10 client.18579 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d6bd66740 20 client.18579 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c8dbe51bc0 0x55c8dbe53fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d6bd66740 4 client.18579 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.016+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3281436369 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8dbadc720 con 0x55c8dbe51bc0 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.017+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c8dbe51bc0 0x55c8dbe53fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.017+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c8dbe51bc0 0x55c8dbe53fb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9aadc82423a8f242 server_cookie=a58e2e689cdb9ab7 in_seq=0 out_seq=0 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.017+0000 7f6d4a7fc640 10 client.18579 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.017+0000 7f6d4a7fc640 1 -- 172.21.15.18:0/3281436369 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c8dbadc720 con 0x55c8dbe51bc0 2024-02-17T12:58:25.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.017+0000 7f6d4a7fc640 10 client.18579 handle_command_reply: tid=0 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d6bd66740 1 client.18579 shutdown 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3281436369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 msgr2=0x55c8dbe3f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 0x55c8dbe3f220 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c09ac70 tx=0x7f6d5c09b520 comp rx=0 tx=0).stop 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3281436369 shutdown_connections 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 0x55c8dbe3f220 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c8dbe51bc0 0x55c8dbe53fb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38b70 0x55c8dbe3ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/3281436369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38250 0x55c8dbe3e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d4a7fc640 0 client.18579 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3281436369 >> 172.21.15.18:0/3281436369 conn(0x55c8dbe32980 msgr2=0x55c8dbe346d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3281436369 shutdown_connections 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3281436369 wait complete. 2024-02-17T12:58:25.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.018+0000 7f6d6bd66740 20 client.18579 trim_cache size 0 max 16384 2024-02-17T12:58:25.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.029+0000 7f6d6bd66740 1 Processor -- start 2024-02-17T12:58:25.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.029+0000 7f6d6bd66740 1 -- start start 2024-02-17T12:58:25.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d6bd66740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 0x55c8dbe40ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d6bd66740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38b70 0x55c8dbe3f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d6bd66740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe41750 0x55c8dbe39a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d6bd66740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8dbd25130 con 0x55c8dbe38250 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d6bd66740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8dbe33980 con 0x55c8dbe41750 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d6bd66740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8dbe2e2b0 con 0x55c8dbe38b70 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d62d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 0x55c8dbe40ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d62d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 0x55c8dbe40ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42556/0 (socket says 172.21.15.18:42556) 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d62d9d640 1 -- 172.21.15.18:0/3992915345 learned_addr learned my addr 172.21.15.18:0/3992915345 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d6359e640 1 --2- 172.21.15.18:0/3992915345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe41750 0x55c8dbe39a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d5bfff640 1 --2- 172.21.15.18:0/3992915345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38b70 0x55c8dbe3f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d62d9d640 1 -- 172.21.15.18:0/3992915345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38b70 msgr2=0x55c8dbe3f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/3992915345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38b70 0x55c8dbe3f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d62d9d640 1 -- 172.21.15.18:0/3992915345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe41750 msgr2=0x55c8dbe39a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/3992915345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe41750 0x55c8dbe39a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d62d9d640 1 -- 172.21.15.18:0/3992915345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8dbd3ac20 con 0x55c8dbe38250 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d6359e640 1 --2- 172.21.15.18:0/3992915345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe41750 0x55c8dbe39a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d5bfff640 1 --2- 172.21.15.18:0/3992915345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38b70 0x55c8dbe3f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/3992915345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 0x55c8dbe40ed0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006010 tx=0x7f6d5400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704bb8f7358eed5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.030+0000 7f6d58ff9640 1 -- 172.21.15.18:0/3992915345 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54020040 con 0x55c8dbe38250 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.031+0000 7f6d58ff9640 1 -- 172.21.15.18:0/3992915345 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d54021040 con 0x55c8dbe38250 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.031+0000 7f6d58ff9640 1 -- 172.21.15.18:0/3992915345 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54011770 con 0x55c8dbe38250 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.031+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3992915345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 msgr2=0x55c8dbe40ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.031+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/3992915345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 0x55c8dbe40ed0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006010 tx=0x7f6d5400b5e0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.031+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3992915345 shutdown_connections 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.031+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/3992915345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe41750 0x55c8dbe39a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.031+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/3992915345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38b70 0x55c8dbe3f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.031+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/3992915345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe38250 0x55c8dbe40ed0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.031+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3992915345 >> 172.21.15.18:0/3992915345 conn(0x55c8dbe32980 msgr2=0x55c8dbe2da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.031+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3992915345 shutdown_connections 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.031+0000 7f6d6bd66740 1 -- 172.21.15.18:0/3992915345 wait complete. 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6bd66740 1 -- start start 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6bd66740 1 Processor -- start 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6bd66740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38250 0x55c8dbe47100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6bd66740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38b70 0x55c8dbe47640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6bd66740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 0x55c8dbe47b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6bd66740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8dbea80a0 con 0x55c8dbe41750 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6bd66740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8dbea8210 con 0x55c8dbe38250 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6bd66740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8dbea83d0 con 0x55c8dbe38b70 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6bd66740 10 client.0 _ll_register_callbacks cb 0x7f6d5400b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 0x55c8dbe47b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 0x55c8dbe47b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42562/0 (socket says 172.21.15.18:42562) 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6359e640 1 -- 172.21.15.18:0/1050981270 learned_addr learned my addr 172.21.15.18:0/1050981270 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d5bfff640 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38b70 0x55c8dbe47640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38250 0x55c8dbe47100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6359e640 1 -- 172.21.15.18:0/1050981270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38b70 msgr2=0x55c8dbe47640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6359e640 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38b70 0x55c8dbe47640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6359e640 1 -- 172.21.15.18:0/1050981270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38250 msgr2=0x55c8dbe47100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6359e640 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38250 0x55c8dbe47100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6359e640 1 -- 172.21.15.18:0/1050981270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8dbd62060 con 0x55c8dbe41750 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d5bfff640 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38b70 0x55c8dbe47640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d6359e640 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 0x55c8dbe47b80 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00efd0 tx=0x7f6d5c0a3320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4d45a00efac3bf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.032+0000 7f6d4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d4a7fc640 1 -- 172.21.15.18:0/1050981270 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0b9240 con 0x55c8dbe41750 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d4a7fc640 1 -- 172.21.15.18:0/1050981270 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6d5c0cd070 con 0x55c8dbe41750 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d6bd66740 1 -- 172.21.15.18:0/1050981270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d5401aaa0 con 0x55c8dbe41750 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d4a7fc640 1 -- 172.21.15.18:0/1050981270 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d5c09a640 con 0x55c8dbe41750 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d6bd66740 10 client.18585 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:25.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d6bd66740 1 -- 172.21.15.18:0/1050981270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55c8dbd608c0 con 0x55c8dbe41750 2024-02-17T12:58:25.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d4a7fc640 1 -- 172.21.15.18:0/1050981270 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d5401aaa0 con 0x55c8dbe41750 2024-02-17T12:58:25.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d4a7fc640 1 -- 172.21.15.18:0/1050981270 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f6d5c018040 con 0x55c8dbe41750 2024-02-17T12:58:25.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d6bd66740 10 client.18585 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:25.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d6bd66740 10 client.18585 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:58:25.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d6bd66740 20 client.18585 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:25.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c8dbea9720 0x55c8dbeabb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d6bd66740 4 client.18585 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d6bd66740 1 -- 172.21.15.18:0/1050981270 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c8dbb2b890 con 0x55c8dbea9720 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.033+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c8dbea9720 0x55c8dbeabb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d4a7fc640 10 client.18585 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d62d9d640 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c8dbea9720 0x55c8dbeabb10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9cc86c9f618770f8 server_cookie=25b7909fd615ef6 in_seq=0 out_seq=0 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d4a7fc640 1 -- 172.21.15.18:0/1050981270 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c8dbb2b890 con 0x55c8dbea9720 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d4a7fc640 10 client.18585 handle_command_reply: tid=0 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d6bd66740 1 client.18585 shutdown 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d6bd66740 1 -- 172.21.15.18:0/1050981270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 msgr2=0x55c8dbe47b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 0x55c8dbe47b80 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00efd0 tx=0x7f6d5c0a3320 comp rx=0 tx=0).stop 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d6bd66740 1 -- 172.21.15.18:0/1050981270 shutdown_connections 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c8dbe41750 0x55c8dbe47b80 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c8dbe38b70 0x55c8dbe47640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c8dbea9720 0x55c8dbeabb10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d6bd66740 1 --2- 172.21.15.18:0/1050981270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8dbe38250 0x55c8dbe47100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d4a7fc640 0 client.18585 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:58:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d6bd66740 1 -- 172.21.15.18:0/1050981270 >> 172.21.15.18:0/1050981270 conn(0x55c8dbe32980 msgr2=0x55c8dbe39a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d6bd66740 1 -- 172.21.15.18:0/1050981270 shutdown_connections 2024-02-17T12:58:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d6bd66740 1 -- 172.21.15.18:0/1050981270 wait complete. 2024-02-17T12:58:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.034+0000 7f6d6bd66740 20 client.18585 trim_cache size 0 max 16384 2024-02-17T12:58:25.033 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:58:25.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.035+0000 7f6d690b5640 1 -- 172.21.15.18:0/2595021232 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6d500a2230 msgr2=0x7f6d500a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.035+0000 7f6d690b5640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6d500a2230 0x7f6d500a46f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009230 tx=0x7f6d4c02c680 comp rx=0 tx=0).stop 2024-02-17T12:58:25.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.035+0000 7f6d690b5640 1 -- 172.21.15.18:0/2595021232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d64197590 msgr2=0x7f6d64197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.035+0000 7f6d690b5640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d64197590 0x7f6d64197a40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00eaa0 tx=0x7f6d5c00ef60 comp rx=0 tx=0).stop 2024-02-17T12:58:25.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.035+0000 7f6d690b5640 1 -- 172.21.15.18:0/2595021232 shutdown_connections 2024-02-17T12:58:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.035+0000 7f6d690b5640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6d500a2230 0x7f6d500a46f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.035+0000 7f6d690b5640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d64197590 0x7f6d64197a40 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.035+0000 7f6d690b5640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d64106520 0x7f6d6419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.035+0000 7f6d690b5640 1 --2- 172.21.15.18:0/2595021232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d640ff910 0x7f6d6419d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.035+0000 7f6d690b5640 1 -- 172.21.15.18:0/2595021232 >> 172.21.15.18:0/2595021232 conn(0x7f6d64076640 msgr2=0x7f6d641047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.035+0000 7f6d690b5640 1 -- 172.21.15.18:0/2595021232 shutdown_connections 2024-02-17T12:58:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.035+0000 7f6d690b5640 1 -- 172.21.15.18:0/2595021232 wait complete. 2024-02-17T12:58:25.037 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:58:25.037 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:58:25.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.153+0000 7f926b8df640 1 -- 172.21.15.18:0/3361336359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9264101380 msgr2=0x7f9264101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.153+0000 7f926b8df640 1 --2- 172.21.15.18:0/3361336359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9264101380 0x7f9264101760 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f925c009930 tx=0x7f925c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:25.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 -- 172.21.15.18:0/3361336359 shutdown_connections 2024-02-17T12:58:25.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 --2- 172.21.15.18:0/3361336359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9264106450 0x7f926410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 --2- 172.21.15.18:0/3361336359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9264101d30 0x7f9264105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 --2- 172.21.15.18:0/3361336359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9264101380 0x7f9264101760 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 -- 172.21.15.18:0/3361336359 >> 172.21.15.18:0/3361336359 conn(0x7f9264078580 msgr2=0x7f9264078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 -- 172.21.15.18:0/3361336359 shutdown_connections 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 -- 172.21.15.18:0/3361336359 wait complete. 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 Processor -- start 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 -- start start 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9264101380 0x7f9264197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9264106450 0x7f9264197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f926419c010 0x7f926419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926419ca00 con 0x7f9264106450 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926419cb70 con 0x7f9264101380 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.154+0000 7f926b8df640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926419cce0 con 0x7f926419c010 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f9268e7a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9264106450 0x7f9264197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f9268e7a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9264106450 0x7f9264197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42590/0 (socket says 172.21.15.18:42590) 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f926967b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9264101380 0x7f9264197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f9268e7a640 1 -- 172.21.15.18:0/4148884725 learned_addr learned my addr 172.21.15.18:0/4148884725 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f9269e7c640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f926419c010 0x7f926419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f9268e7a640 1 -- 172.21.15.18:0/4148884725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f926419c010 msgr2=0x7f926419c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f9268e7a640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f926419c010 0x7f926419c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f9268e7a640 1 -- 172.21.15.18:0/4148884725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9264101380 msgr2=0x7f9264197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f9268e7a640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9264101380 0x7f9264197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f9268e7a640 1 -- 172.21.15.18:0/4148884725 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c009560 con 0x7f9264106450 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f926967b640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9264101380 0x7f9264197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f9268e7a640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9264106450 0x7f9264197c90 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f925800e9c0 tx=0x7f925800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f92627fc640 1 -- 172.21.15.18:0/4148884725 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258009700 con 0x7f9264106450 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f92627fc640 1 -- 172.21.15.18:0/4148884725 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f925800cd80 con 0x7f9264106450 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f926b8df640 1 -- 172.21.15.18:0/4148884725 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92641a42c0 con 0x7f9264106450 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f92627fc640 1 -- 172.21.15.18:0/4148884725 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258010430 con 0x7f9264106450 2024-02-17T12:58:25.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.155+0000 7f926b8df640 1 -- 172.21.15.18:0/4148884725 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92641a47c0 con 0x7f9264106450 2024-02-17T12:58:25.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.157+0000 7f92627fc640 1 -- 172.21.15.18:0/4148884725 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9258005000 con 0x7f9264106450 2024-02-17T12:58:25.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f926e590740 1 Processor -- start 2024-02-17T12:58:25.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f926e590740 1 -- start start 2024-02-17T12:58:25.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f92627fc640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f924c0a22b0 0x7f924c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f926e590740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 0x55a2c42f6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f926e590740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6b70 0x55a2c42ff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f926e590740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c4301b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f926e590740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c42b1960 con 0x55a2c42f6250 2024-02-17T12:58:25.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f926e590740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c4265310 con 0x55a2c42ff750 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f926e590740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c4262840 con 0x55a2c42f6b70 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f926967b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 0x55a2c42f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f9269e7c640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f924c0a22b0 0x7f924c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f92627fc640 1 -- 172.21.15.18:0/4148884725 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9258014070 con 0x7f9264106450 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f926967b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 0x55a2c42f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42602/0 (socket says 172.21.15.18:42602) 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f926967b640 1 -- 172.21.15.18:0/1594850204 learned_addr learned my addr 172.21.15.18:0/1594850204 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f9268e7a640 1 --2- 172.21.15.18:0/1594850204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6b70 0x55a2c42ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.158+0000 7f9269e7c640 1 --2- 172.21.15.18:0/1594850204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c4301b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.159+0000 7f926967b640 1 -- 172.21.15.18:0/1594850204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6b70 msgr2=0x55a2c42ff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.159+0000 7f926967b640 1 --2- 172.21.15.18:0/1594850204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6b70 0x55a2c42ff210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.159+0000 7f926967b640 1 -- 172.21.15.18:0/1594850204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 msgr2=0x55a2c4301b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.159+0000 7f926967b640 1 --2- 172.21.15.18:0/1594850204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c4301b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.159+0000 7f926967b640 1 -- 172.21.15.18:0/1594850204 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2c4220060 con 0x55a2c42f6250 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.159+0000 7f9269e7c640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f924c0a22b0 0x7f924c0a4770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9254012590 tx=0x7f9254012990 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.159+0000 7f926967b640 1 --2- 172.21.15.18:0/1594850204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 0x55a2c42f6630 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f925c005e00 tx=0x7f925c004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90be31502a389123 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.159+0000 7f924b7fe640 1 -- 172.21.15.18:0/1594850204 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c02c8a0 con 0x55a2c42f6250 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.159+0000 7f924b7fe640 1 -- 172.21.15.18:0/1594850204 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f925c004510 con 0x55a2c42f6250 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.159+0000 7f924b7fe640 1 -- 172.21.15.18:0/1594850204 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c005490 con 0x55a2c42f6250 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.160+0000 7f926e590740 1 -- 172.21.15.18:0/1594850204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 msgr2=0x55a2c42f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.160+0000 7f926e590740 1 --2- 172.21.15.18:0/1594850204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 0x55a2c42f6630 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f925c005e00 tx=0x7f925c004a70 comp rx=0 tx=0).stop 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.160+0000 7f926e590740 1 -- 172.21.15.18:0/1594850204 shutdown_connections 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.160+0000 7f926e590740 1 --2- 172.21.15.18:0/1594850204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c4301b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.160+0000 7f926e590740 1 --2- 172.21.15.18:0/1594850204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6b70 0x55a2c42ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.160+0000 7f926e590740 1 --2- 172.21.15.18:0/1594850204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 0x55a2c42f6630 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.160+0000 7f926e590740 1 -- 172.21.15.18:0/1594850204 >> 172.21.15.18:0/1594850204 conn(0x55a2c42f0980 msgr2=0x55a2c42f0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.160+0000 7f926e590740 1 -- 172.21.15.18:0/1594850204 shutdown_connections 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.160+0000 7f926e590740 1 -- 172.21.15.18:0/1594850204 wait complete. 2024-02-17T12:58:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.160+0000 7f926e590740 1 -- start start 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.160+0000 7f926e590740 1 Processor -- start 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f926e590740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42f6250 0x55a2c42fc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f926e590740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 0x55a2c42fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f926e590740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42ff750 0x55a2c42fd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f926e590740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c425e020 con 0x55a2c42f6b70 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f926e590740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c3f31430 con 0x55a2c42f6250 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f926e590740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c3eede00 con 0x55a2c42ff750 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f926e590740 10 client.0 _ll_register_callbacks cb 0x55a2c4205cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9268e7a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 0x55a2c42fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9268e7a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 0x55a2c42fcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42614/0 (socket says 172.21.15.18:42614) 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9268e7a640 1 -- 172.21.15.18:0/3677785958 learned_addr learned my addr 172.21.15.18:0/3677785958 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f926967b640 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42f6250 0x55a2c42fc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9269e7c640 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42ff750 0x55a2c42fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9268e7a640 1 -- 172.21.15.18:0/3677785958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42ff750 msgr2=0x55a2c42fd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9268e7a640 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42ff750 0x55a2c42fd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9268e7a640 1 -- 172.21.15.18:0/3677785958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42f6250 msgr2=0x55a2c42fc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9268e7a640 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42f6250 0x55a2c42fc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9268e7a640 1 -- 172.21.15.18:0/3677785958 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926410e830 con 0x55a2c42f6b70 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9269e7c640 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42ff750 0x55a2c42fd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9268e7a640 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 0x55a2c42fcc30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f9258097310 tx=0x7f9258064580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d242d3dcf9d23a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9248ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9248ff9640 1 -- 172.21.15.18:0/3677785958 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925809f000 con 0x55a2c42f6b70 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9248ff9640 1 -- 172.21.15.18:0/3677785958 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f92580a3650 con 0x55a2c42f6b70 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f926e590740 1 -- 172.21.15.18:0/3677785958 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2c3f5d7e0 con 0x55a2c42f6b70 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f9248ff9640 1 -- 172.21.15.18:0/3677785958 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92580991b0 con 0x55a2c42f6b70 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.161+0000 7f926e590740 10 client.18597 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:25.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.162+0000 7f926e590740 1 -- 172.21.15.18:0/3677785958 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55a2c425c6a0 con 0x55a2c42f6b70 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.162+0000 7f9248ff9640 1 -- 172.21.15.18:0/3677785958 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92580b7020 con 0x55a2c42f6b70 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.162+0000 7f9248ff9640 1 -- 172.21.15.18:0/3677785958 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f925801f060 con 0x55a2c42f6b70 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.162+0000 7f926e590740 10 client.18597 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.162+0000 7f926e590740 10 client.18597 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.162+0000 7f926e590740 20 client.18597 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.162+0000 7f926e590740 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a2c430faf0 0x55a2c4311ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.162+0000 7f926e590740 4 client.18597 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.162+0000 7f926e590740 1 -- 172.21.15.18:0/3677785958 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2c3f9a720 con 0x55a2c430faf0 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.162+0000 7f926967b640 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a2c430faf0 0x55a2c4311ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f926967b640 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a2c430faf0 0x55a2c4311ee0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b43dff5153954822 server_cookie=175fbf93c0b67669 in_seq=0 out_seq=0 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f9248ff9640 10 client.18597 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f9248ff9640 1 -- 172.21.15.18:0/3677785958 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a2c3f9a720 con 0x55a2c430faf0 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f9248ff9640 10 client.18597 handle_command_reply: tid=0 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f926e590740 1 client.18597 shutdown 2024-02-17T12:58:25.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f926e590740 1 -- 172.21.15.18:0/3677785958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 msgr2=0x55a2c42fcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f926e590740 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 0x55a2c42fcc30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f9258097310 tx=0x7f9258064580 comp rx=0 tx=0).stop 2024-02-17T12:58:25.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f926e590740 1 -- 172.21.15.18:0/3677785958 shutdown_connections 2024-02-17T12:58:25.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f926e590740 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a2c430faf0 0x55a2c4311ee0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f926e590740 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42ff750 0x55a2c42fd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f926e590740 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 0x55a2c42fcc30 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f9248ff9640 0 client.18597 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:58:25.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f926e590740 1 --2- 172.21.15.18:0/3677785958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42f6250 0x55a2c42fc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f926e590740 1 -- 172.21.15.18:0/3677785958 >> 172.21.15.18:0/3677785958 conn(0x55a2c42f0980 msgr2=0x55a2c42f26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.163+0000 7f926e590740 1 -- 172.21.15.18:0/3677785958 shutdown_connections 2024-02-17T12:58:25.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.164+0000 7f926e590740 1 -- 172.21.15.18:0/3677785958 wait complete. 2024-02-17T12:58:25.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.164+0000 7f926e590740 20 client.18597 trim_cache size 0 max 16384 2024-02-17T12:58:25.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926e590740 1 Processor -- start 2024-02-17T12:58:25.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926e590740 1 -- start start 2024-02-17T12:58:25.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926e590740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 0x55a2c42fee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926e590740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6b70 0x55a2c42fcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926e590740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c42f9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926e590740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c41e3130 con 0x55a2c42f6250 2024-02-17T12:58:25.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926e590740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c42ec5c0 con 0x55a2c42ff750 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926e590740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c43170f0 con 0x55a2c42f6b70 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926967b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 0x55a2c42fee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926967b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 0x55a2c42fee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42620/0 (socket says 172.21.15.18:42620) 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926967b640 1 -- 172.21.15.18:0/3584738164 learned_addr learned my addr 172.21.15.18:0/3584738164 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f9268e7a640 1 --2- 172.21.15.18:0/3584738164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6b70 0x55a2c42fcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f9269e7c640 1 --2- 172.21.15.18:0/3584738164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c42f9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926967b640 1 -- 172.21.15.18:0/3584738164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6b70 msgr2=0x55a2c42fcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926967b640 1 --2- 172.21.15.18:0/3584738164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6b70 0x55a2c42fcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926967b640 1 -- 172.21.15.18:0/3584738164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 msgr2=0x55a2c42f9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926967b640 1 --2- 172.21.15.18:0/3584738164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c42f9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f926967b640 1 -- 172.21.15.18:0/3584738164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2c41f8c20 con 0x55a2c42f6250 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.176+0000 7f9269e7c640 1 --2- 172.21.15.18:0/3584738164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c42f9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.177+0000 7f926967b640 1 --2- 172.21.15.18:0/3584738164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 0x55a2c42fee20 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f925c004c50 tx=0x7f925c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5956809f96d42ed7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.177+0000 7f924affd640 1 -- 172.21.15.18:0/3584738164 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c02c7e0 con 0x55a2c42f6250 2024-02-17T12:58:25.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.177+0000 7f924affd640 1 -- 172.21.15.18:0/3584738164 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f925c005680 con 0x55a2c42f6250 2024-02-17T12:58:25.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.177+0000 7f924affd640 1 -- 172.21.15.18:0/3584738164 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c0346a0 con 0x55a2c42f6250 2024-02-17T12:58:25.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 -- 172.21.15.18:0/3584738164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 msgr2=0x55a2c42fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 --2- 172.21.15.18:0/3584738164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 0x55a2c42fee20 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f925c004c50 tx=0x7f925c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:25.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 -- 172.21.15.18:0/3584738164 shutdown_connections 2024-02-17T12:58:25.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 --2- 172.21.15.18:0/3584738164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c42f9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 --2- 172.21.15.18:0/3584738164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6b70 0x55a2c42fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 --2- 172.21.15.18:0/3584738164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6250 0x55a2c42fee20 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 -- 172.21.15.18:0/3584738164 >> 172.21.15.18:0/3584738164 conn(0x55a2c42f0980 msgr2=0x55a2c42f79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 -- 172.21.15.18:0/3584738164 shutdown_connections 2024-02-17T12:58:25.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 -- 172.21.15.18:0/3584738164 wait complete. 2024-02-17T12:58:25.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 -- start start 2024-02-17T12:58:25.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 Processor -- start 2024-02-17T12:58:25.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6250 0x55a2c4304fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 0x55a2c4305520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c4305a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c4365fd0 con 0x55a2c42f6b70 2024-02-17T12:58:25.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c4366110 con 0x55a2c42ff750 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c4366250 con 0x55a2c42f6250 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f926e590740 10 client.0 _ll_register_callbacks cb 0x7f925c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.178+0000 7f9268e7a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 0x55a2c4305520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9268e7a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 0x55a2c4305520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42624/0 (socket says 172.21.15.18:42624) 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9268e7a640 1 -- 172.21.15.18:0/749921234 learned_addr learned my addr 172.21.15.18:0/749921234 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f926967b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6250 0x55a2c4304fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9269e7c640 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c4305a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9268e7a640 1 -- 172.21.15.18:0/749921234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6250 msgr2=0x55a2c4304fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9268e7a640 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6250 0x55a2c4304fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9268e7a640 1 -- 172.21.15.18:0/749921234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 msgr2=0x55a2c4305a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9268e7a640 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c4305a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9268e7a640 1 -- 172.21.15.18:0/749921234 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2c4220060 con 0x55a2c42f6b70 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f926967b640 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6250 0x55a2c4304fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:25.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9268e7a640 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 0x55a2c4305520 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f92580646d0 tx=0x7f925809b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e5580afa5c3f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9248ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9248ff9640 1 -- 172.21.15.18:0/749921234 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258064300 con 0x55a2c42f6b70 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9248ff9640 1 -- 172.21.15.18:0/749921234 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f92580b8080 con 0x55a2c42f6b70 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f926e590740 1 -- 172.21.15.18:0/749921234 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f925c004510 con 0x55a2c42f6b70 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9248ff9640 1 -- 172.21.15.18:0/749921234 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f925809a9f0 con 0x55a2c42f6b70 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f926e590740 10 client.18603 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f926e590740 1 -- 172.21.15.18:0/749921234 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55a2c421e8c0 con 0x55a2c42f6b70 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.179+0000 7f9248ff9640 1 -- 172.21.15.18:0/749921234 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f925c004510 con 0x55a2c42f6b70 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.180+0000 7f9248ff9640 1 -- 172.21.15.18:0/749921234 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f92580b92b0 con 0x55a2c42f6b70 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.180+0000 7f926e590740 10 client.18603 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.180+0000 7f926e590740 10 client.18603 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.180+0000 7f926e590740 20 client.18603 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.180+0000 7f926e590740 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a2c43674f0 0x55a2c4369910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.180+0000 7f926e590740 4 client.18603 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.180+0000 7f926e590740 1 -- 172.21.15.18:0/749921234 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a2c3fe9890 con 0x55a2c43674f0 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.180+0000 7f926967b640 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a2c43674f0 0x55a2c4369910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.180+0000 7f9248ff9640 10 client.18603 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.180+0000 7f926967b640 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a2c43674f0 0x55a2c4369910 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f676ba08ff88308c server_cookie=99b204ff8606998d in_seq=0 out_seq=0 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.180+0000 7f9248ff9640 1 -- 172.21.15.18:0/749921234 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a2c3fe9890 con 0x55a2c43674f0 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.180+0000 7f9248ff9640 10 client.18603 handle_command_reply: tid=0 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926e590740 1 client.18603 shutdown 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926e590740 1 -- 172.21.15.18:0/749921234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 msgr2=0x55a2c4305520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926e590740 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 0x55a2c4305520 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f92580646d0 tx=0x7f925809b630 comp rx=0 tx=0).stop 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926e590740 1 -- 172.21.15.18:0/749921234 shutdown_connections 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926e590740 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55a2c43674f0 0x55a2c4369910 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926e590740 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2c42ff750 0x55a2c4305a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f9248ff9640 0 client.18603 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926e590740 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a2c42f6b70 0x55a2c4305520 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926e590740 1 --2- 172.21.15.18:0/749921234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2c42f6250 0x55a2c4304fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926e590740 1 -- 172.21.15.18:0/749921234 >> 172.21.15.18:0/749921234 conn(0x55a2c42f0980 msgr2=0x55a2c42ef2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926e590740 1 -- 172.21.15.18:0/749921234 shutdown_connections 2024-02-17T12:58:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926e590740 1 -- 172.21.15.18:0/749921234 wait complete. 2024-02-17T12:58:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926e590740 20 client.18603 trim_cache size 0 max 16384 2024-02-17T12:58:25.179 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:58:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926b8df640 1 -- 172.21.15.18:0/4148884725 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f924c0a22b0 msgr2=0x7f924c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926b8df640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f924c0a22b0 0x7f924c0a4770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9254012590 tx=0x7f9254012990 comp rx=0 tx=0).stop 2024-02-17T12:58:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926b8df640 1 -- 172.21.15.18:0/4148884725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9264106450 msgr2=0x7f9264197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926b8df640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9264106450 0x7f9264197c90 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f925800e9c0 tx=0x7f925800ee80 comp rx=0 tx=0).stop 2024-02-17T12:58:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926b8df640 1 -- 172.21.15.18:0/4148884725 shutdown_connections 2024-02-17T12:58:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926b8df640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f924c0a22b0 0x7f924c0a4770 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926b8df640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f926419c010 0x7f926419c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926b8df640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9264106450 0x7f9264197c90 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926b8df640 1 --2- 172.21.15.18:0/4148884725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9264101380 0x7f9264197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926b8df640 1 -- 172.21.15.18:0/4148884725 >> 172.21.15.18:0/4148884725 conn(0x7f9264078580 msgr2=0x7f92640ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.181+0000 7f926b8df640 1 -- 172.21.15.18:0/4148884725 shutdown_connections 2024-02-17T12:58:25.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.182+0000 7f926b8df640 1 -- 172.21.15.18:0/4148884725 wait complete. 2024-02-17T12:58:25.182 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:24 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/963584111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:58:25.183 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:58:25.183 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:58:25.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:24 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/963584111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:58:25.286 INFO:tasks.workunit.client.0.smithi018.stdout: 1 438005 5.26 MB/sec execute 340 sec latency 401.375 ms 2024-02-17T12:58:25.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.296+0000 7f6535156640 1 -- 172.21.15.18:0/2491932724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6530101360 msgr2=0x7f6530101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.296+0000 7f6535156640 1 --2- 172.21.15.18:0/2491932724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6530101360 0x7f6530101740 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6520009890 tx=0x7f652002bb70 comp rx=0 tx=0).stop 2024-02-17T12:58:25.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.296+0000 7f6535156640 1 -- 172.21.15.18:0/2491932724 shutdown_connections 2024-02-17T12:58:25.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.296+0000 7f6535156640 1 --2- 172.21.15.18:0/2491932724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6530106430 0x7f653010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.296+0000 7f6535156640 1 --2- 172.21.15.18:0/2491932724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6530101d10 0x7f6530105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.296+0000 7f6535156640 1 --2- 172.21.15.18:0/2491932724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6530101360 0x7f6530101740 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.296+0000 7f6535156640 1 -- 172.21.15.18:0/2491932724 >> 172.21.15.18:0/2491932724 conn(0x7f65300fcbe0 msgr2=0x7f65300ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.296+0000 7f6535156640 1 -- 172.21.15.18:0/2491932724 shutdown_connections 2024-02-17T12:58:25.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.296+0000 7f6535156640 1 -- 172.21.15.18:0/2491932724 wait complete. 2024-02-17T12:58:25.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f6535156640 1 Processor -- start 2024-02-17T12:58:25.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f6535156640 1 -- start start 2024-02-17T12:58:25.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f6535156640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6530101360 0x7f6530073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f6535156640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6530106430 0x7f65300739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f6535156640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6530073f00 0x7f65301a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f6535156640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65301a6000 con 0x7f6530073f00 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f6535156640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65301a6140 con 0x7f6530106430 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f6535156640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65301a62d0 con 0x7f6530101360 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f652f59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6530073f00 0x7f65301a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f652ed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6530101360 0x7f6530073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f652f59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6530073f00 0x7f65301a5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42642/0 (socket says 172.21.15.18:42642) 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f652ed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6530101360 0x7f6530073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40992/0 (socket says 172.21.15.18:40992) 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f652f59e640 1 -- 172.21.15.18:0/2685027985 learned_addr learned my addr 172.21.15.18:0/2685027985 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.297+0000 7f6527fff640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6530106430 0x7f65300739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.298+0000 7f652f59e640 1 -- 172.21.15.18:0/2685027985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6530101360 msgr2=0x7f6530073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.298+0000 7f652f59e640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6530101360 0x7f6530073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.298+0000 7f652f59e640 1 -- 172.21.15.18:0/2685027985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6530106430 msgr2=0x7f65300739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.298+0000 7f652f59e640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6530106430 0x7f65300739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.298+0000 7f652f59e640 1 -- 172.21.15.18:0/2685027985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6518009560 con 0x7f6530073f00 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.298+0000 7f6527fff640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6530106430 0x7f65300739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:25.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.298+0000 7f652f59e640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6530073f00 0x7f65301a5ac0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f652800ead0 tx=0x7f652800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.298+0000 7f652cd99640 1 -- 172.21.15.18:0/2685027985 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652800cde0 con 0x7f6530073f00 2024-02-17T12:58:25.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.298+0000 7f6535156640 1 -- 172.21.15.18:0/2685027985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6520009490 con 0x7f6530073f00 2024-02-17T12:58:25.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.298+0000 7f652cd99640 1 -- 172.21.15.18:0/2685027985 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6528005020 con 0x7f6530073f00 2024-02-17T12:58:25.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.298+0000 7f652cd99640 1 -- 172.21.15.18:0/2685027985 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65280116b0 con 0x7f6530073f00 2024-02-17T12:58:25.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.298+0000 7f6535156640 1 -- 172.21.15.18:0/2685027985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65301a6930 con 0x7f6530073f00 2024-02-17T12:58:25.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.300+0000 7f652cd99640 1 -- 172.21.15.18:0/2685027985 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6528005520 con 0x7f6530073f00 2024-02-17T12:58:25.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f6537e06740 1 Processor -- start 2024-02-17T12:58:25.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f6537e06740 1 -- start start 2024-02-17T12:58:25.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f6537e06740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891250 0x5616e2891630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f6537e06740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891b70 0x5616e289a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f6537e06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e289cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f6537e06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e284c960 con 0x5616e289a750 2024-02-17T12:58:25.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f6537e06740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e2800310 con 0x5616e2891250 2024-02-17T12:58:25.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f6537e06740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e27fd840 con 0x5616e2891b70 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f652cd99640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f651c0a22b0 0x7f651c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f6527fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e289cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f6527fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e289cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42648/0 (socket says 172.21.15.18:42648) 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f6527fff640 1 -- 172.21.15.18:0/3565814926 learned_addr learned my addr 172.21.15.18:0/3565814926 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f652ed9d640 1 --2- 172.21.15.18:0/3565814926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891b70 0x5616e289a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f6527fff640 1 --2- 172.21.15.18:0/3565814926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891250 0x5616e2891630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.301+0000 7f652f59e640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f651c0a22b0 0x7f651c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.302+0000 7f652cd99640 1 -- 172.21.15.18:0/2685027985 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f65280ce230 con 0x7f6530073f00 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.302+0000 7f6527fff640 1 -- 172.21.15.18:0/3565814926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891b70 msgr2=0x5616e289a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.302+0000 7f6527fff640 1 --2- 172.21.15.18:0/3565814926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891b70 0x5616e289a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.302+0000 7f6527fff640 1 -- 172.21.15.18:0/3565814926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891250 msgr2=0x5616e2891630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.302+0000 7f6527fff640 1 --2- 172.21.15.18:0/3565814926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891250 0x5616e2891630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.302+0000 7f6527fff640 1 -- 172.21.15.18:0/3565814926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e27bb060 con 0x5616e289a750 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.302+0000 7f652ed9d640 1 --2- 172.21.15.18:0/3565814926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891b70 0x5616e289a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.302+0000 7f6527fff640 1 --2- 172.21.15.18:0/3565814926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e289cb40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6518011d10 tx=0x7f6518009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e10a81ff6dac2ce server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.302+0000 7f652f59e640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f651c0a22b0 0x7f651c0a4770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f652800cfd0 tx=0x7f652800cf60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.302+0000 7f65257fa640 1 -- 172.21.15.18:0/3565814926 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651800fc50 con 0x5616e289a750 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.302+0000 7f65257fa640 1 -- 172.21.15.18:0/3565814926 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f651800fdb0 con 0x5616e289a750 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.302+0000 7f65257fa640 1 -- 172.21.15.18:0/3565814926 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65180047a0 con 0x5616e289a750 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 -- 172.21.15.18:0/3565814926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 msgr2=0x5616e289cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 --2- 172.21.15.18:0/3565814926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e289cb40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6518011d10 tx=0x7f6518009c90 comp rx=0 tx=0).stop 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 -- 172.21.15.18:0/3565814926 shutdown_connections 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 --2- 172.21.15.18:0/3565814926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e289cb40 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 --2- 172.21.15.18:0/3565814926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891b70 0x5616e289a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 --2- 172.21.15.18:0/3565814926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891250 0x5616e2891630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 -- 172.21.15.18:0/3565814926 >> 172.21.15.18:0/3565814926 conn(0x5616e288b980 msgr2=0x5616e288bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 -- 172.21.15.18:0/3565814926 shutdown_connections 2024-02-17T12:58:25.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 -- 172.21.15.18:0/3565814926 wait complete. 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 -- start start 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 Processor -- start 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891250 0x5616e28977b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891b70 0x5616e2897cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.303+0000 7f6537e06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e2898230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6537e06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e27f9020 con 0x5616e289a750 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6537e06740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e24cc430 con 0x5616e2891b70 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6537e06740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e2488e00 con 0x5616e2891250 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6537e06740 10 client.0 _ll_register_callbacks cb 0x5616e27a0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6527fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e2898230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6527fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e2898230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42658/0 (socket says 172.21.15.18:42658) 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6527fff640 1 -- 172.21.15.18:0/4205639307 learned_addr learned my addr 172.21.15.18:0/4205639307 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6527fff640 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891250 0x5616e28977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f652ed9d640 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891b70 0x5616e2897cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6527fff640 1 -- 172.21.15.18:0/4205639307 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891250 msgr2=0x5616e28977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6527fff640 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891250 0x5616e28977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6527fff640 1 -- 172.21.15.18:0/4205639307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891b70 msgr2=0x5616e2897cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6527fff640 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891b70 0x5616e2897cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6527fff640 1 -- 172.21.15.18:0/4205639307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6518019040 con 0x5616e289a750 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f652ed9d640 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891b70 0x5616e2897cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6527fff640 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e2898230 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6518001eb0 tx=0x7f6518005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3385574820605ca server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f65167fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f65167fc640 1 -- 172.21.15.18:0/4205639307 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518004020 con 0x5616e289a750 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f65167fc640 1 -- 172.21.15.18:0/4205639307 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6518000e40 con 0x5616e289a750 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6537e06740 1 -- 172.21.15.18:0/4205639307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616e24f87e0 con 0x5616e289a750 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f65167fc640 1 -- 172.21.15.18:0/4205639307 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6518004180 con 0x5616e289a750 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6537e06740 10 client.18615 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.304+0000 7f6537e06740 1 -- 172.21.15.18:0/4205639307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5616e27f76a0 con 0x5616e289a750 2024-02-17T12:58:25.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.305+0000 7f65167fc640 1 -- 172.21.15.18:0/4205639307 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5616e24f87e0 con 0x5616e289a750 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.305+0000 7f65167fc640 1 -- 172.21.15.18:0/4205639307 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f65180035f0 con 0x5616e289a750 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.305+0000 7f6537e06740 10 client.18615 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.305+0000 7f6537e06740 10 client.18615 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.305+0000 7f6537e06740 20 client.18615 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.305+0000 7f6537e06740 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5616e28aa9d0 0x5616e28acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.305+0000 7f6537e06740 4 client.18615 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.305+0000 7f6537e06740 1 -- 172.21.15.18:0/4205639307 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616e2535720 con 0x5616e28aa9d0 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.305+0000 7f652ed9d640 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5616e28aa9d0 0x5616e28acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.305+0000 7f65167fc640 10 client.18615 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.305+0000 7f652ed9d640 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5616e28aa9d0 0x5616e28acdc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27c6a26fa48aaed4 server_cookie=670bc5cccfa0848e in_seq=0 out_seq=0 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f65167fc640 1 -- 172.21.15.18:0/4205639307 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5616e2535720 con 0x5616e28aa9d0 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f65167fc640 10 client.18615 handle_command_reply: tid=0 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f6537e06740 1 client.18615 shutdown 2024-02-17T12:58:25.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f6537e06740 1 -- 172.21.15.18:0/4205639307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 msgr2=0x5616e2898230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f6537e06740 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e2898230 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6518001eb0 tx=0x7f6518005640 comp rx=0 tx=0).stop 2024-02-17T12:58:25.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f6537e06740 1 -- 172.21.15.18:0/4205639307 shutdown_connections 2024-02-17T12:58:25.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f6537e06740 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5616e28aa9d0 0x5616e28acdc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f6537e06740 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e2898230 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f6537e06740 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891b70 0x5616e2897cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f6537e06740 1 --2- 172.21.15.18:0/4205639307 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891250 0x5616e28977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f65167fc640 0 client.18615 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:58:25.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f6537e06740 1 -- 172.21.15.18:0/4205639307 >> 172.21.15.18:0/4205639307 conn(0x5616e288b980 msgr2=0x5616e288d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f6537e06740 1 -- 172.21.15.18:0/4205639307 shutdown_connections 2024-02-17T12:58:25.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f6537e06740 1 -- 172.21.15.18:0/4205639307 wait complete. 2024-02-17T12:58:25.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.306+0000 7f6537e06740 20 client.18615 trim_cache size 0 max 16384 2024-02-17T12:58:25.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f6537e06740 1 Processor -- start 2024-02-17T12:58:25.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f6537e06740 1 -- start start 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f6537e06740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891250 0x5616e2899ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f6537e06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e2891b70 0x5616e2896080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f6537e06740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e289a750 0x5616e28965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f6537e06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e277e130 con 0x5616e2891b70 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f6537e06740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e2890eb0 con 0x5616e2891250 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f6537e06740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e28975a0 con 0x5616e289a750 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f652ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e2891b70 0x5616e2896080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f652ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e2891b70 0x5616e2896080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42664/0 (socket says 172.21.15.18:42664) 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f652ed9d640 1 -- 172.21.15.18:0/2753031576 learned_addr learned my addr 172.21.15.18:0/2753031576 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f6527fff640 1 --2- 172.21.15.18:0/2753031576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891250 0x5616e2899ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f652ed9d640 1 -- 172.21.15.18:0/2753031576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e289a750 msgr2=0x5616e28965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f652ed9d640 1 --2- 172.21.15.18:0/2753031576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e289a750 0x5616e28965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f652ed9d640 1 -- 172.21.15.18:0/2753031576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891250 msgr2=0x5616e2899ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f652ed9d640 1 --2- 172.21.15.18:0/2753031576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891250 0x5616e2899ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f652ed9d640 1 -- 172.21.15.18:0/2753031576 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e2793c20 con 0x5616e2891b70 2024-02-17T12:58:25.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.318+0000 7f6527fff640 1 --2- 172.21.15.18:0/2753031576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891250 0x5616e2899ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:25.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.319+0000 7f652ed9d640 1 --2- 172.21.15.18:0/2753031576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e2891b70 0x5616e2896080 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6520005e00 tx=0x7f6520005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676e41b951825cd0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.319+0000 7f6524ff9640 1 -- 172.21.15.18:0/2753031576 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520045d00 con 0x5616e2891b70 2024-02-17T12:58:25.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.319+0000 7f6524ff9640 1 -- 172.21.15.18:0/2753031576 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6520045e60 con 0x5616e2891b70 2024-02-17T12:58:25.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.319+0000 7f6524ff9640 1 -- 172.21.15.18:0/2753031576 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520034730 con 0x5616e2891b70 2024-02-17T12:58:25.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.319+0000 7f6537e06740 1 -- 172.21.15.18:0/2753031576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e2891b70 msgr2=0x5616e2896080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.319+0000 7f6537e06740 1 --2- 172.21.15.18:0/2753031576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e2891b70 0x5616e2896080 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6520005e00 tx=0x7f6520005f90 comp rx=0 tx=0).stop 2024-02-17T12:58:25.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 -- 172.21.15.18:0/2753031576 shutdown_connections 2024-02-17T12:58:25.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 --2- 172.21.15.18:0/2753031576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e289a750 0x5616e28965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 --2- 172.21.15.18:0/2753031576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e2891b70 0x5616e2896080 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 --2- 172.21.15.18:0/2753031576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891250 0x5616e2899ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 -- 172.21.15.18:0/2753031576 >> 172.21.15.18:0/2753031576 conn(0x5616e288b980 msgr2=0x5616e2892a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 -- 172.21.15.18:0/2753031576 shutdown_connections 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 -- 172.21.15.18:0/2753031576 wait complete. 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 -- start start 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 Processor -- start 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891250 0x5616e28a0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891b70 0x5616e28a0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e28a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e2900e40 con 0x5616e289a750 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e2900f80 con 0x5616e2891b70 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e29010c0 con 0x5616e2891250 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6537e06740 10 client.0 _ll_register_callbacks cb 0x7f6518011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:25.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6527fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e28a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6527fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e28a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42674/0 (socket says 172.21.15.18:42674) 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6527fff640 1 -- 172.21.15.18:0/1995691439 learned_addr learned my addr 172.21.15.18:0/1995691439 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f652ed9d640 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891b70 0x5616e28a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6527fff640 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891250 0x5616e28a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6527fff640 1 -- 172.21.15.18:0/1995691439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891250 msgr2=0x5616e28a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6527fff640 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891250 0x5616e28a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6527fff640 1 -- 172.21.15.18:0/1995691439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891b70 msgr2=0x5616e28a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6527fff640 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891b70 0x5616e28a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f6527fff640 1 -- 172.21.15.18:0/1995691439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653010e810 con 0x5616e289a750 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.320+0000 7f652ed9d640 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891b70 0x5616e28a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f6527fff640 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e28a0aa0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f6518011b20 tx=0x7f6518011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96891c5a2d26e79e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f65167fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f65167fc640 1 -- 172.21.15.18:0/1995691439 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518022070 con 0x5616e289a750 2024-02-17T12:58:25.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f65167fc640 1 -- 172.21.15.18:0/1995691439 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6518003cf0 con 0x5616e289a750 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f6537e06740 1 -- 172.21.15.18:0/1995691439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f651800fdb0 con 0x5616e289a750 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f65167fc640 1 -- 172.21.15.18:0/1995691439 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6518005640 con 0x5616e289a750 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f6537e06740 10 client.18621 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f6537e06740 1 -- 172.21.15.18:0/1995691439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5616e27b98c0 con 0x5616e289a750 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f65167fc640 1 -- 172.21.15.18:0/1995691439 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6518047020 con 0x5616e289a750 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f65167fc640 1 -- 172.21.15.18:0/1995691439 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f6518004660 con 0x5616e289a750 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f6537e06740 10 client.18621 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f6537e06740 10 client.18621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f6537e06740 20 client.18621 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f6537e06740 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5616e2902540 0x5616e2904960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f6537e06740 4 client.18621 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f6537e06740 1 -- 172.21.15.18:0/1995691439 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5616e2584890 con 0x5616e2902540 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.321+0000 7f652ed9d640 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5616e2902540 0x5616e2904960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f65167fc640 10 client.18621 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:58:25.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f652ed9d640 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5616e2902540 0x5616e2904960 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5bcf4e871f1d6e71 server_cookie=16b772cd70c6b0db in_seq=0 out_seq=0 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f65167fc640 1 -- 172.21.15.18:0/1995691439 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5616e2584890 con 0x5616e2902540 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f65167fc640 10 client.18621 handle_command_reply: tid=0 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f6537e06740 1 client.18621 shutdown 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f6537e06740 1 -- 172.21.15.18:0/1995691439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 msgr2=0x5616e28a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f6537e06740 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e28a0aa0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f6518011b20 tx=0x7f6518011ab0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f6537e06740 1 -- 172.21.15.18:0/1995691439 shutdown_connections 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f6537e06740 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5616e2902540 0x5616e2904960 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f6537e06740 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5616e289a750 0x5616e28a0aa0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f6537e06740 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e2891b70 0x5616e28a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f6537e06740 1 --2- 172.21.15.18:0/1995691439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616e2891250 0x5616e28a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f6537e06740 1 -- 172.21.15.18:0/1995691439 >> 172.21.15.18:0/1995691439 conn(0x5616e288b980 msgr2=0x5616e2895fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f65167fc640 0 client.18621 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f6537e06740 1 -- 172.21.15.18:0/1995691439 shutdown_connections 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f6537e06740 1 -- 172.21.15.18:0/1995691439 wait complete. 2024-02-17T12:58:25.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.322+0000 7f6537e06740 20 client.18621 trim_cache size 0 max 16384 2024-02-17T12:58:25.321 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:58:25.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.323+0000 7f6535156640 1 -- 172.21.15.18:0/2685027985 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f651c0a22b0 msgr2=0x7f651c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.323+0000 7f6535156640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f651c0a22b0 0x7f651c0a4770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f652800cfd0 tx=0x7f652800cf60 comp rx=0 tx=0).stop 2024-02-17T12:58:25.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.323+0000 7f6535156640 1 -- 172.21.15.18:0/2685027985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6530073f00 msgr2=0x7f65301a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.323+0000 7f6535156640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6530073f00 0x7f65301a5ac0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f652800ead0 tx=0x7f652800ef90 comp rx=0 tx=0).stop 2024-02-17T12:58:25.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.323+0000 7f6535156640 1 -- 172.21.15.18:0/2685027985 shutdown_connections 2024-02-17T12:58:25.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.323+0000 7f6535156640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f651c0a22b0 0x7f651c0a4770 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.323+0000 7f6535156640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6530073f00 0x7f65301a5ac0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.323+0000 7f6535156640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6530106430 0x7f65300739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.323+0000 7f6535156640 1 --2- 172.21.15.18:0/2685027985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6530101360 0x7f6530073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.323+0000 7f6535156640 1 -- 172.21.15.18:0/2685027985 >> 172.21.15.18:0/2685027985 conn(0x7f65300fcbe0 msgr2=0x7f65300fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.323+0000 7f6535156640 1 -- 172.21.15.18:0/2685027985 shutdown_connections 2024-02-17T12:58:25.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.323+0000 7f6535156640 1 -- 172.21.15.18:0/2685027985 wait complete. 2024-02-17T12:58:25.324 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:58:25.325 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:58:25.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:24 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/963584111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:58:25.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.439+0000 7f6299d6c640 1 -- 172.21.15.18:0/1057918960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6294069aa0 msgr2=0x7f6294069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.439+0000 7f6299d6c640 1 --2- 172.21.15.18:0/1057918960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6294069aa0 0x7f6294069e80 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f6284009930 tx=0x7f628402bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:25.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.439+0000 7f6299d6c640 1 -- 172.21.15.18:0/1057918960 shutdown_connections 2024-02-17T12:58:25.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.439+0000 7f6299d6c640 1 --2- 172.21.15.18:0/1057918960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62941064e0 0x7f629410e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.439+0000 7f6299d6c640 1 --2- 172.21.15.18:0/1057918960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f629406a450 0x7f6294105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.439+0000 7f6299d6c640 1 --2- 172.21.15.18:0/1057918960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6294069aa0 0x7f6294069e80 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.439+0000 7f6299d6c640 1 -- 172.21.15.18:0/1057918960 >> 172.21.15.18:0/1057918960 conn(0x7f6294077150 msgr2=0x7f6294077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.439+0000 7f6299d6c640 1 -- 172.21.15.18:0/1057918960 shutdown_connections 2024-02-17T12:58:25.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.439+0000 7f6299d6c640 1 -- 172.21.15.18:0/1057918960 wait complete. 2024-02-17T12:58:25.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.439+0000 7f6299d6c640 1 Processor -- start 2024-02-17T12:58:25.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.439+0000 7f6299d6c640 1 -- start start 2024-02-17T12:58:25.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6299d6c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6294069aa0 0x7f629419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6299d6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62941064e0 0x7f629419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6299d6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62941975a0 0x7f6294197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6299d6c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6294197f90 con 0x7f62941064e0 2024-02-17T12:58:25.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6299d6c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6294198100 con 0x7f62941975a0 2024-02-17T12:58:25.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6299d6c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62941982c0 con 0x7f6294069aa0 2024-02-17T12:58:25.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6292ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62941064e0 0x7f629419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6292ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62941064e0 0x7f629419d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42686/0 (socket says 172.21.15.18:42686) 2024-02-17T12:58:25.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6292ffd640 1 -- 172.21.15.18:0/1618255764 learned_addr learned my addr 172.21.15.18:0/1618255764 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f62937fe640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6294069aa0 0x7f629419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6293fff640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62941975a0 0x7f6294197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6292ffd640 1 -- 172.21.15.18:0/1618255764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6294069aa0 msgr2=0x7f629419d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6292ffd640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6294069aa0 0x7f629419d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6292ffd640 1 -- 172.21.15.18:0/1618255764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62941975a0 msgr2=0x7f6294197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6292ffd640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62941975a0 0x7f6294197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6292ffd640 1 -- 172.21.15.18:0/1618255764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284009560 con 0x7f62941064e0 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6293fff640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62941975a0 0x7f6294197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6292ffd640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62941064e0 0x7f629419d930 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f628800e9c0 tx=0x7f628800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6290ff9640 1 -- 172.21.15.18:0/1618255764 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62880096c0 con 0x7f62941064e0 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6299d6c640 1 -- 172.21.15.18:0/1618255764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62941a42b0 con 0x7f62941064e0 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6290ff9640 1 -- 172.21.15.18:0/1618255764 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6288005500 con 0x7f62941064e0 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6290ff9640 1 -- 172.21.15.18:0/1618255764 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288010430 con 0x7f62941064e0 2024-02-17T12:58:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.440+0000 7f6299d6c640 1 -- 172.21.15.18:0/1618255764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62941a4770 con 0x7f62941064e0 2024-02-17T12:58:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.442+0000 7f6290ff9640 1 -- 172.21.15.18:0/1618255764 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6288005020 con 0x7f62941064e0 2024-02-17T12:58:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f629ca1b740 1 Processor -- start 2024-02-17T12:58:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f629ca1b740 1 -- start start 2024-02-17T12:58:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f629ca1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea35f00 0x562e4ea362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f6290ff9640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f627c0a2230 0x7f627c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f629ca1b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea36820 0x562e4ea43620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f629ca1b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea43b60 0x562e4ea45f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f629ca1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4e9f1960 con 0x562e4ea35f00 2024-02-17T12:58:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f629ca1b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4e9a5310 con 0x562e4ea36820 2024-02-17T12:58:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f629ca1b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4e9a2840 con 0x562e4ea43b60 2024-02-17T12:58:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f6293fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea35f00 0x562e4ea362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f6293fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea35f00 0x562e4ea362e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42694/0 (socket says 172.21.15.18:42694) 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f6293fff640 1 -- 172.21.15.18:0/1915884160 learned_addr learned my addr 172.21.15.18:0/1915884160 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f62937fe640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f627c0a2230 0x7f627c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f6290ff9640 1 -- 172.21.15.18:0/1618255764 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6288014070 con 0x7f62941064e0 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f6293fff640 1 --2- 172.21.15.18:0/1915884160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea43b60 0x562e4ea45f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f6292ffd640 1 --2- 172.21.15.18:0/1915884160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea36820 0x562e4ea43620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f6293fff640 1 -- 172.21.15.18:0/1915884160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea43b60 msgr2=0x562e4ea45f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f6293fff640 1 --2- 172.21.15.18:0/1915884160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea43b60 0x562e4ea45f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f6293fff640 1 -- 172.21.15.18:0/1915884160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea36820 msgr2=0x562e4ea43620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f6293fff640 1 --2- 172.21.15.18:0/1915884160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea36820 0x562e4ea43620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.443+0000 7f6293fff640 1 -- 172.21.15.18:0/1915884160 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e4e960060 con 0x562e4ea35f00 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.444+0000 7f6293fff640 1 --2- 172.21.15.18:0/1915884160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea35f00 0x562e4ea362e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f628c007fd0 tx=0x7f628c0078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea39a411b3a223f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.444+0000 7f62937fe640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f627c0a2230 0x7f627c0a46f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6284005ab0 tx=0x7f6284005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.444+0000 7f6279ffb640 1 -- 172.21.15.18:0/1915884160 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c004b20 con 0x562e4ea35f00 2024-02-17T12:58:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.444+0000 7f6279ffb640 1 -- 172.21.15.18:0/1915884160 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f628c010dd0 con 0x562e4ea35f00 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.444+0000 7f6279ffb640 1 -- 172.21.15.18:0/1915884160 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c0055f0 con 0x562e4ea35f00 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.444+0000 7f629ca1b740 1 -- 172.21.15.18:0/1915884160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea35f00 msgr2=0x562e4ea362e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.444+0000 7f629ca1b740 1 --2- 172.21.15.18:0/1915884160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea35f00 0x562e4ea362e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f628c007fd0 tx=0x7f628c0078f0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.444+0000 7f629ca1b740 1 -- 172.21.15.18:0/1915884160 shutdown_connections 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.444+0000 7f629ca1b740 1 --2- 172.21.15.18:0/1915884160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea43b60 0x562e4ea45f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.444+0000 7f629ca1b740 1 --2- 172.21.15.18:0/1915884160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea36820 0x562e4ea43620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.444+0000 7f629ca1b740 1 --2- 172.21.15.18:0/1915884160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea35f00 0x562e4ea362e0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.444+0000 7f629ca1b740 1 -- 172.21.15.18:0/1915884160 >> 172.21.15.18:0/1915884160 conn(0x562e4ea30980 msgr2=0x562e4ea30d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f629ca1b740 1 -- 172.21.15.18:0/1915884160 shutdown_connections 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f629ca1b740 1 -- 172.21.15.18:0/1915884160 wait complete. 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f629ca1b740 1 -- start start 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f629ca1b740 1 Processor -- start 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f629ca1b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea35f00 0x562e4ea40af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f629ca1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 0x562e4ea41030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f629ca1b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea43b60 0x562e4ea41570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f629ca1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4e99e020 con 0x562e4ea36820 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f629ca1b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4e671430 con 0x562e4ea35f00 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f629ca1b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4e62de00 con 0x562e4ea43b60 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f629ca1b740 10 client.0 _ll_register_callbacks cb 0x562e4e945cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f6292ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 0x562e4ea41030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f6292ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 0x562e4ea41030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42698/0 (socket says 172.21.15.18:42698) 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.445+0000 7f6292ffd640 1 -- 172.21.15.18:0/1537433656 learned_addr learned my addr 172.21.15.18:0/1537433656 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6293fff640 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea43b60 0x562e4ea41570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f62937fe640 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea35f00 0x562e4ea40af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6292ffd640 1 -- 172.21.15.18:0/1537433656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea43b60 msgr2=0x562e4ea41570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6292ffd640 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea43b60 0x562e4ea41570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6292ffd640 1 -- 172.21.15.18:0/1537433656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea35f00 msgr2=0x562e4ea40af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6292ffd640 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea35f00 0x562e4ea40af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6292ffd640 1 -- 172.21.15.18:0/1537433656 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c012470 con 0x562e4ea36820 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f62937fe640 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea35f00 0x562e4ea40af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6292ffd640 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 0x562e4ea41030 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f628809ab70 tx=0x7f628809a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22342ce2160d45b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6272ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6272ffd640 1 -- 172.21.15.18:0/1537433656 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628809ae70 con 0x562e4ea36820 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6272ffd640 1 -- 172.21.15.18:0/1537433656 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f628809b800 con 0x562e4ea36820 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f629ca1b740 1 -- 172.21.15.18:0/1537433656 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e4e69d7e0 con 0x562e4ea36820 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6272ffd640 1 -- 172.21.15.18:0/1537433656 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f628809c1a0 con 0x562e4ea36820 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f629ca1b740 10 client.18633 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f629ca1b740 1 -- 172.21.15.18:0/1537433656 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x562e4e99c6a0 con 0x562e4ea36820 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6272ffd640 1 -- 172.21.15.18:0/1537433656 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562e4e69d7e0 con 0x562e4ea36820 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.446+0000 7f6272ffd640 1 -- 172.21.15.18:0/1537433656 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f628801f060 con 0x562e4ea36820 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.447+0000 7f629ca1b740 10 client.18633 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.447+0000 7f629ca1b740 10 client.18633 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.447+0000 7f629ca1b740 20 client.18633 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:25.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.447+0000 7f629ca1b740 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x562e4ea53ee0 0x562e4ea56300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.447+0000 7f629ca1b740 4 client.18633 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.447+0000 7f629ca1b740 1 -- 172.21.15.18:0/1537433656 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e4e6da720 con 0x562e4ea53ee0 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.447+0000 7f62937fe640 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x562e4ea53ee0 0x562e4ea56300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.447+0000 7f62937fe640 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x562e4ea53ee0 0x562e4ea56300 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=401ffa40e64ea084 server_cookie=b855810009bd1445 in_seq=0 out_seq=0 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.447+0000 7f6272ffd640 10 client.18633 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f6272ffd640 1 -- 172.21.15.18:0/1537433656 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e4e6da720 con 0x562e4ea53ee0 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f6272ffd640 10 client.18633 handle_command_reply: tid=0 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f629ca1b740 1 client.18633 shutdown 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f629ca1b740 1 -- 172.21.15.18:0/1537433656 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 msgr2=0x562e4ea41030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f629ca1b740 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 0x562e4ea41030 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f628809ab70 tx=0x7f628809a890 comp rx=0 tx=0).stop 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f629ca1b740 1 -- 172.21.15.18:0/1537433656 shutdown_connections 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f629ca1b740 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x562e4ea53ee0 0x562e4ea56300 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f629ca1b740 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea43b60 0x562e4ea41570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f629ca1b740 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 0x562e4ea41030 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f6272ffd640 0 client.18633 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:58:25.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f629ca1b740 1 --2- 172.21.15.18:0/1537433656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea35f00 0x562e4ea40af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f629ca1b740 1 -- 172.21.15.18:0/1537433656 >> 172.21.15.18:0/1537433656 conn(0x562e4ea30980 msgr2=0x562e4ea326d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f629ca1b740 1 -- 172.21.15.18:0/1537433656 shutdown_connections 2024-02-17T12:58:25.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f629ca1b740 1 -- 172.21.15.18:0/1537433656 wait complete. 2024-02-17T12:58:25.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.448+0000 7f629ca1b740 20 client.18633 trim_cache size 0 max 16384 2024-02-17T12:58:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.459+0000 7f629ca1b740 1 Processor -- start 2024-02-17T12:58:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f629ca1b740 1 -- start start 2024-02-17T12:58:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f629ca1b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea35f00 0x562e4ea43220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f629ca1b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea36820 0x562e4ea41390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f629ca1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea43b60 0x562e4ea5aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f629ca1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4e923130 con 0x562e4ea43b60 2024-02-17T12:58:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f629ca1b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4ea3c6f0 con 0x562e4ea35f00 2024-02-17T12:58:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f629ca1b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4ea35340 con 0x562e4ea36820 2024-02-17T12:58:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f6293fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea43b60 0x562e4ea5aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f6293fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea43b60 0x562e4ea5aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42700/0 (socket says 172.21.15.18:42700) 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f6293fff640 1 -- 172.21.15.18:0/2662369821 learned_addr learned my addr 172.21.15.18:0/2662369821 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f6292ffd640 1 --2- 172.21.15.18:0/2662369821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea36820 0x562e4ea41390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f62937fe640 1 --2- 172.21.15.18:0/2662369821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea35f00 0x562e4ea43220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f6293fff640 1 -- 172.21.15.18:0/2662369821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea36820 msgr2=0x562e4ea41390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f6293fff640 1 --2- 172.21.15.18:0/2662369821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea36820 0x562e4ea41390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f6293fff640 1 -- 172.21.15.18:0/2662369821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea35f00 msgr2=0x562e4ea43220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f6293fff640 1 --2- 172.21.15.18:0/2662369821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea35f00 0x562e4ea43220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f6293fff640 1 -- 172.21.15.18:0/2662369821 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e4e938c20 con 0x562e4ea43b60 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f6293fff640 1 --2- 172.21.15.18:0/2662369821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea43b60 0x562e4ea5aca0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f628c010740 tx=0x7f628c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e296eee23be26de3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f62797fa640 1 -- 172.21.15.18:0/2662369821 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c007520 con 0x562e4ea43b60 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f62797fa640 1 -- 172.21.15.18:0/2662369821 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f628c023040 con 0x562e4ea43b60 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.460+0000 7f62797fa640 1 -- 172.21.15.18:0/2662369821 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c0054a0 con 0x562e4ea43b60 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.461+0000 7f629ca1b740 1 -- 172.21.15.18:0/2662369821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea43b60 msgr2=0x562e4ea5aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.461+0000 7f629ca1b740 1 --2- 172.21.15.18:0/2662369821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea43b60 0x562e4ea5aca0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f628c010740 tx=0x7f628c010f40 comp rx=0 tx=0).stop 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.461+0000 7f629ca1b740 1 -- 172.21.15.18:0/2662369821 shutdown_connections 2024-02-17T12:58:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.461+0000 7f629ca1b740 1 --2- 172.21.15.18:0/2662369821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea43b60 0x562e4ea5aca0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.461+0000 7f629ca1b740 1 --2- 172.21.15.18:0/2662369821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea36820 0x562e4ea41390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.461+0000 7f629ca1b740 1 --2- 172.21.15.18:0/2662369821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea35f00 0x562e4ea43220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.461+0000 7f629ca1b740 1 -- 172.21.15.18:0/2662369821 >> 172.21.15.18:0/2662369821 conn(0x562e4ea30980 msgr2=0x562e4ea2ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.461+0000 7f629ca1b740 1 -- 172.21.15.18:0/2662369821 shutdown_connections 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.461+0000 7f629ca1b740 1 -- 172.21.15.18:0/2662369821 wait complete. 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.461+0000 7f629ca1b740 1 -- start start 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f629ca1b740 1 Processor -- start 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f629ca1b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea35f00 0x562e4ea49c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f629ca1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 0x562e4ea4a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f629ca1b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea43b60 0x562e4eaaa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f629ca1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4eaaa970 con 0x562e4ea36820 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f629ca1b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4eaaaab0 con 0x562e4ea43b60 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f629ca1b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4eaaabf0 con 0x562e4ea35f00 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f629ca1b740 10 client.0 _ll_register_callbacks cb 0x7f628c007750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f6292ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 0x562e4ea4a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f62937fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea35f00 0x562e4ea49c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f6292ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 0x562e4ea4a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42706/0 (socket says 172.21.15.18:42706) 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f6292ffd640 1 -- 172.21.15.18:0/4091814838 learned_addr learned my addr 172.21.15.18:0/4091814838 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f6293fff640 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea43b60 0x562e4eaaa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f6292ffd640 1 -- 172.21.15.18:0/4091814838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea35f00 msgr2=0x562e4ea49c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f6292ffd640 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea35f00 0x562e4ea49c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f6292ffd640 1 -- 172.21.15.18:0/4091814838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea43b60 msgr2=0x562e4eaaa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f6292ffd640 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea43b60 0x562e4eaaa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f6292ffd640 1 -- 172.21.15.18:0/4091814838 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e4e960060 con 0x562e4ea36820 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f62937fe640 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea35f00 0x562e4ea49c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f6292ffd640 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 0x562e4ea4a1d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f628800c6a0 tx=0x7f6288009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254171b4421aa38b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f6272ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.462+0000 7f6272ffd640 1 -- 172.21.15.18:0/4091814838 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62880a3380 con 0x562e4ea36820 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f6272ffd640 1 -- 172.21.15.18:0/4091814838 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f62880a3610 con 0x562e4ea36820 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f629ca1b740 1 -- 172.21.15.18:0/4091814838 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f628c010dd0 con 0x562e4ea36820 2024-02-17T12:58:25.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f6272ffd640 1 -- 172.21.15.18:0/4091814838 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f628809a9b0 con 0x562e4ea36820 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f629ca1b740 10 client.18639 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f629ca1b740 1 -- 172.21.15.18:0/4091814838 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x562e4e95e8c0 con 0x562e4ea36820 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f6272ffd640 1 -- 172.21.15.18:0/4091814838 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f628c010dd0 con 0x562e4ea36820 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f6272ffd640 1 -- 172.21.15.18:0/4091814838 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f628801f040 con 0x562e4ea36820 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f629ca1b740 10 client.18639 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f629ca1b740 10 client.18639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f629ca1b740 20 client.18639 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f629ca1b740 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x562e4eaabee0 0x562e4eaae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f629ca1b740 4 client.18639 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f629ca1b740 1 -- 172.21.15.18:0/4091814838 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562e4e729890 con 0x562e4eaabee0 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.463+0000 7f62937fe640 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x562e4eaabee0 0x562e4eaae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f62937fe640 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x562e4eaabee0 0x562e4eaae2d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85b192c883735c23 server_cookie=784db04e87aee5d6 in_seq=0 out_seq=0 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f6272ffd640 10 client.18639 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f6272ffd640 1 -- 172.21.15.18:0/4091814838 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562e4e729890 con 0x562e4eaabee0 2024-02-17T12:58:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f6272ffd640 10 client.18639 handle_command_reply: tid=0 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f629ca1b740 1 client.18639 shutdown 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f629ca1b740 1 -- 172.21.15.18:0/4091814838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 msgr2=0x562e4ea4a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f629ca1b740 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 0x562e4ea4a1d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f628800c6a0 tx=0x7f6288009310 comp rx=0 tx=0).stop 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f629ca1b740 1 -- 172.21.15.18:0/4091814838 shutdown_connections 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f629ca1b740 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x562e4eaabee0 0x562e4eaae2d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f629ca1b740 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e4ea43b60 0x562e4eaaa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f629ca1b740 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e4ea36820 0x562e4ea4a1d0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f6272ffd640 0 client.18639 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f629ca1b740 1 --2- 172.21.15.18:0/4091814838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e4ea35f00 0x562e4ea49c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f629ca1b740 1 -- 172.21.15.18:0/4091814838 >> 172.21.15.18:0/4091814838 conn(0x562e4ea30980 msgr2=0x562e4ea5aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f629ca1b740 1 -- 172.21.15.18:0/4091814838 shutdown_connections 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f629ca1b740 1 -- 172.21.15.18:0/4091814838 wait complete. 2024-02-17T12:58:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.464+0000 7f629ca1b740 20 client.18639 trim_cache size 0 max 16384 2024-02-17T12:58:25.462 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:58:25.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.465+0000 7f6299d6c640 1 -- 172.21.15.18:0/1618255764 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f627c0a2230 msgr2=0x7f627c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.465+0000 7f6299d6c640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f627c0a2230 0x7f627c0a46f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6284005ab0 tx=0x7f6284005fd0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.465+0000 7f6299d6c640 1 -- 172.21.15.18:0/1618255764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62941064e0 msgr2=0x7f629419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.465+0000 7f6299d6c640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62941064e0 0x7f629419d930 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f628800e9c0 tx=0x7f628800ee80 comp rx=0 tx=0).stop 2024-02-17T12:58:25.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.465+0000 7f6299d6c640 1 -- 172.21.15.18:0/1618255764 shutdown_connections 2024-02-17T12:58:25.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.465+0000 7f6299d6c640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f627c0a2230 0x7f627c0a46f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.465+0000 7f6299d6c640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62941975a0 0x7f6294197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.465+0000 7f6299d6c640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62941064e0 0x7f629419d930 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.465+0000 7f6299d6c640 1 --2- 172.21.15.18:0/1618255764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6294069aa0 0x7f629419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.465+0000 7f6299d6c640 1 -- 172.21.15.18:0/1618255764 >> 172.21.15.18:0/1618255764 conn(0x7f6294077150 msgr2=0x7f62940fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.465+0000 7f6299d6c640 1 -- 172.21.15.18:0/1618255764 shutdown_connections 2024-02-17T12:58:25.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.465+0000 7f6299d6c640 1 -- 172.21.15.18:0/1618255764 wait complete. 2024-02-17T12:58:25.466 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:58:25.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:58:25.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.580+0000 7f1eb0040640 1 -- 172.21.15.18:0/933398367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ea8101d40 msgr2=0x7f1ea8105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.580+0000 7f1eb0040640 1 --2- 172.21.15.18:0/933398367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ea8101d40 0x7f1ea8105d80 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009930 tx=0x7f1e9802bc20 comp rx=0 tx=0).stop 2024-02-17T12:58:25.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.580+0000 7f1eb0040640 1 -- 172.21.15.18:0/933398367 shutdown_connections 2024-02-17T12:58:25.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.580+0000 7f1eb0040640 1 --2- 172.21.15.18:0/933398367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8106460 0x7f1ea810e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.580+0000 7f1eb0040640 1 --2- 172.21.15.18:0/933398367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ea8101d40 0x7f1ea8105d80 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.580+0000 7f1eb0040640 1 --2- 172.21.15.18:0/933398367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8101390 0x7f1ea8101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.580+0000 7f1eb0040640 1 -- 172.21.15.18:0/933398367 >> 172.21.15.18:0/933398367 conn(0x7f1ea80fcbb0 msgr2=0x7f1ea80fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.580+0000 7f1eb0040640 1 -- 172.21.15.18:0/933398367 shutdown_connections 2024-02-17T12:58:25.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.580+0000 7f1eb0040640 1 -- 172.21.15.18:0/933398367 wait complete. 2024-02-17T12:58:25.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eb0040640 1 Processor -- start 2024-02-17T12:58:25.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eb0040640 1 -- start start 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eb0040640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8101390 0x7f1ea8197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eb0040640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8106460 0x7f1ea8197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eb0040640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ea819bfd0 0x7f1ea819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eb0040640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea819c9c0 con 0x7f1ea819bfd0 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eb0040640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea819cb30 con 0x7f1ea8101390 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eb0040640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea819cca0 con 0x7f1ea8106460 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eae5dd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ea819bfd0 0x7f1ea819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eae5dd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ea819bfd0 0x7f1ea819c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42730/0 (socket says 172.21.15.18:42730) 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eae5dd640 1 -- 172.21.15.18:0/547347114 learned_addr learned my addr 172.21.15.18:0/547347114 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1ead5db640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8106460 0x7f1ea8197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eadddc640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8101390 0x7f1ea8197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eae5dd640 1 -- 172.21.15.18:0/547347114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8106460 msgr2=0x7f1ea8197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eae5dd640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8106460 0x7f1ea8197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eae5dd640 1 -- 172.21.15.18:0/547347114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8101390 msgr2=0x7f1ea8197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eae5dd640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8101390 0x7f1ea8197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.581+0000 7f1eae5dd640 1 -- 172.21.15.18:0/547347114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e98009560 con 0x7f1ea819bfd0 2024-02-17T12:58:25.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.582+0000 7f1eae5dd640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ea819bfd0 0x7f1ea819c480 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000cce0 tx=0x7f1ea000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.582+0000 7f1e9effd640 1 -- 172.21.15.18:0/547347114 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000f450 con 0x7f1ea819bfd0 2024-02-17T12:58:25.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.582+0000 7f1eb0040640 1 -- 172.21.15.18:0/547347114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea81a4280 con 0x7f1ea819bfd0 2024-02-17T12:58:25.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.582+0000 7f1e9effd640 1 -- 172.21.15.18:0/547347114 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ea000fa90 con 0x7f1ea819bfd0 2024-02-17T12:58:25.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.582+0000 7f1e9effd640 1 -- 172.21.15.18:0/547347114 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0018b90 con 0x7f1ea819bfd0 2024-02-17T12:58:25.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.582+0000 7f1eb0040640 1 -- 172.21.15.18:0/547347114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea81a46b0 con 0x7f1ea819bfd0 2024-02-17T12:58:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1e9effd640 1 -- 172.21.15.18:0/547347114 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1ea000fc00 con 0x7f1ea819bfd0 2024-02-17T12:58:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1eb2c0f740 1 Processor -- start 2024-02-17T12:58:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1eb2c0f740 1 -- start start 2024-02-17T12:58:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1eb2c0f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a595a250 0x5641a595a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1eb2c0f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595ab70 0x5641a5963210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1eb2c0f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 0x5641a5965b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1eb2c0f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a5915960 con 0x5641a5963750 2024-02-17T12:58:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1eb2c0f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a58c9310 con 0x5641a595ab70 2024-02-17T12:58:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1eb2c0f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a58c6840 con 0x5641a595a250 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1e9effd640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1e940a2230 0x7f1e940a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1ead5db640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 0x5641a5965b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1ead5db640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 0x5641a5965b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42744/0 (socket says 172.21.15.18:42744) 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1ead5db640 1 -- 172.21.15.18:0/3502536172 learned_addr learned my addr 172.21.15.18:0/3502536172 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1eadddc640 1 --2- 172.21.15.18:0/3502536172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595ab70 0x5641a5963210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1eae5dd640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1e940a2230 0x7f1e940a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.585+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3502536172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a595a250 0x5641a595a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.586+0000 7f1e9effd640 1 -- 172.21.15.18:0/547347114 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1ea00ce7f0 con 0x7f1ea819bfd0 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.586+0000 7f1ead5db640 1 -- 172.21.15.18:0/3502536172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a595a250 msgr2=0x5641a595a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.586+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3502536172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a595a250 0x5641a595a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.586+0000 7f1ead5db640 1 -- 172.21.15.18:0/3502536172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595ab70 msgr2=0x5641a5963210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.586+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3502536172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595ab70 0x5641a5963210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.586+0000 7f1ead5db640 1 -- 172.21.15.18:0/3502536172 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641a5884060 con 0x5641a5963750 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.586+0000 7f1eadddc640 1 --2- 172.21.15.18:0/3502536172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595ab70 0x5641a5963210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.586+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3502536172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 0x5641a5965b40 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009930 tx=0x7f1e98004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfe70048ae2836b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.586+0000 7f1eae5dd640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1e940a2230 0x7f1e940a46f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ea009b190 tx=0x7f1ea00908d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.586+0000 7f1e93fff640 1 -- 172.21.15.18:0/3502536172 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9802c8d0 con 0x5641a5963750 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.586+0000 7f1e93fff640 1 -- 172.21.15.18:0/3502536172 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e98004510 con 0x5641a5963750 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.586+0000 7f1e93fff640 1 -- 172.21.15.18:0/3502536172 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98005540 con 0x5641a5963750 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.587+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3502536172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 msgr2=0x5641a5965b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.587+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3502536172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 0x5641a5965b40 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009930 tx=0x7f1e98004740 comp rx=0 tx=0).stop 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.587+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3502536172 shutdown_connections 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.587+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3502536172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 0x5641a5965b40 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.587+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3502536172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595ab70 0x5641a5963210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.587+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3502536172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a595a250 0x5641a595a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.587+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3502536172 >> 172.21.15.18:0/3502536172 conn(0x5641a5954980 msgr2=0x5641a5954d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.587+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3502536172 shutdown_connections 2024-02-17T12:58:25.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.587+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3502536172 wait complete. 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.587+0000 7f1eb2c0f740 1 -- start start 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.587+0000 7f1eb2c0f740 1 Processor -- start 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eb2c0f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a59607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eb2c0f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 0x5641a5960d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eb2c0f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a5963750 0x5641a5961250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eb2c0f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a58c2020 con 0x5641a595ab70 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eb2c0f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a5595430 con 0x5641a595a250 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eb2c0f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a5551e00 con 0x5641a5963750 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eb2c0f740 10 client.0 _ll_register_callbacks cb 0x5641a5869cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eadddc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 0x5641a5960d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eadddc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 0x5641a5960d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42750/0 (socket says 172.21.15.18:42750) 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eadddc640 1 -- 172.21.15.18:0/3696671979 learned_addr learned my addr 172.21.15.18:0/3696671979 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a59607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a5963750 0x5641a5961250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eadddc640 1 -- 172.21.15.18:0/3696671979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a5963750 msgr2=0x5641a5961250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eadddc640 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a5963750 0x5641a5961250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eadddc640 1 -- 172.21.15.18:0/3696671979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 msgr2=0x5641a59607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eadddc640 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a59607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eadddc640 1 -- 172.21.15.18:0/3696671979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea810e840 con 0x5641a595ab70 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a5963750 0x5641a5961250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eadddc640 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 0x5641a5960d10 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400ce80 tx=0x7f1ea40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9821349ef788fa8d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1e917fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1e917fa640 1 -- 172.21.15.18:0/3696671979 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4011a50 con 0x5641a595ab70 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3696671979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641a55c17e0 con 0x5641a595ab70 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1e917fa640 1 -- 172.21.15.18:0/3696671979 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1ea4011bb0 con 0x5641a595ab70 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1e917fa640 1 -- 172.21.15.18:0/3696671979 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ea4011d70 con 0x5641a595ab70 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eb2c0f740 10 client.18651 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:25.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.588+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3696671979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5641a58c06a0 con 0x5641a595ab70 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.589+0000 7f1e917fa640 1 -- 172.21.15.18:0/3696671979 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5641a55c17e0 con 0x5641a595ab70 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.589+0000 7f1e917fa640 1 -- 172.21.15.18:0/3696671979 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1ea4019500 con 0x5641a595ab70 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.589+0000 7f1eb2c0f740 10 client.18651 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.589+0000 7f1eb2c0f740 10 client.18651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.589+0000 7f1eb2c0f740 20 client.18651 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.589+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5641a5973bf0 0x5641a5976010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.589+0000 7f1eb2c0f740 4 client.18651 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.589+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3696671979 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641a55fe720 con 0x5641a5973bf0 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.589+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5641a5973bf0 0x5641a5976010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1e917fa640 10 client.18651 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5641a5973bf0 0x5641a5976010 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23821e8f5940aa0e server_cookie=8d7437d9dc2f0fa8 in_seq=0 out_seq=0 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1e917fa640 1 -- 172.21.15.18:0/3696671979 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5641a55fe720 con 0x5641a5973bf0 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1e917fa640 10 client.18651 handle_command_reply: tid=0 2024-02-17T12:58:25.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1eb2c0f740 1 client.18651 shutdown 2024-02-17T12:58:25.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3696671979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 msgr2=0x5641a5960d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 0x5641a5960d10 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400ce80 tx=0x7f1ea40039c0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3696671979 shutdown_connections 2024-02-17T12:58:25.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5641a5973bf0 0x5641a5976010 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a5963750 0x5641a5961250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 0x5641a5960d10 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1e917fa640 0 client.18651 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:25.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3696671979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a59607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.590+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3696671979 >> 172.21.15.18:0/3696671979 conn(0x5641a5954980 msgr2=0x5641a59566d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.591+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3696671979 shutdown_connections 2024-02-17T12:58:25.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.591+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3696671979 wait complete. 2024-02-17T12:58:25.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.591+0000 7f1eb2c0f740 20 client.18651 trim_cache size 0 max 16384 2024-02-17T12:58:25.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.602+0000 7f1eb2c0f740 1 Processor -- start 2024-02-17T12:58:25.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.602+0000 7f1eb2c0f740 1 -- start start 2024-02-17T12:58:25.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.602+0000 7f1eb2c0f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a5962f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.602+0000 7f1eb2c0f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a595ab70 0x5641a5961070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.602+0000 7f1eb2c0f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 0x5641a595ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.602+0000 7f1eb2c0f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a5847130 con 0x5641a5963750 2024-02-17T12:58:25.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.602+0000 7f1eb2c0f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a597b260 con 0x5641a595a250 2024-02-17T12:58:25.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.602+0000 7f1eb2c0f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a595b5c0 con 0x5641a595ab70 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1ead5db640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 0x5641a595ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1ead5db640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 0x5641a595ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42756/0 (socket says 172.21.15.18:42756) 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1ead5db640 1 -- 172.21.15.18:0/3070874915 learned_addr learned my addr 172.21.15.18:0/3070874915 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3070874915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a5962f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1eadddc640 1 --2- 172.21.15.18:0/3070874915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a595ab70 0x5641a5961070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1ead5db640 1 -- 172.21.15.18:0/3070874915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a595ab70 msgr2=0x5641a5961070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3070874915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a595ab70 0x5641a5961070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1ead5db640 1 -- 172.21.15.18:0/3070874915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 msgr2=0x5641a5962f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3070874915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a5962f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1ead5db640 1 -- 172.21.15.18:0/3070874915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641a585cc20 con 0x5641a5963750 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1ead5db640 1 --2- 172.21.15.18:0/3070874915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 0x5641a595ba90 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f1e9802c160 tx=0x7f1e98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292069bd797b36c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1e937fe640 1 -- 172.21.15.18:0/3070874915 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9802cb50 con 0x5641a5963750 2024-02-17T12:58:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1e937fe640 1 -- 172.21.15.18:0/3070874915 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e9802ccb0 con 0x5641a5963750 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.603+0000 7f1e937fe640 1 -- 172.21.15.18:0/3070874915 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98042ce0 con 0x5641a5963750 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.604+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3070874915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 msgr2=0x5641a595ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.604+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3070874915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 0x5641a595ba90 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f1e9802c160 tx=0x7f1e98004ad0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.604+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3070874915 shutdown_connections 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.604+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3070874915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5963750 0x5641a595ba90 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.604+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3070874915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a595ab70 0x5641a5961070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.604+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/3070874915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a5962f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.604+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3070874915 >> 172.21.15.18:0/3070874915 conn(0x5641a5954980 msgr2=0x5641a594fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.604+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3070874915 shutdown_connections 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.604+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/3070874915 wait complete. 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.604+0000 7f1eb2c0f740 1 -- start start 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.604+0000 7f1eb2c0f740 1 Processor -- start 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eb2c0f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a59691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eb2c0f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 0x5641a5969710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eb2c0f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a5963750 0x5641a5969c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eb2c0f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a59ca2b0 con 0x5641a595ab70 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eb2c0f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a59ca420 con 0x5641a595a250 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eb2c0f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a59ca5e0 con 0x5641a5963750 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eb2c0f740 10 client.0 _ll_register_callbacks cb 0x7f1e9802c620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eadddc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 0x5641a5969710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eadddc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 0x5641a5969710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42768/0 (socket says 172.21.15.18:42768) 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eadddc640 1 -- 172.21.15.18:0/504960267 learned_addr learned my addr 172.21.15.18:0/504960267 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1ead5db640 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a59691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1ead5db640 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a5963750 0x5641a5969c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eadddc640 1 -- 172.21.15.18:0/504960267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a5963750 msgr2=0x5641a5969c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eadddc640 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a5963750 0x5641a5969c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eadddc640 1 -- 172.21.15.18:0/504960267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 msgr2=0x5641a59691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:25.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eadddc640 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a59691d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eadddc640 1 -- 172.21.15.18:0/504960267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641a5884060 con 0x5641a595ab70 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1ead5db640 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a59691d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eadddc640 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 0x5641a5969710 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006010 tx=0x7f1ea4019cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c64db0e5157728 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1e917fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1e917fa640 1 -- 172.21.15.18:0/504960267 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4011a50 con 0x5641a595ab70 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1e917fa640 1 -- 172.21.15.18:0/504960267 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1ea4000e40 con 0x5641a595ab70 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/504960267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e98004510 con 0x5641a595ab70 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1e917fa640 1 -- 172.21.15.18:0/504960267 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ea4019ec0 con 0x5641a595ab70 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eb2c0f740 10 client.18657 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.605+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/504960267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5641a58828c0 con 0x5641a595ab70 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1e917fa640 1 -- 172.21.15.18:0/504960267 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ea4039080 con 0x5641a595ab70 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1e917fa640 1 -- 172.21.15.18:0/504960267 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1ea40046f0 con 0x5641a595ab70 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1eb2c0f740 10 client.18657 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1eb2c0f740 10 client.18657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:25.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1eb2c0f740 20 client.18657 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5641a59cbe00 0x5641a59ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1eb2c0f740 4 client.18657 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/504960267 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5641a564d890 con 0x5641a59cbe00 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1ead5db640 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5641a59cbe00 0x5641a59ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1ead5db640 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5641a59cbe00 0x5641a59ce220 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89c673a5608d7735 server_cookie=a744d1cf11444b9a in_seq=0 out_seq=0 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1e917fa640 10 client.18657 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1e917fa640 1 -- 172.21.15.18:0/504960267 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5641a564d890 con 0x5641a59cbe00 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1e917fa640 10 client.18657 handle_command_reply: tid=0 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.606+0000 7f1eb2c0f740 1 client.18657 shutdown 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/504960267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 msgr2=0x5641a5969710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 0x5641a5969710 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006010 tx=0x7f1ea4019cd0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/504960267 shutdown_connections 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5641a59cbe00 0x5641a59ce220 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641a5963750 0x5641a5969c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a595ab70 0x5641a5969710 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb2c0f740 1 --2- 172.21.15.18:0/504960267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5641a595a250 0x5641a59691d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1e917fa640 0 client.18657 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/504960267 >> 172.21.15.18:0/504960267 conn(0x5641a5954980 msgr2=0x5641a595ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/504960267 shutdown_connections 2024-02-17T12:58:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb2c0f740 1 -- 172.21.15.18:0/504960267 wait complete. 2024-02-17T12:58:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb2c0f740 20 client.18657 trim_cache size 0 max 16384 2024-02-17T12:58:25.605 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:58:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb0040640 1 -- 172.21.15.18:0/547347114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1e940a2230 msgr2=0x7f1e940a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb0040640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1e940a2230 0x7f1e940a46f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ea009b190 tx=0x7f1ea00908d0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb0040640 1 -- 172.21.15.18:0/547347114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ea819bfd0 msgr2=0x7f1ea819c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb0040640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ea819bfd0 0x7f1ea819c480 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000cce0 tx=0x7f1ea000a620 comp rx=0 tx=0).stop 2024-02-17T12:58:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb0040640 1 -- 172.21.15.18:0/547347114 shutdown_connections 2024-02-17T12:58:25.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb0040640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1e940a2230 0x7f1e940a46f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb0040640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ea819bfd0 0x7f1ea819c480 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb0040640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea8106460 0x7f1ea8197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb0040640 1 --2- 172.21.15.18:0/547347114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8101390 0x7f1ea8197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:25.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb0040640 1 -- 172.21.15.18:0/547347114 >> 172.21.15.18:0/547347114 conn(0x7f1ea80fcbb0 msgr2=0x7f1ea80fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:25.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb0040640 1 -- 172.21.15.18:0/547347114 shutdown_connections 2024-02-17T12:58:25.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:25.607+0000 7f1eb0040640 1 -- 172.21.15.18:0/547347114 wait complete. 2024-02-17T12:58:26.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:25 smithi146 ceph-mon[36454]: pgmap v266: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-02-17T12:58:26.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:25 smithi018 ceph-mon[32625]: pgmap v266: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-02-17T12:58:26.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 438005 5.25 MB/sec execute 341 sec latency 1401.471 ms 2024-02-17T12:58:26.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:25 smithi119 ceph-mon[36188]: pgmap v266: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-02-17T12:58:26.609 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:58:26.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:58:26.610 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:58:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.738+0000 7f7c9815e640 1 -- 172.21.15.18:0/2267199397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90068ca0 msgr2=0x7f7c9010d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.738+0000 7f7c9815e640 1 --2- 172.21.15.18:0/2267199397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90068ca0 0x7f7c9010d5e0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009900 tx=0x7f7c8002bcd0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.738+0000 7f7c9815e640 1 -- 172.21.15.18:0/2267199397 shutdown_connections 2024-02-17T12:58:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.738+0000 7f7c9815e640 1 --2- 172.21.15.18:0/2267199397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c9010db20 0x7f7c9010ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.738+0000 7f7c9815e640 1 --2- 172.21.15.18:0/2267199397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90068ca0 0x7f7c9010d5e0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.738+0000 7f7c9815e640 1 --2- 172.21.15.18:0/2267199397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c90068380 0x7f7c90068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.738+0000 7f7c9815e640 1 -- 172.21.15.18:0/2267199397 >> 172.21.15.18:0/2267199397 conn(0x7f7c90077180 msgr2=0x7f7c90077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.738+0000 7f7c9815e640 1 -- 172.21.15.18:0/2267199397 shutdown_connections 2024-02-17T12:58:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.738+0000 7f7c9815e640 1 -- 172.21.15.18:0/2267199397 wait complete. 2024-02-17T12:58:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.738+0000 7f7c9815e640 1 Processor -- start 2024-02-17T12:58:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.738+0000 7f7c9815e640 1 -- start start 2024-02-17T12:58:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c9815e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c90068380 0x7f7c9019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c9815e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c90068ca0 0x7f7c9019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c9815e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9010db20 0x7f7c901997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c9815e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c90199e50 con 0x7f7c9010db20 2024-02-17T12:58:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c9815e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c90199fc0 con 0x7f7c90068ca0 2024-02-17T12:58:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c9815e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9019a180 con 0x7f7c90068380 2024-02-17T12:58:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c966fb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9010db20 0x7f7c901997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c966fb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9010db20 0x7f7c901997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42794/0 (socket says 172.21.15.18:42794) 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c966fb640 1 -- 172.21.15.18:0/1157303659 learned_addr learned my addr 172.21.15.18:0/1157303659 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c956f9640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c90068ca0 0x7f7c9019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c95efa640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c90068380 0x7f7c9019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c966fb640 1 -- 172.21.15.18:0/1157303659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c90068380 msgr2=0x7f7c9019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c966fb640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c90068380 0x7f7c9019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c966fb640 1 -- 172.21.15.18:0/1157303659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c90068ca0 msgr2=0x7f7c9019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c966fb640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c90068ca0 0x7f7c9019fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c966fb640 1 -- 172.21.15.18:0/1157303659 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c80009560 con 0x7f7c9010db20 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c966fb640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9010db20 0x7f7c901997b0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00eaa0 tx=0x7f7c8c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c86ffd640 1 -- 172.21.15.18:0/1157303659 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c00cdb0 con 0x7f7c9010db20 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c86ffd640 1 -- 172.21.15.18:0/1157303659 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c8c005020 con 0x7f7c9010db20 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c9815e640 1 -- 172.21.15.18:0/1157303659 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9019a460 con 0x7f7c9010db20 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.739+0000 7f7c86ffd640 1 -- 172.21.15.18:0/1157303659 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0116b0 con 0x7f7c9010db20 2024-02-17T12:58:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.740+0000 7f7c9815e640 1 -- 172.21.15.18:0/1157303659 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c901a6670 con 0x7f7c9010db20 2024-02-17T12:58:26.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.742+0000 7f7c86ffd640 1 -- 172.21.15.18:0/1157303659 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7c8c005520 con 0x7f7c9010db20 2024-02-17T12:58:26.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.742+0000 7f7c9ae0d740 1 Processor -- start 2024-02-17T12:58:26.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c9ae0d740 1 -- start start 2024-02-17T12:58:26.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c9ae0d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 0x55edea94d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c9ae0d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94dd30 0x55edea9562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c9ae0d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea9568b0 0x55edea958d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c9ae0d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edea8bbf90 con 0x55edea94d380 2024-02-17T12:58:26.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c9ae0d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edea8b94c0 con 0x55edea9568b0 2024-02-17T12:58:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c9ae0d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edea8b4ca0 con 0x55edea94dd30 2024-02-17T12:58:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c86ffd640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7c7c0a22b0 0x7f7c7c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c956f9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 0x55edea94d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c956f9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 0x55edea94d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42796/0 (socket says 172.21.15.18:42796) 2024-02-17T12:58:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c956f9640 1 -- 172.21.15.18:0/1704312366 learned_addr learned my addr 172.21.15.18:0/1704312366 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c966fb640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7c7c0a22b0 0x7f7c7c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c95efa640 1 --2- 172.21.15.18:0/1704312366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94dd30 0x55edea9562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c956f9640 1 --2- 172.21.15.18:0/1704312366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea9568b0 0x55edea958d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c86ffd640 1 -- 172.21.15.18:0/1157303659 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7c8c0ce360 con 0x7f7c9010db20 2024-02-17T12:58:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c956f9640 1 -- 172.21.15.18:0/1704312366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94dd30 msgr2=0x55edea9562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c956f9640 1 --2- 172.21.15.18:0/1704312366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94dd30 0x55edea9562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c956f9640 1 -- 172.21.15.18:0/1704312366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea9568b0 msgr2=0x55edea958d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c956f9640 1 --2- 172.21.15.18:0/1704312366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea9568b0 0x55edea958d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c956f9640 1 -- 172.21.15.18:0/1704312366 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edea8770b0 con 0x55edea94d380 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c95efa640 1 --2- 172.21.15.18:0/1704312366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94dd30 0x55edea9562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.743+0000 7f7c956f9640 1 --2- 172.21.15.18:0/1704312366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 0x55edea94d760 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005ec0 tx=0x7f7c80005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1deb72a421b9b165 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.744+0000 7f7c966fb640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7c7c0a22b0 0x7f7c7c0a4770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0969f0 tx=0x7f7c8c091040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.744+0000 7f7c7bfff640 1 -- 172.21.15.18:0/1704312366 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c800048c0 con 0x55edea94d380 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.744+0000 7f7c7bfff640 1 -- 172.21.15.18:0/1704312366 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c80004510 con 0x55edea94d380 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.744+0000 7f7c7bfff640 1 -- 172.21.15.18:0/1704312366 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c800054c0 con 0x55edea94d380 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.744+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1704312366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 msgr2=0x55edea94d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.744+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/1704312366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 0x55edea94d760 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005ec0 tx=0x7f7c80005b50 comp rx=0 tx=0).stop 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.744+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1704312366 shutdown_connections 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.744+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/1704312366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea9568b0 0x55edea958d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.744+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/1704312366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94dd30 0x55edea9562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.744+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/1704312366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 0x55edea94d760 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.744+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1704312366 >> 172.21.15.18:0/1704312366 conn(0x55edea947840 msgr2=0x55edea947c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.745+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1704312366 shutdown_connections 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.745+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1704312366 wait complete. 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.745+0000 7f7c9ae0d740 1 -- start start 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.745+0000 7f7c9ae0d740 1 Processor -- start 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.745+0000 7f7c9ae0d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94d380 0x55edea953870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.745+0000 7f7c9ae0d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea94dd30 0x55edea953db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.745+0000 7f7c9ae0d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea9568b0 0x55edea9542f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.745+0000 7f7c9ae0d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edea859f40 con 0x55edea9568b0 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.745+0000 7f7c9ae0d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edea5884b0 con 0x55edea94dd30 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.745+0000 7f7c9ae0d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edea83a670 con 0x55edea94d380 2024-02-17T12:58:26.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c9ae0d740 10 client.0 _ll_register_callbacks cb 0x55edea8a7f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c956f9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea9568b0 0x55edea9542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c956f9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea9568b0 0x55edea9542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42804/0 (socket says 172.21.15.18:42804) 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c956f9640 1 -- 172.21.15.18:0/1357126973 learned_addr learned my addr 172.21.15.18:0/1357126973 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c956f9640 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94d380 0x55edea953870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c95efa640 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea94dd30 0x55edea953db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c956f9640 1 -- 172.21.15.18:0/1357126973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94d380 msgr2=0x55edea953870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c956f9640 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94d380 0x55edea953870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c956f9640 1 -- 172.21.15.18:0/1357126973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea94dd30 msgr2=0x55edea953db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c956f9640 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea94dd30 0x55edea953db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c956f9640 1 -- 172.21.15.18:0/1357126973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c90110a70 con 0x55edea9568b0 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c956f9640 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea9568b0 0x55edea9542f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005130 tx=0x7f7c8002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239d553cce1940d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c797fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c797fa640 1 -- 172.21.15.18:0/1357126973 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80004250 con 0x55edea9568b0 2024-02-17T12:58:26.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c797fa640 1 -- 172.21.15.18:0/1357126973 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7c800043b0 con 0x55edea9568b0 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1357126973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edea88ef40 con 0x55edea9568b0 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c797fa640 1 -- 172.21.15.18:0/1357126973 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c800035f0 con 0x55edea9568b0 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c9ae0d740 10 client.18669 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.746+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1357126973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55edea8b3320 con 0x55edea9568b0 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.747+0000 7f7c797fa640 1 -- 172.21.15.18:0/1357126973 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55edea88ef40 con 0x55edea9568b0 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.747+0000 7f7c797fa640 1 -- 172.21.15.18:0/1357126973 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7c80041460 con 0x55edea9568b0 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.747+0000 7f7c9ae0d740 10 client.18669 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.747+0000 7f7c9ae0d740 10 client.18669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.747+0000 7f7c9ae0d740 20 client.18669 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.747+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55edea966ab0 0x55edea968ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.747+0000 7f7c9ae0d740 4 client.18669 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.747+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1357126973 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edea8ab310 con 0x55edea966ab0 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.747+0000 7f7c95efa640 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55edea966ab0 0x55edea968ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.747+0000 7f7c797fa640 10 client.18669 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:26.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.747+0000 7f7c95efa640 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55edea966ab0 0x55edea968ed0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7427904dd280826 server_cookie=b170e9c0124394b3 in_seq=0 out_seq=0 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c797fa640 1 -- 172.21.15.18:0/1357126973 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55edea8ab310 con 0x55edea966ab0 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c797fa640 10 client.18669 handle_command_reply: tid=0 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c9ae0d740 1 client.18669 shutdown 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1357126973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea9568b0 msgr2=0x55edea9542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea9568b0 0x55edea9542f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005130 tx=0x7f7c8002bcd0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1357126973 shutdown_connections 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55edea966ab0 0x55edea968ed0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea9568b0 0x55edea9542f0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea94dd30 0x55edea953db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/1357126973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94d380 0x55edea953870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c797fa640 0 client.18669 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1357126973 >> 172.21.15.18:0/1357126973 conn(0x55edea947840 msgr2=0x55edea9497a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:26.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1357126973 shutdown_connections 2024-02-17T12:58:26.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/1357126973 wait complete. 2024-02-17T12:58:26.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.748+0000 7f7c9ae0d740 20 client.18669 trim_cache size 0 max 16384 2024-02-17T12:58:26.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c9ae0d740 1 Processor -- start 2024-02-17T12:58:26.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c9ae0d740 1 -- start start 2024-02-17T12:58:26.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c9ae0d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 0x55edea955fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c9ae0d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94dd30 0x55edea9529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c9ae0d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea9568b0 0x55edea953f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c9ae0d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edea8ab310 con 0x55edea94d380 2024-02-17T12:58:26.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c9ae0d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edea88ef40 con 0x55edea9568b0 2024-02-17T12:58:26.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c9ae0d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edea94ccd0 con 0x55edea94dd30 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c956f9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 0x55edea955fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c956f9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 0x55edea955fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42820/0 (socket says 172.21.15.18:42820) 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c956f9640 1 -- 172.21.15.18:0/2013544744 learned_addr learned my addr 172.21.15.18:0/2013544744 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c956f9640 1 -- 172.21.15.18:0/2013544744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94dd30 msgr2=0x55edea9529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c95efa640 1 --2- 172.21.15.18:0/2013544744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94dd30 0x55edea9529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c956f9640 1 --2- 172.21.15.18:0/2013544744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94dd30 0x55edea9529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c956f9640 1 -- 172.21.15.18:0/2013544744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea9568b0 msgr2=0x55edea953f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c956f9640 1 --2- 172.21.15.18:0/2013544744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea9568b0 0x55edea953f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c956f9640 1 -- 172.21.15.18:0/2013544744 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edea84fc00 con 0x55edea94d380 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.764+0000 7f7c95efa640 1 --2- 172.21.15.18:0/2013544744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94dd30 0x55edea9529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c956f9640 1 --2- 172.21.15.18:0/2013544744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 0x55edea955fa0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f7c8002cd00 tx=0x7f7c80005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe49f0a2d7c1cb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c7b7fe640 1 -- 172.21.15.18:0/2013544744 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8002c880 con 0x55edea94d380 2024-02-17T12:58:26.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c7b7fe640 1 -- 172.21.15.18:0/2013544744 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c80055070 con 0x55edea94d380 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c7b7fe640 1 -- 172.21.15.18:0/2013544744 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80004f60 con 0x55edea94d380 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/2013544744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 msgr2=0x55edea955fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/2013544744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 0x55edea955fa0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f7c8002cd00 tx=0x7f7c80005950 comp rx=0 tx=0).stop 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/2013544744 shutdown_connections 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/2013544744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea9568b0 0x55edea953f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/2013544744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea94dd30 0x55edea9529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/2013544744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94d380 0x55edea955fa0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/2013544744 >> 172.21.15.18:0/2013544744 conn(0x55edea947840 msgr2=0x55edea953830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/2013544744 shutdown_connections 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.765+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/2013544744 wait complete. 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c9ae0d740 1 -- start start 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c9ae0d740 1 Processor -- start 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c9ae0d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea94d380 0x55edea9b7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c9ae0d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94dd30 0x55edea9b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c9ae0d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea9568b0 0x55edea95cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c9ae0d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edea95d360 con 0x55edea94dd30 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c9ae0d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edea95d4d0 con 0x55edea94d380 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c9ae0d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edea95d690 con 0x55edea9568b0 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c9ae0d740 10 client.0 _ll_register_callbacks cb 0x7f7c80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c95efa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94dd30 0x55edea9b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c95efa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94dd30 0x55edea9b7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42824/0 (socket says 172.21.15.18:42824) 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c95efa640 1 -- 172.21.15.18:0/3292146248 learned_addr learned my addr 172.21.15.18:0/3292146248 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c956f9640 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea94d380 0x55edea9b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c956f9640 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea9568b0 0x55edea95cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c95efa640 1 -- 172.21.15.18:0/3292146248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea9568b0 msgr2=0x55edea95cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.766+0000 7f7c95efa640 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea9568b0 0x55edea95cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c95efa640 1 -- 172.21.15.18:0/3292146248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea94d380 msgr2=0x55edea9b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c95efa640 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea94d380 0x55edea9b7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c95efa640 1 -- 172.21.15.18:0/3292146248 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edea8b3320 con 0x55edea94dd30 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c956f9640 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea94d380 0x55edea9b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:26.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c95efa640 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94dd30 0x55edea9b7ad0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800b450 tx=0x7f7c880128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e000ada719a6da server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c797fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c797fa640 1 -- 172.21.15.18:0/3292146248 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c88017040 con 0x55edea94dd30 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c797fa640 1 -- 172.21.15.18:0/3292146248 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7c88005020 con 0x55edea94dd30 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/3292146248 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edea8da900 con 0x55edea94dd30 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c797fa640 1 -- 172.21.15.18:0/3292146248 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c8801d8e0 con 0x55edea94dd30 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c9ae0d740 10 client.18675 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/3292146248 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55edea875910 con 0x55edea94dd30 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c797fa640 1 -- 172.21.15.18:0/3292146248 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55edea8da900 con 0x55edea94dd30 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c797fa640 1 -- 172.21.15.18:0/3292146248 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7c880114d0 con 0x55edea94dd30 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c9ae0d740 10 client.18675 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c9ae0d740 10 client.18675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c9ae0d740 20 client.18675 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:26.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55edea9bf440 0x55edea9c1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c9ae0d740 4 client.18675 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:58:26.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.767+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/3292146248 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55edea5b4860 con 0x55edea9bf440 2024-02-17T12:58:26.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.768+0000 7f7c956f9640 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55edea9bf440 0x55edea9c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.768+0000 7f7c956f9640 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55edea9bf440 0x55edea9c1860 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f6578af80192046 server_cookie=a6f5257db50ec739 in_seq=0 out_seq=0 2024-02-17T12:58:26.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.768+0000 7f7c797fa640 10 client.18675 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:26.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c797fa640 1 -- 172.21.15.18:0/3292146248 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55edea5b4860 con 0x55edea9bf440 2024-02-17T12:58:26.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c797fa640 10 client.18675 handle_command_reply: tid=0 2024-02-17T12:58:26.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c9ae0d740 1 client.18675 shutdown 2024-02-17T12:58:26.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/3292146248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94dd30 msgr2=0x55edea9b7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94dd30 0x55edea9b7ad0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800b450 tx=0x7f7c880128e0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/3292146248 shutdown_connections 2024-02-17T12:58:26.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55edea9bf440 0x55edea9c1860 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55edea9568b0 0x55edea95cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55edea94dd30 0x55edea9b7ad0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c9ae0d740 1 --2- 172.21.15.18:0/3292146248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55edea94d380 0x55edea9b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c797fa640 0 client.18675 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:26.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/3292146248 >> 172.21.15.18:0/3292146248 conn(0x55edea947840 msgr2=0x55edea9468e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:26.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.769+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/3292146248 shutdown_connections 2024-02-17T12:58:26.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9ae0d740 1 -- 172.21.15.18:0/3292146248 wait complete. 2024-02-17T12:58:26.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9ae0d740 20 client.18675 trim_cache size 0 max 16384 2024-02-17T12:58:26.767 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:58:26.767 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:58:26.767 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "561a32fc-14b9-4e50-baa4-8300a3ab2a4e", 2024-02-17T12:58:26.767 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:58:26.767 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:58:26.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9815e640 1 -- 172.21.15.18:0/1157303659 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7c7c0a22b0 msgr2=0x7f7c7c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9815e640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7c7c0a22b0 0x7f7c7c0a4770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0969f0 tx=0x7f7c8c091040 comp rx=0 tx=0).stop 2024-02-17T12:58:26.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9815e640 1 -- 172.21.15.18:0/1157303659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9010db20 msgr2=0x7f7c901997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9815e640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9010db20 0x7f7c901997b0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00eaa0 tx=0x7f7c8c00ef60 comp rx=0 tx=0).stop 2024-02-17T12:58:26.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9815e640 1 -- 172.21.15.18:0/1157303659 shutdown_connections 2024-02-17T12:58:26.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9815e640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7c7c0a22b0 0x7f7c7c0a4770 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9815e640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9010db20 0x7f7c901997b0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9815e640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c90068ca0 0x7f7c9019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9815e640 1 --2- 172.21.15.18:0/1157303659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c90068380 0x7f7c9019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9815e640 1 -- 172.21.15.18:0/1157303659 >> 172.21.15.18:0/1157303659 conn(0x7f7c90077180 msgr2=0x7f7c9010f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:26.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9815e640 1 -- 172.21.15.18:0/1157303659 shutdown_connections 2024-02-17T12:58:26.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.770+0000 7f7c9815e640 1 -- 172.21.15.18:0/1157303659 wait complete. 2024-02-17T12:58:26.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:58:26.773 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:58:26.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.896+0000 7fe576756640 1 -- 172.21.15.18:0/731300716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe570105340 msgr2=0x7fe57010cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.896+0000 7fe576756640 1 --2- 172.21.15.18:0/731300716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe570105340 0x7fe57010cf10 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fe568009cd0 tx=0x7fe56802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.897+0000 7fe576756640 1 -- 172.21.15.18:0/731300716 shutdown_connections 2024-02-17T12:58:26.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.897+0000 7fe576756640 1 --2- 172.21.15.18:0/731300716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe570105340 0x7fe57010cf10 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.897+0000 7fe576756640 1 --2- 172.21.15.18:0/731300716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe570100c20 0x7fe570104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.897+0000 7fe576756640 1 --2- 172.21.15.18:0/731300716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe570100270 0x7fe570100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.897+0000 7fe576756640 1 -- 172.21.15.18:0/731300716 >> 172.21.15.18:0/731300716 conn(0x7fe570077230 msgr2=0x7fe570077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:26.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.897+0000 7fe576756640 1 -- 172.21.15.18:0/731300716 shutdown_connections 2024-02-17T12:58:26.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.897+0000 7fe576756640 1 -- 172.21.15.18:0/731300716 wait complete. 2024-02-17T12:58:26.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.897+0000 7fe576756640 1 Processor -- start 2024-02-17T12:58:26.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.897+0000 7fe576756640 1 -- start start 2024-02-17T12:58:26.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe576756640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe570100270 0x7fe570197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe576756640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe570105340 0x7fe570197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe576756640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe57019c000 0x7fe57019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe576756640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57019c9f0 con 0x7fe57019c000 2024-02-17T12:58:26.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe576756640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57019cb60 con 0x7fe570105340 2024-02-17T12:58:26.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe576756640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57019ccd0 con 0x7fe570100270 2024-02-17T12:58:26.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe574cf3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe57019c000 0x7fe57019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe574cf3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe57019c000 0x7fe57019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42850/0 (socket says 172.21.15.18:42850) 2024-02-17T12:58:26.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe574cf3640 1 -- 172.21.15.18:0/3257786716 learned_addr learned my addr 172.21.15.18:0/3257786716 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe56ffff640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe570100270 0x7fe570197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe56f7fe640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe570105340 0x7fe570197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe574cf3640 1 -- 172.21.15.18:0/3257786716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe570100270 msgr2=0x7fe570197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe574cf3640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe570100270 0x7fe570197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe574cf3640 1 -- 172.21.15.18:0/3257786716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe570105340 msgr2=0x7fe570197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe574cf3640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe570105340 0x7fe570197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe574cf3640 1 -- 172.21.15.18:0/3257786716 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe568009900 con 0x7fe57019c000 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.898+0000 7fe574cf3640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe57019c000 0x7fe57019c4b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe568005e30 tx=0x7fe5680325e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.899+0000 7fe56d7fa640 1 -- 172.21.15.18:0/3257786716 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568034e20 con 0x7fe57019c000 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.899+0000 7fe576756640 1 -- 172.21.15.18:0/3257786716 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5701a42b0 con 0x7fe57019c000 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.899+0000 7fe56d7fa640 1 -- 172.21.15.18:0/3257786716 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe568003e50 con 0x7fe57019c000 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.899+0000 7fe576756640 1 -- 172.21.15.18:0/3257786716 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5701a4730 con 0x7fe57019c000 2024-02-17T12:58:26.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.900+0000 7fe56d7fa640 1 -- 172.21.15.18:0/3257786716 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568004750 con 0x7fe57019c000 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.901+0000 7fe56d7fa640 1 -- 172.21.15.18:0/3257786716 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe56800a040 con 0x7fe57019c000 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe579407740 1 Processor -- start 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe56d7fa640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe5580a22b0 0x7fe5580a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe579407740 1 -- start start 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe56f7fe640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe5580a22b0 0x7fe5580a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe579407740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da50082b0 0x557da5008690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe579407740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 0x557da5011270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe56d7fa640 1 -- 172.21.15.18:0/3257786716 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe5680f0a80 con 0x7fe57019c000 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe579407740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50117b0 0x557da5013ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe579407740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da4f77280 con 0x557da5008bd0 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe579407740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da4f747b0 con 0x557da50117b0 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe579407740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da4f6ff90 con 0x557da50082b0 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 0x557da5011270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 0x557da5011270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42866/0 (socket says 172.21.15.18:42866) 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe56f7fe640 1 -- 172.21.15.18:0/1216287575 learned_addr learned my addr 172.21.15.18:0/1216287575 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe574cf3640 1 --2- 172.21.15.18:0/1216287575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50117b0 0x557da5013ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe56ffff640 1 --2- 172.21.15.18:0/1216287575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da50082b0 0x557da5008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe574cf3640 1 -- 172.21.15.18:0/1216287575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da50082b0 msgr2=0x557da5008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe574cf3640 1 --2- 172.21.15.18:0/1216287575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da50082b0 0x557da5008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe574cf3640 1 -- 172.21.15.18:0/1216287575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 msgr2=0x557da5011270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe574cf3640 1 --2- 172.21.15.18:0/1216287575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 0x557da5011270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe56f7fe640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe5580a22b0 0x7fe5580a4770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe564013460 tx=0x7fe564013860 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.902+0000 7fe574cf3640 1 -- 172.21.15.18:0/1216287575 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557da4f2e570 con 0x557da50117b0 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.903+0000 7fe574cf3640 1 --2- 172.21.15.18:0/1216287575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50117b0 0x557da5013ba0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe5680ee300 tx=0x7fe5680bd1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f11e8655d194a053 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.903+0000 7fe5567fc640 1 -- 172.21.15.18:0/1216287575 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680c16f0 con 0x557da50117b0 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.903+0000 7fe5567fc640 1 -- 172.21.15.18:0/1216287575 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5680c5d40 con 0x557da50117b0 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.903+0000 7fe5567fc640 1 -- 172.21.15.18:0/1216287575 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680ee5e0 con 0x557da50117b0 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.903+0000 7fe579407740 1 -- 172.21.15.18:0/1216287575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50117b0 msgr2=0x557da5013ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.903+0000 7fe579407740 1 --2- 172.21.15.18:0/1216287575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50117b0 0x557da5013ba0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe5680ee300 tx=0x7fe5680bd1a0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.903+0000 7fe579407740 1 -- 172.21.15.18:0/1216287575 shutdown_connections 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.903+0000 7fe579407740 1 --2- 172.21.15.18:0/1216287575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50117b0 0x557da5013ba0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.903+0000 7fe579407740 1 --2- 172.21.15.18:0/1216287575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 0x557da5011270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.903+0000 7fe579407740 1 --2- 172.21.15.18:0/1216287575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da50082b0 0x557da5008690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.903+0000 7fe579407740 1 -- 172.21.15.18:0/1216287575 >> 172.21.15.18:0/1216287575 conn(0x557da5002990 msgr2=0x557da5002d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.903+0000 7fe579407740 1 -- 172.21.15.18:0/1216287575 shutdown_connections 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe579407740 1 -- 172.21.15.18:0/1216287575 wait complete. 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe579407740 1 -- start start 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe579407740 1 Processor -- start 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe579407740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50082b0 0x557da5011160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe579407740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da5008bd0 0x557da500e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe579407740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50117b0 0x557da500ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe579407740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da4c43430 con 0x557da50117b0 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe579407740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da4bffe00 con 0x557da50082b0 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe579407740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da4ef4fd0 con 0x557da5008bd0 2024-02-17T12:58:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe579407740 10 client.0 _ll_register_callbacks cb 0x557da4f4afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe574cf3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50117b0 0x557da500ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50082b0 0x557da5011160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe574cf3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50117b0 0x557da500ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42870/0 (socket says 172.21.15.18:42870) 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe574cf3640 1 -- 172.21.15.18:0/2222416123 learned_addr learned my addr 172.21.15.18:0/2222416123 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.904+0000 7fe56f7fe640 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da5008bd0 0x557da500e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe574cf3640 1 -- 172.21.15.18:0/2222416123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da5008bd0 msgr2=0x557da500e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe574cf3640 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da5008bd0 0x557da500e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe574cf3640 1 -- 172.21.15.18:0/2222416123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50082b0 msgr2=0x557da5011160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe574cf3640 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50082b0 0x557da5011160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe574cf3640 1 -- 172.21.15.18:0/2222416123 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5701a4730 con 0x557da50117b0 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe56ffff640 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50082b0 0x557da5011160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe574cf3640 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50117b0 0x557da500ea90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe5680ee360 tx=0x7fe56802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b33796df40523d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe54f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe54f7fe640 1 -- 172.21.15.18:0/2222416123 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680c16f0 con 0x557da50117b0 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe54f7fe640 1 -- 172.21.15.18:0/2222416123 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe5680328b0 con 0x557da50117b0 2024-02-17T12:58:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe579407740 1 -- 172.21.15.18:0/2222416123 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557da4c6f7e0 con 0x557da50117b0 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe54f7fe640 1 -- 172.21.15.18:0/2222416123 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe568032a70 con 0x557da50117b0 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe579407740 10 client.18687 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe579407740 1 -- 172.21.15.18:0/2222416123 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x557da4f6e610 con 0x557da50117b0 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.905+0000 7fe54f7fe640 1 -- 172.21.15.18:0/2222416123 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557da4c6f7e0 con 0x557da50117b0 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.906+0000 7fe54f7fe640 1 -- 172.21.15.18:0/2222416123 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe568004b40 con 0x557da50117b0 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.906+0000 7fe579407740 10 client.18687 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.906+0000 7fe579407740 10 client.18687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.906+0000 7fe579407740 20 client.18687 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.906+0000 7fe579407740 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557da50212b0 0x557da50236d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.906+0000 7fe579407740 4 client.18687 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.906+0000 7fe579407740 1 -- 172.21.15.18:0/2222416123 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557da4cac720 con 0x557da50212b0 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.906+0000 7fe56ffff640 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557da50212b0 0x557da50236d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.906+0000 7fe54f7fe640 10 client.18687 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.906+0000 7fe56ffff640 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557da50212b0 0x557da50236d0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f4b59ddfaf1c60 server_cookie=3309336375f57ce9 in_seq=0 out_seq=0 2024-02-17T12:58:26.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe54f7fe640 1 -- 172.21.15.18:0/2222416123 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557da4cac720 con 0x557da50212b0 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe54f7fe640 10 client.18687 handle_command_reply: tid=0 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe579407740 1 client.18687 shutdown 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe579407740 1 -- 172.21.15.18:0/2222416123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50117b0 msgr2=0x557da500ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe579407740 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50117b0 0x557da500ea90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe5680ee360 tx=0x7fe56802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe579407740 1 -- 172.21.15.18:0/2222416123 shutdown_connections 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe579407740 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557da50212b0 0x557da50236d0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe579407740 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50117b0 0x557da500ea90 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe579407740 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da5008bd0 0x557da500e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe54f7fe640 0 client.18687 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe579407740 1 --2- 172.21.15.18:0/2222416123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50082b0 0x557da5011160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe579407740 1 -- 172.21.15.18:0/2222416123 >> 172.21.15.18:0/2222416123 conn(0x557da5002990 msgr2=0x557da5004730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe579407740 1 -- 172.21.15.18:0/2222416123 shutdown_connections 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe579407740 1 -- 172.21.15.18:0/2222416123 wait complete. 2024-02-17T12:58:26.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.907+0000 7fe579407740 20 client.18687 trim_cache size 0 max 16384 2024-02-17T12:58:26.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe579407740 1 Processor -- start 2024-02-17T12:58:26.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe579407740 1 -- start start 2024-02-17T12:58:26.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe579407740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50082b0 0x557da500d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe579407740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 0x557da5010c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe579407740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da50117b0 0x557da5011160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe579407740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da5003a60 con 0x557da5008bd0 2024-02-17T12:58:26.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe579407740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da50073a0 con 0x557da50082b0 2024-02-17T12:58:26.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe579407740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da4ffe2b0 con 0x557da50117b0 2024-02-17T12:58:26.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 0x557da5010c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 0x557da5010c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42880/0 (socket says 172.21.15.18:42880) 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe56f7fe640 1 -- 172.21.15.18:0/2490309989 learned_addr learned my addr 172.21.15.18:0/2490309989 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe574cf3640 1 --2- 172.21.15.18:0/2490309989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da50117b0 0x557da5011160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe56ffff640 1 --2- 172.21.15.18:0/2490309989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50082b0 0x557da500d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe56f7fe640 1 -- 172.21.15.18:0/2490309989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da50117b0 msgr2=0x557da5011160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe56f7fe640 1 --2- 172.21.15.18:0/2490309989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da50117b0 0x557da5011160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe56f7fe640 1 -- 172.21.15.18:0/2490309989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50082b0 msgr2=0x557da500d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe56f7fe640 1 --2- 172.21.15.18:0/2490309989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50082b0 0x557da500d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.924+0000 7fe56f7fe640 1 -- 172.21.15.18:0/2490309989 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557da4f0aac0 con 0x557da5008bd0 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.925+0000 7fe56f7fe640 1 --2- 172.21.15.18:0/2490309989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 0x557da5010c20 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fe564013d60 tx=0x7fe56401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633b3109f935f8ec server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.925+0000 7fe555ffb640 1 -- 172.21.15.18:0/2490309989 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564025070 con 0x557da5008bd0 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.925+0000 7fe555ffb640 1 -- 172.21.15.18:0/2490309989 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5640044a0 con 0x557da5008bd0 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.925+0000 7fe555ffb640 1 -- 172.21.15.18:0/2490309989 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564018040 con 0x557da5008bd0 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.925+0000 7fe579407740 1 -- 172.21.15.18:0/2490309989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 msgr2=0x557da5010c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.925+0000 7fe579407740 1 --2- 172.21.15.18:0/2490309989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 0x557da5010c20 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fe564013d60 tx=0x7fe56401f000 comp rx=0 tx=0).stop 2024-02-17T12:58:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.925+0000 7fe579407740 1 -- 172.21.15.18:0/2490309989 shutdown_connections 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.925+0000 7fe579407740 1 --2- 172.21.15.18:0/2490309989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da50117b0 0x557da5011160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 --2- 172.21.15.18:0/2490309989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da5008bd0 0x557da5010c20 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 --2- 172.21.15.18:0/2490309989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50082b0 0x557da500d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 -- 172.21.15.18:0/2490309989 >> 172.21.15.18:0/2490309989 conn(0x557da5002990 msgr2=0x557da500cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 -- 172.21.15.18:0/2490309989 shutdown_connections 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 -- 172.21.15.18:0/2490309989 wait complete. 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 -- start start 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 Processor -- start 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50082b0 0x557da5014f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da5008bd0 0x557da50154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50117b0 0x557da50159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da50777b0 con 0x557da50082b0 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da5077920 con 0x557da50117b0 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da5077ae0 con 0x557da5008bd0 2024-02-17T12:58:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe579407740 10 client.0 _ll_register_callbacks cb 0x7fe568005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50082b0 0x557da5014f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50082b0 0x557da5014f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42890/0 (socket says 172.21.15.18:42890) 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe56ffff640 1 -- 172.21.15.18:0/1152319944 learned_addr learned my addr 172.21.15.18:0/1152319944 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe56f7fe640 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da5008bd0 0x557da50154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.926+0000 7fe574cf3640 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50117b0 0x557da50159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe56f7fe640 1 -- 172.21.15.18:0/1152319944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50117b0 msgr2=0x557da50159e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe56f7fe640 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50117b0 0x557da50159e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe56f7fe640 1 -- 172.21.15.18:0/1152319944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50082b0 msgr2=0x557da5014f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe56f7fe640 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50082b0 0x557da5014f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe56f7fe640 1 -- 172.21.15.18:0/1152319944 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5640041a0 con 0x557da5008bd0 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe56ffff640 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50082b0 0x557da5014f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe56f7fe640 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da5008bd0 0x557da50154a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x557da4cac830 tx=0x7fe56400fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=880839dc87fee651 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe54f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe54f7fe640 1 -- 172.21.15.18:0/1152319944 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564018070 con 0x557da5008bd0 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe579407740 1 -- 172.21.15.18:0/1152319944 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5680c5d40 con 0x557da5008bd0 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe54f7fe640 1 -- 172.21.15.18:0/1152319944 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe56402a660 con 0x557da5008bd0 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe54f7fe640 1 -- 172.21.15.18:0/1152319944 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe564025050 con 0x557da5008bd0 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe579407740 10 client.27844 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.927+0000 7fe579407740 1 -- 172.21.15.18:0/1152319944 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x557da4f306a0 con 0x557da5008bd0 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.928+0000 7fe54f7fe640 1 -- 172.21.15.18:0/1152319944 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5680c5d40 con 0x557da5008bd0 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.928+0000 7fe54f7fe640 1 -- 172.21.15.18:0/1152319944 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe564020450 con 0x557da5008bd0 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.928+0000 7fe579407740 10 client.27844 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.928+0000 7fe579407740 10 client.27844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.928+0000 7fe579407740 20 client.27844 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.928+0000 7fe579407740 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557da50792a0 0x557da507b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.928+0000 7fe579407740 4 client.27844 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.928+0000 7fe579407740 1 -- 172.21.15.18:0/1152319944 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557da4cfb890 con 0x557da50792a0 2024-02-17T12:58:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.928+0000 7fe56ffff640 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557da50792a0 0x557da507b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.928+0000 7fe56ffff640 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557da50792a0 0x557da507b6c0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beecf2b6f0b49eee server_cookie=84bbb8c9edd51c16 in_seq=0 out_seq=0 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.928+0000 7fe54f7fe640 10 client.27844 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe54f7fe640 1 -- 172.21.15.18:0/1152319944 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557da4cfb890 con 0x557da50792a0 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe54f7fe640 10 client.27844 handle_command_reply: tid=0 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe579407740 1 client.27844 shutdown 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe579407740 1 -- 172.21.15.18:0/1152319944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da5008bd0 msgr2=0x557da50154a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe579407740 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da5008bd0 0x557da50154a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x557da4cac830 tx=0x7fe56400fb00 comp rx=0 tx=0).stop 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe579407740 1 -- 172.21.15.18:0/1152319944 shutdown_connections 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe579407740 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557da50792a0 0x557da507b6c0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe579407740 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557da50117b0 0x557da50159e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe579407740 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557da5008bd0 0x557da50154a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe54f7fe640 0 client.27844 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe579407740 1 --2- 172.21.15.18:0/1152319944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557da50082b0 0x557da5014f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe579407740 1 -- 172.21.15.18:0/1152319944 >> 172.21.15.18:0/1152319944 conn(0x557da5002990 msgr2=0x557da5001a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:26.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe579407740 1 -- 172.21.15.18:0/1152319944 shutdown_connections 2024-02-17T12:58:26.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe579407740 1 -- 172.21.15.18:0/1152319944 wait complete. 2024-02-17T12:58:26.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.929+0000 7fe579407740 20 client.27844 trim_cache size 0 max 16384 2024-02-17T12:58:26.927 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:58:26.927 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:58:26.927 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:58:26.927 INFO:teuthology.orchestra.run.smithi018.stdout: "561a32fc-14b9-4e50-baa4-8300a3ab2a4e": { 2024-02-17T12:58:26.927 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:58:26.927 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "561a32fc-14b9-4e50-baa4-8300a3ab2a4e", 2024-02-17T12:58:26.927 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:58:26.928 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:58:26.928 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:58:26.928 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:58:26.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.930+0000 7fe576756640 1 -- 172.21.15.18:0/3257786716 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe5580a22b0 msgr2=0x7fe5580a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.930+0000 7fe576756640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe5580a22b0 0x7fe5580a4770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe564013460 tx=0x7fe564013860 comp rx=0 tx=0).stop 2024-02-17T12:58:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.930+0000 7fe576756640 1 -- 172.21.15.18:0/3257786716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe57019c000 msgr2=0x7fe57019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.930+0000 7fe576756640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe57019c000 0x7fe57019c4b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe568005e30 tx=0x7fe5680325e0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.930+0000 7fe576756640 1 -- 172.21.15.18:0/3257786716 shutdown_connections 2024-02-17T12:58:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.930+0000 7fe576756640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe5580a22b0 0x7fe5580a4770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.930+0000 7fe576756640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe57019c000 0x7fe57019c4b0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.930+0000 7fe576756640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe570105340 0x7fe570197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.930+0000 7fe576756640 1 --2- 172.21.15.18:0/3257786716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe570100270 0x7fe570197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.930+0000 7fe576756640 1 -- 172.21.15.18:0/3257786716 >> 172.21.15.18:0/3257786716 conn(0x7fe570077230 msgr2=0x7fe570068630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.930+0000 7fe576756640 1 -- 172.21.15.18:0/3257786716 shutdown_connections 2024-02-17T12:58:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:26.930+0000 7fe576756640 1 -- 172.21.15.18:0/3257786716 wait complete. 2024-02-17T12:58:26.934 INFO:tasks.cephfs.filesystem:scrub status for tag:561a32fc-14b9-4e50-baa4-8300a3ab2a4e - {'path': '/', 'tag': '561a32fc-14b9-4e50-baa4-8300a3ab2a4e', 'options': 'recursive,force'} 2024-02-17T12:58:27.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:26 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:58:27.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:26 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:58:27.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:26 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:26 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:26 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:58:27.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:26 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:26 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:58:27.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:26 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:26 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:58:27.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:26 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:26 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:58:27.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:26 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:58:27.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:26 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:26 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:26 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:58:27.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:26 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:26 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:58:27.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:26 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:26 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:58:27.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:26 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 439242 5.26 MB/sec execute 342 sec latency 1907.201 ms 2024-02-17T12:58:27.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:26 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:58:27.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:26 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:58:27.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:26 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:26 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:26 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:58:27.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:26 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:26 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:58:27.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:26 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:27.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:26 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:58:27.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:26 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:27 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:58:28.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:27 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:27 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:58:28.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:27 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:58:28.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:27 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:27 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:58:28.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:27 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:58:28.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:27 smithi146 ceph-mon[36454]: pgmap v267: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-02-17T12:58:28.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:27 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:27 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:27 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:27 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:58:28.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:27 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:27 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:58:28.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:27 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:58:28.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:27 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:27 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:58:28.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:27 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:58:28.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:27 smithi018 ceph-mon[32625]: pgmap v267: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-02-17T12:58:28.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:27 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:27 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:27 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 440779 5.26 MB/sec execute 343 sec latency 13.103 ms 2024-02-17T12:58:28.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:27 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:58:28.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:27 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:27 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:58:28.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:27 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:58:28.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:27 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:27 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:58:28.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:27 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:58:28.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:27 smithi119 ceph-mon[36188]: pgmap v267: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-02-17T12:58:28.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:27 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:27 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:28.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:27 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:58:29.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 442615 5.28 MB/sec execute 344 sec latency 64.628 ms 2024-02-17T12:58:30.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:29 smithi146 ceph-mon[36454]: pgmap v268: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 234 op/s 2024-02-17T12:58:30.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:29 smithi018 ceph-mon[32625]: pgmap v268: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 234 op/s 2024-02-17T12:58:30.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 444127 5.28 MB/sec execute 345 sec latency 18.456 ms 2024-02-17T12:58:30.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:29 smithi119 ceph-mon[36188]: pgmap v268: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 234 op/s 2024-02-17T12:58:31.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 445096 5.28 MB/sec execute 346 sec latency 341.701 ms 2024-02-17T12:58:32.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:31 smithi146 ceph-mon[36454]: pgmap v269: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-02-17T12:58:32.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:31 smithi018 ceph-mon[32625]: pgmap v269: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-02-17T12:58:32.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 445096 5.27 MB/sec execute 347 sec latency 1341.779 ms 2024-02-17T12:58:32.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:31 smithi119 ceph-mon[36188]: pgmap v269: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-02-17T12:58:33.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 446163 5.27 MB/sec execute 348 sec latency 1907.052 ms 2024-02-17T12:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.599+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/2423513274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ed01013c0 msgr2=0x7f1ed01017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.599+0000 7f1ed4fb6640 1 --2- 172.21.15.18:0/2423513274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ed01013c0 0x7f1ed01017a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009930 tx=0x7f1ec002bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.599+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/2423513274 shutdown_connections 2024-02-17T12:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.599+0000 7f1ed4fb6640 1 --2- 172.21.15.18:0/2423513274 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ed0106490 0x7f1ed010e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.599+0000 7f1ed4fb6640 1 --2- 172.21.15.18:0/2423513274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ed0101d70 0x7f1ed0105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.599+0000 7f1ed4fb6640 1 --2- 172.21.15.18:0/2423513274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ed01013c0 0x7f1ed01017a0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.599+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/2423513274 >> 172.21.15.18:0/2423513274 conn(0x7f1ed00fcc60 msgr2=0x7f1ed00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.599+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/2423513274 shutdown_connections 2024-02-17T12:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.599+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/2423513274 wait complete. 2024-02-17T12:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ed4fb6640 1 Processor -- start 2024-02-17T12:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ed4fb6640 1 -- start start 2024-02-17T12:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ed4fb6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ed01013c0 0x7f1ed0073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ed4fb6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ed0106490 0x7f1ed00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ed4fb6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ed0073f00 0x7f1ed01a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ed4fb6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed01a5f20 con 0x7f1ed0073f00 2024-02-17T12:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ed4fb6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed01a6090 con 0x7f1ed0106490 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ed4fb6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed01a6250 con 0x7f1ed01013c0 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1eced9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ed0073f00 0x7f1ed01a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ecdd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ed0106490 0x7f1ed00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1eced9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ed0073f00 0x7f1ed01a5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51574/0 (socket says 172.21.15.18:51574) 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ece59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ed01013c0 0x7f1ed0073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1eced9d640 1 -- 172.21.15.18:0/1575177544 learned_addr learned my addr 172.21.15.18:0/1575177544 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ecdd9b640 1 -- 172.21.15.18:0/1575177544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ed01013c0 msgr2=0x7f1ed0073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/1575177544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ed01013c0 0x7f1ed0073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ecdd9b640 1 -- 172.21.15.18:0/1575177544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ed0073f00 msgr2=0x7f1ed01a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/1575177544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ed0073f00 0x7f1ed01a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.600+0000 7f1ecdd9b640 1 -- 172.21.15.18:0/1575177544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec0009560 con 0x7f1ed0106490 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.601+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/1575177544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ed0106490 0x7f1ed00739c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400e9c0 tx=0x7f1ec400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.601+0000 7f1ebf7fe640 1 -- 172.21.15.18:0/1575177544 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec400ccc0 con 0x7f1ed0106490 2024-02-17T12:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.601+0000 7f1ebf7fe640 1 -- 172.21.15.18:0/1575177544 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ec400ce20 con 0x7f1ed0106490 2024-02-17T12:58:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.601+0000 7f1ebf7fe640 1 -- 172.21.15.18:0/1575177544 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4018880 con 0x7f1ed0106490 2024-02-17T12:58:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.601+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/1575177544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed01a6530 con 0x7f1ed0106490 2024-02-17T12:58:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.601+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/1575177544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed01a69f0 con 0x7f1ed0106490 2024-02-17T12:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.603+0000 7f1ebf7fe640 1 -- 172.21.15.18:0/1575177544 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1ec40189e0 con 0x7f1ed0106490 2024-02-17T12:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.603+0000 7f1ed7b84740 1 Processor -- start 2024-02-17T12:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.603+0000 7f1ebf7fe640 1 --2- 172.21.15.18:0/1575177544 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1eb80a2380 0x7f1eb80a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1eced9d640 1 --2- 172.21.15.18:0/1575177544 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1eb80a2380 0x7f1eb80a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ebf7fe640 1 -- 172.21.15.18:0/1575177544 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1ec4014070 con 0x7f1ed0106490 2024-02-17T12:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ed7b84740 1 -- start start 2024-02-17T12:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ed7b84740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ed7b84740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee2d60 0x5598c2ee31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ed7b84740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2eee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ed7b84740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c2e519f0 con 0x5598c2ee3ad0 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ed7b84740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c2e4ef20 con 0x5598c2eec2e0 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ed7b84740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c2e4a700 con 0x5598c2ee2d60 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ece59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ece59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51580/0 (socket says 172.21.15.18:51580) 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ece59c640 1 -- 172.21.15.18:0/1503585631 learned_addr learned my addr 172.21.15.18:0/1503585631 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1eced9d640 1 --2- 172.21.15.18:0/1503585631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2eee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/1503585631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee2d60 0x5598c2ee31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1eced9d640 1 --2- 172.21.15.18:0/1575177544 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1eb80a2380 0x7f1eb80a4840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80049d0 tx=0x7f1ec8009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ece59c640 1 -- 172.21.15.18:0/1503585631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee2d60 msgr2=0x5598c2ee31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ece59c640 1 --2- 172.21.15.18:0/1503585631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee2d60 0x5598c2ee31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ece59c640 1 -- 172.21.15.18:0/1503585631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 msgr2=0x5598c2eee6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ece59c640 1 --2- 172.21.15.18:0/1503585631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2eee6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ece59c640 1 -- 172.21.15.18:0/1503585631 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598c2de07b0 con 0x5598c2ee3ad0 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/1503585631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee2d60 0x5598c2ee31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1eced9d640 1 --2- 172.21.15.18:0/1503585631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2eee6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.604+0000 7f1ece59c640 1 --2- 172.21.15.18:0/1503585631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee2820 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0036040 tx=0x7f1ec002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcff35253a0e1b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.605+0000 7f1ebcff9640 1 -- 172.21.15.18:0/1503585631 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0004800 con 0x5598c2ee3ad0 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.605+0000 7f1ebcff9640 1 -- 172.21.15.18:0/1503585631 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ec0004960 con 0x5598c2ee3ad0 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.605+0000 7f1ebcff9640 1 -- 172.21.15.18:0/1503585631 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0005120 con 0x5598c2ee3ad0 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.605+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1503585631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee3ad0 msgr2=0x5598c2ee2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.605+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/1503585631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee2820 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0036040 tx=0x7f1ec002c5d0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.605+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1503585631 shutdown_connections 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.605+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/1503585631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2eee6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.605+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/1503585631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee2d60 0x5598c2ee31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.605+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/1503585631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee2820 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.605+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1503585631 >> 172.21.15.18:0/1503585631 conn(0x5598c2eaa550 msgr2=0x5598c2edda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1503585631 shutdown_connections 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1503585631 wait complete. 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ed7b84740 1 -- start start 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ed7b84740 1 Processor -- start 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ed7b84740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2ee92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ed7b84740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ed7b84740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2eec2e0 0x5598c2ee9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ed7b84740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c2defbf0 con 0x5598c2ee2d60 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ed7b84740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c2b1e250 con 0x5598c2ee3ad0 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ed7b84740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c2dd0200 con 0x5598c2eec2e0 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ed7b84740 10 client.0 _ll_register_callbacks cb 0x5598c2e3ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ece59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2ee92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ece59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2ee92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51596/0 (socket says 172.21.15.18:51596) 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ece59c640 1 -- 172.21.15.18:0/2918708254 learned_addr learned my addr 172.21.15.18:0/2918708254 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.606+0000 7f1eced9d640 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2eec2e0 0x5598c2ee9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1ece59c640 1 -- 172.21.15.18:0/2918708254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2eec2e0 msgr2=0x5598c2ee9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1ece59c640 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2eec2e0 0x5598c2ee9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1ece59c640 1 -- 172.21.15.18:0/2918708254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2ee3ad0 msgr2=0x5598c2ee9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1ece59c640 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1ece59c640 1 -- 172.21.15.18:0/2918708254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed010e870 con 0x5598c2ee2d60 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1eced9d640 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2eec2e0 0x5598c2ee9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1ece59c640 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2ee92f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0005090 tx=0x7f1ec00050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ef45c605dfcea5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1eb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1eb5ffb640 1 -- 172.21.15.18:0/2918708254 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0004800 con 0x5598c2ee2d60 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1eb5ffb640 1 -- 172.21.15.18:0/2918708254 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1ec003f440 con 0x5598c2ee2d60 2024-02-17T12:58:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1ed7b84740 1 -- 172.21.15.18:0/2918708254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598c2b4a600 con 0x5598c2ee2d60 2024-02-17T12:58:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1eb5ffb640 1 -- 172.21.15.18:0/2918708254 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ec0004ad0 con 0x5598c2ee2d60 2024-02-17T12:58:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1ed7b84740 10 client.18705 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1ed7b84740 1 -- 172.21.15.18:0/2918708254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5598c2e48d80 con 0x5598c2ee2d60 2024-02-17T12:58:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.607+0000 7f1eb5ffb640 1 -- 172.21.15.18:0/2918708254 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5598c2b4a600 con 0x5598c2ee2d60 2024-02-17T12:58:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.608+0000 7f1eb5ffb640 1 -- 172.21.15.18:0/2918708254 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1ec003c050 con 0x5598c2ee2d60 2024-02-17T12:58:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.608+0000 7f1ed7b84740 10 client.18705 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.608+0000 7f1ed7b84740 10 client.18705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.608+0000 7f1ed7b84740 20 client.18705 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.608+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5598c2efc1c0 0x5598c2efe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.608+0000 7f1ed7b84740 4 client.18705 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.608+0000 7f1ed7b84740 1 -- 172.21.15.18:0/2918708254 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598c2b9b1b0 con 0x5598c2efc1c0 2024-02-17T12:58:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.608+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5598c2efc1c0 0x5598c2efe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.608+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5598c2efc1c0 0x5598c2efe5e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28adbb4ca910f6a3 server_cookie=b27b7a6647569966 in_seq=0 out_seq=0 2024-02-17T12:58:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.608+0000 7f1eb5ffb640 10 client.18705 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1eb5ffb640 1 -- 172.21.15.18:0/2918708254 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5598c2b9b1b0 con 0x5598c2efc1c0 2024-02-17T12:58:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1eb5ffb640 10 client.18705 handle_command_reply: tid=0 2024-02-17T12:58:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1ed7b84740 1 client.18705 shutdown 2024-02-17T12:58:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1ed7b84740 1 -- 172.21.15.18:0/2918708254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 msgr2=0x5598c2ee92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2ee92f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0005090 tx=0x7f1ec00050c0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1ed7b84740 1 -- 172.21.15.18:0/2918708254 shutdown_connections 2024-02-17T12:58:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5598c2efc1c0 0x5598c2efe5e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2eec2e0 0x5598c2ee9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/2918708254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2ee92f0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1ed7b84740 1 -- 172.21.15.18:0/2918708254 >> 172.21.15.18:0/2918708254 conn(0x5598c2eaa550 msgr2=0x5598c2edf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1eb5ffb640 0 client.18705 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1ed7b84740 1 -- 172.21.15.18:0/2918708254 shutdown_connections 2024-02-17T12:58:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1ed7b84740 1 -- 172.21.15.18:0/2918708254 wait complete. 2024-02-17T12:58:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.609+0000 7f1ed7b84740 20 client.18705 trim_cache size 0 max 16384 2024-02-17T12:58:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.622+0000 7f1ed7b84740 1 Processor -- start 2024-02-17T12:58:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.622+0000 7f1ed7b84740 1 -- start start 2024-02-17T12:58:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ed7b84740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2f02f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ed7b84740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ed7b84740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2ee9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ed7b84740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c2edfda0 con 0x5598c2ee2d60 2024-02-17T12:58:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ed7b84740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c2edea50 con 0x5598c2eec2e0 2024-02-17T12:58:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ed7b84740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c2ee2030 con 0x5598c2ee3ad0 2024-02-17T12:58:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ece59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2f02f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ecdd9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ecdd9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:52770/0 (socket says 172.21.15.18:52770) 2024-02-17T12:58:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ecdd9b640 1 -- 172.21.15.18:0/3363614905 learned_addr learned my addr 172.21.15.18:0/3363614905 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ece59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2f02f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51598/0 (socket says 172.21.15.18:51598) 2024-02-17T12:58:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1eced9d640 1 --2- 172.21.15.18:0/3363614905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2ee9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ecdd9b640 1 -- 172.21.15.18:0/3363614905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 msgr2=0x5598c2ee9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/3363614905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2ee9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ecdd9b640 1 -- 172.21.15.18:0/3363614905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 msgr2=0x5598c2f02f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/3363614905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2f02f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ecdd9b640 1 -- 172.21.15.18:0/3363614905 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598c2de57b0 con 0x5598c2ee3ad0 2024-02-17T12:58:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ece59c640 1 --2- 172.21.15.18:0/3363614905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2f02f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.623+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/3363614905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee9840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40cbb40 tx=0x7f1ec400cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ff3822c5802d5fa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.624+0000 7f1eb7fff640 1 -- 172.21.15.18:0/3363614905 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40990e0 con 0x5598c2ee3ad0 2024-02-17T12:58:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.624+0000 7f1eb7fff640 1 -- 172.21.15.18:0/3363614905 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ec4010aa0 con 0x5598c2ee3ad0 2024-02-17T12:58:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.624+0000 7f1eb7fff640 1 -- 172.21.15.18:0/3363614905 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec409a490 con 0x5598c2ee3ad0 2024-02-17T12:58:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.624+0000 7f1ed7b84740 1 -- 172.21.15.18:0/3363614905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee3ad0 msgr2=0x5598c2ee9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.624+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/3363614905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee9840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40cbb40 tx=0x7f1ec400cf90 comp rx=0 tx=0).stop 2024-02-17T12:58:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.624+0000 7f1ed7b84740 1 -- 172.21.15.18:0/3363614905 shutdown_connections 2024-02-17T12:58:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.624+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/3363614905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2ee9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.624+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/3363614905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee3ad0 0x5598c2ee9840 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.624+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/3363614905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2f02f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.624+0000 7f1ed7b84740 1 -- 172.21.15.18:0/3363614905 >> 172.21.15.18:0/3363614905 conn(0x5598c2eaa550 msgr2=0x5598c2ee46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.624+0000 7f1ed7b84740 1 -- 172.21.15.18:0/3363614905 shutdown_connections 2024-02-17T12:58:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ed7b84740 1 -- 172.21.15.18:0/3363614905 wait complete. 2024-02-17T12:58:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ed7b84740 1 -- start start 2024-02-17T12:58:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ed7b84740 1 Processor -- start 2024-02-17T12:58:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ed7b84740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2ef3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ed7b84740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee3ad0 0x5598c2ef35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ed7b84740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2ef1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ed7b84740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c2ef1cd0 con 0x5598c2ee2d60 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ed7b84740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c2ef1e40 con 0x5598c2eec2e0 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ed7b84740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c2ef2000 con 0x5598c2ee3ad0 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ed7b84740 10 client.0 _ll_register_callbacks cb 0x7f1ec80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ece59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2ef3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ece59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2ef3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51602/0 (socket says 172.21.15.18:51602) 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ece59c640 1 -- 172.21.15.18:0/1548083471 learned_addr learned my addr 172.21.15.18:0/1548083471 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1eced9d640 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2ef1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.625+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee3ad0 0x5598c2ef35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ece59c640 1 -- 172.21.15.18:0/1548083471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee3ad0 msgr2=0x5598c2ef35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ece59c640 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee3ad0 0x5598c2ef35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ece59c640 1 -- 172.21.15.18:0/1548083471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 msgr2=0x5598c2ef1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ece59c640 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2ef1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ece59c640 1 -- 172.21.15.18:0/1548083471 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598c2e48d80 con 0x5598c2ee2d60 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1eced9d640 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2ef1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ece59c640 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2ef3060 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0005090 tx=0x7f1ec00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0567915d90c98c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1eb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1eb5ffb640 1 -- 172.21.15.18:0/1548083471 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0004ad0 con 0x5598c2ee2d60 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1eb5ffb640 1 -- 172.21.15.18:0/1548083471 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1ec0000e40 con 0x5598c2ee2d60 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1548083471 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ec0004960 con 0x5598c2ee2d60 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1eb5ffb640 1 -- 172.21.15.18:0/1548083471 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ec0005130 con 0x5598c2ee2d60 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ed7b84740 10 client.18711 fetch_fsmap learned FSMap version 44 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1548083471 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f1ed01a69f0 con 0x5598c2ee2d60 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1eb5ffb640 1 -- 172.21.15.18:0/1548083471 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ec0004960 con 0x5598c2ee2d60 2024-02-17T12:58:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1eb5ffb640 1 -- 172.21.15.18:0/1548083471 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1ec003c070 con 0x5598c2ee2d60 2024-02-17T12:58:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ed7b84740 10 client.18711 fetch_fsmap finished waiting for FSMap version 44 2024-02-17T12:58:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ed7b84740 10 client.18711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ed7b84740 20 client.18711 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5598c2f53600 0x5598c2f55a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ed7b84740 4 client.18711 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:58:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.626+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1548083471 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5598c2b87550 con 0x5598c2f53600 2024-02-17T12:58:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.627+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5598c2f53600 0x5598c2f55a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.627+0000 7f1ecdd9b640 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5598c2f53600 0x5598c2f55a20 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3406ac96743fe16 server_cookie=79e88ba13cadc67a in_seq=0 out_seq=0 2024-02-17T12:58:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.627+0000 7f1eb5ffb640 10 client.18711 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.660+0000 7f1eb5ffb640 1 -- 172.21.15.18:0/1548083471 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5598c2b87550 con 0x5598c2f53600 2024-02-17T12:58:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.660+0000 7f1eb5ffb640 10 client.18711 handle_command_reply: tid=0 2024-02-17T12:58:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.660+0000 7f1ed7b84740 1 client.18711 shutdown 2024-02-17T12:58:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.660+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1548083471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 msgr2=0x5598c2ef3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.660+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2ef3060 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0005090 tx=0x7f1ec00048f0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.660+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1548083471 shutdown_connections 2024-02-17T12:58:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.660+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5598c2f53600 0x5598c2f55a20 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.660+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5598c2eec2e0 0x5598c2ef1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.660+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5598c2ee3ad0 0x5598c2ef35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.660+0000 7f1ed7b84740 1 --2- 172.21.15.18:0/1548083471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5598c2ee2d60 0x5598c2ef3060 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.660+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1548083471 >> 172.21.15.18:0/1548083471 conn(0x5598c2eaa550 msgr2=0x5598c2f02f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.660+0000 7f1eb5ffb640 0 client.18711 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1548083471 shutdown_connections 2024-02-17T12:58:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed7b84740 1 -- 172.21.15.18:0/1548083471 wait complete. 2024-02-17T12:58:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed7b84740 20 client.18711 trim_cache size 0 max 16384 2024-02-17T12:58:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:58:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:58:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:58:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:58:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/1575177544 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1eb80a2380 msgr2=0x7f1eb80a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed4fb6640 1 --2- 172.21.15.18:0/1575177544 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1eb80a2380 0x7f1eb80a4840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80049d0 tx=0x7f1ec8009500 comp rx=0 tx=0).stop 2024-02-17T12:58:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/1575177544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ed0106490 msgr2=0x7f1ed00739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed4fb6640 1 --2- 172.21.15.18:0/1575177544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ed0106490 0x7f1ed00739c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400e9c0 tx=0x7f1ec400ee80 comp rx=0 tx=0).stop 2024-02-17T12:58:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/1575177544 shutdown_connections 2024-02-17T12:58:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed4fb6640 1 --2- 172.21.15.18:0/1575177544 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1eb80a2380 0x7f1eb80a4840 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed4fb6640 1 --2- 172.21.15.18:0/1575177544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ed0073f00 0x7f1ed01a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed4fb6640 1 --2- 172.21.15.18:0/1575177544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ed0106490 0x7f1ed00739c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed4fb6640 1 --2- 172.21.15.18:0/1575177544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ed01013c0 0x7f1ed0073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/1575177544 >> 172.21.15.18:0/1575177544 conn(0x7f1ed00fcc60 msgr2=0x7f1ed00fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.661+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/1575177544 shutdown_connections 2024-02-17T12:58:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:33.662+0000 7f1ed4fb6640 1 -- 172.21.15.18:0/1575177544 wait complete. 2024-02-17T12:58:34.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:33 smithi146 ceph-mon[36454]: pgmap v270: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-02-17T12:58:34.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:33 smithi018 ceph-mon[32625]: pgmap v270: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-02-17T12:58:34.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 447668 5.28 MB/sec execute 349 sec latency 22.976 ms 2024-02-17T12:58:34.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:33 smithi119 ceph-mon[36188]: pgmap v270: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-02-17T12:58:35.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 448638 5.28 MB/sec execute 350 sec latency 354.525 ms 2024-02-17T12:58:36.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:35 smithi146 ceph-mon[36454]: pgmap v271: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 270 op/s 2024-02-17T12:58:36.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:35 smithi018 ceph-mon[32625]: pgmap v271: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 270 op/s 2024-02-17T12:58:36.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 448638 5.26 MB/sec execute 351 sec latency 1354.612 ms 2024-02-17T12:58:36.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:35 smithi119 ceph-mon[36188]: pgmap v271: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 270 op/s 2024-02-17T12:58:37.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 448638 5.25 MB/sec execute 352 sec latency 2354.711 ms 2024-02-17T12:58:38.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:37 smithi146 ceph-mon[36454]: pgmap v272: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-02-17T12:58:38.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:37 smithi018 ceph-mon[32625]: pgmap v272: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-02-17T12:58:38.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 448638 5.23 MB/sec execute 353 sec latency 3354.784 ms 2024-02-17T12:58:38.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:37 smithi119 ceph-mon[36188]: pgmap v272: 97 pgs: 97 active+clean; 156 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-02-17T12:58:39.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 450599 5.25 MB/sec execute 354 sec latency 3546.489 ms 2024-02-17T12:58:40.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:39 smithi146 ceph-mon[36454]: pgmap v273: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-02-17T12:58:40.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:39 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:58:40.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:39 smithi018 ceph-mon[32625]: pgmap v273: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-02-17T12:58:40.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:39 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:58:40.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 451884 5.26 MB/sec execute 355 sec latency 15.033 ms 2024-02-17T12:58:40.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:39 smithi119 ceph-mon[36188]: pgmap v273: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-02-17T12:58:40.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:39 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:58:41.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:40 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:58:41.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:40 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:58:41.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:40 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:58:41.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:40 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:58:41.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 452187 5.24 MB/sec execute 356 sec latency 640.973 ms 2024-02-17T12:58:41.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:40 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:58:41.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:40 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:58:41.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:40 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:58:41.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:40 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:58:41.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:40 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:58:41.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:40 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:58:41.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:40 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:58:41.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:40 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:58:42.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:41 smithi018 ceph-mon[32625]: pgmap v274: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-02-17T12:58:42.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 454035 5.26 MB/sec execute 357 sec latency 857.695 ms 2024-02-17T12:58:42.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:41 smithi119 ceph-mon[36188]: pgmap v274: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-02-17T12:58:42.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:41 smithi146 ceph-mon[36454]: pgmap v274: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-02-17T12:58:43.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 455225 5.26 MB/sec execute 358 sec latency 14.990 ms 2024-02-17T12:58:44.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:43 smithi018 ceph-mon[32625]: pgmap v275: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-02-17T12:58:44.287 INFO:tasks.workunit.client.0.smithi018.stdout: 1 456642 5.26 MB/sec execute 359 sec latency 166.953 ms 2024-02-17T12:58:44.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:43 smithi119 ceph-mon[36188]: pgmap v275: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-02-17T12:58:44.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:43 smithi146 ceph-mon[36454]: pgmap v275: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-02-17T12:58:45.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:44 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:58:45.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 458180 5.28 MB/sec execute 360 sec latency 16.556 ms 2024-02-17T12:58:45.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:44 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:58:45.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:44 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:58:46.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:45 smithi018 ceph-mon[32625]: pgmap v276: 97 pgs: 97 active+clean; 162 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-02-17T12:58:46.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 459238 5.32 MB/sec execute 361 sec latency 64.472 ms 2024-02-17T12:58:46.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:45 smithi119 ceph-mon[36188]: pgmap v276: 97 pgs: 97 active+clean; 162 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-02-17T12:58:46.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:45 smithi146 ceph-mon[36454]: pgmap v276: 97 pgs: 97 active+clean; 162 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-02-17T12:58:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.752+0000 7f85b9b8c640 1 -- 172.21.15.18:0/905847180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85b4100040 msgr2=0x7f85b410e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.752+0000 7f85b9b8c640 1 --2- 172.21.15.18:0/905847180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85b4100040 0x7f85b410e010 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009930 tx=0x7f85ac02bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.752+0000 7f85b9b8c640 1 -- 172.21.15.18:0/905847180 shutdown_connections 2024-02-17T12:58:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.752+0000 7f85b9b8c640 1 --2- 172.21.15.18:0/905847180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85b4100040 0x7f85b410e010 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.752+0000 7f85b9b8c640 1 --2- 172.21.15.18:0/905847180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85b40ff680 0x7f85b40ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.752+0000 7f85b9b8c640 1 --2- 172.21.15.18:0/905847180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85b4106160 0x7f85b4106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.752+0000 7f85b9b8c640 1 -- 172.21.15.18:0/905847180 >> 172.21.15.18:0/905847180 conn(0x7f85b4076610 msgr2=0x7f85b40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.752+0000 7f85b9b8c640 1 -- 172.21.15.18:0/905847180 shutdown_connections 2024-02-17T12:58:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.752+0000 7f85b9b8c640 1 -- 172.21.15.18:0/905847180 wait complete. 2024-02-17T12:58:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b9b8c640 1 Processor -- start 2024-02-17T12:58:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b9b8c640 1 -- start start 2024-02-17T12:58:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b9b8c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85b40ff680 0x7f85b4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b9b8c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85b4100040 0x7f85b4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b9b8c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85b4106160 0x7f85b419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b9b8c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b419c5c0 con 0x7f85b4106160 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b9b8c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b419c730 con 0x7f85b4100040 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b9b8c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b419c8f0 con 0x7f85b40ff680 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85b4106160 0x7f85b419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85b4106160 0x7f85b419bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56174/0 (socket says 172.21.15.18:56174) 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b3fff640 1 -- 172.21.15.18:0/1556684596 learned_addr learned my addr 172.21.15.18:0/1556684596 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b2ffd640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85b4100040 0x7f85b4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.753+0000 7f85b37fe640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85b40ff680 0x7f85b4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.754+0000 7f85b3fff640 1 -- 172.21.15.18:0/1556684596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85b40ff680 msgr2=0x7f85b4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.754+0000 7f85b3fff640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85b40ff680 0x7f85b4197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.754+0000 7f85b3fff640 1 -- 172.21.15.18:0/1556684596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85b4100040 msgr2=0x7f85b4197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.754+0000 7f85b3fff640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85b4100040 0x7f85b4197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.754+0000 7f85b3fff640 1 -- 172.21.15.18:0/1556684596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac009560 con 0x7f85b4106160 2024-02-17T12:58:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.754+0000 7f85b2ffd640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85b4100040 0x7f85b4197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.754+0000 7f85b3fff640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85b4106160 0x7f85b419bfc0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f85ac005940 tx=0x7f85ac02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.754+0000 7f85b0ff9640 1 -- 172.21.15.18:0/1556684596 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac004350 con 0x7f85b4106160 2024-02-17T12:58:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.754+0000 7f85b0ff9640 1 -- 172.21.15.18:0/1556684596 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85ac0044b0 con 0x7f85b4106160 2024-02-17T12:58:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.754+0000 7f85b0ff9640 1 -- 172.21.15.18:0/1556684596 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac003cc0 con 0x7f85b4106160 2024-02-17T12:58:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.754+0000 7f85b9b8c640 1 -- 172.21.15.18:0/1556684596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b419cb70 con 0x7f85b4106160 2024-02-17T12:58:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.754+0000 7f85b9b8c640 1 -- 172.21.15.18:0/1556684596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b41a4450 con 0x7f85b4106160 2024-02-17T12:58:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.756+0000 7f85b0ff9640 1 -- 172.21.15.18:0/1556684596 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f85ac005580 con 0x7f85b4106160 2024-02-17T12:58:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85bc767740 1 Processor -- start 2024-02-17T12:58:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85bc767740 1 -- start start 2024-02-17T12:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b0ff9640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f859c0a22d0 0x7f859c0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b2ffd640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f859c0a22d0 0x7f859c0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b0ff9640 1 -- 172.21.15.18:0/1556684596 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f85ac0f0060 con 0x7f85b4106160 2024-02-17T12:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85bc767740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 0x55616cda5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85bc767740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 0x55616cda61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85bc767740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdb16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85bc767740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616cd149f0 con 0x55616cda6ad0 2024-02-17T12:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85bc767740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616cd11f20 con 0x55616cdaf2e0 2024-02-17T12:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85bc767740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616cd0d700 con 0x55616cda5d60 2024-02-17T12:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 0x55616cda5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 0x55616cda5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56182/0 (socket says 172.21.15.18:56182) 2024-02-17T12:58:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b37fe640 1 -- 172.21.15.18:0/3365499253 learned_addr learned my addr 172.21.15.18:0/3365499253 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b2ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 0x55616cda61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b3fff640 1 --2- 172.21.15.18:0/3365499253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdb16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b37fe640 1 -- 172.21.15.18:0/3365499253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 msgr2=0x55616cda61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b37fe640 1 --2- 172.21.15.18:0/3365499253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 0x55616cda61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b37fe640 1 -- 172.21.15.18:0/3365499253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 msgr2=0x55616cdb16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b37fe640 1 --2- 172.21.15.18:0/3365499253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdb16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b37fe640 1 -- 172.21.15.18:0/3365499253 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55616cca37b0 con 0x55616cda6ad0 2024-02-17T12:58:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.757+0000 7f85b2ffd640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f859c0a22d0 0x7f859c0a4790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009fd0 tx=0x7f85a80136e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.758+0000 7f85b37fe640 1 --2- 172.21.15.18:0/3365499253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 0x55616cda5820 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f85a400bc50 tx=0x7f85a40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba5a42260dfc98b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.758+0000 7f8599ffb640 1 -- 172.21.15.18:0/3365499253 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4005020 con 0x55616cda6ad0 2024-02-17T12:58:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.758+0000 7f8599ffb640 1 -- 172.21.15.18:0/3365499253 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85a4015070 con 0x55616cda6ad0 2024-02-17T12:58:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.758+0000 7f8599ffb640 1 -- 172.21.15.18:0/3365499253 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a40116a0 con 0x55616cda6ad0 2024-02-17T12:58:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.758+0000 7f85bc767740 1 -- 172.21.15.18:0/3365499253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 msgr2=0x55616cda5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.758+0000 7f85bc767740 1 --2- 172.21.15.18:0/3365499253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 0x55616cda5820 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f85a400bc50 tx=0x7f85a40107a0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 -- 172.21.15.18:0/3365499253 shutdown_connections 2024-02-17T12:58:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 --2- 172.21.15.18:0/3365499253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdb16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 --2- 172.21.15.18:0/3365499253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 0x55616cda61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 --2- 172.21.15.18:0/3365499253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 0x55616cda5820 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 -- 172.21.15.18:0/3365499253 >> 172.21.15.18:0/3365499253 conn(0x55616cd6d550 msgr2=0x55616cda0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 -- 172.21.15.18:0/3365499253 shutdown_connections 2024-02-17T12:58:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 -- 172.21.15.18:0/3365499253 wait complete. 2024-02-17T12:58:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 -- start start 2024-02-17T12:58:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 Processor -- start 2024-02-17T12:58:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 0x55616cdac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 0x55616cdac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdacd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616ccb2bf0 con 0x55616cda6ad0 2024-02-17T12:58:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616c9e1250 con 0x55616cdaf2e0 2024-02-17T12:58:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616cc93200 con 0x55616cda5d60 2024-02-17T12:58:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85bc767740 10 client.0 _ll_register_callbacks cb 0x55616cd01d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85b2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 0x55616cdac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.759+0000 7f85b2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 0x55616cdac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56198/0 (socket says 172.21.15.18:56198) 2024-02-17T12:58:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85b2ffd640 1 -- 172.21.15.18:0/2008385668 learned_addr learned my addr 172.21.15.18:0/2008385668 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85b3fff640 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdacd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85b37fe640 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 0x55616cdac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85b2ffd640 1 -- 172.21.15.18:0/2008385668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 msgr2=0x55616cdac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85b2ffd640 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 0x55616cdac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85b2ffd640 1 -- 172.21.15.18:0/2008385668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 msgr2=0x55616cdacd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85b2ffd640 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdacd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85b2ffd640 1 -- 172.21.15.18:0/2008385668 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a400b900 con 0x55616cda6ad0 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85b3fff640 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdacd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85b2ffd640 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 0x55616cdac800 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f85a8013340 tx=0x7f85a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74284d63334208a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f8592ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f8592ffd640 1 -- 172.21.15.18:0/2008385668 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8010640 con 0x55616cda6ad0 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f8592ffd640 1 -- 172.21.15.18:0/2008385668 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f85a80107a0 con 0x55616cda6ad0 2024-02-17T12:58:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85bc767740 1 -- 172.21.15.18:0/2008385668 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55616ca0d600 con 0x55616cda6ad0 2024-02-17T12:58:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f8592ffd640 1 -- 172.21.15.18:0/2008385668 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85a8010960 con 0x55616cda6ad0 2024-02-17T12:58:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85bc767740 10 client.18723 fetch_fsmap learned FSMap version 45 2024-02-17T12:58:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f85bc767740 1 -- 172.21.15.18:0/2008385668 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55616cd0bd80 con 0x55616cda6ad0 2024-02-17T12:58:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.760+0000 7f8592ffd640 1 -- 172.21.15.18:0/2008385668 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55616ca0d600 con 0x55616cda6ad0 2024-02-17T12:58:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.761+0000 7f8592ffd640 1 -- 172.21.15.18:0/2008385668 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 45) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f85a8032c00 con 0x55616cda6ad0 2024-02-17T12:58:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.761+0000 7f85bc767740 10 client.18723 fetch_fsmap finished waiting for FSMap version 45 2024-02-17T12:58:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.761+0000 7f85bc767740 10 client.18723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.761+0000 7f85bc767740 20 client.18723 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.761+0000 7f85bc767740 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55616cdbf650 0x55616cdc1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.761+0000 7f85bc767740 4 client.18723 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.761+0000 7f85bc767740 1 -- 172.21.15.18:0/2008385668 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55616ca5e1b0 con 0x55616cdbf650 2024-02-17T12:58:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.761+0000 7f85b37fe640 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55616cdbf650 0x55616cdc1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.761+0000 7f85b37fe640 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55616cdbf650 0x55616cdc1a70 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b92090052d28ba3 server_cookie=2f400941ed4026d8 in_seq=0 out_seq=0 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.761+0000 7f8592ffd640 10 client.18723 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f8592ffd640 1 -- 172.21.15.18:0/2008385668 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55616ca5e1b0 con 0x55616cdbf650 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f8592ffd640 10 client.18723 handle_command_reply: tid=0 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f85bc767740 1 client.18723 shutdown 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f85bc767740 1 -- 172.21.15.18:0/2008385668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 msgr2=0x55616cdac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f85bc767740 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 0x55616cdac800 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f85a8013340 tx=0x7f85a8004ab0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f85bc767740 1 -- 172.21.15.18:0/2008385668 shutdown_connections 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f85bc767740 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55616cdbf650 0x55616cdc1a70 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f85bc767740 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdacd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f85bc767740 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda6ad0 0x55616cdac800 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f85bc767740 1 --2- 172.21.15.18:0/2008385668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 0x55616cdac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f8592ffd640 0 client.18723 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f85bc767740 1 -- 172.21.15.18:0/2008385668 >> 172.21.15.18:0/2008385668 conn(0x55616cd6d550 msgr2=0x55616cda27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f85bc767740 1 -- 172.21.15.18:0/2008385668 shutdown_connections 2024-02-17T12:58:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.762+0000 7f85bc767740 1 -- 172.21.15.18:0/2008385668 wait complete. 2024-02-17T12:58:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.763+0000 7f85bc767740 20 client.18723 trim_cache size 0 max 16384 2024-02-17T12:58:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.776+0000 7f85bc767740 1 Processor -- start 2024-02-17T12:58:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.776+0000 7f85bc767740 1 -- start start 2024-02-17T12:58:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.776+0000 7f85bc767740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 0x55616cdac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.776+0000 7f85bc767740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cda6ad0 0x55616cdacd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.776+0000 7f85bc767740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cdaf2e0 0x55616cdaab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.776+0000 7f85bc767740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616ca5e1b0 con 0x55616cdaf2e0 2024-02-17T12:58:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.776+0000 7f85bc767740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616ca0d600 con 0x55616cda6ad0 2024-02-17T12:58:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.776+0000 7f85bc767740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616cda2da0 con 0x55616cda5d60 2024-02-17T12:58:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85b3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cdaf2e0 0x55616cdaab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85b3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cdaf2e0 0x55616cdaab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56202/0 (socket says 172.21.15.18:56202) 2024-02-17T12:58:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85b37fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 0x55616cdac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85b3fff640 1 -- 172.21.15.18:0/3160644493 learned_addr learned my addr 172.21.15.18:0/3160644493 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85b2ffd640 1 --2- 172.21.15.18:0/3160644493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cda6ad0 0x55616cdacd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85b3fff640 1 -- 172.21.15.18:0/3160644493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 msgr2=0x55616cdac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85b3fff640 1 --2- 172.21.15.18:0/3160644493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 0x55616cdac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85b3fff640 1 -- 172.21.15.18:0/3160644493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cda6ad0 msgr2=0x55616cdacd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85b3fff640 1 --2- 172.21.15.18:0/3160644493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cda6ad0 0x55616cdacd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85b3fff640 1 -- 172.21.15.18:0/3160644493 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55616cca87b0 con 0x55616cdaf2e0 2024-02-17T12:58:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85b3fff640 1 --2- 172.21.15.18:0/3160644493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cdaf2e0 0x55616cdaab90 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0eddd0 tx=0x7f85ac04a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f207adfbe4496be server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85997fa640 1 -- 172.21.15.18:0/3160644493 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0bd280 con 0x55616cdaf2e0 2024-02-17T12:58:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85997fa640 1 -- 172.21.15.18:0/3160644493 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85ac0bcb80 con 0x55616cdaf2e0 2024-02-17T12:58:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.777+0000 7f85997fa640 1 -- 172.21.15.18:0/3160644493 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0c50f0 con 0x55616cdaf2e0 2024-02-17T12:58:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.778+0000 7f85bc767740 1 -- 172.21.15.18:0/3160644493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cdaf2e0 msgr2=0x55616cdaab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.778+0000 7f85bc767740 1 --2- 172.21.15.18:0/3160644493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cdaf2e0 0x55616cdaab90 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0eddd0 tx=0x7f85ac04a040 comp rx=0 tx=0).stop 2024-02-17T12:58:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.778+0000 7f85bc767740 1 -- 172.21.15.18:0/3160644493 shutdown_connections 2024-02-17T12:58:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.778+0000 7f85bc767740 1 --2- 172.21.15.18:0/3160644493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cdaf2e0 0x55616cdaab90 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.778+0000 7f85bc767740 1 --2- 172.21.15.18:0/3160644493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cda6ad0 0x55616cdacd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.778+0000 7f85bc767740 1 --2- 172.21.15.18:0/3160644493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda5d60 0x55616cdac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.778+0000 7f85bc767740 1 -- 172.21.15.18:0/3160644493 >> 172.21.15.18:0/3160644493 conn(0x55616cd6d550 msgr2=0x55616cdc63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.778+0000 7f85bc767740 1 -- 172.21.15.18:0/3160644493 shutdown_connections 2024-02-17T12:58:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.778+0000 7f85bc767740 1 -- 172.21.15.18:0/3160644493 wait complete. 2024-02-17T12:58:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.778+0000 7f85bc767740 1 -- start start 2024-02-17T12:58:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.778+0000 7f85bc767740 1 Processor -- start 2024-02-17T12:58:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85bc767740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda5d60 0x55616cdb4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85bc767740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda6ad0 0x55616cdb4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85bc767740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdb54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85bc767740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616cdb69c0 con 0x55616cda5d60 2024-02-17T12:58:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85bc767740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616ce15b60 con 0x55616cdaf2e0 2024-02-17T12:58:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85bc767740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616ce15ca0 con 0x55616cda6ad0 2024-02-17T12:58:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85bc767740 10 client.0 _ll_register_callbacks cb 0x7f85a400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85b37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda5d60 0x55616cdb4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85b37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda5d60 0x55616cdb4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56212/0 (socket says 172.21.15.18:56212) 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85b37fe640 1 -- 172.21.15.18:0/2503029411 learned_addr learned my addr 172.21.15.18:0/2503029411 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85b3fff640 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdb54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85b2ffd640 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda6ad0 0x55616cdb4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85b37fe640 1 -- 172.21.15.18:0/2503029411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda6ad0 msgr2=0x55616cdb4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85b37fe640 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda6ad0 0x55616cdb4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85b37fe640 1 -- 172.21.15.18:0/2503029411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 msgr2=0x55616cdb54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85b37fe640 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdb54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85b37fe640 1 -- 172.21.15.18:0/2503029411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b41a4450 con 0x55616cda5d60 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85b2ffd640 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda6ad0 0x55616cdb4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f85b37fe640 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda5d60 0x55616cdb4a20 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f85a4004cc0 tx=0x7f85a4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213f25b16c8b1d90 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f8592ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.779+0000 7f8592ffd640 1 -- 172.21.15.18:0/2503029411 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4028420 con 0x55616cda5d60 2024-02-17T12:58:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f8592ffd640 1 -- 172.21.15.18:0/2503029411 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f85a4017070 con 0x55616cda5d60 2024-02-17T12:58:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f85bc767740 1 -- 172.21.15.18:0/2503029411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85a4015070 con 0x55616cda5d60 2024-02-17T12:58:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f8592ffd640 1 -- 172.21.15.18:0/2503029411 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85a4028580 con 0x55616cda5d60 2024-02-17T12:58:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f85bc767740 10 client.18729 fetch_fsmap learned FSMap version 45 2024-02-17T12:58:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f85bc767740 1 -- 172.21.15.18:0/2503029411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55616ccce540 con 0x55616cda5d60 2024-02-17T12:58:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f8592ffd640 1 -- 172.21.15.18:0/2503029411 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85a4015070 con 0x55616cda5d60 2024-02-17T12:58:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f8592ffd640 1 -- 172.21.15.18:0/2503029411 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 45) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f85a4011440 con 0x55616cda5d60 2024-02-17T12:58:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f85bc767740 10 client.18729 fetch_fsmap finished waiting for FSMap version 45 2024-02-17T12:58:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f85bc767740 10 client.18729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f85bc767740 20 client.18729 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f85bc767740 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55616ce173e0 0x55616ce19800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f85bc767740 4 client.18729 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:58:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f85bc767740 1 -- 172.21.15.18:0/2503029411 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55616ca4a550 con 0x55616ce173e0 2024-02-17T12:58:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.780+0000 7f85b2ffd640 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55616ce173e0 0x55616ce19800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.781+0000 7f8592ffd640 10 client.18729 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.781+0000 7f85b2ffd640 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55616ce173e0 0x55616ce19800 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3385d0505e82da7 server_cookie=f619b21149903c22 in_seq=0 out_seq=0 2024-02-17T12:58:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.818+0000 7f8592ffd640 1 -- 172.21.15.18:0/2503029411 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55616ca4a550 con 0x55616ce173e0 2024-02-17T12:58:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.818+0000 7f8592ffd640 10 client.18729 handle_command_reply: tid=0 2024-02-17T12:58:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.818+0000 7f85bc767740 1 client.18729 shutdown 2024-02-17T12:58:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.818+0000 7f85bc767740 1 -- 172.21.15.18:0/2503029411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda5d60 msgr2=0x55616cdb4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.818+0000 7f85bc767740 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda5d60 0x55616cdb4a20 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f85a4004cc0 tx=0x7f85a4004770 comp rx=0 tx=0).stop 2024-02-17T12:58:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85bc767740 1 -- 172.21.15.18:0/2503029411 shutdown_connections 2024-02-17T12:58:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85bc767740 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55616ce173e0 0x55616ce19800 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f8592ffd640 0 client.18729 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85bc767740 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55616cdaf2e0 0x55616cdb54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85bc767740 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55616cda6ad0 0x55616cdb4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85bc767740 1 --2- 172.21.15.18:0/2503029411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55616cda5d60 0x55616cdb4a20 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85bc767740 1 -- 172.21.15.18:0/2503029411 >> 172.21.15.18:0/2503029411 conn(0x55616cd6d550 msgr2=0x55616cda1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85bc767740 1 -- 172.21.15.18:0/2503029411 shutdown_connections 2024-02-17T12:58:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85bc767740 1 -- 172.21.15.18:0/2503029411 wait complete. 2024-02-17T12:58:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85bc767740 20 client.18729 trim_cache size 0 max 16384 2024-02-17T12:58:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:58:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:58:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:58:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:58:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85b9b8c640 1 -- 172.21.15.18:0/1556684596 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f859c0a22d0 msgr2=0x7f859c0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85b9b8c640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f859c0a22d0 0x7f859c0a4790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009fd0 tx=0x7f85a80136e0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85b9b8c640 1 -- 172.21.15.18:0/1556684596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85b4106160 msgr2=0x7f85b419bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.819+0000 7f85b9b8c640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85b4106160 0x7f85b419bfc0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f85ac005940 tx=0x7f85ac02c650 comp rx=0 tx=0).stop 2024-02-17T12:58:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.820+0000 7f85b9b8c640 1 -- 172.21.15.18:0/1556684596 shutdown_connections 2024-02-17T12:58:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.820+0000 7f85b9b8c640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f859c0a22d0 0x7f859c0a4790 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.820+0000 7f85b9b8c640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85b4106160 0x7f85b419bfc0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.820+0000 7f85b9b8c640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85b4100040 0x7f85b4197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.820+0000 7f85b9b8c640 1 --2- 172.21.15.18:0/1556684596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85b40ff680 0x7f85b4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.820+0000 7f85b9b8c640 1 -- 172.21.15.18:0/1556684596 >> 172.21.15.18:0/1556684596 conn(0x7f85b4076610 msgr2=0x7f85b4103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.820+0000 7f85b9b8c640 1 -- 172.21.15.18:0/1556684596 shutdown_connections 2024-02-17T12:58:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:46.820+0000 7f85b9b8c640 1 -- 172.21.15.18:0/1556684596 wait complete. 2024-02-17T12:58:47.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 460030 5.32 MB/sec execute 362 sec latency 515.941 ms 2024-02-17T12:58:48.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:47 smithi018 ceph-mon[32625]: pgmap v277: 97 pgs: 97 active+clean; 162 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-02-17T12:58:48.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 460030 5.31 MB/sec execute 363 sec latency 1516.090 ms 2024-02-17T12:58:48.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:47 smithi119 ceph-mon[36188]: pgmap v277: 97 pgs: 97 active+clean; 162 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-02-17T12:58:48.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:47 smithi146 ceph-mon[36454]: pgmap v277: 97 pgs: 97 active+clean; 162 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-02-17T12:58:49.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:48 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:58:49.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 460946 5.30 MB/sec execute 364 sec latency 2115.732 ms 2024-02-17T12:58:49.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:48 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:58:49.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:48 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:58:50.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:49 smithi018 ceph-mon[32625]: pgmap v278: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-02-17T12:58:50.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 462693 5.32 MB/sec execute 365 sec latency 12.634 ms 2024-02-17T12:58:50.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:49 smithi119 ceph-mon[36188]: pgmap v278: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-02-17T12:58:50.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:49 smithi146 ceph-mon[36454]: pgmap v278: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-02-17T12:58:51.291 INFO:tasks.workunit.client.0.smithi018.stdout: 1 463595 5.31 MB/sec execute 366 sec latency 514.784 ms 2024-02-17T12:58:52.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:51 smithi018 ceph-mon[32625]: pgmap v279: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 186 op/s 2024-02-17T12:58:52.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 463595 5.30 MB/sec execute 367 sec latency 1514.819 ms 2024-02-17T12:58:52.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:51 smithi119 ceph-mon[36188]: pgmap v279: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 186 op/s 2024-02-17T12:58:52.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:51 smithi146 ceph-mon[36454]: pgmap v279: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 186 op/s 2024-02-17T12:58:53.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 464661 5.30 MB/sec execute 368 sec latency 2079.971 ms 2024-02-17T12:58:54.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:53 smithi018 ceph-mon[32625]: pgmap v280: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 186 op/s 2024-02-17T12:58:54.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 466258 5.31 MB/sec execute 369 sec latency 14.248 ms 2024-02-17T12:58:54.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:53 smithi119 ceph-mon[36188]: pgmap v280: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 186 op/s 2024-02-17T12:58:54.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:53 smithi146 ceph-mon[36454]: pgmap v280: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 186 op/s 2024-02-17T12:58:55.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 467138 5.31 MB/sec execute 370 sec latency 541.823 ms 2024-02-17T12:58:56.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:55 smithi018 ceph-mon[32625]: pgmap v281: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 261 op/s 2024-02-17T12:58:56.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 467138 5.29 MB/sec execute 371 sec latency 1541.929 ms 2024-02-17T12:58:56.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:55 smithi119 ceph-mon[36188]: pgmap v281: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 261 op/s 2024-02-17T12:58:56.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:55 smithi146 ceph-mon[36454]: pgmap v281: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 261 op/s 2024-02-17T12:58:56.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:58:56.935 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:58:57.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.062+0000 7f25c470d640 1 -- 172.21.15.18:0/2718484182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25bc106110 msgr2=0x7f25bc1064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.062+0000 7f25c470d640 1 --2- 172.21.15.18:0/2718484182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25bc106110 0x7f25bc1064f0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009900 tx=0x7f25b802bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:57.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.062+0000 7f25c470d640 1 -- 172.21.15.18:0/2718484182 shutdown_connections 2024-02-17T12:58:57.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.062+0000 7f25c470d640 1 --2- 172.21.15.18:0/2718484182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25bc100c90 0x7f25bc10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.062+0000 7f25c470d640 1 --2- 172.21.15.18:0/2718484182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc1002d0 0x7f25bc100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.062+0000 7f25c470d640 1 --2- 172.21.15.18:0/2718484182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25bc106110 0x7f25bc1064f0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.062+0000 7f25c470d640 1 -- 172.21.15.18:0/2718484182 >> 172.21.15.18:0/2718484182 conn(0x7f25bc076e20 msgr2=0x7f25bc077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.062+0000 7f25c470d640 1 -- 172.21.15.18:0/2718484182 shutdown_connections 2024-02-17T12:58:57.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.062+0000 7f25c470d640 1 -- 172.21.15.18:0/2718484182 wait complete. 2024-02-17T12:58:57.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c470d640 1 Processor -- start 2024-02-17T12:58:57.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c470d640 1 -- start start 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c470d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25bc1002d0 0x7f25bc197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c470d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc100c90 0x7f25bc197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c470d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25bc106110 0x7f25bc19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c470d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc19c610 con 0x7f25bc106110 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c470d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc19c780 con 0x7f25bc1002d0 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c470d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc19c940 con 0x7f25bc100c90 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c2caa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25bc106110 0x7f25bc19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c2caa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25bc106110 0x7f25bc19c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58562/0 (socket says 172.21.15.18:58562) 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c2caa640 1 -- 172.21.15.18:0/3626196694 learned_addr learned my addr 172.21.15.18:0/3626196694 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c24a9640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25bc1002d0 0x7f25bc197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc100c90 0x7f25bc197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c2caa640 1 -- 172.21.15.18:0/3626196694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc100c90 msgr2=0x7f25bc197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c2caa640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc100c90 0x7f25bc197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c2caa640 1 -- 172.21.15.18:0/3626196694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25bc1002d0 msgr2=0x7f25bc197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c2caa640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25bc1002d0 0x7f25bc197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c2caa640 1 -- 172.21.15.18:0/3626196694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b8009560 con 0x7f25bc106110 2024-02-17T12:58:57.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.063+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc100c90 0x7f25bc197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:57.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.064+0000 7f25c2caa640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25bc106110 0x7f25bc19c010 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f25b400eb10 tx=0x7f25b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.064+0000 7f25b37fe640 1 -- 172.21.15.18:0/3626196694 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b4004b20 con 0x7f25bc106110 2024-02-17T12:58:57.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.064+0000 7f25b37fe640 1 -- 172.21.15.18:0/3626196694 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25b4004c80 con 0x7f25bc106110 2024-02-17T12:58:57.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.064+0000 7f25b37fe640 1 -- 172.21.15.18:0/3626196694 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b4010650 con 0x7f25bc106110 2024-02-17T12:58:57.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.064+0000 7f25c470d640 1 -- 172.21.15.18:0/3626196694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc19cc20 con 0x7f25bc106110 2024-02-17T12:58:57.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.064+0000 7f25c470d640 1 -- 172.21.15.18:0/3626196694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc1a44a0 con 0x7f25bc106110 2024-02-17T12:58:57.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.066+0000 7f25b37fe640 1 -- 172.21.15.18:0/3626196694 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f25b4005020 con 0x7f25bc106110 2024-02-17T12:58:57.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.066+0000 7f25c73ba740 1 Processor -- start 2024-02-17T12:58:57.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c73ba740 1 -- start start 2024-02-17T12:58:57.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25b37fe640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f25a80a2230 0x7f25a80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c73ba740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 0x560df1b663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c73ba740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b66900 0x560df1b73680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c73ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1b75fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c73ba740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df1ad5280 con 0x560df1b65fe0 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c73ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df1ad27b0 con 0x560df1b73bc0 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c73ba740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df1acdf90 con 0x560df1b66900 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25b37fe640 1 -- 172.21.15.18:0/3626196694 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f25b4014070 con 0x7f25bc106110 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f25a80a2230 0x7f25a80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c24a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 0x560df1b663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c24a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 0x560df1b663c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58572/0 (socket says 172.21.15.18:58572) 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c24a9640 1 -- 172.21.15.18:0/1734808127 learned_addr learned my addr 172.21.15.18:0/1734808127 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/1734808127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b66900 0x560df1b73680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c2caa640 1 --2- 172.21.15.18:0/1734808127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1b75fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c24a9640 1 -- 172.21.15.18:0/1734808127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b66900 msgr2=0x560df1b73680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c24a9640 1 --2- 172.21.15.18:0/1734808127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b66900 0x560df1b73680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c24a9640 1 -- 172.21.15.18:0/1734808127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 msgr2=0x560df1b75fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c24a9640 1 --2- 172.21.15.18:0/1734808127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1b75fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c24a9640 1 -- 172.21.15.18:0/1734808127 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560df1a8c570 con 0x560df1b65fe0 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.067+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f25a80a2230 0x7f25a80a46f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f25ac005d00 tx=0x7f25ac009b10 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.068+0000 7f25c24a9640 1 --2- 172.21.15.18:0/1734808127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 0x560df1b663c0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f25b8000c00 tx=0x7f25b802c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438739e7b7b97f48 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.068+0000 7f25b0ff9640 1 -- 172.21.15.18:0/1734808127 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8004800 con 0x560df1b65fe0 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.068+0000 7f25b0ff9640 1 -- 172.21.15.18:0/1734808127 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25b8004960 con 0x560df1b65fe0 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.068+0000 7f25b0ff9640 1 -- 172.21.15.18:0/1734808127 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8005050 con 0x560df1b65fe0 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.068+0000 7f25c73ba740 1 -- 172.21.15.18:0/1734808127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 msgr2=0x560df1b663c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.068+0000 7f25c73ba740 1 --2- 172.21.15.18:0/1734808127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 0x560df1b663c0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f25b8000c00 tx=0x7f25b802c670 comp rx=0 tx=0).stop 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.068+0000 7f25c73ba740 1 -- 172.21.15.18:0/1734808127 shutdown_connections 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.068+0000 7f25c73ba740 1 --2- 172.21.15.18:0/1734808127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1b75fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.068+0000 7f25c73ba740 1 --2- 172.21.15.18:0/1734808127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b66900 0x560df1b73680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.068+0000 7f25c73ba740 1 --2- 172.21.15.18:0/1734808127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 0x560df1b663c0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.068+0000 7f25c73ba740 1 -- 172.21.15.18:0/1734808127 >> 172.21.15.18:0/1734808127 conn(0x560df1b60990 msgr2=0x560df1b60d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c73ba740 1 -- 172.21.15.18:0/1734808127 shutdown_connections 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c73ba740 1 -- 172.21.15.18:0/1734808127 wait complete. 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c73ba740 1 -- start start 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c73ba740 1 Processor -- start 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c73ba740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 0x560df1b73610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c73ba740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b66900 0x560df1b709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c73ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1b710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c73ba740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df17a1430 con 0x560df1b65fe0 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c73ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df175de00 con 0x560df1b73bc0 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c73ba740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df1a52fd0 con 0x560df1b66900 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c73ba740 10 client.0 _ll_register_callbacks cb 0x560df1aa8fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c24a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 0x560df1b73610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c24a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 0x560df1b73610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58574/0 (socket says 172.21.15.18:58574) 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c24a9640 1 -- 172.21.15.18:0/3085372611 learned_addr learned my addr 172.21.15.18:0/3085372611 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.069+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b66900 0x560df1b709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25c2caa640 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1b710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25c24a9640 1 -- 172.21.15.18:0/3085372611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b66900 msgr2=0x560df1b709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25c24a9640 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b66900 0x560df1b709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25c24a9640 1 -- 172.21.15.18:0/3085372611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 msgr2=0x560df1b710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25c24a9640 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1b710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25c24a9640 1 -- 172.21.15.18:0/3085372611 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc0697f0 con 0x560df1b65fe0 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25c2caa640 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1b710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25c24a9640 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 0x560df1b73610 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f25b802c780 tx=0x7f25b8004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adebdb43f763ed3f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25a5ffb640 1 -- 172.21.15.18:0/3085372611 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8004800 con 0x560df1b65fe0 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25a5ffb640 1 -- 172.21.15.18:0/3085372611 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f25b80053b0 con 0x560df1b65fe0 2024-02-17T12:58:57.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25c73ba740 1 -- 172.21.15.18:0/3085372611 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560df17cd7e0 con 0x560df1b65fe0 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25a5ffb640 1 -- 172.21.15.18:0/3085372611 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b8004ad0 con 0x560df1b65fe0 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25c73ba740 10 client.18741 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25c73ba740 1 -- 172.21.15.18:0/3085372611 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x560df1acc610 con 0x560df1b65fe0 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.070+0000 7f25a5ffb640 1 -- 172.21.15.18:0/3085372611 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560df17cd7e0 con 0x560df1b65fe0 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.071+0000 7f25a5ffb640 1 -- 172.21.15.18:0/3085372611 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f25b803c050 con 0x560df1b65fe0 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.071+0000 7f25c73ba740 10 client.18741 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.071+0000 7f25c73ba740 10 client.18741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.071+0000 7f25c73ba740 20 client.18741 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.071+0000 7f25c73ba740 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560df1b83990 0x560df1b85db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.071+0000 7f25c73ba740 4 client.18741 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.071+0000 7f25c73ba740 1 -- 172.21.15.18:0/3085372611 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560df180a720 con 0x560df1b83990 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.071+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560df1b83990 0x560df1b85db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.071+0000 7f25a5ffb640 10 client.18741 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:57.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.071+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560df1b83990 0x560df1b85db0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1af6c9ce149b5e81 server_cookie=c7d4d41bac12bfea in_seq=0 out_seq=0 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25a5ffb640 1 -- 172.21.15.18:0/3085372611 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560df180a720 con 0x560df1b83990 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25a5ffb640 10 client.18741 handle_command_reply: tid=0 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25c73ba740 1 client.18741 shutdown 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25c73ba740 1 -- 172.21.15.18:0/3085372611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 msgr2=0x560df1b73610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25c73ba740 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 0x560df1b73610 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f25b802c780 tx=0x7f25b8004460 comp rx=0 tx=0).stop 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25c73ba740 1 -- 172.21.15.18:0/3085372611 shutdown_connections 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25c73ba740 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560df1b83990 0x560df1b85db0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25c73ba740 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1b710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25a5ffb640 0 client.18741 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25c73ba740 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b66900 0x560df1b709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25c73ba740 1 --2- 172.21.15.18:0/3085372611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b65fe0 0x560df1b73610 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25c73ba740 1 -- 172.21.15.18:0/3085372611 >> 172.21.15.18:0/3085372611 conn(0x560df1b60990 msgr2=0x560df1b62730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25c73ba740 1 -- 172.21.15.18:0/3085372611 shutdown_connections 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25c73ba740 1 -- 172.21.15.18:0/3085372611 wait complete. 2024-02-17T12:58:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.072+0000 7f25c73ba740 20 client.18741 trim_cache size 0 max 16384 2024-02-17T12:58:57.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c73ba740 1 Processor -- start 2024-02-17T12:58:57.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c73ba740 1 -- start start 2024-02-17T12:58:57.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c73ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b65fe0 0x560df1b730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c73ba740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 0x560df1b73610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c73ba740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b73bc0 0x560df1b70230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c73ba740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df1b601d0 con 0x560df1b66900 2024-02-17T12:58:57.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c73ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df1b5c420 con 0x560df1b65fe0 2024-02-17T12:58:57.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c73ba740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df1b67030 con 0x560df1b73bc0 2024-02-17T12:58:57.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c1ca8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 0x560df1b73610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c1ca8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 0x560df1b73610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58578/0 (socket says 172.21.15.18:58578) 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c1ca8640 1 -- 172.21.15.18:0/1105368624 learned_addr learned my addr 172.21.15.18:0/1105368624 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c24a9640 1 --2- 172.21.15.18:0/1105368624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b65fe0 0x560df1b730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c2caa640 1 --2- 172.21.15.18:0/1105368624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b73bc0 0x560df1b70230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c1ca8640 1 -- 172.21.15.18:0/1105368624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b73bc0 msgr2=0x560df1b70230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/1105368624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b73bc0 0x560df1b70230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c1ca8640 1 -- 172.21.15.18:0/1105368624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b65fe0 msgr2=0x560df1b730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/1105368624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b65fe0 0x560df1b730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c1ca8640 1 -- 172.21.15.18:0/1105368624 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560df1a68ac0 con 0x560df1b66900 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c2caa640 1 --2- 172.21.15.18:0/1105368624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b73bc0 0x560df1b70230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.089+0000 7f25c24a9640 1 --2- 172.21.15.18:0/1105368624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b65fe0 0x560df1b730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.090+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/1105368624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 0x560df1b73610 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0058c0 tx=0x7f25ac01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d695d846b50359f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.090+0000 7f25a7fff640 1 -- 172.21.15.18:0/1105368624 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac02a3e0 con 0x560df1b66900 2024-02-17T12:58:57.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.090+0000 7f25a7fff640 1 -- 172.21.15.18:0/1105368624 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25ac01b040 con 0x560df1b66900 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.090+0000 7f25a7fff640 1 -- 172.21.15.18:0/1105368624 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac005280 con 0x560df1b66900 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.090+0000 7f25c73ba740 1 -- 172.21.15.18:0/1105368624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 msgr2=0x560df1b73610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.090+0000 7f25c73ba740 1 --2- 172.21.15.18:0/1105368624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 0x560df1b73610 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0058c0 tx=0x7f25ac01f000 comp rx=0 tx=0).stop 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 -- 172.21.15.18:0/1105368624 shutdown_connections 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 --2- 172.21.15.18:0/1105368624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b73bc0 0x560df1b70230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 --2- 172.21.15.18:0/1105368624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 0x560df1b73610 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 --2- 172.21.15.18:0/1105368624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b65fe0 0x560df1b730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 -- 172.21.15.18:0/1105368624 >> 172.21.15.18:0/1105368624 conn(0x560df1b60990 msgr2=0x560df1b6f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 -- 172.21.15.18:0/1105368624 shutdown_connections 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 -- 172.21.15.18:0/1105368624 wait complete. 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 -- start start 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 Processor -- start 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b65fe0 0x560df1b77ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 0x560df1b78000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1bd9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df1b78720 con 0x560df1b66900 2024-02-17T12:58:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df1bda410 con 0x560df1b73bc0 2024-02-17T12:58:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df1bda580 con 0x560df1b65fe0 2024-02-17T12:58:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c73ba740 10 client.0 _ll_register_callbacks cb 0x7f25ac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c1ca8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 0x560df1b78000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c1ca8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 0x560df1b78000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58592/0 (socket says 172.21.15.18:58592) 2024-02-17T12:58:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c1ca8640 1 -- 172.21.15.18:0/32179128 learned_addr learned my addr 172.21.15.18:0/32179128 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c24a9640 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b65fe0 0x560df1b77ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.091+0000 7f25c2caa640 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1bd9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25c1ca8640 1 -- 172.21.15.18:0/32179128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b65fe0 msgr2=0x560df1b77ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b65fe0 0x560df1b77ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25c1ca8640 1 -- 172.21.15.18:0/32179128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 msgr2=0x560df1bd9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1bd9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25c1ca8640 1 -- 172.21.15.18:0/32179128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560df1acc610 con 0x560df1b66900 2024-02-17T12:58:57.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25c1ca8640 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 0x560df1b78000 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f25ac015f20 tx=0x7f25ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3782f1918d7804cc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:57.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25a5ffb640 1 -- 172.21.15.18:0/32179128 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac012b70 con 0x560df1b66900 2024-02-17T12:58:57.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25a5ffb640 1 -- 172.21.15.18:0/32179128 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f25ac027040 con 0x560df1b66900 2024-02-17T12:58:57.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25c73ba740 1 -- 172.21.15.18:0/32179128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25b8004960 con 0x560df1b66900 2024-02-17T12:58:57.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25a5ffb640 1 -- 172.21.15.18:0/32179128 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ac02a3e0 con 0x560df1b66900 2024-02-17T12:58:57.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25c73ba740 10 client.18747 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:57.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.092+0000 7f25c73ba740 1 -- 172.21.15.18:0/32179128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x560df1a8e6a0 con 0x560df1b66900 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.093+0000 7f25a5ffb640 1 -- 172.21.15.18:0/32179128 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25b8004960 con 0x560df1b66900 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.093+0000 7f25a5ffb640 1 -- 172.21.15.18:0/32179128 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f25ac0046c0 con 0x560df1b66900 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.093+0000 7f25c73ba740 10 client.18747 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.093+0000 7f25c73ba740 10 client.18747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.093+0000 7f25c73ba740 20 client.18747 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.093+0000 7f25c73ba740 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560df1bdbd30 0x560df1bde150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.093+0000 7f25c73ba740 4 client.18747 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.093+0000 7f25c73ba740 1 -- 172.21.15.18:0/32179128 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560df1859890 con 0x560df1bdbd30 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.093+0000 7f25c24a9640 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560df1bdbd30 0x560df1bde150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c24a9640 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560df1bdbd30 0x560df1bde150 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd79812032366890 server_cookie=4c2784c3bc5dbfb9 in_seq=0 out_seq=0 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25a5ffb640 10 client.18747 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25a5ffb640 1 -- 172.21.15.18:0/32179128 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560df1859890 con 0x560df1bdbd30 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25a5ffb640 10 client.18747 handle_command_reply: tid=0 2024-02-17T12:58:57.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c73ba740 1 client.18747 shutdown 2024-02-17T12:58:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c73ba740 1 -- 172.21.15.18:0/32179128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 msgr2=0x560df1b78000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c73ba740 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 0x560df1b78000 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f25ac015f20 tx=0x7f25ac000bc0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c73ba740 1 -- 172.21.15.18:0/32179128 shutdown_connections 2024-02-17T12:58:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c73ba740 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560df1bdbd30 0x560df1bde150 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c73ba740 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560df1b73bc0 0x560df1bd9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c73ba740 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560df1b66900 0x560df1b78000 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25a5ffb640 0 client.18747 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c73ba740 1 --2- 172.21.15.18:0/32179128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560df1b65fe0 0x560df1b77ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c73ba740 1 -- 172.21.15.18:0/32179128 >> 172.21.15.18:0/32179128 conn(0x560df1b60990 msgr2=0x560df1b70200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c73ba740 1 -- 172.21.15.18:0/32179128 shutdown_connections 2024-02-17T12:58:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c73ba740 1 -- 172.21.15.18:0/32179128 wait complete. 2024-02-17T12:58:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.094+0000 7f25c73ba740 20 client.18747 trim_cache size 0 max 16384 2024-02-17T12:58:57.092 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:58:57.092 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:58:57.092 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:58:57.092 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:58:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.095+0000 7f25c470d640 1 -- 172.21.15.18:0/3626196694 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f25a80a2230 msgr2=0x7f25a80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.095+0000 7f25c470d640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f25a80a2230 0x7f25a80a46f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f25ac005d00 tx=0x7f25ac009b10 comp rx=0 tx=0).stop 2024-02-17T12:58:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.095+0000 7f25c470d640 1 -- 172.21.15.18:0/3626196694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25bc106110 msgr2=0x7f25bc19c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.095+0000 7f25c470d640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25bc106110 0x7f25bc19c010 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f25b400eb10 tx=0x7f25b400c2d0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.095+0000 7f25c470d640 1 -- 172.21.15.18:0/3626196694 shutdown_connections 2024-02-17T12:58:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.095+0000 7f25c470d640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f25a80a2230 0x7f25a80a46f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.095+0000 7f25c470d640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25bc106110 0x7f25bc19c010 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.095+0000 7f25c470d640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc100c90 0x7f25bc197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.095+0000 7f25c470d640 1 --2- 172.21.15.18:0/3626196694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25bc1002d0 0x7f25bc197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.095+0000 7f25c470d640 1 -- 172.21.15.18:0/3626196694 >> 172.21.15.18:0/3626196694 conn(0x7f25bc076e20 msgr2=0x7f25bc0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.095+0000 7f25c470d640 1 -- 172.21.15.18:0/3626196694 shutdown_connections 2024-02-17T12:58:57.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.095+0000 7f25c470d640 1 -- 172.21.15.18:0/3626196694 wait complete. 2024-02-17T12:58:57.096 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:58:57.096 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:58:57.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.211+0000 7f44feac3640 1 -- 172.21.15.18:0/530565460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f81002f0 msgr2=0x7f44f81006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.211+0000 7f44feac3640 1 --2- 172.21.15.18:0/530565460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f81002f0 0x7f44f81006d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009930 tx=0x7f44ec02bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:57.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.211+0000 7f44feac3640 1 -- 172.21.15.18:0/530565460 shutdown_connections 2024-02-17T12:58:57.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.211+0000 7f44feac3640 1 --2- 172.21.15.18:0/530565460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f81053c0 0x7f44f810cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.211+0000 7f44feac3640 1 --2- 172.21.15.18:0/530565460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f8100ca0 0x7f44f8104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.211+0000 7f44feac3640 1 --2- 172.21.15.18:0/530565460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f81002f0 0x7f44f81006d0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.211+0000 7f44feac3640 1 -- 172.21.15.18:0/530565460 >> 172.21.15.18:0/530565460 conn(0x7f44f8077180 msgr2=0x7f44f8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.211+0000 7f44feac3640 1 -- 172.21.15.18:0/530565460 shutdown_connections 2024-02-17T12:58:57.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.211+0000 7f44feac3640 1 -- 172.21.15.18:0/530565460 wait complete. 2024-02-17T12:58:57.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.211+0000 7f44feac3640 1 Processor -- start 2024-02-17T12:58:57.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.211+0000 7f44feac3640 1 -- start start 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44feac3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f81002f0 0x7f44f8196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44feac3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f81053c0 0x7f44f8196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44feac3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f819af70 0x7f44f819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44feac3640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f819b960 con 0x7f44f81002f0 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44feac3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f819bad0 con 0x7f44f81053c0 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44feac3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f819bc40 con 0x7f44f819af70 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44fc85f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f81002f0 0x7f44f8196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44fc85f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f81002f0 0x7f44f8196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58616/0 (socket says 172.21.15.18:58616) 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44fc85f640 1 -- 172.21.15.18:0/1560534679 learned_addr learned my addr 172.21.15.18:0/1560534679 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44f7fff640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f81053c0 0x7f44f8196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44fd060640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f819af70 0x7f44f819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44fc85f640 1 -- 172.21.15.18:0/1560534679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f819af70 msgr2=0x7f44f819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44fc85f640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f819af70 0x7f44f819b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44fc85f640 1 -- 172.21.15.18:0/1560534679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f81053c0 msgr2=0x7f44f8196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44fc85f640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f81053c0 0x7f44f8196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44fc85f640 1 -- 172.21.15.18:0/1560534679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec009560 con 0x7f44f81002f0 2024-02-17T12:58:57.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44fd060640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f819af70 0x7f44f819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:57.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44fc85f640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f81002f0 0x7f44f8196700 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009230 tx=0x7f44ec004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44f5ffb640 1 -- 172.21.15.18:0/1560534679 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec004440 con 0x7f44f81002f0 2024-02-17T12:58:57.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44f5ffb640 1 -- 172.21.15.18:0/1560534679 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44ec0052d0 con 0x7f44f81002f0 2024-02-17T12:58:57.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44feac3640 1 -- 172.21.15.18:0/1560534679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f81a3220 con 0x7f44f81002f0 2024-02-17T12:58:57.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.212+0000 7f44f5ffb640 1 -- 172.21.15.18:0/1560534679 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec003d70 con 0x7f44f81002f0 2024-02-17T12:58:57.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.213+0000 7f44feac3640 1 -- 172.21.15.18:0/1560534679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f81a36a0 con 0x7f44f81002f0 2024-02-17T12:58:57.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.214+0000 7f44feac3640 1 -- 172.21.15.18:0/1560534679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f8067f70 con 0x7f44f81002f0 2024-02-17T12:58:57.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.215+0000 7f44f5ffb640 1 -- 172.21.15.18:0/1560534679 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f44ec005440 con 0x7f44f81002f0 2024-02-17T12:58:57.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.215+0000 7f44f5ffb640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44e80a2230 0x7f44e80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.215+0000 7f44f5ffb640 1 -- 172.21.15.18:0/1560534679 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f44ec0f01d0 con 0x7f44f81002f0 2024-02-17T12:58:57.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.215+0000 7f44f7fff640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44e80a2230 0x7f44e80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.216+0000 7f44f7fff640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44e80a2230 0x7f44e80a46f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f44f80ffba0 tx=0x7f44e400b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.220+0000 7f44f5ffb640 1 -- 172.21.15.18:0/1560534679 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f44ec0b8f90 con 0x7f44f81002f0 2024-02-17T12:58:57.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 467138 5.28 MB/sec execute 372 sec latency 2541.988 ms 2024-02-17T12:58:57.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.438+0000 7f44feac3640 1 -- 172.21.15.18:0/1560534679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44f81085f0 con 0x7f44f81002f0 2024-02-17T12:58:57.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.439+0000 7f44f5ffb640 1 -- 172.21.15.18:0/1560534679 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+11393 (secure 0 0 0) 0x7f44ec0bc650 con 0x7f44f81002f0 2024-02-17T12:58:57.429 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:58:57.429 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:58:47.850187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:58:57.429 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 46 2024-02-17T12:58:57.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.440+0000 7f44feac3640 1 -- 172.21.15.18:0/1560534679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44e80a2230 msgr2=0x7f44e80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.440+0000 7f44feac3640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44e80a2230 0x7f44e80a46f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f44f80ffba0 tx=0x7f44e400b390 comp rx=0 tx=0).stop 2024-02-17T12:58:57.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.440+0000 7f44feac3640 1 -- 172.21.15.18:0/1560534679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f81002f0 msgr2=0x7f44f8196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.440+0000 7f44feac3640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f81002f0 0x7f44f8196700 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009230 tx=0x7f44ec004040 comp rx=0 tx=0).stop 2024-02-17T12:58:57.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.441+0000 7f44feac3640 1 -- 172.21.15.18:0/1560534679 shutdown_connections 2024-02-17T12:58:57.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.441+0000 7f44feac3640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44e80a2230 0x7f44e80a46f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.441+0000 7f44feac3640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f819af70 0x7f44f819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.441+0000 7f44feac3640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f81053c0 0x7f44f8196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.441+0000 7f44feac3640 1 --2- 172.21.15.18:0/1560534679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f81002f0 0x7f44f8196700 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.441+0000 7f44feac3640 1 -- 172.21.15.18:0/1560534679 >> 172.21.15.18:0/1560534679 conn(0x7f44f8077180 msgr2=0x7f44f80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.441+0000 7f44feac3640 1 -- 172.21.15.18:0/1560534679 shutdown_connections 2024-02-17T12:58:57.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.441+0000 7f44feac3640 1 -- 172.21.15.18:0/1560534679 wait complete. 2024-02-17T12:58:57.442 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:58:57.442 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:58:57.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.558+0000 7fb0863d0640 1 -- 172.21.15.18:0/351906052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0801002a0 msgr2=0x7fb080100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.558+0000 7fb0863d0640 1 --2- 172.21.15.18:0/351906052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0801002a0 0x7fb080100680 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb070009930 tx=0x7fb07002bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:57.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.558+0000 7fb0863d0640 1 -- 172.21.15.18:0/351906052 shutdown_connections 2024-02-17T12:58:57.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.558+0000 7fb0863d0640 1 --2- 172.21.15.18:0/351906052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb080105370 0x7fb08010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.558+0000 7fb0863d0640 1 --2- 172.21.15.18:0/351906052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb080100c50 0x7fb080104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.558+0000 7fb0863d0640 1 --2- 172.21.15.18:0/351906052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0801002a0 0x7fb080100680 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.558+0000 7fb0863d0640 1 -- 172.21.15.18:0/351906052 >> 172.21.15.18:0/351906052 conn(0x7fb080076400 msgr2=0x7fb080076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.558+0000 7fb0863d0640 1 -- 172.21.15.18:0/351906052 shutdown_connections 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.558+0000 7fb0863d0640 1 -- 172.21.15.18:0/351906052 wait complete. 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.558+0000 7fb0863d0640 1 Processor -- start 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.558+0000 7fb0863d0640 1 -- start start 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb0863d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0801002a0 0x7fb08019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb0863d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb080105370 0x7fb08019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb0863d0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0801975f0 0x7fb080197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb0863d0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb080197fe0 con 0x7fb0801975f0 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb0863d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb080198150 con 0x7fb0801002a0 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb0863d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb080198310 con 0x7fb080105370 2024-02-17T12:58:57.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb08496d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0801975f0 0x7fb080197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb08496d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0801975f0 0x7fb080197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58620/0 (socket says 172.21.15.18:58620) 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb08496d640 1 -- 172.21.15.18:0/2202922039 learned_addr learned my addr 172.21.15.18:0/2202922039 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb080105370 0x7fb08019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb07ffff640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0801002a0 0x7fb08019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb08496d640 1 -- 172.21.15.18:0/2202922039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb080105370 msgr2=0x7fb08019d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb08496d640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb080105370 0x7fb08019d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb08496d640 1 -- 172.21.15.18:0/2202922039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0801002a0 msgr2=0x7fb08019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb08496d640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0801002a0 0x7fb08019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb08496d640 1 -- 172.21.15.18:0/2202922039 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb070009560 con 0x7fb0801975f0 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb08496d640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0801975f0 0x7fb080197aa0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb07800eaa0 tx=0x7fb07800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.559+0000 7fb07d7fa640 1 -- 172.21.15.18:0/2202922039 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07800cdb0 con 0x7fb0801975f0 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.560+0000 7fb07d7fa640 1 -- 172.21.15.18:0/2202922039 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb078005020 con 0x7fb0801975f0 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.560+0000 7fb0863d0640 1 -- 172.21.15.18:0/2202922039 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0801a4300 con 0x7fb0801975f0 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.560+0000 7fb07d7fa640 1 -- 172.21.15.18:0/2202922039 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780116b0 con 0x7fb0801975f0 2024-02-17T12:58:57.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.560+0000 7fb0863d0640 1 -- 172.21.15.18:0/2202922039 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0801a47c0 con 0x7fb0801975f0 2024-02-17T12:58:57.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb07d7fa640 1 -- 172.21.15.18:0/2202922039 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb078005520 con 0x7fb0801975f0 2024-02-17T12:58:57.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb088f9b740 1 Processor -- start 2024-02-17T12:58:57.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb088f9b740 1 -- start start 2024-02-17T12:58:57.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb088f9b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb088f9b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 0x5572337b9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb088f9b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b9750 0x5572337bbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb088f9b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723376b960 con 0x5572337b0b70 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb088f9b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723371f310 con 0x5572337b0250 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb088f9b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723371c840 con 0x5572337b9750 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb07d7fa640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb0680a2230 0x7fb0680a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb07ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 0x5572337b9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb07ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 0x5572337b9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58626/0 (socket says 172.21.15.18:58626) 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb07ffff640 1 -- 172.21.15.18:0/4007541140 learned_addr learned my addr 172.21.15.18:0/4007541140 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb08496d640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb0680a2230 0x7fb0680a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.562+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/4007541140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b9750 0x5572337bbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/4007541140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb07d7fa640 1 -- 172.21.15.18:0/2202922039 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb0780ce240 con 0x7fb0801975f0 2024-02-17T12:58:57.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb07ffff640 1 -- 172.21.15.18:0/4007541140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b9750 msgr2=0x5572337bbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb07ffff640 1 --2- 172.21.15.18:0/4007541140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b9750 0x5572337bbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb07ffff640 1 -- 172.21.15.18:0/4007541140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 msgr2=0x5572337b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb07ffff640 1 --2- 172.21.15.18:0/4007541140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb07ffff640 1 -- 172.21.15.18:0/4007541140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572336da060 con 0x5572337b0b70 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/4007541140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb08496d640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb0680a2230 0x7fb0680a46f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb0780cbfb0 tx=0x7fb07806ab60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb07ffff640 1 --2- 172.21.15.18:0/4007541140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 0x5572337b9210 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fb070036040 tx=0x7fb070005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c2bd2e931f328 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb0667fc640 1 -- 172.21.15.18:0/4007541140 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb070004900 con 0x5572337b0b70 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb0667fc640 1 -- 172.21.15.18:0/4007541140 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb070004a60 con 0x5572337b0b70 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.563+0000 7fb0667fc640 1 -- 172.21.15.18:0/4007541140 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0700051f0 con 0x5572337b0b70 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.564+0000 7fb088f9b740 1 -- 172.21.15.18:0/4007541140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 msgr2=0x5572337b9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.564+0000 7fb088f9b740 1 --2- 172.21.15.18:0/4007541140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 0x5572337b9210 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fb070036040 tx=0x7fb070005c00 comp rx=0 tx=0).stop 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.564+0000 7fb088f9b740 1 -- 172.21.15.18:0/4007541140 shutdown_connections 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.564+0000 7fb088f9b740 1 --2- 172.21.15.18:0/4007541140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b9750 0x5572337bbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.564+0000 7fb088f9b740 1 --2- 172.21.15.18:0/4007541140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 0x5572337b9210 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.564+0000 7fb088f9b740 1 --2- 172.21.15.18:0/4007541140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.564+0000 7fb088f9b740 1 -- 172.21.15.18:0/4007541140 >> 172.21.15.18:0/4007541140 conn(0x5572337aa980 msgr2=0x5572337aad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.564+0000 7fb088f9b740 1 -- 172.21.15.18:0/4007541140 shutdown_connections 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.564+0000 7fb088f9b740 1 -- 172.21.15.18:0/4007541140 wait complete. 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.564+0000 7fb088f9b740 1 -- start start 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.564+0000 7fb088f9b740 1 Processor -- start 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb088f9b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb088f9b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b0b70 0x5572337b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb088f9b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 0x5572337b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb088f9b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557233718020 con 0x5572337b9750 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb088f9b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572333eb430 con 0x5572337b0250 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb088f9b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572333a7e00 con 0x5572337b0b70 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb088f9b740 10 client.0 _ll_register_callbacks cb 0x5572336bfcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:57.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 0x5572337b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 0x5572337b7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58628/0 (socket says 172.21.15.18:58628) 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb07f7fe640 1 -- 172.21.15.18:0/546981669 learned_addr learned my addr 172.21.15.18:0/546981669 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb07ffff640 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b0b70 0x5572337b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb07f7fe640 1 -- 172.21.15.18:0/546981669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b0b70 msgr2=0x5572337b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b0b70 0x5572337b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb07f7fe640 1 -- 172.21.15.18:0/546981669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 msgr2=0x5572337b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb07f7fe640 1 -- 172.21.15.18:0/546981669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb080069770 con 0x5572337b9750 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 0x5572337b7240 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb074005fd0 tx=0x7fb0740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c0a14044346347 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.565+0000 7fb05f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb05f7fe640 1 -- 172.21.15.18:0/546981669 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074011d00 con 0x5572337b9750 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb05f7fe640 1 -- 172.21.15.18:0/546981669 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb074011e60 con 0x5572337b9750 2024-02-17T12:58:57.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb088f9b740 1 -- 172.21.15.18:0/546981669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572334177e0 con 0x5572337b9750 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb05f7fe640 1 -- 172.21.15.18:0/546981669 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb07400b4a0 con 0x5572337b9750 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb088f9b740 10 client.18765 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb088f9b740 1 -- 172.21.15.18:0/546981669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5572337166a0 con 0x5572337b9750 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb05f7fe640 1 -- 172.21.15.18:0/546981669 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5572334177e0 con 0x5572337b9750 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb05f7fe640 1 -- 172.21.15.18:0/546981669 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb074005500 con 0x5572337b9750 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb088f9b740 10 client.18765 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb088f9b740 10 client.18765 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb088f9b740 20 client.18765 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb088f9b740 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5572337c9b70 0x5572337cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb088f9b740 4 client.18765 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.566+0000 7fb088f9b740 1 -- 172.21.15.18:0/546981669 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557233454720 con 0x5572337c9b70 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.567+0000 7fb07ffff640 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5572337c9b70 0x5572337cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.567+0000 7fb07ffff640 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5572337c9b70 0x5572337cbf90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=93abb6638dfa0647 server_cookie=2d2624a311c6b1ba in_seq=0 out_seq=0 2024-02-17T12:58:57.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.567+0000 7fb05f7fe640 10 client.18765 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb05f7fe640 1 -- 172.21.15.18:0/546981669 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557233454720 con 0x5572337c9b70 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb05f7fe640 10 client.18765 handle_command_reply: tid=0 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb088f9b740 1 client.18765 shutdown 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb088f9b740 1 -- 172.21.15.18:0/546981669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 msgr2=0x5572337b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb088f9b740 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 0x5572337b7240 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb074005fd0 tx=0x7fb0740046c0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb088f9b740 1 -- 172.21.15.18:0/546981669 shutdown_connections 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb088f9b740 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 0x5572337b7240 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb088f9b740 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b0b70 0x5572337b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb088f9b740 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5572337c9b70 0x5572337cbf90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb088f9b740 1 --2- 172.21.15.18:0/546981669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb05f7fe640 0 client.18765 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb088f9b740 1 -- 172.21.15.18:0/546981669 >> 172.21.15.18:0/546981669 conn(0x5572337aa980 msgr2=0x5572337ac6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb088f9b740 1 -- 172.21.15.18:0/546981669 shutdown_connections 2024-02-17T12:58:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb088f9b740 1 -- 172.21.15.18:0/546981669 wait complete. 2024-02-17T12:58:57.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.568+0000 7fb088f9b740 20 client.18765 trim_cache size 0 max 16384 2024-02-17T12:58:57.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.584+0000 7fb088f9b740 1 Processor -- start 2024-02-17T12:58:57.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.584+0000 7fb088f9b740 1 -- start start 2024-02-17T12:58:57.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb088f9b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb088f9b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 0x5572337b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb088f9b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b9750 0x5572337b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb088f9b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723369d130 con 0x5572337b0b70 2024-02-17T12:58:57.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb088f9b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572337c55b0 con 0x5572337b0250 2024-02-17T12:58:57.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb088f9b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572337afeb0 con 0x5572337b9750 2024-02-17T12:58:57.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb07ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 0x5572337b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb07ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 0x5572337b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58634/0 (socket says 172.21.15.18:58634) 2024-02-17T12:58:57.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb07ffff640 1 -- 172.21.15.18:0/942904499 learned_addr learned my addr 172.21.15.18:0/942904499 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/942904499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/942904499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b9750 0x5572337b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb07ffff640 1 -- 172.21.15.18:0/942904499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b9750 msgr2=0x5572337b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb07ffff640 1 --2- 172.21.15.18:0/942904499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b9750 0x5572337b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb07ffff640 1 -- 172.21.15.18:0/942904499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 msgr2=0x5572337b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb07ffff640 1 --2- 172.21.15.18:0/942904499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb07ffff640 1 -- 172.21.15.18:0/942904499 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572336b2c20 con 0x5572337b0b70 2024-02-17T12:58:57.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.585+0000 7fb07ffff640 1 --2- 172.21.15.18:0/942904499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 0x5572337b5090 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x5572337afd20 tx=0x7fb070004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4239aed0292242a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.586+0000 7fb065ffb640 1 -- 172.21.15.18:0/942904499 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0700047c0 con 0x5572337b0b70 2024-02-17T12:58:57.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.586+0000 7fb065ffb640 1 -- 172.21.15.18:0/942904499 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb07002c860 con 0x5572337b0b70 2024-02-17T12:58:57.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.586+0000 7fb065ffb640 1 -- 172.21.15.18:0/942904499 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0700053c0 con 0x5572337b0b70 2024-02-17T12:58:57.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.586+0000 7fb088f9b740 1 -- 172.21.15.18:0/942904499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 msgr2=0x5572337b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.586+0000 7fb088f9b740 1 --2- 172.21.15.18:0/942904499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 0x5572337b5090 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x5572337afd20 tx=0x7fb070004e90 comp rx=0 tx=0).stop 2024-02-17T12:58:57.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.586+0000 7fb088f9b740 1 -- 172.21.15.18:0/942904499 shutdown_connections 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.586+0000 7fb088f9b740 1 --2- 172.21.15.18:0/942904499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b9750 0x5572337b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.586+0000 7fb088f9b740 1 --2- 172.21.15.18:0/942904499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b0b70 0x5572337b5090 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.586+0000 7fb088f9b740 1 --2- 172.21.15.18:0/942904499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0250 0x5572337b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.586+0000 7fb088f9b740 1 -- 172.21.15.18:0/942904499 >> 172.21.15.18:0/942904499 conn(0x5572337aa980 msgr2=0x5572337b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.586+0000 7fb088f9b740 1 -- 172.21.15.18:0/942904499 shutdown_connections 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.586+0000 7fb088f9b740 1 -- 172.21.15.18:0/942904499 wait complete. 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb088f9b740 1 -- start start 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb088f9b740 1 Processor -- start 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb088f9b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b0250 0x5572337c08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb088f9b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0b70 0x5572337bec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb088f9b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 0x5572337bf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb088f9b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572337bf800 con 0x5572337b9750 2024-02-17T12:58:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb088f9b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572337bf970 con 0x5572337b0b70 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb088f9b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557233820060 con 0x5572337b0250 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb088f9b740 10 client.0 _ll_register_callbacks cb 0x7fb070005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 0x5572337bf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 0x5572337bf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58636/0 (socket says 172.21.15.18:58636) 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb07f7fe640 1 -- 172.21.15.18:0/1743587406 learned_addr learned my addr 172.21.15.18:0/1743587406 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b0250 0x5572337c08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.587+0000 7fb07ffff640 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0b70 0x5572337bec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb07f7fe640 1 -- 172.21.15.18:0/1743587406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b0250 msgr2=0x5572337c08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b0250 0x5572337c08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb07f7fe640 1 -- 172.21.15.18:0/1743587406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0b70 msgr2=0x5572337bec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0b70 0x5572337bec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb07f7fe640 1 -- 172.21.15.18:0/1743587406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572336da060 con 0x5572337b9750 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb07f7fe640 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 0x5572337bf180 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fb07400bfc0 tx=0x7fb07400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af039142ea600714 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb05f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb05f7fe640 1 -- 172.21.15.18:0/1743587406 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074013260 con 0x5572337b9750 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb05f7fe640 1 -- 172.21.15.18:0/1743587406 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb074000e40 con 0x5572337b9750 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb088f9b740 1 -- 172.21.15.18:0/1743587406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb070004a60 con 0x5572337b9750 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb05f7fe640 1 -- 172.21.15.18:0/1743587406 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0740133c0 con 0x5572337b9750 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb088f9b740 10 client.18771 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb088f9b740 1 -- 172.21.15.18:0/1743587406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5572336d88c0 con 0x5572337b9750 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.588+0000 7fb05f7fe640 1 -- 172.21.15.18:0/1743587406 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb070004a60 con 0x5572337b9750 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb05f7fe640 1 -- 172.21.15.18:0/1743587406 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb074043460 con 0x5572337b9750 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb088f9b740 10 client.18771 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb088f9b740 10 client.18771 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb088f9b740 20 client.18771 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb088f9b740 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5572338215d0 0x5572338239f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb088f9b740 4 client.18771 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb088f9b740 1 -- 172.21.15.18:0/1743587406 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5572334a3890 con 0x5572338215d0 2024-02-17T12:58:57.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb07ffff640 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5572338215d0 0x5572338239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb05f7fe640 10 client.18771 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb07ffff640 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5572338215d0 0x5572338239f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c8359d014ef1dad server_cookie=394ca3fdc8131f7f in_seq=0 out_seq=0 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb05f7fe640 1 -- 172.21.15.18:0/1743587406 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5572334a3890 con 0x5572338215d0 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb05f7fe640 10 client.18771 handle_command_reply: tid=0 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.589+0000 7fb088f9b740 1 client.18771 shutdown 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb088f9b740 1 -- 172.21.15.18:0/1743587406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 msgr2=0x5572337bf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb088f9b740 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 0x5572337bf180 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fb07400bfc0 tx=0x7fb07400bbb0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb088f9b740 1 -- 172.21.15.18:0/1743587406 shutdown_connections 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb088f9b740 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572337b9750 0x5572337bf180 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb088f9b740 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5572338215d0 0x5572338239f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb088f9b740 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572337b0b70 0x5572337bec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb088f9b740 1 --2- 172.21.15.18:0/1743587406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5572337b0250 0x5572337c08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb05f7fe640 0 client.18771 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb088f9b740 1 -- 172.21.15.18:0/1743587406 >> 172.21.15.18:0/1743587406 conn(0x5572337aa980 msgr2=0x5572337b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb088f9b740 1 -- 172.21.15.18:0/1743587406 shutdown_connections 2024-02-17T12:58:57.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb088f9b740 1 -- 172.21.15.18:0/1743587406 wait complete. 2024-02-17T12:58:57.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb088f9b740 20 client.18771 trim_cache size 0 max 16384 2024-02-17T12:58:57.587 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:58:57.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb0863d0640 1 -- 172.21.15.18:0/2202922039 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb0680a2230 msgr2=0x7fb0680a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb0863d0640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb0680a2230 0x7fb0680a46f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb0780cbfb0 tx=0x7fb07806ab60 comp rx=0 tx=0).stop 2024-02-17T12:58:57.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb0863d0640 1 -- 172.21.15.18:0/2202922039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0801975f0 msgr2=0x7fb080197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb0863d0640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0801975f0 0x7fb080197aa0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb07800eaa0 tx=0x7fb07800ef60 comp rx=0 tx=0).stop 2024-02-17T12:58:57.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb0863d0640 1 -- 172.21.15.18:0/2202922039 shutdown_connections 2024-02-17T12:58:57.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb0863d0640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb0680a2230 0x7fb0680a46f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb0863d0640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0801975f0 0x7fb080197aa0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb0863d0640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb080105370 0x7fb08019d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb0863d0640 1 --2- 172.21.15.18:0/2202922039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0801002a0 0x7fb08019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb0863d0640 1 -- 172.21.15.18:0/2202922039 >> 172.21.15.18:0/2202922039 conn(0x7fb080076400 msgr2=0x7fb0800fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb0863d0640 1 -- 172.21.15.18:0/2202922039 shutdown_connections 2024-02-17T12:58:57.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.590+0000 7fb0863d0640 1 -- 172.21.15.18:0/2202922039 wait complete. 2024-02-17T12:58:57.591 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:58:57.591 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:58:57.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.705+0000 7f60aeb0d640 1 -- 172.21.15.18:0/3690306473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60a8101d40 msgr2=0x7f60a8105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.705+0000 7f60aeb0d640 1 --2- 172.21.15.18:0/3690306473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60a8101d40 0x7f60a8105d80 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f6094009930 tx=0x7f609402bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.705+0000 7f60aeb0d640 1 -- 172.21.15.18:0/3690306473 shutdown_connections 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.705+0000 7f60aeb0d640 1 --2- 172.21.15.18:0/3690306473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60a8106460 0x7f60a810e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.705+0000 7f60aeb0d640 1 --2- 172.21.15.18:0/3690306473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60a8101d40 0x7f60a8105d80 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.705+0000 7f60aeb0d640 1 --2- 172.21.15.18:0/3690306473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a8101390 0x7f60a8101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.705+0000 7f60aeb0d640 1 -- 172.21.15.18:0/3690306473 >> 172.21.15.18:0/3690306473 conn(0x7f60a80fcbb0 msgr2=0x7f60a80fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.705+0000 7f60aeb0d640 1 -- 172.21.15.18:0/3690306473 shutdown_connections 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.705+0000 7f60aeb0d640 1 -- 172.21.15.18:0/3690306473 wait complete. 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60aeb0d640 1 Processor -- start 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60aeb0d640 1 -- start start 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60aeb0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a8101390 0x7f60a819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60aeb0d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60a8106460 0x7f60a819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60aeb0d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60a8197600 0x7f60a8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60aeb0d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a8197ff0 con 0x7f60a8106460 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60aeb0d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a8198160 con 0x7f60a8101390 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60aeb0d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a8198320 con 0x7f60a8197600 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60a8106460 0x7f60a819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60a8106460 0x7f60a819d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58650/0 (socket says 172.21.15.18:58650) 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60a7fff640 1 -- 172.21.15.18:0/2792115388 learned_addr learned my addr 172.21.15.18:0/2792115388 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60ad0aa640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60a8197600 0x7f60a8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a8101390 0x7f60a819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60a7fff640 1 -- 172.21.15.18:0/2792115388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60a8197600 msgr2=0x7f60a8197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60a7fff640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60a8197600 0x7f60a8197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60a7fff640 1 -- 172.21.15.18:0/2792115388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a8101390 msgr2=0x7f60a819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60a7fff640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a8101390 0x7f60a819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.706+0000 7f60a7fff640 1 -- 172.21.15.18:0/2792115388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6094009560 con 0x7f60a8106460 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.707+0000 7f60a7fff640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60a8106460 0x7f60a819d990 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f6094000c00 tx=0x7f6094004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.707+0000 7f60a5ffb640 1 -- 172.21.15.18:0/2792115388 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6094004df0 con 0x7f60a8106460 2024-02-17T12:58:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.707+0000 7f60a5ffb640 1 -- 172.21.15.18:0/2792115388 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6094005430 con 0x7f60a8106460 2024-02-17T12:58:57.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.707+0000 7f60aeb0d640 1 -- 172.21.15.18:0/2792115388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a81a4310 con 0x7f60a8106460 2024-02-17T12:58:57.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.707+0000 7f60a5ffb640 1 -- 172.21.15.18:0/2792115388 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6094003e50 con 0x7f60a8106460 2024-02-17T12:58:57.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.707+0000 7f60aeb0d640 1 -- 172.21.15.18:0/2792115388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a81a47c0 con 0x7f60a8106460 2024-02-17T12:58:57.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.709+0000 7f60a5ffb640 1 -- 172.21.15.18:0/2792115388 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f60940055a0 con 0x7f60a8106460 2024-02-17T12:58:57.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.709+0000 7f60b17b8740 1 Processor -- start 2024-02-17T12:58:57.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60b17b8740 1 -- start start 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60b17b8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf3f250 0x556a5bf3f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60b17b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3fb70 0x556a5bf48210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60b17b8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf48750 0x556a5bf4ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60b17b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5befa960 con 0x556a5bf3fb70 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60b17b8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5beae310 con 0x556a5bf48750 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60b17b8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5beab840 con 0x556a5bf3f250 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60ac8a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3fb70 0x556a5bf48210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60ac8a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3fb70 0x556a5bf48210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58656/0 (socket says 172.21.15.18:58656) 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60ac8a9640 1 -- 172.21.15.18:0/1521323297 learned_addr learned my addr 172.21.15.18:0/1521323297 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60a5ffb640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f60980a2230 0x7f60980a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60a7fff640 1 --2- 172.21.15.18:0/1521323297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf48750 0x556a5bf4ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60ad0aa640 1 --2- 172.21.15.18:0/1521323297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf3f250 0x556a5bf3f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60ad0aa640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f60980a2230 0x7f60980a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60ac8a9640 1 -- 172.21.15.18:0/1521323297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf3f250 msgr2=0x556a5bf3f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/1521323297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf3f250 0x556a5bf3f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60ac8a9640 1 -- 172.21.15.18:0/1521323297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf48750 msgr2=0x556a5bf4ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/1521323297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf48750 0x556a5bf4ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60ac8a9640 1 -- 172.21.15.18:0/1521323297 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a5be69060 con 0x556a5bf3fb70 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60a7fff640 1 --2- 172.21.15.18:0/1521323297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf48750 0x556a5bf4ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60a5ffb640 1 -- 172.21.15.18:0/2792115388 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f60940f03e0 con 0x7f60a8106460 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.710+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/1521323297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3fb70 0x556a5bf48210 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f609c00ad00 tx=0x7f609c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a86f59edab5127a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f60ad0aa640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f60980a2230 0x7f60980a46f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009900 tx=0x7f60a0009650 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f6092ffd640 1 -- 172.21.15.18:0/1521323297 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c00a800 con 0x556a5bf3fb70 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f6092ffd640 1 -- 172.21.15.18:0/1521323297 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f609c018b20 con 0x556a5bf3fb70 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f6092ffd640 1 -- 172.21.15.18:0/1521323297 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c010d20 con 0x556a5bf3fb70 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f60b17b8740 1 -- 172.21.15.18:0/1521323297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3fb70 msgr2=0x556a5bf48210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f60b17b8740 1 --2- 172.21.15.18:0/1521323297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3fb70 0x556a5bf48210 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f609c00ad00 tx=0x7f609c00f790 comp rx=0 tx=0).stop 2024-02-17T12:58:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f60b17b8740 1 -- 172.21.15.18:0/1521323297 shutdown_connections 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f60b17b8740 1 --2- 172.21.15.18:0/1521323297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf48750 0x556a5bf4ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f60b17b8740 1 --2- 172.21.15.18:0/1521323297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3fb70 0x556a5bf48210 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f60b17b8740 1 --2- 172.21.15.18:0/1521323297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf3f250 0x556a5bf3f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f60b17b8740 1 -- 172.21.15.18:0/1521323297 >> 172.21.15.18:0/1521323297 conn(0x556a5bf39980 msgr2=0x556a5bf39d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f60b17b8740 1 -- 172.21.15.18:0/1521323297 shutdown_connections 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.711+0000 7f60b17b8740 1 -- 172.21.15.18:0/1521323297 wait complete. 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60b17b8740 1 -- start start 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60b17b8740 1 Processor -- start 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60b17b8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf3f250 0x556a5bf457a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60b17b8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf45ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60b17b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf48750 0x556a5bf46220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60b17b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5bea7020 con 0x556a5bf48750 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60b17b8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5bb7a430 con 0x556a5bf3fb70 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60b17b8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5bb36e00 con 0x556a5bf3f250 2024-02-17T12:58:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60b17b8740 10 client.0 _ll_register_callbacks cb 0x556a5be4ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60ad0aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf48750 0x556a5bf46220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60ad0aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf48750 0x556a5bf46220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58668/0 (socket says 172.21.15.18:58668) 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60ad0aa640 1 -- 172.21.15.18:0/347794572 learned_addr learned my addr 172.21.15.18:0/347794572 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf3f250 0x556a5bf457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.712+0000 7f60a7fff640 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf45ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60ad0aa640 1 -- 172.21.15.18:0/347794572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf3f250 msgr2=0x556a5bf457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60ad0aa640 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf3f250 0x556a5bf457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60ad0aa640 1 -- 172.21.15.18:0/347794572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 msgr2=0x556a5bf45ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60ad0aa640 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf45ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60ad0aa640 1 -- 172.21.15.18:0/347794572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609c00a960 con 0x556a5bf48750 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60a7fff640 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf45ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60ad0aa640 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf48750 0x556a5bf46220 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f60a0017e80 tx=0x7f60a0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5755b7221bb1ffc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f6087fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f6087fff640 1 -- 172.21.15.18:0/347794572 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a0020070 con 0x556a5bf48750 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f6087fff640 1 -- 172.21.15.18:0/347794572 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f60a000be30 con 0x556a5bf48750 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60b17b8740 1 -- 172.21.15.18:0/347794572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a5bba67e0 con 0x556a5bf48750 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f6087fff640 1 -- 172.21.15.18:0/347794572 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60a001bd90 con 0x556a5bf48750 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60b17b8740 10 client.18783 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60b17b8740 1 -- 172.21.15.18:0/347794572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x556a5bea56a0 con 0x556a5bf48750 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f6087fff640 1 -- 172.21.15.18:0/347794572 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556a5bba67e0 con 0x556a5bf48750 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f6087fff640 1 -- 172.21.15.18:0/347794572 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f60a0008040 con 0x556a5bf48750 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60b17b8740 10 client.18783 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60b17b8740 10 client.18783 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60b17b8740 20 client.18783 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60b17b8740 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556a5bf58c10 0x556a5bf5b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60b17b8740 4 client.18783 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.713+0000 7f60b17b8740 1 -- 172.21.15.18:0/347794572 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a5bbe3720 con 0x556a5bf58c10 2024-02-17T12:58:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.714+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556a5bf58c10 0x556a5bf5b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.714+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556a5bf58c10 0x556a5bf5b030 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=705f4ce8bfe931dd server_cookie=6170f5abf85f7244 in_seq=0 out_seq=0 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.714+0000 7f6087fff640 10 client.18783 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f6087fff640 1 -- 172.21.15.18:0/347794572 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a5bbe3720 con 0x556a5bf58c10 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f6087fff640 10 client.18783 handle_command_reply: tid=0 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f60b17b8740 1 client.18783 shutdown 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f60b17b8740 1 -- 172.21.15.18:0/347794572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf48750 msgr2=0x556a5bf46220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f60b17b8740 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf48750 0x556a5bf46220 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f60a0017e80 tx=0x7f60a0004b50 comp rx=0 tx=0).stop 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f60b17b8740 1 -- 172.21.15.18:0/347794572 shutdown_connections 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f60b17b8740 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556a5bf58c10 0x556a5bf5b030 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f60b17b8740 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf48750 0x556a5bf46220 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f6087fff640 0 client.18783 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f60b17b8740 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf45ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f60b17b8740 1 --2- 172.21.15.18:0/347794572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf3f250 0x556a5bf457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f60b17b8740 1 -- 172.21.15.18:0/347794572 >> 172.21.15.18:0/347794572 conn(0x556a5bf39980 msgr2=0x556a5bf3b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f60b17b8740 1 -- 172.21.15.18:0/347794572 shutdown_connections 2024-02-17T12:58:57.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f60b17b8740 1 -- 172.21.15.18:0/347794572 wait complete. 2024-02-17T12:58:57.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.715+0000 7f60b17b8740 20 client.18783 trim_cache size 0 max 16384 2024-02-17T12:58:57.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.727+0000 7f60b17b8740 1 Processor -- start 2024-02-17T12:58:57.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.727+0000 7f60b17b8740 1 -- start start 2024-02-17T12:58:57.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.727+0000 7f60b17b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 0x556a5bf47ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.727+0000 7f60b17b8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf46040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.727+0000 7f60b17b8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf48750 0x556a5bf42a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.727+0000 7f60b17b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5be2c130 con 0x556a5bf3f250 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.727+0000 7f60b17b8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5bf3eeb0 con 0x556a5bf3fb70 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.727+0000 7f60b17b8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5bf3a980 con 0x556a5bf48750 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.727+0000 7f60ac8a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 0x556a5bf47ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.727+0000 7f60ac8a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 0x556a5bf47ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58678/0 (socket says 172.21.15.18:58678) 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.727+0000 7f60ac8a9640 1 -- 172.21.15.18:0/214988349 learned_addr learned my addr 172.21.15.18:0/214988349 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60a7fff640 1 --2- 172.21.15.18:0/214988349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf46040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60ac8a9640 1 -- 172.21.15.18:0/214988349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf48750 msgr2=0x556a5bf42a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/214988349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf48750 0x556a5bf42a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60ac8a9640 1 -- 172.21.15.18:0/214988349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 msgr2=0x556a5bf46040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/214988349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf46040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60ac8a9640 1 -- 172.21.15.18:0/214988349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a5be41c20 con 0x556a5bf3f250 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60a7fff640 1 --2- 172.21.15.18:0/214988349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf46040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/214988349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 0x556a5bf47ed0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f609c00ad00 tx=0x7f609c0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72446837091c025a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60927fc640 1 -- 172.21.15.18:0/214988349 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c028070 con 0x556a5bf3f250 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60927fc640 1 -- 172.21.15.18:0/214988349 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f609c010e80 con 0x556a5bf3f250 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60927fc640 1 -- 172.21.15.18:0/214988349 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c0052a0 con 0x556a5bf3f250 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60b17b8740 1 -- 172.21.15.18:0/214988349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 msgr2=0x556a5bf47ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60b17b8740 1 --2- 172.21.15.18:0/214988349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 0x556a5bf47ed0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f609c00ad00 tx=0x7f609c0108e0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60b17b8740 1 -- 172.21.15.18:0/214988349 shutdown_connections 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60b17b8740 1 --2- 172.21.15.18:0/214988349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf48750 0x556a5bf42a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60b17b8740 1 --2- 172.21.15.18:0/214988349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf46040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60b17b8740 1 --2- 172.21.15.18:0/214988349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 0x556a5bf47ed0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.728+0000 7f60b17b8740 1 -- 172.21.15.18:0/214988349 >> 172.21.15.18:0/214988349 conn(0x556a5bf39980 msgr2=0x556a5bf40a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60b17b8740 1 -- 172.21.15.18:0/214988349 shutdown_connections 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60b17b8740 1 -- 172.21.15.18:0/214988349 wait complete. 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60b17b8740 1 -- start start 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60b17b8740 1 Processor -- start 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60b17b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 0x556a5bf4e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60b17b8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf4e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60b17b8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf48750 0x556a5bf4ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60b17b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5bfaf2d0 con 0x556a5bf3f250 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60b17b8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5bfaf440 con 0x556a5bf3fb70 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60b17b8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5bfaf600 con 0x556a5bf48750 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60b17b8740 10 client.0 _ll_register_callbacks cb 0x7f609c00a730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60ac8a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 0x556a5bf4e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60ac8a9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 0x556a5bf4e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58686/0 (socket says 172.21.15.18:58686) 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60ac8a9640 1 -- 172.21.15.18:0/2535515296 learned_addr learned my addr 172.21.15.18:0/2535515296 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60a7fff640 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf4e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.729+0000 7f60ad0aa640 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf48750 0x556a5bf4ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60ac8a9640 1 -- 172.21.15.18:0/2535515296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf48750 msgr2=0x556a5bf4ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf48750 0x556a5bf4ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60ac8a9640 1 -- 172.21.15.18:0/2535515296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 msgr2=0x556a5bf4e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf4e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60ac8a9640 1 -- 172.21.15.18:0/2535515296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a5be69060 con 0x556a5bf3f250 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60ad0aa640 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf48750 0x556a5bf4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60a7fff640 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf4e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60ac8a9640 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 0x556a5bf4e250 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f609c00f7d0 tx=0x7f609c00a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9079379bcd1c47 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f6087fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f6087fff640 1 -- 172.21.15.18:0/2535515296 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c028070 con 0x556a5bf3f250 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f6087fff640 1 -- 172.21.15.18:0/2535515296 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f609c018500 con 0x556a5bf3f250 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60b17b8740 1 -- 172.21.15.18:0/2535515296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a5bc32890 con 0x556a5bf3f250 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f6087fff640 1 -- 172.21.15.18:0/2535515296 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f609c010950 con 0x556a5bf3f250 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60b17b8740 10 client.18789 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60b17b8740 1 -- 172.21.15.18:0/2535515296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x556a5be678c0 con 0x556a5bf3f250 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f6087fff640 1 -- 172.21.15.18:0/2535515296 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556a5bc32890 con 0x556a5bf3f250 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f6087fff640 1 -- 172.21.15.18:0/2535515296 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f609c033020 con 0x556a5bf3f250 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60b17b8740 10 client.18789 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:57.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60b17b8740 10 client.18789 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60b17b8740 20 client.18789 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60b17b8740 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556a5bfb0a50 0x556a5bfb2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60b17b8740 4 client.18789 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.730+0000 7f60b17b8740 1 -- 172.21.15.18:0/2535515296 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a5bb52700 con 0x556a5bfb0a50 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f60a7fff640 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556a5bfb0a50 0x556a5bfb2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f60a7fff640 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556a5bfb0a50 0x556a5bfb2e40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8fdde858a40cd232 server_cookie=a6d3b3d00e382270 in_seq=0 out_seq=0 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f6087fff640 10 client.18789 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f6087fff640 1 -- 172.21.15.18:0/2535515296 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a5bb52700 con 0x556a5bfb0a50 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f6087fff640 10 client.18789 handle_command_reply: tid=0 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f60b17b8740 1 client.18789 shutdown 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f60b17b8740 1 -- 172.21.15.18:0/2535515296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 msgr2=0x556a5bf4e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f60b17b8740 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 0x556a5bf4e250 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f609c00f7d0 tx=0x7f609c00a6a0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f60b17b8740 1 -- 172.21.15.18:0/2535515296 shutdown_connections 2024-02-17T12:58:57.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f60b17b8740 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x556a5bfb0a50 0x556a5bfb2e40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f60b17b8740 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a5bf48750 0x556a5bf4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f60b17b8740 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a5bf3fb70 0x556a5bf4e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f6087fff640 0 client.18789 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:58:57.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f60b17b8740 1 --2- 172.21.15.18:0/2535515296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a5bf3f250 0x556a5bf4e250 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.731+0000 7f60b17b8740 1 -- 172.21.15.18:0/2535515296 >> 172.21.15.18:0/2535515296 conn(0x556a5bf39980 msgr2=0x556a5bf382f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60b17b8740 1 -- 172.21.15.18:0/2535515296 shutdown_connections 2024-02-17T12:58:57.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60b17b8740 1 -- 172.21.15.18:0/2535515296 wait complete. 2024-02-17T12:58:57.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60b17b8740 20 client.18789 trim_cache size 0 max 16384 2024-02-17T12:58:57.729 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:58:57.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60aeb0d640 1 -- 172.21.15.18:0/2792115388 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f60980a2230 msgr2=0x7f60980a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60aeb0d640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f60980a2230 0x7f60980a46f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009900 tx=0x7f60a0009650 comp rx=0 tx=0).stop 2024-02-17T12:58:57.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60aeb0d640 1 -- 172.21.15.18:0/2792115388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60a8106460 msgr2=0x7f60a819d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60aeb0d640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60a8106460 0x7f60a819d990 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f6094000c00 tx=0x7f6094004270 comp rx=0 tx=0).stop 2024-02-17T12:58:57.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60aeb0d640 1 -- 172.21.15.18:0/2792115388 shutdown_connections 2024-02-17T12:58:57.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60aeb0d640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f60980a2230 0x7f60980a46f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60aeb0d640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60a8197600 0x7f60a8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60aeb0d640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f60a8106460 0x7f60a819d990 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60aeb0d640 1 --2- 172.21.15.18:0/2792115388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a8101390 0x7f60a819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60aeb0d640 1 -- 172.21.15.18:0/2792115388 >> 172.21.15.18:0/2792115388 conn(0x7f60a80fcbb0 msgr2=0x7f60a80feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60aeb0d640 1 -- 172.21.15.18:0/2792115388 shutdown_connections 2024-02-17T12:58:57.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.732+0000 7f60aeb0d640 1 -- 172.21.15.18:0/2792115388 wait complete. 2024-02-17T12:58:57.733 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:58:57.733 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:58:57.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.847+0000 7f9a76c6f640 1 -- 172.21.15.18:0/3725969601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a70106450 msgr2=0x7f9a7010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.847+0000 7f9a76c6f640 1 --2- 172.21.15.18:0/3725969601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a70106450 0x7f9a7010e020 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009930 tx=0x7f9a6002bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:57.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.847+0000 7f9a76c6f640 1 -- 172.21.15.18:0/3725969601 shutdown_connections 2024-02-17T12:58:57.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.847+0000 7f9a76c6f640 1 --2- 172.21.15.18:0/3725969601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a70106450 0x7f9a7010e020 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.847+0000 7f9a76c6f640 1 --2- 172.21.15.18:0/3725969601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a70101d30 0x7f9a70105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.847+0000 7f9a76c6f640 1 --2- 172.21.15.18:0/3725969601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a70101380 0x7f9a70101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.847+0000 7f9a76c6f640 1 -- 172.21.15.18:0/3725969601 >> 172.21.15.18:0/3725969601 conn(0x7f9a70078580 msgr2=0x7f9a70078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.847+0000 7f9a76c6f640 1 -- 172.21.15.18:0/3725969601 shutdown_connections 2024-02-17T12:58:57.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.847+0000 7f9a76c6f640 1 -- 172.21.15.18:0/3725969601 wait complete. 2024-02-17T12:58:57.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.847+0000 7f9a76c6f640 1 Processor -- start 2024-02-17T12:58:57.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.847+0000 7f9a76c6f640 1 -- start start 2024-02-17T12:58:57.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a76c6f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a70101380 0x7f9a7019d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a76c6f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a70106450 0x7f9a7019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a76c6f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a70197650 0x7f9a70197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a76c6f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a70198120 con 0x7f9a70197650 2024-02-17T12:58:57.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a76c6f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a70198290 con 0x7f9a70101380 2024-02-17T12:58:57.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a76c6f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a70198450 con 0x7f9a70106450 2024-02-17T12:58:57.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a7520c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a70197650 0x7f9a70197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a7520c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a70197650 0x7f9a70197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58712/0 (socket says 172.21.15.18:58712) 2024-02-17T12:58:57.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a7520c640 1 -- 172.21.15.18:0/876582880 learned_addr learned my addr 172.21.15.18:0/876582880 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a74a0b640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a70101380 0x7f9a7019d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a70106450 0x7f9a7019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a7520c640 1 -- 172.21.15.18:0/876582880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a70106450 msgr2=0x7f9a7019dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a7520c640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a70106450 0x7f9a7019dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a7520c640 1 -- 172.21.15.18:0/876582880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a70101380 msgr2=0x7f9a7019d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a7520c640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a70101380 0x7f9a7019d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a7520c640 1 -- 172.21.15.18:0/876582880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a60009560 con 0x7f9a70197650 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a7520c640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a70197650 0x7f9a70197ad0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f9a60005940 tx=0x7f9a6002c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a6dffb640 1 -- 172.21.15.18:0/876582880 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600043d0 con 0x7f9a70197650 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a6dffb640 1 -- 172.21.15.18:0/876582880 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a600052d0 con 0x7f9a70197650 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a76c6f640 1 -- 172.21.15.18:0/876582880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a701a4390 con 0x7f9a70197650 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.848+0000 7f9a6dffb640 1 -- 172.21.15.18:0/876582880 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60003d10 con 0x7f9a70197650 2024-02-17T12:58:57.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.849+0000 7f9a76c6f640 1 -- 172.21.15.18:0/876582880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a701a4870 con 0x7f9a70197650 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.851+0000 7f9a6dffb640 1 -- 172.21.15.18:0/876582880 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9a60005440 con 0x7f9a70197650 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.851+0000 7f9a79841740 1 Processor -- start 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.851+0000 7f9a79841740 1 -- start start 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a79841740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26250 0x560b1af26630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a79841740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 0x560b1af2f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a79841740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af2f750 0x560b1af31b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a79841740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b1aee1960 con 0x560b1af26b70 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a79841740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b1ae95310 con 0x560b1af2f750 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a79841740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b1ae92840 con 0x560b1af26250 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a6dffb640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f9a580a2230 0x7f9a580a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a74a0b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 0x560b1af2f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a74a0b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 0x560b1af2f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58724/0 (socket says 172.21.15.18:58724) 2024-02-17T12:58:57.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a74a0b640 1 -- 172.21.15.18:0/1580902306 learned_addr learned my addr 172.21.15.18:0/1580902306 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a7520c640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f9a580a2230 0x7f9a580a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/1580902306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af2f750 0x560b1af31b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/1580902306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26250 0x560b1af26630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a6dffb640 1 -- 172.21.15.18:0/876582880 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9a600f0260 con 0x7f9a70197650 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a74a0b640 1 -- 172.21.15.18:0/1580902306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26250 msgr2=0x560b1af26630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a74a0b640 1 --2- 172.21.15.18:0/1580902306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26250 0x560b1af26630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a74a0b640 1 -- 172.21.15.18:0/1580902306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af2f750 msgr2=0x560b1af31b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a74a0b640 1 --2- 172.21.15.18:0/1580902306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af2f750 0x560b1af31b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a74a0b640 1 -- 172.21.15.18:0/1580902306 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b1ae50060 con 0x560b1af26b70 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a74a0b640 1 --2- 172.21.15.18:0/1580902306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 0x560b1af2f210 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009fa0 tx=0x7f9a6800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb97af3a18ae446 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a7520c640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f9a580a2230 0x7f9a580a46f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9a600b9750 tx=0x7f9a6008fdf0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.852+0000 7f9a56ffd640 1 -- 172.21.15.18:0/1580902306 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68011070 con 0x560b1af26b70 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.853+0000 7f9a56ffd640 1 -- 172.21.15.18:0/1580902306 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a68009ad0 con 0x560b1af26b70 2024-02-17T12:58:57.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.853+0000 7f9a56ffd640 1 -- 172.21.15.18:0/1580902306 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800a850 con 0x560b1af26b70 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.853+0000 7f9a79841740 1 -- 172.21.15.18:0/1580902306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 msgr2=0x560b1af2f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.853+0000 7f9a79841740 1 --2- 172.21.15.18:0/1580902306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 0x560b1af2f210 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009fa0 tx=0x7f9a6800c8b0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.853+0000 7f9a79841740 1 -- 172.21.15.18:0/1580902306 shutdown_connections 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.853+0000 7f9a79841740 1 --2- 172.21.15.18:0/1580902306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af2f750 0x560b1af31b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.853+0000 7f9a79841740 1 --2- 172.21.15.18:0/1580902306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 0x560b1af2f210 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.853+0000 7f9a79841740 1 --2- 172.21.15.18:0/1580902306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26250 0x560b1af26630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.853+0000 7f9a79841740 1 -- 172.21.15.18:0/1580902306 >> 172.21.15.18:0/1580902306 conn(0x560b1af20980 msgr2=0x560b1af20d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.853+0000 7f9a79841740 1 -- 172.21.15.18:0/1580902306 shutdown_connections 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.853+0000 7f9a79841740 1 -- 172.21.15.18:0/1580902306 wait complete. 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a79841740 1 -- start start 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a79841740 1 Processor -- start 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a79841740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af26250 0x560b1af2c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a79841740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 0x560b1af2cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a79841740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af2f750 0x560b1af2d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a79841740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b1ae8e020 con 0x560b1af26b70 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a79841740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b1ab61430 con 0x560b1af26250 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a79841740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b1ab1de00 con 0x560b1af2f750 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a79841740 10 client.0 _ll_register_callbacks cb 0x560b1ae35cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a74a0b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 0x560b1af2cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a74a0b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 0x560b1af2cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58728/0 (socket says 172.21.15.18:58728) 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a74a0b640 1 -- 172.21.15.18:0/259015131 learned_addr learned my addr 172.21.15.18:0/259015131 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af26250 0x560b1af2c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.854+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af2f750 0x560b1af2d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a74a0b640 1 -- 172.21.15.18:0/259015131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af2f750 msgr2=0x560b1af2d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a74a0b640 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af2f750 0x560b1af2d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a74a0b640 1 -- 172.21.15.18:0/259015131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af26250 msgr2=0x560b1af2c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a74a0b640 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af26250 0x560b1af2c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a74a0b640 1 -- 172.21.15.18:0/259015131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a68009c50 con 0x560b1af26b70 2024-02-17T12:58:57.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a74a0b640 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 0x560b1af2cce0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f9a68018040 tx=0x7f9a68015a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c575b4dd4f96b9ea server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a4ffff640 1 -- 172.21.15.18:0/259015131 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68011040 con 0x560b1af26b70 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a79841740 1 -- 172.21.15.18:0/259015131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b1abca720 con 0x560b1af26b70 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a4ffff640 1 -- 172.21.15.18:0/259015131 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9a6800a960 con 0x560b1af26b70 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a4ffff640 1 -- 172.21.15.18:0/259015131 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a6800ab20 con 0x560b1af26b70 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a79841740 10 client.18801 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a79841740 1 -- 172.21.15.18:0/259015131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x560b1ae8c6a0 con 0x560b1af26b70 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a4ffff640 1 -- 172.21.15.18:0/259015131 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560b1abca720 con 0x560b1af26b70 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.855+0000 7f9a4ffff640 1 -- 172.21.15.18:0/259015131 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9a6801c030 con 0x560b1af26b70 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.856+0000 7f9a79841740 10 client.18801 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.856+0000 7f9a79841740 10 client.18801 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.856+0000 7f9a79841740 20 client.18801 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.856+0000 7f9a79841740 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560b1af3fa60 0x560b1af41e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.856+0000 7f9a79841740 4 client.18801 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:57.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.856+0000 7f9a79841740 1 -- 172.21.15.18:0/259015131 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b1ac19890 con 0x560b1af3fa60 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.856+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560b1af3fa60 0x560b1af41e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.856+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560b1af3fa60 0x560b1af41e80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=799ef8dd014ecbe1 server_cookie=650ee7a8d6dbe01c in_seq=0 out_seq=0 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.856+0000 7f9a4ffff640 10 client.18801 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a4ffff640 1 -- 172.21.15.18:0/259015131 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560b1ac19890 con 0x560b1af3fa60 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a4ffff640 10 client.18801 handle_command_reply: tid=0 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a79841740 1 client.18801 shutdown 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a79841740 1 -- 172.21.15.18:0/259015131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 msgr2=0x560b1af2cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a79841740 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 0x560b1af2cce0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f9a68018040 tx=0x7f9a68015a90 comp rx=0 tx=0).stop 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a79841740 1 -- 172.21.15.18:0/259015131 shutdown_connections 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a79841740 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560b1af3fa60 0x560b1af41e80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a79841740 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af2f750 0x560b1af2d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a79841740 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26b70 0x560b1af2cce0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a4ffff640 0 client.18801 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:58:57.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a79841740 1 --2- 172.21.15.18:0/259015131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af26250 0x560b1af2c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a79841740 1 -- 172.21.15.18:0/259015131 >> 172.21.15.18:0/259015131 conn(0x560b1af20980 msgr2=0x560b1af226d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a79841740 1 -- 172.21.15.18:0/259015131 shutdown_connections 2024-02-17T12:58:57.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a79841740 1 -- 172.21.15.18:0/259015131 wait complete. 2024-02-17T12:58:57.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.857+0000 7f9a79841740 20 client.18801 trim_cache size 0 max 16384 2024-02-17T12:58:57.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a79841740 1 Processor -- start 2024-02-17T12:58:57.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a79841740 1 -- start start 2024-02-17T12:58:57.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a79841740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26250 0x560b1af2eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a79841740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af26b70 0x560b1af2d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a79841740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af2f750 0x560b1af29a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a79841740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b1ae13130 con 0x560b1af2f750 2024-02-17T12:58:57.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a79841740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b1af255d0 con 0x560b1af26b70 2024-02-17T12:58:57.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a79841740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b1af25eb0 con 0x560b1af26250 2024-02-17T12:58:57.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a6ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af2f750 0x560b1af29a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a6ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af2f750 0x560b1af29a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58740/0 (socket says 172.21.15.18:58740) 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a6ffff640 1 -- 172.21.15.18:0/3077556 learned_addr learned my addr 172.21.15.18:0/3077556 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/3077556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26250 0x560b1af2eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a6ffff640 1 -- 172.21.15.18:0/3077556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26250 msgr2=0x560b1af2eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.869+0000 7f9a74a0b640 1 --2- 172.21.15.18:0/3077556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af26b70 0x560b1af2d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/3077556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26250 0x560b1af2eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a6ffff640 1 -- 172.21.15.18:0/3077556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af26b70 msgr2=0x560b1af2d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/3077556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af26b70 0x560b1af2d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a6ffff640 1 -- 172.21.15.18:0/3077556 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b1ae28c20 con 0x560b1af2f750 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a74a0b640 1 --2- 172.21.15.18:0/3077556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af26b70 0x560b1af2d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/3077556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af2f750 0x560b1af29a00 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f9a640119c0 tx=0x7f9a6400ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ff0b5b3e8c34e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a567fc640 1 -- 172.21.15.18:0/3077556 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64005e50 con 0x560b1af2f750 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a567fc640 1 -- 172.21.15.18:0/3077556 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a64019040 con 0x560b1af2f750 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a567fc640 1 -- 172.21.15.18:0/3077556 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64005020 con 0x560b1af2f750 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a79841740 1 -- 172.21.15.18:0/3077556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af2f750 msgr2=0x560b1af29a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a79841740 1 --2- 172.21.15.18:0/3077556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af2f750 0x560b1af29a00 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f9a640119c0 tx=0x7f9a6400ea70 comp rx=0 tx=0).stop 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a79841740 1 -- 172.21.15.18:0/3077556 shutdown_connections 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a79841740 1 --2- 172.21.15.18:0/3077556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af2f750 0x560b1af29a00 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a79841740 1 --2- 172.21.15.18:0/3077556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af26b70 0x560b1af2d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a79841740 1 --2- 172.21.15.18:0/3077556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26250 0x560b1af2eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a79841740 1 -- 172.21.15.18:0/3077556 >> 172.21.15.18:0/3077556 conn(0x560b1af20980 msgr2=0x560b1af27a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a79841740 1 -- 172.21.15.18:0/3077556 shutdown_connections 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.870+0000 7f9a79841740 1 -- 172.21.15.18:0/3077556 wait complete. 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a79841740 1 -- start start 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a79841740 1 Processor -- start 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a79841740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26250 0x560b1af35380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a79841740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26b70 0x560b1af358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a79841740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af2f750 0x560b1af9a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a79841740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b1af36100 con 0x560b1af26250 2024-02-17T12:58:57.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a79841740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b1af9a900 con 0x560b1af2f750 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a79841740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b1af9aa40 con 0x560b1af26b70 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a79841740 10 client.0 _ll_register_callbacks cb 0x7f9a68009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a6ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26250 0x560b1af35380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a6ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26250 0x560b1af35380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58742/0 (socket says 172.21.15.18:58742) 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a6ffff640 1 -- 172.21.15.18:0/4064124315 learned_addr learned my addr 172.21.15.18:0/4064124315 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a74a0b640 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26b70 0x560b1af358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.871+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af2f750 0x560b1af9a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a6ffff640 1 -- 172.21.15.18:0/4064124315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26b70 msgr2=0x560b1af358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26b70 0x560b1af358c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a6ffff640 1 -- 172.21.15.18:0/4064124315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af2f750 msgr2=0x560b1af9a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af2f750 0x560b1af9a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a6ffff640 1 -- 172.21.15.18:0/4064124315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a64011b50 con 0x560b1af26250 2024-02-17T12:58:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a6ffff640 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26250 0x560b1af35380 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f9a64001eb0 tx=0x7f9a64004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbd4ea3f7f70790 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a4ffff640 1 -- 172.21.15.18:0/4064124315 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64004520 con 0x560b1af26250 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a4ffff640 1 -- 172.21.15.18:0/4064124315 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9a64000e40 con 0x560b1af26250 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a79841740 1 -- 172.21.15.18:0/4064124315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b1ab8d7e0 con 0x560b1af26250 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a4ffff640 1 -- 172.21.15.18:0/4064124315 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a64005ed0 con 0x560b1af26250 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a79841740 10 client.18807 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a79841740 1 -- 172.21.15.18:0/4064124315 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x560b1ae4e8c0 con 0x560b1af26250 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a4ffff640 1 -- 172.21.15.18:0/4064124315 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560b1ab8d7e0 con 0x560b1af26250 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a4ffff640 1 -- 172.21.15.18:0/4064124315 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9a64004750 con 0x560b1af26250 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.872+0000 7f9a79841740 10 client.18807 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 10 client.18807 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 20 client.18807 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560b1af9c010 0x560b1af9e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 4 client.18807 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 1 -- 172.21.15.18:0/4064124315 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560b1ab39700 con 0x560b1af9c010 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a74a0b640 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560b1af9c010 0x560b1af9e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a74a0b640 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560b1af9c010 0x560b1af9e430 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5bd3c423e8d309 server_cookie=f4e275732b939991 in_seq=0 out_seq=0 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a4ffff640 10 client.18807 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a4ffff640 1 -- 172.21.15.18:0/4064124315 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560b1ab39700 con 0x560b1af9c010 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a4ffff640 10 client.18807 handle_command_reply: tid=0 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 1 client.18807 shutdown 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 1 -- 172.21.15.18:0/4064124315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26250 msgr2=0x560b1af35380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26250 0x560b1af35380 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f9a64001eb0 tx=0x7f9a64004180 comp rx=0 tx=0).stop 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 1 -- 172.21.15.18:0/4064124315 shutdown_connections 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560b1af9c010 0x560b1af9e430 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b1af2f750 0x560b1af9a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b1af26b70 0x560b1af358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a4ffff640 0 client.18807 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:58:57.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 1 --2- 172.21.15.18:0/4064124315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b1af26250 0x560b1af35380 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.873+0000 7f9a79841740 1 -- 172.21.15.18:0/4064124315 >> 172.21.15.18:0/4064124315 conn(0x560b1af20980 msgr2=0x560b1af1f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a79841740 1 -- 172.21.15.18:0/4064124315 shutdown_connections 2024-02-17T12:58:57.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a79841740 1 -- 172.21.15.18:0/4064124315 wait complete. 2024-02-17T12:58:57.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a79841740 20 client.18807 trim_cache size 0 max 16384 2024-02-17T12:58:57.871 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:58:57.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a76c6f640 1 -- 172.21.15.18:0/876582880 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f9a580a2230 msgr2=0x7f9a580a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a76c6f640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f9a580a2230 0x7f9a580a46f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9a600b9750 tx=0x7f9a6008fdf0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a76c6f640 1 -- 172.21.15.18:0/876582880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a70197650 msgr2=0x7f9a70197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a76c6f640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a70197650 0x7f9a70197ad0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f9a60005940 tx=0x7f9a6002c6f0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a76c6f640 1 -- 172.21.15.18:0/876582880 shutdown_connections 2024-02-17T12:58:57.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a76c6f640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f9a580a2230 0x7f9a580a46f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a76c6f640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a70197650 0x7f9a70197ad0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a76c6f640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a70106450 0x7f9a7019dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a76c6f640 1 --2- 172.21.15.18:0/876582880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a70101380 0x7f9a7019d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a76c6f640 1 -- 172.21.15.18:0/876582880 >> 172.21.15.18:0/876582880 conn(0x7f9a70078580 msgr2=0x7f9a700ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a76c6f640 1 -- 172.21.15.18:0/876582880 shutdown_connections 2024-02-17T12:58:57.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.874+0000 7f9a76c6f640 1 -- 172.21.15.18:0/876582880 wait complete. 2024-02-17T12:58:57.875 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:58:57.875 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:58:57.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.989+0000 7f2d28c15640 1 -- 172.21.15.18:0/2812662309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d24100260 msgr2=0x7f2d24100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.989+0000 7f2d28c15640 1 --2- 172.21.15.18:0/2812662309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d24100260 0x7f2d24100640 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009930 tx=0x7f2d1c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:57.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.989+0000 7f2d28c15640 1 -- 172.21.15.18:0/2812662309 shutdown_connections 2024-02-17T12:58:57.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.989+0000 7f2d28c15640 1 --2- 172.21.15.18:0/2812662309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d24105330 0x7f2d2410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.989+0000 7f2d28c15640 1 --2- 172.21.15.18:0/2812662309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d24100c10 0x7f2d24104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.989+0000 7f2d28c15640 1 --2- 172.21.15.18:0/2812662309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d24100260 0x7f2d24100640 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.990+0000 7f2d28c15640 1 -- 172.21.15.18:0/2812662309 >> 172.21.15.18:0/2812662309 conn(0x7f2d24077180 msgr2=0x7f2d24077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.990+0000 7f2d28c15640 1 -- 172.21.15.18:0/2812662309 shutdown_connections 2024-02-17T12:58:57.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.990+0000 7f2d28c15640 1 -- 172.21.15.18:0/2812662309 wait complete. 2024-02-17T12:58:57.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.990+0000 7f2d28c15640 1 Processor -- start 2024-02-17T12:58:57.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.990+0000 7f2d28c15640 1 -- start start 2024-02-17T12:58:57.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.990+0000 7f2d28c15640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d24100260 0x7f2d2419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.990+0000 7f2d28c15640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d24105330 0x7f2d2419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.990+0000 7f2d28c15640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d241975d0 0x7f2d24197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.990+0000 7f2d28c15640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d24197fc0 con 0x7f2d24105330 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.990+0000 7f2d28c15640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d24198130 con 0x7f2d241975d0 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.990+0000 7f2d28c15640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241982f0 con 0x7f2d24100260 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d21d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d24105330 0x7f2d2419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d21d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d24105330 0x7f2d2419d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58766/0 (socket says 172.21.15.18:58766) 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d21d9b640 1 -- 172.21.15.18:0/3520362644 learned_addr learned my addr 172.21.15.18:0/3520362644 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d241975d0 0x7f2d24197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d2259c640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d24100260 0x7f2d2419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d21d9b640 1 -- 172.21.15.18:0/3520362644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d24100260 msgr2=0x7f2d2419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d21d9b640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d24100260 0x7f2d2419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d21d9b640 1 -- 172.21.15.18:0/3520362644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d241975d0 msgr2=0x7f2d24197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d21d9b640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d241975d0 0x7f2d24197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d21d9b640 1 -- 172.21.15.18:0/3520362644 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c009560 con 0x7f2d24105330 2024-02-17T12:58:57.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d21d9b640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d24105330 0x7f2d2419d960 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00d770 tx=0x7f2d0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d1b7fe640 1 -- 172.21.15.18:0/3520362644 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c009c30 con 0x7f2d24105330 2024-02-17T12:58:57.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d28c15640 1 -- 172.21.15.18:0/3520362644 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d241a42e0 con 0x7f2d24105330 2024-02-17T12:58:57.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d1b7fe640 1 -- 172.21.15.18:0/3520362644 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d0c00de80 con 0x7f2d24105330 2024-02-17T12:58:57.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d1b7fe640 1 -- 172.21.15.18:0/3520362644 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00f6b0 con 0x7f2d24105330 2024-02-17T12:58:57.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.991+0000 7f2d28c15640 1 -- 172.21.15.18:0/3520362644 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d241a4840 con 0x7f2d24105330 2024-02-17T12:58:57.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.993+0000 7f2d1b7fe640 1 -- 172.21.15.18:0/3520362644 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2d0c010460 con 0x7f2d24105330 2024-02-17T12:58:57.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d2b7eb740 1 Processor -- start 2024-02-17T12:58:57.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d2b7eb740 1 -- start start 2024-02-17T12:58:57.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d2b7eb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 0x55d5c787a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d2b7eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c787a820 0x55d5c7887620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d2b7eb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c7887b60 0x55d5c7889f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d2b7eb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c7835960 con 0x55d5c7879f00 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d2b7eb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c77e9310 con 0x55d5c7887b60 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d2b7eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c77e6840 con 0x55d5c787a820 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d1b7fe640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2d100a2230 0x7f2d100a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d22d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 0x55d5c787a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d22d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 0x55d5c787a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58780/0 (socket says 172.21.15.18:58780) 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d22d9d640 1 -- 172.21.15.18:0/4170450218 learned_addr learned my addr 172.21.15.18:0/4170450218 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d21d9b640 1 --2- 172.21.15.18:0/4170450218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c7887b60 0x55d5c7889f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2d100a2230 0x7f2d100a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d1b7fe640 1 -- 172.21.15.18:0/3520362644 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2d0c0cde50 con 0x7f2d24105330 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.994+0000 7f2d2259c640 1 --2- 172.21.15.18:0/4170450218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c787a820 0x55d5c7887620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.995+0000 7f2d22d9d640 1 -- 172.21.15.18:0/4170450218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c787a820 msgr2=0x55d5c7887620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.995+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/4170450218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c787a820 0x55d5c7887620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.995+0000 7f2d22d9d640 1 -- 172.21.15.18:0/4170450218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c7887b60 msgr2=0x55d5c7889f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.995+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/4170450218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c7887b60 0x55d5c7889f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.995+0000 7f2d22d9d640 1 -- 172.21.15.18:0/4170450218 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5c77a4060 con 0x55d5c7879f00 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.995+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/4170450218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 0x55d5c787a2e0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f2d14011da0 tx=0x7f2d1400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be460b6e581ba9ce server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.995+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2d100a2230 0x7f2d100a46f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400fc90 tx=0x7f2d1401e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.995+0000 7f2d18ff9640 1 -- 172.21.15.18:0/4170450218 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1401dbf0 con 0x55d5c7879f00 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.995+0000 7f2d18ff9640 1 -- 172.21.15.18:0/4170450218 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d14009dc0 con 0x55d5c7879f00 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.995+0000 7f2d18ff9640 1 -- 172.21.15.18:0/4170450218 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14004290 con 0x55d5c7879f00 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.996+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/4170450218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 msgr2=0x55d5c787a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.996+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/4170450218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 0x55d5c787a2e0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f2d14011da0 tx=0x7f2d1400f790 comp rx=0 tx=0).stop 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.996+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/4170450218 shutdown_connections 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.996+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/4170450218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c7887b60 0x55d5c7889f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.996+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/4170450218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c787a820 0x55d5c7887620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.996+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/4170450218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 0x55d5c787a2e0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.996+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/4170450218 >> 172.21.15.18:0/4170450218 conn(0x55d5c7874980 msgr2=0x55d5c7874d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.996+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/4170450218 shutdown_connections 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.996+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/4170450218 wait complete. 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d2b7eb740 1 -- start start 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d2b7eb740 1 Processor -- start 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d2b7eb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c7879f00 0x55d5c7884b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d2b7eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c787a820 0x55d5c78850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d2b7eb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 0x55d5c78855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d2b7eb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c77e2020 con 0x55d5c7887b60 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d2b7eb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c74b5430 con 0x55d5c7879f00 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d2b7eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c7471e00 con 0x55d5c787a820 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d2b7eb740 10 client.0 _ll_register_callbacks cb 0x55d5c7789cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d22d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 0x55d5c78855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d22d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 0x55d5c78855e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58788/0 (socket says 172.21.15.18:58788) 2024-02-17T12:58:57.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d22d9d640 1 -- 172.21.15.18:0/2462521622 learned_addr learned my addr 172.21.15.18:0/2462521622 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d21d9b640 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c787a820 0x55d5c78850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d2259c640 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c7879f00 0x55d5c7884b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d22d9d640 1 -- 172.21.15.18:0/2462521622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c787a820 msgr2=0x55d5c78850a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c787a820 0x55d5c78850a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d22d9d640 1 -- 172.21.15.18:0/2462521622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c7879f00 msgr2=0x55d5c7884b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c7879f00 0x55d5c7884b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.997+0000 7f2d22d9d640 1 -- 172.21.15.18:0/2462521622 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d14019040 con 0x55d5c7887b60 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.998+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 0x55d5c78855e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009540 tx=0x7f2d14003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d0f5702a0298a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.998+0000 7f2d09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d09ffb640 1 -- 172.21.15.18:0/2462521622 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14033070 con 0x55d5c7887b60 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d09ffb640 1 -- 172.21.15.18:0/2462521622 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2d14011910 con 0x55d5c7887b60 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2462521622 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5c74e17e0 con 0x55d5c7887b60 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d09ffb640 1 -- 172.21.15.18:0/2462521622 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d140052d0 con 0x55d5c7887b60 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d2b7eb740 10 client.18819 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:57.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2462521622 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55d5c77e06a0 con 0x55d5c7887b60 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d09ffb640 1 -- 172.21.15.18:0/2462521622 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d5c74e17e0 con 0x55d5c7887b60 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d09ffb640 1 -- 172.21.15.18:0/2462521622 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2d14036020 con 0x55d5c7887b60 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d2b7eb740 10 client.18819 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d2b7eb740 10 client.18819 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d2b7eb740 20 client.18819 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55d5c7897f50 0x55d5c789a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d2b7eb740 4 client.18819 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:57.999+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2462521622 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5c751e720 con 0x55d5c7897f50 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.000+0000 7f2d2259c640 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55d5c7897f50 0x55d5c789a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.000+0000 7f2d2259c640 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55d5c7897f50 0x55d5c789a370 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d3d4bd77187e47f server_cookie=b11ec667726f134a in_seq=0 out_seq=0 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.000+0000 7f2d09ffb640 10 client.18819 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d09ffb640 1 -- 172.21.15.18:0/2462521622 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5c751e720 con 0x55d5c7897f50 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d09ffb640 10 client.18819 handle_command_reply: tid=0 2024-02-17T12:58:57.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d2b7eb740 1 client.18819 shutdown 2024-02-17T12:58:57.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2462521622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 msgr2=0x55d5c78855e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:57.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 0x55d5c78855e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009540 tx=0x7f2d14003b90 comp rx=0 tx=0).stop 2024-02-17T12:58:57.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2462521622 shutdown_connections 2024-02-17T12:58:57.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55d5c7897f50 0x55d5c789a370 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 0x55d5c78855e0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c787a820 0x55d5c78850a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d09ffb640 0 client.18819 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:58:57.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/2462521622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c7879f00 0x55d5c7884b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:57.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2462521622 >> 172.21.15.18:0/2462521622 conn(0x55d5c7874980 msgr2=0x55d5c78766d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:57.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2462521622 shutdown_connections 2024-02-17T12:58:57.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2462521622 wait complete. 2024-02-17T12:58:57.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.001+0000 7f2d2b7eb740 20 client.18819 trim_cache size 0 max 16384 2024-02-17T12:58:58.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d2b7eb740 1 Processor -- start 2024-02-17T12:58:58.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d2b7eb740 1 -- start start 2024-02-17T12:58:58.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d2b7eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c7879f00 0x55d5c7887290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d2b7eb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c787a820 0x55d5c7885400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d2b7eb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 0x55d5c787fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d2b7eb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c7767130 con 0x55d5c7887b60 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d2b7eb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c788a5f0 con 0x55d5c787a820 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d2b7eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c7897d00 con 0x55d5c7879f00 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d22d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 0x55d5c787fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d22d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 0x55d5c787fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58800/0 (socket says 172.21.15.18:58800) 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d22d9d640 1 -- 172.21.15.18:0/2201323890 learned_addr learned my addr 172.21.15.18:0/2201323890 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d22d9d640 1 -- 172.21.15.18:0/2201323890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c7879f00 msgr2=0x55d5c7887290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/2201323890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c7879f00 0x55d5c7887290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d22d9d640 1 -- 172.21.15.18:0/2201323890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c787a820 msgr2=0x55d5c7885400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/2201323890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c787a820 0x55d5c7885400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d22d9d640 1 -- 172.21.15.18:0/2201323890 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5c777cc20 con 0x55d5c7887b60 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.013+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/2201323890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 0x55d5c787fe40 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f2d14011fd0 tx=0x7f2d14011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8310df9b889f52 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.014+0000 7f2d0bfff640 1 -- 172.21.15.18:0/2201323890 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14033070 con 0x55d5c7887b60 2024-02-17T12:58:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.014+0000 7f2d0bfff640 1 -- 172.21.15.18:0/2201323890 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d14026d20 con 0x55d5c7887b60 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.014+0000 7f2d0bfff640 1 -- 172.21.15.18:0/2201323890 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14003970 con 0x55d5c7887b60 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.014+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2201323890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 msgr2=0x55d5c787fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.014+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/2201323890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 0x55d5c787fe40 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f2d14011fd0 tx=0x7f2d14011ba0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.014+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2201323890 shutdown_connections 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.014+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/2201323890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7887b60 0x55d5c787fe40 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.014+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/2201323890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c787a820 0x55d5c7885400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.014+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/2201323890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c7879f00 0x55d5c7887290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.014+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2201323890 >> 172.21.15.18:0/2201323890 conn(0x55d5c7874980 msgr2=0x55d5c786fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.014+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2201323890 shutdown_connections 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.014+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/2201323890 wait complete. 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2b7eb740 1 -- start start 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2b7eb740 1 Processor -- start 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2b7eb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 0x55d5c788ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2b7eb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c787a820 0x55d5c788d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2b7eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c7887b60 0x55d5c788d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2b7eb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c788deb0 con 0x55d5c7879f00 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2b7eb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c788e020 con 0x55d5c787a820 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2b7eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c78ee460 con 0x55d5c7887b60 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2b7eb740 10 client.0 _ll_register_callbacks cb 0x7f2d14011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 0x55d5c788ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 0x55d5c788ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58812/0 (socket says 172.21.15.18:58812) 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2259c640 1 -- 172.21.15.18:0/1294977009 learned_addr learned my addr 172.21.15.18:0/1294977009 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d21d9b640 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c787a820 0x55d5c788d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2259c640 1 -- 172.21.15.18:0/1294977009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c7887b60 msgr2=0x55d5c788d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d22d9d640 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c7887b60 0x55d5c788d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2259c640 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c7887b60 0x55d5c788d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2259c640 1 -- 172.21.15.18:0/1294977009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c787a820 msgr2=0x55d5c788d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2259c640 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c787a820 0x55d5c788d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2259c640 1 -- 172.21.15.18:0/1294977009 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5c77e06a0 con 0x55d5c7879f00 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d21d9b640 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c787a820 0x55d5c788d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d2259c640 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 0x55d5c788ee70 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c036040 tx=0x7f2d1c02c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97472e7d2790c8c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.015+0000 7f2d09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d09ffb640 1 -- 172.21.15.18:0/1294977009 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c004b00 con 0x55d5c7879f00 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d09ffb640 1 -- 172.21.15.18:0/1294977009 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2d1c044040 con 0x55d5c7879f00 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/1294977009 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d14009dc0 con 0x55d5c7879f00 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d09ffb640 1 -- 172.21.15.18:0/1294977009 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d1c004040 con 0x55d5c7879f00 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d2b7eb740 10 client.18825 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/1294977009 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55d5c77a28c0 con 0x55d5c7879f00 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d09ffb640 1 -- 172.21.15.18:0/1294977009 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d14009dc0 con 0x55d5c7879f00 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d09ffb640 1 -- 172.21.15.18:0/1294977009 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2d1c047670 con 0x55d5c7879f00 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d2b7eb740 10 client.18825 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d2b7eb740 10 client.18825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:58:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d2b7eb740 20 client.18825 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55d5c78efc60 0x55d5c78f2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d2b7eb740 4 client.18825 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/1294977009 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d5c756d890 con 0x55d5c78efc60 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.016+0000 7f2d21d9b640 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55d5c78efc60 0x55d5c78f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d09ffb640 10 client.18825 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d21d9b640 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55d5c78efc60 0x55d5c78f2080 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41c9e1df52f1b18a server_cookie=bd142918f8de7b5d in_seq=0 out_seq=0 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d09ffb640 1 -- 172.21.15.18:0/1294977009 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d5c756d890 con 0x55d5c78efc60 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d09ffb640 10 client.18825 handle_command_reply: tid=0 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d2b7eb740 1 client.18825 shutdown 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/1294977009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 msgr2=0x55d5c788ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 0x55d5c788ee70 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c036040 tx=0x7f2d1c02c1f0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/1294977009 shutdown_connections 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55d5c78efc60 0x55d5c78f2080 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5c7887b60 0x55d5c788d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5c787a820 0x55d5c788d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d09ffb640 0 client.18825 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:58:58.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d2b7eb740 1 --2- 172.21.15.18:0/1294977009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d5c7879f00 0x55d5c788ee70 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/1294977009 >> 172.21.15.18:0/1294977009 conn(0x55d5c7874980 msgr2=0x55d5c787fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:58.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/1294977009 shutdown_connections 2024-02-17T12:58:58.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d2b7eb740 1 -- 172.21.15.18:0/1294977009 wait complete. 2024-02-17T12:58:58.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.017+0000 7f2d2b7eb740 20 client.18825 trim_cache size 0 max 16384 2024-02-17T12:58:58.014 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:58:58.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.018+0000 7f2d28c15640 1 -- 172.21.15.18:0/3520362644 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2d100a2230 msgr2=0x7f2d100a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.018+0000 7f2d28c15640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2d100a2230 0x7f2d100a46f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400fc90 tx=0x7f2d1401e000 comp rx=0 tx=0).stop 2024-02-17T12:58:58.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.018+0000 7f2d28c15640 1 -- 172.21.15.18:0/3520362644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d24105330 msgr2=0x7f2d2419d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.018+0000 7f2d28c15640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d24105330 0x7f2d2419d960 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00d770 tx=0x7f2d0c00dc30 comp rx=0 tx=0).stop 2024-02-17T12:58:58.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.018+0000 7f2d28c15640 1 -- 172.21.15.18:0/3520362644 shutdown_connections 2024-02-17T12:58:58.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.018+0000 7f2d28c15640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2d100a2230 0x7f2d100a46f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.018+0000 7f2d28c15640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d241975d0 0x7f2d24197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.018+0000 7f2d28c15640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d24105330 0x7f2d2419d960 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.018+0000 7f2d28c15640 1 --2- 172.21.15.18:0/3520362644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d24100260 0x7f2d2419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.018+0000 7f2d28c15640 1 -- 172.21.15.18:0/3520362644 >> 172.21.15.18:0/3520362644 conn(0x7f2d24077180 msgr2=0x7f2d240fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:58.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.018+0000 7f2d28c15640 1 -- 172.21.15.18:0/3520362644 shutdown_connections 2024-02-17T12:58:58.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.018+0000 7f2d28c15640 1 -- 172.21.15.18:0/3520362644 wait complete. 2024-02-17T12:58:58.018 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:58:58.019 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:58:58.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.137+0000 7f7ee69c7640 1 -- 172.21.15.18:0/2910918264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ee00ffa40 msgr2=0x7f7ee00ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.137+0000 7f7ee69c7640 1 --2- 172.21.15.18:0/2910918264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ee00ffa40 0x7f7ee00ffec0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009900 tx=0x7f7ed002bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:58.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.137+0000 7f7ee69c7640 1 -- 172.21.15.18:0/2910918264 shutdown_connections 2024-02-17T12:58:58.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.137+0000 7f7ee69c7640 1 --2- 172.21.15.18:0/2910918264 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee0100400 0x7f7ee010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.137+0000 7f7ee69c7640 1 --2- 172.21.15.18:0/2910918264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ee00ffa40 0x7f7ee00ffec0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.137+0000 7f7ee69c7640 1 --2- 172.21.15.18:0/2910918264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee0107380 0x7f7ee0107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.137+0000 7f7ee69c7640 1 -- 172.21.15.18:0/2910918264 >> 172.21.15.18:0/2910918264 conn(0x7f7ee00753c0 msgr2=0x7f7ee00757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:58.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.137+0000 7f7ee69c7640 1 -- 172.21.15.18:0/2910918264 shutdown_connections 2024-02-17T12:58:58.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.137+0000 7f7ee69c7640 1 -- 172.21.15.18:0/2910918264 wait complete. 2024-02-17T12:58:58.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7ee69c7640 1 Processor -- start 2024-02-17T12:58:58.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7ee69c7640 1 -- start start 2024-02-17T12:58:58.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7ee69c7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee00ffa40 0x7f7ee0100c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7ee69c7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ee0100400 0x7f7ee0101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7ee69c7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee0107380 0x7f7ee01016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7ee69c7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee01a5c20 con 0x7f7ee0100400 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7ee69c7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee01a5d90 con 0x7f7ee0107380 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7ee69c7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee01a5f50 con 0x7f7ee00ffa40 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7edf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ee0100400 0x7f7ee0101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7edf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ee0100400 0x7f7ee0101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58832/0 (socket says 172.21.15.18:58832) 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7edf7fe640 1 -- 172.21.15.18:0/1472879417 learned_addr learned my addr 172.21.15.18:0/1472879417 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7edffff640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee00ffa40 0x7f7ee0100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.138+0000 7f7ee4f64640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee0107380 0x7f7ee01016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.139+0000 7f7edf7fe640 1 -- 172.21.15.18:0/1472879417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee00ffa40 msgr2=0x7f7ee0100c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.139+0000 7f7edf7fe640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee00ffa40 0x7f7ee0100c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.139+0000 7f7edf7fe640 1 -- 172.21.15.18:0/1472879417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee0107380 msgr2=0x7f7ee01016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.139+0000 7f7edf7fe640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee0107380 0x7f7ee01016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.139+0000 7f7edf7fe640 1 -- 172.21.15.18:0/1472879417 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed0009560 con 0x7f7ee0100400 2024-02-17T12:58:58.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.139+0000 7f7edf7fe640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ee0100400 0x7f7ee0101160 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0005940 tx=0x7f7ed0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:58.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.139+0000 7f7edd7fa640 1 -- 172.21.15.18:0/1472879417 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0004440 con 0x7f7ee0100400 2024-02-17T12:58:58.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.139+0000 7f7ee69c7640 1 -- 172.21.15.18:0/1472879417 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee01a61d0 con 0x7f7ee0100400 2024-02-17T12:58:58.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.139+0000 7f7edd7fa640 1 -- 172.21.15.18:0/1472879417 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ed00052d0 con 0x7f7ee0100400 2024-02-17T12:58:58.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.139+0000 7f7edd7fa640 1 -- 172.21.15.18:0/1472879417 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0003da0 con 0x7f7ee0100400 2024-02-17T12:58:58.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.139+0000 7f7ee69c7640 1 -- 172.21.15.18:0/1472879417 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee01a6730 con 0x7f7ee0100400 2024-02-17T12:58:58.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.141+0000 7f7edd7fa640 1 -- 172.21.15.18:0/1472879417 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7ed0005440 con 0x7f7ee0100400 2024-02-17T12:58:58.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.141+0000 7f7ee9597740 1 Processor -- start 2024-02-17T12:58:58.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.141+0000 7f7ee9597740 1 -- start start 2024-02-17T12:58:58.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.141+0000 7f7ee9597740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834250 0x55ea15834630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7ee9597740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834b70 0x55ea1583d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7ee9597740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 0x55ea1583fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7ee9597740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea157ef960 con 0x55ea1583d750 2024-02-17T12:58:58.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7ee9597740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea157a3310 con 0x55ea15834b70 2024-02-17T12:58:58.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7ee9597740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea157a0840 con 0x55ea15834250 2024-02-17T12:58:58.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 0x55ea1583fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 0x55ea1583fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58844/0 (socket says 172.21.15.18:58844) 2024-02-17T12:58:58.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834b70 0x55ea1583d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edf7fe640 1 -- 172.21.15.18:0/1534671048 learned_addr learned my addr 172.21.15.18:0/1534671048 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:58.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edd7fa640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7ec80a2230 0x7f7ec80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7ee4f64640 1 --2- 172.21.15.18:0/1534671048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834250 0x55ea15834630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7ee4f64640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7ec80a2230 0x7f7ec80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edd7fa640 1 -- 172.21.15.18:0/1472879417 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7ed00f00c0 con 0x7f7ee0100400 2024-02-17T12:58:58.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edffff640 1 -- 172.21.15.18:0/1534671048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834250 msgr2=0x55ea15834630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:58.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edffff640 1 --2- 172.21.15.18:0/1534671048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834250 0x55ea15834630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edffff640 1 -- 172.21.15.18:0/1534671048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 msgr2=0x55ea1583fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:58.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edffff640 1 --2- 172.21.15.18:0/1534671048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 0x55ea1583fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edffff640 1 -- 172.21.15.18:0/1534671048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea1575e060 con 0x55ea15834b70 2024-02-17T12:58:58.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edf7fe640 1 --2- 172.21.15.18:0/1534671048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 0x55ea1583fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7edffff640 1 --2- 172.21.15.18:0/1534671048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834b70 0x55ea1583d210 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400bb70 tx=0x7f7ed40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a92842c27fe5dde server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7ee4f64640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7ec80a2230 0x7f7ec80a46f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800a780 tx=0x7f7ed800a4d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.142+0000 7f7ec67fc640 1 -- 172.21.15.18:0/1534671048 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4004030 con 0x55ea15834b70 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.143+0000 7f7ec67fc640 1 -- 172.21.15.18:0/1534671048 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ed4015070 con 0x55ea15834b70 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.143+0000 7f7ec67fc640 1 -- 172.21.15.18:0/1534671048 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed40116a0 con 0x55ea15834b70 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.143+0000 7f7ee9597740 1 -- 172.21.15.18:0/1534671048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834b70 msgr2=0x55ea1583d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.143+0000 7f7ee9597740 1 --2- 172.21.15.18:0/1534671048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834b70 0x55ea1583d210 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400bb70 tx=0x7f7ed40107a0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.143+0000 7f7ee9597740 1 -- 172.21.15.18:0/1534671048 shutdown_connections 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.143+0000 7f7ee9597740 1 --2- 172.21.15.18:0/1534671048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 0x55ea1583fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.143+0000 7f7ee9597740 1 --2- 172.21.15.18:0/1534671048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834b70 0x55ea1583d210 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.143+0000 7f7ee9597740 1 --2- 172.21.15.18:0/1534671048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834250 0x55ea15834630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.143+0000 7f7ee9597740 1 -- 172.21.15.18:0/1534671048 >> 172.21.15.18:0/1534671048 conn(0x55ea1582e980 msgr2=0x55ea1582ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:58.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.143+0000 7f7ee9597740 1 -- 172.21.15.18:0/1534671048 shutdown_connections 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.143+0000 7f7ee9597740 1 -- 172.21.15.18:0/1534671048 wait complete. 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7ee9597740 1 -- start start 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7ee9597740 1 Processor -- start 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7ee9597740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 0x55ea1583a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7ee9597740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834b70 0x55ea1583ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7ee9597740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1583d750 0x55ea1583b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7ee9597740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1579c020 con 0x55ea15834250 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7ee9597740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1546f430 con 0x55ea1583d750 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7ee9597740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1542be00 con 0x55ea15834b70 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7ee9597740 10 client.0 _ll_register_callbacks cb 0x55ea15743cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7edffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 0x55ea1583a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7edffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 0x55ea1583a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58852/0 (socket says 172.21.15.18:58852) 2024-02-17T12:58:58.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7edffff640 1 -- 172.21.15.18:0/2477760999 learned_addr learned my addr 172.21.15.18:0/2477760999 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7ee4f64640 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1583d750 0x55ea1583b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7edf7fe640 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834b70 0x55ea1583ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7edffff640 1 -- 172.21.15.18:0/2477760999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834b70 msgr2=0x55ea1583ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7edffff640 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834b70 0x55ea1583ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7edffff640 1 -- 172.21.15.18:0/2477760999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1583d750 msgr2=0x55ea1583b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7edffff640 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1583d750 0x55ea1583b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7edffff640 1 -- 172.21.15.18:0/2477760999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed400b7a0 con 0x55ea15834250 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.144+0000 7f7ee4f64640 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1583d750 0x55ea1583b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7edffff640 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 0x55ea1583a7a0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4011010 tx=0x7f7ed400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f2c3e91a6e2ecd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ebf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ebf7fe640 1 -- 172.21.15.18:0/2477760999 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4021070 con 0x55ea15834250 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ebf7fe640 1 -- 172.21.15.18:0/2477760999 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7ed4017070 con 0x55ea15834250 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ee9597740 1 -- 172.21.15.18:0/2477760999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea1549b7e0 con 0x55ea15834250 2024-02-17T12:58:58.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ebf7fe640 1 -- 172.21.15.18:0/2477760999 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ed4003880 con 0x55ea15834250 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ee9597740 10 client.18837 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ee9597740 1 -- 172.21.15.18:0/2477760999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ea1579a6a0 con 0x55ea15834250 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ebf7fe640 1 -- 172.21.15.18:0/2477760999 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ea1549b7e0 con 0x55ea15834250 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ebf7fe640 1 -- 172.21.15.18:0/2477760999 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7ed40046c0 con 0x55ea15834250 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ee9597740 10 client.18837 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ee9597740 10 client.18837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ee9597740 20 client.18837 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ee9597740 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ea1584da70 0x55ea1584fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ee9597740 4 client.18837 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.145+0000 7f7ee9597740 1 -- 172.21.15.18:0/2477760999 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea154d8720 con 0x55ea1584da70 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.146+0000 7f7edf7fe640 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ea1584da70 0x55ea1584fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.146+0000 7f7edf7fe640 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ea1584da70 0x55ea1584fe60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6605cbf47543a3e server_cookie=d6da8537ed5b210 in_seq=0 out_seq=0 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.146+0000 7f7ebf7fe640 10 client.18837 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:58.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ebf7fe640 1 -- 172.21.15.18:0/2477760999 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea154d8720 con 0x55ea1584da70 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ebf7fe640 10 client.18837 handle_command_reply: tid=0 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ee9597740 1 client.18837 shutdown 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ee9597740 1 -- 172.21.15.18:0/2477760999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 msgr2=0x55ea1583a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ee9597740 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 0x55ea1583a7a0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4011010 tx=0x7f7ed400bf50 comp rx=0 tx=0).stop 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ee9597740 1 -- 172.21.15.18:0/2477760999 shutdown_connections 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ee9597740 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ea1584da70 0x55ea1584fe60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ee9597740 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea1583d750 0x55ea1583b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ee9597740 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834b70 0x55ea1583ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ebf7fe640 0 client.18837 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ee9597740 1 --2- 172.21.15.18:0/2477760999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 0x55ea1583a7a0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ee9597740 1 -- 172.21.15.18:0/2477760999 >> 172.21.15.18:0/2477760999 conn(0x55ea1582e980 msgr2=0x55ea158306d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ee9597740 1 -- 172.21.15.18:0/2477760999 shutdown_connections 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ee9597740 1 -- 172.21.15.18:0/2477760999 wait complete. 2024-02-17T12:58:58.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.147+0000 7f7ee9597740 20 client.18837 trim_cache size 0 max 16384 2024-02-17T12:58:58.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7ee9597740 1 Processor -- start 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7ee9597740 1 -- start start 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7ee9597740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 0x55ea1583ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7ee9597740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834b70 0x55ea1583b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7ee9597740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea1583d750 0x55ea15837a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7ee9597740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea15721130 con 0x55ea15834250 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7ee9597740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea15833eb0 con 0x55ea15834b70 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7ee9597740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea1582f980 con 0x55ea1583d750 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7edffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 0x55ea1583ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7edffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 0x55ea1583ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58866/0 (socket says 172.21.15.18:58866) 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7edffff640 1 -- 172.21.15.18:0/3035120291 learned_addr learned my addr 172.21.15.18:0/3035120291 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7edf7fe640 1 --2- 172.21.15.18:0/3035120291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834b70 0x55ea1583b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7edffff640 1 -- 172.21.15.18:0/3035120291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea1583d750 msgr2=0x55ea15837a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7edffff640 1 --2- 172.21.15.18:0/3035120291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea1583d750 0x55ea15837a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7edffff640 1 -- 172.21.15.18:0/3035120291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834b70 msgr2=0x55ea1583b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7edffff640 1 --2- 172.21.15.18:0/3035120291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834b70 0x55ea1583b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7edffff640 1 -- 172.21.15.18:0/3035120291 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea15736c20 con 0x55ea15834250 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7edf7fe640 1 --2- 172.21.15.18:0/3035120291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834b70 0x55ea1583b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.159+0000 7f7edffff640 1 --2- 172.21.15.18:0/3035120291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 0x55ea1583ced0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4010ca0 tx=0x7f7ed40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc323e32dc5d8619 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ec5ffb640 1 -- 172.21.15.18:0/3035120291 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4021040 con 0x55ea15834250 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ec5ffb640 1 -- 172.21.15.18:0/3035120291 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ed40054b0 con 0x55ea15834250 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ec5ffb640 1 -- 172.21.15.18:0/3035120291 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed402fa40 con 0x55ea15834250 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ee9597740 1 -- 172.21.15.18:0/3035120291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 msgr2=0x55ea1583ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ee9597740 1 --2- 172.21.15.18:0/3035120291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 0x55ea1583ced0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4010ca0 tx=0x7f7ed40107a0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ee9597740 1 -- 172.21.15.18:0/3035120291 shutdown_connections 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ee9597740 1 --2- 172.21.15.18:0/3035120291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea1583d750 0x55ea15837a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ee9597740 1 --2- 172.21.15.18:0/3035120291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834b70 0x55ea1583b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ee9597740 1 --2- 172.21.15.18:0/3035120291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea15834250 0x55ea1583ced0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ee9597740 1 -- 172.21.15.18:0/3035120291 >> 172.21.15.18:0/3035120291 conn(0x55ea1582e980 msgr2=0x55ea15835ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ee9597740 1 -- 172.21.15.18:0/3035120291 shutdown_connections 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ee9597740 1 -- 172.21.15.18:0/3035120291 wait complete. 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.160+0000 7f7ee9597740 1 -- start start 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee9597740 1 Processor -- start 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee9597740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834250 0x55ea15843650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee9597740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834b70 0x55ea15843b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee9597740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 0x55ea158a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee9597740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea15844290 con 0x55ea1583d750 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee9597740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea158a44d0 con 0x55ea15834250 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee9597740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea158a4610 con 0x55ea15834b70 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee9597740 10 client.0 _ll_register_callbacks cb 0x7f7ed400b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee4f64640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 0x55ea158a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee4f64640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 0x55ea158a3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58876/0 (socket says 172.21.15.18:58876) 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee4f64640 1 -- 172.21.15.18:0/1869443287 learned_addr learned my addr 172.21.15.18:0/1869443287 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7edf7fe640 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834b70 0x55ea15843b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7edffff640 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834250 0x55ea15843650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee4f64640 1 -- 172.21.15.18:0/1869443287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834b70 msgr2=0x55ea15843b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee4f64640 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834b70 0x55ea15843b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee4f64640 1 -- 172.21.15.18:0/1869443287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834250 msgr2=0x55ea15843650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee4f64640 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834250 0x55ea15843650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7ee4f64640 1 -- 172.21.15.18:0/1869443287 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea1579a6a0 con 0x55ea1583d750 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.161+0000 7f7edffff640 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834250 0x55ea15843650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ee4f64640 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 0x55ea158a3f90 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800fbd0 tx=0x7f7ed800aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d64c9cf92692c86 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ebf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ebf7fe640 1 -- 172.21.15.18:0/1869443287 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed800fcc0 con 0x55ea1583d750 2024-02-17T12:58:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ebf7fe640 1 -- 172.21.15.18:0/1869443287 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7ed800fe20 con 0x55ea1583d750 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ee9597740 1 -- 172.21.15.18:0/1869443287 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ed4015070 con 0x55ea1583d750 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ebf7fe640 1 -- 172.21.15.18:0/1869443287 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ed8004b20 con 0x55ea1583d750 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ee9597740 10 client.18843 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ee9597740 1 -- 172.21.15.18:0/1869443287 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ea1575c8c0 con 0x55ea1583d750 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ebf7fe640 1 -- 172.21.15.18:0/1869443287 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ed4015070 con 0x55ea1583d750 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ebf7fe640 1 -- 172.21.15.18:0/1869443287 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7ed8005140 con 0x55ea1583d750 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ee9597740 10 client.18843 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ee9597740 10 client.18843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ee9597740 20 client.18843 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ee9597740 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ea158a5e70 0x55ea158a8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ee9597740 4 client.18843 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.162+0000 7f7ee9597740 1 -- 172.21.15.18:0/1869443287 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ea15527890 con 0x55ea158a5e70 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7edffff640 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ea158a5e70 0x55ea158a8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7edffff640 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ea158a5e70 0x55ea158a8290 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48ce054e78460578 server_cookie=c43eff48f6db7fe4 in_seq=0 out_seq=0 2024-02-17T12:58:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ebf7fe640 10 client.18843 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ebf7fe640 1 -- 172.21.15.18:0/1869443287 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ea15527890 con 0x55ea158a5e70 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ebf7fe640 10 client.18843 handle_command_reply: tid=0 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ee9597740 1 client.18843 shutdown 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ee9597740 1 -- 172.21.15.18:0/1869443287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 msgr2=0x55ea158a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ee9597740 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 0x55ea158a3f90 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800fbd0 tx=0x7f7ed800aa60 comp rx=0 tx=0).stop 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ee9597740 1 -- 172.21.15.18:0/1869443287 shutdown_connections 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ee9597740 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ea158a5e70 0x55ea158a8290 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ee9597740 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea1583d750 0x55ea158a3f90 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ee9597740 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea15834b70 0x55ea15843b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ee9597740 1 --2- 172.21.15.18:0/1869443287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea15834250 0x55ea15843650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ebf7fe640 0 client.18843 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ee9597740 1 -- 172.21.15.18:0/1869443287 >> 172.21.15.18:0/1869443287 conn(0x55ea1582e980 msgr2=0x55ea1582d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ee9597740 1 -- 172.21.15.18:0/1869443287 shutdown_connections 2024-02-17T12:58:58.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.163+0000 7f7ee9597740 1 -- 172.21.15.18:0/1869443287 wait complete. 2024-02-17T12:58:58.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee9597740 20 client.18843 trim_cache size 0 max 16384 2024-02-17T12:58:58.161 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:58:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee69c7640 1 -- 172.21.15.18:0/1472879417 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7ec80a2230 msgr2=0x7f7ec80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee69c7640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7ec80a2230 0x7f7ec80a46f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800a780 tx=0x7f7ed800a4d0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee69c7640 1 -- 172.21.15.18:0/1472879417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ee0100400 msgr2=0x7f7ee0101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee69c7640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ee0100400 0x7f7ee0101160 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0005940 tx=0x7f7ed0004040 comp rx=0 tx=0).stop 2024-02-17T12:58:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee69c7640 1 -- 172.21.15.18:0/1472879417 shutdown_connections 2024-02-17T12:58:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee69c7640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7ec80a2230 0x7f7ec80a46f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee69c7640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee0107380 0x7f7ee01016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee69c7640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ee0100400 0x7f7ee0101160 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee69c7640 1 --2- 172.21.15.18:0/1472879417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee00ffa40 0x7f7ee0100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee69c7640 1 -- 172.21.15.18:0/1472879417 >> 172.21.15.18:0/1472879417 conn(0x7f7ee00753c0 msgr2=0x7f7ee00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee69c7640 1 -- 172.21.15.18:0/1472879417 shutdown_connections 2024-02-17T12:58:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:58.164+0000 7f7ee69c7640 1 -- 172.21.15.18:0/1472879417 wait complete. 2024-02-17T12:58:58.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:57 smithi018 ceph-mon[32625]: pgmap v282: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-02-17T12:58:58.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:57 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1560534679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:58:58.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 467138 5.27 MB/sec execute 373 sec latency 3542.097 ms 2024-02-17T12:58:58.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:57 smithi119 ceph-mon[36188]: pgmap v282: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-02-17T12:58:58.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:57 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1560534679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:58:58.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:57 smithi146 ceph-mon[36454]: pgmap v282: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-02-17T12:58:58.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:58:57 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1560534679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:58:59.165 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:58:59.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:58:59.166 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:58:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 -- 172.21.15.18:0/3926044969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2b010d730 msgr2=0x7fb2b010db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 --2- 172.21.15.18:0/3926044969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2b010d730 0x7fb2b010db10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009930 tx=0x7fb2ac02bc30 comp rx=0 tx=0).stop 2024-02-17T12:58:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 -- 172.21.15.18:0/3926044969 shutdown_connections 2024-02-17T12:58:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 --2- 172.21.15.18:0/3926044969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2b01080e0 0x7fb2b01145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 --2- 172.21.15.18:0/3926044969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2b0107720 0x7fb2b0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 --2- 172.21.15.18:0/3926044969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2b010d730 0x7fb2b010db10 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 -- 172.21.15.18:0/3926044969 >> 172.21.15.18:0/3926044969 conn(0x7fb2b00756c0 msgr2=0x7fb2b0077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 -- 172.21.15.18:0/3926044969 shutdown_connections 2024-02-17T12:58:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 -- 172.21.15.18:0/3926044969 wait complete. 2024-02-17T12:58:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 Processor -- start 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 -- start start 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2b0107720 0x7fb2b019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2b01080e0 0x7fb2b019e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2b010d730 0x7fb2b01a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b01a2c60 con 0x7fb2b010d730 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.285+0000 7fb2b833e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b01a2dd0 con 0x7fb2b0107720 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b833e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b01a2f90 con 0x7fb2b01080e0 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b68db640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2b010d730 0x7fb2b01a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b58d9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2b01080e0 0x7fb2b019e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b60da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2b0107720 0x7fb2b019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b68db640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2b010d730 0x7fb2b01a2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58892/0 (socket says 172.21.15.18:58892) 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b68db640 1 -- 172.21.15.18:0/3621023314 learned_addr learned my addr 172.21.15.18:0/3621023314 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b68db640 1 -- 172.21.15.18:0/3621023314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2b01080e0 msgr2=0x7fb2b019e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b68db640 1 --2- 172.21.15.18:0/3621023314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2b01080e0 0x7fb2b019e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b68db640 1 -- 172.21.15.18:0/3621023314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2b0107720 msgr2=0x7fb2b019dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b68db640 1 --2- 172.21.15.18:0/3621023314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2b0107720 0x7fb2b019dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b68db640 1 -- 172.21.15.18:0/3621023314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac009560 con 0x7fb2b010d730 2024-02-17T12:58:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b68db640 1 --2- 172.21.15.18:0/3621023314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2b010d730 0x7fb2b01a2660 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800cce0 tx=0x7fb2a800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2a37fe640 1 -- 172.21.15.18:0/3621023314 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a800f450 con 0x7fb2b010d730 2024-02-17T12:58:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b833e640 1 -- 172.21.15.18:0/3621023314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b01a3270 con 0x7fb2b010d730 2024-02-17T12:58:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2a37fe640 1 -- 172.21.15.18:0/3621023314 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2a800fa90 con 0x7fb2b010d730 2024-02-17T12:58:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2a37fe640 1 -- 172.21.15.18:0/3621023314 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8018b90 con 0x7fb2b010d730 2024-02-17T12:58:59.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.286+0000 7fb2b833e640 1 -- 172.21.15.18:0/3621023314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b01aaaf0 con 0x7fb2b010d730 2024-02-17T12:58:59.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.288+0000 7fb2a37fe640 1 -- 172.21.15.18:0/3621023314 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb2a800fc00 con 0x7fb2b010d730 2024-02-17T12:58:59.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2bafef740 1 Processor -- start 2024-02-17T12:58:59.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2bafef740 1 -- start start 2024-02-17T12:58:59.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2bafef740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85cc380 0x5571a85cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2bafef740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a85d52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2bafef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 0x5571a85d7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2bafef740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a853af90 con 0x5571a85ccd30 2024-02-17T12:58:59.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2bafef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a85384c0 con 0x5571a85d58b0 2024-02-17T12:58:59.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2bafef740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a8533ca0 con 0x5571a85cc380 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2a37fe640 1 --2- 172.21.15.18:0/3621023314 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb29c0a2230 0x7fb29c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2b60da640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a85d52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2b60da640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a85d52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58904/0 (socket says 172.21.15.18:58904) 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2b60da640 1 -- 172.21.15.18:0/3707843137 learned_addr learned my addr 172.21.15.18:0/3707843137 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/3707843137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85cc380 0x5571a85cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2a37fe640 1 -- 172.21.15.18:0/3621023314 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb2a80ce7a0 con 0x7fb2b010d730 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2b68db640 1 --2- 172.21.15.18:0/3621023314 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb29c0a2230 0x7fb29c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.289+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/3707843137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 0x5571a85d7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.290+0000 7fb2b60da640 1 -- 172.21.15.18:0/3707843137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85cc380 msgr2=0x5571a85cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.290+0000 7fb2b60da640 1 --2- 172.21.15.18:0/3707843137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85cc380 0x5571a85cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.290+0000 7fb2b60da640 1 -- 172.21.15.18:0/3707843137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 msgr2=0x5571a85d7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.290+0000 7fb2b60da640 1 --2- 172.21.15.18:0/3707843137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 0x5571a85d7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.290+0000 7fb2b60da640 1 -- 172.21.15.18:0/3707843137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a84f60b0 con 0x5571a85ccd30 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.290+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/3707843137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85cc380 0x5571a85cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.290+0000 7fb2b68db640 1 --2- 172.21.15.18:0/3621023314 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb29c0a2230 0x7fb29c0a46f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800ab20 tx=0x7fb2a800af30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.290+0000 7fb2b60da640 1 --2- 172.21.15.18:0/3707843137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a85d52e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac036040 tx=0x7fb2ac02c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9985b81c4de036c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.290+0000 7fb2a0bfe640 1 -- 172.21.15.18:0/3707843137 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac004900 con 0x5571a85ccd30 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.290+0000 7fb2a0bfe640 1 -- 172.21.15.18:0/3707843137 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2ac004a60 con 0x5571a85ccd30 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.290+0000 7fb2a0bfe640 1 -- 172.21.15.18:0/3707843137 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0050e0 con 0x5571a85ccd30 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.291+0000 7fb2bafef740 1 -- 172.21.15.18:0/3707843137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 msgr2=0x5571a85d52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.291+0000 7fb2bafef740 1 --2- 172.21.15.18:0/3707843137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a85d52e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac036040 tx=0x7fb2ac02c730 comp rx=0 tx=0).stop 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.291+0000 7fb2bafef740 1 -- 172.21.15.18:0/3707843137 shutdown_connections 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.291+0000 7fb2bafef740 1 --2- 172.21.15.18:0/3707843137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 0x5571a85d7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.291+0000 7fb2bafef740 1 --2- 172.21.15.18:0/3707843137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a85d52e0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.291+0000 7fb2bafef740 1 --2- 172.21.15.18:0/3707843137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85cc380 0x5571a85cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.291+0000 7fb2bafef740 1 -- 172.21.15.18:0/3707843137 >> 172.21.15.18:0/3707843137 conn(0x5571a85c6840 msgr2=0x5571a85c6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.291+0000 7fb2bafef740 1 -- 172.21.15.18:0/3707843137 shutdown_connections 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.291+0000 7fb2bafef740 1 -- 172.21.15.18:0/3707843137 wait complete. 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.291+0000 7fb2bafef740 1 -- start start 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.291+0000 7fb2bafef740 1 Processor -- start 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2bafef740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85cc380 0x5571a85d2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2bafef740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a85d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2bafef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 0x5571a85d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2bafef740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a84d8f40 con 0x5571a85ccd30 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2bafef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a82074b0 con 0x5571a85d58b0 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2bafef740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a84b9670 con 0x5571a85cc380 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2b60da640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a85d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2bafef740 10 client.0 _ll_register_callbacks cb 0x5571a8526f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2b60da640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a85d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58914/0 (socket says 172.21.15.18:58914) 2024-02-17T12:58:59.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2b60da640 1 -- 172.21.15.18:0/4111812198 learned_addr learned my addr 172.21.15.18:0/4111812198 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85cc380 0x5571a85d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 0x5571a85d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2b60da640 1 -- 172.21.15.18:0/4111812198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85cc380 msgr2=0x5571a85d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2b60da640 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85cc380 0x5571a85d2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2b60da640 1 -- 172.21.15.18:0/4111812198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 msgr2=0x5571a85d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2b60da640 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 0x5571a85d3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2b60da640 1 -- 172.21.15.18:0/4111812198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b0114ef0 con 0x5571a85ccd30 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2b60da640 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a85d2cd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac005ab0 tx=0x7fb2ac02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a091ef2319cc98 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb299ffb640 1 -- 172.21.15.18:0/4111812198 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac004520 con 0x5571a85ccd30 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb299ffb640 1 -- 172.21.15.18:0/4111812198 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb2ac04d070 con 0x5571a85ccd30 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.292+0000 7fb2bafef740 1 -- 172.21.15.18:0/4111812198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571a850df40 con 0x5571a85ccd30 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb299ffb640 1 -- 172.21.15.18:0/4111812198 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2ac005060 con 0x5571a85ccd30 2024-02-17T12:58:59.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb2bafef740 10 client.18855 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb2bafef740 1 -- 172.21.15.18:0/4111812198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5571a8532320 con 0x5571a85ccd30 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb299ffb640 1 -- 172.21.15.18:0/4111812198 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5571a850df40 con 0x5571a85ccd30 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb299ffb640 1 -- 172.21.15.18:0/4111812198 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb2ac03c050 con 0x5571a85ccd30 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb2bafef740 10 client.18855 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb2bafef740 10 client.18855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb2bafef740 20 client.18855 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb2bafef740 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5571a85e59e0 0x5571a85e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb2bafef740 4 client.18855 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb2bafef740 1 -- 172.21.15.18:0/4111812198 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571a852a310 con 0x5571a85e59e0 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5571a85e59e0 0x5571a85e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5571a85e59e0 0x5571a85e7e00 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed8f0eeee0159f9 server_cookie=c1f9aba5d504a5cc in_seq=0 out_seq=0 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.293+0000 7fb299ffb640 10 client.18855 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb299ffb640 1 -- 172.21.15.18:0/4111812198 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5571a852a310 con 0x5571a85e59e0 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb299ffb640 10 client.18855 handle_command_reply: tid=0 2024-02-17T12:58:59.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb2bafef740 1 client.18855 shutdown 2024-02-17T12:58:59.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb2bafef740 1 -- 172.21.15.18:0/4111812198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 msgr2=0x5571a85d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb2bafef740 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a85d2cd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac005ab0 tx=0x7fb2ac02ca10 comp rx=0 tx=0).stop 2024-02-17T12:58:59.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb2bafef740 1 -- 172.21.15.18:0/4111812198 shutdown_connections 2024-02-17T12:58:59.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb2bafef740 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5571a85e59e0 0x5571a85e7e00 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb2bafef740 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 0x5571a85d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb2bafef740 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a85d2cd0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb299ffb640 0 client.18855 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:59.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb2bafef740 1 --2- 172.21.15.18:0/4111812198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85cc380 0x5571a85d2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb2bafef740 1 -- 172.21.15.18:0/4111812198 >> 172.21.15.18:0/4111812198 conn(0x5571a85c6840 msgr2=0x5571a85c87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb2bafef740 1 -- 172.21.15.18:0/4111812198 shutdown_connections 2024-02-17T12:58:59.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb2bafef740 1 -- 172.21.15.18:0/4111812198 wait complete. 2024-02-17T12:58:59.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.294+0000 7fb2bafef740 20 client.18855 trim_cache size 0 max 16384 2024-02-17T12:58:59.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 469108 5.29 MB/sec execute 374 sec latency 3737.939 ms 2024-02-17T12:58:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2bafef740 1 Processor -- start 2024-02-17T12:58:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2bafef740 1 -- start start 2024-02-17T12:58:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2bafef740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85cc380 0x5571a85d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2bafef740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85ccd30 0x5571a85c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2bafef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 0x5571a85dc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2bafef740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a85e28a0 con 0x5571a85cc380 2024-02-17T12:58:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2bafef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a85c6670 con 0x5571a85d58b0 2024-02-17T12:58:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2bafef740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a85cb510 con 0x5571a85ccd30 2024-02-17T12:58:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2b58d9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85cc380 0x5571a85d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2b58d9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85cc380 0x5571a85d3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58926/0 (socket says 172.21.15.18:58926) 2024-02-17T12:58:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2b58d9640 1 -- 172.21.15.18:0/3549439293 learned_addr learned my addr 172.21.15.18:0/3549439293 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2b60da640 1 --2- 172.21.15.18:0/3549439293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85ccd30 0x5571a85c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/3549439293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 0x5571a85dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2b58d9640 1 -- 172.21.15.18:0/3549439293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85ccd30 msgr2=0x5571a85c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/3549439293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85ccd30 0x5571a85c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2b58d9640 1 -- 172.21.15.18:0/3549439293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 msgr2=0x5571a85dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/3549439293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 0x5571a85dc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.310+0000 7fb2b58d9640 1 -- 172.21.15.18:0/3549439293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a84cec00 con 0x5571a85cc380 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/3549439293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85cc380 0x5571a85d3030 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009730 tx=0x7fb2a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650f759f1faa0cde server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb29bfff640 1 -- 172.21.15.18:0/3549439293 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4016040 con 0x5571a85cc380 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb29bfff640 1 -- 172.21.15.18:0/3549439293 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2a4026e90 con 0x5571a85cc380 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb29bfff640 1 -- 172.21.15.18:0/3549439293 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4022470 con 0x5571a85cc380 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb2bafef740 1 -- 172.21.15.18:0/3549439293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85cc380 msgr2=0x5571a85d3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb2bafef740 1 --2- 172.21.15.18:0/3549439293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85cc380 0x5571a85d3030 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009730 tx=0x7fb2a4005f90 comp rx=0 tx=0).stop 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb2bafef740 1 -- 172.21.15.18:0/3549439293 shutdown_connections 2024-02-17T12:58:59.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb2bafef740 1 --2- 172.21.15.18:0/3549439293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85d58b0 0x5571a85dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb2bafef740 1 --2- 172.21.15.18:0/3549439293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85ccd30 0x5571a85c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb2bafef740 1 --2- 172.21.15.18:0/3549439293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85cc380 0x5571a85d3030 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb2bafef740 1 -- 172.21.15.18:0/3549439293 >> 172.21.15.18:0/3549439293 conn(0x5571a85c6840 msgr2=0x5571a85c5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb2bafef740 1 -- 172.21.15.18:0/3549439293 shutdown_connections 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.311+0000 7fb2bafef740 1 -- 172.21.15.18:0/3549439293 wait complete. 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2bafef740 1 -- start start 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2bafef740 1 Processor -- start 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2bafef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85cc380 0x5571a8635180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2bafef740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a86356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2bafef740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85d58b0 0x5571a85dbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2bafef740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a85dc130 con 0x5571a85ccd30 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2bafef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a85dc2a0 con 0x5571a85cc380 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2bafef740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a85dc460 con 0x5571a85d58b0 2024-02-17T12:58:59.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2bafef740 10 client.0 _ll_register_callbacks cb 0x7fb2ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2b60da640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a86356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2b60da640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a86356c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58930/0 (socket says 172.21.15.18:58930) 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2b60da640 1 -- 172.21.15.18:0/2860791774 learned_addr learned my addr 172.21.15.18:0/2860791774 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85cc380 0x5571a8635180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85d58b0 0x5571a85dbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2b60da640 1 -- 172.21.15.18:0/2860791774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85d58b0 msgr2=0x5571a85dbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2b60da640 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85d58b0 0x5571a85dbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2b60da640 1 -- 172.21.15.18:0/2860791774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85cc380 msgr2=0x5571a8635180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2b60da640 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85cc380 0x5571a8635180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2b60da640 1 -- 172.21.15.18:0/2860791774 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a4009dc0 con 0x5571a85ccd30 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.312+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85d58b0 0x5571a85dbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb2b60da640 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a86356c0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac02bc70 tx=0x7fb2ac0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e297e9bb3ebcec server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:59.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb299ffb640 1 -- 172.21.15.18:0/2860791774 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac05f070 con 0x5571a85ccd30 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb299ffb640 1 -- 172.21.15.18:0/2860791774 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb2ac04d040 con 0x5571a85ccd30 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb2bafef740 1 -- 172.21.15.18:0/2860791774 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571a8559900 con 0x5571a85ccd30 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb299ffb640 1 -- 172.21.15.18:0/2860791774 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2ac0044f0 con 0x5571a85ccd30 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb2bafef740 10 client.18861 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb2bafef740 1 -- 172.21.15.18:0/2860791774 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5571a84f4910 con 0x5571a85ccd30 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb299ffb640 1 -- 172.21.15.18:0/2860791774 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5571a8559900 con 0x5571a85ccd30 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb299ffb640 1 -- 172.21.15.18:0/2860791774 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb2ac062020 con 0x5571a85ccd30 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb2bafef740 10 client.18861 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb2bafef740 10 client.18861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb2bafef740 20 client.18861 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb2bafef740 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5571a863dd60 0x5571a8640150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb2bafef740 4 client.18861 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb2bafef740 1 -- 172.21.15.18:0/2860791774 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5571a8233860 con 0x5571a863dd60 2024-02-17T12:58:59.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.313+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5571a863dd60 0x5571a8640150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.314+0000 7fb2b58d9640 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5571a863dd60 0x5571a8640150 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8c1c4bb0fe35b4 server_cookie=e56c2b242b5ba1df in_seq=0 out_seq=0 2024-02-17T12:58:59.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.314+0000 7fb299ffb640 10 client.18861 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:59.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.315+0000 7fb299ffb640 1 -- 172.21.15.18:0/2860791774 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5571a8233860 con 0x5571a863dd60 2024-02-17T12:58:59.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.315+0000 7fb299ffb640 10 client.18861 handle_command_reply: tid=0 2024-02-17T12:58:59.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2bafef740 1 client.18861 shutdown 2024-02-17T12:58:59.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2bafef740 1 -- 172.21.15.18:0/2860791774 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 msgr2=0x5571a86356c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2bafef740 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a86356c0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac02bc70 tx=0x7fb2ac0098a0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2bafef740 1 -- 172.21.15.18:0/2860791774 shutdown_connections 2024-02-17T12:58:59.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2bafef740 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5571a863dd60 0x5571a8640150 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2bafef740 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5571a85d58b0 0x5571a85dbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2bafef740 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5571a85ccd30 0x5571a86356c0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2bafef740 1 --2- 172.21.15.18:0/2860791774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5571a85cc380 0x5571a8635180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb299ffb640 0 client.18861 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:59.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2bafef740 1 -- 172.21.15.18:0/2860791774 >> 172.21.15.18:0/2860791774 conn(0x5571a85c6840 msgr2=0x5571a85c58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2bafef740 1 -- 172.21.15.18:0/2860791774 shutdown_connections 2024-02-17T12:58:59.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2bafef740 1 -- 172.21.15.18:0/2860791774 wait complete. 2024-02-17T12:58:59.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2bafef740 20 client.18861 trim_cache size 0 max 16384 2024-02-17T12:58:59.312 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:58:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:58:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "c1ae062b-56e9-4e03-8bad-8170f7b1767c", 2024-02-17T12:58:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:58:59.312 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:58:59.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2b833e640 1 -- 172.21.15.18:0/3621023314 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb29c0a2230 msgr2=0x7fb29c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2b833e640 1 --2- 172.21.15.18:0/3621023314 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb29c0a2230 0x7fb29c0a46f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800ab20 tx=0x7fb2a800af30 comp rx=0 tx=0).stop 2024-02-17T12:58:59.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2b833e640 1 -- 172.21.15.18:0/3621023314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2b010d730 msgr2=0x7fb2b01a2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.316+0000 7fb2b833e640 1 --2- 172.21.15.18:0/3621023314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2b010d730 0x7fb2b01a2660 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800cce0 tx=0x7fb2a800a620 comp rx=0 tx=0).stop 2024-02-17T12:58:59.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.317+0000 7fb2b833e640 1 -- 172.21.15.18:0/3621023314 shutdown_connections 2024-02-17T12:58:59.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.317+0000 7fb2b833e640 1 --2- 172.21.15.18:0/3621023314 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb29c0a2230 0x7fb29c0a46f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.317+0000 7fb2b833e640 1 --2- 172.21.15.18:0/3621023314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2b010d730 0x7fb2b01a2660 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.317+0000 7fb2b833e640 1 --2- 172.21.15.18:0/3621023314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2b01080e0 0x7fb2b019e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.317+0000 7fb2b833e640 1 --2- 172.21.15.18:0/3621023314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2b0107720 0x7fb2b019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.317+0000 7fb2b833e640 1 -- 172.21.15.18:0/3621023314 >> 172.21.15.18:0/3621023314 conn(0x7fb2b00756c0 msgr2=0x7fb2b0075ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.317+0000 7fb2b833e640 1 -- 172.21.15.18:0/3621023314 shutdown_connections 2024-02-17T12:58:59.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.317+0000 7fb2b833e640 1 -- 172.21.15.18:0/3621023314 wait complete. 2024-02-17T12:58:59.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:58:59.319 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:58:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.437+0000 7f22ed0df640 1 -- 172.21.15.18:0/1225468096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22e81007f0 msgr2=0x7f22e8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.437+0000 7f22ed0df640 1 --2- 172.21.15.18:0/1225468096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22e81007f0 0x7f22e8100c50 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009900 tx=0x7f22d002bc80 comp rx=0 tx=0).stop 2024-02-17T12:58:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.437+0000 7f22ed0df640 1 -- 172.21.15.18:0/1225468096 shutdown_connections 2024-02-17T12:58:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.437+0000 7f22ed0df640 1 --2- 172.21.15.18:0/1225468096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22e8101190 0x7f22e810cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.437+0000 7f22ed0df640 1 --2- 172.21.15.18:0/1225468096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22e81007f0 0x7f22e8100c50 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.437+0000 7f22ed0df640 1 --2- 172.21.15.18:0/1225468096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22e8105050 0x7f22e8105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.437+0000 7f22ed0df640 1 -- 172.21.15.18:0/1225468096 >> 172.21.15.18:0/1225468096 conn(0x7f22e80765a0 msgr2=0x7f22e80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.437+0000 7f22ed0df640 1 -- 172.21.15.18:0/1225468096 shutdown_connections 2024-02-17T12:58:59.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.437+0000 7f22ed0df640 1 -- 172.21.15.18:0/1225468096 wait complete. 2024-02-17T12:58:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.437+0000 7f22ed0df640 1 Processor -- start 2024-02-17T12:58:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22ed0df640 1 -- start start 2024-02-17T12:58:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22ed0df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22e81007f0 0x7f22e819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22ed0df640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22e8101190 0x7f22e819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22ed0df640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22e8105050 0x7f22e8197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22ed0df640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e8197dd0 con 0x7f22e81007f0 2024-02-17T12:58:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22ed0df640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e8197f40 con 0x7f22e8101190 2024-02-17T12:58:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22ed0df640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e8198100 con 0x7f22e8105050 2024-02-17T12:58:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22e6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22e81007f0 0x7f22e819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22e6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22e81007f0 0x7f22e819d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58950/0 (socket says 172.21.15.18:58950) 2024-02-17T12:58:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22e6d9d640 1 -- 172.21.15.18:0/2098384677 learned_addr learned my addr 172.21.15.18:0/2098384677 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22e659c640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22e8101190 0x7f22e819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22e759e640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22e8105050 0x7f22e8197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22e6d9d640 1 -- 172.21.15.18:0/2098384677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22e8105050 msgr2=0x7f22e8197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22e6d9d640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22e8105050 0x7f22e8197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22e6d9d640 1 -- 172.21.15.18:0/2098384677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22e8101190 msgr2=0x7f22e819dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22e6d9d640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22e8101190 0x7f22e819dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22e6d9d640 1 -- 172.21.15.18:0/2098384677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d0009560 con 0x7f22e81007f0 2024-02-17T12:58:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22e6d9d640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22e81007f0 0x7f22e819d580 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f22e000e9f0 tx=0x7f22e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22dffff640 1 -- 172.21.15.18:0/2098384677 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0009720 con 0x7f22e81007f0 2024-02-17T12:58:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.438+0000 7f22dffff640 1 -- 172.21.15.18:0/2098384677 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22e000cd80 con 0x7f22e81007f0 2024-02-17T12:58:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.439+0000 7f22ed0df640 1 -- 172.21.15.18:0/2098384677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e81983e0 con 0x7f22e81007f0 2024-02-17T12:58:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.439+0000 7f22dffff640 1 -- 172.21.15.18:0/2098384677 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0010690 con 0x7f22e81007f0 2024-02-17T12:58:59.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.439+0000 7f22ed0df640 1 -- 172.21.15.18:0/2098384677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e81a4500 con 0x7f22e81007f0 2024-02-17T12:58:59.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.441+0000 7f22dffff640 1 -- 172.21.15.18:0/2098384677 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f22e00046c0 con 0x7f22e81007f0 2024-02-17T12:58:59.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.441+0000 7f22efd93740 1 Processor -- start 2024-02-17T12:58:59.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.441+0000 7f22efd93740 1 -- start start 2024-02-17T12:58:59.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.441+0000 7f22dffff640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f22d40a2230 0x7f22d40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.441+0000 7f22dffff640 1 -- 172.21.15.18:0/2098384677 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f22e0014070 con 0x7f22e81007f0 2024-02-17T12:58:59.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.441+0000 7f22e759e640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f22d40a2230 0x7f22d40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.441+0000 7f22efd93740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e92690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.441+0000 7f22efd93740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 0x559807e9b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.441+0000 7f22efd93740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e9b7b0 0x559807e9dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.441+0000 7f22efd93740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559807e01280 con 0x559807e92bd0 2024-02-17T12:58:59.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22efd93740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559807dfe7b0 con 0x559807e9b7b0 2024-02-17T12:58:59.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22efd93740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559807df9f90 con 0x559807e922b0 2024-02-17T12:58:59.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 0x559807e9b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e6d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e92690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 0x559807e9b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51558/0 (socket says 172.21.15.18:51558) 2024-02-17T12:58:59.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e659c640 1 -- 172.21.15.18:0/3766694700 learned_addr learned my addr 172.21.15.18:0/3766694700 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e759e640 1 --2- 172.21.15.18:0/3766694700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e9b7b0 0x559807e9dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e659c640 1 -- 172.21.15.18:0/3766694700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 msgr2=0x559807e92690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e659c640 1 --2- 172.21.15.18:0/3766694700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e92690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e659c640 1 -- 172.21.15.18:0/3766694700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e9b7b0 msgr2=0x559807e9dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e659c640 1 --2- 172.21.15.18:0/3766694700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e9b7b0 0x559807e9dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e759e640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f22d40a2230 0x7f22d40a46f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f22d8005fd0 tx=0x7f22d8011810 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e659c640 1 -- 172.21.15.18:0/3766694700 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559807db8570 con 0x559807e92bd0 2024-02-17T12:58:59.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e6d9d640 1 --2- 172.21.15.18:0/3766694700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e92690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:59.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22e659c640 1 --2- 172.21.15.18:0/3766694700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 0x559807e9b270 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009900 tx=0x7f22d002c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654de5f0252ff788 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22dd7fa640 1 -- 172.21.15.18:0/3766694700 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d002c900 con 0x559807e92bd0 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22dd7fa640 1 -- 172.21.15.18:0/3766694700 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22d002ca60 con 0x559807e92bd0 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.442+0000 7f22dd7fa640 1 -- 172.21.15.18:0/3766694700 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0005170 con 0x559807e92bd0 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.443+0000 7f22efd93740 1 -- 172.21.15.18:0/3766694700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 msgr2=0x559807e9b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.443+0000 7f22efd93740 1 --2- 172.21.15.18:0/3766694700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 0x559807e9b270 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009900 tx=0x7f22d002c690 comp rx=0 tx=0).stop 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.443+0000 7f22efd93740 1 -- 172.21.15.18:0/3766694700 shutdown_connections 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.443+0000 7f22efd93740 1 --2- 172.21.15.18:0/3766694700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e9b7b0 0x559807e9dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.443+0000 7f22efd93740 1 --2- 172.21.15.18:0/3766694700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 0x559807e9b270 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.443+0000 7f22efd93740 1 --2- 172.21.15.18:0/3766694700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e92690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.443+0000 7f22efd93740 1 -- 172.21.15.18:0/3766694700 >> 172.21.15.18:0/3766694700 conn(0x559807e8c990 msgr2=0x559807e8cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.443+0000 7f22efd93740 1 -- 172.21.15.18:0/3766694700 shutdown_connections 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.443+0000 7f22efd93740 1 -- 172.21.15.18:0/3766694700 wait complete. 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.443+0000 7f22efd93740 1 -- start start 2024-02-17T12:58:59.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22efd93740 1 Processor -- start 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22efd93740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e9b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22efd93740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 0x559807e985d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22efd93740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e9b7b0 0x559807e98cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22efd93740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559807acd430 con 0x559807e92bd0 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22efd93740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559807a89e00 con 0x559807e9b7b0 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22efd93740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559807d7efd0 con 0x559807e922b0 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22efd93740 10 client.0 _ll_register_callbacks cb 0x559807dd4fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 0x559807e985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e6d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e9b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e9b7b0 0x559807e98cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e6d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e9b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:58224/0 (socket says 172.21.15.18:58224) 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e6d9d640 1 -- 172.21.15.18:0/1916872641 learned_addr learned my addr 172.21.15.18:0/1916872641 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 0x559807e985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51562/0 (socket says 172.21.15.18:51562) 2024-02-17T12:58:59.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e759e640 1 -- 172.21.15.18:0/1916872641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 msgr2=0x559807e9b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e759e640 1 --2- 172.21.15.18:0/1916872641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e9b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e759e640 1 -- 172.21.15.18:0/1916872641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 msgr2=0x559807e985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e759e640 1 --2- 172.21.15.18:0/1916872641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 0x559807e985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e759e640 1 -- 172.21.15.18:0/1916872641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e810d870 con 0x559807e9b7b0 2024-02-17T12:58:59.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e659c640 1 --2- 172.21.15.18:0/1916872641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 0x559807e985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:59.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22e759e640 1 --2- 172.21.15.18:0/1916872641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e9b7b0 0x559807e98cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009d60 tx=0x7f22d8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54850a7902d24f45 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.444+0000 7f22ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:58:59.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22ce7fc640 1 -- 172.21.15.18:0/1916872641 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8022070 con 0x559807e9b7b0 2024-02-17T12:58:59.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22ce7fc640 1 -- 172.21.15.18:0/1916872641 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f22d8004b20 con 0x559807e9b7b0 2024-02-17T12:58:59.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22efd93740 1 -- 172.21.15.18:0/1916872641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559807af97e0 con 0x559807e9b7b0 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22ce7fc640 1 -- 172.21.15.18:0/1916872641 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d8004ce0 con 0x559807e9b7b0 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22efd93740 10 client.27944 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22efd93740 1 -- 172.21.15.18:0/1916872641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x559807df8610 con 0x559807e9b7b0 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22ce7fc640 1 -- 172.21.15.18:0/1916872641 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559807af97e0 con 0x559807e9b7b0 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22ce7fc640 1 -- 172.21.15.18:0/1916872641 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f22d80055b0 con 0x559807e9b7b0 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22efd93740 10 client.27944 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22efd93740 10 client.27944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22efd93740 20 client.27944 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22efd93740 1 --2- 172.21.15.18:0/1916872641 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559807eab950 0x559807eadd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22efd93740 4 client.27944 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22efd93740 1 -- 172.21.15.18:0/1916872641 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559807b36720 con 0x559807eab950 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.445+0000 7f22e6d9d640 1 --2- 172.21.15.18:0/1916872641 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559807eab950 0x559807eadd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.446+0000 7f22ce7fc640 10 client.27944 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:59.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.446+0000 7f22e6d9d640 1 --2- 172.21.15.18:0/1916872641 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559807eab950 0x559807eadd70 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15306c00e74412ad server_cookie=3db54f001dd6da3e in_seq=0 out_seq=0 2024-02-17T12:58:59.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.446+0000 7f22ce7fc640 1 -- 172.21.15.18:0/1916872641 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559807b36720 con 0x559807eab950 2024-02-17T12:58:59.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.446+0000 7f22ce7fc640 10 client.27944 handle_command_reply: tid=0 2024-02-17T12:58:59.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.446+0000 7f22efd93740 1 client.27944 shutdown 2024-02-17T12:58:59.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.446+0000 7f22efd93740 1 -- 172.21.15.18:0/1916872641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e9b7b0 msgr2=0x559807e98cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.446+0000 7f22efd93740 1 --2- 172.21.15.18:0/1916872641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e9b7b0 0x559807e98cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009d60 tx=0x7f22d8004ab0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.447+0000 7f22efd93740 1 -- 172.21.15.18:0/1916872641 shutdown_connections 2024-02-17T12:58:59.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.447+0000 7f22efd93740 1 --2- 172.21.15.18:0/1916872641 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559807eab950 0x559807eadd70 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.447+0000 7f22efd93740 1 --2- 172.21.15.18:0/1916872641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e9b7b0 0x559807e98cc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.447+0000 7f22efd93740 1 --2- 172.21.15.18:0/1916872641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e92bd0 0x559807e985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.447+0000 7f22efd93740 1 --2- 172.21.15.18:0/1916872641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e9b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.447+0000 7f22ce7fc640 0 client.27944 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:59.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.447+0000 7f22efd93740 1 -- 172.21.15.18:0/1916872641 >> 172.21.15.18:0/1916872641 conn(0x559807e8c990 msgr2=0x559807e8e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.447+0000 7f22efd93740 1 -- 172.21.15.18:0/1916872641 shutdown_connections 2024-02-17T12:58:59.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.447+0000 7f22efd93740 1 -- 172.21.15.18:0/1916872641 wait complete. 2024-02-17T12:58:59.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.447+0000 7f22efd93740 20 client.27944 trim_cache size 0 max 16384 2024-02-17T12:58:59.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22efd93740 1 Processor -- start 2024-02-17T12:58:59.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22efd93740 1 -- start start 2024-02-17T12:58:59.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22efd93740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e9ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22efd93740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e92bd0 0x559807e9b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22efd93740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 0x559807ea2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22efd93740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559807e8d9e0 con 0x559807e9b7b0 2024-02-17T12:58:59.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22efd93740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559807e8d790 con 0x559807e92bd0 2024-02-17T12:58:59.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22efd93740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559807e9df20 con 0x559807e922b0 2024-02-17T12:58:59.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22e759e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 0x559807ea2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22e759e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 0x559807ea2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51572/0 (socket says 172.21.15.18:51572) 2024-02-17T12:58:59.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22e759e640 1 -- 172.21.15.18:0/3623166039 learned_addr learned my addr 172.21.15.18:0/3623166039 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22e6d9d640 1 --2- 172.21.15.18:0/3623166039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e9ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.462+0000 7f22e659c640 1 --2- 172.21.15.18:0/3623166039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e92bd0 0x559807e9b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.463+0000 7f22e759e640 1 -- 172.21.15.18:0/3623166039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 msgr2=0x559807e9ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.463+0000 7f22e759e640 1 --2- 172.21.15.18:0/3623166039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e9ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.463+0000 7f22e759e640 1 -- 172.21.15.18:0/3623166039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e92bd0 msgr2=0x559807e9b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.463+0000 7f22e759e640 1 --2- 172.21.15.18:0/3623166039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e92bd0 0x559807e9b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.463+0000 7f22e759e640 1 -- 172.21.15.18:0/3623166039 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559807d94ac0 con 0x559807e9b7b0 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.463+0000 7f22e759e640 1 --2- 172.21.15.18:0/3623166039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 0x559807ea2600 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009c40 tx=0x7f22d801d5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fa1009327df530 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.463+0000 7f22dcff9640 1 -- 172.21.15.18:0/3623166039 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8022040 con 0x559807e9b7b0 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.463+0000 7f22dcff9640 1 -- 172.21.15.18:0/3623166039 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22d801d9a0 con 0x559807e9b7b0 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.463+0000 7f22dcff9640 1 -- 172.21.15.18:0/3623166039 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8027680 con 0x559807e9b7b0 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 -- 172.21.15.18:0/3623166039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 msgr2=0x559807ea2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 --2- 172.21.15.18:0/3623166039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 0x559807ea2600 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009c40 tx=0x7f22d801d5c0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 -- 172.21.15.18:0/3623166039 shutdown_connections 2024-02-17T12:58:59.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 --2- 172.21.15.18:0/3623166039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 0x559807ea2600 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 --2- 172.21.15.18:0/3623166039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e92bd0 0x559807e9b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 --2- 172.21.15.18:0/3623166039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807e9ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 -- 172.21.15.18:0/3623166039 >> 172.21.15.18:0/3623166039 conn(0x559807e8c990 msgr2=0x559807e96ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 -- 172.21.15.18:0/3623166039 shutdown_connections 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 -- 172.21.15.18:0/3623166039 wait complete. 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 -- start start 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 Processor -- start 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807ea18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e92bd0 0x559807ea1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 0x559807f002b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559807f007f0 con 0x559807e9b7b0 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559807f00960 con 0x559807e92bd0 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559807f00b20 con 0x559807e922b0 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.464+0000 7f22efd93740 10 client.0 _ll_register_callbacks cb 0x7f22e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e759e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 0x559807f002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e92bd0 0x559807ea1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e6d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807ea18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e92bd0 0x559807ea1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:58178/0 (socket says 172.21.15.18:58178) 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e659c640 1 -- 172.21.15.18:0/2046635200 learned_addr learned my addr 172.21.15.18:0/2046635200 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e759e640 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 0x559807f002b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51578/0 (socket says 172.21.15.18:51578) 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e659c640 1 -- 172.21.15.18:0/2046635200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 msgr2=0x559807ea18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e659c640 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807ea18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e659c640 1 -- 172.21.15.18:0/2046635200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 msgr2=0x559807f002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e659c640 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 0x559807f002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e659c640 1 -- 172.21.15.18:0/2046635200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559807df8610 con 0x559807e92bd0 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e759e640 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 0x559807f002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22e659c640 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e92bd0 0x559807ea1e30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x559807e91d80 tx=0x7f22d002bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e24a0a7dccc8fb07 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22ce7fc640 1 -- 172.21.15.18:0/2046635200 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d002c900 con 0x559807e92bd0 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22efd93740 1 -- 172.21.15.18:0/2046635200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22d002ca60 con 0x559807e92bd0 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22ce7fc640 1 -- 172.21.15.18:0/2046635200 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f22d0034440 con 0x559807e92bd0 2024-02-17T12:58:59.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22ce7fc640 1 -- 172.21.15.18:0/2046635200 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d0034600 con 0x559807e92bd0 2024-02-17T12:58:59.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22efd93740 10 client.27950 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:59.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.465+0000 7f22efd93740 1 -- 172.21.15.18:0/2046635200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x559807dba6a0 con 0x559807e92bd0 2024-02-17T12:58:59.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.466+0000 7f22ce7fc640 1 -- 172.21.15.18:0/2046635200 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22d002ca60 con 0x559807e92bd0 2024-02-17T12:58:59.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.466+0000 7f22ce7fc640 1 -- 172.21.15.18:0/2046635200 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f22d003c050 con 0x559807e92bd0 2024-02-17T12:58:59.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.466+0000 7f22efd93740 10 client.27950 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:59.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.466+0000 7f22efd93740 10 client.27950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:59.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.466+0000 7f22efd93740 20 client.27950 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:59.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.466+0000 7f22efd93740 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559807f08080 0x559807f0a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.466+0000 7f22efd93740 4 client.27950 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:58:59.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.466+0000 7f22efd93740 1 -- 172.21.15.18:0/2046635200 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559807b85890 con 0x559807f08080 2024-02-17T12:58:59.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.466+0000 7f22e6d9d640 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559807f08080 0x559807f0a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.466+0000 7f22e6d9d640 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559807f08080 0x559807f0a4a0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ea24bce87b64eeb server_cookie=5878020f7bbfe853 in_seq=0 out_seq=0 2024-02-17T12:58:59.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.466+0000 7f22ce7fc640 10 client.27950 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:59.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ce7fc640 1 -- 172.21.15.18:0/2046635200 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559807b85890 con 0x559807f08080 2024-02-17T12:58:59.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ce7fc640 10 client.27950 handle_command_reply: tid=0 2024-02-17T12:58:59.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22efd93740 1 client.27950 shutdown 2024-02-17T12:58:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22efd93740 1 -- 172.21.15.18:0/2046635200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e92bd0 msgr2=0x559807ea1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22efd93740 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e92bd0 0x559807ea1e30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x559807e91d80 tx=0x7f22d002bc80 comp rx=0 tx=0).stop 2024-02-17T12:58:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22efd93740 1 -- 172.21.15.18:0/2046635200 shutdown_connections 2024-02-17T12:58:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22efd93740 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559807f08080 0x559807f0a4a0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22efd93740 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559807e9b7b0 0x559807f002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22efd93740 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559807e92bd0 0x559807ea1e30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ce7fc640 0 client.27950 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22efd93740 1 --2- 172.21.15.18:0/2046635200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559807e922b0 0x559807ea18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22efd93740 1 -- 172.21.15.18:0/2046635200 >> 172.21.15.18:0/2046635200 conn(0x559807e8c990 msgr2=0x559807e97e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22efd93740 1 -- 172.21.15.18:0/2046635200 shutdown_connections 2024-02-17T12:58:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22efd93740 1 -- 172.21.15.18:0/2046635200 wait complete. 2024-02-17T12:58:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22efd93740 20 client.27950 trim_cache size 0 max 16384 2024-02-17T12:58:59.470 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:58:59.470 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:58:59.470 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:58:59.470 INFO:teuthology.orchestra.run.smithi018.stdout: "c1ae062b-56e9-4e03-8bad-8170f7b1767c": { 2024-02-17T12:58:59.470 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:58:59.470 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "c1ae062b-56e9-4e03-8bad-8170f7b1767c", 2024-02-17T12:58:59.470 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:58:59.470 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:58:59.470 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:58:59.471 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:58:59.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ed0df640 1 -- 172.21.15.18:0/2098384677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f22d40a2230 msgr2=0x7f22d40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ed0df640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f22d40a2230 0x7f22d40a46f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f22d8005fd0 tx=0x7f22d8011810 comp rx=0 tx=0).stop 2024-02-17T12:58:59.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ed0df640 1 -- 172.21.15.18:0/2098384677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22e81007f0 msgr2=0x7f22e819d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ed0df640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22e81007f0 0x7f22e819d580 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f22e000e9f0 tx=0x7f22e000edf0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ed0df640 1 -- 172.21.15.18:0/2098384677 shutdown_connections 2024-02-17T12:58:59.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ed0df640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f22d40a2230 0x7f22d40a46f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ed0df640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22e8105050 0x7f22e8197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ed0df640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22e8101190 0x7f22e819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ed0df640 1 --2- 172.21.15.18:0/2098384677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22e81007f0 0x7f22e819d580 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.467+0000 7f22ed0df640 1 -- 172.21.15.18:0/2098384677 >> 172.21.15.18:0/2098384677 conn(0x7f22e80765a0 msgr2=0x7f22e80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.468+0000 7f22ed0df640 1 -- 172.21.15.18:0/2098384677 shutdown_connections 2024-02-17T12:58:59.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:58:59.468+0000 7f22ed0df640 1 -- 172.21.15.18:0/2098384677 wait complete. 2024-02-17T12:58:59.473 INFO:tasks.cephfs.filesystem:scrub status for tag:c1ae062b-56e9-4e03-8bad-8170f7b1767c - {'path': '/', 'tag': 'c1ae062b-56e9-4e03-8bad-8170f7b1767c', 'options': 'recursive,force'} 2024-02-17T12:58:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.913+0000 7f96652c9640 1 -- 172.21.15.18:0/289806583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9660100190 msgr2=0x7f966010c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.913+0000 7f96652c9640 1 --2- 172.21.15.18:0/289806583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9660100190 0x7f966010c6a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f965800acc0 tx=0x7f965802bec0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.913+0000 7f96652c9640 1 -- 172.21.15.18:0/289806583 shutdown_connections 2024-02-17T12:58:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.913+0000 7f96652c9640 1 --2- 172.21.15.18:0/289806583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9660100190 0x7f966010c6a0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.913+0000 7f96652c9640 1 --2- 172.21.15.18:0/289806583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96600ff7d0 0x7f96600ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.913+0000 7f96652c9640 1 --2- 172.21.15.18:0/289806583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96601057e0 0x7f9660105bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.913+0000 7f96652c9640 1 -- 172.21.15.18:0/289806583 >> 172.21.15.18:0/289806583 conn(0x7f9660076640 msgr2=0x7f96600fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.913+0000 7f96652c9640 1 -- 172.21.15.18:0/289806583 shutdown_connections 2024-02-17T12:58:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.913+0000 7f96652c9640 1 -- 172.21.15.18:0/289806583 wait complete. 2024-02-17T12:58:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f96652c9640 1 Processor -- start 2024-02-17T12:58:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f96652c9640 1 -- start start 2024-02-17T12:58:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f96652c9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96600ff7d0 0x7f9660195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f96652c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9660100190 0x7f96601959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f96652c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96601057e0 0x7f9660199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f96652c9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966019a330 con 0x7f96600ff7d0 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f96652c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966019a4a0 con 0x7f9660100190 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f96652c9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966019a660 con 0x7f96601057e0 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f965effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96600ff7d0 0x7f9660195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f965f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96601057e0 0x7f9660199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f965effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96600ff7d0 0x7f9660195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51604/0 (socket says 172.21.15.18:51604) 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f965e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9660100190 0x7f96601959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.914+0000 7f965effd640 1 -- 172.21.15.18:0/3063647438 learned_addr learned my addr 172.21.15.18:0/3063647438 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.915+0000 7f965f7fe640 1 -- 172.21.15.18:0/3063647438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9660100190 msgr2=0x7f96601959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.915+0000 7f965f7fe640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9660100190 0x7f96601959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.915+0000 7f965f7fe640 1 -- 172.21.15.18:0/3063647438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96600ff7d0 msgr2=0x7f9660195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.915+0000 7f965f7fe640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96600ff7d0 0x7f9660195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.915+0000 7f965f7fe640 1 -- 172.21.15.18:0/3063647438 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965400f790 con 0x7f96601057e0 2024-02-17T12:58:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.915+0000 7f965effd640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96600ff7d0 0x7f9660195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.915+0000 7f965f7fe640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96601057e0 0x7f9660199d30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f965802c400 tx=0x7f96580326b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.915+0000 7f9647fff640 1 -- 172.21.15.18:0/3063647438 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658003d80 con 0x7f96601057e0 2024-02-17T12:58:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.915+0000 7f96652c9640 1 -- 172.21.15.18:0/3063647438 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965800a920 con 0x7f96601057e0 2024-02-17T12:58:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.915+0000 7f9647fff640 1 -- 172.21.15.18:0/3063647438 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96580328a0 con 0x7f96601057e0 2024-02-17T12:58:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.915+0000 7f96652c9640 1 -- 172.21.15.18:0/3063647438 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96601a1fe0 con 0x7f96601057e0 2024-02-17T12:58:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.915+0000 7f9647fff640 1 -- 172.21.15.18:0/3063647438 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96580347c0 con 0x7f96601057e0 2024-02-17T12:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f9647fff640 1 -- 172.21.15.18:0/3063647438 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f965803b040 con 0x7f96601057e0 2024-02-17T12:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f9667f70740 1 Processor -- start 2024-02-17T12:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f9647fff640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96480a22d0 0x7f96480a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f9667f70740 1 -- start start 2024-02-17T12:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f9647fff640 1 -- 172.21.15.18:0/3063647438 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f96580f00a0 con 0x7f96601057e0 2024-02-17T12:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f965e7fc640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96480a22d0 0x7f96480a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f9667f70740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dbaad0 0x561cf9db9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f9667f70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f9667f70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dc32e0 0x561cf9dc56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f9667f70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf9d289f0 con 0x561cf9db9d60 2024-02-17T12:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f9667f70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf9d25f20 con 0x561cf9dc32e0 2024-02-17T12:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f9667f70740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf9d21700 con 0x561cf9dbaad0 2024-02-17T12:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.918+0000 7f965e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f965e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dba1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51606/0 (socket says 172.21.15.18:51606) 2024-02-17T12:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f965e7fc640 1 -- 172.21.15.18:0/2762037836 learned_addr learned my addr 172.21.15.18:0/2762037836 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f965effd640 1 --2- 172.21.15.18:0/2762037836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dbaad0 0x561cf9db9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f965f7fe640 1 --2- 172.21.15.18:0/2762037836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dc32e0 0x561cf9dc56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f965e7fc640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96480a22d0 0x7f96480a4790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f965400f630 tx=0x7f965400ea30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f965effd640 1 -- 172.21.15.18:0/2762037836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dc32e0 msgr2=0x561cf9dc56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f965effd640 1 --2- 172.21.15.18:0/2762037836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dc32e0 0x561cf9dc56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f965effd640 1 -- 172.21.15.18:0/2762037836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 msgr2=0x561cf9dba1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f965effd640 1 --2- 172.21.15.18:0/2762037836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dba1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f965effd640 1 -- 172.21.15.18:0/2762037836 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cf9cb77b0 con 0x561cf9dbaad0 2024-02-17T12:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f965e7fc640 1 --2- 172.21.15.18:0/2762037836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f965effd640 1 --2- 172.21.15.18:0/2762037836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dbaad0 0x561cf9db9820 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9650011440 tx=0x7f9650011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c8e81612d7c35 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f96457fa640 1 -- 172.21.15.18:0/2762037836 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650004030 con 0x561cf9dbaad0 2024-02-17T12:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f96457fa640 1 -- 172.21.15.18:0/2762037836 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9650004190 con 0x561cf9dbaad0 2024-02-17T12:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.919+0000 7f96457fa640 1 -- 172.21.15.18:0/2762037836 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965000f880 con 0x561cf9dbaad0 2024-02-17T12:58:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.920+0000 7f9667f70740 1 -- 172.21.15.18:0/2762037836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dbaad0 msgr2=0x561cf9db9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.920+0000 7f9667f70740 1 --2- 172.21.15.18:0/2762037836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dbaad0 0x561cf9db9820 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9650011440 tx=0x7f9650011900 comp rx=0 tx=0).stop 2024-02-17T12:58:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.920+0000 7f9667f70740 1 -- 172.21.15.18:0/2762037836 shutdown_connections 2024-02-17T12:58:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.920+0000 7f9667f70740 1 --2- 172.21.15.18:0/2762037836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dc32e0 0x561cf9dc56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.920+0000 7f9667f70740 1 --2- 172.21.15.18:0/2762037836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.920+0000 7f9667f70740 1 --2- 172.21.15.18:0/2762037836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dbaad0 0x561cf9db9820 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.920+0000 7f9667f70740 1 -- 172.21.15.18:0/2762037836 >> 172.21.15.18:0/2762037836 conn(0x561cf9d81550 msgr2=0x561cf9db4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.920+0000 7f9667f70740 1 -- 172.21.15.18:0/2762037836 shutdown_connections 2024-02-17T12:58:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.920+0000 7f9667f70740 1 -- 172.21.15.18:0/2762037836 wait complete. 2024-02-17T12:58:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f9667f70740 1 -- start start 2024-02-17T12:58:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f9667f70740 1 Processor -- start 2024-02-17T12:58:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f9667f70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f9667f70740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dbaad0 0x561cf9dc0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f9667f70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dc32e0 0x561cf9dc0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f9667f70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf9cc6bf0 con 0x561cf9db9d60 2024-02-17T12:58:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f9667f70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf99f5250 con 0x561cf9dc32e0 2024-02-17T12:58:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f9667f70740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf9ca7200 con 0x561cf9dbaad0 2024-02-17T12:58:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f9667f70740 10 client.0 _ll_register_callbacks cb 0x561cf9d15d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f965effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f965effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51608/0 (socket says 172.21.15.18:51608) 2024-02-17T12:58:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f965effd640 1 -- 172.21.15.18:0/1018855796 learned_addr learned my addr 172.21.15.18:0/1018855796 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f965e7fc640 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dbaad0 0x561cf9dc0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f965f7fe640 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dc32e0 0x561cf9dc0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f965effd640 1 -- 172.21.15.18:0/1018855796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dbaad0 msgr2=0x561cf9dc0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f965effd640 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dbaad0 0x561cf9dc0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f965effd640 1 -- 172.21.15.18:0/1018855796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dc32e0 msgr2=0x561cf9dc0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f965effd640 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dc32e0 0x561cf9dc0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f965effd640 1 -- 172.21.15.18:0/1018855796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9650009c50 con 0x561cf9db9d60 2024-02-17T12:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.921+0000 7f965f7fe640 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dc32e0 0x561cf9dc0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f965effd640 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc02c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9650000c00 tx=0x7f9650005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5055914a82167809 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9642ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9642ff5640 1 -- 172.21.15.18:0/1018855796 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650005670 con 0x561cf9db9d60 2024-02-17T12:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9642ff5640 1 -- 172.21.15.18:0/1018855796 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f96500035f0 con 0x561cf9db9d60 2024-02-17T12:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9667f70740 1 -- 172.21.15.18:0/1018855796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cf9a21600 con 0x561cf9db9d60 2024-02-17T12:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9642ff5640 1 -- 172.21.15.18:0/1018855796 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96500037b0 con 0x561cf9db9d60 2024-02-17T12:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9667f70740 10 client.18891 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9667f70740 1 -- 172.21.15.18:0/1018855796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x561cf9d1fd80 con 0x561cf9db9d60 2024-02-17T12:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9642ff5640 1 -- 172.21.15.18:0/1018855796 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561cf9a21600 con 0x561cf9db9d60 2024-02-17T12:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9642ff5640 1 -- 172.21.15.18:0/1018855796 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9650021040 con 0x561cf9db9d60 2024-02-17T12:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9667f70740 10 client.18891 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9667f70740 10 client.18891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9667f70740 20 client.18891 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9667f70740 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561cf9dd3520 0x561cf9dd5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9667f70740 4 client.18891 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.922+0000 7f9667f70740 1 -- 172.21.15.18:0/1018855796 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cf9a721b0 con 0x561cf9dd3520 2024-02-17T12:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.923+0000 7f965e7fc640 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561cf9dd3520 0x561cf9dd5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.923+0000 7f9642ff5640 10 client.18891 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.923+0000 7f965e7fc640 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561cf9dd3520 0x561cf9dd5960 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54cb32dc7dc5abd server_cookie=6c78f76a41947a26 in_seq=0 out_seq=0 2024-02-17T12:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9642ff5640 1 -- 172.21.15.18:0/1018855796 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561cf9a721b0 con 0x561cf9dd3520 2024-02-17T12:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9642ff5640 10 client.18891 handle_command_reply: tid=0 2024-02-17T12:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9667f70740 1 client.18891 shutdown 2024-02-17T12:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9667f70740 1 -- 172.21.15.18:0/1018855796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 msgr2=0x561cf9dc02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9667f70740 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc02c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9650000c00 tx=0x7f9650005270 comp rx=0 tx=0).stop 2024-02-17T12:58:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9667f70740 1 -- 172.21.15.18:0/1018855796 shutdown_connections 2024-02-17T12:58:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9667f70740 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561cf9dd3520 0x561cf9dd5960 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9667f70740 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dc32e0 0x561cf9dc0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9642ff5640 0 client.18891 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9667f70740 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dbaad0 0x561cf9dc0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9667f70740 1 --2- 172.21.15.18:0/1018855796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc02c0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9667f70740 1 -- 172.21.15.18:0/1018855796 >> 172.21.15.18:0/1018855796 conn(0x561cf9d81550 msgr2=0x561cf9db67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9667f70740 1 -- 172.21.15.18:0/1018855796 shutdown_connections 2024-02-17T12:58:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9667f70740 1 -- 172.21.15.18:0/1018855796 wait complete. 2024-02-17T12:58:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.924+0000 7f9667f70740 20 client.18891 trim_cache size 0 max 16384 2024-02-17T12:58:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f9667f70740 1 Processor -- start 2024-02-17T12:58:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f9667f70740 1 -- start start 2024-02-17T12:58:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f9667f70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f9667f70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dbaad0 0x561cf9dc0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f9667f70740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dc32e0 0x561cf9dc0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f9667f70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf9dd0120 con 0x561cf9db9d60 2024-02-17T12:58:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f9667f70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf9dd3320 con 0x561cf9dbaad0 2024-02-17T12:58:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f9667f70740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf9db5620 con 0x561cf9dc32e0 2024-02-17T12:58:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f965effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f965f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dc32e0 0x561cf9dc0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f965effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51614/0 (socket says 172.21.15.18:51614) 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f965effd640 1 -- 172.21.15.18:0/1589226121 learned_addr learned my addr 172.21.15.18:0/1589226121 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f965e7fc640 1 --2- 172.21.15.18:0/1589226121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dbaad0 0x561cf9dc0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f965f7fe640 1 -- 172.21.15.18:0/1589226121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dbaad0 msgr2=0x561cf9dc0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f965f7fe640 1 --2- 172.21.15.18:0/1589226121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dbaad0 0x561cf9dc0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f965f7fe640 1 -- 172.21.15.18:0/1589226121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 msgr2=0x561cf9dc29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f965f7fe640 1 --2- 172.21.15.18:0/1589226121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f965f7fe640 1 -- 172.21.15.18:0/1589226121 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cf9cbc7b0 con 0x561cf9dc32e0 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f965e7fc640 1 --2- 172.21.15.18:0/1589226121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dbaad0 0x561cf9dc0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.938+0000 7f965effd640 1 --2- 172.21.15.18:0/1589226121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.939+0000 7f965f7fe640 1 --2- 172.21.15.18:0/1589226121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dc32e0 0x561cf9dc0d70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f96580ed920 tx=0x7f965802cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61f9ca7324cf0afc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.939+0000 7f9644ff9640 1 -- 172.21.15.18:0/1589226121 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96580bcc70 con 0x561cf9dc32e0 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.939+0000 7f9644ff9640 1 -- 172.21.15.18:0/1589226121 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96580c0d10 con 0x561cf9dc32e0 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.939+0000 7f9644ff9640 1 -- 172.21.15.18:0/1589226121 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96580c5130 con 0x561cf9dc32e0 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.939+0000 7f9667f70740 1 -- 172.21.15.18:0/1589226121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dc32e0 msgr2=0x561cf9dc0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.939+0000 7f9667f70740 1 --2- 172.21.15.18:0/1589226121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dc32e0 0x561cf9dc0d70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f96580ed920 tx=0x7f965802cd30 comp rx=0 tx=0).stop 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.939+0000 7f9667f70740 1 -- 172.21.15.18:0/1589226121 shutdown_connections 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 --2- 172.21.15.18:0/1589226121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dc32e0 0x561cf9dc0d70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 --2- 172.21.15.18:0/1589226121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dbaad0 0x561cf9dc0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 --2- 172.21.15.18:0/1589226121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 -- 172.21.15.18:0/1589226121 >> 172.21.15.18:0/1589226121 conn(0x561cf9d81550 msgr2=0x561cf9dbebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 -- 172.21.15.18:0/1589226121 shutdown_connections 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 -- 172.21.15.18:0/1589226121 wait complete. 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 -- start start 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 Processor -- start 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dbaad0 0x561cf9dca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dc32e0 0x561cf9dc84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf9dc8a80 con 0x561cf9db9d60 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf9dc8bf0 con 0x561cf9dbaad0 2024-02-17T12:58:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf9dc8db0 con 0x561cf9dc32e0 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f9667f70740 10 client.0 _ll_register_callbacks cb 0x7f9650009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f965effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f965effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51624/0 (socket says 172.21.15.18:51624) 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f965e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dbaad0 0x561cf9dca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f965effd640 1 -- 172.21.15.18:0/903964460 learned_addr learned my addr 172.21.15.18:0/903964460 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.940+0000 7f965f7fe640 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dc32e0 0x561cf9dc84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f965e7fc640 1 -- 172.21.15.18:0/903964460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dc32e0 msgr2=0x561cf9dc84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f965e7fc640 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dc32e0 0x561cf9dc84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f965e7fc640 1 -- 172.21.15.18:0/903964460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 msgr2=0x561cf9dc9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f965e7fc640 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f965e7fc640 1 -- 172.21.15.18:0/903964460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cf9d1fd80 con 0x561cf9dbaad0 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f965effd640 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f965e7fc640 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dbaad0 0x561cf9dca330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f965400fcc0 tx=0x7f96540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2a898036a887ff6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:58:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f9642ff5640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f9642ff5640 1 -- 172.21.15.18:0/903964460 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654031440 con 0x561cf9dbaad0 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f9667f70740 1 -- 172.21.15.18:0/903964460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9650004190 con 0x561cf9dbaad0 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f9642ff5640 1 -- 172.21.15.18:0/903964460 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f96540315a0 con 0x561cf9dbaad0 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f9642ff5640 1 -- 172.21.15.18:0/903964460 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9654031760 con 0x561cf9dbaad0 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f9667f70740 10 client.27968 fetch_fsmap learned FSMap version 46 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f9667f70740 1 -- 172.21.15.18:0/903964460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f96540099c0 con 0x561cf9dbaad0 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f9642ff5640 1 -- 172.21.15.18:0/903964460 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9650004190 con 0x561cf9dbaad0 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f9642ff5640 1 -- 172.21.15.18:0/903964460 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9654020470 con 0x561cf9dbaad0 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f9667f70740 10 client.27968 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f9667f70740 10 client.27968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.941+0000 7f9667f70740 20 client.27968 populate_metadata read hostname 'smithi018' 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.942+0000 7f9667f70740 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561cf9e2aca0 0x561cf9e2d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.942+0000 7f9667f70740 4 client.27968 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.942+0000 7f9667f70740 1 -- 172.21.15.18:0/903964460 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561cf9a5e550 con 0x561cf9e2aca0 2024-02-17T12:58:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.942+0000 7f965effd640 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561cf9e2aca0 0x561cf9e2d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:58:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.942+0000 7f965effd640 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561cf9e2aca0 0x561cf9e2d0c0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9141b03c1a2e05e2 server_cookie=d94c81eff69ecec2 in_seq=0 out_seq=0 2024-02-17T12:58:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.942+0000 7f9642ff5640 10 client.27968 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9642ff5640 1 -- 172.21.15.18:0/903964460 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561cf9a5e550 con 0x561cf9e2aca0 2024-02-17T12:58:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9642ff5640 10 client.27968 handle_command_reply: tid=0 2024-02-17T12:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9667f70740 1 client.27968 shutdown 2024-02-17T12:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9667f70740 1 -- 172.21.15.18:0/903964460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dbaad0 msgr2=0x561cf9dca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9667f70740 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dbaad0 0x561cf9dca330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f965400fcc0 tx=0x7f96540046c0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9667f70740 1 -- 172.21.15.18:0/903964460 shutdown_connections 2024-02-17T12:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9667f70740 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561cf9e2aca0 0x561cf9e2d0c0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9667f70740 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cf9dc32e0 0x561cf9dc84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9642ff5640 0 client.27968 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9667f70740 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cf9dbaad0 0x561cf9dca330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9667f70740 1 --2- 172.21.15.18:0/903964460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561cf9db9d60 0x561cf9dc9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9667f70740 1 -- 172.21.15.18:0/903964460 >> 172.21.15.18:0/903964460 conn(0x561cf9d81550 msgr2=0x561cf9db3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9667f70740 1 -- 172.21.15.18:0/903964460 shutdown_connections 2024-02-17T12:58:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.982+0000 7f9667f70740 1 -- 172.21.15.18:0/903964460 wait complete. 2024-02-17T12:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f9667f70740 20 client.27968 trim_cache size 0 max 16384 2024-02-17T12:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:58:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:58:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f96652c9640 1 -- 172.21.15.18:0/3063647438 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96480a22d0 msgr2=0x7f96480a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f96652c9640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96480a22d0 0x7f96480a4790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f965400f630 tx=0x7f965400ea30 comp rx=0 tx=0).stop 2024-02-17T12:58:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f96652c9640 1 -- 172.21.15.18:0/3063647438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96601057e0 msgr2=0x7f9660199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:58:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f96652c9640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96601057e0 0x7f9660199d30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f965802c400 tx=0x7f96580326b0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f96652c9640 1 -- 172.21.15.18:0/3063647438 shutdown_connections 2024-02-17T12:58:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f96652c9640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96480a22d0 0x7f96480a4790 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f96652c9640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96601057e0 0x7f9660199d30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f96652c9640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9660100190 0x7f96601959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f96652c9640 1 --2- 172.21.15.18:0/3063647438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96600ff7d0 0x7f9660195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:58:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f96652c9640 1 -- 172.21.15.18:0/3063647438 >> 172.21.15.18:0/3063647438 conn(0x7f9660076640 msgr2=0x7f96601024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:58:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f96652c9640 1 -- 172.21.15.18:0/3063647438 shutdown_connections 2024-02-17T12:58:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:58:59.983+0000 7f96652c9640 1 -- 172.21.15.18:0/3063647438 wait complete. 2024-02-17T12:59:00.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: pgmap v283: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:58:59 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.288 INFO:tasks.workunit.client.0.smithi018.stdout: 1 470486 5.29 MB/sec execute 375 sec latency 18.244 ms 2024-02-17T12:59:00.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: pgmap v283: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-02-17T12:59:00.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:00.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:00.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:00.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:00.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:00.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:00.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:00.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:58:59 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: pgmap v283: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-02-17T12:59:00.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:00.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:00.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:00.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:00.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:00.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:00.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:00.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:00.451 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:00 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:01.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:01 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:59:01.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:01 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:01.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:01 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:01.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:01 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:01.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 470685 5.27 MB/sec execute 376 sec latency 865.083 ms 2024-02-17T12:59:01.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:01 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:59:01.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:01 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:01.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:01 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:01.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:01 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:01.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:01 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:59:01.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:01 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:01.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:01 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:01.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:01 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:02.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:02 smithi018 ceph-mon[32625]: pgmap v284: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-02-17T12:59:02.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 470685 5.26 MB/sec execute 377 sec latency 1865.201 ms 2024-02-17T12:59:02.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:02 smithi119 ceph-mon[36188]: pgmap v284: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-02-17T12:59:02.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:02 smithi146 ceph-mon[36454]: pgmap v284: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-02-17T12:59:03.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 470685 5.25 MB/sec execute 378 sec latency 2865.258 ms 2024-02-17T12:59:04.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:04 smithi018 ceph-mon[32625]: pgmap v285: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-02-17T12:59:04.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 471283 5.24 MB/sec execute 379 sec latency 3477.139 ms 2024-02-17T12:59:04.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:04 smithi119 ceph-mon[36188]: pgmap v285: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-02-17T12:59:04.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:04 smithi146 ceph-mon[36454]: pgmap v285: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-02-17T12:59:05.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:05 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:59:05.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:05 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:59:05.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 472784 5.25 MB/sec execute 380 sec latency 26.373 ms 2024-02-17T12:59:05.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:05 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:59:05.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:05 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:59:05.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:05 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:59:05.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:05 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:59:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:06 smithi018 ceph-mon[32625]: pgmap v286: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-02-17T12:59:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:06 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:59:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:06 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:59:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:06 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:59:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:06 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:59:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:06 smithi018 ceph-mon[32625]: osdmap e97: 12 total, 12 up, 12 in 2024-02-17T12:59:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:06 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:59:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:06 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:59:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:06 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:59:06.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 474188 5.26 MB/sec execute 381 sec latency 15.905 ms 2024-02-17T12:59:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:06 smithi119 ceph-mon[36188]: pgmap v286: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-02-17T12:59:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:06 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:59:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:06 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:59:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:06 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:59:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:06 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:59:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:06 smithi119 ceph-mon[36188]: osdmap e97: 12 total, 12 up, 12 in 2024-02-17T12:59:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:06 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:59:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:06 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:59:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:06 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:59:06.350 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:06 smithi146 ceph-mon[36454]: pgmap v286: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-02-17T12:59:06.350 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:06 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:59:06.350 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:06 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:59:06.350 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:06 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:59:06.350 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:06 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:59:06.350 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:06 smithi146 ceph-mon[36454]: osdmap e97: 12 total, 12 up, 12 in 2024-02-17T12:59:06.350 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:06 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:59:06.350 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:06 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:59:06.350 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:06 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T12:59:07.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 474231 5.24 MB/sec execute 382 sec latency 958.160 ms 2024-02-17T12:59:08.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:08 smithi018 ceph-mon[32625]: pgmap v288: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-02-17T12:59:08.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 475147 5.24 MB/sec execute 383 sec latency 1523.236 ms 2024-02-17T12:59:08.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:08 smithi119 ceph-mon[36188]: pgmap v288: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-02-17T12:59:08.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:08 smithi146 ceph-mon[36454]: pgmap v288: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-02-17T12:59:09.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 476868 5.25 MB/sec execute 384 sec latency 17.013 ms 2024-02-17T12:59:10.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:10 smithi018 ceph-mon[32625]: pgmap v289: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-02-17T12:59:10.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 477780 5.25 MB/sec execute 385 sec latency 506.817 ms 2024-02-17T12:59:10.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:10 smithi119 ceph-mon[36188]: pgmap v289: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-02-17T12:59:10.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:10 smithi146 ceph-mon[36454]: pgmap v289: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-02-17T12:59:11.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 477780 5.24 MB/sec execute 386 sec latency 1506.931 ms 2024-02-17T12:59:12.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:12 smithi018 ceph-mon[32625]: pgmap v290: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-02-17T12:59:12.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 477780 5.22 MB/sec execute 387 sec latency 2506.982 ms 2024-02-17T12:59:12.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:12 smithi119 ceph-mon[36188]: pgmap v290: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-02-17T12:59:12.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:12 smithi146 ceph-mon[36454]: pgmap v290: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-02-17T12:59:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.086+0000 7f46d2b48640 1 -- 172.21.15.18:0/182276032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102c80 msgr2=0x7f46cc10b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.086+0000 7f46d2b48640 1 --2- 172.21.15.18:0/182276032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102c80 0x7f46cc10b160 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009900 tx=0x7f46b802bc30 comp rx=0 tx=0).stop 2024-02-17T12:59:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.086+0000 7f46d2b48640 1 -- 172.21.15.18:0/182276032 shutdown_connections 2024-02-17T12:59:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.086+0000 7f46d2b48640 1 --2- 172.21.15.18:0/182276032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46cc10b7d0 0x7f46cc10dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.086+0000 7f46d2b48640 1 --2- 172.21.15.18:0/182276032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102c80 0x7f46cc10b160 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.086+0000 7f46d2b48640 1 --2- 172.21.15.18:0/182276032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46cc102360 0x7f46cc102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.086+0000 7f46d2b48640 1 -- 172.21.15.18:0/182276032 >> 172.21.15.18:0/182276032 conn(0x7f46cc0771b0 msgr2=0x7f46cc0775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.086+0000 7f46d2b48640 1 -- 172.21.15.18:0/182276032 shutdown_connections 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.086+0000 7f46d2b48640 1 -- 172.21.15.18:0/182276032 wait complete. 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46d2b48640 1 Processor -- start 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46d2b48640 1 -- start start 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46d2b48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46cc102360 0x7f46cc0fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46d2b48640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102c80 0x7f46cc1004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46d2b48640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46cc10b7d0 0x7f46cc100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46d2b48640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc1a5ac0 con 0x7f46cc102c80 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46d2b48640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc1a5c30 con 0x7f46cc102360 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46d2b48640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc1a5df0 con 0x7f46cc10b7d0 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102c80 0x7f46cc1004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46d10e5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46cc10b7d0 0x7f46cc100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102c80 0x7f46cc1004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56240/0 (socket says 172.21.15.18:56240) 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46cbfff640 1 -- 172.21.15.18:0/808702714 learned_addr learned my addr 172.21.15.18:0/808702714 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.087+0000 7f46d08e4640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46cc102360 0x7f46cc0fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.088+0000 7f46cbfff640 1 -- 172.21.15.18:0/808702714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46cc10b7d0 msgr2=0x7f46cc100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.088+0000 7f46cbfff640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46cc10b7d0 0x7f46cc100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.088+0000 7f46cbfff640 1 -- 172.21.15.18:0/808702714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46cc102360 msgr2=0x7f46cc0fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.088+0000 7f46cbfff640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46cc102360 0x7f46cc0fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.088+0000 7f46cbfff640 1 -- 172.21.15.18:0/808702714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8009560 con 0x7f46cc102c80 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.088+0000 7f46d10e5640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46cc10b7d0 0x7f46cc100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.088+0000 7f46cbfff640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102c80 0x7f46cc1004f0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f46b8000c00 tx=0x7f46b8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.088+0000 7f46c9ffb640 1 -- 172.21.15.18:0/808702714 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8004500 con 0x7f46cc102c80 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.088+0000 7f46c9ffb640 1 -- 172.21.15.18:0/808702714 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46b80052d0 con 0x7f46cc102c80 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.088+0000 7f46d2b48640 1 -- 172.21.15.18:0/808702714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc1a6070 con 0x7f46cc102c80 2024-02-17T12:59:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.088+0000 7f46c9ffb640 1 -- 172.21.15.18:0/808702714 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8003e50 con 0x7f46cc102c80 2024-02-17T12:59:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.088+0000 7f46d2b48640 1 -- 172.21.15.18:0/808702714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc1a6580 con 0x7f46cc102c80 2024-02-17T12:59:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.090+0000 7f46c9ffb640 1 -- 172.21.15.18:0/808702714 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f46b80035f0 con 0x7f46cc102c80 2024-02-17T12:59:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46d57fa740 1 Processor -- start 2024-02-17T12:59:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46d57fa740 1 -- start start 2024-02-17T12:59:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46c9ffb640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46bc0a22d0 0x7f46bc0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46d57fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b0e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46d57fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b0f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46d57fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b182e0 0x55a9a1b1a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46d57fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a1a7d9f0 con 0x55a9a1b0ed60 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46d57fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a1a7af20 con 0x55a9a1b0fad0 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46d57fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a1a76700 con 0x55a9a1b182e0 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46d10e5640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46bc0a22d0 0x7f46bc0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46c9ffb640 1 -- 172.21.15.18:0/808702714 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f46b80f07a0 con 0x7f46cc102c80 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b0f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b0f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56252/0 (socket says 172.21.15.18:56252) 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.091+0000 7f46cbfff640 1 -- 172.21.15.18:0/2901969694 learned_addr learned my addr 172.21.15.18:0/2901969694 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.092+0000 7f46d08e4640 1 --2- 172.21.15.18:0/2901969694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b0e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.092+0000 7f46d10e5640 1 --2- 172.21.15.18:0/2901969694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b182e0 0x55a9a1b1a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.092+0000 7f46cbfff640 1 -- 172.21.15.18:0/2901969694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b182e0 msgr2=0x55a9a1b1a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.092+0000 7f46cbfff640 1 --2- 172.21.15.18:0/2901969694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b182e0 0x55a9a1b1a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.092+0000 7f46cbfff640 1 -- 172.21.15.18:0/2901969694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b0fad0 msgr2=0x55a9a1b0e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.092+0000 7f46cbfff640 1 --2- 172.21.15.18:0/2901969694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b0e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.092+0000 7f46cbfff640 1 -- 172.21.15.18:0/2901969694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9a1a0c7b0 con 0x55a9a1b0ed60 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.092+0000 7f46d10e5640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46bc0a22d0 0x7f46bc0a4790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f46c4005fd0 tx=0x7f46c40125b0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.092+0000 7f46cbfff640 1 --2- 172.21.15.18:0/2901969694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b0f1e0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f46b80b97e0 tx=0x7f46b8003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc9e53cc2abf0d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.092+0000 7f46b6ffd640 1 -- 172.21.15.18:0/2901969694 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b80c14d0 con 0x55a9a1b0ed60 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.092+0000 7f46b6ffd640 1 -- 172.21.15.18:0/2901969694 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46b80c5b20 con 0x55a9a1b0ed60 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.092+0000 7f46b6ffd640 1 -- 172.21.15.18:0/2901969694 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b80c5690 con 0x55a9a1b0ed60 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.093+0000 7f46d57fa740 1 -- 172.21.15.18:0/2901969694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 msgr2=0x55a9a1b0f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.093+0000 7f46d57fa740 1 --2- 172.21.15.18:0/2901969694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b0f1e0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f46b80b97e0 tx=0x7f46b8003920 comp rx=0 tx=0).stop 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.093+0000 7f46d57fa740 1 -- 172.21.15.18:0/2901969694 shutdown_connections 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.093+0000 7f46d57fa740 1 --2- 172.21.15.18:0/2901969694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b182e0 0x55a9a1b1a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.093+0000 7f46d57fa740 1 --2- 172.21.15.18:0/2901969694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b0f1e0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.093+0000 7f46d57fa740 1 --2- 172.21.15.18:0/2901969694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b0e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.093+0000 7f46d57fa740 1 -- 172.21.15.18:0/2901969694 >> 172.21.15.18:0/2901969694 conn(0x55a9a1ad6550 msgr2=0x55a9a1b09a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.093+0000 7f46d57fa740 1 -- 172.21.15.18:0/2901969694 shutdown_connections 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.093+0000 7f46d57fa740 1 -- 172.21.15.18:0/2901969694 wait complete. 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.093+0000 7f46d57fa740 1 -- start start 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.093+0000 7f46d57fa740 1 Processor -- start 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46d57fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b15200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46d57fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b15740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46d57fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b182e0 0x55a9a1b15c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46d57fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a1a1bbf0 con 0x55a9a1b0fad0 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46d57fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a174a250 con 0x55a9a1b182e0 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46d57fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a19fc200 con 0x55a9a1b0ed60 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46d57fa740 10 client.0 _ll_register_callbacks cb 0x55a9a1a6ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b15740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b15740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56256/0 (socket says 172.21.15.18:56256) 2024-02-17T12:59:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46cbfff640 1 -- 172.21.15.18:0/1171252254 learned_addr learned my addr 172.21.15.18:0/1171252254 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46d10e5640 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b182e0 0x55a9a1b15c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46d08e4640 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b15200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46cbfff640 1 -- 172.21.15.18:0/1171252254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b0ed60 msgr2=0x55a9a1b15200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46cbfff640 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b15200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46cbfff640 1 -- 172.21.15.18:0/1171252254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b182e0 msgr2=0x55a9a1b15c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46cbfff640 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b182e0 0x55a9a1b15c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46cbfff640 1 -- 172.21.15.18:0/1171252254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc1a6580 con 0x55a9a1b0fad0 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46cbfff640 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b15740 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f46b80bd030 tx=0x7f46b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63ebcbfe8bc2b57 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.094+0000 7f46abfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46abfff640 1 -- 172.21.15.18:0/1171252254 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b80c14d0 con 0x55a9a1b0fad0 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46abfff640 1 -- 172.21.15.18:0/1171252254 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f46b80c1170 con 0x55a9a1b0fad0 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46d57fa740 1 -- 172.21.15.18:0/1171252254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9a1776600 con 0x55a9a1b0fad0 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46abfff640 1 -- 172.21.15.18:0/1171252254 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46b80c5690 con 0x55a9a1b0fad0 2024-02-17T12:59:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46d57fa740 10 client.18909 fetch_fsmap learned FSMap version 46 2024-02-17T12:59:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46d57fa740 1 -- 172.21.15.18:0/1171252254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55a9a1a74d80 con 0x55a9a1b0fad0 2024-02-17T12:59:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46abfff640 1 -- 172.21.15.18:0/1171252254 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46b80a3080 con 0x55a9a1b0fad0 2024-02-17T12:59:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46abfff640 1 -- 172.21.15.18:0/1171252254 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f46b803c070 con 0x55a9a1b0fad0 2024-02-17T12:59:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46d57fa740 10 client.18909 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:59:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46d57fa740 10 client.18909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46d57fa740 20 client.18909 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46d57fa740 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9a1b27da0 0x55a9a1b2a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46d57fa740 4 client.18909 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:59:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.095+0000 7f46d57fa740 1 -- 172.21.15.18:0/1171252254 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9a17c71b0 con 0x55a9a1b27da0 2024-02-17T12:59:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.096+0000 7f46d08e4640 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9a1b27da0 0x55a9a1b2a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.096+0000 7f46d08e4640 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9a1b27da0 0x55a9a1b2a190 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e94ec0ccc0f4c0 server_cookie=316a2d459ddc5c93 in_seq=0 out_seq=0 2024-02-17T12:59:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.096+0000 7f46abfff640 10 client.18909 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46abfff640 1 -- 172.21.15.18:0/1171252254 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9a17c71b0 con 0x55a9a1b27da0 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46abfff640 10 client.18909 handle_command_reply: tid=0 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46d57fa740 1 client.18909 shutdown 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46d57fa740 1 -- 172.21.15.18:0/1171252254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 msgr2=0x55a9a1b15740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46d57fa740 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b15740 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f46b80bd030 tx=0x7f46b8009260 comp rx=0 tx=0).stop 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46d57fa740 1 -- 172.21.15.18:0/1171252254 shutdown_connections 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46d57fa740 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9a1b27da0 0x55a9a1b2a190 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46d57fa740 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b182e0 0x55a9a1b15c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46d57fa740 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b15740 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46abfff640 0 client.18909 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46d57fa740 1 --2- 172.21.15.18:0/1171252254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b15200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46d57fa740 1 -- 172.21.15.18:0/1171252254 >> 172.21.15.18:0/1171252254 conn(0x55a9a1ad6550 msgr2=0x55a9a1b0b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46d57fa740 1 -- 172.21.15.18:0/1171252254 shutdown_connections 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46d57fa740 1 -- 172.21.15.18:0/1171252254 wait complete. 2024-02-17T12:59:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.097+0000 7f46d57fa740 20 client.18909 trim_cache size 0 max 16384 2024-02-17T12:59:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46d57fa740 1 Processor -- start 2024-02-17T12:59:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46d57fa740 1 -- start start 2024-02-17T12:59:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46d57fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b17930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46d57fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46d57fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b182e0 0x55a9a1b15aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46d57fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a1b0ebc0 con 0x55a9a1b0fad0 2024-02-17T12:59:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46d57fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a1b100c0 con 0x55a9a1b0ed60 2024-02-17T12:59:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46d57fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a1b0ac10 con 0x55a9a1b182e0 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b141e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56262/0 (socket says 172.21.15.18:56262) 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46cbfff640 1 -- 172.21.15.18:0/952774825 learned_addr learned my addr 172.21.15.18:0/952774825 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46d10e5640 1 --2- 172.21.15.18:0/952774825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b182e0 0x55a9a1b15aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46d08e4640 1 --2- 172.21.15.18:0/952774825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b17930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46cbfff640 1 -- 172.21.15.18:0/952774825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b182e0 msgr2=0x55a9a1b15aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46cbfff640 1 --2- 172.21.15.18:0/952774825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b182e0 0x55a9a1b15aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46cbfff640 1 -- 172.21.15.18:0/952774825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b0ed60 msgr2=0x55a9a1b17930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46cbfff640 1 --2- 172.21.15.18:0/952774825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b17930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46cbfff640 1 -- 172.21.15.18:0/952774825 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9a1a117b0 con 0x55a9a1b0fad0 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.112+0000 7f46d10e5640 1 --2- 172.21.15.18:0/952774825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b182e0 0x55a9a1b15aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.113+0000 7f46cbfff640 1 --2- 172.21.15.18:0/952774825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b141e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f46b8005ec0 tx=0x7f46b802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c992f78a7be599 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.113+0000 7f46b67fc640 1 -- 172.21.15.18:0/952774825 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b80bda80 con 0x55a9a1b0fad0 2024-02-17T12:59:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.113+0000 7f46b67fc640 1 -- 172.21.15.18:0/952774825 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a9a1776600 con 0x55a9a1b0fad0 2024-02-17T12:59:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.113+0000 7f46b67fc640 1 -- 172.21.15.18:0/952774825 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b80b91f0 con 0x55a9a1b0fad0 2024-02-17T12:59:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 -- 172.21.15.18:0/952774825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 msgr2=0x55a9a1b141e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 --2- 172.21.15.18:0/952774825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b141e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f46b8005ec0 tx=0x7f46b802bc30 comp rx=0 tx=0).stop 2024-02-17T12:59:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 -- 172.21.15.18:0/952774825 shutdown_connections 2024-02-17T12:59:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 --2- 172.21.15.18:0/952774825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b182e0 0x55a9a1b15aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 --2- 172.21.15.18:0/952774825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b141e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 --2- 172.21.15.18:0/952774825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b17930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 -- 172.21.15.18:0/952774825 >> 172.21.15.18:0/952774825 conn(0x55a9a1ad6550 msgr2=0x55a9a1b13ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 -- 172.21.15.18:0/952774825 shutdown_connections 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 -- 172.21.15.18:0/952774825 wait complete. 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 -- start start 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 Processor -- start 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b1f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b1d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b182e0 0x55a9a1b1de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a1b1e500 con 0x55a9a1b0ed60 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a1b7e230 con 0x55a9a1b182e0 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9a1b7e370 con 0x55a9a1b0fad0 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.114+0000 7f46d57fa740 10 client.0 _ll_register_callbacks cb 0x7f46c000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d08e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b1f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d08e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b1f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56278/0 (socket says 172.21.15.18:56278) 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d08e4640 1 -- 172.21.15.18:0/241013716 learned_addr learned my addr 172.21.15.18:0/241013716 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46cbfff640 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b1d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d10e5640 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b182e0 0x55a9a1b1de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d08e4640 1 -- 172.21.15.18:0/241013716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b0fad0 msgr2=0x55a9a1b1d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d08e4640 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b1d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d08e4640 1 -- 172.21.15.18:0/241013716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b182e0 msgr2=0x55a9a1b1de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d08e4640 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b182e0 0x55a9a1b1de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d08e4640 1 -- 172.21.15.18:0/241013716 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc1a6580 con 0x55a9a1b0ed60 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d08e4640 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b1f350 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x55a9a1b2aa20 tx=0x7f46c000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af237a07c7cfc40 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46abfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46abfff640 1 -- 172.21.15.18:0/241013716 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0016040 con 0x55a9a1b0ed60 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46abfff640 1 -- 172.21.15.18:0/241013716 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f46c001f0e0 con 0x55a9a1b0ed60 2024-02-17T12:59:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d57fa740 1 -- 172.21.15.18:0/241013716 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46b80c5b20 con 0x55a9a1b0ed60 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46abfff640 1 -- 172.21.15.18:0/241013716 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46c001f2a0 con 0x55a9a1b0ed60 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d57fa740 10 client.18915 fetch_fsmap learned FSMap version 46 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.115+0000 7f46d57fa740 1 -- 172.21.15.18:0/241013716 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55a9a1a37540 con 0x55a9a1b0ed60 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.116+0000 7f46abfff640 1 -- 172.21.15.18:0/241013716 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46b80c5b20 con 0x55a9a1b0ed60 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.116+0000 7f46abfff640 1 -- 172.21.15.18:0/241013716 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 46) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f46c0031af0 con 0x55a9a1b0ed60 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.116+0000 7f46d57fa740 10 client.18915 fetch_fsmap finished waiting for FSMap version 46 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.116+0000 7f46d57fa740 10 client.18915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.116+0000 7f46d57fa740 20 client.18915 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.116+0000 7f46d57fa740 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9a1b7fa90 0x55a9a1b81eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.116+0000 7f46d57fa740 4 client.18915 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.116+0000 7f46d57fa740 1 -- 172.21.15.18:0/241013716 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a9a17b3550 con 0x55a9a1b7fa90 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.116+0000 7f46cbfff640 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9a1b7fa90 0x55a9a1b81eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.116+0000 7f46abfff640 10 client.18915 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.116+0000 7f46cbfff640 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9a1b7fa90 0x55a9a1b81eb0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd5bc475523c1a5 server_cookie=ac27c9399f7b54e6 in_seq=0 out_seq=0 2024-02-17T12:59:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.158+0000 7f46abfff640 1 -- 172.21.15.18:0/241013716 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a9a17b3550 con 0x55a9a1b7fa90 2024-02-17T12:59:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.158+0000 7f46abfff640 10 client.18915 handle_command_reply: tid=0 2024-02-17T12:59:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.158+0000 7f46d57fa740 1 client.18915 shutdown 2024-02-17T12:59:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.158+0000 7f46d57fa740 1 -- 172.21.15.18:0/241013716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 msgr2=0x55a9a1b1f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d57fa740 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b1f350 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x55a9a1b2aa20 tx=0x7f46c000bde0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d57fa740 1 -- 172.21.15.18:0/241013716 shutdown_connections 2024-02-17T12:59:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d57fa740 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a9a1b7fa90 0x55a9a1b81eb0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d57fa740 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a9a1b182e0 0x55a9a1b1de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d57fa740 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a9a1b0fad0 0x55a9a1b1d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d57fa740 1 --2- 172.21.15.18:0/241013716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9a1b0ed60 0x55a9a1b1f350 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46abfff640 0 client.18915 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d57fa740 1 -- 172.21.15.18:0/241013716 >> 172.21.15.18:0/241013716 conn(0x55a9a1ad6550 msgr2=0x55a9a1b0b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d57fa740 1 -- 172.21.15.18:0/241013716 shutdown_connections 2024-02-17T12:59:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d57fa740 1 -- 172.21.15.18:0/241013716 wait complete. 2024-02-17T12:59:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d57fa740 20 client.18915 trim_cache size 0 max 16384 2024-02-17T12:59:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:59:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:59:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:59:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:59:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d2b48640 1 -- 172.21.15.18:0/808702714 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46bc0a22d0 msgr2=0x7f46bc0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d2b48640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46bc0a22d0 0x7f46bc0a4790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f46c4005fd0 tx=0x7f46c40125b0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d2b48640 1 -- 172.21.15.18:0/808702714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102c80 msgr2=0x7f46cc1004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d2b48640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102c80 0x7f46cc1004f0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f46b8000c00 tx=0x7f46b8004040 comp rx=0 tx=0).stop 2024-02-17T12:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d2b48640 1 -- 172.21.15.18:0/808702714 shutdown_connections 2024-02-17T12:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d2b48640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f46bc0a22d0 0x7f46bc0a4790 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d2b48640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46cc10b7d0 0x7f46cc100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d2b48640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102c80 0x7f46cc1004f0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d2b48640 1 --2- 172.21.15.18:0/808702714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46cc102360 0x7f46cc0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.159+0000 7f46d2b48640 1 -- 172.21.15.18:0/808702714 >> 172.21.15.18:0/808702714 conn(0x7f46cc0771b0 msgr2=0x7f46cc0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.160+0000 7f46d2b48640 1 -- 172.21.15.18:0/808702714 shutdown_connections 2024-02-17T12:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:13.160+0000 7f46d2b48640 1 -- 172.21.15.18:0/808702714 wait complete. 2024-02-17T12:59:13.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 477780 5.21 MB/sec execute 388 sec latency 3507.081 ms 2024-02-17T12:59:14.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:14 smithi018 ceph-mon[32625]: pgmap v291: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-02-17T12:59:14.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:14 smithi018 ceph-mon[32625]: osdmap e98: 12 total, 12 up, 12 in 2024-02-17T12:59:14.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 479759 5.23 MB/sec execute 389 sec latency 3701.166 ms 2024-02-17T12:59:14.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:14 smithi119 ceph-mon[36188]: pgmap v291: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-02-17T12:59:14.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:14 smithi119 ceph-mon[36188]: osdmap e98: 12 total, 12 up, 12 in 2024-02-17T12:59:14.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:14 smithi146 ceph-mon[36454]: pgmap v291: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-02-17T12:59:14.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:14 smithi146 ceph-mon[36454]: osdmap e98: 12 total, 12 up, 12 in 2024-02-17T12:59:15.290 INFO:tasks.workunit.client.0.smithi018.stdout: 1 481174 5.23 MB/sec execute 390 sec latency 16.385 ms 2024-02-17T12:59:16.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:16 smithi018 ceph-mon[32625]: pgmap v293: 97 pgs: 97 active+clean; 170 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.0 MiB/s wr, 209 op/s 2024-02-17T12:59:16.289 INFO:tasks.workunit.client.0.smithi018.stdout: 1 481360 5.22 MB/sec execute 391 sec latency 880.957 ms 2024-02-17T12:59:16.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:16 smithi119 ceph-mon[36188]: pgmap v293: 97 pgs: 97 active+clean; 170 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.0 MiB/s wr, 209 op/s 2024-02-17T12:59:16.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:16 smithi146 ceph-mon[36454]: pgmap v293: 97 pgs: 97 active+clean; 170 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.0 MiB/s wr, 209 op/s 2024-02-17T12:59:17.290 INFO:tasks.workunit.client.0.smithi018.stdout: 1 481360 5.21 MB/sec execute 392 sec latency 1881.069 ms 2024-02-17T12:59:18.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:18 smithi018 ceph-mon[32625]: pgmap v294: 97 pgs: 97 active+clean; 170 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-02-17T12:59:18.290 INFO:tasks.workunit.client.0.smithi018.stdout: 1 481360 5.19 MB/sec execute 393 sec latency 2881.177 ms 2024-02-17T12:59:18.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:18 smithi119 ceph-mon[36188]: pgmap v294: 97 pgs: 97 active+clean; 170 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-02-17T12:59:18.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:18 smithi146 ceph-mon[36454]: pgmap v294: 97 pgs: 97 active+clean; 170 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-02-17T12:59:19.290 INFO:tasks.workunit.client.0.smithi018.stdout: 1 482265 5.19 MB/sec execute 394 sec latency 3486.789 ms 2024-02-17T12:59:20.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:20 smithi018 ceph-mon[32625]: pgmap v295: 97 pgs: 97 active+clean; 189 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.5 MiB/s wr, 147 op/s 2024-02-17T12:59:20.290 INFO:tasks.workunit.client.0.smithi018.stdout: 1 483904 5.20 MB/sec execute 395 sec latency 17.004 ms 2024-02-17T12:59:20.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:20 smithi119 ceph-mon[36188]: pgmap v295: 97 pgs: 97 active+clean; 189 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.5 MiB/s wr, 147 op/s 2024-02-17T12:59:20.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:20 smithi146 ceph-mon[36454]: pgmap v295: 97 pgs: 97 active+clean; 189 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.5 MiB/s wr, 147 op/s 2024-02-17T12:59:21.290 INFO:tasks.workunit.client.0.smithi018.stdout: 1 484911 5.20 MB/sec execute 396 sec latency 459.082 ms 2024-02-17T12:59:22.290 INFO:tasks.workunit.client.0.smithi018.stdout: 1 484911 5.19 MB/sec execute 397 sec latency 1459.179 ms 2024-02-17T12:59:22.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:22 smithi119 ceph-mon[36188]: pgmap v296: 97 pgs: 97 active+clean; 189 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.5 MiB/s wr, 147 op/s 2024-02-17T12:59:22.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:22 smithi146 ceph-mon[36454]: pgmap v296: 97 pgs: 97 active+clean; 189 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.5 MiB/s wr, 147 op/s 2024-02-17T12:59:22.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:22 smithi018 ceph-mon[32625]: pgmap v296: 97 pgs: 97 active+clean; 189 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.5 MiB/s wr, 147 op/s 2024-02-17T12:59:23.290 INFO:tasks.workunit.client.0.smithi018.stdout: 1 484911 5.18 MB/sec execute 398 sec latency 2459.285 ms 2024-02-17T12:59:24.290 INFO:tasks.workunit.client.0.smithi018.stdout: 1 485795 5.17 MB/sec execute 399 sec latency 3065.688 ms 2024-02-17T12:59:24.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:24 smithi119 ceph-mon[36188]: pgmap v297: 97 pgs: 97 active+clean; 189 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.5 MiB/s wr, 147 op/s 2024-02-17T12:59:24.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:24 smithi146 ceph-mon[36454]: pgmap v297: 97 pgs: 97 active+clean; 189 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.5 MiB/s wr, 147 op/s 2024-02-17T12:59:24.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:24 smithi018 ceph-mon[32625]: pgmap v297: 97 pgs: 97 active+clean; 189 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.5 MiB/s wr, 147 op/s 2024-02-17T12:59:25.290 INFO:tasks.workunit.client.0.smithi018.stdout: 1 487356 5.18 MB/sec execute 400 sec latency 32.634 ms 2024-02-17T12:59:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.257+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3596705896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7b8106490 msgr2=0x7fd7b810e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.257+0000 7fd7bd3d0640 1 --2- 172.21.15.18:0/3596705896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7b8106490 0x7fd7b810e060 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009cd0 tx=0x7fd7b002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.257+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3596705896 shutdown_connections 2024-02-17T12:59:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.257+0000 7fd7bd3d0640 1 --2- 172.21.15.18:0/3596705896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7b8106490 0x7fd7b810e060 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.257+0000 7fd7bd3d0640 1 --2- 172.21.15.18:0/3596705896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7b8101d70 0x7fd7b8105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.257+0000 7fd7bd3d0640 1 --2- 172.21.15.18:0/3596705896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7b81013c0 0x7fd7b81017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.257+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3596705896 >> 172.21.15.18:0/3596705896 conn(0x7fd7b80fcc60 msgr2=0x7fd7b80ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.257+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3596705896 shutdown_connections 2024-02-17T12:59:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.257+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3596705896 wait complete. 2024-02-17T12:59:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7bd3d0640 1 Processor -- start 2024-02-17T12:59:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7bd3d0640 1 -- start start 2024-02-17T12:59:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7bd3d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7b81013c0 0x7fd7b819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7bd3d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7b8106490 0x7fd7b819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7bd3d0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7b8199700 0x7fd7b8199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7bd3d0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b819a1d0 con 0x7fd7b8199700 2024-02-17T12:59:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7bd3d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b819a340 con 0x7fd7b8106490 2024-02-17T12:59:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7bd3d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b819a500 con 0x7fd7b81013c0 2024-02-17T12:59:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7b77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7b8199700 0x7fd7b8199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7b77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7b8199700 0x7fd7b8199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35292/0 (socket says 172.21.15.18:35292) 2024-02-17T12:59:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7b77fe640 1 -- 172.21.15.18:0/3301196678 learned_addr learned my addr 172.21.15.18:0/3301196678 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7affff640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7b8106490 0x7fd7b819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.258+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7b81013c0 0x7fd7b819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.259+0000 7fd7b77fe640 1 -- 172.21.15.18:0/3301196678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7b81013c0 msgr2=0x7fd7b819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.259+0000 7fd7b77fe640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7b81013c0 0x7fd7b819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.259+0000 7fd7b77fe640 1 -- 172.21.15.18:0/3301196678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7b8106490 msgr2=0x7fd7b819fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.259+0000 7fd7b77fe640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7b8106490 0x7fd7b819fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.259+0000 7fd7b77fe640 1 -- 172.21.15.18:0/3301196678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b0009900 con 0x7fd7b8199700 2024-02-17T12:59:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.259+0000 7fd7b77fe640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7b8199700 0x7fd7b8199b80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00348f0 tx=0x7fd7b0034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.259+0000 7fd7b4ff9640 1 -- 172.21.15.18:0/3301196678 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b00328c0 con 0x7fd7b8199700 2024-02-17T12:59:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.259+0000 7fd7b4ff9640 1 -- 172.21.15.18:0/3301196678 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7b0003e50 con 0x7fd7b8199700 2024-02-17T12:59:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.259+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3301196678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b81a6440 con 0x7fd7b8199700 2024-02-17T12:59:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.259+0000 7fd7b4ff9640 1 -- 172.21.15.18:0/3301196678 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b00048a0 con 0x7fd7b8199700 2024-02-17T12:59:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.259+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3301196678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b81a6910 con 0x7fd7b8199700 2024-02-17T12:59:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.261+0000 7fd7b4ff9640 1 -- 172.21.15.18:0/3301196678 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd7b000a040 con 0x7fd7b8199700 2024-02-17T12:59:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7bffa4740 1 Processor -- start 2024-02-17T12:59:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7bffa4740 1 -- start start 2024-02-17T12:59:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7b4ff9640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd7a40a2350 0x7fd7a40a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7bffa4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6ead0 0x561b8fa6d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7bffa4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa6e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7bffa4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa796d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7bffa4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8f9dc9f0 con 0x561b8fa6dd60 2024-02-17T12:59:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7bffa4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8f9d9f20 con 0x561b8fa6ead0 2024-02-17T12:59:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7bffa4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8f9d5700 con 0x561b8fa772e0 2024-02-17T12:59:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7b4ff9640 1 -- 172.21.15.18:0/3301196678 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd7b00f0840 con 0x7fd7b8199700 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7affff640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd7a40a2350 0x7fd7a40a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7affff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa6e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7affff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa6e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35304/0 (socket says 172.21.15.18:35304) 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7affff640 1 -- 172.21.15.18:0/3557021006 learned_addr learned my addr 172.21.15.18:0/3557021006 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.262+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/3557021006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6ead0 0x561b8fa6d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.263+0000 7fd7b77fe640 1 --2- 172.21.15.18:0/3557021006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa796d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.263+0000 7fd7b6ffd640 1 -- 172.21.15.18:0/3557021006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 msgr2=0x561b8fa796d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.263+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/3557021006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa796d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.263+0000 7fd7b6ffd640 1 -- 172.21.15.18:0/3557021006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 msgr2=0x561b8fa6e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.263+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/3557021006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa6e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.263+0000 7fd7b6ffd640 1 -- 172.21.15.18:0/3557021006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b8f96b7b0 con 0x561b8fa6ead0 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.263+0000 7fd7b77fe640 1 --2- 172.21.15.18:0/3557021006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa796d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.263+0000 7fd7affff640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd7a40a2350 0x7fd7a40a4810 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0005ce0 tx=0x7fd7a000bad0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.263+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/3557021006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6ead0 0x561b8fa6d820 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800abf0 tx=0x7fd7a800f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b24870ee9ae119b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.263+0000 7fd7ad7fa640 1 -- 172.21.15.18:0/3557021006 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8005020 con 0x561b8fa6ead0 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.263+0000 7fd7ad7fa640 1 -- 172.21.15.18:0/3557021006 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7a8005660 con 0x561b8fa6ead0 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.263+0000 7fd7ad7fa640 1 -- 172.21.15.18:0/3557021006 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80186a0 con 0x561b8fa6ead0 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.264+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3557021006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6ead0 msgr2=0x561b8fa6d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.264+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3557021006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6ead0 0x561b8fa6d820 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800abf0 tx=0x7fd7a800f3c0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.264+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3557021006 shutdown_connections 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.264+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3557021006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa796d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.264+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3557021006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa6e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.264+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3557021006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6ead0 0x561b8fa6d820 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.264+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3557021006 >> 172.21.15.18:0/3557021006 conn(0x561b8fa35550 msgr2=0x561b8fa68a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.264+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3557021006 shutdown_connections 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.264+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3557021006 wait complete. 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.264+0000 7fd7bffa4740 1 -- start start 2024-02-17T12:59:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7bffa4740 1 Processor -- start 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7bffa4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa74280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7bffa4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6ead0 0x561b8fa747c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7bffa4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa74d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7bffa4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8f97abf0 con 0x561b8fa6dd60 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7bffa4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8f6a9250 con 0x561b8fa6ead0 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7bffa4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8f95b200 con 0x561b8fa772e0 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7bffa4740 10 client.0 _ll_register_callbacks cb 0x561b8f9c9d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7b6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa74280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7b6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa74280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35306/0 (socket says 172.21.15.18:35306) 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7b6ffd640 1 -- 172.21.15.18:0/846006665 learned_addr learned my addr 172.21.15.18:0/846006665 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7affff640 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6ead0 0x561b8fa747c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7b77fe640 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa74d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7b6ffd640 1 -- 172.21.15.18:0/846006665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 msgr2=0x561b8fa74d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa74d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7b6ffd640 1 -- 172.21.15.18:0/846006665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6ead0 msgr2=0x561b8fa747c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6ead0 0x561b8fa747c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7b6ffd640 1 -- 172.21.15.18:0/846006665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a800a820 con 0x561b8fa6dd60 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa74280 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800abc0 tx=0x7fd7a8004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e75b8870b761529 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.265+0000 7fd79e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd79e7fc640 1 -- 172.21.15.18:0/846006665 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80052b0 con 0x561b8fa6dd60 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd79e7fc640 1 -- 172.21.15.18:0/846006665 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd7a8005410 con 0x561b8fa6dd60 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd7bffa4740 1 -- 172.21.15.18:0/846006665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b8f6d5600 con 0x561b8fa6dd60 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd79e7fc640 1 -- 172.21.15.18:0/846006665 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a8018440 con 0x561b8fa6dd60 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd7bffa4740 10 client.18927 fetch_fsmap learned FSMap version 47 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd7bffa4740 1 -- 172.21.15.18:0/846006665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x561b8f9d3d80 con 0x561b8fa6dd60 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd79e7fc640 1 -- 172.21.15.18:0/846006665 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561b8f6d5600 con 0x561b8fa6dd60 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd79e7fc640 1 -- 172.21.15.18:0/846006665 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 47) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd7a8023690 con 0x561b8fa6dd60 2024-02-17T12:59:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd7bffa4740 10 client.18927 fetch_fsmap finished waiting for FSMap version 47 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd7bffa4740 10 client.18927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd7bffa4740 20 client.18927 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561b8fa875a0 0x561b8fa899e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd7bffa4740 4 client.18927 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.266+0000 7fd7bffa4740 1 -- 172.21.15.18:0/846006665 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b8f7261b0 con 0x561b8fa875a0 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.267+0000 7fd7affff640 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561b8fa875a0 0x561b8fa899e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.267+0000 7fd7affff640 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561b8fa875a0 0x561b8fa899e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b31f0ec2ae70f5 server_cookie=938ca8aaa0fa63ad in_seq=0 out_seq=0 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.267+0000 7fd79e7fc640 10 client.18927 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd79e7fc640 1 -- 172.21.15.18:0/846006665 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b8f7261b0 con 0x561b8fa875a0 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd79e7fc640 10 client.18927 handle_command_reply: tid=0 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd7bffa4740 1 client.18927 shutdown 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd7bffa4740 1 -- 172.21.15.18:0/846006665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 msgr2=0x561b8fa74280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa74280 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800abc0 tx=0x7fd7a8004c30 comp rx=0 tx=0).stop 2024-02-17T12:59:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd7bffa4740 1 -- 172.21.15.18:0/846006665 shutdown_connections 2024-02-17T12:59:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561b8fa875a0 0x561b8fa899e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa74d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6ead0 0x561b8fa747c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd79e7fc640 0 client.18927 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/846006665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa74280 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd7bffa4740 1 -- 172.21.15.18:0/846006665 >> 172.21.15.18:0/846006665 conn(0x561b8fa35550 msgr2=0x561b8fa6a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd7bffa4740 1 -- 172.21.15.18:0/846006665 shutdown_connections 2024-02-17T12:59:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd7bffa4740 1 -- 172.21.15.18:0/846006665 wait complete. 2024-02-17T12:59:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.268+0000 7fd7bffa4740 20 client.18927 trim_cache size 0 max 16384 2024-02-17T12:59:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.281+0000 7fd7bffa4740 1 Processor -- start 2024-02-17T12:59:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.281+0000 7fd7bffa4740 1 -- start start 2024-02-17T12:59:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.281+0000 7fd7bffa4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6dd60 0x561b8fa67a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.281+0000 7fd7bffa4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6ead0 0x561b8fa7e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.281+0000 7fd7bffa4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa7e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.281+0000 7fd7bffa4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8f7261b0 con 0x561b8fa6ead0 2024-02-17T12:59:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.281+0000 7fd7bffa4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8f6d5600 con 0x561b8fa6dd60 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.281+0000 7fd7bffa4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8fa69a50 con 0x561b8fa772e0 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7affff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6ead0 0x561b8fa7e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7affff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6ead0 0x561b8fa7e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35310/0 (socket says 172.21.15.18:35310) 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7affff640 1 -- 172.21.15.18:0/3000535641 learned_addr learned my addr 172.21.15.18:0/3000535641 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/3000535641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6dd60 0x561b8fa67a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7b77fe640 1 --2- 172.21.15.18:0/3000535641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa7e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7affff640 1 -- 172.21.15.18:0/3000535641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 msgr2=0x561b8fa7e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7affff640 1 --2- 172.21.15.18:0/3000535641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa7e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7affff640 1 -- 172.21.15.18:0/3000535641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6dd60 msgr2=0x561b8fa67a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7affff640 1 --2- 172.21.15.18:0/3000535641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6dd60 0x561b8fa67a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7affff640 1 -- 172.21.15.18:0/3000535641 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b8f9707b0 con 0x561b8fa6ead0 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/3000535641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6dd60 0x561b8fa67a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7affff640 1 --2- 172.21.15.18:0/3000535641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6ead0 0x561b8fa7e110 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0010a30 tx=0x7fd7a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa848d80bebdbd65 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7acff9640 1 -- 172.21.15.18:0/3000535641 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a000b750 con 0x561b8fa6ead0 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7acff9640 1 -- 172.21.15.18:0/3000535641 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7a0012e90 con 0x561b8fa6ead0 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.282+0000 7fd7acff9640 1 -- 172.21.15.18:0/3000535641 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0020910 con 0x561b8fa6ead0 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.283+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3000535641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6ead0 msgr2=0x561b8fa7e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.283+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3000535641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6ead0 0x561b8fa7e110 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0010a30 tx=0x7fd7a0004730 comp rx=0 tx=0).stop 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.283+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3000535641 shutdown_connections 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.283+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3000535641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa772e0 0x561b8fa7e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.283+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3000535641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6ead0 0x561b8fa7e110 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.283+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3000535641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa6dd60 0x561b8fa67a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.283+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3000535641 >> 172.21.15.18:0/3000535641 conn(0x561b8fa35550 msgr2=0x561b8fa67300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.283+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3000535641 shutdown_connections 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.283+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3000535641 wait complete. 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.283+0000 7fd7bffa4740 1 -- start start 2024-02-17T12:59:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.283+0000 7fd7bffa4740 1 Processor -- start 2024-02-17T12:59:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7bffa4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa7d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7bffa4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa6ead0 0x561b8fa7d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7bffa4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa772e0 0x561b8faddb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7bffa4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8fade050 con 0x561b8fa6dd60 2024-02-17T12:59:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7bffa4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8fade190 con 0x561b8fa772e0 2024-02-17T12:59:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7bffa4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8fade2d0 con 0x561b8fa6ead0 2024-02-17T12:59:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7bffa4740 10 client.0 _ll_register_callbacks cb 0x7fd7a800a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7b6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa7d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7b6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa7d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35312/0 (socket says 172.21.15.18:35312) 2024-02-17T12:59:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7b6ffd640 1 -- 172.21.15.18:0/3143544568 learned_addr learned my addr 172.21.15.18:0/3143544568 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7affff640 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa6ead0 0x561b8fa7d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7b77fe640 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa772e0 0x561b8faddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7b6ffd640 1 -- 172.21.15.18:0/3143544568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa6ead0 msgr2=0x561b8fa7d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa6ead0 0x561b8fa7d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7b6ffd640 1 -- 172.21.15.18:0/3143544568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa772e0 msgr2=0x561b8faddb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa772e0 0x561b8faddb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7b6ffd640 1 -- 172.21.15.18:0/3143544568 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a001a040 con 0x561b8fa6dd60 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7affff640 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa6ead0 0x561b8fa7d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7b77fe640 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa772e0 0x561b8faddb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7b6ffd640 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa7d220 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800f400 tx=0x7fd7a800aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60698be2e119021c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd79e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd79e7fc640 1 -- 172.21.15.18:0/3143544568 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80052b0 con 0x561b8fa6dd60 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd79e7fc640 1 -- 172.21.15.18:0/3143544568 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd7a8000e40 con 0x561b8fa6dd60 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.284+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3143544568 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7a8005660 con 0x561b8fa6dd60 2024-02-17T12:59:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.285+0000 7fd79e7fc640 1 -- 172.21.15.18:0/3143544568 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a8005410 con 0x561b8fa6dd60 2024-02-17T12:59:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.285+0000 7fd7bffa4740 10 client.18933 fetch_fsmap learned FSMap version 47 2024-02-17T12:59:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.285+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3143544568 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x561b8f996540 con 0x561b8fa6dd60 2024-02-17T12:59:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.285+0000 7fd79e7fc640 1 -- 172.21.15.18:0/3143544568 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7a8005660 con 0x561b8fa6dd60 2024-02-17T12:59:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.285+0000 7fd79e7fc640 1 -- 172.21.15.18:0/3143544568 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 47) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd7a80188b0 con 0x561b8fa6dd60 2024-02-17T12:59:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.285+0000 7fd7bffa4740 10 client.18933 fetch_fsmap finished waiting for FSMap version 47 2024-02-17T12:59:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.285+0000 7fd7bffa4740 10 client.18933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.285+0000 7fd7bffa4740 20 client.18933 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.285+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561b8fadf710 0x561b8fae1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.285+0000 7fd7bffa4740 4 client.18933 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:59:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.285+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3143544568 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561b8f712550 con 0x561b8fadf710 2024-02-17T12:59:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.285+0000 7fd7affff640 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561b8fadf710 0x561b8fae1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.286+0000 7fd7affff640 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561b8fadf710 0x561b8fae1b30 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb13ce2f1641357 server_cookie=ea95d6d87c44a6d8 in_seq=0 out_seq=0 2024-02-17T12:59:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:26.286+0000 7fd79e7fc640 10 client.18933 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:26.291 INFO:tasks.workunit.client.0.smithi018.stdout: 1 487356 5.17 MB/sec execute 401 sec latency 1033.510 ms 2024-02-17T12:59:26.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:26 smithi119 ceph-mon[36188]: pgmap v298: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.0 MiB/s wr, 133 op/s 2024-02-17T12:59:26.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:26 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:59:26.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:26 smithi146 ceph-mon[36454]: pgmap v298: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.0 MiB/s wr, 133 op/s 2024-02-17T12:59:26.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:26 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:59:26.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:26 smithi018 ceph-mon[32625]: pgmap v298: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.0 MiB/s wr, 133 op/s 2024-02-17T12:59:26.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:26 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:59:27.291 INFO:tasks.workunit.client.0.smithi018.stdout: 1 487356 5.16 MB/sec execute 402 sec latency 2033.645 ms 2024-02-17T12:59:28.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 487356 5.15 MB/sec execute 403 sec latency 3033.761 ms 2024-02-17T12:59:28.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:28 smithi119 ceph-mon[36188]: pgmap v299: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 2.7 MiB/s wr, 121 op/s 2024-02-17T12:59:28.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:28 smithi146 ceph-mon[36454]: pgmap v299: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 2.7 MiB/s wr, 121 op/s 2024-02-17T12:59:28.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:28 smithi018 ceph-mon[32625]: pgmap v299: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 2.7 MiB/s wr, 121 op/s 2024-02-17T12:59:29.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 487356 5.13 MB/sec execute 404 sec latency 4033.888 ms 2024-02-17T12:59:29.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:59:29.474 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:59:29.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.602+0000 7fafc298c640 1 -- 172.21.15.18:0/513746170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafbc100c20 msgr2=0x7fafbc104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.602+0000 7fafc298c640 1 --2- 172.21.15.18:0/513746170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafbc100c20 0x7fafbc104c60 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009930 tx=0x7fafb002bc30 comp rx=0 tx=0).stop 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.602+0000 7fafc298c640 1 -- 172.21.15.18:0/513746170 shutdown_connections 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.602+0000 7fafc298c640 1 --2- 172.21.15.18:0/513746170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafbc105340 0x7fafbc10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.602+0000 7fafc298c640 1 --2- 172.21.15.18:0/513746170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafbc100c20 0x7fafbc104c60 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.602+0000 7fafc298c640 1 --2- 172.21.15.18:0/513746170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc100270 0x7fafbc100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.602+0000 7fafc298c640 1 -- 172.21.15.18:0/513746170 >> 172.21.15.18:0/513746170 conn(0x7fafbc077230 msgr2=0x7fafbc077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.602+0000 7fafc298c640 1 -- 172.21.15.18:0/513746170 shutdown_connections 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.602+0000 7fafc298c640 1 -- 172.21.15.18:0/513746170 wait complete. 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.603+0000 7fafc298c640 1 Processor -- start 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.603+0000 7fafc298c640 1 -- start start 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.603+0000 7fafc298c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafbc100270 0x7fafbc101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.603+0000 7fafc298c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafbc105340 0x7fafbc102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.603+0000 7fafc298c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc102780 0x7fafbc1a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.603+0000 7fafc298c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc1a5f70 con 0x7fafbc105340 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.603+0000 7fafc298c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc1a60e0 con 0x7fafbc100270 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.603+0000 7fafc298c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc1a62a0 con 0x7fafbc102780 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.603+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafbc105340 0x7fafbc102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.603+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafbc105340 0x7fafbc102240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55190/0 (socket says 172.21.15.18:55190) 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.603+0000 7fafbb7fe640 1 -- 172.21.15.18:0/43115155 learned_addr learned my addr 172.21.15.18:0/43115155 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.603+0000 7fafbbfff640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafbc100270 0x7fafbc101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.604+0000 7fafc0f29640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc102780 0x7fafbc1a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.604+0000 7fafbb7fe640 1 -- 172.21.15.18:0/43115155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc102780 msgr2=0x7fafbc1a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.604+0000 7fafbb7fe640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc102780 0x7fafbc1a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.604+0000 7fafbb7fe640 1 -- 172.21.15.18:0/43115155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafbc100270 msgr2=0x7fafbc101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.604+0000 7fafbb7fe640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafbc100270 0x7fafbc101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.604+0000 7fafbb7fe640 1 -- 172.21.15.18:0/43115155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0009560 con 0x7fafbc105340 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.604+0000 7fafbb7fe640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafbc105340 0x7fafbc102240 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009230 tx=0x7fafb0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:29.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.604+0000 7fafb97fa640 1 -- 172.21.15.18:0/43115155 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb002cbc0 con 0x7fafbc105340 2024-02-17T12:59:29.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.604+0000 7fafb97fa640 1 -- 172.21.15.18:0/43115155 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafb00052d0 con 0x7fafbc105340 2024-02-17T12:59:29.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.604+0000 7fafb97fa640 1 -- 172.21.15.18:0/43115155 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0003e30 con 0x7fafbc105340 2024-02-17T12:59:29.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.604+0000 7fafc298c640 1 -- 172.21.15.18:0/43115155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafbc1a6520 con 0x7fafbc105340 2024-02-17T12:59:29.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.604+0000 7fafc298c640 1 -- 172.21.15.18:0/43115155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafbc1a6a30 con 0x7fafbc105340 2024-02-17T12:59:29.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.606+0000 7fafb97fa640 1 -- 172.21.15.18:0/43115155 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fafb00035f0 con 0x7fafbc105340 2024-02-17T12:59:29.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.606+0000 7fafc51fd740 1 Processor -- start 2024-02-17T12:59:29.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafc51fd740 1 -- start start 2024-02-17T12:59:29.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafb97fa640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fafa40a2260 0x7fafa40a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:29.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafc51fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f82b0 0x55e1df2f8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:29.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafc51fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f8bd0 0x55e1df301270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:29.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafc51fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df3017b0 0x55e1df303ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:29.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafc51fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df267280 con 0x55e1df2f8bd0 2024-02-17T12:59:29.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafc51fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df2647b0 con 0x55e1df2f82b0 2024-02-17T12:59:29.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafc51fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df25ff90 con 0x55e1df3017b0 2024-02-17T12:59:29.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafc0f29640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fafa40a2260 0x7fafa40a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:29.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f8bd0 0x55e1df301270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f8bd0 0x55e1df301270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55206/0 (socket says 172.21.15.18:55206) 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafbb7fe640 1 -- 172.21.15.18:0/44436167 learned_addr learned my addr 172.21.15.18:0/44436167 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafb97fa640 1 -- 172.21.15.18:0/43115155 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fafb00f01a0 con 0x7fafbc105340 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafbbfff640 1 --2- 172.21.15.18:0/44436167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f82b0 0x55e1df2f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafbb7fe640 1 -- 172.21.15.18:0/44436167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df3017b0 msgr2=0x55e1df303ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafbb7fe640 1 --2- 172.21.15.18:0/44436167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df3017b0 0x55e1df303ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafbb7fe640 1 -- 172.21.15.18:0/44436167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f82b0 msgr2=0x55e1df2f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafbb7fe640 1 --2- 172.21.15.18:0/44436167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f82b0 0x55e1df2f8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafbb7fe640 1 -- 172.21.15.18:0/44436167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1df21e570 con 0x55e1df2f8bd0 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafc0f29640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fafa40a2260 0x7fafa40a4720 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fafb40092f0 tx=0x7fafb40097f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.607+0000 7fafbb7fe640 1 --2- 172.21.15.18:0/44436167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f8bd0 0x55e1df301270 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fafb00b9120 tx=0x7fafb0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77afe184993fc0d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.608+0000 7fafa27fc640 1 -- 172.21.15.18:0/44436167 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb00bccc0 con 0x55e1df2f8bd0 2024-02-17T12:59:29.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.608+0000 7fafa27fc640 1 -- 172.21.15.18:0/44436167 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafb00c0e10 con 0x55e1df2f8bd0 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.608+0000 7fafa27fc640 1 -- 172.21.15.18:0/44436167 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb00b5380 con 0x55e1df2f8bd0 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.608+0000 7fafc51fd740 1 -- 172.21.15.18:0/44436167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f8bd0 msgr2=0x55e1df301270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.608+0000 7fafc51fd740 1 --2- 172.21.15.18:0/44436167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f8bd0 0x55e1df301270 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fafb00b9120 tx=0x7fafb0003f90 comp rx=0 tx=0).stop 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.608+0000 7fafc51fd740 1 -- 172.21.15.18:0/44436167 shutdown_connections 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.608+0000 7fafc51fd740 1 --2- 172.21.15.18:0/44436167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df3017b0 0x55e1df303ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.608+0000 7fafc51fd740 1 --2- 172.21.15.18:0/44436167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f8bd0 0x55e1df301270 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.608+0000 7fafc51fd740 1 --2- 172.21.15.18:0/44436167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f82b0 0x55e1df2f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.608+0000 7fafc51fd740 1 -- 172.21.15.18:0/44436167 >> 172.21.15.18:0/44436167 conn(0x55e1df2f2990 msgr2=0x55e1df2f2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.608+0000 7fafc51fd740 1 -- 172.21.15.18:0/44436167 shutdown_connections 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.608+0000 7fafc51fd740 1 -- 172.21.15.18:0/44436167 wait complete. 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.609+0000 7fafc51fd740 1 -- start start 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.609+0000 7fafc51fd740 1 Processor -- start 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.609+0000 7fafc51fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 0x55e1df2fe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.609+0000 7fafc51fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df2f8bd0 0x55e1df2feb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:29.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.609+0000 7fafc51fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df3017b0 0x55e1df2ff040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.609+0000 7fafc51fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1def33430 con 0x55e1df2f82b0 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.609+0000 7fafc51fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1deeefe00 con 0x55e1df3017b0 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.609+0000 7fafc51fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df1e4fd0 con 0x55e1df2f8bd0 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.609+0000 7fafc51fd740 10 client.0 _ll_register_callbacks cb 0x55e1df23afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.609+0000 7fafbbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 0x55e1df2fe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.609+0000 7fafbbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 0x55e1df2fe5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55210/0 (socket says 172.21.15.18:55210) 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.609+0000 7fafbbfff640 1 -- 172.21.15.18:0/3757055650 learned_addr learned my addr 172.21.15.18:0/3757055650 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7fafc0f29640 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df3017b0 0x55e1df2ff040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7fafbb7fe640 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df2f8bd0 0x55e1df2feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7fafbbfff640 1 -- 172.21.15.18:0/3757055650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df2f8bd0 msgr2=0x55e1df2feb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7fafbbfff640 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df2f8bd0 0x55e1df2feb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7fafbbfff640 1 -- 172.21.15.18:0/3757055650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df3017b0 msgr2=0x55e1df2ff040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:29.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7fafbbfff640 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df3017b0 0x55e1df2ff040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7fafbbfff640 1 -- 172.21.15.18:0/3757055650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafbc1a6a30 con 0x55e1df2f82b0 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7fafc0f29640 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df3017b0 0x55e1df2ff040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7fafbbfff640 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 0x55e1df2fe5c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fafac010a50 tx=0x7fafac010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401413b34be39311 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7faf9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7faf9b7fe640 1 -- 172.21.15.18:0/3757055650 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac015070 con 0x55e1df2f82b0 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7faf9b7fe640 1 -- 172.21.15.18:0/3757055650 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fafac00fbd0 con 0x55e1df2f82b0 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7fafc51fd740 1 -- 172.21.15.18:0/3757055650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1def5f7e0 con 0x55e1df2f82b0 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7faf9b7fe640 1 -- 172.21.15.18:0/3757055650 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafac00fd90 con 0x55e1df2f82b0 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7fafc51fd740 10 client.18945 fetch_fsmap learned FSMap version 47 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7fafc51fd740 1 -- 172.21.15.18:0/3757055650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55e1df25e610 con 0x55e1df2f82b0 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7faf9b7fe640 1 -- 172.21.15.18:0/3757055650 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e1def5f7e0 con 0x55e1df2f82b0 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.610+0000 7faf9b7fe640 1 -- 172.21.15.18:0/3757055650 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 47) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fafac00e570 con 0x55e1df2f82b0 2024-02-17T12:59:29.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.611+0000 7fafc51fd740 10 client.18945 fetch_fsmap finished waiting for FSMap version 47 2024-02-17T12:59:29.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.611+0000 7fafc51fd740 10 client.18945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:29.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.611+0000 7fafc51fd740 20 client.18945 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:29.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.611+0000 7fafc51fd740 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e1df3118f0 0x55e1df313d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:29.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.611+0000 7fafc51fd740 4 client.18945 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:59:29.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.611+0000 7fafc51fd740 1 -- 172.21.15.18:0/3757055650 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1def9c720 con 0x55e1df3118f0 2024-02-17T12:59:29.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.611+0000 7fafbb7fe640 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e1df3118f0 0x55e1df313d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:29.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.611+0000 7fafbb7fe640 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e1df3118f0 0x55e1df313d10 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118d87a7ff548238 server_cookie=8f534a518d1d8e9e in_seq=0 out_seq=0 2024-02-17T12:59:29.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:29.611+0000 7faf9b7fe640 10 client.18945 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:30.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 487356 5.12 MB/sec execute 405 sec latency 5034.004 ms 2024-02-17T12:59:30.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:30 smithi119 ceph-mon[36188]: pgmap v300: 97 pgs: 97 active+clean; 195 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 3.3 MiB/s wr, 179 op/s 2024-02-17T12:59:30.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:30 smithi146 ceph-mon[36454]: pgmap v300: 97 pgs: 97 active+clean; 195 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 3.3 MiB/s wr, 179 op/s 2024-02-17T12:59:30.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:30 smithi018 ceph-mon[32625]: pgmap v300: 97 pgs: 97 active+clean; 195 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 3.3 MiB/s wr, 179 op/s 2024-02-17T12:59:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd79e7fc640 1 -- 172.21.15.18:0/3143544568 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561b8f712550 con 0x561b8fadf710 2024-02-17T12:59:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd79e7fc640 10 client.18933 handle_command_reply: tid=0 2024-02-17T12:59:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd7bffa4740 1 client.18933 shutdown 2024-02-17T12:59:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3143544568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 msgr2=0x561b8fa7d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa7d220 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800f400 tx=0x7fd7a800aa40 comp rx=0 tx=0).stop 2024-02-17T12:59:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3143544568 shutdown_connections 2024-02-17T12:59:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x561b8fadf710 0x561b8fae1b30 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8fa772e0 0x561b8faddb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd79e7fc640 0 client.18933 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b8fa6ead0 0x561b8fa7d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd7bffa4740 1 --2- 172.21.15.18:0/3143544568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561b8fa6dd60 0x561b8fa7d220 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3143544568 >> 172.21.15.18:0/3143544568 conn(0x561b8fa35550 msgr2=0x561b8fa6a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:30.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7faf9b7fe640 1 -- 172.21.15.18:0/3757055650 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e1def9c720 con 0x55e1df3118f0 2024-02-17T12:59:30.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7faf9b7fe640 10 client.18945 handle_command_reply: tid=0 2024-02-17T12:59:30.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fafc51fd740 1 client.18945 shutdown 2024-02-17T12:59:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3143544568 shutdown_connections 2024-02-17T12:59:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd7bffa4740 1 -- 172.21.15.18:0/3143544568 wait complete. 2024-02-17T12:59:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fd7bffa4740 20 client.18933 trim_cache size 0 max 16384 2024-02-17T12:59:30.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fafc51fd740 1 -- 172.21.15.18:0/3757055650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 msgr2=0x55e1df2fe5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:30.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fafc51fd740 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 0x55e1df2fe5c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fafac010a50 tx=0x7fafac010f10 comp rx=0 tx=0).stop 2024-02-17T12:59:30.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fafc51fd740 1 -- 172.21.15.18:0/3757055650 shutdown_connections 2024-02-17T12:59:30.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fafc51fd740 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e1df3118f0 0x55e1df313d10 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fafc51fd740 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df3017b0 0x55e1df2ff040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fafc51fd740 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df2f8bd0 0x55e1df2feb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fafc51fd740 1 --2- 172.21.15.18:0/3757055650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 0x55e1df2fe5c0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:59:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:59:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:59:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:59:30.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7faf9b7fe640 0 client.18945 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:30.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fafc51fd740 1 -- 172.21.15.18:0/3757055650 >> 172.21.15.18:0/3757055650 conn(0x55e1df2f2990 msgr2=0x55e1df2f4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:30.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fafc51fd740 1 -- 172.21.15.18:0/3757055650 shutdown_connections 2024-02-17T12:59:30.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.863+0000 7fafc51fd740 1 -- 172.21.15.18:0/3757055650 wait complete. 2024-02-17T12:59:30.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fafc51fd740 20 client.18945 trim_cache size 0 max 16384 2024-02-17T12:59:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3301196678 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd7a40a2350 msgr2=0x7fd7a40a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fd7bd3d0640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd7a40a2350 0x7fd7a40a4810 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0005ce0 tx=0x7fd7a000bad0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3301196678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7b8199700 msgr2=0x7fd7b8199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fd7bd3d0640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7b8199700 0x7fd7b8199b80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00348f0 tx=0x7fd7b0034cc0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3301196678 shutdown_connections 2024-02-17T12:59:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fd7bd3d0640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd7a40a2350 0x7fd7a40a4810 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fd7bd3d0640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7b8199700 0x7fd7b8199b80 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fd7bd3d0640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7b8106490 0x7fd7b819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fd7bd3d0640 1 --2- 172.21.15.18:0/3301196678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7b81013c0 0x7fd7b819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3301196678 >> 172.21.15.18:0/3301196678 conn(0x7fd7b80fcc60 msgr2=0x7fd7b810c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3301196678 shutdown_connections 2024-02-17T12:59:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:30.864+0000 7fd7bd3d0640 1 -- 172.21.15.18:0/3301196678 wait complete. 2024-02-17T12:59:30.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafc51fd740 1 Processor -- start 2024-02-17T12:59:30.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafc51fd740 1 -- start start 2024-02-17T12:59:30.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafc51fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df2f82b0 0x55e1df2fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:30.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafc51fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f8bd0 0x55e1df2fee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:30.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafc51fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df3017b0 0x55e1df308410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:30.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafc51fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df3116a0 con 0x55e1df3017b0 2024-02-17T12:59:30.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafc51fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df2f3790 con 0x55e1df2f8bd0 2024-02-17T12:59:30.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafc51fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df30dc30 con 0x55e1df2f82b0 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafc0f29640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df3017b0 0x55e1df308410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f8bd0 0x55e1df2fee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafc0f29640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df3017b0 0x55e1df308410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55220/0 (socket says 172.21.15.18:55220) 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafc0f29640 1 -- 172.21.15.18:0/1408972668 learned_addr learned my addr 172.21.15.18:0/1408972668 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.882+0000 7fafbbfff640 1 --2- 172.21.15.18:0/1408972668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df2f82b0 0x55e1df2fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.883+0000 7fafc0f29640 1 -- 172.21.15.18:0/1408972668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df2f82b0 msgr2=0x55e1df2fd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.883+0000 7fafc0f29640 1 --2- 172.21.15.18:0/1408972668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df2f82b0 0x55e1df2fd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.883+0000 7fafc0f29640 1 -- 172.21.15.18:0/1408972668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f8bd0 msgr2=0x55e1df2fee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.883+0000 7fafc0f29640 1 --2- 172.21.15.18:0/1408972668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f8bd0 0x55e1df2fee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.883+0000 7fafc0f29640 1 -- 172.21.15.18:0/1408972668 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1df1faac0 con 0x55e1df3017b0 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.883+0000 7fafbb7fe640 1 --2- 172.21.15.18:0/1408972668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f8bd0 0x55e1df2fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.883+0000 7fafc0f29640 1 --2- 172.21.15.18:0/1408972668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df3017b0 0x55e1df308410 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fafb40137d0 tx=0x7fafb4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33da6e6fc3ab8c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:30.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.883+0000 7fafa1ffb640 1 -- 172.21.15.18:0/1408972668 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb4004b20 con 0x55e1df3017b0 2024-02-17T12:59:30.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.883+0000 7fafa1ffb640 1 -- 172.21.15.18:0/1408972668 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafb400fdc0 con 0x55e1df3017b0 2024-02-17T12:59:30.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.883+0000 7fafa1ffb640 1 -- 172.21.15.18:0/1408972668 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb4005790 con 0x55e1df3017b0 2024-02-17T12:59:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.884+0000 7fafc51fd740 1 -- 172.21.15.18:0/1408972668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df3017b0 msgr2=0x55e1df308410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.884+0000 7fafc51fd740 1 --2- 172.21.15.18:0/1408972668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df3017b0 0x55e1df308410 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fafb40137d0 tx=0x7fafb4015000 comp rx=0 tx=0).stop 2024-02-17T12:59:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.884+0000 7fafc51fd740 1 -- 172.21.15.18:0/1408972668 shutdown_connections 2024-02-17T12:59:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.884+0000 7fafc51fd740 1 --2- 172.21.15.18:0/1408972668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df3017b0 0x55e1df308410 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.884+0000 7fafc51fd740 1 --2- 172.21.15.18:0/1408972668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f8bd0 0x55e1df2fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.884+0000 7fafc51fd740 1 --2- 172.21.15.18:0/1408972668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df2f82b0 0x55e1df2fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.884+0000 7fafc51fd740 1 -- 172.21.15.18:0/1408972668 >> 172.21.15.18:0/1408972668 conn(0x55e1df2f2990 msgr2=0x55e1df2fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.884+0000 7fafc51fd740 1 -- 172.21.15.18:0/1408972668 shutdown_connections 2024-02-17T12:59:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.884+0000 7fafc51fd740 1 -- 172.21.15.18:0/1408972668 wait complete. 2024-02-17T12:59:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.885+0000 7fafc51fd740 1 -- start start 2024-02-17T12:59:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.885+0000 7fafc51fd740 1 Processor -- start 2024-02-17T12:59:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.885+0000 7fafc51fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 0x55e1df304dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.885+0000 7fafc51fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f8bd0 0x55e1df305310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.885+0000 7fafc51fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df3017b0 0x55e1df305850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.885+0000 7fafc51fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df367e10 con 0x55e1df2f82b0 2024-02-17T12:59:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.885+0000 7fafc51fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df367f50 con 0x55e1df2f8bd0 2024-02-17T12:59:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.885+0000 7fafc51fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df368090 con 0x55e1df3017b0 2024-02-17T12:59:30.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.885+0000 7fafc51fd740 10 client.0 _ll_register_callbacks cb 0x7fafb00038a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7fafbbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 0x55e1df304dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f8bd0 0x55e1df305310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7fafbbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 0x55e1df304dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55234/0 (socket says 172.21.15.18:55234) 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7fafbbfff640 1 -- 172.21.15.18:0/127074921 learned_addr learned my addr 172.21.15.18:0/127074921 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7fafc0f29640 1 --2- 172.21.15.18:0/127074921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df3017b0 0x55e1df305850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7fafbbfff640 1 -- 172.21.15.18:0/127074921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df3017b0 msgr2=0x55e1df305850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7fafbbfff640 1 --2- 172.21.15.18:0/127074921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df3017b0 0x55e1df305850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7fafbbfff640 1 -- 172.21.15.18:0/127074921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f8bd0 msgr2=0x55e1df305310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7fafbbfff640 1 --2- 172.21.15.18:0/127074921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f8bd0 0x55e1df305310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7fafbbfff640 1 -- 172.21.15.18:0/127074921 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1df21e570 con 0x55e1df2f82b0 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7fafbbfff640 1 --2- 172.21.15.18:0/127074921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 0x55e1df304dd0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fafac00e980 tx=0x7fafac00e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3359e5e1d8c95c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7faf9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7faf9b7fe640 1 -- 172.21.15.18:0/127074921 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac015070 con 0x55e1df2f82b0 2024-02-17T12:59:30.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7faf9b7fe640 1 -- 172.21.15.18:0/127074921 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fafac00ecf0 con 0x55e1df2f82b0 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.886+0000 7fafc51fd740 1 -- 172.21.15.18:0/127074921 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafb00c0e10 con 0x55e1df2f82b0 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.887+0000 7faf9b7fe640 1 -- 172.21.15.18:0/127074921 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafac00a2c0 con 0x55e1df2f82b0 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.887+0000 7fafc51fd740 10 client.18951 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.887+0000 7fafc51fd740 1 -- 172.21.15.18:0/127074921 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7fafb4013400 con 0x55e1df2f82b0 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.887+0000 7faf9b7fe640 1 -- 172.21.15.18:0/127074921 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafb00c0e10 con 0x55e1df2f82b0 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.887+0000 7faf9b7fe640 1 -- 172.21.15.18:0/127074921 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fafac029020 con 0x55e1df2f82b0 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.887+0000 7fafc51fd740 10 client.18951 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.887+0000 7fafc51fd740 10 client.18951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.887+0000 7fafc51fd740 20 client.18951 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.887+0000 7fafc51fd740 1 --2- 172.21.15.18:0/127074921 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e1df369800 0x55e1df36bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.887+0000 7fafc51fd740 4 client.18951 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.887+0000 7fafc51fd740 1 -- 172.21.15.18:0/127074921 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e1defeb890 con 0x55e1df369800 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.887+0000 7fafbb7fe640 1 --2- 172.21.15.18:0/127074921 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e1df369800 0x55e1df36bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7faf9b7fe640 10 client.18951 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:30.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafbb7fe640 1 --2- 172.21.15.18:0/127074921 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e1df369800 0x55e1df36bc20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd060e393d37fda server_cookie=629a4069ef8e4861 in_seq=0 out_seq=0 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7faf9b7fe640 1 -- 172.21.15.18:0/127074921 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e1defeb890 con 0x55e1df369800 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7faf9b7fe640 10 client.18951 handle_command_reply: tid=0 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafc51fd740 1 client.18951 shutdown 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafc51fd740 1 -- 172.21.15.18:0/127074921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 msgr2=0x55e1df304dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafc51fd740 1 --2- 172.21.15.18:0/127074921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 0x55e1df304dd0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fafac00e980 tx=0x7fafac00e5e0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafc51fd740 1 -- 172.21.15.18:0/127074921 shutdown_connections 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafc51fd740 1 --2- 172.21.15.18:0/127074921 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e1df369800 0x55e1df36bc20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafc51fd740 1 --2- 172.21.15.18:0/127074921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e1df3017b0 0x55e1df305850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafc51fd740 1 --2- 172.21.15.18:0/127074921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df2f8bd0 0x55e1df305310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7faf9b7fe640 0 client.18951 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafc51fd740 1 --2- 172.21.15.18:0/127074921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1df2f82b0 0x55e1df304dd0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafc51fd740 1 -- 172.21.15.18:0/127074921 >> 172.21.15.18:0/127074921 conn(0x55e1df2f2990 msgr2=0x55e1df2f45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafc51fd740 1 -- 172.21.15.18:0/127074921 shutdown_connections 2024-02-17T12:59:30.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafc51fd740 1 -- 172.21.15.18:0/127074921 wait complete. 2024-02-17T12:59:30.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.888+0000 7fafc51fd740 20 client.18951 trim_cache size 0 max 16384 2024-02-17T12:59:30.884 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:59:30.884 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T12:59:30.884 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T12:59:30.884 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:59:30.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.889+0000 7fafc298c640 1 -- 172.21.15.18:0/43115155 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fafa40a2260 msgr2=0x7fafa40a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:30.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.889+0000 7fafc298c640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fafa40a2260 0x7fafa40a4720 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fafb40092f0 tx=0x7fafb40097f0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.889+0000 7fafc298c640 1 -- 172.21.15.18:0/43115155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafbc105340 msgr2=0x7fafbc102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:30.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.889+0000 7fafc298c640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafbc105340 0x7fafbc102240 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009230 tx=0x7fafb0004270 comp rx=0 tx=0).stop 2024-02-17T12:59:30.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.889+0000 7fafc298c640 1 -- 172.21.15.18:0/43115155 shutdown_connections 2024-02-17T12:59:30.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.889+0000 7fafc298c640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fafa40a2260 0x7fafa40a4720 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.889+0000 7fafc298c640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc102780 0x7fafbc1a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.889+0000 7fafc298c640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafbc105340 0x7fafbc102240 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.889+0000 7fafc298c640 1 --2- 172.21.15.18:0/43115155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafbc100270 0x7fafbc101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:30.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.889+0000 7fafc298c640 1 -- 172.21.15.18:0/43115155 >> 172.21.15.18:0/43115155 conn(0x7fafbc077230 msgr2=0x7fafbc0686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:30.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.889+0000 7fafc298c640 1 -- 172.21.15.18:0/43115155 shutdown_connections 2024-02-17T12:59:30.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:30.889+0000 7fafc298c640 1 -- 172.21.15.18:0/43115155 wait complete. 2024-02-17T12:59:30.894 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:59:30.894 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:59:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.026+0000 7f31d0840640 1 -- 172.21.15.18:0/2763575675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc10b7c0 msgr2=0x7f31cc10dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.026+0000 7f31d0840640 1 --2- 172.21.15.18:0/2763575675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc10b7c0 0x7f31cc10dc60 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009ca0 tx=0x7f31c402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.026+0000 7f31d0840640 1 -- 172.21.15.18:0/2763575675 shutdown_connections 2024-02-17T12:59:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.026+0000 7f31d0840640 1 --2- 172.21.15.18:0/2763575675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc10b7c0 0x7f31cc10dc60 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.026+0000 7f31d0840640 1 --2- 172.21.15.18:0/2763575675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31cc102d10 0x7f31cc10b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.026+0000 7f31d0840640 1 --2- 172.21.15.18:0/2763575675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31cc1023f0 0x7f31cc1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.026+0000 7f31d0840640 1 -- 172.21.15.18:0/2763575675 >> 172.21.15.18:0/2763575675 conn(0x7f31cc077180 msgr2=0x7f31cc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.026+0000 7f31d0840640 1 -- 172.21.15.18:0/2763575675 shutdown_connections 2024-02-17T12:59:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.026+0000 7f31d0840640 1 -- 172.21.15.18:0/2763575675 wait complete. 2024-02-17T12:59:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.027+0000 7f31d0840640 1 Processor -- start 2024-02-17T12:59:31.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.027+0000 7f31d0840640 1 -- start start 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.027+0000 7f31d0840640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc1023f0 0x7f31cc19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.027+0000 7f31d0840640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31cc102d10 0x7f31cc19d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.027+0000 7f31d0840640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31cc10b7c0 0x7f31cc197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.027+0000 7f31d0840640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc197c70 con 0x7f31cc1023f0 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.027+0000 7f31d0840640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc197de0 con 0x7f31cc102d10 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.027+0000 7f31d0840640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc197fa0 con 0x7f31cc10b7c0 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31ca59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc1023f0 0x7f31cc19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31ca59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc1023f0 0x7f31cc19d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55242/0 (socket says 172.21.15.18:55242) 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31c1d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31cc102d10 0x7f31cc19d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31ca59c640 1 -- 172.21.15.18:0/540022487 learned_addr learned my addr 172.21.15.18:0/540022487 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31cad9d640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31cc10b7c0 0x7f31cc197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31ca59c640 1 -- 172.21.15.18:0/540022487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31cc10b7c0 msgr2=0x7f31cc197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31ca59c640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31cc10b7c0 0x7f31cc197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31ca59c640 1 -- 172.21.15.18:0/540022487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31cc102d10 msgr2=0x7f31cc19d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31ca59c640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31cc102d10 0x7f31cc19d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31ca59c640 1 -- 172.21.15.18:0/540022487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c4009900 con 0x7f31cc1023f0 2024-02-17T12:59:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31c1d9b640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31cc102d10 0x7f31cc19d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31ca59c640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc1023f0 0x7f31cc19d420 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00e9f0 tx=0x7f31bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31c3fff640 1 -- 172.21.15.18:0/540022487 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc009720 con 0x7f31cc1023f0 2024-02-17T12:59:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31d0840640 1 -- 172.21.15.18:0/540022487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31cc198280 con 0x7f31cc1023f0 2024-02-17T12:59:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31c3fff640 1 -- 172.21.15.18:0/540022487 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31bc00cd80 con 0x7f31cc1023f0 2024-02-17T12:59:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31c3fff640 1 -- 172.21.15.18:0/540022487 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc010690 con 0x7f31cc1023f0 2024-02-17T12:59:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.028+0000 7f31d0840640 1 -- 172.21.15.18:0/540022487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31cc1a44c0 con 0x7f31cc1023f0 2024-02-17T12:59:31.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.029+0000 7f31d0840640 1 -- 172.21.15.18:0/540022487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31cc067f70 con 0x7f31cc1023f0 2024-02-17T12:59:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.030+0000 7f31c3fff640 1 -- 172.21.15.18:0/540022487 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f31bc0046c0 con 0x7f31cc1023f0 2024-02-17T12:59:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.031+0000 7f31c3fff640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f31b80a2230 0x7f31b80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.031+0000 7f31c3fff640 1 -- 172.21.15.18:0/540022487 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f31bc014070 con 0x7f31cc1023f0 2024-02-17T12:59:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.031+0000 7f31c1d9b640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f31b80a2230 0x7f31b80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.032+0000 7f31c1d9b640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f31b80a2230 0x7f31b80a46f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0ffa50 tx=0x7f31b4005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.036+0000 7f31c3fff640 1 -- 172.21.15.18:0/540022487 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31bc097160 con 0x7f31cc1023f0 2024-02-17T12:59:31.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:31 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:59:31.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.284+0000 7f31d0840640 1 -- 172.21.15.18:0/540022487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31cc198b00 con 0x7f31cc1023f0 2024-02-17T12:59:31.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.285+0000 7f31c3fff640 1 -- 172.21.15.18:0/540022487 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+11396 (secure 0 0 0) 0x7f31bc09a820 con 0x7f31cc1023f0 2024-02-17T12:59:31.275 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T12:59:31.275 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:59:29.946156+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:59:31.275 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 48 2024-02-17T12:59:31.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.287+0000 7f31d0840640 1 -- 172.21.15.18:0/540022487 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f31b80a2230 msgr2=0x7f31b80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.287+0000 7f31d0840640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f31b80a2230 0x7f31b80a46f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0ffa50 tx=0x7f31b4005f90 comp rx=0 tx=0).stop 2024-02-17T12:59:31.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.287+0000 7f31d0840640 1 -- 172.21.15.18:0/540022487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc1023f0 msgr2=0x7f31cc19d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.287+0000 7f31d0840640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc1023f0 0x7f31cc19d420 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00e9f0 tx=0x7f31bc00edf0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.287+0000 7f31d0840640 1 -- 172.21.15.18:0/540022487 shutdown_connections 2024-02-17T12:59:31.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.287+0000 7f31d0840640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f31b80a2230 0x7f31b80a46f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.287+0000 7f31d0840640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31cc10b7c0 0x7f31cc197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.287+0000 7f31d0840640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31cc102d10 0x7f31cc19d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.287+0000 7f31d0840640 1 --2- 172.21.15.18:0/540022487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31cc1023f0 0x7f31cc19d420 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.287+0000 7f31d0840640 1 -- 172.21.15.18:0/540022487 >> 172.21.15.18:0/540022487 conn(0x7f31cc077180 msgr2=0x7f31cc0fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.287+0000 7f31d0840640 1 -- 172.21.15.18:0/540022487 shutdown_connections 2024-02-17T12:59:31.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.287+0000 7f31d0840640 1 -- 172.21.15.18:0/540022487 wait complete. 2024-02-17T12:59:31.288 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:59:31.288 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T12:59:31.291 INFO:tasks.workunit.client.0.smithi018.stdout: 1 488171 5.12 MB/sec execute 406 sec latency 5590.260 ms 2024-02-17T12:59:31.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:31 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:59:31.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.410+0000 7f94cb89d640 1 -- 172.21.15.18:0/945723363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94c4102ca0 msgr2=0x7f94c410b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.410+0000 7f94cb89d640 1 --2- 172.21.15.18:0/945723363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94c4102ca0 0x7f94c410b180 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009900 tx=0x7f94b402bcd0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.410+0000 7f94cb89d640 1 -- 172.21.15.18:0/945723363 shutdown_connections 2024-02-17T12:59:31.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.410+0000 7f94cb89d640 1 --2- 172.21.15.18:0/945723363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c410b750 0x7f94c410dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.410+0000 7f94cb89d640 1 --2- 172.21.15.18:0/945723363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94c4102ca0 0x7f94c410b180 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.410+0000 7f94cb89d640 1 --2- 172.21.15.18:0/945723363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94c4102380 0x7f94c4102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.410+0000 7f94cb89d640 1 -- 172.21.15.18:0/945723363 >> 172.21.15.18:0/945723363 conn(0x7f94c4076e30 msgr2=0x7f94c40fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.410+0000 7f94cb89d640 1 -- 172.21.15.18:0/945723363 shutdown_connections 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.410+0000 7f94cb89d640 1 -- 172.21.15.18:0/945723363 wait complete. 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94cb89d640 1 Processor -- start 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94cb89d640 1 -- start start 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94cb89d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c4102380 0x7f94c419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94cb89d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94c4102ca0 0x7f94c419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94cb89d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94c410b750 0x7f94c41975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94cb89d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4197c30 con 0x7f94c410b750 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94cb89d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4197da0 con 0x7f94c4102380 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94cb89d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4197f60 con 0x7f94c4102ca0 2024-02-17T12:59:31.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94c9e3a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94c410b750 0x7f94c41975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94c9e3a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94c410b750 0x7f94c41975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55266/0 (socket says 172.21.15.18:55266) 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94c8e38640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94c4102ca0 0x7f94c419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94c9e3a640 1 -- 172.21.15.18:0/2998148358 learned_addr learned my addr 172.21.15.18:0/2998148358 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94c9639640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c4102380 0x7f94c419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94c9e3a640 1 -- 172.21.15.18:0/2998148358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94c4102ca0 msgr2=0x7f94c419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94c9e3a640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94c4102ca0 0x7f94c419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94c9e3a640 1 -- 172.21.15.18:0/2998148358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c4102380 msgr2=0x7f94c419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94c9e3a640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c4102380 0x7f94c419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94c9e3a640 1 -- 172.21.15.18:0/2998148358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4009560 con 0x7f94c410b750 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.411+0000 7f94c8e38640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94c4102ca0 0x7f94c419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.412+0000 7f94c9e3a640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94c410b750 0x7f94c41975e0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00eaa0 tx=0x7f94bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.412+0000 7f94ba7fc640 1 -- 172.21.15.18:0/2998148358 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00cdb0 con 0x7f94c410b750 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.412+0000 7f94ba7fc640 1 -- 172.21.15.18:0/2998148358 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94bc005020 con 0x7f94c410b750 2024-02-17T12:59:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.412+0000 7f94cb89d640 1 -- 172.21.15.18:0/2998148358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c4198240 con 0x7f94c410b750 2024-02-17T12:59:31.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.412+0000 7f94ba7fc640 1 -- 172.21.15.18:0/2998148358 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0116b0 con 0x7f94c410b750 2024-02-17T12:59:31.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.412+0000 7f94cb89d640 1 -- 172.21.15.18:0/2998148358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c41a44d0 con 0x7f94c410b750 2024-02-17T12:59:31.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94ba7fc640 1 -- 172.21.15.18:0/2998148358 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f94bc005520 con 0x7f94c410b750 2024-02-17T12:59:31.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94ce47c740 1 Processor -- start 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94ce47c740 1 -- start start 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94ce47c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 0x55a672fc2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94ce47c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fcf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94ce47c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fcfb60 0x55a672fd1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94ce47c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a672f7d960 con 0x55a672fc1c30 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94ce47c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a672f31310 con 0x55a672fcfb60 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94ce47c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a672f2e840 con 0x55a672fc25e0 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94c8e38640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 0x55a672fc2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94c8e38640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 0x55a672fc2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55278/0 (socket says 172.21.15.18:55278) 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94c9639640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fcf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94c8e38640 1 -- 172.21.15.18:0/2512988611 learned_addr learned my addr 172.21.15.18:0/2512988611 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.415+0000 7f94ba7fc640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f94b00a2230 0x7f94b00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94c8e38640 1 --2- 172.21.15.18:0/2512988611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fcfb60 0x55a672fd1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94c9e3a640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f94b00a2230 0x7f94b00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94ba7fc640 1 -- 172.21.15.18:0/2998148358 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f94bc0ce240 con 0x7f94c410b750 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94c9639640 1 -- 172.21.15.18:0/2512988611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fcfb60 msgr2=0x55a672fd1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94c9639640 1 --2- 172.21.15.18:0/2512988611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fcfb60 0x55a672fd1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94c9639640 1 -- 172.21.15.18:0/2512988611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 msgr2=0x55a672fc2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94c9639640 1 --2- 172.21.15.18:0/2512988611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 0x55a672fc2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94c9639640 1 -- 172.21.15.18:0/2512988611 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a672eec060 con 0x55a672fc25e0 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94c9639640 1 --2- 172.21.15.18:0/2512988611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fcf620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009c20 tx=0x7f94c0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f428b7b56d5717 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94c9e3a640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f94b00a2230 0x7f94b00a46f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00cfd0 tx=0x7f94bc06ab00 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94af7fe640 1 -- 172.21.15.18:0/2512988611 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0004050 con 0x55a672fc25e0 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94af7fe640 1 -- 172.21.15.18:0/2512988611 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94c0005500 con 0x55a672fc25e0 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.416+0000 7f94af7fe640 1 -- 172.21.15.18:0/2512988611 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0004d30 con 0x55a672fc25e0 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.417+0000 7f94ce47c740 1 -- 172.21.15.18:0/2512988611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 msgr2=0x55a672fcf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.417+0000 7f94ce47c740 1 --2- 172.21.15.18:0/2512988611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fcf620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009c20 tx=0x7f94c0011810 comp rx=0 tx=0).stop 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.417+0000 7f94ce47c740 1 -- 172.21.15.18:0/2512988611 shutdown_connections 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.417+0000 7f94ce47c740 1 --2- 172.21.15.18:0/2512988611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fcfb60 0x55a672fd1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.417+0000 7f94ce47c740 1 --2- 172.21.15.18:0/2512988611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fcf620 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.417+0000 7f94ce47c740 1 --2- 172.21.15.18:0/2512988611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 0x55a672fc2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.417+0000 7f94ce47c740 1 -- 172.21.15.18:0/2512988611 >> 172.21.15.18:0/2512988611 conn(0x55a672fbc980 msgr2=0x55a672fbcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.417+0000 7f94ce47c740 1 -- 172.21.15.18:0/2512988611 shutdown_connections 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.417+0000 7f94ce47c740 1 -- 172.21.15.18:0/2512988611 wait complete. 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.417+0000 7f94ce47c740 1 -- start start 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.417+0000 7f94ce47c740 1 Processor -- start 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94ce47c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fc1c30 0x55a672fccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94ce47c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fcd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94ce47c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 0x55a672fcd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94ce47c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a672f2a020 con 0x55a672fcfb60 2024-02-17T12:59:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94ce47c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a672bfd430 con 0x55a672fc1c30 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94ce47c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a672bb9e00 con 0x55a672fc25e0 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94ce47c740 10 client.0 _ll_register_callbacks cb 0x55a672ed1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94c8e38640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fc1c30 0x55a672fccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94c9639640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fcd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94c8e38640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 0x55a672fcd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94c9639640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fcd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:52634/0 (socket says 172.21.15.18:52634) 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94c9639640 1 -- 172.21.15.18:0/1092273560 learned_addr learned my addr 172.21.15.18:0/1092273560 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94c8e38640 1 -- 172.21.15.18:0/1092273560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 msgr2=0x55a672fcd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94c8e38640 1 --2- 172.21.15.18:0/1092273560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fcd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94c8e38640 1 -- 172.21.15.18:0/1092273560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fc1c30 msgr2=0x55a672fccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94c8e38640 1 --2- 172.21.15.18:0/1092273560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fc1c30 0x55a672fccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.418+0000 7f94c8e38640 1 -- 172.21.15.18:0/1092273560 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c0009850 con 0x55a672fcfb60 2024-02-17T12:59:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94c8e38640 1 --2- 172.21.15.18:0/1092273560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 0x55a672fcd590 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006010 tx=0x7f94b4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a483944062e85070 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94acff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94acff9640 1 -- 172.21.15.18:0/1092273560 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4048030 con 0x55a672fcfb60 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94acff9640 1 -- 172.21.15.18:0/1092273560 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f94b40050c0 con 0x55a672fcfb60 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94ce47c740 1 -- 172.21.15.18:0/1092273560 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a672c297e0 con 0x55a672fcfb60 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94acff9640 1 -- 172.21.15.18:0/1092273560 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94b4005280 con 0x55a672fcfb60 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94ce47c740 10 client.18969 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94ce47c740 1 -- 172.21.15.18:0/1092273560 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a672f286a0 con 0x55a672fcfb60 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94acff9640 1 -- 172.21.15.18:0/1092273560 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a672c297e0 con 0x55a672fcfb60 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94acff9640 1 -- 172.21.15.18:0/1092273560 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f94b403c040 con 0x55a672fcfb60 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94ce47c740 10 client.18969 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94ce47c740 10 client.18969 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94ce47c740 20 client.18969 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94ce47c740 1 --2- 172.21.15.18:0/1092273560 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55a672fdfe00 0x55a672fe21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94ce47c740 4 client.18969 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.419+0000 7f94ce47c740 1 -- 172.21.15.18:0/1092273560 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a672c66720 con 0x55a672fdfe00 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.420+0000 7f94c9639640 1 --2- 172.21.15.18:0/1092273560 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55a672fdfe00 0x55a672fe21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.420+0000 7f94c9639640 1 --2- 172.21.15.18:0/1092273560 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55a672fdfe00 0x55a672fe21f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=90478845cf6b95c6 server_cookie=96bb0b1448bf1795 in_seq=0 out_seq=0 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.420+0000 7f94acff9640 10 client.18969 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.420+0000 7f94acff9640 1 -- 172.21.15.18:0/1092273560 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a672c66720 con 0x55a672fdfe00 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.420+0000 7f94acff9640 10 client.18969 handle_command_reply: tid=0 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94ce47c740 1 client.18969 shutdown 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94ce47c740 1 -- 172.21.15.18:0/1092273560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 msgr2=0x55a672fcd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94ce47c740 1 --2- 172.21.15.18:0/1092273560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 0x55a672fcd590 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006010 tx=0x7f94b4005b50 comp rx=0 tx=0).stop 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94ce47c740 1 -- 172.21.15.18:0/1092273560 shutdown_connections 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94ce47c740 1 --2- 172.21.15.18:0/1092273560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 0x55a672fcd590 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94ce47c740 1 --2- 172.21.15.18:0/1092273560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fcd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94ce47c740 1 --2- 172.21.15.18:0/1092273560 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55a672fdfe00 0x55a672fe21f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94ce47c740 1 --2- 172.21.15.18:0/1092273560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fc1c30 0x55a672fccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94acff9640 0 client.18969 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:59:31.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94ce47c740 1 -- 172.21.15.18:0/1092273560 >> 172.21.15.18:0/1092273560 conn(0x55a672fbc980 msgr2=0x55a672fbe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94ce47c740 1 -- 172.21.15.18:0/1092273560 shutdown_connections 2024-02-17T12:59:31.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94ce47c740 1 -- 172.21.15.18:0/1092273560 wait complete. 2024-02-17T12:59:31.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.421+0000 7f94ce47c740 20 client.18969 trim_cache size 0 max 16384 2024-02-17T12:59:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.435+0000 7f94ce47c740 1 Processor -- start 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94ce47c740 1 -- start start 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94ce47c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 0x55a672fcf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94ce47c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fc9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94ce47c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fcfb60 0x55a672fca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94ce47c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a672eaf130 con 0x55a672fc1c30 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94ce47c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a672c66720 con 0x55a672fcfb60 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94ce47c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a672c297e0 con 0x55a672fc25e0 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94c8e38640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 0x55a672fcf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94c8e38640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 0x55a672fcf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55298/0 (socket says 172.21.15.18:55298) 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94c8e38640 1 -- 172.21.15.18:0/968277503 learned_addr learned my addr 172.21.15.18:0/968277503 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94c9639640 1 --2- 172.21.15.18:0/968277503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fc9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94c8e38640 1 --2- 172.21.15.18:0/968277503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fcfb60 0x55a672fca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94c8e38640 1 -- 172.21.15.18:0/968277503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 msgr2=0x55a672fc9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94c8e38640 1 --2- 172.21.15.18:0/968277503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fc9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94c8e38640 1 -- 172.21.15.18:0/968277503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fcfb60 msgr2=0x55a672fca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94c8e38640 1 --2- 172.21.15.18:0/968277503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fcfb60 0x55a672fca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94c8e38640 1 -- 172.21.15.18:0/968277503 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a672ec4c20 con 0x55a672fc1c30 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.436+0000 7f94c9639640 1 --2- 172.21.15.18:0/968277503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fc9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.437+0000 7f94c8e38640 1 --2- 172.21.15.18:0/968277503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 0x55a672fcf240 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f94b4005c40 tx=0x7f94b4005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ca736545945ef9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.437+0000 7f94aeffd640 1 -- 172.21.15.18:0/968277503 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4048040 con 0x55a672fc1c30 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.437+0000 7f94aeffd640 1 -- 172.21.15.18:0/968277503 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94b4049040 con 0x55a672fc1c30 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.437+0000 7f94aeffd640 1 -- 172.21.15.18:0/968277503 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4003ab0 con 0x55a672fc1c30 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.438+0000 7f94ce47c740 1 -- 172.21.15.18:0/968277503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 msgr2=0x55a672fcf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.438+0000 7f94ce47c740 1 --2- 172.21.15.18:0/968277503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 0x55a672fcf240 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f94b4005c40 tx=0x7f94b4005e40 comp rx=0 tx=0).stop 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.438+0000 7f94ce47c740 1 -- 172.21.15.18:0/968277503 shutdown_connections 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.438+0000 7f94ce47c740 1 --2- 172.21.15.18:0/968277503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fcfb60 0x55a672fca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.438+0000 7f94ce47c740 1 --2- 172.21.15.18:0/968277503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc25e0 0x55a672fc9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.438+0000 7f94ce47c740 1 --2- 172.21.15.18:0/968277503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fc1c30 0x55a672fcf240 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.438+0000 7f94ce47c740 1 -- 172.21.15.18:0/968277503 >> 172.21.15.18:0/968277503 conn(0x55a672fbc980 msgr2=0x55a672fcb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.438+0000 7f94ce47c740 1 -- 172.21.15.18:0/968277503 shutdown_connections 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.438+0000 7f94ce47c740 1 -- 172.21.15.18:0/968277503 wait complete. 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.438+0000 7f94ce47c740 1 -- start start 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.438+0000 7f94ce47c740 1 Processor -- start 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94ce47c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc1c30 0x55a672fd6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94ce47c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fc25e0 0x55a672fd5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94ce47c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 0x55a672fd5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94ce47c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a672fd5f30 con 0x55a672fcfb60 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94ce47c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a672fd60a0 con 0x55a672fc25e0 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94ce47c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a673036300 con 0x55a672fc1c30 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94ce47c740 10 client.0 _ll_register_callbacks cb 0x7f94c00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94c8e38640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 0x55a672fd5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94c9639640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fc25e0 0x55a672fd5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94c8e38640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 0x55a672fd5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55312/0 (socket says 172.21.15.18:55312) 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94c8e38640 1 -- 172.21.15.18:0/1412673180 learned_addr learned my addr 172.21.15.18:0/1412673180 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94c8e38640 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc1c30 0x55a672fd6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94c9639640 1 -- 172.21.15.18:0/1412673180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc1c30 msgr2=0x55a672fd6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94c9639640 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc1c30 0x55a672fd6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94c9639640 1 -- 172.21.15.18:0/1412673180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 msgr2=0x55a672fd5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94c9639640 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 0x55a672fd5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94c9639640 1 -- 172.21.15.18:0/1412673180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c001d040 con 0x55a672fc25e0 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.439+0000 7f94c8e38640 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 0x55a672fd5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94c9639640 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fc25e0 0x55a672fd5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f94c0011d10 tx=0x7f94c0009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d584d65a3219b12b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94acff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94acff9640 1 -- 172.21.15.18:0/1412673180 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c00041a0 con 0x55a672fc25e0 2024-02-17T12:59:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94ce47c740 1 -- 172.21.15.18:0/1412673180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94c0005500 con 0x55a672fc25e0 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94acff9640 1 -- 172.21.15.18:0/1412673180 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f94c0004300 con 0x55a672fc25e0 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94acff9640 1 -- 172.21.15.18:0/1412673180 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94c00044e0 con 0x55a672fc25e0 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94ce47c740 10 client.28040 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94ce47c740 1 -- 172.21.15.18:0/1412673180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a672f286a0 con 0x55a672fc25e0 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94acff9640 1 -- 172.21.15.18:0/1412673180 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94c0005500 con 0x55a672fc25e0 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94acff9640 1 -- 172.21.15.18:0/1412673180 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f94c002f020 con 0x55a672fc25e0 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94ce47c740 10 client.28040 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94ce47c740 10 client.28040 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94ce47c740 20 client.28040 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94ce47c740 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55a673037b50 0x55a673039f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94ce47c740 4 client.28040 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.440+0000 7f94ce47c740 1 -- 172.21.15.18:0/1412673180 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a672cb5890 con 0x55a673037b50 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.441+0000 7f94c8e38640 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55a673037b50 0x55a673039f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.441+0000 7f94acff9640 10 client.28040 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T12:59:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.441+0000 7f94c8e38640 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55a673037b50 0x55a673039f70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7140191e89639aba server_cookie=900d1a9ad869611c in_seq=0 out_seq=0 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.441+0000 7f94acff9640 1 -- 172.21.15.18:0/1412673180 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a672cb5890 con 0x55a673037b50 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.441+0000 7f94acff9640 10 client.28040 handle_command_reply: tid=0 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.441+0000 7f94ce47c740 1 client.28040 shutdown 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94ce47c740 1 -- 172.21.15.18:0/1412673180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fc25e0 msgr2=0x55a672fd5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94ce47c740 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fc25e0 0x55a672fd5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f94c0011d10 tx=0x7f94c0009320 comp rx=0 tx=0).stop 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94ce47c740 1 -- 172.21.15.18:0/1412673180 shutdown_connections 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94ce47c740 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a672fcfb60 0x55a672fd5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94ce47c740 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55a673037b50 0x55a673039f70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94ce47c740 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a672fc25e0 0x55a672fd5230 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94ce47c740 1 --2- 172.21.15.18:0/1412673180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a672fc1c30 0x55a672fd6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94acff9640 0 client.28040 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94ce47c740 1 -- 172.21.15.18:0/1412673180 >> 172.21.15.18:0/1412673180 conn(0x55a672fbc980 msgr2=0x55a672fbdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94ce47c740 1 -- 172.21.15.18:0/1412673180 shutdown_connections 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94ce47c740 1 -- 172.21.15.18:0/1412673180 wait complete. 2024-02-17T12:59:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94ce47c740 20 client.28040 trim_cache size 0 max 16384 2024-02-17T12:59:31.440 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:59:31.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94cb89d640 1 -- 172.21.15.18:0/2998148358 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f94b00a2230 msgr2=0x7f94b00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94cb89d640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f94b00a2230 0x7f94b00a46f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00cfd0 tx=0x7f94bc06ab00 comp rx=0 tx=0).stop 2024-02-17T12:59:31.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94cb89d640 1 -- 172.21.15.18:0/2998148358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94c410b750 msgr2=0x7f94c41975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94cb89d640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94c410b750 0x7f94c41975e0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00eaa0 tx=0x7f94bc00ef60 comp rx=0 tx=0).stop 2024-02-17T12:59:31.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94cb89d640 1 -- 172.21.15.18:0/2998148358 shutdown_connections 2024-02-17T12:59:31.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94cb89d640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f94b00a2230 0x7f94b00a46f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94cb89d640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94c410b750 0x7f94c41975e0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94cb89d640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94c4102ca0 0x7f94c419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94cb89d640 1 --2- 172.21.15.18:0/2998148358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94c4102380 0x7f94c419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.442+0000 7f94cb89d640 1 -- 172.21.15.18:0/2998148358 >> 172.21.15.18:0/2998148358 conn(0x7f94c4076e30 msgr2=0x7f94c40fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.443+0000 7f94cb89d640 1 -- 172.21.15.18:0/2998148358 shutdown_connections 2024-02-17T12:59:31.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.443+0000 7f94cb89d640 1 -- 172.21.15.18:0/2998148358 wait complete. 2024-02-17T12:59:31.443 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:59:31.443 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T12:59:31.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:31 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T12:59:31.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.599+0000 7f1cf47ac640 1 -- 172.21.15.18:0/3853385301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec1053e0 msgr2=0x7f1cec10cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.599+0000 7f1cf47ac640 1 --2- 172.21.15.18:0/3853385301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec1053e0 0x7f1cec10cfb0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009cd0 tx=0x7f1ce402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.599+0000 7f1cf47ac640 1 -- 172.21.15.18:0/3853385301 shutdown_connections 2024-02-17T12:59:31.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.599+0000 7f1cf47ac640 1 --2- 172.21.15.18:0/3853385301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec1053e0 0x7f1cec10cfb0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.599+0000 7f1cf47ac640 1 --2- 172.21.15.18:0/3853385301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cec100cc0 0x7f1cec104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.599+0000 7f1cf47ac640 1 --2- 172.21.15.18:0/3853385301 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cec100310 0x7f1cec1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.599+0000 7f1cf47ac640 1 -- 172.21.15.18:0/3853385301 >> 172.21.15.18:0/3853385301 conn(0x7f1cec0771e0 msgr2=0x7f1cec0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.599+0000 7f1cf47ac640 1 -- 172.21.15.18:0/3853385301 shutdown_connections 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.599+0000 7f1cf47ac640 1 -- 172.21.15.18:0/3853385301 wait complete. 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf47ac640 1 Processor -- start 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf47ac640 1 -- start start 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf47ac640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec100310 0x7f1cec19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf47ac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cec1053e0 0x7f1cec19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf47ac640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cec196590 0x7f1cec196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf47ac640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec196f80 con 0x7f1cec100310 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf47ac640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec1970f0 con 0x7f1cec1053e0 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf47ac640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec1972b0 con 0x7f1cec196590 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf2548640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec100310 0x7f1cec19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf2548640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec100310 0x7f1cec19c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55340/0 (socket says 172.21.15.18:55340) 2024-02-17T12:59:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf2548640 1 -- 172.21.15.18:0/794736809 learned_addr learned my addr 172.21.15.18:0/794736809 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf1d47640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cec1053e0 0x7f1cec19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf2d49640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cec196590 0x7f1cec196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf2548640 1 -- 172.21.15.18:0/794736809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cec196590 msgr2=0x7f1cec196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf2548640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cec196590 0x7f1cec196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf2548640 1 -- 172.21.15.18:0/794736809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cec1053e0 msgr2=0x7f1cec19c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf2548640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cec1053e0 0x7f1cec19c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf2548640 1 -- 172.21.15.18:0/794736809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce4009900 con 0x7f1cec100310 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.600+0000 7f1cf2548640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec100310 0x7f1cec19c3e0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00e9f0 tx=0x7f1cdc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.601+0000 7f1ce37fe640 1 -- 172.21.15.18:0/794736809 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc009720 con 0x7f1cec100310 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.601+0000 7f1ce37fe640 1 -- 172.21.15.18:0/794736809 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cdc00cd80 con 0x7f1cec100310 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.601+0000 7f1ce37fe640 1 -- 172.21.15.18:0/794736809 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc010690 con 0x7f1cec100310 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.601+0000 7f1cf47ac640 1 -- 172.21.15.18:0/794736809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cec069770 con 0x7f1cec100310 2024-02-17T12:59:31.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.601+0000 7f1cf47ac640 1 -- 172.21.15.18:0/794736809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cec069c80 con 0x7f1cec100310 2024-02-17T12:59:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.603+0000 7f1ce37fe640 1 -- 172.21.15.18:0/794736809 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1cdc0046c0 con 0x7f1cec100310 2024-02-17T12:59:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.603+0000 7f1cf737c740 1 Processor -- start 2024-02-17T12:59:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.603+0000 7f1cf737c740 1 -- start start 2024-02-17T12:59:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf737c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdbc250 0x56160bdbc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf737c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbcb70 0x56160bdc5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf737c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 0x56160bdc7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf737c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160bd77960 con 0x56160bdc5750 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf737c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160bd2b310 con 0x56160bdbcb70 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf737c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160bd28840 con 0x56160bdbc250 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1ce37fe640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1cd80a22b0 0x7f1cd80a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf1d47640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 0x56160bdc7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf1d47640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 0x56160bdc7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55356/0 (socket says 172.21.15.18:55356) 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf1d47640 1 -- 172.21.15.18:0/2668322935 learned_addr learned my addr 172.21.15.18:0/2668322935 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf2548640 1 --2- 172.21.15.18:0/2668322935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbcb70 0x56160bdc5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf2d49640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1cd80a22b0 0x7f1cd80a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf2d49640 1 --2- 172.21.15.18:0/2668322935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdbc250 0x56160bdbc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf1d47640 1 -- 172.21.15.18:0/2668322935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdbc250 msgr2=0x56160bdbc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1ce37fe640 1 -- 172.21.15.18:0/794736809 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1cdc014070 con 0x7f1cec100310 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf1d47640 1 --2- 172.21.15.18:0/2668322935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdbc250 0x56160bdbc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf1d47640 1 -- 172.21.15.18:0/2668322935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbcb70 msgr2=0x56160bdc5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf1d47640 1 --2- 172.21.15.18:0/2668322935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbcb70 0x56160bdc5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf1d47640 1 -- 172.21.15.18:0/2668322935 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56160bce6060 con 0x56160bdc5750 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf1d47640 1 --2- 172.21.15.18:0/2668322935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 0x56160bdc7b40 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009ba0 tx=0x7f1ce8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9a11dd19b108ca server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.604+0000 7f1cf2d49640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1cd80a22b0 0x7f1cd80a4770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4005cc0 tx=0x7f1ce4009820 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.605+0000 7f1ce0ff9640 1 -- 172.21.15.18:0/2668322935 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8013070 con 0x56160bdc5750 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.605+0000 7f1ce0ff9640 1 -- 172.21.15.18:0/2668322935 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8014040 con 0x56160bdc5750 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.605+0000 7f1ce0ff9640 1 -- 172.21.15.18:0/2668322935 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8018660 con 0x56160bdc5750 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.605+0000 7f1cf737c740 1 -- 172.21.15.18:0/2668322935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 msgr2=0x56160bdc7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.605+0000 7f1cf737c740 1 --2- 172.21.15.18:0/2668322935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 0x56160bdc7b40 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009ba0 tx=0x7f1ce8005f90 comp rx=0 tx=0).stop 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.605+0000 7f1cf737c740 1 -- 172.21.15.18:0/2668322935 shutdown_connections 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.605+0000 7f1cf737c740 1 --2- 172.21.15.18:0/2668322935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 0x56160bdc7b40 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.605+0000 7f1cf737c740 1 --2- 172.21.15.18:0/2668322935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbcb70 0x56160bdc5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.605+0000 7f1cf737c740 1 --2- 172.21.15.18:0/2668322935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdbc250 0x56160bdbc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.605+0000 7f1cf737c740 1 -- 172.21.15.18:0/2668322935 >> 172.21.15.18:0/2668322935 conn(0x56160bdb6980 msgr2=0x56160bdb6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.605+0000 7f1cf737c740 1 -- 172.21.15.18:0/2668322935 shutdown_connections 2024-02-17T12:59:31.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.605+0000 7f1cf737c740 1 -- 172.21.15.18:0/2668322935 wait complete. 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf737c740 1 -- start start 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf737c740 1 Processor -- start 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf737c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbc250 0x56160bdc2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf737c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbcb70 0x56160bdc2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf737c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdc5750 0x56160bdc3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf737c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160bd24020 con 0x56160bdbc250 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf737c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b9f7430 con 0x56160bdbcb70 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf737c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b9b3e00 con 0x56160bdc5750 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf737c740 10 client.0 _ll_register_callbacks cb 0x56160bccbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf2548640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbc250 0x56160bdc2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf2548640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbc250 0x56160bdc2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55362/0 (socket says 172.21.15.18:55362) 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf2548640 1 -- 172.21.15.18:0/1877138056 learned_addr learned my addr 172.21.15.18:0/1877138056 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf2d49640 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdc5750 0x56160bdc3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf1d47640 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbcb70 0x56160bdc2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf2548640 1 -- 172.21.15.18:0/1877138056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdc5750 msgr2=0x56160bdc3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf2548640 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdc5750 0x56160bdc3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf2548640 1 -- 172.21.15.18:0/1877138056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbcb70 msgr2=0x56160bdc2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf2548640 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbcb70 0x56160bdc2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.606+0000 7f1cf2548640 1 -- 172.21.15.18:0/1877138056 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce8009850 con 0x56160bdbc250 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cf2548640 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbc250 0x56160bdc2780 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc09aab0 tx=0x7f1cdc01da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3050f888e696a776 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cd5ffb640 1 -- 172.21.15.18:0/1877138056 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc00ace0 con 0x56160bdbc250 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cd5ffb640 1 -- 172.21.15.18:0/1877138056 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1cdc09adb0 con 0x56160bdbc250 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cf737c740 1 -- 172.21.15.18:0/1877138056 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56160ba237e0 con 0x56160bdbc250 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cd5ffb640 1 -- 172.21.15.18:0/1877138056 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cdc09ef00 con 0x56160bdbc250 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cf737c740 10 client.18987 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cf737c740 1 -- 172.21.15.18:0/1877138056 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x56160bd226a0 con 0x56160bdbc250 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cd5ffb640 1 -- 172.21.15.18:0/1877138056 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56160ba237e0 con 0x56160bdbc250 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cd5ffb640 1 -- 172.21.15.18:0/1877138056 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1cdc01f060 con 0x56160bdbc250 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cf737c740 10 client.18987 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cf737c740 10 client.18987 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cf737c740 20 client.18987 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cf737c740 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56160bdd5b20 0x56160bdd7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cf737c740 4 client.18987 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.607+0000 7f1cf737c740 1 -- 172.21.15.18:0/1877138056 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56160ba60720 con 0x56160bdd5b20 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.608+0000 7f1cf1d47640 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56160bdd5b20 0x56160bdd7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.608+0000 7f1cf1d47640 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56160bdd5b20 0x56160bdd7f30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=895e629b01c02a41 server_cookie=f0e0097ec5a284ed in_seq=0 out_seq=0 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.608+0000 7f1cd5ffb640 10 client.18987 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.608+0000 7f1cd5ffb640 1 -- 172.21.15.18:0/1877138056 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56160ba60720 con 0x56160bdd5b20 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.608+0000 7f1cd5ffb640 10 client.18987 handle_command_reply: tid=0 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cf737c740 1 client.18987 shutdown 2024-02-17T12:59:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cf737c740 1 -- 172.21.15.18:0/1877138056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbc250 msgr2=0x56160bdc2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cf737c740 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbc250 0x56160bdc2780 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc09aab0 tx=0x7f1cdc01da30 comp rx=0 tx=0).stop 2024-02-17T12:59:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cf737c740 1 -- 172.21.15.18:0/1877138056 shutdown_connections 2024-02-17T12:59:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cf737c740 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56160bdd5b20 0x56160bdd7f30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cf737c740 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdc5750 0x56160bdc3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cf737c740 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbcb70 0x56160bdc2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cd5ffb640 0 client.18987 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:59:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cf737c740 1 --2- 172.21.15.18:0/1877138056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbc250 0x56160bdc2780 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cf737c740 1 -- 172.21.15.18:0/1877138056 >> 172.21.15.18:0/1877138056 conn(0x56160bdb6980 msgr2=0x56160bdb86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cf737c740 1 -- 172.21.15.18:0/1877138056 shutdown_connections 2024-02-17T12:59:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cf737c740 1 -- 172.21.15.18:0/1877138056 wait complete. 2024-02-17T12:59:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.609+0000 7f1cf737c740 20 client.18987 trim_cache size 0 max 16384 2024-02-17T12:59:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.620+0000 7f1cf737c740 1 Processor -- start 2024-02-17T12:59:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf737c740 1 -- start start 2024-02-17T12:59:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf737c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbc250 0x56160bdc4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf737c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbcb70 0x56160bdc3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf737c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdc5750 0x56160bdbf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf737c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160bca9130 con 0x56160bdbcb70 2024-02-17T12:59:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf737c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160bdbbeb0 con 0x56160bdbc250 2024-02-17T12:59:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf737c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160bddd140 con 0x56160bdc5750 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf1d47640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbcb70 0x56160bdc3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf1d47640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbcb70 0x56160bdc3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55376/0 (socket says 172.21.15.18:55376) 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf1d47640 1 -- 172.21.15.18:0/64346807 learned_addr learned my addr 172.21.15.18:0/64346807 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf1d47640 1 -- 172.21.15.18:0/64346807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdc5750 msgr2=0x56160bdbf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf1d47640 1 --2- 172.21.15.18:0/64346807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdc5750 0x56160bdbf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf1d47640 1 -- 172.21.15.18:0/64346807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbc250 msgr2=0x56160bdc4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf1d47640 1 --2- 172.21.15.18:0/64346807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbc250 0x56160bdc4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf1d47640 1 -- 172.21.15.18:0/64346807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56160bcbec20 con 0x56160bdbcb70 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cf1d47640 1 --2- 172.21.15.18:0/64346807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbcb70 0x56160bdc3020 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009250 tx=0x7f1ce800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12723943c890fed server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cd7fff640 1 -- 172.21.15.18:0/64346807 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8027040 con 0x56160bdbcb70 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cd7fff640 1 -- 172.21.15.18:0/64346807 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8009e10 con 0x56160bdbcb70 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.621+0000 7f1cd7fff640 1 -- 172.21.15.18:0/64346807 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce802ba90 con 0x56160bdbcb70 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 -- 172.21.15.18:0/64346807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbcb70 msgr2=0x56160bdc3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 --2- 172.21.15.18:0/64346807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbcb70 0x56160bdc3020 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009250 tx=0x7f1ce800edf0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 -- 172.21.15.18:0/64346807 shutdown_connections 2024-02-17T12:59:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 --2- 172.21.15.18:0/64346807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdc5750 0x56160bdbf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 --2- 172.21.15.18:0/64346807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdbcb70 0x56160bdc3020 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 --2- 172.21.15.18:0/64346807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbc250 0x56160bdc4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 -- 172.21.15.18:0/64346807 >> 172.21.15.18:0/64346807 conn(0x56160bdb6980 msgr2=0x56160bdbda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 -- 172.21.15.18:0/64346807 shutdown_connections 2024-02-17T12:59:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 -- 172.21.15.18:0/64346807 wait complete. 2024-02-17T12:59:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 -- start start 2024-02-17T12:59:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 Processor -- start 2024-02-17T12:59:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbc250 0x56160bdcb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdbcb70 0x56160bdcb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.622+0000 7f1cf737c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 0x56160bdcbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf737c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160be2c020 con 0x56160bdc5750 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf737c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160be2c160 con 0x56160bdbc250 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf737c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160be2c2d0 con 0x56160bdbcb70 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf737c740 10 client.0 _ll_register_callbacks cb 0x7f1ce80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf2d49640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 0x56160bdcbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf2d49640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 0x56160bdcbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55386/0 (socket says 172.21.15.18:55386) 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf2d49640 1 -- 172.21.15.18:0/3017387639 learned_addr learned my addr 172.21.15.18:0/3017387639 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf2548640 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbc250 0x56160bdcb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf1d47640 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdbcb70 0x56160bdcb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf2d49640 1 -- 172.21.15.18:0/3017387639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdbcb70 msgr2=0x56160bdcb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf2d49640 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdbcb70 0x56160bdcb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf2d49640 1 -- 172.21.15.18:0/3017387639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbc250 msgr2=0x56160bdcb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf2d49640 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbc250 0x56160bdcb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf2d49640 1 -- 172.21.15.18:0/3017387639 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56160bce6060 con 0x56160bdc5750 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf1d47640 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdbcb70 0x56160bdcb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf2d49640 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 0x56160bdcbbf0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009cd0 tx=0x7f1ce4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1516b574376b6876 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cd5ffb640 1 -- 172.21.15.18:0/3017387639 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40328c0 con 0x56160bdc5750 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cd5ffb640 1 -- 172.21.15.18:0/3017387639 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1ce4032a20 con 0x56160bdc5750 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf737c740 1 -- 172.21.15.18:0/3017387639 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ce8014040 con 0x56160bdc5750 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cd5ffb640 1 -- 172.21.15.18:0/3017387639 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ce4003970 con 0x56160bdc5750 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf737c740 10 client.18993 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.623+0000 7f1cf737c740 1 -- 172.21.15.18:0/3017387639 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x56160bce48c0 con 0x56160bdc5750 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cd5ffb640 1 -- 172.21.15.18:0/3017387639 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8014040 con 0x56160bdc5750 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cd5ffb640 1 -- 172.21.15.18:0/3017387639 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1ce404e470 con 0x56160bdc5750 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cf737c740 10 client.18993 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:31.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cf737c740 10 client.18993 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cf737c740 20 client.18993 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cf737c740 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56160be2db00 0x56160be2ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cf737c740 4 client.18993 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cf737c740 1 -- 172.21.15.18:0/3017387639 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56160baaf890 con 0x56160be2db00 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cf2548640 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56160be2db00 0x56160be2ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cd5ffb640 10 client.18993 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cf2548640 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56160be2db00 0x56160be2ff20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=19524e8a6ffbef29 server_cookie=a8064403469feb68 in_seq=0 out_seq=0 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cd5ffb640 1 -- 172.21.15.18:0/3017387639 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56160baaf890 con 0x56160be2db00 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cd5ffb640 10 client.18993 handle_command_reply: tid=0 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.624+0000 7f1cf737c740 1 client.18993 shutdown 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf737c740 1 -- 172.21.15.18:0/3017387639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 msgr2=0x56160bdcbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf737c740 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 0x56160bdcbbf0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009cd0 tx=0x7f1ce4005ef0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf737c740 1 -- 172.21.15.18:0/3017387639 shutdown_connections 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf737c740 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56160be2db00 0x56160be2ff20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf737c740 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56160bdc5750 0x56160bdcbbf0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf737c740 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56160bdbcb70 0x56160bdcb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cd5ffb640 0 client.18993 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T12:59:31.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf737c740 1 --2- 172.21.15.18:0/3017387639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56160bdbc250 0x56160bdcb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf737c740 1 -- 172.21.15.18:0/3017387639 >> 172.21.15.18:0/3017387639 conn(0x56160bdb6980 msgr2=0x56160bdb52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf737c740 1 -- 172.21.15.18:0/3017387639 shutdown_connections 2024-02-17T12:59:31.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf737c740 1 -- 172.21.15.18:0/3017387639 wait complete. 2024-02-17T12:59:31.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf737c740 20 client.18993 trim_cache size 0 max 16384 2024-02-17T12:59:31.622 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:59:31.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf47ac640 1 -- 172.21.15.18:0/794736809 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1cd80a22b0 msgr2=0x7f1cd80a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf47ac640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1cd80a22b0 0x7f1cd80a4770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4005cc0 tx=0x7f1ce4009820 comp rx=0 tx=0).stop 2024-02-17T12:59:31.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf47ac640 1 -- 172.21.15.18:0/794736809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec100310 msgr2=0x7f1cec19c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf47ac640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec100310 0x7f1cec19c3e0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00e9f0 tx=0x7f1cdc00edf0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf47ac640 1 -- 172.21.15.18:0/794736809 shutdown_connections 2024-02-17T12:59:31.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf47ac640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1cd80a22b0 0x7f1cd80a4770 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf47ac640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cec196590 0x7f1cec196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf47ac640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cec1053e0 0x7f1cec19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf47ac640 1 --2- 172.21.15.18:0/794736809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec100310 0x7f1cec19c3e0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf47ac640 1 -- 172.21.15.18:0/794736809 >> 172.21.15.18:0/794736809 conn(0x7f1cec0771e0 msgr2=0x7f1cec0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf47ac640 1 -- 172.21.15.18:0/794736809 shutdown_connections 2024-02-17T12:59:31.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.625+0000 7f1cf47ac640 1 -- 172.21.15.18:0/794736809 wait complete. 2024-02-17T12:59:31.626 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:59:31.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:59:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.740+0000 7f8cd929d640 1 -- 172.21.15.18:0/1183086075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cd4100260 msgr2=0x7f8cd4100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.740+0000 7f8cd929d640 1 --2- 172.21.15.18:0/1183086075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cd4100260 0x7f8cd4100640 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009930 tx=0x7f8ccc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:59:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.740+0000 7f8cd929d640 1 -- 172.21.15.18:0/1183086075 shutdown_connections 2024-02-17T12:59:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.740+0000 7f8cd929d640 1 --2- 172.21.15.18:0/1183086075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cd4105330 0x7f8cd410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.740+0000 7f8cd929d640 1 --2- 172.21.15.18:0/1183086075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cd4100c10 0x7f8cd4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.740+0000 7f8cd929d640 1 --2- 172.21.15.18:0/1183086075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cd4100260 0x7f8cd4100640 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.740+0000 7f8cd929d640 1 -- 172.21.15.18:0/1183086075 >> 172.21.15.18:0/1183086075 conn(0x7f8cd4077180 msgr2=0x7f8cd4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.740+0000 7f8cd929d640 1 -- 172.21.15.18:0/1183086075 shutdown_connections 2024-02-17T12:59:31.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.740+0000 7f8cd929d640 1 -- 172.21.15.18:0/1183086075 wait complete. 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.741+0000 7f8cd929d640 1 Processor -- start 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.741+0000 7f8cd929d640 1 -- start start 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.741+0000 7f8cd929d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cd4100260 0x7f8cd4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.741+0000 7f8cd929d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cd4105330 0x7f8cd4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.741+0000 7f8cd929d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cd41990f0 0x7f8cd41a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.741+0000 7f8cd929d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41a3d70 con 0x7f8cd4105330 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.741+0000 7f8cd929d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41a3ee0 con 0x7f8cd4100260 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.741+0000 7f8cd929d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41a40a0 con 0x7f8cd41990f0 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.741+0000 7f8cd27fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cd4105330 0x7f8cd4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.741+0000 7f8cd27fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cd4105330 0x7f8cd4198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55404/0 (socket says 172.21.15.18:55404) 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cd27fc640 1 -- 172.21.15.18:0/802346280 learned_addr learned my addr 172.21.15.18:0/802346280 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cd2ffd640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cd4100260 0x7f8cd4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cd37fe640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cd41990f0 0x7f8cd41a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cd27fc640 1 -- 172.21.15.18:0/802346280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cd41990f0 msgr2=0x7f8cd41a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cd41990f0 0x7f8cd41a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cd27fc640 1 -- 172.21.15.18:0/802346280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cd4100260 msgr2=0x7f8cd4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cd4100260 0x7f8cd4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cd27fc640 1 -- 172.21.15.18:0/802346280 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc009560 con 0x7f8cd4105330 2024-02-17T12:59:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cd4105330 0x7f8cd4198b60 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800d770 tx=0x7f8cc800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cbbfff640 1 -- 172.21.15.18:0/802346280 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8009c30 con 0x7f8cd4105330 2024-02-17T12:59:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cd929d640 1 -- 172.21.15.18:0/802346280 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd41a4380 con 0x7f8cd4105330 2024-02-17T12:59:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cbbfff640 1 -- 172.21.15.18:0/802346280 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cc800de80 con 0x7f8cd4105330 2024-02-17T12:59:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cbbfff640 1 -- 172.21.15.18:0/802346280 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800f6b0 con 0x7f8cd4105330 2024-02-17T12:59:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.742+0000 7f8cd929d640 1 -- 172.21.15.18:0/802346280 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd41a4890 con 0x7f8cd4105330 2024-02-17T12:59:31.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.744+0000 7f8cbbfff640 1 -- 172.21.15.18:0/802346280 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8cc8010460 con 0x7f8cd4105330 2024-02-17T12:59:31.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cdbe72740 1 Processor -- start 2024-02-17T12:59:31.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cdbe72740 1 -- start start 2024-02-17T12:59:31.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cbbfff640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8cbc0a2230 0x7f8cbc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cdbe72740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 0x55a8bc0fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cdbe72740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc107620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cdbe72740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 0x55a8bc109f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cdbe72740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bc0b5960 con 0x55a8bc0f9f00 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cdbe72740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bc069310 con 0x55a8bc0fa820 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cdbe72740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bc066840 con 0x55a8bc107b60 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cbbfff640 1 -- 172.21.15.18:0/802346280 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8cc80cdeb0 con 0x7f8cd4105330 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd37fe640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8cbc0a2230 0x7f8cbc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 0x55a8bc0fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 0x55a8bc0fa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55416/0 (socket says 172.21.15.18:55416) 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd2ffd640 1 -- 172.21.15.18:0/2088163734 learned_addr learned my addr 172.21.15.18:0/2088163734 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/2088163734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc107620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd2ffd640 1 -- 172.21.15.18:0/2088163734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 msgr2=0x55a8bc109f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd2ffd640 1 --2- 172.21.15.18:0/2088163734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 0x55a8bc109f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd2ffd640 1 -- 172.21.15.18:0/2088163734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 msgr2=0x55a8bc107620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd2ffd640 1 --2- 172.21.15.18:0/2088163734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc107620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd2ffd640 1 -- 172.21.15.18:0/2088163734 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8bc024060 con 0x55a8bc0f9f00 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/2088163734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd37fe640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8cbc0a2230 0x7f8cbc0a46f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005fd0 tx=0x7f8cc40107a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.745+0000 7f8cd2ffd640 1 --2- 172.21.15.18:0/2088163734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 0x55a8bc0fa2e0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc036040 tx=0x7f8ccc02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d164c42118251b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.746+0000 7f8cb97fa640 1 -- 172.21.15.18:0/2088163734 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc004800 con 0x55a8bc0f9f00 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.746+0000 7f8cb97fa640 1 -- 172.21.15.18:0/2088163734 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ccc004960 con 0x55a8bc0f9f00 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.746+0000 7f8cb97fa640 1 -- 172.21.15.18:0/2088163734 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc005080 con 0x55a8bc0f9f00 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.746+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2088163734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 msgr2=0x55a8bc0fa2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.746+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/2088163734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 0x55a8bc0fa2e0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc036040 tx=0x7f8ccc02c680 comp rx=0 tx=0).stop 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.746+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2088163734 shutdown_connections 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.746+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/2088163734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 0x55a8bc109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.746+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/2088163734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.746+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/2088163734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 0x55a8bc0fa2e0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.746+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2088163734 >> 172.21.15.18:0/2088163734 conn(0x55a8bc0f4980 msgr2=0x55a8bc0f4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.746+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2088163734 shutdown_connections 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.746+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2088163734 wait complete. 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cdbe72740 1 -- start start 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cdbe72740 1 Processor -- start 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cdbe72740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0f9f00 0x55a8bc104a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cdbe72740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0fa820 0x55a8bc104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cdbe72740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 0x55a8bc105500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cdbe72740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bc062020 con 0x55a8bc0fa820 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cdbe72740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bbd35430 con 0x55a8bc0f9f00 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cdbe72740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bbcf1e00 con 0x55a8bc107b60 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cdbe72740 10 client.0 _ll_register_callbacks cb 0x55a8bc009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cd27fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0fa820 0x55a8bc104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cd27fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0fa820 0x55a8bc104fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55418/0 (socket says 172.21.15.18:55418) 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cd27fc640 1 -- 172.21.15.18:0/1483855007 learned_addr learned my addr 172.21.15.18:0/1483855007 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cd37fe640 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 0x55a8bc105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cd2ffd640 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0f9f00 0x55a8bc104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cd27fc640 1 -- 172.21.15.18:0/1483855007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 msgr2=0x55a8bc105500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 0x55a8bc105500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cd27fc640 1 -- 172.21.15.18:0/1483855007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0f9f00 msgr2=0x55a8bc104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0f9f00 0x55a8bc104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cd27fc640 1 -- 172.21.15.18:0/1483855007 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd410d710 con 0x55a8bc0fa820 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.747+0000 7f8cd37fe640 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 0x55a8bc105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0fa820 0x55a8bc104fc0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8096cc0 tx=0x7f8cc809a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594785a9e8140351 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cb6ff5640 1 -- 172.21.15.18:0/1483855007 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80a3000 con 0x55a8bc0fa820 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cb6ff5640 1 -- 172.21.15.18:0/1483855007 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8cc805d070 con 0x55a8bc0fa820 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cdbe72740 1 -- 172.21.15.18:0/1483855007 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8bbd617e0 con 0x55a8bc0fa820 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cb6ff5640 1 -- 172.21.15.18:0/1483855007 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cc805d230 con 0x55a8bc0fa820 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cdbe72740 10 client.19005 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:31.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cdbe72740 1 -- 172.21.15.18:0/1483855007 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a8bc0606a0 con 0x55a8bc0fa820 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cb6ff5640 1 -- 172.21.15.18:0/1483855007 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a8bbd617e0 con 0x55a8bc0fa820 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cb6ff5640 1 -- 172.21.15.18:0/1483855007 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8cc80cb840 con 0x55a8bc0fa820 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cdbe72740 10 client.19005 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cdbe72740 10 client.19005 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cdbe72740 20 client.19005 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a8bc117de0 0x55a8bc11a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cdbe72740 4 client.19005 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cdbe72740 1 -- 172.21.15.18:0/1483855007 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8bbd9e720 con 0x55a8bc117de0 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.748+0000 7f8cd2ffd640 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a8bc117de0 0x55a8bc11a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.749+0000 7f8cb6ff5640 10 client.19005 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.749+0000 7f8cd2ffd640 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a8bc117de0 0x55a8bc11a1d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c2d2de6e38d2a63 server_cookie=dd0f47acb81bb240 in_seq=0 out_seq=0 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.749+0000 7f8cb6ff5640 1 -- 172.21.15.18:0/1483855007 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a8bbd9e720 con 0x55a8bc117de0 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.749+0000 7f8cb6ff5640 10 client.19005 handle_command_reply: tid=0 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.749+0000 7f8cdbe72740 1 client.19005 shutdown 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.750+0000 7f8cdbe72740 1 -- 172.21.15.18:0/1483855007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0fa820 msgr2=0x55a8bc104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.750+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0fa820 0x55a8bc104fc0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8096cc0 tx=0x7f8cc809a2b0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.750+0000 7f8cdbe72740 1 -- 172.21.15.18:0/1483855007 shutdown_connections 2024-02-17T12:59:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.750+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a8bc117de0 0x55a8bc11a1d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.750+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 0x55a8bc105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.750+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0fa820 0x55a8bc104fc0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.750+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/1483855007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0f9f00 0x55a8bc104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.750+0000 7f8cb6ff5640 0 client.19005 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:59:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.750+0000 7f8cdbe72740 1 -- 172.21.15.18:0/1483855007 >> 172.21.15.18:0/1483855007 conn(0x55a8bc0f4980 msgr2=0x55a8bc0f66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.750+0000 7f8cdbe72740 1 -- 172.21.15.18:0/1483855007 shutdown_connections 2024-02-17T12:59:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.750+0000 7f8cdbe72740 1 -- 172.21.15.18:0/1483855007 wait complete. 2024-02-17T12:59:31.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.750+0000 7f8cdbe72740 20 client.19005 trim_cache size 0 max 16384 2024-02-17T12:59:31.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.761+0000 7f8cdbe72740 1 Processor -- start 2024-02-17T12:59:31.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.761+0000 7f8cdbe72740 1 -- start start 2024-02-17T12:59:31.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.761+0000 7f8cdbe72740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc0f9f00 0x55a8bc103be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.761+0000 7f8cdbe72740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc105320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.761+0000 7f8cdbe72740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc107b60 0x55a8bc0f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.761+0000 7f8cdbe72740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bbfe7130 con 0x55a8bc107b60 2024-02-17T12:59:31.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.761+0000 7f8cdbe72740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bc0f05c0 con 0x55a8bc0fa820 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cdbe72740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bc11f3d0 con 0x55a8bc0f9f00 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc107b60 0x55a8bc0f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc107b60 0x55a8bc0f64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55428/0 (socket says 172.21.15.18:55428) 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cd37fe640 1 -- 172.21.15.18:0/4023591799 learned_addr learned my addr 172.21.15.18:0/4023591799 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cd2ffd640 1 --2- 172.21.15.18:0/4023591799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc0f9f00 0x55a8bc103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/4023591799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc105320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cd37fe640 1 -- 172.21.15.18:0/4023591799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc0f9f00 msgr2=0x55a8bc103be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cd37fe640 1 --2- 172.21.15.18:0/4023591799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc0f9f00 0x55a8bc103be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cd37fe640 1 -- 172.21.15.18:0/4023591799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 msgr2=0x55a8bc105320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cd37fe640 1 --2- 172.21.15.18:0/4023591799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc105320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cd37fe640 1 -- 172.21.15.18:0/4023591799 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8bbffcc20 con 0x55a8bc107b60 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cd37fe640 1 --2- 172.21.15.18:0/4023591799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc107b60 0x55a8bc0f64e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40117c0 tx=0x7f8cc4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5bf47a36b32ee1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cb8ff9640 1 -- 172.21.15.18:0/4023591799 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400e940 con 0x55a8bc107b60 2024-02-17T12:59:31.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cb8ff9640 1 -- 172.21.15.18:0/4023591799 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cc4021430 con 0x55a8bc107b60 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.762+0000 7f8cb8ff9640 1 -- 172.21.15.18:0/4023591799 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40205b0 con 0x55a8bc107b60 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 -- 172.21.15.18:0/4023591799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc107b60 msgr2=0x55a8bc0f64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/4023591799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc107b60 0x55a8bc0f64e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40117c0 tx=0x7f8cc4015000 comp rx=0 tx=0).stop 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 -- 172.21.15.18:0/4023591799 shutdown_connections 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/4023591799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc107b60 0x55a8bc0f64e0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/4023591799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/4023591799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc0f9f00 0x55a8bc103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 -- 172.21.15.18:0/4023591799 >> 172.21.15.18:0/4023591799 conn(0x55a8bc0f4980 msgr2=0x55a8bc0f5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 -- 172.21.15.18:0/4023591799 shutdown_connections 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 -- 172.21.15.18:0/4023591799 wait complete. 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 -- start start 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 Processor -- start 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 0x55a8bc10d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc10dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 0x55a8bc10e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bc16e450 con 0x55a8bc0f9f00 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bc16e5c0 con 0x55a8bc0fa820 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.763+0000 7f8cdbe72740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bc16e780 con 0x55a8bc107b60 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cdbe72740 10 client.0 _ll_register_callbacks cb 0x7f8ccc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 0x55a8bc10d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd27fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc10dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 0x55a8bc10d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55438/0 (socket says 172.21.15.18:55438) 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd2ffd640 1 -- 172.21.15.18:0/2953758793 learned_addr learned my addr 172.21.15.18:0/2953758793 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc10dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:52028/0 (socket says 172.21.15.18:52028) 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd37fe640 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 0x55a8bc10e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd2ffd640 1 -- 172.21.15.18:0/2953758793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 msgr2=0x55a8bc10e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd2ffd640 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 0x55a8bc10e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd2ffd640 1 -- 172.21.15.18:0/2953758793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 msgr2=0x55a8bc10dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd2ffd640 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc10dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd2ffd640 1 -- 172.21.15.18:0/2953758793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc4011440 con 0x55a8bc0f9f00 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cd2ffd640 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 0x55a8bc10d6e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0045e0 tx=0x7f8ccc004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f317869d4feeba server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cb6ff5640 1 -- 172.21.15.18:0/2953758793 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc04e040 con 0x55a8bc0f9f00 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cb6ff5640 1 -- 172.21.15.18:0/2953758793 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8ccc050dd0 con 0x55a8bc0f9f00 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2953758793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ccc004960 con 0x55a8bc0f9f00 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cb6ff5640 1 -- 172.21.15.18:0/2953758793 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ccc02c8e0 con 0x55a8bc0f9f00 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cdbe72740 10 client.19011 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2953758793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a8bc0228c0 con 0x55a8bc0f9f00 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.764+0000 7f8cb6ff5640 1 -- 172.21.15.18:0/2953758793 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ccc004960 con 0x55a8bc0f9f00 2024-02-17T12:59:31.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cb6ff5640 1 -- 172.21.15.18:0/2953758793 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8ccc03c050 con 0x55a8bc0f9f00 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cdbe72740 10 client.19011 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cdbe72740 10 client.19011 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cdbe72740 20 client.19011 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a8bc16fb30 0x55a8bc171f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cdbe72740 4 client.19011 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2953758793 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a8bbded890 con 0x55a8bc16fb30 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a8bc16fb30 0x55a8bc171f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cb6ff5640 10 client.19011 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cd27fc640 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a8bc16fb30 0x55a8bc171f20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed49538a1cf5896c server_cookie=a4daa13c07b705a2 in_seq=0 out_seq=0 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cb6ff5640 1 -- 172.21.15.18:0/2953758793 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a8bbded890 con 0x55a8bc16fb30 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cb6ff5640 10 client.19011 handle_command_reply: tid=0 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cdbe72740 1 client.19011 shutdown 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2953758793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 msgr2=0x55a8bc10d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 0x55a8bc10d6e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0045e0 tx=0x7f8ccc004570 comp rx=0 tx=0).stop 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2953758793 shutdown_connections 2024-02-17T12:59:31.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55a8bc16fb30 0x55a8bc171f20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.765+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8bc107b60 0x55a8bc10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8bc0fa820 0x55a8bc10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cb6ff5640 0 client.19011 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T12:59:31.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cdbe72740 1 --2- 172.21.15.18:0/2953758793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8bc0f9f00 0x55a8bc10d6e0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2953758793 >> 172.21.15.18:0/2953758793 conn(0x55a8bc0f4980 msgr2=0x55a8bc0f3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2953758793 shutdown_connections 2024-02-17T12:59:31.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cdbe72740 1 -- 172.21.15.18:0/2953758793 wait complete. 2024-02-17T12:59:31.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cdbe72740 20 client.19011 trim_cache size 0 max 16384 2024-02-17T12:59:31.763 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:59:31.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cd929d640 1 -- 172.21.15.18:0/802346280 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8cbc0a2230 msgr2=0x7f8cbc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cd929d640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8cbc0a2230 0x7f8cbc0a46f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005fd0 tx=0x7f8cc40107a0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cd929d640 1 -- 172.21.15.18:0/802346280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cd4105330 msgr2=0x7f8cd4198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cd929d640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cd4105330 0x7f8cd4198b60 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800d770 tx=0x7f8cc800dc30 comp rx=0 tx=0).stop 2024-02-17T12:59:31.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cd929d640 1 -- 172.21.15.18:0/802346280 shutdown_connections 2024-02-17T12:59:31.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cd929d640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8cbc0a2230 0x7f8cbc0a46f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cd929d640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8cd41990f0 0x7f8cd41a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cd929d640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cd4105330 0x7f8cd4198b60 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cd929d640 1 --2- 172.21.15.18:0/802346280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cd4100260 0x7f8cd4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cd929d640 1 -- 172.21.15.18:0/802346280 >> 172.21.15.18:0/802346280 conn(0x7f8cd4077180 msgr2=0x7f8cd40fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cd929d640 1 -- 172.21.15.18:0/802346280 shutdown_connections 2024-02-17T12:59:31.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.766+0000 7f8cd929d640 1 -- 172.21.15.18:0/802346280 wait complete. 2024-02-17T12:59:31.767 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:59:31.767 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:59:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.881+0000 7fd0901d0640 1 -- 172.21.15.18:0/3802708615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd088100260 msgr2=0x7fd088100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.881+0000 7fd0901d0640 1 --2- 172.21.15.18:0/3802708615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd088100260 0x7fd088100640 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd084009930 tx=0x7fd08402bc30 comp rx=0 tx=0).stop 2024-02-17T12:59:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 -- 172.21.15.18:0/3802708615 shutdown_connections 2024-02-17T12:59:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 --2- 172.21.15.18:0/3802708615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd088105330 0x7fd08810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 --2- 172.21.15.18:0/3802708615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd088100c10 0x7fd088104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 --2- 172.21.15.18:0/3802708615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd088100260 0x7fd088100640 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 -- 172.21.15.18:0/3802708615 >> 172.21.15.18:0/3802708615 conn(0x7fd088077180 msgr2=0x7fd088077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 -- 172.21.15.18:0/3802708615 shutdown_connections 2024-02-17T12:59:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 -- 172.21.15.18:0/3802708615 wait complete. 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 Processor -- start 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 -- start start 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd088100260 0x7fd0881966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd088105330 0x7fd088196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd08819afa0 0x7fd08819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08819b990 con 0x7fd088100260 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08819bb00 con 0x7fd08819afa0 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.882+0000 7fd0901d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08819bc70 con 0x7fd088105330 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08df6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd088100260 0x7fd0881966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08df6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd088100260 0x7fd0881966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55462/0 (socket says 172.21.15.18:55462) 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08df6c640 1 -- 172.21.15.18:0/2933327912 learned_addr learned my addr 172.21.15.18:0/2933327912 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08e76d640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd08819afa0 0x7fd08819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08df6c640 1 -- 172.21.15.18:0/2933327912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd088105330 msgr2=0x7fd088196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08d76b640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd088105330 0x7fd088196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08df6c640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd088105330 0x7fd088196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08df6c640 1 -- 172.21.15.18:0/2933327912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd08819afa0 msgr2=0x7fd08819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08df6c640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd08819afa0 0x7fd08819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08df6c640 1 -- 172.21.15.18:0/2933327912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd084009560 con 0x7fd088100260 2024-02-17T12:59:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08e76d640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd08819afa0 0x7fd08819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08d76b640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd088105330 0x7fd088196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:59:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd08df6c640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd088100260 0x7fd0881966e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd084009230 tx=0x7fd08402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd07effd640 1 -- 172.21.15.18:0/2933327912 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084004350 con 0x7fd088100260 2024-02-17T12:59:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd07effd640 1 -- 172.21.15.18:0/2933327912 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0840044b0 con 0x7fd088100260 2024-02-17T12:59:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd0901d0640 1 -- 172.21.15.18:0/2933327912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0881a3250 con 0x7fd088100260 2024-02-17T12:59:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd07effd640 1 -- 172.21.15.18:0/2933327912 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084003d70 con 0x7fd088100260 2024-02-17T12:59:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.883+0000 7fd0901d0640 1 -- 172.21.15.18:0/2933327912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0881a36d0 con 0x7fd088100260 2024-02-17T12:59:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.886+0000 7fd07effd640 1 -- 172.21.15.18:0/2933327912 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd084004df0 con 0x7fd088100260 2024-02-17T12:59:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.886+0000 7fd092da4740 1 Processor -- start 2024-02-17T12:59:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.886+0000 7fd092da4740 1 -- start start 2024-02-17T12:59:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.886+0000 7fd092da4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56068079d250 0x56068079d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.886+0000 7fd092da4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x5606807a6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd092da4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606807a6750 0x5606807a8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd092da4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560680758960 con 0x5606807a6750 2024-02-17T12:59:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd092da4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068070c310 con 0x56068079db70 2024-02-17T12:59:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd092da4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560680709840 con 0x56068079d250 2024-02-17T12:59:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd07effd640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd0740a2230 0x7fd0740a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08d76b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606807a6750 0x5606807a8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08d76b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606807a6750 0x5606807a8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55472/0 (socket says 172.21.15.18:55472) 2024-02-17T12:59:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08d76b640 1 -- 172.21.15.18:0/3745188215 learned_addr learned my addr 172.21.15.18:0/3745188215 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08e76d640 1 --2- 172.21.15.18:0/3745188215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56068079d250 0x56068079d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08df6c640 1 --2- 172.21.15.18:0/3745188215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x5606807a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08e76d640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd0740a2230 0x7fd0740a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd07effd640 1 -- 172.21.15.18:0/2933327912 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd0840f01f0 con 0x7fd088100260 2024-02-17T12:59:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08d76b640 1 -- 172.21.15.18:0/3745188215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56068079d250 msgr2=0x56068079d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08d76b640 1 --2- 172.21.15.18:0/3745188215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56068079d250 0x56068079d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08d76b640 1 -- 172.21.15.18:0/3745188215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 msgr2=0x5606807a6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08d76b640 1 --2- 172.21.15.18:0/3745188215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x5606807a6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08d76b640 1 -- 172.21.15.18:0/3745188215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606806c7060 con 0x5606807a6750 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08df6c640 1 --2- 172.21.15.18:0/3745188215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x5606807a6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08d76b640 1 --2- 172.21.15.18:0/3745188215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606807a6750 0x5606807a8b40 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd078009bf0 tx=0x7fd07800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200a31457a29311b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd08e76d640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd0740a2230 0x7fd0740a46f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd080009fd0 tx=0x7fd080009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd073fff640 1 -- 172.21.15.18:0/3745188215 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078005020 con 0x5606807a6750 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd073fff640 1 -- 172.21.15.18:0/3745188215 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd078005660 con 0x5606807a6750 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.887+0000 7fd073fff640 1 -- 172.21.15.18:0/3745188215 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0780116d0 con 0x5606807a6750 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.888+0000 7fd092da4740 1 -- 172.21.15.18:0/3745188215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606807a6750 msgr2=0x5606807a8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.888+0000 7fd092da4740 1 --2- 172.21.15.18:0/3745188215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606807a6750 0x5606807a8b40 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd078009bf0 tx=0x7fd07800e6c0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.888+0000 7fd092da4740 1 -- 172.21.15.18:0/3745188215 shutdown_connections 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.888+0000 7fd092da4740 1 --2- 172.21.15.18:0/3745188215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606807a6750 0x5606807a8b40 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.888+0000 7fd092da4740 1 --2- 172.21.15.18:0/3745188215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x5606807a6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.888+0000 7fd092da4740 1 --2- 172.21.15.18:0/3745188215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56068079d250 0x56068079d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.888+0000 7fd092da4740 1 -- 172.21.15.18:0/3745188215 >> 172.21.15.18:0/3745188215 conn(0x560680797980 msgr2=0x560680797d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.888+0000 7fd092da4740 1 -- 172.21.15.18:0/3745188215 shutdown_connections 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.888+0000 7fd092da4740 1 -- 172.21.15.18:0/3745188215 wait complete. 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd092da4740 1 -- start start 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd092da4740 1 Processor -- start 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd092da4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56068079d250 0x5606807a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd092da4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079db70 0x5606807a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd092da4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606807a6750 0x5606807a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd092da4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560680705020 con 0x56068079db70 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd092da4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606803d8430 con 0x5606807a6750 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd092da4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560680394e00 con 0x56068079d250 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd092da4740 10 client.0 _ll_register_callbacks cb 0x5606806accb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd08d76b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079db70 0x5606807a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd08d76b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079db70 0x5606807a3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55478/0 (socket says 172.21.15.18:55478) 2024-02-17T12:59:31.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd08d76b640 1 -- 172.21.15.18:0/3407796967 learned_addr learned my addr 172.21.15.18:0/3407796967 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd08df6c640 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56068079d250 0x5606807a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd08e76d640 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606807a6750 0x5606807a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd08d76b640 1 -- 172.21.15.18:0/3407796967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56068079d250 msgr2=0x5606807a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd08d76b640 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56068079d250 0x5606807a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd08d76b640 1 -- 172.21.15.18:0/3407796967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606807a6750 msgr2=0x5606807a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd08d76b640 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606807a6750 0x5606807a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd08d76b640 1 -- 172.21.15.18:0/3407796967 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd078009820 con 0x56068079db70 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.889+0000 7fd08df6c640 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56068079d250 0x5606807a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd08d76b640 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079db70 0x5606807a3c70 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd07800a010 tx=0x7fd0780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c940cb0654c5843 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd0717fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd0717fa640 1 -- 172.21.15.18:0/3407796967 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078021070 con 0x56068079db70 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd0717fa640 1 -- 172.21.15.18:0/3407796967 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd0780050d0 con 0x56068079db70 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd092da4740 1 -- 172.21.15.18:0/3407796967 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606804047e0 con 0x56068079db70 2024-02-17T12:59:31.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd0717fa640 1 -- 172.21.15.18:0/3407796967 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd078005290 con 0x56068079db70 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd092da4740 10 client.19023 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd092da4740 1 -- 172.21.15.18:0/3407796967 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5606807036a0 con 0x56068079db70 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd0717fa640 1 -- 172.21.15.18:0/3407796967 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5606804047e0 con 0x56068079db70 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd0717fa640 1 -- 172.21.15.18:0/3407796967 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd07800c070 con 0x56068079db70 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd092da4740 10 client.19023 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd092da4740 10 client.19023 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd092da4740 20 client.19023 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd092da4740 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5606807b6a10 0x5606807b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd092da4740 4 client.19023 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.890+0000 7fd092da4740 1 -- 172.21.15.18:0/3407796967 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560680441720 con 0x5606807b6a10 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.891+0000 7fd08df6c640 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5606807b6a10 0x5606807b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.891+0000 7fd0717fa640 10 client.19023 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.891+0000 7fd08df6c640 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5606807b6a10 0x5606807b8e30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d8c0ba0747046eb server_cookie=644fc68f51ac4179 in_seq=0 out_seq=0 2024-02-17T12:59:31.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd0717fa640 1 -- 172.21.15.18:0/3407796967 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560680441720 con 0x5606807b6a10 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd0717fa640 10 client.19023 handle_command_reply: tid=0 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd092da4740 1 client.19023 shutdown 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd092da4740 1 -- 172.21.15.18:0/3407796967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079db70 msgr2=0x5606807a3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd092da4740 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079db70 0x5606807a3c70 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd07800a010 tx=0x7fd0780048f0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd092da4740 1 -- 172.21.15.18:0/3407796967 shutdown_connections 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd092da4740 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5606807b6a10 0x5606807b8e30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd092da4740 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606807a6750 0x5606807a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd0717fa640 0 client.19023 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd092da4740 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079db70 0x5606807a3c70 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd092da4740 1 --2- 172.21.15.18:0/3407796967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56068079d250 0x5606807a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd092da4740 1 -- 172.21.15.18:0/3407796967 >> 172.21.15.18:0/3407796967 conn(0x560680797980 msgr2=0x5606807996d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd092da4740 1 -- 172.21.15.18:0/3407796967 shutdown_connections 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd092da4740 1 -- 172.21.15.18:0/3407796967 wait complete. 2024-02-17T12:59:31.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.892+0000 7fd092da4740 20 client.19023 trim_cache size 0 max 16384 2024-02-17T12:59:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd092da4740 1 Processor -- start 2024-02-17T12:59:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd092da4740 1 -- start start 2024-02-17T12:59:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd092da4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 0x5606807a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd092da4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x560680796a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd092da4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5606807a6750 0x5606807ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd092da4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068068a130 con 0x56068079d250 2024-02-17T12:59:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd092da4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606807977b0 con 0x56068079db70 2024-02-17T12:59:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd092da4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606807a8b40 con 0x5606807a6750 2024-02-17T12:59:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd08df6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 0x5606807a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd08df6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 0x5606807a3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55484/0 (socket says 172.21.15.18:55484) 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd08df6c640 1 -- 172.21.15.18:0/1437179369 learned_addr learned my addr 172.21.15.18:0/1437179369 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd08e76d640 1 --2- 172.21.15.18:0/1437179369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5606807a6750 0x5606807ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd08d76b640 1 --2- 172.21.15.18:0/1437179369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x560680796a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd08df6c640 1 -- 172.21.15.18:0/1437179369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5606807a6750 msgr2=0x5606807ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd08df6c640 1 --2- 172.21.15.18:0/1437179369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5606807a6750 0x5606807ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd08df6c640 1 -- 172.21.15.18:0/1437179369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 msgr2=0x560680796a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd08df6c640 1 --2- 172.21.15.18:0/1437179369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x560680796a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd08df6c640 1 -- 172.21.15.18:0/1437179369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56068069fc20 con 0x56068079d250 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd08df6c640 1 --2- 172.21.15.18:0/1437179369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 0x5606807a3fd0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fd08402cde0 tx=0x7fd08408cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca14104aae57b30 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd0737fe640 1 -- 172.21.15.18:0/1437179369 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0840bd3b0 con 0x56068079d250 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd0737fe640 1 -- 172.21.15.18:0/1437179369 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0840bccb0 con 0x56068079d250 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.904+0000 7fd0737fe640 1 -- 172.21.15.18:0/1437179369 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0840bafb0 con 0x56068079d250 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.905+0000 7fd092da4740 1 -- 172.21.15.18:0/1437179369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 msgr2=0x5606807a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.905+0000 7fd092da4740 1 --2- 172.21.15.18:0/1437179369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 0x5606807a3fd0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fd08402cde0 tx=0x7fd08408cb10 comp rx=0 tx=0).stop 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.905+0000 7fd092da4740 1 -- 172.21.15.18:0/1437179369 shutdown_connections 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.905+0000 7fd092da4740 1 --2- 172.21.15.18:0/1437179369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5606807a6750 0x5606807ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.905+0000 7fd092da4740 1 --2- 172.21.15.18:0/1437179369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x560680796a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.905+0000 7fd092da4740 1 --2- 172.21.15.18:0/1437179369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 0x5606807a3fd0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.905+0000 7fd092da4740 1 -- 172.21.15.18:0/1437179369 >> 172.21.15.18:0/1437179369 conn(0x560680797980 msgr2=0x5606807962f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.905+0000 7fd092da4740 1 -- 172.21.15.18:0/1437179369 shutdown_connections 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.905+0000 7fd092da4740 1 -- 172.21.15.18:0/1437179369 wait complete. 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.905+0000 7fd092da4740 1 -- start start 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.905+0000 7fd092da4740 1 Processor -- start 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd092da4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 0x5606807aa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd092da4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x5606807aa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd092da4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5606807a6750 0x5606807aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd092da4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560680811430 con 0x56068079d250 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd092da4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606808115a0 con 0x56068079db70 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd092da4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560680811760 con 0x5606807a6750 2024-02-17T12:59:31.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd092da4740 10 client.0 _ll_register_callbacks cb 0x7fd0780093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd08df6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 0x5606807aa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd08df6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 0x5606807aa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55494/0 (socket says 172.21.15.18:55494) 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd08df6c640 1 -- 172.21.15.18:0/769924200 learned_addr learned my addr 172.21.15.18:0/769924200 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd08d76b640 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x5606807aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd08e76d640 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5606807a6750 0x5606807aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd08df6c640 1 -- 172.21.15.18:0/769924200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5606807a6750 msgr2=0x5606807aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd08df6c640 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5606807a6750 0x5606807aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd08df6c640 1 -- 172.21.15.18:0/769924200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 msgr2=0x5606807aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd08df6c640 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x5606807aa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd08df6c640 1 -- 172.21.15.18:0/769924200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0881a36d0 con 0x56068079d250 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd08df6c640 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 0x5606807aa200 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fd084009430 tx=0x7fd0840035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb1dd134f8d4e69 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd0717fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.906+0000 7fd0717fa640 1 -- 172.21.15.18:0/769924200 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0840c5450 con 0x56068079d250 2024-02-17T12:59:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd0717fa640 1 -- 172.21.15.18:0/769924200 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd0840bd3b0 con 0x56068079d250 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd092da4740 1 -- 172.21.15.18:0/769924200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd078005660 con 0x56068079d250 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd0717fa640 1 -- 172.21.15.18:0/769924200 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0840b9320 con 0x56068079d250 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd092da4740 10 client.19029 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd092da4740 1 -- 172.21.15.18:0/769924200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5606806c58c0 con 0x56068079d250 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd0717fa640 1 -- 172.21.15.18:0/769924200 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd078005660 con 0x56068079d250 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd0717fa640 1 -- 172.21.15.18:0/769924200 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd08403c070 con 0x56068079d250 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd092da4740 10 client.19029 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd092da4740 10 client.19029 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd092da4740 20 client.19029 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd092da4740 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x560680812af0 0x560680814ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd092da4740 4 client.19029 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd092da4740 1 -- 172.21.15.18:0/769924200 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560680490890 con 0x560680812af0 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.907+0000 7fd08d76b640 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x560680812af0 0x560680814ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd0717fa640 10 client.19029 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T12:59:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd08d76b640 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x560680812af0 0x560680814ee0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17b1e9fee42bbc6 server_cookie=aed759c13d307552 in_seq=0 out_seq=0 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd0717fa640 1 -- 172.21.15.18:0/769924200 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560680490890 con 0x560680812af0 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd0717fa640 10 client.19029 handle_command_reply: tid=0 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd092da4740 1 client.19029 shutdown 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd092da4740 1 -- 172.21.15.18:0/769924200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 msgr2=0x5606807aa200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd092da4740 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 0x5606807aa200 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fd084009430 tx=0x7fd0840035f0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd092da4740 1 -- 172.21.15.18:0/769924200 shutdown_connections 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd092da4740 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x560680812af0 0x560680814ee0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd092da4740 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5606807a6750 0x5606807aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd092da4740 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56068079db70 0x5606807aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd092da4740 1 --2- 172.21.15.18:0/769924200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56068079d250 0x5606807aa200 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd0717fa640 0 client.19029 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd092da4740 1 -- 172.21.15.18:0/769924200 >> 172.21.15.18:0/769924200 conn(0x560680797980 msgr2=0x560680796b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd092da4740 1 -- 172.21.15.18:0/769924200 shutdown_connections 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd092da4740 1 -- 172.21.15.18:0/769924200 wait complete. 2024-02-17T12:59:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.908+0000 7fd092da4740 20 client.19029 trim_cache size 0 max 16384 2024-02-17T12:59:31.905 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:59:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.909+0000 7fd0901d0640 1 -- 172.21.15.18:0/2933327912 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd0740a2230 msgr2=0x7fd0740a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.909+0000 7fd0901d0640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd0740a2230 0x7fd0740a46f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd080009fd0 tx=0x7fd080009500 comp rx=0 tx=0).stop 2024-02-17T12:59:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.909+0000 7fd0901d0640 1 -- 172.21.15.18:0/2933327912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd088100260 msgr2=0x7fd0881966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.909+0000 7fd0901d0640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd088100260 0x7fd0881966e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd084009230 tx=0x7fd08402c6c0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.909+0000 7fd0901d0640 1 -- 172.21.15.18:0/2933327912 shutdown_connections 2024-02-17T12:59:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.909+0000 7fd0901d0640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd0740a2230 0x7fd0740a46f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.909+0000 7fd0901d0640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd08819afa0 0x7fd08819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.909+0000 7fd0901d0640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd088105330 0x7fd088196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.909+0000 7fd0901d0640 1 --2- 172.21.15.18:0/2933327912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd088100260 0x7fd0881966e0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.909+0000 7fd0901d0640 1 -- 172.21.15.18:0/2933327912 >> 172.21.15.18:0/2933327912 conn(0x7fd088077180 msgr2=0x7fd0880fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.909+0000 7fd0901d0640 1 -- 172.21.15.18:0/2933327912 shutdown_connections 2024-02-17T12:59:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:31.909+0000 7fd0901d0640 1 -- 172.21.15.18:0/2933327912 wait complete. 2024-02-17T12:59:31.909 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:59:31.910 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:59:32.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.024+0000 7f2b52fc9640 1 -- 172.21.15.18:0/1069647505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b4c101380 msgr2=0x7f2b4c101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:32.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.024+0000 7f2b52fc9640 1 --2- 172.21.15.18:0/1069647505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b4c101380 0x7f2b4c101760 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009930 tx=0x7f2b4002bc30 comp rx=0 tx=0).stop 2024-02-17T12:59:32.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.024+0000 7f2b52fc9640 1 -- 172.21.15.18:0/1069647505 shutdown_connections 2024-02-17T12:59:32.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.024+0000 7f2b52fc9640 1 --2- 172.21.15.18:0/1069647505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b4c106450 0x7f2b4c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.024+0000 7f2b52fc9640 1 --2- 172.21.15.18:0/1069647505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b4c101d30 0x7f2b4c105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.024+0000 7f2b52fc9640 1 --2- 172.21.15.18:0/1069647505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b4c101380 0x7f2b4c101760 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.024+0000 7f2b52fc9640 1 -- 172.21.15.18:0/1069647505 >> 172.21.15.18:0/1069647505 conn(0x7f2b4c078580 msgr2=0x7f2b4c078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:32.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.024+0000 7f2b52fc9640 1 -- 172.21.15.18:0/1069647505 shutdown_connections 2024-02-17T12:59:32.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.024+0000 7f2b52fc9640 1 -- 172.21.15.18:0/1069647505 wait complete. 2024-02-17T12:59:32.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b52fc9640 1 Processor -- start 2024-02-17T12:59:32.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b52fc9640 1 -- start start 2024-02-17T12:59:32.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b52fc9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b4c101380 0x7f2b4c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b52fc9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b4c106450 0x7f2b4c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b52fc9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b4c197650 0x7f2b4c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b52fc9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c198120 con 0x7f2b4c101380 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b52fc9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c198290 con 0x7f2b4c197650 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b52fc9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c198450 con 0x7f2b4c106450 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b50d65640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b4c101380 0x7f2b4c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b50d65640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b4c101380 0x7f2b4c19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55508/0 (socket says 172.21.15.18:55508) 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b50d65640 1 -- 172.21.15.18:0/4069552581 learned_addr learned my addr 172.21.15.18:0/4069552581 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b4bfff640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b4c106450 0x7f2b4c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b51566640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b4c197650 0x7f2b4c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b50d65640 1 -- 172.21.15.18:0/4069552581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b4c106450 msgr2=0x7f2b4c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b50d65640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b4c106450 0x7f2b4c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b50d65640 1 -- 172.21.15.18:0/4069552581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b4c197650 msgr2=0x7f2b4c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:32.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b50d65640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b4c197650 0x7f2b4c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b50d65640 1 -- 172.21.15.18:0/4069552581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b40009560 con 0x7f2b4c101380 2024-02-17T12:59:32.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b4bfff640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b4c106450 0x7f2b4c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:32.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.025+0000 7f2b50d65640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b4c101380 0x7f2b4c19d590 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009230 tx=0x7f2b40004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:32.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.026+0000 7f2b49ffb640 1 -- 172.21.15.18:0/4069552581 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40004440 con 0x7f2b4c101380 2024-02-17T12:59:32.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.026+0000 7f2b49ffb640 1 -- 172.21.15.18:0/4069552581 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b400052d0 con 0x7f2b4c101380 2024-02-17T12:59:32.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.026+0000 7f2b52fc9640 1 -- 172.21.15.18:0/4069552581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4c1a4390 con 0x7f2b4c101380 2024-02-17T12:59:32.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.026+0000 7f2b49ffb640 1 -- 172.21.15.18:0/4069552581 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40003d70 con 0x7f2b4c101380 2024-02-17T12:59:32.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.026+0000 7f2b52fc9640 1 -- 172.21.15.18:0/4069552581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4c1a48c0 con 0x7f2b4c101380 2024-02-17T12:59:32.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.028+0000 7f2b49ffb640 1 -- 172.21.15.18:0/4069552581 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2b40005440 con 0x7f2b4c101380 2024-02-17T12:59:32.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.028+0000 7f2b55b95740 1 Processor -- start 2024-02-17T12:59:32.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.028+0000 7f2b55b95740 1 -- start start 2024-02-17T12:59:32.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b55b95740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eec6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b55b95740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eec6b70 0x562d5eecf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b49ffb640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b3c0a22b0 0x7f2b3c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b55b95740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 0x562d5eed1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b55b95740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5ee81960 con 0x562d5eecf750 2024-02-17T12:59:32.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b55b95740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5ee35310 con 0x562d5eec6250 2024-02-17T12:59:32.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b55b95740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5ee32840 con 0x562d5eec6b70 2024-02-17T12:59:32.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b51566640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 0x562d5eed1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b51566640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 0x562d5eed1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55524/0 (socket says 172.21.15.18:55524) 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b51566640 1 -- 172.21.15.18:0/1228953939 learned_addr learned my addr 172.21.15.18:0/1228953939 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b50d65640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b3c0a22b0 0x7f2b3c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b51566640 1 --2- 172.21.15.18:0/1228953939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eec6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b4bfff640 1 --2- 172.21.15.18:0/1228953939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eec6b70 0x562d5eecf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b49ffb640 1 -- 172.21.15.18:0/4069552581 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2b400f0270 con 0x7f2b4c101380 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b51566640 1 -- 172.21.15.18:0/1228953939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eec6b70 msgr2=0x562d5eecf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b51566640 1 --2- 172.21.15.18:0/1228953939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eec6b70 0x562d5eecf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b51566640 1 -- 172.21.15.18:0/1228953939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 msgr2=0x562d5eec6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b51566640 1 --2- 172.21.15.18:0/1228953939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eec6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b51566640 1 -- 172.21.15.18:0/1228953939 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d5edf0060 con 0x562d5eecf750 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b4bfff640 1 --2- 172.21.15.18:0/1228953939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eec6b70 0x562d5eecf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b50d65640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b3c0a22b0 0x7f2b3c0a4770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009430 tx=0x7f2b4008bed0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b51566640 1 --2- 172.21.15.18:0/1228953939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 0x562d5eed1b40 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009370 tx=0x7f2b44009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544dc28b2908ba40 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:32.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.029+0000 7f2b36ffd640 1 -- 172.21.15.18:0/1228953939 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44014110 con 0x562d5eecf750 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.030+0000 7f2b36ffd640 1 -- 172.21.15.18:0/1228953939 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b4401a070 con 0x562d5eecf750 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.030+0000 7f2b36ffd640 1 -- 172.21.15.18:0/1228953939 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44004230 con 0x562d5eecf750 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.030+0000 7f2b55b95740 1 -- 172.21.15.18:0/1228953939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 msgr2=0x562d5eed1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.030+0000 7f2b55b95740 1 --2- 172.21.15.18:0/1228953939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 0x562d5eed1b40 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009370 tx=0x7f2b44009770 comp rx=0 tx=0).stop 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.030+0000 7f2b55b95740 1 -- 172.21.15.18:0/1228953939 shutdown_connections 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.030+0000 7f2b55b95740 1 --2- 172.21.15.18:0/1228953939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 0x562d5eed1b40 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.030+0000 7f2b55b95740 1 --2- 172.21.15.18:0/1228953939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eec6b70 0x562d5eecf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.030+0000 7f2b55b95740 1 --2- 172.21.15.18:0/1228953939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eec6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.030+0000 7f2b55b95740 1 -- 172.21.15.18:0/1228953939 >> 172.21.15.18:0/1228953939 conn(0x562d5eec0980 msgr2=0x562d5eec0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.030+0000 7f2b55b95740 1 -- 172.21.15.18:0/1228953939 shutdown_connections 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.030+0000 7f2b55b95740 1 -- 172.21.15.18:0/1228953939 wait complete. 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.030+0000 7f2b55b95740 1 -- start start 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b55b95740 1 Processor -- start 2024-02-17T12:59:32.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b55b95740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eecc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b55b95740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6b70 0x562d5eecccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b55b95740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eecf750 0x562d5eecd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b55b95740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5ee2e020 con 0x562d5eec6b70 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b55b95740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5eb01430 con 0x562d5eec6250 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b55b95740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5eabde00 con 0x562d5eecf750 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b55b95740 10 client.0 _ll_register_callbacks cb 0x562d5edd5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b4bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6b70 0x562d5eecccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b4bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6b70 0x562d5eecccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55530/0 (socket says 172.21.15.18:55530) 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b4bfff640 1 -- 172.21.15.18:0/264716664 learned_addr learned my addr 172.21.15.18:0/264716664 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b51566640 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eecc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b51566640 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eecf750 0x562d5eecd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b4bfff640 1 -- 172.21.15.18:0/264716664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eecf750 msgr2=0x562d5eecd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b4bfff640 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eecf750 0x562d5eecd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b4bfff640 1 -- 172.21.15.18:0/264716664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 msgr2=0x562d5eecc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b4bfff640 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eecc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b4bfff640 1 -- 172.21.15.18:0/264716664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b440148f0 con 0x562d5eec6b70 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.031+0000 7f2b51566640 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eecc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b4bfff640 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6b70 0x562d5eecccc0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800bf30 tx=0x7f2b38011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15db69f718a65e78 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b2bfff640 1 -- 172.21.15.18:0/264716664 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38005220 con 0x562d5eec6b70 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b2bfff640 1 -- 172.21.15.18:0/264716664 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2b38005380 con 0x562d5eec6b70 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b55b95740 1 -- 172.21.15.18:0/264716664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d5eb2d7e0 con 0x562d5eec6b70 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b2bfff640 1 -- 172.21.15.18:0/264716664 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b38005540 con 0x562d5eec6b70 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b55b95740 10 client.19041 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b55b95740 1 -- 172.21.15.18:0/264716664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x562d5ee2c6a0 con 0x562d5eec6b70 2024-02-17T12:59:32.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b2bfff640 1 -- 172.21.15.18:0/264716664 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562d5eb2d7e0 con 0x562d5eec6b70 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b2bfff640 1 -- 172.21.15.18:0/264716664 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2b380129c0 con 0x562d5eec6b70 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b55b95740 10 client.19041 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b55b95740 10 client.19041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b55b95740 20 client.19041 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b55b95740 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d5eedfbb0 0x562d5eee1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b55b95740 4 client.19041 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.032+0000 7f2b55b95740 1 -- 172.21.15.18:0/264716664 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d5eb6a720 con 0x562d5eedfbb0 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.033+0000 7f2b51566640 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d5eedfbb0 0x562d5eee1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.033+0000 7f2b2bfff640 10 client.19041 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.033+0000 7f2b51566640 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d5eedfbb0 0x562d5eee1fd0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8376a49236849a98 server_cookie=cfbe598574d896ac in_seq=0 out_seq=0 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b2bfff640 1 -- 172.21.15.18:0/264716664 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d5eb6a720 con 0x562d5eedfbb0 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b2bfff640 10 client.19041 handle_command_reply: tid=0 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b55b95740 1 client.19041 shutdown 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b55b95740 1 -- 172.21.15.18:0/264716664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6b70 msgr2=0x562d5eecccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b55b95740 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6b70 0x562d5eecccc0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800bf30 tx=0x7f2b38011870 comp rx=0 tx=0).stop 2024-02-17T12:59:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b55b95740 1 -- 172.21.15.18:0/264716664 shutdown_connections 2024-02-17T12:59:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b55b95740 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d5eedfbb0 0x562d5eee1fd0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b55b95740 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eecf750 0x562d5eecd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b2bfff640 0 client.19041 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b55b95740 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6b70 0x562d5eecccc0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b55b95740 1 --2- 172.21.15.18:0/264716664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eecc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b55b95740 1 -- 172.21.15.18:0/264716664 >> 172.21.15.18:0/264716664 conn(0x562d5eec0980 msgr2=0x562d5eec26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b55b95740 1 -- 172.21.15.18:0/264716664 shutdown_connections 2024-02-17T12:59:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b55b95740 1 -- 172.21.15.18:0/264716664 wait complete. 2024-02-17T12:59:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.034+0000 7f2b55b95740 20 client.19041 trim_cache size 0 max 16384 2024-02-17T12:59:32.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b55b95740 1 Processor -- start 2024-02-17T12:59:32.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b55b95740 1 -- start start 2024-02-17T12:59:32.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b55b95740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6250 0x562d5eeceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b55b95740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6b70 0x562d5eecd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b55b95740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eecf750 0x562d5eee6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b55b95740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5edb3130 con 0x562d5eec6250 2024-02-17T12:59:32.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b55b95740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5eedc6c0 con 0x562d5eec6b70 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b55b95740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5eede3d0 con 0x562d5eecf750 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b51566640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6250 0x562d5eeceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b51566640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6250 0x562d5eeceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55536/0 (socket says 172.21.15.18:55536) 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b51566640 1 -- 172.21.15.18:0/3465265178 learned_addr learned my addr 172.21.15.18:0/3465265178 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b51566640 1 -- 172.21.15.18:0/3465265178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eecf750 msgr2=0x562d5eee6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b51566640 1 --2- 172.21.15.18:0/3465265178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eecf750 0x562d5eee6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b51566640 1 -- 172.21.15.18:0/3465265178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6b70 msgr2=0x562d5eecd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b51566640 1 --2- 172.21.15.18:0/3465265178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6b70 0x562d5eecd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b51566640 1 -- 172.21.15.18:0/3465265178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d5edc8c20 con 0x562d5eec6250 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b51566640 1 --2- 172.21.15.18:0/3465265178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6250 0x562d5eeceeb0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2b44005c40 tx=0x7f2b44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f21e20db679982b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b367fc640 1 -- 172.21.15.18:0/3465265178 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44025040 con 0x562d5eec6250 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b367fc640 1 -- 172.21.15.18:0/3465265178 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b44009670 con 0x562d5eec6250 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.046+0000 7f2b367fc640 1 -- 172.21.15.18:0/3465265178 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4402b5c0 con 0x562d5eec6250 2024-02-17T12:59:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.047+0000 7f2b55b95740 1 -- 172.21.15.18:0/3465265178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6250 msgr2=0x562d5eeceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.047+0000 7f2b55b95740 1 --2- 172.21.15.18:0/3465265178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6250 0x562d5eeceeb0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2b44005c40 tx=0x7f2b44009e30 comp rx=0 tx=0).stop 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.047+0000 7f2b55b95740 1 -- 172.21.15.18:0/3465265178 shutdown_connections 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.047+0000 7f2b55b95740 1 --2- 172.21.15.18:0/3465265178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eecf750 0x562d5eee6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.047+0000 7f2b55b95740 1 --2- 172.21.15.18:0/3465265178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6b70 0x562d5eecd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.047+0000 7f2b55b95740 1 --2- 172.21.15.18:0/3465265178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eec6250 0x562d5eeceeb0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.047+0000 7f2b55b95740 1 -- 172.21.15.18:0/3465265178 >> 172.21.15.18:0/3465265178 conn(0x562d5eec0980 msgr2=0x562d5eecb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.047+0000 7f2b55b95740 1 -- 172.21.15.18:0/3465265178 shutdown_connections 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.047+0000 7f2b55b95740 1 -- 172.21.15.18:0/3465265178 wait complete. 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.047+0000 7f2b55b95740 1 -- start start 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.047+0000 7f2b55b95740 1 Processor -- start 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b55b95740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eed6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b55b95740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eec6b70 0x562d5eed4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b55b95740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 0x562d5eed5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b55b95740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5eed5d30 con 0x562d5eecf750 2024-02-17T12:59:32.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b55b95740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5ef360c0 con 0x562d5eec6250 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b55b95740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5ef36200 con 0x562d5eec6b70 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b55b95740 10 client.0 _ll_register_callbacks cb 0x7f2b44009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b51566640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 0x562d5eed5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b51566640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 0x562d5eed5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55546/0 (socket says 172.21.15.18:55546) 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b51566640 1 -- 172.21.15.18:0/625986935 learned_addr learned my addr 172.21.15.18:0/625986935 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b4bfff640 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eec6b70 0x562d5eed4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b51566640 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eed6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b51566640 1 -- 172.21.15.18:0/625986935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eec6b70 msgr2=0x562d5eed4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b51566640 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eec6b70 0x562d5eed4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b51566640 1 -- 172.21.15.18:0/625986935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 msgr2=0x562d5eed6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b51566640 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eed6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b51566640 1 -- 172.21.15.18:0/625986935 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b44028040 con 0x562d5eecf750 2024-02-17T12:59:32.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b51566640 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 0x562d5eed5660 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f2b44005040 tx=0x7f2b44003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9726b38aff126dc4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.048+0000 7f2b2bfff640 1 -- 172.21.15.18:0/625986935 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44003bf0 con 0x562d5eecf750 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b2bfff640 1 -- 172.21.15.18:0/625986935 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2b44014040 con 0x562d5eecf750 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b55b95740 1 -- 172.21.15.18:0/625986935 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b4401a070 con 0x562d5eecf750 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b2bfff640 1 -- 172.21.15.18:0/625986935 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b44014200 con 0x562d5eecf750 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b55b95740 10 client.19047 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b55b95740 1 -- 172.21.15.18:0/625986935 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x562d5edee8c0 con 0x562d5eecf750 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b2bfff640 1 -- 172.21.15.18:0/625986935 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b44043020 con 0x562d5eecf750 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b2bfff640 1 -- 172.21.15.18:0/625986935 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2b440054c0 con 0x562d5eecf750 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b55b95740 10 client.19047 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b55b95740 10 client.19047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b55b95740 20 client.19047 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:32.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b55b95740 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d5ef37560 0x562d5ef39980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b55b95740 4 client.19047 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b55b95740 1 -- 172.21.15.18:0/625986935 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562d5ebb9890 con 0x562d5ef37560 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.049+0000 7f2b4bfff640 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d5ef37560 0x562d5ef39980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b2bfff640 10 client.19047 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b4bfff640 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d5ef37560 0x562d5ef39980 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e3db72d1293fecc server_cookie=4b7d5238f34e4382 in_seq=0 out_seq=0 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b2bfff640 1 -- 172.21.15.18:0/625986935 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562d5ebb9890 con 0x562d5ef37560 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b2bfff640 10 client.19047 handle_command_reply: tid=0 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b55b95740 1 client.19047 shutdown 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b55b95740 1 -- 172.21.15.18:0/625986935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 msgr2=0x562d5eed5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b55b95740 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 0x562d5eed5660 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f2b44005040 tx=0x7f2b44003db0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b55b95740 1 -- 172.21.15.18:0/625986935 shutdown_connections 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b55b95740 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562d5ef37560 0x562d5ef39980 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b55b95740 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562d5eecf750 0x562d5eed5660 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b2bfff640 0 client.19047 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:32.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b55b95740 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562d5eec6b70 0x562d5eed4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b55b95740 1 --2- 172.21.15.18:0/625986935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d5eec6250 0x562d5eed6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b55b95740 1 -- 172.21.15.18:0/625986935 >> 172.21.15.18:0/625986935 conn(0x562d5eec0980 msgr2=0x562d5eee6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:32.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b55b95740 1 -- 172.21.15.18:0/625986935 shutdown_connections 2024-02-17T12:59:32.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b55b95740 1 -- 172.21.15.18:0/625986935 wait complete. 2024-02-17T12:59:32.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.050+0000 7f2b55b95740 20 client.19047 trim_cache size 0 max 16384 2024-02-17T12:59:32.047 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T12:59:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.051+0000 7f2b52fc9640 1 -- 172.21.15.18:0/4069552581 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b3c0a22b0 msgr2=0x7f2b3c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.051+0000 7f2b52fc9640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b3c0a22b0 0x7f2b3c0a4770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009430 tx=0x7f2b4008bed0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.051+0000 7f2b52fc9640 1 -- 172.21.15.18:0/4069552581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b4c101380 msgr2=0x7f2b4c19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.051+0000 7f2b52fc9640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b4c101380 0x7f2b4c19d590 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009230 tx=0x7f2b40004040 comp rx=0 tx=0).stop 2024-02-17T12:59:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.051+0000 7f2b52fc9640 1 -- 172.21.15.18:0/4069552581 shutdown_connections 2024-02-17T12:59:32.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.051+0000 7f2b52fc9640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b3c0a22b0 0x7f2b3c0a4770 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.051+0000 7f2b52fc9640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b4c197650 0x7f2b4c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.051+0000 7f2b52fc9640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b4c106450 0x7f2b4c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.051+0000 7f2b52fc9640 1 --2- 172.21.15.18:0/4069552581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b4c101380 0x7f2b4c19d590 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.051+0000 7f2b52fc9640 1 -- 172.21.15.18:0/4069552581 >> 172.21.15.18:0/4069552581 conn(0x7f2b4c078580 msgr2=0x7f2b4c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.051+0000 7f2b52fc9640 1 -- 172.21.15.18:0/4069552581 shutdown_connections 2024-02-17T12:59:32.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:32.051+0000 7f2b52fc9640 1 -- 172.21.15.18:0/4069552581 wait complete. 2024-02-17T12:59:32.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:32 smithi018 ceph-mon[32625]: pgmap v301: 97 pgs: 97 active+clean; 195 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-02-17T12:59:32.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:32 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/540022487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:59:32.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 488453 5.11 MB/sec execute 407 sec latency 799.859 ms 2024-02-17T12:59:32.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:32 smithi119 ceph-mon[36188]: pgmap v301: 97 pgs: 97 active+clean; 195 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-02-17T12:59:32.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:32 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/540022487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:59:32.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:32 smithi146 ceph-mon[36454]: pgmap v301: 97 pgs: 97 active+clean; 195 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-02-17T12:59:32.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:32 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/540022487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:59:33.053 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:59:33.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:59:33.054 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:59:33.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.180+0000 7fad375bc640 1 -- 172.21.15.18:0/2049555208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad30101390 msgr2=0x7fad30101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.180+0000 7fad375bc640 1 --2- 172.21.15.18:0/2049555208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad30101390 0x7fad30101770 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009930 tx=0x7fad2c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:59:33.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.180+0000 7fad375bc640 1 -- 172.21.15.18:0/2049555208 shutdown_connections 2024-02-17T12:59:33.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.180+0000 7fad375bc640 1 --2- 172.21.15.18:0/2049555208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad30106460 0x7fad3010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.180+0000 7fad375bc640 1 --2- 172.21.15.18:0/2049555208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad30101d40 0x7fad30105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.180+0000 7fad375bc640 1 --2- 172.21.15.18:0/2049555208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad30101390 0x7fad30101770 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.180+0000 7fad375bc640 1 -- 172.21.15.18:0/2049555208 >> 172.21.15.18:0/2049555208 conn(0x7fad300fcbb0 msgr2=0x7fad300fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:33.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad375bc640 1 -- 172.21.15.18:0/2049555208 shutdown_connections 2024-02-17T12:59:33.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad375bc640 1 -- 172.21.15.18:0/2049555208 wait complete. 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad375bc640 1 Processor -- start 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad375bc640 1 -- start start 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad375bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad30101390 0x7fad30073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad375bc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad30106460 0x7fad300739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad375bc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad30073f00 0x7fad301a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad375bc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad301a5f30 con 0x7fad30106460 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad375bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad301a60a0 con 0x7fad30101390 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad375bc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad301a6260 con 0x7fad30073f00 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad34b57640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad30106460 0x7fad300739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad34b57640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad30106460 0x7fad300739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55564/0 (socket says 172.21.15.18:55564) 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.181+0000 7fad34b57640 1 -- 172.21.15.18:0/65567792 learned_addr learned my addr 172.21.15.18:0/65567792 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad35b59640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad30073f00 0x7fad301a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad35358640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad30101390 0x7fad30073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad34b57640 1 -- 172.21.15.18:0/65567792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad30073f00 msgr2=0x7fad301a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad34b57640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad30073f00 0x7fad301a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad34b57640 1 -- 172.21.15.18:0/65567792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad30101390 msgr2=0x7fad30073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad34b57640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad30101390 0x7fad30073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad34b57640 1 -- 172.21.15.18:0/65567792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c009560 con 0x7fad30106460 2024-02-17T12:59:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad35b59640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad30073f00 0x7fad301a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad34b57640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad30106460 0x7fad300739c0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c970 tx=0x7fad2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad267fc640 1 -- 172.21.15.18:0/65567792 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20004440 con 0x7fad30106460 2024-02-17T12:59:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad267fc640 1 -- 172.21.15.18:0/65567792 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad20007d40 con 0x7fad30106460 2024-02-17T12:59:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad375bc640 1 -- 172.21.15.18:0/65567792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad301a6540 con 0x7fad30106460 2024-02-17T12:59:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad267fc640 1 -- 172.21.15.18:0/65567792 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2000f450 con 0x7fad30106460 2024-02-17T12:59:33.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.182+0000 7fad375bc640 1 -- 172.21.15.18:0/65567792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad301a6a50 con 0x7fad30106460 2024-02-17T12:59:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.184+0000 7fad267fc640 1 -- 172.21.15.18:0/65567792 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fad20007860 con 0x7fad30106460 2024-02-17T12:59:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad3a190740 1 Processor -- start 2024-02-17T12:59:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad3a190740 1 -- start start 2024-02-17T12:59:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad267fc640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fad1c0a2230 0x7fad1c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad3a190740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fc380 0x560ce53fc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad3a190740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce53fcd30 0x560ce54052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad3a190740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 0x560ce5407d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad3a190740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce536af90 con 0x560ce54058b0 2024-02-17T12:59:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad3a190740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce53684c0 con 0x560ce53fcd30 2024-02-17T12:59:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad3a190740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce5363ca0 con 0x560ce53fc380 2024-02-17T12:59:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad35b59640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fad1c0a2230 0x7fad1c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad267fc640 1 -- 172.21.15.18:0/65567792 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fad200ce200 con 0x7fad30106460 2024-02-17T12:59:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad35b59640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 0x560ce5407d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad34b57640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce53fcd30 0x560ce54052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad35b59640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 0x560ce5407d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55568/0 (socket says 172.21.15.18:55568) 2024-02-17T12:59:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad34b57640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce53fcd30 0x560ce54052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:52150/0 (socket says 172.21.15.18:52150) 2024-02-17T12:59:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad35b59640 1 -- 172.21.15.18:0/4148280831 learned_addr learned my addr 172.21.15.18:0/4148280831 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.185+0000 7fad35358640 1 --2- 172.21.15.18:0/4148280831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fc380 0x560ce53fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.186+0000 7fad34b57640 1 -- 172.21.15.18:0/4148280831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fc380 msgr2=0x560ce53fc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.186+0000 7fad34b57640 1 --2- 172.21.15.18:0/4148280831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fc380 0x560ce53fc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.186+0000 7fad34b57640 1 -- 172.21.15.18:0/4148280831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 msgr2=0x560ce5407d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.186+0000 7fad34b57640 1 --2- 172.21.15.18:0/4148280831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 0x560ce5407d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.186+0000 7fad34b57640 1 -- 172.21.15.18:0/4148280831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ce53260b0 con 0x560ce53fcd30 2024-02-17T12:59:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.186+0000 7fad35b59640 1 --2- 172.21.15.18:0/4148280831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 0x560ce5407d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.186+0000 7fad35b59640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fad1c0a2230 0x7fad1c0a46f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fad28012be0 tx=0x7fad28009db0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.186+0000 7fad34b57640 1 --2- 172.21.15.18:0/4148280831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce53fcd30 0x560ce54052e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fad200cb970 tx=0x7fad2009a9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=805a8ed3f145d2d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.186+0000 7fad1b7fe640 1 -- 172.21.15.18:0/4148280831 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad200ce070 con 0x560ce53fcd30 2024-02-17T12:59:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.186+0000 7fad1b7fe640 1 -- 172.21.15.18:0/4148280831 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad2009b310 con 0x560ce53fcd30 2024-02-17T12:59:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.186+0000 7fad1b7fe640 1 -- 172.21.15.18:0/4148280831 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20098f10 con 0x560ce53fcd30 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.187+0000 7fad3a190740 1 -- 172.21.15.18:0/4148280831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce53fcd30 msgr2=0x560ce54052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.187+0000 7fad3a190740 1 --2- 172.21.15.18:0/4148280831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce53fcd30 0x560ce54052e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fad200cb970 tx=0x7fad2009a9e0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.187+0000 7fad3a190740 1 -- 172.21.15.18:0/4148280831 shutdown_connections 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.187+0000 7fad3a190740 1 --2- 172.21.15.18:0/4148280831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 0x560ce5407d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.187+0000 7fad3a190740 1 --2- 172.21.15.18:0/4148280831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce53fcd30 0x560ce54052e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.187+0000 7fad3a190740 1 --2- 172.21.15.18:0/4148280831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fc380 0x560ce53fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.187+0000 7fad3a190740 1 -- 172.21.15.18:0/4148280831 >> 172.21.15.18:0/4148280831 conn(0x560ce53f6840 msgr2=0x560ce53f6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.187+0000 7fad3a190740 1 -- 172.21.15.18:0/4148280831 shutdown_connections 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.187+0000 7fad3a190740 1 -- 172.21.15.18:0/4148280831 wait complete. 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.187+0000 7fad3a190740 1 -- start start 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad3a190740 1 Processor -- start 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad3a190740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce53fc380 0x560ce5402800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad3a190740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fcd30 0x560ce5402d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad3a190740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 0x560ce5403280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad3a190740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce5308f40 con 0x560ce54058b0 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad3a190740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce50374b0 con 0x560ce53fc380 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad3a190740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce52e9670 con 0x560ce53fcd30 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad3a190740 10 client.0 _ll_register_callbacks cb 0x560ce5356f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad35b59640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 0x560ce5403280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad35b59640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 0x560ce5403280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55576/0 (socket says 172.21.15.18:55576) 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad35b59640 1 -- 172.21.15.18:0/1805634091 learned_addr learned my addr 172.21.15.18:0/1805634091 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad34b57640 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fcd30 0x560ce5402d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad35358640 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce53fc380 0x560ce5402800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad35b59640 1 -- 172.21.15.18:0/1805634091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fcd30 msgr2=0x560ce5402d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad35b59640 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fcd30 0x560ce5402d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad35b59640 1 -- 172.21.15.18:0/1805634091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce53fc380 msgr2=0x560ce5402800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad35b59640 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce53fc380 0x560ce5402800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.188+0000 7fad35b59640 1 -- 172.21.15.18:0/1805634091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad301a6a50 con 0x560ce54058b0 2024-02-17T12:59:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad35b59640 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 0x560ce5403280 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fad28010d80 tx=0x7fad28009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb82179265e5ced server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad18ff9640 1 -- 172.21.15.18:0/1805634091 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28010e70 con 0x560ce54058b0 2024-02-17T12:59:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad18ff9640 1 -- 172.21.15.18:0/1805634091 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fad28024070 con 0x560ce54058b0 2024-02-17T12:59:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad3a190740 1 -- 172.21.15.18:0/1805634091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ce533df40 con 0x560ce54058b0 2024-02-17T12:59:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad18ff9640 1 -- 172.21.15.18:0/1805634091 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad28004b20 con 0x560ce54058b0 2024-02-17T12:59:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad3a190740 10 client.19059 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad3a190740 1 -- 172.21.15.18:0/1805634091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x560ce5362320 con 0x560ce54058b0 2024-02-17T12:59:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad18ff9640 1 -- 172.21.15.18:0/1805634091 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560ce533df40 con 0x560ce54058b0 2024-02-17T12:59:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad18ff9640 1 -- 172.21.15.18:0/1805634091 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fad28030410 con 0x560ce54058b0 2024-02-17T12:59:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad3a190740 10 client.19059 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad3a190740 10 client.19059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad3a190740 20 client.19059 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad3a190740 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560ce5415e10 0x560ce5418230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad3a190740 4 client.19059 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.189+0000 7fad3a190740 1 -- 172.21.15.18:0/1805634091 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ce535a310 con 0x560ce5415e10 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.190+0000 7fad35358640 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560ce5415e10 0x560ce5418230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.190+0000 7fad35358640 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560ce5415e10 0x560ce5418230 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac353f9786f1a620 server_cookie=d1b1a8c506baff87 in_seq=0 out_seq=0 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.190+0000 7fad18ff9640 10 client.19059 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad18ff9640 1 -- 172.21.15.18:0/1805634091 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560ce535a310 con 0x560ce5415e10 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad18ff9640 10 client.19059 handle_command_reply: tid=0 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad3a190740 1 client.19059 shutdown 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad3a190740 1 -- 172.21.15.18:0/1805634091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 msgr2=0x560ce5403280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad3a190740 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 0x560ce5403280 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fad28010d80 tx=0x7fad28009280 comp rx=0 tx=0).stop 2024-02-17T12:59:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad3a190740 1 -- 172.21.15.18:0/1805634091 shutdown_connections 2024-02-17T12:59:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad3a190740 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560ce5415e10 0x560ce5418230 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad3a190740 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce54058b0 0x560ce5403280 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad3a190740 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fcd30 0x560ce5402d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad3a190740 1 --2- 172.21.15.18:0/1805634091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce53fc380 0x560ce5402800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad18ff9640 0 client.19059 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad3a190740 1 -- 172.21.15.18:0/1805634091 >> 172.21.15.18:0/1805634091 conn(0x560ce53f6840 msgr2=0x560ce53f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad3a190740 1 -- 172.21.15.18:0/1805634091 shutdown_connections 2024-02-17T12:59:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad3a190740 1 -- 172.21.15.18:0/1805634091 wait complete. 2024-02-17T12:59:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.191+0000 7fad3a190740 20 client.19059 trim_cache size 0 max 16384 2024-02-17T12:59:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad3a190740 1 Processor -- start 2024-02-17T12:59:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad3a190740 1 -- start start 2024-02-17T12:59:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad3a190740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fc380 0x560ce54030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad3a190740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fcd30 0x560ce540c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad3a190740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce54058b0 0x560ce540ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad3a190740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce53f7410 con 0x560ce53fc380 2024-02-17T12:59:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad3a190740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce53f0dc0 con 0x560ce54058b0 2024-02-17T12:59:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad3a190740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce5415be0 con 0x560ce53fcd30 2024-02-17T12:59:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad35358640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fc380 0x560ce54030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad35358640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fc380 0x560ce54030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55592/0 (socket says 172.21.15.18:55592) 2024-02-17T12:59:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad35358640 1 -- 172.21.15.18:0/1160366428 learned_addr learned my addr 172.21.15.18:0/1160366428 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad34b57640 1 --2- 172.21.15.18:0/1160366428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fcd30 0x560ce540c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.208+0000 7fad35b59640 1 --2- 172.21.15.18:0/1160366428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce54058b0 0x560ce540ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.209+0000 7fad35358640 1 -- 172.21.15.18:0/1160366428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fcd30 msgr2=0x560ce540c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.209+0000 7fad35358640 1 --2- 172.21.15.18:0/1160366428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fcd30 0x560ce540c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.209+0000 7fad35358640 1 -- 172.21.15.18:0/1160366428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce54058b0 msgr2=0x560ce540ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.209+0000 7fad35358640 1 --2- 172.21.15.18:0/1160366428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce54058b0 0x560ce540ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.209+0000 7fad35358640 1 -- 172.21.15.18:0/1160366428 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ce52fec00 con 0x560ce53fc380 2024-02-17T12:59:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.209+0000 7fad34b57640 1 --2- 172.21.15.18:0/1160366428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fcd30 0x560ce540c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.209+0000 7fad35b59640 1 --2- 172.21.15.18:0/1160366428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce54058b0 0x560ce540ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:59:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.209+0000 7fad35358640 1 --2- 172.21.15.18:0/1160366428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fc380 0x560ce54030a0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009900 tx=0x7fad2c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871dfb38663addf9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.209+0000 7fad1affd640 1 -- 172.21.15.18:0/1160366428 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c045040 con 0x560ce53fc380 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.209+0000 7fad1affd640 1 -- 172.21.15.18:0/1160366428 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad2c048be0 con 0x560ce53fc380 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.209+0000 7fad1affd640 1 -- 172.21.15.18:0/1160366428 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c048330 con 0x560ce53fc380 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 -- 172.21.15.18:0/1160366428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fc380 msgr2=0x560ce54030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 --2- 172.21.15.18:0/1160366428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fc380 0x560ce54030a0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009900 tx=0x7fad2c005fd0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 -- 172.21.15.18:0/1160366428 shutdown_connections 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 --2- 172.21.15.18:0/1160366428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce54058b0 0x560ce540ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 --2- 172.21.15.18:0/1160366428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fcd30 0x560ce540c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 --2- 172.21.15.18:0/1160366428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fc380 0x560ce54030a0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 -- 172.21.15.18:0/1160366428 >> 172.21.15.18:0/1160366428 conn(0x560ce53f6840 msgr2=0x560ce53f5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 -- 172.21.15.18:0/1160366428 shutdown_connections 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 -- 172.21.15.18:0/1160366428 wait complete. 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 -- start start 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 Processor -- start 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fc380 0x560ce540b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fcd30 0x560ce540bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce54058b0 0x560ce546c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce540c570 con 0x560ce53fcd30 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.210+0000 7fad3a190740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce546c880 con 0x560ce54058b0 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad3a190740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce546c9c0 con 0x560ce53fc380 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad3a190740 10 client.0 _ll_register_callbacks cb 0x7fad20003950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad34b57640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fcd30 0x560ce540bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad34b57640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fcd30 0x560ce540bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55606/0 (socket says 172.21.15.18:55606) 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad34b57640 1 -- 172.21.15.18:0/2538388339 learned_addr learned my addr 172.21.15.18:0/2538388339 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad35358640 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fc380 0x560ce540b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad35b59640 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce54058b0 0x560ce546c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad34b57640 1 -- 172.21.15.18:0/2538388339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fc380 msgr2=0x560ce540b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad34b57640 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fc380 0x560ce540b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad34b57640 1 -- 172.21.15.18:0/2538388339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce54058b0 msgr2=0x560ce546c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad34b57640 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce54058b0 0x560ce546c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad34b57640 1 -- 172.21.15.18:0/2538388339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3010e840 con 0x560ce53fcd30 2024-02-17T12:59:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad34b57640 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fcd30 0x560ce540bc90 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fad20003a00 tx=0x7fad20003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f328e88e38fbd71 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad18ff9640 1 -- 172.21.15.18:0/2538388339 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2009b310 con 0x560ce53fcd30 2024-02-17T12:59:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad18ff9640 1 -- 172.21.15.18:0/2538388339 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fad200b7050 con 0x560ce53fcd30 2024-02-17T12:59:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad3a190740 1 -- 172.21.15.18:0/2538388339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ce5389900 con 0x560ce53fcd30 2024-02-17T12:59:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad18ff9640 1 -- 172.21.15.18:0/2538388339 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad2009ac10 con 0x560ce53fcd30 2024-02-17T12:59:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad3a190740 10 client.19065 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.211+0000 7fad3a190740 1 -- 172.21.15.18:0/2538388339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x560ce5324910 con 0x560ce53fcd30 2024-02-17T12:59:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.212+0000 7fad18ff9640 1 -- 172.21.15.18:0/2538388339 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560ce5389900 con 0x560ce53fcd30 2024-02-17T12:59:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.212+0000 7fad18ff9640 1 -- 172.21.15.18:0/2538388339 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fad200b9020 con 0x560ce53fcd30 2024-02-17T12:59:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.212+0000 7fad3a190740 10 client.19065 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.212+0000 7fad3a190740 10 client.19065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.212+0000 7fad3a190740 20 client.19065 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.212+0000 7fad3a190740 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560ce546de20 0x560ce5470240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.212+0000 7fad3a190740 4 client.19065 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:59:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.212+0000 7fad3a190740 1 -- 172.21.15.18:0/2538388339 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560ce5063860 con 0x560ce546de20 2024-02-17T12:59:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.212+0000 7fad35358640 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560ce546de20 0x560ce5470240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.212+0000 7fad35358640 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560ce546de20 0x560ce5470240 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a85b2a2a50f06805 server_cookie=4b55e54abb2abae8 in_seq=0 out_seq=0 2024-02-17T12:59:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.212+0000 7fad18ff9640 10 client.19065 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:33.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.213+0000 7fad18ff9640 1 -- 172.21.15.18:0/2538388339 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560ce5063860 con 0x560ce546de20 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.213+0000 7fad18ff9640 10 client.19065 handle_command_reply: tid=0 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.213+0000 7fad3a190740 1 client.19065 shutdown 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.213+0000 7fad3a190740 1 -- 172.21.15.18:0/2538388339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fcd30 msgr2=0x560ce540bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.213+0000 7fad3a190740 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fcd30 0x560ce540bc90 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fad20003a00 tx=0x7fad20003a50 comp rx=0 tx=0).stop 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad3a190740 1 -- 172.21.15.18:0/2538388339 shutdown_connections 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad3a190740 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560ce546de20 0x560ce5470240 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad3a190740 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ce54058b0 0x560ce546c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad3a190740 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560ce53fcd30 0x560ce540bc90 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad18ff9640 0 client.19065 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad3a190740 1 --2- 172.21.15.18:0/2538388339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ce53fc380 0x560ce540b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad3a190740 1 -- 172.21.15.18:0/2538388339 >> 172.21.15.18:0/2538388339 conn(0x560ce53f6840 msgr2=0x560ce5401570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad3a190740 1 -- 172.21.15.18:0/2538388339 shutdown_connections 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad3a190740 1 -- 172.21.15.18:0/2538388339 wait complete. 2024-02-17T12:59:33.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad3a190740 20 client.19065 trim_cache size 0 max 16384 2024-02-17T12:59:33.212 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:59:33.212 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T12:59:33.212 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "4cfd9d80-267c-430c-af59-5cab3c547e5f", 2024-02-17T12:59:33.212 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T12:59:33.212 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:59:33.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad375bc640 1 -- 172.21.15.18:0/65567792 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fad1c0a2230 msgr2=0x7fad1c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad375bc640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fad1c0a2230 0x7fad1c0a46f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fad28012be0 tx=0x7fad28009db0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad375bc640 1 -- 172.21.15.18:0/65567792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad30106460 msgr2=0x7fad300739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad375bc640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad30106460 0x7fad300739c0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c970 tx=0x7fad2000ce30 comp rx=0 tx=0).stop 2024-02-17T12:59:33.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad375bc640 1 -- 172.21.15.18:0/65567792 shutdown_connections 2024-02-17T12:59:33.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad375bc640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fad1c0a2230 0x7fad1c0a46f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad375bc640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad30073f00 0x7fad301a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad375bc640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad30106460 0x7fad300739c0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad375bc640 1 --2- 172.21.15.18:0/65567792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad30101390 0x7fad30073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad375bc640 1 -- 172.21.15.18:0/65567792 >> 172.21.15.18:0/65567792 conn(0x7fad300fcbb0 msgr2=0x7fad300fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:33.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad375bc640 1 -- 172.21.15.18:0/65567792 shutdown_connections 2024-02-17T12:59:33.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.214+0000 7fad375bc640 1 -- 172.21.15.18:0/65567792 wait complete. 2024-02-17T12:59:33.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:59:33.217 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:59:33.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 488569 5.10 MB/sec execute 408 sec latency 1741.273 ms 2024-02-17T12:59:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.337+0000 7efd0a54f640 1 -- 172.21.15.18:0/3002249409 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd0410b6e0 msgr2=0x7efd0410db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.337+0000 7efd0a54f640 1 --2- 172.21.15.18:0/3002249409 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd0410b6e0 0x7efd0410db80 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009ca0 tx=0x7efcfc02c180 comp rx=0 tx=0).stop 2024-02-17T12:59:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.337+0000 7efd0a54f640 1 -- 172.21.15.18:0/3002249409 shutdown_connections 2024-02-17T12:59:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.337+0000 7efd0a54f640 1 --2- 172.21.15.18:0/3002249409 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd0410b6e0 0x7efd0410db80 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.337+0000 7efd0a54f640 1 --2- 172.21.15.18:0/3002249409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd04102cc0 0x7efd0410b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.337+0000 7efd0a54f640 1 --2- 172.21.15.18:0/3002249409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd041023a0 0x7efd04102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.337+0000 7efd0a54f640 1 -- 172.21.15.18:0/3002249409 >> 172.21.15.18:0/3002249409 conn(0x7efd04077230 msgr2=0x7efd04077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.338+0000 7efd0a54f640 1 -- 172.21.15.18:0/3002249409 shutdown_connections 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.338+0000 7efd0a54f640 1 -- 172.21.15.18:0/3002249409 wait complete. 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.338+0000 7efd0a54f640 1 Processor -- start 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.338+0000 7efd0a54f640 1 -- start start 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd0a54f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd041023a0 0x7efd04197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd0a54f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd04102cc0 0x7efd04197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd0a54f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd0410b6e0 0x7efd0419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd0a54f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0419c5b0 con 0x7efd04102cc0 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd0a54f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0419c720 con 0x7efd041023a0 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd0a54f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0419c8e0 con 0x7efd0410b6e0 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd037fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd04102cc0 0x7efd04197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd037fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd04102cc0 0x7efd04197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55618/0 (socket says 172.21.15.18:55618) 2024-02-17T12:59:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd037fe640 1 -- 172.21.15.18:0/899406061 learned_addr learned my addr 172.21.15.18:0/899406061 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd03fff640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd041023a0 0x7efd04197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd08aec640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd0410b6e0 0x7efd0419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd037fe640 1 -- 172.21.15.18:0/899406061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd0410b6e0 msgr2=0x7efd0419bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd037fe640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd0410b6e0 0x7efd0419bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd037fe640 1 -- 172.21.15.18:0/899406061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd041023a0 msgr2=0x7efd04197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd037fe640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd041023a0 0x7efd04197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd037fe640 1 -- 172.21.15.18:0/899406061 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc009900 con 0x7efd04102cc0 2024-02-17T12:59:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd037fe640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd04102cc0 0x7efd04197c80 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7efcf800e9f0 tx=0x7efcf800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd017fa640 1 -- 172.21.15.18:0/899406061 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf8009720 con 0x7efd04102cc0 2024-02-17T12:59:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd0a54f640 1 -- 172.21.15.18:0/899406061 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0419cbc0 con 0x7efd04102cc0 2024-02-17T12:59:33.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd017fa640 1 -- 172.21.15.18:0/899406061 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcf800cd80 con 0x7efd04102cc0 2024-02-17T12:59:33.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.339+0000 7efd017fa640 1 -- 172.21.15.18:0/899406061 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf8010690 con 0x7efd04102cc0 2024-02-17T12:59:33.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.340+0000 7efd0a54f640 1 -- 172.21.15.18:0/899406061 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd041a4440 con 0x7efd04102cc0 2024-02-17T12:59:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.342+0000 7efd017fa640 1 -- 172.21.15.18:0/899406061 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7efcf80046c0 con 0x7efd04102cc0 2024-02-17T12:59:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.342+0000 7efd0d202740 1 Processor -- start 2024-02-17T12:59:33.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.342+0000 7efd0d202740 1 -- start start 2024-02-17T12:59:33.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.342+0000 7efd0d202740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65d72b0 0x5645d65d7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.342+0000 7efd0d202740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d7bd0 0x5645d65e0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.342+0000 7efd0d202740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65e07b0 0x5645d65e2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.342+0000 7efd0d202740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d6546280 con 0x5645d65e07b0 2024-02-17T12:59:33.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.342+0000 7efd0d202740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d65437b0 con 0x5645d65d7bd0 2024-02-17T12:59:33.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.342+0000 7efd0d202740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d653ef90 con 0x5645d65d72b0 2024-02-17T12:59:33.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd037fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65e07b0 0x5645d65e2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd03fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d7bd0 0x5645d65e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd037fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65e07b0 0x5645d65e2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55630/0 (socket says 172.21.15.18:55630) 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd08aec640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65d72b0 0x5645d65d7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd037fe640 1 -- 172.21.15.18:0/3483113339 learned_addr learned my addr 172.21.15.18:0/3483113339 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd017fa640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efcec0a22b0 0x7efcec0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd03fff640 1 -- 172.21.15.18:0/3483113339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65d72b0 msgr2=0x5645d65d7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd03fff640 1 --2- 172.21.15.18:0/3483113339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65d72b0 0x5645d65d7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd03fff640 1 -- 172.21.15.18:0/3483113339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65e07b0 msgr2=0x5645d65e2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd03fff640 1 --2- 172.21.15.18:0/3483113339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65e07b0 0x5645d65e2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd03fff640 1 -- 172.21.15.18:0/3483113339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645d64fd570 con 0x5645d65d7bd0 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd017fa640 1 -- 172.21.15.18:0/899406061 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efcf8014070 con 0x7efd04102cc0 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd08aec640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efcec0a22b0 0x7efcec0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efd03fff640 1 --2- 172.21.15.18:0/3483113339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d7bd0 0x5645d65e0270 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009c20 tx=0x7efcf4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ae5fa1f557ac5a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efcea7fc640 1 -- 172.21.15.18:0/3483113339 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4004050 con 0x5645d65d7bd0 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efcea7fc640 1 -- 172.21.15.18:0/3483113339 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcf4005500 con 0x5645d65d7bd0 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.343+0000 7efcea7fc640 1 -- 172.21.15.18:0/3483113339 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4004cd0 con 0x5645d65d7bd0 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.344+0000 7efd0d202740 1 -- 172.21.15.18:0/3483113339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d7bd0 msgr2=0x5645d65e0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.344+0000 7efd0d202740 1 --2- 172.21.15.18:0/3483113339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d7bd0 0x5645d65e0270 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009c20 tx=0x7efcf4011810 comp rx=0 tx=0).stop 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.344+0000 7efd08aec640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efcec0a22b0 0x7efcec0a4770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efcfc004af0 tx=0x7efcfc0095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.344+0000 7efd0d202740 1 -- 172.21.15.18:0/3483113339 shutdown_connections 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.344+0000 7efd0d202740 1 --2- 172.21.15.18:0/3483113339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65e07b0 0x5645d65e2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.344+0000 7efd0d202740 1 --2- 172.21.15.18:0/3483113339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d7bd0 0x5645d65e0270 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.344+0000 7efd0d202740 1 --2- 172.21.15.18:0/3483113339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65d72b0 0x5645d65d7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.344+0000 7efd0d202740 1 -- 172.21.15.18:0/3483113339 >> 172.21.15.18:0/3483113339 conn(0x5645d65d1990 msgr2=0x5645d65d1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.344+0000 7efd0d202740 1 -- 172.21.15.18:0/3483113339 shutdown_connections 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.344+0000 7efd0d202740 1 -- 172.21.15.18:0/3483113339 wait complete. 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd0d202740 1 -- start start 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd0d202740 1 Processor -- start 2024-02-17T12:59:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd0d202740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d72b0 0x5645d65e0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd0d202740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d7bd0 0x5645d65dd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd0d202740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65e07b0 0x5645d65ddb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd0d202740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d6212430 con 0x5645d65d7bd0 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd0d202740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d61cee00 con 0x5645d65d72b0 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd0d202740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d64c3fd0 con 0x5645d65e07b0 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd0d202740 10 client.0 _ll_register_callbacks cb 0x5645d6519fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd037fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d7bd0 0x5645d65dd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd03fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d72b0 0x5645d65e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd037fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d7bd0 0x5645d65dd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55646/0 (socket says 172.21.15.18:55646) 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd037fe640 1 -- 172.21.15.18:0/3199959009 learned_addr learned my addr 172.21.15.18:0/3199959009 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.345+0000 7efd08aec640 1 --2- 172.21.15.18:0/3199959009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65e07b0 0x5645d65ddb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efd037fe640 1 -- 172.21.15.18:0/3199959009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65e07b0 msgr2=0x5645d65ddb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efd037fe640 1 --2- 172.21.15.18:0/3199959009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65e07b0 0x5645d65ddb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efd037fe640 1 -- 172.21.15.18:0/3199959009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d72b0 msgr2=0x5645d65e0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efd037fe640 1 --2- 172.21.15.18:0/3199959009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d72b0 0x5645d65e0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efd037fe640 1 -- 172.21.15.18:0/3199959009 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf4009850 con 0x5645d65d7bd0 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efd037fe640 1 --2- 172.21.15.18:0/3199959009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d7bd0 0x5645d65dd530 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7efcf800c6a0 tx=0x7efcf80d1750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c40444e2d895fd4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efce37fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efce37fe640 1 -- 172.21.15.18:0/3199959009 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf809a700 con 0x5645d65d7bd0 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efce37fe640 1 -- 172.21.15.18:0/3199959009 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efcf809b4a0 con 0x5645d65d7bd0 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efd0d202740 1 -- 172.21.15.18:0/3199959009 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645d623e7e0 con 0x5645d65d7bd0 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efce37fe640 1 -- 172.21.15.18:0/3199959009 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcf809ada0 con 0x5645d65d7bd0 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efd0d202740 10 client.19077 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efd0d202740 1 -- 172.21.15.18:0/3199959009 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5645d653d610 con 0x5645d65d7bd0 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efce37fe640 1 -- 172.21.15.18:0/3199959009 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcf80b7080 con 0x5645d65d7bd0 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efce37fe640 1 -- 172.21.15.18:0/3199959009 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7efcf801f060 con 0x5645d65d7bd0 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efd0d202740 10 client.19077 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efd0d202740 10 client.19077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.346+0000 7efd0d202740 20 client.19077 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.347+0000 7efd0d202740 1 --2- 172.21.15.18:0/3199959009 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5645d65f0940 0x5645d65f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.347+0000 7efd0d202740 4 client.19077 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.347+0000 7efd0d202740 1 -- 172.21.15.18:0/3199959009 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645d627b720 con 0x5645d65f0940 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.347+0000 7efd03fff640 1 --2- 172.21.15.18:0/3199959009 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5645d65f0940 0x5645d65f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.347+0000 7efd03fff640 1 --2- 172.21.15.18:0/3199959009 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5645d65f0940 0x5645d65f2d60 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4549262c75d6db9 server_cookie=a47aca56610b34c3 in_seq=0 out_seq=0 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.347+0000 7efce37fe640 10 client.19077 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efce37fe640 1 -- 172.21.15.18:0/3199959009 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5645d627b720 con 0x5645d65f0940 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efce37fe640 10 client.19077 handle_command_reply: tid=0 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efd0d202740 1 client.19077 shutdown 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efd0d202740 1 -- 172.21.15.18:0/3199959009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d7bd0 msgr2=0x5645d65dd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efd0d202740 1 --2- 172.21.15.18:0/3199959009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d7bd0 0x5645d65dd530 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7efcf800c6a0 tx=0x7efcf80d1750 comp rx=0 tx=0).stop 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efd0d202740 1 -- 172.21.15.18:0/3199959009 shutdown_connections 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efd0d202740 1 --2- 172.21.15.18:0/3199959009 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5645d65f0940 0x5645d65f2d60 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efd0d202740 1 --2- 172.21.15.18:0/3199959009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65e07b0 0x5645d65ddb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efd0d202740 1 --2- 172.21.15.18:0/3199959009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d7bd0 0x5645d65dd530 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efce37fe640 0 client.19077 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efd0d202740 1 --2- 172.21.15.18:0/3199959009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d72b0 0x5645d65e0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efd0d202740 1 -- 172.21.15.18:0/3199959009 >> 172.21.15.18:0/3199959009 conn(0x5645d65d1990 msgr2=0x5645d65d3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efd0d202740 1 -- 172.21.15.18:0/3199959009 shutdown_connections 2024-02-17T12:59:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efd0d202740 1 -- 172.21.15.18:0/3199959009 wait complete. 2024-02-17T12:59:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.348+0000 7efd0d202740 20 client.19077 trim_cache size 0 max 16384 2024-02-17T12:59:33.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.364+0000 7efd0d202740 1 Processor -- start 2024-02-17T12:59:33.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.364+0000 7efd0d202740 1 -- start start 2024-02-17T12:59:33.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.364+0000 7efd0d202740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 0x5645d65d87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.364+0000 7efd0d202740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d7bd0 0x5645d65d8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.364+0000 7efd0d202740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65e07b0 0x5645d65dbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.364+0000 7efd0d202740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d65d29e0 con 0x5645d65d72b0 2024-02-17T12:59:33.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.364+0000 7efd0d202740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d65d2790 con 0x5645d65d7bd0 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.364+0000 7efd0d202740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d65d82e0 con 0x5645d65e07b0 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efd03fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 0x5645d65d87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efd03fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 0x5645d65d87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55650/0 (socket says 172.21.15.18:55650) 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efd03fff640 1 -- 172.21.15.18:0/1425727054 learned_addr learned my addr 172.21.15.18:0/1425727054 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efd08aec640 1 --2- 172.21.15.18:0/1425727054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65e07b0 0x5645d65dbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efd037fe640 1 --2- 172.21.15.18:0/1425727054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d7bd0 0x5645d65d8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efd03fff640 1 -- 172.21.15.18:0/1425727054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65e07b0 msgr2=0x5645d65dbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efd03fff640 1 --2- 172.21.15.18:0/1425727054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65e07b0 0x5645d65dbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efd03fff640 1 -- 172.21.15.18:0/1425727054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d7bd0 msgr2=0x5645d65d8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efd03fff640 1 --2- 172.21.15.18:0/1425727054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d7bd0 0x5645d65d8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efd03fff640 1 -- 172.21.15.18:0/1425727054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645d64d9ac0 con 0x5645d65d72b0 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efd03fff640 1 --2- 172.21.15.18:0/1425727054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 0x5645d65d87b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009c20 tx=0x7efcf4004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887cd4cdd4fd0372 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efce9ffb640 1 -- 172.21.15.18:0/1425727054 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf402dd80 con 0x5645d65d72b0 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efce9ffb640 1 -- 172.21.15.18:0/1425727054 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcf40047e0 con 0x5645d65d72b0 2024-02-17T12:59:33.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.365+0000 7efce9ffb640 1 -- 172.21.15.18:0/1425727054 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf40044c0 con 0x5645d65d72b0 2024-02-17T12:59:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.366+0000 7efd0d202740 1 -- 172.21.15.18:0/1425727054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 msgr2=0x5645d65d87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.366+0000 7efd0d202740 1 --2- 172.21.15.18:0/1425727054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 0x5645d65d87b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009c20 tx=0x7efcf4004950 comp rx=0 tx=0).stop 2024-02-17T12:59:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 -- 172.21.15.18:0/1425727054 shutdown_connections 2024-02-17T12:59:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 --2- 172.21.15.18:0/1425727054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65e07b0 0x5645d65dbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 --2- 172.21.15.18:0/1425727054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65d7bd0 0x5645d65d8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 --2- 172.21.15.18:0/1425727054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 0x5645d65d87b0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 -- 172.21.15.18:0/1425727054 >> 172.21.15.18:0/1425727054 conn(0x5645d65d1990 msgr2=0x5645d65f7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 -- 172.21.15.18:0/1425727054 shutdown_connections 2024-02-17T12:59:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 -- 172.21.15.18:0/1425727054 wait complete. 2024-02-17T12:59:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 -- start start 2024-02-17T12:59:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 Processor -- start 2024-02-17T12:59:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 0x5645d65e4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65d7bd0 0x5645d65e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65e07b0 0x5645d65e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d6646e80 con 0x5645d65d72b0 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d6646fc0 con 0x5645d65e07b0 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645d6647100 con 0x5645d65d7bd0 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.367+0000 7efd0d202740 10 client.0 _ll_register_callbacks cb 0x7efcf40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd03fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 0x5645d65e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd037fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65d7bd0 0x5645d65e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd03fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 0x5645d65e4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55664/0 (socket says 172.21.15.18:55664) 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd03fff640 1 -- 172.21.15.18:0/1049335454 learned_addr learned my addr 172.21.15.18:0/1049335454 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd08aec640 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65e07b0 0x5645d65e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd037fe640 1 -- 172.21.15.18:0/1049335454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65e07b0 msgr2=0x5645d65e4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd037fe640 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65e07b0 0x5645d65e4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd037fe640 1 -- 172.21.15.18:0/1049335454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 msgr2=0x5645d65e4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd037fe640 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 0x5645d65e4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd037fe640 1 -- 172.21.15.18:0/1049335454 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645d64fd570 con 0x5645d65d7bd0 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd03fff640 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 0x5645d65e4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd037fe640 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65d7bd0 0x5645d65e4590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efcf8097200 tx=0x7efcf800c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e4f006d901085ab server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efce37fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efce37fe640 1 -- 172.21.15.18:0/1049335454 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf8097040 con 0x5645d65d7bd0 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efce37fe640 1 -- 172.21.15.18:0/1049335454 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efcf809b4a0 con 0x5645d65d7bd0 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd0d202740 1 -- 172.21.15.18:0/1049335454 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcf4005500 con 0x5645d65d7bd0 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efce37fe640 1 -- 172.21.15.18:0/1049335454 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcf80d1910 con 0x5645d65d7bd0 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd0d202740 10 client.28168 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.368+0000 7efd0d202740 1 -- 172.21.15.18:0/1049335454 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5645d64ff6a0 con 0x5645d65d7bd0 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efce37fe640 1 -- 172.21.15.18:0/1049335454 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcf4005500 con 0x5645d65d7bd0 2024-02-17T12:59:33.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efce37fe640 1 -- 172.21.15.18:0/1049335454 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7efcf801f040 con 0x5645d65d7bd0 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efd0d202740 10 client.28168 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efd0d202740 10 client.28168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efd0d202740 20 client.28168 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efd0d202740 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5645d6648430 0x5645d664a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efd0d202740 4 client.28168 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efd0d202740 1 -- 172.21.15.18:0/1049335454 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5645d62ca890 con 0x5645d6648430 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efd03fff640 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5645d6648430 0x5645d664a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efd03fff640 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5645d6648430 0x5645d664a850 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6588daf6e19d1f25 server_cookie=ea81bb742c6f895 in_seq=0 out_seq=0 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efce37fe640 10 client.28168 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efce37fe640 1 -- 172.21.15.18:0/1049335454 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5645d62ca890 con 0x5645d6648430 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efce37fe640 10 client.28168 handle_command_reply: tid=0 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.369+0000 7efd0d202740 1 client.28168 shutdown 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0d202740 1 -- 172.21.15.18:0/1049335454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65d7bd0 msgr2=0x5645d65e4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0d202740 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65d7bd0 0x5645d65e4590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efcf8097200 tx=0x7efcf800c7d0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0d202740 1 -- 172.21.15.18:0/1049335454 shutdown_connections 2024-02-17T12:59:33.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0d202740 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5645d6648430 0x5645d664a850 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0d202740 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645d65e07b0 0x5645d65e4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0d202740 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5645d65d7bd0 0x5645d65e4590 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0d202740 1 --2- 172.21.15.18:0/1049335454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5645d65d72b0 0x5645d65e4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efce37fe640 0 client.28168 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:33.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0d202740 1 -- 172.21.15.18:0/1049335454 >> 172.21.15.18:0/1049335454 conn(0x5645d65d1990 msgr2=0x5645d65d87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:33.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0d202740 1 -- 172.21.15.18:0/1049335454 shutdown_connections 2024-02-17T12:59:33.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0d202740 1 -- 172.21.15.18:0/1049335454 wait complete. 2024-02-17T12:59:33.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0d202740 20 client.28168 trim_cache size 0 max 16384 2024-02-17T12:59:33.368 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T12:59:33.368 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:59:33.368 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T12:59:33.368 INFO:teuthology.orchestra.run.smithi018.stdout: "4cfd9d80-267c-430c-af59-5cab3c547e5f": { 2024-02-17T12:59:33.368 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T12:59:33.368 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "4cfd9d80-267c-430c-af59-5cab3c547e5f", 2024-02-17T12:59:33.368 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T12:59:33.368 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:59:33.368 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T12:59:33.368 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T12:59:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0a54f640 1 -- 172.21.15.18:0/899406061 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efcec0a22b0 msgr2=0x7efcec0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0a54f640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efcec0a22b0 0x7efcec0a4770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efcfc004af0 tx=0x7efcfc0095c0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0a54f640 1 -- 172.21.15.18:0/899406061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd04102cc0 msgr2=0x7efd04197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0a54f640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd04102cc0 0x7efd04197c80 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7efcf800e9f0 tx=0x7efcf800edf0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0a54f640 1 -- 172.21.15.18:0/899406061 shutdown_connections 2024-02-17T12:59:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0a54f640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efcec0a22b0 0x7efcec0a4770 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0a54f640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd0410b6e0 0x7efd0419bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0a54f640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd04102cc0 0x7efd04197c80 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0a54f640 1 --2- 172.21.15.18:0/899406061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd041023a0 0x7efd04197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.370+0000 7efd0a54f640 1 -- 172.21.15.18:0/899406061 >> 172.21.15.18:0/899406061 conn(0x7efd04077230 msgr2=0x7efd040fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.371+0000 7efd0a54f640 1 -- 172.21.15.18:0/899406061 shutdown_connections 2024-02-17T12:59:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T12:59:33.371+0000 7efd0a54f640 1 -- 172.21.15.18:0/899406061 wait complete. 2024-02-17T12:59:33.371 INFO:tasks.cephfs.filesystem:scrub status for tag:4cfd9d80-267c-430c-af59-5cab3c547e5f - {'path': '/', 'tag': '4cfd9d80-267c-430c-af59-5cab3c547e5f', 'options': 'recursive,force'} 2024-02-17T12:59:34.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 490634 5.12 MB/sec execute 409 sec latency 13.786 ms 2024-02-17T12:59:34.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: pgmap v302: 97 pgs: 97 active+clean; 195 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-02-17T12:59:34.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T12:59:34.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:34.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:34.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:34.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:34.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:34.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:34.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T12:59:34.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:34 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: pgmap v302: 97 pgs: 97 active+clean; 195 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-02-17T12:59:34.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T12:59:34.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:34.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:34.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:34.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:34.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:34.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:34.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T12:59:34.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:34 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: pgmap v302: 97 pgs: 97 active+clean; 195 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-02-17T12:59:34.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T12:59:34.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:34.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:34 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T12:59:35.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 492000 5.12 MB/sec execute 410 sec latency 94.565 ms 2024-02-17T12:59:36.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 492000 5.10 MB/sec execute 411 sec latency 1094.613 ms 2024-02-17T12:59:36.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:36 smithi119 ceph-mon[36188]: pgmap v303: 97 pgs: 97 active+clean; 194 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-02-17T12:59:36.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:36 smithi146 ceph-mon[36454]: pgmap v303: 97 pgs: 97 active+clean; 194 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-02-17T12:59:36.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:36 smithi018 ceph-mon[32625]: pgmap v303: 97 pgs: 97 active+clean; 194 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-02-17T12:59:37.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 492000 5.09 MB/sec execute 412 sec latency 2094.709 ms 2024-02-17T12:59:38.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 492000 5.08 MB/sec execute 413 sec latency 3094.831 ms 2024-02-17T12:59:38.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:38 smithi119 ceph-mon[36188]: pgmap v304: 97 pgs: 97 active+clean; 194 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-02-17T12:59:38.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:38 smithi146 ceph-mon[36454]: pgmap v304: 97 pgs: 97 active+clean; 194 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-02-17T12:59:38.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:38 smithi018 ceph-mon[32625]: pgmap v304: 97 pgs: 97 active+clean; 194 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-02-17T12:59:39.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 493931 5.10 MB/sec execute 414 sec latency 3285.785 ms 2024-02-17T12:59:40.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 495282 5.10 MB/sec execute 415 sec latency 15.883 ms 2024-02-17T12:59:40.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:40 smithi119 ceph-mon[36188]: pgmap v305: 97 pgs: 97 active+clean; 196 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-02-17T12:59:40.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:40 smithi146 ceph-mon[36454]: pgmap v305: 97 pgs: 97 active+clean; 196 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-02-17T12:59:40.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:40 smithi018 ceph-mon[32625]: pgmap v305: 97 pgs: 97 active+clean; 196 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-02-17T12:59:41.292 INFO:tasks.workunit.client.0.smithi018.stdout: 1 495544 5.09 MB/sec execute 416 sec latency 825.124 ms 2024-02-17T12:59:42.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 495544 5.08 MB/sec execute 417 sec latency 1825.195 ms 2024-02-17T12:59:42.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:42 smithi119 ceph-mon[36188]: pgmap v306: 97 pgs: 97 active+clean; 196 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-02-17T12:59:42.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:42 smithi146 ceph-mon[36454]: pgmap v306: 97 pgs: 97 active+clean; 196 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-02-17T12:59:42.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:42 smithi018 ceph-mon[32625]: pgmap v306: 97 pgs: 97 active+clean; 196 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-02-17T12:59:43.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 495544 5.07 MB/sec execute 418 sec latency 2825.286 ms 2024-02-17T12:59:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.968+0000 7f421e977640 1 -- 172.21.15.18:0/387231114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4218106540 msgr2=0x7f421810e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.968+0000 7f421e977640 1 --2- 172.21.15.18:0/387231114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4218106540 0x7f421810e0a0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f4210009ca0 tx=0x7f421002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.968+0000 7f421e977640 1 -- 172.21.15.18:0/387231114 shutdown_connections 2024-02-17T12:59:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.968+0000 7f421e977640 1 --2- 172.21.15.18:0/387231114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4218106540 0x7f421810e0a0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.968+0000 7f421e977640 1 --2- 172.21.15.18:0/387231114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42181002e0 0x7f4218105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.968+0000 7f421e977640 1 --2- 172.21.15.18:0/387231114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42180ff930 0x7f42180ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.968+0000 7f421e977640 1 -- 172.21.15.18:0/387231114 >> 172.21.15.18:0/387231114 conn(0x7f4218076640 msgr2=0x7f42180fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.968+0000 7f421e977640 1 -- 172.21.15.18:0/387231114 shutdown_connections 2024-02-17T12:59:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.968+0000 7f421e977640 1 -- 172.21.15.18:0/387231114 wait complete. 2024-02-17T12:59:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.969+0000 7f421e977640 1 Processor -- start 2024-02-17T12:59:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.969+0000 7f421e977640 1 -- start start 2024-02-17T12:59:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421e977640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42180ff930 0x7f42181976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421e977640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4218106540 0x7f4218197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421e977640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f421819bfa0 0x7f421819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421e977640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421819c990 con 0x7f421819bfa0 2024-02-17T12:59:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421e977640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421819cb00 con 0x7f4218106540 2024-02-17T12:59:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421e977640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421819cc70 con 0x7f42180ff930 2024-02-17T12:59:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421cf14640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f421819bfa0 0x7f421819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f4217fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42180ff930 0x7f42181976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421cf14640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f421819bfa0 0x7f421819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36748/0 (socket says 172.21.15.18:36748) 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f4217fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42180ff930 0x7f42181976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:59226/0 (socket says 172.21.15.18:59226) 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f42177fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4218106540 0x7f4218197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421cf14640 1 -- 172.21.15.18:0/2414372296 learned_addr learned my addr 172.21.15.18:0/2414372296 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421cf14640 1 -- 172.21.15.18:0/2414372296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42180ff930 msgr2=0x7f42181976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421cf14640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42180ff930 0x7f42181976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421cf14640 1 -- 172.21.15.18:0/2414372296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4218106540 msgr2=0x7f4218197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421cf14640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4218106540 0x7f4218197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421cf14640 1 -- 172.21.15.18:0/2414372296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4210009900 con 0x7f421819bfa0 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f4217fff640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42180ff930 0x7f42181976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421cf14640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f421819bfa0 0x7f421819c450 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f421002c700 tx=0x7f4210034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f42157fa640 1 -- 172.21.15.18:0/2414372296 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42100328b0 con 0x7f421819bfa0 2024-02-17T12:59:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.970+0000 7f421e977640 1 -- 172.21.15.18:0/2414372296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42181a4250 con 0x7f421819bfa0 2024-02-17T12:59:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.971+0000 7f42157fa640 1 -- 172.21.15.18:0/2414372296 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4210003e50 con 0x7f421819bfa0 2024-02-17T12:59:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.971+0000 7f421e977640 1 -- 172.21.15.18:0/2414372296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42181a46d0 con 0x7f421819bfa0 2024-02-17T12:59:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.971+0000 7f42157fa640 1 -- 172.21.15.18:0/2414372296 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42100047f0 con 0x7f421819bfa0 2024-02-17T12:59:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.973+0000 7f42157fa640 1 -- 172.21.15.18:0/2414372296 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f421000a040 con 0x7f421819bfa0 2024-02-17T12:59:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.973+0000 7f4221621740 1 Processor -- start 2024-02-17T12:59:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.973+0000 7f4221621740 1 -- start start 2024-02-17T12:59:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.973+0000 7f4221621740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 0x560e7a276820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.973+0000 7f4221621740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a276d60 0x560e7a2771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.973+0000 7f4221621740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a2802e0 0x560e7a2826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.973+0000 7f4221621740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7a1e59f0 con 0x560e7a277ad0 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.973+0000 7f4221621740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7a1e2f20 con 0x560e7a276d60 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.973+0000 7f4221621740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7a1de700 con 0x560e7a2802e0 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.973+0000 7f42177fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 0x560e7a276820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f42177fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 0x560e7a276820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36758/0 (socket says 172.21.15.18:36758) 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f42177fe640 1 -- 172.21.15.18:0/237318260 learned_addr learned my addr 172.21.15.18:0/237318260 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f4217fff640 1 --2- 172.21.15.18:0/237318260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a276d60 0x560e7a2771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f42177fe640 1 --2- 172.21.15.18:0/237318260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a2802e0 0x560e7a2826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f42157fa640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42000a22d0 0x7f42000a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f4217fff640 1 -- 172.21.15.18:0/237318260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a2802e0 msgr2=0x560e7a2826d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f4217fff640 1 --2- 172.21.15.18:0/237318260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a2802e0 0x560e7a2826d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f4217fff640 1 -- 172.21.15.18:0/237318260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 msgr2=0x560e7a276820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f4217fff640 1 --2- 172.21.15.18:0/237318260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 0x560e7a276820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f4217fff640 1 -- 172.21.15.18:0/237318260 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e7a1747b0 con 0x560e7a276d60 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f421cf14640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42000a22d0 0x7f42000a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f4217fff640 1 --2- 172.21.15.18:0/237318260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a276d60 0x560e7a2771e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4208009fd0 tx=0x7f42080118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a808c2918e1b6c1c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f42157fa640 1 -- 172.21.15.18:0/2414372296 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f42100f07e0 con 0x7f421819bfa0 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f41fe7fc640 1 -- 172.21.15.18:0/237318260 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208004030 con 0x560e7a276d60 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f41fe7fc640 1 -- 172.21.15.18:0/237318260 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4208005500 con 0x560e7a276d60 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.974+0000 7f41fe7fc640 1 -- 172.21.15.18:0/237318260 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208004c40 con 0x560e7a276d60 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.975+0000 7f4221621740 1 -- 172.21.15.18:0/237318260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a276d60 msgr2=0x560e7a2771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.975+0000 7f4221621740 1 --2- 172.21.15.18:0/237318260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a276d60 0x560e7a2771e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4208009fd0 tx=0x7f42080118d0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.975+0000 7f421cf14640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42000a22d0 0x7f42000a4790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f42100b9810 tx=0x7f4210090420 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.975+0000 7f4221621740 1 -- 172.21.15.18:0/237318260 shutdown_connections 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.975+0000 7f4221621740 1 --2- 172.21.15.18:0/237318260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a2802e0 0x560e7a2826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.975+0000 7f4221621740 1 --2- 172.21.15.18:0/237318260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a276d60 0x560e7a2771e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.975+0000 7f4221621740 1 --2- 172.21.15.18:0/237318260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 0x560e7a276820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.975+0000 7f4221621740 1 -- 172.21.15.18:0/237318260 >> 172.21.15.18:0/237318260 conn(0x560e7a23e550 msgr2=0x560e7a271a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.975+0000 7f4221621740 1 -- 172.21.15.18:0/237318260 shutdown_connections 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.975+0000 7f4221621740 1 -- 172.21.15.18:0/237318260 wait complete. 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f4221621740 1 -- start start 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f4221621740 1 Processor -- start 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f4221621740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 0x560e7a27d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f4221621740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a277ad0 0x560e7a27d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f4221621740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 0x560e7a27dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f4221621740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7a183bf0 con 0x560e7a2802e0 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f4221621740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e79eb2250 con 0x560e7a277ad0 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f4221621740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7a164200 con 0x560e7a276d60 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f4221621740 10 client.0 _ll_register_callbacks cb 0x560e7a1d2d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f42177fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 0x560e7a27dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f4217fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a277ad0 0x560e7a27d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f42177fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 0x560e7a27dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36770/0 (socket says 172.21.15.18:36770) 2024-02-17T12:59:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f42177fe640 1 -- 172.21.15.18:0/2370449830 learned_addr learned my addr 172.21.15.18:0/2370449830 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.976+0000 7f42177fe640 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 0x560e7a27d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.977+0000 7f4217fff640 1 -- 172.21.15.18:0/2370449830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 msgr2=0x560e7a27d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.977+0000 7f4217fff640 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 0x560e7a27d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.977+0000 7f4217fff640 1 -- 172.21.15.18:0/2370449830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 msgr2=0x560e7a27dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.977+0000 7f4217fff640 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 0x560e7a27dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.977+0000 7f4217fff640 1 -- 172.21.15.18:0/2370449830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4208009c00 con 0x560e7a277ad0 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.977+0000 7f42177fe640 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 0x560e7a27dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.977+0000 7f4217fff640 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a277ad0 0x560e7a27d7d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4208011dd0 tx=0x7f420800fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ad73df746efa85b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.977+0000 7f41f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.977+0000 7f41f77fe640 1 -- 172.21.15.18:0/2370449830 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208005670 con 0x560e7a277ad0 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.977+0000 7f41f77fe640 1 -- 172.21.15.18:0/2370449830 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4208025470 con 0x560e7a277ad0 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.977+0000 7f4221621740 1 -- 172.21.15.18:0/2370449830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e79ede600 con 0x560e7a277ad0 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.978+0000 7f41f77fe640 1 -- 172.21.15.18:0/2370449830 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4208025630 con 0x560e7a277ad0 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.978+0000 7f4221621740 10 client.28142 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.978+0000 7f4221621740 1 -- 172.21.15.18:0/2370449830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x560e7a1dcd80 con 0x560e7a277ad0 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.978+0000 7f41f77fe640 1 -- 172.21.15.18:0/2370449830 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560e79ede600 con 0x560e7a277ad0 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.978+0000 7f41f77fe640 1 -- 172.21.15.18:0/2370449830 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4208021040 con 0x560e7a277ad0 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.978+0000 7f4221621740 10 client.28142 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.978+0000 7f4221621740 10 client.28142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.978+0000 7f4221621740 20 client.28142 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.978+0000 7f4221621740 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560e7a290670 0x560e7a292a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.978+0000 7f4221621740 4 client.28142 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.978+0000 7f4221621740 1 -- 172.21.15.18:0/2370449830 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e79f2f1b0 con 0x560e7a290670 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.978+0000 7f42177fe640 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560e7a290670 0x560e7a292a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.979+0000 7f41f77fe640 10 client.28142 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.979+0000 7f42177fe640 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560e7a290670 0x560e7a292a90 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231e3a2a9e81371b server_cookie=a5e65774d3585af in_seq=0 out_seq=0 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.979+0000 7f41f77fe640 1 -- 172.21.15.18:0/2370449830 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e79f2f1b0 con 0x560e7a290670 2024-02-17T12:59:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.979+0000 7f41f77fe640 10 client.28142 handle_command_reply: tid=0 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.979+0000 7f4221621740 1 client.28142 shutdown 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.979+0000 7f4221621740 1 -- 172.21.15.18:0/2370449830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a277ad0 msgr2=0x560e7a27d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.979+0000 7f4221621740 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a277ad0 0x560e7a27d7d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4208011dd0 tx=0x7f420800fc30 comp rx=0 tx=0).stop 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.980+0000 7f4221621740 1 -- 172.21.15.18:0/2370449830 shutdown_connections 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.980+0000 7f4221621740 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560e7a290670 0x560e7a292a90 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.980+0000 7f4221621740 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 0x560e7a27dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.980+0000 7f4221621740 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a277ad0 0x560e7a27d7d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.980+0000 7f41f77fe640 0 client.28142 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.980+0000 7f4221621740 1 --2- 172.21.15.18:0/2370449830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 0x560e7a27d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.980+0000 7f4221621740 1 -- 172.21.15.18:0/2370449830 >> 172.21.15.18:0/2370449830 conn(0x560e7a23e550 msgr2=0x560e7a2737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.980+0000 7f4221621740 1 -- 172.21.15.18:0/2370449830 shutdown_connections 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.980+0000 7f4221621740 1 -- 172.21.15.18:0/2370449830 wait complete. 2024-02-17T12:59:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.980+0000 7f4221621740 20 client.28142 trim_cache size 0 max 16384 2024-02-17T12:59:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.994+0000 7f4221621740 1 Processor -- start 2024-02-17T12:59:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.994+0000 7f4221621740 1 -- start start 2024-02-17T12:59:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.994+0000 7f4221621740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 0x560e7a27f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.994+0000 7f4221621740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 0x560e7a276280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.994+0000 7f4221621740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a2802e0 0x560e7a287100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.994+0000 7f4221621740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7a272850 con 0x560e7a277ad0 2024-02-17T12:59:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.994+0000 7f4221621740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7a27e260 con 0x560e7a2802e0 2024-02-17T12:59:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.994+0000 7f4221621740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7a276030 con 0x560e7a276d60 2024-02-17T12:59:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f4217fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 0x560e7a276280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f4217fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 0x560e7a276280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36780/0 (socket says 172.21.15.18:36780) 2024-02-17T12:59:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f4217fff640 1 -- 172.21.15.18:0/4269752662 learned_addr learned my addr 172.21.15.18:0/4269752662 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f42177fe640 1 --2- 172.21.15.18:0/4269752662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 0x560e7a27f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f42177fe640 1 --2- 172.21.15.18:0/4269752662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a2802e0 0x560e7a287100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f4217fff640 1 -- 172.21.15.18:0/4269752662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 msgr2=0x560e7a27f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f4217fff640 1 --2- 172.21.15.18:0/4269752662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 0x560e7a27f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f4217fff640 1 -- 172.21.15.18:0/4269752662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a2802e0 msgr2=0x560e7a287100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f4217fff640 1 --2- 172.21.15.18:0/4269752662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a2802e0 0x560e7a287100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f4217fff640 1 -- 172.21.15.18:0/4269752662 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e7a1797b0 con 0x560e7a277ad0 2024-02-17T12:59:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f4217fff640 1 --2- 172.21.15.18:0/4269752662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 0x560e7a276280 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f4208009720 tx=0x7f420800f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3541a5ddfeb29c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f41fdffb640 1 -- 172.21.15.18:0/4269752662 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420800fca0 con 0x560e7a277ad0 2024-02-17T12:59:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f41fdffb640 1 -- 172.21.15.18:0/4269752662 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4208025470 con 0x560e7a277ad0 2024-02-17T12:59:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.995+0000 7f41fdffb640 1 -- 172.21.15.18:0/4269752662 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42080196b0 con 0x560e7a277ad0 2024-02-17T12:59:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.996+0000 7f4221621740 1 -- 172.21.15.18:0/4269752662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 msgr2=0x560e7a276280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.996+0000 7f4221621740 1 --2- 172.21.15.18:0/4269752662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 0x560e7a276280 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f4208009720 tx=0x7f420800f8d0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.996+0000 7f4221621740 1 -- 172.21.15.18:0/4269752662 shutdown_connections 2024-02-17T12:59:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.996+0000 7f4221621740 1 --2- 172.21.15.18:0/4269752662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a2802e0 0x560e7a287100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.996+0000 7f4221621740 1 --2- 172.21.15.18:0/4269752662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a277ad0 0x560e7a276280 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.996+0000 7f4221621740 1 --2- 172.21.15.18:0/4269752662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 0x560e7a27f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.996+0000 7f4221621740 1 -- 172.21.15.18:0/4269752662 >> 172.21.15.18:0/4269752662 conn(0x560e7a23e550 msgr2=0x560e7a26cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.996+0000 7f4221621740 1 -- 172.21.15.18:0/4269752662 shutdown_connections 2024-02-17T12:59:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.996+0000 7f4221621740 1 -- 172.21.15.18:0/4269752662 wait complete. 2024-02-17T12:59:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.996+0000 7f4221621740 1 -- start start 2024-02-17T12:59:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f4221621740 1 Processor -- start 2024-02-17T12:59:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f4221621740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 0x560e7a287110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f4221621740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a277ad0 0x560e7a287650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f4221621740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 0x560e7a279640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f4221621740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7a279d10 con 0x560e7a2802e0 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f4221621740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7a279e80 con 0x560e7a277ad0 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f4221621740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7a27a040 con 0x560e7a276d60 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f4221621740 10 client.0 _ll_register_callbacks cb 0x7f4208009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f42177fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 0x560e7a279640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f42177fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 0x560e7a279640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36794/0 (socket says 172.21.15.18:36794) 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f42177fe640 1 -- 172.21.15.18:0/1758291388 learned_addr learned my addr 172.21.15.18:0/1758291388 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f42177fe640 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 0x560e7a287110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f4217fff640 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a277ad0 0x560e7a287650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f42177fe640 1 -- 172.21.15.18:0/1758291388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 msgr2=0x560e7a287110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f42177fe640 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 0x560e7a287110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f42177fe640 1 -- 172.21.15.18:0/1758291388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a277ad0 msgr2=0x560e7a287650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f42177fe640 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a277ad0 0x560e7a287650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.997+0000 7f42177fe640 1 -- 172.21.15.18:0/1758291388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e7a1dcd80 con 0x560e7a2802e0 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f42177fe640 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 0x560e7a279640 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f420c000c00 tx=0x7f420c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2328b27b7d197190 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f41f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f41f77fe640 1 -- 172.21.15.18:0/1758291388 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c02c040 con 0x560e7a2802e0 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f41f77fe640 1 -- 172.21.15.18:0/1758291388 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f420c0260d0 con 0x560e7a2802e0 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f4221621740 1 -- 172.21.15.18:0/1758291388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4208005500 con 0x560e7a2802e0 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f41f77fe640 1 -- 172.21.15.18:0/1758291388 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f420c026290 con 0x560e7a2802e0 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f4221621740 10 client.19101 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f4221621740 1 -- 172.21.15.18:0/1758291388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x560e7a19f540 con 0x560e7a2802e0 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f41f77fe640 1 -- 172.21.15.18:0/1758291388 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4208005500 con 0x560e7a2802e0 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f41f77fe640 1 -- 172.21.15.18:0/1758291388 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f420c003cc0 con 0x560e7a2802e0 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f4221621740 10 client.19101 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f4221621740 10 client.19101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f4221621740 20 client.19101 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f4221621740 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560e7a2e7dc0 0x560e7a2ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.998+0000 7f4221621740 4 client.19101 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:59:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.999+0000 7f4221621740 1 -- 172.21.15.18:0/1758291388 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560e79f1b550 con 0x560e7a2e7dc0 2024-02-17T12:59:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.999+0000 7f4217fff640 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560e7a2e7dc0 0x560e7a2ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.999+0000 7f41f77fe640 10 client.19101 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:43.999+0000 7f4217fff640 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560e7a2e7dc0 0x560e7a2ea1e0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884d0b1ed2fb3d8 server_cookie=d672795411307abd in_seq=0 out_seq=0 2024-02-17T12:59:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.038+0000 7f41f77fe640 1 -- 172.21.15.18:0/1758291388 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560e79f1b550 con 0x560e7a2e7dc0 2024-02-17T12:59:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f41f77fe640 10 client.19101 handle_command_reply: tid=0 2024-02-17T12:59:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f4221621740 1 client.19101 shutdown 2024-02-17T12:59:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f4221621740 1 -- 172.21.15.18:0/1758291388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 msgr2=0x560e7a279640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f4221621740 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 0x560e7a279640 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f420c000c00 tx=0x7f420c004730 comp rx=0 tx=0).stop 2024-02-17T12:59:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f4221621740 1 -- 172.21.15.18:0/1758291388 shutdown_connections 2024-02-17T12:59:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f4221621740 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x560e7a2e7dc0 0x560e7a2ea1e0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f4221621740 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e7a2802e0 0x560e7a279640 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f4221621740 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560e7a277ad0 0x560e7a287650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f4221621740 1 --2- 172.21.15.18:0/1758291388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e7a276d60 0x560e7a287110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f4221621740 1 -- 172.21.15.18:0/1758291388 >> 172.21.15.18:0/1758291388 conn(0x560e7a23e550 msgr2=0x560e7a271580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f41f77fe640 0 client.19101 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f4221621740 1 -- 172.21.15.18:0/1758291388 shutdown_connections 2024-02-17T12:59:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f4221621740 1 -- 172.21.15.18:0/1758291388 wait complete. 2024-02-17T12:59:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.039+0000 7f4221621740 20 client.19101 trim_cache size 0 max 16384 2024-02-17T12:59:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:59:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:59:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:59:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:59:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.040+0000 7f421e977640 1 -- 172.21.15.18:0/2414372296 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42000a22d0 msgr2=0x7f42000a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.040+0000 7f421e977640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42000a22d0 0x7f42000a4790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f42100b9810 tx=0x7f4210090420 comp rx=0 tx=0).stop 2024-02-17T12:59:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.040+0000 7f421e977640 1 -- 172.21.15.18:0/2414372296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f421819bfa0 msgr2=0x7f421819c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.040+0000 7f421e977640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f421819bfa0 0x7f421819c450 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f421002c700 tx=0x7f4210034a90 comp rx=0 tx=0).stop 2024-02-17T12:59:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.040+0000 7f421e977640 1 -- 172.21.15.18:0/2414372296 shutdown_connections 2024-02-17T12:59:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.040+0000 7f421e977640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f42000a22d0 0x7f42000a4790 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.040+0000 7f421e977640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f421819bfa0 0x7f421819c450 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.040+0000 7f421e977640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4218106540 0x7f4218197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.040+0000 7f421e977640 1 --2- 172.21.15.18:0/2414372296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42180ff930 0x7f42181976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.040+0000 7f421e977640 1 -- 172.21.15.18:0/2414372296 >> 172.21.15.18:0/2414372296 conn(0x7f4218076640 msgr2=0x7f4218104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.040+0000 7f421e977640 1 -- 172.21.15.18:0/2414372296 shutdown_connections 2024-02-17T12:59:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:44.040+0000 7f421e977640 1 -- 172.21.15.18:0/2414372296 wait complete. 2024-02-17T12:59:44.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 496427 5.06 MB/sec execute 419 sec latency 3431.380 ms 2024-02-17T12:59:44.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:44 smithi119 ceph-mon[36188]: pgmap v307: 97 pgs: 97 active+clean; 196 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-02-17T12:59:44.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:44 smithi146 ceph-mon[36454]: pgmap v307: 97 pgs: 97 active+clean; 196 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-02-17T12:59:44.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:44 smithi018 ceph-mon[32625]: pgmap v307: 97 pgs: 97 active+clean; 196 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-02-17T12:59:45.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 498002 5.07 MB/sec execute 420 sec latency 27.418 ms 2024-02-17T12:59:45.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:45 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:59:45.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:45 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:59:45.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:45 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:59:45.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:45 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:59:45.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:45 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:59:45.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:45 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:59:46.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 499092 5.07 MB/sec execute 421 sec latency 420.236 ms 2024-02-17T12:59:46.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:46 smithi119 ceph-mon[36188]: pgmap v308: 97 pgs: 97 active+clean; 199 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-02-17T12:59:46.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:46 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:59:46.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:46 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:59:46.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:46 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T12:59:46.423 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:46 smithi018 ceph-mon[32625]: pgmap v308: 97 pgs: 97 active+clean; 199 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-02-17T12:59:46.423 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:46 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:59:46.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:46 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:59:46.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:46 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T12:59:46.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:46 smithi146 ceph-mon[36454]: pgmap v308: 97 pgs: 97 active+clean; 199 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-02-17T12:59:46.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:46 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:59:46.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:46 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:59:46.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:46 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T12:59:47.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 501022 5.09 MB/sec execute 422 sec latency 651.556 ms 2024-02-17T12:59:48.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 502434 5.10 MB/sec execute 423 sec latency 21.734 ms 2024-02-17T12:59:48.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:48 smithi119 ceph-mon[36188]: pgmap v309: 97 pgs: 97 active+clean; 199 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-02-17T12:59:48.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:48 smithi146 ceph-mon[36454]: pgmap v309: 97 pgs: 97 active+clean; 199 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-02-17T12:59:48.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:48 smithi018 ceph-mon[32625]: pgmap v309: 97 pgs: 97 active+clean; 199 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-02-17T12:59:49.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 503524 5.09 MB/sec execute 424 sec latency 464.762 ms 2024-02-17T12:59:50.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 505281 5.11 MB/sec execute 425 sec latency 11.832 ms 2024-02-17T12:59:50.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:50 smithi119 ceph-mon[36188]: pgmap v310: 97 pgs: 97 active+clean; 199 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.0 MiB/s wr, 211 op/s 2024-02-17T12:59:50.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:50 smithi146 ceph-mon[36454]: pgmap v310: 97 pgs: 97 active+clean; 199 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.0 MiB/s wr, 211 op/s 2024-02-17T12:59:50.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:50 smithi018 ceph-mon[32625]: pgmap v310: 97 pgs: 97 active+clean; 199 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.0 MiB/s wr, 211 op/s 2024-02-17T12:59:51.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 506183 5.10 MB/sec execute 426 sec latency 519.549 ms 2024-02-17T12:59:52.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 506183 5.09 MB/sec execute 427 sec latency 1519.613 ms 2024-02-17T12:59:52.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:52 smithi119 ceph-mon[36188]: pgmap v311: 97 pgs: 97 active+clean; 199 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.5 MiB/s wr, 159 op/s 2024-02-17T12:59:52.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:52 smithi146 ceph-mon[36454]: pgmap v311: 97 pgs: 97 active+clean; 199 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.5 MiB/s wr, 159 op/s 2024-02-17T12:59:52.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:52 smithi018 ceph-mon[32625]: pgmap v311: 97 pgs: 97 active+clean; 199 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.5 MiB/s wr, 159 op/s 2024-02-17T12:59:53.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 507086 5.09 MB/sec execute 428 sec latency 2082.805 ms 2024-02-17T12:59:54.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 508694 5.10 MB/sec execute 429 sec latency 19.103 ms 2024-02-17T12:59:54.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:54 smithi119 ceph-mon[36188]: pgmap v312: 97 pgs: 97 active+clean; 199 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.5 MiB/s wr, 159 op/s 2024-02-17T12:59:54.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:54 smithi146 ceph-mon[36454]: pgmap v312: 97 pgs: 97 active+clean; 199 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.5 MiB/s wr, 159 op/s 2024-02-17T12:59:54.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:54 smithi018 ceph-mon[32625]: pgmap v312: 97 pgs: 97 active+clean; 199 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.5 MiB/s wr, 159 op/s 2024-02-17T12:59:55.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 509726 5.10 MB/sec execute 430 sec latency 380.259 ms 2024-02-17T12:59:56.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 509726 5.09 MB/sec execute 431 sec latency 1380.381 ms 2024-02-17T12:59:56.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:56 smithi119 ceph-mon[36188]: pgmap v313: 97 pgs: 97 active+clean; 206 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 230 op/s 2024-02-17T12:59:56.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:56 smithi146 ceph-mon[36454]: pgmap v313: 97 pgs: 97 active+clean; 206 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 230 op/s 2024-02-17T12:59:56.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:56 smithi018 ceph-mon[32625]: pgmap v313: 97 pgs: 97 active+clean; 206 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 230 op/s 2024-02-17T12:59:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.142+0000 7f35ccedd640 1 -- 172.21.15.18:0/448079747 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c8107270 msgr2=0x7f35c8107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.142+0000 7f35ccedd640 1 --2- 172.21.15.18:0/448079747 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c8107270 0x7f35c8107650 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009900 tx=0x7f35b802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.142+0000 7f35ccedd640 1 -- 172.21.15.18:0/448079747 shutdown_connections 2024-02-17T12:59:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.142+0000 7f35ccedd640 1 --2- 172.21.15.18:0/448079747 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35c80691a0 0x7f35c810e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.142+0000 7f35ccedd640 1 --2- 172.21.15.18:0/448079747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35c8068800 0x7f35c8068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.142+0000 7f35ccedd640 1 --2- 172.21.15.18:0/448079747 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c8107270 0x7f35c8107650 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.142+0000 7f35ccedd640 1 -- 172.21.15.18:0/448079747 >> 172.21.15.18:0/448079747 conn(0x7f35c8076ed0 msgr2=0x7f35c80772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.142+0000 7f35ccedd640 1 -- 172.21.15.18:0/448079747 shutdown_connections 2024-02-17T12:59:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.142+0000 7f35ccedd640 1 -- 172.21.15.18:0/448079747 wait complete. 2024-02-17T12:59:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.143+0000 7f35ccedd640 1 Processor -- start 2024-02-17T12:59:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.143+0000 7f35ccedd640 1 -- start start 2024-02-17T12:59:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.143+0000 7f35ccedd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35c8068800 0x7f35c81003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.143+0000 7f35ccedd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c80691a0 0x7f35c81008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.143+0000 7f35ccedd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35c8107270 0x7f35c8100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.143+0000 7f35ccedd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c81a5a20 con 0x7f35c80691a0 2024-02-17T12:59:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.143+0000 7f35ccedd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c81a5b90 con 0x7f35c8068800 2024-02-17T12:59:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.143+0000 7f35ccedd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c81a5d50 con 0x7f35c8107270 2024-02-17T12:59:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35bdd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c80691a0 0x7f35c81008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35bdd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c80691a0 0x7f35c81008e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46954/0 (socket says 172.21.15.18:46954) 2024-02-17T12:59:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35bdd9b640 1 -- 172.21.15.18:0/48381662 learned_addr learned my addr 172.21.15.18:0/48381662 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35c659c640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35c8068800 0x7f35c81003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35c6d9d640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35c8107270 0x7f35c8100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35bdd9b640 1 -- 172.21.15.18:0/48381662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35c8107270 msgr2=0x7f35c8100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35bdd9b640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35c8107270 0x7f35c8100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35bdd9b640 1 -- 172.21.15.18:0/48381662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35c8068800 msgr2=0x7f35c81003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35bdd9b640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35c8068800 0x7f35c81003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35bdd9b640 1 -- 172.21.15.18:0/48381662 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b8009560 con 0x7f35c80691a0 2024-02-17T12:59:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35c659c640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35c8068800 0x7f35c81003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35bdd9b640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c80691a0 0x7f35c81008e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f35b000e9f0 tx=0x7f35b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35bffff640 1 -- 172.21.15.18:0/48381662 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0009700 con 0x7f35c80691a0 2024-02-17T12:59:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35bffff640 1 -- 172.21.15.18:0/48381662 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b000cd80 con 0x7f35c80691a0 2024-02-17T12:59:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35ccedd640 1 -- 172.21.15.18:0/48381662 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c81a5fd0 con 0x7f35c80691a0 2024-02-17T12:59:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35bffff640 1 -- 172.21.15.18:0/48381662 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0018920 con 0x7f35c80691a0 2024-02-17T12:59:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.144+0000 7f35ccedd640 1 -- 172.21.15.18:0/48381662 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c81a64b0 con 0x7f35c80691a0 2024-02-17T12:59:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.147+0000 7f35bffff640 1 -- 172.21.15.18:0/48381662 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f35b0018a80 con 0x7f35c80691a0 2024-02-17T12:59:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.147+0000 7f35cfb81740 1 Processor -- start 2024-02-17T12:59:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.147+0000 7f35cfb81740 1 -- start start 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35cfb81740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e51aad0 0x55dd7e519820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35cfb81740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e519d60 0x55dd7e51a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35cfb81740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 0x55dd7e5256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35cfb81740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7e4889f0 con 0x55dd7e5232e0 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35cfb81740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7e485f20 con 0x55dd7e51aad0 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35cfb81740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7e481700 con 0x55dd7e519d60 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35bdd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 0x55dd7e5256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35bffff640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f35b40a22d0 0x7f35b40a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35bdd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 0x55dd7e5256d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46966/0 (socket says 172.21.15.18:46966) 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35bdd9b640 1 -- 172.21.15.18:0/3012171589 learned_addr learned my addr 172.21.15.18:0/3012171589 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35c6d9d640 1 --2- 172.21.15.18:0/3012171589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e51aad0 0x55dd7e519820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35c659c640 1 --2- 172.21.15.18:0/3012171589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e519d60 0x55dd7e51a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35c6d9d640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f35b40a22d0 0x7f35b40a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35bffff640 1 -- 172.21.15.18:0/48381662 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f35b0014070 con 0x7f35c80691a0 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35bdd9b640 1 -- 172.21.15.18:0/3012171589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e519d60 msgr2=0x55dd7e51a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35bdd9b640 1 --2- 172.21.15.18:0/3012171589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e519d60 0x55dd7e51a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35bdd9b640 1 -- 172.21.15.18:0/3012171589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e51aad0 msgr2=0x55dd7e519820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35bdd9b640 1 --2- 172.21.15.18:0/3012171589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e51aad0 0x55dd7e519820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35bdd9b640 1 -- 172.21.15.18:0/3012171589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd7e4177b0 con 0x55dd7e5232e0 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.148+0000 7f35c6d9d640 1 --2- 172.21.15.18:0/3012171589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e51aad0 0x55dd7e519820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.149+0000 7f35bdd9b640 1 --2- 172.21.15.18:0/3012171589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 0x55dd7e5256d0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f35b000c6a0 tx=0x7f35b001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12918300d419b5fd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.149+0000 7f35c6d9d640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f35b40a22d0 0x7f35b40a4790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f35c0005cc0 tx=0x7f35c0004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.149+0000 7f35bcd99640 1 -- 172.21.15.18:0/3012171589 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b009f000 con 0x55dd7e5232e0 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.149+0000 7f35bcd99640 1 -- 172.21.15.18:0/3012171589 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b00a3650 con 0x55dd7e5232e0 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.149+0000 7f35bcd99640 1 -- 172.21.15.18:0/3012171589 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b009a300 con 0x55dd7e5232e0 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.149+0000 7f35cfb81740 1 -- 172.21.15.18:0/3012171589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 msgr2=0x55dd7e5256d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 --2- 172.21.15.18:0/3012171589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 0x55dd7e5256d0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f35b000c6a0 tx=0x7f35b001f000 comp rx=0 tx=0).stop 2024-02-17T12:59:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 -- 172.21.15.18:0/3012171589 shutdown_connections 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 --2- 172.21.15.18:0/3012171589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 0x55dd7e5256d0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 --2- 172.21.15.18:0/3012171589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e519d60 0x55dd7e51a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 --2- 172.21.15.18:0/3012171589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e51aad0 0x55dd7e519820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 -- 172.21.15.18:0/3012171589 >> 172.21.15.18:0/3012171589 conn(0x55dd7e4e1550 msgr2=0x55dd7e514a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 -- 172.21.15.18:0/3012171589 shutdown_connections 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 -- 172.21.15.18:0/3012171589 wait complete. 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 -- start start 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 Processor -- start 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e519d60 0x55dd7e520340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e51aad0 0x55dd7e520880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 0x55dd7e520dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7e426bf0 con 0x55dd7e5232e0 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7e155250 con 0x55dd7e519d60 2024-02-17T12:59:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7e407200 con 0x55dd7e51aad0 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35cfb81740 10 client.0 _ll_register_callbacks cb 0x55dd7e475d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35c6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 0x55dd7e520dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35c6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 0x55dd7e520dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46976/0 (socket says 172.21.15.18:46976) 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35c6d9d640 1 -- 172.21.15.18:0/3628704397 learned_addr learned my addr 172.21.15.18:0/3628704397 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.150+0000 7f35c659c640 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e519d60 0x55dd7e520340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35bdd9b640 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e51aad0 0x55dd7e520880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35c6d9d640 1 -- 172.21.15.18:0/3628704397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e51aad0 msgr2=0x55dd7e520880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35c6d9d640 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e51aad0 0x55dd7e520880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35c6d9d640 1 -- 172.21.15.18:0/3628704397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e519d60 msgr2=0x55dd7e520340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35c6d9d640 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e519d60 0x55dd7e520340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35c6d9d640 1 -- 172.21.15.18:0/3628704397 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c81a64b0 con 0x55dd7e5232e0 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35bdd9b640 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e51aad0 0x55dd7e520880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:59:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35c6d9d640 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 0x55dd7e520dc0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f35c0010df0 tx=0x7f35c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdc78eedfa69b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35adffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35adffb640 1 -- 172.21.15.18:0/3628704397 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0018070 con 0x55dd7e5232e0 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35adffb640 1 -- 172.21.15.18:0/3628704397 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f35c0004b20 con 0x55dd7e5232e0 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35cfb81740 1 -- 172.21.15.18:0/3628704397 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd7e181600 con 0x55dd7e5232e0 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35adffb640 1 -- 172.21.15.18:0/3628704397 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35c0004ce0 con 0x55dd7e5232e0 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35cfb81740 10 client.19113 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.151+0000 7f35cfb81740 1 -- 172.21.15.18:0/3628704397 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55dd7e47fd80 con 0x55dd7e5232e0 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.152+0000 7f35adffb640 1 -- 172.21.15.18:0/3628704397 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dd7e181600 con 0x55dd7e5232e0 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.152+0000 7f35adffb640 1 -- 172.21.15.18:0/3628704397 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f35c0025030 con 0x55dd7e5232e0 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.152+0000 7f35cfb81740 10 client.19113 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.152+0000 7f35cfb81740 10 client.19113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.152+0000 7f35cfb81740 20 client.19113 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.152+0000 7f35cfb81740 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55dd7e533730 0x55dd7e535b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.152+0000 7f35cfb81740 4 client.19113 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.152+0000 7f35cfb81740 1 -- 172.21.15.18:0/3628704397 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd7e1d21b0 con 0x55dd7e533730 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.152+0000 7f35c659c640 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55dd7e533730 0x55dd7e535b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35c659c640 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55dd7e533730 0x55dd7e535b50 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad8399827fd8306 server_cookie=f1393cac849b98ef in_seq=0 out_seq=0 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35adffb640 10 client.19113 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35adffb640 1 -- 172.21.15.18:0/3628704397 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd7e1d21b0 con 0x55dd7e533730 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35adffb640 10 client.19113 handle_command_reply: tid=0 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35cfb81740 1 client.19113 shutdown 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35cfb81740 1 -- 172.21.15.18:0/3628704397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 msgr2=0x55dd7e520dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35cfb81740 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 0x55dd7e520dc0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f35c0010df0 tx=0x7f35c0004ab0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35cfb81740 1 -- 172.21.15.18:0/3628704397 shutdown_connections 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35cfb81740 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55dd7e533730 0x55dd7e535b50 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35cfb81740 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e5232e0 0x55dd7e520dc0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35cfb81740 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e51aad0 0x55dd7e520880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35cfb81740 1 --2- 172.21.15.18:0/3628704397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e519d60 0x55dd7e520340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35cfb81740 1 -- 172.21.15.18:0/3628704397 >> 172.21.15.18:0/3628704397 conn(0x55dd7e4e1550 msgr2=0x55dd7e5167a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35adffb640 0 client.19113 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35cfb81740 1 -- 172.21.15.18:0/3628704397 shutdown_connections 2024-02-17T12:59:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.153+0000 7f35cfb81740 1 -- 172.21.15.18:0/3628704397 wait complete. 2024-02-17T12:59:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.154+0000 7f35cfb81740 20 client.19113 trim_cache size 0 max 16384 2024-02-17T12:59:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35cfb81740 1 Processor -- start 2024-02-17T12:59:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35cfb81740 1 -- start start 2024-02-17T12:59:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35cfb81740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 0x55dd7e522a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35cfb81740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e519d60 0x55dd7e520be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35cfb81740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e51aad0 0x55dd7e52a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35cfb81740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7e52f0c0 con 0x55dd7e4e1550 2024-02-17T12:59:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35cfb81740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7e519030 con 0x55dd7e51aad0 2024-02-17T12:59:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35cfb81740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7e515620 con 0x55dd7e519d60 2024-02-17T12:59:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35c659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 0x55dd7e522a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35c659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 0x55dd7e522a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46990/0 (socket says 172.21.15.18:46990) 2024-02-17T12:59:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35c659c640 1 -- 172.21.15.18:0/1672681024 learned_addr learned my addr 172.21.15.18:0/1672681024 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35bdd9b640 1 --2- 172.21.15.18:0/1672681024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e519d60 0x55dd7e520be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35c6d9d640 1 --2- 172.21.15.18:0/1672681024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e51aad0 0x55dd7e52a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35c659c640 1 -- 172.21.15.18:0/1672681024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e519d60 msgr2=0x55dd7e520be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35c659c640 1 --2- 172.21.15.18:0/1672681024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e519d60 0x55dd7e520be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35c659c640 1 -- 172.21.15.18:0/1672681024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e51aad0 msgr2=0x55dd7e52a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35c659c640 1 --2- 172.21.15.18:0/1672681024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e51aad0 0x55dd7e52a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35c659c640 1 -- 172.21.15.18:0/1672681024 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd7e41c7b0 con 0x55dd7e4e1550 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.167+0000 7f35c659c640 1 --2- 172.21.15.18:0/1672681024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 0x55dd7e522a70 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009900 tx=0x7f35b80041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199e3f8f624529e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35affff640 1 -- 172.21.15.18:0/1672681024 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8045040 con 0x55dd7e4e1550 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35affff640 1 -- 172.21.15.18:0/1672681024 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b8050070 con 0x55dd7e4e1550 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35affff640 1 -- 172.21.15.18:0/1672681024 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8004df0 con 0x55dd7e4e1550 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35cfb81740 1 -- 172.21.15.18:0/1672681024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 msgr2=0x55dd7e522a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35cfb81740 1 --2- 172.21.15.18:0/1672681024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 0x55dd7e522a70 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009900 tx=0x7f35b80041a0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35cfb81740 1 -- 172.21.15.18:0/1672681024 shutdown_connections 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35cfb81740 1 --2- 172.21.15.18:0/1672681024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e51aad0 0x55dd7e52a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35cfb81740 1 --2- 172.21.15.18:0/1672681024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e519d60 0x55dd7e520be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35cfb81740 1 --2- 172.21.15.18:0/1672681024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 0x55dd7e522a70 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35cfb81740 1 -- 172.21.15.18:0/1672681024 >> 172.21.15.18:0/1672681024 conn(0x55dd7e5258b0 msgr2=0x55dd7e514580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35cfb81740 1 -- 172.21.15.18:0/1672681024 shutdown_connections 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35cfb81740 1 -- 172.21.15.18:0/1672681024 wait complete. 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35cfb81740 1 -- start start 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.168+0000 7f35cfb81740 1 Processor -- start 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35cfb81740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 0x55dd7e52a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35cfb81740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e519d60 0x55dd7e51cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35cfb81740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e51aad0 0x55dd7e51d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35cfb81740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7e51d650 con 0x55dd7e4e1550 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35cfb81740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7e51d7c0 con 0x55dd7e519d60 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35cfb81740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd7e51d980 con 0x55dd7e51aad0 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35cfb81740 10 client.0 _ll_register_callbacks cb 0x7f35b009ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35c659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 0x55dd7e52a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35c659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 0x55dd7e52a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47004/0 (socket says 172.21.15.18:47004) 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35c659c640 1 -- 172.21.15.18:0/2215419075 learned_addr learned my addr 172.21.15.18:0/2215419075 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T12:59:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35bdd9b640 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e519d60 0x55dd7e51cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35c6d9d640 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e51aad0 0x55dd7e51d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35c659c640 1 -- 172.21.15.18:0/2215419075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e51aad0 msgr2=0x55dd7e51d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35c659c640 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e51aad0 0x55dd7e51d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35c659c640 1 -- 172.21.15.18:0/2215419075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e519d60 msgr2=0x55dd7e51cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35c659c640 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e519d60 0x55dd7e51cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35c659c640 1 -- 172.21.15.18:0/2215419075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c810e850 con 0x55dd7e4e1550 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35c659c640 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 0x55dd7e52a2a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006010 tx=0x7f35b804c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88be4597649efc1a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35adffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35adffb640 1 -- 172.21.15.18:0/2215419075 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8045040 con 0x55dd7e4e1550 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.169+0000 7f35adffb640 1 -- 172.21.15.18:0/2215419075 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f35b802c9a0 con 0x55dd7e4e1550 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35cfb81740 1 -- 172.21.15.18:0/2215419075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35b00a3650 con 0x55dd7e4e1550 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35adffb640 1 -- 172.21.15.18:0/2215419075 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35b8004510 con 0x55dd7e4e1550 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35cfb81740 10 client.19119 fetch_fsmap learned FSMap version 48 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35cfb81740 1 -- 172.21.15.18:0/2215419075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55dd7e442540 con 0x55dd7e4e1550 2024-02-17T12:59:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35adffb640 1 -- 172.21.15.18:0/2215419075 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b00a3650 con 0x55dd7e4e1550 2024-02-17T12:59:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35adffb640 1 -- 172.21.15.18:0/2215419075 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f35b803c040 con 0x55dd7e4e1550 2024-02-17T12:59:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35cfb81740 10 client.19119 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T12:59:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35cfb81740 10 client.19119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:59:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35cfb81740 20 client.19119 populate_metadata read hostname 'smithi018' 2024-02-17T12:59:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35cfb81740 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55dd7e5232e0 0x55dd7e58cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:59:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35cfb81740 4 client.19119 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:59:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35cfb81740 1 -- 172.21.15.18:0/2215419075 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dd7e1be550 con 0x55dd7e5232e0 2024-02-17T12:59:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.170+0000 7f35bdd9b640 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55dd7e5232e0 0x55dd7e58cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:59:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.171+0000 7f35adffb640 10 client.19119 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.171+0000 7f35bdd9b640 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55dd7e5232e0 0x55dd7e58cef0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eace468fedbfd89d server_cookie=7cf1dfb3e366537 in_seq=0 out_seq=0 2024-02-17T12:59:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.215+0000 7f35adffb640 1 -- 172.21.15.18:0/2215419075 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dd7e1be550 con 0x55dd7e5232e0 2024-02-17T12:59:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.215+0000 7f35adffb640 10 client.19119 handle_command_reply: tid=0 2024-02-17T12:59:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.215+0000 7f35cfb81740 1 client.19119 shutdown 2024-02-17T12:59:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.216+0000 7f35cfb81740 1 -- 172.21.15.18:0/2215419075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 msgr2=0x55dd7e52a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.216+0000 7f35cfb81740 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 0x55dd7e52a2a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006010 tx=0x7f35b804c550 comp rx=0 tx=0).stop 2024-02-17T12:59:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.216+0000 7f35cfb81740 1 -- 172.21.15.18:0/2215419075 shutdown_connections 2024-02-17T12:59:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.216+0000 7f35cfb81740 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55dd7e5232e0 0x55dd7e58cef0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.216+0000 7f35cfb81740 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dd7e51aad0 0x55dd7e51d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.216+0000 7f35cfb81740 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dd7e519d60 0x55dd7e51cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.216+0000 7f35cfb81740 1 --2- 172.21.15.18:0/2215419075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dd7e4e1550 0x55dd7e52a2a0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.216+0000 7f35cfb81740 1 -- 172.21.15.18:0/2215419075 >> 172.21.15.18:0/2215419075 conn(0x55dd7e5258b0 msgr2=0x55dd7e51e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.216+0000 7f35adffb640 0 client.19119 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T12:59:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.216+0000 7f35cfb81740 1 -- 172.21.15.18:0/2215419075 shutdown_connections 2024-02-17T12:59:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.216+0000 7f35cfb81740 1 -- 172.21.15.18:0/2215419075 wait complete. 2024-02-17T12:59:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.216+0000 7f35cfb81740 20 client.19119 trim_cache size 0 max 16384 2024-02-17T12:59:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T12:59:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T12:59:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T12:59:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T12:59:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.217+0000 7f35ccedd640 1 -- 172.21.15.18:0/48381662 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f35b40a22d0 msgr2=0x7f35b40a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.217+0000 7f35ccedd640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f35b40a22d0 0x7f35b40a4790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f35c0005cc0 tx=0x7f35c0004a40 comp rx=0 tx=0).stop 2024-02-17T12:59:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.217+0000 7f35ccedd640 1 -- 172.21.15.18:0/48381662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c80691a0 msgr2=0x7f35c81008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:59:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.217+0000 7f35ccedd640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c80691a0 0x7f35c81008e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f35b000e9f0 tx=0x7f35b000eeb0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.217+0000 7f35ccedd640 1 -- 172.21.15.18:0/48381662 shutdown_connections 2024-02-17T12:59:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.217+0000 7f35ccedd640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f35b40a22d0 0x7f35b40a4790 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.217+0000 7f35ccedd640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35c8107270 0x7f35c8100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.217+0000 7f35ccedd640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c80691a0 0x7f35c81008e0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.217+0000 7f35ccedd640 1 --2- 172.21.15.18:0/48381662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35c8068800 0x7f35c81003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:59:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.217+0000 7f35ccedd640 1 -- 172.21.15.18:0/48381662 >> 172.21.15.18:0/48381662 conn(0x7f35c8076ed0 msgr2=0x7f35c80fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:59:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.217+0000 7f35ccedd640 1 -- 172.21.15.18:0/48381662 shutdown_connections 2024-02-17T12:59:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T12:59:57.217+0000 7f35ccedd640 1 -- 172.21.15.18:0/48381662 wait complete. 2024-02-17T12:59:57.293 INFO:tasks.workunit.client.0.smithi018.stdout: 1 509726 5.08 MB/sec execute 432 sec latency 2380.452 ms 2024-02-17T12:59:58.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 509726 5.07 MB/sec execute 433 sec latency 3380.485 ms 2024-02-17T12:59:58.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 12:59:58 smithi119 ceph-mon[36188]: pgmap v314: 97 pgs: 97 active+clean; 206 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-02-17T12:59:58.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 12:59:58 smithi146 ceph-mon[36454]: pgmap v314: 97 pgs: 97 active+clean; 206 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-02-17T12:59:58.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 12:59:58 smithi018 ceph-mon[32625]: pgmap v314: 97 pgs: 97 active+clean; 206 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-02-17T12:59:59.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 511586 5.08 MB/sec execute 434 sec latency 3575.132 ms 2024-02-17T13:00:00.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 512983 5.09 MB/sec execute 435 sec latency 15.987 ms 2024-02-17T13:00:00.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:00 smithi119 ceph-mon[36188]: pgmap v315: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 234 op/s 2024-02-17T13:00:00.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:00 smithi119 ceph-mon[36188]: overall HEALTH_OK 2024-02-17T13:00:00.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:00 smithi146 ceph-mon[36454]: pgmap v315: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 234 op/s 2024-02-17T13:00:00.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:00 smithi146 ceph-mon[36454]: overall HEALTH_OK 2024-02-17T13:00:00.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:00 smithi018 ceph-mon[32625]: pgmap v315: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 234 op/s 2024-02-17T13:00:00.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:00 smithi018 ceph-mon[32625]: overall HEALTH_OK 2024-02-17T13:00:01.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 513272 5.08 MB/sec execute 436 sec latency 793.098 ms 2024-02-17T13:00:02.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 513272 5.06 MB/sec execute 437 sec latency 1793.182 ms 2024-02-17T13:00:02.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:02 smithi119 ceph-mon[36188]: pgmap v316: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-02-17T13:00:02.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:02 smithi146 ceph-mon[36454]: pgmap v316: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-02-17T13:00:02.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:02 smithi018 ceph-mon[32625]: pgmap v316: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-02-17T13:00:03.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 513272 5.05 MB/sec execute 438 sec latency 2793.293 ms 2024-02-17T13:00:03.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:00:03.372 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:00:03.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.505+0000 7ff4e129d640 1 -- 172.21.15.18:0/1627709428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc106430 msgr2=0x7ff4dc10e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.505+0000 7ff4e129d640 1 --2- 172.21.15.18:0/1627709428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc106430 0x7ff4dc10e020 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009cd0 tx=0x7ff4d402c0f0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.505+0000 7ff4e129d640 1 -- 172.21.15.18:0/1627709428 shutdown_connections 2024-02-17T13:00:03.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.505+0000 7ff4e129d640 1 --2- 172.21.15.18:0/1627709428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc106430 0x7ff4dc10e020 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.505+0000 7ff4e129d640 1 --2- 172.21.15.18:0/1627709428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4dc06a3d0 0x7ff4dc105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.505+0000 7ff4e129d640 1 --2- 172.21.15.18:0/1627709428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc069a20 0x7ff4dc069e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.505+0000 7ff4e129d640 1 -- 172.21.15.18:0/1627709428 >> 172.21.15.18:0/1627709428 conn(0x7ff4dc076e20 msgr2=0x7ff4dc077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:03.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.505+0000 7ff4e129d640 1 -- 172.21.15.18:0/1627709428 shutdown_connections 2024-02-17T13:00:03.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.505+0000 7ff4e129d640 1 -- 172.21.15.18:0/1627709428 wait complete. 2024-02-17T13:00:03.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.506+0000 7ff4e129d640 1 Processor -- start 2024-02-17T13:00:03.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.506+0000 7ff4e129d640 1 -- start start 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.506+0000 7ff4e129d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc069a20 0x7ff4dc1003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.506+0000 7ff4e129d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4dc106430 0x7ff4dc1008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.506+0000 7ff4e129d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc100e20 0x7ff4dc1a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.506+0000 7ff4e129d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc1a5e20 con 0x7ff4dc069a20 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.506+0000 7ff4e129d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc1a5f90 con 0x7ff4dc100e20 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.506+0000 7ff4e129d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc1a6150 con 0x7ff4dc106430 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.506+0000 7ff4daffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc069a20 0x7ff4dc1003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.506+0000 7ff4daffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc069a20 0x7ff4dc1003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54870/0 (socket says 172.21.15.18:54870) 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.506+0000 7ff4daffd640 1 -- 172.21.15.18:0/1110286142 learned_addr learned my addr 172.21.15.18:0/1110286142 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4db7fe640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc100e20 0x7ff4dc1a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4d3fff640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4dc106430 0x7ff4dc1008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4daffd640 1 -- 172.21.15.18:0/1110286142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4dc106430 msgr2=0x7ff4dc1008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4daffd640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4dc106430 0x7ff4dc1008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4daffd640 1 -- 172.21.15.18:0/1110286142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc100e20 msgr2=0x7ff4dc1a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4daffd640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc100e20 0x7ff4dc1a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4daffd640 1 -- 172.21.15.18:0/1110286142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d4009900 con 0x7ff4dc069a20 2024-02-17T13:00:03.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4daffd640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc069a20 0x7ff4dc1003a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00c920 tx=0x7ff4cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:03.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4d8ff9640 1 -- 172.21.15.18:0/1110286142 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc007c70 con 0x7ff4dc069a20 2024-02-17T13:00:03.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4e129d640 1 -- 172.21.15.18:0/1110286142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4dc1a6430 con 0x7ff4dc069a20 2024-02-17T13:00:03.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4d8ff9640 1 -- 172.21.15.18:0/1110286142 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc007dd0 con 0x7ff4dc069a20 2024-02-17T13:00:03.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4d8ff9640 1 -- 172.21.15.18:0/1110286142 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc018860 con 0x7ff4dc069a20 2024-02-17T13:00:03.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.507+0000 7ff4e129d640 1 -- 172.21.15.18:0/1110286142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc1a6940 con 0x7ff4dc069a20 2024-02-17T13:00:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.509+0000 7ff4d8ff9640 1 -- 172.21.15.18:0/1110286142 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff4cc00fac0 con 0x7ff4dc069a20 2024-02-17T13:00:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.510+0000 7ff4e3e7b740 1 Processor -- start 2024-02-17T13:00:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.510+0000 7ff4e3e7b740 1 -- start start 2024-02-17T13:00:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.510+0000 7ff4e3e7b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a632b0 0x55e7f9a63690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.510+0000 7ff4e3e7b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a6c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.510+0000 7ff4e3e7b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a6eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.510+0000 7ff4e3e7b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f99d2280 con 0x55e7f9a63bd0 2024-02-17T13:00:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.510+0000 7ff4e3e7b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f99cf7b0 con 0x55e7f9a6c7b0 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.510+0000 7ff4d8ff9640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4c80a2260 0x7ff4c80a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.510+0000 7ff4e3e7b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f99caf90 con 0x55e7f9a632b0 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4daffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a6c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4db7fe640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4c80a2260 0x7ff4c80a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4daffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a6c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54886/0 (socket says 172.21.15.18:54886) 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4daffd640 1 -- 172.21.15.18:0/1175200055 learned_addr learned my addr 172.21.15.18:0/1175200055 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4d3fff640 1 --2- 172.21.15.18:0/1175200055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a6eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4db7fe640 1 --2- 172.21.15.18:0/1175200055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a632b0 0x55e7f9a63690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4d8ff9640 1 -- 172.21.15.18:0/1110286142 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff4cc0ce220 con 0x7ff4dc069a20 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4daffd640 1 -- 172.21.15.18:0/1175200055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a632b0 msgr2=0x55e7f9a63690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4daffd640 1 --2- 172.21.15.18:0/1175200055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a632b0 0x55e7f9a63690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4daffd640 1 -- 172.21.15.18:0/1175200055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a6c7b0 msgr2=0x55e7f9a6eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4daffd640 1 --2- 172.21.15.18:0/1175200055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a6eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4daffd640 1 -- 172.21.15.18:0/1175200055 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7f9989570 con 0x55e7f9a63bd0 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4d3fff640 1 --2- 172.21.15.18:0/1175200055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a6eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4db7fe640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4c80a2260 0x7ff4c80a4720 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4006010 tx=0x7ff4d40345b0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4daffd640 1 --2- 172.21.15.18:0/1175200055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a6c270 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0a2ea0 tx=0x7ff4cc00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a3aaf717cf2a97 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4d17fa640 1 -- 172.21.15.18:0/1175200055 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc09fca0 con 0x55e7f9a63bd0 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4d17fa640 1 -- 172.21.15.18:0/1175200055 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc07f040 con 0x55e7f9a63bd0 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.511+0000 7ff4d17fa640 1 -- 172.21.15.18:0/1175200055 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc09ac30 con 0x55e7f9a63bd0 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.512+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/1175200055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 msgr2=0x55e7f9a6c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.512+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/1175200055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a6c270 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0a2ea0 tx=0x7ff4cc00fec0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.512+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/1175200055 shutdown_connections 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.512+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/1175200055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a6eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.512+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/1175200055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a6c270 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.512+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/1175200055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a632b0 0x55e7f9a63690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.512+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/1175200055 >> 172.21.15.18:0/1175200055 conn(0x55e7f9a5d990 msgr2=0x55e7f9a5dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.512+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/1175200055 shutdown_connections 2024-02-17T13:00:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.512+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/1175200055 wait complete. 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4e3e7b740 1 -- start start 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4e3e7b740 1 Processor -- start 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4e3e7b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a632b0 0x55e7f9a6c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4e3e7b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4e3e7b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a69cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4e3e7b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f969e430 con 0x55e7f9a63bd0 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4e3e7b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f965ae00 con 0x55e7f9a632b0 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4e3e7b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f994ffd0 con 0x55e7f9a6c7b0 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4e3e7b740 10 client.0 _ll_register_callbacks cb 0x55e7f99a5fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4d3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4d3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a695a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54892/0 (socket says 172.21.15.18:54892) 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4d3fff640 1 -- 172.21.15.18:0/3486270612 learned_addr learned my addr 172.21.15.18:0/3486270612 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4daffd640 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a632b0 0x55e7f9a6c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4d3fff640 1 -- 172.21.15.18:0/3486270612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a6c7b0 msgr2=0x55e7f9a69cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4db7fe640 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a69cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4d3fff640 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a69cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4d3fff640 1 -- 172.21.15.18:0/3486270612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a632b0 msgr2=0x55e7f9a6c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4d3fff640 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a632b0 0x55e7f9a6c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4d3fff640 1 -- 172.21.15.18:0/3486270612 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc1a6940 con 0x55e7f9a63bd0 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4daffd640 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a632b0 0x55e7f9a6c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.513+0000 7ff4db7fe640 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a69cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4d3fff640 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a695a0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40116c0 tx=0x7ff4c4011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ccb91b11e79dec server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4c27fc640 1 -- 172.21.15.18:0/3486270612 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400fe70 con 0x55e7f9a63bd0 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3486270612 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7f96ca7e0 con 0x55e7f9a63bd0 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4c27fc640 1 -- 172.21.15.18:0/3486270612 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff4c4005020 con 0x55e7f9a63bd0 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4c27fc640 1 -- 172.21.15.18:0/3486270612 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c4005c20 con 0x55e7f9a63bd0 2024-02-17T13:00:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4e3e7b740 10 client.19131 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3486270612 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55e7f99c9610 con 0x55e7f9a63bd0 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4c27fc640 1 -- 172.21.15.18:0/3486270612 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4c4026020 con 0x55e7f9a63bd0 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4c27fc640 1 -- 172.21.15.18:0/3486270612 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff4c4022440 con 0x55e7f9a63bd0 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4e3e7b740 10 client.19131 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4e3e7b740 10 client.19131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4e3e7b740 20 client.19131 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e7f9a7cab0 0x55e7f9a7eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4e3e7b740 4 client.19131 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.514+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3486270612 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7f9707720 con 0x55e7f9a7cab0 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.515+0000 7ff4daffd640 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e7f9a7cab0 0x55e7f9a7eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.515+0000 7ff4daffd640 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e7f9a7cab0 0x55e7f9a7eed0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d564b5ec383d69e server_cookie=4b646f52300077a2 in_seq=0 out_seq=0 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.515+0000 7ff4c27fc640 10 client.19131 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.515+0000 7ff4c27fc640 1 -- 172.21.15.18:0/3486270612 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e7f9707720 con 0x55e7f9a7cab0 2024-02-17T13:00:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.515+0000 7ff4c27fc640 10 client.19131 handle_command_reply: tid=0 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4e3e7b740 1 client.19131 shutdown 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3486270612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 msgr2=0x55e7f9a695a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a695a0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40116c0 tx=0x7ff4c4011b80 comp rx=0 tx=0).stop 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3486270612 shutdown_connections 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e7f9a7cab0 0x55e7f9a7eed0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a69cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4c27fc640 0 client.19131 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a695a0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/3486270612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a632b0 0x55e7f9a6c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3486270612 >> 172.21.15.18:0/3486270612 conn(0x55e7f9a5d990 msgr2=0x55e7f9a5f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3486270612 shutdown_connections 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3486270612 wait complete. 2024-02-17T13:00:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.516+0000 7ff4e3e7b740 20 client.19131 trim_cache size 0 max 16384 2024-02-17T13:00:03.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.531+0000 7ff4e3e7b740 1 Processor -- start 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.531+0000 7ff4e3e7b740 1 -- start start 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.531+0000 7ff4e3e7b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a632b0 0x55e7f9a69540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.531+0000 7ff4e3e7b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a73660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.531+0000 7ff4e3e7b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a73ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.531+0000 7ff4e3e7b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f9a59410 con 0x55e7f9a6c7b0 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.531+0000 7ff4e3e7b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f9a7b1d0 con 0x55e7f9a632b0 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.531+0000 7ff4e3e7b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f9a62f10 con 0x55e7f9a63bd0 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.531+0000 7ff4db7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a73ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.531+0000 7ff4db7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a73ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54894/0 (socket says 172.21.15.18:54894) 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.531+0000 7ff4db7fe640 1 -- 172.21.15.18:0/3758307734 learned_addr learned my addr 172.21.15.18:0/3758307734 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.531+0000 7ff4d3fff640 1 --2- 172.21.15.18:0/3758307734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a73660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.532+0000 7ff4daffd640 1 --2- 172.21.15.18:0/3758307734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a632b0 0x55e7f9a69540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.532+0000 7ff4db7fe640 1 -- 172.21.15.18:0/3758307734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a63bd0 msgr2=0x55e7f9a73660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.532+0000 7ff4db7fe640 1 --2- 172.21.15.18:0/3758307734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a73660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.532+0000 7ff4db7fe640 1 -- 172.21.15.18:0/3758307734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a632b0 msgr2=0x55e7f9a69540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.532+0000 7ff4db7fe640 1 --2- 172.21.15.18:0/3758307734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a632b0 0x55e7f9a69540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.532+0000 7ff4db7fe640 1 -- 172.21.15.18:0/3758307734 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7f9965ac0 con 0x55e7f9a6c7b0 2024-02-17T13:00:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.532+0000 7ff4db7fe640 1 --2- 172.21.15.18:0/3758307734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a73ba0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40348f0 tx=0x7ff4d4048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c900f20b96303f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.532+0000 7ff4d0ff9640 1 -- 172.21.15.18:0/3758307734 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4034d50 con 0x55e7f9a6c7b0 2024-02-17T13:00:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.532+0000 7ff4d0ff9640 1 -- 172.21.15.18:0/3758307734 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4d4003970 con 0x55e7f9a6c7b0 2024-02-17T13:00:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.532+0000 7ff4d0ff9640 1 -- 172.21.15.18:0/3758307734 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40043c0 con 0x55e7f9a6c7b0 2024-02-17T13:00:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3758307734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a6c7b0 msgr2=0x55e7f9a73ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/3758307734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a73ba0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40348f0 tx=0x7ff4d4048000 comp rx=0 tx=0).stop 2024-02-17T13:00:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3758307734 shutdown_connections 2024-02-17T13:00:03.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/3758307734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9a73ba0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/3758307734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a73660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/3758307734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a632b0 0x55e7f9a69540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3758307734 >> 172.21.15.18:0/3758307734 conn(0x55e7f9a5d990 msgr2=0x55e7f9a68e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3758307734 shutdown_connections 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/3758307734 wait complete. 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 -- start start 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 Processor -- start 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a632b0 0x55e7f9a723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9ad2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f9a730b0 con 0x55e7f9a632b0 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f9a73220 con 0x55e7f9a6c7b0 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f9ad3520 con 0x55e7f9a63bd0 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4e3e7b740 10 client.0 _ll_register_callbacks cb 0x7ff4cc0967b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:03.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.533+0000 7ff4daffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a632b0 0x55e7f9a723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4daffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a632b0 0x55e7f9a723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54906/0 (socket says 172.21.15.18:54906) 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4daffd640 1 -- 172.21.15.18:0/2126737392 learned_addr learned my addr 172.21.15.18:0/2126737392 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4d3fff640 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4db7fe640 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9ad2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4daffd640 1 -- 172.21.15.18:0/2126737392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a63bd0 msgr2=0x55e7f9a728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4daffd640 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4daffd640 1 -- 172.21.15.18:0/2126737392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a6c7b0 msgr2=0x55e7f9ad2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4daffd640 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9ad2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4daffd640 1 -- 172.21.15.18:0/2126737392 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc10e830 con 0x55e7f9a632b0 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4d3fff640 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4db7fe640 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9ad2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4daffd640 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a632b0 0x55e7f9a723b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc09a930 tx=0x7ff4cc0cb930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4847dd45ab397d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:03.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4c27fc640 1 -- 172.21.15.18:0/2126737392 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc09a590 con 0x55e7f9a632b0 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4c27fc640 1 -- 172.21.15.18:0/2126737392 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff4cc0b3070 con 0x55e7f9a632b0 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/2126737392 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4cc07f040 con 0x55e7f9a632b0 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4c27fc640 1 -- 172.21.15.18:0/2126737392 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4cc09fca0 con 0x55e7f9a632b0 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4e3e7b740 10 client.19137 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.534+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/2126737392 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55e7f998b6a0 con 0x55e7f9a632b0 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.535+0000 7ff4c27fc640 1 -- 172.21.15.18:0/2126737392 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc07f040 con 0x55e7f9a632b0 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.535+0000 7ff4c27fc640 1 -- 172.21.15.18:0/2126737392 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff4cc016020 con 0x55e7f9a632b0 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.535+0000 7ff4e3e7b740 10 client.19137 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.535+0000 7ff4e3e7b740 10 client.19137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.535+0000 7ff4e3e7b740 20 client.19137 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.535+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e7f9a7cab0 0x55e7f9ad6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.535+0000 7ff4e3e7b740 4 client.19137 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.535+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/2126737392 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e7f9756890 con 0x55e7f9a7cab0 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.535+0000 7ff4d3fff640 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e7f9a7cab0 0x55e7f9ad6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.535+0000 7ff4d3fff640 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e7f9a7cab0 0x55e7f9ad6940 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86934ae345e82433 server_cookie=70f25116258b6e75 in_seq=0 out_seq=0 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.535+0000 7ff4c27fc640 10 client.19137 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4c27fc640 1 -- 172.21.15.18:0/2126737392 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e7f9756890 con 0x55e7f9a7cab0 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4c27fc640 10 client.19137 handle_command_reply: tid=0 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e3e7b740 1 client.19137 shutdown 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/2126737392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a632b0 msgr2=0x55e7f9a723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a632b0 0x55e7f9a723b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc09a930 tx=0x7ff4cc0cb930 comp rx=0 tx=0).stop 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/2126737392 shutdown_connections 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e7f9a7cab0 0x55e7f9ad6940 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e7f9a6c7b0 0x55e7f9ad2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e7f9a63bd0 0x55e7f9a728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4c27fc640 0 client.19137 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:03.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e3e7b740 1 --2- 172.21.15.18:0/2126737392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7f9a632b0 0x55e7f9a723b0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/2126737392 >> 172.21.15.18:0/2126737392 conn(0x55e7f9a5d990 msgr2=0x55e7f9a67e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:03.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/2126737392 shutdown_connections 2024-02-17T13:00:03.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e3e7b740 1 -- 172.21.15.18:0/2126737392 wait complete. 2024-02-17T13:00:03.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e3e7b740 20 client.19137 trim_cache size 0 max 16384 2024-02-17T13:00:03.533 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:00:03.533 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T13:00:03.533 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T13:00:03.534 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:00:03.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e129d640 1 -- 172.21.15.18:0/1110286142 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4c80a2260 msgr2=0x7ff4c80a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e129d640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4c80a2260 0x7ff4c80a4720 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4006010 tx=0x7ff4d40345b0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e129d640 1 -- 172.21.15.18:0/1110286142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc069a20 msgr2=0x7ff4dc1003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.536+0000 7ff4e129d640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc069a20 0x7ff4dc1003a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00c920 tx=0x7ff4cc00cde0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.537+0000 7ff4e129d640 1 -- 172.21.15.18:0/1110286142 shutdown_connections 2024-02-17T13:00:03.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.537+0000 7ff4e129d640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff4c80a2260 0x7ff4c80a4720 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.537+0000 7ff4e129d640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc100e20 0x7ff4dc1a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.537+0000 7ff4e129d640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4dc106430 0x7ff4dc1008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.537+0000 7ff4e129d640 1 --2- 172.21.15.18:0/1110286142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4dc069a20 0x7ff4dc1003a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.537+0000 7ff4e129d640 1 -- 172.21.15.18:0/1110286142 >> 172.21.15.18:0/1110286142 conn(0x7ff4dc076e20 msgr2=0x7ff4dc0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:03.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.537+0000 7ff4e129d640 1 -- 172.21.15.18:0/1110286142 shutdown_connections 2024-02-17T13:00:03.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.537+0000 7ff4e129d640 1 -- 172.21.15.18:0/1110286142 wait complete. 2024-02-17T13:00:03.537 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T13:00:03.538 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T13:00:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.656+0000 7f64bf39a640 1 -- 172.21.15.18:0/2035709047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64b8069aa0 msgr2=0x7f64b8069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.656+0000 7f64bf39a640 1 --2- 172.21.15.18:0/2035709047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64b8069aa0 0x7f64b8069e80 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009930 tx=0x7f64b402bc50 comp rx=0 tx=0).stop 2024-02-17T13:00:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.656+0000 7f64bf39a640 1 -- 172.21.15.18:0/2035709047 shutdown_connections 2024-02-17T13:00:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.656+0000 7f64bf39a640 1 --2- 172.21.15.18:0/2035709047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b81064a0 0x7f64b810e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.656+0000 7f64bf39a640 1 --2- 172.21.15.18:0/2035709047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64b806a450 0x7f64b8105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.656+0000 7f64bf39a640 1 --2- 172.21.15.18:0/2035709047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64b8069aa0 0x7f64b8069e80 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.656+0000 7f64bf39a640 1 -- 172.21.15.18:0/2035709047 >> 172.21.15.18:0/2035709047 conn(0x7f64b8077120 msgr2=0x7f64b8077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.656+0000 7f64bf39a640 1 -- 172.21.15.18:0/2035709047 shutdown_connections 2024-02-17T13:00:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.656+0000 7f64bf39a640 1 -- 172.21.15.18:0/2035709047 wait complete. 2024-02-17T13:00:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bf39a640 1 Processor -- start 2024-02-17T13:00:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bf39a640 1 -- start start 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bf39a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64b8069aa0 0x7f64b819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bf39a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b81064a0 0x7f64b819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bf39a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64b8197640 0x7f64b8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bf39a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8198110 con 0x7f64b8197640 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bf39a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8198280 con 0x7f64b81064a0 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bf39a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8198440 con 0x7f64b8069aa0 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bd937640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64b8197640 0x7f64b8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bd937640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64b8197640 0x7f64b8197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54930/0 (socket says 172.21.15.18:54930) 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bd937640 1 -- 172.21.15.18:0/1761252504 learned_addr learned my addr 172.21.15.18:0/1761252504 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bd136640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64b8069aa0 0x7f64b819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.657+0000 7f64bc935640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b81064a0 0x7f64b819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.658+0000 7f64bd937640 1 -- 172.21.15.18:0/1761252504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64b8069aa0 msgr2=0x7f64b819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.658+0000 7f64bd937640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64b8069aa0 0x7f64b819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.658+0000 7f64bd937640 1 -- 172.21.15.18:0/1761252504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b81064a0 msgr2=0x7f64b819dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.658+0000 7f64bd937640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b81064a0 0x7f64b819dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.658+0000 7f64bd937640 1 -- 172.21.15.18:0/1761252504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b4009560 con 0x7f64b8197640 2024-02-17T13:00:03.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.658+0000 7f64bc935640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b81064a0 0x7f64b819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:00:03.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.658+0000 7f64bd937640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64b8197640 0x7f64b8197ac0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f64b000cce0 tx=0x7f64b000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:03.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.658+0000 7f64ae7fc640 1 -- 172.21.15.18:0/1761252504 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b000f450 con 0x7f64b8197640 2024-02-17T13:00:03.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.658+0000 7f64ae7fc640 1 -- 172.21.15.18:0/1761252504 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64b000fa90 con 0x7f64b8197640 2024-02-17T13:00:03.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.658+0000 7f64bf39a640 1 -- 172.21.15.18:0/1761252504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b81a43e0 con 0x7f64b8197640 2024-02-17T13:00:03.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.658+0000 7f64ae7fc640 1 -- 172.21.15.18:0/1761252504 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0018b90 con 0x7f64b8197640 2024-02-17T13:00:03.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.658+0000 7f64bf39a640 1 -- 172.21.15.18:0/1761252504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b81a4870 con 0x7f64b8197640 2024-02-17T13:00:03.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.659+0000 7f64bf39a640 1 -- 172.21.15.18:0/1761252504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b81028a0 con 0x7f64b8197640 2024-02-17T13:00:03.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.660+0000 7f64ae7fc640 1 -- 172.21.15.18:0/1761252504 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f64b000fc00 con 0x7f64b8197640 2024-02-17T13:00:03.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.660+0000 7f64ae7fc640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64a40a2230 0x7f64a40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:03.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.661+0000 7f64bd136640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64a40a2230 0x7f64a40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:03.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.661+0000 7f64ae7fc640 1 -- 172.21.15.18:0/1761252504 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f64b00ce950 con 0x7f64b8197640 2024-02-17T13:00:03.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.661+0000 7f64bd136640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64a40a2230 0x7f64a40a46f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f64b4006010 tx=0x7f64b4000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:03.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.664+0000 7f64ae7fc640 1 -- 172.21.15.18:0/1761252504 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64b0097690 con 0x7f64b8197640 2024-02-17T13:00:03.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.885+0000 7f64bf39a640 1 -- 172.21.15.18:0/1761252504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64b80683c0 con 0x7f64b8197640 2024-02-17T13:00:03.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.886+0000 7f64ae7fc640 1 -- 172.21.15.18:0/1761252504 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+11396 (secure 0 0 0) 0x7f64b009ad50 con 0x7f64b8197640 2024-02-17T13:00:03.876 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:00:03.876 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:59:29.946156+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T13:00:03.876 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 48 2024-02-17T13:00:03.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.888+0000 7f64bf39a640 1 -- 172.21.15.18:0/1761252504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64a40a2230 msgr2=0x7f64a40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.888+0000 7f64bf39a640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64a40a2230 0x7f64a40a46f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f64b4006010 tx=0x7f64b4000bc0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.888+0000 7f64bf39a640 1 -- 172.21.15.18:0/1761252504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64b8197640 msgr2=0x7f64b8197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:03.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.888+0000 7f64bf39a640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64b8197640 0x7f64b8197ac0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f64b000cce0 tx=0x7f64b000a620 comp rx=0 tx=0).stop 2024-02-17T13:00:03.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.888+0000 7f64bf39a640 1 -- 172.21.15.18:0/1761252504 shutdown_connections 2024-02-17T13:00:03.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.888+0000 7f64bf39a640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f64a40a2230 0x7f64a40a46f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.888+0000 7f64bf39a640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64b8197640 0x7f64b8197ac0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.888+0000 7f64bf39a640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64b81064a0 0x7f64b819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.888+0000 7f64bf39a640 1 --2- 172.21.15.18:0/1761252504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64b8069aa0 0x7f64b819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:03.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.888+0000 7f64bf39a640 1 -- 172.21.15.18:0/1761252504 >> 172.21.15.18:0/1761252504 conn(0x7f64b8077120 msgr2=0x7f64b80ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:03.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.888+0000 7f64bf39a640 1 -- 172.21.15.18:0/1761252504 shutdown_connections 2024-02-17T13:00:03.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:03.888+0000 7f64bf39a640 1 -- 172.21.15.18:0/1761252504 wait complete. 2024-02-17T13:00:03.889 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:00:03.889 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T13:00:04.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.019+0000 7f2311b65640 1 -- 172.21.15.18:0/2835797140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f230c100c10 msgr2=0x7f230c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.019+0000 7f2311b65640 1 --2- 172.21.15.18:0/2835797140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f230c100c10 0x7f230c104c50 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2300009930 tx=0x7f230002bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.019+0000 7f2311b65640 1 -- 172.21.15.18:0/2835797140 shutdown_connections 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.019+0000 7f2311b65640 1 --2- 172.21.15.18:0/2835797140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f230c105330 0x7f230c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.019+0000 7f2311b65640 1 --2- 172.21.15.18:0/2835797140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f230c100c10 0x7f230c104c50 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.019+0000 7f2311b65640 1 --2- 172.21.15.18:0/2835797140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c100260 0x7f230c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.019+0000 7f2311b65640 1 -- 172.21.15.18:0/2835797140 >> 172.21.15.18:0/2835797140 conn(0x7f230c077180 msgr2=0x7f230c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.019+0000 7f2311b65640 1 -- 172.21.15.18:0/2835797140 shutdown_connections 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.019+0000 7f2311b65640 1 -- 172.21.15.18:0/2835797140 wait complete. 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.020+0000 7f2311b65640 1 Processor -- start 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.020+0000 7f2311b65640 1 -- start start 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.020+0000 7f2311b65640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f230c100260 0x7f230c19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.020+0000 7f2311b65640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f230c105330 0x7f230c19d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.020+0000 7f2311b65640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c1975d0 0x7f230c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.020+0000 7f2311b65640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c197fc0 con 0x7f230c100260 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.020+0000 7f2311b65640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c198130 con 0x7f230c1975d0 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.020+0000 7f2311b65640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c1982f0 con 0x7f230c105330 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.020+0000 7f230b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f230c100260 0x7f230c19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.020+0000 7f230b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f230c100260 0x7f230c19d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54944/0 (socket says 172.21.15.18:54944) 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.020+0000 7f230b7fe640 1 -- 172.21.15.18:0/2969192040 learned_addr learned my addr 172.21.15.18:0/2969192040 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.020+0000 7f230affd640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f230c105330 0x7f230c19d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.021+0000 7f230bfff640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c1975d0 0x7f230c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.021+0000 7f230b7fe640 1 -- 172.21.15.18:0/2969192040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f230c105330 msgr2=0x7f230c19d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.021+0000 7f230b7fe640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f230c105330 0x7f230c19d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.021+0000 7f230b7fe640 1 -- 172.21.15.18:0/2969192040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c1975d0 msgr2=0x7f230c197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.021+0000 7f230b7fe640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c1975d0 0x7f230c197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.021+0000 7f230b7fe640 1 -- 172.21.15.18:0/2969192040 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2300009560 con 0x7f230c100260 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.021+0000 7f230b7fe640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f230c100260 0x7f230c19d420 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00e9f0 tx=0x7f22fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.021+0000 7f2308ff9640 1 -- 172.21.15.18:0/2969192040 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc009700 con 0x7f230c100260 2024-02-17T13:00:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.021+0000 7f2308ff9640 1 -- 172.21.15.18:0/2969192040 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22fc00cd80 con 0x7f230c100260 2024-02-17T13:00:04.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.021+0000 7f2311b65640 1 -- 172.21.15.18:0/2969192040 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230c1a42e0 con 0x7f230c100260 2024-02-17T13:00:04.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.021+0000 7f2308ff9640 1 -- 172.21.15.18:0/2969192040 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc018920 con 0x7f230c100260 2024-02-17T13:00:04.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.021+0000 7f2311b65640 1 -- 172.21.15.18:0/2969192040 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c1a47c0 con 0x7f230c100260 2024-02-17T13:00:04.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.023+0000 7f2308ff9640 1 -- 172.21.15.18:0/2969192040 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f22fc018a80 con 0x7f230c100260 2024-02-17T13:00:04.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f2314812740 1 Processor -- start 2024-02-17T13:00:04.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f2314812740 1 -- start start 2024-02-17T13:00:04.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f2314812740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f02250 0x556a46f02630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f2314812740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02b70 0x556a46f0b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f2314812740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f0b750 0x556a46f0db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f2314812740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a46ebd960 con 0x556a46f0b750 2024-02-17T13:00:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f2314812740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a46e71310 con 0x556a46f02b70 2024-02-17T13:00:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f2314812740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a46e6e840 con 0x556a46f02250 2024-02-17T13:00:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f230affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f0b750 0x556a46f0db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f230affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f0b750 0x556a46f0db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54952/0 (socket says 172.21.15.18:54952) 2024-02-17T13:00:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f230affd640 1 -- 172.21.15.18:0/138447352 learned_addr learned my addr 172.21.15.18:0/138447352 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f2308ff9640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f22f40a2230 0x7f22f40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.024+0000 7f230b7fe640 1 --2- 172.21.15.18:0/138447352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02b70 0x556a46f0b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f230bfff640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f22f40a2230 0x7f22f40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f230affd640 1 -- 172.21.15.18:0/138447352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f02250 msgr2=0x556a46f02630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f230bfff640 1 --2- 172.21.15.18:0/138447352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f02250 0x556a46f02630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f230affd640 1 --2- 172.21.15.18:0/138447352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f02250 0x556a46f02630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f230affd640 1 -- 172.21.15.18:0/138447352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02b70 msgr2=0x556a46f0b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f230affd640 1 --2- 172.21.15.18:0/138447352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02b70 0x556a46f0b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f230affd640 1 -- 172.21.15.18:0/138447352 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a46e2c060 con 0x556a46f0b750 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f2308ff9640 1 -- 172.21.15.18:0/2969192040 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f22fc014070 con 0x7f230c100260 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f230bfff640 1 --2- 172.21.15.18:0/138447352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f02250 0x556a46f02630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f230b7fe640 1 --2- 172.21.15.18:0/138447352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02b70 0x556a46f0b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f230affd640 1 --2- 172.21.15.18:0/138447352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f0b750 0x556a46f0db40 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f2300005ab0 tx=0x7f230002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15acafa321cb8600 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f230bfff640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f22f40a2230 0x7f22f40a46f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2304005fd0 tx=0x7f2304013040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f22f1ffb640 1 -- 172.21.15.18:0/138447352 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300004800 con 0x556a46f0b750 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f22f1ffb640 1 -- 172.21.15.18:0/138447352 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2300004960 con 0x556a46f0b750 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.025+0000 7f22f1ffb640 1 -- 172.21.15.18:0/138447352 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300005030 con 0x556a46f0b750 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.026+0000 7f2314812740 1 -- 172.21.15.18:0/138447352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f0b750 msgr2=0x556a46f0db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.026+0000 7f2314812740 1 --2- 172.21.15.18:0/138447352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f0b750 0x556a46f0db40 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f2300005ab0 tx=0x7f230002c5d0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.026+0000 7f2314812740 1 -- 172.21.15.18:0/138447352 shutdown_connections 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.026+0000 7f2314812740 1 --2- 172.21.15.18:0/138447352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f0b750 0x556a46f0db40 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.026+0000 7f2314812740 1 --2- 172.21.15.18:0/138447352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02b70 0x556a46f0b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.026+0000 7f2314812740 1 --2- 172.21.15.18:0/138447352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f02250 0x556a46f02630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.026+0000 7f2314812740 1 -- 172.21.15.18:0/138447352 >> 172.21.15.18:0/138447352 conn(0x556a46efc980 msgr2=0x556a46efcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.026+0000 7f2314812740 1 -- 172.21.15.18:0/138447352 shutdown_connections 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.026+0000 7f2314812740 1 -- 172.21.15.18:0/138447352 wait complete. 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.026+0000 7f2314812740 1 -- start start 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f2314812740 1 Processor -- start 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f2314812740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 0x556a46f086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f2314812740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02b70 0x556a46f08c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f2314812740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f0b750 0x556a46f09160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f2314812740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a46e6a020 con 0x556a46f02250 2024-02-17T13:00:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f2314812740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a46b3d430 con 0x556a46f02b70 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f2314812740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a46af9e00 con 0x556a46f0b750 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f2314812740 10 client.0 _ll_register_callbacks cb 0x556a46e11cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f230b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 0x556a46f086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f230b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 0x556a46f086e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54960/0 (socket says 172.21.15.18:54960) 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f230b7fe640 1 -- 172.21.15.18:0/508916192 learned_addr learned my addr 172.21.15.18:0/508916192 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f230bfff640 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f0b750 0x556a46f09160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f230affd640 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02b70 0x556a46f08c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f230b7fe640 1 -- 172.21.15.18:0/508916192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f0b750 msgr2=0x556a46f09160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f230b7fe640 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f0b750 0x556a46f09160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f230b7fe640 1 -- 172.21.15.18:0/508916192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02b70 msgr2=0x556a46f08c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f230b7fe640 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02b70 0x556a46f08c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f230b7fe640 1 -- 172.21.15.18:0/508916192 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c0682e0 con 0x556a46f02250 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f230b7fe640 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 0x556a46f086e0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f22fc000c00 tx=0x7f22fc09ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1240944ed28fa036 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.027+0000 7f22eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f22eaffd640 1 -- 172.21.15.18:0/508916192 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc085db0 con 0x556a46f02250 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f22eaffd640 1 -- 172.21.15.18:0/508916192 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f22fc09ef90 con 0x556a46f02250 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f2314812740 1 -- 172.21.15.18:0/508916192 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a46b697e0 con 0x556a46f02250 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f22eaffd640 1 -- 172.21.15.18:0/508916192 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22fc0a35e0 con 0x556a46f02250 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f2314812740 10 client.19155 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f2314812740 1 -- 172.21.15.18:0/508916192 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x556a46e686a0 con 0x556a46f02250 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f22eaffd640 1 -- 172.21.15.18:0/508916192 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556a46b697e0 con 0x556a46f02250 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f22eaffd640 1 -- 172.21.15.18:0/508916192 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f22fc09a4d0 con 0x556a46f02250 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f2314812740 10 client.19155 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f2314812740 10 client.19155 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f2314812740 20 client.19155 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f2314812740 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x556a46f1b980 0x556a46f1dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f2314812740 4 client.19155 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f2314812740 1 -- 172.21.15.18:0/508916192 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a46ba6720 con 0x556a46f1b980 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.028+0000 7f230affd640 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x556a46f1b980 0x556a46f1dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f22eaffd640 10 client.19155 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f230affd640 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x556a46f1b980 0x556a46f1dd90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10e624baa754f0f server_cookie=4d241791c47f9a48 in_seq=0 out_seq=0 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f22eaffd640 1 -- 172.21.15.18:0/508916192 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a46ba6720 con 0x556a46f1b980 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f22eaffd640 10 client.19155 handle_command_reply: tid=0 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f2314812740 1 client.19155 shutdown 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f2314812740 1 -- 172.21.15.18:0/508916192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 msgr2=0x556a46f086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f2314812740 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 0x556a46f086e0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f22fc000c00 tx=0x7f22fc09ad80 comp rx=0 tx=0).stop 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f2314812740 1 -- 172.21.15.18:0/508916192 shutdown_connections 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f2314812740 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f0b750 0x556a46f09160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f2314812740 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x556a46f1b980 0x556a46f1dd90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f2314812740 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02b70 0x556a46f08c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f22eaffd640 0 client.19155 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f2314812740 1 --2- 172.21.15.18:0/508916192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 0x556a46f086e0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.029+0000 7f2314812740 1 -- 172.21.15.18:0/508916192 >> 172.21.15.18:0/508916192 conn(0x556a46efc980 msgr2=0x556a46efe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.030+0000 7f2314812740 1 -- 172.21.15.18:0/508916192 shutdown_connections 2024-02-17T13:00:04.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.030+0000 7f2314812740 1 -- 172.21.15.18:0/508916192 wait complete. 2024-02-17T13:00:04.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.030+0000 7f2314812740 20 client.19155 trim_cache size 0 max 16384 2024-02-17T13:00:04.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f2314812740 1 Processor -- start 2024-02-17T13:00:04.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f2314812740 1 -- start start 2024-02-17T13:00:04.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f2314812740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 0x556a46efba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f2314812740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f02b70 0x556a46f125a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f2314812740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f0b750 0x556a46f12ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f2314812740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a46def130 con 0x556a46f02250 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f2314812740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a46f01c10 con 0x556a46f0b750 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f2314812740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a46f01eb0 con 0x556a46f02b70 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f230b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 0x556a46efba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f230b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 0x556a46efba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54968/0 (socket says 172.21.15.18:54968) 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f230b7fe640 1 -- 172.21.15.18:0/3110421299 learned_addr learned my addr 172.21.15.18:0/3110421299 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f230bfff640 1 --2- 172.21.15.18:0/3110421299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f0b750 0x556a46f12ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f230affd640 1 --2- 172.21.15.18:0/3110421299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f02b70 0x556a46f125a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f230b7fe640 1 -- 172.21.15.18:0/3110421299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f02b70 msgr2=0x556a46f125a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f230b7fe640 1 --2- 172.21.15.18:0/3110421299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f02b70 0x556a46f125a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f230b7fe640 1 -- 172.21.15.18:0/3110421299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f0b750 msgr2=0x556a46f12ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f230b7fe640 1 --2- 172.21.15.18:0/3110421299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f0b750 0x556a46f12ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f230b7fe640 1 -- 172.21.15.18:0/3110421299 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a46e04c20 con 0x556a46f02250 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f230bfff640 1 --2- 172.21.15.18:0/3110421299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f0b750 0x556a46f12ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:04.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f230b7fe640 1 --2- 172.21.15.18:0/3110421299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 0x556a46efba00 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0b0040 tx=0x7f22fc09adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83b4720368b4d20 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f22f17fa640 1 -- 172.21.15.18:0/3110421299 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc085db0 con 0x556a46f02250 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f22f17fa640 1 -- 172.21.15.18:0/3110421299 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22fc0aee90 con 0x556a46f02250 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.042+0000 7f22f17fa640 1 -- 172.21.15.18:0/3110421299 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc09a5a0 con 0x556a46f02250 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.043+0000 7f2314812740 1 -- 172.21.15.18:0/3110421299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 msgr2=0x556a46efba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.043+0000 7f2314812740 1 --2- 172.21.15.18:0/3110421299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 0x556a46efba00 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0b0040 tx=0x7f22fc09adf0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.043+0000 7f2314812740 1 -- 172.21.15.18:0/3110421299 shutdown_connections 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.043+0000 7f2314812740 1 --2- 172.21.15.18:0/3110421299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f0b750 0x556a46f12ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.043+0000 7f2314812740 1 --2- 172.21.15.18:0/3110421299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f02b70 0x556a46f125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.043+0000 7f2314812740 1 --2- 172.21.15.18:0/3110421299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02250 0x556a46efba00 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.043+0000 7f2314812740 1 -- 172.21.15.18:0/3110421299 >> 172.21.15.18:0/3110421299 conn(0x556a46efc980 msgr2=0x556a46efb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.043+0000 7f2314812740 1 -- 172.21.15.18:0/3110421299 shutdown_connections 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.043+0000 7f2314812740 1 -- 172.21.15.18:0/3110421299 wait complete. 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.043+0000 7f2314812740 1 -- start start 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f2314812740 1 Processor -- start 2024-02-17T13:00:04.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f2314812740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02250 0x556a46f10ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f2314812740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02b70 0x556a46f11410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f2314812740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f0b750 0x556a46f11950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f2314812740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a46f12e70 con 0x556a46f02b70 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f2314812740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a46f71ec0 con 0x556a46f02250 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f2314812740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a46f72030 con 0x556a46f0b750 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f2314812740 10 client.0 _ll_register_callbacks cb 0x7f23040093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f230affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02b70 0x556a46f11410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f230affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02b70 0x556a46f11410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54978/0 (socket says 172.21.15.18:54978) 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f230affd640 1 -- 172.21.15.18:0/2608579040 learned_addr learned my addr 172.21.15.18:0/2608579040 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f230b7fe640 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02250 0x556a46f10ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f230bfff640 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f0b750 0x556a46f11950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f230affd640 1 -- 172.21.15.18:0/2608579040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f0b750 msgr2=0x556a46f11950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f230affd640 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f0b750 0x556a46f11950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f230affd640 1 -- 172.21.15.18:0/2608579040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02250 msgr2=0x556a46f10ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f230affd640 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02250 0x556a46f10ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.044+0000 7f230affd640 1 -- 172.21.15.18:0/2608579040 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a46e686a0 con 0x556a46f02b70 2024-02-17T13:00:04.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f230affd640 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02b70 0x556a46f11410 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f2300005c40 tx=0x7f230002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab59a12a658ee7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f22eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:04.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f22eaffd640 1 -- 172.21.15.18:0/2608579040 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230004f040 con 0x556a46f02b70 2024-02-17T13:00:04.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f22eaffd640 1 -- 172.21.15.18:0/2608579040 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f230002c860 con 0x556a46f02b70 2024-02-17T13:00:04.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f2314812740 1 -- 172.21.15.18:0/2608579040 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2300004960 con 0x556a46f02b70 2024-02-17T13:00:04.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f22eaffd640 1 -- 172.21.15.18:0/2608579040 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2300004800 con 0x556a46f02b70 2024-02-17T13:00:04.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f2314812740 10 client.19161 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:04.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f2314812740 1 -- 172.21.15.18:0/2608579040 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x556a46e2a8c0 con 0x556a46f02b70 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f22eaffd640 1 -- 172.21.15.18:0/2608579040 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2300004960 con 0x556a46f02b70 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f22eaffd640 1 -- 172.21.15.18:0/2608579040 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f230003c050 con 0x556a46f02b70 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f2314812740 10 client.19161 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f2314812740 10 client.19161 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f2314812740 20 client.19161 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f2314812740 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x556a46f1b980 0x556a46f75540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f2314812740 4 client.19161 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.045+0000 7f2314812740 1 -- 172.21.15.18:0/2608579040 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a46bf5890 con 0x556a46f1b980 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f230b7fe640 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x556a46f1b980 0x556a46f75540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f230b7fe640 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x556a46f1b980 0x556a46f75540 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb9ac7770ed2eebc server_cookie=829597b04be1894 in_seq=0 out_seq=0 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f22eaffd640 10 client.19161 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f22eaffd640 1 -- 172.21.15.18:0/2608579040 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a46bf5890 con 0x556a46f1b980 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f22eaffd640 10 client.19161 handle_command_reply: tid=0 2024-02-17T13:00:04.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f2314812740 1 client.19161 shutdown 2024-02-17T13:00:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f2314812740 1 -- 172.21.15.18:0/2608579040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02b70 msgr2=0x556a46f11410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f2314812740 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02b70 0x556a46f11410 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f2300005c40 tx=0x7f230002bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f2314812740 1 -- 172.21.15.18:0/2608579040 shutdown_connections 2024-02-17T13:00:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f2314812740 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a46f0b750 0x556a46f11950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f2314812740 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a46f02b70 0x556a46f11410 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f2314812740 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x556a46f1b980 0x556a46f75540 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f2314812740 1 --2- 172.21.15.18:0/2608579040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a46f02250 0x556a46f10ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f2314812740 1 -- 172.21.15.18:0/2608579040 >> 172.21.15.18:0/2608579040 conn(0x556a46efc980 msgr2=0x556a46efbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f22eaffd640 0 client.19161 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:00:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f2314812740 1 -- 172.21.15.18:0/2608579040 shutdown_connections 2024-02-17T13:00:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f2314812740 1 -- 172.21.15.18:0/2608579040 wait complete. 2024-02-17T13:00:04.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.046+0000 7f2314812740 20 client.19161 trim_cache size 0 max 16384 2024-02-17T13:00:04.044 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:00:04.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.047+0000 7f2311b65640 1 -- 172.21.15.18:0/2969192040 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f22f40a2230 msgr2=0x7f22f40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.047+0000 7f2311b65640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f22f40a2230 0x7f22f40a46f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2304005fd0 tx=0x7f2304013040 comp rx=0 tx=0).stop 2024-02-17T13:00:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.047+0000 7f2311b65640 1 -- 172.21.15.18:0/2969192040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f230c100260 msgr2=0x7f230c19d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.047+0000 7f2311b65640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f230c100260 0x7f230c19d420 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00e9f0 tx=0x7f22fc00eeb0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.047+0000 7f2311b65640 1 -- 172.21.15.18:0/2969192040 shutdown_connections 2024-02-17T13:00:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.047+0000 7f2311b65640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f22f40a2230 0x7f22f40a46f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.047+0000 7f2311b65640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f230c1975d0 0x7f230c197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.047+0000 7f2311b65640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f230c105330 0x7f230c19d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.047+0000 7f2311b65640 1 --2- 172.21.15.18:0/2969192040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f230c100260 0x7f230c19d420 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.047+0000 7f2311b65640 1 -- 172.21.15.18:0/2969192040 >> 172.21.15.18:0/2969192040 conn(0x7f230c077180 msgr2=0x7f230c0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.047+0000 7f2311b65640 1 -- 172.21.15.18:0/2969192040 shutdown_connections 2024-02-17T13:00:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.047+0000 7f2311b65640 1 -- 172.21.15.18:0/2969192040 wait complete. 2024-02-17T13:00:04.049 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:00:04.049 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T13:00:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.172+0000 7f50fff6c640 1 -- 172.21.15.18:0/3323044454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50f8100260 msgr2=0x7f50f8100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.172+0000 7f50fff6c640 1 --2- 172.21.15.18:0/3323044454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50f8100260 0x7f50f8100640 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009930 tx=0x7f50f402bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.172+0000 7f50fff6c640 1 -- 172.21.15.18:0/3323044454 shutdown_connections 2024-02-17T13:00:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.172+0000 7f50fff6c640 1 --2- 172.21.15.18:0/3323044454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50f8105330 0x7f50f810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.172+0000 7f50fff6c640 1 --2- 172.21.15.18:0/3323044454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f8100c10 0x7f50f8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.172+0000 7f50fff6c640 1 --2- 172.21.15.18:0/3323044454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50f8100260 0x7f50f8100640 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.172+0000 7f50fff6c640 1 -- 172.21.15.18:0/3323044454 >> 172.21.15.18:0/3323044454 conn(0x7f50f8077180 msgr2=0x7f50f8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.172+0000 7f50fff6c640 1 -- 172.21.15.18:0/3323044454 shutdown_connections 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.172+0000 7f50fff6c640 1 -- 172.21.15.18:0/3323044454 wait complete. 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fff6c640 1 Processor -- start 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fff6c640 1 -- start start 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fff6c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f8100260 0x7f50f819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fff6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50f8105330 0x7f50f819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fff6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50f8197750 0x7f50f8197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fff6c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f8198220 con 0x7f50f8105330 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fff6c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f8198390 con 0x7f50f8197750 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fff6c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f8198550 con 0x7f50f8100260 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fd507640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50f8105330 0x7f50f819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fd507640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50f8105330 0x7f50f819dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54994/0 (socket says 172.21.15.18:54994) 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fdd08640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f8100260 0x7f50f819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fd507640 1 -- 172.21.15.18:0/142318698 learned_addr learned my addr 172.21.15.18:0/142318698 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.173+0000 7f50fe509640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50f8197750 0x7f50f8197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.174+0000 7f50fd507640 1 -- 172.21.15.18:0/142318698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f8100260 msgr2=0x7f50f819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.174+0000 7f50fd507640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f8100260 0x7f50f819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.174+0000 7f50fd507640 1 -- 172.21.15.18:0/142318698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50f8197750 msgr2=0x7f50f8197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.174+0000 7f50fd507640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50f8197750 0x7f50f8197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.174+0000 7f50fd507640 1 -- 172.21.15.18:0/142318698 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f0009840 con 0x7f50f8105330 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.174+0000 7f50fe509640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50f8197750 0x7f50f8197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.174+0000 7f50fd507640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50f8105330 0x7f50f819dbd0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f50e800e9f0 tx=0x7f50e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.174+0000 7f50eeffd640 1 -- 172.21.15.18:0/142318698 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e80096c0 con 0x7f50f8105330 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.174+0000 7f50eeffd640 1 -- 172.21.15.18:0/142318698 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50e8005500 con 0x7f50f8105330 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.174+0000 7f50fff6c640 1 -- 172.21.15.18:0/142318698 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f4009560 con 0x7f50f8105330 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.174+0000 7f50eeffd640 1 -- 172.21.15.18:0/142318698 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e8010430 con 0x7f50f8105330 2024-02-17T13:00:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.174+0000 7f50fff6c640 1 -- 172.21.15.18:0/142318698 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f81a4840 con 0x7f50f8105330 2024-02-17T13:00:04.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.177+0000 7f5102c1a740 1 Processor -- start 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.177+0000 7f5102c1a740 1 -- start start 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.177+0000 7f50eeffd640 1 -- 172.21.15.18:0/142318698 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f50e8005020 con 0x7f50f8105330 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.177+0000 7f5102c1a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bba2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.177+0000 7f5102c1a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bbab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.177+0000 7f5102c1a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbadb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.177+0000 7f5102c1a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846bb5d960 con 0x55846bba2b70 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.177+0000 7f5102c1a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846bb11310 con 0x55846bbab750 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.177+0000 7f5102c1a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846bb0e840 con 0x55846bba2250 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fdd08640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bbab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fe509640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bba2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fdd08640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bbab210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55008/0 (socket says 172.21.15.18:55008) 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fe509640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bba2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34392/0 (socket says 172.21.15.18:34392) 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fdd08640 1 -- 172.21.15.18:0/23409178 learned_addr learned my addr 172.21.15.18:0/23409178 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fd507640 1 --2- 172.21.15.18:0/23409178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbadb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50eeffd640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f50e40a22b0 0x7f50e40a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fe509640 1 -- 172.21.15.18:0/23409178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 msgr2=0x55846bbadb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fe509640 1 --2- 172.21.15.18:0/23409178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbadb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fe509640 1 -- 172.21.15.18:0/23409178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 msgr2=0x55846bbab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fe509640 1 --2- 172.21.15.18:0/23409178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bbab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fe509640 1 -- 172.21.15.18:0/23409178 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55846bacc060 con 0x55846bba2250 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fd507640 1 --2- 172.21.15.18:0/23409178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbadb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fe509640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f50e40a22b0 0x7f50e40a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50eeffd640 1 -- 172.21.15.18:0/142318698 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f50e8014070 con 0x7f50f8105330 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50fe509640 1 --2- 172.21.15.18:0/23409178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bba2630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009fb0 tx=0x7f50f000f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2651a47d5ac62dbc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.178+0000 7f50e3fff640 1 -- 172.21.15.18:0/23409178 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0011ba0 con 0x55846bba2250 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.179+0000 7f50e3fff640 1 -- 172.21.15.18:0/23409178 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50f0011e40 con 0x55846bba2250 2024-02-17T13:00:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.179+0000 7f50e3fff640 1 -- 172.21.15.18:0/23409178 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0010630 con 0x55846bba2250 2024-02-17T13:00:04.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.179+0000 7f50fe509640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f50e40a22b0 0x7f50e40a4770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f50f001e000 tx=0x7f50f0021000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.179+0000 7f5102c1a740 1 -- 172.21.15.18:0/23409178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 msgr2=0x55846bba2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.179+0000 7f5102c1a740 1 --2- 172.21.15.18:0/23409178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bba2630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009fb0 tx=0x7f50f000f7f0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.179+0000 7f5102c1a740 1 -- 172.21.15.18:0/23409178 shutdown_connections 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.179+0000 7f5102c1a740 1 --2- 172.21.15.18:0/23409178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbadb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.179+0000 7f5102c1a740 1 --2- 172.21.15.18:0/23409178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bbab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.179+0000 7f5102c1a740 1 --2- 172.21.15.18:0/23409178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bba2630 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.179+0000 7f5102c1a740 1 -- 172.21.15.18:0/23409178 >> 172.21.15.18:0/23409178 conn(0x55846bb9c980 msgr2=0x55846bb9cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.180+0000 7f5102c1a740 1 -- 172.21.15.18:0/23409178 shutdown_connections 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.180+0000 7f5102c1a740 1 -- 172.21.15.18:0/23409178 wait complete. 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.180+0000 7f5102c1a740 1 -- start start 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.180+0000 7f5102c1a740 1 Processor -- start 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f5102c1a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bba8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f5102c1a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bba8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f5102c1a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bba91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f5102c1a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846bb0a020 con 0x55846bba2b70 2024-02-17T13:00:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f5102c1a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b7dd430 con 0x55846bbab750 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f5102c1a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b799e00 con 0x55846bba2250 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50fd507640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bba8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50fdd08640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bba8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f5102c1a740 10 client.0 _ll_register_callbacks cb 0x55846bab1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50fdd08640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bba8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34406/0 (socket says 172.21.15.18:34406) 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50fd507640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bba8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55014/0 (socket says 172.21.15.18:55014) 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50fdd08640 1 -- 172.21.15.18:0/2091220092 learned_addr learned my addr 172.21.15.18:0/2091220092 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50fe509640 1 --2- 172.21.15.18:0/2091220092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bba91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50fdd08640 1 -- 172.21.15.18:0/2091220092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 msgr2=0x55846bba91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50fdd08640 1 --2- 172.21.15.18:0/2091220092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bba91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50fdd08640 1 -- 172.21.15.18:0/2091220092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 msgr2=0x55846bba8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50fdd08640 1 --2- 172.21.15.18:0/2091220092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bba8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50fdd08640 1 -- 172.21.15.18:0/2091220092 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f0009c90 con 0x55846bba2250 2024-02-17T13:00:04.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50fdd08640 1 --2- 172.21.15.18:0/2091220092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bba8730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009230 tx=0x7f50f402ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eceb473f617bdc03 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.181+0000 7f50e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f50e17fa640 1 -- 172.21.15.18:0/2091220092 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4004050 con 0x55846bba2250 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f5102c1a740 1 -- 172.21.15.18:0/2091220092 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55846b8097e0 con 0x55846bba2250 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f50e17fa640 1 -- 172.21.15.18:0/2091220092 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f50f40041b0 con 0x55846bba2250 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f50e17fa640 1 -- 172.21.15.18:0/2091220092 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50f4004370 con 0x55846bba2250 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f5102c1a740 10 client.28243 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f5102c1a740 1 -- 172.21.15.18:0/2091220092 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55846bb086a0 con 0x55846bba2250 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f50e17fa640 1 -- 172.21.15.18:0/2091220092 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55846b8097e0 con 0x55846bba2250 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f50e17fa640 1 -- 172.21.15.18:0/2091220092 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f50f403c040 con 0x55846bba2250 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f5102c1a740 10 client.28243 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f5102c1a740 10 client.28243 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f5102c1a740 20 client.28243 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f5102c1a740 1 --2- 172.21.15.18:0/2091220092 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55846bbbbba0 0x55846bbbdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f5102c1a740 4 client.28243 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:04.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.182+0000 7f5102c1a740 1 -- 172.21.15.18:0/2091220092 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55846b846720 con 0x55846bbbbba0 2024-02-17T13:00:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.183+0000 7f50fd507640 1 --2- 172.21.15.18:0/2091220092 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55846bbbbba0 0x55846bbbdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.183+0000 7f50fd507640 1 --2- 172.21.15.18:0/2091220092 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55846bbbbba0 0x55846bbbdf90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7e6f7e2d99f9a72 server_cookie=48ab23f994e263c2 in_seq=0 out_seq=0 2024-02-17T13:00:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.183+0000 7f50e17fa640 10 client.28243 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:00:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f50e17fa640 1 -- 172.21.15.18:0/2091220092 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55846b846720 con 0x55846bbbbba0 2024-02-17T13:00:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f50e17fa640 10 client.28243 handle_command_reply: tid=0 2024-02-17T13:00:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f5102c1a740 1 client.28243 shutdown 2024-02-17T13:00:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f5102c1a740 1 -- 172.21.15.18:0/2091220092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 msgr2=0x55846bba8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f5102c1a740 1 --2- 172.21.15.18:0/2091220092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bba8730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009230 tx=0x7f50f402ca10 comp rx=0 tx=0).stop 2024-02-17T13:00:04.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f5102c1a740 1 -- 172.21.15.18:0/2091220092 shutdown_connections 2024-02-17T13:00:04.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f5102c1a740 1 --2- 172.21.15.18:0/2091220092 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55846bbbbba0 0x55846bbbdf90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f5102c1a740 1 --2- 172.21.15.18:0/2091220092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bba91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f5102c1a740 1 --2- 172.21.15.18:0/2091220092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bba8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f5102c1a740 1 --2- 172.21.15.18:0/2091220092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bba8730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f50e17fa640 0 client.28243 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:00:04.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f5102c1a740 1 -- 172.21.15.18:0/2091220092 >> 172.21.15.18:0/2091220092 conn(0x55846bb9c980 msgr2=0x55846bb9e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f5102c1a740 1 -- 172.21.15.18:0/2091220092 shutdown_connections 2024-02-17T13:00:04.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f5102c1a740 1 -- 172.21.15.18:0/2091220092 wait complete. 2024-02-17T13:00:04.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.184+0000 7f5102c1a740 20 client.28243 trim_cache size 0 max 16384 2024-02-17T13:00:04.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.199+0000 7f5102c1a740 1 Processor -- start 2024-02-17T13:00:04.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.199+0000 7f5102c1a740 1 -- start start 2024-02-17T13:00:04.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.199+0000 7f5102c1a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bb9ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.199+0000 7f5102c1a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bbb2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.199+0000 7f5102c1a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbb2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.199+0000 7f5102c1a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846ba8f130 con 0x55846bba2b70 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.199+0000 7f5102c1a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846bb9c7f0 con 0x55846bbab750 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.199+0000 7f5102c1a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846bba1eb0 con 0x55846bba2250 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50fd507640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bbb2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50fdd08640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bb9ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50fd507640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bbb2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55022/0 (socket says 172.21.15.18:55022) 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50fd507640 1 -- 172.21.15.18:0/278251747 learned_addr learned my addr 172.21.15.18:0/278251747 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50fe509640 1 --2- 172.21.15.18:0/278251747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbb2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50fd507640 1 -- 172.21.15.18:0/278251747 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 msgr2=0x55846bb9ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50fd507640 1 --2- 172.21.15.18:0/278251747 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bb9ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50fd507640 1 -- 172.21.15.18:0/278251747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 msgr2=0x55846bbb2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50fd507640 1 --2- 172.21.15.18:0/278251747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbb2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50fd507640 1 -- 172.21.15.18:0/278251747 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55846baa4c20 con 0x55846bba2b70 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50fd507640 1 --2- 172.21.15.18:0/278251747 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bbb2650 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f50e8097280 tx=0x7f50e80ce5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5c5651a6de6181 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50e37fe640 1 -- 172.21.15.18:0/278251747 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e809ae20 con 0x55846bba2b70 2024-02-17T13:00:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50e37fe640 1 -- 172.21.15.18:0/278251747 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50e809ef70 con 0x55846bba2b70 2024-02-17T13:00:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.200+0000 7f50e37fe640 1 -- 172.21.15.18:0/278251747 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e80991b0 con 0x55846bba2b70 2024-02-17T13:00:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.201+0000 7f5102c1a740 1 -- 172.21.15.18:0/278251747 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 msgr2=0x55846bbb2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.201+0000 7f5102c1a740 1 --2- 172.21.15.18:0/278251747 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bbb2650 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f50e8097280 tx=0x7f50e80ce5a0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.201+0000 7f5102c1a740 1 -- 172.21.15.18:0/278251747 shutdown_connections 2024-02-17T13:00:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.201+0000 7f5102c1a740 1 --2- 172.21.15.18:0/278251747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbb2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.201+0000 7f5102c1a740 1 --2- 172.21.15.18:0/278251747 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2b70 0x55846bbb2650 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.201+0000 7f5102c1a740 1 --2- 172.21.15.18:0/278251747 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2250 0x55846bb9ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.201+0000 7f5102c1a740 1 -- 172.21.15.18:0/278251747 >> 172.21.15.18:0/278251747 conn(0x55846bb9c980 msgr2=0x55846bb9b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.201+0000 7f5102c1a740 1 -- 172.21.15.18:0/278251747 shutdown_connections 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.201+0000 7f5102c1a740 1 -- 172.21.15.18:0/278251747 wait complete. 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.201+0000 7f5102c1a740 1 -- start start 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f5102c1a740 1 Processor -- start 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f5102c1a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2250 0x55846bbb1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f5102c1a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2b70 0x55846bbb15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f5102c1a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbb1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f5102c1a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846bc120c0 con 0x55846bba2250 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f5102c1a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846bc12200 con 0x55846bbab750 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f5102c1a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846bc12340 con 0x55846bba2b70 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f5102c1a740 10 client.0 _ll_register_callbacks cb 0x7f50f0009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f50fdd08640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2250 0x55846bbb1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f50fdd08640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2250 0x55846bbb1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55028/0 (socket says 172.21.15.18:55028) 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f50fdd08640 1 -- 172.21.15.18:0/258549915 learned_addr learned my addr 172.21.15.18:0/258549915 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f50fd507640 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2b70 0x55846bbb15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f50fe509640 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbb1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f50fdd08640 1 -- 172.21.15.18:0/258549915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2b70 msgr2=0x55846bbb15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f50fdd08640 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2b70 0x55846bbb15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f50fdd08640 1 -- 172.21.15.18:0/258549915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 msgr2=0x55846bbb1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f50fdd08640 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbb1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.202+0000 7f50fdd08640 1 -- 172.21.15.18:0/258549915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f81a4840 con 0x55846bba2250 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.203+0000 7f50fdd08640 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2250 0x55846bbb1090 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f50f4004430 tx=0x7f50f4004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569df005db7bd32e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.203+0000 7f50e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.203+0000 7f50e17fa640 1 -- 172.21.15.18:0/258549915 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4004a60 con 0x55846bba2250 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.203+0000 7f50e17fa640 1 -- 172.21.15.18:0/258549915 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f50f4000e40 con 0x55846bba2250 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.203+0000 7f5102c1a740 1 -- 172.21.15.18:0/258549915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50f0011e40 con 0x55846bba2250 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.203+0000 7f50e17fa640 1 -- 172.21.15.18:0/258549915 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50f4003940 con 0x55846bba2250 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.203+0000 7f5102c1a740 10 client.19179 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.203+0000 7f5102c1a740 1 -- 172.21.15.18:0/258549915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55846baca8c0 con 0x55846bba2250 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.203+0000 7f50e17fa640 1 -- 172.21.15.18:0/258549915 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50f0011e40 con 0x55846bba2250 2024-02-17T13:00:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.203+0000 7f50e17fa640 1 -- 172.21.15.18:0/258549915 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f50f403c040 con 0x55846bba2250 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f5102c1a740 10 client.19179 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f5102c1a740 10 client.19179 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f5102c1a740 20 client.19179 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f5102c1a740 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55846bc13830 0x55846bc15c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f5102c1a740 4 client.19179 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f5102c1a740 1 -- 172.21.15.18:0/258549915 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55846b895890 con 0x55846bc13830 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f50fd507640 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55846bc13830 0x55846bc15c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f50e17fa640 10 client.19179 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f50fd507640 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55846bc13830 0x55846bc15c20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a7c91572e61cc58a server_cookie=14ef9bfaaa149d09 in_seq=0 out_seq=0 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f50e17fa640 1 -- 172.21.15.18:0/258549915 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55846b895890 con 0x55846bc13830 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f50e17fa640 10 client.19179 handle_command_reply: tid=0 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f5102c1a740 1 client.19179 shutdown 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.204+0000 7f5102c1a740 1 -- 172.21.15.18:0/258549915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2250 msgr2=0x55846bbb1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f5102c1a740 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2250 0x55846bbb1090 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f50f4004430 tx=0x7f50f4004460 comp rx=0 tx=0).stop 2024-02-17T13:00:04.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f5102c1a740 1 -- 172.21.15.18:0/258549915 shutdown_connections 2024-02-17T13:00:04.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f5102c1a740 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55846bc13830 0x55846bc15c20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f5102c1a740 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55846bbab750 0x55846bbb1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f50e17fa640 0 client.19179 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:00:04.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f5102c1a740 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55846bba2b70 0x55846bbb15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f5102c1a740 1 --2- 172.21.15.18:0/258549915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846bba2250 0x55846bbb1090 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f5102c1a740 1 -- 172.21.15.18:0/258549915 >> 172.21.15.18:0/258549915 conn(0x55846bb9c980 msgr2=0x55846bb9bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f5102c1a740 1 -- 172.21.15.18:0/258549915 shutdown_connections 2024-02-17T13:00:04.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f5102c1a740 1 -- 172.21.15.18:0/258549915 wait complete. 2024-02-17T13:00:04.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f5102c1a740 20 client.19179 trim_cache size 0 max 16384 2024-02-17T13:00:04.200 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:00:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f50fff6c640 1 -- 172.21.15.18:0/142318698 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f50e40a22b0 msgr2=0x7f50e40a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f50fff6c640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f50e40a22b0 0x7f50e40a4770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f50f001e000 tx=0x7f50f0021000 comp rx=0 tx=0).stop 2024-02-17T13:00:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f50fff6c640 1 -- 172.21.15.18:0/142318698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50f8105330 msgr2=0x7f50f819dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f50fff6c640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50f8105330 0x7f50f819dbd0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f50e800e9f0 tx=0x7f50e800eeb0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f50fff6c640 1 -- 172.21.15.18:0/142318698 shutdown_connections 2024-02-17T13:00:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f50fff6c640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f50e40a22b0 0x7f50e40a4770 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f50fff6c640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50f8197750 0x7f50f8197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f50fff6c640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50f8105330 0x7f50f819dbd0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f50fff6c640 1 --2- 172.21.15.18:0/142318698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f8100260 0x7f50f819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.205+0000 7f50fff6c640 1 -- 172.21.15.18:0/142318698 >> 172.21.15.18:0/142318698 conn(0x7f50f8077180 msgr2=0x7f50f80687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.206+0000 7f50fff6c640 1 -- 172.21.15.18:0/142318698 shutdown_connections 2024-02-17T13:00:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.206+0000 7f50fff6c640 1 -- 172.21.15.18:0/142318698 wait complete. 2024-02-17T13:00:04.206 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:04 smithi018 ceph-mon[32625]: pgmap v317: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-02-17T13:00:04.206 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:04 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1761252504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:00:04.207 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:00:04.207 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T13:00:04.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 513867 5.04 MB/sec execute 439 sec latency 3395.242 ms 2024-02-17T13:00:04.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.343+0000 7f478a927640 1 -- 172.21.15.18:0/966872296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4784101380 msgr2=0x7f4784101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.343+0000 7f478a927640 1 --2- 172.21.15.18:0/966872296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4784101380 0x7f4784101760 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f4774009930 tx=0x7f477402bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:04.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.343+0000 7f478a927640 1 -- 172.21.15.18:0/966872296 shutdown_connections 2024-02-17T13:00:04.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.343+0000 7f478a927640 1 --2- 172.21.15.18:0/966872296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4784106450 0x7f478410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.343+0000 7f478a927640 1 --2- 172.21.15.18:0/966872296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4784101d30 0x7f4784105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.343+0000 7f478a927640 1 --2- 172.21.15.18:0/966872296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4784101380 0x7f4784101760 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.343+0000 7f478a927640 1 -- 172.21.15.18:0/966872296 >> 172.21.15.18:0/966872296 conn(0x7f4784078540 msgr2=0x7f4784078950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.343+0000 7f478a927640 1 -- 172.21.15.18:0/966872296 shutdown_connections 2024-02-17T13:00:04.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.343+0000 7f478a927640 1 -- 172.21.15.18:0/966872296 wait complete. 2024-02-17T13:00:04.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.344+0000 7f478a927640 1 Processor -- start 2024-02-17T13:00:04.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.344+0000 7f478a927640 1 -- start start 2024-02-17T13:00:04.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.344+0000 7f478a927640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4784101380 0x7f478419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.344+0000 7f478a927640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4784106450 0x7f478419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.344+0000 7f478a927640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4784197650 0x7f4784197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.344+0000 7f478a927640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4784198120 con 0x7f4784101380 2024-02-17T13:00:04.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.344+0000 7f478a927640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4784198290 con 0x7f4784197650 2024-02-17T13:00:04.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.344+0000 7f478a927640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4784198450 con 0x7f4784106450 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.344+0000 7f4783fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4784101380 0x7f478419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.344+0000 7f4783fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4784101380 0x7f478419d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55038/0 (socket says 172.21.15.18:55038) 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.344+0000 7f4783fff640 1 -- 172.21.15.18:0/1017296445 learned_addr learned my addr 172.21.15.18:0/1017296445 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.344+0000 7f47837fe640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4784106450 0x7f478419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f4788ec4640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4784197650 0x7f4784197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f4783fff640 1 -- 172.21.15.18:0/1017296445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4784106450 msgr2=0x7f478419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f4783fff640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4784106450 0x7f478419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f4783fff640 1 -- 172.21.15.18:0/1017296445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4784197650 msgr2=0x7f4784197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f4783fff640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4784197650 0x7f4784197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f4783fff640 1 -- 172.21.15.18:0/1017296445 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4774009560 con 0x7f4784101380 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f47837fe640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4784106450 0x7f478419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f4788ec4640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4784197650 0x7f4784197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f4783fff640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4784101380 0x7f478419d590 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f4774009230 tx=0x7f477402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f47817fa640 1 -- 172.21.15.18:0/1017296445 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774004350 con 0x7f4784101380 2024-02-17T13:00:04.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f47817fa640 1 -- 172.21.15.18:0/1017296445 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47740044b0 con 0x7f4784101380 2024-02-17T13:00:04.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f47817fa640 1 -- 172.21.15.18:0/1017296445 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774003d00 con 0x7f4784101380 2024-02-17T13:00:04.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f478a927640 1 -- 172.21.15.18:0/1017296445 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47841a4390 con 0x7f4784101380 2024-02-17T13:00:04.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.345+0000 7f478a927640 1 -- 172.21.15.18:0/1017296445 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47841a48c0 con 0x7f4784101380 2024-02-17T13:00:04.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.348+0000 7f47817fa640 1 -- 172.21.15.18:0/1017296445 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4774004df0 con 0x7f4784101380 2024-02-17T13:00:04.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f47817fa640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f476c0a22b0 0x7f476c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f478d5ce740 1 Processor -- start 2024-02-17T13:00:04.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f478d5ce740 1 -- start start 2024-02-17T13:00:04.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f47817fa640 1 -- 172.21.15.18:0/1017296445 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f47740eff70 con 0x7f4784101380 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f478d5ce740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e125fa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f478d5ce740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125faa60 0x559e125faee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f478d5ce740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125fb6a0 0x559e125fdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f478d5ce740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e125b1960 con 0x559e125faa60 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f478d5ce740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e12565310 con 0x559e125fb6a0 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f478d5ce740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e12562840 con 0x559e125fa0b0 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f47837fe640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f476c0a22b0 0x7f476c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f4783fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e125fa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f4783fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e125fa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34454/0 (socket says 172.21.15.18:34454) 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f4783fff640 1 -- 172.21.15.18:0/1175214020 learned_addr learned my addr 172.21.15.18:0/1175214020 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f47837fe640 1 --2- 172.21.15.18:0/1175214020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125faa60 0x559e125faee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.349+0000 7f4788ec4640 1 --2- 172.21.15.18:0/1175214020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125fb6a0 0x559e125fdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.350+0000 7f4783fff640 1 -- 172.21.15.18:0/1175214020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125fb6a0 msgr2=0x559e125fdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.350+0000 7f4783fff640 1 --2- 172.21.15.18:0/1175214020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125fb6a0 0x559e125fdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.350+0000 7f4783fff640 1 -- 172.21.15.18:0/1175214020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125faa60 msgr2=0x559e125faee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.350+0000 7f4783fff640 1 --2- 172.21.15.18:0/1175214020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125faa60 0x559e125faee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.350+0000 7f4783fff640 1 -- 172.21.15.18:0/1175214020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e12520060 con 0x559e125fa0b0 2024-02-17T13:00:04.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.350+0000 7f4783fff640 1 --2- 172.21.15.18:0/1175214020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e125fa490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47740b9460 tx=0x7f477402c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca14a4587a142adb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.350+0000 7f47837fe640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f476c0a22b0 0x7f476c0a4770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4784198ea0 tx=0x7f4778013c30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.350+0000 7f476a7fc640 1 -- 172.21.15.18:0/1175214020 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47740bca90 con 0x559e125fa0b0 2024-02-17T13:00:04.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.350+0000 7f476a7fc640 1 -- 172.21.15.18:0/1175214020 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47740c0be0 con 0x559e125fa0b0 2024-02-17T13:00:04.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.350+0000 7f476a7fc640 1 -- 172.21.15.18:0/1175214020 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47740c4da0 con 0x559e125fa0b0 2024-02-17T13:00:04.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:04 smithi119 ceph-mon[36188]: pgmap v317: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-02-17T13:00:04.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:04 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1761252504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:00:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.351+0000 7f478d5ce740 1 -- 172.21.15.18:0/1175214020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 msgr2=0x559e125fa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.351+0000 7f478d5ce740 1 --2- 172.21.15.18:0/1175214020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e125fa490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47740b9460 tx=0x7f477402c740 comp rx=0 tx=0).stop 2024-02-17T13:00:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.351+0000 7f478d5ce740 1 -- 172.21.15.18:0/1175214020 shutdown_connections 2024-02-17T13:00:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.351+0000 7f478d5ce740 1 --2- 172.21.15.18:0/1175214020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125fb6a0 0x559e125fdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.351+0000 7f478d5ce740 1 --2- 172.21.15.18:0/1175214020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125faa60 0x559e125faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.351+0000 7f478d5ce740 1 --2- 172.21.15.18:0/1175214020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e125fa490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.351+0000 7f478d5ce740 1 -- 172.21.15.18:0/1175214020 >> 172.21.15.18:0/1175214020 conn(0x559e125f4d90 msgr2=0x559e125f5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.351+0000 7f478d5ce740 1 -- 172.21.15.18:0/1175214020 shutdown_connections 2024-02-17T13:00:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.351+0000 7f478d5ce740 1 -- 172.21.15.18:0/1175214020 wait complete. 2024-02-17T13:00:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.351+0000 7f478d5ce740 1 -- start start 2024-02-17T13:00:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.352+0000 7f478d5ce740 1 Processor -- start 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.352+0000 7f478d5ce740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e12607110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.352+0000 7f478d5ce740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e12607650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.352+0000 7f478d5ce740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 0x559e125ff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.352+0000 7f4783fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e12607110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f47837fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e12607650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f47837fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e12607650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:45168/0 (socket says 172.21.15.18:45168) 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f4783fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e12607110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34464/0 (socket says 172.21.15.18:34464) 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f4788ec4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 0x559e125ff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f4788ec4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 0x559e125ff3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55052/0 (socket says 172.21.15.18:55052) 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f478d5ce740 1 -- 172.21.15.18:0/4120079842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1255e020 con 0x559e125fb6a0 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f47837fe640 1 -- 172.21.15.18:0/4120079842 learned_addr learned my addr 172.21.15.18:0/4120079842 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f478d5ce740 1 -- 172.21.15.18:0/4120079842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e12231430 con 0x559e125faa60 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f478d5ce740 1 -- 172.21.15.18:0/4120079842 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e121ede00 con 0x559e125fa0b0 2024-02-17T13:00:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f478d5ce740 10 client.0 _ll_register_callbacks cb 0x559e12505cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f4783fff640 1 -- 172.21.15.18:0/4120079842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 msgr2=0x559e12607650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f4783fff640 1 --2- 172.21.15.18:0/4120079842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e12607650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f4783fff640 1 -- 172.21.15.18:0/4120079842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 msgr2=0x559e125ff3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f4783fff640 1 --2- 172.21.15.18:0/4120079842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 0x559e125ff3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f4783fff640 1 -- 172.21.15.18:0/4120079842 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47841a48c0 con 0x559e125fa0b0 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f47837fe640 1 --2- 172.21.15.18:0/4120079842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e12607650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f4783fff640 1 --2- 172.21.15.18:0/4120079842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e12607110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4774005d00 tx=0x7f477402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74885e5bf7b61a5f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f47637fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.353+0000 7f47637fe640 1 -- 172.21.15.18:0/4120079842 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47740c4fa0 con 0x559e125fa0b0 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.354+0000 7f47637fe640 1 -- 172.21.15.18:0/4120079842 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f47740c5230 con 0x559e125fa0b0 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.354+0000 7f478d5ce740 1 -- 172.21.15.18:0/4120079842 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e1225d7e0 con 0x559e125fa0b0 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.354+0000 7f47637fe640 1 -- 172.21.15.18:0/4120079842 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47740bbee0 con 0x559e125fa0b0 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.354+0000 7f478d5ce740 10 client.28261 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.354+0000 7f478d5ce740 1 -- 172.21.15.18:0/4120079842 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x559e1255c6a0 con 0x559e125fa0b0 2024-02-17T13:00:04.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.354+0000 7f47637fe640 1 -- 172.21.15.18:0/4120079842 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559e1225d7e0 con 0x559e125fa0b0 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.355+0000 7f47637fe640 1 -- 172.21.15.18:0/4120079842 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f477403c070 con 0x559e125fa0b0 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.355+0000 7f478d5ce740 10 client.28261 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.355+0000 7f478d5ce740 10 client.28261 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.355+0000 7f478d5ce740 20 client.28261 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.355+0000 7f478d5ce740 1 --2- 172.21.15.18:0/4120079842 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559e12620400 0x559e126227f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.355+0000 7f478d5ce740 4 client.28261 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.355+0000 7f478d5ce740 1 -- 172.21.15.18:0/4120079842 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e1229a720 con 0x559e12620400 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.355+0000 7f47837fe640 1 --2- 172.21.15.18:0/4120079842 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559e12620400 0x559e126227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.355+0000 7f47637fe640 10 client.28261 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.355+0000 7f47837fe640 1 --2- 172.21.15.18:0/4120079842 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559e12620400 0x559e126227f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a2b879556f91b66 server_cookie=a2fa375ba6aa8750 in_seq=0 out_seq=0 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.356+0000 7f47637fe640 1 -- 172.21.15.18:0/4120079842 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559e1229a720 con 0x559e12620400 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.356+0000 7f47637fe640 10 client.28261 handle_command_reply: tid=0 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.356+0000 7f478d5ce740 1 client.28261 shutdown 2024-02-17T13:00:04.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.356+0000 7f478d5ce740 1 -- 172.21.15.18:0/4120079842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 msgr2=0x559e12607110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.356+0000 7f478d5ce740 1 --2- 172.21.15.18:0/4120079842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e12607110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4774005d00 tx=0x7f477402bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.356+0000 7f478d5ce740 1 -- 172.21.15.18:0/4120079842 shutdown_connections 2024-02-17T13:00:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.356+0000 7f478d5ce740 1 --2- 172.21.15.18:0/4120079842 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559e12620400 0x559e126227f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.356+0000 7f478d5ce740 1 --2- 172.21.15.18:0/4120079842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 0x559e125ff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.356+0000 7f478d5ce740 1 --2- 172.21.15.18:0/4120079842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e12607650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.356+0000 7f47637fe640 0 client.28261 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:00:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.356+0000 7f478d5ce740 1 --2- 172.21.15.18:0/4120079842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e12607110 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.356+0000 7f478d5ce740 1 -- 172.21.15.18:0/4120079842 >> 172.21.15.18:0/4120079842 conn(0x559e125f4d90 msgr2=0x559e125f6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.357+0000 7f478d5ce740 1 -- 172.21.15.18:0/4120079842 shutdown_connections 2024-02-17T13:00:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.357+0000 7f478d5ce740 1 -- 172.21.15.18:0/4120079842 wait complete. 2024-02-17T13:00:04.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.357+0000 7f478d5ce740 20 client.28261 trim_cache size 0 max 16384 2024-02-17T13:00:04.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f478d5ce740 1 Processor -- start 2024-02-17T13:00:04.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f478d5ce740 1 -- start start 2024-02-17T13:00:04.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f478d5ce740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e125ff360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f478d5ce740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e126306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f478d5ce740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 0x559e12630bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f478d5ce740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e124e3130 con 0x559e125fb6a0 2024-02-17T13:00:04.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f478d5ce740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e125fae40 con 0x559e125faa60 2024-02-17T13:00:04.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f478d5ce740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e126049e0 con 0x559e125fa0b0 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f4788ec4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 0x559e12630bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f4783fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e125ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f4783fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e125ff360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34480/0 (socket says 172.21.15.18:34480) 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f4788ec4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 0x559e12630bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55054/0 (socket says 172.21.15.18:55054) 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f4783fff640 1 -- 172.21.15.18:0/369395221 learned_addr learned my addr 172.21.15.18:0/369395221 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f47837fe640 1 --2- 172.21.15.18:0/369395221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e126306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f4783fff640 1 -- 172.21.15.18:0/369395221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 msgr2=0x559e126306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f4783fff640 1 --2- 172.21.15.18:0/369395221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e126306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f4783fff640 1 -- 172.21.15.18:0/369395221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 msgr2=0x559e12630bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.372+0000 7f4783fff640 1 --2- 172.21.15.18:0/369395221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 0x559e12630bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.373+0000 7f4783fff640 1 -- 172.21.15.18:0/369395221 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e124f8c20 con 0x559e125fa0b0 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.373+0000 7f4788ec4640 1 --2- 172.21.15.18:0/369395221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 0x559e12630bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:04.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.373+0000 7f4783fff640 1 --2- 172.21.15.18:0/369395221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e125ff360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f47740b8ef0 tx=0x7f4774034000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=234398bbc8b2c02b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.373+0000 7f4769ffb640 1 -- 172.21.15.18:0/369395221 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47740c5230 con 0x559e125fa0b0 2024-02-17T13:00:04.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.373+0000 7f4769ffb640 1 -- 172.21.15.18:0/369395221 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47740ed8a0 con 0x559e125fa0b0 2024-02-17T13:00:04.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.373+0000 7f4769ffb640 1 -- 172.21.15.18:0/369395221 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47740c4da0 con 0x559e125fa0b0 2024-02-17T13:00:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.374+0000 7f478d5ce740 1 -- 172.21.15.18:0/369395221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 msgr2=0x559e125ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.374+0000 7f478d5ce740 1 --2- 172.21.15.18:0/369395221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e125ff360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f47740b8ef0 tx=0x7f4774034000 comp rx=0 tx=0).stop 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.374+0000 7f478d5ce740 1 -- 172.21.15.18:0/369395221 shutdown_connections 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.374+0000 7f478d5ce740 1 --2- 172.21.15.18:0/369395221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fb6a0 0x559e12630bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.374+0000 7f478d5ce740 1 --2- 172.21.15.18:0/369395221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e126306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.374+0000 7f478d5ce740 1 --2- 172.21.15.18:0/369395221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fa0b0 0x559e125ff360 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.374+0000 7f478d5ce740 1 -- 172.21.15.18:0/369395221 >> 172.21.15.18:0/369395221 conn(0x559e125f4d90 msgr2=0x559e125fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.374+0000 7f478d5ce740 1 -- 172.21.15.18:0/369395221 shutdown_connections 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.374+0000 7f478d5ce740 1 -- 172.21.15.18:0/369395221 wait complete. 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.375+0000 7f478d5ce740 1 -- start start 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.375+0000 7f478d5ce740 1 Processor -- start 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.375+0000 7f478d5ce740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fa0b0 0x559e1266b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.375+0000 7f478d5ce740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e1266ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.375+0000 7f478d5ce740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fb6a0 0x559e12676390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.375+0000 7f4783fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fa0b0 0x559e1266b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.375+0000 7f478d5ce740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e1266c140 con 0x559e125fa0b0 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.375+0000 7f4783fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fa0b0 0x559e1266b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55070/0 (socket says 172.21.15.18:55070) 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.375+0000 7f4783fff640 1 -- 172.21.15.18:0/864439470 learned_addr learned my addr 172.21.15.18:0/864439470 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.375+0000 7f478d5ce740 1 -- 172.21.15.18:0/864439470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e126768d0 con 0x559e125faa60 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f478d5ce740 1 -- 172.21.15.18:0/864439470 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e12676a10 con 0x559e125fb6a0 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f47837fe640 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e1266ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f478d5ce740 10 client.0 _ll_register_callbacks cb 0x7f47740b8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f4788ec4640 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fb6a0 0x559e12676390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f4783fff640 1 -- 172.21.15.18:0/864439470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fb6a0 msgr2=0x559e12676390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f4783fff640 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fb6a0 0x559e12676390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f4783fff640 1 -- 172.21.15.18:0/864439470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 msgr2=0x559e1266ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f4783fff640 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e1266ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f4783fff640 1 -- 172.21.15.18:0/864439470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c009de0 con 0x559e125fa0b0 2024-02-17T13:00:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f47837fe640 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e1266ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f4783fff640 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fa0b0 0x559e1266b500 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f47740b9460 tx=0x7f47740037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efea7c9e57c227 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f47637fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f47637fe640 1 -- 172.21.15.18:0/864439470 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47740bc0e0 con 0x559e125fa0b0 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f47637fe640 1 -- 172.21.15.18:0/864439470 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f47740c5230 con 0x559e125fa0b0 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.376+0000 7f478d5ce740 1 -- 172.21.15.18:0/864439470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47740c0be0 con 0x559e125fa0b0 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.377+0000 7f47637fe640 1 -- 172.21.15.18:0/864439470 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47740edb20 con 0x559e125fa0b0 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.377+0000 7f478d5ce740 10 client.19197 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.377+0000 7f478d5ce740 1 -- 172.21.15.18:0/864439470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x559e1255c6a0 con 0x559e125fa0b0 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.377+0000 7f47637fe640 1 -- 172.21.15.18:0/864439470 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47740c0be0 con 0x559e125fa0b0 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.377+0000 7f47637fe640 1 -- 172.21.15.18:0/864439470 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f477403c070 con 0x559e125fa0b0 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.377+0000 7f478d5ce740 10 client.19197 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.377+0000 7f478d5ce740 10 client.19197 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.377+0000 7f478d5ce740 20 client.19197 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.377+0000 7f478d5ce740 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559e12677cb0 0x559e1267a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.377+0000 7f478d5ce740 4 client.19197 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.377+0000 7f478d5ce740 1 -- 172.21.15.18:0/864439470 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559e122e9890 con 0x559e12677cb0 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.377+0000 7f47837fe640 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559e12677cb0 0x559e1267a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f47837fe640 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559e12677cb0 0x559e1267a0a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c080feb7a7be40b server_cookie=2ec11dd95619a0a1 in_seq=0 out_seq=0 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f47637fe640 10 client.19197 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f47637fe640 1 -- 172.21.15.18:0/864439470 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559e122e9890 con 0x559e12677cb0 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f47637fe640 10 client.19197 handle_command_reply: tid=0 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f478d5ce740 1 client.19197 shutdown 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f478d5ce740 1 -- 172.21.15.18:0/864439470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fa0b0 msgr2=0x559e1266b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f478d5ce740 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fa0b0 0x559e1266b500 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f47740b9460 tx=0x7f47740037d0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f478d5ce740 1 -- 172.21.15.18:0/864439470 shutdown_connections 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f478d5ce740 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559e12677cb0 0x559e1267a0a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f47637fe640 0 client.19197 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f478d5ce740 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e125fb6a0 0x559e12676390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f478d5ce740 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e125faa60 0x559e1266ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f478d5ce740 1 --2- 172.21.15.18:0/864439470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559e125fa0b0 0x559e1266b500 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f478d5ce740 1 -- 172.21.15.18:0/864439470 >> 172.21.15.18:0/864439470 conn(0x559e125f4d90 msgr2=0x559e125ef2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f478d5ce740 1 -- 172.21.15.18:0/864439470 shutdown_connections 2024-02-17T13:00:04.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.378+0000 7f478d5ce740 1 -- 172.21.15.18:0/864439470 wait complete. 2024-02-17T13:00:04.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478d5ce740 20 client.19197 trim_cache size 0 max 16384 2024-02-17T13:00:04.375 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:00:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478a927640 1 -- 172.21.15.18:0/1017296445 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f476c0a22b0 msgr2=0x7f476c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478a927640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f476c0a22b0 0x7f476c0a4770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4784198ea0 tx=0x7f4778013c30 comp rx=0 tx=0).stop 2024-02-17T13:00:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478a927640 1 -- 172.21.15.18:0/1017296445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4784101380 msgr2=0x7f478419d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478a927640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4784101380 0x7f478419d590 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f4774009230 tx=0x7f477402c650 comp rx=0 tx=0).stop 2024-02-17T13:00:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478a927640 1 -- 172.21.15.18:0/1017296445 shutdown_connections 2024-02-17T13:00:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478a927640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f476c0a22b0 0x7f476c0a4770 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478a927640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4784197650 0x7f4784197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478a927640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4784106450 0x7f478419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478a927640 1 --2- 172.21.15.18:0/1017296445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4784101380 0x7f478419d590 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478a927640 1 -- 172.21.15.18:0/1017296445 >> 172.21.15.18:0/1017296445 conn(0x7f4784078540 msgr2=0x7f47840ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478a927640 1 -- 172.21.15.18:0/1017296445 shutdown_connections 2024-02-17T13:00:04.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.379+0000 7f478a927640 1 -- 172.21.15.18:0/1017296445 wait complete. 2024-02-17T13:00:04.381 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:00:04.382 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T13:00:04.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:04 smithi146 ceph-mon[36454]: pgmap v317: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-02-17T13:00:04.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:04 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1761252504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:00:04.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.502+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3511682290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a00685f0 msgr2=0x7fd1a00689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.502+0000 7fd1a5e9a640 1 --2- 172.21.15.18:0/3511682290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a00685f0 0x7fd1a00689d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd190009930 tx=0x7fd19002bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:04.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.502+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3511682290 shutdown_connections 2024-02-17T13:00:04.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.502+0000 7fd1a5e9a640 1 --2- 172.21.15.18:0/3511682290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1a01064c0 0x7fd1a010e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.502+0000 7fd1a5e9a640 1 --2- 172.21.15.18:0/3511682290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1a0068fa0 0x7fd1a0105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.502+0000 7fd1a5e9a640 1 --2- 172.21.15.18:0/3511682290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a00685f0 0x7fd1a00689d0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.502+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3511682290 >> 172.21.15.18:0/3511682290 conn(0x7fd1a0077180 msgr2=0x7fd1a0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.502+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3511682290 shutdown_connections 2024-02-17T13:00:04.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.502+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3511682290 wait complete. 2024-02-17T13:00:04.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.502+0000 7fd1a5e9a640 1 Processor -- start 2024-02-17T13:00:04.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd1a5e9a640 1 -- start start 2024-02-17T13:00:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd1a5e9a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a00685f0 0x7fd1a0100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd1a5e9a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1a01064c0 0x7fd1a0100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd1a5e9a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1a0100ac0 0x7fd1a01a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd1a5e9a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a01a5f40 con 0x7fd1a00685f0 2024-02-17T13:00:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd1a5e9a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a01a60b0 con 0x7fd1a0100ac0 2024-02-17T13:00:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd1a5e9a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a01a6270 con 0x7fd1a01064c0 2024-02-17T13:00:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd19f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a00685f0 0x7fd1a0100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd19f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a00685f0 0x7fd1a0100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55092/0 (socket says 172.21.15.18:55092) 2024-02-17T13:00:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd19f7fe640 1 -- 172.21.15.18:0/3840268317 learned_addr learned my addr 172.21.15.18:0/3840268317 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd19ffff640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1a0100ac0 0x7fd1a01a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd19effd640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1a01064c0 0x7fd1a0100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd19f7fe640 1 -- 172.21.15.18:0/3840268317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1a01064c0 msgr2=0x7fd1a0100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1a01064c0 0x7fd1a0100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd19f7fe640 1 -- 172.21.15.18:0/3840268317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1a0100ac0 msgr2=0x7fd1a01a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1a0100ac0 0x7fd1a01a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd19f7fe640 1 -- 172.21.15.18:0/3840268317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd190009560 con 0x7fd1a00685f0 2024-02-17T13:00:04.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.503+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a00685f0 0x7fd1a0100040 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fd190009230 tx=0x7fd19002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.504+0000 7fd19cff9640 1 -- 172.21.15.18:0/3840268317 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190004350 con 0x7fd1a00685f0 2024-02-17T13:00:04.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.504+0000 7fd19cff9640 1 -- 172.21.15.18:0/3840268317 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1900044b0 con 0x7fd1a00685f0 2024-02-17T13:00:04.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.504+0000 7fd19cff9640 1 -- 172.21.15.18:0/3840268317 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190003d70 con 0x7fd1a00685f0 2024-02-17T13:00:04.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.504+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3840268317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a01a64f0 con 0x7fd1a00685f0 2024-02-17T13:00:04.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.504+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3840268317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a01a6a00 con 0x7fd1a00685f0 2024-02-17T13:00:04.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.506+0000 7fd19cff9640 1 -- 172.21.15.18:0/3840268317 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd190004df0 con 0x7fd1a00685f0 2024-02-17T13:00:04.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.506+0000 7fd1a8a6d740 1 Processor -- start 2024-02-17T13:00:04.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd1a8a6d740 1 -- start start 2024-02-17T13:00:04.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19cff9640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd1880a2260 0x7fd1880a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd1a8a6d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 0x55acf4079630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd1a8a6d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4082210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd1a8a6d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4082750 0x55acf4084b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd1a8a6d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf4034960 con 0x55acf4082750 2024-02-17T13:00:04.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19ffff640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd1880a2260 0x7fd1880a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd1a8a6d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3fe8310 con 0x55acf4079b70 2024-02-17T13:00:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd1a8a6d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3fe5840 con 0x55acf4079250 2024-02-17T13:00:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19cff9640 1 -- 172.21.15.18:0/3840268317 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd1900effe0 con 0x7fd1a00685f0 2024-02-17T13:00:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 0x55acf4079630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4082210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 0x55acf4079630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34512/0 (socket says 172.21.15.18:34512) 2024-02-17T13:00:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19f7fe640 1 -- 172.21.15.18:0/3085113389 learned_addr learned my addr 172.21.15.18:0/3085113389 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19ffff640 1 --2- 172.21.15.18:0/3085113389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4082750 0x55acf4084b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19effd640 1 -- 172.21.15.18:0/3085113389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 msgr2=0x55acf4079630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19effd640 1 --2- 172.21.15.18:0/3085113389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 0x55acf4079630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19effd640 1 -- 172.21.15.18:0/3085113389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4082750 msgr2=0x55acf4084b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19effd640 1 --2- 172.21.15.18:0/3085113389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4082750 0x55acf4084b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19effd640 1 -- 172.21.15.18:0/3085113389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf3fa3060 con 0x55acf4079b70 2024-02-17T13:00:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/3085113389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 0x55acf4079630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.507+0000 7fd19ffff640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd1880a2260 0x7fd1880a4720 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd198005fd0 tx=0x7fd198013ac0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd19effd640 1 --2- 172.21.15.18:0/3085113389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4082210 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd194005d00 tx=0x7fd1940107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e209101cf9e13960 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd185ffb640 1 -- 172.21.15.18:0/3085113389 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19400bde0 con 0x55acf4079b70 2024-02-17T13:00:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd185ffb640 1 -- 172.21.15.18:0/3085113389 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd194015070 con 0x55acf4079b70 2024-02-17T13:00:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd185ffb640 1 -- 172.21.15.18:0/3085113389 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1940118a0 con 0x55acf4079b70 2024-02-17T13:00:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/3085113389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 msgr2=0x55acf4082210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/3085113389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4082210 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd194005d00 tx=0x7fd1940107a0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/3085113389 shutdown_connections 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/3085113389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4082750 0x55acf4084b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/3085113389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4082210 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/3085113389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 0x55acf4079630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/3085113389 >> 172.21.15.18:0/3085113389 conn(0x55acf4073980 msgr2=0x55acf4073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/3085113389 shutdown_connections 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.508+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/3085113389 wait complete. 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd1a8a6d740 1 -- start start 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd1a8a6d740 1 Processor -- start 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd1a8a6d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 0x55acf407f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd1a8a6d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079b70 0x55acf407fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd1a8a6d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4082750 0x55acf40801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd1a8a6d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3fe1020 con 0x55acf4079b70 2024-02-17T13:00:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd1a8a6d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3cb4430 con 0x55acf4082750 2024-02-17T13:00:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd1a8a6d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3c70e00 con 0x55acf4079250 2024-02-17T13:00:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd1a8a6d740 10 client.0 _ll_register_callbacks cb 0x55acf3f88cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd19effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079b70 0x55acf407fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd19f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 0x55acf407f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd19f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 0x55acf407f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34522/0 (socket says 172.21.15.18:34522) 2024-02-17T13:00:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd19effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079b70 0x55acf407fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55120/0 (socket says 172.21.15.18:55120) 2024-02-17T13:00:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd19effd640 1 -- 172.21.15.18:0/4277566960 learned_addr learned my addr 172.21.15.18:0/4277566960 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.509+0000 7fd19ffff640 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4082750 0x55acf40801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd19effd640 1 -- 172.21.15.18:0/4277566960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 msgr2=0x55acf407f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd19effd640 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 0x55acf407f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd19effd640 1 -- 172.21.15.18:0/4277566960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4082750 msgr2=0x55acf40801c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd19effd640 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4082750 0x55acf40801c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd19effd640 1 -- 172.21.15.18:0/4277566960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19400b8c0 con 0x55acf4079b70 2024-02-17T13:00:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 0x55acf407f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd19ffff640 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4082750 0x55acf40801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd19effd640 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079b70 0x55acf407fc80 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd19400bf50 tx=0x7fd19400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5de4567cfc81666 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd17effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd17effd640 1 -- 172.21.15.18:0/4277566960 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194021070 con 0x55acf4079b70 2024-02-17T13:00:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd17effd640 1 -- 172.21.15.18:0/4277566960 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd194017070 con 0x55acf4079b70 2024-02-17T13:00:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/4277566960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acf3ce07e0 con 0x55acf4079b70 2024-02-17T13:00:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd17effd640 1 -- 172.21.15.18:0/4277566960 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd194005260 con 0x55acf4079b70 2024-02-17T13:00:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd1a8a6d740 10 client.19209 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.510+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/4277566960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55acf3fdf6a0 con 0x55acf4079b70 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.511+0000 7fd17effd640 1 -- 172.21.15.18:0/4277566960 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd194032020 con 0x55acf4079b70 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.511+0000 7fd17effd640 1 -- 172.21.15.18:0/4277566960 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd194004a50 con 0x55acf4079b70 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.511+0000 7fd1a8a6d740 10 client.19209 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.511+0000 7fd1a8a6d740 10 client.19209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.511+0000 7fd1a8a6d740 20 client.19209 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.511+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55acf4092b10 0x55acf4094f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.511+0000 7fd1a8a6d740 4 client.19209 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.511+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/4277566960 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acf3d1d720 con 0x55acf4092b10 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.511+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55acf4092b10 0x55acf4094f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.511+0000 7fd17effd640 10 client.19209 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.511+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55acf4092b10 0x55acf4094f30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c0ab15ef3dc370f server_cookie=74cadb4a90dc4314 in_seq=0 out_seq=0 2024-02-17T13:00:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd17effd640 1 -- 172.21.15.18:0/4277566960 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55acf3d1d720 con 0x55acf4092b10 2024-02-17T13:00:04.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd17effd640 10 client.19209 handle_command_reply: tid=0 2024-02-17T13:00:04.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd1a8a6d740 1 client.19209 shutdown 2024-02-17T13:00:04.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/4277566960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079b70 msgr2=0x55acf407fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079b70 0x55acf407fc80 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd19400bf50 tx=0x7fd19400bf80 comp rx=0 tx=0).stop 2024-02-17T13:00:04.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/4277566960 shutdown_connections 2024-02-17T13:00:04.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55acf4092b10 0x55acf4094f30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4082750 0x55acf40801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079b70 0x55acf407fc80 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/4277566960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4079250 0x55acf407f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd17effd640 0 client.19209 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:00:04.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/4277566960 >> 172.21.15.18:0/4277566960 conn(0x55acf4073980 msgr2=0x55acf40756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/4277566960 shutdown_connections 2024-02-17T13:00:04.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/4277566960 wait complete. 2024-02-17T13:00:04.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.512+0000 7fd1a8a6d740 20 client.19209 trim_cache size 0 max 16384 2024-02-17T13:00:04.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.524+0000 7fd1a8a6d740 1 Processor -- start 2024-02-17T13:00:04.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.524+0000 7fd1a8a6d740 1 -- start start 2024-02-17T13:00:04.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.524+0000 7fd1a8a6d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 0x55acf407ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.524+0000 7fd1a8a6d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4072a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.524+0000 7fd1a8a6d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4082750 0x55acf4089600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.524+0000 7fd1a8a6d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf3f66130 con 0x55acf4079250 2024-02-17T13:00:04.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.524+0000 7fd1a8a6d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf40737b0 con 0x55acf4079b70 2024-02-17T13:00:04.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.524+0000 7fd1a8a6d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf409a140 con 0x55acf4082750 2024-02-17T13:00:04.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd19f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 0x55acf407ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd19f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 0x55acf407ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55124/0 (socket says 172.21.15.18:55124) 2024-02-17T13:00:04.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd19f7fe640 1 -- 172.21.15.18:0/2097632461 learned_addr learned my addr 172.21.15.18:0/2097632461 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd19effd640 1 --2- 172.21.15.18:0/2097632461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4072a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd19ffff640 1 --2- 172.21.15.18:0/2097632461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4082750 0x55acf4089600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd19f7fe640 1 -- 172.21.15.18:0/2097632461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4082750 msgr2=0x55acf4089600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/2097632461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4082750 0x55acf4089600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd19f7fe640 1 -- 172.21.15.18:0/2097632461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 msgr2=0x55acf4072a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/2097632461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4072a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd19f7fe640 1 -- 172.21.15.18:0/2097632461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf3f7bc20 con 0x55acf4079250 2024-02-17T13:00:04.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/2097632461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 0x55acf407ffe0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd190006010 tx=0x7fd19002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98539709ce762eaf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd1857fa640 1 -- 172.21.15.18:0/2097632461 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1900edea0 con 0x55acf4079250 2024-02-17T13:00:04.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd1857fa640 1 -- 172.21.15.18:0/2097632461 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1900c0c50 con 0x55acf4079250 2024-02-17T13:00:04.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.525+0000 7fd1857fa640 1 -- 172.21.15.18:0/2097632461 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1900bac30 con 0x55acf4079250 2024-02-17T13:00:04.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.526+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/2097632461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 msgr2=0x55acf407ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.526+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/2097632461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 0x55acf407ffe0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd190006010 tx=0x7fd19002bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:04.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.526+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/2097632461 shutdown_connections 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.526+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/2097632461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4082750 0x55acf4089600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.526+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/2097632461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4072a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.526+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/2097632461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 0x55acf407ffe0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.526+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/2097632461 >> 172.21.15.18:0/2097632461 conn(0x55acf4073980 msgr2=0x55acf40722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.526+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/2097632461 shutdown_connections 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.526+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/2097632461 wait complete. 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.526+0000 7fd1a8a6d740 1 -- start start 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.526+0000 7fd1a8a6d740 1 Processor -- start 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd1a8a6d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 0x55acf4088740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd1a8a6d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4088c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd1a8a6d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4082750 0x55acf40e9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd1a8a6d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf4089410 con 0x55acf4079250 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd1a8a6d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf40e95b0 con 0x55acf4079b70 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd1a8a6d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf40e9770 con 0x55acf4082750 2024-02-17T13:00:04.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd1a8a6d740 10 client.0 _ll_register_callbacks cb 0x7fd19400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd19f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 0x55acf4088740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd19f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 0x55acf4088740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55136/0 (socket says 172.21.15.18:55136) 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd19f7fe640 1 -- 172.21.15.18:0/588087341 learned_addr learned my addr 172.21.15.18:0/588087341 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd19effd640 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4088c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd19ffff640 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4082750 0x55acf40e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd19f7fe640 1 -- 172.21.15.18:0/588087341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4082750 msgr2=0x55acf40e9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4082750 0x55acf40e9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd19f7fe640 1 -- 172.21.15.18:0/588087341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 msgr2=0x55acf4088c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4088c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd19f7fe640 1 -- 172.21.15.18:0/588087341 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a01a64f0 con 0x55acf4079250 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd19f7fe640 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 0x55acf4088740 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fd1900edd50 tx=0x7fd190003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59af2d809122608 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd17effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd17effd640 1 -- 172.21.15.18:0/588087341 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1900d4070 con 0x55acf4079250 2024-02-17T13:00:04.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd17effd640 1 -- 172.21.15.18:0/588087341 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd1900c52a0 con 0x55acf4079250 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.527+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/588087341 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd194015070 con 0x55acf4079250 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd17effd640 1 -- 172.21.15.18:0/588087341 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1900baba0 con 0x55acf4079250 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd1a8a6d740 10 client.19215 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/588087341 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55acf3fa18c0 con 0x55acf4079250 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd17effd640 1 -- 172.21.15.18:0/588087341 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd194015070 con 0x55acf4079250 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd17effd640 1 -- 172.21.15.18:0/588087341 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd19003c070 con 0x55acf4079250 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd1a8a6d740 10 client.19215 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd1a8a6d740 10 client.19215 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd1a8a6d740 20 client.19215 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55acf40eac60 0x55acf40ed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd1a8a6d740 4 client.19215 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/588087341 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55acf3d6c890 con 0x55acf40eac60 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd19effd640 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55acf40eac60 0x55acf40ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd19effd640 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55acf40eac60 0x55acf40ed050 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ee03b8beb4ba4db server_cookie=87965c7722c9f605 in_seq=0 out_seq=0 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.528+0000 7fd17effd640 10 client.19215 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:00:04.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd17effd640 1 -- 172.21.15.18:0/588087341 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55acf3d6c890 con 0x55acf40eac60 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd17effd640 10 client.19215 handle_command_reply: tid=0 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a8a6d740 1 client.19215 shutdown 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/588087341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 msgr2=0x55acf4088740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 0x55acf4088740 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fd1900edd50 tx=0x7fd190003ef0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/588087341 shutdown_connections 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55acf40eac60 0x55acf40ed050 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55acf4082750 0x55acf40e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acf4079b70 0x55acf4088c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd17effd640 0 client.19215 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a8a6d740 1 --2- 172.21.15.18:0/588087341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55acf4079250 0x55acf4088740 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/588087341 >> 172.21.15.18:0/588087341 conn(0x55acf4073980 msgr2=0x55acf4072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/588087341 shutdown_connections 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a8a6d740 1 -- 172.21.15.18:0/588087341 wait complete. 2024-02-17T13:00:04.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a8a6d740 20 client.19215 trim_cache size 0 max 16384 2024-02-17T13:00:04.527 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:00:04.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3840268317 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd1880a2260 msgr2=0x7fd1880a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a5e9a640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd1880a2260 0x7fd1880a4720 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd198005fd0 tx=0x7fd198013ac0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3840268317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a00685f0 msgr2=0x7fd1a0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a5e9a640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a00685f0 0x7fd1a0100040 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fd190009230 tx=0x7fd19002c6c0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3840268317 shutdown_connections 2024-02-17T13:00:04.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a5e9a640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd1880a2260 0x7fd1880a4720 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a5e9a640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1a0100ac0 0x7fd1a01a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a5e9a640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1a01064c0 0x7fd1a0100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a5e9a640 1 --2- 172.21.15.18:0/3840268317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a00685f0 0x7fd1a0100040 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.529+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3840268317 >> 172.21.15.18:0/3840268317 conn(0x7fd1a0077180 msgr2=0x7fd1a00fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.530+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3840268317 shutdown_connections 2024-02-17T13:00:04.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.530+0000 7fd1a5e9a640 1 -- 172.21.15.18:0/3840268317 wait complete. 2024-02-17T13:00:04.532 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:00:04.532 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T13:00:04.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.652+0000 7ff012513640 1 -- 172.21.15.18:0/350649462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c100260 msgr2=0x7ff00c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.652+0000 7ff012513640 1 --2- 172.21.15.18:0/350649462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c100260 0x7ff00c100640 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7feffc009930 tx=0x7feffc02bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:04.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.652+0000 7ff012513640 1 -- 172.21.15.18:0/350649462 shutdown_connections 2024-02-17T13:00:04.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.652+0000 7ff012513640 1 --2- 172.21.15.18:0/350649462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00c105330 0x7ff00c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.652+0000 7ff012513640 1 --2- 172.21.15.18:0/350649462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff00c100c10 0x7ff00c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.652+0000 7ff012513640 1 --2- 172.21.15.18:0/350649462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c100260 0x7ff00c100640 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.652+0000 7ff012513640 1 -- 172.21.15.18:0/350649462 >> 172.21.15.18:0/350649462 conn(0x7ff00c077180 msgr2=0x7ff00c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.652+0000 7ff012513640 1 -- 172.21.15.18:0/350649462 shutdown_connections 2024-02-17T13:00:04.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.652+0000 7ff012513640 1 -- 172.21.15.18:0/350649462 wait complete. 2024-02-17T13:00:04.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.652+0000 7ff012513640 1 Processor -- start 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.652+0000 7ff012513640 1 -- start start 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff012513640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00c100260 0x7ff00c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff012513640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff00c105330 0x7ff00c19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff012513640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c1975f0 0x7ff00c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff012513640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c197fe0 con 0x7ff00c1975f0 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff012513640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c198150 con 0x7ff00c100260 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff012513640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c198310 con 0x7ff00c105330 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff010ab0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c1975f0 0x7ff00c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff010ab0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c1975f0 0x7ff00c197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55164/0 (socket says 172.21.15.18:55164) 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff010ab0640 1 -- 172.21.15.18:0/334946986 learned_addr learned my addr 172.21.15.18:0/334946986 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff00b7fe640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff00c105330 0x7ff00c19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff00bfff640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00c100260 0x7ff00c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff010ab0640 1 -- 172.21.15.18:0/334946986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff00c105330 msgr2=0x7ff00c19d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff010ab0640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff00c105330 0x7ff00c19d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff010ab0640 1 -- 172.21.15.18:0/334946986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00c100260 msgr2=0x7ff00c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff010ab0640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00c100260 0x7ff00c19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff010ab0640 1 -- 172.21.15.18:0/334946986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc009560 con 0x7ff00c1975f0 2024-02-17T13:00:04.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.653+0000 7ff010ab0640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c1975f0 0x7ff00c197aa0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7ff00400eb10 tx=0x7ff00400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.654+0000 7ff0097fa640 1 -- 172.21.15.18:0/334946986 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004004b20 con 0x7ff00c1975f0 2024-02-17T13:00:04.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.654+0000 7ff0097fa640 1 -- 172.21.15.18:0/334946986 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff004004c80 con 0x7ff00c1975f0 2024-02-17T13:00:04.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.654+0000 7ff012513640 1 -- 172.21.15.18:0/334946986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00c1a4300 con 0x7ff00c1975f0 2024-02-17T13:00:04.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.654+0000 7ff0097fa640 1 -- 172.21.15.18:0/334946986 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004010650 con 0x7ff00c1975f0 2024-02-17T13:00:04.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.654+0000 7ff012513640 1 -- 172.21.15.18:0/334946986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00c1a47c0 con 0x7ff00c1975f0 2024-02-17T13:00:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.656+0000 7ff0097fa640 1 -- 172.21.15.18:0/334946986 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff004005020 con 0x7ff00c1975f0 2024-02-17T13:00:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.656+0000 7ff0097fa640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7feff40a2230 0x7feff40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.656+0000 7ff0151c7740 1 Processor -- start 2024-02-17T13:00:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.656+0000 7ff0151c7740 1 -- start start 2024-02-17T13:00:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.656+0000 7ff00bfff640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7feff40a2230 0x7feff40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.656+0000 7ff0097fa640 1 -- 172.21.15.18:0/334946986 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff004014070 con 0x7ff00c1975f0 2024-02-17T13:00:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.656+0000 7ff0151c7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442250 0x5639d8442630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.656+0000 7ff0151c7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442b70 0x5639d844b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff0151c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d844b750 0x5639d844db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff0151c7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d83fd960 con 0x5639d8442250 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff0151c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d83b1310 con 0x5639d844b750 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff0151c7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d83ae840 con 0x5639d8442b70 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff00bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442250 0x5639d8442630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff00b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442b70 0x5639d844b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff010ab0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d844b750 0x5639d844db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff00b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442b70 0x5639d844b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34586/0 (socket says 172.21.15.18:34586) 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff00b7fe640 1 -- 172.21.15.18:0/1127082586 learned_addr learned my addr 172.21.15.18:0/1127082586 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff00bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442250 0x5639d8442630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55172/0 (socket says 172.21.15.18:55172) 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff00b7fe640 1 -- 172.21.15.18:0/1127082586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d844b750 msgr2=0x5639d844db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff00b7fe640 1 --2- 172.21.15.18:0/1127082586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d844b750 0x5639d844db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff00b7fe640 1 -- 172.21.15.18:0/1127082586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442250 msgr2=0x5639d8442630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff00b7fe640 1 --2- 172.21.15.18:0/1127082586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442250 0x5639d8442630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff00b7fe640 1 -- 172.21.15.18:0/1127082586 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639d836c060 con 0x5639d8442b70 2024-02-17T13:00:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff00bfff640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7feff40a2230 0x7feff40a46f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feffc005ab0 tx=0x7feffc0046c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7ff00b7fe640 1 --2- 172.21.15.18:0/1127082586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442b70 0x5639d844b210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff000009c20 tx=0x7ff000011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aa6b979651c1ccb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7feff27fc640 1 -- 172.21.15.18:0/1127082586 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000004230 con 0x5639d8442b70 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7feff27fc640 1 -- 172.21.15.18:0/1127082586 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff000009e90 con 0x5639d8442b70 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.657+0000 7feff27fc640 1 -- 172.21.15.18:0/1127082586 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000004c60 con 0x5639d8442b70 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.658+0000 7ff0151c7740 1 -- 172.21.15.18:0/1127082586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442b70 msgr2=0x5639d844b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.658+0000 7ff0151c7740 1 --2- 172.21.15.18:0/1127082586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442b70 0x5639d844b210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff000009c20 tx=0x7ff000011810 comp rx=0 tx=0).stop 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.658+0000 7ff0151c7740 1 -- 172.21.15.18:0/1127082586 shutdown_connections 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.658+0000 7ff0151c7740 1 --2- 172.21.15.18:0/1127082586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d844b750 0x5639d844db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.658+0000 7ff0151c7740 1 --2- 172.21.15.18:0/1127082586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442b70 0x5639d844b210 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.658+0000 7ff0151c7740 1 --2- 172.21.15.18:0/1127082586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442250 0x5639d8442630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.658+0000 7ff0151c7740 1 -- 172.21.15.18:0/1127082586 >> 172.21.15.18:0/1127082586 conn(0x5639d843c980 msgr2=0x5639d843cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.658+0000 7ff0151c7740 1 -- 172.21.15.18:0/1127082586 shutdown_connections 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.658+0000 7ff0151c7740 1 -- 172.21.15.18:0/1127082586 wait complete. 2024-02-17T13:00:04.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.658+0000 7ff0151c7740 1 -- start start 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff0151c7740 1 Processor -- start 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff0151c7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d8448760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff0151c7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442b70 0x5639d8448ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff0151c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d844b750 0x5639d84491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff0151c7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d83aa020 con 0x5639d8442b70 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff0151c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d807d430 con 0x5639d844b750 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff0151c7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d8039e00 con 0x5639d8442250 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff0151c7740 10 client.0 _ll_register_callbacks cb 0x5639d8351cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff00b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442b70 0x5639d8448ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff00bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d8448760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff00b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442b70 0x5639d8448ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55174/0 (socket says 172.21.15.18:55174) 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff00b7fe640 1 -- 172.21.15.18:0/3676646758 learned_addr learned my addr 172.21.15.18:0/3676646758 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff010ab0640 1 --2- 172.21.15.18:0/3676646758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d844b750 0x5639d84491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff00b7fe640 1 -- 172.21.15.18:0/3676646758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 msgr2=0x5639d8448760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff00b7fe640 1 --2- 172.21.15.18:0/3676646758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d8448760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff00b7fe640 1 -- 172.21.15.18:0/3676646758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d844b750 msgr2=0x5639d84491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff00b7fe640 1 --2- 172.21.15.18:0/3676646758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d844b750 0x5639d84491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.659+0000 7ff00b7fe640 1 -- 172.21.15.18:0/3676646758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff000009850 con 0x5639d8442b70 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7ff00b7fe640 1 --2- 172.21.15.18:0/3676646758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442b70 0x5639d8448ca0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7ff000004620 tx=0x7ff000019af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77718b752130f195 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7fefeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7fefeb7fe640 1 -- 172.21.15.18:0/3676646758 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000024470 con 0x5639d8442b70 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7fefeb7fe640 1 -- 172.21.15.18:0/3676646758 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff0000245d0 con 0x5639d8442b70 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7ff0151c7740 1 -- 172.21.15.18:0/3676646758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639d80a97e0 con 0x5639d8442b70 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7fefeb7fe640 1 -- 172.21.15.18:0/3676646758 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff000024790 con 0x5639d8442b70 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7ff0151c7740 10 client.19227 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7ff0151c7740 1 -- 172.21.15.18:0/3676646758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5639d83a86a0 con 0x5639d8442b70 2024-02-17T13:00:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7fefeb7fe640 1 -- 172.21.15.18:0/3676646758 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5639d80a97e0 con 0x5639d8442b70 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7fefeb7fe640 1 -- 172.21.15.18:0/3676646758 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff000023960 con 0x5639d8442b70 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7ff0151c7740 10 client.19227 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7ff0151c7740 10 client.19227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7ff0151c7740 20 client.19227 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7ff0151c7740 1 --2- 172.21.15.18:0/3676646758 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5639d845ff50 0x5639d8462370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7ff0151c7740 4 client.19227 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.660+0000 7ff0151c7740 1 -- 172.21.15.18:0/3676646758 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639d80e6720 con 0x5639d845ff50 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.661+0000 7ff00bfff640 1 --2- 172.21.15.18:0/3676646758 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5639d845ff50 0x5639d8462370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.661+0000 7fefeb7fe640 10 client.19227 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.661+0000 7ff00bfff640 1 --2- 172.21.15.18:0/3676646758 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5639d845ff50 0x5639d8462370 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca9a5daa8e90b318 server_cookie=89be83884e62f625 in_seq=0 out_seq=0 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.661+0000 7fefeb7fe640 1 -- 172.21.15.18:0/3676646758 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639d80e6720 con 0x5639d845ff50 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.661+0000 7fefeb7fe640 10 client.19227 handle_command_reply: tid=0 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.661+0000 7ff0151c7740 1 client.19227 shutdown 2024-02-17T13:00:04.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.662+0000 7ff0151c7740 1 -- 172.21.15.18:0/3676646758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442b70 msgr2=0x5639d8448ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.662+0000 7ff0151c7740 1 --2- 172.21.15.18:0/3676646758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442b70 0x5639d8448ca0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7ff000004620 tx=0x7ff000019af0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.662+0000 7ff0151c7740 1 -- 172.21.15.18:0/3676646758 shutdown_connections 2024-02-17T13:00:04.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.662+0000 7ff0151c7740 1 --2- 172.21.15.18:0/3676646758 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5639d845ff50 0x5639d8462370 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.662+0000 7ff0151c7740 1 --2- 172.21.15.18:0/3676646758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d844b750 0x5639d84491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.662+0000 7ff0151c7740 1 --2- 172.21.15.18:0/3676646758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d8442b70 0x5639d8448ca0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.662+0000 7ff0151c7740 1 --2- 172.21.15.18:0/3676646758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d8448760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.662+0000 7fefeb7fe640 0 client.19227 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:04.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.662+0000 7ff0151c7740 1 -- 172.21.15.18:0/3676646758 >> 172.21.15.18:0/3676646758 conn(0x5639d843c980 msgr2=0x5639d843e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.662+0000 7ff0151c7740 1 -- 172.21.15.18:0/3676646758 shutdown_connections 2024-02-17T13:00:04.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.662+0000 7ff0151c7740 1 -- 172.21.15.18:0/3676646758 wait complete. 2024-02-17T13:00:04.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.662+0000 7ff0151c7740 20 client.19227 trim_cache size 0 max 16384 2024-02-17T13:00:04.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff0151c7740 1 Processor -- start 2024-02-17T13:00:04.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff0151c7740 1 -- start start 2024-02-17T13:00:04.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff0151c7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d844ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff0151c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d8442b70 0x5639d8449000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff0151c7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 0x5639d8466cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff0151c7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d832f130 con 0x5639d844b750 2024-02-17T13:00:04.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff0151c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d845ca50 con 0x5639d8442b70 2024-02-17T13:00:04.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff0151c7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d8444380 con 0x5639d8442250 2024-02-17T13:00:04.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff010ab0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 0x5639d8466cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff010ab0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 0x5639d8466cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55192/0 (socket says 172.21.15.18:55192) 2024-02-17T13:00:04.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff00b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d8442b70 0x5639d8449000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff010ab0640 1 -- 172.21.15.18:0/233325170 learned_addr learned my addr 172.21.15.18:0/233325170 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.674+0000 7ff00bfff640 1 --2- 172.21.15.18:0/233325170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d844ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.675+0000 7ff010ab0640 1 -- 172.21.15.18:0/233325170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 msgr2=0x5639d844ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.675+0000 7ff010ab0640 1 --2- 172.21.15.18:0/233325170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d844ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.675+0000 7ff010ab0640 1 -- 172.21.15.18:0/233325170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d8442b70 msgr2=0x5639d8449000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.675+0000 7ff010ab0640 1 --2- 172.21.15.18:0/233325170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d8442b70 0x5639d8449000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.675+0000 7ff010ab0640 1 -- 172.21.15.18:0/233325170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639d8344c20 con 0x5639d844b750 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.675+0000 7ff00b7fe640 1 --2- 172.21.15.18:0/233325170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d8442b70 0x5639d8449000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.675+0000 7ff010ab0640 1 --2- 172.21.15.18:0/233325170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 0x5639d8466cf0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff004097450 tx=0x7ff0040ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76796197dc08bfb5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.675+0000 7feff1ffb640 1 -- 172.21.15.18:0/233325170 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00409f140 con 0x5639d844b750 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.675+0000 7feff1ffb640 1 -- 172.21.15.18:0/233325170 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0040a03e0 con 0x5639d844b750 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.675+0000 7feff1ffb640 1 -- 172.21.15.18:0/233325170 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00409b250 con 0x5639d844b750 2024-02-17T13:00:04.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 -- 172.21.15.18:0/233325170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 msgr2=0x5639d8466cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 --2- 172.21.15.18:0/233325170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 0x5639d8466cf0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff004097450 tx=0x7ff0040ce040 comp rx=0 tx=0).stop 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 -- 172.21.15.18:0/233325170 shutdown_connections 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 --2- 172.21.15.18:0/233325170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 0x5639d8466cf0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 --2- 172.21.15.18:0/233325170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d8442b70 0x5639d8449000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 --2- 172.21.15.18:0/233325170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d844ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 -- 172.21.15.18:0/233325170 >> 172.21.15.18:0/233325170 conn(0x5639d843c980 msgr2=0x5639d8437a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 -- 172.21.15.18:0/233325170 shutdown_connections 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 -- 172.21.15.18:0/233325170 wait complete. 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 -- start start 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 Processor -- start 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d84558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d8442b70 0x5639d8455e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.676+0000 7ff0151c7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 0x5639d84b6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff0151c7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d8456710 con 0x5639d844b750 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff0151c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d84b69c0 con 0x5639d8442b70 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff0151c7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d84b6b00 con 0x5639d8442250 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff0151c7740 10 client.0 _ll_register_callbacks cb 0x7ff000009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff010ab0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 0x5639d84b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff00bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d84558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff010ab0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 0x5639d84b6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55194/0 (socket says 172.21.15.18:55194) 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff00bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d84558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34604/0 (socket says 172.21.15.18:34604) 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff00b7fe640 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d8442b70 0x5639d8455e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff010ab0640 1 -- 172.21.15.18:0/1975644301 learned_addr learned my addr 172.21.15.18:0/1975644301 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff010ab0640 1 -- 172.21.15.18:0/1975644301 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 msgr2=0x5639d84558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff010ab0640 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d84558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff010ab0640 1 -- 172.21.15.18:0/1975644301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d8442b70 msgr2=0x5639d8455e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:04.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff010ab0640 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d8442b70 0x5639d8455e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff010ab0640 1 -- 172.21.15.18:0/1975644301 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639d83a86a0 con 0x5639d844b750 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff00b7fe640 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d8442b70 0x5639d8455e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff010ab0640 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 0x5639d84b6480 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff0040979c0 tx=0x7ff004004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b581b738d843e8d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7fefeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7fefeb7fe640 1 -- 172.21.15.18:0/1975644301 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00409f140 con 0x5639d844b750 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7fefeb7fe640 1 -- 172.21.15.18:0/1975644301 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff00409b250 con 0x5639d844b750 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff0151c7740 1 -- 172.21.15.18:0/1975644301 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff000009e90 con 0x5639d844b750 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7fefeb7fe640 1 -- 172.21.15.18:0/1975644301 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff00409b410 con 0x5639d844b750 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff0151c7740 10 client.19233 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.677+0000 7ff0151c7740 1 -- 172.21.15.18:0/1975644301 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7ff00c1a47c0 con 0x5639d844b750 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7fefeb7fe640 1 -- 172.21.15.18:0/1975644301 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0040a7020 con 0x5639d844b750 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7fefeb7fe640 1 -- 172.21.15.18:0/1975644301 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff0040b3d20 con 0x5639d844b750 2024-02-17T13:00:04.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7ff0151c7740 10 client.19233 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7ff0151c7740 10 client.19233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7ff0151c7740 20 client.19233 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7ff0151c7740 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5639d84b8040 0x5639d84ba460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7ff0151c7740 4 client.19233 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7ff0151c7740 1 -- 172.21.15.18:0/1975644301 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5639d8135890 con 0x5639d84b8040 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7ff00bfff640 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5639d84b8040 0x5639d84ba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7ff00bfff640 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5639d84b8040 0x5639d84ba460 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e58fb01ae2aa34c4 server_cookie=43ac464837c474d3 in_seq=0 out_seq=0 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7fefeb7fe640 10 client.19233 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7fefeb7fe640 1 -- 172.21.15.18:0/1975644301 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5639d8135890 con 0x5639d84b8040 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7fefeb7fe640 10 client.19233 handle_command_reply: tid=0 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.678+0000 7ff0151c7740 1 client.19233 shutdown 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff0151c7740 1 -- 172.21.15.18:0/1975644301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 msgr2=0x5639d84b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff0151c7740 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 0x5639d84b6480 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff0040979c0 tx=0x7ff004004a40 comp rx=0 tx=0).stop 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff0151c7740 1 -- 172.21.15.18:0/1975644301 shutdown_connections 2024-02-17T13:00:04.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff0151c7740 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5639d84b8040 0x5639d84ba460 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff0151c7740 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5639d844b750 0x5639d84b6480 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff0151c7740 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639d8442b70 0x5639d8455e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff0151c7740 1 --2- 172.21.15.18:0/1975644301 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5639d8442250 0x5639d84558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7fefeb7fe640 0 client.19233 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:04.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff0151c7740 1 -- 172.21.15.18:0/1975644301 >> 172.21.15.18:0/1975644301 conn(0x5639d843c980 msgr2=0x5639d8466cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff0151c7740 1 -- 172.21.15.18:0/1975644301 shutdown_connections 2024-02-17T13:00:04.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff0151c7740 1 -- 172.21.15.18:0/1975644301 wait complete. 2024-02-17T13:00:04.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff0151c7740 20 client.19233 trim_cache size 0 max 16384 2024-02-17T13:00:04.676 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:00:04.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff012513640 1 -- 172.21.15.18:0/334946986 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7feff40a2230 msgr2=0x7feff40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff012513640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7feff40a2230 0x7feff40a46f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feffc005ab0 tx=0x7feffc0046c0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff012513640 1 -- 172.21.15.18:0/334946986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c1975f0 msgr2=0x7ff00c197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:04.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff012513640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c1975f0 0x7ff00c197aa0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7ff00400eb10 tx=0x7ff00400c2d0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff012513640 1 -- 172.21.15.18:0/334946986 shutdown_connections 2024-02-17T13:00:04.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff012513640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7feff40a2230 0x7feff40a46f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff012513640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c1975f0 0x7ff00c197aa0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff012513640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff00c105330 0x7ff00c19d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff012513640 1 --2- 172.21.15.18:0/334946986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00c100260 0x7ff00c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:04.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.679+0000 7ff012513640 1 -- 172.21.15.18:0/334946986 >> 172.21.15.18:0/334946986 conn(0x7ff00c077180 msgr2=0x7ff00c0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:04.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.680+0000 7ff012513640 1 -- 172.21.15.18:0/334946986 shutdown_connections 2024-02-17T13:00:04.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:04.680+0000 7ff012513640 1 -- 172.21.15.18:0/334946986 wait complete. 2024-02-17T13:00:05.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 515360 5.06 MB/sec execute 440 sec latency 23.476 ms 2024-02-17T13:00:05.680 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T13:00:05.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:00:05.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T13:00:05.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.808+0000 7f52c56f4640 1 -- 172.21.15.18:0/1912237638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c0101380 msgr2=0x7f52c0101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.808+0000 7f52c56f4640 1 --2- 172.21.15.18:0/1912237638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c0101380 0x7f52c0101760 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009930 tx=0x7f52b002bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:05.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.808+0000 7f52c56f4640 1 -- 172.21.15.18:0/1912237638 shutdown_connections 2024-02-17T13:00:05.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.808+0000 7f52c56f4640 1 --2- 172.21.15.18:0/1912237638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c0106450 0x7f52c010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.808+0000 7f52c56f4640 1 --2- 172.21.15.18:0/1912237638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52c0101d30 0x7f52c0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.808+0000 7f52c56f4640 1 --2- 172.21.15.18:0/1912237638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c0101380 0x7f52c0101760 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.809+0000 7f52c56f4640 1 -- 172.21.15.18:0/1912237638 >> 172.21.15.18:0/1912237638 conn(0x7f52c00fcc20 msgr2=0x7f52c00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:05.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.809+0000 7f52c56f4640 1 -- 172.21.15.18:0/1912237638 shutdown_connections 2024-02-17T13:00:05.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.809+0000 7f52c56f4640 1 -- 172.21.15.18:0/1912237638 wait complete. 2024-02-17T13:00:05.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.809+0000 7f52c56f4640 1 Processor -- start 2024-02-17T13:00:05.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.809+0000 7f52c56f4640 1 -- start start 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.809+0000 7f52c56f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c0101380 0x7f52c019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52c56f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52c0106450 0x7f52c019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52c56f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c0197680 0x7f52c0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52c56f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0198150 con 0x7f52c0197680 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52c56f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c01982c0 con 0x7f52c0106450 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52c56f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0198480 con 0x7f52c0101380 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52beffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c0101380 0x7f52c019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52c0106450 0x7f52c019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52beffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c0101380 0x7f52c019d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34630/0 (socket says 172.21.15.18:34630) 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52beffd640 1 -- 172.21.15.18:0/516211009 learned_addr learned my addr 172.21.15.18:0/516211009 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52bf7fe640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c0197680 0x7f52c0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52beffd640 1 -- 172.21.15.18:0/516211009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52c0106450 msgr2=0x7f52c019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52beffd640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52c0106450 0x7f52c019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52beffd640 1 -- 172.21.15.18:0/516211009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c0197680 msgr2=0x7f52c0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52beffd640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c0197680 0x7f52c0197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52beffd640 1 -- 172.21.15.18:0/516211009 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b0009560 con 0x7f52c0101380 2024-02-17T13:00:05.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52bf7fe640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c0197680 0x7f52c0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:05.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52b7fff640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52c0106450 0x7f52c019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:05.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.810+0000 7f52beffd640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c0101380 0x7f52c019d5c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009230 tx=0x7f52b002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:05.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.811+0000 7f52bcff9640 1 -- 172.21.15.18:0/516211009 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0004350 con 0x7f52c0101380 2024-02-17T13:00:05.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.811+0000 7f52bcff9640 1 -- 172.21.15.18:0/516211009 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52b00044b0 con 0x7f52c0101380 2024-02-17T13:00:05.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.811+0000 7f52c56f4640 1 -- 172.21.15.18:0/516211009 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c01a43c0 con 0x7f52c0101380 2024-02-17T13:00:05.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.811+0000 7f52bcff9640 1 -- 172.21.15.18:0/516211009 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0003d00 con 0x7f52c0101380 2024-02-17T13:00:05.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.811+0000 7f52c56f4640 1 -- 172.21.15.18:0/516211009 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c01a48f0 con 0x7f52c0101380 2024-02-17T13:00:05.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.814+0000 7f52bcff9640 1 -- 172.21.15.18:0/516211009 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f52b0004df0 con 0x7f52c0101380 2024-02-17T13:00:05.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.814+0000 7f52c83a4740 1 Processor -- start 2024-02-17T13:00:05.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.814+0000 7f52c83a4740 1 -- start start 2024-02-17T13:00:05.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.814+0000 7f52bcff9640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52ac0a22b0 0x7f52ac0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.814+0000 7f52bcff9640 1 -- 172.21.15.18:0/516211009 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f52b00effb0 con 0x7f52c0101380 2024-02-17T13:00:05.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.814+0000 7f52c83a4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c52640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52c83a4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52c10 0x55fd00c5b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52bf7fe640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52ac0a22b0 0x7f52ac0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52c83a4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c5bd10 0x55fd00c5c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52c83a4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd00bc0f90 con 0x55fd00c52c10 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52c83a4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd00bbe4c0 con 0x55fd00c5bd10 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52c83a4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd00bb9ca0 con 0x55fd00c52260 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52c10 0x55fd00c5b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52c10 0x55fd00c5b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55208/0 (socket says 172.21.15.18:55208) 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52b7fff640 1 -- 172.21.15.18:0/2028345348 learned_addr learned my addr 172.21.15.18:0/2028345348 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52bf7fe640 1 --2- 172.21.15.18:0/2028345348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c5bd10 0x55fd00c5c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52beffd640 1 --2- 172.21.15.18:0/2028345348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c52640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52b7fff640 1 -- 172.21.15.18:0/2028345348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 msgr2=0x55fd00c52640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52b7fff640 1 --2- 172.21.15.18:0/2028345348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c52640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52b7fff640 1 -- 172.21.15.18:0/2028345348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c5bd10 msgr2=0x55fd00c5c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52b7fff640 1 --2- 172.21.15.18:0/2028345348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c5bd10 0x55fd00c5c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52b7fff640 1 -- 172.21.15.18:0/2028345348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd00b7c0b0 con 0x55fd00c52c10 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52bf7fe640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52ac0a22b0 0x7f52ac0a4770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009fd0 tx=0x7f52b8014c30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52b7fff640 1 --2- 172.21.15.18:0/2028345348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52c10 0x55fd00c5b740 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f52a80114f0 tx=0x7f52a80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff48a877f941628 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.815+0000 7f52b57fa640 1 -- 172.21.15.18:0/2028345348 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8005020 con 0x55fd00c52c10 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.816+0000 7f52b57fa640 1 -- 172.21.15.18:0/2028345348 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52a8005660 con 0x55fd00c52c10 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.816+0000 7f52b57fa640 1 -- 172.21.15.18:0/2028345348 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a800f7a0 con 0x55fd00c52c10 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.816+0000 7f52c83a4740 1 -- 172.21.15.18:0/2028345348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52c10 msgr2=0x55fd00c5b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.816+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2028345348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52c10 0x55fd00c5b740 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f52a80114f0 tx=0x7f52a80119b0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.816+0000 7f52c83a4740 1 -- 172.21.15.18:0/2028345348 shutdown_connections 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.816+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2028345348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c5bd10 0x55fd00c5c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.816+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2028345348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52c10 0x55fd00c5b740 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.816+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2028345348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c52640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.816+0000 7f52c83a4740 1 -- 172.21.15.18:0/2028345348 >> 172.21.15.18:0/2028345348 conn(0x55fd00c4c840 msgr2=0x55fd00c4cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.816+0000 7f52c83a4740 1 -- 172.21.15.18:0/2028345348 shutdown_connections 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.816+0000 7f52c83a4740 1 -- 172.21.15.18:0/2028345348 wait complete. 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52c83a4740 1 -- start start 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52c83a4740 1 Processor -- start 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52c83a4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c5f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52c83a4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c52c10 0x55fd00c5fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52c83a4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c5bd10 0x55fd00c60170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52c83a4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd00b5ef40 con 0x55fd00c5bd10 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52c83a4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0088d4b0 con 0x55fd00c52c10 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52c83a4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd00b3f670 con 0x55fd00c52260 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52c83a4740 10 client.0 _ll_register_callbacks cb 0x55fd00bacf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52beffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c5f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52beffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c5f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34646/0 (socket says 172.21.15.18:34646) 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52beffd640 1 -- 172.21.15.18:0/2000425753 learned_addr learned my addr 172.21.15.18:0/2000425753 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c52c10 0x55fd00c5fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52bf7fe640 1 --2- 172.21.15.18:0/2000425753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c5bd10 0x55fd00c60170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52beffd640 1 -- 172.21.15.18:0/2000425753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c52c10 msgr2=0x55fd00c5fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52beffd640 1 --2- 172.21.15.18:0/2000425753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c52c10 0x55fd00c5fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52beffd640 1 -- 172.21.15.18:0/2000425753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c5bd10 msgr2=0x55fd00c60170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52beffd640 1 --2- 172.21.15.18:0/2000425753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c5bd10 0x55fd00c60170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.817+0000 7f52beffd640 1 -- 172.21.15.18:0/2000425753 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a8009c80 con 0x55fd00c52260 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52beffd640 1 --2- 172.21.15.18:0/2000425753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c5f6f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f52b00b9480 tx=0x7f52b0003f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21bf54552976857c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52a67fc640 1 -- 172.21.15.18:0/2000425753 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00c0c00 con 0x55fd00c52260 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52c83a4740 1 -- 172.21.15.18:0/2000425753 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd00b93f40 con 0x55fd00c52260 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52a67fc640 1 -- 172.21.15.18:0/2000425753 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f52b00c5250 con 0x55fd00c52260 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52a67fc640 1 -- 172.21.15.18:0/2000425753 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52b00c4dc0 con 0x55fd00c52260 2024-02-17T13:00:05.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52c83a4740 10 client.28315 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:05.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52c83a4740 1 -- 172.21.15.18:0/2000425753 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55fd00bb8320 con 0x55fd00c52260 2024-02-17T13:00:05.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52a67fc640 1 -- 172.21.15.18:0/2000425753 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fd00b93f40 con 0x55fd00c52260 2024-02-17T13:00:05.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52a67fc640 1 -- 172.21.15.18:0/2000425753 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f52b003c070 con 0x55fd00c52260 2024-02-17T13:00:05.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52c83a4740 10 client.28315 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:05.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52c83a4740 10 client.28315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:05.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52c83a4740 20 client.28315 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:05.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2000425753 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fd00c6ff70 0x55fd00c72390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52c83a4740 4 client.28315 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:05.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.818+0000 7f52c83a4740 1 -- 172.21.15.18:0/2000425753 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd00bb0310 con 0x55fd00c6ff70 2024-02-17T13:00:05.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.819+0000 7f52b7fff640 1 --2- 172.21.15.18:0/2000425753 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fd00c6ff70 0x55fd00c72390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.819+0000 7f52b7fff640 1 --2- 172.21.15.18:0/2000425753 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fd00c6ff70 0x55fd00c72390 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cb7875e57532dfc server_cookie=ce499d19ccaf1b9b in_seq=0 out_seq=0 2024-02-17T13:00:05.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.819+0000 7f52a67fc640 10 client.28315 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:05.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52a67fc640 1 -- 172.21.15.18:0/2000425753 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd00bb0310 con 0x55fd00c6ff70 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52a67fc640 10 client.28315 handle_command_reply: tid=0 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52c83a4740 1 client.28315 shutdown 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52c83a4740 1 -- 172.21.15.18:0/2000425753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 msgr2=0x55fd00c5f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2000425753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c5f6f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f52b00b9480 tx=0x7f52b0003f20 comp rx=0 tx=0).stop 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52c83a4740 1 -- 172.21.15.18:0/2000425753 shutdown_connections 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2000425753 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fd00c6ff70 0x55fd00c72390 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2000425753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c5bd10 0x55fd00c60170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52a67fc640 0 client.28315 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2000425753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c52c10 0x55fd00c5fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2000425753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c5f6f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52c83a4740 1 -- 172.21.15.18:0/2000425753 >> 172.21.15.18:0/2000425753 conn(0x55fd00c4c840 msgr2=0x55fd00c4e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52c83a4740 1 -- 172.21.15.18:0/2000425753 shutdown_connections 2024-02-17T13:00:05.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52c83a4740 1 -- 172.21.15.18:0/2000425753 wait complete. 2024-02-17T13:00:05.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.820+0000 7f52c83a4740 20 client.28315 trim_cache size 0 max 16384 2024-02-17T13:00:05.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52c83a4740 1 Processor -- start 2024-02-17T13:00:05.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52c83a4740 1 -- start start 2024-02-17T13:00:05.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52c83a4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c5d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52c83a4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c52c10 0x55fd00c66db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52c83a4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c5bd10 0x55fd00c672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52c83a4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd00c4d460 con 0x55fd00c5bd10 2024-02-17T13:00:05.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52c83a4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd00c516c0 con 0x55fd00c52c10 2024-02-17T13:00:05.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52c83a4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd00c62bb0 con 0x55fd00c52260 2024-02-17T13:00:05.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52beffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c5d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52beffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c5d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34654/0 (socket says 172.21.15.18:34654) 2024-02-17T13:00:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52beffd640 1 -- 172.21.15.18:0/2863819127 learned_addr learned my addr 172.21.15.18:0/2863819127 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52bf7fe640 1 --2- 172.21.15.18:0/2863819127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c5bd10 0x55fd00c672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.837+0000 7f52b7fff640 1 --2- 172.21.15.18:0/2863819127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c52c10 0x55fd00c66db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.838+0000 7f52beffd640 1 -- 172.21.15.18:0/2863819127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c52c10 msgr2=0x55fd00c66db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.838+0000 7f52beffd640 1 --2- 172.21.15.18:0/2863819127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c52c10 0x55fd00c66db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.838+0000 7f52beffd640 1 -- 172.21.15.18:0/2863819127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c5bd10 msgr2=0x55fd00c672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.838+0000 7f52beffd640 1 --2- 172.21.15.18:0/2863819127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c5bd10 0x55fd00c672f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.838+0000 7f52beffd640 1 -- 172.21.15.18:0/2863819127 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd00b54c00 con 0x55fd00c52260 2024-02-17T13:00:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.838+0000 7f52beffd640 1 --2- 172.21.15.18:0/2863819127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c5d100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f52b0000c00 tx=0x7f52b00057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21a3a2ec75ac8205 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.838+0000 7f52b4ff9640 1 -- 172.21.15.18:0/2863819127 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00050a0 con 0x55fd00c52260 2024-02-17T13:00:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.838+0000 7f52b4ff9640 1 -- 172.21.15.18:0/2863819127 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52b00c5250 con 0x55fd00c52260 2024-02-17T13:00:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.838+0000 7f52b4ff9640 1 -- 172.21.15.18:0/2863819127 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00b9120 con 0x55fd00c52260 2024-02-17T13:00:05.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.839+0000 7f52c83a4740 1 -- 172.21.15.18:0/2863819127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 msgr2=0x55fd00c5d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.839+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2863819127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c5d100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f52b0000c00 tx=0x7f52b00057e0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.839+0000 7f52c83a4740 1 -- 172.21.15.18:0/2863819127 shutdown_connections 2024-02-17T13:00:05.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.839+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2863819127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c5bd10 0x55fd00c672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.839+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2863819127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c52c10 0x55fd00c66db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.839+0000 7f52c83a4740 1 --2- 172.21.15.18:0/2863819127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52260 0x55fd00c5d100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.839+0000 7f52c83a4740 1 -- 172.21.15.18:0/2863819127 >> 172.21.15.18:0/2863819127 conn(0x55fd00c4c840 msgr2=0x55fd00c76d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:05.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.839+0000 7f52c83a4740 1 -- 172.21.15.18:0/2863819127 shutdown_connections 2024-02-17T13:00:05.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.839+0000 7f52c83a4740 1 -- 172.21.15.18:0/2863819127 wait complete. 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.839+0000 7f52c83a4740 1 -- start start 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52c83a4740 1 Processor -- start 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52c83a4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52260 0x55fd00cbfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52c83a4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52c10 0x55fd00cc03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52c83a4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c5bd10 0x55fd00cba420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52c83a4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd00cba9e0 con 0x55fd00c52260 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52c83a4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd00cbab50 con 0x55fd00c5bd10 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52c83a4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd00cbad10 con 0x55fd00c52c10 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52c83a4740 10 client.0 _ll_register_callbacks cb 0x7f52a8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52beffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52260 0x55fd00cbfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52beffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52260 0x55fd00cbfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55222/0 (socket says 172.21.15.18:55222) 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52beffd640 1 -- 172.21.15.18:0/1381541666 learned_addr learned my addr 172.21.15.18:0/1381541666 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52bf7fe640 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c5bd10 0x55fd00cba420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52b7fff640 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52c10 0x55fd00cc03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52beffd640 1 -- 172.21.15.18:0/1381541666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52c10 msgr2=0x55fd00cc03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52beffd640 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52c10 0x55fd00cc03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52beffd640 1 -- 172.21.15.18:0/1381541666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c5bd10 msgr2=0x55fd00cba420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52beffd640 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c5bd10 0x55fd00cba420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.840+0000 7f52beffd640 1 -- 172.21.15.18:0/1381541666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd00bb8320 con 0x55fd00c52260 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52beffd640 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52260 0x55fd00cbfe60 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f52b0003810 tx=0x7f52b002c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2580b3f5b34b4f3b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52a67fc640 1 -- 172.21.15.18:0/1381541666 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00050a0 con 0x55fd00c52260 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52a67fc640 1 -- 172.21.15.18:0/1381541666 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f52b00bbf00 con 0x55fd00c52260 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52c83a4740 1 -- 172.21.15.18:0/1381541666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd00bdf900 con 0x55fd00c52260 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52a67fc640 1 -- 172.21.15.18:0/1381541666 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52b00bc0c0 con 0x55fd00c52260 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52c83a4740 10 client.19251 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52c83a4740 1 -- 172.21.15.18:0/1381541666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55fd00b7a910 con 0x55fd00c52260 2024-02-17T13:00:05.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52a67fc640 1 -- 172.21.15.18:0/1381541666 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fd00bdf900 con 0x55fd00c52260 2024-02-17T13:00:05.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52a67fc640 1 -- 172.21.15.18:0/1381541666 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f52b00cb050 con 0x55fd00c52260 2024-02-17T13:00:05.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52c83a4740 10 client.19251 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:05.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52c83a4740 10 client.19251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:05.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52c83a4740 20 client.19251 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:05.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52c83a4740 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fd00cc78a0 0x55fd00cc9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52c83a4740 4 client.19251 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T13:00:05.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.841+0000 7f52c83a4740 1 -- 172.21.15.18:0/1381541666 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fd008b9860 con 0x55fd00cc78a0 2024-02-17T13:00:05.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.842+0000 7f52b7fff640 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fd00cc78a0 0x55fd00cc9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.842+0000 7f52b7fff640 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fd00cc78a0 0x55fd00cc9c90 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e9ba5bcdef5afd server_cookie=1eff3ced91f5c83f in_seq=0 out_seq=0 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.842+0000 7f52a67fc640 10 client.19251 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52a67fc640 1 -- 172.21.15.18:0/1381541666 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fd008b9860 con 0x55fd00cc78a0 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52a67fc640 10 client.19251 handle_command_reply: tid=0 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52c83a4740 1 client.19251 shutdown 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52c83a4740 1 -- 172.21.15.18:0/1381541666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52260 msgr2=0x55fd00cbfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52c83a4740 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52260 0x55fd00cbfe60 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f52b0003810 tx=0x7f52b002c450 comp rx=0 tx=0).stop 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52c83a4740 1 -- 172.21.15.18:0/1381541666 shutdown_connections 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52c83a4740 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fd00cc78a0 0x55fd00cc9c90 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52c83a4740 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd00c5bd10 0x55fd00cba420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52c83a4740 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd00c52c10 0x55fd00cc03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52a67fc640 0 client.19251 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52c83a4740 1 --2- 172.21.15.18:0/1381541666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd00c52260 0x55fd00cbfe60 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52c83a4740 1 -- 172.21.15.18:0/1381541666 >> 172.21.15.18:0/1381541666 conn(0x55fd00c4c840 msgr2=0x55fd00c4b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52c83a4740 1 -- 172.21.15.18:0/1381541666 shutdown_connections 2024-02-17T13:00:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52c83a4740 1 -- 172.21.15.18:0/1381541666 wait complete. 2024-02-17T13:00:05.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.843+0000 7f52c83a4740 20 client.19251 trim_cache size 0 max 16384 2024-02-17T13:00:05.839 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:00:05.839 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T13:00:05.839 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "3da9fb8e-030c-4fbc-97a4-fe3150ef9fd6", 2024-02-17T13:00:05.839 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T13:00:05.839 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:00:05.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.844+0000 7f52c56f4640 1 -- 172.21.15.18:0/516211009 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52ac0a22b0 msgr2=0x7f52ac0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.844+0000 7f52c56f4640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52ac0a22b0 0x7f52ac0a4770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009fd0 tx=0x7f52b8014c30 comp rx=0 tx=0).stop 2024-02-17T13:00:05.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.844+0000 7f52c56f4640 1 -- 172.21.15.18:0/516211009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c0101380 msgr2=0x7f52c019d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.844+0000 7f52c56f4640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c0101380 0x7f52c019d5c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009230 tx=0x7f52b002c650 comp rx=0 tx=0).stop 2024-02-17T13:00:05.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.844+0000 7f52c56f4640 1 -- 172.21.15.18:0/516211009 shutdown_connections 2024-02-17T13:00:05.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.844+0000 7f52c56f4640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f52ac0a22b0 0x7f52ac0a4770 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.844+0000 7f52c56f4640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c0197680 0x7f52c0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.844+0000 7f52c56f4640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52c0106450 0x7f52c019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.844+0000 7f52c56f4640 1 --2- 172.21.15.18:0/516211009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c0101380 0x7f52c019d5c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.844+0000 7f52c56f4640 1 -- 172.21.15.18:0/516211009 >> 172.21.15.18:0/516211009 conn(0x7f52c00fcc20 msgr2=0x7f52c00fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:05.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.844+0000 7f52c56f4640 1 -- 172.21.15.18:0/516211009 shutdown_connections 2024-02-17T13:00:05.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.844+0000 7f52c56f4640 1 -- 172.21.15.18:0/516211009 wait complete. 2024-02-17T13:00:05.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:00:05.846 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:00:05.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.968+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1331595956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a4106170 msgr2=0x7ff8a4106550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.968+0000 7ff8ab92c640 1 --2- 172.21.15.18:0/1331595956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a4106170 0x7ff8a4106550 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009930 tx=0x7ff8a002bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.968+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1331595956 shutdown_connections 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.968+0000 7ff8ab92c640 1 --2- 172.21.15.18:0/1331595956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4100390 0x7ff8a410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.968+0000 7ff8ab92c640 1 --2- 172.21.15.18:0/1331595956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8a40ff9d0 0x7ff8a40ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.968+0000 7ff8ab92c640 1 --2- 172.21.15.18:0/1331595956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a4106170 0x7ff8a4106550 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.968+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1331595956 >> 172.21.15.18:0/1331595956 conn(0x7ff8a4075380 msgr2=0x7ff8a4075790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.968+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1331595956 shutdown_connections 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.968+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1331595956 wait complete. 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.968+0000 7ff8ab92c640 1 Processor -- start 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.968+0000 7ff8ab92c640 1 -- start start 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8ab92c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8a40ff9d0 0x7ff8a4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8ab92c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4100390 0x7ff8a4197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8ab92c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a4106170 0x7ff8a419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8ab92c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a419c600 con 0x7ff8a4106170 2024-02-17T13:00:05.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8ab92c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a419c770 con 0x7ff8a40ff9d0 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8ab92c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a419c930 con 0x7ff8a4100390 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8a9ec9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a4106170 0x7ff8a419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8a9ec9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a4106170 0x7ff8a419c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55232/0 (socket says 172.21.15.18:55232) 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8a96c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8a40ff9d0 0x7ff8a4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8a9ec9640 1 -- 172.21.15.18:0/1741459067 learned_addr learned my addr 172.21.15.18:0/1741459067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8a8ec7640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4100390 0x7ff8a4197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8a96c8640 1 -- 172.21.15.18:0/1741459067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4100390 msgr2=0x7ff8a4197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4100390 0x7ff8a4197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8a96c8640 1 -- 172.21.15.18:0/1741459067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a4106170 msgr2=0x7ff8a419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a4106170 0x7ff8a419c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8a96c8640 1 -- 172.21.15.18:0/1741459067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a0009560 con 0x7ff8a40ff9d0 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.969+0000 7ff8a9ec9640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a4106170 0x7ff8a419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.970+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8a40ff9d0 0x7ff8a4197790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff8a002c170 tx=0x7ff8a0005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.970+0000 7ff89a7fc640 1 -- 172.21.15.18:0/1741459067 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0004350 con 0x7ff8a40ff9d0 2024-02-17T13:00:05.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.970+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1741459067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a419cbb0 con 0x7ff8a40ff9d0 2024-02-17T13:00:05.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.970+0000 7ff89a7fc640 1 -- 172.21.15.18:0/1741459067 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8a00044b0 con 0x7ff8a40ff9d0 2024-02-17T13:00:05.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.970+0000 7ff89a7fc640 1 -- 172.21.15.18:0/1741459067 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0003cd0 con 0x7ff8a40ff9d0 2024-02-17T13:00:05.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.970+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1741459067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a41a4490 con 0x7ff8a40ff9d0 2024-02-17T13:00:05.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.972+0000 7ff89a7fc640 1 -- 172.21.15.18:0/1741459067 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff8a0004df0 con 0x7ff8a40ff9d0 2024-02-17T13:00:05.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8ae5dc740 1 Processor -- start 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8ae5dc740 1 -- start start 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff89a7fc640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff8900a2230 0x7ff8900a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff89a7fc640 1 -- 172.21.15.18:0/1741459067 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7ff8a00eff40 con 0x7ff8a40ff9d0 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8a9ec9640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff8900a2230 0x7ff8900a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8ae5dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1c2b0 0x55d872f1c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8ae5dc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1cbd0 0x55d872f25270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8ae5dc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f27ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8ae5dc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d872e8b280 con 0x55d872f257b0 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8ae5dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d872e887b0 con 0x55d872f1c2b0 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8ae5dc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d872e83f90 con 0x55d872f1cbd0 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8a96c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1c2b0 0x55d872f1c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8a9ec9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f27ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8a8ec7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1cbd0 0x55d872f25270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.973+0000 7ff8a9ec9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f27ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55244/0 (socket says 172.21.15.18:55244) 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff8a9ec9640 1 -- 172.21.15.18:0/2219370113 learned_addr learned my addr 172.21.15.18:0/2219370113 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff8a9ec9640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff8900a2230 0x7ff8900a46f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009fd0 tx=0x7ff89c011b90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff8a9ec9640 1 -- 172.21.15.18:0/2219370113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1cbd0 msgr2=0x55d872f25270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff8a9ec9640 1 --2- 172.21.15.18:0/2219370113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1cbd0 0x55d872f25270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff8a9ec9640 1 -- 172.21.15.18:0/2219370113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1c2b0 msgr2=0x55d872f1c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff8a9ec9640 1 --2- 172.21.15.18:0/2219370113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1c2b0 0x55d872f1c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff8a9ec9640 1 -- 172.21.15.18:0/2219370113 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d872e42570 con 0x55d872f257b0 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff8a8ec7640 1 --2- 172.21.15.18:0/2219370113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1cbd0 0x55d872f25270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/2219370113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1c2b0 0x55d872f1c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff8a9ec9640 1 --2- 172.21.15.18:0/2219370113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f27ba0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00fc50 tx=0x7ff89c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c49ccbc311b6bb9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff88f7fe640 1 -- 172.21.15.18:0/2219370113 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c014e80 con 0x55d872f257b0 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff88f7fe640 1 -- 172.21.15.18:0/2219370113 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff89c00fe80 con 0x55d872f257b0 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.974+0000 7ff88f7fe640 1 -- 172.21.15.18:0/2219370113 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c01f640 con 0x55d872f257b0 2024-02-17T13:00:05.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.975+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2219370113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 msgr2=0x55d872f27ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.975+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/2219370113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f27ba0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00fc50 tx=0x7ff89c01c000 comp rx=0 tx=0).stop 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.975+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2219370113 shutdown_connections 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.975+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/2219370113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f27ba0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.975+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/2219370113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1cbd0 0x55d872f25270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.975+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/2219370113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1c2b0 0x55d872f1c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.975+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2219370113 >> 172.21.15.18:0/2219370113 conn(0x55d872f16990 msgr2=0x55d872f16d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.975+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2219370113 shutdown_connections 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.975+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2219370113 wait complete. 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.975+0000 7ff8ae5dc740 1 -- start start 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8ae5dc740 1 Processor -- start 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8ae5dc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 0x55d872f251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8ae5dc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f1cbd0 0x55d872f22560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8ae5dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f257b0 0x55d872f22b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8ae5dc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d872b57430 con 0x55d872f1cbd0 2024-02-17T13:00:05.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8ae5dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d872b13e00 con 0x55d872f257b0 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8ae5dc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d872e08fd0 con 0x55d872f1c2b0 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8ae5dc740 10 client.0 _ll_register_callbacks cb 0x55d872e5efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8a96c8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 0x55d872f251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8a9ec9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f257b0 0x55d872f22b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8a9ec9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f257b0 0x55d872f22b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:45370/0 (socket says 172.21.15.18:45370) 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8a9ec9640 1 -- 172.21.15.18:0/78547095 learned_addr learned my addr 172.21.15.18:0/78547095 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8a96c8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 0x55d872f251d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34694/0 (socket says 172.21.15.18:34694) 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.976+0000 7ff8a8ec7640 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f1cbd0 0x55d872f22560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff8a96c8640 1 -- 172.21.15.18:0/78547095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f257b0 msgr2=0x55d872f22b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f257b0 0x55d872f22b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff8a96c8640 1 -- 172.21.15.18:0/78547095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f1cbd0 msgr2=0x55d872f22560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f1cbd0 0x55d872f22560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff8a96c8640 1 -- 172.21.15.18:0/78547095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89c00f950 con 0x55d872f1c2b0 2024-02-17T13:00:05.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff8a9ec9640 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f257b0 0x55d872f22b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 0x55d872f251d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00ed7a0 tx=0x7ff8a00035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=599c57ca681f628d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff88cff9640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff88cff9640 1 -- 172.21.15.18:0/78547095 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a00c4f50 con 0x55d872f1c2b0 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/78547095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d872bc0720 con 0x55d872f1c2b0 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff88cff9640 1 -- 172.21.15.18:0/78547095 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7ff8a00bd140 con 0x55d872f1c2b0 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff88cff9640 1 -- 172.21.15.18:0/78547095 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8a00bca40 con 0x55d872f1c2b0 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff8ae5dc740 10 client.28333 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/78547095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55d872e82610 con 0x55d872f1c2b0 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.977+0000 7ff88cff9640 1 -- 172.21.15.18:0/78547095 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d872bc0720 con 0x55d872f1c2b0 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.978+0000 7ff88cff9640 1 -- 172.21.15.18:0/78547095 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff8a003c070 con 0x55d872f1c2b0 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.978+0000 7ff8ae5dc740 10 client.28333 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.978+0000 7ff8ae5dc740 10 client.28333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.978+0000 7ff8ae5dc740 20 client.28333 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.978+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d872f356e0 0x55d872f37b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.978+0000 7ff8ae5dc740 4 client.28333 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.978+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/78547095 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d872c0f890 con 0x55d872f356e0 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.978+0000 7ff8a8ec7640 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d872f356e0 0x55d872f37b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.978+0000 7ff88cff9640 10 client.28333 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.978+0000 7ff8a8ec7640 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d872f356e0 0x55d872f37b00 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df6fcb8a4b7c2b37 server_cookie=99344c9d8c5526b8 in_seq=0 out_seq=0 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff88cff9640 1 -- 172.21.15.18:0/78547095 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d872c0f890 con 0x55d872f356e0 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff88cff9640 10 client.28333 handle_command_reply: tid=0 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff8ae5dc740 1 client.28333 shutdown 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/78547095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 msgr2=0x55d872f251d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 0x55d872f251d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00ed7a0 tx=0x7ff8a00035f0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/78547095 shutdown_connections 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d872f356e0 0x55d872f37b00 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f257b0 0x55d872f22b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f1cbd0 0x55d872f22560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/78547095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 0x55d872f251d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/78547095 >> 172.21.15.18:0/78547095 conn(0x55d872f16990 msgr2=0x55d872f18730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:05.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff88cff9640 0 client.28333 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:05.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/78547095 shutdown_connections 2024-02-17T13:00:05.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.979+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/78547095 wait complete. 2024-02-17T13:00:05.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.980+0000 7ff8ae5dc740 20 client.28333 trim_cache size 0 max 16384 2024-02-17T13:00:05.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.997+0000 7ff8ae5dc740 1 Processor -- start 2024-02-17T13:00:05.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.997+0000 7ff8ae5dc740 1 -- start start 2024-02-17T13:00:05.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8ae5dc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 0x55d872f3c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8ae5dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1cbd0 0x55d872f24c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8ae5dc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8ae5dc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d872bc0720 con 0x55d872f257b0 2024-02-17T13:00:05.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8ae5dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d872f1e1a0 con 0x55d872f1cbd0 2024-02-17T13:00:05.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8ae5dc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d872f32430 con 0x55d872f1c2b0 2024-02-17T13:00:05.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8a96c8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 0x55d872f3c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8a96c8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 0x55d872f3c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34710/0 (socket says 172.21.15.18:34710) 2024-02-17T13:00:05.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8a96c8640 1 -- 172.21.15.18:0/958817898 learned_addr learned my addr 172.21.15.18:0/958817898 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:05.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8a9ec9640 1 --2- 172.21.15.18:0/958817898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8a8ec7640 1 --2- 172.21.15.18:0/958817898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1cbd0 0x55d872f24c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8a96c8640 1 -- 172.21.15.18:0/958817898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1cbd0 msgr2=0x55d872f24c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/958817898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1cbd0 0x55d872f24c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8a96c8640 1 -- 172.21.15.18:0/958817898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 msgr2=0x55d872f251d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/958817898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f251d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8a96c8640 1 -- 172.21.15.18:0/958817898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d872e1eac0 con 0x55d872f1c2b0 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8a9ec9640 1 --2- 172.21.15.18:0/958817898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.998+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/958817898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 0x55d872f3c480 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00bc510 tx=0x7ff8a00037d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6552bc74205b85b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.999+0000 7ff88effd640 1 -- 172.21.15.18:0/958817898 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a00bc580 con 0x55d872f1c2b0 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.999+0000 7ff88effd640 1 -- 172.21.15.18:0/958817898 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8a00bd140 con 0x55d872f1c2b0 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.999+0000 7ff88effd640 1 -- 172.21.15.18:0/958817898 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a00b8b20 con 0x55d872f1c2b0 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.999+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/958817898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 msgr2=0x55d872f3c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.999+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/958817898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 0x55d872f3c480 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00bc510 tx=0x7ff8a00037d0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.999+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/958817898 shutdown_connections 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.999+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/958817898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.999+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/958817898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1cbd0 0x55d872f24c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.999+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/958817898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1c2b0 0x55d872f3c480 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.999+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/958817898 >> 172.21.15.18:0/958817898 conn(0x55d872f16990 msgr2=0x55d872f11840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.999+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/958817898 shutdown_connections 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:05.999+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/958817898 wait complete. 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8ae5dc740 1 -- start start 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8ae5dc740 1 Processor -- start 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8ae5dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1c2b0 0x55d872f2c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8ae5dc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1cbd0 0x55d872f2aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8ae5dc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f2afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8ae5dc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d872f2cf90 con 0x55d872f257b0 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8ae5dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d872f2b510 con 0x55d872f1c2b0 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8ae5dc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d872f2b6d0 con 0x55d872f1cbd0 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8ae5dc740 10 client.0 _ll_register_callbacks cb 0x7ff89c011420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8a9ec9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f2afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8a96c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1c2b0 0x55d872f2c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8a9ec9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f2afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55274/0 (socket says 172.21.15.18:55274) 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8a8ec7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1cbd0 0x55d872f2aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.000+0000 7ff8a9ec9640 1 -- 172.21.15.18:0/2027668254 learned_addr learned my addr 172.21.15.18:0/2027668254 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff8a96c8640 1 -- 172.21.15.18:0/2027668254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1cbd0 msgr2=0x55d872f2aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/2027668254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1cbd0 0x55d872f2aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff8a96c8640 1 -- 172.21.15.18:0/2027668254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 msgr2=0x55d872f2afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/2027668254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f2afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff8a96c8640 1 -- 172.21.15.18:0/2027668254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d872e42570 con 0x55d872f1c2b0 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff8a9ec9640 1 --2- 172.21.15.18:0/2027668254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f2afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff8a96c8640 1 --2- 172.21.15.18:0/2027668254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1c2b0 0x55d872f2c6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0005c20 tx=0x7ff8a0034000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9416abd968805cd1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff88cff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff88cff9640 1 -- 172.21.15.18:0/2027668254 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a00bc580 con 0x55d872f1c2b0 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff88cff9640 1 -- 172.21.15.18:0/2027668254 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7ff8a00bca40 con 0x55d872f1c2b0 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2027668254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d872b837e0 con 0x55d872f1c2b0 2024-02-17T13:00:05.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff88cff9640 1 -- 172.21.15.18:0/2027668254 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8a00c0b90 con 0x55d872f1c2b0 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff8ae5dc740 10 client.28304 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.001+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2027668254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55d872e82610 con 0x55d872f1c2b0 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.002+0000 7ff88cff9640 1 -- 172.21.15.18:0/2027668254 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d872b837e0 con 0x55d872f1c2b0 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.002+0000 7ff88cff9640 1 -- 172.21.15.18:0/2027668254 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff8a003c070 con 0x55d872f1c2b0 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.002+0000 7ff8ae5dc740 10 client.28304 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.002+0000 7ff8ae5dc740 10 client.28304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.002+0000 7ff8ae5dc740 20 client.28304 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.002+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/2027668254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d872f86b20 0x55d872f86f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.002+0000 7ff8ae5dc740 4 client.28304 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.002+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2027668254 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d872b724e0 con 0x55d872f86b20 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.002+0000 7ff8a8ec7640 1 --2- 172.21.15.18:0/2027668254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d872f86b20 0x55d872f86f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.002+0000 7ff88cff9640 10 client.28304 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:05.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.002+0000 7ff8a8ec7640 1 --2- 172.21.15.18:0/2027668254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d872f86b20 0x55d872f86f30 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69fbc9027aa7a12 server_cookie=1bab319919de40e8 in_seq=0 out_seq=0 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.003+0000 7ff88cff9640 1 -- 172.21.15.18:0/2027668254 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d872b724e0 con 0x55d872f86b20 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.003+0000 7ff88cff9640 10 client.28304 handle_command_reply: tid=0 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.003+0000 7ff8ae5dc740 1 client.28304 shutdown 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.004+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2027668254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1c2b0 msgr2=0x55d872f2c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.004+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/2027668254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1c2b0 0x55d872f2c6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0005c20 tx=0x7ff8a0034000 comp rx=0 tx=0).stop 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.004+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2027668254 shutdown_connections 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.004+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/2027668254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d872f86b20 0x55d872f86f30 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.004+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/2027668254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d872f257b0 0x55d872f2afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.004+0000 7ff88cff9640 0 client.28304 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.004+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/2027668254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d872f1cbd0 0x55d872f2aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.004+0000 7ff8ae5dc740 1 --2- 172.21.15.18:0/2027668254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d872f1c2b0 0x55d872f2c6c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.004+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2027668254 >> 172.21.15.18:0/2027668254 conn(0x55d872f16990 msgr2=0x55d872f3c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:05.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.004+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2027668254 shutdown_connections 2024-02-17T13:00:05.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.004+0000 7ff8ae5dc740 1 -- 172.21.15.18:0/2027668254 wait complete. 2024-02-17T13:00:05.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.004+0000 7ff8ae5dc740 20 client.28304 trim_cache size 0 max 16384 2024-02-17T13:00:05.999 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:00:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T13:00:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T13:00:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "3da9fb8e-030c-4fbc-97a4-fe3150ef9fd6": { 2024-02-17T13:00:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T13:00:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "3da9fb8e-030c-4fbc-97a4-fe3150ef9fd6", 2024-02-17T13:00:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T13:00:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:00:06.000 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:00:06.000 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:00:06.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.005+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1741459067 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff8900a2230 msgr2=0x7ff8900a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:06.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.005+0000 7ff8ab92c640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff8900a2230 0x7ff8900a46f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009fd0 tx=0x7ff89c011b90 comp rx=0 tx=0).stop 2024-02-17T13:00:06.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.005+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1741459067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8a40ff9d0 msgr2=0x7ff8a4197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:06.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.005+0000 7ff8ab92c640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8a40ff9d0 0x7ff8a4197790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff8a002c170 tx=0x7ff8a0005e70 comp rx=0 tx=0).stop 2024-02-17T13:00:06.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.005+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1741459067 shutdown_connections 2024-02-17T13:00:06.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.005+0000 7ff8ab92c640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff8900a2230 0x7ff8900a46f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:06.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.005+0000 7ff8ab92c640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a4106170 0x7ff8a419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:06.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.005+0000 7ff8ab92c640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4100390 0x7ff8a4197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:06.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.005+0000 7ff8ab92c640 1 --2- 172.21.15.18:0/1741459067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8a40ff9d0 0x7ff8a4197790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:06.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.005+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1741459067 >> 172.21.15.18:0/1741459067 conn(0x7ff8a4075380 msgr2=0x7ff8a40fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:06.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.005+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1741459067 shutdown_connections 2024-02-17T13:00:06.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:06.005+0000 7ff8ab92c640 1 -- 172.21.15.18:0/1741459067 wait complete. 2024-02-17T13:00:06.006 INFO:tasks.cephfs.filesystem:scrub status for tag:3da9fb8e-030c-4fbc-97a4-fe3150ef9fd6 - {'path': '/', 'tag': '3da9fb8e-030c-4fbc-97a4-fe3150ef9fd6', 'options': 'recursive,force'} 2024-02-17T13:00:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: pgmap v318: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-02-17T13:00:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: osdmap e99: 12 total, 12 up, 12 in 2024-02-17T13:00:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:06 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:00:06.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 516816 5.06 MB/sec execute 441 sec latency 27.767 ms 2024-02-17T13:00:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: pgmap v318: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-02-17T13:00:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: osdmap e99: 12 total, 12 up, 12 in 2024-02-17T13:00:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T13:00:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:06.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:06.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:06.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:06.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:06 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:00:06.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: pgmap v318: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-02-17T13:00:06.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: osdmap e99: 12 total, 12 up, 12 in 2024-02-17T13:00:06.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T13:00:06.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:06.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:06.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:06.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:06.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:06.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:06.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:06.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:06.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:06 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:00:07.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:07 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:07.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:07 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:07.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 516816 5.05 MB/sec execute 442 sec latency 1011.042 ms 2024-02-17T13:00:07.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:07 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:07.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:07 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:07.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:07 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:07.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:07 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:08.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 516816 5.04 MB/sec execute 443 sec latency 2011.099 ms 2024-02-17T13:00:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: pgmap v320: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 962 KiB/s wr, 125 op/s 2024-02-17T13:00:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:08.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:08.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:08.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:08.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:08.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:08.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:08.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:08.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:08.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:00:08.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:00:08.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: pgmap v320: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 962 KiB/s wr, 125 op/s 2024-02-17T13:00:09.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:00:09.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:00:09.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: pgmap v320: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 962 KiB/s wr, 125 op/s 2024-02-17T13:00:09.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T13:00:09.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:00:09.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:00:09.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:00:09.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 517731 5.03 MB/sec execute 444 sec latency 2612.413 ms 2024-02-17T13:00:09.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:09 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi146 to 174.1M 2024-02-17T13:00:09.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:09 smithi146 ceph-mon[36454]: Unable to set osd_memory_target on smithi146 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-02-17T13:00:09.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:09 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi119 to 174.1M 2024-02-17T13:00:09.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:09 smithi146 ceph-mon[36454]: Unable to set osd_memory_target on smithi119 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-02-17T13:00:09.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:09 smithi146 ceph-mon[36454]: Adjusting osd_memory_target on smithi018 to 177.2M 2024-02-17T13:00:09.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:09 smithi146 ceph-mon[36454]: Unable to set osd_memory_target on smithi018 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-02-17T13:00:10.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:09 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi146 to 174.1M 2024-02-17T13:00:10.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:09 smithi018 ceph-mon[32625]: Unable to set osd_memory_target on smithi146 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-02-17T13:00:10.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:09 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi119 to 174.1M 2024-02-17T13:00:10.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:09 smithi018 ceph-mon[32625]: Unable to set osd_memory_target on smithi119 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-02-17T13:00:10.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:09 smithi018 ceph-mon[32625]: Adjusting osd_memory_target on smithi018 to 177.2M 2024-02-17T13:00:10.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:09 smithi018 ceph-mon[32625]: Unable to set osd_memory_target on smithi018 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-02-17T13:00:10.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:09 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi146 to 174.1M 2024-02-17T13:00:10.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:09 smithi119 ceph-mon[36188]: Unable to set osd_memory_target on smithi146 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-02-17T13:00:10.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:09 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi119 to 174.1M 2024-02-17T13:00:10.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:09 smithi119 ceph-mon[36188]: Unable to set osd_memory_target on smithi119 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-02-17T13:00:10.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:09 smithi119 ceph-mon[36188]: Adjusting osd_memory_target on smithi018 to 177.2M 2024-02-17T13:00:10.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:09 smithi119 ceph-mon[36188]: Unable to set osd_memory_target on smithi018 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-02-17T13:00:10.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 519499 5.05 MB/sec execute 445 sec latency 17.055 ms 2024-02-17T13:00:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.315+0000 7f8f12e83640 1 -- 172.21.15.18:0/1960282838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f0c1002d0 msgr2=0x7f8f0c100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.315+0000 7f8f12e83640 1 --2- 172.21.15.18:0/1960282838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f0c1002d0 0x7f8f0c100750 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009900 tx=0x7f8ef802bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 -- 172.21.15.18:0/1960282838 shutdown_connections 2024-02-17T13:00:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 --2- 172.21.15.18:0/1960282838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0c100c90 0x7f8f0c10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 --2- 172.21.15.18:0/1960282838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f0c1002d0 0x7f8f0c100750 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 --2- 172.21.15.18:0/1960282838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f0c105020 0x7f8f0c105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 -- 172.21.15.18:0/1960282838 >> 172.21.15.18:0/1960282838 conn(0x7f8f0c076ed0 msgr2=0x7f8f0c0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 -- 172.21.15.18:0/1960282838 shutdown_connections 2024-02-17T13:00:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 -- 172.21.15.18:0/1960282838 wait complete. 2024-02-17T13:00:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 Processor -- start 2024-02-17T13:00:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 -- start start 2024-02-17T13:00:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f0c1002d0 0x7f8f0c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f0c100c90 0x7f8f0c197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0c105020 0x7f8f0c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c19c5f0 con 0x7f8f0c100c90 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c19c760 con 0x7f8f0c105020 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f12e83640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c19c920 con 0x7f8f0c1002d0 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f0bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f0c100c90 0x7f8f0c197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.316+0000 7f8f0bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f0c100c90 0x7f8f0c197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39130/0 (socket says 172.21.15.18:39130) 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f0bfff640 1 -- 172.21.15.18:0/1320658679 learned_addr learned my addr 172.21.15.18:0/1320658679 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f11420640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0c105020 0x7f8f0c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f10c1f640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f0c1002d0 0x7f8f0c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f0bfff640 1 -- 172.21.15.18:0/1320658679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f0c1002d0 msgr2=0x7f8f0c197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f0bfff640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f0c1002d0 0x7f8f0c197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f0bfff640 1 -- 172.21.15.18:0/1320658679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0c105020 msgr2=0x7f8f0c19bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f0bfff640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0c105020 0x7f8f0c19bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f0bfff640 1 -- 172.21.15.18:0/1320658679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef8009560 con 0x7f8f0c100c90 2024-02-17T13:00:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f11420640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0c105020 0x7f8f0c19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f0bfff640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f0c100c90 0x7f8f0c197cc0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8000c00 tx=0x7f8ef802c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f09ffb640 1 -- 172.21.15.18:0/1320658679 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef802dea0 con 0x7f8f0c100c90 2024-02-17T13:00:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f12e83640 1 -- 172.21.15.18:0/1320658679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c19cba0 con 0x7f8f0c100c90 2024-02-17T13:00:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f09ffb640 1 -- 172.21.15.18:0/1320658679 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ef8004410 con 0x7f8f0c100c90 2024-02-17T13:00:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f09ffb640 1 -- 172.21.15.18:0/1320658679 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8003bb0 con 0x7f8f0c100c90 2024-02-17T13:00:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.317+0000 7f8f12e83640 1 -- 172.21.15.18:0/1320658679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c1a4480 con 0x7f8f0c100c90 2024-02-17T13:00:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.320+0000 7f8f09ffb640 1 -- 172.21.15.18:0/1320658679 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8ef8003d10 con 0x7f8f0c100c90 2024-02-17T13:00:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.320+0000 7f8f15a4c740 1 Processor -- start 2024-02-17T13:00:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.320+0000 7f8f15a4c740 1 -- start start 2024-02-17T13:00:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f15a4c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 0x55db5f04b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f15a4c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 0x55db5f04c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f15a4c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f0576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f15a4c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5efba9f0 con 0x55db5f04bd60 2024-02-17T13:00:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f15a4c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5efb7f20 con 0x55db5f0552e0 2024-02-17T13:00:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f09ffb640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8efc0a22d0 0x7f8efc0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f15a4c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5efb3700 con 0x55db5f04cad0 2024-02-17T13:00:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f10c1f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 0x55db5f04c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f11420640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8efc0a22d0 0x7f8efc0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f10c1f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 0x55db5f04c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39144/0 (socket says 172.21.15.18:39144) 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f10c1f640 1 -- 172.21.15.18:0/2105604792 learned_addr learned my addr 172.21.15.18:0/2105604792 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f11420640 1 --2- 172.21.15.18:0/2105604792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 0x55db5f04b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f0bfff640 1 --2- 172.21.15.18:0/2105604792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f0576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f09ffb640 1 -- 172.21.15.18:0/1320658679 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f8ef80f0570 con 0x7f8f0c100c90 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f10c1f640 1 -- 172.21.15.18:0/2105604792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 msgr2=0x55db5f04b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f10c1f640 1 --2- 172.21.15.18:0/2105604792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 0x55db5f04b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f10c1f640 1 -- 172.21.15.18:0/2105604792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 msgr2=0x55db5f0576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f10c1f640 1 --2- 172.21.15.18:0/2105604792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f0576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f10c1f640 1 -- 172.21.15.18:0/2105604792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db5ef497b0 con 0x55db5f04bd60 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f11420640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8efc0a22d0 0x7f8efc0a4790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8f04005fd0 tx=0x7f8f04011700 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.321+0000 7f8f10c1f640 1 --2- 172.21.15.18:0/2105604792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 0x55db5f04c1e0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009cf0 tx=0x7f8f00011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4b480dfda46aef server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.322+0000 7f8ef6ffd640 1 -- 172.21.15.18:0/2105604792 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00004020 con 0x55db5f04bd60 2024-02-17T13:00:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.322+0000 7f8ef6ffd640 1 -- 172.21.15.18:0/2105604792 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f000035f0 con 0x55db5f04bd60 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.322+0000 7f8ef6ffd640 1 -- 172.21.15.18:0/2105604792 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00018700 con 0x55db5f04bd60 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.322+0000 7f8f15a4c740 1 -- 172.21.15.18:0/2105604792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 msgr2=0x55db5f04c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.322+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/2105604792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 0x55db5f04c1e0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009cf0 tx=0x7f8f00011810 comp rx=0 tx=0).stop 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.322+0000 7f8f15a4c740 1 -- 172.21.15.18:0/2105604792 shutdown_connections 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.322+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/2105604792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f0576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.322+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/2105604792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 0x55db5f04c1e0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.322+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/2105604792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 0x55db5f04b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.322+0000 7f8f15a4c740 1 -- 172.21.15.18:0/2105604792 >> 172.21.15.18:0/2105604792 conn(0x55db5f013550 msgr2=0x55db5f046a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.322+0000 7f8f15a4c740 1 -- 172.21.15.18:0/2105604792 shutdown_connections 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.322+0000 7f8f15a4c740 1 -- 172.21.15.18:0/2105604792 wait complete. 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.323+0000 7f8f15a4c740 1 -- start start 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.323+0000 7f8f15a4c740 1 Processor -- start 2024-02-17T13:00:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.323+0000 7f8f15a4c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 0x55db5f052350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.323+0000 7f8f15a4c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 0x55db5f052890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.323+0000 7f8f15a4c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f052dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.323+0000 7f8f15a4c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5ef58bf0 con 0x55db5f04bd60 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.323+0000 7f8f15a4c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5ec87250 con 0x55db5f0552e0 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.323+0000 7f8f15a4c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5ef39200 con 0x55db5f04cad0 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.323+0000 7f8f15a4c740 10 client.0 _ll_register_callbacks cb 0x55db5efa7d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.323+0000 7f8f10c1f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 0x55db5f052350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f10c1f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 0x55db5f052350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39150/0 (socket says 172.21.15.18:39150) 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f0bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 0x55db5f052890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f10c1f640 1 -- 172.21.15.18:0/3558216923 learned_addr learned my addr 172.21.15.18:0/3558216923 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f11420640 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f052dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f10c1f640 1 -- 172.21.15.18:0/3558216923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 msgr2=0x55db5f052890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f10c1f640 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 0x55db5f052890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f10c1f640 1 -- 172.21.15.18:0/3558216923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 msgr2=0x55db5f052dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f10c1f640 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f052dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f10c1f640 1 -- 172.21.15.18:0/3558216923 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f00009950 con 0x55db5f04bd60 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f11420640 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f052dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f10c1f640 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 0x55db5f052350 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f8f00000c00 tx=0x7f8f00018940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a9a83c7c235c07 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8eebfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8eebfff640 1 -- 172.21.15.18:0/3558216923 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00018b30 con 0x55db5f04bd60 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8eebfff640 1 -- 172.21.15.18:0/3558216923 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f8f00018c90 con 0x55db5f04bd60 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3558216923 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db5ecb3600 con 0x55db5f04bd60 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8eebfff640 1 -- 172.21.15.18:0/3558216923 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f00018e50 con 0x55db5f04bd60 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f15a4c740 10 client.19281 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.324+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3558216923 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55db5efb1d80 con 0x55db5f04bd60 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.325+0000 7f8eebfff640 1 -- 172.21.15.18:0/3558216923 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55db5ecb3600 con 0x55db5f04bd60 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.325+0000 7f8eebfff640 1 -- 172.21.15.18:0/3558216923 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8f00022700 con 0x55db5f04bd60 2024-02-17T13:00:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.325+0000 7f8f15a4c740 10 client.19281 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.325+0000 7f8f15a4c740 10 client.19281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.325+0000 7f8f15a4c740 20 client.19281 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.325+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55db5f0655b0 0x55db5f0679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.325+0000 7f8f15a4c740 4 client.19281 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.325+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3558216923 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db5ed041b0 con 0x55db5f0655b0 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.325+0000 7f8f0bfff640 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55db5f0655b0 0x55db5f0679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.325+0000 7f8eebfff640 10 client.19281 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.325+0000 7f8f0bfff640 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55db5f0655b0 0x55db5f0679d0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59892d971437d04 server_cookie=c36e9cd5183dad86 in_seq=0 out_seq=0 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8eebfff640 1 -- 172.21.15.18:0/3558216923 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f8ef809be90 con 0x55db5f0655b0 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8eebfff640 10 client.19281 handle_command_reply: tid=0 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8f15a4c740 1 client.19281 shutdown 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3558216923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 msgr2=0x55db5f052350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 0x55db5f052350 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f8f00000c00 tx=0x7f8f00018940 comp rx=0 tx=0).stop 2024-02-17T13:00:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3558216923 shutdown_connections 2024-02-17T13:00:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55db5f0655b0 0x55db5f0679d0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f052dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 0x55db5f052890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8eebfff640 0 client.19281 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/3558216923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04bd60 0x55db5f052350 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3558216923 >> 172.21.15.18:0/3558216923 conn(0x55db5f013550 msgr2=0x55db5f0487a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3558216923 shutdown_connections 2024-02-17T13:00:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.326+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3558216923 wait complete. 2024-02-17T13:00:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.327+0000 7f8f15a4c740 20 client.19281 trim_cache size 0 max 16384 2024-02-17T13:00:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.340+0000 7f8f15a4c740 1 Processor -- start 2024-02-17T13:00:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.340+0000 7f8f15a4c740 1 -- start start 2024-02-17T13:00:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.340+0000 7f8f15a4c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f04bd60 0x55db5f054a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.340+0000 7f8f15a4c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 0x55db5f052bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.340+0000 7f8f15a4c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f0552e0 0x55db5f05c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f15a4c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5f04df50 con 0x55db5f0552e0 2024-02-17T13:00:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f15a4c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5f057bb0 con 0x55db5f04bd60 2024-02-17T13:00:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f15a4c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5f04b030 con 0x55db5f04cad0 2024-02-17T13:00:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f11420640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f0552e0 0x55db5f05c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f11420640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f0552e0 0x55db5f05c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39158/0 (socket says 172.21.15.18:39158) 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f11420640 1 -- 172.21.15.18:0/3121473961 learned_addr learned my addr 172.21.15.18:0/3121473961 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f10c1f640 1 --2- 172.21.15.18:0/3121473961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f04bd60 0x55db5f054a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f11420640 1 -- 172.21.15.18:0/3121473961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 msgr2=0x55db5f052bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f11420640 1 --2- 172.21.15.18:0/3121473961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 0x55db5f052bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f11420640 1 -- 172.21.15.18:0/3121473961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f04bd60 msgr2=0x55db5f054a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f11420640 1 --2- 172.21.15.18:0/3121473961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f04bd60 0x55db5f054a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f11420640 1 -- 172.21.15.18:0/3121473961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db5ef4e7b0 con 0x55db5f0552e0 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8f11420640 1 --2- 172.21.15.18:0/3121473961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f0552e0 0x55db5f05c100 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f8f04011b50 tx=0x7f8f04021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d8f5d49e837b26 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8ef67fc640 1 -- 172.21.15.18:0/3121473961 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04016050 con 0x55db5f0552e0 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8ef67fc640 1 -- 172.21.15.18:0/3121473961 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f0401ed90 con 0x55db5f0552e0 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.341+0000 7f8ef67fc640 1 -- 172.21.15.18:0/3121473961 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04010670 con 0x55db5f0552e0 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.342+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3121473961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f0552e0 msgr2=0x55db5f05c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.342+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/3121473961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f0552e0 0x55db5f05c100 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f8f04011b50 tx=0x7f8f04021000 comp rx=0 tx=0).stop 2024-02-17T13:00:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.342+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3121473961 shutdown_connections 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.342+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/3121473961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f0552e0 0x55db5f05c100 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.342+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/3121473961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04cad0 0x55db5f052bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.342+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/3121473961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f04bd60 0x55db5f054a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.342+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3121473961 >> 172.21.15.18:0/3121473961 conn(0x55db5f013550 msgr2=0x55db5f046580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.342+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3121473961 shutdown_connections 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.342+0000 7f8f15a4c740 1 -- 172.21.15.18:0/3121473961 wait complete. 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.342+0000 7f8f15a4c740 1 -- start start 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.342+0000 7f8f15a4c740 1 Processor -- start 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f15a4c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04bd60 0x55db5f0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f15a4c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04cad0 0x55db5f0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f15a4c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f058510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f15a4c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5f058b20 con 0x55db5f04cad0 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f15a4c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5f058c90 con 0x55db5f0552e0 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f15a4c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5f058e50 con 0x55db5f04bd60 2024-02-17T13:00:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f15a4c740 10 client.0 _ll_register_callbacks cb 0x7f8f000097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f0bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04cad0 0x55db5f0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f10c1f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04bd60 0x55db5f0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f0bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04cad0 0x55db5f0b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39162/0 (socket says 172.21.15.18:39162) 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f0bfff640 1 -- 172.21.15.18:0/1579633844 learned_addr learned my addr 172.21.15.18:0/1579633844 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f11420640 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f058510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f0bfff640 1 -- 172.21.15.18:0/1579633844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04bd60 msgr2=0x55db5f0b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f0bfff640 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04bd60 0x55db5f0b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f0bfff640 1 -- 172.21.15.18:0/1579633844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 msgr2=0x55db5f058510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f0bfff640 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f058510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f0bfff640 1 -- 172.21.15.18:0/1579633844 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f04019040 con 0x55db5f04cad0 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.343+0000 7f8f11420640 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f058510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8f0bfff640 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04cad0 0x55db5f0b5370 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80ede30 tx=0x7f8ef8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5b4d2816e45d8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8eebfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8eebfff640 1 -- 172.21.15.18:0/1579633844 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80c11c0 con 0x55db5f04cad0 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8eebfff640 1 -- 172.21.15.18:0/1579633844 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f8ef80c5810 con 0x55db5f04cad0 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8f15a4c740 1 -- 172.21.15.18:0/1579633844 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f000035f0 con 0x55db5f04cad0 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8eebfff640 1 -- 172.21.15.18:0/1579633844 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ef80b96e0 con 0x55db5f04cad0 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8f15a4c740 10 client.19287 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8f15a4c740 1 -- 172.21.15.18:0/1579633844 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55db5ef74540 con 0x55db5f04cad0 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8eebfff640 1 -- 172.21.15.18:0/1579633844 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f000035f0 con 0x55db5f04cad0 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8eebfff640 1 -- 172.21.15.18:0/1579633844 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8ef803c070 con 0x55db5f04cad0 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8f15a4c740 10 client.19287 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8f15a4c740 10 client.19287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8f15a4c740 20 client.19287 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55db5f0bca00 0x55db5f0bee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8f15a4c740 4 client.19287 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:00:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8f15a4c740 1 -- 172.21.15.18:0/1579633844 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55db5ecf0550 con 0x55db5f0bca00 2024-02-17T13:00:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.344+0000 7f8f10c1f640 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55db5f0bca00 0x55db5f0bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.345+0000 7f8f10c1f640 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55db5f0bca00 0x55db5f0bee20 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edbcbb81cc857f93 server_cookie=ada6d2758fedb5c7 in_seq=0 out_seq=0 2024-02-17T13:00:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.345+0000 7f8eebfff640 10 client.19287 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.375+0000 7f8eebfff640 1 -- 172.21.15.18:0/1579633844 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55db5ecf0550 con 0x55db5f0bca00 2024-02-17T13:00:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.375+0000 7f8eebfff640 10 client.19287 handle_command_reply: tid=0 2024-02-17T13:00:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.375+0000 7f8f15a4c740 1 client.19287 shutdown 2024-02-17T13:00:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.375+0000 7f8f15a4c740 1 -- 172.21.15.18:0/1579633844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04cad0 msgr2=0x55db5f0b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.375+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04cad0 0x55db5f0b5370 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80ede30 tx=0x7f8ef8009260 comp rx=0 tx=0).stop 2024-02-17T13:00:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.375+0000 7f8f15a4c740 1 -- 172.21.15.18:0/1579633844 shutdown_connections 2024-02-17T13:00:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.375+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55db5f0bca00 0x55db5f0bee20 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55db5f0552e0 0x55db5f058510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55db5f04cad0 0x55db5f0b5370 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f15a4c740 1 --2- 172.21.15.18:0/1579633844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db5f04bd60 0x55db5f0b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8eebfff640 0 client.19287 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f15a4c740 1 -- 172.21.15.18:0/1579633844 >> 172.21.15.18:0/1579633844 conn(0x55db5f013550 msgr2=0x55db5f050b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f15a4c740 1 -- 172.21.15.18:0/1579633844 shutdown_connections 2024-02-17T13:00:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f15a4c740 1 -- 172.21.15.18:0/1579633844 wait complete. 2024-02-17T13:00:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f15a4c740 20 client.19287 trim_cache size 0 max 16384 2024-02-17T13:00:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:00:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:00:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:00:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:00:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f12e83640 1 -- 172.21.15.18:0/1320658679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8efc0a22d0 msgr2=0x7f8efc0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f12e83640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8efc0a22d0 0x7f8efc0a4790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8f04005fd0 tx=0x7f8f04011700 comp rx=0 tx=0).stop 2024-02-17T13:00:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f12e83640 1 -- 172.21.15.18:0/1320658679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f0c100c90 msgr2=0x7f8f0c197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f12e83640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f0c100c90 0x7f8f0c197cc0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8000c00 tx=0x7f8ef802c6b0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f12e83640 1 -- 172.21.15.18:0/1320658679 shutdown_connections 2024-02-17T13:00:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f12e83640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8efc0a22d0 0x7f8efc0a4790 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f12e83640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0c105020 0x7f8f0c19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f12e83640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f0c100c90 0x7f8f0c197cc0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f12e83640 1 --2- 172.21.15.18:0/1320658679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f0c1002d0 0x7f8f0c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f12e83640 1 -- 172.21.15.18:0/1320658679 >> 172.21.15.18:0/1320658679 conn(0x7f8f0c076ed0 msgr2=0x7f8f0c0689e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f12e83640 1 -- 172.21.15.18:0/1320658679 shutdown_connections 2024-02-17T13:00:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:10.376+0000 7f8f12e83640 1 -- 172.21.15.18:0/1320658679 wait complete. 2024-02-17T13:00:10.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:10 smithi146 ceph-mon[36454]: pgmap v321: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-02-17T13:00:11.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:10 smithi018 ceph-mon[32625]: pgmap v321: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-02-17T13:00:11.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:10 smithi119 ceph-mon[36188]: pgmap v321: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-02-17T13:00:11.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 520365 5.05 MB/sec execute 446 sec latency 539.337 ms 2024-02-17T13:00:12.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 520365 5.04 MB/sec execute 447 sec latency 1539.427 ms 2024-02-17T13:00:12.951 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:12 smithi146 ceph-mon[36454]: pgmap v322: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-02-17T13:00:13.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:12 smithi018 ceph-mon[32625]: pgmap v322: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-02-17T13:00:13.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:12 smithi119 ceph-mon[36188]: pgmap v322: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-02-17T13:00:13.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 521546 5.04 MB/sec execute 448 sec latency 2098.000 ms 2024-02-17T13:00:14.294 INFO:tasks.workunit.client.0.smithi018.stdout: 1 522994 5.05 MB/sec execute 449 sec latency 27.337 ms 2024-02-17T13:00:14.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:14 smithi146 ceph-mon[36454]: pgmap v323: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-02-17T13:00:14.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:14 smithi146 ceph-mon[36454]: osdmap e100: 12 total, 12 up, 12 in 2024-02-17T13:00:15.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:14 smithi018 ceph-mon[32625]: pgmap v323: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-02-17T13:00:15.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:14 smithi018 ceph-mon[32625]: osdmap e100: 12 total, 12 up, 12 in 2024-02-17T13:00:15.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:14 smithi119 ceph-mon[36188]: pgmap v323: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-02-17T13:00:15.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:14 smithi119 ceph-mon[36188]: osdmap e100: 12 total, 12 up, 12 in 2024-02-17T13:00:15.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 523906 5.04 MB/sec execute 450 sec latency 505.648 ms 2024-02-17T13:00:15.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:15 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:00:15.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:15 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:00:15.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:15 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:00:16.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:15 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:00:16.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:15 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:00:16.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:15 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:00:16.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:15 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:00:16.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:15 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:00:16.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:15 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:00:16.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 523906 5.03 MB/sec execute 451 sec latency 1505.764 ms 2024-02-17T13:00:16.923 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:16 smithi018 ceph-mon[32625]: pgmap v325: 97 pgs: 97 active+clean; 204 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-02-17T13:00:16.924 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:16 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:00:16.924 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:16 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:00:16.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:16 smithi146 ceph-mon[36454]: pgmap v325: 97 pgs: 97 active+clean; 204 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-02-17T13:00:16.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:16 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:00:16.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:16 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:00:17.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:16 smithi119 ceph-mon[36188]: pgmap v325: 97 pgs: 97 active+clean; 204 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-02-17T13:00:17.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:16 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:00:17.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:16 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:00:17.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 523906 5.02 MB/sec execute 452 sec latency 2505.827 ms 2024-02-17T13:00:18.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 523906 5.01 MB/sec execute 453 sec latency 3505.889 ms 2024-02-17T13:00:18.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:18 smithi146 ceph-mon[36454]: pgmap v326: 97 pgs: 97 active+clean; 204 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-02-17T13:00:19.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:18 smithi018 ceph-mon[32625]: pgmap v326: 97 pgs: 97 active+clean; 204 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-02-17T13:00:19.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:18 smithi119 ceph-mon[36188]: pgmap v326: 97 pgs: 97 active+clean; 204 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-02-17T13:00:19.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 525823 5.03 MB/sec execute 454 sec latency 3687.934 ms 2024-02-17T13:00:20.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 527212 5.03 MB/sec execute 455 sec latency 20.440 ms 2024-02-17T13:00:20.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:20 smithi146 ceph-mon[36454]: pgmap v327: 97 pgs: 97 active+clean; 205 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-02-17T13:00:21.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:20 smithi018 ceph-mon[32625]: pgmap v327: 97 pgs: 97 active+clean; 205 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-02-17T13:00:21.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:20 smithi119 ceph-mon[36188]: pgmap v327: 97 pgs: 97 active+clean; 205 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-02-17T13:00:21.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 527456 5.02 MB/sec execute 456 sec latency 840.853 ms 2024-02-17T13:00:22.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 527456 5.01 MB/sec execute 457 sec latency 1840.969 ms 2024-02-17T13:00:22.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:22 smithi146 ceph-mon[36454]: pgmap v328: 97 pgs: 97 active+clean; 205 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-02-17T13:00:23.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:22 smithi018 ceph-mon[32625]: pgmap v328: 97 pgs: 97 active+clean; 205 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-02-17T13:00:23.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:22 smithi119 ceph-mon[36188]: pgmap v328: 97 pgs: 97 active+clean; 205 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-02-17T13:00:23.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 527456 5.00 MB/sec execute 458 sec latency 2841.069 ms 2024-02-17T13:00:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.476+0000 7f7fd936c640 1 -- 172.21.15.18:0/707487929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fd41013a0 msgr2=0x7f7fd4101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.476+0000 7f7fd936c640 1 --2- 172.21.15.18:0/707487929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fd41013a0 0x7f7fd4101780 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009930 tx=0x7f7fc402bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.476+0000 7f7fd936c640 1 -- 172.21.15.18:0/707487929 shutdown_connections 2024-02-17T13:00:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.476+0000 7f7fd936c640 1 --2- 172.21.15.18:0/707487929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fd4106470 0x7f7fd410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.476+0000 7f7fd936c640 1 --2- 172.21.15.18:0/707487929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4101d50 0x7f7fd4105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.476+0000 7f7fd936c640 1 --2- 172.21.15.18:0/707487929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fd41013a0 0x7f7fd4101780 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.476+0000 7f7fd936c640 1 -- 172.21.15.18:0/707487929 >> 172.21.15.18:0/707487929 conn(0x7f7fd40fcc40 msgr2=0x7f7fd40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.476+0000 7f7fd936c640 1 -- 172.21.15.18:0/707487929 shutdown_connections 2024-02-17T13:00:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.476+0000 7f7fd936c640 1 -- 172.21.15.18:0/707487929 wait complete. 2024-02-17T13:00:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.476+0000 7f7fd936c640 1 Processor -- start 2024-02-17T13:00:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.476+0000 7f7fd936c640 1 -- start start 2024-02-17T13:00:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd936c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fd41013a0 0x7f7fd4197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd936c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4106470 0x7f7fd4197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd936c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fd419bff0 0x7f7fd419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd936c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd419c9e0 con 0x7f7fd41013a0 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd936c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd419cb50 con 0x7f7fd4106470 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd936c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd419ccc0 con 0x7f7fd419bff0 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fd41013a0 0x7f7fd4197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fd41013a0 0x7f7fd4197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53782/0 (socket says 172.21.15.18:53782) 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd2ffd640 1 -- 172.21.15.18:0/1686926733 learned_addr learned my addr 172.21.15.18:0/1686926733 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd27fc640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4106470 0x7f7fd4197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fd419bff0 0x7f7fd419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd2ffd640 1 -- 172.21.15.18:0/1686926733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fd419bff0 msgr2=0x7f7fd419c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd2ffd640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fd419bff0 0x7f7fd419c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd2ffd640 1 -- 172.21.15.18:0/1686926733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4106470 msgr2=0x7f7fd4197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd2ffd640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4106470 0x7f7fd4197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.477+0000 7f7fd2ffd640 1 -- 172.21.15.18:0/1686926733 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc4009560 con 0x7f7fd41013a0 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.478+0000 7f7fd2ffd640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fd41013a0 0x7f7fd4197730 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009230 tx=0x7f7fc402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.478+0000 7f7fbbfff640 1 -- 172.21.15.18:0/1686926733 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4004350 con 0x7f7fd41013a0 2024-02-17T13:00:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.478+0000 7f7fbbfff640 1 -- 172.21.15.18:0/1686926733 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc40044b0 con 0x7f7fd41013a0 2024-02-17T13:00:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.478+0000 7f7fd936c640 1 -- 172.21.15.18:0/1686926733 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd41a42a0 con 0x7f7fd41013a0 2024-02-17T13:00:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.478+0000 7f7fbbfff640 1 -- 172.21.15.18:0/1686926733 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4003d70 con 0x7f7fd41013a0 2024-02-17T13:00:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.478+0000 7f7fd936c640 1 -- 172.21.15.18:0/1686926733 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd41a4720 con 0x7f7fd41013a0 2024-02-17T13:00:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.480+0000 7f7fbbfff640 1 -- 172.21.15.18:0/1686926733 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7fc4004df0 con 0x7f7fd41013a0 2024-02-17T13:00:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fdc019740 1 Processor -- start 2024-02-17T13:00:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fdc019740 1 -- start start 2024-02-17T13:00:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fdc019740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc232b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fbbfff640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7fbc0a22d0 0x7f7fbc0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fdc019740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc232bd60 0x564cc232c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fdc019740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 0x564cc233bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fdc019740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc229a9f0 con 0x564cc23396f0 2024-02-17T13:00:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fdc019740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc2297f20 con 0x564cc232cad0 2024-02-17T13:00:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fdc019740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc2293700 con 0x564cc232bd60 2024-02-17T13:00:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fd2ffd640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7fbc0a22d0 0x7f7fbc0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 0x564cc233bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fd27fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc232bd60 0x564cc232c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 0x564cc233bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53796/0 (socket says 172.21.15.18:53796) 2024-02-17T13:00:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fd27fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc232bd60 0x564cc232c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:41270/0 (socket says 172.21.15.18:41270) 2024-02-17T13:00:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fd37fe640 1 -- 172.21.15.18:0/244778582 learned_addr learned my addr 172.21.15.18:0/244778582 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fbbfff640 1 -- 172.21.15.18:0/1686926733 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7fc40f0010 con 0x7f7fd41013a0 2024-02-17T13:00:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.481+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/244778582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc232b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.482+0000 7f7fd27fc640 1 -- 172.21.15.18:0/244778582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 msgr2=0x564cc232b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.482+0000 7f7fd27fc640 1 --2- 172.21.15.18:0/244778582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc232b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.482+0000 7f7fd27fc640 1 -- 172.21.15.18:0/244778582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 msgr2=0x564cc233bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.482+0000 7f7fd27fc640 1 --2- 172.21.15.18:0/244778582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 0x564cc233bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.482+0000 7f7fd27fc640 1 -- 172.21.15.18:0/244778582 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cc22297b0 con 0x564cc232bd60 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.482+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/244778582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 0x564cc233bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.482+0000 7f7fd2ffd640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7fbc0a22d0 0x7f7fbc0a4790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40bc830 tx=0x7f7fc408bcd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.482+0000 7f7fd27fc640 1 --2- 172.21.15.18:0/244778582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc232bd60 0x564cc232c1e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7fd406a830 tx=0x7f7fc8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=784d36f24971bd3e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.482+0000 7f7fb97fa640 1 -- 172.21.15.18:0/244778582 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8004050 con 0x564cc232bd60 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.482+0000 7f7fb97fa640 1 -- 172.21.15.18:0/244778582 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc8005500 con 0x564cc232bd60 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.482+0000 7f7fb97fa640 1 -- 172.21.15.18:0/244778582 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8004b80 con 0x564cc232bd60 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.483+0000 7f7fdc019740 1 -- 172.21.15.18:0/244778582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc232bd60 msgr2=0x564cc232c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.483+0000 7f7fdc019740 1 --2- 172.21.15.18:0/244778582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc232bd60 0x564cc232c1e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7fd406a830 tx=0x7f7fc8011810 comp rx=0 tx=0).stop 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.483+0000 7f7fdc019740 1 -- 172.21.15.18:0/244778582 shutdown_connections 2024-02-17T13:00:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.483+0000 7f7fdc019740 1 --2- 172.21.15.18:0/244778582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 0x564cc233bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.483+0000 7f7fdc019740 1 --2- 172.21.15.18:0/244778582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc232bd60 0x564cc232c1e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.483+0000 7f7fdc019740 1 --2- 172.21.15.18:0/244778582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc232b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.483+0000 7f7fdc019740 1 -- 172.21.15.18:0/244778582 >> 172.21.15.18:0/244778582 conn(0x564cc22f3550 msgr2=0x564cc2326a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.483+0000 7f7fdc019740 1 -- 172.21.15.18:0/244778582 shutdown_connections 2024-02-17T13:00:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.483+0000 7f7fdc019740 1 -- 172.21.15.18:0/244778582 wait complete. 2024-02-17T13:00:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.483+0000 7f7fdc019740 1 -- start start 2024-02-17T13:00:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.483+0000 7f7fdc019740 1 Processor -- start 2024-02-17T13:00:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fdc019740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232bd60 0x564cc2336700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fdc019740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc2336c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fdc019740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc23396f0 0x564cc2337180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fdc019740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc2238bf0 con 0x564cc232bd60 2024-02-17T13:00:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fdc019740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc1f67250 con 0x564cc232cad0 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fdc019740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc2219200 con 0x564cc23396f0 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fdc019740 10 client.0 _ll_register_callbacks cb 0x564cc2287d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232bd60 0x564cc2336700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232bd60 0x564cc2336700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53798/0 (socket says 172.21.15.18:53798) 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fd37fe640 1 -- 172.21.15.18:0/3833089585 learned_addr learned my addr 172.21.15.18:0/3833089585 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc23396f0 0x564cc2337180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fd27fc640 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc2336c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fd37fe640 1 -- 172.21.15.18:0/3833089585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc23396f0 msgr2=0x564cc2337180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc23396f0 0x564cc2337180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fd37fe640 1 -- 172.21.15.18:0/3833089585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 msgr2=0x564cc2336c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc2336c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fd37fe640 1 -- 172.21.15.18:0/3833089585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc8009850 con 0x564cc232bd60 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232bd60 0x564cc2336700 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc012e10 tx=0x7f7fcc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f081ec9c81a64393 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fb6ff5640 1 -- 172.21.15.18:0/3833089585 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc004d20 con 0x564cc232bd60 2024-02-17T13:00:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fb6ff5640 1 -- 172.21.15.18:0/3833089585 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7fcc020060 con 0x564cc232bd60 2024-02-17T13:00:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.484+0000 7f7fdc019740 1 -- 172.21.15.18:0/3833089585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564cc1f93600 con 0x564cc232bd60 2024-02-17T13:00:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.485+0000 7f7fb6ff5640 1 -- 172.21.15.18:0/3833089585 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fcc004e80 con 0x564cc232bd60 2024-02-17T13:00:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.485+0000 7f7fdc019740 10 client.19299 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.485+0000 7f7fdc019740 1 -- 172.21.15.18:0/3833089585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x564cc2291d80 con 0x564cc232bd60 2024-02-17T13:00:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.485+0000 7f7fb6ff5640 1 -- 172.21.15.18:0/3833089585 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc028020 con 0x564cc232bd60 2024-02-17T13:00:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.485+0000 7f7fb6ff5640 1 -- 172.21.15.18:0/3833089585 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7fcc005000 con 0x564cc232bd60 2024-02-17T13:00:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.485+0000 7f7fdc019740 10 client.19299 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.485+0000 7f7fdc019740 10 client.19299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.485+0000 7f7fdc019740 20 client.19299 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.485+0000 7f7fdc019740 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564cc2349b50 0x564cc234bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.485+0000 7f7fdc019740 4 client.19299 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.485+0000 7f7fdc019740 1 -- 172.21.15.18:0/3833089585 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cc1fe41b0 con 0x564cc2349b50 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.485+0000 7f7fd27fc640 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564cc2349b50 0x564cc234bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.486+0000 7f7fb6ff5640 10 client.19299 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.486+0000 7f7fd27fc640 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564cc2349b50 0x564cc234bf70 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e69bbbae5a25c7f server_cookie=8fe2d7d4ac06800f in_seq=0 out_seq=0 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fb6ff5640 1 -- 172.21.15.18:0/3833089585 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564cc1fe41b0 con 0x564cc2349b50 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fb6ff5640 10 client.19299 handle_command_reply: tid=0 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fdc019740 1 client.19299 shutdown 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fdc019740 1 -- 172.21.15.18:0/3833089585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232bd60 msgr2=0x564cc2336700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fdc019740 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232bd60 0x564cc2336700 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc012e10 tx=0x7f7fcc014040 comp rx=0 tx=0).stop 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fdc019740 1 -- 172.21.15.18:0/3833089585 shutdown_connections 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fdc019740 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564cc2349b50 0x564cc234bf70 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fdc019740 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc23396f0 0x564cc2337180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fdc019740 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc2336c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fb6ff5640 0 client.19299 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fdc019740 1 --2- 172.21.15.18:0/3833089585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232bd60 0x564cc2336700 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fdc019740 1 -- 172.21.15.18:0/3833089585 >> 172.21.15.18:0/3833089585 conn(0x564cc22f3550 msgr2=0x564cc23287a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fdc019740 1 -- 172.21.15.18:0/3833089585 shutdown_connections 2024-02-17T13:00:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fdc019740 1 -- 172.21.15.18:0/3833089585 wait complete. 2024-02-17T13:00:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.487+0000 7f7fdc019740 20 client.19299 trim_cache size 0 max 16384 2024-02-17T13:00:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.500+0000 7f7fdc019740 1 Processor -- start 2024-02-17T13:00:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.500+0000 7f7fdc019740 1 -- start start 2024-02-17T13:00:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.500+0000 7f7fdc019740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc232bd60 0x564cc2338e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.500+0000 7f7fdc019740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc2340510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.500+0000 7f7fdc019740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 0x564cc2340a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.500+0000 7f7fdc019740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc1fe41b0 con 0x564cc23396f0 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.500+0000 7f7fdc019740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc2346560 con 0x564cc232cad0 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.500+0000 7f7fdc019740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc23482a0 con 0x564cc232bd60 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 0x564cc2340a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 0x564cc2340a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53808/0 (socket says 172.21.15.18:53808) 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fd37fe640 1 -- 172.21.15.18:0/1785651179 learned_addr learned my addr 172.21.15.18:0/1785651179 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/1785651179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc232bd60 0x564cc2338e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fd27fc640 1 --2- 172.21.15.18:0/1785651179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc2340510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fd37fe640 1 -- 172.21.15.18:0/1785651179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc232bd60 msgr2=0x564cc2338e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/1785651179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc232bd60 0x564cc2338e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fd37fe640 1 -- 172.21.15.18:0/1785651179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 msgr2=0x564cc2340510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/1785651179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc2340510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fd37fe640 1 -- 172.21.15.18:0/1785651179 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cc222e7b0 con 0x564cc23396f0 2024-02-17T13:00:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fd27fc640 1 --2- 172.21.15.18:0/1785651179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc2340510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:00:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/1785651179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 0x564cc2340a50 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009d30 tx=0x7f7fcc009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41781f7de4d79c87 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fb8ff9640 1 -- 172.21.15.18:0/1785651179 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc005780 con 0x564cc23396f0 2024-02-17T13:00:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fb8ff9640 1 -- 172.21.15.18:0/1785651179 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc028070 con 0x564cc23396f0 2024-02-17T13:00:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.501+0000 7f7fb8ff9640 1 -- 172.21.15.18:0/1785651179 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0051c0 con 0x564cc23396f0 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 -- 172.21.15.18:0/1785651179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 msgr2=0x564cc2340a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 --2- 172.21.15.18:0/1785651179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 0x564cc2340a50 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009d30 tx=0x7f7fcc009bc0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 -- 172.21.15.18:0/1785651179 shutdown_connections 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 --2- 172.21.15.18:0/1785651179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc23396f0 0x564cc2340a50 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 --2- 172.21.15.18:0/1785651179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232cad0 0x564cc2340510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 --2- 172.21.15.18:0/1785651179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc232bd60 0x564cc2338e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 -- 172.21.15.18:0/1785651179 >> 172.21.15.18:0/1785651179 conn(0x564cc22f3550 msgr2=0x564cc2326580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 -- 172.21.15.18:0/1785651179 shutdown_connections 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 -- 172.21.15.18:0/1785651179 wait complete. 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 -- start start 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 Processor -- start 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232bd60 0x564cc23409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232cad0 0x564cc233cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc23396f0 0x564cc233d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc233dbc0 con 0x564cc232cad0 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc23a0030 con 0x564cc232bd60 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc23a0170 con 0x564cc23396f0 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fdc019740 10 client.0 _ll_register_callbacks cb 0x7f7fc80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fd27fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232cad0 0x564cc233cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fd27fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232cad0 0x564cc233cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53814/0 (socket says 172.21.15.18:53814) 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.502+0000 7f7fd27fc640 1 -- 172.21.15.18:0/1938327380 learned_addr learned my addr 172.21.15.18:0/1938327380 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232bd60 0x564cc23409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc23396f0 0x564cc233d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fd27fc640 1 -- 172.21.15.18:0/1938327380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc23396f0 msgr2=0x564cc233d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fd27fc640 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc23396f0 0x564cc233d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fd27fc640 1 -- 172.21.15.18:0/1938327380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232bd60 msgr2=0x564cc23409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fd27fc640 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232bd60 0x564cc23409e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fd27fc640 1 -- 172.21.15.18:0/1938327380 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cc2291d80 con 0x564cc232cad0 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232bd60 0x564cc23409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fd27fc640 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232cad0 0x564cc233cdb0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009380 tx=0x7f7fc8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b393cd088aa76359 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fb6ff5640 1 -- 172.21.15.18:0/1938327380 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8029070 con 0x564cc232cad0 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fb6ff5640 1 -- 172.21.15.18:0/1938327380 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7fc802f030 con 0x564cc232cad0 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fdc019740 1 -- 172.21.15.18:0/1938327380 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fc8005500 con 0x564cc232cad0 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fb6ff5640 1 -- 172.21.15.18:0/1938327380 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fc8009cb0 con 0x564cc232cad0 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fdc019740 10 client.19305 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fdc019740 1 -- 172.21.15.18:0/1938327380 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x564cc2254540 con 0x564cc232cad0 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.503+0000 7f7fb6ff5640 1 -- 172.21.15.18:0/1938327380 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc8005500 con 0x564cc232cad0 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.504+0000 7f7fb6ff5640 1 -- 172.21.15.18:0/1938327380 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7fc8021040 con 0x564cc232cad0 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.504+0000 7f7fdc019740 10 client.19305 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.504+0000 7f7fdc019740 10 client.19305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.504+0000 7f7fdc019740 20 client.19305 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.504+0000 7f7fdc019740 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564cc23a1960 0x564cc23a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.504+0000 7f7fdc019740 4 client.19305 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:00:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.504+0000 7f7fdc019740 1 -- 172.21.15.18:0/1938327380 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564cc1fd0550 con 0x564cc23a1960 2024-02-17T13:00:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.504+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564cc23a1960 0x564cc23a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.504+0000 7f7fb6ff5640 10 client.19305 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.504+0000 7f7fd37fe640 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564cc23a1960 0x564cc23a3d80 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f998e6d16453bf38 server_cookie=986eb093345ced0a in_seq=0 out_seq=0 2024-02-17T13:00:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fb6ff5640 1 -- 172.21.15.18:0/1938327380 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564cc1fd0550 con 0x564cc23a1960 2024-02-17T13:00:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fb6ff5640 10 client.19305 handle_command_reply: tid=0 2024-02-17T13:00:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fdc019740 1 client.19305 shutdown 2024-02-17T13:00:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fdc019740 1 -- 172.21.15.18:0/1938327380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232cad0 msgr2=0x564cc233cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fdc019740 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232cad0 0x564cc233cdb0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009380 tx=0x7f7fc8005f10 comp rx=0 tx=0).stop 2024-02-17T13:00:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fdc019740 1 -- 172.21.15.18:0/1938327380 shutdown_connections 2024-02-17T13:00:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fdc019740 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564cc23a1960 0x564cc23a3d80 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fdc019740 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cc23396f0 0x564cc233d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fdc019740 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564cc232cad0 0x564cc233cdb0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fb6ff5640 0 client.19305 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fdc019740 1 --2- 172.21.15.18:0/1938327380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564cc232bd60 0x564cc23409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fdc019740 1 -- 172.21.15.18:0/1938327380 >> 172.21.15.18:0/1938327380 conn(0x564cc22f3550 msgr2=0x564cc2334f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.537+0000 7f7fdc019740 1 -- 172.21.15.18:0/1938327380 shutdown_connections 2024-02-17T13:00:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fdc019740 1 -- 172.21.15.18:0/1938327380 wait complete. 2024-02-17T13:00:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fdc019740 20 client.19305 trim_cache size 0 max 16384 2024-02-17T13:00:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:00:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:00:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:00:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:00:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fd936c640 1 -- 172.21.15.18:0/1686926733 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7fbc0a22d0 msgr2=0x7f7fbc0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fd936c640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7fbc0a22d0 0x7f7fbc0a4790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40bc830 tx=0x7f7fc408bcd0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fd936c640 1 -- 172.21.15.18:0/1686926733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fd41013a0 msgr2=0x7f7fd4197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fd936c640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fd41013a0 0x7f7fd4197730 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009230 tx=0x7f7fc402c6c0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fd936c640 1 -- 172.21.15.18:0/1686926733 shutdown_connections 2024-02-17T13:00:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fd936c640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7fbc0a22d0 0x7f7fbc0a4790 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fd936c640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fd419bff0 0x7f7fd419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fd936c640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4106470 0x7f7fd4197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fd936c640 1 --2- 172.21.15.18:0/1686926733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fd41013a0 0x7f7fd4197730 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fd936c640 1 -- 172.21.15.18:0/1686926733 >> 172.21.15.18:0/1686926733 conn(0x7f7fd40fcc40 msgr2=0x7f7fd40fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fd936c640 1 -- 172.21.15.18:0/1686926733 shutdown_connections 2024-02-17T13:00:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:23.538+0000 7f7fd936c640 1 -- 172.21.15.18:0/1686926733 wait complete. 2024-02-17T13:00:24.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 528322 4.99 MB/sec execute 459 sec latency 3467.281 ms 2024-02-17T13:00:24.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:24 smithi146 ceph-mon[36454]: pgmap v329: 97 pgs: 97 active+clean; 205 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-02-17T13:00:25.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:24 smithi018 ceph-mon[32625]: pgmap v329: 97 pgs: 97 active+clean; 205 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-02-17T13:00:25.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:24 smithi119 ceph-mon[36188]: pgmap v329: 97 pgs: 97 active+clean; 205 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-02-17T13:00:25.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 529988 5.01 MB/sec execute 460 sec latency 13.836 ms 2024-02-17T13:00:26.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 530999 5.01 MB/sec execute 461 sec latency 439.788 ms 2024-02-17T13:00:26.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:26 smithi146 ceph-mon[36454]: pgmap v330: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-17T13:00:27.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:26 smithi018 ceph-mon[32625]: pgmap v330: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-17T13:00:27.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:26 smithi119 ceph-mon[36188]: pgmap v330: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-17T13:00:27.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 530999 5.00 MB/sec execute 462 sec latency 1439.887 ms 2024-02-17T13:00:28.295 INFO:tasks.workunit.client.0.smithi018.stdout: 1 530999 4.99 MB/sec execute 463 sec latency 2439.980 ms 2024-02-17T13:00:28.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:28 smithi146 ceph-mon[36454]: pgmap v331: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-02-17T13:00:29.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:28 smithi018 ceph-mon[32625]: pgmap v331: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-02-17T13:00:29.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:28 smithi119 ceph-mon[36188]: pgmap v331: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-02-17T13:00:29.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 531936 4.98 MB/sec execute 464 sec latency 3041.665 ms 2024-02-17T13:00:30.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 533564 4.99 MB/sec execute 465 sec latency 11.980 ms 2024-02-17T13:00:30.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:30 smithi146 ceph-mon[36454]: pgmap v332: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2024-02-17T13:00:31.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:30 smithi018 ceph-mon[32625]: pgmap v332: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2024-02-17T13:00:31.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:30 smithi119 ceph-mon[36188]: pgmap v332: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2024-02-17T13:00:31.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 534579 4.99 MB/sec execute 466 sec latency 373.444 ms 2024-02-17T13:00:32.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 534579 4.98 MB/sec execute 467 sec latency 1373.601 ms 2024-02-17T13:00:32.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:32 smithi146 ceph-mon[36454]: pgmap v333: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-02-17T13:00:33.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:32 smithi018 ceph-mon[32625]: pgmap v333: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-02-17T13:00:33.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:32 smithi119 ceph-mon[36188]: pgmap v333: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-02-17T13:00:33.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 535648 4.99 MB/sec execute 468 sec latency 1931.271 ms 2024-02-17T13:00:34.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 537042 4.99 MB/sec execute 469 sec latency 64.435 ms 2024-02-17T13:00:34.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:34 smithi146 ceph-mon[36454]: pgmap v334: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-02-17T13:00:35.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:34 smithi018 ceph-mon[32625]: pgmap v334: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-02-17T13:00:35.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:34 smithi119 ceph-mon[36188]: pgmap v334: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-02-17T13:00:35.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 538122 4.99 MB/sec execute 470 sec latency 392.257 ms 2024-02-17T13:00:36.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:00:36.008 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:00:36.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.138+0000 7fa076b65640 1 -- 172.21.15.18:0/1433767848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0701023a0 msgr2=0x7fa070102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.138+0000 7fa076b65640 1 --2- 172.21.15.18:0/1433767848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0701023a0 0x7fa070102780 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fa064009900 tx=0x7fa06402bcd0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.138+0000 7fa076b65640 1 -- 172.21.15.18:0/1433767848 shutdown_connections 2024-02-17T13:00:36.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.138+0000 7fa076b65640 1 --2- 172.21.15.18:0/1433767848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa07010b6e0 0x7fa07010db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.138+0000 7fa076b65640 1 --2- 172.21.15.18:0/1433767848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa070102cc0 0x7fa07010b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.138+0000 7fa076b65640 1 --2- 172.21.15.18:0/1433767848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0701023a0 0x7fa070102780 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.138+0000 7fa076b65640 1 -- 172.21.15.18:0/1433767848 >> 172.21.15.18:0/1433767848 conn(0x7fa070077230 msgr2=0x7fa070077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.138+0000 7fa076b65640 1 -- 172.21.15.18:0/1433767848 shutdown_connections 2024-02-17T13:00:36.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.138+0000 7fa076b65640 1 -- 172.21.15.18:0/1433767848 wait complete. 2024-02-17T13:00:36.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa076b65640 1 Processor -- start 2024-02-17T13:00:36.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa076b65640 1 -- start start 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa076b65640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0701023a0 0x7fa0700ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa076b65640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa070102cc0 0x7fa070100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa076b65640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa07010b6e0 0x7fa070100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa076b65640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0701a5bd0 con 0x7fa070102cc0 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa076b65640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0701a5d40 con 0x7fa07010b6e0 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa076b65640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0701a5f00 con 0x7fa0701023a0 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa070102cc0 0x7fa070100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa070102cc0 0x7fa070100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41880/0 (socket says 172.21.15.18:41880) 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa06ffff640 1 -- 172.21.15.18:0/2640779510 learned_addr learned my addr 172.21.15.18:0/2640779510 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa074901640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0701023a0 0x7fa0700ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.139+0000 7fa075102640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa07010b6e0 0x7fa070100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.140+0000 7fa06ffff640 1 -- 172.21.15.18:0/2640779510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0701023a0 msgr2=0x7fa0700ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.140+0000 7fa06ffff640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0701023a0 0x7fa0700ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.140+0000 7fa06ffff640 1 -- 172.21.15.18:0/2640779510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa07010b6e0 msgr2=0x7fa070100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.140+0000 7fa06ffff640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa07010b6e0 0x7fa070100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.140+0000 7fa06ffff640 1 -- 172.21.15.18:0/2640779510 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064009560 con 0x7fa070102cc0 2024-02-17T13:00:36.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.140+0000 7fa06ffff640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa070102cc0 0x7fa070100530 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00d770 tx=0x7fa05c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.140+0000 7fa06dffb640 1 -- 172.21.15.18:0/2640779510 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c009c30 con 0x7fa070102cc0 2024-02-17T13:00:36.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.140+0000 7fa06dffb640 1 -- 172.21.15.18:0/2640779510 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa05c00de80 con 0x7fa070102cc0 2024-02-17T13:00:36.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.140+0000 7fa06dffb640 1 -- 172.21.15.18:0/2640779510 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c004e80 con 0x7fa070102cc0 2024-02-17T13:00:36.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.140+0000 7fa076b65640 1 -- 172.21.15.18:0/2640779510 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0701a61e0 con 0x7fa070102cc0 2024-02-17T13:00:36.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.140+0000 7fa076b65640 1 -- 172.21.15.18:0/2640779510 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0701a66a0 con 0x7fa070102cc0 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.142+0000 7fa06dffb640 1 -- 172.21.15.18:0/2640779510 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa05c005020 con 0x7fa070102cc0 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa079811740 1 Processor -- start 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa079811740 1 -- start start 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa06dffb640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0600a2230 0x7fa0600a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa079811740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 0x5638a8b36690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa079811740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8b3f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa075102640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0600a2230 0x7fa0600a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa079811740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b41ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa079811740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a8aa5280 con 0x5638a8b3f7b0 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa079811740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a8aa27b0 con 0x5638a8b36bd0 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa079811740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a8a9df90 con 0x5638a8b362b0 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa06dffb640 1 -- 172.21.15.18:0/2640779510 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa05c0cda30 con 0x7fa070102cc0 2024-02-17T13:00:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa075102640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b41ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa075102640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b41ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41894/0 (socket says 172.21.15.18:41894) 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa075102640 1 -- 172.21.15.18:0/1564133281 learned_addr learned my addr 172.21.15.18:0/1564133281 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa074901640 1 --2- 172.21.15.18:0/1564133281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 0x5638a8b36690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.143+0000 7fa06ffff640 1 --2- 172.21.15.18:0/1564133281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8b3f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.144+0000 7fa075102640 1 -- 172.21.15.18:0/1564133281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 msgr2=0x5638a8b36690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.144+0000 7fa075102640 1 --2- 172.21.15.18:0/1564133281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 0x5638a8b36690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.144+0000 7fa075102640 1 -- 172.21.15.18:0/1564133281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 msgr2=0x5638a8b3f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.144+0000 7fa075102640 1 --2- 172.21.15.18:0/1564133281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8b3f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.144+0000 7fa075102640 1 -- 172.21.15.18:0/1564133281 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638a8a5c570 con 0x5638a8b3f7b0 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.144+0000 7fa075102640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0600a2230 0x7fa0600a46f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa068012c60 tx=0x7fa068009e60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.144+0000 7fa075102640 1 --2- 172.21.15.18:0/1564133281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b41ba0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa068010bd0 tx=0x7fa06801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4238f5d1ca428d95 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.144+0000 7fa05affd640 1 -- 172.21.15.18:0/1564133281 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06801fbd0 con 0x5638a8b3f7b0 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.144+0000 7fa05affd640 1 -- 172.21.15.18:0/1564133281 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa06801fd30 con 0x5638a8b3f7b0 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.144+0000 7fa05affd640 1 -- 172.21.15.18:0/1564133281 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068020610 con 0x5638a8b3f7b0 2024-02-17T13:00:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.145+0000 7fa079811740 1 -- 172.21.15.18:0/1564133281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 msgr2=0x5638a8b41ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.145+0000 7fa079811740 1 --2- 172.21.15.18:0/1564133281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b41ba0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa068010bd0 tx=0x7fa06801c000 comp rx=0 tx=0).stop 2024-02-17T13:00:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.145+0000 7fa079811740 1 -- 172.21.15.18:0/1564133281 shutdown_connections 2024-02-17T13:00:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.145+0000 7fa079811740 1 --2- 172.21.15.18:0/1564133281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b41ba0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.145+0000 7fa079811740 1 --2- 172.21.15.18:0/1564133281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8b3f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.145+0000 7fa079811740 1 --2- 172.21.15.18:0/1564133281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 0x5638a8b36690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.145+0000 7fa079811740 1 -- 172.21.15.18:0/1564133281 >> 172.21.15.18:0/1564133281 conn(0x5638a8b30990 msgr2=0x5638a8b30d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.145+0000 7fa079811740 1 -- 172.21.15.18:0/1564133281 shutdown_connections 2024-02-17T13:00:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.145+0000 7fa079811740 1 -- 172.21.15.18:0/1564133281 wait complete. 2024-02-17T13:00:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.145+0000 7fa079811740 1 -- start start 2024-02-17T13:00:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.145+0000 7fa079811740 1 Processor -- start 2024-02-17T13:00:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.145+0000 7fa079811740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 0x5638a8b3f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa079811740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8b3c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa079811740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b3cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa079811740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a8771430 con 0x5638a8b3f7b0 2024-02-17T13:00:36.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa079811740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a872de00 con 0x5638a8b36bd0 2024-02-17T13:00:36.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa079811740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a8a22fd0 con 0x5638a8b362b0 2024-02-17T13:00:36.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa079811740 10 client.0 _ll_register_callbacks cb 0x5638a8a78fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:36.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa075102640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b3cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa075102640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b3cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41908/0 (socket says 172.21.15.18:41908) 2024-02-17T13:00:36.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa075102640 1 -- 172.21.15.18:0/456012815 learned_addr learned my addr 172.21.15.18:0/456012815 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa074901640 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 0x5638a8b3f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa06ffff640 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8b3c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa075102640 1 -- 172.21.15.18:0/456012815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 msgr2=0x5638a8b3f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa075102640 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 0x5638a8b3f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa075102640 1 -- 172.21.15.18:0/456012815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 msgr2=0x5638a8b3c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa075102640 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8b3c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa075102640 1 -- 172.21.15.18:0/456012815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06801a040 con 0x5638a8b3f7b0 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa075102640 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b3cd00 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fa068016010 tx=0x7fa068014440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dd0e584e3fd76d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa04ffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa04ffff640 1 -- 172.21.15.18:0/456012815 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06801fbd0 con 0x5638a8b3f7b0 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa04ffff640 1 -- 172.21.15.18:0/456012815 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa068003b80 con 0x5638a8b3f7b0 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa079811740 1 -- 172.21.15.18:0/456012815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638a879d7e0 con 0x5638a8b3f7b0 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.146+0000 7fa04ffff640 1 -- 172.21.15.18:0/456012815 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0680147e0 con 0x5638a8b3f7b0 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.147+0000 7fa079811740 10 client.19317 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.147+0000 7fa079811740 1 -- 172.21.15.18:0/456012815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5638a8a9c610 con 0x5638a8b3f7b0 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.147+0000 7fa04ffff640 1 -- 172.21.15.18:0/456012815 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5638a879d7e0 con 0x5638a8b3f7b0 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.147+0000 7fa04ffff640 1 -- 172.21.15.18:0/456012815 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa0680055c0 con 0x5638a8b3f7b0 2024-02-17T13:00:36.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.147+0000 7fa079811740 10 client.19317 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.147+0000 7fa079811740 10 client.19317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.147+0000 7fa079811740 20 client.19317 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.147+0000 7fa079811740 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638a8b4f990 0x5638a8b51db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.147+0000 7fa079811740 4 client.19317 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.147+0000 7fa079811740 1 -- 172.21.15.18:0/456012815 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638a87da720 con 0x5638a8b4f990 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.147+0000 7fa074901640 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638a8b4f990 0x5638a8b51db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa074901640 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638a8b4f990 0x5638a8b51db0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b177982b83793 server_cookie=28ea067394f570c in_seq=0 out_seq=0 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa04ffff640 10 client.19317 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa04ffff640 1 -- 172.21.15.18:0/456012815 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5638a87da720 con 0x5638a8b4f990 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa04ffff640 10 client.19317 handle_command_reply: tid=0 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa079811740 1 client.19317 shutdown 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa079811740 1 -- 172.21.15.18:0/456012815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 msgr2=0x5638a8b3cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa079811740 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b3cd00 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fa068016010 tx=0x7fa068014440 comp rx=0 tx=0).stop 2024-02-17T13:00:36.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa079811740 1 -- 172.21.15.18:0/456012815 shutdown_connections 2024-02-17T13:00:36.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa079811740 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638a8b4f990 0x5638a8b51db0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa079811740 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b3cd00 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa04ffff640 0 client.19317 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:36.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa079811740 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8b3c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa079811740 1 --2- 172.21.15.18:0/456012815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 0x5638a8b3f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.148+0000 7fa079811740 1 -- 172.21.15.18:0/456012815 >> 172.21.15.18:0/456012815 conn(0x5638a8b30990 msgr2=0x5638a8b32730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.149+0000 7fa079811740 1 -- 172.21.15.18:0/456012815 shutdown_connections 2024-02-17T13:00:36.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.149+0000 7fa079811740 1 -- 172.21.15.18:0/456012815 wait complete. 2024-02-17T13:00:36.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.149+0000 7fa079811740 20 client.19317 trim_cache size 0 max 16384 2024-02-17T13:00:36.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.164+0000 7fa079811740 1 Processor -- start 2024-02-17T13:00:36.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.164+0000 7fa079811740 1 -- start start 2024-02-17T13:00:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.164+0000 7fa079811740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b362b0 0x5638a8b3b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.164+0000 7fa079811740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b36bd0 0x5638a8b3ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.164+0000 7fa079811740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b3f7b0 0x5638a8b3f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.164+0000 7fa079811740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a8b353a0 con 0x5638a8b36bd0 2024-02-17T13:00:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.164+0000 7fa079811740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a8b30240 con 0x5638a8b362b0 2024-02-17T13:00:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.164+0000 7fa079811740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a8b35f10 con 0x5638a8b3f7b0 2024-02-17T13:00:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.164+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b36bd0 0x5638a8b3ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.164+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b36bd0 0x5638a8b3ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41916/0 (socket says 172.21.15.18:41916) 2024-02-17T13:00:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.164+0000 7fa06ffff640 1 -- 172.21.15.18:0/281896533 learned_addr learned my addr 172.21.15.18:0/281896533 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa075102640 1 --2- 172.21.15.18:0/281896533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b3f7b0 0x5638a8b3f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa06ffff640 1 -- 172.21.15.18:0/281896533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b3f7b0 msgr2=0x5638a8b3f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa06ffff640 1 --2- 172.21.15.18:0/281896533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b3f7b0 0x5638a8b3f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa06ffff640 1 -- 172.21.15.18:0/281896533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b362b0 msgr2=0x5638a8b3b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa06ffff640 1 --2- 172.21.15.18:0/281896533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b362b0 0x5638a8b3b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa06ffff640 1 -- 172.21.15.18:0/281896533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638a8a38ac0 con 0x5638a8b36bd0 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa075102640 1 --2- 172.21.15.18:0/281896533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b3f7b0 0x5638a8b3f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa06ffff640 1 --2- 172.21.15.18:0/281896533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b36bd0 0x5638a8b3ecf0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa05c096950 tx=0x7fa05c09a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24494421fb0bc059 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa05a7fc640 1 -- 172.21.15.18:0/281896533 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0a2be0 con 0x5638a8b36bd0 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa05a7fc640 1 -- 172.21.15.18:0/281896533 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa05c0987f0 con 0x5638a8b36bd0 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa05a7fc640 1 -- 172.21.15.18:0/281896533 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c09e170 con 0x5638a8b36bd0 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa079811740 1 -- 172.21.15.18:0/281896533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b36bd0 msgr2=0x5638a8b3ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa079811740 1 --2- 172.21.15.18:0/281896533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b36bd0 0x5638a8b3ecf0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa05c096950 tx=0x7fa05c09a0f0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa079811740 1 -- 172.21.15.18:0/281896533 shutdown_connections 2024-02-17T13:00:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa079811740 1 --2- 172.21.15.18:0/281896533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b3f7b0 0x5638a8b3f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa079811740 1 --2- 172.21.15.18:0/281896533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b36bd0 0x5638a8b3ecf0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa079811740 1 --2- 172.21.15.18:0/281896533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b362b0 0x5638a8b3b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.165+0000 7fa079811740 1 -- 172.21.15.18:0/281896533 >> 172.21.15.18:0/281896533 conn(0x5638a8b30990 msgr2=0x5638a8b3ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa079811740 1 -- 172.21.15.18:0/281896533 shutdown_connections 2024-02-17T13:00:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa079811740 1 -- 172.21.15.18:0/281896533 wait complete. 2024-02-17T13:00:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa079811740 1 -- start start 2024-02-17T13:00:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa079811740 1 Processor -- start 2024-02-17T13:00:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa079811740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 0x5638a8ba02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa079811740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8ba07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa079811740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b43900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa079811740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a8b43ec0 con 0x5638a8b3f7b0 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa079811740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a8b44030 con 0x5638a8b36bd0 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa079811740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a8b441f0 con 0x5638a8b362b0 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa079811740 10 client.0 _ll_register_callbacks cb 0x7fa0680127b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa075102640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b43900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa075102640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b43900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41926/0 (socket says 172.21.15.18:41926) 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa075102640 1 -- 172.21.15.18:0/663675500 learned_addr learned my addr 172.21.15.18:0/663675500 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.166+0000 7fa074901640 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 0x5638a8ba02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa06ffff640 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8ba07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa075102640 1 -- 172.21.15.18:0/663675500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 msgr2=0x5638a8ba02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa075102640 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 0x5638a8ba02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa075102640 1 -- 172.21.15.18:0/663675500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 msgr2=0x5638a8ba07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa075102640 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8ba07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa075102640 1 -- 172.21.15.18:0/663675500 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0701a66a0 con 0x5638a8b3f7b0 2024-02-17T13:00:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa06ffff640 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8ba07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa075102640 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b43900 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa068005bc0 tx=0x7fa068009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28e1aa0d4b7e07a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa04ffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa04ffff640 1 -- 172.21.15.18:0/663675500 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06801fbd0 con 0x5638a8b3f7b0 2024-02-17T13:00:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa04ffff640 1 -- 172.21.15.18:0/663675500 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa068003d90 con 0x5638a8b3f7b0 2024-02-17T13:00:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa079811740 1 -- 172.21.15.18:0/663675500 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa06801fd30 con 0x5638a8b3f7b0 2024-02-17T13:00:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa04ffff640 1 -- 172.21.15.18:0/663675500 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa068012650 con 0x5638a8b3f7b0 2024-02-17T13:00:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa079811740 10 client.19323 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa079811740 1 -- 172.21.15.18:0/663675500 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5638a8a5e6a0 con 0x5638a8b3f7b0 2024-02-17T13:00:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.167+0000 7fa04ffff640 1 -- 172.21.15.18:0/663675500 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa068045020 con 0x5638a8b3f7b0 2024-02-17T13:00:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa04ffff640 1 -- 172.21.15.18:0/663675500 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa068005110 con 0x5638a8b3f7b0 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa079811740 10 client.19323 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa079811740 10 client.19323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa079811740 20 client.19323 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa079811740 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638a8ba7d20 0x5638a8baa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa079811740 4 client.19323 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa079811740 1 -- 172.21.15.18:0/663675500 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5638a8829890 con 0x5638a8ba7d20 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa074901640 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638a8ba7d20 0x5638a8baa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa04ffff640 10 client.19323 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa074901640 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638a8ba7d20 0x5638a8baa110 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=836227b9b4b5ba6e server_cookie=3a96a565062f3d97 in_seq=0 out_seq=0 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa04ffff640 1 -- 172.21.15.18:0/663675500 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5638a8829890 con 0x5638a8ba7d20 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa04ffff640 10 client.19323 handle_command_reply: tid=0 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.168+0000 7fa079811740 1 client.19323 shutdown 2024-02-17T13:00:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa079811740 1 -- 172.21.15.18:0/663675500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 msgr2=0x5638a8b43900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa079811740 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b43900 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa068005bc0 tx=0x7fa068009990 comp rx=0 tx=0).stop 2024-02-17T13:00:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa079811740 1 -- 172.21.15.18:0/663675500 shutdown_connections 2024-02-17T13:00:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa079811740 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5638a8ba7d20 0x5638a8baa110 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa079811740 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5638a8b3f7b0 0x5638a8b43900 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa04ffff640 0 client.19323 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa079811740 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638a8b36bd0 0x5638a8ba07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa079811740 1 --2- 172.21.15.18:0/663675500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638a8b362b0 0x5638a8ba02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa079811740 1 -- 172.21.15.18:0/663675500 >> 172.21.15.18:0/663675500 conn(0x5638a8b30990 msgr2=0x5638a8b31d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa079811740 1 -- 172.21.15.18:0/663675500 shutdown_connections 2024-02-17T13:00:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa079811740 1 -- 172.21.15.18:0/663675500 wait complete. 2024-02-17T13:00:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa079811740 20 client.19323 trim_cache size 0 max 16384 2024-02-17T13:00:36.167 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:00:36.167 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T13:00:36.167 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T13:00:36.167 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:00:36.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa076b65640 1 -- 172.21.15.18:0/2640779510 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0600a2230 msgr2=0x7fa0600a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa076b65640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0600a2230 0x7fa0600a46f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa068012c60 tx=0x7fa068009e60 comp rx=0 tx=0).stop 2024-02-17T13:00:36.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa076b65640 1 -- 172.21.15.18:0/2640779510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa070102cc0 msgr2=0x7fa070100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa076b65640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa070102cc0 0x7fa070100530 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00d770 tx=0x7fa05c00dc30 comp rx=0 tx=0).stop 2024-02-17T13:00:36.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa076b65640 1 -- 172.21.15.18:0/2640779510 shutdown_connections 2024-02-17T13:00:36.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa076b65640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa0600a2230 0x7fa0600a46f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa076b65640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa07010b6e0 0x7fa070100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa076b65640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa070102cc0 0x7fa070100530 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa076b65640 1 --2- 172.21.15.18:0/2640779510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0701023a0 0x7fa0700ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa076b65640 1 -- 172.21.15.18:0/2640779510 >> 172.21.15.18:0/2640779510 conn(0x7fa070077230 msgr2=0x7fa0700fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa076b65640 1 -- 172.21.15.18:0/2640779510 shutdown_connections 2024-02-17T13:00:36.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.169+0000 7fa076b65640 1 -- 172.21.15.18:0/2640779510 wait complete. 2024-02-17T13:00:36.170 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T13:00:36.170 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T13:00:36.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.286+0000 7f379629d640 1 -- 172.21.15.18:0/2577255853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3790100ca0 msgr2=0x7f3790104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.286+0000 7f379629d640 1 --2- 172.21.15.18:0/2577255853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3790100ca0 0x7f3790104ce0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f3784009930 tx=0x7f378402bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:36.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.286+0000 7f379629d640 1 -- 172.21.15.18:0/2577255853 shutdown_connections 2024-02-17T13:00:36.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.286+0000 7f379629d640 1 --2- 172.21.15.18:0/2577255853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37901053c0 0x7f379010cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.286+0000 7f379629d640 1 --2- 172.21.15.18:0/2577255853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3790100ca0 0x7f3790104ce0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.286+0000 7f379629d640 1 --2- 172.21.15.18:0/2577255853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37901002f0 0x7f37901006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.286+0000 7f379629d640 1 -- 172.21.15.18:0/2577255853 >> 172.21.15.18:0/2577255853 conn(0x7f3790077180 msgr2=0x7f3790077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.286+0000 7f379629d640 1 -- 172.21.15.18:0/2577255853 shutdown_connections 2024-02-17T13:00:36.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.286+0000 7f379629d640 1 -- 172.21.15.18:0/2577255853 wait complete. 2024-02-17T13:00:36.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.287+0000 7f379629d640 1 Processor -- start 2024-02-17T13:00:36.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.287+0000 7f379629d640 1 -- start start 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.287+0000 7f379629d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37901002f0 0x7f379019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.287+0000 7f379629d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37901053c0 0x7f379019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.287+0000 7f379629d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3790197680 0x7f3790197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.287+0000 7f379629d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3790198150 con 0x7f3790197680 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.287+0000 7f379629d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37901982c0 con 0x7f37901053c0 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.287+0000 7f379629d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3790198480 con 0x7f37901002f0 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f379483a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3790197680 0x7f3790197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f379483a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3790197680 0x7f3790197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41942/0 (socket says 172.21.15.18:41942) 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f379483a640 1 -- 172.21.15.18:0/3594650441 learned_addr learned my addr 172.21.15.18:0/3594650441 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f378f7fe640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37901053c0 0x7f379019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f378ffff640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37901002f0 0x7f379019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f379483a640 1 -- 172.21.15.18:0/3594650441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37901002f0 msgr2=0x7f379019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f379483a640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37901002f0 0x7f379019d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f379483a640 1 -- 172.21.15.18:0/3594650441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37901053c0 msgr2=0x7f379019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f379483a640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37901053c0 0x7f379019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f379483a640 1 -- 172.21.15.18:0/3594650441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3784009560 con 0x7f3790197680 2024-02-17T13:00:36.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f379483a640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3790197680 0x7f3790197b00 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f3788009fd0 tx=0x7f378800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f378d7fa640 1 -- 172.21.15.18:0/3594650441 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378800bea0 con 0x7f3790197680 2024-02-17T13:00:36.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f378d7fa640 1 -- 172.21.15.18:0/3594650441 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3788009940 con 0x7f3790197680 2024-02-17T13:00:36.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f379629d640 1 -- 172.21.15.18:0/3594650441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37901a4420 con 0x7f3790197680 2024-02-17T13:00:36.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f378d7fa640 1 -- 172.21.15.18:0/3594650441 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37880106c0 con 0x7f3790197680 2024-02-17T13:00:36.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.288+0000 7f379629d640 1 -- 172.21.15.18:0/3594650441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37901a48b0 con 0x7f3790197680 2024-02-17T13:00:36.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.289+0000 7f379629d640 1 -- 172.21.15.18:0/3594650441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37901017c0 con 0x7f3790197680 2024-02-17T13:00:36.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.290+0000 7f378d7fa640 1 -- 172.21.15.18:0/3594650441 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3788004240 con 0x7f3790197680 2024-02-17T13:00:36.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.291+0000 7f378d7fa640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37780a2230 0x7f37780a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.291+0000 7f378d7fa640 1 -- 172.21.15.18:0/3594650441 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f37880cdd60 con 0x7f3790197680 2024-02-17T13:00:36.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.291+0000 7f378ffff640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37780a2230 0x7f37780a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.291+0000 7f378ffff640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37780a2230 0x7f37780a46f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3780005fd0 tx=0x7f3780005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.296+0000 7f378d7fa640 1 -- 172.21.15.18:0/3594650441 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3788096a10 con 0x7f3790197680 2024-02-17T13:00:36.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 538122 4.98 MB/sec execute 471 sec latency 1392.304 ms 2024-02-17T13:00:36.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.513+0000 7f379629d640 1 -- 172.21.15.18:0/3594650441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3790109600 con 0x7f3790197680 2024-02-17T13:00:36.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.514+0000 7f378d7fa640 1 -- 172.21.15.18:0/3594650441 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+11396 (secure 0 0 0) 0x7f378809a0d0 con 0x7f3790197680 2024-02-17T13:00:36.504 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:00:36.504 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T12:59:29.946156+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T13:00:36.504 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 48 2024-02-17T13:00:36.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.516+0000 7f379629d640 1 -- 172.21.15.18:0/3594650441 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37780a2230 msgr2=0x7f37780a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.516+0000 7f379629d640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37780a2230 0x7f37780a46f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3780005fd0 tx=0x7f3780005850 comp rx=0 tx=0).stop 2024-02-17T13:00:36.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.516+0000 7f379629d640 1 -- 172.21.15.18:0/3594650441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3790197680 msgr2=0x7f3790197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.516+0000 7f379629d640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3790197680 0x7f3790197b00 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f3788009fd0 tx=0x7f378800b260 comp rx=0 tx=0).stop 2024-02-17T13:00:36.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.516+0000 7f379629d640 1 -- 172.21.15.18:0/3594650441 shutdown_connections 2024-02-17T13:00:36.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.516+0000 7f379629d640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37780a2230 0x7f37780a46f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.516+0000 7f379629d640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3790197680 0x7f3790197b00 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.516+0000 7f379629d640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37901053c0 0x7f379019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.516+0000 7f379629d640 1 --2- 172.21.15.18:0/3594650441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37901002f0 0x7f379019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.516+0000 7f379629d640 1 -- 172.21.15.18:0/3594650441 >> 172.21.15.18:0/3594650441 conn(0x7f3790077180 msgr2=0x7f37900ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.516+0000 7f379629d640 1 -- 172.21.15.18:0/3594650441 shutdown_connections 2024-02-17T13:00:36.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.516+0000 7f379629d640 1 -- 172.21.15.18:0/3594650441 wait complete. 2024-02-17T13:00:36.517 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:00:36.517 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T13:00:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.628+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3966551127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb58105020 msgr2=0x7fcb58105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.628+0000 7fcb5df4b640 1 --2- 172.21.15.18:0/3966551127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb58105020 0x7fcb58105400 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009900 tx=0x7fcb4002bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.628+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3966551127 shutdown_connections 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.628+0000 7fcb5df4b640 1 --2- 172.21.15.18:0/3966551127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb58100c90 0x7fcb5810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.628+0000 7fcb5df4b640 1 --2- 172.21.15.18:0/3966551127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb581002d0 0x7fcb58100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.628+0000 7fcb5df4b640 1 --2- 172.21.15.18:0/3966551127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb58105020 0x7fcb58105400 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.628+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3966551127 >> 172.21.15.18:0/3966551127 conn(0x7fcb58076ed0 msgr2=0x7fcb580772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.628+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3966551127 shutdown_connections 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.628+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3966551127 wait complete. 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.629+0000 7fcb5df4b640 1 Processor -- start 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.629+0000 7fcb5df4b640 1 -- start start 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.629+0000 7fcb5df4b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb581002d0 0x7fcb5819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.629+0000 7fcb5df4b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb58100c90 0x7fcb5819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.629+0000 7fcb5df4b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb58105020 0x7fcb58197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.629+0000 7fcb5df4b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb58197ca0 con 0x7fcb581002d0 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.629+0000 7fcb5df4b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb58197e10 con 0x7fcb58105020 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.629+0000 7fcb5df4b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb58197fd0 con 0x7fcb58100c90 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.629+0000 7fcb577fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb581002d0 0x7fcb5819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb577fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb581002d0 0x7fcb5819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41958/0 (socket says 172.21.15.18:41958) 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb577fe640 1 -- 172.21.15.18:0/3787847506 learned_addr learned my addr 172.21.15.18:0/3787847506 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb57fff640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb58105020 0x7fcb58197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb56ffd640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb58100c90 0x7fcb5819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb577fe640 1 -- 172.21.15.18:0/3787847506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb58100c90 msgr2=0x7fcb5819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb577fe640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb58100c90 0x7fcb5819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb577fe640 1 -- 172.21.15.18:0/3787847506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb58105020 msgr2=0x7fcb58197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb577fe640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb58105020 0x7fcb58197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb577fe640 1 -- 172.21.15.18:0/3787847506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb40009560 con 0x7fcb581002d0 2024-02-17T13:00:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb577fe640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb581002d0 0x7fcb5819d450 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fcb40005940 tx=0x7fcb40004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb54ff9640 1 -- 172.21.15.18:0/3787847506 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40004440 con 0x7fcb581002d0 2024-02-17T13:00:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb54ff9640 1 -- 172.21.15.18:0/3787847506 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb400052d0 con 0x7fcb581002d0 2024-02-17T13:00:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3787847506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb58198250 con 0x7fcb581002d0 2024-02-17T13:00:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb54ff9640 1 -- 172.21.15.18:0/3787847506 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40003da0 con 0x7fcb581002d0 2024-02-17T13:00:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.630+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3787847506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb581a44f0 con 0x7fcb581002d0 2024-02-17T13:00:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.633+0000 7fcb54ff9640 1 -- 172.21.15.18:0/3787847506 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcb40005440 con 0x7fcb581002d0 2024-02-17T13:00:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.633+0000 7fcb60bf1740 1 Processor -- start 2024-02-17T13:00:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.633+0000 7fcb54ff9640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb440a22d0 0x7fcb440a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.633+0000 7f869e857640 1 -- 172.21.15.18:0/1519203022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8698106440 msgr2=0x7f869810e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.633+0000 7f869e857640 1 --2- 172.21.15.18:0/1519203022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8698106440 0x7f869810e010 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8690009cd0 tx=0x7f869002c0f0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.633+0000 7fcb60bf1740 1 -- start start 2024-02-17T13:00:36.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.633+0000 7f869e857640 1 -- 172.21.15.18:0/1519203022 shutdown_connections 2024-02-17T13:00:36.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.633+0000 7f869e857640 1 --2- 172.21.15.18:0/1519203022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8698106440 0x7f869810e010 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.633+0000 7f869e857640 1 --2- 172.21.15.18:0/1519203022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8698101d20 0x7f8698105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.633+0000 7f869e857640 1 --2- 172.21.15.18:0/1519203022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698101370 0x7f8698101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.633+0000 7f869e857640 1 -- 172.21.15.18:0/1519203022 >> 172.21.15.18:0/1519203022 conn(0x7f86980fcc10 msgr2=0x7f86980ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.633+0000 7fcb57fff640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb440a22d0 0x7fcb440a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.633+0000 7f869e857640 1 -- 172.21.15.18:0/1519203022 shutdown_connections 2024-02-17T13:00:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.633+0000 7fcb54ff9640 1 -- 172.21.15.18:0/3787847506 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fcb400f0130 con 0x7fcb581002d0 2024-02-17T13:00:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.633+0000 7fcb60bf1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d27d0 0x5621ad2d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.633+0000 7f869e857640 1 -- 172.21.15.18:0/1519203022 wait complete. 2024-02-17T13:00:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb60bf1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d30f0 0x5621ad2d3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb60bf1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d3b90 0x5621ad2e6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb60bf1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ad2419f0 con 0x5621ad2d27d0 2024-02-17T13:00:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb60bf1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ad23ef20 con 0x5621ad2d30f0 2024-02-17T13:00:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb60bf1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ad23a700 con 0x5621ad2d3b90 2024-02-17T13:00:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb577fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d27d0 0x5621ad2d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb577fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d27d0 0x5621ad2d2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41988/0 (socket says 172.21.15.18:41988) 2024-02-17T13:00:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb577fe640 1 -- 172.21.15.18:0/1087398481 learned_addr learned my addr 172.21.15.18:0/1087398481 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb56ffd640 1 --2- 172.21.15.18:0/1087398481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d30f0 0x5621ad2d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb577fe640 1 -- 172.21.15.18:0/1087398481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d3b90 msgr2=0x5621ad2e6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb577fe640 1 --2- 172.21.15.18:0/1087398481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d3b90 0x5621ad2e6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb577fe640 1 -- 172.21.15.18:0/1087398481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d30f0 msgr2=0x5621ad2d3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb577fe640 1 --2- 172.21.15.18:0/1087398481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d30f0 0x5621ad2d3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb57fff640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb440a22d0 0x7fcb440a4790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcb50007df0 tx=0x7fcb50007470 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb577fe640 1 -- 172.21.15.18:0/1087398481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621ad1d07b0 con 0x5621ad2d27d0 2024-02-17T13:00:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb56ffd640 1 --2- 172.21.15.18:0/1087398481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d30f0 0x5621ad2d3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb577fe640 1 --2- 172.21.15.18:0/1087398481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d27d0 0x5621ad2d2bb0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fcb400b90b0 tx=0x7fcb400bc5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bba8fc71b969f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.634+0000 7f869e857640 1 Processor -- start 2024-02-17T13:00:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb4dffb640 1 -- 172.21.15.18:0/1087398481 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb400c0da0 con 0x5621ad2d27d0 2024-02-17T13:00:36.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.634+0000 7f869e857640 1 -- start start 2024-02-17T13:00:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb4dffb640 1 -- 172.21.15.18:0/1087398481 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb400c53f0 con 0x5621ad2d27d0 2024-02-17T13:00:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.634+0000 7fcb4dffb640 1 -- 172.21.15.18:0/1087398481 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb400c4f60 con 0x5621ad2d27d0 2024-02-17T13:00:36.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f869e857640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8698101370 0x7f869819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f869e857640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698106440 0x7f869819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f869e857640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8698199820 0x7f8698199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f869e857640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869819a2d0 con 0x7f8698101370 2024-02-17T13:00:36.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f869e857640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869819a440 con 0x7f8698199820 2024-02-17T13:00:36.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f869e857640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869819a5b0 con 0x7f8698106440 2024-02-17T13:00:36.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f8697fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8698101370 0x7f869819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f8697fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8698101370 0x7f869819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41996/0 (socket says 172.21.15.18:41996) 2024-02-17T13:00:36.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f8697fff640 1 -- 172.21.15.18:0/3401395750 learned_addr learned my addr 172.21.15.18:0/3401395750 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f86977fe640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698106440 0x7f869819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f869cdf4640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8698199820 0x7f8698199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f8697fff640 1 -- 172.21.15.18:0/3401395750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698106440 msgr2=0x7f869819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f8697fff640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698106440 0x7f869819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.635+0000 7fcb60bf1740 1 -- 172.21.15.18:0/1087398481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d27d0 msgr2=0x5621ad2d2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.635+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/1087398481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d27d0 0x5621ad2d2bb0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fcb400b90b0 tx=0x7fcb400bc5b0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f8697fff640 1 -- 172.21.15.18:0/3401395750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8698199820 msgr2=0x7f8698199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f8697fff640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8698199820 0x7f8698199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f8697fff640 1 -- 172.21.15.18:0/3401395750 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8690009900 con 0x7f8698101370 2024-02-17T13:00:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.635+0000 7fcb60bf1740 1 -- 172.21.15.18:0/1087398481 shutdown_connections 2024-02-17T13:00:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.635+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/1087398481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d3b90 0x5621ad2e6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.635+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/1087398481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d30f0 0x5621ad2d3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.635+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/1087398481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d27d0 0x5621ad2d2bb0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.635+0000 7fcb60bf1740 1 -- 172.21.15.18:0/1087398481 >> 172.21.15.18:0/1087398481 conn(0x5621ad29a550 msgr2=0x5621ad2cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.635+0000 7fcb60bf1740 1 -- 172.21.15.18:0/1087398481 shutdown_connections 2024-02-17T13:00:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.635+0000 7fcb60bf1740 1 -- 172.21.15.18:0/1087398481 wait complete. 2024-02-17T13:00:36.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.635+0000 7f8697fff640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8698101370 0x7f869819f6b0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f868800d950 tx=0x7f868800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.636+0000 7f86957fa640 1 -- 172.21.15.18:0/3401395750 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688009c30 con 0x7f8698101370 2024-02-17T13:00:36.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.636+0000 7f869e857640 1 -- 172.21.15.18:0/3401395750 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86981a6560 con 0x7f8698101370 2024-02-17T13:00:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.636+0000 7fcb60bf1740 1 -- start start 2024-02-17T13:00:36.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.636+0000 7f86957fa640 1 -- 172.21.15.18:0/3401395750 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8688009d90 con 0x7f8698101370 2024-02-17T13:00:36.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.636+0000 7f869e857640 1 -- 172.21.15.18:0/3401395750 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86981a6a70 con 0x7f8698101370 2024-02-17T13:00:36.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.636+0000 7f86957fa640 1 -- 172.21.15.18:0/3401395750 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868800f7e0 con 0x7f8698101370 2024-02-17T13:00:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.636+0000 7fcb60bf1740 1 Processor -- start 2024-02-17T13:00:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.636+0000 7fcb60bf1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 0x5621ad2e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.636+0000 7fcb60bf1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d30f0 0x5621ad2e48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.636+0000 7fcb60bf1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad2e4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.636+0000 7fcb60bf1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ad1dfbf0 con 0x5621ad2d3b90 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.636+0000 7fcb60bf1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621acf0e250 con 0x5621ad2d27d0 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb60bf1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ad1c0200 con 0x5621ad2d30f0 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad2e4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad2e4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41998/0 (socket says 172.21.15.18:41998) 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb57fff640 1 -- 172.21.15.18:0/2697915679 learned_addr learned my addr 172.21.15.18:0/2697915679 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb60bf1740 10 client.0 _ll_register_callbacks cb 0x5621ad22ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb577fe640 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 0x5621ad2e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb56ffd640 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d30f0 0x5621ad2e48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb57fff640 1 -- 172.21.15.18:0/2697915679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d30f0 msgr2=0x5621ad2e48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb57fff640 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d30f0 0x5621ad2e48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb57fff640 1 -- 172.21.15.18:0/2697915679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 msgr2=0x5621ad2e43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb57fff640 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 0x5621ad2e43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb57fff640 1 -- 172.21.15.18:0/2697915679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb581a44f0 con 0x5621ad2d3b90 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb577fe640 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 0x5621ad2e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb57fff640 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad2e4e30 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fcb50007b80 tx=0x7fcb50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e2f265f3a6fde9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb36ffd640 1 -- 172.21.15.18:0/2697915679 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5001bee0 con 0x5621ad2d3b90 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb36ffd640 1 -- 172.21.15.18:0/2697915679 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fcb50021070 con 0x5621ad2d3b90 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb60bf1740 1 -- 172.21.15.18:0/2697915679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621acf3a600 con 0x5621ad2d3b90 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.637+0000 7fcb36ffd640 1 -- 172.21.15.18:0/2697915679 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb50010e90 con 0x5621ad2d3b90 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.638+0000 7fcb60bf1740 10 client.19347 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.638+0000 7fcb60bf1740 1 -- 172.21.15.18:0/2697915679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5621ad238d80 con 0x5621ad2d3b90 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.638+0000 7fcb36ffd640 1 -- 172.21.15.18:0/2697915679 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5621acf3a600 con 0x5621ad2d3b90 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.638+0000 7fcb36ffd640 1 -- 172.21.15.18:0/2697915679 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fcb5002f940 con 0x5621ad2d3b90 2024-02-17T13:00:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.638+0000 7fcb60bf1740 10 client.19347 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.638+0000 7fcb60bf1740 10 client.19347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.638+0000 7fcb60bf1740 20 client.19347 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.638+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5621ad2f4c70 0x5621ad2f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.638+0000 7fcb60bf1740 4 client.19347 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.638+0000 7fcb60bf1740 1 -- 172.21.15.18:0/2697915679 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621acf8b1b0 con 0x5621ad2f4c70 2024-02-17T13:00:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.638+0000 7fcb577fe640 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5621ad2f4c70 0x5621ad2f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.638+0000 7f86a141f740 1 Processor -- start 2024-02-17T13:00:36.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.638+0000 7f86957fa640 1 -- 172.21.15.18:0/3401395750 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8688010ad0 con 0x7f8698101370 2024-02-17T13:00:36.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f86a141f740 1 -- start start 2024-02-17T13:00:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.639+0000 7fcb36ffd640 10 client.19347 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.639+0000 7fcb577fe640 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5621ad2f4c70 0x5621ad2f7090 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e993b00217c5ea0 server_cookie=5078b15205a190d8 in_seq=0 out_seq=0 2024-02-17T13:00:36.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f86a141f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1250 0x5652053e1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f86a141f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f86a141f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053ea750 0x5652053ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f86a141f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520539c960 con 0x5652053e1250 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f86a141f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565205350310 con 0x5652053e1b70 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f86a141f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520534d840 con 0x5652053ea750 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f869cdf4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1250 0x5652053e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f869cdf4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1250 0x5652053e1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42014/0 (socket says 172.21.15.18:42014) 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f869cdf4640 1 -- 172.21.15.18:0/3567552813 learned_addr learned my addr 172.21.15.18:0/3567552813 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f8697fff640 1 --2- 172.21.15.18:0/3567552813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f86957fa640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f86800a2230 0x7f86800a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f869cdf4640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f86800a2230 0x7f86800a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f86977fe640 1 --2- 172.21.15.18:0/3567552813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053ea750 0x5652053ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f869cdf4640 1 -- 172.21.15.18:0/3567552813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053ea750 msgr2=0x5652053ecb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f869cdf4640 1 --2- 172.21.15.18:0/3567552813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053ea750 0x5652053ecb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f869cdf4640 1 -- 172.21.15.18:0/3567552813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 msgr2=0x5652053ea210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f86957fa640 1 -- 172.21.15.18:0/3401395750 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f86880cdfb0 con 0x7f8698101370 2024-02-17T13:00:36.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f869cdf4640 1 --2- 172.21.15.18:0/3567552813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053ea210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.639+0000 7f869cdf4640 1 -- 172.21.15.18:0/3567552813 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56520530b060 con 0x5652053e1250 2024-02-17T13:00:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.639+0000 7fcb36ffd640 1 -- 172.21.15.18:0/2697915679 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5621acf8b1b0 con 0x5621ad2f4c70 2024-02-17T13:00:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.639+0000 7fcb36ffd640 10 client.19347 handle_command_reply: tid=0 2024-02-17T13:00:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb60bf1740 1 client.19347 shutdown 2024-02-17T13:00:36.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.640+0000 7f869cdf4640 1 --2- 172.21.15.18:0/3567552813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1250 0x5652053e1630 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f8690009430 tx=0x7f8690032610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac43fdda9696d48 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb60bf1740 1 -- 172.21.15.18:0/2697915679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 msgr2=0x5621ad2e4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad2e4e30 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fcb50007b80 tx=0x7fcb50004a40 comp rx=0 tx=0).stop 2024-02-17T13:00:36.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.640+0000 7f869cdf4640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f86800a2230 0x7f86800a46f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8690004af0 tx=0x7f869004b000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb60bf1740 1 -- 172.21.15.18:0/2697915679 shutdown_connections 2024-02-17T13:00:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5621ad2f4c70 0x5621ad2f7090 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad2e4e30 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d30f0 0x5621ad2e48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb36ffd640 0 client.19347 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/2697915679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 0x5621ad2e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb60bf1740 1 -- 172.21.15.18:0/2697915679 >> 172.21.15.18:0/2697915679 conn(0x5621ad29a550 msgr2=0x5621ad2cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.640+0000 7f867e7fc640 1 -- 172.21.15.18:0/3567552813 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86900326b0 con 0x5652053e1250 2024-02-17T13:00:36.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.640+0000 7f867e7fc640 1 -- 172.21.15.18:0/3567552813 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8690032810 con 0x5652053e1250 2024-02-17T13:00:36.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.640+0000 7f867e7fc640 1 -- 172.21.15.18:0/3567552813 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869004e430 con 0x5652053e1250 2024-02-17T13:00:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb60bf1740 1 -- 172.21.15.18:0/2697915679 shutdown_connections 2024-02-17T13:00:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb60bf1740 1 -- 172.21.15.18:0/2697915679 wait complete. 2024-02-17T13:00:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.640+0000 7fcb60bf1740 20 client.19347 trim_cache size 0 max 16384 2024-02-17T13:00:36.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.641+0000 7f86a141f740 1 -- 172.21.15.18:0/3567552813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1250 msgr2=0x5652053e1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.641+0000 7f86a141f740 1 --2- 172.21.15.18:0/3567552813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1250 0x5652053e1630 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f8690009430 tx=0x7f8690032610 comp rx=0 tx=0).stop 2024-02-17T13:00:36.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.641+0000 7f86a141f740 1 -- 172.21.15.18:0/3567552813 shutdown_connections 2024-02-17T13:00:36.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.641+0000 7f86a141f740 1 --2- 172.21.15.18:0/3567552813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053ea750 0x5652053ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.641+0000 7f86a141f740 1 --2- 172.21.15.18:0/3567552813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053ea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.641+0000 7f86a141f740 1 --2- 172.21.15.18:0/3567552813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1250 0x5652053e1630 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.641+0000 7f86a141f740 1 -- 172.21.15.18:0/3567552813 >> 172.21.15.18:0/3567552813 conn(0x5652053db980 msgr2=0x5652053dbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.641+0000 7f86a141f740 1 -- 172.21.15.18:0/3567552813 shutdown_connections 2024-02-17T13:00:36.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.641+0000 7f86a141f740 1 -- 172.21.15.18:0/3567552813 wait complete. 2024-02-17T13:00:36.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.641+0000 7f86a141f740 1 -- start start 2024-02-17T13:00:36.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.641+0000 7f86a141f740 1 Processor -- start 2024-02-17T13:00:36.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86a141f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86a141f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86a141f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 0x5652053e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86a141f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x565205349020 con 0x5652053ea750 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86a141f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56520501c430 con 0x5652053e1b70 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86a141f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565204fd8e00 con 0x5652053e1250 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f869cdf4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 0x5652053e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f869cdf4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 0x5652053e8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42020/0 (socket says 172.21.15.18:42020) 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f869cdf4640 1 -- 172.21.15.18:0/292460687 learned_addr learned my addr 172.21.15.18:0/292460687 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f8697fff640 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86a141f740 10 client.0 _ll_register_callbacks cb 0x5652052f0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86977fe640 1 -- 172.21.15.18:0/292460687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 msgr2=0x5652053e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86977fe640 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86977fe640 1 -- 172.21.15.18:0/292460687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 msgr2=0x5652053e8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86977fe640 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 0x5652053e8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86977fe640 1 -- 172.21.15.18:0/292460687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869810e820 con 0x5652053e1b70 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f869cdf4640 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 0x5652053e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86977fe640 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053e7cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8698107b10 tx=0x7f868c012aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae21d003a33aab51 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.642+0000 7f86777fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86777fe640 1 -- 172.21.15.18:0/292460687 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c017070 con 0x5652053e1b70 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86777fe640 1 -- 172.21.15.18:0/292460687 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f868c004050 con 0x5652053e1b70 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86a141f740 1 -- 172.21.15.18:0/292460687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652050487e0 con 0x5652053e1b70 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86777fe640 1 -- 172.21.15.18:0/292460687 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f868c004210 con 0x5652053e1b70 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86a141f740 10 client.28379 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86a141f740 1 -- 172.21.15.18:0/292460687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5652053476a0 con 0x5652053e1b70 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86777fe640 1 -- 172.21.15.18:0/292460687 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f868c004370 con 0x5652053e1b70 2024-02-17T13:00:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86777fe640 1 -- 172.21.15.18:0/292460687 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f868c00f640 con 0x5652053e1b70 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86a141f740 10 client.28379 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86a141f740 10 client.28379 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86a141f740 20 client.28379 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86a141f740 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5652053faa30 0x5652053fced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86a141f740 4 client.28379 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.643+0000 7f86a141f740 1 -- 172.21.15.18:0/292460687 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652050487e0 con 0x5652053faa30 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.644+0000 7f8697fff640 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5652053faa30 0x5652053fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.644+0000 7f86777fe640 10 client.28379 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.644+0000 7f8697fff640 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5652053faa30 0x5652053fced0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2a9cb9af0452e898 server_cookie=409f0ced11984c64 in_seq=0 out_seq=0 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86777fe640 1 -- 172.21.15.18:0/292460687 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652050487e0 con 0x5652053faa30 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86777fe640 10 client.28379 handle_command_reply: tid=0 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86a141f740 1 client.28379 shutdown 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86a141f740 1 -- 172.21.15.18:0/292460687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 msgr2=0x5652053e7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86a141f740 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053e7cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8698107b10 tx=0x7f868c012aa0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86a141f740 1 -- 172.21.15.18:0/292460687 shutdown_connections 2024-02-17T13:00:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86a141f740 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 0x5652053e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86a141f740 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5652053faa30 0x5652053fced0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86a141f740 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053e7cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86a141f740 1 --2- 172.21.15.18:0/292460687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86777fe640 0 client.28379 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:00:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86a141f740 1 -- 172.21.15.18:0/292460687 >> 172.21.15.18:0/292460687 conn(0x5652053db980 msgr2=0x5652053dd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86a141f740 1 -- 172.21.15.18:0/292460687 shutdown_connections 2024-02-17T13:00:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86a141f740 1 -- 172.21.15.18:0/292460687 wait complete. 2024-02-17T13:00:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.645+0000 7f86a141f740 20 client.28379 trim_cache size 0 max 16384 2024-02-17T13:00:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.657+0000 7fcb60bf1740 1 Processor -- start 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.657+0000 7fcb60bf1740 1 -- start start 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb60bf1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d27d0 0x5621ad2e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb60bf1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d30f0 0x5621ad2e1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb60bf1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad2e4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb60bf1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621acf8b1b0 con 0x5621ad2d3b90 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb60bf1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621acf3a600 con 0x5621ad2d30f0 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb60bf1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ad2e23f0 con 0x5621ad2d27d0 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad2e4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb577fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d27d0 0x5621ad2e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad2e4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42030/0 (socket says 172.21.15.18:42030) 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb57fff640 1 -- 172.21.15.18:0/1598016361 learned_addr learned my addr 172.21.15.18:0/1598016361 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb56ffd640 1 --2- 172.21.15.18:0/1598016361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d30f0 0x5621ad2e1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb57fff640 1 -- 172.21.15.18:0/1598016361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d27d0 msgr2=0x5621ad2e6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb57fff640 1 --2- 172.21.15.18:0/1598016361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d27d0 0x5621ad2e6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb57fff640 1 -- 172.21.15.18:0/1598016361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d30f0 msgr2=0x5621ad2e1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb57fff640 1 --2- 172.21.15.18:0/1598016361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d30f0 0x5621ad2e1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb57fff640 1 -- 172.21.15.18:0/1598016361 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621ad1d57b0 con 0x5621ad2d3b90 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb57fff640 1 --2- 172.21.15.18:0/1598016361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad2e4c50 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fcb5002fd50 tx=0x7fcb50010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df32571e47253fc0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.658+0000 7fcb4d7fa640 1 -- 172.21.15.18:0/1598016361 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5001bee0 con 0x5621ad2d3b90 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.659+0000 7fcb4d7fa640 1 -- 172.21.15.18:0/1598016361 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb50010e90 con 0x5621ad2d3b90 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.659+0000 7fcb4d7fa640 1 -- 172.21.15.18:0/1598016361 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5001d930 con 0x5621ad2d3b90 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.659+0000 7fcb60bf1740 1 -- 172.21.15.18:0/1598016361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 msgr2=0x5621ad2e4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.659+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/1598016361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad2e4c50 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fcb5002fd50 tx=0x7fcb50010b30 comp rx=0 tx=0).stop 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.659+0000 7fcb60bf1740 1 -- 172.21.15.18:0/1598016361 shutdown_connections 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.659+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/1598016361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad2e4c50 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.659+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/1598016361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d30f0 0x5621ad2e1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.659+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/1598016361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d27d0 0x5621ad2e6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.659+0000 7fcb60bf1740 1 -- 172.21.15.18:0/1598016361 >> 172.21.15.18:0/1598016361 conn(0x5621ad29a550 msgr2=0x5621ad2e2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.659+0000 7fcb60bf1740 1 -- 172.21.15.18:0/1598016361 shutdown_connections 2024-02-17T13:00:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.659+0000 7fcb60bf1740 1 -- 172.21.15.18:0/1598016361 wait complete. 2024-02-17T13:00:36.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.659+0000 7f86a141f740 1 Processor -- start 2024-02-17T13:00:36.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.659+0000 7f86a141f740 1 -- start start 2024-02-17T13:00:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.660+0000 7fcb60bf1740 1 -- start start 2024-02-17T13:00:36.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86a141f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053e9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86a141f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1b70 0x5652053e8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86a141f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053ea750 0x5652053e2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86a141f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652052ce130 con 0x5652053e1b70 2024-02-17T13:00:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86a141f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565205402100 con 0x5652053ea750 2024-02-17T13:00:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86a141f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652053e5cd0 con 0x5652053e1250 2024-02-17T13:00:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1b70 0x5652053e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1b70 0x5652053e8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42042/0 (socket says 172.21.15.18:42042) 2024-02-17T13:00:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86977fe640 1 -- 172.21.15.18:0/2258167663 learned_addr learned my addr 172.21.15.18:0/2258167663 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f8697fff640 1 --2- 172.21.15.18:0/2258167663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053e9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f869cdf4640 1 --2- 172.21.15.18:0/2258167663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053ea750 0x5652053e2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.660+0000 7fcb60bf1740 1 Processor -- start 2024-02-17T13:00:36.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86977fe640 1 -- 172.21.15.18:0/2258167663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 msgr2=0x5652053e9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86977fe640 1 --2- 172.21.15.18:0/2258167663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053e9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86977fe640 1 -- 172.21.15.18:0/2258167663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053ea750 msgr2=0x5652053e2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86977fe640 1 --2- 172.21.15.18:0/2258167663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053ea750 0x5652053e2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86977fe640 1 -- 172.21.15.18:0/2258167663 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652052e3c20 con 0x5652053e1b70 2024-02-17T13:00:36.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f86977fe640 1 --2- 172.21.15.18:0/2258167663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1b70 0x5652053e8020 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f868c0058c0 tx=0x7f868c009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddea1942f0a2321 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.660+0000 7fcb60bf1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 0x5621ad2ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.660+0000 7fcb60bf1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d30f0 0x5621ad2eaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.660+0000 7fcb60bf1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad3451c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.660+0000 7fcb60bf1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ad345700 con 0x5621ad2d3b90 2024-02-17T13:00:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.660+0000 7fcb60bf1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ad345870 con 0x5621ad2d27d0 2024-02-17T13:00:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.660+0000 7fcb60bf1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ad345a30 con 0x5621ad2d30f0 2024-02-17T13:00:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.660+0000 7fcb60bf1740 10 client.0 _ll_register_callbacks cb 0x7fcb400045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:36.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.660+0000 7f867dffb640 1 -- 172.21.15.18:0/2258167663 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c017040 con 0x5652053e1b70 2024-02-17T13:00:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.660+0000 7fcb577fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 0x5621ad2ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.660+0000 7fcb56ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d30f0 0x5621ad2eaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb577fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 0x5621ad2ea9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:33640/0 (socket says 172.21.15.18:33640) 2024-02-17T13:00:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb577fe640 1 -- 172.21.15.18:0/3568570956 learned_addr learned my addr 172.21.15.18:0/3568570956 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb57fff640 1 --2- 172.21.15.18:0/3568570956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad3451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.661+0000 7f867dffb640 1 -- 172.21.15.18:0/2258167663 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f868c00fbc0 con 0x5652053e1b70 2024-02-17T13:00:36.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.661+0000 7f867dffb640 1 -- 172.21.15.18:0/2258167663 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c003850 con 0x5652053e1b70 2024-02-17T13:00:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb577fe640 1 -- 172.21.15.18:0/3568570956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d30f0 msgr2=0x5621ad2eaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb577fe640 1 --2- 172.21.15.18:0/3568570956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d30f0 0x5621ad2eaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb577fe640 1 -- 172.21.15.18:0/3568570956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 msgr2=0x5621ad3451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb577fe640 1 --2- 172.21.15.18:0/3568570956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad3451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb577fe640 1 -- 172.21.15.18:0/3568570956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621ad238d80 con 0x5621ad2d27d0 2024-02-17T13:00:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb577fe640 1 --2- 172.21.15.18:0/3568570956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 0x5621ad2ea9a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcb4002cdd0 tx=0x7fcb40009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa53b88a93a4f370 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:00:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb36ffd640 1 -- 172.21.15.18:0/3568570956 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb400db070 con 0x5621ad2d27d0 2024-02-17T13:00:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb36ffd640 1 -- 172.21.15.18:0/3568570956 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fcb400a7040 con 0x5621ad2d27d0 2024-02-17T13:00:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb60bf1740 1 -- 172.21.15.18:0/3568570956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb400c53f0 con 0x5621ad2d27d0 2024-02-17T13:00:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb36ffd640 1 -- 172.21.15.18:0/3568570956 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb400db1d0 con 0x5621ad2d27d0 2024-02-17T13:00:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb60bf1740 10 client.28388 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.661+0000 7fcb60bf1740 1 -- 172.21.15.18:0/3568570956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5621ad1fb540 con 0x5621ad2d27d0 2024-02-17T13:00:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.661+0000 7f86a141f740 1 -- 172.21.15.18:0/2258167663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1b70 msgr2=0x5652053e8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.661+0000 7f86a141f740 1 --2- 172.21.15.18:0/2258167663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1b70 0x5652053e8020 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f868c0058c0 tx=0x7f868c009340 comp rx=0 tx=0).stop 2024-02-17T13:00:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.661+0000 7f86a141f740 1 -- 172.21.15.18:0/2258167663 shutdown_connections 2024-02-17T13:00:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.661+0000 7f86a141f740 1 --2- 172.21.15.18:0/2258167663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053ea750 0x5652053e2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.661+0000 7f86a141f740 1 --2- 172.21.15.18:0/2258167663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053e1b70 0x5652053e8020 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.661+0000 7f86a141f740 1 --2- 172.21.15.18:0/2258167663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053e9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.661+0000 7f86a141f740 1 -- 172.21.15.18:0/2258167663 >> 172.21.15.18:0/2258167663 conn(0x5652053db980 msgr2=0x5652053d6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.662+0000 7f86a141f740 1 -- 172.21.15.18:0/2258167663 shutdown_connections 2024-02-17T13:00:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.662+0000 7f86a141f740 1 -- 172.21.15.18:0/2258167663 wait complete. 2024-02-17T13:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.662+0000 7fcb36ffd640 1 -- 172.21.15.18:0/3568570956 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb400c53f0 con 0x5621ad2d27d0 2024-02-17T13:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.662+0000 7fcb36ffd640 1 -- 172.21.15.18:0/3568570956 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fcb4003c070 con 0x5621ad2d27d0 2024-02-17T13:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.662+0000 7fcb60bf1740 10 client.28388 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.662+0000 7fcb60bf1740 10 client.28388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.662+0000 7fcb60bf1740 20 client.28388 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.662+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/3568570956 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5621ad34cde0 0x5621ad34f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.662+0000 7fcb60bf1740 4 client.28388 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.662+0000 7fcb60bf1740 1 -- 172.21.15.18:0/3568570956 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5621acf77550 con 0x5621ad34cde0 2024-02-17T13:00:36.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.662+0000 7f86a141f740 1 -- start start 2024-02-17T13:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.662+0000 7fcb56ffd640 1 --2- 172.21.15.18:0/3568570956 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5621ad34cde0 0x5621ad34f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.662+0000 7f86a141f740 1 Processor -- start 2024-02-17T13:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.662+0000 7fcb56ffd640 1 --2- 172.21.15.18:0/3568570956 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5621ad34cde0 0x5621ad34f200 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b0436a0d7407b2 server_cookie=8d4cbe671da01b41 in_seq=0 out_seq=0 2024-02-17T13:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.662+0000 7fcb36ffd640 10 client.28388 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:36.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.662+0000 7f86a141f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.662+0000 7f86a141f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f86a141f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 0x5652053efee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f86a141f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652053f0420 con 0x5652053ea750 2024-02-17T13:00:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f8697fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f86a141f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652053f0590 con 0x5652053e1b70 2024-02-17T13:00:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f86a141f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652053f0750 con 0x5652053e1250 2024-02-17T13:00:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f8697fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053f1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:43476/0 (socket says 172.21.15.18:43476) 2024-02-17T13:00:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f8697fff640 1 -- 172.21.15.18:0/848628460 learned_addr learned my addr 172.21.15.18:0/848628460 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f86977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f86a141f740 10 client.0 _ll_register_callbacks cb 0x7f868c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f869cdf4640 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 0x5652053efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f8697fff640 1 -- 172.21.15.18:0/848628460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 msgr2=0x5652053ef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f8697fff640 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053ef9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f8697fff640 1 -- 172.21.15.18:0/848628460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 msgr2=0x5652053efee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f8697fff640 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 0x5652053efee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f8697fff640 1 -- 172.21.15.18:0/848628460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652053476a0 con 0x5652053e1250 2024-02-17T13:00:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f86977fe640 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f869cdf4640 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 0x5652053efee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f8697fff640 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053f1600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8688096ed0 tx=0x7f8688085040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac1e2b11b0c5254c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f86777fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:00:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f86777fe640 1 -- 172.21.15.18:0/848628460 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86880b2070 con 0x5652053e1250 2024-02-17T13:00:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f86a141f740 1 -- 172.21.15.18:0/848628460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8690032810 con 0x5652053e1250 2024-02-17T13:00:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.663+0000 7f86777fe640 1 -- 172.21.15.18:0/848628460 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f86880cb9a0 con 0x5652053e1250 2024-02-17T13:00:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.664+0000 7f86777fe640 1 -- 172.21.15.18:0/848628460 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86880ba040 con 0x5652053e1250 2024-02-17T13:00:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.664+0000 7f86a141f740 10 client.28426 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.664+0000 7f86a141f740 1 -- 172.21.15.18:0/848628460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5652053098c0 con 0x5652053e1250 2024-02-17T13:00:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.664+0000 7f86777fe640 1 -- 172.21.15.18:0/848628460 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86880b7020 con 0x5652053e1250 2024-02-17T13:00:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.664+0000 7f86777fe640 1 -- 172.21.15.18:0/848628460 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8688023070 con 0x5652053e1250 2024-02-17T13:00:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.664+0000 7f86a141f740 10 client.28426 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.664+0000 7f86a141f740 10 client.28426 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:00:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.664+0000 7f86a141f740 20 client.28426 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.664+0000 7f86a141f740 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x565205452330 0x565205454750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.664+0000 7f86a141f740 4 client.28426 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:00:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.664+0000 7f86a141f740 1 -- 172.21.15.18:0/848628460 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565205085720 con 0x565205452330 2024-02-17T13:00:36.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.664+0000 7f86977fe640 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x565205452330 0x565205454750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86777fe640 10 client.28426 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:00:36.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86977fe640 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x565205452330 0x565205454750 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b10469686d3b801 server_cookie=f32f8eeb21e55c83 in_seq=0 out_seq=0 2024-02-17T13:00:36.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86777fe640 1 -- 172.21.15.18:0/848628460 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565205085720 con 0x565205452330 2024-02-17T13:00:36.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86777fe640 10 client.28426 handle_command_reply: tid=0 2024-02-17T13:00:36.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86a141f740 1 client.28426 shutdown 2024-02-17T13:00:36.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86a141f740 1 -- 172.21.15.18:0/848628460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 msgr2=0x5652053f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86a141f740 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053f1600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8688096ed0 tx=0x7f8688085040 comp rx=0 tx=0).stop 2024-02-17T13:00:36.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86a141f740 1 -- 172.21.15.18:0/848628460 shutdown_connections 2024-02-17T13:00:36.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86a141f740 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652053ea750 0x5652053efee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86a141f740 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x565205452330 0x565205454750 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86a141f740 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5652053e1b70 0x5652053ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86777fe640 0 client.28426 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:00:36.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86a141f740 1 --2- 172.21.15.18:0/848628460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5652053e1250 0x5652053f1600 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86a141f740 1 -- 172.21.15.18:0/848628460 >> 172.21.15.18:0/848628460 conn(0x5652053db980 msgr2=0x5652053e2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86a141f740 1 -- 172.21.15.18:0/848628460 shutdown_connections 2024-02-17T13:00:36.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86a141f740 1 -- 172.21.15.18:0/848628460 wait complete. 2024-02-17T13:00:36.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.665+0000 7f86a141f740 20 client.28426 trim_cache size 0 max 16384 2024-02-17T13:00:36.697 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:00:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.666+0000 7f869e857640 1 -- 172.21.15.18:0/3401395750 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f86800a2230 msgr2=0x7f86800a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.666+0000 7f869e857640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f86800a2230 0x7f86800a46f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8690004af0 tx=0x7f869004b000 comp rx=0 tx=0).stop 2024-02-17T13:00:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.666+0000 7f869e857640 1 -- 172.21.15.18:0/3401395750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8698101370 msgr2=0x7f869819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.666+0000 7f869e857640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8698101370 0x7f869819f6b0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f868800d950 tx=0x7f868800de10 comp rx=0 tx=0).stop 2024-02-17T13:00:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.666+0000 7f869e857640 1 -- 172.21.15.18:0/3401395750 shutdown_connections 2024-02-17T13:00:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.666+0000 7f869e857640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f86800a2230 0x7f86800a46f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.666+0000 7f869e857640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8698199820 0x7f8698199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.666+0000 7f869e857640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698106440 0x7f869819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.666+0000 7f869e857640 1 --2- 172.21.15.18:0/3401395750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8698101370 0x7f869819f6b0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.666+0000 7f869e857640 1 -- 172.21.15.18:0/3401395750 >> 172.21.15.18:0/3401395750 conn(0x7f86980fcc10 msgr2=0x7f86980fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.666+0000 7f869e857640 1 -- 172.21.15.18:0/3401395750 shutdown_connections 2024-02-17T13:00:36.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.666+0000 7f869e857640 1 -- 172.21.15.18:0/3401395750 wait complete. 2024-02-17T13:00:36.700 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:00:36.700 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T13:00:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.701+0000 7fcb36ffd640 1 -- 172.21.15.18:0/3568570956 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5621acf77550 con 0x5621ad34cde0 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb36ffd640 10 client.28388 handle_command_reply: tid=0 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb60bf1740 1 client.28388 shutdown 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb60bf1740 1 -- 172.21.15.18:0/3568570956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 msgr2=0x5621ad2ea9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/3568570956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 0x5621ad2ea9a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcb4002cdd0 tx=0x7fcb40009880 comp rx=0 tx=0).stop 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb60bf1740 1 -- 172.21.15.18:0/3568570956 shutdown_connections 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/3568570956 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5621ad34cde0 0x5621ad34f200 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/3568570956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621ad2d3b90 0x5621ad3451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/3568570956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5621ad2d30f0 0x5621ad2eaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb60bf1740 1 --2- 172.21.15.18:0/3568570956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621ad2d27d0 0x5621ad2ea9a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb36ffd640 0 client.28388 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb60bf1740 1 -- 172.21.15.18:0/3568570956 >> 172.21.15.18:0/3568570956 conn(0x5621ad29a550 msgr2=0x5621ad2e30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb60bf1740 1 -- 172.21.15.18:0/3568570956 shutdown_connections 2024-02-17T13:00:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb60bf1740 1 -- 172.21.15.18:0/3568570956 wait complete. 2024-02-17T13:00:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.702+0000 7fcb60bf1740 20 client.28388 trim_cache size 0 max 16384 2024-02-17T13:00:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:00:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:00:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:00:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:00:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.703+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3787847506 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb440a22d0 msgr2=0x7fcb440a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.703+0000 7fcb5df4b640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb440a22d0 0x7fcb440a4790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcb50007df0 tx=0x7fcb50007470 comp rx=0 tx=0).stop 2024-02-17T13:00:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.703+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3787847506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb581002d0 msgr2=0x7fcb5819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.703+0000 7fcb5df4b640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb581002d0 0x7fcb5819d450 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fcb40005940 tx=0x7fcb40004040 comp rx=0 tx=0).stop 2024-02-17T13:00:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.703+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3787847506 shutdown_connections 2024-02-17T13:00:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.703+0000 7fcb5df4b640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcb440a22d0 0x7fcb440a4790 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.703+0000 7fcb5df4b640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb58105020 0x7fcb58197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.703+0000 7fcb5df4b640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb58100c90 0x7fcb5819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.703+0000 7fcb5df4b640 1 --2- 172.21.15.18:0/3787847506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb581002d0 0x7fcb5819d450 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.703+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3787847506 >> 172.21.15.18:0/3787847506 conn(0x7fcb58076ed0 msgr2=0x7fcb58068a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.703+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3787847506 shutdown_connections 2024-02-17T13:00:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:36.703+0000 7fcb5df4b640 1 -- 172.21.15.18:0/3787847506 wait complete. 2024-02-17T13:00:36.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:36 smithi018 ceph-mon[32625]: pgmap v335: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-02-17T13:00:36.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:36 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3594650441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:00:36.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 -- 172.21.15.18:0/92526070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1760100d20 msgr2=0x7f176010ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 --2- 172.21.15.18:0/92526070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1760100d20 0x7f176010ce70 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f1758009ca0 tx=0x7f175802c0f0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 -- 172.21.15.18:0/92526070 shutdown_connections 2024-02-17T13:00:36.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 --2- 172.21.15.18:0/92526070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1760100d20 0x7f176010ce70 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 --2- 172.21.15.18:0/92526070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1760100360 0x7f17601007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 --2- 172.21.15.18:0/92526070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1760106240 0x7f1760106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 -- 172.21.15.18:0/92526070 >> 172.21.15.18:0/92526070 conn(0x7f1760077060 msgr2=0x7f1760077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 -- 172.21.15.18:0/92526070 shutdown_connections 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 -- 172.21.15.18:0/92526070 wait complete. 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 Processor -- start 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 -- start start 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1760100360 0x7f17601976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.854+0000 7f176854b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1760100d20 0x7f1760197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f176854b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1760106240 0x7f176019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f176854b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176019c5a0 con 0x7f1760106240 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f176854b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176019c710 con 0x7f1760100360 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f176854b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176019c8d0 con 0x7f1760100d20 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f1766ae8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1760106240 0x7f176019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f1766ae8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1760106240 0x7f176019bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42070/0 (socket says 172.21.15.18:42070) 2024-02-17T13:00:36.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f1766ae8640 1 -- 172.21.15.18:0/859333343 learned_addr learned my addr 172.21.15.18:0/859333343 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f17662e7640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1760100360 0x7f17601976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f1765ae6640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1760100d20 0x7f1760197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f1766ae8640 1 -- 172.21.15.18:0/859333343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1760100d20 msgr2=0x7f1760197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f1766ae8640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1760100d20 0x7f1760197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f1766ae8640 1 -- 172.21.15.18:0/859333343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1760100360 msgr2=0x7f17601976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f1766ae8640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1760100360 0x7f17601976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f1766ae8640 1 -- 172.21.15.18:0/859333343 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1758009900 con 0x7f1760106240 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f1765ae6640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1760100d20 0x7f1760197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f1766ae8640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1760106240 0x7f176019bfa0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f1758037040 tx=0x7f1758034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f17577fe640 1 -- 172.21.15.18:0/859333343 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17580328b0 con 0x7f1760106240 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f17577fe640 1 -- 172.21.15.18:0/859333343 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1758003e50 con 0x7f1760106240 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f17577fe640 1 -- 172.21.15.18:0/859333343 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1758004830 con 0x7f1760106240 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f176854b640 1 -- 172.21.15.18:0/859333343 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176019cb50 con 0x7f1760106240 2024-02-17T13:00:36.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.855+0000 7f176854b640 1 -- 172.21.15.18:0/859333343 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17601a4430 con 0x7f1760106240 2024-02-17T13:00:36.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.858+0000 7f17577fe640 1 -- 172.21.15.18:0/859333343 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f175800a040 con 0x7f1760106240 2024-02-17T13:00:36.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.858+0000 7f176b1f4740 1 Processor -- start 2024-02-17T13:00:36.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f176b1f4740 1 -- start start 2024-02-17T13:00:36.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f176b1f4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78c250 0x558f9c78c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f176b1f4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 0x558f9c795210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f176b1f4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c795750 0x558f9c797b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f176b1f4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9c747960 con 0x558f9c78cb70 2024-02-17T13:00:36.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f176b1f4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9c6fb310 con 0x558f9c795750 2024-02-17T13:00:36.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f176b1f4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9c6f8840 con 0x558f9c78c250 2024-02-17T13:00:36.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f17577fe640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f174c0a22b0 0x7f174c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f17662e7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 0x558f9c795210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f17662e7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 0x558f9c795210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42082/0 (socket says 172.21.15.18:42082) 2024-02-17T13:00:36.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f17662e7640 1 -- 172.21.15.18:0/1895769618 learned_addr learned my addr 172.21.15.18:0/1895769618 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f1766ae8640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f174c0a22b0 0x7f174c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f17577fe640 1 -- 172.21.15.18:0/859333343 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f17580f0a20 con 0x7f1760106240 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f1765ae6640 1 --2- 172.21.15.18:0/1895769618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c795750 0x558f9c797b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f1765ae6640 1 --2- 172.21.15.18:0/1895769618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78c250 0x558f9c78c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f17662e7640 1 -- 172.21.15.18:0/1895769618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78c250 msgr2=0x558f9c78c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f17662e7640 1 --2- 172.21.15.18:0/1895769618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78c250 0x558f9c78c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f17662e7640 1 -- 172.21.15.18:0/1895769618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c795750 msgr2=0x558f9c797b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f17662e7640 1 --2- 172.21.15.18:0/1895769618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c795750 0x558f9c797b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f17662e7640 1 -- 172.21.15.18:0/1895769618 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f9c6b6060 con 0x558f9c78cb70 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f1765ae6640 1 --2- 172.21.15.18:0/1895769618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c795750 0x558f9c797b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.859+0000 7f1766ae8640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f174c0a22b0 0x7f174c0a4770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f17580ee2a0 tx=0x7f17580dd040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.860+0000 7f17662e7640 1 --2- 172.21.15.18:0/1895769618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 0x558f9c795210 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f175c00ac10 tx=0x7f175c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae89d930fe398ff server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.860+0000 7f1754bfe640 1 -- 172.21.15.18:0/1895769618 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c00a6e0 con 0x558f9c78cb70 2024-02-17T13:00:36.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.860+0000 7f1754bfe640 1 -- 172.21.15.18:0/1895769618 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f175c018b20 con 0x558f9c78cb70 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.860+0000 7f1754bfe640 1 -- 172.21.15.18:0/1895769618 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c010c40 con 0x558f9c78cb70 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.860+0000 7f176b1f4740 1 -- 172.21.15.18:0/1895769618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 msgr2=0x558f9c795210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.860+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1895769618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 0x558f9c795210 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f175c00ac10 tx=0x7f175c00f790 comp rx=0 tx=0).stop 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.860+0000 7f176b1f4740 1 -- 172.21.15.18:0/1895769618 shutdown_connections 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.860+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1895769618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c795750 0x558f9c797b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.860+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1895769618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 0x558f9c795210 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.860+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1895769618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78c250 0x558f9c78c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.860+0000 7f176b1f4740 1 -- 172.21.15.18:0/1895769618 >> 172.21.15.18:0/1895769618 conn(0x558f9c786980 msgr2=0x558f9c786d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f176b1f4740 1 -- 172.21.15.18:0/1895769618 shutdown_connections 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f176b1f4740 1 -- 172.21.15.18:0/1895769618 wait complete. 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f176b1f4740 1 -- start start 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f176b1f4740 1 Processor -- start 2024-02-17T13:00:36.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f176b1f4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78c250 0x558f9c7927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f176b1f4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c78cb70 0x558f9c792d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f176b1f4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c795750 0x558f9c793260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f176b1f4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9c6f4020 con 0x558f9c795750 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f176b1f4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9c3c7430 con 0x558f9c78cb70 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f176b1f4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9c383e00 con 0x558f9c78c250 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f176b1f4740 10 client.0 _ll_register_callbacks cb 0x558f9c69bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f1765ae6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c795750 0x558f9c793260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f1765ae6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c795750 0x558f9c793260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42092/0 (socket says 172.21.15.18:42092) 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f1765ae6640 1 -- 172.21.15.18:0/3184316953 learned_addr learned my addr 172.21.15.18:0/3184316953 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f1765ae6640 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78c250 0x558f9c7927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.861+0000 7f17662e7640 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c78cb70 0x558f9c792d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f1765ae6640 1 -- 172.21.15.18:0/3184316953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78c250 msgr2=0x558f9c7927e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f1765ae6640 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78c250 0x558f9c7927e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f1765ae6640 1 -- 172.21.15.18:0/3184316953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c78cb70 msgr2=0x558f9c792d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f1765ae6640 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c78cb70 0x558f9c792d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f1765ae6640 1 -- 172.21.15.18:0/3184316953 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c00a840 con 0x558f9c795750 2024-02-17T13:00:36.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f17662e7640 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c78cb70 0x558f9c792d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:00:36.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f1765ae6640 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c795750 0x558f9c793260 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f175000cf70 tx=0x7f1750016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80eb46a029289a08 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f1749ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:36.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f1749ffb640 1 -- 172.21.15.18:0/3184316953 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750019070 con 0x558f9c795750 2024-02-17T13:00:36.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f1749ffb640 1 -- 172.21.15.18:0/3184316953 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f17500140d0 con 0x558f9c795750 2024-02-17T13:00:36.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f176b1f4740 1 -- 172.21.15.18:0/3184316953 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f9c3f37e0 con 0x558f9c795750 2024-02-17T13:00:36.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f1749ffb640 1 -- 172.21.15.18:0/3184316953 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1750014290 con 0x558f9c795750 2024-02-17T13:00:36.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f176b1f4740 10 client.19377 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:36.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f176b1f4740 1 -- 172.21.15.18:0/3184316953 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x558f9c6f26a0 con 0x558f9c795750 2024-02-17T13:00:36.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f1749ffb640 1 -- 172.21.15.18:0/3184316953 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558f9c3f37e0 con 0x558f9c795750 2024-02-17T13:00:36.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f1749ffb640 1 -- 172.21.15.18:0/3184316953 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f175001e030 con 0x558f9c795750 2024-02-17T13:00:36.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f176b1f4740 10 client.19377 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:36.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f176b1f4740 10 client.19377 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:00:36.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f176b1f4740 20 client.19377 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:36.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f176b1f4740 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558f9c7a5c90 0x558f9c7a80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f176b1f4740 4 client.19377 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:36.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.862+0000 7f176b1f4740 1 -- 172.21.15.18:0/3184316953 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f9c430720 con 0x558f9c7a5c90 2024-02-17T13:00:36.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.863+0000 7f17662e7640 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558f9c7a5c90 0x558f9c7a80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.863+0000 7f17662e7640 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558f9c7a5c90 0x558f9c7a80b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4eaa0dd9423f6303 server_cookie=6c269b9284c6a707 in_seq=0 out_seq=0 2024-02-17T13:00:36.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.863+0000 7f1749ffb640 10 client.19377 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:00:36.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f1749ffb640 1 -- 172.21.15.18:0/3184316953 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f9c430720 con 0x558f9c7a5c90 2024-02-17T13:00:36.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f1749ffb640 10 client.19377 handle_command_reply: tid=0 2024-02-17T13:00:36.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f176b1f4740 1 client.19377 shutdown 2024-02-17T13:00:36.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f176b1f4740 1 -- 172.21.15.18:0/3184316953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c795750 msgr2=0x558f9c793260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f176b1f4740 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c795750 0x558f9c793260 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f175000cf70 tx=0x7f1750016040 comp rx=0 tx=0).stop 2024-02-17T13:00:36.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f176b1f4740 1 -- 172.21.15.18:0/3184316953 shutdown_connections 2024-02-17T13:00:36.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f176b1f4740 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558f9c7a5c90 0x558f9c7a80b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f176b1f4740 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c795750 0x558f9c793260 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f1749ffb640 0 client.19377 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:00:36.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f176b1f4740 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c78cb70 0x558f9c792d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f176b1f4740 1 --2- 172.21.15.18:0/3184316953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78c250 0x558f9c7927e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f176b1f4740 1 -- 172.21.15.18:0/3184316953 >> 172.21.15.18:0/3184316953 conn(0x558f9c786980 msgr2=0x558f9c7886d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f176b1f4740 1 -- 172.21.15.18:0/3184316953 shutdown_connections 2024-02-17T13:00:36.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f176b1f4740 1 -- 172.21.15.18:0/3184316953 wait complete. 2024-02-17T13:00:36.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.864+0000 7f176b1f4740 20 client.19377 trim_cache size 0 max 16384 2024-02-17T13:00:36.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f176b1f4740 1 Processor -- start 2024-02-17T13:00:36.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f176b1f4740 1 -- start start 2024-02-17T13:00:36.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f176b1f4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c78c250 0x558f9c794f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f176b1f4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 0x558f9c78fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f176b1f4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c795750 0x558f9c78ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f176b1f4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9c679130 con 0x558f9c78cb70 2024-02-17T13:00:36.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f176b1f4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9c7822b0 con 0x558f9c78c250 2024-02-17T13:00:36.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f176b1f4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9c7ad2a0 con 0x558f9c795750 2024-02-17T13:00:36.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f17662e7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 0x558f9c78fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f17662e7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 0x558f9c78fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42104/0 (socket says 172.21.15.18:42104) 2024-02-17T13:00:36.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f17662e7640 1 -- 172.21.15.18:0/1928317638 learned_addr learned my addr 172.21.15.18:0/1928317638 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f17662e7640 1 -- 172.21.15.18:0/1928317638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c795750 msgr2=0x558f9c78ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f17662e7640 1 --2- 172.21.15.18:0/1928317638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c795750 0x558f9c78ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f17662e7640 1 -- 172.21.15.18:0/1928317638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c78c250 msgr2=0x558f9c794f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f17662e7640 1 --2- 172.21.15.18:0/1928317638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c78c250 0x558f9c794f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f17662e7640 1 -- 172.21.15.18:0/1928317638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f9c68ec20 con 0x558f9c78cb70 2024-02-17T13:00:36.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.876+0000 7f17662e7640 1 --2- 172.21.15.18:0/1928317638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 0x558f9c78fa40 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f175c00fc90 tx=0x7f175c00af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38475f52ca93d26 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f174bfff640 1 -- 172.21.15.18:0/1928317638 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c027ea0 con 0x558f9c78cb70 2024-02-17T13:00:36.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f174bfff640 1 -- 172.21.15.18:0/1928317638 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f175c010890 con 0x558f9c78cb70 2024-02-17T13:00:36.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f174bfff640 1 -- 172.21.15.18:0/1928317638 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0055a0 con 0x558f9c78cb70 2024-02-17T13:00:36.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f176b1f4740 1 -- 172.21.15.18:0/1928317638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 msgr2=0x558f9c78fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1928317638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 0x558f9c78fa40 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f175c00fc90 tx=0x7f175c00af10 comp rx=0 tx=0).stop 2024-02-17T13:00:36.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f176b1f4740 1 -- 172.21.15.18:0/1928317638 shutdown_connections 2024-02-17T13:00:36.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1928317638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c795750 0x558f9c78ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1928317638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78cb70 0x558f9c78fa40 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1928317638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c78c250 0x558f9c794f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f176b1f4740 1 -- 172.21.15.18:0/1928317638 >> 172.21.15.18:0/1928317638 conn(0x558f9c786980 msgr2=0x558f9c78dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f176b1f4740 1 -- 172.21.15.18:0/1928317638 shutdown_connections 2024-02-17T13:00:36.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f176b1f4740 1 -- 172.21.15.18:0/1928317638 wait complete. 2024-02-17T13:00:36.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.877+0000 7f176b1f4740 1 -- start start 2024-02-17T13:00:36.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f176b1f4740 1 Processor -- start 2024-02-17T13:00:36.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f176b1f4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78c250 0x558f9c79ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f176b1f4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78cb70 0x558f9c79bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f176b1f4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c795750 0x558f9c7f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f176b1f4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9c7f5680 con 0x558f9c78c250 2024-02-17T13:00:36.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f176b1f4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9c7f57f0 con 0x558f9c795750 2024-02-17T13:00:36.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f176b1f4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9c7f59b0 con 0x558f9c78cb70 2024-02-17T13:00:36.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f176b1f4740 10 client.0 _ll_register_callbacks cb 0x7f175c00a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:36.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f1765ae6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78c250 0x558f9c79ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f1765ae6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78c250 0x558f9c79ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42112/0 (socket says 172.21.15.18:42112) 2024-02-17T13:00:36.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f1765ae6640 1 -- 172.21.15.18:0/1383290446 learned_addr learned my addr 172.21.15.18:0/1383290446 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:36.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f1765ae6640 1 -- 172.21.15.18:0/1383290446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78cb70 msgr2=0x558f9c79bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f1765ae6640 1 --2- 172.21.15.18:0/1383290446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78cb70 0x558f9c79bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f1765ae6640 1 -- 172.21.15.18:0/1383290446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c795750 msgr2=0x558f9c7f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:36.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f1765ae6640 1 --2- 172.21.15.18:0/1383290446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c795750 0x558f9c7f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f1765ae6640 1 -- 172.21.15.18:0/1383290446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f9c6b6060 con 0x558f9c78c250 2024-02-17T13:00:36.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f1765ae6640 1 --2- 172.21.15.18:0/1383290446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78c250 0x558f9c79ba70 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f1750001eb0 tx=0x7f175000c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b805c98ba7d1b06c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:36.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.878+0000 7f1749ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:36.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f1749ffb640 1 -- 172.21.15.18:0/1383290446 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175002d070 con 0x558f9c78c250 2024-02-17T13:00:36.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f176b1f4740 1 -- 172.21.15.18:0/1383290446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f175c018b20 con 0x558f9c78c250 2024-02-17T13:00:36.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f1749ffb640 1 -- 172.21.15.18:0/1383290446 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1750000e40 con 0x558f9c78c250 2024-02-17T13:00:36.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f1749ffb640 1 -- 172.21.15.18:0/1383290446 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1750019070 con 0x558f9c78c250 2024-02-17T13:00:36.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f176b1f4740 10 client.19383 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:36.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f176b1f4740 1 -- 172.21.15.18:0/1383290446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x558f9c6b48c0 con 0x558f9c78c250 2024-02-17T13:00:36.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f1749ffb640 1 -- 172.21.15.18:0/1383290446 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f175c018b20 con 0x558f9c78c250 2024-02-17T13:00:36.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f1749ffb640 1 -- 172.21.15.18:0/1383290446 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1750030020 con 0x558f9c78c250 2024-02-17T13:00:36.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f176b1f4740 10 client.19383 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:36.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f176b1f4740 10 client.19383 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:00:36.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f176b1f4740 20 client.19383 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:36.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1383290446 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558f9c7fe3a0 0x558f9c8007c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:36.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f176b1f4740 4 client.19383 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:00:36.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f176b1f4740 1 -- 172.21.15.18:0/1383290446 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558f9c47f890 con 0x558f9c7fe3a0 2024-02-17T13:00:36.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.879+0000 7f17662e7640 1 --2- 172.21.15.18:0/1383290446 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558f9c7fe3a0 0x558f9c8007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f17662e7640 1 --2- 172.21.15.18:0/1383290446 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558f9c7fe3a0 0x558f9c8007c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=272f3e56808ef3ff server_cookie=19f6092567cd319f in_seq=0 out_seq=0 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f1749ffb640 10 client.19383 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f1749ffb640 1 -- 172.21.15.18:0/1383290446 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558f9c47f890 con 0x558f9c7fe3a0 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f1749ffb640 10 client.19383 handle_command_reply: tid=0 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f176b1f4740 1 client.19383 shutdown 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f176b1f4740 1 -- 172.21.15.18:0/1383290446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78c250 msgr2=0x558f9c79ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1383290446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78c250 0x558f9c79ba70 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f1750001eb0 tx=0x7f175000c690 comp rx=0 tx=0).stop 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f176b1f4740 1 -- 172.21.15.18:0/1383290446 shutdown_connections 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1383290446 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x558f9c7fe3a0 0x558f9c8007c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1383290446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558f9c795750 0x558f9c7f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1383290446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558f9c78cb70 0x558f9c79bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f1749ffb640 0 client.19383 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:00:36.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f176b1f4740 1 --2- 172.21.15.18:0/1383290446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f9c78c250 0x558f9c79ba70 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f176b1f4740 1 -- 172.21.15.18:0/1383290446 >> 172.21.15.18:0/1383290446 conn(0x558f9c786980 msgr2=0x558f9c7852f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f176b1f4740 1 -- 172.21.15.18:0/1383290446 shutdown_connections 2024-02-17T13:00:36.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f176b1f4740 1 -- 172.21.15.18:0/1383290446 wait complete. 2024-02-17T13:00:36.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.880+0000 7f176b1f4740 20 client.19383 trim_cache size 0 max 16384 2024-02-17T13:00:36.882 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:00:36.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.881+0000 7f176854b640 1 -- 172.21.15.18:0/859333343 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f174c0a22b0 msgr2=0x7f174c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.881+0000 7f176854b640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f174c0a22b0 0x7f174c0a4770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f17580ee2a0 tx=0x7f17580dd040 comp rx=0 tx=0).stop 2024-02-17T13:00:36.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.881+0000 7f176854b640 1 -- 172.21.15.18:0/859333343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1760106240 msgr2=0x7f176019bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:36.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.881+0000 7f176854b640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1760106240 0x7f176019bfa0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f1758037040 tx=0x7f1758034a90 comp rx=0 tx=0).stop 2024-02-17T13:00:36.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.881+0000 7f176854b640 1 -- 172.21.15.18:0/859333343 shutdown_connections 2024-02-17T13:00:36.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.881+0000 7f176854b640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f174c0a22b0 0x7f174c0a4770 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.881+0000 7f176854b640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1760106240 0x7f176019bfa0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.881+0000 7f176854b640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1760100d20 0x7f1760197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.881+0000 7f176854b640 1 --2- 172.21.15.18:0/859333343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1760100360 0x7f17601976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:36.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.881+0000 7f176854b640 1 -- 172.21.15.18:0/859333343 >> 172.21.15.18:0/859333343 conn(0x7f1760077060 msgr2=0x7f17600fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:36.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.881+0000 7f176854b640 1 -- 172.21.15.18:0/859333343 shutdown_connections 2024-02-17T13:00:36.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:36.881+0000 7f176854b640 1 -- 172.21.15.18:0/859333343 wait complete. 2024-02-17T13:00:36.885 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:00:36.886 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T13:00:36.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:36 smithi146 ceph-mon[36454]: pgmap v335: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-02-17T13:00:36.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:36 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3594650441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:00:37.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.039+0000 7f8e09320640 1 -- 172.21.15.18:0/1065517387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e04101380 msgr2=0x7f8e04101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.039+0000 7f8e09320640 1 --2- 172.21.15.18:0/1065517387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e04101380 0x7f8e04101760 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009930 tx=0x7f8df402bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:37.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.039+0000 7f8e09320640 1 -- 172.21.15.18:0/1065517387 shutdown_connections 2024-02-17T13:00:37.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.039+0000 7f8e09320640 1 --2- 172.21.15.18:0/1065517387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e04106450 0x7f8e0410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.039+0000 7f8e09320640 1 --2- 172.21.15.18:0/1065517387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e04101d30 0x7f8e04105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.039+0000 7f8e09320640 1 --2- 172.21.15.18:0/1065517387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e04101380 0x7f8e04101760 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.039+0000 7f8e09320640 1 -- 172.21.15.18:0/1065517387 >> 172.21.15.18:0/1065517387 conn(0x7f8e04078580 msgr2=0x7f8e04078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.039+0000 7f8e09320640 1 -- 172.21.15.18:0/1065517387 shutdown_connections 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.039+0000 7f8e09320640 1 -- 172.21.15.18:0/1065517387 wait complete. 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.039+0000 7f8e09320640 1 Processor -- start 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e09320640 1 -- start start 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e09320640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e04101380 0x7f8e04073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e09320640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e04106450 0x7f8e040739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e09320640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e04073f00 0x7f8e041a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e09320640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e041a5fe0 con 0x7f8e04101380 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e09320640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e041a6150 con 0x7f8e04073f00 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e09320640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e041a6310 con 0x7f8e04106450 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e02ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e04101380 0x7f8e04073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e02ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e04101380 0x7f8e04073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42132/0 (socket says 172.21.15.18:42132) 2024-02-17T13:00:37.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e02ffd640 1 -- 172.21.15.18:0/4144794629 learned_addr learned my addr 172.21.15.18:0/4144794629 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e027fc640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e04106450 0x7f8e040739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e037fe640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e04073f00 0x7f8e041a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e02ffd640 1 -- 172.21.15.18:0/4144794629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e04106450 msgr2=0x7f8e040739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e04106450 0x7f8e040739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e02ffd640 1 -- 172.21.15.18:0/4144794629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e04073f00 msgr2=0x7f8e041a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e04073f00 0x7f8e041a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e02ffd640 1 -- 172.21.15.18:0/4144794629 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df4009560 con 0x7f8e04101380 2024-02-17T13:00:37.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e027fc640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e04106450 0x7f8e040739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:37.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e037fe640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e04073f00 0x7f8e041a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:37.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.040+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e04101380 0x7f8e04073480 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f8df4000c00 tx=0x7f8df4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.041+0000 7f8debfff640 1 -- 172.21.15.18:0/4144794629 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4004440 con 0x7f8e04101380 2024-02-17T13:00:37.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.041+0000 7f8debfff640 1 -- 172.21.15.18:0/4144794629 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8df40052d0 con 0x7f8e04101380 2024-02-17T13:00:37.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.041+0000 7f8e09320640 1 -- 172.21.15.18:0/4144794629 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e041a6590 con 0x7f8e04101380 2024-02-17T13:00:37.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.041+0000 7f8debfff640 1 -- 172.21.15.18:0/4144794629 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4003e10 con 0x7f8e04101380 2024-02-17T13:00:37.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.041+0000 7f8e09320640 1 -- 172.21.15.18:0/4144794629 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e041a6aa0 con 0x7f8e04101380 2024-02-17T13:00:37.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.043+0000 7f8debfff640 1 -- 172.21.15.18:0/4144794629 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8df4005440 con 0x7f8e04101380 2024-02-17T13:00:37.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.043+0000 7f8debfff640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8dec0a2260 0x7f8dec0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.043+0000 7f8e0bfcb740 1 Processor -- start 2024-02-17T13:00:37.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.043+0000 7f8e0bfcb740 1 -- start start 2024-02-17T13:00:37.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.043+0000 7f8e027fc640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8dec0a2260 0x7f8dec0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.043+0000 7f8debfff640 1 -- 172.21.15.18:0/4144794629 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8df40f0990 con 0x7f8e04101380 2024-02-17T13:00:37.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.043+0000 7f8e0bfcb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b50250 0x559a15b50630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.043+0000 7f8e0bfcb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b59210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.043+0000 7f8e0bfcb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b59750 0x559a15b5bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.043+0000 7f8e0bfcb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a15b0b960 con 0x559a15b50b70 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.043+0000 7f8e0bfcb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a15abf310 con 0x559a15b50250 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.043+0000 7f8e0bfcb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a15abc840 con 0x559a15b59750 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e027fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b59210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e027fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b59210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42140/0 (socket says 172.21.15.18:42140) 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e027fc640 1 -- 172.21.15.18:0/702043188 learned_addr learned my addr 172.21.15.18:0/702043188 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/702043188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b50250 0x559a15b50630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e037fe640 1 --2- 172.21.15.18:0/702043188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b59750 0x559a15b5bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e027fc640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8dec0a2260 0x7f8dec0a4720 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8df80058c0 tx=0x7f8df800f730 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e027fc640 1 -- 172.21.15.18:0/702043188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b59750 msgr2=0x559a15b5bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e027fc640 1 --2- 172.21.15.18:0/702043188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b59750 0x559a15b5bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e027fc640 1 -- 172.21.15.18:0/702043188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b50250 msgr2=0x559a15b50630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e027fc640 1 --2- 172.21.15.18:0/702043188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b50250 0x559a15b50630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e027fc640 1 -- 172.21.15.18:0/702043188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a15a7a060 con 0x559a15b50b70 2024-02-17T13:00:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e037fe640 1 --2- 172.21.15.18:0/702043188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b59750 0x559a15b5bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8e027fc640 1 --2- 172.21.15.18:0/702043188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b59210 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f8df800bfd0 tx=0x7f8df801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f227817c36b1ac5e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8de97fa640 1 -- 172.21.15.18:0/702043188 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df801d620 con 0x559a15b50b70 2024-02-17T13:00:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8de97fa640 1 -- 172.21.15.18:0/702043188 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8df801dc60 con 0x559a15b50b70 2024-02-17T13:00:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.044+0000 7f8de97fa640 1 -- 172.21.15.18:0/702043188 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8010c40 con 0x559a15b50b70 2024-02-17T13:00:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.045+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/702043188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 msgr2=0x559a15b59210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.045+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/702043188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b59210 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f8df800bfd0 tx=0x7f8df801a000 comp rx=0 tx=0).stop 2024-02-17T13:00:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.045+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/702043188 shutdown_connections 2024-02-17T13:00:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.045+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/702043188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b59750 0x559a15b5bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.045+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/702043188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b59210 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.045+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/702043188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b50250 0x559a15b50630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.045+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/702043188 >> 172.21.15.18:0/702043188 conn(0x559a15b4a980 msgr2=0x559a15b4ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.045+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/702043188 shutdown_connections 2024-02-17T13:00:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.045+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/702043188 wait complete. 2024-02-17T13:00:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.045+0000 7f8e0bfcb740 1 -- start start 2024-02-17T13:00:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.045+0000 7f8e0bfcb740 1 Processor -- start 2024-02-17T13:00:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e0bfcb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b50250 0x559a15b56600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e0bfcb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b56b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e0bfcb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b59750 0x559a15b57080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e0bfcb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a15ab8020 con 0x559a15b50b70 2024-02-17T13:00:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e0bfcb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1578b430 con 0x559a15b59750 2024-02-17T13:00:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e0bfcb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a15747e00 con 0x559a15b50250 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e0bfcb740 10 client.0 _ll_register_callbacks cb 0x559a15a5fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e027fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b56b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e027fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b56b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42148/0 (socket says 172.21.15.18:42148) 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e027fc640 1 -- 172.21.15.18:0/3190176455 learned_addr learned my addr 172.21.15.18:0/3190176455 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e037fe640 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b59750 0x559a15b57080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b50250 0x559a15b56600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e027fc640 1 -- 172.21.15.18:0/3190176455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b50250 msgr2=0x559a15b56600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e027fc640 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b50250 0x559a15b56600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e027fc640 1 -- 172.21.15.18:0/3190176455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b59750 msgr2=0x559a15b57080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e027fc640 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b59750 0x559a15b57080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e027fc640 1 -- 172.21.15.18:0/3190176455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df800bdb0 con 0x559a15b50b70 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e037fe640 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b59750 0x559a15b57080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e027fc640 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b56b40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f8df8000c00 tx=0x7f8df8010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb341038c15daf49 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8de6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8de6ff5640 1 -- 172.21.15.18:0/3190176455 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8031070 con 0x559a15b50b70 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8de6ff5640 1 -- 172.21.15.18:0/3190176455 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8df8018050 con 0x559a15b50b70 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/3190176455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a157b77e0 con 0x559a15b50b70 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8de6ff5640 1 -- 172.21.15.18:0/3190176455 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8df801d420 con 0x559a15b50b70 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e0bfcb740 10 client.19395 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.046+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/3190176455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x559a15ab66a0 con 0x559a15b50b70 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.047+0000 7f8de6ff5640 1 -- 172.21.15.18:0/3190176455 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559a157b77e0 con 0x559a15b50b70 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.047+0000 7f8de6ff5640 1 -- 172.21.15.18:0/3190176455 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8df8034020 con 0x559a15b50b70 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.047+0000 7f8e0bfcb740 10 client.19395 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.047+0000 7f8e0bfcb740 10 client.19395 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.047+0000 7f8e0bfcb740 20 client.19395 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.047+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559a15b699d0 0x559a15b6bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.047+0000 7f8e0bfcb740 4 client.19395 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:37.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.047+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/3190176455 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a157f4720 con 0x559a15b699d0 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.047+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559a15b699d0 0x559a15b6bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.047+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559a15b699d0 0x559a15b6bdf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17944a7d32fc1a9e server_cookie=cc4689cfa4ec6681 in_seq=0 out_seq=0 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.047+0000 7f8de6ff5640 10 client.19395 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8de6ff5640 1 -- 172.21.15.18:0/3190176455 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559a157f4720 con 0x559a15b699d0 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8de6ff5640 10 client.19395 handle_command_reply: tid=0 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8e0bfcb740 1 client.19395 shutdown 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/3190176455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 msgr2=0x559a15b56b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b56b40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f8df8000c00 tx=0x7f8df8010bf0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/3190176455 shutdown_connections 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559a15b699d0 0x559a15b6bdf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b59750 0x559a15b57080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b56b40 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8de6ff5640 0 client.19395 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:00:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/3190176455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b50250 0x559a15b56600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/3190176455 >> 172.21.15.18:0/3190176455 conn(0x559a15b4a980 msgr2=0x559a15b4c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/3190176455 shutdown_connections 2024-02-17T13:00:37.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/3190176455 wait complete. 2024-02-17T13:00:37.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.048+0000 7f8e0bfcb740 20 client.19395 trim_cache size 0 max 16384 2024-02-17T13:00:37.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e0bfcb740 1 Processor -- start 2024-02-17T13:00:37.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e0bfcb740 1 -- start start 2024-02-17T13:00:37.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e0bfcb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50250 0x559a15b55760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e0bfcb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b50b70 0x559a15b56ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e0bfcb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b59750 0x559a15b49a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e0bfcb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a15a3d130 con 0x559a15b50250 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e0bfcb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a15b71000 con 0x559a15b59750 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e0bfcb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a15b55ca0 con 0x559a15b50b70 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e02ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50250 0x559a15b55760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e02ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50250 0x559a15b55760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42152/0 (socket says 172.21.15.18:42152) 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e02ffd640 1 -- 172.21.15.18:0/1410098814 learned_addr learned my addr 172.21.15.18:0/1410098814 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e02ffd640 1 -- 172.21.15.18:0/1410098814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b50b70 msgr2=0x559a15b56ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/1410098814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b50b70 0x559a15b56ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e02ffd640 1 -- 172.21.15.18:0/1410098814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b59750 msgr2=0x559a15b49a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/1410098814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b59750 0x559a15b49a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.060+0000 7f8e02ffd640 1 -- 172.21.15.18:0/1410098814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a15a52c20 con 0x559a15b50250 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/1410098814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50250 0x559a15b55760 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f8df40b99e0 tx=0x7f8df40f0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62782a32745575b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8de8ff9640 1 -- 172.21.15.18:0/1410098814 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df40bdcd0 con 0x559a15b50250 2024-02-17T13:00:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8de8ff9640 1 -- 172.21.15.18:0/1410098814 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8df40b9040 con 0x559a15b50250 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8de8ff9640 1 -- 172.21.15.18:0/1410098814 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df40035f0 con 0x559a15b50250 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/1410098814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50250 msgr2=0x559a15b55760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/1410098814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50250 0x559a15b55760 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f8df40b99e0 tx=0x7f8df40f0040 comp rx=0 tx=0).stop 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/1410098814 shutdown_connections 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/1410098814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b59750 0x559a15b49a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/1410098814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b50b70 0x559a15b56ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/1410098814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50250 0x559a15b55760 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/1410098814 >> 172.21.15.18:0/1410098814 conn(0x559a15b4a980 msgr2=0x559a15b492f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/1410098814 shutdown_connections 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.061+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/1410098814 wait complete. 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e0bfcb740 1 -- start start 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e0bfcb740 1 Processor -- start 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e0bfcb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b50250 0x559a15b60b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e0bfcb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b5eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e0bfcb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b59750 0x559a15b5f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e0bfcb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a15b5fba0 con 0x559a15b50b70 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e0bfcb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a15b5fd10 con 0x559a15b50250 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e0bfcb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a15bbff10 con 0x559a15b59750 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e0bfcb740 10 client.0 _ll_register_callbacks cb 0x7f8df80123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e027fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b5eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e027fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b5eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42168/0 (socket says 172.21.15.18:42168) 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e027fc640 1 -- 172.21.15.18:0/2508461885 learned_addr learned my addr 172.21.15.18:0/2508461885 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b50250 0x559a15b60b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e027fc640 1 -- 172.21.15.18:0/2508461885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b59750 msgr2=0x559a15b5f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e027fc640 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b59750 0x559a15b5f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e027fc640 1 -- 172.21.15.18:0/2508461885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b50250 msgr2=0x559a15b60b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e027fc640 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b50250 0x559a15b60b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e027fc640 1 -- 172.21.15.18:0/2508461885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e041a6aa0 con 0x559a15b50b70 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b50250 0x559a15b60b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8e027fc640 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b5eea0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f8df800bad0 tx=0x7f8df8010460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9c12fbe0d6cfc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8de6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.062+0000 7f8de6ff5640 1 -- 172.21.15.18:0/2508461885 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8031030 con 0x559a15b50b70 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.063+0000 7f8de6ff5640 1 -- 172.21.15.18:0/2508461885 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8df8032070 con 0x559a15b50b70 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.063+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/2508461885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8df801dc60 con 0x559a15b50b70 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.063+0000 7f8de6ff5640 1 -- 172.21.15.18:0/2508461885 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8df802d420 con 0x559a15b50b70 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.063+0000 7f8e0bfcb740 10 client.19401 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.063+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/2508461885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x559a15a788c0 con 0x559a15b50b70 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.063+0000 7f8de6ff5640 1 -- 172.21.15.18:0/2508461885 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8df801dc60 con 0x559a15b50b70 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.063+0000 7f8de6ff5640 1 -- 172.21.15.18:0/2508461885 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8df8034070 con 0x559a15b50b70 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 10 client.19401 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 10 client.19401 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:00:37.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 20 client.19401 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559a15bc12c0 0x559a15bc36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 4 client.19401 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/2508461885 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559a15843890 con 0x559a15bc12c0 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559a15bc12c0 0x559a15bc36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8de6ff5640 10 client.19401 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e02ffd640 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559a15bc12c0 0x559a15bc36e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a25f8ec7d5b5656 server_cookie=9da7232e19536433 in_seq=0 out_seq=0 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8de6ff5640 1 -- 172.21.15.18:0/2508461885 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559a15843890 con 0x559a15bc12c0 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8de6ff5640 10 client.19401 handle_command_reply: tid=0 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 1 client.19401 shutdown 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/2508461885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 msgr2=0x559a15b5eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b5eea0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f8df800bad0 tx=0x7f8df8010460 comp rx=0 tx=0).stop 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/2508461885 shutdown_connections 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x559a15bc12c0 0x559a15bc36e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559a15b59750 0x559a15b5f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8de6ff5640 0 client.19401 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:00:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a15b50b70 0x559a15b5eea0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 1 --2- 172.21.15.18:0/2508461885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a15b50250 0x559a15b60b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.064+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/2508461885 >> 172.21.15.18:0/2508461885 conn(0x559a15b4a980 msgr2=0x559a15b49b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/2508461885 shutdown_connections 2024-02-17T13:00:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e0bfcb740 1 -- 172.21.15.18:0/2508461885 wait complete. 2024-02-17T13:00:37.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e0bfcb740 20 client.19401 trim_cache size 0 max 16384 2024-02-17T13:00:37.061 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:00:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e09320640 1 -- 172.21.15.18:0/4144794629 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8dec0a2260 msgr2=0x7f8dec0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e09320640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8dec0a2260 0x7f8dec0a4720 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8df80058c0 tx=0x7f8df800f730 comp rx=0 tx=0).stop 2024-02-17T13:00:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e09320640 1 -- 172.21.15.18:0/4144794629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e04101380 msgr2=0x7f8e04073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e09320640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e04101380 0x7f8e04073480 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f8df4000c00 tx=0x7f8df4004040 comp rx=0 tx=0).stop 2024-02-17T13:00:37.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e09320640 1 -- 172.21.15.18:0/4144794629 shutdown_connections 2024-02-17T13:00:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e09320640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8dec0a2260 0x7f8dec0a4720 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e09320640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e04073f00 0x7f8e041a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e09320640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e04106450 0x7f8e040739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e09320640 1 --2- 172.21.15.18:0/4144794629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e04101380 0x7f8e04073480 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e09320640 1 -- 172.21.15.18:0/4144794629 >> 172.21.15.18:0/4144794629 conn(0x7f8e04078580 msgr2=0x7f8e040ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e09320640 1 -- 172.21.15.18:0/4144794629 shutdown_connections 2024-02-17T13:00:37.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.065+0000 7f8e09320640 1 -- 172.21.15.18:0/4144794629 wait complete. 2024-02-17T13:00:37.065 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:00:37.066 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T13:00:37.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:36 smithi119 ceph-mon[36188]: pgmap v335: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-02-17T13:00:37.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:36 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3594650441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:00:37.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.180+0000 7f90d6afb640 1 -- 172.21.15.18:0/2142474797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90d0100c10 msgr2=0x7f90d0104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.180+0000 7f90d6afb640 1 --2- 172.21.15.18:0/2142474797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90d0100c10 0x7f90d0104c50 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009930 tx=0x7f90bc02bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:37.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.180+0000 7f90d6afb640 1 -- 172.21.15.18:0/2142474797 shutdown_connections 2024-02-17T13:00:37.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.180+0000 7f90d6afb640 1 --2- 172.21.15.18:0/2142474797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d0105330 0x7f90d010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.180+0000 7f90d6afb640 1 --2- 172.21.15.18:0/2142474797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90d0100c10 0x7f90d0104c50 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.180+0000 7f90d6afb640 1 --2- 172.21.15.18:0/2142474797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d0100260 0x7f90d0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.180+0000 7f90d6afb640 1 -- 172.21.15.18:0/2142474797 >> 172.21.15.18:0/2142474797 conn(0x7f90d0077180 msgr2=0x7f90d0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.180+0000 7f90d6afb640 1 -- 172.21.15.18:0/2142474797 shutdown_connections 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.180+0000 7f90d6afb640 1 -- 172.21.15.18:0/2142474797 wait complete. 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.180+0000 7f90d6afb640 1 Processor -- start 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.180+0000 7f90d6afb640 1 -- start start 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90d6afb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d0100260 0x7f90d0101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90d6afb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90d0105330 0x7f90d0102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90d6afb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d0102770 0x7f90d01a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90d6afb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d01a5f60 con 0x7f90d0105330 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90d6afb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d01a60d0 con 0x7f90d0100260 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90d6afb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d01a6290 con 0x7f90d0102770 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90cffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90d0105330 0x7f90d0102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90cffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90d0105330 0x7f90d0102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42194/0 (socket says 172.21.15.18:42194) 2024-02-17T13:00:37.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90cffff640 1 -- 172.21.15.18:0/1606943711 learned_addr learned my addr 172.21.15.18:0/1606943711 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90d4897640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d0100260 0x7f90d0101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90d5098640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d0102770 0x7f90d01a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90cffff640 1 -- 172.21.15.18:0/1606943711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d0102770 msgr2=0x7f90d01a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90cffff640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d0102770 0x7f90d01a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90cffff640 1 -- 172.21.15.18:0/1606943711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d0100260 msgr2=0x7f90d0101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90cffff640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d0100260 0x7f90d0101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90cffff640 1 -- 172.21.15.18:0/1606943711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc009560 con 0x7f90d0105330 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90cffff640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90d0105330 0x7f90d0102230 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009230 tx=0x7f90bc02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90cdffb640 1 -- 172.21.15.18:0/1606943711 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc004350 con 0x7f90d0105330 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90cdffb640 1 -- 172.21.15.18:0/1606943711 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90bc0044b0 con 0x7f90d0105330 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90d6afb640 1 -- 172.21.15.18:0/1606943711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d01a6510 con 0x7f90d0105330 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.181+0000 7f90cdffb640 1 -- 172.21.15.18:0/1606943711 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc003da0 con 0x7f90d0105330 2024-02-17T13:00:37.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.182+0000 7f90d6afb640 1 -- 172.21.15.18:0/1606943711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d01a6a20 con 0x7f90d0105330 2024-02-17T13:00:37.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.184+0000 7f90cdffb640 1 -- 172.21.15.18:0/1606943711 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f90bc004df0 con 0x7f90d0105330 2024-02-17T13:00:37.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.184+0000 7f90d97af740 1 Processor -- start 2024-02-17T13:00:37.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.184+0000 7f90d97af740 1 -- start start 2024-02-17T13:00:37.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.184+0000 7f90d97af740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 0x557256ccc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d97af740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cd5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d97af740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256cd5750 0x557256cd7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d97af740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557256c87960 con 0x557256ccc250 2024-02-17T13:00:37.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d97af740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557256c3b310 con 0x557256cccb70 2024-02-17T13:00:37.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d97af740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557256c38840 con 0x557256cd5750 2024-02-17T13:00:37.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90cdffb640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f90c00a2260 0x7f90c00a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d5098640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 0x557256ccc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90cdffb640 1 -- 172.21.15.18:0/1606943711 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f90bc0f0000 con 0x7f90d0105330 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d5098640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 0x557256ccc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42208/0 (socket says 172.21.15.18:42208) 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d5098640 1 -- 172.21.15.18:0/527549128 learned_addr learned my addr 172.21.15.18:0/527549128 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d5098640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f90c00a2260 0x7f90c00a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d4897640 1 --2- 172.21.15.18:0/527549128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cd5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90cffff640 1 --2- 172.21.15.18:0/527549128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256cd5750 0x557256cd7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d5098640 1 -- 172.21.15.18:0/527549128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256cd5750 msgr2=0x557256cd7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d5098640 1 --2- 172.21.15.18:0/527549128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256cd5750 0x557256cd7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d5098640 1 -- 172.21.15.18:0/527549128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 msgr2=0x557256cd5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d5098640 1 --2- 172.21.15.18:0/527549128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cd5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d5098640 1 -- 172.21.15.18:0/527549128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557256bf6060 con 0x557256ccc250 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90cffff640 1 --2- 172.21.15.18:0/527549128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256cd5750 0x557256cd7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d5098640 1 --2- 172.21.15.18:0/527549128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 0x557256ccc630 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f90c8011da0 tx=0x7f90c800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2476dd0f5da236a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90d5098640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f90c00a2260 0x7f90c00a4720 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f90c800fc90 tx=0x7f90c801e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90baffd640 1 -- 172.21.15.18:0/527549128 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c801dbf0 con 0x557256ccc250 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90baffd640 1 -- 172.21.15.18:0/527549128 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90c8009dc0 con 0x557256ccc250 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.185+0000 7f90baffd640 1 -- 172.21.15.18:0/527549128 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c80042b0 con 0x557256ccc250 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.186+0000 7f90d97af740 1 -- 172.21.15.18:0/527549128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 msgr2=0x557256ccc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.186+0000 7f90d97af740 1 --2- 172.21.15.18:0/527549128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 0x557256ccc630 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f90c8011da0 tx=0x7f90c800f790 comp rx=0 tx=0).stop 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.186+0000 7f90d97af740 1 -- 172.21.15.18:0/527549128 shutdown_connections 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.186+0000 7f90d97af740 1 --2- 172.21.15.18:0/527549128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256cd5750 0x557256cd7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.186+0000 7f90d97af740 1 --2- 172.21.15.18:0/527549128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cd5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.186+0000 7f90d97af740 1 --2- 172.21.15.18:0/527549128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 0x557256ccc630 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.186+0000 7f90d97af740 1 -- 172.21.15.18:0/527549128 >> 172.21.15.18:0/527549128 conn(0x557256cc6980 msgr2=0x557256cc6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.186+0000 7f90d97af740 1 -- 172.21.15.18:0/527549128 shutdown_connections 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.186+0000 7f90d97af740 1 -- 172.21.15.18:0/527549128 wait complete. 2024-02-17T13:00:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.186+0000 7f90d97af740 1 -- start start 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d97af740 1 Processor -- start 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d97af740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256ccc250 0x557256cd27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d97af740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cd2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d97af740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 0x557256cd3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d97af740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557256c34020 con 0x557256cd5750 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d97af740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557256907430 con 0x557256cccb70 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d97af740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572568c3e00 con 0x557256ccc250 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d97af740 10 client.0 _ll_register_callbacks cb 0x557256bdbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d5098640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 0x557256cd3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d5098640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 0x557256cd3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42210/0 (socket says 172.21.15.18:42210) 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d5098640 1 -- 172.21.15.18:0/553351386 learned_addr learned my addr 172.21.15.18:0/553351386 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90cffff640 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cd2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d4897640 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256ccc250 0x557256cd27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d5098640 1 -- 172.21.15.18:0/553351386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256ccc250 msgr2=0x557256cd27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d5098640 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256ccc250 0x557256cd27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d5098640 1 -- 172.21.15.18:0/553351386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 msgr2=0x557256cd2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d5098640 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cd2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d5098640 1 -- 172.21.15.18:0/553351386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c8019040 con 0x557256cd5750 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90cffff640 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cd2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d4897640 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256ccc250 0x557256cd27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.187+0000 7f90d5098640 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 0x557256cd3230 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f90c80099b0 tx=0x7f90c8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd813c2a6928fa0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90affff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90affff640 1 -- 172.21.15.18:0/553351386 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8011910 con 0x557256cd5750 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90affff640 1 -- 172.21.15.18:0/553351386 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f90c8004260 con 0x557256cd5750 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90d97af740 1 -- 172.21.15.18:0/553351386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572569337e0 con 0x557256cd5750 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90affff640 1 -- 172.21.15.18:0/553351386 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90c8005050 con 0x557256cd5750 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90d97af740 10 client.19413 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90d97af740 1 -- 172.21.15.18:0/553351386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x557256c326a0 con 0x557256cd5750 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90affff640 1 -- 172.21.15.18:0/553351386 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5572569337e0 con 0x557256cd5750 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90affff640 1 -- 172.21.15.18:0/553351386 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f90c8024030 con 0x557256cd5750 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90d97af740 10 client.19413 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90d97af740 10 client.19413 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90d97af740 20 client.19413 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90d97af740 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x557256ce5a90 0x557256ce7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90d97af740 4 client.19413 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.188+0000 7f90d97af740 1 -- 172.21.15.18:0/553351386 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557256970720 con 0x557256ce5a90 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.189+0000 7f90d4897640 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x557256ce5a90 0x557256ce7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.189+0000 7f90d4897640 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x557256ce5a90 0x557256ce7e80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9efed32cdbbb80a4 server_cookie=dc2592c1dcf92462 in_seq=0 out_seq=0 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.189+0000 7f90affff640 10 client.19413 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90affff640 1 -- 172.21.15.18:0/553351386 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557256970720 con 0x557256ce5a90 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90affff640 10 client.19413 handle_command_reply: tid=0 2024-02-17T13:00:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90d97af740 1 client.19413 shutdown 2024-02-17T13:00:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90d97af740 1 -- 172.21.15.18:0/553351386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 msgr2=0x557256cd3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90d97af740 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 0x557256cd3230 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f90c80099b0 tx=0x7f90c8003970 comp rx=0 tx=0).stop 2024-02-17T13:00:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90d97af740 1 -- 172.21.15.18:0/553351386 shutdown_connections 2024-02-17T13:00:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90d97af740 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x557256ce5a90 0x557256ce7e80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90d97af740 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 0x557256cd3230 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90d97af740 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cd2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90affff640 0 client.19413 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:00:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90d97af740 1 --2- 172.21.15.18:0/553351386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256ccc250 0x557256cd27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90d97af740 1 -- 172.21.15.18:0/553351386 >> 172.21.15.18:0/553351386 conn(0x557256cc6980 msgr2=0x557256cc86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90d97af740 1 -- 172.21.15.18:0/553351386 shutdown_connections 2024-02-17T13:00:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90d97af740 1 -- 172.21.15.18:0/553351386 wait complete. 2024-02-17T13:00:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.190+0000 7f90d97af740 20 client.19413 trim_cache size 0 max 16384 2024-02-17T13:00:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d97af740 1 Processor -- start 2024-02-17T13:00:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d97af740 1 -- start start 2024-02-17T13:00:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d97af740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256ccc250 0x557256cd4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d97af740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cd3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d97af740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 0x557256ccfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d97af740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557256bb9130 con 0x557256cd5750 2024-02-17T13:00:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d97af740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557256ccbef0 con 0x557256cccb70 2024-02-17T13:00:37.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d97af740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557256ccb540 con 0x557256ccc250 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d5098640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 0x557256ccfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d5098640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 0x557256ccfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42216/0 (socket says 172.21.15.18:42216) 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d5098640 1 -- 172.21.15.18:0/3757119242 learned_addr learned my addr 172.21.15.18:0/3757119242 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d5098640 1 -- 172.21.15.18:0/3757119242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256ccc250 msgr2=0x557256cd4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d5098640 1 --2- 172.21.15.18:0/3757119242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256ccc250 0x557256cd4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d5098640 1 -- 172.21.15.18:0/3757119242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 msgr2=0x557256cd3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d5098640 1 --2- 172.21.15.18:0/3757119242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cd3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d5098640 1 -- 172.21.15.18:0/3757119242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557256bcec20 con 0x557256cd5750 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90d5098640 1 --2- 172.21.15.18:0/3757119242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 0x557256ccfa10 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f90c8011fd0 tx=0x7f90c8009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afad9bdb1ef2dfb2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90ba7fc640 1 -- 172.21.15.18:0/3757119242 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8011910 con 0x557256cd5750 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.202+0000 7f90ba7fc640 1 -- 172.21.15.18:0/3757119242 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90c80039e0 con 0x557256cd5750 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.203+0000 7f90ba7fc640 1 -- 172.21.15.18:0/3757119242 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c80386b0 con 0x557256cd5750 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.203+0000 7f90d97af740 1 -- 172.21.15.18:0/3757119242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 msgr2=0x557256ccfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.203+0000 7f90d97af740 1 --2- 172.21.15.18:0/3757119242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 0x557256ccfa10 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f90c8011fd0 tx=0x7f90c8009ce0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.203+0000 7f90d97af740 1 -- 172.21.15.18:0/3757119242 shutdown_connections 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.203+0000 7f90d97af740 1 --2- 172.21.15.18:0/3757119242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256cd5750 0x557256ccfa10 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.203+0000 7f90d97af740 1 --2- 172.21.15.18:0/3757119242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cd3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.203+0000 7f90d97af740 1 --2- 172.21.15.18:0/3757119242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256ccc250 0x557256cd4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.203+0000 7f90d97af740 1 -- 172.21.15.18:0/3757119242 >> 172.21.15.18:0/3757119242 conn(0x557256cc6980 msgr2=0x557256cec800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.203+0000 7f90d97af740 1 -- 172.21.15.18:0/3757119242 shutdown_connections 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.203+0000 7f90d97af740 1 -- 172.21.15.18:0/3757119242 wait complete. 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.203+0000 7f90d97af740 1 -- start start 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.203+0000 7f90d97af740 1 Processor -- start 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d97af740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 0x557256cdca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d97af740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cdade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d97af740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256cd5750 0x557256cdb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d97af740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557256cdbae0 con 0x557256ccc250 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d97af740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557256cdbc50 con 0x557256cccb70 2024-02-17T13:00:37.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d97af740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557256d3bfc0 con 0x557256cd5750 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d97af740 10 client.0 _ll_register_callbacks cb 0x7f90c8011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d4897640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 0x557256cdca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d4897640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 0x557256cdca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42224/0 (socket says 172.21.15.18:42224) 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d4897640 1 -- 172.21.15.18:0/91244833 learned_addr learned my addr 172.21.15.18:0/91244833 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d4897640 1 -- 172.21.15.18:0/91244833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256cd5750 msgr2=0x557256cdb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d4897640 1 --2- 172.21.15.18:0/91244833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256cd5750 0x557256cdb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d4897640 1 -- 172.21.15.18:0/91244833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 msgr2=0x557256cdade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d4897640 1 --2- 172.21.15.18:0/91244833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cdade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d4897640 1 -- 172.21.15.18:0/91244833 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557256c326a0 con 0x557256ccc250 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90d4897640 1 --2- 172.21.15.18:0/91244833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 0x557256cdca40 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x557256ce8750 tx=0x7f90c4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adf8cca29aaaf41 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90affff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90affff640 1 -- 172.21.15.18:0/91244833 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c40122d0 con 0x557256ccc250 2024-02-17T13:00:37.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.204+0000 7f90affff640 1 -- 172.21.15.18:0/91244833 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f90c4004030 con 0x557256ccc250 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90d97af740 1 -- 172.21.15.18:0/91244833 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90c8009dc0 con 0x557256ccc250 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90affff640 1 -- 172.21.15.18:0/91244833 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90c40041f0 con 0x557256ccc250 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90d97af740 10 client.19419 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90d97af740 1 -- 172.21.15.18:0/91244833 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x557256bf48c0 con 0x557256ccc250 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90affff640 1 -- 172.21.15.18:0/91244833 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90c8009dc0 con 0x557256ccc250 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90affff640 1 -- 172.21.15.18:0/91244833 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f90c401e5d0 con 0x557256ccc250 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90d97af740 10 client.19419 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90d97af740 10 client.19419 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90d97af740 20 client.19419 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90d97af740 1 --2- 172.21.15.18:0/91244833 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x557256d3d840 0x557256d3fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90d97af740 4 client.19419 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90d97af740 1 -- 172.21.15.18:0/91244833 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5572569bf890 con 0x557256d3d840 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.205+0000 7f90cffff640 1 --2- 172.21.15.18:0/91244833 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x557256d3d840 0x557256d3fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90cffff640 1 --2- 172.21.15.18:0/91244833 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x557256d3d840 0x557256d3fc80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29e15c0a6d3ea8cb server_cookie=d1033c4912cd35a in_seq=0 out_seq=0 2024-02-17T13:00:37.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90affff640 10 client.19419 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90affff640 1 -- 172.21.15.18:0/91244833 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5572569bf890 con 0x557256d3d840 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90affff640 10 client.19419 handle_command_reply: tid=0 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90d97af740 1 client.19419 shutdown 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90d97af740 1 -- 172.21.15.18:0/91244833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 msgr2=0x557256cdca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90d97af740 1 --2- 172.21.15.18:0/91244833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 0x557256cdca40 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x557256ce8750 tx=0x7f90c4012820 comp rx=0 tx=0).stop 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90d97af740 1 -- 172.21.15.18:0/91244833 shutdown_connections 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90d97af740 1 --2- 172.21.15.18:0/91244833 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x557256d3d840 0x557256d3fc80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90d97af740 1 --2- 172.21.15.18:0/91244833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557256cd5750 0x557256cdb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90d97af740 1 --2- 172.21.15.18:0/91244833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557256cccb70 0x557256cdade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90affff640 0 client.19419 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90d97af740 1 --2- 172.21.15.18:0/91244833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557256ccc250 0x557256cdca40 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90d97af740 1 -- 172.21.15.18:0/91244833 >> 172.21.15.18:0/91244833 conn(0x557256cc6980 msgr2=0x557256cc52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90d97af740 1 -- 172.21.15.18:0/91244833 shutdown_connections 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90d97af740 1 -- 172.21.15.18:0/91244833 wait complete. 2024-02-17T13:00:37.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.206+0000 7f90d97af740 20 client.19419 trim_cache size 0 max 16384 2024-02-17T13:00:37.203 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:00:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.207+0000 7f90d6afb640 1 -- 172.21.15.18:0/1606943711 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f90c00a2260 msgr2=0x7f90c00a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.207+0000 7f90d6afb640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f90c00a2260 0x7f90c00a4720 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f90c800fc90 tx=0x7f90c801e000 comp rx=0 tx=0).stop 2024-02-17T13:00:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.207+0000 7f90d6afb640 1 -- 172.21.15.18:0/1606943711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90d0105330 msgr2=0x7f90d0102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.207+0000 7f90d6afb640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90d0105330 0x7f90d0102230 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009230 tx=0x7f90bc02c680 comp rx=0 tx=0).stop 2024-02-17T13:00:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.207+0000 7f90d6afb640 1 -- 172.21.15.18:0/1606943711 shutdown_connections 2024-02-17T13:00:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.207+0000 7f90d6afb640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f90c00a2260 0x7f90c00a4720 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.207+0000 7f90d6afb640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d0102770 0x7f90d01a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.207+0000 7f90d6afb640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90d0105330 0x7f90d0102230 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.207+0000 7f90d6afb640 1 --2- 172.21.15.18:0/1606943711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d0100260 0x7f90d0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.207+0000 7f90d6afb640 1 -- 172.21.15.18:0/1606943711 >> 172.21.15.18:0/1606943711 conn(0x7f90d0077180 msgr2=0x7f90d0068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.207+0000 7f90d6afb640 1 -- 172.21.15.18:0/1606943711 shutdown_connections 2024-02-17T13:00:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.207+0000 7f90d6afb640 1 -- 172.21.15.18:0/1606943711 wait complete. 2024-02-17T13:00:37.207 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:00:37.208 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T13:00:37.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 538122 4.97 MB/sec execute 472 sec latency 2392.334 ms 2024-02-17T13:00:37.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.322+0000 7fba7a240640 1 -- 172.21.15.18:0/2719662052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba741064e0 msgr2=0x7fba7410e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.322+0000 7fba7a240640 1 --2- 172.21.15.18:0/2719662052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba741064e0 0x7fba7410e090 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009cd0 tx=0x7fba6c02c100 comp rx=0 tx=0).stop 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.322+0000 7fba7a240640 1 -- 172.21.15.18:0/2719662052 shutdown_connections 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.322+0000 7fba7a240640 1 --2- 172.21.15.18:0/2719662052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba741064e0 0x7fba7410e090 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.322+0000 7fba7a240640 1 --2- 172.21.15.18:0/2719662052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba7406a450 0x7fba74105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.322+0000 7fba7a240640 1 --2- 172.21.15.18:0/2719662052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba74069aa0 0x7fba74069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.322+0000 7fba7a240640 1 -- 172.21.15.18:0/2719662052 >> 172.21.15.18:0/2719662052 conn(0x7fba74077150 msgr2=0x7fba74077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.322+0000 7fba7a240640 1 -- 172.21.15.18:0/2719662052 shutdown_connections 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.322+0000 7fba7a240640 1 -- 172.21.15.18:0/2719662052 wait complete. 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.323+0000 7fba7a240640 1 Processor -- start 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.323+0000 7fba7a240640 1 -- start start 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.323+0000 7fba7a240640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba74069aa0 0x7fba7419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.323+0000 7fba7a240640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba741064e0 0x7fba7419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.323+0000 7fba7a240640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba74197600 0x7fba74197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.323+0000 7fba7a240640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba74197ff0 con 0x7fba741064e0 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.323+0000 7fba7a240640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba74198160 con 0x7fba74197600 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.323+0000 7fba7a240640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba74198320 con 0x7fba74069aa0 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.323+0000 7fba72ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba741064e0 0x7fba7419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.323+0000 7fba72ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba741064e0 0x7fba7419d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42230/0 (socket says 172.21.15.18:42230) 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.323+0000 7fba72ffd640 1 -- 172.21.15.18:0/3188134265 learned_addr learned my addr 172.21.15.18:0/3188134265 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba73fff640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba74197600 0x7fba74197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba737fe640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba74069aa0 0x7fba7419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba72ffd640 1 -- 172.21.15.18:0/3188134265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba74069aa0 msgr2=0x7fba7419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba72ffd640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba74069aa0 0x7fba7419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba72ffd640 1 -- 172.21.15.18:0/3188134265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba74197600 msgr2=0x7fba74197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba72ffd640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba74197600 0x7fba74197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba72ffd640 1 -- 172.21.15.18:0/3188134265 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c009900 con 0x7fba741064e0 2024-02-17T13:00:37.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba72ffd640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba741064e0 0x7fba7419d990 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fba68006fd0 tx=0x7fba6800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba70ff9640 1 -- 172.21.15.18:0/3188134265 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68011830 con 0x7fba741064e0 2024-02-17T13:00:37.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba70ff9640 1 -- 172.21.15.18:0/3188134265 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba68011e70 con 0x7fba741064e0 2024-02-17T13:00:37.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba70ff9640 1 -- 172.21.15.18:0/3188134265 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6800f2e0 con 0x7fba741064e0 2024-02-17T13:00:37.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba7a240640 1 -- 172.21.15.18:0/3188134265 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba741a4310 con 0x7fba741064e0 2024-02-17T13:00:37.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.324+0000 7fba7a240640 1 -- 172.21.15.18:0/3188134265 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba741a4820 con 0x7fba741064e0 2024-02-17T13:00:37.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.326+0000 7fba70ff9640 1 -- 172.21.15.18:0/3188134265 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fba68011990 con 0x7fba741064e0 2024-02-17T13:00:37.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba7ce11740 1 Processor -- start 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba7ce11740 1 -- start start 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba7ce11740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7250 0x5575ddde7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba7ce11740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7b70 0x5575dddf0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba7ce11740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575dddf0750 0x5575dddf2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba7ce11740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ddda2960 con 0x5575ddde7250 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba7ce11740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ddd56310 con 0x5575dddf0750 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba7ce11740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ddd53840 con 0x5575ddde7b70 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba70ff9640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fba5c0a2230 0x7fba5c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba73fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7250 0x5575ddde7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba73fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7250 0x5575ddde7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42232/0 (socket says 172.21.15.18:42232) 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba73fff640 1 -- 172.21.15.18:0/4292282961 learned_addr learned my addr 172.21.15.18:0/4292282961 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba73fff640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fba5c0a2230 0x7fba5c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba70ff9640 1 -- 172.21.15.18:0/3188134265 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fba680cd6f0 con 0x7fba741064e0 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba737fe640 1 --2- 172.21.15.18:0/4292282961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7b70 0x5575dddf0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba72ffd640 1 --2- 172.21.15.18:0/4292282961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575dddf0750 0x5575dddf2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba73fff640 1 -- 172.21.15.18:0/4292282961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7b70 msgr2=0x5575dddf0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba73fff640 1 --2- 172.21.15.18:0/4292282961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7b70 0x5575dddf0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba73fff640 1 -- 172.21.15.18:0/4292282961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575dddf0750 msgr2=0x5575dddf2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba73fff640 1 --2- 172.21.15.18:0/4292282961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575dddf0750 0x5575dddf2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.327+0000 7fba73fff640 1 -- 172.21.15.18:0/4292282961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575ddd11060 con 0x5575ddde7250 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.328+0000 7fba73fff640 1 --2- 172.21.15.18:0/4292282961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7250 0x5575ddde7630 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02c710 tx=0x7fba6c032890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a12632d9248c49 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.328+0000 7fba73fff640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fba5c0a2230 0x7fba5c0a46f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009cd0 tx=0x7fba6c04a000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.328+0000 7fba59ffb640 1 -- 172.21.15.18:0/4292282961 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c049070 con 0x5575ddde7250 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.328+0000 7fba59ffb640 1 -- 172.21.15.18:0/4292282961 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba6c003e50 con 0x5575ddde7250 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.328+0000 7fba59ffb640 1 -- 172.21.15.18:0/4292282961 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c004040 con 0x5575ddde7250 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.328+0000 7fba7ce11740 1 -- 172.21.15.18:0/4292282961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7250 msgr2=0x5575ddde7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.328+0000 7fba7ce11740 1 --2- 172.21.15.18:0/4292282961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7250 0x5575ddde7630 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02c710 tx=0x7fba6c032890 comp rx=0 tx=0).stop 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.328+0000 7fba7ce11740 1 -- 172.21.15.18:0/4292282961 shutdown_connections 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.328+0000 7fba7ce11740 1 --2- 172.21.15.18:0/4292282961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575dddf0750 0x5575dddf2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.328+0000 7fba7ce11740 1 --2- 172.21.15.18:0/4292282961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7b70 0x5575dddf0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.328+0000 7fba7ce11740 1 --2- 172.21.15.18:0/4292282961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7250 0x5575ddde7630 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.328+0000 7fba7ce11740 1 -- 172.21.15.18:0/4292282961 >> 172.21.15.18:0/4292282961 conn(0x5575ddde1980 msgr2=0x5575ddde1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba7ce11740 1 -- 172.21.15.18:0/4292282961 shutdown_connections 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba7ce11740 1 -- 172.21.15.18:0/4292282961 wait complete. 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba7ce11740 1 -- start start 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba7ce11740 1 Processor -- start 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba7ce11740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7250 0x5575ddded7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba7ce11740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7b70 0x5575dddedd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba7ce11740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575dddf0750 0x5575dddee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba7ce11740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ddd4f020 con 0x5575ddde7b70 2024-02-17T13:00:37.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba7ce11740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575dda22430 con 0x5575ddde7250 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba7ce11740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575dd9dee00 con 0x5575dddf0750 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba7ce11740 10 client.0 _ll_register_callbacks cb 0x5575ddcf6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba72ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7b70 0x5575dddedd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba72ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7b70 0x5575dddedd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42246/0 (socket says 172.21.15.18:42246) 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba72ffd640 1 -- 172.21.15.18:0/1307755452 learned_addr learned my addr 172.21.15.18:0/1307755452 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.329+0000 7fba73fff640 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575dddf0750 0x5575dddee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba72ffd640 1 -- 172.21.15.18:0/1307755452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575dddf0750 msgr2=0x5575dddee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba72ffd640 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575dddf0750 0x5575dddee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba72ffd640 1 -- 172.21.15.18:0/1307755452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7250 msgr2=0x5575ddded7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba737fe640 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7250 0x5575ddded7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba72ffd640 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7250 0x5575ddded7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba72ffd640 1 -- 172.21.15.18:0/1307755452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7410e8a0 con 0x5575ddde7b70 2024-02-17T13:00:37.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba72ffd640 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7b70 0x5575dddedd10 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fba68096be0 tx=0x7fba680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9486a4b3d5365da server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba52ffd640 1 -- 172.21.15.18:0/1307755452 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6809e360 con 0x5575ddde7b70 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba52ffd640 1 -- 172.21.15.18:0/1307755452 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fba680a29b0 con 0x5575ddde7b70 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba7ce11740 1 -- 172.21.15.18:0/1307755452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575dda4e7e0 con 0x5575ddde7b70 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba52ffd640 1 -- 172.21.15.18:0/1307755452 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba680105d0 con 0x5575ddde7b70 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba7ce11740 10 client.19431 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba7ce11740 1 -- 172.21.15.18:0/1307755452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5575ddd4d6a0 con 0x5575ddde7b70 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba52ffd640 1 -- 172.21.15.18:0/1307755452 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5575dda4e7e0 con 0x5575ddde7b70 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.330+0000 7fba52ffd640 1 -- 172.21.15.18:0/1307755452 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fba68099660 con 0x5575ddde7b70 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.331+0000 7fba7ce11740 10 client.19431 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.331+0000 7fba7ce11740 10 client.19431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.331+0000 7fba7ce11740 20 client.19431 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.331+0000 7fba7ce11740 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5575dde00ba0 0x5575dde02fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.331+0000 7fba7ce11740 4 client.19431 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.331+0000 7fba7ce11740 1 -- 172.21.15.18:0/1307755452 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575dda8b720 con 0x5575dde00ba0 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.331+0000 7fba737fe640 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5575dde00ba0 0x5575dde02fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.331+0000 7fba52ffd640 10 client.19431 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.331+0000 7fba737fe640 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5575dde00ba0 0x5575dde02fc0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8cd5f8fc349487 server_cookie=4ee2c79b59b39cf7 in_seq=0 out_seq=0 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba52ffd640 1 -- 172.21.15.18:0/1307755452 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5575dda8b720 con 0x5575dde00ba0 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba52ffd640 10 client.19431 handle_command_reply: tid=0 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba7ce11740 1 client.19431 shutdown 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba7ce11740 1 -- 172.21.15.18:0/1307755452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7b70 msgr2=0x5575dddedd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba7ce11740 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7b70 0x5575dddedd10 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fba68096be0 tx=0x7fba680046c0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba7ce11740 1 -- 172.21.15.18:0/1307755452 shutdown_connections 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba7ce11740 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5575dde00ba0 0x5575dde02fc0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba7ce11740 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575dddf0750 0x5575dddee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba7ce11740 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575ddde7b70 0x5575dddedd10 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba7ce11740 1 --2- 172.21.15.18:0/1307755452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7250 0x5575ddded7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba52ffd640 0 client.19431 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:37.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba7ce11740 1 -- 172.21.15.18:0/1307755452 >> 172.21.15.18:0/1307755452 conn(0x5575ddde1980 msgr2=0x5575ddde36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba7ce11740 1 -- 172.21.15.18:0/1307755452 shutdown_connections 2024-02-17T13:00:37.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba7ce11740 1 -- 172.21.15.18:0/1307755452 wait complete. 2024-02-17T13:00:37.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.332+0000 7fba7ce11740 20 client.19431 trim_cache size 0 max 16384 2024-02-17T13:00:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba7ce11740 1 Processor -- start 2024-02-17T13:00:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba7ce11740 1 -- start start 2024-02-17T13:00:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba7ce11740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7250 0x5575dddeff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba7ce11740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7b70 0x5575dddee070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba7ce11740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 0x5575dddeaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba7ce11740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ddcd4130 con 0x5575dddf0750 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba7ce11740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575dda4e7e0 con 0x5575ddde7b70 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba7ce11740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ddde79d0 con 0x5575ddde7250 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba73fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 0x5575dddeaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba73fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 0x5575dddeaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42254/0 (socket says 172.21.15.18:42254) 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba73fff640 1 -- 172.21.15.18:0/273029735 learned_addr learned my addr 172.21.15.18:0/273029735 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba72ffd640 1 --2- 172.21.15.18:0/273029735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7b70 0x5575dddee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba73fff640 1 -- 172.21.15.18:0/273029735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7250 msgr2=0x5575dddeff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba73fff640 1 --2- 172.21.15.18:0/273029735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7250 0x5575dddeff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba73fff640 1 -- 172.21.15.18:0/273029735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7b70 msgr2=0x5575dddee070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba73fff640 1 --2- 172.21.15.18:0/273029735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7b70 0x5575dddee070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba73fff640 1 -- 172.21.15.18:0/273029735 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575ddce9c20 con 0x5575dddf0750 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba72ffd640 1 --2- 172.21.15.18:0/273029735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7b70 0x5575dddee070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.344+0000 7fba73fff640 1 --2- 172.21.15.18:0/273029735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 0x5575dddeaa70 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02c640 tx=0x7fba6c004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff6d0a76a216cba server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba597fa640 1 -- 172.21.15.18:0/273029735 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c049050 con 0x5575dddf0750 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba597fa640 1 -- 172.21.15.18:0/273029735 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba6c034d90 con 0x5575dddf0750 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba597fa640 1 -- 172.21.15.18:0/273029735 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c004460 con 0x5575dddf0750 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba7ce11740 1 -- 172.21.15.18:0/273029735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 msgr2=0x5575dddeaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba7ce11740 1 --2- 172.21.15.18:0/273029735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 0x5575dddeaa70 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02c640 tx=0x7fba6c004cf0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba7ce11740 1 -- 172.21.15.18:0/273029735 shutdown_connections 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba7ce11740 1 --2- 172.21.15.18:0/273029735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 0x5575dddeaa70 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba7ce11740 1 --2- 172.21.15.18:0/273029735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7b70 0x5575dddee070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba7ce11740 1 --2- 172.21.15.18:0/273029735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7250 0x5575dddeff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba7ce11740 1 -- 172.21.15.18:0/273029735 >> 172.21.15.18:0/273029735 conn(0x5575ddde1980 msgr2=0x5575dddec0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba7ce11740 1 -- 172.21.15.18:0/273029735 shutdown_connections 2024-02-17T13:00:37.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba7ce11740 1 -- 172.21.15.18:0/273029735 wait complete. 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.345+0000 7fba7ce11740 1 -- start start 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba7ce11740 1 Processor -- start 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba7ce11740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7250 0x5575dddf6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba7ce11740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7b70 0x5575dddf6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba7ce11740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 0x5575dde570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba7ce11740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575dde57630 con 0x5575dddf0750 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba7ce11740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575dde57770 con 0x5575ddde7b70 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba7ce11740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575dde578b0 con 0x5575ddde7250 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba7ce11740 10 client.0 _ll_register_callbacks cb 0x7fba6c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba73fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 0x5575dde570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba73fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 0x5575dde570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42270/0 (socket says 172.21.15.18:42270) 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba73fff640 1 -- 172.21.15.18:0/2946967971 learned_addr learned my addr 172.21.15.18:0/2946967971 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:37.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba73fff640 1 -- 172.21.15.18:0/2946967971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7250 msgr2=0x5575dddf6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba73fff640 1 --2- 172.21.15.18:0/2946967971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7250 0x5575dddf6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba73fff640 1 -- 172.21.15.18:0/2946967971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7b70 msgr2=0x5575dddf6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba73fff640 1 --2- 172.21.15.18:0/2946967971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7b70 0x5575dddf6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba73fff640 1 -- 172.21.15.18:0/2946967971 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575ddd11060 con 0x5575dddf0750 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba73fff640 1 --2- 172.21.15.18:0/2946967971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 0x5575dde570f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02c6e0 tx=0x7fba6c032890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb58fa7e8ff31c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.346+0000 7fba52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba52ffd640 1 -- 172.21.15.18:0/2946967971 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c067070 con 0x5575dddf0750 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba52ffd640 1 -- 172.21.15.18:0/2946967971 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fba6c0042f0 con 0x5575dddf0750 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba7ce11740 1 -- 172.21.15.18:0/2946967971 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba6c003e50 con 0x5575dddf0750 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba52ffd640 1 -- 172.21.15.18:0/2946967971 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba6c049020 con 0x5575dddf0750 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba7ce11740 10 client.19437 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba7ce11740 1 -- 172.21.15.18:0/2946967971 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5575ddd0f8c0 con 0x5575dddf0750 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba52ffd640 1 -- 172.21.15.18:0/2946967971 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba6c003e50 con 0x5575dddf0750 2024-02-17T13:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba52ffd640 1 -- 172.21.15.18:0/2946967971 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fba6c06a020 con 0x5575dddf0750 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba7ce11740 10 client.19437 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba7ce11740 10 client.19437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba7ce11740 20 client.19437 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba7ce11740 1 --2- 172.21.15.18:0/2946967971 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5575dde58bd0 0x5575dde5b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba7ce11740 4 client.19437 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba7ce11740 1 -- 172.21.15.18:0/2946967971 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5575ddada890 con 0x5575dde58bd0 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.347+0000 7fba737fe640 1 --2- 172.21.15.18:0/2946967971 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5575dde58bd0 0x5575dde5b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba52ffd640 10 client.19437 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba737fe640 1 --2- 172.21.15.18:0/2946967971 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5575dde58bd0 0x5575dde5b000 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abbc9499a4903e38 server_cookie=b98acef22a08b51c in_seq=0 out_seq=0 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba52ffd640 1 -- 172.21.15.18:0/2946967971 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5575ddada890 con 0x5575dde58bd0 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba52ffd640 10 client.19437 handle_command_reply: tid=0 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba7ce11740 1 client.19437 shutdown 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba7ce11740 1 -- 172.21.15.18:0/2946967971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 msgr2=0x5575dde570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba7ce11740 1 --2- 172.21.15.18:0/2946967971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 0x5575dde570f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02c6e0 tx=0x7fba6c032890 comp rx=0 tx=0).stop 2024-02-17T13:00:37.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba7ce11740 1 -- 172.21.15.18:0/2946967971 shutdown_connections 2024-02-17T13:00:37.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba7ce11740 1 --2- 172.21.15.18:0/2946967971 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5575dde58bd0 0x5575dde5b000 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba7ce11740 1 --2- 172.21.15.18:0/2946967971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575dddf0750 0x5575dde570f0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba52ffd640 0 client.19437 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:37.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba7ce11740 1 --2- 172.21.15.18:0/2946967971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575ddde7b70 0x5575dddf6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba7ce11740 1 --2- 172.21.15.18:0/2946967971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5575ddde7250 0x5575dddf6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba7ce11740 1 -- 172.21.15.18:0/2946967971 >> 172.21.15.18:0/2946967971 conn(0x5575ddde1980 msgr2=0x5575ddde02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba7ce11740 1 -- 172.21.15.18:0/2946967971 shutdown_connections 2024-02-17T13:00:37.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba7ce11740 1 -- 172.21.15.18:0/2946967971 wait complete. 2024-02-17T13:00:37.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.348+0000 7fba7ce11740 20 client.19437 trim_cache size 0 max 16384 2024-02-17T13:00:37.346 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:00:37.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.349+0000 7fba7a240640 1 -- 172.21.15.18:0/3188134265 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fba5c0a2230 msgr2=0x7fba5c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.349+0000 7fba7a240640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fba5c0a2230 0x7fba5c0a46f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009cd0 tx=0x7fba6c04a000 comp rx=0 tx=0).stop 2024-02-17T13:00:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.349+0000 7fba7a240640 1 -- 172.21.15.18:0/3188134265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba741064e0 msgr2=0x7fba7419d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.349+0000 7fba7a240640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba741064e0 0x7fba7419d990 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fba68006fd0 tx=0x7fba6800b8e0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.349+0000 7fba7a240640 1 -- 172.21.15.18:0/3188134265 shutdown_connections 2024-02-17T13:00:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.349+0000 7fba7a240640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fba5c0a2230 0x7fba5c0a46f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.349+0000 7fba7a240640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba74197600 0x7fba74197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.349+0000 7fba7a240640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba741064e0 0x7fba7419d990 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.349+0000 7fba7a240640 1 --2- 172.21.15.18:0/3188134265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba74069aa0 0x7fba7419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.349+0000 7fba7a240640 1 -- 172.21.15.18:0/3188134265 >> 172.21.15.18:0/3188134265 conn(0x7fba74077150 msgr2=0x7fba740fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.349+0000 7fba7a240640 1 -- 172.21.15.18:0/3188134265 shutdown_connections 2024-02-17T13:00:37.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:37.349+0000 7fba7a240640 1 -- 172.21.15.18:0/3188134265 wait complete. 2024-02-17T13:00:38.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 538122 4.96 MB/sec execute 473 sec latency 3392.444 ms 2024-02-17T13:00:38.350 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T13:00:38.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:00:38.350 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T13:00:38.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.481+0000 7fb8b3edb640 1 -- 172.21.15.18:0/2864290163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8ac1023f0 msgr2=0x7fb8ac1027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.481+0000 7fb8b3edb640 1 --2- 172.21.15.18:0/2864290163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8ac1023f0 0x7fb8ac1027d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009860 tx=0x7fb8a802bc00 comp rx=0 tx=0).stop 2024-02-17T13:00:38.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.481+0000 7fb8b3edb640 1 -- 172.21.15.18:0/2864290163 shutdown_connections 2024-02-17T13:00:38.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.481+0000 7fb8b3edb640 1 --2- 172.21.15.18:0/2864290163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ac10b7c0 0x7fb8ac10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.481+0000 7fb8b3edb640 1 --2- 172.21.15.18:0/2864290163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8ac102d10 0x7fb8ac10b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.481+0000 7fb8b3edb640 1 --2- 172.21.15.18:0/2864290163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8ac1023f0 0x7fb8ac1027d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.481+0000 7fb8b3edb640 1 -- 172.21.15.18:0/2864290163 >> 172.21.15.18:0/2864290163 conn(0x7fb8ac0771e0 msgr2=0x7fb8ac0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:38.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.481+0000 7fb8b3edb640 1 -- 172.21.15.18:0/2864290163 shutdown_connections 2024-02-17T13:00:38.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.481+0000 7fb8b3edb640 1 -- 172.21.15.18:0/2864290163 wait complete. 2024-02-17T13:00:38.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.481+0000 7fb8b3edb640 1 Processor -- start 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.481+0000 7fb8b3edb640 1 -- start start 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b3edb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ac1023f0 0x7fb8ac19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b3edb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8ac102d10 0x7fb8ac19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b3edb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8ac10b7c0 0x7fb8ac197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b3edb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac197e20 con 0x7fb8ac10b7c0 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b3edb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac197f90 con 0x7fb8ac1023f0 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b3edb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac198150 con 0x7fb8ac102d10 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b2478640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8ac10b7c0 0x7fb8ac197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b2478640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8ac10b7c0 0x7fb8ac197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42290/0 (socket says 172.21.15.18:42290) 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b2478640 1 -- 172.21.15.18:0/3020254018 learned_addr learned my addr 172.21.15.18:0/3020254018 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b1c77640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ac1023f0 0x7fb8ac19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b1476640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8ac102d10 0x7fb8ac19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b2478640 1 -- 172.21.15.18:0/3020254018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8ac102d10 msgr2=0x7fb8ac19db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b2478640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8ac102d10 0x7fb8ac19db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b2478640 1 -- 172.21.15.18:0/3020254018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ac1023f0 msgr2=0x7fb8ac19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b2478640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ac1023f0 0x7fb8ac19d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b2478640 1 -- 172.21.15.18:0/3020254018 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c009560 con 0x7fb8ac10b7c0 2024-02-17T13:00:38.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8b2478640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8ac10b7c0 0x7fb8ac197690 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400eb10 tx=0x7fb8a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:38.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.482+0000 7fb8a2ffd640 1 -- 172.21.15.18:0/3020254018 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4004b20 con 0x7fb8ac10b7c0 2024-02-17T13:00:38.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.483+0000 7fb8b3edb640 1 -- 172.21.15.18:0/3020254018 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a8009490 con 0x7fb8ac10b7c0 2024-02-17T13:00:38.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.483+0000 7fb8a2ffd640 1 -- 172.21.15.18:0/3020254018 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8a4004c80 con 0x7fb8ac10b7c0 2024-02-17T13:00:38.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.483+0000 7fb8a2ffd640 1 -- 172.21.15.18:0/3020254018 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4010650 con 0x7fb8ac10b7c0 2024-02-17T13:00:38.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.483+0000 7fb8b3edb640 1 -- 172.21.15.18:0/3020254018 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ac1a4550 con 0x7fb8ac10b7c0 2024-02-17T13:00:38.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.485+0000 7fb8a2ffd640 1 -- 172.21.15.18:0/3020254018 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb8a4005020 con 0x7fb8ac10b7c0 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.485+0000 7fb8b6b83740 1 Processor -- start 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.485+0000 7fb8b6b83740 1 -- start start 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.485+0000 7fb8b6b83740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2380 0x556be9da2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.485+0000 7fb8b6b83740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 0x556be9dab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.485+0000 7fb8b6b83740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9dab8b0 0x556be9dadd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.485+0000 7fb8b6b83740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9d10f90 con 0x556be9da2d30 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.485+0000 7fb8b6b83740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9d0e4c0 con 0x556be9dab8b0 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.485+0000 7fb8b6b83740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9d09ca0 con 0x556be9da2380 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b1c77640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 0x556be9dab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8a2ffd640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb8980a2230 0x7fb8980a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b1c77640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 0x556be9dab2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42296/0 (socket says 172.21.15.18:42296) 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b1c77640 1 -- 172.21.15.18:0/1057873623 learned_addr learned my addr 172.21.15.18:0/1057873623 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:38.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b1476640 1 --2- 172.21.15.18:0/1057873623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2380 0x556be9da2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b1476640 1 --2- 172.21.15.18:0/1057873623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9dab8b0 0x556be9dadd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b2478640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb8980a2230 0x7fb8980a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b1c77640 1 -- 172.21.15.18:0/1057873623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2380 msgr2=0x556be9da2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b1c77640 1 --2- 172.21.15.18:0/1057873623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2380 0x556be9da2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b1c77640 1 -- 172.21.15.18:0/1057873623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9dab8b0 msgr2=0x556be9dadd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b1c77640 1 --2- 172.21.15.18:0/1057873623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9dab8b0 0x556be9dadd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b1c77640 1 -- 172.21.15.18:0/1057873623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556be9ccc0b0 con 0x556be9da2d30 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8a2ffd640 1 -- 172.21.15.18:0/3020254018 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb8a4014070 con 0x7fb8ac10b7c0 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b1476640 1 --2- 172.21.15.18:0/1057873623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9dab8b0 0x556be9dadd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b1c77640 1 --2- 172.21.15.18:0/1057873623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 0x556be9dab2e0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8005e00 tx=0x7fb8a80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed610aa746a7f41 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb8b2478640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb8980a2230 0x7fb8980a46f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40cbde0 tx=0x7fb8a400cf70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb897fff640 1 -- 172.21.15.18:0/1057873623 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8004370 con 0x556be9da2d30 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb897fff640 1 -- 172.21.15.18:0/1057873623 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8a80044d0 con 0x556be9da2d30 2024-02-17T13:00:38.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.486+0000 7fb897fff640 1 -- 172.21.15.18:0/1057873623 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a802cc80 con 0x556be9da2d30 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.487+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1057873623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 msgr2=0x556be9dab2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.487+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/1057873623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 0x556be9dab2e0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8005e00 tx=0x7fb8a80040c0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.487+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1057873623 shutdown_connections 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.487+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/1057873623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9dab8b0 0x556be9dadd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.487+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/1057873623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 0x556be9dab2e0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.487+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/1057873623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2380 0x556be9da2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.487+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1057873623 >> 172.21.15.18:0/1057873623 conn(0x556be9d9c840 msgr2=0x556be9d9cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.487+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1057873623 shutdown_connections 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.487+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1057873623 wait complete. 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b6b83740 1 -- start start 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b6b83740 1 Processor -- start 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b6b83740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9da2380 0x556be9da8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b6b83740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 0x556be9da8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b6b83740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9dab8b0 0x556be9da9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b6b83740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9caef40 con 0x556be9da2d30 2024-02-17T13:00:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b6b83740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be99dd4b0 con 0x556be9da2380 2024-02-17T13:00:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b6b83740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9c8f670 con 0x556be9dab8b0 2024-02-17T13:00:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b6b83740 10 client.0 _ll_register_callbacks cb 0x556be9cfcf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b1c77640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 0x556be9da8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b1c77640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 0x556be9da8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42308/0 (socket says 172.21.15.18:42308) 2024-02-17T13:00:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b1c77640 1 -- 172.21.15.18:0/1269974861 learned_addr learned my addr 172.21.15.18:0/1269974861 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b1476640 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9dab8b0 0x556be9da9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.488+0000 7fb8b1476640 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9da2380 0x556be9da8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8b1c77640 1 -- 172.21.15.18:0/1269974861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9dab8b0 msgr2=0x556be9da9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8b1c77640 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9dab8b0 0x556be9da9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8b1c77640 1 -- 172.21.15.18:0/1269974861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9da2380 msgr2=0x556be9da8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8b1c77640 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9da2380 0x556be9da8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8b1c77640 1 -- 172.21.15.18:0/1269974861 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ac10e8b0 con 0x556be9da2d30 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8b1c77640 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 0x556be9da8cd0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fb8a802c140 tx=0x7fb8a802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04c8914a048e60c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8957fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8957fa640 1 -- 172.21.15.18:0/1269974861 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a802cc80 con 0x556be9da2d30 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8957fa640 1 -- 172.21.15.18:0/1269974861 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb8a8034650 con 0x556be9da2d30 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1269974861 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556be9ce3f40 con 0x556be9da2d30 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8957fa640 1 -- 172.21.15.18:0/1269974861 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8a8034830 con 0x556be9da2d30 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8b6b83740 10 client.19449 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1269974861 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x556be9d08320 con 0x556be9da2d30 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8957fa640 1 -- 172.21.15.18:0/1269974861 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8a805d020 con 0x556be9da2d30 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.489+0000 7fb8957fa640 1 -- 172.21.15.18:0/1269974861 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb8a803f480 con 0x556be9da2d30 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.490+0000 7fb8b6b83740 10 client.19449 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:38.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.490+0000 7fb8b6b83740 10 client.19449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:38.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.490+0000 7fb8b6b83740 20 client.19449 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:38.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.490+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556be9dbbb10 0x556be9dbdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.490+0000 7fb8b6b83740 4 client.19449 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:38.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.490+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1269974861 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556be9d00310 con 0x556be9dbbb10 2024-02-17T13:00:38.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.490+0000 7fb8b1476640 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556be9dbbb10 0x556be9dbdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.490+0000 7fb8957fa640 10 client.19449 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:38.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.490+0000 7fb8b1476640 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556be9dbbb10 0x556be9dbdf30 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f3c5c704af56ef server_cookie=b81ee728093b139d in_seq=0 out_seq=0 2024-02-17T13:00:38.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.492+0000 7fb8957fa640 1 -- 172.21.15.18:0/1269974861 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556be9d00310 con 0x556be9dbbb10 2024-02-17T13:00:38.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.492+0000 7fb8957fa640 10 client.19449 handle_command_reply: tid=0 2024-02-17T13:00:38.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.492+0000 7fb8b6b83740 1 client.19449 shutdown 2024-02-17T13:00:38.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.493+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1269974861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 msgr2=0x556be9da8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.493+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 0x556be9da8cd0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fb8a802c140 tx=0x7fb8a802c850 comp rx=0 tx=0).stop 2024-02-17T13:00:38.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.493+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1269974861 shutdown_connections 2024-02-17T13:00:38.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.493+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556be9dbbb10 0x556be9dbdf30 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.493+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9dab8b0 0x556be9da9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.493+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2d30 0x556be9da8cd0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.493+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/1269974861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9da2380 0x556be9da8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.493+0000 7fb8957fa640 0 client.19449 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:38.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.493+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1269974861 >> 172.21.15.18:0/1269974861 conn(0x556be9d9c840 msgr2=0x556be9d9e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:38.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.493+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1269974861 shutdown_connections 2024-02-17T13:00:38.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.493+0000 7fb8b6b83740 1 -- 172.21.15.18:0/1269974861 wait complete. 2024-02-17T13:00:38.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.493+0000 7fb8b6b83740 20 client.19449 trim_cache size 0 max 16384 2024-02-17T13:00:38.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b6b83740 1 Processor -- start 2024-02-17T13:00:38.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b6b83740 1 -- start start 2024-02-17T13:00:38.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b6b83740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2380 0x556be9da9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b6b83740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2d30 0x556be9d9e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b6b83740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9dab8b0 0x556be9db28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b6b83740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9d9dd20 con 0x556be9da2380 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b6b83740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9d9c090 con 0x556be9dab8b0 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b6b83740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9d9c6f0 con 0x556be9da2d30 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b1476640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2380 0x556be9da9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b1476640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2380 0x556be9da9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42314/0 (socket says 172.21.15.18:42314) 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b1476640 1 -- 172.21.15.18:0/2903182113 learned_addr learned my addr 172.21.15.18:0/2903182113 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b1c77640 1 --2- 172.21.15.18:0/2903182113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2d30 0x556be9d9e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b1476640 1 --2- 172.21.15.18:0/2903182113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9dab8b0 0x556be9db28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b1476640 1 -- 172.21.15.18:0/2903182113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2d30 msgr2=0x556be9d9e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b1476640 1 --2- 172.21.15.18:0/2903182113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2d30 0x556be9d9e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b1476640 1 -- 172.21.15.18:0/2903182113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9dab8b0 msgr2=0x556be9db28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b1476640 1 --2- 172.21.15.18:0/2903182113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9dab8b0 0x556be9db28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.509+0000 7fb8b1476640 1 -- 172.21.15.18:0/2903182113 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556be9ca4c00 con 0x556be9da2380 2024-02-17T13:00:38.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.510+0000 7fb8b1476640 1 --2- 172.21.15.18:0/2903182113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2380 0x556be9da9030 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fb89c011220 tx=0x7fb89c028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e9e7d79f441512 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.510+0000 7fb8977fe640 1 -- 172.21.15.18:0/2903182113 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c02b070 con 0x556be9da2380 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.510+0000 7fb8977fe640 1 -- 172.21.15.18:0/2903182113 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb89c026e90 con 0x556be9da2380 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.510+0000 7fb8977fe640 1 -- 172.21.15.18:0/2903182113 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c016040 con 0x556be9da2380 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.510+0000 7fb8b6b83740 1 -- 172.21.15.18:0/2903182113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2380 msgr2=0x556be9da9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.510+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/2903182113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2380 0x556be9da9030 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fb89c011220 tx=0x7fb89c028040 comp rx=0 tx=0).stop 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.510+0000 7fb8b6b83740 1 -- 172.21.15.18:0/2903182113 shutdown_connections 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.510+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/2903182113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9dab8b0 0x556be9db28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.510+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/2903182113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2d30 0x556be9d9e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.510+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/2903182113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9da2380 0x556be9da9030 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.510+0000 7fb8b6b83740 1 -- 172.21.15.18:0/2903182113 >> 172.21.15.18:0/2903182113 conn(0x556be9d9c840 msgr2=0x556be9d9de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b6b83740 1 -- 172.21.15.18:0/2903182113 shutdown_connections 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b6b83740 1 -- 172.21.15.18:0/2903182113 wait complete. 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b6b83740 1 -- start start 2024-02-17T13:00:38.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b6b83740 1 Processor -- start 2024-02-17T13:00:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b6b83740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2380 0x556be9e0c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b6b83740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9da2d30 0x556be9e0ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b6b83740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9dab8b0 0x556be9db1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b6b83740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9db25a0 con 0x556be9dab8b0 2024-02-17T13:00:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b6b83740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9db2710 con 0x556be9da2d30 2024-02-17T13:00:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b6b83740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556be9db28d0 con 0x556be9da2380 2024-02-17T13:00:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b6b83740 10 client.0 _ll_register_callbacks cb 0x7fb8a8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b1476640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9dab8b0 0x556be9db1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b1476640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9dab8b0 0x556be9db1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42324/0 (socket says 172.21.15.18:42324) 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.511+0000 7fb8b1476640 1 -- 172.21.15.18:0/3111988098 learned_addr learned my addr 172.21.15.18:0/3111988098 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8b1c77640 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9da2d30 0x556be9e0ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8b1476640 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2380 0x556be9e0c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8b1476640 1 -- 172.21.15.18:0/3111988098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2380 msgr2=0x556be9e0c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8b1476640 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2380 0x556be9e0c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8b1476640 1 -- 172.21.15.18:0/3111988098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9da2d30 msgr2=0x556be9e0ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8b1476640 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9da2d30 0x556be9e0ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8b1476640 1 -- 172.21.15.18:0/3111988098 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c019040 con 0x556be9dab8b0 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8b1476640 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9dab8b0 0x556be9db1f90 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009310 tx=0x7fb89c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5122916a25d6d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8957fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8957fa640 1 -- 172.21.15.18:0/3111988098 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c02b070 con 0x556be9dab8b0 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8957fa640 1 -- 172.21.15.18:0/3111988098 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb89c023660 con 0x556be9dab8b0 2024-02-17T13:00:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8b6b83740 1 -- 172.21.15.18:0/3111988098 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556be9d2f900 con 0x556be9dab8b0 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8957fa640 1 -- 172.21.15.18:0/3111988098 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb89c016040 con 0x556be9dab8b0 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8b6b83740 10 client.19455 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8b6b83740 1 -- 172.21.15.18:0/3111988098 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x556be9cca910 con 0x556be9dab8b0 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.512+0000 7fb8957fa640 1 -- 172.21.15.18:0/3111988098 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556be9d2f900 con 0x556be9dab8b0 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.513+0000 7fb8957fa640 1 -- 172.21.15.18:0/3111988098 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb89c03f850 con 0x556be9dab8b0 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.513+0000 7fb8b6b83740 10 client.19455 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.513+0000 7fb8b6b83740 10 client.19455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.513+0000 7fb8b6b83740 20 client.19455 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.513+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556be9e14680 0x556be9e16aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.513+0000 7fb8b6b83740 4 client.19455 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.513+0000 7fb8b6b83740 1 -- 172.21.15.18:0/3111988098 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556be9a09860 con 0x556be9e14680 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.513+0000 7fb8b1c77640 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556be9e14680 0x556be9e16aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.513+0000 7fb8957fa640 10 client.19455 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:38.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.513+0000 7fb8b1c77640 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556be9e14680 0x556be9e16aa0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b41d9ded3862f519 server_cookie=e116e67748913c6b in_seq=0 out_seq=0 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8957fa640 1 -- 172.21.15.18:0/3111988098 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556be9a09860 con 0x556be9e14680 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8957fa640 10 client.19455 handle_command_reply: tid=0 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b6b83740 1 client.19455 shutdown 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b6b83740 1 -- 172.21.15.18:0/3111988098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9dab8b0 msgr2=0x556be9db1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9dab8b0 0x556be9db1f90 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009310 tx=0x7fb89c004730 comp rx=0 tx=0).stop 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b6b83740 1 -- 172.21.15.18:0/3111988098 shutdown_connections 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556be9e14680 0x556be9e16aa0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556be9dab8b0 0x556be9db1f90 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556be9da2d30 0x556be9e0ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b6b83740 1 --2- 172.21.15.18:0/3111988098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556be9da2380 0x556be9e0c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8957fa640 0 client.19455 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b6b83740 1 -- 172.21.15.18:0/3111988098 >> 172.21.15.18:0/3111988098 conn(0x556be9d9c840 msgr2=0x556be9d9e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b6b83740 1 -- 172.21.15.18:0/3111988098 shutdown_connections 2024-02-17T13:00:38.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b6b83740 1 -- 172.21.15.18:0/3111988098 wait complete. 2024-02-17T13:00:38.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b6b83740 20 client.19455 trim_cache size 0 max 16384 2024-02-17T13:00:38.513 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:00:38.513 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T13:00:38.513 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "ab7eec6e-2fca-46f5-832e-cc72edca2775", 2024-02-17T13:00:38.513 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T13:00:38.513 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:00:38.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b3edb640 1 -- 172.21.15.18:0/3020254018 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb8980a2230 msgr2=0x7fb8980a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b3edb640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb8980a2230 0x7fb8980a46f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40cbde0 tx=0x7fb8a400cf70 comp rx=0 tx=0).stop 2024-02-17T13:00:38.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b3edb640 1 -- 172.21.15.18:0/3020254018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8ac10b7c0 msgr2=0x7fb8ac197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.516+0000 7fb8b3edb640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8ac10b7c0 0x7fb8ac197690 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400eb10 tx=0x7fb8a400c2d0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.517+0000 7fb8b3edb640 1 -- 172.21.15.18:0/3020254018 shutdown_connections 2024-02-17T13:00:38.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.517+0000 7fb8b3edb640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb8980a2230 0x7fb8980a46f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.517+0000 7fb8b3edb640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8ac10b7c0 0x7fb8ac197690 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.517+0000 7fb8b3edb640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8ac102d10 0x7fb8ac19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.517+0000 7fb8b3edb640 1 --2- 172.21.15.18:0/3020254018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ac1023f0 0x7fb8ac19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.517+0000 7fb8b3edb640 1 -- 172.21.15.18:0/3020254018 >> 172.21.15.18:0/3020254018 conn(0x7fb8ac0771e0 msgr2=0x7fb8ac0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:38.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.517+0000 7fb8b3edb640 1 -- 172.21.15.18:0/3020254018 shutdown_connections 2024-02-17T13:00:38.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.517+0000 7fb8b3edb640 1 -- 172.21.15.18:0/3020254018 wait complete. 2024-02-17T13:00:38.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:00:38.518 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:00:38.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.637+0000 7f3a1c779640 1 -- 172.21.15.18:0/173888090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a14101370 msgr2=0x7f3a14101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.637+0000 7f3a1c779640 1 --2- 172.21.15.18:0/173888090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a14101370 0x7f3a14101750 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009930 tx=0x7f3a1002bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:38.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.637+0000 7f3a1c779640 1 -- 172.21.15.18:0/173888090 shutdown_connections 2024-02-17T13:00:38.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.637+0000 7f3a1c779640 1 --2- 172.21.15.18:0/173888090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a14106440 0x7f3a1410e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.637+0000 7f3a1c779640 1 --2- 172.21.15.18:0/173888090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a14101d20 0x7f3a14105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.637+0000 7f3a1c779640 1 --2- 172.21.15.18:0/173888090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a14101370 0x7f3a14101750 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.637+0000 7f3a1c779640 1 -- 172.21.15.18:0/173888090 >> 172.21.15.18:0/173888090 conn(0x7f3a140fcc30 msgr2=0x7f3a140ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.637+0000 7f3a1c779640 1 -- 172.21.15.18:0/173888090 shutdown_connections 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.637+0000 7f3a1c779640 1 -- 172.21.15.18:0/173888090 wait complete. 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a1c779640 1 Processor -- start 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a1c779640 1 -- start start 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a1c779640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a14101370 0x7f3a1419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a1c779640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a14106440 0x7f3a1419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a1c779640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a14197620 0x7f3a14197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a1c779640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14198010 con 0x7f3a14106440 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a1c779640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14198180 con 0x7f3a14197620 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a1c779640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14198340 con 0x7f3a14101370 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a19d14640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a14106440 0x7f3a1419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a19d14640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a14106440 0x7f3a1419d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42334/0 (socket says 172.21.15.18:42334) 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a19d14640 1 -- 172.21.15.18:0/960912491 learned_addr learned my addr 172.21.15.18:0/960912491 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a1a515640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a14101370 0x7f3a1419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a1ad16640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a14197620 0x7f3a14197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a19d14640 1 -- 172.21.15.18:0/960912491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a14101370 msgr2=0x7f3a1419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a19d14640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a14101370 0x7f3a1419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a19d14640 1 -- 172.21.15.18:0/960912491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a14197620 msgr2=0x7f3a14197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a19d14640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a14197620 0x7f3a14197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a19d14640 1 -- 172.21.15.18:0/960912491 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10009560 con 0x7f3a14106440 2024-02-17T13:00:38.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.638+0000 7f3a19d14640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a14106440 0x7f3a1419d9b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400d740 tx=0x7f3a0400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:38.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.639+0000 7f3a0b7fe640 1 -- 172.21.15.18:0/960912491 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04009c30 con 0x7f3a14106440 2024-02-17T13:00:38.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.639+0000 7f3a0b7fe640 1 -- 172.21.15.18:0/960912491 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a0400de50 con 0x7f3a14106440 2024-02-17T13:00:38.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.639+0000 7f3a1c779640 1 -- 172.21.15.18:0/960912491 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a141a4330 con 0x7f3a14106440 2024-02-17T13:00:38.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.639+0000 7f3a0b7fe640 1 -- 172.21.15.18:0/960912491 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04010460 con 0x7f3a14106440 2024-02-17T13:00:38.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.639+0000 7f3a1c779640 1 -- 172.21.15.18:0/960912491 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a141a47f0 con 0x7f3a14106440 2024-02-17T13:00:38.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.641+0000 7f3a0b7fe640 1 -- 172.21.15.18:0/960912491 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3a04004f10 con 0x7f3a14106440 2024-02-17T13:00:38.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.641+0000 7f3a1f420740 1 Processor -- start 2024-02-17T13:00:38.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.641+0000 7f3a1f420740 1 -- start start 2024-02-17T13:00:38.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1f420740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccbf60 0x5595a7ccc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1f420740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccc880 0x5595a7cd9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1f420740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7cd9bc0 0x5595a7cdbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1f420740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a7c3b280 con 0x5595a7ccbf60 2024-02-17T13:00:38.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1f420740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a7c387b0 con 0x5595a7ccc880 2024-02-17T13:00:38.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1f420740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a7c33f90 con 0x5595a7cd9bc0 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a0b7fe640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3a000a2230 0x7f3a000a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1ad16640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccbf60 0x5595a7ccc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1ad16640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccbf60 0x5595a7ccc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42340/0 (socket says 172.21.15.18:42340) 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1ad16640 1 -- 172.21.15.18:0/3676497293 learned_addr learned my addr 172.21.15.18:0/3676497293 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1ad16640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3a000a2230 0x7f3a000a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1a515640 1 --2- 172.21.15.18:0/3676497293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccc880 0x5595a7cd9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a19d14640 1 --2- 172.21.15.18:0/3676497293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7cd9bc0 0x5595a7cdbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a0b7fe640 1 -- 172.21.15.18:0/960912491 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3a040cd9e0 con 0x7f3a14106440 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1ad16640 1 -- 172.21.15.18:0/3676497293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7cd9bc0 msgr2=0x5595a7cdbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1ad16640 1 --2- 172.21.15.18:0/3676497293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7cd9bc0 0x5595a7cdbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1ad16640 1 -- 172.21.15.18:0/3676497293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccc880 msgr2=0x5595a7cd9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1ad16640 1 --2- 172.21.15.18:0/3676497293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccc880 0x5595a7cd9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1ad16640 1 -- 172.21.15.18:0/3676497293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595a7bf2570 con 0x5595a7ccbf60 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1ad16640 1 --2- 172.21.15.18:0/3676497293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccbf60 0x5595a7ccc340 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c011da0 tx=0x7f3a0c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86542029e0f268e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:38.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.642+0000 7f3a1ad16640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3a000a2230 0x7f3a000a46f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00fc90 tx=0x7f3a0c01e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.643+0000 7f3a08ff9640 1 -- 172.21.15.18:0/3676497293 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c01dbf0 con 0x5595a7ccbf60 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.643+0000 7f3a08ff9640 1 -- 172.21.15.18:0/3676497293 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c009dc0 con 0x5595a7ccbf60 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.643+0000 7f3a08ff9640 1 -- 172.21.15.18:0/3676497293 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0042b0 con 0x5595a7ccbf60 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.643+0000 7f3a1f420740 1 -- 172.21.15.18:0/3676497293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccbf60 msgr2=0x5595a7ccc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.643+0000 7f3a1f420740 1 --2- 172.21.15.18:0/3676497293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccbf60 0x5595a7ccc340 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c011da0 tx=0x7f3a0c00f790 comp rx=0 tx=0).stop 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.643+0000 7f3a1f420740 1 -- 172.21.15.18:0/3676497293 shutdown_connections 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.643+0000 7f3a1f420740 1 --2- 172.21.15.18:0/3676497293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7cd9bc0 0x5595a7cdbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.643+0000 7f3a1f420740 1 --2- 172.21.15.18:0/3676497293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccc880 0x5595a7cd9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.643+0000 7f3a1f420740 1 --2- 172.21.15.18:0/3676497293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccbf60 0x5595a7ccc340 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.643+0000 7f3a1f420740 1 -- 172.21.15.18:0/3676497293 >> 172.21.15.18:0/3676497293 conn(0x5595a7cc6990 msgr2=0x5595a7cc6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.643+0000 7f3a1f420740 1 -- 172.21.15.18:0/3676497293 shutdown_connections 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.643+0000 7f3a1f420740 1 -- 172.21.15.18:0/3676497293 wait complete. 2024-02-17T13:00:38.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1f420740 1 -- start start 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1f420740 1 Processor -- start 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1f420740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 0x5595a7cd95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1f420740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7ccc880 0x5595a7cd6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1f420740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 0x5595a7cd6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1f420740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a7907430 con 0x5595a7cd9bc0 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1f420740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a78c3e00 con 0x5595a7ccbf60 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1f420740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a7bb8fd0 con 0x5595a7ccc880 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1f420740 10 client.0 _ll_register_callbacks cb 0x5595a7c0efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1ad16640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 0x5595a7cd6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1a515640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 0x5595a7cd95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1ad16640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 0x5595a7cd6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42344/0 (socket says 172.21.15.18:42344) 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a19d14640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7ccc880 0x5595a7cd6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1a515640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 0x5595a7cd95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:33938/0 (socket says 172.21.15.18:33938) 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1ad16640 1 -- 172.21.15.18:0/772270590 learned_addr learned my addr 172.21.15.18:0/772270590 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1ad16640 1 -- 172.21.15.18:0/772270590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7ccc880 msgr2=0x5595a7cd6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1ad16640 1 --2- 172.21.15.18:0/772270590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7ccc880 0x5595a7cd6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1ad16640 1 -- 172.21.15.18:0/772270590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 msgr2=0x5595a7cd95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1ad16640 1 --2- 172.21.15.18:0/772270590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 0x5595a7cd95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.644+0000 7f3a1ad16640 1 -- 172.21.15.18:0/772270590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c019040 con 0x5595a7cd9bc0 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f3a1ad16640 1 --2- 172.21.15.18:0/772270590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 0x5595a7cd6f50 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0099b0 tx=0x7f3a0c003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a5d7a4474cf2cb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f39fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f39fdffb640 1 -- 172.21.15.18:0/772270590 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c011910 con 0x5595a7cd9bc0 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f39fdffb640 1 -- 172.21.15.18:0/772270590 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3a0c01ddd0 con 0x5595a7cd9bc0 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f3a1f420740 1 -- 172.21.15.18:0/772270590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595a79337e0 con 0x5595a7cd9bc0 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f39fdffb640 1 -- 172.21.15.18:0/772270590 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a0c005050 con 0x5595a7cd9bc0 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f3a1f420740 10 client.19467 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f3a1f420740 1 -- 172.21.15.18:0/772270590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5595a7c32610 con 0x5595a7cd9bc0 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f39fdffb640 1 -- 172.21.15.18:0/772270590 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5595a79337e0 con 0x5595a7cd9bc0 2024-02-17T13:00:38.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f39fdffb640 1 -- 172.21.15.18:0/772270590 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3a0c024030 con 0x5595a7cd9bc0 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f3a1f420740 10 client.19467 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f3a1f420740 10 client.19467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f3a1f420740 20 client.19467 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f3a1f420740 1 --2- 172.21.15.18:0/772270590 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5595a7ce9d50 0x5595a7cec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f3a1f420740 4 client.19467 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.645+0000 7f3a1f420740 1 -- 172.21.15.18:0/772270590 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595a7970720 con 0x5595a7ce9d50 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.646+0000 7f3a1a515640 1 --2- 172.21.15.18:0/772270590 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5595a7ce9d50 0x5595a7cec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.646+0000 7f3a1a515640 1 --2- 172.21.15.18:0/772270590 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5595a7ce9d50 0x5595a7cec170 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b51b39cb487a00f server_cookie=cc58b112744308f2 in_seq=0 out_seq=0 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.646+0000 7f39fdffb640 10 client.19467 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f39fdffb640 1 -- 172.21.15.18:0/772270590 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5595a7970720 con 0x5595a7ce9d50 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f39fdffb640 10 client.19467 handle_command_reply: tid=0 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f3a1f420740 1 client.19467 shutdown 2024-02-17T13:00:38.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f3a1f420740 1 -- 172.21.15.18:0/772270590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 msgr2=0x5595a7cd6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f3a1f420740 1 --2- 172.21.15.18:0/772270590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 0x5595a7cd6f50 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0099b0 tx=0x7f3a0c003990 comp rx=0 tx=0).stop 2024-02-17T13:00:38.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f3a1f420740 1 -- 172.21.15.18:0/772270590 shutdown_connections 2024-02-17T13:00:38.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f3a1f420740 1 --2- 172.21.15.18:0/772270590 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5595a7ce9d50 0x5595a7cec170 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f3a1f420740 1 --2- 172.21.15.18:0/772270590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 0x5595a7cd6f50 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f3a1f420740 1 --2- 172.21.15.18:0/772270590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7ccc880 0x5595a7cd6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f39fdffb640 0 client.19467 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:38.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f3a1f420740 1 --2- 172.21.15.18:0/772270590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 0x5595a7cd95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f3a1f420740 1 -- 172.21.15.18:0/772270590 >> 172.21.15.18:0/772270590 conn(0x5595a7cc6990 msgr2=0x5595a7cc8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:38.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f3a1f420740 1 -- 172.21.15.18:0/772270590 shutdown_connections 2024-02-17T13:00:38.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f3a1f420740 1 -- 172.21.15.18:0/772270590 wait complete. 2024-02-17T13:00:38.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.647+0000 7f3a1f420740 20 client.19467 trim_cache size 0 max 16384 2024-02-17T13:00:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.662+0000 7f3a1f420740 1 Processor -- start 2024-02-17T13:00:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.662+0000 7f3a1f420740 1 -- start start 2024-02-17T13:00:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1f420740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7ccbf60 0x5595a7cd9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1f420740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccc880 0x5595a7cd95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1f420740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 0x5595a7ce0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1f420740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a7ce6250 con 0x5595a7cd9bc0 2024-02-17T13:00:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1f420740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a7970720 con 0x5595a7ccc880 2024-02-17T13:00:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1f420740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a7cc7ab0 con 0x5595a7ccbf60 2024-02-17T13:00:38.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1ad16640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 0x5595a7ce0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1a515640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7ccbf60 0x5595a7cd9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1a515640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7ccbf60 0x5595a7cd9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:43814/0 (socket says 172.21.15.18:43814) 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1a515640 1 -- 172.21.15.18:0/3957794992 learned_addr learned my addr 172.21.15.18:0/3957794992 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a19d14640 1 --2- 172.21.15.18:0/3957794992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccc880 0x5595a7cd95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1a515640 1 -- 172.21.15.18:0/3957794992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccc880 msgr2=0x5595a7cd95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1a515640 1 --2- 172.21.15.18:0/3957794992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccc880 0x5595a7cd95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1a515640 1 -- 172.21.15.18:0/3957794992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 msgr2=0x5595a7ce0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1a515640 1 --2- 172.21.15.18:0/3957794992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 0x5595a7ce0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1a515640 1 -- 172.21.15.18:0/3957794992 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595a7bceac0 con 0x5595a7ccbf60 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1ad16640 1 --2- 172.21.15.18:0/3957794992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 0x5595a7ce0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f3a1a515640 1 --2- 172.21.15.18:0/3957794992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7ccbf60 0x5595a7cd9070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3a10005ab0 tx=0x7f3a10009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61d006bd39965635 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.663+0000 7f39fffff640 1 -- 172.21.15.18:0/3957794992 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1002c650 con 0x5595a7ccbf60 2024-02-17T13:00:38.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.664+0000 7f39fffff640 1 -- 172.21.15.18:0/3957794992 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a100049e0 con 0x5595a7ccbf60 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.664+0000 7f39fffff640 1 -- 172.21.15.18:0/3957794992 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100493a0 con 0x5595a7ccbf60 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.664+0000 7f3a1f420740 1 -- 172.21.15.18:0/3957794992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7ccbf60 msgr2=0x5595a7cd9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.664+0000 7f3a1f420740 1 --2- 172.21.15.18:0/3957794992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7ccbf60 0x5595a7cd9070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3a10005ab0 tx=0x7f3a10009260 comp rx=0 tx=0).stop 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.664+0000 7f3a1f420740 1 -- 172.21.15.18:0/3957794992 shutdown_connections 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.664+0000 7f3a1f420740 1 --2- 172.21.15.18:0/3957794992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7cd9bc0 0x5595a7ce0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.664+0000 7f3a1f420740 1 --2- 172.21.15.18:0/3957794992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccc880 0x5595a7cd95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.664+0000 7f3a1f420740 1 --2- 172.21.15.18:0/3957794992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7ccbf60 0x5595a7cd9070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.664+0000 7f3a1f420740 1 -- 172.21.15.18:0/3957794992 >> 172.21.15.18:0/3957794992 conn(0x5595a7cc6990 msgr2=0x5595a7cd5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.664+0000 7f3a1f420740 1 -- 172.21.15.18:0/3957794992 shutdown_connections 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.664+0000 7f3a1f420740 1 -- 172.21.15.18:0/3957794992 wait complete. 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.664+0000 7f3a1f420740 1 -- start start 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1f420740 1 Processor -- start 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1f420740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 0x5595a7cddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1f420740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccc880 0x5595a7cde080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1f420740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7cd9bc0 0x5595a7d40250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1f420740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a7cde750 con 0x5595a7ccc880 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1f420740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a7d40790 con 0x5595a7ccbf60 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1f420740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595a7d408d0 con 0x5595a7cd9bc0 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1f420740 10 client.0 _ll_register_callbacks cb 0x7f3a0c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a19d14640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccc880 0x5595a7cde080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1a515640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 0x5595a7cddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1a515640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 0x5595a7cddb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:33964/0 (socket says 172.21.15.18:33964) 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a19d14640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccc880 0x5595a7cde080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42370/0 (socket says 172.21.15.18:42370) 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1ad16640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7cd9bc0 0x5595a7d40250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1a515640 1 -- 172.21.15.18:0/1290290445 learned_addr learned my addr 172.21.15.18:0/1290290445 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1a515640 1 -- 172.21.15.18:0/1290290445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7cd9bc0 msgr2=0x5595a7d40250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1a515640 1 --2- 172.21.15.18:0/1290290445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7cd9bc0 0x5595a7d40250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1a515640 1 -- 172.21.15.18:0/1290290445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccc880 msgr2=0x5595a7cde080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:38.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1a515640 1 --2- 172.21.15.18:0/1290290445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccc880 0x5595a7cde080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a1a515640 1 -- 172.21.15.18:0/1290290445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595a7c32610 con 0x5595a7ccbf60 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.665+0000 7f3a19d14640 1 --2- 172.21.15.18:0/1290290445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccc880 0x5595a7cde080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f3a1a515640 1 --2- 172.21.15.18:0/1290290445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 0x5595a7cddb40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009230 tx=0x7f3a1002c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d57b7d51e6f2d08 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f39fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f39fdffb640 1 -- 172.21.15.18:0/1290290445 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1005b070 con 0x5595a7ccbf60 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f3a1f420740 1 -- 172.21.15.18:0/1290290445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a0c009dc0 con 0x5595a7ccbf60 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f39fdffb640 1 -- 172.21.15.18:0/1290290445 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3a1004d030 con 0x5595a7ccbf60 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f39fdffb640 1 -- 172.21.15.18:0/1290290445 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a10049da0 con 0x5595a7ccbf60 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f3a1f420740 10 client.28469 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f3a1f420740 1 -- 172.21.15.18:0/1290290445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7f3a1410e820 con 0x5595a7ccbf60 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f39fdffb640 1 -- 172.21.15.18:0/1290290445 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c009dc0 con 0x5595a7ccbf60 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f39fdffb640 1 -- 172.21.15.18:0/1290290445 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3a1005e020 con 0x5595a7ccbf60 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f3a1f420740 10 client.28469 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f3a1f420740 10 client.28469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f3a1f420740 20 client.28469 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f3a1f420740 1 --2- 172.21.15.18:0/1290290445 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5595a7d41e80 0x5595a7d442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f3a1f420740 4 client.28469 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f3a1f420740 1 -- 172.21.15.18:0/1290290445 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5595a79bf890 con 0x5595a7d41e80 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.666+0000 7f3a19d14640 1 --2- 172.21.15.18:0/1290290445 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5595a7d41e80 0x5595a7d442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.667+0000 7f3a19d14640 1 --2- 172.21.15.18:0/1290290445 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5595a7d41e80 0x5595a7d442a0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9601455b02de02b7 server_cookie=505e267a68e927f2 in_seq=0 out_seq=0 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.667+0000 7f39fdffb640 10 client.28469 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f39fdffb640 1 -- 172.21.15.18:0/1290290445 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5595a79bf890 con 0x5595a7d41e80 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f39fdffb640 10 client.28469 handle_command_reply: tid=0 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1f420740 1 client.28469 shutdown 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1f420740 1 -- 172.21.15.18:0/1290290445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 msgr2=0x5595a7cddb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1f420740 1 --2- 172.21.15.18:0/1290290445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 0x5595a7cddb40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009230 tx=0x7f3a1002c660 comp rx=0 tx=0).stop 2024-02-17T13:00:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1f420740 1 -- 172.21.15.18:0/1290290445 shutdown_connections 2024-02-17T13:00:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1f420740 1 --2- 172.21.15.18:0/1290290445 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5595a7d41e80 0x5595a7d442a0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1f420740 1 --2- 172.21.15.18:0/1290290445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5595a7cd9bc0 0x5595a7d40250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f39fdffb640 0 client.28469 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1f420740 1 --2- 172.21.15.18:0/1290290445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595a7ccc880 0x5595a7cde080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1f420740 1 --2- 172.21.15.18:0/1290290445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595a7ccbf60 0x5595a7cddb40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1f420740 1 -- 172.21.15.18:0/1290290445 >> 172.21.15.18:0/1290290445 conn(0x5595a7cc6990 msgr2=0x5595a7cc7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1f420740 1 -- 172.21.15.18:0/1290290445 shutdown_connections 2024-02-17T13:00:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1f420740 1 -- 172.21.15.18:0/1290290445 wait complete. 2024-02-17T13:00:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1f420740 20 client.28469 trim_cache size 0 max 16384 2024-02-17T13:00:38.665 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:00:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T13:00:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T13:00:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "ab7eec6e-2fca-46f5-832e-cc72edca2775": { 2024-02-17T13:00:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T13:00:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "ab7eec6e-2fca-46f5-832e-cc72edca2775", 2024-02-17T13:00:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T13:00:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:00:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:00:38.666 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:00:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1c779640 1 -- 172.21.15.18:0/960912491 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3a000a2230 msgr2=0x7f3a000a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1c779640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3a000a2230 0x7f3a000a46f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00fc90 tx=0x7f3a0c01e000 comp rx=0 tx=0).stop 2024-02-17T13:00:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1c779640 1 -- 172.21.15.18:0/960912491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a14106440 msgr2=0x7f3a1419d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1c779640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a14106440 0x7f3a1419d9b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400d740 tx=0x7f3a0400dc00 comp rx=0 tx=0).stop 2024-02-17T13:00:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1c779640 1 -- 172.21.15.18:0/960912491 shutdown_connections 2024-02-17T13:00:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1c779640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3a000a2230 0x7f3a000a46f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1c779640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a14197620 0x7f3a14197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1c779640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a14106440 0x7f3a1419d9b0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1c779640 1 --2- 172.21.15.18:0/960912491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a14101370 0x7f3a1419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:38.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.668+0000 7f3a1c779640 1 -- 172.21.15.18:0/960912491 >> 172.21.15.18:0/960912491 conn(0x7f3a140fcc30 msgr2=0x7f3a140fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:38.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.669+0000 7f3a1c779640 1 -- 172.21.15.18:0/960912491 shutdown_connections 2024-02-17T13:00:38.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:00:38.669+0000 7f3a1c779640 1 -- 172.21.15.18:0/960912491 wait complete. 2024-02-17T13:00:38.669 INFO:tasks.cephfs.filesystem:scrub status for tag:ab7eec6e-2fca-46f5-832e-cc72edca2775 - {'path': '/', 'tag': 'ab7eec6e-2fca-46f5-832e-cc72edca2775', 'options': 'recursive,force'} 2024-02-17T13:00:38.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:38 smithi146 ceph-mon[36454]: pgmap v336: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.5 MiB/s wr, 133 op/s 2024-02-17T13:00:38.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:38 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:38.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:38 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:38.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:38 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:38.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:38 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:38 smithi018 ceph-mon[32625]: pgmap v336: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.5 MiB/s wr, 133 op/s 2024-02-17T13:00:39.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:38 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:39.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:38 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:38 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:39.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:38 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:38 smithi119 ceph-mon[36188]: pgmap v336: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.5 MiB/s wr, 133 op/s 2024-02-17T13:00:39.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:38 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:39.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:38 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:38 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:39.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:38 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 540047 4.98 MB/sec execute 474 sec latency 3576.759 ms 2024-02-17T13:00:39.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T13:00:39.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:39.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:39.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:39.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:39.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:00:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.950 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:39 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.992 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T13:00:39.992 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.992 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:39.992 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:39.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:39.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:39.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:39.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:39.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:00:39.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:39.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:39 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:40.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T13:00:40.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:40.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:40.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:40.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:40.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:40.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:40.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:40.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:40.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:40.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:40.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:40.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:40.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:00:40.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:40.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:39 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:00:40.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 541432 4.98 MB/sec execute 475 sec latency 18.285 ms 2024-02-17T13:00:40.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:40 smithi146 ceph-mon[36454]: pgmap v337: 97 pgs: 97 active+clean; 209 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-02-17T13:00:41.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:40 smithi018 ceph-mon[32625]: pgmap v337: 97 pgs: 97 active+clean; 209 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-02-17T13:00:41.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:40 smithi119 ceph-mon[36188]: pgmap v337: 97 pgs: 97 active+clean; 209 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-02-17T13:00:41.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 541671 4.97 MB/sec execute 476 sec latency 856.527 ms 2024-02-17T13:00:42.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 541671 4.96 MB/sec execute 477 sec latency 1856.579 ms 2024-02-17T13:00:42.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:42 smithi146 ceph-mon[36454]: pgmap v338: 97 pgs: 97 active+clean; 209 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-02-17T13:00:43.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:42 smithi018 ceph-mon[32625]: pgmap v338: 97 pgs: 97 active+clean; 209 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-02-17T13:00:43.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:42 smithi119 ceph-mon[36188]: pgmap v338: 97 pgs: 97 active+clean; 209 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-02-17T13:00:43.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 541671 4.95 MB/sec execute 478 sec latency 2856.686 ms 2024-02-17T13:00:44.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 542561 4.94 MB/sec execute 479 sec latency 3459.417 ms 2024-02-17T13:00:44.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:44 smithi146 ceph-mon[36454]: pgmap v339: 97 pgs: 97 active+clean; 209 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-02-17T13:00:45.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:44 smithi018 ceph-mon[32625]: pgmap v339: 97 pgs: 97 active+clean; 209 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-02-17T13:00:45.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:44 smithi119 ceph-mon[36188]: pgmap v339: 97 pgs: 97 active+clean; 209 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-02-17T13:00:45.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 544118 4.96 MB/sec execute 480 sec latency 17.465 ms 2024-02-17T13:00:45.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:45 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:00:45.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:45 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:00:45.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:45 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:00:46.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:45 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:00:46.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:45 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:00:46.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:45 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:00:46.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:45 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:00:46.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:45 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:00:46.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:45 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:00:46.296 INFO:tasks.workunit.client.0.smithi018.stdout: 1 545212 4.96 MB/sec execute 481 sec latency 395.102 ms 2024-02-17T13:00:46.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:46 smithi146 ceph-mon[36454]: pgmap v340: 97 pgs: 97 active+clean; 212 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-02-17T13:00:46.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:46 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:00:46.949 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:46 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:00:47.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:46 smithi018 ceph-mon[32625]: pgmap v340: 97 pgs: 97 active+clean; 212 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-02-17T13:00:47.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:46 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:00:47.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:46 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:00:47.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:46 smithi119 ceph-mon[36188]: pgmap v340: 97 pgs: 97 active+clean; 212 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-02-17T13:00:47.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:46 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:00:47.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:46 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:00:47.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 545212 4.95 MB/sec execute 482 sec latency 1395.177 ms 2024-02-17T13:00:48.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 545212 4.94 MB/sec execute 483 sec latency 2395.280 ms 2024-02-17T13:00:48.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:48 smithi146 ceph-mon[36454]: pgmap v341: 97 pgs: 97 active+clean; 212 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-02-17T13:00:49.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:48 smithi018 ceph-mon[32625]: pgmap v341: 97 pgs: 97 active+clean; 212 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-02-17T13:00:49.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:48 smithi119 ceph-mon[36188]: pgmap v341: 97 pgs: 97 active+clean; 212 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-02-17T13:00:49.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 546097 4.93 MB/sec execute 484 sec latency 2998.554 ms 2024-02-17T13:00:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.794+0000 7fefdffb6640 1 -- 172.21.15.18:0/2460938621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd81002d0 msgr2=0x7fefd8100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.794+0000 7fefdffb6640 1 --2- 172.21.15.18:0/2460938621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd81002d0 0x7fefd8100750 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009900 tx=0x7fefc802bc30 comp rx=0 tx=0).stop 2024-02-17T13:00:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.794+0000 7fefdffb6640 1 -- 172.21.15.18:0/2460938621 shutdown_connections 2024-02-17T13:00:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.794+0000 7fefdffb6640 1 --2- 172.21.15.18:0/2460938621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd8100c90 0x7fefd810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.794+0000 7fefdffb6640 1 --2- 172.21.15.18:0/2460938621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd81002d0 0x7fefd8100750 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.794+0000 7fefdffb6640 1 --2- 172.21.15.18:0/2460938621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd8105020 0x7fefd8105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.794+0000 7fefdffb6640 1 -- 172.21.15.18:0/2460938621 >> 172.21.15.18:0/2460938621 conn(0x7fefd8076ed0 msgr2=0x7fefd80772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.794+0000 7fefdffb6640 1 -- 172.21.15.18:0/2460938621 shutdown_connections 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.794+0000 7fefdffb6640 1 -- 172.21.15.18:0/2460938621 wait complete. 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.794+0000 7fefdffb6640 1 Processor -- start 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.794+0000 7fefdffb6640 1 -- start start 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdffb6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd81002d0 0x7fefd8101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdffb6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd8100c90 0x7fefd8102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdffb6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd8105020 0x7fefd8102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdffb6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd81a5ba0 con 0x7fefd8100c90 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdffb6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd81a5d10 con 0x7fefd8105020 2024-02-17T13:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdffb6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd81a5ed0 con 0x7fefd81002d0 2024-02-17T13:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdd551640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd8100c90 0x7fefd8102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdd551640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd8100c90 0x7fefd8102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59622/0 (socket says 172.21.15.18:59622) 2024-02-17T13:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefddd52640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd81002d0 0x7fefd8101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdd551640 1 -- 172.21.15.18:0/3340715542 learned_addr learned my addr 172.21.15.18:0/3340715542 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefde553640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd8105020 0x7fefd8102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdd551640 1 -- 172.21.15.18:0/3340715542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd81002d0 msgr2=0x7fefd8101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdd551640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd81002d0 0x7fefd8101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdd551640 1 -- 172.21.15.18:0/3340715542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd8105020 msgr2=0x7fefd8102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdd551640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd8105020 0x7fefd8102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdd551640 1 -- 172.21.15.18:0/3340715542 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc8009560 con 0x7fefd8100c90 2024-02-17T13:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefde553640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd8105020 0x7fefd8102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:00:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefdd551640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd8100c90 0x7fefd8102230 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fefc8005940 tx=0x7fefc802c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.795+0000 7fefceffd640 1 -- 172.21.15.18:0/3340715542 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80042d0 con 0x7fefd8100c90 2024-02-17T13:00:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.796+0000 7fefdffb6640 1 -- 172.21.15.18:0/3340715542 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd81a6150 con 0x7fefd8100c90 2024-02-17T13:00:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.796+0000 7fefceffd640 1 -- 172.21.15.18:0/3340715542 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefc8004430 con 0x7fefd8100c90 2024-02-17T13:00:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.796+0000 7fefdffb6640 1 -- 172.21.15.18:0/3340715542 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd81a66b0 con 0x7fefd8100c90 2024-02-17T13:00:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.796+0000 7fefceffd640 1 -- 172.21.15.18:0/3340715542 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8003c50 con 0x7fefd8100c90 2024-02-17T13:00:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.798+0000 7fefceffd640 1 -- 172.21.15.18:0/3340715542 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fefc80054f0 con 0x7fefd8100c90 2024-02-17T13:00:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.798+0000 7fefe2b87740 1 Processor -- start 2024-02-17T13:00:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.798+0000 7fefe2b87740 1 -- start start 2024-02-17T13:00:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.798+0000 7fefe2b87740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 0x55a111e4f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.798+0000 7fefe2b87740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e4fd60 0x55a111e501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefe2b87740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e592e0 0x55a111e5b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefe2b87740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a111dbe9f0 con 0x55a111e592e0 2024-02-17T13:00:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefe2b87740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a111dbbf20 con 0x55a111e50ad0 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefe2b87740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a111db7700 con 0x55a111e4fd60 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefde553640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 0x55a111e4f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefdd551640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e592e0 0x55a111e5b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefceffd640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fefc40a2300 0x7fefc40a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefde553640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 0x55a111e4f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:56564/0 (socket says 172.21.15.18:56564) 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefde553640 1 -- 172.21.15.18:0/84841096 learned_addr learned my addr 172.21.15.18:0/84841096 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefddd52640 1 --2- 172.21.15.18:0/84841096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e4fd60 0x55a111e501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefde553640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fefc40a2300 0x7fefc40a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefceffd640 1 -- 172.21.15.18:0/3340715542 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fefc80f00b0 con 0x7fefd8100c90 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefde553640 1 -- 172.21.15.18:0/84841096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e4fd60 msgr2=0x55a111e501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefde553640 1 --2- 172.21.15.18:0/84841096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e4fd60 0x55a111e501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefde553640 1 -- 172.21.15.18:0/84841096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e592e0 msgr2=0x55a111e5b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefde553640 1 --2- 172.21.15.18:0/84841096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e592e0 0x55a111e5b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefde553640 1 -- 172.21.15.18:0/84841096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a111d4d7b0 con 0x55a111e50ad0 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefdd551640 1 --2- 172.21.15.18:0/84841096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e592e0 0x55a111e5b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefde553640 1 --2- 172.21.15.18:0/84841096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 0x55a111e4f820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefd0013b00 tx=0x7fefd000f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=660aaaf4f492bc7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.799+0000 7fefde553640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fefc40a2300 0x7fefc40a47c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fefd001e000 tx=0x7fefd0020000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.800+0000 7fefc3fff640 1 -- 172.21.15.18:0/84841096 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000fcb0 con 0x55a111e50ad0 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.800+0000 7fefc3fff640 1 -- 172.21.15.18:0/84841096 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefd000fe10 con 0x55a111e50ad0 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.800+0000 7fefc3fff640 1 -- 172.21.15.18:0/84841096 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0011da0 con 0x55a111e50ad0 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.800+0000 7fefe2b87740 1 -- 172.21.15.18:0/84841096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 msgr2=0x55a111e4f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.800+0000 7fefe2b87740 1 --2- 172.21.15.18:0/84841096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 0x55a111e4f820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefd0013b00 tx=0x7fefd000f360 comp rx=0 tx=0).stop 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.800+0000 7fefe2b87740 1 -- 172.21.15.18:0/84841096 shutdown_connections 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.800+0000 7fefe2b87740 1 --2- 172.21.15.18:0/84841096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e592e0 0x55a111e5b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.800+0000 7fefe2b87740 1 --2- 172.21.15.18:0/84841096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e4fd60 0x55a111e501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.800+0000 7fefe2b87740 1 --2- 172.21.15.18:0/84841096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 0x55a111e4f820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.800+0000 7fefe2b87740 1 -- 172.21.15.18:0/84841096 >> 172.21.15.18:0/84841096 conn(0x55a111e17550 msgr2=0x55a111e4aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.801+0000 7fefe2b87740 1 -- 172.21.15.18:0/84841096 shutdown_connections 2024-02-17T13:00:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.801+0000 7fefe2b87740 1 -- 172.21.15.18:0/84841096 wait complete. 2024-02-17T13:00:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.801+0000 7fefe2b87740 1 -- start start 2024-02-17T13:00:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.801+0000 7fefe2b87740 1 Processor -- start 2024-02-17T13:00:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.801+0000 7fefe2b87740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 0x55a111e56340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.801+0000 7fefe2b87740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e50ad0 0x55a111e56880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.801+0000 7fefe2b87740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e592e0 0x55a111e56dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.801+0000 7fefe2b87740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a111d5cbf0 con 0x55a111e4fd60 2024-02-17T13:00:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.801+0000 7fefe2b87740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a111a8b250 con 0x55a111e592e0 2024-02-17T13:00:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.801+0000 7fefe2b87740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a111d3d200 con 0x55a111e50ad0 2024-02-17T13:00:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.801+0000 7fefe2b87740 10 client.0 _ll_register_callbacks cb 0x55a111dabd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.801+0000 7fefddd52640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 0x55a111e56340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefddd52640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 0x55a111e56340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59644/0 (socket says 172.21.15.18:59644) 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefdd551640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e50ad0 0x55a111e56880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefde553640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e592e0 0x55a111e56dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefddd52640 1 -- 172.21.15.18:0/214716664 learned_addr learned my addr 172.21.15.18:0/214716664 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefdd551640 1 -- 172.21.15.18:0/214716664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e592e0 msgr2=0x55a111e56dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefdd551640 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e592e0 0x55a111e56dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefdd551640 1 -- 172.21.15.18:0/214716664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 msgr2=0x55a111e56340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefdd551640 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 0x55a111e56340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefdd551640 1 -- 172.21.15.18:0/214716664 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd00136e0 con 0x55a111e50ad0 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefddd52640 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 0x55a111e56340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefde553640 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e592e0 0x55a111e56dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefdd551640 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e50ad0 0x55a111e56880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fefc80ed8e0 tx=0x7fefc8005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8918a3433ce4fb9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:00:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefc17fa640 1 -- 172.21.15.18:0/214716664 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80edc80 con 0x55a111e50ad0 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefc17fa640 1 -- 172.21.15.18:0/214716664 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fefc80bcbb0 con 0x55a111e50ad0 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefe2b87740 1 -- 172.21.15.18:0/214716664 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a111ab7600 con 0x55a111e50ad0 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefc17fa640 1 -- 172.21.15.18:0/214716664 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefc80c0d00 con 0x55a111e50ad0 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefe2b87740 10 client.28510 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.802+0000 7fefe2b87740 1 -- 172.21.15.18:0/214716664 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a111db5d80 con 0x55a111e50ad0 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.803+0000 7fefc17fa640 1 -- 172.21.15.18:0/214716664 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a111ab7600 con 0x55a111e50ad0 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.803+0000 7fefc17fa640 1 -- 172.21.15.18:0/214716664 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fefc803c070 con 0x55a111e50ad0 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.803+0000 7fefe2b87740 10 client.28510 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.803+0000 7fefe2b87740 10 client.28510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.803+0000 7fefe2b87740 20 client.28510 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.803+0000 7fefe2b87740 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a111e693f0 0x55a111e6b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.803+0000 7fefe2b87740 4 client.28510 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.803+0000 7fefe2b87740 1 -- 172.21.15.18:0/214716664 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a111b081b0 con 0x55a111e693f0 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.803+0000 7fefddd52640 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a111e693f0 0x55a111e6b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.803+0000 7fefc17fa640 10 client.28510 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.803+0000 7fefddd52640 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a111e693f0 0x55a111e6b810 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3acec5db4f054392 server_cookie=f7f84666b9cedf44 in_seq=0 out_seq=0 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefc17fa640 1 -- 172.21.15.18:0/214716664 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a111b081b0 con 0x55a111e693f0 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefc17fa640 10 client.28510 handle_command_reply: tid=0 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefe2b87740 1 client.28510 shutdown 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefe2b87740 1 -- 172.21.15.18:0/214716664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e50ad0 msgr2=0x55a111e56880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefe2b87740 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e50ad0 0x55a111e56880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fefc80ed8e0 tx=0x7fefc8005bf0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefe2b87740 1 -- 172.21.15.18:0/214716664 shutdown_connections 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefe2b87740 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a111e693f0 0x55a111e6b810 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefe2b87740 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e592e0 0x55a111e56dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefc17fa640 0 client.28510 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefe2b87740 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e50ad0 0x55a111e56880 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefe2b87740 1 --2- 172.21.15.18:0/214716664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 0x55a111e56340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefe2b87740 1 -- 172.21.15.18:0/214716664 >> 172.21.15.18:0/214716664 conn(0x55a111e17550 msgr2=0x55a111e4c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefe2b87740 1 -- 172.21.15.18:0/214716664 shutdown_connections 2024-02-17T13:00:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefe2b87740 1 -- 172.21.15.18:0/214716664 wait complete. 2024-02-17T13:00:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.804+0000 7fefe2b87740 20 client.28510 trim_cache size 0 max 16384 2024-02-17T13:00:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.818+0000 7fefe2b87740 1 Processor -- start 2024-02-17T13:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.818+0000 7fefe2b87740 1 -- start start 2024-02-17T13:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefe2b87740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e4fd60 0x55a111e56be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefe2b87740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e50ad0 0x55a111e54c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefe2b87740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e592e0 0x55a111e55150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefe2b87740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a111e51230 con 0x55a111e50ad0 2024-02-17T13:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefe2b87740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a111e4bc80 con 0x55a111e592e0 2024-02-17T13:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefe2b87740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a111e66140 con 0x55a111e4fd60 2024-02-17T13:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefdd551640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e50ad0 0x55a111e54c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefdd551640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e50ad0 0x55a111e54c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59660/0 (socket says 172.21.15.18:59660) 2024-02-17T13:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefdd551640 1 -- 172.21.15.18:0/2752891060 learned_addr learned my addr 172.21.15.18:0/2752891060 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefde553640 1 --2- 172.21.15.18:0/2752891060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e592e0 0x55a111e55150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefddd52640 1 --2- 172.21.15.18:0/2752891060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e4fd60 0x55a111e56be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefdd551640 1 -- 172.21.15.18:0/2752891060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e4fd60 msgr2=0x55a111e56be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefdd551640 1 --2- 172.21.15.18:0/2752891060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e4fd60 0x55a111e56be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefdd551640 1 -- 172.21.15.18:0/2752891060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e592e0 msgr2=0x55a111e55150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefdd551640 1 --2- 172.21.15.18:0/2752891060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e592e0 0x55a111e55150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefdd551640 1 -- 172.21.15.18:0/2752891060 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a111d527b0 con 0x55a111e50ad0 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefdd551640 1 --2- 172.21.15.18:0/2752891060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e50ad0 0x55a111e54c10 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fefc80bcb40 tx=0x7fefc80bb0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bae3e79aa58e880 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefc37fe640 1 -- 172.21.15.18:0/2752891060 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80bc510 con 0x55a111e50ad0 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefc37fe640 1 -- 172.21.15.18:0/2752891060 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefc80bcbb0 con 0x55a111e50ad0 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.819+0000 7fefc37fe640 1 -- 172.21.15.18:0/2752891060 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80bc260 con 0x55a111e50ad0 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.820+0000 7fefe2b87740 1 -- 172.21.15.18:0/2752891060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e50ad0 msgr2=0x55a111e54c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.820+0000 7fefe2b87740 1 --2- 172.21.15.18:0/2752891060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e50ad0 0x55a111e54c10 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fefc80bcb40 tx=0x7fefc80bb0c0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.820+0000 7fefe2b87740 1 -- 172.21.15.18:0/2752891060 shutdown_connections 2024-02-17T13:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.820+0000 7fefe2b87740 1 --2- 172.21.15.18:0/2752891060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e592e0 0x55a111e55150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.820+0000 7fefe2b87740 1 --2- 172.21.15.18:0/2752891060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e50ad0 0x55a111e54c10 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.820+0000 7fefe2b87740 1 --2- 172.21.15.18:0/2752891060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e4fd60 0x55a111e56be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.820+0000 7fefe2b87740 1 -- 172.21.15.18:0/2752891060 >> 172.21.15.18:0/2752891060 conn(0x55a111e17550 msgr2=0x55a111e4a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.820+0000 7fefe2b87740 1 -- 172.21.15.18:0/2752891060 shutdown_connections 2024-02-17T13:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.820+0000 7fefe2b87740 1 -- 172.21.15.18:0/2752891060 wait complete. 2024-02-17T13:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.820+0000 7fefe2b87740 1 -- start start 2024-02-17T13:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefe2b87740 1 Processor -- start 2024-02-17T13:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefe2b87740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 0x55a111e5e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefe2b87740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 0x55a111e5e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefe2b87740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e592e0 0x55a111ebe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefe2b87740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a111e5ee80 con 0x55a111e4fd60 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefe2b87740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a111ebee30 con 0x55a111e50ad0 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefe2b87740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a111ebef70 con 0x55a111e592e0 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefe2b87740 10 client.0 _ll_register_callbacks cb 0x7fefd806aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefddd52640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 0x55a111e5e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefdd551640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 0x55a111e5e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefddd52640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 0x55a111e5e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59670/0 (socket says 172.21.15.18:59670) 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefddd52640 1 -- 172.21.15.18:0/2297569766 learned_addr learned my addr 172.21.15.18:0/2297569766 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefde553640 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e592e0 0x55a111ebe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefddd52640 1 -- 172.21.15.18:0/2297569766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e592e0 msgr2=0x55a111ebe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefddd52640 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e592e0 0x55a111ebe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefddd52640 1 -- 172.21.15.18:0/2297569766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 msgr2=0x55a111e5e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefddd52640 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 0x55a111e5e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefddd52640 1 -- 172.21.15.18:0/2297569766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a111db5d80 con 0x55a111e4fd60 2024-02-17T13:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.821+0000 7fefdd551640 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 0x55a111e5e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefddd52640 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 0x55a111e5e090 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fefd4000c00 tx=0x7fefd4012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9747b8b7ee43f3e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefc17fa640 1 -- 172.21.15.18:0/2297569766 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4017040 con 0x55a111e4fd60 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefc17fa640 1 -- 172.21.15.18:0/2297569766 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fefd4004230 con 0x55a111e4fd60 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefe2b87740 1 -- 172.21.15.18:0/2297569766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefd000fe10 con 0x55a111e4fd60 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefc17fa640 1 -- 172.21.15.18:0/2297569766 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefd40043f0 con 0x55a111e4fd60 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefe2b87740 10 client.19491 fetch_fsmap learned FSMap version 48 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefe2b87740 1 -- 172.21.15.18:0/2297569766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a111d78540 con 0x55a111e4fd60 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefc17fa640 1 -- 172.21.15.18:0/2297569766 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefd000fe10 con 0x55a111e4fd60 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefc17fa640 1 -- 172.21.15.18:0/2297569766 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 48) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fefd4003a30 con 0x55a111e4fd60 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefe2b87740 10 client.19491 fetch_fsmap finished waiting for FSMap version 48 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefe2b87740 10 client.19491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefe2b87740 20 client.19491 populate_metadata read hostname 'smithi018' 2024-02-17T13:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefe2b87740 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a111ec0770 0x55a111ec2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefe2b87740 4 client.19491 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.822+0000 7fefe2b87740 1 -- 172.21.15.18:0/2297569766 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a111af4550 con 0x55a111ec0770 2024-02-17T13:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.823+0000 7fefdd551640 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a111ec0770 0x55a111ec2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.823+0000 7fefc17fa640 10 client.19491 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.823+0000 7fefdd551640 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a111ec0770 0x55a111ec2b90 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0078dde7fcb43b1 server_cookie=b74c0889e45bdf19 in_seq=0 out_seq=0 2024-02-17T13:00:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.854+0000 7fefc17fa640 1 -- 172.21.15.18:0/2297569766 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a111af4550 con 0x55a111ec0770 2024-02-17T13:00:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.854+0000 7fefc17fa640 10 client.19491 handle_command_reply: tid=0 2024-02-17T13:00:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.854+0000 7fefe2b87740 1 client.19491 shutdown 2024-02-17T13:00:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.854+0000 7fefe2b87740 1 -- 172.21.15.18:0/2297569766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 msgr2=0x55a111e5e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.854+0000 7fefe2b87740 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 0x55a111e5e090 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fefd4000c00 tx=0x7fefd4012b20 comp rx=0 tx=0).stop 2024-02-17T13:00:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.854+0000 7fefe2b87740 1 -- 172.21.15.18:0/2297569766 shutdown_connections 2024-02-17T13:00:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.854+0000 7fefe2b87740 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a111ec0770 0x55a111ec2b90 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefe2b87740 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a111e592e0 0x55a111ebe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefe2b87740 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a111e50ad0 0x55a111e5e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefe2b87740 1 --2- 172.21.15.18:0/2297569766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a111e4fd60 0x55a111e5e090 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefc17fa640 0 client.19491 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:00:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefe2b87740 1 -- 172.21.15.18:0/2297569766 >> 172.21.15.18:0/2297569766 conn(0x55a111e17550 msgr2=0x55a111e54b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefe2b87740 1 -- 172.21.15.18:0/2297569766 shutdown_connections 2024-02-17T13:00:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefe2b87740 1 -- 172.21.15.18:0/2297569766 wait complete. 2024-02-17T13:00:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefe2b87740 20 client.19491 trim_cache size 0 max 16384 2024-02-17T13:00:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:00:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:00:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:00:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:00:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefdffb6640 1 -- 172.21.15.18:0/3340715542 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fefc40a2300 msgr2=0x7fefc40a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefdffb6640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fefc40a2300 0x7fefc40a47c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fefd001e000 tx=0x7fefd0020000 comp rx=0 tx=0).stop 2024-02-17T13:00:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefdffb6640 1 -- 172.21.15.18:0/3340715542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd8100c90 msgr2=0x7fefd8102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:00:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefdffb6640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd8100c90 0x7fefd8102230 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fefc8005940 tx=0x7fefc802c670 comp rx=0 tx=0).stop 2024-02-17T13:00:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefdffb6640 1 -- 172.21.15.18:0/3340715542 shutdown_connections 2024-02-17T13:00:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefdffb6640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fefc40a2300 0x7fefc40a47c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefdffb6640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd8105020 0x7fefd8102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefdffb6640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd8100c90 0x7fefd8102230 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefdffb6640 1 --2- 172.21.15.18:0/3340715542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd81002d0 0x7fefd8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:00:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefdffb6640 1 -- 172.21.15.18:0/3340715542 >> 172.21.15.18:0/3340715542 conn(0x7fefd8076ed0 msgr2=0x7fefd8068a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:00:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefdffb6640 1 -- 172.21.15.18:0/3340715542 shutdown_connections 2024-02-17T13:00:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:00:49.855+0000 7fefdffb6640 1 -- 172.21.15.18:0/3340715542 wait complete. 2024-02-17T13:00:50.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 547662 4.95 MB/sec execute 485 sec latency 20.249 ms 2024-02-17T13:00:50.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:50 smithi146 ceph-mon[36454]: pgmap v342: 97 pgs: 97 active+clean; 214 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-02-17T13:00:51.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:50 smithi018 ceph-mon[32625]: pgmap v342: 97 pgs: 97 active+clean; 214 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-02-17T13:00:51.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:50 smithi119 ceph-mon[36188]: pgmap v342: 97 pgs: 97 active+clean; 214 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-02-17T13:00:51.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 548762 4.95 MB/sec execute 486 sec latency 359.843 ms 2024-02-17T13:00:52.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 548762 4.94 MB/sec execute 487 sec latency 1359.930 ms 2024-02-17T13:00:53.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:52 smithi018 ceph-mon[32625]: pgmap v343: 97 pgs: 97 active+clean; 214 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-02-17T13:00:53.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:52 smithi119 ceph-mon[36188]: pgmap v343: 97 pgs: 97 active+clean; 214 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-02-17T13:00:53.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:52 smithi146 ceph-mon[36454]: pgmap v343: 97 pgs: 97 active+clean; 214 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-02-17T13:00:53.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 549667 4.93 MB/sec execute 488 sec latency 1922.996 ms 2024-02-17T13:00:54.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 551233 4.94 MB/sec execute 489 sec latency 14.981 ms 2024-02-17T13:00:55.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:54 smithi018 ceph-mon[32625]: pgmap v344: 97 pgs: 97 active+clean; 214 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-02-17T13:00:55.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:54 smithi119 ceph-mon[36188]: pgmap v344: 97 pgs: 97 active+clean; 214 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-02-17T13:00:55.213 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:54 smithi146 ceph-mon[36454]: pgmap v344: 97 pgs: 97 active+clean; 214 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-02-17T13:00:55.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 552305 4.95 MB/sec execute 490 sec latency 353.157 ms 2024-02-17T13:00:56.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:55 smithi018 ceph-mon[32625]: pgmap v345: 97 pgs: 97 active+clean; 215 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-02-17T13:00:56.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:55 smithi119 ceph-mon[36188]: pgmap v345: 97 pgs: 97 active+clean; 215 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-02-17T13:00:56.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:55 smithi146 ceph-mon[36454]: pgmap v345: 97 pgs: 97 active+clean; 215 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-02-17T13:00:56.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 552305 4.94 MB/sec execute 491 sec latency 1353.243 ms 2024-02-17T13:00:57.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 552305 4.93 MB/sec execute 492 sec latency 2353.317 ms 2024-02-17T13:00:58.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:57 smithi146 ceph-mon[36454]: pgmap v346: 97 pgs: 97 active+clean; 215 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-02-17T13:00:58.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:57 smithi018 ceph-mon[32625]: pgmap v346: 97 pgs: 97 active+clean; 215 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-02-17T13:00:58.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 552305 4.92 MB/sec execute 493 sec latency 3353.354 ms 2024-02-17T13:00:58.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:57 smithi119 ceph-mon[36188]: pgmap v346: 97 pgs: 97 active+clean; 215 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-02-17T13:00:59.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:58 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:00:59.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:58 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:00:59.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 554166 4.93 MB/sec execute 494 sec latency 3534.888 ms 2024-02-17T13:00:59.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:58 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:01:00.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:00:59 smithi146 ceph-mon[36454]: pgmap v347: 97 pgs: 97 active+clean; 218 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.0 MiB/s wr, 185 op/s 2024-02-17T13:01:00.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:00:59 smithi018 ceph-mon[32625]: pgmap v347: 97 pgs: 97 active+clean; 218 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.0 MiB/s wr, 185 op/s 2024-02-17T13:01:00.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 555348 4.93 MB/sec execute 495 sec latency 22.798 ms 2024-02-17T13:01:00.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:00:59 smithi119 ceph-mon[36188]: pgmap v347: 97 pgs: 97 active+clean; 218 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.0 MiB/s wr, 185 op/s 2024-02-17T13:01:01.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 555852 4.92 MB/sec execute 496 sec latency 712.080 ms 2024-02-17T13:01:02.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:01 smithi146 ceph-mon[36454]: pgmap v348: 97 pgs: 97 active+clean; 218 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T13:01:02.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:01 smithi018 ceph-mon[32625]: pgmap v348: 97 pgs: 97 active+clean; 218 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T13:01:02.297 INFO:tasks.workunit.client.0.smithi018.stdout: 1 555852 4.91 MB/sec execute 497 sec latency 1712.116 ms 2024-02-17T13:01:02.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:01 smithi119 ceph-mon[36188]: pgmap v348: 97 pgs: 97 active+clean; 218 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T13:01:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.954+0000 7fa260a88640 1 -- 172.21.15.18:0/2456914148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa25c1002d0 msgr2=0x7fa25c100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.954+0000 7fa260a88640 1 --2- 172.21.15.18:0/2456914148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa25c1002d0 0x7fa25c100750 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fa244009900 tx=0x7fa24402bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.954+0000 7fa260a88640 1 -- 172.21.15.18:0/2456914148 shutdown_connections 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.954+0000 7fa260a88640 1 --2- 172.21.15.18:0/2456914148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa25c100c90 0x7fa25c10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.954+0000 7fa260a88640 1 --2- 172.21.15.18:0/2456914148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa25c1002d0 0x7fa25c100750 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.954+0000 7fa260a88640 1 --2- 172.21.15.18:0/2456914148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c105020 0x7fa25c105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.954+0000 7fa260a88640 1 -- 172.21.15.18:0/2456914148 >> 172.21.15.18:0/2456914148 conn(0x7fa25c076ed0 msgr2=0x7fa25c0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.954+0000 7fa260a88640 1 -- 172.21.15.18:0/2456914148 shutdown_connections 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.954+0000 7fa260a88640 1 -- 172.21.15.18:0/2456914148 wait complete. 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.955+0000 7fa260a88640 1 Processor -- start 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.955+0000 7fa260a88640 1 -- start start 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.955+0000 7fa260a88640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa25c1002d0 0x7fa25c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.955+0000 7fa260a88640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa25c100c90 0x7fa25c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.955+0000 7fa260a88640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c105020 0x7fa25c19c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa260a88640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c19c620 con 0x7fa25c1002d0 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa260a88640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c19c790 con 0x7fa25c100c90 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa260a88640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c19c950 con 0x7fa25c105020 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa25a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa25c1002d0 0x7fa25c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa25a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa25c1002d0 0x7fa25c197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44734/0 (socket says 172.21.15.18:44734) 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa25a59c640 1 -- 172.21.15.18:0/296910198 learned_addr learned my addr 172.21.15.18:0/296910198 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa25ad9d640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c105020 0x7fa25c19c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa251d9b640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa25c100c90 0x7fa25c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa25a59c640 1 -- 172.21.15.18:0/296910198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c105020 msgr2=0x7fa25c19c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa25a59c640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c105020 0x7fa25c19c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa25a59c640 1 -- 172.21.15.18:0/296910198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa25c100c90 msgr2=0x7fa25c197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa25a59c640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa25c100c90 0x7fa25c197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa25a59c640 1 -- 172.21.15.18:0/296910198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa244009560 con 0x7fa25c1002d0 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa25ad9d640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c105020 0x7fa25c19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa25a59c640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa25c1002d0 0x7fa25c197760 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00e9f0 tx=0x7fa24c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa253fff640 1 -- 172.21.15.18:0/296910198 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c009720 con 0x7fa25c1002d0 2024-02-17T13:01:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa260a88640 1 -- 172.21.15.18:0/296910198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25c19cc30 con 0x7fa25c1002d0 2024-02-17T13:01:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa253fff640 1 -- 172.21.15.18:0/296910198 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa24c00cd80 con 0x7fa25c1002d0 2024-02-17T13:01:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa253fff640 1 -- 172.21.15.18:0/296910198 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c010690 con 0x7fa25c1002d0 2024-02-17T13:01:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.956+0000 7fa260a88640 1 -- 172.21.15.18:0/296910198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c1a44b0 con 0x7fa25c1002d0 2024-02-17T13:01:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.959+0000 7fa253fff640 1 -- 172.21.15.18:0/296910198 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa24c0046c0 con 0x7fa25c1002d0 2024-02-17T13:01:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.959+0000 7fa263664740 1 Processor -- start 2024-02-17T13:01:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.959+0000 7fa263664740 1 -- start start 2024-02-17T13:01:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.959+0000 7fa263664740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028ead0 0x55561028d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.959+0000 7fa263664740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028dd60 0x55561028e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.959+0000 7fa263664740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 0x55561029dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.959+0000 7fa263664740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556101fc9f0 con 0x55561029b6f0 2024-02-17T13:01:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.959+0000 7fa263664740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556101f9f20 con 0x55561028ead0 2024-02-17T13:01:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.959+0000 7fa263664740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556101f5700 con 0x55561028dd60 2024-02-17T13:01:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa251d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 0x55561029dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa251d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 0x55561029dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44748/0 (socket says 172.21.15.18:44748) 2024-02-17T13:01:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa253fff640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa2480a22d0 0x7fa2480a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa251d9b640 1 -- 172.21.15.18:0/3553261343 learned_addr learned my addr 172.21.15.18:0/3553261343 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa25ad9d640 1 --2- 172.21.15.18:0/3553261343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028ead0 0x55561028d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa25a59c640 1 --2- 172.21.15.18:0/3553261343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028dd60 0x55561028e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa25ad9d640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa2480a22d0 0x7fa2480a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa251d9b640 1 -- 172.21.15.18:0/3553261343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028dd60 msgr2=0x55561028e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa253fff640 1 -- 172.21.15.18:0/296910198 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa24c014070 con 0x7fa25c1002d0 2024-02-17T13:01:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa251d9b640 1 --2- 172.21.15.18:0/3553261343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028dd60 0x55561028e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa251d9b640 1 -- 172.21.15.18:0/3553261343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028ead0 msgr2=0x55561028d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa251d9b640 1 --2- 172.21.15.18:0/3553261343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028ead0 0x55561028d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa251d9b640 1 -- 172.21.15.18:0/3553261343 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55561018b7b0 con 0x55561029b6f0 2024-02-17T13:01:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa25a59c640 1 --2- 172.21.15.18:0/3553261343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028dd60 0x55561028e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:01:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa251d9b640 1 --2- 172.21.15.18:0/3553261343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 0x55561029dae0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa24402c170 tx=0x7fa24402c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22bd8eef0b1c609 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa25ad9d640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa2480a22d0 0x7fa2480a4790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa254007310 tx=0x7fa2540072a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.960+0000 7fa250d99640 1 -- 172.21.15.18:0/3553261343 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24402cb10 con 0x55561029b6f0 2024-02-17T13:01:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.961+0000 7fa250d99640 1 -- 172.21.15.18:0/3553261343 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa244004030 con 0x55561029b6f0 2024-02-17T13:01:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.961+0000 7fa250d99640 1 -- 172.21.15.18:0/3553261343 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244005290 con 0x55561029b6f0 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.961+0000 7fa263664740 1 -- 172.21.15.18:0/3553261343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 msgr2=0x55561029dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.961+0000 7fa263664740 1 --2- 172.21.15.18:0/3553261343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 0x55561029dae0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa24402c170 tx=0x7fa24402c6e0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.961+0000 7fa263664740 1 -- 172.21.15.18:0/3553261343 shutdown_connections 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.961+0000 7fa263664740 1 --2- 172.21.15.18:0/3553261343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 0x55561029dae0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.961+0000 7fa263664740 1 --2- 172.21.15.18:0/3553261343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028dd60 0x55561028e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.961+0000 7fa263664740 1 --2- 172.21.15.18:0/3553261343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028ead0 0x55561028d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.961+0000 7fa263664740 1 -- 172.21.15.18:0/3553261343 >> 172.21.15.18:0/3553261343 conn(0x555610255550 msgr2=0x555610288a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.961+0000 7fa263664740 1 -- 172.21.15.18:0/3553261343 shutdown_connections 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.961+0000 7fa263664740 1 -- 172.21.15.18:0/3553261343 wait complete. 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa263664740 1 -- start start 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa263664740 1 Processor -- start 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa263664740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028dd60 0x555610298620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa263664740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028ead0 0x555610298b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa263664740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 0x5556102990a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa263664740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55561019abf0 con 0x55561029b6f0 2024-02-17T13:01:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa263664740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560fec9250 con 0x55561028dd60 2024-02-17T13:01:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa263664740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55561017b200 con 0x55561028ead0 2024-02-17T13:01:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa263664740 10 client.0 _ll_register_callbacks cb 0x5556101e9d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa25ad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 0x5556102990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa25ad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 0x5556102990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44754/0 (socket says 172.21.15.18:44754) 2024-02-17T13:01:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa25ad9d640 1 -- 172.21.15.18:0/1042679141 learned_addr learned my addr 172.21.15.18:0/1042679141 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa251d9b640 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028ead0 0x555610298b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.962+0000 7fa25a59c640 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028dd60 0x555610298620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa25ad9d640 1 -- 172.21.15.18:0/1042679141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028ead0 msgr2=0x555610298b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa25ad9d640 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028ead0 0x555610298b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa25ad9d640 1 -- 172.21.15.18:0/1042679141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028dd60 msgr2=0x555610298620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa25ad9d640 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028dd60 0x555610298620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa25ad9d640 1 -- 172.21.15.18:0/1042679141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c068800 con 0x55561029b6f0 2024-02-17T13:01:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa25ad9d640 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 0x5556102990a0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa25400eb60 tx=0x7fa254004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67081498eb22d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa241ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa241ffb640 1 -- 172.21.15.18:0/1042679141 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25400cab0 con 0x55561029b6f0 2024-02-17T13:01:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa241ffb640 1 -- 172.21.15.18:0/1042679141 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa25400cc10 con 0x55561029b6f0 2024-02-17T13:01:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa263664740 1 -- 172.21.15.18:0/1042679141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55560fef5600 con 0x55561029b6f0 2024-02-17T13:01:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa241ffb640 1 -- 172.21.15.18:0/1042679141 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa25400cdd0 con 0x55561029b6f0 2024-02-17T13:01:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa263664740 10 client.19503 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.963+0000 7fa263664740 1 -- 172.21.15.18:0/1042679141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5556101f3d80 con 0x55561029b6f0 2024-02-17T13:01:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.964+0000 7fa241ffb640 1 -- 172.21.15.18:0/1042679141 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55560fef5600 con 0x55561029b6f0 2024-02-17T13:01:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.964+0000 7fa241ffb640 1 -- 172.21.15.18:0/1042679141 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa25401e3b0 con 0x55561029b6f0 2024-02-17T13:01:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.964+0000 7fa263664740 10 client.19503 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.964+0000 7fa263664740 10 client.19503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.964+0000 7fa263664740 20 client.19503 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.964+0000 7fa263664740 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5556102ab920 0x5556102add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.964+0000 7fa263664740 4 client.19503 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.964+0000 7fa263664740 1 -- 172.21.15.18:0/1042679141 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55560ff461b0 con 0x5556102ab920 2024-02-17T13:01:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.964+0000 7fa25a59c640 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5556102ab920 0x5556102add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa241ffb640 10 client.19503 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa25a59c640 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5556102ab920 0x5556102add40 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf4e736971bf2a6b server_cookie=dd34b4fbe9dcb83c in_seq=0 out_seq=0 2024-02-17T13:01:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa241ffb640 1 -- 172.21.15.18:0/1042679141 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55560ff461b0 con 0x5556102ab920 2024-02-17T13:01:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa241ffb640 10 client.19503 handle_command_reply: tid=0 2024-02-17T13:01:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa263664740 1 client.19503 shutdown 2024-02-17T13:01:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa263664740 1 -- 172.21.15.18:0/1042679141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 msgr2=0x5556102990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa263664740 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 0x5556102990a0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa25400eb60 tx=0x7fa254004a40 comp rx=0 tx=0).stop 2024-02-17T13:01:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa263664740 1 -- 172.21.15.18:0/1042679141 shutdown_connections 2024-02-17T13:01:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa263664740 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5556102ab920 0x5556102add40 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa263664740 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561029b6f0 0x5556102990a0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa263664740 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028ead0 0x555610298b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa241ffb640 0 client.19503 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa263664740 1 --2- 172.21.15.18:0/1042679141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028dd60 0x555610298620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.965+0000 7fa263664740 1 -- 172.21.15.18:0/1042679141 >> 172.21.15.18:0/1042679141 conn(0x555610255550 msgr2=0x55561028a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.966+0000 7fa263664740 1 -- 172.21.15.18:0/1042679141 shutdown_connections 2024-02-17T13:01:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.966+0000 7fa263664740 1 -- 172.21.15.18:0/1042679141 wait complete. 2024-02-17T13:01:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.966+0000 7fa263664740 20 client.19503 trim_cache size 0 max 16384 2024-02-17T13:01:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa263664740 1 Processor -- start 2024-02-17T13:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa263664740 1 -- start start 2024-02-17T13:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa263664740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028dd60 0x55561028a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa263664740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028ead0 0x5556102a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa263664740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561029b6f0 0x5556102a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa263664740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560ff461b0 con 0x55561028ead0 2024-02-17T13:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa263664740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560fef5600 con 0x55561029b6f0 2024-02-17T13:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa263664740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555610288310 con 0x55561028dd60 2024-02-17T13:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa251d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028ead0 0x5556102a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa251d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028ead0 0x5556102a24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44766/0 (socket says 172.21.15.18:44766) 2024-02-17T13:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa251d9b640 1 -- 172.21.15.18:0/1509403922 learned_addr learned my addr 172.21.15.18:0/1509403922 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa25a59c640 1 --2- 172.21.15.18:0/1509403922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028dd60 0x55561028a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa251d9b640 1 -- 172.21.15.18:0/1509403922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028dd60 msgr2=0x55561028a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa251d9b640 1 --2- 172.21.15.18:0/1509403922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028dd60 0x55561028a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa251d9b640 1 -- 172.21.15.18:0/1509403922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561029b6f0 msgr2=0x5556102a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa251d9b640 1 --2- 172.21.15.18:0/1509403922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561029b6f0 0x5556102a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.979+0000 7fa251d9b640 1 -- 172.21.15.18:0/1509403922 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556101907b0 con 0x55561028ead0 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.980+0000 7fa251d9b640 1 --2- 172.21.15.18:0/1509403922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028ead0 0x5556102a24c0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa244004e60 tx=0x7fa244004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0f55409311b8f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.980+0000 7fa243fff640 1 -- 172.21.15.18:0/1509403922 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24402c910 con 0x55561028ead0 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.980+0000 7fa243fff640 1 -- 172.21.15.18:0/1509403922 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa24402ca70 con 0x55561028ead0 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.980+0000 7fa243fff640 1 -- 172.21.15.18:0/1509403922 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2440035f0 con 0x55561028ead0 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.980+0000 7fa263664740 1 -- 172.21.15.18:0/1509403922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028ead0 msgr2=0x5556102a24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.980+0000 7fa263664740 1 --2- 172.21.15.18:0/1509403922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028ead0 0x5556102a24c0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa244004e60 tx=0x7fa244004f30 comp rx=0 tx=0).stop 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.980+0000 7fa263664740 1 -- 172.21.15.18:0/1509403922 shutdown_connections 2024-02-17T13:01:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.980+0000 7fa263664740 1 --2- 172.21.15.18:0/1509403922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561029b6f0 0x5556102a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.980+0000 7fa263664740 1 --2- 172.21.15.18:0/1509403922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028ead0 0x5556102a24c0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.980+0000 7fa263664740 1 --2- 172.21.15.18:0/1509403922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561028dd60 0x55561028a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.980+0000 7fa263664740 1 -- 172.21.15.18:0/1509403922 >> 172.21.15.18:0/1509403922 conn(0x555610255550 msgr2=0x555610289e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.980+0000 7fa263664740 1 -- 172.21.15.18:0/1509403922 shutdown_connections 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa263664740 1 -- 172.21.15.18:0/1509403922 wait complete. 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa263664740 1 -- start start 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa263664740 1 Processor -- start 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa263664740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028dd60 0x5556102a1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa263664740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028ead0 0x5556102a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa263664740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561029b6f0 0x5556102a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa263664740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555610301ef0 con 0x55561028dd60 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa263664740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555610302030 con 0x55561028ead0 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa263664740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556103021f0 con 0x55561029b6f0 2024-02-17T13:01:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa263664740 10 client.0 _ll_register_callbacks cb 0x7fa24c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa25a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028dd60 0x5556102a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa25a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028dd60 0x5556102a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44768/0 (socket says 172.21.15.18:44768) 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa25a59c640 1 -- 172.21.15.18:0/2682644275 learned_addr learned my addr 172.21.15.18:0/2682644275 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa25ad9d640 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561029b6f0 0x5556102a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa251d9b640 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028ead0 0x5556102a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa25a59c640 1 -- 172.21.15.18:0/2682644275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561029b6f0 msgr2=0x5556102a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa25a59c640 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561029b6f0 0x5556102a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa25a59c640 1 -- 172.21.15.18:0/2682644275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028ead0 msgr2=0x5556102a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa25a59c640 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028ead0 0x5556102a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa25a59c640 1 -- 172.21.15.18:0/2682644275 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55561018b7b0 con 0x55561028dd60 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.981+0000 7fa251d9b640 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028ead0 0x5556102a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa25a59c640 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028dd60 0x5556102a1070 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa24c09af20 tx=0x7fa24c06d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c09d473228fe8e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa241ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa241ffb640 1 -- 172.21.15.18:0/2682644275 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c09af90 con 0x55561028dd60 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa241ffb640 1 -- 172.21.15.18:0/2682644275 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa24c09f0e0 con 0x55561028dd60 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa263664740 1 -- 172.21.15.18:0/2682644275 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa244004030 con 0x55561028dd60 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa241ffb640 1 -- 172.21.15.18:0/2682644275 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa24c06a080 con 0x55561028dd60 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa263664740 10 client.19509 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa263664740 1 -- 172.21.15.18:0/2682644275 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5556101b6540 con 0x55561028dd60 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa241ffb640 1 -- 172.21.15.18:0/2682644275 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa244004030 con 0x55561028dd60 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa241ffb640 1 -- 172.21.15.18:0/2682644275 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa24c01f060 con 0x55561028dd60 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa263664740 10 client.19509 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa263664740 10 client.19509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa263664740 20 client.19509 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa263664740 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555610303a70 0x555610305e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa263664740 4 client.19509 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.982+0000 7fa263664740 1 -- 172.21.15.18:0/2682644275 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55560ff32550 con 0x555610303a70 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.983+0000 7fa251d9b640 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555610303a70 0x555610305e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.983+0000 7fa251d9b640 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555610303a70 0x555610305e90 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25fa4d5d0ff19e23 server_cookie=d6260da4dffeccd9 in_seq=0 out_seq=0 2024-02-17T13:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:02.983+0000 7fa241ffb640 10 client.19509 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.028+0000 7fa241ffb640 1 -- 172.21.15.18:0/2682644275 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55560ff32550 con 0x555610303a70 2024-02-17T13:01:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.028+0000 7fa241ffb640 10 client.19509 handle_command_reply: tid=0 2024-02-17T13:01:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.028+0000 7fa263664740 1 client.19509 shutdown 2024-02-17T13:01:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.028+0000 7fa263664740 1 -- 172.21.15.18:0/2682644275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028dd60 msgr2=0x5556102a1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.028+0000 7fa263664740 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028dd60 0x5556102a1070 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa24c09af20 tx=0x7fa24c06d010 comp rx=0 tx=0).stop 2024-02-17T13:01:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.028+0000 7fa263664740 1 -- 172.21.15.18:0/2682644275 shutdown_connections 2024-02-17T13:01:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.028+0000 7fa263664740 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555610303a70 0x555610305e90 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.028+0000 7fa263664740 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55561029b6f0 0x5556102a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.028+0000 7fa241ffb640 0 client.19509 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.028+0000 7fa263664740 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55561028ead0 0x5556102a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.028+0000 7fa263664740 1 --2- 172.21.15.18:0/2682644275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55561028dd60 0x5556102a1070 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.028+0000 7fa263664740 1 -- 172.21.15.18:0/2682644275 >> 172.21.15.18:0/2682644275 conn(0x555610255550 msgr2=0x555610298f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa263664740 1 -- 172.21.15.18:0/2682644275 shutdown_connections 2024-02-17T13:01:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa263664740 1 -- 172.21.15.18:0/2682644275 wait complete. 2024-02-17T13:01:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa263664740 20 client.19509 trim_cache size 0 max 16384 2024-02-17T13:01:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:01:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:01:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:01:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:01:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa260a88640 1 -- 172.21.15.18:0/296910198 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa2480a22d0 msgr2=0x7fa2480a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa260a88640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa2480a22d0 0x7fa2480a4790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa254007310 tx=0x7fa2540072a0 comp rx=0 tx=0).stop 2024-02-17T13:01:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa260a88640 1 -- 172.21.15.18:0/296910198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa25c1002d0 msgr2=0x7fa25c197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa260a88640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa25c1002d0 0x7fa25c197760 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00e9f0 tx=0x7fa24c00edf0 comp rx=0 tx=0).stop 2024-02-17T13:01:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa260a88640 1 -- 172.21.15.18:0/296910198 shutdown_connections 2024-02-17T13:01:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa260a88640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa2480a22d0 0x7fa2480a4790 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa260a88640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa25c105020 0x7fa25c19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa260a88640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa25c100c90 0x7fa25c197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa260a88640 1 --2- 172.21.15.18:0/296910198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa25c1002d0 0x7fa25c197760 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.029+0000 7fa260a88640 1 -- 172.21.15.18:0/296910198 >> 172.21.15.18:0/296910198 conn(0x7fa25c076ed0 msgr2=0x7fa25c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.030+0000 7fa260a88640 1 -- 172.21.15.18:0/296910198 shutdown_connections 2024-02-17T13:01:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:03.030+0000 7fa260a88640 1 -- 172.21.15.18:0/296910198 wait complete. 2024-02-17T13:01:03.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 555852 4.90 MB/sec execute 498 sec latency 2712.216 ms 2024-02-17T13:01:04.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:03 smithi146 ceph-mon[36454]: pgmap v349: 97 pgs: 97 active+clean; 218 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T13:01:04.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:03 smithi018 ceph-mon[32625]: pgmap v349: 97 pgs: 97 active+clean; 218 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T13:01:04.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 556446 4.90 MB/sec execute 499 sec latency 3318.776 ms 2024-02-17T13:01:04.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:03 smithi119 ceph-mon[36188]: pgmap v349: 97 pgs: 97 active+clean; 218 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-02-17T13:01:05.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 557969 4.91 MB/sec execute 500 sec latency 26.630 ms 2024-02-17T13:01:06.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:05 smithi146 ceph-mon[36454]: pgmap v350: 97 pgs: 97 active+clean; 216 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-02-17T13:01:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:05 smithi018 ceph-mon[32625]: pgmap v350: 97 pgs: 97 active+clean; 216 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-02-17T13:01:06.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 559235 4.91 MB/sec execute 501 sec latency 42.770 ms 2024-02-17T13:01:06.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:05 smithi119 ceph-mon[36188]: pgmap v350: 97 pgs: 97 active+clean; 216 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-02-17T13:01:07.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:06 smithi146 ceph-mon[36454]: osdmap e101: 12 total, 12 up, 12 in 2024-02-17T13:01:07.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:06 smithi018 ceph-mon[32625]: osdmap e101: 12 total, 12 up, 12 in 2024-02-17T13:01:07.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 559397 4.90 MB/sec execute 502 sec latency 908.359 ms 2024-02-17T13:01:07.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:06 smithi119 ceph-mon[36188]: osdmap e101: 12 total, 12 up, 12 in 2024-02-17T13:01:08.194 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:07 smithi146 ceph-mon[36454]: pgmap v352: 97 pgs: 97 active+clean; 216 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-02-17T13:01:08.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:07 smithi018 ceph-mon[32625]: pgmap v352: 97 pgs: 97 active+clean; 216 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-02-17T13:01:08.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 559397 4.89 MB/sec execute 503 sec latency 1908.485 ms 2024-02-17T13:01:08.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:07 smithi119 ceph-mon[36188]: pgmap v352: 97 pgs: 97 active+clean; 216 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-02-17T13:01:08.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:01:08.671 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:01:08.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.796+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1528874184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc74106160 msgr2=0x7fcc74106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:08.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.796+0000 7fcc7b2df640 1 --2- 172.21.15.18:0/1528874184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc74106160 0x7fcc74106540 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009890 tx=0x7fcc6402bb70 comp rx=0 tx=0).stop 2024-02-17T13:01:08.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.796+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1528874184 shutdown_connections 2024-02-17T13:01:08.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.796+0000 7fcc7b2df640 1 --2- 172.21.15.18:0/1528874184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc74100040 0x7fcc7410e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.796+0000 7fcc7b2df640 1 --2- 172.21.15.18:0/1528874184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc740ff680 0x7fcc740ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.796+0000 7fcc7b2df640 1 --2- 172.21.15.18:0/1528874184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc74106160 0x7fcc74106540 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.796+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1528874184 >> 172.21.15.18:0/1528874184 conn(0x7fcc740765a0 msgr2=0x7fcc740fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:08.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.796+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1528874184 shutdown_connections 2024-02-17T13:01:08.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.796+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1528874184 wait complete. 2024-02-17T13:01:08.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.796+0000 7fcc7b2df640 1 Processor -- start 2024-02-17T13:01:08.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.796+0000 7fcc7b2df640 1 -- start start 2024-02-17T13:01:08.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7b2df640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc740ff680 0x7fcc7419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7b2df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc74100040 0x7fcc7419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7b2df640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc74106160 0x7fcc74199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7b2df640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc74199ec0 con 0x7fcc74100040 2024-02-17T13:01:08.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7b2df640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7419a030 con 0x7fcc74106160 2024-02-17T13:01:08.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7b2df640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7419a1f0 con 0x7fcc740ff680 2024-02-17T13:01:08.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7887a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc74100040 0x7fcc7419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7887a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc74100040 0x7fcc7419fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44782/0 (socket says 172.21.15.18:44782) 2024-02-17T13:01:08.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7907b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc740ff680 0x7fcc7419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7887a640 1 -- 172.21.15.18:0/1329210576 learned_addr learned my addr 172.21.15.18:0/1329210576 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:08.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7987c640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc74106160 0x7fcc74199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7887a640 1 -- 172.21.15.18:0/1329210576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc740ff680 msgr2=0x7fcc7419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7887a640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc740ff680 0x7fcc7419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7887a640 1 -- 172.21.15.18:0/1329210576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc74106160 msgr2=0x7fcc74199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7887a640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc74106160 0x7fcc74199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7887a640 1 -- 172.21.15.18:0/1329210576 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c009900 con 0x7fcc74100040 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7887a640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc74100040 0x7fcc7419fc00 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000e9f0 tx=0x7fcc7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc6a7fc640 1 -- 172.21.15.18:0/1329210576 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70009700 con 0x7fcc74100040 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.797+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1329210576 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc64009490 con 0x7fcc74100040 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.798+0000 7fcc6a7fc640 1 -- 172.21.15.18:0/1329210576 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc7000cd80 con 0x7fcc74100040 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.798+0000 7fcc6a7fc640 1 -- 172.21.15.18:0/1329210576 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70018920 con 0x7fcc74100040 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.798+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1329210576 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7419a6c0 con 0x7fcc74100040 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.800+0000 7fcc6a7fc640 1 -- 172.21.15.18:0/1329210576 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcc70018a80 con 0x7fcc74100040 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.800+0000 7fcc7df94740 1 Processor -- start 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.800+0000 7fcc7df94740 1 -- start start 2024-02-17T13:01:08.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7df94740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767f99690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7df94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7df94740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767fa4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7df94740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564767f08280 con 0x564767fa27b0 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7df94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564767f057b0 con 0x564767f99bd0 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7df94740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564767f00f90 con 0x564767f992b0 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc6a7fc640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc600a2230 0x7fcc600a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7887a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767fa4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7907b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7987c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767f99690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7907b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:40702/0 (socket says 172.21.15.18:40702) 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7887a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767fa4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44792/0 (socket says 172.21.15.18:44792) 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7907b640 1 -- 172.21.15.18:0/662931490 learned_addr learned my addr 172.21.15.18:0/662931490 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:08.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7987c640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc600a2230 0x7fcc600a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc6a7fc640 1 -- 172.21.15.18:0/1329210576 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fcc70014070 con 0x7fcc74100040 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7907b640 1 -- 172.21.15.18:0/662931490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 msgr2=0x564767f99690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7907b640 1 --2- 172.21.15.18:0/662931490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767f99690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7907b640 1 -- 172.21.15.18:0/662931490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 msgr2=0x564767fa4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7907b640 1 --2- 172.21.15.18:0/662931490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767fa4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7907b640 1 -- 172.21.15.18:0/662931490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564767ebf570 con 0x564767f99bd0 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.801+0000 7fcc7987c640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc600a2230 0x7fcc600a46f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c005fd0 tx=0x7fcc6c0095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.802+0000 7fcc7907b640 1 --2- 172.21.15.18:0/662931490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa2270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcc640097b0 tx=0x7fcc640323c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b07cf3aaf4ff59 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.802+0000 7fcc5f7fe640 1 -- 172.21.15.18:0/662931490 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64004030 con 0x564767f99bd0 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.802+0000 7fcc5f7fe640 1 -- 172.21.15.18:0/662931490 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc64034440 con 0x564767f99bd0 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.802+0000 7fcc5f7fe640 1 -- 172.21.15.18:0/662931490 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6402c9f0 con 0x564767f99bd0 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.802+0000 7fcc7df94740 1 -- 172.21.15.18:0/662931490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 msgr2=0x564767fa2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.802+0000 7fcc7df94740 1 --2- 172.21.15.18:0/662931490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa2270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcc640097b0 tx=0x7fcc640323c0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.802+0000 7fcc7df94740 1 -- 172.21.15.18:0/662931490 shutdown_connections 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.802+0000 7fcc7df94740 1 --2- 172.21.15.18:0/662931490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767fa4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.802+0000 7fcc7df94740 1 --2- 172.21.15.18:0/662931490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa2270 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.802+0000 7fcc7df94740 1 --2- 172.21.15.18:0/662931490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767f99690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.802+0000 7fcc7df94740 1 -- 172.21.15.18:0/662931490 >> 172.21.15.18:0/662931490 conn(0x564767f93990 msgr2=0x564767f93d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7df94740 1 -- 172.21.15.18:0/662931490 shutdown_connections 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7df94740 1 -- 172.21.15.18:0/662931490 wait complete. 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7df94740 1 -- start start 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7df94740 1 Processor -- start 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7df94740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767fa2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7df94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767f9f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7df94740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767f9fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7df94740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564767bd4430 con 0x564767fa27b0 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7df94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564767b90e00 con 0x564767f99bd0 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7df94740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564767e85fd0 con 0x564767f992b0 2024-02-17T13:01:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7df94740 10 client.0 _ll_register_callbacks cb 0x564767edbfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7987c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767f9fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7987c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767f9fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44796/0 (socket says 172.21.15.18:44796) 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7987c640 1 -- 172.21.15.18:0/3851755881 learned_addr learned my addr 172.21.15.18:0/3851755881 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.803+0000 7fcc7887a640 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767f9f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc7907b640 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767fa2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc7987c640 1 -- 172.21.15.18:0/3851755881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 msgr2=0x564767fa2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc7987c640 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767fa2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc7987c640 1 -- 172.21.15.18:0/3851755881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 msgr2=0x564767f9f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc7987c640 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767f9f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc7987c640 1 -- 172.21.15.18:0/3851755881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7410e990 con 0x564767fa27b0 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc7887a640 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767f9f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc7987c640 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767f9fa50 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c003f20 tx=0x7fcc6c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f6c5196032c71e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:08.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc5cff9640 1 -- 172.21.15.18:0/3851755881 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c013030 con 0x564767fa27b0 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc5cff9640 1 -- 172.21.15.18:0/3851755881 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fcc6c004b20 con 0x564767fa27b0 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc7df94740 1 -- 172.21.15.18:0/3851755881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564767c007e0 con 0x564767fa27b0 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc5cff9640 1 -- 172.21.15.18:0/3851755881 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc6c004ce0 con 0x564767fa27b0 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc7df94740 10 client.19521 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.804+0000 7fcc7df94740 1 -- 172.21.15.18:0/3851755881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564767eff610 con 0x564767fa27b0 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.805+0000 7fcc5cff9640 1 -- 172.21.15.18:0/3851755881 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564767c007e0 con 0x564767fa27b0 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.805+0000 7fcc5cff9640 1 -- 172.21.15.18:0/3851755881 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcc6c01f810 con 0x564767fa27b0 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.805+0000 7fcc7df94740 10 client.19521 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.805+0000 7fcc7df94740 10 client.19521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.805+0000 7fcc7df94740 20 client.19521 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.805+0000 7fcc7df94740 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564767fb2990 0x564767fb4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.805+0000 7fcc7df94740 4 client.19521 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:08.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.805+0000 7fcc7df94740 1 -- 172.21.15.18:0/3851755881 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564767c3d720 con 0x564767fb2990 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.805+0000 7fcc7907b640 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564767fb2990 0x564767fb4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.805+0000 7fcc5cff9640 10 client.19521 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.805+0000 7fcc7907b640 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564767fb2990 0x564767fb4db0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1308423e3a4976be server_cookie=bdc5a2eafd8228c7 in_seq=0 out_seq=0 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc5cff9640 1 -- 172.21.15.18:0/3851755881 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564767c3d720 con 0x564767fb2990 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc5cff9640 10 client.19521 handle_command_reply: tid=0 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc7df94740 1 client.19521 shutdown 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc7df94740 1 -- 172.21.15.18:0/3851755881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 msgr2=0x564767f9fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc7df94740 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767f9fa50 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c003f20 tx=0x7fcc6c004ab0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc7df94740 1 -- 172.21.15.18:0/3851755881 shutdown_connections 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc7df94740 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564767fb2990 0x564767fb4db0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc7df94740 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767f9fa50 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc7df94740 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767f9f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc7df94740 1 --2- 172.21.15.18:0/3851755881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767fa2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc5cff9640 0 client.19521 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:08.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc7df94740 1 -- 172.21.15.18:0/3851755881 >> 172.21.15.18:0/3851755881 conn(0x564767f93990 msgr2=0x564767f95730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:08.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc7df94740 1 -- 172.21.15.18:0/3851755881 shutdown_connections 2024-02-17T13:01:08.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.806+0000 7fcc7df94740 1 -- 172.21.15.18:0/3851755881 wait complete. 2024-02-17T13:01:08.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.807+0000 7fcc7df94740 20 client.19521 trim_cache size 0 max 16384 2024-02-17T13:01:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.821+0000 7fcc7df94740 1 Processor -- start 2024-02-17T13:01:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.821+0000 7fcc7df94740 1 -- start start 2024-02-17T13:01:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7df94740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767fa4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7df94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7df94740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767fa2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7df94740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564767f98f10 con 0x564767fa27b0 2024-02-17T13:01:08.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7df94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564767f949e0 con 0x564767f99bd0 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7df94740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564767f9b0a0 con 0x564767f992b0 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7987c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767fa2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7987c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767fa2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44802/0 (socket says 172.21.15.18:44802) 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7987c640 1 -- 172.21.15.18:0/2031491892 learned_addr learned my addr 172.21.15.18:0/2031491892 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7907b640 1 --2- 172.21.15.18:0/2031491892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767fa4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7887a640 1 --2- 172.21.15.18:0/2031491892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7987c640 1 -- 172.21.15.18:0/2031491892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 msgr2=0x564767fa4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7987c640 1 --2- 172.21.15.18:0/2031491892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767fa4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7987c640 1 -- 172.21.15.18:0/2031491892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 msgr2=0x564767fa1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7987c640 1 --2- 172.21.15.18:0/2031491892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7987c640 1 -- 172.21.15.18:0/2031491892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564767e9bac0 con 0x564767fa27b0 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc7987c640 1 --2- 172.21.15.18:0/2031491892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767fa2140 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c010010 tx=0x7fcc6c003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e66ee3775d8dff server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc5effd640 1 -- 172.21.15.18:0/2031491892 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c00e680 con 0x564767fa27b0 2024-02-17T13:01:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.822+0000 7fcc5effd640 1 -- 172.21.15.18:0/2031491892 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc6c014040 con 0x564767fa27b0 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.823+0000 7fcc5effd640 1 -- 172.21.15.18:0/2031491892 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c020d50 con 0x564767fa27b0 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.823+0000 7fcc7df94740 1 -- 172.21.15.18:0/2031491892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 msgr2=0x564767fa2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.823+0000 7fcc7df94740 1 --2- 172.21.15.18:0/2031491892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767fa2140 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c010010 tx=0x7fcc6c003c00 comp rx=0 tx=0).stop 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.823+0000 7fcc7df94740 1 -- 172.21.15.18:0/2031491892 shutdown_connections 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.823+0000 7fcc7df94740 1 --2- 172.21.15.18:0/2031491892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564767fa2140 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.823+0000 7fcc7df94740 1 --2- 172.21.15.18:0/2031491892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.823+0000 7fcc7df94740 1 --2- 172.21.15.18:0/2031491892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767fa4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.823+0000 7fcc7df94740 1 -- 172.21.15.18:0/2031491892 >> 172.21.15.18:0/2031491892 conn(0x564767f93990 msgr2=0x564767fb9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.823+0000 7fcc7df94740 1 -- 172.21.15.18:0/2031491892 shutdown_connections 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.823+0000 7fcc7df94740 1 -- 172.21.15.18:0/2031491892 wait complete. 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7df94740 1 -- start start 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7df94740 1 Processor -- start 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7df94740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767fa5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7df94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7df94740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564768007e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7df94740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564767fa6470 con 0x564767fa27b0 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7df94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647680083f0 con 0x564767f99bd0 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7df94740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647680085b0 con 0x564767f992b0 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7df94740 10 client.0 _ll_register_callbacks cb 0x7fcc6c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7987c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564768007e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7987c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564768007e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44806/0 (socket says 172.21.15.18:44806) 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7987c640 1 -- 172.21.15.18:0/2844216854 learned_addr learned my addr 172.21.15.18:0/2844216854 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7907b640 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767fa5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7887a640 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7987c640 1 -- 172.21.15.18:0/2844216854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 msgr2=0x564767fa5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7987c640 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767fa5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7987c640 1 -- 172.21.15.18:0/2844216854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 msgr2=0x564767fa5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7987c640 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.824+0000 7fcc7987c640 1 -- 172.21.15.18:0/2844216854 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564767eff610 con 0x564767fa27b0 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc7987c640 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564768007e80 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c005c10 tx=0x7fcc6c0207e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5cffb4b5bef9cc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc5cff9640 1 -- 172.21.15.18:0/2844216854 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c01f650 con 0x564767fa27b0 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc5cff9640 1 -- 172.21.15.18:0/2844216854 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fcc6c01f7b0 con 0x564767fa27b0 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc7df94740 1 -- 172.21.15.18:0/2844216854 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc64034440 con 0x564767fa27b0 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc5cff9640 1 -- 172.21.15.18:0/2844216854 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc6c00e700 con 0x564767fa27b0 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc7df94740 10 client.19527 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc7df94740 1 -- 172.21.15.18:0/2844216854 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564767ec16a0 con 0x564767fa27b0 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc5cff9640 1 -- 172.21.15.18:0/2844216854 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc64034440 con 0x564767fa27b0 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc5cff9640 1 -- 172.21.15.18:0/2844216854 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcc6c005370 con 0x564767fa27b0 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc7df94740 10 client.19527 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc7df94740 10 client.19527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc7df94740 20 client.19527 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.825+0000 7fcc7df94740 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564768009b30 0x56476800bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.826+0000 7fcc7df94740 4 client.19527 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.826+0000 7fcc7df94740 1 -- 172.21.15.18:0/2844216854 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564767c8c890 con 0x564768009b30 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.826+0000 7fcc7907b640 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564768009b30 0x56476800bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.826+0000 7fcc7907b640 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564768009b30 0x56476800bf50 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=980d9caf6b1f80be server_cookie=ce3d3b96f334e2b0 in_seq=0 out_seq=0 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.826+0000 7fcc5cff9640 10 client.19527 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.826+0000 7fcc5cff9640 1 -- 172.21.15.18:0/2844216854 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564767c8c890 con 0x564768009b30 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.826+0000 7fcc5cff9640 10 client.19527 handle_command_reply: tid=0 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.826+0000 7fcc7df94740 1 client.19527 shutdown 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.826+0000 7fcc7df94740 1 -- 172.21.15.18:0/2844216854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 msgr2=0x564768007e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.826+0000 7fcc7df94740 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564768007e80 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c005c10 tx=0x7fcc6c0207e0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7df94740 1 -- 172.21.15.18:0/2844216854 shutdown_connections 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7df94740 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564768009b30 0x56476800bf50 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7df94740 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564767fa27b0 0x564768007e80 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7df94740 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564767f99bd0 0x564767fa5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7df94740 1 --2- 172.21.15.18:0/2844216854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564767f992b0 0x564767fa5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc5cff9640 0 client.19527 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:08.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7df94740 1 -- 172.21.15.18:0/2844216854 >> 172.21.15.18:0/2844216854 conn(0x564767f93990 msgr2=0x564767f9dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:08.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7df94740 1 -- 172.21.15.18:0/2844216854 shutdown_connections 2024-02-17T13:01:08.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7df94740 1 -- 172.21.15.18:0/2844216854 wait complete. 2024-02-17T13:01:08.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7df94740 20 client.19527 trim_cache size 0 max 16384 2024-02-17T13:01:08.824 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:01:08.824 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T13:01:08.824 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T13:01:08.825 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:01:08.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1329210576 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc600a2230 msgr2=0x7fcc600a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:08.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7b2df640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc600a2230 0x7fcc600a46f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c005fd0 tx=0x7fcc6c0095c0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1329210576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc74100040 msgr2=0x7fcc7419fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:08.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7b2df640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc74100040 0x7fcc7419fc00 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000e9f0 tx=0x7fcc7000eeb0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1329210576 shutdown_connections 2024-02-17T13:01:08.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7b2df640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcc600a2230 0x7fcc600a46f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7b2df640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc74106160 0x7fcc74199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7b2df640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc74100040 0x7fcc7419fc00 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7b2df640 1 --2- 172.21.15.18:0/1329210576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc740ff680 0x7fcc7419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.827+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1329210576 >> 172.21.15.18:0/1329210576 conn(0x7fcc740765a0 msgr2=0x7fcc74104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:08.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.828+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1329210576 shutdown_connections 2024-02-17T13:01:08.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.828+0000 7fcc7b2df640 1 -- 172.21.15.18:0/1329210576 wait complete. 2024-02-17T13:01:08.832 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T13:01:08.832 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T13:01:08.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.950+0000 7fed27e15640 1 -- 172.21.15.18:0/3836494463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed2010b7c0 msgr2=0x7fed2010dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:08.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.950+0000 7fed27e15640 1 --2- 172.21.15.18:0/3836494463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed2010b7c0 0x7fed2010dc60 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fed18009ca0 tx=0x7fed1802c0f0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.950+0000 7fed27e15640 1 -- 172.21.15.18:0/3836494463 shutdown_connections 2024-02-17T13:01:08.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.950+0000 7fed27e15640 1 --2- 172.21.15.18:0/3836494463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed2010b7c0 0x7fed2010dc60 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.950+0000 7fed27e15640 1 --2- 172.21.15.18:0/3836494463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed20102d10 0x7fed2010b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.950+0000 7fed27e15640 1 --2- 172.21.15.18:0/3836494463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed201023f0 0x7fed201027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.950+0000 7fed27e15640 1 -- 172.21.15.18:0/3836494463 >> 172.21.15.18:0/3836494463 conn(0x7fed20077180 msgr2=0x7fed20077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:08.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.950+0000 7fed27e15640 1 -- 172.21.15.18:0/3836494463 shutdown_connections 2024-02-17T13:01:08.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.950+0000 7fed27e15640 1 -- 172.21.15.18:0/3836494463 wait complete. 2024-02-17T13:01:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.951+0000 7fed27e15640 1 Processor -- start 2024-02-17T13:01:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.951+0000 7fed27e15640 1 -- start start 2024-02-17T13:01:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed27e15640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed201023f0 0x7fed2019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed27e15640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed20102d10 0x7fed2019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed27e15640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed2010b7c0 0x7fed201975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed27e15640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed20197bf0 con 0x7fed2010b7c0 2024-02-17T13:01:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed27e15640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed20197d60 con 0x7fed20102d10 2024-02-17T13:01:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed27e15640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed20197f20 con 0x7fed201023f0 2024-02-17T13:01:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed263b2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed2010b7c0 0x7fed201975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed253b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed20102d10 0x7fed2019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed25bb1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed201023f0 0x7fed2019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed253b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed20102d10 0x7fed2019d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:40768/0 (socket says 172.21.15.18:40768) 2024-02-17T13:01:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed253b0640 1 -- 172.21.15.18:0/3275639975 learned_addr learned my addr 172.21.15.18:0/3275639975 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed253b0640 1 -- 172.21.15.18:0/3275639975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed201023f0 msgr2=0x7fed2019d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed253b0640 1 --2- 172.21.15.18:0/3275639975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed201023f0 0x7fed2019d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed253b0640 1 -- 172.21.15.18:0/3275639975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed2010b7c0 msgr2=0x7fed201975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed253b0640 1 --2- 172.21.15.18:0/3275639975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed2010b7c0 0x7fed201975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed253b0640 1 -- 172.21.15.18:0/3275639975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed18009900 con 0x7fed20102d10 2024-02-17T13:01:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.952+0000 7fed253b0640 1 --2- 172.21.15.18:0/3275639975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed20102d10 0x7fed2019d930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fed1000e9f0 tx=0x7fed1000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.953+0000 7fed16ffd640 1 -- 172.21.15.18:0/3275639975 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10009700 con 0x7fed20102d10 2024-02-17T13:01:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.953+0000 7fed27e15640 1 -- 172.21.15.18:0/3275639975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed20198200 con 0x7fed20102d10 2024-02-17T13:01:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.953+0000 7fed16ffd640 1 -- 172.21.15.18:0/3275639975 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed1000cd80 con 0x7fed20102d10 2024-02-17T13:01:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.953+0000 7fed16ffd640 1 -- 172.21.15.18:0/3275639975 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10018920 con 0x7fed20102d10 2024-02-17T13:01:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.953+0000 7fed27e15640 1 -- 172.21.15.18:0/3275639975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed201a4490 con 0x7fed20102d10 2024-02-17T13:01:08.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.954+0000 7fed27e15640 1 -- 172.21.15.18:0/3275639975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed20067f70 con 0x7fed20102d10 2024-02-17T13:01:08.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.955+0000 7fed16ffd640 1 -- 172.21.15.18:0/3275639975 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fed10018a80 con 0x7fed20102d10 2024-02-17T13:01:08.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.956+0000 7fed16ffd640 1 --2- 172.21.15.18:0/3275639975 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fed0c0a2230 0x7fed0c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:08.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.956+0000 7fed25bb1640 1 --2- 172.21.15.18:0/3275639975 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fed0c0a2230 0x7fed0c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:08.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.956+0000 7fed16ffd640 1 -- 172.21.15.18:0/3275639975 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fed10014070 con 0x7fed20102d10 2024-02-17T13:01:08.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.956+0000 7fed25bb1640 1 --2- 172.21.15.18:0/3275639975 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fed0c0a2230 0x7fed0c0a46f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009bd0 tx=0x7fed1c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:08.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:08.964+0000 7fed16ffd640 1 -- 172.21.15.18:0/3275639975 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed100970e0 con 0x7fed20102d10 2024-02-17T13:01:09.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:08 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:01:09.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:08 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:01:09.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.209+0000 7fed27e15640 1 -- 172.21.15.18:0/3275639975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed20198ad0 con 0x7fed20102d10 2024-02-17T13:01:09.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.210+0000 7fed16ffd640 1 -- 172.21.15.18:0/3275639975 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+11395 (secure 0 0 0) 0x7fed1009a7a0 con 0x7fed20102d10 2024-02-17T13:01:09.201 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:01:09.201 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T13:00:57.852826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T13:01:09.201 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 49 2024-02-17T13:01:09.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.212+0000 7fed27e15640 1 -- 172.21.15.18:0/3275639975 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fed0c0a2230 msgr2=0x7fed0c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.212+0000 7fed27e15640 1 --2- 172.21.15.18:0/3275639975 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fed0c0a2230 0x7fed0c0a46f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009bd0 tx=0x7fed1c005f90 comp rx=0 tx=0).stop 2024-02-17T13:01:09.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.212+0000 7fed27e15640 1 -- 172.21.15.18:0/3275639975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed20102d10 msgr2=0x7fed2019d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.212+0000 7fed27e15640 1 --2- 172.21.15.18:0/3275639975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed20102d10 0x7fed2019d930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fed1000e9f0 tx=0x7fed1000eeb0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.212+0000 7fed27e15640 1 -- 172.21.15.18:0/3275639975 shutdown_connections 2024-02-17T13:01:09.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.212+0000 7fed27e15640 1 --2- 172.21.15.18:0/3275639975 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fed0c0a2230 0x7fed0c0a46f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.212+0000 7fed27e15640 1 --2- 172.21.15.18:0/3275639975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed2010b7c0 0x7fed201975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.212+0000 7fed27e15640 1 --2- 172.21.15.18:0/3275639975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed20102d10 0x7fed2019d930 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.212+0000 7fed27e15640 1 --2- 172.21.15.18:0/3275639975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed201023f0 0x7fed2019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.213+0000 7fed27e15640 1 -- 172.21.15.18:0/3275639975 >> 172.21.15.18:0/3275639975 conn(0x7fed20077180 msgr2=0x7fed200fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.213+0000 7fed27e15640 1 -- 172.21.15.18:0/3275639975 shutdown_connections 2024-02-17T13:01:09.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.213+0000 7fed27e15640 1 -- 172.21.15.18:0/3275639975 wait complete. 2024-02-17T13:01:09.215 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:01:09.215 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T13:01:09.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 560262 4.89 MB/sec execute 504 sec latency 2513.186 ms 2024-02-17T13:01:09.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.340+0000 7f7ce31e9640 1 -- 172.21.15.18:0/2958001953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc102d10 msgr2=0x7f7cdc10b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.340+0000 7f7ce31e9640 1 --2- 172.21.15.18:0/2958001953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc102d10 0x7f7cdc10b1f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009860 tx=0x7f7cc802bc00 comp rx=0 tx=0).stop 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.340+0000 7f7ce31e9640 1 -- 172.21.15.18:0/2958001953 shutdown_connections 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.340+0000 7f7ce31e9640 1 --2- 172.21.15.18:0/2958001953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cdc10b7c0 0x7f7cdc10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.340+0000 7f7ce31e9640 1 --2- 172.21.15.18:0/2958001953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc102d10 0x7f7cdc10b1f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.340+0000 7f7ce31e9640 1 --2- 172.21.15.18:0/2958001953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cdc1023f0 0x7f7cdc1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.340+0000 7f7ce31e9640 1 -- 172.21.15.18:0/2958001953 >> 172.21.15.18:0/2958001953 conn(0x7f7cdc0771e0 msgr2=0x7f7cdc0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.340+0000 7f7ce31e9640 1 -- 172.21.15.18:0/2958001953 shutdown_connections 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.340+0000 7f7ce31e9640 1 -- 172.21.15.18:0/2958001953 wait complete. 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7ce31e9640 1 Processor -- start 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7ce31e9640 1 -- start start 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7ce31e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cdc1023f0 0x7f7cdc100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7ce31e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cdc102d10 0x7f7cdc100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7ce31e9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc10b7c0 0x7f7cdc100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7ce31e9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc1a5ba0 con 0x7f7cdc1023f0 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7ce31e9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc1a5d10 con 0x7f7cdc102d10 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7ce31e9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc1a5ed0 con 0x7f7cdc10b7c0 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7ce0f85640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cdc1023f0 0x7f7cdc100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7ce0f85640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cdc1023f0 0x7f7cdc100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44834/0 (socket says 172.21.15.18:44834) 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7ce0f85640 1 -- 172.21.15.18:0/3040885607 learned_addr learned my addr 172.21.15.18:0/3040885607 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7ce1786640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc10b7c0 0x7f7cdc100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.341+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cdc102d10 0x7f7cdc100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7ce1786640 1 -- 172.21.15.18:0/3040885607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cdc102d10 msgr2=0x7f7cdc100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7ce1786640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cdc102d10 0x7f7cdc100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7ce1786640 1 -- 172.21.15.18:0/3040885607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cdc1023f0 msgr2=0x7f7cdc100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7ce1786640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cdc1023f0 0x7f7cdc100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7ce1786640 1 -- 172.21.15.18:0/3040885607 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc8009490 con 0x7f7cdc10b7c0 2024-02-17T13:01:09.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7ce0f85640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cdc1023f0 0x7f7cdc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cdc102d10 0x7f7cdc100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7ce1786640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc10b7c0 0x7f7cdc100ac0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009fd0 tx=0x7f7cd400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7cd9ffb640 1 -- 172.21.15.18:0/3040885607 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd400ce20 con 0x7f7cdc10b7c0 2024-02-17T13:01:09.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7cd9ffb640 1 -- 172.21.15.18:0/3040885607 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cd4005020 con 0x7f7cdc10b7c0 2024-02-17T13:01:09.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7ce31e9640 1 -- 172.21.15.18:0/3040885607 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cdc1a61b0 con 0x7f7cdc10b7c0 2024-02-17T13:01:09.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7cd9ffb640 1 -- 172.21.15.18:0/3040885607 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4010690 con 0x7f7cdc10b7c0 2024-02-17T13:01:09.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.342+0000 7f7ce31e9640 1 -- 172.21.15.18:0/3040885607 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cdc1a6690 con 0x7f7cdc10b7c0 2024-02-17T13:01:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.345+0000 7f7cd9ffb640 1 -- 172.21.15.18:0/3040885607 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7cd4005500 con 0x7f7cdc10b7c0 2024-02-17T13:01:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.345+0000 7f7ce5dbf740 1 Processor -- start 2024-02-17T13:01:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.345+0000 7f7ce5dbf740 1 -- start start 2024-02-17T13:01:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.345+0000 7f7ce5dbf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648a2250 0x5618648a2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce5dbf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2b70 0x5618648ab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce5dbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648ab750 0x5618648adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce5dbf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56186485d960 con 0x5618648a2b70 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce5dbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561864811310 con 0x5618648ab750 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce5dbf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56186480e840 con 0x5618648a2250 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce0f85640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2b70 0x5618648ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648a2250 0x5618648a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce0f85640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2b70 0x5618648ab210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44850/0 (socket says 172.21.15.18:44850) 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce0f85640 1 -- 172.21.15.18:0/2693422821 learned_addr learned my addr 172.21.15.18:0/2693422821 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7cd9ffb640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7ccc0a2260 0x7f7ccc0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/2693422821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648ab750 0x5618648adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce1786640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7ccc0a2260 0x7f7ccc0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce0f85640 1 -- 172.21.15.18:0/2693422821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648a2250 msgr2=0x5618648a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce0f85640 1 --2- 172.21.15.18:0/2693422821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648a2250 0x5618648a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce0f85640 1 -- 172.21.15.18:0/2693422821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648ab750 msgr2=0x5618648adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce0f85640 1 --2- 172.21.15.18:0/2693422821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648ab750 0x5618648adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce0f85640 1 -- 172.21.15.18:0/2693422821 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618647cc060 con 0x5618648a2b70 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/2693422821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648a2250 0x5618648a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7cd9ffb640 1 -- 172.21.15.18:0/3040885607 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f7cd4014070 con 0x7f7cdc10b7c0 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7ce0f85640 1 --2- 172.21.15.18:0/2693422821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2b70 0x5618648ab210 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00115c0 tx=0x7f7cd0011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3762a6357d922813 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.346+0000 7f7cc6ffd640 1 -- 172.21.15.18:0/2693422821 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd000f6c0 con 0x5618648a2b70 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.347+0000 7f7ce1786640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7ccc0a2260 0x7f7ccc0a4720 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7cd409adc0 tx=0x7f7cd4090790 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.347+0000 7f7cc6ffd640 1 -- 172.21.15.18:0/2693422821 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cd000fd00 con 0x5618648a2b70 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.347+0000 7f7cc6ffd640 1 -- 172.21.15.18:0/2693422821 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0019e30 con 0x5618648a2b70 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.347+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/2693422821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2b70 msgr2=0x5618648ab210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.347+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/2693422821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2b70 0x5618648ab210 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00115c0 tx=0x7f7cd0011a80 comp rx=0 tx=0).stop 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.347+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/2693422821 shutdown_connections 2024-02-17T13:01:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.347+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/2693422821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648ab750 0x5618648adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.347+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/2693422821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2b70 0x5618648ab210 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.347+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/2693422821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648a2250 0x5618648a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.347+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/2693422821 >> 172.21.15.18:0/2693422821 conn(0x56186489c980 msgr2=0x56186489cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.347+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/2693422821 shutdown_connections 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.347+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/2693422821 wait complete. 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7ce5dbf740 1 -- start start 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7ce5dbf740 1 Processor -- start 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7ce5dbf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 0x5618648a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7ce5dbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648a2b70 0x5618648a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7ce5dbf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648ab750 0x5618648a9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7ce5dbf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56186480a020 con 0x5618648a2250 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7ce5dbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618644dd430 con 0x5618648a2b70 2024-02-17T13:01:09.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7ce5dbf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561864499e00 con 0x5618648ab750 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7ce5dbf740 10 client.0 _ll_register_callbacks cb 0x5618647b1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 0x5618648a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 0x5618648a87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44862/0 (socket says 172.21.15.18:44862) 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7cdbfff640 1 -- 172.21.15.18:0/290083111 learned_addr learned my addr 172.21.15.18:0/290083111 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.348+0000 7f7ce0f85640 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648a2b70 0x5618648a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648ab750 0x5618648a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7ce0f85640 1 -- 172.21.15.18:0/290083111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648ab750 msgr2=0x5618648a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7ce0f85640 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648ab750 0x5618648a9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7ce0f85640 1 -- 172.21.15.18:0/290083111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 msgr2=0x5618648a87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7ce0f85640 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 0x5618648a87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7ce0f85640 1 -- 172.21.15.18:0/290083111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd0009db0 con 0x5618648a2b70 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 0x5618648a87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7ce0f85640 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648a2b70 0x5618648a8d20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7cd001b040 tx=0x7f7cd0019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e03cf374024c3fd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7cbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:01:09.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7cbbfff640 1 -- 172.21.15.18:0/290083111 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0004250 con 0x5618648a2b70 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7cbbfff640 1 -- 172.21.15.18:0/290083111 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f7cd00043b0 con 0x5618648a2b70 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/290083111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618645097e0 con 0x5618648a2b70 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7cbbfff640 1 -- 172.21.15.18:0/290083111 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cd000f6c0 con 0x5618648a2b70 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7ce5dbf740 10 client.28532 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.349+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/290083111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5618648086a0 con 0x5618648a2b70 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.350+0000 7f7cbbfff640 1 -- 172.21.15.18:0/290083111 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5618645097e0 con 0x5618648a2b70 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.350+0000 7f7cbbfff640 1 -- 172.21.15.18:0/290083111 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7cd0021070 con 0x5618648a2b70 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.350+0000 7f7ce5dbf740 10 client.28532 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.350+0000 7f7ce5dbf740 10 client.28532 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.350+0000 7f7ce5dbf740 20 client.28532 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.350+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5618648bbb80 0x5618648bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.350+0000 7f7ce5dbf740 4 client.28532 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.350+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/290083111 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561864546720 con 0x5618648bbb80 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.350+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5618648bbb80 0x5618648bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7cbbfff640 10 client.28532 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:01:09.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5618648bbb80 0x5618648bdfa0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aac122a0308e9e97 server_cookie=ad05e16f8611f766 in_seq=0 out_seq=0 2024-02-17T13:01:09.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:08 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:01:09.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7cbbfff640 1 -- 172.21.15.18:0/290083111 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f7cc8056a90 con 0x5618648bbb80 2024-02-17T13:01:09.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7cbbfff640 10 client.28532 handle_command_reply: tid=0 2024-02-17T13:01:09.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7ce5dbf740 1 client.28532 shutdown 2024-02-17T13:01:09.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/290083111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648a2b70 msgr2=0x5618648a8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648a2b70 0x5618648a8d20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7cd001b040 tx=0x7f7cd0019640 comp rx=0 tx=0).stop 2024-02-17T13:01:09.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/290083111 shutdown_connections 2024-02-17T13:01:09.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648ab750 0x5618648a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5618648bbb80 0x5618648bdfa0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648a2b70 0x5618648a8d20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7cbbfff640 0 client.28532 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:01:09.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/290083111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 0x5618648a87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.351+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/290083111 >> 172.21.15.18:0/290083111 conn(0x56186489c980 msgr2=0x56186489e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.352+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/290083111 shutdown_connections 2024-02-17T13:01:09.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.352+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/290083111 wait complete. 2024-02-17T13:01:09.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.352+0000 7f7ce5dbf740 20 client.28532 trim_cache size 0 max 16384 2024-02-17T13:01:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.364+0000 7f7ce5dbf740 1 Processor -- start 2024-02-17T13:01:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7ce5dbf740 1 -- start start 2024-02-17T13:01:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7ce5dbf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 0x5618648aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7ce5dbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648a2b70 0x5618648a9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7ce5dbf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648ab750 0x5618648a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7ce5dbf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56186478f130 con 0x5618648a2250 2024-02-17T13:01:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7ce5dbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618648ae040 con 0x5618648a2b70 2024-02-17T13:01:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7ce5dbf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618648a6d30 con 0x5618648ab750 2024-02-17T13:01:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 0x5618648aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7ce0f85640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648a2b70 0x5618648a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 0x5618648aaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44868/0 (socket says 172.21.15.18:44868) 2024-02-17T13:01:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7cdbfff640 1 -- 172.21.15.18:0/3439974953 learned_addr learned my addr 172.21.15.18:0/3439974953 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/3439974953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648ab750 0x5618648a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7cdbfff640 1 -- 172.21.15.18:0/3439974953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648ab750 msgr2=0x5618648a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/3439974953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648ab750 0x5618648a5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7cdbfff640 1 -- 172.21.15.18:0/3439974953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648a2b70 msgr2=0x5618648a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/3439974953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648a2b70 0x5618648a9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7cdbfff640 1 -- 172.21.15.18:0/3439974953 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618647a4c20 con 0x5618648a2250 2024-02-17T13:01:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.365+0000 7f7ce0f85640 1 --2- 172.21.15.18:0/3439974953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648a2b70 0x5618648a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.366+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/3439974953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 0x5618648aaf10 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7cc802c140 tx=0x7f7cc8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439e1c9af458e1c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.366+0000 7f7cc67fc640 1 -- 172.21.15.18:0/3439974953 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc804cd10 con 0x5618648a2250 2024-02-17T13:01:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.366+0000 7f7cc67fc640 1 -- 172.21.15.18:0/3439974953 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cc8052050 con 0x5618648a2250 2024-02-17T13:01:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.366+0000 7f7cc67fc640 1 -- 172.21.15.18:0/3439974953 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc80035f0 con 0x5618648a2250 2024-02-17T13:01:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.367+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3439974953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 msgr2=0x5618648aaf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.367+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/3439974953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 0x5618648aaf10 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7cc802c140 tx=0x7f7cc8005f90 comp rx=0 tx=0).stop 2024-02-17T13:01:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.367+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3439974953 shutdown_connections 2024-02-17T13:01:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.367+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/3439974953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648ab750 0x5618648a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.367+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/3439974953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5618648a2b70 0x5618648a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.367+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/3439974953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648a2250 0x5618648aaf10 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.367+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3439974953 >> 172.21.15.18:0/3439974953 conn(0x56186489c980 msgr2=0x5618648a3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.367+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3439974953 shutdown_connections 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.367+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3439974953 wait complete. 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.367+0000 7f7ce5dbf740 1 -- start start 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.367+0000 7f7ce5dbf740 1 Processor -- start 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce5dbf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648a2b70 0x5618648b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce5dbf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648ab750 0x5618648b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce5dbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56186490b070 0x56186490b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce5dbf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56186490ba30 con 0x5618648ab750 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce5dbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56186490bba0 con 0x56186490b070 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce5dbf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56186490bd60 con 0x5618648a2b70 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce5dbf740 10 client.0 _ll_register_callbacks cb 0x7f7cd0009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:09.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce0f85640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648ab750 0x5618648b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648a2b70 0x5618648b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce0f85640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648ab750 0x5618648b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44882/0 (socket says 172.21.15.18:44882) 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648a2b70 0x5618648b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:35486/0 (socket says 172.21.15.18:35486) 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce0f85640 1 -- 172.21.15.18:0/3563892371 learned_addr learned my addr 172.21.15.18:0/3563892371 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/3563892371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56186490b070 0x56186490b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce0f85640 1 -- 172.21.15.18:0/3563892371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648a2b70 msgr2=0x5618648b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce0f85640 1 --2- 172.21.15.18:0/3563892371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648a2b70 0x5618648b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce0f85640 1 -- 172.21.15.18:0/3563892371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56186490b070 msgr2=0x56186490b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce0f85640 1 --2- 172.21.15.18:0/3563892371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56186490b070 0x56186490b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce0f85640 1 -- 172.21.15.18:0/3563892371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618648086a0 con 0x5618648ab750 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7ce0f85640 1 --2- 172.21.15.18:0/3563892371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648ab750 0x5618648b1ed0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00058c0 tx=0x7f7cd0011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5adda15bae0489 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.368+0000 7f7cbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7cbbfff640 1 -- 172.21.15.18:0/3563892371 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0009c50 con 0x5618648ab750 2024-02-17T13:01:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3563892371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cd000fd00 con 0x5618648ab750 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7cbbfff640 1 -- 172.21.15.18:0/3563892371 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f7cd0000e40 con 0x5618648ab750 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7cbbfff640 1 -- 172.21.15.18:0/3563892371 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cd0003d30 con 0x5618648ab750 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7ce5dbf740 10 client.19551 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3563892371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f7cc8030070 con 0x5618648ab750 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7cbbfff640 1 -- 172.21.15.18:0/3563892371 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cd0043080 con 0x5618648ab750 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7cbbfff640 1 -- 172.21.15.18:0/3563892371 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7cd0021040 con 0x5618648ab750 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7ce5dbf740 10 client.19551 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7ce5dbf740 10 client.19551 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7ce5dbf740 20 client.19551 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/3563892371 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x561864914490 0x5618649168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7ce5dbf740 4 client.19551 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3563892371 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561864595890 con 0x561864914490 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.369+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/3563892371 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x561864914490 0x5618649168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7cbbfff640 10 client.19551 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:01:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7cdbfff640 1 --2- 172.21.15.18:0/3563892371 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x561864914490 0x5618649168b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2a3a2224f4477c4f server_cookie=26fe13b74989034 in_seq=0 out_seq=0 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7cbbfff640 1 -- 172.21.15.18:0/3563892371 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561864595890 con 0x561864914490 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7cbbfff640 10 client.19551 handle_command_reply: tid=0 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7ce5dbf740 1 client.19551 shutdown 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3563892371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648ab750 msgr2=0x5618648b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/3563892371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648ab750 0x5618648b1ed0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00058c0 tx=0x7f7cd0011a80 comp rx=0 tx=0).stop 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3563892371 shutdown_connections 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/3563892371 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x561864914490 0x5618649168b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/3563892371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56186490b070 0x56186490b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7cbbfff640 0 client.19551 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/3563892371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5618648ab750 0x5618648b1ed0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7ce5dbf740 1 --2- 172.21.15.18:0/3563892371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618648a2b70 0x5618648b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3563892371 >> 172.21.15.18:0/3563892371 conn(0x56186489c980 msgr2=0x56186489b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3563892371 shutdown_connections 2024-02-17T13:01:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.370+0000 7f7ce5dbf740 1 -- 172.21.15.18:0/3563892371 wait complete. 2024-02-17T13:01:09.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.371+0000 7f7ce5dbf740 20 client.19551 trim_cache size 0 max 16384 2024-02-17T13:01:09.368 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:01:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.371+0000 7f7ce31e9640 1 -- 172.21.15.18:0/3040885607 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7ccc0a2260 msgr2=0x7f7ccc0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.371+0000 7f7ce31e9640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7ccc0a2260 0x7f7ccc0a4720 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7cd409adc0 tx=0x7f7cd4090790 comp rx=0 tx=0).stop 2024-02-17T13:01:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.371+0000 7f7ce31e9640 1 -- 172.21.15.18:0/3040885607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc10b7c0 msgr2=0x7f7cdc100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.371+0000 7f7ce31e9640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc10b7c0 0x7f7cdc100ac0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009fd0 tx=0x7f7cd400ef10 comp rx=0 tx=0).stop 2024-02-17T13:01:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.371+0000 7f7ce31e9640 1 -- 172.21.15.18:0/3040885607 shutdown_connections 2024-02-17T13:01:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.371+0000 7f7ce31e9640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7ccc0a2260 0x7f7ccc0a4720 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.371+0000 7f7ce31e9640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc10b7c0 0x7f7cdc100ac0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.371+0000 7f7ce31e9640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cdc102d10 0x7f7cdc100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.371+0000 7f7ce31e9640 1 --2- 172.21.15.18:0/3040885607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cdc1023f0 0x7f7cdc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.371+0000 7f7ce31e9640 1 -- 172.21.15.18:0/3040885607 >> 172.21.15.18:0/3040885607 conn(0x7f7cdc0771e0 msgr2=0x7f7cdc0fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.371+0000 7f7ce31e9640 1 -- 172.21.15.18:0/3040885607 shutdown_connections 2024-02-17T13:01:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.372+0000 7f7ce31e9640 1 -- 172.21.15.18:0/3040885607 wait complete. 2024-02-17T13:01:09.373 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:01:09.374 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T13:01:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.493+0000 7ffac4483640 1 -- 172.21.15.18:0/3117634078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffabc100260 msgr2=0x7ffabc100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.493+0000 7ffac4483640 1 --2- 172.21.15.18:0/3117634078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffabc100260 0x7ffabc100640 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009930 tx=0x7ffab802bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.493+0000 7ffac4483640 1 -- 172.21.15.18:0/3117634078 shutdown_connections 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.493+0000 7ffac4483640 1 --2- 172.21.15.18:0/3117634078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc105330 0x7ffabc10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.493+0000 7ffac4483640 1 --2- 172.21.15.18:0/3117634078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc100c10 0x7ffabc104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.493+0000 7ffac4483640 1 --2- 172.21.15.18:0/3117634078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffabc100260 0x7ffabc100640 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.493+0000 7ffac4483640 1 -- 172.21.15.18:0/3117634078 >> 172.21.15.18:0/3117634078 conn(0x7ffabc077180 msgr2=0x7ffabc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.493+0000 7ffac4483640 1 -- 172.21.15.18:0/3117634078 shutdown_connections 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac4483640 1 -- 172.21.15.18:0/3117634078 wait complete. 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac4483640 1 Processor -- start 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac4483640 1 -- start start 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac4483640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc100260 0x7ffabc19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac4483640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffabc105330 0x7ffabc19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac4483640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc1975f0 0x7ffabc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac4483640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc197fe0 con 0x7ffabc105330 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac4483640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc198150 con 0x7ffabc100260 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac4483640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc198310 con 0x7ffabc1975f0 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac1a1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffabc105330 0x7ffabc19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac1a1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffabc105330 0x7ffabc19d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40254/0 (socket says 172.21.15.18:40254) 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac221f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc100260 0x7ffabc19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac1a1e640 1 -- 172.21.15.18:0/3340581452 learned_addr learned my addr 172.21.15.18:0/3340581452 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.494+0000 7ffac2a20640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc1975f0 0x7ffabc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.495+0000 7ffac1a1e640 1 -- 172.21.15.18:0/3340581452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc1975f0 msgr2=0x7ffabc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.495+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc1975f0 0x7ffabc197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.495+0000 7ffac1a1e640 1 -- 172.21.15.18:0/3340581452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc100260 msgr2=0x7ffabc19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.495+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc100260 0x7ffabc19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.495+0000 7ffac1a1e640 1 -- 172.21.15.18:0/3340581452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab8009560 con 0x7ffabc105330 2024-02-17T13:01:09.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.495+0000 7ffac221f640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc100260 0x7ffabc19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.495+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffabc105330 0x7ffabc19d980 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00e9c0 tx=0x7ffaac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.495+0000 7ffab37fe640 1 -- 172.21.15.18:0/3340581452 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac009700 con 0x7ffabc105330 2024-02-17T13:01:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.495+0000 7ffac4483640 1 -- 172.21.15.18:0/3340581452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc1a4300 con 0x7ffabc105330 2024-02-17T13:01:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.495+0000 7ffab37fe640 1 -- 172.21.15.18:0/3340581452 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffaac00cd80 con 0x7ffabc105330 2024-02-17T13:01:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.495+0000 7ffab37fe640 1 -- 172.21.15.18:0/3340581452 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac010430 con 0x7ffabc105330 2024-02-17T13:01:09.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.495+0000 7ffac4483640 1 -- 172.21.15.18:0/3340581452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc1a4860 con 0x7ffabc105330 2024-02-17T13:01:09.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac704b740 1 Processor -- start 2024-02-17T13:01:09.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac704b740 1 -- start start 2024-02-17T13:01:09.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffab37fe640 1 -- 172.21.15.18:0/3340581452 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ffaac005000 con 0x7ffabc105330 2024-02-17T13:01:09.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac704b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181a250 0x56203181a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac704b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181ab70 0x562031823210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac704b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x562031825b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac704b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620317d5960 con 0x562031823750 2024-02-17T13:01:09.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac704b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562031789310 con 0x56203181a250 2024-02-17T13:01:09.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac704b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562031786840 con 0x56203181ab70 2024-02-17T13:01:09.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac1a1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x562031825b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac2a20640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181a250 0x56203181a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac1a1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x562031825b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40262/0 (socket says 172.21.15.18:40262) 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac221f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181ab70 0x562031823210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac1a1e640 1 -- 172.21.15.18:0/2193144291 learned_addr learned my addr 172.21.15.18:0/2193144291 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.498+0000 7ffac2a20640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181a250 0x56203181a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:54540/0 (socket says 172.21.15.18:54540) 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.499+0000 7ffac2a20640 1 -- 172.21.15.18:0/2193144291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181ab70 msgr2=0x562031823210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.499+0000 7ffac2a20640 1 --2- 172.21.15.18:0/2193144291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181ab70 0x562031823210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.499+0000 7ffac2a20640 1 -- 172.21.15.18:0/2193144291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 msgr2=0x562031825b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.499+0000 7ffac2a20640 1 --2- 172.21.15.18:0/2193144291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x562031825b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.499+0000 7ffac2a20640 1 -- 172.21.15.18:0/2193144291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562031744060 con 0x56203181a250 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.499+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/2193144291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x562031825b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.499+0000 7ffab37fe640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffaa80a2230 0x7ffaa80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.499+0000 7ffac2a20640 1 --2- 172.21.15.18:0/2193144291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181a250 0x56203181a630 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009f00 tx=0x7ffab4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34977b3937d8c45e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.499+0000 7ffac221f640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffaa80a2230 0x7ffaa80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.499+0000 7ffab37fe640 1 -- 172.21.15.18:0/3340581452 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7ffaac014070 con 0x7ffabc105330 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.499+0000 7ffab0bfe640 1 -- 172.21.15.18:0/2193144291 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4004d20 con 0x56203181a250 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.499+0000 7ffab0bfe640 1 -- 172.21.15.18:0/2193144291 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffab4004e80 con 0x56203181a250 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.500+0000 7ffab0bfe640 1 -- 172.21.15.18:0/2193144291 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4005260 con 0x56203181a250 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.500+0000 7ffac221f640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffaa80a2230 0x7ffaa80a46f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffab8005ab0 tx=0x7ffab8005d90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.500+0000 7ffac704b740 1 -- 172.21.15.18:0/2193144291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181a250 msgr2=0x56203181a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.500+0000 7ffac704b740 1 --2- 172.21.15.18:0/2193144291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181a250 0x56203181a630 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009f00 tx=0x7ffab4011810 comp rx=0 tx=0).stop 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.500+0000 7ffac704b740 1 -- 172.21.15.18:0/2193144291 shutdown_connections 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.500+0000 7ffac704b740 1 --2- 172.21.15.18:0/2193144291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x562031825b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.500+0000 7ffac704b740 1 --2- 172.21.15.18:0/2193144291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181ab70 0x562031823210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.500+0000 7ffac704b740 1 --2- 172.21.15.18:0/2193144291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181a250 0x56203181a630 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.500+0000 7ffac704b740 1 -- 172.21.15.18:0/2193144291 >> 172.21.15.18:0/2193144291 conn(0x562031814980 msgr2=0x562031814d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.500+0000 7ffac704b740 1 -- 172.21.15.18:0/2193144291 shutdown_connections 2024-02-17T13:01:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.500+0000 7ffac704b740 1 -- 172.21.15.18:0/2193144291 wait complete. 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac704b740 1 -- start start 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac704b740 1 Processor -- start 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac704b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56203181a250 0x5620318206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac704b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181ab70 0x562031820c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac704b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562031823750 0x562031821170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac704b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562031782020 con 0x56203181a250 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac704b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562031455430 con 0x562031823750 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac704b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562031411e00 con 0x56203181ab70 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac704b740 10 client.0 _ll_register_callbacks cb 0x562031729cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac221f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56203181a250 0x5620318206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac221f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56203181a250 0x5620318206f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40278/0 (socket says 172.21.15.18:40278) 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac221f640 1 -- 172.21.15.18:0/188164201 learned_addr learned my addr 172.21.15.18:0/188164201 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181ab70 0x562031820c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac2a20640 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562031823750 0x562031821170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac221f640 1 -- 172.21.15.18:0/188164201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181ab70 msgr2=0x562031820c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac221f640 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181ab70 0x562031820c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac221f640 1 -- 172.21.15.18:0/188164201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562031823750 msgr2=0x562031821170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac221f640 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562031823750 0x562031821170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac221f640 1 -- 172.21.15.18:0/188164201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab4009be0 con 0x56203181a250 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.501+0000 7ffac2a20640 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562031823750 0x562031821170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffac221f640 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56203181a250 0x5620318206f0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009900 tx=0x7ffab802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d6e9c0e1bf11ce server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffaa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffaa5ffb640 1 -- 172.21.15.18:0/188164201 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8004510 con 0x56203181a250 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffaa5ffb640 1 -- 172.21.15.18:0/188164201 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7ffab8005450 con 0x56203181a250 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffac704b740 1 -- 172.21.15.18:0/188164201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620314be720 con 0x56203181a250 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffaa5ffb640 1 -- 172.21.15.18:0/188164201 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffab8005610 con 0x56203181a250 2024-02-17T13:01:09.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffac704b740 10 client.19563 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffac704b740 1 -- 172.21.15.18:0/188164201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5620317806a0 con 0x56203181a250 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffaa5ffb640 1 -- 172.21.15.18:0/188164201 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5620314be720 con 0x56203181a250 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffaa5ffb640 1 -- 172.21.15.18:0/188164201 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffab803c040 con 0x56203181a250 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffac704b740 10 client.19563 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffac704b740 10 client.19563 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffac704b740 20 client.19563 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffac704b740 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x562031833b00 0x562031835ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffac704b740 4 client.19563 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.502+0000 7ffac704b740 1 -- 172.21.15.18:0/188164201 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56203150d890 con 0x562031833b00 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.503+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x562031833b00 0x562031835ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.503+0000 7ffaa5ffb640 10 client.19563 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.503+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x562031833b00 0x562031835ef0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eda83faa4b3a6cb3 server_cookie=f59599888ad30900 in_seq=0 out_seq=0 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffaa5ffb640 1 -- 172.21.15.18:0/188164201 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56203150d890 con 0x562031833b00 2024-02-17T13:01:09.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffaa5ffb640 10 client.19563 handle_command_reply: tid=0 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffac704b740 1 client.19563 shutdown 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffac704b740 1 -- 172.21.15.18:0/188164201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56203181a250 msgr2=0x5620318206f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffac704b740 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56203181a250 0x5620318206f0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009900 tx=0x7ffab802bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffac704b740 1 -- 172.21.15.18:0/188164201 shutdown_connections 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffac704b740 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x562031833b00 0x562031835ef0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffac704b740 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562031823750 0x562031821170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffaa5ffb640 0 client.19563 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffac704b740 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181ab70 0x562031820c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffac704b740 1 --2- 172.21.15.18:0/188164201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56203181a250 0x5620318206f0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffac704b740 1 -- 172.21.15.18:0/188164201 >> 172.21.15.18:0/188164201 conn(0x562031814980 msgr2=0x5620318166d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffac704b740 1 -- 172.21.15.18:0/188164201 shutdown_connections 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffac704b740 1 -- 172.21.15.18:0/188164201 wait complete. 2024-02-17T13:01:09.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.504+0000 7ffac704b740 20 client.19563 trim_cache size 0 max 16384 2024-02-17T13:01:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac704b740 1 Processor -- start 2024-02-17T13:01:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac704b740 1 -- start start 2024-02-17T13:01:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac704b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181a250 0x562031822e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac704b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 0x562031820f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac704b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x56203181d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac704b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562031707130 con 0x562031823750 2024-02-17T13:01:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac704b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562031833900 con 0x56203181ab70 2024-02-17T13:01:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac704b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562031826420 con 0x56203181a250 2024-02-17T13:01:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac2a20640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x56203181d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac1a1e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 0x562031820f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac1a1e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 0x562031820f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:54554/0 (socket says 172.21.15.18:54554) 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac2a20640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x56203181d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40280/0 (socket says 172.21.15.18:40280) 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac1a1e640 1 -- 172.21.15.18:0/188756966 learned_addr learned my addr 172.21.15.18:0/188756966 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac221f640 1 --2- 172.21.15.18:0/188756966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181a250 0x562031822e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac1a1e640 1 -- 172.21.15.18:0/188756966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181a250 msgr2=0x562031822e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/188756966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181a250 0x562031822e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac1a1e640 1 -- 172.21.15.18:0/188756966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 msgr2=0x56203181d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/188756966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x56203181d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac1a1e640 1 -- 172.21.15.18:0/188756966 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56203171cc20 con 0x56203181ab70 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac221f640 1 --2- 172.21.15.18:0/188756966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181a250 0x562031822e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.517+0000 7ffac2a20640 1 --2- 172.21.15.18:0/188756966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x56203181d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.518+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/188756966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 0x562031820f90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffaac097850 tx=0x7ffaac0d1d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=177cdab42e01ef6c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.518+0000 7ffaa7fff640 1 -- 172.21.15.18:0/188756966 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac09ac50 con 0x56203181ab70 2024-02-17T13:01:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.518+0000 7ffaa7fff640 1 -- 172.21.15.18:0/188756966 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffaac09a9c0 con 0x56203181ab70 2024-02-17T13:01:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.518+0000 7ffaa7fff640 1 -- 172.21.15.18:0/188756966 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac099180 con 0x56203181ab70 2024-02-17T13:01:09.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 -- 172.21.15.18:0/188756966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 msgr2=0x562031820f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 --2- 172.21.15.18:0/188756966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 0x562031820f90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffaac097850 tx=0x7ffaac0d1d80 comp rx=0 tx=0).stop 2024-02-17T13:01:09.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 -- 172.21.15.18:0/188756966 shutdown_connections 2024-02-17T13:01:09.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 --2- 172.21.15.18:0/188756966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x56203181d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 --2- 172.21.15.18:0/188756966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 0x562031820f90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 --2- 172.21.15.18:0/188756966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181a250 0x562031822e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 -- 172.21.15.18:0/188756966 >> 172.21.15.18:0/188756966 conn(0x562031814980 msgr2=0x56203183a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 -- 172.21.15.18:0/188756966 shutdown_connections 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 -- 172.21.15.18:0/188756966 wait complete. 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 -- start start 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 Processor -- start 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181a250 0x562031829100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 0x562031829640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x562031829b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562031889fe0 con 0x562031823750 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56203188a120 con 0x56203181ab70 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56203188a260 con 0x56203181a250 2024-02-17T13:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac704b740 10 client.0 _ll_register_callbacks cb 0x7ffab4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac1a1e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 0x562031829640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac1a1e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 0x562031829640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:54558/0 (socket says 172.21.15.18:54558) 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac1a1e640 1 -- 172.21.15.18:0/783093184 learned_addr learned my addr 172.21.15.18:0/783093184 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.519+0000 7ffac2a20640 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x562031829b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffac1a1e640 1 -- 172.21.15.18:0/783093184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181a250 msgr2=0x562031829100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffac221f640 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181a250 0x562031829100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181a250 0x562031829100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffac1a1e640 1 -- 172.21.15.18:0/783093184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 msgr2=0x562031829b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x562031829b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffac1a1e640 1 -- 172.21.15.18:0/783093184 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620317806a0 con 0x56203181ab70 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffac2a20640 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x562031829b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffac221f640 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181a250 0x562031829100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffac1a1e640 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 0x562031829640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffaac09ae10 tx=0x7ffaac0d1ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd2b913839eb5730 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffaa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffaa5ffb640 1 -- 172.21.15.18:0/783093184 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac09ac50 con 0x56203181ab70 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffaa5ffb640 1 -- 172.21.15.18:0/783093184 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7ffaac09ae80 con 0x56203181ab70 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.520+0000 7ffac704b740 1 -- 172.21.15.18:0/783093184 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620314817e0 con 0x56203181ab70 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.521+0000 7ffaa5ffb640 1 -- 172.21.15.18:0/783093184 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaac09b580 con 0x56203181ab70 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.521+0000 7ffac704b740 10 client.28556 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.521+0000 7ffac704b740 1 -- 172.21.15.18:0/783093184 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x562031744060 con 0x56203181ab70 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.521+0000 7ffaa5ffb640 1 -- 172.21.15.18:0/783093184 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5620314817e0 con 0x56203181ab70 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.521+0000 7ffaa5ffb640 1 -- 172.21.15.18:0/783093184 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffaac01f060 con 0x56203181ab70 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.521+0000 7ffac704b740 10 client.28556 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.521+0000 7ffac704b740 10 client.28556 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.521+0000 7ffac704b740 20 client.28556 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.521+0000 7ffac704b740 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56203188b800 0x56203188dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.521+0000 7ffac704b740 4 client.28556 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:01:09.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.521+0000 7ffac704b740 1 -- 172.21.15.18:0/783093184 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56203142d700 con 0x56203188b800 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.521+0000 7ffac221f640 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56203188b800 0x56203188dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac221f640 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56203188b800 0x56203188dc20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68793782ba7386a1 server_cookie=50ba00e8843c961c in_seq=0 out_seq=0 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffaa5ffb640 10 client.28556 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffaa5ffb640 1 -- 172.21.15.18:0/783093184 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56203142d700 con 0x56203188b800 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffaa5ffb640 10 client.28556 handle_command_reply: tid=0 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac704b740 1 client.28556 shutdown 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac704b740 1 -- 172.21.15.18:0/783093184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 msgr2=0x562031829640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac704b740 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 0x562031829640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffaac09ae10 tx=0x7ffaac0d1ae0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac704b740 1 -- 172.21.15.18:0/783093184 shutdown_connections 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac704b740 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x56203188b800 0x56203188dc20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac704b740 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562031823750 0x562031829b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac704b740 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56203181ab70 0x562031829640 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac704b740 1 --2- 172.21.15.18:0/783093184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56203181a250 0x562031829100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffaa5ffb640 0 client.28556 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:01:09.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac704b740 1 -- 172.21.15.18:0/783093184 >> 172.21.15.18:0/783093184 conn(0x562031814980 msgr2=0x5620318132f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac704b740 1 -- 172.21.15.18:0/783093184 shutdown_connections 2024-02-17T13:01:09.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac704b740 1 -- 172.21.15.18:0/783093184 wait complete. 2024-02-17T13:01:09.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac704b740 20 client.28556 trim_cache size 0 max 16384 2024-02-17T13:01:09.520 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:01:09.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac4483640 1 -- 172.21.15.18:0/3340581452 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffaa80a2230 msgr2=0x7ffaa80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac4483640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffaa80a2230 0x7ffaa80a46f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffab8005ab0 tx=0x7ffab8005d90 comp rx=0 tx=0).stop 2024-02-17T13:01:09.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.522+0000 7ffac4483640 1 -- 172.21.15.18:0/3340581452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffabc105330 msgr2=0x7ffabc19d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.523+0000 7ffac4483640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffabc105330 0x7ffabc19d980 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00e9c0 tx=0x7ffaac00ee80 comp rx=0 tx=0).stop 2024-02-17T13:01:09.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.523+0000 7ffac4483640 1 -- 172.21.15.18:0/3340581452 shutdown_connections 2024-02-17T13:01:09.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.523+0000 7ffac4483640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffaa80a2230 0x7ffaa80a46f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.523+0000 7ffac4483640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc1975f0 0x7ffabc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.523+0000 7ffac4483640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffabc105330 0x7ffabc19d980 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.523+0000 7ffac4483640 1 --2- 172.21.15.18:0/3340581452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc100260 0x7ffabc19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.523+0000 7ffac4483640 1 -- 172.21.15.18:0/3340581452 >> 172.21.15.18:0/3340581452 conn(0x7ffabc077180 msgr2=0x7ffabc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.523+0000 7ffac4483640 1 -- 172.21.15.18:0/3340581452 shutdown_connections 2024-02-17T13:01:09.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.523+0000 7ffac4483640 1 -- 172.21.15.18:0/3340581452 wait complete. 2024-02-17T13:01:09.525 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:01:09.525 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T13:01:09.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.648+0000 7f976455c640 1 -- 172.21.15.18:0/550661829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f975c069960 msgr2=0x7f975c069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.648+0000 7f976455c640 1 --2- 172.21.15.18:0/550661829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f975c069960 0x7f975c069d40 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f9758009930 tx=0x7f975802bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:09.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.648+0000 7f976455c640 1 -- 172.21.15.18:0/550661829 shutdown_connections 2024-02-17T13:01:09.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.648+0000 7f976455c640 1 --2- 172.21.15.18:0/550661829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c106450 0x7f975c10e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.648+0000 7f976455c640 1 --2- 172.21.15.18:0/550661829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f975c06a310 0x7f975c105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.648+0000 7f976455c640 1 --2- 172.21.15.18:0/550661829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f975c069960 0x7f975c069d40 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.648+0000 7f976455c640 1 -- 172.21.15.18:0/550661829 >> 172.21.15.18:0/550661829 conn(0x7f975c076f70 msgr2=0x7f975c077380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.648+0000 7f976455c640 1 -- 172.21.15.18:0/550661829 shutdown_connections 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.648+0000 7f976455c640 1 -- 172.21.15.18:0/550661829 wait complete. 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f976455c640 1 Processor -- start 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f976455c640 1 -- start start 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f976455c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c069960 0x7f975c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f976455c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f975c106450 0x7f975c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f976455c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f975c1975e0 0x7f975c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f976455c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c197fd0 con 0x7f975c1975e0 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f976455c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c198140 con 0x7f975c106450 2024-02-17T13:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f976455c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c198300 con 0x7f975c069960 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f9762af9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f975c1975e0 0x7f975c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f97622f8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c069960 0x7f975c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f9761af7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f975c106450 0x7f975c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f9762af9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f975c1975e0 0x7f975c197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40306/0 (socket says 172.21.15.18:40306) 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.649+0000 7f9762af9640 1 -- 172.21.15.18:0/2723305929 learned_addr learned my addr 172.21.15.18:0/2723305929 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f9761af7640 1 -- 172.21.15.18:0/2723305929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c069960 msgr2=0x7f975c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f9761af7640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c069960 0x7f975c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f9761af7640 1 -- 172.21.15.18:0/2723305929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f975c1975e0 msgr2=0x7f975c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f9761af7640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f975c1975e0 0x7f975c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f9761af7640 1 -- 172.21.15.18:0/2723305929 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9758009560 con 0x7f975c106450 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f97622f8640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c069960 0x7f975c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f9762af9640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f975c1975e0 0x7f975c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f9761af7640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f975c106450 0x7f975c19d970 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f974c00e9c0 tx=0x7f974c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f97537fe640 1 -- 172.21.15.18:0/2723305929 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c00ccc0 con 0x7f975c106450 2024-02-17T13:01:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f976455c640 1 -- 172.21.15.18:0/2723305929 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975c1a42f0 con 0x7f975c106450 2024-02-17T13:01:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f97537fe640 1 -- 172.21.15.18:0/2723305929 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f974c00ce20 con 0x7f975c106450 2024-02-17T13:01:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f976455c640 1 -- 172.21.15.18:0/2723305929 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975c1a4850 con 0x7f975c106450 2024-02-17T13:01:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.650+0000 7f97537fe640 1 -- 172.21.15.18:0/2723305929 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c018880 con 0x7f975c106450 2024-02-17T13:01:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.653+0000 7f97537fe640 1 -- 172.21.15.18:0/2723305929 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f974c0189e0 con 0x7f975c106450 2024-02-17T13:01:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.653+0000 7f97537fe640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f97480a2230 0x7f97480a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.653+0000 7f97622f8640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f97480a2230 0x7f97480a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.653+0000 7f97537fe640 1 -- 172.21.15.18:0/2723305929 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f974c014070 con 0x7f975c106450 2024-02-17T13:01:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.654+0000 7f97622f8640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f97480a2230 0x7f97480a46f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9758005ab0 tx=0x7f9758005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.654+0000 7f9767205740 1 Processor -- start 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.654+0000 7f9767205740 1 -- start start 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.654+0000 7f9767205740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4250 0x5654617b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9767205740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617b4b70 0x5654617bd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9767205740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617bd750 0x5654617bfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9767205740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56546176f960 con 0x5654617bd750 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9767205740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565461723310 con 0x5654617b4b70 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9767205740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565461720840 con 0x5654617b4250 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9762af9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617bd750 0x5654617bfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f97622f8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4250 0x5654617b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f97622f8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4250 0x5654617b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:38204/0 (socket says 172.21.15.18:38204) 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9762af9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617bd750 0x5654617bfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40312/0 (socket says 172.21.15.18:40312) 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f97622f8640 1 -- 172.21.15.18:0/4256399685 learned_addr learned my addr 172.21.15.18:0/4256399685 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9761af7640 1 --2- 172.21.15.18:0/4256399685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617b4b70 0x5654617bd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9762af9640 1 -- 172.21.15.18:0/4256399685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4250 msgr2=0x5654617b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9762af9640 1 --2- 172.21.15.18:0/4256399685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4250 0x5654617b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9762af9640 1 -- 172.21.15.18:0/4256399685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617b4b70 msgr2=0x5654617bd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9762af9640 1 --2- 172.21.15.18:0/4256399685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617b4b70 0x5654617bd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9762af9640 1 -- 172.21.15.18:0/4256399685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654616de060 con 0x5654617bd750 2024-02-17T13:01:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f97622f8640 1 --2- 172.21.15.18:0/4256399685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4250 0x5654617b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9761af7640 1 --2- 172.21.15.18:0/4256399685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617b4b70 0x5654617bd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9762af9640 1 --2- 172.21.15.18:0/4256399685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617bd750 0x5654617bfb40 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f9754011780 tx=0x7f9754011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eff06de563e360a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.655+0000 7f9750bfe640 1 -- 172.21.15.18:0/4256399685 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754004d20 con 0x5654617bd750 2024-02-17T13:01:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.656+0000 7f9750bfe640 1 -- 172.21.15.18:0/4256399685 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9754004e80 con 0x5654617bd750 2024-02-17T13:01:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.656+0000 7f9750bfe640 1 -- 172.21.15.18:0/4256399685 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97540057f0 con 0x5654617bd750 2024-02-17T13:01:09.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.656+0000 7f9767205740 1 -- 172.21.15.18:0/4256399685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617bd750 msgr2=0x5654617bfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.656+0000 7f9767205740 1 --2- 172.21.15.18:0/4256399685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617bd750 0x5654617bfb40 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f9754011780 tx=0x7f9754011c40 comp rx=0 tx=0).stop 2024-02-17T13:01:09.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.656+0000 7f9767205740 1 -- 172.21.15.18:0/4256399685 shutdown_connections 2024-02-17T13:01:09.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.656+0000 7f9767205740 1 --2- 172.21.15.18:0/4256399685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617bd750 0x5654617bfb40 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.656+0000 7f9767205740 1 --2- 172.21.15.18:0/4256399685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617b4b70 0x5654617bd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.656+0000 7f9767205740 1 --2- 172.21.15.18:0/4256399685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4250 0x5654617b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.656+0000 7f9767205740 1 -- 172.21.15.18:0/4256399685 >> 172.21.15.18:0/4256399685 conn(0x5654617ae980 msgr2=0x5654617aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9767205740 1 -- 172.21.15.18:0/4256399685 shutdown_connections 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9767205740 1 -- 172.21.15.18:0/4256399685 wait complete. 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9767205740 1 -- start start 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9767205740 1 Processor -- start 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9767205740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9767205740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617b4b70 0x5654617bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9767205740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617bd750 0x5654617bb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9767205740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56546171c020 con 0x5654617b4250 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9767205740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654613ef430 con 0x5654617b4b70 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9767205740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654613abe00 con 0x5654617bd750 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9767205740 10 client.0 _ll_register_callbacks cb 0x5654616c3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f97622f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f97622f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617ba6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40322/0 (socket says 172.21.15.18:40322) 2024-02-17T13:01:09.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f97622f8640 1 -- 172.21.15.18:0/613273627 learned_addr learned my addr 172.21.15.18:0/613273627 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9761af7640 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617b4b70 0x5654617bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.657+0000 7f9762af9640 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617bd750 0x5654617bb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f97622f8640 1 -- 172.21.15.18:0/613273627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617bd750 msgr2=0x5654617bb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f97622f8640 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617bd750 0x5654617bb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f97622f8640 1 -- 172.21.15.18:0/613273627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617b4b70 msgr2=0x5654617bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f97622f8640 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617b4b70 0x5654617bac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f97622f8640 1 -- 172.21.15.18:0/613273627 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97540113e0 con 0x5654617b4250 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f97622f8640 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617ba6d0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f9758000c00 tx=0x7f9758009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e795114c3acbb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f9745ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f9745ffb640 1 -- 172.21.15.18:0/613273627 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97580050c0 con 0x5654617b4250 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f9745ffb640 1 -- 172.21.15.18:0/613273627 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f9758047040 con 0x5654617b4250 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f9767205740 1 -- 172.21.15.18:0/613273627 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565461458720 con 0x5654617b4250 2024-02-17T13:01:09.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f9745ffb640 1 -- 172.21.15.18:0/613273627 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9758005220 con 0x5654617b4250 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f9767205740 10 client.19581 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f9767205740 1 -- 172.21.15.18:0/613273627 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56546171a6a0 con 0x5654617b4250 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f9745ffb640 1 -- 172.21.15.18:0/613273627 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x565461458720 con 0x5654617b4250 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.658+0000 7f9745ffb640 1 -- 172.21.15.18:0/613273627 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f975803f440 con 0x5654617b4250 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.659+0000 7f9767205740 10 client.19581 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.659+0000 7f9767205740 10 client.19581 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.659+0000 7f9767205740 20 client.19581 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.659+0000 7f9767205740 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5654617cda40 0x5654617cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.659+0000 7f9767205740 4 client.19581 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.659+0000 7f9767205740 1 -- 172.21.15.18:0/613273627 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654614a7890 con 0x5654617cda40 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.659+0000 7f9761af7640 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5654617cda40 0x5654617cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.659+0000 7f9745ffb640 10 client.19581 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:01:09.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.659+0000 7f9761af7640 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5654617cda40 0x5654617cfe60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2bc70de183a517e3 server_cookie=ce30019a2ab5a880 in_seq=0 out_seq=0 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9745ffb640 1 -- 172.21.15.18:0/613273627 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5654614a7890 con 0x5654617cda40 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9745ffb640 10 client.19581 handle_command_reply: tid=0 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9767205740 1 client.19581 shutdown 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9767205740 1 -- 172.21.15.18:0/613273627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 msgr2=0x5654617ba6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9767205740 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617ba6d0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f9758000c00 tx=0x7f9758009260 comp rx=0 tx=0).stop 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9767205740 1 -- 172.21.15.18:0/613273627 shutdown_connections 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9767205740 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5654617cda40 0x5654617cfe60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9767205740 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617bd750 0x5654617bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9767205740 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617b4b70 0x5654617bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9745ffb640 0 client.19581 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9767205740 1 --2- 172.21.15.18:0/613273627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617ba6d0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9767205740 1 -- 172.21.15.18:0/613273627 >> 172.21.15.18:0/613273627 conn(0x5654617ae980 msgr2=0x5654617b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9767205740 1 -- 172.21.15.18:0/613273627 shutdown_connections 2024-02-17T13:01:09.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9767205740 1 -- 172.21.15.18:0/613273627 wait complete. 2024-02-17T13:01:09.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.660+0000 7f9767205740 20 client.19581 trim_cache size 0 max 16384 2024-02-17T13:01:09.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.685+0000 7f9767205740 1 Processor -- start 2024-02-17T13:01:09.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.685+0000 7f9767205740 1 -- start start 2024-02-17T13:01:09.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.685+0000 7f9767205740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617baf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f9767205740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4b70 0x5654617ada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f9767205740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617bd750 0x5654617c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f9767205740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654616a1130 con 0x5654617b4250 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f9767205740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654617b35c0 con 0x5654617bd750 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f9767205740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654617cd840 con 0x5654617b4b70 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f97622f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617baf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f97622f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617baf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40328/0 (socket says 172.21.15.18:40328) 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f97622f8640 1 -- 172.21.15.18:0/2601446846 learned_addr learned my addr 172.21.15.18:0/2601446846 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f9762af9640 1 --2- 172.21.15.18:0/2601446846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617bd750 0x5654617c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f9761af7640 1 --2- 172.21.15.18:0/2601446846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4b70 0x5654617ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f97622f8640 1 -- 172.21.15.18:0/2601446846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4b70 msgr2=0x5654617ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f97622f8640 1 --2- 172.21.15.18:0/2601446846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4b70 0x5654617ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f97622f8640 1 -- 172.21.15.18:0/2601446846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617bd750 msgr2=0x5654617c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f97622f8640 1 --2- 172.21.15.18:0/2601446846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617bd750 0x5654617c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f97622f8640 1 -- 172.21.15.18:0/2601446846 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654616b6c20 con 0x5654617b4250 2024-02-17T13:01:09.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f97622f8640 1 --2- 172.21.15.18:0/2601446846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617baf70 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9758005c00 tx=0x7f97580092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8211403decde5b3c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f9747fff640 1 -- 172.21.15.18:0/2601446846 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975803f440 con 0x5654617b4250 2024-02-17T13:01:09.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f9747fff640 1 -- 172.21.15.18:0/2601446846 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f975803f5a0 con 0x5654617b4250 2024-02-17T13:01:09.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.686+0000 7f9747fff640 1 -- 172.21.15.18:0/2601446846 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758003790 con 0x5654617b4250 2024-02-17T13:01:09.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.687+0000 7f9767205740 1 -- 172.21.15.18:0/2601446846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 msgr2=0x5654617baf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.687+0000 7f9767205740 1 --2- 172.21.15.18:0/2601446846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617baf70 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9758005c00 tx=0x7f97580092d0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.687+0000 7f9767205740 1 -- 172.21.15.18:0/2601446846 shutdown_connections 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.687+0000 7f9767205740 1 --2- 172.21.15.18:0/2601446846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617bd750 0x5654617c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.687+0000 7f9767205740 1 --2- 172.21.15.18:0/2601446846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4b70 0x5654617ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.687+0000 7f9767205740 1 --2- 172.21.15.18:0/2601446846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617baf70 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.687+0000 7f9767205740 1 -- 172.21.15.18:0/2601446846 >> 172.21.15.18:0/2601446846 conn(0x5654617ae980 msgr2=0x5654617ad2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.687+0000 7f9767205740 1 -- 172.21.15.18:0/2601446846 shutdown_connections 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.687+0000 7f9767205740 1 -- 172.21.15.18:0/2601446846 wait complete. 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f9767205740 1 -- start start 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f9767205740 1 Processor -- start 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f9767205740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f9767205740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4b70 0x5654617c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f9767205740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617bd750 0x5654617c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f9767205740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654618240b0 con 0x5654617b4250 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f9767205740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565461824220 con 0x5654617bd750 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f9767205740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654618243e0 con 0x5654617b4b70 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f9767205740 10 client.0 _ll_register_callbacks cb 0x7f9754009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f97622f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f9761af7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4b70 0x5654617c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f97622f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617c32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40340/0 (socket says 172.21.15.18:40340) 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f97622f8640 1 -- 172.21.15.18:0/4200146730 learned_addr learned my addr 172.21.15.18:0/4200146730 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f9762af9640 1 --2- 172.21.15.18:0/4200146730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617bd750 0x5654617c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f97622f8640 1 -- 172.21.15.18:0/4200146730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4b70 msgr2=0x5654617c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f97622f8640 1 --2- 172.21.15.18:0/4200146730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4b70 0x5654617c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f97622f8640 1 -- 172.21.15.18:0/4200146730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617bd750 msgr2=0x5654617c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f97622f8640 1 --2- 172.21.15.18:0/4200146730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617bd750 0x5654617c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.688+0000 7f97622f8640 1 -- 172.21.15.18:0/4200146730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56546171a6a0 con 0x5654617b4250 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f97622f8640 1 --2- 172.21.15.18:0/4200146730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617c32b0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9758005dd0 tx=0x7f97580052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d15da9ff75713b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9745ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9745ffb640 1 -- 172.21.15.18:0/4200146730 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975803f440 con 0x5654617b4250 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9745ffb640 1 -- 172.21.15.18:0/4200146730 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f97580056f0 con 0x5654617b4250 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9767205740 1 -- 172.21.15.18:0/4200146730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56546141b7e0 con 0x5654617b4250 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9745ffb640 1 -- 172.21.15.18:0/4200146730 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97580036d0 con 0x5654617b4250 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9767205740 10 client.19587 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9767205740 1 -- 172.21.15.18:0/4200146730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5654616dc8c0 con 0x5654617b4250 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9745ffb640 1 -- 172.21.15.18:0/4200146730 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9758074020 con 0x5654617b4250 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9745ffb640 1 -- 172.21.15.18:0/4200146730 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9758067a40 con 0x5654617b4250 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9767205740 10 client.19587 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9767205740 10 client.19587 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:01:09.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9767205740 20 client.19587 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9767205740 1 --2- 172.21.15.18:0/4200146730 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5654618256d0 0x565461827ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9767205740 4 client.19587 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.689+0000 7f9767205740 1 -- 172.21.15.18:0/4200146730 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5654613c7700 con 0x5654618256d0 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.690+0000 7f9761af7640 1 --2- 172.21.15.18:0/4200146730 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5654618256d0 0x565461827ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.690+0000 7f9745ffb640 10 client.19587 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.690+0000 7f9761af7640 1 --2- 172.21.15.18:0/4200146730 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5654618256d0 0x565461827ac0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6361a1272fb13875 server_cookie=ac3d140a8aebe866 in_seq=0 out_seq=0 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.690+0000 7f9745ffb640 1 -- 172.21.15.18:0/4200146730 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5654613c7700 con 0x5654618256d0 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.690+0000 7f9745ffb640 10 client.19587 handle_command_reply: tid=0 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.690+0000 7f9767205740 1 client.19587 shutdown 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.690+0000 7f9767205740 1 -- 172.21.15.18:0/4200146730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 msgr2=0x5654617c32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.690+0000 7f9767205740 1 --2- 172.21.15.18:0/4200146730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617c32b0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9758005dd0 tx=0x7f97580052f0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f9767205740 1 -- 172.21.15.18:0/4200146730 shutdown_connections 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f9767205740 1 --2- 172.21.15.18:0/4200146730 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5654618256d0 0x565461827ac0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f9767205740 1 --2- 172.21.15.18:0/4200146730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654617bd750 0x5654617c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f9767205740 1 --2- 172.21.15.18:0/4200146730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654617b4b70 0x5654617c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f9745ffb640 0 client.19587 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:01:09.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f9767205740 1 --2- 172.21.15.18:0/4200146730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654617b4250 0x5654617c32b0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f9767205740 1 -- 172.21.15.18:0/4200146730 >> 172.21.15.18:0/4200146730 conn(0x5654617ae980 msgr2=0x5654617adb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f9767205740 1 -- 172.21.15.18:0/4200146730 shutdown_connections 2024-02-17T13:01:09.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f9767205740 1 -- 172.21.15.18:0/4200146730 wait complete. 2024-02-17T13:01:09.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f9767205740 20 client.19587 trim_cache size 0 max 16384 2024-02-17T13:01:09.688 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:01:09.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f976455c640 1 -- 172.21.15.18:0/2723305929 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f97480a2230 msgr2=0x7f97480a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f976455c640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f97480a2230 0x7f97480a46f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9758005ab0 tx=0x7f9758005fd0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f976455c640 1 -- 172.21.15.18:0/2723305929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f975c106450 msgr2=0x7f975c19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f976455c640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f975c106450 0x7f975c19d970 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f974c00e9c0 tx=0x7f974c00ee80 comp rx=0 tx=0).stop 2024-02-17T13:01:09.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f976455c640 1 -- 172.21.15.18:0/2723305929 shutdown_connections 2024-02-17T13:01:09.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f976455c640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f97480a2230 0x7f97480a46f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f976455c640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f975c1975e0 0x7f975c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f976455c640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f975c106450 0x7f975c19d970 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f976455c640 1 --2- 172.21.15.18:0/2723305929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c069960 0x7f975c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f976455c640 1 -- 172.21.15.18:0/2723305929 >> 172.21.15.18:0/2723305929 conn(0x7f975c076f70 msgr2=0x7f975c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.691+0000 7f976455c640 1 -- 172.21.15.18:0/2723305929 shutdown_connections 2024-02-17T13:01:09.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.692+0000 7f976455c640 1 -- 172.21.15.18:0/2723305929 wait complete. 2024-02-17T13:01:09.697 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:01:09.697 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T13:01:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.821+0000 7fdc1b457640 1 -- 172.21.15.18:0/3628722528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc14101370 msgr2=0x7fdc14101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.821+0000 7fdc1b457640 1 --2- 172.21.15.18:0/3628722528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc14101370 0x7fdc14101750 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009cd0 tx=0x7fdc0402c0f0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.821+0000 7fdc1b457640 1 -- 172.21.15.18:0/3628722528 shutdown_connections 2024-02-17T13:01:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.821+0000 7fdc1b457640 1 --2- 172.21.15.18:0/3628722528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc14106440 0x7fdc1410e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.821+0000 7fdc1b457640 1 --2- 172.21.15.18:0/3628722528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc14101d20 0x7fdc14105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.821+0000 7fdc1b457640 1 --2- 172.21.15.18:0/3628722528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc14101370 0x7fdc14101750 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.821+0000 7fdc1b457640 1 -- 172.21.15.18:0/3628722528 >> 172.21.15.18:0/3628722528 conn(0x7fdc140fcc10 msgr2=0x7fdc140ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.821+0000 7fdc1b457640 1 -- 172.21.15.18:0/3628722528 shutdown_connections 2024-02-17T13:01:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.821+0000 7fdc1b457640 1 -- 172.21.15.18:0/3628722528 wait complete. 2024-02-17T13:01:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc1b457640 1 Processor -- start 2024-02-17T13:01:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc1b457640 1 -- start start 2024-02-17T13:01:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc1b457640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc14101370 0x7fdc14073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc1b457640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc14106440 0x7fdc140739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc1b457640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc14073f00 0x7fdc141a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc1b457640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc141a5f90 con 0x7fdc14106440 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc1b457640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc141a6100 con 0x7fdc14101370 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc1b457640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc141a62c0 con 0x7fdc14073f00 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc189f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc14106440 0x7fdc140739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc191f3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc14101370 0x7fdc14073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc189f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc14106440 0x7fdc140739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40352/0 (socket says 172.21.15.18:40352) 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc199f4640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc14073f00 0x7fdc141a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc189f2640 1 -- 172.21.15.18:0/1795576216 learned_addr learned my addr 172.21.15.18:0/1795576216 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc191f3640 1 -- 172.21.15.18:0/1795576216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc14073f00 msgr2=0x7fdc141a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc191f3640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc14073f00 0x7fdc141a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc191f3640 1 -- 172.21.15.18:0/1795576216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc14106440 msgr2=0x7fdc140739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc191f3640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc14106440 0x7fdc140739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc191f3640 1 -- 172.21.15.18:0/1795576216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc08009490 con 0x7fdc14101370 2024-02-17T13:01:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.822+0000 7fdc189f2640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc14106440 0x7fdc140739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.823+0000 7fdc191f3640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc14101370 0x7fdc14073480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdc0402c700 tx=0x7fdc040325d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.823+0000 7fdc127fc640 1 -- 172.21.15.18:0/1795576216 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc040328b0 con 0x7fdc14101370 2024-02-17T13:01:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.823+0000 7fdc127fc640 1 -- 172.21.15.18:0/1795576216 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc04003e50 con 0x7fdc14101370 2024-02-17T13:01:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.823+0000 7fdc1b457640 1 -- 172.21.15.18:0/1795576216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc04009900 con 0x7fdc14101370 2024-02-17T13:01:09.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.823+0000 7fdc127fc640 1 -- 172.21.15.18:0/1795576216 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc04004860 con 0x7fdc14101370 2024-02-17T13:01:09.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.823+0000 7fdc1b457640 1 -- 172.21.15.18:0/1795576216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc141a6830 con 0x7fdc14101370 2024-02-17T13:01:09.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.826+0000 7fdc127fc640 1 -- 172.21.15.18:0/1795576216 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fdc0400a040 con 0x7fdc14101370 2024-02-17T13:01:09.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.826+0000 7fdc1e020740 1 Processor -- start 2024-02-17T13:01:09.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.826+0000 7fdc1e020740 1 -- start start 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.826+0000 7fdc1e020740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d07d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.826+0000 7fdc1e020740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d086210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.826+0000 7fdc1e020740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 0x56067d088b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.826+0000 7fdc1e020740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067d038960 con 0x56067d07db70 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.826+0000 7fdc1e020740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067cfec310 con 0x56067d086750 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.826+0000 7fdc1e020740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067cfe9840 con 0x56067d07d250 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc191f3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d086210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc191f3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d086210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40364/0 (socket says 172.21.15.18:40364) 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc191f3640 1 -- 172.21.15.18:0/3581990109 learned_addr learned my addr 172.21.15.18:0/3581990109 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc189f2640 1 --2- 172.21.15.18:0/3581990109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 0x56067d088b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc199f4640 1 --2- 172.21.15.18:0/3581990109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d07d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc127fc640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdbfc0a2230 0x7fdbfc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc191f3640 1 -- 172.21.15.18:0/3581990109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 msgr2=0x56067d07d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc191f3640 1 --2- 172.21.15.18:0/3581990109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d07d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc191f3640 1 -- 172.21.15.18:0/3581990109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 msgr2=0x56067d088b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc191f3640 1 --2- 172.21.15.18:0/3581990109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 0x56067d088b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc191f3640 1 -- 172.21.15.18:0/3581990109 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56067cfa7060 con 0x56067d07db70 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc199f4640 1 --2- 172.21.15.18:0/3581990109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d07d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc199f4640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdbfc0a2230 0x7fdbfc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc127fc640 1 -- 172.21.15.18:0/1795576216 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fdc040f0790 con 0x7fdc14101370 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdc191f3640 1 --2- 172.21.15.18:0/3581990109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d086210 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fdc040edf30 tx=0x7fdc04050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541047a8a1614d08 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.827+0000 7fdbfb7fe640 1 -- 172.21.15.18:0/3581990109 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc04032a10 con 0x56067d07db70 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.828+0000 7fdbfb7fe640 1 -- 172.21.15.18:0/3581990109 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc040b7e90 con 0x56067d07db70 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.828+0000 7fdbfb7fe640 1 -- 172.21.15.18:0/3581990109 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc04004040 con 0x56067d07db70 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.828+0000 7fdc1e020740 1 -- 172.21.15.18:0/3581990109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 msgr2=0x56067d086210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.828+0000 7fdc199f4640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdbfc0a2230 0x7fdbfc0a46f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0058c0 tx=0x7fdc0c0046c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.828+0000 7fdc1e020740 1 --2- 172.21.15.18:0/3581990109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d086210 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fdc040edf30 tx=0x7fdc04050000 comp rx=0 tx=0).stop 2024-02-17T13:01:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.828+0000 7fdc1e020740 1 -- 172.21.15.18:0/3581990109 shutdown_connections 2024-02-17T13:01:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.828+0000 7fdc1e020740 1 --2- 172.21.15.18:0/3581990109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 0x56067d088b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.828+0000 7fdc1e020740 1 --2- 172.21.15.18:0/3581990109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d086210 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.828+0000 7fdc1e020740 1 --2- 172.21.15.18:0/3581990109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d07d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.828+0000 7fdc1e020740 1 -- 172.21.15.18:0/3581990109 >> 172.21.15.18:0/3581990109 conn(0x56067d077980 msgr2=0x56067d077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.828+0000 7fdc1e020740 1 -- 172.21.15.18:0/3581990109 shutdown_connections 2024-02-17T13:01:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.828+0000 7fdc1e020740 1 -- 172.21.15.18:0/3581990109 wait complete. 2024-02-17T13:01:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc1e020740 1 -- start start 2024-02-17T13:01:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc1e020740 1 Processor -- start 2024-02-17T13:01:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc1e020740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d07d250 0x56067d083780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc1e020740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d083cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc1e020740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d086750 0x56067d084200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc1e020740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067cfe5020 con 0x56067d07db70 2024-02-17T13:01:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc1e020740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067ccb8430 con 0x56067d07d250 2024-02-17T13:01:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc1e020740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067cc74e00 con 0x56067d086750 2024-02-17T13:01:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc1e020740 10 client.0 _ll_register_callbacks cb 0x56067cf8ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc189f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d083cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc189f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d083cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40370/0 (socket says 172.21.15.18:40370) 2024-02-17T13:01:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc189f2640 1 -- 172.21.15.18:0/784958131 learned_addr learned my addr 172.21.15.18:0/784958131 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc191f3640 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d07d250 0x56067d083780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.829+0000 7fdc199f4640 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d086750 0x56067d084200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdc189f2640 1 -- 172.21.15.18:0/784958131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d086750 msgr2=0x56067d084200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdc189f2640 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d086750 0x56067d084200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdc189f2640 1 -- 172.21.15.18:0/784958131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d07d250 msgr2=0x56067d083780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdc189f2640 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d07d250 0x56067d083780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdc189f2640 1 -- 172.21.15.18:0/784958131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc141a6830 con 0x56067d07db70 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdc189f2640 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d083cc0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fdc08012d50 tx=0x7fdc0800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574d2669b6f6b8ca server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdbf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdbf8ff9640 1 -- 172.21.15.18:0/784958131 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08017070 con 0x56067d07db70 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdbf8ff9640 1 -- 172.21.15.18:0/784958131 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fdc080120d0 con 0x56067d07db70 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdc1e020740 1 -- 172.21.15.18:0/784958131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56067cce47e0 con 0x56067d07db70 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdbf8ff9640 1 -- 172.21.15.18:0/784958131 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc08012290 con 0x56067d07db70 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdc1e020740 10 client.19599 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdc1e020740 1 -- 172.21.15.18:0/784958131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56067cfe36a0 con 0x56067d07db70 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.830+0000 7fdbf8ff9640 1 -- 172.21.15.18:0/784958131 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56067cce47e0 con 0x56067d07db70 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.831+0000 7fdbf8ff9640 1 -- 172.21.15.18:0/784958131 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdc0801c470 con 0x56067d07db70 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.831+0000 7fdc1e020740 10 client.19599 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.831+0000 7fdc1e020740 10 client.19599 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.831+0000 7fdc1e020740 20 client.19599 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.831+0000 7fdc1e020740 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56067d0968f0 0x56067d098d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.831+0000 7fdc1e020740 4 client.19599 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.831+0000 7fdc1e020740 1 -- 172.21.15.18:0/784958131 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56067cd21720 con 0x56067d0968f0 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.831+0000 7fdc191f3640 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56067d0968f0 0x56067d098d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.831+0000 7fdbf8ff9640 10 client.19599 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.831+0000 7fdc191f3640 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56067d0968f0 0x56067d098d30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30263e527cf2f009 server_cookie=6136239bcde08968 in_seq=0 out_seq=0 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdbf8ff9640 1 -- 172.21.15.18:0/784958131 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fdc040a1a90 con 0x56067d0968f0 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdbf8ff9640 10 client.19599 handle_command_reply: tid=0 2024-02-17T13:01:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdc1e020740 1 client.19599 shutdown 2024-02-17T13:01:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdc1e020740 1 -- 172.21.15.18:0/784958131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 msgr2=0x56067d083cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdc1e020740 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d083cc0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fdc08012d50 tx=0x7fdc0800f790 comp rx=0 tx=0).stop 2024-02-17T13:01:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdc1e020740 1 -- 172.21.15.18:0/784958131 shutdown_connections 2024-02-17T13:01:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdc1e020740 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56067d0968f0 0x56067d098d30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdbf8ff9640 0 client.19599 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:01:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdc1e020740 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d086750 0x56067d084200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdc1e020740 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d083cc0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdc1e020740 1 --2- 172.21.15.18:0/784958131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d07d250 0x56067d083780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdc1e020740 1 -- 172.21.15.18:0/784958131 >> 172.21.15.18:0/784958131 conn(0x56067d077980 msgr2=0x56067d0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdc1e020740 1 -- 172.21.15.18:0/784958131 shutdown_connections 2024-02-17T13:01:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdc1e020740 1 -- 172.21.15.18:0/784958131 wait complete. 2024-02-17T13:01:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.832+0000 7fdc1e020740 20 client.19599 trim_cache size 0 max 16384 2024-02-17T13:01:09.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc1e020740 1 Processor -- start 2024-02-17T13:01:09.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc1e020740 1 -- start start 2024-02-17T13:01:09.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc1e020740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d085eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc1e020740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d084020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc1e020740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 0x56067d0809e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc1e020740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067cf6a130 con 0x56067d07db70 2024-02-17T13:01:09.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc1e020740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067d0774b0 con 0x56067d086750 2024-02-17T13:01:09.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc1e020740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067d0735c0 con 0x56067d07d250 2024-02-17T13:01:09.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc189f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d084020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc189f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d084020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40378/0 (socket says 172.21.15.18:40378) 2024-02-17T13:01:09.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc189f2640 1 -- 172.21.15.18:0/3737665676 learned_addr learned my addr 172.21.15.18:0/3737665676 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc199f4640 1 --2- 172.21.15.18:0/3737665676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 0x56067d0809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc191f3640 1 --2- 172.21.15.18:0/3737665676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d085eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc189f2640 1 -- 172.21.15.18:0/3737665676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 msgr2=0x56067d085eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc189f2640 1 --2- 172.21.15.18:0/3737665676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d085eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc189f2640 1 -- 172.21.15.18:0/3737665676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 msgr2=0x56067d0809e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc189f2640 1 --2- 172.21.15.18:0/3737665676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 0x56067d0809e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc189f2640 1 -- 172.21.15.18:0/3737665676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56067cf7fc20 con 0x56067d07db70 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc191f3640 1 --2- 172.21.15.18:0/3737665676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d085eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.846+0000 7fdc189f2640 1 --2- 172.21.15.18:0/3737665676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d084020 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800ffd0 tx=0x7fdc08012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c5f7cf23094a51 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.847+0000 7fdbfaffd640 1 -- 172.21.15.18:0/3737665676 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08017040 con 0x56067d07db70 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.847+0000 7fdbfaffd640 1 -- 172.21.15.18:0/3737665676 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc08012110 con 0x56067d07db70 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.847+0000 7fdbfaffd640 1 -- 172.21.15.18:0/3737665676 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08004680 con 0x56067d07db70 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.848+0000 7fdc1e020740 1 -- 172.21.15.18:0/3737665676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 msgr2=0x56067d084020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.848+0000 7fdc1e020740 1 --2- 172.21.15.18:0/3737665676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d084020 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800ffd0 tx=0x7fdc08012910 comp rx=0 tx=0).stop 2024-02-17T13:01:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.848+0000 7fdc1e020740 1 -- 172.21.15.18:0/3737665676 shutdown_connections 2024-02-17T13:01:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.848+0000 7fdc1e020740 1 --2- 172.21.15.18:0/3737665676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 0x56067d0809e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.848+0000 7fdc1e020740 1 --2- 172.21.15.18:0/3737665676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d084020 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.848+0000 7fdc1e020740 1 --2- 172.21.15.18:0/3737665676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d085eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.848+0000 7fdc1e020740 1 -- 172.21.15.18:0/3737665676 >> 172.21.15.18:0/3737665676 conn(0x56067d077980 msgr2=0x56067d082050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.848+0000 7fdc1e020740 1 -- 172.21.15.18:0/3737665676 shutdown_connections 2024-02-17T13:01:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.848+0000 7fdc1e020740 1 -- 172.21.15.18:0/3737665676 wait complete. 2024-02-17T13:01:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.848+0000 7fdc1e020740 1 -- start start 2024-02-17T13:01:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.848+0000 7fdc1e020740 1 Processor -- start 2024-02-17T13:01:09.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc1e020740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d08d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc1e020740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d08b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc1e020740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 0x56067d08be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc1e020740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067d08c370 con 0x56067d07db70 2024-02-17T13:01:09.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc1e020740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067d08c4e0 con 0x56067d086750 2024-02-17T13:01:09.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc1e020740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067d08c6a0 con 0x56067d07d250 2024-02-17T13:01:09.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc1e020740 10 client.0 _ll_register_callbacks cb 0x7fdc040f6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:09.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc189f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d08b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc189f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d08b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40386/0 (socket says 172.21.15.18:40386) 2024-02-17T13:01:09.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc189f2640 1 -- 172.21.15.18:0/2752239778 learned_addr learned my addr 172.21.15.18:0/2752239778 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc191f3640 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d08d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc199f4640 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 0x56067d08be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc189f2640 1 -- 172.21.15.18:0/2752239778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 msgr2=0x56067d08d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc189f2640 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d08d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc189f2640 1 -- 172.21.15.18:0/2752239778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 msgr2=0x56067d08be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc189f2640 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 0x56067d08be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc189f2640 1 -- 172.21.15.18:0/2752239778 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56067cfe36a0 con 0x56067d07db70 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc189f2640 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d08b8f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fdc0801c7d0 tx=0x7fdc08012ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8ef7961608618f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdbf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdbf8ff9640 1 -- 172.21.15.18:0/2752239778 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08017040 con 0x56067d07db70 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdbf8ff9640 1 -- 172.21.15.18:0/2752239778 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fdc08000e40 con 0x56067d07db70 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.849+0000 7fdc1e020740 1 -- 172.21.15.18:0/2752239778 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc040b7e90 con 0x56067d07db70 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.850+0000 7fdbf8ff9640 1 -- 172.21.15.18:0/2752239778 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc0802e450 con 0x56067d07db70 2024-02-17T13:01:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.850+0000 7fdc1e020740 10 client.19605 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.850+0000 7fdc1e020740 1 -- 172.21.15.18:0/2752239778 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56067cfa58c0 con 0x56067d07db70 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.850+0000 7fdbf8ff9640 1 -- 172.21.15.18:0/2752239778 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc040b7e90 con 0x56067d07db70 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.850+0000 7fdbf8ff9640 1 -- 172.21.15.18:0/2752239778 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdc08039070 con 0x56067d07db70 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.850+0000 7fdc1e020740 10 client.19605 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.850+0000 7fdc1e020740 10 client.19605 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.850+0000 7fdc1e020740 20 client.19605 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.850+0000 7fdc1e020740 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56067d0968f0 0x56067d0f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.850+0000 7fdc1e020740 4 client.19605 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.850+0000 7fdc1e020740 1 -- 172.21.15.18:0/2752239778 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56067cd70890 con 0x56067d0968f0 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.850+0000 7fdc191f3640 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56067d0968f0 0x56067d0f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc191f3640 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56067d0968f0 0x56067d0f0060 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c842066bac61f9e server_cookie=8dfc8d3bc748a2e2 in_seq=0 out_seq=0 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdbf8ff9640 10 client.19605 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdbf8ff9640 1 -- 172.21.15.18:0/2752239778 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56067cd70890 con 0x56067d0968f0 2024-02-17T13:01:09.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdbf8ff9640 10 client.19605 handle_command_reply: tid=0 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc1e020740 1 client.19605 shutdown 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc1e020740 1 -- 172.21.15.18:0/2752239778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 msgr2=0x56067d08b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc1e020740 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d08b8f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fdc0801c7d0 tx=0x7fdc08012ce0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc1e020740 1 -- 172.21.15.18:0/2752239778 shutdown_connections 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc1e020740 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x56067d0968f0 0x56067d0f0060 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc1e020740 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067d086750 0x56067d08be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc1e020740 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56067d07db70 0x56067d08b8f0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc1e020740 1 --2- 172.21.15.18:0/2752239778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56067d07d250 0x56067d08d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdbf8ff9640 0 client.19605 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc1e020740 1 -- 172.21.15.18:0/2752239778 >> 172.21.15.18:0/2752239778 conn(0x56067d077980 msgr2=0x56067d0762f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc1e020740 1 -- 172.21.15.18:0/2752239778 shutdown_connections 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc1e020740 1 -- 172.21.15.18:0/2752239778 wait complete. 2024-02-17T13:01:09.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.851+0000 7fdc1e020740 20 client.19605 trim_cache size 0 max 16384 2024-02-17T13:01:09.849 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:01:09.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.852+0000 7fdc1b457640 1 -- 172.21.15.18:0/1795576216 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdbfc0a2230 msgr2=0x7fdbfc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.852+0000 7fdc1b457640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdbfc0a2230 0x7fdbfc0a46f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0058c0 tx=0x7fdc0c0046c0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.852+0000 7fdc1b457640 1 -- 172.21.15.18:0/1795576216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc14101370 msgr2=0x7fdc14073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.852+0000 7fdc1b457640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc14101370 0x7fdc14073480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdc0402c700 tx=0x7fdc040325d0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.852+0000 7fdc1b457640 1 -- 172.21.15.18:0/1795576216 shutdown_connections 2024-02-17T13:01:09.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.852+0000 7fdc1b457640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdbfc0a2230 0x7fdbfc0a46f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.852+0000 7fdc1b457640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc14073f00 0x7fdc141a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.852+0000 7fdc1b457640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc14106440 0x7fdc140739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.852+0000 7fdc1b457640 1 --2- 172.21.15.18:0/1795576216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc14101370 0x7fdc14073480 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.852+0000 7fdc1b457640 1 -- 172.21.15.18:0/1795576216 >> 172.21.15.18:0/1795576216 conn(0x7fdc140fcc10 msgr2=0x7fdc140fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.852+0000 7fdc1b457640 1 -- 172.21.15.18:0/1795576216 shutdown_connections 2024-02-17T13:01:09.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.852+0000 7fdc1b457640 1 -- 172.21.15.18:0/1795576216 wait complete. 2024-02-17T13:01:09.857 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:01:09.857 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T13:01:09.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.988+0000 7fd8df60f640 1 -- 172.21.15.18:0/2526450772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d8100d20 msgr2=0x7fd8d810ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.988+0000 7fd8df60f640 1 --2- 172.21.15.18:0/2526450772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d8100d20 0x7fd8d810ce70 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009ca0 tx=0x7fd8d002c0f0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.988+0000 7fd8df60f640 1 -- 172.21.15.18:0/2526450772 shutdown_connections 2024-02-17T13:01:09.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.988+0000 7fd8df60f640 1 --2- 172.21.15.18:0/2526450772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d8100d20 0x7fd8d810ce70 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.988+0000 7fd8df60f640 1 --2- 172.21.15.18:0/2526450772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d8100360 0x7fd8d81007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.988+0000 7fd8df60f640 1 --2- 172.21.15.18:0/2526450772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d8106240 0x7fd8d8106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.988+0000 7fd8df60f640 1 -- 172.21.15.18:0/2526450772 >> 172.21.15.18:0/2526450772 conn(0x7fd8d8077060 msgr2=0x7fd8d8077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.988+0000 7fd8df60f640 1 -- 172.21.15.18:0/2526450772 shutdown_connections 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.988+0000 7fd8df60f640 1 -- 172.21.15.18:0/2526450772 wait complete. 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8df60f640 1 Processor -- start 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8df60f640 1 -- start start 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8df60f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d8100360 0x7fd8d81977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8df60f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d8100d20 0x7fd8d8197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8df60f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d8106240 0x7fd8d819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8df60f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d819c620 con 0x7fd8d8100360 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8df60f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d819c790 con 0x7fd8d8106240 2024-02-17T13:01:09.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8df60f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d819c950 con 0x7fd8d8100d20 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8dd3ab640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d8100360 0x7fd8d81977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8dd3ab640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d8100360 0x7fd8d81977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40410/0 (socket says 172.21.15.18:40410) 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8dd3ab640 1 -- 172.21.15.18:0/2738668475 learned_addr learned my addr 172.21.15.18:0/2738668475 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8dcbaa640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d8100d20 0x7fd8d8197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.989+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d8106240 0x7fd8d819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.990+0000 7fd8ddbac640 1 -- 172.21.15.18:0/2738668475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d8100d20 msgr2=0x7fd8d8197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.990+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d8100d20 0x7fd8d8197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.990+0000 7fd8ddbac640 1 -- 172.21.15.18:0/2738668475 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d8100360 msgr2=0x7fd8d81977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.990+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d8100360 0x7fd8d81977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.990+0000 7fd8ddbac640 1 -- 172.21.15.18:0/2738668475 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d4009490 con 0x7fd8d8106240 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.990+0000 7fd8dd3ab640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d8100360 0x7fd8d81977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.990+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d8106240 0x7fd8d819c020 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd8d002c700 tx=0x7fd8d0034b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.990+0000 7fd8ce7fc640 1 -- 172.21.15.18:0/2738668475 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00328b0 con 0x7fd8d8106240 2024-02-17T13:01:09.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.990+0000 7fd8df60f640 1 -- 172.21.15.18:0/2738668475 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d0009900 con 0x7fd8d8106240 2024-02-17T13:01:09.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.990+0000 7fd8ce7fc640 1 -- 172.21.15.18:0/2738668475 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8d0003e50 con 0x7fd8d8106240 2024-02-17T13:01:09.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.990+0000 7fd8ce7fc640 1 -- 172.21.15.18:0/2738668475 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0004840 con 0x7fd8d8106240 2024-02-17T13:01:09.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.990+0000 7fd8df60f640 1 -- 172.21.15.18:0/2738668475 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d81a42d0 con 0x7fd8d8106240 2024-02-17T13:01:09.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.992+0000 7fd8ce7fc640 1 -- 172.21.15.18:0/2738668475 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd8d000a040 con 0x7fd8d8106240 2024-02-17T13:01:09.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8e21d9740 1 Processor -- start 2024-02-17T13:01:09.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8ce7fc640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8c40a2230 0x7fd8c40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8e21d9740 1 -- start start 2024-02-17T13:01:09.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8e21d9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d47f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8dcbaa640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8c40a2230 0x7fd8c40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8ce7fc640 1 -- 172.21.15.18:0/2738668475 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fd8d00f0ce0 con 0x7fd8d8106240 2024-02-17T13:01:09.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8e21d9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d47f820 0x555c5d48c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8e21d9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 0x555c5d48ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8e21d9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5d43a960 con 0x555c5d47ef00 2024-02-17T13:01:09.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8e21d9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5d3ee310 con 0x555c5d48cb60 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8e21d9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5d3eb840 con 0x555c5d47f820 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8ddbac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 0x555c5d48ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8ddbac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 0x555c5d48ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:54698/0 (socket says 172.21.15.18:54698) 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8dd3ab640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d47f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.993+0000 7fd8ddbac640 1 -- 172.21.15.18:0/2589100040 learned_addr learned my addr 172.21.15.18:0/2589100040 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.994+0000 7fd8dcbaa640 1 --2- 172.21.15.18:0/2589100040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d47f820 0x555c5d48c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.994+0000 7fd8ddbac640 1 -- 172.21.15.18:0/2589100040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d47f820 msgr2=0x555c5d48c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.994+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/2589100040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d47f820 0x555c5d48c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.994+0000 7fd8ddbac640 1 -- 172.21.15.18:0/2589100040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 msgr2=0x555c5d47f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.994+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/2589100040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d47f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.994+0000 7fd8ddbac640 1 -- 172.21.15.18:0/2589100040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c5d3a9060 con 0x555c5d48cb60 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.994+0000 7fd8dcbaa640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8c40a2230 0x7fd8c40a46f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8004650 tx=0x7fd8c800bef0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.994+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/2589100040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 0x555c5d48ef50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00b9c00 tx=0x7fd8d00bd1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3160a21638e0d24e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.994+0000 7fd8c37fe640 1 -- 172.21.15.18:0/2589100040 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00c18f0 con 0x555c5d48cb60 2024-02-17T13:01:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.994+0000 7fd8c37fe640 1 -- 172.21.15.18:0/2589100040 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8d00c5f40 con 0x555c5d48cb60 2024-02-17T13:01:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.994+0000 7fd8c37fe640 1 -- 172.21.15.18:0/2589100040 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00b9e10 con 0x555c5d48cb60 2024-02-17T13:01:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.995+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2589100040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 msgr2=0x555c5d48ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.995+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2589100040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 0x555c5d48ef50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00b9c00 tx=0x7fd8d00bd1c0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.995+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2589100040 shutdown_connections 2024-02-17T13:01:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.995+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2589100040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 0x555c5d48ef50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.995+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2589100040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d47f820 0x555c5d48c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.995+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2589100040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d47f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.995+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2589100040 >> 172.21.15.18:0/2589100040 conn(0x555c5d479980 msgr2=0x555c5d479d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.995+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2589100040 shutdown_connections 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.995+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2589100040 wait complete. 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.995+0000 7fd8e21d9740 1 -- start start 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.995+0000 7fd8e21d9740 1 Processor -- start 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8e21d9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d47ef00 0x555c5d489ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8e21d9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47f820 0x555c5d48a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8e21d9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d48cb60 0x555c5d48a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8e21d9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5d3e7020 con 0x555c5d47f820 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8e21d9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5d0ba430 con 0x555c5d47ef00 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8e21d9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5d076e00 con 0x555c5d48cb60 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8e21d9740 10 client.0 _ll_register_callbacks cb 0x555c5d38ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8ddbac640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d48cb60 0x555c5d48a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8ddbac640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d48cb60 0x555c5d48a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:38312/0 (socket says 172.21.15.18:38312) 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8ddbac640 1 -- 172.21.15.18:0/2927707830 learned_addr learned my addr 172.21.15.18:0/2927707830 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8dd3ab640 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d47ef00 0x555c5d489ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8dcbaa640 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47f820 0x555c5d48a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8ddbac640 1 -- 172.21.15.18:0/2927707830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d47ef00 msgr2=0x555c5d489ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d47ef00 0x555c5d489ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8ddbac640 1 -- 172.21.15.18:0/2927707830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47f820 msgr2=0x555c5d48a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47f820 0x555c5d48a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.996+0000 7fd8ddbac640 1 -- 172.21.15.18:0/2927707830 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d81a42d0 con 0x555c5d48cb60 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.997+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d48cb60 0x555c5d48a550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0037040 tx=0x7fd8d00ee520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8e0005510ee6265 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.997+0000 7fd8c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.997+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/2927707830 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00b9e10 con 0x555c5d48cb60 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.997+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/2927707830 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fd8d00b9f70 con 0x555c5d48cb60 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.997+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2927707830 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c5d0e67e0 con 0x555c5d48cb60 2024-02-17T13:01:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.997+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/2927707830 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8d00bda00 con 0x555c5d48cb60 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.997+0000 7fd8e21d9740 10 client.28636 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.997+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2927707830 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x555c5d3e56a0 con 0x555c5d48cb60 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.998+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/2927707830 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555c5d0e67e0 con 0x555c5d48cb60 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.998+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/2927707830 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd8d00eeb40 con 0x555c5d48cb60 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.998+0000 7fd8e21d9740 10 client.28636 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.998+0000 7fd8e21d9740 10 client.28636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.998+0000 7fd8e21d9740 20 client.28636 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.998+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555c5d49c890 0x555c5d49ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.998+0000 7fd8e21d9740 4 client.28636 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.998+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2927707830 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c5d123720 con 0x555c5d49c890 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.998+0000 7fd8dd3ab640 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555c5d49c890 0x555c5d49ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.999+0000 7fd8dd3ab640 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555c5d49c890 0x555c5d49ec80 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb95f77216fc3e45 server_cookie=47716bff310e0497 in_seq=0 out_seq=0 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.999+0000 7fd8c0ff9640 10 client.28636 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.999+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/2927707830 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c5d123720 con 0x555c5d49c890 2024-02-17T13:01:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.999+0000 7fd8c0ff9640 10 client.28636 handle_command_reply: tid=0 2024-02-17T13:01:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:09.999+0000 7fd8e21d9740 1 client.28636 shutdown 2024-02-17T13:01:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.000+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2927707830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d48cb60 msgr2=0x555c5d48a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.000+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d48cb60 0x555c5d48a550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0037040 tx=0x7fd8d00ee520 comp rx=0 tx=0).stop 2024-02-17T13:01:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.000+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2927707830 shutdown_connections 2024-02-17T13:01:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.000+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555c5d49c890 0x555c5d49ec80 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.000+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d48cb60 0x555c5d48a550 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.000+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47f820 0x555c5d48a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.000+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2927707830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d47ef00 0x555c5d489ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.000+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2927707830 >> 172.21.15.18:0/2927707830 conn(0x555c5d479980 msgr2=0x555c5d47b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.000+0000 7fd8c0ff9640 0 client.28636 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.000+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2927707830 shutdown_connections 2024-02-17T13:01:10.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.000+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2927707830 wait complete. 2024-02-17T13:01:10.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.000+0000 7fd8e21d9740 20 client.28636 trim_cache size 0 max 16384 2024-02-17T13:01:10.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.013+0000 7fd8e21d9740 1 Processor -- start 2024-02-17T13:01:10.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.013+0000 7fd8e21d9740 1 -- start start 2024-02-17T13:01:10.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.013+0000 7fd8e21d9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d48c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:10.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.013+0000 7fd8e21d9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d47f820 0x555c5d48a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.013+0000 7fd8e21d9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 0x555c5d488ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.013+0000 7fd8e21d9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5d36c130 con 0x555c5d47ef00 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.013+0000 7fd8e21d9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5d0e67e0 con 0x555c5d48cb60 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.013+0000 7fd8e21d9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5d46ba40 con 0x555c5d47f820 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.013+0000 7fd8dd3ab640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d48c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8dd3ab640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d48c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40444/0 (socket says 172.21.15.18:40444) 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8dcbaa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d47f820 0x555c5d48a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8ddbac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 0x555c5d488ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8dd3ab640 1 -- 172.21.15.18:0/1738830628 learned_addr learned my addr 172.21.15.18:0/1738830628 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8ddbac640 1 -- 172.21.15.18:0/1738830628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d47f820 msgr2=0x555c5d48a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/1738830628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d47f820 0x555c5d48a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8ddbac640 1 -- 172.21.15.18:0/1738830628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 msgr2=0x555c5d48c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/1738830628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d48c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8ddbac640 1 -- 172.21.15.18:0/1738830628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c5d381c20 con 0x555c5d48cb60 2024-02-17T13:01:10.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8dd3ab640 1 --2- 172.21.15.18:0/1738830628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d48c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:10.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8dcbaa640 1 --2- 172.21.15.18:0/1738830628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d47f820 0x555c5d48a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:10.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/1738830628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 0x555c5d488ab0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00b9c00 tx=0x7fd8d00bd1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8161c8b5effb3538 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:10.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8c2ffd640 1 -- 172.21.15.18:0/1738830628 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00d62b0 con 0x555c5d48cb60 2024-02-17T13:01:10.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8c2ffd640 1 -- 172.21.15.18:0/1738830628 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8d00b9e10 con 0x555c5d48cb60 2024-02-17T13:01:10.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.014+0000 7fd8c2ffd640 1 -- 172.21.15.18:0/1738830628 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00c5ab0 con 0x555c5d48cb60 2024-02-17T13:01:10.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.015+0000 7fd8e21d9740 1 -- 172.21.15.18:0/1738830628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 msgr2=0x555c5d488ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:10.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.015+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/1738830628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 0x555c5d488ab0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00b9c00 tx=0x7fd8d00bd1c0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.015+0000 7fd8e21d9740 1 -- 172.21.15.18:0/1738830628 shutdown_connections 2024-02-17T13:01:10.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.015+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/1738830628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d48cb60 0x555c5d488ab0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.015+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/1738830628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d47f820 0x555c5d48a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.015+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/1738830628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d48c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.015+0000 7fd8e21d9740 1 -- 172.21.15.18:0/1738830628 >> 172.21.15.18:0/1738830628 conn(0x555c5d479980 msgr2=0x555c5d4883a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:10.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.015+0000 7fd8e21d9740 1 -- 172.21.15.18:0/1738830628 shutdown_connections 2024-02-17T13:01:10.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.015+0000 7fd8e21d9740 1 -- 172.21.15.18:0/1738830628 wait complete. 2024-02-17T13:01:10.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.015+0000 7fd8e21d9740 1 -- start start 2024-02-17T13:01:10.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8e21d9740 1 Processor -- start 2024-02-17T13:01:10.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8e21d9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d491af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:10.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8e21d9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d47f820 0x555c5d492030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:10.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8e21d9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d48cb60 0x555c5d4ebd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:10.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8e21d9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5d4ec2b0 con 0x555c5d47ef00 2024-02-17T13:01:10.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8e21d9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5d4ec420 con 0x555c5d47f820 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8e21d9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5d4ec5e0 con 0x555c5d48cb60 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8e21d9740 10 client.0 _ll_register_callbacks cb 0x7fd8d00b9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8dd3ab640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d491af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8dcbaa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d47f820 0x555c5d492030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8dd3ab640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d491af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40456/0 (socket says 172.21.15.18:40456) 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8dd3ab640 1 -- 172.21.15.18:0/2864866574 learned_addr learned my addr 172.21.15.18:0/2864866574 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8dcbaa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d47f820 0x555c5d492030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:54724/0 (socket says 172.21.15.18:54724) 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8ddbac640 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d48cb60 0x555c5d4ebd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8dcbaa640 1 -- 172.21.15.18:0/2864866574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d48cb60 msgr2=0x555c5d4ebd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8dcbaa640 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d48cb60 0x555c5d4ebd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8dcbaa640 1 -- 172.21.15.18:0/2864866574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 msgr2=0x555c5d491af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8dcbaa640 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d491af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8dcbaa640 1 -- 172.21.15.18:0/2864866574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d4009d00 con 0x555c5d47f820 2024-02-17T13:01:10.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.016+0000 7fd8dd3ab640 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d491af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:10.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.017+0000 7fd8dcbaa640 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d47f820 0x555c5d492030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x555c5d48b0a0 tx=0x7fd8c8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c7ddbc9c3db5c46 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:10.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.017+0000 7fd8c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:01:10.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.017+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/2864866574 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8021800 con 0x555c5d47f820 2024-02-17T13:01:10.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.017+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/2864866574 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fd8c8021960 con 0x555c5d47f820 2024-02-17T13:01:10.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.017+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2864866574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8d00c5f40 con 0x555c5d47f820 2024-02-17T13:01:10.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.017+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/2864866574 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8c8021b60 con 0x555c5d47f820 2024-02-17T13:01:10.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.017+0000 7fd8e21d9740 10 client.28610 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:10.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.017+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2864866574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7fd8c8003dc0 con 0x555c5d47f820 2024-02-17T13:01:10.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.017+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/2864866574 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8d00c5f40 con 0x555c5d47f820 2024-02-17T13:01:10.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.017+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/2864866574 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd8c802a3e0 con 0x555c5d47f820 2024-02-17T13:01:10.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.018+0000 7fd8e21d9740 10 client.28610 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:10.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.018+0000 7fd8e21d9740 10 client.28610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:10.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.018+0000 7fd8e21d9740 20 client.28610 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:10.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.018+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555c5d4f3f00 0x555c5d4f6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:10.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.018+0000 7fd8e21d9740 4 client.28610 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:01:10.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.018+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2864866574 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c5d172890 con 0x555c5d4f3f00 2024-02-17T13:01:10.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.018+0000 7fd8dd3ab640 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555c5d4f3f00 0x555c5d4f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:10.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.018+0000 7fd8c0ff9640 10 client.28610 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:10.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.018+0000 7fd8dd3ab640 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555c5d4f3f00 0x555c5d4f6320 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c546efd853403748 server_cookie=4e67ce7948ab32a5 in_seq=0 out_seq=0 2024-02-17T13:01:10.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/2864866574 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c5d172890 con 0x555c5d4f3f00 2024-02-17T13:01:10.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8c0ff9640 10 client.28610 handle_command_reply: tid=0 2024-02-17T13:01:10.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8e21d9740 1 client.28610 shutdown 2024-02-17T13:01:10.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2864866574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d47f820 msgr2=0x555c5d492030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:10.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d47f820 0x555c5d492030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x555c5d48b0a0 tx=0x7fd8c8004730 comp rx=0 tx=0).stop 2024-02-17T13:01:10.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2864866574 shutdown_connections 2024-02-17T13:01:10.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x555c5d4f3f00 0x555c5d4f6320 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555c5d48cb60 0x555c5d4ebd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5d47f820 0x555c5d492030 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8c0ff9640 0 client.28610 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:10.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8e21d9740 1 --2- 172.21.15.18:0/2864866574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555c5d47ef00 0x555c5d491af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2864866574 >> 172.21.15.18:0/2864866574 conn(0x555c5d479980 msgr2=0x555c5d4782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:10.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2864866574 shutdown_connections 2024-02-17T13:01:10.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8e21d9740 1 -- 172.21.15.18:0/2864866574 wait complete. 2024-02-17T13:01:10.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8e21d9740 20 client.28610 trim_cache size 0 max 16384 2024-02-17T13:01:10.019 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:01:10.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8df60f640 1 -- 172.21.15.18:0/2738668475 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8c40a2230 msgr2=0x7fd8c40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:10.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8df60f640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8c40a2230 0x7fd8c40a46f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8004650 tx=0x7fd8c800bef0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8df60f640 1 -- 172.21.15.18:0/2738668475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d8106240 msgr2=0x7fd8d819c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:10.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.019+0000 7fd8df60f640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d8106240 0x7fd8d819c020 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd8d002c700 tx=0x7fd8d0034b10 comp rx=0 tx=0).stop 2024-02-17T13:01:10.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.020+0000 7fd8df60f640 1 -- 172.21.15.18:0/2738668475 shutdown_connections 2024-02-17T13:01:10.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.020+0000 7fd8df60f640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8c40a2230 0x7fd8c40a46f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.020+0000 7fd8df60f640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d8106240 0x7fd8d819c020 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.020+0000 7fd8df60f640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d8100d20 0x7fd8d8197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.020+0000 7fd8df60f640 1 --2- 172.21.15.18:0/2738668475 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d8100360 0x7fd8d81977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:10.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.020+0000 7fd8df60f640 1 -- 172.21.15.18:0/2738668475 >> 172.21.15.18:0/2738668475 conn(0x7fd8d8077060 msgr2=0x7fd8d80ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:10.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.020+0000 7fd8df60f640 1 -- 172.21.15.18:0/2738668475 shutdown_connections 2024-02-17T13:01:10.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:10.020+0000 7fd8df60f640 1 -- 172.21.15.18:0/2738668475 wait complete. 2024-02-17T13:01:10.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 561782 4.90 MB/sec execute 505 sec latency 20.658 ms 2024-02-17T13:01:10.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:10 smithi146 ceph-mon[36454]: pgmap v353: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-02-17T13:01:10.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:10 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:01:10.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:10 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:01:10.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:10 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3275639975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:01:10.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:10 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:01:10.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:10 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:01:10.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:10 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:01:10.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:10 smithi018 ceph-mon[32625]: pgmap v353: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-02-17T13:01:10.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:10 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:01:10.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:10 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:01:10.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:10 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3275639975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:01:10.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:10 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:01:10.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:10 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:01:10.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:10 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:01:10.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:10 smithi119 ceph-mon[36188]: pgmap v353: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-02-17T13:01:10.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:10 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:01:10.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:10 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:01:10.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:10 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3275639975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:01:10.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:10 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:01:10.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:10 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:01:10.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:10 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:01:11.023 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T13:01:11.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:01:11.023 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T13:01:11.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.141+0000 7fdf56313640 1 -- 172.21.15.18:0/2749642374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf50105330 msgr2=0x7fdf5010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.141+0000 7fdf56313640 1 --2- 172.21.15.18:0/2749642374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf50105330 0x7fdf5010cf00 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009cd0 tx=0x7fdf4802c0f0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.141+0000 7fdf56313640 1 -- 172.21.15.18:0/2749642374 shutdown_connections 2024-02-17T13:01:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.141+0000 7fdf56313640 1 --2- 172.21.15.18:0/2749642374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf50105330 0x7fdf5010cf00 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.141+0000 7fdf56313640 1 --2- 172.21.15.18:0/2749642374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf50100c10 0x7fdf50104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.141+0000 7fdf56313640 1 --2- 172.21.15.18:0/2749642374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf50100260 0x7fdf50100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.141+0000 7fdf56313640 1 -- 172.21.15.18:0/2749642374 >> 172.21.15.18:0/2749642374 conn(0x7fdf50077180 msgr2=0x7fdf50077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.141+0000 7fdf56313640 1 -- 172.21.15.18:0/2749642374 shutdown_connections 2024-02-17T13:01:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.141+0000 7fdf56313640 1 -- 172.21.15.18:0/2749642374 wait complete. 2024-02-17T13:01:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.142+0000 7fdf56313640 1 Processor -- start 2024-02-17T13:01:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.142+0000 7fdf56313640 1 -- start start 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.142+0000 7fdf56313640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf50100260 0x7fdf50197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.142+0000 7fdf56313640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf50105330 0x7fdf50197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.142+0000 7fdf56313640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf5019bff0 0x7fdf5019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.142+0000 7fdf56313640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5019c9e0 con 0x7fdf5019bff0 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.142+0000 7fdf56313640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5019cb50 con 0x7fdf50100260 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.142+0000 7fdf56313640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5019ccc0 con 0x7fdf50105330 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf548b0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf5019bff0 0x7fdf5019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf548b0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf5019bff0 0x7fdf5019c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40478/0 (socket says 172.21.15.18:40478) 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf548b0640 1 -- 172.21.15.18:0/1370414115 learned_addr learned my addr 172.21.15.18:0/1370414115 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf50105330 0x7fdf50197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf4ffff640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf50100260 0x7fdf50197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf548b0640 1 -- 172.21.15.18:0/1370414115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf50105330 msgr2=0x7fdf50197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf548b0640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf50105330 0x7fdf50197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf548b0640 1 -- 172.21.15.18:0/1370414115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf50100260 msgr2=0x7fdf50197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf548b0640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf50100260 0x7fdf50197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf548b0640 1 -- 172.21.15.18:0/1370414115 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf48009900 con 0x7fdf5019bff0 2024-02-17T13:01:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf548b0640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf5019bff0 0x7fdf5019c4a0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fdf480348f0 tx=0x7fdf48034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf4d7fa640 1 -- 172.21.15.18:0/1370414115 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480328c0 con 0x7fdf5019bff0 2024-02-17T13:01:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf56313640 1 -- 172.21.15.18:0/1370414115 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf501a42a0 con 0x7fdf5019bff0 2024-02-17T13:01:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf4d7fa640 1 -- 172.21.15.18:0/1370414115 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf48003e50 con 0x7fdf5019bff0 2024-02-17T13:01:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf4d7fa640 1 -- 172.21.15.18:0/1370414115 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480048a0 con 0x7fdf5019bff0 2024-02-17T13:01:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.143+0000 7fdf56313640 1 -- 172.21.15.18:0/1370414115 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf501a46d0 con 0x7fdf5019bff0 2024-02-17T13:01:11.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.145+0000 7fdf4d7fa640 1 -- 172.21.15.18:0/1370414115 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fdf4800a040 con 0x7fdf5019bff0 2024-02-17T13:01:11.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf58fc9740 1 Processor -- start 2024-02-17T13:01:11.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf58fc9740 1 -- start start 2024-02-17T13:01:11.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf58fc9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 0x55deecb81760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf58fc9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81d30 0x55deecb8a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf58fc9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 0x55deecb8cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf58fc9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deecaeff90 con 0x55deecb8a8b0 2024-02-17T13:01:11.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf58fc9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deecaed4c0 con 0x55deecb81380 2024-02-17T13:01:11.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf58fc9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deecae8ca0 con 0x55deecb81d30 2024-02-17T13:01:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf4d7fa640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdf380a2230 0x7fdf380a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf4f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 0x55deecb8cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf4f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 0x55deecb8cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40484/0 (socket says 172.21.15.18:40484) 2024-02-17T13:01:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf4f7fe640 1 -- 172.21.15.18:0/3992418165 learned_addr learned my addr 172.21.15.18:0/3992418165 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf548b0640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdf380a2230 0x7fdf380a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/3992418165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 0x55deecb81760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf4ffff640 1 --2- 172.21.15.18:0/3992418165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81d30 0x55deecb8a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf4d7fa640 1 -- 172.21.15.18:0/1370414115 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fdf480f0c50 con 0x7fdf5019bff0 2024-02-17T13:01:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf4f7fe640 1 -- 172.21.15.18:0/3992418165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81d30 msgr2=0x55deecb8a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/3992418165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81d30 0x55deecb8a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf4f7fe640 1 -- 172.21.15.18:0/3992418165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 msgr2=0x55deecb81760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/3992418165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 0x55deecb81760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.146+0000 7fdf4f7fe640 1 -- 172.21.15.18:0/3992418165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55deecaab0b0 con 0x55deecb8a8b0 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/3992418165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 0x55deecb8cd00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fdf440126f0 tx=0x7fdf4400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce205baca4472949 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf548b0640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdf380a2230 0x7fdf380a46f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdf480b9b70 tx=0x7fdf48090780 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf367fc640 1 -- 172.21.15.18:0/3992418165 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44012c70 con 0x55deecb8a8b0 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf367fc640 1 -- 172.21.15.18:0/3992418165 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf44012dd0 con 0x55deecb8a8b0 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf367fc640 1 -- 172.21.15.18:0/3992418165 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf440049e0 con 0x55deecb8a8b0 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3992418165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 msgr2=0x55deecb8cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3992418165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 0x55deecb8cd00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fdf440126f0 tx=0x7fdf4400b6e0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3992418165 shutdown_connections 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3992418165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 0x55deecb8cd00 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3992418165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81d30 0x55deecb8a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3992418165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 0x55deecb81760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3992418165 >> 172.21.15.18:0/3992418165 conn(0x55deecb7b840 msgr2=0x55deecb7bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3992418165 shutdown_connections 2024-02-17T13:01:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.147+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3992418165 wait complete. 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf58fc9740 1 -- start start 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf58fc9740 1 Processor -- start 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf58fc9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 0x55deecb87830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf58fc9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81d30 0x55deecb87d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf58fc9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 0x55deecb882b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf58fc9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deeca8df40 con 0x55deecb8a8b0 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf58fc9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deec7bc4b0 con 0x55deecb81380 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf58fc9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deeca6e670 con 0x55deecb81d30 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf58fc9740 10 client.0 _ll_register_callbacks cb 0x55deecadbf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf4f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 0x55deecb882b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf4f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 0x55deecb882b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40494/0 (socket says 172.21.15.18:40494) 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81d30 0x55deecb87d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf4f7fe640 1 -- 172.21.15.18:0/3511806560 learned_addr learned my addr 172.21.15.18:0/3511806560 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.148+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/3511806560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 0x55deecb87830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.149+0000 7fdf4ffff640 1 -- 172.21.15.18:0/3511806560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 msgr2=0x55deecb87830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.149+0000 7fdf4ffff640 1 --2- 172.21.15.18:0/3511806560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 0x55deecb87830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.149+0000 7fdf4ffff640 1 -- 172.21.15.18:0/3511806560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 msgr2=0x55deecb882b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.149+0000 7fdf4ffff640 1 --2- 172.21.15.18:0/3511806560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 0x55deecb882b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.149+0000 7fdf4ffff640 1 -- 172.21.15.18:0/3511806560 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4401a040 con 0x55deecb81d30 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.149+0000 7fdf4ffff640 1 --2- 172.21.15.18:0/3511806560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81d30 0x55deecb87d70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55deecb80e50 tx=0x7fdf40012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f451f9649278382 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.149+0000 7fdf2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.149+0000 7fdf2f7fe640 1 -- 172.21.15.18:0/3511806560 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40010d40 con 0x55deecb81d30 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.149+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3511806560 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55deecac2f40 con 0x55deecb81d30 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.149+0000 7fdf2f7fe640 1 -- 172.21.15.18:0/3511806560 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fdf40012e40 con 0x55deecb81d30 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.149+0000 7fdf2f7fe640 1 -- 172.21.15.18:0/3511806560 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf40010ea0 con 0x55deecb81d30 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.150+0000 7fdf58fc9740 10 client.28654 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.150+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3511806560 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55deecae7320 con 0x55deecb81d30 2024-02-17T13:01:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.150+0000 7fdf2f7fe640 1 -- 172.21.15.18:0/3511806560 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55deecac2f40 con 0x55deecb81d30 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.150+0000 7fdf2f7fe640 1 -- 172.21.15.18:0/3511806560 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf40022070 con 0x55deecb81d30 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.150+0000 7fdf58fc9740 10 client.28654 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.150+0000 7fdf58fc9740 10 client.28654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.150+0000 7fdf58fc9740 20 client.28654 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.150+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3511806560 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55deecb9aeb0 0x55deecb9d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.150+0000 7fdf58fc9740 4 client.28654 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.150+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3511806560 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55deecadf310 con 0x55deecb9aeb0 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.150+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/3511806560 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55deecb9aeb0 0x55deecb9d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/3511806560 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55deecb9aeb0 0x55deecb9d2d0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e8cd5d5e65016a server_cookie=da3fddcf44fabcd7 in_seq=0 out_seq=0 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf2f7fe640 10 client.28654 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf2f7fe640 1 -- 172.21.15.18:0/3511806560 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55deecadf310 con 0x55deecb9aeb0 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf2f7fe640 10 client.28654 handle_command_reply: tid=0 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf58fc9740 1 client.28654 shutdown 2024-02-17T13:01:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3511806560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81d30 msgr2=0x55deecb87d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3511806560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81d30 0x55deecb87d70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55deecb80e50 tx=0x7fdf40012bf0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3511806560 shutdown_connections 2024-02-17T13:01:11.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3511806560 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55deecb9aeb0 0x55deecb9d2d0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3511806560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb8a8b0 0x55deecb882b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3511806560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81d30 0x55deecb87d70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf2f7fe640 0 client.28654 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:11.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3511806560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 0x55deecb87830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3511806560 >> 172.21.15.18:0/3511806560 conn(0x55deecb7b840 msgr2=0x55deecb7d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:11.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3511806560 shutdown_connections 2024-02-17T13:01:11.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.151+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3511806560 wait complete. 2024-02-17T13:01:11.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.152+0000 7fdf58fc9740 20 client.28654 trim_cache size 0 max 16384 2024-02-17T13:01:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf58fc9740 1 Processor -- start 2024-02-17T13:01:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf58fc9740 1 -- start start 2024-02-17T13:01:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf58fc9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81380 0x55deecb89f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf58fc9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 0x55deecb880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf58fc9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb8a8b0 0x55deecb84ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf58fc9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deecb97920 con 0x55deecb81d30 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf58fc9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deecb80410 con 0x55deecb8a8b0 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf58fc9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deecba24f0 con 0x55deecb81380 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 0x55deecb880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 0x55deecb880d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40500/0 (socket says 172.21.15.18:40500) 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf4f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81380 0x55deecb89f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf4ffff640 1 -- 172.21.15.18:0/2647868821 learned_addr learned my addr 172.21.15.18:0/2647868821 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/2647868821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb8a8b0 0x55deecb84ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf4ffff640 1 -- 172.21.15.18:0/2647868821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81380 msgr2=0x55deecb89f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf4ffff640 1 --2- 172.21.15.18:0/2647868821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81380 0x55deecb89f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf4ffff640 1 -- 172.21.15.18:0/2647868821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb8a8b0 msgr2=0x55deecb84ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf4ffff640 1 --2- 172.21.15.18:0/2647868821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb8a8b0 0x55deecb84ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf4ffff640 1 -- 172.21.15.18:0/2647868821 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55deeca83c00 con 0x55deecb81d30 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/2647868821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81380 0x55deecb89f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf4ffff640 1 --2- 172.21.15.18:0/2647868821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 0x55deecb880d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000afb0 tx=0x7fdf40005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe49f5175a09aad server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.167+0000 7fdf35ffb640 1 -- 172.21.15.18:0/2647868821 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4000a340 con 0x55deecb81d30 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.168+0000 7fdf35ffb640 1 -- 172.21.15.18:0/2647868821 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf4000e3b0 con 0x55deecb81d30 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.168+0000 7fdf35ffb640 1 -- 172.21.15.18:0/2647868821 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40005400 con 0x55deecb81d30 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.168+0000 7fdf58fc9740 1 -- 172.21.15.18:0/2647868821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 msgr2=0x55deecb880d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.168+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/2647868821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 0x55deecb880d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000afb0 tx=0x7fdf40005b90 comp rx=0 tx=0).stop 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.168+0000 7fdf58fc9740 1 -- 172.21.15.18:0/2647868821 shutdown_connections 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.168+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/2647868821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb8a8b0 0x55deecb84ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.168+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/2647868821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 0x55deecb880d0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.168+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/2647868821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb81380 0x55deecb89f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.168+0000 7fdf58fc9740 1 -- 172.21.15.18:0/2647868821 >> 172.21.15.18:0/2647868821 conn(0x55deecb7b840 msgr2=0x55deecb86100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.168+0000 7fdf58fc9740 1 -- 172.21.15.18:0/2647868821 shutdown_connections 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.168+0000 7fdf58fc9740 1 -- 172.21.15.18:0/2647868821 wait complete. 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.168+0000 7fdf58fc9740 1 -- start start 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf58fc9740 1 Processor -- start 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf58fc9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 0x55deecb90a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf58fc9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 0x55deecb90f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf58fc9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb8a8b0 0x55deecbf13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf58fc9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deecb91700 con 0x55deecb81d30 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf58fc9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deecbf1930 con 0x55deecb81380 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf58fc9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deecbf1af0 con 0x55deecb8a8b0 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf58fc9740 10 client.0 _ll_register_callbacks cb 0x7fdf4400ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 0x55deecb90f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf4ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 0x55deecb90f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40512/0 (socket says 172.21.15.18:40512) 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf4ffff640 1 -- 172.21.15.18:0/3697919855 learned_addr learned my addr 172.21.15.18:0/3697919855 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb8a8b0 0x55deecbf13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 0x55deecb90a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf4ffff640 1 -- 172.21.15.18:0/3697919855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb8a8b0 msgr2=0x55deecbf13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf4ffff640 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb8a8b0 0x55deecbf13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf4ffff640 1 -- 172.21.15.18:0/3697919855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 msgr2=0x55deecb90a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf4ffff640 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 0x55deecb90a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf4ffff640 1 -- 172.21.15.18:0/3697919855 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf44027040 con 0x55deecb81d30 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf4ffff640 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 0x55deecb90f70 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fdf40000c00 tx=0x7fdf40005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defaa559b7144f3a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.169+0000 7fdf2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf2f7fe640 1 -- 172.21.15.18:0/3697919855 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4000ea90 con 0x55deecb81d30 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf2f7fe640 1 -- 172.21.15.18:0/3697919855 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fdf40005140 con 0x55deecb81d30 2024-02-17T13:01:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3697919855 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55deecb0e900 con 0x55deecb81d30 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf2f7fe640 1 -- 172.21.15.18:0/3697919855 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf40005300 con 0x55deecb81d30 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf58fc9740 10 client.19641 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3697919855 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55deecae7320 con 0x55deecb81d30 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf2f7fe640 1 -- 172.21.15.18:0/3697919855 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55deecb0e900 con 0x55deecb81d30 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf2f7fe640 1 -- 172.21.15.18:0/3697919855 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf40022070 con 0x55deecb81d30 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf58fc9740 10 client.19641 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf58fc9740 10 client.19641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf58fc9740 20 client.19641 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55deecbf31c0 0x55deecbf55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf58fc9740 4 client.19641 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3697919855 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55deec7e8860 con 0x55deecbf31c0 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.170+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55deecbf31c0 0x55deecbf55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.171+0000 7fdf4f7fe640 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55deecbf31c0 0x55deecbf55e0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9878c598e008d12b server_cookie=10fe638796776ec0 in_seq=0 out_seq=0 2024-02-17T13:01:11.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.171+0000 7fdf2f7fe640 10 client.19641 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:11.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf2f7fe640 1 -- 172.21.15.18:0/3697919855 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55deec7e8860 con 0x55deecbf31c0 2024-02-17T13:01:11.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf2f7fe640 10 client.19641 handle_command_reply: tid=0 2024-02-17T13:01:11.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf58fc9740 1 client.19641 shutdown 2024-02-17T13:01:11.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3697919855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 msgr2=0x55deecb90f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 0x55deecb90f70 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fdf40000c00 tx=0x7fdf40005eb0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3697919855 shutdown_connections 2024-02-17T13:01:11.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55deecbf31c0 0x55deecbf55e0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deecb8a8b0 0x55deecbf13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf2f7fe640 0 client.19641 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:11.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55deecb81d30 0x55deecb90f70 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf58fc9740 1 --2- 172.21.15.18:0/3697919855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55deecb81380 0x55deecb90a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3697919855 >> 172.21.15.18:0/3697919855 conn(0x55deecb7b840 msgr2=0x55deecb7a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:11.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3697919855 shutdown_connections 2024-02-17T13:01:11.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf58fc9740 1 -- 172.21.15.18:0/3697919855 wait complete. 2024-02-17T13:01:11.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.172+0000 7fdf58fc9740 20 client.19641 trim_cache size 0 max 16384 2024-02-17T13:01:11.170 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:01:11.170 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T13:01:11.170 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "6a32dc09-be83-48e8-a0c9-4bdb5828ae73", 2024-02-17T13:01:11.170 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T13:01:11.170 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:01:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.173+0000 7fdf56313640 1 -- 172.21.15.18:0/1370414115 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdf380a2230 msgr2=0x7fdf380a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.173+0000 7fdf56313640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdf380a2230 0x7fdf380a46f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdf480b9b70 tx=0x7fdf48090780 comp rx=0 tx=0).stop 2024-02-17T13:01:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.173+0000 7fdf56313640 1 -- 172.21.15.18:0/1370414115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf5019bff0 msgr2=0x7fdf5019c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.173+0000 7fdf56313640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf5019bff0 0x7fdf5019c4a0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fdf480348f0 tx=0x7fdf48034cc0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.173+0000 7fdf56313640 1 -- 172.21.15.18:0/1370414115 shutdown_connections 2024-02-17T13:01:11.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.173+0000 7fdf56313640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdf380a2230 0x7fdf380a46f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.173+0000 7fdf56313640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf5019bff0 0x7fdf5019c4a0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.173+0000 7fdf56313640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf50105330 0x7fdf50197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.173+0000 7fdf56313640 1 --2- 172.21.15.18:0/1370414115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf50100260 0x7fdf50197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.173+0000 7fdf56313640 1 -- 172.21.15.18:0/1370414115 >> 172.21.15.18:0/1370414115 conn(0x7fdf50077180 msgr2=0x7fdf500fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:11.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.173+0000 7fdf56313640 1 -- 172.21.15.18:0/1370414115 shutdown_connections 2024-02-17T13:01:11.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.173+0000 7fdf56313640 1 -- 172.21.15.18:0/1370414115 wait complete. 2024-02-17T13:01:11.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:01:11.177 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:01:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.300+0000 7f9448348640 1 -- 172.21.15.18:0/3981739157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9440100270 msgr2=0x7f9440100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.300+0000 7f9448348640 1 --2- 172.21.15.18:0/3981739157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9440100270 0x7f9440100650 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f943c009930 tx=0x7f943c02bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.300+0000 7f9448348640 1 -- 172.21.15.18:0/3981739157 shutdown_connections 2024-02-17T13:01:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.300+0000 7f9448348640 1 --2- 172.21.15.18:0/3981739157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9440105340 0x7f944010cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.300+0000 7f9448348640 1 --2- 172.21.15.18:0/3981739157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9440100c20 0x7f9440104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.300+0000 7f9448348640 1 --2- 172.21.15.18:0/3981739157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9440100270 0x7f9440100650 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.300+0000 7f9448348640 1 -- 172.21.15.18:0/3981739157 >> 172.21.15.18:0/3981739157 conn(0x7f9440077230 msgr2=0x7f9440077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.300+0000 7f9448348640 1 -- 172.21.15.18:0/3981739157 shutdown_connections 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.300+0000 7f9448348640 1 -- 172.21.15.18:0/3981739157 wait complete. 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f9448348640 1 Processor -- start 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f9448348640 1 -- start start 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f9448348640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9440100270 0x7f9440196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f9448348640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9440105340 0x7f9440196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f9448348640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944019af50 0x7f944019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f9448348640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944019b940 con 0x7f944019af50 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f9448348640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944019bab0 con 0x7f9440105340 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f9448348640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944019bc20 con 0x7f9440100270 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f94468e5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944019af50 0x7f944019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f94458e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9440105340 0x7f9440196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f94468e5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944019af50 0x7f944019b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40536/0 (socket says 172.21.15.18:40536) 2024-02-17T13:01:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f94468e5640 1 -- 172.21.15.18:0/3337836256 learned_addr learned my addr 172.21.15.18:0/3337836256 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f94460e4640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9440100270 0x7f9440196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f94468e5640 1 -- 172.21.15.18:0/3337836256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9440100270 msgr2=0x7f9440196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f94468e5640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9440100270 0x7f9440196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f94468e5640 1 -- 172.21.15.18:0/3337836256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9440105340 msgr2=0x7f9440196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f94468e5640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9440105340 0x7f9440196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f94468e5640 1 -- 172.21.15.18:0/3337836256 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c009560 con 0x7f944019af50 2024-02-17T13:01:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.301+0000 7f94468e5640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944019af50 0x7f944019b400 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f943800eb10 tx=0x7f943800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.302+0000 7f94377fe640 1 -- 172.21.15.18:0/3337836256 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438004b20 con 0x7f944019af50 2024-02-17T13:01:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.302+0000 7f9448348640 1 -- 172.21.15.18:0/3337836256 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94401a3200 con 0x7f944019af50 2024-02-17T13:01:11.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.302+0000 7f94377fe640 1 -- 172.21.15.18:0/3337836256 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9438004c80 con 0x7f944019af50 2024-02-17T13:01:11.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.302+0000 7f94377fe640 1 -- 172.21.15.18:0/3337836256 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438010670 con 0x7f944019af50 2024-02-17T13:01:11.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.302+0000 7f9448348640 1 -- 172.21.15.18:0/3337836256 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94401a3700 con 0x7f944019af50 2024-02-17T13:01:11.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.304+0000 7f94377fe640 1 -- 172.21.15.18:0/3337836256 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9438005020 con 0x7f944019af50 2024-02-17T13:01:11.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.304+0000 7f944affb740 1 Processor -- start 2024-02-17T13:01:11.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.304+0000 7f944affb740 1 -- start start 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.304+0000 7f944affb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 0x55fb9890a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f944affb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb98913270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f944affb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb98915ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f944affb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb98879280 con 0x55fb9890a2b0 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f944affb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb988767b0 con 0x55fb9890abd0 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f944affb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb98871f90 con 0x55fb989137b0 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94458e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 0x55fb9890a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94458e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 0x55fb9890a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40548/0 (socket says 172.21.15.18:40548) 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94458e3640 1 -- 172.21.15.18:0/507484285 learned_addr learned my addr 172.21.15.18:0/507484285 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94460e4640 1 --2- 172.21.15.18:0/507484285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb98913270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94458e3640 1 --2- 172.21.15.18:0/507484285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb98915ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94377fe640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f942c0a2230 0x7f942c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94468e5640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f942c0a2230 0x7f942c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94460e4640 1 -- 172.21.15.18:0/507484285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 msgr2=0x55fb98915ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94460e4640 1 --2- 172.21.15.18:0/507484285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb98915ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94460e4640 1 -- 172.21.15.18:0/507484285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 msgr2=0x55fb9890a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94460e4640 1 --2- 172.21.15.18:0/507484285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 0x55fb9890a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94460e4640 1 -- 172.21.15.18:0/507484285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb98830570 con 0x55fb9890abd0 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94458e3640 1 --2- 172.21.15.18:0/507484285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 0x55fb9890a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94377fe640 1 -- 172.21.15.18:0/3337836256 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f9438014070 con 0x7f944019af50 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f94460e4640 1 --2- 172.21.15.18:0/507484285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb98913270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f943c009230 tx=0x7f943c02c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1b5c82203f89704 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.306+0000 7f94468e5640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f942c0a2230 0x7f942c0a46f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f94380973e0 tx=0x7f943801b4f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.305+0000 7f9434bfe640 1 -- 172.21.15.18:0/507484285 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c004880 con 0x55fb9890abd0 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.306+0000 7f9434bfe640 1 -- 172.21.15.18:0/507484285 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f943c0049e0 con 0x55fb9890abd0 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.306+0000 7f9434bfe640 1 -- 172.21.15.18:0/507484285 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c005030 con 0x55fb9890abd0 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.306+0000 7f944affb740 1 -- 172.21.15.18:0/507484285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 msgr2=0x55fb98913270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.306+0000 7f944affb740 1 --2- 172.21.15.18:0/507484285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb98913270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f943c009230 tx=0x7f943c02c680 comp rx=0 tx=0).stop 2024-02-17T13:01:11.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.306+0000 7f944affb740 1 -- 172.21.15.18:0/507484285 shutdown_connections 2024-02-17T13:01:11.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.306+0000 7f944affb740 1 --2- 172.21.15.18:0/507484285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb98915ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.306+0000 7f944affb740 1 --2- 172.21.15.18:0/507484285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb98913270 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.306+0000 7f944affb740 1 --2- 172.21.15.18:0/507484285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 0x55fb9890a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.306+0000 7f944affb740 1 -- 172.21.15.18:0/507484285 >> 172.21.15.18:0/507484285 conn(0x55fb98904990 msgr2=0x55fb98904d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:11.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f944affb740 1 -- 172.21.15.18:0/507484285 shutdown_connections 2024-02-17T13:01:11.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f944affb740 1 -- 172.21.15.18:0/507484285 wait complete. 2024-02-17T13:01:11.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f944affb740 1 -- start start 2024-02-17T13:01:11.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f944affb740 1 Processor -- start 2024-02-17T13:01:11.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f944affb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb9890a2b0 0x55fb98910630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f944affb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb98910b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f944affb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb989137b0 0x55fb989110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f944affb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb98545430 con 0x55fb989137b0 2024-02-17T13:01:11.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f944affb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb98501e00 con 0x55fb9890abd0 2024-02-17T13:01:11.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f944affb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb987f6fd0 con 0x55fb9890a2b0 2024-02-17T13:01:11.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 563211 4.90 MB/sec execute 506 sec latency 200.160 ms 2024-02-17T13:01:11.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f944affb740 10 client.0 _ll_register_callbacks cb 0x55fb9884cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f94458e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb989137b0 0x55fb989110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f94458e3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb9890a2b0 0x55fb98910630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f94458e3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb9890a2b0 0x55fb98910630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:38392/0 (socket says 172.21.15.18:38392) 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f94460e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb98910b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.307+0000 7f94458e3640 1 -- 172.21.15.18:0/2348049153 learned_addr learned my addr 172.21.15.18:0/2348049153 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f94458e3640 1 -- 172.21.15.18:0/2348049153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 msgr2=0x55fb98910b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f94458e3640 1 --2- 172.21.15.18:0/2348049153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb98910b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f94458e3640 1 -- 172.21.15.18:0/2348049153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb989137b0 msgr2=0x55fb989110b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f94458e3640 1 --2- 172.21.15.18:0/2348049153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb989137b0 0x55fb989110b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f94458e3640 1 -- 172.21.15.18:0/2348049153 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944010d720 con 0x55fb9890a2b0 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f94460e4640 1 --2- 172.21.15.18:0/2348049153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb98910b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f94458e3640 1 --2- 172.21.15.18:0/2348049153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb9890a2b0 0x55fb98910630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9430012e20 tx=0x7f943000b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecc0a25b82c785ef server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:11.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f9429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f9429ffb640 1 -- 172.21.15.18:0/2348049153 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430005220 con 0x55fb9890a2b0 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f944affb740 1 -- 172.21.15.18:0/2348049153 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb985717e0 con 0x55fb9890a2b0 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f9429ffb640 1 -- 172.21.15.18:0/2348049153 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f9430010e70 con 0x55fb9890a2b0 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f9429ffb640 1 -- 172.21.15.18:0/2348049153 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9430005380 con 0x55fb9890a2b0 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f944affb740 10 client.28672 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f944affb740 1 -- 172.21.15.18:0/2348049153 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55fb98870610 con 0x55fb9890a2b0 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.308+0000 7f9429ffb640 1 -- 172.21.15.18:0/2348049153 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fb985717e0 con 0x55fb9890a2b0 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.309+0000 7f9429ffb640 1 -- 172.21.15.18:0/2348049153 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f94300243c0 con 0x55fb9890a2b0 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.309+0000 7f944affb740 10 client.28672 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.309+0000 7f944affb740 10 client.28672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.309+0000 7f944affb740 20 client.28672 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.309+0000 7f944affb740 1 --2- 172.21.15.18:0/2348049153 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fb98925ca0 0x55fb989280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.309+0000 7f944affb740 4 client.28672 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:11.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.309+0000 7f944affb740 1 -- 172.21.15.18:0/2348049153 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb985ae720 con 0x55fb98925ca0 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.309+0000 7f94460e4640 1 --2- 172.21.15.18:0/2348049153 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fb98925ca0 0x55fb989280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.309+0000 7f94460e4640 1 --2- 172.21.15.18:0/2348049153 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fb98925ca0 0x55fb989280c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81c9b7da3d6603b0 server_cookie=4590eb90c4705739 in_seq=0 out_seq=0 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.309+0000 7f9429ffb640 10 client.28672 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f9429ffb640 1 -- 172.21.15.18:0/2348049153 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb985ae720 con 0x55fb98925ca0 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f9429ffb640 10 client.28672 handle_command_reply: tid=0 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f944affb740 1 client.28672 shutdown 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f944affb740 1 -- 172.21.15.18:0/2348049153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb9890a2b0 msgr2=0x55fb98910630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f944affb740 1 --2- 172.21.15.18:0/2348049153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb9890a2b0 0x55fb98910630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9430012e20 tx=0x7f943000b390 comp rx=0 tx=0).stop 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f944affb740 1 -- 172.21.15.18:0/2348049153 shutdown_connections 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f944affb740 1 --2- 172.21.15.18:0/2348049153 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fb98925ca0 0x55fb989280c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f944affb740 1 --2- 172.21.15.18:0/2348049153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb989137b0 0x55fb989110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f944affb740 1 --2- 172.21.15.18:0/2348049153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb98910b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f944affb740 1 --2- 172.21.15.18:0/2348049153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb9890a2b0 0x55fb98910630 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f9429ffb640 0 client.28672 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f944affb740 1 -- 172.21.15.18:0/2348049153 >> 172.21.15.18:0/2348049153 conn(0x55fb98904990 msgr2=0x55fb98906730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:11.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f944affb740 1 -- 172.21.15.18:0/2348049153 shutdown_connections 2024-02-17T13:01:11.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.310+0000 7f944affb740 1 -- 172.21.15.18:0/2348049153 wait complete. 2024-02-17T13:01:11.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.311+0000 7f944affb740 20 client.28672 trim_cache size 0 max 16384 2024-02-17T13:01:11.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f944affb740 1 Processor -- start 2024-02-17T13:01:11.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f944affb740 1 -- start start 2024-02-17T13:01:11.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f944affb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 0x55fb98912d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f944affb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb9890f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f944affb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb9890d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f944affb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb98909f10 con 0x55fb9890a2b0 2024-02-17T13:01:11.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f944affb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb989059e0 con 0x55fb9890abd0 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f944affb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb988fee20 con 0x55fb989137b0 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f94458e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 0x55fb98912d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f94460e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb9890f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f94458e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 0x55fb98912d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40578/0 (socket says 172.21.15.18:40578) 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f94458e3640 1 -- 172.21.15.18:0/962631105 learned_addr learned my addr 172.21.15.18:0/962631105 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f94458e3640 1 --2- 172.21.15.18:0/962631105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb9890d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f94460e4640 1 -- 172.21.15.18:0/962631105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 msgr2=0x55fb9890d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f94460e4640 1 --2- 172.21.15.18:0/962631105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb9890d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f94460e4640 1 -- 172.21.15.18:0/962631105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 msgr2=0x55fb98912d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f94460e4640 1 --2- 172.21.15.18:0/962631105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 0x55fb98912d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f94460e4640 1 -- 172.21.15.18:0/962631105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb9880cac0 con 0x55fb9890abd0 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.327+0000 7f94458e3640 1 --2- 172.21.15.18:0/962631105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb9890d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:01:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.328+0000 7f94460e4640 1 --2- 172.21.15.18:0/962631105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb9890f610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f943c005c40 tx=0x7f943c041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a419b5f4979351a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.328+0000 7f942bfff640 1 -- 172.21.15.18:0/962631105 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c054070 con 0x55fb9890abd0 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.328+0000 7f942bfff640 1 -- 172.21.15.18:0/962631105 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f943c004740 con 0x55fb9890abd0 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.328+0000 7f942bfff640 1 -- 172.21.15.18:0/962631105 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c050d60 con 0x55fb9890abd0 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.328+0000 7f944affb740 1 -- 172.21.15.18:0/962631105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 msgr2=0x55fb9890f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.328+0000 7f944affb740 1 --2- 172.21.15.18:0/962631105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb9890f610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f943c005c40 tx=0x7f943c041040 comp rx=0 tx=0).stop 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.328+0000 7f944affb740 1 -- 172.21.15.18:0/962631105 shutdown_connections 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.328+0000 7f944affb740 1 --2- 172.21.15.18:0/962631105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb9890d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.328+0000 7f944affb740 1 --2- 172.21.15.18:0/962631105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890abd0 0x55fb9890f610 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.328+0000 7f944affb740 1 --2- 172.21.15.18:0/962631105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890a2b0 0x55fb98912d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.328+0000 7f944affb740 1 -- 172.21.15.18:0/962631105 >> 172.21.15.18:0/962631105 conn(0x55fb98904990 msgr2=0x55fb9890ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f944affb740 1 -- 172.21.15.18:0/962631105 shutdown_connections 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f944affb740 1 -- 172.21.15.18:0/962631105 wait complete. 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f944affb740 1 -- start start 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f944affb740 1 Processor -- start 2024-02-17T13:01:11.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f944affb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890a2b0 0x55fb98918fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f944affb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890abd0 0x55fb989194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f944affb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb98919a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f944affb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb9891aef0 con 0x55fb9890abd0 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f944affb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb9897c1b0 con 0x55fb9890a2b0 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f944affb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb9897c2f0 con 0x55fb989137b0 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f944affb740 10 client.0 _ll_register_callbacks cb 0x7f943c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f94460e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890abd0 0x55fb989194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f94460e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890abd0 0x55fb989194e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40594/0 (socket says 172.21.15.18:40594) 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f94460e4640 1 -- 172.21.15.18:0/3682620778 learned_addr learned my addr 172.21.15.18:0/3682620778 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f94458e3640 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890a2b0 0x55fb98918fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f94458e3640 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb98919a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f94460e4640 1 -- 172.21.15.18:0/3682620778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 msgr2=0x55fb98919a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f94460e4640 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb98919a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f94460e4640 1 -- 172.21.15.18:0/3682620778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890a2b0 msgr2=0x55fb98918fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f94460e4640 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890a2b0 0x55fb98918fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f94460e4640 1 -- 172.21.15.18:0/3682620778 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb98830570 con 0x55fb9890abd0 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.329+0000 7f94458e3640 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890a2b0 0x55fb98918fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f94460e4640 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890abd0 0x55fb989194e0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f943c02c170 tx=0x7f943c004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf11c043708782e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f9429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f9429ffb640 1 -- 172.21.15.18:0/3682620778 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c054070 con 0x55fb9890abd0 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f9429ffb640 1 -- 172.21.15.18:0/3682620778 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f943c050330 con 0x55fb9890abd0 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f944affb740 1 -- 172.21.15.18:0/3682620778 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f943c0049e0 con 0x55fb9890abd0 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f9429ffb640 1 -- 172.21.15.18:0/3682620778 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f943c0504f0 con 0x55fb9890abd0 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f944affb740 10 client.19659 fetch_fsmap learned FSMap version 49 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f944affb740 1 -- 172.21.15.18:0/3682620778 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55fb988326a0 con 0x55fb9890abd0 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f9429ffb640 1 -- 172.21.15.18:0/3682620778 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f943c066020 con 0x55fb9890abd0 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f9429ffb640 1 -- 172.21.15.18:0/3682620778 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 49) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f943c03c050 con 0x55fb9890abd0 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f944affb740 10 client.19659 fetch_fsmap finished waiting for FSMap version 49 2024-02-17T13:01:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f944affb740 10 client.19659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f944affb740 20 client.19659 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f944affb740 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fb9897d7f0 0x55fb9897fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f944affb740 4 client.19659 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:01:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.330+0000 7f944affb740 1 -- 172.21.15.18:0/3682620778 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fb985fd890 con 0x55fb9897d7f0 2024-02-17T13:01:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f94458e3640 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fb9897d7f0 0x55fb9897fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f94458e3640 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fb9897d7f0 0x55fb9897fc10 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee246fe37bd2266 server_cookie=e06820f726fba4b8 in_seq=0 out_seq=0 2024-02-17T13:01:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f9429ffb640 10 client.19659 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:11.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f9429ffb640 1 -- 172.21.15.18:0/3682620778 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fb985fd890 con 0x55fb9897d7f0 2024-02-17T13:01:11.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f9429ffb640 10 client.19659 handle_command_reply: tid=0 2024-02-17T13:01:11.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f944affb740 1 client.19659 shutdown 2024-02-17T13:01:11.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f944affb740 1 -- 172.21.15.18:0/3682620778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890abd0 msgr2=0x55fb989194e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f944affb740 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890abd0 0x55fb989194e0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f943c02c170 tx=0x7f943c004520 comp rx=0 tx=0).stop 2024-02-17T13:01:11.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f944affb740 1 -- 172.21.15.18:0/3682620778 shutdown_connections 2024-02-17T13:01:11.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f944affb740 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fb9897d7f0 0x55fb9897fc10 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f944affb740 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb989137b0 0x55fb98919a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f944affb740 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb9890abd0 0x55fb989194e0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f9429ffb640 0 client.19659 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:11.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f944affb740 1 --2- 172.21.15.18:0/3682620778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fb9890a2b0 0x55fb98918fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f944affb740 1 -- 172.21.15.18:0/3682620778 >> 172.21.15.18:0/3682620778 conn(0x55fb98904990 msgr2=0x55fb9890f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:11.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.331+0000 7f944affb740 1 -- 172.21.15.18:0/3682620778 shutdown_connections 2024-02-17T13:01:11.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f944affb740 1 -- 172.21.15.18:0/3682620778 wait complete. 2024-02-17T13:01:11.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f944affb740 20 client.19659 trim_cache size 0 max 16384 2024-02-17T13:01:11.330 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:01:11.330 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T13:01:11.331 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T13:01:11.331 INFO:teuthology.orchestra.run.smithi018.stdout: "6a32dc09-be83-48e8-a0c9-4bdb5828ae73": { 2024-02-17T13:01:11.331 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T13:01:11.331 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "6a32dc09-be83-48e8-a0c9-4bdb5828ae73", 2024-02-17T13:01:11.331 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T13:01:11.331 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:01:11.331 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:01:11.331 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:01:11.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f9448348640 1 -- 172.21.15.18:0/3337836256 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f942c0a2230 msgr2=0x7f942c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f9448348640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f942c0a2230 0x7f942c0a46f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f94380973e0 tx=0x7f943801b4f0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f9448348640 1 -- 172.21.15.18:0/3337836256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944019af50 msgr2=0x7f944019b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:11.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f9448348640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944019af50 0x7f944019b400 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f943800eb10 tx=0x7f943800c2d0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f9448348640 1 -- 172.21.15.18:0/3337836256 shutdown_connections 2024-02-17T13:01:11.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f9448348640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f942c0a2230 0x7f942c0a46f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f9448348640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944019af50 0x7f944019b400 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f9448348640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9440105340 0x7f9440196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f9448348640 1 --2- 172.21.15.18:0/3337836256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9440100270 0x7f9440196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:11.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f9448348640 1 -- 172.21.15.18:0/3337836256 >> 172.21.15.18:0/3337836256 conn(0x7f9440077230 msgr2=0x7f94400fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:11.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f9448348640 1 -- 172.21.15.18:0/3337836256 shutdown_connections 2024-02-17T13:01:11.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:11.332+0000 7f9448348640 1 -- 172.21.15.18:0/3337836256 wait complete. 2024-02-17T13:01:11.334 INFO:tasks.cephfs.filesystem:scrub status for tag:6a32dc09-be83-48e8-a0c9-4bdb5828ae73 - {'path': '/', 'tag': '6a32dc09-be83-48e8-a0c9-4bdb5828ae73', 'options': 'recursive,force'} 2024-02-17T13:01:12.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 565190 4.92 MB/sec execute 507 sec latency 13.237 ms 2024-02-17T13:01:12.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: pgmap v354: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-02-17T13:01:12.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T13:01:12.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:12.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:12.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:12.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:12.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.450 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:12 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: pgmap v354: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-02-17T13:01:12.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T13:01:12.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:12.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:12.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:12.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:12.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:12 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: pgmap v354: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-02-17T13:01:12.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T13:01:12.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:12.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:12.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:12.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:12.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:12.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:12 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:13.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 566486 4.92 MB/sec execute 508 sec latency 231.537 ms 2024-02-17T13:01:14.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 567353 4.92 MB/sec execute 509 sec latency 838.370 ms 2024-02-17T13:01:14.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:14 smithi146 ceph-mon[36454]: pgmap v355: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-02-17T13:01:14.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:14 smithi146 ceph-mon[36454]: osdmap e102: 12 total, 12 up, 12 in 2024-02-17T13:01:14.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:14 smithi018 ceph-mon[32625]: pgmap v355: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-02-17T13:01:14.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:14 smithi018 ceph-mon[32625]: osdmap e102: 12 total, 12 up, 12 in 2024-02-17T13:01:14.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:14 smithi119 ceph-mon[36188]: pgmap v355: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-02-17T13:01:14.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:14 smithi119 ceph-mon[36188]: osdmap e102: 12 total, 12 up, 12 in 2024-02-17T13:01:15.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 569011 4.93 MB/sec execute 510 sec latency 20.570 ms 2024-02-17T13:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.120+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/3154956965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d4105020 msgr2=0x7fd8d4105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.120+0000 7fd8d8f4b640 1 --2- 172.21.15.18:0/3154956965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d4105020 0x7fd8d4105400 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009890 tx=0x7fd8c402bb70 comp rx=0 tx=0).stop 2024-02-17T13:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.120+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/3154956965 shutdown_connections 2024-02-17T13:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.121+0000 7fd8d8f4b640 1 --2- 172.21.15.18:0/3154956965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d4100c90 0x7fd8d410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.121+0000 7fd8d8f4b640 1 --2- 172.21.15.18:0/3154956965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d41002d0 0x7fd8d4100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.121+0000 7fd8d8f4b640 1 --2- 172.21.15.18:0/3154956965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d4105020 0x7fd8d4105400 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.121+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/3154956965 >> 172.21.15.18:0/3154956965 conn(0x7fd8d4076ed0 msgr2=0x7fd8d40772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.121+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/3154956965 shutdown_connections 2024-02-17T13:01:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.121+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/3154956965 wait complete. 2024-02-17T13:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.121+0000 7fd8d8f4b640 1 Processor -- start 2024-02-17T13:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.121+0000 7fd8d8f4b640 1 -- start start 2024-02-17T13:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d8f4b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d41002d0 0x7fd8d410d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d8f4b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d4100c90 0x7fd8d410de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d8f4b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d4105020 0x7fd8d410e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d8f4b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d41a5b70 con 0x7fd8d41002d0 2024-02-17T13:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d8f4b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d41a5ce0 con 0x7fd8d4105020 2024-02-17T13:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d8f4b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d41a5ea0 con 0x7fd8d4100c90 2024-02-17T13:01:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d41002d0 0x7fd8d410d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d1d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d4100c90 0x7fd8d410de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d41002d0 0x7fd8d410d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40620/0 (socket says 172.21.15.18:40620) 2024-02-17T13:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d1d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d4100c90 0x7fd8d410de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:38434/0 (socket says 172.21.15.18:38434) 2024-02-17T13:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d259c640 1 -- 172.21.15.18:0/2207329473 learned_addr learned my addr 172.21.15.18:0/2207329473 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d4105020 0x7fd8d410e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d259c640 1 -- 172.21.15.18:0/2207329473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d4100c90 msgr2=0x7fd8d410de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d259c640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d4100c90 0x7fd8d410de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d259c640 1 -- 172.21.15.18:0/2207329473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d4105020 msgr2=0x7fd8d410e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d259c640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d4105020 0x7fd8d410e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d259c640 1 -- 172.21.15.18:0/2207329473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c4009490 con 0x7fd8d41002d0 2024-02-17T13:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d4105020 0x7fd8d410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d1d9b640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d4100c90 0x7fd8d410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8d259c640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d41002d0 0x7fd8d410d930 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009860 tx=0x7fd8c4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.122+0000 7fd8c37fe640 1 -- 172.21.15.18:0/2207329473 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4034950 con 0x7fd8d41002d0 2024-02-17T13:01:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.123+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/2207329473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d41a6120 con 0x7fd8d41002d0 2024-02-17T13:01:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.123+0000 7fd8c37fe640 1 -- 172.21.15.18:0/2207329473 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8c402c9f0 con 0x7fd8d41002d0 2024-02-17T13:01:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.123+0000 7fd8c37fe640 1 -- 172.21.15.18:0/2207329473 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4032600 con 0x7fd8d41002d0 2024-02-17T13:01:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.123+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/2207329473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d41a64e0 con 0x7fd8d41002d0 2024-02-17T13:01:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.125+0000 7fd8c37fe640 1 -- 172.21.15.18:0/2207329473 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd8c403b040 con 0x7fd8d41002d0 2024-02-17T13:01:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.125+0000 7fd8dbbf5740 1 Processor -- start 2024-02-17T13:01:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8dbbf5740 1 -- start start 2024-02-17T13:01:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8dbbf5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 0x5563bd5b9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8dbbf5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5b9d60 0x5563bd5ba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8dbbf5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5c32e0 0x5563bd5c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8dbbf5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bd5289f0 con 0x5563bd5baad0 2024-02-17T13:01:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8dbbf5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bd525f20 con 0x5563bd5b9d60 2024-02-17T13:01:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8dbbf5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bd521700 con 0x5563bd5c32e0 2024-02-17T13:01:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8d2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 0x5563bd5b9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8c37fe640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8bc0a22d0 0x7fd8bc0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8d2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 0x5563bd5b9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40626/0 (socket says 172.21.15.18:40626) 2024-02-17T13:01:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8d2d9d640 1 -- 172.21.15.18:0/852466039 learned_addr learned my addr 172.21.15.18:0/852466039 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8d1d9b640 1 --2- 172.21.15.18:0/852466039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5c32e0 0x5563bd5c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8bc0a22d0 0x7fd8bc0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8c37fe640 1 -- 172.21.15.18:0/2207329473 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd8c40efa70 con 0x7fd8d41002d0 2024-02-17T13:01:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8d259c640 1 --2- 172.21.15.18:0/852466039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5b9d60 0x5563bd5ba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8d2d9d640 1 -- 172.21.15.18:0/852466039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5c32e0 msgr2=0x5563bd5c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/852466039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5c32e0 0x5563bd5c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8d2d9d640 1 -- 172.21.15.18:0/852466039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5b9d60 msgr2=0x5563bd5ba1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/852466039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5b9d60 0x5563bd5ba1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8d2d9d640 1 -- 172.21.15.18:0/852466039 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563bd4b77b0 con 0x5563bd5baad0 2024-02-17T13:01:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.126+0000 7fd8d1d9b640 1 --2- 172.21.15.18:0/852466039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5c32e0 0x5563bd5c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.127+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/852466039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 0x5563bd5b9820 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00ed80 tx=0x7fd8cc00d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c01cab20362345 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.127+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/852466039 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc025450 con 0x5563bd5baad0 2024-02-17T13:01:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.127+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8bc0a22d0 0x7fd8bc0a4790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc022000 tx=0x7fd8cc01e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.127+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/852466039 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8cc025a90 con 0x5563bd5baad0 2024-02-17T13:01:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.127+0000 7fd8c0ff9640 1 -- 172.21.15.18:0/852466039 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc01b9d0 con 0x5563bd5baad0 2024-02-17T13:01:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/852466039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 msgr2=0x5563bd5b9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/852466039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 0x5563bd5b9820 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00ed80 tx=0x7fd8cc00d7b0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/852466039 shutdown_connections 2024-02-17T13:01:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/852466039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5c32e0 0x5563bd5c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/852466039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5b9d60 0x5563bd5ba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/852466039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 0x5563bd5b9820 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/852466039 >> 172.21.15.18:0/852466039 conn(0x5563bd581550 msgr2=0x5563bd5b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/852466039 shutdown_connections 2024-02-17T13:01:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/852466039 wait complete. 2024-02-17T13:01:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 -- start start 2024-02-17T13:01:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 Processor -- start 2024-02-17T13:01:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5b9d60 0x5563bd5c0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5baad0 0x5563bd5c0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 0x5563bd5c0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bd4c6bf0 con 0x5563bd5c32e0 2024-02-17T13:01:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bd1f5250 con 0x5563bd5b9d60 2024-02-17T13:01:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bd4a7200 con 0x5563bd5baad0 2024-02-17T13:01:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.128+0000 7fd8dbbf5740 10 client.0 _ll_register_callbacks cb 0x5563bd515d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8d2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 0x5563bd5c0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8d259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5b9d60 0x5563bd5c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8d259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5b9d60 0x5563bd5c0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:54864/0 (socket says 172.21.15.18:54864) 2024-02-17T13:01:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8d259c640 1 -- 172.21.15.18:0/2750694960 learned_addr learned my addr 172.21.15.18:0/2750694960 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8d1d9b640 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5baad0 0x5563bd5c0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8d2d9d640 1 -- 172.21.15.18:0/2750694960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5baad0 msgr2=0x5563bd5c0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5baad0 0x5563bd5c0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8d2d9d640 1 -- 172.21.15.18:0/2750694960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5b9d60 msgr2=0x5563bd5c0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5b9d60 0x5563bd5c0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8d2d9d640 1 -- 172.21.15.18:0/2750694960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8cc016040 con 0x5563bd5c32e0 2024-02-17T13:01:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8d259c640 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5b9d60 0x5563bd5c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 0x5563bd5c0dc0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009250 tx=0x7fd8cc005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c3680792bb5573 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8b9ffb640 1 -- 172.21.15.18:0/2750694960 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc01bdc0 con 0x5563bd5c32e0 2024-02-17T13:01:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8b9ffb640 1 -- 172.21.15.18:0/2750694960 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd8cc02b040 con 0x5563bd5c32e0 2024-02-17T13:01:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2750694960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563bd221600 con 0x5563bd5c32e0 2024-02-17T13:01:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8b9ffb640 1 -- 172.21.15.18:0/2750694960 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8cc01b370 con 0x5563bd5c32e0 2024-02-17T13:01:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8dbbf5740 10 client.19671 fetch_fsmap learned FSMap version 50 2024-02-17T13:01:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.129+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2750694960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5563bd51fd80 con 0x5563bd5c32e0 2024-02-17T13:01:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.130+0000 7fd8b9ffb640 1 -- 172.21.15.18:0/2750694960 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5563bd221600 con 0x5563bd5c32e0 2024-02-17T13:01:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.130+0000 7fd8b9ffb640 1 -- 172.21.15.18:0/2750694960 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd8cc02fba0 con 0x5563bd5c32e0 2024-02-17T13:01:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.130+0000 7fd8dbbf5740 10 client.19671 fetch_fsmap finished waiting for FSMap version 50 2024-02-17T13:01:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.130+0000 7fd8dbbf5740 10 client.19671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.130+0000 7fd8dbbf5740 20 client.19671 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.130+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5563bd5d35e0 0x5563bd5d5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.130+0000 7fd8dbbf5740 4 client.19671 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.130+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2750694960 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563bd2721b0 con 0x5563bd5d35e0 2024-02-17T13:01:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.130+0000 7fd8d259c640 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5563bd5d35e0 0x5563bd5d5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8b9ffb640 10 client.19671 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8d259c640 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5563bd5d35e0 0x5563bd5d5a00 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed2e81775bdddcf server_cookie=5c7d38abd033cb30 in_seq=0 out_seq=0 2024-02-17T13:01:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8b9ffb640 1 -- 172.21.15.18:0/2750694960 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5563bd2721b0 con 0x5563bd5d35e0 2024-02-17T13:01:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8b9ffb640 10 client.19671 handle_command_reply: tid=0 2024-02-17T13:01:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8dbbf5740 1 client.19671 shutdown 2024-02-17T13:01:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2750694960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 msgr2=0x5563bd5c0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 0x5563bd5c0dc0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009250 tx=0x7fd8cc005a50 comp rx=0 tx=0).stop 2024-02-17T13:01:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2750694960 shutdown_connections 2024-02-17T13:01:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5563bd5d35e0 0x5563bd5d5a00 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 0x5563bd5c0dc0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5baad0 0x5563bd5c0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8b9ffb640 0 client.19671 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/2750694960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5b9d60 0x5563bd5c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2750694960 >> 172.21.15.18:0/2750694960 conn(0x5563bd581550 msgr2=0x5563bd5b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2750694960 shutdown_connections 2024-02-17T13:01:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2750694960 wait complete. 2024-02-17T13:01:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.131+0000 7fd8dbbf5740 20 client.19671 trim_cache size 0 max 16384 2024-02-17T13:01:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.144+0000 7fd8dbbf5740 1 Processor -- start 2024-02-17T13:01:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.144+0000 7fd8dbbf5740 1 -- start start 2024-02-17T13:01:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8dbbf5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5b9d60 0x5563bd5c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8dbbf5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5baad0 0x5563bd5c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8dbbf5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 0x5563bd5bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8dbbf5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bd5b5a80 con 0x5563bd5c32e0 2024-02-17T13:01:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8dbbf5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bd5b9060 con 0x5563bd5baad0 2024-02-17T13:01:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8dbbf5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bd5ba450 con 0x5563bd5b9d60 2024-02-17T13:01:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8d2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 0x5563bd5bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8d2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 0x5563bd5bd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40642/0 (socket says 172.21.15.18:40642) 2024-02-17T13:01:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8d2d9d640 1 -- 172.21.15.18:0/3889670051 learned_addr learned my addr 172.21.15.18:0/3889670051 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8d1d9b640 1 --2- 172.21.15.18:0/3889670051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5baad0 0x5563bd5c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8d259c640 1 --2- 172.21.15.18:0/3889670051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5b9d60 0x5563bd5c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8d2d9d640 1 -- 172.21.15.18:0/3889670051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5b9d60 msgr2=0x5563bd5c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/3889670051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5b9d60 0x5563bd5c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8d2d9d640 1 -- 172.21.15.18:0/3889670051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5baad0 msgr2=0x5563bd5c0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/3889670051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5baad0 0x5563bd5c0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8d2d9d640 1 -- 172.21.15.18:0/3889670051 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563bd4bc7b0 con 0x5563bd5c32e0 2024-02-17T13:01:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8d259c640 1 --2- 172.21.15.18:0/3889670051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5b9d60 0x5563bd5c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:01:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/3889670051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 0x5563bd5bd5a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00e6f0 tx=0x7fd8cc00d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec952dbd398ab471 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8bbfff640 1 -- 172.21.15.18:0/3889670051 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc01b570 con 0x5563bd5c32e0 2024-02-17T13:01:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8bbfff640 1 -- 172.21.15.18:0/3889670051 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8cc025660 con 0x5563bd5c32e0 2024-02-17T13:01:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.145+0000 7fd8bbfff640 1 -- 172.21.15.18:0/3889670051 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc02fb90 con 0x5563bd5c32e0 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.146+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/3889670051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 msgr2=0x5563bd5bd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.146+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/3889670051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 0x5563bd5bd5a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00e6f0 tx=0x7fd8cc00d7b0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.146+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/3889670051 shutdown_connections 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.146+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/3889670051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5c32e0 0x5563bd5bd5a0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.146+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/3889670051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5baad0 0x5563bd5c0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.146+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/3889670051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5b9d60 0x5563bd5c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.146+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/3889670051 >> 172.21.15.18:0/3889670051 conn(0x5563bd581550 msgr2=0x5563bd5bec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.146+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/3889670051 shutdown_connections 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.146+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/3889670051 wait complete. 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.146+0000 7fd8dbbf5740 1 -- start start 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.146+0000 7fd8dbbf5740 1 Processor -- start 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8dbbf5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5b9d60 0x5563bd5c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8dbbf5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 0x5563bd5c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8dbbf5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5c32e0 0x5563bd628aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8dbbf5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bd5c8ef0 con 0x5563bd5baad0 2024-02-17T13:01:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8dbbf5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bd628fe0 con 0x5563bd5c32e0 2024-02-17T13:01:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8dbbf5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bd629170 con 0x5563bd5b9d60 2024-02-17T13:01:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8dbbf5740 10 client.0 _ll_register_callbacks cb 0x7fd8cc00e450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8d1d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 0x5563bd5c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8d1d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 0x5563bd5c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40656/0 (socket says 172.21.15.18:40656) 2024-02-17T13:01:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8d1d9b640 1 -- 172.21.15.18:0/2208775344 learned_addr learned my addr 172.21.15.18:0/2208775344 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8d259c640 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5b9d60 0x5563bd5c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8d2d9d640 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5c32e0 0x5563bd628aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8d1d9b640 1 -- 172.21.15.18:0/2208775344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5b9d60 msgr2=0x5563bd5c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8d1d9b640 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5b9d60 0x5563bd5c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8d1d9b640 1 -- 172.21.15.18:0/2208775344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5c32e0 msgr2=0x5563bd628aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8d1d9b640 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5c32e0 0x5563bd628aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8d1d9b640 1 -- 172.21.15.18:0/2208775344 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563bd51fd80 con 0x5563bd5baad0 2024-02-17T13:01:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8d259c640 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5b9d60 0x5563bd5c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:01:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8d1d9b640 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 0x5563bd5c87b0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x5563bd5ba800 tx=0x7fd8c800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e41dd9fb378512 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8b9ffb640 1 -- 172.21.15.18:0/2208775344 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8017040 con 0x5563bd5baad0 2024-02-17T13:01:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8b9ffb640 1 -- 172.21.15.18:0/2208775344 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd8c8005220 con 0x5563bd5baad0 2024-02-17T13:01:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2208775344 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8cc025a90 con 0x5563bd5baad0 2024-02-17T13:01:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8b9ffb640 1 -- 172.21.15.18:0/2208775344 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8c80122e0 con 0x5563bd5baad0 2024-02-17T13:01:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8dbbf5740 10 client.19677 fetch_fsmap learned FSMap version 50 2024-02-17T13:01:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.147+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2208775344 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5563bd4e2540 con 0x5563bd5baad0 2024-02-17T13:01:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.148+0000 7fd8b9ffb640 1 -- 172.21.15.18:0/2208775344 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8c802b020 con 0x5563bd5baad0 2024-02-17T13:01:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.148+0000 7fd8b9ffb640 1 -- 172.21.15.18:0/2208775344 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd8c8025070 con 0x5563bd5baad0 2024-02-17T13:01:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.148+0000 7fd8dbbf5740 10 client.19677 fetch_fsmap finished waiting for FSMap version 50 2024-02-17T13:01:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.148+0000 7fd8dbbf5740 10 client.19677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.148+0000 7fd8dbbf5740 20 client.19677 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.148+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5563bd5d35e0 0x5563bd62ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.148+0000 7fd8dbbf5740 4 client.19677 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:01:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.148+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2208775344 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5563bd25e550 con 0x5563bd5d35e0 2024-02-17T13:01:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.148+0000 7fd8d259c640 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5563bd5d35e0 0x5563bd62ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.148+0000 7fd8b9ffb640 10 client.19677 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.148+0000 7fd8d259c640 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5563bd5d35e0 0x5563bd62ca90 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47ac4f9de4c591e server_cookie=72d8398fd7f711ac in_seq=0 out_seq=0 2024-02-17T13:01:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.184+0000 7fd8b9ffb640 1 -- 172.21.15.18:0/2208775344 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5563bd25e550 con 0x5563bd5d35e0 2024-02-17T13:01:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.184+0000 7fd8b9ffb640 10 client.19677 handle_command_reply: tid=0 2024-02-17T13:01:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.184+0000 7fd8dbbf5740 1 client.19677 shutdown 2024-02-17T13:01:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.184+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2208775344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 msgr2=0x5563bd5c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.184+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 0x5563bd5c87b0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x5563bd5ba800 tx=0x7fd8c800f790 comp rx=0 tx=0).stop 2024-02-17T13:01:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2208775344 shutdown_connections 2024-02-17T13:01:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5563bd5d35e0 0x5563bd62ca90 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bd5c32e0 0x5563bd628aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5563bd5baad0 0x5563bd5c87b0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8dbbf5740 1 --2- 172.21.15.18:0/2208775344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5563bd5b9d60 0x5563bd5c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8b9ffb640 0 client.19677 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2208775344 >> 172.21.15.18:0/2208775344 conn(0x5563bd581550 msgr2=0x5563bd5b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2208775344 shutdown_connections 2024-02-17T13:01:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8dbbf5740 1 -- 172.21.15.18:0/2208775344 wait complete. 2024-02-17T13:01:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8dbbf5740 20 client.19677 trim_cache size 0 max 16384 2024-02-17T13:01:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:01:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:01:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:01:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:01:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/2207329473 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8bc0a22d0 msgr2=0x7fd8bc0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8d8f4b640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8bc0a22d0 0x7fd8bc0a4790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc022000 tx=0x7fd8cc01e000 comp rx=0 tx=0).stop 2024-02-17T13:01:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/2207329473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d41002d0 msgr2=0x7fd8d410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.185+0000 7fd8d8f4b640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d41002d0 0x7fd8d410d930 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009860 tx=0x7fd8c4004530 comp rx=0 tx=0).stop 2024-02-17T13:01:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.186+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/2207329473 shutdown_connections 2024-02-17T13:01:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.186+0000 7fd8d8f4b640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd8bc0a22d0 0x7fd8bc0a4790 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.186+0000 7fd8d8f4b640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8d4105020 0x7fd8d410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.186+0000 7fd8d8f4b640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8d4100c90 0x7fd8d410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.186+0000 7fd8d8f4b640 1 --2- 172.21.15.18:0/2207329473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8d41002d0 0x7fd8d410d930 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.186+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/2207329473 >> 172.21.15.18:0/2207329473 conn(0x7fd8d4076ed0 msgr2=0x7fd8d40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.186+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/2207329473 shutdown_connections 2024-02-17T13:01:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:16.186+0000 7fd8d8f4b640 1 -- 172.21.15.18:0/2207329473 wait complete. 2024-02-17T13:01:16.298 INFO:tasks.workunit.client.0.smithi018.stdout: 1 570036 4.93 MB/sec execute 511 sec latency 438.529 ms 2024-02-17T13:01:16.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:16 smithi146 ceph-mon[36454]: pgmap v357: 97 pgs: 97 active+clean; 226 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 256 op/s 2024-02-17T13:01:16.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:16 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:01:16.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:16 smithi018 ceph-mon[32625]: pgmap v357: 97 pgs: 97 active+clean; 226 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 256 op/s 2024-02-17T13:01:16.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:16 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:01:16.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:16 smithi119 ceph-mon[36188]: pgmap v357: 97 pgs: 97 active+clean; 226 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 256 op/s 2024-02-17T13:01:16.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:16 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:01:17.299 INFO:tasks.workunit.client.0.smithi018.stdout: 1 570036 4.92 MB/sec execute 512 sec latency 1438.577 ms 2024-02-17T13:01:18.299 INFO:tasks.workunit.client.0.smithi018.stdout: 1 571116 4.92 MB/sec execute 513 sec latency 1993.565 ms 2024-02-17T13:01:18.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:18 smithi146 ceph-mon[36454]: pgmap v358: 97 pgs: 97 active+clean; 226 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-02-17T13:01:18.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:18 smithi018 ceph-mon[32625]: pgmap v358: 97 pgs: 97 active+clean; 226 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-02-17T13:01:18.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:18 smithi119 ceph-mon[36188]: pgmap v358: 97 pgs: 97 active+clean; 226 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-02-17T13:01:19.299 INFO:tasks.workunit.client.0.smithi018.stdout: 1 572678 4.93 MB/sec execute 514 sec latency 17.454 ms 2024-02-17T13:01:20.299 INFO:tasks.workunit.client.0.smithi018.stdout: 1 573579 4.93 MB/sec execute 515 sec latency 514.101 ms 2024-02-17T13:01:20.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:20 smithi146 ceph-mon[36454]: pgmap v359: 97 pgs: 97 active+clean; 223 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-02-17T13:01:20.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:20 smithi018 ceph-mon[32625]: pgmap v359: 97 pgs: 97 active+clean; 223 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-02-17T13:01:20.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:20 smithi119 ceph-mon[36188]: pgmap v359: 97 pgs: 97 active+clean; 223 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-02-17T13:01:21.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 573579 4.92 MB/sec execute 516 sec latency 1514.229 ms 2024-02-17T13:01:22.299 INFO:tasks.workunit.client.0.smithi018.stdout: 1 573579 4.91 MB/sec execute 517 sec latency 2514.315 ms 2024-02-17T13:01:22.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:22 smithi146 ceph-mon[36454]: pgmap v360: 97 pgs: 97 active+clean; 223 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-02-17T13:01:22.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:22 smithi018 ceph-mon[32625]: pgmap v360: 97 pgs: 97 active+clean; 223 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-02-17T13:01:22.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:22 smithi119 ceph-mon[36188]: pgmap v360: 97 pgs: 97 active+clean; 223 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-02-17T13:01:23.299 INFO:tasks.workunit.client.0.smithi018.stdout: 1 573579 4.90 MB/sec execute 518 sec latency 3514.428 ms 2024-02-17T13:01:24.299 INFO:tasks.workunit.client.0.smithi018.stdout: 1 575510 4.91 MB/sec execute 519 sec latency 3699.115 ms 2024-02-17T13:01:24.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:24 smithi146 ceph-mon[36454]: pgmap v361: 97 pgs: 97 active+clean; 223 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-02-17T13:01:24.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:24 smithi018 ceph-mon[32625]: pgmap v361: 97 pgs: 97 active+clean; 223 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-02-17T13:01:24.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:24 smithi119 ceph-mon[36188]: pgmap v361: 97 pgs: 97 active+clean; 223 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-02-17T13:01:25.299 INFO:tasks.workunit.client.0.smithi018.stdout: 1 576886 4.92 MB/sec execute 520 sec latency 16.131 ms 2024-02-17T13:01:25.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:25 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:01:25.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:25 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:01:25.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:25 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:01:25.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:25 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:01:25.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:25 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:01:25.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:25 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:01:26.299 INFO:tasks.workunit.client.0.smithi018.stdout: 1 577126 4.91 MB/sec execute 521 sec latency 805.232 ms 2024-02-17T13:01:26.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:26 smithi146 ceph-mon[36454]: pgmap v362: 97 pgs: 97 active+clean; 226 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-02-17T13:01:26.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:26 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:01:26.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:26 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:01:26.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:26 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:01:26.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:26 smithi018 ceph-mon[32625]: pgmap v362: 97 pgs: 97 active+clean; 226 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-02-17T13:01:26.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:26 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:01:26.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:26 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:01:26.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:26 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:01:26.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:26 smithi119 ceph-mon[36188]: pgmap v362: 97 pgs: 97 active+clean; 226 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-02-17T13:01:26.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:26 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:01:26.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:26 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:01:26.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:26 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:01:27.299 INFO:tasks.workunit.client.0.smithi018.stdout: 1 577126 4.90 MB/sec execute 522 sec latency 1805.312 ms 2024-02-17T13:01:28.299 INFO:tasks.workunit.client.0.smithi018.stdout: 1 577126 4.89 MB/sec execute 523 sec latency 2805.375 ms 2024-02-17T13:01:28.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:28 smithi018 ceph-mon[32625]: pgmap v363: 97 pgs: 97 active+clean; 226 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-02-17T13:01:28.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:28 smithi119 ceph-mon[36188]: pgmap v363: 97 pgs: 97 active+clean; 226 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-02-17T13:01:28.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:28 smithi146 ceph-mon[36454]: pgmap v363: 97 pgs: 97 active+clean; 226 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-02-17T13:01:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.285+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/50653712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6cc1002d0 msgr2=0x7fe6cc100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.285+0000 7fe6d3cdf640 1 --2- 172.21.15.18:0/50653712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6cc1002d0 0x7fe6cc100750 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009900 tx=0x7fe6bc02bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.285+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/50653712 shutdown_connections 2024-02-17T13:01:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.285+0000 7fe6d3cdf640 1 --2- 172.21.15.18:0/50653712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6cc100c90 0x7fe6cc10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.285+0000 7fe6d3cdf640 1 --2- 172.21.15.18:0/50653712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6cc1002d0 0x7fe6cc100750 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.285+0000 7fe6d3cdf640 1 --2- 172.21.15.18:0/50653712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6cc105020 0x7fe6cc105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.285+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/50653712 >> 172.21.15.18:0/50653712 conn(0x7fe6cc076ed0 msgr2=0x7fe6cc0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.286+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/50653712 shutdown_connections 2024-02-17T13:01:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.286+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/50653712 wait complete. 2024-02-17T13:01:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.286+0000 7fe6d3cdf640 1 Processor -- start 2024-02-17T13:01:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.286+0000 7fe6d3cdf640 1 -- start start 2024-02-17T13:01:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.286+0000 7fe6d3cdf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6cc1002d0 0x7fe6cc101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d3cdf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6cc100c90 0x7fe6cc102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d3cdf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6cc105020 0x7fe6cc102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d3cdf640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc1a5ba0 con 0x7fe6cc1002d0 2024-02-17T13:01:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d3cdf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc1a5d10 con 0x7fe6cc100c90 2024-02-17T13:01:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d3cdf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc1a5ed0 con 0x7fe6cc105020 2024-02-17T13:01:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d1a7b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6cc1002d0 0x7fe6cc101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d1a7b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6cc1002d0 0x7fe6cc101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44412/0 (socket says 172.21.15.18:44412) 2024-02-17T13:01:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d1a7b640 1 -- 172.21.15.18:0/1712961623 learned_addr learned my addr 172.21.15.18:0/1712961623 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d127a640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6cc100c90 0x7fe6cc102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d227c640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6cc105020 0x7fe6cc102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d1a7b640 1 -- 172.21.15.18:0/1712961623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6cc105020 msgr2=0x7fe6cc102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d1a7b640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6cc105020 0x7fe6cc102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d1a7b640 1 -- 172.21.15.18:0/1712961623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6cc100c90 msgr2=0x7fe6cc102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d1a7b640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6cc100c90 0x7fe6cc102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d1a7b640 1 -- 172.21.15.18:0/1712961623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc009560 con 0x7fe6cc1002d0 2024-02-17T13:01:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d1a7b640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6cc1002d0 0x7fe6cc101cf0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e9f0 tx=0x7fe6c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6c2ffd640 1 -- 172.21.15.18:0/1712961623 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8009700 con 0x7fe6cc1002d0 2024-02-17T13:01:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/1712961623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc1a61b0 con 0x7fe6cc1002d0 2024-02-17T13:01:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6c2ffd640 1 -- 172.21.15.18:0/1712961623 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6c800cd80 con 0x7fe6cc1002d0 2024-02-17T13:01:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6c2ffd640 1 -- 172.21.15.18:0/1712961623 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8018920 con 0x7fe6cc1002d0 2024-02-17T13:01:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.287+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/1712961623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc1a6690 con 0x7fe6cc1002d0 2024-02-17T13:01:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.290+0000 7fe6c2ffd640 1 -- 172.21.15.18:0/1712961623 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe6c8018a80 con 0x7fe6cc1002d0 2024-02-17T13:01:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.290+0000 7fe6d6993740 1 Processor -- start 2024-02-17T13:01:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d6993740 1 -- start start 2024-02-17T13:01:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d6993740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc37820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d6993740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc37d60 0x55fc6cc381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d6993740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 0x55fc6cc436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d6993740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6cba69f0 con 0x55fc6cc412e0 2024-02-17T13:01:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d6993740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6cba3f20 con 0x55fc6cc37d60 2024-02-17T13:01:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d6993740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6cb9f700 con 0x55fc6cc38ad0 2024-02-17T13:01:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d127a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 0x55fc6cc436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d1a7b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc37d60 0x55fc6cc381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d227c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc37820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d127a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 0x55fc6cc436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44422/0 (socket says 172.21.15.18:44422) 2024-02-17T13:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d127a640 1 -- 172.21.15.18:0/2526812557 learned_addr learned my addr 172.21.15.18:0/2526812557 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6c2ffd640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe6b80a2300 0x7fe6b80a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d127a640 1 -- 172.21.15.18:0/2526812557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc38ad0 msgr2=0x55fc6cc37820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d127a640 1 --2- 172.21.15.18:0/2526812557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc37820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d127a640 1 -- 172.21.15.18:0/2526812557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc37d60 msgr2=0x55fc6cc381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d127a640 1 --2- 172.21.15.18:0/2526812557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc37d60 0x55fc6cc381e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d127a640 1 -- 172.21.15.18:0/2526812557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc6cb357b0 con 0x55fc6cc412e0 2024-02-17T13:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d227c640 1 --2- 172.21.15.18:0/2526812557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc37820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.291+0000 7fe6d227c640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe6b80a2300 0x7fe6b80a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.292+0000 7fe6c2ffd640 1 -- 172.21.15.18:0/1712961623 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe6c8014070 con 0x7fe6cc1002d0 2024-02-17T13:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.292+0000 7fe6d127a640 1 --2- 172.21.15.18:0/2526812557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 0x55fc6cc436d0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009900 tx=0x7fe6bc02c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168bacb7fdc26df7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.292+0000 7fe6b7fff640 1 -- 172.21.15.18:0/2526812557 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc02c910 con 0x55fc6cc412e0 2024-02-17T13:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.292+0000 7fe6b7fff640 1 -- 172.21.15.18:0/2526812557 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc02ca70 con 0x55fc6cc412e0 2024-02-17T13:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.292+0000 7fe6b7fff640 1 -- 172.21.15.18:0/2526812557 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc005170 con 0x55fc6cc412e0 2024-02-17T13:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.292+0000 7fe6d227c640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe6b80a2300 0x7fe6b80a47c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4005fd0 tx=0x7fe6c4004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.293+0000 7fe6d6993740 1 -- 172.21.15.18:0/2526812557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 msgr2=0x55fc6cc436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.293+0000 7fe6d6993740 1 --2- 172.21.15.18:0/2526812557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 0x55fc6cc436d0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009900 tx=0x7fe6bc02c6a0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.293+0000 7fe6d6993740 1 -- 172.21.15.18:0/2526812557 shutdown_connections 2024-02-17T13:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.293+0000 7fe6d6993740 1 --2- 172.21.15.18:0/2526812557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 0x55fc6cc436d0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.293+0000 7fe6d6993740 1 --2- 172.21.15.18:0/2526812557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc37d60 0x55fc6cc381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.293+0000 7fe6d6993740 1 --2- 172.21.15.18:0/2526812557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc37820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.293+0000 7fe6d6993740 1 -- 172.21.15.18:0/2526812557 >> 172.21.15.18:0/2526812557 conn(0x55fc6cbff550 msgr2=0x55fc6cc32a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.293+0000 7fe6d6993740 1 -- 172.21.15.18:0/2526812557 shutdown_connections 2024-02-17T13:01:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.293+0000 7fe6d6993740 1 -- 172.21.15.18:0/2526812557 wait complete. 2024-02-17T13:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.293+0000 7fe6d6993740 1 -- start start 2024-02-17T13:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.293+0000 7fe6d6993740 1 Processor -- start 2024-02-17T13:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d6993740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc37d60 0x55fc6cc3e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d6993740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d6993740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc412e0 0x55fc6cc3ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d6993740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6cb44bf0 con 0x55fc6cc38ad0 2024-02-17T13:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d6993740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6c873250 con 0x55fc6cc412e0 2024-02-17T13:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d6993740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6cb25200 con 0x55fc6cc37d60 2024-02-17T13:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d6993740 10 client.0 _ll_register_callbacks cb 0x55fc6cb93d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d127a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d127a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44438/0 (socket says 172.21.15.18:44438) 2024-02-17T13:01:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d127a640 1 -- 172.21.15.18:0/4132533632 learned_addr learned my addr 172.21.15.18:0/4132533632 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d1a7b640 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc37d60 0x55fc6cc3e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d227c640 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc412e0 0x55fc6cc3ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d127a640 1 -- 172.21.15.18:0/4132533632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc37d60 msgr2=0x55fc6cc3e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d127a640 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc37d60 0x55fc6cc3e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d127a640 1 -- 172.21.15.18:0/4132533632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc412e0 msgr2=0x55fc6cc3ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d127a640 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc412e0 0x55fc6cc3ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d127a640 1 -- 172.21.15.18:0/4132533632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc10d860 con 0x55fc6cc38ad0 2024-02-17T13:01:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d1a7b640 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc37d60 0x55fc6cc3e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6d127a640 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3e820 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc02c7b0 tx=0x7fe6bc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd76cffb30c8f2cc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.294+0000 7fe6b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6b57fa640 1 -- 172.21.15.18:0/4132533632 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc04f070 con 0x55fc6cc38ad0 2024-02-17T13:01:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6b57fa640 1 -- 172.21.15.18:0/4132533632 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe6bc0050e0 con 0x55fc6cc38ad0 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6d6993740 1 -- 172.21.15.18:0/4132533632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc6c89f600 con 0x55fc6cc38ad0 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6b57fa640 1 -- 172.21.15.18:0/4132533632 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6bc02c910 con 0x55fc6cc38ad0 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6d6993740 10 client.19689 fetch_fsmap learned FSMap version 50 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6d6993740 1 -- 172.21.15.18:0/4132533632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55fc6cb9dd80 con 0x55fc6cc38ad0 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6b57fa640 1 -- 172.21.15.18:0/4132533632 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fc6c89f600 con 0x55fc6cc38ad0 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6b57fa640 1 -- 172.21.15.18:0/4132533632 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe6bc052020 con 0x55fc6cc38ad0 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6d6993740 10 client.19689 fetch_fsmap finished waiting for FSMap version 50 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6d6993740 10 client.19689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6d6993740 20 client.19689 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6d6993740 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fc6cc512c0 0x55fc6cc536e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6d6993740 4 client.19689 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6d6993740 1 -- 172.21.15.18:0/4132533632 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc6c8f01b0 con 0x55fc6cc512c0 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.295+0000 7fe6d1a7b640 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fc6cc512c0 0x55fc6cc536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.296+0000 7fe6d1a7b640 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fc6cc512c0 0x55fc6cc536e0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98bf63a51752a4eb server_cookie=ce77b28deefbab52 in_seq=0 out_seq=0 2024-02-17T13:01:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.296+0000 7fe6b57fa640 10 client.19689 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6b57fa640 1 -- 172.21.15.18:0/4132533632 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fc6c8f01b0 con 0x55fc6cc512c0 2024-02-17T13:01:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6b57fa640 10 client.19689 handle_command_reply: tid=0 2024-02-17T13:01:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6d6993740 1 client.19689 shutdown 2024-02-17T13:01:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6d6993740 1 -- 172.21.15.18:0/4132533632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 msgr2=0x55fc6cc3e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6d6993740 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3e820 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc02c7b0 tx=0x7fe6bc02bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6d6993740 1 -- 172.21.15.18:0/4132533632 shutdown_connections 2024-02-17T13:01:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6d6993740 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fc6cc512c0 0x55fc6cc536e0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6d6993740 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc412e0 0x55fc6cc3ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6d6993740 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3e820 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6d6993740 1 --2- 172.21.15.18:0/4132533632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc37d60 0x55fc6cc3e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6b57fa640 0 client.19689 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6d6993740 1 -- 172.21.15.18:0/4132533632 >> 172.21.15.18:0/4132533632 conn(0x55fc6cbff550 msgr2=0x55fc6cc347a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6d6993740 1 -- 172.21.15.18:0/4132533632 shutdown_connections 2024-02-17T13:01:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6d6993740 1 -- 172.21.15.18:0/4132533632 wait complete. 2024-02-17T13:01:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.297+0000 7fe6d6993740 20 client.19689 trim_cache size 0 max 16384 2024-02-17T13:01:29.299 INFO:tasks.workunit.client.0.smithi018.stdout: 1 577993 4.89 MB/sec execute 524 sec latency 3404.827 ms 2024-02-17T13:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.311+0000 7fe6d6993740 1 Processor -- start 2024-02-17T13:01:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d6993740 1 -- start start 2024-02-17T13:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d6993740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc37d60 0x55fc6cc40a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d6993740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d6993740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 0x55fc6cc480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d6993740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6cc39f50 con 0x55fc6cc412e0 2024-02-17T13:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d6993740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6cc4e0d0 con 0x55fc6cc38ad0 2024-02-17T13:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d6993740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6cc43b50 con 0x55fc6cc37d60 2024-02-17T13:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d227c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 0x55fc6cc480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d227c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 0x55fc6cc480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44454/0 (socket says 172.21.15.18:44454) 2024-02-17T13:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d227c640 1 -- 172.21.15.18:0/429354188 learned_addr learned my addr 172.21.15.18:0/429354188 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d127a640 1 --2- 172.21.15.18:0/429354188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d1a7b640 1 --2- 172.21.15.18:0/429354188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc37d60 0x55fc6cc40a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d227c640 1 -- 172.21.15.18:0/429354188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc37d60 msgr2=0x55fc6cc40a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d227c640 1 --2- 172.21.15.18:0/429354188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc37d60 0x55fc6cc40a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d227c640 1 -- 172.21.15.18:0/429354188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc38ad0 msgr2=0x55fc6cc3eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d227c640 1 --2- 172.21.15.18:0/429354188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d227c640 1 -- 172.21.15.18:0/429354188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc6cb3a7b0 con 0x55fc6cc412e0 2024-02-17T13:01:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d127a640 1 --2- 172.21.15.18:0/429354188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.312+0000 7fe6d227c640 1 --2- 172.21.15.18:0/429354188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 0x55fc6cc480a0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4014470 tx=0x7fe6c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bffabc5cd06b1e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.313+0000 7fe6b77fe640 1 -- 172.21.15.18:0/429354188 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4024040 con 0x55fc6cc412e0 2024-02-17T13:01:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.313+0000 7fe6b77fe640 1 -- 172.21.15.18:0/429354188 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6c4011e00 con 0x55fc6cc412e0 2024-02-17T13:01:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.313+0000 7fe6b77fe640 1 -- 172.21.15.18:0/429354188 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4012880 con 0x55fc6cc412e0 2024-02-17T13:01:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.313+0000 7fe6d6993740 1 -- 172.21.15.18:0/429354188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 msgr2=0x55fc6cc480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.313+0000 7fe6d6993740 1 --2- 172.21.15.18:0/429354188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 0x55fc6cc480a0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4014470 tx=0x7fe6c4019000 comp rx=0 tx=0).stop 2024-02-17T13:01:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.313+0000 7fe6d6993740 1 -- 172.21.15.18:0/429354188 shutdown_connections 2024-02-17T13:01:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.313+0000 7fe6d6993740 1 --2- 172.21.15.18:0/429354188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc412e0 0x55fc6cc480a0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.313+0000 7fe6d6993740 1 --2- 172.21.15.18:0/429354188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.313+0000 7fe6d6993740 1 --2- 172.21.15.18:0/429354188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc37d60 0x55fc6cc40a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.313+0000 7fe6d6993740 1 -- 172.21.15.18:0/429354188 >> 172.21.15.18:0/429354188 conn(0x55fc6cbff550 msgr2=0x55fc6cc32580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.313+0000 7fe6d6993740 1 -- 172.21.15.18:0/429354188 shutdown_connections 2024-02-17T13:01:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.313+0000 7fe6d6993740 1 -- 172.21.15.18:0/429354188 wait complete. 2024-02-17T13:01:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d6993740 1 -- start start 2024-02-17T13:01:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d6993740 1 Processor -- start 2024-02-17T13:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d6993740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc37d60 0x55fc6cc3af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d6993740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d6993740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc412e0 0x55fc6cc3b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d6993740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6cca77a0 con 0x55fc6cc38ad0 2024-02-17T13:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d6993740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6cca78e0 con 0x55fc6cc37d60 2024-02-17T13:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d6993740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6cca7a20 con 0x55fc6cc412e0 2024-02-17T13:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d6993740 10 client.0 _ll_register_callbacks cb 0x7fe6c40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d127a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d1a7b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc37d60 0x55fc6cc3af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d127a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44462/0 (socket says 172.21.15.18:44462) 2024-02-17T13:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.314+0000 7fe6d127a640 1 -- 172.21.15.18:0/7459601 learned_addr learned my addr 172.21.15.18:0/7459601 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6d227c640 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc412e0 0x55fc6cc3b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6d1a7b640 1 -- 172.21.15.18:0/7459601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc412e0 msgr2=0x55fc6cc3b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6d1a7b640 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc412e0 0x55fc6cc3b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6d1a7b640 1 -- 172.21.15.18:0/7459601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 msgr2=0x55fc6cc3b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6d1a7b640 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6d1a7b640 1 -- 172.21.15.18:0/7459601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc6cb9dd80 con 0x55fc6cc37d60 2024-02-17T13:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6d127a640 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6d1a7b640 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc37d60 0x55fc6cc3af50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80cbb90 tx=0x7fe6c8010f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9174f456e514c3b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6b57fa640 1 -- 172.21.15.18:0/7459601 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c809b530 con 0x55fc6cc37d60 2024-02-17T13:01:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6b57fa640 1 -- 172.21.15.18:0/7459601 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe6c809ae30 con 0x55fc6cc37d60 2024-02-17T13:01:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6d6993740 1 -- 172.21.15.18:0/7459601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6bc02ca70 con 0x55fc6cc37d60 2024-02-17T13:01:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6b57fa640 1 -- 172.21.15.18:0/7459601 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6c809ef80 con 0x55fc6cc37d60 2024-02-17T13:01:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6d6993740 10 client.28676 fetch_fsmap learned FSMap version 50 2024-02-17T13:01:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.315+0000 7fe6d6993740 1 -- 172.21.15.18:0/7459601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7fe6c4014180 con 0x55fc6cc37d60 2024-02-17T13:01:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.316+0000 7fe6b57fa640 1 -- 172.21.15.18:0/7459601 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6c80a7020 con 0x55fc6cc37d60 2024-02-17T13:01:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.316+0000 7fe6b57fa640 1 -- 172.21.15.18:0/7459601 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 50) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe6c80b3b60 con 0x55fc6cc37d60 2024-02-17T13:01:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.316+0000 7fe6d6993740 10 client.28676 fetch_fsmap finished waiting for FSMap version 50 2024-02-17T13:01:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.316+0000 7fe6d6993740 10 client.28676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.316+0000 7fe6d6993740 20 client.28676 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.316+0000 7fe6d6993740 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fc6cca91c0 0x55fc6ccab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.316+0000 7fe6d6993740 4 client.28676 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:01:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.316+0000 7fe6d6993740 1 -- 172.21.15.18:0/7459601 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc6c8dc550 con 0x55fc6cca91c0 2024-02-17T13:01:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.316+0000 7fe6d127a640 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fc6cca91c0 0x55fc6ccab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.316+0000 7fe6b57fa640 10 client.28676 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.316+0000 7fe6d127a640 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fc6cca91c0 0x55fc6ccab5e0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91da9f2888a61c5c server_cookie=52882e780b2631f1 in_seq=0 out_seq=0 2024-02-17T13:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6b57fa640 1 -- 172.21.15.18:0/7459601 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc6c8dc550 con 0x55fc6cca91c0 2024-02-17T13:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6b57fa640 10 client.28676 handle_command_reply: tid=0 2024-02-17T13:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6d6993740 1 client.28676 shutdown 2024-02-17T13:01:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6d6993740 1 -- 172.21.15.18:0/7459601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc37d60 msgr2=0x55fc6cc3af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6d6993740 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc37d60 0x55fc6cc3af50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80cbb90 tx=0x7fe6c8010f50 comp rx=0 tx=0).stop 2024-02-17T13:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6d6993740 1 -- 172.21.15.18:0/7459601 shutdown_connections 2024-02-17T13:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6d6993740 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fc6cca91c0 0x55fc6ccab5e0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6d6993740 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6cc412e0 0x55fc6cc3b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6d6993740 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc6cc38ad0 0x55fc6cc3b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6d6993740 1 --2- 172.21.15.18:0/7459601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fc6cc37d60 0x55fc6cc3af50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6b57fa640 0 client.28676 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6d6993740 1 -- 172.21.15.18:0/7459601 >> 172.21.15.18:0/7459601 conn(0x55fc6cbff550 msgr2=0x55fc6cc3cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6d6993740 1 -- 172.21.15.18:0/7459601 shutdown_connections 2024-02-17T13:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6d6993740 1 -- 172.21.15.18:0/7459601 wait complete. 2024-02-17T13:01:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.354+0000 7fe6d6993740 20 client.28676 trim_cache size 0 max 16384 2024-02-17T13:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:01:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:01:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.355+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/1712961623 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe6b80a2300 msgr2=0x7fe6b80a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.355+0000 7fe6d3cdf640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe6b80a2300 0x7fe6b80a47c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4005fd0 tx=0x7fe6c4004a40 comp rx=0 tx=0).stop 2024-02-17T13:01:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.355+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/1712961623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6cc1002d0 msgr2=0x7fe6cc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.355+0000 7fe6d3cdf640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6cc1002d0 0x7fe6cc101cf0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e9f0 tx=0x7fe6c800eeb0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.355+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/1712961623 shutdown_connections 2024-02-17T13:01:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.355+0000 7fe6d3cdf640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe6b80a2300 0x7fe6b80a47c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.355+0000 7fe6d3cdf640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6cc105020 0x7fe6cc102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.355+0000 7fe6d3cdf640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6cc100c90 0x7fe6cc102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.355+0000 7fe6d3cdf640 1 --2- 172.21.15.18:0/1712961623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6cc1002d0 0x7fe6cc101cf0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.355+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/1712961623 >> 172.21.15.18:0/1712961623 conn(0x7fe6cc076ed0 msgr2=0x7fe6cc068a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.355+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/1712961623 shutdown_connections 2024-02-17T13:01:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:29.355+0000 7fe6d3cdf640 1 -- 172.21.15.18:0/1712961623 wait complete. 2024-02-17T13:01:30.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 579573 4.90 MB/sec execute 525 sec latency 14.027 ms 2024-02-17T13:01:30.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:30 smithi018 ceph-mon[32625]: pgmap v364: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-02-17T13:01:30.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:30 smithi119 ceph-mon[36188]: pgmap v364: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-02-17T13:01:30.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:30 smithi146 ceph-mon[36454]: pgmap v364: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-02-17T13:01:31.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 580669 4.90 MB/sec execute 526 sec latency 350.609 ms 2024-02-17T13:01:32.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 580669 4.89 MB/sec execute 527 sec latency 1350.709 ms 2024-02-17T13:01:32.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:32 smithi018 ceph-mon[32625]: pgmap v365: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-02-17T13:01:32.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:32 smithi119 ceph-mon[36188]: pgmap v365: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-02-17T13:01:32.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:32 smithi146 ceph-mon[36454]: pgmap v365: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-02-17T13:01:33.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 580669 4.88 MB/sec execute 528 sec latency 2350.775 ms 2024-02-17T13:01:34.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 581535 4.88 MB/sec execute 529 sec latency 2948.061 ms 2024-02-17T13:01:34.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:34 smithi018 ceph-mon[32625]: pgmap v366: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-02-17T13:01:34.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:34 smithi119 ceph-mon[36188]: pgmap v366: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-02-17T13:01:34.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:34 smithi146 ceph-mon[36454]: pgmap v366: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-02-17T13:01:35.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 583137 4.89 MB/sec execute 530 sec latency 11.409 ms 2024-02-17T13:01:36.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 584222 4.89 MB/sec execute 531 sec latency 335.296 ms 2024-02-17T13:01:36.500 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:36 smithi146 ceph-mon[36454]: pgmap v367: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.7 MiB/s wr, 165 op/s 2024-02-17T13:01:36.500 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:36 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:01:36.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:36 smithi018 ceph-mon[32625]: pgmap v367: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.7 MiB/s wr, 165 op/s 2024-02-17T13:01:36.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:36 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:01:36.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:36 smithi119 ceph-mon[36188]: pgmap v367: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.7 MiB/s wr, 165 op/s 2024-02-17T13:01:36.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:36 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:01:37.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 584222 4.88 MB/sec execute 532 sec latency 1335.398 ms 2024-02-17T13:01:38.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 585126 4.87 MB/sec execute 533 sec latency 1889.209 ms 2024-02-17T13:01:38.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:38 smithi018 ceph-mon[32625]: pgmap v368: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-02-17T13:01:38.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:38 smithi119 ceph-mon[36188]: pgmap v368: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-02-17T13:01:38.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:38 smithi146 ceph-mon[36454]: pgmap v368: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-02-17T13:01:39.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 586781 4.89 MB/sec execute 534 sec latency 12.668 ms 2024-02-17T13:01:40.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 587763 4.89 MB/sec execute 535 sec latency 448.467 ms 2024-02-17T13:01:40.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:40 smithi018 ceph-mon[32625]: pgmap v369: 97 pgs: 97 active+clean; 232 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-02-17T13:01:40.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:40 smithi119 ceph-mon[36188]: pgmap v369: 97 pgs: 97 active+clean; 232 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-02-17T13:01:40.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:40 smithi146 ceph-mon[36454]: pgmap v369: 97 pgs: 97 active+clean; 232 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-02-17T13:01:41.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 587763 4.88 MB/sec execute 536 sec latency 1448.513 ms 2024-02-17T13:01:41.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:01:41.336 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:01:41.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.465+0000 7f3593e4d640 1 -- 172.21.15.18:0/2749481102 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f358c0697b0 msgr2=0x7f358c069c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.465+0000 7f3593e4d640 1 --2- 172.21.15.18:0/2749481102 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f358c0697b0 0x7f358c069c10 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f357c009930 tx=0x7f357c02bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:41.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.465+0000 7f3593e4d640 1 -- 172.21.15.18:0/2749481102 shutdown_connections 2024-02-17T13:01:41.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.465+0000 7f3593e4d640 1 --2- 172.21.15.18:0/2749481102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f358c06a150 0x7f358c10dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.465+0000 7f3593e4d640 1 --2- 172.21.15.18:0/2749481102 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f358c0697b0 0x7f358c069c10 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.465+0000 7f3593e4d640 1 --2- 172.21.15.18:0/2749481102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f358c1071b0 0x7f358c107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.465+0000 7f3593e4d640 1 -- 172.21.15.18:0/2749481102 >> 172.21.15.18:0/2749481102 conn(0x7f358c076e20 msgr2=0x7f358c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.465+0000 7f3593e4d640 1 -- 172.21.15.18:0/2749481102 shutdown_connections 2024-02-17T13:01:41.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.465+0000 7f3593e4d640 1 -- 172.21.15.18:0/2749481102 wait complete. 2024-02-17T13:01:41.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.466+0000 7f3593e4d640 1 Processor -- start 2024-02-17T13:01:41.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.466+0000 7f3593e4d640 1 -- start start 2024-02-17T13:01:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.466+0000 7f3593e4d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f358c0697b0 0x7f358c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.466+0000 7f3593e4d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f358c06a150 0x7f358c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.466+0000 7f3593e4d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f358c1071b0 0x7f358c19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.466+0000 7f3593e4d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c19c600 con 0x7f358c0697b0 2024-02-17T13:01:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.466+0000 7f3593e4d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c19c770 con 0x7f358c1071b0 2024-02-17T13:01:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.466+0000 7f3593e4d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c19c930 con 0x7f358c06a150 2024-02-17T13:01:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.466+0000 7f3591be9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f358c0697b0 0x7f358c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.466+0000 7f3591be9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f358c0697b0 0x7f358c197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40956/0 (socket says 172.21.15.18:40956) 2024-02-17T13:01:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.466+0000 7f3591be9640 1 -- 172.21.15.18:0/644537152 learned_addr learned my addr 172.21.15.18:0/644537152 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.466+0000 7f35913e8640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f358c06a150 0x7f358c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.467+0000 7f3591be9640 1 -- 172.21.15.18:0/644537152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f358c06a150 msgr2=0x7f358c197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.467+0000 7f3591be9640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f358c06a150 0x7f358c197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.467+0000 7f3591be9640 1 -- 172.21.15.18:0/644537152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f358c1071b0 msgr2=0x7f358c19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.467+0000 7f3591be9640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f358c1071b0 0x7f358c19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.467+0000 7f3591be9640 1 -- 172.21.15.18:0/644537152 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c009560 con 0x7f358c0697b0 2024-02-17T13:01:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.467+0000 7f3591be9640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f358c0697b0 0x7f358c197740 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f358800d950 tx=0x7f358800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.467+0000 7f3582ffd640 1 -- 172.21.15.18:0/644537152 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3588009c30 con 0x7f358c0697b0 2024-02-17T13:01:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.467+0000 7f3582ffd640 1 -- 172.21.15.18:0/644537152 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3588009d90 con 0x7f358c0697b0 2024-02-17T13:01:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.467+0000 7f3582ffd640 1 -- 172.21.15.18:0/644537152 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358800f7e0 con 0x7f358c0697b0 2024-02-17T13:01:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.467+0000 7f3593e4d640 1 -- 172.21.15.18:0/644537152 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c19cc10 con 0x7f358c0697b0 2024-02-17T13:01:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.467+0000 7f3593e4d640 1 -- 172.21.15.18:0/644537152 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c1a4490 con 0x7f358c0697b0 2024-02-17T13:01:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.469+0000 7f3582ffd640 1 -- 172.21.15.18:0/644537152 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3588010ad0 con 0x7f358c0697b0 2024-02-17T13:01:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.470+0000 7f3596a1b740 1 Processor -- start 2024-02-17T13:01:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.470+0000 7f3596a1b740 1 -- start start 2024-02-17T13:01:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.470+0000 7f3582ffd640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f35780a2230 0x7f35780a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.470+0000 7f3596a1b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b2722b0 0x563f4b272690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.470+0000 7f3596a1b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b272bd0 0x563f4b27b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.470+0000 7f3596a1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 0x563f4b27dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.470+0000 7f3596a1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4b1e1280 con 0x563f4b27b7b0 2024-02-17T13:01:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.470+0000 7f3596a1b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4b1de7b0 con 0x563f4b272bd0 2024-02-17T13:01:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.470+0000 7f3596a1b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4b1d9f90 con 0x563f4b2722b0 2024-02-17T13:01:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f3591be9640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f35780a2230 0x7f35780a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f35923ea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 0x563f4b27dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f3582ffd640 1 -- 172.21.15.18:0/644537152 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f35880cdc60 con 0x7f358c0697b0 2024-02-17T13:01:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f35923ea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 0x563f4b27dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40970/0 (socket says 172.21.15.18:40970) 2024-02-17T13:01:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f35923ea640 1 -- 172.21.15.18:0/995603595 learned_addr learned my addr 172.21.15.18:0/995603595 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f35913e8640 1 --2- 172.21.15.18:0/995603595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b272bd0 0x563f4b27b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f35923ea640 1 --2- 172.21.15.18:0/995603595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b2722b0 0x563f4b272690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f35923ea640 1 -- 172.21.15.18:0/995603595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b2722b0 msgr2=0x563f4b272690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f35923ea640 1 --2- 172.21.15.18:0/995603595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b2722b0 0x563f4b272690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f35923ea640 1 -- 172.21.15.18:0/995603595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b272bd0 msgr2=0x563f4b27b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f35923ea640 1 --2- 172.21.15.18:0/995603595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b272bd0 0x563f4b27b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f35923ea640 1 -- 172.21.15.18:0/995603595 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f4b198570 con 0x563f4b27b7b0 2024-02-17T13:01:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f3591be9640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f35780a2230 0x7f35780a46f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3588010fd0 tx=0x7f358800dfa0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f35923ea640 1 --2- 172.21.15.18:0/995603595 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 0x563f4b27dba0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f3584012a40 tx=0x7f35840096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e94ad97cbd04d24 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f3577fff640 1 -- 172.21.15.18:0/995603595 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358400d9a0 con 0x563f4b27b7b0 2024-02-17T13:01:41.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f3577fff640 1 -- 172.21.15.18:0/995603595 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3584018030 con 0x563f4b27b7b0 2024-02-17T13:01:41.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.471+0000 7f3577fff640 1 -- 172.21.15.18:0/995603595 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584010e30 con 0x563f4b27b7b0 2024-02-17T13:01:41.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.472+0000 7f3596a1b740 1 -- 172.21.15.18:0/995603595 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 msgr2=0x563f4b27dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.472+0000 7f3596a1b740 1 --2- 172.21.15.18:0/995603595 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 0x563f4b27dba0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f3584012a40 tx=0x7f35840096c0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.472+0000 7f3596a1b740 1 -- 172.21.15.18:0/995603595 shutdown_connections 2024-02-17T13:01:41.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.472+0000 7f3596a1b740 1 --2- 172.21.15.18:0/995603595 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 0x563f4b27dba0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.472+0000 7f3596a1b740 1 --2- 172.21.15.18:0/995603595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b272bd0 0x563f4b27b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.472+0000 7f3596a1b740 1 --2- 172.21.15.18:0/995603595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b2722b0 0x563f4b272690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.472+0000 7f3596a1b740 1 -- 172.21.15.18:0/995603595 >> 172.21.15.18:0/995603595 conn(0x563f4b26c990 msgr2=0x563f4b26cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.472+0000 7f3596a1b740 1 -- 172.21.15.18:0/995603595 shutdown_connections 2024-02-17T13:01:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.472+0000 7f3596a1b740 1 -- 172.21.15.18:0/995603595 wait complete. 2024-02-17T13:01:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.472+0000 7f3596a1b740 1 -- start start 2024-02-17T13:01:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f3596a1b740 1 Processor -- start 2024-02-17T13:01:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f3596a1b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b27b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f3596a1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 0x563f4b2785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f3596a1b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b27b7b0 0x563f4b278cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f3596a1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4aead430 con 0x563f4b272bd0 2024-02-17T13:01:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f3596a1b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4ae69e00 con 0x563f4b2722b0 2024-02-17T13:01:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f3596a1b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4b15efd0 con 0x563f4b27b7b0 2024-02-17T13:01:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f3596a1b740 10 client.0 _ll_register_callbacks cb 0x563f4b1b4fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f35913e8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 0x563f4b2785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f35913e8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 0x563f4b2785d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40984/0 (socket says 172.21.15.18:40984) 2024-02-17T13:01:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f35913e8640 1 -- 172.21.15.18:0/860028035 learned_addr learned my addr 172.21.15.18:0/860028035 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f35923ea640 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b27b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f35923ea640 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b27b7b0 0x563f4b278cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f35913e8640 1 -- 172.21.15.18:0/860028035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b27b7b0 msgr2=0x563f4b278cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f35913e8640 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b27b7b0 0x563f4b278cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f35913e8640 1 -- 172.21.15.18:0/860028035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 msgr2=0x563f4b27b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f35913e8640 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b27b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.473+0000 7f35913e8640 1 -- 172.21.15.18:0/860028035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35840126f0 con 0x563f4b272bd0 2024-02-17T13:01:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.474+0000 7f35913e8640 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 0x563f4b2785d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f357c036040 tx=0x7f357c02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7cfc88255bee6b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.474+0000 7f35757fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.474+0000 7f35757fa640 1 -- 172.21.15.18:0/860028035 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c004040 con 0x563f4b272bd0 2024-02-17T13:01:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.474+0000 7f35757fa640 1 -- 172.21.15.18:0/860028035 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f357c0041a0 con 0x563f4b272bd0 2024-02-17T13:01:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.474+0000 7f3596a1b740 1 -- 172.21.15.18:0/860028035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f4aed97e0 con 0x563f4b272bd0 2024-02-17T13:01:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.474+0000 7f35757fa640 1 -- 172.21.15.18:0/860028035 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f357c0043a0 con 0x563f4b272bd0 2024-02-17T13:01:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.474+0000 7f3596a1b740 10 client.19707 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.474+0000 7f3596a1b740 1 -- 172.21.15.18:0/860028035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x563f4b1d8610 con 0x563f4b272bd0 2024-02-17T13:01:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.474+0000 7f35757fa640 1 -- 172.21.15.18:0/860028035 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563f4aed97e0 con 0x563f4b272bd0 2024-02-17T13:01:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.474+0000 7f35757fa640 1 -- 172.21.15.18:0/860028035 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f357c03c040 con 0x563f4b272bd0 2024-02-17T13:01:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.474+0000 7f3596a1b740 10 client.19707 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.474+0000 7f3596a1b740 10 client.19707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.475+0000 7f3596a1b740 20 client.19707 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.475+0000 7f3596a1b740 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563f4b28b9c0 0x563f4b28dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.475+0000 7f3596a1b740 4 client.19707 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.475+0000 7f3596a1b740 1 -- 172.21.15.18:0/860028035 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f4af16720 con 0x563f4b28b9c0 2024-02-17T13:01:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.475+0000 7f35923ea640 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563f4b28b9c0 0x563f4b28dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.475+0000 7f35923ea640 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563f4b28b9c0 0x563f4b28dde0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2db9298f77cb242e server_cookie=6c3bee67f453d954 in_seq=0 out_seq=0 2024-02-17T13:01:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.475+0000 7f35757fa640 10 client.19707 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f35757fa640 1 -- 172.21.15.18:0/860028035 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563f4af16720 con 0x563f4b28b9c0 2024-02-17T13:01:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f35757fa640 10 client.19707 handle_command_reply: tid=0 2024-02-17T13:01:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f3596a1b740 1 client.19707 shutdown 2024-02-17T13:01:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f3596a1b740 1 -- 172.21.15.18:0/860028035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 msgr2=0x563f4b2785d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f3596a1b740 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 0x563f4b2785d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f357c036040 tx=0x7f357c02ca10 comp rx=0 tx=0).stop 2024-02-17T13:01:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f3596a1b740 1 -- 172.21.15.18:0/860028035 shutdown_connections 2024-02-17T13:01:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f3596a1b740 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563f4b28b9c0 0x563f4b28dde0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f3596a1b740 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b27b7b0 0x563f4b278cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f3596a1b740 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 0x563f4b2785d0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f3596a1b740 1 --2- 172.21.15.18:0/860028035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b27b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f35757fa640 0 client.19707 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:41.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f3596a1b740 1 -- 172.21.15.18:0/860028035 >> 172.21.15.18:0/860028035 conn(0x563f4b26c990 msgr2=0x563f4b26e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f3596a1b740 1 -- 172.21.15.18:0/860028035 shutdown_connections 2024-02-17T13:01:41.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f3596a1b740 1 -- 172.21.15.18:0/860028035 wait complete. 2024-02-17T13:01:41.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.476+0000 7f3596a1b740 20 client.19707 trim_cache size 0 max 16384 2024-02-17T13:01:41.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.491+0000 7f3596a1b740 1 Processor -- start 2024-02-17T13:01:41.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.491+0000 7f3596a1b740 1 -- start start 2024-02-17T13:01:41.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.491+0000 7f3596a1b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b27ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.491+0000 7f3596a1b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b272bd0 0x563f4b27b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.491+0000 7f3596a1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 0x563f4b282660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.491+0000 7f3596a1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4b2680f0 con 0x563f4b27b7b0 2024-02-17T13:01:41.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.491+0000 7f3596a1b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4b268320 con 0x563f4b2722b0 2024-02-17T13:01:41.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.491+0000 7f3596a1b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4b26d9e0 con 0x563f4b272bd0 2024-02-17T13:01:41.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35923ea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 0x563f4b282660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35923ea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 0x563f4b282660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41000/0 (socket says 172.21.15.18:41000) 2024-02-17T13:01:41.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35923ea640 1 -- 172.21.15.18:0/4234248645 learned_addr learned my addr 172.21.15.18:0/4234248645 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:41.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35923ea640 1 --2- 172.21.15.18:0/4234248645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b27ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35913e8640 1 --2- 172.21.15.18:0/4234248645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b272bd0 0x563f4b27b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35923ea640 1 -- 172.21.15.18:0/4234248645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b272bd0 msgr2=0x563f4b27b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35923ea640 1 --2- 172.21.15.18:0/4234248645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b272bd0 0x563f4b27b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35923ea640 1 -- 172.21.15.18:0/4234248645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 msgr2=0x563f4b27ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35923ea640 1 --2- 172.21.15.18:0/4234248645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b27ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35923ea640 1 -- 172.21.15.18:0/4234248645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f4b174ac0 con 0x563f4b27b7b0 2024-02-17T13:01:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35923ea640 1 --2- 172.21.15.18:0/4234248645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 0x563f4b282660 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f35840129e0 tx=0x7f3584010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377379cf1eb6df03 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35777fe640 1 -- 172.21.15.18:0/4234248645 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584010a80 con 0x563f4b27b7b0 2024-02-17T13:01:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35777fe640 1 -- 172.21.15.18:0/4234248645 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3584010df0 con 0x563f4b27b7b0 2024-02-17T13:01:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.492+0000 7f35777fe640 1 -- 172.21.15.18:0/4234248645 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584003970 con 0x563f4b27b7b0 2024-02-17T13:01:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.493+0000 7f3596a1b740 1 -- 172.21.15.18:0/4234248645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 msgr2=0x563f4b282660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.493+0000 7f3596a1b740 1 --2- 172.21.15.18:0/4234248645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 0x563f4b282660 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f35840129e0 tx=0x7f3584010a10 comp rx=0 tx=0).stop 2024-02-17T13:01:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.493+0000 7f3596a1b740 1 -- 172.21.15.18:0/4234248645 shutdown_connections 2024-02-17T13:01:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.493+0000 7f3596a1b740 1 --2- 172.21.15.18:0/4234248645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b27b7b0 0x563f4b282660 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.493+0000 7f3596a1b740 1 --2- 172.21.15.18:0/4234248645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b272bd0 0x563f4b27b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.493+0000 7f3596a1b740 1 --2- 172.21.15.18:0/4234248645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b27ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.493+0000 7f3596a1b740 1 -- 172.21.15.18:0/4234248645 >> 172.21.15.18:0/4234248645 conn(0x563f4b26c990 msgr2=0x563f4b276ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.493+0000 7f3596a1b740 1 -- 172.21.15.18:0/4234248645 shutdown_connections 2024-02-17T13:01:41.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.493+0000 7f3596a1b740 1 -- 172.21.15.18:0/4234248645 wait complete. 2024-02-17T13:01:41.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.493+0000 7f3596a1b740 1 -- start start 2024-02-17T13:01:41.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.493+0000 7f3596a1b740 1 Processor -- start 2024-02-17T13:01:41.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f3596a1b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b2818a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f3596a1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 0x563f4b281de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f3596a1b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b27b7b0 0x563f4b2e1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f3596a1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4b2e2430 con 0x563f4b272bd0 2024-02-17T13:01:41.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f3596a1b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4b2e2570 con 0x563f4b2722b0 2024-02-17T13:01:41.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f3596a1b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4b2e2730 con 0x563f4b27b7b0 2024-02-17T13:01:41.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f3596a1b740 10 client.0 _ll_register_callbacks cb 0x7f3584009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:41.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35913e8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 0x563f4b281de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35913e8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 0x563f4b281de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41008/0 (socket says 172.21.15.18:41008) 2024-02-17T13:01:41.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35913e8640 1 -- 172.21.15.18:0/2845476047 learned_addr learned my addr 172.21.15.18:0/2845476047 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:41.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35923ea640 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b2818a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35913e8640 1 -- 172.21.15.18:0/2845476047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b27b7b0 msgr2=0x563f4b2e1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35923ea640 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b27b7b0 0x563f4b2e1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35913e8640 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b27b7b0 0x563f4b2e1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35913e8640 1 -- 172.21.15.18:0/2845476047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 msgr2=0x563f4b2818a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35913e8640 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b2818a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35913e8640 1 -- 172.21.15.18:0/2845476047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3584026040 con 0x563f4b272bd0 2024-02-17T13:01:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35923ea640 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b2818a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:01:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35913e8640 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 0x563f4b281de0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f357c02bc70 tx=0x7f357c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ee310faff9f10 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35757fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35757fa640 1 -- 172.21.15.18:0/2845476047 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c005850 con 0x563f4b272bd0 2024-02-17T13:01:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f3596a1b740 1 -- 172.21.15.18:0/2845476047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3584018030 con 0x563f4b272bd0 2024-02-17T13:01:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.494+0000 7f35757fa640 1 -- 172.21.15.18:0/2845476047 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f357c000e40 con 0x563f4b272bd0 2024-02-17T13:01:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.495+0000 7f35757fa640 1 -- 172.21.15.18:0/2845476047 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f357c0035f0 con 0x563f4b272bd0 2024-02-17T13:01:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.495+0000 7f3596a1b740 10 client.19713 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.495+0000 7f3596a1b740 1 -- 172.21.15.18:0/2845476047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x563f4b19a6a0 con 0x563f4b272bd0 2024-02-17T13:01:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.495+0000 7f35757fa640 1 -- 172.21.15.18:0/2845476047 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3584018030 con 0x563f4b272bd0 2024-02-17T13:01:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.495+0000 7f35757fa640 1 -- 172.21.15.18:0/2845476047 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f357c059030 con 0x563f4b272bd0 2024-02-17T13:01:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.495+0000 7f3596a1b740 10 client.19713 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.495+0000 7f3596a1b740 10 client.19713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.495+0000 7f3596a1b740 20 client.19713 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.495+0000 7f3596a1b740 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563f4b2e3ce0 0x563f4b2e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.495+0000 7f3596a1b740 4 client.19713 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.495+0000 7f3596a1b740 1 -- 172.21.15.18:0/2845476047 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563f4af65890 con 0x563f4b2e3ce0 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.495+0000 7f35923ea640 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563f4b2e3ce0 0x563f4b2e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f35757fa640 10 client.19713 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f35923ea640 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563f4b2e3ce0 0x563f4b2e6100 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3126e0591a3528d server_cookie=fcfc3f21b53c7b77 in_seq=0 out_seq=0 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f35757fa640 1 -- 172.21.15.18:0/2845476047 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563f4af65890 con 0x563f4b2e3ce0 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f35757fa640 10 client.19713 handle_command_reply: tid=0 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f3596a1b740 1 client.19713 shutdown 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f3596a1b740 1 -- 172.21.15.18:0/2845476047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 msgr2=0x563f4b281de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f3596a1b740 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 0x563f4b281de0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f357c02bc70 tx=0x7f357c004610 comp rx=0 tx=0).stop 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f3596a1b740 1 -- 172.21.15.18:0/2845476047 shutdown_connections 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f3596a1b740 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x563f4b2e3ce0 0x563f4b2e6100 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f3596a1b740 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563f4b27b7b0 0x563f4b2e1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f3596a1b740 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563f4b272bd0 0x563f4b281de0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f35757fa640 0 client.19713 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f3596a1b740 1 --2- 172.21.15.18:0/2845476047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f4b2722b0 0x563f4b2818a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f3596a1b740 1 -- 172.21.15.18:0/2845476047 >> 172.21.15.18:0/2845476047 conn(0x563f4b26c990 msgr2=0x563f4b277e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f3596a1b740 1 -- 172.21.15.18:0/2845476047 shutdown_connections 2024-02-17T13:01:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f3596a1b740 1 -- 172.21.15.18:0/2845476047 wait complete. 2024-02-17T13:01:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.496+0000 7f3596a1b740 20 client.19713 trim_cache size 0 max 16384 2024-02-17T13:01:41.499 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:01:41.499 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T13:01:41.500 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T13:01:41.500 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:01:41.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.497+0000 7f3593e4d640 1 -- 172.21.15.18:0/644537152 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f35780a2230 msgr2=0x7f35780a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.497+0000 7f3593e4d640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f35780a2230 0x7f35780a46f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3588010fd0 tx=0x7f358800dfa0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.497+0000 7f3593e4d640 1 -- 172.21.15.18:0/644537152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f358c0697b0 msgr2=0x7f358c197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.497+0000 7f3593e4d640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f358c0697b0 0x7f358c197740 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f358800d950 tx=0x7f358800de10 comp rx=0 tx=0).stop 2024-02-17T13:01:41.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.497+0000 7f3593e4d640 1 -- 172.21.15.18:0/644537152 shutdown_connections 2024-02-17T13:01:41.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.497+0000 7f3593e4d640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f35780a2230 0x7f35780a46f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.497+0000 7f3593e4d640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f358c1071b0 0x7f358c19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.497+0000 7f3593e4d640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f358c06a150 0x7f358c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.497+0000 7f3593e4d640 1 --2- 172.21.15.18:0/644537152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f358c0697b0 0x7f358c197740 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.497+0000 7f3593e4d640 1 -- 172.21.15.18:0/644537152 >> 172.21.15.18:0/644537152 conn(0x7f358c076e20 msgr2=0x7f358c0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.497+0000 7f3593e4d640 1 -- 172.21.15.18:0/644537152 shutdown_connections 2024-02-17T13:01:41.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.497+0000 7f3593e4d640 1 -- 172.21.15.18:0/644537152 wait complete. 2024-02-17T13:01:41.502 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T13:01:41.502 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T13:01:41.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.617+0000 7f2b9460e640 1 -- 172.21.15.18:0/3698951263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b8c106450 msgr2=0x7f2b8c10e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.617+0000 7f2b9460e640 1 --2- 172.21.15.18:0/3698951263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b8c106450 0x7f2b8c10e020 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009cd0 tx=0x7f2b8402c0f0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.617+0000 7f2b9460e640 1 -- 172.21.15.18:0/3698951263 shutdown_connections 2024-02-17T13:01:41.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.617+0000 7f2b9460e640 1 --2- 172.21.15.18:0/3698951263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b8c106450 0x7f2b8c10e020 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.617+0000 7f2b9460e640 1 --2- 172.21.15.18:0/3698951263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b8c101d30 0x7f2b8c105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.617+0000 7f2b9460e640 1 --2- 172.21.15.18:0/3698951263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b8c101380 0x7f2b8c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.617+0000 7f2b9460e640 1 -- 172.21.15.18:0/3698951263 >> 172.21.15.18:0/3698951263 conn(0x7f2b8c0fcc20 msgr2=0x7f2b8c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.617+0000 7f2b9460e640 1 -- 172.21.15.18:0/3698951263 shutdown_connections 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.617+0000 7f2b9460e640 1 -- 172.21.15.18:0/3698951263 wait complete. 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b9460e640 1 Processor -- start 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b9460e640 1 -- start start 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b9460e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b8c101380 0x7f2b8c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b9460e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b8c106450 0x7f2b8c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b9460e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b8c19c020 0x7f2b8c19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b9460e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c19ca10 con 0x7f2b8c19c020 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b9460e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c19cb80 con 0x7f2b8c106450 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b9460e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c19ccf0 con 0x7f2b8c101380 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b92bab640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b8c19c020 0x7f2b8c19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b92bab640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b8c19c020 0x7f2b8c19c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41018/0 (socket says 172.21.15.18:41018) 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b92bab640 1 -- 172.21.15.18:0/1135504014 learned_addr learned my addr 172.21.15.18:0/1135504014 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b923aa640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b8c101380 0x7f2b8c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b91ba9640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b8c106450 0x7f2b8c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b92bab640 1 -- 172.21.15.18:0/1135504014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b8c101380 msgr2=0x7f2b8c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b92bab640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b8c101380 0x7f2b8c197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b92bab640 1 -- 172.21.15.18:0/1135504014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b8c106450 msgr2=0x7f2b8c197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b92bab640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b8c106450 0x7f2b8c197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b92bab640 1 -- 172.21.15.18:0/1135504014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b84009900 con 0x7f2b8c19c020 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.618+0000 7f2b92bab640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b8c19c020 0x7f2b8c19c4d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f2b840348f0 tx=0x7f2b84034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.619+0000 7f2b837fe640 1 -- 172.21.15.18:0/1135504014 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840328c0 con 0x7f2b8c19c020 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.619+0000 7f2b837fe640 1 -- 172.21.15.18:0/1135504014 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b84003e50 con 0x7f2b8c19c020 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.619+0000 7f2b9460e640 1 -- 172.21.15.18:0/1135504014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8c1a42d0 con 0x7f2b8c19c020 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.619+0000 7f2b837fe640 1 -- 172.21.15.18:0/1135504014 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840048a0 con 0x7f2b8c19c020 2024-02-17T13:01:41.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.619+0000 7f2b9460e640 1 -- 172.21.15.18:0/1135504014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c1a46f0 con 0x7f2b8c19c020 2024-02-17T13:01:41.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.620+0000 7f2b9460e640 1 -- 172.21.15.18:0/1135504014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8c0683c0 con 0x7f2b8c19c020 2024-02-17T13:01:41.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.621+0000 7f2b837fe640 1 -- 172.21.15.18:0/1135504014 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2b8400a040 con 0x7f2b8c19c020 2024-02-17T13:01:41.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.621+0000 7f2b837fe640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b780a2230 0x7f2b780a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.622+0000 7f2b923aa640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b780a2230 0x7f2b780a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.622+0000 7f2b837fe640 1 -- 172.21.15.18:0/1135504014 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2b840f1260 con 0x7f2b8c19c020 2024-02-17T13:01:41.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.622+0000 7f2b923aa640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b780a2230 0x7f2b780a46f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800abe0 tx=0x7f2b8800a2d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.625+0000 7f2b837fe640 1 -- 172.21.15.18:0/1135504014 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b840ba140 con 0x7f2b8c19c020 2024-02-17T13:01:41.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.843+0000 7f2b9460e640 1 -- 172.21.15.18:0/1135504014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b8c101d30 con 0x7f2b8c19c020 2024-02-17T13:01:41.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.844+0000 7f2b837fe640 1 -- 172.21.15.18:0/1135504014 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+11392 (secure 0 0 0) 0x7f2b840bd800 con 0x7f2b8c19c020 2024-02-17T13:01:41.834 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:01:41.834 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T13:01:34.948363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T13:01:41.834 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 51 2024-02-17T13:01:41.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.846+0000 7f2b9460e640 1 -- 172.21.15.18:0/1135504014 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b780a2230 msgr2=0x7f2b780a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.846+0000 7f2b9460e640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b780a2230 0x7f2b780a46f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800abe0 tx=0x7f2b8800a2d0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.846+0000 7f2b9460e640 1 -- 172.21.15.18:0/1135504014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b8c19c020 msgr2=0x7f2b8c19c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.846+0000 7f2b9460e640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b8c19c020 0x7f2b8c19c4d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f2b840348f0 tx=0x7f2b84034cc0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.846+0000 7f2b9460e640 1 -- 172.21.15.18:0/1135504014 shutdown_connections 2024-02-17T13:01:41.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.846+0000 7f2b9460e640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b780a2230 0x7f2b780a46f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.846+0000 7f2b9460e640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b8c19c020 0x7f2b8c19c4d0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.846+0000 7f2b9460e640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b8c106450 0x7f2b8c197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.846+0000 7f2b9460e640 1 --2- 172.21.15.18:0/1135504014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b8c101380 0x7f2b8c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.846+0000 7f2b9460e640 1 -- 172.21.15.18:0/1135504014 >> 172.21.15.18:0/1135504014 conn(0x7f2b8c0fcc20 msgr2=0x7f2b8c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.846+0000 7f2b9460e640 1 -- 172.21.15.18:0/1135504014 shutdown_connections 2024-02-17T13:01:41.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.846+0000 7f2b9460e640 1 -- 172.21.15.18:0/1135504014 wait complete. 2024-02-17T13:01:41.847 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:01:41.847 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T13:01:41.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.963+0000 7fd23cbb5640 1 -- 172.21.15.18:0/2966975655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238100310 msgr2=0x7fd2381006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.963+0000 7fd23cbb5640 1 --2- 172.21.15.18:0/2966975655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238100310 0x7fd2381006f0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fd228009930 tx=0x7fd22802bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.963+0000 7fd23cbb5640 1 -- 172.21.15.18:0/2966975655 shutdown_connections 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.963+0000 7fd23cbb5640 1 --2- 172.21.15.18:0/2966975655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2381053e0 0x7fd23810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.963+0000 7fd23cbb5640 1 --2- 172.21.15.18:0/2966975655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd238100cc0 0x7fd238104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.963+0000 7fd23cbb5640 1 --2- 172.21.15.18:0/2966975655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238100310 0x7fd2381006f0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.963+0000 7fd23cbb5640 1 -- 172.21.15.18:0/2966975655 >> 172.21.15.18:0/2966975655 conn(0x7fd2380771e0 msgr2=0x7fd2380775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.963+0000 7fd23cbb5640 1 -- 172.21.15.18:0/2966975655 shutdown_connections 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.963+0000 7fd23cbb5640 1 -- 172.21.15.18:0/2966975655 wait complete. 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23cbb5640 1 Processor -- start 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23cbb5640 1 -- start start 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23cbb5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238100310 0x7fd23819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23cbb5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2381053e0 0x7fd23819dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23cbb5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd238197770 0x7fd238197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23cbb5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd238198240 con 0x7fd238100310 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23cbb5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2381983b0 con 0x7fd2381053e0 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23cbb5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd238198570 con 0x7fd238197770 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238100310 0x7fd23819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238100310 0x7fd23819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41030/0 (socket says 172.21.15.18:41030) 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23659c640 1 -- 172.21.15.18:0/3546375108 learned_addr learned my addr 172.21.15.18:0/3546375108 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd22ffff640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2381053e0 0x7fd23819dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23659c640 1 -- 172.21.15.18:0/3546375108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd238197770 msgr2=0x7fd238197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23659c640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd238197770 0x7fd238197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23659c640 1 -- 172.21.15.18:0/3546375108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2381053e0 msgr2=0x7fd23819dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23659c640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2381053e0 0x7fd23819dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23659c640 1 -- 172.21.15.18:0/3546375108 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd228009560 con 0x7fd238100310 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd22ffff640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2381053e0 0x7fd23819dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.964+0000 7fd23659c640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238100310 0x7fd23819d6b0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fd228005940 tx=0x7fd2280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.965+0000 7fd22f7fe640 1 -- 172.21.15.18:0/3546375108 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2280044e0 con 0x7fd238100310 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.965+0000 7fd23cbb5640 1 -- 172.21.15.18:0/3546375108 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2381a44b0 con 0x7fd238100310 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.965+0000 7fd22f7fe640 1 -- 172.21.15.18:0/3546375108 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd22802cbc0 con 0x7fd238100310 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.965+0000 7fd22f7fe640 1 -- 172.21.15.18:0/3546375108 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228003e20 con 0x7fd238100310 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.965+0000 7fd23cbb5640 1 -- 172.21.15.18:0/3546375108 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2381a4990 con 0x7fd238100310 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.967+0000 7fd22f7fe640 1 -- 172.21.15.18:0/3546375108 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd2280035f0 con 0x7fd238100310 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.967+0000 7fd23f781740 1 Processor -- start 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.967+0000 7fd23f781740 1 -- start start 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.967+0000 7fd23f781740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d729f80 0x55b13d72a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.967+0000 7fd23f781740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d72a8a0 0x55b13d737620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.967+0000 7fd23f781740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d737b60 0x55b13d739f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.967+0000 7fd23f781740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d6e5960 con 0x55b13d737b60 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.967+0000 7fd23f781740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d699310 con 0x55b13d72a8a0 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.967+0000 7fd23f781740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d696840 con 0x55b13d729f80 2024-02-17T13:01:41.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d737b60 0x55b13d739f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d737b60 0x55b13d739f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41036/0 (socket says 172.21.15.18:41036) 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22ffff640 1 -- 172.21.15.18:0/4036350783 learned_addr learned my addr 172.21.15.18:0/4036350783 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd23659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d72a8a0 0x55b13d737620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22f7fe640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd2240a2230 0x7fd2240a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd236d9d640 1 --2- 172.21.15.18:0/4036350783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d729f80 0x55b13d72a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd236d9d640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd2240a2230 0x7fd2240a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22ffff640 1 -- 172.21.15.18:0/4036350783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d729f80 msgr2=0x55b13d72a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22ffff640 1 --2- 172.21.15.18:0/4036350783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d729f80 0x55b13d72a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22ffff640 1 -- 172.21.15.18:0/4036350783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d72a8a0 msgr2=0x55b13d737620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22ffff640 1 --2- 172.21.15.18:0/4036350783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d72a8a0 0x55b13d737620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22ffff640 1 -- 172.21.15.18:0/4036350783 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b13d654060 con 0x55b13d737b60 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22f7fe640 1 -- 172.21.15.18:0/3546375108 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd2280f0320 con 0x7fd238100310 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22ffff640 1 --2- 172.21.15.18:0/4036350783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d737b60 0x55b13d739f50 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fd22000a9a0 tx=0x7fd22000ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be518a6e3d6b5fda server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd236d9d640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd2240a2230 0x7fd2240a46f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd230008fd0 tx=0x7fd230006ce0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22cff9640 1 -- 172.21.15.18:0/4036350783 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22000cc00 con 0x55b13d737b60 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22cff9640 1 -- 172.21.15.18:0/4036350783 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd22000cd60 con 0x55b13d737b60 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.968+0000 7fd22cff9640 1 -- 172.21.15.18:0/4036350783 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220018720 con 0x55b13d737b60 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.969+0000 7fd23f781740 1 -- 172.21.15.18:0/4036350783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d737b60 msgr2=0x55b13d739f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.969+0000 7fd23f781740 1 --2- 172.21.15.18:0/4036350783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d737b60 0x55b13d739f50 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fd22000a9a0 tx=0x7fd22000ae60 comp rx=0 tx=0).stop 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.969+0000 7fd23f781740 1 -- 172.21.15.18:0/4036350783 shutdown_connections 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.969+0000 7fd23f781740 1 --2- 172.21.15.18:0/4036350783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d737b60 0x55b13d739f50 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.969+0000 7fd23f781740 1 --2- 172.21.15.18:0/4036350783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d72a8a0 0x55b13d737620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.969+0000 7fd23f781740 1 --2- 172.21.15.18:0/4036350783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d729f80 0x55b13d72a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.969+0000 7fd23f781740 1 -- 172.21.15.18:0/4036350783 >> 172.21.15.18:0/4036350783 conn(0x55b13d724980 msgr2=0x55b13d724d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.969+0000 7fd23f781740 1 -- 172.21.15.18:0/4036350783 shutdown_connections 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.969+0000 7fd23f781740 1 -- 172.21.15.18:0/4036350783 wait complete. 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23f781740 1 -- start start 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23f781740 1 Processor -- start 2024-02-17T13:01:41.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23f781740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 0x55b13d734b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23f781740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d72a8a0 0x55b13d735080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23f781740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d7355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23f781740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d692020 con 0x55b13d729f80 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23f781740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d365430 con 0x55b13d737b60 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23f781740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d321e00 con 0x55b13d72a8a0 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23f781740 10 client.0 _ll_register_callbacks cb 0x55b13d639cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 0x55b13d734b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 0x55b13d734b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41038/0 (socket says 172.21.15.18:41038) 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23659c640 1 -- 172.21.15.18:0/3846582354 learned_addr learned my addr 172.21.15.18:0/3846582354 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd22ffff640 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d72a8a0 0x55b13d735080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd236d9d640 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d7355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23659c640 1 -- 172.21.15.18:0/3846582354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d72a8a0 msgr2=0x55b13d735080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23659c640 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d72a8a0 0x55b13d735080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23659c640 1 -- 172.21.15.18:0/3846582354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 msgr2=0x55b13d7355c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23659c640 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d7355c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23659c640 1 -- 172.21.15.18:0/3846582354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22000a5d0 con 0x55b13d729f80 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd236d9d640 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d7355c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd23659c640 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 0x55b13d734b40 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd2280b8b20 tx=0x7fd228003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6786c4fc63c3d8ed server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd21dffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.970+0000 7fd21dffb640 1 -- 172.21.15.18:0/3846582354 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2280bd540 con 0x55b13d729f80 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd21dffb640 1 -- 172.21.15.18:0/3846582354 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd22803de90 con 0x55b13d729f80 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd23f781740 1 -- 172.21.15.18:0/3846582354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b13d3917e0 con 0x55b13d729f80 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd21dffb640 1 -- 172.21.15.18:0/3846582354 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2280bce40 con 0x55b13d729f80 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd23f781740 10 client.19731 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd23f781740 1 -- 172.21.15.18:0/3846582354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55b13d6906a0 con 0x55b13d729f80 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd21dffb640 1 -- 172.21.15.18:0/3846582354 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2280d4030 con 0x55b13d729f80 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd21dffb640 1 -- 172.21.15.18:0/3846582354 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd22803c070 con 0x55b13d729f80 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd23f781740 10 client.19731 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd23f781740 10 client.19731 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:01:41.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd23f781740 20 client.19731 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd23f781740 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55b13d747dc0 0x55b13d74a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd23f781740 4 client.19731 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd23f781740 1 -- 172.21.15.18:0/3846582354 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b13d3ce720 con 0x55b13d747dc0 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.971+0000 7fd22ffff640 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55b13d747dc0 0x55b13d74a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd21dffb640 10 client.19731 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd22ffff640 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55b13d747dc0 0x55b13d74a1b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=93cb0a751f0d47f6 server_cookie=8f89cd96020b8e23 in_seq=0 out_seq=0 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd21dffb640 1 -- 172.21.15.18:0/3846582354 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b13d3ce720 con 0x55b13d747dc0 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd21dffb640 10 client.19731 handle_command_reply: tid=0 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd23f781740 1 client.19731 shutdown 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd23f781740 1 -- 172.21.15.18:0/3846582354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 msgr2=0x55b13d734b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd23f781740 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 0x55b13d734b40 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd2280b8b20 tx=0x7fd228003920 comp rx=0 tx=0).stop 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd23f781740 1 -- 172.21.15.18:0/3846582354 shutdown_connections 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd23f781740 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55b13d747dc0 0x55b13d74a1b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd23f781740 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d7355c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd23f781740 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d72a8a0 0x55b13d735080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd23f781740 1 --2- 172.21.15.18:0/3846582354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 0x55b13d734b40 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd21dffb640 0 client.19731 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:01:41.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.972+0000 7fd23f781740 1 -- 172.21.15.18:0/3846582354 >> 172.21.15.18:0/3846582354 conn(0x55b13d724980 msgr2=0x55b13d7266d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.973+0000 7fd23f781740 1 -- 172.21.15.18:0/3846582354 shutdown_connections 2024-02-17T13:01:41.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.973+0000 7fd23f781740 1 -- 172.21.15.18:0/3846582354 wait complete. 2024-02-17T13:01:41.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.973+0000 7fd23f781740 20 client.19731 trim_cache size 0 max 16384 2024-02-17T13:01:41.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.984+0000 7fd23f781740 1 Processor -- start 2024-02-17T13:01:41.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.984+0000 7fd23f781740 1 -- start start 2024-02-17T13:01:41.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.984+0000 7fd23f781740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 0x55b13d7353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.984+0000 7fd23f781740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d72a8a0 0x55b13d73ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.984+0000 7fd23f781740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d73ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.984+0000 7fd23f781740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d617130 con 0x55b13d729f80 2024-02-17T13:01:41.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.984+0000 7fd23f781740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d730470 con 0x55b13d737b60 2024-02-17T13:01:41.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.984+0000 7fd23f781740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d3ce720 con 0x55b13d72a8a0 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd23659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 0x55b13d7353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd23659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 0x55b13d7353e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41048/0 (socket says 172.21.15.18:41048) 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd23659c640 1 -- 172.21.15.18:0/3480377447 learned_addr learned my addr 172.21.15.18:0/3480377447 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd23659c640 1 -- 172.21.15.18:0/3480377447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d72a8a0 msgr2=0x55b13d73ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd236d9d640 1 --2- 172.21.15.18:0/3480377447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d73ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd23659c640 1 --2- 172.21.15.18:0/3480377447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d72a8a0 0x55b13d73ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd23659c640 1 -- 172.21.15.18:0/3480377447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 msgr2=0x55b13d73ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd23659c640 1 --2- 172.21.15.18:0/3480377447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d73ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd23659c640 1 -- 172.21.15.18:0/3480377447 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b13d62cc20 con 0x55b13d729f80 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd236d9d640 1 --2- 172.21.15.18:0/3480377447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d73ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd23659c640 1 --2- 172.21.15.18:0/3480377447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 0x55b13d7353e0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd2280bcb40 tx=0x7fd2280056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db01390eeeaa415 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd21ffff640 1 -- 172.21.15.18:0/3480377447 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22803de90 con 0x55b13d729f80 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd21ffff640 1 -- 172.21.15.18:0/3480377447 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2280d4070 con 0x55b13d729f80 2024-02-17T13:01:41.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.985+0000 7fd21ffff640 1 -- 172.21.15.18:0/3480377447 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2280883e0 con 0x55b13d729f80 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 -- 172.21.15.18:0/3480377447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 msgr2=0x55b13d7353e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 --2- 172.21.15.18:0/3480377447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 0x55b13d7353e0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd2280bcb40 tx=0x7fd2280056c0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 -- 172.21.15.18:0/3480377447 shutdown_connections 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 --2- 172.21.15.18:0/3480377447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d73ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 --2- 172.21.15.18:0/3480377447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d72a8a0 0x55b13d73ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 --2- 172.21.15.18:0/3480377447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d729f80 0x55b13d7353e0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 -- 172.21.15.18:0/3480377447 >> 172.21.15.18:0/3480377447 conn(0x55b13d724980 msgr2=0x55b13d723b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 -- 172.21.15.18:0/3480377447 shutdown_connections 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 -- 172.21.15.18:0/3480377447 wait complete. 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 -- start start 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 Processor -- start 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d729f80 0x55b13d73dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d72a8a0 0x55b13d73e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d798220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d798760 con 0x55b13d72a8a0 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d7988d0 con 0x55b13d737b60 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d798a90 con 0x55b13d729f80 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd23f781740 10 client.0 _ll_register_callbacks cb 0x7fd22000a1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd22ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d72a8a0 0x55b13d73e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd22ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d72a8a0 0x55b13d73e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41058/0 (socket says 172.21.15.18:41058) 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd22ffff640 1 -- 172.21.15.18:0/1336352607 learned_addr learned my addr 172.21.15.18:0/1336352607 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.986+0000 7fd236d9d640 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d798220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd23659c640 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d729f80 0x55b13d73dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd22ffff640 1 -- 172.21.15.18:0/1336352607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d729f80 msgr2=0x55b13d73dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd22ffff640 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d729f80 0x55b13d73dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd22ffff640 1 -- 172.21.15.18:0/1336352607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 msgr2=0x55b13d798220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd22ffff640 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d798220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd22ffff640 1 -- 172.21.15.18:0/1336352607 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b13d6906a0 con 0x55b13d72a8a0 2024-02-17T13:01:41.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd23659c640 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d729f80 0x55b13d73dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:01:41.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd22ffff640 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d72a8a0 0x55b13d73e1a0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fd220011f30 tx=0x7fd220006f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23122d17705474eb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:41.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd21dffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:41.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd21dffb640 1 -- 172.21.15.18:0/1336352607 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2200115f0 con 0x55b13d72a8a0 2024-02-17T13:01:41.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd21dffb640 1 -- 172.21.15.18:0/1336352607 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd22001f040 con 0x55b13d72a8a0 2024-02-17T13:01:41.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd23f781740 1 -- 172.21.15.18:0/1336352607 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd22000cd60 con 0x55b13d72a8a0 2024-02-17T13:01:41.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd21dffb640 1 -- 172.21.15.18:0/1336352607 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd220011750 con 0x55b13d72a8a0 2024-02-17T13:01:41.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd23f781740 10 client.19737 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:41.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd23f781740 1 -- 172.21.15.18:0/1336352607 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55b13d6528c0 con 0x55b13d72a8a0 2024-02-17T13:01:41.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd21dffb640 1 -- 172.21.15.18:0/1336352607 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd22000cd60 con 0x55b13d72a8a0 2024-02-17T13:01:41.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd21dffb640 1 -- 172.21.15.18:0/1336352607 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd2200384f0 con 0x55b13d72a8a0 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd23f781740 10 client.19737 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd23f781740 10 client.19737 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd23f781740 20 client.19737 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd23f781740 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55b13d7a0160 0x55b13d7a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd23f781740 4 client.19737 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.987+0000 7fd23f781740 1 -- 172.21.15.18:0/1336352607 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b13d41d890 con 0x55b13d7a0160 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23659c640 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55b13d7a0160 0x55b13d7a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd21dffb640 10 client.19737 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23659c640 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55b13d7a0160 0x55b13d7a2580 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=84d09c282437ef57 server_cookie=b7223c2474aed917 in_seq=0 out_seq=0 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd21dffb640 1 -- 172.21.15.18:0/1336352607 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b13d41d890 con 0x55b13d7a0160 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd21dffb640 10 client.19737 handle_command_reply: tid=0 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23f781740 1 client.19737 shutdown 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23f781740 1 -- 172.21.15.18:0/1336352607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d72a8a0 msgr2=0x55b13d73e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23f781740 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d72a8a0 0x55b13d73e1a0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fd220011f30 tx=0x7fd220006f10 comp rx=0 tx=0).stop 2024-02-17T13:01:41.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23f781740 1 -- 172.21.15.18:0/1336352607 shutdown_connections 2024-02-17T13:01:41.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23f781740 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55b13d7a0160 0x55b13d7a2580 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23f781740 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b13d737b60 0x55b13d798220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23f781740 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b13d72a8a0 0x55b13d73e1a0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd21dffb640 0 client.19737 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:01:41.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23f781740 1 --2- 172.21.15.18:0/1336352607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b13d729f80 0x55b13d73dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23f781740 1 -- 172.21.15.18:0/1336352607 >> 172.21.15.18:0/1336352607 conn(0x55b13d724980 msgr2=0x55b13d725c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23f781740 1 -- 172.21.15.18:0/1336352607 shutdown_connections 2024-02-17T13:01:41.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23f781740 1 -- 172.21.15.18:0/1336352607 wait complete. 2024-02-17T13:01:41.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.988+0000 7fd23f781740 20 client.19737 trim_cache size 0 max 16384 2024-02-17T13:01:41.985 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:01:41.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.989+0000 7fd23cbb5640 1 -- 172.21.15.18:0/3546375108 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd2240a2230 msgr2=0x7fd2240a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.989+0000 7fd23cbb5640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd2240a2230 0x7fd2240a46f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd230008fd0 tx=0x7fd230006ce0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.989+0000 7fd23cbb5640 1 -- 172.21.15.18:0/3546375108 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238100310 msgr2=0x7fd23819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:41.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.989+0000 7fd23cbb5640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238100310 0x7fd23819d6b0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fd228005940 tx=0x7fd2280042f0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.989+0000 7fd23cbb5640 1 -- 172.21.15.18:0/3546375108 shutdown_connections 2024-02-17T13:01:41.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.989+0000 7fd23cbb5640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd2240a2230 0x7fd2240a46f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.989+0000 7fd23cbb5640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd238197770 0x7fd238197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.989+0000 7fd23cbb5640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2381053e0 0x7fd23819dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.989+0000 7fd23cbb5640 1 --2- 172.21.15.18:0/3546375108 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd238100310 0x7fd23819d6b0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:41.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.989+0000 7fd23cbb5640 1 -- 172.21.15.18:0/3546375108 >> 172.21.15.18:0/3546375108 conn(0x7fd2380771e0 msgr2=0x7fd238069ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:41.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.989+0000 7fd23cbb5640 1 -- 172.21.15.18:0/3546375108 shutdown_connections 2024-02-17T13:01:41.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:41.989+0000 7fd23cbb5640 1 -- 172.21.15.18:0/3546375108 wait complete. 2024-02-17T13:01:41.989 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:01:41.989 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T13:01:42.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.104+0000 7f2ba57c2640 1 -- 172.21.15.18:0/18028469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ba0105330 msgr2=0x7f2ba010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.104+0000 7f2ba57c2640 1 --2- 172.21.15.18:0/18028469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ba0105330 0x7f2ba010cf00 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009cd0 tx=0x7f2b9002c0f0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.104+0000 7f2ba57c2640 1 -- 172.21.15.18:0/18028469 shutdown_connections 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.104+0000 7f2ba57c2640 1 --2- 172.21.15.18:0/18028469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ba0105330 0x7f2ba010cf00 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.104+0000 7f2ba57c2640 1 --2- 172.21.15.18:0/18028469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ba0100c10 0x7f2ba0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.104+0000 7f2ba57c2640 1 --2- 172.21.15.18:0/18028469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ba0100260 0x7f2ba0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.104+0000 7f2ba57c2640 1 -- 172.21.15.18:0/18028469 >> 172.21.15.18:0/18028469 conn(0x7f2ba0077180 msgr2=0x7f2ba0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.104+0000 7f2ba57c2640 1 -- 172.21.15.18:0/18028469 shutdown_connections 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.104+0000 7f2ba57c2640 1 -- 172.21.15.18:0/18028469 wait complete. 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.105+0000 7f2ba57c2640 1 Processor -- start 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.105+0000 7f2ba57c2640 1 -- start start 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.105+0000 7f2ba57c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ba0100260 0x7f2ba019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.105+0000 7f2ba57c2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ba0105330 0x7f2ba019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.105+0000 7f2ba57c2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ba0197650 0x7f2ba0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.105+0000 7f2ba57c2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba0198040 con 0x7f2ba0105330 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.105+0000 7f2ba57c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba01981b0 con 0x7f2ba0100260 2024-02-17T13:01:42.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.105+0000 7f2ba57c2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba0198370 con 0x7f2ba0197650 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.105+0000 7f2b9e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ba0105330 0x7f2ba019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b9e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ba0105330 0x7f2ba019d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41076/0 (socket says 172.21.15.18:41076) 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/2849668429 learned_addr learned my addr 172.21.15.18:0/2849668429 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b9effd640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ba0100260 0x7f2ba019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b9f7fe640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ba0197650 0x7f2ba0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/2849668429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ba0197650 msgr2=0x7f2ba0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ba0197650 0x7f2ba0197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/2849668429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ba0100260 msgr2=0x7f2ba019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ba0100260 0x7f2ba019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/2849668429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90009900 con 0x7f2ba0105330 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b9f7fe640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ba0197650 0x7f2ba0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ba0105330 0x7f2ba019d990 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400c970 tx=0x7f2b9400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b87fff640 1 -- 172.21.15.18:0/2849668429 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9400cea0 con 0x7f2ba0105330 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2ba57c2640 1 -- 172.21.15.18:0/2849668429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba01a4310 con 0x7f2ba0105330 2024-02-17T13:01:42.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b87fff640 1 -- 172.21.15.18:0/2849668429 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b94007bb0 con 0x7f2ba0105330 2024-02-17T13:01:42.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2b87fff640 1 -- 172.21.15.18:0/2849668429 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94018730 con 0x7f2ba0105330 2024-02-17T13:01:42.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.106+0000 7f2ba57c2640 1 -- 172.21.15.18:0/2849668429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba01a47d0 con 0x7f2ba0105330 2024-02-17T13:01:42.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.108+0000 7f2b87fff640 1 -- 172.21.15.18:0/2849668429 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2b94007d20 con 0x7f2ba0105330 2024-02-17T13:01:42.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.108+0000 7f2ba838a740 1 Processor -- start 2024-02-17T13:01:42.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2ba838a740 1 -- start start 2024-02-17T13:01:42.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2ba838a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 0x561281b4e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2ba838a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b4e5e0 0x561281b5b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2ba838a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b5bb60 0x561281b5df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2ba838a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561281b09960 con 0x561281b5bb60 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2ba838a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561281abd310 con 0x561281b4e5e0 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2ba838a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561281aba840 con 0x561281b4dc30 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b9e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b5bb60 0x561281b5df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b9e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b5bb60 0x561281b5df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41084/0 (socket says 172.21.15.18:41084) 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/2229474127 learned_addr learned my addr 172.21.15.18:0/2229474127 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b9f7fe640 1 --2- 172.21.15.18:0/2229474127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 0x561281b4e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b9effd640 1 --2- 172.21.15.18:0/2229474127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b4e5e0 0x561281b5b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b87fff640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b880a2230 0x7f2b880a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b9f7fe640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b880a2230 0x7f2b880a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/2229474127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 msgr2=0x561281b4e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/2229474127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 0x561281b4e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/2229474127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b4e5e0 msgr2=0x561281b5b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/2229474127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b4e5e0 0x561281b5b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/2229474127 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561281a78060 con 0x561281b5bb60 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.109+0000 7f2b87fff640 1 -- 172.21.15.18:0/2849668429 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2b940ce0f0 con 0x7f2ba0105330 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/2229474127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b5bb60 0x561281b5df50 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2b940cb870 tx=0x7f2b94096f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e90e7a57c6d0758 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2b857fa640 1 -- 172.21.15.18:0/2229474127 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9409a500 con 0x561281b5bb60 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2b9f7fe640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b880a2230 0x7f2b880a46f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b90006010 tx=0x7f2b9002c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2b857fa640 1 -- 172.21.15.18:0/2229474127 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b940d1d80 con 0x561281b5bb60 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2b857fa640 1 -- 172.21.15.18:0/2229474127 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94097210 con 0x561281b5bb60 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2ba838a740 1 -- 172.21.15.18:0/2229474127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b5bb60 msgr2=0x561281b5df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2ba838a740 1 --2- 172.21.15.18:0/2229474127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b5bb60 0x561281b5df50 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2b940cb870 tx=0x7f2b94096f70 comp rx=0 tx=0).stop 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2ba838a740 1 -- 172.21.15.18:0/2229474127 shutdown_connections 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2ba838a740 1 --2- 172.21.15.18:0/2229474127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b5bb60 0x561281b5df50 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2ba838a740 1 --2- 172.21.15.18:0/2229474127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b4e5e0 0x561281b5b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2ba838a740 1 --2- 172.21.15.18:0/2229474127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 0x561281b4e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2ba838a740 1 -- 172.21.15.18:0/2229474127 >> 172.21.15.18:0/2229474127 conn(0x561281b48980 msgr2=0x561281b48d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2ba838a740 1 -- 172.21.15.18:0/2229474127 shutdown_connections 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.110+0000 7f2ba838a740 1 -- 172.21.15.18:0/2229474127 wait complete. 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2ba838a740 1 -- start start 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2ba838a740 1 Processor -- start 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2ba838a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b4dc30 0x561281b58b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2ba838a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b59080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2ba838a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b5bb60 0x561281b595c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2ba838a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561281ab6020 con 0x561281b4e5e0 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2ba838a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561281789430 con 0x561281b4dc30 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2ba838a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561281745e00 con 0x561281b5bb60 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2ba838a740 10 client.0 _ll_register_callbacks cb 0x561281a5dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2b9e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b59080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2b9effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b4dc30 0x561281b58b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2b9e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b59080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41090/0 (socket says 172.21.15.18:41090) 2024-02-17T13:01:42.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/3845089791 learned_addr learned my addr 172.21.15.18:0/3845089791 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2b9f7fe640 1 --2- 172.21.15.18:0/3845089791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b5bb60 0x561281b595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/3845089791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b5bb60 msgr2=0x561281b595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/3845089791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b5bb60 0x561281b595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/3845089791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b4dc30 msgr2=0x561281b58b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.111+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/3845089791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b4dc30 0x561281b58b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/3845089791 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba01a47d0 con 0x561281b4e5e0 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/3845089791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b59080 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f2b940d1ef0 tx=0x7f2b94097440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30dab60a0a89fd6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2b82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2b82ff5640 1 -- 172.21.15.18:0/3845089791 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9409a500 con 0x561281b4e5e0 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2b82ff5640 1 -- 172.21.15.18:0/3845089791 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2b9409b2a0 con 0x561281b4e5e0 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2ba838a740 1 -- 172.21.15.18:0/3845089791 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612817b57e0 con 0x561281b4e5e0 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2b82ff5640 1 -- 172.21.15.18:0/3845089791 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b9409aba0 con 0x561281b4e5e0 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2ba838a740 10 client.19749 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2ba838a740 1 -- 172.21.15.18:0/3845089791 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x561281ab46a0 con 0x561281b4e5e0 2024-02-17T13:01:42.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2b82ff5640 1 -- 172.21.15.18:0/3845089791 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5612817b57e0 con 0x561281b4e5e0 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2b82ff5640 1 -- 172.21.15.18:0/3845089791 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2b94016020 con 0x561281b4e5e0 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2ba838a740 10 client.19749 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2ba838a740 10 client.19749 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2ba838a740 20 client.19749 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2ba838a740 1 --2- 172.21.15.18:0/3845089791 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x561281b6b970 0x561281b6dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2ba838a740 4 client.19749 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.112+0000 7f2ba838a740 1 -- 172.21.15.18:0/3845089791 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612817f2720 con 0x561281b6b970 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.113+0000 7f2b9effd640 1 --2- 172.21.15.18:0/3845089791 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x561281b6b970 0x561281b6dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.113+0000 7f2b9effd640 1 --2- 172.21.15.18:0/3845089791 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x561281b6b970 0x561281b6dd60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8db9cb720ad281df server_cookie=bae92c61591f8599 in_seq=0 out_seq=0 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.113+0000 7f2b82ff5640 10 client.19749 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2b82ff5640 1 -- 172.21.15.18:0/3845089791 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612817f2720 con 0x561281b6b970 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2b82ff5640 10 client.19749 handle_command_reply: tid=0 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2ba838a740 1 client.19749 shutdown 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2ba838a740 1 -- 172.21.15.18:0/3845089791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 msgr2=0x561281b59080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2ba838a740 1 --2- 172.21.15.18:0/3845089791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b59080 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f2b940d1ef0 tx=0x7f2b94097440 comp rx=0 tx=0).stop 2024-02-17T13:01:42.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2ba838a740 1 -- 172.21.15.18:0/3845089791 shutdown_connections 2024-02-17T13:01:42.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2ba838a740 1 --2- 172.21.15.18:0/3845089791 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x561281b6b970 0x561281b6dd60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2ba838a740 1 --2- 172.21.15.18:0/3845089791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b5bb60 0x561281b595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2ba838a740 1 --2- 172.21.15.18:0/3845089791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b59080 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2b82ff5640 0 client.19749 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:01:42.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2ba838a740 1 --2- 172.21.15.18:0/3845089791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b4dc30 0x561281b58b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2ba838a740 1 -- 172.21.15.18:0/3845089791 >> 172.21.15.18:0/3845089791 conn(0x561281b48980 msgr2=0x561281b4a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2ba838a740 1 -- 172.21.15.18:0/3845089791 shutdown_connections 2024-02-17T13:01:42.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2ba838a740 1 -- 172.21.15.18:0/3845089791 wait complete. 2024-02-17T13:01:42.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.114+0000 7f2ba838a740 20 client.19749 trim_cache size 0 max 16384 2024-02-17T13:01:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2ba838a740 1 Processor -- start 2024-02-17T13:01:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2ba838a740 1 -- start start 2024-02-17T13:01:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2ba838a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 0x561281b593e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2ba838a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b47a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2ba838a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b5bb60 0x561281b62960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2ba838a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561281a3b130 con 0x561281b4e5e0 2024-02-17T13:01:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2ba838a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561281b3aa70 con 0x561281b5bb60 2024-02-17T13:01:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2ba838a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561281b68970 con 0x561281b4dc30 2024-02-17T13:01:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2b9e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b47a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2b9e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b47a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41100/0 (socket says 172.21.15.18:41100) 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/280401261 learned_addr learned my addr 172.21.15.18:0/280401261 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/280401261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 msgr2=0x561281b593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/280401261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 0x561281b593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/280401261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b5bb60 msgr2=0x561281b62960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2b9f7fe640 1 --2- 172.21.15.18:0/280401261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b5bb60 0x561281b62960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/280401261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b5bb60 0x561281b62960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/280401261 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561281a50c20 con 0x561281b4e5e0 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2b9f7fe640 1 --2- 172.21.15.18:0/280401261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b5bb60 0x561281b62960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.126+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/280401261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b47a00 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f2b94007590 tx=0x7f2b94096f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852e088a0daf6a0b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2b84ff9640 1 -- 172.21.15.18:0/280401261 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9409aba0 con 0x561281b4e5e0 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2b84ff9640 1 -- 172.21.15.18:0/280401261 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b940a3340 con 0x561281b4e5e0 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2b84ff9640 1 -- 172.21.15.18:0/280401261 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94099cc0 con 0x561281b4e5e0 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2ba838a740 1 -- 172.21.15.18:0/280401261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 msgr2=0x561281b47a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2ba838a740 1 --2- 172.21.15.18:0/280401261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b47a00 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f2b94007590 tx=0x7f2b94096f70 comp rx=0 tx=0).stop 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2ba838a740 1 -- 172.21.15.18:0/280401261 shutdown_connections 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2ba838a740 1 --2- 172.21.15.18:0/280401261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b5bb60 0x561281b62960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2ba838a740 1 --2- 172.21.15.18:0/280401261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b47a00 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2ba838a740 1 --2- 172.21.15.18:0/280401261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 0x561281b593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2ba838a740 1 -- 172.21.15.18:0/280401261 >> 172.21.15.18:0/280401261 conn(0x561281b48980 msgr2=0x561281b472f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2ba838a740 1 -- 172.21.15.18:0/280401261 shutdown_connections 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2ba838a740 1 -- 172.21.15.18:0/280401261 wait complete. 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2ba838a740 1 -- start start 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.127+0000 7f2ba838a740 1 Processor -- start 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2ba838a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 0x561281b61620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2ba838a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b61b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2ba838a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b5bb60 0x561281b620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2ba838a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x561281bc1fe0 con 0x561281b4e5e0 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2ba838a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561281bc2150 con 0x561281b5bb60 2024-02-17T13:01:42.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2ba838a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x561281bc2310 con 0x561281b4dc30 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2ba838a740 10 client.0 _ll_register_callbacks cb 0x7f2b940d1b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b9e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b61b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b9e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b61b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41108/0 (socket says 172.21.15.18:41108) 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/2875329028 learned_addr learned my addr 172.21.15.18:0/2875329028 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b9effd640 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 0x561281b61620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b9f7fe640 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b5bb60 0x561281b620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/2875329028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 msgr2=0x561281b61620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 0x561281b61620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/2875329028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b5bb60 msgr2=0x561281b620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b5bb60 0x561281b620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b9e7fc640 1 -- 172.21.15.18:0/2875329028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561281ab46a0 con 0x561281b4e5e0 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b9effd640 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 0x561281b61620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b9e7fc640 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b61b60 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f2b9409a3a0 tx=0x7f2b9409a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b96135ac54c597 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b82ff5640 1 -- 172.21.15.18:0/2875329028 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9409aba0 con 0x561281b4e5e0 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b82ff5640 1 -- 172.21.15.18:0/2875329028 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2b9409b2a0 con 0x561281b4e5e0 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2ba838a740 1 -- 172.21.15.18:0/2875329028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b940d1d80 con 0x561281b4e5e0 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.128+0000 7f2b82ff5640 1 -- 172.21.15.18:0/2875329028 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b94096ca0 con 0x561281b4e5e0 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2ba838a740 10 client.19755 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2ba838a740 1 -- 172.21.15.18:0/2875329028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x561281a768c0 con 0x561281b4e5e0 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2b82ff5640 1 -- 172.21.15.18:0/2875329028 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b940d1d80 con 0x561281b4e5e0 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2b82ff5640 1 -- 172.21.15.18:0/2875329028 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2b940ba490 con 0x561281b4e5e0 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2ba838a740 10 client.19755 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2ba838a740 10 client.19755 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2ba838a740 20 client.19755 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2ba838a740 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x561281bc35d0 0x561281bc59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2ba838a740 4 client.19755 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2ba838a740 1 -- 172.21.15.18:0/2875329028 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561281841890 con 0x561281bc35d0 2024-02-17T13:01:42.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2b9effd640 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x561281bc35d0 0x561281bc59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2b82ff5640 10 client.19755 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.129+0000 7f2b9effd640 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x561281bc35d0 0x561281bc59c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2345a034531b9282 server_cookie=73446b16d4957821 in_seq=0 out_seq=0 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2b82ff5640 1 -- 172.21.15.18:0/2875329028 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561281841890 con 0x561281bc35d0 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2b82ff5640 10 client.19755 handle_command_reply: tid=0 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2ba838a740 1 client.19755 shutdown 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2ba838a740 1 -- 172.21.15.18:0/2875329028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 msgr2=0x561281b61b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2ba838a740 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b61b60 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f2b9409a3a0 tx=0x7f2b9409a450 comp rx=0 tx=0).stop 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2ba838a740 1 -- 172.21.15.18:0/2875329028 shutdown_connections 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2ba838a740 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x561281bc35d0 0x561281bc59c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2ba838a740 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561281b5bb60 0x561281b620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2b82ff5640 0 client.19755 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2ba838a740 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561281b4e5e0 0x561281b61b60 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2ba838a740 1 --2- 172.21.15.18:0/2875329028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561281b4dc30 0x561281b61620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2ba838a740 1 -- 172.21.15.18:0/2875329028 >> 172.21.15.18:0/2875329028 conn(0x561281b48980 msgr2=0x561281b47b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2ba838a740 1 -- 172.21.15.18:0/2875329028 shutdown_connections 2024-02-17T13:01:42.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2ba838a740 1 -- 172.21.15.18:0/2875329028 wait complete. 2024-02-17T13:01:42.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.130+0000 7f2ba838a740 20 client.19755 trim_cache size 0 max 16384 2024-02-17T13:01:42.127 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:01:42.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.131+0000 7f2ba57c2640 1 -- 172.21.15.18:0/2849668429 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b880a2230 msgr2=0x7f2b880a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.131+0000 7f2ba57c2640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b880a2230 0x7f2b880a46f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b90006010 tx=0x7f2b9002c0f0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.131+0000 7f2ba57c2640 1 -- 172.21.15.18:0/2849668429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ba0105330 msgr2=0x7f2ba019d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.131+0000 7f2ba57c2640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ba0105330 0x7f2ba019d990 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400c970 tx=0x7f2b9400cd70 comp rx=0 tx=0).stop 2024-02-17T13:01:42.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.131+0000 7f2ba57c2640 1 -- 172.21.15.18:0/2849668429 shutdown_connections 2024-02-17T13:01:42.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.131+0000 7f2ba57c2640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2b880a2230 0x7f2b880a46f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.131+0000 7f2ba57c2640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ba0197650 0x7f2ba0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.131+0000 7f2ba57c2640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ba0105330 0x7f2ba019d990 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.131+0000 7f2ba57c2640 1 --2- 172.21.15.18:0/2849668429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ba0100260 0x7f2ba019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.131+0000 7f2ba57c2640 1 -- 172.21.15.18:0/2849668429 >> 172.21.15.18:0/2849668429 conn(0x7f2ba0077180 msgr2=0x7f2ba00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.131+0000 7f2ba57c2640 1 -- 172.21.15.18:0/2849668429 shutdown_connections 2024-02-17T13:01:42.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.131+0000 7f2ba57c2640 1 -- 172.21.15.18:0/2849668429 wait complete. 2024-02-17T13:01:42.131 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:01:42.131 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T13:01:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.246+0000 7f3174eff640 1 -- 172.21.15.18:0/2763950023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3170100cc0 msgr2=0x7f3170104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.246+0000 7f3174eff640 1 --2- 172.21.15.18:0/2763950023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3170100cc0 0x7f3170104d00 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f3158009930 tx=0x7f315802bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.246+0000 7f3174eff640 1 -- 172.21.15.18:0/2763950023 shutdown_connections 2024-02-17T13:01:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.246+0000 7f3174eff640 1 --2- 172.21.15.18:0/2763950023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31701053e0 0x7f317010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.246+0000 7f3174eff640 1 --2- 172.21.15.18:0/2763950023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3170100cc0 0x7f3170104d00 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.246+0000 7f3174eff640 1 --2- 172.21.15.18:0/2763950023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3170100310 0x7f31701006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.246+0000 7f3174eff640 1 -- 172.21.15.18:0/2763950023 >> 172.21.15.18:0/2763950023 conn(0x7f31700771e0 msgr2=0x7f31700775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.246+0000 7f3174eff640 1 -- 172.21.15.18:0/2763950023 shutdown_connections 2024-02-17T13:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.246+0000 7f3174eff640 1 -- 172.21.15.18:0/2763950023 wait complete. 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f3174eff640 1 Processor -- start 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f3174eff640 1 -- start start 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f3174eff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3170100310 0x7f3170196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f3174eff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31701053e0 0x7f3170196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f3174eff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f317019b020 0x7f317019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f3174eff640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317019ba10 con 0x7f317019b020 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f3174eff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317019bb80 con 0x7f3170100310 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f3174eff640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317019bcf0 con 0x7f31701053e0 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f316ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f317019b020 0x7f317019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f316ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f317019b020 0x7f317019b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41116/0 (socket says 172.21.15.18:41116) 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f316ed9d640 1 -- 172.21.15.18:0/2418608978 learned_addr learned my addr 172.21.15.18:0/2418608978 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f316e59c640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3170100310 0x7f3170196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.247+0000 7f3167fff640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31701053e0 0x7f3170196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.248+0000 7f316ed9d640 1 -- 172.21.15.18:0/2418608978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31701053e0 msgr2=0x7f3170196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.248+0000 7f316ed9d640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31701053e0 0x7f3170196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.248+0000 7f316ed9d640 1 -- 172.21.15.18:0/2418608978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3170100310 msgr2=0x7f3170196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.248+0000 7f316ed9d640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3170100310 0x7f3170196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.248+0000 7f316ed9d640 1 -- 172.21.15.18:0/2418608978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3158009560 con 0x7f317019b020 2024-02-17T13:01:42.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.248+0000 7f316ed9d640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f317019b020 0x7f317019b4d0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f316800cce0 tx=0x7f316800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.248+0000 7f31677fe640 1 -- 172.21.15.18:0/2418608978 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316800f450 con 0x7f317019b020 2024-02-17T13:01:42.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.248+0000 7f31677fe640 1 -- 172.21.15.18:0/2418608978 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f316800fa90 con 0x7f317019b020 2024-02-17T13:01:42.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.248+0000 7f3174eff640 1 -- 172.21.15.18:0/2418608978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31701a32d0 con 0x7f317019b020 2024-02-17T13:01:42.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.248+0000 7f31677fe640 1 -- 172.21.15.18:0/2418608978 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168018b90 con 0x7f317019b020 2024-02-17T13:01:42.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.248+0000 7f3174eff640 1 -- 172.21.15.18:0/2418608978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31701a3700 con 0x7f317019b020 2024-02-17T13:01:42.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.250+0000 7f31677fe640 1 -- 172.21.15.18:0/2418608978 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f316800fc00 con 0x7f317019b020 2024-02-17T13:01:42.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.250+0000 7f3177ba9740 1 Processor -- start 2024-02-17T13:01:42.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f3177ba9740 1 -- start start 2024-02-17T13:01:42.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f3177ba9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e2f10 0x55fd652e32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f31677fe640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f315c0a2230 0x7f315c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f3177ba9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652e3830 0x55fd652ee420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f3177ba9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652ee960 0x55fd652f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f3177ba9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6529e960 con 0x55fd652ee960 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f3177ba9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd65252310 con 0x55fd652e2f10 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f3177ba9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6524f840 con 0x55fd652e3830 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f316ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652ee960 0x55fd652f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f316e59c640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f315c0a2230 0x7f315c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f316ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652ee960 0x55fd652f0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41128/0 (socket says 172.21.15.18:41128) 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f316ed9d640 1 -- 172.21.15.18:0/3543652080 learned_addr learned my addr 172.21.15.18:0/3543652080 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f31677fe640 1 -- 172.21.15.18:0/2418608978 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f31680ce7a0 con 0x7f317019b020 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f3167fff640 1 --2- 172.21.15.18:0/3543652080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e2f10 0x55fd652e32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f3167fff640 1 --2- 172.21.15.18:0/3543652080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652e3830 0x55fd652ee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f316ed9d640 1 -- 172.21.15.18:0/3543652080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652e3830 msgr2=0x55fd652ee420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f316ed9d640 1 --2- 172.21.15.18:0/3543652080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652e3830 0x55fd652ee420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f316ed9d640 1 -- 172.21.15.18:0/3543652080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e2f10 msgr2=0x55fd652e32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f316ed9d640 1 --2- 172.21.15.18:0/3543652080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e2f10 0x55fd652e32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f316ed9d640 1 -- 172.21.15.18:0/3543652080 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd6520d060 con 0x55fd652ee960 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f316e59c640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f315c0a2230 0x7f315c0a46f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3160005fd0 tx=0x7f316000b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.251+0000 7f316ed9d640 1 --2- 172.21.15.18:0/3543652080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652ee960 0x55fd652f0d50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f31680a33c0 tx=0x7f3168018590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c6666305c2017f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.252+0000 7f3164ff9640 1 -- 172.21.15.18:0/3543652080 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31680b3e80 con 0x55fd652ee960 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.252+0000 7f3164ff9640 1 -- 172.21.15.18:0/3543652080 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f316809b850 con 0x55fd652ee960 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.252+0000 7f3164ff9640 1 -- 172.21.15.18:0/3543652080 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31680977c0 con 0x55fd652ee960 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.252+0000 7f3177ba9740 1 -- 172.21.15.18:0/3543652080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652ee960 msgr2=0x55fd652f0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.252+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3543652080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652ee960 0x55fd652f0d50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f31680a33c0 tx=0x7f3168018590 comp rx=0 tx=0).stop 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.252+0000 7f3177ba9740 1 -- 172.21.15.18:0/3543652080 shutdown_connections 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.252+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3543652080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652ee960 0x55fd652f0d50 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.252+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3543652080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652e3830 0x55fd652ee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.252+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3543652080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e2f10 0x55fd652e32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.252+0000 7f3177ba9740 1 -- 172.21.15.18:0/3543652080 >> 172.21.15.18:0/3543652080 conn(0x55fd652dd980 msgr2=0x55fd652ddd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.252+0000 7f3177ba9740 1 -- 172.21.15.18:0/3543652080 shutdown_connections 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.252+0000 7f3177ba9740 1 -- 172.21.15.18:0/3543652080 wait complete. 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f3177ba9740 1 -- start start 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f3177ba9740 1 Processor -- start 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f3177ba9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e2f10 0x55fd652eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f3177ba9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e3830 0x55fd652ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f3177ba9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652ee960 0x55fd652ec470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f3177ba9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6524b020 con 0x55fd652e2f10 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f3177ba9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd64f1e430 con 0x55fd652e3830 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f3177ba9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd64edae00 con 0x55fd652ee960 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f3177ba9740 10 client.0 _ll_register_callbacks cb 0x55fd651f2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f316e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e2f10 0x55fd652eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f316e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e2f10 0x55fd652eb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41130/0 (socket says 172.21.15.18:41130) 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f316e59c640 1 -- 172.21.15.18:0/786542532 learned_addr learned my addr 172.21.15.18:0/786542532 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f3167fff640 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e3830 0x55fd652ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f316ed9d640 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652ee960 0x55fd652ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f316e59c640 1 -- 172.21.15.18:0/786542532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652ee960 msgr2=0x55fd652ec470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f316e59c640 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652ee960 0x55fd652ec470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f316e59c640 1 -- 172.21.15.18:0/786542532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e3830 msgr2=0x55fd652ebf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f316e59c640 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e3830 0x55fd652ebf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f316e59c640 1 -- 172.21.15.18:0/786542532 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31701a3700 con 0x55fd652e2f10 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f316ed9d640 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652ee960 0x55fd652ec470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.253+0000 7f3167fff640 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e3830 0x55fd652ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f316e59c640 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e2f10 0x55fd652eb9f0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f316000b620 tx=0x7f3160004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486c1e04429f10b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3155ffb640 1 -- 172.21.15.18:0/786542532 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160005220 con 0x55fd652e2f10 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3155ffb640 1 -- 172.21.15.18:0/786542532 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3160017070 con 0x55fd652e2f10 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3177ba9740 1 -- 172.21.15.18:0/786542532 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd64f4a7e0 con 0x55fd652e2f10 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3155ffb640 1 -- 172.21.15.18:0/786542532 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3160005380 con 0x55fd652e2f10 2024-02-17T13:01:42.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3177ba9740 10 client.19767 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3177ba9740 1 -- 172.21.15.18:0/786542532 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55fd652496a0 con 0x55fd652e2f10 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3155ffb640 1 -- 172.21.15.18:0/786542532 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fd64f4a7e0 con 0x55fd652e2f10 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3155ffb640 1 -- 172.21.15.18:0/786542532 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3160024030 con 0x55fd652e2f10 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3177ba9740 10 client.19767 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3177ba9740 10 client.19767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3177ba9740 20 client.19767 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3177ba9740 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55fd652fee70 0x55fd65301290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3177ba9740 4 client.19767 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.254+0000 7f3177ba9740 1 -- 172.21.15.18:0/786542532 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd64f87720 con 0x55fd652fee70 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.255+0000 7f3167fff640 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55fd652fee70 0x55fd65301290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.255+0000 7f3167fff640 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55fd652fee70 0x55fd65301290 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e81e35ea8e66bf1 server_cookie=ace465cfda219ca9 in_seq=0 out_seq=0 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.255+0000 7f3155ffb640 10 client.19767 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3155ffb640 1 -- 172.21.15.18:0/786542532 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd64f87720 con 0x55fd652fee70 2024-02-17T13:01:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3155ffb640 10 client.19767 handle_command_reply: tid=0 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3177ba9740 1 client.19767 shutdown 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3177ba9740 1 -- 172.21.15.18:0/786542532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e2f10 msgr2=0x55fd652eb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3177ba9740 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e2f10 0x55fd652eb9f0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f316000b620 tx=0x7f3160004730 comp rx=0 tx=0).stop 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3177ba9740 1 -- 172.21.15.18:0/786542532 shutdown_connections 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3177ba9740 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55fd652fee70 0x55fd65301290 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3177ba9740 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652ee960 0x55fd652ec470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3177ba9740 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e3830 0x55fd652ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3177ba9740 1 --2- 172.21.15.18:0/786542532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e2f10 0x55fd652eb9f0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3155ffb640 0 client.19767 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3177ba9740 1 -- 172.21.15.18:0/786542532 >> 172.21.15.18:0/786542532 conn(0x55fd652dd980 msgr2=0x55fd652df6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3177ba9740 1 -- 172.21.15.18:0/786542532 shutdown_connections 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3177ba9740 1 -- 172.21.15.18:0/786542532 wait complete. 2024-02-17T13:01:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.256+0000 7f3177ba9740 20 client.19767 trim_cache size 0 max 16384 2024-02-17T13:01:42.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3177ba9740 1 Processor -- start 2024-02-17T13:01:42.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3177ba9740 1 -- start start 2024-02-17T13:01:42.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3177ba9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652e2f10 0x55fd652ec290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3177ba9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 0x55fd652dca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3177ba9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652ee960 0x55fd652f5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3177ba9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd651d0130 con 0x55fd652e3830 2024-02-17T13:01:42.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3177ba9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd64f87750 con 0x55fd652ee960 2024-02-17T13:01:42.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3177ba9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd652e3690 con 0x55fd652e2f10 2024-02-17T13:01:42.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3167fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 0x55fd652dca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3167fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 0x55fd652dca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41146/0 (socket says 172.21.15.18:41146) 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3167fff640 1 -- 172.21.15.18:0/3294090932 learned_addr learned my addr 172.21.15.18:0/3294090932 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f316e59c640 1 --2- 172.21.15.18:0/3294090932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652e2f10 0x55fd652ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3167fff640 1 -- 172.21.15.18:0/3294090932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652e2f10 msgr2=0x55fd652ec290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3167fff640 1 --2- 172.21.15.18:0/3294090932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652e2f10 0x55fd652ec290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3167fff640 1 -- 172.21.15.18:0/3294090932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652ee960 msgr2=0x55fd652f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3167fff640 1 --2- 172.21.15.18:0/3294090932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652ee960 0x55fd652f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3167fff640 1 -- 172.21.15.18:0/3294090932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd651e5c20 con 0x55fd652e3830 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3167fff640 1 --2- 172.21.15.18:0/3294090932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 0x55fd652dca00 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f3158009900 tx=0x7f3158004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3524073ecc4c2a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3157fff640 1 -- 172.21.15.18:0/3294090932 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315802c9c0 con 0x55fd652e3830 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3157fff640 1 -- 172.21.15.18:0/3294090932 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f315802cb20 con 0x55fd652e3830 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.268+0000 7f3157fff640 1 -- 172.21.15.18:0/3294090932 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158005650 con 0x55fd652e3830 2024-02-17T13:01:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.269+0000 7f3177ba9740 1 -- 172.21.15.18:0/3294090932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 msgr2=0x55fd652dca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.269+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3294090932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 0x55fd652dca00 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f3158009900 tx=0x7f3158004770 comp rx=0 tx=0).stop 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.269+0000 7f3177ba9740 1 -- 172.21.15.18:0/3294090932 shutdown_connections 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.269+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3294090932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652ee960 0x55fd652f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.269+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3294090932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 0x55fd652dca00 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.269+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3294090932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652e2f10 0x55fd652ec290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.269+0000 7f3177ba9740 1 -- 172.21.15.18:0/3294090932 >> 172.21.15.18:0/3294090932 conn(0x55fd652dd980 msgr2=0x55fd652dc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.269+0000 7f3177ba9740 1 -- 172.21.15.18:0/3294090932 shutdown_connections 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.269+0000 7f3177ba9740 1 -- 172.21.15.18:0/3294090932 wait complete. 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.269+0000 7f3177ba9740 1 -- start start 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.269+0000 7f3177ba9740 1 Processor -- start 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3177ba9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e2f10 0x55fd6534e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3177ba9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 0x55fd6534ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3177ba9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652ee960 0x55fd652f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3177ba9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd652f5340 con 0x55fd652e3830 2024-02-17T13:01:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3177ba9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd652f54b0 con 0x55fd652e2f10 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3177ba9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd652f5670 con 0x55fd652ee960 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3177ba9740 10 client.0 _ll_register_callbacks cb 0x7f3168096db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3167fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 0x55fd6534ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3167fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 0x55fd6534ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41152/0 (socket says 172.21.15.18:41152) 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3167fff640 1 -- 172.21.15.18:0/3703034990 learned_addr learned my addr 172.21.15.18:0/3703034990 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f316e59c640 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e2f10 0x55fd6534e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f316ed9d640 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652ee960 0x55fd652f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3167fff640 1 -- 172.21.15.18:0/3703034990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652ee960 msgr2=0x55fd652f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3167fff640 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652ee960 0x55fd652f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3167fff640 1 -- 172.21.15.18:0/3703034990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e2f10 msgr2=0x55fd6534e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3167fff640 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e2f10 0x55fd6534e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3167fff640 1 -- 172.21.15.18:0/3703034990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317010d7c0 con 0x55fd652e3830 2024-02-17T13:01:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3167fff640 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 0x55fd6534ed20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f3158005ab0 tx=0x7f3158005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3c262e5af09a6c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3155ffb640 1 -- 172.21.15.18:0/3703034990 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158005390 con 0x55fd652e3830 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3155ffb640 1 -- 172.21.15.18:0/3703034990 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3158055070 con 0x55fd652e3830 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3177ba9740 1 -- 172.21.15.18:0/3703034990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f316809b850 con 0x55fd652e3830 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3155ffb640 1 -- 172.21.15.18:0/3703034990 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31580054f0 con 0x55fd652e3830 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.270+0000 7f3177ba9740 10 client.19773 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.271+0000 7f3177ba9740 1 -- 172.21.15.18:0/3703034990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55fd6520b8c0 con 0x55fd652e3830 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.271+0000 7f3155ffb640 1 -- 172.21.15.18:0/3703034990 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f316809b850 con 0x55fd652e3830 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.271+0000 7f3155ffb640 1 -- 172.21.15.18:0/3703034990 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3158003b90 con 0x55fd652e3830 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.271+0000 7f3177ba9740 10 client.19773 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.271+0000 7f3177ba9740 10 client.19773 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.271+0000 7f3177ba9740 20 client.19773 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.271+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55fd65357580 0x55fd653599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.271+0000 7f3177ba9740 4 client.19773 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:01:42.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.271+0000 7f3177ba9740 1 -- 172.21.15.18:0/3703034990 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd64fd6890 con 0x55fd65357580 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.271+0000 7f316e59c640 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55fd65357580 0x55fd653599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.271+0000 7f316e59c640 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55fd65357580 0x55fd653599a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4a15cd5a928cc46 server_cookie=fa76bf06aa17a013 in_seq=0 out_seq=0 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.271+0000 7f3155ffb640 10 client.19773 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3155ffb640 1 -- 172.21.15.18:0/3703034990 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd64fd6890 con 0x55fd65357580 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3155ffb640 10 client.19773 handle_command_reply: tid=0 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3177ba9740 1 client.19773 shutdown 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3177ba9740 1 -- 172.21.15.18:0/3703034990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 msgr2=0x55fd6534ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 0x55fd6534ed20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f3158005ab0 tx=0x7f3158005900 comp rx=0 tx=0).stop 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3177ba9740 1 -- 172.21.15.18:0/3703034990 shutdown_connections 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x55fd65357580 0x55fd653599a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd652ee960 0x55fd652f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd652e3830 0x55fd6534ed20 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3155ffb640 0 client.19773 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3177ba9740 1 --2- 172.21.15.18:0/3703034990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd652e2f10 0x55fd6534e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3177ba9740 1 -- 172.21.15.18:0/3703034990 >> 172.21.15.18:0/3703034990 conn(0x55fd652dd980 msgr2=0x55fd652dcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3177ba9740 1 -- 172.21.15.18:0/3703034990 shutdown_connections 2024-02-17T13:01:42.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3177ba9740 1 -- 172.21.15.18:0/3703034990 wait complete. 2024-02-17T13:01:42.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3177ba9740 20 client.19773 trim_cache size 0 max 16384 2024-02-17T13:01:42.269 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:01:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3174eff640 1 -- 172.21.15.18:0/2418608978 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f315c0a2230 msgr2=0x7f315c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3174eff640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f315c0a2230 0x7f315c0a46f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3160005fd0 tx=0x7f316000b390 comp rx=0 tx=0).stop 2024-02-17T13:01:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3174eff640 1 -- 172.21.15.18:0/2418608978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f317019b020 msgr2=0x7f317019b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3174eff640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f317019b020 0x7f317019b4d0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f316800cce0 tx=0x7f316800a620 comp rx=0 tx=0).stop 2024-02-17T13:01:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3174eff640 1 -- 172.21.15.18:0/2418608978 shutdown_connections 2024-02-17T13:01:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3174eff640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f315c0a2230 0x7f315c0a46f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3174eff640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f317019b020 0x7f317019b4d0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3174eff640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31701053e0 0x7f3170196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3174eff640 1 --2- 172.21.15.18:0/2418608978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3170100310 0x7f3170196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3174eff640 1 -- 172.21.15.18:0/2418608978 >> 172.21.15.18:0/2418608978 conn(0x7f31700771e0 msgr2=0x7f31700fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3174eff640 1 -- 172.21.15.18:0/2418608978 shutdown_connections 2024-02-17T13:01:42.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.272+0000 7f3174eff640 1 -- 172.21.15.18:0/2418608978 wait complete. 2024-02-17T13:01:42.273 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:01:42.273 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T13:01:42.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 587763 4.87 MB/sec execute 537 sec latency 2448.542 ms 2024-02-17T13:01:42.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.388+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3903706613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a08101d40 msgr2=0x7f3a08105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.388+0000 7f3a0d3d0640 1 --2- 172.21.15.18:0/3903706613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a08101d40 0x7f3a08105d80 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009930 tx=0x7f39fc02bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:42.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.388+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3903706613 shutdown_connections 2024-02-17T13:01:42.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.388+0000 7f3a0d3d0640 1 --2- 172.21.15.18:0/3903706613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a08106460 0x7f3a0810e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.388+0000 7f3a0d3d0640 1 --2- 172.21.15.18:0/3903706613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a08101d40 0x7f3a08105d80 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.388+0000 7f3a0d3d0640 1 --2- 172.21.15.18:0/3903706613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a08101390 0x7f3a08101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.388+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3903706613 >> 172.21.15.18:0/3903706613 conn(0x7f3a080fcbb0 msgr2=0x7f3a080fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.388+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3903706613 shutdown_connections 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.388+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3903706613 wait complete. 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.389+0000 7f3a0d3d0640 1 Processor -- start 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.389+0000 7f3a0d3d0640 1 -- start start 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.389+0000 7f3a0d3d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a08101390 0x7f3a0819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.389+0000 7f3a0d3d0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a08106460 0x7f3a0819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.389+0000 7f3a0d3d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a08197640 0x7f3a08197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.389+0000 7f3a0d3d0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a08198110 con 0x7f3a08106460 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.389+0000 7f3a0d3d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a08198280 con 0x7f3a08197640 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.389+0000 7f3a0d3d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a08198440 con 0x7f3a08101390 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.389+0000 7f3a067fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a08106460 0x7f3a0819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.389+0000 7f3a067fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a08106460 0x7f3a0819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41160/0 (socket says 172.21.15.18:41160) 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.389+0000 7f3a067fc640 1 -- 172.21.15.18:0/3729464097 learned_addr learned my addr 172.21.15.18:0/3729464097 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f3a06ffd640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a08101390 0x7f3a0819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f3a077fe640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a08197640 0x7f3a08197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f3a067fc640 1 -- 172.21.15.18:0/3729464097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a08101390 msgr2=0x7f3a0819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f3a067fc640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a08101390 0x7f3a0819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f3a067fc640 1 -- 172.21.15.18:0/3729464097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a08197640 msgr2=0x7f3a08197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f3a067fc640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a08197640 0x7f3a08197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f3a067fc640 1 -- 172.21.15.18:0/3729464097 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc009560 con 0x7f3a08106460 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f3a067fc640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a08106460 0x7f3a0819dac0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009230 tx=0x7f39fc02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f39effff640 1 -- 172.21.15.18:0/3729464097 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc004350 con 0x7f3a08106460 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3729464097 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a081a4380 con 0x7f3a08106460 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f39effff640 1 -- 172.21.15.18:0/3729464097 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39fc0044b0 con 0x7f3a08106460 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f39effff640 1 -- 172.21.15.18:0/3729464097 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc003da0 con 0x7f3a08106460 2024-02-17T13:01:42.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.390+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3729464097 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a081a4860 con 0x7f3a08106460 2024-02-17T13:01:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.392+0000 7f39effff640 1 -- 172.21.15.18:0/3729464097 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f39fc004df0 con 0x7f3a08106460 2024-02-17T13:01:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a0ff9c740 1 Processor -- start 2024-02-17T13:01:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a0ff9c740 1 -- start start 2024-02-17T13:01:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a0ff9c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fabf0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a0ff9c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabf05e0 0x5601fabfd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a0ff9c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabfdb60 0x5601fabfff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a0ff9c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fabab960 con 0x5601fabefc30 2024-02-17T13:01:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a0ff9c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fab5f310 con 0x5601fabfdb60 2024-02-17T13:01:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a0ff9c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fab5c840 con 0x5601fabf05e0 2024-02-17T13:01:42.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f39effff640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f39f00a2230 0x7f39f00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a077fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fabf0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a077fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fabf0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41174/0 (socket says 172.21.15.18:41174) 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a077fe640 1 -- 172.21.15.18:0/3396369045 learned_addr learned my addr 172.21.15.18:0/3396369045 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a077fe640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f39f00a2230 0x7f39f00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a06ffd640 1 --2- 172.21.15.18:0/3396369045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabf05e0 0x5601fabfd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f39effff640 1 -- 172.21.15.18:0/3729464097 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f39fc0f0140 con 0x7f3a08106460 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a067fc640 1 --2- 172.21.15.18:0/3396369045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabfdb60 0x5601fabfff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a077fe640 1 -- 172.21.15.18:0/3396369045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabf05e0 msgr2=0x5601fabfd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a077fe640 1 --2- 172.21.15.18:0/3396369045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabf05e0 0x5601fabfd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a077fe640 1 -- 172.21.15.18:0/3396369045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabfdb60 msgr2=0x5601fabfff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a077fe640 1 --2- 172.21.15.18:0/3396369045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabfdb60 0x5601fabfff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.393+0000 7f3a077fe640 1 -- 172.21.15.18:0/3396369045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601fab1a060 con 0x5601fabefc30 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.394+0000 7f3a067fc640 1 --2- 172.21.15.18:0/3396369045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabfdb60 0x5601fabfff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.394+0000 7f3a077fe640 1 --2- 172.21.15.18:0/3396369045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fabf0010 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3a000129f0 tx=0x7f3a00012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c770758432a7e138 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.394+0000 7f3a077fe640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f39f00a2230 0x7f39f00a46f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3a0001c000 tx=0x7f3a00020000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.394+0000 7f39ed7fa640 1 -- 172.21.15.18:0/3396369045 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a000120d0 con 0x5601fabefc30 2024-02-17T13:01:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.394+0000 7f39ed7fa640 1 -- 172.21.15.18:0/3396369045 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a0001fe60 con 0x5601fabefc30 2024-02-17T13:01:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.394+0000 7f39ed7fa640 1 -- 172.21.15.18:0/3396369045 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00004230 con 0x5601fabefc30 2024-02-17T13:01:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/3396369045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 msgr2=0x5601fabf0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/3396369045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fabf0010 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3a000129f0 tx=0x7f3a00012eb0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/3396369045 shutdown_connections 2024-02-17T13:01:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/3396369045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabfdb60 0x5601fabfff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/3396369045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabf05e0 0x5601fabfd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/3396369045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fabf0010 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/3396369045 >> 172.21.15.18:0/3396369045 conn(0x5601fabea980 msgr2=0x5601fabead60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/3396369045 shutdown_connections 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/3396369045 wait complete. 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 -- start start 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 Processor -- start 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fabfabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabf05e0 0x5601fabfb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabfdb60 0x5601fabfb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fab58020 con 0x5601fabefc30 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fa82b430 con 0x5601fabf05e0 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.395+0000 7f3a0ff9c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fa7e7e00 con 0x5601fabfdb60 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a0ff9c740 10 client.0 _ll_register_callbacks cb 0x5601faaffcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a06ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fabfabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a06ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fabfabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41176/0 (socket says 172.21.15.18:41176) 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a06ffd640 1 -- 172.21.15.18:0/2735134691 learned_addr learned my addr 172.21.15.18:0/2735134691 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a067fc640 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabf05e0 0x5601fabfb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a077fe640 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabfdb60 0x5601fabfb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a06ffd640 1 -- 172.21.15.18:0/2735134691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabfdb60 msgr2=0x5601fabfb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a06ffd640 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabfdb60 0x5601fabfb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a06ffd640 1 -- 172.21.15.18:0/2735134691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabf05e0 msgr2=0x5601fabfb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a06ffd640 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabf05e0 0x5601fabfb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a06ffd640 1 -- 172.21.15.18:0/2735134691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a000125d0 con 0x5601fabefc30 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a06ffd640 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fabfabb0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f39f8012770 tx=0x7f39f8012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0f76db3c4c11f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f39eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f39eaff5640 1 -- 172.21.15.18:0/2735134691 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8017070 con 0x5601fabefc30 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2735134691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601fa8577e0 con 0x5601fabefc30 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f39eaff5640 1 -- 172.21.15.18:0/2735134691 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f39f800f920 con 0x5601fabefc30 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f39eaff5640 1 -- 172.21.15.18:0/2735134691 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f800fae0 con 0x5601fabefc30 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a0ff9c740 10 client.19785 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.396+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2735134691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5601fab566a0 con 0x5601fabefc30 2024-02-17T13:01:42.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.397+0000 7f39eaff5640 1 -- 172.21.15.18:0/2735134691 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5601fa8577e0 con 0x5601fabefc30 2024-02-17T13:01:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.397+0000 7f39eaff5640 1 -- 172.21.15.18:0/2735134691 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39f801c420 con 0x5601fabefc30 2024-02-17T13:01:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.397+0000 7f3a0ff9c740 10 client.19785 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.397+0000 7f3a0ff9c740 10 client.19785 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:01:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.397+0000 7f3a0ff9c740 20 client.19785 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.397+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5601fac0df70 0x5601fac10390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.397+0000 7f3a0ff9c740 4 client.19785 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.397+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2735134691 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601fa894720 con 0x5601fac0df70 2024-02-17T13:01:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.397+0000 7f3a067fc640 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5601fac0df70 0x5601fac10390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f3a067fc640 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5601fac0df70 0x5601fac10390 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=405896ac6eb6da51 server_cookie=33ec1443d09d6037 in_seq=0 out_seq=0 2024-02-17T13:01:42.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f39eaff5640 10 client.19785 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f39eaff5640 1 -- 172.21.15.18:0/2735134691 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5601fa894720 con 0x5601fac0df70 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f39eaff5640 10 client.19785 handle_command_reply: tid=0 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f3a0ff9c740 1 client.19785 shutdown 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2735134691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 msgr2=0x5601fabfabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fabfabb0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f39f8012770 tx=0x7f39f8012c30 comp rx=0 tx=0).stop 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2735134691 shutdown_connections 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5601fac0df70 0x5601fac10390 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabfdb60 0x5601fabfb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabf05e0 0x5601fabfb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f39eaff5640 0 client.19785 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/2735134691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fabfabb0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2735134691 >> 172.21.15.18:0/2735134691 conn(0x5601fabea980 msgr2=0x5601fabec6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2735134691 shutdown_connections 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.398+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2735134691 wait complete. 2024-02-17T13:01:42.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.399+0000 7f3a0ff9c740 20 client.19785 trim_cache size 0 max 16384 2024-02-17T13:01:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.410+0000 7f3a0ff9c740 1 Processor -- start 2024-02-17T13:01:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.410+0000 7f3a0ff9c740 1 -- start start 2024-02-17T13:01:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.410+0000 7f3a0ff9c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabefc30 0x5601fabfb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a0ff9c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabf05e0 0x5601fabf7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a0ff9c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabfdb60 0x5601fabf8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a0ff9c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601faadd130 con 0x5601fabfdb60 2024-02-17T13:01:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a0ff9c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fabf67f0 con 0x5601fabefc30 2024-02-17T13:01:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a0ff9c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fabef340 con 0x5601fabf05e0 2024-02-17T13:01:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a077fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabfdb60 0x5601fabf8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a077fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabfdb60 0x5601fabf8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41182/0 (socket says 172.21.15.18:41182) 2024-02-17T13:01:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a077fe640 1 -- 172.21.15.18:0/4093610036 learned_addr learned my addr 172.21.15.18:0/4093610036 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a06ffd640 1 --2- 172.21.15.18:0/4093610036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabefc30 0x5601fabfb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a067fc640 1 --2- 172.21.15.18:0/4093610036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabf05e0 0x5601fabf7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a077fe640 1 -- 172.21.15.18:0/4093610036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabf05e0 msgr2=0x5601fabf7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a077fe640 1 --2- 172.21.15.18:0/4093610036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabf05e0 0x5601fabf7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a077fe640 1 -- 172.21.15.18:0/4093610036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabefc30 msgr2=0x5601fabfb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a077fe640 1 --2- 172.21.15.18:0/4093610036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabefc30 0x5601fabfb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a077fe640 1 -- 172.21.15.18:0/4093610036 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601faaf2c20 con 0x5601fabfdb60 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f3a077fe640 1 --2- 172.21.15.18:0/4093610036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabfdb60 0x5601fabf8390 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f3a00011000 tx=0x7f3a00009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0329b8ad48117 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f39ecff9640 1 -- 172.21.15.18:0/4093610036 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00003970 con 0x5601fabfdb60 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f39ecff9640 1 -- 172.21.15.18:0/4093610036 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a00012430 con 0x5601fabfdb60 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.411+0000 7f39ecff9640 1 -- 172.21.15.18:0/4093610036 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00005420 con 0x5601fabfdb60 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.412+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/4093610036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabfdb60 msgr2=0x5601fabf8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.412+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/4093610036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabfdb60 0x5601fabf8390 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f3a00011000 tx=0x7f3a00009480 comp rx=0 tx=0).stop 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.412+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/4093610036 shutdown_connections 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.412+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/4093610036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabfdb60 0x5601fabf8390 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.412+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/4093610036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabf05e0 0x5601fabf7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.412+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/4093610036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabefc30 0x5601fabfb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.412+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/4093610036 >> 172.21.15.18:0/4093610036 conn(0x5601fabea980 msgr2=0x5601fac14d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.412+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/4093610036 shutdown_connections 2024-02-17T13:01:42.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.412+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/4093610036 wait complete. 2024-02-17T13:01:42.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.412+0000 7f3a0ff9c740 1 -- start start 2024-02-17T13:01:42.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a0ff9c740 1 Processor -- start 2024-02-17T13:01:42.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a0ff9c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fac5f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a0ff9c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabf05e0 0x5601fac5f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a0ff9c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabfdb60 0x5601fac04370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a0ff9c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fac04a40 con 0x5601fabefc30 2024-02-17T13:01:42.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a0ff9c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fac04bb0 con 0x5601fabf05e0 2024-02-17T13:01:42.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a0ff9c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fac04d70 con 0x5601fabfdb60 2024-02-17T13:01:42.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a0ff9c740 10 client.0 _ll_register_callbacks cb 0x7f3a00009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:42.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a06ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fac5f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a06ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fac5f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41190/0 (socket says 172.21.15.18:41190) 2024-02-17T13:01:42.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a06ffd640 1 -- 172.21.15.18:0/2416644324 learned_addr learned my addr 172.21.15.18:0/2416644324 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a077fe640 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabfdb60 0x5601fac04370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a067fc640 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabf05e0 0x5601fac5f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a06ffd640 1 -- 172.21.15.18:0/2416644324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabfdb60 msgr2=0x5601fac04370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a06ffd640 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabfdb60 0x5601fac04370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a06ffd640 1 -- 172.21.15.18:0/2416644324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabf05e0 msgr2=0x5601fac5f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a06ffd640 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabf05e0 0x5601fac5f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a06ffd640 1 -- 172.21.15.18:0/2416644324 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601fab1a060 con 0x5601fabefc30 2024-02-17T13:01:42.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f3a06ffd640 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fac5f060 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f39f80059f0 tx=0x7f39f8009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f371638764d2091 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.413+0000 7f39eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:42.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f39eaff5640 1 -- 172.21.15.18:0/2416644324 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8017040 con 0x5601fabefc30 2024-02-17T13:01:42.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f39eaff5640 1 -- 172.21.15.18:0/2416644324 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f39f8000e40 con 0x5601fabefc30 2024-02-17T13:01:42.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2416644324 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a0001fe60 con 0x5601fabefc30 2024-02-17T13:01:42.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f39eaff5640 1 -- 172.21.15.18:0/2416644324 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f80098c0 con 0x5601fabefc30 2024-02-17T13:01:42.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f3a0ff9c740 10 client.19791 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:42.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2416644324 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5601fab188c0 con 0x5601fabefc30 2024-02-17T13:01:42.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f39eaff5640 1 -- 172.21.15.18:0/2416644324 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a0001fe60 con 0x5601fabefc30 2024-02-17T13:01:42.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f39eaff5640 1 -- 172.21.15.18:0/2416644324 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39f8025460 con 0x5601fabefc30 2024-02-17T13:01:42.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f3a0ff9c740 10 client.19791 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:42.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f3a0ff9c740 10 client.19791 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:01:42.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f3a0ff9c740 20 client.19791 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:42.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5601fac66ee0 0x5601fac69300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f3a0ff9c740 4 client.19791 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:01:42.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.414+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2416644324 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5601fa8e3890 con 0x5601fac66ee0 2024-02-17T13:01:42.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a067fc640 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5601fac66ee0 0x5601fac69300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a067fc640 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5601fac66ee0 0x5601fac69300 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3b0b7021ebe5635 server_cookie=a54a1b4e33678548 in_seq=0 out_seq=0 2024-02-17T13:01:42.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f39eaff5640 10 client.19791 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:01:42.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f39eaff5640 1 -- 172.21.15.18:0/2416644324 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5601fa8e3890 con 0x5601fac66ee0 2024-02-17T13:01:42.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f39eaff5640 10 client.19791 handle_command_reply: tid=0 2024-02-17T13:01:42.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a0ff9c740 1 client.19791 shutdown 2024-02-17T13:01:42.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2416644324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 msgr2=0x5601fac5f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fac5f060 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f39f80059f0 tx=0x7f39f8009e70 comp rx=0 tx=0).stop 2024-02-17T13:01:42.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2416644324 shutdown_connections 2024-02-17T13:01:42.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5601fac66ee0 0x5601fac69300 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5601fabfdb60 0x5601fac04370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601fabf05e0 0x5601fac5f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f39eaff5640 0 client.19791 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:01:42.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a0ff9c740 1 --2- 172.21.15.18:0/2416644324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5601fabefc30 0x5601fac5f060 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2416644324 >> 172.21.15.18:0/2416644324 conn(0x5601fabea980 msgr2=0x5601fabe92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2416644324 shutdown_connections 2024-02-17T13:01:42.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.415+0000 7f3a0ff9c740 1 -- 172.21.15.18:0/2416644324 wait complete. 2024-02-17T13:01:42.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0ff9c740 20 client.19791 trim_cache size 0 max 16384 2024-02-17T13:01:42.414 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:01:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3729464097 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f39f00a2230 msgr2=0x7f39f00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0d3d0640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f39f00a2230 0x7f39f00a46f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3a0001c000 tx=0x7f3a00020000 comp rx=0 tx=0).stop 2024-02-17T13:01:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3729464097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a08106460 msgr2=0x7f3a0819dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0d3d0640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a08106460 0x7f3a0819dac0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009230 tx=0x7f39fc02c680 comp rx=0 tx=0).stop 2024-02-17T13:01:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3729464097 shutdown_connections 2024-02-17T13:01:42.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0d3d0640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f39f00a2230 0x7f39f00a46f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0d3d0640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a08197640 0x7f3a08197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0d3d0640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a08106460 0x7f3a0819dac0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0d3d0640 1 --2- 172.21.15.18:0/3729464097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a08101390 0x7f3a0819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3729464097 >> 172.21.15.18:0/3729464097 conn(0x7f3a080fcbb0 msgr2=0x7f3a080fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3729464097 shutdown_connections 2024-02-17T13:01:42.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.416+0000 7f3a0d3d0640 1 -- 172.21.15.18:0/3729464097 wait complete. 2024-02-17T13:01:42.416 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:42 smithi018 ceph-mon[32625]: pgmap v370: 97 pgs: 97 active+clean; 232 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-02-17T13:01:42.417 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:42 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1135504014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:01:42.417 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:01:42.418 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T13:01:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.446+0000 7f859d439640 1 -- 172.21.15.18:0/1628243123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85981013c0 msgr2=0x7f85981017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.446+0000 7f859d439640 1 --2- 172.21.15.18:0/1628243123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85981013c0 0x7f85981017a0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8590009930 tx=0x7f859002bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.446+0000 7f859d439640 1 -- 172.21.15.18:0/1628243123 shutdown_connections 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.446+0000 7f859d439640 1 --2- 172.21.15.18:0/1628243123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8598106490 0x7f859810e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.446+0000 7f859d439640 1 --2- 172.21.15.18:0/1628243123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8598101d70 0x7f8598105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.446+0000 7f859d439640 1 --2- 172.21.15.18:0/1628243123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85981013c0 0x7f85981017a0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.446+0000 7f859d439640 1 -- 172.21.15.18:0/1628243123 >> 172.21.15.18:0/1628243123 conn(0x7f85980fcc60 msgr2=0x7f85980ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.447+0000 7f859d439640 1 -- 172.21.15.18:0/1628243123 shutdown_connections 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.447+0000 7f859d439640 1 -- 172.21.15.18:0/1628243123 wait complete. 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.447+0000 7f859d439640 1 Processor -- start 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.447+0000 7f859d439640 1 -- start start 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.447+0000 7f859d439640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85981013c0 0x7f859819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.447+0000 7f859d439640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8598106490 0x7f859819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.447+0000 7f859d439640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85981975e0 0x7f8598197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.447+0000 7f859d439640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8598197fd0 con 0x7f8598106490 2024-02-17T13:01:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.447+0000 7f859d439640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8598198140 con 0x7f85981013c0 2024-02-17T13:01:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.447+0000 7f859d439640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8598198300 con 0x7f85981975e0 2024-02-17T13:01:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f85967fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8598106490 0x7f859819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f85967fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8598106490 0x7f859819d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41212/0 (socket says 172.21.15.18:41212) 2024-02-17T13:01:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f85967fc640 1 -- 172.21.15.18:0/118222147 learned_addr learned my addr 172.21.15.18:0/118222147 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f8596ffd640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85981013c0 0x7f859819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f85977fe640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85981975e0 0x7f8598197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f85967fc640 1 -- 172.21.15.18:0/118222147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85981975e0 msgr2=0x7f8598197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f85967fc640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85981975e0 0x7f8598197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f85967fc640 1 -- 172.21.15.18:0/118222147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85981013c0 msgr2=0x7f859819d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f85967fc640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85981013c0 0x7f859819d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f85967fc640 1 -- 172.21.15.18:0/118222147 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8590009560 con 0x7f8598106490 2024-02-17T13:01:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f85967fc640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8598106490 0x7f859819d920 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f858c00e9c0 tx=0x7f858c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f857ffff640 1 -- 172.21.15.18:0/118222147 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c009700 con 0x7f8598106490 2024-02-17T13:01:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f857ffff640 1 -- 172.21.15.18:0/118222147 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f858c00cd80 con 0x7f8598106490 2024-02-17T13:01:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f859d439640 1 -- 172.21.15.18:0/118222147 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85981a42a0 con 0x7f8598106490 2024-02-17T13:01:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f857ffff640 1 -- 172.21.15.18:0/118222147 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c010430 con 0x7f8598106490 2024-02-17T13:01:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.448+0000 7f859d439640 1 -- 172.21.15.18:0/118222147 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85981a47b0 con 0x7f8598106490 2024-02-17T13:01:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f857ffff640 1 -- 172.21.15.18:0/118222147 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f858c005000 con 0x7f8598106490 2024-02-17T13:01:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f85a000a740 1 Processor -- start 2024-02-17T13:01:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f85a000a740 1 -- start start 2024-02-17T13:01:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f857ffff640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f85800a22d0 0x7f85800a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f85a000a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610afad0 0x558c610ae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f85a000a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610aed60 0x558c610af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f85a000a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610b82e0 0x558c610ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f85a000a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c6101d9f0 con 0x558c610afad0 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f85a000a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c6101af20 con 0x558c610aed60 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f85a000a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c61016700 con 0x558c610b82e0 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f857ffff640 1 -- 172.21.15.18:0/118222147 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f858c014070 con 0x7f8598106490 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f85977fe640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f85800a22d0 0x7f85800a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f8596ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610afad0 0x558c610ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f8596ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610afad0 0x558c610ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41228/0 (socket says 172.21.15.18:41228) 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f8596ffd640 1 -- 172.21.15.18:0/3131417066 learned_addr learned my addr 172.21.15.18:0/3131417066 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.451+0000 7f85967fc640 1 --2- 172.21.15.18:0/3131417066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610aed60 0x558c610af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.452+0000 7f85977fe640 1 --2- 172.21.15.18:0/3131417066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610b82e0 0x558c610ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.452+0000 7f8596ffd640 1 -- 172.21.15.18:0/3131417066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610b82e0 msgr2=0x558c610ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.452+0000 7f8596ffd640 1 --2- 172.21.15.18:0/3131417066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610b82e0 0x558c610ba6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.452+0000 7f8596ffd640 1 -- 172.21.15.18:0/3131417066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610aed60 msgr2=0x558c610af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.452+0000 7f8596ffd640 1 --2- 172.21.15.18:0/3131417066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610aed60 0x558c610af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.452+0000 7f8596ffd640 1 -- 172.21.15.18:0/3131417066 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c60fac7b0 con 0x558c610afad0 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.452+0000 7f85977fe640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f85800a22d0 0x7f85800a4790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f85880092f0 tx=0x7f8588014a70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.452+0000 7f8596ffd640 1 --2- 172.21.15.18:0/3131417066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610afad0 0x558c610ae820 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8590036040 tx=0x7f859002c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e2b1a5bc7217d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.452+0000 7f857d7fa640 1 -- 172.21.15.18:0/3131417066 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590004880 con 0x558c610afad0 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.452+0000 7f857d7fa640 1 -- 172.21.15.18:0/3131417066 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85900049e0 con 0x558c610afad0 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.452+0000 7f857d7fa640 1 -- 172.21.15.18:0/3131417066 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590005150 con 0x558c610afad0 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 -- 172.21.15.18:0/3131417066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610afad0 msgr2=0x558c610ae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 --2- 172.21.15.18:0/3131417066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610afad0 0x558c610ae820 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8590036040 tx=0x7f859002c680 comp rx=0 tx=0).stop 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 -- 172.21.15.18:0/3131417066 shutdown_connections 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 --2- 172.21.15.18:0/3131417066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610b82e0 0x558c610ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 --2- 172.21.15.18:0/3131417066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610aed60 0x558c610af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 --2- 172.21.15.18:0/3131417066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610afad0 0x558c610ae820 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 -- 172.21.15.18:0/3131417066 >> 172.21.15.18:0/3131417066 conn(0x558c61076550 msgr2=0x558c610a9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 -- 172.21.15.18:0/3131417066 shutdown_connections 2024-02-17T13:01:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 -- 172.21.15.18:0/3131417066 wait complete. 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 -- start start 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 Processor -- start 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610aed60 0x558c610b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.453+0000 7f85a000a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 0x558c610b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f85a000a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610b82e0 0x558c610b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f85a000a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c60fbbbf0 con 0x558c610aed60 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f85a000a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c60cea250 con 0x558c610afad0 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f85a000a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c60f9c200 con 0x558c610b82e0 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f85a000a740 10 client.0 _ll_register_callbacks cb 0x558c6100ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f8596ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610aed60 0x558c610b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f8596ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610aed60 0x558c610b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41238/0 (socket says 172.21.15.18:41238) 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f8596ffd640 1 -- 172.21.15.18:0/79699682 learned_addr learned my addr 172.21.15.18:0/79699682 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f85967fc640 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 0x558c610b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f85977fe640 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610b82e0 0x558c610b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f8596ffd640 1 -- 172.21.15.18:0/79699682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610b82e0 msgr2=0x558c610b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f8596ffd640 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610b82e0 0x558c610b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f8596ffd640 1 -- 172.21.15.18:0/79699682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 msgr2=0x558c610b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f8596ffd640 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 0x558c610b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f8596ffd640 1 -- 172.21.15.18:0/79699682 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859810e870 con 0x558c610aed60 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f85977fe640 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610b82e0 0x558c610b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f8596ffd640 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610aed60 0x558c610b5270 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f85900051d0 tx=0x7f859002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6e2e1c4d73a342 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f857aff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f857aff5640 1 -- 172.21.15.18:0/79699682 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590004880 con 0x558c610aed60 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f857aff5640 1 -- 172.21.15.18:0/79699682 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f859004d070 con 0x558c610aed60 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f85a000a740 1 -- 172.21.15.18:0/79699682 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c60d16600 con 0x558c610aed60 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.454+0000 7f857aff5640 1 -- 172.21.15.18:0/79699682 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f859002cbe0 con 0x558c610aed60 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.455+0000 7f85a000a740 10 client.19803 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.455+0000 7f85a000a740 1 -- 172.21.15.18:0/79699682 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x558c61014d80 con 0x558c610aed60 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.455+0000 7f857aff5640 1 -- 172.21.15.18:0/79699682 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558c60d16600 con 0x558c610aed60 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.455+0000 7f857aff5640 1 -- 172.21.15.18:0/79699682 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f859003c050 con 0x558c610aed60 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.455+0000 7f85a000a740 10 client.19803 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.455+0000 7f85a000a740 10 client.19803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.455+0000 7f85a000a740 20 client.19803 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.455+0000 7f85a000a740 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558c610cc5e0 0x558c610ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.455+0000 7f85a000a740 4 client.19803 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.455+0000 7f85a000a740 1 -- 172.21.15.18:0/79699682 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c60d671b0 con 0x558c610cc5e0 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.455+0000 7f85967fc640 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558c610cc5e0 0x558c610ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f857aff5640 10 client.19803 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f85967fc640 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558c610cc5e0 0x558c610ce9d0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74befd63fae12ae7 server_cookie=a4951359dc011a61 in_seq=0 out_seq=0 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f857aff5640 1 -- 172.21.15.18:0/79699682 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c60d671b0 con 0x558c610cc5e0 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f857aff5640 10 client.19803 handle_command_reply: tid=0 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f85a000a740 1 client.19803 shutdown 2024-02-17T13:01:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f85a000a740 1 -- 172.21.15.18:0/79699682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610aed60 msgr2=0x558c610b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f85a000a740 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610aed60 0x558c610b5270 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f85900051d0 tx=0x7f859002bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f85a000a740 1 -- 172.21.15.18:0/79699682 shutdown_connections 2024-02-17T13:01:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f85a000a740 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558c610cc5e0 0x558c610ce9d0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f85a000a740 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610b82e0 0x558c610b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f85a000a740 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 0x558c610b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f85a000a740 1 --2- 172.21.15.18:0/79699682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610aed60 0x558c610b5270 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f85a000a740 1 -- 172.21.15.18:0/79699682 >> 172.21.15.18:0/79699682 conn(0x558c61076550 msgr2=0x558c610ab7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f857aff5640 0 client.19803 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f85a000a740 1 -- 172.21.15.18:0/79699682 shutdown_connections 2024-02-17T13:01:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.456+0000 7f85a000a740 1 -- 172.21.15.18:0/79699682 wait complete. 2024-02-17T13:01:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.457+0000 7f85a000a740 20 client.19803 trim_cache size 0 max 16384 2024-02-17T13:01:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85a000a740 1 Processor -- start 2024-02-17T13:01:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85a000a740 1 -- start start 2024-02-17T13:01:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85a000a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610aed60 0x558c610b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85a000a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 0x558c610b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85a000a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 0x558c610b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85a000a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c610aaa50 con 0x558c610b82e0 2024-02-17T13:01:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85a000a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c610aa620 con 0x558c610afad0 2024-02-17T13:01:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85a000a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c610ae030 con 0x558c610aed60 2024-02-17T13:01:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 0x558c610b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 0x558c610b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41248/0 (socket says 172.21.15.18:41248) 2024-02-17T13:01:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85977fe640 1 -- 172.21.15.18:0/3169244135 learned_addr learned my addr 172.21.15.18:0/3169244135 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f8596ffd640 1 --2- 172.21.15.18:0/3169244135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610aed60 0x558c610b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85967fc640 1 --2- 172.21.15.18:0/3169244135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 0x558c610b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85977fe640 1 -- 172.21.15.18:0/3169244135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610aed60 msgr2=0x558c610b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85977fe640 1 --2- 172.21.15.18:0/3169244135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610aed60 0x558c610b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85977fe640 1 -- 172.21.15.18:0/3169244135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 msgr2=0x558c610b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85977fe640 1 --2- 172.21.15.18:0/3169244135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 0x558c610b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.470+0000 7f85977fe640 1 -- 172.21.15.18:0/3169244135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c60fb17b0 con 0x558c610b82e0 2024-02-17T13:01:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.471+0000 7f85977fe640 1 --2- 172.21.15.18:0/3169244135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 0x558c610b3b40 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f85880129a0 tx=0x7f8588009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92396921a58f6e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.471+0000 7f857cff9640 1 -- 172.21.15.18:0/3169244135 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858800fae0 con 0x558c610b82e0 2024-02-17T13:01:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.471+0000 7f857cff9640 1 -- 172.21.15.18:0/3169244135 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f858801a070 con 0x558c610b82e0 2024-02-17T13:01:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.471+0000 7f857cff9640 1 -- 172.21.15.18:0/3169244135 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588020380 con 0x558c610b82e0 2024-02-17T13:01:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 -- 172.21.15.18:0/3169244135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 msgr2=0x558c610b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 --2- 172.21.15.18:0/3169244135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 0x558c610b3b40 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f85880129a0 tx=0x7f8588009280 comp rx=0 tx=0).stop 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 -- 172.21.15.18:0/3169244135 shutdown_connections 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 --2- 172.21.15.18:0/3169244135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 0x558c610b3b40 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 --2- 172.21.15.18:0/3169244135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 0x558c610b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 --2- 172.21.15.18:0/3169244135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610aed60 0x558c610b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 -- 172.21.15.18:0/3169244135 >> 172.21.15.18:0/3169244135 conn(0x558c61076550 msgr2=0x558c610d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 -- 172.21.15.18:0/3169244135 shutdown_connections 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 -- 172.21.15.18:0/3169244135 wait complete. 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 -- start start 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 Processor -- start 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610aed60 0x558c610c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 0x558c610c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 0x558c61121aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c61121fe0 con 0x558c610b82e0 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c61122120 con 0x558c610afad0 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c61122290 con 0x558c610aed60 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.472+0000 7f85a000a740 10 client.0 _ll_register_callbacks cb 0x7f8590009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f85977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 0x558c61121aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f85977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 0x558c61121aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41252/0 (socket says 172.21.15.18:41252) 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f85977fe640 1 -- 172.21.15.18:0/3056160016 learned_addr learned my addr 172.21.15.18:0/3056160016 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f85967fc640 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 0x558c610c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f8596ffd640 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610aed60 0x558c610c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f85977fe640 1 -- 172.21.15.18:0/3056160016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610aed60 msgr2=0x558c610c16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f85977fe640 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610aed60 0x558c610c16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f85977fe640 1 -- 172.21.15.18:0/3056160016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 msgr2=0x558c610c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f85977fe640 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 0x558c610c1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f85977fe640 1 -- 172.21.15.18:0/3056160016 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85880126e0 con 0x558c610b82e0 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f85977fe640 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 0x558c61121aa0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8588012970 tx=0x7f8588009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469638e2c689a7f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f857aff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f857aff5640 1 -- 172.21.15.18:0/3056160016 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858800fae0 con 0x558c610b82e0 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f857aff5640 1 -- 172.21.15.18:0/3056160016 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f858800fc40 con 0x558c610b82e0 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.473+0000 7f85a000a740 1 -- 172.21.15.18:0/3056160016 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85900049e0 con 0x558c610b82e0 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.474+0000 7f857aff5640 1 -- 172.21.15.18:0/3056160016 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8588021420 con 0x558c610b82e0 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.474+0000 7f85a000a740 10 client.19809 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.474+0000 7f85a000a740 1 -- 172.21.15.18:0/3056160016 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x558c60fd7540 con 0x558c610b82e0 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.474+0000 7f857aff5640 1 -- 172.21.15.18:0/3056160016 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8588040020 con 0x558c610b82e0 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.474+0000 7f857aff5640 1 -- 172.21.15.18:0/3056160016 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8588029c80 con 0x558c610b82e0 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.474+0000 7f85a000a740 10 client.19809 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.474+0000 7f85a000a740 10 client.19809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.474+0000 7f85a000a740 20 client.19809 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.474+0000 7f85a000a740 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558c61123a40 0x558c61125e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.474+0000 7f85a000a740 4 client.19809 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:01:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.474+0000 7f85a000a740 1 -- 172.21.15.18:0/3056160016 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558c60d53550 con 0x558c61123a40 2024-02-17T13:01:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.474+0000 7f8596ffd640 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558c61123a40 0x558c61125e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.475+0000 7f8596ffd640 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558c61123a40 0x558c61125e60 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd176a564434b24 server_cookie=4f92b627139ff47e in_seq=0 out_seq=0 2024-02-17T13:01:42.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.475+0000 7f857aff5640 10 client.19809 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f857aff5640 1 -- 172.21.15.18:0/3056160016 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558c60d53550 con 0x558c61123a40 2024-02-17T13:01:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f857aff5640 10 client.19809 handle_command_reply: tid=0 2024-02-17T13:01:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f85a000a740 1 client.19809 shutdown 2024-02-17T13:01:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f85a000a740 1 -- 172.21.15.18:0/3056160016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 msgr2=0x558c61121aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f85a000a740 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 0x558c61121aa0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8588012970 tx=0x7f8588009f30 comp rx=0 tx=0).stop 2024-02-17T13:01:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f85a000a740 1 -- 172.21.15.18:0/3056160016 shutdown_connections 2024-02-17T13:01:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f85a000a740 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x558c61123a40 0x558c61125e60 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f85a000a740 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c610b82e0 0x558c61121aa0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f85a000a740 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558c610afad0 0x558c610c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f857aff5640 0 client.19809 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f85a000a740 1 --2- 172.21.15.18:0/3056160016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c610aed60 0x558c610c16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f85a000a740 1 -- 172.21.15.18:0/3056160016 >> 172.21.15.18:0/3056160016 conn(0x558c61076550 msgr2=0x558c610b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f85a000a740 1 -- 172.21.15.18:0/3056160016 shutdown_connections 2024-02-17T13:01:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f85a000a740 1 -- 172.21.15.18:0/3056160016 wait complete. 2024-02-17T13:01:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.515+0000 7f85a000a740 20 client.19809 trim_cache size 0 max 16384 2024-02-17T13:01:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:01:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:01:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:01:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:01:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.516+0000 7f859d439640 1 -- 172.21.15.18:0/118222147 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f85800a22d0 msgr2=0x7f85800a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.516+0000 7f859d439640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f85800a22d0 0x7f85800a4790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f85880092f0 tx=0x7f8588014a70 comp rx=0 tx=0).stop 2024-02-17T13:01:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.516+0000 7f859d439640 1 -- 172.21.15.18:0/118222147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8598106490 msgr2=0x7f859819d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.516+0000 7f859d439640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8598106490 0x7f859819d920 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f858c00e9c0 tx=0x7f858c00ee80 comp rx=0 tx=0).stop 2024-02-17T13:01:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.516+0000 7f859d439640 1 -- 172.21.15.18:0/118222147 shutdown_connections 2024-02-17T13:01:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.516+0000 7f859d439640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f85800a22d0 0x7f85800a4790 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.516+0000 7f859d439640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85981975e0 0x7f8598197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.516+0000 7f859d439640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8598106490 0x7f859819d920 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.516+0000 7f859d439640 1 --2- 172.21.15.18:0/118222147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85981013c0 0x7f859819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.516+0000 7f859d439640 1 -- 172.21.15.18:0/118222147 >> 172.21.15.18:0/118222147 conn(0x7f85980fcc60 msgr2=0x7f85980fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.516+0000 7f859d439640 1 -- 172.21.15.18:0/118222147 shutdown_connections 2024-02-17T13:01:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:42.516+0000 7f859d439640 1 -- 172.21.15.18:0/118222147 wait complete. 2024-02-17T13:01:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.558+0000 7f36e276c640 1 -- 172.21.15.18:0/3737376113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc106450 msgr2=0x7f36dc10e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.558+0000 7f36e276c640 1 --2- 172.21.15.18:0/3737376113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc106450 0x7f36dc10e020 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009cd0 tx=0x7f36d402c0f0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.558+0000 7f36e276c640 1 -- 172.21.15.18:0/3737376113 shutdown_connections 2024-02-17T13:01:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.558+0000 7f36e276c640 1 --2- 172.21.15.18:0/3737376113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc106450 0x7f36dc10e020 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.558+0000 7f36e276c640 1 --2- 172.21.15.18:0/3737376113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36dc101d30 0x7f36dc105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.558+0000 7f36e276c640 1 --2- 172.21.15.18:0/3737376113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36dc101380 0x7f36dc101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.558+0000 7f36e276c640 1 -- 172.21.15.18:0/3737376113 >> 172.21.15.18:0/3737376113 conn(0x7f36dc078580 msgr2=0x7f36dc078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.558+0000 7f36e276c640 1 -- 172.21.15.18:0/3737376113 shutdown_connections 2024-02-17T13:01:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.558+0000 7f36e276c640 1 -- 172.21.15.18:0/3737376113 wait complete. 2024-02-17T13:01:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36e276c640 1 Processor -- start 2024-02-17T13:01:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36e276c640 1 -- start start 2024-02-17T13:01:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36e276c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36dc101380 0x7f36dc197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36e276c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36dc106450 0x7f36dc197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36e276c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc19c020 0x7f36dc19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36e276c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc19ca10 con 0x7f36dc19c020 2024-02-17T13:01:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36e276c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc19cb80 con 0x7f36dc106450 2024-02-17T13:01:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36e276c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc19ccf0 con 0x7f36dc101380 2024-02-17T13:01:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36e0d09640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc19c020 0x7f36dc19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36dbfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36dc101380 0x7f36dc197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36e0d09640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc19c020 0x7f36dc19c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41264/0 (socket says 172.21.15.18:41264) 2024-02-17T13:01:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36dbfff640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36dc101380 0x7f36dc197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:44674/0 (socket says 172.21.15.18:44674) 2024-02-17T13:01:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.559+0000 7f36e0d09640 1 -- 172.21.15.18:0/3732645824 learned_addr learned my addr 172.21.15.18:0/3732645824 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36db7fe640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36dc106450 0x7f36dc197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36e0d09640 1 -- 172.21.15.18:0/3732645824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36dc101380 msgr2=0x7f36dc197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36e0d09640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36dc101380 0x7f36dc197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36e0d09640 1 -- 172.21.15.18:0/3732645824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36dc106450 msgr2=0x7f36dc197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36e0d09640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36dc106450 0x7f36dc197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36e0d09640 1 -- 172.21.15.18:0/3732645824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d4009900 con 0x7f36dc19c020 2024-02-17T13:01:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36dbfff640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36dc101380 0x7f36dc197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36e0d09640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc19c020 0x7f36dc19c4d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f36d40348f0 tx=0x7f36d4034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36d97fa640 1 -- 172.21.15.18:0/3732645824 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d40328c0 con 0x7f36dc19c020 2024-02-17T13:01:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36e276c640 1 -- 172.21.15.18:0/3732645824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36dc1a42d0 con 0x7f36dc19c020 2024-02-17T13:01:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36d97fa640 1 -- 172.21.15.18:0/3732645824 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36d4003e50 con 0x7f36dc19c020 2024-02-17T13:01:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36d97fa640 1 -- 172.21.15.18:0/3732645824 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4040450 con 0x7f36dc19c020 2024-02-17T13:01:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.560+0000 7f36e276c640 1 -- 172.21.15.18:0/3732645824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36dc1a46f0 con 0x7f36dc19c020 2024-02-17T13:01:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.563+0000 7f36d97fa640 1 -- 172.21.15.18:0/3732645824 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f36d400a040 con 0x7f36dc19c020 2024-02-17T13:01:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.563+0000 7f36e541e740 1 Processor -- start 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.563+0000 7f36e541e740 1 -- start start 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.563+0000 7f36e541e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8b250 0x55b932e8b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.563+0000 7f36e541e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 0x55b932e94210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.563+0000 7f36e541e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e94750 0x55b932e96b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.563+0000 7f36e541e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b932e46960 con 0x55b932e8bb70 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.563+0000 7f36e541e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b932dfa310 con 0x55b932e8b250 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.563+0000 7f36e541e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b932df7840 con 0x55b932e94750 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.563+0000 7f36dbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 0x55b932e94210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36db7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8b250 0x55b932e8b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36dbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 0x55b932e94210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41280/0 (socket says 172.21.15.18:41280) 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36dbfff640 1 -- 172.21.15.18:0/144189507 learned_addr learned my addr 172.21.15.18:0/144189507 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36d97fa640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f36c80a2230 0x7f36c80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36db7fe640 1 --2- 172.21.15.18:0/144189507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e94750 0x55b932e96b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36e0d09640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f36c80a2230 0x7f36c80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36dbfff640 1 -- 172.21.15.18:0/144189507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e94750 msgr2=0x55b932e96b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36dbfff640 1 --2- 172.21.15.18:0/144189507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e94750 0x55b932e96b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36dbfff640 1 -- 172.21.15.18:0/144189507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8b250 msgr2=0x55b932e8b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36dbfff640 1 --2- 172.21.15.18:0/144189507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8b250 0x55b932e8b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36dbfff640 1 -- 172.21.15.18:0/144189507 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b932db5060 con 0x55b932e8bb70 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36d97fa640 1 -- 172.21.15.18:0/3732645824 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f36d40f0db0 con 0x7f36dc19c020 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36db7fe640 1 --2- 172.21.15.18:0/144189507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8b250 0x55b932e8b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36dbfff640 1 --2- 172.21.15.18:0/144189507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 0x55b932e94210 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f36c4010480 tx=0x7f36c4010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b2e0b0a6b1a583 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36e0d09640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f36c80a2230 0x7f36c80a46f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f36d40bd570 tx=0x7f36d40dd040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36d27fc640 1 -- 172.21.15.18:0/144189507 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4011760 con 0x55b932e8bb70 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36d27fc640 1 -- 172.21.15.18:0/144189507 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36c4015070 con 0x55b932e8bb70 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.564+0000 7f36d27fc640 1 -- 172.21.15.18:0/144189507 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c401cd40 con 0x55b932e8bb70 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.565+0000 7f36e541e740 1 -- 172.21.15.18:0/144189507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 msgr2=0x55b932e94210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.565+0000 7f36e541e740 1 --2- 172.21.15.18:0/144189507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 0x55b932e94210 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f36c4010480 tx=0x7f36c4010940 comp rx=0 tx=0).stop 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.565+0000 7f36e541e740 1 -- 172.21.15.18:0/144189507 shutdown_connections 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.565+0000 7f36e541e740 1 --2- 172.21.15.18:0/144189507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e94750 0x55b932e96b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.565+0000 7f36e541e740 1 --2- 172.21.15.18:0/144189507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 0x55b932e94210 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.565+0000 7f36e541e740 1 --2- 172.21.15.18:0/144189507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8b250 0x55b932e8b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.565+0000 7f36e541e740 1 -- 172.21.15.18:0/144189507 >> 172.21.15.18:0/144189507 conn(0x55b932e85980 msgr2=0x55b932e85d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.565+0000 7f36e541e740 1 -- 172.21.15.18:0/144189507 shutdown_connections 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.565+0000 7f36e541e740 1 -- 172.21.15.18:0/144189507 wait complete. 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.565+0000 7f36e541e740 1 -- start start 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36e541e740 1 Processor -- start 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36e541e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36e541e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8bb70 0x55b932e91c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36e541e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 0x55b932e92150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36e541e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b932df3020 con 0x55b932e94750 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36e541e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b932ac6430 con 0x55b932e8bb70 2024-02-17T13:01:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36e541e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b932a82e00 con 0x55b932e8b250 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36e541e740 10 client.0 _ll_register_callbacks cb 0x55b932d9acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36db7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 0x55b932e92150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36db7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 0x55b932e92150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41290/0 (socket says 172.21.15.18:41290) 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36db7fe640 1 -- 172.21.15.18:0/3971709400 learned_addr learned my addr 172.21.15.18:0/3971709400 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36dbfff640 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8bb70 0x55b932e91c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36db7fe640 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36db7fe640 1 -- 172.21.15.18:0/3971709400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 msgr2=0x55b932e916d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36db7fe640 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e916d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36db7fe640 1 -- 172.21.15.18:0/3971709400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8bb70 msgr2=0x55b932e91c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36db7fe640 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8bb70 0x55b932e91c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36db7fe640 1 -- 172.21.15.18:0/3971709400 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c400bcd0 con 0x55b932e94750 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36db7fe640 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 0x55b932e92150 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f36dc06aa20 tx=0x7f36cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbd656ff7aa015c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.566+0000 7f36bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36bb7fe640 1 -- 172.21.15.18:0/3971709400 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc004220 con 0x55b932e94750 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36bb7fe640 1 -- 172.21.15.18:0/3971709400 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f36cc004380 con 0x55b932e94750 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36e541e740 1 -- 172.21.15.18:0/3971709400 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b932af27e0 con 0x55b932e94750 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36bb7fe640 1 -- 172.21.15.18:0/3971709400 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36cc0118e0 con 0x55b932e94750 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36e541e740 10 client.19821 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36e541e740 1 -- 172.21.15.18:0/3971709400 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55b932df16a0 con 0x55b932e94750 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36bb7fe640 1 -- 172.21.15.18:0/3971709400 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b932af27e0 con 0x55b932e94750 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36bb7fe640 1 -- 172.21.15.18:0/3971709400 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f36cc0047a0 con 0x55b932e94750 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36e541e740 10 client.19821 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36e541e740 10 client.19821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36e541e740 20 client.19821 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36e541e740 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b932ea4a30 0x55b932ea6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36e541e740 4 client.19821 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36e541e740 1 -- 172.21.15.18:0/3971709400 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b932b2f720 con 0x55b932ea4a30 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.567+0000 7f36dbfff640 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b932ea4a30 0x55b932ea6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.568+0000 7f36dbfff640 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b932ea4a30 0x55b932ea6e50 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=354e2af5e8950716 server_cookie=192f2c93eb72fa44 in_seq=0 out_seq=0 2024-02-17T13:01:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.568+0000 7f36bb7fe640 10 client.19821 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.568+0000 7f36bb7fe640 1 -- 172.21.15.18:0/3971709400 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b932b2f720 con 0x55b932ea4a30 2024-02-17T13:01:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.568+0000 7f36bb7fe640 10 client.19821 handle_command_reply: tid=0 2024-02-17T13:01:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.568+0000 7f36e541e740 1 client.19821 shutdown 2024-02-17T13:01:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.569+0000 7f36e541e740 1 -- 172.21.15.18:0/3971709400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 msgr2=0x55b932e92150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.569+0000 7f36e541e740 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 0x55b932e92150 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f36dc06aa20 tx=0x7f36cc004730 comp rx=0 tx=0).stop 2024-02-17T13:01:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.569+0000 7f36e541e740 1 -- 172.21.15.18:0/3971709400 shutdown_connections 2024-02-17T13:01:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.569+0000 7f36e541e740 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b932ea4a30 0x55b932ea6e50 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.569+0000 7f36e541e740 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 0x55b932e92150 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.569+0000 7f36e541e740 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8bb70 0x55b932e91c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.569+0000 7f36bb7fe640 0 client.19821 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.569+0000 7f36e541e740 1 --2- 172.21.15.18:0/3971709400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e916d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.569+0000 7f36e541e740 1 -- 172.21.15.18:0/3971709400 >> 172.21.15.18:0/3971709400 conn(0x55b932e85980 msgr2=0x55b932e876d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.569+0000 7f36e541e740 1 -- 172.21.15.18:0/3971709400 shutdown_connections 2024-02-17T13:01:42.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.569+0000 7f36e541e740 1 -- 172.21.15.18:0/3971709400 wait complete. 2024-02-17T13:01:42.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.569+0000 7f36e541e740 20 client.19821 trim_cache size 0 max 16384 2024-02-17T13:01:42.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.580+0000 7f36e541e740 1 Processor -- start 2024-02-17T13:01:42.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.580+0000 7f36e541e740 1 -- start start 2024-02-17T13:01:42.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36e541e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e93e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36e541e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8bb70 0x55b932e8c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36e541e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 0x55b932e8ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36e541e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b932d78130 con 0x55b932e94750 2024-02-17T13:01:42.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36e541e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b932e77a40 con 0x55b932e8bb70 2024-02-17T13:01:42.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36e541e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b932ea15d0 con 0x55b932e8b250 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36db7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 0x55b932e8ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36db7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 0x55b932e8ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41294/0 (socket says 172.21.15.18:41294) 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36db7fe640 1 -- 172.21.15.18:0/1667327126 learned_addr learned my addr 172.21.15.18:0/1667327126 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36db7fe640 1 --2- 172.21.15.18:0/1667327126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e93e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36dbfff640 1 --2- 172.21.15.18:0/1667327126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8bb70 0x55b932e8c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36db7fe640 1 -- 172.21.15.18:0/1667327126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 msgr2=0x55b932e93e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36db7fe640 1 --2- 172.21.15.18:0/1667327126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e93e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36db7fe640 1 -- 172.21.15.18:0/1667327126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8bb70 msgr2=0x55b932e8c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36db7fe640 1 --2- 172.21.15.18:0/1667327126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8bb70 0x55b932e8c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36db7fe640 1 -- 172.21.15.18:0/1667327126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b932d8dc20 con 0x55b932e94750 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36dbfff640 1 --2- 172.21.15.18:0/1667327126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8bb70 0x55b932e8c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36db7fe640 1 --2- 172.21.15.18:0/1667327126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 0x55b932e8ced0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0124d0 tx=0x7f36cc00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973db14ec9931484 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36d1ffb640 1 -- 172.21.15.18:0/1667327126 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0049a0 con 0x55b932e94750 2024-02-17T13:01:42.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36d1ffb640 1 -- 172.21.15.18:0/1667327126 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36cc011c90 con 0x55b932e94750 2024-02-17T13:01:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.581+0000 7f36d1ffb640 1 -- 172.21.15.18:0/1667327126 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0054f0 con 0x55b932e94750 2024-02-17T13:01:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.582+0000 7f36e541e740 1 -- 172.21.15.18:0/1667327126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 msgr2=0x55b932e8ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.582+0000 7f36e541e740 1 --2- 172.21.15.18:0/1667327126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 0x55b932e8ced0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0124d0 tx=0x7f36cc00b410 comp rx=0 tx=0).stop 2024-02-17T13:01:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.582+0000 7f36e541e740 1 -- 172.21.15.18:0/1667327126 shutdown_connections 2024-02-17T13:01:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.582+0000 7f36e541e740 1 --2- 172.21.15.18:0/1667327126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e94750 0x55b932e8ced0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.582+0000 7f36e541e740 1 --2- 172.21.15.18:0/1667327126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e8bb70 0x55b932e8c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.582+0000 7f36e541e740 1 --2- 172.21.15.18:0/1667327126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e93e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.582+0000 7f36e541e740 1 -- 172.21.15.18:0/1667327126 >> 172.21.15.18:0/1667327126 conn(0x55b932e85980 msgr2=0x55b932e80a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.582+0000 7f36e541e740 1 -- 172.21.15.18:0/1667327126 shutdown_connections 2024-02-17T13:01:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.582+0000 7f36e541e740 1 -- 172.21.15.18:0/1667327126 wait complete. 2024-02-17T13:01:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.582+0000 7f36e541e740 1 -- start start 2024-02-17T13:01:42.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36e541e740 1 Processor -- start 2024-02-17T13:01:42.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36e541e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e9a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36e541e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 0x55b932e9a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36e541e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e94750 0x55b932e9ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36e541e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b932efafc0 con 0x55b932e8bb70 2024-02-17T13:01:42.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36e541e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b932efb130 con 0x55b932e94750 2024-02-17T13:01:42.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36e541e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b932efb2f0 con 0x55b932e8b250 2024-02-17T13:01:42.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36e541e740 10 client.0 _ll_register_callbacks cb 0x7f36c400bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36dbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 0x55b932e9a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36dbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 0x55b932e9a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41304/0 (socket says 172.21.15.18:41304) 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36dbfff640 1 -- 172.21.15.18:0/2054092834 learned_addr learned my addr 172.21.15.18:0/2054092834 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36db7fe640 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e9a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36dbfff640 1 -- 172.21.15.18:0/2054092834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 msgr2=0x55b932e9a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36dbfff640 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e9a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36dbfff640 1 -- 172.21.15.18:0/2054092834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e94750 msgr2=0x55b932e9ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36dbfff640 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e94750 0x55b932e9ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36dbfff640 1 -- 172.21.15.18:0/2054092834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b932df16a0 con 0x55b932e8bb70 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36db7fe640 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e9a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36dbfff640 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 0x55b932e9a6d0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f36c4010480 tx=0x7f36c400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871ddb2055c50d63 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.583+0000 7f36bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36bb7fe640 1 -- 172.21.15.18:0/2054092834 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c401dd10 con 0x55b932e8bb70 2024-02-17T13:01:42.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36e541e740 1 -- 172.21.15.18:0/2054092834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36c4015070 con 0x55b932e8bb70 2024-02-17T13:01:42.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36bb7fe640 1 -- 172.21.15.18:0/2054092834 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f36c4017070 con 0x55b932e8bb70 2024-02-17T13:01:42.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36bb7fe640 1 -- 172.21.15.18:0/2054092834 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36c401de70 con 0x55b932e8bb70 2024-02-17T13:01:42.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36e541e740 10 client.19827 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:42.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36e541e740 1 -- 172.21.15.18:0/2054092834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55b932db38c0 con 0x55b932e8bb70 2024-02-17T13:01:42.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36bb7fe640 1 -- 172.21.15.18:0/2054092834 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36c4015070 con 0x55b932e8bb70 2024-02-17T13:01:42.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36bb7fe640 1 -- 172.21.15.18:0/2054092834 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f36c4030030 con 0x55b932e8bb70 2024-02-17T13:01:42.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36e541e740 10 client.19827 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:42.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36e541e740 10 client.19827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:42.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36e541e740 20 client.19827 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36e541e740 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b932efcb20 0x55b932efef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36e541e740 4 client.19827 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36e541e740 1 -- 172.21.15.18:0/2054092834 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b932b7e890 con 0x55b932efcb20 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.584+0000 7f36db7fe640 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b932efcb20 0x55b932efef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36db7fe640 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b932efcb20 0x55b932efef40 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aaf885969848994 server_cookie=4aaa22bfa4f75d0a in_seq=0 out_seq=0 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36bb7fe640 10 client.19827 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36bb7fe640 1 -- 172.21.15.18:0/2054092834 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b932b7e890 con 0x55b932efcb20 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36bb7fe640 10 client.19827 handle_command_reply: tid=0 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36e541e740 1 client.19827 shutdown 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36e541e740 1 -- 172.21.15.18:0/2054092834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 msgr2=0x55b932e9a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36e541e740 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 0x55b932e9a6d0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f36c4010480 tx=0x7f36c400bb90 comp rx=0 tx=0).stop 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36e541e740 1 -- 172.21.15.18:0/2054092834 shutdown_connections 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36e541e740 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b932efcb20 0x55b932efef40 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36e541e740 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b932e94750 0x55b932e9ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36e541e740 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b932e8bb70 0x55b932e9a6d0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36bb7fe640 0 client.19827 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:42.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36e541e740 1 --2- 172.21.15.18:0/2054092834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b932e8b250 0x55b932e9a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36e541e740 1 -- 172.21.15.18:0/2054092834 >> 172.21.15.18:0/2054092834 conn(0x55b932e85980 msgr2=0x55b932e8c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36e541e740 1 -- 172.21.15.18:0/2054092834 shutdown_connections 2024-02-17T13:01:42.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36e541e740 1 -- 172.21.15.18:0/2054092834 wait complete. 2024-02-17T13:01:42.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.585+0000 7f36e541e740 20 client.19827 trim_cache size 0 max 16384 2024-02-17T13:01:42.582 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:01:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.586+0000 7f36e276c640 1 -- 172.21.15.18:0/3732645824 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f36c80a2230 msgr2=0x7f36c80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.586+0000 7f36e276c640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f36c80a2230 0x7f36c80a46f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f36d40bd570 tx=0x7f36d40dd040 comp rx=0 tx=0).stop 2024-02-17T13:01:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.586+0000 7f36e276c640 1 -- 172.21.15.18:0/3732645824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc19c020 msgr2=0x7f36dc19c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.586+0000 7f36e276c640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc19c020 0x7f36dc19c4d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f36d40348f0 tx=0x7f36d4034cc0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.586+0000 7f36e276c640 1 -- 172.21.15.18:0/3732645824 shutdown_connections 2024-02-17T13:01:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.586+0000 7f36e276c640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f36c80a2230 0x7f36c80a46f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.586+0000 7f36e276c640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc19c020 0x7f36dc19c4d0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.586+0000 7f36e276c640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36dc106450 0x7f36dc197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.586+0000 7f36e276c640 1 --2- 172.21.15.18:0/3732645824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36dc101380 0x7f36dc197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.586+0000 7f36e276c640 1 -- 172.21.15.18:0/3732645824 >> 172.21.15.18:0/3732645824 conn(0x7f36dc078580 msgr2=0x7f36dc0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.586+0000 7f36e276c640 1 -- 172.21.15.18:0/3732645824 shutdown_connections 2024-02-17T13:01:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:42.586+0000 7f36e276c640 1 -- 172.21.15.18:0/3732645824 wait complete. 2024-02-17T13:01:42.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:42 smithi119 ceph-mon[36188]: pgmap v370: 97 pgs: 97 active+clean; 232 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-02-17T13:01:42.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:42 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1135504014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:01:42.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:42 smithi146 ceph-mon[36454]: pgmap v370: 97 pgs: 97 active+clean; 232 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-02-17T13:01:42.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:42 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1135504014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:01:43.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 587763 4.86 MB/sec execute 538 sec latency 3448.652 ms 2024-02-17T13:01:43.587 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T13:01:43.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:01:43.588 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T13:01:43.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.723+0000 7fa01a98a640 1 -- 172.21.15.18:0/1518156801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa014100240 msgr2=0x7fa014105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.724+0000 7fa01a98a640 1 --2- 172.21.15.18:0/1518156801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa014100240 0x7fa014105d70 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fa008009930 tx=0x7fa00802bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:43.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.724+0000 7fa01a98a640 1 -- 172.21.15.18:0/1518156801 shutdown_connections 2024-02-17T13:01:43.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.724+0000 7fa01a98a640 1 --2- 172.21.15.18:0/1518156801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014106450 0x7fa01410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.724+0000 7fa01a98a640 1 --2- 172.21.15.18:0/1518156801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa014100240 0x7fa014105d70 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.724+0000 7fa01a98a640 1 --2- 172.21.15.18:0/1518156801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0140ff890 0x7fa0140ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.724+0000 7fa01a98a640 1 -- 172.21.15.18:0/1518156801 >> 172.21.15.18:0/1518156801 conn(0x7fa0140765a0 msgr2=0x7fa0140fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:43.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.724+0000 7fa01a98a640 1 -- 172.21.15.18:0/1518156801 shutdown_connections 2024-02-17T13:01:43.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.724+0000 7fa01a98a640 1 -- 172.21.15.18:0/1518156801 wait complete. 2024-02-17T13:01:43.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.724+0000 7fa01a98a640 1 Processor -- start 2024-02-17T13:01:43.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.724+0000 7fa01a98a640 1 -- start start 2024-02-17T13:01:43.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa01a98a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0140ff890 0x7fa014100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa01a98a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa014106450 0x7fa014100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa01a98a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014101280 0x7fa0141a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa01a98a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0141a5fd0 con 0x7fa0140ff890 2024-02-17T13:01:43.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa01a98a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0141a6140 con 0x7fa014101280 2024-02-17T13:01:43.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa01a98a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0141a6300 con 0x7fa014106450 2024-02-17T13:01:43.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa013fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0140ff890 0x7fa014100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa013fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0140ff890 0x7fa014100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41322/0 (socket says 172.21.15.18:41322) 2024-02-17T13:01:43.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa013fff640 1 -- 172.21.15.18:0/2638299650 learned_addr learned my addr 172.21.15.18:0/2638299650 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:43.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa0137fe640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa014106450 0x7fa014100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa018f27640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014101280 0x7fa0141a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa013fff640 1 -- 172.21.15.18:0/2638299650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa014106450 msgr2=0x7fa014100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa013fff640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa014106450 0x7fa014100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa013fff640 1 -- 172.21.15.18:0/2638299650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014101280 msgr2=0x7fa0141a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa013fff640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014101280 0x7fa0141a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa013fff640 1 -- 172.21.15.18:0/2638299650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa008009560 con 0x7fa0140ff890 2024-02-17T13:01:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.725+0000 7fa0137fe640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa014106450 0x7fa014100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.726+0000 7fa013fff640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0140ff890 0x7fa014100800 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fa00400e9f0 tx=0x7fa00400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.726+0000 7fa0117fa640 1 -- 172.21.15.18:0/2638299650 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004009720 con 0x7fa0140ff890 2024-02-17T13:01:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.726+0000 7fa0117fa640 1 -- 172.21.15.18:0/2638299650 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa00400cd80 con 0x7fa0140ff890 2024-02-17T13:01:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.726+0000 7fa01a98a640 1 -- 172.21.15.18:0/2638299650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0141a65e0 con 0x7fa0140ff890 2024-02-17T13:01:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.726+0000 7fa0117fa640 1 -- 172.21.15.18:0/2638299650 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004010690 con 0x7fa0140ff890 2024-02-17T13:01:43.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.726+0000 7fa01a98a640 1 -- 172.21.15.18:0/2638299650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0141a6a70 con 0x7fa0140ff890 2024-02-17T13:01:43.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa0117fa640 1 -- 172.21.15.18:0/2638299650 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa0040046c0 con 0x7fa0140ff890 2024-02-17T13:01:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa01d633740 1 Processor -- start 2024-02-17T13:01:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa01d633740 1 -- start start 2024-02-17T13:01:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa01d633740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777c380 0x5603c777c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa01d633740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777cd30 0x5603c77852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa01d633740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c77858b0 0x5603c7787d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa01d633740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c76eaf90 con 0x5603c777c380 2024-02-17T13:01:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa01d633740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c76e84c0 con 0x5603c777cd30 2024-02-17T13:01:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa01d633740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c76e3ca0 con 0x5603c77858b0 2024-02-17T13:01:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa0117fa640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f9ffc0a2260 0x7f9ffc0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa018f27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777c380 0x5603c777c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa013fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777cd30 0x5603c77852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa018f27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777c380 0x5603c777c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41332/0 (socket says 172.21.15.18:41332) 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa018f27640 1 -- 172.21.15.18:0/7192170 learned_addr learned my addr 172.21.15.18:0/7192170 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa018f27640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f9ffc0a2260 0x7f9ffc0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa0137fe640 1 --2- 172.21.15.18:0/7192170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c77858b0 0x5603c7787d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.729+0000 7fa0117fa640 1 -- 172.21.15.18:0/2638299650 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa004014070 con 0x7fa0140ff890 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.730+0000 7fa018f27640 1 -- 172.21.15.18:0/7192170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c77858b0 msgr2=0x5603c7787d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.730+0000 7fa018f27640 1 --2- 172.21.15.18:0/7192170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c77858b0 0x5603c7787d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.730+0000 7fa018f27640 1 -- 172.21.15.18:0/7192170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777cd30 msgr2=0x5603c77852e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.730+0000 7fa018f27640 1 --2- 172.21.15.18:0/7192170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777cd30 0x5603c77852e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.730+0000 7fa018f27640 1 -- 172.21.15.18:0/7192170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603c76a60b0 con 0x5603c777c380 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.730+0000 7fa018f27640 1 --2- 172.21.15.18:0/7192170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777c380 0x5603c777c760 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fa00c012d00 tx=0x7fa00c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc85d3e57a2a3e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.730+0000 7fa018f27640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f9ffc0a2260 0x7f9ffc0a4720 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa00c010c40 tx=0x7fa00c00c000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.730+0000 7f9ffa7fc640 1 -- 172.21.15.18:0/7192170 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c024070 con 0x5603c777c380 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.730+0000 7f9ffa7fc640 1 -- 172.21.15.18:0/7192170 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa00c012520 con 0x5603c777c380 2024-02-17T13:01:43.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.730+0000 7f9ffa7fc640 1 -- 172.21.15.18:0/7192170 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c01f960 con 0x5603c777c380 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.731+0000 7fa01d633740 1 -- 172.21.15.18:0/7192170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777c380 msgr2=0x5603c777c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.731+0000 7fa01d633740 1 --2- 172.21.15.18:0/7192170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777c380 0x5603c777c760 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fa00c012d00 tx=0x7fa00c010740 comp rx=0 tx=0).stop 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.731+0000 7fa01d633740 1 -- 172.21.15.18:0/7192170 shutdown_connections 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.731+0000 7fa01d633740 1 --2- 172.21.15.18:0/7192170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c77858b0 0x5603c7787d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.731+0000 7fa01d633740 1 --2- 172.21.15.18:0/7192170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777cd30 0x5603c77852e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.731+0000 7fa01d633740 1 --2- 172.21.15.18:0/7192170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777c380 0x5603c777c760 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.731+0000 7fa01d633740 1 -- 172.21.15.18:0/7192170 >> 172.21.15.18:0/7192170 conn(0x5603c7776840 msgr2=0x5603c7776c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.731+0000 7fa01d633740 1 -- 172.21.15.18:0/7192170 shutdown_connections 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.731+0000 7fa01d633740 1 -- 172.21.15.18:0/7192170 wait complete. 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.731+0000 7fa01d633740 1 -- start start 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa01d633740 1 Processor -- start 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa01d633740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777c380 0x5603c7782980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa01d633740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c777cd30 0x5603c7782ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa01d633740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c77858b0 0x5603c7783400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa01d633740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c7688f40 con 0x5603c77858b0 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa01d633740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c73b74b0 con 0x5603c777c380 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa01d633740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c7669670 con 0x5603c777cd30 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa01d633740 10 client.0 _ll_register_callbacks cb 0x5603c76d6f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa018f27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c77858b0 0x5603c7783400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa0137fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c777cd30 0x5603c7782ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa018f27640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c77858b0 0x5603c7783400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41334/0 (socket says 172.21.15.18:41334) 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa018f27640 1 -- 172.21.15.18:0/3470171654 learned_addr learned my addr 172.21.15.18:0/3470171654 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.732+0000 7fa013fff640 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777c380 0x5603c7782980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7fa0137fe640 1 -- 172.21.15.18:0/3470171654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777c380 msgr2=0x5603c7782980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7fa0137fe640 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777c380 0x5603c7782980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7fa0137fe640 1 -- 172.21.15.18:0/3470171654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c77858b0 msgr2=0x5603c7783400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7fa0137fe640 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c77858b0 0x5603c7783400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7fa0137fe640 1 -- 172.21.15.18:0/3470171654 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00c01a040 con 0x5603c777cd30 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7fa018f27640 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c77858b0 0x5603c7783400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7fa0137fe640 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c777cd30 0x5603c7782ec0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa00802c170 tx=0x7fa00802cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c14703823555e052 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7f9ff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7f9ff37fe640 1 -- 172.21.15.18:0/3470171654 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008004410 con 0x5603c777cd30 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7f9ff37fe640 1 -- 172.21.15.18:0/3470171654 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa008004df0 con 0x5603c777cd30 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7fa01d633740 1 -- 172.21.15.18:0/3470171654 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603c76bdf40 con 0x5603c777cd30 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7f9ff37fe640 1 -- 172.21.15.18:0/3470171654 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa008004fd0 con 0x5603c777cd30 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7fa01d633740 10 client.28825 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.733+0000 7fa01d633740 1 -- 172.21.15.18:0/3470171654 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5603c76e2320 con 0x5603c777cd30 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.734+0000 7f9ff37fe640 1 -- 172.21.15.18:0/3470171654 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5603c76bdf40 con 0x5603c777cd30 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.734+0000 7f9ff37fe640 1 -- 172.21.15.18:0/3470171654 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa00803c040 con 0x5603c777cd30 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.734+0000 7fa01d633740 10 client.28825 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.734+0000 7fa01d633740 10 client.28825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.734+0000 7fa01d633740 20 client.28825 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.734+0000 7fa01d633740 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5603c7795fb0 0x5603c77983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.734+0000 7fa01d633740 4 client.28825 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.734+0000 7fa01d633740 1 -- 172.21.15.18:0/3470171654 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603c76da310 con 0x5603c7795fb0 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.734+0000 7fa013fff640 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5603c7795fb0 0x5603c77983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.734+0000 7f9ff37fe640 10 client.28825 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.734+0000 7fa013fff640 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5603c7795fb0 0x5603c77983d0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=948fa13df109b9da server_cookie=3a4eb62621f719bd in_seq=0 out_seq=0 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7f9ff37fe640 1 -- 172.21.15.18:0/3470171654 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5603c76da310 con 0x5603c7795fb0 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7f9ff37fe640 10 client.28825 handle_command_reply: tid=0 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7fa01d633740 1 client.28825 shutdown 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7fa01d633740 1 -- 172.21.15.18:0/3470171654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c777cd30 msgr2=0x5603c7782ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7fa01d633740 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c777cd30 0x5603c7782ec0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa00802c170 tx=0x7fa00802cd20 comp rx=0 tx=0).stop 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7fa01d633740 1 -- 172.21.15.18:0/3470171654 shutdown_connections 2024-02-17T13:01:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7fa01d633740 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5603c7795fb0 0x5603c77983d0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7fa01d633740 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c77858b0 0x5603c7783400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7fa01d633740 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c777cd30 0x5603c7782ec0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7fa01d633740 1 --2- 172.21.15.18:0/3470171654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777c380 0x5603c7782980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7f9ff37fe640 0 client.28825 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7fa01d633740 1 -- 172.21.15.18:0/3470171654 >> 172.21.15.18:0/3470171654 conn(0x5603c7776840 msgr2=0x5603c77787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7fa01d633740 1 -- 172.21.15.18:0/3470171654 shutdown_connections 2024-02-17T13:01:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7fa01d633740 1 -- 172.21.15.18:0/3470171654 wait complete. 2024-02-17T13:01:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.735+0000 7fa01d633740 20 client.28825 trim_cache size 0 max 16384 2024-02-17T13:01:43.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa01d633740 1 Processor -- start 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa01d633740 1 -- start start 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa01d633740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777c380 0x5603c7782ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa01d633740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 0x5603c7783430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa01d633740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c77858b0 0x5603c777fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa01d633740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c7772110 con 0x5603c777cd30 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa01d633740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c77853e0 con 0x5603c777c380 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa01d633740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c77766a0 con 0x5603c77858b0 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa0137fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 0x5603c7783430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa0137fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 0x5603c7783430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41340/0 (socket says 172.21.15.18:41340) 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa0137fe640 1 -- 172.21.15.18:0/3490686357 learned_addr learned my addr 172.21.15.18:0/3490686357 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa018f27640 1 --2- 172.21.15.18:0/3490686357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c77858b0 0x5603c777fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.753+0000 7fa013fff640 1 --2- 172.21.15.18:0/3490686357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777c380 0x5603c7782ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.754+0000 7fa0137fe640 1 -- 172.21.15.18:0/3490686357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c77858b0 msgr2=0x5603c777fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.754+0000 7fa0137fe640 1 --2- 172.21.15.18:0/3490686357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c77858b0 0x5603c777fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.754+0000 7fa0137fe640 1 -- 172.21.15.18:0/3490686357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777c380 msgr2=0x5603c7782ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.754+0000 7fa0137fe640 1 --2- 172.21.15.18:0/3490686357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777c380 0x5603c7782ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.754+0000 7fa0137fe640 1 -- 172.21.15.18:0/3490686357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603c767ec00 con 0x5603c777cd30 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.754+0000 7fa0137fe640 1 --2- 172.21.15.18:0/3490686357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 0x5603c7783430 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fa008005e00 tx=0x7fa008005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf799e98956c166 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.754+0000 7f9ff9ffb640 1 -- 172.21.15.18:0/3490686357 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008003fc0 con 0x5603c777cd30 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.754+0000 7f9ff9ffb640 1 -- 172.21.15.18:0/3490686357 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa00802c5d0 con 0x5603c777cd30 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.754+0000 7f9ff9ffb640 1 -- 172.21.15.18:0/3490686357 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008034a90 con 0x5603c777cd30 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.755+0000 7fa01d633740 1 -- 172.21.15.18:0/3490686357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 msgr2=0x5603c7783430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.755+0000 7fa01d633740 1 --2- 172.21.15.18:0/3490686357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 0x5603c7783430 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fa008005e00 tx=0x7fa008005e50 comp rx=0 tx=0).stop 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.755+0000 7fa01d633740 1 -- 172.21.15.18:0/3490686357 shutdown_connections 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.755+0000 7fa01d633740 1 --2- 172.21.15.18:0/3490686357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c77858b0 0x5603c777fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.755+0000 7fa01d633740 1 --2- 172.21.15.18:0/3490686357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 0x5603c7783430 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.755+0000 7fa01d633740 1 --2- 172.21.15.18:0/3490686357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c777c380 0x5603c7782ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.755+0000 7fa01d633740 1 -- 172.21.15.18:0/3490686357 >> 172.21.15.18:0/3490686357 conn(0x5603c7776840 msgr2=0x5603c7781270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:43.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.755+0000 7fa01d633740 1 -- 172.21.15.18:0/3490686357 shutdown_connections 2024-02-17T13:01:43.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.755+0000 7fa01d633740 1 -- 172.21.15.18:0/3490686357 wait complete. 2024-02-17T13:01:43.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.755+0000 7fa01d633740 1 -- start start 2024-02-17T13:01:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.755+0000 7fa01d633740 1 Processor -- start 2024-02-17T13:01:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa01d633740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c777c380 0x5603c778b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa01d633740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 0x5603c778b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa01d633740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c77858b0 0x5603c778bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa01d633740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c77ec4c0 con 0x5603c777cd30 2024-02-17T13:01:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa01d633740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c77ec600 con 0x5603c77858b0 2024-02-17T13:01:43.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa01d633740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c77ec790 con 0x5603c777c380 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa01d633740 10 client.0 _ll_register_callbacks cb 0x7fa00c0123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa0137fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 0x5603c778b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa0137fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 0x5603c778b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41344/0 (socket says 172.21.15.18:41344) 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa0137fe640 1 -- 172.21.15.18:0/2472039812 learned_addr learned my addr 172.21.15.18:0/2472039812 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa013fff640 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c777c380 0x5603c778b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa018f27640 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c77858b0 0x5603c778bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa0137fe640 1 -- 172.21.15.18:0/2472039812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c777c380 msgr2=0x5603c778b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa0137fe640 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c777c380 0x5603c778b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa0137fe640 1 -- 172.21.15.18:0/2472039812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c77858b0 msgr2=0x5603c778bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa0137fe640 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c77858b0 0x5603c778bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa0137fe640 1 -- 172.21.15.18:0/2472039812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603c76e2320 con 0x5603c777cd30 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa0137fe640 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 0x5603c778b840 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fa008000c00 tx=0x7fa00802cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93005851ab77297 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:43.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7f9ff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7f9ff37fe640 1 -- 172.21.15.18:0/2472039812 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008034810 con 0x5603c777cd30 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7fa01d633740 1 -- 172.21.15.18:0/2472039812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603c7709900 con 0x5603c777cd30 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.756+0000 7f9ff37fe640 1 -- 172.21.15.18:0/2472039812 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa008000e40 con 0x5603c777cd30 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.757+0000 7f9ff37fe640 1 -- 172.21.15.18:0/2472039812 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa008034970 con 0x5603c777cd30 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.757+0000 7fa01d633740 10 client.19845 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.757+0000 7fa01d633740 1 -- 172.21.15.18:0/2472039812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5603c76a4910 con 0x5603c777cd30 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.757+0000 7f9ff37fe640 1 -- 172.21.15.18:0/2472039812 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5603c7709900 con 0x5603c777cd30 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.757+0000 7f9ff37fe640 1 -- 172.21.15.18:0/2472039812 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa008059020 con 0x5603c777cd30 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.757+0000 7fa01d633740 10 client.19845 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.757+0000 7fa01d633740 10 client.19845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.757+0000 7fa01d633740 20 client.19845 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.757+0000 7fa01d633740 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5603c77edb80 0x5603c77effa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.757+0000 7fa01d633740 4 client.19845 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T13:01:43.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.757+0000 7fa01d633740 1 -- 172.21.15.18:0/2472039812 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5603c73e3860 con 0x5603c77edb80 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.757+0000 7fa013fff640 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5603c77edb80 0x5603c77effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.758+0000 7f9ff37fe640 10 client.19845 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.758+0000 7fa013fff640 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5603c77edb80 0x5603c77effa0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77497a8094eeed3f server_cookie=5aceb30b055fc5ec in_seq=0 out_seq=0 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.758+0000 7f9ff37fe640 1 -- 172.21.15.18:0/2472039812 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5603c73e3860 con 0x5603c77edb80 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.758+0000 7f9ff37fe640 10 client.19845 handle_command_reply: tid=0 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.758+0000 7fa01d633740 1 client.19845 shutdown 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.759+0000 7fa01d633740 1 -- 172.21.15.18:0/2472039812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 msgr2=0x5603c778b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.759+0000 7fa01d633740 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 0x5603c778b840 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fa008000c00 tx=0x7fa00802cd00 comp rx=0 tx=0).stop 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.759+0000 7fa01d633740 1 -- 172.21.15.18:0/2472039812 shutdown_connections 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.759+0000 7fa01d633740 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5603c77edb80 0x5603c77effa0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.759+0000 7fa01d633740 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c77858b0 0x5603c778bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.759+0000 7fa01d633740 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603c777cd30 0x5603c778b840 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.759+0000 7f9ff37fe640 0 client.19845 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.759+0000 7fa01d633740 1 --2- 172.21.15.18:0/2472039812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603c777c380 0x5603c778b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.759+0000 7fa01d633740 1 -- 172.21.15.18:0/2472039812 >> 172.21.15.18:0/2472039812 conn(0x5603c7776840 msgr2=0x5603c7784030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.759+0000 7fa01d633740 1 -- 172.21.15.18:0/2472039812 shutdown_connections 2024-02-17T13:01:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.759+0000 7fa01d633740 1 -- 172.21.15.18:0/2472039812 wait complete. 2024-02-17T13:01:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.759+0000 7fa01d633740 20 client.19845 trim_cache size 0 max 16384 2024-02-17T13:01:43.755 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:01:43.755 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T13:01:43.755 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "c7aaf3ee-5c8f-4478-80e6-77281280af77", 2024-02-17T13:01:43.755 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T13:01:43.755 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:01:43.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.760+0000 7fa01a98a640 1 -- 172.21.15.18:0/2638299650 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f9ffc0a2260 msgr2=0x7f9ffc0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.760+0000 7fa01a98a640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f9ffc0a2260 0x7f9ffc0a4720 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa00c010c40 tx=0x7fa00c00c000 comp rx=0 tx=0).stop 2024-02-17T13:01:43.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.760+0000 7fa01a98a640 1 -- 172.21.15.18:0/2638299650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0140ff890 msgr2=0x7fa014100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.760+0000 7fa01a98a640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0140ff890 0x7fa014100800 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fa00400e9f0 tx=0x7fa00400edf0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.760+0000 7fa01a98a640 1 -- 172.21.15.18:0/2638299650 shutdown_connections 2024-02-17T13:01:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.760+0000 7fa01a98a640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f9ffc0a2260 0x7f9ffc0a4720 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.760+0000 7fa01a98a640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014101280 0x7fa0141a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.760+0000 7fa01a98a640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa014106450 0x7fa014100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.760+0000 7fa01a98a640 1 --2- 172.21.15.18:0/2638299650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0140ff890 0x7fa014100800 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.760+0000 7fa01a98a640 1 -- 172.21.15.18:0/2638299650 >> 172.21.15.18:0/2638299650 conn(0x7fa0140765a0 msgr2=0x7fa014104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.760+0000 7fa01a98a640 1 -- 172.21.15.18:0/2638299650 shutdown_connections 2024-02-17T13:01:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.760+0000 7fa01a98a640 1 -- 172.21.15.18:0/2638299650 wait complete. 2024-02-17T13:01:43.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:01:43.761 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:01:43.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.883+0000 7fc2a39b5640 1 -- 172.21.15.18:0/353221715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c100c60 msgr2=0x7fc29c104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.883+0000 7fc2a39b5640 1 --2- 172.21.15.18:0/353221715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c100c60 0x7fc29c104ca0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009930 tx=0x7fc28c02bc30 comp rx=0 tx=0).stop 2024-02-17T13:01:43.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.883+0000 7fc2a39b5640 1 -- 172.21.15.18:0/353221715 shutdown_connections 2024-02-17T13:01:43.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.883+0000 7fc2a39b5640 1 --2- 172.21.15.18:0/353221715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc29c105380 0x7fc29c10cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.883+0000 7fc2a39b5640 1 --2- 172.21.15.18:0/353221715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c100c60 0x7fc29c104ca0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.883+0000 7fc2a39b5640 1 --2- 172.21.15.18:0/353221715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c1002b0 0x7fc29c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.883+0000 7fc2a39b5640 1 -- 172.21.15.18:0/353221715 >> 172.21.15.18:0/353221715 conn(0x7fc29c077190 msgr2=0x7fc29c0775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:43.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.883+0000 7fc2a39b5640 1 -- 172.21.15.18:0/353221715 shutdown_connections 2024-02-17T13:01:43.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.883+0000 7fc2a39b5640 1 -- 172.21.15.18:0/353221715 wait complete. 2024-02-17T13:01:43.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a39b5640 1 Processor -- start 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a39b5640 1 -- start start 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a39b5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c1002b0 0x7fc29c196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a39b5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc29c105380 0x7fc29c196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a39b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c19afe0 0x7fc29c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a39b5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c19b9d0 con 0x7fc29c1002b0 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a39b5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c19bb40 con 0x7fc29c19afe0 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a39b5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c19bcb0 con 0x7fc29c105380 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a1751640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c1002b0 0x7fc29c196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a0f50640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc29c105380 0x7fc29c196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a0f50640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc29c105380 0x7fc29c196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:44796/0 (socket says 172.21.15.18:44796) 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a1751640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c1002b0 0x7fc29c196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41370/0 (socket says 172.21.15.18:41370) 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a0f50640 1 -- 172.21.15.18:0/2700775416 learned_addr learned my addr 172.21.15.18:0/2700775416 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a1f52640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c19afe0 0x7fc29c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a0f50640 1 -- 172.21.15.18:0/2700775416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c19afe0 msgr2=0x7fc29c19b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c19afe0 0x7fc29c19b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a0f50640 1 -- 172.21.15.18:0/2700775416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c1002b0 msgr2=0x7fc29c196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c1002b0 0x7fc29c196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a0f50640 1 -- 172.21.15.18:0/2700775416 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c009560 con 0x7fc29c105380 2024-02-17T13:01:43.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a1f52640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c19afe0 0x7fc29c19b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:43.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.884+0000 7fc2a1751640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c1002b0 0x7fc29c196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:43.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.885+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc29c105380 0x7fc29c196c60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009230 tx=0x7fc28c02c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:43.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.885+0000 7fc2927fc640 1 -- 172.21.15.18:0/2700775416 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c004350 con 0x7fc29c105380 2024-02-17T13:01:43.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.885+0000 7fc2927fc640 1 -- 172.21.15.18:0/2700775416 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc28c0044b0 con 0x7fc29c105380 2024-02-17T13:01:43.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.885+0000 7fc2a39b5640 1 -- 172.21.15.18:0/2700775416 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29c0697b0 con 0x7fc29c105380 2024-02-17T13:01:43.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.885+0000 7fc2927fc640 1 -- 172.21.15.18:0/2700775416 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c003da0 con 0x7fc29c105380 2024-02-17T13:01:43.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.885+0000 7fc2a39b5640 1 -- 172.21.15.18:0/2700775416 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29c069cc0 con 0x7fc29c105380 2024-02-17T13:01:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a6588740 1 Processor -- start 2024-02-17T13:01:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a6588740 1 -- start start 2024-02-17T13:01:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2927fc640 1 -- 172.21.15.18:0/2700775416 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc28c004df0 con 0x7fc29c105380 2024-02-17T13:01:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a6588740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 0x55a1446d1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a6588740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 0x55a1446da270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a6588740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446dcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a6588740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a144640280 con 0x55a1446d1bd0 2024-02-17T13:01:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a6588740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14463d7b0 con 0x55a1446d12b0 2024-02-17T13:01:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a6588740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a144638f90 con 0x55a1446da7b0 2024-02-17T13:01:43.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a1751640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 0x55a1446da270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a1f52640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 0x55a1446d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a1f52640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 0x55a1446d1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:54410/0 (socket says 172.21.15.18:54410) 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a1f52640 1 -- 172.21.15.18:0/3353806478 learned_addr learned my addr 172.21.15.18:0/3353806478 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a1751640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 0x55a1446da270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41372/0 (socket says 172.21.15.18:41372) 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/3353806478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446dcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a1f52640 1 -- 172.21.15.18:0/3353806478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 msgr2=0x55a1446dcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a1f52640 1 --2- 172.21.15.18:0/3353806478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446dcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a1f52640 1 -- 172.21.15.18:0/3353806478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 msgr2=0x55a1446da270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a1f52640 1 --2- 172.21.15.18:0/3353806478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 0x55a1446da270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a1f52640 1 -- 172.21.15.18:0/3353806478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1445f7570 con 0x55a1446d12b0 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.888+0000 7fc2a1751640 1 --2- 172.21.15.18:0/3353806478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 0x55a1446da270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.889+0000 7fc2927fc640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc2880a2230 0x7fc2880a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.889+0000 7fc2a1751640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc2880a2230 0x7fc2880a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.889+0000 7fc2927fc640 1 -- 172.21.15.18:0/2700775416 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc28c0effb0 con 0x7fc29c105380 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.889+0000 7fc2a1f52640 1 --2- 172.21.15.18:0/3353806478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 0x55a1446d1690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc294011440 tx=0x7fc294011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ca2c01f8f5c4847 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.889+0000 7fc2877fe640 1 -- 172.21.15.18:0/3353806478 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294004020 con 0x55a1446d12b0 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.889+0000 7fc2877fe640 1 -- 172.21.15.18:0/3353806478 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc294004180 con 0x55a1446d12b0 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.889+0000 7fc2877fe640 1 -- 172.21.15.18:0/3353806478 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29400f820 con 0x55a1446d12b0 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.890+0000 7fc2a6588740 1 -- 172.21.15.18:0/3353806478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 msgr2=0x55a1446d1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.890+0000 7fc2a1751640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc2880a2230 0x7fc2880a46f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55a1446d0d80 tx=0x7fc298012040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.890+0000 7fc2a6588740 1 --2- 172.21.15.18:0/3353806478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 0x55a1446d1690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc294011440 tx=0x7fc294011900 comp rx=0 tx=0).stop 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.890+0000 7fc2a6588740 1 -- 172.21.15.18:0/3353806478 shutdown_connections 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.890+0000 7fc2a6588740 1 --2- 172.21.15.18:0/3353806478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446dcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.890+0000 7fc2a6588740 1 --2- 172.21.15.18:0/3353806478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 0x55a1446da270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.890+0000 7fc2a6588740 1 --2- 172.21.15.18:0/3353806478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 0x55a1446d1690 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.890+0000 7fc2a6588740 1 -- 172.21.15.18:0/3353806478 >> 172.21.15.18:0/3353806478 conn(0x55a1446cb990 msgr2=0x55a1446cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.890+0000 7fc2a6588740 1 -- 172.21.15.18:0/3353806478 shutdown_connections 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.890+0000 7fc2a6588740 1 -- 172.21.15.18:0/3353806478 wait complete. 2024-02-17T13:01:43.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.890+0000 7fc2a6588740 1 -- start start 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.890+0000 7fc2a6588740 1 Processor -- start 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a6588740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 0x55a1446d7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a6588740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d1bd0 0x55a1446d7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a6588740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446d80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a6588740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14430c430 con 0x55a1446d12b0 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a6588740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1442c8e00 con 0x55a1446d1bd0 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a6588740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1445bdfd0 con 0x55a1446da7b0 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a6588740 10 client.0 _ll_register_callbacks cb 0x55a144613fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a1751640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 0x55a1446d7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a1751640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 0x55a1446d7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41384/0 (socket says 172.21.15.18:41384) 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a1751640 1 -- 172.21.15.18:0/1884994520 learned_addr learned my addr 172.21.15.18:0/1884994520 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a1f52640 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446d80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d1bd0 0x55a1446d7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a1751640 1 -- 172.21.15.18:0/1884994520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 msgr2=0x55a1446d80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a1751640 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446d80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a1751640 1 -- 172.21.15.18:0/1884994520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d1bd0 msgr2=0x55a1446d7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a1751640 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d1bd0 0x55a1446d7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a1751640 1 -- 172.21.15.18:0/1884994520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc294009cc0 con 0x55a1446d12b0 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a1751640 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 0x55a1446d7630 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fc29801c700 tx=0x7fc298004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4939d0a040797ff5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc284ff9640 1 -- 172.21.15.18:0/1884994520 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298015040 con 0x55a1446d12b0 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc284ff9640 1 -- 172.21.15.18:0/1884994520 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc298004b20 con 0x55a1446d12b0 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc2a6588740 1 -- 172.21.15.18:0/1884994520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1443387e0 con 0x55a1446d12b0 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.891+0000 7fc284ff9640 1 -- 172.21.15.18:0/1884994520 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc298004ce0 con 0x55a1446d12b0 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.892+0000 7fc2a6588740 10 client.19857 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.892+0000 7fc2a6588740 1 -- 172.21.15.18:0/1884994520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a144637610 con 0x55a1446d12b0 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.892+0000 7fc284ff9640 1 -- 172.21.15.18:0/1884994520 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a1443387e0 con 0x55a1446d12b0 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.892+0000 7fc284ff9640 1 -- 172.21.15.18:0/1884994520 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc2980265b0 con 0x55a1446d12b0 2024-02-17T13:01:43.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.892+0000 7fc2a6588740 10 client.19857 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.892+0000 7fc2a6588740 10 client.19857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.892+0000 7fc2a6588740 20 client.19857 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.892+0000 7fc2a6588740 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a1446eab00 0x55a1446ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.892+0000 7fc2a6588740 4 client.19857 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.892+0000 7fc2a6588740 1 -- 172.21.15.18:0/1884994520 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a144375720 con 0x55a1446eab00 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.892+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a1446eab00 0x55a1446ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.893+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a1446eab00 0x55a1446ecf20 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7bfeb89b3053219 server_cookie=bc6ad66d229ed29d in_seq=0 out_seq=0 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.893+0000 7fc284ff9640 10 client.19857 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.893+0000 7fc284ff9640 1 -- 172.21.15.18:0/1884994520 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a144375720 con 0x55a1446eab00 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.893+0000 7fc284ff9640 10 client.19857 handle_command_reply: tid=0 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.893+0000 7fc2a6588740 1 client.19857 shutdown 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.893+0000 7fc2a6588740 1 -- 172.21.15.18:0/1884994520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 msgr2=0x55a1446d7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.893+0000 7fc2a6588740 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 0x55a1446d7630 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fc29801c700 tx=0x7fc298004ab0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.893+0000 7fc2a6588740 1 -- 172.21.15.18:0/1884994520 shutdown_connections 2024-02-17T13:01:43.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.893+0000 7fc2a6588740 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a1446eab00 0x55a1446ecf20 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.894+0000 7fc2a6588740 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446d80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.894+0000 7fc2a6588740 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d1bd0 0x55a1446d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.894+0000 7fc284ff9640 0 client.19857 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:43.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.894+0000 7fc2a6588740 1 --2- 172.21.15.18:0/1884994520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 0x55a1446d7630 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.894+0000 7fc2a6588740 1 -- 172.21.15.18:0/1884994520 >> 172.21.15.18:0/1884994520 conn(0x55a1446cb990 msgr2=0x55a1446cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:43.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.894+0000 7fc2a6588740 1 -- 172.21.15.18:0/1884994520 shutdown_connections 2024-02-17T13:01:43.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.894+0000 7fc2a6588740 1 -- 172.21.15.18:0/1884994520 wait complete. 2024-02-17T13:01:43.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.894+0000 7fc2a6588740 20 client.19857 trim_cache size 0 max 16384 2024-02-17T13:01:43.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a6588740 1 Processor -- start 2024-02-17T13:01:43.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a6588740 1 -- start start 2024-02-17T13:01:43.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a6588740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 0x55a1446d9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a6588740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 0x55a1446d7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a6588740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446d6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a6588740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1446d3250 con 0x55a1446d1bd0 2024-02-17T13:01:43.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a6588740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1446d0f10 con 0x55a1446d12b0 2024-02-17T13:01:43.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a6588740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1446cc9e0 con 0x55a1446da7b0 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a0f50640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 0x55a1446d7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a1751640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 0x55a1446d9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a0f50640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 0x55a1446d7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41400/0 (socket says 172.21.15.18:41400) 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a0f50640 1 -- 172.21.15.18:0/2333971294 learned_addr learned my addr 172.21.15.18:0/2333971294 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.910+0000 7fc2a1f52640 1 --2- 172.21.15.18:0/2333971294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446d6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.911+0000 7fc2a0f50640 1 -- 172.21.15.18:0/2333971294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 msgr2=0x55a1446d6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.911+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/2333971294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446d6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.911+0000 7fc2a0f50640 1 -- 172.21.15.18:0/2333971294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 msgr2=0x55a1446d9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.911+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/2333971294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 0x55a1446d9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.911+0000 7fc2a0f50640 1 -- 172.21.15.18:0/2333971294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1445d3ac0 con 0x55a1446d1bd0 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.911+0000 7fc2a1751640 1 --2- 172.21.15.18:0/2333971294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 0x55a1446d9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.911+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/2333971294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 0x55a1446d7ed0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0b94a0 tx=0x7fc28c0f3ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a0325518137a43 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.911+0000 7fc286ffd640 1 -- 172.21.15.18:0/2333971294 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0bc610 con 0x55a1446d1bd0 2024-02-17T13:01:43.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.911+0000 7fc286ffd640 1 -- 172.21.15.18:0/2333971294 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc28c0bc430 con 0x55a1446d1bd0 2024-02-17T13:01:43.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.911+0000 7fc286ffd640 1 -- 172.21.15.18:0/2333971294 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0c5270 con 0x55a1446d1bd0 2024-02-17T13:01:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.912+0000 7fc2a6588740 1 -- 172.21.15.18:0/2333971294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 msgr2=0x55a1446d7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.912+0000 7fc2a6588740 1 --2- 172.21.15.18:0/2333971294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 0x55a1446d7ed0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0b94a0 tx=0x7fc28c0f3ac0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.912+0000 7fc2a6588740 1 -- 172.21.15.18:0/2333971294 shutdown_connections 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.912+0000 7fc2a6588740 1 --2- 172.21.15.18:0/2333971294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446d6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.912+0000 7fc2a6588740 1 --2- 172.21.15.18:0/2333971294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d1bd0 0x55a1446d7ed0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.912+0000 7fc2a6588740 1 --2- 172.21.15.18:0/2333971294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d12b0 0x55a1446d9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.912+0000 7fc2a6588740 1 -- 172.21.15.18:0/2333971294 >> 172.21.15.18:0/2333971294 conn(0x55a1446cb990 msgr2=0x55a1446d5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.912+0000 7fc2a6588740 1 -- 172.21.15.18:0/2333971294 shutdown_connections 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.912+0000 7fc2a6588740 1 -- 172.21.15.18:0/2333971294 wait complete. 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.912+0000 7fc2a6588740 1 -- start start 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.912+0000 7fc2a6588740 1 Processor -- start 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a6588740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 0x55a1446e0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a6588740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d1bd0 0x55a1446e07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a6588740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446e0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a6588740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a144741140 con 0x55a1446d12b0 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a6588740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1447412b0 con 0x55a1446d1bd0 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a6588740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a144741470 con 0x55a1446da7b0 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a6588740 10 client.0 _ll_register_callbacks cb 0x7fc294009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a1751640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 0x55a1446e0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a1751640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 0x55a1446e0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41414/0 (socket says 172.21.15.18:41414) 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a1751640 1 -- 172.21.15.18:0/2186898803 learned_addr learned my addr 172.21.15.18:0/2186898803 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d1bd0 0x55a1446e07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a1f52640 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446e0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a1751640 1 -- 172.21.15.18:0/2186898803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 msgr2=0x55a1446e0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a1751640 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446e0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a1751640 1 -- 172.21.15.18:0/2186898803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d1bd0 msgr2=0x55a1446e07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a1751640 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d1bd0 0x55a1446e07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a1751640 1 -- 172.21.15.18:0/2186898803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a144637610 con 0x55a1446d12b0 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d1bd0 0x55a1446e07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:43.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc2a1751640 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 0x55a1446e0290 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fc298009690 tx=0x7fc298026b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef34f5793f27d08 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.913+0000 7fc284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc284ff9640 1 -- 172.21.15.18:0/2186898803 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298015040 con 0x55a1446d12b0 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc284ff9640 1 -- 172.21.15.18:0/2186898803 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc298026da0 con 0x55a1446d12b0 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc2a6588740 1 -- 172.21.15.18:0/2186898803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc294004180 con 0x55a1446d12b0 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc284ff9640 1 -- 172.21.15.18:0/2186898803 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc298026600 con 0x55a1446d12b0 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc2a6588740 10 client.19863 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc2a6588740 1 -- 172.21.15.18:0/2186898803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fc29c069cc0 con 0x55a1446d12b0 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc284ff9640 1 -- 172.21.15.18:0/2186898803 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc294004180 con 0x55a1446d12b0 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc284ff9640 1 -- 172.21.15.18:0/2186898803 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc29801a4b0 con 0x55a1446d12b0 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc2a6588740 10 client.19863 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc2a6588740 10 client.19863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc2a6588740 20 client.19863 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc2a6588740 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a144742c20 0x55a144745040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:43.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc2a6588740 4 client.19863 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc2a6588740 1 -- 172.21.15.18:0/2186898803 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a1443c4890 con 0x55a144742c20 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.914+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a144742c20 0x55a144745040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc2a0f50640 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a144742c20 0x55a144745040 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d929f1893cc2837 server_cookie=282508380de9b4aa in_seq=0 out_seq=0 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc284ff9640 10 client.19863 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc284ff9640 1 -- 172.21.15.18:0/2186898803 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a1443c4890 con 0x55a144742c20 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc284ff9640 10 client.19863 handle_command_reply: tid=0 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc2a6588740 1 client.19863 shutdown 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc2a6588740 1 -- 172.21.15.18:0/2186898803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 msgr2=0x55a1446e0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc2a6588740 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 0x55a1446e0290 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fc298009690 tx=0x7fc298026b80 comp rx=0 tx=0).stop 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc2a6588740 1 -- 172.21.15.18:0/2186898803 shutdown_connections 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc2a6588740 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a144742c20 0x55a144745040 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc2a6588740 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a1446da7b0 0x55a1446e0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc284ff9640 0 client.19863 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc2a6588740 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1446d1bd0 0x55a1446e07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc2a6588740 1 --2- 172.21.15.18:0/2186898803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a1446d12b0 0x55a1446e0290 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc2a6588740 1 -- 172.21.15.18:0/2186898803 >> 172.21.15.18:0/2186898803 conn(0x55a1446cb990 msgr2=0x55a1446cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:43.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc2a6588740 1 -- 172.21.15.18:0/2186898803 shutdown_connections 2024-02-17T13:01:43.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.915+0000 7fc2a6588740 1 -- 172.21.15.18:0/2186898803 wait complete. 2024-02-17T13:01:43.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a6588740 20 client.19863 trim_cache size 0 max 16384 2024-02-17T13:01:43.912 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:01:43.912 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T13:01:43.912 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T13:01:43.912 INFO:teuthology.orchestra.run.smithi018.stdout: "c7aaf3ee-5c8f-4478-80e6-77281280af77": { 2024-02-17T13:01:43.912 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T13:01:43.912 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "c7aaf3ee-5c8f-4478-80e6-77281280af77", 2024-02-17T13:01:43.912 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T13:01:43.913 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:01:43.913 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:01:43.913 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:01:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a39b5640 1 -- 172.21.15.18:0/2700775416 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc2880a2230 msgr2=0x7fc2880a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a39b5640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc2880a2230 0x7fc2880a46f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55a1446d0d80 tx=0x7fc298012040 comp rx=0 tx=0).stop 2024-02-17T13:01:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a39b5640 1 -- 172.21.15.18:0/2700775416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc29c105380 msgr2=0x7fc29c196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a39b5640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc29c105380 0x7fc29c196c60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009230 tx=0x7fc28c02c680 comp rx=0 tx=0).stop 2024-02-17T13:01:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a39b5640 1 -- 172.21.15.18:0/2700775416 shutdown_connections 2024-02-17T13:01:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a39b5640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc2880a2230 0x7fc2880a46f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a39b5640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc29c19afe0 0x7fc29c19b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a39b5640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc29c105380 0x7fc29c196c60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a39b5640 1 --2- 172.21.15.18:0/2700775416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c1002b0 0x7fc29c196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a39b5640 1 -- 172.21.15.18:0/2700775416 >> 172.21.15.18:0/2700775416 conn(0x7fc29c077190 msgr2=0x7fc29c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a39b5640 1 -- 172.21.15.18:0/2700775416 shutdown_connections 2024-02-17T13:01:43.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:01:43.916+0000 7fc2a39b5640 1 -- 172.21.15.18:0/2700775416 wait complete. 2024-02-17T13:01:43.917 INFO:tasks.cephfs.filesystem:scrub status for tag:c7aaf3ee-5c8f-4478-80e6-77281280af77 - {'path': '/', 'tag': 'c7aaf3ee-5c8f-4478-80e6-77281280af77', 'options': 'recursive,force'} 2024-02-17T13:01:44.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 589625 4.87 MB/sec execute 539 sec latency 3634.917 ms 2024-02-17T13:01:44.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: pgmap v371: 97 pgs: 97 active+clean; 232 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-02-17T13:01:44.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T13:01:44.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:44.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:44 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: pgmap v371: 97 pgs: 97 active+clean; 232 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-02-17T13:01:44.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T13:01:44.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:44.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:44.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:44.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:44.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:44.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:44.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:44.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:44.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:44 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: pgmap v371: 97 pgs: 97 active+clean; 232 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-02-17T13:01:44.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:44.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:44.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:44.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:44 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:45.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 591073 4.88 MB/sec execute 540 sec latency 15.964 ms 2024-02-17T13:01:45.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:45 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:45.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:45 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:45.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:45 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:01:45.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:45 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:45.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:45 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:45.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:45 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:45.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:45 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:01:45.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:45 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:45.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:45 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:45.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:45 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:45.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:45 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:01:45.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:45 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:01:46.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 591312 4.87 MB/sec execute 541 sec latency 854.310 ms 2024-02-17T13:01:46.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:46 smithi018 ceph-mon[32625]: pgmap v372: 97 pgs: 97 active+clean; 231 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-02-17T13:01:46.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:46 smithi119 ceph-mon[36188]: pgmap v372: 97 pgs: 97 active+clean; 231 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-02-17T13:01:46.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:46 smithi146 ceph-mon[36454]: pgmap v372: 97 pgs: 97 active+clean; 231 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-02-17T13:01:47.300 INFO:tasks.workunit.client.0.smithi018.stdout: 1 592217 4.86 MB/sec execute 542 sec latency 1402.221 ms 2024-02-17T13:01:48.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 593843 4.88 MB/sec execute 543 sec latency 17.250 ms 2024-02-17T13:01:48.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:48 smithi018 ceph-mon[32625]: pgmap v373: 97 pgs: 97 active+clean; 231 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-02-17T13:01:48.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:48 smithi119 ceph-mon[36188]: pgmap v373: 97 pgs: 97 active+clean; 231 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-02-17T13:01:48.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:48 smithi146 ceph-mon[36454]: pgmap v373: 97 pgs: 97 active+clean; 231 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-02-17T13:01:49.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 595914 4.89 MB/sec execute 544 sec latency 28.380 ms 2024-02-17T13:01:50.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 597492 4.89 MB/sec execute 545 sec latency 12.655 ms 2024-02-17T13:01:50.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:50 smithi018 ceph-mon[32625]: pgmap v374: 97 pgs: 97 active+clean; 236 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-02-17T13:01:50.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:50 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:01:50.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:50 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:01:50.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:50 smithi119 ceph-mon[36188]: pgmap v374: 97 pgs: 97 active+clean; 236 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-02-17T13:01:50.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:50 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:01:50.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:50 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:01:50.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:50 smithi146 ceph-mon[36454]: pgmap v374: 97 pgs: 97 active+clean; 236 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-02-17T13:01:50.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:50 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:01:50.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:50 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:01:51.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 598403 4.89 MB/sec execute 546 sec latency 428.765 ms 2024-02-17T13:01:51.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:51 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:01:51.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:51 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:01:51.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:51 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:01:51.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:51 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:01:51.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:51 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:01:51.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:51 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:01:51.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:51 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:01:51.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:51 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:01:51.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:51 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:01:52.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 598403 4.88 MB/sec execute 547 sec latency 1428.853 ms 2024-02-17T13:01:52.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:52 smithi018 ceph-mon[32625]: pgmap v375: 97 pgs: 97 active+clean; 236 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-02-17T13:01:52.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:52 smithi119 ceph-mon[36188]: pgmap v375: 97 pgs: 97 active+clean; 236 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-02-17T13:01:52.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:52 smithi146 ceph-mon[36454]: pgmap v375: 97 pgs: 97 active+clean; 236 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-02-17T13:01:53.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 599319 4.88 MB/sec execute 548 sec latency 1988.714 ms 2024-02-17T13:01:54.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 600948 4.89 MB/sec execute 549 sec latency 18.303 ms 2024-02-17T13:01:54.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:54 smithi018 ceph-mon[32625]: pgmap v376: 97 pgs: 97 active+clean; 236 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-02-17T13:01:54.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:54 smithi119 ceph-mon[36188]: pgmap v376: 97 pgs: 97 active+clean; 236 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-02-17T13:01:54.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:54 smithi146 ceph-mon[36454]: pgmap v376: 97 pgs: 97 active+clean; 236 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-02-17T13:01:55.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 601947 4.89 MB/sec execute 550 sec latency 438.769 ms 2024-02-17T13:01:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.613+0000 7f24fac4e640 1 -- 172.21.15.18:0/565368923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24f4073520 msgr2=0x7f24f4073900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.613+0000 7f24fac4e640 1 --2- 172.21.15.18:0/565368923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24f4073520 0x7f24f4073900 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009ca0 tx=0x7f24f002c180 comp rx=0 tx=0).stop 2024-02-17T13:01:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.614+0000 7f24fac4e640 1 -- 172.21.15.18:0/565368923 shutdown_connections 2024-02-17T13:01:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.614+0000 7f24fac4e640 1 --2- 172.21.15.18:0/565368923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f4112160 0x7f24f4114550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.614+0000 7f24fac4e640 1 --2- 172.21.15.18:0/565368923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f4073e40 0x7f24f4111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.614+0000 7f24fac4e640 1 --2- 172.21.15.18:0/565368923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24f4073520 0x7f24f4073900 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.614+0000 7f24fac4e640 1 -- 172.21.15.18:0/565368923 >> 172.21.15.18:0/565368923 conn(0x7f24f40fcc60 msgr2=0x7f24f40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.614+0000 7f24fac4e640 1 -- 172.21.15.18:0/565368923 shutdown_connections 2024-02-17T13:01:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.614+0000 7f24fac4e640 1 -- 172.21.15.18:0/565368923 wait complete. 2024-02-17T13:01:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.614+0000 7f24fac4e640 1 Processor -- start 2024-02-17T13:01:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.614+0000 7f24fac4e640 1 -- start start 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24fac4e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f4073520 0x7f24f419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24fac4e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f4073e40 0x7f24f419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24fac4e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24f4112160 0x7f24f41a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24fac4e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f41a2be0 con 0x7f24f4112160 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24fac4e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f41a2d50 con 0x7f24f4073520 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24fac4e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f41a2f10 con 0x7f24f4073e40 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24f91eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24f4112160 0x7f24f41a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24f89ea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f4073520 0x7f24f419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24f91eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24f4112160 0x7f24f41a25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32888/0 (socket says 172.21.15.18:32888) 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24f91eb640 1 -- 172.21.15.18:0/3674909933 learned_addr learned my addr 172.21.15.18:0/3674909933 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24ebfff640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f4073e40 0x7f24f419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24f89ea640 1 -- 172.21.15.18:0/3674909933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f4073e40 msgr2=0x7f24f419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24f89ea640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f4073e40 0x7f24f419e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24f89ea640 1 -- 172.21.15.18:0/3674909933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24f4112160 msgr2=0x7f24f41a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24f89ea640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24f4112160 0x7f24f41a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24f89ea640 1 -- 172.21.15.18:0/3674909933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e4009490 con 0x7f24f4073520 2024-02-17T13:01:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24f91eb640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24f4112160 0x7f24f41a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24f89ea640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f4073520 0x7f24f419dd20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f24f0006010 tx=0x7f24f00346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.615+0000 7f24e9ffb640 1 -- 172.21.15.18:0/3674909933 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0003d80 con 0x7f24f4073520 2024-02-17T13:01:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.616+0000 7f24e9ffb640 1 -- 172.21.15.18:0/3674909933 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24f00348e0 con 0x7f24f4073520 2024-02-17T13:01:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.616+0000 7f24fac4e640 1 -- 172.21.15.18:0/3674909933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f0009900 con 0x7f24f4073520 2024-02-17T13:01:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.616+0000 7f24e9ffb640 1 -- 172.21.15.18:0/3674909933 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0004790 con 0x7f24f4073520 2024-02-17T13:01:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.616+0000 7f24fac4e640 1 -- 172.21.15.18:0/3674909933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f41aa890 con 0x7f24f4073520 2024-02-17T13:01:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.618+0000 7f24e9ffb640 1 -- 172.21.15.18:0/3674909933 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f24f000a040 con 0x7f24f4073520 2024-02-17T13:01:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.618+0000 7f24fd827740 1 Processor -- start 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.618+0000 7f24fd827740 1 -- start start 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.618+0000 7f24fd827740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f1ad0 0x55d3852f0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.618+0000 7f24e9ffb640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f24dc0a22d0 0x7f24dc0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24fd827740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 0x55d3852f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24fd827740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852fa2e0 0x55d3852fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24fd827740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38525f9f0 con 0x55d3852f0d60 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24fd827740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38525cf20 con 0x55d3852f1ad0 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24fd827740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d385258700 con 0x55d3852fa2e0 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24ebfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 0x55d3852f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24ebfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 0x55d3852f11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32900/0 (socket says 172.21.15.18:32900) 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24f89ea640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f24dc0a22d0 0x7f24dc0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24ebfff640 1 -- 172.21.15.18:0/2191620946 learned_addr learned my addr 172.21.15.18:0/2191620946 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24e9ffb640 1 -- 172.21.15.18:0/3674909933 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f24f00f0990 con 0x7f24f4073520 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24f91eb640 1 --2- 172.21.15.18:0/2191620946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f1ad0 0x55d3852f0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24f91eb640 1 --2- 172.21.15.18:0/2191620946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852fa2e0 0x55d3852fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24ebfff640 1 -- 172.21.15.18:0/2191620946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852fa2e0 msgr2=0x55d3852fc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24ebfff640 1 --2- 172.21.15.18:0/2191620946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852fa2e0 0x55d3852fc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24ebfff640 1 -- 172.21.15.18:0/2191620946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f1ad0 msgr2=0x55d3852f0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24ebfff640 1 --2- 172.21.15.18:0/2191620946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f1ad0 0x55d3852f0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24ebfff640 1 -- 172.21.15.18:0/2191620946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3851ee7b0 con 0x55d3852f0d60 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24ebfff640 1 --2- 172.21.15.18:0/2191620946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 0x55d3852f11e0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009e20 tx=0x7f24e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8355ee5ef24c7a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24f89ea640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f24dc0a22d0 0x7f24dc0a4790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f24f00bd1b0 tx=0x7f24f008c650 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24daffd640 1 -- 172.21.15.18:0/2191620946 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e4019410 con 0x55d3852f0d60 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24daffd640 1 -- 172.21.15.18:0/2191620946 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24e4019a50 con 0x55d3852f0d60 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.619+0000 7f24daffd640 1 -- 172.21.15.18:0/2191620946 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e4018b30 con 0x55d3852f0d60 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.620+0000 7f24fd827740 1 -- 172.21.15.18:0/2191620946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 msgr2=0x55d3852f11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.620+0000 7f24fd827740 1 --2- 172.21.15.18:0/2191620946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 0x55d3852f11e0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009e20 tx=0x7f24e4011810 comp rx=0 tx=0).stop 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.620+0000 7f24fd827740 1 -- 172.21.15.18:0/2191620946 shutdown_connections 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.620+0000 7f24fd827740 1 --2- 172.21.15.18:0/2191620946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852fa2e0 0x55d3852fc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.620+0000 7f24fd827740 1 --2- 172.21.15.18:0/2191620946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 0x55d3852f11e0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.620+0000 7f24fd827740 1 --2- 172.21.15.18:0/2191620946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f1ad0 0x55d3852f0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.620+0000 7f24fd827740 1 -- 172.21.15.18:0/2191620946 >> 172.21.15.18:0/2191620946 conn(0x55d3852b8550 msgr2=0x55d3852eba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.620+0000 7f24fd827740 1 -- 172.21.15.18:0/2191620946 shutdown_connections 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.620+0000 7f24fd827740 1 -- 172.21.15.18:0/2191620946 wait complete. 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.620+0000 7f24fd827740 1 -- start start 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24fd827740 1 Processor -- start 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24fd827740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f0d60 0x55d3852f7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24fd827740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852f7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24fd827740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 0x55d3852f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24fd827740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3851fdbf0 con 0x55d3852fa2e0 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24fd827740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d384f2c250 con 0x55d3852f0d60 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24fd827740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3851de200 con 0x55d3852f1ad0 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24fd827740 10 client.0 _ll_register_callbacks cb 0x55d38524cd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24f91eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 0x55d3852f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24f91eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 0x55d3852f7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32914/0 (socket says 172.21.15.18:32914) 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24f91eb640 1 -- 172.21.15.18:0/1021472709 learned_addr learned my addr 172.21.15.18:0/1021472709 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24ebfff640 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.621+0000 7f24f91eb640 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f0d60 0x55d3852f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24f91eb640 1 -- 172.21.15.18:0/1021472709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 msgr2=0x55d3852f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24f91eb640 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24f91eb640 1 -- 172.21.15.18:0/1021472709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f0d60 msgr2=0x55d3852f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24f91eb640 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f0d60 0x55d3852f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24f91eb640 1 -- 172.21.15.18:0/1021472709 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e4009a80 con 0x55d3852fa2e0 2024-02-17T13:01:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24ebfff640 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24f91eb640 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 0x55d3852f7dc0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00f4a0 tx=0x7f24ec009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cf9f49c2874064 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24d3fff640 1 -- 172.21.15.18:0/1021472709 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec016030 con 0x55d3852fa2e0 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24d3fff640 1 -- 172.21.15.18:0/1021472709 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f24ec005020 con 0x55d3852fa2e0 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24fd827740 1 -- 172.21.15.18:0/1021472709 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d384f58600 con 0x55d3852fa2e0 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24d3fff640 1 -- 172.21.15.18:0/1021472709 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24ec00fee0 con 0x55d3852fa2e0 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24fd827740 10 client.19875 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24fd827740 1 -- 172.21.15.18:0/1021472709 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55d385256d80 con 0x55d3852fa2e0 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24d3fff640 1 -- 172.21.15.18:0/1021472709 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d384f58600 con 0x55d3852fa2e0 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.622+0000 7f24d3fff640 1 -- 172.21.15.18:0/1021472709 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f24ec004d20 con 0x55d3852fa2e0 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.623+0000 7f24fd827740 10 client.19875 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.623+0000 7f24fd827740 10 client.19875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.623+0000 7f24fd827740 20 client.19875 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.623+0000 7f24fd827740 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d38530a6a0 0x55d38530cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.623+0000 7f24fd827740 4 client.19875 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.623+0000 7f24fd827740 1 -- 172.21.15.18:0/1021472709 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d384fa91b0 con 0x55d38530a6a0 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.623+0000 7f24ebfff640 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d38530a6a0 0x55d38530cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.623+0000 7f24ebfff640 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d38530a6a0 0x55d38530cac0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb4c3a1705c65324 server_cookie=87fea59f1e5e9400 in_seq=0 out_seq=0 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.623+0000 7f24d3fff640 10 client.19875 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24d3fff640 1 -- 172.21.15.18:0/1021472709 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d384fa91b0 con 0x55d38530a6a0 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24d3fff640 10 client.19875 handle_command_reply: tid=0 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24fd827740 1 client.19875 shutdown 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24fd827740 1 -- 172.21.15.18:0/1021472709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 msgr2=0x55d3852f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24fd827740 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 0x55d3852f7dc0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00f4a0 tx=0x7f24ec009eb0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24fd827740 1 -- 172.21.15.18:0/1021472709 shutdown_connections 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24fd827740 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d38530a6a0 0x55d38530cac0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24fd827740 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 0x55d3852f7dc0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24fd827740 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24fd827740 1 --2- 172.21.15.18:0/1021472709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f0d60 0x55d3852f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24d3fff640 0 client.19875 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24fd827740 1 -- 172.21.15.18:0/1021472709 >> 172.21.15.18:0/1021472709 conn(0x55d3852b8550 msgr2=0x55d3852ed7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24fd827740 1 -- 172.21.15.18:0/1021472709 shutdown_connections 2024-02-17T13:01:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24fd827740 1 -- 172.21.15.18:0/1021472709 wait complete. 2024-02-17T13:01:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.624+0000 7f24fd827740 20 client.19875 trim_cache size 0 max 16384 2024-02-17T13:01:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.637+0000 7f24fd827740 1 Processor -- start 2024-02-17T13:01:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24fd827740 1 -- start start 2024-02-17T13:01:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24fd827740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f0d60 0x55d3852f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24fd827740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24fd827740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 0x55d3852f6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24fd827740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d384f58600 con 0x55d3852fa2e0 2024-02-17T13:01:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24fd827740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3852f20c0 con 0x55d3852f0d60 2024-02-17T13:01:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24fd827740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3852eb310 con 0x55d3852f1ad0 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24f91eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 0x55d3852f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24f91eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 0x55d3852f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32918/0 (socket says 172.21.15.18:32918) 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24f91eb640 1 -- 172.21.15.18:0/430184299 learned_addr learned my addr 172.21.15.18:0/430184299 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24ebfff640 1 --2- 172.21.15.18:0/430184299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24f91eb640 1 --2- 172.21.15.18:0/430184299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f0d60 0x55d3852f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24f91eb640 1 -- 172.21.15.18:0/430184299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 msgr2=0x55d3852f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24f91eb640 1 --2- 172.21.15.18:0/430184299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24f91eb640 1 -- 172.21.15.18:0/430184299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f0d60 msgr2=0x55d3852f9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24f91eb640 1 --2- 172.21.15.18:0/430184299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f0d60 0x55d3852f9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24f91eb640 1 -- 172.21.15.18:0/430184299 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3851f37b0 con 0x55d3852fa2e0 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24f91eb640 1 --2- 172.21.15.18:0/430184299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 0x55d3852f6150 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f24ec011e40 tx=0x7f24ec011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455ee6819d6433b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24da7fc640 1 -- 172.21.15.18:0/430184299 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec016040 con 0x55d3852fa2e0 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24da7fc640 1 -- 172.21.15.18:0/430184299 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24ec0046c0 con 0x55d3852fa2e0 2024-02-17T13:01:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.638+0000 7f24da7fc640 1 -- 172.21.15.18:0/430184299 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00eea0 con 0x55d3852fa2e0 2024-02-17T13:01:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.639+0000 7f24fd827740 1 -- 172.21.15.18:0/430184299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 msgr2=0x55d3852f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.639+0000 7f24fd827740 1 --2- 172.21.15.18:0/430184299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 0x55d3852f6150 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f24ec011e40 tx=0x7f24ec011440 comp rx=0 tx=0).stop 2024-02-17T13:01:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.639+0000 7f24fd827740 1 -- 172.21.15.18:0/430184299 shutdown_connections 2024-02-17T13:01:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.639+0000 7f24fd827740 1 --2- 172.21.15.18:0/430184299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852fa2e0 0x55d3852f6150 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.639+0000 7f24fd827740 1 --2- 172.21.15.18:0/430184299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.639+0000 7f24fd827740 1 --2- 172.21.15.18:0/430184299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852f0d60 0x55d3852f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.639+0000 7f24fd827740 1 -- 172.21.15.18:0/430184299 >> 172.21.15.18:0/430184299 conn(0x55d3852b8550 msgr2=0x55d3852eb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.639+0000 7f24fd827740 1 -- 172.21.15.18:0/430184299 shutdown_connections 2024-02-17T13:01:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.639+0000 7f24fd827740 1 -- 172.21.15.18:0/430184299 wait complete. 2024-02-17T13:01:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.639+0000 7f24fd827740 1 -- start start 2024-02-17T13:01:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.639+0000 7f24fd827740 1 Processor -- start 2024-02-17T13:01:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24fd827740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 0x55d385301270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24fd827740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852ff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24fd827740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852fa2e0 0x55d3852ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24fd827740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d385300090 con 0x55d3852f0d60 2024-02-17T13:01:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24fd827740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d385300200 con 0x55d3852fa2e0 2024-02-17T13:01:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24fd827740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3853003c0 con 0x55d3852f1ad0 2024-02-17T13:01:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24fd827740 10 client.0 _ll_register_callbacks cb 0x7f24e4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24f91eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 0x55d385301270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24f91eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 0x55d385301270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32928/0 (socket says 172.21.15.18:32928) 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24f91eb640 1 -- 172.21.15.18:0/2203898739 learned_addr learned my addr 172.21.15.18:0/2203898739 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24ebfff640 1 --2- 172.21.15.18:0/2203898739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24f91eb640 1 -- 172.21.15.18:0/2203898739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 msgr2=0x55d3852ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24f91eb640 1 --2- 172.21.15.18:0/2203898739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24f91eb640 1 -- 172.21.15.18:0/2203898739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852fa2e0 msgr2=0x55d3852ffb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24f91eb640 1 --2- 172.21.15.18:0/2203898739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852fa2e0 0x55d3852ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24f91eb640 1 -- 172.21.15.18:0/2203898739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d385256d80 con 0x55d3852f0d60 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24f91eb640 1 --2- 172.21.15.18:0/2203898739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 0x55d385301270 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00e680 tx=0x7f24ec011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab58b2c9eb29f0f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.640+0000 7f24d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24d3fff640 1 -- 172.21.15.18:0/2203898739 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec016040 con 0x55d3852f0d60 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24d3fff640 1 -- 172.21.15.18:0/2203898739 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f24ec000e40 con 0x55d3852f0d60 2024-02-17T13:01:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24fd827740 1 -- 172.21.15.18:0/2203898739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24e4019a50 con 0x55d3852f0d60 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24d3fff640 1 -- 172.21.15.18:0/2203898739 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24ec00fee0 con 0x55d3852f0d60 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24fd827740 10 client.19881 fetch_fsmap learned FSMap version 51 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24fd827740 1 -- 172.21.15.18:0/2203898739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55d385219540 con 0x55d3852f0d60 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24d3fff640 1 -- 172.21.15.18:0/2203898739 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24e4019a50 con 0x55d3852f0d60 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24d3fff640 1 -- 172.21.15.18:0/2203898739 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f24ec035470 con 0x55d3852f0d60 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24fd827740 10 client.19881 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24fd827740 10 client.19881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24fd827740 20 client.19881 populate_metadata read hostname 'smithi018' 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24fd827740 1 --2- 172.21.15.18:0/2203898739 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d385361c60 0x55d385364080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24fd827740 4 client.19881 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.641+0000 7f24fd827740 1 -- 172.21.15.18:0/2203898739 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d384f95550 con 0x55d385361c60 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.642+0000 7f24ebfff640 1 --2- 172.21.15.18:0/2203898739 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d385361c60 0x55d385364080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.642+0000 7f24ebfff640 1 --2- 172.21.15.18:0/2203898739 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d385361c60 0x55d385364080 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16d2b876ed4e9076 server_cookie=dc0f36a7e56136c7 in_seq=0 out_seq=0 2024-02-17T13:01:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.642+0000 7f24d3fff640 10 client.19881 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.679+0000 7f24d3fff640 1 -- 172.21.15.18:0/2203898739 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d384f95550 con 0x55d385361c60 2024-02-17T13:01:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.679+0000 7f24d3fff640 10 client.19881 handle_command_reply: tid=0 2024-02-17T13:01:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.679+0000 7f24fd827740 1 client.19881 shutdown 2024-02-17T13:01:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.680+0000 7f24fd827740 1 -- 172.21.15.18:0/2203898739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 msgr2=0x55d385301270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.680+0000 7f24fd827740 1 --2- 172.21.15.18:0/2203898739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 0x55d385301270 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00e680 tx=0x7f24ec011c50 comp rx=0 tx=0).stop 2024-02-17T13:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.680+0000 7f24fd827740 1 -- 172.21.15.18:0/2203898739 shutdown_connections 2024-02-17T13:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.680+0000 7f24fd827740 1 --2- 172.21.15.18:0/2203898739 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d385361c60 0x55d385364080 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.680+0000 7f24fd827740 1 --2- 172.21.15.18:0/2203898739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3852fa2e0 0x55d3852ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.680+0000 7f24fd827740 1 --2- 172.21.15.18:0/2203898739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d3852f1ad0 0x55d3852ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.680+0000 7f24d3fff640 0 client.19881 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.680+0000 7f24fd827740 1 --2- 172.21.15.18:0/2203898739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3852f0d60 0x55d385301270 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.680+0000 7f24fd827740 1 -- 172.21.15.18:0/2203898739 >> 172.21.15.18:0/2203898739 conn(0x55d3852b8550 msgr2=0x55d3852f7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.680+0000 7f24fd827740 1 -- 172.21.15.18:0/2203898739 shutdown_connections 2024-02-17T13:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.680+0000 7f24fd827740 1 -- 172.21.15.18:0/2203898739 wait complete. 2024-02-17T13:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.680+0000 7f24fd827740 20 client.19881 trim_cache size 0 max 16384 2024-02-17T13:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:01:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:01:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:01:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.681+0000 7f24fac4e640 1 -- 172.21.15.18:0/3674909933 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f24dc0a22d0 msgr2=0x7f24dc0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.681+0000 7f24fac4e640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f24dc0a22d0 0x7f24dc0a4790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f24f00bd1b0 tx=0x7f24f008c650 comp rx=0 tx=0).stop 2024-02-17T13:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.681+0000 7f24fac4e640 1 -- 172.21.15.18:0/3674909933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f4073520 msgr2=0x7f24f419dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.681+0000 7f24fac4e640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f4073520 0x7f24f419dd20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f24f0006010 tx=0x7f24f00346f0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.681+0000 7f24fac4e640 1 -- 172.21.15.18:0/3674909933 shutdown_connections 2024-02-17T13:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.681+0000 7f24fac4e640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f24dc0a22d0 0x7f24dc0a4790 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.681+0000 7f24fac4e640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24f4112160 0x7f24f41a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.681+0000 7f24fac4e640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f4073e40 0x7f24f419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.681+0000 7f24fac4e640 1 --2- 172.21.15.18:0/3674909933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f4073520 0x7f24f419dd20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.681+0000 7f24fac4e640 1 -- 172.21.15.18:0/3674909933 >> 172.21.15.18:0/3674909933 conn(0x7f24f40fcc60 msgr2=0x7f24f40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.681+0000 7f24fac4e640 1 -- 172.21.15.18:0/3674909933 shutdown_connections 2024-02-17T13:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:01:55.681+0000 7f24fac4e640 1 -- 172.21.15.18:0/3674909933 wait complete. 2024-02-17T13:01:56.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 601947 4.88 MB/sec execute 551 sec latency 1438.874 ms 2024-02-17T13:01:56.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:56 smithi018 ceph-mon[32625]: pgmap v377: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 233 op/s 2024-02-17T13:01:56.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:56 smithi119 ceph-mon[36188]: pgmap v377: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 233 op/s 2024-02-17T13:01:56.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:56 smithi146 ceph-mon[36454]: pgmap v377: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 233 op/s 2024-02-17T13:01:57.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 601947 4.87 MB/sec execute 552 sec latency 2438.938 ms 2024-02-17T13:01:58.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 601947 4.87 MB/sec execute 553 sec latency 3438.980 ms 2024-02-17T13:01:58.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:01:58 smithi018 ceph-mon[32625]: pgmap v378: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-02-17T13:01:58.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:01:58 smithi119 ceph-mon[36188]: pgmap v378: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-02-17T13:01:58.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:01:58 smithi146 ceph-mon[36454]: pgmap v378: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-02-17T13:01:59.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 603893 4.88 MB/sec execute 554 sec latency 3622.186 ms 2024-02-17T13:02:00.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 605296 4.88 MB/sec execute 555 sec latency 15.123 ms 2024-02-17T13:02:00.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:00 smithi018 ceph-mon[32625]: pgmap v379: 97 pgs: 97 active+clean; 238 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-02-17T13:02:00.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:00 smithi119 ceph-mon[36188]: pgmap v379: 97 pgs: 97 active+clean; 238 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-02-17T13:02:00.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:00 smithi146 ceph-mon[36454]: pgmap v379: 97 pgs: 97 active+clean; 238 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-02-17T13:02:01.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 605495 4.87 MB/sec execute 556 sec latency 865.881 ms 2024-02-17T13:02:02.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 605495 4.87 MB/sec execute 557 sec latency 1865.944 ms 2024-02-17T13:02:02.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:02 smithi018 ceph-mon[32625]: pgmap v380: 97 pgs: 97 active+clean; 238 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-02-17T13:02:02.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:02 smithi119 ceph-mon[36188]: pgmap v380: 97 pgs: 97 active+clean; 238 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-02-17T13:02:02.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:02 smithi146 ceph-mon[36454]: pgmap v380: 97 pgs: 97 active+clean; 238 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-02-17T13:02:03.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 605495 4.86 MB/sec execute 558 sec latency 2866.075 ms 2024-02-17T13:02:04.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 606380 4.85 MB/sec execute 559 sec latency 3461.576 ms 2024-02-17T13:02:04.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:04 smithi018 ceph-mon[32625]: pgmap v381: 97 pgs: 97 active+clean; 238 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-02-17T13:02:04.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:04 smithi119 ceph-mon[36188]: pgmap v381: 97 pgs: 97 active+clean; 238 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-02-17T13:02:04.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:04 smithi146 ceph-mon[36454]: pgmap v381: 97 pgs: 97 active+clean; 238 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-02-17T13:02:05.301 INFO:tasks.workunit.client.0.smithi018.stdout: 1 607924 4.86 MB/sec execute 560 sec latency 13.630 ms 2024-02-17T13:02:06.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 609036 4.86 MB/sec execute 561 sec latency 305.711 ms 2024-02-17T13:02:06.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:06 smithi018 ceph-mon[32625]: pgmap v382: 97 pgs: 97 active+clean; 241 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-02-17T13:02:06.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:06 smithi018 ceph-mon[32625]: osdmap e103: 12 total, 12 up, 12 in 2024-02-17T13:02:06.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:06 smithi119 ceph-mon[36188]: pgmap v382: 97 pgs: 97 active+clean; 241 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-02-17T13:02:06.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:06 smithi119 ceph-mon[36188]: osdmap e103: 12 total, 12 up, 12 in 2024-02-17T13:02:06.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:06 smithi146 ceph-mon[36454]: pgmap v382: 97 pgs: 97 active+clean; 241 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-02-17T13:02:06.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:06 smithi146 ceph-mon[36454]: osdmap e103: 12 total, 12 up, 12 in 2024-02-17T13:02:07.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 609036 4.86 MB/sec execute 562 sec latency 1305.780 ms 2024-02-17T13:02:08.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 609036 4.85 MB/sec execute 563 sec latency 2305.915 ms 2024-02-17T13:02:08.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:08 smithi018 ceph-mon[32625]: pgmap v384: 97 pgs: 97 active+clean; 241 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-02-17T13:02:08.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:08 smithi119 ceph-mon[36188]: pgmap v384: 97 pgs: 97 active+clean; 241 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-02-17T13:02:08.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:08 smithi146 ceph-mon[36454]: pgmap v384: 97 pgs: 97 active+clean; 241 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-02-17T13:02:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.780+0000 7f014e5d7640 1 -- 172.21.15.18:0/2557510990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0148107270 msgr2=0x7f0148107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.780+0000 7f014e5d7640 1 --2- 172.21.15.18:0/2557510990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0148107270 0x7f0148107650 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f0138009900 tx=0x7f013802bcd0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.780+0000 7f014e5d7640 1 -- 172.21.15.18:0/2557510990 shutdown_connections 2024-02-17T13:02:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.780+0000 7f014e5d7640 1 --2- 172.21.15.18:0/2557510990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01480691a0 0x7f014810e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.780+0000 7f014e5d7640 1 --2- 172.21.15.18:0/2557510990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0148068800 0x7f0148068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.780+0000 7f014e5d7640 1 --2- 172.21.15.18:0/2557510990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0148107270 0x7f0148107650 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.780+0000 7f014e5d7640 1 -- 172.21.15.18:0/2557510990 >> 172.21.15.18:0/2557510990 conn(0x7f0148076ed0 msgr2=0x7f01480772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.780+0000 7f014e5d7640 1 -- 172.21.15.18:0/2557510990 shutdown_connections 2024-02-17T13:02:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.780+0000 7f014e5d7640 1 -- 172.21.15.18:0/2557510990 wait complete. 2024-02-17T13:02:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.780+0000 7f014e5d7640 1 Processor -- start 2024-02-17T13:02:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014e5d7640 1 -- start start 2024-02-17T13:02:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014e5d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0148068800 0x7f0148197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014e5d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01480691a0 0x7f0148197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014e5d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0148107270 0x7f014819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014e5d7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014819c550 con 0x7f0148107270 2024-02-17T13:02:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014e5d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014819c6c0 con 0x7f0148068800 2024-02-17T13:02:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014e5d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014819c880 con 0x7f01480691a0 2024-02-17T13:02:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014cb74640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0148107270 0x7f014819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014cb74640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0148107270 0x7f014819bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40008/0 (socket says 172.21.15.18:40008) 2024-02-17T13:02:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014cb74640 1 -- 172.21.15.18:0/81204727 learned_addr learned my addr 172.21.15.18:0/81204727 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f01477fe640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01480691a0 0x7f0148197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f0147fff640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0148068800 0x7f0148197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014cb74640 1 -- 172.21.15.18:0/81204727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01480691a0 msgr2=0x7f0148197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014cb74640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01480691a0 0x7f0148197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014cb74640 1 -- 172.21.15.18:0/81204727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0148068800 msgr2=0x7f0148197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014cb74640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0148068800 0x7f0148197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.781+0000 7f014cb74640 1 -- 172.21.15.18:0/81204727 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0138009560 con 0x7f0148107270 2024-02-17T13:02:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.782+0000 7f014cb74640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0148107270 0x7f014819bf50 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f014000e960 tx=0x7f014000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.782+0000 7f01457fa640 1 -- 172.21.15.18:0/81204727 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140004d20 con 0x7f0148107270 2024-02-17T13:02:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.782+0000 7f01457fa640 1 -- 172.21.15.18:0/81204727 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0140004e80 con 0x7f0148107270 2024-02-17T13:02:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.782+0000 7f014e5d7640 1 -- 172.21.15.18:0/81204727 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014819cb60 con 0x7f0148107270 2024-02-17T13:02:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.782+0000 7f01457fa640 1 -- 172.21.15.18:0/81204727 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01400116d0 con 0x7f0148107270 2024-02-17T13:02:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.782+0000 7f014e5d7640 1 -- 172.21.15.18:0/81204727 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01481a43e0 con 0x7f0148107270 2024-02-17T13:02:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.784+0000 7f01457fa640 1 -- 172.21.15.18:0/81204727 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0140005340 con 0x7f0148107270 2024-02-17T13:02:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.784+0000 7f01511aa740 1 Processor -- start 2024-02-17T13:02:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.784+0000 7f01511aa740 1 -- start start 2024-02-17T13:02:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f01511aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae510ad0 0x55c3ae50f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f01511aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 0x55c3ae5101e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f01511aa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae5192e0 0x55c3ae51b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f01511aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ae47e9f0 con 0x55c3ae50fd60 2024-02-17T13:02:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f01511aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ae47bf20 con 0x55c3ae510ad0 2024-02-17T13:02:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f01511aa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ae477700 con 0x55c3ae5192e0 2024-02-17T13:02:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f0147fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 0x55c3ae5101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f0147fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 0x55c3ae5101e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40022/0 (socket says 172.21.15.18:40022) 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f0147fff640 1 -- 172.21.15.18:0/3837649995 learned_addr learned my addr 172.21.15.18:0/3837649995 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f01477fe640 1 --2- 172.21.15.18:0/3837649995 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae5192e0 0x55c3ae51b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f01457fa640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f01300a22d0 0x7f01300a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f01477fe640 1 --2- 172.21.15.18:0/3837649995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae510ad0 0x55c3ae50f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f0147fff640 1 -- 172.21.15.18:0/3837649995 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae5192e0 msgr2=0x55c3ae51b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f0147fff640 1 --2- 172.21.15.18:0/3837649995 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae5192e0 0x55c3ae51b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f0147fff640 1 -- 172.21.15.18:0/3837649995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae510ad0 msgr2=0x55c3ae50f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f0147fff640 1 --2- 172.21.15.18:0/3837649995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae510ad0 0x55c3ae50f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f0147fff640 1 -- 172.21.15.18:0/3837649995 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3ae40d7b0 con 0x55c3ae50fd60 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f014cb74640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f01300a22d0 0x7f01300a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f01477fe640 1 --2- 172.21.15.18:0/3837649995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae510ad0 0x55c3ae50f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f01457fa640 1 -- 172.21.15.18:0/81204727 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f01400ce290 con 0x7f0148107270 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.785+0000 7f0147fff640 1 --2- 172.21.15.18:0/3837649995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 0x55c3ae5101e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f0138009900 tx=0x7f013802c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc81cf606aa8288a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f012e7fc640 1 -- 172.21.15.18:0/3837649995 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138004a40 con 0x55c3ae50fd60 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f012e7fc640 1 -- 172.21.15.18:0/3837649995 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0138004040 con 0x55c3ae50fd60 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f012e7fc640 1 -- 172.21.15.18:0/3837649995 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138005330 con 0x55c3ae50fd60 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f01511aa740 1 -- 172.21.15.18:0/3837649995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 msgr2=0x55c3ae5101e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f014cb74640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f01300a22d0 0x7f01300a4790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f014009a980 tx=0x7f0140096fa0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f01511aa740 1 --2- 172.21.15.18:0/3837649995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 0x55c3ae5101e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f0138009900 tx=0x7f013802c790 comp rx=0 tx=0).stop 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f01511aa740 1 -- 172.21.15.18:0/3837649995 shutdown_connections 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f01511aa740 1 --2- 172.21.15.18:0/3837649995 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae5192e0 0x55c3ae51b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f01511aa740 1 --2- 172.21.15.18:0/3837649995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 0x55c3ae5101e0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f01511aa740 1 --2- 172.21.15.18:0/3837649995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae510ad0 0x55c3ae50f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f01511aa740 1 -- 172.21.15.18:0/3837649995 >> 172.21.15.18:0/3837649995 conn(0x55c3ae4d7550 msgr2=0x55c3ae50aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f01511aa740 1 -- 172.21.15.18:0/3837649995 shutdown_connections 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.786+0000 7f01511aa740 1 -- 172.21.15.18:0/3837649995 wait complete. 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.787+0000 7f01511aa740 1 -- start start 2024-02-17T13:02:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.787+0000 7f01511aa740 1 Processor -- start 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.787+0000 7f01511aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 0x55c3ae516320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.787+0000 7f01511aa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae510ad0 0x55c3ae516860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.787+0000 7f01511aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae5192e0 0x55c3ae516da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.787+0000 7f01511aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ae41cbf0 con 0x55c3ae50fd60 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.787+0000 7f01511aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ae14b250 con 0x55c3ae5192e0 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.787+0000 7f01511aa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ae3fd200 con 0x55c3ae510ad0 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.787+0000 7f01511aa740 10 client.0 _ll_register_callbacks cb 0x55c3ae46bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.787+0000 7f01477fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 0x55c3ae516320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.787+0000 7f01477fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 0x55c3ae516320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40038/0 (socket says 172.21.15.18:40038) 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.787+0000 7f01477fe640 1 -- 172.21.15.18:0/4126006218 learned_addr learned my addr 172.21.15.18:0/4126006218 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f0147fff640 1 --2- 172.21.15.18:0/4126006218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae510ad0 0x55c3ae516860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01477fe640 1 -- 172.21.15.18:0/4126006218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae510ad0 msgr2=0x55c3ae516860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01477fe640 1 --2- 172.21.15.18:0/4126006218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae510ad0 0x55c3ae516860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01477fe640 1 -- 172.21.15.18:0/4126006218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae5192e0 msgr2=0x55c3ae516da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01477fe640 1 --2- 172.21.15.18:0/4126006218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae5192e0 0x55c3ae516da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01477fe640 1 -- 172.21.15.18:0/4126006218 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014810e850 con 0x55c3ae50fd60 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01477fe640 1 --2- 172.21.15.18:0/4126006218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 0x55c3ae516320 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f013c004330 tx=0x7f013c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11650ad13bbea88e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01277fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01277fe640 1 -- 172.21.15.18:0/4126006218 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c005220 con 0x55c3ae50fd60 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01277fe640 1 -- 172.21.15.18:0/4126006218 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f013c005380 con 0x55c3ae50fd60 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01511aa740 1 -- 172.21.15.18:0/4126006218 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3ae177600 con 0x55c3ae50fd60 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01277fe640 1 -- 172.21.15.18:0/4126006218 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f013c005540 con 0x55c3ae50fd60 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01511aa740 10 client.19893 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01511aa740 1 -- 172.21.15.18:0/4126006218 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55c3ae475d80 con 0x55c3ae50fd60 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.788+0000 7f01277fe640 1 -- 172.21.15.18:0/4126006218 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c3ae177600 con 0x55c3ae50fd60 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.789+0000 7f01277fe640 1 -- 172.21.15.18:0/4126006218 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f013c01c070 con 0x55c3ae50fd60 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.789+0000 7f01511aa740 10 client.19893 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.789+0000 7f01511aa740 10 client.19893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.789+0000 7f01511aa740 20 client.19893 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.789+0000 7f01511aa740 1 --2- 172.21.15.18:0/4126006218 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c3ae529580 0x55c3ae52b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.789+0000 7f01511aa740 4 client.19893 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.789+0000 7f01511aa740 1 -- 172.21.15.18:0/4126006218 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3ae1c81b0 con 0x55c3ae529580 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.789+0000 7f0147fff640 1 --2- 172.21.15.18:0/4126006218 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c3ae529580 0x55c3ae52b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01277fe640 10 client.19893 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f0147fff640 1 --2- 172.21.15.18:0/4126006218 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c3ae529580 0x55c3ae52b9a0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c827cd823c101c server_cookie=4029fd3488b65717 in_seq=0 out_seq=0 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01277fe640 1 -- 172.21.15.18:0/4126006218 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3ae1c81b0 con 0x55c3ae529580 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01277fe640 10 client.19893 handle_command_reply: tid=0 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01511aa740 1 client.19893 shutdown 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01511aa740 1 -- 172.21.15.18:0/4126006218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 msgr2=0x55c3ae516320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01511aa740 1 --2- 172.21.15.18:0/4126006218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 0x55c3ae516320 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f013c004330 tx=0x7f013c0039c0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01511aa740 1 -- 172.21.15.18:0/4126006218 shutdown_connections 2024-02-17T13:02:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01511aa740 1 --2- 172.21.15.18:0/4126006218 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c3ae529580 0x55c3ae52b9a0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01511aa740 1 --2- 172.21.15.18:0/4126006218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae5192e0 0x55c3ae516da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01511aa740 1 --2- 172.21.15.18:0/4126006218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae510ad0 0x55c3ae516860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01277fe640 0 client.19893 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01511aa740 1 --2- 172.21.15.18:0/4126006218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae50fd60 0x55c3ae516320 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01511aa740 1 -- 172.21.15.18:0/4126006218 >> 172.21.15.18:0/4126006218 conn(0x55c3ae4d7550 msgr2=0x55c3ae50c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.790+0000 7f01511aa740 1 -- 172.21.15.18:0/4126006218 shutdown_connections 2024-02-17T13:02:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.791+0000 7f01511aa740 1 -- 172.21.15.18:0/4126006218 wait complete. 2024-02-17T13:02:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.791+0000 7f01511aa740 20 client.19893 trim_cache size 0 max 16384 2024-02-17T13:02:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01511aa740 1 Processor -- start 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01511aa740 1 -- start start 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01511aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae50fd60 0x55c3ae518a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01511aa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae510ad0 0x55c3ae51ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01511aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae5192e0 0x55c3ae5204c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01511aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ae50ba50 con 0x55c3ae5192e0 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01511aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ae530b70 con 0x55c3ae50fd60 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01511aa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ae50f030 con 0x55c3ae510ad0 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01477fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae5192e0 0x55c3ae5204c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01477fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae5192e0 0x55c3ae5204c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40052/0 (socket says 172.21.15.18:40052) 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01477fe640 1 -- 172.21.15.18:0/947820369 learned_addr learned my addr 172.21.15.18:0/947820369 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f0147fff640 1 --2- 172.21.15.18:0/947820369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae510ad0 0x55c3ae51ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01477fe640 1 -- 172.21.15.18:0/947820369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae510ad0 msgr2=0x55c3ae51ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01477fe640 1 --2- 172.21.15.18:0/947820369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae510ad0 0x55c3ae51ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01477fe640 1 -- 172.21.15.18:0/947820369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae50fd60 msgr2=0x55c3ae518a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01477fe640 1 --2- 172.21.15.18:0/947820369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae50fd60 0x55c3ae518a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f01477fe640 1 -- 172.21.15.18:0/947820369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3ae4127b0 con 0x55c3ae5192e0 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.804+0000 7f0147fff640 1 --2- 172.21.15.18:0/947820369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae510ad0 0x55c3ae51ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f01477fe640 1 --2- 172.21.15.18:0/947820369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae5192e0 0x55c3ae5204c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f013c00b5b0 tx=0x7f013c00b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954b21bd99fe1df5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f012dffb640 1 -- 172.21.15.18:0/947820369 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00bc80 con 0x55c3ae5192e0 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f012dffb640 1 -- 172.21.15.18:0/947820369 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f013c004c10 con 0x55c3ae5192e0 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f012dffb640 1 -- 172.21.15.18:0/947820369 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c005220 con 0x55c3ae5192e0 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f01511aa740 1 -- 172.21.15.18:0/947820369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae5192e0 msgr2=0x55c3ae5204c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f01511aa740 1 --2- 172.21.15.18:0/947820369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae5192e0 0x55c3ae5204c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f013c00b5b0 tx=0x7f013c00b5e0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f01511aa740 1 -- 172.21.15.18:0/947820369 shutdown_connections 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f01511aa740 1 --2- 172.21.15.18:0/947820369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae5192e0 0x55c3ae5204c0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f01511aa740 1 --2- 172.21.15.18:0/947820369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae510ad0 0x55c3ae51ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f01511aa740 1 --2- 172.21.15.18:0/947820369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae50fd60 0x55c3ae518a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f01511aa740 1 -- 172.21.15.18:0/947820369 >> 172.21.15.18:0/947820369 conn(0x55c3ae4d7550 msgr2=0x55c3ae511640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f01511aa740 1 -- 172.21.15.18:0/947820369 shutdown_connections 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.805+0000 7f01511aa740 1 -- 172.21.15.18:0/947820369 wait complete. 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f01511aa740 1 -- start start 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f01511aa740 1 Processor -- start 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f01511aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae50fd60 0x55c3ae513260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f01511aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae510ad0 0x55c3ae5137a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f01511aa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae5192e0 0x55c3ae513ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f01511aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ae57fbb0 con 0x55c3ae510ad0 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f01511aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ae57fd20 con 0x55c3ae50fd60 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f01511aa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ae57fee0 con 0x55c3ae5192e0 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f01511aa740 10 client.0 _ll_register_callbacks cb 0x7f0138009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f0147fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae510ad0 0x55c3ae5137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f0147fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae510ad0 0x55c3ae5137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40066/0 (socket says 172.21.15.18:40066) 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f0147fff640 1 -- 172.21.15.18:0/488424677 learned_addr learned my addr 172.21.15.18:0/488424677 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f01477fe640 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae50fd60 0x55c3ae513260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f0147fff640 1 -- 172.21.15.18:0/488424677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae5192e0 msgr2=0x55c3ae513ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f0147fff640 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae5192e0 0x55c3ae513ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f0147fff640 1 -- 172.21.15.18:0/488424677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae50fd60 msgr2=0x55c3ae513260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f0147fff640 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae50fd60 0x55c3ae513260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.806+0000 7f0147fff640 1 -- 172.21.15.18:0/488424677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3ae475d80 con 0x55c3ae510ad0 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01477fe640 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae50fd60 0x55c3ae513260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f0147fff640 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae510ad0 0x55c3ae5137a0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f0138004f00 tx=0x7f0138004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4f22f4c1c6ee58 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01277fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01277fe640 1 -- 172.21.15.18:0/488424677 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01380049c0 con 0x55c3ae510ad0 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01277fe640 1 -- 172.21.15.18:0/488424677 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f013802c880 con 0x55c3ae510ad0 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01511aa740 1 -- 172.21.15.18:0/488424677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3ae516150 con 0x55c3ae510ad0 2024-02-17T13:02:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01277fe640 1 -- 172.21.15.18:0/488424677 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f013802ca40 con 0x55c3ae510ad0 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01511aa740 10 client.19899 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01511aa740 1 -- 172.21.15.18:0/488424677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55c3ae438540 con 0x55c3ae510ad0 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01277fe640 1 -- 172.21.15.18:0/488424677 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f013805d080 con 0x55c3ae510ad0 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01277fe640 1 -- 172.21.15.18:0/488424677 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f013803c040 con 0x55c3ae510ad0 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01511aa740 10 client.19899 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01511aa740 10 client.19899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01511aa740 20 client.19899 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01511aa740 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c3ae581310 0x55c3ae583730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01511aa740 4 client.19899 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.807+0000 7f01511aa740 1 -- 172.21.15.18:0/488424677 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f0138004040 con 0x55c3ae581310 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.808+0000 7f01477fe640 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c3ae581310 0x55c3ae583730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.808+0000 7f01477fe640 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c3ae581310 0x55c3ae583730 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f922dce75b22231b server_cookie=3c2951c788cb7b2d in_seq=0 out_seq=0 2024-02-17T13:02:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.808+0000 7f01277fe640 10 client.19899 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01277fe640 1 -- 172.21.15.18:0/488424677 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f0138004040 con 0x55c3ae581310 2024-02-17T13:02:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01277fe640 10 client.19899 handle_command_reply: tid=0 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01511aa740 1 client.19899 shutdown 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01511aa740 1 -- 172.21.15.18:0/488424677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae510ad0 msgr2=0x55c3ae5137a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01511aa740 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae510ad0 0x55c3ae5137a0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f0138004f00 tx=0x7f0138004e90 comp rx=0 tx=0).stop 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01511aa740 1 -- 172.21.15.18:0/488424677 shutdown_connections 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01511aa740 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c3ae581310 0x55c3ae583730 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01511aa740 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c3ae5192e0 0x55c3ae513ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01511aa740 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c3ae510ad0 0x55c3ae5137a0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01277fe640 0 client.19899 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01511aa740 1 --2- 172.21.15.18:0/488424677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c3ae50fd60 0x55c3ae513260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01511aa740 1 -- 172.21.15.18:0/488424677 >> 172.21.15.18:0/488424677 conn(0x55c3ae4d7550 msgr2=0x55c3ae514bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01511aa740 1 -- 172.21.15.18:0/488424677 shutdown_connections 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01511aa740 1 -- 172.21.15.18:0/488424677 wait complete. 2024-02-17T13:02:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.850+0000 7f01511aa740 20 client.19899 trim_cache size 0 max 16384 2024-02-17T13:02:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:02:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:02:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:02:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:02:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.851+0000 7f014e5d7640 1 -- 172.21.15.18:0/81204727 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f01300a22d0 msgr2=0x7f01300a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.851+0000 7f014e5d7640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f01300a22d0 0x7f01300a4790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f014009a980 tx=0x7f0140096fa0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.851+0000 7f014e5d7640 1 -- 172.21.15.18:0/81204727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0148107270 msgr2=0x7f014819bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.851+0000 7f014e5d7640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0148107270 0x7f014819bf50 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f014000e960 tx=0x7f014000ee20 comp rx=0 tx=0).stop 2024-02-17T13:02:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.851+0000 7f014e5d7640 1 -- 172.21.15.18:0/81204727 shutdown_connections 2024-02-17T13:02:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.851+0000 7f014e5d7640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f01300a22d0 0x7f01300a4790 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.851+0000 7f014e5d7640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0148107270 0x7f014819bf50 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.851+0000 7f014e5d7640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01480691a0 0x7f0148197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.851+0000 7f014e5d7640 1 --2- 172.21.15.18:0/81204727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0148068800 0x7f0148197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.851+0000 7f014e5d7640 1 -- 172.21.15.18:0/81204727 >> 172.21.15.18:0/81204727 conn(0x7f0148076ed0 msgr2=0x7f01480fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.851+0000 7f014e5d7640 1 -- 172.21.15.18:0/81204727 shutdown_connections 2024-02-17T13:02:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:08.851+0000 7f014e5d7640 1 -- 172.21.15.18:0/81204727 wait complete. 2024-02-17T13:02:09.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 609946 4.84 MB/sec execute 564 sec latency 2902.435 ms 2024-02-17T13:02:10.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 611514 4.85 MB/sec execute 565 sec latency 11.992 ms 2024-02-17T13:02:10.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:10 smithi018 ceph-mon[32625]: pgmap v385: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-02-17T13:02:10.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:10 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:02:10.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:10 smithi119 ceph-mon[36188]: pgmap v385: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-02-17T13:02:10.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:10 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:02:10.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:10 smithi146 ceph-mon[36454]: pgmap v385: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-02-17T13:02:10.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:10 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:02:11.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 612586 4.86 MB/sec execute 566 sec latency 323.024 ms 2024-02-17T13:02:11.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:11 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:11.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:11 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:11.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:11 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:11.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:11 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:11.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:11 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:02:11.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:11 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:02:11.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:11 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:12.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:11 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:12.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:11 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:12.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:11 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:12.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:11 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:12.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:11 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:02:12.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:11 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:02:12.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:11 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:12.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:11 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:12.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:11 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:12.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:11 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:12.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:11 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:12.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:11 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:02:12.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:11 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:02:12.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:11 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:02:12.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 612586 4.85 MB/sec execute 567 sec latency 1323.074 ms 2024-02-17T13:02:13.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:12 smithi018 ceph-mon[32625]: pgmap v386: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-02-17T13:02:13.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:12 smithi119 ceph-mon[36188]: pgmap v386: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-02-17T13:02:13.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:12 smithi146 ceph-mon[36454]: pgmap v386: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-02-17T13:02:13.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 613500 4.84 MB/sec execute 568 sec latency 1877.497 ms 2024-02-17T13:02:13.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:02:13.918 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:02:14.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:13 smithi018 ceph-mon[32625]: pgmap v387: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-02-17T13:02:14.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.040+0000 7ff993c9d640 1 -- 172.21.15.18:0/3840681680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff98c102cc0 msgr2=0x7ff98c10b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.040+0000 7ff993c9d640 1 --2- 172.21.15.18:0/3840681680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff98c102cc0 0x7ff98c10b1a0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009900 tx=0x7ff97c02bcd0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.040+0000 7ff993c9d640 1 -- 172.21.15.18:0/3840681680 shutdown_connections 2024-02-17T13:02:14.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.040+0000 7ff993c9d640 1 --2- 172.21.15.18:0/3840681680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff98c10b6e0 0x7ff98c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.040+0000 7ff993c9d640 1 --2- 172.21.15.18:0/3840681680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff98c102cc0 0x7ff98c10b1a0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.040+0000 7ff993c9d640 1 --2- 172.21.15.18:0/3840681680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c1023a0 0x7ff98c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.040+0000 7ff993c9d640 1 -- 172.21.15.18:0/3840681680 >> 172.21.15.18:0/3840681680 conn(0x7ff98c077230 msgr2=0x7ff98c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.040+0000 7ff993c9d640 1 -- 172.21.15.18:0/3840681680 shutdown_connections 2024-02-17T13:02:14.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.040+0000 7ff993c9d640 1 -- 172.21.15.18:0/3840681680 wait complete. 2024-02-17T13:02:14.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.040+0000 7ff993c9d640 1 Processor -- start 2024-02-17T13:02:14.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.040+0000 7ff993c9d640 1 -- start start 2024-02-17T13:02:14.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff993c9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff98c1023a0 0x7ff98c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff993c9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c102cc0 0x7ff98c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff993c9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff98c10b6e0 0x7ff98c197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff993c9d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c197c50 con 0x7ff98c10b6e0 2024-02-17T13:02:14.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff993c9d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c197dc0 con 0x7ff98c1023a0 2024-02-17T13:02:14.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff993c9d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c197f80 con 0x7ff98c102cc0 2024-02-17T13:02:14.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff99223a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff98c10b6e0 0x7ff98c197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff991a39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff98c1023a0 0x7ff98c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff991a39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff98c1023a0 0x7ff98c19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:58724/0 (socket says 172.21.15.18:58724) 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff991a39640 1 -- 172.21.15.18:0/3718001942 learned_addr learned my addr 172.21.15.18:0/3718001942 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff99223a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff98c10b6e0 0x7ff98c197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39314/0 (socket says 172.21.15.18:39314) 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff991238640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c102cc0 0x7ff98c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff99223a640 1 -- 172.21.15.18:0/3718001942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c102cc0 msgr2=0x7ff98c19d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff99223a640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c102cc0 0x7ff98c19d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff99223a640 1 -- 172.21.15.18:0/3718001942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff98c1023a0 msgr2=0x7ff98c19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff99223a640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff98c1023a0 0x7ff98c19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff99223a640 1 -- 172.21.15.18:0/3718001942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c009560 con 0x7ff98c10b6e0 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff991a39640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff98c1023a0 0x7ff98c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff99223a640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff98c10b6e0 0x7ff98c197600 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7ff984009fd0 tx=0x7ff98400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff982ffd640 1 -- 172.21.15.18:0/3718001942 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984015070 con 0x7ff98c10b6e0 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff993c9d640 1 -- 172.21.15.18:0/3718001942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98c198260 con 0x7ff98c10b6e0 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff982ffd640 1 -- 172.21.15.18:0/3718001942 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff984019450 con 0x7ff98c10b6e0 2024-02-17T13:02:14.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.041+0000 7ff982ffd640 1 -- 172.21.15.18:0/3718001942 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9840052c0 con 0x7ff98c10b6e0 2024-02-17T13:02:14.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.042+0000 7ff993c9d640 1 -- 172.21.15.18:0/3718001942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98c100100 con 0x7ff98c10b6e0 2024-02-17T13:02:14.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.043+0000 7ff982ffd640 1 -- 172.21.15.18:0/3718001942 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff9840195c0 con 0x7ff98c10b6e0 2024-02-17T13:02:14.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.044+0000 7ff996874740 1 Processor -- start 2024-02-17T13:02:14.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.044+0000 7ff996874740 1 -- start start 2024-02-17T13:02:14.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.044+0000 7ff996874740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37dd2b0 0x5632c37dd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.044+0000 7ff996874740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37e6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.044+0000 7ff996874740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37e67b0 0x5632c37e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.044+0000 7ff996874740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632c374c280 con 0x5632c37e67b0 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.044+0000 7ff996874740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632c37497b0 con 0x5632c37ddbd0 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.044+0000 7ff996874740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632c3744f90 con 0x5632c37dd2b0 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.044+0000 7ff982ffd640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff9780a2230 0x7ff9780a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff991a39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff991a39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37e6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:58730/0 (socket says 172.21.15.18:58730) 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff991238640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37e67b0 0x5632c37e8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff991a39640 1 -- 172.21.15.18:0/3584960999 learned_addr learned my addr 172.21.15.18:0/3584960999 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff99223a640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff9780a2230 0x7ff9780a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff991238640 1 --2- 172.21.15.18:0/3584960999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37dd2b0 0x5632c37dd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff982ffd640 1 -- 172.21.15.18:0/3718001942 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff9840d1ee0 con 0x7ff98c10b6e0 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff991a39640 1 -- 172.21.15.18:0/3584960999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37dd2b0 msgr2=0x5632c37dd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff991a39640 1 --2- 172.21.15.18:0/3584960999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37dd2b0 0x5632c37dd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff991a39640 1 -- 172.21.15.18:0/3584960999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37e67b0 msgr2=0x5632c37e8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff991a39640 1 --2- 172.21.15.18:0/3584960999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37e67b0 0x5632c37e8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff991a39640 1 -- 172.21.15.18:0/3584960999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632c3703570 con 0x5632c37ddbd0 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff991a39640 1 --2- 172.21.15.18:0/3584960999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37e6270 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff9880105c0 tx=0x7ff988010a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=221e2fe34f3ba6e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff99223a640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff9780a2230 0x7ff9780a46f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff9840cf700 tx=0x7ff984070960 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff977fff640 1 -- 172.21.15.18:0/3584960999 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98800e7d0 con 0x5632c37ddbd0 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff977fff640 1 -- 172.21.15.18:0/3584960999 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff988015070 con 0x5632c37ddbd0 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.045+0000 7ff977fff640 1 -- 172.21.15.18:0/3584960999 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988011d10 con 0x5632c37ddbd0 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.046+0000 7ff996874740 1 -- 172.21.15.18:0/3584960999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 msgr2=0x5632c37e6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.046+0000 7ff996874740 1 --2- 172.21.15.18:0/3584960999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37e6270 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff9880105c0 tx=0x7ff988010a80 comp rx=0 tx=0).stop 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.046+0000 7ff996874740 1 -- 172.21.15.18:0/3584960999 shutdown_connections 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.046+0000 7ff996874740 1 --2- 172.21.15.18:0/3584960999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37e67b0 0x5632c37e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.046+0000 7ff996874740 1 --2- 172.21.15.18:0/3584960999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37e6270 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.046+0000 7ff996874740 1 --2- 172.21.15.18:0/3584960999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37dd2b0 0x5632c37dd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.046+0000 7ff996874740 1 -- 172.21.15.18:0/3584960999 >> 172.21.15.18:0/3584960999 conn(0x5632c37d7990 msgr2=0x5632c37d7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.047+0000 7ff996874740 1 -- 172.21.15.18:0/3584960999 shutdown_connections 2024-02-17T13:02:14.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.047+0000 7ff996874740 1 -- 172.21.15.18:0/3584960999 wait complete. 2024-02-17T13:02:14.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.047+0000 7ff996874740 1 -- start start 2024-02-17T13:02:14.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.047+0000 7ff996874740 1 Processor -- start 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.047+0000 7ff996874740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 0x5632c37e6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.047+0000 7ff996874740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37e34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.047+0000 7ff996874740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37e67b0 0x5632c37e3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.047+0000 7ff996874740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632c3418430 con 0x5632c37dd2b0 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.047+0000 7ff996874740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632c33d4e00 con 0x5632c37ddbd0 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.047+0000 7ff996874740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632c36c9fd0 con 0x5632c37e67b0 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff996874740 10 client.0 _ll_register_callbacks cb 0x5632c371ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff991238640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 0x5632c37e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff991238640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 0x5632c37e6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39326/0 (socket says 172.21.15.18:39326) 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff991a39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37e34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff991238640 1 -- 172.21.15.18:0/895040984 learned_addr learned my addr 172.21.15.18:0/895040984 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff991238640 1 --2- 172.21.15.18:0/895040984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37e67b0 0x5632c37e3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff991238640 1 -- 172.21.15.18:0/895040984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37e67b0 msgr2=0x5632c37e3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff991238640 1 --2- 172.21.15.18:0/895040984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37e67b0 0x5632c37e3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff991238640 1 -- 172.21.15.18:0/895040984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 msgr2=0x5632c37e34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff991238640 1 --2- 172.21.15.18:0/895040984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37e34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff991238640 1 -- 172.21.15.18:0/895040984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98800be20 con 0x5632c37dd2b0 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff991238640 1 --2- 172.21.15.18:0/895040984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 0x5632c37e6130 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7ff97c02c210 tx=0x7ff97c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6a91cdd56df0b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff9757fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff9757fa640 1 -- 172.21.15.18:0/895040984 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c004350 con 0x5632c37dd2b0 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff9757fa640 1 -- 172.21.15.18:0/895040984 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff97c0044b0 con 0x5632c37dd2b0 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff996874740 1 -- 172.21.15.18:0/895040984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632c34447e0 con 0x5632c37dd2b0 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff9757fa640 1 -- 172.21.15.18:0/895040984 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff97c004df0 con 0x5632c37dd2b0 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.048+0000 7ff996874740 10 client.19911 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.049+0000 7ff996874740 1 -- 172.21.15.18:0/895040984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5632c3743610 con 0x5632c37dd2b0 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.049+0000 7ff9757fa640 1 -- 172.21.15.18:0/895040984 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5632c34447e0 con 0x5632c37dd2b0 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.049+0000 7ff9757fa640 1 -- 172.21.15.18:0/895040984 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff97c003b20 con 0x5632c37dd2b0 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.049+0000 7ff996874740 10 client.19911 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:14.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.049+0000 7ff996874740 10 client.19911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.049+0000 7ff996874740 20 client.19911 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.049+0000 7ff996874740 1 --2- 172.21.15.18:0/895040984 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5632c37f68f0 0x5632c37f8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.049+0000 7ff996874740 4 client.19911 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.049+0000 7ff996874740 1 -- 172.21.15.18:0/895040984 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632c3481720 con 0x5632c37f68f0 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.049+0000 7ff991a39640 1 --2- 172.21.15.18:0/895040984 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5632c37f68f0 0x5632c37f8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff9757fa640 10 client.19911 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff991a39640 1 --2- 172.21.15.18:0/895040984 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5632c37f68f0 0x5632c37f8ce0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fbfdd0d7b958946 server_cookie=d5c1da94fce6d9d2 in_seq=0 out_seq=0 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff9757fa640 1 -- 172.21.15.18:0/895040984 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632c3481720 con 0x5632c37f68f0 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff9757fa640 10 client.19911 handle_command_reply: tid=0 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff996874740 1 client.19911 shutdown 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff996874740 1 -- 172.21.15.18:0/895040984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 msgr2=0x5632c37e6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff996874740 1 --2- 172.21.15.18:0/895040984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 0x5632c37e6130 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7ff97c02c210 tx=0x7ff97c02bcd0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff996874740 1 -- 172.21.15.18:0/895040984 shutdown_connections 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff996874740 1 --2- 172.21.15.18:0/895040984 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5632c37f68f0 0x5632c37f8ce0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff996874740 1 --2- 172.21.15.18:0/895040984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37e67b0 0x5632c37e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff9757fa640 0 client.19911 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:14.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff996874740 1 --2- 172.21.15.18:0/895040984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37e34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff996874740 1 --2- 172.21.15.18:0/895040984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 0x5632c37e6130 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff996874740 1 -- 172.21.15.18:0/895040984 >> 172.21.15.18:0/895040984 conn(0x5632c37d7990 msgr2=0x5632c37d9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff996874740 1 -- 172.21.15.18:0/895040984 shutdown_connections 2024-02-17T13:02:14.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.050+0000 7ff996874740 1 -- 172.21.15.18:0/895040984 wait complete. 2024-02-17T13:02:14.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.051+0000 7ff996874740 20 client.19911 trim_cache size 0 max 16384 2024-02-17T13:02:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.066+0000 7ff996874740 1 Processor -- start 2024-02-17T13:02:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.066+0000 7ff996874740 1 -- start start 2024-02-17T13:02:14.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff996874740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 0x5632c37e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff996874740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff996874740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37e67b0 0x5632c37e5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff996874740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632c37de320 con 0x5632c37dd2b0 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff996874740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632c37d32b0 con 0x5632c37ddbd0 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff996874740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632c37d89e0 con 0x5632c37e67b0 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff991238640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 0x5632c37e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff991a39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff991238640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 0x5632c37e8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39338/0 (socket says 172.21.15.18:39338) 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff991238640 1 -- 172.21.15.18:0/1756744369 learned_addr learned my addr 172.21.15.18:0/1756744369 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff991238640 1 --2- 172.21.15.18:0/1756744369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37e67b0 0x5632c37e5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff991a39640 1 -- 172.21.15.18:0/1756744369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37e67b0 msgr2=0x5632c37e5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff991a39640 1 --2- 172.21.15.18:0/1756744369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37e67b0 0x5632c37e5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff991a39640 1 -- 172.21.15.18:0/1756744369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 msgr2=0x5632c37e8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff991a39640 1 --2- 172.21.15.18:0/1756744369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 0x5632c37e8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff991a39640 1 -- 172.21.15.18:0/1756744369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632c36dfac0 con 0x5632c37ddbd0 2024-02-17T13:02:14.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff991238640 1 --2- 172.21.15.18:0/1756744369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 0x5632c37e8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:14.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.067+0000 7ff991a39640 1 --2- 172.21.15.18:0/1756744369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37d7d70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff988006010 tx=0x7ff98800b4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6fc48d3fe3558c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.068+0000 7ff9777fe640 1 -- 172.21.15.18:0/1756744369 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988011560 con 0x5632c37ddbd0 2024-02-17T13:02:14.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.068+0000 7ff9777fe640 1 -- 172.21.15.18:0/1756744369 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff988005520 con 0x5632c37ddbd0 2024-02-17T13:02:14.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.068+0000 7ff9777fe640 1 -- 172.21.15.18:0/1756744369 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880294b0 con 0x5632c37ddbd0 2024-02-17T13:02:14.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.068+0000 7ff996874740 1 -- 172.21.15.18:0/1756744369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 msgr2=0x5632c37d7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.068+0000 7ff996874740 1 --2- 172.21.15.18:0/1756744369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37d7d70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff988006010 tx=0x7ff98800b4a0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 -- 172.21.15.18:0/1756744369 shutdown_connections 2024-02-17T13:02:14.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 --2- 172.21.15.18:0/1756744369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37e67b0 0x5632c37e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 --2- 172.21.15.18:0/1756744369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37ddbd0 0x5632c37d7d70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 --2- 172.21.15.18:0/1756744369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37dd2b0 0x5632c37e8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 -- 172.21.15.18:0/1756744369 >> 172.21.15.18:0/1756744369 conn(0x5632c37d7990 msgr2=0x5632c37e2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 -- 172.21.15.18:0/1756744369 shutdown_connections 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 -- 172.21.15.18:0/1756744369 wait complete. 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 -- start start 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 Processor -- start 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37dd2b0 0x5632c37ea3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37ddbd0 0x5632c37ea8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37e67b0 0x5632c384cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632c37eb050 con 0x5632c37ddbd0 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632c37eb1c0 con 0x5632c37e67b0 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632c384d300 con 0x5632c37dd2b0 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff996874740 10 client.0 _ll_register_callbacks cb 0x7ff98800bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff991a39640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37ddbd0 0x5632c37ea8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff991a39640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37ddbd0 0x5632c37ea8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39354/0 (socket says 172.21.15.18:39354) 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.069+0000 7ff991a39640 1 -- 172.21.15.18:0/3256893625 learned_addr learned my addr 172.21.15.18:0/3256893625 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff991238640 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37dd2b0 0x5632c37ea3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff991238640 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37e67b0 0x5632c384cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff991a39640 1 -- 172.21.15.18:0/3256893625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37dd2b0 msgr2=0x5632c37ea3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff991a39640 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37dd2b0 0x5632c37ea3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff991a39640 1 -- 172.21.15.18:0/3256893625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37e67b0 msgr2=0x5632c384cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff991a39640 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37e67b0 0x5632c384cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff991a39640 1 -- 172.21.15.18:0/3256893625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632c3743610 con 0x5632c37ddbd0 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff991a39640 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37ddbd0 0x5632c37ea8f0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7ff98802b040 tx=0x7ff9880056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901ae0dde8d8f988 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff9757fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff9757fa640 1 -- 172.21.15.18:0/3256893625 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880119e0 con 0x5632c37ddbd0 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff9757fa640 1 -- 172.21.15.18:0/3256893625 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff988029250 con 0x5632c37ddbd0 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff996874740 1 -- 172.21.15.18:0/3256893625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff988015070 con 0x5632c37ddbd0 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff9757fa640 1 -- 172.21.15.18:0/3256893625 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff988029410 con 0x5632c37ddbd0 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff996874740 10 client.19917 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff996874740 1 -- 172.21.15.18:0/3256893625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7ff98801e040 con 0x5632c37ddbd0 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.070+0000 7ff9757fa640 1 -- 172.21.15.18:0/3256893625 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff988015070 con 0x5632c37ddbd0 2024-02-17T13:02:14.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff9757fa640 1 -- 172.21.15.18:0/3256893625 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff98800ec00 con 0x5632c37ddbd0 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff996874740 10 client.19917 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff996874740 10 client.19917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff996874740 20 client.19917 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff996874740 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5632c384e7b0 0x5632c3850bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff996874740 4 client.19917 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff996874740 1 -- 172.21.15.18:0/3256893625 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5632c34d0890 con 0x5632c384e7b0 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff991238640 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5632c384e7b0 0x5632c3850bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff9757fa640 10 client.19917 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff991238640 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5632c384e7b0 0x5632c3850bd0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=366c415da0be78a6 server_cookie=95cbe447a2c2f85d in_seq=0 out_seq=0 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff9757fa640 1 -- 172.21.15.18:0/3256893625 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5632c34d0890 con 0x5632c384e7b0 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff9757fa640 10 client.19917 handle_command_reply: tid=0 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.071+0000 7ff996874740 1 client.19917 shutdown 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff996874740 1 -- 172.21.15.18:0/3256893625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37ddbd0 msgr2=0x5632c37ea8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff996874740 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37ddbd0 0x5632c37ea8f0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7ff98802b040 tx=0x7ff9880056b0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff996874740 1 -- 172.21.15.18:0/3256893625 shutdown_connections 2024-02-17T13:02:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff996874740 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5632c384e7b0 0x5632c3850bd0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff996874740 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632c37e67b0 0x5632c384cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff996874740 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5632c37ddbd0 0x5632c37ea8f0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff996874740 1 --2- 172.21.15.18:0/3256893625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632c37dd2b0 0x5632c37ea3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff9757fa640 0 client.19917 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff996874740 1 -- 172.21.15.18:0/3256893625 >> 172.21.15.18:0/3256893625 conn(0x5632c37d7990 msgr2=0x5632c37d7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff996874740 1 -- 172.21.15.18:0/3256893625 shutdown_connections 2024-02-17T13:02:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff996874740 1 -- 172.21.15.18:0/3256893625 wait complete. 2024-02-17T13:02:14.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff996874740 20 client.19917 trim_cache size 0 max 16384 2024-02-17T13:02:14.067 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:02:14.067 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T13:02:14.067 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T13:02:14.068 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:02:14.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff993c9d640 1 -- 172.21.15.18:0/3718001942 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff9780a2230 msgr2=0x7ff9780a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff993c9d640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff9780a2230 0x7ff9780a46f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff9840cf700 tx=0x7ff984070960 comp rx=0 tx=0).stop 2024-02-17T13:02:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff993c9d640 1 -- 172.21.15.18:0/3718001942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff98c10b6e0 msgr2=0x7ff98c197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff993c9d640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff98c10b6e0 0x7ff98c197600 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7ff984009fd0 tx=0x7ff98400b690 comp rx=0 tx=0).stop 2024-02-17T13:02:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff993c9d640 1 -- 172.21.15.18:0/3718001942 shutdown_connections 2024-02-17T13:02:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff993c9d640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ff9780a2230 0x7ff9780a46f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff993c9d640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff98c10b6e0 0x7ff98c197600 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff993c9d640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff98c102cc0 0x7ff98c19d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff993c9d640 1 --2- 172.21.15.18:0/3718001942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff98c1023a0 0x7ff98c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff993c9d640 1 -- 172.21.15.18:0/3718001942 >> 172.21.15.18:0/3718001942 conn(0x7ff98c077230 msgr2=0x7ff98c0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff993c9d640 1 -- 172.21.15.18:0/3718001942 shutdown_connections 2024-02-17T13:02:14.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.072+0000 7ff993c9d640 1 -- 172.21.15.18:0/3718001942 wait complete. 2024-02-17T13:02:14.073 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T13:02:14.073 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T13:02:14.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:13 smithi119 ceph-mon[36188]: pgmap v387: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-02-17T13:02:14.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.194+0000 7f0573925640 1 -- 172.21.15.18:0/2632131271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c105380 msgr2=0x7f056c10cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.194+0000 7f0573925640 1 --2- 172.21.15.18:0/2632131271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c105380 0x7f056c10cf50 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f0564009cd0 tx=0x7f056402c0f0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.194+0000 7f0573925640 1 -- 172.21.15.18:0/2632131271 shutdown_connections 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.194+0000 7f0573925640 1 --2- 172.21.15.18:0/2632131271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c105380 0x7f056c10cf50 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.194+0000 7f0573925640 1 --2- 172.21.15.18:0/2632131271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c100c60 0x7f056c104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.194+0000 7f0573925640 1 --2- 172.21.15.18:0/2632131271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f056c1002b0 0x7f056c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.194+0000 7f0573925640 1 -- 172.21.15.18:0/2632131271 >> 172.21.15.18:0/2632131271 conn(0x7f056c077120 msgr2=0x7f056c077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.194+0000 7f0573925640 1 -- 172.21.15.18:0/2632131271 shutdown_connections 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.194+0000 7f0573925640 1 -- 172.21.15.18:0/2632131271 wait complete. 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0573925640 1 Processor -- start 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0573925640 1 -- start start 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0573925640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f056c1002b0 0x7f056c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0573925640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c105380 0x7f056c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0573925640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c19af90 0x7f056c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0573925640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c19b980 con 0x7f056c105380 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0573925640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c19baf0 con 0x7f056c19af90 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0573925640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c19bc60 con 0x7f056c1002b0 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0570ec0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c105380 0x7f056c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0570ec0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c105380 0x7f056c196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39370/0 (socket says 172.21.15.18:39370) 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0570ec0640 1 -- 172.21.15.18:0/3376631734 learned_addr learned my addr 172.21.15.18:0/3376631734 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f05716c1640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f056c1002b0 0x7f056c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0571ec2640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c19af90 0x7f056c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0570ec0640 1 -- 172.21.15.18:0/3376631734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f056c1002b0 msgr2=0x7f056c1966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0570ec0640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f056c1002b0 0x7f056c1966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0570ec0640 1 -- 172.21.15.18:0/3376631734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c19af90 msgr2=0x7f056c19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0570ec0640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c19af90 0x7f056c19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0570ec0640 1 -- 172.21.15.18:0/3376631734 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0564009900 con 0x7f056c105380 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.195+0000 7f0570ec0640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c105380 0x7f056c196c10 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f055c00d770 tx=0x7f055c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.196+0000 7f05627fc640 1 -- 172.21.15.18:0/3376631734 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c009c30 con 0x7f056c105380 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.196+0000 7f05627fc640 1 -- 172.21.15.18:0/3376631734 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f055c00de80 con 0x7f056c105380 2024-02-17T13:02:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.196+0000 7f0573925640 1 -- 172.21.15.18:0/3376631734 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c1a3240 con 0x7f056c105380 2024-02-17T13:02:14.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.196+0000 7f05627fc640 1 -- 172.21.15.18:0/3376631734 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c00f6b0 con 0x7f056c105380 2024-02-17T13:02:14.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.196+0000 7f0573925640 1 -- 172.21.15.18:0/3376631734 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c1a36f0 con 0x7f056c105380 2024-02-17T13:02:14.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.197+0000 7f0573925640 1 -- 172.21.15.18:0/3376631734 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c101780 con 0x7f056c105380 2024-02-17T13:02:14.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.198+0000 7f05627fc640 1 -- 172.21.15.18:0/3376631734 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f055c005030 con 0x7f056c105380 2024-02-17T13:02:14.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.198+0000 7f05627fc640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f05580a2230 0x7f05580a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.199+0000 7f05716c1640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f05580a2230 0x7f05580a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.199+0000 7f05627fc640 1 -- 172.21.15.18:0/3376631734 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f055c0cdd50 con 0x7f056c105380 2024-02-17T13:02:14.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.199+0000 7f05716c1640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f05580a2230 0x7f05580a46f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0568009bd0 tx=0x7f0568005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.202+0000 7f05627fc640 1 -- 172.21.15.18:0/3376631734 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f055c096a50 con 0x7f056c105380 2024-02-17T13:02:14.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:13 smithi146 ceph-mon[36454]: pgmap v387: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-02-17T13:02:14.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 615056 4.85 MB/sec execute 569 sec latency 14.297 ms 2024-02-17T13:02:14.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.434+0000 7f0573925640 1 -- 172.21.15.18:0/3376631734 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f056c108610 con 0x7f056c105380 2024-02-17T13:02:14.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.435+0000 7f05627fc640 1 -- 172.21.15.18:0/3376631734 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+11392 (secure 0 0 0) 0x7f055c021090 con 0x7f056c105380 2024-02-17T13:02:14.425 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:02:14.425 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T13:01:34.948363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T13:02:14.425 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 51 2024-02-17T13:02:14.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.437+0000 7f0573925640 1 -- 172.21.15.18:0/3376631734 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f05580a2230 msgr2=0x7f05580a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.437+0000 7f0573925640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f05580a2230 0x7f05580a46f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0568009bd0 tx=0x7f0568005f90 comp rx=0 tx=0).stop 2024-02-17T13:02:14.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.437+0000 7f0573925640 1 -- 172.21.15.18:0/3376631734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c105380 msgr2=0x7f056c196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.437+0000 7f0573925640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c105380 0x7f056c196c10 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f055c00d770 tx=0x7f055c00dc30 comp rx=0 tx=0).stop 2024-02-17T13:02:14.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.437+0000 7f0573925640 1 -- 172.21.15.18:0/3376631734 shutdown_connections 2024-02-17T13:02:14.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.437+0000 7f0573925640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f05580a2230 0x7f05580a46f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.437+0000 7f0573925640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c19af90 0x7f056c19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.437+0000 7f0573925640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f056c105380 0x7f056c196c10 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.437+0000 7f0573925640 1 --2- 172.21.15.18:0/3376631734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f056c1002b0 0x7f056c1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.437+0000 7f0573925640 1 -- 172.21.15.18:0/3376631734 >> 172.21.15.18:0/3376631734 conn(0x7f056c077120 msgr2=0x7f056c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.437+0000 7f0573925640 1 -- 172.21.15.18:0/3376631734 shutdown_connections 2024-02-17T13:02:14.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.437+0000 7f0573925640 1 -- 172.21.15.18:0/3376631734 wait complete. 2024-02-17T13:02:14.438 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:02:14.438 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T13:02:14.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.560+0000 7f100ae39640 1 -- 172.21.15.18:0/2285210392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10040ff7b0 msgr2=0x7f10040ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.560+0000 7f100ae39640 1 --2- 172.21.15.18:0/2285210392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10040ff7b0 0x7f10040ffc30 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009900 tx=0x7f0ff002bc30 comp rx=0 tx=0).stop 2024-02-17T13:02:14.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.560+0000 7f100ae39640 1 -- 172.21.15.18:0/2285210392 shutdown_connections 2024-02-17T13:02:14.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.560+0000 7f100ae39640 1 --2- 172.21.15.18:0/2285210392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1004100170 0x7f100410c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.560+0000 7f100ae39640 1 --2- 172.21.15.18:0/2285210392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10040ff7b0 0x7f10040ffc30 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.560+0000 7f100ae39640 1 --2- 172.21.15.18:0/2285210392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10041057c0 0x7f1004105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.560+0000 7f100ae39640 1 -- 172.21.15.18:0/2285210392 >> 172.21.15.18:0/2285210392 conn(0x7f1004076640 msgr2=0x7f10040fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.560+0000 7f100ae39640 1 -- 172.21.15.18:0/2285210392 shutdown_connections 2024-02-17T13:02:14.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.560+0000 7f100ae39640 1 -- 172.21.15.18:0/2285210392 wait complete. 2024-02-17T13:02:14.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f100ae39640 1 Processor -- start 2024-02-17T13:02:14.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f100ae39640 1 -- start start 2024-02-17T13:02:14.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f100ae39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10040ff7b0 0x7f100419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f100ae39640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1004100170 0x7f100419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f100ae39640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10041057c0 0x7f10041975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f100ae39640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1004197c10 con 0x7f10041057c0 2024-02-17T13:02:14.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f100ae39640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1004197d80 con 0x7f10040ff7b0 2024-02-17T13:02:14.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f100ae39640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1004197f40 con 0x7f1004100170 2024-02-17T13:02:14.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f10093d6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10041057c0 0x7f10041975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f1008bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10040ff7b0 0x7f100419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f1003fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1004100170 0x7f100419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f10093d6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10041057c0 0x7f10041975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39390/0 (socket says 172.21.15.18:39390) 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.561+0000 7f10093d6640 1 -- 172.21.15.18:0/4240303014 learned_addr learned my addr 172.21.15.18:0/4240303014 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.562+0000 7f1008bd5640 1 -- 172.21.15.18:0/4240303014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1004100170 msgr2=0x7f100419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.562+0000 7f1008bd5640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1004100170 0x7f100419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.562+0000 7f1008bd5640 1 -- 172.21.15.18:0/4240303014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10041057c0 msgr2=0x7f10041975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.562+0000 7f1008bd5640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10041057c0 0x7f10041975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.562+0000 7f1008bd5640 1 -- 172.21.15.18:0/4240303014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0009560 con 0x7f10040ff7b0 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.562+0000 7f10093d6640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10041057c0 0x7f10041975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.562+0000 7f1008bd5640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10040ff7b0 0x7f100419d410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800e9f0 tx=0x7f0ff800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.562+0000 7f1001ffb640 1 -- 172.21.15.18:0/4240303014 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff800ccc0 con 0x7f10040ff7b0 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.562+0000 7f100ae39640 1 -- 172.21.15.18:0/4240303014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1004198220 con 0x7f10040ff7b0 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.562+0000 7f1001ffb640 1 -- 172.21.15.18:0/4240303014 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ff800ce20 con 0x7f10040ff7b0 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.562+0000 7f1001ffb640 1 -- 172.21.15.18:0/4240303014 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8004a00 con 0x7f10040ff7b0 2024-02-17T13:02:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.562+0000 7f100ae39640 1 -- 172.21.15.18:0/4240303014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10041a44b0 con 0x7f10040ff7b0 2024-02-17T13:02:14.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.564+0000 7f1001ffb640 1 -- 172.21.15.18:0/4240303014 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0ff80050b0 con 0x7f10040ff7b0 2024-02-17T13:02:14.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f100da09740 1 Processor -- start 2024-02-17T13:02:14.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f100da09740 1 -- start start 2024-02-17T13:02:14.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f1001ffb640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0ff40a2230 0x7f0ff40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f100da09740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f100da09740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f100da09740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab3d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f100da09740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ab383960 con 0x55c6ab3c8b70 2024-02-17T13:02:14.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f100da09740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ab337310 con 0x55c6ab3c8250 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f100da09740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ab334840 con 0x55c6ab3d1750 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f10093d6640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0ff40a2230 0x7f0ff40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f10093d6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab3d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f1001ffb640 1 -- 172.21.15.18:0/4240303014 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0ff8014070 con 0x7f10040ff7b0 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f10093d6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab3d3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:33970/0 (socket says 172.21.15.18:33970) 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f10093d6640 1 -- 172.21.15.18:0/4141338068 learned_addr learned my addr 172.21.15.18:0/4141338068 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f1008bd5640 1 --2- 172.21.15.18:0/4141338068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.565+0000 7f1003fff640 1 --2- 172.21.15.18:0/4141338068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.566+0000 7f10093d6640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0ff40a2230 0x7f0ff40a46f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc013d20 tx=0x7f0ffc013460 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.566+0000 7f10093d6640 1 -- 172.21.15.18:0/4141338068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 msgr2=0x55c6ab3c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.566+0000 7f10093d6640 1 --2- 172.21.15.18:0/4141338068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.566+0000 7f10093d6640 1 -- 172.21.15.18:0/4141338068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 msgr2=0x55c6ab3d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.566+0000 7f10093d6640 1 --2- 172.21.15.18:0/4141338068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.566+0000 7f10093d6640 1 -- 172.21.15.18:0/4141338068 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6ab2f2060 con 0x55c6ab3d1750 2024-02-17T13:02:14.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.566+0000 7f1008bd5640 1 --2- 172.21.15.18:0/4141338068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.566+0000 7f10093d6640 1 --2- 172.21.15.18:0/4141338068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab3d3b40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00fc90 tx=0x7f0ffc020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d003ff882d113eb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.566+0000 7f0feeffd640 1 -- 172.21.15.18:0/4141338068 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc020d40 con 0x55c6ab3d1750 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.566+0000 7f0feeffd640 1 -- 172.21.15.18:0/4141338068 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ffc0130d0 con 0x55c6ab3d1750 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.566+0000 7f0feeffd640 1 -- 172.21.15.18:0/4141338068 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc00e610 con 0x55c6ab3d1750 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.567+0000 7f100da09740 1 -- 172.21.15.18:0/4141338068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 msgr2=0x55c6ab3d3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.567+0000 7f100da09740 1 --2- 172.21.15.18:0/4141338068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab3d3b40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00fc90 tx=0x7f0ffc020000 comp rx=0 tx=0).stop 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.567+0000 7f100da09740 1 -- 172.21.15.18:0/4141338068 shutdown_connections 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.567+0000 7f100da09740 1 --2- 172.21.15.18:0/4141338068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab3d3b40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.567+0000 7f100da09740 1 --2- 172.21.15.18:0/4141338068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.567+0000 7f100da09740 1 --2- 172.21.15.18:0/4141338068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.567+0000 7f100da09740 1 -- 172.21.15.18:0/4141338068 >> 172.21.15.18:0/4141338068 conn(0x55c6ab3c2980 msgr2=0x55c6ab3c2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.567+0000 7f100da09740 1 -- 172.21.15.18:0/4141338068 shutdown_connections 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.567+0000 7f100da09740 1 -- 172.21.15.18:0/4141338068 wait complete. 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.567+0000 7f100da09740 1 -- start start 2024-02-17T13:02:14.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.567+0000 7f100da09740 1 Processor -- start 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f100da09740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8250 0x55c6ab3ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f100da09740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f100da09740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3d1750 0x55c6ab3cf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f100da09740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ab330020 con 0x55c6ab3c8250 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f100da09740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ab003430 con 0x55c6ab3d1750 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f100da09740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6aafbfe00 con 0x55c6ab3c8b70 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f100da09740 10 client.0 _ll_register_callbacks cb 0x55c6ab2d7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f1008bd5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8250 0x55c6ab3ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f1008bd5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8250 0x55c6ab3ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39412/0 (socket says 172.21.15.18:39412) 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f1008bd5640 1 -- 172.21.15.18:0/2506704279 learned_addr learned my addr 172.21.15.18:0/2506704279 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f1003fff640 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f10093d6640 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3d1750 0x55c6ab3cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f1008bd5640 1 -- 172.21.15.18:0/2506704279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3c8b70 msgr2=0x55c6ab3cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f1008bd5640 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f1008bd5640 1 -- 172.21.15.18:0/2506704279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3d1750 msgr2=0x55c6ab3cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f1008bd5640 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3d1750 0x55c6ab3cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f1008bd5640 1 -- 172.21.15.18:0/2506704279 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ffc013e10 con 0x55c6ab3c8250 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f1003fff640 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f1008bd5640 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8250 0x55c6ab3ce7a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f0ff809a6d0 tx=0x7f0ff809b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a4d00fee1f0ab3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f0fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f0fe3fff640 1 -- 172.21.15.18:0/2506704279 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8048040 con 0x55c6ab3c8250 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f0fe3fff640 1 -- 172.21.15.18:0/2506704279 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0ff80483a0 con 0x55c6ab3c8250 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.568+0000 7f100da09740 1 -- 172.21.15.18:0/2506704279 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6ab02f7e0 con 0x55c6ab3c8250 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.569+0000 7f0fe3fff640 1 -- 172.21.15.18:0/2506704279 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ff80cbbc0 con 0x55c6ab3c8250 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.569+0000 7f100da09740 10 client.19935 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.569+0000 7f100da09740 1 -- 172.21.15.18:0/2506704279 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55c6ab32e6a0 con 0x55c6ab3c8250 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.569+0000 7f0fe3fff640 1 -- 172.21.15.18:0/2506704279 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c6ab02f7e0 con 0x55c6ab3c8250 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.569+0000 7f0fe3fff640 1 -- 172.21.15.18:0/2506704279 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ff80a2d70 con 0x55c6ab3c8250 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.569+0000 7f100da09740 10 client.19935 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:14.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.569+0000 7f100da09740 10 client.19935 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.569+0000 7f100da09740 20 client.19935 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.569+0000 7f100da09740 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c6ab3e1a70 0x55c6ab3e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.569+0000 7f100da09740 4 client.19935 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.569+0000 7f100da09740 1 -- 172.21.15.18:0/2506704279 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6ab06c720 con 0x55c6ab3e1a70 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.569+0000 7f1003fff640 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c6ab3e1a70 0x55c6ab3e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f1003fff640 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c6ab3e1a70 0x55c6ab3e3e60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c3ae06469d873d16 server_cookie=77c42baf0110f0ae in_seq=0 out_seq=0 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f0fe3fff640 10 client.19935 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f0fe3fff640 1 -- 172.21.15.18:0/2506704279 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c6ab06c720 con 0x55c6ab3e1a70 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f0fe3fff640 10 client.19935 handle_command_reply: tid=0 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f100da09740 1 client.19935 shutdown 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f100da09740 1 -- 172.21.15.18:0/2506704279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8250 msgr2=0x55c6ab3ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f100da09740 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8250 0x55c6ab3ce7a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f0ff809a6d0 tx=0x7f0ff809b0d0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f100da09740 1 -- 172.21.15.18:0/2506704279 shutdown_connections 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f100da09740 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c6ab3e1a70 0x55c6ab3e3e60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f100da09740 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3d1750 0x55c6ab3cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f100da09740 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f100da09740 1 --2- 172.21.15.18:0/2506704279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8250 0x55c6ab3ce7a0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f0fe3fff640 0 client.19935 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:02:14.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f100da09740 1 -- 172.21.15.18:0/2506704279 >> 172.21.15.18:0/2506704279 conn(0x55c6ab3c2980 msgr2=0x55c6ab3c46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f100da09740 1 -- 172.21.15.18:0/2506704279 shutdown_connections 2024-02-17T13:02:14.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.570+0000 7f100da09740 1 -- 172.21.15.18:0/2506704279 wait complete. 2024-02-17T13:02:14.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.571+0000 7f100da09740 20 client.19935 trim_cache size 0 max 16384 2024-02-17T13:02:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.582+0000 7f100da09740 1 Processor -- start 2024-02-17T13:02:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.582+0000 7f100da09740 1 -- start start 2024-02-17T13:02:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.582+0000 7f100da09740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.582+0000 7f100da09740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.582+0000 7f100da09740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab3cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.582+0000 7f100da09740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ab2b5130 con 0x55c6ab3c8b70 2024-02-17T13:02:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.582+0000 7f100da09740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ab3be080 con 0x55c6ab3c8250 2024-02-17T13:02:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.582+0000 7f100da09740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ab3e1820 con 0x55c6ab3d1750 2024-02-17T13:02:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f1003fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f1003fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39420/0 (socket says 172.21.15.18:39420) 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f1008bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f1003fff640 1 -- 172.21.15.18:0/1724405647 learned_addr learned my addr 172.21.15.18:0/1724405647 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f10093d6640 1 --2- 172.21.15.18:0/1724405647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab3cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f1003fff640 1 -- 172.21.15.18:0/1724405647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 msgr2=0x55c6ab3cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f1003fff640 1 --2- 172.21.15.18:0/1724405647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab3cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f1003fff640 1 -- 172.21.15.18:0/1724405647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 msgr2=0x55c6ab3d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f1003fff640 1 --2- 172.21.15.18:0/1724405647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3d0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f1003fff640 1 -- 172.21.15.18:0/1724405647 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6ab2cac20 con 0x55c6ab3c8b70 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f1008bd5640 1 --2- 172.21.15.18:0/1724405647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f1003fff640 1 --2- 172.21.15.18:0/1724405647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3cf040 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009900 tx=0x7f0ff0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4843773fe9878939 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f0fee7fc640 1 -- 172.21.15.18:0/1724405647 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0049ca0 con 0x55c6ab3c8b70 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f0fee7fc640 1 -- 172.21.15.18:0/1724405647 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ff0049e00 con 0x55c6ab3c8b70 2024-02-17T13:02:14.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.583+0000 7f0fee7fc640 1 -- 172.21.15.18:0/1724405647 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00495e0 con 0x55c6ab3c8b70 2024-02-17T13:02:14.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.584+0000 7f100da09740 1 -- 172.21.15.18:0/1724405647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 msgr2=0x55c6ab3cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.584+0000 7f100da09740 1 --2- 172.21.15.18:0/1724405647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3cf040 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009900 tx=0x7f0ff0009260 comp rx=0 tx=0).stop 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.584+0000 7f100da09740 1 -- 172.21.15.18:0/1724405647 shutdown_connections 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.584+0000 7f100da09740 1 --2- 172.21.15.18:0/1724405647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab3cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.584+0000 7f100da09740 1 --2- 172.21.15.18:0/1724405647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3cf040 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.584+0000 7f100da09740 1 --2- 172.21.15.18:0/1724405647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.584+0000 7f100da09740 1 -- 172.21.15.18:0/1724405647 >> 172.21.15.18:0/1724405647 conn(0x55c6ab3c2980 msgr2=0x55c6ab3c9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.584+0000 7f100da09740 1 -- 172.21.15.18:0/1724405647 shutdown_connections 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.584+0000 7f100da09740 1 -- 172.21.15.18:0/1724405647 wait complete. 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.584+0000 7f100da09740 1 -- start start 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.584+0000 7f100da09740 1 Processor -- start 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f100da09740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3d7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f100da09740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3d7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f100da09740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab437f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f100da09740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ab3d8300 con 0x55c6ab3c8b70 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f100da09740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ab4384a0 con 0x55c6ab3c8250 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f100da09740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ab438630 con 0x55c6ab3d1750 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f100da09740 10 client.0 _ll_register_callbacks cb 0x7f0ffc009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f1003fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3d7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f1003fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3d7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39432/0 (socket says 172.21.15.18:39432) 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f1008bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3d7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f1003fff640 1 -- 172.21.15.18:0/283108272 learned_addr learned my addr 172.21.15.18:0/283108272 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f10093d6640 1 --2- 172.21.15.18:0/283108272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab437f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f1003fff640 1 -- 172.21.15.18:0/283108272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 msgr2=0x55c6ab437f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f1003fff640 1 --2- 172.21.15.18:0/283108272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab437f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f1003fff640 1 -- 172.21.15.18:0/283108272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 msgr2=0x55c6ab3d7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f1003fff640 1 --2- 172.21.15.18:0/283108272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3d7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f1003fff640 1 -- 172.21.15.18:0/283108272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6ab32e6a0 con 0x55c6ab3c8b70 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f1003fff640 1 --2- 172.21.15.18:0/283108272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3d7bc0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009230 tx=0x7f0ff002c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba2004c22ff0fdd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f0fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f0fe3fff640 1 -- 172.21.15.18:0/283108272 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff002cc70 con 0x55c6ab3c8b70 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f100da09740 1 -- 172.21.15.18:0/283108272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ffc0130d0 con 0x55c6ab3c8b70 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.585+0000 7f0fe3fff640 1 -- 172.21.15.18:0/283108272 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0ff0058030 con 0x55c6ab3c8b70 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f0fe3fff640 1 -- 172.21.15.18:0/283108272 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ff00042a0 con 0x55c6ab3c8b70 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f100da09740 10 client.19941 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f100da09740 1 -- 172.21.15.18:0/283108272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f10040692c0 con 0x55c6ab3c8b70 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f0fe3fff640 1 -- 172.21.15.18:0/283108272 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ffc0130d0 con 0x55c6ab3c8b70 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f0fe3fff640 1 -- 172.21.15.18:0/283108272 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ff003c040 con 0x55c6ab3c8b70 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f100da09740 10 client.19941 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f100da09740 10 client.19941 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f100da09740 20 client.19941 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f100da09740 1 --2- 172.21.15.18:0/283108272 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c6ab4399e0 0x55c6ab43bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f100da09740 4 client.19941 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f100da09740 1 -- 172.21.15.18:0/283108272 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6ab0bb890 con 0x55c6ab4399e0 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f1008bd5640 1 --2- 172.21.15.18:0/283108272 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c6ab4399e0 0x55c6ab43bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f1008bd5640 1 --2- 172.21.15.18:0/283108272 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c6ab4399e0 0x55c6ab43bdd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c429aab186678583 server_cookie=b905c866f13b6dc0 in_seq=0 out_seq=0 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.586+0000 7f0fe3fff640 10 client.19941 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f0fe3fff640 1 -- 172.21.15.18:0/283108272 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6ab0bb890 con 0x55c6ab4399e0 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f0fe3fff640 10 client.19941 handle_command_reply: tid=0 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f100da09740 1 client.19941 shutdown 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f100da09740 1 -- 172.21.15.18:0/283108272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 msgr2=0x55c6ab3d7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f100da09740 1 --2- 172.21.15.18:0/283108272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3d7bc0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009230 tx=0x7f0ff002c710 comp rx=0 tx=0).stop 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f100da09740 1 -- 172.21.15.18:0/283108272 shutdown_connections 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f100da09740 1 --2- 172.21.15.18:0/283108272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6ab3d1750 0x55c6ab437f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f100da09740 1 --2- 172.21.15.18:0/283108272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6ab3c8b70 0x55c6ab3d7bc0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f100da09740 1 --2- 172.21.15.18:0/283108272 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x55c6ab4399e0 0x55c6ab43bdd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f100da09740 1 --2- 172.21.15.18:0/283108272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c6ab3c8250 0x55c6ab3d7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f0fe3fff640 0 client.19941 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:02:14.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f100da09740 1 -- 172.21.15.18:0/283108272 >> 172.21.15.18:0/283108272 conn(0x55c6ab3c2980 msgr2=0x55c6ab3c12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f100da09740 1 -- 172.21.15.18:0/283108272 shutdown_connections 2024-02-17T13:02:14.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f100da09740 1 -- 172.21.15.18:0/283108272 wait complete. 2024-02-17T13:02:14.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.587+0000 7f100da09740 20 client.19941 trim_cache size 0 max 16384 2024-02-17T13:02:14.583 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:02:14.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.588+0000 7f100ae39640 1 -- 172.21.15.18:0/4240303014 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0ff40a2230 msgr2=0x7f0ff40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.588+0000 7f100ae39640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0ff40a2230 0x7f0ff40a46f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc013d20 tx=0x7f0ffc013460 comp rx=0 tx=0).stop 2024-02-17T13:02:14.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.588+0000 7f100ae39640 1 -- 172.21.15.18:0/4240303014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10040ff7b0 msgr2=0x7f100419d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.588+0000 7f100ae39640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10040ff7b0 0x7f100419d410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800e9f0 tx=0x7f0ff800eeb0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.588+0000 7f100ae39640 1 -- 172.21.15.18:0/4240303014 shutdown_connections 2024-02-17T13:02:14.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.588+0000 7f100ae39640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0ff40a2230 0x7f0ff40a46f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.588+0000 7f100ae39640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10041057c0 0x7f10041975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.588+0000 7f100ae39640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1004100170 0x7f100419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.588+0000 7f100ae39640 1 --2- 172.21.15.18:0/4240303014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10040ff7b0 0x7f100419d410 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.588+0000 7f100ae39640 1 -- 172.21.15.18:0/4240303014 >> 172.21.15.18:0/4240303014 conn(0x7f1004076640 msgr2=0x7f1004102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.588+0000 7f100ae39640 1 -- 172.21.15.18:0/4240303014 shutdown_connections 2024-02-17T13:02:14.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.588+0000 7f100ae39640 1 -- 172.21.15.18:0/4240303014 wait complete. 2024-02-17T13:02:14.588 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:02:14.588 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T13:02:14.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.704+0000 7f568594f640 1 -- 172.21.15.18:0/1759832887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5680100260 msgr2=0x7f5680100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.704+0000 7f568594f640 1 --2- 172.21.15.18:0/1759832887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5680100260 0x7f5680100640 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f5670009cd0 tx=0x7f567002c0f0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.704+0000 7f568594f640 1 -- 172.21.15.18:0/1759832887 shutdown_connections 2024-02-17T13:02:14.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.704+0000 7f568594f640 1 --2- 172.21.15.18:0/1759832887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5680105330 0x7f568010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.704+0000 7f568594f640 1 --2- 172.21.15.18:0/1759832887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5680100c10 0x7f5680104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.704+0000 7f568594f640 1 --2- 172.21.15.18:0/1759832887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5680100260 0x7f5680100640 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.704+0000 7f568594f640 1 -- 172.21.15.18:0/1759832887 >> 172.21.15.18:0/1759832887 conn(0x7f5680077180 msgr2=0x7f5680077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.704+0000 7f568594f640 1 -- 172.21.15.18:0/1759832887 shutdown_connections 2024-02-17T13:02:14.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.704+0000 7f568594f640 1 -- 172.21.15.18:0/1759832887 wait complete. 2024-02-17T13:02:14.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.705+0000 7f568594f640 1 Processor -- start 2024-02-17T13:02:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.705+0000 7f568594f640 1 -- start start 2024-02-17T13:02:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.705+0000 7f568594f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5680100260 0x7f5680198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.705+0000 7f568594f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5680105330 0x7f5680198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.705+0000 7f568594f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56801990e0 0x7f56801a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.705+0000 7f568594f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801a3d60 con 0x7f56801990e0 2024-02-17T13:02:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.705+0000 7f568594f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801a3ed0 con 0x7f5680105330 2024-02-17T13:02:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.705+0000 7f568594f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801a4090 con 0x7f5680100260 2024-02-17T13:02:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.705+0000 7f567f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56801990e0 0x7f56801a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.705+0000 7f567f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56801990e0 0x7f56801a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39456/0 (socket says 172.21.15.18:39456) 2024-02-17T13:02:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.705+0000 7f567f7fe640 1 -- 172.21.15.18:0/696288362 learned_addr learned my addr 172.21.15.18:0/696288362 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.705+0000 7f567e7fc640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5680105330 0x7f5680198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.706+0000 7f567effd640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5680100260 0x7f5680198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.706+0000 7f567f7fe640 1 -- 172.21.15.18:0/696288362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5680100260 msgr2=0x7f5680198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.706+0000 7f567f7fe640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5680100260 0x7f5680198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.706+0000 7f567f7fe640 1 -- 172.21.15.18:0/696288362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5680105330 msgr2=0x7f5680198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.706+0000 7f567f7fe640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5680105330 0x7f5680198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.706+0000 7f567f7fe640 1 -- 172.21.15.18:0/696288362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5670009900 con 0x7f56801990e0 2024-02-17T13:02:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.706+0000 7f567f7fe640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56801990e0 0x7f56801a3690 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f567800e970 tx=0x7f567800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.706+0000 7f568494d640 1 -- 172.21.15.18:0/696288362 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678009720 con 0x7f56801990e0 2024-02-17T13:02:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.706+0000 7f568494d640 1 -- 172.21.15.18:0/696288362 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5678005500 con 0x7f56801990e0 2024-02-17T13:02:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.706+0000 7f568594f640 1 -- 172.21.15.18:0/696288362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56801a4370 con 0x7f56801990e0 2024-02-17T13:02:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.706+0000 7f568494d640 1 -- 172.21.15.18:0/696288362 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678010430 con 0x7f56801990e0 2024-02-17T13:02:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.706+0000 7f568594f640 1 -- 172.21.15.18:0/696288362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56801a4830 con 0x7f56801990e0 2024-02-17T13:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.708+0000 7f568494d640 1 -- 172.21.15.18:0/696288362 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5678005020 con 0x7f56801990e0 2024-02-17T13:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.708+0000 7f5688603740 1 Processor -- start 2024-02-17T13:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f5688603740 1 -- start start 2024-02-17T13:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f5688603740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 0x55c516c0a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f5688603740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0ab70 0x55c516c13210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f5688603740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c13750 0x55c516c15b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f5688603740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c516bc5960 con 0x55c516c0a250 2024-02-17T13:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f5688603740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c516b79310 con 0x55c516c0ab70 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f5688603740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c516b76840 con 0x55c516c13750 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f568494d640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f566c0a2230 0x7f566c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f567e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 0x55c516c0a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f567e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 0x55c516c0a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39458/0 (socket says 172.21.15.18:39458) 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f567effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0ab70 0x55c516c13210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f567e7fc640 1 -- 172.21.15.18:0/2662939819 learned_addr learned my addr 172.21.15.18:0/2662939819 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f567e7fc640 1 --2- 172.21.15.18:0/2662939819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c13750 0x55c516c15b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f567f7fe640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f566c0a2230 0x7f566c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f568494d640 1 -- 172.21.15.18:0/696288362 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5678014070 con 0x7f56801990e0 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f567effd640 1 -- 172.21.15.18:0/2662939819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c13750 msgr2=0x55c516c15b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f567effd640 1 --2- 172.21.15.18:0/2662939819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c13750 0x55c516c15b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f567effd640 1 -- 172.21.15.18:0/2662939819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 msgr2=0x55c516c0a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f567effd640 1 --2- 172.21.15.18:0/2662939819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 0x55c516c0a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f567effd640 1 -- 172.21.15.18:0/2662939819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c516b34060 con 0x55c516c0ab70 2024-02-17T13:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.709+0000 7f567e7fc640 1 --2- 172.21.15.18:0/2662939819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 0x55c516c0a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.710+0000 7f567f7fe640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f566c0a2230 0x7f566c0a46f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f56780977c0 tx=0x7f56780049b0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.710+0000 7f567effd640 1 --2- 172.21.15.18:0/2662939819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0ab70 0x55c516c13210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f567002c700 tx=0x7f5670005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed1b8775a084026 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.710+0000 7f5675ffb640 1 -- 172.21.15.18:0/2662939819 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5670034440 con 0x55c516c0ab70 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.710+0000 7f5675ffb640 1 -- 172.21.15.18:0/2662939819 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56700345a0 con 0x55c516c0ab70 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.710+0000 7f5675ffb640 1 -- 172.21.15.18:0/2662939819 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5670003bd0 con 0x55c516c0ab70 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.710+0000 7f5688603740 1 -- 172.21.15.18:0/2662939819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0ab70 msgr2=0x55c516c13210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.710+0000 7f5688603740 1 --2- 172.21.15.18:0/2662939819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0ab70 0x55c516c13210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f567002c700 tx=0x7f5670005f70 comp rx=0 tx=0).stop 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 -- 172.21.15.18:0/2662939819 shutdown_connections 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 --2- 172.21.15.18:0/2662939819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c13750 0x55c516c15b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 --2- 172.21.15.18:0/2662939819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0ab70 0x55c516c13210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 --2- 172.21.15.18:0/2662939819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 0x55c516c0a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 -- 172.21.15.18:0/2662939819 >> 172.21.15.18:0/2662939819 conn(0x55c516c04980 msgr2=0x55c516c04d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 -- 172.21.15.18:0/2662939819 shutdown_connections 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 -- 172.21.15.18:0/2662939819 wait complete. 2024-02-17T13:02:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 -- start start 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 Processor -- start 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 0x55c516c107a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c0ab70 0x55c516c10ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c13750 0x55c516c11220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c516b72020 con 0x55c516c0a250 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c516845430 con 0x55c516c13750 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c516801e00 con 0x55c516c0ab70 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f5688603740 10 client.0 _ll_register_callbacks cb 0x55c516b19cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f567e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 0x55c516c107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f567e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 0x55c516c107a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39464/0 (socket says 172.21.15.18:39464) 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f567e7fc640 1 -- 172.21.15.18:0/4030044891 learned_addr learned my addr 172.21.15.18:0/4030044891 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.711+0000 7f567effd640 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c0ab70 0x55c516c10ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f567e7fc640 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c13750 0x55c516c11220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f567e7fc640 1 -- 172.21.15.18:0/4030044891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c0ab70 msgr2=0x55c516c10ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f567e7fc640 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c0ab70 0x55c516c10ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f567e7fc640 1 -- 172.21.15.18:0/4030044891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c13750 msgr2=0x55c516c11220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f567e7fc640 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c13750 0x55c516c11220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f567e7fc640 1 -- 172.21.15.18:0/4030044891 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568010d710 con 0x55c516c0a250 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f567e7fc640 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 0x55c516c107a0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f5668013e20 tx=0x7f5668005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a8bb1c5286b40a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f565effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f565effd640 1 -- 172.21.15.18:0/4030044891 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668004310 con 0x55c516c0a250 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f565effd640 1 -- 172.21.15.18:0/4030044891 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5668004470 con 0x55c516c0a250 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f5688603740 1 -- 172.21.15.18:0/4030044891 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5168717e0 con 0x55c516c0a250 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f565effd640 1 -- 172.21.15.18:0/4030044891 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5668005020 con 0x55c516c0a250 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f5688603740 10 client.19953 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f5688603740 1 -- 172.21.15.18:0/4030044891 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55c516b706a0 con 0x55c516c0a250 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.712+0000 7f565effd640 1 -- 172.21.15.18:0/4030044891 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5668028020 con 0x55c516c0a250 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.713+0000 7f565effd640 1 -- 172.21.15.18:0/4030044891 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5668023070 con 0x55c516c0a250 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.713+0000 7f5688603740 10 client.19953 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.713+0000 7f5688603740 10 client.19953 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.713+0000 7f5688603740 20 client.19953 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.713+0000 7f5688603740 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c516c23b80 0x55c516c25fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.713+0000 7f5688603740 4 client.19953 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.713+0000 7f5688603740 1 -- 172.21.15.18:0/4030044891 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5168ae720 con 0x55c516c23b80 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.713+0000 7f567effd640 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c516c23b80 0x55c516c25fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.713+0000 7f567effd640 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c516c23b80 0x55c516c25fa0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb82aecc133d00bd server_cookie=e3472262992c133d in_seq=0 out_seq=0 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.713+0000 7f565effd640 10 client.19953 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f565effd640 1 -- 172.21.15.18:0/4030044891 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5168ae720 con 0x55c516c23b80 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f565effd640 10 client.19953 handle_command_reply: tid=0 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f5688603740 1 client.19953 shutdown 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f5688603740 1 -- 172.21.15.18:0/4030044891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 msgr2=0x55c516c107a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f5688603740 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 0x55c516c107a0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f5668013e20 tx=0x7f5668005f90 comp rx=0 tx=0).stop 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f5688603740 1 -- 172.21.15.18:0/4030044891 shutdown_connections 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f5688603740 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c516c23b80 0x55c516c25fa0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f5688603740 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c13750 0x55c516c11220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f5688603740 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c0ab70 0x55c516c10ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f5688603740 1 --2- 172.21.15.18:0/4030044891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0a250 0x55c516c107a0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f565effd640 0 client.19953 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:02:14.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f5688603740 1 -- 172.21.15.18:0/4030044891 >> 172.21.15.18:0/4030044891 conn(0x55c516c04980 msgr2=0x55c516c066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f5688603740 1 -- 172.21.15.18:0/4030044891 shutdown_connections 2024-02-17T13:02:14.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f5688603740 1 -- 172.21.15.18:0/4030044891 wait complete. 2024-02-17T13:02:14.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.714+0000 7f5688603740 20 client.19953 trim_cache size 0 max 16384 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f5688603740 1 Processor -- start 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f5688603740 1 -- start start 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f5688603740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0a250 0x55c516c12ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f5688603740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c0ab70 0x55c516c0bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f5688603740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c13750 0x55c516c0c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f5688603740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c516af7130 con 0x55c516c13750 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f5688603740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c516c20620 con 0x55c516c0a250 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f5688603740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c516c09eb0 con 0x55c516c0ab70 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f567e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c13750 0x55c516c0c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f567e7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c13750 0x55c516c0c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39476/0 (socket says 172.21.15.18:39476) 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f567e7fc640 1 -- 172.21.15.18:0/2699899549 learned_addr learned my addr 172.21.15.18:0/2699899549 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f567e7fc640 1 --2- 172.21.15.18:0/2699899549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0a250 0x55c516c12ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f567effd640 1 --2- 172.21.15.18:0/2699899549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c0ab70 0x55c516c0bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f567e7fc640 1 -- 172.21.15.18:0/2699899549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c0ab70 msgr2=0x55c516c0bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f567e7fc640 1 --2- 172.21.15.18:0/2699899549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c0ab70 0x55c516c0bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f567e7fc640 1 -- 172.21.15.18:0/2699899549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0a250 msgr2=0x55c516c12ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f567e7fc640 1 --2- 172.21.15.18:0/2699899549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0a250 0x55c516c12ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.727+0000 7f567e7fc640 1 -- 172.21.15.18:0/2699899549 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c516b0cc20 con 0x55c516c13750 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f567e7fc640 1 --2- 172.21.15.18:0/2699899549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c13750 0x55c516c0c010 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f5668009f90 tx=0x7f5668005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70b18b156db64e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f56757fa640 1 -- 172.21.15.18:0/2699899549 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56680139e0 con 0x55c516c13750 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f56757fa640 1 -- 172.21.15.18:0/2699899549 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5668028030 con 0x55c516c13750 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f56757fa640 1 -- 172.21.15.18:0/2699899549 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668004ab0 con 0x55c516c13750 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f5688603740 1 -- 172.21.15.18:0/2699899549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c13750 msgr2=0x55c516c0c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f5688603740 1 --2- 172.21.15.18:0/2699899549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c13750 0x55c516c0c010 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f5668009f90 tx=0x7f5668005970 comp rx=0 tx=0).stop 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f5688603740 1 -- 172.21.15.18:0/2699899549 shutdown_connections 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f5688603740 1 --2- 172.21.15.18:0/2699899549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c13750 0x55c516c0c010 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f5688603740 1 --2- 172.21.15.18:0/2699899549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c0ab70 0x55c516c0bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f5688603740 1 --2- 172.21.15.18:0/2699899549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0a250 0x55c516c12ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f5688603740 1 -- 172.21.15.18:0/2699899549 >> 172.21.15.18:0/2699899549 conn(0x55c516c04980 msgr2=0x55c516bffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f5688603740 1 -- 172.21.15.18:0/2699899549 shutdown_connections 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.728+0000 7f5688603740 1 -- 172.21.15.18:0/2699899549 wait complete. 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f5688603740 1 -- start start 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f5688603740 1 Processor -- start 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f5688603740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0a250 0x55c516c195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f5688603740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0ab70 0x55c516c19b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f5688603740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c13750 0x55c516c7a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f5688603740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c516c1a210 con 0x55c516c0ab70 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f5688603740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c516c7a5d0 con 0x55c516c0a250 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f5688603740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c516c7a710 con 0x55c516c13750 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f5688603740 10 client.0 _ll_register_callbacks cb 0x7f5678005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f567effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0ab70 0x55c516c19b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f567effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0ab70 0x55c516c19b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39482/0 (socket says 172.21.15.18:39482) 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f567effd640 1 -- 172.21.15.18:0/1227345590 learned_addr learned my addr 172.21.15.18:0/1227345590 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f567e7fc640 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0a250 0x55c516c195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f567e7fc640 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c13750 0x55c516c7a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f567effd640 1 -- 172.21.15.18:0/1227345590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c13750 msgr2=0x55c516c7a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f567effd640 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c13750 0x55c516c7a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f567effd640 1 -- 172.21.15.18:0/1227345590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0a250 msgr2=0x55c516c195d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f567effd640 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0a250 0x55c516c195d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.729+0000 7f567effd640 1 -- 172.21.15.18:0/1227345590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c516b706a0 con 0x55c516c0ab70 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f567effd640 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0ab70 0x55c516c19b10 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f567002c630 tx=0x7f5670000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef5764b6e80391a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f565effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f565effd640 1 -- 172.21.15.18:0/1227345590 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5670034440 con 0x55c516c0ab70 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f565effd640 1 -- 172.21.15.18:0/1227345590 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f567004f050 con 0x55c516c0ab70 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f5688603740 1 -- 172.21.15.18:0/1227345590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56700345a0 con 0x55c516c0ab70 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f565effd640 1 -- 172.21.15.18:0/1227345590 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5670032b60 con 0x55c516c0ab70 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f5688603740 10 client.19959 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f5688603740 1 -- 172.21.15.18:0/1227345590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55c516b328c0 con 0x55c516c0ab70 2024-02-17T13:02:14.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f565effd640 1 -- 172.21.15.18:0/1227345590 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5670062080 con 0x55c516c0ab70 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f565effd640 1 -- 172.21.15.18:0/1227345590 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f567005d070 con 0x55c516c0ab70 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f5688603740 10 client.19959 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f5688603740 10 client.19959 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f5688603740 20 client.19959 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f5688603740 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c516c7bba0 0x55c516c7dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f5688603740 4 client.19959 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.730+0000 7f5688603740 1 -- 172.21.15.18:0/1227345590 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c5168fd890 con 0x55c516c7bba0 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f567e7fc640 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c516c7bba0 0x55c516c7dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f567e7fc640 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c516c7bba0 0x55c516c7dfc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=65ea3e14537bde90 server_cookie=f2b5e2223b4ed092 in_seq=0 out_seq=0 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f565effd640 10 client.19959 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f565effd640 1 -- 172.21.15.18:0/1227345590 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c5168fd890 con 0x55c516c7bba0 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f565effd640 10 client.19959 handle_command_reply: tid=0 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f5688603740 1 client.19959 shutdown 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f5688603740 1 -- 172.21.15.18:0/1227345590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0ab70 msgr2=0x55c516c19b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f5688603740 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0ab70 0x55c516c19b10 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f567002c630 tx=0x7f5670000f40 comp rx=0 tx=0).stop 2024-02-17T13:02:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f5688603740 1 -- 172.21.15.18:0/1227345590 shutdown_connections 2024-02-17T13:02:14.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f5688603740 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c516c7bba0 0x55c516c7dfc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f5688603740 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c516c13750 0x55c516c7a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f5688603740 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c516c0ab70 0x55c516c19b10 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f5688603740 1 --2- 172.21.15.18:0/1227345590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c516c0a250 0x55c516c195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f5688603740 1 -- 172.21.15.18:0/1227345590 >> 172.21.15.18:0/1227345590 conn(0x55c516c04980 msgr2=0x55c516c0bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f565effd640 0 client.19959 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:02:14.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.731+0000 7f5688603740 1 -- 172.21.15.18:0/1227345590 shutdown_connections 2024-02-17T13:02:14.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f5688603740 1 -- 172.21.15.18:0/1227345590 wait complete. 2024-02-17T13:02:14.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f5688603740 20 client.19959 trim_cache size 0 max 16384 2024-02-17T13:02:14.727 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:02:14.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f568594f640 1 -- 172.21.15.18:0/696288362 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f566c0a2230 msgr2=0x7f566c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f568594f640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f566c0a2230 0x7f566c0a46f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f56780977c0 tx=0x7f56780049b0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f568594f640 1 -- 172.21.15.18:0/696288362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56801990e0 msgr2=0x7f56801a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f568594f640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56801990e0 0x7f56801a3690 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f567800e970 tx=0x7f567800ed70 comp rx=0 tx=0).stop 2024-02-17T13:02:14.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f568594f640 1 -- 172.21.15.18:0/696288362 shutdown_connections 2024-02-17T13:02:14.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f568594f640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f566c0a2230 0x7f566c0a46f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f568594f640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56801990e0 0x7f56801a3690 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f568594f640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5680105330 0x7f5680198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f568594f640 1 --2- 172.21.15.18:0/696288362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5680100260 0x7f5680198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f568594f640 1 -- 172.21.15.18:0/696288362 >> 172.21.15.18:0/696288362 conn(0x7f5680077180 msgr2=0x7f56800fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.732+0000 7f568594f640 1 -- 172.21.15.18:0/696288362 shutdown_connections 2024-02-17T13:02:14.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.733+0000 7f568594f640 1 -- 172.21.15.18:0/696288362 wait complete. 2024-02-17T13:02:14.736 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:02:14.736 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T13:02:14.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.857+0000 7f3733799640 1 -- 172.21.15.18:0/657847646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f372c100c10 msgr2=0x7f372c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.857+0000 7f3733799640 1 --2- 172.21.15.18:0/657847646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f372c100c10 0x7f372c104c50 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f371c009930 tx=0x7f371c02bc30 comp rx=0 tx=0).stop 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.857+0000 7f3733799640 1 -- 172.21.15.18:0/657847646 shutdown_connections 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.857+0000 7f3733799640 1 --2- 172.21.15.18:0/657847646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f372c105330 0x7f372c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.857+0000 7f3733799640 1 --2- 172.21.15.18:0/657847646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f372c100c10 0x7f372c104c50 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.857+0000 7f3733799640 1 --2- 172.21.15.18:0/657847646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f372c100260 0x7f372c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.857+0000 7f3733799640 1 -- 172.21.15.18:0/657847646 >> 172.21.15.18:0/657847646 conn(0x7f372c077180 msgr2=0x7f372c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.857+0000 7f3733799640 1 -- 172.21.15.18:0/657847646 shutdown_connections 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.857+0000 7f3733799640 1 -- 172.21.15.18:0/657847646 wait complete. 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.857+0000 7f3733799640 1 Processor -- start 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.857+0000 7f3733799640 1 -- start start 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3733799640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f372c100260 0x7f372c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3733799640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f372c105330 0x7f372c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3733799640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f372c197650 0x7f372c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3733799640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c198120 con 0x7f372c197650 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3733799640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c198290 con 0x7f372c105330 2024-02-17T13:02:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3733799640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c198450 con 0x7f372c100260 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3731d36640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f372c197650 0x7f372c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3731d36640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f372c197650 0x7f372c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39498/0 (socket says 172.21.15.18:39498) 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3731d36640 1 -- 172.21.15.18:0/3721691281 learned_addr learned my addr 172.21.15.18:0/3721691281 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3731535640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f372c100260 0x7f372c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3730d34640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f372c105330 0x7f372c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3731d36640 1 -- 172.21.15.18:0/3721691281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f372c100260 msgr2=0x7f372c19d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3731d36640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f372c100260 0x7f372c19d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3731d36640 1 -- 172.21.15.18:0/3721691281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f372c105330 msgr2=0x7f372c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3731d36640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f372c105330 0x7f372c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3731d36640 1 -- 172.21.15.18:0/3721691281 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c009560 con 0x7f372c197650 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3731d36640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f372c197650 0x7f372c197ad0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f372400e970 tx=0x7f372400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f37227fc640 1 -- 172.21.15.18:0/3721691281 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724009720 con 0x7f372c197650 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3733799640 1 -- 172.21.15.18:0/3721691281 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372c1a43f0 con 0x7f372c197650 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f37227fc640 1 -- 172.21.15.18:0/3721691281 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3724005500 con 0x7f372c197650 2024-02-17T13:02:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f37227fc640 1 -- 172.21.15.18:0/3721691281 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724010430 con 0x7f372c197650 2024-02-17T13:02:14.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.858+0000 7f3733799640 1 -- 172.21.15.18:0/3721691281 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372c1a48d0 con 0x7f372c197650 2024-02-17T13:02:14.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.861+0000 7f37227fc640 1 -- 172.21.15.18:0/3721691281 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3724005020 con 0x7f372c197650 2024-02-17T13:02:14.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.861+0000 7f3736368740 1 Processor -- start 2024-02-17T13:02:14.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.861+0000 7f3736368740 1 -- start start 2024-02-17T13:02:14.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.861+0000 7f3736368740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6d250 0x557a69f6d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.861+0000 7f3736368740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6db70 0x557a69f76210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.861+0000 7f3736368740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f76750 0x557a69f78b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.861+0000 7f3736368740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a69f28960 con 0x557a69f76750 2024-02-17T13:02:14.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.861+0000 7f3736368740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a69edc310 con 0x557a69f6d250 2024-02-17T13:02:14.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.861+0000 7f3736368740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a69ed9840 con 0x557a69f6db70 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f37227fc640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37180a2230 0x7f37180a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3730d34640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6d250 0x557a69f6d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3730d34640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6d250 0x557a69f6d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:58900/0 (socket says 172.21.15.18:58900) 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3730d34640 1 -- 172.21.15.18:0/2543218507 learned_addr learned my addr 172.21.15.18:0/2543218507 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3731535640 1 --2- 172.21.15.18:0/2543218507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6db70 0x557a69f76210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3731d36640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37180a2230 0x7f37180a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3730d34640 1 --2- 172.21.15.18:0/2543218507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f76750 0x557a69f78b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f37227fc640 1 -- 172.21.15.18:0/3721691281 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3724014070 con 0x7f372c197650 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3730d34640 1 -- 172.21.15.18:0/2543218507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6db70 msgr2=0x557a69f76210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3730d34640 1 --2- 172.21.15.18:0/2543218507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6db70 0x557a69f76210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3730d34640 1 -- 172.21.15.18:0/2543218507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f76750 msgr2=0x557a69f78b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3730d34640 1 --2- 172.21.15.18:0/2543218507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f76750 0x557a69f78b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3730d34640 1 -- 172.21.15.18:0/2543218507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a69e97060 con 0x557a69f6d250 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3731d36640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37180a2230 0x7f37180a46f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f37240cc0f0 tx=0x7f37240049b0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f3730d34640 1 --2- 172.21.15.18:0/2543218507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6d250 0x557a69f6d630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f371c005e00 tx=0x7f371c005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56a5efaf759f76b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f37177fe640 1 -- 172.21.15.18:0/2543218507 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c02c890 con 0x557a69f6d250 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f37177fe640 1 -- 172.21.15.18:0/2543218507 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f371c004500 con 0x557a69f6d250 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.862+0000 7f37177fe640 1 -- 172.21.15.18:0/2543218507 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c003bf0 con 0x557a69f6d250 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.863+0000 7f3736368740 1 -- 172.21.15.18:0/2543218507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6d250 msgr2=0x557a69f6d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.863+0000 7f3736368740 1 --2- 172.21.15.18:0/2543218507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6d250 0x557a69f6d630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f371c005e00 tx=0x7f371c005d90 comp rx=0 tx=0).stop 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.863+0000 7f3736368740 1 -- 172.21.15.18:0/2543218507 shutdown_connections 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.863+0000 7f3736368740 1 --2- 172.21.15.18:0/2543218507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f76750 0x557a69f78b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.863+0000 7f3736368740 1 --2- 172.21.15.18:0/2543218507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6db70 0x557a69f76210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.863+0000 7f3736368740 1 --2- 172.21.15.18:0/2543218507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6d250 0x557a69f6d630 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.863+0000 7f3736368740 1 -- 172.21.15.18:0/2543218507 >> 172.21.15.18:0/2543218507 conn(0x557a69f67980 msgr2=0x557a69f67d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.863+0000 7f3736368740 1 -- 172.21.15.18:0/2543218507 shutdown_connections 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.863+0000 7f3736368740 1 -- 172.21.15.18:0/2543218507 wait complete. 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3736368740 1 -- start start 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3736368740 1 Processor -- start 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3736368740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 0x557a69f73790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3736368740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 0x557a69f73cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3736368740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f76750 0x557a69f74210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3736368740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a69ed5020 con 0x557a69f6d250 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3736368740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a69ba8430 con 0x557a69f6db70 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3736368740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a69b64e00 con 0x557a69f76750 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3736368740 10 client.0 _ll_register_callbacks cb 0x557a69e7ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3730d34640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 0x557a69f73790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3731535640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 0x557a69f73cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3731535640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 0x557a69f73cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:58912/0 (socket says 172.21.15.18:58912) 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3731535640 1 -- 172.21.15.18:0/3246114466 learned_addr learned my addr 172.21.15.18:0/3246114466 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3730d34640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 0x557a69f73790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39522/0 (socket says 172.21.15.18:39522) 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3730d34640 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f76750 0x557a69f74210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3731535640 1 -- 172.21.15.18:0/3246114466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f76750 msgr2=0x557a69f74210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3731535640 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f76750 0x557a69f74210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3731535640 1 -- 172.21.15.18:0/3246114466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 msgr2=0x557a69f73790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3731535640 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 0x557a69f73790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3731535640 1 -- 172.21.15.18:0/3246114466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372c0682e0 con 0x557a69f6db70 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.864+0000 7f3730d34640 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 0x557a69f73790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3731535640 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 0x557a69f73cd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3728011d10 tx=0x7f372800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a8290baf9e0ddf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3714ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3714ff9640 1 -- 172.21.15.18:0/3246114466 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728016070 con 0x557a69f6db70 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3736368740 1 -- 172.21.15.18:0/3246114466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a69c11720 con 0x557a69f6db70 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3714ff9640 1 -- 172.21.15.18:0/3246114466 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3728004b20 con 0x557a69f6db70 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3714ff9640 1 -- 172.21.15.18:0/3246114466 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3728004ce0 con 0x557a69f6db70 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3736368740 10 client.28904 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3736368740 1 -- 172.21.15.18:0/3246114466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x557a69ed36a0 con 0x557a69f6db70 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3714ff9640 1 -- 172.21.15.18:0/3246114466 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557a69c11720 con 0x557a69f6db70 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3714ff9640 1 -- 172.21.15.18:0/3246114466 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f372801a3f0 con 0x557a69f6db70 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3736368740 10 client.28904 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3736368740 10 client.28904 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3736368740 20 client.28904 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3736368740 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x557a69f8afa0 0x557a69f8d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3736368740 4 client.28904 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.865+0000 7f3736368740 1 -- 172.21.15.18:0/3246114466 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a69c60890 con 0x557a69f8afa0 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.866+0000 7f3730d34640 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x557a69f8afa0 0x557a69f8d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.866+0000 7f3730d34640 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x557a69f8afa0 0x557a69f8d3c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ae259f9197a1272 server_cookie=56344eddaf7bc532 in_seq=0 out_seq=0 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.866+0000 7f3714ff9640 10 client.28904 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.866+0000 7f3714ff9640 1 -- 172.21.15.18:0/3246114466 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a69c60890 con 0x557a69f8afa0 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.866+0000 7f3714ff9640 10 client.28904 handle_command_reply: tid=0 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.866+0000 7f3736368740 1 client.28904 shutdown 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.867+0000 7f3736368740 1 -- 172.21.15.18:0/3246114466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 msgr2=0x557a69f73cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.867+0000 7f3736368740 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 0x557a69f73cd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3728011d10 tx=0x7f372800f790 comp rx=0 tx=0).stop 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.867+0000 7f3736368740 1 -- 172.21.15.18:0/3246114466 shutdown_connections 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.867+0000 7f3736368740 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x557a69f8afa0 0x557a69f8d3c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.867+0000 7f3736368740 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f76750 0x557a69f74210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.867+0000 7f3714ff9640 0 client.28904 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.867+0000 7f3736368740 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 0x557a69f73cd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.867+0000 7f3736368740 1 --2- 172.21.15.18:0/3246114466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 0x557a69f73790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.867+0000 7f3736368740 1 -- 172.21.15.18:0/3246114466 >> 172.21.15.18:0/3246114466 conn(0x557a69f67980 msgr2=0x557a69f696d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.867+0000 7f3736368740 1 -- 172.21.15.18:0/3246114466 shutdown_connections 2024-02-17T13:02:14.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.867+0000 7f3736368740 1 -- 172.21.15.18:0/3246114466 wait complete. 2024-02-17T13:02:14.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.867+0000 7f3736368740 20 client.28904 trim_cache size 0 max 16384 2024-02-17T13:02:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.879+0000 7f3736368740 1 Processor -- start 2024-02-17T13:02:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.879+0000 7f3736368740 1 -- start start 2024-02-17T13:02:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3736368740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6d250 0x557a69f75ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3736368740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 0x557a69f74030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3736368740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f76750 0x557a69f70a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3736368740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a69e5a130 con 0x557a69f76750 2024-02-17T13:02:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3736368740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a69f632b0 con 0x557a69f6db70 2024-02-17T13:02:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3736368740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a69f6ceb0 con 0x557a69f6d250 2024-02-17T13:02:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3730d34640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f76750 0x557a69f70a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3730d34640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f76750 0x557a69f70a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39530/0 (socket says 172.21.15.18:39530) 2024-02-17T13:02:14.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3731535640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 0x557a69f74030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3730d34640 1 -- 172.21.15.18:0/3548136243 learned_addr learned my addr 172.21.15.18:0/3548136243 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3730d34640 1 --2- 172.21.15.18:0/3548136243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6d250 0x557a69f75ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3730d34640 1 -- 172.21.15.18:0/3548136243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6d250 msgr2=0x557a69f75ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3730d34640 1 --2- 172.21.15.18:0/3548136243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6d250 0x557a69f75ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3730d34640 1 -- 172.21.15.18:0/3548136243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 msgr2=0x557a69f74030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3730d34640 1 --2- 172.21.15.18:0/3548136243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 0x557a69f74030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3730d34640 1 -- 172.21.15.18:0/3548136243 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a69e6fc20 con 0x557a69f76750 2024-02-17T13:02:14.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3731535640 1 --2- 172.21.15.18:0/3548136243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 0x557a69f74030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:14.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.880+0000 7f3730d34640 1 --2- 172.21.15.18:0/3548136243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f76750 0x557a69f70a10 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f371c0038c0 tx=0x7f371c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc39f9079333e294 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.883+0000 7f3716ffd640 1 -- 172.21.15.18:0/3548136243 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c05c070 con 0x557a69f76750 2024-02-17T13:02:14.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.883+0000 7f3716ffd640 1 -- 172.21.15.18:0/3548136243 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f371c05d040 con 0x557a69f76750 2024-02-17T13:02:14.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.883+0000 7f3716ffd640 1 -- 172.21.15.18:0/3548136243 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c003ba0 con 0x557a69f76750 2024-02-17T13:02:14.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.883+0000 7f3736368740 1 -- 172.21.15.18:0/3548136243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f76750 msgr2=0x557a69f70a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.883+0000 7f3736368740 1 --2- 172.21.15.18:0/3548136243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f76750 0x557a69f70a10 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f371c0038c0 tx=0x7f371c000bc0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.883+0000 7f3736368740 1 -- 172.21.15.18:0/3548136243 shutdown_connections 2024-02-17T13:02:14.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.883+0000 7f3736368740 1 --2- 172.21.15.18:0/3548136243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f76750 0x557a69f70a10 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.883+0000 7f3736368740 1 --2- 172.21.15.18:0/3548136243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f6db70 0x557a69f74030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.883+0000 7f3736368740 1 --2- 172.21.15.18:0/3548136243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6d250 0x557a69f75ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.883+0000 7f3736368740 1 -- 172.21.15.18:0/3548136243 >> 172.21.15.18:0/3548136243 conn(0x557a69f67980 msgr2=0x557a69f6ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.883+0000 7f3736368740 1 -- 172.21.15.18:0/3548136243 shutdown_connections 2024-02-17T13:02:14.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.883+0000 7f3736368740 1 -- 172.21.15.18:0/3548136243 wait complete. 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3736368740 1 -- start start 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3736368740 1 Processor -- start 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3736368740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 0x557a69f80680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3736368740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6db70 0x557a69f80bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3736368740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f76750 0x557a69f81100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3736368740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a69fe1640 con 0x557a69f6d250 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3736368740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a69fe17b0 con 0x557a69f76750 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3736368740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a69fe1970 con 0x557a69f6db70 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3736368740 10 client.0 _ll_register_callbacks cb 0x7f37280093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3730d34640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 0x557a69f80680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3730d34640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 0x557a69f80680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39534/0 (socket says 172.21.15.18:39534) 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3730d34640 1 -- 172.21.15.18:0/3853986805 learned_addr learned my addr 172.21.15.18:0/3853986805 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3731535640 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6db70 0x557a69f80bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.884+0000 7f3730d34640 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f76750 0x557a69f81100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3730d34640 1 -- 172.21.15.18:0/3853986805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6db70 msgr2=0x557a69f80bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3730d34640 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6db70 0x557a69f80bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3730d34640 1 -- 172.21.15.18:0/3853986805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f76750 msgr2=0x557a69f81100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3730d34640 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f76750 0x557a69f81100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3730d34640 1 -- 172.21.15.18:0/3853986805 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a69ed36a0 con 0x557a69f6d250 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3730d34640 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 0x557a69f80680 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f371c046040 tx=0x7f371c0056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3add8c49e2d52b12 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3714ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3714ff9640 1 -- 172.21.15.18:0/3853986805 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c05c040 con 0x557a69f6d250 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3714ff9640 1 -- 172.21.15.18:0/3853986805 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f371c004ff0 con 0x557a69f6d250 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3736368740 1 -- 172.21.15.18:0/3853986805 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a69bd47e0 con 0x557a69f6d250 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3714ff9640 1 -- 172.21.15.18:0/3853986805 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f371c02ccb0 con 0x557a69f6d250 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3736368740 10 client.19977 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3736368740 1 -- 172.21.15.18:0/3853986805 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x557a69e97060 con 0x557a69f6d250 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3714ff9640 1 -- 172.21.15.18:0/3853986805 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f371c070020 con 0x557a69f6d250 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3714ff9640 1 -- 172.21.15.18:0/3853986805 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f371c04b040 con 0x557a69f6d250 2024-02-17T13:02:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3736368740 10 client.19977 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3736368740 10 client.19977 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3736368740 20 client.19977 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3736368740 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x557a69fe2c90 0x557a69fe5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3736368740 4 client.19977 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.885+0000 7f3736368740 1 -- 172.21.15.18:0/3853986805 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557a69b80700 con 0x557a69fe2c90 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3731535640 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x557a69fe2c90 0x557a69fe5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3731535640 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x557a69fe2c90 0x557a69fe5080 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb980fa569f9fa23 server_cookie=31bf734259530f0d in_seq=0 out_seq=0 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3714ff9640 10 client.19977 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3714ff9640 1 -- 172.21.15.18:0/3853986805 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557a69b80700 con 0x557a69fe2c90 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3714ff9640 10 client.19977 handle_command_reply: tid=0 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3736368740 1 client.19977 shutdown 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3736368740 1 -- 172.21.15.18:0/3853986805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 msgr2=0x557a69f80680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3736368740 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 0x557a69f80680 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f371c046040 tx=0x7f371c0056f0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3736368740 1 -- 172.21.15.18:0/3853986805 shutdown_connections 2024-02-17T13:02:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3736368740 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x557a69fe2c90 0x557a69fe5080 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3736368740 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a69f76750 0x557a69f81100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3736368740 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a69f6db70 0x557a69f80bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3714ff9640 0 client.19977 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:02:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3736368740 1 --2- 172.21.15.18:0/3853986805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557a69f6d250 0x557a69f80680 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3736368740 1 -- 172.21.15.18:0/3853986805 >> 172.21.15.18:0/3853986805 conn(0x557a69f67980 msgr2=0x557a69f662f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.886+0000 7f3736368740 1 -- 172.21.15.18:0/3853986805 shutdown_connections 2024-02-17T13:02:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3736368740 1 -- 172.21.15.18:0/3853986805 wait complete. 2024-02-17T13:02:14.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3736368740 20 client.19977 trim_cache size 0 max 16384 2024-02-17T13:02:14.881 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:02:14.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3733799640 1 -- 172.21.15.18:0/3721691281 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37180a2230 msgr2=0x7f37180a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3733799640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37180a2230 0x7f37180a46f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f37240cc0f0 tx=0x7f37240049b0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3733799640 1 -- 172.21.15.18:0/3721691281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f372c197650 msgr2=0x7f372c197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3733799640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f372c197650 0x7f372c197ad0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f372400e970 tx=0x7f372400ed70 comp rx=0 tx=0).stop 2024-02-17T13:02:14.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3733799640 1 -- 172.21.15.18:0/3721691281 shutdown_connections 2024-02-17T13:02:14.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3733799640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f37180a2230 0x7f37180a46f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3733799640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f372c197650 0x7f372c197ad0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3733799640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f372c105330 0x7f372c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3733799640 1 --2- 172.21.15.18:0/3721691281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f372c100260 0x7f372c19d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3733799640 1 -- 172.21.15.18:0/3721691281 >> 172.21.15.18:0/3721691281 conn(0x7f372c077180 msgr2=0x7f372c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3733799640 1 -- 172.21.15.18:0/3721691281 shutdown_connections 2024-02-17T13:02:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:14.887+0000 7f3733799640 1 -- 172.21.15.18:0/3721691281 wait complete. 2024-02-17T13:02:14.887 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:02:14.887 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T13:02:14.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.005+0000 7fa27eeaa640 1 -- 172.21.15.18:0/3308840484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278100280 msgr2=0x7fa278100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.005+0000 7fa27eeaa640 1 --2- 172.21.15.18:0/3308840484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278100280 0x7fa278100660 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009930 tx=0x7fa26c02bc30 comp rx=0 tx=0).stop 2024-02-17T13:02:14.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.005+0000 7fa27eeaa640 1 -- 172.21.15.18:0/3308840484 shutdown_connections 2024-02-17T13:02:14.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.005+0000 7fa27eeaa640 1 --2- 172.21.15.18:0/3308840484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa278105350 0x7fa27810cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.005+0000 7fa27eeaa640 1 --2- 172.21.15.18:0/3308840484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa278100c30 0x7fa278104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.005+0000 7fa27eeaa640 1 --2- 172.21.15.18:0/3308840484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278100280 0x7fa278100660 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.005+0000 7fa27eeaa640 1 -- 172.21.15.18:0/3308840484 >> 172.21.15.18:0/3308840484 conn(0x7fa278077150 msgr2=0x7fa278077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.005+0000 7fa27eeaa640 1 -- 172.21.15.18:0/3308840484 shutdown_connections 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.005+0000 7fa27eeaa640 1 -- 172.21.15.18:0/3308840484 wait complete. 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa27eeaa640 1 Processor -- start 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa27eeaa640 1 -- start start 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa27eeaa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa278100280 0x7fa27819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa27eeaa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa278105350 0x7fa27819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa27eeaa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278197670 0x7fa278197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa27eeaa640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa278198140 con 0x7fa278197670 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa27eeaa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2781982b0 con 0x7fa278105350 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa27eeaa640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa278198470 con 0x7fa278100280 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa277fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa278105350 0x7fa27819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa27d447640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278197670 0x7fa278197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa277fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa278105350 0x7fa27819daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:58936/0 (socket says 172.21.15.18:58936) 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa277fff640 1 -- 172.21.15.18:0/1059952922 learned_addr learned my addr 172.21.15.18:0/1059952922 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.006+0000 7fa27cc46640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa278100280 0x7fa27819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa277fff640 1 -- 172.21.15.18:0/1059952922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa278100280 msgr2=0x7fa27819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa277fff640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa278100280 0x7fa27819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa277fff640 1 -- 172.21.15.18:0/1059952922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278197670 msgr2=0x7fa278197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa277fff640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278197670 0x7fa278197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa277fff640 1 -- 172.21.15.18:0/1059952922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c009560 con 0x7fa278105350 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa27cc46640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa278100280 0x7fa27819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa27d447640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278197670 0x7fa278197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa277fff640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa278105350 0x7fa27819daf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa26400e9f0 tx=0x7fa26400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa275ffb640 1 -- 172.21.15.18:0/1059952922 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2640096c0 con 0x7fa278105350 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa275ffb640 1 -- 172.21.15.18:0/1059952922 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa264005500 con 0x7fa278105350 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa27eeaa640 1 -- 172.21.15.18:0/1059952922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2781a4410 con 0x7fa278105350 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa275ffb640 1 -- 172.21.15.18:0/1059952922 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa264010430 con 0x7fa278105350 2024-02-17T13:02:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.007+0000 7fa27eeaa640 1 -- 172.21.15.18:0/1059952922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2781a48a0 con 0x7fa278105350 2024-02-17T13:02:15.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.009+0000 7fa275ffb640 1 -- 172.21.15.18:0/1059952922 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa264005020 con 0x7fa278105350 2024-02-17T13:02:15.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa281a7d740 1 Processor -- start 2024-02-17T13:02:15.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa281a7d740 1 -- start start 2024-02-17T13:02:15.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa281a7d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 0x55a412481630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa281a7d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 0x55a41248a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa281a7d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a41248a750 0x55a41248cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa281a7d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41243c960 con 0x55a412481250 2024-02-17T13:02:15.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa281a7d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4123f0310 con 0x55a41248a750 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa281a7d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4123ed840 con 0x55a412481b70 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa275ffb640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa2680a2230 0x7fa2680a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa27d447640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 0x55a412481630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa27cc46640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 0x55a41248a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa27d447640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 0x55a412481630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39552/0 (socket says 172.21.15.18:39552) 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa27d447640 1 -- 172.21.15.18:0/3182598804 learned_addr learned my addr 172.21.15.18:0/3182598804 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa277fff640 1 --2- 172.21.15.18:0/3182598804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a41248a750 0x55a41248cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa27d447640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa2680a2230 0x7fa2680a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa275ffb640 1 -- 172.21.15.18:0/1059952922 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa264014070 con 0x7fa278105350 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa27cc46640 1 -- 172.21.15.18:0/3182598804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a41248a750 msgr2=0x55a41248cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa27cc46640 1 --2- 172.21.15.18:0/3182598804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a41248a750 0x55a41248cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa27cc46640 1 -- 172.21.15.18:0/3182598804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 msgr2=0x55a412481630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa27cc46640 1 --2- 172.21.15.18:0/3182598804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 0x55a412481630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.010+0000 7fa27cc46640 1 -- 172.21.15.18:0/3182598804 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4123ab060 con 0x55a412481b70 2024-02-17T13:02:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.011+0000 7fa27d447640 1 --2- 172.21.15.18:0/3182598804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 0x55a412481630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.011+0000 7fa277fff640 1 --2- 172.21.15.18:0/3182598804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a41248a750 0x55a41248cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.011+0000 7fa27d447640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa2680a2230 0x7fa2680a46f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa2700099e0 tx=0x7fa270009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.011+0000 7fa27cc46640 1 --2- 172.21.15.18:0/3182598804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 0x55a41248a210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa26c036040 tx=0x7fa26c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc8c57be517badc1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.011+0000 7fa262ffd640 1 -- 172.21.15.18:0/3182598804 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c02c7e0 con 0x55a412481b70 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.011+0000 7fa262ffd640 1 -- 172.21.15.18:0/3182598804 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa26c02c940 con 0x55a412481b70 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.011+0000 7fa262ffd640 1 -- 172.21.15.18:0/3182598804 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c005300 con 0x55a412481b70 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.012+0000 7fa281a7d740 1 -- 172.21.15.18:0/3182598804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 msgr2=0x55a41248a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.012+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3182598804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 0x55a41248a210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa26c036040 tx=0x7fa26c02c650 comp rx=0 tx=0).stop 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.012+0000 7fa281a7d740 1 -- 172.21.15.18:0/3182598804 shutdown_connections 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.012+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3182598804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a41248a750 0x55a41248cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.012+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3182598804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 0x55a41248a210 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.012+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3182598804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 0x55a412481630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.012+0000 7fa281a7d740 1 -- 172.21.15.18:0/3182598804 >> 172.21.15.18:0/3182598804 conn(0x55a41247b980 msgr2=0x55a41247bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:15.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.012+0000 7fa281a7d740 1 -- 172.21.15.18:0/3182598804 shutdown_connections 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.012+0000 7fa281a7d740 1 -- 172.21.15.18:0/3182598804 wait complete. 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.012+0000 7fa281a7d740 1 -- start start 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.012+0000 7fa281a7d740 1 Processor -- start 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa281a7d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481250 0x55a412487770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa281a7d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 0x55a412487cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa281a7d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a41248a750 0x55a4124881f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa281a7d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4123e9020 con 0x55a41248a750 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa281a7d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4120bc430 con 0x55a412481250 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa281a7d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a412078e00 con 0x55a412481b70 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa281a7d740 10 client.0 _ll_register_callbacks cb 0x55a412390cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa27d447640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a41248a750 0x55a4124881f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa27cc46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481250 0x55a412487770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa277fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 0x55a412487cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa27cc46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481250 0x55a412487770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:58958/0 (socket says 172.21.15.18:58958) 2024-02-17T13:02:15.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa27cc46640 1 -- 172.21.15.18:0/1388742324 learned_addr learned my addr 172.21.15.18:0/1388742324 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa277fff640 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 0x55a412487cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34096/0 (socket says 172.21.15.18:34096) 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa27cc46640 1 -- 172.21.15.18:0/1388742324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 msgr2=0x55a412487cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa27cc46640 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 0x55a412487cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa27cc46640 1 -- 172.21.15.18:0/1388742324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a41248a750 msgr2=0x55a4124881f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa27cc46640 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a41248a750 0x55a4124881f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa27cc46640 1 -- 172.21.15.18:0/1388742324 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa278069790 con 0x55a412481250 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa277fff640 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 0x55a412487cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa27d447640 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a41248a750 0x55a4124881f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa27cc46640 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481250 0x55a412487770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa26c02c240 tx=0x7fa26c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b2a3cbe7774f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa257fff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa257fff640 1 -- 172.21.15.18:0/1388742324 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c02c7e0 con 0x55a412481250 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa257fff640 1 -- 172.21.15.18:0/1388742324 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa26c004df0 con 0x55a412481250 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.013+0000 7fa281a7d740 1 -- 172.21.15.18:0/1388742324 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4120e87e0 con 0x55a412481250 2024-02-17T13:02:15.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.014+0000 7fa257fff640 1 -- 172.21.15.18:0/1388742324 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa26c005670 con 0x55a412481250 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.014+0000 7fa281a7d740 10 client.28922 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.014+0000 7fa281a7d740 1 -- 172.21.15.18:0/1388742324 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a4123e76a0 con 0x55a412481250 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.014+0000 7fa257fff640 1 -- 172.21.15.18:0/1388742324 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa26c05b020 con 0x55a412481250 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.014+0000 7fa257fff640 1 -- 172.21.15.18:0/1388742324 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa26c055070 con 0x55a412481250 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.014+0000 7fa281a7d740 10 client.28922 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.014+0000 7fa281a7d740 10 client.28922 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.014+0000 7fa281a7d740 20 client.28922 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.014+0000 7fa281a7d740 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55a41249a740 0x55a41249cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.014+0000 7fa281a7d740 4 client.28922 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.014+0000 7fa281a7d740 1 -- 172.21.15.18:0/1388742324 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a412125720 con 0x55a41249a740 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.014+0000 7fa277fff640 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55a41249a740 0x55a41249cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa277fff640 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55a41249a740 0x55a41249cb30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=583c183141c1b3ea server_cookie=eaaf041fe5b7541d in_seq=0 out_seq=0 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa257fff640 10 client.28922 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa257fff640 1 -- 172.21.15.18:0/1388742324 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a412125720 con 0x55a41249a740 2024-02-17T13:02:15.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa257fff640 10 client.28922 handle_command_reply: tid=0 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa281a7d740 1 client.28922 shutdown 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa281a7d740 1 -- 172.21.15.18:0/1388742324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481250 msgr2=0x55a412487770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa281a7d740 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481250 0x55a412487770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa26c02c240 tx=0x7fa26c005270 comp rx=0 tx=0).stop 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa281a7d740 1 -- 172.21.15.18:0/1388742324 shutdown_connections 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa281a7d740 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55a41249a740 0x55a41249cb30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa281a7d740 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a41248a750 0x55a4124881f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa281a7d740 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a412481b70 0x55a412487cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa281a7d740 1 --2- 172.21.15.18:0/1388742324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481250 0x55a412487770 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa257fff640 0 client.28922 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.015+0000 7fa281a7d740 1 -- 172.21.15.18:0/1388742324 >> 172.21.15.18:0/1388742324 conn(0x55a41247b980 msgr2=0x55a41247d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.016+0000 7fa281a7d740 1 -- 172.21.15.18:0/1388742324 shutdown_connections 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.016+0000 7fa281a7d740 1 -- 172.21.15.18:0/1388742324 wait complete. 2024-02-17T13:02:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.016+0000 7fa281a7d740 20 client.28922 trim_cache size 0 max 16384 2024-02-17T13:02:15.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.030+0000 7fa281a7d740 1 Processor -- start 2024-02-17T13:02:15.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.030+0000 7fa281a7d740 1 -- start start 2024-02-17T13:02:15.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa281a7d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481250 0x55a412489ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa281a7d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481b70 0x55a412488010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa281a7d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a41248a750 0x55a4124849d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa281a7d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41236e130 con 0x55a412481b70 2024-02-17T13:02:15.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa281a7d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a412480ec0 con 0x55a412481250 2024-02-17T13:02:15.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa281a7d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41247c980 con 0x55a41248a750 2024-02-17T13:02:15.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa277fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481b70 0x55a412488010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa277fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481b70 0x55a412488010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39564/0 (socket says 172.21.15.18:39564) 2024-02-17T13:02:15.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa277fff640 1 -- 172.21.15.18:0/3943926558 learned_addr learned my addr 172.21.15.18:0/3943926558 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa27d447640 1 --2- 172.21.15.18:0/3943926558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a41248a750 0x55a4124849d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa27cc46640 1 --2- 172.21.15.18:0/3943926558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481250 0x55a412489ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa277fff640 1 -- 172.21.15.18:0/3943926558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a41248a750 msgr2=0x55a4124849d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa277fff640 1 --2- 172.21.15.18:0/3943926558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a41248a750 0x55a4124849d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa277fff640 1 -- 172.21.15.18:0/3943926558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481250 msgr2=0x55a412489ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa277fff640 1 --2- 172.21.15.18:0/3943926558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481250 0x55a412489ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa277fff640 1 -- 172.21.15.18:0/3943926558 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a412383c20 con 0x55a412481b70 2024-02-17T13:02:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa277fff640 1 --2- 172.21.15.18:0/3943926558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481b70 0x55a412488010 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fa26400c6a0 tx=0x7fa26406ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2215a227ec3bbc2d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa2627fc640 1 -- 172.21.15.18:0/3943926558 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa264005670 con 0x55a412481b70 2024-02-17T13:02:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa2627fc640 1 -- 172.21.15.18:0/3943926558 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa26409ae20 con 0x55a412481b70 2024-02-17T13:02:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.031+0000 7fa2627fc640 1 -- 172.21.15.18:0/3943926558 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26409a4b0 con 0x55a412481b70 2024-02-17T13:02:15.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:14 smithi018 ceph-mon[32625]: osdmap e104: 12 total, 12 up, 12 in 2024-02-17T13:02:15.024 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:14 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3376631734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.032+0000 7fa281a7d740 1 -- 172.21.15.18:0/3943926558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481b70 msgr2=0x55a412488010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.032+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3943926558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481b70 0x55a412488010 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fa26400c6a0 tx=0x7fa26406ac80 comp rx=0 tx=0).stop 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.032+0000 7fa281a7d740 1 -- 172.21.15.18:0/3943926558 shutdown_connections 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.032+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3943926558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a41248a750 0x55a4124849d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.032+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3943926558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481b70 0x55a412488010 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.032+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3943926558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481250 0x55a412489ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.032+0000 7fa281a7d740 1 -- 172.21.15.18:0/3943926558 >> 172.21.15.18:0/3943926558 conn(0x55a41247b980 msgr2=0x55a412482a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.032+0000 7fa281a7d740 1 -- 172.21.15.18:0/3943926558 shutdown_connections 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.032+0000 7fa281a7d740 1 -- 172.21.15.18:0/3943926558 wait complete. 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa281a7d740 1 -- start start 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa281a7d740 1 Processor -- start 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa281a7d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 0x55a4124900e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa281a7d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481b70 0x55a412490620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa281a7d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a41248a750 0x55a412490b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa281a7d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4124f0c30 con 0x55a412481250 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa281a7d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4124f0d70 con 0x55a412481b70 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa281a7d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4124f0f30 con 0x55a41248a750 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa281a7d740 10 client.0 _ll_register_callbacks cb 0x7fa264005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa27cc46640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 0x55a4124900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa27cc46640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 0x55a4124900e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39570/0 (socket says 172.21.15.18:39570) 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa27cc46640 1 -- 172.21.15.18:0/3325844677 learned_addr learned my addr 172.21.15.18:0/3325844677 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa277fff640 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481b70 0x55a412490620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa27d447640 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a41248a750 0x55a412490b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa27cc46640 1 -- 172.21.15.18:0/3325844677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a41248a750 msgr2=0x55a412490b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa27cc46640 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a41248a750 0x55a412490b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa27cc46640 1 -- 172.21.15.18:0/3325844677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481b70 msgr2=0x55a412490620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa27cc46640 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481b70 0x55a412490620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.033+0000 7fa27cc46640 1 -- 172.21.15.18:0/3325844677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2781a4410 con 0x55a412481250 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa277fff640 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481b70 0x55a412490620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa27d447640 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a41248a750 0x55a412490b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa27cc46640 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 0x55a4124900e0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fa26c02c210 tx=0x7fa26c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3745e7e6bfe0f9c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa257fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa257fff640 1 -- 172.21.15.18:0/3325844677 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c004450 con 0x55a412481250 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa257fff640 1 -- 172.21.15.18:0/3325844677 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa26c05c080 con 0x55a412481250 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa281a7d740 1 -- 172.21.15.18:0/3325844677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa26c02c940 con 0x55a412481250 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa257fff640 1 -- 172.21.15.18:0/3325844677 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa26c0052e0 con 0x55a412481250 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa281a7d740 10 client.19995 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa281a7d740 1 -- 172.21.15.18:0/3325844677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a4123a98c0 con 0x55a412481250 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa257fff640 1 -- 172.21.15.18:0/3325844677 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa26c02c940 con 0x55a412481250 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa257fff640 1 -- 172.21.15.18:0/3325844677 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa26c055040 con 0x55a412481250 2024-02-17T13:02:15.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa281a7d740 10 client.19995 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa281a7d740 10 client.19995 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa281a7d740 20 client.19995 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55a4124f23b0 0x55a4124f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa281a7d740 4 client.19995 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.034+0000 7fa281a7d740 1 -- 172.21.15.18:0/3325844677 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a412174890 con 0x55a4124f23b0 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa277fff640 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55a4124f23b0 0x55a4124f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa257fff640 10 client.19995 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa277fff640 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55a4124f23b0 0x55a4124f47d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f66bb5e0595a1cf server_cookie=3d463c0627056f88 in_seq=0 out_seq=0 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa257fff640 1 -- 172.21.15.18:0/3325844677 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a412174890 con 0x55a4124f23b0 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa257fff640 10 client.19995 handle_command_reply: tid=0 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa281a7d740 1 client.19995 shutdown 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa281a7d740 1 -- 172.21.15.18:0/3325844677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 msgr2=0x55a4124900e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 0x55a4124900e0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fa26c02c210 tx=0x7fa26c02c7e0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa281a7d740 1 -- 172.21.15.18:0/3325844677 shutdown_connections 2024-02-17T13:02:15.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x55a4124f23b0 0x55a4124f47d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a41248a750 0x55a412490b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa257fff640 0 client.19995 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:02:15.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a412481b70 0x55a412490620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa281a7d740 1 --2- 172.21.15.18:0/3325844677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a412481250 0x55a4124900e0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa281a7d740 1 -- 172.21.15.18:0/3325844677 >> 172.21.15.18:0/3325844677 conn(0x55a41247b980 msgr2=0x55a41247a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:15.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa281a7d740 1 -- 172.21.15.18:0/3325844677 shutdown_connections 2024-02-17T13:02:15.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.035+0000 7fa281a7d740 1 -- 172.21.15.18:0/3325844677 wait complete. 2024-02-17T13:02:15.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa281a7d740 20 client.19995 trim_cache size 0 max 16384 2024-02-17T13:02:15.033 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:02:15.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa27eeaa640 1 -- 172.21.15.18:0/1059952922 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa2680a2230 msgr2=0x7fa2680a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa27eeaa640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa2680a2230 0x7fa2680a46f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa2700099e0 tx=0x7fa270009500 comp rx=0 tx=0).stop 2024-02-17T13:02:15.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa27eeaa640 1 -- 172.21.15.18:0/1059952922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa278105350 msgr2=0x7fa27819daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa27eeaa640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa278105350 0x7fa27819daf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa26400e9f0 tx=0x7fa26400eeb0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa27eeaa640 1 -- 172.21.15.18:0/1059952922 shutdown_connections 2024-02-17T13:02:15.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa27eeaa640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa2680a2230 0x7fa2680a46f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa27eeaa640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278197670 0x7fa278197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa27eeaa640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa278105350 0x7fa27819daf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa27eeaa640 1 --2- 172.21.15.18:0/1059952922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa278100280 0x7fa27819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa27eeaa640 1 -- 172.21.15.18:0/1059952922 >> 172.21.15.18:0/1059952922 conn(0x7fa278077150 msgr2=0x7fa2780fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:15.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa27eeaa640 1 -- 172.21.15.18:0/1059952922 shutdown_connections 2024-02-17T13:02:15.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.036+0000 7fa27eeaa640 1 -- 172.21.15.18:0/1059952922 wait complete. 2024-02-17T13:02:15.036 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:02:15.036 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T13:02:15.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.152+0000 7f8bb356b640 1 -- 172.21.15.18:0/220280051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bac0685f0 msgr2=0x7f8bac0689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.152+0000 7f8bb356b640 1 --2- 172.21.15.18:0/220280051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bac0685f0 0x7f8bac0689d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009890 tx=0x7f8ba802bb70 comp rx=0 tx=0).stop 2024-02-17T13:02:15.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.152+0000 7f8bb356b640 1 -- 172.21.15.18:0/220280051 shutdown_connections 2024-02-17T13:02:15.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.152+0000 7f8bb356b640 1 --2- 172.21.15.18:0/220280051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac1064c0 0x7f8bac10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.152+0000 7f8bb356b640 1 --2- 172.21.15.18:0/220280051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bac068fa0 0x7f8bac105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.152+0000 7f8bb356b640 1 --2- 172.21.15.18:0/220280051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bac0685f0 0x7f8bac0689d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.152+0000 7f8bb356b640 1 -- 172.21.15.18:0/220280051 >> 172.21.15.18:0/220280051 conn(0x7f8bac077180 msgr2=0x7f8bac077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:15.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.153+0000 7f8bb356b640 1 -- 172.21.15.18:0/220280051 shutdown_connections 2024-02-17T13:02:15.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.153+0000 7f8bb356b640 1 -- 172.21.15.18:0/220280051 wait complete. 2024-02-17T13:02:15.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.153+0000 7f8bb356b640 1 Processor -- start 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.153+0000 7f8bb356b640 1 -- start start 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.153+0000 7f8bb356b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bac0685f0 0x7f8bac100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.153+0000 7f8bb356b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bac1064c0 0x7f8bac100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.153+0000 7f8bb356b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac100ac0 0x7f8bac1a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.153+0000 7f8bb356b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac1a5f80 con 0x7f8bac100ac0 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.153+0000 7f8bb356b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac1a60f0 con 0x7f8bac1064c0 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.153+0000 7f8bb356b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac1a62b0 con 0x7f8bac0685f0 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb0b06640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bac1064c0 0x7f8bac100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb1b08640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac100ac0 0x7f8bac1a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb1307640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bac0685f0 0x7f8bac100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb0b06640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bac1064c0 0x7f8bac100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:59018/0 (socket says 172.21.15.18:59018) 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb0b06640 1 -- 172.21.15.18:0/4081527549 learned_addr learned my addr 172.21.15.18:0/4081527549 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb0b06640 1 -- 172.21.15.18:0/4081527549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bac0685f0 msgr2=0x7f8bac100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb0b06640 1 --2- 172.21.15.18:0/4081527549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bac0685f0 0x7f8bac100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb0b06640 1 -- 172.21.15.18:0/4081527549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac100ac0 msgr2=0x7f8bac1a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb0b06640 1 --2- 172.21.15.18:0/4081527549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac100ac0 0x7f8bac1a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb0b06640 1 -- 172.21.15.18:0/4081527549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba4009900 con 0x7f8bac1064c0 2024-02-17T13:02:15.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb0b06640 1 --2- 172.21.15.18:0/4081527549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bac1064c0 0x7f8bac100580 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00e9c0 tx=0x7f8b9c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:15.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8ba27fc640 1 -- 172.21.15.18:0/4081527549 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0096c0 con 0x7f8bac1064c0 2024-02-17T13:02:15.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb356b640 1 -- 172.21.15.18:0/4081527549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba8009490 con 0x7f8bac1064c0 2024-02-17T13:02:15.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8ba27fc640 1 -- 172.21.15.18:0/4081527549 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b9c005500 con 0x7f8bac1064c0 2024-02-17T13:02:15.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8ba27fc640 1 -- 172.21.15.18:0/4081527549 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c010430 con 0x7f8bac1064c0 2024-02-17T13:02:15.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.154+0000 7f8bb356b640 1 -- 172.21.15.18:0/4081527549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac1a68c0 con 0x7f8bac1064c0 2024-02-17T13:02:15.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.156+0000 7f8ba27fc640 1 -- 172.21.15.18:0/4081527549 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8b9c005020 con 0x7f8bac1064c0 2024-02-17T13:02:15.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb620e740 1 Processor -- start 2024-02-17T13:02:15.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb620e740 1 -- start start 2024-02-17T13:02:15.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb620e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca17630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb620e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 0x55e33ca20210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb620e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca20750 0x55e33ca22b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb620e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33c9d2960 con 0x55e33ca17b70 2024-02-17T13:02:15.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb620e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33c986310 con 0x55e33ca20750 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb620e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33c983840 con 0x55e33ca17250 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8ba27fc640 1 --2- 172.21.15.18:0/4081527549 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8b980a2260 0x7f8b980a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb1307640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 0x55e33ca20210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb1b08640 1 --2- 172.21.15.18:0/4081527549 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8b980a2260 0x7f8b980a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb0b06640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca20750 0x55e33ca22b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb1307640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 0x55e33ca20210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39602/0 (socket says 172.21.15.18:39602) 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb1307640 1 -- 172.21.15.18:0/216409766 learned_addr learned my addr 172.21.15.18:0/216409766 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.157+0000 7f8bb1b08640 1 --2- 172.21.15.18:0/216409766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca17630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.158+0000 7f8ba27fc640 1 -- 172.21.15.18:0/4081527549 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8b9c014070 con 0x7f8bac1064c0 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.158+0000 7f8bb1307640 1 -- 172.21.15.18:0/216409766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 msgr2=0x55e33ca17630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.158+0000 7f8bb1307640 1 --2- 172.21.15.18:0/216409766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca17630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.158+0000 7f8bb1307640 1 -- 172.21.15.18:0/216409766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca20750 msgr2=0x55e33ca22b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.158+0000 7f8bb1307640 1 --2- 172.21.15.18:0/216409766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca20750 0x55e33ca22b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.158+0000 7f8bb1307640 1 -- 172.21.15.18:0/216409766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e33c941060 con 0x55e33ca17b70 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.158+0000 7f8bb1b08640 1 --2- 172.21.15.18:0/4081527549 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8b980a2260 0x7f8b980a4720 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4005fd0 tx=0x7f8ba4009f20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.158+0000 7f8bb1307640 1 --2- 172.21.15.18:0/216409766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 0x55e33ca20210 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009860 tx=0x7f8ba80323c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cdbe25cbc8cd2a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.158+0000 7f8b977fe640 1 -- 172.21.15.18:0/216409766 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8004030 con 0x55e33ca17b70 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.158+0000 7f8b977fe640 1 -- 172.21.15.18:0/216409766 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ba8034440 con 0x55e33ca17b70 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.158+0000 7f8b977fe640 1 -- 172.21.15.18:0/216409766 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8005590 con 0x55e33ca17b70 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.159+0000 7f8bb620e740 1 -- 172.21.15.18:0/216409766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 msgr2=0x55e33ca20210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.159+0000 7f8bb620e740 1 --2- 172.21.15.18:0/216409766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 0x55e33ca20210 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009860 tx=0x7f8ba80323c0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.159+0000 7f8bb620e740 1 -- 172.21.15.18:0/216409766 shutdown_connections 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.159+0000 7f8bb620e740 1 --2- 172.21.15.18:0/216409766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca20750 0x55e33ca22b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.159+0000 7f8bb620e740 1 --2- 172.21.15.18:0/216409766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 0x55e33ca20210 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.159+0000 7f8bb620e740 1 --2- 172.21.15.18:0/216409766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca17630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.159+0000 7f8bb620e740 1 -- 172.21.15.18:0/216409766 >> 172.21.15.18:0/216409766 conn(0x55e33ca11980 msgr2=0x55e33ca11d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.159+0000 7f8bb620e740 1 -- 172.21.15.18:0/216409766 shutdown_connections 2024-02-17T13:02:15.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.159+0000 7f8bb620e740 1 -- 172.21.15.18:0/216409766 wait complete. 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.159+0000 7f8bb620e740 1 -- start start 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.159+0000 7f8bb620e740 1 Processor -- start 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb620e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca17250 0x55e33ca1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb620e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17b70 0x55e33ca1dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb620e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 0x55e33ca1e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb620e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33c97f020 con 0x55e33ca20750 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb620e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33c652430 con 0x55e33ca17250 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb620e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33c60ee00 con 0x55e33ca17b70 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb620e740 10 client.0 _ll_register_callbacks cb 0x55e33c926cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb1b08640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 0x55e33ca1e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb1307640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca17250 0x55e33ca1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb1b08640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 0x55e33ca1e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39608/0 (socket says 172.21.15.18:39608) 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb1b08640 1 -- 172.21.15.18:0/3121090682 learned_addr learned my addr 172.21.15.18:0/3121090682 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:15.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb0b06640 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17b70 0x55e33ca1dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb1b08640 1 -- 172.21.15.18:0/3121090682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17b70 msgr2=0x55e33ca1dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb1b08640 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17b70 0x55e33ca1dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb1b08640 1 -- 172.21.15.18:0/3121090682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca17250 msgr2=0x55e33ca1d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb1b08640 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca17250 0x55e33ca1d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb1b08640 1 -- 172.21.15.18:0/3121090682 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac10e880 con 0x55e33ca20750 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb1307640 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca17250 0x55e33ca1d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8bb1b08640 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 0x55e33ca1e1e0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4004260 tx=0x7f8ba40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98c0bc3ac8432b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8b94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.160+0000 7f8b94ff9640 1 -- 172.21.15.18:0/3121090682 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4013040 con 0x55e33ca20750 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8b94ff9640 1 -- 172.21.15.18:0/3121090682 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8ba4025070 con 0x55e33ca20750 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8bb620e740 1 -- 172.21.15.18:0/3121090682 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e33c67e7e0 con 0x55e33ca20750 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8b94ff9640 1 -- 172.21.15.18:0/3121090682 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ba4004720 con 0x55e33ca20750 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8bb620e740 10 client.20007 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8bb620e740 1 -- 172.21.15.18:0/3121090682 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55e33c97d6a0 con 0x55e33ca20750 2024-02-17T13:02:15.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8b94ff9640 1 -- 172.21.15.18:0/3121090682 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e33c67e7e0 con 0x55e33ca20750 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8b94ff9640 1 -- 172.21.15.18:0/3121090682 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ba400ed20 con 0x55e33ca20750 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8bb620e740 10 client.20007 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8bb620e740 10 client.20007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8bb620e740 20 client.20007 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8bb620e740 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e33ca30a50 0x55e33ca32e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8bb620e740 4 client.20007 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8bb620e740 1 -- 172.21.15.18:0/3121090682 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e33c6bb720 con 0x55e33ca30a50 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.161+0000 7f8bb1307640 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e33ca30a50 0x55e33ca32e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8bb1307640 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e33ca30a50 0x55e33ca32e70 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9439e0fc76dea213 server_cookie=7a03c690b3b55e51 in_seq=0 out_seq=0 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8b94ff9640 10 client.20007 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8b94ff9640 1 -- 172.21.15.18:0/3121090682 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e33c6bb720 con 0x55e33ca30a50 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8b94ff9640 10 client.20007 handle_command_reply: tid=0 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8bb620e740 1 client.20007 shutdown 2024-02-17T13:02:15.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8bb620e740 1 -- 172.21.15.18:0/3121090682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 msgr2=0x55e33ca1e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8bb620e740 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 0x55e33ca1e1e0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4004260 tx=0x7f8ba40095c0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8bb620e740 1 -- 172.21.15.18:0/3121090682 shutdown_connections 2024-02-17T13:02:15.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8bb620e740 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e33ca30a50 0x55e33ca32e70 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8bb620e740 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 0x55e33ca1e1e0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8bb620e740 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17b70 0x55e33ca1dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8b94ff9640 0 client.20007 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:15.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8bb620e740 1 --2- 172.21.15.18:0/3121090682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca17250 0x55e33ca1d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.162+0000 7f8bb620e740 1 -- 172.21.15.18:0/3121090682 >> 172.21.15.18:0/3121090682 conn(0x55e33ca11980 msgr2=0x55e33ca136d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:15.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.163+0000 7f8bb620e740 1 -- 172.21.15.18:0/3121090682 shutdown_connections 2024-02-17T13:02:15.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.163+0000 7f8bb620e740 1 -- 172.21.15.18:0/3121090682 wait complete. 2024-02-17T13:02:15.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.163+0000 7f8bb620e740 20 client.20007 trim_cache size 0 max 16384 2024-02-17T13:02:15.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.174+0000 7f8bb620e740 1 Processor -- start 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.174+0000 7f8bb620e740 1 -- start start 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.174+0000 7f8bb620e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca1e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.174+0000 7f8bb620e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 0x55e33ca10a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.174+0000 7f8bb620e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca20750 0x55e33ca27480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.174+0000 7f8bb620e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33c904130 con 0x55e33ca17b70 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.174+0000 7f8bb620e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33ca03a40 con 0x55e33ca20750 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.174+0000 7f8bb620e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33ca12780 con 0x55e33ca17250 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb0b06640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 0x55e33ca10a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb0b06640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 0x55e33ca10a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39620/0 (socket says 172.21.15.18:39620) 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb0b06640 1 -- 172.21.15.18:0/1801511288 learned_addr learned my addr 172.21.15.18:0/1801511288 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb1b08640 1 --2- 172.21.15.18:0/1801511288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca20750 0x55e33ca27480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb1307640 1 --2- 172.21.15.18:0/1801511288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca1e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb0b06640 1 -- 172.21.15.18:0/1801511288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 msgr2=0x55e33ca1e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb0b06640 1 --2- 172.21.15.18:0/1801511288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca1e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb0b06640 1 -- 172.21.15.18:0/1801511288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca20750 msgr2=0x55e33ca27480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb0b06640 1 --2- 172.21.15.18:0/1801511288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca20750 0x55e33ca27480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb0b06640 1 -- 172.21.15.18:0/1801511288 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e33c919c20 con 0x55e33ca17b70 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb1b08640 1 --2- 172.21.15.18:0/1801511288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca20750 0x55e33ca27480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb1307640 1 --2- 172.21.15.18:0/1801511288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca1e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8bb0b06640 1 --2- 172.21.15.18:0/1801511288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 0x55e33ca10a00 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c005780 tx=0x7f8b9c065e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e890d541588e5f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8b96ffd640 1 -- 172.21.15.18:0/1801511288 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c09e070 con 0x55e33ca17b70 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8b96ffd640 1 -- 172.21.15.18:0/1801511288 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b9c0120f0 con 0x55e33ca17b70 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.175+0000 7f8b96ffd640 1 -- 172.21.15.18:0/1801511288 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c087040 con 0x55e33ca17b70 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 -- 172.21.15.18:0/1801511288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 msgr2=0x55e33ca10a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 --2- 172.21.15.18:0/1801511288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 0x55e33ca10a00 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c005780 tx=0x7f8b9c065e90 comp rx=0 tx=0).stop 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 -- 172.21.15.18:0/1801511288 shutdown_connections 2024-02-17T13:02:15.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 --2- 172.21.15.18:0/1801511288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca20750 0x55e33ca27480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 --2- 172.21.15.18:0/1801511288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca17b70 0x55e33ca10a00 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 --2- 172.21.15.18:0/1801511288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca1e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 -- 172.21.15.18:0/1801511288 >> 172.21.15.18:0/1801511288 conn(0x55e33ca11980 msgr2=0x55e33ca102f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 -- 172.21.15.18:0/1801511288 shutdown_connections 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 -- 172.21.15.18:0/1801511288 wait complete. 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 -- start start 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 Processor -- start 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca267a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca17b70 0x55e33ca26ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 0x55e33ca80fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33ca81500 con 0x55e33ca20750 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33ca81670 con 0x55e33ca17b70 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.176+0000 7f8bb620e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33ca81830 con 0x55e33ca17250 2024-02-17T13:02:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb620e740 10 client.0 _ll_register_callbacks cb 0x7f8ba4009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb1b08640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 0x55e33ca80fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb1b08640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 0x55e33ca80fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39624/0 (socket says 172.21.15.18:39624) 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb1b08640 1 -- 172.21.15.18:0/1875074201 learned_addr learned my addr 172.21.15.18:0/1875074201 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb0b06640 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca17b70 0x55e33ca26ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb1307640 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca267a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb1b08640 1 -- 172.21.15.18:0/1875074201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 msgr2=0x55e33ca267a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb1b08640 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca267a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb1b08640 1 -- 172.21.15.18:0/1875074201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca17b70 msgr2=0x55e33ca26ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb1b08640 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca17b70 0x55e33ca26ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb1b08640 1 -- 172.21.15.18:0/1875074201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac1a68c0 con 0x55e33ca20750 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb0b06640 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca17b70 0x55e33ca26ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb1b08640 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 0x55e33ca80fc0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4029e60 tx=0x7f8ba40049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4b83593c0c3b14 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8b94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:15.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8b94ff9640 1 -- 172.21.15.18:0/1875074201 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4013070 con 0x55e33ca20750 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8b94ff9640 1 -- 172.21.15.18:0/1875074201 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8ba40096d0 con 0x55e33ca20750 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb620e740 1 -- 172.21.15.18:0/1875074201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ba8034440 con 0x55e33ca20750 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8b94ff9640 1 -- 172.21.15.18:0/1875074201 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ba401e830 con 0x55e33ca20750 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb620e740 10 client.20013 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.177+0000 7f8bb620e740 1 -- 172.21.15.18:0/1875074201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55e33c93f8c0 con 0x55e33ca20750 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.178+0000 7f8b94ff9640 1 -- 172.21.15.18:0/1875074201 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ba8034440 con 0x55e33ca20750 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.178+0000 7f8b94ff9640 1 -- 172.21.15.18:0/1875074201 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ba400ed20 con 0x55e33ca20750 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.178+0000 7f8bb620e740 10 client.20013 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.178+0000 7f8bb620e740 10 client.20013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.178+0000 7f8bb620e740 20 client.20013 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.178+0000 7f8bb620e740 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e33ca89020 0x55e33ca8b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.178+0000 7f8bb620e740 4 client.20013 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.178+0000 7f8bb620e740 1 -- 172.21.15.18:0/1875074201 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e33c70a890 con 0x55e33ca89020 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.178+0000 7f8bb1307640 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e33ca89020 0x55e33ca8b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:15.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8b94ff9640 10 client.20013 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb1307640 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e33ca89020 0x55e33ca8b440 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4661473a207752 server_cookie=e6f4570bcb2fcadc in_seq=0 out_seq=0 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8b94ff9640 1 -- 172.21.15.18:0/1875074201 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e33c70a890 con 0x55e33ca89020 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8b94ff9640 10 client.20013 handle_command_reply: tid=0 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb620e740 1 client.20013 shutdown 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb620e740 1 -- 172.21.15.18:0/1875074201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 msgr2=0x55e33ca80fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb620e740 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 0x55e33ca80fc0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4029e60 tx=0x7f8ba40049d0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb620e740 1 -- 172.21.15.18:0/1875074201 shutdown_connections 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb620e740 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e33ca89020 0x55e33ca8b440 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb620e740 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e33ca20750 0x55e33ca80fc0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb620e740 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e33ca17b70 0x55e33ca26ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8b94ff9640 0 client.20013 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb620e740 1 --2- 172.21.15.18:0/1875074201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e33ca17250 0x55e33ca267a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb620e740 1 -- 172.21.15.18:0/1875074201 >> 172.21.15.18:0/1875074201 conn(0x55e33ca11980 msgr2=0x55e33ca10b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb620e740 1 -- 172.21.15.18:0/1875074201 shutdown_connections 2024-02-17T13:02:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb620e740 1 -- 172.21.15.18:0/1875074201 wait complete. 2024-02-17T13:02:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb620e740 20 client.20013 trim_cache size 0 max 16384 2024-02-17T13:02:15.176 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:02:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb356b640 1 -- 172.21.15.18:0/4081527549 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8b980a2260 msgr2=0x7f8b980a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.179+0000 7f8bb356b640 1 --2- 172.21.15.18:0/4081527549 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8b980a2260 0x7f8b980a4720 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4005fd0 tx=0x7f8ba4009f20 comp rx=0 tx=0).stop 2024-02-17T13:02:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.180+0000 7f8bb356b640 1 -- 172.21.15.18:0/4081527549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bac1064c0 msgr2=0x7f8bac100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.180+0000 7f8bb356b640 1 --2- 172.21.15.18:0/4081527549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bac1064c0 0x7f8bac100580 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00e9c0 tx=0x7f8b9c00ee80 comp rx=0 tx=0).stop 2024-02-17T13:02:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.180+0000 7f8bb356b640 1 -- 172.21.15.18:0/4081527549 shutdown_connections 2024-02-17T13:02:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.180+0000 7f8bb356b640 1 --2- 172.21.15.18:0/4081527549 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8b980a2260 0x7f8b980a4720 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.180+0000 7f8bb356b640 1 --2- 172.21.15.18:0/4081527549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac100ac0 0x7f8bac1a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.180+0000 7f8bb356b640 1 --2- 172.21.15.18:0/4081527549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bac1064c0 0x7f8bac100580 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.180+0000 7f8bb356b640 1 --2- 172.21.15.18:0/4081527549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bac0685f0 0x7f8bac100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.180+0000 7f8bb356b640 1 -- 172.21.15.18:0/4081527549 >> 172.21.15.18:0/4081527549 conn(0x7f8bac077180 msgr2=0x7f8bac0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:15.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.180+0000 7f8bb356b640 1 -- 172.21.15.18:0/4081527549 shutdown_connections 2024-02-17T13:02:15.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:15.180+0000 7f8bb356b640 1 -- 172.21.15.18:0/4081527549 wait complete. 2024-02-17T13:02:15.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:14 smithi146 ceph-mon[36454]: osdmap e104: 12 total, 12 up, 12 in 2024-02-17T13:02:15.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:14 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3376631734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:02:15.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 616129 4.86 MB/sec execute 570 sec latency 294.600 ms 2024-02-17T13:02:15.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:14 smithi119 ceph-mon[36188]: osdmap e104: 12 total, 12 up, 12 in 2024-02-17T13:02:15.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:14 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3376631734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:02:16.180 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T13:02:16.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:02:16.181 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T13:02:16.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:15 smithi146 ceph-mon[36454]: pgmap v389: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-02-17T13:02:16.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:15 smithi018 ceph-mon[32625]: pgmap v389: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-02-17T13:02:16.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.310+0000 7f8af0c49640 1 -- 172.21.15.18:0/435860424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aec101380 msgr2=0x7f8aec101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.310+0000 7f8af0c49640 1 --2- 172.21.15.18:0/435860424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aec101380 0x7f8aec101760 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009930 tx=0x7f8adc02bc30 comp rx=0 tx=0).stop 2024-02-17T13:02:16.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.310+0000 7f8af0c49640 1 -- 172.21.15.18:0/435860424 shutdown_connections 2024-02-17T13:02:16.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.310+0000 7f8af0c49640 1 --2- 172.21.15.18:0/435860424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8aec106450 0x7f8aec10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.310+0000 7f8af0c49640 1 --2- 172.21.15.18:0/435860424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8aec101d30 0x7f8aec105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.310+0000 7f8af0c49640 1 --2- 172.21.15.18:0/435860424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aec101380 0x7f8aec101760 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.310+0000 7f8af0c49640 1 -- 172.21.15.18:0/435860424 >> 172.21.15.18:0/435860424 conn(0x7f8aec078580 msgr2=0x7f8aec078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.310+0000 7f8af0c49640 1 -- 172.21.15.18:0/435860424 shutdown_connections 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.310+0000 7f8af0c49640 1 -- 172.21.15.18:0/435860424 wait complete. 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8af0c49640 1 Processor -- start 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8af0c49640 1 -- start start 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8af0c49640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aec101380 0x7f8aec19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8af0c49640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8aec106450 0x7f8aec19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8af0c49640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8aec197670 0x7f8aec197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8af0c49640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec198140 con 0x7f8aec101380 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8af0c49640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec1982b0 con 0x7f8aec106450 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8af0c49640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec198470 con 0x7f8aec197670 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8aea59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aec101380 0x7f8aec19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8aea59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aec101380 0x7f8aec19d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39650/0 (socket says 172.21.15.18:39650) 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8aea59c640 1 -- 172.21.15.18:0/3334034027 learned_addr learned my addr 172.21.15.18:0/3334034027 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8aead9d640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8aec197670 0x7f8aec197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8ae3fff640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8aec106450 0x7f8aec19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8aea59c640 1 -- 172.21.15.18:0/3334034027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8aec197670 msgr2=0x7f8aec197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8aea59c640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8aec197670 0x7f8aec197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.311+0000 7f8aea59c640 1 -- 172.21.15.18:0/3334034027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8aec106450 msgr2=0x7f8aec19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.312+0000 7f8aea59c640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8aec106450 0x7f8aec19daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.312+0000 7f8aea59c640 1 -- 172.21.15.18:0/3334034027 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc009560 con 0x7f8aec101380 2024-02-17T13:02:16.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.312+0000 7f8aea59c640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aec101380 0x7f8aec19d5b0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f8adc000c00 tx=0x7f8adc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:16.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.312+0000 7f8ae37fe640 1 -- 172.21.15.18:0/3334034027 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc004440 con 0x7f8aec101380 2024-02-17T13:02:16.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.312+0000 7f8ae37fe640 1 -- 172.21.15.18:0/3334034027 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8adc0052d0 con 0x7f8aec101380 2024-02-17T13:02:16.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.312+0000 7f8af0c49640 1 -- 172.21.15.18:0/3334034027 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec1a43b0 con 0x7f8aec101380 2024-02-17T13:02:16.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.312+0000 7f8ae37fe640 1 -- 172.21.15.18:0/3334034027 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc003da0 con 0x7f8aec101380 2024-02-17T13:02:16.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.312+0000 7f8af0c49640 1 -- 172.21.15.18:0/3334034027 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec1a4890 con 0x7f8aec101380 2024-02-17T13:02:16.305 INFO:tasks.workunit.client.0.smithi018.stdout: 1 616129 4.85 MB/sec execute 571 sec latency 1294.633 ms 2024-02-17T13:02:16.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.314+0000 7f8ae37fe640 1 -- 172.21.15.18:0/3334034027 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8adc005440 con 0x7f8aec101380 2024-02-17T13:02:16.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8af3817740 1 Processor -- start 2024-02-17T13:02:16.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8af3817740 1 -- start start 2024-02-17T13:02:16.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8af3817740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc6a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8af3817740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc6ad30 0x557e1fc732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8af3817740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 0x557e1fc75d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8af3817740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e1fbd8f90 con 0x557e1fc738b0 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8af3817740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e1fbd64c0 con 0x557e1fc6a380 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8af3817740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e1fbd1ca0 con 0x557e1fc6ad30 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8ae37fe640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8ad80a2230 0x7f8ad80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8ae3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 0x557e1fc75d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8ae3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 0x557e1fc75d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39666/0 (socket says 172.21.15.18:39666) 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8ae3fff640 1 -- 172.21.15.18:0/4062137574 learned_addr learned my addr 172.21.15.18:0/4062137574 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.315+0000 7f8aead9d640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8ad80a2230 0x7f8ad80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8aea59c640 1 --2- 172.21.15.18:0/4062137574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc6ad30 0x557e1fc732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8aead9d640 1 --2- 172.21.15.18:0/4062137574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc6a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8ae37fe640 1 -- 172.21.15.18:0/3334034027 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8adc0f0890 con 0x7f8aec101380 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8ae3fff640 1 -- 172.21.15.18:0/4062137574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc6ad30 msgr2=0x557e1fc732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8ae3fff640 1 --2- 172.21.15.18:0/4062137574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc6ad30 0x557e1fc732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8ae3fff640 1 -- 172.21.15.18:0/4062137574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 msgr2=0x557e1fc6a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8ae3fff640 1 --2- 172.21.15.18:0/4062137574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc6a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8ae3fff640 1 -- 172.21.15.18:0/4062137574 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e1fb940b0 con 0x557e1fc738b0 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8aead9d640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8ad80a2230 0x7f8ad80a46f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4005c40 tx=0x7f8ae4009c90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8ae3fff640 1 --2- 172.21.15.18:0/4062137574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 0x557e1fc75d00 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400dc50 tx=0x7f8ad400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4daef07a79a18931 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8ae0ff9640 1 -- 172.21.15.18:0/4062137574 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4004030 con 0x557e1fc738b0 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8ae0ff9640 1 -- 172.21.15.18:0/4062137574 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ad4004190 con 0x557e1fc738b0 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.316+0000 7f8ae0ff9640 1 -- 172.21.15.18:0/4062137574 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40116a0 con 0x557e1fc738b0 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.317+0000 7f8af3817740 1 -- 172.21.15.18:0/4062137574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 msgr2=0x557e1fc75d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.317+0000 7f8af3817740 1 --2- 172.21.15.18:0/4062137574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 0x557e1fc75d00 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400dc50 tx=0x7f8ad400f730 comp rx=0 tx=0).stop 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.317+0000 7f8af3817740 1 -- 172.21.15.18:0/4062137574 shutdown_connections 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.317+0000 7f8af3817740 1 --2- 172.21.15.18:0/4062137574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 0x557e1fc75d00 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.317+0000 7f8af3817740 1 --2- 172.21.15.18:0/4062137574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc6ad30 0x557e1fc732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.317+0000 7f8af3817740 1 --2- 172.21.15.18:0/4062137574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc6a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.317+0000 7f8af3817740 1 -- 172.21.15.18:0/4062137574 >> 172.21.15.18:0/4062137574 conn(0x557e1fc64840 msgr2=0x557e1fc64c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:16.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.317+0000 7f8af3817740 1 -- 172.21.15.18:0/4062137574 shutdown_connections 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.317+0000 7f8af3817740 1 -- 172.21.15.18:0/4062137574 wait complete. 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.317+0000 7f8af3817740 1 -- start start 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.317+0000 7f8af3817740 1 Processor -- start 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8af3817740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8af3817740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 0x557e1fc70ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8af3817740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc738b0 0x557e1fc71220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8af3817740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e1fb76f40 con 0x557e1fc6ad30 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8af3817740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e1f8a54b0 con 0x557e1fc6a380 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8af3817740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e1fb57670 con 0x557e1fc738b0 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8af3817740 10 client.0 _ll_register_callbacks cb 0x557e1fbc4f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8ae3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 0x557e1fc70ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8ae3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 0x557e1fc70ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39678/0 (socket says 172.21.15.18:39678) 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8ae3fff640 1 -- 172.21.15.18:0/2964385018 learned_addr learned my addr 172.21.15.18:0/2964385018 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8aea59c640 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8aead9d640 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc738b0 0x557e1fc71220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8ae3fff640 1 -- 172.21.15.18:0/2964385018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc738b0 msgr2=0x557e1fc71220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8ae3fff640 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc738b0 0x557e1fc71220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8ae3fff640 1 -- 172.21.15.18:0/2964385018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 msgr2=0x557e1fc707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8ae3fff640 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8ae3fff640 1 -- 172.21.15.18:0/2964385018 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad400d880 con 0x557e1fc6ad30 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8aead9d640 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc738b0 0x557e1fc71220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8ae3fff640 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 0x557e1fc70ce0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4003d30 tx=0x7f8ad400df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6353d87e9ec64c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8ad1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.318+0000 7f8ad1ffb640 1 -- 172.21.15.18:0/2964385018 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4020070 con 0x557e1fc6ad30 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8ad1ffb640 1 -- 172.21.15.18:0/2964385018 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8ad40038c0 con 0x557e1fc6ad30 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8af3817740 1 -- 172.21.15.18:0/2964385018 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e1fbabf40 con 0x557e1fc6ad30 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8ad1ffb640 1 -- 172.21.15.18:0/2964385018 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ad4003f30 con 0x557e1fc6ad30 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8af3817740 10 client.20025 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8af3817740 1 -- 172.21.15.18:0/2964385018 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x557e1fbd0320 con 0x557e1fc6ad30 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8ad1ffb640 1 -- 172.21.15.18:0/2964385018 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ad4031020 con 0x557e1fc6ad30 2024-02-17T13:02:16.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8ad1ffb640 1 -- 172.21.15.18:0/2964385018 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ad4017020 con 0x557e1fc6ad30 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8af3817740 10 client.20025 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8af3817740 10 client.20025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8af3817740 20 client.20025 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8af3817740 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557e1fc83d60 0x557e1fc86180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8af3817740 4 client.20025 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8af3817740 1 -- 172.21.15.18:0/2964385018 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e1fbc8310 con 0x557e1fc83d60 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.319+0000 7f8aea59c640 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557e1fc83d60 0x557e1fc86180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.320+0000 7f8aea59c640 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557e1fc83d60 0x557e1fc86180 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b48d914aa3cf7b server_cookie=db9de08245891efc in_seq=0 out_seq=0 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.320+0000 7f8ad1ffb640 10 client.20025 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.320+0000 7f8ad1ffb640 1 -- 172.21.15.18:0/2964385018 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e1fbc8310 con 0x557e1fc83d60 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.320+0000 7f8ad1ffb640 10 client.20025 handle_command_reply: tid=0 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8af3817740 1 client.20025 shutdown 2024-02-17T13:02:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8af3817740 1 -- 172.21.15.18:0/2964385018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 msgr2=0x557e1fc70ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8af3817740 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 0x557e1fc70ce0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4003d30 tx=0x7f8ad400df50 comp rx=0 tx=0).stop 2024-02-17T13:02:16.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8af3817740 1 -- 172.21.15.18:0/2964385018 shutdown_connections 2024-02-17T13:02:16.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8af3817740 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557e1fc83d60 0x557e1fc86180 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8af3817740 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc738b0 0x557e1fc71220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8ad1ffb640 0 client.20025 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:16.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8af3817740 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 0x557e1fc70ce0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8af3817740 1 --2- 172.21.15.18:0/2964385018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8af3817740 1 -- 172.21.15.18:0/2964385018 >> 172.21.15.18:0/2964385018 conn(0x557e1fc64840 msgr2=0x557e1fc667a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:16.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8af3817740 1 -- 172.21.15.18:0/2964385018 shutdown_connections 2024-02-17T13:02:16.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8af3817740 1 -- 172.21.15.18:0/2964385018 wait complete. 2024-02-17T13:02:16.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.321+0000 7f8af3817740 20 client.20025 trim_cache size 0 max 16384 2024-02-17T13:02:16.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.336+0000 7f8af3817740 1 Processor -- start 2024-02-17T13:02:16.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.336+0000 7f8af3817740 1 -- start start 2024-02-17T13:02:16.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.336+0000 7f8af3817740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc71040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.336+0000 7f8af3817740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc6ad30 0x557e1fc7a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.336+0000 7f8af3817740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 0x557e1fc7ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.336+0000 7f8af3817740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e1fc5edf0 con 0x557e1fc738b0 2024-02-17T13:02:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.336+0000 7f8af3817740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e1fc71af0 con 0x557e1fc6a380 2024-02-17T13:02:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.336+0000 7f8af3817740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e1fc6a1a0 con 0x557e1fc6ad30 2024-02-17T13:02:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.336+0000 7f8aead9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 0x557e1fc7ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.336+0000 7f8aead9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 0x557e1fc7ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39690/0 (socket says 172.21.15.18:39690) 2024-02-17T13:02:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.336+0000 7f8aead9d640 1 -- 172.21.15.18:0/2319209658 learned_addr learned my addr 172.21.15.18:0/2319209658 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.336+0000 7f8ae3fff640 1 --2- 172.21.15.18:0/2319209658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc6ad30 0x557e1fc7a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8aea59c640 1 --2- 172.21.15.18:0/2319209658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc71040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8aead9d640 1 -- 172.21.15.18:0/2319209658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc6ad30 msgr2=0x557e1fc7a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8aead9d640 1 --2- 172.21.15.18:0/2319209658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc6ad30 0x557e1fc7a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8aead9d640 1 -- 172.21.15.18:0/2319209658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 msgr2=0x557e1fc71040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8aead9d640 1 --2- 172.21.15.18:0/2319209658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc71040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8aead9d640 1 -- 172.21.15.18:0/2319209658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e1fb6cc00 con 0x557e1fc738b0 2024-02-17T13:02:16.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8aea59c640 1 --2- 172.21.15.18:0/2319209658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc71040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:16.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8aead9d640 1 --2- 172.21.15.18:0/2319209658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 0x557e1fc7ae20 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009be0 tx=0x7f8ae401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468b0c3a8b74c75e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:16.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8ad3fff640 1 -- 172.21.15.18:0/2319209658 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4012570 con 0x557e1fc738b0 2024-02-17T13:02:16.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8ad3fff640 1 -- 172.21.15.18:0/2319209658 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ae401edd0 con 0x557e1fc738b0 2024-02-17T13:02:16.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8ad3fff640 1 -- 172.21.15.18:0/2319209658 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae402ac00 con 0x557e1fc738b0 2024-02-17T13:02:16.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8af3817740 1 -- 172.21.15.18:0/2319209658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 msgr2=0x557e1fc7ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.337+0000 7f8af3817740 1 --2- 172.21.15.18:0/2319209658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 0x557e1fc7ae20 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009be0 tx=0x7f8ae401f000 comp rx=0 tx=0).stop 2024-02-17T13:02:16.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 -- 172.21.15.18:0/2319209658 shutdown_connections 2024-02-17T13:02:16.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 --2- 172.21.15.18:0/2319209658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc738b0 0x557e1fc7ae20 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 --2- 172.21.15.18:0/2319209658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc6ad30 0x557e1fc7a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 --2- 172.21.15.18:0/2319209658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fc71040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 -- 172.21.15.18:0/2319209658 >> 172.21.15.18:0/2319209658 conn(0x557e1fc64840 msgr2=0x557e1fc63050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:16.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 -- 172.21.15.18:0/2319209658 shutdown_connections 2024-02-17T13:02:16.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 -- 172.21.15.18:0/2319209658 wait complete. 2024-02-17T13:02:16.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 -- start start 2024-02-17T13:02:16.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 Processor -- start 2024-02-17T13:02:16.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fcd4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 0x557e1fcd5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc738b0 0x557e1fc7a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e1fc7a760 con 0x557e1fc6ad30 2024-02-17T13:02:16.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e1fc7a8d0 con 0x557e1fc6a380 2024-02-17T13:02:16.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e1fc7aa90 con 0x557e1fc738b0 2024-02-17T13:02:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8af3817740 10 client.0 _ll_register_callbacks cb 0x7f8ad400d3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8ae3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 0x557e1fcd5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8ae3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 0x557e1fcd5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39704/0 (socket says 172.21.15.18:39704) 2024-02-17T13:02:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8ae3fff640 1 -- 172.21.15.18:0/2823660963 learned_addr learned my addr 172.21.15.18:0/2823660963 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8aead9d640 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc738b0 0x557e1fc7a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8aea59c640 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fcd4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.338+0000 7f8ae3fff640 1 -- 172.21.15.18:0/2823660963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc738b0 msgr2=0x557e1fc7a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8ae3fff640 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc738b0 0x557e1fc7a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8ae3fff640 1 -- 172.21.15.18:0/2823660963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 msgr2=0x557e1fcd4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8ae3fff640 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fcd4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8ae3fff640 1 -- 172.21.15.18:0/2823660963 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae401a040 con 0x557e1fc6ad30 2024-02-17T13:02:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8aea59c640 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fcd4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8aead9d640 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc738b0 0x557e1fc7a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8ae3fff640 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 0x557e1fcd5140 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x557e1fc71d70 tx=0x7f8ad400d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891aa169424dbb87 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8ad1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8ad1ffb640 1 -- 172.21.15.18:0/2823660963 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4020040 con 0x557e1fc6ad30 2024-02-17T13:02:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8ad1ffb640 1 -- 172.21.15.18:0/2823660963 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8ad4000e40 con 0x557e1fc6ad30 2024-02-17T13:02:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8af3817740 1 -- 172.21.15.18:0/2823660963 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e1fbf7900 con 0x557e1fc6ad30 2024-02-17T13:02:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8ad1ffb640 1 -- 172.21.15.18:0/2823660963 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ad4012ea0 con 0x557e1fc6ad30 2024-02-17T13:02:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8af3817740 10 client.20031 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8af3817740 1 -- 172.21.15.18:0/2823660963 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x557e1fb92910 con 0x557e1fc6ad30 2024-02-17T13:02:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8ad1ffb640 1 -- 172.21.15.18:0/2823660963 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557e1fbf7900 con 0x557e1fc6ad30 2024-02-17T13:02:16.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8ad1ffb640 1 -- 172.21.15.18:0/2823660963 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8ad4017070 con 0x557e1fc6ad30 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8af3817740 10 client.20031 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8af3817740 10 client.20031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8af3817740 20 client.20031 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8af3817740 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557e1fcdc790 0x557e1fcdebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8af3817740 4 client.20031 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.339+0000 7f8af3817740 1 -- 172.21.15.18:0/2823660963 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557e1f8d1860 con 0x557e1fcdc790 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.340+0000 7f8aea59c640 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557e1fcdc790 0x557e1fcdebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.340+0000 7f8aea59c640 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557e1fcdc790 0x557e1fcdebb0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1674f7af588ebfeb server_cookie=327c698841f8120b in_seq=0 out_seq=0 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.340+0000 7f8ad1ffb640 10 client.20031 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8ad1ffb640 1 -- 172.21.15.18:0/2823660963 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557e1f8d1860 con 0x557e1fcdc790 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8ad1ffb640 10 client.20031 handle_command_reply: tid=0 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8af3817740 1 client.20031 shutdown 2024-02-17T13:02:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8af3817740 1 -- 172.21.15.18:0/2823660963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 msgr2=0x557e1fcd5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8af3817740 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 0x557e1fcd5140 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x557e1fc71d70 tx=0x7f8ad400d5d0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8af3817740 1 -- 172.21.15.18:0/2823660963 shutdown_connections 2024-02-17T13:02:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8af3817740 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x557e1fcdc790 0x557e1fcdebb0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8af3817740 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557e1fc738b0 0x557e1fc7a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8af3817740 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1fc6ad30 0x557e1fcd5140 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8af3817740 1 --2- 172.21.15.18:0/2823660963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e1fc6a380 0x557e1fcd4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8af3817740 1 -- 172.21.15.18:0/2823660963 >> 172.21.15.18:0/2823660963 conn(0x557e1fc64840 msgr2=0x557e1fc66380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8ad1ffb640 0 client.20031 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8af3817740 1 -- 172.21.15.18:0/2823660963 shutdown_connections 2024-02-17T13:02:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8af3817740 1 -- 172.21.15.18:0/2823660963 wait complete. 2024-02-17T13:02:16.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.341+0000 7f8af3817740 20 client.20031 trim_cache size 0 max 16384 2024-02-17T13:02:16.340 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:02:16.340 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T13:02:16.340 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "2d8d6a5f-fa16-4785-947e-014783876bf2", 2024-02-17T13:02:16.340 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T13:02:16.340 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:02:16.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.342+0000 7f8af0c49640 1 -- 172.21.15.18:0/3334034027 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8ad80a2230 msgr2=0x7f8ad80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.342+0000 7f8af0c49640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8ad80a2230 0x7f8ad80a46f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4005c40 tx=0x7f8ae4009c90 comp rx=0 tx=0).stop 2024-02-17T13:02:16.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.342+0000 7f8af0c49640 1 -- 172.21.15.18:0/3334034027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aec101380 msgr2=0x7f8aec19d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.342+0000 7f8af0c49640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aec101380 0x7f8aec19d5b0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f8adc000c00 tx=0x7f8adc004040 comp rx=0 tx=0).stop 2024-02-17T13:02:16.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.342+0000 7f8af0c49640 1 -- 172.21.15.18:0/3334034027 shutdown_connections 2024-02-17T13:02:16.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.342+0000 7f8af0c49640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8ad80a2230 0x7f8ad80a46f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.342+0000 7f8af0c49640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8aec197670 0x7f8aec197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.342+0000 7f8af0c49640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8aec106450 0x7f8aec19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.342+0000 7f8af0c49640 1 --2- 172.21.15.18:0/3334034027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aec101380 0x7f8aec19d5b0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.342+0000 7f8af0c49640 1 -- 172.21.15.18:0/3334034027 >> 172.21.15.18:0/3334034027 conn(0x7f8aec078580 msgr2=0x7f8aec0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:16.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.342+0000 7f8af0c49640 1 -- 172.21.15.18:0/3334034027 shutdown_connections 2024-02-17T13:02:16.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.342+0000 7f8af0c49640 1 -- 172.21.15.18:0/3334034027 wait complete. 2024-02-17T13:02:16.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:15 smithi119 ceph-mon[36188]: pgmap v389: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-02-17T13:02:16.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:02:16.343 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:02:16.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.502+0000 7f3b87e23640 1 -- 172.21.15.18:0/2412966427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b80105060 msgr2=0x7f3b80105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.502+0000 7f3b87e23640 1 --2- 172.21.15.18:0/2412966427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b80105060 0x7f3b80105440 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009930 tx=0x7f3b7c02bc30 comp rx=0 tx=0).stop 2024-02-17T13:02:16.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.502+0000 7f3b87e23640 1 -- 172.21.15.18:0/2412966427 shutdown_connections 2024-02-17T13:02:16.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.502+0000 7f3b87e23640 1 --2- 172.21.15.18:0/2412966427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b80100d00 0x7f3b8010cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.502+0000 7f3b87e23640 1 --2- 172.21.15.18:0/2412966427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b80100360 0x7f3b801007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.502+0000 7f3b87e23640 1 --2- 172.21.15.18:0/2412966427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b80105060 0x7f3b80105440 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.502+0000 7f3b87e23640 1 -- 172.21.15.18:0/2412966427 >> 172.21.15.18:0/2412966427 conn(0x7f3b80076db0 msgr2=0x7f3b800771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:16.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.502+0000 7f3b87e23640 1 -- 172.21.15.18:0/2412966427 shutdown_connections 2024-02-17T13:02:16.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.502+0000 7f3b87e23640 1 -- 172.21.15.18:0/2412966427 wait complete. 2024-02-17T13:02:16.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b87e23640 1 Processor -- start 2024-02-17T13:02:16.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b87e23640 1 -- start start 2024-02-17T13:02:16.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b87e23640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b80100360 0x7f3b8019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b87e23640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b80100d00 0x7f3b8019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b87e23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b80105060 0x7f3b80197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b87e23640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b80197be0 con 0x7f3b80100360 2024-02-17T13:02:16.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b87e23640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b80197d50 con 0x7f3b80105060 2024-02-17T13:02:16.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b87e23640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b80197f10 con 0x7f3b80100d00 2024-02-17T13:02:16.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b85bbf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b80100360 0x7f3b8019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b85bbf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b80100360 0x7f3b8019d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39724/0 (socket says 172.21.15.18:39724) 2024-02-17T13:02:16.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b85bbf640 1 -- 172.21.15.18:0/903367084 learned_addr learned my addr 172.21.15.18:0/903367084 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:16.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b853be640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b80100d00 0x7f3b8019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.503+0000 7f3b863c0640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b80105060 0x7f3b80197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b85bbf640 1 -- 172.21.15.18:0/903367084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b80100d00 msgr2=0x7f3b8019d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b85bbf640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b80100d00 0x7f3b8019d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b85bbf640 1 -- 172.21.15.18:0/903367084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b80105060 msgr2=0x7f3b80197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b85bbf640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b80105060 0x7f3b80197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b85bbf640 1 -- 172.21.15.18:0/903367084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b70009490 con 0x7f3b80100360 2024-02-17T13:02:16.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b853be640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b80100d00 0x7f3b8019d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:16.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b863c0640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b80105060 0x7f3b80197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:16.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b85bbf640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b80100360 0x7f3b8019d3e0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009230 tx=0x7f3b7c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:16.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b76ffd640 1 -- 172.21.15.18:0/903367084 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c004350 con 0x7f3b80100360 2024-02-17T13:02:16.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b87e23640 1 -- 172.21.15.18:0/903367084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c009560 con 0x7f3b80100360 2024-02-17T13:02:16.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b76ffd640 1 -- 172.21.15.18:0/903367084 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b7c0044b0 con 0x7f3b80100360 2024-02-17T13:02:16.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b76ffd640 1 -- 172.21.15.18:0/903367084 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c003d00 con 0x7f3b80100360 2024-02-17T13:02:16.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.504+0000 7f3b87e23640 1 -- 172.21.15.18:0/903367084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b801a42a0 con 0x7f3b80100360 2024-02-17T13:02:16.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.506+0000 7f3b76ffd640 1 -- 172.21.15.18:0/903367084 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3b7c004df0 con 0x7f3b80100360 2024-02-17T13:02:16.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b8a9f8740 1 Processor -- start 2024-02-17T13:02:16.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b8a9f8740 1 -- start start 2024-02-17T13:02:16.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b8a9f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1c2b0 0x559b3ee1c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b8a9f8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1cbd0 0x559b3ee25270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b8a9f8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee27ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b8a9f8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ed8b280 con 0x559b3ee1cbd0 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b8a9f8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ed887b0 con 0x559b3ee257b0 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b8a9f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ed83f90 con 0x559b3ee1c2b0 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b85bbf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1cbd0 0x559b3ee25270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b863c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1c2b0 0x559b3ee1c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b853be640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee27ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b85bbf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1cbd0 0x559b3ee25270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39736/0 (socket says 172.21.15.18:39736) 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b85bbf640 1 -- 172.21.15.18:0/3916876677 learned_addr learned my addr 172.21.15.18:0/3916876677 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.507+0000 7f3b76ffd640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3b6c0a2230 0x7f3b6c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b863c0640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3b6c0a2230 0x7f3b6c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b76ffd640 1 -- 172.21.15.18:0/903367084 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3b7c0eff40 con 0x7f3b80100360 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b85bbf640 1 -- 172.21.15.18:0/3916876677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1c2b0 msgr2=0x559b3ee1c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b85bbf640 1 --2- 172.21.15.18:0/3916876677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1c2b0 0x559b3ee1c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b85bbf640 1 -- 172.21.15.18:0/3916876677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 msgr2=0x559b3ee27ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b85bbf640 1 --2- 172.21.15.18:0/3916876677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee27ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b85bbf640 1 -- 172.21.15.18:0/3916876677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b3ed42570 con 0x559b3ee1cbd0 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b853be640 1 --2- 172.21.15.18:0/3916876677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee27ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b85bbf640 1 --2- 172.21.15.18:0/3916876677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1cbd0 0x559b3ee25270 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0ed760 tx=0x7f3b7c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aec0c9eb2b2773c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b863c0640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3b6c0a2230 0x7f3b6c0a46f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b780049d0 tx=0x7f3b78004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b6bfff640 1 -- 172.21.15.18:0/3916876677 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c003e60 con 0x559b3ee1cbd0 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b6bfff640 1 -- 172.21.15.18:0/3916876677 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b7c0035f0 con 0x559b3ee1cbd0 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.508+0000 7f3b6bfff640 1 -- 172.21.15.18:0/3916876677 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0055d0 con 0x559b3ee1cbd0 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.509+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/3916876677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1cbd0 msgr2=0x559b3ee25270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.509+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/3916876677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1cbd0 0x559b3ee25270 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0ed760 tx=0x7f3b7c009260 comp rx=0 tx=0).stop 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.509+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/3916876677 shutdown_connections 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.509+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/3916876677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee27ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.509+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/3916876677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1cbd0 0x559b3ee25270 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.509+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/3916876677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1c2b0 0x559b3ee1c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.509+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/3916876677 >> 172.21.15.18:0/3916876677 conn(0x559b3ee16990 msgr2=0x559b3ee16d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.509+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/3916876677 shutdown_connections 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.509+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/3916876677 wait complete. 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.509+0000 7f3b8a9f8740 1 -- start start 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.509+0000 7f3b8a9f8740 1 Processor -- start 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b8a9f8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1c2b0 0x559b3ee25200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b8a9f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1cbd0 0x559b3ee22590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b8a9f8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee22cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b8a9f8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ea57430 con 0x559b3ee1c2b0 2024-02-17T13:02:16.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b8a9f8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ea13e00 con 0x559b3ee257b0 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b8a9f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ed08fd0 con 0x559b3ee1cbd0 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b8a9f8740 10 client.0 _ll_register_callbacks cb 0x559b3ed5efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b85bbf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1c2b0 0x559b3ee25200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b85bbf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1c2b0 0x559b3ee25200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39738/0 (socket says 172.21.15.18:39738) 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b85bbf640 1 -- 172.21.15.18:0/2072690354 learned_addr learned my addr 172.21.15.18:0/2072690354 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b863c0640 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee22cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b853be640 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1cbd0 0x559b3ee22590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b85bbf640 1 -- 172.21.15.18:0/2072690354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1cbd0 msgr2=0x559b3ee22590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b85bbf640 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1cbd0 0x559b3ee22590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b85bbf640 1 -- 172.21.15.18:0/2072690354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 msgr2=0x559b3ee22cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b85bbf640 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee22cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b85bbf640 1 -- 172.21.15.18:0/2072690354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b801a42a0 con 0x559b3ee1c2b0 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b85bbf640 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1c2b0 0x559b3ee25200 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0b9430 tx=0x7f3b7c0037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5a5456b748048e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:16.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.510+0000 7f3b697fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b697fa640 1 -- 172.21.15.18:0/2072690354 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0b8d00 con 0x559b3ee1c2b0 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b697fa640 1 -- 172.21.15.18:0/2072690354 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3b7c0bca60 con 0x559b3ee1c2b0 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/2072690354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b3ea837e0 con 0x559b3ee1c2b0 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b697fa640 1 -- 172.21.15.18:0/2072690354 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b7c0bd160 con 0x559b3ee1c2b0 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b8a9f8740 10 client.20043 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/2072690354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x559b3ed82610 con 0x559b3ee1c2b0 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b697fa640 1 -- 172.21.15.18:0/2072690354 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b7c0d4070 con 0x559b3ee1c2b0 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b697fa640 1 -- 172.21.15.18:0/2072690354 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b7c03c070 con 0x559b3ee1c2b0 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b8a9f8740 10 client.20043 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b8a9f8740 10 client.20043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b8a9f8740 20 client.20043 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559b3ee35310 0x559b3ee37700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b8a9f8740 4 client.20043 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/2072690354 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b3eac0720 con 0x559b3ee35310 2024-02-17T13:02:16.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.511+0000 7f3b853be640 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559b3ee35310 0x559b3ee37700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.512+0000 7f3b853be640 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559b3ee35310 0x559b3ee37700 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65534a426914ed41 server_cookie=49ce18be4c3db339 in_seq=0 out_seq=0 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.512+0000 7f3b697fa640 10 client.20043 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.512+0000 7f3b697fa640 1 -- 172.21.15.18:0/2072690354 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b3eac0720 con 0x559b3ee35310 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.512+0000 7f3b697fa640 10 client.20043 handle_command_reply: tid=0 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.512+0000 7f3b8a9f8740 1 client.20043 shutdown 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.512+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/2072690354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1c2b0 msgr2=0x559b3ee25200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.512+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1c2b0 0x559b3ee25200 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0b9430 tx=0x7f3b7c0037d0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.513+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/2072690354 shutdown_connections 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.513+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559b3ee35310 0x559b3ee37700 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.513+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee22cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.513+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1cbd0 0x559b3ee22590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.513+0000 7f3b697fa640 0 client.20043 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:16.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.513+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/2072690354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1c2b0 0x559b3ee25200 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.513+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/2072690354 >> 172.21.15.18:0/2072690354 conn(0x559b3ee16990 msgr2=0x559b3ee18730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:16.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.513+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/2072690354 shutdown_connections 2024-02-17T13:02:16.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.513+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/2072690354 wait complete. 2024-02-17T13:02:16.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.513+0000 7f3b8a9f8740 20 client.20043 trim_cache size 0 max 16384 2024-02-17T13:02:16.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.530+0000 7f3b8a9f8740 1 Processor -- start 2024-02-17T13:02:16.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.531+0000 7f3b8a9f8740 1 -- start start 2024-02-17T13:02:16.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.532+0000 7f3b8a9f8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1c2b0 0x559b3ee2c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.532+0000 7f3b8a9f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1cbd0 0x559b3ee2c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.532+0000 7f3b8a9f8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee24cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.532+0000 7f3b8a9f8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ee1b3a0 con 0x559b3ee1c2b0 2024-02-17T13:02:16.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.532+0000 7f3b8a9f8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ee179e0 con 0x559b3ee257b0 2024-02-17T13:02:16.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.532+0000 7f3b8a9f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ee1bcf0 con 0x559b3ee1cbd0 2024-02-17T13:02:16.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.534+0000 7f3b863c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee24cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.534+0000 7f3b853be640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1cbd0 0x559b3ee2c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.534+0000 7f3b853be640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1cbd0 0x559b3ee2c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34256/0 (socket says 172.21.15.18:34256) 2024-02-17T13:02:16.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.534+0000 7f3b853be640 1 -- 172.21.15.18:0/672440180 learned_addr learned my addr 172.21.15.18:0/672440180 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:16.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.535+0000 7f3b85bbf640 1 --2- 172.21.15.18:0/672440180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1c2b0 0x559b3ee2c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.535+0000 7f3b853be640 1 -- 172.21.15.18:0/672440180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 msgr2=0x559b3ee24cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.535+0000 7f3b853be640 1 --2- 172.21.15.18:0/672440180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee24cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.535+0000 7f3b853be640 1 -- 172.21.15.18:0/672440180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1c2b0 msgr2=0x559b3ee2c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.535+0000 7f3b853be640 1 --2- 172.21.15.18:0/672440180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1c2b0 0x559b3ee2c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.535+0000 7f3b853be640 1 -- 172.21.15.18:0/672440180 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b3ed1eac0 con 0x559b3ee1cbd0 2024-02-17T13:02:16.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.535+0000 7f3b853be640 1 --2- 172.21.15.18:0/672440180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1cbd0 0x559b3ee2c960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3b8006a8b0 tx=0x7f3b70005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81affd568baedd7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:16.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.536+0000 7f3b6b7fe640 1 -- 172.21.15.18:0/672440180 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70004310 con 0x559b3ee1cbd0 2024-02-17T13:02:16.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.536+0000 7f3b6b7fe640 1 -- 172.21.15.18:0/672440180 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b70018040 con 0x559b3ee1cbd0 2024-02-17T13:02:16.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.538+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/672440180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1cbd0 msgr2=0x559b3ee2c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.538+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/672440180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1cbd0 0x559b3ee2c960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3b8006a8b0 tx=0x7f3b70005f90 comp rx=0 tx=0).stop 2024-02-17T13:02:16.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.538+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/672440180 shutdown_connections 2024-02-17T13:02:16.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.538+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/672440180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee257b0 0x559b3ee24cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.538+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/672440180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee1cbd0 0x559b3ee2c960 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.538+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/672440180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1c2b0 0x559b3ee2c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.538+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/672440180 >> 172.21.15.18:0/672440180 conn(0x559b3ee16990 msgr2=0x559b3ee20e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:16.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.539+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/672440180 shutdown_connections 2024-02-17T13:02:16.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.539+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/672440180 wait complete. 2024-02-17T13:02:16.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.539+0000 7f3b8a9f8740 1 -- start start 2024-02-17T13:02:16.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.539+0000 7f3b8a9f8740 1 Processor -- start 2024-02-17T13:02:16.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.540+0000 7f3b8a9f8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee1c2b0 0x559b3ee2b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.540+0000 7f3b8a9f8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1cbd0 0x559b3ee2ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.540+0000 7f3b8a9f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee257b0 0x559b3ee8b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.540+0000 7f3b8a9f8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ee2c1d0 con 0x559b3ee1cbd0 2024-02-17T13:02:16.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.540+0000 7f3b8a9f8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ee8bcb0 con 0x559b3ee1c2b0 2024-02-17T13:02:16.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.540+0000 7f3b8a9f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ee8be40 con 0x559b3ee257b0 2024-02-17T13:02:16.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.540+0000 7f3b8a9f8740 10 client.0 _ll_register_callbacks cb 0x7f3b7c0b8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.541+0000 7f3b85bbf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee1c2b0 0x559b3ee2b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.541+0000 7f3b853be640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1cbd0 0x559b3ee2ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.541+0000 7f3b863c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee257b0 0x559b3ee8b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.541+0000 7f3b863c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee257b0 0x559b3ee8b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:34260/0 (socket says 172.21.15.18:34260) 2024-02-17T13:02:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.541+0000 7f3b863c0640 1 -- 172.21.15.18:0/616997028 learned_addr learned my addr 172.21.15.18:0/616997028 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.541+0000 7f3b863c0640 1 -- 172.21.15.18:0/616997028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee1c2b0 msgr2=0x559b3ee2b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.541+0000 7f3b863c0640 1 --2- 172.21.15.18:0/616997028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee1c2b0 0x559b3ee2b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.541+0000 7f3b863c0640 1 -- 172.21.15.18:0/616997028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1cbd0 msgr2=0x559b3ee2ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.541+0000 7f3b863c0640 1 --2- 172.21.15.18:0/616997028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1cbd0 0x559b3ee2ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.541+0000 7f3b863c0640 1 -- 172.21.15.18:0/616997028 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b70009c00 con 0x559b3ee257b0 2024-02-17T13:02:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.542+0000 7f3b863c0640 1 --2- 172.21.15.18:0/616997028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee257b0 0x559b3ee8b770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3b78023bc0 tx=0x7f3b78004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba06d820c696288c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.542+0000 7f3b697fa640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:02:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.542+0000 7f3b697fa640 1 -- 172.21.15.18:0/616997028 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78028070 con 0x559b3ee257b0 2024-02-17T13:02:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.542+0000 7f3b697fa640 1 -- 172.21.15.18:0/616997028 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3b78004b20 con 0x559b3ee257b0 2024-02-17T13:02:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.542+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/616997028 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b7c0035f0 con 0x559b3ee257b0 2024-02-17T13:02:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.542+0000 7f3b697fa640 1 -- 172.21.15.18:0/616997028 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b780132f0 con 0x559b3ee257b0 2024-02-17T13:02:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.542+0000 7f3b8a9f8740 10 client.29023 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.542+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/616997028 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x559b3ed446a0 con 0x559b3ee257b0 2024-02-17T13:02:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.542+0000 7f3b697fa640 1 -- 172.21.15.18:0/616997028 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b7c0035f0 con 0x559b3ee257b0 2024-02-17T13:02:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.543+0000 7f3b697fa640 1 -- 172.21.15.18:0/616997028 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b780052d0 con 0x559b3ee257b0 2024-02-17T13:02:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.543+0000 7f3b8a9f8740 10 client.29023 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.543+0000 7f3b8a9f8740 10 client.29023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.543+0000 7f3b8a9f8740 20 client.29023 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.543+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/616997028 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559b3ee8d720 0x559b3ee8fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.543+0000 7f3b8a9f8740 4 client.29023 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:02:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.543+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/616997028 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559b3eb0f890 con 0x559b3ee8d720 2024-02-17T13:02:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.543+0000 7f3b85bbf640 1 --2- 172.21.15.18:0/616997028 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559b3ee8d720 0x559b3ee8fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.543+0000 7f3b85bbf640 1 --2- 172.21.15.18:0/616997028 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559b3ee8d720 0x559b3ee8fb40 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eecf30570da4453e server_cookie=f158759a87c3d7db in_seq=0 out_seq=0 2024-02-17T13:02:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.543+0000 7f3b697fa640 10 client.29023 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:16.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b697fa640 1 -- 172.21.15.18:0/616997028 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559b3eb0f890 con 0x559b3ee8d720 2024-02-17T13:02:16.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b697fa640 10 client.29023 handle_command_reply: tid=0 2024-02-17T13:02:16.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b8a9f8740 1 client.29023 shutdown 2024-02-17T13:02:16.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/616997028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee257b0 msgr2=0x559b3ee8b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/616997028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee257b0 0x559b3ee8b770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3b78023bc0 tx=0x7f3b78004a40 comp rx=0 tx=0).stop 2024-02-17T13:02:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/616997028 shutdown_connections 2024-02-17T13:02:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/616997028 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559b3ee8d720 0x559b3ee8fb40 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/616997028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b3ee257b0 0x559b3ee8b770 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/616997028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b3ee1cbd0 0x559b3ee2ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b697fa640 0 client.29023 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b8a9f8740 1 --2- 172.21.15.18:0/616997028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3ee1c2b0 0x559b3ee2b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/616997028 >> 172.21.15.18:0/616997028 conn(0x559b3ee16990 msgr2=0x559b3ee20da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/616997028 shutdown_connections 2024-02-17T13:02:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b8a9f8740 1 -- 172.21.15.18:0/616997028 wait complete. 2024-02-17T13:02:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.545+0000 7f3b8a9f8740 20 client.29023 trim_cache size 0 max 16384 2024-02-17T13:02:16.539 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:02:16.539 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T13:02:16.539 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T13:02:16.539 INFO:teuthology.orchestra.run.smithi018.stdout: "2d8d6a5f-fa16-4785-947e-014783876bf2": { 2024-02-17T13:02:16.539 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T13:02:16.539 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "2d8d6a5f-fa16-4785-947e-014783876bf2", 2024-02-17T13:02:16.539 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T13:02:16.539 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:02:16.539 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:02:16.539 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:02:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.546+0000 7f3b87e23640 1 -- 172.21.15.18:0/903367084 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3b6c0a2230 msgr2=0x7f3b6c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.546+0000 7f3b87e23640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3b6c0a2230 0x7f3b6c0a46f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b780049d0 tx=0x7f3b78004ab0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.546+0000 7f3b87e23640 1 -- 172.21.15.18:0/903367084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b80100360 msgr2=0x7f3b8019d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.546+0000 7f3b87e23640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b80100360 0x7f3b8019d3e0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009230 tx=0x7f3b7c02c650 comp rx=0 tx=0).stop 2024-02-17T13:02:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.546+0000 7f3b87e23640 1 -- 172.21.15.18:0/903367084 shutdown_connections 2024-02-17T13:02:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.546+0000 7f3b87e23640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3b6c0a2230 0x7f3b6c0a46f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.546+0000 7f3b87e23640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b80105060 0x7f3b80197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.546+0000 7f3b87e23640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b80100d00 0x7f3b8019d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.546+0000 7f3b87e23640 1 --2- 172.21.15.18:0/903367084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b80100360 0x7f3b8019d3e0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.546+0000 7f3b87e23640 1 -- 172.21.15.18:0/903367084 >> 172.21.15.18:0/903367084 conn(0x7f3b80076db0 msgr2=0x7f3b80103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.546+0000 7f3b87e23640 1 -- 172.21.15.18:0/903367084 shutdown_connections 2024-02-17T13:02:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:16.546+0000 7f3b87e23640 1 -- 172.21.15.18:0/903367084 wait complete. 2024-02-17T13:02:16.546 INFO:tasks.cephfs.filesystem:scrub status for tag:2d8d6a5f-fa16-4785-947e-014783876bf2 - {'path': '/', 'tag': '2d8d6a5f-fa16-4785-947e-014783876bf2', 'options': 'recursive,force'} 2024-02-17T13:02:17.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:16 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:17.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:16 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:17.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:16 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:17.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:16 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:17.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:16 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:17.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:16 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:17.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:16 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:17.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:16 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:17.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 618196 4.86 MB/sec execute 572 sec latency 1352.625 ms 2024-02-17T13:02:17.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:16 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:17.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:16 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:17.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:16 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:17.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:16 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T13:02:18.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:18.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:18.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:18.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:18.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:17 smithi146 ceph-mon[36454]: pgmap v390: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-02-17T13:02:18.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T13:02:18.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:18.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:18.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:18.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:17 smithi018 ceph-mon[32625]: pgmap v390: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-02-17T13:02:18.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 619489 4.86 MB/sec execute 573 sec latency 16.991 ms 2024-02-17T13:02:18.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T13:02:18.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:18.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:18.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:18.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:18.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:18.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:17 smithi119 ceph-mon[36188]: pgmap v390: 97 pgs: 97 active+clean; 243 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-02-17T13:02:19.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 620555 4.86 MB/sec execute 574 sec latency 456.283 ms 2024-02-17T13:02:20.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:19 smithi146 ceph-mon[36454]: pgmap v391: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-02-17T13:02:20.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:19 smithi018 ceph-mon[32625]: pgmap v391: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-02-17T13:02:20.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 622065 4.87 MB/sec execute 575 sec latency 14.527 ms 2024-02-17T13:02:20.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:19 smithi119 ceph-mon[36188]: pgmap v391: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-02-17T13:02:21.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 623219 4.87 MB/sec execute 576 sec latency 284.492 ms 2024-02-17T13:02:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.944+0000 7fede56f4640 1 -- 172.21.15.18:0/1649785027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede01002c0 msgr2=0x7fede01006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.944+0000 7fede56f4640 1 --2- 172.21.15.18:0/1649785027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede01002c0 0x7fede01006a0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009930 tx=0x7fedd402bc30 comp rx=0 tx=0).stop 2024-02-17T13:02:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.944+0000 7fede56f4640 1 -- 172.21.15.18:0/1649785027 shutdown_connections 2024-02-17T13:02:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.944+0000 7fede56f4640 1 --2- 172.21.15.18:0/1649785027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fede0105390 0x7fede010cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.944+0000 7fede56f4640 1 --2- 172.21.15.18:0/1649785027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fede0100c70 0x7fede0104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.944+0000 7fede56f4640 1 --2- 172.21.15.18:0/1649785027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede01002c0 0x7fede01006a0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.944+0000 7fede56f4640 1 -- 172.21.15.18:0/1649785027 >> 172.21.15.18:0/1649785027 conn(0x7fede00771b0 msgr2=0x7fede00775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.944+0000 7fede56f4640 1 -- 172.21.15.18:0/1649785027 shutdown_connections 2024-02-17T13:02:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.944+0000 7fede56f4640 1 -- 172.21.15.18:0/1649785027 wait complete. 2024-02-17T13:02:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.945+0000 7fede56f4640 1 Processor -- start 2024-02-17T13:02:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.945+0000 7fede56f4640 1 -- start start 2024-02-17T13:02:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.945+0000 7fede56f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede01002c0 0x7fede019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.945+0000 7fede56f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fede0105390 0x7fede019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.945+0000 7fede56f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fede01997a0 0x7fede0199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.945+0000 7fede56f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede019a250 con 0x7fede01002c0 2024-02-17T13:02:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.945+0000 7fede56f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede019a3c0 con 0x7fede01997a0 2024-02-17T13:02:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.945+0000 7fede56f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede019a530 con 0x7fede0105390 2024-02-17T13:02:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.945+0000 7feddeffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede01002c0 0x7fede019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.945+0000 7feddeffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede01002c0 0x7fede019f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36064/0 (socket says 172.21.15.18:36064) 2024-02-17T13:02:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.945+0000 7feddeffd640 1 -- 172.21.15.18:0/2840741569 learned_addr learned my addr 172.21.15.18:0/2840741569 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7fedde7fc640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fede0105390 0x7fede019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7feddf7fe640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fede01997a0 0x7fede0199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7feddeffd640 1 -- 172.21.15.18:0/2840741569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fede0105390 msgr2=0x7fede019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7feddeffd640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fede0105390 0x7fede019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7feddeffd640 1 -- 172.21.15.18:0/2840741569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fede01997a0 msgr2=0x7fede0199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7feddeffd640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fede01997a0 0x7fede0199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7feddeffd640 1 -- 172.21.15.18:0/2840741569 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd4009560 con 0x7fede01002c0 2024-02-17T13:02:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7fedde7fc640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fede0105390 0x7fede019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7feddf7fe640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fede01997a0 0x7fede0199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7feddeffd640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede01002c0 0x7fede019f630 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fedd4005940 tx=0x7fedd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7fedc7fff640 1 -- 172.21.15.18:0/2840741569 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd40044e0 con 0x7fede01002c0 2024-02-17T13:02:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7fede56f4640 1 -- 172.21.15.18:0/2840741569 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede01a6480 con 0x7fede01002c0 2024-02-17T13:02:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7fedc7fff640 1 -- 172.21.15.18:0/2840741569 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd402cbc0 con 0x7fede01002c0 2024-02-17T13:02:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7fedc7fff640 1 -- 172.21.15.18:0/2840741569 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4003e20 con 0x7fede01002c0 2024-02-17T13:02:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.946+0000 7fede56f4640 1 -- 172.21.15.18:0/2840741569 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede01a6990 con 0x7fede01002c0 2024-02-17T13:02:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.948+0000 7fedc7fff640 1 -- 172.21.15.18:0/2840741569 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fedd40035f0 con 0x7fede01002c0 2024-02-17T13:02:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7fede839c740 1 Processor -- start 2024-02-17T13:02:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7fede839c740 1 -- start start 2024-02-17T13:02:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7fedc7fff640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fedc80a22d0 0x7fedc80a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7fede839c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7fede839c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b1d60 0x55d27f6b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7fede839c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6bb2e0 0x55d27f6bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7fede839c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27f6209f0 con 0x55d27f6b2ad0 2024-02-17T13:02:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7fede839c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27f61df20 con 0x55d27f6bb2e0 2024-02-17T13:02:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7fede839c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27f619700 con 0x55d27f6b1d60 2024-02-17T13:02:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7feddf7fe640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fedc80a22d0 0x7fedc80a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7fedc7fff640 1 -- 172.21.15.18:0/2840741569 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fedd40f03e0 con 0x7fede01002c0 2024-02-17T13:02:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7feddeffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7feddeffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36080/0 (socket says 172.21.15.18:36080) 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7feddeffd640 1 -- 172.21.15.18:0/2000397827 learned_addr learned my addr 172.21.15.18:0/2000397827 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7fedde7fc640 1 --2- 172.21.15.18:0/2000397827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b1d60 0x55d27f6b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7feddf7fe640 1 --2- 172.21.15.18:0/2000397827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6bb2e0 0x55d27f6bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7feddeffd640 1 -- 172.21.15.18:0/2000397827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b1d60 msgr2=0x55d27f6b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7feddeffd640 1 --2- 172.21.15.18:0/2000397827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b1d60 0x55d27f6b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7feddeffd640 1 -- 172.21.15.18:0/2000397827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6bb2e0 msgr2=0x55d27f6bd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7feddeffd640 1 --2- 172.21.15.18:0/2000397827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6bb2e0 0x55d27f6bd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.949+0000 7feddeffd640 1 -- 172.21.15.18:0/2000397827 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d27f5af7b0 con 0x55d27f6b2ad0 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.950+0000 7feddf7fe640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fedc80a22d0 0x7fedc80a4790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fedd8005fd0 tx=0x7fedd8014a30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.950+0000 7feddeffd640 1 --2- 172.21.15.18:0/2000397827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b1820 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fedd40edc60 tx=0x7fedd4003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c5df908c9bf671 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.950+0000 7fedc57fa640 1 -- 172.21.15.18:0/2000397827 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd40cf510 con 0x55d27f6b2ad0 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.950+0000 7fedc57fa640 1 -- 172.21.15.18:0/2000397827 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd403de90 con 0x55d27f6b2ad0 2024-02-17T13:02:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.950+0000 7fedc57fa640 1 -- 172.21.15.18:0/2000397827 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd40c5640 con 0x55d27f6b2ad0 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.950+0000 7fede839c740 1 -- 172.21.15.18:0/2000397827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 msgr2=0x55d27f6b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.950+0000 7fede839c740 1 --2- 172.21.15.18:0/2000397827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b1820 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fedd40edc60 tx=0x7fedd4003920 comp rx=0 tx=0).stop 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.950+0000 7fede839c740 1 -- 172.21.15.18:0/2000397827 shutdown_connections 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.950+0000 7fede839c740 1 --2- 172.21.15.18:0/2000397827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6bb2e0 0x55d27f6bd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.950+0000 7fede839c740 1 --2- 172.21.15.18:0/2000397827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b1d60 0x55d27f6b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.950+0000 7fede839c740 1 --2- 172.21.15.18:0/2000397827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b1820 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.950+0000 7fede839c740 1 -- 172.21.15.18:0/2000397827 >> 172.21.15.18:0/2000397827 conn(0x55d27f679550 msgr2=0x55d27f6aca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fede839c740 1 -- 172.21.15.18:0/2000397827 shutdown_connections 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fede839c740 1 -- 172.21.15.18:0/2000397827 wait complete. 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fede839c740 1 -- start start 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fede839c740 1 Processor -- start 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fede839c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b1d60 0x55d27f6b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fede839c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fede839c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6bb2e0 0x55d27f6b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fede839c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27f5bebf0 con 0x55d27f6b2ad0 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fede839c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27f2ed250 con 0x55d27f6bb2e0 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fede839c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27f59f200 con 0x55d27f6b1d60 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fede839c740 10 client.0 _ll_register_callbacks cb 0x55d27f60dd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fedde7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fedde7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36092/0 (socket says 172.21.15.18:36092) 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7fedde7fc640 1 -- 172.21.15.18:0/21699956 learned_addr learned my addr 172.21.15.18:0/21699956 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7feddeffd640 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b1d60 0x55d27f6b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.951+0000 7feddf7fe640 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6bb2e0 0x55d27f6b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fedde7fc640 1 -- 172.21.15.18:0/21699956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b1d60 msgr2=0x55d27f6b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fedde7fc640 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b1d60 0x55d27f6b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fedde7fc640 1 -- 172.21.15.18:0/21699956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6bb2e0 msgr2=0x55d27f6b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fedde7fc640 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6bb2e0 0x55d27f6b8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fedde7fc640 1 -- 172.21.15.18:0/21699956 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede01a6990 con 0x55d27f6b2ad0 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7feddf7fe640 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6bb2e0 0x55d27f6b8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fedde7fc640 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b8840 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fedd000cfd0 tx=0x7fedd0011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbba9947606b39c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fedc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fedc2ff5640 1 -- 172.21.15.18:0/21699956 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd000fe70 con 0x55d27f6b2ad0 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fedc2ff5640 1 -- 172.21.15.18:0/21699956 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fedd0004020 con 0x55d27f6b2ad0 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fede839c740 1 -- 172.21.15.18:0/21699956 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d27f319600 con 0x55d27f6b2ad0 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fedc2ff5640 1 -- 172.21.15.18:0/21699956 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedd00041e0 con 0x55d27f6b2ad0 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fede839c740 10 client.20061 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fede839c740 1 -- 172.21.15.18:0/21699956 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55d27f617d80 con 0x55d27f6b2ad0 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fedc2ff5640 1 -- 172.21.15.18:0/21699956 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd0025080 con 0x55d27f6b2ad0 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fedc2ff5640 1 -- 172.21.15.18:0/21699956 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fedd0004b30 con 0x55d27f6b2ad0 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fede839c740 10 client.20061 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fede839c740 10 client.20061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.952+0000 7fede839c740 20 client.20061 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.953+0000 7fede839c740 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d27f6cb4d0 0x55d27f6cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.953+0000 7fede839c740 4 client.20061 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.953+0000 7fede839c740 1 -- 172.21.15.18:0/21699956 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d27f36a1b0 con 0x55d27f6cb4d0 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.953+0000 7feddeffd640 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d27f6cb4d0 0x55d27f6cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.953+0000 7feddeffd640 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d27f6cb4d0 0x55d27f6cd8f0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d13e01c7b6d90149 server_cookie=cebfdf32f2b1024e in_seq=0 out_seq=0 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.953+0000 7fedc2ff5640 10 client.20061 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fedc2ff5640 1 -- 172.21.15.18:0/21699956 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d27f36a1b0 con 0x55d27f6cb4d0 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fedc2ff5640 10 client.20061 handle_command_reply: tid=0 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fede839c740 1 client.20061 shutdown 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fede839c740 1 -- 172.21.15.18:0/21699956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 msgr2=0x55d27f6b8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fede839c740 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b8840 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fedd000cfd0 tx=0x7fedd0011c90 comp rx=0 tx=0).stop 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fede839c740 1 -- 172.21.15.18:0/21699956 shutdown_connections 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fede839c740 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d27f6cb4d0 0x55d27f6cd8f0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fede839c740 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6bb2e0 0x55d27f6b8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fede839c740 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b8840 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fede839c740 1 --2- 172.21.15.18:0/21699956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b1d60 0x55d27f6b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fedc2ff5640 0 client.20061 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fede839c740 1 -- 172.21.15.18:0/21699956 >> 172.21.15.18:0/21699956 conn(0x55d27f679550 msgr2=0x55d27f6ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fede839c740 1 -- 172.21.15.18:0/21699956 shutdown_connections 2024-02-17T13:02:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fede839c740 1 -- 172.21.15.18:0/21699956 wait complete. 2024-02-17T13:02:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.954+0000 7fede839c740 20 client.20061 trim_cache size 0 max 16384 2024-02-17T13:02:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.967+0000 7fede839c740 1 Processor -- start 2024-02-17T13:02:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.967+0000 7fede839c740 1 -- start start 2024-02-17T13:02:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.967+0000 7fede839c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6b1d60 0x55d27f6baa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.967+0000 7fede839c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.967+0000 7fede839c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6bb2e0 0x55d27f6b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.967+0000 7fede839c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27f6b3f50 con 0x55d27f6b2ad0 2024-02-17T13:02:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.967+0000 7fede839c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27f6adc10 con 0x55d27f6b1d60 2024-02-17T13:02:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.967+0000 7fede839c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27f6ada80 con 0x55d27f6bb2e0 2024-02-17T13:02:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.967+0000 7fedde7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.967+0000 7fedde7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36100/0 (socket says 172.21.15.18:36100) 2024-02-17T13:02:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.967+0000 7fedde7fc640 1 -- 172.21.15.18:0/4179538855 learned_addr learned my addr 172.21.15.18:0/4179538855 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7feddf7fe640 1 --2- 172.21.15.18:0/4179538855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6bb2e0 0x55d27f6b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7fedde7fc640 1 -- 172.21.15.18:0/4179538855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6bb2e0 msgr2=0x55d27f6b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7fedde7fc640 1 --2- 172.21.15.18:0/4179538855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6bb2e0 0x55d27f6b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7feddeffd640 1 --2- 172.21.15.18:0/4179538855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6b1d60 0x55d27f6baa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7fedde7fc640 1 -- 172.21.15.18:0/4179538855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6b1d60 msgr2=0x55d27f6baa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7fedde7fc640 1 --2- 172.21.15.18:0/4179538855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6b1d60 0x55d27f6baa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7fedde7fc640 1 -- 172.21.15.18:0/4179538855 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d27f5b47b0 con 0x55d27f6b2ad0 2024-02-17T13:02:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7feddf7fe640 1 --2- 172.21.15.18:0/4179538855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6bb2e0 0x55d27f6b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7feddeffd640 1 --2- 172.21.15.18:0/4179538855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6b1d60 0x55d27f6baa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7fedde7fc640 1 --2- 172.21.15.18:0/4179538855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b8ba0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fedd0004f60 tx=0x7fedd000f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e523e2159938820 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7fedc4ff9640 1 -- 172.21.15.18:0/4179538855 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd0003890 con 0x55d27f6b2ad0 2024-02-17T13:02:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7fedc4ff9640 1 -- 172.21.15.18:0/4179538855 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d27f319600 con 0x55d27f6b2ad0 2024-02-17T13:02:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.968+0000 7fedc4ff9640 1 -- 172.21.15.18:0/4179538855 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd0005240 con 0x55d27f6b2ad0 2024-02-17T13:02:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 -- 172.21.15.18:0/4179538855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 msgr2=0x55d27f6b8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 --2- 172.21.15.18:0/4179538855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b8ba0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fedd0004f60 tx=0x7fedd000f840 comp rx=0 tx=0).stop 2024-02-17T13:02:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 -- 172.21.15.18:0/4179538855 shutdown_connections 2024-02-17T13:02:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 --2- 172.21.15.18:0/4179538855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6bb2e0 0x55d27f6b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 --2- 172.21.15.18:0/4179538855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6b2ad0 0x55d27f6b8ba0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 --2- 172.21.15.18:0/4179538855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6b1d60 0x55d27f6baa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 -- 172.21.15.18:0/4179538855 >> 172.21.15.18:0/4179538855 conn(0x55d27f679550 msgr2=0x55d27f6b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 -- 172.21.15.18:0/4179538855 shutdown_connections 2024-02-17T13:02:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 -- 172.21.15.18:0/4179538855 wait complete. 2024-02-17T13:02:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 -- start start 2024-02-17T13:02:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 Processor -- start 2024-02-17T13:02:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6b1d60 0x55d27f6c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b2ad0 0x55d27f6c1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6bb2e0 0x55d27f7219e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27f6c1940 con 0x55d27f6bb2e0 2024-02-17T13:02:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27f6c1ab0 con 0x55d27f6b1d60 2024-02-17T13:02:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27f721fa0 con 0x55d27f6b2ad0 2024-02-17T13:02:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fede839c740 10 client.0 _ll_register_callbacks cb 0x7fedd402cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7feddf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6bb2e0 0x55d27f7219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7feddf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6bb2e0 0x55d27f7219e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36102/0 (socket says 172.21.15.18:36102) 2024-02-17T13:02:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7feddf7fe640 1 -- 172.21.15.18:0/196862834 learned_addr learned my addr 172.21.15.18:0/196862834 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7feddeffd640 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6b1d60 0x55d27f6c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.969+0000 7fedde7fc640 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b2ad0 0x55d27f6c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7feddf7fe640 1 -- 172.21.15.18:0/196862834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b2ad0 msgr2=0x55d27f6c1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7feddf7fe640 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b2ad0 0x55d27f6c1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7feddf7fe640 1 -- 172.21.15.18:0/196862834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6b1d60 msgr2=0x55d27f6c0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7feddf7fe640 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6b1d60 0x55d27f6c0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7feddf7fe640 1 -- 172.21.15.18:0/196862834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d27f617d80 con 0x55d27f6bb2e0 2024-02-17T13:02:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7feddeffd640 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6b1d60 0x55d27f6c0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7feddf7fe640 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6bb2e0 0x55d27f7219e0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fedd8014f30 tx=0x7fedd8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e901a18733ea89 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fedc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fedc2ff5640 1 -- 172.21.15.18:0/196862834 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8012d60 con 0x55d27f6bb2e0 2024-02-17T13:02:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fedc2ff5640 1 -- 172.21.15.18:0/196862834 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fedd8021480 con 0x55d27f6bb2e0 2024-02-17T13:02:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fede839c740 1 -- 172.21.15.18:0/196862834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedd403de90 con 0x55d27f6bb2e0 2024-02-17T13:02:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fedc2ff5640 1 -- 172.21.15.18:0/196862834 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedd8021640 con 0x55d27f6bb2e0 2024-02-17T13:02:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fede839c740 10 client.20067 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fede839c740 1 -- 172.21.15.18:0/196862834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55d27f5da540 con 0x55d27f6bb2e0 2024-02-17T13:02:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fedc2ff5640 1 -- 172.21.15.18:0/196862834 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd403de90 con 0x55d27f6bb2e0 2024-02-17T13:02:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fedc2ff5640 1 -- 172.21.15.18:0/196862834 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fedd800b050 con 0x55d27f6bb2e0 2024-02-17T13:02:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fede839c740 10 client.20067 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fede839c740 10 client.20067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fede839c740 20 client.20067 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.970+0000 7fede839c740 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d27f723770 0x55d27f725b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.971+0000 7fede839c740 4 client.20067 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:02:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.971+0000 7fede839c740 1 -- 172.21.15.18:0/196862834 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d27f356550 con 0x55d27f723770 2024-02-17T13:02:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.971+0000 7feddeffd640 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d27f723770 0x55d27f725b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.971+0000 7fedc2ff5640 10 client.20067 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:21.971+0000 7feddeffd640 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d27f723770 0x55d27f725b90 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9551aeabf99c57 server_cookie=57de6b61b73f910f in_seq=0 out_seq=0 2024-02-17T13:02:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fedc2ff5640 1 -- 172.21.15.18:0/196862834 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d27f356550 con 0x55d27f723770 2024-02-17T13:02:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fedc2ff5640 10 client.20067 handle_command_reply: tid=0 2024-02-17T13:02:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fede839c740 1 client.20067 shutdown 2024-02-17T13:02:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fede839c740 1 -- 172.21.15.18:0/196862834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6bb2e0 msgr2=0x55d27f7219e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fede839c740 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6bb2e0 0x55d27f7219e0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fedd8014f30 tx=0x7fedd8004ab0 comp rx=0 tx=0).stop 2024-02-17T13:02:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fede839c740 1 -- 172.21.15.18:0/196862834 shutdown_connections 2024-02-17T13:02:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fede839c740 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d27f723770 0x55d27f725b90 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fede839c740 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d27f6bb2e0 0x55d27f7219e0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fede839c740 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d27f6b2ad0 0x55d27f6c1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fedc2ff5640 0 client.20067 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fede839c740 1 --2- 172.21.15.18:0/196862834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27f6b1d60 0x55d27f6c0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fede839c740 1 -- 172.21.15.18:0/196862834 >> 172.21.15.18:0/196862834 conn(0x55d27f679550 msgr2=0x55d27f6adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fede839c740 1 -- 172.21.15.18:0/196862834 shutdown_connections 2024-02-17T13:02:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fede839c740 1 -- 172.21.15.18:0/196862834 wait complete. 2024-02-17T13:02:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.014+0000 7fede839c740 20 client.20067 trim_cache size 0 max 16384 2024-02-17T13:02:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:02:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:02:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:02:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:02:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.015+0000 7fede56f4640 1 -- 172.21.15.18:0/2840741569 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fedc80a22d0 msgr2=0x7fedc80a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.015+0000 7fede56f4640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fedc80a22d0 0x7fedc80a4790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fedd8005fd0 tx=0x7fedd8014a30 comp rx=0 tx=0).stop 2024-02-17T13:02:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.015+0000 7fede56f4640 1 -- 172.21.15.18:0/2840741569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede01002c0 msgr2=0x7fede019f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.015+0000 7fede56f4640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede01002c0 0x7fede019f630 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fedd4005940 tx=0x7fedd40042f0 comp rx=0 tx=0).stop 2024-02-17T13:02:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.015+0000 7fede56f4640 1 -- 172.21.15.18:0/2840741569 shutdown_connections 2024-02-17T13:02:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.015+0000 7fede56f4640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fedc80a22d0 0x7fedc80a4790 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.015+0000 7fede56f4640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fede01997a0 0x7fede0199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.015+0000 7fede56f4640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fede0105390 0x7fede019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.015+0000 7fede56f4640 1 --2- 172.21.15.18:0/2840741569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede01002c0 0x7fede019f630 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.015+0000 7fede56f4640 1 -- 172.21.15.18:0/2840741569 >> 172.21.15.18:0/2840741569 conn(0x7fede00771b0 msgr2=0x7fede010bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.015+0000 7fede56f4640 1 -- 172.21.15.18:0/2840741569 shutdown_connections 2024-02-17T13:02:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:22.015+0000 7fede56f4640 1 -- 172.21.15.18:0/2840741569 wait complete. 2024-02-17T13:02:22.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:21 smithi146 ceph-mon[36454]: pgmap v392: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-02-17T13:02:22.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:21 smithi018 ceph-mon[32625]: pgmap v392: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-02-17T13:02:22.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 623219 4.86 MB/sec execute 577 sec latency 1284.602 ms 2024-02-17T13:02:22.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:21 smithi119 ceph-mon[36188]: pgmap v392: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-02-17T13:02:23.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 624123 4.86 MB/sec execute 578 sec latency 1846.598 ms 2024-02-17T13:02:24.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:23 smithi146 ceph-mon[36454]: pgmap v393: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-02-17T13:02:24.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:23 smithi018 ceph-mon[32625]: pgmap v393: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-02-17T13:02:24.302 INFO:tasks.workunit.client.0.smithi018.stdout: 1 625696 4.87 MB/sec execute 579 sec latency 13.505 ms 2024-02-17T13:02:24.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:23 smithi119 ceph-mon[36188]: pgmap v393: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-02-17T13:02:25.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 626768 4.87 MB/sec execute 580 sec latency 394.534 ms 2024-02-17T13:02:26.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:25 smithi146 ceph-mon[36454]: pgmap v394: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-02-17T13:02:26.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:25 smithi018 ceph-mon[32625]: pgmap v394: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-02-17T13:02:26.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 626768 4.86 MB/sec execute 581 sec latency 1394.631 ms 2024-02-17T13:02:26.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:25 smithi119 ceph-mon[36188]: pgmap v394: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-02-17T13:02:27.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 626768 4.85 MB/sec execute 582 sec latency 2394.746 ms 2024-02-17T13:02:28.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:27 smithi146 ceph-mon[36454]: pgmap v395: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.9 MiB/s wr, 179 op/s 2024-02-17T13:02:28.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:27 smithi018 ceph-mon[32625]: pgmap v395: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.9 MiB/s wr, 179 op/s 2024-02-17T13:02:28.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 626768 4.85 MB/sec execute 583 sec latency 3394.852 ms 2024-02-17T13:02:28.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:27 smithi119 ceph-mon[36188]: pgmap v395: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.9 MiB/s wr, 179 op/s 2024-02-17T13:02:29.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 628626 4.86 MB/sec execute 584 sec latency 3577.853 ms 2024-02-17T13:02:30.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:29 smithi146 ceph-mon[36454]: pgmap v396: 97 pgs: 97 active+clean; 253 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-02-17T13:02:30.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:29 smithi018 ceph-mon[32625]: pgmap v396: 97 pgs: 97 active+clean; 253 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-02-17T13:02:30.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 630027 4.86 MB/sec execute 585 sec latency 16.329 ms 2024-02-17T13:02:30.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:29 smithi119 ceph-mon[36188]: pgmap v396: 97 pgs: 97 active+clean; 253 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-02-17T13:02:31.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:30 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:02:31.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:30 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:02:31.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:30 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:02:31.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:30 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:02:31.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:30 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:02:31.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:30 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:02:31.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 630309 4.85 MB/sec execute 586 sec latency 788.204 ms 2024-02-17T13:02:31.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:30 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:02:31.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:30 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:02:31.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:30 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:02:32.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:31 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:02:32.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:31 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:02:32.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:31 smithi146 ceph-mon[36454]: pgmap v397: 97 pgs: 97 active+clean; 253 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-02-17T13:02:32.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:31 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:02:32.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:31 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:02:32.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:31 smithi018 ceph-mon[32625]: pgmap v397: 97 pgs: 97 active+clean; 253 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-02-17T13:02:32.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 630309 4.84 MB/sec execute 587 sec latency 1788.237 ms 2024-02-17T13:02:32.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:31 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:02:32.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:31 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:02:32.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:31 smithi119 ceph-mon[36188]: pgmap v397: 97 pgs: 97 active+clean; 253 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-02-17T13:02:33.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 630309 4.84 MB/sec execute 588 sec latency 2788.349 ms 2024-02-17T13:02:34.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:33 smithi146 ceph-mon[36454]: pgmap v398: 97 pgs: 97 active+clean; 253 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-02-17T13:02:34.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:33 smithi018 ceph-mon[32625]: pgmap v398: 97 pgs: 97 active+clean; 253 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-02-17T13:02:34.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 631179 4.83 MB/sec execute 589 sec latency 3389.637 ms 2024-02-17T13:02:34.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:33 smithi119 ceph-mon[36188]: pgmap v398: 97 pgs: 97 active+clean; 253 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-02-17T13:02:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.106+0000 7f6ea28ff640 1 -- 172.21.15.18:0/3697315329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e9c068800 msgr2=0x7f6e9c068c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.106+0000 7f6ea28ff640 1 --2- 172.21.15.18:0/3697315329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e9c068800 0x7f6e9c068c60 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009900 tx=0x7f6e9002bcd0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.106+0000 7f6ea28ff640 1 -- 172.21.15.18:0/3697315329 shutdown_connections 2024-02-17T13:02:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.106+0000 7f6ea28ff640 1 --2- 172.21.15.18:0/3697315329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e9c0691a0 0x7f6e9c10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.106+0000 7f6ea28ff640 1 --2- 172.21.15.18:0/3697315329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e9c068800 0x7f6e9c068c60 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.106+0000 7f6ea28ff640 1 --2- 172.21.15.18:0/3697315329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e9c107270 0x7f6e9c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.106+0000 7f6ea28ff640 1 -- 172.21.15.18:0/3697315329 >> 172.21.15.18:0/3697315329 conn(0x7f6e9c076ed0 msgr2=0x7f6e9c0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.106+0000 7f6ea28ff640 1 -- 172.21.15.18:0/3697315329 shutdown_connections 2024-02-17T13:02:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.106+0000 7f6ea28ff640 1 -- 172.21.15.18:0/3697315329 wait complete. 2024-02-17T13:02:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.106+0000 7f6ea28ff640 1 Processor -- start 2024-02-17T13:02:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6ea28ff640 1 -- start start 2024-02-17T13:02:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6ea28ff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e9c068800 0x7f6e9c197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6ea28ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e9c0691a0 0x7f6e9c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6ea28ff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e9c107270 0x7f6e9c19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6ea28ff640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c19c510 con 0x7f6e9c068800 2024-02-17T13:02:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6ea28ff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c19c680 con 0x7f6e9c0691a0 2024-02-17T13:02:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6ea28ff640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c19c840 con 0x7f6e9c107270 2024-02-17T13:02:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6e9bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e9c068800 0x7f6e9c197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6e9bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e9c068800 0x7f6e9c197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47518/0 (socket says 172.21.15.18:47518) 2024-02-17T13:02:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6e9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e9c0691a0 0x7f6e9c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6e9bfff640 1 -- 172.21.15.18:0/2869027686 learned_addr learned my addr 172.21.15.18:0/2869027686 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6ea0e9c640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e9c107270 0x7f6e9c19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6e9bfff640 1 -- 172.21.15.18:0/2869027686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e9c107270 msgr2=0x7f6e9c19bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e9c107270 0x7f6e9c19bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6e9bfff640 1 -- 172.21.15.18:0/2869027686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e9c0691a0 msgr2=0x7f6e9c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e9c0691a0 0x7f6e9c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6e9bfff640 1 -- 172.21.15.18:0/2869027686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c009490 con 0x7f6e9c068800 2024-02-17T13:02:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.107+0000 7f6e9b7fe640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e9c0691a0 0x7f6e9c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.108+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e9c068800 0x7f6e9c197650 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00ea60 tx=0x7f6e8c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.108+0000 7f6e997fa640 1 -- 172.21.15.18:0/2869027686 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c00cde0 con 0x7f6e9c068800 2024-02-17T13:02:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.108+0000 7f6ea28ff640 1 -- 172.21.15.18:0/2869027686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e90009560 con 0x7f6e9c068800 2024-02-17T13:02:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.108+0000 7f6e997fa640 1 -- 172.21.15.18:0/2869027686 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e8c0035f0 con 0x7f6e9c068800 2024-02-17T13:02:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.108+0000 7f6e997fa640 1 -- 172.21.15.18:0/2869027686 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c010690 con 0x7f6e9c068800 2024-02-17T13:02:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.108+0000 7f6ea28ff640 1 -- 172.21.15.18:0/2869027686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9c1a41c0 con 0x7f6e9c068800 2024-02-17T13:02:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.110+0000 7f6e997fa640 1 -- 172.21.15.18:0/2869027686 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6e8c003af0 con 0x7f6e9c068800 2024-02-17T13:02:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.110+0000 7f6ea55aa740 1 Processor -- start 2024-02-17T13:02:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.110+0000 7f6ea55aa740 1 -- start start 2024-02-17T13:02:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.110+0000 7f6e997fa640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6e840a22d0 0x7f6e840a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6e997fa640 1 -- 172.21.15.18:0/2869027686 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6e8c014070 con 0x7f6e9c068800 2024-02-17T13:02:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6ea55aa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd19673820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6ea55aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd196741e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6ea0e9c640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6e840a22d0 0x7f6e840a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6ea55aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd1967d2e0 0x55cd1967f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6ea55aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd195e29f0 con 0x55cd1967d2e0 2024-02-17T13:02:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6ea55aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd195dff20 con 0x55cd19673d60 2024-02-17T13:02:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6ea55aa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd195db700 con 0x55cd19674ad0 2024-02-17T13:02:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6e9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd196741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6ea0e9c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd1967d2e0 0x55cd1967f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6e9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd196741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:43862/0 (socket says 172.21.15.18:43862) 2024-02-17T13:02:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6e9b7fe640 1 -- 172.21.15.18:0/575901702 learned_addr learned my addr 172.21.15.18:0/575901702 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6e9bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd19673820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6e9b7fe640 1 -- 172.21.15.18:0/575901702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 msgr2=0x55cd19673820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6e9b7fe640 1 --2- 172.21.15.18:0/575901702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd19673820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6e9b7fe640 1 -- 172.21.15.18:0/575901702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd1967d2e0 msgr2=0x55cd1967f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6e9b7fe640 1 --2- 172.21.15.18:0/575901702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd1967d2e0 0x55cd1967f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6ea0e9c640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6e840a22d0 0x7f6e840a4790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6e94005fd0 tx=0x7f6e94013b60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6e9b7fe640 1 -- 172.21.15.18:0/575901702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd195717b0 con 0x55cd19673d60 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/575901702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd19673820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.111+0000 7f6e9b7fe640 1 --2- 172.21.15.18:0/575901702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd196741e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009900 tx=0x7f6e9002c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84249f2470080124 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.112+0000 7f6e827fc640 1 -- 172.21.15.18:0/575901702 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90004af0 con 0x55cd19673d60 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.112+0000 7f6e827fc640 1 -- 172.21.15.18:0/575901702 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e90004040 con 0x55cd19673d60 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.112+0000 7f6e827fc640 1 -- 172.21.15.18:0/575901702 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e900053e0 con 0x55cd19673d60 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.112+0000 7f6ea55aa740 1 -- 172.21.15.18:0/575901702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 msgr2=0x55cd196741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.112+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/575901702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd196741e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009900 tx=0x7f6e9002c720 comp rx=0 tx=0).stop 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.112+0000 7f6ea55aa740 1 -- 172.21.15.18:0/575901702 shutdown_connections 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.112+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/575901702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd1967d2e0 0x55cd1967f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.112+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/575901702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd196741e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.112+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/575901702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd19673820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.112+0000 7f6ea55aa740 1 -- 172.21.15.18:0/575901702 >> 172.21.15.18:0/575901702 conn(0x55cd1963b550 msgr2=0x55cd1966ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6ea55aa740 1 -- 172.21.15.18:0/575901702 shutdown_connections 2024-02-17T13:02:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6ea55aa740 1 -- 172.21.15.18:0/575901702 wait complete. 2024-02-17T13:02:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6ea55aa740 1 -- start start 2024-02-17T13:02:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6ea55aa740 1 Processor -- start 2024-02-17T13:02:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6ea55aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd1967a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6ea55aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19674ad0 0x55cd1967a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6ea55aa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd1967d2e0 0x55cd1967ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6ea55aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd19580bf0 con 0x55cd19674ad0 2024-02-17T13:02:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6ea55aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd192af250 con 0x55cd19673d60 2024-02-17T13:02:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6ea55aa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd19561200 con 0x55cd1967d2e0 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6ea55aa740 10 client.0 _ll_register_callbacks cb 0x55cd195cfd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6e9b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19674ad0 0x55cd1967a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6e9b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19674ad0 0x55cd1967a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47536/0 (socket says 172.21.15.18:47536) 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.113+0000 7f6e9b7fe640 1 -- 172.21.15.18:0/3715086272 learned_addr learned my addr 172.21.15.18:0/3715086272 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd1967a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6ea0e9c640 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd1967d2e0 0x55cd1967ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6e9b7fe640 1 -- 172.21.15.18:0/3715086272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd1967d2e0 msgr2=0x55cd1967ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6e9b7fe640 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd1967d2e0 0x55cd1967ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6e9b7fe640 1 -- 172.21.15.18:0/3715086272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 msgr2=0x55cd1967a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6e9b7fe640 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd1967a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6e9b7fe640 1 -- 172.21.15.18:0/3715086272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9c10e850 con 0x55cd19674ad0 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd1967a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6e9b7fe640 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19674ad0 0x55cd1967a840 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f6e9002cab0 tx=0x7f6e9002cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4037da0324de5ad server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6e7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6e7b7fe640 1 -- 172.21.15.18:0/3715086272 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90004af0 con 0x55cd19674ad0 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6e7b7fe640 1 -- 172.21.15.18:0/3715086272 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6e90004df0 con 0x55cd19674ad0 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6ea55aa740 1 -- 172.21.15.18:0/3715086272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd192db600 con 0x55cd19674ad0 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6e7b7fe640 1 -- 172.21.15.18:0/3715086272 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e9002cd00 con 0x55cd19674ad0 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6ea55aa740 10 client.20079 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.114+0000 7f6ea55aa740 1 -- 172.21.15.18:0/3715086272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55cd195d9d80 con 0x55cd19674ad0 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.115+0000 7f6e7b7fe640 1 -- 172.21.15.18:0/3715086272 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cd192db600 con 0x55cd19674ad0 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.115+0000 7f6e7b7fe640 1 -- 172.21.15.18:0/3715086272 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6e9003c050 con 0x55cd19674ad0 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.115+0000 7f6ea55aa740 10 client.20079 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.115+0000 7f6ea55aa740 10 client.20079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.115+0000 7f6ea55aa740 20 client.20079 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.115+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd1968d090 0x55cd1968f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.115+0000 7f6ea55aa740 4 client.20079 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.115+0000 7f6ea55aa740 1 -- 172.21.15.18:0/3715086272 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd1932c1b0 con 0x55cd1968d090 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.115+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd1968d090 0x55cd1968f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.115+0000 7f6e7b7fe640 10 client.20079 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.115+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd1968d090 0x55cd1968f480 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b788997ff40db5b server_cookie=f2911eed60ca441a in_seq=0 out_seq=0 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6e7b7fe640 1 -- 172.21.15.18:0/3715086272 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd1932c1b0 con 0x55cd1968d090 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6e7b7fe640 10 client.20079 handle_command_reply: tid=0 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6ea55aa740 1 client.20079 shutdown 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6ea55aa740 1 -- 172.21.15.18:0/3715086272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19674ad0 msgr2=0x55cd1967a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19674ad0 0x55cd1967a840 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f6e9002cab0 tx=0x7f6e9002cae0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6ea55aa740 1 -- 172.21.15.18:0/3715086272 shutdown_connections 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd1968d090 0x55cd1968f480 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd1967d2e0 0x55cd1967ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19674ad0 0x55cd1967a840 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6e7b7fe640 0 client.20079 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/3715086272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd1967a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6ea55aa740 1 -- 172.21.15.18:0/3715086272 >> 172.21.15.18:0/3715086272 conn(0x55cd1963b550 msgr2=0x55cd196707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6ea55aa740 1 -- 172.21.15.18:0/3715086272 shutdown_connections 2024-02-17T13:02:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6ea55aa740 1 -- 172.21.15.18:0/3715086272 wait complete. 2024-02-17T13:02:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.117+0000 7f6ea55aa740 20 client.20079 trim_cache size 0 max 16384 2024-02-17T13:02:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6ea55aa740 1 Processor -- start 2024-02-17T13:02:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6ea55aa740 1 -- start start 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6ea55aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19673d60 0x55cd1967ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6ea55aa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd1967aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6ea55aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd1967d2e0 0x55cd19683f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6ea55aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd1966f850 con 0x55cd19673d60 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6ea55aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd19675f50 con 0x55cd1967d2e0 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6ea55aa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd1966a140 con 0x55cd19674ad0 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6e9bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19673d60 0x55cd1967ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6e9bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19673d60 0x55cd1967ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47550/0 (socket says 172.21.15.18:47550) 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6e9bfff640 1 -- 172.21.15.18:0/1104845037 learned_addr learned my addr 172.21.15.18:0/1104845037 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6ea0e9c640 1 --2- 172.21.15.18:0/1104845037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd1967d2e0 0x55cd19683f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6e9b7fe640 1 --2- 172.21.15.18:0/1104845037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd1967aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6e9bfff640 1 -- 172.21.15.18:0/1104845037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 msgr2=0x55cd1967aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/1104845037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd1967aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6e9bfff640 1 -- 172.21.15.18:0/1104845037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd1967d2e0 msgr2=0x55cd19683f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/1104845037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd1967d2e0 0x55cd19683f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.131+0000 7f6e9bfff640 1 -- 172.21.15.18:0/1104845037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd195767b0 con 0x55cd19673d60 2024-02-17T13:02:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.132+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/1104845037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19673d60 0x55cd1967ca30 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00c760 tx=0x7f6e8c0971b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856d577a85614d51 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.132+0000 7f6e81ffb640 1 -- 172.21.15.18:0/1104845037 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c0a36b0 con 0x55cd19673d60 2024-02-17T13:02:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.132+0000 7f6e81ffb640 1 -- 172.21.15.18:0/1104845037 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e8c098fb0 con 0x55cd19673d60 2024-02-17T13:02:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.132+0000 7f6e81ffb640 1 -- 172.21.15.18:0/1104845037 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c097580 con 0x55cd19673d60 2024-02-17T13:02:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1104845037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19673d60 msgr2=0x55cd1967ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/1104845037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19673d60 0x55cd1967ca30 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00c760 tx=0x7f6e8c0971b0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1104845037 shutdown_connections 2024-02-17T13:02:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/1104845037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd1967d2e0 0x55cd19683f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/1104845037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd1967aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/1104845037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd19673d60 0x55cd1967ca30 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1104845037 >> 172.21.15.18:0/1104845037 conn(0x55cd1963b550 msgr2=0x55cd1966e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1104845037 shutdown_connections 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1104845037 wait complete. 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 -- start start 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 Processor -- start 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd19683f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd196844d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd1967d2e0 0x55cd19676b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd19677250 con 0x55cd1967d2e0 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd196773c0 con 0x55cd19673d60 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd19677580 con 0x55cd19674ad0 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.133+0000 7f6ea55aa740 10 client.0 _ll_register_callbacks cb 0x7f6e940093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6ea0e9c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd1967d2e0 0x55cd19676b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6e9bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd19683f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6ea0e9c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd1967d2e0 0x55cd19676b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47564/0 (socket says 172.21.15.18:47564) 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6ea0e9c640 1 -- 172.21.15.18:0/1003303718 learned_addr learned my addr 172.21.15.18:0/1003303718 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6e9b7fe640 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd196844d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6ea0e9c640 1 -- 172.21.15.18:0/1003303718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 msgr2=0x55cd196844d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6ea0e9c640 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd196844d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6ea0e9c640 1 -- 172.21.15.18:0/1003303718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 msgr2=0x55cd19683f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6ea0e9c640 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd19683f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6ea0e9c640 1 -- 172.21.15.18:0/1003303718 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e90009560 con 0x55cd1967d2e0 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd19683f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6ea0e9c640 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd1967d2e0 0x55cd19676b80 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009fd0 tx=0x7f6e94004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4168bf7d8245dc78 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6e7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6e7b7fe640 1 -- 172.21.15.18:0/1003303718 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94025070 con 0x55cd1967d2e0 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6e7b7fe640 1 -- 172.21.15.18:0/1003303718 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6e94003e00 con 0x55cd1967d2e0 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1003303718 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e90004040 con 0x55cd1967d2e0 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6e7b7fe640 1 -- 172.21.15.18:0/1003303718 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e94004b20 con 0x55cd1967d2e0 2024-02-17T13:02:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6ea55aa740 10 client.20085 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.134+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1003303718 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55cd1959c540 con 0x55cd1967d2e0 2024-02-17T13:02:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.135+0000 7f6e7b7fe640 1 -- 172.21.15.18:0/1003303718 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e94038020 con 0x55cd1967d2e0 2024-02-17T13:02:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.135+0000 7f6e7b7fe640 1 -- 172.21.15.18:0/1003303718 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6e94034880 con 0x55cd1967d2e0 2024-02-17T13:02:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.135+0000 7f6ea55aa740 10 client.20085 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.135+0000 7f6ea55aa740 10 client.20085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.135+0000 7f6ea55aa740 20 client.20085 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.135+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd196e4970 0x55cd196e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.135+0000 7f6ea55aa740 4 client.20085 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:02:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.135+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1003303718 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cd19318550 con 0x55cd196e4970 2024-02-17T13:02:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.135+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd196e4970 0x55cd196e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.135+0000 7f6e7b7fe640 10 client.20085 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.135+0000 7f6e9bfff640 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd196e4970 0x55cd196e6d90 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecda73cf406c2bec server_cookie=d4cc6a349d72fb71 in_seq=0 out_seq=0 2024-02-17T13:02:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.170+0000 7f6e7b7fe640 1 -- 172.21.15.18:0/1003303718 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cd19318550 con 0x55cd196e4970 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.170+0000 7f6e7b7fe640 10 client.20085 handle_command_reply: tid=0 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.170+0000 7f6ea55aa740 1 client.20085 shutdown 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.170+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1003303718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd1967d2e0 msgr2=0x55cd19676b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.170+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd1967d2e0 0x55cd19676b80 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009fd0 tx=0x7f6e94004ab0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.170+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1003303718 shutdown_connections 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.170+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd196e4970 0x55cd196e6d90 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.170+0000 7f6e7b7fe640 0 client.20085 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.170+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd1967d2e0 0x55cd19676b80 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.170+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd19674ad0 0x55cd196844d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.170+0000 7f6ea55aa740 1 --2- 172.21.15.18:0/1003303718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd19673d60 0x55cd19683f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.170+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1003303718 >> 172.21.15.18:0/1003303718 conn(0x55cd1963b550 msgr2=0x55cd19678b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1003303718 shutdown_connections 2024-02-17T13:02:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea55aa740 1 -- 172.21.15.18:0/1003303718 wait complete. 2024-02-17T13:02:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea55aa740 20 client.20085 trim_cache size 0 max 16384 2024-02-17T13:02:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:02:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:02:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:02:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:02:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea28ff640 1 -- 172.21.15.18:0/2869027686 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6e840a22d0 msgr2=0x7f6e840a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea28ff640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6e840a22d0 0x7f6e840a4790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6e94005fd0 tx=0x7f6e94013b60 comp rx=0 tx=0).stop 2024-02-17T13:02:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea28ff640 1 -- 172.21.15.18:0/2869027686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e9c068800 msgr2=0x7f6e9c197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea28ff640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e9c068800 0x7f6e9c197650 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00ea60 tx=0x7f6e8c00ef20 comp rx=0 tx=0).stop 2024-02-17T13:02:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea28ff640 1 -- 172.21.15.18:0/2869027686 shutdown_connections 2024-02-17T13:02:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea28ff640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6e840a22d0 0x7f6e840a4790 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea28ff640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e9c107270 0x7f6e9c19bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea28ff640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e9c0691a0 0x7f6e9c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea28ff640 1 --2- 172.21.15.18:0/2869027686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e9c068800 0x7f6e9c197650 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.171+0000 7f6ea28ff640 1 -- 172.21.15.18:0/2869027686 >> 172.21.15.18:0/2869027686 conn(0x7f6e9c076ed0 msgr2=0x7f6e9c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.172+0000 7f6ea28ff640 1 -- 172.21.15.18:0/2869027686 shutdown_connections 2024-02-17T13:02:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:35.172+0000 7f6ea28ff640 1 -- 172.21.15.18:0/2869027686 wait complete. 2024-02-17T13:02:35.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 632724 4.84 MB/sec execute 590 sec latency 26.470 ms 2024-02-17T13:02:36.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:35 smithi146 ceph-mon[36454]: pgmap v399: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-02-17T13:02:36.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:35 smithi018 ceph-mon[32625]: pgmap v399: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-02-17T13:02:36.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 633858 4.84 MB/sec execute 591 sec latency 358.226 ms 2024-02-17T13:02:36.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:35 smithi119 ceph-mon[36188]: pgmap v399: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-02-17T13:02:37.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 633858 4.84 MB/sec execute 592 sec latency 1358.325 ms 2024-02-17T13:02:38.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:37 smithi146 ceph-mon[36454]: pgmap v400: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-02-17T13:02:38.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:37 smithi018 ceph-mon[32625]: pgmap v400: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-02-17T13:02:38.304 INFO:tasks.workunit.client.0.smithi018.stdout: 1 633858 4.83 MB/sec execute 593 sec latency 2358.426 ms 2024-02-17T13:02:38.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:37 smithi119 ceph-mon[36188]: pgmap v400: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-02-17T13:02:39.304 INFO:tasks.workunit.client.0.smithi018.stdout: 1 634742 4.82 MB/sec execute 594 sec latency 2965.594 ms 2024-02-17T13:02:40.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:39 smithi146 ceph-mon[36454]: pgmap v401: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 1.7 MiB/s wr, 165 op/s 2024-02-17T13:02:40.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:39 smithi018 ceph-mon[32625]: pgmap v401: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 1.7 MiB/s wr, 165 op/s 2024-02-17T13:02:40.303 INFO:tasks.workunit.client.0.smithi018.stdout: 1 636431 4.84 MB/sec execute 595 sec latency 14.760 ms 2024-02-17T13:02:40.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:39 smithi119 ceph-mon[36188]: pgmap v401: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 1.7 MiB/s wr, 165 op/s 2024-02-17T13:02:41.304 INFO:tasks.workunit.client.0.smithi018.stdout: 1 637402 4.84 MB/sec execute 596 sec latency 473.519 ms 2024-02-17T13:02:42.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:41 smithi146 ceph-mon[36454]: pgmap v402: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-02-17T13:02:42.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:41 smithi018 ceph-mon[32625]: pgmap v402: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-02-17T13:02:42.304 INFO:tasks.workunit.client.0.smithi018.stdout: 1 637402 4.83 MB/sec execute 597 sec latency 1473.590 ms 2024-02-17T13:02:42.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:41 smithi119 ceph-mon[36188]: pgmap v402: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-02-17T13:02:43.304 INFO:tasks.workunit.client.0.smithi018.stdout: 1 638427 4.83 MB/sec execute 598 sec latency 2031.175 ms 2024-02-17T13:02:44.200 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:43 smithi146 ceph-mon[36454]: pgmap v403: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-02-17T13:02:44.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:43 smithi018 ceph-mon[32625]: pgmap v403: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-02-17T13:02:44.322 INFO:tasks.workunit.client.0.smithi018.stdout: 1 639979 4.84 MB/sec execute 599 sec latency 18.451 ms 2024-02-17T13:02:44.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:43 smithi119 ceph-mon[36188]: pgmap v403: 97 pgs: 97 active+clean; 252 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-02-17T13:02:45.304 INFO:tasks.workunit.client.0.smithi018.stdout: 1 cleanup 600 sec 2024-02-17T13:02:46.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:45 smithi146 ceph-mon[36454]: pgmap v404: 97 pgs: 97 active+clean; 254 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-02-17T13:02:46.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:45 smithi018 ceph-mon[32625]: pgmap v404: 97 pgs: 97 active+clean; 254 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-02-17T13:02:46.304 INFO:tasks.workunit.client.0.smithi018.stdout: 1 cleanup 601 sec 2024-02-17T13:02:46.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:45 smithi119 ceph-mon[36188]: pgmap v404: 97 pgs: 97 active+clean; 254 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-02-17T13:02:46.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:02:46.547 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:02:46.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.674+0000 7f669db6c640 1 -- 172.21.15.18:0/2649963226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f669806a170 msgr2=0x7f669810e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:46.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.674+0000 7f669db6c640 1 --2- 172.21.15.18:0/2649963226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f669806a170 0x7f669810e000 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f6690009cd0 tx=0x7f669002c0f0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.674+0000 7f669db6c640 1 -- 172.21.15.18:0/2649963226 shutdown_connections 2024-02-17T13:02:46.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.674+0000 7f669db6c640 1 --2- 172.21.15.18:0/2649963226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f669806a170 0x7f669810e000 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.674+0000 7f669db6c640 1 --2- 172.21.15.18:0/2649963226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66980697b0 0x7f6698069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.674+0000 7f669db6c640 1 --2- 172.21.15.18:0/2649963226 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6698106130 0x7f6698106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.674+0000 7f669db6c640 1 -- 172.21.15.18:0/2649963226 >> 172.21.15.18:0/2649963226 conn(0x7f6698076e20 msgr2=0x7f6698077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:46.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.674+0000 7f669db6c640 1 -- 172.21.15.18:0/2649963226 shutdown_connections 2024-02-17T13:02:46.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.674+0000 7f669db6c640 1 -- 172.21.15.18:0/2649963226 wait complete. 2024-02-17T13:02:46.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.675+0000 7f669db6c640 1 Processor -- start 2024-02-17T13:02:46.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.675+0000 7f669db6c640 1 -- start start 2024-02-17T13:02:46.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.675+0000 7f669db6c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66980697b0 0x7f66981003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.675+0000 7f669db6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f669806a170 0x7f66981008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.675+0000 7f669db6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6698106130 0x7f6698100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.675+0000 7f669db6c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66981a59b0 con 0x7f6698106130 2024-02-17T13:02:46.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.675+0000 7f669db6c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66981a5b20 con 0x7f669806a170 2024-02-17T13:02:46.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.675+0000 7f669db6c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66981a5ce0 con 0x7f66980697b0 2024-02-17T13:02:46.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.675+0000 7f6697fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6698106130 0x7f6698100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.675+0000 7f6697fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6698106130 0x7f6698100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53654/0 (socket says 172.21.15.18:53654) 2024-02-17T13:02:46.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.675+0000 7f6697fff640 1 -- 172.21.15.18:0/3501829504 learned_addr learned my addr 172.21.15.18:0/3501829504 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:46.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.675+0000 7f66977fe640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66980697b0 0x7f66981003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.676+0000 7f6696ffd640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f669806a170 0x7f66981008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.676+0000 7f6697fff640 1 -- 172.21.15.18:0/3501829504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66980697b0 msgr2=0x7f66981003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:46.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.676+0000 7f6697fff640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66980697b0 0x7f66981003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.676+0000 7f6697fff640 1 -- 172.21.15.18:0/3501829504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f669806a170 msgr2=0x7f66981008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:46.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.676+0000 7f6697fff640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f669806a170 0x7f66981008e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.676+0000 7f6697fff640 1 -- 172.21.15.18:0/3501829504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6690009900 con 0x7f6698106130 2024-02-17T13:02:46.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.676+0000 7f6697fff640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6698106130 0x7f6698100e20 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f6690004af0 tx=0x7f6690034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:46.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.676+0000 7f6694ff9640 1 -- 172.21.15.18:0/3501829504 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66900328c0 con 0x7f6698106130 2024-02-17T13:02:46.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.676+0000 7f6694ff9640 1 -- 172.21.15.18:0/3501829504 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6690003e50 con 0x7f6698106130 2024-02-17T13:02:46.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.676+0000 7f669db6c640 1 -- 172.21.15.18:0/3501829504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66981a5f60 con 0x7f6698106130 2024-02-17T13:02:46.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.676+0000 7f6694ff9640 1 -- 172.21.15.18:0/3501829504 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690004860 con 0x7f6698106130 2024-02-17T13:02:46.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.676+0000 7f669db6c640 1 -- 172.21.15.18:0/3501829504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66981a63e0 con 0x7f6698106130 2024-02-17T13:02:46.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.678+0000 7f6694ff9640 1 -- 172.21.15.18:0/3501829504 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f669000a040 con 0x7f6698106130 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.678+0000 7f66a081c740 1 Processor -- start 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.678+0000 7f66a081c740 1 -- start start 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6694ff9640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f66800a2230 0x7f66800a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f66a081c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 0x5624f76b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f66a081c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f66a081c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76bdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f66a081c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f7621280 con 0x5624f76b2bd0 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f66a081c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f761e7b0 con 0x5624f76b22b0 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f66a081c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f7619f90 con 0x5624f76bb7b0 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6696ffd640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f66800a2230 0x7f66800a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6694ff9640 1 -- 172.21.15.18:0/3501829504 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f66900f0d60 con 0x7f6698106130 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6697fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76bdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6696ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f66977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 0x5624f76b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6697fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76bdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:51538/0 (socket says 172.21.15.18:51538) 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6696ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53662/0 (socket says 172.21.15.18:53662) 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6697fff640 1 -- 172.21.15.18:0/4099524629 learned_addr learned my addr 172.21.15.18:0/4099524629 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6697fff640 1 -- 172.21.15.18:0/4099524629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 msgr2=0x5624f76b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6697fff640 1 --2- 172.21.15.18:0/4099524629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 0x5624f76b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6697fff640 1 -- 172.21.15.18:0/4099524629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 msgr2=0x5624f76bb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6697fff640 1 --2- 172.21.15.18:0/4099524629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6697fff640 1 -- 172.21.15.18:0/4099524629 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f75d8570 con 0x5624f76bb7b0 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6696ffd640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f66800a2230 0x7f66800a46f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6698101e30 tx=0x7f668c00a950 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.679+0000 7f6697fff640 1 --2- 172.21.15.18:0/4099524629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76bdba0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f66900ba1f0 tx=0x7f6690050000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23515b39d73372a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.680+0000 7f667dffb640 1 -- 172.21.15.18:0/4099524629 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669009ae40 con 0x5624f76bb7b0 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.680+0000 7f667dffb640 1 -- 172.21.15.18:0/4099524629 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f669009ac60 con 0x5624f76bb7b0 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.680+0000 7f667dffb640 1 -- 172.21.15.18:0/4099524629 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66900bd820 con 0x5624f76bb7b0 2024-02-17T13:02:46.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.680+0000 7f66a081c740 1 -- 172.21.15.18:0/4099524629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 msgr2=0x5624f76bdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.680+0000 7f66a081c740 1 --2- 172.21.15.18:0/4099524629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76bdba0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f66900ba1f0 tx=0x7f6690050000 comp rx=0 tx=0).stop 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.680+0000 7f66a081c740 1 -- 172.21.15.18:0/4099524629 shutdown_connections 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.680+0000 7f66a081c740 1 --2- 172.21.15.18:0/4099524629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76bdba0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.680+0000 7f66a081c740 1 --2- 172.21.15.18:0/4099524629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.680+0000 7f66a081c740 1 --2- 172.21.15.18:0/4099524629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 0x5624f76b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.680+0000 7f66a081c740 1 -- 172.21.15.18:0/4099524629 >> 172.21.15.18:0/4099524629 conn(0x5624f76ac990 msgr2=0x5624f76acd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.680+0000 7f66a081c740 1 -- 172.21.15.18:0/4099524629 shutdown_connections 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.680+0000 7f66a081c740 1 -- 172.21.15.18:0/4099524629 wait complete. 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f66a081c740 1 -- start start 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f66a081c740 1 Processor -- start 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f66a081c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b22b0 0x5624f76bb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f66a081c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76b2bd0 0x5624f76b8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f66a081c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76bb7b0 0x5624f76b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f66a081c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f72ed430 con 0x5624f76b22b0 2024-02-17T13:02:46.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f66a081c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f72a9e00 con 0x5624f76bb7b0 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f66a081c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f759efd0 con 0x5624f76b2bd0 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f66a081c740 10 client.0 _ll_register_callbacks cb 0x5624f75f4fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f66977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b22b0 0x5624f76bb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f66977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b22b0 0x5624f76bb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53678/0 (socket says 172.21.15.18:53678) 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f66977fe640 1 -- 172.21.15.18:0/2404317912 learned_addr learned my addr 172.21.15.18:0/2404317912 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.681+0000 7f6696ffd640 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76b2bd0 0x5624f76b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f6697fff640 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76bb7b0 0x5624f76b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f66977fe640 1 -- 172.21.15.18:0/2404317912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76b2bd0 msgr2=0x5624f76b8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f66977fe640 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76b2bd0 0x5624f76b8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f66977fe640 1 -- 172.21.15.18:0/2404317912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76bb7b0 msgr2=0x5624f76b8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f66977fe640 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76bb7b0 0x5624f76b8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f66977fe640 1 -- 172.21.15.18:0/2404317912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66981a63e0 con 0x5624f76b22b0 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f66977fe640 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b22b0 0x5624f76bb1b0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f6688012740 tx=0x7f6688012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eef3375987c0c8f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f6676ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f6676ffd640 1 -- 172.21.15.18:0/2404317912 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688017070 con 0x5624f76b22b0 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f6676ffd640 1 -- 172.21.15.18:0/2404317912 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6688004030 con 0x5624f76b22b0 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f66a081c740 1 -- 172.21.15.18:0/2404317912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624f73197e0 con 0x5624f76b22b0 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f6676ffd640 1 -- 172.21.15.18:0/2404317912 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66880122e0 con 0x5624f76b22b0 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f66a081c740 10 client.20097 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f66a081c740 1 -- 172.21.15.18:0/2404317912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5624f7618610 con 0x5624f76b22b0 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.682+0000 7f6676ffd640 1 -- 172.21.15.18:0/2404317912 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5624f73197e0 con 0x5624f76b22b0 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.683+0000 7f6676ffd640 1 -- 172.21.15.18:0/2404317912 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6688003950 con 0x5624f76b22b0 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.683+0000 7f66a081c740 10 client.20097 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.683+0000 7f66a081c740 10 client.20097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.683+0000 7f66a081c740 20 client.20097 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.683+0000 7f66a081c740 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5624f76cb760 0x5624f76cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.683+0000 7f66a081c740 4 client.20097 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.683+0000 7f66a081c740 1 -- 172.21.15.18:0/2404317912 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624f7356720 con 0x5624f76cb760 2024-02-17T13:02:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.683+0000 7f6696ffd640 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5624f76cb760 0x5624f76cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.683+0000 7f6676ffd640 10 client.20097 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.683+0000 7f6696ffd640 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5624f76cb760 0x5624f76cdb80 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff847731254ca10b server_cookie=9889d800216319a8 in_seq=0 out_seq=0 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f6676ffd640 1 -- 172.21.15.18:0/2404317912 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5624f7356720 con 0x5624f76cb760 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f6676ffd640 10 client.20097 handle_command_reply: tid=0 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f66a081c740 1 client.20097 shutdown 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f66a081c740 1 -- 172.21.15.18:0/2404317912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b22b0 msgr2=0x5624f76bb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f66a081c740 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b22b0 0x5624f76bb1b0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f6688012740 tx=0x7f6688012c00 comp rx=0 tx=0).stop 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f66a081c740 1 -- 172.21.15.18:0/2404317912 shutdown_connections 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f66a081c740 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5624f76cb760 0x5624f76cdb80 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f66a081c740 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76bb7b0 0x5624f76b8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f66a081c740 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76b2bd0 0x5624f76b8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f66a081c740 1 --2- 172.21.15.18:0/2404317912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b22b0 0x5624f76bb1b0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f6676ffd640 0 client.20097 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:46.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f66a081c740 1 -- 172.21.15.18:0/2404317912 >> 172.21.15.18:0/2404317912 conn(0x5624f76ac990 msgr2=0x5624f76ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:46.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f66a081c740 1 -- 172.21.15.18:0/2404317912 shutdown_connections 2024-02-17T13:02:46.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f66a081c740 1 -- 172.21.15.18:0/2404317912 wait complete. 2024-02-17T13:02:46.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.684+0000 7f66a081c740 20 client.20097 trim_cache size 0 max 16384 2024-02-17T13:02:46.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.699+0000 7f66a081c740 1 Processor -- start 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f66a081c740 1 -- start start 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f66a081c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 0x5624f76bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f66a081c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f66a081c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f66a081c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f76a6e50 con 0x5624f76b2bd0 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f66a081c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f76b1a80 con 0x5624f76b22b0 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f66a081c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f76b1650 con 0x5624f76bb7b0 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f6696ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f6696ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53680/0 (socket says 172.21.15.18:53680) 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f6696ffd640 1 -- 172.21.15.18:0/3962808327 learned_addr learned my addr 172.21.15.18:0/3962808327 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f6697fff640 1 --2- 172.21.15.18:0/3962808327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f6696ffd640 1 -- 172.21.15.18:0/3962808327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 msgr2=0x5624f76b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:46.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f6696ffd640 1 --2- 172.21.15.18:0/3962808327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f6696ffd640 1 -- 172.21.15.18:0/3962808327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 msgr2=0x5624f76bac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f6696ffd640 1 --2- 172.21.15.18:0/3962808327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 0x5624f76bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f6696ffd640 1 -- 172.21.15.18:0/3962808327 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f75b4ac0 con 0x5624f76b2bd0 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f6696ffd640 1 --2- 172.21.15.18:0/3962808327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bb1b0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f668c012480 tx=0x7f668c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43ea1f7a70044b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f667d7fa640 1 -- 172.21.15.18:0/3962808327 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c00fd30 con 0x5624f76b2bd0 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f667d7fa640 1 -- 172.21.15.18:0/3962808327 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f668c018040 con 0x5624f76b2bd0 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.700+0000 7f667d7fa640 1 -- 172.21.15.18:0/3962808327 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c028910 con 0x5624f76b2bd0 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.701+0000 7f66a081c740 1 -- 172.21.15.18:0/3962808327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 msgr2=0x5624f76bb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.701+0000 7f66a081c740 1 --2- 172.21.15.18:0/3962808327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bb1b0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f668c012480 tx=0x7f668c004730 comp rx=0 tx=0).stop 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.701+0000 7f66a081c740 1 -- 172.21.15.18:0/3962808327 shutdown_connections 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.701+0000 7f66a081c740 1 --2- 172.21.15.18:0/3962808327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.701+0000 7f66a081c740 1 --2- 172.21.15.18:0/3962808327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bb1b0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.701+0000 7f66a081c740 1 --2- 172.21.15.18:0/3962808327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 0x5624f76bac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.701+0000 7f66a081c740 1 -- 172.21.15.18:0/3962808327 >> 172.21.15.18:0/3962808327 conn(0x5624f76ac990 msgr2=0x5624f76d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.701+0000 7f66a081c740 1 -- 172.21.15.18:0/3962808327 shutdown_connections 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.701+0000 7f66a081c740 1 -- 172.21.15.18:0/3962808327 wait complete. 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.701+0000 7f66a081c740 1 -- start start 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f66a081c740 1 Processor -- start 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f66a081c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 0x5624f76bef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f66a081c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f66a081c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76bf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f66a081c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f7721c20 con 0x5624f76b2bd0 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f66a081c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f7721d60 con 0x5624f76b22b0 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f66a081c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f7721f20 con 0x5624f76bb7b0 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f66a081c740 10 client.0 _ll_register_callbacks cb 0x7f6690032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f6696ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f6696ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53696/0 (socket says 172.21.15.18:53696) 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f6696ffd640 1 -- 172.21.15.18:0/1952686504 learned_addr learned my addr 172.21.15.18:0/1952686504 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:46.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f6697fff640 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f66977fe640 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 0x5624f76bef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f6696ffd640 1 -- 172.21.15.18:0/1952686504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 msgr2=0x5624f76bf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f6696ffd640 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76bf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f6696ffd640 1 -- 172.21.15.18:0/1952686504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 msgr2=0x5624f76bef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f6696ffd640 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 0x5624f76bef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f6696ffd640 1 -- 172.21.15.18:0/1952686504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c028b70 con 0x5624f76b2bd0 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f66977fe640 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 0x5624f76bef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f6696ffd640 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bf450 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f668c00a4a0 tx=0x7f668c00ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76bde024fc977bbd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.702+0000 7f6676ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f6676ffd640 1 -- 172.21.15.18:0/1952686504 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c023040 con 0x5624f76b2bd0 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f6676ffd640 1 -- 172.21.15.18:0/1952686504 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f668c026070 con 0x5624f76b2bd0 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f66a081c740 1 -- 172.21.15.18:0/1952686504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f669009ac60 con 0x5624f76b2bd0 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f6676ffd640 1 -- 172.21.15.18:0/1952686504 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f668c01f840 con 0x5624f76b2bd0 2024-02-17T13:02:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f66a081c740 10 client.20103 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f66a081c740 1 -- 172.21.15.18:0/1952686504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5624f75da6a0 con 0x5624f76b2bd0 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f6676ffd640 1 -- 172.21.15.18:0/1952686504 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f668c047080 con 0x5624f76b2bd0 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f6676ffd640 1 -- 172.21.15.18:0/1952686504 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f668c039030 con 0x5624f76b2bd0 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f66a081c740 10 client.20103 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f66a081c740 10 client.20103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f66a081c740 20 client.20103 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f66a081c740 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5624f77237a0 0x5624f7725bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f66a081c740 4 client.20103 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f66a081c740 1 -- 172.21.15.18:0/1952686504 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5624f73a5890 con 0x5624f77237a0 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.703+0000 7f66977fe640 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5624f77237a0 0x5624f7725bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f66977fe640 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5624f77237a0 0x5624f7725bc0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a64a320c0927e45 server_cookie=b9730839f878da2e in_seq=0 out_seq=0 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f6676ffd640 10 client.20103 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f6676ffd640 1 -- 172.21.15.18:0/1952686504 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5624f73a5890 con 0x5624f77237a0 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f6676ffd640 10 client.20103 handle_command_reply: tid=0 2024-02-17T13:02:46.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f66a081c740 1 client.20103 shutdown 2024-02-17T13:02:46.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f66a081c740 1 -- 172.21.15.18:0/1952686504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 msgr2=0x5624f76bf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:46.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f66a081c740 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bf450 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f668c00a4a0 tx=0x7f668c00ff60 comp rx=0 tx=0).stop 2024-02-17T13:02:46.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f66a081c740 1 -- 172.21.15.18:0/1952686504 shutdown_connections 2024-02-17T13:02:46.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f66a081c740 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5624f77237a0 0x5624f7725bc0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f66a081c740 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5624f76bb7b0 0x5624f76bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f66a081c740 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624f76b2bd0 0x5624f76bf450 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f6676ffd640 0 client.20103 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:46.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f66a081c740 1 --2- 172.21.15.18:0/1952686504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624f76b22b0 0x5624f76bef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.704+0000 7f66a081c740 1 -- 172.21.15.18:0/1952686504 >> 172.21.15.18:0/1952686504 conn(0x5624f76ac990 msgr2=0x5624f76b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:46.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f66a081c740 1 -- 172.21.15.18:0/1952686504 shutdown_connections 2024-02-17T13:02:46.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f66a081c740 1 -- 172.21.15.18:0/1952686504 wait complete. 2024-02-17T13:02:46.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f66a081c740 20 client.20103 trim_cache size 0 max 16384 2024-02-17T13:02:46.701 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:02:46.701 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T13:02:46.701 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T13:02:46.701 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:02:46.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f669db6c640 1 -- 172.21.15.18:0/3501829504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f66800a2230 msgr2=0x7f66800a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:46.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f669db6c640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f66800a2230 0x7f66800a46f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6698101e30 tx=0x7f668c00a950 comp rx=0 tx=0).stop 2024-02-17T13:02:46.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f669db6c640 1 -- 172.21.15.18:0/3501829504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6698106130 msgr2=0x7f6698100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:46.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f669db6c640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6698106130 0x7f6698100e20 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f6690004af0 tx=0x7f6690034cc0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f669db6c640 1 -- 172.21.15.18:0/3501829504 shutdown_connections 2024-02-17T13:02:46.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f669db6c640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f66800a2230 0x7f66800a46f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f669db6c640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6698106130 0x7f6698100e20 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f669db6c640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f669806a170 0x7f66981008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f669db6c640 1 --2- 172.21.15.18:0/3501829504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66980697b0 0x7f66981003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f669db6c640 1 -- 172.21.15.18:0/3501829504 >> 172.21.15.18:0/3501829504 conn(0x7f6698076e20 msgr2=0x7f66980fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:46.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f669db6c640 1 -- 172.21.15.18:0/3501829504 shutdown_connections 2024-02-17T13:02:46.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.705+0000 7f669db6c640 1 -- 172.21.15.18:0/3501829504 wait complete. 2024-02-17T13:02:46.705 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T13:02:46.706 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T13:02:46.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.825+0000 7fdd0bd35640 1 -- 172.21.15.18:0/1969911438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd04100410 msgr2=0x7fdd0410e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:46.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.825+0000 7fdd0bd35640 1 --2- 172.21.15.18:0/1969911438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd04100410 0x7fdd0410e010 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009cd0 tx=0x7fdcfc02c180 comp rx=0 tx=0).stop 2024-02-17T13:02:46.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.825+0000 7fdd0bd35640 1 -- 172.21.15.18:0/1969911438 shutdown_connections 2024-02-17T13:02:46.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.825+0000 7fdd0bd35640 1 --2- 172.21.15.18:0/1969911438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd04100410 0x7fdd0410e010 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.825+0000 7fdd0bd35640 1 --2- 172.21.15.18:0/1969911438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd040ffa50 0x7fdd040ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.825+0000 7fdd0bd35640 1 --2- 172.21.15.18:0/1969911438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd04107390 0x7fdd04107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.825+0000 7fdd0bd35640 1 -- 172.21.15.18:0/1969911438 >> 172.21.15.18:0/1969911438 conn(0x7fdd040753d0 msgr2=0x7fdd040757e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:46.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.825+0000 7fdd0bd35640 1 -- 172.21.15.18:0/1969911438 shutdown_connections 2024-02-17T13:02:46.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.825+0000 7fdd0bd35640 1 -- 172.21.15.18:0/1969911438 wait complete. 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd0bd35640 1 Processor -- start 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd0bd35640 1 -- start start 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd0bd35640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd040ffa50 0x7fdd04197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd0bd35640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd04100410 0x7fdd04197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd0bd35640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd04107390 0x7fdd0419bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd0bd35640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0419c530 con 0x7fdd040ffa50 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd0bd35640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0419c6a0 con 0x7fdd04100410 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd0bd35640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0419c860 con 0x7fdd04107390 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd09ad1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd040ffa50 0x7fdd04197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd09ad1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd040ffa50 0x7fdd04197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53718/0 (socket says 172.21.15.18:53718) 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd09ad1640 1 -- 172.21.15.18:0/2254999348 learned_addr learned my addr 172.21.15.18:0/2254999348 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd0a2d2640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd04107390 0x7fdd0419bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd092d0640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd04100410 0x7fdd04197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd09ad1640 1 -- 172.21.15.18:0/2254999348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd04107390 msgr2=0x7fdd0419bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:46.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd09ad1640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd04107390 0x7fdd0419bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd09ad1640 1 -- 172.21.15.18:0/2254999348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd04100410 msgr2=0x7fdd04197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:46.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd09ad1640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd04100410 0x7fdd04197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:46.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.826+0000 7fdd09ad1640 1 -- 172.21.15.18:0/2254999348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc009900 con 0x7fdd040ffa50 2024-02-17T13:02:46.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.827+0000 7fdd09ad1640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd040ffa50 0x7fdd04197670 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000c920 tx=0x7fdd0000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:46.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.827+0000 7fdcfaffd640 1 -- 172.21.15.18:0/2254999348 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00007c70 con 0x7fdd040ffa50 2024-02-17T13:02:46.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.827+0000 7fdd0bd35640 1 -- 172.21.15.18:0/2254999348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0419cb40 con 0x7fdd040ffa50 2024-02-17T13:02:46.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.827+0000 7fdcfaffd640 1 -- 172.21.15.18:0/2254999348 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd00007dd0 con 0x7fdd040ffa50 2024-02-17T13:02:46.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.827+0000 7fdcfaffd640 1 -- 172.21.15.18:0/2254999348 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00018860 con 0x7fdd040ffa50 2024-02-17T13:02:46.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.827+0000 7fdd0bd35640 1 -- 172.21.15.18:0/2254999348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd041a43c0 con 0x7fdd040ffa50 2024-02-17T13:02:46.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.828+0000 7fdd0bd35640 1 -- 172.21.15.18:0/2254999348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd040683c0 con 0x7fdd040ffa50 2024-02-17T13:02:46.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.829+0000 7fdcfaffd640 1 -- 172.21.15.18:0/2254999348 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fdd0000fac0 con 0x7fdd040ffa50 2024-02-17T13:02:46.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.829+0000 7fdcfaffd640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdcf00a2230 0x7fdcf00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:46.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.829+0000 7fdd092d0640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdcf00a2230 0x7fdcf00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:46.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.829+0000 7fdcfaffd640 1 -- 172.21.15.18:0/2254999348 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdd000ce4d0 con 0x7fdd040ffa50 2024-02-17T13:02:46.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.830+0000 7fdd092d0640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdcf00a2230 0x7fdcf00a46f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4005e10 tx=0x7fdcf4005da0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:46.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:46.833+0000 7fdcfaffd640 1 -- 172.21.15.18:0/2254999348 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd00097210 con 0x7fdd040ffa50 2024-02-17T13:02:47.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.052+0000 7fdd0bd35640 1 -- 172.21.15.18:0/2254999348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd041989c0 con 0x7fdd040ffa50 2024-02-17T13:02:47.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.053+0000 7fdcfaffd640 1 -- 172.21.15.18:0/2254999348 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+11392 (secure 0 0 0) 0x7fdd0009a8d0 con 0x7fdd040ffa50 2024-02-17T13:02:47.043 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:02:47.043 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T13:01:34.948363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T13:02:47.043 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 51 2024-02-17T13:02:47.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.055+0000 7fdd0bd35640 1 -- 172.21.15.18:0/2254999348 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdcf00a2230 msgr2=0x7fdcf00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.055+0000 7fdd0bd35640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdcf00a2230 0x7fdcf00a46f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4005e10 tx=0x7fdcf4005da0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.055+0000 7fdd0bd35640 1 -- 172.21.15.18:0/2254999348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd040ffa50 msgr2=0x7fdd04197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.055+0000 7fdd0bd35640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd040ffa50 0x7fdd04197670 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000c920 tx=0x7fdd0000cde0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.055+0000 7fdd0bd35640 1 -- 172.21.15.18:0/2254999348 shutdown_connections 2024-02-17T13:02:47.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.055+0000 7fdd0bd35640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fdcf00a2230 0x7fdcf00a46f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.055+0000 7fdd0bd35640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd04107390 0x7fdd0419bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.055+0000 7fdd0bd35640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd04100410 0x7fdd04197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.055+0000 7fdd0bd35640 1 --2- 172.21.15.18:0/2254999348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd040ffa50 0x7fdd04197670 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.055+0000 7fdd0bd35640 1 -- 172.21.15.18:0/2254999348 >> 172.21.15.18:0/2254999348 conn(0x7fdd040753d0 msgr2=0x7fdd040fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.055+0000 7fdd0bd35640 1 -- 172.21.15.18:0/2254999348 shutdown_connections 2024-02-17T13:02:47.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.055+0000 7fdd0bd35640 1 -- 172.21.15.18:0/2254999348 wait complete. 2024-02-17T13:02:47.055 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:02:47.056 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T13:02:47.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.172+0000 7f8c663bc640 1 -- 172.21.15.18:0/3965822667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c60106450 msgr2=0x7f8c6010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.172+0000 7f8c663bc640 1 --2- 172.21.15.18:0/3965822667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c60106450 0x7f8c6010e020 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009cd0 tx=0x7f8c5802c0f0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.172+0000 7f8c663bc640 1 -- 172.21.15.18:0/3965822667 shutdown_connections 2024-02-17T13:02:47.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.172+0000 7f8c663bc640 1 --2- 172.21.15.18:0/3965822667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c60106450 0x7f8c6010e020 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.172+0000 7f8c663bc640 1 --2- 172.21.15.18:0/3965822667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c60101d30 0x7f8c60105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.172+0000 7f8c663bc640 1 --2- 172.21.15.18:0/3965822667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c60101380 0x7f8c60101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.172+0000 7f8c663bc640 1 -- 172.21.15.18:0/3965822667 >> 172.21.15.18:0/3965822667 conn(0x7f8c60078580 msgr2=0x7f8c60078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.172+0000 7f8c663bc640 1 -- 172.21.15.18:0/3965822667 shutdown_connections 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.172+0000 7f8c663bc640 1 -- 172.21.15.18:0/3965822667 wait complete. 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.172+0000 7f8c663bc640 1 Processor -- start 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.172+0000 7f8c663bc640 1 -- start start 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c663bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c60101380 0x7f8c6019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c663bc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c60106450 0x7f8c6019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c663bc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c601975c0 0x7f8c60197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c663bc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c60197fb0 con 0x7f8c601975c0 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c663bc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c60198120 con 0x7f8c60101380 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c663bc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c601982e0 con 0x7f8c60106450 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c64959640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c601975c0 0x7f8c60197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c64959640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c601975c0 0x7f8c60197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53742/0 (socket says 172.21.15.18:53742) 2024-02-17T13:02:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c64959640 1 -- 172.21.15.18:0/4246038626 learned_addr learned my addr 172.21.15.18:0/4246038626 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c60106450 0x7f8c6019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c5ffff640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c60101380 0x7f8c6019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c64959640 1 -- 172.21.15.18:0/4246038626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c60106450 msgr2=0x7f8c6019d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c64959640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c60106450 0x7f8c6019d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c64959640 1 -- 172.21.15.18:0/4246038626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c60101380 msgr2=0x7f8c6019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c64959640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c60101380 0x7f8c6019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c64959640 1 -- 172.21.15.18:0/4246038626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c58009900 con 0x7f8c601975c0 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c64959640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c601975c0 0x7f8c60197a70 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f8c58005e30 tx=0x7f8c58034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c5d7fa640 1 -- 172.21.15.18:0/4246038626 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c580328b0 con 0x7f8c601975c0 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c663bc640 1 -- 172.21.15.18:0/4246038626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c601a42d0 con 0x7f8c601975c0 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c5d7fa640 1 -- 172.21.15.18:0/4246038626 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c58003e50 con 0x7f8c601975c0 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c5d7fa640 1 -- 172.21.15.18:0/4246038626 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58004860 con 0x7f8c601975c0 2024-02-17T13:02:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.173+0000 7f8c663bc640 1 -- 172.21.15.18:0/4246038626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c601a4780 con 0x7f8c601975c0 2024-02-17T13:02:47.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.176+0000 7f8c5d7fa640 1 -- 172.21.15.18:0/4246038626 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8c5800a040 con 0x7f8c601975c0 2024-02-17T13:02:47.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c68f8e740 1 Processor -- start 2024-02-17T13:02:47.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c68f8e740 1 -- start start 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c68f8e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e79630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c68f8e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e79b70 0x559ff9e82210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c68f8e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e82750 0x559ff9e84b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c68f8e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff9e34960 con 0x559ff9e79250 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c68f8e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff9de8310 con 0x559ff9e82750 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c68f8e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff9de5840 con 0x559ff9e79b70 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c5d7fa640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8c480a2230 0x7f8c480a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c5f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e79630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c5f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e79630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53758/0 (socket says 172.21.15.18:53758) 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/3907513855 learned_addr learned my addr 172.21.15.18:0/3907513855 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c64959640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8c480a2230 0x7f8c480a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c5ffff640 1 --2- 172.21.15.18:0/3907513855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e79b70 0x559ff9e82210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c5d7fa640 1 -- 172.21.15.18:0/4246038626 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8c580f0c70 con 0x7f8c601975c0 2024-02-17T13:02:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/3907513855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e82750 0x559ff9e84b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/3907513855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e79b70 msgr2=0x559ff9e82210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/3907513855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e79b70 0x559ff9e82210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/3907513855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e82750 msgr2=0x559ff9e84b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/3907513855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e82750 0x559ff9e84b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.177+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/3907513855 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ff9da3060 con 0x559ff9e79250 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.178+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/3907513855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e79630 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f8c54013530 tx=0x7f8c540096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3aa75c789c162a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.178+0000 7f8c64959640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8c480a2230 0x7f8c480a46f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8c580b9bf0 tx=0x7f8c5808f750 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.178+0000 7f8c467fc640 1 -- 172.21.15.18:0/3907513855 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54004020 con 0x559ff9e79250 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.178+0000 7f8c467fc640 1 -- 172.21.15.18:0/3907513855 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c54004180 con 0x559ff9e79250 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.178+0000 7f8c467fc640 1 -- 172.21.15.18:0/3907513855 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c540106f0 con 0x559ff9e79250 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.178+0000 7f8c68f8e740 1 -- 172.21.15.18:0/3907513855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 msgr2=0x559ff9e79630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.178+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/3907513855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e79630 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f8c54013530 tx=0x7f8c540096c0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.178+0000 7f8c68f8e740 1 -- 172.21.15.18:0/3907513855 shutdown_connections 2024-02-17T13:02:47.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.178+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/3907513855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e82750 0x559ff9e84b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.178+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/3907513855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e79b70 0x559ff9e82210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.178+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/3907513855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e79630 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.178+0000 7f8c68f8e740 1 -- 172.21.15.18:0/3907513855 >> 172.21.15.18:0/3907513855 conn(0x559ff9e73980 msgr2=0x559ff9e73d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c68f8e740 1 -- 172.21.15.18:0/3907513855 shutdown_connections 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c68f8e740 1 -- 172.21.15.18:0/3907513855 wait complete. 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c68f8e740 1 -- start start 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c68f8e740 1 Processor -- start 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c68f8e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e7f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c68f8e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e7fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c68f8e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e82750 0x559ff9e80220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c68f8e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff9de1020 con 0x559ff9e79250 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c68f8e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff9ab4430 con 0x559ff9e79b70 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c68f8e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff9a70e00 con 0x559ff9e82750 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c68f8e740 10 client.0 _ll_register_callbacks cb 0x559ff9d88cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:47.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c5f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e7f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c5f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e7f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53764/0 (socket says 172.21.15.18:53764) 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.179+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/2361360346 learned_addr learned my addr 172.21.15.18:0/2361360346 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c5ffff640 1 --2- 172.21.15.18:0/2361360346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e7fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/2361360346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e82750 msgr2=0x559ff9e80220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/2361360346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e82750 0x559ff9e80220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/2361360346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 msgr2=0x559ff9e7fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/2361360346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e7fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/2361360346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c54009c80 con 0x559ff9e79250 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/2361360346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e7f7a0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f8c54013430 tx=0x7f8c540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a95456ad1ac9152 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c3f7fe640 1 -- 172.21.15.18:0/2361360346 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54003a00 con 0x559ff9e79250 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c3f7fe640 1 -- 172.21.15.18:0/2361360346 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8c54005050 con 0x559ff9e79250 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c68f8e740 1 -- 172.21.15.18:0/2361360346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ff9ae07e0 con 0x559ff9e79250 2024-02-17T13:02:47.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c3f7fe640 1 -- 172.21.15.18:0/2361360346 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c54005210 con 0x559ff9e79250 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c68f8e740 10 client.20121 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c68f8e740 1 -- 172.21.15.18:0/2361360346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x559ff9ddf6a0 con 0x559ff9e79250 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.180+0000 7f8c3f7fe640 1 -- 172.21.15.18:0/2361360346 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559ff9ae07e0 con 0x559ff9e79250 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.181+0000 7f8c3f7fe640 1 -- 172.21.15.18:0/2361360346 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8c5401d070 con 0x559ff9e79250 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.181+0000 7f8c68f8e740 10 client.20121 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.181+0000 7f8c68f8e740 10 client.20121 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.181+0000 7f8c68f8e740 20 client.20121 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.181+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/2361360346 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x559ff9e92a20 0x559ff9e94e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.181+0000 7f8c68f8e740 4 client.20121 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.181+0000 7f8c68f8e740 1 -- 172.21.15.18:0/2361360346 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ff9b1d720 con 0x559ff9e92a20 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.181+0000 7f8c5ffff640 1 --2- 172.21.15.18:0/2361360346 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x559ff9e92a20 0x559ff9e94e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.181+0000 7f8c5ffff640 1 --2- 172.21.15.18:0/2361360346 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x559ff9e92a20 0x559ff9e94e40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3f0eeadf5c810716 server_cookie=68d99d1f2a4ffbc1 in_seq=0 out_seq=0 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.181+0000 7f8c3f7fe640 10 client.20121 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c3f7fe640 1 -- 172.21.15.18:0/2361360346 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559ff9b1d720 con 0x559ff9e92a20 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c3f7fe640 10 client.20121 handle_command_reply: tid=0 2024-02-17T13:02:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c68f8e740 1 client.20121 shutdown 2024-02-17T13:02:47.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c68f8e740 1 -- 172.21.15.18:0/2361360346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 msgr2=0x559ff9e7f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/2361360346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e7f7a0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f8c54013430 tx=0x7f8c540042f0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c68f8e740 1 -- 172.21.15.18:0/2361360346 shutdown_connections 2024-02-17T13:02:47.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/2361360346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e82750 0x559ff9e80220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/2361360346 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x559ff9e92a20 0x559ff9e94e40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/2361360346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e7fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/2361360346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e7f7a0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c3f7fe640 0 client.20121 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:02:47.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c68f8e740 1 -- 172.21.15.18:0/2361360346 >> 172.21.15.18:0/2361360346 conn(0x559ff9e73980 msgr2=0x559ff9e756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c68f8e740 1 -- 172.21.15.18:0/2361360346 shutdown_connections 2024-02-17T13:02:47.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c68f8e740 1 -- 172.21.15.18:0/2361360346 wait complete. 2024-02-17T13:02:47.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.182+0000 7f8c68f8e740 20 client.20121 trim_cache size 0 max 16384 2024-02-17T13:02:47.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.194+0000 7f8c68f8e740 1 Processor -- start 2024-02-17T13:02:47.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.194+0000 7f8c68f8e740 1 -- start start 2024-02-17T13:02:47.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.194+0000 7f8c68f8e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e79250 0x559ff9e81ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.194+0000 7f8c68f8e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e7aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.194+0000 7f8c68f8e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e82750 0x559ff9e7afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.194+0000 7f8c68f8e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff9d66130 con 0x559ff9e82750 2024-02-17T13:02:47.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.194+0000 7f8c68f8e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff9e65a40 con 0x559ff9e79b70 2024-02-17T13:02:47.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.194+0000 7f8c68f8e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff9e8f680 con 0x559ff9e79250 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.194+0000 7f8c5f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e82750 0x559ff9e7afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.194+0000 7f8c5f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e82750 0x559ff9e7afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53776/0 (socket says 172.21.15.18:53776) 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.194+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/3701886674 learned_addr learned my addr 172.21.15.18:0/3701886674 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.195+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/3701886674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e79250 0x559ff9e81ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.195+0000 7f8c5ffff640 1 --2- 172.21.15.18:0/3701886674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e7aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.195+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/3701886674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e79250 msgr2=0x559ff9e81ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.195+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/3701886674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e79250 0x559ff9e81ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.195+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/3701886674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 msgr2=0x559ff9e7aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.195+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/3701886674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e7aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.195+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/3701886674 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ff9d7bc20 con 0x559ff9e82750 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.195+0000 7f8c5ffff640 1 --2- 172.21.15.18:0/3701886674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e7aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.195+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/3701886674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e82750 0x559ff9e7afd0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f8c54000c00 tx=0x7f8c540045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f17feadc4441070 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.195+0000 7f8c45ffb640 1 -- 172.21.15.18:0/3701886674 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54003a00 con 0x559ff9e82750 2024-02-17T13:02:47.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.195+0000 7f8c45ffb640 1 -- 172.21.15.18:0/3701886674 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c54009ea0 con 0x559ff9e82750 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.195+0000 7f8c45ffb640 1 -- 172.21.15.18:0/3701886674 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5402b470 con 0x559ff9e82750 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 -- 172.21.15.18:0/3701886674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e82750 msgr2=0x559ff9e7afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/3701886674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e82750 0x559ff9e7afd0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f8c54000c00 tx=0x7f8c540045d0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 -- 172.21.15.18:0/3701886674 shutdown_connections 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/3701886674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e82750 0x559ff9e7afd0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/3701886674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e7aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/3701886674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e79250 0x559ff9e81ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 -- 172.21.15.18:0/3701886674 >> 172.21.15.18:0/3701886674 conn(0x559ff9e73980 msgr2=0x559ff9e6ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 -- 172.21.15.18:0/3701886674 shutdown_connections 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 -- 172.21.15.18:0/3701886674 wait complete. 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 -- start start 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 Processor -- start 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e88800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e88d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e82750 0x559ff9ee8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff9ee9470 con 0x559ff9e79250 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff9ee95b0 con 0x559ff9e79b70 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff9ee9720 con 0x559ff9e82750 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c68f8e740 10 client.0 _ll_register_callbacks cb 0x7f8c540097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c5f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e88800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.196+0000 7f8c5f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e88800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53790/0 (socket says 172.21.15.18:53790) 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/1510089246 learned_addr learned my addr 172.21.15.18:0/1510089246 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c5ffff640 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e88d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/1510089246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e82750 msgr2=0x559ff9ee8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e82750 0x559ff9ee8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/1510089246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 msgr2=0x559ff9e88d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e88d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c5f7fe640 1 -- 172.21.15.18:0/1510089246 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ff9ddf6a0 con 0x559ff9e79250 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c5ffff640 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e88d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c5f7fe640 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e88800 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f8c540108e0 tx=0x7f8c54010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869330dbb88ec788 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c3f7fe640 1 -- 172.21.15.18:0/1510089246 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54003a00 con 0x559ff9e79250 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c3f7fe640 1 -- 172.21.15.18:0/1510089246 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8c54000e40 con 0x559ff9e79250 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c68f8e740 1 -- 172.21.15.18:0/1510089246 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c54004180 con 0x559ff9e79250 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c3f7fe640 1 -- 172.21.15.18:0/1510089246 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c54003b60 con 0x559ff9e79250 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c68f8e740 10 client.20127 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c68f8e740 1 -- 172.21.15.18:0/1510089246 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x559ff9da18c0 con 0x559ff9e79250 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.197+0000 7f8c3f7fe640 1 -- 172.21.15.18:0/1510089246 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c54004180 con 0x559ff9e79250 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c3f7fe640 1 -- 172.21.15.18:0/1510089246 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8c5401d020 con 0x559ff9e79250 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c68f8e740 10 client.20127 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c68f8e740 10 client.20127 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:02:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c68f8e740 20 client.20127 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x559ff9eea980 0x559ff9eecd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c68f8e740 4 client.20127 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c68f8e740 1 -- 172.21.15.18:0/1510089246 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559ff9b6c890 con 0x559ff9eea980 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c5ffff640 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x559ff9eea980 0x559ff9eecd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c5ffff640 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x559ff9eea980 0x559ff9eecd70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bf951aea2de298fe server_cookie=834ddf36cdc0cf5 in_seq=0 out_seq=0 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c3f7fe640 10 client.20127 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c3f7fe640 1 -- 172.21.15.18:0/1510089246 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559ff9b6c890 con 0x559ff9eea980 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c3f7fe640 10 client.20127 handle_command_reply: tid=0 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.198+0000 7f8c68f8e740 1 client.20127 shutdown 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c68f8e740 1 -- 172.21.15.18:0/1510089246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 msgr2=0x559ff9e88800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e88800 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f8c540108e0 tx=0x7f8c54010a00 comp rx=0 tx=0).stop 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c68f8e740 1 -- 172.21.15.18:0/1510089246 shutdown_connections 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ff9e82750 0x559ff9ee8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x559ff9eea980 0x559ff9eecd70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ff9e79b70 0x559ff9e88d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c68f8e740 1 --2- 172.21.15.18:0/1510089246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ff9e79250 0x559ff9e88800 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c3f7fe640 0 client.20127 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:02:47.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c68f8e740 1 -- 172.21.15.18:0/1510089246 >> 172.21.15.18:0/1510089246 conn(0x559ff9e73980 msgr2=0x559ff9e7aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c68f8e740 1 -- 172.21.15.18:0/1510089246 shutdown_connections 2024-02-17T13:02:47.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c68f8e740 1 -- 172.21.15.18:0/1510089246 wait complete. 2024-02-17T13:02:47.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c68f8e740 20 client.20127 trim_cache size 0 max 16384 2024-02-17T13:02:47.195 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:02:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c663bc640 1 -- 172.21.15.18:0/4246038626 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8c480a2230 msgr2=0x7f8c480a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c663bc640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8c480a2230 0x7f8c480a46f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8c580b9bf0 tx=0x7f8c5808f750 comp rx=0 tx=0).stop 2024-02-17T13:02:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c663bc640 1 -- 172.21.15.18:0/4246038626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c601975c0 msgr2=0x7f8c60197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c663bc640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c601975c0 0x7f8c60197a70 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f8c58005e30 tx=0x7f8c58034a90 comp rx=0 tx=0).stop 2024-02-17T13:02:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c663bc640 1 -- 172.21.15.18:0/4246038626 shutdown_connections 2024-02-17T13:02:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c663bc640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8c480a2230 0x7f8c480a46f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c663bc640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c601975c0 0x7f8c60197a70 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c663bc640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c60106450 0x7f8c6019d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c663bc640 1 --2- 172.21.15.18:0/4246038626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c60101380 0x7f8c6019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c663bc640 1 -- 172.21.15.18:0/4246038626 >> 172.21.15.18:0/4246038626 conn(0x7f8c60078580 msgr2=0x7f8c600ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c663bc640 1 -- 172.21.15.18:0/4246038626 shutdown_connections 2024-02-17T13:02:47.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.199+0000 7f8c663bc640 1 -- 172.21.15.18:0/4246038626 wait complete. 2024-02-17T13:02:47.199 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:02:47.199 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T13:02:47.304 INFO:tasks.workunit.client.0.smithi018.stdout: 1 cleanup 602 sec 2024-02-17T13:02:47.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.326+0000 7fe4eef2e640 1 -- 172.21.15.18:0/618965407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4e8101380 msgr2=0x7fe4e8101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.326+0000 7fe4eef2e640 1 --2- 172.21.15.18:0/618965407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4e8101380 0x7fe4e8101760 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009930 tx=0x7fe4dc02bc30 comp rx=0 tx=0).stop 2024-02-17T13:02:47.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.326+0000 7fe4eef2e640 1 -- 172.21.15.18:0/618965407 shutdown_connections 2024-02-17T13:02:47.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.326+0000 7fe4eef2e640 1 --2- 172.21.15.18:0/618965407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e8106450 0x7fe4e810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.326+0000 7fe4eef2e640 1 --2- 172.21.15.18:0/618965407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e8101d30 0x7fe4e8105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.326+0000 7fe4eef2e640 1 --2- 172.21.15.18:0/618965407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4e8101380 0x7fe4e8101760 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.326+0000 7fe4eef2e640 1 -- 172.21.15.18:0/618965407 >> 172.21.15.18:0/618965407 conn(0x7fe4e8078580 msgr2=0x7fe4e8078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.326+0000 7fe4eef2e640 1 -- 172.21.15.18:0/618965407 shutdown_connections 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.326+0000 7fe4eef2e640 1 -- 172.21.15.18:0/618965407 wait complete. 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4eef2e640 1 Processor -- start 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4eef2e640 1 -- start start 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4eef2e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e8101380 0x7fe4e81976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4eef2e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4e8106450 0x7fe4e8197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4eef2e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e819bf70 0x7fe4e819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4eef2e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e819c960 con 0x7fe4e8106450 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4eef2e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e819cad0 con 0x7fe4e8101380 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4eef2e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e819cc40 con 0x7fe4e819bf70 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4e7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4e8106450 0x7fe4e8197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4e7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4e8106450 0x7fe4e8197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53806/0 (socket says 172.21.15.18:53806) 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4e7fff640 1 -- 172.21.15.18:0/595395785 learned_addr learned my addr 172.21.15.18:0/595395785 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e819bf70 0x7fe4e819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4eccca640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e8101380 0x7fe4e81976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4e7fff640 1 -- 172.21.15.18:0/595395785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e819bf70 msgr2=0x7fe4e819c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4e7fff640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e819bf70 0x7fe4e819c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4e7fff640 1 -- 172.21.15.18:0/595395785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e8101380 msgr2=0x7fe4e81976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4e7fff640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e8101380 0x7fe4e81976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4e7fff640 1 -- 172.21.15.18:0/595395785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc009560 con 0x7fe4e8106450 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e819bf70 0x7fe4e819c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.327+0000 7fe4eccca640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e8101380 0x7fe4e81976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.328+0000 7fe4e7fff640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4e8106450 0x7fe4e8197bf0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400c970 tx=0x7fe4d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.328+0000 7fe4e5ffb640 1 -- 172.21.15.18:0/595395785 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4004440 con 0x7fe4e8106450 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.328+0000 7fe4e5ffb640 1 -- 172.21.15.18:0/595395785 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4d4007d40 con 0x7fe4e8106450 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.328+0000 7fe4eef2e640 1 -- 172.21.15.18:0/595395785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e81a4220 con 0x7fe4e8106450 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.328+0000 7fe4e5ffb640 1 -- 172.21.15.18:0/595395785 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d400f450 con 0x7fe4e8106450 2024-02-17T13:02:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.328+0000 7fe4eef2e640 1 -- 172.21.15.18:0/595395785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e81a46d0 con 0x7fe4e8106450 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4e5ffb640 1 -- 172.21.15.18:0/595395785 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe4d4007860 con 0x7fe4e8106450 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4f1bd7740 1 Processor -- start 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4f1bd7740 1 -- start start 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4f1bd7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 0x5609a0595360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4f1bd7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a05a2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4f1bd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a05a2b60 0x5609a05a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4f1bd7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a0550960 con 0x5609a0594f80 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4f1bd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a0504310 con 0x5609a05a2b60 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4f1bd7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a0501840 con 0x5609a05958a0 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4e5ffb640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe4d80a2230 0x7fe4d80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4ed4cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 0x5609a0595360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4ed4cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 0x5609a0595360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53820/0 (socket says 172.21.15.18:53820) 2024-02-17T13:02:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4ed4cb640 1 -- 172.21.15.18:0/1239747569 learned_addr learned my addr 172.21.15.18:0/1239747569 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4eccca640 1 --2- 172.21.15.18:0/1239747569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a05a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe4d80a2230 0x7fe4d80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4e5ffb640 1 -- 172.21.15.18:0/595395785 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe4d40cdff0 con 0x7fe4e8106450 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.331+0000 7fe4e7fff640 1 --2- 172.21.15.18:0/1239747569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a05a2b60 0x5609a05a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.332+0000 7fe4ed4cb640 1 -- 172.21.15.18:0/1239747569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 msgr2=0x5609a05a2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.332+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/1239747569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a05a2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.332+0000 7fe4ed4cb640 1 -- 172.21.15.18:0/1239747569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a05a2b60 msgr2=0x5609a05a4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.332+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/1239747569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a05a2b60 0x5609a05a4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.332+0000 7fe4ed4cb640 1 -- 172.21.15.18:0/1239747569 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a04bf060 con 0x5609a0594f80 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.332+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/1239747569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 0x5609a0595360 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0012d80 tx=0x7fe4e000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f9c30b3d02a4aa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.332+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe4d80a2230 0x7fe4d80a46f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000fc30 tx=0x7fe4e001a000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.332+0000 7fe4d2ffd640 1 -- 172.21.15.18:0/1239747569 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0019ce0 con 0x5609a0594f80 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.332+0000 7fe4d2ffd640 1 -- 172.21.15.18:0/1239747569 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4e0018040 con 0x5609a0594f80 2024-02-17T13:02:47.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.332+0000 7fe4d2ffd640 1 -- 172.21.15.18:0/1239747569 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e000d3d0 con 0x5609a0594f80 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/1239747569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 msgr2=0x5609a0595360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/1239747569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 0x5609a0595360 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0012d80 tx=0x7fe4e000f730 comp rx=0 tx=0).stop 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/1239747569 shutdown_connections 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/1239747569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a05a2b60 0x5609a05a4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/1239747569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a05a2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/1239747569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 0x5609a0595360 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/1239747569 >> 172.21.15.18:0/1239747569 conn(0x5609a058f980 msgr2=0x5609a058fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/1239747569 shutdown_connections 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/1239747569 wait complete. 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 -- start start 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 Processor -- start 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a0594f80 0x5609a059fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.333+0000 7fe4f1bd7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a05a0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4f1bd7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 0x5609a05a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4f1bd7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a04fd020 con 0x5609a05a2b60 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4f1bd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a01d0430 con 0x5609a0594f80 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4f1bd7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a018ce00 con 0x5609a05958a0 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4f1bd7740 10 client.0 _ll_register_callbacks cb 0x5609a04a4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4ed4cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 0x5609a05a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4ed4cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 0x5609a05a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53832/0 (socket says 172.21.15.18:53832) 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4ed4cb640 1 -- 172.21.15.18:0/3873080280 learned_addr learned my addr 172.21.15.18:0/3873080280 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4eccca640 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a0594f80 0x5609a059fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4e7fff640 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a05a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4ed4cb640 1 -- 172.21.15.18:0/3873080280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 msgr2=0x5609a05a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a05a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4ed4cb640 1 -- 172.21.15.18:0/3873080280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a0594f80 msgr2=0x5609a059fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a0594f80 0x5609a059fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4ed4cb640 1 -- 172.21.15.18:0/3873080280 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e0012930 con 0x5609a05a2b60 2024-02-17T13:02:47.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 0x5609a05a06b0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fe4e001ca80 tx=0x7fe4e0019d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ace3a48a77145c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4c7fff640 1 -- 172.21.15.18:0/3873080280 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e001c840 con 0x5609a05a2b60 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4c7fff640 1 -- 172.21.15.18:0/3873080280 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe4e0004a00 con 0x5609a05a2b60 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/3873080280 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609a01fc7e0 con 0x5609a05a2b60 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4c7fff640 1 -- 172.21.15.18:0/3873080280 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e0004c00 con 0x5609a05a2b60 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4f1bd7740 10 client.20139 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.334+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/3873080280 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5609a04fb6a0 con 0x5609a05a2b60 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.335+0000 7fe4c7fff640 1 -- 172.21.15.18:0/3873080280 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5609a01fc7e0 con 0x5609a05a2b60 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.335+0000 7fe4c7fff640 1 -- 172.21.15.18:0/3873080280 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe4e0005880 con 0x5609a05a2b60 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.335+0000 7fe4f1bd7740 10 client.20139 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.335+0000 7fe4f1bd7740 10 client.20139 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.335+0000 7fe4f1bd7740 20 client.20139 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.335+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5609a05b2fa0 0x5609a05b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.335+0000 7fe4f1bd7740 4 client.20139 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.335+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/3873080280 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609a0239720 con 0x5609a05b2fa0 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.335+0000 7fe4eccca640 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5609a05b2fa0 0x5609a05b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.335+0000 7fe4c7fff640 10 client.20139 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.335+0000 7fe4eccca640 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5609a05b2fa0 0x5609a05b5390 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4d120d57040cb5cd server_cookie=921c454c33a44d62 in_seq=0 out_seq=0 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4c7fff640 1 -- 172.21.15.18:0/3873080280 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5609a0239720 con 0x5609a05b2fa0 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4c7fff640 10 client.20139 handle_command_reply: tid=0 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4f1bd7740 1 client.20139 shutdown 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/3873080280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 msgr2=0x5609a05a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 0x5609a05a06b0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fe4e001ca80 tx=0x7fe4e0019d60 comp rx=0 tx=0).stop 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/3873080280 shutdown_connections 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5609a05b2fa0 0x5609a05b5390 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 0x5609a05a06b0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a05a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4c7fff640 0 client.20139 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:02:47.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/3873080280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a0594f80 0x5609a059fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/3873080280 >> 172.21.15.18:0/3873080280 conn(0x5609a058f980 msgr2=0x5609a05916d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/3873080280 shutdown_connections 2024-02-17T13:02:47.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.336+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/3873080280 wait complete. 2024-02-17T13:02:47.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.337+0000 7fe4f1bd7740 20 client.20139 trim_cache size 0 max 16384 2024-02-17T13:02:47.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.352+0000 7fe4f1bd7740 1 Processor -- start 2024-02-17T13:02:47.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.352+0000 7fe4f1bd7740 1 -- start start 2024-02-17T13:02:47.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.352+0000 7fe4f1bd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a0594f80 0x5609a05a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.352+0000 7fe4f1bd7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a058ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.352+0000 7fe4f1bd7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 0x5609a05a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.352+0000 7fe4f1bd7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a0482130 con 0x5609a05a2b60 2024-02-17T13:02:47.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.352+0000 7fe4f1bd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a059b830 con 0x5609a0594f80 2024-02-17T13:02:47.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.352+0000 7fe4f1bd7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a0596010 con 0x5609a05958a0 2024-02-17T13:02:47.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4ed4cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 0x5609a05a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4ed4cb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 0x5609a05a99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53834/0 (socket says 172.21.15.18:53834) 2024-02-17T13:02:47.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4ed4cb640 1 -- 172.21.15.18:0/1122166887 learned_addr learned my addr 172.21.15.18:0/1122166887 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4e7fff640 1 --2- 172.21.15.18:0/1122166887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a058ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4eccca640 1 --2- 172.21.15.18:0/1122166887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a0594f80 0x5609a05a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4ed4cb640 1 -- 172.21.15.18:0/1122166887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 msgr2=0x5609a058ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/1122166887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a058ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4ed4cb640 1 -- 172.21.15.18:0/1122166887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a0594f80 msgr2=0x5609a05a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/1122166887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a0594f80 0x5609a05a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4ed4cb640 1 -- 172.21.15.18:0/1122166887 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a0497c20 con 0x5609a05a2b60 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4eccca640 1 --2- 172.21.15.18:0/1122166887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a0594f80 0x5609a05a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/1122166887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 0x5609a05a99c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0005fd0 tx=0x7fe4e0005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e1197a451b7257 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4d27fc640 1 -- 172.21.15.18:0/1122166887 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e000a4e0 con 0x5609a05a2b60 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4d27fc640 1 -- 172.21.15.18:0/1122166887 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4e001c640 con 0x5609a05a2b60 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.353+0000 7fe4d27fc640 1 -- 172.21.15.18:0/1122166887 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0025b20 con 0x5609a05a2b60 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.354+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/1122166887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 msgr2=0x5609a05a99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.354+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/1122166887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 0x5609a05a99c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0005fd0 tx=0x7fe4e0005c80 comp rx=0 tx=0).stop 2024-02-17T13:02:47.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.354+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/1122166887 shutdown_connections 2024-02-17T13:02:47.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.354+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/1122166887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a05a2b60 0x5609a05a99c0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.354+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/1122166887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a058ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.354+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/1122166887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a0594f80 0x5609a05a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.354+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/1122166887 >> 172.21.15.18:0/1122166887 conn(0x5609a058f980 msgr2=0x5609a058e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.354+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/1122166887 shutdown_connections 2024-02-17T13:02:47.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.354+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/1122166887 wait complete. 2024-02-17T13:02:47.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.354+0000 7fe4f1bd7740 1 -- start start 2024-02-17T13:02:47.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.354+0000 7fe4f1bd7740 1 Processor -- start 2024-02-17T13:02:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4f1bd7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 0x5609a05a8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4f1bd7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a05a8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4f1bd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a05a2b60 0x5609a05a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4f1bd7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a0609480 con 0x5609a0594f80 2024-02-17T13:02:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4f1bd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a06095c0 con 0x5609a05a2b60 2024-02-17T13:02:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4f1bd7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a0609700 con 0x5609a05958a0 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4f1bd7740 10 client.0 _ll_register_callbacks cb 0x7fe4e000add0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4eccca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 0x5609a05a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4eccca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 0x5609a05a8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53848/0 (socket says 172.21.15.18:53848) 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4eccca640 1 -- 172.21.15.18:0/875710646 learned_addr learned my addr 172.21.15.18:0/875710646 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4eccca640 1 -- 172.21.15.18:0/875710646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 msgr2=0x5609a05a8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4eccca640 1 --2- 172.21.15.18:0/875710646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a05a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4eccca640 1 -- 172.21.15.18:0/875710646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a05a2b60 msgr2=0x5609a05a8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4ed4cb640 1 --2- 172.21.15.18:0/875710646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a05a2b60 0x5609a05a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4eccca640 1 --2- 172.21.15.18:0/875710646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a05a2b60 0x5609a05a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4eccca640 1 -- 172.21.15.18:0/875710646 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609a04fb6a0 con 0x5609a0594f80 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4eccca640 1 --2- 172.21.15.18:0/875710646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 0x5609a05a8430 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc02e010 tx=0x7fe4dc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26e4b02044678cb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4c7fff640 1 -- 172.21.15.18:0/875710646 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc045040 con 0x5609a0594f80 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4c7fff640 1 -- 172.21.15.18:0/875710646 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe4dc004040 con 0x5609a0594f80 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.355+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/875710646 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4e0018040 con 0x5609a0594f80 2024-02-17T13:02:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.356+0000 7fe4c7fff640 1 -- 172.21.15.18:0/875710646 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4dc004200 con 0x5609a0594f80 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.356+0000 7fe4f1bd7740 10 client.20145 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.356+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/875710646 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5609a04bd8c0 con 0x5609a0594f80 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.356+0000 7fe4c7fff640 1 -- 172.21.15.18:0/875710646 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4e0018040 con 0x5609a0594f80 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.356+0000 7fe4c7fff640 1 -- 172.21.15.18:0/875710646 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe4dc03c040 con 0x5609a0594f80 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.356+0000 7fe4f1bd7740 10 client.20145 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.356+0000 7fe4f1bd7740 10 client.20145 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.356+0000 7fe4f1bd7740 20 client.20145 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.356+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/875710646 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5609a060af10 0x5609a060d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.356+0000 7fe4f1bd7740 4 client.20145 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.356+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/875710646 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5609a0288890 con 0x5609a060af10 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.356+0000 7fe4e7fff640 1 --2- 172.21.15.18:0/875710646 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5609a060af10 0x5609a060d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4c7fff640 10 client.20145 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4e7fff640 1 --2- 172.21.15.18:0/875710646 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5609a060af10 0x5609a060d330 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6956f6c5f36c5b88 server_cookie=e2c388119b2a044c in_seq=0 out_seq=0 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4c7fff640 1 -- 172.21.15.18:0/875710646 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5609a0288890 con 0x5609a060af10 2024-02-17T13:02:47.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4c7fff640 10 client.20145 handle_command_reply: tid=0 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4f1bd7740 1 client.20145 shutdown 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/875710646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 msgr2=0x5609a05a8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/875710646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 0x5609a05a8430 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc02e010 tx=0x7fe4dc005a20 comp rx=0 tx=0).stop 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/875710646 shutdown_connections 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/875710646 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5609a060af10 0x5609a060d330 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/875710646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609a05a2b60 0x5609a05a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/875710646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5609a05958a0 0x5609a05a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4f1bd7740 1 --2- 172.21.15.18:0/875710646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5609a0594f80 0x5609a05a8430 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4c7fff640 0 client.20145 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/875710646 >> 172.21.15.18:0/875710646 conn(0x5609a058f980 msgr2=0x5609a058eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/875710646 shutdown_connections 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4f1bd7740 1 -- 172.21.15.18:0/875710646 wait complete. 2024-02-17T13:02:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.357+0000 7fe4f1bd7740 20 client.20145 trim_cache size 0 max 16384 2024-02-17T13:02:47.353 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:02:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.358+0000 7fe4eef2e640 1 -- 172.21.15.18:0/595395785 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe4d80a2230 msgr2=0x7fe4d80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.358+0000 7fe4eef2e640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe4d80a2230 0x7fe4d80a46f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000fc30 tx=0x7fe4e001a000 comp rx=0 tx=0).stop 2024-02-17T13:02:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.358+0000 7fe4eef2e640 1 -- 172.21.15.18:0/595395785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4e8106450 msgr2=0x7fe4e8197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.358+0000 7fe4eef2e640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4e8106450 0x7fe4e8197bf0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400c970 tx=0x7fe4d400ce30 comp rx=0 tx=0).stop 2024-02-17T13:02:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.358+0000 7fe4eef2e640 1 -- 172.21.15.18:0/595395785 shutdown_connections 2024-02-17T13:02:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.358+0000 7fe4eef2e640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe4d80a2230 0x7fe4d80a46f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.358+0000 7fe4eef2e640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4e819bf70 0x7fe4e819c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.358+0000 7fe4eef2e640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4e8106450 0x7fe4e8197bf0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.358+0000 7fe4eef2e640 1 --2- 172.21.15.18:0/595395785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4e8101380 0x7fe4e81976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.358+0000 7fe4eef2e640 1 -- 172.21.15.18:0/595395785 >> 172.21.15.18:0/595395785 conn(0x7fe4e8078580 msgr2=0x7fe4e80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.358+0000 7fe4eef2e640 1 -- 172.21.15.18:0/595395785 shutdown_connections 2024-02-17T13:02:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.358+0000 7fe4eef2e640 1 -- 172.21.15.18:0/595395785 wait complete. 2024-02-17T13:02:47.358 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:02:47.358 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T13:02:47.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.474+0000 7fd5d3b48640 1 -- 172.21.15.18:0/509901967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc106460 msgr2=0x7fd5cc10e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.474+0000 7fd5d3b48640 1 --2- 172.21.15.18:0/509901967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc106460 0x7fd5cc10e030 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009be0 tx=0x7fd5c402bef0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.474+0000 7fd5d3b48640 1 -- 172.21.15.18:0/509901967 shutdown_connections 2024-02-17T13:02:47.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.474+0000 7fd5d3b48640 1 --2- 172.21.15.18:0/509901967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc106460 0x7fd5cc10e030 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.474+0000 7fd5d3b48640 1 --2- 172.21.15.18:0/509901967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5cc101d40 0x7fd5cc105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.474+0000 7fd5d3b48640 1 --2- 172.21.15.18:0/509901967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5cc101390 0x7fd5cc101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.474+0000 7fd5d3b48640 1 -- 172.21.15.18:0/509901967 >> 172.21.15.18:0/509901967 conn(0x7fd5cc0fcbb0 msgr2=0x7fd5cc0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.474+0000 7fd5d3b48640 1 -- 172.21.15.18:0/509901967 shutdown_connections 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.474+0000 7fd5d3b48640 1 -- 172.21.15.18:0/509901967 wait complete. 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d3b48640 1 Processor -- start 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d3b48640 1 -- start start 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d3b48640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5cc101390 0x7fd5cc19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d3b48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5cc106460 0x7fd5cc19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d3b48640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc197690 0x7fd5cc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d3b48640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc198160 con 0x7fd5cc101390 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d3b48640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc1982d0 con 0x7fd5cc106460 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d3b48640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc198490 con 0x7fd5cc197690 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d18e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5cc101390 0x7fd5cc19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d18e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5cc101390 0x7fd5cc19d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53866/0 (socket says 172.21.15.18:53866) 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d18e4640 1 -- 172.21.15.18:0/515497046 learned_addr learned my addr 172.21.15.18:0/515497046 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d20e5640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc197690 0x7fd5cc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d10e3640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5cc106460 0x7fd5cc19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d18e4640 1 -- 172.21.15.18:0/515497046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc197690 msgr2=0x7fd5cc197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc197690 0x7fd5cc197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d18e4640 1 -- 172.21.15.18:0/515497046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5cc106460 msgr2=0x7fd5cc19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.475+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5cc106460 0x7fd5cc19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.476+0000 7fd5d18e4640 1 -- 172.21.15.18:0/515497046 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c8009560 con 0x7fd5cc101390 2024-02-17T13:02:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.476+0000 7fd5d10e3640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5cc106460 0x7fd5cc19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.476+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5cc101390 0x7fd5cc19d580 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e9c0 tx=0x7fd5c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.476+0000 7fd5c2ffd640 1 -- 172.21.15.18:0/515497046 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800cde0 con 0x7fd5cc101390 2024-02-17T13:02:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.476+0000 7fd5d3b48640 1 -- 172.21.15.18:0/515497046 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c4009840 con 0x7fd5cc101390 2024-02-17T13:02:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.476+0000 7fd5c2ffd640 1 -- 172.21.15.18:0/515497046 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5c80035f0 con 0x7fd5cc101390 2024-02-17T13:02:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.476+0000 7fd5c2ffd640 1 -- 172.21.15.18:0/515497046 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8010650 con 0x7fd5cc101390 2024-02-17T13:02:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.476+0000 7fd5d3b48640 1 -- 172.21.15.18:0/515497046 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5cc1a4780 con 0x7fd5cc101390 2024-02-17T13:02:47.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.478+0000 7fd5c2ffd640 1 -- 172.21.15.18:0/515497046 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd5c8003ad0 con 0x7fd5cc101390 2024-02-17T13:02:47.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.478+0000 7fd5d67fa740 1 Processor -- start 2024-02-17T13:02:47.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d67fa740 1 -- start start 2024-02-17T13:02:47.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d67fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614863c30 0x556614864010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d67fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x556614871620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d67fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614871b60 0x556614873f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d67fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661481f960 con 0x5566148645e0 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d67fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566147d3310 con 0x556614871b60 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5c2ffd640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd5b80a2230 0x7fd5b80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d67fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566147d0840 con 0x556614863c30 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d18e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x556614871620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d20e5640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd5b80a2230 0x7fd5b80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d18e4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x556614871620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53868/0 (socket says 172.21.15.18:53868) 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d18e4640 1 -- 172.21.15.18:0/4102571231 learned_addr learned my addr 172.21.15.18:0/4102571231 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5c2ffd640 1 -- 172.21.15.18:0/515497046 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd5c8014070 con 0x7fd5cc101390 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d10e3640 1 --2- 172.21.15.18:0/4102571231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614871b60 0x556614873f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d20e5640 1 --2- 172.21.15.18:0/4102571231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614863c30 0x556614864010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d18e4640 1 -- 172.21.15.18:0/4102571231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614863c30 msgr2=0x556614864010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/4102571231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614863c30 0x556614864010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d18e4640 1 -- 172.21.15.18:0/4102571231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614871b60 msgr2=0x556614873f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/4102571231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614871b60 0x556614873f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d18e4640 1 -- 172.21.15.18:0/4102571231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55661478e060 con 0x5566148645e0 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.479+0000 7fd5d20e5640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd5b80a2230 0x7fd5b80a46f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4004af0 tx=0x7fd5c4033ad0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.480+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/4102571231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x556614871620 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009530 tx=0x7fd5c806aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed041e701ec7e1a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.480+0000 7fd5b7fff640 1 -- 172.21.15.18:0/4102571231 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c809b750 con 0x5566148645e0 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.480+0000 7fd5b7fff640 1 -- 172.21.15.18:0/4102571231 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5c80665b0 con 0x5566148645e0 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.480+0000 7fd5b7fff640 1 -- 172.21.15.18:0/4102571231 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8099470 con 0x5566148645e0 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.480+0000 7fd5d67fa740 1 -- 172.21.15.18:0/4102571231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 msgr2=0x556614871620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.480+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/4102571231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x556614871620 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009530 tx=0x7fd5c806aeb0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 -- 172.21.15.18:0/4102571231 shutdown_connections 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/4102571231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614871b60 0x556614873f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/4102571231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x556614871620 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/4102571231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614863c30 0x556614864010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 -- 172.21.15.18:0/4102571231 >> 172.21.15.18:0/4102571231 conn(0x55661485e980 msgr2=0x55661485ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 -- 172.21.15.18:0/4102571231 shutdown_connections 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 -- 172.21.15.18:0/4102571231 wait complete. 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 -- start start 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 Processor -- start 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614863c30 0x55661486eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x55661486f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614871b60 0x55661486f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566147cc020 con 0x5566148645e0 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661449f430 con 0x556614871b60 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661445be00 con 0x556614863c30 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d67fa740 10 client.0 _ll_register_callbacks cb 0x556614773cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.481+0000 7fd5d10e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x55661486f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d10e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x55661486f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53880/0 (socket says 172.21.15.18:53880) 2024-02-17T13:02:47.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d10e3640 1 -- 172.21.15.18:0/2785590535 learned_addr learned my addr 172.21.15.18:0/2785590535 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d20e5640 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614871b60 0x55661486f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614863c30 0x55661486eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d10e3640 1 -- 172.21.15.18:0/2785590535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614863c30 msgr2=0x55661486eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d10e3640 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614863c30 0x55661486eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d10e3640 1 -- 172.21.15.18:0/2785590535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614871b60 msgr2=0x55661486f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d10e3640 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614871b60 0x55661486f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d10e3640 1 -- 172.21.15.18:0/2785590535 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5cc1a4780 con 0x5566148645e0 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614863c30 0x55661486eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d20e5640 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614871b60 0x55661486f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d10e3640 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x55661486f0e0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc013740 tx=0x7fd5bc013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2fd8e1cbd3ba4a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5b57fa640 1 -- 172.21.15.18:0/2785590535 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc004500 con 0x5566148645e0 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5b57fa640 1 -- 172.21.15.18:0/2785590535 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd5bc0119c0 con 0x5566148645e0 2024-02-17T13:02:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d67fa740 1 -- 172.21.15.18:0/2785590535 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566144cb7e0 con 0x5566148645e0 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5b57fa640 1 -- 172.21.15.18:0/2785590535 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5bc011b80 con 0x5566148645e0 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d67fa740 10 client.20157 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5d67fa740 1 -- 172.21.15.18:0/2785590535 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5566147ca6a0 con 0x5566148645e0 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.482+0000 7fd5b57fa640 1 -- 172.21.15.18:0/2785590535 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5566144cb7e0 con 0x5566148645e0 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.483+0000 7fd5b57fa640 1 -- 172.21.15.18:0/2785590535 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd5bc01c4b0 con 0x5566148645e0 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.483+0000 7fd5d67fa740 10 client.20157 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.483+0000 7fd5d67fa740 10 client.20157 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.483+0000 7fd5d67fa740 20 client.20157 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.483+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5566148820c0 0x5566148844e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.483+0000 7fd5d67fa740 4 client.20157 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.483+0000 7fd5d67fa740 1 -- 172.21.15.18:0/2785590535 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556614508720 con 0x5566148820c0 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.483+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5566148820c0 0x5566148844e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.483+0000 7fd5b57fa640 10 client.20157 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.483+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5566148820c0 0x5566148844e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6a135b409180ead server_cookie=6791878b686f434e in_seq=0 out_seq=0 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5b57fa640 1 -- 172.21.15.18:0/2785590535 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556614508720 con 0x5566148820c0 2024-02-17T13:02:47.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5b57fa640 10 client.20157 handle_command_reply: tid=0 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5d67fa740 1 client.20157 shutdown 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5d67fa740 1 -- 172.21.15.18:0/2785590535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 msgr2=0x55661486f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x55661486f0e0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc013740 tx=0x7fd5bc013c00 comp rx=0 tx=0).stop 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5d67fa740 1 -- 172.21.15.18:0/2785590535 shutdown_connections 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5566148820c0 0x5566148844e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614871b60 0x55661486f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x55661486f0e0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5b57fa640 0 client.20157 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/2785590535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614863c30 0x55661486eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5d67fa740 1 -- 172.21.15.18:0/2785590535 >> 172.21.15.18:0/2785590535 conn(0x55661485e980 msgr2=0x5566148606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5d67fa740 1 -- 172.21.15.18:0/2785590535 shutdown_connections 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5d67fa740 1 -- 172.21.15.18:0/2785590535 wait complete. 2024-02-17T13:02:47.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.484+0000 7fd5d67fa740 20 client.20157 trim_cache size 0 max 16384 2024-02-17T13:02:47.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.496+0000 7fd5d67fa740 1 Processor -- start 2024-02-17T13:02:47.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.496+0000 7fd5d67fa740 1 -- start start 2024-02-17T13:02:47.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.496+0000 7fd5d67fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614863c30 0x5566148712d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.496+0000 7fd5d67fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x55661486f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.496+0000 7fd5d67fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614871b60 0x55661486be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.496+0000 7fd5d67fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556614751130 con 0x5566148645e0 2024-02-17T13:02:47.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.496+0000 7fd5d67fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556614863340 con 0x556614863c30 2024-02-17T13:02:47.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.496+0000 7fd5d67fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661485a080 con 0x556614871b60 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.496+0000 7fd5d10e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x55661486f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.496+0000 7fd5d10e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x55661486f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53882/0 (socket says 172.21.15.18:53882) 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.496+0000 7fd5d10e3640 1 -- 172.21.15.18:0/886637429 learned_addr learned my addr 172.21.15.18:0/886637429 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.497+0000 7fd5d20e5640 1 --2- 172.21.15.18:0/886637429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614871b60 0x55661486be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.497+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/886637429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614863c30 0x5566148712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.497+0000 7fd5d10e3640 1 -- 172.21.15.18:0/886637429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614871b60 msgr2=0x55661486be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.497+0000 7fd5d10e3640 1 --2- 172.21.15.18:0/886637429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614871b60 0x55661486be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.497+0000 7fd5d10e3640 1 -- 172.21.15.18:0/886637429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614863c30 msgr2=0x5566148712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.497+0000 7fd5d10e3640 1 --2- 172.21.15.18:0/886637429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614863c30 0x5566148712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.497+0000 7fd5d10e3640 1 -- 172.21.15.18:0/886637429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556614766c20 con 0x5566148645e0 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.497+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/886637429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614863c30 0x5566148712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.497+0000 7fd5d10e3640 1 --2- 172.21.15.18:0/886637429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x55661486f440 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00a010 tx=0x7fd5bc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b007a18749dfc96c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.497+0000 7fd5b77fe640 1 -- 172.21.15.18:0/886637429 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0117b0 con 0x5566148645e0 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.497+0000 7fd5b77fe640 1 -- 172.21.15.18:0/886637429 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc011ea0 con 0x5566148645e0 2024-02-17T13:02:47.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.497+0000 7fd5b77fe640 1 -- 172.21.15.18:0/886637429 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc003870 con 0x5566148645e0 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 -- 172.21.15.18:0/886637429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 msgr2=0x55661486f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/886637429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x55661486f440 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00a010 tx=0x7fd5bc009750 comp rx=0 tx=0).stop 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 -- 172.21.15.18:0/886637429 shutdown_connections 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/886637429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614871b60 0x55661486be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/886637429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x55661486f440 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/886637429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614863c30 0x5566148712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 -- 172.21.15.18:0/886637429 >> 172.21.15.18:0/886637429 conn(0x55661485e980 msgr2=0x556614869ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 -- 172.21.15.18:0/886637429 shutdown_connections 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 -- 172.21.15.18:0/886637429 wait complete. 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 -- start start 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 Processor -- start 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614863c30 0x556614877a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x556614877fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614871b60 0x5566148d85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566148786d0 con 0x5566148645e0 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566148d8b10 con 0x556614863c30 2024-02-17T13:02:47.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566148d8c80 con 0x556614871b60 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d67fa740 10 client.0 _ll_register_callbacks cb 0x7fd5c80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d10e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x556614877fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d10e3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x556614877fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53892/0 (socket says 172.21.15.18:53892) 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.498+0000 7fd5d10e3640 1 -- 172.21.15.18:0/3122689426 learned_addr learned my addr 172.21.15.18:0/3122689426 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614863c30 0x556614877a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5d20e5640 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614871b60 0x5566148d85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5d10e3640 1 -- 172.21.15.18:0/3122689426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614871b60 msgr2=0x5566148d85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5d10e3640 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614871b60 0x5566148d85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5d10e3640 1 -- 172.21.15.18:0/3122689426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614863c30 msgr2=0x556614877a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5d10e3640 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614863c30 0x556614877a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5d10e3640 1 -- 172.21.15.18:0/3122689426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566147ca6a0 con 0x5566148645e0 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5d10e3640 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x556614877fb0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc005b90 tx=0x7fd5bc003670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f757188710bdfbde server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5b57fa640 1 -- 172.21.15.18:0/3122689426 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0119c0 con 0x5566148645e0 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5b57fa640 1 -- 172.21.15.18:0/3122689426 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd5bc000e40 con 0x5566148645e0 2024-02-17T13:02:47.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5d67fa740 1 -- 172.21.15.18:0/3122689426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5c80665b0 con 0x5566148645e0 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5b57fa640 1 -- 172.21.15.18:0/3122689426 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5bc011b20 con 0x5566148645e0 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5d67fa740 10 client.20163 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5d67fa740 1 -- 172.21.15.18:0/3122689426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55661478c8c0 con 0x5566148645e0 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.499+0000 7fd5b57fa640 1 -- 172.21.15.18:0/3122689426 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5c80665b0 con 0x5566148645e0 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5b57fa640 1 -- 172.21.15.18:0/3122689426 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd5bc037570 con 0x5566148645e0 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5d67fa740 10 client.20163 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5d67fa740 10 client.20163 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5d67fa740 20 client.20163 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5566148da040 0x5566148dc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5d67fa740 4 client.20163 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5d67fa740 1 -- 172.21.15.18:0/3122689426 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556614557890 con 0x5566148da040 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5566148da040 0x5566148dc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5b57fa640 10 client.20163 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5d18e4640 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5566148da040 0x5566148dc460 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a30644ac54f69221 server_cookie=128e250c1a0df9c in_seq=0 out_seq=0 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5b57fa640 1 -- 172.21.15.18:0/3122689426 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556614557890 con 0x5566148da040 2024-02-17T13:02:47.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5b57fa640 10 client.20163 handle_command_reply: tid=0 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5d67fa740 1 client.20163 shutdown 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5d67fa740 1 -- 172.21.15.18:0/3122689426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 msgr2=0x556614877fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.500+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x556614877fb0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc005b90 tx=0x7fd5bc003670 comp rx=0 tx=0).stop 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d67fa740 1 -- 172.21.15.18:0/3122689426 shutdown_connections 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x5566148da040 0x5566148dc460 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556614871b60 0x5566148d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5566148645e0 0x556614877fb0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d67fa740 1 --2- 172.21.15.18:0/3122689426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556614863c30 0x556614877a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5b57fa640 0 client.20163 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d67fa740 1 -- 172.21.15.18:0/3122689426 >> 172.21.15.18:0/3122689426 conn(0x55661485e980 msgr2=0x55661485d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d67fa740 1 -- 172.21.15.18:0/3122689426 shutdown_connections 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d67fa740 1 -- 172.21.15.18:0/3122689426 wait complete. 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d67fa740 20 client.20163 trim_cache size 0 max 16384 2024-02-17T13:02:47.496 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:02:47.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d3b48640 1 -- 172.21.15.18:0/515497046 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd5b80a2230 msgr2=0x7fd5b80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d3b48640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd5b80a2230 0x7fd5b80a46f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4004af0 tx=0x7fd5c4033ad0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d3b48640 1 -- 172.21.15.18:0/515497046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5cc101390 msgr2=0x7fd5cc19d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d3b48640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5cc101390 0x7fd5cc19d580 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e9c0 tx=0x7fd5c800ee80 comp rx=0 tx=0).stop 2024-02-17T13:02:47.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d3b48640 1 -- 172.21.15.18:0/515497046 shutdown_connections 2024-02-17T13:02:47.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d3b48640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd5b80a2230 0x7fd5b80a46f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d3b48640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc197690 0x7fd5cc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d3b48640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5cc106460 0x7fd5cc19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d3b48640 1 --2- 172.21.15.18:0/515497046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5cc101390 0x7fd5cc19d580 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d3b48640 1 -- 172.21.15.18:0/515497046 >> 172.21.15.18:0/515497046 conn(0x7fd5cc0fcbb0 msgr2=0x7fd5cc0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d3b48640 1 -- 172.21.15.18:0/515497046 shutdown_connections 2024-02-17T13:02:47.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.501+0000 7fd5d3b48640 1 -- 172.21.15.18:0/515497046 wait complete. 2024-02-17T13:02:47.501 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:02:47.501 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T13:02:47.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.616+0000 7f2c32888640 1 -- 172.21.15.18:0/3917207953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c2c101d30 msgr2=0x7f2c2c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.616+0000 7f2c32888640 1 --2- 172.21.15.18:0/3917207953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c2c101d30 0x7f2c2c105d70 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009930 tx=0x7f2c2002bc30 comp rx=0 tx=0).stop 2024-02-17T13:02:47.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.616+0000 7f2c32888640 1 -- 172.21.15.18:0/3917207953 shutdown_connections 2024-02-17T13:02:47.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.616+0000 7f2c32888640 1 --2- 172.21.15.18:0/3917207953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c106450 0x7f2c2c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.616+0000 7f2c32888640 1 --2- 172.21.15.18:0/3917207953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c2c101d30 0x7f2c2c105d70 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.616+0000 7f2c32888640 1 --2- 172.21.15.18:0/3917207953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c101380 0x7f2c2c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.616+0000 7f2c32888640 1 -- 172.21.15.18:0/3917207953 >> 172.21.15.18:0/3917207953 conn(0x7f2c2c078580 msgr2=0x7f2c2c078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.616+0000 7f2c32888640 1 -- 172.21.15.18:0/3917207953 shutdown_connections 2024-02-17T13:02:47.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.616+0000 7f2c32888640 1 -- 172.21.15.18:0/3917207953 wait complete. 2024-02-17T13:02:47.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c32888640 1 Processor -- start 2024-02-17T13:02:47.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c32888640 1 -- start start 2024-02-17T13:02:47.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c32888640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c101380 0x7f2c2c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c32888640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c2c106450 0x7f2c2c197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c32888640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c19c020 0x7f2c2c19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c32888640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c19ca10 con 0x7f2c2c106450 2024-02-17T13:02:47.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c32888640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c19cb80 con 0x7f2c2c19c020 2024-02-17T13:02:47.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c32888640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c19ccf0 con 0x7f2c2c101380 2024-02-17T13:02:47.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c2c106450 0x7f2c2c197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c2c106450 0x7f2c2c197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53908/0 (socket says 172.21.15.18:53908) 2024-02-17T13:02:47.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/2977848997 learned_addr learned my addr 172.21.15.18:0/2977848997 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c30e25640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c19c020 0x7f2c2c19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c2bfff640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c101380 0x7f2c2c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/2977848997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c101380 msgr2=0x7f2c2c1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c101380 0x7f2c2c1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/2977848997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c19c020 msgr2=0x7f2c2c19c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c19c020 0x7f2c2c19c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.617+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/2977848997 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c20009560 con 0x7f2c2c106450 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.618+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c2c106450 0x7f2c2c197cf0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009230 tx=0x7f2c20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.618+0000 7f2c297fa640 1 -- 172.21.15.18:0/2977848997 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2002cbc0 con 0x7f2c2c106450 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.618+0000 7f2c297fa640 1 -- 172.21.15.18:0/2977848997 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c200052d0 con 0x7f2c2c106450 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.618+0000 7f2c32888640 1 -- 172.21.15.18:0/2977848997 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c1a42d0 con 0x7f2c2c106450 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.618+0000 7f2c297fa640 1 -- 172.21.15.18:0/2977848997 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20003dc0 con 0x7f2c2c106450 2024-02-17T13:02:47.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.618+0000 7f2c32888640 1 -- 172.21.15.18:0/2977848997 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c1a4750 con 0x7f2c2c106450 2024-02-17T13:02:47.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.620+0000 7f2c297fa640 1 -- 172.21.15.18:0/2977848997 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2c200035f0 con 0x7f2c2c106450 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.620+0000 7f2c35467740 1 Processor -- start 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.620+0000 7f2c35467740 1 -- start start 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.620+0000 7f2c297fa640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2c140a2230 0x7f2c140a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c35467740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa14d250 0x5654fa14d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c35467740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa156210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c35467740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa156750 0x5654fa158b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c35467740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654fa108960 con 0x5654fa14db70 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c35467740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654fa0bc310 con 0x5654fa156750 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c35467740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654fa0b9840 con 0x5654fa14d250 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c30e25640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2c140a2230 0x7f2c140a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53918/0 (socket says 172.21.15.18:53918) 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/1144478076 learned_addr learned my addr 172.21.15.18:0/1144478076 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c297fa640 1 -- 172.21.15.18:0/2977848997 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2c200f0340 con 0x7f2c2c106450 2024-02-17T13:02:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c2bfff640 1 --2- 172.21.15.18:0/1144478076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa14d250 0x5654fa14d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c30e25640 1 --2- 172.21.15.18:0/1144478076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa156750 0x5654fa158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/1144478076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa14d250 msgr2=0x5654fa14d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/1144478076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa14d250 0x5654fa14d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/1144478076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa156750 msgr2=0x5654fa158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/1144478076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa156750 0x5654fa158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/1144478076 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654fa077060 con 0x5654fa14db70 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c30e25640 1 --2- 172.21.15.18:0/1144478076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa156750 0x5654fa158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c30e25640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2c140a2230 0x7f2c140a46f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2c24005fd0 tx=0x7f2c240128f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/1144478076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa156210 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009900 tx=0x7f2c20005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cf78c040c3d5af server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c127fc640 1 -- 172.21.15.18:0/1144478076 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2009a480 con 0x5654fa14db70 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c127fc640 1 -- 172.21.15.18:0/1144478076 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c2009a2a0 con 0x5654fa14db70 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.621+0000 7f2c127fc640 1 -- 172.21.15.18:0/1144478076 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200c0fb0 con 0x5654fa14db70 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.622+0000 7f2c35467740 1 -- 172.21.15.18:0/1144478076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 msgr2=0x5654fa156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.622+0000 7f2c35467740 1 --2- 172.21.15.18:0/1144478076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa156210 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009900 tx=0x7f2c20005fd0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.622+0000 7f2c35467740 1 -- 172.21.15.18:0/1144478076 shutdown_connections 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.622+0000 7f2c35467740 1 --2- 172.21.15.18:0/1144478076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa156750 0x5654fa158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.622+0000 7f2c35467740 1 --2- 172.21.15.18:0/1144478076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa156210 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.622+0000 7f2c35467740 1 --2- 172.21.15.18:0/1144478076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa14d250 0x5654fa14d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.622+0000 7f2c35467740 1 -- 172.21.15.18:0/1144478076 >> 172.21.15.18:0/1144478076 conn(0x5654fa147980 msgr2=0x5654fa147d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.622+0000 7f2c35467740 1 -- 172.21.15.18:0/1144478076 shutdown_connections 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.622+0000 7f2c35467740 1 -- 172.21.15.18:0/1144478076 wait complete. 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.622+0000 7f2c35467740 1 -- start start 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c35467740 1 Processor -- start 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c35467740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa14d250 0x5654fa153750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c35467740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa153c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c35467740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa156750 0x5654fa1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c35467740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654fa0b5020 con 0x5654fa14db70 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c35467740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f9d88430 con 0x5654fa14d250 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c35467740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f9d44e00 con 0x5654fa156750 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c35467740 10 client.0 _ll_register_callbacks cb 0x5654fa05ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53930/0 (socket says 172.21.15.18:53930) 2024-02-17T13:02:47.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/3351543000 learned_addr learned my addr 172.21.15.18:0/3351543000 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c2bfff640 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa14d250 0x5654fa153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/3351543000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa156750 msgr2=0x5654fa1541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c30e25640 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa156750 0x5654fa1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa156750 0x5654fa1541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/3351543000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa14d250 msgr2=0x5654fa153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa14d250 0x5654fa153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/3351543000 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c1a4750 con 0x5654fa14db70 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c2bfff640 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa14d250 0x5654fa153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c30e25640 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa156750 0x5654fa1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa153c90 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f2c200edc70 tx=0x7f2c20005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109850623d29b500 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.623+0000 7f2c0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c0b7fe640 1 -- 172.21.15.18:0/3351543000 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2009ba20 con 0x5654fa14db70 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c0b7fe640 1 -- 172.21.15.18:0/3351543000 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2c200bf030 con 0x5654fa14db70 2024-02-17T13:02:47.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c35467740 1 -- 172.21.15.18:0/3351543000 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654f9db47e0 con 0x5654fa14db70 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c0b7fe640 1 -- 172.21.15.18:0/3351543000 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c2009a440 con 0x5654fa14db70 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c35467740 10 client.20175 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c35467740 1 -- 172.21.15.18:0/3351543000 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5654fa0b36a0 con 0x5654fa14db70 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c0b7fe640 1 -- 172.21.15.18:0/3351543000 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5654f9db47e0 con 0x5654fa14db70 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c0b7fe640 1 -- 172.21.15.18:0/3351543000 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2c2003c070 con 0x5654fa14db70 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c35467740 10 client.20175 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c35467740 10 client.20175 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c35467740 20 client.20175 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c35467740 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5654fa166440 0x5654fa168860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c35467740 4 client.20175 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c35467740 1 -- 172.21.15.18:0/3351543000 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654f9df1720 con 0x5654fa166440 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.624+0000 7f2c2bfff640 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5654fa166440 0x5654fa168860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.625+0000 7f2c2bfff640 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5654fa166440 0x5654fa168860 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f11be83efa0fc74 server_cookie=88706890f604900 in_seq=0 out_seq=0 2024-02-17T13:02:47.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.625+0000 7f2c0b7fe640 10 client.20175 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.625+0000 7f2c0b7fe640 1 -- 172.21.15.18:0/3351543000 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5654f9df1720 con 0x5654fa166440 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.625+0000 7f2c0b7fe640 10 client.20175 handle_command_reply: tid=0 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.625+0000 7f2c35467740 1 client.20175 shutdown 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.626+0000 7f2c35467740 1 -- 172.21.15.18:0/3351543000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 msgr2=0x5654fa153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.626+0000 7f2c35467740 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa153c90 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f2c200edc70 tx=0x7f2c20005e80 comp rx=0 tx=0).stop 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.626+0000 7f2c35467740 1 -- 172.21.15.18:0/3351543000 shutdown_connections 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.626+0000 7f2c35467740 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5654fa166440 0x5654fa168860 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.626+0000 7f2c35467740 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa156750 0x5654fa1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.626+0000 7f2c35467740 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa153c90 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.626+0000 7f2c35467740 1 --2- 172.21.15.18:0/3351543000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa14d250 0x5654fa153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.626+0000 7f2c0b7fe640 0 client.20175 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.626+0000 7f2c35467740 1 -- 172.21.15.18:0/3351543000 >> 172.21.15.18:0/3351543000 conn(0x5654fa147980 msgr2=0x5654fa1496d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.626+0000 7f2c35467740 1 -- 172.21.15.18:0/3351543000 shutdown_connections 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.626+0000 7f2c35467740 1 -- 172.21.15.18:0/3351543000 wait complete. 2024-02-17T13:02:47.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.626+0000 7f2c35467740 20 client.20175 trim_cache size 0 max 16384 2024-02-17T13:02:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.637+0000 7f2c35467740 1 Processor -- start 2024-02-17T13:02:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.637+0000 7f2c35467740 1 -- start start 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.637+0000 7f2c35467740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa14d250 0x5654fa155e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.637+0000 7f2c35467740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.637+0000 7f2c35467740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa156750 0x5654fa1509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.637+0000 7f2c35467740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654fa03a130 con 0x5654fa14db70 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c35467740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654fa16da90 con 0x5654fa14d250 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c35467740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654fa158bc0 con 0x5654fa156750 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53936/0 (socket says 172.21.15.18:53936) 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/2836107941 learned_addr learned my addr 172.21.15.18:0/2836107941 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/2836107941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa156750 msgr2=0x5654fa1509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/2836107941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa156750 0x5654fa1509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/2836107941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa14d250 msgr2=0x5654fa155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/2836107941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa14d250 0x5654fa155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/2836107941 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654fa04fc20 con 0x5654fa14db70 2024-02-17T13:02:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/2836107941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa153ff0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f2c200b92c0 tx=0x7f2c200bb370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c53c0040b79f90d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c11ffb640 1 -- 172.21.15.18:0/2836107941 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200bc7c0 con 0x5654fa14db70 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c11ffb640 1 -- 172.21.15.18:0/2836107941 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c200bd560 con 0x5654fa14db70 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.638+0000 7f2c11ffb640 1 -- 172.21.15.18:0/2836107941 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200db1e0 con 0x5654fa14db70 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 -- 172.21.15.18:0/2836107941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 msgr2=0x5654fa153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 --2- 172.21.15.18:0/2836107941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa153ff0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f2c200b92c0 tx=0x7f2c200bb370 comp rx=0 tx=0).stop 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 -- 172.21.15.18:0/2836107941 shutdown_connections 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 --2- 172.21.15.18:0/2836107941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa156750 0x5654fa1509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 --2- 172.21.15.18:0/2836107941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa153ff0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 --2- 172.21.15.18:0/2836107941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa14d250 0x5654fa155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 -- 172.21.15.18:0/2836107941 >> 172.21.15.18:0/2836107941 conn(0x5654fa147980 msgr2=0x5654fa14ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 -- 172.21.15.18:0/2836107941 shutdown_connections 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 -- 172.21.15.18:0/2836107941 wait complete. 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 -- start start 2024-02-17T13:02:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 Processor -- start 2024-02-17T13:02:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa14d250 0x5654fa15d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa156750 0x5654fa15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654fa15c8c0 con 0x5654fa14db70 2024-02-17T13:02:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654fa15ca30 con 0x5654fa156750 2024-02-17T13:02:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.639+0000 7f2c35467740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654fa1bc910 con 0x5654fa14d250 2024-02-17T13:02:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c35467740 10 client.0 _ll_register_callbacks cb 0x7f2c20003f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa15bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53940/0 (socket says 172.21.15.18:53940) 2024-02-17T13:02:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/1013469316 learned_addr learned my addr 172.21.15.18:0/1013469316 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c30e25640 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa156750 0x5654fa15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c2bfff640 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa14d250 0x5654fa15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/1013469316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa14d250 msgr2=0x5654fa15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa14d250 0x5654fa15d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/1013469316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa156750 msgr2=0x5654fa15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa156750 0x5654fa15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c2b7fe640 1 -- 172.21.15.18:0/1013469316 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654fa0b36a0 con 0x5654fa14db70 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c30e25640 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa156750 0x5654fa15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c2bfff640 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa14d250 0x5654fa15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c2b7fe640 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa15bbf0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2c20003fd0 tx=0x7f2c20005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ae49fdf63fa86b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c0b7fe640 1 -- 172.21.15.18:0/1013469316 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20087070 con 0x5654fa14db70 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c0b7fe640 1 -- 172.21.15.18:0/1013469316 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2c200bf070 con 0x5654fa14db70 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c35467740 1 -- 172.21.15.18:0/1013469316 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c2009a2a0 con 0x5654fa14db70 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c0b7fe640 1 -- 172.21.15.18:0/1013469316 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c200db3f0 con 0x5654fa14db70 2024-02-17T13:02:47.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c35467740 10 client.20181 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c35467740 1 -- 172.21.15.18:0/1013469316 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5654fa0758c0 con 0x5654fa14db70 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.640+0000 7f2c0b7fe640 1 -- 172.21.15.18:0/1013469316 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c2009a2a0 con 0x5654fa14db70 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.641+0000 7f2c0b7fe640 1 -- 172.21.15.18:0/1013469316 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2c2003c050 con 0x5654fa14db70 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.641+0000 7f2c35467740 10 client.20181 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.641+0000 7f2c35467740 10 client.20181 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.641+0000 7f2c35467740 20 client.20181 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.641+0000 7f2c35467740 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5654fa1bdc00 0x5654fa1bfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.641+0000 7f2c35467740 4 client.20181 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.641+0000 7f2c35467740 1 -- 172.21.15.18:0/1013469316 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5654f9e40890 con 0x5654fa1bdc00 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.641+0000 7f2c2bfff640 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5654fa1bdc00 0x5654fa1bfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.641+0000 7f2c0b7fe640 10 client.20181 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.641+0000 7f2c2bfff640 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5654fa1bdc00 0x5654fa1bfff0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2104a651ec746d39 server_cookie=f594edfc32d8ce88 in_seq=0 out_seq=0 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c0b7fe640 1 -- 172.21.15.18:0/1013469316 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5654f9e40890 con 0x5654fa1bdc00 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c0b7fe640 10 client.20181 handle_command_reply: tid=0 2024-02-17T13:02:47.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c35467740 1 client.20181 shutdown 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c35467740 1 -- 172.21.15.18:0/1013469316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 msgr2=0x5654fa15bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c35467740 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa15bbf0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2c20003fd0 tx=0x7f2c20005fd0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c35467740 1 -- 172.21.15.18:0/1013469316 shutdown_connections 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c35467740 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5654fa1bdc00 0x5654fa1bfff0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c35467740 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5654fa156750 0x5654fa15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c35467740 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5654fa14db70 0x5654fa15bbf0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c0b7fe640 0 client.20181 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c35467740 1 --2- 172.21.15.18:0/1013469316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654fa14d250 0x5654fa15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c35467740 1 -- 172.21.15.18:0/1013469316 >> 172.21.15.18:0/1013469316 conn(0x5654fa147980 msgr2=0x5654fa1462f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c35467740 1 -- 172.21.15.18:0/1013469316 shutdown_connections 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c35467740 1 -- 172.21.15.18:0/1013469316 wait complete. 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c35467740 20 client.20181 trim_cache size 0 max 16384 2024-02-17T13:02:47.637 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:02:47.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c32888640 1 -- 172.21.15.18:0/2977848997 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2c140a2230 msgr2=0x7f2c140a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c32888640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2c140a2230 0x7f2c140a46f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2c24005fd0 tx=0x7f2c240128f0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c32888640 1 -- 172.21.15.18:0/2977848997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c2c106450 msgr2=0x7f2c2c197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.642+0000 7f2c32888640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c2c106450 0x7f2c2c197cf0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009230 tx=0x7f2c20004270 comp rx=0 tx=0).stop 2024-02-17T13:02:47.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.643+0000 7f2c32888640 1 -- 172.21.15.18:0/2977848997 shutdown_connections 2024-02-17T13:02:47.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.643+0000 7f2c32888640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2c140a2230 0x7f2c140a46f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.643+0000 7f2c32888640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c19c020 0x7f2c2c19c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.643+0000 7f2c32888640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c2c106450 0x7f2c2c197cf0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.643+0000 7f2c32888640 1 --2- 172.21.15.18:0/2977848997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c2c101380 0x7f2c2c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.643+0000 7f2c32888640 1 -- 172.21.15.18:0/2977848997 >> 172.21.15.18:0/2977848997 conn(0x7f2c2c078580 msgr2=0x7f2c2c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.643+0000 7f2c32888640 1 -- 172.21.15.18:0/2977848997 shutdown_connections 2024-02-17T13:02:47.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.643+0000 7f2c32888640 1 -- 172.21.15.18:0/2977848997 wait complete. 2024-02-17T13:02:47.642 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:02:47.643 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T13:02:47.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.757+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/2445433761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c54105330 msgr2=0x7f6c5410cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.757+0000 7f6c5c5f4640 1 --2- 172.21.15.18:0/2445433761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c54105330 0x7f6c5410cf00 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009cd0 tx=0x7f6c4402c0f0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/2445433761 shutdown_connections 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 --2- 172.21.15.18:0/2445433761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c54105330 0x7f6c5410cf00 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 --2- 172.21.15.18:0/2445433761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c54100c10 0x7f6c54104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 --2- 172.21.15.18:0/2445433761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c54100260 0x7f6c54100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/2445433761 >> 172.21.15.18:0/2445433761 conn(0x7f6c54077180 msgr2=0x7f6c54077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/2445433761 shutdown_connections 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/2445433761 wait complete. 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 Processor -- start 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 -- start start 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c54100260 0x7f6c54197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c54105330 0x7f6c54197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c5419bfd0 0x7f6c5419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419c9c0 con 0x7f6c5419bfd0 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419cb30 con 0x7f6c54100260 2024-02-17T13:02:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5c5f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419cca0 con 0x7f6c54105330 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5ab91640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c5419bfd0 0x7f6c5419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5ab91640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c5419bfd0 0x7f6c5419c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53958/0 (socket says 172.21.15.18:53958) 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c5ab91640 1 -- 172.21.15.18:0/1125537364 learned_addr learned my addr 172.21.15.18:0/1125537364 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.758+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c54105330 0x7f6c54197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.759+0000 7f6c5a390640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c54100260 0x7f6c54197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.759+0000 7f6c5ab91640 1 -- 172.21.15.18:0/1125537364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c54105330 msgr2=0x7f6c54197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.759+0000 7f6c5ab91640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c54105330 0x7f6c54197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.759+0000 7f6c5ab91640 1 -- 172.21.15.18:0/1125537364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c54100260 msgr2=0x7f6c54197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.759+0000 7f6c5ab91640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c54100260 0x7f6c54197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.759+0000 7f6c5ab91640 1 -- 172.21.15.18:0/1125537364 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c44009900 con 0x7f6c5419bfd0 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.759+0000 7f6c5ab91640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c5419bfd0 0x7f6c5419c480 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f6c440348f0 tx=0x7f6c44034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.759+0000 7f6c537fe640 1 -- 172.21.15.18:0/1125537364 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440328c0 con 0x7f6c5419bfd0 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.759+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/1125537364 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c541a4280 con 0x7f6c5419bfd0 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.759+0000 7f6c537fe640 1 -- 172.21.15.18:0/1125537364 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c44003e50 con 0x7f6c5419bfd0 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.759+0000 7f6c537fe640 1 -- 172.21.15.18:0/1125537364 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440048a0 con 0x7f6c5419bfd0 2024-02-17T13:02:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.759+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/1125537364 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c541a4700 con 0x7f6c5419bfd0 2024-02-17T13:02:47.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.761+0000 7f6c537fe640 1 -- 172.21.15.18:0/1125537364 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6c4400a040 con 0x7f6c5419bfd0 2024-02-17T13:02:47.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c5f1c8740 1 Processor -- start 2024-02-17T13:02:47.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c5f1c8740 1 -- start start 2024-02-17T13:02:47.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c537fe640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6c3c0a2230 0x7f6c3c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c5f1c8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec79f00 0x55a16ec7a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c5f1c8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec7a820 0x55a16ec87620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c5f1c8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec87b60 0x55a16ec89f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c5f1c8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16ec35960 con 0x55a16ec87b60 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c5f1c8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16ebe9310 con 0x55a16ec79f00 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c5f1c8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16ebe6840 con 0x55a16ec7a820 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6c3c0a2230 0x7f6c3c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c5ab91640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec87b60 0x55a16ec89f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c5ab91640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec87b60 0x55a16ec89f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53966/0 (socket says 172.21.15.18:53966) 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c537fe640 1 -- 172.21.15.18:0/1125537364 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6c440f0c10 con 0x7f6c5419bfd0 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.762+0000 7f6c5ab91640 1 -- 172.21.15.18:0/13141332 learned_addr learned my addr 172.21.15.18:0/13141332 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.763+0000 7f6c5a390640 1 --2- 172.21.15.18:0/13141332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec79f00 0x55a16ec7a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.763+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/13141332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec7a820 0x55a16ec87620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.763+0000 7f6c5ab91640 1 -- 172.21.15.18:0/13141332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec7a820 msgr2=0x55a16ec87620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.763+0000 7f6c5ab91640 1 --2- 172.21.15.18:0/13141332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec7a820 0x55a16ec87620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.763+0000 7f6c5ab91640 1 -- 172.21.15.18:0/13141332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec79f00 msgr2=0x55a16ec7a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.763+0000 7f6c5ab91640 1 --2- 172.21.15.18:0/13141332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec79f00 0x55a16ec7a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.763+0000 7f6c5ab91640 1 -- 172.21.15.18:0/13141332 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a16eba4060 con 0x55a16ec87b60 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.763+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6c3c0a2230 0x7f6c3c0a46f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800a5c0 tx=0x7f6c48013940 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.763+0000 7f6c5ab91640 1 --2- 172.21.15.18:0/13141332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec87b60 0x55a16ec89f50 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009430 tx=0x7f6c44034eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f713791e643bc9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.763+0000 7f6c50bfe640 1 -- 172.21.15.18:0/13141332 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440bd270 con 0x55a16ec87b60 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.763+0000 7f6c50bfe640 1 -- 172.21.15.18:0/13141332 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c440bd090 con 0x55a16ec87b60 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.763+0000 7f6c50bfe640 1 -- 172.21.15.18:0/13141332 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440bbac0 con 0x55a16ec87b60 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.764+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/13141332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec87b60 msgr2=0x55a16ec89f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.764+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/13141332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec87b60 0x55a16ec89f50 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009430 tx=0x7f6c44034eb0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.764+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/13141332 shutdown_connections 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.764+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/13141332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec87b60 0x55a16ec89f50 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.764+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/13141332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec7a820 0x55a16ec87620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.764+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/13141332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec79f00 0x55a16ec7a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.764+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/13141332 >> 172.21.15.18:0/13141332 conn(0x55a16ec74980 msgr2=0x55a16ec74d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.764+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/13141332 shutdown_connections 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.764+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/13141332 wait complete. 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.764+0000 7f6c5f1c8740 1 -- start start 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.764+0000 7f6c5f1c8740 1 Processor -- start 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5f1c8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 0x55a16ec849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5f1c8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec7a820 0x55a16ec84f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5f1c8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec87b60 0x55a16ec85470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5f1c8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16ebe2020 con 0x55a16ec79f00 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5f1c8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16e8b5430 con 0x55a16ec7a820 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5f1c8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16e871e00 con 0x55a16ec87b60 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5f1c8740 10 client.0 _ll_register_callbacks cb 0x55a16eb89cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5a390640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 0x55a16ec849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5a390640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 0x55a16ec849f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53970/0 (socket says 172.21.15.18:53970) 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5a390640 1 -- 172.21.15.18:0/1812215664 learned_addr learned my addr 172.21.15.18:0/1812215664 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec7a820 0x55a16ec84f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5ab91640 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec87b60 0x55a16ec85470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5a390640 1 -- 172.21.15.18:0/1812215664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec87b60 msgr2=0x55a16ec85470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5a390640 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec87b60 0x55a16ec85470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5a390640 1 -- 172.21.15.18:0/1812215664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec7a820 msgr2=0x55a16ec84f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5a390640 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec7a820 0x55a16ec84f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5a390640 1 -- 172.21.15.18:0/1812215664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c541a4700 con 0x55a16ec79f00 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5ab91640 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec87b60 0x55a16ec85470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec7a820 0x55a16ec84f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5a390640 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 0x55a16ec849f0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00ba60 tx=0x7f6c4c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7fbf54b9d79899 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c39ffb640 1 -- 172.21.15.18:0/1812215664 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c016070 con 0x55a16ec79f00 2024-02-17T13:02:47.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c39ffb640 1 -- 172.21.15.18:0/1812215664 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6c4c005020 con 0x55a16ec79f00 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/1812215664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a16e8e17e0 con 0x55a16ec79f00 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c39ffb640 1 -- 172.21.15.18:0/1812215664 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c4c003ee0 con 0x55a16ec79f00 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5f1c8740 10 client.20193 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.765+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/1812215664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a16ebe06a0 con 0x55a16ec79f00 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.766+0000 7f6c39ffb640 1 -- 172.21.15.18:0/1812215664 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a16e8e17e0 con 0x55a16ec79f00 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.766+0000 7f6c39ffb640 1 -- 172.21.15.18:0/1812215664 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6c4c01b500 con 0x55a16ec79f00 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.766+0000 7f6c5f1c8740 10 client.20193 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.766+0000 7f6c5f1c8740 10 client.20193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.766+0000 7f6c5f1c8740 20 client.20193 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.766+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a16ec97d10 0x55a16ec9a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.766+0000 7f6c5f1c8740 4 client.20193 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.766+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/1812215664 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a16e91e720 con 0x55a16ec97d10 2024-02-17T13:02:47.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.766+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a16ec97d10 0x55a16ec9a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.767+0000 7f6c39ffb640 10 client.20193 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.767+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a16ec97d10 0x55a16ec9a150 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4970a0d95cdf1d4d server_cookie=2c7e079eda5541e6 in_seq=0 out_seq=0 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.767+0000 7f6c39ffb640 1 -- 172.21.15.18:0/1812215664 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a16e91e720 con 0x55a16ec97d10 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.767+0000 7f6c39ffb640 10 client.20193 handle_command_reply: tid=0 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.767+0000 7f6c5f1c8740 1 client.20193 shutdown 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.767+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/1812215664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 msgr2=0x55a16ec849f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.767+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 0x55a16ec849f0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00ba60 tx=0x7f6c4c00bf20 comp rx=0 tx=0).stop 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.767+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/1812215664 shutdown_connections 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.767+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a16ec97d10 0x55a16ec9a150 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.767+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec87b60 0x55a16ec85470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.767+0000 7f6c39ffb640 0 client.20193 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.767+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec7a820 0x55a16ec84f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.768+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/1812215664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 0x55a16ec849f0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.768+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/1812215664 >> 172.21.15.18:0/1812215664 conn(0x55a16ec74980 msgr2=0x55a16ec766d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.768+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/1812215664 shutdown_connections 2024-02-17T13:02:47.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.768+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/1812215664 wait complete. 2024-02-17T13:02:47.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.768+0000 7f6c5f1c8740 20 client.20193 trim_cache size 0 max 16384 2024-02-17T13:02:47.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.779+0000 7f6c5f1c8740 1 Processor -- start 2024-02-17T13:02:47.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.779+0000 7f6c5f1c8740 1 -- start start 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c5f1c8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec79f00 0x55a16ec83b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c5f1c8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec7a820 0x55a16ec85290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c5f1c8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec87b60 0x55a16ec73a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c5f1c8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16eb67130 con 0x55a16ec7a820 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c5f1c8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16ec8a360 con 0x55a16ec87b60 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c5f1c8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16ec89f50 con 0x55a16ec79f00 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c59b8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec7a820 0x55a16ec85290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c59b8f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec7a820 0x55a16ec85290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53974/0 (socket says 172.21.15.18:53974) 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c59b8f640 1 -- 172.21.15.18:0/370216386 learned_addr learned my addr 172.21.15.18:0/370216386 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c59b8f640 1 -- 172.21.15.18:0/370216386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec79f00 msgr2=0x55a16ec83b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c5a390640 1 --2- 172.21.15.18:0/370216386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec79f00 0x55a16ec83b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/370216386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec79f00 0x55a16ec83b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c59b8f640 1 -- 172.21.15.18:0/370216386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec87b60 msgr2=0x55a16ec73a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c5ab91640 1 --2- 172.21.15.18:0/370216386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec87b60 0x55a16ec73a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/370216386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec87b60 0x55a16ec73a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c59b8f640 1 -- 172.21.15.18:0/370216386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a16eb7cc20 con 0x55a16ec7a820 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c5a390640 1 --2- 172.21.15.18:0/370216386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec79f00 0x55a16ec83b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/370216386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec7a820 0x55a16ec85290 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f6c48011b60 tx=0x7f6c4801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6032287779364250 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c3bfff640 1 -- 172.21.15.18:0/370216386 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48026040 con 0x55a16ec7a820 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c3bfff640 1 -- 172.21.15.18:0/370216386 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c48019070 con 0x55a16ec7a820 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.780+0000 7f6c3bfff640 1 -- 172.21.15.18:0/370216386 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48029730 con 0x55a16ec7a820 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/370216386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec7a820 msgr2=0x55a16ec85290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/370216386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec7a820 0x55a16ec85290 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f6c48011b60 tx=0x7f6c4801d000 comp rx=0 tx=0).stop 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/370216386 shutdown_connections 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/370216386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec87b60 0x55a16ec73a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/370216386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec7a820 0x55a16ec85290 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/370216386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec79f00 0x55a16ec83b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/370216386 >> 172.21.15.18:0/370216386 conn(0x55a16ec74980 msgr2=0x55a16ec732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/370216386 shutdown_connections 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/370216386 wait complete. 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 -- start start 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 Processor -- start 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 0x55a16ec8ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec7a820 0x55a16ec8ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec87b60 0x55a16ec8d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16ec8d900 con 0x55a16ec79f00 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16ec8da70 con 0x55a16ec7a820 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16ec8dc30 con 0x55a16ec87b60 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5f1c8740 10 client.0 _ll_register_callbacks cb 0x7f6c4c0042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.781+0000 7f6c5a390640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 0x55a16ec8ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c5a390640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 0x55a16ec8ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53976/0 (socket says 172.21.15.18:53976) 2024-02-17T13:02:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c5a390640 1 -- 172.21.15.18:0/618396254 learned_addr learned my addr 172.21.15.18:0/618396254 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c5ab91640 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec87b60 0x55a16ec8d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c5a390640 1 -- 172.21.15.18:0/618396254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec87b60 msgr2=0x55a16ec8d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c5a390640 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec87b60 0x55a16ec8d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec7a820 0x55a16ec8ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c5a390640 1 -- 172.21.15.18:0/618396254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec7a820 msgr2=0x55a16ec8ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c5a390640 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec7a820 0x55a16ec8ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c5a390640 1 -- 172.21.15.18:0/618396254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c480118d0 con 0x55a16ec79f00 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c5a390640 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 0x55a16ec8ea60 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c004280 tx=0x7f6c4c003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0240f251e2fb5af server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c39ffb640 1 -- 172.21.15.18:0/618396254 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c016040 con 0x55a16ec79f00 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c39ffb640 1 -- 172.21.15.18:0/618396254 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6c4c000e40 con 0x55a16ec79f00 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/618396254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c440bd090 con 0x55a16ec79f00 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c39ffb640 1 -- 172.21.15.18:0/618396254 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c4c01b470 con 0x55a16ec79f00 2024-02-17T13:02:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c5f1c8740 10 client.20199 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/618396254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a16eba28c0 con 0x55a16ec79f00 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.782+0000 7f6c39ffb640 1 -- 172.21.15.18:0/618396254 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c4c039020 con 0x55a16ec79f00 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c39ffb640 1 -- 172.21.15.18:0/618396254 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6c4c0244a0 con 0x55a16ec79f00 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c5f1c8740 10 client.20199 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c5f1c8740 10 client.20199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c5f1c8740 20 client.20199 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a16ec97d10 0x55a16ecf18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c5f1c8740 4 client.20199 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/618396254 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a16e96d890 con 0x55a16ec97d10 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a16ec97d10 0x55a16ecf18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c39ffb640 10 client.20199 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c59b8f640 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a16ec97d10 0x55a16ecf18d0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c54018ef514cdce7 server_cookie=78f25498775841e3 in_seq=0 out_seq=0 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c39ffb640 1 -- 172.21.15.18:0/618396254 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a16e96d890 con 0x55a16ec97d10 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c39ffb640 10 client.20199 handle_command_reply: tid=0 2024-02-17T13:02:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c5f1c8740 1 client.20199 shutdown 2024-02-17T13:02:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/618396254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 msgr2=0x55a16ec8ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.783+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 0x55a16ec8ea60 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c004280 tx=0x7f6c4c003fd0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/618396254 shutdown_connections 2024-02-17T13:02:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a16ec97d10 0x55a16ecf18d0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a16ec87b60 0x55a16ec8d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a16ec7a820 0x55a16ec8ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c39ffb640 0 client.20199 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5f1c8740 1 --2- 172.21.15.18:0/618396254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a16ec79f00 0x55a16ec8ea60 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/618396254 >> 172.21.15.18:0/618396254 conn(0x55a16ec74980 msgr2=0x55a16ec852d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/618396254 shutdown_connections 2024-02-17T13:02:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5f1c8740 1 -- 172.21.15.18:0/618396254 wait complete. 2024-02-17T13:02:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5f1c8740 20 client.20199 trim_cache size 0 max 16384 2024-02-17T13:02:47.781 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:02:47.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/1125537364 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6c3c0a2230 msgr2=0x7f6c3c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5c5f4640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6c3c0a2230 0x7f6c3c0a46f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800a5c0 tx=0x7f6c48013940 comp rx=0 tx=0).stop 2024-02-17T13:02:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/1125537364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c5419bfd0 msgr2=0x7f6c5419c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5c5f4640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c5419bfd0 0x7f6c5419c480 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f6c440348f0 tx=0x7f6c44034cc0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/1125537364 shutdown_connections 2024-02-17T13:02:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5c5f4640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6c3c0a2230 0x7f6c3c0a46f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5c5f4640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c5419bfd0 0x7f6c5419c480 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5c5f4640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c54105330 0x7f6c54197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5c5f4640 1 --2- 172.21.15.18:0/1125537364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c54100260 0x7f6c54197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/1125537364 >> 172.21.15.18:0/1125537364 conn(0x7f6c54077180 msgr2=0x7f6c540fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/1125537364 shutdown_connections 2024-02-17T13:02:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:47.784+0000 7f6c5c5f4640 1 -- 172.21.15.18:0/1125537364 wait complete. 2024-02-17T13:02:48.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:47 smithi146 ceph-mon[36454]: pgmap v405: 97 pgs: 97 active+clean; 254 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-02-17T13:02:48.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:47 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2254999348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:02:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.265+0000 7f88ba393640 1 -- 172.21.15.18:0/342914984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88b4101d70 msgr2=0x7f88b4105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.265+0000 7f88ba393640 1 --2- 172.21.15.18:0/342914984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88b4101d70 0x7f88b4105db0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009930 tx=0x7f88a402bc30 comp rx=0 tx=0).stop 2024-02-17T13:02:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.265+0000 7f88ba393640 1 -- 172.21.15.18:0/342914984 shutdown_connections 2024-02-17T13:02:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.265+0000 7f88ba393640 1 --2- 172.21.15.18:0/342914984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b4106490 0x7f88b410e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.265+0000 7f88ba393640 1 --2- 172.21.15.18:0/342914984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88b4101d70 0x7f88b4105db0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.265+0000 7f88ba393640 1 --2- 172.21.15.18:0/342914984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88b41013c0 0x7f88b41017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.265+0000 7f88ba393640 1 -- 172.21.15.18:0/342914984 >> 172.21.15.18:0/342914984 conn(0x7f88b40fcc60 msgr2=0x7f88b40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.265+0000 7f88ba393640 1 -- 172.21.15.18:0/342914984 shutdown_connections 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.265+0000 7f88ba393640 1 -- 172.21.15.18:0/342914984 wait complete. 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.266+0000 7f88ba393640 1 Processor -- start 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.266+0000 7f88ba393640 1 -- start start 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.266+0000 7f88ba393640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b41013c0 0x7f88b419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.266+0000 7f88ba393640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88b4106490 0x7f88b419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.266+0000 7f88ba393640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88b4197590 0x7f88b4197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.266+0000 7f88ba393640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b4197f80 con 0x7f88b4197590 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.266+0000 7f88ba393640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b41980f0 con 0x7f88b41013c0 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.266+0000 7f88ba393640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b41982b0 con 0x7f88b4106490 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.266+0000 7f88b8930640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88b4197590 0x7f88b4197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.266+0000 7f88b3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b41013c0 0x7f88b419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88b8930640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88b4197590 0x7f88b4197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53992/0 (socket says 172.21.15.18:53992) 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88b8930640 1 -- 172.21.15.18:0/2990049192 learned_addr learned my addr 172.21.15.18:0/2990049192 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88b37fe640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88b4106490 0x7f88b419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88b8930640 1 -- 172.21.15.18:0/2990049192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88b4106490 msgr2=0x7f88b419d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88b8930640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88b4106490 0x7f88b419d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88b8930640 1 -- 172.21.15.18:0/2990049192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b41013c0 msgr2=0x7f88b419d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88b8930640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b41013c0 0x7f88b419d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88b8930640 1 -- 172.21.15.18:0/2990049192 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a4009560 con 0x7f88b4197590 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88b8930640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88b4197590 0x7f88b4197a40 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00eaa0 tx=0x7f88ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88b17fa640 1 -- 172.21.15.18:0/2990049192 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac00cdb0 con 0x7f88b4197590 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88ba393640 1 -- 172.21.15.18:0/2990049192 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b41a42a0 con 0x7f88b4197590 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88b17fa640 1 -- 172.21.15.18:0/2990049192 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f88ac005020 con 0x7f88b4197590 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88b17fa640 1 -- 172.21.15.18:0/2990049192 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0116b0 con 0x7f88b4197590 2024-02-17T13:02:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.267+0000 7f88ba393640 1 -- 172.21.15.18:0/2990049192 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b41a4780 con 0x7f88b4197590 2024-02-17T13:02:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.269+0000 7f88b17fa640 1 -- 172.21.15.18:0/2990049192 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f88ac005520 con 0x7f88b4197590 2024-02-17T13:02:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88bcf69740 1 Processor -- start 2024-02-17T13:02:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88bcf69740 1 -- start start 2024-02-17T13:02:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88bcf69740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1796ad0 0x5605b1795820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88bcf69740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1795d60 0x5605b17961e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88b17fa640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f88a00a22d0 0x7f88a00a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88bcf69740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b179f2e0 0x5605b17a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88bcf69740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b17049f0 con 0x5605b1796ad0 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88bcf69740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b1701f20 con 0x5605b179f2e0 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88bcf69740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b16fd700 con 0x5605b1795d60 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88b37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1796ad0 0x5605b1795820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88b3fff640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f88a00a22d0 0x7f88a00a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88b37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1796ad0 0x5605b1795820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53998/0 (socket says 172.21.15.18:53998) 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88b37fe640 1 -- 172.21.15.18:0/2563774297 learned_addr learned my addr 172.21.15.18:0/2563774297 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88b17fa640 1 -- 172.21.15.18:0/2990049192 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f88ac0ce2f0 con 0x7f88b4197590 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88b37fe640 1 --2- 172.21.15.18:0/2563774297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1795d60 0x5605b17961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.270+0000 7f88b8930640 1 --2- 172.21.15.18:0/2563774297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b179f2e0 0x5605b17a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.271+0000 7f88b37fe640 1 -- 172.21.15.18:0/2563774297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1795d60 msgr2=0x5605b17961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.271+0000 7f88b37fe640 1 --2- 172.21.15.18:0/2563774297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1795d60 0x5605b17961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.271+0000 7f88b37fe640 1 -- 172.21.15.18:0/2563774297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b179f2e0 msgr2=0x5605b17a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.271+0000 7f88b37fe640 1 --2- 172.21.15.18:0/2563774297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b179f2e0 0x5605b17a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.271+0000 7f88b37fe640 1 -- 172.21.15.18:0/2563774297 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605b16937b0 con 0x5605b1796ad0 2024-02-17T13:02:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.271+0000 7f88b3fff640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f88a00a22d0 0x7f88a00a4790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f889c005d00 tx=0x7f889c00b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.271+0000 7f88b37fe640 1 --2- 172.21.15.18:0/2563774297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1796ad0 0x5605b1795820 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f88a4005940 tx=0x7f88a40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6966a0d1fdd359 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.271+0000 7f88aa7fc640 1 -- 172.21.15.18:0/2563774297 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a402c9e0 con 0x5605b1796ad0 2024-02-17T13:02:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.271+0000 7f88aa7fc640 1 -- 172.21.15.18:0/2563774297 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f88a4004510 con 0x5605b1796ad0 2024-02-17T13:02:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.271+0000 7f88aa7fc640 1 -- 172.21.15.18:0/2563774297 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4005610 con 0x5605b1796ad0 2024-02-17T13:02:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 -- 172.21.15.18:0/2563774297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1796ad0 msgr2=0x5605b1795820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 --2- 172.21.15.18:0/2563774297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1796ad0 0x5605b1795820 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f88a4005940 tx=0x7f88a40058b0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 -- 172.21.15.18:0/2563774297 shutdown_connections 2024-02-17T13:02:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 --2- 172.21.15.18:0/2563774297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b179f2e0 0x5605b17a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 --2- 172.21.15.18:0/2563774297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1795d60 0x5605b17961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 --2- 172.21.15.18:0/2563774297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1796ad0 0x5605b1795820 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 -- 172.21.15.18:0/2563774297 >> 172.21.15.18:0/2563774297 conn(0x5605b175d550 msgr2=0x5605b1790a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 -- 172.21.15.18:0/2563774297 shutdown_connections 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 -- 172.21.15.18:0/2563774297 wait complete. 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 -- start start 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 Processor -- start 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1795d60 0x5605b179c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b1796ad0 0x5605b179c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.272+0000 7f88bcf69740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 0x5605b179cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88bcf69740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b16a2bf0 con 0x5605b179f2e0 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88bcf69740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b13d1250 con 0x5605b1796ad0 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88bcf69740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b1683200 con 0x5605b1795d60 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88bcf69740 10 client.0 _ll_register_callbacks cb 0x5605b16f1d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88b8930640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 0x5605b179cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88b8930640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 0x5605b179cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54000/0 (socket says 172.21.15.18:54000) 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88b8930640 1 -- 172.21.15.18:0/876970624 learned_addr learned my addr 172.21.15.18:0/876970624 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88b3fff640 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1795d60 0x5605b179c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88b37fe640 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b1796ad0 0x5605b179c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88b8930640 1 -- 172.21.15.18:0/876970624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1795d60 msgr2=0x5605b179c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88b8930640 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1795d60 0x5605b179c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88b8930640 1 -- 172.21.15.18:0/876970624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b1796ad0 msgr2=0x5605b179c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88b8930640 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b1796ad0 0x5605b179c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88b8930640 1 -- 172.21.15.18:0/876970624 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b410e870 con 0x5605b179f2e0 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88b8930640 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 0x5605b179cda0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f88ac09ac30 tx=0x7f88ac09bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49863103ebe0892c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88937fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88937fe640 1 -- 172.21.15.18:0/876970624 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac005390 con 0x5605b179f2e0 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88937fe640 1 -- 172.21.15.18:0/876970624 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f88ac09bfd0 con 0x5605b179f2e0 2024-02-17T13:02:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88bcf69740 1 -- 172.21.15.18:0/876970624 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605b13fd600 con 0x5605b179f2e0 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88937fe640 1 -- 172.21.15.18:0/876970624 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88ac09edf0 con 0x5605b179f2e0 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88bcf69740 10 client.20211 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.273+0000 7f88bcf69740 1 -- 172.21.15.18:0/876970624 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5605b16fbd80 con 0x5605b179f2e0 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.274+0000 7f88937fe640 1 -- 172.21.15.18:0/876970624 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f88ac0b7020 con 0x5605b179f2e0 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.274+0000 7f88937fe640 1 -- 172.21.15.18:0/876970624 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f88ac018030 con 0x5605b179f2e0 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.274+0000 7f88bcf69740 10 client.20211 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.274+0000 7f88bcf69740 10 client.20211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.274+0000 7f88bcf69740 20 client.20211 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.274+0000 7f88bcf69740 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5605b17af650 0x5605b17b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.274+0000 7f88bcf69740 4 client.20211 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.274+0000 7f88bcf69740 1 -- 172.21.15.18:0/876970624 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605b144e1b0 con 0x5605b17af650 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.274+0000 7f88b3fff640 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5605b17af650 0x5605b17b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.275+0000 7f88b3fff640 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5605b17af650 0x5605b17b1a70 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=378a0d3b32ae01f8 server_cookie=83a47ace38653c1f in_seq=0 out_seq=0 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.275+0000 7f88937fe640 10 client.20211 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.275+0000 7f88937fe640 1 -- 172.21.15.18:0/876970624 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5605b144e1b0 con 0x5605b17af650 2024-02-17T13:02:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.275+0000 7f88937fe640 10 client.20211 handle_command_reply: tid=0 2024-02-17T13:02:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.275+0000 7f88bcf69740 1 client.20211 shutdown 2024-02-17T13:02:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.275+0000 7f88bcf69740 1 -- 172.21.15.18:0/876970624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 msgr2=0x5605b179cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.275+0000 7f88bcf69740 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 0x5605b179cda0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f88ac09ac30 tx=0x7f88ac09bec0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.275+0000 7f88bcf69740 1 -- 172.21.15.18:0/876970624 shutdown_connections 2024-02-17T13:02:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.275+0000 7f88bcf69740 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5605b17af650 0x5605b17b1a70 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.275+0000 7f88bcf69740 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 0x5605b179cda0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.275+0000 7f88937fe640 0 client.20211 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.275+0000 7f88bcf69740 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b1796ad0 0x5605b179c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.276+0000 7f88bcf69740 1 --2- 172.21.15.18:0/876970624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1795d60 0x5605b179c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.276+0000 7f88bcf69740 1 -- 172.21.15.18:0/876970624 >> 172.21.15.18:0/876970624 conn(0x5605b175d550 msgr2=0x5605b17927a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.276+0000 7f88bcf69740 1 -- 172.21.15.18:0/876970624 shutdown_connections 2024-02-17T13:02:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.276+0000 7f88bcf69740 1 -- 172.21.15.18:0/876970624 wait complete. 2024-02-17T13:02:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.276+0000 7f88bcf69740 20 client.20211 trim_cache size 0 max 16384 2024-02-17T13:02:48.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:47 smithi018 ceph-mon[32625]: pgmap v405: 97 pgs: 97 active+clean; 254 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-02-17T13:02:48.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:47 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2254999348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:02:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88bcf69740 1 Processor -- start 2024-02-17T13:02:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88bcf69740 1 -- start start 2024-02-17T13:02:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88bcf69740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b1795d60 0x5605b179ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88bcf69740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1796ad0 0x5605b179cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88bcf69740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 0x5605b1799580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88bcf69740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b144e1b0 con 0x5605b179f2e0 2024-02-17T13:02:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88bcf69740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b1791c80 con 0x5605b1795d60 2024-02-17T13:02:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88bcf69740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b17ac160 con 0x5605b1796ad0 2024-02-17T13:02:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88b8930640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 0x5605b1799580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88b8930640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 0x5605b1799580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54006/0 (socket says 172.21.15.18:54006) 2024-02-17T13:02:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88b8930640 1 -- 172.21.15.18:0/3821804883 learned_addr learned my addr 172.21.15.18:0/3821804883 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88b8930640 1 -- 172.21.15.18:0/3821804883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1796ad0 msgr2=0x5605b179cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88b8930640 1 --2- 172.21.15.18:0/3821804883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1796ad0 0x5605b179cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88b8930640 1 -- 172.21.15.18:0/3821804883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b1795d60 msgr2=0x5605b179ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88b8930640 1 --2- 172.21.15.18:0/3821804883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b1795d60 0x5605b179ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88b8930640 1 -- 172.21.15.18:0/3821804883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605b16987b0 con 0x5605b179f2e0 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.289+0000 7f88b8930640 1 --2- 172.21.15.18:0/3821804883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 0x5605b1799580 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0bc860 tx=0x7f88ac096f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d2b35a56b18bdc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.290+0000 7f88a9ffb640 1 -- 172.21.15.18:0/3821804883 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac005390 con 0x5605b179f2e0 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.290+0000 7f88a9ffb640 1 -- 172.21.15.18:0/3821804883 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f88ac0b7070 con 0x5605b179f2e0 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.290+0000 7f88a9ffb640 1 -- 172.21.15.18:0/3821804883 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac09aca0 con 0x5605b179f2e0 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.290+0000 7f88bcf69740 1 -- 172.21.15.18:0/3821804883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 msgr2=0x5605b1799580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.290+0000 7f88bcf69740 1 --2- 172.21.15.18:0/3821804883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 0x5605b1799580 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0bc860 tx=0x7f88ac096f40 comp rx=0 tx=0).stop 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.290+0000 7f88bcf69740 1 -- 172.21.15.18:0/3821804883 shutdown_connections 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.290+0000 7f88bcf69740 1 --2- 172.21.15.18:0/3821804883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b179f2e0 0x5605b1799580 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.290+0000 7f88bcf69740 1 --2- 172.21.15.18:0/3821804883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1796ad0 0x5605b179cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.290+0000 7f88bcf69740 1 --2- 172.21.15.18:0/3821804883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b1795d60 0x5605b179ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.290+0000 7f88bcf69740 1 -- 172.21.15.18:0/3821804883 >> 172.21.15.18:0/3821804883 conn(0x5605b175d550 msgr2=0x5605b179abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.290+0000 7f88bcf69740 1 -- 172.21.15.18:0/3821804883 shutdown_connections 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.290+0000 7f88bcf69740 1 -- 172.21.15.18:0/3821804883 wait complete. 2024-02-17T13:02:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88bcf69740 1 -- start start 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88bcf69740 1 Processor -- start 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88bcf69740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1795d60 0x5605b17a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88bcf69740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1796ad0 0x5605b17a4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88bcf69740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b179f2e0 0x5605b17a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88bcf69740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b17a5170 con 0x5605b1795d60 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88bcf69740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b17a52e0 con 0x5605b179f2e0 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88bcf69740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b17a54a0 con 0x5605b1796ad0 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88bcf69740 10 client.0 _ll_register_callbacks cb 0x7f88a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88b3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1795d60 0x5605b17a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88b3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1795d60 0x5605b17a62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54010/0 (socket says 172.21.15.18:54010) 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88b3fff640 1 -- 172.21.15.18:0/4123110225 learned_addr learned my addr 172.21.15.18:0/4123110225 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88b37fe640 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1796ad0 0x5605b17a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88b3fff640 1 -- 172.21.15.18:0/4123110225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1796ad0 msgr2=0x5605b17a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88b3fff640 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1796ad0 0x5605b17a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88b3fff640 1 -- 172.21.15.18:0/4123110225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b179f2e0 msgr2=0x5605b17a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.291+0000 7f88b8930640 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b179f2e0 0x5605b17a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88b3fff640 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b179f2e0 0x5605b17a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88b3fff640 1 -- 172.21.15.18:0/4123110225 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac0bae70 con 0x5605b1795d60 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88b37fe640 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1796ad0 0x5605b17a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88b8930640 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b179f2e0 0x5605b17a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88b3fff640 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1795d60 0x5605b17a62d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f889c011010 tx=0x7f889c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb4088a8c6c15e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88937fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88937fe640 1 -- 172.21.15.18:0/4123110225 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c025070 con 0x5605b1795d60 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88937fe640 1 -- 172.21.15.18:0/4123110225 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f889c017070 con 0x5605b1795d60 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88bcf69740 1 -- 172.21.15.18:0/4123110225 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88a4004510 con 0x5605b1795d60 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88937fe640 1 -- 172.21.15.18:0/4123110225 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f889c020500 con 0x5605b1795d60 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88bcf69740 10 client.20217 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88bcf69740 1 -- 172.21.15.18:0/4123110225 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5605b16be540 con 0x5605b1795d60 2024-02-17T13:02:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88937fe640 1 -- 172.21.15.18:0/4123110225 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f88a4004510 con 0x5605b1795d60 2024-02-17T13:02:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88937fe640 1 -- 172.21.15.18:0/4123110225 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f889c01a040 con 0x5605b1795d60 2024-02-17T13:02:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88bcf69740 10 client.20217 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88bcf69740 10 client.20217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88bcf69740 20 client.20217 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88bcf69740 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5605b18071d0 0x5605b18095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88bcf69740 4 client.20217 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:02:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.292+0000 7f88bcf69740 1 -- 172.21.15.18:0/4123110225 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5605b143a550 con 0x5605b18071d0 2024-02-17T13:02:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.293+0000 7f88b37fe640 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5605b18071d0 0x5605b18095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.293+0000 7f88937fe640 10 client.20217 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.293+0000 7f88b37fe640 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5605b18071d0 0x5605b18095f0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b98af0c9bb63c3e7 server_cookie=182ba09e4fcde3c3 in_seq=0 out_seq=0 2024-02-17T13:02:48.304 INFO:tasks.workunit.client.0.smithi018.stdout: 1 cleanup 603 sec 2024-02-17T13:02:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.336+0000 7f88937fe640 1 -- 172.21.15.18:0/4123110225 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5605b143a550 con 0x5605b18071d0 2024-02-17T13:02:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.336+0000 7f88937fe640 10 client.20217 handle_command_reply: tid=0 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.336+0000 7f88bcf69740 1 client.20217 shutdown 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.336+0000 7f88bcf69740 1 -- 172.21.15.18:0/4123110225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1795d60 msgr2=0x5605b17a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.336+0000 7f88bcf69740 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1795d60 0x5605b17a62d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f889c011010 tx=0x7f889c0046c0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88bcf69740 1 -- 172.21.15.18:0/4123110225 shutdown_connections 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88bcf69740 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5605b18071d0 0x5605b18095f0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88bcf69740 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605b179f2e0 0x5605b17a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88bcf69740 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1796ad0 0x5605b17a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88937fe640 0 client.20217 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88bcf69740 1 --2- 172.21.15.18:0/4123110225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5605b1795d60 0x5605b17a62d0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88bcf69740 1 -- 172.21.15.18:0/4123110225 >> 172.21.15.18:0/4123110225 conn(0x5605b175d550 msgr2=0x5605b178f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88bcf69740 1 -- 172.21.15.18:0/4123110225 shutdown_connections 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88bcf69740 1 -- 172.21.15.18:0/4123110225 wait complete. 2024-02-17T13:02:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88bcf69740 20 client.20217 trim_cache size 0 max 16384 2024-02-17T13:02:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:02:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:02:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:02:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:02:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88ba393640 1 -- 172.21.15.18:0/2990049192 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f88a00a22d0 msgr2=0x7f88a00a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88ba393640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f88a00a22d0 0x7f88a00a4790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f889c005d00 tx=0x7f889c00b410 comp rx=0 tx=0).stop 2024-02-17T13:02:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88ba393640 1 -- 172.21.15.18:0/2990049192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88b4197590 msgr2=0x7f88b4197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88ba393640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88b4197590 0x7f88b4197a40 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00eaa0 tx=0x7f88ac00ef60 comp rx=0 tx=0).stop 2024-02-17T13:02:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88ba393640 1 -- 172.21.15.18:0/2990049192 shutdown_connections 2024-02-17T13:02:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88ba393640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f88a00a22d0 0x7f88a00a4790 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88ba393640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88b4197590 0x7f88b4197a40 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88ba393640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88b4106490 0x7f88b419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88ba393640 1 --2- 172.21.15.18:0/2990049192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b41013c0 0x7f88b419d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88ba393640 1 -- 172.21.15.18:0/2990049192 >> 172.21.15.18:0/2990049192 conn(0x7f88b40fcc60 msgr2=0x7f88b40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88ba393640 1 -- 172.21.15.18:0/2990049192 shutdown_connections 2024-02-17T13:02:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:02:48.337+0000 7f88ba393640 1 -- 172.21.15.18:0/2990049192 wait complete. 2024-02-17T13:02:48.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:47 smithi119 ceph-mon[36188]: pgmap v405: 97 pgs: 97 active+clean; 254 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-02-17T13:02:48.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:47 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2254999348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:02:48.683 INFO:tasks.workunit.client.0.smithi018.stdout: 0 cleanup 603 sec 2024-02-17T13:02:48.684 INFO:tasks.workunit.client.0.smithi018.stdout: 2024-02-17T13:02:48.684 INFO:tasks.workunit.client.0.smithi018.stdout: Operation Count AvgLat MaxLat 2024-02-17T13:02:48.684 INFO:tasks.workunit.client.0.smithi018.stdout: ---------------------------------------- 2024-02-17T13:02:48.684 INFO:tasks.workunit.client.0.smithi018.stdout: NTCreateX 92515 1.432 4948.268 2024-02-17T13:02:48.684 INFO:tasks.workunit.client.0.smithi018.stdout: Close 67938 0.002 0.082 2024-02-17T13:02:48.684 INFO:tasks.workunit.client.0.smithi018.stdout: Rename 3919 71.866 4860.853 2024-02-17T13:02:48.684 INFO:tasks.workunit.client.0.smithi018.stdout: Unlink 18705 1.773 5590.249 2024-02-17T13:02:48.684 INFO:tasks.workunit.client.0.smithi018.stdout: Deltree 2 92.897 121.337 2024-02-17T13:02:48.684 INFO:tasks.workunit.client.0.smithi018.stdout: Mkdir 1 0.013 0.013 2024-02-17T13:02:48.684 INFO:tasks.workunit.client.0.smithi018.stdout: Qpathinfo 83832 1.460 4928.914 2024-02-17T13:02:48.684 INFO:tasks.workunit.client.0.smithi018.stdout: Qfileinfo 14668 0.002 0.034 2024-02-17T13:02:48.684 INFO:tasks.workunit.client.0.smithi018.stdout: Qfsinfo 15390 0.133 2.635 2024-02-17T13:02:48.685 INFO:tasks.workunit.client.0.smithi018.stdout: Sfileinfo 7538 0.011 4.368 2024-02-17T13:02:48.685 INFO:tasks.workunit.client.0.smithi018.stdout: Find 32425 0.246 2434.565 2024-02-17T13:02:48.685 INFO:tasks.workunit.client.0.smithi018.stdout: WriteX 46049 0.075 25.147 2024-02-17T13:02:48.685 INFO:tasks.workunit.client.0.smithi018.stdout: ReadX 144988 0.010 3.663 2024-02-17T13:02:48.685 INFO:tasks.workunit.client.0.smithi018.stdout: LockX 300 0.671 14.904 2024-02-17T13:02:48.685 INFO:tasks.workunit.client.0.smithi018.stdout: UnlockX 300 0.591 6.098 2024-02-17T13:02:48.685 INFO:tasks.workunit.client.0.smithi018.stdout: Flush 6490 2.577 82.248 2024-02-17T13:02:48.685 INFO:tasks.workunit.client.0.smithi018.stdout: 2024-02-17T13:02:48.685 INFO:tasks.workunit.client.0.smithi018.stdout:Throughput 4.83508 MB/sec 1 clients 1 procs max_latency=5590.260 ms 2024-02-17T13:02:48.685 INFO:tasks.workunit.client.0.smithi018.stderr:+ dbench 10 2024-02-17T13:02:48.686 INFO:tasks.workunit.client.0.smithi018.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-02-17T13:02:48.686 INFO:tasks.workunit.client.0.smithi018.stdout: 2024-02-17T13:02:48.686 INFO:tasks.workunit.client.0.smithi018.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-02-17T13:02:48.686 INFO:tasks.workunit.client.0.smithi018.stdout:8 of 10 processes prepared for launch 0 sec 2024-02-17T13:02:48.785 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T13:02:48.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:02:48.785 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T13:02:48.786 INFO:tasks.workunit.client.0.smithi018.stdout:10 of 10 processes prepared for launch 0 sec 2024-02-17T13:02:48.786 INFO:tasks.workunit.client.0.smithi018.stdout:releasing clients 2024-02-17T13:02:48.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.919+0000 7fc530965640 1 -- 172.21.15.18:0/2007251674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c104bf0 msgr2=0x7fc52c10c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.919+0000 7fc530965640 1 --2- 172.21.15.18:0/2007251674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c104bf0 0x7fc52c10c770 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fc524009ca0 tx=0x7fc52402c0f0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.920+0000 7fc530965640 1 -- 172.21.15.18:0/2007251674 shutdown_connections 2024-02-17T13:02:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.920+0000 7fc530965640 1 --2- 172.21.15.18:0/2007251674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c104bf0 0x7fc52c10c770 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.920+0000 7fc530965640 1 --2- 172.21.15.18:0/2007251674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc52c1006d0 0x7fc52c1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.920+0000 7fc530965640 1 --2- 172.21.15.18:0/2007251674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c0ffd20 0x7fc52c100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.920+0000 7fc530965640 1 -- 172.21.15.18:0/2007251674 >> 172.21.15.18:0/2007251674 conn(0x7fc52c075430 msgr2=0x7fc52c075840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.920+0000 7fc530965640 1 -- 172.21.15.18:0/2007251674 shutdown_connections 2024-02-17T13:02:48.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.920+0000 7fc530965640 1 -- 172.21.15.18:0/2007251674 wait complete. 2024-02-17T13:02:48.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc530965640 1 Processor -- start 2024-02-17T13:02:48.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc530965640 1 -- start start 2024-02-17T13:02:48.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc530965640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c0ffd20 0x7fc52c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc530965640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c104bf0 0x7fc52c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc530965640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc52c199130 0x7fc52c1a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc530965640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c1a3db0 con 0x7fc52c104bf0 2024-02-17T13:02:48.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc530965640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c1a3f20 con 0x7fc52c0ffd20 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc530965640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c1a40e0 con 0x7fc52c199130 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc529d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c104bf0 0x7fc52c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc529d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c104bf0 0x7fc52c198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54018/0 (socket says 172.21.15.18:54018) 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc529d9b640 1 -- 172.21.15.18:0/651195601 learned_addr learned my addr 172.21.15.18:0/651195601 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc52a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c0ffd20 0x7fc52c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc52ad9d640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc52c199130 0x7fc52c1a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc529d9b640 1 -- 172.21.15.18:0/651195601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc52c199130 msgr2=0x7fc52c1a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc529d9b640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc52c199130 0x7fc52c1a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc529d9b640 1 -- 172.21.15.18:0/651195601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c0ffd20 msgr2=0x7fc52c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc529d9b640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c0ffd20 0x7fc52c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc529d9b640 1 -- 172.21.15.18:0/651195601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc524009900 con 0x7fc52c104bf0 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.921+0000 7fc52a59c640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c0ffd20 0x7fc52c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.922+0000 7fc529d9b640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c104bf0 0x7fc52c198ba0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fc52000e9f0 tx=0x7fc52000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:48.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.922+0000 7fc51b7fe640 1 -- 172.21.15.18:0/651195601 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5200096c0 con 0x7fc52c104bf0 2024-02-17T13:02:48.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.922+0000 7fc530965640 1 -- 172.21.15.18:0/651195601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c1a43c0 con 0x7fc52c104bf0 2024-02-17T13:02:48.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.922+0000 7fc51b7fe640 1 -- 172.21.15.18:0/651195601 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc520005500 con 0x7fc52c104bf0 2024-02-17T13:02:48.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.922+0000 7fc51b7fe640 1 -- 172.21.15.18:0/651195601 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520010430 con 0x7fc52c104bf0 2024-02-17T13:02:48.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.922+0000 7fc530965640 1 -- 172.21.15.18:0/651195601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c1a4880 con 0x7fc52c104bf0 2024-02-17T13:02:48.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.924+0000 7fc51b7fe640 1 -- 172.21.15.18:0/651195601 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc520005020 con 0x7fc52c104bf0 2024-02-17T13:02:48.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc533614740 1 Processor -- start 2024-02-17T13:02:48.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc533614740 1 -- start start 2024-02-17T13:02:48.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc533614740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6380 0x55f38b0d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc533614740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0df2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc533614740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 0x55f38b0e1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc533614740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38b044f90 con 0x55f38b0df8b0 2024-02-17T13:02:48.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc533614740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38b0424c0 con 0x55f38b0d6380 2024-02-17T13:02:48.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc533614740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38b03dca0 con 0x55f38b0d6d30 2024-02-17T13:02:48.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc51b7fe640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5140a2230 0x7fc5140a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc52a59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0df2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc52ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6380 0x55f38b0d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc52a59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0df2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:51966/0 (socket says 172.21.15.18:51966) 2024-02-17T13:02:48.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc529d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 0x55f38b0e1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.925+0000 7fc52a59c640 1 -- 172.21.15.18:0/2466881326 learned_addr learned my addr 172.21.15.18:0/2466881326 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc52ad9d640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5140a2230 0x7fc5140a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc51b7fe640 1 -- 172.21.15.18:0/651195601 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc520014070 con 0x7fc52c104bf0 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc52a59c640 1 -- 172.21.15.18:0/2466881326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6380 msgr2=0x55f38b0d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc52a59c640 1 --2- 172.21.15.18:0/2466881326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6380 0x55f38b0d6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc52a59c640 1 -- 172.21.15.18:0/2466881326 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 msgr2=0x55f38b0e1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc52a59c640 1 --2- 172.21.15.18:0/2466881326 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 0x55f38b0e1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc52a59c640 1 -- 172.21.15.18:0/2466881326 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f38b0000b0 con 0x55f38b0d6d30 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc529d9b640 1 --2- 172.21.15.18:0/2466881326 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 0x55f38b0e1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc52a59c640 1 --2- 172.21.15.18:0/2466881326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0df2e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0115f0 tx=0x7fc51c011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c1bf7aaafd2d963 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc52ad9d640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5140a2230 0x7fc5140a46f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc524004af0 tx=0x7fc524005c50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc518ff9640 1 -- 172.21.15.18:0/2466881326 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00f640 con 0x55f38b0d6d30 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc518ff9640 1 -- 172.21.15.18:0/2466881326 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc51c00fc80 con 0x55f38b0d6d30 2024-02-17T13:02:48.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.926+0000 7fc518ff9640 1 -- 172.21.15.18:0/2466881326 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c019d70 con 0x55f38b0d6d30 2024-02-17T13:02:48.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.927+0000 7fc533614740 1 -- 172.21.15.18:0/2466881326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 msgr2=0x55f38b0df2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.927+0000 7fc533614740 1 --2- 172.21.15.18:0/2466881326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0df2e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0115f0 tx=0x7fc51c011ab0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.927+0000 7fc533614740 1 -- 172.21.15.18:0/2466881326 shutdown_connections 2024-02-17T13:02:48.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.927+0000 7fc533614740 1 --2- 172.21.15.18:0/2466881326 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 0x55f38b0e1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.927+0000 7fc533614740 1 --2- 172.21.15.18:0/2466881326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0df2e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.927+0000 7fc533614740 1 --2- 172.21.15.18:0/2466881326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6380 0x55f38b0d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.927+0000 7fc533614740 1 -- 172.21.15.18:0/2466881326 >> 172.21.15.18:0/2466881326 conn(0x55f38b0d0840 msgr2=0x55f38b0d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:48.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.927+0000 7fc533614740 1 -- 172.21.15.18:0/2466881326 shutdown_connections 2024-02-17T13:02:48.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.927+0000 7fc533614740 1 -- 172.21.15.18:0/2466881326 wait complete. 2024-02-17T13:02:48.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc533614740 1 -- start start 2024-02-17T13:02:48.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc533614740 1 Processor -- start 2024-02-17T13:02:48.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc533614740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 0x55f38b0dc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc533614740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6d30 0x55f38b0dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc533614740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0df8b0 0x55f38b0dd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc533614740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38afe2f40 con 0x55f38b0d6380 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc533614740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38ad114b0 con 0x55f38b0d6d30 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc533614740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38afc3670 con 0x55f38b0df8b0 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc533614740 10 client.0 _ll_register_callbacks cb 0x55f38b030f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc52a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 0x55f38b0dc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc529d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6d30 0x55f38b0dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc52a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 0x55f38b0dc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54040/0 (socket says 172.21.15.18:54040) 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc52a59c640 1 -- 172.21.15.18:0/1894778149 learned_addr learned my addr 172.21.15.18:0/1894778149 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc52ad9d640 1 --2- 172.21.15.18:0/1894778149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0df8b0 0x55f38b0dd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc52a59c640 1 -- 172.21.15.18:0/1894778149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0df8b0 msgr2=0x55f38b0dd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc52a59c640 1 --2- 172.21.15.18:0/1894778149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0df8b0 0x55f38b0dd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc52a59c640 1 -- 172.21.15.18:0/1894778149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6d30 msgr2=0x55f38b0dcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc52a59c640 1 --2- 172.21.15.18:0/1894778149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6d30 0x55f38b0dcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.928+0000 7fc52a59c640 1 -- 172.21.15.18:0/1894778149 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c009d40 con 0x55f38b0d6380 2024-02-17T13:02:48.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.929+0000 7fc52a59c640 1 --2- 172.21.15.18:0/1894778149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 0x55f38b0dc860 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fc51c01b000 tx=0x7fc51c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1f21c5594a404d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:48.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.929+0000 7fc511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:48.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.929+0000 7fc511ffb640 1 -- 172.21.15.18:0/1894778149 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c019710 con 0x55f38b0d6380 2024-02-17T13:02:48.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.929+0000 7fc511ffb640 1 -- 172.21.15.18:0/1894778149 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc51c019410 con 0x55f38b0d6380 2024-02-17T13:02:48.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.929+0000 7fc533614740 1 -- 172.21.15.18:0/1894778149 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f38b017f40 con 0x55f38b0d6380 2024-02-17T13:02:48.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.930+0000 7fc511ffb640 1 -- 172.21.15.18:0/1894778149 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc51c004030 con 0x55f38b0d6380 2024-02-17T13:02:48.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.930+0000 7fc533614740 10 client.20229 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.930+0000 7fc533614740 1 -- 172.21.15.18:0/1894778149 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55f38b03c320 con 0x55f38b0d6380 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.930+0000 7fc511ffb640 1 -- 172.21.15.18:0/1894778149 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f38b017f40 con 0x55f38b0d6380 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.930+0000 7fc511ffb640 1 -- 172.21.15.18:0/1894778149 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc51c00f640 con 0x55f38b0d6380 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.930+0000 7fc533614740 10 client.20229 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.930+0000 7fc533614740 10 client.20229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.930+0000 7fc533614740 20 client.20229 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.930+0000 7fc533614740 1 --2- 172.21.15.18:0/1894778149 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f38b0efcc0 0x55f38b0f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.930+0000 7fc533614740 4 client.20229 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.930+0000 7fc533614740 1 -- 172.21.15.18:0/1894778149 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f38b034310 con 0x55f38b0efcc0 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.930+0000 7fc529d9b640 1 --2- 172.21.15.18:0/1894778149 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f38b0efcc0 0x55f38b0f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.931+0000 7fc529d9b640 1 --2- 172.21.15.18:0/1894778149 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f38b0efcc0 0x55f38b0f20b0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e933f4ba7f43eec1 server_cookie=90d2623d0517fcba in_seq=0 out_seq=0 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.931+0000 7fc511ffb640 10 client.20229 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc511ffb640 1 -- 172.21.15.18:0/1894778149 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f38b034310 con 0x55f38b0efcc0 2024-02-17T13:02:48.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc511ffb640 10 client.20229 handle_command_reply: tid=0 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc533614740 1 client.20229 shutdown 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc533614740 1 -- 172.21.15.18:0/1894778149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 msgr2=0x55f38b0dc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc533614740 1 --2- 172.21.15.18:0/1894778149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 0x55f38b0dc860 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fc51c01b000 tx=0x7fc51c005270 comp rx=0 tx=0).stop 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc533614740 1 -- 172.21.15.18:0/1894778149 shutdown_connections 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc533614740 1 --2- 172.21.15.18:0/1894778149 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f38b0efcc0 0x55f38b0f20b0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc533614740 1 --2- 172.21.15.18:0/1894778149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0df8b0 0x55f38b0dd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc533614740 1 --2- 172.21.15.18:0/1894778149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6d30 0x55f38b0dcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc511ffb640 0 client.20229 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc533614740 1 --2- 172.21.15.18:0/1894778149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 0x55f38b0dc860 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc533614740 1 -- 172.21.15.18:0/1894778149 >> 172.21.15.18:0/1894778149 conn(0x55f38b0d0840 msgr2=0x55f38b0d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc533614740 1 -- 172.21.15.18:0/1894778149 shutdown_connections 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc533614740 1 -- 172.21.15.18:0/1894778149 wait complete. 2024-02-17T13:02:48.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.932+0000 7fc533614740 20 client.20229 trim_cache size 0 max 16384 2024-02-17T13:02:48.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.950+0000 7fc533614740 1 Processor -- start 2024-02-17T13:02:48.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.950+0000 7fc533614740 1 -- start start 2024-02-17T13:02:48.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc533614740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6380 0x55f38b0d9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc533614740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0da000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc533614740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 0x55f38b0dc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc533614740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38b0d2d60 con 0x55f38b0df8b0 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc533614740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38b0d00a0 con 0x55f38b0d6380 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc533614740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38b0d5410 con 0x55f38b0d6d30 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc52ad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 0x55f38b0dc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc52a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6380 0x55f38b0d9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc52a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6380 0x55f38b0d9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:50716/0 (socket says 172.21.15.18:50716) 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc52ad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 0x55f38b0dc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54046/0 (socket says 172.21.15.18:54046) 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc52a59c640 1 -- 172.21.15.18:0/541258300 learned_addr learned my addr 172.21.15.18:0/541258300 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc529d9b640 1 --2- 172.21.15.18:0/541258300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0da000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc52ad9d640 1 -- 172.21.15.18:0/541258300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 msgr2=0x55f38b0da000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc52ad9d640 1 --2- 172.21.15.18:0/541258300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0da000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc52ad9d640 1 -- 172.21.15.18:0/541258300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6380 msgr2=0x55f38b0d9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc52ad9d640 1 --2- 172.21.15.18:0/541258300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6380 0x55f38b0d9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc52ad9d640 1 -- 172.21.15.18:0/541258300 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f38afd8c00 con 0x55f38b0df8b0 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc52a59c640 1 --2- 172.21.15.18:0/541258300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6380 0x55f38b0d9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.951+0000 7fc52ad9d640 1 --2- 172.21.15.18:0/541258300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 0x55f38b0dc820 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fc52402c700 tx=0x7fc524009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f75ee9e716a5ea server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.952+0000 7fc513fff640 1 -- 172.21.15.18:0/541258300 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524034c90 con 0x55f38b0df8b0 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.952+0000 7fc513fff640 1 -- 172.21.15.18:0/541258300 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc524034df0 con 0x55f38b0df8b0 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.952+0000 7fc513fff640 1 -- 172.21.15.18:0/541258300 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5240042a0 con 0x55f38b0df8b0 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.952+0000 7fc533614740 1 -- 172.21.15.18:0/541258300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 msgr2=0x55f38b0dc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.952+0000 7fc533614740 1 --2- 172.21.15.18:0/541258300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 0x55f38b0dc820 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fc52402c700 tx=0x7fc524009360 comp rx=0 tx=0).stop 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.952+0000 7fc533614740 1 -- 172.21.15.18:0/541258300 shutdown_connections 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 --2- 172.21.15.18:0/541258300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0df8b0 0x55f38b0dc820 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 --2- 172.21.15.18:0/541258300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0da000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 --2- 172.21.15.18:0/541258300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0d6380 0x55f38b0d9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 -- 172.21.15.18:0/541258300 >> 172.21.15.18:0/541258300 conn(0x55f38b0d0840 msgr2=0x55f38b0db130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 -- 172.21.15.18:0/541258300 shutdown_connections 2024-02-17T13:02:48.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 -- 172.21.15.18:0/541258300 wait complete. 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 -- start start 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 Processor -- start 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 0x55f38b0e6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0df8b0 0x55f38b143850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38b143d90 con 0x55f38b0d6380 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38b143f00 con 0x55f38b0df8b0 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38b1440c0 con 0x55f38b0d6d30 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc533614740 10 client.0 _ll_register_callbacks cb 0x7fc51c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc52a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 0x55f38b0e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc52a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 0x55f38b0e6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54054/0 (socket says 172.21.15.18:54054) 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc52a59c640 1 -- 172.21.15.18:0/2773331314 learned_addr learned my addr 172.21.15.18:0/2773331314 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc529d9b640 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.953+0000 7fc52ad9d640 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0df8b0 0x55f38b143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc52a59c640 1 -- 172.21.15.18:0/2773331314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 msgr2=0x55f38b0e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc52a59c640 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0e6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc52a59c640 1 -- 172.21.15.18:0/2773331314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0df8b0 msgr2=0x55f38b143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc52a59c640 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0df8b0 0x55f38b143850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc52a59c640 1 -- 172.21.15.18:0/2773331314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f38b03c320 con 0x55f38b0d6380 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc529d9b640 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc52a59c640 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 0x55f38b0e6050 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0115c0 tx=0x7fc51c01ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f47881f455a043 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc511ffb640 1 -- 172.21.15.18:0/2773331314 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00f3c0 con 0x55f38b0d6380 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc511ffb640 1 -- 172.21.15.18:0/2773331314 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc51c019410 con 0x55f38b0d6380 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc533614740 1 -- 172.21.15.18:0/2773331314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f38b063900 con 0x55f38b0d6380 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc511ffb640 1 -- 172.21.15.18:0/2773331314 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc51c005d30 con 0x55f38b0d6380 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc533614740 10 client.20235 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc533614740 1 -- 172.21.15.18:0/2773331314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fc52c10cf60 con 0x55f38b0d6380 2024-02-17T13:02:48.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.954+0000 7fc511ffb640 1 -- 172.21.15.18:0/2773331314 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f38b063900 con 0x55f38b0d6380 2024-02-17T13:02:48.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.955+0000 7fc511ffb640 1 -- 172.21.15.18:0/2773331314 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc51c034430 con 0x55f38b0d6380 2024-02-17T13:02:48.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.955+0000 7fc533614740 10 client.20235 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:48.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.955+0000 7fc533614740 10 client.20235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:48.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.955+0000 7fc533614740 20 client.20235 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:48.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.955+0000 7fc533614740 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f38b14c830 0x55f38b14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:48.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.955+0000 7fc533614740 4 client.20235 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T13:02:48.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.955+0000 7fc533614740 1 -- 172.21.15.18:0/2773331314 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f38ad3d860 con 0x55f38b14c830 2024-02-17T13:02:48.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.955+0000 7fc529d9b640 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f38b14c830 0x55f38b14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:48.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.955+0000 7fc511ffb640 10 client.20235 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:48.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.955+0000 7fc529d9b640 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f38b14c830 0x55f38b14ec50 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=332456fa61065a9 server_cookie=191553dbaa142cfa in_seq=0 out_seq=0 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.957+0000 7fc511ffb640 1 -- 172.21.15.18:0/2773331314 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f38ad3d860 con 0x55f38b14c830 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.957+0000 7fc511ffb640 10 client.20235 handle_command_reply: tid=0 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.957+0000 7fc533614740 1 client.20235 shutdown 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.957+0000 7fc533614740 1 -- 172.21.15.18:0/2773331314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 msgr2=0x55f38b0e6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.957+0000 7fc533614740 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 0x55f38b0e6050 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0115c0 tx=0x7fc51c01ae80 comp rx=0 tx=0).stop 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.957+0000 7fc533614740 1 -- 172.21.15.18:0/2773331314 shutdown_connections 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.957+0000 7fc533614740 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f38b14c830 0x55f38b14ec50 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.957+0000 7fc533614740 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f38b0df8b0 0x55f38b143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.957+0000 7fc533614740 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f38b0d6d30 0x55f38b0e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.957+0000 7fc533614740 1 --2- 172.21.15.18:0/2773331314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f38b0d6380 0x55f38b0e6050 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.957+0000 7fc533614740 1 -- 172.21.15.18:0/2773331314 >> 172.21.15.18:0/2773331314 conn(0x55f38b0d0840 msgr2=0x55f38b0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.957+0000 7fc511ffb640 0 client.20235 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:48.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc533614740 1 -- 172.21.15.18:0/2773331314 shutdown_connections 2024-02-17T13:02:48.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc533614740 1 -- 172.21.15.18:0/2773331314 wait complete. 2024-02-17T13:02:48.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc533614740 20 client.20235 trim_cache size 0 max 16384 2024-02-17T13:02:48.953 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:02:48.953 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T13:02:48.953 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "72ca207e-7924-4fa4-9970-f36c1805dcd8", 2024-02-17T13:02:48.953 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T13:02:48.953 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:02:48.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc530965640 1 -- 172.21.15.18:0/651195601 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5140a2230 msgr2=0x7fc5140a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc530965640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5140a2230 0x7fc5140a46f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc524004af0 tx=0x7fc524005c50 comp rx=0 tx=0).stop 2024-02-17T13:02:48.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc530965640 1 -- 172.21.15.18:0/651195601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c104bf0 msgr2=0x7fc52c198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:48.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc530965640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c104bf0 0x7fc52c198ba0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fc52000e9f0 tx=0x7fc52000eeb0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc530965640 1 -- 172.21.15.18:0/651195601 shutdown_connections 2024-02-17T13:02:48.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc530965640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5140a2230 0x7fc5140a46f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc530965640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc52c199130 0x7fc52c1a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc530965640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c104bf0 0x7fc52c198ba0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc530965640 1 --2- 172.21.15.18:0/651195601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c0ffd20 0x7fc52c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc530965640 1 -- 172.21.15.18:0/651195601 >> 172.21.15.18:0/651195601 conn(0x7fc52c075430 msgr2=0x7fc52c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc530965640 1 -- 172.21.15.18:0/651195601 shutdown_connections 2024-02-17T13:02:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:48.958+0000 7fc530965640 1 -- 172.21.15.18:0/651195601 wait complete. 2024-02-17T13:02:48.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:02:48.959 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:02:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.086+0000 7fa035963640 1 -- 172.21.15.18:0/3764172167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa030100c20 msgr2=0x7fa030104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.086+0000 7fa035963640 1 --2- 172.21.15.18:0/3764172167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa030100c20 0x7fa030104c60 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa018009930 tx=0x7fa01802bc30 comp rx=0 tx=0).stop 2024-02-17T13:02:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.086+0000 7fa035963640 1 -- 172.21.15.18:0/3764172167 shutdown_connections 2024-02-17T13:02:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.086+0000 7fa035963640 1 --2- 172.21.15.18:0/3764172167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030105340 0x7fa03010cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.086+0000 7fa035963640 1 --2- 172.21.15.18:0/3764172167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa030100c20 0x7fa030104c60 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.086+0000 7fa035963640 1 --2- 172.21.15.18:0/3764172167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa030100270 0x7fa030100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.086+0000 7fa035963640 1 -- 172.21.15.18:0/3764172167 >> 172.21.15.18:0/3764172167 conn(0x7fa030077230 msgr2=0x7fa030077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.086+0000 7fa035963640 1 -- 172.21.15.18:0/3764172167 shutdown_connections 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.086+0000 7fa035963640 1 -- 172.21.15.18:0/3764172167 wait complete. 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa035963640 1 Processor -- start 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa035963640 1 -- start start 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa035963640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa030100270 0x7fa030101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa035963640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030105340 0x7fa030102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa035963640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa030102780 0x7fa0301a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa035963640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301a5ef0 con 0x7fa030102780 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa035963640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301a6060 con 0x7fa030105340 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa035963640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301a6220 con 0x7fa030100270 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa02f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa030102780 0x7fa0301a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa02effd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa030100270 0x7fa030101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa02f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa030102780 0x7fa0301a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54078/0 (socket says 172.21.15.18:54078) 2024-02-17T13:02:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa02f7fe640 1 -- 172.21.15.18:0/2192989112 learned_addr learned my addr 172.21.15.18:0/2192989112 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.087+0000 7fa027fff640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030105340 0x7fa030102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa027fff640 1 -- 172.21.15.18:0/2192989112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa030100270 msgr2=0x7fa030101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa027fff640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa030100270 0x7fa030101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa027fff640 1 -- 172.21.15.18:0/2192989112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa030102780 msgr2=0x7fa0301a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa027fff640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa030102780 0x7fa0301a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa027fff640 1 -- 172.21.15.18:0/2192989112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa018009560 con 0x7fa030105340 2024-02-17T13:02:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa02effd640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa030100270 0x7fa030101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa02f7fe640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa030102780 0x7fa0301a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa027fff640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030105340 0x7fa030102240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa01802c170 tx=0x7fa018005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa034961640 1 -- 172.21.15.18:0/2192989112 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018004350 con 0x7fa030105340 2024-02-17T13:02:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa035963640 1 -- 172.21.15.18:0/2192989112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0301a64a0 con 0x7fa030105340 2024-02-17T13:02:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa034961640 1 -- 172.21.15.18:0/2192989112 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0180044b0 con 0x7fa030105340 2024-02-17T13:02:49.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa034961640 1 -- 172.21.15.18:0/2192989112 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018003cd0 con 0x7fa030105340 2024-02-17T13:02:49.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.088+0000 7fa035963640 1 -- 172.21.15.18:0/2192989112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0301a6980 con 0x7fa030105340 2024-02-17T13:02:49.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa034961640 1 -- 172.21.15.18:0/2192989112 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa018004df0 con 0x7fa030105340 2024-02-17T13:02:49.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa03860c740 1 Processor -- start 2024-02-17T13:02:49.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa03860c740 1 -- start start 2024-02-17T13:02:49.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa03860c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 0x56047e2e9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa03860c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2f2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa03860c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e2f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa03860c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047e258280 con 0x56047e2e92b0 2024-02-17T13:02:49.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa03860c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047e2557b0 con 0x56047e2f27b0 2024-02-17T13:02:49.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa03860c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047e250f90 con 0x56047e2e9bd0 2024-02-17T13:02:49.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa034961640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa01c0a2260 0x7fa01c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa02f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 0x56047e2e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa02f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 0x56047e2e9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54080/0 (socket says 172.21.15.18:54080) 2024-02-17T13:02:49.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.091+0000 7fa02f7fe640 1 -- 172.21.15.18:0/1177544500 learned_addr learned my addr 172.21.15.18:0/1177544500 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:49.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa034961640 1 -- 172.21.15.18:0/2192989112 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa0180eff10 con 0x7fa030105340 2024-02-17T13:02:49.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa02effd640 1 --2- 172.21.15.18:0/1177544500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa027fff640 1 --2- 172.21.15.18:0/1177544500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e2f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa02f7fe640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa01c0a2260 0x7fa01c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa02f7fe640 1 -- 172.21.15.18:0/1177544500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 msgr2=0x56047e2f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa02f7fe640 1 --2- 172.21.15.18:0/1177544500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2f2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa02f7fe640 1 -- 172.21.15.18:0/1177544500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 msgr2=0x56047e2f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa02f7fe640 1 --2- 172.21.15.18:0/1177544500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e2f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa02f7fe640 1 -- 172.21.15.18:0/1177544500 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56047e20f570 con 0x56047e2e92b0 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa027fff640 1 --2- 172.21.15.18:0/1177544500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e2f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa02f7fe640 1 --2- 172.21.15.18:0/1177544500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 0x56047e2e9690 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa028013f10 tx=0x7fa028011820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a370264fa58835 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa025ffb640 1 -- 172.21.15.18:0/1177544500 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028013910 con 0x56047e2e92b0 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa025ffb640 1 -- 172.21.15.18:0/1177544500 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa02800ee10 con 0x56047e2e92b0 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.092+0000 7fa025ffb640 1 -- 172.21.15.18:0/1177544500 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02800f600 con 0x56047e2e92b0 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.093+0000 7fa03860c740 1 -- 172.21.15.18:0/1177544500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 msgr2=0x56047e2e9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.093+0000 7fa02f7fe640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa01c0a2260 0x7fa01c0a4720 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa028022000 tx=0x7fa02801e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.093+0000 7fa03860c740 1 --2- 172.21.15.18:0/1177544500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 0x56047e2e9690 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa028013f10 tx=0x7fa028011820 comp rx=0 tx=0).stop 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.093+0000 7fa03860c740 1 -- 172.21.15.18:0/1177544500 shutdown_connections 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.093+0000 7fa03860c740 1 --2- 172.21.15.18:0/1177544500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e2f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.093+0000 7fa03860c740 1 --2- 172.21.15.18:0/1177544500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.093+0000 7fa03860c740 1 --2- 172.21.15.18:0/1177544500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 0x56047e2e9690 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.093+0000 7fa03860c740 1 -- 172.21.15.18:0/1177544500 >> 172.21.15.18:0/1177544500 conn(0x56047e2e3990 msgr2=0x56047e2e3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.093+0000 7fa03860c740 1 -- 172.21.15.18:0/1177544500 shutdown_connections 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.093+0000 7fa03860c740 1 -- 172.21.15.18:0/1177544500 wait complete. 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.093+0000 7fa03860c740 1 -- start start 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa03860c740 1 Processor -- start 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa03860c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2e92b0 0x56047e2f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa03860c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa03860c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2f27b0 0x56047e2efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa03860c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047df24430 con 0x56047e2f27b0 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa03860c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047dee0e00 con 0x56047e2e92b0 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa03860c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047e1d5fd0 con 0x56047e2e9bd0 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa03860c740 10 client.0 _ll_register_callbacks cb 0x56047e22bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa02f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2f27b0 0x56047e2efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa02effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2e92b0 0x56047e2f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa027fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa027fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2ef5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:52008/0 (socket says 172.21.15.18:52008) 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa02effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2e92b0 0x56047e2f2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:50772/0 (socket says 172.21.15.18:50772) 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.094+0000 7fa027fff640 1 -- 172.21.15.18:0/2264893751 learned_addr learned my addr 172.21.15.18:0/2264893751 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa02f7fe640 1 -- 172.21.15.18:0/2264893751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 msgr2=0x56047e2ef5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa02f7fe640 1 --2- 172.21.15.18:0/2264893751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2ef5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa02f7fe640 1 -- 172.21.15.18:0/2264893751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2e92b0 msgr2=0x56047e2f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa02f7fe640 1 --2- 172.21.15.18:0/2264893751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2e92b0 0x56047e2f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa02f7fe640 1 -- 172.21.15.18:0/2264893751 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa028013af0 con 0x56047e2f27b0 2024-02-17T13:02:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa02effd640 1 --2- 172.21.15.18:0/2264893751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2e92b0 0x56047e2f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa02f7fe640 1 --2- 172.21.15.18:0/2264893751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2f27b0 0x56047e2efcf0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fa028011d20 tx=0x7fa028004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1690c2eb9997335f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa00effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa00effd640 1 -- 172.21.15.18:0/2264893751 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02800fcf0 con 0x56047e2f27b0 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa00effd640 1 -- 172.21.15.18:0/2264893751 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa028013910 con 0x56047e2f27b0 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa03860c740 1 -- 172.21.15.18:0/2264893751 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56047df507e0 con 0x56047e2f27b0 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa00effd640 1 -- 172.21.15.18:0/2264893751 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa028004e70 con 0x56047e2f27b0 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa03860c740 10 client.20247 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa03860c740 1 -- 172.21.15.18:0/2264893751 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x56047e24f610 con 0x56047e2f27b0 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.095+0000 7fa00effd640 1 -- 172.21.15.18:0/2264893751 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa02803e020 con 0x56047e2f27b0 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.096+0000 7fa00effd640 1 -- 172.21.15.18:0/2264893751 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa028005160 con 0x56047e2f27b0 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.096+0000 7fa03860c740 10 client.20247 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.096+0000 7fa03860c740 10 client.20247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.096+0000 7fa03860c740 20 client.20247 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.096+0000 7fa03860c740 1 --2- 172.21.15.18:0/2264893751 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56047e302810 0x56047e304c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.096+0000 7fa03860c740 4 client.20247 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:02:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.096+0000 7fa03860c740 1 -- 172.21.15.18:0/2264893751 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56047df8d720 con 0x56047e302810 2024-02-17T13:02:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.096+0000 7fa02effd640 1 --2- 172.21.15.18:0/2264893751 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56047e302810 0x56047e304c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.096+0000 7fa02effd640 1 --2- 172.21.15.18:0/2264893751 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56047e302810 0x56047e304c00 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d430d56ed358e93b server_cookie=4cffb9289dac94c9 in_seq=0 out_seq=0 2024-02-17T13:02:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.096+0000 7fa00effd640 10 client.20247 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.098+0000 7fa00effd640 1 -- 172.21.15.18:0/2264893751 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56047df8d720 con 0x56047e302810 2024-02-17T13:02:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.098+0000 7fa00effd640 10 client.20247 handle_command_reply: tid=0 2024-02-17T13:02:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.098+0000 7fa03860c740 1 client.20247 shutdown 2024-02-17T13:02:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.098+0000 7fa03860c740 1 -- 172.21.15.18:0/2264893751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2f27b0 msgr2=0x56047e2efcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.098+0000 7fa03860c740 1 --2- 172.21.15.18:0/2264893751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2f27b0 0x56047e2efcf0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fa028011d20 tx=0x7fa028004c80 comp rx=0 tx=0).stop 2024-02-17T13:02:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.098+0000 7fa03860c740 1 -- 172.21.15.18:0/2264893751 shutdown_connections 2024-02-17T13:02:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.098+0000 7fa03860c740 1 --2- 172.21.15.18:0/2264893751 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56047e302810 0x56047e304c00 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.098+0000 7fa00effd640 0 client.20247 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.098+0000 7fa03860c740 1 --2- 172.21.15.18:0/2264893751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2f27b0 0x56047e2efcf0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.098+0000 7fa03860c740 1 --2- 172.21.15.18:0/2264893751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2ef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.098+0000 7fa03860c740 1 --2- 172.21.15.18:0/2264893751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2e92b0 0x56047e2f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.098+0000 7fa03860c740 1 -- 172.21.15.18:0/2264893751 >> 172.21.15.18:0/2264893751 conn(0x56047e2e3990 msgr2=0x56047e2e5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.099+0000 7fa03860c740 1 -- 172.21.15.18:0/2264893751 shutdown_connections 2024-02-17T13:02:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.099+0000 7fa03860c740 1 -- 172.21.15.18:0/2264893751 wait complete. 2024-02-17T13:02:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.099+0000 7fa03860c740 20 client.20247 trim_cache size 0 max 16384 2024-02-17T13:02:49.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.114+0000 7fa03860c740 1 Processor -- start 2024-02-17T13:02:49.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.114+0000 7fa03860c740 1 -- start start 2024-02-17T13:02:49.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.114+0000 7fa03860c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e92b0 0x56047e2ef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.114+0000 7fa03860c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e9bd0 0x56047e2f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.114+0000 7fa03860c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e2f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.114+0000 7fa03860c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047e2df0f0 con 0x56047e2e9bd0 2024-02-17T13:02:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.114+0000 7fa03860c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047e302610 con 0x56047e2f27b0 2024-02-17T13:02:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.114+0000 7fa03860c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047e2e8f10 con 0x56047e2e92b0 2024-02-17T13:02:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa027fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e9bd0 0x56047e2f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa027fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e9bd0 0x56047e2f1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54088/0 (socket says 172.21.15.18:54088) 2024-02-17T13:02:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa027fff640 1 -- 172.21.15.18:0/4140864790 learned_addr learned my addr 172.21.15.18:0/4140864790 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa02f7fe640 1 --2- 172.21.15.18:0/4140864790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e2f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa02effd640 1 --2- 172.21.15.18:0/4140864790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e92b0 0x56047e2ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa027fff640 1 -- 172.21.15.18:0/4140864790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e92b0 msgr2=0x56047e2ef550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa027fff640 1 --2- 172.21.15.18:0/4140864790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e92b0 0x56047e2ef550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa027fff640 1 -- 172.21.15.18:0/4140864790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 msgr2=0x56047e2f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa027fff640 1 --2- 172.21.15.18:0/4140864790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e2f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa027fff640 1 -- 172.21.15.18:0/4140864790 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56047e1ebac0 con 0x56047e2e9bd0 2024-02-17T13:02:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa02f7fe640 1 --2- 172.21.15.18:0/4140864790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e2f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:02:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa027fff640 1 --2- 172.21.15.18:0/4140864790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e9bd0 0x56047e2f1ce0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fa0180bc730 tx=0x7fa01802c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258e433d27885074 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa0257fa640 1 -- 172.21.15.18:0/4140864790 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0180b8cd0 con 0x56047e2e9bd0 2024-02-17T13:02:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa0257fa640 1 -- 172.21.15.18:0/4140864790 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0180035f0 con 0x56047e2e9bd0 2024-02-17T13:02:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.115+0000 7fa0257fa640 1 -- 172.21.15.18:0/4140864790 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0180c51d0 con 0x56047e2e9bd0 2024-02-17T13:02:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.116+0000 7fa03860c740 1 -- 172.21.15.18:0/4140864790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e9bd0 msgr2=0x56047e2f1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.116+0000 7fa03860c740 1 --2- 172.21.15.18:0/4140864790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e9bd0 0x56047e2f1ce0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fa0180bc730 tx=0x7fa01802c450 comp rx=0 tx=0).stop 2024-02-17T13:02:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.116+0000 7fa03860c740 1 -- 172.21.15.18:0/4140864790 shutdown_connections 2024-02-17T13:02:49.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.116+0000 7fa03860c740 1 --2- 172.21.15.18:0/4140864790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e2f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.116+0000 7fa03860c740 1 --2- 172.21.15.18:0/4140864790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e9bd0 0x56047e2f1ce0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.116+0000 7fa03860c740 1 --2- 172.21.15.18:0/4140864790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e92b0 0x56047e2ef550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.116+0000 7fa03860c740 1 -- 172.21.15.18:0/4140864790 >> 172.21.15.18:0/4140864790 conn(0x56047e2e3990 msgr2=0x56047e2eee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:49.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.116+0000 7fa03860c740 1 -- 172.21.15.18:0/4140864790 shutdown_connections 2024-02-17T13:02:49.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.116+0000 7fa03860c740 1 -- 172.21.15.18:0/4140864790 wait complete. 2024-02-17T13:02:49.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.116+0000 7fa03860c740 1 -- start start 2024-02-17T13:02:49.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa03860c740 1 Processor -- start 2024-02-17T13:02:49.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa03860c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 0x56047e2f6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa03860c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa03860c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e352d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa03860c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047e353260 con 0x56047e2e92b0 2024-02-17T13:02:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa03860c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047e3533d0 con 0x56047e2f27b0 2024-02-17T13:02:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa03860c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047e353590 con 0x56047e2e9bd0 2024-02-17T13:02:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa02effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 0x56047e2f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa03860c740 10 client.0 _ll_register_callbacks cb 0x7fa028013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:02:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa027fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa02f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e352d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa02effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 0x56047e2f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54102/0 (socket says 172.21.15.18:54102) 2024-02-17T13:02:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa02effd640 1 -- 172.21.15.18:0/3274443026 learned_addr learned my addr 172.21.15.18:0/3274443026 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:02:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa02effd640 1 -- 172.21.15.18:0/3274443026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 msgr2=0x56047e2f6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa02effd640 1 --2- 172.21.15.18:0/3274443026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2f6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa02effd640 1 -- 172.21.15.18:0/3274443026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 msgr2=0x56047e352d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa02effd640 1 --2- 172.21.15.18:0/3274443026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e352d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.117+0000 7fa02effd640 1 -- 172.21.15.18:0/3274443026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56047e24f610 con 0x56047e2e92b0 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.118+0000 7fa02effd640 1 --2- 172.21.15.18:0/3274443026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 0x56047e2f6920 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fa020000c00 tx=0x7fa020012da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1913c3f0dcfb520a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.118+0000 7fa00effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.118+0000 7fa00effd640 1 -- 172.21.15.18:0/3274443026 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020016040 con 0x56047e2e92b0 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.118+0000 7fa00effd640 1 -- 172.21.15.18:0/3274443026 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa020025070 con 0x56047e2e92b0 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.118+0000 7fa03860c740 1 -- 172.21.15.18:0/3274443026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa02800ee10 con 0x56047e2e92b0 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.118+0000 7fa00effd640 1 -- 172.21.15.18:0/3274443026 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa020010e70 con 0x56047e2e92b0 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.118+0000 7fa03860c740 10 client.20253 fetch_fsmap learned FSMap version 51 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.118+0000 7fa03860c740 1 -- 172.21.15.18:0/3274443026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fa0301a6980 con 0x56047e2e92b0 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.118+0000 7fa00effd640 1 -- 172.21.15.18:0/3274443026 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa02800ee10 con 0x56047e2e92b0 2024-02-17T13:02:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.118+0000 7fa00effd640 1 -- 172.21.15.18:0/3274443026 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 51) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa0200297b0 con 0x56047e2e92b0 2024-02-17T13:02:49.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.118+0000 7fa03860c740 10 client.20253 fetch_fsmap finished waiting for FSMap version 51 2024-02-17T13:02:49.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.119+0000 7fa03860c740 10 client.20253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:02:49.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.119+0000 7fa03860c740 20 client.20253 populate_metadata read hostname 'smithi018' 2024-02-17T13:02:49.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.119+0000 7fa03860c740 1 --2- 172.21.15.18:0/3274443026 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56047e35ae30 0x56047e35d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:02:49.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.119+0000 7fa03860c740 4 client.20253 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:02:49.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.119+0000 7fa03860c740 1 -- 172.21.15.18:0/3274443026 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56047dfdc890 con 0x56047e35ae30 2024-02-17T13:02:49.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.119+0000 7fa027fff640 1 --2- 172.21.15.18:0/3274443026 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56047e35ae30 0x56047e35d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:02:49.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.119+0000 7fa00effd640 10 client.20253 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:49.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.119+0000 7fa027fff640 1 --2- 172.21.15.18:0/3274443026 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56047e35ae30 0x56047e35d250 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01475fabe4d01e5 server_cookie=dcbc4b74a90e21bb in_seq=0 out_seq=0 2024-02-17T13:02:49.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa00effd640 1 -- 172.21.15.18:0/3274443026 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56047dfdc890 con 0x56047e35ae30 2024-02-17T13:02:49.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa00effd640 10 client.20253 handle_command_reply: tid=0 2024-02-17T13:02:49.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa03860c740 1 client.20253 shutdown 2024-02-17T13:02:49.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa03860c740 1 -- 172.21.15.18:0/3274443026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 msgr2=0x56047e2f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:49.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa03860c740 1 --2- 172.21.15.18:0/3274443026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 0x56047e2f6920 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fa020000c00 tx=0x7fa020012da0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa03860c740 1 -- 172.21.15.18:0/3274443026 shutdown_connections 2024-02-17T13:02:49.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa03860c740 1 --2- 172.21.15.18:0/3274443026 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56047e35ae30 0x56047e35d250 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa03860c740 1 --2- 172.21.15.18:0/3274443026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56047e2f27b0 0x56047e352d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa03860c740 1 --2- 172.21.15.18:0/3274443026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56047e2e9bd0 0x56047e2f6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa00effd640 0 client.20253 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:02:49.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa03860c740 1 --2- 172.21.15.18:0/3274443026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56047e2e92b0 0x56047e2f6920 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa03860c740 1 -- 172.21.15.18:0/3274443026 >> 172.21.15.18:0/3274443026 conn(0x56047e2e3990 msgr2=0x56047e2ede80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:49.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa03860c740 1 -- 172.21.15.18:0/3274443026 shutdown_connections 2024-02-17T13:02:49.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa03860c740 1 -- 172.21.15.18:0/3274443026 wait complete. 2024-02-17T13:02:49.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.122+0000 7fa03860c740 20 client.20253 trim_cache size 0 max 16384 2024-02-17T13:02:49.119 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:02:49.119 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T13:02:49.119 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T13:02:49.119 INFO:teuthology.orchestra.run.smithi018.stdout: "72ca207e-7924-4fa4-9970-f36c1805dcd8": { 2024-02-17T13:02:49.119 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T13:02:49.119 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "72ca207e-7924-4fa4-9970-f36c1805dcd8", 2024-02-17T13:02:49.119 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T13:02:49.119 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:02:49.119 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:02:49.119 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:02:49.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.123+0000 7fa035963640 1 -- 172.21.15.18:0/2192989112 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa01c0a2260 msgr2=0x7fa01c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.123+0000 7fa035963640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa01c0a2260 0x7fa01c0a4720 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa028022000 tx=0x7fa02801e000 comp rx=0 tx=0).stop 2024-02-17T13:02:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.123+0000 7fa035963640 1 -- 172.21.15.18:0/2192989112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030105340 msgr2=0x7fa030102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:02:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.123+0000 7fa035963640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030105340 0x7fa030102240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa01802c170 tx=0x7fa018005e70 comp rx=0 tx=0).stop 2024-02-17T13:02:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.123+0000 7fa035963640 1 -- 172.21.15.18:0/2192989112 shutdown_connections 2024-02-17T13:02:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.123+0000 7fa035963640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa01c0a2260 0x7fa01c0a4720 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.123+0000 7fa035963640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa030102780 0x7fa0301a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.123+0000 7fa035963640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa030105340 0x7fa030102240 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.123+0000 7fa035963640 1 --2- 172.21.15.18:0/2192989112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa030100270 0x7fa030101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:02:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.123+0000 7fa035963640 1 -- 172.21.15.18:0/2192989112 >> 172.21.15.18:0/2192989112 conn(0x7fa030077230 msgr2=0x7fa0300686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:02:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.123+0000 7fa035963640 1 -- 172.21.15.18:0/2192989112 shutdown_connections 2024-02-17T13:02:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:02:49.123+0000 7fa035963640 1 -- 172.21.15.18:0/2192989112 wait complete. 2024-02-17T13:02:49.125 INFO:tasks.cephfs.filesystem:scrub status for tag:72ca207e-7924-4fa4-9970-f36c1805dcd8 - {'path': '/', 'tag': '72ca207e-7924-4fa4-9970-f36c1805dcd8', 'options': 'recursive,force'} 2024-02-17T13:02:49.785 INFO:tasks.workunit.client.0.smithi018.stdout: 10 373 133.31 MB/sec warmup 1 sec latency 48.880 ms 2024-02-17T13:02:50.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:49 smithi146 ceph-mon[36454]: pgmap v406: 97 pgs: 97 active+clean; 257 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-02-17T13:02:50.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:49 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T13:02:50.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:49 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:50.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:49 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:50.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:49 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:50.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:49 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:50.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:49 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:50.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:49 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:50.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:49 smithi018 ceph-mon[32625]: pgmap v406: 97 pgs: 97 active+clean; 257 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-02-17T13:02:50.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:49 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T13:02:50.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:49 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:50.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:49 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:50.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:49 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:50.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:49 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:50.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:49 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:50.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:49 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:50.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:49 smithi119 ceph-mon[36188]: pgmap v406: 97 pgs: 97 active+clean; 257 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-02-17T13:02:50.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:49 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T13:02:50.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:49 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:50.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:49 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:50.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:49 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:50.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:49 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:50.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:49 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:50.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:49 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:50.785 INFO:tasks.workunit.client.0.smithi018.stdout: 10 802 105.01 MB/sec warmup 2 sec latency 88.553 ms 2024-02-17T13:02:51.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:51.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.199 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:50 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:51.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:51.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:51.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:51.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:02:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:50 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:51.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:50 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:02:51.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 1388 85.33 MB/sec warmup 3 sec latency 81.314 ms 2024-02-17T13:02:52.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:51 smithi146 ceph-mon[36454]: pgmap v407: 97 pgs: 97 active+clean; 257 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-02-17T13:02:52.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:51 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:02:52.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:51 smithi018 ceph-mon[32625]: pgmap v407: 97 pgs: 97 active+clean; 257 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-02-17T13:02:52.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:51 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:02:52.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:51 smithi119 ceph-mon[36188]: pgmap v407: 97 pgs: 97 active+clean; 257 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-02-17T13:02:52.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:51 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:02:52.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 2236 68.71 MB/sec warmup 4 sec latency 86.569 ms 2024-02-17T13:02:53.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 2764 64.03 MB/sec warmup 5 sec latency 295.571 ms 2024-02-17T13:02:54.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:53 smithi146 ceph-mon[36454]: pgmap v408: 97 pgs: 97 active+clean; 257 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-02-17T13:02:54.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:53 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:02:54.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:53 smithi018 ceph-mon[32625]: pgmap v408: 97 pgs: 97 active+clean; 257 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-02-17T13:02:54.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:53 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:02:54.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:53 smithi119 ceph-mon[36188]: pgmap v408: 97 pgs: 97 active+clean; 257 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-02-17T13:02:54.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:53 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:02:54.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 3407 61.77 MB/sec warmup 6 sec latency 409.283 ms 2024-02-17T13:02:55.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:54 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:02:55.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:54 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:02:55.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:54 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:02:55.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 3698 55.27 MB/sec warmup 7 sec latency 73.338 ms 2024-02-17T13:02:56.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:55 smithi146 ceph-mon[36454]: pgmap v409: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2024-02-17T13:02:56.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:55 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:02:56.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:55 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:02:56.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:55 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:02:56.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:55 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:02:56.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:55 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:02:56.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:55 smithi018 ceph-mon[32625]: pgmap v409: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2024-02-17T13:02:56.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:55 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:02:56.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:55 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:02:56.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:55 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:02:56.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:55 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:02:56.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:55 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:02:56.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:55 smithi119 ceph-mon[36188]: pgmap v409: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2024-02-17T13:02:56.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:55 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:02:56.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:55 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:02:56.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:55 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:02:56.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:55 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:02:56.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:55 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:02:56.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 3876 49.26 MB/sec warmup 8 sec latency 84.626 ms 2024-02-17T13:02:57.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4104 44.37 MB/sec warmup 9 sec latency 88.044 ms 2024-02-17T13:02:58.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:57 smithi146 ceph-mon[36454]: pgmap v410: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.4 MiB/s wr, 209 op/s 2024-02-17T13:02:58.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:57 smithi018 ceph-mon[32625]: pgmap v410: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.4 MiB/s wr, 209 op/s 2024-02-17T13:02:58.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:57 smithi119 ceph-mon[36188]: pgmap v410: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.4 MiB/s wr, 209 op/s 2024-02-17T13:02:58.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4396 40.61 MB/sec warmup 10 sec latency 147.821 ms 2024-02-17T13:02:59.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:58 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:02:59.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:58 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:02:59.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:58 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:02:59.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4907 40.55 MB/sec warmup 11 sec latency 59.211 ms 2024-02-17T13:03:00.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:59 smithi018 ceph-mon[32625]: pgmap v411: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 329 op/s 2024-02-17T13:03:00.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:02:59 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:00.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:59 smithi119 ceph-mon[36188]: pgmap v411: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 329 op/s 2024-02-17T13:03:00.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:02:59 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:00.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:59 smithi146 ceph-mon[36454]: pgmap v411: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 329 op/s 2024-02-17T13:03:00.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:02:59 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:00.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 5718 38.57 MB/sec warmup 12 sec latency 56.623 ms 2024-02-17T13:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.434+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4193963903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e10100360 msgr2=0x7f8e101007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.434+0000 7f8e16f3a640 1 --2- 172.21.15.18:0/4193963903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e10100360 0x7f8e101007c0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009930 tx=0x7f8dfc02bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.434+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4193963903 shutdown_connections 2024-02-17T13:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.434+0000 7f8e16f3a640 1 --2- 172.21.15.18:0/4193963903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e10100d00 0x7f8e1010cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.434+0000 7f8e16f3a640 1 --2- 172.21.15.18:0/4193963903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e10100360 0x7f8e101007c0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.434+0000 7f8e16f3a640 1 --2- 172.21.15.18:0/4193963903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e101050a0 0x7f8e10105480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.434+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4193963903 >> 172.21.15.18:0/4193963903 conn(0x7f8e10077060 msgr2=0x7f8e10077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.434+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4193963903 shutdown_connections 2024-02-17T13:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.434+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4193963903 wait complete. 2024-02-17T13:03:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.435+0000 7f8e16f3a640 1 Processor -- start 2024-02-17T13:03:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.435+0000 7f8e16f3a640 1 -- start start 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.435+0000 7f8e16f3a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e10100360 0x7f8e10106f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.435+0000 7f8e16f3a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e10100d00 0x7f8e10107480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.435+0000 7f8e16f3a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e101050a0 0x7f8e101079c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.435+0000 7f8e16f3a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e101a5ad0 con 0x7f8e10100d00 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.435+0000 7f8e16f3a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e101a5c40 con 0x7f8e10100360 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.435+0000 7f8e16f3a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e101a5e00 con 0x7f8e101050a0 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e14cd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e10100360 0x7f8e10106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e0ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e10100d00 0x7f8e10107480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e154d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e101050a0 0x7f8e101079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e14cd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e10100360 0x7f8e10106f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:57680/0 (socket says 172.21.15.18:57680) 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e14cd6640 1 -- 172.21.15.18:0/4008437679 learned_addr learned my addr 172.21.15.18:0/4008437679 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e10100d00 0x7f8e10107480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40790/0 (socket says 172.21.15.18:40790) 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e0ffff640 1 -- 172.21.15.18:0/4008437679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e101050a0 msgr2=0x7f8e101079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e101050a0 0x7f8e101079c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e0ffff640 1 -- 172.21.15.18:0/4008437679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e10100360 msgr2=0x7f8e10106f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e10100360 0x7f8e10106f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e0ffff640 1 -- 172.21.15.18:0/4008437679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc009560 con 0x7f8e10100d00 2024-02-17T13:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e154d7640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e101050a0 0x7f8e101079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e14cd6640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e10100360 0x7f8e10106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e10100d00 0x7f8e10107480 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009230 tx=0x7f8dfc004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e0dffb640 1 -- 172.21.15.18:0/4008437679 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc004250 con 0x7f8e10100d00 2024-02-17T13:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e0dffb640 1 -- 172.21.15.18:0/4008437679 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8dfc0043b0 con 0x7f8e10100d00 2024-02-17T13:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4008437679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e101a6080 con 0x7f8e10100d00 2024-02-17T13:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.436+0000 7f8e0dffb640 1 -- 172.21.15.18:0/4008437679 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc003b90 con 0x7f8e10100d00 2024-02-17T13:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.437+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4008437679 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e101a6560 con 0x7f8e10100d00 2024-02-17T13:03:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.439+0000 7f8e0dffb640 1 -- 172.21.15.18:0/4008437679 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8dfc003cf0 con 0x7f8e10100d00 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e19be5740 1 Processor -- start 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e0dffb640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8e000a2300 0x7f8e000a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e19be5740 1 -- start start 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e154d7640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8e000a2300 0x7f8e000a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e0dffb640 1 -- 172.21.15.18:0/4008437679 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8dfc0eff00 con 0x7f8e10100d00 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e19be5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 0x55bab1f22820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e19be5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e19be5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 0x55bab1f2e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e19be5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab1e919f0 con 0x55bab1f23ad0 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e19be5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab1e8ef20 con 0x55bab1f2c2e0 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e19be5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab1e8a700 con 0x55bab1f22d60 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e14cd6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 0x55bab1f22820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e0ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e14cd6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 0x55bab1f22820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40798/0 (socket says 172.21.15.18:40798) 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e14cd6640 1 -- 172.21.15.18:0/2452676386 learned_addr learned my addr 172.21.15.18:0/2452676386 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e154d7640 1 --2- 172.21.15.18:0/2452676386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 0x55bab1f2e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e0ffff640 1 -- 172.21.15.18:0/2452676386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 msgr2=0x55bab1f2e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/2452676386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 0x55bab1f2e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e0ffff640 1 -- 172.21.15.18:0/2452676386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 msgr2=0x55bab1f22820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/2452676386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 0x55bab1f22820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.440+0000 7f8e0ffff640 1 -- 172.21.15.18:0/2452676386 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bab1e207b0 con 0x55bab1f22d60 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.441+0000 7f8e154d7640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8e000a2300 0x7f8e000a47c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8e08005fd0 tx=0x7f8e08011d20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.441+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/2452676386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f231e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0ed780 tx=0x7f8dfc02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=436468a8dc95c5ee server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.441+0000 7f8dfaffd640 1 -- 172.21.15.18:0/2452676386 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0c0b70 con 0x55bab1f22d60 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.441+0000 7f8dfaffd640 1 -- 172.21.15.18:0/2452676386 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8dfc0c51c0 con 0x55bab1f22d60 2024-02-17T13:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.441+0000 7f8dfaffd640 1 -- 172.21.15.18:0/2452676386 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0c4d30 con 0x55bab1f22d60 2024-02-17T13:03:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 -- 172.21.15.18:0/2452676386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 msgr2=0x55bab1f231e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2452676386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f231e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0ed780 tx=0x7f8dfc02bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 -- 172.21.15.18:0/2452676386 shutdown_connections 2024-02-17T13:03:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2452676386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 0x55bab1f2e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2452676386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f231e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2452676386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 0x55bab1f22820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 -- 172.21.15.18:0/2452676386 >> 172.21.15.18:0/2452676386 conn(0x55bab1eea550 msgr2=0x55bab1f1da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 -- 172.21.15.18:0/2452676386 shutdown_connections 2024-02-17T13:03:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 -- 172.21.15.18:0/2452676386 wait complete. 2024-02-17T13:03:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 -- start start 2024-02-17T13:03:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 Processor -- start 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f29330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 0x55bab1f29870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 0x55bab1f29db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab1e2fbf0 con 0x55bab1f23ad0 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab1b5e250 con 0x55bab1f2c2e0 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.442+0000 7f8e19be5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab1e10200 con 0x55bab1f22d60 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e19be5740 10 client.0 _ll_register_callbacks cb 0x55bab1e7ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e0ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 0x55bab1f29870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e0ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 0x55bab1f29870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40800/0 (socket says 172.21.15.18:40800) 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e14cd6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f29330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e0ffff640 1 -- 172.21.15.18:0/2947354827 learned_addr learned my addr 172.21.15.18:0/2947354827 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e154d7640 1 --2- 172.21.15.18:0/2947354827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 0x55bab1f29db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e0ffff640 1 -- 172.21.15.18:0/2947354827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 msgr2=0x55bab1f29330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/2947354827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f29330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e0ffff640 1 -- 172.21.15.18:0/2947354827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 msgr2=0x55bab1f29db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/2947354827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 0x55bab1f29db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e0ffff640 1 -- 172.21.15.18:0/2947354827 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e101a6560 con 0x55bab1f23ad0 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/2947354827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 0x55bab1f29870 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0bc9b0 tx=0x7f8dfc004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9264de57f1d062 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8deffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8deffff640 1 -- 172.21.15.18:0/2947354827 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc005640 con 0x55bab1f23ad0 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8deffff640 1 -- 172.21.15.18:0/2947354827 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8dfc0dc040 con 0x55bab1f23ad0 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e19be5740 1 -- 172.21.15.18:0/2947354827 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bab1b8a600 con 0x55bab1f23ad0 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8deffff640 1 -- 172.21.15.18:0/2947354827 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dfc0c0b70 con 0x55bab1f23ad0 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e19be5740 10 client.20265 fetch_fsmap learned FSMap version 56 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.443+0000 7f8e19be5740 1 -- 172.21.15.18:0/2947354827 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55bab1e88d80 con 0x55bab1f23ad0 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.444+0000 7f8deffff640 1 -- 172.21.15.18:0/2947354827 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bab1b8a600 con 0x55bab1f23ad0 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.444+0000 7f8deffff640 1 -- 172.21.15.18:0/2947354827 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8dfc03c070 con 0x55bab1f23ad0 2024-02-17T13:03:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.444+0000 7f8e19be5740 10 client.20265 fetch_fsmap finished waiting for FSMap version 56 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.444+0000 7f8e19be5740 10 client.20265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.444+0000 7f8e19be5740 20 client.20265 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.444+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2947354827 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bab1f3bf20 0x55bab1f3e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.444+0000 7f8e19be5740 4 client.20265 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.444+0000 7f8e19be5740 1 -- 172.21.15.18:0/2947354827 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bab1bdb1b0 con 0x55bab1f3bf20 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.444+0000 7f8e14cd6640 1 --2- 172.21.15.18:0/2947354827 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bab1f3bf20 0x55bab1f3e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.444+0000 7f8deffff640 10 client.20265 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.444+0000 7f8e14cd6640 1 --2- 172.21.15.18:0/2947354827 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bab1f3bf20 0x55bab1f3e310 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0fb8dff626493ae server_cookie=3951ea4c7c939db5 in_seq=0 out_seq=0 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.446+0000 7f8deffff640 1 -- 172.21.15.18:0/2947354827 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bab1bdb1b0 con 0x55bab1f3bf20 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.446+0000 7f8deffff640 10 client.20265 handle_command_reply: tid=0 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.446+0000 7f8e19be5740 1 client.20265 shutdown 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.446+0000 7f8e19be5740 1 -- 172.21.15.18:0/2947354827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 msgr2=0x55bab1f29870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.447+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2947354827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 0x55bab1f29870 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0bc9b0 tx=0x7f8dfc004520 comp rx=0 tx=0).stop 2024-02-17T13:03:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.447+0000 7f8e19be5740 1 -- 172.21.15.18:0/2947354827 shutdown_connections 2024-02-17T13:03:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.447+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2947354827 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bab1f3bf20 0x55bab1f3e310 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.447+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2947354827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 0x55bab1f29db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.447+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2947354827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f23ad0 0x55bab1f29870 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.447+0000 7f8deffff640 0 client.20265 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.447+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2947354827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f29330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.447+0000 7f8e19be5740 1 -- 172.21.15.18:0/2947354827 >> 172.21.15.18:0/2947354827 conn(0x55bab1eea550 msgr2=0x55bab1f1f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.447+0000 7f8e19be5740 1 -- 172.21.15.18:0/2947354827 shutdown_connections 2024-02-17T13:03:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.447+0000 7f8e19be5740 1 -- 172.21.15.18:0/2947354827 wait complete. 2024-02-17T13:03:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.447+0000 7f8e19be5740 20 client.20265 trim_cache size 0 max 16384 2024-02-17T13:03:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e19be5740 1 Processor -- start 2024-02-17T13:03:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e19be5740 1 -- start start 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e19be5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f298c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e19be5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f23ad0 0x55bab1f29e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e19be5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f2c2e0 0x55bab1f265f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e19be5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab1f3e870 con 0x55bab1f2c2e0 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e19be5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab1f22030 con 0x55bab1f23ad0 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e19be5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab1f2eb30 con 0x55bab1f22d60 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e154d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f2c2e0 0x55bab1f265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e154d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f2c2e0 0x55bab1f265f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40814/0 (socket says 172.21.15.18:40814) 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e154d7640 1 -- 172.21.15.18:0/2134109833 learned_addr learned my addr 172.21.15.18:0/2134109833 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e14cd6640 1 --2- 172.21.15.18:0/2134109833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.462+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/2134109833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f23ad0 0x55bab1f29e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.463+0000 7f8e154d7640 1 -- 172.21.15.18:0/2134109833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 msgr2=0x55bab1f298c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.463+0000 7f8e154d7640 1 --2- 172.21.15.18:0/2134109833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f298c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.463+0000 7f8e154d7640 1 -- 172.21.15.18:0/2134109833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f23ad0 msgr2=0x55bab1f29e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.463+0000 7f8e154d7640 1 --2- 172.21.15.18:0/2134109833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f23ad0 0x55bab1f29e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.463+0000 7f8e154d7640 1 -- 172.21.15.18:0/2134109833 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bab1e257b0 con 0x55bab1f2c2e0 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.463+0000 7f8e154d7640 1 --2- 172.21.15.18:0/2134109833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f2c2e0 0x55bab1f265f0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800fe40 tx=0x7f8e0801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4b1728f8901532 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.463+0000 7f8dfa7fc640 1 -- 172.21.15.18:0/2134109833 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08004530 con 0x55bab1f2c2e0 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.463+0000 7f8dfa7fc640 1 -- 172.21.15.18:0/2134109833 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e08004690 con 0x55bab1f2c2e0 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.463+0000 7f8dfa7fc640 1 -- 172.21.15.18:0/2134109833 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08020660 con 0x55bab1f2c2e0 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.464+0000 7f8e19be5740 1 -- 172.21.15.18:0/2134109833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f2c2e0 msgr2=0x55bab1f265f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.464+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2134109833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f2c2e0 0x55bab1f265f0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800fe40 tx=0x7f8e0801c000 comp rx=0 tx=0).stop 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.464+0000 7f8e19be5740 1 -- 172.21.15.18:0/2134109833 shutdown_connections 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.464+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2134109833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f2c2e0 0x55bab1f265f0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.464+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2134109833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f23ad0 0x55bab1f29e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.464+0000 7f8e19be5740 1 --2- 172.21.15.18:0/2134109833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f22d60 0x55bab1f298c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.464+0000 7f8e19be5740 1 -- 172.21.15.18:0/2134109833 >> 172.21.15.18:0/2134109833 conn(0x55bab1eea550 msgr2=0x55bab1f27c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.464+0000 7f8e19be5740 1 -- 172.21.15.18:0/2134109833 shutdown_connections 2024-02-17T13:03:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.464+0000 7f8e19be5740 1 -- 172.21.15.18:0/2134109833 wait complete. 2024-02-17T13:03:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.464+0000 7f8e19be5740 1 -- start start 2024-02-17T13:03:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.464+0000 7f8e19be5740 1 Processor -- start 2024-02-17T13:03:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e19be5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f22d60 0x55bab1f319a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e19be5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f23ad0 0x55bab1f31ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e19be5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 0x55bab1f32420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e19be5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab1f92270 con 0x55bab1f22d60 2024-02-17T13:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e19be5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab1f923b0 con 0x55bab1f2c2e0 2024-02-17T13:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e19be5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab1f924f0 con 0x55bab1f23ad0 2024-02-17T13:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e19be5740 10 client.0 _ll_register_callbacks cb 0x7f8dfc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e14cd6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f22d60 0x55bab1f319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e14cd6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f22d60 0x55bab1f319a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40818/0 (socket says 172.21.15.18:40818) 2024-02-17T13:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e14cd6640 1 -- 172.21.15.18:0/713362178 learned_addr learned my addr 172.21.15.18:0/713362178 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e154d7640 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 0x55bab1f32420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f23ad0 0x55bab1f31ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e14cd6640 1 -- 172.21.15.18:0/713362178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f23ad0 msgr2=0x55bab1f31ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e14cd6640 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f23ad0 0x55bab1f31ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e14cd6640 1 -- 172.21.15.18:0/713362178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 msgr2=0x55bab1f32420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e14cd6640 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 0x55bab1f32420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e14cd6640 1 -- 172.21.15.18:0/713362178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0800fbd0 con 0x55bab1f22d60 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e14cd6640 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f22d60 0x55bab1f319a0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f8e04012290 tx=0x7f8e04013c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778821d836ad54f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8deffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8deffff640 1 -- 172.21.15.18:0/713362178 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04017070 con 0x55bab1f22d60 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8deffff640 1 -- 172.21.15.18:0/713362178 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8e04020740 con 0x55bab1f22d60 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.465+0000 7f8e19be5740 1 -- 172.21.15.18:0/713362178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8dfc0c51c0 con 0x55bab1f22d60 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.466+0000 7f8deffff640 1 -- 172.21.15.18:0/713362178 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e04020900 con 0x55bab1f22d60 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.466+0000 7f8e19be5740 10 client.20271 fetch_fsmap learned FSMap version 56 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.466+0000 7f8e19be5740 1 -- 172.21.15.18:0/713362178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55bab1e4b540 con 0x55bab1f22d60 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.466+0000 7f8deffff640 1 -- 172.21.15.18:0/713362178 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8dfc0c51c0 con 0x55bab1f22d60 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.466+0000 7f8deffff640 1 -- 172.21.15.18:0/713362178 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8e04010550 con 0x55bab1f22d60 2024-02-17T13:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.466+0000 7f8e19be5740 10 client.20271 fetch_fsmap finished waiting for FSMap version 56 2024-02-17T13:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.466+0000 7f8e19be5740 10 client.20271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.466+0000 7f8e19be5740 20 client.20271 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.466+0000 7f8e19be5740 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bab1f3bf20 0x55bab1f95e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.466+0000 7f8e19be5740 4 client.20271 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.466+0000 7f8e19be5740 1 -- 172.21.15.18:0/713362178 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bab1bc7550 con 0x55bab1f3bf20 2024-02-17T13:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.466+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bab1f3bf20 0x55bab1f95e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.467+0000 7f8e0ffff640 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bab1f3bf20 0x55bab1f95e20 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d924f5ccae8de0d server_cookie=1891ddef3ebc1096 in_seq=0 out_seq=0 2024-02-17T13:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.467+0000 7f8deffff640 10 client.20271 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:01.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 5935 36.36 MB/sec warmup 13 sec latency 235.766 ms 2024-02-17T13:03:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.932+0000 7f8deffff640 1 -- 172.21.15.18:0/713362178 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bab1bc7550 con 0x55bab1f3bf20 2024-02-17T13:03:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.932+0000 7f8deffff640 10 client.20271 handle_command_reply: tid=0 2024-02-17T13:03:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.932+0000 7f8e19be5740 1 client.20271 shutdown 2024-02-17T13:03:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.933+0000 7f8e19be5740 1 -- 172.21.15.18:0/713362178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f22d60 msgr2=0x55bab1f319a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.933+0000 7f8e19be5740 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f22d60 0x55bab1f319a0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f8e04012290 tx=0x7f8e04013c40 comp rx=0 tx=0).stop 2024-02-17T13:03:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.933+0000 7f8e19be5740 1 -- 172.21.15.18:0/713362178 shutdown_connections 2024-02-17T13:03:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.933+0000 7f8e19be5740 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bab1f3bf20 0x55bab1f95e20 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.933+0000 7f8deffff640 0 client.20271 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.933+0000 7f8e19be5740 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bab1f2c2e0 0x55bab1f32420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.933+0000 7f8e19be5740 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bab1f23ad0 0x55bab1f31ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.933+0000 7f8e19be5740 1 --2- 172.21.15.18:0/713362178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bab1f22d60 0x55bab1f319a0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.933+0000 7f8e19be5740 1 -- 172.21.15.18:0/713362178 >> 172.21.15.18:0/713362178 conn(0x55bab1eea550 msgr2=0x55bab1f1c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.933+0000 7f8e19be5740 1 -- 172.21.15.18:0/713362178 shutdown_connections 2024-02-17T13:03:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.933+0000 7f8e19be5740 1 -- 172.21.15.18:0/713362178 wait complete. 2024-02-17T13:03:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.933+0000 7f8e19be5740 20 client.20271 trim_cache size 0 max 16384 2024-02-17T13:03:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:03:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:03:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:03:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:03:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.934+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4008437679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8e000a2300 msgr2=0x7f8e000a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.934+0000 7f8e16f3a640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8e000a2300 0x7f8e000a47c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8e08005fd0 tx=0x7f8e08011d20 comp rx=0 tx=0).stop 2024-02-17T13:03:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.934+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4008437679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e10100d00 msgr2=0x7f8e10107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.934+0000 7f8e16f3a640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e10100d00 0x7f8e10107480 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009230 tx=0x7f8dfc004a50 comp rx=0 tx=0).stop 2024-02-17T13:03:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.934+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4008437679 shutdown_connections 2024-02-17T13:03:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.934+0000 7f8e16f3a640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8e000a2300 0x7f8e000a47c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.934+0000 7f8e16f3a640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e101050a0 0x7f8e101079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.934+0000 7f8e16f3a640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e10100d00 0x7f8e10107480 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.934+0000 7f8e16f3a640 1 --2- 172.21.15.18:0/4008437679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e10100360 0x7f8e10106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.934+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4008437679 >> 172.21.15.18:0/4008437679 conn(0x7f8e10077060 msgr2=0x7f8e100fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.934+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4008437679 shutdown_connections 2024-02-17T13:03:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:01.934+0000 7f8e16f3a640 1 -- 172.21.15.18:0/4008437679 wait complete. 2024-02-17T13:03:02.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:01 smithi018 ceph-mon[32625]: pgmap v412: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2024-02-17T13:03:02.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:01 smithi119 ceph-mon[36188]: pgmap v412: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2024-02-17T13:03:02.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:01 smithi146 ceph-mon[36454]: pgmap v412: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2024-02-17T13:03:02.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 6528 37.27 MB/sec warmup 14 sec latency 271.665 ms 2024-02-17T13:03:03.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 6778 35.97 MB/sec warmup 15 sec latency 100.035 ms 2024-02-17T13:03:04.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:03 smithi018 ceph-mon[32625]: pgmap v413: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2024-02-17T13:03:04.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:03 smithi119 ceph-mon[36188]: pgmap v413: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2024-02-17T13:03:04.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:03 smithi146 ceph-mon[36454]: pgmap v413: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2024-02-17T13:03:04.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 7209 36.06 MB/sec warmup 16 sec latency 111.842 ms 2024-02-17T13:03:05.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 7387 34.16 MB/sec warmup 17 sec latency 49.281 ms 2024-02-17T13:03:06.250 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:05 smithi146 ceph-mon[36454]: pgmap v414: 97 pgs: 97 active+clean; 280 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 896 op/s 2024-02-17T13:03:06.250 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:05 smithi146 ceph-mon[36454]: osdmap e105: 12 total, 12 up, 12 in 2024-02-17T13:03:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:05 smithi018 ceph-mon[32625]: pgmap v414: 97 pgs: 97 active+clean; 280 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 896 op/s 2024-02-17T13:03:06.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:05 smithi018 ceph-mon[32625]: osdmap e105: 12 total, 12 up, 12 in 2024-02-17T13:03:06.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:05 smithi119 ceph-mon[36188]: pgmap v414: 97 pgs: 97 active+clean; 280 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 896 op/s 2024-02-17T13:03:06.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:05 smithi119 ceph-mon[36188]: osdmap e105: 12 total, 12 up, 12 in 2024-02-17T13:03:06.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 7621 32.58 MB/sec warmup 18 sec latency 72.463 ms 2024-02-17T13:03:07.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 7995 31.54 MB/sec warmup 19 sec latency 122.978 ms 2024-02-17T13:03:08.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:07 smithi018 ceph-mon[32625]: pgmap v416: 97 pgs: 97 active+clean; 280 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 888 op/s 2024-02-17T13:03:08.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:07 smithi119 ceph-mon[36188]: pgmap v416: 97 pgs: 97 active+clean; 280 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 888 op/s 2024-02-17T13:03:08.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:07 smithi146 ceph-mon[36454]: pgmap v416: 97 pgs: 97 active+clean; 280 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 888 op/s 2024-02-17T13:03:08.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 8452 31.87 MB/sec warmup 20 sec latency 60.196 ms 2024-02-17T13:03:09.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 9221 31.12 MB/sec warmup 21 sec latency 76.619 ms 2024-02-17T13:03:10.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:09 smithi018 ceph-mon[32625]: pgmap v417: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 872 op/s 2024-02-17T13:03:10.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:09 smithi119 ceph-mon[36188]: pgmap v417: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 872 op/s 2024-02-17T13:03:10.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:09 smithi146 ceph-mon[36454]: pgmap v417: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 872 op/s 2024-02-17T13:03:10.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 9655 31.47 MB/sec warmup 22 sec latency 80.041 ms 2024-02-17T13:03:11.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 10264 31.63 MB/sec warmup 23 sec latency 45.674 ms 2024-02-17T13:03:12.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:11 smithi018 ceph-mon[32625]: pgmap v418: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 872 op/s 2024-02-17T13:03:12.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:11 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:03:12.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:11 smithi119 ceph-mon[36188]: pgmap v418: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 872 op/s 2024-02-17T13:03:12.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:11 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:03:12.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:11 smithi146 ceph-mon[36454]: pgmap v418: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 872 op/s 2024-02-17T13:03:12.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:11 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:03:12.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 10655 31.93 MB/sec warmup 24 sec latency 88.912 ms 2024-02-17T13:03:13.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:12 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:03:13.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:12 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:03:13.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:12 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:03:13.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:12 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:03:13.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:12 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:03:13.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:12 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:03:13.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:12 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:03:13.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:12 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:03:13.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:12 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:03:13.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 10823 30.77 MB/sec warmup 25 sec latency 464.741 ms 2024-02-17T13:03:14.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:14 smithi146 ceph-mon[36454]: pgmap v419: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 872 op/s 2024-02-17T13:03:14.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:14 smithi146 ceph-mon[36454]: osdmap e106: 12 total, 12 up, 12 in 2024-02-17T13:03:14.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:14 smithi018 ceph-mon[32625]: pgmap v419: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 872 op/s 2024-02-17T13:03:14.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:14 smithi018 ceph-mon[32625]: osdmap e106: 12 total, 12 up, 12 in 2024-02-17T13:03:14.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 11039 29.65 MB/sec warmup 26 sec latency 642.360 ms 2024-02-17T13:03:14.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:14 smithi119 ceph-mon[36188]: pgmap v419: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 872 op/s 2024-02-17T13:03:14.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:14 smithi119 ceph-mon[36188]: osdmap e106: 12 total, 12 up, 12 in 2024-02-17T13:03:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.051+0000 7fa19f356640 1 -- 172.21.15.18:0/4277439980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa198100c90 msgr2=0x7fa19810ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.051+0000 7fa19f356640 1 --2- 172.21.15.18:0/4277439980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa198100c90 0x7fa19810ceb0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fa190009900 tx=0x7fa19002bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.051+0000 7fa19f356640 1 -- 172.21.15.18:0/4277439980 shutdown_connections 2024-02-17T13:03:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.051+0000 7fa19f356640 1 --2- 172.21.15.18:0/4277439980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa198100c90 0x7fa19810ceb0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.051+0000 7fa19f356640 1 --2- 172.21.15.18:0/4277439980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1981002d0 0x7fa198100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.051+0000 7fa19f356640 1 --2- 172.21.15.18:0/4277439980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa198105020 0x7fa198105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.051+0000 7fa19f356640 1 -- 172.21.15.18:0/4277439980 >> 172.21.15.18:0/4277439980 conn(0x7fa198076ed0 msgr2=0x7fa1980772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.051+0000 7fa19f356640 1 -- 172.21.15.18:0/4277439980 shutdown_connections 2024-02-17T13:03:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.051+0000 7fa19f356640 1 -- 172.21.15.18:0/4277439980 wait complete. 2024-02-17T13:03:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.051+0000 7fa19f356640 1 Processor -- start 2024-02-17T13:03:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19f356640 1 -- start start 2024-02-17T13:03:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19f356640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1981002d0 0x7fa19819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19f356640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa198100c90 0x7fa19819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19f356640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa198105020 0x7fa1981975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19f356640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198197c20 con 0x7fa1981002d0 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19f356640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198197d90 con 0x7fa198105020 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19f356640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198197f50 con 0x7fa198100c90 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19d0f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1981002d0 0x7fa19819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19d0f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1981002d0 0x7fa19819d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48348/0 (socket says 172.21.15.18:48348) 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19d0f2640 1 -- 172.21.15.18:0/4131193008 learned_addr learned my addr 172.21.15.18:0/4131193008 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa198100c90 0x7fa19819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19d8f3640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa198105020 0x7fa1981975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19d0f2640 1 -- 172.21.15.18:0/4131193008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa198100c90 msgr2=0x7fa19819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19d0f2640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa198100c90 0x7fa19819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19d0f2640 1 -- 172.21.15.18:0/4131193008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa198105020 msgr2=0x7fa1981975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19d0f2640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa198105020 0x7fa1981975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19d0f2640 1 -- 172.21.15.18:0/4131193008 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa190009560 con 0x7fa1981002d0 2024-02-17T13:03:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19d8f3640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa198105020 0x7fa1981975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.052+0000 7fa19d0f2640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1981002d0 0x7fa19819d420 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa19400eb90 tx=0x7fa19400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.053+0000 7fa18e7fc640 1 -- 172.21.15.18:0/4131193008 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1940099b0 con 0x7fa1981002d0 2024-02-17T13:03:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.053+0000 7fa18e7fc640 1 -- 172.21.15.18:0/4131193008 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa19400ee90 con 0x7fa1981002d0 2024-02-17T13:03:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.053+0000 7fa19f356640 1 -- 172.21.15.18:0/4131193008 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa198198230 con 0x7fa1981002d0 2024-02-17T13:03:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.053+0000 7fa18e7fc640 1 -- 172.21.15.18:0/4131193008 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1940107a0 con 0x7fa1981002d0 2024-02-17T13:03:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.053+0000 7fa19f356640 1 -- 172.21.15.18:0/4131193008 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1981a44c0 con 0x7fa1981002d0 2024-02-17T13:03:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.055+0000 7fa18e7fc640 1 -- 172.21.15.18:0/4131193008 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa194005020 con 0x7fa1981002d0 2024-02-17T13:03:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa1a2009740 1 Processor -- start 2024-02-17T13:03:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa1a2009740 1 -- start start 2024-02-17T13:03:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa18e7fc640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa1840a22d0 0x7fa1840a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa18e7fc640 1 -- 172.21.15.18:0/4131193008 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa194014070 con 0x7fa1981002d0 2024-02-17T13:03:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19d8f3640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa1840a22d0 0x7fa1840a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa1a2009740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e06bad0 0x55b48e06a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa1a2009740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 0x55b48e06b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa1a2009740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e0742e0 0x55b48e0766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa1a2009740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48dfd99f0 con 0x55b48e06ad60 2024-02-17T13:03:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa1a2009740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48dfd6f20 con 0x55b48e06bad0 2024-02-17T13:03:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa1a2009740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48dfd2700 con 0x55b48e0742e0 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19c8f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 0x55b48e06b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19c8f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 0x55b48e06b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48360/0 (socket says 172.21.15.18:48360) 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19d0f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e06bad0 0x55b48e06a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19c8f1640 1 -- 172.21.15.18:0/4175439495 learned_addr learned my addr 172.21.15.18:0/4175439495 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19d8f3640 1 --2- 172.21.15.18:0/4175439495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e0742e0 0x55b48e0766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19c8f1640 1 -- 172.21.15.18:0/4175439495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e0742e0 msgr2=0x55b48e0766d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/4175439495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e0742e0 0x55b48e0766d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19c8f1640 1 -- 172.21.15.18:0/4175439495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e06bad0 msgr2=0x55b48e06a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/4175439495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e06bad0 0x55b48e06a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19c8f1640 1 -- 172.21.15.18:0/4175439495 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b48df687b0 con 0x55b48e06ad60 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19d0f2640 1 --2- 172.21.15.18:0/4175439495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e06bad0 0x55b48e06a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.056+0000 7fa19d8f3640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa1840a22d0 0x7fa1840a4790 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa190005ec0 tx=0x7fa19002c8d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.057+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/4175439495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 0x55b48e06b1e0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa188009c60 tx=0x7fa188011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad9e7bc24948a50 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.057+0000 7fa1837fe640 1 -- 172.21.15.18:0/4175439495 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188004230 con 0x55b48e06ad60 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.057+0000 7fa1837fe640 1 -- 172.21.15.18:0/4175439495 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa188009e10 con 0x55b48e06ad60 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.057+0000 7fa1837fe640 1 -- 172.21.15.18:0/4175439495 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188004c70 con 0x55b48e06ad60 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.057+0000 7fa1a2009740 1 -- 172.21.15.18:0/4175439495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 msgr2=0x55b48e06b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.057+0000 7fa1a2009740 1 --2- 172.21.15.18:0/4175439495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 0x55b48e06b1e0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa188009c60 tx=0x7fa188011810 comp rx=0 tx=0).stop 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.057+0000 7fa1a2009740 1 -- 172.21.15.18:0/4175439495 shutdown_connections 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.057+0000 7fa1a2009740 1 --2- 172.21.15.18:0/4175439495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e0742e0 0x55b48e0766d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.057+0000 7fa1a2009740 1 --2- 172.21.15.18:0/4175439495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 0x55b48e06b1e0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.057+0000 7fa1a2009740 1 --2- 172.21.15.18:0/4175439495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e06bad0 0x55b48e06a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.057+0000 7fa1a2009740 1 -- 172.21.15.18:0/4175439495 >> 172.21.15.18:0/4175439495 conn(0x55b48e032550 msgr2=0x55b48e065a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.058+0000 7fa1a2009740 1 -- 172.21.15.18:0/4175439495 shutdown_connections 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.058+0000 7fa1a2009740 1 -- 172.21.15.18:0/4175439495 wait complete. 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.058+0000 7fa1a2009740 1 -- start start 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.058+0000 7fa1a2009740 1 Processor -- start 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.058+0000 7fa1a2009740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06ad60 0x55b48e0712d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.058+0000 7fa1a2009740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 0x55b48e071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.058+0000 7fa1a2009740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.058+0000 7fa1a2009740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48df77bf0 con 0x55b48e06bad0 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.058+0000 7fa1a2009740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48dca6250 con 0x55b48e0742e0 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.058+0000 7fa1a2009740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48df58200 con 0x55b48e06ad60 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.058+0000 7fa1a2009740 10 client.0 _ll_register_callbacks cb 0x55b48dfc6d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.058+0000 7fa19c8f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 0x55b48e071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa19c8f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 0x55b48e071810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48370/0 (socket says 172.21.15.18:48370) 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa19c8f1640 1 -- 172.21.15.18:0/3942042694 learned_addr learned my addr 172.21.15.18:0/3942042694 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa19d8f3640 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa19d0f2640 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06ad60 0x55b48e0712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa19c8f1640 1 -- 172.21.15.18:0/3942042694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06ad60 msgr2=0x55b48e0712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06ad60 0x55b48e0712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa19c8f1640 1 -- 172.21.15.18:0/3942042694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 msgr2=0x55b48e071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa19c8f1640 1 -- 172.21.15.18:0/3942042694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa188009890 con 0x55b48e06bad0 2024-02-17T13:03:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa19d8f3640 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 0x55b48e071810 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fa188000c00 tx=0x7fa188019c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a3ad87f72166e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa180ff9640 1 -- 172.21.15.18:0/3942042694 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188023610 con 0x55b48e06bad0 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa180ff9640 1 -- 172.21.15.18:0/3942042694 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa188023770 con 0x55b48e06bad0 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa1a2009740 1 -- 172.21.15.18:0/3942042694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b48dcd2600 con 0x55b48e06bad0 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa180ff9640 1 -- 172.21.15.18:0/3942042694 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa188023930 con 0x55b48e06bad0 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa1a2009740 10 client.20283 fetch_fsmap learned FSMap version 56 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.059+0000 7fa1a2009740 1 -- 172.21.15.18:0/3942042694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55b48dfd0d80 con 0x55b48e06bad0 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.060+0000 7fa180ff9640 1 -- 172.21.15.18:0/3942042694 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b48dcd2600 con 0x55b48e06bad0 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.060+0000 7fa180ff9640 1 -- 172.21.15.18:0/3942042694 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa188018b80 con 0x55b48e06bad0 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.060+0000 7fa1a2009740 10 client.20283 fetch_fsmap finished waiting for FSMap version 56 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.060+0000 7fa1a2009740 10 client.20283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.060+0000 7fa1a2009740 20 client.20283 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.060+0000 7fa1a2009740 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b48e0845b0 0x55b48e0869d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.060+0000 7fa1a2009740 4 client.20283 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.060+0000 7fa1a2009740 1 -- 172.21.15.18:0/3942042694 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b48dd231b0 con 0x55b48e0845b0 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.060+0000 7fa19d0f2640 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b48e0845b0 0x55b48e0869d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.060+0000 7fa19d0f2640 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b48e0845b0 0x55b48e0869d0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67fc47c827d3977 server_cookie=9b064a0087b21034 in_seq=0 out_seq=0 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.060+0000 7fa180ff9640 10 client.20283 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa180ff9640 1 -- 172.21.15.18:0/3942042694 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b48dd231b0 con 0x55b48e0845b0 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa180ff9640 10 client.20283 handle_command_reply: tid=0 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa1a2009740 1 client.20283 shutdown 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa1a2009740 1 -- 172.21.15.18:0/3942042694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 msgr2=0x55b48e071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa1a2009740 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 0x55b48e071810 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fa188000c00 tx=0x7fa188019c30 comp rx=0 tx=0).stop 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa1a2009740 1 -- 172.21.15.18:0/3942042694 shutdown_connections 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa1a2009740 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b48e0845b0 0x55b48e0869d0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa1a2009740 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa1a2009740 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 0x55b48e071810 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa180ff9640 0 client.20283 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa1a2009740 1 --2- 172.21.15.18:0/3942042694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06ad60 0x55b48e0712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa1a2009740 1 -- 172.21.15.18:0/3942042694 >> 172.21.15.18:0/3942042694 conn(0x55b48e032550 msgr2=0x55b48e0677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa1a2009740 1 -- 172.21.15.18:0/3942042694 shutdown_connections 2024-02-17T13:03:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa1a2009740 1 -- 172.21.15.18:0/3942042694 wait complete. 2024-02-17T13:03:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.062+0000 7fa1a2009740 20 client.20283 trim_cache size 0 max 16384 2024-02-17T13:03:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa1a2009740 1 Processor -- start 2024-02-17T13:03:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa1a2009740 1 -- start start 2024-02-17T13:03:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa1a2009740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06ad60 0x55b48e07b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa1a2009740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 0x55b48e07b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa1a2009740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa1a2009740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48e0843b0 con 0x55b48e06bad0 2024-02-17T13:03:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa1a2009740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48e066a50 con 0x55b48e0742e0 2024-02-17T13:03:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa1a2009740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48e06a030 con 0x55b48e06ad60 2024-02-17T13:03:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa19c8f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 0x55b48e07b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa19c8f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 0x55b48e07b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48378/0 (socket says 172.21.15.18:48378) 2024-02-17T13:03:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa19c8f1640 1 -- 172.21.15.18:0/1898430280 learned_addr learned my addr 172.21.15.18:0/1898430280 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa19d8f3640 1 --2- 172.21.15.18:0/1898430280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa19d0f2640 1 --2- 172.21.15.18:0/1898430280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06ad60 0x55b48e07b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa19c8f1640 1 -- 172.21.15.18:0/1898430280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06ad60 msgr2=0x55b48e07b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.076+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/1898430280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06ad60 0x55b48e07b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.077+0000 7fa19c8f1640 1 -- 172.21.15.18:0/1898430280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 msgr2=0x55b48e071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.077+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/1898430280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.077+0000 7fa19c8f1640 1 -- 172.21.15.18:0/1898430280 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b48df6d7b0 con 0x55b48e06bad0 2024-02-17T13:03:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.077+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/1898430280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 0x55b48e07b640 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fa188012fd0 tx=0x7fa188009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948bf49b0ea269b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.077+0000 7fa182ffd640 1 -- 172.21.15.18:0/1898430280 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188019820 con 0x55b48e06bad0 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.077+0000 7fa182ffd640 1 -- 172.21.15.18:0/1898430280 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa18802be50 con 0x55b48e06bad0 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.077+0000 7fa182ffd640 1 -- 172.21.15.18:0/1898430280 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1880187b0 con 0x55b48e06bad0 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.077+0000 7fa1a2009740 1 -- 172.21.15.18:0/1898430280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 msgr2=0x55b48e07b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.077+0000 7fa1a2009740 1 --2- 172.21.15.18:0/1898430280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 0x55b48e07b640 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fa188012fd0 tx=0x7fa188009d70 comp rx=0 tx=0).stop 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 -- 172.21.15.18:0/1898430280 shutdown_connections 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 --2- 172.21.15.18:0/1898430280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 --2- 172.21.15.18:0/1898430280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06bad0 0x55b48e07b640 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 --2- 172.21.15.18:0/1898430280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06ad60 0x55b48e07b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 -- 172.21.15.18:0/1898430280 >> 172.21.15.18:0/1898430280 conn(0x55b48e032550 msgr2=0x55b48e06a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 -- 172.21.15.18:0/1898430280 shutdown_connections 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 -- 172.21.15.18:0/1898430280 wait complete. 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 -- start start 2024-02-17T13:03:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 Processor -- start 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 0x55b48e0d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06bad0 0x55b48e0d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e07ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48e07b050 con 0x55b48e06ad60 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48e07b190 con 0x55b48e0742e0 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48e07b350 con 0x55b48e06bad0 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa1a2009740 10 client.0 _ll_register_callbacks cb 0x7fa1880097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa19d0f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 0x55b48e0d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa19d0f2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 0x55b48e0d2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48392/0 (socket says 172.21.15.18:48392) 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.078+0000 7fa19d0f2640 1 -- 172.21.15.18:0/1874745717 learned_addr learned my addr 172.21.15.18:0/1874745717 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa19d8f3640 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e07ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06bad0 0x55b48e0d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa19d0f2640 1 -- 172.21.15.18:0/1874745717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06bad0 msgr2=0x55b48e0d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa19d0f2640 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06bad0 0x55b48e0d3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa19d0f2640 1 -- 172.21.15.18:0/1874745717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 msgr2=0x55b48e07ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa19d0f2640 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e07ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa19d0f2640 1 -- 172.21.15.18:0/1874745717 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b48dfd0d80 con 0x55b48e06ad60 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa19d0f2640 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 0x55b48e0d2c50 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fa194097720 tx=0x7fa1940058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf23e9cb4139173 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa180ff9640 1 -- 172.21.15.18:0/1874745717 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194004de0 con 0x55b48e06ad60 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa180ff9640 1 -- 172.21.15.18:0/1874745717 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa194097930 con 0x55b48e06ad60 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa1a2009740 1 -- 172.21.15.18:0/1874745717 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa188009e10 con 0x55b48e06ad60 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa180ff9640 1 -- 172.21.15.18:0/1874745717 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa19407c050 con 0x55b48e06ad60 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa1a2009740 10 client.20289 fetch_fsmap learned FSMap version 56 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa1a2009740 1 -- 172.21.15.18:0/1874745717 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55b48df93540 con 0x55b48e06ad60 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa180ff9640 1 -- 172.21.15.18:0/1874745717 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa188009e10 con 0x55b48e06ad60 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.079+0000 7fa180ff9640 1 -- 172.21.15.18:0/1874745717 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa19401f060 con 0x55b48e06ad60 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.080+0000 7fa1a2009740 10 client.20289 fetch_fsmap finished waiting for FSMap version 56 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.080+0000 7fa1a2009740 10 client.20289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.080+0000 7fa1a2009740 20 client.20289 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.080+0000 7fa1a2009740 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b48e0dbd50 0x55b48e0de170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.080+0000 7fa1a2009740 4 client.20289 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:03:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.080+0000 7fa1a2009740 1 -- 172.21.15.18:0/1874745717 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b48dd0f550 con 0x55b48e0dbd50 2024-02-17T13:03:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.080+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b48e0dbd50 0x55b48e0de170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.080+0000 7fa180ff9640 10 client.20289 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.080+0000 7fa19c8f1640 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b48e0dbd50 0x55b48e0de170 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbcd836193a64039 server_cookie=400e993afca282b6 in_seq=0 out_seq=0 2024-02-17T13:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa180ff9640 1 -- 172.21.15.18:0/1874745717 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b48dd0f550 con 0x55b48e0dbd50 2024-02-17T13:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa180ff9640 10 client.20289 handle_command_reply: tid=0 2024-02-17T13:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa1a2009740 1 client.20289 shutdown 2024-02-17T13:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa1a2009740 1 -- 172.21.15.18:0/1874745717 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 msgr2=0x55b48e0d2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa1a2009740 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 0x55b48e0d2c50 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fa194097720 tx=0x7fa1940058d0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa1a2009740 1 -- 172.21.15.18:0/1874745717 shutdown_connections 2024-02-17T13:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa1a2009740 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55b48e0dbd50 0x55b48e0de170 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa1a2009740 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b48e0742e0 0x55b48e07ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa1a2009740 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b48e06bad0 0x55b48e0d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa1a2009740 1 --2- 172.21.15.18:0/1874745717 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b48e06ad60 0x55b48e0d2c50 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa180ff9640 0 client.20289 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa1a2009740 1 -- 172.21.15.18:0/1874745717 >> 172.21.15.18:0/1874745717 conn(0x55b48e032550 msgr2=0x55b48e06fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa1a2009740 1 -- 172.21.15.18:0/1874745717 shutdown_connections 2024-02-17T13:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.584+0000 7fa1a2009740 1 -- 172.21.15.18:0/1874745717 wait complete. 2024-02-17T13:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa1a2009740 20 client.20289 trim_cache size 0 max 16384 2024-02-17T13:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa19f356640 1 -- 172.21.15.18:0/4131193008 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa1840a22d0 msgr2=0x7fa1840a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa19f356640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa1840a22d0 0x7fa1840a4790 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa190005ec0 tx=0x7fa19002c8d0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa19f356640 1 -- 172.21.15.18:0/4131193008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1981002d0 msgr2=0x7fa19819d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa19f356640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1981002d0 0x7fa19819d420 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa19400eb90 tx=0x7fa19400c6a0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa19f356640 1 -- 172.21.15.18:0/4131193008 shutdown_connections 2024-02-17T13:03:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa19f356640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa1840a22d0 0x7fa1840a4790 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa19f356640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa198105020 0x7fa1981975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa19f356640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa198100c90 0x7fa19819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa19f356640 1 --2- 172.21.15.18:0/4131193008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1981002d0 0x7fa19819d420 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa19f356640 1 -- 172.21.15.18:0/4131193008 >> 172.21.15.18:0/4131193008 conn(0x7fa198076ed0 msgr2=0x7fa198103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa19f356640 1 -- 172.21.15.18:0/4131193008 shutdown_connections 2024-02-17T13:03:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:15.585+0000 7fa19f356640 1 -- 172.21.15.18:0/4131193008 wait complete. 2024-02-17T13:03:15.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 11223 28.68 MB/sec warmup 27 sec latency 454.897 ms 2024-02-17T13:03:16.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:16 smithi146 ceph-mon[36454]: pgmap v421: 97 pgs: 97 active+clean; 303 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2024-02-17T13:03:16.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:16 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:16.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:16 smithi018 ceph-mon[32625]: pgmap v421: 97 pgs: 97 active+clean; 303 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2024-02-17T13:03:16.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:16 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:16.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 11632 28.49 MB/sec warmup 28 sec latency 1454.957 ms 2024-02-17T13:03:16.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:16 smithi119 ceph-mon[36188]: pgmap v421: 97 pgs: 97 active+clean; 303 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2024-02-17T13:03:16.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:16 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:17.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 12437 28.63 MB/sec warmup 29 sec latency 2455.015 ms 2024-02-17T13:03:18.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:18 smithi146 ceph-mon[36454]: pgmap v422: 97 pgs: 97 active+clean; 303 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 258 op/s 2024-02-17T13:03:18.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:18 smithi018 ceph-mon[32625]: pgmap v422: 97 pgs: 97 active+clean; 303 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 258 op/s 2024-02-17T13:03:18.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 12578 27.94 MB/sec warmup 30 sec latency 3455.047 ms 2024-02-17T13:03:18.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:18 smithi119 ceph-mon[36188]: pgmap v422: 97 pgs: 97 active+clean; 303 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 258 op/s 2024-02-17T13:03:19.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:03:19.126 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:03:19.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.254+0000 7f6190496640 1 -- 172.21.15.18:0/1684027285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6188101d20 msgr2=0x7f6188105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.254+0000 7f6190496640 1 --2- 172.21.15.18:0/1684027285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6188101d20 0x7f6188105d60 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f6178009930 tx=0x7f617802bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:19.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.254+0000 7f6190496640 1 -- 172.21.15.18:0/1684027285 shutdown_connections 2024-02-17T13:03:19.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.254+0000 7f6190496640 1 --2- 172.21.15.18:0/1684027285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6188106440 0x7f618810e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.254+0000 7f6190496640 1 --2- 172.21.15.18:0/1684027285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6188101d20 0x7f6188105d60 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.254+0000 7f6190496640 1 --2- 172.21.15.18:0/1684027285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6188101370 0x7f6188101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.254+0000 7f6190496640 1 -- 172.21.15.18:0/1684027285 >> 172.21.15.18:0/1684027285 conn(0x7f61880fcc30 msgr2=0x7f61880ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.254+0000 7f6190496640 1 -- 172.21.15.18:0/1684027285 shutdown_connections 2024-02-17T13:03:19.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.255+0000 7f6190496640 1 -- 172.21.15.18:0/1684027285 wait complete. 2024-02-17T13:03:19.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.255+0000 7f6190496640 1 Processor -- start 2024-02-17T13:03:19.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.255+0000 7f6190496640 1 -- start start 2024-02-17T13:03:19.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f6190496640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6188101370 0x7f6188074e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f6190496640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6188106440 0x7f6188070f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f6190496640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6188071490 0x7f6188071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f6190496640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61881ae780 con 0x7f6188101370 2024-02-17T13:03:19.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f6190496640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61881ae8f0 con 0x7f6188106440 2024-02-17T13:03:19.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f6190496640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61881aeab0 con 0x7f6188071490 2024-02-17T13:03:19.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f618e232640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6188101370 0x7f6188074e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f618ea33640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6188071490 0x7f6188071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f618e232640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6188101370 0x7f6188074e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48420/0 (socket says 172.21.15.18:48420) 2024-02-17T13:03:19.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f618ea33640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6188071490 0x7f6188071940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:50836/0 (socket says 172.21.15.18:50836) 2024-02-17T13:03:19.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f618ea33640 1 -- 172.21.15.18:0/1051462628 learned_addr learned my addr 172.21.15.18:0/1051462628 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:19.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f618da31640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6188106440 0x7f6188070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f618da31640 1 -- 172.21.15.18:0/1051462628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6188071490 msgr2=0x7f6188071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f618da31640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6188071490 0x7f6188071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f618da31640 1 -- 172.21.15.18:0/1051462628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6188101370 msgr2=0x7f6188074e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f618da31640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6188101370 0x7f6188074e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f618da31640 1 -- 172.21.15.18:0/1051462628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6178009560 con 0x7f6188106440 2024-02-17T13:03:19.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.256+0000 7f618ea33640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6188071490 0x7f6188071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:19.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.257+0000 7f618da31640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6188106440 0x7f6188070f50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6178009230 tx=0x7f617802cd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.257+0000 7f617f7fe640 1 -- 172.21.15.18:0/1051462628 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178004250 con 0x7f6188106440 2024-02-17T13:03:19.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.257+0000 7f617f7fe640 1 -- 172.21.15.18:0/1051462628 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61780043b0 con 0x7f6188106440 2024-02-17T13:03:19.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.257+0000 7f617f7fe640 1 -- 172.21.15.18:0/1051462628 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178003af0 con 0x7f6188106440 2024-02-17T13:03:19.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.257+0000 7f6190496640 1 -- 172.21.15.18:0/1051462628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61881aed30 con 0x7f6188106440 2024-02-17T13:03:19.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.257+0000 7f6190496640 1 -- 172.21.15.18:0/1051462628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618806ca70 con 0x7f6188106440 2024-02-17T13:03:19.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.260+0000 7f617f7fe640 1 -- 172.21.15.18:0/1051462628 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6178003c50 con 0x7f6188106440 2024-02-17T13:03:19.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.260+0000 7f617f7fe640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f61740a23c0 0x7f61740a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.261+0000 7f617f7fe640 1 -- 172.21.15.18:0/1051462628 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f61780f0410 con 0x7f6188106440 2024-02-17T13:03:19.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.261+0000 7f618e232640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f61740a23c0 0x7f61740a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.261+0000 7f6193062740 1 Processor -- start 2024-02-17T13:03:19.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.261+0000 7f618e232640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f61740a23c0 0x7f61740a4880 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6184009bd0 tx=0x7f6184005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.261+0000 7f6193062740 1 -- start start 2024-02-17T13:03:19.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f6193062740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4aa2b0 0x5646ed4aa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f6193062740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed4b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f6193062740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4b37b0 0x5646ed4b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f6193062740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ed419280 con 0x5646ed4aa2b0 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f6193062740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ed4167b0 con 0x5646ed4b37b0 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f6193062740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ed411f90 con 0x5646ed4aabd0 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618e232640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4aa2b0 0x5646ed4aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618e232640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4aa2b0 0x5646ed4aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48422/0 (socket says 172.21.15.18:48422) 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618ea33640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4b37b0 0x5646ed4b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618e232640 1 -- 172.21.15.18:0/924787742 learned_addr learned my addr 172.21.15.18:0/924787742 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618da31640 1 --2- 172.21.15.18:0/924787742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed4b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618e232640 1 -- 172.21.15.18:0/924787742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 msgr2=0x5646ed4b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618e232640 1 --2- 172.21.15.18:0/924787742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed4b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618e232640 1 -- 172.21.15.18:0/924787742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4b37b0 msgr2=0x5646ed4b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618e232640 1 --2- 172.21.15.18:0/924787742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4b37b0 0x5646ed4b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618e232640 1 -- 172.21.15.18:0/924787742 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646ed3d0570 con 0x5646ed4aa2b0 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618da31640 1 --2- 172.21.15.18:0/924787742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed4b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618ea33640 1 --2- 172.21.15.18:0/924787742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4b37b0 0x5646ed4b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:19.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.262+0000 7f618e232640 1 --2- 172.21.15.18:0/924787742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4aa2b0 0x5646ed4aa690 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f618400e630 tx=0x7f6184014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a103845d57c729f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.263+0000 7f617cbfe640 1 -- 172.21.15.18:0/924787742 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184004020 con 0x5646ed4aa2b0 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.263+0000 7f617cbfe640 1 -- 172.21.15.18:0/924787742 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6184021070 con 0x5646ed4aa2b0 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.263+0000 7f617cbfe640 1 -- 172.21.15.18:0/924787742 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618401c670 con 0x5646ed4aa2b0 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 -- 172.21.15.18:0/924787742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4aa2b0 msgr2=0x5646ed4aa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 --2- 172.21.15.18:0/924787742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4aa2b0 0x5646ed4aa690 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f618400e630 tx=0x7f6184014000 comp rx=0 tx=0).stop 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 -- 172.21.15.18:0/924787742 shutdown_connections 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 --2- 172.21.15.18:0/924787742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4b37b0 0x5646ed4b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 --2- 172.21.15.18:0/924787742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed4b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 --2- 172.21.15.18:0/924787742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4aa2b0 0x5646ed4aa690 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 -- 172.21.15.18:0/924787742 >> 172.21.15.18:0/924787742 conn(0x5646ed4a4990 msgr2=0x5646ed4a4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 -- 172.21.15.18:0/924787742 shutdown_connections 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 -- 172.21.15.18:0/924787742 wait complete. 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 -- start start 2024-02-17T13:03:19.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 Processor -- start 2024-02-17T13:03:19.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4aa2b0 0x5646ed4b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aabd0 0x5646ed4b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4b37b0 0x5646ed4b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ed0e5430 con 0x5646ed4aa2b0 2024-02-17T13:03:19.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ed0a1e00 con 0x5646ed4aabd0 2024-02-17T13:03:19.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.264+0000 7f6193062740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ed396fd0 con 0x5646ed4b37b0 2024-02-17T13:03:19.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f6193062740 10 client.0 _ll_register_callbacks cb 0x5646ed3ecfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:19.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f618da31640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aabd0 0x5646ed4b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f618da31640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aabd0 0x5646ed4b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:55086/0 (socket says 172.21.15.18:55086) 2024-02-17T13:03:19.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f618da31640 1 -- 172.21.15.18:0/477375590 learned_addr learned my addr 172.21.15.18:0/477375590 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f618e232640 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4aa2b0 0x5646ed4b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f618ea33640 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4b37b0 0x5646ed4b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f618da31640 1 -- 172.21.15.18:0/477375590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4b37b0 msgr2=0x5646ed4b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f618da31640 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4b37b0 0x5646ed4b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f618da31640 1 -- 172.21.15.18:0/477375590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4aa2b0 msgr2=0x5646ed4b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f618da31640 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4aa2b0 0x5646ed4b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f618da31640 1 -- 172.21.15.18:0/477375590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6184017070 con 0x5646ed4aabd0 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f618da31640 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aabd0 0x5646ed4b0560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f61780edcf0 tx=0x7f61780ba890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee836fe43443cab2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.265+0000 7f6171ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6171ffb640 1 -- 172.21.15.18:0/477375590 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61780d4070 con 0x5646ed4aabd0 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6171ffb640 1 -- 172.21.15.18:0/477375590 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f61780be780 con 0x5646ed4aabd0 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6193062740 1 -- 172.21.15.18:0/477375590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646ed1117e0 con 0x5646ed4aabd0 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6171ffb640 1 -- 172.21.15.18:0/477375590 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61780be080 con 0x5646ed4aabd0 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6193062740 10 client.29159 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:19.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6193062740 1 -- 172.21.15.18:0/477375590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5646ed410610 con 0x5646ed4aabd0 2024-02-17T13:03:19.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6171ffb640 1 -- 172.21.15.18:0/477375590 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5646ed1117e0 con 0x5646ed4aabd0 2024-02-17T13:03:19.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6171ffb640 1 -- 172.21.15.18:0/477375590 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f617803c070 con 0x5646ed4aabd0 2024-02-17T13:03:19.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6193062740 10 client.29159 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:19.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6193062740 10 client.29159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:19.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6193062740 20 client.29159 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:19.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6193062740 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5646ed4c3620 0x5646ed4c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6193062740 4 client.29159 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:19.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f6193062740 1 -- 172.21.15.18:0/477375590 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646ed14e720 con 0x5646ed4c3620 2024-02-17T13:03:19.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.266+0000 7f618e232640 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5646ed4c3620 0x5646ed4c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.267+0000 7f6171ffb640 10 client.29159 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:19.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.267+0000 7f618e232640 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5646ed4c3620 0x5646ed4c5a40 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f3f1ad15322ef0 server_cookie=5fae8951f8bc1db in_seq=0 out_seq=0 2024-02-17T13:03:19.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.269+0000 7f6171ffb640 1 -- 172.21.15.18:0/477375590 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5646ed14e720 con 0x5646ed4c3620 2024-02-17T13:03:19.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.269+0000 7f6171ffb640 10 client.29159 handle_command_reply: tid=0 2024-02-17T13:03:19.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.269+0000 7f6193062740 1 client.29159 shutdown 2024-02-17T13:03:19.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.269+0000 7f6193062740 1 -- 172.21.15.18:0/477375590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aabd0 msgr2=0x5646ed4b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.269+0000 7f6193062740 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aabd0 0x5646ed4b0560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f61780edcf0 tx=0x7f61780ba890 comp rx=0 tx=0).stop 2024-02-17T13:03:19.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.269+0000 7f6193062740 1 -- 172.21.15.18:0/477375590 shutdown_connections 2024-02-17T13:03:19.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.269+0000 7f6193062740 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5646ed4c3620 0x5646ed4c5a40 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.269+0000 7f6193062740 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4b37b0 0x5646ed4b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.269+0000 7f6193062740 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aabd0 0x5646ed4b0560 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.269+0000 7f6171ffb640 0 client.29159 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:19.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.269+0000 7f6193062740 1 --2- 172.21.15.18:0/477375590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4aa2b0 0x5646ed4b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.269+0000 7f6193062740 1 -- 172.21.15.18:0/477375590 >> 172.21.15.18:0/477375590 conn(0x5646ed4a4990 msgr2=0x5646ed4a6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.270+0000 7f6193062740 1 -- 172.21.15.18:0/477375590 shutdown_connections 2024-02-17T13:03:19.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.270+0000 7f6193062740 1 -- 172.21.15.18:0/477375590 wait complete. 2024-02-17T13:03:19.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.270+0000 7f6193062740 20 client.29159 trim_cache size 0 max 16384 2024-02-17T13:03:19.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.289+0000 7f6193062740 1 Processor -- start 2024-02-17T13:03:19.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f6193062740 1 -- start start 2024-02-17T13:03:19.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f6193062740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aa2b0 0x5646ed4b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f6193062740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed4b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f6193062740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 0x5646ed4ad800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f6193062740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ed4bfe70 con 0x5646ed4b37b0 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f6193062740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ed4b6480 con 0x5646ed4aa2b0 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f6193062740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ed4a59e0 con 0x5646ed4aabd0 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f618ea33640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 0x5646ed4ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f618da31640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed4b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f618ea33640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 0x5646ed4ad800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48438/0 (socket says 172.21.15.18:48438) 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f618ea33640 1 -- 172.21.15.18:0/2578605659 learned_addr learned my addr 172.21.15.18:0/2578605659 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f618e232640 1 --2- 172.21.15.18:0/2578605659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aa2b0 0x5646ed4b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f618da31640 1 -- 172.21.15.18:0/2578605659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aa2b0 msgr2=0x5646ed4b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f618da31640 1 --2- 172.21.15.18:0/2578605659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aa2b0 0x5646ed4b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f618da31640 1 -- 172.21.15.18:0/2578605659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 msgr2=0x5646ed4ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f618da31640 1 --2- 172.21.15.18:0/2578605659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 0x5646ed4ad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f618da31640 1 -- 172.21.15.18:0/2578605659 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646ed3acac0 con 0x5646ed4aabd0 2024-02-17T13:03:19.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f618ea33640 1 --2- 172.21.15.18:0/2578605659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 0x5646ed4ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.290+0000 7f618da31640 1 --2- 172.21.15.18:0/2578605659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed4b31d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6178003740 tx=0x7f617802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21aecbbc2ec8be31 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.291+0000 7f6173fff640 1 -- 172.21.15.18:0/2578605659 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61780d4030 con 0x5646ed4aabd0 2024-02-17T13:03:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.291+0000 7f6173fff640 1 -- 172.21.15.18:0/2578605659 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f617809ce90 con 0x5646ed4aabd0 2024-02-17T13:03:19.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.291+0000 7f6173fff640 1 -- 172.21.15.18:0/2578605659 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61780c21d0 con 0x5646ed4aabd0 2024-02-17T13:03:19.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.291+0000 7f6193062740 1 -- 172.21.15.18:0/2578605659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 msgr2=0x5646ed4b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.291+0000 7f6193062740 1 --2- 172.21.15.18:0/2578605659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed4b31d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6178003740 tx=0x7f617802bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:19.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.291+0000 7f6193062740 1 -- 172.21.15.18:0/2578605659 shutdown_connections 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.291+0000 7f6193062740 1 --2- 172.21.15.18:0/2578605659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 0x5646ed4ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.291+0000 7f6193062740 1 --2- 172.21.15.18:0/2578605659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed4b31d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.291+0000 7f6193062740 1 --2- 172.21.15.18:0/2578605659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aa2b0 0x5646ed4b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.291+0000 7f6193062740 1 -- 172.21.15.18:0/2578605659 >> 172.21.15.18:0/2578605659 conn(0x5646ed4a4990 msgr2=0x5646ed4aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.291+0000 7f6193062740 1 -- 172.21.15.18:0/2578605659 shutdown_connections 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.291+0000 7f6193062740 1 -- 172.21.15.18:0/2578605659 wait complete. 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f6193062740 1 -- start start 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f6193062740 1 Processor -- start 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f6193062740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aa2b0 0x5646ed513ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f6193062740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed514410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f6193062740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 0x5646ed4b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f6193062740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ed4b7f60 con 0x5646ed4b37b0 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f6193062740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ed4b80d0 con 0x5646ed4aa2b0 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f6193062740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ed4b8290 con 0x5646ed4aabd0 2024-02-17T13:03:19.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f6193062740 10 client.0 _ll_register_callbacks cb 0x7f6184009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f618e232640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aa2b0 0x5646ed513ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f618e232640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aa2b0 0x5646ed513ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:55102/0 (socket says 172.21.15.18:55102) 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f618e232640 1 -- 172.21.15.18:0/486572038 learned_addr learned my addr 172.21.15.18:0/486572038 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f618ea33640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 0x5646ed4b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f618da31640 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed514410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f618e232640 1 -- 172.21.15.18:0/486572038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 msgr2=0x5646ed514410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f618e232640 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed514410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f618e232640 1 -- 172.21.15.18:0/486572038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 msgr2=0x5646ed4b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f618e232640 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 0x5646ed4b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f618e232640 1 -- 172.21.15.18:0/486572038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6180010540 con 0x5646ed4aa2b0 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.292+0000 7f618ea33640 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 0x5646ed4b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.293+0000 7f618e232640 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aa2b0 0x5646ed513ed0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f618400eaf0 tx=0x7f6184003c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ddecf394672c3d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.293+0000 7f6171ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.293+0000 7f6171ffb640 1 -- 172.21.15.18:0/486572038 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618400ee60 con 0x5646ed4aa2b0 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.293+0000 7f6171ffb640 1 -- 172.21.15.18:0/486572038 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6184035e40 con 0x5646ed4aa2b0 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.293+0000 7f6193062740 1 -- 172.21.15.18:0/486572038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646ed4b0390 con 0x5646ed4aa2b0 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.293+0000 7f6171ffb640 1 -- 172.21.15.18:0/486572038 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61840039c0 con 0x5646ed4aa2b0 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.293+0000 7f6193062740 10 client.29165 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.293+0000 7f6193062740 1 -- 172.21.15.18:0/486572038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5646ed410610 con 0x5646ed4aa2b0 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.293+0000 7f6171ffb640 1 -- 172.21.15.18:0/486572038 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f618402b040 con 0x5646ed4aa2b0 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.294+0000 7f6171ffb640 1 -- 172.21.15.18:0/486572038 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f61840356e0 con 0x5646ed4aa2b0 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.294+0000 7f6193062740 10 client.29165 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.294+0000 7f6193062740 10 client.29165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.294+0000 7f6193062740 20 client.29165 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.294+0000 7f6193062740 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5646ed51bd20 0x5646ed51e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.294+0000 7f6193062740 4 client.29165 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.294+0000 7f6193062740 1 -- 172.21.15.18:0/486572038 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5646ed4b0390 con 0x5646ed51bd20 2024-02-17T13:03:19.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.294+0000 7f618da31640 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5646ed51bd20 0x5646ed51e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.294+0000 7f618da31640 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5646ed51bd20 0x5646ed51e140 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258680c1acc4e639 server_cookie=637a18973be20eee in_seq=0 out_seq=0 2024-02-17T13:03:19.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.294+0000 7f6171ffb640 10 client.29165 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:19.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.297+0000 7f6171ffb640 1 -- 172.21.15.18:0/486572038 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5646ed4b0390 con 0x5646ed51bd20 2024-02-17T13:03:19.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.297+0000 7f6171ffb640 10 client.29165 handle_command_reply: tid=0 2024-02-17T13:03:19.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.297+0000 7f6193062740 1 client.29165 shutdown 2024-02-17T13:03:19.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.297+0000 7f6193062740 1 -- 172.21.15.18:0/486572038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aa2b0 msgr2=0x5646ed513ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.297+0000 7f6193062740 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aa2b0 0x5646ed513ed0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f618400eaf0 tx=0x7f6184003c70 comp rx=0 tx=0).stop 2024-02-17T13:03:19.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6193062740 1 -- 172.21.15.18:0/486572038 shutdown_connections 2024-02-17T13:03:19.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6193062740 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5646ed51bd20 0x5646ed51e140 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6193062740 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646ed4b37b0 0x5646ed4b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6193062740 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5646ed4aabd0 0x5646ed514410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6193062740 1 --2- 172.21.15.18:0/486572038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646ed4aa2b0 0x5646ed513ed0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6171ffb640 0 client.29165 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:19.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6193062740 1 -- 172.21.15.18:0/486572038 >> 172.21.15.18:0/486572038 conn(0x5646ed4a4990 msgr2=0x5646ed4a3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6193062740 1 -- 172.21.15.18:0/486572038 shutdown_connections 2024-02-17T13:03:19.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6193062740 1 -- 172.21.15.18:0/486572038 wait complete. 2024-02-17T13:03:19.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6193062740 20 client.29165 trim_cache size 0 max 16384 2024-02-17T13:03:19.291 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:03:19.291 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T13:03:19.291 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T13:03:19.291 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:03:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6190496640 1 -- 172.21.15.18:0/1051462628 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f61740a23c0 msgr2=0x7f61740a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6190496640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f61740a23c0 0x7f61740a4880 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6184009bd0 tx=0x7f6184005f90 comp rx=0 tx=0).stop 2024-02-17T13:03:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6190496640 1 -- 172.21.15.18:0/1051462628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6188106440 msgr2=0x7f6188070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6190496640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6188106440 0x7f6188070f50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6178009230 tx=0x7f617802cd60 comp rx=0 tx=0).stop 2024-02-17T13:03:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6190496640 1 -- 172.21.15.18:0/1051462628 shutdown_connections 2024-02-17T13:03:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6190496640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f61740a23c0 0x7f61740a4880 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6190496640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6188071490 0x7f6188071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6190496640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6188106440 0x7f6188070f50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6190496640 1 --2- 172.21.15.18:0/1051462628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6188101370 0x7f6188074e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6190496640 1 -- 172.21.15.18:0/1051462628 >> 172.21.15.18:0/1051462628 conn(0x7f61880fcc30 msgr2=0x7f61880fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6190496640 1 -- 172.21.15.18:0/1051462628 shutdown_connections 2024-02-17T13:03:19.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.298+0000 7f6190496640 1 -- 172.21.15.18:0/1051462628 wait complete. 2024-02-17T13:03:19.300 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T13:03:19.300 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T13:03:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.433+0000 7fa3a26d4640 1 -- 172.21.15.18:0/3984681402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa39c1002f0 msgr2=0x7fa39c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.433+0000 7fa3a26d4640 1 --2- 172.21.15.18:0/3984681402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa39c1002f0 0x7fa39c1006d0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009930 tx=0x7fa38c02bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.433+0000 7fa3a26d4640 1 -- 172.21.15.18:0/3984681402 shutdown_connections 2024-02-17T13:03:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.433+0000 7fa3a26d4640 1 --2- 172.21.15.18:0/3984681402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c1053c0 0x7fa39c10cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.433+0000 7fa3a26d4640 1 --2- 172.21.15.18:0/3984681402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa39c100ca0 0x7fa39c104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.433+0000 7fa3a26d4640 1 --2- 172.21.15.18:0/3984681402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa39c1002f0 0x7fa39c1006d0 secure :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009930 tx=0x7fa38c02bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.433+0000 7fa3a26d4640 1 -- 172.21.15.18:0/3984681402 >> 172.21.15.18:0/3984681402 conn(0x7fa39c077180 msgr2=0x7fa39c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.433+0000 7fa3a26d4640 1 -- 172.21.15.18:0/3984681402 shutdown_connections 2024-02-17T13:03:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.433+0000 7fa3a26d4640 1 -- 172.21.15.18:0/3984681402 wait complete. 2024-02-17T13:03:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.434+0000 7fa3a26d4640 1 Processor -- start 2024-02-17T13:03:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.434+0000 7fa3a26d4640 1 -- start start 2024-02-17T13:03:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.434+0000 7fa3a26d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c1053c0 0x7fa39c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.434+0000 7fa3a26d4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa39c198e00 0x7fa39c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa3a26d4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa39c1a1930 0x7fa39c1a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa3a26d4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c1a43c0 con 0x7fa39c198e00 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa39b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa39c198e00 0x7fa39c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa39b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa39c198e00 0x7fa39c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48468/0 (socket says 172.21.15.18:48468) 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa39b7fe640 1 -- 172.21.15.18:0/673299401 learned_addr learned my addr 172.21.15.18:0/673299401 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa39bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c1053c0 0x7fa39c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa3a26d4640 1 -- 172.21.15.18:0/673299401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c1a4530 con 0x7fa39c1053c0 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa3a26d4640 1 -- 172.21.15.18:0/673299401 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c1a46f0 con 0x7fa39c1a1930 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa3a0c71640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa39c1a1930 0x7fa39c1a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa39b7fe640 1 -- 172.21.15.18:0/673299401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa39c1a1930 msgr2=0x7fa39c1a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa39b7fe640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa39c1a1930 0x7fa39c1a3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa39b7fe640 1 -- 172.21.15.18:0/673299401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c1053c0 msgr2=0x7fa39c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa39b7fe640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c1053c0 0x7fa39c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa39b7fe640 1 -- 172.21.15.18:0/673299401 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c009560 con 0x7fa39c198e00 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa39bfff640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c1053c0 0x7fa39c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa39b7fe640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa39c198e00 0x7fa39c199260 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fa39000d740 tx=0x7fa39000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.435+0000 7fa3997fa640 1 -- 172.21.15.18:0/673299401 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390009c30 con 0x7fa39c198e00 2024-02-17T13:03:19.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.436+0000 7fa3997fa640 1 -- 172.21.15.18:0/673299401 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa39000de50 con 0x7fa39c198e00 2024-02-17T13:03:19.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.436+0000 7fa3997fa640 1 -- 172.21.15.18:0/673299401 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390010460 con 0x7fa39c198e00 2024-02-17T13:03:19.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.436+0000 7fa3a26d4640 1 -- 172.21.15.18:0/673299401 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39c1a49d0 con 0x7fa39c198e00 2024-02-17T13:03:19.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.436+0000 7fa3a26d4640 1 -- 172.21.15.18:0/673299401 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39c1a4e90 con 0x7fa39c198e00 2024-02-17T13:03:19.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.437+0000 7fa3a26d4640 1 -- 172.21.15.18:0/673299401 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39c067f70 con 0x7fa39c198e00 2024-02-17T13:03:19.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.438+0000 7fa3997fa640 1 -- 172.21.15.18:0/673299401 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa390004f10 con 0x7fa39c198e00 2024-02-17T13:03:19.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.438+0000 7fa3997fa640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa3840a2230 0x7fa3840a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.438+0000 7fa39bfff640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa3840a2230 0x7fa3840a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.438+0000 7fa3997fa640 1 -- 172.21.15.18:0/673299401 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa3900cd980 con 0x7fa39c198e00 2024-02-17T13:03:19.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.439+0000 7fa39bfff640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa3840a2230 0x7fa3840a46f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009230 tx=0x7fa38c005e80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.444+0000 7fa3997fa640 1 -- 172.21.15.18:0/673299401 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa3900966e0 con 0x7fa39c198e00 2024-02-17T13:03:19.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.699+0000 7fa3a26d4640 1 -- 172.21.15.18:0/673299401 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa39c1085f0 con 0x7fa39c198e00 2024-02-17T13:03:19.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.700+0000 7fa3997fa640 1 -- 172.21.15.18:0/673299401 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+11401 (secure 0 0 0) 0x7fa390099da0 con 0x7fa39c198e00 2024-02-17T13:03:19.689 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:03:19.689 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T13:03:14.961869+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T13:03:19.690 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 57 2024-02-17T13:03:19.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.702+0000 7fa3a26d4640 1 -- 172.21.15.18:0/673299401 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa3840a2230 msgr2=0x7fa3840a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.702+0000 7fa3a26d4640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa3840a2230 0x7fa3840a46f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009230 tx=0x7fa38c005e80 comp rx=0 tx=0).stop 2024-02-17T13:03:19.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.702+0000 7fa3a26d4640 1 -- 172.21.15.18:0/673299401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa39c198e00 msgr2=0x7fa39c199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.702+0000 7fa3a26d4640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa39c198e00 0x7fa39c199260 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fa39000d740 tx=0x7fa39000dc00 comp rx=0 tx=0).stop 2024-02-17T13:03:19.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.702+0000 7fa3a26d4640 1 -- 172.21.15.18:0/673299401 shutdown_connections 2024-02-17T13:03:19.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.702+0000 7fa3a26d4640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa3840a2230 0x7fa3840a46f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.702+0000 7fa3a26d4640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa39c1a1930 0x7fa39c1a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.702+0000 7fa3a26d4640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa39c198e00 0x7fa39c199260 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.702+0000 7fa3a26d4640 1 --2- 172.21.15.18:0/673299401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c1053c0 0x7fa39c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.702+0000 7fa3a26d4640 1 -- 172.21.15.18:0/673299401 >> 172.21.15.18:0/673299401 conn(0x7fa39c077180 msgr2=0x7fa39c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.702+0000 7fa3a26d4640 1 -- 172.21.15.18:0/673299401 shutdown_connections 2024-02-17T13:03:19.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.702+0000 7fa3a26d4640 1 -- 172.21.15.18:0/673299401 wait complete. 2024-02-17T13:03:19.703 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:03:19.704 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T13:03:19.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 13144 28.49 MB/sec warmup 31 sec latency 4455.083 ms 2024-02-17T13:03:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.844+0000 7f747f088640 1 -- 172.21.15.18:0/1188403522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7478100c50 msgr2=0x7f7478104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.844+0000 7f747f088640 1 --2- 172.21.15.18:0/1188403522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7478100c50 0x7f7478104c90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7468009890 tx=0x7f746802bb70 comp rx=0 tx=0).stop 2024-02-17T13:03:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.845+0000 7f747f088640 1 -- 172.21.15.18:0/1188403522 shutdown_connections 2024-02-17T13:03:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.845+0000 7f747f088640 1 --2- 172.21.15.18:0/1188403522 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7478105370 0x7f747810cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.845+0000 7f747f088640 1 --2- 172.21.15.18:0/1188403522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7478100c50 0x7f7478104c90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.845+0000 7f747f088640 1 --2- 172.21.15.18:0/1188403522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74781002a0 0x7f7478100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.845+0000 7f747f088640 1 -- 172.21.15.18:0/1188403522 >> 172.21.15.18:0/1188403522 conn(0x7f7478076400 msgr2=0x7f7478076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.845+0000 7f747f088640 1 -- 172.21.15.18:0/1188403522 shutdown_connections 2024-02-17T13:03:19.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.845+0000 7f747f088640 1 -- 172.21.15.18:0/1188403522 wait complete. 2024-02-17T13:03:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f747f088640 1 Processor -- start 2024-02-17T13:03:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f747f088640 1 -- start start 2024-02-17T13:03:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f747f088640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74781002a0 0x7f7478101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f747f088640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7478105370 0x7f7478102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f747f088640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74781027b0 0x7f74781a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f747f088640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74781a5fb0 con 0x7f74781027b0 2024-02-17T13:03:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f747f088640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74781a6120 con 0x7f7478105370 2024-02-17T13:03:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f747f088640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74781a62e0 con 0x7f74781002a0 2024-02-17T13:03:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f746ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7478105370 0x7f7478102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f746ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7478105370 0x7f7478102270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:46358/0 (socket says 172.21.15.18:46358) 2024-02-17T13:03:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f746ffff640 1 -- 172.21.15.18:0/2729712225 learned_addr learned my addr 172.21.15.18:0/2729712225 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f747ce24640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74781002a0 0x7f7478101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.846+0000 7f747d625640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74781027b0 0x7f74781a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.847+0000 7f747d625640 1 -- 172.21.15.18:0/2729712225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74781002a0 msgr2=0x7f7478101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.847+0000 7f747d625640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74781002a0 0x7f7478101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.847+0000 7f747d625640 1 -- 172.21.15.18:0/2729712225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7478105370 msgr2=0x7f7478102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.847+0000 7f747d625640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7478105370 0x7f7478102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.847+0000 7f747d625640 1 -- 172.21.15.18:0/2729712225 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7468009490 con 0x7f74781027b0 2024-02-17T13:03:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.847+0000 7f747d625640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74781027b0 0x7f74781a59e0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f747000ef00 tx=0x7f747000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.847+0000 7f746dffb640 1 -- 172.21.15.18:0/2729712225 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470004b20 con 0x7f74781027b0 2024-02-17T13:03:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.847+0000 7f746dffb640 1 -- 172.21.15.18:0/2729712225 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7470003e50 con 0x7f74781027b0 2024-02-17T13:03:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.847+0000 7f746dffb640 1 -- 172.21.15.18:0/2729712225 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747000b570 con 0x7f74781027b0 2024-02-17T13:03:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.847+0000 7f747f088640 1 -- 172.21.15.18:0/2729712225 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74781a65c0 con 0x7f74781027b0 2024-02-17T13:03:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.847+0000 7f747f088640 1 -- 172.21.15.18:0/2729712225 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74781a6a80 con 0x7f74781027b0 2024-02-17T13:03:19.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.850+0000 7f746dffb640 1 -- 172.21.15.18:0/2729712225 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f747000a040 con 0x7f74781027b0 2024-02-17T13:03:19.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.851+0000 7f7481c68740 1 Processor -- start 2024-02-17T13:03:19.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.851+0000 7f7481c68740 1 -- start start 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.851+0000 7f746dffb640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74600a22e0 0x7f74600a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.851+0000 7f7481c68740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdaf060 0x5619bfdaf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.851+0000 7f7481c68740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdafa10 0x5619bfdc0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.851+0000 7f7481c68740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 0x5619bfdc3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.851+0000 7f7481c68740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bfd6a960 con 0x5619bfdaf060 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.851+0000 7f7481c68740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bfd1e310 con 0x5619bfdc0f60 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.851+0000 7f7481c68740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bfd1b840 con 0x5619bfdafa10 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.851+0000 7f746dffb640 1 -- 172.21.15.18:0/2729712225 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f74700cdee0 con 0x7f74781027b0 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f747d625640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 0x5619bfdc3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f747d625640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 0x5619bfdc3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:46370/0 (socket says 172.21.15.18:46370) 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f747d625640 1 -- 172.21.15.18:0/4000261343 learned_addr learned my addr 172.21.15.18:0/4000261343 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f746ffff640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74600a22e0 0x7f74600a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f746ffff640 1 --2- 172.21.15.18:0/4000261343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdafa10 0x5619bfdc0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f747ce24640 1 --2- 172.21.15.18:0/4000261343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdaf060 0x5619bfdaf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f747d625640 1 -- 172.21.15.18:0/4000261343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdafa10 msgr2=0x5619bfdc0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f747d625640 1 --2- 172.21.15.18:0/4000261343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdafa10 0x5619bfdc0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f747d625640 1 -- 172.21.15.18:0/4000261343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdaf060 msgr2=0x5619bfdaf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f747d625640 1 --2- 172.21.15.18:0/4000261343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdaf060 0x5619bfdaf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f747d625640 1 -- 172.21.15.18:0/4000261343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619bfcd9060 con 0x5619bfdc0f60 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f746ffff640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74600a22e0 0x7f74600a47a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f74781037c0 tx=0x7f7468035c70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f747d625640 1 --2- 172.21.15.18:0/4000261343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 0x5619bfdc3350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f747009a590 tx=0x7f747009a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf187db5c56b49d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f745effd640 1 -- 172.21.15.18:0/4000261343 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470077eb0 con 0x5619bfdc0f60 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f745effd640 1 -- 172.21.15.18:0/4000261343 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7470077cd0 con 0x5619bfdc0f60 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.852+0000 7f745effd640 1 -- 172.21.15.18:0/4000261343 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747009ea70 con 0x5619bfdc0f60 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.853+0000 7f7481c68740 1 -- 172.21.15.18:0/4000261343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 msgr2=0x5619bfdc3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.853+0000 7f7481c68740 1 --2- 172.21.15.18:0/4000261343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 0x5619bfdc3350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f747009a590 tx=0x7f747009a3d0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.853+0000 7f7481c68740 1 -- 172.21.15.18:0/4000261343 shutdown_connections 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.853+0000 7f7481c68740 1 --2- 172.21.15.18:0/4000261343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 0x5619bfdc3350 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.853+0000 7f7481c68740 1 --2- 172.21.15.18:0/4000261343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdafa10 0x5619bfdc0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.853+0000 7f7481c68740 1 --2- 172.21.15.18:0/4000261343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdaf060 0x5619bfdaf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.853+0000 7f7481c68740 1 -- 172.21.15.18:0/4000261343 >> 172.21.15.18:0/4000261343 conn(0x5619bfda9980 msgr2=0x5619bfda9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.853+0000 7f7481c68740 1 -- 172.21.15.18:0/4000261343 shutdown_connections 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.853+0000 7f7481c68740 1 -- 172.21.15.18:0/4000261343 wait complete. 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.853+0000 7f7481c68740 1 -- start start 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f7481c68740 1 Processor -- start 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f7481c68740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdaf060 0x5619bfdba820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f7481c68740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 0x5619bfdbad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f7481c68740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 0x5619bfdc0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f7481c68740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bfd17020 con 0x5619bfdafa10 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f7481c68740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf9ea430 con 0x5619bfdc0f60 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f7481c68740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf9a6e00 con 0x5619bfdaf060 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f746ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 0x5619bfdbad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f746ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 0x5619bfdbad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43780/0 (socket says 172.21.15.18:43780) 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f746ffff640 1 -- 172.21.15.18:0/3548961701 learned_addr learned my addr 172.21.15.18:0/3548961701 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:19.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f747d625640 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 0x5619bfdc0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f7481c68740 10 client.0 _ll_register_callbacks cb 0x5619bfcbecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f747ce24640 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdaf060 0x5619bfdba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f746ffff640 1 -- 172.21.15.18:0/3548961701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdaf060 msgr2=0x5619bfdba820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f746ffff640 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdaf060 0x5619bfdba820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f746ffff640 1 -- 172.21.15.18:0/3548961701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 msgr2=0x5619bfdc0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f746ffff640 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 0x5619bfdc0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f746ffff640 1 -- 172.21.15.18:0/3548961701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74781a6a80 con 0x5619bfdafa10 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.854+0000 7f747d625640 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 0x5619bfdc0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f746ffff640 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 0x5619bfdbad60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f746802c180 tx=0x7f74680058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe570ee491cd5439 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7457fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7457fff640 1 -- 172.21.15.18:0/3548961701 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468048070 con 0x5619bfdafa10 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7457fff640 1 -- 172.21.15.18:0/3548961701 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7468034920 con 0x5619bfdafa10 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7481c68740 1 -- 172.21.15.18:0/3548961701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619bfa167e0 con 0x5619bfdafa10 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7457fff640 1 -- 172.21.15.18:0/3548961701 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7468004520 con 0x5619bfdafa10 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7481c68740 10 client.20325 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:19.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7481c68740 1 -- 172.21.15.18:0/3548961701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5619bfd156a0 con 0x5619bfdafa10 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7457fff640 1 -- 172.21.15.18:0/3548961701 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5619bfa167e0 con 0x5619bfdafa10 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7457fff640 1 -- 172.21.15.18:0/3548961701 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f7468004ac0 con 0x5619bfdafa10 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7481c68740 10 client.20325 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7481c68740 10 client.20325 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7481c68740 20 client.20325 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.855+0000 7f7481c68740 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5619bfdd9c40 0x5619bfddc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.856+0000 7f7481c68740 4 client.20325 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.856+0000 7f7481c68740 1 -- 172.21.15.18:0/3548961701 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619bfa53720 con 0x5619bfdd9c40 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.856+0000 7f747ce24640 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5619bfdd9c40 0x5619bfddc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.856+0000 7f7457fff640 10 client.20325 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.856+0000 7f747ce24640 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5619bfdd9c40 0x5619bfddc060 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1db52a294af1cc9f server_cookie=59567d21150c015 in_seq=0 out_seq=0 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.857+0000 7f7457fff640 1 -- 172.21.15.18:0/3548961701 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619bfa53720 con 0x5619bfdd9c40 2024-02-17T13:03:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.857+0000 7f7457fff640 10 client.20325 handle_command_reply: tid=0 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.857+0000 7f7481c68740 1 client.20325 shutdown 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.857+0000 7f7481c68740 1 -- 172.21.15.18:0/3548961701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 msgr2=0x5619bfdbad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.857+0000 7f7481c68740 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 0x5619bfdbad60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f746802c180 tx=0x7f74680058d0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.857+0000 7f7481c68740 1 -- 172.21.15.18:0/3548961701 shutdown_connections 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.857+0000 7f7481c68740 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5619bfdd9c40 0x5619bfddc060 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.857+0000 7f7481c68740 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfdc0f60 0x5619bfdc0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.857+0000 7f7481c68740 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 0x5619bfdbad60 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.857+0000 7f7481c68740 1 --2- 172.21.15.18:0/3548961701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdaf060 0x5619bfdba820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.857+0000 7f7481c68740 1 -- 172.21.15.18:0/3548961701 >> 172.21.15.18:0/3548961701 conn(0x5619bfda9980 msgr2=0x5619bfdab6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.857+0000 7f7457fff640 0 client.20325 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.858+0000 7f7481c68740 1 -- 172.21.15.18:0/3548961701 shutdown_connections 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.858+0000 7f7481c68740 1 -- 172.21.15.18:0/3548961701 wait complete. 2024-02-17T13:03:19.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.858+0000 7f7481c68740 20 client.20325 trim_cache size 0 max 16384 2024-02-17T13:03:19.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f7481c68740 1 Processor -- start 2024-02-17T13:03:19.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f7481c68740 1 -- start start 2024-02-17T13:03:19.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f7481c68740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfda9980 0x5619bfdc04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f7481c68740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdaf060 0x5619bfdb2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f7481c68740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdafa10 0x5619bfdb33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f7481c68740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bfc9c130 con 0x5619bfdaf060 2024-02-17T13:03:19.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f7481c68740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bfdb56e0 con 0x5619bfda9980 2024-02-17T13:03:19.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f7481c68740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bfdaa1c0 con 0x5619bfdafa10 2024-02-17T13:03:19.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f746ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdaf060 0x5619bfdb2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f747ce24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfda9980 0x5619bfdc04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f746ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdaf060 0x5619bfdb2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43784/0 (socket says 172.21.15.18:43784) 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f746ffff640 1 -- 172.21.15.18:0/494521757 learned_addr learned my addr 172.21.15.18:0/494521757 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.873+0000 7f747d625640 1 --2- 172.21.15.18:0/494521757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdafa10 0x5619bfdb33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.874+0000 7f747ce24640 1 -- 172.21.15.18:0/494521757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdafa10 msgr2=0x5619bfdb33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.874+0000 7f747ce24640 1 --2- 172.21.15.18:0/494521757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdafa10 0x5619bfdb33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.874+0000 7f747ce24640 1 -- 172.21.15.18:0/494521757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdaf060 msgr2=0x5619bfdb2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.874+0000 7f747ce24640 1 --2- 172.21.15.18:0/494521757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdaf060 0x5619bfdb2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.874+0000 7f747ce24640 1 -- 172.21.15.18:0/494521757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619bfcb1c20 con 0x5619bfda9980 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.874+0000 7f746ffff640 1 --2- 172.21.15.18:0/494521757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdaf060 0x5619bfdb2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.874+0000 7f747ce24640 1 --2- 172.21.15.18:0/494521757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfda9980 0x5619bfdc04c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x5619bfddc8d0 tx=0x7f74740046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96eded4961359a1b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.874+0000 7f745e7fc640 1 -- 172.21.15.18:0/494521757 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474003d20 con 0x5619bfda9980 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.874+0000 7f745e7fc640 1 -- 172.21.15.18:0/494521757 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7474018040 con 0x5619bfda9980 2024-02-17T13:03:19.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.874+0000 7f745e7fc640 1 -- 172.21.15.18:0/494521757 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74740108f0 con 0x5619bfda9980 2024-02-17T13:03:19.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.875+0000 7f7481c68740 1 -- 172.21.15.18:0/494521757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfda9980 msgr2=0x5619bfdc04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.875+0000 7f7481c68740 1 --2- 172.21.15.18:0/494521757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfda9980 0x5619bfdc04c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x5619bfddc8d0 tx=0x7f74740046c0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.875+0000 7f7481c68740 1 -- 172.21.15.18:0/494521757 shutdown_connections 2024-02-17T13:03:19.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.875+0000 7f7481c68740 1 --2- 172.21.15.18:0/494521757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdafa10 0x5619bfdb33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.875+0000 7f7481c68740 1 --2- 172.21.15.18:0/494521757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdaf060 0x5619bfdb2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.875+0000 7f7481c68740 1 --2- 172.21.15.18:0/494521757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfda9980 0x5619bfdc04c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.875+0000 7f7481c68740 1 -- 172.21.15.18:0/494521757 >> 172.21.15.18:0/494521757 conn(0x5619bfda4e80 msgr2=0x5619bfdb44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.875+0000 7f7481c68740 1 -- 172.21.15.18:0/494521757 shutdown_connections 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.875+0000 7f7481c68740 1 -- 172.21.15.18:0/494521757 wait complete. 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.875+0000 7f7481c68740 1 -- start start 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.875+0000 7f7481c68740 1 Processor -- start 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f7481c68740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfda9980 0x5619bfe29de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f7481c68740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdaf060 0x5619bfe2a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f7481c68740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 0x5619bfe241f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f7481c68740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bfe247b0 con 0x5619bfdafa10 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f7481c68740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bfe24920 con 0x5619bfda9980 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f7481c68740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bfe24ae0 con 0x5619bfdaf060 2024-02-17T13:03:19.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f7481c68740 10 client.0 _ll_register_callbacks cb 0x7f7474005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f747d625640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 0x5619bfe241f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f747d625640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 0x5619bfe241f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43786/0 (socket says 172.21.15.18:43786) 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f747d625640 1 -- 172.21.15.18:0/2044328029 learned_addr learned my addr 172.21.15.18:0/2044328029 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f747ce24640 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfda9980 0x5619bfe29de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f746ffff640 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdaf060 0x5619bfe2a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f747d625640 1 -- 172.21.15.18:0/2044328029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdaf060 msgr2=0x5619bfe2a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f747d625640 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdaf060 0x5619bfe2a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f747d625640 1 -- 172.21.15.18:0/2044328029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfda9980 msgr2=0x5619bfe29de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f747d625640 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfda9980 0x5619bfe29de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f747d625640 1 -- 172.21.15.18:0/2044328029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619bfd156a0 con 0x5619bfdafa10 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.876+0000 7f747ce24640 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfda9980 0x5619bfe29de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.877+0000 7f747d625640 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 0x5619bfe241f0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7470009be0 tx=0x7f7470013ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf3715b5b645d2a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.877+0000 7f7457fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.877+0000 7f7457fff640 1 -- 172.21.15.18:0/2044328029 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747009af90 con 0x5619bfdafa10 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.877+0000 7f7457fff640 1 -- 172.21.15.18:0/2044328029 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f74700a3030 con 0x5619bfdafa10 2024-02-17T13:03:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.877+0000 7f7481c68740 1 -- 172.21.15.18:0/2044328029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7470077cd0 con 0x5619bfdafa10 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.877+0000 7f7457fff640 1 -- 172.21.15.18:0/2044328029 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747009a890 con 0x5619bfdafa10 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.877+0000 7f7481c68740 10 client.20331 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.877+0000 7f7481c68740 1 -- 172.21.15.18:0/2044328029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x7f7474009e30 con 0x5619bfdafa10 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.877+0000 7f7457fff640 1 -- 172.21.15.18:0/2044328029 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f747007d080 con 0x5619bfdafa10 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f7457fff640 1 -- 172.21.15.18:0/2044328029 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f74700cb870 con 0x5619bfdafa10 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f7481c68740 10 client.20331 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f7481c68740 10 client.20331 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f7481c68740 20 client.20331 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f7481c68740 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5619bfdc0f60 0x5619bfe33aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f7481c68740 4 client.20331 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f7481c68740 1 -- 172.21.15.18:0/2044328029 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5619bfaa2890 con 0x5619bfdc0f60 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f747ce24640 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5619bfdc0f60 0x5619bfe33aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f7457fff640 10 client.20331 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:03:19.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f747ce24640 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5619bfdc0f60 0x5619bfe33aa0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=27c0144847e760e2 server_cookie=4918006e6929fd04 in_seq=0 out_seq=0 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f7457fff640 1 -- 172.21.15.18:0/2044328029 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5619bfaa2890 con 0x5619bfdc0f60 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f7457fff640 10 client.20331 handle_command_reply: tid=0 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.878+0000 7f7481c68740 1 client.20331 shutdown 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f7481c68740 1 -- 172.21.15.18:0/2044328029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 msgr2=0x5619bfe241f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f7481c68740 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 0x5619bfe241f0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7470009be0 tx=0x7f7470013ff0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f7481c68740 1 -- 172.21.15.18:0/2044328029 shutdown_connections 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f7481c68740 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619bfdafa10 0x5619bfe241f0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f7481c68740 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619bfdaf060 0x5619bfe2a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f7481c68740 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x5619bfdc0f60 0x5619bfe33aa0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f7481c68740 1 --2- 172.21.15.18:0/2044328029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bfda9980 0x5619bfe29de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f7457fff640 0 client.20331 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f7481c68740 1 -- 172.21.15.18:0/2044328029 >> 172.21.15.18:0/2044328029 conn(0x5619bfda4e80 msgr2=0x5619bfdaadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f7481c68740 1 -- 172.21.15.18:0/2044328029 shutdown_connections 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f7481c68740 1 -- 172.21.15.18:0/2044328029 wait complete. 2024-02-17T13:03:19.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f7481c68740 20 client.20331 trim_cache size 0 max 16384 2024-02-17T13:03:19.874 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:03:19.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f747f088640 1 -- 172.21.15.18:0/2729712225 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74600a22e0 msgr2=0x7f74600a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f747f088640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74600a22e0 0x7f74600a47a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f74781037c0 tx=0x7f7468035c70 comp rx=0 tx=0).stop 2024-02-17T13:03:19.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f747f088640 1 -- 172.21.15.18:0/2729712225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74781027b0 msgr2=0x7f74781a59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:19.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f747f088640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74781027b0 0x7f74781a59e0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f747000ef00 tx=0x7f747000c740 comp rx=0 tx=0).stop 2024-02-17T13:03:19.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f747f088640 1 -- 172.21.15.18:0/2729712225 shutdown_connections 2024-02-17T13:03:19.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f747f088640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74600a22e0 0x7f74600a47a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f747f088640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74781027b0 0x7f74781a59e0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f747f088640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7478105370 0x7f7478102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f747f088640 1 --2- 172.21.15.18:0/2729712225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74781002a0 0x7f7478101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:19.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.879+0000 7f747f088640 1 -- 172.21.15.18:0/2729712225 >> 172.21.15.18:0/2729712225 conn(0x7f7478076400 msgr2=0x7f7478069bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:19.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.880+0000 7f747f088640 1 -- 172.21.15.18:0/2729712225 shutdown_connections 2024-02-17T13:03:19.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:19.880+0000 7f747f088640 1 -- 172.21.15.18:0/2729712225 wait complete. 2024-02-17T13:03:19.880 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:03:19.880 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T13:03:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.012+0000 7fae518d4640 1 -- 172.21.15.18:0/1268845372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae4c100c10 msgr2=0x7fae4c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.012+0000 7fae518d4640 1 --2- 172.21.15.18:0/1268845372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae4c100c10 0x7fae4c104c50 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fae40009930 tx=0x7fae4002bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.012+0000 7fae518d4640 1 -- 172.21.15.18:0/1268845372 shutdown_connections 2024-02-17T13:03:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.012+0000 7fae518d4640 1 --2- 172.21.15.18:0/1268845372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae4c105330 0x7fae4c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.012+0000 7fae518d4640 1 --2- 172.21.15.18:0/1268845372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae4c100c10 0x7fae4c104c50 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.012+0000 7fae518d4640 1 --2- 172.21.15.18:0/1268845372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae4c100260 0x7fae4c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.012+0000 7fae518d4640 1 -- 172.21.15.18:0/1268845372 >> 172.21.15.18:0/1268845372 conn(0x7fae4c077180 msgr2=0x7fae4c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.012+0000 7fae518d4640 1 -- 172.21.15.18:0/1268845372 shutdown_connections 2024-02-17T13:03:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.013+0000 7fae518d4640 1 -- 172.21.15.18:0/1268845372 wait complete. 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.013+0000 7fae518d4640 1 Processor -- start 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.013+0000 7fae518d4640 1 -- start start 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.013+0000 7fae518d4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae4c100260 0x7fae4c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.013+0000 7fae518d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae4c105330 0x7fae4c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.013+0000 7fae518d4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae4c197600 0x7fae4c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.013+0000 7fae518d4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c197ff0 con 0x7fae4c197600 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.013+0000 7fae518d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c198160 con 0x7fae4c105330 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.013+0000 7fae518d4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c198320 con 0x7fae4c100260 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae4b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae4c197600 0x7fae4c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae4b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae4c197600 0x7fae4c197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43800/0 (socket says 172.21.15.18:43800) 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae4b7fe640 1 -- 172.21.15.18:0/2942334932 learned_addr learned my addr 172.21.15.18:0/2942334932 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae4affd640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae4c100260 0x7fae4c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae4c105330 0x7fae4c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae4b7fe640 1 -- 172.21.15.18:0/2942334932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae4c100260 msgr2=0x7fae4c19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae4b7fe640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae4c100260 0x7fae4c19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae4b7fe640 1 -- 172.21.15.18:0/2942334932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae4c105330 msgr2=0x7fae4c19d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae4b7fe640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae4c105330 0x7fae4c19d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae4b7fe640 1 -- 172.21.15.18:0/2942334932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c009490 con 0x7fae4c197600 2024-02-17T13:03:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae4affd640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae4c100260 0x7fae4c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae4b7fe640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae4c197600 0x7fae4c197ab0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fae4400eb10 tx=0x7fae4400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae508d2640 1 -- 172.21.15.18:0/2942334932 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44004b20 con 0x7fae4c197600 2024-02-17T13:03:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae518d4640 1 -- 172.21.15.18:0/2942334932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae40009560 con 0x7fae4c197600 2024-02-17T13:03:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae508d2640 1 -- 172.21.15.18:0/2942334932 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae44004c80 con 0x7fae4c197600 2024-02-17T13:03:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae508d2640 1 -- 172.21.15.18:0/2942334932 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44010650 con 0x7fae4c197600 2024-02-17T13:03:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.014+0000 7fae518d4640 1 -- 172.21.15.18:0/2942334932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c1a4660 con 0x7fae4c197600 2024-02-17T13:03:20.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.017+0000 7fae54587740 1 Processor -- start 2024-02-17T13:03:20.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.017+0000 7fae508d2640 1 -- 172.21.15.18:0/2942334932 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fae44005020 con 0x7fae4c197600 2024-02-17T13:03:20.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae54587740 1 -- start start 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae54587740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4250 0x55c5847e4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae54587740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4b70 0x55c5847ed210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae54587740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847ed750 0x55c5847efb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae54587740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58479f960 con 0x55c5847e4b70 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae54587740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c584753310 con 0x55c5847ed750 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae54587740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c584750840 con 0x55c5847e4250 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae4a7fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4250 0x55c5847e4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae4affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4b70 0x55c5847ed210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae4affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4b70 0x55c5847ed210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43816/0 (socket says 172.21.15.18:43816) 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae4affd640 1 -- 172.21.15.18:0/3294794086 learned_addr learned my addr 172.21.15.18:0/3294794086 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/3294794086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847ed750 0x55c5847efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae4affd640 1 -- 172.21.15.18:0/3294794086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4250 msgr2=0x55c5847e4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae508d2640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fae340a22b0 0x7fae340a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae4affd640 1 --2- 172.21.15.18:0/3294794086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4250 0x55c5847e4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae4affd640 1 -- 172.21.15.18:0/3294794086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847ed750 msgr2=0x55c5847efb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae4affd640 1 --2- 172.21.15.18:0/3294794086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847ed750 0x55c5847efb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae4affd640 1 -- 172.21.15.18:0/3294794086 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c58470e060 con 0x55c5847e4b70 2024-02-17T13:03:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/3294794086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847ed750 0x55c5847efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.018+0000 7fae4b7fe640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fae340a22b0 0x7fae340a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.019+0000 7fae508d2640 1 -- 172.21.15.18:0/2942334932 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fae44014070 con 0x7fae4c197600 2024-02-17T13:03:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.019+0000 7fae4affd640 1 --2- 172.21.15.18:0/3294794086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4b70 0x55c5847ed210 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0117e0 tx=0x7fae3c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cef48402f25fe9b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.019+0000 7fae31ffb640 1 -- 172.21.15.18:0/3294794086 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00fc60 con 0x55c5847e4b70 2024-02-17T13:03:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.019+0000 7fae31ffb640 1 -- 172.21.15.18:0/3294794086 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae3c00fdc0 con 0x55c5847e4b70 2024-02-17T13:03:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.019+0000 7fae31ffb640 1 -- 172.21.15.18:0/3294794086 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00e7e0 con 0x55c5847e4b70 2024-02-17T13:03:20.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.020+0000 7fae54587740 1 -- 172.21.15.18:0/3294794086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4b70 msgr2=0x55c5847ed210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.020+0000 7fae4b7fe640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fae340a22b0 0x7fae340a4770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fae440cbd40 tx=0x7fae4406cef0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.020+0000 7fae54587740 1 --2- 172.21.15.18:0/3294794086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4b70 0x55c5847ed210 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0117e0 tx=0x7fae3c011ca0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.020+0000 7fae54587740 1 -- 172.21.15.18:0/3294794086 shutdown_connections 2024-02-17T13:03:20.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.020+0000 7fae54587740 1 --2- 172.21.15.18:0/3294794086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847ed750 0x55c5847efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.020+0000 7fae54587740 1 --2- 172.21.15.18:0/3294794086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4b70 0x55c5847ed210 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.020+0000 7fae54587740 1 --2- 172.21.15.18:0/3294794086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4250 0x55c5847e4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.020+0000 7fae54587740 1 -- 172.21.15.18:0/3294794086 >> 172.21.15.18:0/3294794086 conn(0x55c5847de980 msgr2=0x55c5847ded60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.020+0000 7fae54587740 1 -- 172.21.15.18:0/3294794086 shutdown_connections 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.020+0000 7fae54587740 1 -- 172.21.15.18:0/3294794086 wait complete. 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.020+0000 7fae54587740 1 -- start start 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.020+0000 7fae54587740 1 Processor -- start 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae54587740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 0x55c5847ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae54587740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4b70 0x55c5847ead50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae54587740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847ed750 0x55c5847eb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae54587740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58474c020 con 0x55c5847e4250 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae4a7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 0x55c5847ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae54587740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58441f430 con 0x55c5847ed750 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae54587740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5843dbe00 con 0x55c5847e4b70 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae4a7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 0x55c5847ea810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43830/0 (socket says 172.21.15.18:43830) 2024-02-17T13:03:20.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae4a7fc640 1 -- 172.21.15.18:0/3402129901 learned_addr learned my addr 172.21.15.18:0/3402129901 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae4affd640 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4b70 0x55c5847ead50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae54587740 10 client.0 _ll_register_callbacks cb 0x55c5846f3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847ed750 0x55c5847eb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae4a7fc640 1 -- 172.21.15.18:0/3402129901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4b70 msgr2=0x55c5847ead50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4b70 0x55c5847ead50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae4a7fc640 1 -- 172.21.15.18:0/3402129901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847ed750 msgr2=0x55c5847eb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847ed750 0x55c5847eb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.021+0000 7fae4a7fc640 1 -- 172.21.15.18:0/3402129901 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c011440 con 0x55c5847e4250 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 0x55c5847ea810 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fae40005ab0 tx=0x7fae40009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73372f868b474ece server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae2affd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae2affd640 1 -- 172.21.15.18:0/3402129901 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40047070 con 0x55c5847e4250 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae2affd640 1 -- 172.21.15.18:0/3402129901 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fae40004100 con 0x55c5847e4250 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae54587740 1 -- 172.21.15.18:0/3402129901 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c58444b7e0 con 0x55c5847e4250 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae2affd640 1 -- 172.21.15.18:0/3402129901 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae400042c0 con 0x55c5847e4250 2024-02-17T13:03:20.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae54587740 10 client.20343 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae54587740 1 -- 172.21.15.18:0/3402129901 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55c58474a6a0 con 0x55c5847e4250 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae2affd640 1 -- 172.21.15.18:0/3402129901 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c58444b7e0 con 0x55c5847e4250 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae2affd640 1 -- 172.21.15.18:0/3402129901 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fae4003c040 con 0x55c5847e4250 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae54587740 10 client.20343 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae54587740 10 client.20343 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae54587740 20 client.20343 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae54587740 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c5847fdb00 0x55c5847fff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae54587740 4 client.20343 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.022+0000 7fae54587740 1 -- 172.21.15.18:0/3402129901 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c584488720 con 0x55c5847fdb00 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.023+0000 7fae4affd640 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c5847fdb00 0x55c5847fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.023+0000 7fae4affd640 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c5847fdb00 0x55c5847fff40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d0ead39f8e2f3164 server_cookie=6af6db523d9811aa in_seq=0 out_seq=0 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.023+0000 7fae2affd640 10 client.20343 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae2affd640 1 -- 172.21.15.18:0/3402129901 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c584488720 con 0x55c5847fdb00 2024-02-17T13:03:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae2affd640 10 client.20343 handle_command_reply: tid=0 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae54587740 1 client.20343 shutdown 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae54587740 1 -- 172.21.15.18:0/3402129901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 msgr2=0x55c5847ea810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae54587740 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 0x55c5847ea810 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fae40005ab0 tx=0x7fae40009260 comp rx=0 tx=0).stop 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae54587740 1 -- 172.21.15.18:0/3402129901 shutdown_connections 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae54587740 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c5847fdb00 0x55c5847fff40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae54587740 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847ed750 0x55c5847eb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae54587740 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4b70 0x55c5847ead50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae54587740 1 --2- 172.21.15.18:0/3402129901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 0x55c5847ea810 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae2affd640 0 client.20343 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae54587740 1 -- 172.21.15.18:0/3402129901 >> 172.21.15.18:0/3402129901 conn(0x55c5847de980 msgr2=0x55c5847e06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae54587740 1 -- 172.21.15.18:0/3402129901 shutdown_connections 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae54587740 1 -- 172.21.15.18:0/3402129901 wait complete. 2024-02-17T13:03:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.024+0000 7fae54587740 20 client.20343 trim_cache size 0 max 16384 2024-02-17T13:03:20.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.038+0000 7fae54587740 1 Processor -- start 2024-02-17T13:03:20.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.038+0000 7fae54587740 1 -- start start 2024-02-17T13:03:20.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae54587740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 0x55c5847ecf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae54587740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 0x55c5847e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae54587740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847ed750 0x55c5847e9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae54587740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5846d1130 con 0x55c5847e4250 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae54587740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5847fd900 con 0x55c5847e4b70 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae54587740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5847ea600 con 0x55c5847ed750 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae4a7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 0x55c5847ecf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae4affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 0x55c5847e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae4a7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 0x55c5847ecf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43846/0 (socket says 172.21.15.18:43846) 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae4a7fc640 1 -- 172.21.15.18:0/1942992406 learned_addr learned my addr 172.21.15.18:0/1942992406 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/1942992406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847ed750 0x55c5847e9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae4a7fc640 1 -- 172.21.15.18:0/1942992406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847ed750 msgr2=0x55c5847e9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/1942992406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847ed750 0x55c5847e9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae4a7fc640 1 -- 172.21.15.18:0/1942992406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 msgr2=0x55c5847e90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/1942992406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 0x55c5847e90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae4a7fc640 1 -- 172.21.15.18:0/1942992406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5846e6c20 con 0x55c5847e4250 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae4affd640 1 --2- 172.21.15.18:0/1942992406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 0x55c5847e90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/1942992406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 0x55c5847ecf40 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fae4002cd90 tx=0x7fae4002cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4080aeb27ed3561 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.039+0000 7fae317fa640 1 -- 172.21.15.18:0/1942992406 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40047040 con 0x55c5847e4250 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.040+0000 7fae317fa640 1 -- 172.21.15.18:0/1942992406 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae40049070 con 0x55c5847e4250 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.040+0000 7fae317fa640 1 -- 172.21.15.18:0/1942992406 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae400035f0 con 0x55c5847e4250 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.040+0000 7fae54587740 1 -- 172.21.15.18:0/1942992406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 msgr2=0x55c5847ecf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.040+0000 7fae54587740 1 --2- 172.21.15.18:0/1942992406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 0x55c5847ecf40 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fae4002cd90 tx=0x7fae4002cd20 comp rx=0 tx=0).stop 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.040+0000 7fae54587740 1 -- 172.21.15.18:0/1942992406 shutdown_connections 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.040+0000 7fae54587740 1 --2- 172.21.15.18:0/1942992406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847ed750 0x55c5847e9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.040+0000 7fae54587740 1 --2- 172.21.15.18:0/1942992406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 0x55c5847e90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.040+0000 7fae54587740 1 --2- 172.21.15.18:0/1942992406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847e4250 0x55c5847ecf40 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.040+0000 7fae54587740 1 -- 172.21.15.18:0/1942992406 >> 172.21.15.18:0/1942992406 conn(0x55c5847de980 msgr2=0x55c5847e5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.040+0000 7fae54587740 1 -- 172.21.15.18:0/1942992406 shutdown_connections 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.040+0000 7fae54587740 1 -- 172.21.15.18:0/1942992406 wait complete. 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.041+0000 7fae54587740 1 -- start start 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.041+0000 7fae54587740 1 Processor -- start 2024-02-17T13:03:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.041+0000 7fae54587740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4250 0x55c5847f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.041+0000 7fae54587740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 0x55c5847f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.041+0000 7fae54587740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847ed750 0x55c5847f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.041+0000 7fae54587740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5847f3ad0 con 0x55c5847ed750 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.041+0000 7fae54587740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5847f3c10 con 0x55c5847e4b70 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.041+0000 7fae54587740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c584854070 con 0x55c5847e4250 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.041+0000 7fae54587740 10 client.0 _ll_register_callbacks cb 0x7fae3c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.041+0000 7fae4a7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847ed750 0x55c5847f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.041+0000 7fae4affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 0x55c5847f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.041+0000 7fae4affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 0x55c5847f2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:46448/0 (socket says 172.21.15.18:46448) 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae4affd640 1 -- 172.21.15.18:0/2426165854 learned_addr learned my addr 172.21.15.18:0/2426165854 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/2426165854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4250 0x55c5847f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae4affd640 1 -- 172.21.15.18:0/2426165854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4250 msgr2=0x55c5847f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae4affd640 1 --2- 172.21.15.18:0/2426165854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4250 0x55c5847f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae4affd640 1 -- 172.21.15.18:0/2426165854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847ed750 msgr2=0x55c5847f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae4affd640 1 --2- 172.21.15.18:0/2426165854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847ed750 0x55c5847f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae4affd640 1 -- 172.21.15.18:0/2426165854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c58470e060 con 0x55c5847e4b70 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae4affd640 1 --2- 172.21.15.18:0/2426165854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 0x55c5847f2e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae3c01afd0 tx=0x7fae3c009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c060c8ac22ad5b8a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae2affd640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae2affd640 1 -- 172.21.15.18:0/2426165854 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c029070 con 0x55c5847e4b70 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae54587740 1 -- 172.21.15.18:0/2426165854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5847da0f0 con 0x55c5847e4b70 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae2affd640 1 -- 172.21.15.18:0/2426165854 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fae3c02ad10 con 0x55c5847e4b70 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae2affd640 1 -- 172.21.15.18:0/2426165854 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae3c009260 con 0x55c5847e4b70 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.042+0000 7fae54587740 10 client.29207 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.043+0000 7fae54587740 1 -- 172.21.15.18:0/2426165854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55c58474a6a0 con 0x55c5847e4b70 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.043+0000 7fae2affd640 1 -- 172.21.15.18:0/2426165854 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c5847da0f0 con 0x55c5847e4b70 2024-02-17T13:03:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.043+0000 7fae2affd640 1 -- 172.21.15.18:0/2426165854 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fae3c02a6e0 con 0x55c5847e4b70 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.043+0000 7fae54587740 10 client.29207 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.043+0000 7fae54587740 10 client.29207 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.043+0000 7fae54587740 20 client.29207 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.043+0000 7fae54587740 1 --2- 172.21.15.18:0/2426165854 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c584855780 0x55c584857ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.043+0000 7fae54587740 4 client.29207 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.043+0000 7fae54587740 1 -- 172.21.15.18:0/2426165854 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fae3c00fdc0 con 0x55c584855780 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.043+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/2426165854 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c584855780 0x55c584857ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.043+0000 7fae2affd640 10 client.29207 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae4a7fc640 1 --2- 172.21.15.18:0/2426165854 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c584855780 0x55c584857ba0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab2b8d26c3d43d65 server_cookie=50d6d27c4bcfe702 in_seq=0 out_seq=0 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae2affd640 1 -- 172.21.15.18:0/2426165854 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fae3c00fdc0 con 0x55c584855780 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae2affd640 10 client.29207 handle_command_reply: tid=0 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae54587740 1 client.29207 shutdown 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae54587740 1 -- 172.21.15.18:0/2426165854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 msgr2=0x55c5847f2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae54587740 1 --2- 172.21.15.18:0/2426165854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 0x55c5847f2e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae3c01afd0 tx=0x7fae3c009770 comp rx=0 tx=0).stop 2024-02-17T13:03:20.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae54587740 1 -- 172.21.15.18:0/2426165854 shutdown_connections 2024-02-17T13:03:20.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae54587740 1 --2- 172.21.15.18:0/2426165854 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x55c584855780 0x55c584857ba0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae54587740 1 --2- 172.21.15.18:0/2426165854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5847ed750 0x55c5847f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae54587740 1 --2- 172.21.15.18:0/2426165854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5847e4b70 0x55c5847f2e00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae54587740 1 --2- 172.21.15.18:0/2426165854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5847e4250 0x55c5847f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae2affd640 0 client.29207 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:03:20.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae54587740 1 -- 172.21.15.18:0/2426165854 >> 172.21.15.18:0/2426165854 conn(0x55c5847de980 msgr2=0x55c5847e8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae54587740 1 -- 172.21.15.18:0/2426165854 shutdown_connections 2024-02-17T13:03:20.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae54587740 1 -- 172.21.15.18:0/2426165854 wait complete. 2024-02-17T13:03:20.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.044+0000 7fae54587740 20 client.29207 trim_cache size 0 max 16384 2024-02-17T13:03:20.041 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:03:20.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.045+0000 7fae518d4640 1 -- 172.21.15.18:0/2942334932 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fae340a22b0 msgr2=0x7fae340a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.045+0000 7fae518d4640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fae340a22b0 0x7fae340a4770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fae440cbd40 tx=0x7fae4406cef0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.045+0000 7fae518d4640 1 -- 172.21.15.18:0/2942334932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae4c197600 msgr2=0x7fae4c197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.045+0000 7fae518d4640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae4c197600 0x7fae4c197ab0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fae4400eb10 tx=0x7fae4400c2d0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.045+0000 7fae518d4640 1 -- 172.21.15.18:0/2942334932 shutdown_connections 2024-02-17T13:03:20.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.045+0000 7fae518d4640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fae340a22b0 0x7fae340a4770 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.045+0000 7fae518d4640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae4c197600 0x7fae4c197ab0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.045+0000 7fae518d4640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae4c105330 0x7fae4c19d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.045+0000 7fae518d4640 1 --2- 172.21.15.18:0/2942334932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae4c100260 0x7fae4c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.045+0000 7fae518d4640 1 -- 172.21.15.18:0/2942334932 >> 172.21.15.18:0/2942334932 conn(0x7fae4c077180 msgr2=0x7fae4c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.045+0000 7fae518d4640 1 -- 172.21.15.18:0/2942334932 shutdown_connections 2024-02-17T13:03:20.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.045+0000 7fae518d4640 1 -- 172.21.15.18:0/2942334932 wait complete. 2024-02-17T13:03:20.047 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:03:20.047 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T13:03:20.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.167+0000 7fec00b0d640 1 -- 172.21.15.18:0/1486943394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc100260 msgr2=0x7febfc100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.167+0000 7fec00b0d640 1 --2- 172.21.15.18:0/1486943394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc100260 0x7febfc100640 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7febec009930 tx=0x7febec02bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:20.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.167+0000 7fec00b0d640 1 -- 172.21.15.18:0/1486943394 shutdown_connections 2024-02-17T13:03:20.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.167+0000 7fec00b0d640 1 --2- 172.21.15.18:0/1486943394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febfc105330 0x7febfc10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.167+0000 7fec00b0d640 1 --2- 172.21.15.18:0/1486943394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febfc100c10 0x7febfc104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.167+0000 7fec00b0d640 1 --2- 172.21.15.18:0/1486943394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc100260 0x7febfc100640 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.167+0000 7fec00b0d640 1 -- 172.21.15.18:0/1486943394 >> 172.21.15.18:0/1486943394 conn(0x7febfc077180 msgr2=0x7febfc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.168+0000 7fec00b0d640 1 -- 172.21.15.18:0/1486943394 shutdown_connections 2024-02-17T13:03:20.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.168+0000 7fec00b0d640 1 -- 172.21.15.18:0/1486943394 wait complete. 2024-02-17T13:03:20.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.168+0000 7fec00b0d640 1 Processor -- start 2024-02-17T13:03:20.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.168+0000 7fec00b0d640 1 -- start start 2024-02-17T13:03:20.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7fec00b0d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febfc100260 0x7febfc19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7fec00b0d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc105330 0x7febfc19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7fec00b0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febfc1975c0 0x7febfc197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7fec00b0d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc197fb0 con 0x7febfc105330 2024-02-17T13:03:20.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7fec00b0d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc198120 con 0x7febfc1975c0 2024-02-17T13:03:20.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7fec00b0d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc1982e0 con 0x7febfc100260 2024-02-17T13:03:20.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febf1d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc105330 0x7febfc19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febf1d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc105330 0x7febfc19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43858/0 (socket says 172.21.15.18:43858) 2024-02-17T13:03:20.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febf1d9b640 1 -- 172.21.15.18:0/1569105593 learned_addr learned my addr 172.21.15.18:0/1569105593 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febfad9d640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febfc1975c0 0x7febfc197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febfa59c640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febfc100260 0x7febfc19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febf1d9b640 1 -- 172.21.15.18:0/1569105593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febfc100260 msgr2=0x7febfc19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febf1d9b640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febfc100260 0x7febfc19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febf1d9b640 1 -- 172.21.15.18:0/1569105593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febfc1975c0 msgr2=0x7febfc197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febf1d9b640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febfc1975c0 0x7febfc197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febf1d9b640 1 -- 172.21.15.18:0/1569105593 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec009560 con 0x7febfc105330 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febfad9d640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febfc1975c0 0x7febfc197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febf1d9b640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc105330 0x7febfc19d950 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7febe400e9c0 tx=0x7febe400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febf3fff640 1 -- 172.21.15.18:0/1569105593 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4009700 con 0x7febfc105330 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7fec00b0d640 1 -- 172.21.15.18:0/1569105593 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febfc1a42d0 con 0x7febfc105330 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.169+0000 7febf3fff640 1 -- 172.21.15.18:0/1569105593 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febe400cd80 con 0x7febfc105330 2024-02-17T13:03:20.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.170+0000 7fec00b0d640 1 -- 172.21.15.18:0/1569105593 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febfc1a4830 con 0x7febfc105330 2024-02-17T13:03:20.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.170+0000 7febf3fff640 1 -- 172.21.15.18:0/1569105593 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4010430 con 0x7febfc105330 2024-02-17T13:03:20.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.172+0000 7febf3fff640 1 -- 172.21.15.18:0/1569105593 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7febe4005000 con 0x7febfc105330 2024-02-17T13:03:20.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.172+0000 7fec037bc740 1 Processor -- start 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.172+0000 7fec037bc740 1 -- start start 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.172+0000 7febf3fff640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7febe80a2230 0x7febe80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.172+0000 7fec037bc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 0x560a4fd81630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.172+0000 7fec037bc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81b70 0x560a4fd8a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.172+0000 7fec037bc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd8a750 0x560a4fd8cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.172+0000 7fec037bc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4fd3c960 con 0x560a4fd81b70 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.172+0000 7fec037bc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4fcf0310 con 0x560a4fd81250 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.172+0000 7fec037bc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4fced840 con 0x560a4fd8a750 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febfad9d640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7febe80a2230 0x7febe80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febf1d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81b70 0x560a4fd8a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febfa59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 0x560a4fd81630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febf1d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81b70 0x560a4fd8a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43860/0 (socket says 172.21.15.18:43860) 2024-02-17T13:03:20.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febfa59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 0x560a4fd81630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:46472/0 (socket says 172.21.15.18:46472) 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febf1d9b640 1 -- 172.21.15.18:0/2728247568 learned_addr learned my addr 172.21.15.18:0/2728247568 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febf3fff640 1 -- 172.21.15.18:0/1569105593 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7febe4014070 con 0x7febfc105330 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febfad9d640 1 --2- 172.21.15.18:0/2728247568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd8a750 0x560a4fd8cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febfa59c640 1 -- 172.21.15.18:0/2728247568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd8a750 msgr2=0x560a4fd8cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febfa59c640 1 --2- 172.21.15.18:0/2728247568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd8a750 0x560a4fd8cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febfa59c640 1 -- 172.21.15.18:0/2728247568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81b70 msgr2=0x560a4fd8a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febfa59c640 1 --2- 172.21.15.18:0/2728247568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81b70 0x560a4fd8a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febfa59c640 1 -- 172.21.15.18:0/2728247568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a4fcab060 con 0x560a4fd81250 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febf1d9b640 1 --2- 172.21.15.18:0/2728247568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81b70 0x560a4fd8a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febfad9d640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7febe80a2230 0x7febe80a46f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7febf4009fd0 tx=0x7febf4011f10 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febfa59c640 1 --2- 172.21.15.18:0/2728247568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 0x560a4fd81630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7febec009230 tx=0x7febec02c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5a8ae9b341f8783 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febf0d99640 1 -- 172.21.15.18:0/2728247568 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec004940 con 0x560a4fd81250 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febf0d99640 1 -- 172.21.15.18:0/2728247568 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febec004aa0 con 0x560a4fd81250 2024-02-17T13:03:20.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.173+0000 7febf0d99640 1 -- 172.21.15.18:0/2728247568 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec0050e0 con 0x560a4fd81250 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.174+0000 7fec037bc740 1 -- 172.21.15.18:0/2728247568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 msgr2=0x560a4fd81630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.174+0000 7fec037bc740 1 --2- 172.21.15.18:0/2728247568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 0x560a4fd81630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7febec009230 tx=0x7febec02c5d0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.174+0000 7fec037bc740 1 -- 172.21.15.18:0/2728247568 shutdown_connections 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.174+0000 7fec037bc740 1 --2- 172.21.15.18:0/2728247568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd8a750 0x560a4fd8cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.174+0000 7fec037bc740 1 --2- 172.21.15.18:0/2728247568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81b70 0x560a4fd8a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.174+0000 7fec037bc740 1 --2- 172.21.15.18:0/2728247568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 0x560a4fd81630 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.174+0000 7fec037bc740 1 -- 172.21.15.18:0/2728247568 >> 172.21.15.18:0/2728247568 conn(0x560a4fd7b980 msgr2=0x560a4fd7bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.174+0000 7fec037bc740 1 -- 172.21.15.18:0/2728247568 shutdown_connections 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.174+0000 7fec037bc740 1 -- 172.21.15.18:0/2728247568 wait complete. 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7fec037bc740 1 -- start start 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7fec037bc740 1 Processor -- start 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7fec037bc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 0x560a4fd87750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7fec037bc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81b70 0x560a4fd87c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7fec037bc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd8a750 0x560a4fd881d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7fec037bc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4fce9020 con 0x560a4fd81250 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7fec037bc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4f9bc430 con 0x560a4fd81b70 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7fec037bc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4f978e00 con 0x560a4fd8a750 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7fec037bc740 10 client.0 _ll_register_callbacks cb 0x560a4fc90cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febfa59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 0x560a4fd87750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febfa59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 0x560a4fd87750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43870/0 (socket says 172.21.15.18:43870) 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febf1d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81b70 0x560a4fd87c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febfa59c640 1 -- 172.21.15.18:0/2389771113 learned_addr learned my addr 172.21.15.18:0/2389771113 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febfad9d640 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd8a750 0x560a4fd881d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febfa59c640 1 -- 172.21.15.18:0/2389771113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd8a750 msgr2=0x560a4fd881d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febfa59c640 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd8a750 0x560a4fd881d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febfa59c640 1 -- 172.21.15.18:0/2389771113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81b70 msgr2=0x560a4fd87c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febfa59c640 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81b70 0x560a4fd87c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febfa59c640 1 -- 172.21.15.18:0/2389771113 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febfc10d710 con 0x560a4fd81250 2024-02-17T13:03:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febf1d9b640 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81b70 0x560a4fd87c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febfad9d640 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd8a750 0x560a4fd881d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febfa59c640 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 0x560a4fd87750 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7febec004a70 tx=0x7febec004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fbaec79931ec9a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.175+0000 7febe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7febe1ffb640 1 -- 172.21.15.18:0/2389771113 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec04e070 con 0x560a4fd81250 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7febe1ffb640 1 -- 172.21.15.18:0/2389771113 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7febec02c9f0 con 0x560a4fd81250 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7fec037bc740 1 -- 172.21.15.18:0/2389771113 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a4f9e87e0 con 0x560a4fd81250 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7febe1ffb640 1 -- 172.21.15.18:0/2389771113 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7febec02cbb0 con 0x560a4fd81250 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7fec037bc740 10 client.20361 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7fec037bc740 1 -- 172.21.15.18:0/2389771113 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x560a4fce76a0 con 0x560a4fd81250 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7febe1ffb640 1 -- 172.21.15.18:0/2389771113 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560a4f9e87e0 con 0x560a4fd81250 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7febe1ffb640 1 -- 172.21.15.18:0/2389771113 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7febec051020 con 0x560a4fd81250 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7fec037bc740 10 client.20361 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7fec037bc740 10 client.20361 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7fec037bc740 20 client.20361 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7fec037bc740 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560a4fd9a770 0x560a4fd9cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7fec037bc740 4 client.20361 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7fec037bc740 1 -- 172.21.15.18:0/2389771113 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a4fa25720 con 0x560a4fd9a770 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.176+0000 7febf1d9b640 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560a4fd9a770 0x560a4fd9cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.177+0000 7febe1ffb640 10 client.20361 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.177+0000 7febf1d9b640 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560a4fd9a770 0x560a4fd9cb90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56f532d19be344da server_cookie=9c7d96658045d5e7 in_seq=0 out_seq=0 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.177+0000 7febe1ffb640 1 -- 172.21.15.18:0/2389771113 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a4fa25720 con 0x560a4fd9a770 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.177+0000 7febe1ffb640 10 client.20361 handle_command_reply: tid=0 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.177+0000 7fec037bc740 1 client.20361 shutdown 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.177+0000 7fec037bc740 1 -- 172.21.15.18:0/2389771113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 msgr2=0x560a4fd87750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.177+0000 7fec037bc740 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 0x560a4fd87750 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7febec004a70 tx=0x7febec004c10 comp rx=0 tx=0).stop 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.178+0000 7fec037bc740 1 -- 172.21.15.18:0/2389771113 shutdown_connections 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.178+0000 7fec037bc740 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560a4fd9a770 0x560a4fd9cb90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.178+0000 7fec037bc740 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd8a750 0x560a4fd881d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.178+0000 7fec037bc740 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81b70 0x560a4fd87c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.178+0000 7febe1ffb640 0 client.20361 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.178+0000 7fec037bc740 1 --2- 172.21.15.18:0/2389771113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 0x560a4fd87750 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.178+0000 7fec037bc740 1 -- 172.21.15.18:0/2389771113 >> 172.21.15.18:0/2389771113 conn(0x560a4fd7b980 msgr2=0x560a4fd7d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.178+0000 7fec037bc740 1 -- 172.21.15.18:0/2389771113 shutdown_connections 2024-02-17T13:03:20.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.178+0000 7fec037bc740 1 -- 172.21.15.18:0/2389771113 wait complete. 2024-02-17T13:03:20.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.178+0000 7fec037bc740 20 client.20361 trim_cache size 0 max 16384 2024-02-17T13:03:20.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7fec037bc740 1 Processor -- start 2024-02-17T13:03:20.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7fec037bc740 1 -- start start 2024-02-17T13:03:20.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7fec037bc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 0x560a4fd7aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7fec037bc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd81b70 0x560a4fd915a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7fec037bc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd8a750 0x560a4fd91ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7fec037bc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4fc6e130 con 0x560a4fd81250 2024-02-17T13:03:20.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7fec037bc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4f9e87e0 con 0x560a4fd8a750 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7fec037bc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4fd7c980 con 0x560a4fd81b70 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7febfa59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 0x560a4fd7aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7febfa59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 0x560a4fd7aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43876/0 (socket says 172.21.15.18:43876) 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7febfa59c640 1 -- 172.21.15.18:0/2742586213 learned_addr learned my addr 172.21.15.18:0/2742586213 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7febfad9d640 1 --2- 172.21.15.18:0/2742586213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd8a750 0x560a4fd91ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.191+0000 7febf1d9b640 1 --2- 172.21.15.18:0/2742586213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd81b70 0x560a4fd915a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.192+0000 7febfa59c640 1 -- 172.21.15.18:0/2742586213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd81b70 msgr2=0x560a4fd915a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.192+0000 7febfa59c640 1 --2- 172.21.15.18:0/2742586213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd81b70 0x560a4fd915a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.192+0000 7febfa59c640 1 -- 172.21.15.18:0/2742586213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd8a750 msgr2=0x560a4fd91ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.192+0000 7febfa59c640 1 --2- 172.21.15.18:0/2742586213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd8a750 0x560a4fd91ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.192+0000 7febfa59c640 1 -- 172.21.15.18:0/2742586213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a4fc83c20 con 0x560a4fd81250 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.192+0000 7febfa59c640 1 --2- 172.21.15.18:0/2742586213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 0x560a4fd7aa00 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7febec003af0 tx=0x7febec02c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f40d7b737320cd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.192+0000 7febe3fff640 1 -- 172.21.15.18:0/2742586213 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec04e020 con 0x560a4fd81250 2024-02-17T13:03:20.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.192+0000 7febe3fff640 1 -- 172.21.15.18:0/2742586213 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febec02c9f0 con 0x560a4fd81250 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.192+0000 7febe3fff640 1 -- 172.21.15.18:0/2742586213 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec0052d0 con 0x560a4fd81250 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 -- 172.21.15.18:0/2742586213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 msgr2=0x560a4fd7aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 --2- 172.21.15.18:0/2742586213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 0x560a4fd7aa00 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7febec003af0 tx=0x7febec02c2b0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 -- 172.21.15.18:0/2742586213 shutdown_connections 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 --2- 172.21.15.18:0/2742586213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd8a750 0x560a4fd91ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 --2- 172.21.15.18:0/2742586213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd81b70 0x560a4fd915a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 --2- 172.21.15.18:0/2742586213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd81250 0x560a4fd7aa00 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 -- 172.21.15.18:0/2742586213 >> 172.21.15.18:0/2742586213 conn(0x560a4fd7b980 msgr2=0x560a4fd7a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 -- 172.21.15.18:0/2742586213 shutdown_connections 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 -- 172.21.15.18:0/2742586213 wait complete. 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 -- start start 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 Processor -- start 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 0x560a4fd901f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd81b70 0x560a4fd90730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd8a750 0x560a4fd90c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4fdf0db0 con 0x560a4fd8a750 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4fdf0f20 con 0x560a4fd81250 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4fdf10e0 con 0x560a4fd81b70 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7fec037bc740 10 client.0 _ll_register_callbacks cb 0x7febe4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.193+0000 7febfad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd8a750 0x560a4fd90c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febfad9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd8a750 0x560a4fd90c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43892/0 (socket says 172.21.15.18:43892) 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febfad9d640 1 -- 172.21.15.18:0/2360298723 learned_addr learned my addr 172.21.15.18:0/2360298723 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febfa59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 0x560a4fd901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febf1d9b640 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd81b70 0x560a4fd90730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febfad9d640 1 -- 172.21.15.18:0/2360298723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd81b70 msgr2=0x560a4fd90730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febfad9d640 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd81b70 0x560a4fd90730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febfad9d640 1 -- 172.21.15.18:0/2360298723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 msgr2=0x560a4fd901f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febfad9d640 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 0x560a4fd901f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febfad9d640 1 -- 172.21.15.18:0/2360298723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a4fce76a0 con 0x560a4fd8a750 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febfa59c640 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 0x560a4fd901f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febfad9d640 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd8a750 0x560a4fd90c70 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7febf4005fd0 tx=0x7febf4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a855798e67442f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febe1ffb640 1 -- 172.21.15.18:0/2360298723 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4026070 con 0x560a4fd8a750 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febe1ffb640 1 -- 172.21.15.18:0/2360298723 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7febf40206d0 con 0x560a4fd8a750 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7fec037bc740 1 -- 172.21.15.18:0/2360298723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febec004aa0 con 0x560a4fd8a750 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febe1ffb640 1 -- 172.21.15.18:0/2360298723 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7febf4020890 con 0x560a4fd8a750 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7fec037bc740 10 client.20367 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7fec037bc740 1 -- 172.21.15.18:0/2360298723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x560a4fca98c0 con 0x560a4fd8a750 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.194+0000 7febe1ffb640 1 -- 172.21.15.18:0/2360298723 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febf4037020 con 0x560a4fd8a750 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7febe1ffb640 1 -- 172.21.15.18:0/2360298723 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7febf402ac00 con 0x560a4fd8a750 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7fec037bc740 10 client.20367 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7fec037bc740 10 client.20367 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:03:20.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7fec037bc740 20 client.20367 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7fec037bc740 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560a4fdf2970 0x560a4fdf4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7fec037bc740 4 client.20367 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7fec037bc740 1 -- 172.21.15.18:0/2360298723 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560a4fa74890 con 0x560a4fdf2970 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7febfa59c640 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560a4fdf2970 0x560a4fdf4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7febfa59c640 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560a4fdf2970 0x560a4fdf4d90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a585e6c1ddcc6f68 server_cookie=e2d0caa8e81288ff in_seq=0 out_seq=0 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7febe1ffb640 10 client.20367 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7febe1ffb640 1 -- 172.21.15.18:0/2360298723 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560a4fa74890 con 0x560a4fdf2970 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7febe1ffb640 10 client.20367 handle_command_reply: tid=0 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7fec037bc740 1 client.20367 shutdown 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7fec037bc740 1 -- 172.21.15.18:0/2360298723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd8a750 msgr2=0x560a4fd90c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7fec037bc740 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd8a750 0x560a4fd90c70 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7febf4005fd0 tx=0x7febf4004ab0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7fec037bc740 1 -- 172.21.15.18:0/2360298723 shutdown_connections 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.195+0000 7fec037bc740 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x560a4fdf2970 0x560a4fdf4d90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec037bc740 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4fd8a750 0x560a4fd90c70 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec037bc740 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a4fd81b70 0x560a4fd90730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec037bc740 1 --2- 172.21.15.18:0/2360298723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a4fd81250 0x560a4fd901f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7febe1ffb640 0 client.20367 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:03:20.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec037bc740 1 -- 172.21.15.18:0/2360298723 >> 172.21.15.18:0/2360298723 conn(0x560a4fd7b980 msgr2=0x560a4fd7ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec037bc740 1 -- 172.21.15.18:0/2360298723 shutdown_connections 2024-02-17T13:03:20.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec037bc740 1 -- 172.21.15.18:0/2360298723 wait complete. 2024-02-17T13:03:20.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec037bc740 20 client.20367 trim_cache size 0 max 16384 2024-02-17T13:03:20.192 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:03:20.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec00b0d640 1 -- 172.21.15.18:0/1569105593 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7febe80a2230 msgr2=0x7febe80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec00b0d640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7febe80a2230 0x7febe80a46f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7febf4009fd0 tx=0x7febf4011f10 comp rx=0 tx=0).stop 2024-02-17T13:03:20.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec00b0d640 1 -- 172.21.15.18:0/1569105593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc105330 msgr2=0x7febfc19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec00b0d640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc105330 0x7febfc19d950 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7febe400e9c0 tx=0x7febe400ee80 comp rx=0 tx=0).stop 2024-02-17T13:03:20.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec00b0d640 1 -- 172.21.15.18:0/1569105593 shutdown_connections 2024-02-17T13:03:20.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec00b0d640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7febe80a2230 0x7febe80a46f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec00b0d640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7febfc1975c0 0x7febfc197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec00b0d640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc105330 0x7febfc19d950 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec00b0d640 1 --2- 172.21.15.18:0/1569105593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febfc100260 0x7febfc19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec00b0d640 1 -- 172.21.15.18:0/1569105593 >> 172.21.15.18:0/1569105593 conn(0x7febfc077180 msgr2=0x7febfc0686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec00b0d640 1 -- 172.21.15.18:0/1569105593 shutdown_connections 2024-02-17T13:03:20.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.196+0000 7fec00b0d640 1 -- 172.21.15.18:0/1569105593 wait complete. 2024-02-17T13:03:20.197 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:03:20.197 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T13:03:20.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.320+0000 7f3d0182c640 1 -- 172.21.15.18:0/1508517665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cfc100360 msgr2=0x7f3cfc1007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.320+0000 7f3d0182c640 1 --2- 172.21.15.18:0/1508517665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cfc100360 0x7f3cfc1007e0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009900 tx=0x7f3cf002bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:20.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.320+0000 7f3d0182c640 1 -- 172.21.15.18:0/1508517665 shutdown_connections 2024-02-17T13:03:20.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.320+0000 7f3d0182c640 1 --2- 172.21.15.18:0/1508517665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cfc100d20 0x7f3cfc10ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.320+0000 7f3d0182c640 1 --2- 172.21.15.18:0/1508517665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cfc100360 0x7f3cfc1007e0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.320+0000 7f3d0182c640 1 --2- 172.21.15.18:0/1508517665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc106240 0x7f3cfc106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.320+0000 7f3d0182c640 1 -- 172.21.15.18:0/1508517665 >> 172.21.15.18:0/1508517665 conn(0x7f3cfc077060 msgr2=0x7f3cfc077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.320+0000 7f3d0182c640 1 -- 172.21.15.18:0/1508517665 shutdown_connections 2024-02-17T13:03:20.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.320+0000 7f3d0182c640 1 -- 172.21.15.18:0/1508517665 wait complete. 2024-02-17T13:03:20.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3d0182c640 1 Processor -- start 2024-02-17T13:03:20.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3d0182c640 1 -- start start 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3d0182c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cfc100360 0x7f3cfc101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3d0182c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cfc100d20 0x7f3cfc1022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3d0182c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc106240 0x7f3cfc1027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3d0182c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc1a5b00 con 0x7f3cfc100360 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3d0182c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc1a5c70 con 0x7f3cfc106240 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3d0182c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc1a5e30 con 0x7f3cfc100d20 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3cfaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cfc100360 0x7f3cfc101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3cfaffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cfc100360 0x7f3cfc101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43916/0 (socket says 172.21.15.18:43916) 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3cfaffd640 1 -- 172.21.15.18:0/1692246234 learned_addr learned my addr 172.21.15.18:0/1692246234 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3cfa7fc640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cfc100d20 0x7f3cfc1022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.321+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc106240 0x7f3cfc1027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.322+0000 7f3cfaffd640 1 -- 172.21.15.18:0/1692246234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cfc100d20 msgr2=0x7f3cfc1022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.322+0000 7f3cfaffd640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cfc100d20 0x7f3cfc1022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.322+0000 7f3cfaffd640 1 -- 172.21.15.18:0/1692246234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc106240 msgr2=0x7f3cfc1027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.322+0000 7f3cfaffd640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc106240 0x7f3cfc1027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.322+0000 7f3cfaffd640 1 -- 172.21.15.18:0/1692246234 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf0009560 con 0x7f3cfc100360 2024-02-17T13:03:20.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.322+0000 7f3cfa7fc640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cfc100d20 0x7f3cfc1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:20.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.322+0000 7f3cfaffd640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cfc100360 0x7f3cfc101d60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00ebc0 tx=0x7f3cec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.322+0000 7f3d0082a640 1 -- 172.21.15.18:0/1692246234 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec0099b0 con 0x7f3cfc100360 2024-02-17T13:03:20.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.322+0000 7f3d0082a640 1 -- 172.21.15.18:0/1692246234 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cec00eec0 con 0x7f3cfc100360 2024-02-17T13:03:20.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.322+0000 7f3d0182c640 1 -- 172.21.15.18:0/1692246234 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc1a6110 con 0x7f3cfc100360 2024-02-17T13:03:20.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.322+0000 7f3d0082a640 1 -- 172.21.15.18:0/1692246234 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec018b60 con 0x7f3cfc100360 2024-02-17T13:03:20.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.322+0000 7f3d0182c640 1 -- 172.21.15.18:0/1692246234 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc1a6640 con 0x7f3cfc100360 2024-02-17T13:03:20.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.324+0000 7f3d0082a640 1 -- 172.21.15.18:0/1692246234 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3cec018cc0 con 0x7f3cfc100360 2024-02-17T13:03:20.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.324+0000 7f3d043fd740 1 Processor -- start 2024-02-17T13:03:20.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.324+0000 7f3d043fd740 1 -- start start 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3d043fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250de2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3d043fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 0x565250deb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3d0082a640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ce40a22e0 0x7f3ce40a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3d043fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250deb750 0x565250dedb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3d043fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x565250d9d960 con 0x565250de2b70 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3d043fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565250d51310 con 0x565250deb750 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3d043fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565250d4e840 con 0x565250de2250 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfa7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 0x565250deb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfaffd640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ce40a22e0 0x7f3ce40a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfa7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 0x565250deb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43932/0 (socket says 172.21.15.18:43932) 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfa7fc640 1 -- 172.21.15.18:0/3154313216 learned_addr learned my addr 172.21.15.18:0/3154313216 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/3154313216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250de2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/3154313216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250deb750 0x565250dedb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3d0082a640 1 -- 172.21.15.18:0/1692246234 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3cec014070 con 0x7f3cfc100360 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfa7fc640 1 -- 172.21.15.18:0/3154313216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 msgr2=0x565250de2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfa7fc640 1 --2- 172.21.15.18:0/3154313216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250de2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfa7fc640 1 -- 172.21.15.18:0/3154313216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250deb750 msgr2=0x565250dedb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfa7fc640 1 --2- 172.21.15.18:0/3154313216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250deb750 0x565250dedb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfa7fc640 1 -- 172.21.15.18:0/3154313216 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565250d0c060 con 0x565250de2b70 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/3154313216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250de2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfa7fc640 1 --2- 172.21.15.18:0/3154313216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 0x565250deb210 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009900 tx=0x7f3cf002c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60469023f061ea7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.325+0000 7f3cfaffd640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ce40a22e0 0x7f3ce40a47a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3cec09aff0 tx=0x7f3cec06e0d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.326+0000 7f3ce1ffb640 1 -- 172.21.15.18:0/3154313216 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf002c7e0 con 0x565250de2b70 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.326+0000 7f3ce1ffb640 1 -- 172.21.15.18:0/3154313216 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cf0004510 con 0x565250de2b70 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.326+0000 7f3ce1ffb640 1 -- 172.21.15.18:0/3154313216 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0005410 con 0x565250de2b70 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.326+0000 7f3d043fd740 1 -- 172.21.15.18:0/3154313216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 msgr2=0x565250deb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.326+0000 7f3d043fd740 1 --2- 172.21.15.18:0/3154313216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 0x565250deb210 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009900 tx=0x7f3cf002c680 comp rx=0 tx=0).stop 2024-02-17T13:03:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.326+0000 7f3d043fd740 1 -- 172.21.15.18:0/3154313216 shutdown_connections 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.326+0000 7f3d043fd740 1 --2- 172.21.15.18:0/3154313216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250deb750 0x565250dedb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.326+0000 7f3d043fd740 1 --2- 172.21.15.18:0/3154313216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 0x565250deb210 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.326+0000 7f3d043fd740 1 --2- 172.21.15.18:0/3154313216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250de2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.326+0000 7f3d043fd740 1 -- 172.21.15.18:0/3154313216 >> 172.21.15.18:0/3154313216 conn(0x565250ddc980 msgr2=0x565250ddcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.326+0000 7f3d043fd740 1 -- 172.21.15.18:0/3154313216 shutdown_connections 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.327+0000 7f3d043fd740 1 -- 172.21.15.18:0/3154313216 wait complete. 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.327+0000 7f3d043fd740 1 -- start start 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.327+0000 7f3d043fd740 1 Processor -- start 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.327+0000 7f3d043fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250de87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.327+0000 7f3d043fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 0x565250de8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.327+0000 7f3d043fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250deb750 0x565250de9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.327+0000 7f3d043fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x565250d4a020 con 0x565250de2b70 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.327+0000 7f3d043fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565250a1d430 con 0x565250deb750 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.327+0000 7f3d043fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652509d9e00 con 0x565250de2250 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.327+0000 7f3d043fd740 10 client.0 _ll_register_callbacks cb 0x565250cf1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.327+0000 7f3cfa7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 0x565250de8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.327+0000 7f3cfa7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 0x565250de8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43948/0 (socket says 172.21.15.18:43948) 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cfa7fc640 1 -- 172.21.15.18:0/3463709614 learned_addr learned my addr 172.21.15.18:0/3463709614 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250deb750 0x565250de9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250de87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cfa7fc640 1 -- 172.21.15.18:0/3463709614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 msgr2=0x565250de87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cfa7fc640 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250de87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cfa7fc640 1 -- 172.21.15.18:0/3463709614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250deb750 msgr2=0x565250de9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cfa7fc640 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250deb750 0x565250de9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cfa7fc640 1 -- 172.21.15.18:0/3463709614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc10d7a0 con 0x565250de2b70 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250deb750 0x565250de9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cfa7fc640 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 0x565250de8cf0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0005470 tx=0x7f3cf0004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6cb93058320d77 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cdaffd640 1 -- 172.21.15.18:0/3463709614 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf002cb30 con 0x565250de2b70 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cdaffd640 1 -- 172.21.15.18:0/3463709614 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3cf0034850 con 0x565250de2b70 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3d043fd740 1 -- 172.21.15.18:0/3463709614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565250a497e0 con 0x565250de2b70 2024-02-17T13:03:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3cdaffd640 1 -- 172.21.15.18:0/3463709614 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cf0005650 con 0x565250de2b70 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3d043fd740 10 client.20379 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.328+0000 7f3d043fd740 1 -- 172.21.15.18:0/3463709614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x565250d486a0 con 0x565250de2b70 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.329+0000 7f3cdaffd640 1 -- 172.21.15.18:0/3463709614 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x565250a497e0 con 0x565250de2b70 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.329+0000 7f3cdaffd640 1 -- 172.21.15.18:0/3463709614 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f3cf003c050 con 0x565250de2b70 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.329+0000 7f3d043fd740 10 client.20379 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.329+0000 7f3d043fd740 10 client.20379 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.329+0000 7f3d043fd740 20 client.20379 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.329+0000 7f3d043fd740 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x565250dfb7f0 0x565250dfdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.329+0000 7f3d043fd740 4 client.20379 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.329+0000 7f3d043fd740 1 -- 172.21.15.18:0/3463709614 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565250a86720 con 0x565250dfb7f0 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.329+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x565250dfb7f0 0x565250dfdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.329+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x565250dfb7f0 0x565250dfdc10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a04da68192a9178e server_cookie=982530bd02b3353d in_seq=0 out_seq=0 2024-02-17T13:03:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.329+0000 7f3cdaffd640 10 client.20379 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:03:20.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3cdaffd640 1 -- 172.21.15.18:0/3463709614 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565250a86720 con 0x565250dfb7f0 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3cdaffd640 10 client.20379 handle_command_reply: tid=0 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3d043fd740 1 client.20379 shutdown 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3d043fd740 1 -- 172.21.15.18:0/3463709614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 msgr2=0x565250de8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3d043fd740 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 0x565250de8cf0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0005470 tx=0x7f3cf0004f70 comp rx=0 tx=0).stop 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3d043fd740 1 -- 172.21.15.18:0/3463709614 shutdown_connections 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3d043fd740 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x565250dfb7f0 0x565250dfdc10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3d043fd740 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250deb750 0x565250de9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3d043fd740 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2b70 0x565250de8cf0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3cdaffd640 0 client.20379 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3d043fd740 1 --2- 172.21.15.18:0/3463709614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250de87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3d043fd740 1 -- 172.21.15.18:0/3463709614 >> 172.21.15.18:0/3463709614 conn(0x565250ddc980 msgr2=0x565250dde6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3d043fd740 1 -- 172.21.15.18:0/3463709614 shutdown_connections 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.331+0000 7f3d043fd740 1 -- 172.21.15.18:0/3463709614 wait complete. 2024-02-17T13:03:20.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.332+0000 7f3d043fd740 20 client.20379 trim_cache size 0 max 16384 2024-02-17T13:03:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.345+0000 7f3d043fd740 1 Processor -- start 2024-02-17T13:03:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.345+0000 7f3d043fd740 1 -- start start 2024-02-17T13:03:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3d043fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250deaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3d043fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250de2b70 0x565250de9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3d043fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250deb750 0x565250de5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3d043fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x565250ccf130 con 0x565250deb750 2024-02-17T13:03:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3d043fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565250ddd9b0 con 0x565250de2b70 2024-02-17T13:03:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3d043fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565250dd85c0 con 0x565250de2250 2024-02-17T13:03:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3cfb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250deb750 0x565250de5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3cfb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250deb750 0x565250de5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43960/0 (socket says 172.21.15.18:43960) 2024-02-17T13:03:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3cfb7fe640 1 -- 172.21.15.18:0/552423957 learned_addr learned my addr 172.21.15.18:0/552423957 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3cfa7fc640 1 --2- 172.21.15.18:0/552423957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250de2b70 0x565250de9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/552423957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250deaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3cfb7fe640 1 -- 172.21.15.18:0/552423957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 msgr2=0x565250deaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/552423957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250deaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3cfb7fe640 1 -- 172.21.15.18:0/552423957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250de2b70 msgr2=0x565250de9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/552423957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250de2b70 0x565250de9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3cfb7fe640 1 -- 172.21.15.18:0/552423957 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565250ce4c20 con 0x565250deb750 2024-02-17T13:03:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.346+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/552423957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250deb750 0x565250de5a10 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4020580 tx=0x7f3cf4009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b25959f747e954 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.347+0000 7f3ce17fa640 1 -- 172.21.15.18:0/552423957 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4016040 con 0x565250deb750 2024-02-17T13:03:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.347+0000 7f3ce17fa640 1 -- 172.21.15.18:0/552423957 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cf4026e90 con 0x565250deb750 2024-02-17T13:03:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.347+0000 7f3ce17fa640 1 -- 172.21.15.18:0/552423957 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4005020 con 0x565250deb750 2024-02-17T13:03:20.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 -- 172.21.15.18:0/552423957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250deb750 msgr2=0x565250de5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 --2- 172.21.15.18:0/552423957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250deb750 0x565250de5a10 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4020580 tx=0x7f3cf4009ac0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 -- 172.21.15.18:0/552423957 shutdown_connections 2024-02-17T13:03:20.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 --2- 172.21.15.18:0/552423957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250deb750 0x565250de5a10 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 --2- 172.21.15.18:0/552423957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250de2b70 0x565250de9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 --2- 172.21.15.18:0/552423957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250de2250 0x565250deaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 -- 172.21.15.18:0/552423957 >> 172.21.15.18:0/552423957 conn(0x565250ddc980 msgr2=0x565250e02570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 -- 172.21.15.18:0/552423957 shutdown_connections 2024-02-17T13:03:20.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 -- 172.21.15.18:0/552423957 wait complete. 2024-02-17T13:03:20.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 -- start start 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 Processor -- start 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2250 0x565250df2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250de2b70 0x565250df0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.348+0000 7f3d043fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250deb750 0x565250df1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3d043fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x565250df1b10 con 0x565250de2250 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3d043fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565250df1c80 con 0x565250de2b70 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3d043fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x565250e51d80 con 0x565250deb750 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3d043fd740 10 client.0 _ll_register_callbacks cb 0x7f3cf4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cfb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2250 0x565250df2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cfb7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2250 0x565250df2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43970/0 (socket says 172.21.15.18:43970) 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cfb7fe640 1 -- 172.21.15.18:0/513707241 learned_addr learned my addr 172.21.15.18:0/513707241 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cfa7fc640 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250de2b70 0x565250df0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250deb750 0x565250df1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cfb7fe640 1 -- 172.21.15.18:0/513707241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250deb750 msgr2=0x565250df1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250deb750 0x565250df1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cfb7fe640 1 -- 172.21.15.18:0/513707241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250de2b70 msgr2=0x565250df0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250de2b70 0x565250df0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cfb7fe640 1 -- 172.21.15.18:0/513707241 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf4004040 con 0x565250de2250 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cfb7fe640 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2250 0x565250df2a20 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009f10 tx=0x7f3cf4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93bd34f486cccf43 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cdaffd640 1 -- 172.21.15.18:0/513707241 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4016070 con 0x565250de2250 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cdaffd640 1 -- 172.21.15.18:0/513707241 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3cf4011aa0 con 0x565250de2250 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3d043fd740 1 -- 172.21.15.18:0/513707241 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cf0004510 con 0x565250de2250 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3cdaffd640 1 -- 172.21.15.18:0/513707241 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cf40035f0 con 0x565250de2250 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3d043fd740 10 client.20385 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.349+0000 7f3d043fd740 1 -- 172.21.15.18:0/513707241 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x565250d0a8c0 con 0x565250de2250 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.350+0000 7f3cdaffd640 1 -- 172.21.15.18:0/513707241 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cf0004510 con 0x565250de2250 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.350+0000 7f3cdaffd640 1 -- 172.21.15.18:0/513707241 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f3cf40209b0 con 0x565250de2250 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.350+0000 7f3d043fd740 10 client.20385 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.350+0000 7f3d043fd740 10 client.20385 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:03:20.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.350+0000 7f3d043fd740 20 client.20385 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.350+0000 7f3d043fd740 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x565250e53600 0x565250e55a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.350+0000 7f3d043fd740 4 client.20385 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.350+0000 7f3d043fd740 1 -- 172.21.15.18:0/513707241 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565250ad5890 con 0x565250e53600 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.350+0000 7f3cfa7fc640 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x565250e53600 0x565250e55a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.350+0000 7f3cdaffd640 10 client.20385 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.351+0000 7f3cfa7fc640 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x565250e53600 0x565250e55a20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c745780e268ff30 server_cookie=30d9605bdb25a400 in_seq=0 out_seq=0 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3cdaffd640 1 -- 172.21.15.18:0/513707241 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565250ad5890 con 0x565250e53600 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3cdaffd640 10 client.20385 handle_command_reply: tid=0 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3d043fd740 1 client.20385 shutdown 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3d043fd740 1 -- 172.21.15.18:0/513707241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2250 msgr2=0x565250df2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3d043fd740 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2250 0x565250df2a20 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009f10 tx=0x7f3cf4004ac0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3d043fd740 1 -- 172.21.15.18:0/513707241 shutdown_connections 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3d043fd740 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x565250e53600 0x565250e55a20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3d043fd740 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565250deb750 0x565250df1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3d043fd740 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565250de2b70 0x565250df0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3cdaffd640 0 client.20385 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:03:20.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3d043fd740 1 --2- 172.21.15.18:0/513707241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565250de2250 0x565250df2a20 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3d043fd740 1 -- 172.21.15.18:0/513707241 >> 172.21.15.18:0/513707241 conn(0x565250ddc980 msgr2=0x565250ddb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3d043fd740 1 -- 172.21.15.18:0/513707241 shutdown_connections 2024-02-17T13:03:20.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3d043fd740 1 -- 172.21.15.18:0/513707241 wait complete. 2024-02-17T13:03:20.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.352+0000 7f3d043fd740 20 client.20385 trim_cache size 0 max 16384 2024-02-17T13:03:20.346 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:03:20.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.353+0000 7f3d0182c640 1 -- 172.21.15.18:0/1692246234 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ce40a22e0 msgr2=0x7f3ce40a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.353+0000 7f3d0182c640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ce40a22e0 0x7f3ce40a47a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3cec09aff0 tx=0x7f3cec06e0d0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.353+0000 7f3d0182c640 1 -- 172.21.15.18:0/1692246234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cfc100360 msgr2=0x7f3cfc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.353+0000 7f3d0182c640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cfc100360 0x7f3cfc101d60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00ebc0 tx=0x7f3cec00c6a0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.353+0000 7f3d0182c640 1 -- 172.21.15.18:0/1692246234 shutdown_connections 2024-02-17T13:03:20.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.353+0000 7f3d0182c640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ce40a22e0 0x7f3ce40a47a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.353+0000 7f3d0182c640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc106240 0x7f3cfc1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.353+0000 7f3d0182c640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3cfc100d20 0x7f3cfc1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.353+0000 7f3d0182c640 1 --2- 172.21.15.18:0/1692246234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cfc100360 0x7f3cfc101d60 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.353+0000 7f3d0182c640 1 -- 172.21.15.18:0/1692246234 >> 172.21.15.18:0/1692246234 conn(0x7f3cfc077060 msgr2=0x7f3cfc0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.353+0000 7f3d0182c640 1 -- 172.21.15.18:0/1692246234 shutdown_connections 2024-02-17T13:03:20.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.353+0000 7f3d0182c640 1 -- 172.21.15.18:0/1692246234 wait complete. 2024-02-17T13:03:20.353 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:03:20.354 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T13:03:20.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.474+0000 7f7b39040640 1 -- 172.21.15.18:0/3404699820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b340685f0 msgr2=0x7f7b340689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.474+0000 7f7b39040640 1 --2- 172.21.15.18:0/3404699820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b340685f0 0x7f7b340689d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009860 tx=0x7f7b2402bb70 comp rx=0 tx=0).stop 2024-02-17T13:03:20.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.474+0000 7f7b39040640 1 -- 172.21.15.18:0/3404699820 shutdown_connections 2024-02-17T13:03:20.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.474+0000 7f7b39040640 1 --2- 172.21.15.18:0/3404699820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b341064c0 0x7f7b3410e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.474+0000 7f7b39040640 1 --2- 172.21.15.18:0/3404699820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b34068fa0 0x7f7b34105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.474+0000 7f7b39040640 1 --2- 172.21.15.18:0/3404699820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b340685f0 0x7f7b340689d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.474+0000 7f7b39040640 1 -- 172.21.15.18:0/3404699820 >> 172.21.15.18:0/3404699820 conn(0x7f7b34077180 msgr2=0x7f7b34077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.474+0000 7f7b39040640 1 -- 172.21.15.18:0/3404699820 shutdown_connections 2024-02-17T13:03:20.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.474+0000 7f7b39040640 1 -- 172.21.15.18:0/3404699820 wait complete. 2024-02-17T13:03:20.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b39040640 1 Processor -- start 2024-02-17T13:03:20.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b39040640 1 -- start start 2024-02-17T13:03:20.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b39040640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b340685f0 0x7f7b3419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b39040640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b341064c0 0x7f7b3419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b39040640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b341975c0 0x7f7b34197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b39040640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34197fb0 con 0x7f7b340685f0 2024-02-17T13:03:20.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b39040640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34198120 con 0x7f7b341064c0 2024-02-17T13:03:20.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b39040640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b341982e0 con 0x7f7b341975c0 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b32d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b340685f0 0x7f7b3419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b32d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b340685f0 0x7f7b3419d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43986/0 (socket says 172.21.15.18:43986) 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b32d9d640 1 -- 172.21.15.18:0/288604196 learned_addr learned my addr 172.21.15.18:0/288604196 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b341064c0 0x7f7b3419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.475+0000 7f7b3359e640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b341975c0 0x7f7b34197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.476+0000 7f7b32d9d640 1 -- 172.21.15.18:0/288604196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b341975c0 msgr2=0x7f7b34197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.476+0000 7f7b32d9d640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b341975c0 0x7f7b34197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.476+0000 7f7b32d9d640 1 -- 172.21.15.18:0/288604196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b341064c0 msgr2=0x7f7b3419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.476+0000 7f7b32d9d640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b341064c0 0x7f7b3419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.476+0000 7f7b32d9d640 1 -- 172.21.15.18:0/288604196 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c009900 con 0x7f7b340685f0 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.476+0000 7f7b32d9d640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b340685f0 0x7f7b3419d410 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f7b24037040 tx=0x7f7b240045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.476+0000 7f7b30d99640 1 -- 172.21.15.18:0/288604196 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b240349d0 con 0x7f7b340685f0 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.476+0000 7f7b39040640 1 -- 172.21.15.18:0/288604196 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b24009490 con 0x7f7b340685f0 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.476+0000 7f7b30d99640 1 -- 172.21.15.18:0/288604196 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b2402c9f0 con 0x7f7b340685f0 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.476+0000 7f7b30d99640 1 -- 172.21.15.18:0/288604196 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24032600 con 0x7f7b340685f0 2024-02-17T13:03:20.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.476+0000 7f7b39040640 1 -- 172.21.15.18:0/288604196 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b341a44b0 con 0x7f7b340685f0 2024-02-17T13:03:20.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.478+0000 7f7b30d99640 1 -- 172.21.15.18:0/288604196 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7b2403b040 con 0x7f7b340685f0 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b30d99640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b200a2230 0x7f7b200a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b200a2230 0x7f7b200a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b3bc10740 1 Processor -- start 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b30d99640 1 -- 172.21.15.18:0/288604196 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7b240efb00 con 0x7f7b340685f0 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b3bc10740 1 -- start start 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b3bc10740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767d250 0x56437767d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b3bc10740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56437767db70 0x564377686210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b200a2230 0x7f7b200a46f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7b340ffb80 tx=0x7f7b1c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b3bc10740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 0x564377688b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b3bc10740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564377638960 con 0x564377686750 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b3bc10740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643775ec310 con 0x56437767db70 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b3bc10740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643775e9840 con 0x56437767d250 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.479+0000 7f7b3359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 0x564377688b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b3359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 0x564377688b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43994/0 (socket says 172.21.15.18:43994) 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b3359e640 1 -- 172.21.15.18:0/725494911 learned_addr learned my addr 172.21.15.18:0/725494911 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/725494911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56437767db70 0x564377686210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b32d9d640 1 --2- 172.21.15.18:0/725494911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767d250 0x56437767d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b3359e640 1 -- 172.21.15.18:0/725494911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767d250 msgr2=0x56437767d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b3359e640 1 --2- 172.21.15.18:0/725494911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767d250 0x56437767d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b3359e640 1 -- 172.21.15.18:0/725494911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56437767db70 msgr2=0x564377686210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b3359e640 1 --2- 172.21.15.18:0/725494911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56437767db70 0x564377686210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b3359e640 1 -- 172.21.15.18:0/725494911 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643775a7060 con 0x564377686750 2024-02-17T13:03:20.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b3359e640 1 --2- 172.21.15.18:0/725494911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 0x564377688b40 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00e5e0 tx=0x7f7b2c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a37d7be0797323 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b297fa640 1 -- 172.21.15.18:0/725494911 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c013040 con 0x564377686750 2024-02-17T13:03:20.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b297fa640 1 -- 172.21.15.18:0/725494911 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b2c00de70 con 0x564377686750 2024-02-17T13:03:20.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.480+0000 7f7b297fa640 1 -- 172.21.15.18:0/725494911 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c016380 con 0x564377686750 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.481+0000 7f7b3bc10740 1 -- 172.21.15.18:0/725494911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 msgr2=0x564377688b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.481+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/725494911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 0x564377688b40 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00e5e0 tx=0x7f7b2c0095c0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.481+0000 7f7b3bc10740 1 -- 172.21.15.18:0/725494911 shutdown_connections 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.481+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/725494911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 0x564377688b40 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.481+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/725494911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56437767db70 0x564377686210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.481+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/725494911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767d250 0x56437767d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.481+0000 7f7b3bc10740 1 -- 172.21.15.18:0/725494911 >> 172.21.15.18:0/725494911 conn(0x564377677980 msgr2=0x564377677d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.481+0000 7f7b3bc10740 1 -- 172.21.15.18:0/725494911 shutdown_connections 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.481+0000 7f7b3bc10740 1 -- 172.21.15.18:0/725494911 wait complete. 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.481+0000 7f7b3bc10740 1 -- start start 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.481+0000 7f7b3bc10740 1 Processor -- start 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b3bc10740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767d250 0x564377683730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b3bc10740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767db70 0x564377683c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b3bc10740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564377686750 0x5643776841b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b3bc10740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643775e5020 con 0x56437767db70 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b3bc10740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643772b8430 con 0x564377686750 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b3bc10740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564377274e00 con 0x56437767d250 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b3bc10740 10 client.0 _ll_register_callbacks cb 0x56437758ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b2bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767db70 0x564377683c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b2bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767db70 0x564377683c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44000/0 (socket says 172.21.15.18:44000) 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b2bfff640 1 -- 172.21.15.18:0/1380686229 learned_addr learned my addr 172.21.15.18:0/1380686229 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b3359e640 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564377686750 0x5643776841b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b32d9d640 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767d250 0x564377683730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b2bfff640 1 -- 172.21.15.18:0/1380686229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767d250 msgr2=0x564377683730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767d250 0x564377683730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b2bfff640 1 -- 172.21.15.18:0/1380686229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564377686750 msgr2=0x5643776841b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564377686750 0x5643776841b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b2bfff640 1 -- 172.21.15.18:0/1380686229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2c009e30 con 0x56437767db70 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b3359e640 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564377686750 0x5643776841b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767db70 0x564377683c70 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c003850 tx=0x7f7b1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c408d425300667 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b1a7fc640 1 -- 172.21.15.18:0/1380686229 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c022070 con 0x56437767db70 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.482+0000 7f7b1a7fc640 1 -- 172.21.15.18:0/1380686229 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7b1c00e6a0 con 0x56437767db70 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b3bc10740 1 -- 172.21.15.18:0/1380686229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643772e47e0 con 0x56437767db70 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b1a7fc640 1 -- 172.21.15.18:0/1380686229 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b1c00e860 con 0x56437767db70 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b3bc10740 10 client.20397 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b3bc10740 1 -- 172.21.15.18:0/1380686229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5643775e36a0 con 0x56437767db70 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b1a7fc640 1 -- 172.21.15.18:0/1380686229 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5643772e47e0 con 0x56437767db70 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b1a7fc640 1 -- 172.21.15.18:0/1380686229 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f7b1c027b80 con 0x56437767db70 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b3bc10740 10 client.20397 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b3bc10740 10 client.20397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b3bc10740 20 client.20397 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564377696cb0 0x5643776990d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b3bc10740 4 client.20397 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b3bc10740 1 -- 172.21.15.18:0/1380686229 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564377321720 con 0x564377696cb0 2024-02-17T13:03:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.483+0000 7f7b32d9d640 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564377696cb0 0x5643776990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.484+0000 7f7b32d9d640 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564377696cb0 0x5643776990d0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eaf6a98424251b7 server_cookie=beacd7499024f0d in_seq=0 out_seq=0 2024-02-17T13:03:20.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.484+0000 7f7b1a7fc640 10 client.20397 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.486+0000 7f7b1a7fc640 1 -- 172.21.15.18:0/1380686229 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564377321720 con 0x564377696cb0 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b1a7fc640 10 client.20397 handle_command_reply: tid=0 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b3bc10740 1 client.20397 shutdown 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b3bc10740 1 -- 172.21.15.18:0/1380686229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767db70 msgr2=0x564377683c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767db70 0x564377683c70 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c003850 tx=0x7f7b1c004730 comp rx=0 tx=0).stop 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b3bc10740 1 -- 172.21.15.18:0/1380686229 shutdown_connections 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564377696cb0 0x5643776990d0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564377686750 0x5643776841b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767db70 0x564377683c70 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/1380686229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767d250 0x564377683730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b1a7fc640 0 client.20397 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b3bc10740 1 -- 172.21.15.18:0/1380686229 >> 172.21.15.18:0/1380686229 conn(0x564377677980 msgr2=0x5643776796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b3bc10740 1 -- 172.21.15.18:0/1380686229 shutdown_connections 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b3bc10740 1 -- 172.21.15.18:0/1380686229 wait complete. 2024-02-17T13:03:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.487+0000 7f7b3bc10740 20 client.20397 trim_cache size 0 max 16384 2024-02-17T13:03:20.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.501+0000 7f7b3bc10740 1 Processor -- start 2024-02-17T13:03:20.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.501+0000 7f7b3bc10740 1 -- start start 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.501+0000 7f7b3bc10740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767d250 0x564377685e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.501+0000 7f7b3bc10740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767db70 0x564377683fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.501+0000 7f7b3bc10740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564377686750 0x564377680990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.501+0000 7f7b3bc10740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437756a130 con 0x56437767d250 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.501+0000 7f7b3bc10740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437767c9f0 con 0x564377686750 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.501+0000 7f7b3bc10740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643776735c0 con 0x56437767db70 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.501+0000 7f7b32d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767d250 0x564377685e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.501+0000 7f7b32d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767d250 0x564377685e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44008/0 (socket says 172.21.15.18:44008) 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.501+0000 7f7b32d9d640 1 -- 172.21.15.18:0/2390884881 learned_addr learned my addr 172.21.15.18:0/2390884881 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.501+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/2390884881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767db70 0x564377683fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.502+0000 7f7b3359e640 1 --2- 172.21.15.18:0/2390884881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564377686750 0x564377680990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.502+0000 7f7b2bfff640 1 -- 172.21.15.18:0/2390884881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564377686750 msgr2=0x564377680990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.502+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/2390884881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564377686750 0x564377680990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.502+0000 7f7b2bfff640 1 -- 172.21.15.18:0/2390884881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767d250 msgr2=0x564377685e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.502+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/2390884881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767d250 0x564377685e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.502+0000 7f7b2bfff640 1 -- 172.21.15.18:0/2390884881 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56437757fc20 con 0x56437767db70 2024-02-17T13:03:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.502+0000 7f7b32d9d640 1 --2- 172.21.15.18:0/2390884881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767d250 0x564377685e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.502+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/2390884881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767db70 0x564377683fd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c027f30 tx=0x7f7b1c005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff97a3d27ada063 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.502+0000 7f7b28ff9640 1 -- 172.21.15.18:0/2390884881 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c022040 con 0x56437767db70 2024-02-17T13:03:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.503+0000 7f7b28ff9640 1 -- 172.21.15.18:0/2390884881 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b1c004490 con 0x56437767db70 2024-02-17T13:03:20.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.503+0000 7f7b28ff9640 1 -- 172.21.15.18:0/2390884881 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c0279c0 con 0x56437767db70 2024-02-17T13:03:20.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.503+0000 7f7b3bc10740 1 -- 172.21.15.18:0/2390884881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767db70 msgr2=0x564377683fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.503+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/2390884881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767db70 0x564377683fd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c027f30 tx=0x7f7b1c005870 comp rx=0 tx=0).stop 2024-02-17T13:03:20.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.503+0000 7f7b3bc10740 1 -- 172.21.15.18:0/2390884881 shutdown_connections 2024-02-17T13:03:20.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.503+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/2390884881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564377686750 0x564377680990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.503+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/2390884881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767db70 0x564377683fd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.503+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/2390884881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56437767d250 0x564377685e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.503+0000 7f7b3bc10740 1 -- 172.21.15.18:0/2390884881 >> 172.21.15.18:0/2390884881 conn(0x564377677980 msgr2=0x56437767e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.503+0000 7f7b3bc10740 1 -- 172.21.15.18:0/2390884881 shutdown_connections 2024-02-17T13:03:20.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b3bc10740 1 -- 172.21.15.18:0/2390884881 wait complete. 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b3bc10740 1 -- start start 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b3bc10740 1 Processor -- start 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b3bc10740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56437767d250 0x56437768c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b3bc10740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767db70 0x56437768c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b3bc10740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 0x56437768cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b3bc10740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643776ed1c0 con 0x564377686750 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b3bc10740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643776ed300 con 0x56437767d250 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b3bc10740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643776ed440 con 0x56437767db70 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b3bc10740 10 client.0 _ll_register_callbacks cb 0x7f7b2c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b3359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 0x56437768cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b2bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767db70 0x56437768c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.504+0000 7f7b32d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56437767d250 0x56437768c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.505+0000 7f7b3359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 0x56437768cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44020/0 (socket says 172.21.15.18:44020) 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.505+0000 7f7b3359e640 1 -- 172.21.15.18:0/3917631937 learned_addr learned my addr 172.21.15.18:0/3917631937 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:20.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.505+0000 7f7b2bfff640 1 -- 172.21.15.18:0/3917631937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56437767d250 msgr2=0x56437768c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.505+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/3917631937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56437767d250 0x56437768c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.505+0000 7f7b2bfff640 1 -- 172.21.15.18:0/3917631937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 msgr2=0x56437768cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:20.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.505+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/3917631937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 0x56437768cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.505+0000 7f7b2bfff640 1 -- 172.21.15.18:0/3917631937 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b341a44b0 con 0x56437767db70 2024-02-17T13:03:20.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.505+0000 7f7b2bfff640 1 --2- 172.21.15.18:0/3917631937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767db70 0x56437768c5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x564377669a80 tx=0x7f7b1c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e9524bc28e97503 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:20.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.505+0000 7f7b1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:03:20.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.505+0000 7f7b1a7fc640 1 -- 172.21.15.18:0/3917631937 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c027c60 con 0x56437767db70 2024-02-17T13:03:20.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.505+0000 7f7b1a7fc640 1 -- 172.21.15.18:0/3917631937 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7b1c00e8b0 con 0x56437767db70 2024-02-17T13:03:20.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.505+0000 7f7b3bc10740 1 -- 172.21.15.18:0/3917631937 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b2c00de70 con 0x56437767db70 2024-02-17T13:03:20.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.506+0000 7f7b1a7fc640 1 -- 172.21.15.18:0/3917631937 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b1c00ea70 con 0x56437767db70 2024-02-17T13:03:20.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.506+0000 7f7b3bc10740 10 client.29329 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:20.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.506+0000 7f7b3bc10740 1 -- 172.21.15.18:0/3917631937 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5643775e36a0 con 0x56437767db70 2024-02-17T13:03:20.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.506+0000 7f7b1a7fc640 1 -- 172.21.15.18:0/3917631937 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b1c04d080 con 0x56437767db70 2024-02-17T13:03:20.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.506+0000 7f7b1a7fc640 1 -- 172.21.15.18:0/3917631937 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f7b1c039460 con 0x56437767db70 2024-02-17T13:03:20.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.506+0000 7f7b3bc10740 10 client.29329 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:20.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.506+0000 7f7b3bc10740 10 client.29329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:20.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.506+0000 7f7b3bc10740 20 client.29329 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:20.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.506+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/3917631937 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5643776ee8c0 0x5643776f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:20.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.506+0000 7f7b3bc10740 4 client.29329 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:03:20.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.506+0000 7f7b3bc10740 1 -- 172.21.15.18:0/3917631937 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564377370890 con 0x5643776ee8c0 2024-02-17T13:03:20.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.507+0000 7f7b32d9d640 1 --2- 172.21.15.18:0/3917631937 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5643776ee8c0 0x5643776f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:20.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.508+0000 7f7b32d9d640 1 --2- 172.21.15.18:0/3917631937 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5643776ee8c0 0x5643776f0ce0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52e971db6d2253 server_cookie=13d95687f41af153 in_seq=0 out_seq=0 2024-02-17T13:03:20.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.508+0000 7f7b1a7fc640 10 client.29329 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:20.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b1a7fc640 1 -- 172.21.15.18:0/3917631937 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564377370890 con 0x5643776ee8c0 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b1a7fc640 10 client.29329 handle_command_reply: tid=0 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b3bc10740 1 client.29329 shutdown 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b3bc10740 1 -- 172.21.15.18:0/3917631937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767db70 msgr2=0x56437768c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/3917631937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767db70 0x56437768c5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x564377669a80 tx=0x7f7b1c014000 comp rx=0 tx=0).stop 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b3bc10740 1 -- 172.21.15.18:0/3917631937 shutdown_connections 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/3917631937 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5643776ee8c0 0x5643776f0ce0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/3917631937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564377686750 0x56437768cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/3917631937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56437767db70 0x56437768c5a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b1a7fc640 0 client.29329 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b3bc10740 1 --2- 172.21.15.18:0/3917631937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56437767d250 0x56437768c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b3bc10740 1 -- 172.21.15.18:0/3917631937 >> 172.21.15.18:0/3917631937 conn(0x564377677980 msgr2=0x5643776762f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b3bc10740 1 -- 172.21.15.18:0/3917631937 shutdown_connections 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b3bc10740 1 -- 172.21.15.18:0/3917631937 wait complete. 2024-02-17T13:03:20.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.512+0000 7f7b3bc10740 20 client.29329 trim_cache size 0 max 16384 2024-02-17T13:03:20.503 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:03:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.513+0000 7f7b39040640 1 -- 172.21.15.18:0/288604196 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b200a2230 msgr2=0x7f7b200a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.513+0000 7f7b39040640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b200a2230 0x7f7b200a46f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7b340ffb80 tx=0x7f7b1c005f90 comp rx=0 tx=0).stop 2024-02-17T13:03:20.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.513+0000 7f7b39040640 1 -- 172.21.15.18:0/288604196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b340685f0 msgr2=0x7f7b3419d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.513+0000 7f7b39040640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b340685f0 0x7f7b3419d410 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f7b24037040 tx=0x7f7b240045b0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.513+0000 7f7b39040640 1 -- 172.21.15.18:0/288604196 shutdown_connections 2024-02-17T13:03:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.513+0000 7f7b39040640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f7b200a2230 0x7f7b200a46f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.513+0000 7f7b39040640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b341975c0 0x7f7b34197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.513+0000 7f7b39040640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b341064c0 0x7f7b3419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.513+0000 7f7b39040640 1 --2- 172.21.15.18:0/288604196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b340685f0 0x7f7b3419d410 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.513+0000 7f7b39040640 1 -- 172.21.15.18:0/288604196 >> 172.21.15.18:0/288604196 conn(0x7f7b34077180 msgr2=0x7f7b340fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.513+0000 7f7b39040640 1 -- 172.21.15.18:0/288604196 shutdown_connections 2024-02-17T13:03:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:20.513+0000 7f7b39040640 1 -- 172.21.15.18:0/288604196 wait complete. 2024-02-17T13:03:20.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:20 smithi146 ceph-mon[36454]: pgmap v423: 97 pgs: 97 active+clean; 479 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 21 MiB/s wr, 538 op/s 2024-02-17T13:03:20.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:20 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/673299401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:03:20.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:20 smithi018 ceph-mon[32625]: pgmap v423: 97 pgs: 97 active+clean; 479 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 21 MiB/s wr, 538 op/s 2024-02-17T13:03:20.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:20 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/673299401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:03:20.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 13393 28.28 MB/sec warmup 32 sec latency 4942.499 ms 2024-02-17T13:03:20.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:20 smithi119 ceph-mon[36188]: pgmap v423: 97 pgs: 97 active+clean; 479 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 21 MiB/s wr, 538 op/s 2024-02-17T13:03:20.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:20 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/673299401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:03:21.519 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T13:03:21.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:03:21.519 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T13:03:21.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.645+0000 7fefdb1d0640 1 -- 172.21.15.18:0/360626540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd4105330 msgr2=0x7fefd410cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.645+0000 7fefdb1d0640 1 --2- 172.21.15.18:0/360626540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd4105330 0x7fefd410cf00 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009930 tx=0x7fefbc02bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:21.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.645+0000 7fefdb1d0640 1 -- 172.21.15.18:0/360626540 shutdown_connections 2024-02-17T13:03:21.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.645+0000 7fefdb1d0640 1 --2- 172.21.15.18:0/360626540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd4105330 0x7fefd410cf00 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.645+0000 7fefdb1d0640 1 --2- 172.21.15.18:0/360626540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd4100c10 0x7fefd4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.645+0000 7fefdb1d0640 1 --2- 172.21.15.18:0/360626540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd4100260 0x7fefd4100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.645+0000 7fefdb1d0640 1 -- 172.21.15.18:0/360626540 >> 172.21.15.18:0/360626540 conn(0x7fefd4077180 msgr2=0x7fefd4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:21.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.645+0000 7fefdb1d0640 1 -- 172.21.15.18:0/360626540 shutdown_connections 2024-02-17T13:03:21.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.645+0000 7fefdb1d0640 1 -- 172.21.15.18:0/360626540 wait complete. 2024-02-17T13:03:21.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.645+0000 7fefdb1d0640 1 Processor -- start 2024-02-17T13:03:21.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.645+0000 7fefdb1d0640 1 -- start start 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefdb1d0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd4100260 0x7fefd4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefdb1d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd4105330 0x7fefd4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefdb1d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd41990d0 0x7fefd41a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefdb1d0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd41a3d20 con 0x7fefd4100260 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefdb1d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd41a3e90 con 0x7fefd4105330 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefdb1d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd41a4050 con 0x7fefd41990d0 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd8f6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd4100260 0x7fefd4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd8f6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd4100260 0x7fefd4198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44036/0 (socket says 172.21.15.18:44036) 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd8f6c640 1 -- 172.21.15.18:0/2562323198 learned_addr learned my addr 172.21.15.18:0/2562323198 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd976d640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd41990d0 0x7fefd41a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd3fff640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd4105330 0x7fefd4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd8f6c640 1 -- 172.21.15.18:0/2562323198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd41990d0 msgr2=0x7fefd41a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd41990d0 0x7fefd41a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd8f6c640 1 -- 172.21.15.18:0/2562323198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd4105330 msgr2=0x7fefd4198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd4105330 0x7fefd4198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd8f6c640 1 -- 172.21.15.18:0/2562323198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc009490 con 0x7fefd4100260 2024-02-17T13:03:21.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd976d640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd41990d0 0x7fefd41a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:21.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd3fff640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd4105330 0x7fefd4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:21.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd4100260 0x7fefd4198600 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00ea60 tx=0x7fefcc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:21.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefd1ffb640 1 -- 172.21.15.18:0/2562323198 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc00cde0 con 0x7fefd4100260 2024-02-17T13:03:21.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.646+0000 7fefdb1d0640 1 -- 172.21.15.18:0/2562323198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc009560 con 0x7fefd4100260 2024-02-17T13:03:21.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.647+0000 7fefd1ffb640 1 -- 172.21.15.18:0/2562323198 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefcc0035f0 con 0x7fefd4100260 2024-02-17T13:03:21.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.647+0000 7fefd1ffb640 1 -- 172.21.15.18:0/2562323198 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc010690 con 0x7fefd4100260 2024-02-17T13:03:21.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.647+0000 7fefdb1d0640 1 -- 172.21.15.18:0/2562323198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd41a4700 con 0x7fefd4100260 2024-02-17T13:03:21.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.649+0000 7fefd1ffb640 1 -- 172.21.15.18:0/2562323198 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fefcc003af0 con 0x7fefd4100260 2024-02-17T13:03:21.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.649+0000 7fefddd9b740 1 Processor -- start 2024-02-17T13:03:21.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefddd9b740 1 -- start start 2024-02-17T13:03:21.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd1ffb640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fefc00a22b0 0x7fefc00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefddd9b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 0x55cc1ac7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefddd9b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefddd9b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac838b0 0x55cc1ac85d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefddd9b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1abe8f90 con 0x55cc1ac7a380 2024-02-17T13:03:21.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefddd9b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1abe64c0 con 0x55cc1ac7ad30 2024-02-17T13:03:21.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefddd9b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1abe1ca0 con 0x55cc1ac838b0 2024-02-17T13:03:21.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd976d640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fefc00a22b0 0x7fefc00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd8f6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 0x55cc1ac7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd8f6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 0x55cc1ac7a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44050/0 (socket says 172.21.15.18:44050) 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd8f6c640 1 -- 172.21.15.18:0/3842448912 learned_addr learned my addr 172.21.15.18:0/3842448912 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd1ffb640 1 -- 172.21.15.18:0/2562323198 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fefcc014070 con 0x7fefd4100260 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd976d640 1 --2- 172.21.15.18:0/3842448912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac838b0 0x55cc1ac85d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd8f6c640 1 -- 172.21.15.18:0/3842448912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac838b0 msgr2=0x55cc1ac85d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/3842448912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac838b0 0x55cc1ac85d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd8f6c640 1 -- 172.21.15.18:0/3842448912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 msgr2=0x55cc1ac832e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/3842448912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac832e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd8f6c640 1 -- 172.21.15.18:0/3842448912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc1aba40b0 con 0x55cc1ac7a380 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.650+0000 7fefd3fff640 1 --2- 172.21.15.18:0/3842448912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.651+0000 7fefd976d640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fefc00a22b0 0x7fefc00a4770 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fefbc005940 tx=0x7fefbc005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:21.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.651+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/3842448912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 0x55cc1ac7a760 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0a31c0 tx=0x7fefcc007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b813b7e44ccfdf8f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.651+0000 7fefcaffd640 1 -- 172.21.15.18:0/3842448912 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc099090 con 0x55cc1ac7a380 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.651+0000 7fefcaffd640 1 -- 172.21.15.18:0/3842448912 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefcc098e30 con 0x55cc1ac7a380 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.651+0000 7fefcaffd640 1 -- 172.21.15.18:0/3842448912 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc09af50 con 0x55cc1ac7a380 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.651+0000 7fefddd9b740 1 -- 172.21.15.18:0/3842448912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 msgr2=0x55cc1ac7a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.652+0000 7fefddd9b740 1 --2- 172.21.15.18:0/3842448912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 0x55cc1ac7a760 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0a31c0 tx=0x7fefcc007b90 comp rx=0 tx=0).stop 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.652+0000 7fefddd9b740 1 -- 172.21.15.18:0/3842448912 shutdown_connections 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.652+0000 7fefddd9b740 1 --2- 172.21.15.18:0/3842448912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac838b0 0x55cc1ac85d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.652+0000 7fefddd9b740 1 --2- 172.21.15.18:0/3842448912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.652+0000 7fefddd9b740 1 --2- 172.21.15.18:0/3842448912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 0x55cc1ac7a760 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.652+0000 7fefddd9b740 1 -- 172.21.15.18:0/3842448912 >> 172.21.15.18:0/3842448912 conn(0x55cc1ac74840 msgr2=0x55cc1ac74c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.652+0000 7fefddd9b740 1 -- 172.21.15.18:0/3842448912 shutdown_connections 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.652+0000 7fefddd9b740 1 -- 172.21.15.18:0/3842448912 wait complete. 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.652+0000 7fefddd9b740 1 -- start start 2024-02-17T13:03:21.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.652+0000 7fefddd9b740 1 Processor -- start 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.652+0000 7fefddd9b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac7a380 0x55cc1ac80800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.652+0000 7fefddd9b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac80d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefddd9b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 0x55cc1ac81280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefddd9b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1ab86f40 con 0x55cc1ac838b0 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefddd9b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1a8b54b0 con 0x55cc1ac7ad30 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefddd9b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1ab67670 con 0x55cc1ac7a380 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefddd9b740 10 client.0 _ll_register_callbacks cb 0x55cc1abd4f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd976d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 0x55cc1ac81280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac80d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd976d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 0x55cc1ac81280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44052/0 (socket says 172.21.15.18:44052) 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd976d640 1 -- 172.21.15.18:0/2463794231 learned_addr learned my addr 172.21.15.18:0/2463794231 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd8f6c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac7a380 0x55cc1ac80800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd976d640 1 -- 172.21.15.18:0/2463794231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac7a380 msgr2=0x55cc1ac80800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd976d640 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac7a380 0x55cc1ac80800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd976d640 1 -- 172.21.15.18:0/2463794231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 msgr2=0x55cc1ac80d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd976d640 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac80d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd976d640 1 -- 172.21.15.18:0/2463794231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd41a4700 con 0x55cc1ac838b0 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd3fff640 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac80d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac7a380 0x55cc1ac80800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefd976d640 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 0x55cc1ac81280 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0044c0 tx=0x7fefbc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10d2bdc4b43fa93 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefb3fff640 1 -- 172.21.15.18:0/2463794231 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc005580 con 0x55cc1ac838b0 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefb3fff640 1 -- 172.21.15.18:0/2463794231 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fefbc0035f0 con 0x55cc1ac838b0 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.653+0000 7fefddd9b740 1 -- 172.21.15.18:0/2463794231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc1abbbf40 con 0x55cc1ac838b0 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.654+0000 7fefb3fff640 1 -- 172.21.15.18:0/2463794231 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefbc0037b0 con 0x55cc1ac838b0 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.654+0000 7fefddd9b740 10 client.20415 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:21.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.654+0000 7fefddd9b740 1 -- 172.21.15.18:0/2463794231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55cc1abe0320 con 0x55cc1ac838b0 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.654+0000 7fefb3fff640 1 -- 172.21.15.18:0/2463794231 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cc1abbbf40 con 0x55cc1ac838b0 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.654+0000 7fefb3fff640 1 -- 172.21.15.18:0/2463794231 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fefbc050840 con 0x55cc1ac838b0 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.654+0000 7fefddd9b740 10 client.20415 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.654+0000 7fefddd9b740 10 client.20415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.654+0000 7fefddd9b740 20 client.20415 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.654+0000 7fefddd9b740 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cc1ac93cd0 0x55cc1ac960f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.654+0000 7fefddd9b740 4 client.20415 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.654+0000 7fefddd9b740 1 -- 172.21.15.18:0/2463794231 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc1abd8310 con 0x55cc1ac93cd0 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.654+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cc1ac93cd0 0x55cc1ac960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.655+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cc1ac93cd0 0x55cc1ac960f0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc4d665289fc255 server_cookie=73f10c1edce31d22 in_seq=0 out_seq=0 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.655+0000 7fefb3fff640 10 client.20415 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.655+0000 7fefb3fff640 1 -- 172.21.15.18:0/2463794231 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc1abd8310 con 0x55cc1ac93cd0 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.655+0000 7fefb3fff640 10 client.20415 handle_command_reply: tid=0 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.655+0000 7fefddd9b740 1 client.20415 shutdown 2024-02-17T13:03:21.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.656+0000 7fefddd9b740 1 -- 172.21.15.18:0/2463794231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 msgr2=0x55cc1ac81280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.656+0000 7fefddd9b740 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 0x55cc1ac81280 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0044c0 tx=0x7fefbc0058d0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.656+0000 7fefddd9b740 1 -- 172.21.15.18:0/2463794231 shutdown_connections 2024-02-17T13:03:21.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.656+0000 7fefddd9b740 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cc1ac93cd0 0x55cc1ac960f0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.656+0000 7fefddd9b740 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 0x55cc1ac81280 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.656+0000 7fefddd9b740 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac80d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.656+0000 7fefb3fff640 0 client.20415 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:21.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.656+0000 7fefddd9b740 1 --2- 172.21.15.18:0/2463794231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac7a380 0x55cc1ac80800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.656+0000 7fefddd9b740 1 -- 172.21.15.18:0/2463794231 >> 172.21.15.18:0/2463794231 conn(0x55cc1ac74840 msgr2=0x55cc1ac767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:21.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.656+0000 7fefddd9b740 1 -- 172.21.15.18:0/2463794231 shutdown_connections 2024-02-17T13:03:21.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.656+0000 7fefddd9b740 1 -- 172.21.15.18:0/2463794231 wait complete. 2024-02-17T13:03:21.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.656+0000 7fefddd9b740 20 client.20415 trim_cache size 0 max 16384 2024-02-17T13:03:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.672+0000 7fefddd9b740 1 Processor -- start 2024-02-17T13:03:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.672+0000 7fefddd9b740 1 -- start start 2024-02-17T13:03:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.672+0000 7fefddd9b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7a380 0x55cc1ac82f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.672+0000 7fefddd9b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.672+0000 7fefddd9b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 0x55cc1ac9aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.672+0000 7fefddd9b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1ac93aa0 con 0x55cc1ac838b0 2024-02-17T13:03:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.672+0000 7fefddd9b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1ac7c410 con 0x55cc1ac7a380 2024-02-17T13:03:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.672+0000 7fefddd9b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1ac7ed50 con 0x55cc1ac7ad30 2024-02-17T13:03:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.672+0000 7fefd976d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 0x55cc1ac9aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.672+0000 7fefd976d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 0x55cc1ac9aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44066/0 (socket says 172.21.15.18:44066) 2024-02-17T13:03:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.672+0000 7fefd976d640 1 -- 172.21.15.18:0/865305544 learned_addr learned my addr 172.21.15.18:0/865305544 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.673+0000 7fefd3fff640 1 --2- 172.21.15.18:0/865305544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.673+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/865305544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7a380 0x55cc1ac82f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.673+0000 7fefd976d640 1 -- 172.21.15.18:0/865305544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac7ad30 msgr2=0x55cc1ac810a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.673+0000 7fefd976d640 1 --2- 172.21.15.18:0/865305544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac810a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.673+0000 7fefd976d640 1 -- 172.21.15.18:0/865305544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7a380 msgr2=0x55cc1ac82f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.673+0000 7fefd976d640 1 --2- 172.21.15.18:0/865305544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7a380 0x55cc1ac82f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.673+0000 7fefd976d640 1 -- 172.21.15.18:0/865305544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc1ab7cc00 con 0x55cc1ac838b0 2024-02-17T13:03:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.673+0000 7fefd976d640 1 --2- 172.21.15.18:0/865305544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 0x55cc1ac9aa90 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fefbc050bc0 tx=0x7fefbc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1b83d6b46d8cd2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.673+0000 7fefca7fc640 1 -- 172.21.15.18:0/865305544 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0051c0 con 0x55cc1ac838b0 2024-02-17T13:03:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.673+0000 7fefca7fc640 1 -- 172.21.15.18:0/865305544 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefbc050960 con 0x55cc1ac838b0 2024-02-17T13:03:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.673+0000 7fefca7fc640 1 -- 172.21.15.18:0/865305544 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc034950 con 0x55cc1ac838b0 2024-02-17T13:03:21.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.674+0000 7fefddd9b740 1 -- 172.21.15.18:0/865305544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 msgr2=0x55cc1ac9aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.674+0000 7fefddd9b740 1 --2- 172.21.15.18:0/865305544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 0x55cc1ac9aa90 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fefbc050bc0 tx=0x7fefbc005b50 comp rx=0 tx=0).stop 2024-02-17T13:03:21.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.674+0000 7fefddd9b740 1 -- 172.21.15.18:0/865305544 shutdown_connections 2024-02-17T13:03:21.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.674+0000 7fefddd9b740 1 --2- 172.21.15.18:0/865305544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac838b0 0x55cc1ac9aa90 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.674+0000 7fefddd9b740 1 --2- 172.21.15.18:0/865305544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.674+0000 7fefddd9b740 1 --2- 172.21.15.18:0/865305544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7a380 0x55cc1ac82f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.674+0000 7fefddd9b740 1 -- 172.21.15.18:0/865305544 >> 172.21.15.18:0/865305544 conn(0x55cc1ac74840 msgr2=0x55cc1ac74c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.674+0000 7fefddd9b740 1 -- 172.21.15.18:0/865305544 shutdown_connections 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.674+0000 7fefddd9b740 1 -- 172.21.15.18:0/865305544 wait complete. 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.674+0000 7fefddd9b740 1 -- start start 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.674+0000 7fefddd9b740 1 Processor -- start 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefddd9b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 0x55cc1ac8ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefddd9b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac88fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefddd9b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac838b0 0x55cc1ac894f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefddd9b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1ac89bf0 con 0x55cc1ac7a380 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefddd9b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1ac89d60 con 0x55cc1ac7ad30 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefddd9b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1acea220 con 0x55cc1ac838b0 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefddd9b740 10 client.0 _ll_register_callbacks cb 0x7fefcc003df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefd8f6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 0x55cc1ac8ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefd8f6c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 0x55cc1ac8ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44082/0 (socket says 172.21.15.18:44082) 2024-02-17T13:03:21.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefd8f6c640 1 -- 172.21.15.18:0/1750661482 learned_addr learned my addr 172.21.15.18:0/1750661482 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefd976d640 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac838b0 0x55cc1ac894f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefd3fff640 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac88fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefd8f6c640 1 -- 172.21.15.18:0/1750661482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac838b0 msgr2=0x55cc1ac894f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac838b0 0x55cc1ac894f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefd8f6c640 1 -- 172.21.15.18:0/1750661482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 msgr2=0x55cc1ac88fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac88fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefd8f6c640 1 -- 172.21.15.18:0/1750661482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc1abe0320 con 0x55cc1ac7a380 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefd3fff640 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac88fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefd8f6c640 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 0x55cc1ac8ac10 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fefcc097920 tx=0x7fefcc003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb30f12ba79207ec server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefb3fff640 1 -- 172.21.15.18:0/1750661482 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc09b650 con 0x55cc1ac7a380 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefb3fff640 1 -- 172.21.15.18:0/1750661482 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fefcc09f0a0 con 0x55cc1ac7a380 2024-02-17T13:03:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.675+0000 7fefddd9b740 1 -- 172.21.15.18:0/1750661482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc1ac07900 con 0x55cc1ac7a380 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.676+0000 7fefb3fff640 1 -- 172.21.15.18:0/1750661482 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefcc0a36f0 con 0x55cc1ac7a380 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.676+0000 7fefddd9b740 10 client.20421 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.676+0000 7fefddd9b740 1 -- 172.21.15.18:0/1750661482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55cc1aba2910 con 0x55cc1ac7a380 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.676+0000 7fefb3fff640 1 -- 172.21.15.18:0/1750661482 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cc1ac07900 con 0x55cc1ac7a380 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.676+0000 7fefb3fff640 1 -- 172.21.15.18:0/1750661482 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fefcc09b1b0 con 0x55cc1ac7a380 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.676+0000 7fefddd9b740 10 client.20421 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.676+0000 7fefddd9b740 10 client.20421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.676+0000 7fefddd9b740 20 client.20421 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.676+0000 7fefddd9b740 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cc1aceb710 0x55cc1acedb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.676+0000 7fefddd9b740 4 client.20421 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.676+0000 7fefddd9b740 1 -- 172.21.15.18:0/1750661482 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cc1a8e1860 con 0x55cc1aceb710 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.676+0000 7fefd3fff640 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cc1aceb710 0x55cc1acedb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.677+0000 7fefd3fff640 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cc1aceb710 0x55cc1acedb30 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c771769eb682c262 server_cookie=4f0772aca7d6c306 in_seq=0 out_seq=0 2024-02-17T13:03:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.677+0000 7fefb3fff640 10 client.20421 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefb3fff640 1 -- 172.21.15.18:0/1750661482 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cc1a8e1860 con 0x55cc1aceb710 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefb3fff640 10 client.20421 handle_command_reply: tid=0 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefddd9b740 1 client.20421 shutdown 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefddd9b740 1 -- 172.21.15.18:0/1750661482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 msgr2=0x55cc1ac8ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefddd9b740 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 0x55cc1ac8ac10 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fefcc097920 tx=0x7fefcc003ef0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefddd9b740 1 -- 172.21.15.18:0/1750661482 shutdown_connections 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefddd9b740 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cc1aceb710 0x55cc1acedb30 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefddd9b740 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc1ac838b0 0x55cc1ac894f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefddd9b740 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc1ac7ad30 0x55cc1ac88fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefb3fff640 0 client.20421 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefddd9b740 1 --2- 172.21.15.18:0/1750661482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cc1ac7a380 0x55cc1ac8ac10 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefddd9b740 1 -- 172.21.15.18:0/1750661482 >> 172.21.15.18:0/1750661482 conn(0x55cc1ac74840 msgr2=0x55cc1ac9aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefddd9b740 1 -- 172.21.15.18:0/1750661482 shutdown_connections 2024-02-17T13:03:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.680+0000 7fefddd9b740 1 -- 172.21.15.18:0/1750661482 wait complete. 2024-02-17T13:03:21.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefddd9b740 20 client.20421 trim_cache size 0 max 16384 2024-02-17T13:03:21.674 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:03:21.674 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-02-17T13:03:21.674 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "6e3955b4-a093-4924-a887-6fea290f9ecb", 2024-02-17T13:03:21.674 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-02-17T13:03:21.674 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:03:21.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefdb1d0640 1 -- 172.21.15.18:0/2562323198 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fefc00a22b0 msgr2=0x7fefc00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefdb1d0640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fefc00a22b0 0x7fefc00a4770 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fefbc005940 tx=0x7fefbc005fd0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefdb1d0640 1 -- 172.21.15.18:0/2562323198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd4100260 msgr2=0x7fefd4198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefdb1d0640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd4100260 0x7fefd4198600 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00ea60 tx=0x7fefcc00ef20 comp rx=0 tx=0).stop 2024-02-17T13:03:21.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefdb1d0640 1 -- 172.21.15.18:0/2562323198 shutdown_connections 2024-02-17T13:03:21.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefdb1d0640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fefc00a22b0 0x7fefc00a4770 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefdb1d0640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefd41990d0 0x7fefd41a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefdb1d0640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd4105330 0x7fefd4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefdb1d0640 1 --2- 172.21.15.18:0/2562323198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefd4100260 0x7fefd4198600 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefdb1d0640 1 -- 172.21.15.18:0/2562323198 >> 172.21.15.18:0/2562323198 conn(0x7fefd4077180 msgr2=0x7fefd40fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:21.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefdb1d0640 1 -- 172.21.15.18:0/2562323198 shutdown_connections 2024-02-17T13:03:21.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.681+0000 7fefdb1d0640 1 -- 172.21.15.18:0/2562323198 wait complete. 2024-02-17T13:03:21.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:03:21.683 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:03:21.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 13918 28.19 MB/sec warmup 33 sec latency 1667.066 ms 2024-02-17T13:03:21.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.809+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3503916986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8b4100c20 msgr2=0x7fa8b4104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.809+0000 7fa8bb42e640 1 --2- 172.21.15.18:0/3503916986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8b4100c20 0x7fa8b4104c60 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009930 tx=0x7fa8a402bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:21.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.809+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3503916986 shutdown_connections 2024-02-17T13:03:21.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.809+0000 7fa8bb42e640 1 --2- 172.21.15.18:0/3503916986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8b4105340 0x7fa8b410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.809+0000 7fa8bb42e640 1 --2- 172.21.15.18:0/3503916986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8b4100c20 0x7fa8b4104c60 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.809+0000 7fa8bb42e640 1 --2- 172.21.15.18:0/3503916986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b4100270 0x7fa8b4100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.809+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3503916986 >> 172.21.15.18:0/3503916986 conn(0x7fa8b4077230 msgr2=0x7fa8b4077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:21.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.809+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3503916986 shutdown_connections 2024-02-17T13:03:21.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.809+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3503916986 wait complete. 2024-02-17T13:03:21.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.810+0000 7fa8bb42e640 1 Processor -- start 2024-02-17T13:03:21.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.810+0000 7fa8bb42e640 1 -- start start 2024-02-17T13:03:21.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.810+0000 7fa8bb42e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8b4100270 0x7fa8b4196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.810+0000 7fa8bb42e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b4105340 0x7fa8b4196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.810+0000 7fa8bb42e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8b419af20 0x7fa8b419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.810+0000 7fa8bb42e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b419b910 con 0x7fa8b4100270 2024-02-17T13:03:21.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.810+0000 7fa8bb42e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b419ba80 con 0x7fa8b4105340 2024-02-17T13:03:21.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.810+0000 7fa8bb42e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b419bbf0 con 0x7fa8b419af20 2024-02-17T13:03:21.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.810+0000 7fa8b91ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8b4100270 0x7fa8b4196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.810+0000 7fa8b91ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8b4100270 0x7fa8b4196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44096/0 (socket says 172.21.15.18:44096) 2024-02-17T13:03:21.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.810+0000 7fa8b91ca640 1 -- 172.21.15.18:0/3377358754 learned_addr learned my addr 172.21.15.18:0/3377358754 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:21.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b4105340 0x7fa8b4196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8b99cb640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8b419af20 0x7fa8b419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8b91ca640 1 -- 172.21.15.18:0/3377358754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8b419af20 msgr2=0x7fa8b419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8b91ca640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8b419af20 0x7fa8b419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8b91ca640 1 -- 172.21.15.18:0/3377358754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b4105340 msgr2=0x7fa8b4196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8b91ca640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b4105340 0x7fa8b4196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8b91ca640 1 -- 172.21.15.18:0/3377358754 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a4009560 con 0x7fa8b4100270 2024-02-17T13:03:21.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8b91ca640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8b4100270 0x7fa8b4196660 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000e9f0 tx=0x7fa8b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:21.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8aa7fc640 1 -- 172.21.15.18:0/3377358754 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0009700 con 0x7fa8b4100270 2024-02-17T13:03:21.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8aa7fc640 1 -- 172.21.15.18:0/3377358754 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8b000cd80 con 0x7fa8b4100270 2024-02-17T13:03:21.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8aa7fc640 1 -- 172.21.15.18:0/3377358754 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0018920 con 0x7fa8b4100270 2024-02-17T13:03:21.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3377358754 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b40682e0 con 0x7fa8b4100270 2024-02-17T13:03:21.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.811+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3377358754 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b4068760 con 0x7fa8b4100270 2024-02-17T13:03:21.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.813+0000 7fa8aa7fc640 1 -- 172.21.15.18:0/3377358754 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa8b0018a80 con 0x7fa8b4100270 2024-02-17T13:03:21.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8be000740 1 Processor -- start 2024-02-17T13:03:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8be000740 1 -- start start 2024-02-17T13:03:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8aa7fc640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa8a00a2230 0x7fa8a00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8be000740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 0x55bde75c1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8be000740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c1bd0 0x55bde75ca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8be000740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 0x55bde75ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8be000740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde7530280 con 0x55bde75c12b0 2024-02-17T13:03:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8be000740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde752d7b0 con 0x55bde75ca7b0 2024-02-17T13:03:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8be000740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde7528f90 con 0x55bde75c1bd0 2024-02-17T13:03:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8b99cb640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa8a00a2230 0x7fa8a00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8aa7fc640 1 -- 172.21.15.18:0/3377358754 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa8b0014070 con 0x7fa8b4100270 2024-02-17T13:03:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8b91ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 0x55bde75c1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8b91ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 0x55bde75c1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44102/0 (socket says 172.21.15.18:44102) 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8b91ca640 1 -- 172.21.15.18:0/2254701197 learned_addr learned my addr 172.21.15.18:0/2254701197 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8b89c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c1bd0 0x55bde75ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8b99cb640 1 --2- 172.21.15.18:0/2254701197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 0x55bde75ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8b89c9640 1 -- 172.21.15.18:0/2254701197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 msgr2=0x55bde75ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/2254701197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 0x55bde75ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8b89c9640 1 -- 172.21.15.18:0/2254701197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 msgr2=0x55bde75c1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/2254701197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 0x55bde75c1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8b89c9640 1 -- 172.21.15.18:0/2254701197 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bde74e7570 con 0x55bde75c1bd0 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.814+0000 7fa8b91ca640 1 --2- 172.21.15.18:0/2254701197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 0x55bde75c1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.815+0000 7fa8b99cb640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa8a00a2230 0x7fa8a00a46f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009fd0 tx=0x7fa8ac009d80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.815+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/2254701197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c1bd0 0x55bde75ca270 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa8a402c170 tx=0x7fa8a402c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e022a5e4870934 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.815+0000 7fa89f7fe640 1 -- 172.21.15.18:0/2254701197 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4004740 con 0x55bde75c1bd0 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.815+0000 7fa89f7fe640 1 -- 172.21.15.18:0/2254701197 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8a40048a0 con 0x55bde75c1bd0 2024-02-17T13:03:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.815+0000 7fa89f7fe640 1 -- 172.21.15.18:0/2254701197 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4005050 con 0x55bde75c1bd0 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.816+0000 7fa8be000740 1 -- 172.21.15.18:0/2254701197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c1bd0 msgr2=0x55bde75ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.816+0000 7fa8be000740 1 --2- 172.21.15.18:0/2254701197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c1bd0 0x55bde75ca270 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa8a402c170 tx=0x7fa8a402c5d0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.816+0000 7fa8be000740 1 -- 172.21.15.18:0/2254701197 shutdown_connections 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.816+0000 7fa8be000740 1 --2- 172.21.15.18:0/2254701197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 0x55bde75ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.816+0000 7fa8be000740 1 --2- 172.21.15.18:0/2254701197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c1bd0 0x55bde75ca270 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.816+0000 7fa8be000740 1 --2- 172.21.15.18:0/2254701197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 0x55bde75c1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.816+0000 7fa8be000740 1 -- 172.21.15.18:0/2254701197 >> 172.21.15.18:0/2254701197 conn(0x55bde75bb990 msgr2=0x55bde75bbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.816+0000 7fa8be000740 1 -- 172.21.15.18:0/2254701197 shutdown_connections 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.816+0000 7fa8be000740 1 -- 172.21.15.18:0/2254701197 wait complete. 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.816+0000 7fa8be000740 1 -- start start 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.816+0000 7fa8be000740 1 Processor -- start 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8be000740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 0x55bde75ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8be000740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c1bd0 0x55bde75c7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8be000740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 0x55bde75c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8be000740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde71fc430 con 0x55bde75c12b0 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8be000740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde71b8e00 con 0x55bde75ca7b0 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8be000740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde74adfd0 con 0x55bde75c1bd0 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8be000740 10 client.0 _ll_register_callbacks cb 0x55bde7503fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8b91ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 0x55bde75ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8b91ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 0x55bde75ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44106/0 (socket says 172.21.15.18:44106) 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8b91ca640 1 -- 172.21.15.18:0/4083443251 learned_addr learned my addr 172.21.15.18:0/4083443251 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8b99cb640 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 0x55bde75c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c1bd0 0x55bde75c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8b91ca640 1 -- 172.21.15.18:0/4083443251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c1bd0 msgr2=0x55bde75c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8b91ca640 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c1bd0 0x55bde75c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8b91ca640 1 -- 172.21.15.18:0/4083443251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 msgr2=0x55bde75c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8b91ca640 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 0x55bde75c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8b91ca640 1 -- 172.21.15.18:0/4083443251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b410d720 con 0x55bde75c12b0 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8b91ca640 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 0x55bde75ca1e0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fa8b009ae60 tx=0x7fa8b009a830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f451c8c47964630c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa89cff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa89cff9640 1 -- 172.21.15.18:0/4083443251 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0074070 con 0x55bde75c12b0 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa89cff9640 1 -- 172.21.15.18:0/4083443251 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa8b009aed0 con 0x55bde75c12b0 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.817+0000 7fa8be000740 1 -- 172.21.15.18:0/4083443251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bde72287e0 con 0x55bde75c12b0 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.818+0000 7fa89cff9640 1 -- 172.21.15.18:0/4083443251 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8b009ef70 con 0x55bde75c12b0 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.818+0000 7fa8be000740 10 client.20433 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.818+0000 7fa8be000740 1 -- 172.21.15.18:0/4083443251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55bde7527610 con 0x55bde75c12b0 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.818+0000 7fa89cff9640 1 -- 172.21.15.18:0/4083443251 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bde72287e0 con 0x55bde75c12b0 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.818+0000 7fa89cff9640 1 -- 172.21.15.18:0/4083443251 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fa8b00cbe00 con 0x55bde75c12b0 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.818+0000 7fa8be000740 10 client.20433 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.818+0000 7fa8be000740 10 client.20433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:21.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.818+0000 7fa8be000740 20 client.20433 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:21.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.818+0000 7fa8be000740 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bde75da800 0x55bde75dcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.818+0000 7fa8be000740 4 client.20433 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:21.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.818+0000 7fa8be000740 1 -- 172.21.15.18:0/4083443251 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bde7265720 con 0x55bde75da800 2024-02-17T13:03:21.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.818+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bde75da800 0x55bde75dcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.819+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bde75da800 0x55bde75dcbf0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de530f0ec4bd8fb server_cookie=a4d1f14693298822 in_seq=0 out_seq=0 2024-02-17T13:03:21.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.819+0000 7fa89cff9640 10 client.20433 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:21.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa89cff9640 1 -- 172.21.15.18:0/4083443251 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bde7265720 con 0x55bde75da800 2024-02-17T13:03:21.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa89cff9640 10 client.20433 handle_command_reply: tid=0 2024-02-17T13:03:21.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa8be000740 1 client.20433 shutdown 2024-02-17T13:03:21.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa8be000740 1 -- 172.21.15.18:0/4083443251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 msgr2=0x55bde75ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa8be000740 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 0x55bde75ca1e0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fa8b009ae60 tx=0x7fa8b009a830 comp rx=0 tx=0).stop 2024-02-17T13:03:21.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa8be000740 1 -- 172.21.15.18:0/4083443251 shutdown_connections 2024-02-17T13:03:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa8be000740 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bde75da800 0x55bde75dcbf0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa8be000740 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 0x55bde75c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa8be000740 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c1bd0 0x55bde75c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa8be000740 1 --2- 172.21.15.18:0/4083443251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c12b0 0x55bde75ca1e0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa89cff9640 0 client.20433 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa8be000740 1 -- 172.21.15.18:0/4083443251 >> 172.21.15.18:0/4083443251 conn(0x55bde75bb990 msgr2=0x55bde75bd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa8be000740 1 -- 172.21.15.18:0/4083443251 shutdown_connections 2024-02-17T13:03:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa8be000740 1 -- 172.21.15.18:0/4083443251 wait complete. 2024-02-17T13:03:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.824+0000 7fa8be000740 20 client.20433 trim_cache size 0 max 16384 2024-02-17T13:03:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.841+0000 7fa8be000740 1 Processor -- start 2024-02-17T13:03:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.841+0000 7fa8be000740 1 -- start start 2024-02-17T13:03:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8be000740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75c12b0 0x55bde75c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8be000740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 0x55bde75ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8be000740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75ca7b0 0x55bde75c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8be000740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde75c89d0 con 0x55bde75c1bd0 2024-02-17T13:03:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8be000740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde75b70f0 con 0x55bde75c12b0 2024-02-17T13:03:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8be000740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde75bc9e0 con 0x55bde75ca7b0 2024-02-17T13:03:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8b89c9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 0x55bde75ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8b89c9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 0x55bde75ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44122/0 (socket says 172.21.15.18:44122) 2024-02-17T13:03:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8b89c9640 1 -- 172.21.15.18:0/2298178964 learned_addr learned my addr 172.21.15.18:0/2298178964 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8b99cb640 1 --2- 172.21.15.18:0/2298178964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75ca7b0 0x55bde75c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8b91ca640 1 --2- 172.21.15.18:0/2298178964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75c12b0 0x55bde75c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8b89c9640 1 -- 172.21.15.18:0/2298178964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75ca7b0 msgr2=0x55bde75c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/2298178964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75ca7b0 0x55bde75c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8b89c9640 1 -- 172.21.15.18:0/2298178964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75c12b0 msgr2=0x55bde75c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/2298178964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75c12b0 0x55bde75c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8b89c9640 1 -- 172.21.15.18:0/2298178964 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bde74c3ac0 con 0x55bde75c1bd0 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8b91ca640 1 --2- 172.21.15.18:0/2298178964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75c12b0 0x55bde75c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/2298178964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 0x55bde75ca1e0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40092a0 tx=0x7fa8a4041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf0a0357f8b15e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa89effd640 1 -- 172.21.15.18:0/2298178964 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4004b10 con 0x55bde75c1bd0 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa89effd640 1 -- 172.21.15.18:0/2298178964 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8a4052330 con 0x55bde75c1bd0 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.842+0000 7fa89effd640 1 -- 172.21.15.18:0/2298178964 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a40050c0 con 0x55bde75c1bd0 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.843+0000 7fa8be000740 1 -- 172.21.15.18:0/2298178964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 msgr2=0x55bde75ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.843+0000 7fa8be000740 1 --2- 172.21.15.18:0/2298178964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 0x55bde75ca1e0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40092a0 tx=0x7fa8a4041040 comp rx=0 tx=0).stop 2024-02-17T13:03:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.843+0000 7fa8be000740 1 -- 172.21.15.18:0/2298178964 shutdown_connections 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.843+0000 7fa8be000740 1 --2- 172.21.15.18:0/2298178964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75ca7b0 0x55bde75c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.843+0000 7fa8be000740 1 --2- 172.21.15.18:0/2298178964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 0x55bde75ca1e0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.843+0000 7fa8be000740 1 --2- 172.21.15.18:0/2298178964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75c12b0 0x55bde75c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.843+0000 7fa8be000740 1 -- 172.21.15.18:0/2298178964 >> 172.21.15.18:0/2298178964 conn(0x55bde75bb990 msgr2=0x55bde75c5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.843+0000 7fa8be000740 1 -- 172.21.15.18:0/2298178964 shutdown_connections 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.843+0000 7fa8be000740 1 -- 172.21.15.18:0/2298178964 wait complete. 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8be000740 1 -- start start 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8be000740 1 Processor -- start 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8be000740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c12b0 0x55bde75ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8be000740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 0x55bde75ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8be000740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 0x55bde75cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8be000740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde7630df0 con 0x55bde75c1bd0 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8be000740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde7630f60 con 0x55bde75ca7b0 2024-02-17T13:03:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8be000740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde7631120 con 0x55bde75c12b0 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8be000740 10 client.0 _ll_register_callbacks cb 0x7fa8ac0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8b89c9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 0x55bde75ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8b89c9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 0x55bde75ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44128/0 (socket says 172.21.15.18:44128) 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8b89c9640 1 -- 172.21.15.18:0/2145367201 learned_addr learned my addr 172.21.15.18:0/2145367201 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8b91ca640 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c12b0 0x55bde75ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8b99cb640 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 0x55bde75cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8b89c9640 1 -- 172.21.15.18:0/2145367201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c12b0 msgr2=0x55bde75ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c12b0 0x55bde75ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8b89c9640 1 -- 172.21.15.18:0/2145367201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 msgr2=0x55bde75cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 0x55bde75cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.844+0000 7fa8b89c9640 1 -- 172.21.15.18:0/2145367201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bde74e7570 con 0x55bde75c1bd0 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa89cff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:21.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa8b89c9640 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 0x55bde75ce770 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009930 tx=0x7fa8a402cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621e63f3e9f02bc6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa89cff9640 1 -- 172.21.15.18:0/2145367201 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a402c9f0 con 0x55bde75c1bd0 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa89cff9640 1 -- 172.21.15.18:0/2145367201 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa8a404e040 con 0x55bde75c1bd0 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa8be000740 1 -- 172.21.15.18:0/2145367201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8a40048a0 con 0x55bde75c1bd0 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa89cff9640 1 -- 172.21.15.18:0/2145367201 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8a402cb50 con 0x55bde75c1bd0 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa8be000740 10 client.20439 fetch_fsmap learned FSMap version 57 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa8be000740 1 -- 172.21.15.18:0/2145367201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55bde74e96a0 con 0x55bde75c1bd0 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa89cff9640 1 -- 172.21.15.18:0/2145367201 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8a40048a0 con 0x55bde75c1bd0 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa89cff9640 1 -- 172.21.15.18:0/2145367201 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 57) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fa8a4052050 con 0x55bde75c1bd0 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa8be000740 10 client.20439 fetch_fsmap finished waiting for FSMap version 57 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa8be000740 10 client.20439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa8be000740 20 client.20439 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa8be000740 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bde7632610 0x55bde7634a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa8be000740 4 client.20439 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:03:21.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.845+0000 7fa8be000740 1 -- 172.21.15.18:0/2145367201 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bde72b4890 con 0x55bde7632610 2024-02-17T13:03:21.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.846+0000 7fa8b91ca640 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bde7632610 0x55bde7634a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:21.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.846+0000 7fa8b91ca640 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bde7632610 0x55bde7634a30 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=871bdb26cf3462c2 server_cookie=ecdf0dd09c8f74e0 in_seq=0 out_seq=0 2024-02-17T13:03:21.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.846+0000 7fa89cff9640 10 client.20439 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:21.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa89cff9640 1 -- 172.21.15.18:0/2145367201 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55bde72b4890 con 0x55bde7632610 2024-02-17T13:03:21.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa89cff9640 10 client.20439 handle_command_reply: tid=0 2024-02-17T13:03:21.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa8be000740 1 client.20439 shutdown 2024-02-17T13:03:21.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa8be000740 1 -- 172.21.15.18:0/2145367201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 msgr2=0x55bde75ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa8be000740 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 0x55bde75ce770 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009930 tx=0x7fa8a402cd60 comp rx=0 tx=0).stop 2024-02-17T13:03:21.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa8be000740 1 -- 172.21.15.18:0/2145367201 shutdown_connections 2024-02-17T13:03:21.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa8be000740 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55bde7632610 0x55bde7634a30 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa8be000740 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bde75ca7b0 0x55bde75cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa8be000740 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bde75c1bd0 0x55bde75ce770 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa8be000740 1 --2- 172.21.15.18:0/2145367201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde75c12b0 0x55bde75ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa8be000740 1 -- 172.21.15.18:0/2145367201 >> 172.21.15.18:0/2145367201 conn(0x55bde75bb990 msgr2=0x55bde75c6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:21.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa89cff9640 0 client.20439 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:21.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa8be000740 1 -- 172.21.15.18:0/2145367201 shutdown_connections 2024-02-17T13:03:21.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa8be000740 1 -- 172.21.15.18:0/2145367201 wait complete. 2024-02-17T13:03:21.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.850+0000 7fa8be000740 20 client.20439 trim_cache size 0 max 16384 2024-02-17T13:03:21.843 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:03:21.843 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (223 inodes in the stack)", 2024-02-17T13:03:21.843 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-02-17T13:03:21.843 INFO:teuthology.orchestra.run.smithi018.stdout: "6e3955b4-a093-4924-a887-6fea290f9ecb": { 2024-02-17T13:03:21.843 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-02-17T13:03:21.843 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "6e3955b4-a093-4924-a887-6fea290f9ecb", 2024-02-17T13:03:21.844 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-02-17T13:03:21.844 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:03:21.844 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:03:21.844 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:03:21.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.851+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3377358754 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa8a00a2230 msgr2=0x7fa8a00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.851+0000 7fa8bb42e640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa8a00a2230 0x7fa8a00a46f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009fd0 tx=0x7fa8ac009d80 comp rx=0 tx=0).stop 2024-02-17T13:03:21.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.851+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3377358754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8b4100270 msgr2=0x7fa8b4196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:21.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.851+0000 7fa8bb42e640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8b4100270 0x7fa8b4196660 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000e9f0 tx=0x7fa8b000eeb0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.851+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3377358754 shutdown_connections 2024-02-17T13:03:21.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.851+0000 7fa8bb42e640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fa8a00a2230 0x7fa8a00a46f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.851+0000 7fa8bb42e640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8b419af20 0x7fa8b419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.851+0000 7fa8bb42e640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b4105340 0x7fa8b4196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.851+0000 7fa8bb42e640 1 --2- 172.21.15.18:0/3377358754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8b4100270 0x7fa8b4196660 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:21.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.851+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3377358754 >> 172.21.15.18:0/3377358754 conn(0x7fa8b4077230 msgr2=0x7fa8b40fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:21.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.851+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3377358754 shutdown_connections 2024-02-17T13:03:21.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:21.851+0000 7fa8bb42e640 1 -- 172.21.15.18:0/3377358754 wait complete. 2024-02-17T13:03:21.852 INFO:tasks.cephfs.filesystem:scrub status for tag:6e3955b4-a093-4924-a887-6fea290f9ecb - {'path': '/', 'tag': '6e3955b4-a093-4924-a887-6fea290f9ecb', 'options': 'recursive,force'} 2024-02-17T13:03:22.091 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.693+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000104f [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ auth{0=1} v136 ap=1 f(v0 m2024-02-17T13:02:50.215697+0000 9=0+9) n(v4 rc2024-02-17T13:03:21.282322+0000 b15890742 70=60+10) old_inodes=1 (isnap sync r=9) (inest mix w=1 dirty) (ipolicy sync r=5) (ifile excl) (iversion lock) caps={15354=pAsLsXsFsx/pAsLsXsFsxcral@3},l=15354 | dirtyscattered=1 request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x560e3dc94100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000104f:[<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x1000000104f:[<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:02:50.215697+0000 9=0+9)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:21.282322+0000 b15890742 70=60+10)","memory_value.dirstat":"f(v0 m2024-02-17T13:02:50.215697+0000 9=0+9)","memory_value.rstat":"n(v4 rc2024-02-17T13:03:21.282322+0000 b15890742 70=60+10)","error_str":""},"return_code":0} 2024-02-17T13:03:22.091 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.694+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001063 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.001 auth v118 ap=1 s=1048576 n(v0 rc2024-02-17T13:02:48.863713+0000 b1048576 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dac2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001063:[<0x1000000104e/filler.001 v6>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001063:[<0x1000000104e/filler.001 v118>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.091 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.694+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000106c [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.002 auth v130 ap=1 s=1048576 n(v0 rc2024-02-17T13:02:48.893713+0000 b1048576 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcb2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000106c:[<0x1000000104e/filler.002 v8>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000106c:[<0x1000000104e/filler.002 v130>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.092 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.696+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001076 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.003 auth v122 ap=1 s=1048576 n(v0 rc2024-02-17T13:02:48.924712+0000 b1048576 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dbee680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001076:[<0x1000000104e/filler.003 v10>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001076:[<0x1000000104e/filler.003 v122>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.092 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.696+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001080 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.004 auth v134 ap=1 s=1048576 n(v0 rc2024-02-17T13:02:48.950712+0000 b1048576 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3deb9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001080:[<0x1000000104e/filler.004 v12>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001080:[<0x1000000104e/filler.004 v134>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.092 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.697+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001057 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.000 auth v126 ap=1 s=1048576 n(v0 rc2024-02-17T13:02:48.841713+0000 b1048576 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcae100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001057:[<0x1000000104e/filler.000 v4>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001057:[<0x1000000104e/filler.000 v126>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.092 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.697+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001101 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ auth v2501 ap=1 f(v0 m2024-02-17T13:03:14.799400+0000 11=11+0) n(v0 rc2024-02-17T13:03:14.799400+0000 b2861056 12=11+1) old_inodes=1 (inest lock) (ifile excl) (iversion lock) caps={15354=pAsLsXsFsx/pAsLsXsFsxcral@2},l=15354 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc3d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000001101:[<0x1000000104f/COREL v779>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x10000001101:[<0x1000000104f/COREL v2501>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:14.799400+0000 11=11+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:14.799400+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:14.799400+0000 11=11+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:14.799400+0000 b2861056 12=11+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.092 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.698+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010ab [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/ auth v2901 ap=2 f(v0 m2024-02-17T13:03:21.590318+0000 8=8+0) n(v0 rc2024-02-17T13:03:21.590318+0000 b608576 9=8+1) old_inodes=1 (iauth sync r=1) (isnap sync r=1) (inest lock w=1) (ipolicy sync r=1) (ifile excl w=1) (iversion lock) caps={15354=pAsLsXsFsxc/pAsLsXsFsxcral@132},l=15354 | request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x560e3da57700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000010ab:[<0x1000000104f/EXCEL v1113>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x100000010ab:[<0x1000000104f/EXCEL v2901>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:21.590318+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:21.590318+0000 b608576 9=8+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:21.590318+0000 8=8+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:21.590318+0000 b608576 9=8+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.092 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.699+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001088 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/ auth v2697 ap=3 f(v0 m2024-02-17T13:03:21.000325+0000 5=5+0) n(v0 rc2024-02-17T13:03:21.000325+0000 b780288 6=5+1) old_inodes=1 (iauth sync r=1) (isnap sync r=2) (inest lock w=2) (ipolicy sync r=1) (ifile excl w=2) (iversion lock) caps={15354=pAsLsXsFsxcr/pAsLsXsFsxcral@89},l=15354 | request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x560e3da80680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000001088:[<0x1000000104f/WORD v971>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x10000001088:[<0x1000000104f/WORD v2697>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:21.000325+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:21.000325+0000 b780288 6=5+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:21.000325+0000 5=5+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:21.000325+0000 b780288 6=5+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.092 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.700+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001282 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/ auth v2635 ap=1 f(v0 m2024-02-17T13:02:50.272696+0000 3=3+0) n(v0 rc2024-02-17T13:03:12.596426+0000 b3244032 4=3+1) old_inodes=1 (inest lock) (ifile excl) (iversion lock) caps={15354=pAsLsXsFsx/pAsLsXsFsxcral@2},l=15354 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3da7f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000001282:[<0x1000000104f/SEED v120>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x10000001282:[<0x1000000104f/SEED v2635>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:02:50.272696+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:12.596426+0000 b3244032 4=3+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:02:50.272696+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:12.596426+0000 b3244032 4=3+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.092 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.700+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000116a [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/ auth v2933 pv2935 ap=5 f(v0 m2024-02-17T13:03:21.681317+0000 7=7+0)->f(v0 m2024-02-17T13:03:21.689317+0000 6=6+0) n(v0 rc2024-02-17T13:03:21.683317+0000 b803406 8=7+1)->n(v0 rc2024-02-17T13:03:21.689317+0000 b792831 7=6+1) old_inodes=1 (isnap sync r=2) (inest lock w=1) (ifile excl->sync w=1) (iversion lock w=1 last_client=15354) caps={15354=pAsLsXsFs/pAsLsXsFsxcral@45},l=15354 | request=1 lock=4 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x560e3c7c3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000116a:[<0x1000000104f/WORDPRO v791>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x1000000116a:[<0x1000000104f/WORDPRO v2933>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:21.681317+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:21.683317+0000 b803406 8=7+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:21.681317+0000 7=7+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:21.683317+0000 b803406 8=7+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.092 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.700+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011b2 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ auth v2813 ap=3 f(v0 m2024-02-17T13:03:21.299321+0000 24=24+0) n(v0 rc2024-02-17T13:03:21.299321+0000 b5201920 25=24+1) old_inodes=1 (iauth sync r=1) (isnap sync r=2) (inest lock w=2) (ipolicy sync r=1) (ifile excl w=2) (iversion lock) caps={15354=pAsLsXsFsxcr/pAsLsXsFsxcral@154},l=15354 | request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x560e3da55600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000011b2:[<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x100000011b2:[<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:21.299321+0000 24=24+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:21.299321+0000 b5201920 25=24+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:21.299321+0000 24=24+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:21.299321+0000 b5201920 25=24+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.092 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.701+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000118e [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/ auth v2481 ap=1 f(v0 m2024-02-17T13:03:14.903399+0000 3=3+0) n(v0 rc2024-02-17T13:03:14.903399+0000 b780288 4=3+1) old_inodes=1 (inest lock) (ifile excl) (iversion lock) caps={15354=pAsLsXsFsx/pAsLsXsFsxcral@2},l=15354 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3da81180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000118e:[<0x1000000104f/PM v793>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x1000000118e:[<0x1000000104f/PM v2481>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:14.903399+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:14.903399+0000 b780288 4=3+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:14.903399+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:14.903399+0000 b780288 4=3+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.101 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.702+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010c6 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/ auth v2629 ap=3 f(v0 m2024-02-17T13:03:20.839327+0000 7=7+0) n(v0 rc2024-02-17T13:03:20.839327+0000 b1462582 8=7+1) old_inodes=1 (iauth sync r=1) (isnap sync r=2) (inest lock w=2) (ipolicy sync r=1) (ifile excl w=2) (iversion lock) caps={15354=pAsLsXsFsxcr/pAsLsXsFsxcral@87},l=15354 | request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x560e3dcd2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000010c6:[<0x1000000104f/PWRPNT v1079>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x100000010c6:[<0x1000000104f/PWRPNT v2629>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:20.839327+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:20.839327+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:20.839327+0000 7=7+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:20.839327+0000 b1462582 8=7+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.101 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.704+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001276 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ACCESS/ auth v2201 ap=1 f(v0 m2024-02-17T13:03:15.192395+0000 1=1+0) n(v0 rc2024-02-17T13:03:15.192395+0000 b260096 2=1+1) old_inodes=1 (inest lock) (ifile excl) (iversion lock) caps={15354=pAsLsXsFsx/pAsLsXsFsxcral@2},l=15354 | importingcaps=0 dirfrag=1 caps=1 replicated=0 authpin=1 scrubqueue=0 0x560e3d8e6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000001276:[<0x1000000104f/ACCESS v807>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x10000001276:[<0x1000000104f/ACCESS v2201>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:15.192395+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:15.192395+0000 b260096 2=1+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:15.192395+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:15.192395+0000 b260096 2=1+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.101 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.704+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001159 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ROSE.CPT auth v254 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.448706+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3d9cf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001159:[<0x10000001101/ROSE.CPT v20>,<0x1000000104f/COREL v779>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001159:[<0x10000001101/ROSE.CPT v254>,<0x1000000104f/COREL v2501>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.101 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.705+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001137 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/TEAPOT2.CPT auth v246 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.357707+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3d949700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001137:[<0x10000001101/TEAPOT2.CPT v12>,<0x1000000104f/COREL v779>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001137:[<0x10000001101/TEAPOT2.CPT v246>,<0x1000000104f/COREL v2501>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.101 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.706+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000112e [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/MAILBOX2.CPT auth v262 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.330707+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc20580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000112e:[<0x10000001101/MAILBOX2.CPT v10>,<0x1000000104f/COREL v779>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000112e:[<0x10000001101/MAILBOX2.CPT v262>,<0x1000000104f/COREL v2501>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.101 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.706+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001164 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CORELDRW.CDT auth v258 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.481706+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3da7f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001164:[<0x10000001101/CORELDRW.CDT v22>,<0x1000000104f/COREL v779>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001164:[<0x10000001101/CORELDRW.CDT v258>,<0x1000000104f/COREL v2501>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.101 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.707+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001152 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/GRAPHIC2.CDR auth v270 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.428706+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc92000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001152:[<0x10000001101/GRAPHIC2.CDR v18>,<0x1000000104f/COREL v779>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001152:[<0x10000001101/GRAPHIC2.CDR v270>,<0x1000000104f/COREL v2501>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.101 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.707+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000111b [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ARTISTIC.ACL auth v266 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.286708+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc21b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000111b:[<0x10000001101/ARTISTIC.ACL v6>,<0x1000000104f/COREL v779>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000111b:[<0x10000001101/ARTISTIC.ACL v266>,<0x1000000104f/COREL v2501>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.101 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.707+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001140 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/SIGN2.CPT auth v278 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.380707+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dee0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001140:[<0x10000001101/SIGN2.CPT v14>,<0x1000000104f/COREL v779>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001140:[<0x10000001101/SIGN2.CPT v278>,<0x1000000104f/COREL v2501>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.101 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.707+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000110e [16,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CDRBARS.CFG auth v238 ap=1 s=260096 n(v0 rc2024-02-17T13:02:55.331635+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15354 | ptrwaiter=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcd3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000110e:[<0x10000001101/CDRBARS.CFG v4>,<0x1000000104f/COREL v779>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000110e:[<0x10000001101/CDRBARS.CFG v238>,<0x1000000104f/COREL v2501>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.102 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.708+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001149 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/TRAIN.CDR auth v274 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.404707+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc93080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001149:[<0x10000001101/TRAIN.CDR v16>,<0x1000000104f/COREL v779>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001149:[<0x10000001101/TRAIN.CDR v274>,<0x1000000104f/COREL v2501>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.102 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.708+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001125 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/GRAPHIC1.CDR auth v250 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.309708+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3d9ce580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001125:[<0x10000001101/GRAPHIC1.CDR v8>,<0x1000000104f/COREL v779>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001125:[<0x10000001101/GRAPHIC1.CDR v250>,<0x1000000104f/COREL v2501>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.102 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.708+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001104 [16,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CDRROLS.CFG auth v224 ap=1 s=260096 n(v0 rc2024-02-17T13:02:55.362635+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15354 | ptrwaiter=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc20b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001104:[<0x10000001101/CDRROLS.CFG v2>,<0x1000000104f/COREL v779>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001104:[<0x10000001101/CDRROLS.CFG v224>,<0x1000000104f/COREL v2501>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.102 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.708+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010af [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/PCMAGCD.XLS auth v300 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.889459+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@24},l=15354 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dee3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010af:[<0x100000010ab/PCMAGCD.XLS v2>,<0x1000000104f/EXCEL v1113>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010af:[<0x100000010ab/PCMAGCD.XLS v300>,<0x1000000104f/EXCEL v2901>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.102 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.709+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010ba [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/SALES.XLS auth v362 ap=2 s=260096 n(v0 rc2024-02-17T13:03:10.505452+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@24},l=15354 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcb3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010ba:[<0x100000010ab/SALES.XLS v4>,<0x1000000104f/EXCEL v1113>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010ba:[<0x100000010ab/SALES.XLS v362>,<0x1000000104f/EXCEL v2901>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.102 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.710+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001058 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ auth{0=1} v132 ap=1 f(v0 m2024-02-17T13:02:50.262696+0000 9=0+9) n(v3 rc2024-02-17T13:03:21.355321+0000 b15890742 67=57+10) old_inodes=1 (isnap sync r=6) (inest mix dirty) (ipolicy sync r=3) (ifile excl) (iversion lock) caps={15354=pAsLsXsFsx/pAsLsXsFsxcral@3},l=15354 | dirtyscattered=1 request=0 lock=2 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x560e3dc8e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000001058:[<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x10000001058:[<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:02:50.262696+0000 9=0+9)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:21.355321+0000 b15890742 67=57+10)","memory_value.dirstat":"f(v0 m2024-02-17T13:02:50.262696+0000 9=0+9)","memory_value.rstat":"n(v3 rc2024-02-17T13:03:21.355321+0000 b15890742 67=57+10)","error_str":""},"return_code":0} 2024-02-17T13:03:22.102 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.712+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000106d [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.001 auth v114 ap=1 s=1048576 n(v0 rc2024-02-17T13:02:48.894713+0000 b1048576 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3da56100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000106d:[<0x10000001055/filler.001 v6>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000106d:[<0x10000001055/filler.001 v114>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.102 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.713+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001077 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.002 auth v122 ap=1 s=1048576 n(v0 rc2024-02-17T13:02:48.925712+0000 b1048576 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3db0ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001077:[<0x10000001055/filler.002 v8>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001077:[<0x10000001055/filler.002 v122>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.102 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.713+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001091 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.004 auth v118 ap=1 s=1048576 n(v0 rc2024-02-17T13:02:48.989712+0000 b1048576 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3da96000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001091:[<0x10000001055/filler.004 v12>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001091:[<0x10000001055/filler.004 v118>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.102 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.713+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001081 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.003 auth v126 ap=1 s=1048576 n(v0 rc2024-02-17T13:02:48.950712+0000 b1048576 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc16b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001081:[<0x10000001055/filler.003 v10>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001081:[<0x10000001055/filler.003 v126>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.102 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.714+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001064 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.000 auth v130 ap=1 s=1048576 n(v0 rc2024-02-17T13:02:48.866713+0000 b1048576 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcb1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001064:[<0x10000001055/filler.000 v4>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001064:[<0x10000001055/filler.000 v130>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.103 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.714+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001095 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/BASEMACH.DOC auth v232 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.002711+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@19},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3d9d0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001095:[<0x10000001088/BASEMACH.DOC v4>,<0x1000000104f/WORD v971>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001095:[<0x10000001088/BASEMACH.DOC v232>,<0x1000000104f/WORD v2697>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.103 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.714+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010a5 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/FACTS.DOC auth v244 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.029711+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@19},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dee2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010a5:[<0x10000001088/FACTS.DOC v6>,<0x1000000104f/WORD v971>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010a5:[<0x10000001088/FACTS.DOC v244>,<0x1000000104f/WORD v2697>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.103 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.715+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000108c [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/CHAP10.DOC auth v216 ap=1 s=260096 n(v0 rc2024-02-17T13:02:48.975712+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@19},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc8a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000108c:[<0x10000001088/CHAP10.DOC v2>,<0x1000000104f/WORD v971>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000108c:[<0x10000001088/CHAP10.DOC v216>,<0x1000000104f/WORD v2697>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.103 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.716+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001285 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/SMALL.FIL auth v14 ap=2 s=327680 n(v0 rc2024-02-17T13:03:12.596426+0000 b327680 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc20000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001285:[<0x10000001282/SMALL.FIL v2>,<0x1000000104f/SEED v120>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001285:[<0x10000001282/SMALL.FIL v14>,<0x1000000104f/SEED v2635>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.103 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.716+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001290 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/MEDIUM.FIL auth v16 ap=2 s=1048576 n(v0 rc2024-02-17T13:03:12.596426+0000 b1048576 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3d9d0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001290:[<0x10000001282/MEDIUM.FIL v4>,<0x1000000104f/SEED v120>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001290:[<0x10000001282/MEDIUM.FIL v16>,<0x1000000104f/SEED v2635>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.103 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.717+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000129d [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/LARGE.FIL auth v18 ap=2 s=1867776 n(v0 rc2024-02-17T13:03:12.596426+0000 b1867776 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc4ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000129d:[<0x10000001282/LARGE.FIL v6>,<0x1000000104f/SEED v120>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000129d:[<0x10000001282/LARGE.FIL v18>,<0x1000000104f/SEED v2635>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.103 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.717+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000116f [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/BENCHS.LWP auth v302 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.512705+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@14},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc94680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000116f:[<0x1000000116a/BENCHS.LWP v2>,<0x1000000104f/WORDPRO v791>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000116f:[<0x1000000116a/BENCHS.LWP v302>,<0x1000000104f/WORDPRO v2941>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.103 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.718+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001177 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/RESULTS.XLS auth v316 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.532705+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@14},l=15354 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc1db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001177:[<0x1000000116a/RESULTS.XLS v4>,<0x1000000104f/WORDPRO v791>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001177:[<0x1000000116a/RESULTS.XLS v316>,<0x1000000104f/WORDPRO v2941>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.103 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.718+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001186 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/NEWS1_1.LWP auth v340 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.559705+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@14},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3e32ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001186:[<0x1000000116a/NEWS1_1.LWP v6>,<0x1000000104f/WORDPRO v791>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001186:[<0x1000000116a/NEWS1_1.LWP v340>,<0x1000000104f/WORDPRO v2941>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.103 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.719+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011bf [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.PX auth v432 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.594463+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dbedb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011bf:[<0x100000011b2/COURSES.PX v4>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011bf:[<0x100000011b2/COURSES.PX v432>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.103 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.720+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011cb [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.X04 auth v434 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.594463+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc3ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011cb:[<0x100000011b2/COURSES.X04 v6>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011cb:[<0x100000011b2/COURSES.X04 v434>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.103 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.721+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011fa [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG0 auth v466 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.845701+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dbef700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011fa:[<0x100000011b2/STUDENTS.YG0 v16>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011fa:[<0x100000011b2/STUDENTS.YG0 v466>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.104 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.722+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001232 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.PX auth v478 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.015699+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3d8e7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001232:[<0x100000011b2/ENROLL.PX v28>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001232:[<0x100000011b2/ENROLL.PX v478>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.104 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.722+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011e8 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.PX auth v462 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.592463+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc3e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011e8:[<0x100000011b2/STUDENTS.PX v12>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011e8:[<0x100000011b2/STUDENTS.PX v462>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.104 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.723+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001258 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.TV auth v528 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.124698+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dac2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001258:[<0x100000011b2/STUDENTS.TV v36>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001258:[<0x100000011b2/STUDENTS.TV v528>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.104 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.723+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001263 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.FSL auth v530 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.152698+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc18100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001263:[<0x100000011b2/STUDENTS.FSL v38>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001263:[<0x100000011b2/STUDENTS.FSL v530>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.104 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.723+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000120d [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG1 auth v470 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.897701+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3e328b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000120d:[<0x100000011b2/STUDENTS.YG1 v20>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000120d:[<0x100000011b2/STUDENTS.YG1 v470>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.104 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.723+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001245 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.Y03 auth v482 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.067699+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcb0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001245:[<0x100000011b2/ENROLL.Y03 v32>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001245:[<0x100000011b2/ENROLL.Y03 v482>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.104 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.723+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000126e [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.FSL auth v532 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.181697+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc18680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000126e:[<0x100000011b2/COURSES.FSL v40>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000126e:[<0x100000011b2/COURSES.FSL v532>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.104 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.723+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001216 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.VAL auth v472 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.923700+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3e32a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001216:[<0x100000011b2/STUDENTS.VAL v22>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001216:[<0x100000011b2/STUDENTS.VAL v472>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.104 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.724+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011b7 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.DB auth v430 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.594463+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3d9cf080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011b7:[<0x100000011b2/COURSES.DB v2>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011b7:[<0x100000011b2/COURSES.DB v430>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.104 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.724+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011d6 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.Y04 auth v436 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.594463+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3d73b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011d6:[<0x100000011b2/COURSES.Y04 v8>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011d6:[<0x100000011b2/COURSES.Y04 v436>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.104 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.724+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000123c [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.X03 auth v480 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.041699+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3debac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000123c:[<0x100000011b2/ENROLL.X03 v30>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000123c:[<0x100000011b2/ENROLL.X03 v480>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.105 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.724+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001220 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.MB auth v474 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.957700+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcac000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001220:[<0x100000011b2/STUDENTS.MB v24>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001220:[<0x100000011b2/STUDENTS.MB v474>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.105 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.724+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000124e [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.VAL auth v484 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.097698+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc91700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000124e:[<0x100000011b2/ENROLL.VAL v34>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000124e:[<0x100000011b2/ENROLL.VAL v484>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.105 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.724+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001229 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.DB auth v476 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.989700+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3d9d1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001229:[<0x100000011b2/ENROLL.DB v26>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001229:[<0x100000011b2/ENROLL.DB v476>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.105 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.725+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001204 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG1 auth v468 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.871701+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dac1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001204:[<0x100000011b2/STUDENTS.XG1 v18>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001204:[<0x100000011b2/STUDENTS.XG1 v468>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.105 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.725+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011f1 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG0 auth v464 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.819702+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcd3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011f1:[<0x100000011b2/STUDENTS.XG0 v14>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011f1:[<0x100000011b2/STUDENTS.XG0 v464>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.105 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.725+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011df [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.DB auth v460 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.590463+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3debb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011df:[<0x100000011b2/STUDENTS.DB v10>,<0x1000000104f/PARADOX v1049>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011df:[<0x100000011b2/STUDENTS.DB v460>,<0x1000000104f/PARADOX v2813>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.105 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.725+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001193 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/T1.XLS auth v466 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.587704+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dbeec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001193:[<0x1000000118e/T1.XLS v2>,<0x1000000104f/PM v793>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001193:[<0x1000000118e/T1.XLS v466>,<0x1000000104f/PM v2481>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.105 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.726+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000119b [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/EVENTS.DOC auth v470 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.607704+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc22680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000119b:[<0x1000000118e/EVENTS.DOC v4>,<0x1000000104f/PM v793>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000119b:[<0x1000000118e/EVENTS.DOC v470>,<0x1000000104f/PM v2481>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.105 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.726+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011aa [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/MOVED.DOC auth v474 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.634704+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcccb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011aa:[<0x1000000118e/MOVED.DOC v6>,<0x1000000104f/PM v793>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011aa:[<0x1000000118e/MOVED.DOC v474>,<0x1000000104f/PM v2481>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.105 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.726+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010ca [16,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/PCBENCHM.PPT auth v362 ap=1 s=260096 n(v0 rc2024-02-17T13:02:57.958603+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@15},l=15354 | ptrwaiter=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dac2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010ca:[<0x100000010c6/PCBENCHM.PPT v2>,<0x1000000104f/PWRPNT v1079>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010ca:[<0x100000010c6/PCBENCHM.PPT v362>,<0x1000000104f/PWRPNT v2629>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.105 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.726+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010d6 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/ZD16.BMP auth v408 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.139710+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFscr/-@14} | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3deb9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010d6:[<0x100000010c6/ZD16.BMP v4>,<0x1000000104f/PWRPNT v1079>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010d6:[<0x100000010c6/ZD16.BMP v408>,<0x1000000104f/PWRPNT v2629>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.105 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.727+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010fa [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/TRIDOTS.POT auth v484 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.212709+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@19},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dbbf080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010fa:[<0x100000010c6/TRIDOTS.POT v10>,<0x1000000104f/PWRPNT v1079>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010fa:[<0x100000010c6/TRIDOTS.POT v484>,<0x1000000104f/PWRPNT v2629>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.727+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010f0 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/TIPS.PPT auth v456 ap=2 s=260096 n(v0 rc2024-02-17T13:03:07.509488+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@15},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3da56c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010f0:[<0x100000010c6/TIPS.PPT v8>,<0x1000000104f/PWRPNT v1079>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010f0:[<0x100000010c6/TIPS.PPT v456>,<0x1000000104f/PWRPNT v2629>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.728+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010e4 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/PPTOOLS1.PPA auth v430 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.162709+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15354 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcd2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010e4:[<0x100000010c6/PPTOOLS1.PPA v6>,<0x1000000104f/PWRPNT v1079>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010e4:[<0x100000010c6/PPTOOLS1.PPA v430>,<0x1000000104f/PWRPNT v2629>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.728+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000127b [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ACCESS/FASTENER.MDB auth v112 ap=1 s=260096 n(v0 rc2024-02-17T13:03:11.849435+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x560e3d8e6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000127b:[<0x10000001276/FASTENER.MDB v12>,<0x1000000104f/ACCESS v807>,<0x1000000104e/~dmtmp v56>,<0x10000000007/client3 v1766>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000127b:[<0x10000001276/FASTENER.MDB v112>,<0x1000000104f/ACCESS v2201>,<0x1000000104e/~dmtmp v136>,<0x10000000007/client3 v2094>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.729+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001111 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ auth{0=1} v2511 ap=1 f(v0 m2024-02-17T13:03:14.768400+0000 11=11+0) n(v0 rc2024-02-17T13:03:14.768400+0000 b2861056 12=11+1) old_inodes=1 (inest mix) (ifile excl) (iversion lock) caps={15354=pAsLsXsFsx/pAsLsXsFsxcral@2},l=15354 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x560e3deb8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000001111:[<0x10000001058/COREL v781>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x10000001111:[<0x10000001058/COREL v2511>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:14.768400+0000 11=11+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:14.768400+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:14.768400+0000 11=11+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:14.768400+0000 b2861056 12=11+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.729+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010c0 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/ auth{0=1} v2369 pv2813 ap=2 f(v0 m2024-02-17T13:03:14.623402+0000 2=2+0)->f(v0 m2024-02-17T13:03:14.623402+0000 2=2+0) n(v0 rc2024-02-17T13:03:14.623402+0000 b520192 3=2+1)->n(v0 rc2024-02-17T13:03:14.623402+0000 b520192 3=2+1) old_inodes=1 (inest mix w=1) (ifile excl) (iversion lock w=1) caps={15354=pAsLsXsFsx/pAsLsXsFsxcral@11},l=15354 | request=0 lock=2 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x560e3dc1f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000010c0:[<0x10000001058/EXCEL v1115>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x100000010c0:[<0x10000001058/EXCEL v2369>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:14.623402+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:14.623402+0000 b520192 3=2+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:14.623402+0000 2=2+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:14.623402+0000 b520192 3=2+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.731+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001096 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/ auth{0=1} v2695 ap=3 f(v0 m2024-02-17T13:03:21.183323+0000 5=5+0) n(v0 rc2024-02-17T13:03:21.183323+0000 b780288 6=5+1) old_inodes=1 (iauth sync r=1) (isnap sync r=2) (inest mix w=2) (ipolicy sync r=1) (ifile excl w=2) (iversion lock) caps={15354=pAsLsXsFsxcr/pAsLsXsFsxcral@79},l=15354 | request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x560e3d8e7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000001096:[<0x10000001058/WORD v973>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x10000001096:[<0x10000001058/WORD v2695>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:21.183323+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:21.183323+0000 b780288 6=5+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:21.183323+0000 5=5+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:21.183323+0000 b780288 6=5+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.733+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001298 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/ auth v2633 ap=1 f(v0 m2024-02-17T13:02:50.316696+0000 3=3+0) n(v0 rc2024-02-17T13:03:12.541427+0000 b3244032 4=3+1) old_inodes=1 (inest lock) (ifile excl) (iversion lock) caps={15354=pAsLsXsFsx/pAsLsXsFsxcral@2},l=15354 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dee1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000001298:[<0x10000001058/SEED v120>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x10000001298:[<0x10000001058/SEED v2633>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:02:50.316696+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:12.541427+0000 b3244032 4=3+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:02:50.316696+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:12.541427+0000 b3244032 4=3+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.733+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000117f [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/ auth{0=1} v2515 ap=1 f(v0 m2024-02-17T13:03:14.851399+0000 3=3+0) n(v0 rc2024-02-17T13:03:14.851399+0000 b780288 4=3+1) old_inodes=1 (inest mix) (ifile excl) (iversion lock) caps={15354=pAsLsXsFsx/pAsLsXsFsxcral@2},l=15354 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x560e3dbef180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000117f:[<0x10000001058/WORDPRO v793>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x1000000117f:[<0x10000001058/WORDPRO v2515>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:14.851399+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:14.851399+0000 b780288 4=3+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:14.851399+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:14.851399+0000 b780288 4=3+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.734+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011c7 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ auth{0=1} v2811 ap=3 f(v0 m2024-02-17T13:03:21.664317+0000 24=24+0) n(v0 rc2024-02-17T13:03:21.664317+0000 b5201920 25=24+1) old_inodes=1 (iauth sync r=1) (isnap sync r=2) (inest mix w=2) (ipolicy sync r=1) (ifile excl w=2) (iversion lock) caps={15354=pAsLsXsFsxcr/pAsLsXsFsxcral@149},l=15354 | request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x560e3d948680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000011c7:[<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x100000011c7:[<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:21.664317+0000 24=24+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:21.664317+0000 b5201920 25=24+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:21.664317+0000 24=24+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:21.664317+0000 b5201920 25=24+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.735+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011a3 [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/ auth{0=1} v2487 ap=1 f(v0 m2024-02-17T13:03:14.876399+0000 3=3+0) n(v0 rc2024-02-17T13:03:14.876399+0000 b780288 4=3+1) old_inodes=1 (inest mix) (ifile excl) (iversion lock) caps={15354=pAsLsXsFsx/pAsLsXsFsxcral@2},l=15354 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x560e3dc90c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000011a3:[<0x10000001058/PM v795>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x100000011a3:[<0x10000001058/PM v2487>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:14.876399+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:14.876399+0000 b780288 4=3+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:14.876399+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:14.876399+0000 b780288 4=3+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.735+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010da [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/ auth{0=1} v2627 ap=3 f(v0 m2024-02-17T13:03:20.913326+0000 7=7+0) n(v0 rc2024-02-17T13:03:20.913326+0000 b1462582 8=7+1) old_inodes=1 (iauth sync r=1) (isnap sync r=2) (inest mix w=2) (ipolicy sync r=1) (ifile excl w=2) (iversion lock) caps={15354=pAsLsXsFsxcr/pAsLsXsFsxcral@78},l=15354 | request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x560e3dc1d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000010da:[<0x10000001058/PWRPNT v1079>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x100000010da:[<0x10000001058/PWRPNT v2627>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:20.913326+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:20.913326+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:20.913326+0000 7=7+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:20.913326+0000 b1462582 8=7+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.735+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000128b [...18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ACCESS/ auth{0=1} v2201 ap=1 f(v0 m2024-02-17T13:03:15.201395+0000 1=1+0) n(v0 rc2024-02-17T13:03:15.201395+0000 b260096 2=1+1) old_inodes=1 (inest lock) (ifile excl) (iversion lock) caps={15354=pAsLsXsFsx/pAsLsXsFsxcral@2},l=15354 | importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x560e3deb9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000128b:[<0x10000001058/ACCESS v809>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(2)0x1000000128b:[<0x10000001058/ACCESS v2201>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2098>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-17T13:03:15.201395+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2024-02-17T13:03:15.201395+0000 b260096 2=1+1)","memory_value.dirstat":"f(v0 m2024-02-17T13:03:15.201395+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-02-17T13:03:15.201395+0000 b260096 2=1+1)","error_str":""},"return_code":0} 2024-02-17T13:03:22.106 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.736+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001169 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ROSE.CPT auth v262 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.505705+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3db0c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001169:[<0x10000001111/ROSE.CPT v20>,<0x10000001058/COREL v781>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001169:[<0x10000001111/ROSE.CPT v262>,<0x10000001058/COREL v2511>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.107 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.736+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001143 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/TEAPOT2.CPT auth v274 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.392707+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcad080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001143:[<0x10000001111/TEAPOT2.CPT v12>,<0x10000001058/COREL v781>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001143:[<0x10000001111/TEAPOT2.CPT v274>,<0x10000001058/COREL v2511>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.107 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.736+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000113a [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/MAILBOX2.CPT auth v266 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.367707+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc1e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000113a:[<0x10000001111/MAILBOX2.CPT v10>,<0x10000001058/COREL v781>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000113a:[<0x10000001111/MAILBOX2.CPT v266>,<0x10000001058/COREL v2511>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.107 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.737+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001176 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CORELDRW.CDT auth v278 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.531705+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3e328000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001176:[<0x10000001111/CORELDRW.CDT v22>,<0x10000001058/COREL v781>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001176:[<0x10000001111/CORELDRW.CDT v278>,<0x10000001058/COREL v2511>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.107 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.737+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000115f [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/GRAPHIC2.CDR auth v270 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.470706+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc1ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000115f:[<0x10000001111/GRAPHIC2.CDR v18>,<0x10000001058/COREL v781>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000115f:[<0x10000001111/GRAPHIC2.CDR v270>,<0x10000001058/COREL v2511>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.107 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.737+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001127 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ARTISTIC.ACL auth v258 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.319708+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dac1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001127:[<0x10000001111/ARTISTIC.ACL v6>,<0x10000001058/COREL v781>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001127:[<0x10000001111/ARTISTIC.ACL v258>,<0x10000001058/COREL v2511>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.107 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.737+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000111e [16,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CDRBARS.CFG auth v224 ap=1 s=260096 n(v0 rc2024-02-17T13:02:55.338635+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15354 | ptrwaiter=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc1c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000111e:[<0x10000001111/CDRBARS.CFG v4>,<0x10000001058/COREL v781>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000111e:[<0x10000001111/CDRBARS.CFG v224>,<0x10000001058/COREL v2511>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.107 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.738+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000114c [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/SIGN2.CPT auth v250 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.415706+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3d9d0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000114c:[<0x10000001111/SIGN2.CPT v14>,<0x10000001058/COREL v781>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000114c:[<0x10000001111/SIGN2.CPT v250>,<0x10000001058/COREL v2511>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.107 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.738+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001155 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/TRAIN.CDR auth v254 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.440706+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3da96580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001155:[<0x10000001111/TRAIN.CDR v16>,<0x10000001058/COREL v781>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001155:[<0x10000001111/TRAIN.CDR v254>,<0x10000001058/COREL v2511>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.107 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.738+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001131 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/GRAPHIC1.CDR auth v246 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.341707+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3d9ce000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001131:[<0x10000001111/GRAPHIC1.CDR v8>,<0x10000001058/COREL v781>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001131:[<0x10000001111/GRAPHIC1.CDR v246>,<0x10000001058/COREL v2511>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.107 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.744+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010c4 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/PCMAGCD.XLS auth v300 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.909459+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3e329600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010c4:[<0x100000010c0/PCMAGCD.XLS v2>,<0x10000001058/EXCEL v1115>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010c4:[<0x100000010c0/PCMAGCD.XLS v300>,<0x10000001058/EXCEL v2813>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.108 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.745+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001115 [16,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CDRROLS.CFG auth v242 ap=1 s=260096 n(v0 rc2024-02-17T13:02:55.379634+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15354 | ptrwaiter=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3e32b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001115:[<0x10000001111/CDRROLS.CFG v2>,<0x10000001058/COREL v781>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001115:[<0x10000001111/CDRROLS.CFG v242>,<0x10000001058/COREL v2511>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.108 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.745+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010d1 [16,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/SALES.XLS auth v296 ap=1 s=260096 n(v0 rc2024-02-17T13:02:53.513657+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15354 | ptrwaiter=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3da80100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010d1:[<0x100000010c0/SALES.XLS v4>,<0x10000001058/EXCEL v1115>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010d1:[<0x100000010c0/SALES.XLS v296>,<0x10000001058/EXCEL v2813>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.108 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.745+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010a9 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/BASEMACH.DOC auth v232 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.040711+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@19},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3db0cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010a9:[<0x10000001096/BASEMACH.DOC v4>,<0x10000001058/WORD v973>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010a9:[<0x10000001096/BASEMACH.DOC v232>,<0x10000001058/WORD v2695>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.108 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.745+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010b8 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/FACTS.DOC auth v244 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.089710+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@19},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc92b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010b8:[<0x10000001096/FACTS.DOC v6>,<0x10000001058/WORD v973>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010b8:[<0x10000001096/FACTS.DOC v244>,<0x10000001058/WORD v2695>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.108 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.747+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000109b [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/CHAP10.DOC auth v216 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.010711+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@19},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcd2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000109b:[<0x10000001096/CHAP10.DOC v2>,<0x10000001058/WORD v973>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000109b:[<0x10000001096/CHAP10.DOC v216>,<0x10000001058/WORD v2695>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.108 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.748+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000129a [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/SMALL.FIL auth v12 ap=2 s=327680 n(v0 rc2024-02-17T13:03:12.540427+0000 b327680 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc95180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000129a:[<0x10000001298/SMALL.FIL v2>,<0x10000001058/SEED v120>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000129a:[<0x10000001298/SMALL.FIL v12>,<0x10000001058/SEED v2633>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.108 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.748+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012a4 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/MEDIUM.FIL auth v14 ap=2 s=1048576 n(v0 rc2024-02-17T13:03:12.540427+0000 b1048576 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3da98c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000012a4:[<0x10000001298/MEDIUM.FIL v4>,<0x10000001058/SEED v120>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000012a4:[<0x10000001298/MEDIUM.FIL v14>,<0x10000001058/SEED v2633>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.108 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.749+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012ab [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/LARGE.FIL auth v16 ap=2 s=1867776 n(v0 rc2024-02-17T13:03:12.541427+0000 b1867776 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@15} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc3cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000012ab:[<0x10000001298/LARGE.FIL v6>,<0x10000001058/SEED v120>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000012ab:[<0x10000001298/LARGE.FIL v16>,<0x10000001058/SEED v2633>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.108 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.749+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001185 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP auth v296 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.558705+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc93b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001185:[<0x1000000117f/BENCHS.LWP v2>,<0x10000001058/WORDPRO v793>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001185:[<0x1000000117f/BENCHS.LWP v296>,<0x10000001058/WORDPRO v2515>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.108 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.749+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000118d [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/RESULTS.XLS auth v300 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.580704+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3e329080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000118d:[<0x1000000117f/RESULTS.XLS v4>,<0x10000001058/WORDPRO v793>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000118d:[<0x1000000117f/RESULTS.XLS v300>,<0x10000001058/WORDPRO v2515>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.108 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.749+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000119a [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/NEWS1_1.LWP auth v292 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.606704+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc8f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000119a:[<0x1000000117f/NEWS1_1.LWP v6>,<0x10000001058/WORDPRO v793>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000119a:[<0x1000000117f/NEWS1_1.LWP v292>,<0x10000001058/WORDPRO v2515>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.108 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.749+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011d4 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.PX auth v434 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.595463+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3e328580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011d4:[<0x100000011c7/COURSES.PX v4>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011d4:[<0x100000011c7/COURSES.PX v434>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.109 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.750+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011dd [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.X04 auth v436 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.595463+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcb3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011dd:[<0x100000011c7/COURSES.X04 v6>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011dd:[<0x100000011c7/COURSES.X04 v436>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.109 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.750+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000120a [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.YG0 auth v468 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.888701+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcb1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000120a:[<0x100000011c7/STUDENTS.YG0 v16>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000120a:[<0x100000011c7/STUDENTS.YG0 v468>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.109 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.750+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001241 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.PX auth v480 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.056699+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dbbfb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001241:[<0x100000011c7/ENROLL.PX v28>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001241:[<0x100000011c7/ENROLL.PX v480>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.109 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.750+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011f8 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.PX auth v464 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.594463+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dbc0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011f8:[<0x100000011c7/STUDENTS.PX v12>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011f8:[<0x100000011c7/STUDENTS.PX v464>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.109 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.751+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001269 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.TV auth v530 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.172697+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dbed080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001269:[<0x100000011c7/STUDENTS.TV v36>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001269:[<0x100000011c7/STUDENTS.TV v530>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.109 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.751+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001274 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.FSL auth v532 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.199697+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcd1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001274:[<0x100000011c7/STUDENTS.FSL v38>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001274:[<0x100000011c7/STUDENTS.FSL v532>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.109 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.751+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000121d [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.YG1 auth v472 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.947700+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc89080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000121d:[<0x100000011c7/STUDENTS.YG1 v20>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000121d:[<0x100000011c7/STUDENTS.YG1 v472>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.109 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.755+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001254 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.Y03 auth v484 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.115698+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc90680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001254:[<0x100000011c7/ENROLL.Y03 v32>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001254:[<0x100000011c7/ENROLL.Y03 v484>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.109 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.756+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001280 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.FSL auth v534 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.219697+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3d9ceb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001280:[<0x100000011c7/COURSES.FSL v40>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001280:[<0x100000011c7/COURSES.FSL v534>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.109 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.757+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001226 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.VAL auth v474 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.981700+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc8b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001226:[<0x100000011c7/STUDENTS.VAL v22>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001226:[<0x100000011c7/STUDENTS.VAL v474>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.109 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.759+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011ca [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.DB auth v432 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.595463+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3da96b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011ca:[<0x100000011c7/COURSES.DB v2>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011ca:[<0x100000011c7/COURSES.DB v432>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.110 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.760+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011e6 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.Y04 auth v438 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.596463+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3e32a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011e6:[<0x100000011c7/COURSES.Y04 v8>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011e6:[<0x100000011c7/COURSES.Y04 v438>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.110 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.761+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000124a [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.X03 auth v482 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.086698+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc8eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000124a:[<0x100000011c7/ENROLL.X03 v30>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000124a:[<0x100000011c7/ENROLL.X03 v482>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.110 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.762+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000122f [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.MB auth v476 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.006699+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcd0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000122f:[<0x100000011c7/STUDENTS.MB v24>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000122f:[<0x100000011c7/STUDENTS.MB v476>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.110 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.762+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000125e [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.VAL auth v486 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.142698+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc95700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000125e:[<0x100000011c7/ENROLL.VAL v34>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000125e:[<0x100000011c7/ENROLL.VAL v486>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.110 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.763+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001238 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.DB auth v478 ap=1 s=260096 n(v0 rc2024-02-17T13:02:50.032699+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3da99700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001238:[<0x100000011c7/ENROLL.DB v26>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001238:[<0x100000011c7/ENROLL.DB v478>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.110 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.763+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001213 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.XG1 auth v470 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.916700+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dccc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001213:[<0x100000011c7/STUDENTS.XG1 v18>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001213:[<0x100000011c7/STUDENTS.XG1 v470>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.110 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.763+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011ef [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.DB auth v462 ap=2 s=260096 n(v0 rc2024-02-17T13:03:09.591463+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@26},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcce100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011ef:[<0x100000011c7/STUDENTS.DB v10>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011ef:[<0x100000011c7/STUDENTS.DB v462>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.110 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.763+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011a9 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/T1.XLS auth v474 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.633704+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dccc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011a9:[<0x100000011a3/T1.XLS v2>,<0x10000001058/PM v795>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011a9:[<0x100000011a3/T1.XLS v474>,<0x10000001058/PM v2487>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.110 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.763+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011be [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/MOVED.DOC auth v466 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.680703+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc92580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011be:[<0x100000011a3/MOVED.DOC v6>,<0x10000001058/PM v795>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011be:[<0x100000011a3/MOVED.DOC v466>,<0x10000001058/PM v2487>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.110 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.764+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000011b1 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/EVENTS.DOC auth v470 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.653704+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFsc/-@2},l=15354(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc93600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000011b1:[<0x100000011a3/EVENTS.DOC v4>,<0x10000001058/PM v795>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000011b1:[<0x100000011a3/EVENTS.DOC v470>,<0x10000001058/PM v2487>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.110 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.764+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001201 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.XG0 auth v466 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.862701+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@29},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dccd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001201:[<0x100000011c7/STUDENTS.XG0 v14>,<0x10000001058/PARADOX v1051>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001201:[<0x100000011c7/STUDENTS.XG0 v466>,<0x10000001058/PARADOX v2811>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.111 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.764+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010e8 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/ZD16.BMP auth v408 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.174709+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFscr/-@15} | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dac3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010e8:[<0x100000010da/ZD16.BMP v4>,<0x10000001058/PWRPNT v1079>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010e8:[<0x100000010da/ZD16.BMP v408>,<0x10000001058/PWRPNT v2627>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.111 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.765+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010de [16,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/PCBENCHM.PPT auth v360 ap=1 s=260096 n(v0 rc2024-02-17T13:02:57.956603+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@16},l=15354 | ptrwaiter=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dac1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010de:[<0x100000010da/PCBENCHM.PPT v2>,<0x10000001058/PWRPNT v1079>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010de:[<0x100000010da/PCBENCHM.PPT v360>,<0x10000001058/PWRPNT v2627>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.111 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.768+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001108 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/TRIDOTS.POT auth v484 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.255708+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@20},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dcb2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001108:[<0x100000010da/TRIDOTS.POT v10>,<0x10000001058/PWRPNT v1079>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x10000001108:[<0x100000010da/TRIDOTS.POT v484>,<0x10000001058/PWRPNT v2627>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.111 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.768+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010fc [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/TIPS.PPT auth v456 ap=2 s=260096 n(v0 rc2024-02-17T13:03:07.510488+0000 b260096 1=1+0) need_snapflush={16=15354} (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@16} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@16},l=15354 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 needsnapflush=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc1d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010fc:[<0x100000010da/TIPS.PPT v8>,<0x10000001058/PWRPNT v1079>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010fc:[<0x100000010da/TIPS.PPT v456>,<0x10000001058/PWRPNT v2627>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.111 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.768+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000010f2 [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/PPTOOLS1.PPA auth v430 ap=1 s=260096 n(v0 rc2024-02-17T13:02:49.199709+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15354=0-4194304@17} caps={15354=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=15354 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 needsnapflush=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560e3dc1f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000010f2:[<0x100000010da/PPTOOLS1.PPA v6>,<0x10000001058/PWRPNT v1079>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x100000010f2:[<0x100000010da/PPTOOLS1.PPA v430>,<0x10000001058/PWRPNT v2627>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.111 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:03:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:03:21.768+0000 7f44ccbff700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000128e [18,head] /volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ACCESS/FASTENER.MDB auth v112 ap=1 s=260096 n(v0 rc2024-02-17T13:03:11.912435+0000 b260096 1=1+0) (iversion lock) caps={15354=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x560e3dcae680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000128e:[<0x1000000128b/FASTENER.MDB v12>,<0x10000001058/ACCESS v809>,<0x10000001055/~dmtmp v56>,<0x10000000007/client6 v1770>,<0x10000000006/clients v1349>,<0x10000000005/tmp v1303>,<0x100000001fb/client.0 v1291>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1272>,<0x10000000001/sv_1 v1279>,<0x10000000000/qa v1259>,<0x1/volumes v1203>]//[]","memoryvalue":"(3)0x1000000128e:[<0x1000000128b/FASTENER.MDB v112>,<0x10000001058/ACCESS v2201>,<0x10000001055/~dmtmp v132>,<0x10000000007/client6 v2352>,<0x10000000006/clients v1398>,<0x10000000005/tmp v1342>,<0x100000001fb/client.0 v1317>,<0x100000001fa/323825cb-6c90-4285-a584-8b73e60121df v1260>,<0x10000000001/sv_1 v1308>,<0x10000000000/qa v1287>,<0x1/volumes v1229>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-17T13:03:22.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: pgmap v424: 97 pgs: 97 active+clean; 479 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 21 MiB/s wr, 538 op/s 2024-02-17T13:03:22.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: scrub queued for path: / 2024-02-17T13:03:22.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:22.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:03:22.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:03:22.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000104f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001063 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.001) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000106c (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.002) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001076 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.003) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001080 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.004) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001057 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.000) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001101 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010ab (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001088 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001282 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000116a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011b2 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000118e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010c6 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001276 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ACCESS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001159 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001137 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000112e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001164 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001152 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000111b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001140 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000110e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001149 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001125 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001104 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010af (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010ba (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001058 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000106d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.001) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001077 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.002) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001091 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.004) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001081 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.003) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001064 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.000) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001095 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/BASEMACH.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010a5 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/FACTS.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000108c (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/CHAP10.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001285 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001290 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000129d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000116f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001177 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001186 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011bf (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011cb (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011fa (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001232 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011e8 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001258 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001263 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000120d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001245 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000126e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001216 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011b7 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011d6 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000123c (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001220 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.703 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000124e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001229 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001204 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011f1 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011df (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001193 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/T1.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000119b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/EVENTS.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011aa (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/MOVED.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010ca (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010d6 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010fa (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010f0 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010e4 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.704 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000127b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ACCESS/FASTENER.MDB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001111 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010c0 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001096 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001298 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000117f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011c7 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011a3 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010da (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000128b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ACCESS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001169 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001143 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000113a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001176 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.705 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000115f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.706 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001127 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.706 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000111e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.706 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000114c (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.706 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001155 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001131 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010c4 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001115 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010d1 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010a9 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/BASEMACH.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010b8 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/FACTS.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000109b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/CHAP10.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000129a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000012a4 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000012ab (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001185 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000118d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000119a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.707 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011d4 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011dd (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000120a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001241 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011f8 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001269 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001274 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000121d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001254 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001280 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001226 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011ca (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011e6 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000124a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.708 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000122f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000125e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001238 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001213 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011ef (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011a9 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/T1.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011be (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/MOVED.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000011b1 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/EVENTS.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001201 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010e8 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010de (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x10000001108 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010fc (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x100000010f2 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.709 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: Scrub error on inode 0x1000000128e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ACCESS/FASTENER.MDB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.710 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:22.710 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:03:22.710 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:22 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:22.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: pgmap v424: 97 pgs: 97 active+clean; 479 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 21 MiB/s wr, 538 op/s 2024-02-17T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: scrub queued for path: / 2024-02-17T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000104f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001063 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.001) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000106c (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.002) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001076 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.003) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001080 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.004) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001057 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.000) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001101 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010ab (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001088 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001282 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000116a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011b2 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000118e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010c6 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001276 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ACCESS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001159 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001137 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000112e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001164 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001152 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000111b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001140 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000110e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001149 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001125 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001104 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010af (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010ba (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001058 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000106d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.001) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001077 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.002) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001091 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.004) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001081 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.003) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001064 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.000) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001095 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/BASEMACH.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010a5 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/FACTS.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000108c (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/CHAP10.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001285 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001290 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000129d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000116f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001177 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001186 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011bf (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011cb (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011fa (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.776 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001232 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011e8 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001258 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001263 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000120d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001245 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000126e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001216 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011b7 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011d6 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000123c (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001220 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000124e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001229 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001204 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.777 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011f1 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011df (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001193 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/T1.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000119b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/EVENTS.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011aa (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/MOVED.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010ca (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010d6 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010fa (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010f0 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010e4 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000127b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ACCESS/FASTENER.MDB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001111 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010c0 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001096 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.778 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001298 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000117f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011c7 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011a3 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010da (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000128b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ACCESS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001169 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001143 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000113a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001176 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.779 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000115f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001127 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000111e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000114c (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001155 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001131 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010c4 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001115 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010d1 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010a9 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/BASEMACH.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010b8 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/FACTS.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.780 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000109b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/CHAP10.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000129a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000012a4 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000012ab (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001185 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000118d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000119a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011d4 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011dd (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000120a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001241 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011f8 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001269 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001274 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000121d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001254 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001280 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001226 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011ca (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011e6 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000124a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000122f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000125e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001238 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001213 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011ef (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011a9 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/T1.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011be (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/MOVED.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000011b1 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/EVENTS.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001201 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010e8 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010de (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x10000001108 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010fc (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x100000010f2 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: Scrub error on inode 0x1000000128e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ACCESS/FASTENER.MDB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:22.784 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:03:22.784 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:22 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:22.786 INFO:tasks.workunit.client.0.smithi018.stdout: 10 14297 28.01 MB/sec warmup 34 sec latency 2667.091 ms 2024-02-17T13:03:22.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: pgmap v424: 97 pgs: 97 active+clean; 479 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 21 MiB/s wr, 538 op/s 2024-02-17T13:03:22.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: scrub queued for path: / 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000104f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001063 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.001) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000106c (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.002) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001076 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.003) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001080 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.004) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001057 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.000) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001101 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010ab (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001088 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001282 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000116a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011b2 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000118e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010c6 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001276 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ACCESS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001159 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001137 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000112e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001164 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001152 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000111b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001140 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000110e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001149 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001125 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001104 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010af (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010ba (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001058 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000106d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.001) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001077 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.002) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001091 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.004) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001081 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.003) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001064 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.000) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001095 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/BASEMACH.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010a5 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/FACTS.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000108c (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/CHAP10.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001285 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001290 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000129d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000116f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001177 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001186 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011bf (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011cb (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011fa (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001232 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011e8 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001258 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001263 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000120d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001245 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000126e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001216 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011b7 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011d6 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000123c (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001220 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000124e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001229 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001204 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011f1 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011df (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001193 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/T1.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000119b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/EVENTS.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011aa (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/MOVED.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010ca (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010d6 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010fa (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010f0 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010e4 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000127b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ACCESS/FASTENER.MDB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001111 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010c0 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001096 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001298 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000117f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011c7 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011a3 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010da (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000128b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ACCESS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001169 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001143 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000113a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001176 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000115f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001127 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000111e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000114c (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001155 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001131 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010c4 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001115 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010d1 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010a9 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/BASEMACH.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010b8 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/FACTS.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000109b (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/CHAP10.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000129a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000012a4 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000012ab (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001185 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000118d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000119a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011d4 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011dd (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000120a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001241 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011f8 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001269 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001274 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000121d (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001254 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001280 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001226 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011ca (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011e6 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000124a (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000122f (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000125e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001238 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001213 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011ef (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.851 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011a9 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/T1.XLS) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011be (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/MOVED.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000011b1 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/EVENTS.DOC) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001201 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010e8 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010de (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x10000001108 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010fc (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x100000010f2 (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: Scrub error on inode 0x1000000128e (/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ACCESS/FASTENER.MDB) see mds.h log and `damage ls` output for details 2024-02-17T13:03:22.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:22.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:03:22.852 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:22 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:23.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:23 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:03:23.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:23 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:23.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:23 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:03:23.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:23 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:03:23.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:23 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:23.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:23 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:23.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:23 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:03:23.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:23 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:23.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:23 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:03:23.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:23 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:23.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:23 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:03:23.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:23 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:03:23.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:23 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:23.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:23 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:23.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:23 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:03:23.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:23 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:23.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 14771 27.66 MB/sec warmup 35 sec latency 3667.120 ms 2024-02-17T13:03:23.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:23 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:03:23.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:23 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:23.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:23 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:03:23.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:23 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:03:23.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:23 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:23.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:23 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:23.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:23 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:03:23.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:23 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:24.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:24 smithi146 ceph-mon[36454]: pgmap v425: 97 pgs: 97 active+clean; 479 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 21 MiB/s wr, 538 op/s 2024-02-17T13:03:24.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:24 smithi018 ceph-mon[32625]: pgmap v425: 97 pgs: 97 active+clean; 479 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 21 MiB/s wr, 538 op/s 2024-02-17T13:03:24.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:24 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a[32621]: 2024-02-17T13:03:24.405+0000 7f9b678e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-17T13:03:24.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 15457 27.96 MB/sec warmup 36 sec latency 4667.150 ms 2024-02-17T13:03:24.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:24 smithi119 ceph-mon[36188]: pgmap v425: 97 pgs: 97 active+clean; 479 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 21 MiB/s wr, 538 op/s 2024-02-17T13:03:25.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:25 smithi146 ceph-mon[36454]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-17T13:03:25.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:25 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] up:active 2024-02-17T13:03:25.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:25 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:25.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:25 smithi018 ceph-mon[32625]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-17T13:03:25.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:25 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] up:active 2024-02-17T13:03:25.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:25 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:25.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 16307 28.13 MB/sec warmup 37 sec latency 5667.184 ms 2024-02-17T13:03:25.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:25 smithi119 ceph-mon[36188]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-17T13:03:25.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:25 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] up:active 2024-02-17T13:03:25.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:25 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:26.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:26 smithi146 ceph-mon[36454]: pgmap v426: 97 pgs: 97 active+clean; 494 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 20 MiB/s wr, 614 op/s 2024-02-17T13:03:26.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:26 smithi018 ceph-mon[32625]: pgmap v426: 97 pgs: 97 active+clean; 494 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 20 MiB/s wr, 614 op/s 2024-02-17T13:03:26.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 17024 28.98 MB/sec warmup 38 sec latency 6667.231 ms 2024-02-17T13:03:26.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:26 smithi119 ceph-mon[36188]: pgmap v426: 97 pgs: 97 active+clean; 494 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 20 MiB/s wr, 614 op/s 2024-02-17T13:03:27.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 17457 28.87 MB/sec warmup 39 sec latency 7667.263 ms 2024-02-17T13:03:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.680+0000 7f576df8c640 1 -- 172.21.15.18:0/4038743713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57681002d0 msgr2=0x7f5768100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.680+0000 7f576df8c640 1 --2- 172.21.15.18:0/4038743713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57681002d0 0x7f5768100750 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f575c009900 tx=0x7f575c02bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.680+0000 7f576df8c640 1 -- 172.21.15.18:0/4038743713 shutdown_connections 2024-02-17T13:03:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.680+0000 7f576df8c640 1 --2- 172.21.15.18:0/4038743713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5768100c90 0x7f576810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.680+0000 7f576df8c640 1 --2- 172.21.15.18:0/4038743713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57681002d0 0x7f5768100750 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.680+0000 7f576df8c640 1 --2- 172.21.15.18:0/4038743713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5768105020 0x7f5768105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.680+0000 7f576df8c640 1 -- 172.21.15.18:0/4038743713 >> 172.21.15.18:0/4038743713 conn(0x7f5768076ed0 msgr2=0x7f57680772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.680+0000 7f576df8c640 1 -- 172.21.15.18:0/4038743713 shutdown_connections 2024-02-17T13:03:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.680+0000 7f576df8c640 1 -- 172.21.15.18:0/4038743713 wait complete. 2024-02-17T13:03:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f576df8c640 1 Processor -- start 2024-02-17T13:03:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f576df8c640 1 -- start start 2024-02-17T13:03:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f576df8c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57681002d0 0x7f576810eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f576df8c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5768100c90 0x7f576810f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f576df8c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5768105020 0x7f576810f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f576df8c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57681a5b30 con 0x7f57681002d0 2024-02-17T13:03:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f576df8c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57681a5ca0 con 0x7f5768100c90 2024-02-17T13:03:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f576df8c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57681a5e60 con 0x7f5768105020 2024-02-17T13:03:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f57677fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57681002d0 0x7f576810eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f57677fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57681002d0 0x7f576810eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44134/0 (socket says 172.21.15.18:44134) 2024-02-17T13:03:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f57677fe640 1 -- 172.21.15.18:0/2127936992 learned_addr learned my addr 172.21.15.18:0/2127936992 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f5767fff640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5768105020 0x7f576810f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.681+0000 7f5766ffd640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5768100c90 0x7f576810f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.682+0000 7f57677fe640 1 -- 172.21.15.18:0/2127936992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5768105020 msgr2=0x7f576810f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.682+0000 7f57677fe640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5768105020 0x7f576810f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.682+0000 7f57677fe640 1 -- 172.21.15.18:0/2127936992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5768100c90 msgr2=0x7f576810f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.682+0000 7f57677fe640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5768100c90 0x7f576810f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.682+0000 7f57677fe640 1 -- 172.21.15.18:0/2127936992 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c009560 con 0x7f57681002d0 2024-02-17T13:03:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.682+0000 7f57677fe640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57681002d0 0x7f576810eb90 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f575800e9f0 tx=0x7f575800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.682+0000 7f5764ff9640 1 -- 172.21.15.18:0/2127936992 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758009700 con 0x7f57681002d0 2024-02-17T13:03:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.682+0000 7f5764ff9640 1 -- 172.21.15.18:0/2127936992 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f575800cd80 con 0x7f57681002d0 2024-02-17T13:03:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.682+0000 7f576df8c640 1 -- 172.21.15.18:0/2127936992 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57681a6140 con 0x7f57681002d0 2024-02-17T13:03:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.682+0000 7f5764ff9640 1 -- 172.21.15.18:0/2127936992 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758018920 con 0x7f57681002d0 2024-02-17T13:03:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.682+0000 7f576df8c640 1 -- 172.21.15.18:0/2127936992 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57681a6620 con 0x7f57681002d0 2024-02-17T13:03:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.684+0000 7f5764ff9640 1 -- 172.21.15.18:0/2127936992 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5758018a80 con 0x7f57681002d0 2024-02-17T13:03:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f57707fd740 1 Processor -- start 2024-02-17T13:03:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f57707fd740 1 -- start start 2024-02-17T13:03:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f57707fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564844348ad0 0x564844347820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f57707fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844347d60 0x5648443481e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f57707fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648443512e0 0x5648443536d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f57707fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648442b69f0 con 0x5648443512e0 2024-02-17T13:03:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f57707fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648442b3f20 con 0x564844348ad0 2024-02-17T13:03:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f57707fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648442af700 con 0x564844347d60 2024-02-17T13:03:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f5764ff9640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f57500a22d0 0x7f57500a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f5766ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648443512e0 0x5648443536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f5767fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564844348ad0 0x564844347820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f5766ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648443512e0 0x5648443536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44142/0 (socket says 172.21.15.18:44142) 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f5766ffd640 1 -- 172.21.15.18:0/2728419754 learned_addr learned my addr 172.21.15.18:0/2728419754 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f5767fff640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f57500a22d0 0x7f57500a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f5764ff9640 1 -- 172.21.15.18:0/2127936992 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5758014070 con 0x7f57681002d0 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f57677fe640 1 --2- 172.21.15.18:0/2728419754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844347d60 0x5648443481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f5767fff640 1 -- 172.21.15.18:0/2728419754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844347d60 msgr2=0x5648443481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f5767fff640 1 --2- 172.21.15.18:0/2728419754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844347d60 0x5648443481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.685+0000 7f5767fff640 1 -- 172.21.15.18:0/2728419754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648443512e0 msgr2=0x5648443536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.686+0000 7f5767fff640 1 --2- 172.21.15.18:0/2728419754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648443512e0 0x5648443536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.686+0000 7f5767fff640 1 -- 172.21.15.18:0/2728419754 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648442457b0 con 0x564844348ad0 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.686+0000 7f5766ffd640 1 --2- 172.21.15.18:0/2728419754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648443512e0 0x5648443536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.686+0000 7f5767fff640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f57500a22d0 0x7f57500a4790 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f576000fc90 tx=0x7f576001e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.686+0000 7f5767fff640 1 --2- 172.21.15.18:0/2728419754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564844348ad0 0x564844347820 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5760005fd0 tx=0x7f576000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c5a7923de25c553 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.686+0000 7f574dffb640 1 -- 172.21.15.18:0/2728419754 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760011550 con 0x564844348ad0 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.686+0000 7f574dffb640 1 -- 172.21.15.18:0/2728419754 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5760009d90 con 0x564844348ad0 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.686+0000 7f574dffb640 1 -- 172.21.15.18:0/2728419754 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760004050 con 0x564844348ad0 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.687+0000 7f57707fd740 1 -- 172.21.15.18:0/2728419754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564844348ad0 msgr2=0x564844347820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.687+0000 7f57707fd740 1 --2- 172.21.15.18:0/2728419754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564844348ad0 0x564844347820 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5760005fd0 tx=0x7f576000f790 comp rx=0 tx=0).stop 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.687+0000 7f57707fd740 1 -- 172.21.15.18:0/2728419754 shutdown_connections 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.687+0000 7f57707fd740 1 --2- 172.21.15.18:0/2728419754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648443512e0 0x5648443536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.687+0000 7f57707fd740 1 --2- 172.21.15.18:0/2728419754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844347d60 0x5648443481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.687+0000 7f57707fd740 1 --2- 172.21.15.18:0/2728419754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564844348ad0 0x564844347820 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.687+0000 7f57707fd740 1 -- 172.21.15.18:0/2728419754 >> 172.21.15.18:0/2728419754 conn(0x56484430f550 msgr2=0x564844342a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.687+0000 7f57707fd740 1 -- 172.21.15.18:0/2728419754 shutdown_connections 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.687+0000 7f57707fd740 1 -- 172.21.15.18:0/2728419754 wait complete. 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.687+0000 7f57707fd740 1 -- start start 2024-02-17T13:03:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.687+0000 7f57707fd740 1 Processor -- start 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f57707fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564844347d60 0x56484434e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f57707fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 0x56484434e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f57707fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5648443512e0 0x56484434edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f57707fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564844254bf0 con 0x564844348ad0 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f57707fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564843f83250 con 0x564844347d60 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f57707fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x564844235200 con 0x5648443512e0 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f57707fd740 10 client.0 _ll_register_callbacks cb 0x5648442a3d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5766ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 0x56484434e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5766ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 0x56484434e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44156/0 (socket says 172.21.15.18:44156) 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f57677fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564844347d60 0x56484434e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5766ffd640 1 -- 172.21.15.18:0/3754682054 learned_addr learned my addr 172.21.15.18:0/3754682054 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5767fff640 1 --2- 172.21.15.18:0/3754682054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5648443512e0 0x56484434edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5766ffd640 1 -- 172.21.15.18:0/3754682054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5648443512e0 msgr2=0x56484434edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5766ffd640 1 --2- 172.21.15.18:0/3754682054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5648443512e0 0x56484434edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5766ffd640 1 -- 172.21.15.18:0/3754682054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564844347d60 msgr2=0x56484434e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5766ffd640 1 --2- 172.21.15.18:0/3754682054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564844347d60 0x56484434e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5766ffd640 1 -- 172.21.15.18:0/3754682054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5760019040 con 0x564844348ad0 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5766ffd640 1 --2- 172.21.15.18:0/3754682054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 0x56484434e870 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f575c035000 tx=0x7f575c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46101b69feb0791b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5746ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5746ffd640 1 -- 172.21.15.18:0/3754682054 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c004a90 con 0x564844348ad0 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f5746ffd640 1 -- 172.21.15.18:0/3754682054 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f575c004260 con 0x564844348ad0 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.688+0000 7f57707fd740 1 -- 172.21.15.18:0/3754682054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564843faf600 con 0x564844348ad0 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.689+0000 7f5746ffd640 1 -- 172.21.15.18:0/3754682054 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f575c004420 con 0x564844348ad0 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.689+0000 7f57707fd740 10 client.20451 fetch_fsmap learned FSMap version 58 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.689+0000 7f57707fd740 1 -- 172.21.15.18:0/3754682054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5648442add80 con 0x564844348ad0 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.689+0000 7f5746ffd640 1 -- 172.21.15.18:0/3754682054 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564843faf600 con 0x564844348ad0 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.689+0000 7f5746ffd640 1 -- 172.21.15.18:0/3754682054 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 58) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f575c03c040 con 0x564844348ad0 2024-02-17T13:03:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.689+0000 7f57707fd740 10 client.20451 fetch_fsmap finished waiting for FSMap version 58 2024-02-17T13:03:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.689+0000 7f57707fd740 10 client.20451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.689+0000 7f57707fd740 20 client.20451 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.689+0000 7f57707fd740 1 --2- 172.21.15.18:0/3754682054 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564844361670 0x564844363a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.689+0000 7f57707fd740 4 client.20451 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.689+0000 7f57707fd740 1 -- 172.21.15.18:0/3754682054 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648440001b0 con 0x564844361670 2024-02-17T13:03:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.689+0000 7f57677fe640 1 --2- 172.21.15.18:0/3754682054 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564844361670 0x564844363a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.690+0000 7f57677fe640 1 --2- 172.21.15.18:0/3754682054 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564844361670 0x564844363a90 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f072b66c364e80 server_cookie=7d53b3870f120478 in_seq=0 out_seq=0 2024-02-17T13:03:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.690+0000 7f5746ffd640 10 client.20451 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.691+0000 7f5746ffd640 1 -- 172.21.15.18:0/3754682054 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648440001b0 con 0x564844361670 2024-02-17T13:03:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.691+0000 7f5746ffd640 10 client.20451 handle_command_reply: tid=0 2024-02-17T13:03:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.691+0000 7f57707fd740 1 client.20451 shutdown 2024-02-17T13:03:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.691+0000 7f57707fd740 1 -- 172.21.15.18:0/3754682054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 msgr2=0x56484434e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.691+0000 7f57707fd740 1 --2- 172.21.15.18:0/3754682054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 0x56484434e870 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f575c035000 tx=0x7f575c0048f0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.691+0000 7f57707fd740 1 -- 172.21.15.18:0/3754682054 shutdown_connections 2024-02-17T13:03:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.691+0000 7f57707fd740 1 --2- 172.21.15.18:0/3754682054 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x564844361670 0x564844363a90 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.691+0000 7f57707fd740 1 --2- 172.21.15.18:0/3754682054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5648443512e0 0x56484434edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.691+0000 7f57707fd740 1 --2- 172.21.15.18:0/3754682054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 0x56484434e870 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.691+0000 7f5746ffd640 0 client.20451 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.691+0000 7f57707fd740 1 --2- 172.21.15.18:0/3754682054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564844347d60 0x56484434e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.691+0000 7f57707fd740 1 -- 172.21.15.18:0/3754682054 >> 172.21.15.18:0/3754682054 conn(0x56484430f550 msgr2=0x5648443447a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.692+0000 7f57707fd740 1 -- 172.21.15.18:0/3754682054 shutdown_connections 2024-02-17T13:03:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.692+0000 7f57707fd740 1 -- 172.21.15.18:0/3754682054 wait complete. 2024-02-17T13:03:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.692+0000 7f57707fd740 20 client.20451 trim_cache size 0 max 16384 2024-02-17T13:03:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.706+0000 7f57707fd740 1 Processor -- start 2024-02-17T13:03:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.706+0000 7f57707fd740 1 -- start start 2024-02-17T13:03:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.706+0000 7f57707fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844347d60 0x564844350a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.706+0000 7f57707fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844348ad0 0x56484434ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.706+0000 7f57707fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648443512e0 0x56484434b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.706+0000 7f57707fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x564844342310 con 0x564844347d60 2024-02-17T13:03:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.706+0000 7f57707fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56484435e130 con 0x5648443512e0 2024-02-17T13:03:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.706+0000 7f57707fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56484433e310 con 0x564844348ad0 2024-02-17T13:03:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f57677fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844347d60 0x564844350a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f57677fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844347d60 0x564844350a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44160/0 (socket says 172.21.15.18:44160) 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f57677fe640 1 -- 172.21.15.18:0/2378846169 learned_addr learned my addr 172.21.15.18:0/2378846169 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f5766ffd640 1 --2- 172.21.15.18:0/2378846169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844348ad0 0x56484434ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f5767fff640 1 --2- 172.21.15.18:0/2378846169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648443512e0 0x56484434b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f57677fe640 1 -- 172.21.15.18:0/2378846169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844348ad0 msgr2=0x56484434ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f57677fe640 1 --2- 172.21.15.18:0/2378846169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844348ad0 0x56484434ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f57677fe640 1 -- 172.21.15.18:0/2378846169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648443512e0 msgr2=0x56484434b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f57677fe640 1 --2- 172.21.15.18:0/2378846169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648443512e0 0x56484434b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f57677fe640 1 -- 172.21.15.18:0/2378846169 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56484424a7b0 con 0x564844347d60 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f5767fff640 1 --2- 172.21.15.18:0/2378846169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648443512e0 0x56484434b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f57677fe640 1 --2- 172.21.15.18:0/2378846169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844347d60 0x564844350a60 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f57580cc0f0 tx=0x7f5758010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd5b5169172316a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f574d7fa640 1 -- 172.21.15.18:0/2378846169 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580b7070 con 0x564844347d60 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f574d7fa640 1 -- 172.21.15.18:0/2378846169 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5758098ee0 con 0x564844347d60 2024-02-17T13:03:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.707+0000 7f574d7fa640 1 -- 172.21.15.18:0/2378846169 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575809e040 con 0x564844347d60 2024-02-17T13:03:28.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:28 smithi146 ceph-mon[36454]: pgmap v427: 97 pgs: 97 active+clean; 494 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 18 MiB/s wr, 561 op/s 2024-02-17T13:03:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.708+0000 7f57707fd740 1 -- 172.21.15.18:0/2378846169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844347d60 msgr2=0x564844350a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.708+0000 7f57707fd740 1 --2- 172.21.15.18:0/2378846169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844347d60 0x564844350a60 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f57580cc0f0 tx=0x7f5758010e50 comp rx=0 tx=0).stop 2024-02-17T13:03:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.708+0000 7f57707fd740 1 -- 172.21.15.18:0/2378846169 shutdown_connections 2024-02-17T13:03:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.708+0000 7f57707fd740 1 --2- 172.21.15.18:0/2378846169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648443512e0 0x56484434b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.708+0000 7f57707fd740 1 --2- 172.21.15.18:0/2378846169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844348ad0 0x56484434ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.708+0000 7f57707fd740 1 --2- 172.21.15.18:0/2378846169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844347d60 0x564844350a60 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.708+0000 7f57707fd740 1 -- 172.21.15.18:0/2378846169 >> 172.21.15.18:0/2378846169 conn(0x56484430f550 msgr2=0x56484434cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.708+0000 7f57707fd740 1 -- 172.21.15.18:0/2378846169 shutdown_connections 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.708+0000 7f57707fd740 1 -- 172.21.15.18:0/2378846169 wait complete. 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57707fd740 1 -- start start 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57707fd740 1 Processor -- start 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57707fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844347d60 0x564844356cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57707fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 0x564844357210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57707fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648443512e0 0x564844357750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57707fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648443b7c40 con 0x564844348ad0 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57707fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648443b7db0 con 0x5648443512e0 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57707fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648443b7f70 con 0x564844347d60 2024-02-17T13:03:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57707fd740 10 client.0 _ll_register_callbacks cb 0x7f5760009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f5766ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 0x564844357210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57677fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844347d60 0x564844356cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f5766ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 0x564844357210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44162/0 (socket says 172.21.15.18:44162) 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f5766ffd640 1 -- 172.21.15.18:0/3678316291 learned_addr learned my addr 172.21.15.18:0/3678316291 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f5767fff640 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648443512e0 0x564844357750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57677fe640 1 -- 172.21.15.18:0/3678316291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648443512e0 msgr2=0x564844357750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57677fe640 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648443512e0 0x564844357750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57677fe640 1 -- 172.21.15.18:0/3678316291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 msgr2=0x564844357210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57677fe640 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 0x564844357210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f57677fe640 1 -- 172.21.15.18:0/3678316291 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57681a6620 con 0x564844347d60 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.709+0000 7f5766ffd640 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 0x564844357210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f57677fe640 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844347d60 0x564844356cd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f57580cc0f0 tx=0x7f5758000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e21efd19b87ceff3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f5746ffd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:03:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f5746ffd640 1 -- 172.21.15.18:0/3678316291 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580b7070 con 0x564844347d60 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f57707fd740 1 -- 172.21.15.18:0/3678316291 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5760009d90 con 0x564844347d60 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f5746ffd640 1 -- 172.21.15.18:0/3678316291 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f57580974f0 con 0x564844347d60 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f5746ffd640 1 -- 172.21.15.18:0/3678316291 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f575809e050 con 0x564844347d60 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f57707fd740 10 client.29383 fetch_fsmap learned FSMap version 58 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f57707fd740 1 -- 172.21.15.18:0/3678316291 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x564844270540 con 0x564844347d60 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f5746ffd640 1 -- 172.21.15.18:0/3678316291 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5760009d90 con 0x564844347d60 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f5746ffd640 1 -- 172.21.15.18:0/3678316291 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 58) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f5758085b00 con 0x564844347d60 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f57707fd740 10 client.29383 fetch_fsmap finished waiting for FSMap version 58 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f57707fd740 10 client.29383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f57707fd740 20 client.29383 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f57707fd740 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5648443b9280 0x5648443bb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f57707fd740 4 client.29383 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.710+0000 7f57707fd740 1 -- 172.21.15.18:0/3678316291 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564843fec550 con 0x5648443b9280 2024-02-17T13:03:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.711+0000 7f5766ffd640 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5648443b9280 0x5648443bb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.711+0000 7f5766ffd640 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5648443b9280 0x5648443bb670 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7ceaada632f02fc server_cookie=c207f72729c9bdca in_seq=0 out_seq=0 2024-02-17T13:03:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:28.711+0000 7f5746ffd640 10 client.29383 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:28.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:28 smithi018 ceph-mon[32625]: pgmap v427: 97 pgs: 97 active+clean; 494 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 18 MiB/s wr, 561 op/s 2024-02-17T13:03:28.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 17918 28.70 MB/sec warmup 40 sec latency 8667.307 ms 2024-02-17T13:03:28.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:28 smithi119 ceph-mon[36188]: pgmap v427: 97 pgs: 97 active+clean; 494 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 18 MiB/s wr, 561 op/s 2024-02-17T13:03:29.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 18231 28.31 MB/sec warmup 41 sec latency 9667.339 ms 2024-02-17T13:03:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.383+0000 7f5746ffd640 1 -- 172.21.15.18:0/3678316291 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564843fec550 con 0x5648443b9280 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.383+0000 7f5746ffd640 10 client.29383 handle_command_reply: tid=0 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.383+0000 7f57707fd740 1 client.29383 shutdown 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.383+0000 7f57707fd740 1 -- 172.21.15.18:0/3678316291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844347d60 msgr2=0x564844356cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.383+0000 7f57707fd740 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844347d60 0x564844356cd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f57580cc0f0 tx=0x7f5758000bc0 comp rx=0 tx=0).stop 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.383+0000 7f57707fd740 1 -- 172.21.15.18:0/3678316291 shutdown_connections 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.383+0000 7f57707fd740 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5648443b9280 0x5648443bb670 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.383+0000 7f57707fd740 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648443512e0 0x564844357750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.383+0000 7f57707fd740 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564844348ad0 0x564844357210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.383+0000 7f57707fd740 1 --2- 172.21.15.18:0/3678316291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564844347d60 0x564844356cd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.384+0000 7f57707fd740 1 -- 172.21.15.18:0/3678316291 >> 172.21.15.18:0/3678316291 conn(0x56484430f550 msgr2=0x564844341300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.384+0000 7f5746ffd640 0 client.29383 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.384+0000 7f57707fd740 1 -- 172.21.15.18:0/3678316291 shutdown_connections 2024-02-17T13:03:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.384+0000 7f57707fd740 1 -- 172.21.15.18:0/3678316291 wait complete. 2024-02-17T13:03:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.384+0000 7f57707fd740 20 client.29383 trim_cache size 0 max 16384 2024-02-17T13:03:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:03:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:03:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:03:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:03:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.384+0000 7f576df8c640 1 -- 172.21.15.18:0/2127936992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f57500a22d0 msgr2=0x7f57500a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.384+0000 7f576df8c640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f57500a22d0 0x7f57500a4790 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f576000fc90 tx=0x7f576001e000 comp rx=0 tx=0).stop 2024-02-17T13:03:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.384+0000 7f576df8c640 1 -- 172.21.15.18:0/2127936992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57681002d0 msgr2=0x7f576810eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.384+0000 7f576df8c640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57681002d0 0x7f576810eb90 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f575800e9f0 tx=0x7f575800eeb0 comp rx=0 tx=0).stop 2024-02-17T13:03:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.385+0000 7f576df8c640 1 -- 172.21.15.18:0/2127936992 shutdown_connections 2024-02-17T13:03:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.385+0000 7f576df8c640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f57500a22d0 0x7f57500a4790 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.385+0000 7f576df8c640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5768105020 0x7f576810f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.385+0000 7f576df8c640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5768100c90 0x7f576810f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.385+0000 7f576df8c640 1 --2- 172.21.15.18:0/2127936992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57681002d0 0x7f576810eb90 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.385+0000 7f576df8c640 1 -- 172.21.15.18:0/2127936992 >> 172.21.15.18:0/2127936992 conn(0x7f5768076ed0 msgr2=0x7f57680fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.385+0000 7f576df8c640 1 -- 172.21.15.18:0/2127936992 shutdown_connections 2024-02-17T13:03:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:30.385+0000 7f576df8c640 1 -- 172.21.15.18:0/2127936992 wait complete. 2024-02-17T13:03:30.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:30 smithi146 ceph-mon[36454]: pgmap v428: 97 pgs: 97 active+clean; 511 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 20 MiB/s wr, 723 op/s 2024-02-17T13:03:30.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:30 smithi018 ceph-mon[32625]: pgmap v428: 97 pgs: 97 active+clean; 511 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 20 MiB/s wr, 723 op/s 2024-02-17T13:03:30.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 18505 27.92 MB/sec warmup 42 sec latency 10281.735 ms 2024-02-17T13:03:30.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:30 smithi119 ceph-mon[36188]: pgmap v428: 97 pgs: 97 active+clean; 511 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 20 MiB/s wr, 723 op/s 2024-02-17T13:03:31.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:31 smithi146 ceph-mon[36454]: scrub summary: active paths [/] 2024-02-17T13:03:31.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:31 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:31.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:31 smithi146 ceph-mon[36454]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:31.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:31 smithi018 ceph-mon[32625]: scrub summary: active paths [/] 2024-02-17T13:03:31.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:31 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:31.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:31 smithi018 ceph-mon[32625]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:31.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 19038 28.00 MB/sec warmup 43 sec latency 156.599 ms 2024-02-17T13:03:31.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:31 smithi119 ceph-mon[36188]: scrub summary: active paths [/] 2024-02-17T13:03:31.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:31 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:31.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:31 smithi119 ceph-mon[36188]: scrub summary: idle+waiting paths [/] 2024-02-17T13:03:32.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:32 smithi146 ceph-mon[36454]: pgmap v429: 97 pgs: 97 active+clean; 511 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-02-17T13:03:32.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:32 smithi018 ceph-mon[32625]: pgmap v429: 97 pgs: 97 active+clean; 511 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-02-17T13:03:32.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 19856 27.99 MB/sec warmup 44 sec latency 157.144 ms 2024-02-17T13:03:32.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:32 smithi119 ceph-mon[36188]: pgmap v429: 97 pgs: 97 active+clean; 511 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-02-17T13:03:33.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 20583 28.34 MB/sec warmup 45 sec latency 54.041 ms 2024-02-17T13:03:34.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:34 smithi146 ceph-mon[36454]: pgmap v430: 97 pgs: 97 active+clean; 511 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-02-17T13:03:34.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:34 smithi018 ceph-mon[32625]: pgmap v430: 97 pgs: 97 active+clean; 511 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-02-17T13:03:34.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21264 28.84 MB/sec warmup 46 sec latency 74.378 ms 2024-02-17T13:03:34.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:34 smithi119 ceph-mon[36188]: pgmap v430: 97 pgs: 97 active+clean; 511 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-02-17T13:03:35.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21734 28.78 MB/sec warmup 47 sec latency 151.601 ms 2024-02-17T13:03:36.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:36 smithi146 ceph-mon[36454]: pgmap v431: 97 pgs: 97 active+clean; 526 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.6 MiB/s wr, 752 op/s 2024-02-17T13:03:36.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:36 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:36.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:36 smithi018 ceph-mon[32625]: pgmap v431: 97 pgs: 97 active+clean; 526 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.6 MiB/s wr, 752 op/s 2024-02-17T13:03:36.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:36 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:36.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 22547 28.94 MB/sec warmup 48 sec latency 47.261 ms 2024-02-17T13:03:36.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:36 smithi119 ceph-mon[36188]: pgmap v431: 97 pgs: 97 active+clean; 526 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.6 MiB/s wr, 752 op/s 2024-02-17T13:03:36.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:36 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:37.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 23382 29.29 MB/sec warmup 49 sec latency 109.082 ms 2024-02-17T13:03:38.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:38 smithi146 ceph-mon[36454]: pgmap v432: 97 pgs: 97 active+clean; 526 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.9 MiB/s wr, 531 op/s 2024-02-17T13:03:38.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:38 smithi018 ceph-mon[32625]: pgmap v432: 97 pgs: 97 active+clean; 526 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.9 MiB/s wr, 531 op/s 2024-02-17T13:03:38.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 24287 29.54 MB/sec warmup 50 sec latency 68.694 ms 2024-02-17T13:03:38.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:38 smithi119 ceph-mon[36188]: pgmap v432: 97 pgs: 97 active+clean; 526 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.9 MiB/s wr, 531 op/s 2024-02-17T13:03:39.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 25215 30.06 MB/sec warmup 51 sec latency 70.102 ms 2024-02-17T13:03:40.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:40 smithi146 ceph-mon[36454]: pgmap v433: 97 pgs: 97 active+clean; 555 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.6 MiB/s wr, 759 op/s 2024-02-17T13:03:40.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:40 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:03:40.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:40 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:03:40.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:40 smithi018 ceph-mon[32625]: pgmap v433: 97 pgs: 97 active+clean; 555 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.6 MiB/s wr, 759 op/s 2024-02-17T13:03:40.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:40 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:03:40.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:40 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:03:40.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 25885 30.11 MB/sec warmup 52 sec latency 141.040 ms 2024-02-17T13:03:40.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:40 smithi119 ceph-mon[36188]: pgmap v433: 97 pgs: 97 active+clean; 555 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.6 MiB/s wr, 759 op/s 2024-02-17T13:03:40.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:40 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:03:40.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:40 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:03:41.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:41 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:03:41.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:41 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:03:41.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:41 smithi146 ceph-mon[36454]: scrub summary: idle 2024-02-17T13:03:41.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:41 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:41.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:41 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:03:41.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:41 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:03:41.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:41 smithi018 ceph-mon[32625]: scrub summary: idle 2024-02-17T13:03:41.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:41 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:41.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 26906 30.68 MB/sec warmup 53 sec latency 47.640 ms 2024-02-17T13:03:41.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:41 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:03:41.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:41 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:03:41.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:41 smithi119 ceph-mon[36188]: scrub summary: idle 2024-02-17T13:03:41.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:41 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:42.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:42 smithi146 ceph-mon[36454]: pgmap v434: 97 pgs: 97 active+clean; 555 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-02-17T13:03:42.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:42 smithi018 ceph-mon[32625]: pgmap v434: 97 pgs: 97 active+clean; 555 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-02-17T13:03:42.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 27841 30.97 MB/sec warmup 54 sec latency 81.275 ms 2024-02-17T13:03:42.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:42 smithi119 ceph-mon[36188]: pgmap v434: 97 pgs: 97 active+clean; 555 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-02-17T13:03:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.480+0000 7f99ab017640 1 -- 172.21.15.18:0/1809168506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99a4069980 msgr2=0x7f99a4069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.480+0000 7f99ab017640 1 --2- 172.21.15.18:0/1809168506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99a4069980 0x7f99a4069e00 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f9990009930 tx=0x7f999002bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.480+0000 7f99ab017640 1 -- 172.21.15.18:0/1809168506 shutdown_connections 2024-02-17T13:03:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.480+0000 7f99ab017640 1 --2- 172.21.15.18:0/1809168506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99a406a340 0x7f99a410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.480+0000 7f99ab017640 1 --2- 172.21.15.18:0/1809168506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99a4069980 0x7f99a4069e00 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.480+0000 7f99ab017640 1 --2- 172.21.15.18:0/1809168506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a41071b0 0x7f99a4107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.480+0000 7f99ab017640 1 -- 172.21.15.18:0/1809168506 >> 172.21.15.18:0/1809168506 conn(0x7f99a4076e50 msgr2=0x7f99a4077260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.480+0000 7f99ab017640 1 -- 172.21.15.18:0/1809168506 shutdown_connections 2024-02-17T13:03:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.480+0000 7f99ab017640 1 -- 172.21.15.18:0/1809168506 wait complete. 2024-02-17T13:03:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.480+0000 7f99ab017640 1 Processor -- start 2024-02-17T13:03:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.480+0000 7f99ab017640 1 -- start start 2024-02-17T13:03:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99ab017640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a4069980 0x7f99a41976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99ab017640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99a406a340 0x7f99a4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99ab017640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99a41071b0 0x7f99a419bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99ab017640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a419c570 con 0x7f99a406a340 2024-02-17T13:03:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99ab017640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a419c6e0 con 0x7f99a4069980 2024-02-17T13:03:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99ab017640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a419c8a0 con 0x7f99a41071b0 2024-02-17T13:03:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99a406a340 0x7f99a4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99a406a340 0x7f99a4197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47900/0 (socket says 172.21.15.18:47900) 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a3fff640 1 -- 172.21.15.18:0/2942108992 learned_addr learned my addr 172.21.15.18:0/2942108992 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a95b4640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99a41071b0 0x7f99a419bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a8db3640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a4069980 0x7f99a41976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a3fff640 1 -- 172.21.15.18:0/2942108992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99a41071b0 msgr2=0x7f99a419bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a3fff640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99a41071b0 0x7f99a419bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a3fff640 1 -- 172.21.15.18:0/2942108992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a4069980 msgr2=0x7f99a41976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a3fff640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a4069980 0x7f99a41976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a3fff640 1 -- 172.21.15.18:0/2942108992 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9990009560 con 0x7f99a406a340 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a3fff640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99a406a340 0x7f99a4197bf0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f9990009230 tx=0x7f999002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a1ffb640 1 -- 172.21.15.18:0/2942108992 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990004350 con 0x7f99a406a340 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99ab017640 1 -- 172.21.15.18:0/2942108992 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a419cb20 con 0x7f99a406a340 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a1ffb640 1 -- 172.21.15.18:0/2942108992 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99900044b0 con 0x7f99a406a340 2024-02-17T13:03:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.481+0000 7f99a1ffb640 1 -- 172.21.15.18:0/2942108992 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990003d70 con 0x7f99a406a340 2024-02-17T13:03:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.482+0000 7f99ab017640 1 -- 172.21.15.18:0/2942108992 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a41a4400 con 0x7f99a406a340 2024-02-17T13:03:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.484+0000 7f99a1ffb640 1 -- 172.21.15.18:0/2942108992 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9990004df0 con 0x7f99a406a340 2024-02-17T13:03:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.484+0000 7f99adbdf740 1 Processor -- start 2024-02-17T13:03:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.484+0000 7f99adbdf740 1 -- start start 2024-02-17T13:03:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a1ffb640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f99940a22d0 0x7f99940a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99adbdf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 0x55e09fc22820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99adbdf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc22d60 0x55e09fc231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99adbdf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc306f0 0x55e09fc32ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99adbdf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09fb919f0 con 0x55e09fc22d60 2024-02-17T13:03:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a95b4640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f99940a22d0 0x7f99940a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99adbdf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09fb8ef20 con 0x55e09fc23ad0 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a1ffb640 1 -- 172.21.15.18:0/2942108992 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f99900f00c0 con 0x7f99a406a340 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99adbdf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09fb8a700 con 0x55e09fc306f0 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc22d60 0x55e09fc231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc22d60 0x55e09fc231e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47916/0 (socket says 172.21.15.18:47916) 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a3fff640 1 -- 172.21.15.18:0/3620455769 learned_addr learned my addr 172.21.15.18:0/3620455769 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a8db3640 1 --2- 172.21.15.18:0/3620455769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 0x55e09fc22820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a95b4640 1 --2- 172.21.15.18:0/3620455769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc306f0 0x55e09fc32ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a3fff640 1 -- 172.21.15.18:0/3620455769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc306f0 msgr2=0x55e09fc32ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a3fff640 1 --2- 172.21.15.18:0/3620455769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc306f0 0x55e09fc32ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a3fff640 1 -- 172.21.15.18:0/3620455769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 msgr2=0x55e09fc22820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a3fff640 1 --2- 172.21.15.18:0/3620455769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 0x55e09fc22820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a3fff640 1 -- 172.21.15.18:0/3620455769 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e09fb207b0 con 0x55e09fc22d60 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a95b4640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f99940a22d0 0x7f99940a4790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f999c009fd0 tx=0x7f999c013b80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f99a3fff640 1 --2- 172.21.15.18:0/3620455769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc22d60 0x55e09fc231e0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f99900bcb70 tx=0x7f99900bc600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad3519883d85567 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.485+0000 7f998effd640 1 -- 172.21.15.18:0/3620455769 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99900c0d30 con 0x55e09fc22d60 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.486+0000 7f998effd640 1 -- 172.21.15.18:0/3620455769 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99900c5380 con 0x55e09fc22d60 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.486+0000 7f998effd640 1 -- 172.21.15.18:0/3620455769 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99900c4ef0 con 0x55e09fc22d60 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.486+0000 7f99adbdf740 1 -- 172.21.15.18:0/3620455769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc22d60 msgr2=0x55e09fc231e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.486+0000 7f99adbdf740 1 --2- 172.21.15.18:0/3620455769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc22d60 0x55e09fc231e0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f99900bcb70 tx=0x7f99900bc600 comp rx=0 tx=0).stop 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.486+0000 7f99adbdf740 1 -- 172.21.15.18:0/3620455769 shutdown_connections 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.486+0000 7f99adbdf740 1 --2- 172.21.15.18:0/3620455769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc306f0 0x55e09fc32ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.486+0000 7f99adbdf740 1 --2- 172.21.15.18:0/3620455769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc22d60 0x55e09fc231e0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.486+0000 7f99adbdf740 1 --2- 172.21.15.18:0/3620455769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 0x55e09fc22820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.486+0000 7f99adbdf740 1 -- 172.21.15.18:0/3620455769 >> 172.21.15.18:0/3620455769 conn(0x55e09fbea550 msgr2=0x55e09fc1da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.486+0000 7f99adbdf740 1 -- 172.21.15.18:0/3620455769 shutdown_connections 2024-02-17T13:03:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.486+0000 7f99adbdf740 1 -- 172.21.15.18:0/3620455769 wait complete. 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99adbdf740 1 -- start start 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99adbdf740 1 Processor -- start 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99adbdf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc22d60 0x55e09fc2d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99adbdf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 0x55e09fc2dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99adbdf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc2e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99adbdf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09fb2fbf0 con 0x55e09fc306f0 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99adbdf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09f85e250 con 0x55e09fc23ad0 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99adbdf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09fb10200 con 0x55e09fc22d60 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99adbdf740 10 client.0 _ll_register_callbacks cb 0x55e09fb7ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99a95b4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc2e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99a95b4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc2e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47930/0 (socket says 172.21.15.18:47930) 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99a8db3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc22d60 0x55e09fc2d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99a95b4640 1 -- 172.21.15.18:0/1633908482 learned_addr learned my addr 172.21.15.18:0/1633908482 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.487+0000 7f99a3fff640 1 --2- 172.21.15.18:0/1633908482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 0x55e09fc2dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f99a95b4640 1 -- 172.21.15.18:0/1633908482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc22d60 msgr2=0x55e09fc2d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f99a95b4640 1 --2- 172.21.15.18:0/1633908482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc22d60 0x55e09fc2d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f99a95b4640 1 -- 172.21.15.18:0/1633908482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 msgr2=0x55e09fc2dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f99a95b4640 1 --2- 172.21.15.18:0/1633908482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 0x55e09fc2dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f99a95b4640 1 -- 172.21.15.18:0/1633908482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a41a4400 con 0x55e09fc306f0 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f99a95b4640 1 --2- 172.21.15.18:0/1633908482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc2e170 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f999c00fea0 tx=0x7f999c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe5cc4bc7fb746e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f9983fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f9983fff640 1 -- 172.21.15.18:0/1633908482 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c004730 con 0x55e09fc306f0 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f9983fff640 1 -- 172.21.15.18:0/1633908482 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f999c020430 con 0x55e09fc306f0 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f99adbdf740 1 -- 172.21.15.18:0/1633908482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e09f88a600 con 0x55e09fc306f0 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f9983fff640 1 -- 172.21.15.18:0/1633908482 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f999c004890 con 0x55e09fc306f0 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f99adbdf740 10 client.20469 fetch_fsmap learned FSMap version 60 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f99adbdf740 1 -- 172.21.15.18:0/1633908482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55e09fb88d80 con 0x55e09fc306f0 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.488+0000 7f9983fff640 1 -- 172.21.15.18:0/1633908482 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e09f88a600 con 0x55e09fc306f0 2024-02-17T13:03:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.489+0000 7f9983fff640 1 -- 172.21.15.18:0/1633908482 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 60) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f999c010660 con 0x55e09fc306f0 2024-02-17T13:03:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.489+0000 7f99adbdf740 10 client.20469 fetch_fsmap finished waiting for FSMap version 60 2024-02-17T13:03:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.489+0000 7f99adbdf740 10 client.20469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.489+0000 7f99adbdf740 20 client.20469 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.489+0000 7f99adbdf740 1 --2- 172.21.15.18:0/1633908482 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e09fc408c0 0x55e09fc42ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.489+0000 7f99adbdf740 4 client.20469 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.489+0000 7f99adbdf740 1 -- 172.21.15.18:0/1633908482 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e09f8db1b0 con 0x55e09fc408c0 2024-02-17T13:03:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.489+0000 7f99a8db3640 1 --2- 172.21.15.18:0/1633908482 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e09fc408c0 0x55e09fc42ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.489+0000 7f9983fff640 10 client.20469 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.489+0000 7f99a8db3640 1 --2- 172.21.15.18:0/1633908482 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e09fc408c0 0x55e09fc42ce0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac87964d4143082 server_cookie=d5e27b0e38736993 in_seq=0 out_seq=0 2024-02-17T13:03:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.492+0000 7f9983fff640 1 -- 172.21.15.18:0/1633908482 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e09f8db1b0 con 0x55e09fc408c0 2024-02-17T13:03:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.492+0000 7f9983fff640 10 client.20469 handle_command_reply: tid=0 2024-02-17T13:03:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.492+0000 7f99adbdf740 1 client.20469 shutdown 2024-02-17T13:03:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.493+0000 7f99adbdf740 1 -- 172.21.15.18:0/1633908482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 msgr2=0x55e09fc2e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.493+0000 7f99adbdf740 1 --2- 172.21.15.18:0/1633908482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc2e170 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f999c00fea0 tx=0x7f999c004ab0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.493+0000 7f99adbdf740 1 -- 172.21.15.18:0/1633908482 shutdown_connections 2024-02-17T13:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.493+0000 7f99adbdf740 1 --2- 172.21.15.18:0/1633908482 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e09fc408c0 0x55e09fc42ce0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.493+0000 7f99adbdf740 1 --2- 172.21.15.18:0/1633908482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc2e170 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.493+0000 7f99adbdf740 1 --2- 172.21.15.18:0/1633908482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 0x55e09fc2dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.493+0000 7f9983fff640 0 client.20469 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.493+0000 7f99adbdf740 1 --2- 172.21.15.18:0/1633908482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc22d60 0x55e09fc2d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.493+0000 7f99adbdf740 1 -- 172.21.15.18:0/1633908482 >> 172.21.15.18:0/1633908482 conn(0x55e09fbea550 msgr2=0x55e09fc1f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.493+0000 7f99adbdf740 1 -- 172.21.15.18:0/1633908482 shutdown_connections 2024-02-17T13:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.493+0000 7f99adbdf740 1 -- 172.21.15.18:0/1633908482 wait complete. 2024-02-17T13:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.493+0000 7f99adbdf740 20 client.20469 trim_cache size 0 max 16384 2024-02-17T13:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.508+0000 7f99adbdf740 1 Processor -- start 2024-02-17T13:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.508+0000 7f99adbdf740 1 -- start start 2024-02-17T13:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.508+0000 7f99adbdf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc22d60 0x55e09fc2fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.508+0000 7f99adbdf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc23ad0 0x55e09fc2dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.508+0000 7f99adbdf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc2a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.508+0000 7f99adbdf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09f88a600 con 0x55e09fc306f0 2024-02-17T13:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.508+0000 7f99adbdf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09fc3d300 con 0x55e09fc22d60 2024-02-17T13:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.508+0000 7f99adbdf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09fc22030 con 0x55e09fc23ad0 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.508+0000 7f99a95b4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc2a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.508+0000 7f99a95b4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc2a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47936/0 (socket says 172.21.15.18:47936) 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f99a95b4640 1 -- 172.21.15.18:0/1480841148 learned_addr learned my addr 172.21.15.18:0/1480841148 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f99a3fff640 1 --2- 172.21.15.18:0/1480841148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc23ad0 0x55e09fc2dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f99a8db3640 1 --2- 172.21.15.18:0/1480841148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc22d60 0x55e09fc2fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f99a95b4640 1 -- 172.21.15.18:0/1480841148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc23ad0 msgr2=0x55e09fc2dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f99a95b4640 1 --2- 172.21.15.18:0/1480841148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc23ad0 0x55e09fc2dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f99a95b4640 1 -- 172.21.15.18:0/1480841148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc22d60 msgr2=0x55e09fc2fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f99a95b4640 1 --2- 172.21.15.18:0/1480841148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc22d60 0x55e09fc2fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f99a95b4640 1 -- 172.21.15.18:0/1480841148 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e09fb257b0 con 0x55e09fc306f0 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f99a3fff640 1 --2- 172.21.15.18:0/1480841148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc23ad0 0x55e09fc2dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f99a8db3640 1 --2- 172.21.15.18:0/1480841148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc22d60 0x55e09fc2fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f99a95b4640 1 --2- 172.21.15.18:0/1480841148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc2a9d0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f999c000f80 tx=0x7f999c010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5117fed0b88ac8d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f998e7fc640 1 -- 172.21.15.18:0/1480841148 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c013540 con 0x55e09fc306f0 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f998e7fc640 1 -- 172.21.15.18:0/1480841148 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f999c00fd20 con 0x55e09fc306f0 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.509+0000 7f998e7fc640 1 -- 172.21.15.18:0/1480841148 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c0296a0 con 0x55e09fc306f0 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.510+0000 7f99adbdf740 1 -- 172.21.15.18:0/1480841148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 msgr2=0x55e09fc2a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.510+0000 7f99adbdf740 1 --2- 172.21.15.18:0/1480841148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc2a9d0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f999c000f80 tx=0x7f999c010be0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.510+0000 7f99adbdf740 1 -- 172.21.15.18:0/1480841148 shutdown_connections 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.510+0000 7f99adbdf740 1 --2- 172.21.15.18:0/1480841148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc2a9d0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.510+0000 7f99adbdf740 1 --2- 172.21.15.18:0/1480841148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc23ad0 0x55e09fc2dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.510+0000 7f99adbdf740 1 --2- 172.21.15.18:0/1480841148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc22d60 0x55e09fc2fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.510+0000 7f99adbdf740 1 -- 172.21.15.18:0/1480841148 >> 172.21.15.18:0/1480841148 conn(0x55e09fbea550 msgr2=0x55e09fc2c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.510+0000 7f99adbdf740 1 -- 172.21.15.18:0/1480841148 shutdown_connections 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.510+0000 7f99adbdf740 1 -- 172.21.15.18:0/1480841148 wait complete. 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.510+0000 7f99adbdf740 1 -- start start 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.510+0000 7f99adbdf740 1 Processor -- start 2024-02-17T13:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99adbdf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc22d60 0x55e09fc35fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99adbdf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 0x55e09fc36510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99adbdf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc36a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99adbdf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09fc96ec0 con 0x55e09fc306f0 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99adbdf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09fc97030 con 0x55e09fc23ad0 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99adbdf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09fc971f0 con 0x55e09fc22d60 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99adbdf740 10 client.0 _ll_register_callbacks cb 0x7f9990003ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99a95b4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc36a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99a95b4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc36a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47944/0 (socket says 172.21.15.18:47944) 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99a95b4640 1 -- 172.21.15.18:0/473672519 learned_addr learned my addr 172.21.15.18:0/473672519 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99a8db3640 1 --2- 172.21.15.18:0/473672519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc22d60 0x55e09fc35fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99a95b4640 1 -- 172.21.15.18:0/473672519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc22d60 msgr2=0x55e09fc35fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99a95b4640 1 --2- 172.21.15.18:0/473672519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc22d60 0x55e09fc35fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99a95b4640 1 -- 172.21.15.18:0/473672519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 msgr2=0x55e09fc36510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99a95b4640 1 --2- 172.21.15.18:0/473672519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 0x55e09fc36510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.511+0000 7f99a95b4640 1 -- 172.21.15.18:0/473672519 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e09fb88d80 con 0x55e09fc306f0 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f99a95b4640 1 --2- 172.21.15.18:0/473672519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc36a50 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f999c006010 tx=0x7f999c010880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fb77fdbf1b1461 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f9983fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f9983fff640 1 -- 172.21.15.18:0/473672519 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c00fd20 con 0x55e09fc306f0 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f9983fff640 1 -- 172.21.15.18:0/473672519 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f999c010640 con 0x55e09fc306f0 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f99adbdf740 1 -- 172.21.15.18:0/473672519 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e09f8c7550 con 0x55e09fc306f0 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f9983fff640 1 -- 172.21.15.18:0/473672519 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f999c004ab0 con 0x55e09fc306f0 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f99adbdf740 10 client.20475 fetch_fsmap learned FSMap version 60 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f99adbdf740 1 -- 172.21.15.18:0/473672519 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55e09fb4b540 con 0x55e09fc306f0 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f9983fff640 1 -- 172.21.15.18:0/473672519 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e09f8c7550 con 0x55e09fc306f0 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f9983fff640 1 -- 172.21.15.18:0/473672519 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 60) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f999c0357b0 con 0x55e09fc306f0 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f99adbdf740 10 client.20475 fetch_fsmap finished waiting for FSMap version 60 2024-02-17T13:03:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f99adbdf740 10 client.20475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f99adbdf740 20 client.20475 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f99adbdf740 1 --2- 172.21.15.18:0/473672519 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e09fc985b0 0x55e09fc9a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f99adbdf740 4 client.20475 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:03:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.512+0000 7f99adbdf740 1 -- 172.21.15.18:0/473672519 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e09f836520 con 0x55e09fc985b0 2024-02-17T13:03:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.513+0000 7f99a8db3640 1 --2- 172.21.15.18:0/473672519 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e09fc985b0 0x55e09fc9a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.515+0000 7f9983fff640 10 client.20475 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.515+0000 7f99a8db3640 1 --2- 172.21.15.18:0/473672519 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e09fc985b0 0x55e09fc9a9d0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdbb4a7aaf43485d server_cookie=a438b470fcbcb14e in_seq=0 out_seq=0 2024-02-17T13:03:43.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 28570 31.27 MB/sec warmup 55 sec latency 166.017 ms 2024-02-17T13:03:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.798+0000 7f9983fff640 1 -- 172.21.15.18:0/473672519 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e09f836520 con 0x55e09fc985b0 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f9983fff640 10 client.20475 handle_command_reply: tid=0 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f99adbdf740 1 client.20475 shutdown 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f99adbdf740 1 -- 172.21.15.18:0/473672519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 msgr2=0x55e09fc36a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f99adbdf740 1 --2- 172.21.15.18:0/473672519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc36a50 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f999c006010 tx=0x7f999c010880 comp rx=0 tx=0).stop 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f99adbdf740 1 -- 172.21.15.18:0/473672519 shutdown_connections 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f99adbdf740 1 --2- 172.21.15.18:0/473672519 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55e09fc985b0 0x55e09fc9a9d0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f99adbdf740 1 --2- 172.21.15.18:0/473672519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e09fc306f0 0x55e09fc36a50 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f99adbdf740 1 --2- 172.21.15.18:0/473672519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e09fc23ad0 0x55e09fc36510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f99adbdf740 1 --2- 172.21.15.18:0/473672519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e09fc22d60 0x55e09fc35fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f99adbdf740 1 -- 172.21.15.18:0/473672519 >> 172.21.15.18:0/473672519 conn(0x55e09fbea550 msgr2=0x55e09fc1c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f9983fff640 0 client.20475 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f99adbdf740 1 -- 172.21.15.18:0/473672519 shutdown_connections 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f99adbdf740 1 -- 172.21.15.18:0/473672519 wait complete. 2024-02-17T13:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.799+0000 7f99adbdf740 20 client.20475 trim_cache size 0 max 16384 2024-02-17T13:03:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:03:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:03:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:03:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.800+0000 7f99ab017640 1 -- 172.21.15.18:0/2942108992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f99940a22d0 msgr2=0x7f99940a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.800+0000 7f99ab017640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f99940a22d0 0x7f99940a4790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f999c009fd0 tx=0x7f999c013b80 comp rx=0 tx=0).stop 2024-02-17T13:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.800+0000 7f99ab017640 1 -- 172.21.15.18:0/2942108992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99a406a340 msgr2=0x7f99a4197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.800+0000 7f99ab017640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99a406a340 0x7f99a4197bf0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f9990009230 tx=0x7f999002c6c0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.800+0000 7f99ab017640 1 -- 172.21.15.18:0/2942108992 shutdown_connections 2024-02-17T13:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.800+0000 7f99ab017640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f99940a22d0 0x7f99940a4790 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.800+0000 7f99ab017640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99a41071b0 0x7f99a419bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.800+0000 7f99ab017640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99a406a340 0x7f99a4197bf0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.800+0000 7f99ab017640 1 --2- 172.21.15.18:0/2942108992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99a4069980 0x7f99a41976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.800+0000 7f99ab017640 1 -- 172.21.15.18:0/2942108992 >> 172.21.15.18:0/2942108992 conn(0x7f99a4076e50 msgr2=0x7f99a40ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.800+0000 7f99ab017640 1 -- 172.21.15.18:0/2942108992 shutdown_connections 2024-02-17T13:03:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:43.800+0000 7f99ab017640 1 -- 172.21.15.18:0/2942108992 wait complete. 2024-02-17T13:03:44.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:44 smithi146 ceph-mon[36454]: pgmap v435: 97 pgs: 97 active+clean; 555 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-02-17T13:03:44.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:44 smithi018 ceph-mon[32625]: pgmap v435: 97 pgs: 97 active+clean; 555 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-02-17T13:03:44.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 29222 31.30 MB/sec warmup 56 sec latency 169.953 ms 2024-02-17T13:03:44.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:44 smithi119 ceph-mon[36188]: pgmap v435: 97 pgs: 97 active+clean; 555 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-02-17T13:03:45.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 30151 31.37 MB/sec warmup 57 sec latency 62.746 ms 2024-02-17T13:03:46.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:46 smithi146 ceph-mon[36454]: pgmap v436: 97 pgs: 97 active+clean; 582 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.6 MiB/s wr, 976 op/s 2024-02-17T13:03:46.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:46 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:46.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:46 smithi018 ceph-mon[32625]: pgmap v436: 97 pgs: 97 active+clean; 582 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.6 MiB/s wr, 976 op/s 2024-02-17T13:03:46.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:46 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:46.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 31297 32.05 MB/sec warmup 58 sec latency 59.350 ms 2024-02-17T13:03:46.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:46 smithi119 ceph-mon[36188]: pgmap v436: 97 pgs: 97 active+clean; 582 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.6 MiB/s wr, 976 op/s 2024-02-17T13:03:46.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:46 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:47.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 32150 32.34 MB/sec warmup 59 sec latency 59.154 ms 2024-02-17T13:03:48.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:48 smithi146 ceph-mon[36454]: pgmap v437: 97 pgs: 97 active+clean; 582 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.2 MiB/s wr, 607 op/s 2024-02-17T13:03:48.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:48 smithi018 ceph-mon[32625]: pgmap v437: 97 pgs: 97 active+clean; 582 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.2 MiB/s wr, 607 op/s 2024-02-17T13:03:48.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 33046 32.58 MB/sec warmup 60 sec latency 110.438 ms 2024-02-17T13:03:48.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:48 smithi119 ceph-mon[36188]: pgmap v437: 97 pgs: 97 active+clean; 582 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.2 MiB/s wr, 607 op/s 2024-02-17T13:03:49.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 33321 32.35 MB/sec warmup 61 sec latency 1000.861 ms 2024-02-17T13:03:50.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:50 smithi018 ceph-mon[32625]: pgmap v438: 97 pgs: 97 active+clean; 592 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 751 op/s 2024-02-17T13:03:50.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:50 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:50.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 33817 32.19 MB/sec warmup 62 sec latency 1126.380 ms 2024-02-17T13:03:50.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:50 smithi119 ceph-mon[36188]: pgmap v438: 97 pgs: 97 active+clean; 592 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 751 op/s 2024-02-17T13:03:50.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:50 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:50.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:50 smithi146 ceph-mon[36454]: pgmap v438: 97 pgs: 97 active+clean; 592 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 751 op/s 2024-02-17T13:03:50.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:50 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:03:51.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 34707 32.45 MB/sec warmup 63 sec latency 97.685 ms 2024-02-17T13:03:51.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T13:03:51.853 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T13:03:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.012+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2997209270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f81002d0 msgr2=0x7f55f8100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.012+0000 7f55ffcaa640 1 --2- 172.21.15.18:0/2997209270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f81002d0 0x7f55f8100730 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009900 tx=0x7f55e802bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.012+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2997209270 shutdown_connections 2024-02-17T13:03:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.012+0000 7f55ffcaa640 1 --2- 172.21.15.18:0/2997209270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f8100c70 0x7f55f810ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.012+0000 7f55ffcaa640 1 --2- 172.21.15.18:0/2997209270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f81002d0 0x7f55f8100730 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.012+0000 7f55ffcaa640 1 --2- 172.21.15.18:0/2997209270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f81060a0 0x7f55f8106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.012+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2997209270 >> 172.21.15.18:0/2997209270 conn(0x7f55f8076e10 msgr2=0x7f55f8077220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.012+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2997209270 shutdown_connections 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.012+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2997209270 wait complete. 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.013+0000 7f55ffcaa640 1 Processor -- start 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.013+0000 7f55ffcaa640 1 -- start start 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.013+0000 7f55ffcaa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f81002d0 0x7f55f819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.013+0000 7f55ffcaa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8100c70 0x7f55f819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.013+0000 7f55ffcaa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f81060a0 0x7f55f81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.013+0000 7f55ffcaa640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f8199e60 con 0x7f55f81002d0 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.013+0000 7f55ffcaa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f8199fd0 con 0x7f55f81060a0 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.013+0000 7f55ffcaa640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f819a190 con 0x7f55f8100c70 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.013+0000 7f55fd245640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8100c70 0x7f55f819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.013+0000 7f55fd245640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8100c70 0x7f55f819fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:55562/0 (socket says 172.21.15.18:55562) 2024-02-17T13:03:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.013+0000 7f55fd245640 1 -- 172.21.15.18:0/2136045590 learned_addr learned my addr 172.21.15.18:0/2136045590 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.013+0000 7f55fe247640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f81060a0 0x7f55f81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.014+0000 7f55fda46640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f81002d0 0x7f55f819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.014+0000 7f55fe247640 1 -- 172.21.15.18:0/2136045590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8100c70 msgr2=0x7f55f819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.014+0000 7f55fe247640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8100c70 0x7f55f819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.014+0000 7f55fe247640 1 -- 172.21.15.18:0/2136045590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f81002d0 msgr2=0x7f55f819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.014+0000 7f55fe247640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f81002d0 0x7f55f819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.014+0000 7f55fe247640 1 -- 172.21.15.18:0/2136045590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e8009560 con 0x7f55f81060a0 2024-02-17T13:03:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.014+0000 7f55fe247640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f81060a0 0x7f55f81997c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55f000da10 tx=0x7f55f000ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.015+0000 7f55eeffd640 1 -- 172.21.15.18:0/2136045590 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f000bd70 con 0x7f55f81060a0 2024-02-17T13:03:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.015+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2136045590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f819a470 con 0x7f55f81060a0 2024-02-17T13:03:52.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.015+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2136045590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f81a6680 con 0x7f55f81060a0 2024-02-17T13:03:52.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.015+0000 7f55eeffd640 1 -- 172.21.15.18:0/2136045590 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55f0003e50 con 0x7f55f81060a0 2024-02-17T13:03:52.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.015+0000 7f55eeffd640 1 -- 172.21.15.18:0/2136045590 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0010660 con 0x7f55f81060a0 2024-02-17T13:03:52.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.018+0000 7f55eeffd640 1 -- 172.21.15.18:0/2136045590 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f55f00107c0 con 0x7f55f81060a0 2024-02-17T13:03:52.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.018+0000 7f560287d740 1 Processor -- start 2024-02-17T13:03:52.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f560287d740 1 -- start start 2024-02-17T13:03:52.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f55eeffd640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f55e40a2230 0x7f55e40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f560287d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 0x55c2e1d03690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f560287d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d0c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f560287d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d0eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f560287d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e1c72280 con 0x55c2e1d032b0 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f560287d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e1c6f7b0 con 0x55c2e1d03bd0 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f560287d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e1c6af90 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f55eeffd640 1 -- 172.21.15.18:0/2136045590 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f55f00ce010 con 0x7f55f81060a0 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f55fd245640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f55e40a2230 0x7f55e40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f55fe247640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d0eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f55fe247640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d0eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:55568/0 (socket says 172.21.15.18:55568) 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f55fe247640 1 -- 172.21.15.18:0/519393520 learned_addr learned my addr 172.21.15.18:0/519393520 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f55fda46640 1 --2- 172.21.15.18:0/519393520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 0x55c2e1d03690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.019+0000 7f55fe247640 1 -- 172.21.15.18:0/519393520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d03bd0 msgr2=0x55c2e1d0c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.020+0000 7f55fe247640 1 --2- 172.21.15.18:0/519393520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d0c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.020+0000 7f55fe247640 1 -- 172.21.15.18:0/519393520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 msgr2=0x55c2e1d03690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.020+0000 7f55fe247640 1 --2- 172.21.15.18:0/519393520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 0x55c2e1d03690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.020+0000 7f55fe247640 1 -- 172.21.15.18:0/519393520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2e1c29570 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.020+0000 7f55fda46640 1 --2- 172.21.15.18:0/519393520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 0x55c2e1d03690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:52.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.020+0000 7f55fd245640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f55e40a2230 0x7f55e40a46f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f55e8000c00 tx=0x7f55e802c5f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.020+0000 7f55fe247640 1 --2- 172.21.15.18:0/519393520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d0eba0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55f00973f0 tx=0x7f55f009b120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76eb6025523ebd91 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.020+0000 7f55e3fff640 1 -- 172.21.15.18:0/519393520 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0066de0 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.020+0000 7f55e3fff640 1 -- 172.21.15.18:0/519393520 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55f009aa20 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.020+0000 7f55e3fff640 1 -- 172.21.15.18:0/519393520 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f00cbb90 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.021+0000 7f560287d740 1 -- 172.21.15.18:0/519393520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d0c7b0 msgr2=0x55c2e1d0eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.021+0000 7f560287d740 1 --2- 172.21.15.18:0/519393520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d0eba0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55f00973f0 tx=0x7f55f009b120 comp rx=0 tx=0).stop 2024-02-17T13:03:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.021+0000 7f560287d740 1 -- 172.21.15.18:0/519393520 shutdown_connections 2024-02-17T13:03:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.021+0000 7f560287d740 1 --2- 172.21.15.18:0/519393520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d0eba0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.021+0000 7f560287d740 1 --2- 172.21.15.18:0/519393520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d0c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.021+0000 7f560287d740 1 --2- 172.21.15.18:0/519393520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 0x55c2e1d03690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.021+0000 7f560287d740 1 -- 172.21.15.18:0/519393520 >> 172.21.15.18:0/519393520 conn(0x55c2e1cfd990 msgr2=0x55c2e1cfdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.022+0000 7f560287d740 1 -- 172.21.15.18:0/519393520 shutdown_connections 2024-02-17T13:03:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.022+0000 7f560287d740 1 -- 172.21.15.18:0/519393520 wait complete. 2024-02-17T13:03:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.022+0000 7f560287d740 1 -- start start 2024-02-17T13:03:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.022+0000 7f560287d740 1 Processor -- start 2024-02-17T13:03:52.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f560287d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d032b0 0x55c2e1d0c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f560287d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d09580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f560287d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d09cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f560287d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e193e430 con 0x55c2e1d03bd0 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f560287d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e18fae00 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f560287d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e1beffd0 con 0x55c2e1d032b0 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f560287d740 10 client.0 _ll_register_callbacks cb 0x55c2e1c45fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f55fd245640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d09580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f55fd245640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d09580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40846/0 (socket says 172.21.15.18:40846) 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f55fd245640 1 -- 172.21.15.18:0/3204003743 learned_addr learned my addr 172.21.15.18:0/3204003743 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f55fda46640 1 --2- 172.21.15.18:0/3204003743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d032b0 0x55c2e1d0c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f55fd245640 1 -- 172.21.15.18:0/3204003743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d032b0 msgr2=0x55c2e1d0c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f55fd245640 1 --2- 172.21.15.18:0/3204003743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d032b0 0x55c2e1d0c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f55fd245640 1 -- 172.21.15.18:0/3204003743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 msgr2=0x55c2e1d09cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f55fd245640 1 --2- 172.21.15.18:0/3204003743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d09cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.023+0000 7f55fd245640 1 -- 172.21.15.18:0/3204003743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f81a6680 con 0x55c2e1d03bd0 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.024+0000 7f55e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.024+0000 7f55fd245640 1 --2- 172.21.15.18:0/3204003743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d09580 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f55e8005e90 tx=0x7f55e8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ed15741c85ed18 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.024+0000 7f55e17fa640 1 -- 172.21.15.18:0/3204003743 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8004df0 con 0x55c2e1d03bd0 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.024+0000 7f55e17fa640 1 -- 172.21.15.18:0/3204003743 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f55e8004f50 con 0x55c2e1d03bd0 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.024+0000 7f560287d740 1 -- 172.21.15.18:0/3204003743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2e196a7e0 con 0x55c2e1d03bd0 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.024+0000 7f55e17fa640 1 -- 172.21.15.18:0/3204003743 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55e8005110 con 0x55c2e1d03bd0 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.024+0000 7f560287d740 10 client.20487 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.024+0000 7f560287d740 1 -- 172.21.15.18:0/3204003743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x55c2e1c69610 con 0x55c2e1d03bd0 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.025+0000 7f55e17fa640 1 -- 172.21.15.18:0/3204003743 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c2e196a7e0 con 0x55c2e1d03bd0 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.025+0000 7f55e17fa640 1 -- 172.21.15.18:0/3204003743 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f55e803c030 con 0x55c2e1d03bd0 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.025+0000 7f560287d740 10 client.20487 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.025+0000 7f560287d740 10 client.20487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.025+0000 7f560287d740 20 client.20487 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:52.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.025+0000 7f560287d740 1 --2- 172.21.15.18:0/3204003743 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c2e1d251a0 0x55c2e1d275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.025+0000 7f560287d740 4 client.20487 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:52.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.025+0000 7f560287d740 1 -- 172.21.15.18:0/3204003743 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2e19a7720 con 0x55c2e1d251a0 2024-02-17T13:03:52.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.025+0000 7f55fda46640 1 --2- 172.21.15.18:0/3204003743 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c2e1d251a0 0x55c2e1d275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.026+0000 7f55fda46640 1 --2- 172.21.15.18:0/3204003743 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c2e1d251a0 0x55c2e1d275c0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a98bcecf92d0c server_cookie=e908c15357791c29 in_seq=0 out_seq=0 2024-02-17T13:03:52.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.026+0000 7f55e17fa640 10 client.20487 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.027+0000 7f55e17fa640 1 -- 172.21.15.18:0/3204003743 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2e19a7720 con 0x55c2e1d251a0 2024-02-17T13:03:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.027+0000 7f55e17fa640 10 client.20487 handle_command_reply: tid=0 2024-02-17T13:03:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.027+0000 7f560287d740 1 client.20487 shutdown 2024-02-17T13:03:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.028+0000 7f560287d740 1 -- 172.21.15.18:0/3204003743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d03bd0 msgr2=0x55c2e1d09580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.028+0000 7f560287d740 1 --2- 172.21.15.18:0/3204003743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d09580 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f55e8005e90 tx=0x7f55e8009260 comp rx=0 tx=0).stop 2024-02-17T13:03:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.028+0000 7f560287d740 1 -- 172.21.15.18:0/3204003743 shutdown_connections 2024-02-17T13:03:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.028+0000 7f560287d740 1 --2- 172.21.15.18:0/3204003743 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c2e1d251a0 0x55c2e1d275c0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.028+0000 7f560287d740 1 --2- 172.21.15.18:0/3204003743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d09cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.028+0000 7f560287d740 1 --2- 172.21.15.18:0/3204003743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d09580 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.028+0000 7f55e17fa640 0 client.20487 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:52.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.028+0000 7f560287d740 1 --2- 172.21.15.18:0/3204003743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d032b0 0x55c2e1d0c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.028+0000 7f560287d740 1 -- 172.21.15.18:0/3204003743 >> 172.21.15.18:0/3204003743 conn(0x55c2e1cfd990 msgr2=0x55c2e1cff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.028+0000 7f560287d740 1 -- 172.21.15.18:0/3204003743 shutdown_connections 2024-02-17T13:03:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.028+0000 7f560287d740 1 -- 172.21.15.18:0/3204003743 wait complete. 2024-02-17T13:03:52.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.028+0000 7f560287d740 20 client.20487 trim_cache size 0 max 16384 2024-02-17T13:03:52.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.051+0000 7f560287d740 1 Processor -- start 2024-02-17T13:03:52.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.051+0000 7f560287d740 1 -- start start 2024-02-17T13:03:52.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.051+0000 7f560287d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 0x55c2e1d0bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.051+0000 7f560287d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d0c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.051+0000 7f55fda46640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 0x55c2e1d0bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.051+0000 7f55fda46640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 0x55c2e1d0bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40860/0 (socket says 172.21.15.18:40860) 2024-02-17T13:03:52.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.051+0000 7f55fda46640 1 -- 172.21.15.18:0/449957111 learned_addr learned my addr 172.21.15.18:0/449957111 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.051+0000 7f560287d740 1 --2- 172.21.15.18:0/449957111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d07e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.051+0000 7f560287d740 1 -- 172.21.15.18:0/449957111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e1d21610 con 0x55c2e1d032b0 2024-02-17T13:03:52.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.051+0000 7f560287d740 1 -- 172.21.15.18:0/449957111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e1d2c7b0 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.051+0000 7f560287d740 1 -- 172.21.15.18:0/449957111 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e1cf92b0 con 0x55c2e1d03bd0 2024-02-17T13:03:52.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.051+0000 7f55fd245640 1 --2- 172.21.15.18:0/449957111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d0c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.052+0000 7f55fe247640 1 --2- 172.21.15.18:0/449957111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d07e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.052+0000 7f55fda46640 1 -- 172.21.15.18:0/449957111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d03bd0 msgr2=0x55c2e1d0c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.052+0000 7f55fda46640 1 --2- 172.21.15.18:0/449957111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d0c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.052+0000 7f55fda46640 1 -- 172.21.15.18:0/449957111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 msgr2=0x55c2e1d07e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.052+0000 7f55fda46640 1 --2- 172.21.15.18:0/449957111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d07e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.052+0000 7f55fda46640 1 -- 172.21.15.18:0/449957111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2e1c05ac0 con 0x55c2e1d032b0 2024-02-17T13:03:52.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.052+0000 7f55fe247640 1 --2- 172.21.15.18:0/449957111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d07e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:03:52.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.052+0000 7f55fda46640 1 --2- 172.21.15.18:0/449957111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 0x55c2e1d0bcb0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f55f400b820 tx=0x7f55f400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b9bc5e751730dd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.052+0000 7f55e37fe640 1 -- 172.21.15.18:0/449957111 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f4016030 con 0x55c2e1d032b0 2024-02-17T13:03:52.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.052+0000 7f55e37fe640 1 -- 172.21.15.18:0/449957111 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55f4017040 con 0x55c2e1d032b0 2024-02-17T13:03:52.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.052+0000 7f55e37fe640 1 -- 172.21.15.18:0/449957111 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f4004e30 con 0x55c2e1d032b0 2024-02-17T13:03:52.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.053+0000 7f560287d740 1 -- 172.21.15.18:0/449957111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 msgr2=0x55c2e1d0bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.053+0000 7f560287d740 1 --2- 172.21.15.18:0/449957111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 0x55c2e1d0bcb0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f55f400b820 tx=0x7f55f400b390 comp rx=0 tx=0).stop 2024-02-17T13:03:52.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.053+0000 7f560287d740 1 -- 172.21.15.18:0/449957111 shutdown_connections 2024-02-17T13:03:52.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.053+0000 7f560287d740 1 --2- 172.21.15.18:0/449957111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d07e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.053+0000 7f560287d740 1 --2- 172.21.15.18:0/449957111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d0c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.053+0000 7f560287d740 1 --2- 172.21.15.18:0/449957111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d032b0 0x55c2e1d0bcb0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.053+0000 7f560287d740 1 -- 172.21.15.18:0/449957111 >> 172.21.15.18:0/449957111 conn(0x55c2e1cfd990 msgr2=0x55c2e1d04810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.053+0000 7f560287d740 1 -- 172.21.15.18:0/449957111 shutdown_connections 2024-02-17T13:03:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.054+0000 7f560287d740 1 -- 172.21.15.18:0/449957111 wait complete. 2024-02-17T13:03:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.054+0000 7f560287d740 1 -- start start 2024-02-17T13:03:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.054+0000 7f560287d740 1 Processor -- start 2024-02-17T13:03:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f560287d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d032b0 0x55c2e1d1c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f560287d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f560287d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d18a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f560287d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e1d18ff0 con 0x55c2e1d03bd0 2024-02-17T13:03:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f560287d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e1d19160 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f560287d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e1d19320 con 0x55c2e1d032b0 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f55fe247640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d18a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f55fe247640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d18a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:35010/0 (socket says 172.21.15.18:35010) 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f55fe247640 1 -- 172.21.15.18:0/2738144974 learned_addr learned my addr 172.21.15.18:0/2738144974 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f55fd245640 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f55fda46640 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d032b0 0x55c2e1d1c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f560287d740 10 client.0 _ll_register_callbacks cb 0x7f55f000bed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f55fe247640 1 -- 172.21.15.18:0/2738144974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d032b0 msgr2=0x55c2e1d1c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f55fe247640 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d032b0 0x55c2e1d1c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.055+0000 7f55fe247640 1 -- 172.21.15.18:0/2738144974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d03bd0 msgr2=0x55c2e1d184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.056+0000 7f55fe247640 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.056+0000 7f55fe247640 1 -- 172.21.15.18:0/2738144974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f400b930 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.056+0000 7f560287d740 1 -- 172.21.15.18:0/2738144974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55f009aa20 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.056+0000 7f55fe247640 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d18a30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f55f009a720 tx=0x7f55f000b7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9313c0fb9f78dd9b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.056+0000 7f55e17fa640 10 client.29336 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.056+0000 7f55e17fa640 1 -- 172.21.15.18:0/2738144974 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f00a31c0 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.057+0000 7f55e17fa640 1 -- 172.21.15.18:0/2738144974 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f55f00ba040 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.057+0000 7f55e17fa640 1 -- 172.21.15.18:0/2738144974 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55f0097090 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.057+0000 7f560287d740 10 client.29336 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.057+0000 7f560287d740 1 -- 172.21.15.18:0/2738144974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=62}) v3 -- 0x55c2e1c2b6a0 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.057+0000 7f55e17fa640 1 -- 172.21.15.18:0/2738144974 <== mon.2 v2:172.21.15.119:3300/0 4 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f55f001a070 con 0x55c2e1d0c7b0 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.057+0000 7f560287d740 10 client.29336 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.057+0000 7f560287d740 10 client.29336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.057+0000 7f560287d740 20 client.29336 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.057+0000 7f560287d740 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c2e1d7c690 0x55c2e1d7ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.058+0000 7f560287d740 4 client.29336 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T13:03:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.058+0000 7f560287d740 1 -- 172.21.15.18:0/2738144974 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f55f009aa20 con 0x55c2e1d7c690 2024-02-17T13:03:52.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.058+0000 7f55fda46640 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c2e1d7c690 0x55c2e1d7ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.059+0000 7f55fda46640 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c2e1d7c690 0x55c2e1d7ea80 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82ee6005d40ce3ef server_cookie=80abd0371d38a46d in_seq=0 out_seq=0 2024-02-17T13:03:52.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.059+0000 7f55e17fa640 10 client.29336 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:52.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.061+0000 7f55e17fa640 1 -- 172.21.15.18:0/2738144974 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f55f009aa20 con 0x55c2e1d7c690 2024-02-17T13:03:52.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.061+0000 7f55e17fa640 10 client.29336 handle_command_reply: tid=0 2024-02-17T13:03:52.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.061+0000 7f560287d740 1 client.29336 shutdown 2024-02-17T13:03:52.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.061+0000 7f560287d740 1 -- 172.21.15.18:0/2738144974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 msgr2=0x55c2e1d18a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.061+0000 7f560287d740 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d18a30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f55f009a720 tx=0x7f55f000b7d0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.062+0000 7f560287d740 1 -- 172.21.15.18:0/2738144974 shutdown_connections 2024-02-17T13:03:52.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.062+0000 7f560287d740 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55c2e1d7c690 0x55c2e1d7ea80 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.062+0000 7f560287d740 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e1d0c7b0 0x55c2e1d18a30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.062+0000 7f560287d740 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c2e1d03bd0 0x55c2e1d184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.062+0000 7f560287d740 1 --2- 172.21.15.18:0/2738144974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2e1d032b0 0x55c2e1d1c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.062+0000 7f55e17fa640 0 client.29336 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:52.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.062+0000 7f560287d740 1 -- 172.21.15.18:0/2738144974 >> 172.21.15.18:0/2738144974 conn(0x55c2e1cfd990 msgr2=0x55c2e1d08e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.063+0000 7f560287d740 1 -- 172.21.15.18:0/2738144974 shutdown_connections 2024-02-17T13:03:52.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.063+0000 7f560287d740 1 -- 172.21.15.18:0/2738144974 wait complete. 2024-02-17T13:03:52.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.063+0000 7f560287d740 20 client.29336 trim_cache size 0 max 16384 2024-02-17T13:03:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:03:52.055 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-02-17T13:03:52.055 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-02-17T13:03:52.055 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:03:52.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.064+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2136045590 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f55e40a2230 msgr2=0x7f55e40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.064+0000 7f55ffcaa640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f55e40a2230 0x7f55e40a46f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f55e8000c00 tx=0x7f55e802c5f0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.064+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2136045590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f81060a0 msgr2=0x7f55f81997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.064+0000 7f55ffcaa640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f81060a0 0x7f55f81997c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55f000da10 tx=0x7f55f000ded0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.064+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2136045590 shutdown_connections 2024-02-17T13:03:52.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.064+0000 7f55ffcaa640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f55e40a2230 0x7f55e40a46f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.064+0000 7f55ffcaa640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f81060a0 0x7f55f81997c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.064+0000 7f55ffcaa640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8100c70 0x7f55f819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.064+0000 7f55ffcaa640 1 --2- 172.21.15.18:0/2136045590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f81002d0 0x7f55f819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.064+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2136045590 >> 172.21.15.18:0/2136045590 conn(0x7f55f8076e10 msgr2=0x7f55f80695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.064+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2136045590 shutdown_connections 2024-02-17T13:03:52.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.064+0000 7f55ffcaa640 1 -- 172.21.15.18:0/2136045590 wait complete. 2024-02-17T13:03:52.069 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T13:03:52.069 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T13:03:52.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.219+0000 7f48665e2640 1 -- 172.21.15.18:0/1914998007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4860101d30 msgr2=0x7f4860105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.219+0000 7f48665e2640 1 --2- 172.21.15.18:0/1914998007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4860101d30 0x7f4860105d70 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f4854009890 tx=0x7f485402bb90 comp rx=0 tx=0).stop 2024-02-17T13:03:52.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.219+0000 7f48665e2640 1 -- 172.21.15.18:0/1914998007 shutdown_connections 2024-02-17T13:03:52.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.219+0000 7f48665e2640 1 --2- 172.21.15.18:0/1914998007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4860106450 0x7f486010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.219+0000 7f48665e2640 1 --2- 172.21.15.18:0/1914998007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4860101d30 0x7f4860105d70 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.219+0000 7f48665e2640 1 --2- 172.21.15.18:0/1914998007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4860101380 0x7f4860101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.219+0000 7f48665e2640 1 -- 172.21.15.18:0/1914998007 >> 172.21.15.18:0/1914998007 conn(0x7f4860078580 msgr2=0x7f4860078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.219+0000 7f48665e2640 1 -- 172.21.15.18:0/1914998007 shutdown_connections 2024-02-17T13:03:52.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.219+0000 7f48665e2640 1 -- 172.21.15.18:0/1914998007 wait complete. 2024-02-17T13:03:52.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.220+0000 7f48665e2640 1 Processor -- start 2024-02-17T13:03:52.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.220+0000 7f48665e2640 1 -- start start 2024-02-17T13:03:52.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.220+0000 7f48665e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4860101380 0x7f486019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.220+0000 7f48665e2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4860106450 0x7f486019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.220+0000 7f48665e2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4860197690 0x7f4860197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.220+0000 7f48665e2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4860198160 con 0x7f4860106450 2024-02-17T13:03:52.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.220+0000 7f48665e2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601982d0 con 0x7f4860101380 2024-02-17T13:03:52.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.220+0000 7f48665e2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4860198490 con 0x7f4860197690 2024-02-17T13:03:52.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.220+0000 7f485ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4860101380 0x7f486019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.220+0000 7f485f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4860106450 0x7f486019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.221+0000 7f485ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4860101380 0x7f486019d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:35018/0 (socket says 172.21.15.18:35018) 2024-02-17T13:03:52.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.221+0000 7f485ffff640 1 -- 172.21.15.18:0/3850103697 learned_addr learned my addr 172.21.15.18:0/3850103697 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.221+0000 7f4864b7f640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4860197690 0x7f4860197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.221+0000 7f485f7fe640 1 -- 172.21.15.18:0/3850103697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4860197690 msgr2=0x7f4860197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.221+0000 7f485f7fe640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4860197690 0x7f4860197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.221+0000 7f485f7fe640 1 -- 172.21.15.18:0/3850103697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4860101380 msgr2=0x7f486019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.221+0000 7f485f7fe640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4860101380 0x7f486019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.221+0000 7f485f7fe640 1 -- 172.21.15.18:0/3850103697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4854009490 con 0x7f4860106450 2024-02-17T13:03:52.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.221+0000 7f485ffff640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4860101380 0x7f486019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:52.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.221+0000 7f485f7fe640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4860106450 0x7f486019db10 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f485402c0d0 tx=0x7f48540045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.222+0000 7f485d7fa640 1 -- 172.21.15.18:0/3850103697 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48540349d0 con 0x7f4860106450 2024-02-17T13:03:52.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.222+0000 7f48665e2640 1 -- 172.21.15.18:0/3850103697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48601a43d0 con 0x7f4860106450 2024-02-17T13:03:52.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.222+0000 7f48665e2640 1 -- 172.21.15.18:0/3850103697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48601a48a0 con 0x7f4860106450 2024-02-17T13:03:52.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.222+0000 7f485d7fa640 1 -- 172.21.15.18:0/3850103697 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f485402ca10 con 0x7f4860106450 2024-02-17T13:03:52.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.222+0000 7f485d7fa640 1 -- 172.21.15.18:0/3850103697 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854032600 con 0x7f4860106450 2024-02-17T13:03:52.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.223+0000 7f48665e2640 1 -- 172.21.15.18:0/3850103697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4860102850 con 0x7f4860106450 2024-02-17T13:03:52.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.225+0000 7f485d7fa640 1 -- 172.21.15.18:0/3850103697 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f485403b040 con 0x7f4860106450 2024-02-17T13:03:52.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.226+0000 7f485d7fa640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f48480a2330 0x7f48480a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.226+0000 7f485d7fa640 1 -- 172.21.15.18:0/3850103697 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f48540ef980 con 0x7f4860106450 2024-02-17T13:03:52.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.226+0000 7f485ffff640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f48480a2330 0x7f48480a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.227+0000 7f485ffff640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f48480a2330 0x7f48480a47f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f485800e930 tx=0x7f48580095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.232+0000 7f485d7fa640 1 -- 172.21.15.18:0/3850103697 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f48540b8790 con 0x7f4860106450 2024-02-17T13:03:52.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.537+0000 7f48665e2640 1 -- 172.21.15.18:0/3850103697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4860109680 con 0x7f4860106450 2024-02-17T13:03:52.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.538+0000 7f485d7fa640 1 -- 172.21.15.18:0/3850103697 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+11413 (secure 0 0 0) 0x7f48540bbe50 con 0x7f4860106450 2024-02-17T13:03:52.529 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:03:52.529 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":62,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T13:03:48.908271+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":240,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26872":{"gid":26872,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/1419682628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1419682628},{"type":"v1","addr":"172.21.15.146:6839","nonce":1419682628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T13:03:52.529 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 62 2024-02-17T13:03:52.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.540+0000 7f48665e2640 1 -- 172.21.15.18:0/3850103697 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f48480a2330 msgr2=0x7f48480a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.540+0000 7f48665e2640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f48480a2330 0x7f48480a47f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f485800e930 tx=0x7f48580095c0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.540+0000 7f48665e2640 1 -- 172.21.15.18:0/3850103697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4860106450 msgr2=0x7f486019db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.540+0000 7f48665e2640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4860106450 0x7f486019db10 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f485402c0d0 tx=0x7f48540045f0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.540+0000 7f48665e2640 1 -- 172.21.15.18:0/3850103697 shutdown_connections 2024-02-17T13:03:52.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.540+0000 7f48665e2640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f48480a2330 0x7f48480a47f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.540+0000 7f48665e2640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4860197690 0x7f4860197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.540+0000 7f48665e2640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4860106450 0x7f486019db10 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.540+0000 7f48665e2640 1 --2- 172.21.15.18:0/3850103697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4860101380 0x7f486019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.540+0000 7f48665e2640 1 -- 172.21.15.18:0/3850103697 >> 172.21.15.18:0/3850103697 conn(0x7f4860078580 msgr2=0x7f48600ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.541+0000 7f48665e2640 1 -- 172.21.15.18:0/3850103697 shutdown_connections 2024-02-17T13:03:52.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.541+0000 7f48665e2640 1 -- 172.21.15.18:0/3850103697 wait complete. 2024-02-17T13:03:52.544 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:03:52.544 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-17T13:03:52.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.667+0000 7fcb111a0640 1 -- 172.21.15.18:0/2386724590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb0c100b00 msgr2=0x7fcb0c104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.667+0000 7fcb111a0640 1 --2- 172.21.15.18:0/2386724590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb0c100b00 0x7fcb0c104b40 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009930 tx=0x7fcaf402bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:52.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.668+0000 7fcb111a0640 1 -- 172.21.15.18:0/2386724590 shutdown_connections 2024-02-17T13:03:52.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.668+0000 7fcb111a0640 1 --2- 172.21.15.18:0/2386724590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c105220 0x7fcb0c10cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.668+0000 7fcb111a0640 1 --2- 172.21.15.18:0/2386724590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb0c100b00 0x7fcb0c104b40 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.668+0000 7fcb111a0640 1 --2- 172.21.15.18:0/2386724590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb0c100150 0x7fcb0c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.668+0000 7fcb111a0640 1 -- 172.21.15.18:0/2386724590 >> 172.21.15.18:0/2386724590 conn(0x7fcb0c0fb9d0 msgr2=0x7fcb0c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.668+0000 7fcb111a0640 1 -- 172.21.15.18:0/2386724590 shutdown_connections 2024-02-17T13:03:52.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.668+0000 7fcb111a0640 1 -- 172.21.15.18:0/2386724590 wait complete. 2024-02-17T13:03:52.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.668+0000 7fcb111a0640 1 Processor -- start 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.668+0000 7fcb111a0640 1 -- start start 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.668+0000 7fcb111a0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb0c100150 0x7fcb0c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb111a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c105220 0x7fcb0c19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb111a0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb0c197680 0x7fcb0c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb111a0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c198150 con 0x7fcb0c100150 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb111a0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1982c0 con 0x7fcb0c105220 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb111a0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c198480 con 0x7fcb0c197680 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb0bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb0c100150 0x7fcb0c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb0bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb0c100150 0x7fcb0c19d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40908/0 (socket says 172.21.15.18:40908) 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb0bfff640 1 -- 172.21.15.18:0/513858634 learned_addr learned my addr 172.21.15.18:0/513858634 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb1099f640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb0c197680 0x7fcb0c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb0b7fe640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c105220 0x7fcb0c19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb0bfff640 1 -- 172.21.15.18:0/513858634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb0c197680 msgr2=0x7fcb0c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb0bfff640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb0c197680 0x7fcb0c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb0bfff640 1 -- 172.21.15.18:0/513858634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c105220 msgr2=0x7fcb0c19da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb0bfff640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c105220 0x7fcb0c19da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb0bfff640 1 -- 172.21.15.18:0/513858634 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf4009560 con 0x7fcb0c100150 2024-02-17T13:03:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb0bfff640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb0c100150 0x7fcb0c19d4d0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00d770 tx=0x7fcafc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb097fa640 1 -- 172.21.15.18:0/513858634 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc009c30 con 0x7fcb0c100150 2024-02-17T13:03:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb097fa640 1 -- 172.21.15.18:0/513858634 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcafc00de80 con 0x7fcb0c100150 2024-02-17T13:03:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb111a0640 1 -- 172.21.15.18:0/513858634 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c1a4330 con 0x7fcb0c100150 2024-02-17T13:03:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb097fa640 1 -- 172.21.15.18:0/513858634 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc004e80 con 0x7fcb0c100150 2024-02-17T13:03:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.669+0000 7fcb111a0640 1 -- 172.21.15.18:0/513858634 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c1a47c0 con 0x7fcb0c100150 2024-02-17T13:03:52.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb097fa640 1 -- 172.21.15.18:0/513858634 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcafc005020 con 0x7fcb0c100150 2024-02-17T13:03:52.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb14fd1740 1 Processor -- start 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb14fd1740 1 -- start start 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb14fd1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206170 0x56016b206550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb14fd1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206a90 0x56016b213810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb14fd1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b216140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb14fd1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56016b1c1960 con 0x56016b206170 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb14fd1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56016b175310 con 0x56016b213d50 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb14fd1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56016b172840 con 0x56016b206a90 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb1099f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206170 0x56016b206550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb097fa640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcaf80a2230 0x7fcaf80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb1099f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206170 0x56016b206550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40916/0 (socket says 172.21.15.18:40916) 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb1099f640 1 -- 172.21.15.18:0/704242968 learned_addr learned my addr 172.21.15.18:0/704242968 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb0bfff640 1 --2- 172.21.15.18:0/704242968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206a90 0x56016b213810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb0b7fe640 1 --2- 172.21.15.18:0/704242968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b216140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.672+0000 7fcb1099f640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcaf80a2230 0x7fcaf80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.673+0000 7fcb1099f640 1 -- 172.21.15.18:0/704242968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206a90 msgr2=0x56016b213810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.673+0000 7fcb1099f640 1 --2- 172.21.15.18:0/704242968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206a90 0x56016b213810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.673+0000 7fcb1099f640 1 -- 172.21.15.18:0/704242968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 msgr2=0x56016b216140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.673+0000 7fcb1099f640 1 --2- 172.21.15.18:0/704242968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b216140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.673+0000 7fcb1099f640 1 -- 172.21.15.18:0/704242968 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56016b130060 con 0x56016b206170 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.673+0000 7fcb097fa640 1 -- 172.21.15.18:0/513858634 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fcafc0cda20 con 0x7fcb0c100150 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.673+0000 7fcb1099f640 1 --2- 172.21.15.18:0/704242968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206170 0x56016b206550 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fcb04014a60 tx=0x7fcb04014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6045f84ba2827e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.673+0000 7fcb027fc640 1 -- 172.21.15.18:0/704242968 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04012e70 con 0x56016b206170 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.673+0000 7fcb027fc640 1 -- 172.21.15.18:0/704242968 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb0401a070 con 0x56016b206170 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.673+0000 7fcb027fc640 1 -- 172.21.15.18:0/704242968 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04005560 con 0x56016b206170 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.673+0000 7fcb1099f640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcaf80a2230 0x7fcaf80a46f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcb04022000 tx=0x7fcb0401e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.674+0000 7fcb14fd1740 1 -- 172.21.15.18:0/704242968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206170 msgr2=0x56016b206550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.674+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/704242968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206170 0x56016b206550 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fcb04014a60 tx=0x7fcb04014f20 comp rx=0 tx=0).stop 2024-02-17T13:03:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.674+0000 7fcb14fd1740 1 -- 172.21.15.18:0/704242968 shutdown_connections 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.674+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/704242968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b216140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.674+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/704242968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206a90 0x56016b213810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.674+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/704242968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206170 0x56016b206550 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.674+0000 7fcb14fd1740 1 -- 172.21.15.18:0/704242968 >> 172.21.15.18:0/704242968 conn(0x56016b200b70 msgr2=0x56016b200f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.674+0000 7fcb14fd1740 1 -- 172.21.15.18:0/704242968 shutdown_connections 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.674+0000 7fcb14fd1740 1 -- 172.21.15.18:0/704242968 wait complete. 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.674+0000 7fcb14fd1740 1 -- start start 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.674+0000 7fcb14fd1740 1 Processor -- start 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.674+0000 7fcb14fd1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b210db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb14fd1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 0x56016b2112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb14fd1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b211830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb14fd1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56016b16e020 con 0x56016b206a90 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb14fd1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56016ae41430 con 0x56016b213d50 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb14fd1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56016adfde00 con 0x56016b206170 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb14fd1740 10 client.0 _ll_register_callbacks cb 0x56016b115cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:52.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb0b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 0x56016b2112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb0b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 0x56016b2112f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40924/0 (socket says 172.21.15.18:40924) 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb0b7fe640 1 -- 172.21.15.18:0/2637143929 learned_addr learned my addr 172.21.15.18:0/2637143929 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb1099f640 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b211830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb0bfff640 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b210db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb0b7fe640 1 -- 172.21.15.18:0/2637143929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 msgr2=0x56016b210db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb0b7fe640 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b210db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb0b7fe640 1 -- 172.21.15.18:0/2637143929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 msgr2=0x56016b211830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb0b7fe640 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b211830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb0b7fe640 1 -- 172.21.15.18:0/2637143929 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb040146d0 con 0x56016b206a90 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb0bfff640 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b210db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb0b7fe640 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 0x56016b2112f0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4005ab0 tx=0x7fcaf402ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f6b3b47f481b9e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcaeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcaeb7fe640 1 -- 172.21.15.18:0/2637143929 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf402cbf0 con 0x56016b206a90 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcaeb7fe640 1 -- 172.21.15.18:0/2637143929 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fcaf4004250 con 0x56016b206a90 2024-02-17T13:03:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb14fd1740 1 -- 172.21.15.18:0/2637143929 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56016ae6d7e0 con 0x56016b206a90 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcaeb7fe640 1 -- 172.21.15.18:0/2637143929 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf4004410 con 0x56016b206a90 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.675+0000 7fcb14fd1740 10 client.20511 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.676+0000 7fcb14fd1740 1 -- 172.21.15.18:0/2637143929 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x56016b16c6a0 con 0x56016b206a90 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.676+0000 7fcaeb7fe640 1 -- 172.21.15.18:0/2637143929 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56016ae6d7e0 con 0x56016b206a90 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.676+0000 7fcaeb7fe640 1 -- 172.21.15.18:0/2637143929 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fcaf403c040 con 0x56016b206a90 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.676+0000 7fcb14fd1740 10 client.20511 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.676+0000 7fcb14fd1740 10 client.20511 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.676+0000 7fcb14fd1740 20 client.20511 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.676+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56016b2241c0 0x56016b2265e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.676+0000 7fcb14fd1740 4 client.20511 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.676+0000 7fcb14fd1740 1 -- 172.21.15.18:0/2637143929 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56016aeaa720 con 0x56016b2241c0 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.676+0000 7fcb0bfff640 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56016b2241c0 0x56016b2265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcaeb7fe640 10 client.20511 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb0bfff640 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56016b2241c0 0x56016b2265e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee39d04f7252abfa server_cookie=cd4a58af62e2d44c in_seq=0 out_seq=0 2024-02-17T13:03:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcaeb7fe640 1 -- 172.21.15.18:0/2637143929 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56016aeaa720 con 0x56016b2241c0 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcaeb7fe640 10 client.20511 handle_command_reply: tid=0 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb14fd1740 1 client.20511 shutdown 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb14fd1740 1 -- 172.21.15.18:0/2637143929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 msgr2=0x56016b2112f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 0x56016b2112f0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4005ab0 tx=0x7fcaf402ca80 comp rx=0 tx=0).stop 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb14fd1740 1 -- 172.21.15.18:0/2637143929 shutdown_connections 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56016b2241c0 0x56016b2265e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b211830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 0x56016b2112f0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcaeb7fe640 0 client.20511 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/2637143929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b210db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb14fd1740 1 -- 172.21.15.18:0/2637143929 >> 172.21.15.18:0/2637143929 conn(0x56016b200b70 msgr2=0x56016b2028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb14fd1740 1 -- 172.21.15.18:0/2637143929 shutdown_connections 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb14fd1740 1 -- 172.21.15.18:0/2637143929 wait complete. 2024-02-17T13:03:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.677+0000 7fcb14fd1740 20 client.20511 trim_cache size 0 max 16384 2024-02-17T13:03:52.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.689+0000 7fcb14fd1740 1 Processor -- start 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.689+0000 7fcb14fd1740 1 -- start start 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.689+0000 7fcb14fd1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b211650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.689+0000 7fcb14fd1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 0x56016b1ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.689+0000 7fcb14fd1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b21ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.689+0000 7fcb14fd1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56016b0f3130 con 0x56016b206a90 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.689+0000 7fcb14fd1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56016b220cd0 con 0x56016b213d50 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.689+0000 7fcb14fd1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56016b2057b0 con 0x56016b206170 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.689+0000 7fcb0b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 0x56016b1ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.689+0000 7fcb0b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 0x56016b1ffb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40934/0 (socket says 172.21.15.18:40934) 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.689+0000 7fcb0b7fe640 1 -- 172.21.15.18:0/1745133260 learned_addr learned my addr 172.21.15.18:0/1745133260 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.690+0000 7fcb0bfff640 1 --2- 172.21.15.18:0/1745133260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b211650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.690+0000 7fcb1099f640 1 --2- 172.21.15.18:0/1745133260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b21ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.690+0000 7fcb0b7fe640 1 -- 172.21.15.18:0/1745133260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 msgr2=0x56016b211650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.690+0000 7fcb0b7fe640 1 --2- 172.21.15.18:0/1745133260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b211650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.690+0000 7fcb0b7fe640 1 -- 172.21.15.18:0/1745133260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 msgr2=0x56016b21ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.690+0000 7fcb0b7fe640 1 --2- 172.21.15.18:0/1745133260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b21ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.690+0000 7fcb0b7fe640 1 -- 172.21.15.18:0/1745133260 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56016b108c20 con 0x56016b206a90 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.690+0000 7fcb1099f640 1 --2- 172.21.15.18:0/1745133260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b21ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.690+0000 7fcb0b7fe640 1 --2- 172.21.15.18:0/1745133260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 0x56016b1ffb50 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009850 tx=0x7fcaf402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad271dc911a750e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.690+0000 7fcb01ffb640 1 -- 172.21.15.18:0/1745133260 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf40035f0 con 0x56016b206a90 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.690+0000 7fcb01ffb640 1 -- 172.21.15.18:0/1745133260 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcaf4003750 con 0x56016b206a90 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.690+0000 7fcb01ffb640 1 -- 172.21.15.18:0/1745133260 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4005530 con 0x56016b206a90 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 -- 172.21.15.18:0/1745133260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 msgr2=0x56016b1ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/1745133260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 0x56016b1ffb50 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009850 tx=0x7fcaf402bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 -- 172.21.15.18:0/1745133260 shutdown_connections 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/1745133260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b213d50 0x56016b21ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/1745133260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b206a90 0x56016b1ffb50 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/1745133260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b211650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 -- 172.21.15.18:0/1745133260 >> 172.21.15.18:0/1745133260 conn(0x56016b200b70 msgr2=0x56016b1ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 -- 172.21.15.18:0/1745133260 shutdown_connections 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 -- 172.21.15.18:0/1745133260 wait complete. 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 -- start start 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 Processor -- start 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b219f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b206a90 0x56016b21a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b213d50 0x56016b274750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56016b274c90 con 0x56016b213d50 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56016b274e00 con 0x56016b206a90 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56016b274fc0 con 0x56016b206170 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb14fd1740 10 client.0 _ll_register_callbacks cb 0x7fcb04009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.691+0000 7fcb1099f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b213d50 0x56016b274750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb0bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b219f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb1099f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b213d50 0x56016b274750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40948/0 (socket says 172.21.15.18:40948) 2024-02-17T13:03:52.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb0bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b219f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:55682/0 (socket says 172.21.15.18:55682) 2024-02-17T13:03:52.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb1099f640 1 -- 172.21.15.18:0/3520832662 learned_addr learned my addr 172.21.15.18:0/3520832662 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb0b7fe640 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b206a90 0x56016b21a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb0bfff640 1 -- 172.21.15.18:0/3520832662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b206a90 msgr2=0x56016b21a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb0bfff640 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b206a90 0x56016b21a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb0bfff640 1 -- 172.21.15.18:0/3520832662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b213d50 msgr2=0x56016b274750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb0bfff640 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b213d50 0x56016b274750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb0bfff640 1 -- 172.21.15.18:0/3520832662 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56016b16c6a0 con 0x56016b206170 2024-02-17T13:03:52.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb1099f640 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b213d50 0x56016b274750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:52.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb0bfff640 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b219f30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0cb190 tx=0x7fcafc0ba040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdb05a22331cd74d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcaeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:03:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcaeb7fe640 1 -- 172.21.15.18:0/3520832662 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0cb410 con 0x56016b206170 2024-02-17T13:03:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcaeb7fe640 1 -- 172.21.15.18:0/3520832662 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fcafc0cb570 con 0x56016b206170 2024-02-17T13:03:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb14fd1740 1 -- 172.21.15.18:0/3520832662 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb0401a070 con 0x56016b206170 2024-02-17T13:03:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcaeb7fe640 1 -- 172.21.15.18:0/3520832662 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcafc096b50 con 0x56016b206170 2024-02-17T13:03:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb14fd1740 10 client.29443 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.692+0000 7fcb14fd1740 1 -- 172.21.15.18:0/3520832662 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x56016b12e8c0 con 0x56016b206170 2024-02-17T13:03:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.693+0000 7fcaeb7fe640 1 -- 172.21.15.18:0/3520832662 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcafc0a6030 con 0x56016b206170 2024-02-17T13:03:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.693+0000 7fcaeb7fe640 1 -- 172.21.15.18:0/3520832662 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fcafc0b8ba0 con 0x56016b206170 2024-02-17T13:03:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.693+0000 7fcb14fd1740 10 client.29443 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.693+0000 7fcb14fd1740 10 client.29443 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-02-17T13:03:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.693+0000 7fcb14fd1740 20 client.29443 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.693+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56016b27c7e0 0x56016b27ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.693+0000 7fcb14fd1740 4 client.29443 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.693+0000 7fcb14fd1740 1 -- 172.21.15.18:0/3520832662 --> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56016aef9890 con 0x56016b27c7e0 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.693+0000 7fcb0b7fe640 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56016b27c7e0 0x56016b27ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.693+0000 7fcaeb7fe640 10 client.29443 ms_handle_connect on v2:172.21.15.119:6832/21392902 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.693+0000 7fcb0b7fe640 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56016b27c7e0 0x56016b27ec00 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cffe9d8cea3c1799 server_cookie=abc44f069b31e8c2 in_seq=0 out_seq=0 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcaeb7fe640 1 -- 172.21.15.18:0/3520832662 <== mds.4 v2:172.21.15.119:6832/21392902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56016aef9890 con 0x56016b27c7e0 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcaeb7fe640 10 client.29443 handle_command_reply: tid=0 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb14fd1740 1 client.29443 shutdown 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb14fd1740 1 -- 172.21.15.18:0/3520832662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 msgr2=0x56016b219f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b219f30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0cb190 tx=0x7fcafc0ba040 comp rx=0 tx=0).stop 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb14fd1740 1 -- 172.21.15.18:0/3520832662 shutdown_connections 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56016b213d50 0x56016b274750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.119:6832/21392902,v1:172.21.15.119:6833/21392902] conn(0x56016b27c7e0 0x56016b27ec00 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56016b206a90 0x56016b21a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb14fd1740 1 --2- 172.21.15.18:0/3520832662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56016b206170 0x56016b219f30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcaeb7fe640 0 client.29443 ms_handle_reset on v2:172.21.15.119:6832/21392902 2024-02-17T13:03:52.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb14fd1740 1 -- 172.21.15.18:0/3520832662 >> 172.21.15.18:0/3520832662 conn(0x56016b200b70 msgr2=0x56016b1ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb14fd1740 1 -- 172.21.15.18:0/3520832662 shutdown_connections 2024-02-17T13:03:52.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb14fd1740 1 -- 172.21.15.18:0/3520832662 wait complete. 2024-02-17T13:03:52.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb14fd1740 20 client.29443 trim_cache size 0 max 16384 2024-02-17T13:03:52.692 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:03:52.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb111a0640 1 -- 172.21.15.18:0/513858634 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcaf80a2230 msgr2=0x7fcaf80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb111a0640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcaf80a2230 0x7fcaf80a46f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcb04022000 tx=0x7fcb0401e000 comp rx=0 tx=0).stop 2024-02-17T13:03:52.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb111a0640 1 -- 172.21.15.18:0/513858634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb0c100150 msgr2=0x7fcb0c19d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.694+0000 7fcb111a0640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb0c100150 0x7fcb0c19d4d0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00d770 tx=0x7fcafc00dc30 comp rx=0 tx=0).stop 2024-02-17T13:03:52.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.695+0000 7fcb111a0640 1 -- 172.21.15.18:0/513858634 shutdown_connections 2024-02-17T13:03:52.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.695+0000 7fcb111a0640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fcaf80a2230 0x7fcaf80a46f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.695+0000 7fcb111a0640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb0c197680 0x7fcb0c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.695+0000 7fcb111a0640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c105220 0x7fcb0c19da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.695+0000 7fcb111a0640 1 --2- 172.21.15.18:0/513858634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb0c100150 0x7fcb0c19d4d0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.695+0000 7fcb111a0640 1 -- 172.21.15.18:0/513858634 >> 172.21.15.18:0/513858634 conn(0x7fcb0c0fb9d0 msgr2=0x7fcb0c0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.695+0000 7fcb111a0640 1 -- 172.21.15.18:0/513858634 shutdown_connections 2024-02-17T13:03:52.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.695+0000 7fcb111a0640 1 -- 172.21.15.18:0/513858634 wait complete. 2024-02-17T13:03:52.695 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:03:52.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-17T13:03:52.718 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:52 smithi018 ceph-mon[32625]: pgmap v439: 97 pgs: 97 active+clean; 629 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 736 op/s 2024-02-17T13:03:52.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 35331 32.49 MB/sec warmup 64 sec latency 133.258 ms 2024-02-17T13:03:52.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.825+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3152695726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bbc100c10 msgr2=0x7f6bbc104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.825+0000 7f6bc2be2640 1 --2- 172.21.15.18:0/3152695726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bbc100c10 0x7f6bbc104c50 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009930 tx=0x7f6ba802bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:52.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.825+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3152695726 shutdown_connections 2024-02-17T13:03:52.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.825+0000 7f6bc2be2640 1 --2- 172.21.15.18:0/3152695726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc105330 0x7f6bbc10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.825+0000 7f6bc2be2640 1 --2- 172.21.15.18:0/3152695726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bbc100c10 0x7f6bbc104c50 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.825+0000 7f6bc2be2640 1 --2- 172.21.15.18:0/3152695726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bbc100260 0x7f6bbc100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.825+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3152695726 >> 172.21.15.18:0/3152695726 conn(0x7f6bbc077180 msgr2=0x7f6bbc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.825+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3152695726 shutdown_connections 2024-02-17T13:03:52.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.825+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3152695726 wait complete. 2024-02-17T13:03:52.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bc2be2640 1 Processor -- start 2024-02-17T13:03:52.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bc2be2640 1 -- start start 2024-02-17T13:03:52.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bc2be2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bbc100260 0x7f6bbc19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bc2be2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bbc105330 0x7f6bbc19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bc2be2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc197640 0x7f6bbc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bc2be2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc198110 con 0x7f6bbc100260 2024-02-17T13:03:52.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bc2be2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc198280 con 0x7f6bbc105330 2024-02-17T13:03:52.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bc2be2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc198440 con 0x7f6bbc197640 2024-02-17T13:03:52.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bc097e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bbc100260 0x7f6bbc19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bc097e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bbc100260 0x7f6bbc19d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40978/0 (socket says 172.21.15.18:40978) 2024-02-17T13:03:52.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bc097e640 1 -- 172.21.15.18:0/3254459183 learned_addr learned my addr 172.21.15.18:0/3254459183 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bc117f640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc197640 0x7f6bbc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.826+0000 7f6bbbfff640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bbc105330 0x7f6bbc19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.827+0000 7f6bc097e640 1 -- 172.21.15.18:0/3254459183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc197640 msgr2=0x7f6bbc197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.827+0000 7f6bc097e640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc197640 0x7f6bbc197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.827+0000 7f6bc097e640 1 -- 172.21.15.18:0/3254459183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bbc105330 msgr2=0x7f6bbc19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.827+0000 7f6bc097e640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bbc105330 0x7f6bbc19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.827+0000 7f6bc097e640 1 -- 172.21.15.18:0/3254459183 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba8009560 con 0x7f6bbc100260 2024-02-17T13:03:52.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.827+0000 7f6bc097e640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bbc100260 0x7f6bbc19d580 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000e9f0 tx=0x7f6bb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.827+0000 7f6bb9ffb640 1 -- 172.21.15.18:0/3254459183 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0009700 con 0x7f6bbc100260 2024-02-17T13:03:52.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.827+0000 7f6bb9ffb640 1 -- 172.21.15.18:0/3254459183 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bb000cd80 con 0x7f6bbc100260 2024-02-17T13:03:52.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.827+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3254459183 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc1a43e0 con 0x7f6bbc100260 2024-02-17T13:03:52.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.827+0000 7f6bb9ffb640 1 -- 172.21.15.18:0/3254459183 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0018920 con 0x7f6bbc100260 2024-02-17T13:03:52.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.827+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3254459183 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc1a48c0 con 0x7f6bbc100260 2024-02-17T13:03:52.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.829+0000 7f6bb9ffb640 1 -- 172.21.15.18:0/3254459183 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6bb0018a80 con 0x7f6bbc100260 2024-02-17T13:03:52.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc57b9740 1 Processor -- start 2024-02-17T13:03:52.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc57b9740 1 -- start start 2024-02-17T13:03:52.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc57b9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8250 0x5555620c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc57b9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 0x5555620d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc57b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 0x5555620d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc57b9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555562083960 con 0x5555620c8b70 2024-02-17T13:03:52.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc57b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555562037310 con 0x5555620d1750 2024-02-17T13:03:52.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc57b9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555562034840 con 0x5555620c8250 2024-02-17T13:03:52.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc097e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 0x5555620d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc117f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8250 0x5555620c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc097e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 0x5555620d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40994/0 (socket says 172.21.15.18:40994) 2024-02-17T13:03:52.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc097e640 1 -- 172.21.15.18:0/3627669476 learned_addr learned my addr 172.21.15.18:0/3627669476 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bb9ffb640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6bac0a2230 0x7f6bac0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bbbfff640 1 --2- 172.21.15.18:0/3627669476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 0x5555620d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.830+0000 7f6bc117f640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6bac0a2230 0x7f6bac0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.831+0000 7f6bc097e640 1 -- 172.21.15.18:0/3627669476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8250 msgr2=0x5555620c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.831+0000 7f6bc097e640 1 --2- 172.21.15.18:0/3627669476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8250 0x5555620c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.831+0000 7f6bc097e640 1 -- 172.21.15.18:0/3627669476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 msgr2=0x5555620d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.831+0000 7f6bc097e640 1 --2- 172.21.15.18:0/3627669476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 0x5555620d3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.831+0000 7f6bb9ffb640 1 -- 172.21.15.18:0/3254459183 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6bb0014070 con 0x7f6bbc100260 2024-02-17T13:03:52.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.831+0000 7f6bc097e640 1 -- 172.21.15.18:0/3627669476 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555561ff2060 con 0x5555620c8b70 2024-02-17T13:03:52.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.831+0000 7f6bc117f640 1 --2- 172.21.15.18:0/3627669476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8250 0x5555620c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:52.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.831+0000 7f6bc097e640 1 --2- 172.21.15.18:0/3627669476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 0x5555620d1210 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f6bb009abb0 tx=0x7f6bb0010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb6cd899806d010 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.831+0000 7f6ba6ffd640 1 -- 172.21.15.18:0/3627669476 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00d1850 con 0x5555620c8b70 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.831+0000 7f6bc117f640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6bac0a2230 0x7f6bac0a46f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009fd0 tx=0x7f6bb4004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.831+0000 7f6ba6ffd640 1 -- 172.21.15.18:0/3627669476 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bb009aeb0 con 0x5555620c8b70 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.831+0000 7f6ba6ffd640 1 -- 172.21.15.18:0/3627669476 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0010c90 con 0x5555620c8b70 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.832+0000 7f6bc57b9740 1 -- 172.21.15.18:0/3627669476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 msgr2=0x5555620d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.832+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/3627669476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 0x5555620d1210 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f6bb009abb0 tx=0x7f6bb0010f50 comp rx=0 tx=0).stop 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.832+0000 7f6bc57b9740 1 -- 172.21.15.18:0/3627669476 shutdown_connections 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.832+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/3627669476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 0x5555620d3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.832+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/3627669476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 0x5555620d1210 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.832+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/3627669476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8250 0x5555620c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.832+0000 7f6bc57b9740 1 -- 172.21.15.18:0/3627669476 >> 172.21.15.18:0/3627669476 conn(0x5555620c2980 msgr2=0x5555620c2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.832+0000 7f6bc57b9740 1 -- 172.21.15.18:0/3627669476 shutdown_connections 2024-02-17T13:03:52.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.832+0000 7f6bc57b9740 1 -- 172.21.15.18:0/3627669476 wait complete. 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.832+0000 7f6bc57b9740 1 -- start start 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bc57b9740 1 Processor -- start 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bc57b9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 0x5555620ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bc57b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620c8b70 0x5555620cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bc57b9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620d1750 0x5555620cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bc57b9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555562030020 con 0x5555620c8250 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bc57b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555561d03430 con 0x5555620c8b70 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bc57b9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x555561cbfe00 con 0x5555620d1750 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bc57b9740 10 client.0 _ll_register_callbacks cb 0x555561fd7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bc097e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 0x5555620ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bc097e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 0x5555620ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41002/0 (socket says 172.21.15.18:41002) 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bc097e640 1 -- 172.21.15.18:0/1385769357 learned_addr learned my addr 172.21.15.18:0/1385769357 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bc117f640 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620d1750 0x5555620cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.833+0000 7f6bbbfff640 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620c8b70 0x5555620cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6bc097e640 1 -- 172.21.15.18:0/1385769357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620d1750 msgr2=0x5555620cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6bc097e640 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620d1750 0x5555620cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6bc097e640 1 -- 172.21.15.18:0/1385769357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620c8b70 msgr2=0x5555620cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6bc097e640 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620c8b70 0x5555620cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6bc097e640 1 -- 172.21.15.18:0/1385769357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc1a48c0 con 0x5555620c8250 2024-02-17T13:03:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6bc097e640 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 0x5555620ce740 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00cbc10 tx=0x7f6bb00ce3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5045c6c168e1d15f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6b9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6b9bfff640 1 -- 172.21.15.18:0/1385769357 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb009d070 con 0x5555620c8250 2024-02-17T13:03:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6b9bfff640 1 -- 172.21.15.18:0/1385769357 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6bb00a35a0 con 0x5555620c8250 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1385769357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555561d2f7e0 con 0x5555620c8250 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6b9bfff640 1 -- 172.21.15.18:0/1385769357 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bb00991b0 con 0x5555620c8250 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6bc57b9740 10 client.20529 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1385769357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x55556202e6a0 con 0x5555620c8250 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.834+0000 7f6b9bfff640 1 -- 172.21.15.18:0/1385769357 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555561d2f7e0 con 0x5555620c8250 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.835+0000 7f6b9bfff640 1 -- 172.21.15.18:0/1385769357 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f6bb00cd020 con 0x5555620c8250 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.835+0000 7f6bc57b9740 10 client.20529 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.835+0000 7f6bc57b9740 10 client.20529 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.835+0000 7f6bc57b9740 20 client.20529 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.835+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5555620e1800 0x5555620e3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.835+0000 7f6bc57b9740 4 client.20529 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.835+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1385769357 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555561d6c720 con 0x5555620e1800 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.835+0000 7f6bbbfff640 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5555620e1800 0x5555620e3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.835+0000 7f6bbbfff640 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5555620e1800 0x5555620e3c30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7c30b9c4552707ef server_cookie=c108095e692125a9 in_seq=0 out_seq=0 2024-02-17T13:03:52.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.835+0000 7f6b9bfff640 10 client.20529 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:03:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6b9bfff640 1 -- 172.21.15.18:0/1385769357 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555561d6c720 con 0x5555620e1800 2024-02-17T13:03:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6b9bfff640 10 client.20529 handle_command_reply: tid=0 2024-02-17T13:03:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6bc57b9740 1 client.20529 shutdown 2024-02-17T13:03:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1385769357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 msgr2=0x5555620ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 0x5555620ce740 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00cbc10 tx=0x7f6bb00ce3d0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1385769357 shutdown_connections 2024-02-17T13:03:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5555620e1800 0x5555620e3c30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620d1750 0x5555620cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6b9bfff640 0 client.20529 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:03:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620c8b70 0x5555620cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/1385769357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 0x5555620ce740 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1385769357 >> 172.21.15.18:0/1385769357 conn(0x5555620c2980 msgr2=0x5555620c46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1385769357 shutdown_connections 2024-02-17T13:03:52.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.838+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1385769357 wait complete. 2024-02-17T13:03:52.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.839+0000 7f6bc57b9740 20 client.20529 trim_cache size 0 max 16384 2024-02-17T13:03:52.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:52 smithi119 ceph-mon[36188]: pgmap v439: 97 pgs: 97 active+clean; 629 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 736 op/s 2024-02-17T13:03:52.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.851+0000 7f6bc57b9740 1 Processor -- start 2024-02-17T13:03:52.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.851+0000 7f6bc57b9740 1 -- start start 2024-02-17T13:03:52.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc57b9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 0x5555620cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc57b9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8b70 0x5555620c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc57b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 0x5555620d85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc57b9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x555561fb5130 con 0x5555620c8250 2024-02-17T13:03:52.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc57b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555620de5b0 con 0x5555620d1750 2024-02-17T13:03:52.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc57b9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555620c7cc0 con 0x5555620c8b70 2024-02-17T13:03:52.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc097e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 0x5555620cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc097e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 0x5555620cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41008/0 (socket says 172.21.15.18:41008) 2024-02-17T13:03:52.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc097e640 1 -- 172.21.15.18:0/4140210609 learned_addr learned my addr 172.21.15.18:0/4140210609 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bbbfff640 1 --2- 172.21.15.18:0/4140210609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8b70 0x5555620c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc117f640 1 --2- 172.21.15.18:0/4140210609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 0x5555620d85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc097e640 1 -- 172.21.15.18:0/4140210609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8b70 msgr2=0x5555620c1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc097e640 1 --2- 172.21.15.18:0/4140210609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8b70 0x5555620c1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc097e640 1 -- 172.21.15.18:0/4140210609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 msgr2=0x5555620d85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc097e640 1 --2- 172.21.15.18:0/4140210609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 0x5555620d85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc097e640 1 -- 172.21.15.18:0/4140210609 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555561fcac20 con 0x5555620c8250 2024-02-17T13:03:52.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.852+0000 7f6bc097e640 1 --2- 172.21.15.18:0/4140210609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 0x5555620cefe0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0097310 tx=0x7f6bb0010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dee53aedbb78069 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.853+0000 7f6ba67fc640 1 -- 172.21.15.18:0/4140210609 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb009d070 con 0x5555620c8250 2024-02-17T13:03:52.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.853+0000 7f6ba67fc640 1 -- 172.21.15.18:0/4140210609 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bb00d2020 con 0x5555620c8250 2024-02-17T13:03:52.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.853+0000 7f6ba67fc640 1 -- 172.21.15.18:0/4140210609 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0096c00 con 0x5555620c8250 2024-02-17T13:03:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.853+0000 7f6bc57b9740 1 -- 172.21.15.18:0/4140210609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 msgr2=0x5555620cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.853+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/4140210609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 0x5555620cefe0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0097310 tx=0x7f6bb0010f50 comp rx=0 tx=0).stop 2024-02-17T13:03:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.853+0000 7f6bc57b9740 1 -- 172.21.15.18:0/4140210609 shutdown_connections 2024-02-17T13:03:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.853+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/4140210609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 0x5555620d85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.853+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/4140210609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8b70 0x5555620c1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.853+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/4140210609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8250 0x5555620cefe0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.853+0000 7f6bc57b9740 1 -- 172.21.15.18:0/4140210609 >> 172.21.15.18:0/4140210609 conn(0x5555620c2980 msgr2=0x5555620c12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.854+0000 7f6bc57b9740 1 -- 172.21.15.18:0/4140210609 shutdown_connections 2024-02-17T13:03:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.854+0000 7f6bc57b9740 1 -- 172.21.15.18:0/4140210609 wait complete. 2024-02-17T13:03:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.854+0000 7f6bc57b9740 1 -- start start 2024-02-17T13:03:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.854+0000 7f6bc57b9740 1 Processor -- start 2024-02-17T13:03:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.854+0000 7f6bc57b9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8250 0x5555620d70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.854+0000 7f6bc57b9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 0x5555620d7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.854+0000 7f6bc57b9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 0x5555620d7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.854+0000 7f6bc57b9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556213c130 con 0x5555620c8b70 2024-02-17T13:03:52.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.854+0000 7f6bc57b9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556213c2a0 con 0x5555620d1750 2024-02-17T13:03:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.854+0000 7f6bc57b9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556213c460 con 0x5555620c8250 2024-02-17T13:03:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bc57b9740 10 client.0 _ll_register_callbacks cb 0x7f6bb0096f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bbbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 0x5555620d7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bc097e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8250 0x5555620d70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bbbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 0x5555620d7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41016/0 (socket says 172.21.15.18:41016) 2024-02-17T13:03:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bbbfff640 1 -- 172.21.15.18:0/1412032982 learned_addr learned my addr 172.21.15.18:0/1412032982 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bc117f640 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 0x5555620d7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bc097e640 1 -- 172.21.15.18:0/1412032982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 msgr2=0x5555620d7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bc097e640 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 0x5555620d7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bc097e640 1 -- 172.21.15.18:0/1412032982 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 msgr2=0x5555620d7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bc097e640 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 0x5555620d7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bc097e640 1 -- 172.21.15.18:0/1412032982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55556202e6a0 con 0x5555620c8250 2024-02-17T13:03:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bbbfff640 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 0x5555620d7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bc097e640 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8250 0x5555620d70e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0096b90 tx=0x7f6bb009ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0a0d9277da151ff server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6b9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:03:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6b9bfff640 1 -- 172.21.15.18:0/1412032982 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb009d070 con 0x5555620c8250 2024-02-17T13:03:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1412032982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bb009aeb0 con 0x5555620c8250 2024-02-17T13:03:52.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.855+0000 7f6b9bfff640 1 -- 172.21.15.18:0/1412032982 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6bb005cbf0 con 0x5555620c8250 2024-02-17T13:03:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.856+0000 7f6b9bfff640 1 -- 172.21.15.18:0/1412032982 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bb005cdb0 con 0x5555620c8250 2024-02-17T13:03:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.856+0000 7f6bc57b9740 10 client.29461 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.856+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1412032982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x555561ff08c0 con 0x5555620c8250 2024-02-17T13:03:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.856+0000 7f6b9bfff640 1 -- 172.21.15.18:0/1412032982 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bb009aeb0 con 0x5555620c8250 2024-02-17T13:03:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.856+0000 7f6b9bfff640 1 -- 172.21.15.18:0/1412032982 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f6bb00cd070 con 0x5555620c8250 2024-02-17T13:03:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.856+0000 7f6bc57b9740 10 client.29461 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:52.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.856+0000 7f6bc57b9740 10 client.29461 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-02-17T13:03:52.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.856+0000 7f6bc57b9740 20 client.29461 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:52.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.856+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5555620e1800 0x55556213f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:52.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.856+0000 7f6bc57b9740 4 client.29461 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:03:52.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.856+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1412032982 --> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555561dbb890 con 0x5555620e1800 2024-02-17T13:03:52.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.856+0000 7f6bbbfff640 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5555620e1800 0x55556213f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:52.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.857+0000 7f6bbbfff640 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5555620e1800 0x55556213f7b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=391dd74147c77b52 server_cookie=edbe7e16b399bc01 in_seq=0 out_seq=0 2024-02-17T13:03:52.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.857+0000 7f6b9bfff640 10 client.29461 ms_handle_connect on v2:172.21.15.146:6834/405923223 2024-02-17T13:03:52.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6b9bfff640 1 -- 172.21.15.18:0/1412032982 <== mds.3 v2:172.21.15.146:6834/405923223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555561dbb890 con 0x5555620e1800 2024-02-17T13:03:52.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6b9bfff640 10 client.29461 handle_command_reply: tid=0 2024-02-17T13:03:52.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6bc57b9740 1 client.29461 shutdown 2024-02-17T13:03:52.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1412032982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8250 msgr2=0x5555620d70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8250 0x5555620d70e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0096b90 tx=0x7f6bb009ac80 comp rx=0 tx=0).stop 2024-02-17T13:03:52.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1412032982 shutdown_connections 2024-02-17T13:03:52.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.146:6834/405923223,v1:172.21.15.146:6835/405923223] conn(0x5555620e1800 0x55556213f7b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5555620d1750 0x5555620d7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555620c8b70 0x5555620d7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6b9bfff640 0 client.29461 ms_handle_reset on v2:172.21.15.146:6834/405923223 2024-02-17T13:03:52.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6bc57b9740 1 --2- 172.21.15.18:0/1412032982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5555620c8250 0x5555620d70e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1412032982 >> 172.21.15.18:0/1412032982 conn(0x5555620c2980 msgr2=0x5555620c1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1412032982 shutdown_connections 2024-02-17T13:03:52.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6bc57b9740 1 -- 172.21.15.18:0/1412032982 wait complete. 2024-02-17T13:03:52.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.860+0000 7f6bc57b9740 20 client.29461 trim_cache size 0 max 16384 2024-02-17T13:03:52.861 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:03:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.861+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3254459183 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6bac0a2230 msgr2=0x7f6bac0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.861+0000 7f6bc2be2640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6bac0a2230 0x7f6bac0a46f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009fd0 tx=0x7f6bb4004ab0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.861+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3254459183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bbc100260 msgr2=0x7f6bbc19d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.861+0000 7f6bc2be2640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bbc100260 0x7f6bbc19d580 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000e9f0 tx=0x7f6bb000eeb0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.861+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3254459183 shutdown_connections 2024-02-17T13:03:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.861+0000 7f6bc2be2640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6bac0a2230 0x7f6bac0a46f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.861+0000 7f6bc2be2640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bbc197640 0x7f6bbc197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.861+0000 7f6bc2be2640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bbc105330 0x7f6bbc19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.861+0000 7f6bc2be2640 1 --2- 172.21.15.18:0/3254459183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bbc100260 0x7f6bbc19d580 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:52.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.861+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3254459183 >> 172.21.15.18:0/3254459183 conn(0x7f6bbc077180 msgr2=0x7f6bbc0686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:52.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.861+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3254459183 shutdown_connections 2024-02-17T13:03:52.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:52.861+0000 7f6bc2be2640 1 -- 172.21.15.18:0/3254459183 wait complete. 2024-02-17T13:03:52.866 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:03:52.866 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T13:03:52.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:52 smithi146 ceph-mon[36454]: pgmap v439: 97 pgs: 97 active+clean; 629 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 736 op/s 2024-02-17T13:03:53.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.059+0000 7f194cf02640 1 -- 172.21.15.18:0/2198042032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1948068fa0 msgr2=0x7f1948105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.059+0000 7f194cf02640 1 --2- 172.21.15.18:0/2198042032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1948068fa0 0x7f1948105de0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f1940009cd0 tx=0x7f194002c0f0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.059+0000 7f194cf02640 1 -- 172.21.15.18:0/2198042032 shutdown_connections 2024-02-17T13:03:53.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.059+0000 7f194cf02640 1 --2- 172.21.15.18:0/2198042032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19481064c0 0x7f194810e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.059+0000 7f194cf02640 1 --2- 172.21.15.18:0/2198042032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1948068fa0 0x7f1948105de0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.059+0000 7f194cf02640 1 --2- 172.21.15.18:0/2198042032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19480685f0 0x7f19480689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.059+0000 7f194cf02640 1 -- 172.21.15.18:0/2198042032 >> 172.21.15.18:0/2198042032 conn(0x7f1948077180 msgr2=0x7f1948077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.059+0000 7f194cf02640 1 -- 172.21.15.18:0/2198042032 shutdown_connections 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.059+0000 7f194cf02640 1 -- 172.21.15.18:0/2198042032 wait complete. 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.060+0000 7f194cf02640 1 Processor -- start 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.060+0000 7f194cf02640 1 -- start start 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.060+0000 7f194cf02640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19480685f0 0x7f194819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.060+0000 7f194cf02640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19481064c0 0x7f194819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.060+0000 7f194cf02640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1948197660 0x7f1948197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.060+0000 7f194cf02640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1948198130 con 0x7f1948197660 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.060+0000 7f194cf02640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19481982a0 con 0x7f19480685f0 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.060+0000 7f194cf02640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1948198460 con 0x7f19481064c0 2024-02-17T13:03:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f1946d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1948197660 0x7f1948197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f1945d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19481064c0 0x7f194819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f1945d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19481064c0 0x7f194819dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:55782/0 (socket says 172.21.15.18:55782) 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f1945d9b640 1 -- 172.21.15.18:0/2644695515 learned_addr learned my addr 172.21.15.18:0/2644695515 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f1946d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1948197660 0x7f1948197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41038/0 (socket says 172.21.15.18:41038) 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f194659c640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19480685f0 0x7f194819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f1945d9b640 1 -- 172.21.15.18:0/2644695515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19480685f0 msgr2=0x7f194819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f1945d9b640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19480685f0 0x7f194819d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f1945d9b640 1 -- 172.21.15.18:0/2644695515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1948197660 msgr2=0x7f1948197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f1945d9b640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1948197660 0x7f1948197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f1945d9b640 1 -- 172.21.15.18:0/2644695515 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1940009900 con 0x7f19481064c0 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f1945d9b640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19481064c0 0x7f194819dae0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1940005cc0 tx=0x7f1940034d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.061+0000 7f19377fe640 1 -- 172.21.15.18:0/2644695515 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400328c0 con 0x7f19481064c0 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.062+0000 7f19377fe640 1 -- 172.21.15.18:0/2644695515 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1940003e50 con 0x7f19481064c0 2024-02-17T13:03:53.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.062+0000 7f194cf02640 1 -- 172.21.15.18:0/2644695515 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19481a43a0 con 0x7f19481064c0 2024-02-17T13:03:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.062+0000 7f19377fe640 1 -- 172.21.15.18:0/2644695515 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940040450 con 0x7f19481064c0 2024-02-17T13:03:53.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.062+0000 7f194cf02640 1 -- 172.21.15.18:0/2644695515 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19481a4760 con 0x7f19481064c0 2024-02-17T13:03:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.064+0000 7f19377fe640 1 -- 172.21.15.18:0/2644695515 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f194000a040 con 0x7f19481064c0 2024-02-17T13:03:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.065+0000 7f194fbb4740 1 Processor -- start 2024-02-17T13:03:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.065+0000 7f194fbb4740 1 -- start start 2024-02-17T13:03:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.065+0000 7f194fbb4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f1fa0 0x563d234f2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.065+0000 7f194fbb4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234f2a40 0x563d234f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.065+0000 7f194fbb4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234ffea0 0x563d23502290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.065+0000 7f19377fe640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f19300a22b0 0x7f19300a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.065+0000 7f194fbb4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d234a9960 con 0x563d234f1fa0 2024-02-17T13:03:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.065+0000 7f194fbb4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d2345d310 con 0x563d234ffea0 2024-02-17T13:03:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.065+0000 7f194fbb4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d2345a840 con 0x563d234f2a40 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f1945d9b640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f19300a22b0 0x7f19300a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f1946d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234ffea0 0x563d23502290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f1946d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f1fa0 0x563d234f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f19377fe640 1 -- 172.21.15.18:0/2644695515 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f19400f0c80 con 0x7f19481064c0 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f1946d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f1fa0 0x563d234f2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41054/0 (socket says 172.21.15.18:41054) 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f1946d9d640 1 -- 172.21.15.18:0/1406628478 learned_addr learned my addr 172.21.15.18:0/1406628478 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f194659c640 1 --2- 172.21.15.18:0/1406628478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234f2a40 0x563d234f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f1946d9d640 1 -- 172.21.15.18:0/1406628478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234f2a40 msgr2=0x563d234f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f1946d9d640 1 --2- 172.21.15.18:0/1406628478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234f2a40 0x563d234f2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f1946d9d640 1 -- 172.21.15.18:0/1406628478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234ffea0 msgr2=0x563d23502290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f1946d9d640 1 --2- 172.21.15.18:0/1406628478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234ffea0 0x563d23502290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f1946d9d640 1 -- 172.21.15.18:0/1406628478 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d23418060 con 0x563d234f1fa0 2024-02-17T13:03:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.066+0000 7f194659c640 1 --2- 172.21.15.18:0/1406628478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234f2a40 0x563d234f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:03:53.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.067+0000 7f1945d9b640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f19300a22b0 0x7f19300a4770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f19400bd6d0 tx=0x7f19400b2e10 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.067+0000 7f1946d9d640 1 --2- 172.21.15.18:0/1406628478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f1fa0 0x563d234f2470 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f193800bef0 tx=0x7f1938005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b754ff767d4e44ff server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.067+0000 7f1934ff9640 1 -- 172.21.15.18:0/1406628478 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193800f720 con 0x563d234f1fa0 2024-02-17T13:03:53.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.067+0000 7f1934ff9640 1 -- 172.21.15.18:0/1406628478 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f193800fd60 con 0x563d234f1fa0 2024-02-17T13:03:53.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.067+0000 7f1934ff9640 1 -- 172.21.15.18:0/1406628478 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193801ed40 con 0x563d234f1fa0 2024-02-17T13:03:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.068+0000 7f194fbb4740 1 -- 172.21.15.18:0/1406628478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f1fa0 msgr2=0x563d234f2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.068+0000 7f194fbb4740 1 --2- 172.21.15.18:0/1406628478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f1fa0 0x563d234f2470 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f193800bef0 tx=0x7f1938005cc0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.068+0000 7f194fbb4740 1 -- 172.21.15.18:0/1406628478 shutdown_connections 2024-02-17T13:03:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.068+0000 7f194fbb4740 1 --2- 172.21.15.18:0/1406628478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234ffea0 0x563d23502290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.068+0000 7f194fbb4740 1 --2- 172.21.15.18:0/1406628478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234f2a40 0x563d234f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.068+0000 7f194fbb4740 1 --2- 172.21.15.18:0/1406628478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f1fa0 0x563d234f2470 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.068+0000 7f194fbb4740 1 -- 172.21.15.18:0/1406628478 >> 172.21.15.18:0/1406628478 conn(0x563d234eccd0 msgr2=0x563d234ed0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.068+0000 7f194fbb4740 1 -- 172.21.15.18:0/1406628478 shutdown_connections 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.068+0000 7f194fbb4740 1 -- 172.21.15.18:0/1406628478 wait complete. 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f194fbb4740 1 -- start start 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f194fbb4740 1 Processor -- start 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f194fbb4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234f1fa0 0x563d234e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f194fbb4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d234e97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f194fbb4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234ffea0 0x563d234e9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f194fbb4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23456020 con 0x563d234f2a40 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f194fbb4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23129430 con 0x563d234ffea0 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f194fbb4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d230e5e00 con 0x563d234f1fa0 2024-02-17T13:03:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f194659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d234e97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f194fbb4740 10 client.0 _ll_register_callbacks cb 0x563d233fdcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f194659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d234e97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41070/0 (socket says 172.21.15.18:41070) 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f194659c640 1 -- 172.21.15.18:0/3853159632 learned_addr learned my addr 172.21.15.18:0/3853159632 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.069+0000 7f1946d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234f1fa0 0x563d234e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.070+0000 7f194659c640 1 -- 172.21.15.18:0/3853159632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234f1fa0 msgr2=0x563d234e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.070+0000 7f194659c640 1 --2- 172.21.15.18:0/3853159632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234f1fa0 0x563d234e9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.070+0000 7f194659c640 1 -- 172.21.15.18:0/3853159632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234ffea0 msgr2=0x563d234e9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.070+0000 7f194659c640 1 --2- 172.21.15.18:0/3853159632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234ffea0 0x563d234e9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.070+0000 7f194659c640 1 -- 172.21.15.18:0/3853159632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19380123f0 con 0x563d234f2a40 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.070+0000 7f194659c640 1 --2- 172.21.15.18:0/3853159632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d234e97c0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f193c0104d0 tx=0x7f193c010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144e912712fd40c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.070+0000 7f192dffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.070+0000 7f192dffb640 1 -- 172.21.15.18:0/3853159632 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c014800 con 0x563d234f2a40 2024-02-17T13:03:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.070+0000 7f192dffb640 1 -- 172.21.15.18:0/3853159632 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f193c014960 con 0x563d234f2a40 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.070+0000 7f194fbb4740 1 -- 172.21.15.18:0/3853159632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d231557e0 con 0x563d234f2a40 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.071+0000 7f192dffb640 1 -- 172.21.15.18:0/3853159632 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f193c014b20 con 0x563d234f2a40 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.071+0000 7f194fbb4740 10 client.20547 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.071+0000 7f194fbb4740 1 -- 172.21.15.18:0/3853159632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x563d234546a0 con 0x563d234f2a40 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.071+0000 7f192dffb640 1 -- 172.21.15.18:0/3853159632 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f193c014c80 con 0x563d234f2a40 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.071+0000 7f192dffb640 1 -- 172.21.15.18:0/3853159632 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f193c012390 con 0x563d234f2a40 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.071+0000 7f194fbb4740 10 client.20547 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.071+0000 7f194fbb4740 10 client.20547 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.071+0000 7f194fbb4740 20 client.20547 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.071+0000 7f194fbb4740 1 --2- 172.21.15.18:0/3853159632 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563d23510310 0x563d23512770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.072+0000 7f194fbb4740 4 client.20547 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.072+0000 7f194fbb4740 1 -- 172.21.15.18:0/3853159632 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d231557e0 con 0x563d23510310 2024-02-17T13:03:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.072+0000 7f1946d9d640 1 --2- 172.21.15.18:0/3853159632 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563d23510310 0x563d23512770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.073+0000 7f1946d9d640 1 --2- 172.21.15.18:0/3853159632 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563d23510310 0x563d23512770 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6f951e758fea0d4 server_cookie=ab0e531ec6af5ef7 in_seq=0 out_seq=0 2024-02-17T13:03:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.073+0000 7f192dffb640 10 client.20547 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:03:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f192dffb640 1 -- 172.21.15.18:0/3853159632 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d231557e0 con 0x563d23510310 2024-02-17T13:03:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f192dffb640 10 client.20547 handle_command_reply: tid=0 2024-02-17T13:03:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f194fbb4740 1 client.20547 shutdown 2024-02-17T13:03:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f194fbb4740 1 -- 172.21.15.18:0/3853159632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 msgr2=0x563d234e97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f194fbb4740 1 --2- 172.21.15.18:0/3853159632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d234e97c0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f193c0104d0 tx=0x7f193c010990 comp rx=0 tx=0).stop 2024-02-17T13:03:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f194fbb4740 1 -- 172.21.15.18:0/3853159632 shutdown_connections 2024-02-17T13:03:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f194fbb4740 1 --2- 172.21.15.18:0/3853159632 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563d23510310 0x563d23512770 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f194fbb4740 1 --2- 172.21.15.18:0/3853159632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234ffea0 0x563d234e9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f194fbb4740 1 --2- 172.21.15.18:0/3853159632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d234e97c0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f192dffb640 0 client.20547 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:03:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f194fbb4740 1 --2- 172.21.15.18:0/3853159632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234f1fa0 0x563d234e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f194fbb4740 1 -- 172.21.15.18:0/3853159632 >> 172.21.15.18:0/3853159632 conn(0x563d234eccd0 msgr2=0x563d234eeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f194fbb4740 1 -- 172.21.15.18:0/3853159632 shutdown_connections 2024-02-17T13:03:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f194fbb4740 1 -- 172.21.15.18:0/3853159632 wait complete. 2024-02-17T13:03:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.074+0000 7f194fbb4740 20 client.20547 trim_cache size 0 max 16384 2024-02-17T13:03:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f194fbb4740 1 Processor -- start 2024-02-17T13:03:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f194fbb4740 1 -- start start 2024-02-17T13:03:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f194fbb4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234f1fa0 0x563d234eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f194fbb4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d234e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f194fbb4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234ffea0 0x563d23517110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f194fbb4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d233db130 con 0x563d234f2a40 2024-02-17T13:03:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f194fbb4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d234e72f0 con 0x563d234f1fa0 2024-02-17T13:03:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f194fbb4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d234fda30 con 0x563d234ffea0 2024-02-17T13:03:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f194659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d234e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f194659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d234e7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41072/0 (socket says 172.21.15.18:41072) 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f194659c640 1 -- 172.21.15.18:0/3640081463 learned_addr learned my addr 172.21.15.18:0/3640081463 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f1946d9d640 1 --2- 172.21.15.18:0/3640081463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234ffea0 0x563d23517110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.088+0000 7f1946d9d640 1 --2- 172.21.15.18:0/3640081463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234f1fa0 0x563d234eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.089+0000 7f194659c640 1 -- 172.21.15.18:0/3640081463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234ffea0 msgr2=0x563d23517110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.089+0000 7f194659c640 1 --2- 172.21.15.18:0/3640081463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234ffea0 0x563d23517110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.089+0000 7f194659c640 1 -- 172.21.15.18:0/3640081463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234f1fa0 msgr2=0x563d234eb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.089+0000 7f194659c640 1 --2- 172.21.15.18:0/3640081463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234f1fa0 0x563d234eb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.089+0000 7f194659c640 1 -- 172.21.15.18:0/3640081463 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d233f0c20 con 0x563d234f2a40 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.089+0000 7f194659c640 1 --2- 172.21.15.18:0/3640081463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d234e7bb0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f193c012770 tx=0x7f193c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42af572a804b3cc9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.089+0000 7f192ffff640 1 -- 172.21.15.18:0/3640081463 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c009d60 con 0x563d234f2a40 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.089+0000 7f192ffff640 1 -- 172.21.15.18:0/3640081463 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f193c012390 con 0x563d234f2a40 2024-02-17T13:03:53.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.089+0000 7f192ffff640 1 -- 172.21.15.18:0/3640081463 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c014600 con 0x563d234f2a40 2024-02-17T13:03:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.090+0000 7f194fbb4740 1 -- 172.21.15.18:0/3640081463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 msgr2=0x563d234e7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.090+0000 7f194fbb4740 1 --2- 172.21.15.18:0/3640081463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d234e7bb0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f193c012770 tx=0x7f193c0046c0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.091+0000 7f194fbb4740 1 -- 172.21.15.18:0/3640081463 shutdown_connections 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.091+0000 7f194fbb4740 1 --2- 172.21.15.18:0/3640081463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234ffea0 0x563d23517110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.091+0000 7f194fbb4740 1 --2- 172.21.15.18:0/3640081463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d234e7bb0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.091+0000 7f194fbb4740 1 --2- 172.21.15.18:0/3640081463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234f1fa0 0x563d234eb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.091+0000 7f194fbb4740 1 -- 172.21.15.18:0/3640081463 >> 172.21.15.18:0/3640081463 conn(0x563d234eccd0 msgr2=0x563d234ed0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.091+0000 7f194fbb4740 1 -- 172.21.15.18:0/3640081463 shutdown_connections 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.091+0000 7f194fbb4740 1 -- 172.21.15.18:0/3640081463 wait complete. 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.091+0000 7f194fbb4740 1 -- start start 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194fbb4740 1 Processor -- start 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194fbb4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234f1fa0 0x563d2355fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194fbb4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d23560200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194fbb4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234ffea0 0x563d235061d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194fbb4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d235068d0 con 0x563d234f2a40 2024-02-17T13:03:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194fbb4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23506a40 con 0x563d234f1fa0 2024-02-17T13:03:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194fbb4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23506c00 con 0x563d234ffea0 2024-02-17T13:03:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194fbb4740 10 client.0 _ll_register_callbacks cb 0x7f193800bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d23560200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d23560200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41078/0 (socket says 172.21.15.18:41078) 2024-02-17T13:03:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194659c640 1 -- 172.21.15.18:0/809699408 learned_addr learned my addr 172.21.15.18:0/809699408 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f1946d9d640 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234ffea0 0x563d235061d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f1946d9d640 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234f1fa0 0x563d2355fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194659c640 1 -- 172.21.15.18:0/809699408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234ffea0 msgr2=0x563d235061d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194659c640 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234ffea0 0x563d235061d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194659c640 1 -- 172.21.15.18:0/809699408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234f1fa0 msgr2=0x563d2355fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194659c640 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234f1fa0 0x563d2355fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f194659c640 1 -- 172.21.15.18:0/809699408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d234546a0 con 0x563d234f2a40 2024-02-17T13:03:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.092+0000 7f1946d9d640 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234f1fa0 0x563d2355fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.093+0000 7f194659c640 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d23560200 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f193c0126f0 tx=0x7f193c009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60dced058d27ffb1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.093+0000 7f192dffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.093+0000 7f192dffb640 1 -- 172.21.15.18:0/809699408 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c012de0 con 0x563d234f2a40 2024-02-17T13:03:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.093+0000 7f192dffb640 1 -- 172.21.15.18:0/809699408 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f193c0059c0 con 0x563d234f2a40 2024-02-17T13:03:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.093+0000 7f194fbb4740 1 -- 172.21.15.18:0/809699408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f193800fd60 con 0x563d234f2a40 2024-02-17T13:03:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.093+0000 7f192dffb640 1 -- 172.21.15.18:0/809699408 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f193c005b80 con 0x563d234f2a40 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.093+0000 7f194fbb4740 10 client.20553 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.093+0000 7f194fbb4740 1 -- 172.21.15.18:0/809699408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x563d234168c0 con 0x563d234f2a40 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.093+0000 7f192dffb640 1 -- 172.21.15.18:0/809699408 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f193800fd60 con 0x563d234f2a40 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.094+0000 7f192dffb640 1 -- 172.21.15.18:0/809699408 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f193c006000 con 0x563d234f2a40 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.094+0000 7f194fbb4740 10 client.20553 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.094+0000 7f194fbb4740 10 client.20553 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.094+0000 7f194fbb4740 20 client.20553 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.094+0000 7f194fbb4740 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563d235688a0 0x563d2356acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.094+0000 7f194fbb4740 4 client.20553 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.094+0000 7f194fbb4740 1 -- 172.21.15.18:0/809699408 --> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563d23192720 con 0x563d235688a0 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.094+0000 7f1946d9d640 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563d235688a0 0x563d2356acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.094+0000 7f1946d9d640 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563d235688a0 0x563d2356acc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3af1ddfa4e8219ac server_cookie=3de95a0fce8afb6e in_seq=0 out_seq=0 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.094+0000 7f192dffb640 10 client.20553 ms_handle_connect on v2:172.21.15.119:6834/2304139172 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f192dffb640 1 -- 172.21.15.18:0/809699408 <== mds.2 v2:172.21.15.119:6834/2304139172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563d23192720 con 0x563d235688a0 2024-02-17T13:03:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f192dffb640 10 client.20553 handle_command_reply: tid=0 2024-02-17T13:03:53.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f194fbb4740 1 client.20553 shutdown 2024-02-17T13:03:53.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f194fbb4740 1 -- 172.21.15.18:0/809699408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 msgr2=0x563d23560200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f194fbb4740 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d23560200 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f193c0126f0 tx=0x7f193c009690 comp rx=0 tx=0).stop 2024-02-17T13:03:53.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f194fbb4740 1 -- 172.21.15.18:0/809699408 shutdown_connections 2024-02-17T13:03:53.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f194fbb4740 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.119:6834/2304139172,v1:172.21.15.119:6835/2304139172] conn(0x563d235688a0 0x563d2356acc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f194fbb4740 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d234ffea0 0x563d235061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f194fbb4740 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d234f2a40 0x563d23560200 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f192dffb640 0 client.20553 ms_handle_reset on v2:172.21.15.119:6834/2304139172 2024-02-17T13:03:53.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f194fbb4740 1 --2- 172.21.15.18:0/809699408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d234f1fa0 0x563d2355fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f194fbb4740 1 -- 172.21.15.18:0/809699408 >> 172.21.15.18:0/809699408 conn(0x563d234eccd0 msgr2=0x563d234fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f194fbb4740 1 -- 172.21.15.18:0/809699408 shutdown_connections 2024-02-17T13:03:53.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f194fbb4740 1 -- 172.21.15.18:0/809699408 wait complete. 2024-02-17T13:03:53.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.095+0000 7f194fbb4740 20 client.20553 trim_cache size 0 max 16384 2024-02-17T13:03:53.095 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:03:53.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.096+0000 7f194cf02640 1 -- 172.21.15.18:0/2644695515 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f19300a22b0 msgr2=0x7f19300a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.096+0000 7f194cf02640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f19300a22b0 0x7f19300a4770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f19400bd6d0 tx=0x7f19400b2e10 comp rx=0 tx=0).stop 2024-02-17T13:03:53.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.096+0000 7f194cf02640 1 -- 172.21.15.18:0/2644695515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19481064c0 msgr2=0x7f194819dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.096+0000 7f194cf02640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19481064c0 0x7f194819dae0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1940005cc0 tx=0x7f1940034d40 comp rx=0 tx=0).stop 2024-02-17T13:03:53.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.096+0000 7f194cf02640 1 -- 172.21.15.18:0/2644695515 shutdown_connections 2024-02-17T13:03:53.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.096+0000 7f194cf02640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f19300a22b0 0x7f19300a4770 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.096+0000 7f194cf02640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1948197660 0x7f1948197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.096+0000 7f194cf02640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19481064c0 0x7f194819dae0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.096+0000 7f194cf02640 1 --2- 172.21.15.18:0/2644695515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19480685f0 0x7f194819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.096+0000 7f194cf02640 1 -- 172.21.15.18:0/2644695515 >> 172.21.15.18:0/2644695515 conn(0x7f1948077180 msgr2=0x7f19480fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.096+0000 7f194cf02640 1 -- 172.21.15.18:0/2644695515 shutdown_connections 2024-02-17T13:03:53.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.096+0000 7f194cf02640 1 -- 172.21.15.18:0/2644695515 wait complete. 2024-02-17T13:03:53.098 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:03:53.098 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T13:03:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.227+0000 7fc2d4c15640 1 -- 172.21.15.18:0/3772657401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2d0106450 msgr2=0x7fc2d010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.227+0000 7fc2d4c15640 1 --2- 172.21.15.18:0/3772657401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2d0106450 0x7fc2d010e020 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009930 tx=0x7fc2c802bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.228+0000 7fc2d4c15640 1 -- 172.21.15.18:0/3772657401 shutdown_connections 2024-02-17T13:03:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.228+0000 7fc2d4c15640 1 --2- 172.21.15.18:0/3772657401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2d0106450 0x7fc2d010e020 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.228+0000 7fc2d4c15640 1 --2- 172.21.15.18:0/3772657401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2d0100240 0x7fc2d0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.228+0000 7fc2d4c15640 1 --2- 172.21.15.18:0/3772657401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d00ff890 0x7fc2d00ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.228+0000 7fc2d4c15640 1 -- 172.21.15.18:0/3772657401 >> 172.21.15.18:0/3772657401 conn(0x7fc2d00765a0 msgr2=0x7fc2d00fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.228+0000 7fc2d4c15640 1 -- 172.21.15.18:0/3772657401 shutdown_connections 2024-02-17T13:03:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.228+0000 7fc2d4c15640 1 -- 172.21.15.18:0/3772657401 wait complete. 2024-02-17T13:03:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.228+0000 7fc2d4c15640 1 Processor -- start 2024-02-17T13:03:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2d4c15640 1 -- start start 2024-02-17T13:03:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2d4c15640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d00ff890 0x7fc2d019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2d4c15640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2d0106450 0x7fc2d019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2d4c15640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2d0197640 0x7fc2d0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2d4c15640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d0198110 con 0x7fc2d0197640 2024-02-17T13:03:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2d4c15640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d0198280 con 0x7fc2d00ff890 2024-02-17T13:03:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2d4c15640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d0198440 con 0x7fc2d0106450 2024-02-17T13:03:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2ced9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2d0197640 0x7fc2d0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2ced9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2d0197640 0x7fc2d0197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41098/0 (socket says 172.21.15.18:41098) 2024-02-17T13:03:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2ced9d640 1 -- 172.21.15.18:0/1460292831 learned_addr learned my addr 172.21.15.18:0/1460292831 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d00ff890 0x7fc2d019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2cdd9b640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2d0106450 0x7fc2d019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2ced9d640 1 -- 172.21.15.18:0/1460292831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2d0106450 msgr2=0x7fc2d019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2ced9d640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2d0106450 0x7fc2d019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2ced9d640 1 -- 172.21.15.18:0/1460292831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d00ff890 msgr2=0x7fc2d019d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2ced9d640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d00ff890 0x7fc2d019d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.229+0000 7fc2ced9d640 1 -- 172.21.15.18:0/1460292831 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c8009560 con 0x7fc2d0197640 2024-02-17T13:03:53.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.230+0000 7fc2ced9d640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2d0197640 0x7fc2d0197ac0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8005940 tx=0x7fc2c802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.230+0000 7fc2c77fe640 1 -- 172.21.15.18:0/1460292831 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8004350 con 0x7fc2d0197640 2024-02-17T13:03:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.230+0000 7fc2c77fe640 1 -- 172.21.15.18:0/1460292831 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2c80044b0 con 0x7fc2d0197640 2024-02-17T13:03:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.230+0000 7fc2d4c15640 1 -- 172.21.15.18:0/1460292831 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d01a4380 con 0x7fc2d0197640 2024-02-17T13:03:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.230+0000 7fc2c77fe640 1 -- 172.21.15.18:0/1460292831 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8003cc0 con 0x7fc2d0197640 2024-02-17T13:03:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.230+0000 7fc2d4c15640 1 -- 172.21.15.18:0/1460292831 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d01a48b0 con 0x7fc2d0197640 2024-02-17T13:03:53.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.233+0000 7fc2c77fe640 1 -- 172.21.15.18:0/1460292831 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc2c8005580 con 0x7fc2d0197640 2024-02-17T13:03:53.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.234+0000 7fc2c77fe640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc2b00a22b0 0x7fc2b00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.234+0000 7fc2d77eb740 1 Processor -- start 2024-02-17T13:03:53.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.234+0000 7fc2d77eb740 1 -- start start 2024-02-17T13:03:53.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.234+0000 7fc2c77fe640 1 -- 172.21.15.18:0/1460292831 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc2c80f0090 con 0x7fc2d0197640 2024-02-17T13:03:53.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.234+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc2b00a22b0 0x7fc2b00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2d77eb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d51cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2d77eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d522a0 0x5593b2d3e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2d77eb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d3ea50 0x5593b2d3eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2d77eb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b2cff960 con 0x5593b2d3ea50 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2d77eb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b2cb3310 con 0x5593b2d518f0 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2d77eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b2cb0840 con 0x5593b2d522a0 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2ced9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d3ea50 0x5593b2d3eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2ced9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d3ea50 0x5593b2d3eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41114/0 (socket says 172.21.15.18:41114) 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2ced9d640 1 -- 172.21.15.18:0/2981311613 learned_addr learned my addr 172.21.15.18:0/2981311613 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2cdd9b640 1 --2- 172.21.15.18:0/2981311613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d522a0 0x5593b2d3e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc2b00a22b0 0x7fc2b00a4770 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000bc30 tx=0x7fc2c000b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2ced9d640 1 -- 172.21.15.18:0/2981311613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d522a0 msgr2=0x5593b2d3e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2ced9d640 1 --2- 172.21.15.18:0/2981311613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d522a0 0x5593b2d3e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2ced9d640 1 -- 172.21.15.18:0/2981311613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 msgr2=0x5593b2d51cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2ced9d640 1 --2- 172.21.15.18:0/2981311613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d51cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2ced9d640 1 -- 172.21.15.18:0/2981311613 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593b2c6e060 con 0x5593b2d3ea50 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.235+0000 7fc2ced9d640 1 --2- 172.21.15.18:0/2981311613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d3ea50 0x5593b2d3eeb0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80b9010 tx=0x7fc2c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facb40cb80ecb0b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.236+0000 7fc2c4ff9640 1 -- 172.21.15.18:0/2981311613 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80bcbb0 con 0x5593b2d3ea50 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.236+0000 7fc2c4ff9640 1 -- 172.21.15.18:0/2981311613 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2c80c0d00 con 0x5593b2d3ea50 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.236+0000 7fc2c4ff9640 1 -- 172.21.15.18:0/2981311613 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80edc40 con 0x5593b2d3ea50 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 -- 172.21.15.18:0/2981311613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d3ea50 msgr2=0x5593b2d3eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/2981311613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d3ea50 0x5593b2d3eeb0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80b9010 tx=0x7fc2c8009260 comp rx=0 tx=0).stop 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 -- 172.21.15.18:0/2981311613 shutdown_connections 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/2981311613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d3ea50 0x5593b2d3eeb0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/2981311613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d522a0 0x5593b2d3e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/2981311613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d51cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 -- 172.21.15.18:0/2981311613 >> 172.21.15.18:0/2981311613 conn(0x5593b2d40ef0 msgr2=0x5593b2d412d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 -- 172.21.15.18:0/2981311613 shutdown_connections 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 -- 172.21.15.18:0/2981311613 wait complete. 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 -- start start 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 Processor -- start 2024-02-17T13:03:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d55df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d56330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b2cac020 con 0x5593b2d522a0 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b297f430 con 0x5593b2d518f0 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.237+0000 7fc2d77eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b293be00 con 0x5593b2d3ea50 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2d77eb740 10 client.0 _ll_register_callbacks cb 0x5593b2c53cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2ced9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d56330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2ced9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d56330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41122/0 (socket says 172.21.15.18:41122) 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2ced9d640 1 -- 172.21.15.18:0/700476325 learned_addr learned my addr 172.21.15.18:0/700476325 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2cdd9b640 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d55df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2ced9d640 1 -- 172.21.15.18:0/700476325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 msgr2=0x5593b2d558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2ced9d640 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2ced9d640 1 -- 172.21.15.18:0/700476325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 msgr2=0x5593b2d55df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2ced9d640 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d55df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2ced9d640 1 -- 172.21.15.18:0/700476325 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d01a48b0 con 0x5593b2d522a0 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2ced9d640 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d56330 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80cc480 tx=0x7fc2c80c5170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcc81a42f760bd0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2bdffb640 1 -- 172.21.15.18:0/700476325 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80edc40 con 0x5593b2d522a0 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2d77eb740 1 -- 172.21.15.18:0/700476325 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593b29ab7e0 con 0x5593b2d522a0 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2bdffb640 1 -- 172.21.15.18:0/700476325 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc2c80c5350 con 0x5593b2d522a0 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2bdffb640 1 -- 172.21.15.18:0/700476325 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2c80ed9f0 con 0x5593b2d522a0 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2d77eb740 10 client.20565 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.238+0000 7fc2d77eb740 1 -- 172.21.15.18:0/700476325 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x5593b2caa6a0 con 0x5593b2d522a0 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.239+0000 7fc2bdffb640 1 -- 172.21.15.18:0/700476325 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5593b29ab7e0 con 0x5593b2d522a0 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.239+0000 7fc2bdffb640 1 -- 172.21.15.18:0/700476325 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fc2c803c070 con 0x5593b2d522a0 2024-02-17T13:03:53.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.239+0000 7fc2d77eb740 10 client.20565 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.239+0000 7fc2d77eb740 10 client.20565 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.239+0000 7fc2d77eb740 20 client.20565 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.239+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5593b2d65e10 0x5593b2d68230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.239+0000 7fc2d77eb740 4 client.20565 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.239+0000 7fc2d77eb740 1 -- 172.21.15.18:0/700476325 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593b29e8720 con 0x5593b2d65e10 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.239+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5593b2d65e10 0x5593b2d68230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.240+0000 7fc2bdffb640 10 client.20565 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.240+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5593b2d65e10 0x5593b2d68230 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80d9cc0cb261b0d3 server_cookie=9c654bd50b185689 in_seq=0 out_seq=0 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2bdffb640 1 -- 172.21.15.18:0/700476325 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5593b29e8720 con 0x5593b2d65e10 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2bdffb640 10 client.20565 handle_command_reply: tid=0 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2d77eb740 1 client.20565 shutdown 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2d77eb740 1 -- 172.21.15.18:0/700476325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 msgr2=0x5593b2d56330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d56330 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80cc480 tx=0x7fc2c80c5170 comp rx=0 tx=0).stop 2024-02-17T13:03:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2d77eb740 1 -- 172.21.15.18:0/700476325 shutdown_connections 2024-02-17T13:03:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5593b2d65e10 0x5593b2d68230 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d56330 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d55df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/700476325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2d77eb740 1 -- 172.21.15.18:0/700476325 >> 172.21.15.18:0/700476325 conn(0x5593b2d40ef0 msgr2=0x5593b2d3fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2bdffb640 0 client.20565 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:03:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2d77eb740 1 -- 172.21.15.18:0/700476325 shutdown_connections 2024-02-17T13:03:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.242+0000 7fc2d77eb740 1 -- 172.21.15.18:0/700476325 wait complete. 2024-02-17T13:03:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.243+0000 7fc2d77eb740 20 client.20565 trim_cache size 0 max 16384 2024-02-17T13:03:53.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.264+0000 7fc2d77eb740 1 Processor -- start 2024-02-17T13:03:53.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.264+0000 7fc2d77eb740 1 -- start start 2024-02-17T13:03:53.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.264+0000 7fc2d77eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d57fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2d77eb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d56150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2d77eb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d50920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2d77eb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b2c31130 con 0x5593b2d522a0 2024-02-17T13:03:53.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2d77eb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b2d3f900 con 0x5593b2d518f0 2024-02-17T13:03:53.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2d77eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b2d61c00 con 0x5593b2d3ea50 2024-02-17T13:03:53.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2ce59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d57fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2ce59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d57fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:55870/0 (socket says 172.21.15.18:55870) 2024-02-17T13:03:53.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2ce59c640 1 -- 172.21.15.18:0/1948922972 learned_addr learned my addr 172.21.15.18:0/1948922972 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2ced9d640 1 --2- 172.21.15.18:0/1948922972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d50920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2ce59c640 1 -- 172.21.15.18:0/1948922972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 msgr2=0x5593b2d56150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/1948922972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d56150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2ce59c640 1 -- 172.21.15.18:0/1948922972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 msgr2=0x5593b2d50920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/1948922972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d50920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2ce59c640 1 -- 172.21.15.18:0/1948922972 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593b2c46c20 con 0x5593b2d3ea50 2024-02-17T13:03:53.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.265+0000 7fc2ced9d640 1 --2- 172.21.15.18:0/1948922972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d50920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:53.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.266+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/1948922972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d57fe0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0010c30 tx=0x7fc2c0013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c683978f389b5873 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.266+0000 7fc2bffff640 1 -- 172.21.15.18:0/1948922972 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c002b070 con 0x5593b2d3ea50 2024-02-17T13:03:53.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.266+0000 7fc2bffff640 1 -- 172.21.15.18:0/1948922972 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2c0015070 con 0x5593b2d3ea50 2024-02-17T13:03:53.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.266+0000 7fc2bffff640 1 -- 172.21.15.18:0/1948922972 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0024040 con 0x5593b2d3ea50 2024-02-17T13:03:53.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.267+0000 7fc2d77eb740 1 -- 172.21.15.18:0/1948922972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 msgr2=0x5593b2d57fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.267+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/1948922972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d57fe0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0010c30 tx=0x7fc2c0013000 comp rx=0 tx=0).stop 2024-02-17T13:03:53.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.267+0000 7fc2d77eb740 1 -- 172.21.15.18:0/1948922972 shutdown_connections 2024-02-17T13:03:53.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.267+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/1948922972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d50920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.267+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/1948922972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d56150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.267+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/1948922972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d57fe0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.267+0000 7fc2d77eb740 1 -- 172.21.15.18:0/1948922972 >> 172.21.15.18:0/1948922972 conn(0x5593b2d40ef0 msgr2=0x5593b2d6cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.267+0000 7fc2d77eb740 1 -- 172.21.15.18:0/1948922972 shutdown_connections 2024-02-17T13:03:53.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.267+0000 7fc2d77eb740 1 -- 172.21.15.18:0/1948922972 wait complete. 2024-02-17T13:03:53.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2d77eb740 1 -- start start 2024-02-17T13:03:53.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2d77eb740 1 Processor -- start 2024-02-17T13:03:53.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2d77eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d5d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2d77eb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d5b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2d77eb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d5ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2d77eb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b2d5c0f0 con 0x5593b2d522a0 2024-02-17T13:03:53.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2ced9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d5ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2ced9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d5ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41146/0 (socket says 172.21.15.18:41146) 2024-02-17T13:03:53.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2ced9d640 1 -- 172.21.15.18:0/4036631837 learned_addr learned my addr 172.21.15.18:0/4036631837 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d5d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2d77eb740 1 -- 172.21.15.18:0/4036631837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b2d5c260 con 0x5593b2d518f0 2024-02-17T13:03:53.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2cdd9b640 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d5b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.268+0000 7fc2d77eb740 1 -- 172.21.15.18:0/4036631837 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b2dbc320 con 0x5593b2d3ea50 2024-02-17T13:03:53.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.269+0000 7fc2d77eb740 10 client.0 _ll_register_callbacks cb 0x7fc2c80035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:53.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.269+0000 7fc2ce59c640 1 -- 172.21.15.18:0/4036631837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 msgr2=0x5593b2d5b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.269+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d5b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.269+0000 7fc2ce59c640 1 -- 172.21.15.18:0/4036631837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 msgr2=0x5593b2d5ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.269+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d5ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.269+0000 7fc2ce59c640 1 -- 172.21.15.18:0/4036631837 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593b2caa6a0 con 0x5593b2d3ea50 2024-02-17T13:03:53.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.269+0000 7fc2ced9d640 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d5ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:53.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.269+0000 7fc2ce59c640 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d5d080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc2c002a000 tx=0x7fc2c0011d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed1de3dad661451 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.269+0000 7fc2bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:03:53.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.269+0000 7fc2bdffb640 1 -- 172.21.15.18:0/4036631837 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c002b040 con 0x5593b2d3ea50 2024-02-17T13:03:53.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.269+0000 7fc2bdffb640 1 -- 172.21.15.18:0/4036631837 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc2c0017070 con 0x5593b2d3ea50 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.269+0000 7fc2d77eb740 1 -- 172.21.15.18:0/4036631837 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2c80c0d00 con 0x5593b2d3ea50 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.270+0000 7fc2bdffb640 1 -- 172.21.15.18:0/4036631837 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2c0024040 con 0x5593b2d3ea50 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.270+0000 7fc2d77eb740 10 client.29494 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.270+0000 7fc2d77eb740 1 -- 172.21.15.18:0/4036631837 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x7fc2c000be50 con 0x5593b2d3ea50 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.270+0000 7fc2bdffb640 1 -- 172.21.15.18:0/4036631837 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2c80c0d00 con 0x5593b2d3ea50 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.270+0000 7fc2bdffb640 1 -- 172.21.15.18:0/4036631837 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fc2c0038070 con 0x5593b2d3ea50 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.271+0000 7fc2d77eb740 10 client.29494 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.271+0000 7fc2d77eb740 10 client.29494 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.271+0000 7fc2d77eb740 20 client.29494 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.271+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5593b2dbdb80 0x5593b2dbffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.271+0000 7fc2d77eb740 4 client.29494 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.271+0000 7fc2d77eb740 1 -- 172.21.15.18:0/4036631837 --> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5593b2a37890 con 0x5593b2dbdb80 2024-02-17T13:03:53.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.271+0000 7fc2cdd9b640 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5593b2dbdb80 0x5593b2dbffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.272+0000 7fc2bdffb640 10 client.29494 ms_handle_connect on v2:172.21.15.119:6836/248613392 2024-02-17T13:03:53.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.272+0000 7fc2cdd9b640 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5593b2dbdb80 0x5593b2dbffa0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3007e02a69fd405a server_cookie=fec2a563bee9d25d in_seq=0 out_seq=0 2024-02-17T13:03:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.276+0000 7fc2bdffb640 1 -- 172.21.15.18:0/4036631837 <== mds.1 v2:172.21.15.119:6836/248613392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+30085 (crc 0 0 0) 0x5593b2a37890 con 0x5593b2dbdb80 2024-02-17T13:03:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.276+0000 7fc2bdffb640 10 client.29494 handle_command_reply: tid=0 2024-02-17T13:03:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.276+0000 7fc2d77eb740 1 client.29494 shutdown 2024-02-17T13:03:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.276+0000 7fc2d77eb740 1 -- 172.21.15.18:0/4036631837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 msgr2=0x5593b2d5d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.276+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d5d080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc2c002a000 tx=0x7fc2c0011d80 comp rx=0 tx=0).stop 2024-02-17T13:03:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.276+0000 7fc2d77eb740 1 -- 172.21.15.18:0/4036631837 shutdown_connections 2024-02-17T13:03:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.276+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.119:6836/248613392,v1:172.21.15.119:6837/248613392] conn(0x5593b2dbdb80 0x5593b2dbffa0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.276+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5593b2d522a0 0x5593b2d5ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.276+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593b2d518f0 0x5593b2d5b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.276+0000 7fc2d77eb740 1 --2- 172.21.15.18:0/4036631837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5593b2d3ea50 0x5593b2d5d080 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d77eb740 1 -- 172.21.15.18:0/4036631837 >> 172.21.15.18:0/4036631837 conn(0x5593b2d40ef0 msgr2=0x5593b2d50920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2bdffb640 0 client.29494 ms_handle_reset on v2:172.21.15.119:6836/248613392 2024-02-17T13:03:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d77eb740 1 -- 172.21.15.18:0/4036631837 shutdown_connections 2024-02-17T13:03:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d77eb740 1 -- 172.21.15.18:0/4036631837 wait complete. 2024-02-17T13:03:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d77eb740 20 client.29494 trim_cache size 0 max 16384 2024-02-17T13:03:53.269 INFO:teuthology.orchestra.run.smithi018.stdout:[ 2024-02-17T13:03:53.269 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.269 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.269 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 17474985, 2024-02-17T13:03:53.269 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632358, 2024-02-17T13:03:53.269 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.Y04" 2024-02-17T13:03:53.269 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 22282374, 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632502, 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ACCESS" 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 44541585, 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632512, 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.FSL" 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.270 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 68402129, 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632555, 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/LARGE.FIL" 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 83205717, 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632319, 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.PX" 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 100114821, 2024-02-17T13:03:53.271 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511631951, 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp" 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 201061690, 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511631960, 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp" 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 217042116, 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511631972, 2024-02-17T13:03:53.272 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.000" 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 228887469, 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632331, 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.X04" 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 250361605, 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632165, 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/GRAPHIC1.CDR" 2024-02-17T13:03:53.273 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.274 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.274 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.274 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 265689946, 2024-02-17T13:03:53.274 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632536, 2024-02-17T13:03:53.274 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED" 2024-02-17T13:03:53.274 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.274 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.274 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.274 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 322599022, 2024-02-17T13:03:53.274 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632342, 2024-02-17T13:03:53.274 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.Y04" 2024-02-17T13:03:53.274 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 353990034, 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632027, 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/CHAP10.DOC" 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 354003082, 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632318, 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/MOVED.DOC" 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.275 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 379030449, 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632145, 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL" 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 412091781, 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632360, 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.PX" 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.276 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 420857839, 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632489, 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.TV" 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 460890606, 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632468, 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.Y03" 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 480069620, 2024-02-17T13:03:53.277 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632001, 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.003" 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 501046091, 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632183, 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/TEAPOT2.CPT" 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 515837876, 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632122, 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/TRIDOTS.POT" 2024-02-17T13:03:53.278 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 520301462, 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632327, 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX" 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 534726608, 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632413, 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.YG1" 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.279 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.280 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 544404046, 2024-02-17T13:03:53.280 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632041, 2024-02-17T13:03:53.280 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/BASEMACH.DOC" 2024-02-17T13:03:53.280 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.280 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.280 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.280 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 581679058, 2024-02-17T13:03:53.280 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632440, 2024-02-17T13:03:53.280 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.DB" 2024-02-17T13:03:53.280 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.280 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.280 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 599604896, 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632483, 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.FSL" 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 656497680, 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632012, 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/CHAP10.DOC" 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.281 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 724049895, 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632100, 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/PPTOOLS1.PPA" 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 724230668, 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632037, 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/FACTS.DOC" 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 741883858, 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632192, 2024-02-17T13:03:53.282 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/SIGN2.CPT" 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 818291441, 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632112, 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/TIPS.PPT" 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 823683864, 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632210, 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/GRAPHIC2.CDR" 2024-02-17T13:03:53.283 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 849382321, 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632247, 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/RESULTS.XLS" 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 882382375, 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632416, 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.MB" 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.284 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 902448862, 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632394, 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.YG0" 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 908746010, 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632478, 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.VAL" 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.285 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 993703516, 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632367, 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.DB" 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1108650990, 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632472, 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.TV" 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1154388300, 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632047, 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/PCMAGCD.XLS" 2024-02-17T13:03:53.286 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1164477043, 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632043, 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL" 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1169383195, 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632058, 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/SALES.XLS" 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.287 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.288 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1222165985, 2024-02-17T13:03:53.288 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632129, 2024-02-17T13:03:53.288 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL" 2024-02-17T13:03:53.288 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.288 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.288 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.288 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1259105962, 2024-02-17T13:03:53.288 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632444, 2024-02-17T13:03:53.288 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.X03" 2024-02-17T13:03:53.288 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.288 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.288 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1311411245, 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632291, 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM" 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1325404355, 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632228, 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CORELDRW.CDT" 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1357913282, 2024-02-17T13:03:53.289 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632528, 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/MEDIUM.FIL" 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1360364410, 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511631959, 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.000" 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1363142189, 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632494, 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.FSL" 2024-02-17T13:03:53.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1400705399, 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632114, 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/PPTOOLS1.PPA" 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1474109852, 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632195, 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/TEAPOT2.CPT" 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.291 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1495218118, 2024-02-17T13:03:53.292 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632297, 2024-02-17T13:03:53.292 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/T1.XLS" 2024-02-17T13:03:53.292 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.292 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.292 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.292 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1513157904, 2024-02-17T13:03:53.292 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632298, 2024-02-17T13:03:53.292 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/MOVED.DOC" 2024-02-17T13:03:53.292 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.292 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1527520179, 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632094, 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/PCBENCHM.PPT" 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1536856915, 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632431, 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.MB" 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.293 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1648270889, 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632158, 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CDRBARS.CFG" 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1722770758, 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632201, 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/TRAIN.CDR" 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1803634382, 2024-02-17T13:03:53.294 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632283, 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/EVENTS.DOC" 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1830575748, 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632514, 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED" 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1831448260, 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632425, 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.DB" 2024-02-17T13:03:53.295 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1858883827, 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632311, 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.DB" 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1872555523, 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632142, 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CDRBARS.CFG" 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.296 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1878802967, 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632167, 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ARTISTIC.ACL" 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 1923080944, 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632022, 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD" 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2005278208, 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632017, 2024-02-17T13:03:53.297 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.004" 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2071251410, 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511631990, 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.003" 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2085423244, 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632246, 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CORELDRW.CDT" 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.298 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2100713521, 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632282, 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/NEWS1_1.LWP" 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2112683687, 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632523, 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ACCESS" 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2125244884, 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632223, 2024-02-17T13:03:53.299 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/GRAPHIC2.CDR" 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2156267498, 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632021, 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/BASEMACH.DOC" 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2171783352, 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632104, 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/ZD16.BMP" 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.300 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2213102250, 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632351, 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.DB" 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2216007626, 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632064, 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL" 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.301 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2280932768, 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632213, 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/TRAIN.CDR" 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2385846216, 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632397, 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG1" 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2393294834, 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511631981, 2024-02-17T13:03:53.302 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.001" 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2482896815, 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632526, 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ACCESS/FASTENER.MDB" 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2489634982, 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632462, 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.VAL" 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.303 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2490995176, 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632270, 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM" 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2496965368, 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632275, 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/T1.XLS" 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2638445658, 2024-02-17T13:03:53.304 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632538, 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/SMALL.FIL" 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2657798143, 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632081, 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/SALES.XLS" 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2676251669, 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511631971, 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.001" 2024-02-17T13:03:53.305 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2689372003, 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632548, 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/MEDIUM.FIL" 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2712507909, 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511631991, 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.002" 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.306 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.307 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2773157621, 2024-02-17T13:03:53.307 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632000, 2024-02-17T13:03:53.307 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.004" 2024-02-17T13:03:53.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.307 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.307 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.307 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2780947645, 2024-02-17T13:03:53.307 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632330, 2024-02-17T13:03:53.307 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.DB" 2024-02-17T13:03:53.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.308 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.308 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.308 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2815618509, 2024-02-17T13:03:53.308 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632434, 2024-02-17T13:03:53.308 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.PX" 2024-02-17T13:03:53.308 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.308 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.308 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.308 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2829938149, 2024-02-17T13:03:53.308 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511631980, 2024-02-17T13:03:53.308 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.002" 2024-02-17T13:03:53.308 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.309 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.309 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.309 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2839137842, 2024-02-17T13:03:53.309 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632217, 2024-02-17T13:03:53.309 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ROSE.CPT" 2024-02-17T13:03:53.309 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.309 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.309 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.309 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2864698331, 2024-02-17T13:03:53.309 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632349, 2024-02-17T13:03:53.309 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.X04" 2024-02-17T13:03:53.309 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2870698099, 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632517, 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/SMALL.FIL" 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2900366143, 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632388, 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG1" 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.310 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2932299629, 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632453, 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.Y03" 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2932747824, 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632233, 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ROSE.CPT" 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.311 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2981302684, 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632149, 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CDRROLS.CFG" 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 2992330079, 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632500, 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.FSL" 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3154793778, 2024-02-17T13:03:53.312 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632177, 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/GRAPHIC1.CDR" 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3234848247, 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632124, 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/TIPS.PPT" 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3261215441, 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632369, 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG0" 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.313 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3293556358, 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632305, 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/EVENTS.DOC" 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3301724578, 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632378, 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG0" 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3324533253, 2024-02-17T13:03:53.314 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632186, 2024-02-17T13:03:53.315 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/MAILBOX2.CPT" 2024-02-17T13:03:53.315 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.315 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.315 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.315 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3328139832, 2024-02-17T13:03:53.315 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632239, 2024-02-17T13:03:53.315 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/BENCHS.LWP" 2024-02-17T13:03:53.315 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.315 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.315 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.315 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3330523578, 2024-02-17T13:03:53.315 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632306, 2024-02-17T13:03:53.316 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX" 2024-02-17T13:03:53.316 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.316 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.316 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.316 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3430028879, 2024-02-17T13:03:53.316 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632406, 2024-02-17T13:03:53.316 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.VAL" 2024-02-17T13:03:53.317 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.317 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.317 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.317 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3509547545, 2024-02-17T13:03:53.317 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632458, 2024-02-17T13:03:53.317 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.X03" 2024-02-17T13:03:53.318 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.318 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.318 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.318 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3532425201, 2024-02-17T13:03:53.318 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632090, 2024-02-17T13:03:53.318 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT" 2024-02-17T13:03:53.318 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.319 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.319 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.319 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3535994564, 2024-02-17T13:03:53.319 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632507, 2024-02-17T13:03:53.319 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ACCESS/FASTENER.MDB" 2024-02-17T13:03:53.319 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.319 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.319 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.319 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3649556935, 2024-02-17T13:03:53.319 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632174, 2024-02-17T13:03:53.319 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/MAILBOX2.CPT" 2024-02-17T13:03:53.320 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.320 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.320 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.320 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3656796612, 2024-02-17T13:03:53.320 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632422, 2024-02-17T13:03:53.320 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.VAL" 2024-02-17T13:03:53.320 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.320 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.320 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.320 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3674886703, 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632008, 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD" 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3677647580, 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632136, 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/TRIDOTS.POT" 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3692077939, 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632385, 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.XG0" 2024-02-17T13:03:53.321 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3717969239, 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632155, 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ARTISTIC.ACL" 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3728128989, 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632541, 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/LARGE.FIL" 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.322 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3747767585, 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632449, 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.PX" 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3756507551, 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632086, 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/ZD16.BMP" 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3811175711, 2024-02-17T13:03:53.323 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632234, 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO" 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 3930020264, 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632403, 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.XG1" 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4012855282, 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632376, 2024-02-17T13:03:53.324 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.PX" 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4013686900, 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632068, 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/PCMAGCD.XLS" 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4046892807, 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632261, 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP" 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.325 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4057049422, 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632132, 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CDRROLS.CFG" 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4128918081, 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632204, 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/SIGN2.CPT" 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4131300286, 2024-02-17T13:03:53.326 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632262, 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/NEWS1_1.LWP" 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4155482369, 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632074, 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/PCBENCHM.PPT" 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4184038870, 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632269, 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/RESULTS.XLS" 2024-02-17T13:03:53.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4209615699, 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632255, 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO" 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4243877982, 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632056, 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/FACTS.DOC" 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.328 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.329 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4272231202, 2024-02-17T13:03:53.329 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632070, 2024-02-17T13:03:53.329 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT" 2024-02-17T13:03:53.329 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:03:53.329 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:03:53.329 INFO:teuthology.orchestra.run.smithi018.stdout: "damage_type": "backtrace", 2024-02-17T13:03:53.329 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 4292119270, 2024-02-17T13:03:53.329 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1099511632340, 2024-02-17T13:03:53.329 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.PX" 2024-02-17T13:03:53.329 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:03:53.329 INFO:teuthology.orchestra.run.smithi018.stdout:] 2024-02-17T13:03:53.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d4c15640 1 -- 172.21.15.18:0/1460292831 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc2b00a22b0 msgr2=0x7fc2b00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d4c15640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc2b00a22b0 0x7fc2b00a4770 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000bc30 tx=0x7fc2c000b390 comp rx=0 tx=0).stop 2024-02-17T13:03:53.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d4c15640 1 -- 172.21.15.18:0/1460292831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2d0197640 msgr2=0x7fc2d0197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d4c15640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2d0197640 0x7fc2d0197ac0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8005940 tx=0x7fc2c802c650 comp rx=0 tx=0).stop 2024-02-17T13:03:53.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d4c15640 1 -- 172.21.15.18:0/1460292831 shutdown_connections 2024-02-17T13:03:53.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d4c15640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc2b00a22b0 0x7fc2b00a4770 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d4c15640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2d0197640 0x7fc2d0197ac0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d4c15640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2d0106450 0x7fc2d019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d4c15640 1 --2- 172.21.15.18:0/1460292831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d00ff890 0x7fc2d019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.277+0000 7fc2d4c15640 1 -- 172.21.15.18:0/1460292831 >> 172.21.15.18:0/1460292831 conn(0x7fc2d00765a0 msgr2=0x7fc2d01046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.278+0000 7fc2d4c15640 1 -- 172.21.15.18:0/1460292831 shutdown_connections 2024-02-17T13:03:53.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.278+0000 7fc2d4c15640 1 -- 172.21.15.18:0/1460292831 wait complete. 2024-02-17T13:03:53.332 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T13:03:53.333 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T13:03:53.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.462+0000 7f01a4edd640 1 -- 172.21.15.18:0/2445298109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a0068fa0 msgr2=0x7f01a0105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.462+0000 7f01a4edd640 1 --2- 172.21.15.18:0/2445298109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a0068fa0 0x7f01a0105de0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f0194009930 tx=0x7f019402bc30 comp rx=0 tx=0).stop 2024-02-17T13:03:53.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.463+0000 7f01a4edd640 1 -- 172.21.15.18:0/2445298109 shutdown_connections 2024-02-17T13:03:53.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.463+0000 7f01a4edd640 1 --2- 172.21.15.18:0/2445298109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01a01064c0 0x7f01a010e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.463+0000 7f01a4edd640 1 --2- 172.21.15.18:0/2445298109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a0068fa0 0x7f01a0105de0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.463+0000 7f01a4edd640 1 --2- 172.21.15.18:0/2445298109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01a00685f0 0x7f01a00689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.463+0000 7f01a4edd640 1 -- 172.21.15.18:0/2445298109 >> 172.21.15.18:0/2445298109 conn(0x7f01a0077180 msgr2=0x7f01a0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.463+0000 7f01a4edd640 1 -- 172.21.15.18:0/2445298109 shutdown_connections 2024-02-17T13:03:53.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.463+0000 7f01a4edd640 1 -- 172.21.15.18:0/2445298109 wait complete. 2024-02-17T13:03:53.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.463+0000 7f01a4edd640 1 Processor -- start 2024-02-17T13:03:53.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f01a4edd640 1 -- start start 2024-02-17T13:03:53.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f01a4edd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a00685f0 0x7f01a019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f01a4edd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01a01064c0 0x7f01a019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f01a4edd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01a0197620 0x7f01a0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f01a4edd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a0198010 con 0x7f01a00685f0 2024-02-17T13:03:53.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f01a4edd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a0198180 con 0x7f01a0197620 2024-02-17T13:03:53.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f01a4edd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a0198340 con 0x7f01a01064c0 2024-02-17T13:03:53.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f019e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a00685f0 0x7f01a019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f019e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a00685f0 0x7f01a019d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41158/0 (socket says 172.21.15.18:41158) 2024-02-17T13:03:53.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f019e59c640 1 -- 172.21.15.18:0/5669017 learned_addr learned my addr 172.21.15.18:0/5669017 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f019dd9b640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01a01064c0 0x7f01a019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f019ed9d640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01a0197620 0x7f01a0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f019e59c640 1 -- 172.21.15.18:0/5669017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01a01064c0 msgr2=0x7f01a019d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f019e59c640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01a01064c0 0x7f01a019d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f019e59c640 1 -- 172.21.15.18:0/5669017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01a0197620 msgr2=0x7f01a0197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f019e59c640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01a0197620 0x7f01a0197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f019e59c640 1 -- 172.21.15.18:0/5669017 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194009560 con 0x7f01a00685f0 2024-02-17T13:03:53.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.464+0000 7f019e59c640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a00685f0 0x7f01a019d470 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f019000e9f0 tx=0x7f019000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.465+0000 7f018f7fe640 1 -- 172.21.15.18:0/5669017 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190009700 con 0x7f01a00685f0 2024-02-17T13:03:53.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.465+0000 7f01a4edd640 1 -- 172.21.15.18:0/5669017 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a01a4330 con 0x7f01a00685f0 2024-02-17T13:03:53.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.465+0000 7f018f7fe640 1 -- 172.21.15.18:0/5669017 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f019000cd80 con 0x7f01a00685f0 2024-02-17T13:03:53.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.465+0000 7f018f7fe640 1 -- 172.21.15.18:0/5669017 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190018920 con 0x7f01a00685f0 2024-02-17T13:03:53.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.465+0000 7f01a4edd640 1 -- 172.21.15.18:0/5669017 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a01a4860 con 0x7f01a00685f0 2024-02-17T13:03:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.468+0000 7f018f7fe640 1 -- 172.21.15.18:0/5669017 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0190018a80 con 0x7f01a00685f0 2024-02-17T13:03:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.468+0000 7f01a7b81740 1 Processor -- start 2024-02-17T13:03:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.468+0000 7f01a7b81740 1 -- start start 2024-02-17T13:03:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f01a7b81740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 0x55d0e87b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f01a7b81740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87b4b70 0x55d0e87bd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f01a7b81740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87bd750 0x55d0e87bfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f01a7b81740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e876f960 con 0x55d0e87b4250 2024-02-17T13:03:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f01a7b81740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8723310 con 0x55d0e87b4b70 2024-02-17T13:03:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f01a7b81740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8720840 con 0x55d0e87bd750 2024-02-17T13:03:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 0x55d0e87b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 0x55d0e87b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41170/0 (socket says 172.21.15.18:41170) 2024-02-17T13:03:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019ed9d640 1 -- 172.21.15.18:0/2229840757 learned_addr learned my addr 172.21.15.18:0/2229840757 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019e59c640 1 --2- 172.21.15.18:0/2229840757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87b4b70 0x55d0e87bd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019dd9b640 1 --2- 172.21.15.18:0/2229840757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87bd750 0x55d0e87bfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f018f7fe640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f01880a22b0 0x7f01880a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019ed9d640 1 -- 172.21.15.18:0/2229840757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87bd750 msgr2=0x55d0e87bfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f018f7fe640 1 -- 172.21.15.18:0/5669017 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0190014070 con 0x7f01a00685f0 2024-02-17T13:03:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019ed9d640 1 --2- 172.21.15.18:0/2229840757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87bd750 0x55d0e87bfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019ed9d640 1 -- 172.21.15.18:0/2229840757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87b4b70 msgr2=0x55d0e87bd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019ed9d640 1 --2- 172.21.15.18:0/2229840757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87b4b70 0x55d0e87bd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019ed9d640 1 -- 172.21.15.18:0/2229840757 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0e86de060 con 0x55d0e87b4250 2024-02-17T13:03:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019e59c640 1 --2- 172.21.15.18:0/2229840757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87b4b70 0x55d0e87bd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019dd9b640 1 --2- 172.21.15.18:0/2229840757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87bd750 0x55d0e87bfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.469+0000 7f019ed9d640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f01880a22b0 0x7f01880a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.470+0000 7f019ed9d640 1 --2- 172.21.15.18:0/2229840757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 0x55d0e87b4630 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f0198011590 tx=0x7f0198011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88d8d3668311376 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.470+0000 7f019ed9d640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f01880a22b0 0x7f01880a4770 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0198022000 tx=0x7f0198023000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.470+0000 7f018cff9640 1 -- 172.21.15.18:0/2229840757 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019800f840 con 0x55d0e87b4250 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.470+0000 7f018cff9640 1 -- 172.21.15.18:0/2229840757 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f019800fe80 con 0x55d0e87b4250 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.470+0000 7f018cff9640 1 -- 172.21.15.18:0/2229840757 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019801bc70 con 0x55d0e87b4250 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.471+0000 7f01a7b81740 1 -- 172.21.15.18:0/2229840757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 msgr2=0x55d0e87b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.471+0000 7f01a7b81740 1 --2- 172.21.15.18:0/2229840757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 0x55d0e87b4630 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f0198011590 tx=0x7f0198011a50 comp rx=0 tx=0).stop 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.471+0000 7f01a7b81740 1 -- 172.21.15.18:0/2229840757 shutdown_connections 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.471+0000 7f01a7b81740 1 --2- 172.21.15.18:0/2229840757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87bd750 0x55d0e87bfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.471+0000 7f01a7b81740 1 --2- 172.21.15.18:0/2229840757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87b4b70 0x55d0e87bd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.471+0000 7f01a7b81740 1 --2- 172.21.15.18:0/2229840757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 0x55d0e87b4630 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.471+0000 7f01a7b81740 1 -- 172.21.15.18:0/2229840757 >> 172.21.15.18:0/2229840757 conn(0x55d0e87ae980 msgr2=0x55d0e87aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.471+0000 7f01a7b81740 1 -- 172.21.15.18:0/2229840757 shutdown_connections 2024-02-17T13:03:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.471+0000 7f01a7b81740 1 -- 172.21.15.18:0/2229840757 wait complete. 2024-02-17T13:03:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.471+0000 7f01a7b81740 1 -- start start 2024-02-17T13:03:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.471+0000 7f01a7b81740 1 Processor -- start 2024-02-17T13:03:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f01a7b81740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87b4250 0x55d0e87ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f01a7b81740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 0x55d0e87bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f01a7b81740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87bd750 0x55d0e87bb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f01a7b81740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e871c020 con 0x55d0e87b4b70 2024-02-17T13:03:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f01a7b81740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e83ef430 con 0x55d0e87bd750 2024-02-17T13:03:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f01a7b81740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e83abe00 con 0x55d0e87b4250 2024-02-17T13:03:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f019dd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 0x55d0e87bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f019e59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87b4250 0x55d0e87ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f019dd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 0x55d0e87bad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41182/0 (socket says 172.21.15.18:41182) 2024-02-17T13:03:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f019dd9b640 1 -- 172.21.15.18:0/529525397 learned_addr learned my addr 172.21.15.18:0/529525397 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f019e59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87b4250 0x55d0e87ba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:55916/0 (socket says 172.21.15.18:55916) 2024-02-17T13:03:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f01a7b81740 10 client.0 _ll_register_callbacks cb 0x55d0e86c3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f019ed9d640 1 --2- 172.21.15.18:0/529525397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87bd750 0x55d0e87bb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f019dd9b640 1 -- 172.21.15.18:0/529525397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87b4250 msgr2=0x55d0e87ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f019dd9b640 1 --2- 172.21.15.18:0/529525397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87b4250 0x55d0e87ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f019dd9b640 1 -- 172.21.15.18:0/529525397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87bd750 msgr2=0x55d0e87bb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f019dd9b640 1 --2- 172.21.15.18:0/529525397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87bd750 0x55d0e87bb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f019dd9b640 1 -- 172.21.15.18:0/529525397 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0198009cd0 con 0x55d0e87b4b70 2024-02-17T13:03:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f019dd9b640 1 --2- 172.21.15.18:0/529525397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 0x55d0e87bad40 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f0194000c00 tx=0x7f019402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e575bbd593415f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.472+0000 7f0185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:53.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.474+0000 7f0185ffb640 1 -- 172.21.15.18:0/529525397 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194004040 con 0x55d0e87b4b70 2024-02-17T13:03:53.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.474+0000 7f0185ffb640 1 -- 172.21.15.18:0/529525397 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f01940041a0 con 0x55d0e87b4b70 2024-02-17T13:03:53.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.474+0000 7f01a7b81740 1 -- 172.21.15.18:0/529525397 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0e8458720 con 0x55d0e87b4b70 2024-02-17T13:03:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.474+0000 7f0185ffb640 1 -- 172.21.15.18:0/529525397 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0194004360 con 0x55d0e87b4b70 2024-02-17T13:03:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.474+0000 7f01a7b81740 10 client.20583 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.474+0000 7f01a7b81740 1 -- 172.21.15.18:0/529525397 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x55d0e871a6a0 con 0x55d0e87b4b70 2024-02-17T13:03:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.474+0000 7f0185ffb640 1 -- 172.21.15.18:0/529525397 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d0e8458720 con 0x55d0e87b4b70 2024-02-17T13:03:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.475+0000 7f0185ffb640 1 -- 172.21.15.18:0/529525397 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f019403c040 con 0x55d0e87b4b70 2024-02-17T13:03:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.475+0000 7f01a7b81740 10 client.20583 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.475+0000 7f01a7b81740 10 client.20583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.475+0000 7f01a7b81740 20 client.20583 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.475+0000 7f01a7b81740 1 --2- 172.21.15.18:0/529525397 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d0e87cdc60 0x55d0e87d0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.475+0000 7f01a7b81740 4 client.20583 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.475+0000 7f01a7b81740 1 -- 172.21.15.18:0/529525397 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0e84a7890 con 0x55d0e87cdc60 2024-02-17T13:03:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.475+0000 7f019e59c640 1 --2- 172.21.15.18:0/529525397 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d0e87cdc60 0x55d0e87d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.475+0000 7f019e59c640 1 --2- 172.21.15.18:0/529525397 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d0e87cdc60 0x55d0e87d0080 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad9d35cfa217624 server_cookie=6c95cf248c4f45ad in_seq=0 out_seq=0 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.475+0000 7f0185ffb640 10 client.20583 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f0185ffb640 1 -- 172.21.15.18:0/529525397 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0e84a7890 con 0x55d0e87cdc60 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f0185ffb640 10 client.20583 handle_command_reply: tid=0 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f01a7b81740 1 client.20583 shutdown 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f01a7b81740 1 -- 172.21.15.18:0/529525397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 msgr2=0x55d0e87bad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f01a7b81740 1 --2- 172.21.15.18:0/529525397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 0x55d0e87bad40 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f0194000c00 tx=0x7f019402ca10 comp rx=0 tx=0).stop 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f01a7b81740 1 -- 172.21.15.18:0/529525397 shutdown_connections 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f01a7b81740 1 --2- 172.21.15.18:0/529525397 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d0e87cdc60 0x55d0e87d0080 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f01a7b81740 1 --2- 172.21.15.18:0/529525397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87bd750 0x55d0e87bb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f01a7b81740 1 --2- 172.21.15.18:0/529525397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 0x55d0e87bad40 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f01a7b81740 1 --2- 172.21.15.18:0/529525397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87b4250 0x55d0e87ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f0185ffb640 0 client.20583 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f01a7b81740 1 -- 172.21.15.18:0/529525397 >> 172.21.15.18:0/529525397 conn(0x55d0e87ae980 msgr2=0x55d0e87b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f01a7b81740 1 -- 172.21.15.18:0/529525397 shutdown_connections 2024-02-17T13:03:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.477+0000 7f01a7b81740 1 -- 172.21.15.18:0/529525397 wait complete. 2024-02-17T13:03:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.478+0000 7f01a7b81740 20 client.20583 trim_cache size 0 max 16384 2024-02-17T13:03:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.492+0000 7f01a7b81740 1 Processor -- start 2024-02-17T13:03:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.492+0000 7f01a7b81740 1 -- start start 2024-02-17T13:03:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.492+0000 7f01a7b81740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87b4250 0x55d0e87bcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.492+0000 7f01a7b81740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 0x55d0e87bb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f01a7b81740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87bd750 0x55d0e87b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f01a7b81740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e86a1130 con 0x55d0e87b4b70 2024-02-17T13:03:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f01a7b81740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e84a7890 con 0x55d0e87bd750 2024-02-17T13:03:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f01a7b81740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8458720 con 0x55d0e87b4250 2024-02-17T13:03:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f019dd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 0x55d0e87bb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f019dd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 0x55d0e87bb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41194/0 (socket says 172.21.15.18:41194) 2024-02-17T13:03:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f019dd9b640 1 -- 172.21.15.18:0/3028359932 learned_addr learned my addr 172.21.15.18:0/3028359932 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f019e59c640 1 --2- 172.21.15.18:0/3028359932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87b4250 0x55d0e87bcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f019ed9d640 1 --2- 172.21.15.18:0/3028359932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87bd750 0x55d0e87b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f019dd9b640 1 -- 172.21.15.18:0/3028359932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87b4250 msgr2=0x55d0e87bcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f019dd9b640 1 --2- 172.21.15.18:0/3028359932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87b4250 0x55d0e87bcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f019dd9b640 1 -- 172.21.15.18:0/3028359932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87bd750 msgr2=0x55d0e87b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f019dd9b640 1 --2- 172.21.15.18:0/3028359932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87bd750 0x55d0e87b7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f019dd9b640 1 -- 172.21.15.18:0/3028359932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0e86b6c20 con 0x55d0e87b4b70 2024-02-17T13:03:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f019dd9b640 1 --2- 172.21.15.18:0/3028359932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 0x55d0e87bb0a0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f0194005aa0 tx=0x7f0194005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94d5a8385b02f5d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f0187fff640 1 -- 172.21.15.18:0/3028359932 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194058070 con 0x55d0e87b4b70 2024-02-17T13:03:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f0187fff640 1 -- 172.21.15.18:0/3028359932 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01940035f0 con 0x55d0e87b4b70 2024-02-17T13:03:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.493+0000 7f0187fff640 1 -- 172.21.15.18:0/3028359932 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194005050 con 0x55d0e87b4b70 2024-02-17T13:03:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.495+0000 7f01a7b81740 1 -- 172.21.15.18:0/3028359932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 msgr2=0x55d0e87bb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.495+0000 7f01a7b81740 1 --2- 172.21.15.18:0/3028359932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 0x55d0e87bb0a0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f0194005aa0 tx=0x7f0194005e30 comp rx=0 tx=0).stop 2024-02-17T13:03:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.495+0000 7f01a7b81740 1 -- 172.21.15.18:0/3028359932 shutdown_connections 2024-02-17T13:03:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.495+0000 7f01a7b81740 1 --2- 172.21.15.18:0/3028359932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87bd750 0x55d0e87b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.495+0000 7f01a7b81740 1 --2- 172.21.15.18:0/3028359932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4b70 0x55d0e87bb0a0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.495+0000 7f01a7b81740 1 --2- 172.21.15.18:0/3028359932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87b4250 0x55d0e87bcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.495+0000 7f01a7b81740 1 -- 172.21.15.18:0/3028359932 >> 172.21.15.18:0/3028359932 conn(0x55d0e87ae980 msgr2=0x55d0e87d49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.495+0000 7f01a7b81740 1 -- 172.21.15.18:0/3028359932 shutdown_connections 2024-02-17T13:03:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.495+0000 7f01a7b81740 1 -- 172.21.15.18:0/3028359932 wait complete. 2024-02-17T13:03:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.495+0000 7f01a7b81740 1 -- start start 2024-02-17T13:03:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.495+0000 7f01a7b81740 1 Processor -- start 2024-02-17T13:03:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f01a7b81740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 0x55d0e87c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f01a7b81740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87b4b70 0x55d0e87c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f01a7b81740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87bd750 0x55d0e87c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f01a7b81740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e87c4f20 con 0x55d0e87b4250 2024-02-17T13:03:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f01a7b81740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e87c3680 con 0x55d0e87b4b70 2024-02-17T13:03:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f01a7b81740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e87c3840 con 0x55d0e87bd750 2024-02-17T13:03:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f01a7b81740 10 client.0 _ll_register_callbacks cb 0x7f0198009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f019e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 0x55d0e87c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f019dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87b4b70 0x55d0e87c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f019e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 0x55d0e87c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41210/0 (socket says 172.21.15.18:41210) 2024-02-17T13:03:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f019e59c640 1 -- 172.21.15.18:0/2354815329 learned_addr learned my addr 172.21.15.18:0/2354815329 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f019ed9d640 1 --2- 172.21.15.18:0/2354815329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87bd750 0x55d0e87c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f019dd9b640 1 -- 172.21.15.18:0/2354815329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87bd750 msgr2=0x55d0e87c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f019dd9b640 1 --2- 172.21.15.18:0/2354815329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87bd750 0x55d0e87c3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f019dd9b640 1 -- 172.21.15.18:0/2354815329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 msgr2=0x55d0e87c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f019dd9b640 1 --2- 172.21.15.18:0/2354815329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 0x55d0e87c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f019dd9b640 1 -- 172.21.15.18:0/2354815329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0e871a6a0 con 0x55d0e87b4b70 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f019dd9b640 1 --2- 172.21.15.18:0/2354815329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87b4b70 0x55d0e87c2c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55d0e87aa2f0 tx=0x7f019402cd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9c519a30f1ba474 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.496+0000 7f0185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f0185ffb640 1 -- 172.21.15.18:0/2354815329 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194060070 con 0x55d0e87b4b70 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f0185ffb640 1 -- 172.21.15.18:0/2354815329 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0194000e40 con 0x55d0e87b4b70 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f01a7b81740 1 -- 172.21.15.18:0/2354815329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0e841b7e0 con 0x55d0e87b4b70 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f0185ffb640 1 -- 172.21.15.18:0/2354815329 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0194058040 con 0x55d0e87b4b70 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f01a7b81740 10 client.29417 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f01a7b81740 1 -- 172.21.15.18:0/2354815329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x55d0e86dc8c0 con 0x55d0e87b4b70 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f0185ffb640 1 -- 172.21.15.18:0/2354815329 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d0e841b7e0 con 0x55d0e87b4b70 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f0185ffb640 1 -- 172.21.15.18:0/2354815329 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f0194063020 con 0x55d0e87b4b70 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f01a7b81740 10 client.29417 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f01a7b81740 10 client.29417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f01a7b81740 20 client.29417 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f01a7b81740 1 --2- 172.21.15.18:0/2354815329 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d0e8825170 0x55d0e8827590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f01a7b81740 4 client.29417 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.497+0000 7f01a7b81740 1 -- 172.21.15.18:0/2354815329 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d0e83c7700 con 0x55d0e8825170 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.499+0000 7f019e59c640 1 --2- 172.21.15.18:0/2354815329 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d0e8825170 0x55d0e8827590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.499+0000 7f019e59c640 1 --2- 172.21.15.18:0/2354815329 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d0e8825170 0x55d0e8827590 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7a8477e933d1f3b server_cookie=7917f838c4f4afef in_seq=0 out_seq=0 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.499+0000 7f0185ffb640 10 client.29417 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f0185ffb640 1 -- 172.21.15.18:0/2354815329 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d0e83c7700 con 0x55d0e8825170 2024-02-17T13:03:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f0185ffb640 10 client.29417 handle_command_reply: tid=0 2024-02-17T13:03:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a7b81740 1 client.29417 shutdown 2024-02-17T13:03:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a7b81740 1 -- 172.21.15.18:0/2354815329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87b4b70 msgr2=0x55d0e87c2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a7b81740 1 --2- 172.21.15.18:0/2354815329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87b4b70 0x55d0e87c2c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55d0e87aa2f0 tx=0x7f019402cd90 comp rx=0 tx=0).stop 2024-02-17T13:03:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a7b81740 1 -- 172.21.15.18:0/2354815329 shutdown_connections 2024-02-17T13:03:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a7b81740 1 --2- 172.21.15.18:0/2354815329 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55d0e8825170 0x55d0e8827590 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a7b81740 1 --2- 172.21.15.18:0/2354815329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d0e87bd750 0x55d0e87c3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f0185ffb640 0 client.29417 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a7b81740 1 --2- 172.21.15.18:0/2354815329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0e87b4b70 0x55d0e87c2c00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a7b81740 1 --2- 172.21.15.18:0/2354815329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0e87b4250 0x55d0e87c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a7b81740 1 -- 172.21.15.18:0/2354815329 >> 172.21.15.18:0/2354815329 conn(0x55d0e87ae980 msgr2=0x55d0e87ad2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a7b81740 1 -- 172.21.15.18:0/2354815329 shutdown_connections 2024-02-17T13:03:53.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a7b81740 1 -- 172.21.15.18:0/2354815329 wait complete. 2024-02-17T13:03:53.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a7b81740 20 client.29417 trim_cache size 0 max 16384 2024-02-17T13:03:53.496 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-02-17T13:03:53.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a4edd640 1 -- 172.21.15.18:0/5669017 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f01880a22b0 msgr2=0x7f01880a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a4edd640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f01880a22b0 0x7f01880a4770 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0198022000 tx=0x7f0198023000 comp rx=0 tx=0).stop 2024-02-17T13:03:53.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a4edd640 1 -- 172.21.15.18:0/5669017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a00685f0 msgr2=0x7f01a019d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:53.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.500+0000 7f01a4edd640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a00685f0 0x7f01a019d470 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f019000e9f0 tx=0x7f019000eeb0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.501+0000 7f01a4edd640 1 -- 172.21.15.18:0/5669017 shutdown_connections 2024-02-17T13:03:53.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.501+0000 7f01a4edd640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f01880a22b0 0x7f01880a4770 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.501+0000 7f01a4edd640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01a0197620 0x7f01a0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.501+0000 7f01a4edd640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01a01064c0 0x7f01a019d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.501+0000 7f01a4edd640 1 --2- 172.21.15.18:0/5669017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a00685f0 0x7f01a019d470 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:53.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.501+0000 7f01a4edd640 1 -- 172.21.15.18:0/5669017 >> 172.21.15.18:0/5669017 conn(0x7f01a0077180 msgr2=0x7f01a00fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:53.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.501+0000 7f01a4edd640 1 -- 172.21.15.18:0/5669017 shutdown_connections 2024-02-17T13:03:53.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:03:53.501+0000 7f01a4edd640 1 -- 172.21.15.18:0/5669017 wait complete. 2024-02-17T13:03:53.507 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 17474985, 'ino': 1099511632358, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.Y04'}, {'damage_type': 'backtrace', 'id': 22282374, 'ino': 1099511632502, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ACCESS'}, {'damage_type': 'backtrace', 'id': 44541585, 'ino': 1099511632512, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.FSL'}, {'damage_type': 'backtrace', 'id': 68402129, 'ino': 1099511632555, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/LARGE.FIL'}, {'damage_type': 'backtrace', 'id': 83205717, 'ino': 1099511632319, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.PX'}, {'damage_type': 'backtrace', 'id': 100114821, 'ino': 1099511631951, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp'}, {'damage_type': 'backtrace', 'id': 201061690, 'ino': 1099511631960, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp'}, {'damage_type': 'backtrace', 'id': 217042116, 'ino': 1099511631972, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.000'}, {'damage_type': 'backtrace', 'id': 228887469, 'ino': 1099511632331, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.X04'}, {'damage_type': 'backtrace', 'id': 250361605, 'ino': 1099511632165, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/GRAPHIC1.CDR'}, {'damage_type': 'backtrace', 'id': 265689946, 'ino': 1099511632536, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED'}, {'damage_type': 'backtrace', 'id': 322599022, 'ino': 1099511632342, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.Y04'}, {'damage_type': 'backtrace', 'id': 353990034, 'ino': 1099511632027, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/CHAP10.DOC'}, {'damage_type': 'backtrace', 'id': 354003082, 'ino': 1099511632318, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/MOVED.DOC'}, {'damage_type': 'backtrace', 'id': 379030449, 'ino': 1099511632145, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL'}, {'damage_type': 'backtrace', 'id': 412091781, 'ino': 1099511632360, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.PX'}, {'damage_type': 'backtrace', 'id': 420857839, 'ino': 1099511632489, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.TV'}, {'damage_type': 'backtrace', 'id': 460890606, 'ino': 1099511632468, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.Y03'}, {'damage_type': 'backtrace', 'id': 480069620, 'ino': 1099511632001, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.003'}, {'damage_type': 'backtrace', 'id': 501046091, 'ino': 1099511632183, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/TEAPOT2.CPT'}, {'damage_type': 'backtrace', 'id': 515837876, 'ino': 1099511632122, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/TRIDOTS.POT'}, {'damage_type': 'backtrace', 'id': 520301462, 'ino': 1099511632327, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX'}, {'damage_type': 'backtrace', 'id': 534726608, 'ino': 1099511632413, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.YG1'}, {'damage_type': 'backtrace', 'id': 544404046, 'ino': 1099511632041, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/BASEMACH.DOC'}, {'damage_type': 'backtrace', 'id': 581679058, 'ino': 1099511632440, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.DB'}, {'damage_type': 'backtrace', 'id': 599604896, 'ino': 1099511632483, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.FSL'}, {'damage_type': 'backtrace', 'id': 656497680, 'ino': 1099511632012, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/CHAP10.DOC'}, {'damage_type': 'backtrace', 'id': 724049895, 'ino': 1099511632100, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/PPTOOLS1.PPA'}, {'damage_type': 'backtrace', 'id': 724230668, 'ino': 1099511632037, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/FACTS.DOC'}, {'damage_type': 'backtrace', 'id': 741883858, 'ino': 1099511632192, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/SIGN2.CPT'}, {'damage_type': 'backtrace', 'id': 818291441, 'ino': 1099511632112, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/TIPS.PPT'}, {'damage_type': 'backtrace', 'id': 823683864, 'ino': 1099511632210, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/GRAPHIC2.CDR'}, {'damage_type': 'backtrace', 'id': 849382321, 'ino': 1099511632247, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/RESULTS.XLS'}, {'damage_type': 'backtrace', 'id': 882382375, 'ino': 1099511632416, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.MB'}, {'damage_type': 'backtrace', 'id': 902448862, 'ino': 1099511632394, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.YG0'}, {'damage_type': 'backtrace', 'id': 908746010, 'ino': 1099511632478, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.VAL'}, {'damage_type': 'backtrace', 'id': 993703516, 'ino': 1099511632367, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.DB'}, {'damage_type': 'backtrace', 'id': 1108650990, 'ino': 1099511632472, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.TV'}, {'damage_type': 'backtrace', 'id': 1154388300, 'ino': 1099511632047, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/PCMAGCD.XLS'}, {'damage_type': 'backtrace', 'id': 1164477043, 'ino': 1099511632043, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL'}, {'damage_type': 'backtrace', 'id': 1169383195, 'ino': 1099511632058, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/EXCEL/SALES.XLS'}, {'damage_type': 'backtrace', 'id': 1222165985, 'ino': 1099511632129, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL'}, {'damage_type': 'backtrace', 'id': 1259105962, 'ino': 1099511632444, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.X03'}, {'damage_type': 'backtrace', 'id': 1311411245, 'ino': 1099511632291, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM'}, {'damage_type': 'backtrace', 'id': 1325404355, 'ino': 1099511632228, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CORELDRW.CDT'}, {'damage_type': 'backtrace', 'id': 1357913282, 'ino': 1099511632528, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/MEDIUM.FIL'}, {'damage_type': 'backtrace', 'id': 1360364410, 'ino': 1099511631959, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.000'}, {'damage_type': 'backtrace', 'id': 1363142189, 'ino': 1099511632494, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.FSL'}, {'damage_type': 'backtrace', 'id': 1400705399, 'ino': 1099511632114, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/PPTOOLS1.PPA'}, {'damage_type': 'backtrace', 'id': 1474109852, 'ino': 1099511632195, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/TEAPOT2.CPT'}, {'damage_type': 'backtrace', 'id': 1495218118, 'ino': 1099511632297, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/T1.XLS'}, {'damage_type': 'backtrace', 'id': 1513157904, 'ino': 1099511632298, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/MOVED.DOC'}, {'damage_type': 'backtrace', 'id': 1527520179, 'ino': 1099511632094, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/PCBENCHM.PPT'}, {'damage_type': 'backtrace', 'id': 1536856915, 'ino': 1099511632431, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.MB'}, {'damage_type': 'backtrace', 'id': 1648270889, 'ino': 1099511632158, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CDRBARS.CFG'}, {'damage_type': 'backtrace', 'id': 1722770758, 'ino': 1099511632201, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/TRAIN.CDR'}, {'damage_type': 'backtrace', 'id': 1803634382, 'ino': 1099511632283, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/EVENTS.DOC'}, {'damage_type': 'backtrace', 'id': 1830575748, 'ino': 1099511632514, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED'}, {'damage_type': 'backtrace', 'id': 1831448260, 'ino': 1099511632425, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.DB'}, {'damage_type': 'backtrace', 'id': 1858883827, 'ino': 1099511632311, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/COURSES.DB'}, {'damage_type': 'backtrace', 'id': 1872555523, 'ino': 1099511632142, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CDRBARS.CFG'}, {'damage_type': 'backtrace', 'id': 1878802967, 'ino': 1099511632167, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ARTISTIC.ACL'}, {'damage_type': 'backtrace', 'id': 1923080944, 'ino': 1099511632022, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD'}, {'damage_type': 'backtrace', 'id': 2005278208, 'ino': 1099511632017, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.004'}, {'damage_type': 'backtrace', 'id': 2071251410, 'ino': 1099511631990, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.003'}, {'damage_type': 'backtrace', 'id': 2085423244, 'ino': 1099511632246, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CORELDRW.CDT'}, {'damage_type': 'backtrace', 'id': 2100713521, 'ino': 1099511632282, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/NEWS1_1.LWP'}, {'damage_type': 'backtrace', 'id': 2112683687, 'ino': 1099511632523, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ACCESS'}, {'damage_type': 'backtrace', 'id': 2125244884, 'ino': 1099511632223, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/GRAPHIC2.CDR'}, {'damage_type': 'backtrace', 'id': 2156267498, 'ino': 1099511632021, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD/BASEMACH.DOC'}, {'damage_type': 'backtrace', 'id': 2171783352, 'ino': 1099511632104, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/ZD16.BMP'}, {'damage_type': 'backtrace', 'id': 2213102250, 'ino': 1099511632351, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.DB'}, {'damage_type': 'backtrace', 'id': 2216007626, 'ino': 1099511632064, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL'}, {'damage_type': 'backtrace', 'id': 2280932768, 'ino': 1099511632213, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/TRAIN.CDR'}, {'damage_type': 'backtrace', 'id': 2385846216, 'ino': 1099511632397, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG1'}, {'damage_type': 'backtrace', 'id': 2393294834, 'ino': 1099511631981, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.001'}, {'damage_type': 'backtrace', 'id': 2482896815, 'ino': 1099511632526, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/ACCESS/FASTENER.MDB'}, {'damage_type': 'backtrace', 'id': 2489634982, 'ino': 1099511632462, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.VAL'}, {'damage_type': 'backtrace', 'id': 2490995176, 'ino': 1099511632270, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM'}, {'damage_type': 'backtrace', 'id': 2496965368, 'ino': 1099511632275, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PM/T1.XLS'}, {'damage_type': 'backtrace', 'id': 2638445658, 'ino': 1099511632538, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/SMALL.FIL'}, {'damage_type': 'backtrace', 'id': 2657798143, 'ino': 1099511632081, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/SALES.XLS'}, {'damage_type': 'backtrace', 'id': 2676251669, 'ino': 1099511631971, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.001'}, {'damage_type': 'backtrace', 'id': 2689372003, 'ino': 1099511632548, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/SEED/MEDIUM.FIL'}, {'damage_type': 'backtrace', 'id': 2712507909, 'ino': 1099511631991, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/filler.002'}, {'damage_type': 'backtrace', 'id': 2773157621, 'ino': 1099511632000, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.004'}, {'damage_type': 'backtrace', 'id': 2780947645, 'ino': 1099511632330, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.DB'}, {'damage_type': 'backtrace', 'id': 2815618509, 'ino': 1099511632434, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.PX'}, {'damage_type': 'backtrace', 'id': 2829938149, 'ino': 1099511631980, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/filler.002'}, {'damage_type': 'backtrace', 'id': 2839137842, 'ino': 1099511632217, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ROSE.CPT'}, {'damage_type': 'backtrace', 'id': 2864698331, 'ino': 1099511632349, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.X04'}, {'damage_type': 'backtrace', 'id': 2870698099, 'ino': 1099511632517, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/SMALL.FIL'}, {'damage_type': 'backtrace', 'id': 2900366143, 'ino': 1099511632388, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG1'}, {'damage_type': 'backtrace', 'id': 2932299629, 'ino': 1099511632453, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/ENROLL.Y03'}, {'damage_type': 'backtrace', 'id': 2932747824, 'ino': 1099511632233, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/ROSE.CPT'}, {'damage_type': 'backtrace', 'id': 2981302684, 'ino': 1099511632149, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/CDRROLS.CFG'}, {'damage_type': 'backtrace', 'id': 2992330079, 'ino': 1099511632500, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.FSL'}, {'damage_type': 'backtrace', 'id': 3154793778, 'ino': 1099511632177, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/GRAPHIC1.CDR'}, {'damage_type': 'backtrace', 'id': 3234848247, 'ino': 1099511632124, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/TIPS.PPT'}, {'damage_type': 'backtrace', 'id': 3261215441, 'ino': 1099511632369, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.XG0'}, {'damage_type': 'backtrace', 'id': 3293556358, 'ino': 1099511632305, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PM/EVENTS.DOC'}, {'damage_type': 'backtrace', 'id': 3301724578, 'ino': 1099511632378, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.YG0'}, {'damage_type': 'backtrace', 'id': 3324533253, 'ino': 1099511632186, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/MAILBOX2.CPT'}, {'damage_type': 'backtrace', 'id': 3328139832, 'ino': 1099511632239, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/BENCHS.LWP'}, {'damage_type': 'backtrace', 'id': 3330523578, 'ino': 1099511632306, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX'}, {'damage_type': 'backtrace', 'id': 3430028879, 'ino': 1099511632406, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PARADOX/STUDENTS.VAL'}, {'damage_type': 'backtrace', 'id': 3509547545, 'ino': 1099511632458, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.X03'}, {'damage_type': 'backtrace', 'id': 3532425201, 'ino': 1099511632090, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT'}, {'damage_type': 'backtrace', 'id': 3535994564, 'ino': 1099511632507, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/ACCESS/FASTENER.MDB'}, {'damage_type': 'backtrace', 'id': 3649556935, 'ino': 1099511632174, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/MAILBOX2.CPT'}, {'damage_type': 'backtrace', 'id': 3656796612, 'ino': 1099511632422, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.VAL'}, {'damage_type': 'backtrace', 'id': 3674886703, 'ino': 1099511632008, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORD'}, {'damage_type': 'backtrace', 'id': 3677647580, 'ino': 1099511632136, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PWRPNT/TRIDOTS.POT'}, {'damage_type': 'backtrace', 'id': 3692077939, 'ino': 1099511632385, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.XG0'}, {'damage_type': 'backtrace', 'id': 3717969239, 'ino': 1099511632155, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/ARTISTIC.ACL'}, {'damage_type': 'backtrace', 'id': 3728128989, 'ino': 1099511632541, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/SEED/LARGE.FIL'}, {'damage_type': 'backtrace', 'id': 3747767585, 'ino': 1099511632449, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/ENROLL.PX'}, {'damage_type': 'backtrace', 'id': 3756507551, 'ino': 1099511632086, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/ZD16.BMP'}, {'damage_type': 'backtrace', 'id': 3811175711, 'ino': 1099511632234, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO'}, {'damage_type': 'backtrace', 'id': 3930020264, 'ino': 1099511632403, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.XG1'}, {'damage_type': 'backtrace', 'id': 4012855282, 'ino': 1099511632376, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/STUDENTS.PX'}, {'damage_type': 'backtrace', 'id': 4013686900, 'ino': 1099511632068, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/EXCEL/PCMAGCD.XLS'}, {'damage_type': 'backtrace', 'id': 4046892807, 'ino': 1099511632261, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/BENCHS.LWP'}, {'damage_type': 'backtrace', 'id': 4057049422, 'ino': 1099511632132, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/COREL/CDRROLS.CFG'}, {'damage_type': 'backtrace', 'id': 4128918081, 'ino': 1099511632204, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/COREL/SIGN2.CPT'}, {'damage_type': 'backtrace', 'id': 4131300286, 'ino': 1099511632262, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/WORDPRO/NEWS1_1.LWP'}, {'damage_type': 'backtrace', 'id': 4155482369, 'ino': 1099511632074, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT/PCBENCHM.PPT'}, {'damage_type': 'backtrace', 'id': 4184038870, 'ino': 1099511632269, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO/RESULTS.XLS'}, {'damage_type': 'backtrace', 'id': 4209615699, 'ino': 1099511632255, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORDPRO'}, {'damage_type': 'backtrace', 'id': 4243877982, 'ino': 1099511632056, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/WORD/FACTS.DOC'}, {'damage_type': 'backtrace', 'id': 4272231202, 'ino': 1099511632070, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client3/~dmtmp/PWRPNT'}, {'damage_type': 'backtrace', 'id': 4292119270, 'ino': 1099511632340, 'path': '/volumes/qa/sv_1/323825cb-6c90-4285-a584-8b73e60121df/client.0/tmp/clients/client6/~dmtmp/PARADOX/COURSES.PX'}] 2024-02-17T13:03:53.507 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-02-17T13:03:53.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:53 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3850103697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:03:53.787 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36057 32.51 MB/sec warmup 65 sec latency 157.814 ms 2024-02-17T13:03:53.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:53 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3850103697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:03:53.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:53 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3850103697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:03:54.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:54 smithi018 ceph-mon[32625]: pgmap v440: 97 pgs: 97 active+clean; 629 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 736 op/s 2024-02-17T13:03:54.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36871 32.71 MB/sec warmup 66 sec latency 189.597 ms 2024-02-17T13:03:54.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:54 smithi119 ceph-mon[36188]: pgmap v440: 97 pgs: 97 active+clean; 629 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 736 op/s 2024-02-17T13:03:54.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:54 smithi146 ceph-mon[36454]: pgmap v440: 97 pgs: 97 active+clean; 629 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 736 op/s 2024-02-17T13:03:55.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 37709 32.91 MB/sec warmup 67 sec latency 132.821 ms 2024-02-17T13:03:56.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:56 smithi018 ceph-mon[32625]: pgmap v441: 97 pgs: 97 active+clean; 617 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-02-17T13:03:56.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 38691 33.23 MB/sec warmup 68 sec latency 39.068 ms 2024-02-17T13:03:56.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:56 smithi119 ceph-mon[36188]: pgmap v441: 97 pgs: 97 active+clean; 617 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-02-17T13:03:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.907+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/3131520581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3df8100360 msgr2=0x7f3df81007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.907+0000 7f3dfcfbc640 1 --2- 172.21.15.18:0/3131520581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3df8100360 0x7f3df81007c0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009890 tx=0x7f3de002bb70 comp rx=0 tx=0).stop 2024-02-17T13:03:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.907+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/3131520581 shutdown_connections 2024-02-17T13:03:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.907+0000 7f3dfcfbc640 1 --2- 172.21.15.18:0/3131520581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df8100d00 0x7f3df810cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.907+0000 7f3dfcfbc640 1 --2- 172.21.15.18:0/3131520581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3df8100360 0x7f3df81007c0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.907+0000 7f3dfcfbc640 1 --2- 172.21.15.18:0/3131520581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df81050a0 0x7f3df8105480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.907+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/3131520581 >> 172.21.15.18:0/3131520581 conn(0x7f3df8077060 msgr2=0x7f3df8077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.907+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/3131520581 shutdown_connections 2024-02-17T13:03:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.907+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/3131520581 wait complete. 2024-02-17T13:03:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.907+0000 7f3dfcfbc640 1 Processor -- start 2024-02-17T13:03:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.907+0000 7f3dfcfbc640 1 -- start start 2024-02-17T13:03:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3dfcfbc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3df8100360 0x7f3df810d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3dfcfbc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df8100d00 0x7f3df810de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3dfcfbc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df81050a0 0x7f3df810e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3dfcfbc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df81a5b20 con 0x7f3df8100360 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3dfcfbc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df81a5c90 con 0x7f3df81050a0 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3dfcfbc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df81a5e50 con 0x7f3df8100d00 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3df659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3df8100360 0x7f3df810d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3df659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3df8100360 0x7f3df810d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41234/0 (socket says 172.21.15.18:41234) 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3df659c640 1 -- 172.21.15.18:0/71405378 learned_addr learned my addr 172.21.15.18:0/71405378 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3deffff640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df8100d00 0x7f3df810de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3df6d9d640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df81050a0 0x7f3df810e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3df659c640 1 -- 172.21.15.18:0/71405378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df8100d00 msgr2=0x7f3df810de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3df659c640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df8100d00 0x7f3df810de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3df659c640 1 -- 172.21.15.18:0/71405378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df81050a0 msgr2=0x7f3df810e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3df659c640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df81050a0 0x7f3df810e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3df659c640 1 -- 172.21.15.18:0/71405378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de0009490 con 0x7f3df8100360 2024-02-17T13:03:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3df6d9d640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df81050a0 0x7f3df810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:03:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.908+0000 7f3df659c640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3df8100360 0x7f3df810d8c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f3de800e9f0 tx=0x7f3de800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.909+0000 7f3def7fe640 1 -- 172.21.15.18:0/71405378 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de800ccc0 con 0x7f3df8100360 2024-02-17T13:03:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.909+0000 7f3def7fe640 1 -- 172.21.15.18:0/71405378 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de800ce20 con 0x7f3df8100360 2024-02-17T13:03:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.909+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/71405378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df81a6130 con 0x7f3df8100360 2024-02-17T13:03:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.909+0000 7f3def7fe640 1 -- 172.21.15.18:0/71405378 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8010690 con 0x7f3df8100360 2024-02-17T13:03:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.909+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/71405378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df81a65c0 con 0x7f3df8100360 2024-02-17T13:03:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.911+0000 7f3def7fe640 1 -- 172.21.15.18:0/71405378 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3de80046c0 con 0x7f3df8100360 2024-02-17T13:03:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.911+0000 7f3dffb8f740 1 Processor -- start 2024-02-17T13:03:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3def7fe640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3de40a22d0 0x7f3de40a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3def7fe640 1 -- 172.21.15.18:0/71405378 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3de8014070 con 0x7f3df8100360 2024-02-17T13:03:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3df6d9d640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3de40a22d0 0x7f3de40a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3dffb8f740 1 -- start start 2024-02-17T13:03:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3dffb8f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4aead0 0x562e0f4ad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3dffb8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4add60 0x562e0f4ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3dffb8f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4b72e0 0x562e0f4b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3dffb8f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0f41c9f0 con 0x562e0f4aead0 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3dffb8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0f419f20 con 0x562e0f4add60 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3dffb8f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0f415700 con 0x562e0f4b72e0 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3df659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4aead0 0x562e0f4ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3df659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4aead0 0x562e0f4ad820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41240/0 (socket says 172.21.15.18:41240) 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3df6d9d640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3de40a22d0 0x7f3de40a4790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3df0005fd0 tx=0x7f3df0009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3df659c640 1 -- 172.21.15.18:0/4275100614 learned_addr learned my addr 172.21.15.18:0/4275100614 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3df6d9d640 1 --2- 172.21.15.18:0/4275100614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4b72e0 0x562e0f4b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.912+0000 7f3deffff640 1 --2- 172.21.15.18:0/4275100614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4add60 0x562e0f4ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.913+0000 7f3df659c640 1 -- 172.21.15.18:0/4275100614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4b72e0 msgr2=0x562e0f4b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.913+0000 7f3df659c640 1 --2- 172.21.15.18:0/4275100614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4b72e0 0x562e0f4b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.913+0000 7f3df659c640 1 -- 172.21.15.18:0/4275100614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4add60 msgr2=0x562e0f4ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.913+0000 7f3df659c640 1 --2- 172.21.15.18:0/4275100614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4add60 0x562e0f4ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.913+0000 7f3df659c640 1 -- 172.21.15.18:0/4275100614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e0f3ab7b0 con 0x562e0f4aead0 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.913+0000 7f3df659c640 1 --2- 172.21.15.18:0/4275100614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4aead0 0x562e0f4ad820 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f3de8097780 tx=0x7f3de8009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde12c635cd13358 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.913+0000 7f3decff9640 1 -- 172.21.15.18:0/4275100614 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de80a3550 con 0x562e0f4aead0 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.913+0000 7f3decff9640 1 -- 172.21.15.18:0/4275100614 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de809e070 con 0x562e0f4aead0 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.913+0000 7f3decff9640 1 -- 172.21.15.18:0/4275100614 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de80a30c0 con 0x562e0f4aead0 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.914+0000 7f3dffb8f740 1 -- 172.21.15.18:0/4275100614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4aead0 msgr2=0x562e0f4ad820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.914+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/4275100614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4aead0 0x562e0f4ad820 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f3de8097780 tx=0x7f3de8009f50 comp rx=0 tx=0).stop 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.914+0000 7f3dffb8f740 1 -- 172.21.15.18:0/4275100614 shutdown_connections 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.914+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/4275100614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4b72e0 0x562e0f4b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.914+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/4275100614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4add60 0x562e0f4ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.914+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/4275100614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4aead0 0x562e0f4ad820 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.914+0000 7f3dffb8f740 1 -- 172.21.15.18:0/4275100614 >> 172.21.15.18:0/4275100614 conn(0x562e0f475550 msgr2=0x562e0f4a8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.914+0000 7f3dffb8f740 1 -- 172.21.15.18:0/4275100614 shutdown_connections 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.914+0000 7f3dffb8f740 1 -- 172.21.15.18:0/4275100614 wait complete. 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.914+0000 7f3dffb8f740 1 -- start start 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.914+0000 7f3dffb8f740 1 Processor -- start 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.914+0000 7f3dffb8f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 0x562e0f4b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3dffb8f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4aead0 0x562e0f4b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3dffb8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4b72e0 0x562e0f4b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3dffb8f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0f3babf0 con 0x562e0f4add60 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3dffb8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0f0e9250 con 0x562e0f4b72e0 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3dffb8f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0f39b200 con 0x562e0f4aead0 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3dffb8f740 10 client.0 _ll_register_callbacks cb 0x562e0f409d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3df659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 0x562e0f4b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3df659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 0x562e0f4b42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41254/0 (socket says 172.21.15.18:41254) 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3df659c640 1 -- 172.21.15.18:0/1649201357 learned_addr learned my addr 172.21.15.18:0/1649201357 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3deffff640 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4aead0 0x562e0f4b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3df6d9d640 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4b72e0 0x562e0f4b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3df659c640 1 -- 172.21.15.18:0/1649201357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4aead0 msgr2=0x562e0f4b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3df659c640 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4aead0 0x562e0f4b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3df659c640 1 -- 172.21.15.18:0/1649201357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4b72e0 msgr2=0x562e0f4b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3df659c640 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4b72e0 0x562e0f4b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3df659c640 1 -- 172.21.15.18:0/1649201357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df81a65c0 con 0x562e0f4add60 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3df659c640 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 0x562e0f4b42e0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f3de800c6a0 tx=0x7f3de8097050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c3d2236d24f6bd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3dddffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3dddffb640 1 -- 172.21.15.18:0/1649201357 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de80b91f0 con 0x562e0f4add60 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3dddffb640 1 -- 172.21.15.18:0/1649201357 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3de80662c0 con 0x562e0f4add60 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.915+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1649201357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e0f115600 con 0x562e0f4add60 2024-02-17T13:03:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.916+0000 7f3dddffb640 1 -- 172.21.15.18:0/1649201357 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3de80a30c0 con 0x562e0f4add60 2024-02-17T13:03:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.916+0000 7f3dffb8f740 10 client.20601 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.916+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1649201357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x562e0f413d80 con 0x562e0f4add60 2024-02-17T13:03:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.916+0000 7f3dddffb640 1 -- 172.21.15.18:0/1649201357 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562e0f115600 con 0x562e0f4add60 2024-02-17T13:03:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.916+0000 7f3dddffb640 1 -- 172.21.15.18:0/1649201357 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f3de801f060 con 0x562e0f4add60 2024-02-17T13:03:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.916+0000 7f3dffb8f740 10 client.20601 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.916+0000 7f3dffb8f740 10 client.20601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.916+0000 7f3dffb8f740 20 client.20601 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.916+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562e0f4c7320 0x562e0f4c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.916+0000 7f3dffb8f740 4 client.20601 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:03:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.916+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1649201357 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e0f1661b0 con 0x562e0f4c7320 2024-02-17T13:03:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.918+0000 7f3deffff640 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562e0f4c7320 0x562e0f4c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.918+0000 7f3deffff640 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562e0f4c7320 0x562e0f4c9740 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45eada37638e8736 server_cookie=f285db5819f33a57 in_seq=0 out_seq=0 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.918+0000 7f3dddffb640 10 client.20601 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dddffb640 1 -- 172.21.15.18:0/1649201357 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e0f1661b0 con 0x562e0f4c7320 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dddffb640 10 client.20601 handle_command_reply: tid=0 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dffb8f740 1 client.20601 shutdown 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1649201357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 msgr2=0x562e0f4b42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 0x562e0f4b42e0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f3de800c6a0 tx=0x7f3de8097050 comp rx=0 tx=0).stop 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1649201357 shutdown_connections 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562e0f4c7320 0x562e0f4c9740 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4b72e0 0x562e0f4b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4aead0 0x562e0f4b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dddffb640 0 client.20601 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/1649201357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 0x562e0f4b42e0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1649201357 >> 172.21.15.18:0/1649201357 conn(0x562e0f475550 msgr2=0x562e0f4aa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1649201357 shutdown_connections 2024-02-17T13:03:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1649201357 wait complete. 2024-02-17T13:03:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.922+0000 7f3dffb8f740 20 client.20601 trim_cache size 0 max 16384 2024-02-17T13:03:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.941+0000 7f3dffb8f740 1 Processor -- start 2024-02-17T13:03:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.941+0000 7f3dffb8f740 1 -- start start 2024-02-17T13:03:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.941+0000 7f3dffb8f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4add60 0x562e0f4aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.941+0000 7f3dffb8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4aead0 0x562e0f4be110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.941+0000 7f3dffb8f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4b72e0 0x562e0f4be650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.941+0000 7f3dffb8f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0f115600 con 0x562e0f4b72e0 2024-02-17T13:03:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.941+0000 7f3dffb8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0f4adb30 con 0x562e0f4aead0 2024-02-17T13:03:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.941+0000 7f3dffb8f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0f4a8310 con 0x562e0f4add60 2024-02-17T13:03:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.942+0000 7f3df6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4b72e0 0x562e0f4be650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.942+0000 7f3df6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4b72e0 0x562e0f4be650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41256/0 (socket says 172.21.15.18:41256) 2024-02-17T13:03:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.942+0000 7f3df6d9d640 1 -- 172.21.15.18:0/3025401753 learned_addr learned my addr 172.21.15.18:0/3025401753 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.942+0000 7f3df659c640 1 --2- 172.21.15.18:0/3025401753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4add60 0x562e0f4aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.942+0000 7f3deffff640 1 --2- 172.21.15.18:0/3025401753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4aead0 0x562e0f4be110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.942+0000 7f3df6d9d640 1 -- 172.21.15.18:0/3025401753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4add60 msgr2=0x562e0f4aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.942+0000 7f3df6d9d640 1 --2- 172.21.15.18:0/3025401753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4add60 0x562e0f4aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.942+0000 7f3df6d9d640 1 -- 172.21.15.18:0/3025401753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4aead0 msgr2=0x562e0f4be110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.942+0000 7f3df6d9d640 1 --2- 172.21.15.18:0/3025401753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4aead0 0x562e0f4be110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.942+0000 7f3df6d9d640 1 -- 172.21.15.18:0/3025401753 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e0f3b07b0 con 0x562e0f4b72e0 2024-02-17T13:03:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.943+0000 7f3df6d9d640 1 --2- 172.21.15.18:0/3025401753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4b72e0 0x562e0f4be650 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f3df00119b0 tx=0x7f3df0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468b43c1a7eee964 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.943+0000 7f3ddffff640 1 -- 172.21.15.18:0/3025401753 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df0024070 con 0x562e0f4b72e0 2024-02-17T13:03:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.943+0000 7f3ddffff640 1 -- 172.21.15.18:0/3025401753 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3df0015070 con 0x562e0f4b72e0 2024-02-17T13:03:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.943+0000 7f3ddffff640 1 -- 172.21.15.18:0/3025401753 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df0024070 con 0x562e0f4b72e0 2024-02-17T13:03:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.944+0000 7f3dffb8f740 1 -- 172.21.15.18:0/3025401753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4b72e0 msgr2=0x562e0f4be650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.944+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/3025401753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4b72e0 0x562e0f4be650 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f3df00119b0 tx=0x7f3df0013000 comp rx=0 tx=0).stop 2024-02-17T13:03:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.945+0000 7f3dffb8f740 1 -- 172.21.15.18:0/3025401753 shutdown_connections 2024-02-17T13:03:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.945+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/3025401753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4b72e0 0x562e0f4be650 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.945+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/3025401753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4aead0 0x562e0f4be110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.945+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/3025401753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4add60 0x562e0f4aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.945+0000 7f3dffb8f740 1 -- 172.21.15.18:0/3025401753 >> 172.21.15.18:0/3025401753 conn(0x562e0f475550 msgr2=0x562e0f4a9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.945+0000 7f3dffb8f740 1 -- 172.21.15.18:0/3025401753 shutdown_connections 2024-02-17T13:03:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.945+0000 7f3dffb8f740 1 -- 172.21.15.18:0/3025401753 wait complete. 2024-02-17T13:03:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3dffb8f740 1 -- start start 2024-02-17T13:03:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3dffb8f740 1 Processor -- start 2024-02-17T13:03:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3dffb8f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 0x562e0f520310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3dffb8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4aead0 0x562e0f520850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3dffb8f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4b72e0 0x562e0f4bd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3dffb8f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0f4bd8d0 con 0x562e0f4add60 2024-02-17T13:03:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3dffb8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0f4bda40 con 0x562e0f4aead0 2024-02-17T13:03:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3dffb8f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0f4bdc00 con 0x562e0f4b72e0 2024-02-17T13:03:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3dffb8f740 10 client.0 _ll_register_callbacks cb 0x7f3df00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:03:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3df659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 0x562e0f520310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3df659c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 0x562e0f520310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41268/0 (socket says 172.21.15.18:41268) 2024-02-17T13:03:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3df659c640 1 -- 172.21.15.18:0/1742363878 learned_addr learned my addr 172.21.15.18:0/1742363878 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:03:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3df6d9d640 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4b72e0 0x562e0f4bd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.946+0000 7f3deffff640 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4aead0 0x562e0f520850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.947+0000 7f3df659c640 1 -- 172.21.15.18:0/1742363878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4b72e0 msgr2=0x562e0f4bd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.947+0000 7f3df659c640 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4b72e0 0x562e0f4bd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.947+0000 7f3df659c640 1 -- 172.21.15.18:0/1742363878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4aead0 msgr2=0x562e0f520850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:03:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.947+0000 7f3df659c640 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4aead0 0x562e0f520850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.947+0000 7f3df659c640 1 -- 172.21.15.18:0/1742363878 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df0011690 con 0x562e0f4add60 2024-02-17T13:03:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.948+0000 7f3df659c640 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 0x562e0f520310 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009230 tx=0x7f3de800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27885647b1903304 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:03:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.948+0000 7f3dddffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:03:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.948+0000 7f3dddffb640 1 -- 172.21.15.18:0/1742363878 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de809a910 con 0x562e0f4add60 2024-02-17T13:03:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.948+0000 7f3dddffb640 1 -- 172.21.15.18:0/1742363878 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3de80a30c0 con 0x562e0f4add60 2024-02-17T13:03:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.949+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1742363878 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3de809e070 con 0x562e0f4add60 2024-02-17T13:03:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.949+0000 7f3dddffb640 1 -- 172.21.15.18:0/1742363878 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3de80a3280 con 0x562e0f4add60 2024-02-17T13:03:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.949+0000 7f3dffb8f740 10 client.20607 fetch_fsmap learned FSMap version 62 2024-02-17T13:03:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.949+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1742363878 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x562e0f3d6540 con 0x562e0f4add60 2024-02-17T13:03:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.949+0000 7f3dddffb640 1 -- 172.21.15.18:0/1742363878 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de808a080 con 0x562e0f4add60 2024-02-17T13:03:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.950+0000 7f3dddffb640 1 -- 172.21.15.18:0/1742363878 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 62) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f3de801f070 con 0x562e0f4add60 2024-02-17T13:03:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.950+0000 7f3dffb8f740 10 client.20607 fetch_fsmap finished waiting for FSMap version 62 2024-02-17T13:03:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.950+0000 7f3dffb8f740 10 client.20607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:03:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.950+0000 7f3dffb8f740 20 client.20607 populate_metadata read hostname 'smithi018' 2024-02-17T13:03:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.950+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562e0f538cf0 0x562e0f53b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:03:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.950+0000 7f3dffb8f740 4 client.20607 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:03:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.950+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1742363878 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562e0f152550 con 0x562e0f538cf0 2024-02-17T13:03:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.952+0000 7f3deffff640 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562e0f538cf0 0x562e0f53b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:03:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.953+0000 7f3dddffb640 10 client.20607 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:56.953+0000 7f3deffff640 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562e0f538cf0 0x562e0f53b0e0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e421342def5019ca server_cookie=2be40b73e1c85bda in_seq=0 out_seq=0 2024-02-17T13:03:56.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:56 smithi146 ceph-mon[36454]: pgmap v441: 97 pgs: 97 active+clean; 617 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-02-17T13:03:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.236+0000 7f3dddffb640 1 -- 172.21.15.18:0/1742363878 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562e0f152550 con 0x562e0f538cf0 2024-02-17T13:03:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.236+0000 7f3dddffb640 10 client.20607 handle_command_reply: tid=0 2024-02-17T13:03:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.236+0000 7f3dffb8f740 1 client.20607 shutdown 2024-02-17T13:03:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.236+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1742363878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 msgr2=0x562e0f520310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.236+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 0x562e0f520310 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009230 tx=0x7f3de800cf90 comp rx=0 tx=0).stop 2024-02-17T13:03:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.236+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1742363878 shutdown_connections 2024-02-17T13:03:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.236+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x562e0f538cf0 0x562e0f53b0e0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.237+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e0f4b72e0 0x562e0f4bd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.237+0000 7f3dddffb640 0 client.20607 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:03:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.237+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e0f4aead0 0x562e0f520850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.237+0000 7f3dffb8f740 1 --2- 172.21.15.18:0/1742363878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562e0f4add60 0x562e0f520310 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.237+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1742363878 >> 172.21.15.18:0/1742363878 conn(0x562e0f475550 msgr2=0x562e0f4b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.237+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1742363878 shutdown_connections 2024-02-17T13:03:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.237+0000 7f3dffb8f740 1 -- 172.21.15.18:0/1742363878 wait complete. 2024-02-17T13:03:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.237+0000 7f3dffb8f740 20 client.20607 trim_cache size 0 max 16384 2024-02-17T13:03:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:03:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:03:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:03:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:03:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.238+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/71405378 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3de40a22d0 msgr2=0x7f3de40a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.238+0000 7f3dfcfbc640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3de40a22d0 0x7f3de40a4790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3df0005fd0 tx=0x7f3df0009500 comp rx=0 tx=0).stop 2024-02-17T13:03:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.238+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/71405378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3df8100360 msgr2=0x7f3df810d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:03:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.238+0000 7f3dfcfbc640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3df8100360 0x7f3df810d8c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f3de800e9f0 tx=0x7f3de800eeb0 comp rx=0 tx=0).stop 2024-02-17T13:03:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.238+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/71405378 shutdown_connections 2024-02-17T13:03:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.238+0000 7f3dfcfbc640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3de40a22d0 0x7f3de40a4790 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.238+0000 7f3dfcfbc640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3df81050a0 0x7f3df810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.238+0000 7f3dfcfbc640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df8100d00 0x7f3df810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.238+0000 7f3dfcfbc640 1 --2- 172.21.15.18:0/71405378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3df8100360 0x7f3df810d8c0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:03:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.238+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/71405378 >> 172.21.15.18:0/71405378 conn(0x7f3df8077060 msgr2=0x7f3df80fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:03:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.238+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/71405378 shutdown_connections 2024-02-17T13:03:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:03:57.238+0000 7f3dfcfbc640 1 -- 172.21.15.18:0/71405378 wait complete. 2024-02-17T13:03:57.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 39353 33.27 MB/sec warmup 69 sec latency 235.200 ms 2024-02-17T13:03:58.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:03:58 smithi018 ceph-mon[32625]: pgmap v442: 97 pgs: 97 active+clean; 617 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 821 op/s 2024-02-17T13:03:58.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 40275 33.38 MB/sec warmup 70 sec latency 44.007 ms 2024-02-17T13:03:58.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:03:58 smithi119 ceph-mon[36188]: pgmap v442: 97 pgs: 97 active+clean; 617 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 821 op/s 2024-02-17T13:03:58.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:03:58 smithi146 ceph-mon[36454]: pgmap v442: 97 pgs: 97 active+clean; 617 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 821 op/s 2024-02-17T13:03:59.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 41041 33.60 MB/sec warmup 71 sec latency 318.287 ms 2024-02-17T13:04:00.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:00 smithi018 ceph-mon[32625]: pgmap v443: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 951 op/s 2024-02-17T13:04:00.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:00 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:00.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 41460 33.50 MB/sec warmup 72 sec latency 427.632 ms 2024-02-17T13:04:00.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:00 smithi119 ceph-mon[36188]: pgmap v443: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 951 op/s 2024-02-17T13:04:00.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:00 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:00.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:00 smithi146 ceph-mon[36454]: pgmap v443: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 951 op/s 2024-02-17T13:04:00.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:00 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:01.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 41992 33.33 MB/sec warmup 73 sec latency 512.007 ms 2024-02-17T13:04:02.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:02 smithi018 ceph-mon[32625]: pgmap v444: 97 pgs: 97 active+clean; 621 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-02-17T13:04:02.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 42517 33.32 MB/sec warmup 74 sec latency 114.635 ms 2024-02-17T13:04:02.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:02 smithi119 ceph-mon[36188]: pgmap v444: 97 pgs: 97 active+clean; 621 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-02-17T13:04:02.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:02 smithi146 ceph-mon[36454]: pgmap v444: 97 pgs: 97 active+clean; 621 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-02-17T13:04:03.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 43561 33.53 MB/sec warmup 75 sec latency 49.903 ms 2024-02-17T13:04:04.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:04 smithi018 ceph-mon[32625]: pgmap v445: 97 pgs: 97 active+clean; 621 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 898 op/s 2024-02-17T13:04:04.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:04 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:04.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 44320 33.64 MB/sec warmup 76 sec latency 222.252 ms 2024-02-17T13:04:04.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:04 smithi119 ceph-mon[36188]: pgmap v445: 97 pgs: 97 active+clean; 621 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 898 op/s 2024-02-17T13:04:04.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:04 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:04.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:04 smithi146 ceph-mon[36454]: pgmap v445: 97 pgs: 97 active+clean; 621 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 898 op/s 2024-02-17T13:04:04.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:04 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:05.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 45008 33.88 MB/sec warmup 77 sec latency 60.968 ms 2024-02-17T13:04:06.750 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:06 smithi146 ceph-mon[36454]: pgmap v446: 97 pgs: 97 active+clean; 640 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 1.43k op/s 2024-02-17T13:04:06.750 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:06 smithi146 ceph-mon[36454]: osdmap e107: 12 total, 12 up, 12 in 2024-02-17T13:04:06.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:06 smithi018 ceph-mon[32625]: pgmap v446: 97 pgs: 97 active+clean; 640 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 1.43k op/s 2024-02-17T13:04:06.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:06 smithi018 ceph-mon[32625]: osdmap e107: 12 total, 12 up, 12 in 2024-02-17T13:04:06.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 45451 33.71 MB/sec warmup 78 sec latency 106.943 ms 2024-02-17T13:04:06.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:06 smithi119 ceph-mon[36188]: pgmap v446: 97 pgs: 97 active+clean; 640 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 1.43k op/s 2024-02-17T13:04:06.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:06 smithi119 ceph-mon[36188]: osdmap e107: 12 total, 12 up, 12 in 2024-02-17T13:04:07.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 46256 33.74 MB/sec warmup 79 sec latency 65.488 ms 2024-02-17T13:04:08.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:08 smithi018 ceph-mon[32625]: pgmap v448: 97 pgs: 97 active+clean; 640 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-02-17T13:04:08.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 47211 34.01 MB/sec warmup 80 sec latency 219.707 ms 2024-02-17T13:04:08.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:08 smithi119 ceph-mon[36188]: pgmap v448: 97 pgs: 97 active+clean; 640 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-02-17T13:04:08.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:08 smithi146 ceph-mon[36454]: pgmap v448: 97 pgs: 97 active+clean; 640 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-02-17T13:04:09.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:09 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:09.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 47891 33.93 MB/sec warmup 81 sec latency 1219.735 ms 2024-02-17T13:04:09.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:09 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:09.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:09 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.375+0000 7f1a751eb640 1 -- 172.21.15.18:0/2362605066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a7011a960 msgr2=0x7f1a7011cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.375+0000 7f1a751eb640 1 --2- 172.21.15.18:0/2362605066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a7011a960 0x7f1a7011cd50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009900 tx=0x7f1a6002bc80 comp rx=0 tx=0).stop 2024-02-17T13:04:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.375+0000 7f1a751eb640 1 -- 172.21.15.18:0/2362605066 shutdown_connections 2024-02-17T13:04:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.375+0000 7f1a751eb640 1 --2- 172.21.15.18:0/2362605066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a7011a960 0x7f1a7011cd50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.375+0000 7f1a751eb640 1 --2- 172.21.15.18:0/2362605066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7010fd30 0x7f1a7010da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.375+0000 7f1a751eb640 1 --2- 172.21.15.18:0/2362605066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a7010f8c0 0x7f1a7010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.375+0000 7f1a751eb640 1 -- 172.21.15.18:0/2362605066 >> 172.21.15.18:0/2362605066 conn(0x7f1a7006c220 msgr2=0x7f1a7006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.376+0000 7f1a751eb640 1 -- 172.21.15.18:0/2362605066 shutdown_connections 2024-02-17T13:04:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.376+0000 7f1a751eb640 1 -- 172.21.15.18:0/2362605066 wait complete. 2024-02-17T13:04:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.377+0000 7f1a751eb640 1 Processor -- start 2024-02-17T13:04:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.377+0000 7f1a751eb640 1 -- start start 2024-02-17T13:04:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.377+0000 7f1a751eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7010f8c0 0x7f1a701a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.377+0000 7f1a751eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a7010fd30 0x7f1a701a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.377+0000 7f1a751eb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a7011a960 0x7f1a701aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.377+0000 7f1a751eb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a701ab590 con 0x7f1a7010fd30 2024-02-17T13:04:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.377+0000 7f1a751eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a701ab700 con 0x7f1a7010f8c0 2024-02-17T13:04:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.377+0000 7f1a751eb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a701ab8c0 con 0x7f1a7011a960 2024-02-17T13:04:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.378+0000 7f1a6ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7010f8c0 0x7f1a701a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.378+0000 7f1a6ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7010f8c0 0x7f1a701a66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:50628/0 (socket says 172.21.15.18:50628) 2024-02-17T13:04:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.378+0000 7f1a6ed9d640 1 -- 172.21.15.18:0/3394990850 learned_addr learned my addr 172.21.15.18:0/3394990850 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.378+0000 7f1a6f59e640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a7011a960 0x7f1a701aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.378+0000 7f1a6e59c640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a7010fd30 0x7f1a701a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.379+0000 7f1a6ed9d640 1 -- 172.21.15.18:0/3394990850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a7011a960 msgr2=0x7f1a701aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.379+0000 7f1a6ed9d640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a7011a960 0x7f1a701aaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.379+0000 7f1a6ed9d640 1 -- 172.21.15.18:0/3394990850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a7010fd30 msgr2=0x7f1a701a6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.379+0000 7f1a6ed9d640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a7010fd30 0x7f1a701a6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.379+0000 7f1a6ed9d640 1 -- 172.21.15.18:0/3394990850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a60009560 con 0x7f1a7010f8c0 2024-02-17T13:04:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.381+0000 7f1a6ed9d640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7010f8c0 0x7f1a701a66d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800ba50 tx=0x7f1a6800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.382+0000 7f1a57fff640 1 -- 172.21.15.18:0/3394990850 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6800cbe0 con 0x7f1a7010f8c0 2024-02-17T13:04:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.382+0000 7f1a751eb640 1 -- 172.21.15.18:0/3394990850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a701abba0 con 0x7f1a7010f8c0 2024-02-17T13:04:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.382+0000 7f1a751eb640 1 -- 172.21.15.18:0/3394990850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a701b3420 con 0x7f1a7010f8c0 2024-02-17T13:04:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.382+0000 7f1a57fff640 1 -- 172.21.15.18:0/3394990850 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a6800cd40 con 0x7f1a7010f8c0 2024-02-17T13:04:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.382+0000 7f1a57fff640 1 -- 172.21.15.18:0/3394990850 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6801d7b0 con 0x7f1a7010f8c0 2024-02-17T13:04:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.385+0000 7f1a57fff640 1 -- 172.21.15.18:0/3394990850 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1a68014440 con 0x7f1a7010f8c0 2024-02-17T13:04:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.386+0000 7f1a77db7740 1 Processor -- start 2024-02-17T13:04:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.386+0000 7f1a77db7740 1 -- start start 2024-02-17T13:04:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.386+0000 7f1a77db7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.386+0000 7f1a77db7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 0x55fdf33c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.386+0000 7f1a77db7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33cf2e0 0x55fdf33d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.386+0000 7f1a77db7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf33349f0 con 0x55fdf33c6ad0 2024-02-17T13:04:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.386+0000 7f1a77db7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf3331f20 con 0x55fdf33c5d60 2024-02-17T13:04:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.386+0000 7f1a77db7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf332d700 con 0x55fdf33cf2e0 2024-02-17T13:04:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a6f59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a6f59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37708/0 (socket says 172.21.15.18:37708) 2024-02-17T13:04:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a6f59e640 1 -- 172.21.15.18:0/4141701714 learned_addr learned my addr 172.21.15.18:0/4141701714 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a6e59c640 1 --2- 172.21.15.18:0/4141701714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33cf2e0 0x55fdf33d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a6ed9d640 1 --2- 172.21.15.18:0/4141701714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 0x55fdf33c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a57fff640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1a580a2350 0x7f1a580a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a6f59e640 1 -- 172.21.15.18:0/4141701714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33cf2e0 msgr2=0x55fdf33d16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a6f59e640 1 --2- 172.21.15.18:0/4141701714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33cf2e0 0x55fdf33d16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a6f59e640 1 -- 172.21.15.18:0/4141701714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 msgr2=0x55fdf33c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a6f59e640 1 --2- 172.21.15.18:0/4141701714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 0x55fdf33c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a6f59e640 1 -- 172.21.15.18:0/4141701714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdf32c37b0 con 0x55fdf33c6ad0 2024-02-17T13:04:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a6e59c640 1 --2- 172.21.15.18:0/4141701714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33cf2e0 0x55fdf33d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:04:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.387+0000 7f1a6ed9d640 1 --2- 172.21.15.18:0/4141701714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 0x55fdf33c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:04:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.388+0000 7f1a57fff640 1 -- 172.21.15.18:0/3394990850 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f1a680cd9a0 con 0x7f1a7010f8c0 2024-02-17T13:04:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.388+0000 7f1a6f59e640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1a580a2350 0x7f1a580a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.388+0000 7f1a6f59e640 1 --2- 172.21.15.18:0/4141701714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33c5820 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f1a60036040 tx=0x7f1a600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a0ebf6a5450731 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.388+0000 7f1a557fa640 1 -- 172.21.15.18:0/4141701714 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60004040 con 0x55fdf33c6ad0 2024-02-17T13:04:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.389+0000 7f1a557fa640 1 -- 172.21.15.18:0/4141701714 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a60004df0 con 0x55fdf33c6ad0 2024-02-17T13:04:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.389+0000 7f1a6f59e640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1a580a2350 0x7f1a580a4810 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1a60047000 tx=0x7f1a6004c000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.389+0000 7f1a557fa640 1 -- 172.21.15.18:0/4141701714 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60003850 con 0x55fdf33c6ad0 2024-02-17T13:04:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.390+0000 7f1a77db7740 1 -- 172.21.15.18:0/4141701714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 msgr2=0x55fdf33c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.390+0000 7f1a77db7740 1 --2- 172.21.15.18:0/4141701714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33c5820 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f1a60036040 tx=0x7f1a600046c0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.390+0000 7f1a77db7740 1 -- 172.21.15.18:0/4141701714 shutdown_connections 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.390+0000 7f1a77db7740 1 --2- 172.21.15.18:0/4141701714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33cf2e0 0x55fdf33d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.390+0000 7f1a77db7740 1 --2- 172.21.15.18:0/4141701714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 0x55fdf33c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.390+0000 7f1a77db7740 1 --2- 172.21.15.18:0/4141701714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33c5820 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.390+0000 7f1a77db7740 1 -- 172.21.15.18:0/4141701714 >> 172.21.15.18:0/4141701714 conn(0x55fdf338d550 msgr2=0x55fdf33c0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.390+0000 7f1a77db7740 1 -- 172.21.15.18:0/4141701714 shutdown_connections 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.390+0000 7f1a77db7740 1 -- 172.21.15.18:0/4141701714 wait complete. 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.390+0000 7f1a77db7740 1 -- start start 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.390+0000 7f1a77db7740 1 Processor -- start 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a77db7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33c5d60 0x55fdf33cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a77db7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a77db7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33cf2e0 0x55fdf33ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a77db7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf32d2bf0 con 0x55fdf33c6ad0 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a77db7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf3001250 con 0x55fdf33cf2e0 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a77db7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf32b3200 con 0x55fdf33c5d60 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a77db7740 10 client.0 _ll_register_callbacks cb 0x55fdf3321d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a6f59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33cf2e0 0x55fdf33ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a6e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a6f59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33cf2e0 0x55fdf33ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:50648/0 (socket says 172.21.15.18:50648) 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a6f59e640 1 -- 172.21.15.18:0/3765514018 learned_addr learned my addr 172.21.15.18:0/3765514018 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a6e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37724/0 (socket says 172.21.15.18:37724) 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a6e59c640 1 -- 172.21.15.18:0/3765514018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33c5d60 msgr2=0x55fdf33cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a6e59c640 1 --2- 172.21.15.18:0/3765514018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33c5d60 0x55fdf33cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a6e59c640 1 -- 172.21.15.18:0/3765514018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33cf2e0 msgr2=0x55fdf33ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a6e59c640 1 --2- 172.21.15.18:0/3765514018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33cf2e0 0x55fdf33ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a6e59c640 1 -- 172.21.15.18:0/3765514018 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7011d640 con 0x55fdf33c6ad0 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a6f59e640 1 --2- 172.21.15.18:0/3765514018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33cf2e0 0x55fdf33ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.391+0000 7f1a6e59c640 1 --2- 172.21.15.18:0/3765514018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33cc840 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x55fdf33c6800 tx=0x7f1a6400f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787f7bf09147a8dd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.392+0000 7f1a52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.392+0000 7f1a52ff5640 1 -- 172.21.15.18:0/3765514018 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64013850 con 0x55fdf33c6ad0 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.392+0000 7f1a52ff5640 1 -- 172.21.15.18:0/3765514018 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f1a640139b0 con 0x55fdf33c6ad0 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.392+0000 7f1a77db7740 1 -- 172.21.15.18:0/3765514018 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fdf302d600 con 0x55fdf33c6ad0 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.392+0000 7f1a52ff5640 1 -- 172.21.15.18:0/3765514018 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a64013b70 con 0x55fdf33c6ad0 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.392+0000 7f1a77db7740 10 client.20613 fetch_fsmap learned FSMap version 65 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.392+0000 7f1a77db7740 1 -- 172.21.15.18:0/3765514018 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x55fdf332bd80 con 0x55fdf33c6ad0 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.392+0000 7f1a52ff5640 1 -- 172.21.15.18:0/3765514018 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fdf302d600 con 0x55fdf33c6ad0 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.393+0000 7f1a52ff5640 1 -- 172.21.15.18:0/3765514018 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 65) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f1a64019020 con 0x55fdf33c6ad0 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.393+0000 7f1a77db7740 10 client.20613 fetch_fsmap finished waiting for FSMap version 65 2024-02-17T13:04:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.393+0000 7f1a77db7740 10 client.20613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.393+0000 7f1a77db7740 20 client.20613 populate_metadata read hostname 'smithi018' 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.393+0000 7f1a77db7740 1 --2- 172.21.15.18:0/3765514018 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fdf33df810 0x55fdf33e1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.393+0000 7f1a77db7740 4 client.20613 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.393+0000 7f1a77db7740 1 -- 172.21.15.18:0/3765514018 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdf307e1b0 con 0x55fdf33df810 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.393+0000 7f1a6ed9d640 1 --2- 172.21.15.18:0/3765514018 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fdf33df810 0x55fdf33e1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.393+0000 7f1a52ff5640 10 client.20613 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.393+0000 7f1a6ed9d640 1 --2- 172.21.15.18:0/3765514018 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fdf33df810 0x55fdf33e1c30 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831f3723603f2b28 server_cookie=9dcd560d306635d0 in_seq=0 out_seq=0 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.394+0000 7f1a52ff5640 1 -- 172.21.15.18:0/3765514018 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fdf307e1b0 con 0x55fdf33df810 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.394+0000 7f1a52ff5640 10 client.20613 handle_command_reply: tid=0 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.394+0000 7f1a77db7740 1 client.20613 shutdown 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.395+0000 7f1a77db7740 1 -- 172.21.15.18:0/3765514018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 msgr2=0x55fdf33cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.395+0000 7f1a77db7740 1 --2- 172.21.15.18:0/3765514018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33cc840 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x55fdf33c6800 tx=0x7f1a6400f950 comp rx=0 tx=0).stop 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.395+0000 7f1a77db7740 1 -- 172.21.15.18:0/3765514018 shutdown_connections 2024-02-17T13:04:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.395+0000 7f1a77db7740 1 --2- 172.21.15.18:0/3765514018 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fdf33df810 0x55fdf33e1c30 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.395+0000 7f1a77db7740 1 --2- 172.21.15.18:0/3765514018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33cf2e0 0x55fdf33ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.395+0000 7f1a77db7740 1 --2- 172.21.15.18:0/3765514018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33cc840 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.395+0000 7f1a52ff5640 0 client.20613 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.395+0000 7f1a77db7740 1 --2- 172.21.15.18:0/3765514018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33c5d60 0x55fdf33cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.395+0000 7f1a77db7740 1 -- 172.21.15.18:0/3765514018 >> 172.21.15.18:0/3765514018 conn(0x55fdf338d550 msgr2=0x55fdf33c27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.395+0000 7f1a77db7740 1 -- 172.21.15.18:0/3765514018 shutdown_connections 2024-02-17T13:04:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.395+0000 7f1a77db7740 1 -- 172.21.15.18:0/3765514018 wait complete. 2024-02-17T13:04:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.395+0000 7f1a77db7740 20 client.20613 trim_cache size 0 max 16384 2024-02-17T13:04:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.415+0000 7f1a77db7740 1 Processor -- start 2024-02-17T13:04:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.415+0000 7f1a77db7740 1 -- start start 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.415+0000 7f1a77db7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 0x55fdf33cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.415+0000 7f1a77db7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.415+0000 7f1a77db7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33cf2e0 0x55fdf33c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a77db7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf33d1ef0 con 0x55fdf33c6ad0 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a77db7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf33ddef0 con 0x55fdf33c5d60 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a77db7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf33ca850 con 0x55fdf33cf2e0 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a6ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 0x55fdf33cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a6ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 0x55fdf33cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:50650/0 (socket says 172.21.15.18:50650) 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a6ed9d640 1 -- 172.21.15.18:0/3819048113 learned_addr learned my addr 172.21.15.18:0/3819048113 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a6f59e640 1 --2- 172.21.15.18:0/3819048113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33cf2e0 0x55fdf33c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a6e59c640 1 --2- 172.21.15.18:0/3819048113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a6ed9d640 1 -- 172.21.15.18:0/3819048113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33cf2e0 msgr2=0x55fdf33c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a6ed9d640 1 --2- 172.21.15.18:0/3819048113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33cf2e0 0x55fdf33c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a6ed9d640 1 -- 172.21.15.18:0/3819048113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 msgr2=0x55fdf33ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a6ed9d640 1 --2- 172.21.15.18:0/3819048113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a6ed9d640 1 -- 172.21.15.18:0/3819048113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdf32c87b0 con 0x55fdf33c5d60 2024-02-17T13:04:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a6e59c640 1 --2- 172.21.15.18:0/3819048113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:04:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.416+0000 7f1a6ed9d640 1 --2- 172.21.15.18:0/3819048113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 0x55fdf33cea30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1a68014c60 tx=0x7f1a68004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f93a3adada7e56 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.417+0000 7f1a54ff9640 1 -- 172.21.15.18:0/3819048113 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68099dc0 con 0x55fdf33c5d60 2024-02-17T13:04:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.417+0000 7f1a54ff9640 1 -- 172.21.15.18:0/3819048113 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a6809ab60 con 0x55fdf33c5d60 2024-02-17T13:04:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.417+0000 7f1a54ff9640 1 -- 172.21.15.18:0/3819048113 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68096ad0 con 0x55fdf33c5d60 2024-02-17T13:04:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.418+0000 7f1a77db7740 1 -- 172.21.15.18:0/3819048113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 msgr2=0x55fdf33cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.418+0000 7f1a77db7740 1 --2- 172.21.15.18:0/3819048113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 0x55fdf33cea30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1a68014c60 tx=0x7f1a68004a80 comp rx=0 tx=0).stop 2024-02-17T13:04:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.418+0000 7f1a77db7740 1 -- 172.21.15.18:0/3819048113 shutdown_connections 2024-02-17T13:04:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.418+0000 7f1a77db7740 1 --2- 172.21.15.18:0/3819048113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33cf2e0 0x55fdf33c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.418+0000 7f1a77db7740 1 --2- 172.21.15.18:0/3819048113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33c6ad0 0x55fdf33ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.418+0000 7f1a77db7740 1 --2- 172.21.15.18:0/3819048113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c5d60 0x55fdf33cea30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.418+0000 7f1a77db7740 1 -- 172.21.15.18:0/3819048113 >> 172.21.15.18:0/3819048113 conn(0x55fdf338d550 msgr2=0x55fdf33cabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.419+0000 7f1a77db7740 1 -- 172.21.15.18:0/3819048113 shutdown_connections 2024-02-17T13:04:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.419+0000 7f1a77db7740 1 -- 172.21.15.18:0/3819048113 wait complete. 2024-02-17T13:04:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.420+0000 7f1a77db7740 1 -- start start 2024-02-17T13:04:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.420+0000 7f1a77db7740 1 Processor -- start 2024-02-17T13:04:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.420+0000 7f1a77db7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33c5d60 0x55fdf33d4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.420+0000 7f1a77db7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c6ad0 0x55fdf33d54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.420+0000 7f1a77db7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33cf2e0 0x55fdf3435d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a77db7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf33d5df0 con 0x55fdf33cf2e0 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a77db7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf3436280 con 0x55fdf33c6ad0 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a6ed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33c5d60 0x55fdf33d4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a77db7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf34363c0 con 0x55fdf33c5d60 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a6ed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33c5d60 0x55fdf33d4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:43614/0 (socket says 172.21.15.18:43614) 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a6ed9d640 1 -- 172.21.15.18:0/1315250871 learned_addr learned my addr 172.21.15.18:0/1315250871 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a6e59c640 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c6ad0 0x55fdf33d54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a6f59e640 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33cf2e0 0x55fdf3435d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a77db7740 10 client.0 _ll_register_callbacks cb 0x7f1a60009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a6e59c640 1 -- 172.21.15.18:0/1315250871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33c5d60 msgr2=0x55fdf33d4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a6e59c640 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33c5d60 0x55fdf33d4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a6e59c640 1 -- 172.21.15.18:0/1315250871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33cf2e0 msgr2=0x55fdf3435d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a6e59c640 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33cf2e0 0x55fdf3435d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a6e59c640 1 -- 172.21.15.18:0/1315250871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a701b3420 con 0x55fdf33c6ad0 2024-02-17T13:04:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a6e59c640 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c6ad0 0x55fdf33d54f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1a64000c00 tx=0x7f1a64005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=493a1ae1e68ebd0a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.421+0000 7f1a52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:04:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.422+0000 7f1a52ff5640 1 -- 172.21.15.18:0/1315250871 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400a310 con 0x55fdf33c6ad0 2024-02-17T13:04:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.422+0000 7f1a77db7740 1 -- 172.21.15.18:0/1315250871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a60004df0 con 0x55fdf33c6ad0 2024-02-17T13:04:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.422+0000 7f1a52ff5640 1 -- 172.21.15.18:0/1315250871 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f1a64005e10 con 0x55fdf33c6ad0 2024-02-17T13:04:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.422+0000 7f1a52ff5640 1 -- 172.21.15.18:0/1315250871 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a64004040 con 0x55fdf33c6ad0 2024-02-17T13:04:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.422+0000 7f1a77db7740 10 client.29450 fetch_fsmap learned FSMap version 65 2024-02-17T13:04:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.422+0000 7f1a77db7740 1 -- 172.21.15.18:0/1315250871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x55fdf32ee540 con 0x55fdf33c6ad0 2024-02-17T13:04:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.424+0000 7f1a52ff5640 1 -- 172.21.15.18:0/1315250871 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a6400c040 con 0x55fdf33c6ad0 2024-02-17T13:04:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.424+0000 7f1a52ff5640 1 -- 172.21.15.18:0/1315250871 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 65) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f1a64019040 con 0x55fdf33c6ad0 2024-02-17T13:04:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.424+0000 7f1a77db7740 10 client.29450 fetch_fsmap finished waiting for FSMap version 65 2024-02-17T13:04:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.424+0000 7f1a77db7740 10 client.29450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:04:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.424+0000 7f1a77db7740 20 client.29450 populate_metadata read hostname 'smithi018' 2024-02-17T13:04:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.424+0000 7f1a77db7740 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fdf3437b50 0x55fdf3439f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.424+0000 7f1a77db7740 4 client.29450 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:04:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.424+0000 7f1a77db7740 1 -- 172.21.15.18:0/1315250871 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f1a60004df0 con 0x55fdf3437b50 2024-02-17T13:04:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.425+0000 7f1a6ed9d640 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fdf3437b50 0x55fdf3439f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.426+0000 7f1a6ed9d640 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fdf3437b50 0x55fdf3439f70 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=154aa0a9d7c9c6a2 server_cookie=d644d19827290bdc in_seq=0 out_seq=0 2024-02-17T13:04:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.426+0000 7f1a52ff5640 10 client.29450 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a52ff5640 1 -- 172.21.15.18:0/1315250871 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f1a60004df0 con 0x55fdf3437b50 2024-02-17T13:04:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a52ff5640 10 client.29450 handle_command_reply: tid=0 2024-02-17T13:04:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a77db7740 1 client.29450 shutdown 2024-02-17T13:04:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a77db7740 1 -- 172.21.15.18:0/1315250871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c6ad0 msgr2=0x55fdf33d54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a77db7740 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c6ad0 0x55fdf33d54f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1a64000c00 tx=0x7f1a64005c20 comp rx=0 tx=0).stop 2024-02-17T13:04:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a77db7740 1 -- 172.21.15.18:0/1315250871 shutdown_connections 2024-02-17T13:04:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a77db7740 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55fdf3437b50 0x55fdf3439f70 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a77db7740 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdf33cf2e0 0x55fdf3435d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a77db7740 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fdf33c6ad0 0x55fdf33d54f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a52ff5640 0 client.29450 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a77db7740 1 --2- 172.21.15.18:0/1315250871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fdf33c5d60 0x55fdf33d4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a77db7740 1 -- 172.21.15.18:0/1315250871 >> 172.21.15.18:0/1315250871 conn(0x55fdf338d550 msgr2=0x55fdf33bf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a77db7740 1 -- 172.21.15.18:0/1315250871 shutdown_connections 2024-02-17T13:04:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a77db7740 1 -- 172.21.15.18:0/1315250871 wait complete. 2024-02-17T13:04:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.711+0000 7f1a77db7740 20 client.29450 trim_cache size 0 max 16384 2024-02-17T13:04:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:04:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:04:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:04:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:04:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.712+0000 7f1a751eb640 1 -- 172.21.15.18:0/3394990850 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1a580a2350 msgr2=0x7f1a580a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.712+0000 7f1a751eb640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1a580a2350 0x7f1a580a4810 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1a60047000 tx=0x7f1a6004c000 comp rx=0 tx=0).stop 2024-02-17T13:04:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.712+0000 7f1a751eb640 1 -- 172.21.15.18:0/3394990850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7010f8c0 msgr2=0x7f1a701a66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.712+0000 7f1a751eb640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7010f8c0 0x7f1a701a66d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800ba50 tx=0x7f1a6800bf10 comp rx=0 tx=0).stop 2024-02-17T13:04:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.712+0000 7f1a751eb640 1 -- 172.21.15.18:0/3394990850 shutdown_connections 2024-02-17T13:04:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.712+0000 7f1a751eb640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1a580a2350 0x7f1a580a4810 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.712+0000 7f1a751eb640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a7011a960 0x7f1a701aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.712+0000 7f1a751eb640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a7010fd30 0x7f1a701a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.712+0000 7f1a751eb640 1 --2- 172.21.15.18:0/3394990850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7010f8c0 0x7f1a701a66d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.712+0000 7f1a751eb640 1 -- 172.21.15.18:0/3394990850 >> 172.21.15.18:0/3394990850 conn(0x7f1a7006c220 msgr2=0x7f1a70119430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.712+0000 7f1a751eb640 1 -- 172.21.15.18:0/3394990850 shutdown_connections 2024-02-17T13:04:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:10.712+0000 7f1a751eb640 1 -- 172.21.15.18:0/3394990850 wait complete. 2024-02-17T13:04:10.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:10 smithi018 ceph-mon[32625]: pgmap v449: 97 pgs: 97 active+clean; 637 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.6 MiB/s wr, 1.03k op/s 2024-02-17T13:04:10.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 48576 34.03 MB/sec warmup 82 sec latency 2219.767 ms 2024-02-17T13:04:10.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:10 smithi119 ceph-mon[36188]: pgmap v449: 97 pgs: 97 active+clean; 637 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.6 MiB/s wr, 1.03k op/s 2024-02-17T13:04:10.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:10 smithi146 ceph-mon[36454]: pgmap v449: 97 pgs: 97 active+clean; 637 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.6 MiB/s wr, 1.03k op/s 2024-02-17T13:04:11.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 49472 34.27 MB/sec warmup 83 sec latency 3219.804 ms 2024-02-17T13:04:12.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:12 smithi018 ceph-mon[32625]: pgmap v450: 97 pgs: 97 active+clean; 646 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-02-17T13:04:12.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 50089 34.26 MB/sec warmup 84 sec latency 4219.836 ms 2024-02-17T13:04:12.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:12 smithi119 ceph-mon[36188]: pgmap v450: 97 pgs: 97 active+clean; 646 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-02-17T13:04:12.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:12 smithi146 ceph-mon[36454]: pgmap v450: 97 pgs: 97 active+clean; 646 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-02-17T13:04:13.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:13 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:04:13.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 50712 34.19 MB/sec warmup 85 sec latency 4434.943 ms 2024-02-17T13:04:13.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:13 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:04:13.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:13 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:04:14.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:14 smithi018 ceph-mon[32625]: pgmap v451: 97 pgs: 97 active+clean; 646 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-02-17T13:04:14.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:14 smithi018 ceph-mon[32625]: osdmap e108: 12 total, 12 up, 12 in 2024-02-17T13:04:14.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:14 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:04:14.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:14 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:04:14.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:14 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:04:14.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 51224 34.10 MB/sec warmup 86 sec latency 5027.943 ms 2024-02-17T13:04:14.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:14 smithi146 ceph-mon[36454]: pgmap v451: 97 pgs: 97 active+clean; 646 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-02-17T13:04:14.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:14 smithi146 ceph-mon[36454]: osdmap e108: 12 total, 12 up, 12 in 2024-02-17T13:04:14.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:14 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:04:14.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:14 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:04:14.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:14 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:04:15.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:14 smithi119 ceph-mon[36188]: pgmap v451: 97 pgs: 97 active+clean; 646 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-02-17T13:04:15.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:14 smithi119 ceph-mon[36188]: osdmap e108: 12 total, 12 up, 12 in 2024-02-17T13:04:15.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:14 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:04:15.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:14 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:04:15.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:14 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:04:15.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 51951 34.10 MB/sec warmup 87 sec latency 6027.970 ms 2024-02-17T13:04:16.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:16 smithi018 ceph-mon[32625]: pgmap v453: 97 pgs: 97 active+clean; 666 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 988 op/s 2024-02-17T13:04:16.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 52865 34.42 MB/sec warmup 88 sec latency 7027.999 ms 2024-02-17T13:04:16.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:16 smithi146 ceph-mon[36454]: pgmap v453: 97 pgs: 97 active+clean; 666 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 988 op/s 2024-02-17T13:04:17.085 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:16 smithi119 ceph-mon[36188]: pgmap v453: 97 pgs: 97 active+clean; 666 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 988 op/s 2024-02-17T13:04:17.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 53429 34.40 MB/sec warmup 89 sec latency 8028.081 ms 2024-02-17T13:04:18.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:18 smithi018 ceph-mon[32625]: pgmap v454: 97 pgs: 97 active+clean; 666 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 926 op/s 2024-02-17T13:04:18.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 54056 34.32 MB/sec warmup 90 sec latency 8150.047 ms 2024-02-17T13:04:18.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:18 smithi119 ceph-mon[36188]: pgmap v454: 97 pgs: 97 active+clean; 666 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 926 op/s 2024-02-17T13:04:18.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:18 smithi146 ceph-mon[36454]: pgmap v454: 97 pgs: 97 active+clean; 666 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 926 op/s 2024-02-17T13:04:19.788 INFO:tasks.workunit.client.0.smithi018.stdout: 10 54693 34.20 MB/sec warmup 91 sec latency 233.565 ms 2024-02-17T13:04:20.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 55703 34.48 MB/sec warmup 92 sec latency 59.294 ms 2024-02-17T13:04:20.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:20 smithi119 ceph-mon[36188]: pgmap v455: 97 pgs: 97 active+clean; 668 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 939 op/s 2024-02-17T13:04:20.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:20 smithi146 ceph-mon[36454]: pgmap v455: 97 pgs: 97 active+clean; 668 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 939 op/s 2024-02-17T13:04:21.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:20 smithi018 ceph-mon[32625]: pgmap v455: 97 pgs: 97 active+clean; 668 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 939 op/s 2024-02-17T13:04:21.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 56511 34.64 MB/sec warmup 93 sec latency 50.889 ms 2024-02-17T13:04:22.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 57497 34.91 MB/sec warmup 94 sec latency 42.796 ms 2024-02-17T13:04:22.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:22 smithi119 ceph-mon[36188]: pgmap v456: 97 pgs: 97 active+clean; 696 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 971 op/s 2024-02-17T13:04:22.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:22 smithi146 ceph-mon[36454]: pgmap v456: 97 pgs: 97 active+clean; 696 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 971 op/s 2024-02-17T13:04:23.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:22 smithi018 ceph-mon[32625]: pgmap v456: 97 pgs: 97 active+clean; 696 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 971 op/s 2024-02-17T13:04:23.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 58191 34.84 MB/sec warmup 95 sec latency 155.071 ms 2024-02-17T13:04:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.817+0000 7fd558548640 1 -- 172.21.15.18:0/3099955080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd55010d320 msgr2=0x7fd550114ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.817+0000 7fd558548640 1 --2- 172.21.15.18:0/3099955080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd55010d320 0x7fd550114ed0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd54800ade0 tx=0x7fd54802cfd0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.817+0000 7fd558548640 1 -- 172.21.15.18:0/3099955080 shutdown_connections 2024-02-17T13:04:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.817+0000 7fd558548640 1 --2- 172.21.15.18:0/3099955080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd55010d320 0x7fd550114ed0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.817+0000 7fd558548640 1 --2- 172.21.15.18:0/3099955080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd550109630 0x7fd55010cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.817+0000 7fd558548640 1 --2- 172.21.15.18:0/3099955080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd550108c80 0x7fd550109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.817+0000 7fd558548640 1 -- 172.21.15.18:0/3099955080 >> 172.21.15.18:0/3099955080 conn(0x7fd55006c6b0 msgr2=0x7fd55006cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.818+0000 7fd558548640 1 -- 172.21.15.18:0/3099955080 shutdown_connections 2024-02-17T13:04:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.818+0000 7fd558548640 1 -- 172.21.15.18:0/3099955080 wait complete. 2024-02-17T13:04:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.818+0000 7fd558548640 1 Processor -- start 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.818+0000 7fd558548640 1 -- start start 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.818+0000 7fd558548640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd550108c80 0x7fd55019ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd558548640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd55010d320 0x7fd5501a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd558548640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5501a4810 0x7fd5501a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd558548640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5501a5200 con 0x7fd5501a4810 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd558548640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5501a5370 con 0x7fd550108c80 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd558548640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5501a54e0 con 0x7fd55010d320 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd555ae3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd55010d320 0x7fd5501a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd555ae3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd55010d320 0x7fd5501a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:59532/0 (socket says 172.21.15.18:59532) 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd555ae3640 1 -- 172.21.15.18:0/1983941719 learned_addr learned my addr 172.21.15.18:0/1983941719 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd556ae5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5501a4810 0x7fd5501a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd5562e4640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd550108c80 0x7fd55019ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd556ae5640 1 -- 172.21.15.18:0/1983941719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd55010d320 msgr2=0x7fd5501a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd556ae5640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd55010d320 0x7fd5501a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd556ae5640 1 -- 172.21.15.18:0/1983941719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd550108c80 msgr2=0x7fd55019ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd556ae5640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd550108c80 0x7fd55019ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd556ae5640 1 -- 172.21.15.18:0/1983941719 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54800aa10 con 0x7fd5501a4810 2024-02-17T13:04:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd555ae3640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd55010d320 0x7fd5501a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:04:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.819+0000 7fd556ae5640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5501a4810 0x7fd5501a4cc0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fd548038040 tx=0x7fd548035670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.820+0000 7fd5477fe640 1 -- 172.21.15.18:0/1983941719 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548033590 con 0x7fd5501a4810 2024-02-17T13:04:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.820+0000 7fd558548640 1 -- 172.21.15.18:0/1983941719 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5501acac0 con 0x7fd5501a4810 2024-02-17T13:04:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.820+0000 7fd558548640 1 -- 172.21.15.18:0/1983941719 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5501acee0 con 0x7fd5501a4810 2024-02-17T13:04:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.820+0000 7fd5477fe640 1 -- 172.21.15.18:0/1983941719 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd548003e50 con 0x7fd5501a4810 2024-02-17T13:04:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.820+0000 7fd5477fe640 1 -- 172.21.15.18:0/1983941719 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548004280 con 0x7fd5501a4810 2024-02-17T13:04:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.823+0000 7fd55b1fa740 1 Processor -- start 2024-02-17T13:04:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.824+0000 7fd5477fe640 1 -- 172.21.15.18:0/1983941719 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd54803c040 con 0x7fd5501a4810 2024-02-17T13:04:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.824+0000 7fd55b1fa740 1 -- start start 2024-02-17T13:04:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.824+0000 7fd55b1fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3a2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.824+0000 7fd55b1fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 0x55ddac3a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.824+0000 7fd55b1fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3b06f0 0x55ddac3b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.824+0000 7fd55b1fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddac3119f0 con 0x55ddac3b06f0 2024-02-17T13:04:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.824+0000 7fd55b1fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddac30ef20 con 0x55ddac3a2d60 2024-02-17T13:04:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.824+0000 7fd55b1fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddac30a700 con 0x55ddac3a3ad0 2024-02-17T13:04:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.824+0000 7fd555ae3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3a2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd555ae3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3a2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:59542/0 (socket says 172.21.15.18:59542) 2024-02-17T13:04:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd555ae3640 1 -- 172.21.15.18:0/4020996402 learned_addr learned my addr 172.21.15.18:0/4020996402 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd555ae3640 1 --2- 172.21.15.18:0/4020996402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3b06f0 0x55ddac3b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd5562e4640 1 --2- 172.21.15.18:0/4020996402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 0x55ddac3a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd5477fe640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd53c0a2350 0x7fd53c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd555ae3640 1 -- 172.21.15.18:0/4020996402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 msgr2=0x55ddac3a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd555ae3640 1 --2- 172.21.15.18:0/4020996402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 0x55ddac3a31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd555ae3640 1 -- 172.21.15.18:0/4020996402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3b06f0 msgr2=0x55ddac3b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd555ae3640 1 --2- 172.21.15.18:0/4020996402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3b06f0 0x55ddac3b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd555ae3640 1 -- 172.21.15.18:0/4020996402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddac2a07b0 con 0x55ddac3a3ad0 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd5477fe640 1 -- 172.21.15.18:0/1983941719 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd5480f0280 con 0x7fd5501a4810 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd556ae5640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd53c0a2350 0x7fd53c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.825+0000 7fd555ae3640 1 --2- 172.21.15.18:0/4020996402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3a2820 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd540012b50 tx=0x7fd54000bb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=332e3488f03e0266 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.826+0000 7fd544bfe640 1 -- 172.21.15.18:0/4020996402 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd540012c40 con 0x55ddac3a3ad0 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.826+0000 7fd544bfe640 1 -- 172.21.15.18:0/4020996402 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd540004510 con 0x55ddac3a3ad0 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.826+0000 7fd544bfe640 1 -- 172.21.15.18:0/4020996402 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5400039f0 con 0x55ddac3a3ad0 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.826+0000 7fd55b1fa740 1 -- 172.21.15.18:0/4020996402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 msgr2=0x55ddac3a2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.826+0000 7fd556ae5640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd53c0a2350 0x7fd53c0a4810 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd5480bcaa0 tx=0x7fd54802cfd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.826+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/4020996402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3a2820 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd540012b50 tx=0x7fd54000bb80 comp rx=0 tx=0).stop 2024-02-17T13:04:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.826+0000 7fd55b1fa740 1 -- 172.21.15.18:0/4020996402 shutdown_connections 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.826+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/4020996402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3b06f0 0x55ddac3b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.826+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/4020996402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 0x55ddac3a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.826+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/4020996402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3a2820 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.826+0000 7fd55b1fa740 1 -- 172.21.15.18:0/4020996402 >> 172.21.15.18:0/4020996402 conn(0x55ddac36a550 msgr2=0x55ddac39da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.827+0000 7fd55b1fa740 1 -- 172.21.15.18:0/4020996402 shutdown_connections 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.827+0000 7fd55b1fa740 1 -- 172.21.15.18:0/4020996402 wait complete. 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.827+0000 7fd55b1fa740 1 -- start start 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.828+0000 7fd55b1fa740 1 Processor -- start 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.828+0000 7fd55b1fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a2d60 0x55ddac3ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.828+0000 7fd55b1fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3adc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.828+0000 7fd55b1fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3b06f0 0x55ddac3ae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.828+0000 7fd55b1fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddac2afbf0 con 0x55ddac3a2d60 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.828+0000 7fd55b1fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddabfde250 con 0x55ddac3b06f0 2024-02-17T13:04:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.828+0000 7fd555ae3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a2d60 0x55ddac3ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.828+0000 7fd5562e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.828+0000 7fd555ae3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a2d60 0x55ddac3ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48124/0 (socket says 172.21.15.18:48124) 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.828+0000 7fd555ae3640 1 -- 172.21.15.18:0/3460591267 learned_addr learned my addr 172.21.15.18:0/3460591267 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.828+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3460591267 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddac290200 con 0x55ddac3a3ad0 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.828+0000 7fd555ae3640 1 --2- 172.21.15.18:0/3460591267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3b06f0 0x55ddac3ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.829+0000 7fd555ae3640 1 -- 172.21.15.18:0/3460591267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 msgr2=0x55ddac3adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.829+0000 7fd555ae3640 1 --2- 172.21.15.18:0/3460591267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.829+0000 7fd555ae3640 1 -- 172.21.15.18:0/3460591267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3b06f0 msgr2=0x55ddac3ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.829+0000 7fd555ae3640 1 --2- 172.21.15.18:0/3460591267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3b06f0 0x55ddac3ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.829+0000 7fd555ae3640 1 -- 172.21.15.18:0/3460591267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54001a040 con 0x55ddac3a2d60 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.829+0000 7fd555ae3640 1 --2- 172.21.15.18:0/3460591267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a2d60 0x55ddac3ad700 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fd540012f90 tx=0x7fd540003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e19adee5785090 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.829+0000 7fd539ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.829+0000 7fd539ffb640 1 -- 172.21.15.18:0/3460591267 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54002b8e0 con 0x55ddac3a2d60 2024-02-17T13:04:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.829+0000 7fd539ffb640 1 -- 172.21.15.18:0/3460591267 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd54002ba40 con 0x55ddac3a2d60 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.829+0000 7fd55b1fa740 10 client.0 _ll_register_callbacks cb 0x55ddac2fed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.830+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3460591267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddac00a600 con 0x55ddac3a2d60 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.830+0000 7fd539ffb640 1 -- 172.21.15.18:0/3460591267 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd54001f420 con 0x55ddac3a2d60 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.830+0000 7fd55b1fa740 10 client.20631 fetch_fsmap learned FSMap version 65 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.830+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3460591267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({fsmap=65}) v3 -- 0x55ddac308d80 con 0x55ddac3a2d60 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.831+0000 7fd539ffb640 1 -- 172.21.15.18:0/3460591267 <== mon.0 v2:172.21.15.18:3300/0 4 ==== fsmap(e 65) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fd54001f580 con 0x55ddac3a2d60 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.831+0000 7fd55b1fa740 10 client.20631 fetch_fsmap finished waiting for FSMap version 65 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.831+0000 7fd55b1fa740 10 client.20631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.831+0000 7fd55b1fa740 20 client.20631 populate_metadata read hostname 'smithi018' 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.831+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3460591267 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ddac3a7340 0x55ddac3a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.831+0000 7fd55b1fa740 4 client.20631 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.831+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3460591267 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddac00a600 con 0x55ddac3a7340 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.831+0000 7fd5562e4640 1 --2- 172.21.15.18:0/3460591267 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ddac3a7340 0x55ddac3a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.831+0000 7fd5562e4640 1 --2- 172.21.15.18:0/3460591267 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ddac3a7340 0x55ddac3a77d0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de07ca8c2ae9e65a server_cookie=e817515885618d49 in_seq=0 out_seq=0 2024-02-17T13:04:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.831+0000 7fd539ffb640 10 client.20631 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.832+0000 7fd539ffb640 1 -- 172.21.15.18:0/3460591267 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ddac00a600 con 0x55ddac3a7340 2024-02-17T13:04:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.832+0000 7fd539ffb640 10 client.20631 handle_command_reply: tid=0 2024-02-17T13:04:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.832+0000 7fd55b1fa740 1 client.20631 shutdown 2024-02-17T13:04:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.832+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3460591267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a2d60 msgr2=0x55ddac3ad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.833+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3460591267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a2d60 0x55ddac3ad700 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fd540012f90 tx=0x7fd540003690 comp rx=0 tx=0).stop 2024-02-17T13:04:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.833+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3460591267 shutdown_connections 2024-02-17T13:04:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.833+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3460591267 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ddac3a7340 0x55ddac3a77d0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.833+0000 7fd539ffb640 0 client.20631 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.833+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3460591267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3b06f0 0x55ddac3ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.833+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3460591267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.833+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3460591267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a2d60 0x55ddac3ad700 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.833+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3460591267 >> 172.21.15.18:0/3460591267 conn(0x55ddac36a550 msgr2=0x55ddac39f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.833+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3460591267 shutdown_connections 2024-02-17T13:04:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.833+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3460591267 wait complete. 2024-02-17T13:04:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.833+0000 7fd55b1fa740 20 client.20631 trim_cache size 0 max 16384 2024-02-17T13:04:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.850+0000 7fd55b1fa740 1 Processor -- start 2024-02-17T13:04:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.850+0000 7fd55b1fa740 1 -- start start 2024-02-17T13:04:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.850+0000 7fd55b1fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 0x55ddac3afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.850+0000 7fd55b1fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.850+0000 7fd55b1fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a7340 0x55ddac3c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.850+0000 7fd55b1fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddac39d310 con 0x55ddac3a7340 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.850+0000 7fd55b1fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddac3bd5b0 con 0x55ddac3a2d60 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.850+0000 7fd55b1fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddac3a7b80 con 0x55ddac3a3ad0 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.850+0000 7fd555ae3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a7340 0x55ddac3c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.850+0000 7fd555ae3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a7340 0x55ddac3c3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48140/0 (socket says 172.21.15.18:48140) 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.851+0000 7fd555ae3640 1 -- 172.21.15.18:0/3765014125 learned_addr learned my addr 172.21.15.18:0/3765014125 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.851+0000 7fd5562e4640 1 --2- 172.21.15.18:0/3765014125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.851+0000 7fd555ae3640 1 --2- 172.21.15.18:0/3765014125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 0x55ddac3afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.851+0000 7fd5562e4640 1 -- 172.21.15.18:0/3765014125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 msgr2=0x55ddac3afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.851+0000 7fd5562e4640 1 --2- 172.21.15.18:0/3765014125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 0x55ddac3afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.851+0000 7fd5562e4640 1 -- 172.21.15.18:0/3765014125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a7340 msgr2=0x55ddac3c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.851+0000 7fd5562e4640 1 --2- 172.21.15.18:0/3765014125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a7340 0x55ddac3c3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.851+0000 7fd5562e4640 1 -- 172.21.15.18:0/3765014125 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddac2a57b0 con 0x55ddac3a3ad0 2024-02-17T13:04:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.851+0000 7fd5562e4640 1 --2- 172.21.15.18:0/3765014125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3c3800 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55ddac3a80b0 tx=0x7fd54c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dbc21399d6411fc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.852+0000 7fd53bfff640 1 -- 172.21.15.18:0/3765014125 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c0042f0 con 0x55ddac3a3ad0 2024-02-17T13:04:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.852+0000 7fd53bfff640 1 -- 172.21.15.18:0/3765014125 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd54c018040 con 0x55ddac3a3ad0 2024-02-17T13:04:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.852+0000 7fd53bfff640 1 -- 172.21.15.18:0/3765014125 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c004d60 con 0x55ddac3a3ad0 2024-02-17T13:04:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.853+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3765014125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 msgr2=0x55ddac3c3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.853+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3765014125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3c3800 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55ddac3a80b0 tx=0x7fd54c005f90 comp rx=0 tx=0).stop 2024-02-17T13:04:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.853+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3765014125 shutdown_connections 2024-02-17T13:04:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.853+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3765014125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a7340 0x55ddac3c3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.853+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3765014125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3c3800 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.853+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3765014125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 0x55ddac3afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.853+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3765014125 >> 172.21.15.18:0/3765014125 conn(0x55ddac36a550 msgr2=0x55ddac3abfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.853+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3765014125 shutdown_connections 2024-02-17T13:04:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.853+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3765014125 wait complete. 2024-02-17T13:04:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.853+0000 7fd55b1fa740 1 -- start start 2024-02-17T13:04:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.853+0000 7fd55b1fa740 1 Processor -- start 2024-02-17T13:04:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd55b1fa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 0x55ddac3c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd55b1fa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd55b1fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a7340 0x55ddac3c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd55b1fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddac3c06f0 con 0x55ddac3a7340 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd55b1fa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddac3c0860 con 0x55ddac3a2d60 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd55b1fa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddac3c0a20 con 0x55ddac3a3ad0 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd55b1fa740 10 client.0 _ll_register_callbacks cb 0x7fd54000be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd5562e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd5562e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3bfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:59568/0 (socket says 172.21.15.18:59568) 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd5562e4640 1 -- 172.21.15.18:0/3699384492 learned_addr learned my addr 172.21.15.18:0/3699384492 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd5562e4640 1 -- 172.21.15.18:0/3699384492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 msgr2=0x55ddac3c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd5562e4640 1 --2- 172.21.15.18:0/3699384492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 0x55ddac3c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd5562e4640 1 -- 172.21.15.18:0/3699384492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a7340 msgr2=0x55ddac3c01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd5562e4640 1 --2- 172.21.15.18:0/3699384492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a7340 0x55ddac3c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.854+0000 7fd5562e4640 1 -- 172.21.15.18:0/3699384492 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54c009ad0 con 0x55ddac3a3ad0 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.855+0000 7fd5562e4640 1 --2- 172.21.15.18:0/3699384492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3bfc70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd54c022040 tx=0x7fd54c0040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=303a5f424f9f457e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.855+0000 7fd539ffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.855+0000 7fd539ffb640 1 -- 172.21.15.18:0/3699384492 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c004360 con 0x55ddac3a3ad0 2024-02-17T13:04:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.855+0000 7fd539ffb640 1 -- 172.21.15.18:0/3699384492 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd54c004940 con 0x55ddac3a3ad0 2024-02-17T13:04:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.855+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3699384492 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd540004510 con 0x55ddac3a3ad0 2024-02-17T13:04:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.855+0000 7fd539ffb640 1 -- 172.21.15.18:0/3699384492 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd54c004b00 con 0x55ddac3a3ad0 2024-02-17T13:04:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.855+0000 7fd55b1fa740 10 client.29560 fetch_fsmap learned FSMap version 65 2024-02-17T13:04:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.855+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3699384492 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x55ddac2cb540 con 0x55ddac3a3ad0 2024-02-17T13:04:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.856+0000 7fd539ffb640 1 -- 172.21.15.18:0/3699384492 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd540004510 con 0x55ddac3a3ad0 2024-02-17T13:04:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.856+0000 7fd539ffb640 1 -- 172.21.15.18:0/3699384492 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 65) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fd54c0037d0 con 0x55ddac3a3ad0 2024-02-17T13:04:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.856+0000 7fd55b1fa740 10 client.29560 fetch_fsmap finished waiting for FSMap version 65 2024-02-17T13:04:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.856+0000 7fd55b1fa740 10 client.29560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:04:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.856+0000 7fd55b1fa740 20 client.29560 populate_metadata read hostname 'smithi018' 2024-02-17T13:04:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.856+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3699384492 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ddac3b06f0 0x55ddac416580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.856+0000 7fd55b1fa740 4 client.29560 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:04:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.856+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3699384492 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ddac05b1b0 con 0x55ddac3b06f0 2024-02-17T13:04:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.856+0000 7fd555ae3640 1 --2- 172.21.15.18:0/3699384492 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ddac3b06f0 0x55ddac416580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.857+0000 7fd539ffb640 10 client.29560 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:23.857+0000 7fd555ae3640 1 --2- 172.21.15.18:0/3699384492 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ddac3b06f0 0x55ddac416580 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f442fbd5623ed8e7 server_cookie=3fffa873531a90d5 in_seq=0 out_seq=0 2024-02-17T13:04:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd539ffb640 1 -- 172.21.15.18:0/3699384492 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ddac05b1b0 con 0x55ddac3b06f0 2024-02-17T13:04:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd539ffb640 10 client.29560 handle_command_reply: tid=0 2024-02-17T13:04:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd55b1fa740 1 client.29560 shutdown 2024-02-17T13:04:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3699384492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 msgr2=0x55ddac3bfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3699384492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3bfc70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd54c022040 tx=0x7fd54c0040f0 comp rx=0 tx=0).stop 2024-02-17T13:04:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3699384492 shutdown_connections 2024-02-17T13:04:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3699384492 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55ddac3b06f0 0x55ddac416580 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3699384492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddac3a7340 0x55ddac3c01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd539ffb640 0 client.29560 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3699384492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ddac3a3ad0 0x55ddac3bfc70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd55b1fa740 1 --2- 172.21.15.18:0/3699384492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddac3a2d60 0x55ddac3c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3699384492 >> 172.21.15.18:0/3699384492 conn(0x55ddac36a550 msgr2=0x55ddac39c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3699384492 shutdown_connections 2024-02-17T13:04:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd55b1fa740 1 -- 172.21.15.18:0/3699384492 wait complete. 2024-02-17T13:04:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.081+0000 7fd55b1fa740 20 client.29560 trim_cache size 0 max 16384 2024-02-17T13:04:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:04:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:04:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:04:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:04:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.082+0000 7fd558548640 1 -- 172.21.15.18:0/1983941719 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd53c0a2350 msgr2=0x7fd53c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.082+0000 7fd558548640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd53c0a2350 0x7fd53c0a4810 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd5480bcaa0 tx=0x7fd54802cfd0 comp rx=0 tx=0).stop 2024-02-17T13:04:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.082+0000 7fd558548640 1 -- 172.21.15.18:0/1983941719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5501a4810 msgr2=0x7fd5501a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.082+0000 7fd558548640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5501a4810 0x7fd5501a4cc0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fd548038040 tx=0x7fd548035670 comp rx=0 tx=0).stop 2024-02-17T13:04:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.082+0000 7fd558548640 1 -- 172.21.15.18:0/1983941719 shutdown_connections 2024-02-17T13:04:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.082+0000 7fd558548640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd53c0a2350 0x7fd53c0a4810 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.082+0000 7fd558548640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5501a4810 0x7fd5501a4cc0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.082+0000 7fd558548640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd55010d320 0x7fd5501a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.082+0000 7fd558548640 1 --2- 172.21.15.18:0/1983941719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd550108c80 0x7fd55019ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.082+0000 7fd558548640 1 -- 172.21.15.18:0/1983941719 >> 172.21.15.18:0/1983941719 conn(0x7fd55006c6b0 msgr2=0x7fd550071550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.082+0000 7fd558548640 1 -- 172.21.15.18:0/1983941719 shutdown_connections 2024-02-17T13:04:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:24.082+0000 7fd558548640 1 -- 172.21.15.18:0/1983941719 wait complete. 2024-02-17T13:04:24.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 58680 34.79 MB/sec warmup 96 sec latency 310.220 ms 2024-02-17T13:04:24.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:24 smithi119 ceph-mon[36188]: pgmap v457: 97 pgs: 97 active+clean; 696 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 971 op/s 2024-02-17T13:04:24.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:24 smithi146 ceph-mon[36454]: pgmap v457: 97 pgs: 97 active+clean; 696 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 971 op/s 2024-02-17T13:04:25.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:24 smithi018 ceph-mon[32625]: pgmap v457: 97 pgs: 97 active+clean; 696 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 971 op/s 2024-02-17T13:04:25.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 59384 34.80 MB/sec warmup 97 sec latency 348.988 ms 2024-02-17T13:04:26.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 60261 34.82 MB/sec warmup 98 sec latency 53.383 ms 2024-02-17T13:04:26.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:26 smithi119 ceph-mon[36188]: pgmap v458: 97 pgs: 97 active+clean; 716 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 1.06k op/s 2024-02-17T13:04:26.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:26 smithi146 ceph-mon[36454]: pgmap v458: 97 pgs: 97 active+clean; 716 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 1.06k op/s 2024-02-17T13:04:27.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:26 smithi018 ceph-mon[32625]: pgmap v458: 97 pgs: 97 active+clean; 716 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 1.06k op/s 2024-02-17T13:04:27.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 61488 35.19 MB/sec warmup 99 sec latency 32.671 ms 2024-02-17T13:04:28.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 62254 35.39 MB/sec warmup 100 sec latency 243.783 ms 2024-02-17T13:04:28.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:28 smithi119 ceph-mon[36188]: pgmap v459: 97 pgs: 97 active+clean; 716 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 768 op/s 2024-02-17T13:04:28.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:28 smithi146 ceph-mon[36454]: pgmap v459: 97 pgs: 97 active+clean; 716 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 768 op/s 2024-02-17T13:04:29.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:28 smithi018 ceph-mon[32625]: pgmap v459: 97 pgs: 97 active+clean; 716 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 768 op/s 2024-02-17T13:04:29.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 62755 35.27 MB/sec warmup 101 sec latency 150.569 ms 2024-02-17T13:04:30.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 63307 35.11 MB/sec warmup 102 sec latency 261.478 ms 2024-02-17T13:04:30.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:30 smithi119 ceph-mon[36188]: pgmap v460: 97 pgs: 97 active+clean; 730 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 937 op/s 2024-02-17T13:04:30.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:30 smithi146 ceph-mon[36454]: pgmap v460: 97 pgs: 97 active+clean; 730 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 937 op/s 2024-02-17T13:04:31.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:30 smithi018 ceph-mon[32625]: pgmap v460: 97 pgs: 97 active+clean; 730 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 937 op/s 2024-02-17T13:04:31.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 64467 35.43 MB/sec warmup 103 sec latency 832.868 ms 2024-02-17T13:04:32.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 65475 35.66 MB/sec warmup 104 sec latency 1832.916 ms 2024-02-17T13:04:32.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:32 smithi119 ceph-mon[36188]: pgmap v461: 97 pgs: 97 active+clean; 741 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 1.15k op/s 2024-02-17T13:04:32.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:32 smithi146 ceph-mon[36454]: pgmap v461: 97 pgs: 97 active+clean; 741 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 1.15k op/s 2024-02-17T13:04:33.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:32 smithi018 ceph-mon[32625]: pgmap v461: 97 pgs: 97 active+clean; 741 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 1.15k op/s 2024-02-17T13:04:33.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 66212 35.71 MB/sec warmup 105 sec latency 2832.944 ms 2024-02-17T13:04:34.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 66929 35.66 MB/sec warmup 106 sec latency 3832.980 ms 2024-02-17T13:04:34.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:34 smithi119 ceph-mon[36188]: pgmap v462: 97 pgs: 97 active+clean; 741 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 819 op/s 2024-02-17T13:04:34.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:34 smithi146 ceph-mon[36454]: pgmap v462: 97 pgs: 97 active+clean; 741 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 819 op/s 2024-02-17T13:04:35.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:34 smithi018 ceph-mon[32625]: pgmap v462: 97 pgs: 97 active+clean; 741 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 819 op/s 2024-02-17T13:04:35.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 67519 35.60 MB/sec warmup 107 sec latency 4833.072 ms 2024-02-17T13:04:36.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 68598 35.86 MB/sec warmup 108 sec latency 5833.113 ms 2024-02-17T13:04:36.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:36 smithi119 ceph-mon[36188]: pgmap v463: 97 pgs: 97 active+clean; 768 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 1.19k op/s 2024-02-17T13:04:36.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:36 smithi146 ceph-mon[36454]: pgmap v463: 97 pgs: 97 active+clean; 768 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 1.19k op/s 2024-02-17T13:04:37.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:36 smithi018 ceph-mon[32625]: pgmap v463: 97 pgs: 97 active+clean; 768 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 1.19k op/s 2024-02-17T13:04:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.186+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/3861290742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc410b7c0 msgr2=0x7f1fc410dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.186+0000 7f1fc8c4d640 1 --2- 172.21.15.18:0/3861290742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc410b7c0 0x7f1fc410dbf0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009ca0 tx=0x7f1fbc02c190 comp rx=0 tx=0).stop 2024-02-17T13:04:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.186+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/3861290742 shutdown_connections 2024-02-17T13:04:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.186+0000 7f1fc8c4d640 1 --2- 172.21.15.18:0/3861290742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc410b7c0 0x7f1fc410dbf0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.186+0000 7f1fc8c4d640 1 --2- 172.21.15.18:0/3861290742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc4102d10 0x7f1fc410b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.186+0000 7f1fc8c4d640 1 --2- 172.21.15.18:0/3861290742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc41023f0 0x7f1fc41027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.186+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/3861290742 >> 172.21.15.18:0/3861290742 conn(0x7f1fc40771a0 msgr2=0x7f1fc40775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.186+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/3861290742 shutdown_connections 2024-02-17T13:04:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.186+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/3861290742 wait complete. 2024-02-17T13:04:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.187+0000 7f1fc8c4d640 1 Processor -- start 2024-02-17T13:04:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.187+0000 7f1fc8c4d640 1 -- start start 2024-02-17T13:04:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.187+0000 7f1fc8c4d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41023f0 0x7f1fc4100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.187+0000 7f1fc8c4d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc4102d10 0x7f1fc4100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.187+0000 7f1fc8c4d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc410b7c0 0x7f1fc4100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.187+0000 7f1fc8c4d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc41a5a80 con 0x7f1fc41023f0 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.187+0000 7f1fc8c4d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc41a5bf0 con 0x7f1fc4102d10 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.187+0000 7f1fc8c4d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc41a5db0 con 0x7f1fc410b7c0 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.188+0000 7f1fbbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc4102d10 0x7f1fc4100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.188+0000 7f1fc259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41023f0 0x7f1fc4100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.188+0000 7f1fc259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41023f0 0x7f1fc4100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52896/0 (socket says 172.21.15.18:52896) 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.188+0000 7f1fc259c640 1 -- 172.21.15.18:0/328063224 learned_addr learned my addr 172.21.15.18:0/328063224 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.188+0000 7f1fc2d9d640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc410b7c0 0x7f1fc4100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.188+0000 7f1fc259c640 1 -- 172.21.15.18:0/328063224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc410b7c0 msgr2=0x7f1fc4100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.188+0000 7f1fc259c640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc410b7c0 0x7f1fc4100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.188+0000 7f1fc259c640 1 -- 172.21.15.18:0/328063224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc4102d10 msgr2=0x7f1fc4100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.188+0000 7f1fc259c640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc4102d10 0x7f1fc4100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.188+0000 7f1fc259c640 1 -- 172.21.15.18:0/328063224 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc009900 con 0x7f1fc41023f0 2024-02-17T13:04:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.188+0000 7f1fc259c640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41023f0 0x7f1fc4100040 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400e9f0 tx=0x7f1fb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.189+0000 7f1fbb7fe640 1 -- 172.21.15.18:0/328063224 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4009700 con 0x7f1fc41023f0 2024-02-17T13:04:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.189+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/328063224 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc41a6090 con 0x7f1fc41023f0 2024-02-17T13:04:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.189+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/328063224 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc41a65f0 con 0x7f1fc41023f0 2024-02-17T13:04:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.189+0000 7f1fbb7fe640 1 -- 172.21.15.18:0/328063224 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fb400cd80 con 0x7f1fc41023f0 2024-02-17T13:04:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.189+0000 7f1fbb7fe640 1 -- 172.21.15.18:0/328063224 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4018920 con 0x7f1fc41023f0 2024-02-17T13:04:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.192+0000 7f1fbb7fe640 1 -- 172.21.15.18:0/328063224 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1fb4018a80 con 0x7f1fc41023f0 2024-02-17T13:04:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.192+0000 7f1fcb81a740 1 Processor -- start 2024-02-17T13:04:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.192+0000 7f1fcb81a740 1 -- start start 2024-02-17T13:04:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fcb81a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df03820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fcb81a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df03d60 0x56070df041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fcb81a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 0x56070df0f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fcb81a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070de729f0 con 0x56070df0d2e0 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fcb81a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070de6ff20 con 0x56070df04ad0 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fcb81a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070de6b700 con 0x56070df03d60 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fbbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 0x56070df0f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fbbfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 0x56070df0f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52900/0 (socket says 172.21.15.18:52900) 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fbbfff640 1 -- 172.21.15.18:0/295231760 learned_addr learned my addr 172.21.15.18:0/295231760 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fbb7fe640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1fb00a2350 0x7f1fb00a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fc259c640 1 --2- 172.21.15.18:0/295231760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df03d60 0x56070df041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fc2d9d640 1 --2- 172.21.15.18:0/295231760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df03820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fbbfff640 1 -- 172.21.15.18:0/295231760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df03d60 msgr2=0x56070df041e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fbb7fe640 1 -- 172.21.15.18:0/328063224 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1fb4014070 con 0x7f1fc41023f0 2024-02-17T13:04:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fbbfff640 1 --2- 172.21.15.18:0/295231760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df03d60 0x56070df041e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fc2d9d640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1fb00a2350 0x7f1fb00a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fbbfff640 1 -- 172.21.15.18:0/295231760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 msgr2=0x56070df03820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fbbfff640 1 --2- 172.21.15.18:0/295231760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df03820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.193+0000 7f1fbbfff640 1 -- 172.21.15.18:0/295231760 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56070de017b0 con 0x56070df0d2e0 2024-02-17T13:04:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.194+0000 7f1fbbfff640 1 --2- 172.21.15.18:0/295231760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 0x56070df0f6d0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00bc30 tx=0x7f1fac0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793dde12b487af1a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.194+0000 7f1fb8ff9640 1 -- 172.21.15.18:0/295231760 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac01c480 con 0x56070df0d2e0 2024-02-17T13:04:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.194+0000 7f1fb8ff9640 1 -- 172.21.15.18:0/295231760 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fac015070 con 0x56070df0d2e0 2024-02-17T13:04:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.194+0000 7f1fb8ff9640 1 -- 172.21.15.18:0/295231760 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac011a90 con 0x56070df0d2e0 2024-02-17T13:04:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.195+0000 7f1fcb81a740 1 -- 172.21.15.18:0/295231760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 msgr2=0x56070df0f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.195+0000 7f1fc2d9d640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1fb00a2350 0x7f1fb00a4810 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc02c6d0 tx=0x7f1fbc02c190 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.195+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/295231760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 0x56070df0f6d0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00bc30 tx=0x7f1fac0107a0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.195+0000 7f1fcb81a740 1 -- 172.21.15.18:0/295231760 shutdown_connections 2024-02-17T13:04:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.195+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/295231760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 0x56070df0f6d0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.195+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/295231760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df03d60 0x56070df041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.195+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/295231760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df03820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.195+0000 7f1fcb81a740 1 -- 172.21.15.18:0/295231760 >> 172.21.15.18:0/295231760 conn(0x56070decb550 msgr2=0x56070defea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.195+0000 7f1fcb81a740 1 -- 172.21.15.18:0/295231760 shutdown_connections 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.195+0000 7f1fcb81a740 1 -- 172.21.15.18:0/295231760 wait complete. 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.195+0000 7f1fcb81a740 1 -- start start 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.195+0000 7f1fcb81a740 1 Processor -- start 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fcb81a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 0x56070df0a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fcb81a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df0a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fcb81a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df0d2e0 0x56070df0acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fcb81a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070de10bf0 con 0x56070df03d60 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fcb81a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070db3f250 con 0x56070df04ad0 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fcb81a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070ddf1200 con 0x56070df0d2e0 2024-02-17T13:04:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fcb81a740 10 client.0 _ll_register_callbacks cb 0x56070de5fd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fc259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 0x56070df0a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fc259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 0x56070df0a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52910/0 (socket says 172.21.15.18:52910) 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fc259c640 1 -- 172.21.15.18:0/1199438793 learned_addr learned my addr 172.21.15.18:0/1199438793 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fc2d9d640 1 --2- 172.21.15.18:0/1199438793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df0d2e0 0x56070df0acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fc259c640 1 -- 172.21.15.18:0/1199438793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df0d2e0 msgr2=0x56070df0acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fc259c640 1 --2- 172.21.15.18:0/1199438793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df0d2e0 0x56070df0acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fc259c640 1 -- 172.21.15.18:0/1199438793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 msgr2=0x56070df0a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fc259c640 1 --2- 172.21.15.18:0/1199438793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df0a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fc259c640 1 -- 172.21.15.18:0/1199438793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac00b860 con 0x56070df03d60 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fc259c640 1 --2- 172.21.15.18:0/1199438793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 0x56070df0a250 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c6a0 tx=0x7f1fb409ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f2db1e9e555113 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fa9ffb640 1 -- 172.21.15.18:0/1199438793 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40d02e0 con 0x56070df03d60 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fcb81a740 1 -- 172.21.15.18:0/1199438793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56070db6b600 con 0x56070df03d60 2024-02-17T13:04:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fa9ffb640 1 -- 172.21.15.18:0/1199438793 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1fb409a1a0 con 0x56070df03d60 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fa9ffb640 1 -- 172.21.15.18:0/1199438793 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb40b9230 con 0x56070df03d60 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fcb81a740 10 client.20646 fetch_fsmap learned FSMap version 65 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.196+0000 7f1fcb81a740 1 -- 172.21.15.18:0/1199438793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x56070de69d80 con 0x56070df03d60 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.197+0000 7f1fa9ffb640 1 -- 172.21.15.18:0/1199438793 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fb40b2020 con 0x56070df03d60 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.197+0000 7f1fa9ffb640 1 -- 172.21.15.18:0/1199438793 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 65) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f1fb409fd10 con 0x56070df03d60 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.197+0000 7f1fcb81a740 10 client.20646 fetch_fsmap finished waiting for FSMap version 65 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.197+0000 7f1fcb81a740 10 client.20646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.197+0000 7f1fcb81a740 20 client.20646 populate_metadata read hostname 'smithi018' 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.197+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/1199438793 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56070df1d7b0 0x56070df1fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.197+0000 7f1fcb81a740 4 client.20646 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.197+0000 7f1fcb81a740 1 -- 172.21.15.18:0/1199438793 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56070dbbc1b0 con 0x56070df1d7b0 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.197+0000 7f1fbbfff640 1 --2- 172.21.15.18:0/1199438793 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56070df1d7b0 0x56070df1fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.198+0000 7f1fa9ffb640 10 client.20646 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.198+0000 7f1fbbfff640 1 --2- 172.21.15.18:0/1199438793 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56070df1d7b0 0x56070df1fc10 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e19ae3117b4170 server_cookie=508725046f035a00 in_seq=0 out_seq=0 2024-02-17T13:04:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fa9ffb640 1 -- 172.21.15.18:0/1199438793 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56070dbbc1b0 con 0x56070df1d7b0 2024-02-17T13:04:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fa9ffb640 10 client.20646 handle_command_reply: tid=0 2024-02-17T13:04:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fcb81a740 1 client.20646 shutdown 2024-02-17T13:04:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fcb81a740 1 -- 172.21.15.18:0/1199438793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 msgr2=0x56070df0a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/1199438793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 0x56070df0a250 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c6a0 tx=0x7f1fb409ef30 comp rx=0 tx=0).stop 2024-02-17T13:04:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fcb81a740 1 -- 172.21.15.18:0/1199438793 shutdown_connections 2024-02-17T13:04:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/1199438793 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56070df1d7b0 0x56070df1fc10 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/1199438793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df0d2e0 0x56070df0acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/1199438793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df0a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/1199438793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 0x56070df0a250 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fa9ffb640 0 client.20646 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fcb81a740 1 -- 172.21.15.18:0/1199438793 >> 172.21.15.18:0/1199438793 conn(0x56070decb550 msgr2=0x56070df007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fcb81a740 1 -- 172.21.15.18:0/1199438793 shutdown_connections 2024-02-17T13:04:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fcb81a740 1 -- 172.21.15.18:0/1199438793 wait complete. 2024-02-17T13:04:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.200+0000 7f1fcb81a740 20 client.20646 trim_cache size 0 max 16384 2024-02-17T13:04:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.216+0000 7f1fcb81a740 1 Processor -- start 2024-02-17T13:04:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.216+0000 7f1fcb81a740 1 -- start start 2024-02-17T13:04:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fcb81a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df03d60 0x56070df0c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fcb81a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df0aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fcb81a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 0x56070df074d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fcb81a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070defec10 con 0x56070df0d2e0 2024-02-17T13:04:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fcb81a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070defefd0 con 0x56070df04ad0 2024-02-17T13:04:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fcb81a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070df1d5a0 con 0x56070df03d60 2024-02-17T13:04:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fc2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 0x56070df074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fc259c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df03d60 0x56070df0c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fc2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 0x56070df074d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52926/0 (socket says 172.21.15.18:52926) 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fc2d9d640 1 -- 172.21.15.18:0/4042383380 learned_addr learned my addr 172.21.15.18:0/4042383380 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fc259c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df03d60 0x56070df0c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40462/0 (socket says 172.21.15.18:40462) 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fbbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df0aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fc2d9d640 1 -- 172.21.15.18:0/4042383380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df03d60 msgr2=0x56070df0c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fc2d9d640 1 --2- 172.21.15.18:0/4042383380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df03d60 0x56070df0c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fc2d9d640 1 -- 172.21.15.18:0/4042383380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 msgr2=0x56070df0aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fc2d9d640 1 --2- 172.21.15.18:0/4042383380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df0aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fc2d9d640 1 -- 172.21.15.18:0/4042383380 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56070de067b0 con 0x56070df0d2e0 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.217+0000 7f1fc2d9d640 1 --2- 172.21.15.18:0/4042383380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 0x56070df074d0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009ca0 tx=0x7f1fbc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7a70722ef1308a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.218+0000 7f1fabfff640 1 -- 172.21.15.18:0/4042383380 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc003cf0 con 0x56070df0d2e0 2024-02-17T13:04:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.218+0000 7f1fabfff640 1 -- 172.21.15.18:0/4042383380 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fbc049040 con 0x56070df0d2e0 2024-02-17T13:04:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.218+0000 7f1fabfff640 1 -- 172.21.15.18:0/4042383380 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0045a0 con 0x56070df0d2e0 2024-02-17T13:04:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.218+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4042383380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 msgr2=0x56070df074d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.218+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/4042383380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 0x56070df074d0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009ca0 tx=0x7f1fbc009360 comp rx=0 tx=0).stop 2024-02-17T13:04:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4042383380 shutdown_connections 2024-02-17T13:04:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/4042383380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df0d2e0 0x56070df074d0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/4042383380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df0aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/4042383380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df03d60 0x56070df0c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4042383380 >> 172.21.15.18:0/4042383380 conn(0x56070decb550 msgr2=0x56070df08b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4042383380 shutdown_connections 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4042383380 wait complete. 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 -- start start 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 Processor -- start 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 0x56070df14360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df12750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df0d2e0 0x56070df12c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070df13200 con 0x56070df03d60 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070df13370 con 0x56070df04ad0 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070df13530 con 0x56070df0d2e0 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fcb81a740 10 client.0 _ll_register_callbacks cb 0x7f1fac00b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fc259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 0x56070df14360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fc259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 0x56070df14360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52928/0 (socket says 172.21.15.18:52928) 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.219+0000 7f1fc259c640 1 -- 172.21.15.18:0/4180436856 learned_addr learned my addr 172.21.15.18:0/4180436856 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fc2d9d640 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df0d2e0 0x56070df12c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fbbfff640 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df12750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fc259c640 1 -- 172.21.15.18:0/4180436856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df0d2e0 msgr2=0x56070df12c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fc259c640 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df0d2e0 0x56070df12c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fc259c640 1 -- 172.21.15.18:0/4180436856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 msgr2=0x56070df12750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fc259c640 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df12750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fc259c640 1 -- 172.21.15.18:0/4180436856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc410e870 con 0x56070df03d60 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fc2d9d640 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df0d2e0 0x56070df12c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fc259c640 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 0x56070df14360 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40cc000 tx=0x7f1fb409ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88e8b5baa87dba1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fa9ffb640 1 -- 172.21.15.18:0/4180436856 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40b9230 con 0x56070df03d60 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fa9ffb640 1 -- 172.21.15.18:0/4180436856 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1fb40b3030 con 0x56070df03d60 2024-02-17T13:04:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4180436856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fac015070 con 0x56070df03d60 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fa9ffb640 1 -- 172.21.15.18:0/4180436856 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb409a3b0 con 0x56070df03d60 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fcb81a740 10 client.20652 fetch_fsmap learned FSMap version 65 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.220+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4180436856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x56070de2c540 con 0x56070df03d60 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.221+0000 7f1fa9ffb640 1 -- 172.21.15.18:0/4180436856 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fac015070 con 0x56070df03d60 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.221+0000 7f1fa9ffb640 1 -- 172.21.15.18:0/4180436856 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 65) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f1fb4085aa0 con 0x56070df03d60 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.221+0000 7f1fcb81a740 10 client.20652 fetch_fsmap finished waiting for FSMap version 65 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.221+0000 7f1fcb81a740 10 client.20652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.221+0000 7f1fcb81a740 20 client.20652 populate_metadata read hostname 'smithi018' 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.221+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56070df75210 0x56070df77630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.221+0000 7f1fcb81a740 4 client.20652 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.221+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4180436856 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56070dba8550 con 0x56070df75210 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.222+0000 7f1fbbfff640 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56070df75210 0x56070df77630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.222+0000 7f1fbbfff640 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56070df75210 0x56070df77630 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62cf7a24bb7ba9a6 server_cookie=73ed2a395f2b6047 in_seq=0 out_seq=0 2024-02-17T13:04:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.222+0000 7f1fa9ffb640 10 client.20652 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.472+0000 7f1fa9ffb640 1 -- 172.21.15.18:0/4180436856 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56070dba8550 con 0x56070df75210 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.472+0000 7f1fa9ffb640 10 client.20652 handle_command_reply: tid=0 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fcb81a740 1 client.20652 shutdown 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4180436856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 msgr2=0x56070df14360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 0x56070df14360 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40cc000 tx=0x7f1fb409ff40 comp rx=0 tx=0).stop 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4180436856 shutdown_connections 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x56070df75210 0x56070df77630 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56070df0d2e0 0x56070df12c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fa9ffb640 0 client.20652 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56070df04ad0 0x56070df12750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fcb81a740 1 --2- 172.21.15.18:0/4180436856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56070df03d60 0x56070df14360 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4180436856 >> 172.21.15.18:0/4180436856 conn(0x56070decb550 msgr2=0x56070defd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4180436856 shutdown_connections 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fcb81a740 1 -- 172.21.15.18:0/4180436856 wait complete. 2024-02-17T13:04:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.473+0000 7f1fcb81a740 20 client.20652 trim_cache size 0 max 16384 2024-02-17T13:04:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:04:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:04:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:04:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:04:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.474+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/328063224 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1fb00a2350 msgr2=0x7f1fb00a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.474+0000 7f1fc8c4d640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1fb00a2350 0x7f1fb00a4810 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc02c6d0 tx=0x7f1fbc02c190 comp rx=0 tx=0).stop 2024-02-17T13:04:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.474+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/328063224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41023f0 msgr2=0x7f1fc4100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.474+0000 7f1fc8c4d640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41023f0 0x7f1fc4100040 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400e9f0 tx=0x7f1fb400eeb0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.474+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/328063224 shutdown_connections 2024-02-17T13:04:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.474+0000 7f1fc8c4d640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f1fb00a2350 0x7f1fb00a4810 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.474+0000 7f1fc8c4d640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc410b7c0 0x7f1fc4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.474+0000 7f1fc8c4d640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc4102d10 0x7f1fc4100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.474+0000 7f1fc8c4d640 1 --2- 172.21.15.18:0/328063224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41023f0 0x7f1fc4100040 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.474+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/328063224 >> 172.21.15.18:0/328063224 conn(0x7f1fc40771a0 msgr2=0x7f1fc40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.474+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/328063224 shutdown_connections 2024-02-17T13:04:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:37.474+0000 7f1fc8c4d640 1 -- 172.21.15.18:0/328063224 wait complete. 2024-02-17T13:04:37.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 69197 35.87 MB/sec warmup 109 sec latency 6833.140 ms 2024-02-17T13:04:38.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 69875 35.80 MB/sec warmup 110 sec latency 7833.168 ms 2024-02-17T13:04:38.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:38 smithi119 ceph-mon[36188]: pgmap v464: 97 pgs: 97 active+clean; 768 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 792 op/s 2024-02-17T13:04:38.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:38 smithi146 ceph-mon[36454]: pgmap v464: 97 pgs: 97 active+clean; 768 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 792 op/s 2024-02-17T13:04:39.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:38 smithi018 ceph-mon[32625]: pgmap v464: 97 pgs: 97 active+clean; 768 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 792 op/s 2024-02-17T13:04:39.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 70979 35.97 MB/sec warmup 111 sec latency 8008.469 ms 2024-02-17T13:04:40.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 71536 35.95 MB/sec warmup 112 sec latency 248.747 ms 2024-02-17T13:04:40.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:40 smithi119 ceph-mon[36188]: pgmap v465: 97 pgs: 97 active+clean; 771 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 934 op/s 2024-02-17T13:04:40.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:40 smithi146 ceph-mon[36454]: pgmap v465: 97 pgs: 97 active+clean; 771 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 934 op/s 2024-02-17T13:04:41.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:40 smithi018 ceph-mon[32625]: pgmap v465: 97 pgs: 97 active+clean; 771 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 934 op/s 2024-02-17T13:04:41.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 72483 36.18 MB/sec warmup 113 sec latency 75.152 ms 2024-02-17T13:04:42.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 73254 36.18 MB/sec warmup 114 sec latency 47.531 ms 2024-02-17T13:04:42.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:42 smithi119 ceph-mon[36188]: pgmap v466: 97 pgs: 97 active+clean; 782 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-02-17T13:04:42.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:42 smithi146 ceph-mon[36454]: pgmap v466: 97 pgs: 97 active+clean; 782 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-02-17T13:04:43.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:42 smithi018 ceph-mon[32625]: pgmap v466: 97 pgs: 97 active+clean; 782 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-02-17T13:04:43.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 73925 36.14 MB/sec warmup 115 sec latency 253.426 ms 2024-02-17T13:04:44.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 74907 36.26 MB/sec warmup 116 sec latency 70.412 ms 2024-02-17T13:04:44.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:44 smithi119 ceph-mon[36188]: pgmap v467: 97 pgs: 97 active+clean; 782 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 820 op/s 2024-02-17T13:04:44.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:44 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:44.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:44 smithi146 ceph-mon[36454]: pgmap v467: 97 pgs: 97 active+clean; 782 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 820 op/s 2024-02-17T13:04:44.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:44 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:45.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:44 smithi018 ceph-mon[32625]: pgmap v467: 97 pgs: 97 active+clean; 782 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 820 op/s 2024-02-17T13:04:45.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:44 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:45.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 75578 36.30 MB/sec warmup 117 sec latency 229.666 ms 2024-02-17T13:04:46.789 INFO:tasks.workunit.client.0.smithi018.stdout: 10 76455 36.39 MB/sec warmup 118 sec latency 51.064 ms 2024-02-17T13:04:46.823 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:46 smithi018 ceph-mon[32625]: pgmap v468: 97 pgs: 97 active+clean; 809 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2024-02-17T13:04:46.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:46 smithi119 ceph-mon[36188]: pgmap v468: 97 pgs: 97 active+clean; 809 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2024-02-17T13:04:46.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:46 smithi146 ceph-mon[36454]: pgmap v468: 97 pgs: 97 active+clean; 809 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2024-02-17T13:04:47.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77255 36.47 MB/sec warmup 119 sec latency 198.582 ms 2024-02-17T13:04:48.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:48 smithi119 ceph-mon[36188]: pgmap v469: 97 pgs: 97 active+clean; 809 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 807 op/s 2024-02-17T13:04:48.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:48 smithi146 ceph-mon[36454]: pgmap v469: 97 pgs: 97 active+clean; 809 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 807 op/s 2024-02-17T13:04:49.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:48 smithi018 ceph-mon[32625]: pgmap v469: 97 pgs: 97 active+clean; 809 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 807 op/s 2024-02-17T13:04:49.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 78692 36.49 MB/sec execute 1 sec latency 191.812 ms 2024-02-17T13:04:50.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.592+0000 7ffa33edf640 1 -- 172.21.15.18:0/2986116669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa2c100c90 msgr2=0x7ffa2c10ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:50.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.592+0000 7ffa33edf640 1 --2- 172.21.15.18:0/2986116669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa2c100c90 0x7ffa2c10ceb0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009be0 tx=0x7ffa2402bef0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.592+0000 7ffa33edf640 1 -- 172.21.15.18:0/2986116669 shutdown_connections 2024-02-17T13:04:50.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.592+0000 7ffa33edf640 1 --2- 172.21.15.18:0/2986116669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa2c100c90 0x7ffa2c10ceb0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.592+0000 7ffa33edf640 1 --2- 172.21.15.18:0/2986116669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa2c1002d0 0x7ffa2c100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.592+0000 7ffa33edf640 1 --2- 172.21.15.18:0/2986116669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa2c105020 0x7ffa2c105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.592+0000 7ffa33edf640 1 -- 172.21.15.18:0/2986116669 >> 172.21.15.18:0/2986116669 conn(0x7ffa2c076ed0 msgr2=0x7ffa2c0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:50.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.592+0000 7ffa33edf640 1 -- 172.21.15.18:0/2986116669 shutdown_connections 2024-02-17T13:04:50.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.592+0000 7ffa33edf640 1 -- 172.21.15.18:0/2986116669 wait complete. 2024-02-17T13:04:50.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.593+0000 7ffa33edf640 1 Processor -- start 2024-02-17T13:04:50.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.593+0000 7ffa33edf640 1 -- start start 2024-02-17T13:04:50.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.593+0000 7ffa33edf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa2c1002d0 0x7ffa2c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.593+0000 7ffa33edf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa2c100c90 0x7ffa2c19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.593+0000 7ffa33edf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa2c105020 0x7ffa2c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa33edf640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c199e40 con 0x7ffa2c100c90 2024-02-17T13:04:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa33edf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c199fb0 con 0x7ffa2c105020 2024-02-17T13:04:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa33edf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c19a170 con 0x7ffa2c1002d0 2024-02-17T13:04:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa31c7b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa2c1002d0 0x7ffa2c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa31c7b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa2c1002d0 0x7ffa2c19f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40350/0 (socket says 172.21.15.18:40350) 2024-02-17T13:04:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa31c7b640 1 -- 172.21.15.18:0/1079039966 learned_addr learned my addr 172.21.15.18:0/1079039966 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa3147a640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa2c100c90 0x7ffa2c19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa3247c640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa2c105020 0x7ffa2c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa31c7b640 1 -- 172.21.15.18:0/1079039966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa2c105020 msgr2=0x7ffa2c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa31c7b640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa2c105020 0x7ffa2c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa31c7b640 1 -- 172.21.15.18:0/1079039966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa2c100c90 msgr2=0x7ffa2c19fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa31c7b640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa2c100c90 0x7ffa2c19fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.594+0000 7ffa31c7b640 1 -- 172.21.15.18:0/1079039966 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa24009840 con 0x7ffa2c1002d0 2024-02-17T13:04:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.595+0000 7ffa3147a640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa2c100c90 0x7ffa2c19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:04:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.597+0000 7ffa31c7b640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa2c1002d0 0x7ffa2c19f630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800e9c0 tx=0x7ffa2800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.598+0000 7ffa22ffd640 1 -- 172.21.15.18:0/1079039966 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2800ccc0 con 0x7ffa2c1002d0 2024-02-17T13:04:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.598+0000 7ffa22ffd640 1 -- 172.21.15.18:0/1079039966 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa2800ce20 con 0x7ffa2c1002d0 2024-02-17T13:04:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.598+0000 7ffa22ffd640 1 -- 172.21.15.18:0/1079039966 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28010430 con 0x7ffa2c1002d0 2024-02-17T13:04:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.598+0000 7ffa33edf640 1 -- 172.21.15.18:0/1079039966 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c19a450 con 0x7ffa2c1002d0 2024-02-17T13:04:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.598+0000 7ffa33edf640 1 -- 172.21.15.18:0/1079039966 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2c107040 con 0x7ffa2c1002d0 2024-02-17T13:04:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.602+0000 7ffa22ffd640 1 -- 172.21.15.18:0/1079039966 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ffa28004f80 con 0x7ffa2c1002d0 2024-02-17T13:04:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.602+0000 7ffa36b8c740 1 Processor -- start 2024-02-17T13:04:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.603+0000 7ffa36b8c740 1 -- start start 2024-02-17T13:04:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.603+0000 7ffa36b8c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 0x5642bd062820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.603+0000 7ffa36b8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd0631e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.603+0000 7ffa36b8c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.603+0000 7ffa36b8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bcfd19f0 con 0x5642bd062d60 2024-02-17T13:04:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.603+0000 7ffa36b8c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bcfcef20 con 0x5642bd063ad0 2024-02-17T13:04:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.603+0000 7ffa36b8c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bcfca700 con 0x5642bd06c2e0 2024-02-17T13:04:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.603+0000 7ffa31c7b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd0631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.603+0000 7ffa3247c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 0x5642bd062820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.603+0000 7ffa3147a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.603+0000 7ffa31c7b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd0631e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35786/0 (socket says 172.21.15.18:35786) 2024-02-17T13:04:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.603+0000 7ffa31c7b640 1 -- 172.21.15.18:0/1372833471 learned_addr learned my addr 172.21.15.18:0/1372833471 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.604+0000 7ffa22ffd640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffa180a2350 0x7ffa180a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.604+0000 7ffa3147a640 1 -- 172.21.15.18:0/1372833471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 msgr2=0x5642bd062820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.604+0000 7ffa3147a640 1 --2- 172.21.15.18:0/1372833471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 0x5642bd062820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.604+0000 7ffa3147a640 1 -- 172.21.15.18:0/1372833471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 msgr2=0x5642bd0631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.604+0000 7ffa3147a640 1 --2- 172.21.15.18:0/1372833471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd0631e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.604+0000 7ffa3147a640 1 -- 172.21.15.18:0/1372833471 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642bcf607b0 con 0x5642bd06c2e0 2024-02-17T13:04:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.604+0000 7ffa22ffd640 1 -- 172.21.15.18:0/1079039966 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ffa28014070 con 0x7ffa2c1002d0 2024-02-17T13:04:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.604+0000 7ffa3147a640 1 --2- 172.21.15.18:0/1372833471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd06e6d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c19aaf0 tx=0x7ffa1c0228e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eb63782ff1672cc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.604+0000 7ffa3247c640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffa180a2350 0x7ffa180a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.605+0000 7ffa17fff640 1 -- 172.21.15.18:0/1372833471 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c004310 con 0x5642bd06c2e0 2024-02-17T13:04:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.605+0000 7ffa17fff640 1 -- 172.21.15.18:0/1372833471 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c004470 con 0x5642bd06c2e0 2024-02-17T13:04:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.605+0000 7ffa17fff640 1 -- 172.21.15.18:0/1372833471 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c020af0 con 0x5642bd06c2e0 2024-02-17T13:04:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.606+0000 7ffa36b8c740 1 -- 172.21.15.18:0/1372833471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 msgr2=0x5642bd06e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:50.601 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.606+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/1372833471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd06e6d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c19aaf0 tx=0x7ffa1c0228e0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.601 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.606+0000 7ffa3247c640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffa180a2350 0x7ffa180a4810 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009370 tx=0x7ffa24009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.607+0000 7ffa36b8c740 1 -- 172.21.15.18:0/1372833471 shutdown_connections 2024-02-17T13:04:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.607+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/1372833471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd06e6d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.607+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/1372833471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd0631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.608+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/1372833471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 0x5642bd062820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.608+0000 7ffa36b8c740 1 -- 172.21.15.18:0/1372833471 >> 172.21.15.18:0/1372833471 conn(0x5642bd02a550 msgr2=0x5642bd05da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.608+0000 7ffa36b8c740 1 -- 172.21.15.18:0/1372833471 shutdown_connections 2024-02-17T13:04:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.608+0000 7ffa36b8c740 1 -- 172.21.15.18:0/1372833471 wait complete. 2024-02-17T13:04:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.609+0000 7ffa36b8c740 1 -- start start 2024-02-17T13:04:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.609+0000 7ffa36b8c740 1 Processor -- start 2024-02-17T13:04:50.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.610+0000 7ffa36b8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd069380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.610+0000 7ffa36b8c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 0x5642bd0698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.610+0000 7ffa36b8c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd069e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.610+0000 7ffa31c7b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.611+0000 7ffa31c7b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd069380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35798/0 (socket says 172.21.15.18:35798) 2024-02-17T13:04:50.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.611+0000 7ffa36b8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bcf6fbf0 con 0x5642bd062d60 2024-02-17T13:04:50.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.611+0000 7ffa36b8c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bcc9e250 con 0x5642bd063ad0 2024-02-17T13:04:50.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.611+0000 7ffa36b8c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bcf50200 con 0x5642bd06c2e0 2024-02-17T13:04:50.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.611+0000 7ffa36b8c740 10 client.0 _ll_register_callbacks cb 0x5642bcfbed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:04:50.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.611+0000 7ffa3247c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.611+0000 7ffa31c7b640 1 -- 172.21.15.18:0/3524984619 learned_addr learned my addr 172.21.15.18:0/3524984619 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:50.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.611+0000 7ffa3247c640 1 -- 172.21.15.18:0/3524984619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 msgr2=0x5642bd0698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:50.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.611+0000 7ffa3247c640 1 --2- 172.21.15.18:0/3524984619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 0x5642bd0698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.611+0000 7ffa3247c640 1 -- 172.21.15.18:0/3524984619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 msgr2=0x5642bd069380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:50.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.611+0000 7ffa3247c640 1 --2- 172.21.15.18:0/3524984619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd069380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.611+0000 7ffa3247c640 1 -- 172.21.15.18:0/3524984619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c009c00 con 0x5642bd06c2e0 2024-02-17T13:04:50.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.612+0000 7ffa3247c640 1 --2- 172.21.15.18:0/3524984619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd069e00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffa24006010 tx=0x7ffa2402e050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16b5d51b9c9d0565 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:50.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.612+0000 7ffa157fa640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:04:50.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.612+0000 7ffa157fa640 1 -- 172.21.15.18:0/3524984619 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24033eb0 con 0x5642bd06c2e0 2024-02-17T13:04:50.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.612+0000 7ffa157fa640 1 -- 172.21.15.18:0/3524984619 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ffa24004590 con 0x5642bd06c2e0 2024-02-17T13:04:50.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.612+0000 7ffa36b8c740 1 -- 172.21.15.18:0/3524984619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642bccca600 con 0x5642bd06c2e0 2024-02-17T13:04:50.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.612+0000 7ffa157fa640 1 -- 172.21.15.18:0/3524984619 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa2402ca60 con 0x5642bd06c2e0 2024-02-17T13:04:50.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.612+0000 7ffa36b8c740 10 client.29590 fetch_fsmap learned FSMap version 66 2024-02-17T13:04:50.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.612+0000 7ffa36b8c740 1 -- 172.21.15.18:0/3524984619 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x5642bcfc8d80 con 0x5642bd06c2e0 2024-02-17T13:04:50.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.613+0000 7ffa157fa640 1 -- 172.21.15.18:0/3524984619 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa24004750 con 0x5642bd06c2e0 2024-02-17T13:04:50.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.613+0000 7ffa157fa640 1 -- 172.21.15.18:0/3524984619 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 66) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffa2403c040 con 0x5642bd06c2e0 2024-02-17T13:04:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.613+0000 7ffa36b8c740 10 client.29590 fetch_fsmap finished waiting for FSMap version 66 2024-02-17T13:04:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.613+0000 7ffa36b8c740 10 client.29590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:04:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.613+0000 7ffa36b8c740 20 client.29590 populate_metadata read hostname 'smithi018' 2024-02-17T13:04:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.613+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/3524984619 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5642bd07c790 0x5642bd07ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.613+0000 7ffa36b8c740 4 client.29590 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:04:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.613+0000 7ffa36b8c740 1 -- 172.21.15.18:0/3524984619 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642bccca600 con 0x5642bd07c790 2024-02-17T13:04:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.616+0000 7ffa31c7b640 1 --2- 172.21.15.18:0/3524984619 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5642bd07c790 0x5642bd07ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.617+0000 7ffa31c7b640 1 --2- 172.21.15.18:0/3524984619 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5642bd07c790 0x5642bd07ec10 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56febfce81cb7d06 server_cookie=2adf5be43096eb2e in_seq=0 out_seq=0 2024-02-17T13:04:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.617+0000 7ffa157fa640 10 client.29590 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.626+0000 7ffa157fa640 1 -- 172.21.15.18:0/3524984619 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5642bccca600 con 0x5642bd07c790 2024-02-17T13:04:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.626+0000 7ffa157fa640 10 client.29590 handle_command_reply: tid=0 2024-02-17T13:04:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.628+0000 7ffa36b8c740 1 client.29590 shutdown 2024-02-17T13:04:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.628+0000 7ffa36b8c740 1 -- 172.21.15.18:0/3524984619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 msgr2=0x5642bd069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.628+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/3524984619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd069e00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffa24006010 tx=0x7ffa2402e050 comp rx=0 tx=0).stop 2024-02-17T13:04:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.629+0000 7ffa36b8c740 1 -- 172.21.15.18:0/3524984619 shutdown_connections 2024-02-17T13:04:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.629+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/3524984619 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5642bd07c790 0x5642bd07ec10 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.629+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/3524984619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd069e00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.629+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/3524984619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 0x5642bd0698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.629+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/3524984619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd069380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.629+0000 7ffa36b8c740 1 -- 172.21.15.18:0/3524984619 >> 172.21.15.18:0/3524984619 conn(0x5642bd02a550 msgr2=0x5642bd05f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.629+0000 7ffa157fa640 0 client.29590 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.630+0000 7ffa36b8c740 1 -- 172.21.15.18:0/3524984619 shutdown_connections 2024-02-17T13:04:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.630+0000 7ffa36b8c740 1 -- 172.21.15.18:0/3524984619 wait complete. 2024-02-17T13:04:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.630+0000 7ffa36b8c740 20 client.29590 trim_cache size 0 max 16384 2024-02-17T13:04:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.654+0000 7ffa36b8c740 1 Processor -- start 2024-02-17T13:04:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.654+0000 7ffa36b8c740 1 -- start start 2024-02-17T13:04:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.655+0000 7ffa36b8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.655+0000 7ffa36b8c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 0x5642bd073100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.655+0000 7ffa36b8c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd073640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.655+0000 7ffa36b8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bd05d310 con 0x5642bd062d60 2024-02-17T13:04:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.655+0000 7ffa36b8c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bd07c580 con 0x5642bd063ad0 2024-02-17T13:04:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.655+0000 7ffa36b8c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bd05ea50 con 0x5642bd06c2e0 2024-02-17T13:04:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.655+0000 7ffa3247c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.655+0000 7ffa3247c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd073640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40390/0 (socket says 172.21.15.18:40390) 2024-02-17T13:04:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.655+0000 7ffa3247c640 1 -- 172.21.15.18:0/1126523230 learned_addr learned my addr 172.21.15.18:0/1126523230 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.656+0000 7ffa3147a640 1 --2- 172.21.15.18:0/1126523230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 0x5642bd073100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.656+0000 7ffa31c7b640 1 --2- 172.21.15.18:0/1126523230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.656+0000 7ffa3247c640 1 -- 172.21.15.18:0/1126523230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 msgr2=0x5642bd073100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.656+0000 7ffa3247c640 1 --2- 172.21.15.18:0/1126523230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 0x5642bd073100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.656+0000 7ffa3247c640 1 -- 172.21.15.18:0/1126523230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 msgr2=0x5642bd06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.656+0000 7ffa3247c640 1 --2- 172.21.15.18:0/1126523230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.656+0000 7ffa3247c640 1 -- 172.21.15.18:0/1126523230 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642bcf657b0 con 0x5642bd06c2e0 2024-02-17T13:04:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.657+0000 7ffa3247c640 1 --2- 172.21.15.18:0/1126523230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd073640 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffa2402cf80 tx=0x7ffa240338f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=465e3c098ff78606 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.657+0000 7ffa31c7b640 1 --2- 172.21.15.18:0/1126523230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:04:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.657+0000 7ffa177fe640 1 -- 172.21.15.18:0/1126523230 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2402c9e0 con 0x5642bd06c2e0 2024-02-17T13:04:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.658+0000 7ffa177fe640 1 -- 172.21.15.18:0/1126523230 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa24034700 con 0x5642bd06c2e0 2024-02-17T13:04:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.658+0000 7ffa177fe640 1 -- 172.21.15.18:0/1126523230 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24004670 con 0x5642bd06c2e0 2024-02-17T13:04:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.659+0000 7ffa36b8c740 1 -- 172.21.15.18:0/1126523230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 msgr2=0x5642bd073640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.659+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/1126523230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd073640 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffa2402cf80 tx=0x7ffa240338f0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.659+0000 7ffa36b8c740 1 -- 172.21.15.18:0/1126523230 shutdown_connections 2024-02-17T13:04:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.659+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/1126523230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd073640 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.659+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/1126523230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd063ad0 0x5642bd073100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.659+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/1126523230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd062d60 0x5642bd06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.659+0000 7ffa36b8c740 1 -- 172.21.15.18:0/1126523230 >> 172.21.15.18:0/1126523230 conn(0x5642bd02a550 msgr2=0x5642bd05d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.660+0000 7ffa36b8c740 1 -- 172.21.15.18:0/1126523230 shutdown_connections 2024-02-17T13:04:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.662+0000 7ffa36b8c740 1 -- 172.21.15.18:0/1126523230 wait complete. 2024-02-17T13:04:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.662+0000 7ffa36b8c740 1 -- start start 2024-02-17T13:04:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.662+0000 7ffa36b8c740 1 Processor -- start 2024-02-17T13:04:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.663+0000 7ffa36b8c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd062d60 0x5642bd073300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.663+0000 7ffa36b8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd063ad0 0x5642bd06f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.663+0000 7ffa36b8c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd06fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.663+0000 7ffa36b8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bd0701a0 con 0x5642bd063ad0 2024-02-17T13:04:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.663+0000 7ffa36b8c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bd070310 con 0x5642bd062d60 2024-02-17T13:04:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.663+0000 7ffa36b8c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bd0704d0 con 0x5642bd06c2e0 2024-02-17T13:04:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.663+0000 7ffa3247c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd06fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.663+0000 7ffa3247c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd06fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40392/0 (socket says 172.21.15.18:40392) 2024-02-17T13:04:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.663+0000 7ffa3247c640 1 -- 172.21.15.18:0/4026751460 learned_addr learned my addr 172.21.15.18:0/4026751460 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:04:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.663+0000 7ffa36b8c740 10 client.0 _ll_register_callbacks cb 0x7ffa1c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:04:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.664+0000 7ffa3247c640 1 -- 172.21.15.18:0/4026751460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd062d60 msgr2=0x5642bd073300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T13:04:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.664+0000 7ffa3247c640 1 --2- 172.21.15.18:0/4026751460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd062d60 0x5642bd073300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.664+0000 7ffa3247c640 1 -- 172.21.15.18:0/4026751460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd063ad0 msgr2=0x5642bd06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:04:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.664+0000 7ffa3247c640 1 --2- 172.21.15.18:0/4026751460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd063ad0 0x5642bd06f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.664+0000 7ffa3247c640 1 -- 172.21.15.18:0/4026751460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642bcfc8d80 con 0x5642bd06c2e0 2024-02-17T13:04:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.664+0000 7ffa3247c640 1 --2- 172.21.15.18:0/4026751460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd06fc60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffa24040040 tx=0x7ffa24034de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=326e15ce68fbb038 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:04:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.664+0000 7ffa157fa640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:04:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.664+0000 7ffa157fa640 1 -- 172.21.15.18:0/4026751460 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24004410 con 0x5642bd06c2e0 2024-02-17T13:04:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.665+0000 7ffa157fa640 1 -- 172.21.15.18:0/4026751460 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ffa24004570 con 0x5642bd06c2e0 2024-02-17T13:04:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.665+0000 7ffa36b8c740 1 -- 172.21.15.18:0/4026751460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa1c004470 con 0x5642bd06c2e0 2024-02-17T13:04:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.665+0000 7ffa157fa640 1 -- 172.21.15.18:0/4026751460 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa24004730 con 0x5642bd06c2e0 2024-02-17T13:04:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.665+0000 7ffa36b8c740 10 client.29596 fetch_fsmap learned FSMap version 66 2024-02-17T13:04:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.665+0000 7ffa36b8c740 1 -- 172.21.15.18:0/4026751460 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x5642bcf8b540 con 0x5642bd06c2e0 2024-02-17T13:04:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.666+0000 7ffa157fa640 1 -- 172.21.15.18:0/4026751460 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa24004890 con 0x5642bd06c2e0 2024-02-17T13:04:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.667+0000 7ffa157fa640 1 -- 172.21.15.18:0/4026751460 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 66) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffa2403c070 con 0x5642bd06c2e0 2024-02-17T13:04:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.667+0000 7ffa36b8c740 10 client.29596 fetch_fsmap finished waiting for FSMap version 66 2024-02-17T13:04:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.667+0000 7ffa36b8c740 10 client.29596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:04:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.667+0000 7ffa36b8c740 20 client.29596 populate_metadata read hostname 'smithi018' 2024-02-17T13:04:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.668+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/4026751460 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5642bd0d3ef0 0x5642bd0d6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:04:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.668+0000 7ffa36b8c740 4 client.29596 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:04:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.668+0000 7ffa36b8c740 1 -- 172.21.15.18:0/4026751460 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7ffa1c004470 con 0x5642bd0d3ef0 2024-02-17T13:04:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.668+0000 7ffa31c7b640 1 --2- 172.21.15.18:0/4026751460 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5642bd0d3ef0 0x5642bd0d6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:04:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.669+0000 7ffa157fa640 10 client.29596 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.669+0000 7ffa31c7b640 1 --2- 172.21.15.18:0/4026751460 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5642bd0d3ef0 0x5642bd0d6310 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3698134819d4261c server_cookie=9a2acb3e7662253f in_seq=0 out_seq=0 2024-02-17T13:04:50.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 79230 29.59 MB/sec execute 2 sec latency 244.780 ms 2024-02-17T13:04:50.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:50 smithi119 ceph-mon[36188]: pgmap v470: 97 pgs: 97 active+clean; 821 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 945 op/s 2024-02-17T13:04:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.925+0000 7ffa157fa640 1 -- 172.21.15.18:0/4026751460 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7ffa1c004470 con 0x5642bd0d3ef0 2024-02-17T13:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.925+0000 7ffa157fa640 10 client.29596 handle_command_reply: tid=0 2024-02-17T13:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.925+0000 7ffa36b8c740 1 client.29596 shutdown 2024-02-17T13:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.926+0000 7ffa36b8c740 1 -- 172.21.15.18:0/4026751460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 msgr2=0x5642bd06fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.926+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/4026751460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd06fc60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffa24040040 tx=0x7ffa24034de0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.926+0000 7ffa36b8c740 1 -- 172.21.15.18:0/4026751460 shutdown_connections 2024-02-17T13:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.926+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/4026751460 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5642bd0d3ef0 0x5642bd0d6310 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.926+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/4026751460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5642bd06c2e0 0x5642bd06fc60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.926+0000 7ffa157fa640 0 client.29596 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.926+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/4026751460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642bd063ad0 0x5642bd06f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.926+0000 7ffa36b8c740 1 --2- 172.21.15.18:0/4026751460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642bd062d60 0x5642bd073300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.926+0000 7ffa36b8c740 1 -- 172.21.15.18:0/4026751460 >> 172.21.15.18:0/4026751460 conn(0x5642bd02a550 msgr2=0x5642bd067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.926+0000 7ffa36b8c740 1 -- 172.21.15.18:0/4026751460 shutdown_connections 2024-02-17T13:04:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.926+0000 7ffa36b8c740 1 -- 172.21.15.18:0/4026751460 wait complete. 2024-02-17T13:04:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.926+0000 7ffa36b8c740 20 client.29596 trim_cache size 0 max 16384 2024-02-17T13:04:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:04:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:04:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:04:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.927+0000 7ffa33edf640 1 -- 172.21.15.18:0/1079039966 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffa180a2350 msgr2=0x7ffa180a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.927+0000 7ffa33edf640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffa180a2350 0x7ffa180a4810 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009370 tx=0x7ffa24009500 comp rx=0 tx=0).stop 2024-02-17T13:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.927+0000 7ffa33edf640 1 -- 172.21.15.18:0/1079039966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa2c1002d0 msgr2=0x7ffa2c19f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.927+0000 7ffa33edf640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa2c1002d0 0x7ffa2c19f630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800e9c0 tx=0x7ffa2800ee80 comp rx=0 tx=0).stop 2024-02-17T13:04:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.927+0000 7ffa33edf640 1 -- 172.21.15.18:0/1079039966 shutdown_connections 2024-02-17T13:04:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.927+0000 7ffa33edf640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7ffa180a2350 0x7ffa180a4810 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.927+0000 7ffa33edf640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa2c105020 0x7ffa2c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.927+0000 7ffa33edf640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa2c100c90 0x7ffa2c19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.927+0000 7ffa33edf640 1 --2- 172.21.15.18:0/1079039966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa2c1002d0 0x7ffa2c19f630 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:04:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.927+0000 7ffa33edf640 1 -- 172.21.15.18:0/1079039966 >> 172.21.15.18:0/1079039966 conn(0x7ffa2c076ed0 msgr2=0x7ffa2c068800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:04:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.927+0000 7ffa33edf640 1 -- 172.21.15.18:0/1079039966 shutdown_connections 2024-02-17T13:04:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:04:50.927+0000 7ffa33edf640 1 -- 172.21.15.18:0/1079039966 wait complete. 2024-02-17T13:04:50.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:50 smithi146 ceph-mon[36454]: pgmap v470: 97 pgs: 97 active+clean; 821 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 945 op/s 2024-02-17T13:04:51.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:50 smithi018 ceph-mon[32625]: pgmap v470: 97 pgs: 97 active+clean; 821 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 945 op/s 2024-02-17T13:04:51.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:51 smithi146 ceph-mon[36454]: daemon mds.i restarted 2024-02-17T13:04:51.698 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:04:51 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i[67657]: ignoring --setuser ceph since I am not root 2024-02-17T13:04:51.699 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:04:51 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i[67657]: ignoring --setgroup ceph since I am not root 2024-02-17T13:04:51.699 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:04:51 smithi146 ceph-mds[67661]: -- 172.21.15.146:0/3061168822 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555bc16cc1a0 con 0x555bc16a3000 2024-02-17T13:04:51.699 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:04:51 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i[67657]: starting mds.i at 2024-02-17T13:04:51.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 80207 41.08 MB/sec execute 3 sec latency 225.403 ms 2024-02-17T13:04:51.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:51 smithi119 ceph-mon[36188]: daemon mds.i restarted 2024-02-17T13:04:52.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:51 smithi018 ceph-mon[32625]: daemon mds.i restarted 2024-02-17T13:04:52.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 80977 40.29 MB/sec execute 4 sec latency 83.354 ms 2024-02-17T13:04:52.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:52 smithi119 ceph-mon[36188]: pgmap v471: 97 pgs: 97 active+clean; 833 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-02-17T13:04:52.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:52 smithi119 ceph-mon[36188]: osdmap e109: 12 total, 12 up, 12 in 2024-02-17T13:04:52.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:52 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.146:6838/3761238301,v1:172.21.15.146:6839/3761238301] up:boot 2024-02-17T13:04:52.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:52 smithi119 ceph-mon[36188]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-02-17T13:04:52.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:52 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T13:04:52.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:52 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:52.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:52 smithi146 ceph-mon[36454]: pgmap v471: 97 pgs: 97 active+clean; 833 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-02-17T13:04:52.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:52 smithi146 ceph-mon[36454]: osdmap e109: 12 total, 12 up, 12 in 2024-02-17T13:04:52.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:52 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.146:6838/3761238301,v1:172.21.15.146:6839/3761238301] up:boot 2024-02-17T13:04:52.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:52 smithi146 ceph-mon[36454]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-02-17T13:04:52.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:52 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T13:04:52.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:52 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:53.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:52 smithi018 ceph-mon[32625]: pgmap v471: 97 pgs: 97 active+clean; 833 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-02-17T13:04:53.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:52 smithi018 ceph-mon[32625]: osdmap e109: 12 total, 12 up, 12 in 2024-02-17T13:04:53.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:52 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.146:6838/3761238301,v1:172.21.15.146:6839/3761238301] up:boot 2024-02-17T13:04:53.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:52 smithi018 ceph-mon[32625]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-02-17T13:04:53.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:52 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T13:04:53.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:52 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:53.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 81666 39.16 MB/sec execute 5 sec latency 162.231 ms 2024-02-17T13:04:54.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 82289 37.91 MB/sec execute 6 sec latency 556.420 ms 2024-02-17T13:04:54.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:54 smithi119 ceph-mon[36188]: pgmap v473: 97 pgs: 97 active+clean; 833 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 864 op/s 2024-02-17T13:04:54.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:54 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:54.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:54 smithi146 ceph-mon[36454]: pgmap v473: 97 pgs: 97 active+clean; 833 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 864 op/s 2024-02-17T13:04:54.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:54 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:55.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:54 smithi018 ceph-mon[32625]: pgmap v473: 97 pgs: 97 active+clean; 833 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 864 op/s 2024-02-17T13:04:55.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:54 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:04:55.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 82941 37.23 MB/sec execute 7 sec latency 294.668 ms 2024-02-17T13:04:56.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 83970 40.22 MB/sec execute 8 sec latency 58.755 ms 2024-02-17T13:04:56.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:56 smithi146 ceph-mon[36454]: pgmap v474: 97 pgs: 97 active+clean; 845 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 956 op/s 2024-02-17T13:04:57.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:56 smithi018 ceph-mon[32625]: pgmap v474: 97 pgs: 97 active+clean; 845 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 956 op/s 2024-02-17T13:04:57.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:56 smithi119 ceph-mon[36188]: pgmap v474: 97 pgs: 97 active+clean; 845 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 956 op/s 2024-02-17T13:04:57.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 84829 41.47 MB/sec execute 9 sec latency 72.598 ms 2024-02-17T13:04:58.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 85424 39.54 MB/sec execute 10 sec latency 162.567 ms 2024-02-17T13:04:58.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:04:58 smithi146 ceph-mon[36454]: pgmap v475: 97 pgs: 97 active+clean; 845 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 956 op/s 2024-02-17T13:04:59.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:04:58 smithi018 ceph-mon[32625]: pgmap v475: 97 pgs: 97 active+clean; 845 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 956 op/s 2024-02-17T13:04:59.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:04:58 smithi119 ceph-mon[36188]: pgmap v475: 97 pgs: 97 active+clean; 845 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 956 op/s 2024-02-17T13:04:59.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 86228 39.75 MB/sec execute 11 sec latency 171.527 ms 2024-02-17T13:05:00.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 86898 39.67 MB/sec execute 12 sec latency 440.157 ms 2024-02-17T13:05:00.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:00 smithi146 ceph-mon[36454]: pgmap v476: 97 pgs: 97 active+clean; 853 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 911 op/s 2024-02-17T13:05:01.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:00 smithi018 ceph-mon[32625]: pgmap v476: 97 pgs: 97 active+clean; 853 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 911 op/s 2024-02-17T13:05:01.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:00 smithi119 ceph-mon[36188]: pgmap v476: 97 pgs: 97 active+clean; 853 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 911 op/s 2024-02-17T13:05:01.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 87517 39.54 MB/sec execute 13 sec latency 568.319 ms 2024-02-17T13:05:02.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88225 39.83 MB/sec execute 14 sec latency 86.545 ms 2024-02-17T13:05:02.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:02 smithi146 ceph-mon[36454]: pgmap v477: 97 pgs: 97 active+clean; 870 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 945 op/s 2024-02-17T13:05:03.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:02 smithi018 ceph-mon[32625]: pgmap v477: 97 pgs: 97 active+clean; 870 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 945 op/s 2024-02-17T13:05:03.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:02 smithi119 ceph-mon[36188]: pgmap v477: 97 pgs: 97 active+clean; 870 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 945 op/s 2024-02-17T13:05:03.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88969 39.53 MB/sec execute 15 sec latency 90.133 ms 2024-02-17T13:05:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.030+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/3459374049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ef41002d0 msgr2=0x7f8ef4100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.030+0000 7f8ef9cf4640 1 --2- 172.21.15.18:0/3459374049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ef41002d0 0x7f8ef4100750 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800d910 tx=0x7f8ee802dbb0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.031+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/3459374049 shutdown_connections 2024-02-17T13:05:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.031+0000 7f8ef9cf4640 1 --2- 172.21.15.18:0/3459374049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef4100c90 0x7f8ef410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.031+0000 7f8ef9cf4640 1 --2- 172.21.15.18:0/3459374049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ef41002d0 0x7f8ef4100750 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.031+0000 7f8ef9cf4640 1 --2- 172.21.15.18:0/3459374049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ef4105020 0x7f8ef4105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.031+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/3459374049 >> 172.21.15.18:0/3459374049 conn(0x7f8ef4076ed0 msgr2=0x7f8ef40772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.031+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/3459374049 shutdown_connections 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.031+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/3459374049 wait complete. 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.031+0000 7f8ef9cf4640 1 Processor -- start 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.031+0000 7f8ef9cf4640 1 -- start start 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef9cf4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef41002d0 0x7f8ef419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef9cf4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ef4100c90 0x7f8ef419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef9cf4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ef4105020 0x7f8ef4199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef9cf4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4199eb0 con 0x7f8ef4100c90 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef9cf4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef419a020 con 0x7f8ef41002d0 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef9cf4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef419a1e0 con 0x7f8ef4105020 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ef4100c90 0x7f8ef419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef41002d0 0x7f8ef419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ef4100c90 0x7f8ef419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49134/0 (socket says 172.21.15.18:49134) 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef2ffd640 1 -- 172.21.15.18:0/1713899205 learned_addr learned my addr 172.21.15.18:0/1713899205 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef3fff640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ef4105020 0x7f8ef4199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef2ffd640 1 -- 172.21.15.18:0/1713899205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ef4105020 msgr2=0x7f8ef4199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef2ffd640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ef4105020 0x7f8ef4199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef2ffd640 1 -- 172.21.15.18:0/1713899205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef41002d0 msgr2=0x7f8ef419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef2ffd640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef41002d0 0x7f8ef419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef2ffd640 1 -- 172.21.15.18:0/1713899205 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee800d570 con 0x7f8ef4100c90 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef3fff640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ef4105020 0x7f8ef4199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.032+0000 7f8ef2ffd640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ef4100c90 0x7f8ef419fbe0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8005ab0 tx=0x7f8ee8030cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.033+0000 7f8ef0ff9640 1 -- 172.21.15.18:0/1713899205 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8030ea0 con 0x7f8ef4100c90 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.033+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/1713899205 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef419a460 con 0x7f8ef4100c90 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.033+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/1713899205 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef4195510 con 0x7f8ef4100c90 2024-02-17T13:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.033+0000 7f8ef0ff9640 1 -- 172.21.15.18:0/1713899205 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ee802ebf0 con 0x7f8ef4100c90 2024-02-17T13:05:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.033+0000 7f8ef0ff9640 1 -- 172.21.15.18:0/1713899205 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8004de0 con 0x7f8ef4100c90 2024-02-17T13:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.036+0000 7f8efc9a2740 1 Processor -- start 2024-02-17T13:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.036+0000 7f8ef0ff9640 1 -- 172.21.15.18:0/1713899205 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8ee8005020 con 0x7f8ef4100c90 2024-02-17T13:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.036+0000 7f8efc9a2740 1 -- start start 2024-02-17T13:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.036+0000 7f8efc9a2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf0023ad0 0x55caf0022820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.036+0000 7f8efc9a2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 0x55caf00231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.036+0000 7f8efc9a2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf002c2e0 0x55caf002e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.036+0000 7f8efc9a2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caeff919f0 con 0x55caf0022d60 2024-02-17T13:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.036+0000 7f8efc9a2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caeff8ef20 con 0x55caf002c2e0 2024-02-17T13:05:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.036+0000 7f8efc9a2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caeff8a700 con 0x55caf0023ad0 2024-02-17T13:05:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.036+0000 7f8ef37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 0x55caf00231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef3fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf0023ad0 0x55caf0022820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 0x55caf00231e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49142/0 (socket says 172.21.15.18:49142) 2024-02-17T13:05:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef37fe640 1 -- 172.21.15.18:0/2269509631 learned_addr learned my addr 172.21.15.18:0/2269509631 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef2ffd640 1 --2- 172.21.15.18:0/2269509631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf002c2e0 0x55caf002e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef3fff640 1 -- 172.21.15.18:0/2269509631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf002c2e0 msgr2=0x55caf002e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef3fff640 1 --2- 172.21.15.18:0/2269509631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf002c2e0 0x55caf002e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef0ff9640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8edc0a2350 0x7f8edc0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef3fff640 1 -- 172.21.15.18:0/2269509631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 msgr2=0x55caf00231e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef3fff640 1 --2- 172.21.15.18:0/2269509631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 0x55caf00231e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef3fff640 1 -- 172.21.15.18:0/2269509631 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caeff207b0 con 0x55caf0023ad0 2024-02-17T13:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef37fe640 1 --2- 172.21.15.18:0/2269509631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 0x55caf00231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef0ff9640 1 -- 172.21.15.18:0/1713899205 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8ee80ef880 con 0x7f8ef4100c90 2024-02-17T13:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef3fff640 1 --2- 172.21.15.18:0/2269509631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf0023ad0 0x55caf0022820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009db0 tx=0x7f8eec009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf07499312c4e257 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ef3fff640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8edc0a2350 0x7f8edc0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ed9ffb640 1 -- 172.21.15.18:0/2269509631 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec019070 con 0x55caf0023ad0 2024-02-17T13:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ed9ffb640 1 -- 172.21.15.18:0/2269509631 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8eec01a040 con 0x55caf0023ad0 2024-02-17T13:05:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.037+0000 7f8ed9ffb640 1 -- 172.21.15.18:0/2269509631 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec02ab20 con 0x55caf0023ad0 2024-02-17T13:05:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.038+0000 7f8efc9a2740 1 -- 172.21.15.18:0/2269509631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf0023ad0 msgr2=0x55caf0022820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.038+0000 7f8ef3fff640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8edc0a2350 0x7f8edc0a4810 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00bfd0 tx=0x7f8eec01e790 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.038+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/2269509631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf0023ad0 0x55caf0022820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009db0 tx=0x7f8eec009500 comp rx=0 tx=0).stop 2024-02-17T13:05:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.038+0000 7f8efc9a2740 1 -- 172.21.15.18:0/2269509631 shutdown_connections 2024-02-17T13:05:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.038+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/2269509631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf002c2e0 0x55caf002e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.038+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/2269509631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 0x55caf00231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.038+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/2269509631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf0023ad0 0x55caf0022820 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.038+0000 7f8efc9a2740 1 -- 172.21.15.18:0/2269509631 >> 172.21.15.18:0/2269509631 conn(0x55caeffea550 msgr2=0x55caf001da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.039+0000 7f8efc9a2740 1 -- 172.21.15.18:0/2269509631 shutdown_connections 2024-02-17T13:05:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.039+0000 7f8efc9a2740 1 -- 172.21.15.18:0/2269509631 wait complete. 2024-02-17T13:05:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.039+0000 7f8efc9a2740 1 -- start start 2024-02-17T13:05:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.039+0000 7f8efc9a2740 1 Processor -- start 2024-02-17T13:05:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8efc9a2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf0022d60 0x55caf00292f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8efc9a2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0023ad0 0x55caf0029830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8efc9a2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf002c2e0 0x55caf0029d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8efc9a2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caeff2fbf0 con 0x55caf002c2e0 2024-02-17T13:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8efc9a2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caefc5e250 con 0x55caf0023ad0 2024-02-17T13:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8efc9a2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caeff10200 con 0x55caf0022d60 2024-02-17T13:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8efc9a2740 10 client.0 _ll_register_callbacks cb 0x55caeff7ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8ef3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf002c2e0 0x55caf0029d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8ef3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf002c2e0 0x55caf0029d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49144/0 (socket says 172.21.15.18:49144) 2024-02-17T13:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8ef3fff640 1 -- 172.21.15.18:0/3007406435 learned_addr learned my addr 172.21.15.18:0/3007406435 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8ef37fe640 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf0022d60 0x55caf00292f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8ef2ffd640 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0023ad0 0x55caf0029830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8ef3fff640 1 -- 172.21.15.18:0/3007406435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf0022d60 msgr2=0x55caf00292f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8ef3fff640 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf0022d60 0x55caf00292f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.040+0000 7f8ef3fff640 1 -- 172.21.15.18:0/3007406435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0023ad0 msgr2=0x55caf0029830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.041+0000 7f8ef3fff640 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0023ad0 0x55caf0029830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.041+0000 7f8ef3fff640 1 -- 172.21.15.18:0/3007406435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eec009a90 con 0x55caf002c2e0 2024-02-17T13:05:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.041+0000 7f8ef2ffd640 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0023ad0 0x55caf0029830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:05:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.041+0000 7f8ef3fff640 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf002c2e0 0x55caf0029d70 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f8eec01f690 tx=0x7f8eec024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c54b5758957f32d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.041+0000 7f8ed2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:05:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.042+0000 7f8ed2ffd640 1 -- 172.21.15.18:0/3007406435 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec01fdd0 con 0x55caf002c2e0 2024-02-17T13:05:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.042+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3007406435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55caefc8a600 con 0x55caf002c2e0 2024-02-17T13:05:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.042+0000 7f8ed2ffd640 1 -- 172.21.15.18:0/3007406435 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8eec01f770 con 0x55caf002c2e0 2024-02-17T13:05:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.042+0000 7f8ed2ffd640 1 -- 172.21.15.18:0/3007406435 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eec01ef00 con 0x55caf002c2e0 2024-02-17T13:05:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.042+0000 7f8efc9a2740 10 client.20685 fetch_fsmap learned FSMap version 69 2024-02-17T13:05:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.042+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3007406435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x55caeff88d80 con 0x55caf002c2e0 2024-02-17T13:05:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.043+0000 7f8ed2ffd640 1 -- 172.21.15.18:0/3007406435 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55caefc8a600 con 0x55caf002c2e0 2024-02-17T13:05:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.043+0000 7f8ed2ffd640 1 -- 172.21.15.18:0/3007406435 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 69) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f8eec005470 con 0x55caf002c2e0 2024-02-17T13:05:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.043+0000 7f8efc9a2740 10 client.20685 fetch_fsmap finished waiting for FSMap version 69 2024-02-17T13:05:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.043+0000 7f8efc9a2740 10 client.20685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:05:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.043+0000 7f8efc9a2740 20 client.20685 populate_metadata read hostname 'smithi018' 2024-02-17T13:05:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.043+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55caf003c6b0 0x55caf003ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.043+0000 7f8efc9a2740 4 client.20685 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:05:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.043+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3007406435 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55caefcdb1b0 con 0x55caf003c6b0 2024-02-17T13:05:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.044+0000 7f8ef37fe640 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55caf003c6b0 0x55caf003ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.045+0000 7f8ed2ffd640 10 client.20685 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.045+0000 7f8ef37fe640 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55caf003c6b0 0x55caf003ead0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a29804d6cc3e47 server_cookie=ef81f4babbcff538 in_seq=0 out_seq=0 2024-02-17T13:05:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8ed2ffd640 1 -- 172.21.15.18:0/3007406435 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55caefcdb1b0 con 0x55caf003c6b0 2024-02-17T13:05:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8ed2ffd640 10 client.20685 handle_command_reply: tid=0 2024-02-17T13:05:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8efc9a2740 1 client.20685 shutdown 2024-02-17T13:05:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3007406435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf002c2e0 msgr2=0x55caf0029d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf002c2e0 0x55caf0029d70 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f8eec01f690 tx=0x7f8eec024000 comp rx=0 tx=0).stop 2024-02-17T13:05:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3007406435 shutdown_connections 2024-02-17T13:05:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55caf003c6b0 0x55caf003ead0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf002c2e0 0x55caf0029d70 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0023ad0 0x55caf0029830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/3007406435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf0022d60 0x55caf00292f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3007406435 >> 172.21.15.18:0/3007406435 conn(0x55caeffea550 msgr2=0x55caf001f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8ed2ffd640 0 client.20685 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3007406435 shutdown_connections 2024-02-17T13:05:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3007406435 wait complete. 2024-02-17T13:05:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.047+0000 7f8efc9a2740 20 client.20685 trim_cache size 0 max 16384 2024-02-17T13:05:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.063+0000 7f8efc9a2740 1 Processor -- start 2024-02-17T13:05:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.063+0000 7f8efc9a2740 1 -- start start 2024-02-17T13:05:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.063+0000 7f8efc9a2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0022d60 0x55caf002ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.063+0000 7f8efc9a2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0023ad0 0x55caf001d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.063+0000 7f8efc9a2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf002c2e0 0x55caf001dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.063+0000 7f8efc9a2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf001d310 con 0x55caf0023ad0 2024-02-17T13:05:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.063+0000 7f8efc9a2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf0039150 con 0x55caf0022d60 2024-02-17T13:05:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.063+0000 7f8efc9a2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf0043ca0 con 0x55caf002c2e0 2024-02-17T13:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.063+0000 7f8ef2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0023ad0 0x55caf001d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ef2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0023ad0 0x55caf001d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49152/0 (socket says 172.21.15.18:49152) 2024-02-17T13:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ef2ffd640 1 -- 172.21.15.18:0/2100094509 learned_addr learned my addr 172.21.15.18:0/2100094509 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ef3fff640 1 --2- 172.21.15.18:0/2100094509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf002c2e0 0x55caf001dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ef37fe640 1 --2- 172.21.15.18:0/2100094509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0022d60 0x55caf002ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ef2ffd640 1 -- 172.21.15.18:0/2100094509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf002c2e0 msgr2=0x55caf001dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ef2ffd640 1 --2- 172.21.15.18:0/2100094509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf002c2e0 0x55caf001dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ef2ffd640 1 -- 172.21.15.18:0/2100094509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0022d60 msgr2=0x55caf002ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ef2ffd640 1 --2- 172.21.15.18:0/2100094509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0022d60 0x55caf002ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ef2ffd640 1 -- 172.21.15.18:0/2100094509 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caeff257b0 con 0x55caf0023ad0 2024-02-17T13:05:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ef2ffd640 1 --2- 172.21.15.18:0/2100094509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0023ad0 0x55caf001d580 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8000c00 tx=0x7f8ee80f5240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5e3e1eab0af583 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ed97fa640 1 -- 172.21.15.18:0/2100094509 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80f2ea0 con 0x55caf0023ad0 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ed97fa640 1 -- 172.21.15.18:0/2100094509 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ee80b8670 con 0x55caf0023ad0 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.064+0000 7f8ed97fa640 1 -- 172.21.15.18:0/2100094509 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80bc3d0 con 0x55caf0023ad0 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.065+0000 7f8efc9a2740 1 -- 172.21.15.18:0/2100094509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0023ad0 msgr2=0x55caf001d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.065+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/2100094509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0023ad0 0x55caf001d580 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8000c00 tx=0x7f8ee80f5240 comp rx=0 tx=0).stop 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.065+0000 7f8efc9a2740 1 -- 172.21.15.18:0/2100094509 shutdown_connections 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.065+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/2100094509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf002c2e0 0x55caf001dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.065+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/2100094509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0023ad0 0x55caf001d580 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.065+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/2100094509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0022d60 0x55caf002ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.065+0000 7f8efc9a2740 1 -- 172.21.15.18:0/2100094509 >> 172.21.15.18:0/2100094509 conn(0x55caeffea550 msgr2=0x55caf0024640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.065+0000 7f8efc9a2740 1 -- 172.21.15.18:0/2100094509 shutdown_connections 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.065+0000 7f8efc9a2740 1 -- 172.21.15.18:0/2100094509 wait complete. 2024-02-17T13:05:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8efc9a2740 1 -- start start 2024-02-17T13:05:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8efc9a2740 1 Processor -- start 2024-02-17T13:05:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8efc9a2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 0x55caf0033550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8efc9a2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf002c2e0 0x55caf0031850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8efc9a2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0031d90 0x55caf0032210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8efc9a2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf0092bf0 con 0x55caf0022d60 2024-02-17T13:05:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8efc9a2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf0092d60 con 0x55caf0031d90 2024-02-17T13:05:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8efc9a2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf0092f20 con 0x55caf002c2e0 2024-02-17T13:05:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8efc9a2740 10 client.0 _ll_register_callbacks cb 0x7f8eec0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:05:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8ef37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 0x55caf0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8ef37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 0x55caf0033550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49160/0 (socket says 172.21.15.18:49160) 2024-02-17T13:05:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8ef37fe640 1 -- 172.21.15.18:0/3343442822 learned_addr learned my addr 172.21.15.18:0/3343442822 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8ef2ffd640 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf002c2e0 0x55caf0031850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.066+0000 7f8ef3fff640 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0031d90 0x55caf0032210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8ef37fe640 1 -- 172.21.15.18:0/3343442822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf002c2e0 msgr2=0x55caf0031850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8ef37fe640 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf002c2e0 0x55caf0031850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8ef37fe640 1 -- 172.21.15.18:0/3343442822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0031d90 msgr2=0x55caf0032210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8ef37fe640 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0031d90 0x55caf0032210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8ef37fe640 1 -- 172.21.15.18:0/3343442822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef4195510 con 0x55caf0022d60 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8ef37fe640 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 0x55caf0033550 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009bc0 tx=0x7f8ee401f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c74b39ab1a4fd1e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8ed2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8ed2ffd640 1 -- 172.21.15.18:0/3343442822 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4004460 con 0x55caf0022d60 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8ed2ffd640 1 -- 172.21.15.18:0/3343442822 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8ee4010990 con 0x55caf0022d60 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3343442822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8eec01a040 con 0x55caf0022d60 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8ed2ffd640 1 -- 172.21.15.18:0/3343442822 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ee4010b70 con 0x55caf0022d60 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8efc9a2740 10 client.20691 fetch_fsmap learned FSMap version 69 2024-02-17T13:05:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3343442822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x55caeff4b540 con 0x55caf0022d60 2024-02-17T13:05:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.067+0000 7f8ed2ffd640 1 -- 172.21.15.18:0/3343442822 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8eec01a040 con 0x55caf0022d60 2024-02-17T13:05:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.068+0000 7f8ed2ffd640 1 -- 172.21.15.18:0/3343442822 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 69) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f8ee400f3e0 con 0x55caf0022d60 2024-02-17T13:05:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.068+0000 7f8efc9a2740 10 client.20691 fetch_fsmap finished waiting for FSMap version 69 2024-02-17T13:05:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.068+0000 7f8efc9a2740 10 client.20691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:05:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.068+0000 7f8efc9a2740 20 client.20691 populate_metadata read hostname 'smithi018' 2024-02-17T13:05:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.068+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55caf0094850 0x55caf0096c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.068+0000 7f8efc9a2740 4 client.20691 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:05:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.068+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3343442822 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55caefcc7550 con 0x55caf0094850 2024-02-17T13:05:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.068+0000 7f8ef2ffd640 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55caf0094850 0x55caf0096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.068+0000 7f8ef2ffd640 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55caf0094850 0x55caf0096c70 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6c77f290bc0193d server_cookie=bfd29d84397f27c1 in_seq=0 out_seq=0 2024-02-17T13:05:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.068+0000 7f8ed2ffd640 10 client.20691 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.283+0000 7f8ed2ffd640 1 -- 172.21.15.18:0/3343442822 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55caefcc7550 con 0x55caf0094850 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.283+0000 7f8ed2ffd640 10 client.20691 handle_command_reply: tid=0 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8efc9a2740 1 client.20691 shutdown 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3343442822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 msgr2=0x55caf0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 0x55caf0033550 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009bc0 tx=0x7f8ee401f770 comp rx=0 tx=0).stop 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3343442822 shutdown_connections 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55caf0094850 0x55caf0096c70 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55caf0031d90 0x55caf0032210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55caf002c2e0 0x55caf0031850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8efc9a2740 1 --2- 172.21.15.18:0/3343442822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55caf0022d60 0x55caf0033550 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8ed2ffd640 0 client.20691 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3343442822 >> 172.21.15.18:0/3343442822 conn(0x55caeffea550 msgr2=0x55caf001d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3343442822 shutdown_connections 2024-02-17T13:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8efc9a2740 1 -- 172.21.15.18:0/3343442822 wait complete. 2024-02-17T13:05:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8efc9a2740 20 client.20691 trim_cache size 0 max 16384 2024-02-17T13:05:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:05:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:05:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:05:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:05:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/1713899205 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8edc0a2350 msgr2=0x7f8edc0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8ef9cf4640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8edc0a2350 0x7f8edc0a4810 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00bfd0 tx=0x7f8eec01e790 comp rx=0 tx=0).stop 2024-02-17T13:05:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/1713899205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ef4100c90 msgr2=0x7f8ef419fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.284+0000 7f8ef9cf4640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ef4100c90 0x7f8ef419fbe0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8005ab0 tx=0x7f8ee8030cb0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.285+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/1713899205 shutdown_connections 2024-02-17T13:05:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.285+0000 7f8ef9cf4640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8edc0a2350 0x7f8edc0a4810 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.285+0000 7f8ef9cf4640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ef4105020 0x7f8ef4199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.285+0000 7f8ef9cf4640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ef4100c90 0x7f8ef419fbe0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.285+0000 7f8ef9cf4640 1 --2- 172.21.15.18:0/1713899205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef41002d0 0x7f8ef419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.285+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/1713899205 >> 172.21.15.18:0/1713899205 conn(0x7f8ef4076ed0 msgr2=0x7f8ef40fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.285+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/1713899205 shutdown_connections 2024-02-17T13:05:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:04.285+0000 7f8ef9cf4640 1 -- 172.21.15.18:0/1713899205 wait complete. 2024-02-17T13:05:04.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 89433 38.82 MB/sec execute 16 sec latency 157.074 ms 2024-02-17T13:05:04.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:04 smithi146 ceph-mon[36454]: pgmap v478: 97 pgs: 97 active+clean; 870 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 837 op/s 2024-02-17T13:05:05.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:04 smithi018 ceph-mon[32625]: pgmap v478: 97 pgs: 97 active+clean; 870 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 837 op/s 2024-02-17T13:05:05.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:04 smithi119 ceph-mon[36188]: pgmap v478: 97 pgs: 97 active+clean; 870 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 837 op/s 2024-02-17T13:05:05.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 89971 37.58 MB/sec execute 17 sec latency 316.319 ms 2024-02-17T13:05:06.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 90823 38.10 MB/sec execute 18 sec latency 790.798 ms 2024-02-17T13:05:06.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:06 smithi146 ceph-mon[36454]: pgmap v479: 97 pgs: 97 active+clean; 895 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 18 MiB/s wr, 1.23k op/s 2024-02-17T13:05:07.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:06 smithi018 ceph-mon[32625]: pgmap v479: 97 pgs: 97 active+clean; 895 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 18 MiB/s wr, 1.23k op/s 2024-02-17T13:05:07.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:06 smithi119 ceph-mon[36188]: pgmap v479: 97 pgs: 97 active+clean; 895 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 18 MiB/s wr, 1.23k op/s 2024-02-17T13:05:07.619 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:07 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:05:07.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 91788 39.42 MB/sec execute 19 sec latency 1790.831 ms 2024-02-17T13:05:08.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:07 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:05:08.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:07 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:05:08.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 92381 38.92 MB/sec execute 20 sec latency 2790.870 ms 2024-02-17T13:05:08.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:08 smithi146 ceph-mon[36454]: pgmap v480: 97 pgs: 97 active+clean; 895 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 800 op/s 2024-02-17T13:05:09.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:08 smithi018 ceph-mon[32625]: pgmap v480: 97 pgs: 97 active+clean; 895 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 800 op/s 2024-02-17T13:05:09.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:08 smithi119 ceph-mon[36188]: pgmap v480: 97 pgs: 97 active+clean; 895 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 800 op/s 2024-02-17T13:05:09.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 92875 38.19 MB/sec execute 21 sec latency 3790.938 ms 2024-02-17T13:05:10.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 93523 38.10 MB/sec execute 22 sec latency 4790.967 ms 2024-02-17T13:05:10.892 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:10 smithi119 ceph-mon[36188]: pgmap v481: 97 pgs: 97 active+clean; 907 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 946 op/s 2024-02-17T13:05:10.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:10 smithi146 ceph-mon[36454]: pgmap v481: 97 pgs: 97 active+clean; 907 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 946 op/s 2024-02-17T13:05:11.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:10 smithi018 ceph-mon[32625]: pgmap v481: 97 pgs: 97 active+clean; 907 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 946 op/s 2024-02-17T13:05:11.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 94143 37.95 MB/sec execute 23 sec latency 5790.999 ms 2024-02-17T13:05:12.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 95117 38.47 MB/sec execute 24 sec latency 6791.037 ms 2024-02-17T13:05:12.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:12 smithi146 ceph-mon[36454]: pgmap v482: 97 pgs: 97 active+clean; 912 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-02-17T13:05:13.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:12 smithi018 ceph-mon[32625]: pgmap v482: 97 pgs: 97 active+clean; 912 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-02-17T13:05:13.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:12 smithi119 ceph-mon[36188]: pgmap v482: 97 pgs: 97 active+clean; 912 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-02-17T13:05:13.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 95831 38.70 MB/sec execute 25 sec latency 7791.071 ms 2024-02-17T13:05:14.790 INFO:tasks.workunit.client.0.smithi018.stdout: 10 96392 38.20 MB/sec execute 26 sec latency 8791.102 ms 2024-02-17T13:05:14.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:14 smithi146 ceph-mon[36454]: pgmap v483: 97 pgs: 97 active+clean; 912 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 863 op/s 2024-02-17T13:05:15.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:14 smithi018 ceph-mon[32625]: pgmap v483: 97 pgs: 97 active+clean; 912 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 863 op/s 2024-02-17T13:05:15.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:14 smithi119 ceph-mon[36188]: pgmap v483: 97 pgs: 97 active+clean; 912 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 863 op/s 2024-02-17T13:05:15.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 97069 38.16 MB/sec execute 27 sec latency 9791.139 ms 2024-02-17T13:05:15.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:15 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:05:16.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:15 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:05:16.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:15 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:05:16.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 97588 37.76 MB/sec execute 28 sec latency 10791.179 ms 2024-02-17T13:05:16.922 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:16 smithi018 ceph-mon[32625]: pgmap v484: 97 pgs: 97 active+clean; 941 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-02-17T13:05:16.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:16 smithi146 ceph-mon[36454]: pgmap v484: 97 pgs: 97 active+clean; 941 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-02-17T13:05:17.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:16 smithi119 ceph-mon[36188]: pgmap v484: 97 pgs: 97 active+clean; 941 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-02-17T13:05:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.391+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3968164196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f90106500 msgr2=0x7f0f9010e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.391+0000 7f0f94c8d640 1 --2- 172.21.15.18:0/3968164196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f90106500 0x7f0f9010e060 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009ca0 tx=0x7f0f8802c100 comp rx=0 tx=0).stop 2024-02-17T13:05:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.391+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3968164196 shutdown_connections 2024-02-17T13:05:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.391+0000 7f0f94c8d640 1 --2- 172.21.15.18:0/3968164196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f90106500 0x7f0f9010e060 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.391+0000 7f0f94c8d640 1 --2- 172.21.15.18:0/3968164196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f901002a0 0x7f0f90105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.391+0000 7f0f94c8d640 1 --2- 172.21.15.18:0/3968164196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f900ff8f0 0x7f0f900ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.391+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3968164196 >> 172.21.15.18:0/3968164196 conn(0x7f0f90076610 msgr2=0x7f0f900fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.391+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3968164196 shutdown_connections 2024-02-17T13:05:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.391+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3968164196 wait complete. 2024-02-17T13:05:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f94c8d640 1 Processor -- start 2024-02-17T13:05:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f94c8d640 1 -- start start 2024-02-17T13:05:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f94c8d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f900ff8f0 0x7f0f90197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f94c8d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f90106500 0x7f0f90197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f94c8d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f9019c010 0x7f0f9019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f94c8d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9019ca00 con 0x7f0f900ff8f0 2024-02-17T13:05:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f94c8d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9019cb70 con 0x7f0f90106500 2024-02-17T13:05:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f94c8d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9019cce0 con 0x7f0f9019c010 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f8e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f900ff8f0 0x7f0f90197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f8e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f900ff8f0 0x7f0f90197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37540/0 (socket says 172.21.15.18:37540) 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f8e59c640 1 -- 172.21.15.18:0/3858023966 learned_addr learned my addr 172.21.15.18:0/3858023966 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f90106500 0x7f0f90197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f8ed9d640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f9019c010 0x7f0f9019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f8e59c640 1 -- 172.21.15.18:0/3858023966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f9019c010 msgr2=0x7f0f9019c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f8e59c640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f9019c010 0x7f0f9019c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f8e59c640 1 -- 172.21.15.18:0/3858023966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f90106500 msgr2=0x7f0f90197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f8e59c640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f90106500 0x7f0f90197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.392+0000 7f0f8e59c640 1 -- 172.21.15.18:0/3858023966 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f88009900 con 0x7f0f900ff8f0 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.393+0000 7f0f8e59c640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f900ff8f0 0x7f0f90197750 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000e9f0 tx=0x7f0f8000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.393+0000 7f0f7f7fe640 1 -- 172.21.15.18:0/3858023966 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80009720 con 0x7f0f900ff8f0 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.393+0000 7f0f7f7fe640 1 -- 172.21.15.18:0/3858023966 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f80005500 con 0x7f0f900ff8f0 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.393+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3858023966 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f901a42c0 con 0x7f0f900ff8f0 2024-02-17T13:05:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.393+0000 7f0f7f7fe640 1 -- 172.21.15.18:0/3858023966 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80010430 con 0x7f0f900ff8f0 2024-02-17T13:05:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.393+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3858023966 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f901a4720 con 0x7f0f900ff8f0 2024-02-17T13:05:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.396+0000 7f0f97859740 1 Processor -- start 2024-02-17T13:05:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.396+0000 7f0f7f7fe640 1 -- 172.21.15.18:0/3858023966 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0f80005020 con 0x7f0f900ff8f0 2024-02-17T13:05:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.396+0000 7f0f97859740 1 -- start start 2024-02-17T13:05:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f97859740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 0x560051285820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f97859740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560051285d60 0x5600512861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f97859740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 0x5600512916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f97859740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600511f49f0 con 0x560051286ad0 2024-02-17T13:05:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f97859740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600511f1f20 con 0x56005128f2e0 2024-02-17T13:05:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f97859740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600511ed700 con 0x560051285d60 2024-02-17T13:05:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f8e59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560051285d60 0x5600512861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f8dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 0x5600512916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f8ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 0x560051285820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f8dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 0x5600512916d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:56278/0 (socket says 172.21.15.18:56278) 2024-02-17T13:05:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f8ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 0x560051285820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37542/0 (socket says 172.21.15.18:37542) 2024-02-17T13:05:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f8dd9b640 1 -- 172.21.15.18:0/2730346096 learned_addr learned my addr 172.21.15.18:0/2730346096 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f7f7fe640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0f780a22d0 0x7f0f780a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f8dd9b640 1 -- 172.21.15.18:0/2730346096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560051285d60 msgr2=0x5600512861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/2730346096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560051285d60 0x5600512861e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f8dd9b640 1 -- 172.21.15.18:0/2730346096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 msgr2=0x560051285820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/2730346096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 0x560051285820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f8dd9b640 1 -- 172.21.15.18:0/2730346096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600511837b0 con 0x56005128f2e0 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.397+0000 7f0f7f7fe640 1 -- 172.21.15.18:0/3858023966 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0f80014070 con 0x7f0f900ff8f0 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.398+0000 7f0f8ed9d640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0f780a22d0 0x7f0f780a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.398+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/2730346096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 0x5600512916d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009a00 tx=0x7f0f84009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfcba38dc59256af server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.398+0000 7f0f7cff9640 1 -- 172.21.15.18:0/2730346096 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400eae0 con 0x56005128f2e0 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.398+0000 7f0f8ed9d640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0f780a22d0 0x7f0f780a4790 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f88004af0 tx=0x7f0f880095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.399+0000 7f0f7cff9640 1 -- 172.21.15.18:0/2730346096 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f8400ec40 con 0x56005128f2e0 2024-02-17T13:05:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.399+0000 7f0f7cff9640 1 -- 172.21.15.18:0/2730346096 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84012610 con 0x56005128f2e0 2024-02-17T13:05:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.399+0000 7f0f97859740 1 -- 172.21.15.18:0/2730346096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 msgr2=0x5600512916d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.399+0000 7f0f97859740 1 --2- 172.21.15.18:0/2730346096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 0x5600512916d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009a00 tx=0x7f0f84009ec0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.399+0000 7f0f97859740 1 -- 172.21.15.18:0/2730346096 shutdown_connections 2024-02-17T13:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.399+0000 7f0f97859740 1 --2- 172.21.15.18:0/2730346096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 0x5600512916d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.399+0000 7f0f97859740 1 --2- 172.21.15.18:0/2730346096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560051285d60 0x5600512861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.399+0000 7f0f97859740 1 --2- 172.21.15.18:0/2730346096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 0x560051285820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.399+0000 7f0f97859740 1 -- 172.21.15.18:0/2730346096 >> 172.21.15.18:0/2730346096 conn(0x56005124d550 msgr2=0x560051280a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.400+0000 7f0f97859740 1 -- 172.21.15.18:0/2730346096 shutdown_connections 2024-02-17T13:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.400+0000 7f0f97859740 1 -- 172.21.15.18:0/2730346096 wait complete. 2024-02-17T13:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.400+0000 7f0f97859740 1 -- start start 2024-02-17T13:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.400+0000 7f0f97859740 1 Processor -- start 2024-02-17T13:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f97859740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 0x56005128ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f97859740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560051286ad0 0x56005128c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f97859740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005128f2e0 0x56005128c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f8e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 0x56005128ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f8e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 0x56005128ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37550/0 (socket says 172.21.15.18:37550) 2024-02-17T13:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f8e59c640 1 -- 172.21.15.18:0/2658480153 learned_addr learned my addr 172.21.15.18:0/2658480153 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f97859740 1 -- 172.21.15.18:0/2658480153 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560051192bf0 con 0x560051285d60 2024-02-17T13:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f97859740 1 -- 172.21.15.18:0/2658480153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560050ec1250 con 0x560051286ad0 2024-02-17T13:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f97859740 1 -- 172.21.15.18:0/2658480153 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560051173200 con 0x56005128f2e0 2024-02-17T13:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560051286ad0 0x56005128c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f97859740 10 client.0 _ll_register_callbacks cb 0x5600511e1d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f8ed9d640 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005128f2e0 0x56005128c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f8e59c640 1 -- 172.21.15.18:0/2658480153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005128f2e0 msgr2=0x56005128c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f8e59c640 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005128f2e0 0x56005128c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f8e59c640 1 -- 172.21.15.18:0/2658480153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560051286ad0 msgr2=0x56005128c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f8e59c640 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560051286ad0 0x56005128c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.401+0000 7f0f8e59c640 1 -- 172.21.15.18:0/2658480153 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f84009630 con 0x560051285d60 2024-02-17T13:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f8e59c640 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 0x56005128ecc0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000c6a0 tx=0x7f0f800971a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166eade09ef87d0e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f75ffb640 1 -- 172.21.15.18:0/2658480153 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80005670 con 0x560051285d60 2024-02-17T13:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f75ffb640 1 -- 172.21.15.18:0/2658480153 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0f8009b580 con 0x560051285d60 2024-02-17T13:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f97859740 1 -- 172.21.15.18:0/2658480153 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560050eed600 con 0x560051285d60 2024-02-17T13:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f75ffb640 1 -- 172.21.15.18:0/2658480153 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f8009ae80 con 0x560051285d60 2024-02-17T13:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f97859740 10 client.20703 fetch_fsmap learned FSMap version 70 2024-02-17T13:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f97859740 1 -- 172.21.15.18:0/2658480153 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5600511ebd80 con 0x560051285d60 2024-02-17T13:05:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f75ffb640 1 -- 172.21.15.18:0/2658480153 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560050eed600 con 0x560051285d60 2024-02-17T13:05:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f75ffb640 1 -- 172.21.15.18:0/2658480153 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 70) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f0f8001f060 con 0x560051285d60 2024-02-17T13:05:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f97859740 10 client.20703 fetch_fsmap finished waiting for FSMap version 70 2024-02-17T13:05:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f97859740 10 client.20703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:05:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f97859740 20 client.20703 populate_metadata read hostname 'smithi018' 2024-02-17T13:05:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f97859740 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5600512a7e40 0x5600512aa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f97859740 4 client.20703 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:05:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.402+0000 7f0f97859740 1 -- 172.21.15.18:0/2658480153 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560050f3e1b0 con 0x5600512a7e40 2024-02-17T13:05:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.403+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5600512a7e40 0x5600512aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.404+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5600512a7e40 0x5600512aa260 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9754a5530f13a9c3 server_cookie=b8e3f6fc194b23af in_seq=0 out_seq=0 2024-02-17T13:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.404+0000 7f0f75ffb640 10 client.20703 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f75ffb640 1 -- 172.21.15.18:0/2658480153 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560050f3e1b0 con 0x5600512a7e40 2024-02-17T13:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f75ffb640 10 client.20703 handle_command_reply: tid=0 2024-02-17T13:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f97859740 1 client.20703 shutdown 2024-02-17T13:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f97859740 1 -- 172.21.15.18:0/2658480153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 msgr2=0x56005128ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f97859740 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 0x56005128ecc0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000c6a0 tx=0x7f0f800971a0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f97859740 1 -- 172.21.15.18:0/2658480153 shutdown_connections 2024-02-17T13:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f97859740 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5600512a7e40 0x5600512aa260 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f97859740 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005128f2e0 0x56005128c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f97859740 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560051286ad0 0x56005128c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f97859740 1 --2- 172.21.15.18:0/2658480153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 0x56005128ecc0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f75ffb640 0 client.20703 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f97859740 1 -- 172.21.15.18:0/2658480153 >> 172.21.15.18:0/2658480153 conn(0x56005124d550 msgr2=0x5600512827a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f97859740 1 -- 172.21.15.18:0/2658480153 shutdown_connections 2024-02-17T13:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f97859740 1 -- 172.21.15.18:0/2658480153 wait complete. 2024-02-17T13:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.406+0000 7f0f97859740 20 client.20703 trim_cache size 0 max 16384 2024-02-17T13:05:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f97859740 1 Processor -- start 2024-02-17T13:05:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f97859740 1 -- start start 2024-02-17T13:05:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f97859740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560051285d60 0x56005128c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f97859740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 0x56005128e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f97859740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005128f2e0 0x5600512906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f97859740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x560051281a50 con 0x560051286ad0 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f97859740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560051295f80 con 0x560051285d60 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f97859740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600512a7c30 con 0x56005128f2e0 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f8dd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 0x56005128e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f8dd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 0x56005128e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37556/0 (socket says 172.21.15.18:37556) 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f8dd9b640 1 -- 172.21.15.18:0/2769932358 learned_addr learned my addr 172.21.15.18:0/2769932358 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f8ed9d640 1 --2- 172.21.15.18:0/2769932358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005128f2e0 0x5600512906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f8e59c640 1 --2- 172.21.15.18:0/2769932358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560051285d60 0x56005128c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f8dd9b640 1 -- 172.21.15.18:0/2769932358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005128f2e0 msgr2=0x5600512906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/2769932358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005128f2e0 0x5600512906d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f8dd9b640 1 -- 172.21.15.18:0/2769932358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560051285d60 msgr2=0x56005128c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/2769932358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560051285d60 0x56005128c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.423+0000 7f0f8dd9b640 1 -- 172.21.15.18:0/2769932358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600511887b0 con 0x560051286ad0 2024-02-17T13:05:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.424+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/2769932358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 0x56005128e9e0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f0f84013530 tx=0x7f0f84008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616735f3f873e5b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.424+0000 7f0f77fff640 1 -- 172.21.15.18:0/2769932358 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84029070 con 0x560051286ad0 2024-02-17T13:05:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.424+0000 7f0f77fff640 1 -- 172.21.15.18:0/2769932358 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f84025880 con 0x560051286ad0 2024-02-17T13:05:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.424+0000 7f0f77fff640 1 -- 172.21.15.18:0/2769932358 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8401c650 con 0x560051286ad0 2024-02-17T13:05:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 -- 172.21.15.18:0/2769932358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 msgr2=0x56005128e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 --2- 172.21.15.18:0/2769932358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 0x56005128e9e0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f0f84013530 tx=0x7f0f84008040 comp rx=0 tx=0).stop 2024-02-17T13:05:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 -- 172.21.15.18:0/2769932358 shutdown_connections 2024-02-17T13:05:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 --2- 172.21.15.18:0/2769932358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005128f2e0 0x5600512906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 --2- 172.21.15.18:0/2769932358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051286ad0 0x56005128e9e0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 --2- 172.21.15.18:0/2769932358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560051285d60 0x56005128c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 -- 172.21.15.18:0/2769932358 >> 172.21.15.18:0/2769932358 conn(0x56005124d550 msgr2=0x56005128ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 -- 172.21.15.18:0/2769932358 shutdown_connections 2024-02-17T13:05:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 -- 172.21.15.18:0/2769932358 wait complete. 2024-02-17T13:05:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 -- start start 2024-02-17T13:05:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 Processor -- start 2024-02-17T13:05:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 0x5600512a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560051286ad0 0x56005129ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 0x56005129f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x56005129f800 con 0x560051285d60 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56005129f970 con 0x56005128f2e0 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x56005129fb30 con 0x560051286ad0 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f97859740 10 client.0 _ll_register_callbacks cb 0x7f0f84009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f8e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 0x5600512a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.425+0000 7f0f8e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 0x5600512a44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37560/0 (socket says 172.21.15.18:37560) 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f8e59c640 1 -- 172.21.15.18:0/1102474257 learned_addr learned my addr 172.21.15.18:0/1102474257 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560051286ad0 0x56005129ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f8ed9d640 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 0x56005129f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f8e59c640 1 -- 172.21.15.18:0/1102474257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560051286ad0 msgr2=0x56005129ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f8e59c640 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560051286ad0 0x56005129ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f8e59c640 1 -- 172.21.15.18:0/1102474257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 msgr2=0x56005129f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f8e59c640 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 0x56005129f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f8e59c640 1 -- 172.21.15.18:0/1102474257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600511837b0 con 0x560051285d60 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f8ed9d640 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 0x56005129f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f8e59c640 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 0x5600512a44f0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f0f80097850 tx=0x7f0f8000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b3cc19896bb95c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f75ffb640 1 -- 172.21.15.18:0/1102474257 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8009b580 con 0x560051285d60 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f75ffb640 1 -- 172.21.15.18:0/1102474257 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0f80005670 con 0x560051285d60 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f97859740 1 -- 172.21.15.18:0/1102474257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f8400ec40 con 0x560051285d60 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f75ffb640 1 -- 172.21.15.18:0/1102474257 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f8009ae80 con 0x560051285d60 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f97859740 10 client.20709 fetch_fsmap learned FSMap version 70 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.426+0000 7f0f97859740 1 -- 172.21.15.18:0/1102474257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5600511ae540 con 0x560051285d60 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.427+0000 7f0f75ffb640 1 -- 172.21.15.18:0/1102474257 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f8400ec40 con 0x560051285d60 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.427+0000 7f0f75ffb640 1 -- 172.21.15.18:0/1102474257 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 70) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f0f8001f040 con 0x560051285d60 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.427+0000 7f0f97859740 10 client.20709 fetch_fsmap finished waiting for FSMap version 70 2024-02-17T13:05:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.427+0000 7f0f97859740 10 client.20709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:05:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.427+0000 7f0f97859740 20 client.20709 populate_metadata read hostname 'smithi018' 2024-02-17T13:05:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.427+0000 7f0f97859740 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5600512ff510 0x560051301930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.427+0000 7f0f97859740 4 client.20709 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:05:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.427+0000 7f0f97859740 1 -- 172.21.15.18:0/1102474257 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560050f2a550 con 0x5600512ff510 2024-02-17T13:05:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.427+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5600512ff510 0x560051301930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.427+0000 7f0f8dd9b640 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5600512ff510 0x560051301930 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61e44e66201e8824 server_cookie=3fe4a7d679fc0c57 in_seq=0 out_seq=0 2024-02-17T13:05:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.427+0000 7f0f75ffb640 10 client.20709 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.664+0000 7f0f75ffb640 1 -- 172.21.15.18:0/1102474257 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560050f2a550 con 0x5600512ff510 2024-02-17T13:05:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.664+0000 7f0f75ffb640 10 client.20709 handle_command_reply: tid=0 2024-02-17T13:05:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.664+0000 7f0f97859740 1 client.20709 shutdown 2024-02-17T13:05:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.664+0000 7f0f97859740 1 -- 172.21.15.18:0/1102474257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 msgr2=0x5600512a44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.664+0000 7f0f97859740 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 0x5600512a44f0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f0f80097850 tx=0x7f0f8000cf50 comp rx=0 tx=0).stop 2024-02-17T13:05:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.664+0000 7f0f97859740 1 -- 172.21.15.18:0/1102474257 shutdown_connections 2024-02-17T13:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.665+0000 7f0f97859740 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5600512ff510 0x560051301930 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.665+0000 7f0f97859740 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005128f2e0 0x56005129f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.665+0000 7f0f75ffb640 0 client.20709 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.665+0000 7f0f97859740 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560051286ad0 0x56005129ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.665+0000 7f0f97859740 1 --2- 172.21.15.18:0/1102474257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560051285d60 0x5600512a44f0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.665+0000 7f0f97859740 1 -- 172.21.15.18:0/1102474257 >> 172.21.15.18:0/1102474257 conn(0x56005124d550 msgr2=0x5600512940f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.665+0000 7f0f97859740 1 -- 172.21.15.18:0/1102474257 shutdown_connections 2024-02-17T13:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.665+0000 7f0f97859740 1 -- 172.21.15.18:0/1102474257 wait complete. 2024-02-17T13:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.665+0000 7f0f97859740 20 client.20709 trim_cache size 0 max 16384 2024-02-17T13:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.666+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3858023966 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0f780a22d0 msgr2=0x7f0f780a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.666+0000 7f0f94c8d640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0f780a22d0 0x7f0f780a4790 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f88004af0 tx=0x7f0f880095c0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.666+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3858023966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f900ff8f0 msgr2=0x7f0f90197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.666+0000 7f0f94c8d640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f900ff8f0 0x7f0f90197750 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000e9f0 tx=0x7f0f8000edf0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.666+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3858023966 shutdown_connections 2024-02-17T13:05:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.666+0000 7f0f94c8d640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0f780a22d0 0x7f0f780a4790 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.666+0000 7f0f94c8d640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f9019c010 0x7f0f9019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.667+0000 7f0f94c8d640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f90106500 0x7f0f90197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.667+0000 7f0f94c8d640 1 --2- 172.21.15.18:0/3858023966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f900ff8f0 0x7f0f90197750 secure :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000e9f0 tx=0x7f0f8000edf0 comp rx=0 tx=0).stop 2024-02-17T13:05:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.667+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3858023966 >> 172.21.15.18:0/3858023966 conn(0x7f0f90076610 msgr2=0x7f0f90104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.667+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3858023966 shutdown_connections 2024-02-17T13:05:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:17.667+0000 7f0f94c8d640 1 -- 172.21.15.18:0/3858023966 wait complete. 2024-02-17T13:05:17.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98245 37.54 MB/sec execute 29 sec latency 11791.232 ms 2024-02-17T13:05:18.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98928 37.57 MB/sec execute 30 sec latency 12791.263 ms 2024-02-17T13:05:18.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:18 smithi146 ceph-mon[36454]: pgmap v485: 97 pgs: 97 active+clean; 941 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 770 op/s 2024-02-17T13:05:18.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:18 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:05:18.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:18 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:05:18.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:18 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:05:19.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:18 smithi018 ceph-mon[32625]: pgmap v485: 97 pgs: 97 active+clean; 941 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 770 op/s 2024-02-17T13:05:19.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:18 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:05:19.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:18 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:05:19.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:18 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:05:19.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:18 smithi119 ceph-mon[36188]: pgmap v485: 97 pgs: 97 active+clean; 941 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 770 op/s 2024-02-17T13:05:19.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:18 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:05:19.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:18 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:05:19.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:18 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:05:19.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 99873 38.11 MB/sec execute 31 sec latency 12975.761 ms 2024-02-17T13:05:20.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 100540 38.04 MB/sec execute 32 sec latency 77.769 ms 2024-02-17T13:05:20.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:20 smithi146 ceph-mon[36454]: pgmap v486: 97 pgs: 97 active+clean; 950 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 926 op/s 2024-02-17T13:05:21.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:20 smithi018 ceph-mon[32625]: pgmap v486: 97 pgs: 97 active+clean; 950 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 926 op/s 2024-02-17T13:05:21.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:20 smithi119 ceph-mon[36188]: pgmap v486: 97 pgs: 97 active+clean; 950 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 926 op/s 2024-02-17T13:05:21.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 101038 37.75 MB/sec execute 33 sec latency 231.372 ms 2024-02-17T13:05:22.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 101876 37.82 MB/sec execute 34 sec latency 52.618 ms 2024-02-17T13:05:22.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:22 smithi146 ceph-mon[36454]: pgmap v487: 97 pgs: 97 active+clean; 964 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 1.06k op/s 2024-02-17T13:05:23.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:22 smithi018 ceph-mon[32625]: pgmap v487: 97 pgs: 97 active+clean; 964 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 1.06k op/s 2024-02-17T13:05:23.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:22 smithi119 ceph-mon[36188]: pgmap v487: 97 pgs: 97 active+clean; 964 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 1.06k op/s 2024-02-17T13:05:23.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 102747 38.14 MB/sec execute 35 sec latency 50.080 ms 2024-02-17T13:05:24.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 103607 38.35 MB/sec execute 36 sec latency 77.537 ms 2024-02-17T13:05:24.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:24 smithi146 ceph-mon[36454]: pgmap v488: 97 pgs: 97 active+clean; 964 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 792 op/s 2024-02-17T13:05:25.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:24 smithi018 ceph-mon[32625]: pgmap v488: 97 pgs: 97 active+clean; 964 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 792 op/s 2024-02-17T13:05:25.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:24 smithi119 ceph-mon[36188]: pgmap v488: 97 pgs: 97 active+clean; 964 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 792 op/s 2024-02-17T13:05:25.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 104244 38.49 MB/sec execute 37 sec latency 72.932 ms 2024-02-17T13:05:26.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 104553 37.69 MB/sec execute 38 sec latency 318.872 ms 2024-02-17T13:05:26.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:26 smithi146 ceph-mon[36454]: pgmap v489: 97 pgs: 97 active+clean; 991 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 1.15k op/s 2024-02-17T13:05:27.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:26 smithi018 ceph-mon[32625]: pgmap v489: 97 pgs: 97 active+clean; 991 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 1.15k op/s 2024-02-17T13:05:27.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:26 smithi119 ceph-mon[36188]: pgmap v489: 97 pgs: 97 active+clean; 991 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 1.15k op/s 2024-02-17T13:05:27.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 105268 37.56 MB/sec execute 39 sec latency 93.816 ms 2024-02-17T13:05:28.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106264 37.97 MB/sec execute 40 sec latency 153.000 ms 2024-02-17T13:05:28.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:28 smithi146 ceph-mon[36454]: pgmap v490: 97 pgs: 97 active+clean; 991 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 796 op/s 2024-02-17T13:05:29.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:28 smithi018 ceph-mon[32625]: pgmap v490: 97 pgs: 97 active+clean; 991 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 796 op/s 2024-02-17T13:05:29.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:28 smithi119 ceph-mon[36188]: pgmap v490: 97 pgs: 97 active+clean; 991 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 796 op/s 2024-02-17T13:05:29.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 107121 38.47 MB/sec execute 41 sec latency 48.301 ms 2024-02-17T13:05:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.787+0000 7f03b3c64640 1 -- 172.21.15.18:0/1273982911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03ac1013e0 msgr2=0x7f03ac1017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.787+0000 7f03b3c64640 1 --2- 172.21.15.18:0/1273982911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03ac1013e0 0x7f03ac1017c0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009cd0 tx=0x7f03a802c0f0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.787+0000 7f03b3c64640 1 -- 172.21.15.18:0/1273982911 shutdown_connections 2024-02-17T13:05:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.787+0000 7f03b3c64640 1 --2- 172.21.15.18:0/1273982911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03ac1064b0 0x7f03ac10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.787+0000 7f03b3c64640 1 --2- 172.21.15.18:0/1273982911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ac101d90 0x7f03ac105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.787+0000 7f03b3c64640 1 --2- 172.21.15.18:0/1273982911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03ac1013e0 0x7f03ac1017c0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.787+0000 7f03b3c64640 1 -- 172.21.15.18:0/1273982911 >> 172.21.15.18:0/1273982911 conn(0x7f03ac0fcc40 msgr2=0x7f03ac0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.788+0000 7f03b3c64640 1 -- 172.21.15.18:0/1273982911 shutdown_connections 2024-02-17T13:05:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.788+0000 7f03b3c64640 1 -- 172.21.15.18:0/1273982911 wait complete. 2024-02-17T13:05:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.789+0000 7f03b3c64640 1 Processor -- start 2024-02-17T13:05:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.789+0000 7f03b3c64640 1 -- start start 2024-02-17T13:05:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.789+0000 7f03b3c64640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03ac1013e0 0x7f03ac070d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.789+0000 7f03b3c64640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ac1064b0 0x7f03ac0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.790+0000 7f03b3c64640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03ac074590 0x7f03ac074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.790+0000 7f03b1a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03ac1013e0 0x7f03ac070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.790+0000 7f03b3c64640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac074f80 con 0x7f03ac1013e0 2024-02-17T13:05:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.790+0000 7f03b1a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03ac1013e0 0x7f03ac070d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58374/0 (socket says 172.21.15.18:58374) 2024-02-17T13:05:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.790+0000 7f03b1a00640 1 -- 172.21.15.18:0/2440987633 learned_addr learned my addr 172.21.15.18:0/2440987633 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.790+0000 7f03b2201640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03ac074590 0x7f03ac074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.790+0000 7f03b3c64640 1 -- 172.21.15.18:0/2440987633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0750f0 con 0x7f03ac1064b0 2024-02-17T13:05:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.790+0000 7f03b3c64640 1 -- 172.21.15.18:0/2440987633 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac075260 con 0x7f03ac074590 2024-02-17T13:05:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.790+0000 7f03b11ff640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ac1064b0 0x7f03ac0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.791+0000 7f03b1a00640 1 -- 172.21.15.18:0/2440987633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03ac074590 msgr2=0x7f03ac074a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.791+0000 7f03b1a00640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03ac074590 0x7f03ac074a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.791+0000 7f03b1a00640 1 -- 172.21.15.18:0/2440987633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ac1064b0 msgr2=0x7f03ac0712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.791+0000 7f03b1a00640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ac1064b0 0x7f03ac0712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.791+0000 7f03b1a00640 1 -- 172.21.15.18:0/2440987633 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a8009900 con 0x7f03ac1013e0 2024-02-17T13:05:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.791+0000 7f03b11ff640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ac1064b0 0x7f03ac0712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:05:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.791+0000 7f03b2201640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03ac074590 0x7f03ac074a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:05:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.791+0000 7f03b1a00640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03ac1013e0 0x7f03ac070d80 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f03a8004af0 tx=0x7f03a8034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.791+0000 7f03a2ffd640 1 -- 172.21.15.18:0/2440987633 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a80328b0 con 0x7f03ac1013e0 2024-02-17T13:05:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.792+0000 7f03a2ffd640 1 -- 172.21.15.18:0/2440987633 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03a8003e50 con 0x7f03ac1013e0 2024-02-17T13:05:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.792+0000 7f03a2ffd640 1 -- 172.21.15.18:0/2440987633 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8004840 con 0x7f03ac1013e0 2024-02-17T13:05:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.792+0000 7f03b3c64640 1 -- 172.21.15.18:0/2440987633 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac1aed60 con 0x7f03ac1013e0 2024-02-17T13:05:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.792+0000 7f03b3c64640 1 -- 172.21.15.18:0/2440987633 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac1af0d0 con 0x7f03ac1013e0 2024-02-17T13:05:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.795+0000 7f03a2ffd640 1 -- 172.21.15.18:0/2440987633 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f03a800a040 con 0x7f03ac1013e0 2024-02-17T13:05:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.796+0000 7f03a2ffd640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03980a22d0 0x7f03980a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.796+0000 7f03b11ff640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03980a22d0 0x7f03980a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.796+0000 7f03a2ffd640 1 -- 172.21.15.18:0/2440987633 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f03a80bdc10 con 0x7f03ac1013e0 2024-02-17T13:05:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.797+0000 7f03b11ff640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03980a22d0 0x7f03980a4790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f03ac10e4d0 tx=0x7f039c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.798+0000 7f03b6838740 1 Processor -- start 2024-02-17T13:05:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.798+0000 7f03b6838740 1 -- start start 2024-02-17T13:05:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.798+0000 7f03b6838740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd282e0 0x5607abd27030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.798+0000 7f03b6838740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 0x5607abd279f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.798+0000 7f03b6838740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 0x5607abd32ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b6838740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607abc8d9f0 con 0x5607abd27570 2024-02-17T13:05:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b1a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd282e0 0x5607abd27030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b6838740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607abc8af20 con 0x5607abd30af0 2024-02-17T13:05:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b6838740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607abc86700 con 0x5607abd282e0 2024-02-17T13:05:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b1a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd282e0 0x5607abd27030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:37812/0 (socket says 172.21.15.18:37812) 2024-02-17T13:05:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b1a00640 1 -- 172.21.15.18:0/2182560929 learned_addr learned my addr 172.21.15.18:0/2182560929 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b11ff640 1 --2- 172.21.15.18:0/2182560929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 0x5607abd279f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b1a00640 1 -- 172.21.15.18:0/2182560929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 msgr2=0x5607abd32ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b1a00640 1 --2- 172.21.15.18:0/2182560929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 0x5607abd32ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b1a00640 1 -- 172.21.15.18:0/2182560929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 msgr2=0x5607abd279f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b1a00640 1 --2- 172.21.15.18:0/2182560929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 0x5607abd279f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b1a00640 1 -- 172.21.15.18:0/2182560929 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607abc1c7b0 con 0x5607abd282e0 2024-02-17T13:05:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b11ff640 1 --2- 172.21.15.18:0/2182560929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 0x5607abd279f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:05:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.799+0000 7f03b1a00640 1 --2- 172.21.15.18:0/2182560929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd282e0 0x5607abd27030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f03a8000f80 tx=0x7f03a80c2510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9419728fd536e40 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.800+0000 7f0397fff640 1 -- 172.21.15.18:0/2182560929 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a80bd2e0 con 0x5607abd282e0 2024-02-17T13:05:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.800+0000 7f0397fff640 1 -- 172.21.15.18:0/2182560929 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03a80bce70 con 0x5607abd282e0 2024-02-17T13:05:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.800+0000 7f0397fff640 1 -- 172.21.15.18:0/2182560929 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a80bd050 con 0x5607abd282e0 2024-02-17T13:05:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.801+0000 7f03b6838740 1 -- 172.21.15.18:0/2182560929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd282e0 msgr2=0x5607abd27030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.801+0000 7f03b6838740 1 --2- 172.21.15.18:0/2182560929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd282e0 0x5607abd27030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f03a8000f80 tx=0x7f03a80c2510 comp rx=0 tx=0).stop 2024-02-17T13:05:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.801+0000 7f03b6838740 1 -- 172.21.15.18:0/2182560929 shutdown_connections 2024-02-17T13:05:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.801+0000 7f03b6838740 1 --2- 172.21.15.18:0/2182560929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 0x5607abd32ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.801+0000 7f03b6838740 1 --2- 172.21.15.18:0/2182560929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 0x5607abd279f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.801+0000 7f03b6838740 1 --2- 172.21.15.18:0/2182560929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd282e0 0x5607abd27030 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.801+0000 7f03b6838740 1 -- 172.21.15.18:0/2182560929 >> 172.21.15.18:0/2182560929 conn(0x5607abce6550 msgr2=0x5607abd22260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.802+0000 7f03b6838740 1 -- 172.21.15.18:0/2182560929 shutdown_connections 2024-02-17T13:05:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.802+0000 7f03b6838740 1 -- 172.21.15.18:0/2182560929 wait complete. 2024-02-17T13:05:30.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 107748 38.34 MB/sec execute 42 sec latency 53.585 ms 2024-02-17T13:05:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.802+0000 7f03b6838740 1 -- start start 2024-02-17T13:05:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.802+0000 7f03b6838740 1 Processor -- start 2024-02-17T13:05:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b6838740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd27570 0x5607abd2db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b6838740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd282e0 0x5607abd2e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b6838740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 0x5607abd2e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b6838740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607abc2bbf0 con 0x5607abd282e0 2024-02-17T13:05:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b6838740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ab95a250 con 0x5607abd30af0 2024-02-17T13:05:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b6838740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607abc0c200 con 0x5607abd27570 2024-02-17T13:05:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b6838740 10 client.0 _ll_register_callbacks cb 0x5607abc7ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:05:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b11ff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd282e0 0x5607abd2e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b11ff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd282e0 0x5607abd2e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58398/0 (socket says 172.21.15.18:58398) 2024-02-17T13:05:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b11ff640 1 -- 172.21.15.18:0/1359907359 learned_addr learned my addr 172.21.15.18:0/1359907359 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b2201640 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 0x5607abd2e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b1a00640 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd27570 0x5607abd2db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b11ff640 1 -- 172.21.15.18:0/1359907359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd27570 msgr2=0x5607abd2db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b11ff640 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd27570 0x5607abd2db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.803+0000 7f03b11ff640 1 -- 172.21.15.18:0/1359907359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 msgr2=0x5607abd2e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.804+0000 7f03b11ff640 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 0x5607abd2e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.804+0000 7f03b11ff640 1 -- 172.21.15.18:0/1359907359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac1af0d0 con 0x5607abd282e0 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.804+0000 7f03b2201640 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 0x5607abd2e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.804+0000 7f03b11ff640 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd282e0 0x5607abd2e080 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f039c003920 tx=0x7f039c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c084437b54c6c884 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.804+0000 7f03957fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.804+0000 7f03957fa640 1 -- 172.21.15.18:0/1359907359 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c00fe10 con 0x5607abd282e0 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.804+0000 7f03b6838740 1 -- 172.21.15.18:0/1359907359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607ab986600 con 0x5607abd282e0 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.804+0000 7f03957fa640 1 -- 172.21.15.18:0/1359907359 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f039c028470 con 0x5607abd282e0 2024-02-17T13:05:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.804+0000 7f03957fa640 1 -- 172.21.15.18:0/1359907359 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f039c028630 con 0x5607abd282e0 2024-02-17T13:05:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.804+0000 7f03b6838740 10 client.20721 fetch_fsmap learned FSMap version 70 2024-02-17T13:05:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.804+0000 7f03b6838740 1 -- 172.21.15.18:0/1359907359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5607abc84d80 con 0x5607abd282e0 2024-02-17T13:05:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.805+0000 7f03957fa640 1 -- 172.21.15.18:0/1359907359 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5607ab986600 con 0x5607abd282e0 2024-02-17T13:05:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.805+0000 7f03957fa640 1 -- 172.21.15.18:0/1359907359 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 70) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f039c031870 con 0x5607abd282e0 2024-02-17T13:05:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.805+0000 7f03b6838740 10 client.20721 fetch_fsmap finished waiting for FSMap version 70 2024-02-17T13:05:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.805+0000 7f03b6838740 10 client.20721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:05:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.805+0000 7f03b6838740 20 client.20721 populate_metadata read hostname 'smithi018' 2024-02-17T13:05:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.805+0000 7f03b6838740 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607abd40cc0 0x5607abd430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.805+0000 7f03b6838740 4 client.20721 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:05:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.805+0000 7f03b6838740 1 -- 172.21.15.18:0/1359907359 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607ab9d71b0 con 0x5607abd40cc0 2024-02-17T13:05:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.807+0000 7f03b1a00640 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607abd40cc0 0x5607abd430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.807+0000 7f03957fa640 10 client.20721 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.807+0000 7f03b1a00640 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607abd40cc0 0x5607abd430e0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa258c246593ebc5 server_cookie=4eabcbc8a6f4aaf in_seq=0 out_seq=0 2024-02-17T13:05:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03957fa640 1 -- 172.21.15.18:0/1359907359 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5607ab9d71b0 con 0x5607abd40cc0 2024-02-17T13:05:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03957fa640 10 client.20721 handle_command_reply: tid=0 2024-02-17T13:05:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03b6838740 1 client.20721 shutdown 2024-02-17T13:05:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03b6838740 1 -- 172.21.15.18:0/1359907359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd282e0 msgr2=0x5607abd2e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03b6838740 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd282e0 0x5607abd2e080 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f039c003920 tx=0x7f039c004730 comp rx=0 tx=0).stop 2024-02-17T13:05:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03b6838740 1 -- 172.21.15.18:0/1359907359 shutdown_connections 2024-02-17T13:05:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03b6838740 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607abd40cc0 0x5607abd430e0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03b6838740 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 0x5607abd2e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03b6838740 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd282e0 0x5607abd2e080 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03b6838740 1 --2- 172.21.15.18:0/1359907359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd27570 0x5607abd2db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03957fa640 0 client.20721 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03b6838740 1 -- 172.21.15.18:0/1359907359 >> 172.21.15.18:0/1359907359 conn(0x5607abce6550 msgr2=0x5607abd23fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03b6838740 1 -- 172.21.15.18:0/1359907359 shutdown_connections 2024-02-17T13:05:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.813+0000 7f03b6838740 1 -- 172.21.15.18:0/1359907359 wait complete. 2024-02-17T13:05:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.814+0000 7f03b6838740 20 client.20721 trim_cache size 0 max 16384 2024-02-17T13:05:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.832+0000 7f03b6838740 1 Processor -- start 2024-02-17T13:05:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.832+0000 7f03b6838740 1 -- start start 2024-02-17T13:05:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b6838740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 0x5607abd30270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b6838740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd282e0 0x5607abd2c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b1a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 0x5607abd30270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b6838740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 0x5607abd2c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b6838740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607abd2ee90 con 0x5607abd27570 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b1a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 0x5607abd30270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58406/0 (socket says 172.21.15.18:58406) 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b6838740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607abd28a00 con 0x5607abd30af0 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b1a00640 1 -- 172.21.15.18:0/1805890242 learned_addr learned my addr 172.21.15.18:0/1805890242 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b6838740 1 -- 172.21.15.18:0/1805890242 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607abd2ce90 con 0x5607abd282e0 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b11ff640 1 --2- 172.21.15.18:0/1805890242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd282e0 0x5607abd2c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b2201640 1 --2- 172.21.15.18:0/1805890242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 0x5607abd2c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b1a00640 1 -- 172.21.15.18:0/1805890242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd282e0 msgr2=0x5607abd2c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b1a00640 1 --2- 172.21.15.18:0/1805890242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd282e0 0x5607abd2c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b1a00640 1 -- 172.21.15.18:0/1805890242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 msgr2=0x5607abd2c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b1a00640 1 --2- 172.21.15.18:0/1805890242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 0x5607abd2c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.833+0000 7f03b1a00640 1 -- 172.21.15.18:0/1805890242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607abc217b0 con 0x5607abd27570 2024-02-17T13:05:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.834+0000 7f03b1a00640 1 --2- 172.21.15.18:0/1805890242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 0x5607abd30270 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f03a8006010 tx=0x7f03a8004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91eaab4b437f878b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.834+0000 7f03977fe640 1 -- 172.21.15.18:0/1805890242 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a80c5cb0 con 0x5607abd27570 2024-02-17T13:05:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.834+0000 7f03977fe640 1 -- 172.21.15.18:0/1805890242 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03a80dd0b0 con 0x5607abd27570 2024-02-17T13:05:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.834+0000 7f03977fe640 1 -- 172.21.15.18:0/1805890242 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a80b95e0 con 0x5607abd27570 2024-02-17T13:05:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.835+0000 7f03b6838740 1 -- 172.21.15.18:0/1805890242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 msgr2=0x5607abd30270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.835+0000 7f03b6838740 1 --2- 172.21.15.18:0/1805890242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 0x5607abd30270 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f03a8006010 tx=0x7f03a8004370 comp rx=0 tx=0).stop 2024-02-17T13:05:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.835+0000 7f03b6838740 1 -- 172.21.15.18:0/1805890242 shutdown_connections 2024-02-17T13:05:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.835+0000 7f03b6838740 1 --2- 172.21.15.18:0/1805890242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd30af0 0x5607abd2c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.835+0000 7f03b6838740 1 --2- 172.21.15.18:0/1805890242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd282e0 0x5607abd2c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.835+0000 7f03b6838740 1 --2- 172.21.15.18:0/1805890242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd27570 0x5607abd30270 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.835+0000 7f03b6838740 1 -- 172.21.15.18:0/1805890242 >> 172.21.15.18:0/1805890242 conn(0x5607abce6550 msgr2=0x5607abd21d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.835+0000 7f03b6838740 1 -- 172.21.15.18:0/1805890242 shutdown_connections 2024-02-17T13:05:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.835+0000 7f03b6838740 1 -- 172.21.15.18:0/1805890242 wait complete. 2024-02-17T13:05:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.835+0000 7f03b6838740 1 -- start start 2024-02-17T13:05:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.835+0000 7f03b6838740 1 Processor -- start 2024-02-17T13:05:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b6838740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd27570 0x5607abd376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b6838740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd282e0 0x5607abd35c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b6838740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd30af0 0x5607abd361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b6838740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607abd37ff0 con 0x5607abd30af0 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b6838740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607abd36700 con 0x5607abd282e0 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b6838740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607abd368c0 con 0x5607abd27570 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b6838740 10 client.0 _ll_register_callbacks cb 0x7f03a80b9040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b11ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd282e0 0x5607abd35c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b1a00640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd27570 0x5607abd376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b11ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd282e0 0x5607abd35c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:46300/0 (socket says 172.21.15.18:46300) 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b11ff640 1 -- 172.21.15.18:0/3005771028 learned_addr learned my addr 172.21.15.18:0/3005771028 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b1a00640 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd27570 0x5607abd376f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:37834/0 (socket says 172.21.15.18:37834) 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b2201640 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd30af0 0x5607abd361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b2201640 1 -- 172.21.15.18:0/3005771028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd27570 msgr2=0x5607abd376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b2201640 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd27570 0x5607abd376f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b2201640 1 -- 172.21.15.18:0/3005771028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd282e0 msgr2=0x5607abd35c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b2201640 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd282e0 0x5607abd35c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b2201640 1 -- 172.21.15.18:0/3005771028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607abc1c7b0 con 0x5607abd30af0 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b11ff640 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd282e0 0x5607abd35c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.836+0000 7f03b1a00640 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd27570 0x5607abd376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.837+0000 7f03b2201640 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd30af0 0x5607abd361c0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f03a4005d80 tx=0x7f03a4012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7542f09d26b2441 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.837+0000 7f03957fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.837+0000 7f03957fa640 1 -- 172.21.15.18:0/3005771028 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4017070 con 0x5607abd30af0 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.837+0000 7f03957fa640 1 -- 172.21.15.18:0/3005771028 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f03a4010a90 con 0x5607abd30af0 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.837+0000 7f03b6838740 1 -- 172.21.15.18:0/3005771028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03a80bce70 con 0x5607abd30af0 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.837+0000 7f03957fa640 1 -- 172.21.15.18:0/3005771028 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03a4010c50 con 0x5607abd30af0 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.837+0000 7f03b6838740 10 client.20727 fetch_fsmap learned FSMap version 70 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.837+0000 7f03b6838740 1 -- 172.21.15.18:0/3005771028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5607abc47540 con 0x5607abd30af0 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.837+0000 7f03957fa640 1 -- 172.21.15.18:0/3005771028 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03a80bce70 con 0x5607abd30af0 2024-02-17T13:05:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.837+0000 7f03957fa640 1 -- 172.21.15.18:0/3005771028 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 70) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f03a401a370 con 0x5607abd30af0 2024-02-17T13:05:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.838+0000 7f03b6838740 10 client.20727 fetch_fsmap finished waiting for FSMap version 70 2024-02-17T13:05:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.838+0000 7f03b6838740 10 client.20727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:05:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.838+0000 7f03b6838740 20 client.20727 populate_metadata read hostname 'smithi018' 2024-02-17T13:05:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.838+0000 7f03b6838740 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607abd40cc0 0x5607abd9a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.838+0000 7f03b6838740 4 client.20727 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:05:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.838+0000 7f03b6838740 1 -- 172.21.15.18:0/3005771028 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5607ab9c3550 con 0x5607abd40cc0 2024-02-17T13:05:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.838+0000 7f03b1a00640 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607abd40cc0 0x5607abd9a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.838+0000 7f03b1a00640 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607abd40cc0 0x5607abd9a7e0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e41e7975890cf2a server_cookie=b09b13fae80ff9e6 in_seq=0 out_seq=0 2024-02-17T13:05:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:30.838+0000 7f03957fa640 10 client.20727 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:30.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:30 smithi146 ceph-mon[36454]: pgmap v491: 97 pgs: 97 active+clean; 1007 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 943 op/s 2024-02-17T13:05:31.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:30 smithi018 ceph-mon[32625]: pgmap v491: 97 pgs: 97 active+clean; 1007 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 943 op/s 2024-02-17T13:05:31.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:30 smithi119 ceph-mon[36188]: pgmap v491: 97 pgs: 97 active+clean; 1007 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 943 op/s 2024-02-17T13:05:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03957fa640 1 -- 172.21.15.18:0/3005771028 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5607ab9c3550 con 0x5607abd40cc0 2024-02-17T13:05:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03957fa640 10 client.20727 handle_command_reply: tid=0 2024-02-17T13:05:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03b6838740 1 client.20727 shutdown 2024-02-17T13:05:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03b6838740 1 -- 172.21.15.18:0/3005771028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd30af0 msgr2=0x5607abd361c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03b6838740 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd30af0 0x5607abd361c0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f03a4005d80 tx=0x7f03a4012b60 comp rx=0 tx=0).stop 2024-02-17T13:05:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03b6838740 1 -- 172.21.15.18:0/3005771028 shutdown_connections 2024-02-17T13:05:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03b6838740 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x5607abd40cc0 0x5607abd9a7e0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03b6838740 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5607abd30af0 0x5607abd361c0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03957fa640 0 client.20727 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03b6838740 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5607abd282e0 0x5607abd35c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03b6838740 1 --2- 172.21.15.18:0/3005771028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5607abd27570 0x5607abd376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03b6838740 1 -- 172.21.15.18:0/3005771028 >> 172.21.15.18:0/3005771028 conn(0x5607abce6550 msgr2=0x5607abd21d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03b6838740 1 -- 172.21.15.18:0/3005771028 shutdown_connections 2024-02-17T13:05:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.117+0000 7f03b6838740 1 -- 172.21.15.18:0/3005771028 wait complete. 2024-02-17T13:05:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.118+0000 7f03b6838740 20 client.20727 trim_cache size 0 max 16384 2024-02-17T13:05:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:05:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:05:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:05:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:05:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.119+0000 7f03b3c64640 1 -- 172.21.15.18:0/2440987633 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03980a22d0 msgr2=0x7f03980a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.119+0000 7f03b3c64640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03980a22d0 0x7f03980a4790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f03ac10e4d0 tx=0x7f039c005f90 comp rx=0 tx=0).stop 2024-02-17T13:05:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.119+0000 7f03b3c64640 1 -- 172.21.15.18:0/2440987633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03ac1013e0 msgr2=0x7f03ac070d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.119+0000 7f03b3c64640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03ac1013e0 0x7f03ac070d80 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f03a8004af0 tx=0x7f03a8034a90 comp rx=0 tx=0).stop 2024-02-17T13:05:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.119+0000 7f03b3c64640 1 -- 172.21.15.18:0/2440987633 shutdown_connections 2024-02-17T13:05:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.119+0000 7f03b3c64640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f03980a22d0 0x7f03980a4790 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.119+0000 7f03b3c64640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03ac074590 0x7f03ac074a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.119+0000 7f03b3c64640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ac1064b0 0x7f03ac0712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.119+0000 7f03b3c64640 1 --2- 172.21.15.18:0/2440987633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03ac1013e0 0x7f03ac070d80 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.119+0000 7f03b3c64640 1 -- 172.21.15.18:0/2440987633 >> 172.21.15.18:0/2440987633 conn(0x7f03ac0fcc40 msgr2=0x7f03ac0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.120+0000 7f03b3c64640 1 -- 172.21.15.18:0/2440987633 shutdown_connections 2024-02-17T13:05:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:31.120+0000 7f03b3c64640 1 -- 172.21.15.18:0/2440987633 wait complete. 2024-02-17T13:05:31.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 108223 37.94 MB/sec execute 43 sec latency 218.598 ms 2024-02-17T13:05:32.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 108984 38.02 MB/sec execute 44 sec latency 235.679 ms 2024-02-17T13:05:32.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:32 smithi146 ceph-mon[36454]: pgmap v492: 97 pgs: 97 active+clean; 1015 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 966 op/s 2024-02-17T13:05:33.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:32 smithi018 ceph-mon[32625]: pgmap v492: 97 pgs: 97 active+clean; 1015 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 966 op/s 2024-02-17T13:05:33.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:32 smithi119 ceph-mon[36188]: pgmap v492: 97 pgs: 97 active+clean; 1015 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 966 op/s 2024-02-17T13:05:33.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 109812 37.92 MB/sec execute 45 sec latency 196.720 ms 2024-02-17T13:05:34.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 110823 38.51 MB/sec execute 46 sec latency 91.681 ms 2024-02-17T13:05:34.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:34 smithi146 ceph-mon[36454]: pgmap v493: 97 pgs: 97 active+clean; 1015 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 682 op/s 2024-02-17T13:05:35.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:34 smithi018 ceph-mon[32625]: pgmap v493: 97 pgs: 97 active+clean; 1015 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 682 op/s 2024-02-17T13:05:35.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:34 smithi119 ceph-mon[36188]: pgmap v493: 97 pgs: 97 active+clean; 1015 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 682 op/s 2024-02-17T13:05:35.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 111558 38.63 MB/sec execute 47 sec latency 71.507 ms 2024-02-17T13:05:36.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 112006 38.32 MB/sec execute 48 sec latency 230.172 ms 2024-02-17T13:05:36.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:36 smithi146 ceph-mon[36454]: pgmap v494: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-02-17T13:05:37.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:36 smithi018 ceph-mon[32625]: pgmap v494: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-02-17T13:05:37.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:36 smithi119 ceph-mon[36188]: pgmap v494: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-02-17T13:05:37.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 112797 38.34 MB/sec execute 49 sec latency 154.763 ms 2024-02-17T13:05:38.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 113575 38.28 MB/sec execute 50 sec latency 98.511 ms 2024-02-17T13:05:38.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:38 smithi146 ceph-mon[36454]: pgmap v495: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 749 op/s 2024-02-17T13:05:39.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:38 smithi018 ceph-mon[32625]: pgmap v495: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 749 op/s 2024-02-17T13:05:39.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:38 smithi119 ceph-mon[36188]: pgmap v495: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 749 op/s 2024-02-17T13:05:39.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 114450 38.47 MB/sec execute 51 sec latency 61.228 ms 2024-02-17T13:05:40.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 115320 38.78 MB/sec execute 52 sec latency 66.689 ms 2024-02-17T13:05:40.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:40 smithi146 ceph-mon[36454]: pgmap v496: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 890 op/s 2024-02-17T13:05:41.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:40 smithi018 ceph-mon[32625]: pgmap v496: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 890 op/s 2024-02-17T13:05:41.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:40 smithi119 ceph-mon[36188]: pgmap v496: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 890 op/s 2024-02-17T13:05:41.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 115745 38.45 MB/sec execute 53 sec latency 282.405 ms 2024-02-17T13:05:42.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 116465 38.51 MB/sec execute 54 sec latency 73.644 ms 2024-02-17T13:05:42.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:42 smithi146 ceph-mon[36454]: pgmap v497: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 941 op/s 2024-02-17T13:05:43.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:42 smithi018 ceph-mon[32625]: pgmap v497: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 941 op/s 2024-02-17T13:05:43.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:42 smithi119 ceph-mon[36188]: pgmap v497: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 941 op/s 2024-02-17T13:05:43.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 117214 38.41 MB/sec execute 55 sec latency 241.603 ms 2024-02-17T13:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.230+0000 7efce5f0d640 1 -- 172.21.15.18:0/2809346330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efce00ff790 msgr2=0x7efce00ffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.230+0000 7efce5f0d640 1 --2- 172.21.15.18:0/2809346330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efce00ff790 0x7efce00ffc10 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009900 tx=0x7efcd402bc30 comp rx=0 tx=0).stop 2024-02-17T13:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.230+0000 7efce5f0d640 1 -- 172.21.15.18:0/2809346330 shutdown_connections 2024-02-17T13:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.230+0000 7efce5f0d640 1 --2- 172.21.15.18:0/2809346330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce0100150 0x7efce010c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.230+0000 7efce5f0d640 1 --2- 172.21.15.18:0/2809346330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efce00ff790 0x7efce00ffc10 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.230+0000 7efce5f0d640 1 --2- 172.21.15.18:0/2809346330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce01057a0 0x7efce0105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.230+0000 7efce5f0d640 1 -- 172.21.15.18:0/2809346330 >> 172.21.15.18:0/2809346330 conn(0x7efce0076610 msgr2=0x7efce00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.230+0000 7efce5f0d640 1 -- 172.21.15.18:0/2809346330 shutdown_connections 2024-02-17T13:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.230+0000 7efce5f0d640 1 -- 172.21.15.18:0/2809346330 wait complete. 2024-02-17T13:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.230+0000 7efce5f0d640 1 Processor -- start 2024-02-17T13:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efce5f0d640 1 -- start start 2024-02-17T13:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efce5f0d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efce00ff790 0x7efce0197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efce5f0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce0100150 0x7efce0197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efce5f0d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce01057a0 0x7efce019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efce5f0d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce019c5d0 con 0x7efce00ff790 2024-02-17T13:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efce5f0d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce019c740 con 0x7efce0100150 2024-02-17T13:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efce5f0d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce019c900 con 0x7efce01057a0 2024-02-17T13:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efcdf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efce00ff790 0x7efce0197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efcdf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efce00ff790 0x7efce0197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57292/0 (socket says 172.21.15.18:57292) 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efcdf7fe640 1 -- 172.21.15.18:0/3323975879 learned_addr learned my addr 172.21.15.18:0/3323975879 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efcdeffd640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce0100150 0x7efce0197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efcdffff640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce01057a0 0x7efce019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efcdf7fe640 1 -- 172.21.15.18:0/3323975879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce01057a0 msgr2=0x7efce019bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce01057a0 0x7efce019bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efcdf7fe640 1 -- 172.21.15.18:0/3323975879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce0100150 msgr2=0x7efce0197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce0100150 0x7efce0197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.231+0000 7efcdf7fe640 1 -- 172.21.15.18:0/3323975879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd4009560 con 0x7efce00ff790 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.232+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efce00ff790 0x7efce0197710 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7efcd000e9f0 tx=0x7efcd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.232+0000 7efcdcff9640 1 -- 172.21.15.18:0/3323975879 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0009720 con 0x7efce00ff790 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.232+0000 7efce5f0d640 1 -- 172.21.15.18:0/3323975879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce019cbe0 con 0x7efce00ff790 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.232+0000 7efce5f0d640 1 -- 172.21.15.18:0/3323975879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce01a4460 con 0x7efce00ff790 2024-02-17T13:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.232+0000 7efcdcff9640 1 -- 172.21.15.18:0/3323975879 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcd000cd80 con 0x7efce00ff790 2024-02-17T13:05:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.232+0000 7efcdcff9640 1 -- 172.21.15.18:0/3323975879 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0010690 con 0x7efce00ff790 2024-02-17T13:05:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efcdcff9640 1 -- 172.21.15.18:0/3323975879 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7efcd00046c0 con 0x7efce00ff790 2024-02-17T13:05:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efce8bbb740 1 Processor -- start 2024-02-17T13:05:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efce8bbb740 1 -- start start 2024-02-17T13:05:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efce8bbb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f363238ad0 0x55f363237820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efce8bbb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363237d60 0x55f3632381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efce8bbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3632412e0 0x55f3632436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efce8bbb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3631a69f0 con 0x55f363238ad0 2024-02-17T13:05:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efce8bbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3631a3f20 con 0x55f3632412e0 2024-02-17T13:05:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efce8bbb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36319f700 con 0x55f363237d60 2024-02-17T13:05:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efcdcff9640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efcc80a22d0 0x7efcc80a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efcdffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f363238ad0 0x55f363237820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efcdffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f363238ad0 0x55f363237820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57306/0 (socket says 172.21.15.18:57306) 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efcdffff640 1 -- 172.21.15.18:0/1594616007 learned_addr learned my addr 172.21.15.18:0/1594616007 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.235+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/1594616007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363237d60 0x55f3632381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcdcff9640 1 -- 172.21.15.18:0/3323975879 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7efcd0014070 con 0x7efce00ff790 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcdffff640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efcc80a22d0 0x7efcc80a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcdeffd640 1 --2- 172.21.15.18:0/1594616007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3632412e0 0x55f3632436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcdf7fe640 1 -- 172.21.15.18:0/1594616007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3632412e0 msgr2=0x55f3632436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/1594616007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3632412e0 0x55f3632436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcdf7fe640 1 -- 172.21.15.18:0/1594616007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f363238ad0 msgr2=0x55f363237820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/1594616007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f363238ad0 0x55f363237820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcdf7fe640 1 -- 172.21.15.18:0/1594616007 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3631357b0 con 0x55f363237d60 2024-02-17T13:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcdffff640 1 --2- 172.21.15.18:0/1594616007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f363238ad0 0x55f363237820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:05:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/1594616007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363237d60 0x55f3632381e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efcd00cbb70 tx=0x7efcd007a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db86eea781242496 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcdffff640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efcc80a22d0 0x7efcc80a4790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009fd0 tx=0x7efcd8004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcc5ffb640 1 -- 172.21.15.18:0/1594616007 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd009b510 con 0x55f363237d60 2024-02-17T13:05:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcc5ffb640 1 -- 172.21.15.18:0/1594616007 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcd009d040 con 0x55f363237d60 2024-02-17T13:05:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.236+0000 7efcc5ffb640 1 -- 172.21.15.18:0/1594616007 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0097480 con 0x55f363237d60 2024-02-17T13:05:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.237+0000 7efce8bbb740 1 -- 172.21.15.18:0/1594616007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363237d60 msgr2=0x55f3632381e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.237+0000 7efce8bbb740 1 --2- 172.21.15.18:0/1594616007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363237d60 0x55f3632381e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efcd00cbb70 tx=0x7efcd007a660 comp rx=0 tx=0).stop 2024-02-17T13:05:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.237+0000 7efce8bbb740 1 -- 172.21.15.18:0/1594616007 shutdown_connections 2024-02-17T13:05:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.237+0000 7efce8bbb740 1 --2- 172.21.15.18:0/1594616007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3632412e0 0x55f3632436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.237+0000 7efce8bbb740 1 --2- 172.21.15.18:0/1594616007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363237d60 0x55f3632381e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.237+0000 7efce8bbb740 1 --2- 172.21.15.18:0/1594616007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f363238ad0 0x55f363237820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.237+0000 7efce8bbb740 1 -- 172.21.15.18:0/1594616007 >> 172.21.15.18:0/1594616007 conn(0x55f3631ff550 msgr2=0x55f363232a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.238+0000 7efce8bbb740 1 -- 172.21.15.18:0/1594616007 shutdown_connections 2024-02-17T13:05:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.238+0000 7efce8bbb740 1 -- 172.21.15.18:0/1594616007 wait complete. 2024-02-17T13:05:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.238+0000 7efce8bbb740 1 -- start start 2024-02-17T13:05:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.238+0000 7efce8bbb740 1 Processor -- start 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efce8bbb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363237d60 0x55f36323e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efce8bbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363238ad0 0x55f36323e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efce8bbb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f36323ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efce8bbb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f363144bf0 con 0x55f3632412e0 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efce8bbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f362e73250 con 0x55f363238ad0 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efce8bbb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f363125200 con 0x55f363237d60 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efce8bbb740 10 client.0 _ll_register_callbacks cb 0x55f363193d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efcdffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f36323ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efcdf7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363237d60 0x55f36323e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efcdffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f36323ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57322/0 (socket says 172.21.15.18:57322) 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efcdffff640 1 -- 172.21.15.18:0/409179692 learned_addr learned my addr 172.21.15.18:0/409179692 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efcdeffd640 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363238ad0 0x55f36323e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efcdf7fe640 1 -- 172.21.15.18:0/409179692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363238ad0 msgr2=0x55f36323e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363238ad0 0x55f36323e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efcdf7fe640 1 -- 172.21.15.18:0/409179692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 msgr2=0x55f36323ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f36323ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efcdf7fe640 1 -- 172.21.15.18:0/409179692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce01a4460 con 0x55f363237d60 2024-02-17T13:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efcdeffd640 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363238ad0 0x55f36323e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efcdffff640 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f36323ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.239+0000 7efce8bbb740 1 -- 172.21.15.18:0/409179692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f362e9f600 con 0x55f363237d60 2024-02-17T13:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.240+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363237d60 0x55f36323e2b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efcd009f120 tx=0x7efcd009ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ec5a18166c851ed server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.240+0000 7efcbeffd640 10 client.29665 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.240+0000 7efcbeffd640 1 -- 172.21.15.18:0/409179692 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd009ae10 con 0x55f363237d60 2024-02-17T13:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.240+0000 7efcbeffd640 1 -- 172.21.15.18:0/409179692 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7efcd00a35b0 con 0x55f363237d60 2024-02-17T13:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.240+0000 7efcbeffd640 1 -- 172.21.15.18:0/409179692 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcd00b9300 con 0x55f363237d60 2024-02-17T13:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.240+0000 7efce8bbb740 10 client.29665 fetch_fsmap learned FSMap version 70 2024-02-17T13:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.240+0000 7efce8bbb740 1 -- 172.21.15.18:0/409179692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({fsmap=70}) v3 -- 0x55f36319dd80 con 0x55f363237d60 2024-02-17T13:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.241+0000 7efcbeffd640 1 -- 172.21.15.18:0/409179692 <== mon.1 v2:172.21.15.146:3300/0 4 ==== fsmap(e 70) v1 ==== 7946+0+0 (secure 0 0 0) 0x7efcd001f060 con 0x55f363237d60 2024-02-17T13:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.241+0000 7efce8bbb740 10 client.29665 fetch_fsmap finished waiting for FSMap version 70 2024-02-17T13:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.241+0000 7efce8bbb740 10 client.29665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.241+0000 7efce8bbb740 20 client.29665 populate_metadata read hostname 'smithi018' 2024-02-17T13:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.241+0000 7efce8bbb740 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f3632511c0 0x55f363253620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.241+0000 7efce8bbb740 4 client.29665 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.241+0000 7efce8bbb740 1 -- 172.21.15.18:0/409179692 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f362ef01b0 con 0x55f3632511c0 2024-02-17T13:05:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.242+0000 7efcdeffd640 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f3632511c0 0x55f363253620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.243+0000 7efcdeffd640 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f3632511c0 0x55f363253620 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c7d2b212942b630 server_cookie=959e02551ae09095 in_seq=0 out_seq=0 2024-02-17T13:05:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.243+0000 7efcbeffd640 10 client.29665 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efcbeffd640 1 -- 172.21.15.18:0/409179692 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f362ef01b0 con 0x55f3632511c0 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efcbeffd640 10 client.29665 handle_command_reply: tid=0 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efce8bbb740 1 client.29665 shutdown 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efce8bbb740 1 -- 172.21.15.18:0/409179692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363237d60 msgr2=0x55f36323e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efce8bbb740 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363237d60 0x55f36323e2b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efcd009f120 tx=0x7efcd009ef60 comp rx=0 tx=0).stop 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efce8bbb740 1 -- 172.21.15.18:0/409179692 shutdown_connections 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efce8bbb740 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f3632511c0 0x55f363253620 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efce8bbb740 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f36323ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efcbeffd640 0 client.29665 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efce8bbb740 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363238ad0 0x55f36323e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efce8bbb740 1 --2- 172.21.15.18:0/409179692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363237d60 0x55f36323e2b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efce8bbb740 1 -- 172.21.15.18:0/409179692 >> 172.21.15.18:0/409179692 conn(0x55f3631ff550 msgr2=0x55f3632347a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efce8bbb740 1 -- 172.21.15.18:0/409179692 shutdown_connections 2024-02-17T13:05:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efce8bbb740 1 -- 172.21.15.18:0/409179692 wait complete. 2024-02-17T13:05:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.247+0000 7efce8bbb740 20 client.29665 trim_cache size 0 max 16384 2024-02-17T13:05:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efce8bbb740 1 Processor -- start 2024-02-17T13:05:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efce8bbb740 1 -- start start 2024-02-17T13:05:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efce8bbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363237d60 0x55f3632409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efce8bbb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363238ad0 0x55f363248110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efce8bbb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f363248650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efce8bbb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36322e4f0 con 0x55f3632412e0 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efce8bbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36322e310 con 0x55f363237d60 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efce8bbb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f363233c80 con 0x55f363238ad0 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efcdffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f363248650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efcdffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f363248650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57324/0 (socket says 172.21.15.18:57324) 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efcdffff640 1 -- 172.21.15.18:0/448559011 learned_addr learned my addr 172.21.15.18:0/448559011 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efcdeffd640 1 --2- 172.21.15.18:0/448559011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363238ad0 0x55f363248110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/448559011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363237d60 0x55f3632409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efcdffff640 1 -- 172.21.15.18:0/448559011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363238ad0 msgr2=0x55f363248110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efcdffff640 1 --2- 172.21.15.18:0/448559011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363238ad0 0x55f363248110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efcdffff640 1 -- 172.21.15.18:0/448559011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363237d60 msgr2=0x55f3632409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efcdffff640 1 --2- 172.21.15.18:0/448559011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363237d60 0x55f3632409e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.263+0000 7efcdffff640 1 -- 172.21.15.18:0/448559011 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f36313a7b0 con 0x55f3632412e0 2024-02-17T13:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.264+0000 7efcdffff640 1 --2- 172.21.15.18:0/448559011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f363248650 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7efcd8012b60 tx=0x7efcd8009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37166086dfddb7d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.264+0000 7efcc57fa640 1 -- 172.21.15.18:0/448559011 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8012e10 con 0x55f3632412e0 2024-02-17T13:05:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.264+0000 7efcc57fa640 1 -- 172.21.15.18:0/448559011 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcd8004050 con 0x55f3632412e0 2024-02-17T13:05:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.264+0000 7efcc57fa640 1 -- 172.21.15.18:0/448559011 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8023680 con 0x55f3632412e0 2024-02-17T13:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.265+0000 7efce8bbb740 1 -- 172.21.15.18:0/448559011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 msgr2=0x55f363248650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.265+0000 7efce8bbb740 1 --2- 172.21.15.18:0/448559011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f363248650 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7efcd8012b60 tx=0x7efcd8009ab0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.265+0000 7efce8bbb740 1 -- 172.21.15.18:0/448559011 shutdown_connections 2024-02-17T13:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.265+0000 7efce8bbb740 1 --2- 172.21.15.18:0/448559011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f363248650 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.265+0000 7efce8bbb740 1 --2- 172.21.15.18:0/448559011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363238ad0 0x55f363248110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.265+0000 7efce8bbb740 1 --2- 172.21.15.18:0/448559011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363237d60 0x55f3632409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.265+0000 7efce8bbb740 1 -- 172.21.15.18:0/448559011 >> 172.21.15.18:0/448559011 conn(0x55f3631ff550 msgr2=0x55f363257f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.265+0000 7efce8bbb740 1 -- 172.21.15.18:0/448559011 shutdown_connections 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.265+0000 7efce8bbb740 1 -- 172.21.15.18:0/448559011 wait complete. 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.265+0000 7efce8bbb740 1 -- start start 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.265+0000 7efce8bbb740 1 Processor -- start 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efce8bbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363237d60 0x55f36329fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efce8bbb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363238ad0 0x55f3632a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efce8bbb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f3632a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efce8bbb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3632a4460 con 0x55f3632412e0 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efce8bbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3632a45d0 con 0x55f363237d60 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efce8bbb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3632a4790 con 0x55f363238ad0 2024-02-17T13:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efce8bbb740 10 client.0 _ll_register_callbacks cb 0x7efcd000cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcdffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f3632a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcdffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f3632a3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57334/0 (socket says 172.21.15.18:57334) 2024-02-17T13:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcdffff640 1 -- 172.21.15.18:0/969945240 learned_addr learned my addr 172.21.15.18:0/969945240 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363237d60 0x55f36329fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcdeffd640 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363238ad0 0x55f3632a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcdffff640 1 -- 172.21.15.18:0/969945240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363238ad0 msgr2=0x55f3632a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcdffff640 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363238ad0 0x55f3632a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcdffff640 1 -- 172.21.15.18:0/969945240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363237d60 msgr2=0x55f36329fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcdffff640 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363237d60 0x55f36329fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcdffff640 1 -- 172.21.15.18:0/969945240 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd801a040 con 0x55f3632412e0 2024-02-17T13:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcdffff640 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f3632a3f20 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7efcd800d010 tx=0x7efcd800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ef128c579c5d03 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcbeffd640 1 -- 172.21.15.18:0/969945240 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd80047e0 con 0x55f3632412e0 2024-02-17T13:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efcbeffd640 1 -- 172.21.15.18:0/969945240 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7efcd80043d0 con 0x55f3632412e0 2024-02-17T13:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.266+0000 7efce8bbb740 1 -- 172.21.15.18:0/969945240 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcd009d040 con 0x55f3632412e0 2024-02-17T13:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.267+0000 7efcbeffd640 1 -- 172.21.15.18:0/969945240 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcd8004590 con 0x55f3632412e0 2024-02-17T13:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.267+0000 7efce8bbb740 10 client.20745 fetch_fsmap learned FSMap version 70 2024-02-17T13:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.267+0000 7efce8bbb740 1 -- 172.21.15.18:0/969945240 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55f363160540 con 0x55f3632412e0 2024-02-17T13:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.267+0000 7efcbeffd640 1 -- 172.21.15.18:0/969945240 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcd009d040 con 0x55f3632412e0 2024-02-17T13:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.267+0000 7efcbeffd640 1 -- 172.21.15.18:0/969945240 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 70) v1 ==== 7946+0+0 (secure 0 0 0) 0x7efcd8029030 con 0x55f3632412e0 2024-02-17T13:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.267+0000 7efce8bbb740 10 client.20745 fetch_fsmap finished waiting for FSMap version 70 2024-02-17T13:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.267+0000 7efce8bbb740 10 client.20745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.267+0000 7efce8bbb740 20 client.20745 populate_metadata read hostname 'smithi018' 2024-02-17T13:05:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.267+0000 7efce8bbb740 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f3632511c0 0x55f3632af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.267+0000 7efce8bbb740 4 client.20745 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:05:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.267+0000 7efce8bbb740 1 -- 172.21.15.18:0/969945240 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f362edc550 con 0x55f3632511c0 2024-02-17T13:05:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.267+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f3632511c0 0x55f3632af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.268+0000 7efcdf7fe640 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f3632511c0 0x55f3632af270 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a5252861c07bf91 server_cookie=f29102a5d24980ef in_seq=0 out_seq=0 2024-02-17T13:05:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.268+0000 7efcbeffd640 10 client.20745 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efcbeffd640 1 -- 172.21.15.18:0/969945240 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f362edc550 con 0x55f3632511c0 2024-02-17T13:05:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efcbeffd640 10 client.20745 handle_command_reply: tid=0 2024-02-17T13:05:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efce8bbb740 1 client.20745 shutdown 2024-02-17T13:05:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efce8bbb740 1 -- 172.21.15.18:0/969945240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 msgr2=0x55f3632a3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efce8bbb740 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f3632a3f20 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7efcd800d010 tx=0x7efcd800cf10 comp rx=0 tx=0).stop 2024-02-17T13:05:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efce8bbb740 1 -- 172.21.15.18:0/969945240 shutdown_connections 2024-02-17T13:05:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efce8bbb740 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55f3632511c0 0x55f3632af270 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efce8bbb740 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3632412e0 0x55f3632a3f20 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efcbeffd640 0 client.20745 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efce8bbb740 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f363238ad0 0x55f3632a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efce8bbb740 1 --2- 172.21.15.18:0/969945240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f363237d60 0x55f36329fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efce8bbb740 1 -- 172.21.15.18:0/969945240 >> 172.21.15.18:0/969945240 conn(0x55f3631ff550 msgr2=0x55f363233e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efce8bbb740 1 -- 172.21.15.18:0/969945240 shutdown_connections 2024-02-17T13:05:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efce8bbb740 1 -- 172.21.15.18:0/969945240 wait complete. 2024-02-17T13:05:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.513+0000 7efce8bbb740 20 client.20745 trim_cache size 0 max 16384 2024-02-17T13:05:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:05:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:05:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:05:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:05:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.514+0000 7efce5f0d640 1 -- 172.21.15.18:0/3323975879 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efcc80a22d0 msgr2=0x7efcc80a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.514+0000 7efce5f0d640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efcc80a22d0 0x7efcc80a4790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009fd0 tx=0x7efcd8004ab0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.514+0000 7efce5f0d640 1 -- 172.21.15.18:0/3323975879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efce00ff790 msgr2=0x7efce0197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.514+0000 7efce5f0d640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efce00ff790 0x7efce0197710 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7efcd000e9f0 tx=0x7efcd000edf0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.514+0000 7efce5f0d640 1 -- 172.21.15.18:0/3323975879 shutdown_connections 2024-02-17T13:05:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.514+0000 7efce5f0d640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7efcc80a22d0 0x7efcc80a4790 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.514+0000 7efce5f0d640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce01057a0 0x7efce019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.514+0000 7efce5f0d640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce0100150 0x7efce0197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.514+0000 7efce5f0d640 1 --2- 172.21.15.18:0/3323975879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efce00ff790 0x7efce0197710 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.514+0000 7efce5f0d640 1 -- 172.21.15.18:0/3323975879 >> 172.21.15.18:0/3323975879 conn(0x7efce0076610 msgr2=0x7efce01025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.514+0000 7efce5f0d640 1 -- 172.21.15.18:0/3323975879 shutdown_connections 2024-02-17T13:05:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:44.514+0000 7efce5f0d640 1 -- 172.21.15.18:0/3323975879 wait complete. 2024-02-17T13:05:44.791 INFO:tasks.workunit.client.0.smithi018.stdout: 10 117692 38.17 MB/sec execute 56 sec latency 579.773 ms 2024-02-17T13:05:44.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:44 smithi146 ceph-mon[36454]: pgmap v498: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-02-17T13:05:45.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:44 smithi018 ceph-mon[32625]: pgmap v498: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-02-17T13:05:45.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:44 smithi119 ceph-mon[36188]: pgmap v498: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-02-17T13:05:45.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 118491 38.25 MB/sec execute 57 sec latency 67.155 ms 2024-02-17T13:05:46.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 119119 38.16 MB/sec execute 58 sec latency 323.748 ms 2024-02-17T13:05:46.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:46 smithi146 ceph-mon[36454]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-02-17T13:05:47.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:46 smithi018 ceph-mon[32625]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-02-17T13:05:47.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:46 smithi119 ceph-mon[36188]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-02-17T13:05:47.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 119684 38.20 MB/sec execute 59 sec latency 400.770 ms 2024-02-17T13:05:48.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 120227 37.97 MB/sec execute 60 sec latency 250.280 ms 2024-02-17T13:05:48.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:48 smithi146 ceph-mon[36454]: pgmap v500: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-02-17T13:05:49.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:48 smithi018 ceph-mon[32625]: pgmap v500: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-02-17T13:05:49.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:48 smithi119 ceph-mon[36188]: pgmap v500: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-02-17T13:05:49.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 121071 38.01 MB/sec execute 61 sec latency 66.632 ms 2024-02-17T13:05:50.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 121995 38.28 MB/sec execute 62 sec latency 103.310 ms 2024-02-17T13:05:50.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:50 smithi146 ceph-mon[36454]: pgmap v501: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 858 op/s 2024-02-17T13:05:51.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:50 smithi018 ceph-mon[32625]: pgmap v501: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 858 op/s 2024-02-17T13:05:51.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:50 smithi119 ceph-mon[36188]: pgmap v501: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 858 op/s 2024-02-17T13:05:51.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 122484 38.12 MB/sec execute 63 sec latency 242.766 ms 2024-02-17T13:05:52.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 123069 37.88 MB/sec execute 64 sec latency 361.971 ms 2024-02-17T13:05:52.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:52 smithi146 ceph-mon[36454]: pgmap v502: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 935 op/s 2024-02-17T13:05:53.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:52 smithi018 ceph-mon[32625]: pgmap v502: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 935 op/s 2024-02-17T13:05:53.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:52 smithi119 ceph-mon[36188]: pgmap v502: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 935 op/s 2024-02-17T13:05:53.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 123739 37.97 MB/sec execute 65 sec latency 158.885 ms 2024-02-17T13:05:54.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 124434 38.07 MB/sec execute 66 sec latency 62.517 ms 2024-02-17T13:05:54.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:54 smithi146 ceph-mon[36454]: pgmap v503: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 737 op/s 2024-02-17T13:05:55.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:54 smithi018 ceph-mon[32625]: pgmap v503: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 737 op/s 2024-02-17T13:05:55.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:54 smithi119 ceph-mon[36188]: pgmap v503: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 737 op/s 2024-02-17T13:05:55.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 125117 37.89 MB/sec execute 67 sec latency 78.033 ms 2024-02-17T13:05:56.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 125944 37.96 MB/sec execute 68 sec latency 119.762 ms 2024-02-17T13:05:57.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:56 smithi018 ceph-mon[32625]: pgmap v504: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-02-17T13:05:57.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:56 smithi119 ceph-mon[36188]: pgmap v504: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-02-17T13:05:57.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:56 smithi146 ceph-mon[36454]: pgmap v504: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-02-17T13:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.620+0000 7fc43d4a7640 1 -- 172.21.15.18:0/3388640004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4381007f0 msgr2=0x7fc438100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.620+0000 7fc43d4a7640 1 --2- 172.21.15.18:0/3388640004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4381007f0 0x7fc438100c50 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009900 tx=0x7fc42c02bc80 comp rx=0 tx=0).stop 2024-02-17T13:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.620+0000 7fc43d4a7640 1 -- 172.21.15.18:0/3388640004 shutdown_connections 2024-02-17T13:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.620+0000 7fc43d4a7640 1 --2- 172.21.15.18:0/3388640004 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc438101190 0x7fc43810cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.620+0000 7fc43d4a7640 1 --2- 172.21.15.18:0/3388640004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4381007f0 0x7fc438100c50 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.620+0000 7fc43d4a7640 1 --2- 172.21.15.18:0/3388640004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc438105050 0x7fc438105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.620+0000 7fc43d4a7640 1 -- 172.21.15.18:0/3388640004 >> 172.21.15.18:0/3388640004 conn(0x7fc438076640 msgr2=0x7fc4380fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.620+0000 7fc43d4a7640 1 -- 172.21.15.18:0/3388640004 shutdown_connections 2024-02-17T13:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.620+0000 7fc43d4a7640 1 -- 172.21.15.18:0/3388640004 wait complete. 2024-02-17T13:05:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc43d4a7640 1 Processor -- start 2024-02-17T13:05:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc43d4a7640 1 -- start start 2024-02-17T13:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc43d4a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4381007f0 0x7fc43819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc43d4a7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc438101190 0x7fc43819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc43d4a7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc438105050 0x7fc438197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc43d4a7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc438197c50 con 0x7fc438105050 2024-02-17T13:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc43d4a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc438197dc0 con 0x7fc4381007f0 2024-02-17T13:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc43d4a7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc438197f80 con 0x7fc438101190 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc4377fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc438105050 0x7fc438197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc4367fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc438101190 0x7fc43819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc436ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4381007f0 0x7fc43819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc4367fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc438101190 0x7fc43819d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:43654/0 (socket says 172.21.15.18:43654) 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.621+0000 7fc4377fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc438105050 0x7fc438197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42422/0 (socket says 172.21.15.18:42422) 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc4377fe640 1 -- 172.21.15.18:0/2153877569 learned_addr learned my addr 172.21.15.18:0/2153877569 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc436ffd640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4381007f0 0x7fc43819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:60648/0 (socket says 172.21.15.18:60648) 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc4377fe640 1 -- 172.21.15.18:0/2153877569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc438101190 msgr2=0x7fc43819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc4377fe640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc438101190 0x7fc43819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc4377fe640 1 -- 172.21.15.18:0/2153877569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4381007f0 msgr2=0x7fc43819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc4377fe640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4381007f0 0x7fc43819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc4377fe640 1 -- 172.21.15.18:0/2153877569 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c009560 con 0x7fc438105050 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc4367fc640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc438101190 0x7fc43819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc4377fe640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc438105050 0x7fc438197600 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc43000ea60 tx=0x7fc43000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc41ffff640 1 -- 172.21.15.18:0/2153877569 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000ce40 con 0x7fc438105050 2024-02-17T13:05:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc43d4a7640 1 -- 172.21.15.18:0/2153877569 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc438198260 con 0x7fc438105050 2024-02-17T13:05:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc43d4a7640 1 -- 172.21.15.18:0/2153877569 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4380fbb60 con 0x7fc438105050 2024-02-17T13:05:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc41ffff640 1 -- 172.21.15.18:0/2153877569 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc430005020 con 0x7fc438105050 2024-02-17T13:05:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.622+0000 7fc41ffff640 1 -- 172.21.15.18:0/2153877569 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc430010690 con 0x7fc438105050 2024-02-17T13:05:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.625+0000 7fc41ffff640 1 -- 172.21.15.18:0/2153877569 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc430004a40 con 0x7fc438105050 2024-02-17T13:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.626+0000 7fc41ffff640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc4200a2350 0x7fc4200a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.626+0000 7fc41ffff640 1 -- 172.21.15.18:0/2153877569 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc430014070 con 0x7fc438105050 2024-02-17T13:05:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.626+0000 7fc44007f740 1 Processor -- start 2024-02-17T13:05:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc44007f740 1 -- start start 2024-02-17T13:05:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc44007f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead182e0 0x559eead17030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc44007f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead179f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc44007f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 0x559eead22ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc44007f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeac7d9f0 con 0x559eead182e0 2024-02-17T13:05:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc44007f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeac7af20 con 0x559eead20af0 2024-02-17T13:05:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc44007f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeac76700 con 0x559eead17570 2024-02-17T13:05:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc436ffd640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc4200a2350 0x7fc4200a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc4367fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead179f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc4367fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead179f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:43658/0 (socket says 172.21.15.18:43658) 2024-02-17T13:05:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc4367fc640 1 -- 172.21.15.18:0/296029986 learned_addr learned my addr 172.21.15.18:0/296029986 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc4377fe640 1 --2- 172.21.15.18:0/296029986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 0x559eead22ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.627+0000 7fc436ffd640 1 --2- 172.21.15.18:0/296029986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead182e0 0x559eead17030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.628+0000 7fc4367fc640 1 -- 172.21.15.18:0/296029986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 msgr2=0x559eead22ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.628+0000 7fc4367fc640 1 --2- 172.21.15.18:0/296029986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 0x559eead22ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.628+0000 7fc4367fc640 1 -- 172.21.15.18:0/296029986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead182e0 msgr2=0x559eead17030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.628+0000 7fc4367fc640 1 --2- 172.21.15.18:0/296029986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead182e0 0x559eead17030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.628+0000 7fc4367fc640 1 -- 172.21.15.18:0/296029986 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eeac0c7b0 con 0x559eead17570 2024-02-17T13:05:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.628+0000 7fc436ffd640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc4200a2350 0x7fc4200a4810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc428009f90 tx=0x7fc4280137d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.628+0000 7fc4367fc640 1 --2- 172.21.15.18:0/296029986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead179f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009900 tx=0x7fc42c02cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db85b1b6de997590 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.628+0000 7fc41d7fa640 1 -- 172.21.15.18:0/296029986 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c02c660 con 0x559eead17570 2024-02-17T13:05:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.628+0000 7fc41d7fa640 1 -- 172.21.15.18:0/296029986 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc42c004030 con 0x559eead17570 2024-02-17T13:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.628+0000 7fc41d7fa640 1 -- 172.21.15.18:0/296029986 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c005420 con 0x559eead17570 2024-02-17T13:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.629+0000 7fc44007f740 1 -- 172.21.15.18:0/296029986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 msgr2=0x559eead179f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.629+0000 7fc44007f740 1 --2- 172.21.15.18:0/296029986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead179f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009900 tx=0x7fc42c02cc40 comp rx=0 tx=0).stop 2024-02-17T13:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.630+0000 7fc44007f740 1 -- 172.21.15.18:0/296029986 shutdown_connections 2024-02-17T13:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.630+0000 7fc44007f740 1 --2- 172.21.15.18:0/296029986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 0x559eead22ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.630+0000 7fc44007f740 1 --2- 172.21.15.18:0/296029986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead179f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.630+0000 7fc44007f740 1 --2- 172.21.15.18:0/296029986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead182e0 0x559eead17030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.630+0000 7fc44007f740 1 -- 172.21.15.18:0/296029986 >> 172.21.15.18:0/296029986 conn(0x559eeacd6550 msgr2=0x559eead12260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.630+0000 7fc44007f740 1 -- 172.21.15.18:0/296029986 shutdown_connections 2024-02-17T13:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.630+0000 7fc44007f740 1 -- 172.21.15.18:0/296029986 wait complete. 2024-02-17T13:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.630+0000 7fc44007f740 1 -- start start 2024-02-17T13:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.630+0000 7fc44007f740 1 Processor -- start 2024-02-17T13:05:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc44007f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead1dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc44007f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead182e0 0x559eead1dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc44007f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 0x559eead1e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc44007f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeac1bbf0 con 0x559eead182e0 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc44007f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eea94a250 con 0x559eead20af0 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc44007f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeabfc200 con 0x559eead17570 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc44007f740 10 client.0 _ll_register_callbacks cb 0x559eeac6ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc4367fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead182e0 0x559eead1dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc436ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead1dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc436ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead1dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:43662/0 (socket says 172.21.15.18:43662) 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc4367fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead182e0 0x559eead1dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42438/0 (socket says 172.21.15.18:42438) 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc436ffd640 1 -- 172.21.15.18:0/1946498602 learned_addr learned my addr 172.21.15.18:0/1946498602 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc4377fe640 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 0x559eead1e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc436ffd640 1 -- 172.21.15.18:0/1946498602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 msgr2=0x559eead1e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc436ffd640 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 0x559eead1e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc436ffd640 1 -- 172.21.15.18:0/1946498602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead182e0 msgr2=0x559eead1dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc436ffd640 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead182e0 0x559eead1dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc436ffd640 1 -- 172.21.15.18:0/1946498602 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43810d870 con 0x559eead17570 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.631+0000 7fc4367fc640 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead182e0 0x559eead1dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.632+0000 7fc436ffd640 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead1dab0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc42800fb80 tx=0x7fc428004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b1c7e26eacac9bc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.632+0000 7fc41aff5640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.632+0000 7fc41aff5640 1 -- 172.21.15.18:0/1946498602 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc428010de0 con 0x559eead17570 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.632+0000 7fc41aff5640 1 -- 172.21.15.18:0/1946498602 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc428029460 con 0x559eead17570 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.632+0000 7fc44007f740 1 -- 172.21.15.18:0/1946498602 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559eea976600 con 0x559eead17570 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.632+0000 7fc41aff5640 1 -- 172.21.15.18:0/1946498602 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc428029620 con 0x559eead17570 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.632+0000 7fc44007f740 10 client.29683 fetch_fsmap learned FSMap version 70 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.632+0000 7fc44007f740 1 -- 172.21.15.18:0/1946498602 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x559eeac74d80 con 0x559eead17570 2024-02-17T13:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.632+0000 7fc41aff5640 1 -- 172.21.15.18:0/1946498602 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559eea976600 con 0x559eead17570 2024-02-17T13:05:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.633+0000 7fc41aff5640 1 -- 172.21.15.18:0/1946498602 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 70) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fc4280205a0 con 0x559eead17570 2024-02-17T13:05:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.633+0000 7fc44007f740 10 client.29683 fetch_fsmap finished waiting for FSMap version 70 2024-02-17T13:05:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.633+0000 7fc44007f740 10 client.29683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:05:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.633+0000 7fc44007f740 20 client.29683 populate_metadata read hostname 'smithi018' 2024-02-17T13:05:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.633+0000 7fc44007f740 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559eead30fc0 0x559eead33400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.633+0000 7fc44007f740 4 client.29683 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:05:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.633+0000 7fc44007f740 1 -- 172.21.15.18:0/1946498602 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559eea9c71b0 con 0x559eead30fc0 2024-02-17T13:05:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.633+0000 7fc4367fc640 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559eead30fc0 0x559eead33400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.635+0000 7fc4367fc640 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559eead30fc0 0x559eead33400 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c189644f4017fe3 server_cookie=3a97d23fa07a45e7 in_seq=0 out_seq=0 2024-02-17T13:05:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.635+0000 7fc41aff5640 10 client.29683 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc41aff5640 1 -- 172.21.15.18:0/1946498602 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559eea9c71b0 con 0x559eead30fc0 2024-02-17T13:05:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc41aff5640 10 client.29683 handle_command_reply: tid=0 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc44007f740 1 client.29683 shutdown 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc44007f740 1 -- 172.21.15.18:0/1946498602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 msgr2=0x559eead1dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc44007f740 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead1dab0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc42800fb80 tx=0x7fc428004730 comp rx=0 tx=0).stop 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc44007f740 1 -- 172.21.15.18:0/1946498602 shutdown_connections 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc44007f740 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559eead30fc0 0x559eead33400 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc44007f740 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 0x559eead1e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc44007f740 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead182e0 0x559eead1dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc41aff5640 0 client.29683 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc44007f740 1 --2- 172.21.15.18:0/1946498602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead1dab0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc44007f740 1 -- 172.21.15.18:0/1946498602 >> 172.21.15.18:0/1946498602 conn(0x559eeacd6550 msgr2=0x559eead13fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc44007f740 1 -- 172.21.15.18:0/1946498602 shutdown_connections 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc44007f740 1 -- 172.21.15.18:0/1946498602 wait complete. 2024-02-17T13:05:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.638+0000 7fc44007f740 20 client.29683 trim_cache size 0 max 16384 2024-02-17T13:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc44007f740 1 Processor -- start 2024-02-17T13:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc44007f740 1 -- start start 2024-02-17T13:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc44007f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead201e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc44007f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead182e0 0x559eead1e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc44007f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead20af0 0x559eead278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc44007f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eead17db0 con 0x559eead20af0 2024-02-17T13:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc44007f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eead2d8f0 con 0x559eead182e0 2024-02-17T13:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc44007f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eead2c800 con 0x559eead17570 2024-02-17T13:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc4377fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead20af0 0x559eead278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc4377fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead20af0 0x559eead278e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42452/0 (socket says 172.21.15.18:42452) 2024-02-17T13:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc4377fe640 1 -- 172.21.15.18:0/3876596904 learned_addr learned my addr 172.21.15.18:0/3876596904 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc436ffd640 1 --2- 172.21.15.18:0/3876596904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead201e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.655+0000 7fc4367fc640 1 --2- 172.21.15.18:0/3876596904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead182e0 0x559eead1e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.656+0000 7fc4377fe640 1 -- 172.21.15.18:0/3876596904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 msgr2=0x559eead201e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.656+0000 7fc4377fe640 1 --2- 172.21.15.18:0/3876596904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead201e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.656+0000 7fc4377fe640 1 -- 172.21.15.18:0/3876596904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead182e0 msgr2=0x559eead1e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.656+0000 7fc4377fe640 1 --2- 172.21.15.18:0/3876596904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead182e0 0x559eead1e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.656+0000 7fc4377fe640 1 -- 172.21.15.18:0/3876596904 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eeac117b0 con 0x559eead20af0 2024-02-17T13:05:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.656+0000 7fc436ffd640 1 --2- 172.21.15.18:0/3876596904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead201e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:05:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.656+0000 7fc4377fe640 1 --2- 172.21.15.18:0/3876596904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead20af0 0x559eead278e0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fc430097850 tx=0x7fc43009ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f50e3ef32073c92 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.656+0000 7fc41cff9640 1 -- 172.21.15.18:0/3876596904 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc430098fe0 con 0x559eead20af0 2024-02-17T13:05:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.656+0000 7fc41cff9640 1 -- 172.21.15.18:0/3876596904 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc430004cf0 con 0x559eead20af0 2024-02-17T13:05:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.656+0000 7fc41cff9640 1 -- 172.21.15.18:0/3876596904 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43009b0e0 con 0x559eead20af0 2024-02-17T13:05:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.657+0000 7fc44007f740 1 -- 172.21.15.18:0/3876596904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead20af0 msgr2=0x559eead278e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.657+0000 7fc44007f740 1 --2- 172.21.15.18:0/3876596904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead20af0 0x559eead278e0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fc430097850 tx=0x7fc43009ad80 comp rx=0 tx=0).stop 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.657+0000 7fc44007f740 1 -- 172.21.15.18:0/3876596904 shutdown_connections 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.657+0000 7fc44007f740 1 --2- 172.21.15.18:0/3876596904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead20af0 0x559eead278e0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.657+0000 7fc44007f740 1 --2- 172.21.15.18:0/3876596904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead182e0 0x559eead1e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.657+0000 7fc44007f740 1 --2- 172.21.15.18:0/3876596904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead17570 0x559eead201e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.657+0000 7fc44007f740 1 -- 172.21.15.18:0/3876596904 >> 172.21.15.18:0/3876596904 conn(0x559eeacd6550 msgr2=0x559eead1c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.657+0000 7fc44007f740 1 -- 172.21.15.18:0/3876596904 shutdown_connections 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.657+0000 7fc44007f740 1 -- 172.21.15.18:0/3876596904 wait complete. 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc44007f740 1 -- start start 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc44007f740 1 Processor -- start 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc44007f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead17570 0x559eead1a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc44007f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead182e0 0x559eead1aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc44007f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 0x559eead1b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc44007f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eead87600 con 0x559eead17570 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc44007f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eead87770 con 0x559eead20af0 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc44007f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eead87930 con 0x559eead182e0 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc4367fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead182e0 0x559eead1aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc436ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead17570 0x559eead1a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc4367fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead182e0 0x559eead1aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:43684/0 (socket says 172.21.15.18:43684) 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc4367fc640 1 -- 172.21.15.18:0/3532110256 learned_addr learned my addr 172.21.15.18:0/3532110256 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.658+0000 7fc4377fe640 1 --2- 172.21.15.18:0/3532110256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 0x559eead1b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.659+0000 7fc44007f740 10 client.0 _ll_register_callbacks cb 0x7fc4300093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.659+0000 7fc4367fc640 1 -- 172.21.15.18:0/3532110256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 msgr2=0x559eead1b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.659+0000 7fc4367fc640 1 --2- 172.21.15.18:0/3532110256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 0x559eead1b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.659+0000 7fc4367fc640 1 -- 172.21.15.18:0/3532110256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead17570 msgr2=0x559eead1a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:05:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.659+0000 7fc4367fc640 1 --2- 172.21.15.18:0/3532110256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead17570 0x559eead1a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.659+0000 7fc4367fc640 1 -- 172.21.15.18:0/3532110256 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559eeac74d80 con 0x559eead182e0 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.659+0000 7fc4367fc640 1 --2- 172.21.15.18:0/3532110256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead182e0 0x559eead1aee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009900 tx=0x7fc42c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f4ae024bc58cffb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.659+0000 7fc41aff5640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.659+0000 7fc41aff5640 1 -- 172.21.15.18:0/3532110256 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0046c0 con 0x559eead182e0 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.659+0000 7fc41aff5640 1 -- 172.21.15.18:0/3532110256 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc42c04e050 con 0x559eead182e0 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.659+0000 7fc44007f740 1 -- 172.21.15.18:0/3532110256 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc42c004030 con 0x559eead182e0 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.660+0000 7fc41aff5640 1 -- 172.21.15.18:0/3532110256 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc42c004820 con 0x559eead182e0 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.660+0000 7fc44007f740 10 client.29689 fetch_fsmap learned FSMap version 70 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.660+0000 7fc44007f740 1 -- 172.21.15.18:0/3532110256 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7fc4380fbb60 con 0x559eead182e0 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.660+0000 7fc41aff5640 1 -- 172.21.15.18:0/3532110256 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc42c004980 con 0x559eead182e0 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.660+0000 7fc41aff5640 1 -- 172.21.15.18:0/3532110256 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 70) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fc42c03c050 con 0x559eead182e0 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.660+0000 7fc44007f740 10 client.29689 fetch_fsmap finished waiting for FSMap version 70 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.660+0000 7fc44007f740 10 client.29689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.660+0000 7fc44007f740 20 client.29689 populate_metadata read hostname 'smithi018' 2024-02-17T13:05:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.660+0000 7fc44007f740 1 --2- 172.21.15.18:0/3532110256 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559eead88eb0 0x559eead8b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:05:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.660+0000 7fc44007f740 4 client.29689 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:05:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.660+0000 7fc44007f740 1 -- 172.21.15.18:0/3532110256 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fc42c004030 con 0x559eead88eb0 2024-02-17T13:05:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.660+0000 7fc436ffd640 1 --2- 172.21.15.18:0/3532110256 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559eead88eb0 0x559eead8b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:05:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.661+0000 7fc41aff5640 10 client.29689 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.661+0000 7fc436ffd640 1 --2- 172.21.15.18:0/3532110256 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559eead88eb0 0x559eead8b2d0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a876bb01741e664 server_cookie=c4b1d6f1dc69108e in_seq=0 out_seq=0 2024-02-17T13:05:57.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 126681 38.07 MB/sec execute 69 sec latency 287.483 ms 2024-02-17T13:05:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.964+0000 7fc41aff5640 1 -- 172.21.15.18:0/3532110256 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fc42c004030 con 0x559eead88eb0 2024-02-17T13:05:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.964+0000 7fc41aff5640 10 client.29689 handle_command_reply: tid=0 2024-02-17T13:05:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.964+0000 7fc44007f740 1 client.29689 shutdown 2024-02-17T13:05:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.964+0000 7fc44007f740 1 -- 172.21.15.18:0/3532110256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead182e0 msgr2=0x559eead1aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.964+0000 7fc44007f740 1 --2- 172.21.15.18:0/3532110256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead182e0 0x559eead1aee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009900 tx=0x7fc42c004610 comp rx=0 tx=0).stop 2024-02-17T13:05:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.964+0000 7fc44007f740 1 -- 172.21.15.18:0/3532110256 shutdown_connections 2024-02-17T13:05:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.964+0000 7fc44007f740 1 --2- 172.21.15.18:0/3532110256 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x559eead88eb0 0x559eead8b2d0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.965+0000 7fc44007f740 1 --2- 172.21.15.18:0/3532110256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559eead20af0 0x559eead1b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.965+0000 7fc41aff5640 0 client.29689 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:05:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.965+0000 7fc44007f740 1 --2- 172.21.15.18:0/3532110256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559eead182e0 0x559eead1aee0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.965+0000 7fc44007f740 1 --2- 172.21.15.18:0/3532110256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559eead17570 0x559eead1a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.965+0000 7fc44007f740 1 -- 172.21.15.18:0/3532110256 >> 172.21.15.18:0/3532110256 conn(0x559eeacd6550 msgr2=0x559eead08300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.965+0000 7fc44007f740 1 -- 172.21.15.18:0/3532110256 shutdown_connections 2024-02-17T13:05:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.965+0000 7fc44007f740 1 -- 172.21.15.18:0/3532110256 wait complete. 2024-02-17T13:05:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.965+0000 7fc44007f740 20 client.29689 trim_cache size 0 max 16384 2024-02-17T13:05:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:05:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:05:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:05:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:05:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.966+0000 7fc43d4a7640 1 -- 172.21.15.18:0/2153877569 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc4200a2350 msgr2=0x7fc4200a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.966+0000 7fc43d4a7640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc4200a2350 0x7fc4200a4810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc428009f90 tx=0x7fc4280137d0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.966+0000 7fc43d4a7640 1 -- 172.21.15.18:0/2153877569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc438105050 msgr2=0x7fc438197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:05:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.966+0000 7fc43d4a7640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc438105050 0x7fc438197600 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc43000ea60 tx=0x7fc43000ef20 comp rx=0 tx=0).stop 2024-02-17T13:05:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.966+0000 7fc43d4a7640 1 -- 172.21.15.18:0/2153877569 shutdown_connections 2024-02-17T13:05:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.966+0000 7fc43d4a7640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc4200a2350 0x7fc4200a4810 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.966+0000 7fc43d4a7640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc438105050 0x7fc438197600 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.966+0000 7fc43d4a7640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc438101190 0x7fc43819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.966+0000 7fc43d4a7640 1 --2- 172.21.15.18:0/2153877569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4381007f0 0x7fc43819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:05:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.966+0000 7fc43d4a7640 1 -- 172.21.15.18:0/2153877569 >> 172.21.15.18:0/2153877569 conn(0x7fc438076640 msgr2=0x7fc4380ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:05:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.966+0000 7fc43d4a7640 1 -- 172.21.15.18:0/2153877569 shutdown_connections 2024-02-17T13:05:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:05:57.966+0000 7fc43d4a7640 1 -- 172.21.15.18:0/2153877569 wait complete. 2024-02-17T13:05:58.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 127279 37.98 MB/sec execute 70 sec latency 211.645 ms 2024-02-17T13:05:59.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:05:58 smithi018 ceph-mon[32625]: pgmap v505: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 730 op/s 2024-02-17T13:05:59.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:05:58 smithi119 ceph-mon[36188]: pgmap v505: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 730 op/s 2024-02-17T13:05:59.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:05:58 smithi146 ceph-mon[36454]: pgmap v505: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 730 op/s 2024-02-17T13:05:59.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 127999 37.96 MB/sec execute 71 sec latency 85.569 ms 2024-02-17T13:06:00.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 128861 38.11 MB/sec execute 72 sec latency 55.716 ms 2024-02-17T13:06:01.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:00 smithi018 ceph-mon[32625]: pgmap v506: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 866 op/s 2024-02-17T13:06:01.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:00 smithi119 ceph-mon[36188]: pgmap v506: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 866 op/s 2024-02-17T13:06:01.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:00 smithi146 ceph-mon[36454]: pgmap v506: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 866 op/s 2024-02-17T13:06:01.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 129440 37.91 MB/sec execute 73 sec latency 97.238 ms 2024-02-17T13:06:02.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 129873 37.61 MB/sec execute 74 sec latency 359.504 ms 2024-02-17T13:06:03.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:02 smithi018 ceph-mon[32625]: pgmap v507: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 980 op/s 2024-02-17T13:06:03.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:02 smithi119 ceph-mon[36188]: pgmap v507: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 980 op/s 2024-02-17T13:06:03.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:02 smithi146 ceph-mon[36454]: pgmap v507: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 980 op/s 2024-02-17T13:06:03.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 130397 37.60 MB/sec execute 75 sec latency 252.297 ms 2024-02-17T13:06:04.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 131103 37.68 MB/sec execute 76 sec latency 171.967 ms 2024-02-17T13:06:05.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:04 smithi018 ceph-mon[32625]: pgmap v508: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 761 op/s 2024-02-17T13:06:05.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:04 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:05.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:04 smithi119 ceph-mon[36188]: pgmap v508: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 761 op/s 2024-02-17T13:06:05.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:04 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:05.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:04 smithi146 ceph-mon[36454]: pgmap v508: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 761 op/s 2024-02-17T13:06:05.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:04 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:05.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 131786 37.69 MB/sec execute 77 sec latency 65.398 ms 2024-02-17T13:06:06.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:05 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:06.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:05 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:06.098 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:05 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:06.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 132163 37.43 MB/sec execute 78 sec latency 162.355 ms 2024-02-17T13:06:07.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:06 smithi018 ceph-mon[32625]: pgmap v509: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 1.27k op/s 2024-02-17T13:06:07.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:06 smithi018 ceph-mon[32625]: osdmap e110: 12 total, 12 up, 12 in 2024-02-17T13:06:07.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:06 smithi119 ceph-mon[36188]: pgmap v509: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 1.27k op/s 2024-02-17T13:06:07.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:06 smithi119 ceph-mon[36188]: osdmap e110: 12 total, 12 up, 12 in 2024-02-17T13:06:07.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:06 smithi146 ceph-mon[36454]: pgmap v509: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 1.27k op/s 2024-02-17T13:06:07.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:06 smithi146 ceph-mon[36454]: osdmap e110: 12 total, 12 up, 12 in 2024-02-17T13:06:07.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 132472 37.19 MB/sec execute 79 sec latency 732.631 ms 2024-02-17T13:06:08.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 132936 37.10 MB/sec execute 80 sec latency 1732.670 ms 2024-02-17T13:06:09.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:08 smithi018 ceph-mon[32625]: pgmap v511: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 1.10k op/s 2024-02-17T13:06:09.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:08 smithi119 ceph-mon[36188]: pgmap v511: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 1.10k op/s 2024-02-17T13:06:09.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:08 smithi146 ceph-mon[36454]: pgmap v511: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 1.10k op/s 2024-02-17T13:06:09.792 INFO:tasks.workunit.client.0.smithi018.stdout: 10 133302 36.85 MB/sec execute 81 sec latency 2732.709 ms 2024-02-17T13:06:10.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 133704 36.58 MB/sec execute 82 sec latency 3732.751 ms 2024-02-17T13:06:11.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:10 smithi018 ceph-mon[32625]: pgmap v512: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2024-02-17T13:06:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.076+0000 7f80fee23640 1 -- 172.21.15.18:0/176910030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f8102d10 msgr2=0x7f80f810b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.076+0000 7f80fee23640 1 --2- 172.21.15.18:0/176910030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f8102d10 0x7f80f810b1f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009900 tx=0x7f80e802bcc0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.076+0000 7f80fee23640 1 -- 172.21.15.18:0/176910030 shutdown_connections 2024-02-17T13:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.076+0000 7f80fee23640 1 --2- 172.21.15.18:0/176910030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80f810b7c0 0x7f80f810dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.076+0000 7f80fee23640 1 --2- 172.21.15.18:0/176910030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f8102d10 0x7f80f810b1f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.076+0000 7f80fee23640 1 --2- 172.21.15.18:0/176910030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80f81023f0 0x7f80f81027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.076+0000 7f80fee23640 1 -- 172.21.15.18:0/176910030 >> 172.21.15.18:0/176910030 conn(0x7f80f80771a0 msgr2=0x7f80f80775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.076+0000 7f80fee23640 1 -- 172.21.15.18:0/176910030 shutdown_connections 2024-02-17T13:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.076+0000 7f80fee23640 1 -- 172.21.15.18:0/176910030 wait complete. 2024-02-17T13:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80fee23640 1 Processor -- start 2024-02-17T13:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80fee23640 1 -- start start 2024-02-17T13:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80fee23640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80f81023f0 0x7f80f8197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80fee23640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80f8102d10 0x7f80f8197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80fee23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f810b7c0 0x7f80f819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80fee23640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f819c5f0 con 0x7f80f81023f0 2024-02-17T13:06:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80fee23640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f819c760 con 0x7f80f810b7c0 2024-02-17T13:06:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80fee23640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f819c920 con 0x7f80f8102d10 2024-02-17T13:06:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80fcbbf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80f81023f0 0x7f80f8197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80fcbbf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80f81023f0 0x7f80f8197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36206/0 (socket says 172.21.15.18:36206) 2024-02-17T13:06:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80fcbbf640 1 -- 172.21.15.18:0/1468988481 learned_addr learned my addr 172.21.15.18:0/1468988481 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80f7fff640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80f8102d10 0x7f80f8197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.077+0000 7f80fd3c0640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f810b7c0 0x7f80f819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.078+0000 7f80fcbbf640 1 -- 172.21.15.18:0/1468988481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80f8102d10 msgr2=0x7f80f8197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.078+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80f8102d10 0x7f80f8197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.078+0000 7f80fcbbf640 1 -- 172.21.15.18:0/1468988481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f810b7c0 msgr2=0x7f80f819bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.078+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f810b7c0 0x7f80f819bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.078+0000 7f80fcbbf640 1 -- 172.21.15.18:0/1468988481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f0009840 con 0x7f80f81023f0 2024-02-17T13:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.078+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80f81023f0 0x7f80f8197730 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009d50 tx=0x7f80f000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.078+0000 7f80f5ffb640 1 -- 172.21.15.18:0/1468988481 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0003b70 con 0x7f80f81023f0 2024-02-17T13:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.078+0000 7f80fee23640 1 -- 172.21.15.18:0/1468988481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e8009560 con 0x7f80f81023f0 2024-02-17T13:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.078+0000 7f80f5ffb640 1 -- 172.21.15.18:0/1468988481 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80f0003cd0 con 0x7f80f81023f0 2024-02-17T13:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.078+0000 7f80f5ffb640 1 -- 172.21.15.18:0/1468988481 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0010990 con 0x7f80f81023f0 2024-02-17T13:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.079+0000 7f80fee23640 1 -- 172.21.15.18:0/1468988481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f81a42a0 con 0x7f80f81023f0 2024-02-17T13:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.081+0000 7f80f5ffb640 1 -- 172.21.15.18:0/1468988481 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f80f0010af0 con 0x7f80f81023f0 2024-02-17T13:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.082+0000 7f80f5ffb640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f80e40a22d0 0x7f80e40a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.082+0000 7f80f5ffb640 1 -- 172.21.15.18:0/1468988481 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f80f0014070 con 0x7f80f81023f0 2024-02-17T13:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.082+0000 7f81019f6740 1 Processor -- start 2024-02-17T13:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.082+0000 7f81019f6740 1 -- start start 2024-02-17T13:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.082+0000 7f80f7fff640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f80e40a22d0 0x7f80e40a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.082+0000 7f81019f6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4dee0 0x55cd93d4cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.082+0000 7f81019f6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4d170 0x55cd93d4d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.082+0000 7f81019f6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d566f0 0x55cd93d58ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.082+0000 7f81019f6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93cb79f0 con 0x55cd93d4d170 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.082+0000 7f81019f6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93cb4f20 con 0x55cd93d4dee0 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f81019f6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93cb0700 con 0x55cd93d566f0 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f80f7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4d170 0x55cd93d4d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f80f7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4d170 0x55cd93d4d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36208/0 (socket says 172.21.15.18:36208) 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f80f7fff640 1 -- 172.21.15.18:0/3312644407 learned_addr learned my addr 172.21.15.18:0/3312644407 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/3312644407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4dee0 0x55cd93d4cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f80fd3c0640 1 --2- 172.21.15.18:0/3312644407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d566f0 0x55cd93d58ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f80fcbbf640 1 -- 172.21.15.18:0/3312644407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d566f0 msgr2=0x55cd93d58ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/3312644407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d566f0 0x55cd93d58ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f80f7fff640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f80e40a22d0 0x7f80e40a4790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f80e802c200 tx=0x7f80e8004b50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f80fcbbf640 1 -- 172.21.15.18:0/3312644407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4d170 msgr2=0x55cd93d4d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/3312644407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4d170 0x55cd93d4d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f80fcbbf640 1 -- 172.21.15.18:0/3312644407 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd93c467b0 con 0x55cd93d4dee0 2024-02-17T13:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.083+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/3312644407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4dee0 0x55cd93d4cc30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80f00cc080 tx=0x7f80f009b210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d23661c899a388 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.084+0000 7f80eeffd640 1 -- 172.21.15.18:0/3312644407 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f009b320 con 0x55cd93d4dee0 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.084+0000 7f80eeffd640 1 -- 172.21.15.18:0/3312644407 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80f009f470 con 0x55cd93d4dee0 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.084+0000 7f80eeffd640 1 -- 172.21.15.18:0/3312644407 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0097990 con 0x55cd93d4dee0 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.084+0000 7f81019f6740 1 -- 172.21.15.18:0/3312644407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4dee0 msgr2=0x55cd93d4cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.084+0000 7f81019f6740 1 --2- 172.21.15.18:0/3312644407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4dee0 0x55cd93d4cc30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80f00cc080 tx=0x7f80f009b210 comp rx=0 tx=0).stop 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.085+0000 7f81019f6740 1 -- 172.21.15.18:0/3312644407 shutdown_connections 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.085+0000 7f81019f6740 1 --2- 172.21.15.18:0/3312644407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d566f0 0x55cd93d58ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.085+0000 7f81019f6740 1 --2- 172.21.15.18:0/3312644407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4d170 0x55cd93d4d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.085+0000 7f81019f6740 1 --2- 172.21.15.18:0/3312644407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4dee0 0x55cd93d4cc30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.085+0000 7f81019f6740 1 -- 172.21.15.18:0/3312644407 >> 172.21.15.18:0/3312644407 conn(0x55cd93d10550 msgr2=0x55cd93d47e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.085+0000 7f81019f6740 1 -- 172.21.15.18:0/3312644407 shutdown_connections 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.085+0000 7f81019f6740 1 -- 172.21.15.18:0/3312644407 wait complete. 2024-02-17T13:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.085+0000 7f81019f6740 1 -- start start 2024-02-17T13:06:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.085+0000 7f81019f6740 1 Processor -- start 2024-02-17T13:06:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.086+0000 7f81019f6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4d170 0x55cd93d4f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.086+0000 7f81019f6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4dee0 0x55cd93d4fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.086+0000 7f80fcbbf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4d170 0x55cd93d4f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.086+0000 7f80fcbbf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4d170 0x55cd93d4f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40096/0 (socket says 172.21.15.18:40096) 2024-02-17T13:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.086+0000 7f80fcbbf640 1 -- 172.21.15.18:0/433778421 learned_addr learned my addr 172.21.15.18:0/433778421 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.086+0000 7f80f7fff640 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4dee0 0x55cd93d4fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.086+0000 7f81019f6740 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d566f0 0x55cd93d50260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.086+0000 7f81019f6740 1 -- 172.21.15.18:0/433778421 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93c55bf0 con 0x55cd93d566f0 2024-02-17T13:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.087+0000 7f81019f6740 1 -- 172.21.15.18:0/433778421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93984250 con 0x55cd93d4dee0 2024-02-17T13:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.087+0000 7f81019f6740 1 -- 172.21.15.18:0/433778421 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93c36200 con 0x55cd93d4d170 2024-02-17T13:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.087+0000 7f80fd3c0640 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d566f0 0x55cd93d50260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.087+0000 7f81019f6740 10 client.0 _ll_register_callbacks cb 0x55cd93ca4d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.087+0000 7f80fd3c0640 1 -- 172.21.15.18:0/433778421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4d170 msgr2=0x55cd93d4f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.087+0000 7f80fd3c0640 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4d170 0x55cd93d4f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.087+0000 7f80fd3c0640 1 -- 172.21.15.18:0/433778421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4dee0 msgr2=0x55cd93d4fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.087+0000 7f80fd3c0640 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4dee0 0x55cd93d4fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.087+0000 7f80fd3c0640 1 -- 172.21.15.18:0/433778421 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f81a42a0 con 0x55cd93d566f0 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.087+0000 7f80f7fff640 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4dee0 0x55cd93d4fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.087+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4d170 0x55cd93d4f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.088+0000 7f80fd3c0640 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d566f0 0x55cd93d50260 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x55cd93d4dc10 tx=0x7f80e0012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c169ee391dab380 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.088+0000 7f80d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.088+0000 7f80d7fff640 1 -- 172.21.15.18:0/433778421 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0017070 con 0x55cd93d566f0 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.088+0000 7f80d7fff640 1 -- 172.21.15.18:0/433778421 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f80e000f870 con 0x55cd93d566f0 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.088+0000 7f81019f6740 1 -- 172.21.15.18:0/433778421 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd939b0600 con 0x55cd93d566f0 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.088+0000 7f80d7fff640 1 -- 172.21.15.18:0/433778421 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80e000fa30 con 0x55cd93d566f0 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.088+0000 7f81019f6740 10 client.20775 fetch_fsmap learned FSMap version 72 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.089+0000 7f81019f6740 1 -- 172.21.15.18:0/433778421 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55cd93caed80 con 0x55cd93d566f0 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.089+0000 7f80d7fff640 1 -- 172.21.15.18:0/433778421 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cd939b0600 con 0x55cd93d566f0 2024-02-17T13:06:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.089+0000 7f80d7fff640 1 -- 172.21.15.18:0/433778421 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 72) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f80e0005140 con 0x55cd93d566f0 2024-02-17T13:06:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.089+0000 7f81019f6740 10 client.20775 fetch_fsmap finished waiting for FSMap version 72 2024-02-17T13:06:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.089+0000 7f81019f6740 10 client.20775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:06:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.089+0000 7f81019f6740 20 client.20775 populate_metadata read hostname 'smithi018' 2024-02-17T13:06:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.089+0000 7f81019f6740 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd93d6af10 0x55cd93d6d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.089+0000 7f81019f6740 4 client.20775 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:06:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.089+0000 7f81019f6740 1 -- 172.21.15.18:0/433778421 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd93a011b0 con 0x55cd93d6af10 2024-02-17T13:06:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.089+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd93d6af10 0x55cd93d6d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.090+0000 7f80d7fff640 10 client.20775 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.090+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd93d6af10 0x55cd93d6d330 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40153858e1f8db69 server_cookie=e601f602fd7863ef in_seq=0 out_seq=0 2024-02-17T13:06:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.094+0000 7f80d7fff640 1 -- 172.21.15.18:0/433778421 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f80f007fa90 con 0x55cd93d6af10 2024-02-17T13:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.094+0000 7f80d7fff640 10 client.20775 handle_command_reply: tid=0 2024-02-17T13:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.094+0000 7f81019f6740 1 client.20775 shutdown 2024-02-17T13:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.094+0000 7f81019f6740 1 -- 172.21.15.18:0/433778421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d566f0 msgr2=0x55cd93d50260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.094+0000 7f81019f6740 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d566f0 0x55cd93d50260 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x55cd93d4dc10 tx=0x7f80e0012b60 comp rx=0 tx=0).stop 2024-02-17T13:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.095+0000 7f81019f6740 1 -- 172.21.15.18:0/433778421 shutdown_connections 2024-02-17T13:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.095+0000 7f81019f6740 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd93d6af10 0x55cd93d6d330 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.095+0000 7f81019f6740 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d566f0 0x55cd93d50260 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.095+0000 7f81019f6740 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4dee0 0x55cd93d4fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.095+0000 7f81019f6740 1 --2- 172.21.15.18:0/433778421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4d170 0x55cd93d4f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.095+0000 7f81019f6740 1 -- 172.21.15.18:0/433778421 >> 172.21.15.18:0/433778421 conn(0x55cd93d10550 msgr2=0x55cd93d49bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.095+0000 7f80d7fff640 0 client.20775 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.095+0000 7f81019f6740 1 -- 172.21.15.18:0/433778421 shutdown_connections 2024-02-17T13:06:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.095+0000 7f81019f6740 1 -- 172.21.15.18:0/433778421 wait complete. 2024-02-17T13:06:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.095+0000 7f81019f6740 20 client.20775 trim_cache size 0 max 16384 2024-02-17T13:06:11.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:10 smithi119 ceph-mon[36188]: pgmap v512: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2024-02-17T13:06:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.110+0000 7f81019f6740 1 Processor -- start 2024-02-17T13:06:11.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.111+0000 7f81019f6740 1 -- start start 2024-02-17T13:06:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.111+0000 7f81019f6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 0x55cd93d55e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.111+0000 7f81019f6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4dee0 0x55cd93d4f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.111+0000 7f81019f6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d566f0 0x55cd93d4fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.111+0000 7f81019f6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93d56550 con 0x55cd93d566f0 2024-02-17T13:06:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.111+0000 7f81019f6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93d53e00 con 0x55cd93d4d170 2024-02-17T13:06:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.111+0000 7f81019f6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93d54060 con 0x55cd93d4dee0 2024-02-17T13:06:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80f7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4dee0 0x55cd93d4f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80fcbbf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 0x55cd93d55e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80fcbbf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 0x55cd93d55e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:35132/0 (socket says 172.21.15.18:35132) 2024-02-17T13:06:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80f7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4dee0 0x55cd93d4f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:40100/0 (socket says 172.21.15.18:40100) 2024-02-17T13:06:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80fcbbf640 1 -- 172.21.15.18:0/631664710 learned_addr learned my addr 172.21.15.18:0/631664710 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80fd3c0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d566f0 0x55cd93d4fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80fcbbf640 1 -- 172.21.15.18:0/631664710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4dee0 msgr2=0x55cd93d4f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/631664710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4dee0 0x55cd93d4f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80fcbbf640 1 -- 172.21.15.18:0/631664710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d566f0 msgr2=0x55cd93d4fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/631664710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d566f0 0x55cd93d4fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80fcbbf640 1 -- 172.21.15.18:0/631664710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd93c4b7b0 con 0x55cd93d4d170 2024-02-17T13:06:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/631664710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 0x55cd93d55e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80f0005fa0 tx=0x7f80f0005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8b82a8783036284 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80ee7fc640 1 -- 172.21.15.18:0/631664710 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f009b3a0 con 0x55cd93d4d170 2024-02-17T13:06:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.112+0000 7f80ee7fc640 1 -- 172.21.15.18:0/631664710 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80f007b030 con 0x55cd93d4d170 2024-02-17T13:06:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.113+0000 7f80ee7fc640 1 -- 172.21.15.18:0/631664710 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f00cc300 con 0x55cd93d4d170 2024-02-17T13:06:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.113+0000 7f81019f6740 1 -- 172.21.15.18:0/631664710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 msgr2=0x55cd93d55e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.113+0000 7f81019f6740 1 --2- 172.21.15.18:0/631664710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 0x55cd93d55e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80f0005fa0 tx=0x7f80f0005f30 comp rx=0 tx=0).stop 2024-02-17T13:06:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.113+0000 7f81019f6740 1 -- 172.21.15.18:0/631664710 shutdown_connections 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.113+0000 7f81019f6740 1 --2- 172.21.15.18:0/631664710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d566f0 0x55cd93d4fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.113+0000 7f81019f6740 1 --2- 172.21.15.18:0/631664710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d4dee0 0x55cd93d4f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.113+0000 7f81019f6740 1 --2- 172.21.15.18:0/631664710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 0x55cd93d55e00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.113+0000 7f81019f6740 1 -- 172.21.15.18:0/631664710 >> 172.21.15.18:0/631664710 conn(0x55cd93d10550 msgr2=0x55cd93d47990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.113+0000 7f81019f6740 1 -- 172.21.15.18:0/631664710 shutdown_connections 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.113+0000 7f81019f6740 1 -- 172.21.15.18:0/631664710 wait complete. 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.114+0000 7f81019f6740 1 -- start start 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.114+0000 7f81019f6740 1 Processor -- start 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.114+0000 7f81019f6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 0x55cd93d66880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.114+0000 7f81019f6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4dee0 0x55cd93d66dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.114+0000 7f81019f6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d566f0 0x55cd93d67300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.114+0000 7f81019f6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93d619a0 con 0x55cd93d4dee0 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.114+0000 7f81019f6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93d61b10 con 0x55cd93d4d170 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.114+0000 7f81019f6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd93d61cd0 con 0x55cd93d566f0 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.114+0000 7f81019f6740 10 client.0 _ll_register_callbacks cb 0x7f80f0004860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:06:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.114+0000 7f80f7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4dee0 0x55cd93d66dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.114+0000 7f80fcbbf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 0x55cd93d66880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80f7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4dee0 0x55cd93d66dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36248/0 (socket says 172.21.15.18:36248) 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80fcbbf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 0x55cd93d66880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:35146/0 (socket says 172.21.15.18:35146) 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80fd3c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d566f0 0x55cd93d67300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80fcbbf640 1 -- 172.21.15.18:0/3118073803 learned_addr learned my addr 172.21.15.18:0/3118073803 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80fcbbf640 1 -- 172.21.15.18:0/3118073803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d566f0 msgr2=0x55cd93d67300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d566f0 0x55cd93d67300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80fcbbf640 1 -- 172.21.15.18:0/3118073803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4dee0 msgr2=0x55cd93d66dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4dee0 0x55cd93d66dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80fcbbf640 1 -- 172.21.15.18:0/3118073803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd93c467b0 con 0x55cd93d4d170 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80f7fff640 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4dee0 0x55cd93d66dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80fd3c0640 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d566f0 0x55cd93d67300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80fcbbf640 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 0x55cd93d66880 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f80f00a3590 tx=0x7f80f009ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2600fd857e3744e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-02-17T13:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80d7fff640 1 -- 172.21.15.18:0/3118073803 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0083070 con 0x55cd93d4d170 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80d7fff640 1 -- 172.21.15.18:0/3118073803 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f80f009b320 con 0x55cd93d4d170 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f81019f6740 1 -- 172.21.15.18:0/3118073803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80f009f470 con 0x55cd93d4d170 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f80d7fff640 1 -- 172.21.15.18:0/3118073803 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80f0099620 con 0x55cd93d4d170 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f81019f6740 10 client.29591 fetch_fsmap learned FSMap version 72 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.115+0000 7f81019f6740 1 -- 172.21.15.18:0/3118073803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55cd93c71540 con 0x55cd93d4d170 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.116+0000 7f80d7fff640 1 -- 172.21.15.18:0/3118073803 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80f009f470 con 0x55cd93d4d170 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.116+0000 7f80d7fff640 1 -- 172.21.15.18:0/3118073803 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 72) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f80f009a800 con 0x55cd93d4d170 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.116+0000 7f81019f6740 10 client.29591 fetch_fsmap finished waiting for FSMap version 72 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.116+0000 7f81019f6740 10 client.29591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.116+0000 7f81019f6740 20 client.29591 populate_metadata read hostname 'smithi018' 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.116+0000 7f81019f6740 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd93dc1bc0 0x55cd93dc3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.116+0000 7f81019f6740 4 client.29591 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.116+0000 7f81019f6740 1 -- 172.21.15.18:0/3118073803 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cd939ed550 con 0x55cd93dc1bc0 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.116+0000 7f80f7fff640 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd93dc1bc0 0x55cd93dc3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.116+0000 7f80d7fff640 10 client.29591 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:11.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:11.116+0000 7f80f7fff640 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd93dc1bc0 0x55cd93dc3fe0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c1e78fa98b9832e server_cookie=6d64922215dd27e in_seq=0 out_seq=0 2024-02-17T13:06:11.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:10 smithi146 ceph-mon[36454]: pgmap v512: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2024-02-17T13:06:11.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 134093 36.47 MB/sec execute 83 sec latency 4732.787 ms 2024-02-17T13:06:12.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 134287 36.18 MB/sec execute 84 sec latency 5732.858 ms 2024-02-17T13:06:13.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:12 smithi018 ceph-mon[32625]: pgmap v513: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 959 op/s 2024-02-17T13:06:13.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:12 smithi119 ceph-mon[36188]: pgmap v513: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 959 op/s 2024-02-17T13:06:13.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:12 smithi146 ceph-mon[36454]: pgmap v513: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 959 op/s 2024-02-17T13:06:13.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 134428 35.80 MB/sec execute 85 sec latency 6732.902 ms 2024-02-17T13:06:14.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:13 smithi119 ceph-mon[36188]: pgmap v514: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 959 op/s 2024-02-17T13:06:14.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:13 smithi146 ceph-mon[36454]: pgmap v514: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 959 op/s 2024-02-17T13:06:14.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:13 smithi018 ceph-mon[32625]: pgmap v514: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 959 op/s 2024-02-17T13:06:14.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 134530 35.48 MB/sec execute 86 sec latency 7732.934 ms 2024-02-17T13:06:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.062+0000 7f80d7fff640 1 -- 172.21.15.18:0/3118073803 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cd939ed550 con 0x55cd93dc1bc0 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.062+0000 7f80d7fff640 10 client.29591 handle_command_reply: tid=0 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.062+0000 7f81019f6740 1 client.29591 shutdown 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.063+0000 7f81019f6740 1 -- 172.21.15.18:0/3118073803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 msgr2=0x55cd93d66880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.063+0000 7f81019f6740 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 0x55cd93d66880 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f80f00a3590 tx=0x7f80f009ad10 comp rx=0 tx=0).stop 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.063+0000 7f81019f6740 1 -- 172.21.15.18:0/3118073803 shutdown_connections 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.063+0000 7f81019f6740 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55cd93dc1bc0 0x55cd93dc3fe0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.063+0000 7f81019f6740 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd93d566f0 0x55cd93d67300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.063+0000 7f81019f6740 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd93d4dee0 0x55cd93d66dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.063+0000 7f81019f6740 1 --2- 172.21.15.18:0/3118073803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd93d4d170 0x55cd93d66880 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.063+0000 7f80d7fff640 0 client.29591 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.063+0000 7f81019f6740 1 -- 172.21.15.18:0/3118073803 >> 172.21.15.18:0/3118073803 conn(0x55cd93d10550 msgr2=0x55cd93d4e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.063+0000 7f81019f6740 1 -- 172.21.15.18:0/3118073803 shutdown_connections 2024-02-17T13:06:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.063+0000 7f81019f6740 1 -- 172.21.15.18:0/3118073803 wait complete. 2024-02-17T13:06:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.063+0000 7f81019f6740 20 client.29591 trim_cache size 0 max 16384 2024-02-17T13:06:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:06:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:06:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:06:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:06:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.064+0000 7f80fee23640 1 -- 172.21.15.18:0/1468988481 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f80e40a22d0 msgr2=0x7f80e40a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.064+0000 7f80fee23640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f80e40a22d0 0x7f80e40a4790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f80e802c200 tx=0x7f80e8004b50 comp rx=0 tx=0).stop 2024-02-17T13:06:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.064+0000 7f80fee23640 1 -- 172.21.15.18:0/1468988481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80f81023f0 msgr2=0x7f80f8197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.064+0000 7f80fee23640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80f81023f0 0x7f80f8197730 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009d50 tx=0x7f80f000c6a0 comp rx=0 tx=0).stop 2024-02-17T13:06:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.064+0000 7f80fee23640 1 -- 172.21.15.18:0/1468988481 shutdown_connections 2024-02-17T13:06:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.064+0000 7f80fee23640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f80e40a22d0 0x7f80e40a4790 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.064+0000 7f80fee23640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f810b7c0 0x7f80f819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.064+0000 7f80fee23640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80f8102d10 0x7f80f8197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.064+0000 7f80fee23640 1 --2- 172.21.15.18:0/1468988481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80f81023f0 0x7f80f8197730 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.064+0000 7f80fee23640 1 -- 172.21.15.18:0/1468988481 >> 172.21.15.18:0/1468988481 conn(0x7f80f80771a0 msgr2=0x7f80f80fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.064+0000 7f80fee23640 1 -- 172.21.15.18:0/1468988481 shutdown_connections 2024-02-17T13:06:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:15.064+0000 7f80fee23640 1 -- 172.21.15.18:0/1468988481 wait complete. 2024-02-17T13:06:15.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:14 smithi146 ceph-mon[36454]: osdmap e111: 12 total, 12 up, 12 in 2024-02-17T13:06:15.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:14 smithi018 ceph-mon[32625]: osdmap e111: 12 total, 12 up, 12 in 2024-02-17T13:06:15.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:14 smithi119 ceph-mon[36188]: osdmap e111: 12 total, 12 up, 12 in 2024-02-17T13:06:15.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 134872 35.24 MB/sec execute 87 sec latency 8732.981 ms 2024-02-17T13:06:16.172 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:15 smithi018 ceph-mon[32625]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 769 op/s 2024-02-17T13:06:16.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:15 smithi146 ceph-mon[36454]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 769 op/s 2024-02-17T13:06:16.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:15 smithi119 ceph-mon[36188]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 769 op/s 2024-02-17T13:06:16.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 135463 35.22 MB/sec execute 88 sec latency 8969.816 ms 2024-02-17T13:06:17.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:16 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:17.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:16 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:17.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:16 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:17.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 135787 35.03 MB/sec execute 89 sec latency 8246.701 ms 2024-02-17T13:06:18.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:17 smithi146 ceph-mon[36454]: pgmap v517: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 704 op/s 2024-02-17T13:06:18.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:18 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:06:18.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:17 smithi018 ceph-mon[32625]: pgmap v517: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 704 op/s 2024-02-17T13:06:18.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:17 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:06:18.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:17 smithi119 ceph-mon[36188]: pgmap v517: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 704 op/s 2024-02-17T13:06:18.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:17 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:06:18.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 136169 34.80 MB/sec execute 90 sec latency 8728.178 ms 2024-02-17T13:06:19.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 136804 34.87 MB/sec execute 91 sec latency 131.336 ms 2024-02-17T13:06:20.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:19 smithi146 ceph-mon[36454]: pgmap v518: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 664 op/s 2024-02-17T13:06:20.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:19 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:06:20.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:19 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:06:20.198 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:19 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:06:20.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:19 smithi018 ceph-mon[32625]: pgmap v518: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 664 op/s 2024-02-17T13:06:20.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:19 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:06:20.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:19 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:06:20.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:19 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:06:20.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:19 smithi119 ceph-mon[36188]: pgmap v518: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 664 op/s 2024-02-17T13:06:20.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:19 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:06:20.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:19 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:06:20.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:19 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:06:20.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 137549 34.90 MB/sec execute 92 sec latency 96.269 ms 2024-02-17T13:06:21.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 138153 34.90 MB/sec execute 93 sec latency 89.268 ms 2024-02-17T13:06:22.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:21 smithi146 ceph-mon[36454]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 690 op/s 2024-02-17T13:06:22.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:21 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:22.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:21 smithi018 ceph-mon[32625]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 690 op/s 2024-02-17T13:06:22.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:21 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:22.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:21 smithi119 ceph-mon[36188]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 690 op/s 2024-02-17T13:06:22.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:21 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:22.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 138563 34.70 MB/sec execute 94 sec latency 263.103 ms 2024-02-17T13:06:23.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 139220 34.74 MB/sec execute 95 sec latency 82.932 ms 2024-02-17T13:06:24.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:23 smithi146 ceph-mon[36454]: pgmap v520: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 690 op/s 2024-02-17T13:06:24.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:23 smithi018 ceph-mon[32625]: pgmap v520: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 690 op/s 2024-02-17T13:06:24.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:23 smithi119 ceph-mon[36188]: pgmap v520: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 690 op/s 2024-02-17T13:06:24.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 139874 34.75 MB/sec execute 96 sec latency 105.589 ms 2024-02-17T13:06:25.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 140707 34.82 MB/sec execute 97 sec latency 107.964 ms 2024-02-17T13:06:26.197 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:25 smithi146 ceph-mon[36454]: pgmap v521: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 835 op/s 2024-02-17T13:06:26.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:25 smithi018 ceph-mon[32625]: pgmap v521: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 835 op/s 2024-02-17T13:06:26.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:25 smithi119 ceph-mon[36188]: pgmap v521: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 835 op/s 2024-02-17T13:06:26.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 141379 34.94 MB/sec execute 98 sec latency 84.874 ms 2024-02-17T13:06:27.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 141743 34.66 MB/sec execute 99 sec latency 338.649 ms 2024-02-17T13:06:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.196+0000 7fc84943b640 1 -- 172.21.15.18:0/1941054873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8441013a0 msgr2=0x7fc844101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.196+0000 7fc84943b640 1 --2- 172.21.15.18:0/1941054873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8441013a0 0x7fc844101780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc834009c10 tx=0x7fc83402bef0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.196+0000 7fc84943b640 1 -- 172.21.15.18:0/1941054873 shutdown_connections 2024-02-17T13:06:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.196+0000 7fc84943b640 1 --2- 172.21.15.18:0/1941054873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc844106470 0x7fc84410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.196+0000 7fc84943b640 1 --2- 172.21.15.18:0/1941054873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc844101d50 0x7fc844105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.196+0000 7fc84943b640 1 --2- 172.21.15.18:0/1941054873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8441013a0 0x7fc844101780 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.196+0000 7fc84943b640 1 -- 172.21.15.18:0/1941054873 >> 172.21.15.18:0/1941054873 conn(0x7fc8440fcc40 msgr2=0x7fc8440ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.197+0000 7fc84943b640 1 -- 172.21.15.18:0/1941054873 shutdown_connections 2024-02-17T13:06:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.197+0000 7fc84943b640 1 -- 172.21.15.18:0/1941054873 wait complete. 2024-02-17T13:06:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc84943b640 1 Processor -- start 2024-02-17T13:06:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc84943b640 1 -- start start 2024-02-17T13:06:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc84943b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8441013a0 0x7fc844197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc84943b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc844106470 0x7fc844197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc84943b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84419be50 0x7fc84419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc84943b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84419c840 con 0x7fc84419be50 2024-02-17T13:06:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc84943b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84419c9b0 con 0x7fc8441013a0 2024-02-17T13:06:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc84943b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84419cb20 con 0x7fc844106470 2024-02-17T13:06:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc842ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8441013a0 0x7fc844197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc8437fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84419be50 0x7fc84419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc842ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8441013a0 0x7fc844197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:53376/0 (socket says 172.21.15.18:53376) 2024-02-17T13:06:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc842ffd640 1 -- 172.21.15.18:0/2529246916 learned_addr learned my addr 172.21.15.18:0/2529246916 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.198+0000 7fc83a7fc640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc844106470 0x7fc844197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.199+0000 7fc8437fe640 1 -- 172.21.15.18:0/2529246916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc844106470 msgr2=0x7fc844197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.199+0000 7fc8437fe640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc844106470 0x7fc844197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.199+0000 7fc8437fe640 1 -- 172.21.15.18:0/2529246916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8441013a0 msgr2=0x7fc844197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.199+0000 7fc8437fe640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8441013a0 0x7fc844197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.199+0000 7fc8437fe640 1 -- 172.21.15.18:0/2529246916 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc834009840 con 0x7fc84419be50 2024-02-17T13:06:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.199+0000 7fc83a7fc640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc844106470 0x7fc844197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:06:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.200+0000 7fc8437fe640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84419be50 0x7fc84419c300 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00d680 tx=0x7fc83c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.200+0000 7fc840ff9640 1 -- 172.21.15.18:0/2529246916 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c005220 con 0x7fc84419be50 2024-02-17T13:06:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.200+0000 7fc84943b640 1 -- 172.21.15.18:0/2529246916 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8440733b0 con 0x7fc84419be50 2024-02-17T13:06:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.200+0000 7fc840ff9640 1 -- 172.21.15.18:0/2529246916 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc83c009d00 con 0x7fc84419be50 2024-02-17T13:06:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.200+0000 7fc84943b640 1 -- 172.21.15.18:0/2529246916 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc844073860 con 0x7fc84419be50 2024-02-17T13:06:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.201+0000 7fc840ff9640 1 -- 172.21.15.18:0/2529246916 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c004e00 con 0x7fc84419be50 2024-02-17T13:06:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.204+0000 7fc840ff9640 1 -- 172.21.15.18:0/2529246916 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc83c010460 con 0x7fc84419be50 2024-02-17T13:06:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.205+0000 7fc84c005740 1 Processor -- start 2024-02-17T13:06:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.206+0000 7fc840ff9640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc8300a2350 0x7fc8300a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.206+0000 7fc840ff9640 1 -- 172.21.15.18:0/2529246916 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc83c0cd770 con 0x7fc84419be50 2024-02-17T13:06:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.206+0000 7fc84c005740 1 -- start start 2024-02-17T13:06:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.206+0000 7fc84c005740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d8ad0 0x55a8635d7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.206+0000 7fc84c005740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635d81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.206+0000 7fc84c005740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635e9af0 0x55a8635ebee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.206+0000 7fc84c005740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8635469f0 con 0x55a8635d8ad0 2024-02-17T13:06:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.206+0000 7fc84c005740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a863543f20 con 0x55a8635e9af0 2024-02-17T13:06:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.206+0000 7fc84c005740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86353f700 con 0x55a8635d7d60 2024-02-17T13:06:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.207+0000 7fc8437fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635e9af0 0x55a8635ebee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.207+0000 7fc8437fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635e9af0 0x55a8635ebee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:53390/0 (socket says 172.21.15.18:53390) 2024-02-17T13:06:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.207+0000 7fc8437fe640 1 -- 172.21.15.18:0/804561061 learned_addr learned my addr 172.21.15.18:0/804561061 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.207+0000 7fc8437fe640 1 -- 172.21.15.18:0/804561061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 msgr2=0x55a8635d81e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T13:06:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.207+0000 7fc8437fe640 1 --2- 172.21.15.18:0/804561061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635d81e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.207+0000 7fc8437fe640 1 -- 172.21.15.18:0/804561061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d8ad0 msgr2=0x55a8635d7820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T13:06:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.207+0000 7fc8437fe640 1 --2- 172.21.15.18:0/804561061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d8ad0 0x55a8635d7820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.207+0000 7fc8437fe640 1 -- 172.21.15.18:0/804561061 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8634d57b0 con 0x55a8635e9af0 2024-02-17T13:06:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.208+0000 7fc8437fe640 1 --2- 172.21.15.18:0/804561061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635e9af0 0x55a8635ebee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc83c09a110 tx=0x7fc83c099ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e755eac6d5fd304e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.208+0000 7fc83a7fc640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc8300a2350 0x7fc8300a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.208+0000 7fc8397fa640 1 -- 172.21.15.18:0/804561061 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c0a2920 con 0x55a8635e9af0 2024-02-17T13:06:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.208+0000 7fc8397fa640 1 -- 172.21.15.18:0/804561061 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc83c098480 con 0x55a8635e9af0 2024-02-17T13:06:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.208+0000 7fc8397fa640 1 -- 172.21.15.18:0/804561061 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c0a2490 con 0x55a8635e9af0 2024-02-17T13:06:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.209+0000 7fc83a7fc640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc8300a2350 0x7fc8300a4810 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55a8635d9230 tx=0x7fc82c00b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.209+0000 7fc84c005740 1 -- 172.21.15.18:0/804561061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635e9af0 msgr2=0x55a8635ebee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.209+0000 7fc84c005740 1 --2- 172.21.15.18:0/804561061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635e9af0 0x55a8635ebee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc83c09a110 tx=0x7fc83c099ba0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.209+0000 7fc84c005740 1 -- 172.21.15.18:0/804561061 shutdown_connections 2024-02-17T13:06:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.209+0000 7fc84c005740 1 --2- 172.21.15.18:0/804561061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635e9af0 0x55a8635ebee0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.209+0000 7fc84c005740 1 --2- 172.21.15.18:0/804561061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635d81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.209+0000 7fc84c005740 1 --2- 172.21.15.18:0/804561061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d8ad0 0x55a8635d7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.209+0000 7fc84c005740 1 -- 172.21.15.18:0/804561061 >> 172.21.15.18:0/804561061 conn(0x55a86359f550 msgr2=0x55a8635d2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.210+0000 7fc84c005740 1 -- 172.21.15.18:0/804561061 shutdown_connections 2024-02-17T13:06:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.210+0000 7fc84c005740 1 -- 172.21.15.18:0/804561061 wait complete. 2024-02-17T13:06:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.210+0000 7fc84c005740 1 -- start start 2024-02-17T13:06:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.210+0000 7fc84c005740 1 Processor -- start 2024-02-17T13:06:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.211+0000 7fc84c005740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635e6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.211+0000 7fc84c005740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635d8ad0 0x55a8635e7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.211+0000 7fc842ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635e6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.211+0000 7fc842ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635e6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:38224/0 (socket says 172.21.15.18:38224) 2024-02-17T13:06:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.211+0000 7fc842ffd640 1 -- 172.21.15.18:0/4175687842 learned_addr learned my addr 172.21.15.18:0/4175687842 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.211+0000 7fc84c005740 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635e9af0 0x55a8635e7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.211+0000 7fc84c005740 1 -- 172.21.15.18:0/4175687842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8634e4bf0 con 0x55a8635e9af0 2024-02-17T13:06:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.211+0000 7fc8437fe640 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635e9af0 0x55a8635e7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.211+0000 7fc83a7fc640 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635d8ad0 0x55a8635e7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.211+0000 7fc84c005740 1 -- 172.21.15.18:0/4175687842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a863213250 con 0x55a8635d8ad0 2024-02-17T13:06:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.212+0000 7fc84c005740 1 -- 172.21.15.18:0/4175687842 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8634c5200 con 0x55a8635d7d60 2024-02-17T13:06:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.212+0000 7fc84c005740 10 client.0 _ll_register_callbacks cb 0x55a863533d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:06:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.212+0000 7fc842ffd640 1 -- 172.21.15.18:0/4175687842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635d8ad0 msgr2=0x55a8635e7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.212+0000 7fc842ffd640 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635d8ad0 0x55a8635e7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.212+0000 7fc842ffd640 1 -- 172.21.15.18:0/4175687842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635e9af0 msgr2=0x55a8635e7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.212+0000 7fc842ffd640 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635e9af0 0x55a8635e7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.212+0000 7fc842ffd640 1 -- 172.21.15.18:0/4175687842 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc844073860 con 0x55a8635d7d60 2024-02-17T13:06:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.212+0000 7fc842ffd640 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635e6b10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc834033e20 tx=0x7fc834033e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1c4de05c0b328a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.212+0000 7fc82a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:06:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.213+0000 7fc82a7fc640 1 -- 172.21.15.18:0/4175687842 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834033670 con 0x55a8635d7d60 2024-02-17T13:06:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.213+0000 7fc82a7fc640 1 -- 172.21.15.18:0/4175687842 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc8340337d0 con 0x55a8635d7d60 2024-02-17T13:06:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.213+0000 7fc84c005740 1 -- 172.21.15.18:0/4175687842 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a86323f600 con 0x55a8635d7d60 2024-02-17T13:06:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.213+0000 7fc82a7fc640 1 -- 172.21.15.18:0/4175687842 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc834003970 con 0x55a8635d7d60 2024-02-17T13:06:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.213+0000 7fc84c005740 10 client.29716 fetch_fsmap learned FSMap version 74 2024-02-17T13:06:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.213+0000 7fc84c005740 1 -- 172.21.15.18:0/4175687842 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55a86353dd80 con 0x55a8635d7d60 2024-02-17T13:06:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.214+0000 7fc82a7fc640 1 -- 172.21.15.18:0/4175687842 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a86323f600 con 0x55a8635d7d60 2024-02-17T13:06:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.214+0000 7fc82a7fc640 1 -- 172.21.15.18:0/4175687842 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 74) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc83403c040 con 0x55a8635d7d60 2024-02-17T13:06:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.214+0000 7fc84c005740 10 client.29716 fetch_fsmap finished waiting for FSMap version 74 2024-02-17T13:06:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.214+0000 7fc84c005740 10 client.29716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:06:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.214+0000 7fc84c005740 20 client.29716 populate_metadata read hostname 'smithi018' 2024-02-17T13:06:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.214+0000 7fc84c005740 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a8635f9e00 0x55a8635fc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.214+0000 7fc84c005740 4 client.29716 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:06:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.214+0000 7fc84c005740 1 -- 172.21.15.18:0/4175687842 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8632901b0 con 0x55a8635f9e00 2024-02-17T13:06:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.215+0000 7fc83a7fc640 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a8635f9e00 0x55a8635fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.222+0000 7fc82a7fc640 10 client.29716 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.222+0000 7fc83a7fc640 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a8635f9e00 0x55a8635fc220 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f2585d79458552b server_cookie=19b32b061ccfa2c in_seq=0 out_seq=0 2024-02-17T13:06:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc82a7fc640 1 -- 172.21.15.18:0/4175687842 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a8632901b0 con 0x55a8635f9e00 2024-02-17T13:06:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc82a7fc640 10 client.29716 handle_command_reply: tid=0 2024-02-17T13:06:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc84c005740 1 client.29716 shutdown 2024-02-17T13:06:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc84c005740 1 -- 172.21.15.18:0/4175687842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 msgr2=0x55a8635e6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc84c005740 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635e6b10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc834033e20 tx=0x7fc834033e50 comp rx=0 tx=0).stop 2024-02-17T13:06:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc84c005740 1 -- 172.21.15.18:0/4175687842 shutdown_connections 2024-02-17T13:06:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc84c005740 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a8635f9e00 0x55a8635fc220 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc84c005740 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635e9af0 0x55a8635e7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc82a7fc640 0 client.29716 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc84c005740 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635d8ad0 0x55a8635e7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc84c005740 1 --2- 172.21.15.18:0/4175687842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635e6b10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc84c005740 1 -- 172.21.15.18:0/4175687842 >> 172.21.15.18:0/4175687842 conn(0x55a86359f550 msgr2=0x55a8635d47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc84c005740 1 -- 172.21.15.18:0/4175687842 shutdown_connections 2024-02-17T13:06:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc84c005740 1 -- 172.21.15.18:0/4175687842 wait complete. 2024-02-17T13:06:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.224+0000 7fc84c005740 20 client.29716 trim_cache size 0 max 16384 2024-02-17T13:06:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.243+0000 7fc84c005740 1 Processor -- start 2024-02-17T13:06:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.243+0000 7fc84c005740 1 -- start start 2024-02-17T13:06:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.243+0000 7fc84c005740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d7d60 0x55a8635e9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc84c005740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635d8ad0 0x55a8635e73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc84c005740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635e9af0 0x55a8635e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc84c005740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8635ec700 con 0x55a8635d7d60 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc84c005740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8635f6970 con 0x55a8635d8ad0 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc84c005740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8636013d0 con 0x55a8635e9af0 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc842ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d7d60 0x55a8635e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc842ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d7d60 0x55a8635e9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47738/0 (socket says 172.21.15.18:47738) 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc842ffd640 1 -- 172.21.15.18:0/1040919063 learned_addr learned my addr 172.21.15.18:0/1040919063 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc8437fe640 1 --2- 172.21.15.18:0/1040919063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635e9af0 0x55a8635e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc83a7fc640 1 --2- 172.21.15.18:0/1040919063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635d8ad0 0x55a8635e73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc842ffd640 1 -- 172.21.15.18:0/1040919063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635e9af0 msgr2=0x55a8635e3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc842ffd640 1 --2- 172.21.15.18:0/1040919063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635e9af0 0x55a8635e3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc842ffd640 1 -- 172.21.15.18:0/1040919063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635d8ad0 msgr2=0x55a8635e73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc842ffd640 1 --2- 172.21.15.18:0/1040919063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635d8ad0 0x55a8635e73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc842ffd640 1 -- 172.21.15.18:0/1040919063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8634da7b0 con 0x55a8635d7d60 2024-02-17T13:06:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.244+0000 7fc8437fe640 1 --2- 172.21.15.18:0/1040919063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635e9af0 0x55a8635e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:06:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.245+0000 7fc842ffd640 1 --2- 172.21.15.18:0/1040919063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d7d60 0x55a8635e9240 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fc834002230 tx=0x7fc834009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ec6edd7514c02f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.245+0000 7fc838ff9640 1 -- 172.21.15.18:0/1040919063 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83403e770 con 0x55a8635d7d60 2024-02-17T13:06:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.245+0000 7fc838ff9640 1 -- 172.21.15.18:0/1040919063 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc83403e8d0 con 0x55a8635d7d60 2024-02-17T13:06:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.246+0000 7fc838ff9640 1 -- 172.21.15.18:0/1040919063 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834034910 con 0x55a8635d7d60 2024-02-17T13:06:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.246+0000 7fc84c005740 1 -- 172.21.15.18:0/1040919063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d7d60 msgr2=0x55a8635e9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.246+0000 7fc84c005740 1 --2- 172.21.15.18:0/1040919063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d7d60 0x55a8635e9240 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fc834002230 tx=0x7fc834009aa0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.247+0000 7fc84c005740 1 -- 172.21.15.18:0/1040919063 shutdown_connections 2024-02-17T13:06:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.247+0000 7fc84c005740 1 --2- 172.21.15.18:0/1040919063 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635e9af0 0x55a8635e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.247+0000 7fc84c005740 1 --2- 172.21.15.18:0/1040919063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635d8ad0 0x55a8635e73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.247+0000 7fc84c005740 1 --2- 172.21.15.18:0/1040919063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d7d60 0x55a8635e9240 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.247+0000 7fc84c005740 1 -- 172.21.15.18:0/1040919063 >> 172.21.15.18:0/1040919063 conn(0x55a86359f550 msgr2=0x55a8635e53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.247+0000 7fc84c005740 1 -- 172.21.15.18:0/1040919063 shutdown_connections 2024-02-17T13:06:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.247+0000 7fc84c005740 1 -- 172.21.15.18:0/1040919063 wait complete. 2024-02-17T13:06:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.248+0000 7fc84c005740 1 -- start start 2024-02-17T13:06:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.248+0000 7fc84c005740 1 Processor -- start 2024-02-17T13:06:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.248+0000 7fc84c005740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.249+0000 7fc84c005740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d8ad0 0x55a8635ef1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.249+0000 7fc84c005740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635e9af0 0x55a8635ef910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.249+0000 7fc842ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.249+0000 7fc842ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635f0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:38240/0 (socket says 172.21.15.18:38240) 2024-02-17T13:06:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.249+0000 7fc842ffd640 1 -- 172.21.15.18:0/952269751 learned_addr learned my addr 172.21.15.18:0/952269751 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.249+0000 7fc84c005740 1 -- 172.21.15.18:0/952269751 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8635effe0 con 0x55a8635d8ad0 2024-02-17T13:06:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.249+0000 7fc84c005740 1 -- 172.21.15.18:0/952269751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8636502d0 con 0x55a8635e9af0 2024-02-17T13:06:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.249+0000 7fc84c005740 1 -- 172.21.15.18:0/952269751 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a863650410 con 0x55a8635d7d60 2024-02-17T13:06:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.250+0000 7fc83a7fc640 1 --2- 172.21.15.18:0/952269751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d8ad0 0x55a8635ef1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.250+0000 7fc84c005740 10 client.0 _ll_register_callbacks cb 0x7fc83c009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:06:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.250+0000 7fc842ffd640 1 -- 172.21.15.18:0/952269751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635e9af0 msgr2=0x55a8635ef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.250+0000 7fc842ffd640 1 --2- 172.21.15.18:0/952269751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635e9af0 0x55a8635ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.250+0000 7fc842ffd640 1 -- 172.21.15.18:0/952269751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d8ad0 msgr2=0x55a8635ef1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.250+0000 7fc842ffd640 1 --2- 172.21.15.18:0/952269751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d8ad0 0x55a8635ef1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.250+0000 7fc842ffd640 1 -- 172.21.15.18:0/952269751 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a86353dd80 con 0x55a8635d7d60 2024-02-17T13:06:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.250+0000 7fc84c005740 1 -- 172.21.15.18:0/952269751 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc83c098480 con 0x55a8635d7d60 2024-02-17T13:06:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.250+0000 7fc842ffd640 1 --2- 172.21.15.18:0/952269751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635f0e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc834006010 tx=0x7fc834003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc861aa89e6afe6c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.250+0000 7fc82a7fc640 10 client.29722 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:06:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.250+0000 7fc82a7fc640 1 -- 172.21.15.18:0/952269751 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83403e770 con 0x55a8635d7d60 2024-02-17T13:06:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.251+0000 7fc82a7fc640 1 -- 172.21.15.18:0/952269751 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc834004b40 con 0x55a8635d7d60 2024-02-17T13:06:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.251+0000 7fc82a7fc640 1 -- 172.21.15.18:0/952269751 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc834004d40 con 0x55a8635d7d60 2024-02-17T13:06:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.251+0000 7fc84c005740 10 client.29722 fetch_fsmap learned FSMap version 74 2024-02-17T13:06:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.251+0000 7fc84c005740 1 -- 172.21.15.18:0/952269751 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({fsmap=74}) v3 -- 0x55a863500540 con 0x55a8635d7d60 2024-02-17T13:06:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.251+0000 7fc82a7fc640 1 -- 172.21.15.18:0/952269751 <== mon.1 v2:172.21.15.146:3300/0 4 ==== fsmap(e 74) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc834061070 con 0x55a8635d7d60 2024-02-17T13:06:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.251+0000 7fc84c005740 10 client.29722 fetch_fsmap finished waiting for FSMap version 74 2024-02-17T13:06:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.251+0000 7fc84c005740 10 client.29722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:06:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.251+0000 7fc84c005740 20 client.29722 populate_metadata read hostname 'smithi018' 2024-02-17T13:06:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.251+0000 7fc84c005740 1 --2- 172.21.15.18:0/952269751 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a863651540 0x55a863653960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.251+0000 7fc84c005740 4 client.29722 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:06:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.251+0000 7fc84c005740 1 -- 172.21.15.18:0/952269751 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a86327c550 con 0x55a863651540 2024-02-17T13:06:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.252+0000 7fc83a7fc640 1 --2- 172.21.15.18:0/952269751 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a863651540 0x55a863653960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.253+0000 7fc82a7fc640 10 client.29722 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.253+0000 7fc83a7fc640 1 --2- 172.21.15.18:0/952269751 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a863651540 0x55a863653960 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87427f3e208c6a3 server_cookie=4ffec1abfcc0585f in_seq=0 out_seq=0 2024-02-17T13:06:28.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:27 smithi018 ceph-mon[32625]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.8 MiB/s wr, 597 op/s 2024-02-17T13:06:28.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:27 smithi119 ceph-mon[36188]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.8 MiB/s wr, 597 op/s 2024-02-17T13:06:28.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:27 smithi146 ceph-mon[36454]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.8 MiB/s wr, 597 op/s 2024-02-17T13:06:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc82a7fc640 1 -- 172.21.15.18:0/952269751 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a86327c550 con 0x55a863651540 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc82a7fc640 10 client.29722 handle_command_reply: tid=0 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc84c005740 1 client.29722 shutdown 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc84c005740 1 -- 172.21.15.18:0/952269751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 msgr2=0x55a8635f0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc84c005740 1 --2- 172.21.15.18:0/952269751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635f0e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc834006010 tx=0x7fc834003c20 comp rx=0 tx=0).stop 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc84c005740 1 -- 172.21.15.18:0/952269751 shutdown_connections 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc84c005740 1 --2- 172.21.15.18:0/952269751 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55a863651540 0x55a863653960 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc84c005740 1 --2- 172.21.15.18:0/952269751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a8635e9af0 0x55a8635ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc84c005740 1 --2- 172.21.15.18:0/952269751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8635d8ad0 0x55a8635ef1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc84c005740 1 --2- 172.21.15.18:0/952269751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a8635d7d60 0x55a8635f0e30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc82a7fc640 0 client.29722 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc84c005740 1 -- 172.21.15.18:0/952269751 >> 172.21.15.18:0/952269751 conn(0x55a86359f550 msgr2=0x55a8635d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc84c005740 1 -- 172.21.15.18:0/952269751 shutdown_connections 2024-02-17T13:06:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc84c005740 1 -- 172.21.15.18:0/952269751 wait complete. 2024-02-17T13:06:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.567+0000 7fc84c005740 20 client.29722 trim_cache size 0 max 16384 2024-02-17T13:06:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:06:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:06:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:06:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:06:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.568+0000 7fc84943b640 1 -- 172.21.15.18:0/2529246916 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc8300a2350 msgr2=0x7fc8300a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.568+0000 7fc84943b640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc8300a2350 0x7fc8300a4810 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55a8635d9230 tx=0x7fc82c00b390 comp rx=0 tx=0).stop 2024-02-17T13:06:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.568+0000 7fc84943b640 1 -- 172.21.15.18:0/2529246916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84419be50 msgr2=0x7fc84419c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.568+0000 7fc84943b640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84419be50 0x7fc84419c300 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00d680 tx=0x7fc83c00db40 comp rx=0 tx=0).stop 2024-02-17T13:06:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.568+0000 7fc84943b640 1 -- 172.21.15.18:0/2529246916 shutdown_connections 2024-02-17T13:06:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.568+0000 7fc84943b640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc8300a2350 0x7fc8300a4810 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.568+0000 7fc84943b640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84419be50 0x7fc84419c300 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.568+0000 7fc84943b640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc844106470 0x7fc844197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.568+0000 7fc84943b640 1 --2- 172.21.15.18:0/2529246916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8441013a0 0x7fc844197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.568+0000 7fc84943b640 1 -- 172.21.15.18:0/2529246916 >> 172.21.15.18:0/2529246916 conn(0x7fc8440fcc40 msgr2=0x7fc8440fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.568+0000 7fc84943b640 1 -- 172.21.15.18:0/2529246916 shutdown_connections 2024-02-17T13:06:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:28.568+0000 7fc84943b640 1 -- 172.21.15.18:0/2529246916 wait complete. 2024-02-17T13:06:28.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 142227 34.62 MB/sec execute 100 sec latency 196.492 ms 2024-02-17T13:06:29.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 142881 34.55 MB/sec execute 101 sec latency 85.610 ms 2024-02-17T13:06:30.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:29 smithi018 ceph-mon[32625]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 728 op/s 2024-02-17T13:06:30.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:29 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:30.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:29 smithi119 ceph-mon[36188]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 728 op/s 2024-02-17T13:06:30.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:29 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:30.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:29 smithi146 ceph-mon[36454]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 728 op/s 2024-02-17T13:06:30.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:29 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:30.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 143620 34.69 MB/sec execute 102 sec latency 93.855 ms 2024-02-17T13:06:31.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 144184 34.65 MB/sec execute 103 sec latency 91.438 ms 2024-02-17T13:06:32.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:31 smithi018 ceph-mon[32625]: pgmap v524: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 922 op/s 2024-02-17T13:06:32.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:31 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:32.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:31 smithi119 ceph-mon[36188]: pgmap v524: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 922 op/s 2024-02-17T13:06:32.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:31 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:32.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:31 smithi146 ceph-mon[36454]: pgmap v524: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 922 op/s 2024-02-17T13:06:32.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:31 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:32.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 144652 34.55 MB/sec execute 104 sec latency 281.032 ms 2024-02-17T13:06:33.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 145244 34.59 MB/sec execute 105 sec latency 377.256 ms 2024-02-17T13:06:34.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:33 smithi018 ceph-mon[32625]: pgmap v525: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 727 op/s 2024-02-17T13:06:34.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:33 smithi119 ceph-mon[36188]: pgmap v525: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 727 op/s 2024-02-17T13:06:34.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:33 smithi146 ceph-mon[36454]: pgmap v525: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 727 op/s 2024-02-17T13:06:34.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 145811 34.43 MB/sec execute 106 sec latency 237.289 ms 2024-02-17T13:06:35.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:35 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:35.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:35 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:35.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:35 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:35.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 146542 34.57 MB/sec execute 107 sec latency 83.544 ms 2024-02-17T13:06:36.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:36 smithi146 ceph-mon[36454]: pgmap v526: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 1.01k op/s 2024-02-17T13:06:36.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:36 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:36.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:36 smithi018 ceph-mon[32625]: pgmap v526: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 1.01k op/s 2024-02-17T13:06:36.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:36 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:36.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:36 smithi119 ceph-mon[36188]: pgmap v526: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 1.01k op/s 2024-02-17T13:06:36.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:36 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:36.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 147098 34.45 MB/sec execute 108 sec latency 101.026 ms 2024-02-17T13:06:37.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 147621 34.38 MB/sec execute 109 sec latency 168.998 ms 2024-02-17T13:06:38.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:38 smithi146 ceph-mon[36454]: pgmap v527: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 694 op/s 2024-02-17T13:06:38.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:38 smithi018 ceph-mon[32625]: pgmap v527: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 694 op/s 2024-02-17T13:06:38.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:38 smithi119 ceph-mon[36188]: pgmap v527: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 694 op/s 2024-02-17T13:06:38.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 148182 34.45 MB/sec execute 110 sec latency 381.254 ms 2024-02-17T13:06:39.793 INFO:tasks.workunit.client.0.smithi018.stdout: 10 148765 34.36 MB/sec execute 111 sec latency 1348.733 ms 2024-02-17T13:06:40.392 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:40 smithi119 ceph-mon[36188]: pgmap v528: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 754 op/s 2024-02-17T13:06:40.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:40 smithi146 ceph-mon[36454]: pgmap v528: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 754 op/s 2024-02-17T13:06:40.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:40 smithi018 ceph-mon[32625]: pgmap v528: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 754 op/s 2024-02-17T13:06:40.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 149463 34.49 MB/sec execute 112 sec latency 2348.768 ms 2024-02-17T13:06:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.665+0000 7fb7bed63640 1 -- 172.21.15.18:0/1436853423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7b81013a0 msgr2=0x7fb7b8101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.665+0000 7fb7bed63640 1 --2- 172.21.15.18:0/1436853423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7b81013a0 0x7fb7b8101780 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009900 tx=0x7fb7ac02bc30 comp rx=0 tx=0).stop 2024-02-17T13:06:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.665+0000 7fb7bed63640 1 -- 172.21.15.18:0/1436853423 shutdown_connections 2024-02-17T13:06:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.665+0000 7fb7bed63640 1 --2- 172.21.15.18:0/1436853423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7b8106470 0x7fb7b810e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.665+0000 7fb7bed63640 1 --2- 172.21.15.18:0/1436853423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7b8101d50 0x7fb7b8105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.665+0000 7fb7bed63640 1 --2- 172.21.15.18:0/1436853423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7b81013a0 0x7fb7b8101780 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.665+0000 7fb7bed63640 1 -- 172.21.15.18:0/1436853423 >> 172.21.15.18:0/1436853423 conn(0x7fb7b80fcc40 msgr2=0x7fb7b80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.665+0000 7fb7bed63640 1 -- 172.21.15.18:0/1436853423 shutdown_connections 2024-02-17T13:06:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.665+0000 7fb7bed63640 1 -- 172.21.15.18:0/1436853423 wait complete. 2024-02-17T13:06:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7bed63640 1 Processor -- start 2024-02-17T13:06:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7bed63640 1 -- start start 2024-02-17T13:06:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7bed63640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7b81013a0 0x7fb7b819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7bed63640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7b8106470 0x7fb7b819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7bed63640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7b8197590 0x7fb7b8197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7bed63640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b8197f80 con 0x7fb7b8106470 2024-02-17T13:06:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7bed63640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b81980f0 con 0x7fb7b8197590 2024-02-17T13:06:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7bed63640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b81982b0 con 0x7fb7b81013a0 2024-02-17T13:06:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7b7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7b8106470 0x7fb7b819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7bcaff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7b81013a0 0x7fb7b819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7bcaff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7b81013a0 0x7fb7b819d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:43118/0 (socket says 172.21.15.18:43118) 2024-02-17T13:06:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7bcaff640 1 -- 172.21.15.18:0/689971091 learned_addr learned my addr 172.21.15.18:0/689971091 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.666+0000 7fb7bd300640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7b8197590 0x7fb7b8197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.667+0000 7fb7b7fff640 1 -- 172.21.15.18:0/689971091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7b81013a0 msgr2=0x7fb7b819d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.667+0000 7fb7b7fff640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7b81013a0 0x7fb7b819d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.667+0000 7fb7b7fff640 1 -- 172.21.15.18:0/689971091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7b8197590 msgr2=0x7fb7b8197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.667+0000 7fb7b7fff640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7b8197590 0x7fb7b8197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.667+0000 7fb7b7fff640 1 -- 172.21.15.18:0/689971091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b0009840 con 0x7fb7b8106470 2024-02-17T13:06:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.667+0000 7fb7b7fff640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7b8106470 0x7fb7b819d920 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800e9c0 tx=0x7fb7a800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.667+0000 7fb7b5ffb640 1 -- 172.21.15.18:0/689971091 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8009620 con 0x7fb7b8106470 2024-02-17T13:06:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.667+0000 7fb7b5ffb640 1 -- 172.21.15.18:0/689971091 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7a800ccc0 con 0x7fb7b8106470 2024-02-17T13:06:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.667+0000 7fb7bed63640 1 -- 172.21.15.18:0/689971091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ac009560 con 0x7fb7b8106470 2024-02-17T13:06:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.667+0000 7fb7b5ffb640 1 -- 172.21.15.18:0/689971091 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8004a00 con 0x7fb7b8106470 2024-02-17T13:06:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.667+0000 7fb7bed63640 1 -- 172.21.15.18:0/689971091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b81a4620 con 0x7fb7b8106470 2024-02-17T13:06:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.669+0000 7fb7b5ffb640 1 -- 172.21.15.18:0/689971091 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb7a8004b60 con 0x7fb7b8106470 2024-02-17T13:06:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.670+0000 7fb7c1a0c740 1 Processor -- start 2024-02-17T13:06:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.670+0000 7fb7c1a0c740 1 -- start start 2024-02-17T13:06:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.670+0000 7fb7b5ffb640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb7a00a2350 0x7fb7a00a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.670+0000 7fb7c1a0c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d15820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.670+0000 7fb7c1a0c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d161e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.670+0000 7fb7c1a0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.670+0000 7fb7c1a0c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556218c849f0 con 0x556218d16ad0 2024-02-17T13:06:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.670+0000 7fb7c1a0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556218c81f20 con 0x556218d1f2e0 2024-02-17T13:06:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.670+0000 7fb7c1a0c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556218c7d700 con 0x556218d15d60 2024-02-17T13:06:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7bcaff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d15820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7bcaff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d15820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60516/0 (socket says 172.21.15.18:60516) 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7bcaff640 1 -- 172.21.15.18:0/3089516865 learned_addr learned my addr 172.21.15.18:0/3089516865 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7bd300640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb7a00a2350 0x7fb7a00a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7b5ffb640 1 -- 172.21.15.18:0/689971091 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb7a8014070 con 0x7fb7b8106470 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7b7fff640 1 --2- 172.21.15.18:0/3089516865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7bd300640 1 --2- 172.21.15.18:0/3089516865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7bcaff640 1 -- 172.21.15.18:0/3089516865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 msgr2=0x556218d161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/3089516865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d161e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7bcaff640 1 -- 172.21.15.18:0/3089516865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 msgr2=0x556218d216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/3089516865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d216d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7bcaff640 1 -- 172.21.15.18:0/3089516865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556218c137b0 con 0x556218d16ad0 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7bd300640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb7a00a2350 0x7fb7a00a4810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0005fd0 tx=0x7fb7b00114d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.671+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/3089516865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d15820 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac036040 tx=0x7fb7ac02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e02fe7435257f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.672+0000 7fb79effd640 1 -- 172.21.15.18:0/3089516865 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac02c950 con 0x556218d16ad0 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.672+0000 7fb79effd640 1 -- 172.21.15.18:0/3089516865 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7ac02cab0 con 0x556218d16ad0 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.672+0000 7fb79effd640 1 -- 172.21.15.18:0/3089516865 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac005120 con 0x556218d16ad0 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.673+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/3089516865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 msgr2=0x556218d15820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.673+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/3089516865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d15820 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac036040 tx=0x7fb7ac02c670 comp rx=0 tx=0).stop 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.673+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/3089516865 shutdown_connections 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.673+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/3089516865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.673+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/3089516865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.673+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/3089516865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d15820 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.673+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/3089516865 >> 172.21.15.18:0/3089516865 conn(0x556218cdd550 msgr2=0x556218d10a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.673+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/3089516865 shutdown_connections 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.673+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/3089516865 wait complete. 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.673+0000 7fb7c1a0c740 1 -- start start 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.673+0000 7fb7c1a0c740 1 Processor -- start 2024-02-17T13:06:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7c1a0c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d1c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7c1a0c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d1c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7c1a0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d1cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7c1a0c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556218c22bf0 con 0x556218d16ad0 2024-02-17T13:06:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7b7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d1c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7b7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d1c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60524/0 (socket says 172.21.15.18:60524) 2024-02-17T13:06:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7b7fff640 1 -- 172.21.15.18:0/2015409140 learned_addr learned my addr 172.21.15.18:0/2015409140 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7c1a0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556218951250 con 0x556218d1f2e0 2024-02-17T13:06:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2015409140 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556218c03200 con 0x556218d15d60 2024-02-17T13:06:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d1c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7bd300640 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d1cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7c1a0c740 10 client.0 _ll_register_callbacks cb 0x556218c71d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:06:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7bcaff640 1 -- 172.21.15.18:0/2015409140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 msgr2=0x556218d1cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d1cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7bcaff640 1 -- 172.21.15.18:0/2015409140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 msgr2=0x556218d1c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d1c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7bcaff640 1 -- 172.21.15.18:0/2015409140 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b810e850 con 0x556218d15d60 2024-02-17T13:06:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.674+0000 7fb7b7fff640 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d1c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:06:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.675+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2015409140 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55621897d600 con 0x556218d15d60 2024-02-17T13:06:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.675+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d1c290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009430 tx=0x7fb7ac005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c443a9675b547d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.675+0000 7fb797fff640 10 client.29734 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:06:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.675+0000 7fb797fff640 1 -- 172.21.15.18:0/2015409140 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac005270 con 0x556218d15d60 2024-02-17T13:06:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.675+0000 7fb797fff640 1 -- 172.21.15.18:0/2015409140 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb7ac04d070 con 0x556218d15d60 2024-02-17T13:06:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.675+0000 7fb797fff640 1 -- 172.21.15.18:0/2015409140 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7ac004330 con 0x556218d15d60 2024-02-17T13:06:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.675+0000 7fb7c1a0c740 10 client.29734 fetch_fsmap learned FSMap version 78 2024-02-17T13:06:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.675+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2015409140 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({fsmap=78}) v3 -- 0x556218c7bd80 con 0x556218d15d60 2024-02-17T13:06:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.676+0000 7fb797fff640 1 -- 172.21.15.18:0/2015409140 <== mon.1 v2:172.21.15.146:3300/0 4 ==== fsmap(e 78) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb7ac03c050 con 0x556218d15d60 2024-02-17T13:06:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.676+0000 7fb7c1a0c740 10 client.29734 fetch_fsmap finished waiting for FSMap version 78 2024-02-17T13:06:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.676+0000 7fb7c1a0c740 10 client.29734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:06:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.676+0000 7fb7c1a0c740 20 client.29734 populate_metadata read hostname 'smithi018' 2024-02-17T13:06:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.676+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556218d33480 0x556218d358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.676+0000 7fb7c1a0c740 4 client.29734 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:06:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.676+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2015409140 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562189ce1b0 con 0x556218d33480 2024-02-17T13:06:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.676+0000 7fb7b7fff640 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556218d33480 0x556218d358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.679+0000 7fb797fff640 10 client.29734 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.679+0000 7fb7b7fff640 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556218d33480 0x556218d358c0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f13f1a6b06ebdfe server_cookie=68e6effc7f914856 in_seq=0 out_seq=0 2024-02-17T13:06:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb797fff640 1 -- 172.21.15.18:0/2015409140 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5562189ce1b0 con 0x556218d33480 2024-02-17T13:06:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb797fff640 10 client.29734 handle_command_reply: tid=0 2024-02-17T13:06:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb7c1a0c740 1 client.29734 shutdown 2024-02-17T13:06:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2015409140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 msgr2=0x556218d1c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d1c290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009430 tx=0x7fb7ac005de0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2015409140 shutdown_connections 2024-02-17T13:06:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556218d33480 0x556218d358c0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d1cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d1c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb797fff640 0 client.29734 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/2015409140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d1c290 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2015409140 >> 172.21.15.18:0/2015409140 conn(0x556218cdd550 msgr2=0x556218d127a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2015409140 shutdown_connections 2024-02-17T13:06:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.682+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2015409140 wait complete. 2024-02-17T13:06:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.683+0000 7fb7c1a0c740 20 client.29734 trim_cache size 0 max 16384 2024-02-17T13:06:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.698+0000 7fb7c1a0c740 1 Processor -- start 2024-02-17T13:06:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.698+0000 7fb7c1a0c740 1 -- start start 2024-02-17T13:06:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.698+0000 7fb7c1a0c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d15d60 0x556218d1c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.698+0000 7fb7c1a0c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d16ad0 0x556218d1cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.698+0000 7fb7c1a0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d1ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.698+0000 7fb7c1a0c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556218d17ed0 con 0x556218d15d60 2024-02-17T13:06:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.698+0000 7fb7c1a0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556218d1d260 con 0x556218d1f2e0 2024-02-17T13:06:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.698+0000 7fb7c1a0c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556218d0c380 con 0x556218d16ad0 2024-02-17T13:06:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb7bcaff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d15d60 0x556218d1c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb7b7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d16ad0 0x556218d1cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb7bcaff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d15d60 0x556218d1c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60534/0 (socket says 172.21.15.18:60534) 2024-02-17T13:06:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb7bcaff640 1 -- 172.21.15.18:0/537805065 learned_addr learned my addr 172.21.15.18:0/537805065 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb7bd300640 1 --2- 172.21.15.18:0/537805065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d1ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb7bcaff640 1 -- 172.21.15.18:0/537805065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d16ad0 msgr2=0x556218d1cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/537805065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d16ad0 0x556218d1cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb7bcaff640 1 -- 172.21.15.18:0/537805065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 msgr2=0x556218d1ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/537805065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d1ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb7bcaff640 1 -- 172.21.15.18:0/537805065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556218c187b0 con 0x556218d15d60 2024-02-17T13:06:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/537805065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d15d60 0x556218d1c7e0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac036040 tx=0x7fb7ac04a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d79a67f255f7a5f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb79e7fc640 1 -- 172.21.15.18:0/537805065 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac005070 con 0x556218d15d60 2024-02-17T13:06:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb79e7fc640 1 -- 172.21.15.18:0/537805065 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7ac058070 con 0x556218d15d60 2024-02-17T13:06:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.699+0000 7fb79e7fc640 1 -- 172.21.15.18:0/537805065 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac03f440 con 0x556218d15d60 2024-02-17T13:06:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.700+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/537805065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d15d60 msgr2=0x556218d1c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.700+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/537805065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d15d60 0x556218d1c7e0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac036040 tx=0x7fb7ac04a000 comp rx=0 tx=0).stop 2024-02-17T13:06:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.700+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/537805065 shutdown_connections 2024-02-17T13:06:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.700+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/537805065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d1ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.700+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/537805065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d16ad0 0x556218d1cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.700+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/537805065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d15d60 0x556218d1c7e0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.700+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/537805065 >> 172.21.15.18:0/537805065 conn(0x556218cdd550 msgr2=0x556218d3a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.700+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/537805065 shutdown_connections 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.700+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/537805065 wait complete. 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7c1a0c740 1 -- start start 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7c1a0c740 1 Processor -- start 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7c1a0c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d2a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7c1a0c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d2a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7c1a0c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d28730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7c1a0c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x556218d28ca0 con 0x556218d16ad0 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7c1a0c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556218d28e10 con 0x556218d1f2e0 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7c1a0c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x556218d28fd0 con 0x556218d15d60 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7c1a0c740 10 client.0 _ll_register_callbacks cb 0x7fb7a8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:06:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7b7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d2a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7bcaff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d2a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7b7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d2a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60544/0 (socket says 172.21.15.18:60544) 2024-02-17T13:06:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7b7fff640 1 -- 172.21.15.18:0/2422809379 learned_addr learned my addr 172.21.15.18:0/2422809379 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.701+0000 7fb7bd300640 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d28730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb7bcaff640 1 -- 172.21.15.18:0/2422809379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 msgr2=0x556218d28730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d28730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb7bcaff640 1 -- 172.21.15.18:0/2422809379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 msgr2=0x556218d2a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d2a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb7bcaff640 1 -- 172.21.15.18:0/2422809379 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556218c7bd80 con 0x556218d15d60 2024-02-17T13:06:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb7b7fff640 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d2a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:06:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb7bcaff640 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d2a120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac060040 tx=0x7fb7ac005480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aade66dfb2f0d790 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb797fff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:06:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb797fff640 1 -- 172.21.15.18:0/2422809379 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac02cc20 con 0x556218d15d60 2024-02-17T13:06:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2422809379 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7ac02cab0 con 0x556218d15d60 2024-02-17T13:06:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb797fff640 1 -- 172.21.15.18:0/2422809379 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb7ac04d040 con 0x556218d15d60 2024-02-17T13:06:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb797fff640 1 -- 172.21.15.18:0/2422809379 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7ac05d460 con 0x556218d15d60 2024-02-17T13:06:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb7c1a0c740 10 client.29740 fetch_fsmap learned FSMap version 78 2024-02-17T13:06:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2422809379 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x556218c3e540 con 0x556218d15d60 2024-02-17T13:06:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.702+0000 7fb797fff640 1 -- 172.21.15.18:0/2422809379 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7ac02cab0 con 0x556218d15d60 2024-02-17T13:06:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.703+0000 7fb797fff640 1 -- 172.21.15.18:0/2422809379 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 78) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb7ac03c040 con 0x556218d15d60 2024-02-17T13:06:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.703+0000 7fb7c1a0c740 10 client.29740 fetch_fsmap finished waiting for FSMap version 78 2024-02-17T13:06:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.703+0000 7fb7c1a0c740 10 client.29740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:06:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.703+0000 7fb7c1a0c740 20 client.29740 populate_metadata read hostname 'smithi018' 2024-02-17T13:06:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.703+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556218d8a7f0 0x556218d8cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.703+0000 7fb7c1a0c740 4 client.29740 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:06:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.703+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2422809379 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5562189ba550 con 0x556218d8a7f0 2024-02-17T13:06:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.703+0000 7fb7b7fff640 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556218d8a7f0 0x556218d8cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.704+0000 7fb7b7fff640 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556218d8a7f0 0x556218d8cc10 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37dae11126abf66 server_cookie=d38d7ef38ee0b660 in_seq=0 out_seq=0 2024-02-17T13:06:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:41.704+0000 7fb797fff640 10 client.29740 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:41.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 150010 34.36 MB/sec execute 113 sec latency 3349.109 ms 2024-02-17T13:06:42.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:42 smithi146 ceph-mon[36454]: pgmap v529: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 853 op/s 2024-02-17T13:06:42.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:42 smithi018 ceph-mon[32625]: pgmap v529: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 853 op/s 2024-02-17T13:06:42.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:42 smithi119 ceph-mon[36188]: pgmap v529: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 853 op/s 2024-02-17T13:06:42.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 150543 34.34 MB/sec execute 114 sec latency 4349.153 ms 2024-02-17T13:06:43.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 150998 34.21 MB/sec execute 115 sec latency 5349.193 ms 2024-02-17T13:06:44.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:44 smithi146 ceph-mon[36454]: pgmap v530: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 572 op/s 2024-02-17T13:06:44.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:44 smithi018 ceph-mon[32625]: pgmap v530: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 572 op/s 2024-02-17T13:06:44.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:44 smithi119 ceph-mon[36188]: pgmap v530: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 572 op/s 2024-02-17T13:06:44.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 151554 34.21 MB/sec execute 116 sec latency 6349.229 ms 2024-02-17T13:06:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.067+0000 7fb797fff640 1 -- 172.21.15.18:0/2422809379 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5562189ba550 con 0x556218d8a7f0 2024-02-17T13:06:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.067+0000 7fb797fff640 10 client.29740 handle_command_reply: tid=0 2024-02-17T13:06:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.067+0000 7fb7c1a0c740 1 client.29740 shutdown 2024-02-17T13:06:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.067+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2422809379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 msgr2=0x556218d2a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.067+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d2a120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac060040 tx=0x7fb7ac005480 comp rx=0 tx=0).stop 2024-02-17T13:06:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.067+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2422809379 shutdown_connections 2024-02-17T13:06:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.067+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x556218d8a7f0 0x556218d8cc10 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.067+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556218d1f2e0 0x556218d28730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.067+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556218d16ad0 0x556218d2a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.067+0000 7fb7c1a0c740 1 --2- 172.21.15.18:0/2422809379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556218d15d60 0x556218d2a120 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.067+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2422809379 >> 172.21.15.18:0/2422809379 conn(0x556218cdd550 msgr2=0x556218d1aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.067+0000 7fb797fff640 0 client.29740 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.068+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2422809379 shutdown_connections 2024-02-17T13:06:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.068+0000 7fb7c1a0c740 1 -- 172.21.15.18:0/2422809379 wait complete. 2024-02-17T13:06:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.068+0000 7fb7c1a0c740 20 client.29740 trim_cache size 0 max 16384 2024-02-17T13:06:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:06:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:06:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:06:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:06:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.068+0000 7fb7bed63640 1 -- 172.21.15.18:0/689971091 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb7a00a2350 msgr2=0x7fb7a00a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.068+0000 7fb7bed63640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb7a00a2350 0x7fb7a00a4810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0005fd0 tx=0x7fb7b00114d0 comp rx=0 tx=0).stop 2024-02-17T13:06:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.068+0000 7fb7bed63640 1 -- 172.21.15.18:0/689971091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7b8106470 msgr2=0x7fb7b819d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.068+0000 7fb7bed63640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7b8106470 0x7fb7b819d920 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800e9c0 tx=0x7fb7a800edc0 comp rx=0 tx=0).stop 2024-02-17T13:06:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.068+0000 7fb7bed63640 1 -- 172.21.15.18:0/689971091 shutdown_connections 2024-02-17T13:06:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.069+0000 7fb7bed63640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fb7a00a2350 0x7fb7a00a4810 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.069+0000 7fb7bed63640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7b8197590 0x7fb7b8197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.069+0000 7fb7bed63640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7b8106470 0x7fb7b819d920 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.069+0000 7fb7bed63640 1 --2- 172.21.15.18:0/689971091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7b81013a0 0x7fb7b819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.069+0000 7fb7bed63640 1 -- 172.21.15.18:0/689971091 >> 172.21.15.18:0/689971091 conn(0x7fb7b80fcc40 msgr2=0x7fb7b80fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.069+0000 7fb7bed63640 1 -- 172.21.15.18:0/689971091 shutdown_connections 2024-02-17T13:06:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:45.069+0000 7fb7bed63640 1 -- 172.21.15.18:0/689971091 wait complete. 2024-02-17T13:06:45.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 152142 34.20 MB/sec execute 117 sec latency 6619.314 ms 2024-02-17T13:06:46.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:46 smithi146 ceph-mon[36454]: pgmap v531: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 846 op/s 2024-02-17T13:06:46.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:46 smithi018 ceph-mon[32625]: pgmap v531: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 846 op/s 2024-02-17T13:06:46.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:46 smithi119 ceph-mon[36188]: pgmap v531: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 846 op/s 2024-02-17T13:06:46.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 152695 34.17 MB/sec execute 118 sec latency 102.340 ms 2024-02-17T13:06:47.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 153403 34.17 MB/sec execute 119 sec latency 66.528 ms 2024-02-17T13:06:48.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:48 smithi146 ceph-mon[36454]: pgmap v532: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.8 MiB/s wr, 607 op/s 2024-02-17T13:06:48.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:48 smithi018 ceph-mon[32625]: pgmap v532: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.8 MiB/s wr, 607 op/s 2024-02-17T13:06:48.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:48 smithi119 ceph-mon[36188]: pgmap v532: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.8 MiB/s wr, 607 op/s 2024-02-17T13:06:48.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 153910 34.10 MB/sec execute 120 sec latency 209.078 ms 2024-02-17T13:06:49.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 154428 34.08 MB/sec execute 121 sec latency 218.981 ms 2024-02-17T13:06:50.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:50 smithi146 ceph-mon[36454]: pgmap v533: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 687 op/s 2024-02-17T13:06:50.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:50 smithi018 ceph-mon[32625]: pgmap v533: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 687 op/s 2024-02-17T13:06:50.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:50 smithi119 ceph-mon[36188]: pgmap v533: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 687 op/s 2024-02-17T13:06:50.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 155150 34.03 MB/sec execute 122 sec latency 82.738 ms 2024-02-17T13:06:51.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 155937 34.12 MB/sec execute 123 sec latency 70.053 ms 2024-02-17T13:06:52.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:52 smithi146 ceph-mon[36454]: pgmap v534: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 810 op/s 2024-02-17T13:06:52.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:52 smithi018 ceph-mon[32625]: pgmap v534: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 810 op/s 2024-02-17T13:06:52.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:52 smithi119 ceph-mon[36188]: pgmap v534: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 810 op/s 2024-02-17T13:06:52.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 156753 34.26 MB/sec execute 124 sec latency 58.131 ms 2024-02-17T13:06:53.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 157178 34.22 MB/sec execute 125 sec latency 300.665 ms 2024-02-17T13:06:54.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:54 smithi146 ceph-mon[36454]: pgmap v535: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 7.6 MiB/s wr, 580 op/s 2024-02-17T13:06:54.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:54 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:54.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:54 smithi018 ceph-mon[32625]: pgmap v535: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 7.6 MiB/s wr, 580 op/s 2024-02-17T13:06:54.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:54 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:54.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:54 smithi119 ceph-mon[36188]: pgmap v535: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 7.6 MiB/s wr, 580 op/s 2024-02-17T13:06:54.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:54 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:06:54.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 157671 34.12 MB/sec execute 126 sec latency 168.321 ms 2024-02-17T13:06:55.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 158291 34.09 MB/sec execute 127 sec latency 94.536 ms 2024-02-17T13:06:56.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:56 smithi146 ceph-mon[36454]: pgmap v536: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 852 op/s 2024-02-17T13:06:56.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:56 smithi018 ceph-mon[32625]: pgmap v536: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 852 op/s 2024-02-17T13:06:56.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:56 smithi119 ceph-mon[36188]: pgmap v536: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 852 op/s 2024-02-17T13:06:56.794 INFO:tasks.workunit.client.0.smithi018.stdout: 10 159052 34.12 MB/sec execute 128 sec latency 90.790 ms 2024-02-17T13:06:57.795 INFO:tasks.workunit.client.0.smithi018.stdout: 10 159814 34.17 MB/sec execute 129 sec latency 89.181 ms 2024-02-17T13:06:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.209+0000 7f74ea06d640 1 -- 172.21.15.18:0/577481002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e4105590 msgr2=0x7f74e4105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.209+0000 7f74ea06d640 1 --2- 172.21.15.18:0/577481002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e4105590 0x7f74e4105a20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f74d801a910 tx=0x7f74d803cd80 comp rx=0 tx=0).stop 2024-02-17T13:06:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.214+0000 7f74ea06d640 1 -- 172.21.15.18:0/577481002 shutdown_connections 2024-02-17T13:06:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.214+0000 7f74ea06d640 1 --2- 172.21.15.18:0/577481002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e4105590 0x7f74e4105a20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.214+0000 7f74ea06d640 1 --2- 172.21.15.18:0/577481002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74e4100ab0 0x7f74e4104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.214+0000 7f74ea06d640 1 --2- 172.21.15.18:0/577481002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74e4100190 0x7f74e4100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.214+0000 7f74ea06d640 1 -- 172.21.15.18:0/577481002 >> 172.21.15.18:0/577481002 conn(0x7f74e4077290 msgr2=0x7f74e40776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.217+0000 7f74ea06d640 1 -- 172.21.15.18:0/577481002 shutdown_connections 2024-02-17T13:06:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.218+0000 7f74ea06d640 1 -- 172.21.15.18:0/577481002 wait complete. 2024-02-17T13:06:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.219+0000 7f74ea06d640 1 Processor -- start 2024-02-17T13:06:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.219+0000 7f74ea06d640 1 -- start start 2024-02-17T13:06:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.219+0000 7f74ea06d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74e4100190 0x7f74e40719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.219+0000 7f74ea06d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74e4100ab0 0x7f74e4071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.219+0000 7f74ea06d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e4105590 0x7f74e410d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.219+0000 7f74ea06d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e41149d0 con 0x7f74e4100190 2024-02-17T13:06:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.219+0000 7f74ea06d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e4114b40 con 0x7f74e4105590 2024-02-17T13:06:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.219+0000 7f74ea06d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e410dd40 con 0x7f74e4100ab0 2024-02-17T13:06:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.220+0000 7f74e3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e4105590 0x7f74e410d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.221+0000 7f74e3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e4105590 0x7f74e410d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:36830/0 (socket says 172.21.15.18:36830) 2024-02-17T13:06:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.221+0000 7f74e3fff640 1 -- 172.21.15.18:0/94679751 learned_addr learned my addr 172.21.15.18:0/94679751 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.223+0000 7f74e37fe640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74e4100190 0x7f74e40719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.223+0000 7f74e2ffd640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74e4100ab0 0x7f74e4071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.223+0000 7f74e3fff640 1 -- 172.21.15.18:0/94679751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74e4100ab0 msgr2=0x7f74e4071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.223+0000 7f74e3fff640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74e4100ab0 0x7f74e4071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.223+0000 7f74e3fff640 1 -- 172.21.15.18:0/94679751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74e4100190 msgr2=0x7f74e40719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.223+0000 7f74e3fff640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74e4100190 0x7f74e40719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.223+0000 7f74e3fff640 1 -- 172.21.15.18:0/94679751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d801a570 con 0x7f74e4105590 2024-02-17T13:06:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.226+0000 7f74e3fff640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e4105590 0x7f74e410d800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f74d8005740 tx=0x7f74d80045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.227+0000 7f74e0ff9640 1 -- 172.21.15.18:0/94679751 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8004be0 con 0x7f74e4105590 2024-02-17T13:06:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.227+0000 7f74e0ff9640 1 -- 172.21.15.18:0/94679751 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74d803ddb0 con 0x7f74e4105590 2024-02-17T13:06:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.227+0000 7f74e0ff9640 1 -- 172.21.15.18:0/94679751 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d804e480 con 0x7f74e4105590 2024-02-17T13:06:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.228+0000 7f74ea06d640 1 -- 172.21.15.18:0/94679751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e410dfc0 con 0x7f74e4105590 2024-02-17T13:06:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.228+0000 7f74ea06d640 1 -- 172.21.15.18:0/94679751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e410e480 con 0x7f74e4105590 2024-02-17T13:06:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.234+0000 7f74ecc3d740 1 Processor -- start 2024-02-17T13:06:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.234+0000 7f74e0ff9640 1 -- 172.21.15.18:0/94679751 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f74d8004d40 con 0x7f74e4105590 2024-02-17T13:06:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.235+0000 7f74e0ff9640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74cc0a2500 0x7f74cc0a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.235+0000 7f74e0ff9640 1 -- 172.21.15.18:0/94679751 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f74d8101430 con 0x7f74e4105590 2024-02-17T13:06:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.235+0000 7f74ecc3d740 1 -- start start 2024-02-17T13:06:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.236+0000 7f74ecc3d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fbfd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.236+0000 7f74ecc3d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 0x55869fbfe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.236+0000 7f74ecc3d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc072e0 0x55869fc096d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.236+0000 7f74ecc3d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869fb6c9f0 con 0x55869fbfdd60 2024-02-17T13:06:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.236+0000 7f74ecc3d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869fb69f20 con 0x55869fc072e0 2024-02-17T13:06:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.236+0000 7f74ecc3d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869fb65700 con 0x55869fbfead0 2024-02-17T13:06:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.236+0000 7f74e3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc072e0 0x55869fc096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.236+0000 7f74e3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc072e0 0x55869fc096d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.18:36832/0 (socket says 172.21.15.18:36832) 2024-02-17T13:06:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.236+0000 7f74e3fff640 1 -- 172.21.15.18:0/3931538096 learned_addr learned my addr 172.21.15.18:0/3931538096 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.237+0000 7f74e2ffd640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74cc0a2500 0x7f74cc0a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.237+0000 7f74e3fff640 1 -- 172.21.15.18:0/3931538096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 msgr2=0x55869fbfd820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T13:06:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.237+0000 7f74e3fff640 1 --2- 172.21.15.18:0/3931538096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fbfd820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.237+0000 7f74e3fff640 1 -- 172.21.15.18:0/3931538096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 msgr2=0x55869fbfe1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.237+0000 7f74e3fff640 1 --2- 172.21.15.18:0/3931538096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 0x55869fbfe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.237+0000 7f74e3fff640 1 -- 172.21.15.18:0/3931538096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55869fafb7b0 con 0x55869fc072e0 2024-02-17T13:06:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.238+0000 7f74e3fff640 1 --2- 172.21.15.18:0/3931538096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc072e0 0x55869fc096d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f74d80ca830 tx=0x7f74d80cdcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=512c33811525d0b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.239+0000 7f74c9ffb640 1 -- 172.21.15.18:0/3931538096 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80e5070 con 0x55869fc072e0 2024-02-17T13:06:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.239+0000 7f74c9ffb640 1 -- 172.21.15.18:0/3931538096 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74d80cc030 con 0x55869fc072e0 2024-02-17T13:06:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.240+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3931538096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc072e0 msgr2=0x55869fc096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.240+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/3931538096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc072e0 0x55869fc096d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f74d80ca830 tx=0x7f74d80cdcc0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.240+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3931538096 shutdown_connections 2024-02-17T13:06:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.240+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/3931538096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc072e0 0x55869fc096d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.240+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/3931538096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 0x55869fbfe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.240+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/3931538096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fbfd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.240+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3931538096 >> 172.21.15.18:0/3931538096 conn(0x55869fbc5550 msgr2=0x55869fbf8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.243+0000 7f74e2ffd640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74cc0a2500 0x7f74cc0a49c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55869fbff230 tx=0x7f74d4009330 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.243+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3931538096 shutdown_connections 2024-02-17T13:06:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.247+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3931538096 wait complete. 2024-02-17T13:06:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.248+0000 7f74ecc3d740 1 -- start start 2024-02-17T13:06:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.248+0000 7f74ecc3d740 1 Processor -- start 2024-02-17T13:06:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.249+0000 7f74ecc3d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 0x55869fc09570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.249+0000 7f74ecc3d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fbfead0 0x55869fc06860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.249+0000 7f74ecc3d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fc06da0 0x55869fc07220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.249+0000 7f74ecc3d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869fb0abf0 con 0x55869fbfdd60 2024-02-17T13:06:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.249+0000 7f74ecc3d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869f839250 con 0x55869fbfead0 2024-02-17T13:06:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.249+0000 7f74ecc3d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869faeb200 con 0x55869fc06da0 2024-02-17T13:06:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.249+0000 7f74ecc3d740 10 client.0 _ll_register_callbacks cb 0x55869fb59d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:06:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.249+0000 7f74e37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 0x55869fc09570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.250+0000 7f74e3fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fc06da0 0x55869fc07220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.250+0000 7f74e3fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fc06da0 0x55869fc07220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:39924/0 (socket says 172.21.15.18:39924) 2024-02-17T13:06:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.250+0000 7f74e3fff640 1 -- 172.21.15.18:0/1418289095 learned_addr learned my addr 172.21.15.18:0/1418289095 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.250+0000 7f74e3fff640 1 -- 172.21.15.18:0/1418289095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fbfead0 msgr2=0x55869fc06860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T13:06:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.250+0000 7f74e3fff640 1 --2- 172.21.15.18:0/1418289095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fbfead0 0x55869fc06860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.250+0000 7f74e3fff640 1 -- 172.21.15.18:0/1418289095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 msgr2=0x55869fc09570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.250+0000 7f74e3fff640 1 --2- 172.21.15.18:0/1418289095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 0x55869fc09570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.250+0000 7f74e3fff640 1 -- 172.21.15.18:0/1418289095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e410e480 con 0x55869fc06da0 2024-02-17T13:06:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.251+0000 7f74e3fff640 1 --2- 172.21.15.18:0/1418289095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fc06da0 0x55869fc07220 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f74d80ca2c0 tx=0x7f74d80ce380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b356015fd9934ae5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.251+0000 7f74c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:06:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.252+0000 7f74c2ffd640 1 -- 172.21.15.18:0/1418289095 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80e5070 con 0x55869fc06da0 2024-02-17T13:06:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.252+0000 7f74c2ffd640 1 -- 172.21.15.18:0/1418289095 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f74d80ce5f0 con 0x55869fc06da0 2024-02-17T13:06:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.253+0000 7f74ecc3d740 1 -- 172.21.15.18:0/1418289095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55869f8b61b0 con 0x55869fc06da0 2024-02-17T13:06:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.256+0000 7f74c2ffd640 1 -- 172.21.15.18:0/1418289095 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74d80ca670 con 0x55869fc06da0 2024-02-17T13:06:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.260+0000 7f74ecc3d740 10 client.29746 fetch_fsmap learned FSMap version 79 2024-02-17T13:06:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.260+0000 7f74ecc3d740 1 -- 172.21.15.18:0/1418289095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x55869fb63d80 con 0x55869fc06da0 2024-02-17T13:06:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.262+0000 7f74c2ffd640 1 -- 172.21.15.18:0/1418289095 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74d80da070 con 0x55869fc06da0 2024-02-17T13:06:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.262+0000 7f74c2ffd640 1 -- 172.21.15.18:0/1418289095 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 79) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f74d80ec1b0 con 0x55869fc06da0 2024-02-17T13:06:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.262+0000 7f74ecc3d740 10 client.29746 fetch_fsmap finished waiting for FSMap version 79 2024-02-17T13:06:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.262+0000 7f74ecc3d740 10 client.29746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:06:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.262+0000 7f74ecc3d740 20 client.29746 populate_metadata read hostname 'smithi018' 2024-02-17T13:06:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.262+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/1418289095 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55869fc17010 0x55869fc19430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.262+0000 7f74ecc3d740 4 client.29746 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T13:06:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.262+0000 7f74ecc3d740 1 -- 172.21.15.18:0/1418289095 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55869f8a2550 con 0x55869fc17010 2024-02-17T13:06:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.263+0000 7f74e37fe640 1 --2- 172.21.15.18:0/1418289095 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55869fc17010 0x55869fc19430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.264+0000 7f74e37fe640 1 --2- 172.21.15.18:0/1418289095 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55869fc17010 0x55869fc19430 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f9bb9b5530cf7a2 server_cookie=13e3f49ae6105bdc in_seq=0 out_seq=0 2024-02-17T13:06:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.266+0000 7f74c2ffd640 10 client.29746 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.272+0000 7f74c2ffd640 1 -- 172.21.15.18:0/1418289095 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55869f8a2550 con 0x55869fc17010 2024-02-17T13:06:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.272+0000 7f74c2ffd640 10 client.29746 handle_command_reply: tid=0 2024-02-17T13:06:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.272+0000 7f74ecc3d740 1 client.29746 shutdown 2024-02-17T13:06:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.273+0000 7f74ecc3d740 1 -- 172.21.15.18:0/1418289095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fc06da0 msgr2=0x55869fc07220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.273+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/1418289095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fc06da0 0x55869fc07220 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f74d80ca2c0 tx=0x7f74d80ce380 comp rx=0 tx=0).stop 2024-02-17T13:06:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.274+0000 7f74ecc3d740 1 -- 172.21.15.18:0/1418289095 shutdown_connections 2024-02-17T13:06:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.274+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/1418289095 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55869fc17010 0x55869fc19430 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.274+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/1418289095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fc06da0 0x55869fc07220 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.274+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/1418289095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fbfead0 0x55869fc06860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.274+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/1418289095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 0x55869fc09570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.274+0000 7f74ecc3d740 1 -- 172.21.15.18:0/1418289095 >> 172.21.15.18:0/1418289095 conn(0x55869fbc5550 msgr2=0x55869fbfa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.276+0000 7f74c2ffd640 0 client.29746 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.277+0000 7f74ecc3d740 1 -- 172.21.15.18:0/1418289095 shutdown_connections 2024-02-17T13:06:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.277+0000 7f74ecc3d740 1 -- 172.21.15.18:0/1418289095 wait complete. 2024-02-17T13:06:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.279+0000 7f74ecc3d740 20 client.29746 trim_cache size 0 max 16384 2024-02-17T13:06:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74ecc3d740 1 Processor -- start 2024-02-17T13:06:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74ecc3d740 1 -- start start 2024-02-17T13:06:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74ecc3d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 0x55869fc090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74ecc3d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fc060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74ecc3d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc06da0 0x55869fc06630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74ecc3d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869fbf3070 con 0x55869fbfdd60 2024-02-17T13:06:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74ecc3d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869fbfad60 con 0x55869fc06da0 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74e37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 0x55869fc090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74ecc3d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869fbf9b50 con 0x55869fbfead0 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74e3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc06da0 0x55869fc06630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74e2ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fc060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74e2ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fc060f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:39936/0 (socket says 172.21.15.18:39936) 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74e2ffd640 1 -- 172.21.15.18:0/31573560 learned_addr learned my addr 172.21.15.18:0/31573560 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.309+0000 7f74e37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 0x55869fc090a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49758/0 (socket says 172.21.15.18:49758) 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.310+0000 7f74e3fff640 1 -- 172.21.15.18:0/31573560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 msgr2=0x55869fc060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.310+0000 7f74e3fff640 1 --2- 172.21.15.18:0/31573560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fc060f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.310+0000 7f74e3fff640 1 -- 172.21.15.18:0/31573560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 msgr2=0x55869fc090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.310+0000 7f74e3fff640 1 --2- 172.21.15.18:0/31573560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 0x55869fc090a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.310+0000 7f74e3fff640 1 -- 172.21.15.18:0/31573560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55869fb007b0 con 0x55869fc06da0 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.310+0000 7f74e3fff640 1 --2- 172.21.15.18:0/31573560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc06da0 0x55869fc06630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f74d8004650 tx=0x7f74d80cddf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37fca8fb798247ed server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.310+0000 7f74c97fa640 1 -- 172.21.15.18:0/31573560 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80e5070 con 0x55869fc06da0 2024-02-17T13:06:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.310+0000 7f74c97fa640 1 -- 172.21.15.18:0/31573560 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74d80da070 con 0x55869fc06da0 2024-02-17T13:06:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.310+0000 7f74c97fa640 1 -- 172.21.15.18:0/31573560 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80ca630 con 0x55869fc06da0 2024-02-17T13:06:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.311+0000 7f74ecc3d740 1 -- 172.21.15.18:0/31573560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc06da0 msgr2=0x55869fc06630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.311+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/31573560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc06da0 0x55869fc06630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f74d8004650 tx=0x7f74d80cddf0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.311+0000 7f74ecc3d740 1 -- 172.21.15.18:0/31573560 shutdown_connections 2024-02-17T13:06:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.311+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/31573560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fc06da0 0x55869fc06630 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.311+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/31573560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fc060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.311+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/31573560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fbfdd60 0x55869fc090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.311+0000 7f74ecc3d740 1 -- 172.21.15.18:0/31573560 >> 172.21.15.18:0/31573560 conn(0x55869fbc5550 msgr2=0x55869fc01b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.311+0000 7f74ecc3d740 1 -- 172.21.15.18:0/31573560 shutdown_connections 2024-02-17T13:06:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.311+0000 7f74ecc3d740 1 -- 172.21.15.18:0/31573560 wait complete. 2024-02-17T13:06:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.311+0000 7f74ecc3d740 1 -- start start 2024-02-17T13:06:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74ecc3d740 1 Processor -- start 2024-02-17T13:06:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74ecc3d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fbfdd60 0x55869fc0bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74ecc3d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fc0c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74ecc3d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fc06da0 0x55869fc6c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74ecc3d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869fc0ca30 con 0x55869fc06da0 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74ecc3d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869fc0cba0 con 0x55869fbfdd60 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74ecc3d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869fc6ca10 con 0x55869fbfead0 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74ecc3d740 10 client.0 _ll_register_callbacks cb 0x7f74d803df20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74e3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fc06da0 0x55869fc6c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74e3fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fc06da0 0x55869fc6c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49766/0 (socket says 172.21.15.18:49766) 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74e3fff640 1 -- 172.21.15.18:0/3029011276 learned_addr learned my addr 172.21.15.18:0/3029011276 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74e2ffd640 1 --2- 172.21.15.18:0/3029011276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fc0c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74e2ffd640 1 -- 172.21.15.18:0/3029011276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fbfdd60 msgr2=0x55869fc0bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74e2ffd640 1 --2- 172.21.15.18:0/3029011276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fbfdd60 0x55869fc0bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74e2ffd640 1 -- 172.21.15.18:0/3029011276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fc06da0 msgr2=0x55869fc6c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74e2ffd640 1 --2- 172.21.15.18:0/3029011276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fc06da0 0x55869fc6c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.312+0000 7f74e2ffd640 1 -- 172.21.15.18:0/3029011276 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55869fb63d80 con 0x55869fbfead0 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.313+0000 7f74e2ffd640 1 --2- 172.21.15.18:0/3029011276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fc0c2c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55869f8a2620 tx=0x7f74d4004700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7adbd0257f0f219b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:06:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.313+0000 7f74c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.313+0000 7f74c2ffd640 1 -- 172.21.15.18:0/3029011276 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d401f070 con 0x55869fbfead0 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.313+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3029011276 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55869f865600 con 0x55869fbfead0 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.313+0000 7f74c2ffd640 1 -- 172.21.15.18:0/3029011276 <== mon.1 v2:172.21.15.146:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f74d40155f0 con 0x55869fbfead0 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.313+0000 7f74c2ffd640 1 -- 172.21.15.18:0/3029011276 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74d40157b0 con 0x55869fbfead0 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.313+0000 7f74ecc3d740 10 client.29752 fetch_fsmap learned FSMap version 79 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.313+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3029011276 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x55869fb26540 con 0x55869fbfead0 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.314+0000 7f74c2ffd640 1 -- 172.21.15.18:0/3029011276 <== mon.1 v2:172.21.15.146:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74d4030020 con 0x55869fbfead0 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.314+0000 7f74c2ffd640 1 -- 172.21.15.18:0/3029011276 <== mon.1 v2:172.21.15.146:3300/0 5 ==== fsmap(e 79) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f74d4023b00 con 0x55869fbfead0 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.314+0000 7f74ecc3d740 10 client.29752 fetch_fsmap finished waiting for FSMap version 79 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.314+0000 7f74ecc3d740 10 client.29752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.314+0000 7f74ecc3d740 20 client.29752 populate_metadata read hostname 'smithi018' 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.314+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/3029011276 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55869fc6e360 0x55869fc70780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:06:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.314+0000 7f74ecc3d740 4 client.29752 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T13:06:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.314+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3029011276 --> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55869f811520 con 0x55869fc6e360 2024-02-17T13:06:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.314+0000 7f74e37fe640 1 --2- 172.21.15.18:0/3029011276 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55869fc6e360 0x55869fc70780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:06:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.315+0000 7f74c2ffd640 10 client.29752 ms_handle_connect on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.315+0000 7f74e37fe640 1 --2- 172.21.15.18:0/3029011276 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55869fc6e360 0x55869fc70780 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f0b89af40ccb7db server_cookie=7ec2d40456d16112 in_seq=0 out_seq=0 2024-02-17T13:06:58.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:06:58 smithi146 ceph-mon[36454]: pgmap v537: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 598 op/s 2024-02-17T13:06:58.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:06:58 smithi018 ceph-mon[32625]: pgmap v537: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 598 op/s 2024-02-17T13:06:58.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:06:58 smithi119 ceph-mon[36188]: pgmap v537: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 598 op/s 2024-02-17T13:06:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.780+0000 7f74c2ffd640 1 -- 172.21.15.18:0/3029011276 <== mds.0 v2:172.21.15.119:6838/2236251663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55869f811520 con 0x55869fc6e360 2024-02-17T13:06:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.780+0000 7f74c2ffd640 10 client.29752 handle_command_reply: tid=0 2024-02-17T13:06:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.780+0000 7f74ecc3d740 1 client.29752 shutdown 2024-02-17T13:06:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.780+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3029011276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 msgr2=0x55869fc0c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.780+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/3029011276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fc0c2c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55869f8a2620 tx=0x7f74d4004700 comp rx=0 tx=0).stop 2024-02-17T13:06:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.780+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3029011276 shutdown_connections 2024-02-17T13:06:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.780+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/3029011276 >> [v2:172.21.15.119:6838/2236251663,v1:172.21.15.119:6839/2236251663] conn(0x55869fc6e360 0x55869fc70780 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.780+0000 7f74c2ffd640 0 client.29752 ms_handle_reset on v2:172.21.15.119:6838/2236251663 2024-02-17T13:06:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.780+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/3029011276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55869fc06da0 0x55869fc6c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.780+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/3029011276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55869fbfead0 0x55869fc0c2c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.780+0000 7f74ecc3d740 1 --2- 172.21.15.18:0/3029011276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55869fbfdd60 0x55869fc0bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.780+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3029011276 >> 172.21.15.18:0/3029011276 conn(0x55869fbc5550 msgr2=0x55869fc06080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.781+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3029011276 shutdown_connections 2024-02-17T13:06:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.781+0000 7f74ecc3d740 1 -- 172.21.15.18:0/3029011276 wait complete. 2024-02-17T13:06:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.781+0000 7f74ecc3d740 20 client.29752 trim_cache size 0 max 16384 2024-02-17T13:06:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-02-17T13:06:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-02-17T13:06:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-02-17T13:06:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-02-17T13:06:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.782+0000 7f74ea06d640 1 -- 172.21.15.18:0/94679751 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74cc0a2500 msgr2=0x7f74cc0a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.782+0000 7f74ea06d640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74cc0a2500 0x7f74cc0a49c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55869fbff230 tx=0x7f74d4009330 comp rx=0 tx=0).stop 2024-02-17T13:06:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.782+0000 7f74ea06d640 1 -- 172.21.15.18:0/94679751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e4105590 msgr2=0x7f74e410d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:06:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.782+0000 7f74ea06d640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e4105590 0x7f74e410d800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f74d8005740 tx=0x7f74d80045e0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.782+0000 7f74ea06d640 1 -- 172.21.15.18:0/94679751 shutdown_connections 2024-02-17T13:06:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.782+0000 7f74ea06d640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f74cc0a2500 0x7f74cc0a49c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.782+0000 7f74ea06d640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e4105590 0x7f74e410d800 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.782+0000 7f74ea06d640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74e4100ab0 0x7f74e4071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.782+0000 7f74ea06d640 1 --2- 172.21.15.18:0/94679751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74e4100190 0x7f74e40719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:06:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.782+0000 7f74ea06d640 1 -- 172.21.15.18:0/94679751 >> 172.21.15.18:0/94679751 conn(0x7f74e4077290 msgr2=0x7f74e4102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:06:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.782+0000 7f74ea06d640 1 -- 172.21.15.18:0/94679751 shutdown_connections 2024-02-17T13:06:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-02-17T13:06:58.782+0000 7f74ea06d640 1 -- 172.21.15.18:0/94679751 wait complete. 2024-02-17T13:06:58.795 INFO:tasks.workunit.client.0.smithi018.stdout: 10 160217 34.11 MB/sec execute 130 sec latency 328.524 ms 2024-02-17T13:06:59.795 INFO:tasks.workunit.client.0.smithi018.stdout: 10 160733 34.08 MB/sec execute 131 sec latency 139.667 ms 2024-02-17T13:07:00.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:00 smithi146 ceph-mon[36454]: pgmap v538: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 633 op/s 2024-02-17T13:07:00.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:00 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:07:00.496 INFO:tasks.workunit.client.0.smithi018.stdout:[232709] write failed on handle 10358 (Resource temporarily unavailable) 2024-02-17T13:07:00.497 INFO:tasks.workunit.client.0.smithi018.stdout:Child failed with status 1 2024-02-17T13:07:00.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T13:07:00.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:00 smithi018 ceph-mon[32625]: pgmap v538: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 633 op/s 2024-02-17T13:07:00.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:00 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:07:00.532 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-02-17T13:07:00.533 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-17T13:07:00.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:00 smithi119 ceph-mon[36188]: pgmap v538: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 633 op/s 2024-02-17T13:07:00.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:00 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:07:00.860 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi018 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6822c7b54e949006be7a0802650d01723d834d1d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-02-17T13:07:01.120 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=3c0730b0578a473eb855034826bbffc6 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi018 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6822c7b54e949006be7a0802650d01723d834d1d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-02-17T13:07:01.123 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-02-17T13:07:01.134 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi018.front.sepia.ceph.com 2024-02-17T13:07:01.148 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-02-17T13:07:01.166 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-02-17T13:07:01.166 INFO:teuthology.task.exec:Executing custom commands... 2024-02-17T13:07:01.167 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-02-17T13:07:01.167 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-02-17T13:07:01.229 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-02-17T13:07:01.298 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-17T13:07:01.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.454+0000 7f135f2a3640 1 -- 172.21.15.18:0/3201217418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13581003b0 msgr2=0x7f1358100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:01.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.454+0000 7f135f2a3640 1 --2- 172.21.15.18:0/3201217418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13581003b0 0x7f1358100790 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f1354009930 tx=0x7f135402bc30 comp rx=0 tx=0).stop 2024-02-17T13:07:01.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.454+0000 7f135f2a3640 1 -- 172.21.15.18:0/3201217418 shutdown_connections 2024-02-17T13:07:01.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.454+0000 7f135f2a3640 1 --2- 172.21.15.18:0/3201217418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1358102630 0x7f135810d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.454+0000 7f135f2a3640 1 --2- 172.21.15.18:0/3201217418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1358101c90 0x7f13581020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.454+0000 7f135f2a3640 1 --2- 172.21.15.18:0/3201217418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13581003b0 0x7f1358100790 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.454+0000 7f135f2a3640 1 -- 172.21.15.18:0/3201217418 >> 172.21.15.18:0/3201217418 conn(0x7f13580771a0 msgr2=0x7f13580775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.454+0000 7f135f2a3640 1 -- 172.21.15.18:0/3201217418 shutdown_connections 2024-02-17T13:07:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.454+0000 7f135f2a3640 1 -- 172.21.15.18:0/3201217418 wait complete. 2024-02-17T13:07:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.454+0000 7f135f2a3640 1 Processor -- start 2024-02-17T13:07:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.454+0000 7f135f2a3640 1 -- start start 2024-02-17T13:07:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135f2a3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13581003b0 0x7f135819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135f2a3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1358101c90 0x7f135819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135f2a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1358102630 0x7f1358197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135f2a3640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1358197c50 con 0x7f1358101c90 2024-02-17T13:07:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135f2a3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1358197dc0 con 0x7f1358102630 2024-02-17T13:07:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135f2a3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1358197f80 con 0x7f13581003b0 2024-02-17T13:07:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135c83e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1358101c90 0x7f135819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135c83e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1358101c90 0x7f135819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46158/0 (socket says 172.21.15.18:46158) 2024-02-17T13:07:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135c83e640 1 -- 172.21.15.18:0/3123637822 learned_addr learned my addr 172.21.15.18:0/3123637822 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135d03f640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13581003b0 0x7f135819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135d840640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1358102630 0x7f1358197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135c83e640 1 -- 172.21.15.18:0/3123637822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13581003b0 msgr2=0x7f135819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135c83e640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13581003b0 0x7f135819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135c83e640 1 -- 172.21.15.18:0/3123637822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1358102630 msgr2=0x7f1358197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135c83e640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1358102630 0x7f1358197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135c83e640 1 -- 172.21.15.18:0/3123637822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1354009560 con 0x7f1358101c90 2024-02-17T13:07:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135c83e640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1358101c90 0x7f135819d940 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f134800e9f0 tx=0x7f134800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f134e7fc640 1 -- 172.21.15.18:0/3123637822 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1348009720 con 0x7f1358101c90 2024-02-17T13:07:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135f2a3640 1 -- 172.21.15.18:0/3123637822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1358198260 con 0x7f1358101c90 2024-02-17T13:07:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f134e7fc640 1 -- 172.21.15.18:0/3123637822 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f134800cd80 con 0x7f1358101c90 2024-02-17T13:07:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f134e7fc640 1 -- 172.21.15.18:0/3123637822 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1348010690 con 0x7f1358101c90 2024-02-17T13:07:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.455+0000 7f135f2a3640 1 -- 172.21.15.18:0/3123637822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13581a44a0 con 0x7f1358101c90 2024-02-17T13:07:01.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.456+0000 7f135f2a3640 1 -- 172.21.15.18:0/3123637822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135804eee0 con 0x7f1358101c90 2024-02-17T13:07:01.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.457+0000 7f134e7fc640 1 -- 172.21.15.18:0/3123637822 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f13480046c0 con 0x7f1358101c90 2024-02-17T13:07:01.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.458+0000 7f134e7fc640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13440a22d0 0x7f13440a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:01.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.458+0000 7f135d03f640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13440a22d0 0x7f13440a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:01.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.458+0000 7f134e7fc640 1 -- 172.21.15.18:0/3123637822 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1348014070 con 0x7f1358101c90 2024-02-17T13:07:01.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.458+0000 7f135d03f640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13440a22d0 0x7f13440a4790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1354005ab0 tx=0x7f1354005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:01.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.463+0000 7f134e7fc640 1 -- 172.21.15.18:0/3123637822 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1348097160 con 0x7f1358101c90 2024-02-17T13:07:01.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.664+0000 7f135f2a3640 1 -- 172.21.15.18:0/3123637822 --> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f13581086d0 con 0x7f13440a22d0 2024-02-17T13:07:01.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.674+0000 7f134e7fc640 1 -- 172.21.15.18:0/3123637822 <== mgr.15528 v2:172.21.15.18:6800/2186095810 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f13581086d0 con 0x7f13440a22d0 2024-02-17T13:07:01.664 INFO:teuthology.orchestra.run.smithi018.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-17T00:00:00", "created": "2024-02-17T12:49:53", "first": "2024-02-17T12:50:00", "last": "2024-02-17T13:07:00", "last_pruned": "2024-02-17T13:07:00", "created_count": 18, "pruned_count": 11, "active": true} 2024-02-17T13:07:01.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.676+0000 7f135f2a3640 1 -- 172.21.15.18:0/3123637822 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13440a22d0 msgr2=0x7f13440a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:01.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.676+0000 7f135f2a3640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13440a22d0 0x7f13440a4790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1354005ab0 tx=0x7f1354005fd0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.676+0000 7f135f2a3640 1 -- 172.21.15.18:0/3123637822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1358101c90 msgr2=0x7f135819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:01.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.676+0000 7f135f2a3640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1358101c90 0x7f135819d940 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f134800e9f0 tx=0x7f134800edf0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.676+0000 7f135f2a3640 1 -- 172.21.15.18:0/3123637822 shutdown_connections 2024-02-17T13:07:01.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.676+0000 7f135f2a3640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f13440a22d0 0x7f13440a4790 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.676+0000 7f135f2a3640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1358102630 0x7f1358197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.676+0000 7f135f2a3640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1358101c90 0x7f135819d940 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.676+0000 7f135f2a3640 1 --2- 172.21.15.18:0/3123637822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13581003b0 0x7f135819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.676+0000 7f135f2a3640 1 -- 172.21.15.18:0/3123637822 >> 172.21.15.18:0/3123637822 conn(0x7f13580771a0 msgr2=0x7f1358103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:01.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.676+0000 7f135f2a3640 1 -- 172.21.15.18:0/3123637822 shutdown_connections 2024-02-17T13:07:01.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.676+0000 7f135f2a3640 1 -- 172.21.15.18:0/3123637822 wait complete. 2024-02-17T13:07:01.676 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-17T13:07:01.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 -- 172.21.15.18:0/50283038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c0c105020 msgr2=0x7f8c0c105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:01.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 --2- 172.21.15.18:0/50283038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c0c105020 0x7f8c0c105400 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009ca0 tx=0x7f8c0802c0f0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 -- 172.21.15.18:0/50283038 shutdown_connections 2024-02-17T13:07:01.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 --2- 172.21.15.18:0/50283038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c100c90 0x7f8c0c10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 --2- 172.21.15.18:0/50283038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c0c1002d0 0x7f8c0c100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 --2- 172.21.15.18:0/50283038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c0c105020 0x7f8c0c105400 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 -- 172.21.15.18:0/50283038 >> 172.21.15.18:0/50283038 conn(0x7f8c0c076ed0 msgr2=0x7f8c0c0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:01.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 -- 172.21.15.18:0/50283038 shutdown_connections 2024-02-17T13:07:01.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 -- 172.21.15.18:0/50283038 wait complete. 2024-02-17T13:07:01.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 Processor -- start 2024-02-17T13:07:01.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 -- start start 2024-02-17T13:07:01.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c0c1002d0 0x7f8c0c197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:01.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c0c100c90 0x7f8c0c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:01.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c105020 0x7f8c0c19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:01.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c19c5c0 con 0x7f8c0c100c90 2024-02-17T13:07:01.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c19c730 con 0x7f8c0c1002d0 2024-02-17T13:07:01.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.792+0000 7f8c13c0f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c19c8f0 con 0x7f8c0c105020 2024-02-17T13:07:01.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c111aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c0c100c90 0x7f8c0c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:01.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c111aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c0c100c90 0x7f8c0c197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46178/0 (socket says 172.21.15.18:46178) 2024-02-17T13:07:01.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c111aa640 1 -- 172.21.15.18:0/705420903 learned_addr learned my addr 172.21.15.18:0/705420903 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:01.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c119ab640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c0c1002d0 0x7f8c0c197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:01.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c121ac640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c105020 0x7f8c0c19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:01.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c111aa640 1 -- 172.21.15.18:0/705420903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c105020 msgr2=0x7f8c0c19bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:01.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c111aa640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c105020 0x7f8c0c19bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c111aa640 1 -- 172.21.15.18:0/705420903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c0c1002d0 msgr2=0x7f8c0c197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:01.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c111aa640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c0c1002d0 0x7f8c0c197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:01.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c111aa640 1 -- 172.21.15.18:0/705420903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c08009900 con 0x7f8c0c100c90 2024-02-17T13:07:01.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c111aa640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c0c100c90 0x7f8c0c197c40 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00c920 tx=0x7f8bfc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:01.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c02ffd640 1 -- 172.21.15.18:0/705420903 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc007c70 con 0x7f8c0c100c90 2024-02-17T13:07:01.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c02ffd640 1 -- 172.21.15.18:0/705420903 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bfc007dd0 con 0x7f8c0c100c90 2024-02-17T13:07:01.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c13c0f640 1 -- 172.21.15.18:0/705420903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0c19cbd0 con 0x7f8c0c100c90 2024-02-17T13:07:01.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c02ffd640 1 -- 172.21.15.18:0/705420903 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc018860 con 0x7f8c0c100c90 2024-02-17T13:07:01.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.793+0000 7f8c13c0f640 1 -- 172.21.15.18:0/705420903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0c1a4450 con 0x7f8c0c100c90 2024-02-17T13:07:01.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.794+0000 7f8c13c0f640 1 -- 172.21.15.18:0/705420903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c0c04eee0 con 0x7f8c0c100c90 2024-02-17T13:07:01.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.795+0000 7f8c02ffd640 1 -- 172.21.15.18:0/705420903 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8bfc00fac0 con 0x7f8c0c100c90 2024-02-17T13:07:01.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.796+0000 7f8c02ffd640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8bf80a2350 0x7f8bf80a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:01.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.796+0000 7f8c119ab640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8bf80a2350 0x7f8bf80a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:01.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.796+0000 7f8c02ffd640 1 -- 172.21.15.18:0/705420903 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8bfc0ce3a0 con 0x7f8c0c100c90 2024-02-17T13:07:01.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.796+0000 7f8c119ab640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8bf80a2350 0x7f8bf80a4810 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8c08004af0 tx=0x7f8c08005c50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:01.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:01.799+0000 7f8c02ffd640 1 -- 172.21.15.18:0/705420903 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8bfc097100 con 0x7f8c0c100c90 2024-02-17T13:07:01.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.004+0000 7f8c13c0f640 1 -- 172.21.15.18:0/705420903 --> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8c0c1113c0 con 0x7f8bf80a2350 2024-02-17T13:07:02.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.014+0000 7f8c02ffd640 1 -- 172.21.15.18:0/705420903 <== mgr.15528 v2:172.21.15.18:6800/2186095810 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f8c0c1113c0 con 0x7f8bf80a2350 2024-02-17T13:07:02.004 INFO:teuthology.orchestra.run.smithi018.stdout:/ 1m 6m3h 2024-02-17T13:07:02.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.016+0000 7f8c13c0f640 1 -- 172.21.15.18:0/705420903 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8bf80a2350 msgr2=0x7f8bf80a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.016+0000 7f8c13c0f640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8bf80a2350 0x7f8bf80a4810 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8c08004af0 tx=0x7f8c08005c50 comp rx=0 tx=0).stop 2024-02-17T13:07:02.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.016+0000 7f8c13c0f640 1 -- 172.21.15.18:0/705420903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c0c100c90 msgr2=0x7f8c0c197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.016+0000 7f8c13c0f640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c0c100c90 0x7f8c0c197c40 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00c920 tx=0x7f8bfc00cde0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.016+0000 7f8c13c0f640 1 -- 172.21.15.18:0/705420903 shutdown_connections 2024-02-17T13:07:02.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.016+0000 7f8c13c0f640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f8bf80a2350 0x7f8bf80a4810 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.016+0000 7f8c13c0f640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c105020 0x7f8c0c19bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.016+0000 7f8c13c0f640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c0c100c90 0x7f8c0c197c40 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.016+0000 7f8c13c0f640 1 --2- 172.21.15.18:0/705420903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c0c1002d0 0x7f8c0c197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.016+0000 7f8c13c0f640 1 -- 172.21.15.18:0/705420903 >> 172.21.15.18:0/705420903 conn(0x7f8c0c076ed0 msgr2=0x7f8c0c0fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:02.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.016+0000 7f8c13c0f640 1 -- 172.21.15.18:0/705420903 shutdown_connections 2024-02-17T13:07:02.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.016+0000 7f8c13c0f640 1 -- 172.21.15.18:0/705420903 wait complete. 2024-02-17T13:07:02.016 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-02-17T13:07:02.027 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-02-17T13:07:02.027 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-02-17T13:07:02.031 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-02-17T13:07:02.067 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T13:07:02.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.185+0000 7f752494f640 1 -- 172.21.15.18:0/3756593713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75201053c0 msgr2=0x7f752010cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.185+0000 7f752494f640 1 --2- 172.21.15.18:0/3756593713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75201053c0 0x7f752010cf90 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f7518009cd0 tx=0x7f751802c0f0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.185+0000 7f752494f640 1 -- 172.21.15.18:0/3756593713 shutdown_connections 2024-02-17T13:07:02.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.185+0000 7f752494f640 1 --2- 172.21.15.18:0/3756593713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75201053c0 0x7f752010cf90 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.185+0000 7f752494f640 1 --2- 172.21.15.18:0/3756593713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7520100ca0 0x7f7520104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.185+0000 7f752494f640 1 --2- 172.21.15.18:0/3756593713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75201002f0 0x7f75201006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.185+0000 7f752494f640 1 -- 172.21.15.18:0/3756593713 >> 172.21.15.18:0/3756593713 conn(0x7f7520077180 msgr2=0x7f7520077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.185+0000 7f752494f640 1 -- 172.21.15.18:0/3756593713 shutdown_connections 2024-02-17T13:07:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.185+0000 7f752494f640 1 -- 172.21.15.18:0/3756593713 wait complete. 2024-02-17T13:07:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f752494f640 1 Processor -- start 2024-02-17T13:07:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f752494f640 1 -- start start 2024-02-17T13:07:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f752494f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75201002f0 0x7f7520101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f752494f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75201053c0 0x7f75201022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f752494f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7520102800 0x7f75201a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f752494f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75201a5f60 con 0x7f7520102800 2024-02-17T13:07:02.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f752494f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75201a60d0 con 0x7f75201002f0 2024-02-17T13:07:02.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f752494f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75201a6290 con 0x7f75201053c0 2024-02-17T13:07:02.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f751ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7520102800 0x7f75201a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:02.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f751ed9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7520102800 0x7f75201a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46186/0 (socket says 172.21.15.18:46186) 2024-02-17T13:07:02.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f751ed9d640 1 -- 172.21.15.18:0/3920365880 learned_addr learned my addr 172.21.15.18:0/3920365880 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:02.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f7517fff640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75201053c0 0x7f75201022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:02.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.186+0000 7f751e59c640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75201002f0 0x7f7520101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:02.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.187+0000 7f751ed9d640 1 -- 172.21.15.18:0/3920365880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75201053c0 msgr2=0x7f75201022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.187+0000 7f751ed9d640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75201053c0 0x7f75201022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.187+0000 7f751ed9d640 1 -- 172.21.15.18:0/3920365880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75201002f0 msgr2=0x7f7520101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.187+0000 7f751ed9d640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75201002f0 0x7f7520101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.187+0000 7f751ed9d640 1 -- 172.21.15.18:0/3920365880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7518009900 con 0x7f7520102800 2024-02-17T13:07:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.187+0000 7f751ed9d640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7520102800 0x7f75201a5990 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f7518005e30 tx=0x7f75180325e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.187+0000 7f75177fe640 1 -- 172.21.15.18:0/3920365880 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518034e20 con 0x7f7520102800 2024-02-17T13:07:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.187+0000 7f75177fe640 1 -- 172.21.15.18:0/3920365880 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7518003e50 con 0x7f7520102800 2024-02-17T13:07:02.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.187+0000 7f75177fe640 1 -- 172.21.15.18:0/3920365880 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518004750 con 0x7f7520102800 2024-02-17T13:07:02.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.187+0000 7f752494f640 1 -- 172.21.15.18:0/3920365880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75201a6510 con 0x7f7520102800 2024-02-17T13:07:02.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.187+0000 7f752494f640 1 -- 172.21.15.18:0/3920365880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75201a69c0 con 0x7f7520102800 2024-02-17T13:07:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.188+0000 7f752494f640 1 -- 172.21.15.18:0/3920365880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7520067f70 con 0x7f7520102800 2024-02-17T13:07:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.189+0000 7f75177fe640 1 -- 172.21.15.18:0/3920365880 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f751800a040 con 0x7f7520102800 2024-02-17T13:07:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.190+0000 7f75177fe640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f750c0a2260 0x7f750c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.190+0000 7f751e59c640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f750c0a2260 0x7f750c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.190+0000 7f75177fe640 1 -- 172.21.15.18:0/3920365880 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f75180f0ba0 con 0x7f7520102800 2024-02-17T13:07:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.190+0000 7f751e59c640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f750c0a2260 0x7f750c0a4720 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7510009bd0 tx=0x7f7510005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:02.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.194+0000 7f75177fe640 1 -- 172.21.15.18:0/3920365880 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f75180b9900 con 0x7f7520102800 2024-02-17T13:07:02.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:02 smithi018 ceph-mon[32625]: pgmap v539: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 859 op/s 2024-02-17T13:07:02.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:02 smithi018 ceph-mon[32625]: from='client.20829 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T13:07:02.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.378+0000 7f752494f640 1 -- 172.21.15.18:0/3920365880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7520109600 con 0x7f7520102800 2024-02-17T13:07:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.379+0000 7f75177fe640 1 -- 172.21.15.18:0/3920365880 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+23079 (secure 0 0 0) 0x7f75180bcfc0 con 0x7f7520102800 2024-02-17T13:07:02.369 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:07:02.369 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":111,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T13:06:13.918938+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":28,"snap_epoch":110,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/1074404031":"2024-02-18T12:49:52.647628+0000","172.21.15.18:6801/3026633658":"2024-02-18T12:49:52.647628+0000","172.21.15.18:6800/3026633658":"2024-02-18T12:49:52.647628+0000","172.21.15.18:0/2513093443":"2024-02-18T12:49:52.647628+0000","172.21.15.18:0/1103049228":"2024-02-18T12:49:52.647628+0000","172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/3776356008":"2024-02-18T12:49:52.647628+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":28,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T13:07:02.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.381+0000 7f752494f640 1 -- 172.21.15.18:0/3920365880 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f750c0a2260 msgr2=0x7f750c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.381+0000 7f752494f640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f750c0a2260 0x7f750c0a4720 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7510009bd0 tx=0x7f7510005f90 comp rx=0 tx=0).stop 2024-02-17T13:07:02.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.381+0000 7f752494f640 1 -- 172.21.15.18:0/3920365880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7520102800 msgr2=0x7f75201a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.381+0000 7f752494f640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7520102800 0x7f75201a5990 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f7518005e30 tx=0x7f75180325e0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.382+0000 7f752494f640 1 -- 172.21.15.18:0/3920365880 shutdown_connections 2024-02-17T13:07:02.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.382+0000 7f752494f640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f750c0a2260 0x7f750c0a4720 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.382+0000 7f752494f640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7520102800 0x7f75201a5990 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.382+0000 7f752494f640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75201053c0 0x7f75201022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.382+0000 7f752494f640 1 --2- 172.21.15.18:0/3920365880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75201002f0 0x7f7520101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.382+0000 7f752494f640 1 -- 172.21.15.18:0/3920365880 >> 172.21.15.18:0/3920365880 conn(0x7f7520077180 msgr2=0x7f75200685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.382+0000 7f752494f640 1 -- 172.21.15.18:0/3920365880 shutdown_connections 2024-02-17T13:07:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.382+0000 7f752494f640 1 -- 172.21.15.18:0/3920365880 wait complete. 2024-02-17T13:07:02.382 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T12:43:12.450140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T12:47:43.690543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T12:47:44.653229+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 28, 'snap_epoch': 110, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T13:07:02.383 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T13:07:02.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:02 smithi146 ceph-mon[36454]: pgmap v539: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 859 op/s 2024-02-17T13:07:02.449 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:02 smithi146 ceph-mon[36454]: from='client.20829 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T13:07:02.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.500+0000 7fc531a62640 1 -- 172.21.15.18:0/2209930050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c10bda0 msgr2=0x7fc52c10c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.500+0000 7fc531a62640 1 --2- 172.21.15.18:0/2209930050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c10bda0 0x7fc52c10c180 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009930 tx=0x7fc51c02bc30 comp rx=0 tx=0).stop 2024-02-17T13:07:02.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.500+0000 7fc531a62640 1 -- 172.21.15.18:0/2209930050 shutdown_connections 2024-02-17T13:07:02.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.500+0000 7fc531a62640 1 --2- 172.21.15.18:0/2209930050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c110e60 0x7fc52c118a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.500+0000 7fc531a62640 1 --2- 172.21.15.18:0/2209930050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc52c10c750 0x7fc52c110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.500+0000 7fc531a62640 1 --2- 172.21.15.18:0/2209930050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c10bda0 0x7fc52c10c180 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.500+0000 7fc531a62640 1 -- 172.21.15.18:0/2209930050 >> 172.21.15.18:0/2209930050 conn(0x7fc52c06c180 msgr2=0x7fc52c10a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:02.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.500+0000 7fc531a62640 1 -- 172.21.15.18:0/2209930050 shutdown_connections 2024-02-17T13:07:02.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.500+0000 7fc531a62640 1 -- 172.21.15.18:0/2209930050 wait complete. 2024-02-17T13:07:02.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc531a62640 1 Processor -- start 2024-02-17T13:07:02.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc531a62640 1 -- start start 2024-02-17T13:07:02.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc531a62640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c10bda0 0x7fc52c1a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:02.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc531a62640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c110e60 0x7fc52c1a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:02.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc531a62640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc52c1a6b20 0x7fc52c1a6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:02.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc531a62640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c1a7510 con 0x7fc52c10bda0 2024-02-17T13:07:02.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc531a62640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c1a7680 con 0x7fc52c110e60 2024-02-17T13:07:02.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc531a62640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c1a77f0 con 0x7fc52c1a6b20 2024-02-17T13:07:02.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc52b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c10bda0 0x7fc52c1a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:02.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc52b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c10bda0 0x7fc52c1a2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46198/0 (socket says 172.21.15.18:46198) 2024-02-17T13:07:02.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc52b7fe640 1 -- 172.21.15.18:0/1615331585 learned_addr learned my addr 172.21.15.18:0/1615331585 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:02.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc52bfff640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc52c1a6b20 0x7fc52c1a6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:02.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc52affd640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c110e60 0x7fc52c1a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:02.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc52b7fe640 1 -- 172.21.15.18:0/1615331585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc52c1a6b20 msgr2=0x7fc52c1a6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc52b7fe640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc52c1a6b20 0x7fc52c1a6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc52b7fe640 1 -- 172.21.15.18:0/1615331585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c110e60 msgr2=0x7fc52c1a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc52b7fe640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c110e60 0x7fc52c1a27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.501+0000 7fc52b7fe640 1 -- 172.21.15.18:0/1615331585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c009560 con 0x7fc52c10bda0 2024-02-17T13:07:02.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.502+0000 7fc52b7fe640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c10bda0 0x7fc52c1a2260 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009230 tx=0x7fc51c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:02.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.502+0000 7fc528ff9640 1 -- 172.21.15.18:0/1615331585 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c004440 con 0x7fc52c10bda0 2024-02-17T13:07:02.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.502+0000 7fc531a62640 1 -- 172.21.15.18:0/1615331585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c1aedd0 con 0x7fc52c10bda0 2024-02-17T13:07:02.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.502+0000 7fc528ff9640 1 -- 172.21.15.18:0/1615331585 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc51c0052d0 con 0x7fc52c10bda0 2024-02-17T13:07:02.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.502+0000 7fc528ff9640 1 -- 172.21.15.18:0/1615331585 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c003d70 con 0x7fc52c10bda0 2024-02-17T13:07:02.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.502+0000 7fc531a62640 1 -- 172.21.15.18:0/1615331585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c1af1f0 con 0x7fc52c10bda0 2024-02-17T13:07:02.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.503+0000 7fc531a62640 1 -- 172.21.15.18:0/1615331585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52c067f70 con 0x7fc52c10bda0 2024-02-17T13:07:02.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.505+0000 7fc528ff9640 1 -- 172.21.15.18:0/1615331585 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc51c005440 con 0x7fc52c10bda0 2024-02-17T13:07:02.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.505+0000 7fc528ff9640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5140a2230 0x7fc5140a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:02.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.505+0000 7fc52affd640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5140a2230 0x7fc5140a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:02.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.505+0000 7fc528ff9640 1 -- 172.21.15.18:0/1615331585 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fc51c0f0a40 con 0x7fc52c10bda0 2024-02-17T13:07:02.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.506+0000 7fc52affd640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5140a2230 0x7fc5140a46f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc520005fd0 tx=0x7fc520005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:02.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.509+0000 7fc528ff9640 1 -- 172.21.15.18:0/1615331585 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc51c0b9800 con 0x7fc52c10bda0 2024-02-17T13:07:02.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:02 smithi119 ceph-mon[36188]: pgmap v539: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 859 op/s 2024-02-17T13:07:02.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:02 smithi119 ceph-mon[36188]: from='client.20829 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T13:07:02.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.689+0000 7fc531a62640 1 -- 172.21.15.18:0/1615331585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc52c10c750 con 0x7fc52c10bda0 2024-02-17T13:07:02.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.690+0000 7fc528ff9640 1 -- 172.21.15.18:0/1615331585 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc51c047e90 con 0x7fc52c10bda0 2024-02-17T13:07:02.679 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-02-17T13:07:02.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.692+0000 7fc531a62640 1 -- 172.21.15.18:0/1615331585 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5140a2230 msgr2=0x7fc5140a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.692+0000 7fc531a62640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5140a2230 0x7fc5140a46f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc520005fd0 tx=0x7fc520005850 comp rx=0 tx=0).stop 2024-02-17T13:07:02.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.692+0000 7fc531a62640 1 -- 172.21.15.18:0/1615331585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c10bda0 msgr2=0x7fc52c1a2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.692+0000 7fc531a62640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c10bda0 0x7fc52c1a2260 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009230 tx=0x7fc51c004040 comp rx=0 tx=0).stop 2024-02-17T13:07:02.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.692+0000 7fc531a62640 1 -- 172.21.15.18:0/1615331585 shutdown_connections 2024-02-17T13:07:02.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.692+0000 7fc531a62640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fc5140a2230 0x7fc5140a46f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.692+0000 7fc531a62640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc52c1a6b20 0x7fc52c1a6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.692+0000 7fc531a62640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc52c110e60 0x7fc52c1a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.692+0000 7fc531a62640 1 --2- 172.21.15.18:0/1615331585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc52c10bda0 0x7fc52c1a2260 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.692+0000 7fc531a62640 1 -- 172.21.15.18:0/1615331585 >> 172.21.15.18:0/1615331585 conn(0x7fc52c06c180 msgr2=0x7fc52c10f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:02.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.692+0000 7fc531a62640 1 -- 172.21.15.18:0/1615331585 shutdown_connections 2024-02-17T13:07:02.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.692+0000 7fc531a62640 1 -- 172.21.15.18:0/1615331585 wait complete. 2024-02-17T13:07:02.692 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T13:07:02.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.809+0000 7fe2414ae640 1 -- 172.21.15.18:0/1029488176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe23c10b7c0 msgr2=0x7fe23c10dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.809+0000 7fe2414ae640 1 --2- 172.21.15.18:0/1029488176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe23c10b7c0 0x7fe23c10dc30 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fe234009ca0 tx=0x7fe23402c180 comp rx=0 tx=0).stop 2024-02-17T13:07:02.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.810+0000 7fe2414ae640 1 -- 172.21.15.18:0/1029488176 shutdown_connections 2024-02-17T13:07:02.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.810+0000 7fe2414ae640 1 --2- 172.21.15.18:0/1029488176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe23c10b7c0 0x7fe23c10dc30 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.810+0000 7fe2414ae640 1 --2- 172.21.15.18:0/1029488176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe23c102d10 0x7fe23c10b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.810+0000 7fe2414ae640 1 --2- 172.21.15.18:0/1029488176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe23c1023f0 0x7fe23c1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.810+0000 7fe2414ae640 1 -- 172.21.15.18:0/1029488176 >> 172.21.15.18:0/1029488176 conn(0x7fe23c0771e0 msgr2=0x7fe23c0775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:02.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.810+0000 7fe2414ae640 1 -- 172.21.15.18:0/1029488176 shutdown_connections 2024-02-17T13:07:02.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.810+0000 7fe2414ae640 1 -- 172.21.15.18:0/1029488176 wait complete. 2024-02-17T13:07:02.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.810+0000 7fe2414ae640 1 Processor -- start 2024-02-17T13:07:02.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.810+0000 7fe2414ae640 1 -- start start 2024-02-17T13:07:02.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.810+0000 7fe2414ae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe23c1023f0 0x7fe23c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:02.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.810+0000 7fe2414ae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe23c102d10 0x7fe23c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:02.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.810+0000 7fe2414ae640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe23c10b7c0 0x7fe23c197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:02.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe2414ae640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c197c80 con 0x7fe23c10b7c0 2024-02-17T13:07:02.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe2414ae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c197df0 con 0x7fe23c1023f0 2024-02-17T13:07:02.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe2414ae640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c197fb0 con 0x7fe23c102d10 2024-02-17T13:07:02.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe23b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe23c10b7c0 0x7fe23c197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:02.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe23b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe23c10b7c0 0x7fe23c197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46218/0 (socket says 172.21.15.18:46218) 2024-02-17T13:07:02.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe23b7fe640 1 -- 172.21.15.18:0/3459163503 learned_addr learned my addr 172.21.15.18:0/3459163503 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:02.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe23affd640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe23c1023f0 0x7fe23c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:02.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe23a7fc640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe23c102d10 0x7fe23c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:02.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe23b7fe640 1 -- 172.21.15.18:0/3459163503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe23c102d10 msgr2=0x7fe23c19d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe23b7fe640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe23c102d10 0x7fe23c19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe23b7fe640 1 -- 172.21.15.18:0/3459163503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe23c1023f0 msgr2=0x7fe23c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe23b7fe640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe23c1023f0 0x7fe23c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe23b7fe640 1 -- 172.21.15.18:0/3459163503 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe234009900 con 0x7fe23c10b7c0 2024-02-17T13:07:02.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe23affd640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe23c1023f0 0x7fe23c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:07:02.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe23b7fe640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe23c10b7c0 0x7fe23c197630 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fe23402c790 tx=0x7fe234034ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:02.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe223fff640 1 -- 172.21.15.18:0/3459163503 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234032790 con 0x7fe23c10b7c0 2024-02-17T13:07:02.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe223fff640 1 -- 172.21.15.18:0/3459163503 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe234032dd0 con 0x7fe23c10b7c0 2024-02-17T13:07:02.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe2414ae640 1 -- 172.21.15.18:0/3459163503 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23c198230 con 0x7fe23c10b7c0 2024-02-17T13:07:02.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe223fff640 1 -- 172.21.15.18:0/3459163503 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234040450 con 0x7fe23c10b7c0 2024-02-17T13:07:02.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.811+0000 7fe2414ae640 1 -- 172.21.15.18:0/3459163503 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23c1a44d0 con 0x7fe23c10b7c0 2024-02-17T13:07:02.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.812+0000 7fe2414ae640 1 -- 172.21.15.18:0/3459163503 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23c0683c0 con 0x7fe23c10b7c0 2024-02-17T13:07:02.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.813+0000 7fe223fff640 1 -- 172.21.15.18:0/3459163503 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe23400a040 con 0x7fe23c10b7c0 2024-02-17T13:07:02.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.814+0000 7fe223fff640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe2240a2230 0x7fe2240a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:02.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.814+0000 7fe23affd640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe2240a2230 0x7fe2240a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:02.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.814+0000 7fe223fff640 1 -- 172.21.15.18:0/3459163503 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe2340f1460 con 0x7fe23c10b7c0 2024-02-17T13:07:02.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.814+0000 7fe23affd640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe2240a2230 0x7fe2240a46f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe22c006fd0 tx=0x7fe22c008040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:02.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:02.817+0000 7fe223fff640 1 -- 172.21.15.18:0/3459163503 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe2340ba1c0 con 0x7fe23c10b7c0 2024-02-17T13:07:02.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.001+0000 7fe2414ae640 1 -- 172.21.15.18:0/3459163503 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe23c111350 con 0x7fe23c10b7c0 2024-02-17T13:07:02.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.001+0000 7fe223fff640 1 -- 172.21.15.18:0/3459163503 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2340bd880 con 0x7fe23c10b7c0 2024-02-17T13:07:02.991 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-02-17T13:07:02.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.003+0000 7fe2414ae640 1 -- 172.21.15.18:0/3459163503 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe2240a2230 msgr2=0x7fe2240a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.003+0000 7fe2414ae640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe2240a2230 0x7fe2240a46f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe22c006fd0 tx=0x7fe22c008040 comp rx=0 tx=0).stop 2024-02-17T13:07:02.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.003+0000 7fe2414ae640 1 -- 172.21.15.18:0/3459163503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe23c10b7c0 msgr2=0x7fe23c197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:02.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.003+0000 7fe2414ae640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe23c10b7c0 0x7fe23c197630 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fe23402c790 tx=0x7fe234034ea0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.003+0000 7fe2414ae640 1 -- 172.21.15.18:0/3459163503 shutdown_connections 2024-02-17T13:07:02.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.003+0000 7fe2414ae640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fe2240a2230 0x7fe2240a46f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.003+0000 7fe2414ae640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe23c10b7c0 0x7fe23c197630 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.003+0000 7fe2414ae640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe23c102d10 0x7fe23c19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.003+0000 7fe2414ae640 1 --2- 172.21.15.18:0/3459163503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe23c1023f0 0x7fe23c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:02.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.003+0000 7fe2414ae640 1 -- 172.21.15.18:0/3459163503 >> 172.21.15.18:0/3459163503 conn(0x7fe23c0771e0 msgr2=0x7fe23c0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:02.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.003+0000 7fe2414ae640 1 -- 172.21.15.18:0/3459163503 shutdown_connections 2024-02-17T13:07:02.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.003+0000 7fe2414ae640 1 -- 172.21.15.18:0/3459163503 wait complete. 2024-02-17T13:07:03.003 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T13:07:03.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.121+0000 7f96b9a55640 1 -- 172.21.15.18:0/1342322012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96b4068380 msgr2=0x7f96b4068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:03.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.121+0000 7f96b9a55640 1 --2- 172.21.15.18:0/1342322012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96b4068380 0x7f96b4068760 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009900 tx=0x7f96a402bcd0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.121+0000 7f96b9a55640 1 -- 172.21.15.18:0/1342322012 shutdown_connections 2024-02-17T13:07:03.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.121+0000 7f96b9a55640 1 --2- 172.21.15.18:0/1342322012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b410db20 0x7f96b410ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.121+0000 7f96b9a55640 1 --2- 172.21.15.18:0/1342322012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b4068ca0 0x7f96b410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.121+0000 7f96b9a55640 1 --2- 172.21.15.18:0/1342322012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96b4068380 0x7f96b4068760 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.121+0000 7f96b9a55640 1 -- 172.21.15.18:0/1342322012 >> 172.21.15.18:0/1342322012 conn(0x7f96b4077180 msgr2=0x7f96b4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:03.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.121+0000 7f96b9a55640 1 -- 172.21.15.18:0/1342322012 shutdown_connections 2024-02-17T13:07:03.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.121+0000 7f96b9a55640 1 -- 172.21.15.18:0/1342322012 wait complete. 2024-02-17T13:07:03.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.121+0000 7f96b9a55640 1 Processor -- start 2024-02-17T13:07:03.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.121+0000 7f96b9a55640 1 -- start start 2024-02-17T13:07:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b9a55640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96b4068380 0x7f96b419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b9a55640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b4068ca0 0x7f96b419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b9a55640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b410db20 0x7f96b4199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b9a55640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b4199ea0 con 0x7f96b4068380 2024-02-17T13:07:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b9a55640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b419a010 con 0x7f96b4068ca0 2024-02-17T13:07:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b9a55640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b419a1d0 con 0x7f96b410db20 2024-02-17T13:07:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96b4068380 0x7f96b419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96b4068380 0x7f96b419f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46236/0 (socket says 172.21.15.18:46236) 2024-02-17T13:07:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b2ffd640 1 -- 172.21.15.18:0/1837246601 learned_addr learned my addr 172.21.15.18:0/1837246601 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b27fc640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b4068ca0 0x7f96b419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b37fe640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b410db20 0x7f96b4199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b2ffd640 1 -- 172.21.15.18:0/1837246601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b410db20 msgr2=0x7f96b4199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b2ffd640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b410db20 0x7f96b4199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b2ffd640 1 -- 172.21.15.18:0/1837246601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b4068ca0 msgr2=0x7f96b419fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b2ffd640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b4068ca0 0x7f96b419fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b2ffd640 1 -- 172.21.15.18:0/1837246601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a4009560 con 0x7f96b4068380 2024-02-17T13:07:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b27fc640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b4068ca0 0x7f96b419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:07:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b2ffd640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96b4068380 0x7f96b419f650 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f96a4005ec0 tx=0x7f96a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:03.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b8a53640 1 -- 172.21.15.18:0/1837246601 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a40044e0 con 0x7f96b4068380 2024-02-17T13:07:03.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b9a55640 1 -- 172.21.15.18:0/1837246601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b419a450 con 0x7f96b4068380 2024-02-17T13:07:03.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b8a53640 1 -- 172.21.15.18:0/1837246601 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96a4004ac0 con 0x7f96b4068380 2024-02-17T13:07:03.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b8a53640 1 -- 172.21.15.18:0/1837246601 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a403e450 con 0x7f96b4068380 2024-02-17T13:07:03.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.122+0000 7f96b9a55640 1 -- 172.21.15.18:0/1837246601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b41a6650 con 0x7f96b4068380 2024-02-17T13:07:03.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.123+0000 7f96b9a55640 1 -- 172.21.15.18:0/1837246601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b4100640 con 0x7f96b4068380 2024-02-17T13:07:03.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.124+0000 7f96b8a53640 1 -- 172.21.15.18:0/1837246601 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f96a40035f0 con 0x7f96b4068380 2024-02-17T13:07:03.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.125+0000 7f96b8a53640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96a00a2230 0x7f96a00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:03.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.125+0000 7f96b27fc640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96a00a2230 0x7f96a00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:03.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.125+0000 7f96b8a53640 1 -- 172.21.15.18:0/1837246601 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f96a40f0a70 con 0x7f96b4068380 2024-02-17T13:07:03.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.125+0000 7f96b27fc640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96a00a2230 0x7f96a00a46f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f96b419af60 tx=0x7f969c00a5e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:03.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.130+0000 7f96b8a53640 1 -- 172.21.15.18:0/1837246601 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96a40b97d0 con 0x7f96b4068380 2024-02-17T13:07:03.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:03 smithi018 ceph-mon[32625]: from='client.20835 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T13:07:03.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:03 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3920365880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T13:07:03.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:03 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1615331585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T13:07:03.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:03 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3459163503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T13:07:03.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.375+0000 7f96b9a55640 1 -- 172.21.15.18:0/1837246601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96b41134b0 con 0x7f96b4068380 2024-02-17T13:07:03.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.376+0000 7f96b8a53640 1 -- 172.21.15.18:0/1837246601 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+11 (secure 0 0 0) 0x7f96a40bce90 con 0x7f96b4068380 2024-02-17T13:07:03.365 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-02-17T13:07:03.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.378+0000 7f96b9a55640 1 -- 172.21.15.18:0/1837246601 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96a00a2230 msgr2=0x7f96a00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:03.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.378+0000 7f96b9a55640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96a00a2230 0x7f96a00a46f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f96b419af60 tx=0x7f969c00a5e0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.378+0000 7f96b9a55640 1 -- 172.21.15.18:0/1837246601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96b4068380 msgr2=0x7f96b419f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:03.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.378+0000 7f96b9a55640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96b4068380 0x7f96b419f650 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f96a4005ec0 tx=0x7f96a40042f0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.379+0000 7f96b9a55640 1 -- 172.21.15.18:0/1837246601 shutdown_connections 2024-02-17T13:07:03.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.379+0000 7f96b9a55640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f96a00a2230 0x7f96a00a46f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.379+0000 7f96b9a55640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b410db20 0x7f96b4199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.379+0000 7f96b9a55640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b4068ca0 0x7f96b419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.379+0000 7f96b9a55640 1 --2- 172.21.15.18:0/1837246601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96b4068380 0x7f96b419f650 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.379+0000 7f96b9a55640 1 -- 172.21.15.18:0/1837246601 >> 172.21.15.18:0/1837246601 conn(0x7f96b4077180 msgr2=0x7f96b410ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:03.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.379+0000 7f96b9a55640 1 -- 172.21.15.18:0/1837246601 shutdown_connections 2024-02-17T13:07:03.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.381+0000 7f96b9a55640 1 -- 172.21.15.18:0/1837246601 wait complete. 2024-02-17T13:07:03.388 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-02-17T13:07:03.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:03 smithi146 ceph-mon[36454]: from='client.20835 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T13:07:03.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:03 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3920365880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T13:07:03.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:03 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1615331585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T13:07:03.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:03 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3459163503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T13:07:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.517+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/758900733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e84106450 msgr2=0x7f2e8410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.517+0000 7f2e8c3e2640 1 --2- 172.21.15.18:0/758900733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e84106450 0x7f2e8410e020 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009930 tx=0x7f2e7c02bc30 comp rx=0 tx=0).stop 2024-02-17T13:07:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/758900733 shutdown_connections 2024-02-17T13:07:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 --2- 172.21.15.18:0/758900733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e84106450 0x7f2e8410e020 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 --2- 172.21.15.18:0/758900733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e84101d30 0x7f2e84105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 --2- 172.21.15.18:0/758900733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e84101380 0x7f2e84101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/758900733 >> 172.21.15.18:0/758900733 conn(0x7f2e840fcc20 msgr2=0x7f2e840ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/758900733 shutdown_connections 2024-02-17T13:07:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/758900733 wait complete. 2024-02-17T13:07:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 Processor -- start 2024-02-17T13:07:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 -- start start 2024-02-17T13:07:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e84101380 0x7f2e84073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e84106450 0x7f2e840739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e84073f00 0x7f2e841a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.518+0000 7f2e8c3e2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e841a5f80 con 0x7f2e84073f00 2024-02-17T13:07:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8c3e2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e841a60f0 con 0x7f2e84106450 2024-02-17T13:07:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8c3e2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e841a62b0 con 0x7f2e84101380 2024-02-17T13:07:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8a97f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e84073f00 0x7f2e841a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8a17e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e84101380 0x7f2e84073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8997d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e84106450 0x7f2e840739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8a97f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e84073f00 0x7f2e841a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46242/0 (socket says 172.21.15.18:46242) 2024-02-17T13:07:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8a17e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e84101380 0x7f2e84073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.18:52902/0 (socket says 172.21.15.18:52902) 2024-02-17T13:07:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8a97f640 1 -- 172.21.15.18:0/2350389238 learned_addr learned my addr 172.21.15.18:0/2350389238 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8997d640 1 -- 172.21.15.18:0/2350389238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e84101380 msgr2=0x7f2e84073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8997d640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e84101380 0x7f2e84073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8997d640 1 -- 172.21.15.18:0/2350389238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e84073f00 msgr2=0x7f2e841a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8997d640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e84073f00 0x7f2e841a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8997d640 1 -- 172.21.15.18:0/2350389238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c009560 con 0x7f2e84106450 2024-02-17T13:07:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8a17e640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e84101380 0x7f2e84073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:07:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8a97f640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e84073f00 0x7f2e841a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:07:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8997d640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e84106450 0x7f2e840739c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e9c0 tx=0x7f2e7800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e837fe640 1 -- 172.21.15.18:0/2350389238 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7800ccc0 con 0x7f2e84106450 2024-02-17T13:07:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/2350389238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e841a6590 con 0x7f2e84106450 2024-02-17T13:07:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/2350389238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e841a6aa0 con 0x7f2e84106450 2024-02-17T13:07:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e837fe640 1 -- 172.21.15.18:0/2350389238 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e78004500 con 0x7f2e84106450 2024-02-17T13:07:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.519+0000 7f2e837fe640 1 -- 172.21.15.18:0/2350389238 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78004a00 con 0x7f2e84106450 2024-02-17T13:07:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.520+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/2350389238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e840683c0 con 0x7f2e84106450 2024-02-17T13:07:03.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.522+0000 7f2e837fe640 1 -- 172.21.15.18:0/2350389238 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2e78005020 con 0x7f2e84106450 2024-02-17T13:07:03.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.522+0000 7f2e837fe640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2e6c0a2230 0x7f2e6c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:03.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.522+0000 7f2e8a17e640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2e6c0a2230 0x7f2e6c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:03.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.522+0000 7f2e837fe640 1 -- 172.21.15.18:0/2350389238 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2e78014070 con 0x7f2e84106450 2024-02-17T13:07:03.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.523+0000 7f2e8a17e640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2e6c0a2230 0x7f2e6c0a46f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2e74005fd0 tx=0x7f2e74009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:03.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.526+0000 7f2e837fe640 1 -- 172.21.15.18:0/2350389238 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e78096fe0 con 0x7f2e84106450 2024-02-17T13:07:03.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:03 smithi119 ceph-mon[36188]: from='client.20835 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T13:07:03.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:03 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3920365880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T13:07:03.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:03 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1615331585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T13:07:03.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:03 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3459163503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T13:07:03.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.759+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/2350389238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f2e841096e0 con 0x7f2e84106450 2024-02-17T13:07:03.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.768+0000 7f2e837fe640 1 -- 172.21.15.18:0/2350389238 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+736943 (secure 0 0 0) 0x7f2e7809a6a0 con 0x7f2e84106450 2024-02-17T13:07:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:07:03.766 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:07:03.767 INFO:teuthology.orchestra.run.smithi018.stdout: "epoch": 26, 2024-02-17T13:07:03.767 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.767 INFO:teuthology.orchestra.run.smithi018.stdout: "active_gid": 15528, 2024-02-17T13:07:03.767 INFO:teuthology.orchestra.run.smithi018.stdout: "active_name": "x", 2024-02-17T13:07:03.767 INFO:teuthology.orchestra.run.smithi018.stdout: "active_addrs": { 2024-02-17T13:07:03.767 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-02-17T13:07:03.767 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:03.767 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-02-17T13:07:03.768 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6800", 2024-02-17T13:07:03.768 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2186095810 2024-02-17T13:07:03.768 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.768 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:03.768 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v1", 2024-02-17T13:07:03.768 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6801", 2024-02-17T13:07:03.768 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2186095810 2024-02-17T13:07:03.768 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-02-17T13:07:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "active_addr": "172.21.15.18:6801/2186095810", 2024-02-17T13:07:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "active_change": "2024-02-17T12:49:52.647833+0000", 2024-02-17T13:07:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "active_mgr_features": 4540701547738038271, 2024-02-17T13:07:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "available": true, 2024-02-17T13:07:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "standbys": [ 2024-02-17T13:07:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "gid": 25028, 2024-02-17T13:07:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "y", 2024-02-17T13:07:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_features": 4540701547738038271, 2024-02-17T13:07:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "available_modules": [ 2024-02-17T13:07:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "alerts", 2024-02-17T13:07:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How frequently to reexamine health status", 2024-02-17T13:07:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_destination": { 2024-02-17T13:07:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_destination", 2024-02-17T13:07:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Email address to send alerts to", 2024-02-17T13:07:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_from_name": { 2024-02-17T13:07:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_from_name", 2024-02-17T13:07:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "Ceph", 2024-02-17T13:07:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Email From: name", 2024-02-17T13:07:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_host": { 2024-02-17T13:07:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_host", 2024-02-17T13:07:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP server", 2024-02-17T13:07:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_password": { 2024-02-17T13:07:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_password", 2024-02-17T13:07:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Password to authenticate with", 2024-02-17T13:07:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_port": { 2024-02-17T13:07:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_port", 2024-02-17T13:07:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "465", 2024-02-17T13:07:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP port", 2024-02-17T13:07:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_sender": { 2024-02-17T13:07:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_sender", 2024-02-17T13:07:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP envelope sender", 2024-02-17T13:07:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_ssl": { 2024-02-17T13:07:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_ssl", 2024-02-17T13:07:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-17T13:07:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_user": { 2024-02-17T13:07:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_user", 2024-02-17T13:07:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "User to authenticate as", 2024-02-17T13:07:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "balancer", 2024-02-17T13:07:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "active": { 2024-02-17T13:07:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "active", 2024-02-17T13:07:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "automatically balance PGs across cluster", 2024-02-17T13:07:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "begin_time": { 2024-02-17T13:07:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "begin_time", 2024-02-17T13:07:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0000", 2024-02-17T13:07:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "beginning time of day to automatically balance", 2024-02-17T13:07:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-17T13:07:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "begin_weekday": { 2024-02-17T13:07:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "begin_weekday", 2024-02-17T13:07:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0", 2024-02-17T13:07:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "6", 2024-02-17T13:07:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-17T13:07:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-17T13:07:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_max_iterations": { 2024-02-17T13:07:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_max_iterations", 2024-02-17T13:07:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "25", 2024-02-17T13:07:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-02-17T13:07:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "250", 2024-02-17T13:07:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-17T13:07:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_metrics": { 2024-02-17T13:07:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_metrics", 2024-02-17T13:07:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "pgs,objects,bytes", 2024-02-17T13:07:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-17T13:07:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-17T13:07:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_step": { 2024-02-17T13:07:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_step", 2024-02-17T13:07:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0.5", 2024-02-17T13:07:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0.001", 2024-02-17T13:07:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "0.999", 2024-02-17T13:07:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "aggressiveness of optimization", 2024-02-17T13:07:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-17T13:07:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "end_time": { 2024-02-17T13:07:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "end_time", 2024-02-17T13:07:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "2359", 2024-02-17T13:07:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "ending time of day to automatically balance", 2024-02-17T13:07:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-17T13:07:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "end_weekday": { 2024-02-17T13:07:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "end_weekday", 2024-02-17T13:07:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0", 2024-02-17T13:07:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "6", 2024-02-17T13:07:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-17T13:07:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-17T13:07:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.814 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.814 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.814 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.814 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:03.814 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:03.814 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.814 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.814 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "min_score": { 2024-02-17T13:07:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "min_score", 2024-02-17T13:07:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-17T13:07:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": { 2024-02-17T13:07:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mode", 2024-02-17T13:07:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "upmap", 2024-02-17T13:07:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "crush-compat", 2024-02-17T13:07:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "none", 2024-02-17T13:07:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "read", 2024-02-17T13:07:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap", 2024-02-17T13:07:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap-read" 2024-02-17T13:07:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Balancer mode", 2024-02-17T13:07:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_ids": { 2024-02-17T13:07:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pool_ids", 2024-02-17T13:07:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-17T13:07:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-17T13:07:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap_max_deviation": { 2024-02-17T13:07:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "upmap_max_deviation", 2024-02-17T13:07:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-02-17T13:07:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-02-17T13:07:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-17T13:07:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-17T13:07:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap_max_optimizations": { 2024-02-17T13:07:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "upmap_max_optimizations", 2024-02-17T13:07:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-17T13:07:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cephadm", 2024-02-17T13:07:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_down_multiplier": { 2024-02-17T13:07:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_down_multiplier", 2024-02-17T13:07:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3.0", 2024-02-17T13:07:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-17T13:07:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_refresh_rate": { 2024-02-17T13:07:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_refresh_rate", 2024-02-17T13:07:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "20", 2024-02-17T13:07:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-17T13:07:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_starting_port": { 2024-02-17T13:07:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_starting_port", 2024-02-17T13:07:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "4721", 2024-02-17T13:07:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-17T13:07:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_ptrace": { 2024-02-17T13:07:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_ptrace", 2024-02-17T13:07:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-17T13:07:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-17T13:07:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "autotune_interval": { 2024-02-17T13:07:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "autotune_interval", 2024-02-17T13:07:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-17T13:07:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "autotune_memory_target_ratio": { 2024-02-17T13:07:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "autotune_memory_target_ratio", 2024-02-17T13:07:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0.7", 2024-02-17T13:07:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-17T13:07:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm_log_destination": { 2024-02-17T13:07:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cephadm_log_destination", 2024-02-17T13:07:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "file", 2024-02-17T13:07:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "file,syslog", 2024-02-17T13:07:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "syslog" 2024-02-17T13:07:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-17T13:07:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "cgroups_split": { 2024-02-17T13:07:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cgroups_split", 2024-02-17T13:07:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-17T13:07:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "config_checks_enabled": { 2024-02-17T13:07:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_checks_enabled", 2024-02-17T13:07:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-17T13:07:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "config_dashboard": { 2024-02-17T13:07:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_dashboard", 2024-02-17T13:07:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-17T13:07:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.849 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.849 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_alertmanager": { 2024-02-17T13:07:03.849 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_alertmanager", 2024-02-17T13:07:03.849 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.849 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.849 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.849 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-17T13:07:03.849 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.850 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.850 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.850 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-02-17T13:07:03.850 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.850 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.850 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.850 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.850 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_base": { 2024-02-17T13:07:03.851 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_base", 2024-02-17T13:07:03.851 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.851 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.851 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.851 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-17T13:07:03.851 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.851 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.851 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.851 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Container image name, without the tag", 2024-02-17T13:07:03.852 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.852 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.852 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.852 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.852 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_elasticsearch": { 2024-02-17T13:07:03.852 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_elasticsearch", 2024-02-17T13:07:03.852 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.852 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.853 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.853 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-17T13:07:03.853 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.853 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.853 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.853 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "elasticsearch container image", 2024-02-17T13:07:03.853 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.853 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.853 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.854 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.854 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_grafana": { 2024-02-17T13:07:03.854 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_grafana", 2024-02-17T13:07:03.854 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.854 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.854 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.854 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-17T13:07:03.854 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.855 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.855 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.855 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-02-17T13:07:03.855 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.855 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.855 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.855 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.855 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_haproxy": { 2024-02-17T13:07:03.855 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_haproxy", 2024-02-17T13:07:03.856 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.856 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.856 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.856 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-17T13:07:03.856 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.856 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.857 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.857 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "HAproxy container image", 2024-02-17T13:07:03.857 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.857 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.857 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.857 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.857 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_agent": { 2024-02-17T13:07:03.857 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_agent", 2024-02-17T13:07:03.858 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.858 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.858 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.858 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-17T13:07:03.858 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.858 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.858 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.858 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger agent container image", 2024-02-17T13:07:03.859 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.859 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.859 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.859 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.859 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_collector": { 2024-02-17T13:07:03.859 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_collector", 2024-02-17T13:07:03.859 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.859 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.860 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.860 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-17T13:07:03.860 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.860 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.860 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.860 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger collector container image", 2024-02-17T13:07:03.860 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.861 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.861 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.861 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.861 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_query": { 2024-02-17T13:07:03.861 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_query", 2024-02-17T13:07:03.861 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.861 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.861 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.862 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-17T13:07:03.862 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.862 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.862 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.862 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger query container image", 2024-02-17T13:07:03.862 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.862 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.862 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.863 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.863 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_keepalived": { 2024-02-17T13:07:03.863 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_keepalived", 2024-02-17T13:07:03.863 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.863 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.863 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.863 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-17T13:07:03.863 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.864 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.864 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.864 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Keepalived container image", 2024-02-17T13:07:03.864 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.864 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.864 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.864 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.864 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_loki": { 2024-02-17T13:07:03.865 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_loki", 2024-02-17T13:07:03.865 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.865 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.865 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.865 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-17T13:07:03.865 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.865 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.865 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.865 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Loki container image", 2024-02-17T13:07:03.866 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.866 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.866 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.866 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.866 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_node_exporter": { 2024-02-17T13:07:03.866 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_node_exporter", 2024-02-17T13:07:03.866 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.866 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.867 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.867 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-17T13:07:03.867 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.867 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.867 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.867 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-02-17T13:07:03.867 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.867 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.867 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.868 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.868 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_nvmeof": { 2024-02-17T13:07:03.868 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_nvmeof", 2024-02-17T13:07:03.868 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.868 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.868 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.868 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-17T13:07:03.868 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.869 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.869 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.869 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Nvme-of container image", 2024-02-17T13:07:03.869 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.869 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.869 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.869 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.869 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_prometheus": { 2024-02-17T13:07:03.870 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_prometheus", 2024-02-17T13:07:03.870 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.870 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.870 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.870 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-17T13:07:03.870 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.870 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.870 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.871 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-02-17T13:07:03.871 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.871 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.871 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.871 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.871 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_promtail": { 2024-02-17T13:07:03.871 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_promtail", 2024-02-17T13:07:03.871 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.872 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.872 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.872 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-17T13:07:03.872 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.872 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.872 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.872 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Promtail container image", 2024-02-17T13:07:03.873 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.873 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.873 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.873 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.873 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_snmp_gateway": { 2024-02-17T13:07:03.873 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_snmp_gateway", 2024-02-17T13:07:03.873 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.873 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.873 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.874 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-17T13:07:03.874 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.874 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.874 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.874 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SNMP Gateway container image", 2024-02-17T13:07:03.874 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.874 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.874 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.874 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.875 INFO:teuthology.orchestra.run.smithi018.stdout: "container_init": { 2024-02-17T13:07:03.875 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_init", 2024-02-17T13:07:03.875 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.875 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.875 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.875 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.875 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.875 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.876 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.876 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Run podman/docker with `--init`", 2024-02-17T13:07:03.876 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.876 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.876 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.876 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.876 INFO:teuthology.orchestra.run.smithi018.stdout: "daemon_cache_timeout": { 2024-02-17T13:07:03.876 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "daemon_cache_timeout", 2024-02-17T13:07:03.877 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:03.877 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.877 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.877 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:03.877 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.877 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.877 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.877 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-17T13:07:03.878 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.878 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.878 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.878 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.878 INFO:teuthology.orchestra.run.smithi018.stdout: "default_cephadm_command_timeout": { 2024-02-17T13:07:03.878 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "default_cephadm_command_timeout", 2024-02-17T13:07:03.878 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:03.878 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.878 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.879 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "900", 2024-02-17T13:07:03.879 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.879 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.879 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.879 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-17T13:07:03.879 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.879 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.879 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.880 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.880 INFO:teuthology.orchestra.run.smithi018.stdout: "default_registry": { 2024-02-17T13:07:03.880 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "default_registry", 2024-02-17T13:07:03.880 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.880 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.880 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.880 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io", 2024-02-17T13:07:03.880 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.881 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.881 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.881 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-17T13:07:03.881 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.881 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.881 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.881 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.881 INFO:teuthology.orchestra.run.smithi018.stdout: "device_cache_timeout": { 2024-02-17T13:07:03.881 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_cache_timeout", 2024-02-17T13:07:03.882 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:03.882 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.882 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.882 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1800", 2024-02-17T13:07:03.882 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.882 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.882 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.882 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache device inventory", 2024-02-17T13:07:03.883 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.883 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.883 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.883 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.883 INFO:teuthology.orchestra.run.smithi018.stdout: "device_enhanced_scan": { 2024-02-17T13:07:03.883 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_enhanced_scan", 2024-02-17T13:07:03.883 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.883 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.884 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.884 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.884 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.884 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.884 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.884 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-17T13:07:03.884 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.884 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.885 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.885 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.885 INFO:teuthology.orchestra.run.smithi018.stdout: "facts_cache_timeout": { 2024-02-17T13:07:03.885 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "facts_cache_timeout", 2024-02-17T13:07:03.885 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:03.885 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.885 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.885 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:03.885 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.886 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.886 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.886 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache host facts data", 2024-02-17T13:07:03.886 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.886 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.886 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.886 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.886 INFO:teuthology.orchestra.run.smithi018.stdout: "host_check_interval": { 2024-02-17T13:07:03.887 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "host_check_interval", 2024-02-17T13:07:03.887 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:03.887 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.887 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.887 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:03.887 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.887 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.887 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.887 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to perform a host check", 2024-02-17T13:07:03.888 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.888 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.888 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.888 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.888 INFO:teuthology.orchestra.run.smithi018.stdout: "hw_monitoring": { 2024-02-17T13:07:03.888 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "hw_monitoring", 2024-02-17T13:07:03.888 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.888 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.889 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.889 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.889 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.889 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.889 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.889 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-17T13:07:03.889 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.889 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.890 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.890 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.890 INFO:teuthology.orchestra.run.smithi018.stdout: "inventory_list_all": { 2024-02-17T13:07:03.890 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "inventory_list_all", 2024-02-17T13:07:03.890 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.890 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.890 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.890 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.890 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-17T13:07:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:03.893 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:03.893 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:03.893 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:03.893 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:03.893 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:03.893 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:03.893 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:03.893 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.894 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.894 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.894 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.894 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.894 INFO:teuthology.orchestra.run.smithi018.stdout: "log_refresh_metadata": { 2024-02-17T13:07:03.895 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_refresh_metadata", 2024-02-17T13:07:03.895 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.895 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.895 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.895 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.895 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.895 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.895 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.896 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-17T13:07:03.896 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.896 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.896 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.896 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.896 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:03.896 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:03.896 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.897 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.897 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.897 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.897 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.897 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.897 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.897 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-17T13:07:03.898 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.898 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.898 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.898 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.898 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:03.898 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:03.898 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.898 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.899 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.899 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:03.899 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.899 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.899 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:03.899 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:03.899 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:03.899 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:03.900 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:03.900 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:03.900 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:03.900 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:03.900 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.900 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.900 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.901 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.901 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.901 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:03.901 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:03.901 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.901 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.901 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.901 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.901 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.902 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.902 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.902 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.902 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.902 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.902 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.902 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.902 INFO:teuthology.orchestra.run.smithi018.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-17T13:07:03.902 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-17T13:07:03.903 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.903 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.903 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.903 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.903 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.903 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.903 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.903 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-17T13:07:03.904 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.904 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.904 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.904 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.904 INFO:teuthology.orchestra.run.smithi018.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-17T13:07:03.904 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-17T13:07:03.904 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.904 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.905 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.905 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "*", 2024-02-17T13:07:03.905 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.905 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.905 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.905 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-17T13:07:03.905 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.905 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.906 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.906 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.906 INFO:teuthology.orchestra.run.smithi018.stdout: "max_count_per_host": { 2024-02-17T13:07:03.906 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_count_per_host", 2024-02-17T13:07:03.906 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:03.906 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.906 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.906 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:03.907 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.907 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.907 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.907 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "max number of daemons per service per host", 2024-02-17T13:07:03.907 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.907 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.907 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.907 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.908 INFO:teuthology.orchestra.run.smithi018.stdout: "max_osd_draining_count": { 2024-02-17T13:07:03.908 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_osd_draining_count", 2024-02-17T13:07:03.908 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:03.908 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.908 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.908 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:03.908 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.908 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.909 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.909 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-17T13:07:03.909 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.909 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.909 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.909 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.909 INFO:teuthology.orchestra.run.smithi018.stdout: "migration_current": { 2024-02-17T13:07:03.909 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "migration_current", 2024-02-17T13:07:03.910 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:03.910 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.910 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.910 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.910 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.910 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.910 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.910 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "internal - do not modify", 2024-02-17T13:07:03.911 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.911 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.911 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.911 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.911 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": { 2024-02-17T13:07:03.911 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mode", 2024-02-17T13:07:03.911 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.911 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.911 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.912 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "root", 2024-02-17T13:07:03.912 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.912 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.912 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:03.912 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm-package", 2024-02-17T13:07:03.912 INFO:teuthology.orchestra.run.smithi018.stdout: "root" 2024-02-17T13:07:03.912 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:03.913 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "mode for remote execution of cephadm", 2024-02-17T13:07:03.913 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.913 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.913 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.913 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.913 INFO:teuthology.orchestra.run.smithi018.stdout: "oob_default_addr": { 2024-02-17T13:07:03.913 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "oob_default_addr", 2024-02-17T13:07:03.913 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.913 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.914 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.914 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "169.254.1.1", 2024-02-17T13:07:03.914 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.914 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.914 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.914 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-17T13:07:03.914 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.914 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.915 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.915 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.915 INFO:teuthology.orchestra.run.smithi018.stdout: "prometheus_alerts_path": { 2024-02-17T13:07:03.915 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prometheus_alerts_path", 2024-02-17T13:07:03.915 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.915 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.915 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.915 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-17T13:07:03.915 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.916 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.916 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.916 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-17T13:07:03.916 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.916 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.916 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.916 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.916 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_insecure": { 2024-02-17T13:07:03.917 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_insecure", 2024-02-17T13:07:03.917 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.917 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.917 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.917 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.917 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.917 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.917 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.918 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-17T13:07:03.918 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.918 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.918 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.918 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.918 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_password": { 2024-02-17T13:07:03.918 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_password", 2024-02-17T13:07:03.918 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.918 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.919 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.919 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.919 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.919 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.919 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.919 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-17T13:07:03.919 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.919 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.919 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.920 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.920 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_url": { 2024-02-17T13:07:03.920 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_url", 2024-02-17T13:07:03.920 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.920 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.920 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.921 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.921 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.921 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.921 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.921 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-17T13:07:03.921 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.921 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.921 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.922 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.922 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_username": { 2024-02-17T13:07:03.922 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_username", 2024-02-17T13:07:03.922 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.922 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.922 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.922 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.922 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.923 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.923 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.923 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-17T13:07:03.923 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.923 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.923 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.923 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.923 INFO:teuthology.orchestra.run.smithi018.stdout: "secure_monitoring_stack": { 2024-02-17T13:07:03.923 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "secure_monitoring_stack", 2024-02-17T13:07:03.924 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.924 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.924 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.924 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.924 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.924 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.924 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.924 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-17T13:07:03.925 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.925 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.925 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.925 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.925 INFO:teuthology.orchestra.run.smithi018.stdout: "service_discovery_port": { 2024-02-17T13:07:03.925 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "service_discovery_port", 2024-02-17T13:07:03.925 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:03.925 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.925 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.926 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8765", 2024-02-17T13:07:03.926 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.926 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.926 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.926 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "cephadm service discovery port", 2024-02-17T13:07:03.926 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.926 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.926 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.927 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.927 INFO:teuthology.orchestra.run.smithi018.stdout: "ssh_config_file": { 2024-02-17T13:07:03.927 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssh_config_file", 2024-02-17T13:07:03.927 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.927 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.927 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.927 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.927 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.927 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.928 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.928 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-17T13:07:03.928 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.928 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.928 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.928 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.928 INFO:teuthology.orchestra.run.smithi018.stdout: "use_agent": { 2024-02-17T13:07:03.928 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "use_agent", 2024-02-17T13:07:03.929 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.929 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.929 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.929 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.929 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.929 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.929 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.929 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-17T13:07:03.929 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.930 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.930 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.930 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.930 INFO:teuthology.orchestra.run.smithi018.stdout: "use_repo_digest": { 2024-02-17T13:07:03.930 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "use_repo_digest", 2024-02-17T13:07:03.930 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.930 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.930 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.931 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.931 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.931 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.931 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.931 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-17T13:07:03.931 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.931 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.931 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.932 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.932 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_failed_host_check": { 2024-02-17T13:07:03.932 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_failed_host_check", 2024-02-17T13:07:03.932 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.932 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.932 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.932 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.932 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.932 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.933 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.933 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if the host check fails", 2024-02-17T13:07:03.933 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.933 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.933 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.933 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.933 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_stray_daemons": { 2024-02-17T13:07:03.933 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_stray_daemons", 2024-02-17T13:07:03.933 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.934 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.934 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.934 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.934 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.934 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.934 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.934 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-17T13:07:03.934 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.935 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.935 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.935 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.935 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_stray_hosts": { 2024-02-17T13:07:03.935 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_stray_hosts", 2024-02-17T13:07:03.935 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.935 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.935 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.935 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.936 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.936 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.936 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.936 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-17T13:07:03.936 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.936 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.936 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.937 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:03.937 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:03.937 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.937 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:03.937 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crash", 2024-02-17T13:07:03.937 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:03.937 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:03.937 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:03.937 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:03.938 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:03.938 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.938 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.938 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.938 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.938 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.938 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.938 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:03.938 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:03.939 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:03.939 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:03.939 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:03.939 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:03.939 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:03.939 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:03.939 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.939 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.940 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.940 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.940 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:03.940 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:03.940 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.940 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.940 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.941 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.941 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.941 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.941 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.941 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.941 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.941 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.941 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.942 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.942 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:03.942 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:03.942 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.942 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.942 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.942 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:03.942 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.943 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.943 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:03.943 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:03.943 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:03.943 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:03.943 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:03.943 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:03.943 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:03.944 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:03.944 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.944 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.944 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.944 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.944 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.944 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:03.944 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:03.945 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.945 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.945 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.945 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.945 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.945 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.945 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.945 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.946 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.946 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.946 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.946 INFO:teuthology.orchestra.run.smithi018.stdout: "retain_interval": { 2024-02-17T13:07:03.946 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "retain_interval", 2024-02-17T13:07:03.946 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:03.946 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.946 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.947 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "31536000", 2024-02-17T13:07:03.947 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.947 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.947 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.947 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-17T13:07:03.947 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.947 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.947 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.948 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.948 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_recent_interval": { 2024-02-17T13:07:03.948 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_recent_interval", 2024-02-17T13:07:03.948 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:03.948 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.948 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:03.948 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1209600", 2024-02-17T13:07:03.948 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.948 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.949 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.949 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-17T13:07:03.949 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.949 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.949 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.949 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:03.949 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:03.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.950 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:03.950 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "dashboard", 2024-02-17T13:07:03.950 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:03.950 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:03.950 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:03.950 INFO:teuthology.orchestra.run.smithi018.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-17T13:07:03.950 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-17T13:07:03.950 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:03.951 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.951 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.951 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:03.951 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.951 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.951 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.951 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.951 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.952 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.952 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.952 INFO:teuthology.orchestra.run.smithi018.stdout: "ALERTMANAGER_API_HOST": { 2024-02-17T13:07:03.952 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-17T13:07:03.952 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.952 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.953 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.953 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.953 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.953 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.953 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.953 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.953 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.953 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.954 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.954 INFO:teuthology.orchestra.run.smithi018.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-17T13:07:03.954 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-17T13:07:03.954 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.954 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.954 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.954 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.955 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.955 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.955 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.955 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.955 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.955 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.955 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.955 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.956 INFO:teuthology.orchestra.run.smithi018.stdout: "AUDIT_API_ENABLED": { 2024-02-17T13:07:03.956 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "AUDIT_API_ENABLED", 2024-02-17T13:07:03.956 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.956 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.956 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.956 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.956 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.956 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.957 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.957 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.957 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.957 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.957 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.957 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.957 INFO:teuthology.orchestra.run.smithi018.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-17T13:07:03.957 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-17T13:07:03.958 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.958 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.958 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.958 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.958 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.958 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.958 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.958 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.958 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.959 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.959 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.959 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.959 INFO:teuthology.orchestra.run.smithi018.stdout: "ENABLE_BROWSABLE_API": { 2024-02-17T13:07:03.959 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-17T13:07:03.959 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.959 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.959 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.959 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.960 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.960 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.960 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.960 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.960 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.960 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.960 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.960 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.961 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-17T13:07:03.961 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-17T13:07:03.961 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.961 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.961 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.961 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.961 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.961 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.962 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.962 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.962 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.962 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.962 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.962 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.962 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-17T13:07:03.962 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-17T13:07:03.962 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.963 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.963 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.963 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.963 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.963 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.963 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.963 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.963 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.964 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.964 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.964 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.964 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-17T13:07:03.964 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-17T13:07:03.964 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.964 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.965 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.965 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.965 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.965 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.965 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.965 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.965 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.965 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.965 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.966 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.966 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-17T13:07:03.966 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-17T13:07:03.966 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.966 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.966 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.966 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.966 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.967 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.967 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.967 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.967 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.967 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.967 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.967 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.967 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-17T13:07:03.968 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-17T13:07:03.968 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.968 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.968 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.968 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.968 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.968 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.968 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.969 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.969 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.969 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.969 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.969 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.969 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-17T13:07:03.969 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-17T13:07:03.969 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.970 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.970 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.970 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.970 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.970 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.970 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.970 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.970 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.971 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.971 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.971 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.971 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-17T13:07:03.971 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-17T13:07:03.971 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.971 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.971 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.971 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.972 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.972 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.972 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.972 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.972 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.972 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.972 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.972 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.973 INFO:teuthology.orchestra.run.smithi018.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-17T13:07:03.973 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-17T13:07:03.973 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.973 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.973 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.973 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.973 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.973 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.974 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.974 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.974 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.974 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.974 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.974 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_PASSWORD": { 2024-02-17T13:07:03.974 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-17T13:07:03.974 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.975 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.975 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.975 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-02-17T13:07:03.975 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.975 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.975 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.975 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.975 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.976 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.976 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.976 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.976 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-17T13:07:03.976 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-17T13:07:03.976 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.976 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.976 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.977 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.977 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.977 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.977 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.977 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.977 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.977 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.977 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.977 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.978 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_URL": { 2024-02-17T13:07:03.978 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_URL", 2024-02-17T13:07:03.978 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.978 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.978 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.978 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.978 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.978 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.979 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.979 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.979 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.979 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.979 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.979 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.979 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_USERNAME": { 2024-02-17T13:07:03.979 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-17T13:07:03.979 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.980 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.980 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.980 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-02-17T13:07:03.980 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.980 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.980 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.980 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.980 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.981 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.981 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.981 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.981 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-17T13:07:03.981 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-17T13:07:03.981 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.981 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.981 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.982 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.982 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.982 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.982 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.982 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.982 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.982 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.982 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.982 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.983 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-17T13:07:03.983 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-17T13:07:03.983 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.983 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.983 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.983 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.983 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.983 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.984 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.984 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.984 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.984 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.984 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.984 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.984 INFO:teuthology.orchestra.run.smithi018.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-17T13:07:03.984 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-17T13:07:03.985 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.985 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.985 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.985 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.985 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.985 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.985 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.985 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.985 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.986 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.986 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.986 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.986 INFO:teuthology.orchestra.run.smithi018.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-17T13:07:03.986 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-17T13:07:03.986 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.986 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.987 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.987 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.987 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.987 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.987 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.987 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.987 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.987 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.987 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.988 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.988 INFO:teuthology.orchestra.run.smithi018.stdout: "PROMETHEUS_API_HOST": { 2024-02-17T13:07:03.988 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-17T13:07:03.988 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:03.988 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.988 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.988 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:03.988 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.989 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.989 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.989 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.989 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.989 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.989 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.989 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.989 INFO:teuthology.orchestra.run.smithi018.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-17T13:07:03.990 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-17T13:07:03.990 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.990 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.990 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.990 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.990 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.990 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.990 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.990 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.991 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.991 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.991 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.991 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.991 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-17T13:07:03.991 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-17T13:07:03.991 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.991 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.992 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.992 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.992 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.992 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.992 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.992 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.992 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.992 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.992 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.993 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-17T13:07:03.993 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-17T13:07:03.993 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.993 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.993 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.993 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.993 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.994 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.994 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.994 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.994 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.994 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.994 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.994 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.994 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-17T13:07:03.995 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-17T13:07:03.995 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.995 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.995 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.995 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.995 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.995 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.995 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.995 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.996 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.996 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.996 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.996 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.996 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-17T13:07:03.996 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-17T13:07:03.996 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.997 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.997 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.997 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:03.997 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.997 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.997 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.997 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.997 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.997 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.998 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.998 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:03.998 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-17T13:07:03.998 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-17T13:07:03.998 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:03.998 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:03.998 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:03.998 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:03.999 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:03.999 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:03.999 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:03.999 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:03.999 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:03.999 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:03.999 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:03.999 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.000 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-17T13:07:04.000 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-17T13:07:04.000 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.000 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.000 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.000 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.000 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.001 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.001 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.001 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.001 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.001 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.001 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.001 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.001 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-17T13:07:04.002 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-17T13:07:04.002 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.002 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.002 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.002 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.002 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.002 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.003 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.003 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.003 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.003 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.003 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.003 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.003 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_ENABLED": { 2024-02-17T13:07:04.003 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-17T13:07:04.004 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.004 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.004 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.004 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.004 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.004 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.004 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.004 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.005 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.005 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.005 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.005 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.005 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-17T13:07:04.005 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-17T13:07:04.005 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.005 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.006 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.006 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-17T13:07:04.006 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.006 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.006 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.006 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.006 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.006 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.007 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.007 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.007 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-17T13:07:04.007 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-17T13:07:04.007 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.007 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.007 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.007 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:04.008 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.008 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.008 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.008 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.008 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.008 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.008 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.008 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.009 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-17T13:07:04.009 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-17T13:07:04.009 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.009 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.009 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.009 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8", 2024-02-17T13:07:04.009 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.009 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.010 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.010 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.010 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.010 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.010 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.010 INFO:teuthology.orchestra.run.smithi018.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-17T13:07:04.010 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-17T13:07:04.011 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.011 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.011 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.011 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "45", 2024-02-17T13:07:04.011 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.011 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.011 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.011 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.012 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.012 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.012 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.012 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_ACCESS_KEY": { 2024-02-17T13:07:04.012 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-17T13:07:04.012 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.012 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.013 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.013 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.013 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.013 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.013 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.013 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.013 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.013 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.014 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.014 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-17T13:07:04.014 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-17T13:07:04.014 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.014 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.014 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.014 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-02-17T13:07:04.014 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.015 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.015 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.015 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.015 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.015 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.015 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.015 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.015 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_SECRET_KEY": { 2024-02-17T13:07:04.015 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-17T13:07:04.016 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.016 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.016 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.016 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.016 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.016 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.016 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.016 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.017 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.017 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.017 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.017 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.017 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_SSL_VERIFY": { 2024-02-17T13:07:04.017 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-17T13:07:04.017 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.017 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.018 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.018 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.018 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.018 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.018 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.018 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.018 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.018 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.018 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.019 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.019 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-17T13:07:04.019 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-17T13:07:04.019 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.019 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.019 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.019 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:04.020 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.020 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.020 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.020 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.020 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.020 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.020 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.021 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.021 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-17T13:07:04.021 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-17T13:07:04.021 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.021 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.021 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.021 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:04.021 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.022 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.022 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.022 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.022 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.022 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.022 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.022 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.023 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-17T13:07:04.023 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-17T13:07:04.023 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.023 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.023 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.023 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-02-17T13:07:04.023 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.023 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.024 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.024 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.024 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.024 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.024 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.024 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.024 INFO:teuthology.orchestra.run.smithi018.stdout: "cross_origin_url": { 2024-02-17T13:07:04.024 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cross_origin_url", 2024-02-17T13:07:04.025 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.025 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.025 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.025 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.025 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.025 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.025 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.025 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.026 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.026 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.026 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.026 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.026 INFO:teuthology.orchestra.run.smithi018.stdout: "crt_file": { 2024-02-17T13:07:04.026 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crt_file", 2024-02-17T13:07:04.026 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.026 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.027 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.027 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.027 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.027 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.027 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.027 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.027 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.027 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.028 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.028 INFO:teuthology.orchestra.run.smithi018.stdout: "debug": { 2024-02-17T13:07:04.028 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "debug", 2024-02-17T13:07:04.028 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.028 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.028 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.028 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.029 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.029 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.029 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.029 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable/disable debug options", 2024-02-17T13:07:04.029 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.029 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.029 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.029 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.029 INFO:teuthology.orchestra.run.smithi018.stdout: "jwt_token_ttl": { 2024-02-17T13:07:04.030 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "jwt_token_ttl", 2024-02-17T13:07:04.030 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.030 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.030 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.030 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "28800", 2024-02-17T13:07:04.030 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.030 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.030 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.031 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.031 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.031 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.031 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.031 INFO:teuthology.orchestra.run.smithi018.stdout: "key_file": { 2024-02-17T13:07:04.031 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "key_file", 2024-02-17T13:07:04.031 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.032 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.032 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.032 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.032 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.032 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.032 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.032 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.033 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.033 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.033 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.033 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.033 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.033 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.033 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.033 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.033 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.034 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.034 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.034 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.034 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.034 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.034 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.034 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.034 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.035 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.035 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.035 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.035 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.035 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.035 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.035 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.035 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.036 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.036 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.036 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.036 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.036 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.036 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.036 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.036 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.037 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.037 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.037 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.037 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.037 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.037 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.037 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.038 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.038 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.038 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.038 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.038 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.038 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.038 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.038 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.038 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.039 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.039 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.039 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.039 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.039 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.039 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.039 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.039 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.040 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.040 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.040 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.040 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.040 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.040 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.040 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.040 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.041 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.041 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.041 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.041 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.041 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.042 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.043 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.043 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.043 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.043 INFO:teuthology.orchestra.run.smithi018.stdout: "motd": { 2024-02-17T13:07:04.043 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "motd", 2024-02-17T13:07:04.043 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.043 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.043 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.044 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.044 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.044 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.044 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.044 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "The message of the day", 2024-02-17T13:07:04.044 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.044 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.045 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.045 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.045 INFO:teuthology.orchestra.run.smithi018.stdout: "redirect_resolve_ip_addr": { 2024-02-17T13:07:04.045 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "redirect_resolve_ip_addr", 2024-02-17T13:07:04.045 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.045 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.045 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.045 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.045 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.046 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.046 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.046 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.046 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.046 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.046 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.046 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.046 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-02-17T13:07:04.047 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-02-17T13:07:04.047 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.047 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.047 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.047 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "::", 2024-02-17T13:07:04.047 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.047 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.048 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.048 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.048 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.048 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.048 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.048 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.048 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-02-17T13:07:04.048 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-02-17T13:07:04.049 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.049 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.049 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.049 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8080", 2024-02-17T13:07:04.049 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.049 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.049 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.050 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.050 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.050 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.050 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.050 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.050 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl": { 2024-02-17T13:07:04.050 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl", 2024-02-17T13:07:04.050 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.051 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.051 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.051 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.051 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.051 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.051 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.051 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.051 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.052 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.052 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.052 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.052 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl_server_port": { 2024-02-17T13:07:04.052 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl_server_port", 2024-02-17T13:07:04.052 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.052 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.052 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.053 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8443", 2024-02-17T13:07:04.053 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.053 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.053 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.053 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.053 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.053 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.053 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.054 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.054 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_behaviour": { 2024-02-17T13:07:04.054 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_behaviour", 2024-02-17T13:07:04.054 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.054 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.054 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.054 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "redirect", 2024-02-17T13:07:04.054 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.054 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.055 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.055 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.055 INFO:teuthology.orchestra.run.smithi018.stdout: "redirect" 2024-02-17T13:07:04.055 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.055 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.055 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.055 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.055 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.055 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.056 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_error_status_code": { 2024-02-17T13:07:04.056 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_error_status_code", 2024-02-17T13:07:04.056 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.056 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.056 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.056 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "500", 2024-02-17T13:07:04.056 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "400", 2024-02-17T13:07:04.056 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "599", 2024-02-17T13:07:04.057 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.057 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.057 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.057 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.057 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.057 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.057 INFO:teuthology.orchestra.run.smithi018.stdout: "url_prefix": { 2024-02-17T13:07:04.057 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "url_prefix", 2024-02-17T13:07:04.058 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.058 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.058 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.058 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.058 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.058 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.058 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.059 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.059 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.059 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.059 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.059 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.059 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.059 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.059 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.059 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "devicehealth", 2024-02-17T13:07:04.060 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.060 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.060 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.060 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_monitoring": { 2024-02-17T13:07:04.060 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enable_monitoring", 2024-02-17T13:07:04.060 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.060 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.061 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.061 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.061 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.061 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.061 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.061 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "monitor device health metrics", 2024-02-17T13:07:04.061 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.061 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.062 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.062 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.062 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.062 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.062 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.062 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.062 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.062 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.062 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.063 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.063 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.063 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.063 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.063 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.063 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.063 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.064 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.064 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.064 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.064 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.064 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.064 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.065 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.065 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.065 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.065 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.065 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.065 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.065 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.065 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.065 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.066 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.066 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.066 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.066 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.066 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.066 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.066 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.066 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.067 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.067 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.067 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.067 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.067 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.067 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.067 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.067 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.068 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.068 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.068 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.068 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.068 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.068 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.068 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.068 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.069 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.069 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.069 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.069 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.069 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.069 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.069 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.069 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.070 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.070 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.070 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.070 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.070 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.070 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.070 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.070 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.070 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.071 INFO:teuthology.orchestra.run.smithi018.stdout: "mark_out_threshold": { 2024-02-17T13:07:04.071 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mark_out_threshold", 2024-02-17T13:07:04.071 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.071 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.071 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.071 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "2419200", 2024-02-17T13:07:04.071 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.071 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.072 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.072 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-17T13:07:04.072 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.072 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.072 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.072 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.072 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": { 2024-02-17T13:07:04.072 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pool_name", 2024-02-17T13:07:04.073 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.073 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.073 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.073 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "device_health_metrics", 2024-02-17T13:07:04.073 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.073 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.073 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.073 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-17T13:07:04.074 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.074 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.074 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.074 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.074 INFO:teuthology.orchestra.run.smithi018.stdout: "retention_period": { 2024-02-17T13:07:04.074 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "retention_period", 2024-02-17T13:07:04.074 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.074 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.074 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.075 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15552000", 2024-02-17T13:07:04.075 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.075 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.075 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.075 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long to retain device health metrics", 2024-02-17T13:07:04.075 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.075 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.075 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.075 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.076 INFO:teuthology.orchestra.run.smithi018.stdout: "scrape_frequency": { 2024-02-17T13:07:04.076 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "scrape_frequency", 2024-02-17T13:07:04.076 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.076 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.076 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.076 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "86400", 2024-02-17T13:07:04.076 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.077 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.077 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.077 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-17T13:07:04.077 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.077 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.077 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.077 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.077 INFO:teuthology.orchestra.run.smithi018.stdout: "self_heal": { 2024-02-17T13:07:04.077 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "self_heal", 2024-02-17T13:07:04.078 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.078 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.078 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.078 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.078 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.078 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.078 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.078 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-17T13:07:04.079 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.079 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.079 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.079 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.079 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:04.079 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:04.079 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.079 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.080 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.080 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:04.080 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.080 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.080 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.080 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to wake up and check device health", 2024-02-17T13:07:04.080 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.081 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.081 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.081 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.081 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_threshold": { 2024-02-17T13:07:04.081 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_threshold", 2024-02-17T13:07:04.081 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.081 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.081 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.082 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "7257600", 2024-02-17T13:07:04.082 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.082 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.082 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.082 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-17T13:07:04.082 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.082 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.082 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.082 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.083 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.083 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.083 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.083 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "diskprediction_local", 2024-02-17T13:07:04.083 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.083 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.083 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.083 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.084 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.084 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.084 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.084 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.084 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.084 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.084 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.084 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.085 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.085 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.085 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.085 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.085 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.085 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.085 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.085 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.085 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.086 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.086 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.086 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.086 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.086 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.086 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.086 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.086 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.087 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.087 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.087 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.087 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.087 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.087 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.087 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.087 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.088 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.088 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.088 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.088 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.088 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.088 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.088 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.088 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.089 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.089 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.089 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.089 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.089 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.089 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.089 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.089 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.090 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.090 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.090 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.090 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.090 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.090 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.090 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.090 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.091 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.091 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.091 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.091 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.091 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.091 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.091 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.091 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.091 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.092 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.092 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.092 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.092 INFO:teuthology.orchestra.run.smithi018.stdout: "predict_interval": { 2024-02-17T13:07:04.092 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "predict_interval", 2024-02-17T13:07:04.092 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.092 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.093 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.093 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "86400", 2024-02-17T13:07:04.093 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.093 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.093 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.093 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.093 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.093 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.093 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.094 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.094 INFO:teuthology.orchestra.run.smithi018.stdout: "predictor_model": { 2024-02-17T13:07:04.094 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "predictor_model", 2024-02-17T13:07:04.094 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.094 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.094 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.094 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "prophetstor", 2024-02-17T13:07:04.094 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.095 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.095 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.095 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.095 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.095 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.095 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.095 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.095 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:04.096 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:04.096 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.096 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.096 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.096 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:04.096 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.096 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.096 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.097 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.097 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.097 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.097 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.097 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.097 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.097 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.097 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.098 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "influx", 2024-02-17T13:07:04.098 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": false, 2024-02-17T13:07:04.098 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "influxdb python module not found", 2024-02-17T13:07:04.098 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.098 INFO:teuthology.orchestra.run.smithi018.stdout: "batch_size": { 2024-02-17T13:07:04.098 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "batch_size", 2024-02-17T13:07:04.098 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.099 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.099 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.099 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5000", 2024-02-17T13:07:04.099 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.099 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.099 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.099 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-17T13:07:04.099 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.099 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.100 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.100 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.100 INFO:teuthology.orchestra.run.smithi018.stdout: "database": { 2024-02-17T13:07:04.100 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "database", 2024-02-17T13:07:04.100 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.100 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.100 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.100 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "ceph", 2024-02-17T13:07:04.101 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.101 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.101 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.101 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-17T13:07:04.101 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.101 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.101 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.101 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.101 INFO:teuthology.orchestra.run.smithi018.stdout: "hostname": { 2024-02-17T13:07:04.102 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "hostname", 2024-02-17T13:07:04.102 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.102 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.102 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.102 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.102 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.102 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.102 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.103 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB server hostname", 2024-02-17T13:07:04.103 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.103 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.103 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.103 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.103 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:04.103 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:04.104 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.104 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.104 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.104 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "30", 2024-02-17T13:07:04.104 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "5", 2024-02-17T13:07:04.104 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.104 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.104 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-17T13:07:04.105 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.105 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.105 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.105 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.105 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.105 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.105 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.105 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.106 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.106 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.106 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.106 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.106 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.106 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.106 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.106 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.107 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.108 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.109 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.110 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.111 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.112 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.113 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: "password": { 2024-02-17T13:07:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "password", 2024-02-17T13:07:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.114 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "password of InfluxDB server user", 2024-02-17T13:07:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: "port": { 2024-02-17T13:07:04.115 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "port", 2024-02-17T13:07:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8086", 2024-02-17T13:07:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.116 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB server port", 2024-02-17T13:07:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl": { 2024-02-17T13:07:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl", 2024-02-17T13:07:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.117 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "false", 2024-02-17T13:07:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-17T13:07:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.118 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: "threads": { 2024-02-17T13:07:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "threads", 2024-02-17T13:07:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-02-17T13:07:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-02-17T13:07:04.119 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "32", 2024-02-17T13:07:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-17T13:07:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: "username": { 2024-02-17T13:07:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "username", 2024-02-17T13:07:04.121 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.121 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.121 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.121 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.121 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.121 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.121 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.121 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "username of InfluxDB server user", 2024-02-17T13:07:04.121 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.122 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.122 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.122 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.122 INFO:teuthology.orchestra.run.smithi018.stdout: "verify_ssl": { 2024-02-17T13:07:04.122 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "verify_ssl", 2024-02-17T13:07:04.122 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.122 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.122 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.123 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "true", 2024-02-17T13:07:04.123 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.123 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.123 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.123 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-17T13:07:04.123 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.123 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.123 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.123 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.124 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.124 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.124 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.124 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "insights", 2024-02-17T13:07:04.124 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.124 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.124 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.124 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.125 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.125 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.125 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.125 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.125 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.125 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.125 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.125 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.126 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.126 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.126 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.126 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.126 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.126 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.126 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.126 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.127 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.127 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.127 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.127 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.127 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.127 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.127 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.127 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.128 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.128 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.128 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.128 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.128 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.128 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.128 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.128 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.129 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.129 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.129 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.129 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.129 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.129 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.129 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.129 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.130 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.130 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.130 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.130 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.130 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.130 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.130 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.130 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.131 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.131 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.131 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.131 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.131 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.131 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.131 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.131 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.132 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.132 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.132 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.132 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.132 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.132 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.132 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.132 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.132 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.133 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.133 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.133 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.133 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.133 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.133 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.133 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.133 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "iostat", 2024-02-17T13:07:04.134 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.134 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.134 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.134 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.134 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.134 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.134 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.134 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.135 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.135 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.135 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.135 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.135 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.135 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.135 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.135 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.135 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.136 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.136 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.136 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.136 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.136 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.136 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.136 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.136 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.137 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.137 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.137 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.137 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.137 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.137 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.137 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.137 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.137 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.138 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.138 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.138 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.138 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.138 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.138 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.138 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.138 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.139 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.139 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.139 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.139 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.139 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.139 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.139 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.139 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.140 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.140 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.140 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.140 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.140 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.140 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.140 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.140 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.141 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.141 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.141 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.141 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.141 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.141 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.141 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.141 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.142 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.142 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.142 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.142 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.142 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.142 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.142 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.142 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.143 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.143 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.143 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "k8sevents", 2024-02-17T13:07:04.143 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.143 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.143 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.143 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_event_retention_days": { 2024-02-17T13:07:04.143 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ceph_event_retention_days", 2024-02-17T13:07:04.144 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.144 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.144 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.144 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "7", 2024-02-17T13:07:04.144 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.144 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.144 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.144 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-17T13:07:04.145 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.145 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.145 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.145 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.145 INFO:teuthology.orchestra.run.smithi018.stdout: "config_check_secs": { 2024-02-17T13:07:04.145 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_check_secs", 2024-02-17T13:07:04.145 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.145 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.146 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.146 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:04.146 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "10", 2024-02-17T13:07:04.146 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.146 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.146 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-17T13:07:04.146 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.146 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.147 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.147 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.147 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.147 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.147 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.147 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.147 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.147 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.147 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.148 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.148 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.148 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.148 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.148 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.148 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.148 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.148 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.149 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.149 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.149 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.149 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.149 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.149 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.149 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.149 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.149 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.150 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.150 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.150 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.150 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.150 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.150 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.150 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.151 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.151 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.151 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.151 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.151 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.151 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.151 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.151 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.151 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.152 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.152 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.152 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.152 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.152 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.152 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.152 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.153 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.153 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.153 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.153 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.153 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.153 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.153 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.153 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.154 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.154 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.154 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.154 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.154 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.154 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.154 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.154 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.155 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.155 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.155 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.155 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.155 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.155 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.155 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.155 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.156 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.156 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.156 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "localpool", 2024-02-17T13:07:04.156 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.156 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.156 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.156 INFO:teuthology.orchestra.run.smithi018.stdout: "failure_domain": { 2024-02-17T13:07:04.156 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "failure_domain", 2024-02-17T13:07:04.156 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.157 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.157 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.157 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "host", 2024-02-17T13:07:04.157 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.157 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.157 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.157 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "failure domain for any created local pool", 2024-02-17T13:07:04.157 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-17T13:07:04.158 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.158 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.158 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.158 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.158 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.158 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.158 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.158 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.159 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.159 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.159 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.159 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.159 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.159 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.159 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.159 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.159 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.160 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.160 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.160 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.160 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.160 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.160 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.160 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.160 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.161 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.161 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.161 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.161 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.161 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.161 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.161 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.161 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.162 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.162 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.162 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.162 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.162 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.162 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.162 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.162 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.162 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.163 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.163 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.163 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.163 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.163 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.163 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.163 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.163 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.164 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.164 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.164 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.164 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.164 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.164 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.164 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.164 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.165 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.165 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.165 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.165 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.165 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.165 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.165 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.165 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.165 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.166 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.166 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.166 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.166 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.166 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.166 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.166 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": { 2024-02-17T13:07:04.166 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "min_size", 2024-02-17T13:07:04.167 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.167 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.167 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.167 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.167 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.167 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.167 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.167 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default min_size for any created local pool", 2024-02-17T13:07:04.168 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-17T13:07:04.168 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.168 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.168 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.168 INFO:teuthology.orchestra.run.smithi018.stdout: "num_rep": { 2024-02-17T13:07:04.168 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "num_rep", 2024-02-17T13:07:04.168 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.168 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.169 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.169 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3", 2024-02-17T13:07:04.169 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.169 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.169 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.169 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default replica count for any created local pool", 2024-02-17T13:07:04.169 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.170 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.170 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.170 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.170 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": { 2024-02-17T13:07:04.170 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pg_num", 2024-02-17T13:07:04.170 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.170 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.170 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.171 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "128", 2024-02-17T13:07:04.171 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.171 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.171 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.171 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default pg_num for any created local pool", 2024-02-17T13:07:04.171 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.171 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.171 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.172 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.172 INFO:teuthology.orchestra.run.smithi018.stdout: "prefix": { 2024-02-17T13:07:04.172 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prefix", 2024-02-17T13:07:04.172 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.172 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.172 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.172 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.172 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.173 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.173 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.173 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "name prefix for any created local pool", 2024-02-17T13:07:04.173 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.173 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.173 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.173 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.173 INFO:teuthology.orchestra.run.smithi018.stdout: "subtree": { 2024-02-17T13:07:04.174 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "subtree", 2024-02-17T13:07:04.174 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.174 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.174 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.174 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "rack", 2024-02-17T13:07:04.174 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.174 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.174 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.175 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-17T13:07:04.175 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-17T13:07:04.175 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.175 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.175 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.175 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.175 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.175 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.175 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mds_autoscaler", 2024-02-17T13:07:04.176 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.176 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.176 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.176 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.176 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.176 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.176 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.176 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.177 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.177 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.177 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.177 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.177 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.177 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.177 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.178 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.178 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.178 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.178 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.178 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.178 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.178 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.178 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.178 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.179 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.179 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.179 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.179 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.179 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.179 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.179 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.179 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.180 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.180 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.180 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.180 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.180 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.180 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.180 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.180 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.181 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.181 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.181 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.181 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.181 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.181 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.181 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.181 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.182 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.182 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.182 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.182 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.182 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.182 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.182 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.182 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.182 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.183 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.183 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.183 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.183 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.183 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.183 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.183 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.183 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.184 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.184 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.184 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.184 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.184 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.184 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.184 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.184 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.185 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.185 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.185 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.185 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mirroring", 2024-02-17T13:07:04.185 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.185 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.185 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.185 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.185 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.186 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.186 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.186 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.186 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.186 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.186 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.186 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.186 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.187 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.187 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.187 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.187 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.187 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.187 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.187 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.187 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.188 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.188 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.188 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.188 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.188 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.188 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.188 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.188 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.189 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.189 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.189 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.189 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.189 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.189 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.189 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.189 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.189 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.190 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.190 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.190 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.190 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.190 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.190 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.190 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.190 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.191 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.191 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.191 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.191 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.191 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.191 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.191 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.191 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.192 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.192 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.192 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.192 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.192 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.192 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.192 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.192 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.193 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.193 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.193 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.193 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.193 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.193 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.193 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.193 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.194 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.194 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.194 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.194 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.194 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.194 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.194 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "nfs", 2024-02-17T13:07:04.194 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.195 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.195 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.195 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.195 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.195 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.195 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.195 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.195 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.195 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.196 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.196 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.196 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.196 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.196 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.196 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.196 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.196 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.197 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.197 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.197 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.197 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.197 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.197 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.197 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.197 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.198 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.198 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.198 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.198 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.198 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.198 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.198 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.198 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.199 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.199 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.199 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.199 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.199 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.199 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.199 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.199 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.200 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.200 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.200 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.200 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.200 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.200 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.200 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.200 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.201 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.201 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.201 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.201 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.201 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.201 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.201 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.201 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.202 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.202 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.202 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.202 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.202 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.202 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.202 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.202 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.202 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "orchestrator", 2024-02-17T13:07:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: "fail_fs": { 2024-02-17T13:07:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "fail_fs", 2024-02-17T13:07:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-17T13:07:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.206 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.206 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.206 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.206 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.206 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.206 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.206 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.206 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.207 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.207 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.207 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.207 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.207 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.207 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.207 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.207 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.207 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.208 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.208 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.208 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.208 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.208 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.208 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.208 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.208 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.209 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.209 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.209 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.209 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.209 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.209 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.209 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.209 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.209 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.210 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.210 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.210 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.210 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.210 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.210 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.210 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.210 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.211 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.211 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.211 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.211 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.211 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.211 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.211 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.211 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.212 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.212 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.212 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.212 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.212 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.212 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.212 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.212 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.213 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.213 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.213 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.213 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.213 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.213 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.213 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.213 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.214 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.214 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.214 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.214 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.214 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.214 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.214 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.214 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator": { 2024-02-17T13:07:04.214 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "orchestrator", 2024-02-17T13:07:04.215 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.215 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.215 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.215 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.215 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.215 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.215 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.215 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm", 2024-02-17T13:07:04.216 INFO:teuthology.orchestra.run.smithi018.stdout: "rook", 2024-02-17T13:07:04.216 INFO:teuthology.orchestra.run.smithi018.stdout: "test_orchestrator" 2024-02-17T13:07:04.216 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.216 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Orchestrator backend", 2024-02-17T13:07:04.216 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.216 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.216 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.216 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.217 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.217 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.217 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.217 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "osd_perf_query", 2024-02-17T13:07:04.217 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.217 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.217 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.217 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.217 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.218 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.218 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.218 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.218 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.218 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.218 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.218 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.218 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.219 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.219 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.219 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.219 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.219 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.219 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.219 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.219 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.220 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.220 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.220 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.220 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.220 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.220 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.220 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.220 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.220 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.221 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.221 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.221 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.221 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.221 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.221 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.221 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.222 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.222 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.222 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.222 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.222 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.222 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.222 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.222 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.223 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.223 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.223 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.223 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.223 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.223 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.223 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.223 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.223 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.224 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.224 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.224 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.224 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.224 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.224 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.224 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.225 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.225 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.225 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.225 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.225 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.225 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.225 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.225 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.225 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.226 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.226 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.226 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.226 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.226 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.226 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.226 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "osd_support", 2024-02-17T13:07:04.227 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.227 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.227 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.227 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.227 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.227 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.227 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.227 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.228 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.228 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.228 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.228 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.228 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.228 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.228 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.228 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.229 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.229 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.229 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.229 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.229 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.229 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.229 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.229 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.230 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.230 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.230 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.230 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.230 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.230 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.230 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.230 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.230 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.231 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.231 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.231 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.231 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.231 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.231 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.231 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.231 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.232 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.232 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.232 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.232 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.232 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.232 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.232 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.232 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.233 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.233 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.233 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.233 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.233 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.233 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.233 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.233 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.233 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.234 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.234 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.234 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.234 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.234 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.234 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.234 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.234 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.235 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.235 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.235 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.235 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.235 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.235 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.235 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.235 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.236 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.236 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.236 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pg_autoscaler", 2024-02-17T13:07:04.236 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.236 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.236 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.236 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.236 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.237 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.237 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.237 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.237 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.237 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.237 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.237 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.237 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.237 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.238 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.238 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.238 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.238 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.238 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.238 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.238 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.239 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.239 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.239 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.239 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.239 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.239 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.239 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.239 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.240 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.240 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.240 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.240 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.240 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.240 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.240 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.240 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.241 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.241 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.241 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.241 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.241 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.241 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.241 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.241 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.242 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.242 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.242 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.242 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.242 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.242 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.242 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.242 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.243 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.243 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.243 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.243 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.243 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.243 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.243 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.243 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.243 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.244 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.244 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.244 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.244 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.244 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.244 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.244 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.244 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.245 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.245 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.245 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.245 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:04.245 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:04.245 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.245 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.245 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.246 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:04.246 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.246 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.246 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.246 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.246 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.246 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.246 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.247 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.247 INFO:teuthology.orchestra.run.smithi018.stdout: "threshold": { 2024-02-17T13:07:04.247 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "threshold", 2024-02-17T13:07:04.247 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:04.247 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.247 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.247 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3.0", 2024-02-17T13:07:04.247 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1.0", 2024-02-17T13:07:04.247 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.248 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.248 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "scaling threshold", 2024-02-17T13:07:04.248 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-17T13:07:04.248 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.248 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.248 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.248 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.248 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.249 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.249 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "progress", 2024-02-17T13:07:04.249 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.249 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.249 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.249 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_pg_recovery_event": { 2024-02-17T13:07:04.249 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_pg_recovery_event", 2024-02-17T13:07:04.249 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.250 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.250 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.250 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.250 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.250 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.250 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.250 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-17T13:07:04.250 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.250 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.251 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.251 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.251 INFO:teuthology.orchestra.run.smithi018.stdout: "enabled": { 2024-02-17T13:07:04.251 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enabled", 2024-02-17T13:07:04.251 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.251 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.251 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.251 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.252 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.252 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.252 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.252 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.252 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.252 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.252 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.252 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.253 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.253 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.253 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.253 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.253 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.253 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.253 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.253 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.253 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.254 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.254 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.254 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.254 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.254 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.254 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.254 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.254 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.255 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.255 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.255 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.255 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.255 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.255 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.255 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.255 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.255 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.256 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.256 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.256 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.256 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.256 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.256 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.256 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.256 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.257 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.257 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.257 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.257 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.257 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.257 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.257 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.257 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.258 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.258 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.258 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.258 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.258 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.258 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.258 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.258 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.259 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.259 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.259 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.259 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.259 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.259 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.259 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.259 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.259 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.260 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.260 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.260 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.260 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.260 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.260 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.260 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.260 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.261 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.261 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.261 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.261 INFO:teuthology.orchestra.run.smithi018.stdout: "max_completed_events": { 2024-02-17T13:07:04.261 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_completed_events", 2024-02-17T13:07:04.261 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.261 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.261 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.262 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "50", 2024-02-17T13:07:04.262 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.262 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.262 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.262 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "number of past completed events to remember", 2024-02-17T13:07:04.262 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.262 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.262 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.262 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.263 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:04.263 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:04.263 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.263 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.263 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.263 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-02-17T13:07:04.263 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.263 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.264 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.264 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long the module is going to sleep", 2024-02-17T13:07:04.264 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.264 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.264 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.264 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.264 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.264 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.265 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.265 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prometheus", 2024-02-17T13:07:04.265 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.265 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.265 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.265 INFO:teuthology.orchestra.run.smithi018.stdout: "cache": { 2024-02-17T13:07:04.265 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cache", 2024-02-17T13:07:04.265 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.265 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.266 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.266 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.266 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.266 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.266 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.266 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.266 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.266 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.267 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.267 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.267 INFO:teuthology.orchestra.run.smithi018.stdout: "exclude_perf_counters": { 2024-02-17T13:07:04.267 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "exclude_perf_counters", 2024-02-17T13:07:04.267 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.267 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.267 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.267 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.267 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.268 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.268 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.268 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-17T13:07:04.268 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-17T13:07:04.268 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.268 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.268 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.269 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.269 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.269 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.269 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.269 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.269 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.269 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.269 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.269 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.270 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.270 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.270 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.270 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.270 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.270 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.270 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.270 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.271 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.271 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.271 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.271 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.271 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.271 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.271 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.271 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.272 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.272 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.272 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.272 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.272 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.272 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.272 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.272 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.273 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.273 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.273 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.273 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.273 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.273 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.273 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.273 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.274 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.274 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.274 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.274 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.274 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.274 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.274 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.274 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.275 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.275 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.275 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.275 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.275 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.275 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.275 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.275 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.276 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.276 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.276 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.276 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.276 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.276 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.276 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.276 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.277 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.277 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.277 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.277 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.277 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.277 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_stats_pools": { 2024-02-17T13:07:04.277 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_stats_pools", 2024-02-17T13:07:04.277 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.278 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.278 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.278 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.278 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.278 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.278 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.278 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.278 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.279 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.279 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.279 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.279 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-17T13:07:04.279 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-17T13:07:04.279 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.279 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.279 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.280 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "300", 2024-02-17T13:07:04.280 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.280 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.280 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.280 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.280 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.280 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.280 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.281 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.281 INFO:teuthology.orchestra.run.smithi018.stdout: "scrape_interval": { 2024-02-17T13:07:04.281 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "scrape_interval", 2024-02-17T13:07:04.281 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:04.281 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.281 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.281 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15.0", 2024-02-17T13:07:04.281 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.282 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.282 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.282 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.282 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.282 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.282 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.282 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.282 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-02-17T13:07:04.283 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-02-17T13:07:04.283 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.283 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.283 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.283 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "::", 2024-02-17T13:07:04.283 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.283 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.283 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.283 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-17T13:07:04.284 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.284 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.284 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.284 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.284 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-02-17T13:07:04.284 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-02-17T13:07:04.284 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.284 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.285 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.285 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "9283", 2024-02-17T13:07:04.285 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.285 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.285 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.285 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-17T13:07:04.285 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.285 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.286 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.286 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.286 INFO:teuthology.orchestra.run.smithi018.stdout: "stale_cache_strategy": { 2024-02-17T13:07:04.286 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "stale_cache_strategy", 2024-02-17T13:07:04.286 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.286 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.286 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.286 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "log", 2024-02-17T13:07:04.287 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.287 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.287 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.287 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.287 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.287 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.287 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.287 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.288 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_behaviour": { 2024-02-17T13:07:04.288 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_behaviour", 2024-02-17T13:07:04.288 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.288 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.288 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.288 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "default", 2024-02-17T13:07:04.288 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.288 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.288 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.289 INFO:teuthology.orchestra.run.smithi018.stdout: "default", 2024-02-17T13:07:04.289 INFO:teuthology.orchestra.run.smithi018.stdout: "error" 2024-02-17T13:07:04.289 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.289 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.289 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.289 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.289 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.289 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.290 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_error_status_code": { 2024-02-17T13:07:04.290 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_error_status_code", 2024-02-17T13:07:04.290 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.290 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.290 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.290 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "500", 2024-02-17T13:07:04.290 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "400", 2024-02-17T13:07:04.290 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "599", 2024-02-17T13:07:04.291 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.291 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.291 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.291 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.291 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.291 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.291 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.291 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.292 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.292 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_support", 2024-02-17T13:07:04.292 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.292 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.292 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.292 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.292 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.292 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.293 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.293 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.293 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.293 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.293 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.293 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.293 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.293 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.294 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.294 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.294 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.294 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.294 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.294 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.294 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.294 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.294 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.295 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.295 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.295 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.295 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.295 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.295 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.295 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.295 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.296 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.296 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.296 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.296 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.296 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.296 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.296 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.296 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.297 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.297 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.297 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.297 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.297 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.297 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.297 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.297 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.297 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.298 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.298 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.298 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.298 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.298 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.298 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.298 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.298 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.299 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.299 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.299 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.299 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.299 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.299 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.299 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.300 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.300 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.300 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.300 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.300 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.300 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.300 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.300 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.301 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.301 INFO:teuthology.orchestra.run.smithi018.stdout: "max_concurrent_snap_create": { 2024-02-17T13:07:04.301 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_concurrent_snap_create", 2024-02-17T13:07:04.301 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.301 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.301 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.301 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:04.301 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.301 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.302 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.302 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.302 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.302 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.302 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.302 INFO:teuthology.orchestra.run.smithi018.stdout: "mirror_snapshot_schedule": { 2024-02-17T13:07:04.302 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mirror_snapshot_schedule", 2024-02-17T13:07:04.302 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.303 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.303 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.303 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.303 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.303 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.303 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.303 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.303 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.304 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.304 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.304 INFO:teuthology.orchestra.run.smithi018.stdout: "trash_purge_schedule": { 2024-02-17T13:07:04.304 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "trash_purge_schedule", 2024-02-17T13:07:04.304 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.304 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.304 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.305 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.305 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.305 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.305 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.305 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.305 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.305 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.305 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.305 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.306 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.306 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.306 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.306 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "restful", 2024-02-17T13:07:04.306 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.306 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.306 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.306 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_auth": { 2024-02-17T13:07:04.307 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enable_auth", 2024-02-17T13:07:04.307 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.307 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.307 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.307 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.307 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.307 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.307 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.308 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.308 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.308 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.308 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.308 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.308 INFO:teuthology.orchestra.run.smithi018.stdout: "key_file": { 2024-02-17T13:07:04.308 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "key_file", 2024-02-17T13:07:04.308 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.309 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.309 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.309 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.309 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.309 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.309 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.309 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.309 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.309 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.310 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.310 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.310 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.310 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.310 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.310 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.310 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.310 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.310 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.311 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.311 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.311 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.311 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.311 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.311 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.311 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.311 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.312 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.312 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.312 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.312 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.312 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.312 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.312 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.312 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.313 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.313 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.313 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.313 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.313 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.313 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.313 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.313 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.314 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.314 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.314 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.314 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.314 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.314 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.314 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.315 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.315 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.315 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.315 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.315 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.315 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.315 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.315 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.316 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.316 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.316 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.316 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.316 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.316 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.316 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.316 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.316 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.317 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.317 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.317 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.317 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.317 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.317 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.317 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.317 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.318 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.318 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.318 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.318 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.318 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.318 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.318 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.318 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-02-17T13:07:04.318 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-02-17T13:07:04.319 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.319 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.319 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.319 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.319 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.319 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.319 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.319 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.320 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.320 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.320 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.320 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.320 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-02-17T13:07:04.320 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-02-17T13:07:04.320 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.320 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.321 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.321 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.321 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.321 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.321 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.321 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.321 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.321 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.321 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.322 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.322 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.322 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.322 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.322 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rgw", 2024-02-17T13:07:04.322 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.322 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.322 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.323 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.323 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.323 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.323 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.323 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.323 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.323 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.323 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.324 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.324 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.324 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.324 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.324 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.324 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.324 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.324 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.325 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.325 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.325 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.325 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.325 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.325 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.325 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.325 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.326 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.326 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.326 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.326 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.326 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.326 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.326 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.326 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.326 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.327 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.327 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.327 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.327 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.327 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.327 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.327 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.328 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.328 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.328 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.328 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.328 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.328 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.328 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.328 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.329 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.329 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.329 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.329 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.329 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.329 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.329 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.329 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.330 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.330 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.330 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.330 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.330 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.330 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.330 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.330 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.331 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.331 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.331 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.331 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.331 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.331 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.331 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.331 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.331 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rook", 2024-02-17T13:07:04.332 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.332 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.332 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.332 INFO:teuthology.orchestra.run.smithi018.stdout: "drive_group_interval": { 2024-02-17T13:07:04.332 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "drive_group_interval", 2024-02-17T13:07:04.332 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:04.332 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.332 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.333 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "300.0", 2024-02-17T13:07:04.333 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.333 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.333 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.333 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-17T13:07:04.333 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.333 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.333 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.334 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.334 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.334 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.334 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.334 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.334 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.334 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.334 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.335 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.335 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.335 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.335 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.335 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.335 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.335 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.335 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.336 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.336 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.336 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.336 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.336 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.336 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.336 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.336 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.337 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.337 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.337 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.337 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.337 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.337 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.337 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.338 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.338 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.338 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.338 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.338 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.338 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.338 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.338 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.339 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.339 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.339 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.339 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.339 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.339 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.339 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.339 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.340 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.340 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.340 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.340 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.340 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.340 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.340 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.340 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.341 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.341 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.341 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.341 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.341 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.341 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.341 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.341 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.342 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.342 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.342 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.342 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.342 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.342 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.342 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.342 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.343 INFO:teuthology.orchestra.run.smithi018.stdout: "storage_class": { 2024-02-17T13:07:04.343 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "storage_class", 2024-02-17T13:07:04.343 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.343 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.343 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.343 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "local", 2024-02-17T13:07:04.343 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.343 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.343 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.344 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-17T13:07:04.344 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.344 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.344 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.344 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.344 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.344 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.344 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.345 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "selftest", 2024-02-17T13:07:04.345 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.345 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.345 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.345 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.345 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.345 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.345 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.345 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.346 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.346 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.346 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.346 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.346 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.346 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.346 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.346 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.347 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.347 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.347 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.347 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.347 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.347 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.347 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.347 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.348 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.348 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.348 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.348 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.348 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.348 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.348 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.348 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.349 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.349 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.349 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.349 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.349 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.349 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.349 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.349 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.350 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.350 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.350 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.350 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.350 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.350 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.350 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.350 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.350 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.351 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.351 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.351 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.351 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.351 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.351 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.351 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.351 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.352 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.352 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.352 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.352 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.352 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.352 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.352 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.352 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.353 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.353 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.353 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.353 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.353 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.353 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.353 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.354 INFO:teuthology.orchestra.run.smithi018.stdout: "roption1": { 2024-02-17T13:07:04.354 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "roption1", 2024-02-17T13:07:04.354 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.354 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.354 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.354 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.354 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.354 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.355 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.355 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.355 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.355 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.355 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.355 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.355 INFO:teuthology.orchestra.run.smithi018.stdout: "roption2": { 2024-02-17T13:07:04.355 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "roption2", 2024-02-17T13:07:04.355 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.356 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.356 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.356 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "xyz", 2024-02-17T13:07:04.356 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.356 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.356 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.356 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.356 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.357 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.357 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.357 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.357 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption1": { 2024-02-17T13:07:04.357 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption1", 2024-02-17T13:07:04.357 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.357 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.357 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.358 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.358 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.358 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.358 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.358 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.358 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.358 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.358 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.358 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.359 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption2": { 2024-02-17T13:07:04.359 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption2", 2024-02-17T13:07:04.359 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.359 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.359 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.359 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.359 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.359 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.360 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.360 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.360 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.360 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.360 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.360 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.360 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption3": { 2024-02-17T13:07:04.360 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption3", 2024-02-17T13:07:04.361 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:04.361 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.361 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.361 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.361 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.361 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.361 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.361 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.362 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.362 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.362 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.362 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.362 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption4": { 2024-02-17T13:07:04.362 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption4", 2024-02-17T13:07:04.362 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.362 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.363 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.363 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.363 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.363 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.363 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.363 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.363 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.363 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.364 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.364 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.364 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption5": { 2024-02-17T13:07:04.364 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption5", 2024-02-17T13:07:04.364 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.364 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.364 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.364 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.365 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.365 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.365 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.365 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.365 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.365 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.365 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.365 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.365 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption6": { 2024-02-17T13:07:04.366 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption6", 2024-02-17T13:07:04.366 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.366 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.366 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.366 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.366 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.366 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.366 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.367 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.367 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.367 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.367 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.367 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.367 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption7": { 2024-02-17T13:07:04.367 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption7", 2024-02-17T13:07:04.367 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.368 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.368 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.368 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.368 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-02-17T13:07:04.368 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "42", 2024-02-17T13:07:04.368 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.368 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.368 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.369 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.369 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.369 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.369 INFO:teuthology.orchestra.run.smithi018.stdout: "testkey": { 2024-02-17T13:07:04.369 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testkey", 2024-02-17T13:07:04.369 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.369 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.369 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.370 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.370 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.370 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.370 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.370 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.370 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.370 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.370 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.370 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.371 INFO:teuthology.orchestra.run.smithi018.stdout: "testlkey": { 2024-02-17T13:07:04.371 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testlkey", 2024-02-17T13:07:04.371 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.371 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.371 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.371 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.371 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.371 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.372 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.372 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.372 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.372 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.372 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.372 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.372 INFO:teuthology.orchestra.run.smithi018.stdout: "testnewline": { 2024-02-17T13:07:04.372 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testnewline", 2024-02-17T13:07:04.373 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.373 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.373 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.373 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.373 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.373 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.373 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.373 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.374 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.374 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.374 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.374 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.374 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.374 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.374 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.374 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snap_schedule", 2024-02-17T13:07:04.374 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.375 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.375 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.375 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_m_granularity": { 2024-02-17T13:07:04.375 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_m_granularity", 2024-02-17T13:07:04.375 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.375 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.375 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.375 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.376 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.376 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.376 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.376 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow minute scheduled snapshots", 2024-02-17T13:07:04.376 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.376 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.376 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.376 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.376 INFO:teuthology.orchestra.run.smithi018.stdout: "dump_on_update": { 2024-02-17T13:07:04.377 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "dump_on_update", 2024-02-17T13:07:04.377 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.377 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.377 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.377 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.377 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.377 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.377 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.378 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "dump database to debug log on update", 2024-02-17T13:07:04.378 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.378 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.378 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.378 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.378 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.378 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.378 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.378 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.379 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.379 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.379 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.379 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.379 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.379 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.379 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.379 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.380 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.380 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.380 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.380 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.380 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.380 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.380 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.380 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.381 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.381 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.381 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.381 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.381 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.381 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.381 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.381 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.382 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.382 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.382 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.382 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.382 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.382 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.382 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.382 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.382 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.383 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.383 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.383 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.383 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.383 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.383 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.383 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.384 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.384 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.384 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.384 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.384 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.384 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.384 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.384 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.385 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.385 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.385 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.385 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.385 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.385 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.385 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.386 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.386 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.386 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.386 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.386 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.386 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.386 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.386 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.387 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.387 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.387 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.387 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.387 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.387 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.387 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "stats", 2024-02-17T13:07:04.387 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.388 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.388 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.388 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.388 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.388 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.388 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.388 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.389 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.389 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.389 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.389 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.389 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.389 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.389 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.389 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.390 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.390 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.390 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.390 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.390 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.390 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.390 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.390 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.391 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.391 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.391 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.391 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.391 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.391 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.391 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.391 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.392 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.392 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.392 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.392 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.392 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.392 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.392 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.393 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.393 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.393 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.393 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.393 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.393 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.393 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.393 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.393 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.394 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.394 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.394 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.394 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.394 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.394 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.394 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.394 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.395 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.395 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.395 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.395 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.395 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.395 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.395 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.395 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.395 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.396 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.396 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.396 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.396 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.396 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.396 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.396 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.396 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.397 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.397 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.397 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.397 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "status", 2024-02-17T13:07:04.397 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.397 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.397 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.397 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.397 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.398 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.398 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.398 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.398 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.398 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.398 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.398 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.398 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.401 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.401 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.401 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.401 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.401 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.401 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.401 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.401 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.402 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.402 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.402 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.402 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.402 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.402 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.402 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.402 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.402 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.403 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.403 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.403 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.403 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.403 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.403 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.403 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.403 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.403 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.404 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.404 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.404 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.404 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.404 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.404 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.404 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.404 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.405 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.405 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.405 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.405 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.405 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.405 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.405 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.405 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.406 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.406 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.406 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.406 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.406 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "telegraf", 2024-02-17T13:07:04.406 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.406 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.406 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.406 INFO:teuthology.orchestra.run.smithi018.stdout: "address": { 2024-02-17T13:07:04.407 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "address", 2024-02-17T13:07:04.407 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.407 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.407 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.407 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-17T13:07:04.407 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.407 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.407 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.408 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.408 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.408 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.408 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.408 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.408 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:04.408 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:04.408 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.409 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.409 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.409 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15", 2024-02-17T13:07:04.409 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.409 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.409 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.409 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.409 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.410 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.410 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.410 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.410 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.410 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.410 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.410 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.410 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.411 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.411 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.411 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.411 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.411 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.411 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.411 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.411 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.411 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.412 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.412 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.412 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.412 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.412 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.412 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.412 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.413 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.413 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.413 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.413 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.413 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.413 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.413 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.413 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.414 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.414 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.414 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.414 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.414 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.414 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.414 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.414 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.414 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.415 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.415 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.415 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.415 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.415 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.415 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.415 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.415 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.415 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.416 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.416 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.416 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.416 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.416 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.416 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.416 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.417 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.417 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.417 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.417 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.417 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.417 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.417 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.417 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.417 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.418 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.418 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.418 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.418 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.418 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.418 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.418 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.418 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.419 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.419 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.419 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "telemetry", 2024-02-17T13:07:04.419 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.419 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.419 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.419 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_basic": { 2024-02-17T13:07:04.419 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_basic", 2024-02-17T13:07:04.419 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.420 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.420 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.420 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.420 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.420 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.420 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.420 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-17T13:07:04.420 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.421 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.421 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.421 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.421 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_crash": { 2024-02-17T13:07:04.421 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_crash", 2024-02-17T13:07:04.421 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.421 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.421 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.422 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.422 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.422 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.422 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.422 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-17T13:07:04.422 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.422 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.422 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.422 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.423 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_device": { 2024-02-17T13:07:04.423 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_device", 2024-02-17T13:07:04.423 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.423 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.423 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.423 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.423 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.423 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.424 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.424 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-17T13:07:04.424 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.424 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.424 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.424 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.424 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_ident": { 2024-02-17T13:07:04.425 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_ident", 2024-02-17T13:07:04.425 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.425 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.425 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.425 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.425 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.425 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.425 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.425 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-17T13:07:04.426 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.426 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.426 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.426 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.426 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_perf": { 2024-02-17T13:07:04.426 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_perf", 2024-02-17T13:07:04.426 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.426 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.427 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.427 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.427 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.427 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.427 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.427 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-17T13:07:04.427 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.427 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.428 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.428 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.428 INFO:teuthology.orchestra.run.smithi018.stdout: "contact": { 2024-02-17T13:07:04.428 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "contact", 2024-02-17T13:07:04.428 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.428 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.428 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.428 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.429 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.429 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.429 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.429 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.429 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.429 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.429 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.429 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.430 INFO:teuthology.orchestra.run.smithi018.stdout: "description": { 2024-02-17T13:07:04.430 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "description", 2024-02-17T13:07:04.430 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.430 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.430 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.430 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.430 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.430 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.431 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.431 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.431 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.431 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.431 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.431 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.431 INFO:teuthology.orchestra.run.smithi018.stdout: "device_url": { 2024-02-17T13:07:04.431 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_url", 2024-02-17T13:07:04.432 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.432 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.432 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.432 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-17T13:07:04.432 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.432 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.432 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.433 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.433 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.433 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.433 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.433 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.433 INFO:teuthology.orchestra.run.smithi018.stdout: "enabled": { 2024-02-17T13:07:04.433 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enabled", 2024-02-17T13:07:04.433 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.434 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.434 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.434 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.434 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.434 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.434 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.434 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.434 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.435 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.435 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.435 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.435 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:04.435 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:04.435 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.435 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.435 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.436 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "24", 2024-02-17T13:07:04.436 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "8", 2024-02-17T13:07:04.436 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.436 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.436 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.436 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.436 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.436 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.437 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.437 INFO:teuthology.orchestra.run.smithi018.stdout: "last_opt_revision": { 2024-02-17T13:07:04.437 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "last_opt_revision", 2024-02-17T13:07:04.437 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.437 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.437 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.437 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1", 2024-02-17T13:07:04.437 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.438 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.438 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.438 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.438 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.438 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.438 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.438 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.438 INFO:teuthology.orchestra.run.smithi018.stdout: "leaderboard": { 2024-02-17T13:07:04.439 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "leaderboard", 2024-02-17T13:07:04.439 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.439 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.439 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.439 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.439 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.439 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.439 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.439 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.440 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.440 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.440 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.440 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.440 INFO:teuthology.orchestra.run.smithi018.stdout: "leaderboard_description": { 2024-02-17T13:07:04.440 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "leaderboard_description", 2024-02-17T13:07:04.440 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.441 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.441 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.441 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.441 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.441 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.441 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.441 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.441 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.442 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.442 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.442 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.442 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.442 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.442 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.442 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.442 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.443 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.443 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.443 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.443 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.443 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.443 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.443 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.443 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.444 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.444 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.444 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.444 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.444 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.444 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.444 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.444 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.445 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.445 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.445 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.445 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.445 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.445 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.445 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.445 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.445 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.446 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.446 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.446 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.446 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.446 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.446 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.446 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.446 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.447 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.447 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.447 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.447 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.447 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.447 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.447 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.447 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.448 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.448 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.448 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.448 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.448 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.448 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.448 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.448 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.449 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.449 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.449 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.449 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.449 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.449 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.449 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.449 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.449 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.450 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.450 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.450 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.450 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.450 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.450 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.450 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.450 INFO:teuthology.orchestra.run.smithi018.stdout: "organization": { 2024-02-17T13:07:04.451 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "organization", 2024-02-17T13:07:04.451 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.451 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.451 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.451 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.451 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.451 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.451 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.452 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.452 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.452 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.452 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.452 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.452 INFO:teuthology.orchestra.run.smithi018.stdout: "proxy": { 2024-02-17T13:07:04.452 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "proxy", 2024-02-17T13:07:04.452 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.453 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.453 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.453 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.453 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.453 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.453 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.453 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.453 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.453 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.454 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.454 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.454 INFO:teuthology.orchestra.run.smithi018.stdout: "url": { 2024-02-17T13:07:04.454 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "url", 2024-02-17T13:07:04.454 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.454 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.454 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.454 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-17T13:07:04.455 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.455 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.455 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.455 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.455 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.455 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.455 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.455 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "test_orchestrator", 2024-02-17T13:07:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.457 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.457 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.457 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.457 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.457 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.457 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.457 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.457 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.458 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.458 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.458 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.458 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.458 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.458 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.458 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.458 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.458 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.459 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.459 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.459 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.459 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.459 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.459 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.459 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.459 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.460 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.460 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.460 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.460 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.460 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.460 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.460 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.460 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.461 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.461 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.461 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.461 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.461 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.461 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.461 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.461 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.462 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.462 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.462 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.462 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.462 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.462 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.462 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.462 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.462 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.463 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.463 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.463 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.463 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.463 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.463 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.463 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.463 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.463 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.464 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.464 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.464 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.464 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.464 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.464 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.464 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.464 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.465 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.465 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.465 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.465 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.465 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.465 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "volumes", 2024-02-17T13:07:04.465 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.465 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.466 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.466 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.466 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.466 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.466 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.466 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.466 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.466 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.467 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.467 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.467 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.467 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.467 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.467 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.467 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.467 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.468 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.468 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.468 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.468 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.468 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.468 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.468 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.468 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.469 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.469 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.469 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.469 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.469 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.469 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.469 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.469 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.470 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.470 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.470 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.470 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.470 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.470 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.470 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.470 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.471 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.471 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.471 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.471 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.471 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.471 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.471 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.471 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.472 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.472 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.472 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.472 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.472 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.472 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.472 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.472 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.472 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.473 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.473 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.473 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.473 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.473 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.473 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.473 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.473 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.474 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.474 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.474 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.474 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.474 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.474 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.474 INFO:teuthology.orchestra.run.smithi018.stdout: "max_concurrent_clones": { 2024-02-17T13:07:04.474 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_concurrent_clones", 2024-02-17T13:07:04.474 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.475 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.475 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.475 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "4", 2024-02-17T13:07:04.475 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.475 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.475 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.475 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-17T13:07:04.475 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.476 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.476 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.476 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.476 INFO:teuthology.orchestra.run.smithi018.stdout: "periodic_async_work": { 2024-02-17T13:07:04.476 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "periodic_async_work", 2024-02-17T13:07:04.476 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.476 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.476 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.477 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.477 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.477 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.477 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.477 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Periodically check for async work", 2024-02-17T13:07:04.477 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.477 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.477 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.478 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.478 INFO:teuthology.orchestra.run.smithi018.stdout: "snapshot_clone_delay": { 2024-02-17T13:07:04.478 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snapshot_clone_delay", 2024-02-17T13:07:04.478 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.478 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.478 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.478 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:04.478 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.478 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.479 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.479 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-17T13:07:04.479 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.479 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.479 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.479 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.479 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.480 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.480 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.480 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix", 2024-02-17T13:07:04.480 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.480 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.480 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.480 INFO:teuthology.orchestra.run.smithi018.stdout: "discovery_interval": { 2024-02-17T13:07:04.480 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "discovery_interval", 2024-02-17T13:07:04.481 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:04.481 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.481 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.481 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "100", 2024-02-17T13:07:04.481 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.481 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.481 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.481 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.482 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.482 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.482 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.482 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.482 INFO:teuthology.orchestra.run.smithi018.stdout: "identifier": { 2024-02-17T13:07:04.482 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "identifier", 2024-02-17T13:07:04.482 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.482 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.483 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.483 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.483 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.483 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.483 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.483 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.483 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.483 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.483 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.484 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.484 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:04.484 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:04.484 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.484 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.484 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.484 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:04.484 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.485 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.485 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.485 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.485 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.485 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.485 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.485 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.485 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.486 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.486 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.486 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.486 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.486 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.486 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.486 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.486 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.486 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.487 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.487 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.487 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.487 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.487 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.487 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.487 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.487 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.488 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.488 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.488 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.488 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.488 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.488 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.488 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.488 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.488 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.489 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.489 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.489 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.489 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.489 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.489 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.489 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.489 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.490 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.490 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.490 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.490 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.490 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.490 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.490 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.490 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.490 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.491 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.491 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.491 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.491 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.491 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.491 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.491 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.491 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.492 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.492 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.492 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.492 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.492 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.492 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.492 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.492 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.492 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.493 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.493 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.493 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.493 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.493 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.493 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.493 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.493 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.494 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.494 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_host": { 2024-02-17T13:07:04.494 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_host", 2024-02-17T13:07:04.494 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.494 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.494 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.494 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.494 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.494 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.495 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.495 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.495 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.495 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.495 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.495 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.495 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_port": { 2024-02-17T13:07:04.495 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_port", 2024-02-17T13:07:04.496 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.496 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.496 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.496 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10051", 2024-02-17T13:07:04.496 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.496 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.496 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.496 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.497 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.497 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.497 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.497 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.497 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_sender": { 2024-02-17T13:07:04.497 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_sender", 2024-02-17T13:07:04.497 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.497 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.498 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.498 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-17T13:07:04.498 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.498 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.498 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.498 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.498 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.498 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.499 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.499 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.499 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.499 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.499 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-02-17T13:07:04.499 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.499 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.499 INFO:teuthology.orchestra.run.smithi018.stdout: "gid": 25108, 2024-02-17T13:07:04.499 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "z", 2024-02-17T13:07:04.500 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_features": 4540701547738038271, 2024-02-17T13:07:04.500 INFO:teuthology.orchestra.run.smithi018.stdout: "available_modules": [ 2024-02-17T13:07:04.500 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.500 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "alerts", 2024-02-17T13:07:04.500 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.500 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.500 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.501 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:04.501 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:04.501 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.501 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.501 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.501 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:04.501 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.501 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.501 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.502 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How frequently to reexamine health status", 2024-02-17T13:07:04.502 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.502 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.502 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.502 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.502 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.502 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.502 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.503 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.503 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.503 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.503 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.503 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.503 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.503 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.503 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.504 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.504 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.504 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.504 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.504 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.504 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.504 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.504 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.505 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.505 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.505 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.505 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.505 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.505 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.505 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.505 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.506 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.506 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.506 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.506 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.506 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.506 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.506 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.506 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.506 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.507 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.507 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.507 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.507 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.507 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.507 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.507 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.507 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.508 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.508 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.508 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.508 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.508 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.508 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.508 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.508 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.509 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.509 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.509 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.509 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.509 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.509 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.509 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.509 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.510 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.510 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.510 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.510 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.510 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.510 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.510 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.510 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.510 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.511 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.511 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_destination": { 2024-02-17T13:07:04.511 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_destination", 2024-02-17T13:07:04.511 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.511 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.511 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.511 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.511 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.512 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.512 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.512 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Email address to send alerts to", 2024-02-17T13:07:04.512 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.512 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.512 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.512 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.512 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_from_name": { 2024-02-17T13:07:04.513 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_from_name", 2024-02-17T13:07:04.513 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.513 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.513 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.513 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "Ceph", 2024-02-17T13:07:04.513 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.513 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.513 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.514 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Email From: name", 2024-02-17T13:07:04.514 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.514 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.514 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.514 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.514 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_host": { 2024-02-17T13:07:04.514 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_host", 2024-02-17T13:07:04.514 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.514 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.515 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.515 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.515 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.515 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.515 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.515 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP server", 2024-02-17T13:07:04.515 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.515 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.516 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.516 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.516 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_password": { 2024-02-17T13:07:04.516 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_password", 2024-02-17T13:07:04.516 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.516 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.516 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.516 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.517 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.517 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.517 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.517 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Password to authenticate with", 2024-02-17T13:07:04.517 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.517 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.517 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.518 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.518 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_port": { 2024-02-17T13:07:04.518 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_port", 2024-02-17T13:07:04.518 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.518 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.518 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.518 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "465", 2024-02-17T13:07:04.518 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.519 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.519 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.519 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP port", 2024-02-17T13:07:04.519 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.519 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.519 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.519 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.519 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_sender": { 2024-02-17T13:07:04.519 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_sender", 2024-02-17T13:07:04.520 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.520 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.520 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.520 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.520 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.520 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.520 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.520 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP envelope sender", 2024-02-17T13:07:04.521 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.521 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.521 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.521 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.521 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_ssl": { 2024-02-17T13:07:04.521 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_ssl", 2024-02-17T13:07:04.521 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.521 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.522 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.522 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.522 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.522 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.522 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.522 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-17T13:07:04.522 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.522 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.523 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.523 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.523 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_user": { 2024-02-17T13:07:04.523 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_user", 2024-02-17T13:07:04.523 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.523 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.523 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.523 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.523 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.524 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.524 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.524 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "User to authenticate as", 2024-02-17T13:07:04.524 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.524 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.524 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.524 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.524 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.525 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.525 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.525 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "balancer", 2024-02-17T13:07:04.525 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.525 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.525 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.525 INFO:teuthology.orchestra.run.smithi018.stdout: "active": { 2024-02-17T13:07:04.525 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "active", 2024-02-17T13:07:04.526 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.526 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.526 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.526 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.526 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.526 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.526 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.526 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "automatically balance PGs across cluster", 2024-02-17T13:07:04.527 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.527 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.527 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.527 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.527 INFO:teuthology.orchestra.run.smithi018.stdout: "begin_time": { 2024-02-17T13:07:04.527 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "begin_time", 2024-02-17T13:07:04.527 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.528 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.528 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.528 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0000", 2024-02-17T13:07:04.528 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.528 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.528 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.528 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "beginning time of day to automatically balance", 2024-02-17T13:07:04.528 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-17T13:07:04.529 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.529 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.529 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.529 INFO:teuthology.orchestra.run.smithi018.stdout: "begin_weekday": { 2024-02-17T13:07:04.529 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "begin_weekday", 2024-02-17T13:07:04.529 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:04.529 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.529 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.530 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:04.530 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0", 2024-02-17T13:07:04.530 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "6", 2024-02-17T13:07:04.530 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.530 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-17T13:07:04.530 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-17T13:07:04.530 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.530 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.531 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.531 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_max_iterations": { 2024-02-17T13:07:04.531 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_max_iterations", 2024-02-17T13:07:04.531 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:04.531 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.531 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.531 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "25", 2024-02-17T13:07:04.531 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-02-17T13:07:04.532 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "250", 2024-02-17T13:07:04.532 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.532 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-17T13:07:04.532 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.532 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.532 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.532 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.532 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_metrics": { 2024-02-17T13:07:04.533 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_metrics", 2024-02-17T13:07:04.533 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.533 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.533 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.533 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "pgs,objects,bytes", 2024-02-17T13:07:04.533 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.533 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.533 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.534 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-17T13:07:04.534 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-17T13:07:04.534 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.534 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.534 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.534 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_step": { 2024-02-17T13:07:04.534 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_step", 2024-02-17T13:07:04.534 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:04.535 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.535 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.535 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0.5", 2024-02-17T13:07:04.535 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0.001", 2024-02-17T13:07:04.535 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "0.999", 2024-02-17T13:07:04.535 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.535 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "aggressiveness of optimization", 2024-02-17T13:07:04.535 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-17T13:07:04.535 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.536 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.536 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.536 INFO:teuthology.orchestra.run.smithi018.stdout: "end_time": { 2024-02-17T13:07:04.536 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "end_time", 2024-02-17T13:07:04.536 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.536 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.536 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.536 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "2359", 2024-02-17T13:07:04.537 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.537 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.537 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.537 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "ending time of day to automatically balance", 2024-02-17T13:07:04.537 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-17T13:07:04.537 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.537 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.537 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.538 INFO:teuthology.orchestra.run.smithi018.stdout: "end_weekday": { 2024-02-17T13:07:04.538 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "end_weekday", 2024-02-17T13:07:04.538 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:04.538 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.538 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.538 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:04.538 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0", 2024-02-17T13:07:04.538 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "6", 2024-02-17T13:07:04.538 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.539 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-17T13:07:04.539 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-17T13:07:04.539 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.539 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.539 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.539 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.539 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.539 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.540 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.540 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.540 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.540 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.540 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.540 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.540 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.540 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.541 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.541 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.541 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.541 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.541 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.541 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.541 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.541 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.542 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.542 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.542 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.542 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.542 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.542 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.542 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.542 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.542 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.543 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.543 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.543 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.543 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.543 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.543 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.543 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.543 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.544 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.544 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.544 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.544 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.544 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.544 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.544 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.544 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.545 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.545 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.545 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.545 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.545 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.545 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.545 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.546 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.546 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.546 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.546 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.546 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.546 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.546 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.546 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.546 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.547 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.547 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.547 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.547 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.547 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.547 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.547 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.547 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.548 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.548 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.548 INFO:teuthology.orchestra.run.smithi018.stdout: "min_score": { 2024-02-17T13:07:04.548 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "min_score", 2024-02-17T13:07:04.548 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:04.548 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.548 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.549 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:04.549 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.549 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.549 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.549 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-17T13:07:04.549 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.549 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.549 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.549 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.550 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": { 2024-02-17T13:07:04.550 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mode", 2024-02-17T13:07:04.550 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.550 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.550 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.550 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "upmap", 2024-02-17T13:07:04.550 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.550 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.551 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.551 INFO:teuthology.orchestra.run.smithi018.stdout: "crush-compat", 2024-02-17T13:07:04.551 INFO:teuthology.orchestra.run.smithi018.stdout: "none", 2024-02-17T13:07:04.551 INFO:teuthology.orchestra.run.smithi018.stdout: "read", 2024-02-17T13:07:04.551 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap", 2024-02-17T13:07:04.551 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap-read" 2024-02-17T13:07:04.551 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.551 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Balancer mode", 2024-02-17T13:07:04.551 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.552 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.552 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.552 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.552 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_ids": { 2024-02-17T13:07:04.552 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pool_ids", 2024-02-17T13:07:04.552 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.552 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.552 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.553 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.553 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.553 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.553 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.553 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-17T13:07:04.553 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.553 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.553 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.553 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.554 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:04.554 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:04.554 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.554 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.554 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.554 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:04.554 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.554 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.555 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.555 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-17T13:07:04.555 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.555 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.555 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.555 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.555 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap_max_deviation": { 2024-02-17T13:07:04.555 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "upmap_max_deviation", 2024-02-17T13:07:04.555 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.556 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.556 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.556 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-02-17T13:07:04.556 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-02-17T13:07:04.556 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.556 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.556 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-17T13:07:04.556 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-17T13:07:04.557 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.557 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.557 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.557 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap_max_optimizations": { 2024-02-17T13:07:04.557 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "upmap_max_optimizations", 2024-02-17T13:07:04.557 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:04.557 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.557 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.557 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:04.558 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.558 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.558 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.558 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-17T13:07:04.558 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.558 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.558 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.558 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.559 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.559 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.559 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.559 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cephadm", 2024-02-17T13:07:04.559 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.559 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.559 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.559 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_down_multiplier": { 2024-02-17T13:07:04.559 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_down_multiplier", 2024-02-17T13:07:04.560 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:04.560 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.560 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.560 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3.0", 2024-02-17T13:07:04.560 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.560 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.560 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.561 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-17T13:07:04.561 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.561 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.561 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.561 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.561 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_refresh_rate": { 2024-02-17T13:07:04.561 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_refresh_rate", 2024-02-17T13:07:04.561 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.561 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.562 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.562 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "20", 2024-02-17T13:07:04.562 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.562 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.562 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.562 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-17T13:07:04.562 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.562 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.563 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.563 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.563 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_starting_port": { 2024-02-17T13:07:04.563 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_starting_port", 2024-02-17T13:07:04.563 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.563 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.563 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.563 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "4721", 2024-02-17T13:07:04.563 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-17T13:07:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_ptrace": { 2024-02-17T13:07:04.565 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_ptrace", 2024-02-17T13:07:04.565 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.565 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.565 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.565 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.565 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.565 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.565 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.566 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-17T13:07:04.566 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-17T13:07:04.566 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.566 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.566 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.566 INFO:teuthology.orchestra.run.smithi018.stdout: "autotune_interval": { 2024-02-17T13:07:04.566 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "autotune_interval", 2024-02-17T13:07:04.566 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.566 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.567 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.567 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:04.567 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.567 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.567 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.567 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-17T13:07:04.567 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.567 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.568 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.568 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.568 INFO:teuthology.orchestra.run.smithi018.stdout: "autotune_memory_target_ratio": { 2024-02-17T13:07:04.568 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "autotune_memory_target_ratio", 2024-02-17T13:07:04.568 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:04.568 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.568 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.568 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0.7", 2024-02-17T13:07:04.569 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.569 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.569 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.569 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-17T13:07:04.569 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.569 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.569 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.569 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.570 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm_log_destination": { 2024-02-17T13:07:04.570 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cephadm_log_destination", 2024-02-17T13:07:04.570 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.570 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.570 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.570 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.570 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.570 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.571 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.571 INFO:teuthology.orchestra.run.smithi018.stdout: "file", 2024-02-17T13:07:04.571 INFO:teuthology.orchestra.run.smithi018.stdout: "file,syslog", 2024-02-17T13:07:04.571 INFO:teuthology.orchestra.run.smithi018.stdout: "syslog" 2024-02-17T13:07:04.571 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.571 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-17T13:07:04.571 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.571 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.572 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.572 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.572 INFO:teuthology.orchestra.run.smithi018.stdout: "cgroups_split": { 2024-02-17T13:07:04.572 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cgroups_split", 2024-02-17T13:07:04.572 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.572 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.572 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.572 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.573 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.573 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.573 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.573 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-17T13:07:04.573 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.573 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.573 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.573 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.573 INFO:teuthology.orchestra.run.smithi018.stdout: "config_checks_enabled": { 2024-02-17T13:07:04.574 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_checks_enabled", 2024-02-17T13:07:04.574 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.574 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.574 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.574 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.574 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.574 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.574 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.575 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-17T13:07:04.575 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.575 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.575 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.575 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.575 INFO:teuthology.orchestra.run.smithi018.stdout: "config_dashboard": { 2024-02-17T13:07:04.575 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_dashboard", 2024-02-17T13:07:04.575 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.576 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.576 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.576 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.576 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.576 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.576 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.576 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-17T13:07:04.576 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.577 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.577 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.577 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.577 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_alertmanager": { 2024-02-17T13:07:04.577 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_alertmanager", 2024-02-17T13:07:04.577 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.577 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.577 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.578 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-17T13:07:04.578 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.578 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.578 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.578 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-02-17T13:07:04.578 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.578 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.578 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.579 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.579 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_base": { 2024-02-17T13:07:04.579 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_base", 2024-02-17T13:07:04.579 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.579 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.579 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.579 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-17T13:07:04.579 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.580 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.580 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.580 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Container image name, without the tag", 2024-02-17T13:07:04.580 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.580 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.580 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.580 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.580 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_elasticsearch": { 2024-02-17T13:07:04.581 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_elasticsearch", 2024-02-17T13:07:04.581 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.581 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.581 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.581 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-17T13:07:04.581 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.581 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.582 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.582 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "elasticsearch container image", 2024-02-17T13:07:04.582 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.582 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.582 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.582 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.582 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_grafana": { 2024-02-17T13:07:04.582 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_grafana", 2024-02-17T13:07:04.583 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.583 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.583 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.583 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-17T13:07:04.583 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.583 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.583 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.583 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-02-17T13:07:04.583 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.584 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.584 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.584 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.584 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_haproxy": { 2024-02-17T13:07:04.584 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_haproxy", 2024-02-17T13:07:04.584 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.584 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.585 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.585 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-17T13:07:04.585 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.585 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.585 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.585 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "HAproxy container image", 2024-02-17T13:07:04.585 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.585 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.585 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.586 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.586 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_agent": { 2024-02-17T13:07:04.586 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_agent", 2024-02-17T13:07:04.586 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.586 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.586 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.586 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-17T13:07:04.586 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.587 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.587 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.587 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger agent container image", 2024-02-17T13:07:04.587 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.587 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.587 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.587 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.587 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_collector": { 2024-02-17T13:07:04.588 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_collector", 2024-02-17T13:07:04.588 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.588 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.588 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.588 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-17T13:07:04.588 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.588 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.588 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.589 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger collector container image", 2024-02-17T13:07:04.589 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.589 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.589 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.589 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.589 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_query": { 2024-02-17T13:07:04.589 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_query", 2024-02-17T13:07:04.589 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.590 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.590 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.590 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-17T13:07:04.590 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.590 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.590 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.590 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger query container image", 2024-02-17T13:07:04.590 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.591 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.591 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.591 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.591 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_keepalived": { 2024-02-17T13:07:04.591 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_keepalived", 2024-02-17T13:07:04.591 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.591 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.591 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.592 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-17T13:07:04.592 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.592 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.592 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.592 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Keepalived container image", 2024-02-17T13:07:04.592 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.592 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.592 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.593 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.593 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_loki": { 2024-02-17T13:07:04.593 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_loki", 2024-02-17T13:07:04.593 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.593 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.593 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.593 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-17T13:07:04.593 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.593 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.594 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.594 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Loki container image", 2024-02-17T13:07:04.594 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.594 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.594 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.594 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.594 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_node_exporter": { 2024-02-17T13:07:04.594 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_node_exporter", 2024-02-17T13:07:04.595 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.595 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.595 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.595 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-17T13:07:04.595 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.595 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.595 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.595 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-02-17T13:07:04.596 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.596 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.596 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.596 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.596 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_nvmeof": { 2024-02-17T13:07:04.596 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_nvmeof", 2024-02-17T13:07:04.596 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.596 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.597 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.597 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-17T13:07:04.597 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.597 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.597 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.597 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Nvme-of container image", 2024-02-17T13:07:04.597 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.597 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.597 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.598 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.598 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_prometheus": { 2024-02-17T13:07:04.598 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_prometheus", 2024-02-17T13:07:04.598 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.598 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.598 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.598 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-17T13:07:04.598 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.599 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.599 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.599 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-02-17T13:07:04.599 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.599 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.599 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.599 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.599 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_promtail": { 2024-02-17T13:07:04.599 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_promtail", 2024-02-17T13:07:04.600 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.600 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.600 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.600 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-17T13:07:04.600 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.600 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.600 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.601 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Promtail container image", 2024-02-17T13:07:04.601 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.601 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.601 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.601 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.601 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_snmp_gateway": { 2024-02-17T13:07:04.601 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_snmp_gateway", 2024-02-17T13:07:04.601 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.602 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.602 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.602 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-17T13:07:04.602 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.602 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.602 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.602 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SNMP Gateway container image", 2024-02-17T13:07:04.602 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.603 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.603 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.603 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.603 INFO:teuthology.orchestra.run.smithi018.stdout: "container_init": { 2024-02-17T13:07:04.603 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_init", 2024-02-17T13:07:04.603 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.603 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.603 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.604 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.604 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.604 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.604 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.604 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Run podman/docker with `--init`", 2024-02-17T13:07:04.604 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.604 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.607 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.607 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.607 INFO:teuthology.orchestra.run.smithi018.stdout: "daemon_cache_timeout": { 2024-02-17T13:07:04.607 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "daemon_cache_timeout", 2024-02-17T13:07:04.607 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.607 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.607 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.608 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:04.608 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.608 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.608 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.608 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-17T13:07:04.608 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.608 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.608 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.609 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.609 INFO:teuthology.orchestra.run.smithi018.stdout: "default_cephadm_command_timeout": { 2024-02-17T13:07:04.609 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "default_cephadm_command_timeout", 2024-02-17T13:07:04.609 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.609 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.609 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.609 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "900", 2024-02-17T13:07:04.610 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.610 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.610 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.610 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-17T13:07:04.610 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.610 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.610 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.610 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.611 INFO:teuthology.orchestra.run.smithi018.stdout: "default_registry": { 2024-02-17T13:07:04.611 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "default_registry", 2024-02-17T13:07:04.611 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.611 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.611 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.611 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io", 2024-02-17T13:07:04.611 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.611 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.612 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.612 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-17T13:07:04.612 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.612 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.612 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.612 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.612 INFO:teuthology.orchestra.run.smithi018.stdout: "device_cache_timeout": { 2024-02-17T13:07:04.612 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_cache_timeout", 2024-02-17T13:07:04.613 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.613 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.613 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.613 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1800", 2024-02-17T13:07:04.613 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.613 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.613 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.613 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache device inventory", 2024-02-17T13:07:04.614 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.614 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.614 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.614 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.614 INFO:teuthology.orchestra.run.smithi018.stdout: "device_enhanced_scan": { 2024-02-17T13:07:04.614 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_enhanced_scan", 2024-02-17T13:07:04.614 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.614 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.615 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.615 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.615 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.615 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.615 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.615 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-17T13:07:04.615 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.615 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.615 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.616 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.616 INFO:teuthology.orchestra.run.smithi018.stdout: "facts_cache_timeout": { 2024-02-17T13:07:04.616 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "facts_cache_timeout", 2024-02-17T13:07:04.616 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.616 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.616 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.616 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:04.616 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.617 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.617 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.617 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache host facts data", 2024-02-17T13:07:04.617 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.617 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.617 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.617 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.617 INFO:teuthology.orchestra.run.smithi018.stdout: "host_check_interval": { 2024-02-17T13:07:04.618 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "host_check_interval", 2024-02-17T13:07:04.618 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.618 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.618 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.618 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:04.618 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.618 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.618 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.618 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to perform a host check", 2024-02-17T13:07:04.619 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.619 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.619 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.619 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.619 INFO:teuthology.orchestra.run.smithi018.stdout: "hw_monitoring": { 2024-02-17T13:07:04.619 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "hw_monitoring", 2024-02-17T13:07:04.619 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.620 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.620 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.620 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.620 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.620 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.620 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.620 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-17T13:07:04.620 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.621 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.621 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.621 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.621 INFO:teuthology.orchestra.run.smithi018.stdout: "inventory_list_all": { 2024-02-17T13:07:04.621 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "inventory_list_all", 2024-02-17T13:07:04.621 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.621 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.621 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.621 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.622 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.622 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.622 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.622 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-17T13:07:04.622 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.622 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.622 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.622 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.623 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.623 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.623 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.623 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.623 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.623 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.623 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.623 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.624 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.624 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.624 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.624 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.624 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.624 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.624 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.624 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.625 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.625 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.625 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.625 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.625 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.625 INFO:teuthology.orchestra.run.smithi018.stdout: "log_refresh_metadata": { 2024-02-17T13:07:04.625 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_refresh_metadata", 2024-02-17T13:07:04.626 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.626 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.626 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.626 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.626 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.626 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.626 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.626 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-17T13:07:04.626 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.627 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.627 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.627 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.627 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.627 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.627 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.627 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.627 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.628 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.628 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.628 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.628 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.628 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-17T13:07:04.628 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.628 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.628 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.629 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.629 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.629 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.629 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.629 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.629 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.629 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.629 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.629 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.630 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.630 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.630 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.630 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.630 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.630 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.630 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.630 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.631 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.631 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.631 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.631 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.631 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.631 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.631 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.631 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.632 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.632 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.632 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.632 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.632 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.632 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.632 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.632 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.633 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.633 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.633 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.633 INFO:teuthology.orchestra.run.smithi018.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-17T13:07:04.633 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-17T13:07:04.633 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.633 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.633 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.634 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.634 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.634 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.634 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.634 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-17T13:07:04.634 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.634 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.634 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.635 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.635 INFO:teuthology.orchestra.run.smithi018.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-17T13:07:04.635 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-17T13:07:04.635 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.635 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.635 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.635 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "*", 2024-02-17T13:07:04.635 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.636 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.636 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.636 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-17T13:07:04.636 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.636 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.636 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.636 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.637 INFO:teuthology.orchestra.run.smithi018.stdout: "max_count_per_host": { 2024-02-17T13:07:04.637 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_count_per_host", 2024-02-17T13:07:04.637 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.637 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.637 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.637 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:04.637 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.637 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.637 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.638 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "max number of daemons per service per host", 2024-02-17T13:07:04.638 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.638 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.638 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.638 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.638 INFO:teuthology.orchestra.run.smithi018.stdout: "max_osd_draining_count": { 2024-02-17T13:07:04.638 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_osd_draining_count", 2024-02-17T13:07:04.638 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.639 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.639 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.639 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:04.639 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.639 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.639 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.639 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-17T13:07:04.639 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.640 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.640 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.640 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.640 INFO:teuthology.orchestra.run.smithi018.stdout: "migration_current": { 2024-02-17T13:07:04.640 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "migration_current", 2024-02-17T13:07:04.640 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.640 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.640 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.640 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.641 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.641 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.641 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.641 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "internal - do not modify", 2024-02-17T13:07:04.641 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.641 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.641 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.641 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.642 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": { 2024-02-17T13:07:04.642 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mode", 2024-02-17T13:07:04.642 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.642 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.642 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.642 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "root", 2024-02-17T13:07:04.642 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.642 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.643 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.643 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm-package", 2024-02-17T13:07:04.643 INFO:teuthology.orchestra.run.smithi018.stdout: "root" 2024-02-17T13:07:04.643 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.643 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "mode for remote execution of cephadm", 2024-02-17T13:07:04.643 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.643 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.643 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.644 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.644 INFO:teuthology.orchestra.run.smithi018.stdout: "oob_default_addr": { 2024-02-17T13:07:04.644 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "oob_default_addr", 2024-02-17T13:07:04.644 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.644 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.644 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.644 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "169.254.1.1", 2024-02-17T13:07:04.644 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.645 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.645 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.645 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-17T13:07:04.645 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.645 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.645 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.645 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.645 INFO:teuthology.orchestra.run.smithi018.stdout: "prometheus_alerts_path": { 2024-02-17T13:07:04.646 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prometheus_alerts_path", 2024-02-17T13:07:04.646 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.646 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.646 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.646 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-17T13:07:04.646 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.646 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.647 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.647 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-17T13:07:04.647 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.647 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.647 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.647 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.647 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_insecure": { 2024-02-17T13:07:04.647 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_insecure", 2024-02-17T13:07:04.648 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.648 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.648 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.648 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.648 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.648 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.648 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.649 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-17T13:07:04.649 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.649 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.649 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.649 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.649 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_password": { 2024-02-17T13:07:04.649 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_password", 2024-02-17T13:07:04.649 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.649 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.650 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.650 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.650 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.650 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.650 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.650 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-17T13:07:04.650 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.650 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.651 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.651 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.651 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_url": { 2024-02-17T13:07:04.651 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_url", 2024-02-17T13:07:04.651 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.651 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.651 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.651 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.651 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.652 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.652 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.652 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-17T13:07:04.652 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.652 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.652 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.652 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.653 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_username": { 2024-02-17T13:07:04.653 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_username", 2024-02-17T13:07:04.653 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.653 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.653 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.653 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.653 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.653 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.653 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.654 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-17T13:07:04.654 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.654 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.654 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.654 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.654 INFO:teuthology.orchestra.run.smithi018.stdout: "secure_monitoring_stack": { 2024-02-17T13:07:04.654 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "secure_monitoring_stack", 2024-02-17T13:07:04.654 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.655 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.655 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.655 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.655 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.655 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.655 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.655 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-17T13:07:04.655 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.655 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.656 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.656 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.656 INFO:teuthology.orchestra.run.smithi018.stdout: "service_discovery_port": { 2024-02-17T13:07:04.656 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "service_discovery_port", 2024-02-17T13:07:04.656 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.656 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.656 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.657 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8765", 2024-02-17T13:07:04.657 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.657 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.657 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.657 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "cephadm service discovery port", 2024-02-17T13:07:04.657 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.657 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.657 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.658 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.658 INFO:teuthology.orchestra.run.smithi018.stdout: "ssh_config_file": { 2024-02-17T13:07:04.658 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssh_config_file", 2024-02-17T13:07:04.658 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.658 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.658 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.658 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.658 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.659 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.659 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.659 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-17T13:07:04.659 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.659 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.659 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.659 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.659 INFO:teuthology.orchestra.run.smithi018.stdout: "use_agent": { 2024-02-17T13:07:04.660 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "use_agent", 2024-02-17T13:07:04.660 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.660 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.660 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.660 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.660 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.660 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.660 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.660 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-17T13:07:04.661 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.661 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.661 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.661 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.661 INFO:teuthology.orchestra.run.smithi018.stdout: "use_repo_digest": { 2024-02-17T13:07:04.661 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "use_repo_digest", 2024-02-17T13:07:04.661 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.661 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.662 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.662 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.662 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.662 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.662 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.662 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-17T13:07:04.662 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.662 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.662 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.663 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.663 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_failed_host_check": { 2024-02-17T13:07:04.663 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_failed_host_check", 2024-02-17T13:07:04.663 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.663 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.663 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.663 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.663 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.664 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.664 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.664 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if the host check fails", 2024-02-17T13:07:04.664 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.664 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.664 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.664 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.664 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_stray_daemons": { 2024-02-17T13:07:04.665 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_stray_daemons", 2024-02-17T13:07:04.665 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.665 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.665 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.665 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.665 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.665 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.665 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.665 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-17T13:07:04.666 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.666 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.666 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.666 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.666 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_stray_hosts": { 2024-02-17T13:07:04.666 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_stray_hosts", 2024-02-17T13:07:04.666 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.666 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.666 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.667 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.667 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.667 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.667 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.667 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-17T13:07:04.667 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.667 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.667 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.668 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.668 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.668 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.668 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.668 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crash", 2024-02-17T13:07:04.668 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.668 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.668 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.669 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.669 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.669 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.669 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.669 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.669 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.669 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.669 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.670 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.670 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.670 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.670 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.670 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.670 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.670 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.670 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.671 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.671 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.671 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.671 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.671 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.671 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.671 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.671 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.672 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.672 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.672 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.672 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.672 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.672 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.672 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.672 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.673 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.673 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.673 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.673 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.673 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.673 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.673 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.674 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.674 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.674 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.674 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.674 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.674 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.674 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.674 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.675 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.675 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.675 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.675 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.675 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.675 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.675 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.675 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.675 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.676 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.676 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.676 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.676 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.676 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.676 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.676 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.677 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.677 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.677 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.677 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.677 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.677 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.677 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.677 INFO:teuthology.orchestra.run.smithi018.stdout: "retain_interval": { 2024-02-17T13:07:04.677 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "retain_interval", 2024-02-17T13:07:04.678 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.678 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.678 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.678 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "31536000", 2024-02-17T13:07:04.678 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.678 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.678 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.679 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-17T13:07:04.679 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.679 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.679 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.679 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.679 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_recent_interval": { 2024-02-17T13:07:04.679 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_recent_interval", 2024-02-17T13:07:04.679 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.680 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.680 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.680 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1209600", 2024-02-17T13:07:04.680 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.680 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.680 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.680 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-17T13:07:04.680 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.681 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.681 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.681 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.681 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.681 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.681 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.681 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "dashboard", 2024-02-17T13:07:04.681 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.682 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.682 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.682 INFO:teuthology.orchestra.run.smithi018.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-17T13:07:04.682 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-17T13:07:04.682 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.682 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.682 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.682 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:04.682 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.683 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.683 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.683 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.683 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.683 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.683 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.683 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.683 INFO:teuthology.orchestra.run.smithi018.stdout: "ALERTMANAGER_API_HOST": { 2024-02-17T13:07:04.684 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-17T13:07:04.684 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.684 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.684 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.684 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.684 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.684 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.684 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.685 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.685 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.685 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.685 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.685 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.685 INFO:teuthology.orchestra.run.smithi018.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-17T13:07:04.685 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-17T13:07:04.685 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.685 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.686 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.687 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.687 INFO:teuthology.orchestra.run.smithi018.stdout: "AUDIT_API_ENABLED": { 2024-02-17T13:07:04.687 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "AUDIT_API_ENABLED", 2024-02-17T13:07:04.687 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.687 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.687 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.687 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.687 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.688 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.688 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.688 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.688 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.688 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.688 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.688 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.688 INFO:teuthology.orchestra.run.smithi018.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-17T13:07:04.689 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-17T13:07:04.689 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.689 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.689 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.689 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.689 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.689 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.689 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.689 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.690 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.690 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.690 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.690 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.690 INFO:teuthology.orchestra.run.smithi018.stdout: "ENABLE_BROWSABLE_API": { 2024-02-17T13:07:04.690 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-17T13:07:04.690 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.690 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.691 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.691 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.691 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.691 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.691 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.691 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.691 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.691 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.691 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.692 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.692 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-17T13:07:04.692 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-17T13:07:04.692 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.692 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.692 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.692 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.692 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.693 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.693 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.693 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.693 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.693 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.693 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.693 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.693 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-17T13:07:04.694 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-17T13:07:04.694 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.694 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.694 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.694 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.694 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.694 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.694 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.694 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.695 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.695 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.695 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.695 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.695 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-17T13:07:04.695 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-17T13:07:04.695 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.695 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.695 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.696 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.696 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.696 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.696 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.696 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.696 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.696 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.697 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.697 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.697 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-17T13:07:04.697 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-17T13:07:04.697 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.697 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.697 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.697 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.697 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.698 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.698 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.698 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.698 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.698 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.698 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.698 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.698 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-17T13:07:04.698 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-17T13:07:04.699 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.699 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.699 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.699 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.699 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.699 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.699 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.699 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.700 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.700 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.700 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.700 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.700 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-17T13:07:04.700 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-17T13:07:04.700 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.700 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.701 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.701 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.701 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.701 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.701 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.701 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.701 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.701 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.702 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.702 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.702 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-17T13:07:04.702 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-17T13:07:04.702 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.702 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.702 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.702 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.703 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.703 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.703 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.703 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.703 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.703 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.703 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.703 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.704 INFO:teuthology.orchestra.run.smithi018.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-17T13:07:04.704 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-17T13:07:04.704 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.704 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.704 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.704 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.704 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.704 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.705 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.705 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.705 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.705 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.705 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.705 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.705 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_PASSWORD": { 2024-02-17T13:07:04.705 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-17T13:07:04.706 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.706 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.706 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.706 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-02-17T13:07:04.706 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.706 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.706 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.706 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.707 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.707 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.707 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.707 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.707 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-17T13:07:04.707 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-17T13:07:04.707 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.707 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.707 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.708 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.708 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.708 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.708 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.708 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.708 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.708 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.709 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.709 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.709 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_URL": { 2024-02-17T13:07:04.709 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_URL", 2024-02-17T13:07:04.709 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.709 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.709 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.709 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.709 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.710 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.710 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.710 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.710 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.710 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.710 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.710 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.710 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_USERNAME": { 2024-02-17T13:07:04.710 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-17T13:07:04.711 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.711 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.711 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.711 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-02-17T13:07:04.711 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.711 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.711 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.711 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.712 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.712 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.712 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.712 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.712 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-17T13:07:04.712 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-17T13:07:04.712 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.712 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.713 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.713 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.713 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.713 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.713 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.713 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.713 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.713 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.713 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.714 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.714 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-17T13:07:04.714 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-17T13:07:04.714 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.714 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.714 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.714 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.714 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.715 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.715 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.715 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.715 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.715 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.715 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.715 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.715 INFO:teuthology.orchestra.run.smithi018.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-17T13:07:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-17T13:07:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-17T13:07:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-17T13:07:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "PROMETHEUS_API_HOST": { 2024-02-17T13:07:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-17T13:07:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-17T13:07:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-17T13:07:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.722 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.722 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.722 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.722 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.722 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.722 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.722 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.722 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-17T13:07:04.723 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-17T13:07:04.723 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.723 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.723 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.723 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.723 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.723 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.723 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.724 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.724 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.724 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.724 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.724 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.724 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-17T13:07:04.724 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-17T13:07:04.724 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.725 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.725 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.725 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.725 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.725 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.725 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.725 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.725 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.726 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.726 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.726 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.726 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-17T13:07:04.726 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-17T13:07:04.726 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.726 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.726 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.726 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.727 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.727 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.727 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.727 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.727 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.727 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.727 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.727 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.727 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-17T13:07:04.728 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-17T13:07:04.728 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.728 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.728 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.728 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.729 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.729 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.729 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.729 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.729 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.729 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.729 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.729 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.730 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-17T13:07:04.730 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-17T13:07:04.730 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.730 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.730 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.730 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.730 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.730 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.730 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.731 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.731 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.731 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.731 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.731 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.731 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-17T13:07:04.731 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-17T13:07:04.731 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.732 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.732 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.732 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.732 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.732 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.732 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.732 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.732 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.733 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.733 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.733 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.733 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-17T13:07:04.733 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-17T13:07:04.733 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.733 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.733 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.734 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.734 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.734 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.734 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.734 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.734 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.734 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.734 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.735 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.735 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_ENABLED": { 2024-02-17T13:07:04.735 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-17T13:07:04.735 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.735 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.735 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.735 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.735 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.736 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.736 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.736 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.736 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.736 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.736 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.736 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.736 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-17T13:07:04.737 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-17T13:07:04.737 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.737 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.737 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.737 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-17T13:07:04.737 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.737 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.738 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.738 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.738 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.738 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.738 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.738 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.738 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-17T13:07:04.738 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-17T13:07:04.738 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.739 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.739 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.739 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:04.739 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.739 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.739 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.739 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.739 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.740 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.740 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.740 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.740 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-17T13:07:04.740 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-17T13:07:04.740 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.740 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.740 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.741 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8", 2024-02-17T13:07:04.741 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.741 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.741 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.741 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.741 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.741 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.742 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.742 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.742 INFO:teuthology.orchestra.run.smithi018.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-17T13:07:04.742 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-17T13:07:04.742 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.742 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.742 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.742 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "45", 2024-02-17T13:07:04.743 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.743 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.743 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.743 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.743 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.743 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.743 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.743 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.743 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_ACCESS_KEY": { 2024-02-17T13:07:04.744 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-17T13:07:04.744 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.744 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.744 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.744 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.744 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.744 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.745 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.745 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.745 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.745 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.745 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.745 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.745 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-17T13:07:04.745 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-17T13:07:04.746 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.746 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.746 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.746 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-02-17T13:07:04.746 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.746 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.746 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.746 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.746 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.747 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.747 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.747 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.747 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_SECRET_KEY": { 2024-02-17T13:07:04.747 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-17T13:07:04.747 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.747 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.747 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.748 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.748 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.748 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.748 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.748 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.748 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.748 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.748 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.749 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.749 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_SSL_VERIFY": { 2024-02-17T13:07:04.749 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-17T13:07:04.749 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.749 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.749 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.749 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.749 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.749 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.750 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.750 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.750 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.750 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.750 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.750 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.750 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-17T13:07:04.750 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-17T13:07:04.751 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.751 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.751 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.751 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:04.751 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.751 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.751 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.751 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.751 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.752 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.752 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.752 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.752 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-17T13:07:04.752 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-17T13:07:04.752 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.752 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.752 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.753 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:04.753 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.753 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.753 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.753 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.753 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.753 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.753 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.753 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.754 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-17T13:07:04.754 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-17T13:07:04.754 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.754 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.754 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.754 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-02-17T13:07:04.754 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.754 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.755 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.755 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.755 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.755 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.755 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.755 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.755 INFO:teuthology.orchestra.run.smithi018.stdout: "cross_origin_url": { 2024-02-17T13:07:04.755 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cross_origin_url", 2024-02-17T13:07:04.756 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.756 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.756 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.756 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.756 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.756 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.756 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.756 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "crt_file": { 2024-02-17T13:07:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crt_file", 2024-02-17T13:07:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.759 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.759 INFO:teuthology.orchestra.run.smithi018.stdout: "debug": { 2024-02-17T13:07:04.759 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "debug", 2024-02-17T13:07:04.759 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.759 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.759 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.759 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.759 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.760 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.760 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.760 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable/disable debug options", 2024-02-17T13:07:04.760 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.760 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.760 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.760 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.760 INFO:teuthology.orchestra.run.smithi018.stdout: "jwt_token_ttl": { 2024-02-17T13:07:04.761 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "jwt_token_ttl", 2024-02-17T13:07:04.761 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.761 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.761 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.761 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "28800", 2024-02-17T13:07:04.761 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.761 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.762 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.762 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.762 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.762 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.762 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.762 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.762 INFO:teuthology.orchestra.run.smithi018.stdout: "key_file": { 2024-02-17T13:07:04.762 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "key_file", 2024-02-17T13:07:04.763 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.763 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.763 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.763 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.763 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.763 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.763 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.763 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.763 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.764 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.764 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.764 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.764 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.764 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.764 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.764 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.764 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.765 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.765 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.765 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.765 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.765 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.765 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.765 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.765 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.766 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.766 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.766 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.766 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.766 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.766 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.766 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.766 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.766 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.767 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.767 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.767 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.767 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.767 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.767 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.767 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.768 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.768 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.768 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.768 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.768 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.768 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.768 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.768 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.769 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.769 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.769 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.769 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.769 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.769 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.769 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.770 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.770 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.770 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.770 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.770 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.770 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.770 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.770 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.771 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.771 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.771 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.771 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.771 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.771 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.771 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.771 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.772 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.772 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.772 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.772 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.772 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.772 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.772 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.773 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.773 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.773 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.773 INFO:teuthology.orchestra.run.smithi018.stdout: "motd": { 2024-02-17T13:07:04.773 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "motd", 2024-02-17T13:07:04.773 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.773 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.773 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.774 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.774 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.774 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.774 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.774 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "The message of the day", 2024-02-17T13:07:04.774 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.774 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.774 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.774 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.775 INFO:teuthology.orchestra.run.smithi018.stdout: "redirect_resolve_ip_addr": { 2024-02-17T13:07:04.775 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "redirect_resolve_ip_addr", 2024-02-17T13:07:04.775 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.775 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.775 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.775 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.775 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.776 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.776 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.776 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.776 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.776 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.776 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.776 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.776 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-02-17T13:07:04.777 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-02-17T13:07:04.777 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.777 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.777 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.777 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "::", 2024-02-17T13:07:04.777 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.777 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.777 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.777 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.778 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.778 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.778 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.778 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.778 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-02-17T13:07:04.778 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-02-17T13:07:04.778 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.778 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.779 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.779 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8080", 2024-02-17T13:07:04.779 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.779 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.779 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.779 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.779 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.779 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.779 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.780 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.780 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl": { 2024-02-17T13:07:04.780 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl", 2024-02-17T13:07:04.780 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.780 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.780 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.780 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.781 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.781 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.781 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.781 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.781 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.781 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.781 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.781 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.782 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl_server_port": { 2024-02-17T13:07:04.782 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl_server_port", 2024-02-17T13:07:04.782 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.782 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.782 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.782 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8443", 2024-02-17T13:07:04.782 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.782 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.782 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.783 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.783 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.783 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.783 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.783 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.783 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_behaviour": { 2024-02-17T13:07:04.783 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_behaviour", 2024-02-17T13:07:04.783 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.784 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.784 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.784 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "redirect", 2024-02-17T13:07:04.784 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.784 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.784 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.784 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.784 INFO:teuthology.orchestra.run.smithi018.stdout: "redirect" 2024-02-17T13:07:04.785 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.785 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.785 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.785 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.785 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.785 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_error_status_code": { 2024-02-17T13:07:04.785 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_error_status_code", 2024-02-17T13:07:04.786 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.786 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.786 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.786 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "500", 2024-02-17T13:07:04.786 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "400", 2024-02-17T13:07:04.786 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "599", 2024-02-17T13:07:04.786 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.786 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.787 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.787 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.787 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.787 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.787 INFO:teuthology.orchestra.run.smithi018.stdout: "url_prefix": { 2024-02-17T13:07:04.787 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "url_prefix", 2024-02-17T13:07:04.787 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.787 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.787 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.788 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.788 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.788 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.788 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.788 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.788 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.788 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.788 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.789 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.789 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.789 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.789 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.789 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "devicehealth", 2024-02-17T13:07:04.789 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.789 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.789 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.790 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_monitoring": { 2024-02-17T13:07:04.790 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enable_monitoring", 2024-02-17T13:07:04.790 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.790 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.790 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.790 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.790 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.790 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.791 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.791 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "monitor device health metrics", 2024-02-17T13:07:04.791 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.791 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.791 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.791 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.791 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.791 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.791 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.792 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.792 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.792 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.792 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.792 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.792 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.792 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.793 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.793 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.793 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.793 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.793 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.793 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.793 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.793 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.793 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.794 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.794 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.794 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.794 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.794 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.794 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.794 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.794 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.795 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.795 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.795 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.795 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.795 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.795 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.795 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.795 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.796 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.796 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.796 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.796 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.796 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.796 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.796 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.796 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.797 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.797 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.797 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.797 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.797 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.797 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.797 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.797 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.798 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.798 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.798 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.798 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.798 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.798 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.798 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.798 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.799 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.799 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.799 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.799 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.799 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.799 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.799 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.799 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.800 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.800 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.800 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.800 INFO:teuthology.orchestra.run.smithi018.stdout: "mark_out_threshold": { 2024-02-17T13:07:04.800 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mark_out_threshold", 2024-02-17T13:07:04.800 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.800 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.800 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.801 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "2419200", 2024-02-17T13:07:04.801 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.801 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.801 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.801 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-17T13:07:04.801 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.801 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.801 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.802 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.802 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": { 2024-02-17T13:07:04.802 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pool_name", 2024-02-17T13:07:04.802 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.802 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.802 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.802 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "device_health_metrics", 2024-02-17T13:07:04.802 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.802 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.803 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.803 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-17T13:07:04.803 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.803 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.803 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.803 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.803 INFO:teuthology.orchestra.run.smithi018.stdout: "retention_period": { 2024-02-17T13:07:04.803 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "retention_period", 2024-02-17T13:07:04.804 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.804 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.804 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.804 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15552000", 2024-02-17T13:07:04.804 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.804 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.804 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.804 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long to retain device health metrics", 2024-02-17T13:07:04.805 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.805 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.805 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.805 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.805 INFO:teuthology.orchestra.run.smithi018.stdout: "scrape_frequency": { 2024-02-17T13:07:04.805 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "scrape_frequency", 2024-02-17T13:07:04.805 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.805 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.805 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.806 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "86400", 2024-02-17T13:07:04.806 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.806 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.806 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.806 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-17T13:07:04.806 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.806 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.806 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.807 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.807 INFO:teuthology.orchestra.run.smithi018.stdout: "self_heal": { 2024-02-17T13:07:04.807 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "self_heal", 2024-02-17T13:07:04.807 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.807 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.807 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.807 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.807 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.808 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.808 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.808 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-17T13:07:04.808 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.808 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.808 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.808 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.808 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:04.809 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:04.809 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.809 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.809 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.809 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:04.809 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.809 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.809 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.809 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to wake up and check device health", 2024-02-17T13:07:04.810 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.810 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.810 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.810 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.810 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_threshold": { 2024-02-17T13:07:04.810 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_threshold", 2024-02-17T13:07:04.810 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.810 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.811 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.811 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "7257600", 2024-02-17T13:07:04.811 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.811 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.811 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.811 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-17T13:07:04.811 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.811 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.811 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.812 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.812 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.812 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.812 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.812 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "diskprediction_local", 2024-02-17T13:07:04.812 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.812 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.813 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.813 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.813 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.813 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.813 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.813 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.813 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.813 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.813 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.814 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.814 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.814 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.814 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.814 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.814 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.814 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.814 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.815 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.815 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.815 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.815 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.815 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.815 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.815 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.816 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.816 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.816 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.816 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.816 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.816 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.816 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.816 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.817 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.817 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.817 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.817 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.817 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.817 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.817 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.817 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.818 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.818 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.818 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.818 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.818 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.818 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.818 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.818 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.819 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.819 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.819 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.819 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.819 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.819 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.819 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.819 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.820 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.820 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.820 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.820 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.820 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.820 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.820 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.821 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.821 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.821 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.821 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.821 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.821 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.821 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.821 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.821 INFO:teuthology.orchestra.run.smithi018.stdout: "predict_interval": { 2024-02-17T13:07:04.822 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "predict_interval", 2024-02-17T13:07:04.822 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.822 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.822 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.822 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "86400", 2024-02-17T13:07:04.822 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.822 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.822 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.823 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.823 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.823 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.823 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.823 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.823 INFO:teuthology.orchestra.run.smithi018.stdout: "predictor_model": { 2024-02-17T13:07:04.823 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "predictor_model", 2024-02-17T13:07:04.823 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.824 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.824 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.824 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "prophetstor", 2024-02-17T13:07:04.824 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.824 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.824 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.824 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.824 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.825 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.825 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.825 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.825 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:04.825 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:04.825 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.825 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.825 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.826 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:04.826 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.826 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.826 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.826 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.826 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.826 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.826 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.826 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.827 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.827 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.827 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.827 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "influx", 2024-02-17T13:07:04.827 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": false, 2024-02-17T13:07:04.827 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "influxdb python module not found", 2024-02-17T13:07:04.827 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.827 INFO:teuthology.orchestra.run.smithi018.stdout: "batch_size": { 2024-02-17T13:07:04.828 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "batch_size", 2024-02-17T13:07:04.828 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.828 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.828 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.828 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5000", 2024-02-17T13:07:04.828 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.828 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.828 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.829 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-17T13:07:04.829 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.829 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.829 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.829 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.829 INFO:teuthology.orchestra.run.smithi018.stdout: "database": { 2024-02-17T13:07:04.829 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "database", 2024-02-17T13:07:04.829 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.830 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.830 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.830 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "ceph", 2024-02-17T13:07:04.830 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.830 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.830 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.830 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-17T13:07:04.831 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.831 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.831 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.831 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.831 INFO:teuthology.orchestra.run.smithi018.stdout: "hostname": { 2024-02-17T13:07:04.831 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "hostname", 2024-02-17T13:07:04.831 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.831 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.832 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.832 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.832 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.832 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.832 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.832 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB server hostname", 2024-02-17T13:07:04.832 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.833 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.833 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.833 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.833 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:04.833 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:04.833 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.833 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.833 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.834 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "30", 2024-02-17T13:07:04.834 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "5", 2024-02-17T13:07:04.834 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.834 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.834 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-17T13:07:04.834 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.834 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.835 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.835 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.835 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.835 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.835 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.835 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.835 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.835 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.836 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.836 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.836 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.836 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.836 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.836 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.836 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.836 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.837 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.837 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.837 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.837 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.837 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.837 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.837 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.838 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.838 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.838 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.838 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.838 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.838 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.838 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.838 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.839 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.839 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.839 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.839 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.839 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.839 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.839 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.840 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.840 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.840 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.840 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.840 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.840 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.840 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.840 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.841 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.841 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.841 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.841 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.841 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.841 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.841 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.841 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.842 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.842 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.842 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.842 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.842 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.842 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.842 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.842 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.843 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.843 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.843 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.843 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.843 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.843 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.843 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.843 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.843 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.844 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.844 INFO:teuthology.orchestra.run.smithi018.stdout: "password": { 2024-02-17T13:07:04.844 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "password", 2024-02-17T13:07:04.844 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.844 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.844 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.844 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.844 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.845 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.845 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.845 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "password of InfluxDB server user", 2024-02-17T13:07:04.845 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.845 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.845 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.845 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.845 INFO:teuthology.orchestra.run.smithi018.stdout: "port": { 2024-02-17T13:07:04.846 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "port", 2024-02-17T13:07:04.846 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.846 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.846 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.846 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8086", 2024-02-17T13:07:04.846 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.846 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.846 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.847 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB server port", 2024-02-17T13:07:04.847 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.847 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.847 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.847 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.847 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl": { 2024-02-17T13:07:04.847 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl", 2024-02-17T13:07:04.847 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.847 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.848 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.848 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "false", 2024-02-17T13:07:04.848 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.848 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.848 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.848 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-17T13:07:04.848 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.848 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.849 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.849 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.849 INFO:teuthology.orchestra.run.smithi018.stdout: "threads": { 2024-02-17T13:07:04.849 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "threads", 2024-02-17T13:07:04.849 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.849 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.849 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.849 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-02-17T13:07:04.850 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-02-17T13:07:04.850 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "32", 2024-02-17T13:07:04.850 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.850 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-17T13:07:04.850 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.850 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.850 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.850 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.851 INFO:teuthology.orchestra.run.smithi018.stdout: "username": { 2024-02-17T13:07:04.851 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "username", 2024-02-17T13:07:04.851 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.851 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.851 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.851 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.851 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.851 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.851 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.852 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "username of InfluxDB server user", 2024-02-17T13:07:04.852 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.852 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.852 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.852 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.852 INFO:teuthology.orchestra.run.smithi018.stdout: "verify_ssl": { 2024-02-17T13:07:04.852 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "verify_ssl", 2024-02-17T13:07:04.853 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.853 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.853 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.853 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "true", 2024-02-17T13:07:04.853 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.853 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.853 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.853 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-17T13:07:04.853 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.854 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.854 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.854 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.854 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.854 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.854 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.854 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "insights", 2024-02-17T13:07:04.854 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.855 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.855 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.855 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.855 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.855 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.855 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.855 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.855 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.855 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.856 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.856 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.856 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.856 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.856 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.856 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.856 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.857 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.857 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.857 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.857 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.857 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.857 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.857 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.857 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.857 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.859 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.859 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.859 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.859 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.859 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.859 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.860 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.860 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.860 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.860 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.860 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.860 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.860 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.860 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.861 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.861 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.861 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.861 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.861 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.861 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.861 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.862 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.862 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.862 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.862 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.862 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.862 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.862 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.862 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.862 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.863 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.863 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.863 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.863 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.863 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.863 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.863 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.864 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.864 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.864 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.864 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.864 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.864 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.864 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.865 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.865 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.865 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.865 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.865 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.865 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.865 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "iostat", 2024-02-17T13:07:04.865 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.865 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.866 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.866 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.866 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.866 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.866 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.866 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.866 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.866 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.867 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.867 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.867 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.867 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.867 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.867 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.867 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.867 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.868 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.868 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.868 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.868 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.868 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.868 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.868 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.868 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.869 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.869 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.869 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.869 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.869 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.869 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.869 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.869 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.869 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.870 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.870 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.870 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.870 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.870 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.870 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.870 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.870 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.871 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.871 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.871 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.871 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.871 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.871 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.871 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.872 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.872 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.872 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.872 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.872 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.872 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.872 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.872 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.873 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.873 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.873 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.873 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.873 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.873 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.873 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.873 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.873 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.874 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.874 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.874 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.874 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.874 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.874 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.874 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.874 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.875 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.875 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "k8sevents", 2024-02-17T13:07:04.875 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.875 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.875 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.875 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_event_retention_days": { 2024-02-17T13:07:04.875 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ceph_event_retention_days", 2024-02-17T13:07:04.875 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.876 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.876 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.876 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "7", 2024-02-17T13:07:04.876 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.876 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.876 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.876 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-17T13:07:04.876 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.877 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.877 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.877 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.877 INFO:teuthology.orchestra.run.smithi018.stdout: "config_check_secs": { 2024-02-17T13:07:04.877 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_check_secs", 2024-02-17T13:07:04.877 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.877 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.877 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.878 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:04.878 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "10", 2024-02-17T13:07:04.878 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.878 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.878 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-17T13:07:04.878 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.878 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.878 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.878 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.879 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.879 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.879 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.879 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.879 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.879 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.879 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.879 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.880 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.880 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.880 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.880 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.880 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.880 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.880 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.881 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.881 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.881 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.881 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.881 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.881 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.881 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.881 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.882 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.882 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.882 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.882 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.882 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.882 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.882 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.882 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.882 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.883 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.883 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.883 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.883 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.883 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.883 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.883 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.883 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.884 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.884 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.884 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.884 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.884 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.884 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.884 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.884 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.885 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.885 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.885 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.885 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.885 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.885 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.885 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.885 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.885 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.886 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.886 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.886 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.886 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.886 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.886 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.886 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.886 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.887 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.887 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.887 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.887 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.887 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.887 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.887 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.887 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.888 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "localpool", 2024-02-17T13:07:04.888 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.888 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.888 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.888 INFO:teuthology.orchestra.run.smithi018.stdout: "failure_domain": { 2024-02-17T13:07:04.888 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "failure_domain", 2024-02-17T13:07:04.888 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.888 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.889 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.889 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "host", 2024-02-17T13:07:04.889 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.889 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.889 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.889 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "failure domain for any created local pool", 2024-02-17T13:07:04.889 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-17T13:07:04.889 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.890 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.890 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.890 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.890 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.890 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.890 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.890 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.890 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.891 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.891 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.891 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.891 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.891 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.891 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.891 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.891 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.892 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.892 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.892 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.892 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.892 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.892 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.892 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.892 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.892 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.893 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.893 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.893 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.893 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.893 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.893 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.893 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.893 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.894 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.894 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.894 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.894 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.894 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.894 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.894 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.894 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.894 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.895 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.895 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.895 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.895 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.895 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.895 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.895 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.895 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.896 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.896 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.896 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.896 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.896 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.896 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.896 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.896 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.897 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.897 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.897 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.897 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.897 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.897 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.897 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.897 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.897 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.898 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.898 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.898 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.898 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.898 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.898 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": { 2024-02-17T13:07:04.898 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "min_size", 2024-02-17T13:07:04.898 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.899 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.899 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.899 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.899 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.899 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.899 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.899 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default min_size for any created local pool", 2024-02-17T13:07:04.899 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-17T13:07:04.899 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.900 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.900 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.900 INFO:teuthology.orchestra.run.smithi018.stdout: "num_rep": { 2024-02-17T13:07:04.900 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "num_rep", 2024-02-17T13:07:04.900 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.900 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.900 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.900 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3", 2024-02-17T13:07:04.901 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.901 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.901 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.901 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default replica count for any created local pool", 2024-02-17T13:07:04.901 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.901 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.901 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.901 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.902 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": { 2024-02-17T13:07:04.902 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pg_num", 2024-02-17T13:07:04.902 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.902 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.902 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.902 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "128", 2024-02-17T13:07:04.902 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.902 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.903 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.903 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default pg_num for any created local pool", 2024-02-17T13:07:04.903 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.903 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.903 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.903 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.903 INFO:teuthology.orchestra.run.smithi018.stdout: "prefix": { 2024-02-17T13:07:04.903 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prefix", 2024-02-17T13:07:04.903 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.904 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.904 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.904 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.904 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.904 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.904 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.904 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "name prefix for any created local pool", 2024-02-17T13:07:04.905 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.905 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.905 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.905 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.905 INFO:teuthology.orchestra.run.smithi018.stdout: "subtree": { 2024-02-17T13:07:04.905 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "subtree", 2024-02-17T13:07:04.905 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.905 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.906 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.906 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "rack", 2024-02-17T13:07:04.906 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.906 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.906 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.906 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-17T13:07:04.906 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-17T13:07:04.906 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.907 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.907 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.907 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.907 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.907 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.907 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mds_autoscaler", 2024-02-17T13:07:04.907 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.907 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.908 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.908 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.908 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.908 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.908 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.908 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.908 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.908 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.909 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.909 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.909 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.909 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.909 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.909 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.909 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.909 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.909 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.910 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.910 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.910 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.910 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.910 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.910 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.911 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.911 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.911 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.911 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.911 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.911 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.911 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.911 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.912 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.912 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.912 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.912 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.912 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.912 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.912 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.913 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.913 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.913 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.913 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.913 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.913 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.913 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.913 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.914 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.914 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.914 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.914 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.914 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.914 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.914 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.914 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.915 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.915 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.915 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.915 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.915 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.915 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.915 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.915 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.915 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.916 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.916 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.916 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.916 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.916 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.916 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.916 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.916 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.917 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.917 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.917 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.917 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mirroring", 2024-02-17T13:07:04.917 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.917 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.917 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.917 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.918 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.918 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.918 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.918 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.918 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.918 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.918 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.918 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.918 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.919 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.919 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.919 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.919 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.919 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.919 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.919 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.919 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.920 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.920 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.920 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.920 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.920 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.920 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.920 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.920 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.921 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.921 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.921 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.921 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.921 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.921 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.921 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.921 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.922 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.922 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.922 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.922 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.922 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.922 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.922 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.922 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.922 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.923 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.923 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.923 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.923 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.923 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.923 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.923 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.923 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.924 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.924 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.924 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.924 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.924 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.924 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.924 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.924 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.925 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.925 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.925 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.925 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.925 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.925 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.925 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.925 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.925 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.926 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.926 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.926 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.926 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.926 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.926 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "nfs", 2024-02-17T13:07:04.926 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.926 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.927 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.927 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.927 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.927 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.927 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.927 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.927 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.927 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.928 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.928 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.928 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.928 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.928 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.928 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.928 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.928 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.929 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.929 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.929 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.929 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.929 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.929 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.929 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.929 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.930 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.930 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.930 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.930 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.930 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.930 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.930 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.930 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.931 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.931 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.931 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.931 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.931 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.931 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.931 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.932 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.932 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.932 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.932 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.932 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.932 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.932 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.932 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.933 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.933 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.933 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.933 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.933 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.933 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.933 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.933 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.934 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.934 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.934 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.934 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.934 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.934 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.934 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.935 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.935 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.935 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.935 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.935 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.935 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.935 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.935 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.936 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.936 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.936 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.936 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.936 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "orchestrator", 2024-02-17T13:07:04.936 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.936 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.937 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.937 INFO:teuthology.orchestra.run.smithi018.stdout: "fail_fs": { 2024-02-17T13:07:04.937 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "fail_fs", 2024-02-17T13:07:04.937 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.937 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.937 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.937 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.937 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.937 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.938 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.938 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-17T13:07:04.938 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.938 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.938 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.938 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.938 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.939 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.939 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.939 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.939 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.939 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.939 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.939 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.939 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.939 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.940 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.940 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.940 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.940 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.940 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.940 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.940 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.940 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.941 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.941 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.941 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.941 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.941 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.941 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.941 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.941 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.942 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.942 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.942 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.942 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.942 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.942 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.942 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.942 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.943 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.943 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.943 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.943 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.943 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.943 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.943 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.943 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.943 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.943 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.944 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.944 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.944 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.944 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.944 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.944 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.944 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.944 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.945 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.945 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.945 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.945 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.945 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.945 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.945 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.945 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator": { 2024-02-17T13:07:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "orchestrator", 2024-02-17T13:07:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.948 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.948 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.948 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm", 2024-02-17T13:07:04.948 INFO:teuthology.orchestra.run.smithi018.stdout: "rook", 2024-02-17T13:07:04.948 INFO:teuthology.orchestra.run.smithi018.stdout: "test_orchestrator" 2024-02-17T13:07:04.948 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.948 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Orchestrator backend", 2024-02-17T13:07:04.948 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.949 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.949 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.949 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.949 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.949 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.949 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "osd_perf_query", 2024-02-17T13:07:04.949 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.950 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.950 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.950 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.950 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.950 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.950 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.950 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.950 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.951 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.951 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.951 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.951 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.951 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.951 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.951 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.951 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.951 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.952 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.952 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.952 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.952 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.952 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.952 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.952 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.953 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.953 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.953 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.953 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.953 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.953 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.953 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.953 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.953 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.954 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.954 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.954 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.954 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.954 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.954 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.954 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.954 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.955 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.955 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.955 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.955 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.955 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.955 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.955 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.955 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.955 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.956 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.956 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.956 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.956 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.956 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.956 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.956 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.957 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.957 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.957 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.957 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.957 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.957 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.957 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.957 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.957 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.958 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.958 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.958 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.958 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.958 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.958 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.958 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.958 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "osd_support", 2024-02-17T13:07:04.959 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.959 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.959 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.959 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.959 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.959 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.959 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.959 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.960 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.960 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.960 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.960 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.960 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.960 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.960 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.960 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.961 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.961 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.961 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.961 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.961 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.961 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.961 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.962 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.962 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.962 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.962 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.962 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.962 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.962 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.962 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.963 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.963 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.963 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.963 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.963 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.963 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.963 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.963 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.963 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.964 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.964 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.964 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.964 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.964 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.964 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.964 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.965 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.965 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.965 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.965 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.965 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.965 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.965 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.965 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.966 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.966 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.966 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.966 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.966 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.966 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.966 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.966 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.966 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.967 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.967 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.967 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.967 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.967 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.967 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.967 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.967 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.968 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.968 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.968 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.968 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pg_autoscaler", 2024-02-17T13:07:04.968 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.968 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.968 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.968 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.969 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.969 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.969 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.969 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.969 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.969 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.969 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.970 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.970 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.970 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.970 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.970 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.970 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.970 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.970 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.971 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.971 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.971 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.971 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.971 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.971 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.971 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.971 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.972 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.972 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.972 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.972 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.972 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.972 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.972 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.972 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.973 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.973 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.973 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.973 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.973 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.973 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.973 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.973 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.974 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.974 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.974 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.974 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.974 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.974 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.974 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.974 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.975 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.975 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.975 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.975 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.975 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.975 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.975 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.975 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.976 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.976 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.976 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.976 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.976 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.976 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.976 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.976 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.977 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.977 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.977 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.977 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.977 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.977 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.977 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:04.977 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:04.977 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.978 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.978 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.978 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:04.978 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.978 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.978 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.978 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.978 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.978 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.979 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.979 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.979 INFO:teuthology.orchestra.run.smithi018.stdout: "threshold": { 2024-02-17T13:07:04.979 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "threshold", 2024-02-17T13:07:04.979 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:04.979 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.979 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.979 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3.0", 2024-02-17T13:07:04.980 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1.0", 2024-02-17T13:07:04.980 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.980 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.980 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "scaling threshold", 2024-02-17T13:07:04.980 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-17T13:07:04.980 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.980 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.980 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.981 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.981 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.981 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.981 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "progress", 2024-02-17T13:07:04.981 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.981 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.981 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.981 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_pg_recovery_event": { 2024-02-17T13:07:04.982 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_pg_recovery_event", 2024-02-17T13:07:04.982 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.982 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.982 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.982 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.982 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.982 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.982 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.983 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-17T13:07:04.983 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.983 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.983 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.983 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.983 INFO:teuthology.orchestra.run.smithi018.stdout: "enabled": { 2024-02-17T13:07:04.983 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enabled", 2024-02-17T13:07:04.983 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.984 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.984 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.984 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.984 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.984 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.984 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.984 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.984 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.985 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.985 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.985 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.985 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:04.985 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:04.985 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.985 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.985 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.986 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:04.986 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.986 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.986 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.986 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.986 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.986 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.986 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.986 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.987 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.987 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.987 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.987 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.987 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.987 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.987 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.987 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:04.988 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:04.988 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.988 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.988 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.988 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.988 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.988 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.988 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.989 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.989 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.989 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.989 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.989 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.989 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:04.989 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:04.989 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:04.990 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.990 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.990 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:04.990 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.990 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.990 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:04.990 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:04.990 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:04.990 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:04.991 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:04.991 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:04.991 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:04.991 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:04.991 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.991 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.991 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.991 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.992 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.992 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:04.992 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:04.992 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.992 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.992 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.992 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:04.992 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.993 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.993 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.993 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.993 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.993 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.993 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.993 INFO:teuthology.orchestra.run.smithi018.stdout: "max_completed_events": { 2024-02-17T13:07:04.994 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_completed_events", 2024-02-17T13:07:04.994 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:04.994 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.994 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.994 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "50", 2024-02-17T13:07:04.994 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.994 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.994 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.994 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "number of past completed events to remember", 2024-02-17T13:07:04.995 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.995 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.995 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.995 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.995 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:04.995 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:04.995 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:04.995 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.996 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:04.996 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-02-17T13:07:04.996 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.996 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.996 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.996 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long the module is going to sleep", 2024-02-17T13:07:04.996 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.996 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.997 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.997 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.997 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:04.997 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.997 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:04.997 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prometheus", 2024-02-17T13:07:04.997 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:04.997 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:04.998 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:04.998 INFO:teuthology.orchestra.run.smithi018.stdout: "cache": { 2024-02-17T13:07:04.998 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cache", 2024-02-17T13:07:04.998 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:04.998 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:04.998 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:04.998 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:04.998 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:04.999 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:04.999 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:04.999 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:04.999 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:04.999 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:04.999 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:04.999 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:04.999 INFO:teuthology.orchestra.run.smithi018.stdout: "exclude_perf_counters": { 2024-02-17T13:07:05.000 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "exclude_perf_counters", 2024-02-17T13:07:05.000 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.000 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.000 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.000 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.000 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.000 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.000 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.001 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-17T13:07:05.001 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-17T13:07:05.001 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.001 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.001 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.001 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.001 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.002 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.002 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.002 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.002 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.002 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.002 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.002 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.002 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.003 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.003 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.003 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.003 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.003 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.003 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.003 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.003 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.004 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.004 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.004 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.004 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.004 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.004 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.004 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.004 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.005 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.005 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.005 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.005 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.005 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.005 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.005 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.005 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.006 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.006 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.006 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.006 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.006 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.006 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.006 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.007 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.007 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.007 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.007 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.007 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.007 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.007 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.007 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.008 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.008 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.008 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.008 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.008 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.008 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.008 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.008 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.009 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.009 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.009 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.009 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.009 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.009 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.009 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.009 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.010 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.010 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.010 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.010 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_stats_pools": { 2024-02-17T13:07:05.010 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_stats_pools", 2024-02-17T13:07:05.010 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.010 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.011 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.011 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.011 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.011 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.011 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.011 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.011 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.012 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.012 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.012 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-17T13:07:05.012 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-17T13:07:05.012 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.012 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.012 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.012 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "300", 2024-02-17T13:07:05.013 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.013 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.013 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.013 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.013 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.013 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.013 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.014 INFO:teuthology.orchestra.run.smithi018.stdout: "scrape_interval": { 2024-02-17T13:07:05.014 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "scrape_interval", 2024-02-17T13:07:05.014 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:05.014 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.014 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.014 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15.0", 2024-02-17T13:07:05.014 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.014 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.015 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.015 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.015 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.015 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.015 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.015 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.015 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-02-17T13:07:05.015 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-02-17T13:07:05.016 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.016 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.016 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.016 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "::", 2024-02-17T13:07:05.016 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.016 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.016 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.016 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-17T13:07:05.017 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.017 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.017 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.017 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.017 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-02-17T13:07:05.017 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-02-17T13:07:05.017 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.017 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.017 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.018 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "9283", 2024-02-17T13:07:05.018 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.018 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.018 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.018 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-17T13:07:05.018 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.018 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.018 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.019 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.019 INFO:teuthology.orchestra.run.smithi018.stdout: "stale_cache_strategy": { 2024-02-17T13:07:05.019 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "stale_cache_strategy", 2024-02-17T13:07:05.019 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.019 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.019 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.019 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "log", 2024-02-17T13:07:05.019 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.020 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.020 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.020 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.020 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.020 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.020 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.020 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.020 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_behaviour": { 2024-02-17T13:07:05.021 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_behaviour", 2024-02-17T13:07:05.021 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.021 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.021 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.021 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "default", 2024-02-17T13:07:05.021 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.021 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.021 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.022 INFO:teuthology.orchestra.run.smithi018.stdout: "default", 2024-02-17T13:07:05.022 INFO:teuthology.orchestra.run.smithi018.stdout: "error" 2024-02-17T13:07:05.022 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.022 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.022 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.022 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.022 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.022 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.023 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_error_status_code": { 2024-02-17T13:07:05.023 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_error_status_code", 2024-02-17T13:07:05.023 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.023 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.023 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.023 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "500", 2024-02-17T13:07:05.023 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "400", 2024-02-17T13:07:05.023 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "599", 2024-02-17T13:07:05.023 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.024 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.024 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.024 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.024 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.024 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.024 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.024 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.025 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.025 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_support", 2024-02-17T13:07:05.025 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.025 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.025 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.025 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.025 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.025 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.026 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.026 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.026 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.026 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.026 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.026 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.026 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.026 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.026 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.027 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.027 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.027 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.027 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.027 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.027 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.027 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.028 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.028 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.028 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.028 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.028 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.028 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.028 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.029 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.029 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.029 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.029 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.029 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.029 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.029 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.029 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.030 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.030 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.030 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.030 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.030 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.030 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.030 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.031 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.031 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.031 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.031 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.031 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.031 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.031 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.031 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.031 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.032 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.032 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.032 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.032 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.032 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.032 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.032 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.032 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.033 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.033 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.033 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.033 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.033 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.033 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.033 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.033 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.034 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.034 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.034 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.034 INFO:teuthology.orchestra.run.smithi018.stdout: "max_concurrent_snap_create": { 2024-02-17T13:07:05.034 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_concurrent_snap_create", 2024-02-17T13:07:05.034 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.034 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.034 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.035 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:05.035 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.035 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.035 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.035 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.035 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.035 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.035 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.036 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.036 INFO:teuthology.orchestra.run.smithi018.stdout: "mirror_snapshot_schedule": { 2024-02-17T13:07:05.036 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mirror_snapshot_schedule", 2024-02-17T13:07:05.036 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.036 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.036 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.036 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.036 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.037 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.037 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.037 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.037 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.037 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.037 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.037 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.037 INFO:teuthology.orchestra.run.smithi018.stdout: "trash_purge_schedule": { 2024-02-17T13:07:05.038 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "trash_purge_schedule", 2024-02-17T13:07:05.038 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.038 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.038 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.038 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.038 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.038 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.038 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.038 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.039 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.039 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.039 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.039 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.039 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.039 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.039 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.039 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "restful", 2024-02-17T13:07:05.040 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.040 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.040 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.040 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_auth": { 2024-02-17T13:07:05.040 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enable_auth", 2024-02-17T13:07:05.040 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.040 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.041 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.041 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.041 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.041 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.041 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.041 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.041 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.041 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.041 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.042 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.042 INFO:teuthology.orchestra.run.smithi018.stdout: "key_file": { 2024-02-17T13:07:05.042 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "key_file", 2024-02-17T13:07:05.042 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.042 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.042 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.042 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.042 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.043 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.043 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.043 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.043 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.043 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.043 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.043 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.043 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.044 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.044 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.044 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.044 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.044 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.044 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.044 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.044 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.045 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.045 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.045 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.045 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.045 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.045 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.045 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.046 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.046 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.046 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.046 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.046 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.046 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.046 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.046 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.047 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.047 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.047 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.047 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.047 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.047 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.047 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.047 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.047 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.048 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.048 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.048 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.048 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.048 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.048 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.048 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.049 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.049 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.049 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.049 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.049 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.049 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.049 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.049 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.049 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.050 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.050 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.050 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.050 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.050 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.050 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.050 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.050 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.051 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.051 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.051 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.051 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.051 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.051 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.051 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.051 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.052 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.052 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.052 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.052 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.052 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.052 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-02-17T13:07:05.052 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-02-17T13:07:05.052 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.052 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.053 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.053 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.053 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.053 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.053 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.053 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.053 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.054 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.054 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.054 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.054 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-02-17T13:07:05.054 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-02-17T13:07:05.054 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.054 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.054 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.054 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.055 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.055 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.055 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.055 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.055 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.055 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.055 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.055 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.056 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.056 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.056 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.056 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rgw", 2024-02-17T13:07:05.056 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.056 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.056 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.056 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.056 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.057 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.057 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.057 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.057 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.057 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.057 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.057 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.057 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.058 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.058 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.058 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.058 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.058 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.058 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.058 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.059 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.059 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.059 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.059 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.059 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.059 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.059 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.059 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.060 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.060 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.060 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.060 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.060 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.061 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.061 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.061 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.061 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.061 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.061 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.061 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.062 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.062 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.062 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.062 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.062 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.062 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.062 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.063 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.063 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.063 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.063 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.063 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.063 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.063 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.064 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.064 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.064 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.064 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.064 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.064 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.064 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.065 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.065 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.065 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.065 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.065 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.065 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.065 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.066 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.066 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.066 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.066 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.066 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.066 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.066 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.066 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.067 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rook", 2024-02-17T13:07:05.067 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.067 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.067 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.067 INFO:teuthology.orchestra.run.smithi018.stdout: "drive_group_interval": { 2024-02-17T13:07:05.067 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "drive_group_interval", 2024-02-17T13:07:05.067 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:05.068 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.068 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.068 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "300.0", 2024-02-17T13:07:05.068 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.068 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.068 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.068 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-17T13:07:05.068 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.069 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.069 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.069 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.069 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.069 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.069 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.069 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.070 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.070 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.070 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.070 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.070 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.070 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.070 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.071 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.071 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.071 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.071 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.071 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.071 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.071 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.072 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.072 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.072 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.072 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.072 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.072 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.072 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.073 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.073 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.073 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.073 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.073 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.073 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.073 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.073 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.074 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.074 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.074 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.074 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.074 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.074 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.074 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.075 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.075 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.075 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.075 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.075 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.075 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.075 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.075 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.076 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.076 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.076 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.076 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.076 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.076 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.077 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.077 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.077 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.077 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.077 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.077 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.077 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.077 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.077 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.078 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.078 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.078 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.078 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.078 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.078 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.079 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.079 INFO:teuthology.orchestra.run.smithi018.stdout: "storage_class": { 2024-02-17T13:07:05.079 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "storage_class", 2024-02-17T13:07:05.079 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.079 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.079 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.079 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "local", 2024-02-17T13:07:05.080 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.080 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.080 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.080 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-17T13:07:05.080 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.080 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.080 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.081 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.081 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.081 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.081 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.081 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "selftest", 2024-02-17T13:07:05.081 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.081 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.081 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.082 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.082 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.082 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.082 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.082 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.082 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.082 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.083 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.083 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.083 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.083 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.083 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.083 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.083 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.084 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.084 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.084 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.084 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.084 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.084 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.084 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.085 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.085 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.085 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.085 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.085 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.085 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.085 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.086 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.086 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.086 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.086 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.086 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.086 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.086 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.087 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.087 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.087 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.087 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.087 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.087 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.087 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.088 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.088 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.088 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.088 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.088 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.088 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.088 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.089 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.089 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.089 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.089 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.089 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.089 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.089 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.089 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.090 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.090 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.090 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.090 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.090 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.090 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.090 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.091 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.091 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.091 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.091 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.091 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.091 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.091 INFO:teuthology.orchestra.run.smithi018.stdout: "roption1": { 2024-02-17T13:07:05.092 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "roption1", 2024-02-17T13:07:05.092 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.092 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.092 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.092 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.092 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.093 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.093 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.093 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.093 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.093 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.093 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.093 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.094 INFO:teuthology.orchestra.run.smithi018.stdout: "roption2": { 2024-02-17T13:07:05.094 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "roption2", 2024-02-17T13:07:05.094 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.094 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.094 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.094 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "xyz", 2024-02-17T13:07:05.095 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.095 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.095 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.095 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.095 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.095 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.096 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.096 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.096 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption1": { 2024-02-17T13:07:05.096 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption1", 2024-02-17T13:07:05.096 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.096 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.096 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.097 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.097 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.097 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.097 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.097 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.097 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.097 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.098 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.098 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.098 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption2": { 2024-02-17T13:07:05.098 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption2", 2024-02-17T13:07:05.098 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.098 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.098 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.099 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.099 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.099 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.099 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.099 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.099 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.099 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.100 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.100 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.100 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption3": { 2024-02-17T13:07:05.100 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption3", 2024-02-17T13:07:05.100 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:05.100 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.100 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.101 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.101 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.101 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.101 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.101 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.101 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.101 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.102 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.102 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.102 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption4": { 2024-02-17T13:07:05.102 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption4", 2024-02-17T13:07:05.102 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.102 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.102 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.102 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.103 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.103 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.103 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.103 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.103 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.103 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.103 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.103 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.103 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption5": { 2024-02-17T13:07:05.104 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption5", 2024-02-17T13:07:05.104 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.104 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.104 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.104 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.104 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.104 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.105 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.105 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.105 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.105 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.105 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.105 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.105 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption6": { 2024-02-17T13:07:05.105 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption6", 2024-02-17T13:07:05.106 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.106 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.106 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.106 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.106 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.106 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.106 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.107 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.107 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.107 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.107 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.107 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.107 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption7": { 2024-02-17T13:07:05.108 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption7", 2024-02-17T13:07:05.108 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.108 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.108 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.108 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.108 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-02-17T13:07:05.109 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "42", 2024-02-17T13:07:05.109 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.109 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.109 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.109 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.109 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.109 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.109 INFO:teuthology.orchestra.run.smithi018.stdout: "testkey": { 2024-02-17T13:07:05.110 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testkey", 2024-02-17T13:07:05.110 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.110 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.110 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.110 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.110 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.110 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.111 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.111 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.111 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.111 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.111 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.111 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.111 INFO:teuthology.orchestra.run.smithi018.stdout: "testlkey": { 2024-02-17T13:07:05.112 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testlkey", 2024-02-17T13:07:05.112 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.112 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.112 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.112 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.112 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.112 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.113 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.113 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.113 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.113 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.113 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.113 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.113 INFO:teuthology.orchestra.run.smithi018.stdout: "testnewline": { 2024-02-17T13:07:05.114 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testnewline", 2024-02-17T13:07:05.114 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.114 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.114 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.114 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.114 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.115 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.115 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.115 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.115 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.115 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.115 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.115 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.116 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.116 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.116 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.116 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snap_schedule", 2024-02-17T13:07:05.116 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.116 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.116 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.117 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_m_granularity": { 2024-02-17T13:07:05.117 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_m_granularity", 2024-02-17T13:07:05.117 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.117 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.117 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.117 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.117 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.118 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.118 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.118 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow minute scheduled snapshots", 2024-02-17T13:07:05.118 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.118 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.118 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.118 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.119 INFO:teuthology.orchestra.run.smithi018.stdout: "dump_on_update": { 2024-02-17T13:07:05.119 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "dump_on_update", 2024-02-17T13:07:05.119 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.119 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.119 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.119 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.119 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.120 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.120 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.120 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "dump database to debug log on update", 2024-02-17T13:07:05.120 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.120 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.120 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.120 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.121 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.121 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.121 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.121 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.121 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.121 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.122 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.122 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.122 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.122 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.122 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.122 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.122 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.123 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.123 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.123 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.123 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.123 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.123 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.123 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.124 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.124 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.124 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.124 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.124 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.124 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.124 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.125 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.125 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.125 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.125 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.125 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.125 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.125 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.126 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.126 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.126 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.126 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.126 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.126 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.127 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.127 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.127 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.127 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.127 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.127 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.128 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.128 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.128 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.128 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.128 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.129 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.129 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.129 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.129 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.129 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.129 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.130 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.130 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.130 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.130 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.130 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.130 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.130 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.131 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.131 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.131 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.131 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.131 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.131 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.131 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.131 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.132 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.132 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "stats", 2024-02-17T13:07:05.132 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.132 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.132 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.132 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.132 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.133 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.133 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.133 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.133 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.133 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.133 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.133 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.133 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.134 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.134 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.134 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.134 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.134 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.134 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.134 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.135 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.135 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.135 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.135 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.135 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.135 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.135 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.136 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.136 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.136 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.136 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.136 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.136 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.136 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.137 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.137 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.137 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.137 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.137 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.137 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.137 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.138 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.138 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.138 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.138 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.138 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.138 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.138 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.138 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.139 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.139 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.139 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.139 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.139 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.139 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.140 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.140 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.140 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.140 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.140 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.141 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.141 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.141 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.141 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.141 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.141 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.142 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.142 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.142 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.142 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.142 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.142 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.143 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.143 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.143 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.143 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.143 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "status", 2024-02-17T13:07:05.143 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.143 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.143 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.144 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.144 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.144 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.144 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.144 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.144 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.144 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.145 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.145 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.145 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.145 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.145 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.145 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.145 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.145 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.146 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.146 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.146 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.146 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.146 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.146 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.146 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.147 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.147 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.147 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.147 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.147 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.147 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.147 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.147 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.148 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.148 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.148 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.148 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.148 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.148 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.148 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.149 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.149 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.149 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.149 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.149 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.149 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.149 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.150 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.150 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.150 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.150 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.150 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.150 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.150 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.150 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.151 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.151 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.151 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.151 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.151 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.151 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.151 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.152 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.152 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.152 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.152 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.152 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.152 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.152 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.153 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.153 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.153 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.153 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.153 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.153 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.154 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.154 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "telegraf", 2024-02-17T13:07:05.154 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.154 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.154 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.154 INFO:teuthology.orchestra.run.smithi018.stdout: "address": { 2024-02-17T13:07:05.154 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "address", 2024-02-17T13:07:05.155 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.155 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.155 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.155 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-17T13:07:05.155 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.155 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.155 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.155 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.156 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.156 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.156 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.156 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.156 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:05.156 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:05.156 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.157 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.157 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.157 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15", 2024-02-17T13:07:05.157 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.157 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.157 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.157 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.157 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.158 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.158 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.158 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.158 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.158 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.158 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.158 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.158 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.159 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.159 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.159 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.159 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.159 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.159 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.159 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.160 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.160 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.160 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.160 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.160 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.160 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.160 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.161 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.161 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.161 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.161 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.161 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.161 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.161 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.161 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.162 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.162 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.162 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.162 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.162 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.162 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.162 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.163 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.163 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.163 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.163 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.163 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.163 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.163 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.164 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.164 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.164 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.164 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.164 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.164 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.164 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.165 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.165 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.165 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.165 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.165 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.165 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.165 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.165 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.166 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.166 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.166 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.166 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.166 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.166 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.166 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.167 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.167 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.167 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.167 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.167 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.167 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.168 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.168 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.168 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.168 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.168 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "telemetry", 2024-02-17T13:07:05.168 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.169 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.169 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.169 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_basic": { 2024-02-17T13:07:05.169 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_basic", 2024-02-17T13:07:05.169 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.169 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.169 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.170 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.170 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.170 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.170 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.170 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-17T13:07:05.170 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.170 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.171 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.171 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.171 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_crash": { 2024-02-17T13:07:05.171 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_crash", 2024-02-17T13:07:05.171 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.171 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.172 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.172 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.172 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.172 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.172 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.172 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-17T13:07:05.173 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.173 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.173 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.173 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.173 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_device": { 2024-02-17T13:07:05.173 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_device", 2024-02-17T13:07:05.173 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.174 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.174 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.174 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.174 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.174 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.174 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.174 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-17T13:07:05.175 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.175 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.175 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.175 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.175 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_ident": { 2024-02-17T13:07:05.175 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_ident", 2024-02-17T13:07:05.175 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.176 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.176 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.176 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.176 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.176 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.176 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.176 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-17T13:07:05.177 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.177 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.177 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.177 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.177 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_perf": { 2024-02-17T13:07:05.177 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_perf", 2024-02-17T13:07:05.177 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.177 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.178 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.178 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.178 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.178 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.178 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.178 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-17T13:07:05.178 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.179 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.179 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.179 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.179 INFO:teuthology.orchestra.run.smithi018.stdout: "contact": { 2024-02-17T13:07:05.179 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "contact", 2024-02-17T13:07:05.179 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.179 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.180 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.180 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.180 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.180 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.180 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.180 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.180 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.181 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.181 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.181 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.181 INFO:teuthology.orchestra.run.smithi018.stdout: "description": { 2024-02-17T13:07:05.181 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "description", 2024-02-17T13:07:05.181 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.182 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.182 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.182 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.182 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.182 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.182 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.182 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.183 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.183 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.183 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.183 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.183 INFO:teuthology.orchestra.run.smithi018.stdout: "device_url": { 2024-02-17T13:07:05.183 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_url", 2024-02-17T13:07:05.183 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.184 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.184 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.184 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-17T13:07:05.184 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.184 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.184 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.184 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.185 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.185 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.185 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.185 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.185 INFO:teuthology.orchestra.run.smithi018.stdout: "enabled": { 2024-02-17T13:07:05.185 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enabled", 2024-02-17T13:07:05.185 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.185 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.186 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.186 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.186 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.186 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.186 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.186 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.186 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.187 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.187 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.187 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.187 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:05.187 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:05.187 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.187 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.188 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.188 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "24", 2024-02-17T13:07:05.188 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "8", 2024-02-17T13:07:05.188 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.188 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.188 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.188 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.189 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.189 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.189 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.189 INFO:teuthology.orchestra.run.smithi018.stdout: "last_opt_revision": { 2024-02-17T13:07:05.189 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "last_opt_revision", 2024-02-17T13:07:05.189 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.189 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.190 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.190 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1", 2024-02-17T13:07:05.190 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.190 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.190 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.190 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.190 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.191 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.191 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.191 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.191 INFO:teuthology.orchestra.run.smithi018.stdout: "leaderboard": { 2024-02-17T13:07:05.191 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "leaderboard", 2024-02-17T13:07:05.191 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.191 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.192 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.192 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.192 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.192 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.192 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.192 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.192 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.193 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.193 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.193 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.193 INFO:teuthology.orchestra.run.smithi018.stdout: "leaderboard_description": { 2024-02-17T13:07:05.193 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "leaderboard_description", 2024-02-17T13:07:05.193 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.193 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.194 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.194 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.194 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.194 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.194 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.194 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.194 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.194 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.195 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.195 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.195 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.195 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.195 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.195 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.195 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.196 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.196 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.196 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.196 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.196 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.196 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.196 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.197 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.197 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.197 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.197 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.197 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.197 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.197 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.198 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.198 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.198 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.198 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.198 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.198 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.198 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.198 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.199 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.199 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.199 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.199 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.199 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.199 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.199 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.200 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.200 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.200 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.200 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.200 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.200 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.201 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.201 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.201 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.201 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.201 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.201 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.201 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.202 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.202 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.202 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.202 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.202 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.202 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.202 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.203 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.203 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.203 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.203 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.203 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.203 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.203 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.203 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.204 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.204 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.204 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.204 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.204 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.204 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.204 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.205 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.205 INFO:teuthology.orchestra.run.smithi018.stdout: "organization": { 2024-02-17T13:07:05.205 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "organization", 2024-02-17T13:07:05.205 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.205 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.205 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.205 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.205 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.206 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.206 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.206 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.206 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.206 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.206 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.206 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.207 INFO:teuthology.orchestra.run.smithi018.stdout: "proxy": { 2024-02-17T13:07:05.207 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "proxy", 2024-02-17T13:07:05.207 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.207 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.207 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.207 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.207 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.208 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.208 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.208 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.208 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.208 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.208 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.209 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.209 INFO:teuthology.orchestra.run.smithi018.stdout: "url": { 2024-02-17T13:07:05.209 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "url", 2024-02-17T13:07:05.209 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.209 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.209 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.209 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-17T13:07:05.209 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.210 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.210 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.210 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.210 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.210 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.210 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.210 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.211 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.211 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.211 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.211 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "test_orchestrator", 2024-02-17T13:07:05.211 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.211 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.211 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.211 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.212 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.212 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.212 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.212 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.212 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.212 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.212 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.214 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.214 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.214 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.214 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.214 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.214 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.214 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.215 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.215 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.215 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.215 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.215 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.215 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.215 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.215 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.216 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.216 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.216 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.216 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.216 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.216 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.216 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.217 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.217 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.217 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.217 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.217 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.217 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.217 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.218 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.218 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.218 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.218 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.218 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.218 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.218 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.219 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.219 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.219 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.219 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.219 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.219 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.219 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.220 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.220 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.220 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.220 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.220 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.220 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.220 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.220 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.221 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.221 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.221 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.221 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.221 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.221 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.221 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.222 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "volumes", 2024-02-17T13:07:05.222 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.222 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.222 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.222 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.222 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.223 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.223 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.223 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.223 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.223 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.223 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.223 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.224 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.224 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.224 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.224 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.224 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.224 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.224 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.224 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.225 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.225 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.225 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.225 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.225 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.225 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.225 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.226 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.226 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.226 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.226 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.226 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.226 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.226 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.227 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.227 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.227 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.227 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.227 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.227 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.227 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.228 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.228 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.228 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.228 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.228 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.228 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.228 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.228 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.230 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.230 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.230 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.230 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.230 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.230 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.230 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.231 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.231 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.231 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.231 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.231 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.231 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.231 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.232 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.232 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.232 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.232 INFO:teuthology.orchestra.run.smithi018.stdout: "max_concurrent_clones": { 2024-02-17T13:07:05.232 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_concurrent_clones", 2024-02-17T13:07:05.232 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.232 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.233 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.233 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "4", 2024-02-17T13:07:05.233 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.233 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.233 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.233 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-17T13:07:05.233 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.234 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.234 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.234 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.234 INFO:teuthology.orchestra.run.smithi018.stdout: "periodic_async_work": { 2024-02-17T13:07:05.234 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "periodic_async_work", 2024-02-17T13:07:05.234 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.234 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.235 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.235 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.235 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.235 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.235 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.235 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Periodically check for async work", 2024-02-17T13:07:05.235 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.236 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.236 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.236 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.236 INFO:teuthology.orchestra.run.smithi018.stdout: "snapshot_clone_delay": { 2024-02-17T13:07:05.236 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snapshot_clone_delay", 2024-02-17T13:07:05.236 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.236 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-17T13:07:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix", 2024-02-17T13:07:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: "discovery_interval": { 2024-02-17T13:07:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "discovery_interval", 2024-02-17T13:07:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.240 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "100", 2024-02-17T13:07:05.240 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.240 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.240 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.240 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.240 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.240 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.241 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.241 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.241 INFO:teuthology.orchestra.run.smithi018.stdout: "identifier": { 2024-02-17T13:07:05.241 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "identifier", 2024-02-17T13:07:05.241 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.241 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.241 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.241 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.242 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.242 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.242 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.242 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.242 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.242 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.242 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.243 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.243 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:05.243 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:05.243 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.243 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.243 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.244 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:05.244 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.244 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.244 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.244 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.244 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.245 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.245 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.245 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.245 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.245 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.245 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.245 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.246 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.246 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.246 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.246 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.246 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.246 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.246 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.247 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.247 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.247 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.247 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.247 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.247 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.247 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.248 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.248 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.248 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.248 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.248 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.248 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.248 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.249 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.249 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.249 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.249 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.249 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.249 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.249 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.250 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.250 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.250 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.250 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.250 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.250 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.251 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.251 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.251 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.251 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.251 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.251 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.252 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.252 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.252 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.252 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.252 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.252 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.253 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.253 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.253 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.253 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.253 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.253 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.253 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.253 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.254 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.254 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.254 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.254 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.254 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.254 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.254 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.255 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.255 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.255 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.255 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.255 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.255 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_host": { 2024-02-17T13:07:05.255 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_host", 2024-02-17T13:07:05.255 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_port": { 2024-02-17T13:07:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_port", 2024-02-17T13:07:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10051", 2024-02-17T13:07:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_sender": { 2024-02-17T13:07:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_sender", 2024-02-17T13:07:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-17T13:07:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-02-17T13:07:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: "modules": [ 2024-02-17T13:07:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm", 2024-02-17T13:07:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: "dashboard", 2024-02-17T13:07:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: "iostat", 2024-02-17T13:07:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: "nfs", 2024-02-17T13:07:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: "restful", 2024-02-17T13:07:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_schedule" 2024-02-17T13:07:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: "available_modules": [ 2024-02-17T13:07:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "alerts", 2024-02-17T13:07:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How frequently to reexamine health status", 2024-02-17T13:07:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.268 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.268 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.268 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.268 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.268 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.268 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.268 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.269 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.269 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.269 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.269 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.269 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.269 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.269 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.270 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.270 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.270 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.270 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.270 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.270 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.270 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.270 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.271 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.271 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.271 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.271 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.271 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.271 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.271 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.272 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.272 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.272 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.272 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.272 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.272 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.272 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.273 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.273 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.273 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.273 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.273 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.273 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.273 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.274 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.274 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.274 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.274 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.274 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.274 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.274 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.274 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.275 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.275 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.275 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.275 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.275 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.275 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.275 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.276 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_destination": { 2024-02-17T13:07:05.276 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_destination", 2024-02-17T13:07:05.276 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.276 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.276 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.276 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.277 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.277 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.277 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.277 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Email address to send alerts to", 2024-02-17T13:07:05.277 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.277 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.277 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.278 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.278 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_from_name": { 2024-02-17T13:07:05.278 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_from_name", 2024-02-17T13:07:05.278 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.278 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.278 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.278 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "Ceph", 2024-02-17T13:07:05.279 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.279 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.279 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.279 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Email From: name", 2024-02-17T13:07:05.279 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.279 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.279 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.280 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.280 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_host": { 2024-02-17T13:07:05.280 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_host", 2024-02-17T13:07:05.280 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.280 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.280 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.281 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.281 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.281 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.281 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.281 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP server", 2024-02-17T13:07:05.281 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.281 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.281 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.282 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.282 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_password": { 2024-02-17T13:07:05.282 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_password", 2024-02-17T13:07:05.282 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.282 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.282 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.282 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.283 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.283 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.283 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.283 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Password to authenticate with", 2024-02-17T13:07:05.283 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.283 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.283 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.283 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.284 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_port": { 2024-02-17T13:07:05.284 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_port", 2024-02-17T13:07:05.284 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.284 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.284 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.284 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "465", 2024-02-17T13:07:05.284 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.285 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.285 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.285 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP port", 2024-02-17T13:07:05.285 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.285 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.285 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.285 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.286 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_sender": { 2024-02-17T13:07:05.286 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_sender", 2024-02-17T13:07:05.286 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.286 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.286 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.286 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.287 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.287 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.287 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.287 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP envelope sender", 2024-02-17T13:07:05.287 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.287 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.287 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.288 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.288 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_ssl": { 2024-02-17T13:07:05.288 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_ssl", 2024-02-17T13:07:05.288 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.288 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.288 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.288 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.289 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.289 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.289 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.289 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-17T13:07:05.289 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.289 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.289 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.289 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.290 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_user": { 2024-02-17T13:07:05.290 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_user", 2024-02-17T13:07:05.290 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.290 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.290 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.290 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.290 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.290 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.291 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.291 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "User to authenticate as", 2024-02-17T13:07:05.291 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.291 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.291 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.291 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.291 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.292 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.292 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.292 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "balancer", 2024-02-17T13:07:05.292 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.292 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.292 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.293 INFO:teuthology.orchestra.run.smithi018.stdout: "active": { 2024-02-17T13:07:05.293 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "active", 2024-02-17T13:07:05.293 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.293 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.293 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.293 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.294 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.294 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.294 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.294 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "automatically balance PGs across cluster", 2024-02-17T13:07:05.294 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.294 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.294 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.294 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.295 INFO:teuthology.orchestra.run.smithi018.stdout: "begin_time": { 2024-02-17T13:07:05.295 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "begin_time", 2024-02-17T13:07:05.295 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.295 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.295 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.295 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0000", 2024-02-17T13:07:05.295 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.296 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.296 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.296 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "beginning time of day to automatically balance", 2024-02-17T13:07:05.296 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-17T13:07:05.296 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.296 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.297 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.297 INFO:teuthology.orchestra.run.smithi018.stdout: "begin_weekday": { 2024-02-17T13:07:05.297 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "begin_weekday", 2024-02-17T13:07:05.297 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:05.297 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.297 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.298 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:05.298 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0", 2024-02-17T13:07:05.298 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "6", 2024-02-17T13:07:05.298 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.298 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-17T13:07:05.298 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-17T13:07:05.298 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.299 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.299 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_max_iterations": { 2024-02-17T13:07:05.299 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_max_iterations", 2024-02-17T13:07:05.299 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:05.299 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.299 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.299 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "25", 2024-02-17T13:07:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-02-17T13:07:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "250", 2024-02-17T13:07:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-17T13:07:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_metrics": { 2024-02-17T13:07:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_metrics", 2024-02-17T13:07:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "pgs,objects,bytes", 2024-02-17T13:07:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-17T13:07:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-17T13:07:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_step": { 2024-02-17T13:07:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_step", 2024-02-17T13:07:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0.5", 2024-02-17T13:07:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0.001", 2024-02-17T13:07:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "0.999", 2024-02-17T13:07:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "aggressiveness of optimization", 2024-02-17T13:07:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-17T13:07:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "end_time": { 2024-02-17T13:07:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "end_time", 2024-02-17T13:07:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "2359", 2024-02-17T13:07:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "ending time of day to automatically balance", 2024-02-17T13:07:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-17T13:07:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "end_weekday": { 2024-02-17T13:07:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "end_weekday", 2024-02-17T13:07:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0", 2024-02-17T13:07:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "6", 2024-02-17T13:07:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-17T13:07:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-17T13:07:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "min_score": { 2024-02-17T13:07:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "min_score", 2024-02-17T13:07:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-17T13:07:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": { 2024-02-17T13:07:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mode", 2024-02-17T13:07:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "upmap", 2024-02-17T13:07:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "crush-compat", 2024-02-17T13:07:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "none", 2024-02-17T13:07:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "read", 2024-02-17T13:07:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap", 2024-02-17T13:07:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap-read" 2024-02-17T13:07:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Balancer mode", 2024-02-17T13:07:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_ids": { 2024-02-17T13:07:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pool_ids", 2024-02-17T13:07:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-17T13:07:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-17T13:07:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap_max_deviation": { 2024-02-17T13:07:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "upmap_max_deviation", 2024-02-17T13:07:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-02-17T13:07:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-02-17T13:07:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-17T13:07:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-17T13:07:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap_max_optimizations": { 2024-02-17T13:07:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "upmap_max_optimizations", 2024-02-17T13:07:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-17T13:07:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cephadm", 2024-02-17T13:07:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_down_multiplier": { 2024-02-17T13:07:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_down_multiplier", 2024-02-17T13:07:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3.0", 2024-02-17T13:07:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-17T13:07:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_refresh_rate": { 2024-02-17T13:07:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_refresh_rate", 2024-02-17T13:07:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "20", 2024-02-17T13:07:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-17T13:07:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_starting_port": { 2024-02-17T13:07:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_starting_port", 2024-02-17T13:07:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "4721", 2024-02-17T13:07:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-17T13:07:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_ptrace": { 2024-02-17T13:07:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_ptrace", 2024-02-17T13:07:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-17T13:07:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-17T13:07:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "autotune_interval": { 2024-02-17T13:07:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "autotune_interval", 2024-02-17T13:07:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-17T13:07:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "autotune_memory_target_ratio": { 2024-02-17T13:07:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "autotune_memory_target_ratio", 2024-02-17T13:07:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0.7", 2024-02-17T13:07:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-17T13:07:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm_log_destination": { 2024-02-17T13:07:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cephadm_log_destination", 2024-02-17T13:07:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "file", 2024-02-17T13:07:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "file,syslog", 2024-02-17T13:07:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "syslog" 2024-02-17T13:07:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-17T13:07:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: "cgroups_split": { 2024-02-17T13:07:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cgroups_split", 2024-02-17T13:07:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-17T13:07:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "config_checks_enabled": { 2024-02-17T13:07:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_checks_enabled", 2024-02-17T13:07:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-17T13:07:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "config_dashboard": { 2024-02-17T13:07:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_dashboard", 2024-02-17T13:07:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-17T13:07:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_alertmanager": { 2024-02-17T13:07:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_alertmanager", 2024-02-17T13:07:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-17T13:07:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-02-17T13:07:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_base": { 2024-02-17T13:07:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_base", 2024-02-17T13:07:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-17T13:07:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Container image name, without the tag", 2024-02-17T13:07:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_elasticsearch": { 2024-02-17T13:07:05.357 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_elasticsearch", 2024-02-17T13:07:05.357 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.357 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.357 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.357 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-17T13:07:05.357 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "elasticsearch container image", 2024-02-17T13:07:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_grafana": { 2024-02-17T13:07:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_grafana", 2024-02-17T13:07:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-17T13:07:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-02-17T13:07:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_haproxy": { 2024-02-17T13:07:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_haproxy", 2024-02-17T13:07:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-17T13:07:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "HAproxy container image", 2024-02-17T13:07:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_agent": { 2024-02-17T13:07:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_agent", 2024-02-17T13:07:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-17T13:07:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger agent container image", 2024-02-17T13:07:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_collector": { 2024-02-17T13:07:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_collector", 2024-02-17T13:07:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-17T13:07:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger collector container image", 2024-02-17T13:07:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_query": { 2024-02-17T13:07:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_query", 2024-02-17T13:07:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-17T13:07:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger query container image", 2024-02-17T13:07:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_keepalived": { 2024-02-17T13:07:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_keepalived", 2024-02-17T13:07:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.370 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-17T13:07:05.370 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.370 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.370 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.370 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Keepalived container image", 2024-02-17T13:07:05.370 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.370 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.371 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.371 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.371 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_loki": { 2024-02-17T13:07:05.371 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_loki", 2024-02-17T13:07:05.371 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.371 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.371 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.372 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-17T13:07:05.372 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.372 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.372 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.372 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Loki container image", 2024-02-17T13:07:05.372 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.372 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.373 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.373 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.373 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_node_exporter": { 2024-02-17T13:07:05.373 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_node_exporter", 2024-02-17T13:07:05.373 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.373 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.373 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.373 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-17T13:07:05.374 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.374 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.374 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.374 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-02-17T13:07:05.374 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.374 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.374 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.375 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.375 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_nvmeof": { 2024-02-17T13:07:05.375 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_nvmeof", 2024-02-17T13:07:05.375 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.375 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.375 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.375 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-17T13:07:05.375 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.376 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.376 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.376 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Nvme-of container image", 2024-02-17T13:07:05.376 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.376 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.376 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.376 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.377 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_prometheus": { 2024-02-17T13:07:05.377 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_prometheus", 2024-02-17T13:07:05.377 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.377 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.377 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.377 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-17T13:07:05.377 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.377 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.378 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.378 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-02-17T13:07:05.378 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.378 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.378 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.378 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.378 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_promtail": { 2024-02-17T13:07:05.379 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_promtail", 2024-02-17T13:07:05.379 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.379 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.379 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.379 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-17T13:07:05.379 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.379 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.380 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.380 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Promtail container image", 2024-02-17T13:07:05.380 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.380 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.380 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.380 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.381 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_snmp_gateway": { 2024-02-17T13:07:05.381 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_snmp_gateway", 2024-02-17T13:07:05.381 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.381 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.381 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.381 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-17T13:07:05.381 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.381 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.382 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.382 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SNMP Gateway container image", 2024-02-17T13:07:05.382 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.382 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.382 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.382 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.382 INFO:teuthology.orchestra.run.smithi018.stdout: "container_init": { 2024-02-17T13:07:05.383 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_init", 2024-02-17T13:07:05.383 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.383 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.383 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.383 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.383 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.383 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.383 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.384 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Run podman/docker with `--init`", 2024-02-17T13:07:05.384 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.384 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.384 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.384 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.384 INFO:teuthology.orchestra.run.smithi018.stdout: "daemon_cache_timeout": { 2024-02-17T13:07:05.384 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "daemon_cache_timeout", 2024-02-17T13:07:05.385 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.385 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.385 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.385 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:05.385 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.385 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.385 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.385 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-17T13:07:05.386 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.386 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.386 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.386 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.386 INFO:teuthology.orchestra.run.smithi018.stdout: "default_cephadm_command_timeout": { 2024-02-17T13:07:05.386 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "default_cephadm_command_timeout", 2024-02-17T13:07:05.386 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.387 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.387 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.387 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "900", 2024-02-17T13:07:05.387 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.387 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.387 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.387 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-17T13:07:05.388 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.388 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.388 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.388 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.388 INFO:teuthology.orchestra.run.smithi018.stdout: "default_registry": { 2024-02-17T13:07:05.388 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "default_registry", 2024-02-17T13:07:05.388 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.388 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.389 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.389 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io", 2024-02-17T13:07:05.389 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.389 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.389 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.389 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-17T13:07:05.389 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.390 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.390 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.390 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.390 INFO:teuthology.orchestra.run.smithi018.stdout: "device_cache_timeout": { 2024-02-17T13:07:05.390 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_cache_timeout", 2024-02-17T13:07:05.390 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.390 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.391 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.391 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1800", 2024-02-17T13:07:05.391 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.391 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.391 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.391 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache device inventory", 2024-02-17T13:07:05.391 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.392 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.392 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.392 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.392 INFO:teuthology.orchestra.run.smithi018.stdout: "device_enhanced_scan": { 2024-02-17T13:07:05.392 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_enhanced_scan", 2024-02-17T13:07:05.392 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.392 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.393 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.393 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.393 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.393 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.393 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.393 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-17T13:07:05.394 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.394 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.394 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.394 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.394 INFO:teuthology.orchestra.run.smithi018.stdout: "facts_cache_timeout": { 2024-02-17T13:07:05.394 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "facts_cache_timeout", 2024-02-17T13:07:05.394 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.395 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.395 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.395 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:05.395 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.395 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.395 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.395 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache host facts data", 2024-02-17T13:07:05.396 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.396 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.396 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.396 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.396 INFO:teuthology.orchestra.run.smithi018.stdout: "host_check_interval": { 2024-02-17T13:07:05.396 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "host_check_interval", 2024-02-17T13:07:05.397 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.397 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.397 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.397 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:05.397 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.397 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.397 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.398 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to perform a host check", 2024-02-17T13:07:05.398 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.398 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.398 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.398 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.398 INFO:teuthology.orchestra.run.smithi018.stdout: "hw_monitoring": { 2024-02-17T13:07:05.398 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "hw_monitoring", 2024-02-17T13:07:05.398 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.399 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.399 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.399 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.399 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.399 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.399 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.399 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-17T13:07:05.400 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.400 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.400 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.400 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.400 INFO:teuthology.orchestra.run.smithi018.stdout: "inventory_list_all": { 2024-02-17T13:07:05.400 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "inventory_list_all", 2024-02-17T13:07:05.400 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.401 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.401 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.401 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.401 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.401 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.401 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.401 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-17T13:07:05.402 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.402 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.402 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.402 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.402 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.402 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.402 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.403 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.403 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.403 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.403 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.403 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.403 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.403 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.403 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.404 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.404 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.404 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.404 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.404 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.404 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.404 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.405 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.405 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.405 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.405 INFO:teuthology.orchestra.run.smithi018.stdout: "log_refresh_metadata": { 2024-02-17T13:07:05.405 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_refresh_metadata", 2024-02-17T13:07:05.405 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.405 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.406 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.406 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.406 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.406 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.406 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.406 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-17T13:07:05.406 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.407 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.407 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.407 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.407 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.407 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.407 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.407 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.407 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.408 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.408 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.408 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.408 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.408 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-17T13:07:05.408 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.408 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.409 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.409 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.409 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.409 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.409 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.409 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.409 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.410 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.410 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.410 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.410 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.410 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.410 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.410 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.411 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.411 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.411 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.411 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.411 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.411 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.411 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.411 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.412 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.412 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.412 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.412 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.412 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.412 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.412 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.413 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.413 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.413 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.413 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.413 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.413 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.413 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.414 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.414 INFO:teuthology.orchestra.run.smithi018.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-17T13:07:05.414 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-17T13:07:05.414 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.414 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.414 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.414 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.415 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.415 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.415 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.415 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-17T13:07:05.415 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.415 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.415 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.415 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.416 INFO:teuthology.orchestra.run.smithi018.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-17T13:07:05.416 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-17T13:07:05.416 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.416 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.416 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.416 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "*", 2024-02-17T13:07:05.417 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.417 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.417 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.417 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-17T13:07:05.417 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.417 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.417 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.417 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.418 INFO:teuthology.orchestra.run.smithi018.stdout: "max_count_per_host": { 2024-02-17T13:07:05.418 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_count_per_host", 2024-02-17T13:07:05.418 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.418 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.418 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.418 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:05.418 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.419 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.419 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.419 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "max number of daemons per service per host", 2024-02-17T13:07:05.419 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.419 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.419 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.419 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.419 INFO:teuthology.orchestra.run.smithi018.stdout: "max_osd_draining_count": { 2024-02-17T13:07:05.420 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_osd_draining_count", 2024-02-17T13:07:05.420 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.420 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.420 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.420 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:05.420 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.420 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.421 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.421 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-17T13:07:05.421 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.421 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.421 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.421 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.421 INFO:teuthology.orchestra.run.smithi018.stdout: "migration_current": { 2024-02-17T13:07:05.421 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "migration_current", 2024-02-17T13:07:05.422 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.422 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.422 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.422 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.422 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.422 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.422 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.423 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "internal - do not modify", 2024-02-17T13:07:05.423 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.423 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.423 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.423 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.423 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": { 2024-02-17T13:07:05.423 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mode", 2024-02-17T13:07:05.423 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.424 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.424 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.424 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "root", 2024-02-17T13:07:05.424 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.424 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.424 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.424 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm-package", 2024-02-17T13:07:05.425 INFO:teuthology.orchestra.run.smithi018.stdout: "root" 2024-02-17T13:07:05.425 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.425 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "mode for remote execution of cephadm", 2024-02-17T13:07:05.425 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.425 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.425 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.425 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.425 INFO:teuthology.orchestra.run.smithi018.stdout: "oob_default_addr": { 2024-02-17T13:07:05.426 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "oob_default_addr", 2024-02-17T13:07:05.426 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.426 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.426 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.426 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "169.254.1.1", 2024-02-17T13:07:05.426 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.426 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.426 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.427 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-17T13:07:05.427 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.427 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.427 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.427 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.427 INFO:teuthology.orchestra.run.smithi018.stdout: "prometheus_alerts_path": { 2024-02-17T13:07:05.427 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prometheus_alerts_path", 2024-02-17T13:07:05.428 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.428 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.428 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.428 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-17T13:07:05.428 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.428 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.428 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.428 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-17T13:07:05.429 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.429 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.429 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.429 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.429 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_insecure": { 2024-02-17T13:07:05.429 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_insecure", 2024-02-17T13:07:05.429 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.430 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.430 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.430 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.430 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.430 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.430 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.430 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-17T13:07:05.430 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.431 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.431 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.431 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.431 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_password": { 2024-02-17T13:07:05.431 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_password", 2024-02-17T13:07:05.431 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.431 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.432 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.432 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.432 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.432 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.432 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.432 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-17T13:07:05.432 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.433 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.433 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.433 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.433 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_url": { 2024-02-17T13:07:05.433 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_url", 2024-02-17T13:07:05.433 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.433 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.434 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.434 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.434 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.434 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.434 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.434 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-17T13:07:05.434 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.435 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.435 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.435 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.435 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_username": { 2024-02-17T13:07:05.435 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_username", 2024-02-17T13:07:05.435 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.435 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.436 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.436 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.436 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.436 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.436 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.436 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-17T13:07:05.436 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.437 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.437 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.437 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.437 INFO:teuthology.orchestra.run.smithi018.stdout: "secure_monitoring_stack": { 2024-02-17T13:07:05.437 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "secure_monitoring_stack", 2024-02-17T13:07:05.437 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.437 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.438 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.438 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.438 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.438 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.438 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.438 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-17T13:07:05.438 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.439 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.439 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.439 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.439 INFO:teuthology.orchestra.run.smithi018.stdout: "service_discovery_port": { 2024-02-17T13:07:05.439 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "service_discovery_port", 2024-02-17T13:07:05.439 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.440 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.440 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.440 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8765", 2024-02-17T13:07:05.440 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.440 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.440 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.440 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "cephadm service discovery port", 2024-02-17T13:07:05.441 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.441 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.441 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.441 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.441 INFO:teuthology.orchestra.run.smithi018.stdout: "ssh_config_file": { 2024-02-17T13:07:05.441 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssh_config_file", 2024-02-17T13:07:05.441 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.442 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.442 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.442 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.442 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.442 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.442 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.442 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-17T13:07:05.442 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.443 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.443 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.443 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.443 INFO:teuthology.orchestra.run.smithi018.stdout: "use_agent": { 2024-02-17T13:07:05.443 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "use_agent", 2024-02-17T13:07:05.443 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.443 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.444 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.444 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.444 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.444 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.444 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.444 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-17T13:07:05.444 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.444 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.445 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.445 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.445 INFO:teuthology.orchestra.run.smithi018.stdout: "use_repo_digest": { 2024-02-17T13:07:05.445 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "use_repo_digest", 2024-02-17T13:07:05.445 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.445 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.445 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.446 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.446 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.446 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.446 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.446 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-17T13:07:05.446 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.446 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.446 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.447 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.447 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_failed_host_check": { 2024-02-17T13:07:05.447 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_failed_host_check", 2024-02-17T13:07:05.447 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.447 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.447 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.447 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.448 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.448 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.448 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.448 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if the host check fails", 2024-02-17T13:07:05.448 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.448 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.448 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.449 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.449 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_stray_daemons": { 2024-02-17T13:07:05.449 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_stray_daemons", 2024-02-17T13:07:05.449 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.449 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.449 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.449 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.450 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.450 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.450 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.450 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-17T13:07:05.450 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.450 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.450 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.451 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.451 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_stray_hosts": { 2024-02-17T13:07:05.451 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_stray_hosts", 2024-02-17T13:07:05.451 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.451 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.451 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.451 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.451 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.452 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.452 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.452 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-17T13:07:05.452 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.452 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.452 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.452 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.453 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.453 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.453 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.453 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crash", 2024-02-17T13:07:05.453 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.453 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.453 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.454 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.454 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.454 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.454 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.454 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.454 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.454 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.455 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.455 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.455 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.455 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.455 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.455 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.455 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.456 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.456 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.456 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.456 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.456 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.456 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.456 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.457 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.457 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.457 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.457 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.457 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.457 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.457 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.458 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.458 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.458 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.458 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.458 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.458 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.458 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.459 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.459 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.459 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.459 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.459 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.459 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.459 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.460 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.460 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.460 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.460 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.460 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.460 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.460 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.461 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.461 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.461 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.461 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.461 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.461 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.461 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.462 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.462 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.462 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.462 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.462 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.462 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.462 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.463 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.463 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.463 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.463 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.463 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.463 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.463 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.464 INFO:teuthology.orchestra.run.smithi018.stdout: "retain_interval": { 2024-02-17T13:07:05.464 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "retain_interval", 2024-02-17T13:07:05.464 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.464 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.464 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.464 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "31536000", 2024-02-17T13:07:05.464 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.464 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.465 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.465 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-17T13:07:05.465 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.465 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.465 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.465 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.465 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_recent_interval": { 2024-02-17T13:07:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_recent_interval", 2024-02-17T13:07:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1209600", 2024-02-17T13:07:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-17T13:07:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.468 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.468 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "dashboard", 2024-02-17T13:07:05.468 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.468 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.468 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.468 INFO:teuthology.orchestra.run.smithi018.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-17T13:07:05.468 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-17T13:07:05.469 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.471 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.471 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.471 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:05.471 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.471 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.471 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.471 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.472 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.472 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.472 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.472 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.472 INFO:teuthology.orchestra.run.smithi018.stdout: "ALERTMANAGER_API_HOST": { 2024-02-17T13:07:05.472 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-17T13:07:05.472 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.473 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.473 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.473 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.473 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.473 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.473 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.473 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.474 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.474 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.474 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.474 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.474 INFO:teuthology.orchestra.run.smithi018.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-17T13:07:05.474 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-17T13:07:05.474 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.475 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.475 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.475 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.475 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.475 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.475 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.475 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.476 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.476 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.476 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.476 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.476 INFO:teuthology.orchestra.run.smithi018.stdout: "AUDIT_API_ENABLED": { 2024-02-17T13:07:05.476 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "AUDIT_API_ENABLED", 2024-02-17T13:07:05.476 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.477 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.477 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.477 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.477 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.477 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.477 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.477 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.478 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.478 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.478 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.478 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.478 INFO:teuthology.orchestra.run.smithi018.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-17T13:07:05.478 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-17T13:07:05.478 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.479 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.479 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.479 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.479 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.479 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.479 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.480 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.480 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.480 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.480 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.480 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.480 INFO:teuthology.orchestra.run.smithi018.stdout: "ENABLE_BROWSABLE_API": { 2024-02-17T13:07:05.480 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-17T13:07:05.480 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.481 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.481 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.481 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.481 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.481 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.481 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.481 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.482 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.482 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.482 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.482 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.482 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-17T13:07:05.482 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-17T13:07:05.482 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.483 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.483 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.483 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.483 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.483 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.483 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.483 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.483 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.484 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.484 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.484 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.484 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-17T13:07:05.484 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-17T13:07:05.484 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.484 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.485 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.485 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.485 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.485 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.485 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.485 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.486 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.486 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.486 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.486 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.486 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-17T13:07:05.486 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-17T13:07:05.487 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.487 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.487 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.487 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.487 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.487 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.488 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.488 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.488 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.488 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.488 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.488 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.488 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-17T13:07:05.488 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-17T13:07:05.489 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.489 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.489 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.489 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.489 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.489 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.489 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.490 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.490 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.490 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.490 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.490 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.490 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-17T13:07:05.490 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-17T13:07:05.490 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.491 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.491 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.491 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.491 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.491 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.491 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.491 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.491 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.492 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.492 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.492 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.492 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-17T13:07:05.492 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-17T13:07:05.492 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.492 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.493 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.493 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.493 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.493 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.493 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.493 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.493 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.493 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.494 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.494 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.494 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-17T13:07:05.494 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-17T13:07:05.494 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.494 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.494 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.494 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.495 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.495 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.495 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.495 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.495 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.495 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.495 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.495 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.496 INFO:teuthology.orchestra.run.smithi018.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-17T13:07:05.496 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-17T13:07:05.496 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.496 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.496 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.496 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.496 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.497 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.497 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.497 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.497 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.497 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.497 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.497 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.498 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_PASSWORD": { 2024-02-17T13:07:05.498 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-17T13:07:05.498 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.498 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.498 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.498 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-02-17T13:07:05.499 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.499 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.499 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.499 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.499 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.499 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.499 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.500 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.500 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-17T13:07:05.500 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-17T13:07:05.500 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.500 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.500 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.500 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.501 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.501 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.501 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.501 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.501 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.501 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.501 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.502 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.502 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_URL": { 2024-02-17T13:07:05.502 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_URL", 2024-02-17T13:07:05.502 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.502 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.502 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.502 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.503 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.503 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.503 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.503 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.503 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.503 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.503 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.503 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.504 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_USERNAME": { 2024-02-17T13:07:05.504 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-17T13:07:05.504 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.504 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.504 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.504 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-02-17T13:07:05.504 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.505 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.505 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.505 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.505 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.505 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.505 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.505 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.506 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-17T13:07:05.506 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-17T13:07:05.506 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.506 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.506 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.506 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.506 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.506 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.507 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.507 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.507 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.507 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.507 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.507 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.507 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-17T13:07:05.508 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-17T13:07:05.508 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.508 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.508 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.508 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.508 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.509 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.509 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.509 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.509 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.509 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.509 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.509 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.509 INFO:teuthology.orchestra.run.smithi018.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-17T13:07:05.510 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-17T13:07:05.510 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.510 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.510 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.510 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.510 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.510 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.511 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.511 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.511 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.511 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.511 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.511 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.511 INFO:teuthology.orchestra.run.smithi018.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-17T13:07:05.512 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-17T13:07:05.512 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.512 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.512 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.512 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.512 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.512 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.513 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.513 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.513 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.513 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.513 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.513 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.513 INFO:teuthology.orchestra.run.smithi018.stdout: "PROMETHEUS_API_HOST": { 2024-02-17T13:07:05.513 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-17T13:07:05.514 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.514 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.514 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.514 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.514 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.514 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.514 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.515 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.515 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.515 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.515 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.515 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.515 INFO:teuthology.orchestra.run.smithi018.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-17T13:07:05.515 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-17T13:07:05.515 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.516 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.516 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.516 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.516 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.516 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.516 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.516 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.517 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.517 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.517 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.517 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.517 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-17T13:07:05.517 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-17T13:07:05.517 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.517 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.518 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.518 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.518 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.518 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.518 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.518 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.519 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.519 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.519 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.519 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.519 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-17T13:07:05.519 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-17T13:07:05.519 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.519 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.520 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.520 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.520 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.520 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.520 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.520 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.520 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.520 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.521 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.521 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.521 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-17T13:07:05.521 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-17T13:07:05.521 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.521 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.521 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.522 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.522 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.522 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.522 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.522 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.522 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.522 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.522 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.523 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.523 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-17T13:07:05.523 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-17T13:07:05.523 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.523 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.523 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.523 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.523 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.524 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.524 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.524 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.524 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.524 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.524 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.525 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.525 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-17T13:07:05.525 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-17T13:07:05.525 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.525 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.525 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.525 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.525 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.526 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.526 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.526 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.526 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.526 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.526 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.526 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.527 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-17T13:07:05.527 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-17T13:07:05.527 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.527 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.527 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.527 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.527 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.528 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.528 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.528 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.528 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.528 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.528 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.528 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.529 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-17T13:07:05.529 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-17T13:07:05.529 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.529 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.529 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.529 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.529 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.530 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.530 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.530 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.530 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.530 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.530 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.530 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.531 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_ENABLED": { 2024-02-17T13:07:05.531 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-17T13:07:05.531 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.531 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.531 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.531 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.531 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.532 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.532 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.532 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.532 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.532 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.532 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.532 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.533 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-17T13:07:05.533 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-17T13:07:05.533 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.533 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.533 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.533 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-17T13:07:05.533 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.534 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.534 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.534 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.534 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.534 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.534 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.534 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.535 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-17T13:07:05.535 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-17T13:07:05.535 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.535 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.535 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.535 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:05.535 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.536 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.536 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.536 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.536 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.536 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.536 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.536 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.537 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-17T13:07:05.537 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-17T13:07:05.537 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.537 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.537 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.537 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8", 2024-02-17T13:07:05.537 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.538 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.538 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.538 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.538 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.538 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.538 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.538 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.539 INFO:teuthology.orchestra.run.smithi018.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-17T13:07:05.539 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-17T13:07:05.539 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.539 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.539 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.539 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "45", 2024-02-17T13:07:05.539 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.540 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.540 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.540 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.540 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.540 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.540 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.540 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.540 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_ACCESS_KEY": { 2024-02-17T13:07:05.541 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-17T13:07:05.541 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.541 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.541 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.541 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.541 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.541 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.542 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.542 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.542 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.542 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.542 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.542 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.542 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-17T13:07:05.543 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-17T13:07:05.543 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.543 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.543 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.543 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-02-17T13:07:05.543 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.543 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.544 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.544 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.544 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.544 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.544 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.544 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.544 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_SECRET_KEY": { 2024-02-17T13:07:05.545 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-17T13:07:05.545 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.545 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.545 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.545 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.545 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.545 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.546 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.546 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.546 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.546 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.546 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.546 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.546 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_SSL_VERIFY": { 2024-02-17T13:07:05.546 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-17T13:07:05.547 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.547 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.547 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.547 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.547 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.547 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.547 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.547 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.548 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.548 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.548 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.548 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.548 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-17T13:07:05.548 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-17T13:07:05.549 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.549 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.549 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.549 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:05.549 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.549 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.549 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.550 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.550 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.550 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.550 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.550 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.550 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-17T13:07:05.550 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-17T13:07:05.551 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.551 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.551 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.551 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:05.551 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.551 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.551 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.552 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.552 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.552 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.552 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.552 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.552 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-17T13:07:05.552 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-17T13:07:05.552 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.553 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.553 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.553 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-02-17T13:07:05.553 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.553 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.553 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.553 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.553 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.554 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.554 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.554 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.554 INFO:teuthology.orchestra.run.smithi018.stdout: "cross_origin_url": { 2024-02-17T13:07:05.554 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cross_origin_url", 2024-02-17T13:07:05.554 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.555 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.555 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.555 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.555 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.555 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.555 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.555 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.556 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.556 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.556 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.556 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.556 INFO:teuthology.orchestra.run.smithi018.stdout: "crt_file": { 2024-02-17T13:07:05.556 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crt_file", 2024-02-17T13:07:05.556 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.556 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.557 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.557 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.557 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.557 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.557 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.557 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.557 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.558 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.558 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.558 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.558 INFO:teuthology.orchestra.run.smithi018.stdout: "debug": { 2024-02-17T13:07:05.558 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "debug", 2024-02-17T13:07:05.558 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.558 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.559 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.559 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.559 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.559 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.559 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.559 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable/disable debug options", 2024-02-17T13:07:05.559 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.560 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.560 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.560 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.560 INFO:teuthology.orchestra.run.smithi018.stdout: "jwt_token_ttl": { 2024-02-17T13:07:05.560 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "jwt_token_ttl", 2024-02-17T13:07:05.560 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.561 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.561 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.561 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "28800", 2024-02-17T13:07:05.561 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.561 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.561 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.561 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.561 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.562 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.562 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.562 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.562 INFO:teuthology.orchestra.run.smithi018.stdout: "key_file": { 2024-02-17T13:07:05.562 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "key_file", 2024-02-17T13:07:05.562 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.562 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.563 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.563 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.563 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.563 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.563 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.563 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.563 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.564 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.564 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.564 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.564 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.564 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.564 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.564 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.564 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.565 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.565 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.565 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.565 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.565 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.565 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.565 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.566 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.566 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.566 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.566 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.566 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.566 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.567 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.567 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.567 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.567 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.567 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.567 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.567 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.568 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.568 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.568 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.568 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.568 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.568 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.568 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.569 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.569 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.569 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.569 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.569 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.569 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.569 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.570 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.570 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.570 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.570 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.570 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.570 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.570 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.570 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.571 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.571 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.571 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.571 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.571 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.571 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.572 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.572 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.572 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.572 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.572 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.572 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.572 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.573 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.573 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.573 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.573 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.573 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.573 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.573 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.574 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.574 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.574 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.574 INFO:teuthology.orchestra.run.smithi018.stdout: "motd": { 2024-02-17T13:07:05.574 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "motd", 2024-02-17T13:07:05.574 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.574 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.575 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.575 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.575 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.575 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.575 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.575 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "The message of the day", 2024-02-17T13:07:05.575 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.575 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.576 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.576 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.576 INFO:teuthology.orchestra.run.smithi018.stdout: "redirect_resolve_ip_addr": { 2024-02-17T13:07:05.576 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "redirect_resolve_ip_addr", 2024-02-17T13:07:05.576 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.576 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.577 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.577 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.577 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.577 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.577 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.577 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.577 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.578 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.578 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.578 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.578 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-02-17T13:07:05.578 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-02-17T13:07:05.578 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.578 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.578 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.579 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "::", 2024-02-17T13:07:05.579 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.579 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.579 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.579 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.579 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.579 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.580 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.580 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.580 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-02-17T13:07:05.580 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-02-17T13:07:05.580 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.580 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.581 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.581 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8080", 2024-02-17T13:07:05.581 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.581 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.581 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.581 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.581 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.582 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.582 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.582 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.582 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl": { 2024-02-17T13:07:05.582 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl", 2024-02-17T13:07:05.582 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.582 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.583 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.583 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.583 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.583 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.583 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.583 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.584 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.584 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.584 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.584 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.584 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl_server_port": { 2024-02-17T13:07:05.584 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl_server_port", 2024-02-17T13:07:05.585 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.585 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.585 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.585 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8443", 2024-02-17T13:07:05.585 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.585 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.585 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.586 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.586 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.586 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.586 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.586 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.586 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_behaviour": { 2024-02-17T13:07:05.586 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_behaviour", 2024-02-17T13:07:05.587 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.587 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.587 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.587 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "redirect", 2024-02-17T13:07:05.587 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.587 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.587 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.587 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.588 INFO:teuthology.orchestra.run.smithi018.stdout: "redirect" 2024-02-17T13:07:05.588 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.588 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.588 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.588 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.588 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.588 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.589 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_error_status_code": { 2024-02-17T13:07:05.589 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_error_status_code", 2024-02-17T13:07:05.589 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.589 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.589 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.589 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "500", 2024-02-17T13:07:05.589 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "400", 2024-02-17T13:07:05.590 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "599", 2024-02-17T13:07:05.590 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.590 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.590 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.590 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.590 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.590 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.591 INFO:teuthology.orchestra.run.smithi018.stdout: "url_prefix": { 2024-02-17T13:07:05.591 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "url_prefix", 2024-02-17T13:07:05.591 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.591 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.591 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.591 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.591 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.592 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.592 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.592 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.592 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.592 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.592 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.593 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.593 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.593 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.593 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.593 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "devicehealth", 2024-02-17T13:07:05.593 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.593 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.594 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.594 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_monitoring": { 2024-02-17T13:07:05.594 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enable_monitoring", 2024-02-17T13:07:05.594 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.594 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.594 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.594 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.595 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.595 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.595 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.595 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "monitor device health metrics", 2024-02-17T13:07:05.595 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.595 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.595 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.596 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.596 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.596 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.596 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.596 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.596 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.596 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.597 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.597 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.597 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.597 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.597 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.597 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.597 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.598 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.598 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.598 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.598 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.598 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.598 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.598 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.599 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.599 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.599 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.599 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.599 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.599 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.599 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.599 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.600 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.600 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.600 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.600 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.600 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.600 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.601 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.601 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.601 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.601 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.601 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.601 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.601 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.601 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.602 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.602 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.602 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.602 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.602 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.602 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.602 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.603 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.603 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.603 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.603 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.603 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.603 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.603 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.603 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.604 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.604 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.604 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.604 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.604 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.604 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.604 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.605 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.605 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.605 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.605 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.605 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.605 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.605 INFO:teuthology.orchestra.run.smithi018.stdout: "mark_out_threshold": { 2024-02-17T13:07:05.606 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mark_out_threshold", 2024-02-17T13:07:05.606 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.606 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.606 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.606 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "2419200", 2024-02-17T13:07:05.606 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.606 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.607 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.607 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-17T13:07:05.607 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.607 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.607 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.607 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.607 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": { 2024-02-17T13:07:05.607 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pool_name", 2024-02-17T13:07:05.608 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.608 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.608 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.608 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "device_health_metrics", 2024-02-17T13:07:05.608 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.608 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.609 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.609 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-17T13:07:05.609 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.609 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.609 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.609 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.609 INFO:teuthology.orchestra.run.smithi018.stdout: "retention_period": { 2024-02-17T13:07:05.610 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "retention_period", 2024-02-17T13:07:05.610 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.610 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.610 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.610 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15552000", 2024-02-17T13:07:05.610 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.610 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.611 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.611 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long to retain device health metrics", 2024-02-17T13:07:05.611 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.611 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.611 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.611 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.611 INFO:teuthology.orchestra.run.smithi018.stdout: "scrape_frequency": { 2024-02-17T13:07:05.611 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "scrape_frequency", 2024-02-17T13:07:05.612 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.612 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.612 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.612 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "86400", 2024-02-17T13:07:05.612 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.612 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.613 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.613 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-17T13:07:05.613 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.613 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.613 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.613 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.613 INFO:teuthology.orchestra.run.smithi018.stdout: "self_heal": { 2024-02-17T13:07:05.614 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "self_heal", 2024-02-17T13:07:05.614 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.614 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.614 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.614 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.614 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.614 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.615 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.615 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-17T13:07:05.615 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.615 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.615 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.615 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.615 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:05.616 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:05.616 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.616 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.616 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.616 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:05.616 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.616 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.616 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.617 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to wake up and check device health", 2024-02-17T13:07:05.617 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.617 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.617 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.617 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.617 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_threshold": { 2024-02-17T13:07:05.617 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_threshold", 2024-02-17T13:07:05.618 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.618 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.618 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.618 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "7257600", 2024-02-17T13:07:05.618 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.618 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.618 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.618 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-17T13:07:05.619 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.619 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.619 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.619 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.619 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.619 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.619 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.619 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "diskprediction_local", 2024-02-17T13:07:05.620 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.620 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.620 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.620 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.620 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.620 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.621 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.621 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.621 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.621 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.621 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.621 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.621 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.622 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.622 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.622 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.622 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.622 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.622 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.622 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.622 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.623 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.623 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.623 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.623 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.623 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.623 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.623 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.624 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.624 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.624 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.624 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.624 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.624 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.624 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.625 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.625 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.625 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.625 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.625 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.625 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.625 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.625 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.626 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.626 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.626 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.626 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.626 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.626 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.627 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.627 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.627 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.627 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.627 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.627 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.627 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.628 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.628 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.628 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.628 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.628 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.628 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.629 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.629 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.629 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.629 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.629 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.629 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.629 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.630 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.630 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.630 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.630 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.630 INFO:teuthology.orchestra.run.smithi018.stdout: "predict_interval": { 2024-02-17T13:07:05.630 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "predict_interval", 2024-02-17T13:07:05.630 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.630 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.631 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.631 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "86400", 2024-02-17T13:07:05.631 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.631 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.631 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.631 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.631 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.632 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.632 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.632 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.632 INFO:teuthology.orchestra.run.smithi018.stdout: "predictor_model": { 2024-02-17T13:07:05.632 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "predictor_model", 2024-02-17T13:07:05.632 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.632 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.633 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.633 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "prophetstor", 2024-02-17T13:07:05.633 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.633 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.633 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.633 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.633 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.633 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.634 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.634 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.634 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:05.634 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:05.634 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.634 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.634 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.634 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-02-17T13:07:05.635 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.635 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.635 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.635 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.635 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.635 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.635 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.636 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.636 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.636 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.636 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.636 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "influx", 2024-02-17T13:07:05.636 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": false, 2024-02-17T13:07:05.636 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "influxdb python module not found", 2024-02-17T13:07:05.636 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.637 INFO:teuthology.orchestra.run.smithi018.stdout: "batch_size": { 2024-02-17T13:07:05.637 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "batch_size", 2024-02-17T13:07:05.637 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.637 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.637 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.637 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5000", 2024-02-17T13:07:05.637 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.638 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.638 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.638 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-17T13:07:05.638 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.638 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.638 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.638 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.638 INFO:teuthology.orchestra.run.smithi018.stdout: "database": { 2024-02-17T13:07:05.639 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "database", 2024-02-17T13:07:05.639 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.639 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.639 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.639 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "ceph", 2024-02-17T13:07:05.639 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.639 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.640 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.640 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-17T13:07:05.640 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.640 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.640 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.640 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.640 INFO:teuthology.orchestra.run.smithi018.stdout: "hostname": { 2024-02-17T13:07:05.641 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "hostname", 2024-02-17T13:07:05.641 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.641 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.641 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.641 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.641 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.641 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.641 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.642 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB server hostname", 2024-02-17T13:07:05.642 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.642 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.642 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.642 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.642 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:05.643 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:05.643 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.643 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.643 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.643 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "30", 2024-02-17T13:07:05.643 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "5", 2024-02-17T13:07:05.643 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.643 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.644 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-17T13:07:05.644 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.644 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.644 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.644 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.644 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.644 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.645 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.645 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.645 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.645 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.645 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.646 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.646 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.646 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.646 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.646 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.646 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.646 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.646 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.647 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.647 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.647 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.647 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.647 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.647 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.647 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.648 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.648 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.648 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.648 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.648 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.648 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.648 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.650 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.650 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.650 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.650 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.650 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.650 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.650 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.651 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.651 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.651 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.651 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.651 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.651 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.651 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.651 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.652 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.652 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.652 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.652 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.652 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.652 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.653 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.653 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.653 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.653 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.653 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.653 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.653 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.654 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.654 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.654 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.654 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.654 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.654 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.654 INFO:teuthology.orchestra.run.smithi018.stdout: "password": { 2024-02-17T13:07:05.655 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "password", 2024-02-17T13:07:05.655 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.655 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.655 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.655 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.655 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.655 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.655 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.656 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "password of InfluxDB server user", 2024-02-17T13:07:05.656 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.656 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.656 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.656 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.656 INFO:teuthology.orchestra.run.smithi018.stdout: "port": { 2024-02-17T13:07:05.656 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "port", 2024-02-17T13:07:05.657 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.657 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.657 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.657 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8086", 2024-02-17T13:07:05.657 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.657 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.657 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.657 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB server port", 2024-02-17T13:07:05.658 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.658 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.658 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.658 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.658 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl": { 2024-02-17T13:07:05.658 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl", 2024-02-17T13:07:05.658 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.659 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.659 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.659 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "false", 2024-02-17T13:07:05.659 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.659 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.659 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.659 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-17T13:07:05.659 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.660 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.660 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.660 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.660 INFO:teuthology.orchestra.run.smithi018.stdout: "threads": { 2024-02-17T13:07:05.660 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "threads", 2024-02-17T13:07:05.660 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.661 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.661 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.661 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-02-17T13:07:05.661 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-02-17T13:07:05.661 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "32", 2024-02-17T13:07:05.661 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.661 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-17T13:07:05.661 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.662 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.662 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.662 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.662 INFO:teuthology.orchestra.run.smithi018.stdout: "username": { 2024-02-17T13:07:05.662 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "username", 2024-02-17T13:07:05.662 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.662 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.663 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.663 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.663 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.663 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.663 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.663 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "username of InfluxDB server user", 2024-02-17T13:07:05.663 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.664 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.664 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.664 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.664 INFO:teuthology.orchestra.run.smithi018.stdout: "verify_ssl": { 2024-02-17T13:07:05.664 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "verify_ssl", 2024-02-17T13:07:05.664 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.664 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.665 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.665 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "true", 2024-02-17T13:07:05.665 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.665 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.665 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.665 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-17T13:07:05.666 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.666 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.666 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.666 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.666 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.666 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.667 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.667 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "insights", 2024-02-17T13:07:05.667 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.667 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.667 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.667 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.667 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.667 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.668 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.668 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.668 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.668 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.668 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.668 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.668 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.669 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.669 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.669 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.669 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.669 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.669 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.669 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.669 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.670 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.670 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.670 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.670 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.670 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.670 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.670 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.671 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.671 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.671 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.671 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.671 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.671 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.672 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.672 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.672 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.672 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.672 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.672 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.672 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.673 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.673 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.673 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.673 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.673 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.673 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.674 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.674 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.674 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.674 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.674 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.674 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.674 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.675 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.675 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.675 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.675 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.675 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.675 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.676 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.676 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.676 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.676 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.676 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.676 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.677 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.677 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.677 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.677 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.679 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.679 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.679 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.679 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.679 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.679 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.680 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "iostat", 2024-02-17T13:07:05.680 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.680 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.680 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.680 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.680 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.680 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.681 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.681 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.681 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.681 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.681 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.681 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.682 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.682 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.682 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.682 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.682 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.682 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.682 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.682 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.683 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.683 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.683 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.683 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.683 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.683 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.683 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.683 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.684 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.684 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.684 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.684 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.684 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.684 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.684 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.685 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.685 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.685 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.685 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.685 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.685 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.685 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.686 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.686 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.686 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.686 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.686 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.686 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.686 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.687 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.687 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.687 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.687 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.687 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.687 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.687 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.688 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.688 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.688 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.688 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.688 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.688 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.689 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.689 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.689 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.689 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.689 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.689 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.689 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.689 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.690 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.690 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.690 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.690 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.690 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.690 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.690 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "k8sevents", 2024-02-17T13:07:05.691 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.691 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.691 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.691 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_event_retention_days": { 2024-02-17T13:07:05.691 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ceph_event_retention_days", 2024-02-17T13:07:05.691 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.691 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.691 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.692 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "7", 2024-02-17T13:07:05.692 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.692 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.692 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.692 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-17T13:07:05.692 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.693 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.693 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.693 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.693 INFO:teuthology.orchestra.run.smithi018.stdout: "config_check_secs": { 2024-02-17T13:07:05.693 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_check_secs", 2024-02-17T13:07:05.693 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.693 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.693 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.694 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:05.694 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "10", 2024-02-17T13:07:05.694 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.694 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.694 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-17T13:07:05.694 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.694 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.695 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.695 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.695 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.695 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.695 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.695 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.695 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.696 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.696 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.696 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.696 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.696 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.696 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.696 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.697 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.697 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.697 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.697 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.697 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.697 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.697 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.698 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.698 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.698 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.698 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.698 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.698 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.698 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.698 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.699 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.699 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.699 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.699 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.699 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.699 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.699 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.700 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.700 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.700 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.700 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.700 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.700 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.700 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.701 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.701 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.701 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.701 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.701 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.701 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.702 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.702 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.702 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.702 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.702 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.702 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.702 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.702 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.703 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.703 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.703 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.703 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.703 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.703 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.703 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.704 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.704 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.704 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.704 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.704 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.704 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.704 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.705 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.705 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.705 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.705 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.705 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "localpool", 2024-02-17T13:07:05.705 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.705 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.705 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.705 INFO:teuthology.orchestra.run.smithi018.stdout: "failure_domain": { 2024-02-17T13:07:05.706 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "failure_domain", 2024-02-17T13:07:05.706 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.706 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.706 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.706 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "host", 2024-02-17T13:07:05.706 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.706 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.706 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.707 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "failure domain for any created local pool", 2024-02-17T13:07:05.707 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-17T13:07:05.707 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.707 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.707 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.707 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.707 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.707 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.708 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.708 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.708 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.708 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.708 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.708 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.709 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.709 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.709 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.709 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.709 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.709 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.709 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.710 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.710 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.710 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.710 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.710 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.710 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.710 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.711 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.711 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.711 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.711 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.711 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.711 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.711 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.712 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.712 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.712 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.712 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.712 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.712 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.712 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.713 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.713 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.713 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.713 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.713 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.713 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.713 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.714 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.714 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.714 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.714 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.714 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.714 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.714 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.714 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.715 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.715 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.715 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.715 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.715 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.715 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.715 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.716 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.716 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.716 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.716 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.716 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.716 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.716 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.716 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.717 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.717 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.717 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.717 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": { 2024-02-17T13:07:05.717 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "min_size", 2024-02-17T13:07:05.717 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.718 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.718 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.718 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.718 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.718 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.718 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.718 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default min_size for any created local pool", 2024-02-17T13:07:05.718 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-17T13:07:05.719 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.719 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.719 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.719 INFO:teuthology.orchestra.run.smithi018.stdout: "num_rep": { 2024-02-17T13:07:05.719 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "num_rep", 2024-02-17T13:07:05.719 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.719 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.720 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.720 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3", 2024-02-17T13:07:05.720 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.720 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.720 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.720 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default replica count for any created local pool", 2024-02-17T13:07:05.721 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.721 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.721 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.721 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.721 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": { 2024-02-17T13:07:05.721 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pg_num", 2024-02-17T13:07:05.721 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.722 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.722 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.722 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "128", 2024-02-17T13:07:05.722 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.722 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.722 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.722 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default pg_num for any created local pool", 2024-02-17T13:07:05.723 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.723 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.723 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.723 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.723 INFO:teuthology.orchestra.run.smithi018.stdout: "prefix": { 2024-02-17T13:07:05.723 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prefix", 2024-02-17T13:07:05.724 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.724 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.724 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.724 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.724 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.724 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.724 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.725 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "name prefix for any created local pool", 2024-02-17T13:07:05.725 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.725 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.725 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.725 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.725 INFO:teuthology.orchestra.run.smithi018.stdout: "subtree": { 2024-02-17T13:07:05.725 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "subtree", 2024-02-17T13:07:05.726 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.726 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.726 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.726 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "rack", 2024-02-17T13:07:05.726 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.726 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.726 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.727 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-17T13:07:05.727 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-17T13:07:05.727 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.727 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.727 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.727 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.727 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.727 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.728 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mds_autoscaler", 2024-02-17T13:07:05.728 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.728 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.728 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.728 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.728 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.729 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.729 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.729 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.729 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.729 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.729 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.729 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.729 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.730 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.730 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.730 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.730 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.730 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.730 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.730 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.730 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.731 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.731 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.731 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.731 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.731 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.731 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.731 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.731 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.732 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.732 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.732 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.732 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.732 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.732 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.732 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.733 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.733 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.733 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.733 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.733 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.733 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.733 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.734 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.734 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.734 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.734 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.734 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.734 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.734 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.734 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.735 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.735 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.735 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.735 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.735 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.735 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.736 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.736 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.736 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.736 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.736 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.736 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.736 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.737 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.737 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.737 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.737 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.737 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.737 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.737 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.738 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.738 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.738 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.738 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.738 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.738 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mirroring", 2024-02-17T13:07:05.738 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.738 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.739 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.739 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.739 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.739 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.739 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.739 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.739 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.739 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.740 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.740 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.740 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.740 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.740 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.740 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.740 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.741 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.741 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.741 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.741 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.741 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.741 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.741 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.742 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.742 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.742 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.742 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.742 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.742 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.742 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.743 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.743 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.743 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.743 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.743 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.743 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.743 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.743 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.744 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.744 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.744 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.744 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.744 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.744 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.745 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.745 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.745 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.745 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.745 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.745 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.745 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.745 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.746 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.746 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.746 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.746 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.746 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.746 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.746 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.747 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.747 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.747 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.747 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.747 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.747 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.747 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.748 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.748 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.748 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.748 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.748 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.748 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.749 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.749 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.749 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.749 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "nfs", 2024-02-17T13:07:05.749 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.749 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.749 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.749 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.750 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.750 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.750 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.750 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.750 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.750 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.750 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.751 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.751 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.751 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.751 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.751 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.751 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.751 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.751 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.752 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.752 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.752 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.752 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.752 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.752 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.752 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.753 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.753 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.753 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.753 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.753 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.753 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.753 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.754 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.754 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.754 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.754 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.754 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.754 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.754 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.754 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.755 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.755 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.755 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.755 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.755 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.755 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.755 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.755 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.756 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.756 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.756 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.756 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.756 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.756 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.756 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.757 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.757 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.757 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.757 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.757 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.757 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.757 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.758 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.758 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.758 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.758 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.758 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.758 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.758 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.759 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.759 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.759 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.759 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.759 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.759 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.759 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "orchestrator", 2024-02-17T13:07:05.760 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.760 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.760 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.760 INFO:teuthology.orchestra.run.smithi018.stdout: "fail_fs": { 2024-02-17T13:07:05.760 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "fail_fs", 2024-02-17T13:07:05.760 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.761 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.761 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.761 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.761 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.761 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.761 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.761 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-17T13:07:05.762 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.762 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.762 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.762 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.762 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.762 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.762 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.763 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.763 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.763 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.763 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.763 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.763 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.763 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.763 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.764 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.764 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.764 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.764 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.764 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.764 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.764 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.765 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.765 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.765 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.765 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.765 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.765 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.765 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.766 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.766 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.766 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.766 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.766 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.766 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.766 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.767 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.767 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.767 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.767 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.767 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.767 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.767 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.768 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.768 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.768 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.768 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.768 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.768 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.769 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.769 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.769 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.769 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.769 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.769 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.769 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.770 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.770 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.770 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.770 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.770 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.770 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.770 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.771 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.771 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.771 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.771 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.771 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.771 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.771 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.772 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.772 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.772 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.772 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.772 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator": { 2024-02-17T13:07:05.772 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "orchestrator", 2024-02-17T13:07:05.772 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.773 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.773 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.773 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.773 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.773 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.773 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.773 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm", 2024-02-17T13:07:05.774 INFO:teuthology.orchestra.run.smithi018.stdout: "rook", 2024-02-17T13:07:05.774 INFO:teuthology.orchestra.run.smithi018.stdout: "test_orchestrator" 2024-02-17T13:07:05.774 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.774 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Orchestrator backend", 2024-02-17T13:07:05.774 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.774 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.774 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.775 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.775 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.775 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.775 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.775 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "osd_perf_query", 2024-02-17T13:07:05.775 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.775 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.776 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.776 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.776 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.776 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.776 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.776 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.776 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.777 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.777 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.777 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.777 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.777 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.777 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.777 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.778 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.778 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.778 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.778 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.778 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.778 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.778 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.778 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.779 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.779 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.779 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.779 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.779 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.779 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.779 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.780 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.780 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.780 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.780 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.780 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.780 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.780 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.781 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.781 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.781 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.781 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.781 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.781 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.781 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.781 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.782 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.782 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.782 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.782 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.782 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.782 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.782 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.782 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.783 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.783 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.783 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.783 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.783 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.783 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.783 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.783 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.784 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.784 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.784 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.784 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.784 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.784 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.784 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.785 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.785 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.785 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.785 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.785 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.785 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.786 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "osd_support", 2024-02-17T13:07:05.786 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.786 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.786 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.786 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.786 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.786 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.787 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.787 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.787 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.787 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.787 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.787 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.787 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.788 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.788 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.788 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.788 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.788 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.788 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.788 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.789 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.789 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.789 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.789 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.789 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.789 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.789 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.789 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.790 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.790 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.790 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.790 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.790 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.790 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.791 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.791 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.791 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.791 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.791 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.791 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.791 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.792 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.792 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.792 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.792 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.792 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.792 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.792 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.793 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.793 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.793 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.793 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.793 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.793 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.794 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.794 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.794 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.794 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.794 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.794 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.794 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.795 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.795 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.795 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.795 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.795 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.795 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.795 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.796 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.796 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.796 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.796 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.796 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.796 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.796 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.797 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.797 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pg_autoscaler", 2024-02-17T13:07:05.797 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.797 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.797 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.797 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.797 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.798 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.798 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.798 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.798 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.798 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.798 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.798 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.799 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.799 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.799 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.799 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.799 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.799 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.799 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.799 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.800 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.800 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.800 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.800 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.800 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.800 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.801 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.801 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.801 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.801 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.801 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.801 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.801 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.801 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.802 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.802 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.802 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.802 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.802 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.802 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.802 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.803 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.803 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.803 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.803 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.803 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.803 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.803 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.804 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.804 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.804 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.804 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.804 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.804 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.804 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.805 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.805 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.805 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.805 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.805 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.805 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.805 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.805 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.806 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.806 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.806 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.806 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.806 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.806 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.806 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.807 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.807 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.807 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.807 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:05.807 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:05.807 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.807 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.808 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.808 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:05.808 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.808 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.808 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.808 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.808 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.809 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.809 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.809 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.809 INFO:teuthology.orchestra.run.smithi018.stdout: "threshold": { 2024-02-17T13:07:05.809 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "threshold", 2024-02-17T13:07:05.809 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:05.809 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.810 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.810 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3.0", 2024-02-17T13:07:05.810 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1.0", 2024-02-17T13:07:05.810 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.810 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.810 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "scaling threshold", 2024-02-17T13:07:05.810 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-17T13:07:05.810 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.811 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.811 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.811 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.811 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.811 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.811 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "progress", 2024-02-17T13:07:05.811 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.811 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.812 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.812 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_pg_recovery_event": { 2024-02-17T13:07:05.812 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_pg_recovery_event", 2024-02-17T13:07:05.812 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.812 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.812 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.812 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.813 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.813 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.813 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.813 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-17T13:07:05.813 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.813 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.813 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.814 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.814 INFO:teuthology.orchestra.run.smithi018.stdout: "enabled": { 2024-02-17T13:07:05.814 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enabled", 2024-02-17T13:07:05.814 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.814 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.814 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.814 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.814 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.815 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.815 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.815 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.815 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.815 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.815 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.816 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.816 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.816 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.816 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.816 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.816 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.816 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.817 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.817 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.817 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.817 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.817 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.817 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.817 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.818 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.818 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.818 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.818 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.818 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.818 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.818 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.819 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.819 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.819 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.819 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.819 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.819 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.819 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.819 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.820 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.820 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.820 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.820 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.820 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.820 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.820 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.821 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.821 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.821 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.821 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.821 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.821 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.821 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.822 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.822 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.822 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.822 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.822 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.822 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.822 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.822 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.823 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.823 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.823 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.823 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.823 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.823 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.823 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.823 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.824 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.824 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.824 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.824 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.824 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.824 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.824 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.825 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.825 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.825 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.825 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.825 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.825 INFO:teuthology.orchestra.run.smithi018.stdout: "max_completed_events": { 2024-02-17T13:07:05.825 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_completed_events", 2024-02-17T13:07:05.826 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.826 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.826 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.826 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "50", 2024-02-17T13:07:05.826 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.826 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.826 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.827 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "number of past completed events to remember", 2024-02-17T13:07:05.827 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.827 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.827 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.827 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.827 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-02-17T13:07:05.827 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-02-17T13:07:05.828 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.828 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.828 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.828 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-02-17T13:07:05.828 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.828 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.828 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.829 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long the module is going to sleep", 2024-02-17T13:07:05.829 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.829 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.829 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.829 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.829 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.829 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.830 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.830 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prometheus", 2024-02-17T13:07:05.830 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.830 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.830 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.830 INFO:teuthology.orchestra.run.smithi018.stdout: "cache": { 2024-02-17T13:07:05.830 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cache", 2024-02-17T13:07:05.831 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.831 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.831 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.831 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.831 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.831 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.831 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.831 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.832 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.832 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.832 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.832 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.832 INFO:teuthology.orchestra.run.smithi018.stdout: "exclude_perf_counters": { 2024-02-17T13:07:05.832 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "exclude_perf_counters", 2024-02-17T13:07:05.832 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.833 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.833 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.833 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.833 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.833 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.833 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.833 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-17T13:07:05.834 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-17T13:07:05.834 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.834 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.834 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.834 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.834 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.834 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.835 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.835 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.835 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.835 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.835 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.835 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.835 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.835 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.836 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.836 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.836 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.836 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.836 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.836 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.836 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.837 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.837 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.837 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.837 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.837 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.837 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.838 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.838 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.838 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.838 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.838 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.838 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.838 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.839 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.839 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.839 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.839 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.839 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.839 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.839 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.839 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.840 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.840 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.840 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.840 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.840 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.840 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.841 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.841 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.841 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.841 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.841 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.841 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.841 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.841 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.842 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.842 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.842 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.842 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.842 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.842 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.842 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.843 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.843 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.843 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.843 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.843 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.843 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.843 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.844 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.844 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.844 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.844 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_stats_pools": { 2024-02-17T13:07:05.844 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_stats_pools", 2024-02-17T13:07:05.844 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.844 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.845 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.845 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.845 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.845 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.845 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.845 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.846 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.846 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.846 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.846 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.846 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-17T13:07:05.846 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-17T13:07:05.846 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "300", 2024-02-17T13:07:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.848 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.848 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.848 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.848 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.848 INFO:teuthology.orchestra.run.smithi018.stdout: "scrape_interval": { 2024-02-17T13:07:05.848 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "scrape_interval", 2024-02-17T13:07:05.848 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:05.849 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.849 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.849 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15.0", 2024-02-17T13:07:05.849 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.849 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.849 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.849 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.849 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.850 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.850 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.850 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.850 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-02-17T13:07:05.850 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-02-17T13:07:05.850 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.851 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.851 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.851 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "::", 2024-02-17T13:07:05.851 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.851 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.851 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.852 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-17T13:07:05.852 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.852 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.852 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.852 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.852 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-02-17T13:07:05.852 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-02-17T13:07:05.853 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.853 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.853 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.853 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "9283", 2024-02-17T13:07:05.853 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.853 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.854 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.854 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-17T13:07:05.854 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.854 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.854 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.854 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.855 INFO:teuthology.orchestra.run.smithi018.stdout: "stale_cache_strategy": { 2024-02-17T13:07:05.855 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "stale_cache_strategy", 2024-02-17T13:07:05.855 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.855 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.855 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.855 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "log", 2024-02-17T13:07:05.855 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.856 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.856 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.856 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.856 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.856 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.857 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.857 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.857 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_behaviour": { 2024-02-17T13:07:05.857 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_behaviour", 2024-02-17T13:07:05.857 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.857 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.857 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.858 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "default", 2024-02-17T13:07:05.858 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.858 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.858 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.858 INFO:teuthology.orchestra.run.smithi018.stdout: "default", 2024-02-17T13:07:05.858 INFO:teuthology.orchestra.run.smithi018.stdout: "error" 2024-02-17T13:07:05.859 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.859 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.859 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.859 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.859 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.859 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.859 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_error_status_code": { 2024-02-17T13:07:05.860 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_error_status_code", 2024-02-17T13:07:05.860 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.860 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.860 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.860 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "500", 2024-02-17T13:07:05.860 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "400", 2024-02-17T13:07:05.860 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "599", 2024-02-17T13:07:05.861 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.861 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.861 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.861 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.861 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.861 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.861 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.862 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.862 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.862 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_support", 2024-02-17T13:07:05.862 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.862 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.862 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.862 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.863 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.863 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.863 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.863 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.863 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.863 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.863 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.863 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.864 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.864 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.864 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.864 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.864 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.864 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.865 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.865 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.865 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.865 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.865 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.865 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.866 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.866 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.866 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.866 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.866 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.866 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.866 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.866 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.867 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.867 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.867 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.867 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.867 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.867 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.867 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.868 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.868 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.868 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.868 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.868 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.868 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.869 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.869 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.869 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.869 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.869 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.869 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.869 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.870 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.870 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.870 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.870 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.870 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.870 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.870 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.870 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.871 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.871 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.871 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.871 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.871 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.871 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.871 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.872 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.872 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.872 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.872 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.872 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.872 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.872 INFO:teuthology.orchestra.run.smithi018.stdout: "max_concurrent_snap_create": { 2024-02-17T13:07:05.873 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_concurrent_snap_create", 2024-02-17T13:07:05.873 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.873 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.873 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.873 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-02-17T13:07:05.873 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.873 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.874 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.874 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.874 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.874 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.874 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.874 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.874 INFO:teuthology.orchestra.run.smithi018.stdout: "mirror_snapshot_schedule": { 2024-02-17T13:07:05.875 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mirror_snapshot_schedule", 2024-02-17T13:07:05.875 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.875 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.875 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.875 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.875 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.875 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.876 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.876 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.876 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.876 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.876 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.876 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.876 INFO:teuthology.orchestra.run.smithi018.stdout: "trash_purge_schedule": { 2024-02-17T13:07:05.877 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "trash_purge_schedule", 2024-02-17T13:07:05.877 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.877 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.877 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.877 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.877 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.877 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.877 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.878 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.878 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.878 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.878 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.878 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.878 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.878 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.879 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.879 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "restful", 2024-02-17T13:07:05.879 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.879 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.879 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.879 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_auth": { 2024-02-17T13:07:05.879 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enable_auth", 2024-02-17T13:07:05.879 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.880 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.880 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.880 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.880 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.880 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.880 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.880 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.881 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.881 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.881 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.881 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.881 INFO:teuthology.orchestra.run.smithi018.stdout: "key_file": { 2024-02-17T13:07:05.881 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "key_file", 2024-02-17T13:07:05.881 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.882 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.882 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.882 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.882 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.882 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.882 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.882 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.882 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.883 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.883 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.883 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.883 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.883 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.883 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.883 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.884 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.884 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.884 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.884 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.884 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.884 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.885 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.885 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.885 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.885 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.885 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.885 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.885 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.886 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.886 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.886 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.886 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.886 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.886 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.886 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.887 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.887 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.887 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.887 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.887 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.887 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.887 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.888 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.888 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.888 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.888 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.888 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.888 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.888 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.888 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.889 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.889 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.889 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.889 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.889 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.889 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.889 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.890 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.890 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.890 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.890 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.890 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.890 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.891 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.891 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.891 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.891 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.891 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.891 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.891 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.892 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.892 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.892 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.892 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.892 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.892 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.892 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.893 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.893 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.893 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.893 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.893 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-02-17T13:07:05.893 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-02-17T13:07:05.893 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.893 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.894 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.894 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.894 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.894 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.894 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.894 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.894 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.895 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.895 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.895 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.895 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-02-17T13:07:05.895 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-02-17T13:07:05.895 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.895 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.896 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.896 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.896 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.896 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.896 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.896 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.896 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.897 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.897 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.897 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.897 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.897 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.897 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.897 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rgw", 2024-02-17T13:07:05.897 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.898 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.898 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.898 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.898 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.898 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.898 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.899 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.899 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.899 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.899 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.899 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.899 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.899 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.900 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.900 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.900 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.900 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.900 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.900 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.900 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.900 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.901 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.901 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.901 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.901 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.901 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.901 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.902 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.902 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.902 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.902 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.902 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.902 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.902 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.903 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.903 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.903 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.903 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.903 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.903 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.903 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.904 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.904 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.904 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.904 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.904 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.904 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.904 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.905 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.905 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.905 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.905 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.905 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.905 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.905 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.906 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.906 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.906 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.906 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.906 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.906 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.907 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.907 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.907 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.907 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.907 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.907 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.907 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.907 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.908 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.908 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.908 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.908 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.908 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.908 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.909 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rook", 2024-02-17T13:07:05.909 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.909 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.909 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.909 INFO:teuthology.orchestra.run.smithi018.stdout: "drive_group_interval": { 2024-02-17T13:07:05.909 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "drive_group_interval", 2024-02-17T13:07:05.909 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:05.910 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.910 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.910 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "300.0", 2024-02-17T13:07:05.910 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.910 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.910 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.910 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-17T13:07:05.911 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.911 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.911 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.911 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.911 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.911 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.911 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.912 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.912 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.912 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.912 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.912 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.912 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.913 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.913 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.913 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.913 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.913 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.913 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.913 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.914 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.914 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.914 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.914 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.914 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.914 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.914 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.915 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.915 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.915 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.915 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.915 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.915 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.915 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.915 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.916 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.916 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.916 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.916 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.916 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.916 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.916 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.917 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.917 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.917 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.917 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.917 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.917 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.917 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.918 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.918 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.918 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.918 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.918 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.918 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.918 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.918 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.919 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.919 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.919 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.919 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.919 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.919 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.919 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.919 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.920 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.920 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.920 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.920 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.920 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.920 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.920 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.921 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.921 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.921 INFO:teuthology.orchestra.run.smithi018.stdout: "storage_class": { 2024-02-17T13:07:05.921 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "storage_class", 2024-02-17T13:07:05.921 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.921 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.921 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.922 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "local", 2024-02-17T13:07:05.922 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.922 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.922 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.922 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-17T13:07:05.922 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.922 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.922 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.923 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.923 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.923 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.923 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.923 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "selftest", 2024-02-17T13:07:05.923 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.923 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.924 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.924 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.924 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.924 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.924 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.924 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.924 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.925 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.925 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.925 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.925 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.925 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.925 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.925 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.926 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.926 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.926 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.926 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.926 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.926 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.926 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.927 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.927 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.927 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.927 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.927 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.927 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.927 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.927 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.928 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.928 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.928 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.928 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.928 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.928 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.928 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.929 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.929 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.929 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.929 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.929 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.929 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.929 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.930 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.930 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.930 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.930 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.930 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.930 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.930 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.931 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.931 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.931 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.931 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.931 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.931 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.931 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.932 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.932 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.932 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.932 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.932 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.932 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.932 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.932 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.933 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.933 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.933 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.933 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.933 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.933 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.933 INFO:teuthology.orchestra.run.smithi018.stdout: "roption1": { 2024-02-17T13:07:05.934 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "roption1", 2024-02-17T13:07:05.934 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.934 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.934 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.934 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.934 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.934 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.935 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.935 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.935 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.935 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.935 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.935 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.935 INFO:teuthology.orchestra.run.smithi018.stdout: "roption2": { 2024-02-17T13:07:05.935 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "roption2", 2024-02-17T13:07:05.936 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.936 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.936 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.936 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "xyz", 2024-02-17T13:07:05.936 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.936 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.936 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.937 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.937 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.937 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.937 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.937 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.937 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption1": { 2024-02-17T13:07:05.937 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption1", 2024-02-17T13:07:05.938 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.938 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.938 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.938 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.938 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.938 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.938 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.938 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.939 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.939 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.939 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.939 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.939 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption2": { 2024-02-17T13:07:05.939 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption2", 2024-02-17T13:07:05.939 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.940 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.940 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.940 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.940 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.940 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.940 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.941 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.941 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.941 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.941 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.941 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.941 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption3": { 2024-02-17T13:07:05.942 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption3", 2024-02-17T13:07:05.942 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-02-17T13:07:05.942 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.942 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.942 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.942 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.942 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.943 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.943 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.943 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.943 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.943 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.943 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.943 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption4": { 2024-02-17T13:07:05.944 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption4", 2024-02-17T13:07:05.944 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.944 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.944 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.944 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.944 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.944 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.945 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.945 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.945 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.945 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.945 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.945 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.945 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption5": { 2024-02-17T13:07:05.945 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption5", 2024-02-17T13:07:05.946 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.946 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.946 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.946 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.946 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.946 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.947 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.947 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.947 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.947 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.947 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.947 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption6": { 2024-02-17T13:07:05.948 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption6", 2024-02-17T13:07:05.948 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.948 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.948 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.948 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:05.948 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.949 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.949 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.949 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.949 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.949 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.949 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.950 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption7": { 2024-02-17T13:07:05.950 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption7", 2024-02-17T13:07:05.950 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:05.950 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.950 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.950 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.950 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-02-17T13:07:05.951 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "42", 2024-02-17T13:07:05.951 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.951 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.951 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.951 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.951 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.951 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.952 INFO:teuthology.orchestra.run.smithi018.stdout: "testkey": { 2024-02-17T13:07:05.952 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testkey", 2024-02-17T13:07:05.952 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.952 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.952 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.952 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.953 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.953 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.953 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.953 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.953 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.953 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.953 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.953 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.954 INFO:teuthology.orchestra.run.smithi018.stdout: "testlkey": { 2024-02-17T13:07:05.954 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testlkey", 2024-02-17T13:07:05.954 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.954 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.954 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.954 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.954 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.955 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.955 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.955 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.955 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.955 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.955 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.955 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.956 INFO:teuthology.orchestra.run.smithi018.stdout: "testnewline": { 2024-02-17T13:07:05.956 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testnewline", 2024-02-17T13:07:05.956 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.956 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.956 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.956 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.956 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.957 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.957 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.957 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.957 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.957 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.957 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.957 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.957 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.958 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.958 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.958 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snap_schedule", 2024-02-17T13:07:05.958 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.958 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.958 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.958 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_m_granularity": { 2024-02-17T13:07:05.959 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_m_granularity", 2024-02-17T13:07:05.959 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.959 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.959 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.959 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.959 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.960 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.960 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.960 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow minute scheduled snapshots", 2024-02-17T13:07:05.960 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.960 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.960 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.960 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.961 INFO:teuthology.orchestra.run.smithi018.stdout: "dump_on_update": { 2024-02-17T13:07:05.961 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "dump_on_update", 2024-02-17T13:07:05.961 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.961 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.961 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.961 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.961 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.962 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.962 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.962 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "dump database to debug log on update", 2024-02-17T13:07:05.962 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.962 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.962 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.962 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.963 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.963 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.963 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.963 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.963 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.963 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.964 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.964 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.964 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.964 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.964 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.964 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.964 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.965 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.965 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.965 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.965 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.965 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.965 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.965 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.966 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.966 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.966 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.966 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.966 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.966 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.966 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.966 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.967 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.967 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.967 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.967 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.967 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.967 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.968 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.968 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.968 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.968 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.968 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.968 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.968 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.969 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.969 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.969 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.969 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.969 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.969 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.970 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.970 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.970 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.970 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.970 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.970 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.971 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.971 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.971 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.971 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.971 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.971 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.971 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.972 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.972 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.972 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.972 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.972 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.973 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.973 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.973 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.973 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.973 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.973 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.973 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.974 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.974 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "stats", 2024-02-17T13:07:05.974 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.974 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.974 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.974 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.975 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.975 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.975 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.975 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.975 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.975 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.976 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.976 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.976 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.976 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.976 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.977 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.977 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.977 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.977 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.977 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.977 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.978 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.978 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.978 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.978 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.978 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.978 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.979 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.979 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.979 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.979 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.979 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.979 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.980 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.980 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.980 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.980 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.980 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.980 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.981 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.981 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.981 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.981 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.981 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.981 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.982 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.982 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.982 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.982 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.982 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.982 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.982 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.982 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.983 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.983 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.983 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.983 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.983 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.983 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.984 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.984 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.984 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.984 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.984 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.984 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.984 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.985 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.985 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.985 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.985 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.985 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.985 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.985 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.986 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.986 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.986 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.986 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "status", 2024-02-17T13:07:05.986 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.986 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.986 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.986 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:05.987 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:05.987 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.987 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.987 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.987 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:05.987 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.988 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.988 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.988 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.988 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.988 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.988 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.988 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.988 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.989 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.989 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.989 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.989 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.989 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.989 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.989 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:05.989 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:05.990 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.990 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.990 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.990 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.990 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.990 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.990 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.991 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.991 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.991 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.991 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.991 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.991 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:05.991 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:05.992 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.992 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.992 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.992 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:05.992 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.992 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.992 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:05.993 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:05.993 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:05.993 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:05.993 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:05.993 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:05.993 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:05.993 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:05.993 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.994 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.994 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.994 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.994 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.994 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:05.994 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:05.994 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:05.995 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.995 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:05.995 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:05.995 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.995 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.995 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.995 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.996 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.996 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.996 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.996 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.996 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:05.996 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.996 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:05.997 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "telegraf", 2024-02-17T13:07:05.997 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:05.997 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:05.997 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:05.997 INFO:teuthology.orchestra.run.smithi018.stdout: "address": { 2024-02-17T13:07:05.997 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "address", 2024-02-17T13:07:05.997 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:05.998 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:05.998 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:05.998 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-17T13:07:05.998 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:05.998 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:05.998 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:05.998 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:05.999 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.000 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.000 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15", 2024-02-17T13:07:06.000 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.000 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.000 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.000 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.000 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.001 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.001 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.001 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.001 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:06.001 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:06.001 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.001 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.002 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.002 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:06.002 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.002 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.002 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:06.002 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:06.002 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:06.002 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:06.003 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:06.003 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:06.003 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:06.003 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.003 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.003 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.003 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.004 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.004 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.004 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:06.004 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:06.004 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.004 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.004 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.005 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.005 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.005 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.005 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.005 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.005 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.005 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.006 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.006 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:06.006 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:06.006 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.006 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.006 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.007 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:06.007 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.007 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.007 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:06.007 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:06.007 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:06.007 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:06.007 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:06.008 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:06.008 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:06.008 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.008 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.008 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.008 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.009 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.009 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.009 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:06.009 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:06.009 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.009 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.009 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.010 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.010 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.010 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.010 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.010 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.010 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.010 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.011 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.011 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.011 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.011 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.011 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.011 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "telemetry", 2024-02-17T13:07:06.011 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:06.012 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:06.012 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:06.012 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_basic": { 2024-02-17T13:07:06.012 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_basic", 2024-02-17T13:07:06.012 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.012 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.012 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.012 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:06.013 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.013 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.013 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.013 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-17T13:07:06.013 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.013 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.013 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.014 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_crash": { 2024-02-17T13:07:06.014 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_crash", 2024-02-17T13:07:06.014 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.014 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.014 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.014 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:06.015 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.015 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.015 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.015 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-17T13:07:06.015 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.015 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.016 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.016 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.016 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_device": { 2024-02-17T13:07:06.016 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_device", 2024-02-17T13:07:06.017 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.017 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.017 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.017 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-02-17T13:07:06.017 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.018 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.018 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.018 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-17T13:07:06.018 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.018 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.018 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.018 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.018 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_ident": { 2024-02-17T13:07:06.019 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_ident", 2024-02-17T13:07:06.019 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.019 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.019 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.019 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.019 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.019 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.020 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.020 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-17T13:07:06.020 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.020 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.020 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.020 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.020 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_perf": { 2024-02-17T13:07:06.021 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_perf", 2024-02-17T13:07:06.021 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.021 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.021 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.021 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.021 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.021 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.021 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.022 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-17T13:07:06.022 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.022 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.022 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.022 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.022 INFO:teuthology.orchestra.run.smithi018.stdout: "contact": { 2024-02-17T13:07:06.022 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "contact", 2024-02-17T13:07:06.022 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.023 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.023 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.023 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:06.023 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.023 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.023 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.023 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.024 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.024 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.024 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.024 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.024 INFO:teuthology.orchestra.run.smithi018.stdout: "description": { 2024-02-17T13:07:06.024 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "description", 2024-02-17T13:07:06.024 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.025 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.025 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.025 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:06.025 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.025 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.025 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.025 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.026 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.026 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.026 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.026 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.026 INFO:teuthology.orchestra.run.smithi018.stdout: "device_url": { 2024-02-17T13:07:06.026 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_url", 2024-02-17T13:07:06.026 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.027 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.027 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.027 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-17T13:07:06.027 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.027 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.027 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.027 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.027 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.028 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.028 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.028 INFO:teuthology.orchestra.run.smithi018.stdout: "enabled": { 2024-02-17T13:07:06.028 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enabled", 2024-02-17T13:07:06.028 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.028 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.029 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.029 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.029 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.029 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.029 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.029 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.029 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.030 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.030 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.030 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.030 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:06.030 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:06.030 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:06.030 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.031 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.031 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "24", 2024-02-17T13:07:06.031 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "8", 2024-02-17T13:07:06.031 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.031 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.031 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.031 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.032 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.032 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.032 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.032 INFO:teuthology.orchestra.run.smithi018.stdout: "last_opt_revision": { 2024-02-17T13:07:06.032 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "last_opt_revision", 2024-02-17T13:07:06.032 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:06.032 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.033 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.033 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1", 2024-02-17T13:07:06.033 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.033 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.033 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.033 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.033 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.034 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.034 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.034 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.034 INFO:teuthology.orchestra.run.smithi018.stdout: "leaderboard": { 2024-02-17T13:07:06.034 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "leaderboard", 2024-02-17T13:07:06.034 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.034 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.035 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.035 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.035 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.035 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.035 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.035 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.035 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.036 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.036 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.036 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.036 INFO:teuthology.orchestra.run.smithi018.stdout: "leaderboard_description": { 2024-02-17T13:07:06.036 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "leaderboard_description", 2024-02-17T13:07:06.036 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.037 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.037 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.037 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:06.037 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.037 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.037 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.037 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.038 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.038 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.038 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.038 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.038 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:06.038 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:06.038 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.039 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.039 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.039 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:06.039 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.039 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.039 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:06.039 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:06.039 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:06.040 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:06.040 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:06.040 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:06.040 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:06.040 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.040 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.040 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.041 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.041 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.041 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.041 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:06.041 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:06.041 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.041 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.041 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.042 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.042 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.042 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.042 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.042 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.042 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.042 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.043 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.043 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.043 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:06.043 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:06.043 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.043 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.043 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.044 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:06.044 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.044 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.044 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:06.044 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:06.044 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:06.044 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:06.044 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:06.045 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:06.045 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:06.045 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.045 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.045 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.045 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.045 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.046 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.046 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:06.046 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:06.046 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.046 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.046 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.046 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.046 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.047 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.047 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.047 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.047 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.047 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.047 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.047 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.048 INFO:teuthology.orchestra.run.smithi018.stdout: "organization": { 2024-02-17T13:07:06.048 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "organization", 2024-02-17T13:07:06.048 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.048 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.048 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.048 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:06.048 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.049 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.049 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.049 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.049 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.049 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.049 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.049 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.049 INFO:teuthology.orchestra.run.smithi018.stdout: "proxy": { 2024-02-17T13:07:06.050 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "proxy", 2024-02-17T13:07:06.050 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.050 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.050 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.050 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:06.050 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.050 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.051 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.051 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.051 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.051 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.051 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.051 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.052 INFO:teuthology.orchestra.run.smithi018.stdout: "url": { 2024-02-17T13:07:06.052 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "url", 2024-02-17T13:07:06.052 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.052 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.052 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.052 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-17T13:07:06.052 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.052 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.053 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.053 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.053 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.053 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.053 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.053 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.053 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.054 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.054 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.054 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "test_orchestrator", 2024-02-17T13:07:06.054 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:06.054 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:06.054 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:06.055 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:06.055 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:06.055 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.055 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.055 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.055 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:06.056 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.056 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.056 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:06.056 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:06.056 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:06.057 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:06.057 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:06.057 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:06.057 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:06.057 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.057 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.058 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.058 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.058 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.058 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.058 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:06.058 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:06.058 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.059 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.059 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.059 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.059 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.059 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.059 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.059 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.060 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.060 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.060 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.060 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.060 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:06.060 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:06.060 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.061 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.061 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.061 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:06.061 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.061 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.061 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:06.061 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:06.062 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:06.062 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:06.062 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:06.062 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:06.062 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:06.062 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.062 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.063 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.063 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.063 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.063 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.063 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:06.063 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:06.063 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.064 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.064 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.064 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.064 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.064 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.064 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.064 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.065 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.065 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.065 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.065 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.065 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.065 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.065 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.066 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "volumes", 2024-02-17T13:07:06.066 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:06.066 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:06.066 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:06.066 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:06.066 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:06.066 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.067 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.067 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.067 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:06.067 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.067 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.067 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:06.067 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:06.068 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:06.068 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:06.068 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:06.068 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:06.068 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:06.068 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.068 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.069 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.069 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.069 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.069 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.069 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:06.069 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:06.069 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.070 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.070 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.070 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.070 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.070 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.070 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.070 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.070 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.071 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.071 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.071 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.071 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:06.071 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:06.071 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.071 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.072 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.072 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:06.072 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.072 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.072 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:06.072 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:06.072 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:06.073 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:06.073 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:06.073 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:06.073 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:06.073 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.073 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.074 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.074 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.074 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.074 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.074 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:06.074 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:06.074 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.075 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.075 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.075 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.075 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.075 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.075 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.075 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.076 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.076 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.076 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.076 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.076 INFO:teuthology.orchestra.run.smithi018.stdout: "max_concurrent_clones": { 2024-02-17T13:07:06.076 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_concurrent_clones", 2024-02-17T13:07:06.076 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:06.076 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.077 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.077 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "4", 2024-02-17T13:07:06.077 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.077 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.077 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.077 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-17T13:07:06.077 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.078 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.078 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.078 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.078 INFO:teuthology.orchestra.run.smithi018.stdout: "periodic_async_work": { 2024-02-17T13:07:06.078 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "periodic_async_work", 2024-02-17T13:07:06.078 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.078 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.079 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.079 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.079 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.079 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.079 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.079 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Periodically check for async work", 2024-02-17T13:07:06.079 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.080 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.080 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.080 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.080 INFO:teuthology.orchestra.run.smithi018.stdout: "snapshot_clone_delay": { 2024-02-17T13:07:06.080 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snapshot_clone_delay", 2024-02-17T13:07:06.080 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:06.080 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.081 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.081 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-02-17T13:07:06.081 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.081 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.081 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.081 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-17T13:07:06.082 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.082 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.082 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.082 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.082 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.082 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.082 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.082 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix", 2024-02-17T13:07:06.083 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-02-17T13:07:06.083 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-02-17T13:07:06.083 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-02-17T13:07:06.083 INFO:teuthology.orchestra.run.smithi018.stdout: "discovery_interval": { 2024-02-17T13:07:06.083 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "discovery_interval", 2024-02-17T13:07:06.083 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-02-17T13:07:06.083 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.083 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.084 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "100", 2024-02-17T13:07:06.084 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.084 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.084 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.084 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.084 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.084 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.085 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.085 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.085 INFO:teuthology.orchestra.run.smithi018.stdout: "identifier": { 2024-02-17T13:07:06.085 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "identifier", 2024-02-17T13:07:06.085 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.085 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.085 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.086 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:06.086 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.086 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.086 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.086 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.086 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.086 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.086 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.087 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.087 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-02-17T13:07:06.087 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-02-17T13:07:06.087 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-02-17T13:07:06.087 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.087 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.087 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-02-17T13:07:06.088 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.088 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.088 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.088 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.088 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.088 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.088 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.089 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.089 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-02-17T13:07:06.089 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-02-17T13:07:06.090 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.091 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.091 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.091 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:06.091 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.091 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.091 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:06.091 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:06.092 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:06.092 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:06.092 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:06.092 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:06.092 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:06.092 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.092 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.093 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.093 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.093 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.093 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.093 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-02-17T13:07:06.093 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-02-17T13:07:06.094 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.094 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.094 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.094 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.094 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.094 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.095 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.095 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.095 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.095 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.095 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.095 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.095 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-02-17T13:07:06.096 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-02-17T13:07:06.096 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.096 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.096 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.096 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-02-17T13:07:06.096 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.097 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.097 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-02-17T13:07:06.097 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-02-17T13:07:06.097 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-02-17T13:07:06.097 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-02-17T13:07:06.098 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-02-17T13:07:06.098 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-02-17T13:07:06.098 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-02-17T13:07:06.098 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.098 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.098 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.099 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.099 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.099 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.099 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-02-17T13:07:06.099 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-02-17T13:07:06.099 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-02-17T13:07:06.100 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.100 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-02-17T13:07:06.100 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-02-17T13:07:06.100 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.100 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.101 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.101 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.101 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.101 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.101 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.101 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.102 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_host": { 2024-02-17T13:07:06.102 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_host", 2024-02-17T13:07:06.102 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.102 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.102 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.102 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-02-17T13:07:06.102 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.102 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.103 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.103 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.103 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.103 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.103 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.103 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.103 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_port": { 2024-02-17T13:07:06.104 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_port", 2024-02-17T13:07:06.104 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-02-17T13:07:06.104 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.104 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.104 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10051", 2024-02-17T13:07:06.104 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.104 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.105 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.105 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.105 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.105 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.105 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.105 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.105 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_sender": { 2024-02-17T13:07:06.106 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_sender", 2024-02-17T13:07:06.106 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-02-17T13:07:06.106 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-02-17T13:07:06.106 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-02-17T13:07:06.106 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-17T13:07:06.106 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-02-17T13:07:06.106 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-02-17T13:07:06.107 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-02-17T13:07:06.107 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-02-17T13:07:06.107 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-02-17T13:07:06.107 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-02-17T13:07:06.107 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-02-17T13:07:06.107 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.107 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.108 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.108 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.108 INFO:teuthology.orchestra.run.smithi018.stdout: "services": { 2024-02-17T13:07:06.108 INFO:teuthology.orchestra.run.smithi018.stdout: "dashboard": "https://172.21.15.18:8443/" 2024-02-17T13:07:06.108 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.108 INFO:teuthology.orchestra.run.smithi018.stdout: "always_on_modules": { 2024-02-17T13:07:06.108 INFO:teuthology.orchestra.run.smithi018.stdout: "octopus": [ 2024-02-17T13:07:06.109 INFO:teuthology.orchestra.run.smithi018.stdout: "balancer", 2024-02-17T13:07:06.109 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-02-17T13:07:06.109 INFO:teuthology.orchestra.run.smithi018.stdout: "devicehealth", 2024-02-17T13:07:06.109 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator", 2024-02-17T13:07:06.109 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscaler", 2024-02-17T13:07:06.109 INFO:teuthology.orchestra.run.smithi018.stdout: "progress", 2024-02-17T13:07:06.109 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_support", 2024-02-17T13:07:06.109 INFO:teuthology.orchestra.run.smithi018.stdout: "status", 2024-02-17T13:07:06.110 INFO:teuthology.orchestra.run.smithi018.stdout: "telemetry", 2024-02-17T13:07:06.110 INFO:teuthology.orchestra.run.smithi018.stdout: "volumes" 2024-02-17T13:07:06.110 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.110 INFO:teuthology.orchestra.run.smithi018.stdout: "pacific": [ 2024-02-17T13:07:06.110 INFO:teuthology.orchestra.run.smithi018.stdout: "balancer", 2024-02-17T13:07:06.110 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-02-17T13:07:06.110 INFO:teuthology.orchestra.run.smithi018.stdout: "devicehealth", 2024-02-17T13:07:06.110 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator", 2024-02-17T13:07:06.111 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscaler", 2024-02-17T13:07:06.111 INFO:teuthology.orchestra.run.smithi018.stdout: "progress", 2024-02-17T13:07:06.111 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_support", 2024-02-17T13:07:06.111 INFO:teuthology.orchestra.run.smithi018.stdout: "status", 2024-02-17T13:07:06.111 INFO:teuthology.orchestra.run.smithi018.stdout: "telemetry", 2024-02-17T13:07:06.111 INFO:teuthology.orchestra.run.smithi018.stdout: "volumes" 2024-02-17T13:07:06.111 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.112 INFO:teuthology.orchestra.run.smithi018.stdout: "quincy": [ 2024-02-17T13:07:06.112 INFO:teuthology.orchestra.run.smithi018.stdout: "balancer", 2024-02-17T13:07:06.112 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-02-17T13:07:06.112 INFO:teuthology.orchestra.run.smithi018.stdout: "devicehealth", 2024-02-17T13:07:06.112 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator", 2024-02-17T13:07:06.112 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscaler", 2024-02-17T13:07:06.112 INFO:teuthology.orchestra.run.smithi018.stdout: "progress", 2024-02-17T13:07:06.113 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_support", 2024-02-17T13:07:06.113 INFO:teuthology.orchestra.run.smithi018.stdout: "status", 2024-02-17T13:07:06.113 INFO:teuthology.orchestra.run.smithi018.stdout: "telemetry", 2024-02-17T13:07:06.113 INFO:teuthology.orchestra.run.smithi018.stdout: "volumes" 2024-02-17T13:07:06.113 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.113 INFO:teuthology.orchestra.run.smithi018.stdout: "reef": [ 2024-02-17T13:07:06.113 INFO:teuthology.orchestra.run.smithi018.stdout: "balancer", 2024-02-17T13:07:06.113 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-02-17T13:07:06.114 INFO:teuthology.orchestra.run.smithi018.stdout: "devicehealth", 2024-02-17T13:07:06.114 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator", 2024-02-17T13:07:06.114 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscaler", 2024-02-17T13:07:06.114 INFO:teuthology.orchestra.run.smithi018.stdout: "progress", 2024-02-17T13:07:06.114 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_support", 2024-02-17T13:07:06.114 INFO:teuthology.orchestra.run.smithi018.stdout: "status", 2024-02-17T13:07:06.114 INFO:teuthology.orchestra.run.smithi018.stdout: "telemetry", 2024-02-17T13:07:06.114 INFO:teuthology.orchestra.run.smithi018.stdout: "volumes" 2024-02-17T13:07:06.115 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-02-17T13:07:06.115 INFO:teuthology.orchestra.run.smithi018.stdout: "squid": [ 2024-02-17T13:07:06.115 INFO:teuthology.orchestra.run.smithi018.stdout: "balancer", 2024-02-17T13:07:06.115 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-02-17T13:07:06.115 INFO:teuthology.orchestra.run.smithi018.stdout: "devicehealth", 2024-02-17T13:07:06.115 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator", 2024-02-17T13:07:06.115 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscaler", 2024-02-17T13:07:06.115 INFO:teuthology.orchestra.run.smithi018.stdout: "progress", 2024-02-17T13:07:06.116 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_support", 2024-02-17T13:07:06.116 INFO:teuthology.orchestra.run.smithi018.stdout: "status", 2024-02-17T13:07:06.116 INFO:teuthology.orchestra.run.smithi018.stdout: "telemetry", 2024-02-17T13:07:06.116 INFO:teuthology.orchestra.run.smithi018.stdout: "volumes" 2024-02-17T13:07:06.116 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-02-17T13:07:06.116 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.116 INFO:teuthology.orchestra.run.smithi018.stdout: "last_failure_osd_epoch": 89, 2024-02-17T13:07:06.116 INFO:teuthology.orchestra.run.smithi018.stdout: "active_clients": [ 2024-02-17T13:07:06.117 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.117 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "devicehealth", 2024-02-17T13:07:06.117 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-02-17T13:07:06.117 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.117 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-02-17T13:07:06.117 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-02-17T13:07:06.117 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 837888122 2024-02-17T13:07:06.118 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.118 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-02-17T13:07:06.118 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.118 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.118 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "libcephsqlite", 2024-02-17T13:07:06.118 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-02-17T13:07:06.118 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.119 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-02-17T13:07:06.119 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-02-17T13:07:06.119 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 141893338 2024-02-17T13:07:06.119 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.119 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-02-17T13:07:06.119 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.119 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.119 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_support", 2024-02-17T13:07:06.120 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-02-17T13:07:06.120 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.120 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-02-17T13:07:06.120 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-02-17T13:07:06.120 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 1853759493 2024-02-17T13:07:06.120 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.120 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-02-17T13:07:06.121 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.121 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.121 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snap_schedule", 2024-02-17T13:07:06.121 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-02-17T13:07:06.121 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.121 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-02-17T13:07:06.121 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-02-17T13:07:06.121 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 1832442328 2024-02-17T13:07:06.122 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.122 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-02-17T13:07:06.122 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.122 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.122 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snap_schedule", 2024-02-17T13:07:06.122 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-02-17T13:07:06.122 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.123 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-02-17T13:07:06.123 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-02-17T13:07:06.123 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2757558601 2024-02-17T13:07:06.123 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.123 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-02-17T13:07:06.123 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:06.123 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.124 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "volumes", 2024-02-17T13:07:06.124 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-02-17T13:07:06.124 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-02-17T13:07:06.124 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-02-17T13:07:06.124 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-02-17T13:07:06.124 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 1417385699 2024-02-17T13:07:06.124 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.125 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-02-17T13:07:06.125 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:06.125 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-02-17T13:07:06.125 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:07:06.127 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:04 smithi119 ceph-mon[36188]: pgmap v540: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 676 op/s 2024-02-17T13:07:06.127 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:04 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1837246601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T13:07:06.127 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:04 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2350389238' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-17T13:07:06.127 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:04 smithi146 ceph-mon[36454]: pgmap v540: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 676 op/s 2024-02-17T13:07:06.127 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:04 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1837246601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T13:07:06.128 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:04 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2350389238' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-17T13:07:06.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.772+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/2350389238 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2e6c0a2230 msgr2=0x7f2e6c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:06.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.772+0000 7f2e8c3e2640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2e6c0a2230 0x7f2e6c0a46f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2e74005fd0 tx=0x7f2e74009500 comp rx=0 tx=0).stop 2024-02-17T13:07:06.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.772+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/2350389238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e84106450 msgr2=0x7f2e840739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:06.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.772+0000 7f2e8c3e2640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e84106450 0x7f2e840739c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e9c0 tx=0x7f2e7800ee80 comp rx=0 tx=0).stop 2024-02-17T13:07:06.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.773+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/2350389238 shutdown_connections 2024-02-17T13:07:06.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.773+0000 7f2e8c3e2640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f2e6c0a2230 0x7f2e6c0a46f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:06.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.773+0000 7f2e8c3e2640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e84073f00 0x7f2e841a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:06.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.773+0000 7f2e8c3e2640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e84106450 0x7f2e840739c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:06.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.773+0000 7f2e8c3e2640 1 --2- 172.21.15.18:0/2350389238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e84101380 0x7f2e84073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:06.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.773+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/2350389238 >> 172.21.15.18:0/2350389238 conn(0x7f2e840fcc20 msgr2=0x7f2e840fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:06.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.773+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/2350389238 shutdown_connections 2024-02-17T13:07:06.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:03.773+0000 7f2e8c3e2640 1 -- 172.21.15.18:0/2350389238 wait complete. 2024-02-17T13:07:06.136 DEBUG:tasks.check_counter:Getting stats from a 2024-02-17T13:07:06.136 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph daemon mds.a perf dump 2024-02-17T13:07:06.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:04 smithi018 ceph-mon[32625]: pgmap v540: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 676 op/s 2024-02-17T13:07:06.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:04 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1837246601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T13:07:06.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:04 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2350389238' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-17T13:07:06.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:06 smithi146 ceph-mon[36454]: pgmap v541: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 837 op/s 2024-02-17T13:07:06.448 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:06 smithi146 ceph-mon[36454]: osdmap e112: 12 total, 12 up, 12 in 2024-02-17T13:07:06.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:06 smithi018 ceph-mon[32625]: pgmap v541: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 837 op/s 2024-02-17T13:07:06.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:06 smithi018 ceph-mon[32625]: osdmap e112: 12 total, 12 up, 12 in 2024-02-17T13:07:06.545 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T13:07:06.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:06 smithi119 ceph-mon[36188]: pgmap v541: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 837 op/s 2024-02-17T13:07:06.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:06 smithi119 ceph-mon[36188]: osdmap e112: 12 total, 12 up, 12 in 2024-02-17T13:07:08.347 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:08 smithi018 ceph-mon[32625]: pgmap v543: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 671 op/s 2024-02-17T13:07:08.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:08 smithi146 ceph-mon[36454]: pgmap v543: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 671 op/s 2024-02-17T13:07:08.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:08 smithi119 ceph-mon[36188]: pgmap v543: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 671 op/s 2024-02-17T13:07:08.655 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:07:08.656 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T13:07:08.656 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 2227, 2024-02-17T13:07:08.656 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 2229, 2024-02-17T13:07:08.656 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 94040149, 2024-02-17T13:07:08.656 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 692598, 2024-02-17T13:07:08.656 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 8, 2024-02-17T13:07:08.657 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 4, 2024-02-17T13:07:08.657 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.239987043, 2024-02-17T13:07:08.657 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.057169487, 2024-02-17T13:07:08.657 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.103906122, 2024-02-17T13:07:08.657 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.061462917, 2024-02-17T13:07:08.657 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:08.657 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 2227, 2024-02-17T13:07:08.657 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.067036293, 2024-02-17T13:07:08.658 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000030101 2024-02-17T13:07:08.658 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.658 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:08.658 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.658 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.658 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.658 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.658 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T13:07:08.659 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T13:07:08.659 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.659 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T13:07:08.659 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 1355, 2024-02-17T13:07:08.659 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 1591, 2024-02-17T13:07:08.659 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 105213808, 2024-02-17T13:07:08.659 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 1090236, 2024-02-17T13:07:08.660 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 10, 2024-02-17T13:07:08.660 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 7, 2024-02-17T13:07:08.660 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.182902062, 2024-02-17T13:07:08.660 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.039965092, 2024-02-17T13:07:08.660 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.096325711, 2024-02-17T13:07:08.660 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.035014087, 2024-02-17T13:07:08.660 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:08.660 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 1588, 2024-02-17T13:07:08.661 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.059720179, 2024-02-17T13:07:08.661 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000037607 2024-02-17T13:07:08.661 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.661 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:08.661 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.661 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.661 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.662 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.662 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-17T13:07:08.662 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 669696 2024-02-17T13:07:08.662 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.662 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T13:07:08.662 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 1100, 2024-02-17T13:07:08.662 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 1032, 2024-02-17T13:07:08.662 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 81859872, 2024-02-17T13:07:08.662 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 407369, 2024-02-17T13:07:08.663 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 5, 2024-02-17T13:07:08.663 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 4, 2024-02-17T13:07:08.663 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.141076879, 2024-02-17T13:07:08.663 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.034980583, 2024-02-17T13:07:08.663 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.090667642, 2024-02-17T13:07:08.663 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.021738890, 2024-02-17T13:07:08.663 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:08.663 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 1032, 2024-02-17T13:07:08.663 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.031685595, 2024-02-17T13:07:08.664 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000030703 2024-02-17T13:07:08.664 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.664 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:08.664 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.664 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.664 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.664 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.664 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 1982240, 2024-02-17T13:07:08.665 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 193712 2024-02-17T13:07:08.665 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.665 INFO:teuthology.orchestra.run.smithi018.stdout: "cct": { 2024-02-17T13:07:08.665 INFO:teuthology.orchestra.run.smithi018.stdout: "total_workers": 1, 2024-02-17T13:07:08.665 INFO:teuthology.orchestra.run.smithi018.stdout: "unhealthy_workers": 0 2024-02-17T13:07:08.665 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.665 INFO:teuthology.orchestra.run.smithi018.stdout: "finisher-MDSRank": { 2024-02-17T13:07:08.665 INFO:teuthology.orchestra.run.smithi018.stdout: "queue_len": 0, 2024-02-17T13:07:08.666 INFO:teuthology.orchestra.run.smithi018.stdout: "complete_latency": { 2024-02-17T13:07:08.666 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 7124, 2024-02-17T13:07:08.666 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.162481006, 2024-02-17T13:07:08.666 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000022807 2024-02-17T13:07:08.666 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:08.666 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.666 INFO:teuthology.orchestra.run.smithi018.stdout: "finisher-PurgeQueue": { 2024-02-17T13:07:08.666 INFO:teuthology.orchestra.run.smithi018.stdout: "queue_len": 0, 2024-02-17T13:07:08.666 INFO:teuthology.orchestra.run.smithi018.stdout: "complete_latency": { 2024-02-17T13:07:08.667 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.667 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.667 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.667 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:08.667 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-02-17T13:07:08.667 INFO:teuthology.orchestra.run.smithi018.stdout: "request": 0, 2024-02-17T13:07:08.667 INFO:teuthology.orchestra.run.smithi018.stdout: "reply": 0, 2024-02-17T13:07:08.668 INFO:teuthology.orchestra.run.smithi018.stdout: "reply_latency": { 2024-02-17T13:07:08.668 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.668 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.668 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.668 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.668 INFO:teuthology.orchestra.run.smithi018.stdout: "slow_reply": 0, 2024-02-17T13:07:08.668 INFO:teuthology.orchestra.run.smithi018.stdout: "forward": 0, 2024-02-17T13:07:08.669 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_fetch_complete": 0, 2024-02-17T13:07:08.669 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_fetch_keys": 0, 2024-02-17T13:07:08.669 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_commit": 0, 2024-02-17T13:07:08.669 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_split": 0, 2024-02-17T13:07:08.669 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_merge": 0, 2024-02-17T13:07:08.669 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes": 14283, 2024-02-17T13:07:08.669 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_top": 119, 2024-02-17T13:07:08.669 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_bottom": 14164, 2024-02-17T13:07:08.669 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_pin_tail": 0, 2024-02-17T13:07:08.670 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_pinned": 14113, 2024-02-17T13:07:08.670 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_expired": 0, 2024-02-17T13:07:08.670 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_with_caps": 0, 2024-02-17T13:07:08.670 INFO:teuthology.orchestra.run.smithi018.stdout: "caps": 0, 2024-02-17T13:07:08.670 INFO:teuthology.orchestra.run.smithi018.stdout: "subtrees": 16, 2024-02-17T13:07:08.670 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse": 0, 2024-02-17T13:07:08.670 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_hit": 0, 2024-02-17T13:07:08.670 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_forward": 0, 2024-02-17T13:07:08.671 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_discover": 0, 2024-02-17T13:07:08.671 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_dir_fetch": 0, 2024-02-17T13:07:08.671 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_remote_ino": 0, 2024-02-17T13:07:08.671 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_lock": 0, 2024-02-17T13:07:08.671 INFO:teuthology.orchestra.run.smithi018.stdout: "load_cent": 0, 2024-02-17T13:07:08.671 INFO:teuthology.orchestra.run.smithi018.stdout: "q": 0, 2024-02-17T13:07:08.671 INFO:teuthology.orchestra.run.smithi018.stdout: "exported": 0, 2024-02-17T13:07:08.671 INFO:teuthology.orchestra.run.smithi018.stdout: "exported_inodes": 0, 2024-02-17T13:07:08.672 INFO:teuthology.orchestra.run.smithi018.stdout: "imported": 0, 2024-02-17T13:07:08.672 INFO:teuthology.orchestra.run.smithi018.stdout: "imported_inodes": 0, 2024-02-17T13:07:08.672 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_dir_fetch": 0, 2024-02-17T13:07:08.672 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_backtrace_fetch": 0, 2024-02-17T13:07:08.672 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_peer_discover": 0, 2024-02-17T13:07:08.672 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rfiles": 0, 2024-02-17T13:07:08.672 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rbytes": 0, 2024-02-17T13:07:08.672 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rsnaps": 0, 2024-02-17T13:07:08.673 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_backtrace_fetch": 0, 2024-02-17T13:07:08.673 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_set_tag": 0, 2024-02-17T13:07:08.673 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T13:07:08.673 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_inotable_repaired": 0, 2024-02-17T13:07:08.673 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dir_inodes": 0, 2024-02-17T13:07:08.673 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T13:07:08.673 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T13:07:08.673 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_file_inodes": 0, 2024-02-17T13:07:08.674 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_inode_file_caps": 0, 2024-02-17T13:07:08.674 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_revoke": 0, 2024-02-17T13:07:08.674 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_grant": 0, 2024-02-17T13:07:08.674 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T13:07:08.674 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-17T13:07:08.674 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T13:07:08.674 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_caps": 0, 2024-02-17T13:07:08.674 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_caps_dirty": 0, 2024-02-17T13:07:08.674 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_cap_release": 0, 2024-02-17T13:07:08.675 INFO:teuthology.orchestra.run.smithi018.stdout: "process_request_cap_release": 0 2024-02-17T13:07:08.675 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.675 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cache": { 2024-02-17T13:07:08.675 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_update": 0, 2024-02-17T13:07:08.675 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_update_receipt": 0, 2024-02-17T13:07:08.675 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_try_discover": 0, 2024-02-17T13:07:08.675 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_send_discover": 0, 2024-02-17T13:07:08.675 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_handle_discover": 0, 2024-02-17T13:07:08.675 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays": 0, 2024-02-17T13:07:08.676 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays_delayed": 0, 2024-02-17T13:07:08.676 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays_enqueuing": 0, 2024-02-17T13:07:08.676 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_created": 0, 2024-02-17T13:07:08.676 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_enqueued": 0, 2024-02-17T13:07:08.676 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_reintegrated": 0, 2024-02-17T13:07:08.676 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_migrated": 0, 2024-02-17T13:07:08.676 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_processing": 0, 2024-02-17T13:07:08.677 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_enqueued": 0, 2024-02-17T13:07:08.677 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_prioritized": 0, 2024-02-17T13:07:08.677 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_started": 0, 2024-02-17T13:07:08.677 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_completed": 0, 2024-02-17T13:07:08.677 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T13:07:08.677 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_exportdir": 0, 2024-02-17T13:07:08.677 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_flush": 0, 2024-02-17T13:07:08.677 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_fragmentdir": 0, 2024-02-17T13:07:08.677 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_fragstats": 0, 2024-02-17T13:07:08.678 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_inodestats": 0 2024-02-17T13:07:08.678 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_log": { 2024-02-17T13:07:08.678 INFO:teuthology.orchestra.run.smithi018.stdout: "evlrg": 0, 2024-02-17T13:07:08.678 INFO:teuthology.orchestra.run.smithi018.stdout: "evadd": 0, 2024-02-17T13:07:08.678 INFO:teuthology.orchestra.run.smithi018.stdout: "evex": 0, 2024-02-17T13:07:08.678 INFO:teuthology.orchestra.run.smithi018.stdout: "evtrm": 0, 2024-02-17T13:07:08.679 INFO:teuthology.orchestra.run.smithi018.stdout: "ev": 77696, 2024-02-17T13:07:08.679 INFO:teuthology.orchestra.run.smithi018.stdout: "evexg": 0, 2024-02-17T13:07:08.679 INFO:teuthology.orchestra.run.smithi018.stdout: "evexd": 0, 2024-02-17T13:07:08.679 INFO:teuthology.orchestra.run.smithi018.stdout: "segadd": 0, 2024-02-17T13:07:08.679 INFO:teuthology.orchestra.run.smithi018.stdout: "segex": 0, 2024-02-17T13:07:08.679 INFO:teuthology.orchestra.run.smithi018.stdout: "segtrm": 0, 2024-02-17T13:07:08.679 INFO:teuthology.orchestra.run.smithi018.stdout: "seg": 104, 2024-02-17T13:07:08.679 INFO:teuthology.orchestra.run.smithi018.stdout: "segmjr": 8, 2024-02-17T13:07:08.680 INFO:teuthology.orchestra.run.smithi018.stdout: "segexg": 0, 2024-02-17T13:07:08.680 INFO:teuthology.orchestra.run.smithi018.stdout: "segexd": 0, 2024-02-17T13:07:08.680 INFO:teuthology.orchestra.run.smithi018.stdout: "expos": 4194304, 2024-02-17T13:07:08.680 INFO:teuthology.orchestra.run.smithi018.stdout: "wrpos": 282121973, 2024-02-17T13:07:08.680 INFO:teuthology.orchestra.run.smithi018.stdout: "rdpos": 282113519, 2024-02-17T13:07:08.680 INFO:teuthology.orchestra.run.smithi018.stdout: "jlat": { 2024-02-17T13:07:08.680 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.680 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.680 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.681 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.681 INFO:teuthology.orchestra.run.smithi018.stdout: "replayed": 77696 2024-02-17T13:07:08.681 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_mem": { 2024-02-17T13:07:08.681 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1280, 2024-02-17T13:07:08.681 INFO:teuthology.orchestra.run.smithi018.stdout: "ino+": 14376, 2024-02-17T13:07:08.681 INFO:teuthology.orchestra.run.smithi018.stdout: "ino-": 13096, 2024-02-17T13:07:08.681 INFO:teuthology.orchestra.run.smithi018.stdout: "dir": 93, 2024-02-17T13:07:08.682 INFO:teuthology.orchestra.run.smithi018.stdout: "dir+": 126, 2024-02-17T13:07:08.682 INFO:teuthology.orchestra.run.smithi018.stdout: "dir-": 33, 2024-02-17T13:07:08.682 INFO:teuthology.orchestra.run.smithi018.stdout: "dn": 14283, 2024-02-17T13:07:08.682 INFO:teuthology.orchestra.run.smithi018.stdout: "dn+": 14570, 2024-02-17T13:07:08.682 INFO:teuthology.orchestra.run.smithi018.stdout: "dn-": 287, 2024-02-17T13:07:08.682 INFO:teuthology.orchestra.run.smithi018.stdout: "cap": 0, 2024-02-17T13:07:08.682 INFO:teuthology.orchestra.run.smithi018.stdout: "cap+": 0, 2024-02-17T13:07:08.682 INFO:teuthology.orchestra.run.smithi018.stdout: "cap-": 0, 2024-02-17T13:07:08.682 INFO:teuthology.orchestra.run.smithi018.stdout: "rss": 68816, 2024-02-17T13:07:08.683 INFO:teuthology.orchestra.run.smithi018.stdout: "heap": 215420 2024-02-17T13:07:08.683 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_server": { 2024-02-17T13:07:08.683 INFO:teuthology.orchestra.run.smithi018.stdout: "dispatch_client_request": 0, 2024-02-17T13:07:08.683 INFO:teuthology.orchestra.run.smithi018.stdout: "dispatch_server_request": 0, 2024-02-17T13:07:08.683 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_request": 0, 2024-02-17T13:07:08.683 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_session": 0, 2024-02-17T13:07:08.683 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_peer_request": 0, 2024-02-17T13:07:08.684 INFO:teuthology.orchestra.run.smithi018.stdout: "req_create_latency": { 2024-02-17T13:07:08.684 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.684 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.684 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.684 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.684 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getattr_latency": { 2024-02-17T13:07:08.684 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.685 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.685 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.685 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.685 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getfilelock_latency": { 2024-02-17T13:07:08.685 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.685 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.685 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.685 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.685 INFO:teuthology.orchestra.run.smithi018.stdout: "req_link_latency": { 2024-02-17T13:07:08.686 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.686 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.686 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.686 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.686 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookup_latency": { 2024-02-17T13:07:08.686 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.686 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.686 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.686 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.686 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookuphash_latency": { 2024-02-17T13:07:08.687 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.687 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.687 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.687 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.687 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupino_latency": { 2024-02-17T13:07:08.687 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.687 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.687 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.687 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.688 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupname_latency": { 2024-02-17T13:07:08.688 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.688 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.688 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.688 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.688 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupparent_latency": { 2024-02-17T13:07:08.688 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.688 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.689 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.689 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.689 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupsnap_latency": { 2024-02-17T13:07:08.689 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.689 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.689 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.689 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.689 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lssnap_latency": { 2024-02-17T13:07:08.689 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.690 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.690 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.690 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.690 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mkdir_latency": { 2024-02-17T13:07:08.690 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.690 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.690 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.691 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.691 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mknod_latency": { 2024-02-17T13:07:08.691 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.691 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.691 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.691 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.691 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mksnap_latency": { 2024-02-17T13:07:08.691 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.692 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.692 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.692 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.692 INFO:teuthology.orchestra.run.smithi018.stdout: "req_open_latency": { 2024-02-17T13:07:08.692 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.692 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.692 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.692 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.693 INFO:teuthology.orchestra.run.smithi018.stdout: "req_readdir_latency": { 2024-02-17T13:07:08.693 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.693 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.693 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.693 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.693 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rename_latency": { 2024-02-17T13:07:08.693 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.693 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.693 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.694 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.694 INFO:teuthology.orchestra.run.smithi018.stdout: "req_renamesnap_latency": { 2024-02-17T13:07:08.694 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.694 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.694 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.694 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.694 INFO:teuthology.orchestra.run.smithi018.stdout: "req_snapdiff_latency": { 2024-02-17T13:07:08.694 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.694 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.695 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.695 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.695 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmdir_latency": { 2024-02-17T13:07:08.695 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.695 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.695 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.695 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.695 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmsnap_latency": { 2024-02-17T13:07:08.695 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.696 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.696 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.696 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.696 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmxattr_latency": { 2024-02-17T13:07:08.696 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.696 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.696 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.696 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.696 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setattr_latency": { 2024-02-17T13:07:08.697 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.697 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.697 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.697 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.697 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setdirlayout_latency": { 2024-02-17T13:07:08.697 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.697 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.697 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.697 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.698 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setfilelock_latency": { 2024-02-17T13:07:08.698 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.698 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.698 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.698 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.698 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setlayout_latency": { 2024-02-17T13:07:08.698 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.698 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.699 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.699 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.699 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setxattr_latency": { 2024-02-17T13:07:08.699 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.699 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.699 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.699 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.699 INFO:teuthology.orchestra.run.smithi018.stdout: "req_symlink_latency": { 2024-02-17T13:07:08.699 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.700 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.700 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.700 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.700 INFO:teuthology.orchestra.run.smithi018.stdout: "req_unlink_latency": { 2024-02-17T13:07:08.700 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.700 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.700 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.701 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.701 INFO:teuthology.orchestra.run.smithi018.stdout: "cap_revoke_eviction": 0, 2024-02-17T13:07:08.701 INFO:teuthology.orchestra.run.smithi018.stdout: "cap_acquisition_throttle": 0, 2024-02-17T13:07:08.701 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getvxattr_latency": { 2024-02-17T13:07:08.701 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.701 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.701 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.701 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:08.702 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.702 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_sessions": { 2024-02-17T13:07:08.702 INFO:teuthology.orchestra.run.smithi018.stdout: "session_count": 2, 2024-02-17T13:07:08.702 INFO:teuthology.orchestra.run.smithi018.stdout: "session_add": 2, 2024-02-17T13:07:08.702 INFO:teuthology.orchestra.run.smithi018.stdout: "session_remove": 0, 2024-02-17T13:07:08.702 INFO:teuthology.orchestra.run.smithi018.stdout: "sessions_open": 2, 2024-02-17T13:07:08.702 INFO:teuthology.orchestra.run.smithi018.stdout: "sessions_stale": 0, 2024-02-17T13:07:08.702 INFO:teuthology.orchestra.run.smithi018.stdout: "total_load": 0, 2024-02-17T13:07:08.702 INFO:teuthology.orchestra.run.smithi018.stdout: "average_load": 0, 2024-02-17T13:07:08.703 INFO:teuthology.orchestra.run.smithi018.stdout: "avg_session_uptime": 2160, 2024-02-17T13:07:08.703 INFO:teuthology.orchestra.run.smithi018.stdout: "mdthresh_evicted": 0 2024-02-17T13:07:08.703 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.703 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool": { 2024-02-17T13:07:08.703 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_bytes": 0, 2024-02-17T13:07:08.703 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_items": 0, 2024-02-17T13:07:08.703 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T13:07:08.703 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_items": 0, 2024-02-17T13:07:08.703 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T13:07:08.704 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_items": 0, 2024-02-17T13:07:08.704 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T13:07:08.704 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T13:07:08.704 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T13:07:08.704 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T13:07:08.704 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T13:07:08.704 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_items": 0, 2024-02-17T13:07:08.704 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T13:07:08.705 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T13:07:08.705 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_bytes": 0, 2024-02-17T13:07:08.705 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_items": 0, 2024-02-17T13:07:08.705 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_bytes": 0, 2024-02-17T13:07:08.705 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_items": 0, 2024-02-17T13:07:08.705 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T13:07:08.705 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T13:07:08.705 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T13:07:08.705 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T13:07:08.706 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T13:07:08.706 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_items": 0, 2024-02-17T13:07:08.706 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_bytes": 0, 2024-02-17T13:07:08.706 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_items": 0, 2024-02-17T13:07:08.706 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T13:07:08.706 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T13:07:08.706 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_bytes": 0, 2024-02-17T13:07:08.706 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_items": 0, 2024-02-17T13:07:08.707 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_bytes": 0, 2024-02-17T13:07:08.707 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_items": 0, 2024-02-17T13:07:08.707 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T13:07:08.707 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_items": 0, 2024-02-17T13:07:08.707 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T13:07:08.707 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_items": 0, 2024-02-17T13:07:08.707 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_bytes": 291887, 2024-02-17T13:07:08.707 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_items": 1458, 2024-02-17T13:07:08.707 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_bytes": 0, 2024-02-17T13:07:08.708 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_items": 0, 2024-02-17T13:07:08.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bytes": 0, 2024-02-17T13:07:08.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_items": 0, 2024-02-17T13:07:08.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_bytes": 0, 2024-02-17T13:07:08.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_items": 0, 2024-02-17T13:07:08.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_bytes": 0, 2024-02-17T13:07:08.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_items": 0, 2024-02-17T13:07:08.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_bytes": 8056, 2024-02-17T13:07:08.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_items": 168, 2024-02-17T13:07:08.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T13:07:08.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_items": 0, 2024-02-17T13:07:08.709 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_bytes": 0, 2024-02-17T13:07:08.709 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_items": 0, 2024-02-17T13:07:08.709 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_bytes": 13340782, 2024-02-17T13:07:08.709 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_items": 154619, 2024-02-17T13:07:08.709 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_bytes": 0, 2024-02-17T13:07:08.709 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_items": 0, 2024-02-17T13:07:08.709 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_bytes": 0, 2024-02-17T13:07:08.710 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_items": 0 2024-02-17T13:07:08.710 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.710 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter": { 2024-02-17T13:07:08.710 INFO:teuthology.orchestra.run.smithi018.stdout: "op_active": 0, 2024-02-17T13:07:08.710 INFO:teuthology.orchestra.run.smithi018.stdout: "op_laggy": 0, 2024-02-17T13:07:08.710 INFO:teuthology.orchestra.run.smithi018.stdout: "op_send": 4266, 2024-02-17T13:07:08.710 INFO:teuthology.orchestra.run.smithi018.stdout: "op_send_bytes": 21, 2024-02-17T13:07:08.710 INFO:teuthology.orchestra.run.smithi018.stdout: "op_resend": 3, 2024-02-17T13:07:08.710 INFO:teuthology.orchestra.run.smithi018.stdout: "op_reply": 4266, 2024-02-17T13:07:08.711 INFO:teuthology.orchestra.run.smithi018.stdout: "op_latency": { 2024-02-17T13:07:08.711 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 4266, 2024-02-17T13:07:08.711 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 4.323948773, 2024-02-17T13:07:08.711 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.001013583 2024-02-17T13:07:08.711 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.711 INFO:teuthology.orchestra.run.smithi018.stdout: "op_inflight": 0, 2024-02-17T13:07:08.711 INFO:teuthology.orchestra.run.smithi018.stdout: "oplen_avg": { 2024-02-17T13:07:08.711 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 4266, 2024-02-17T13:07:08.712 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 4267 2024-02-17T13:07:08.712 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.712 INFO:teuthology.orchestra.run.smithi018.stdout: "op": 4266, 2024-02-17T13:07:08.712 INFO:teuthology.orchestra.run.smithi018.stdout: "op_r": 4266, 2024-02-17T13:07:08.712 INFO:teuthology.orchestra.run.smithi018.stdout: "op_w": 0, 2024-02-17T13:07:08.712 INFO:teuthology.orchestra.run.smithi018.stdout: "op_rmw": 0, 2024-02-17T13:07:08.712 INFO:teuthology.orchestra.run.smithi018.stdout: "op_pg": 0, 2024-02-17T13:07:08.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_stat": 2404, 2024-02-17T13:07:08.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_create": 0, 2024-02-17T13:07:08.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_read": 1860, 2024-02-17T13:07:08.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_write": 0, 2024-02-17T13:07:08.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_writefull": 0, 2024-02-17T13:07:08.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_writesame": 0, 2024-02-17T13:07:08.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_append": 0, 2024-02-17T13:07:08.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_zero": 0, 2024-02-17T13:07:08.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_truncate": 0, 2024-02-17T13:07:08.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_delete": 0, 2024-02-17T13:07:08.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_mapext": 0, 2024-02-17T13:07:08.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_sparse_read": 0, 2024-02-17T13:07:08.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_clonerange": 0, 2024-02-17T13:07:08.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_getxattr": 1, 2024-02-17T13:07:08.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_setxattr": 0, 2024-02-17T13:07:08.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_cmpxattr": 0, 2024-02-17T13:07:08.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_rmxattr": 0, 2024-02-17T13:07:08.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_resetxattrs": 0, 2024-02-17T13:07:08.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_call": 0, 2024-02-17T13:07:08.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_watch": 0, 2024-02-17T13:07:08.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_notify": 0, 2024-02-17T13:07:08.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T13:07:08.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_pgls": 0, 2024-02-17T13:07:08.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_pgls_filter": 0, 2024-02-17T13:07:08.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_other": 0, 2024-02-17T13:07:08.715 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_active": 0, 2024-02-17T13:07:08.715 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_send": 0, 2024-02-17T13:07:08.716 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_resend": 0, 2024-02-17T13:07:08.716 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_ping": 0, 2024-02-17T13:07:08.716 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_active": 0, 2024-02-17T13:07:08.716 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_send": 0, 2024-02-17T13:07:08.716 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_resend": 0, 2024-02-17T13:07:08.716 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_active": 0, 2024-02-17T13:07:08.716 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_send": 0, 2024-02-17T13:07:08.716 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_resend": 0, 2024-02-17T13:07:08.717 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_active": 0, 2024-02-17T13:07:08.717 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_send": 0, 2024-02-17T13:07:08.717 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_resend": 0, 2024-02-17T13:07:08.717 INFO:teuthology.orchestra.run.smithi018.stdout: "command_active": 0, 2024-02-17T13:07:08.717 INFO:teuthology.orchestra.run.smithi018.stdout: "command_send": 0, 2024-02-17T13:07:08.717 INFO:teuthology.orchestra.run.smithi018.stdout: "command_resend": 0, 2024-02-17T13:07:08.717 INFO:teuthology.orchestra.run.smithi018.stdout: "map_epoch": 112, 2024-02-17T13:07:08.717 INFO:teuthology.orchestra.run.smithi018.stdout: "map_full": 0, 2024-02-17T13:07:08.717 INFO:teuthology.orchestra.run.smithi018.stdout: "map_inc": 26, 2024-02-17T13:07:08.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_sessions": 11, 2024-02-17T13:07:08.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_session_open": 11, 2024-02-17T13:07:08.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_session_close": 0, 2024-02-17T13:07:08.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_laggy": 0, 2024-02-17T13:07:08.718 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_wr": 0, 2024-02-17T13:07:08.718 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_rd": 2, 2024-02-17T13:07:08.718 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_del": 0 2024-02-17T13:07:08.718 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.718 INFO:teuthology.orchestra.run.smithi018.stdout: "oft": { 2024-02-17T13:07:08.719 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_objs": 0, 2024-02-17T13:07:08.719 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_kv_pairs": 0, 2024-02-17T13:07:08.719 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_updates": 0, 2024-02-17T13:07:08.719 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_removes": 0 2024-02-17T13:07:08.719 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.719 INFO:teuthology.orchestra.run.smithi018.stdout: "osd-slow-ops": { 2024-02-17T13:07:08.719 INFO:teuthology.orchestra.run.smithi018.stdout: "slow_ops_count": 0 2024-02-17T13:07:08.719 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.720 INFO:teuthology.orchestra.run.smithi018.stdout: "purge_queue": { 2024-02-17T13:07:08.720 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_ops": 0, 2024-02-17T13:07:08.720 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_ops_high_water": 0, 2024-02-17T13:07:08.720 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing": 0, 2024-02-17T13:07:08.720 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_high_water": 0, 2024-02-17T13:07:08.720 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executed_ops": 0, 2024-02-17T13:07:08.720 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executed": 0, 2024-02-17T13:07:08.720 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_item_in_journal": 0 2024-02-17T13:07:08.721 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.721 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T13:07:08.721 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-02-17T13:07:08.721 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-02-17T13:07:08.721 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-02-17T13:07:08.721 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 4678, 2024-02-17T13:07:08.721 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 280669209, 2024-02-17T13:07:08.721 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:08.722 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 4678, 2024-02-17T13:07:08.722 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-02-17T13:07:08.722 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-02-17T13:07:08.722 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 4678, 2024-02-17T13:07:08.722 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 280669209, 2024-02-17T13:07:08.722 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-02-17T13:07:08.722 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.722 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.723 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.723 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:08.723 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.723 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-objecter_bytes": { 2024-02-17T13:07:08.723 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-02-17T13:07:08.723 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-02-17T13:07:08.723 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-02-17T13:07:08.723 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-02-17T13:07:08.724 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-02-17T13:07:08.724 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:08.724 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:08.724 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 4266, 2024-02-17T13:07:08.724 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 277951423, 2024-02-17T13:07:08.724 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 4266, 2024-02-17T13:07:08.724 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 277951423, 2024-02-17T13:07:08.724 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-02-17T13:07:08.725 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.725 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.725 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.725 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:08.725 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.725 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-objecter_ops": { 2024-02-17T13:07:08.725 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-02-17T13:07:08.725 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 1024, 2024-02-17T13:07:08.726 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-02-17T13:07:08.726 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-02-17T13:07:08.726 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-02-17T13:07:08.726 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:08.726 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:08.726 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 4266, 2024-02-17T13:07:08.726 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 4266, 2024-02-17T13:07:08.726 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 4266, 2024-02-17T13:07:08.726 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 4266, 2024-02-17T13:07:08.726 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-02-17T13:07:08.727 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.727 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.727 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.727 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:08.727 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.727 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-write_buf_throttle": { 2024-02-17T13:07:08.727 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-02-17T13:07:08.727 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 3758096384, 2024-02-17T13:07:08.728 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-02-17T13:07:08.728 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-02-17T13:07:08.728 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-02-17T13:07:08.728 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:08.728 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:08.728 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-02-17T13:07:08.728 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-02-17T13:07:08.728 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 0, 2024-02-17T13:07:08.728 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 0, 2024-02-17T13:07:08.728 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-02-17T13:07:08.729 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.729 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.729 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.729 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:08.729 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:08.729 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-write_buf_throttle-0x557aa8798960": { 2024-02-17T13:07:08.729 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-02-17T13:07:08.729 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 3758096384, 2024-02-17T13:07:08.730 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-02-17T13:07:08.730 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-02-17T13:07:08.730 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-02-17T13:07:08.730 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:08.730 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:08.730 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-02-17T13:07:08.730 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-02-17T13:07:08.730 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 0, 2024-02-17T13:07:08.731 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 0, 2024-02-17T13:07:08.731 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-02-17T13:07:08.731 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:08.731 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:08.731 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:08.731 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:08.731 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:08.731 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:07:09.403 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-02-17T13:07:09.403 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-02-17T13:07:09.403 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-02-17T13:07:09.403 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-02-17T13:07:09.404 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T13:07:09.404 DEBUG:tasks.check_counter:Getting stats from d 2024-02-17T13:07:09.404 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph daemon mds.d perf dump 2024-02-17T13:07:09.673 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T13:07:10.248 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:10 smithi018 ceph-mon[32625]: pgmap v544: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.6 MiB/s wr, 605 op/s 2024-02-17T13:07:10.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:10 smithi146 ceph-mon[36454]: pgmap v544: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.6 MiB/s wr, 605 op/s 2024-02-17T13:07:10.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:10 smithi119 ceph-mon[36188]: pgmap v544: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.6 MiB/s wr, 605 op/s 2024-02-17T13:07:11.706 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:07:11.707 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T13:07:11.707 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 0, 2024-02-17T13:07:11.707 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 2, 2024-02-17T13:07:11.707 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 0, 2024-02-17T13:07:11.707 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 0, 2024-02-17T13:07:11.707 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 4, 2024-02-17T13:07:11.707 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 0, 2024-02-17T13:07:11.708 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.001218852, 2024-02-17T13:07:11.708 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.000000000, 2024-02-17T13:07:11.708 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.000346809, 2024-02-17T13:07:11.708 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T13:07:11.708 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:11.708 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:11.708 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:11.708 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:11.709 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:11.709 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:11.709 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:11.709 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:11.709 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:11.709 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:11.709 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T13:07:11.709 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T13:07:11.710 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:11.710 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T13:07:11.710 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 7, 2024-02-17T13:07:11.710 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 246, 2024-02-17T13:07:11.710 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 3472, 2024-02-17T13:07:11.710 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 275712, 2024-02-17T13:07:11.710 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 6, 2024-02-17T13:07:11.710 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 3, 2024-02-17T13:07:11.711 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.018191080, 2024-02-17T13:07:11.711 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.012816144, 2024-02-17T13:07:11.711 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.002688276, 2024-02-17T13:07:11.711 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T13:07:11.711 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:11.711 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 244, 2024-02-17T13:07:11.711 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.020023633, 2024-02-17T13:07:11.711 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000082064 2024-02-17T13:07:11.712 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:11.712 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:11.712 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:11.712 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:11.712 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:11.712 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:11.712 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-02-17T13:07:11.712 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 275712 2024-02-17T13:07:11.713 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:11.713 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T13:07:11.713 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 329, 2024-02-17T13:07:11.713 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 318, 2024-02-17T13:07:11.713 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 1599584, 2024-02-17T13:07:11.713 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 191600, 2024-02-17T13:07:11.713 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 2, 2024-02-17T13:07:11.713 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 1, 2024-02-17T13:07:11.713 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.053130740, 2024-02-17T13:07:11.713 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.017762799, 2024-02-17T13:07:11.714 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.036386149, 2024-02-17T13:07:11.714 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.006108845, 2024-02-17T13:07:11.714 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:11.714 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 317, 2024-02-17T13:07:11.714 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.012104678, 2024-02-17T13:07:11.714 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000038185 2024-02-17T13:07:11.714 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:11.714 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:11.714 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:11.715 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:11.715 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:11.715 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:11.715 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 1599584, 2024-02-17T13:07:11.715 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 191600 2024-02-17T13:07:11.715 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:11.715 INFO:teuthology.orchestra.run.smithi018.stdout: "cct": { 2024-02-17T13:07:11.715 INFO:teuthology.orchestra.run.smithi018.stdout: "total_workers": 0, 2024-02-17T13:07:11.715 INFO:teuthology.orchestra.run.smithi018.stdout: "unhealthy_workers": 0 2024-02-17T13:07:11.716 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:11.716 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool": { 2024-02-17T13:07:11.716 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_bytes": 0, 2024-02-17T13:07:11.716 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_items": 0, 2024-02-17T13:07:11.716 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T13:07:11.716 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_items": 0, 2024-02-17T13:07:11.716 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T13:07:11.716 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_items": 0, 2024-02-17T13:07:11.716 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T13:07:11.717 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T13:07:11.717 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T13:07:11.717 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T13:07:11.717 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T13:07:11.717 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_items": 0, 2024-02-17T13:07:11.717 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T13:07:11.717 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T13:07:11.717 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_bytes": 0, 2024-02-17T13:07:11.717 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_items": 0, 2024-02-17T13:07:11.718 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_bytes": 0, 2024-02-17T13:07:11.718 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_items": 0, 2024-02-17T13:07:11.718 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T13:07:11.718 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T13:07:11.718 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T13:07:11.718 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T13:07:11.718 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T13:07:11.718 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_items": 0, 2024-02-17T13:07:11.718 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_bytes": 0, 2024-02-17T13:07:11.719 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_items": 0, 2024-02-17T13:07:11.719 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T13:07:11.719 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T13:07:11.719 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_bytes": 0, 2024-02-17T13:07:11.719 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_items": 0, 2024-02-17T13:07:11.719 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_bytes": 0, 2024-02-17T13:07:11.719 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_items": 0, 2024-02-17T13:07:11.719 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T13:07:11.719 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_items": 0, 2024-02-17T13:07:11.720 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T13:07:11.720 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_items": 0, 2024-02-17T13:07:11.720 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_bytes": 148794, 2024-02-17T13:07:11.720 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_items": 32, 2024-02-17T13:07:11.720 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_bytes": 0, 2024-02-17T13:07:11.720 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_items": 0, 2024-02-17T13:07:11.720 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bytes": 0, 2024-02-17T13:07:11.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_items": 0, 2024-02-17T13:07:11.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_bytes": 0, 2024-02-17T13:07:11.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_items": 0, 2024-02-17T13:07:11.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_bytes": 0, 2024-02-17T13:07:11.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_items": 0, 2024-02-17T13:07:11.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_bytes": 0, 2024-02-17T13:07:11.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_items": 0, 2024-02-17T13:07:11.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T13:07:11.722 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_items": 0, 2024-02-17T13:07:11.722 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_bytes": 0, 2024-02-17T13:07:11.722 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_items": 0, 2024-02-17T13:07:11.722 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_bytes": 928, 2024-02-17T13:07:11.722 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_items": 2, 2024-02-17T13:07:11.722 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_bytes": 0, 2024-02-17T13:07:11.722 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_items": 0, 2024-02-17T13:07:11.723 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_bytes": 0, 2024-02-17T13:07:11.723 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_items": 0 2024-02-17T13:07:11.723 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:11.723 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T13:07:11.723 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-02-17T13:07:11.723 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-02-17T13:07:11.723 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-02-17T13:07:11.723 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 332, 2024-02-17T13:07:11.724 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 1553129, 2024-02-17T13:07:11.724 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:11.724 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 332, 2024-02-17T13:07:11.724 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-02-17T13:07:11.724 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-02-17T13:07:11.724 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 332, 2024-02-17T13:07:11.724 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 1553129, 2024-02-17T13:07:11.725 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-02-17T13:07:11.725 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:11.725 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:11.725 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:11.725 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:11.725 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:11.725 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:07:12.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:12 smithi146 ceph-mon[36454]: pgmap v545: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 309 op/s 2024-02-17T13:07:12.480 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:12 smithi018 ceph-mon[32625]: pgmap v545: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 309 op/s 2024-02-17T13:07:12.509 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-02-17T13:07:12.509 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-02-17T13:07:12.509 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-02-17T13:07:12.509 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-02-17T13:07:12.509 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-02-17T13:07:12.509 DEBUG:tasks.check_counter:Getting stats from g 2024-02-17T13:07:12.510 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph daemon mds.g perf dump 2024-02-17T13:07:12.589 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:12 smithi119 ceph-mon[36188]: pgmap v545: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 309 op/s 2024-02-17T13:07:12.756 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T13:07:14.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:14 smithi018 ceph-mon[32625]: pgmap v546: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 309 op/s 2024-02-17T13:07:14.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:14 smithi018 ceph-mon[32625]: osdmap e113: 12 total, 12 up, 12 in 2024-02-17T13:07:14.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:14 smithi146 ceph-mon[36454]: pgmap v546: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 309 op/s 2024-02-17T13:07:14.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:14 smithi146 ceph-mon[36454]: osdmap e113: 12 total, 12 up, 12 in 2024-02-17T13:07:14.481 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:07:14.481 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T13:07:14.481 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 2058, 2024-02-17T13:07:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 1992, 2024-02-17T13:07:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 65407566, 2024-02-17T13:07:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 706594, 2024-02-17T13:07:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 7, 2024-02-17T13:07:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 3, 2024-02-17T13:07:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.230142745, 2024-02-17T13:07:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.075528211, 2024-02-17T13:07:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.104605826, 2024-02-17T13:07:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.047666044, 2024-02-17T13:07:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 1991, 2024-02-17T13:07:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.072988785, 2024-02-17T13:07:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000036659 2024-02-17T13:07:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 1980256, 2024-02-17T13:07:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 194688 2024-02-17T13:07:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T13:07:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 1990, 2024-02-17T13:07:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 2227, 2024-02-17T13:07:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 69062962, 2024-02-17T13:07:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 1204057, 2024-02-17T13:07:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 9, 2024-02-17T13:07:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 7, 2024-02-17T13:07:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.182528255, 2024-02-17T13:07:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.044575470, 2024-02-17T13:07:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.081666717, 2024-02-17T13:07:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.044198596, 2024-02-17T13:07:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 2225, 2024-02-17T13:07:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.060699871, 2024-02-17T13:07:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000027280 2024-02-17T13:07:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-02-17T13:07:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 586976 2024-02-17T13:07:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T13:07:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 535, 2024-02-17T13:07:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 537, 2024-02-17T13:07:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 79828751, 2024-02-17T13:07:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 166397, 2024-02-17T13:07:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 7, 2024-02-17T13:07:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 5, 2024-02-17T13:07:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.082468612, 2024-02-17T13:07:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.009672858, 2024-02-17T13:07:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.057292958, 2024-02-17T13:07:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.012199193, 2024-02-17T13:07:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 535, 2024-02-17T13:07:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.013478714, 2024-02-17T13:07:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000025193 2024-02-17T13:07:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T13:07:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T13:07:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "cct": { 2024-02-17T13:07:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "total_workers": 1, 2024-02-17T13:07:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "unhealthy_workers": 0 2024-02-17T13:07:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "finisher-MDSRank": { 2024-02-17T13:07:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "queue_len": 0, 2024-02-17T13:07:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "complete_latency": { 2024-02-17T13:07:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 7056, 2024-02-17T13:07:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.142862593, 2024-02-17T13:07:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000020246 2024-02-17T13:07:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "finisher-PurgeQueue": { 2024-02-17T13:07:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "queue_len": 0, 2024-02-17T13:07:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "complete_latency": { 2024-02-17T13:07:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-02-17T13:07:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "request": 0, 2024-02-17T13:07:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "reply": 0, 2024-02-17T13:07:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: "reply_latency": { 2024-02-17T13:07:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: "slow_reply": 0, 2024-02-17T13:07:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: "forward": 0, 2024-02-17T13:07:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_fetch_complete": 0, 2024-02-17T13:07:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_fetch_keys": 0, 2024-02-17T13:07:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_commit": 0, 2024-02-17T13:07:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_split": 0, 2024-02-17T13:07:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_merge": 0, 2024-02-17T13:07:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes": 11670, 2024-02-17T13:07:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_top": 62, 2024-02-17T13:07:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_bottom": 11608, 2024-02-17T13:07:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_pin_tail": 0, 2024-02-17T13:07:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_pinned": 11581, 2024-02-17T13:07:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_expired": 0, 2024-02-17T13:07:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_with_caps": 0, 2024-02-17T13:07:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "caps": 0, 2024-02-17T13:07:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "subtrees": 6, 2024-02-17T13:07:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse": 0, 2024-02-17T13:07:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_hit": 0, 2024-02-17T13:07:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_forward": 0, 2024-02-17T13:07:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_discover": 0, 2024-02-17T13:07:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_dir_fetch": 0, 2024-02-17T13:07:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_remote_ino": 0, 2024-02-17T13:07:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_lock": 0, 2024-02-17T13:07:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "load_cent": 0, 2024-02-17T13:07:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "q": 0, 2024-02-17T13:07:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "exported": 0, 2024-02-17T13:07:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "exported_inodes": 0, 2024-02-17T13:07:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "imported": 0, 2024-02-17T13:07:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "imported_inodes": 0, 2024-02-17T13:07:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_dir_fetch": 0, 2024-02-17T13:07:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_backtrace_fetch": 0, 2024-02-17T13:07:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_peer_discover": 0, 2024-02-17T13:07:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rfiles": 0, 2024-02-17T13:07:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rbytes": 0, 2024-02-17T13:07:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rsnaps": 0, 2024-02-17T13:07:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_backtrace_fetch": 0, 2024-02-17T13:07:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_set_tag": 0, 2024-02-17T13:07:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T13:07:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_inotable_repaired": 0, 2024-02-17T13:07:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dir_inodes": 0, 2024-02-17T13:07:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T13:07:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T13:07:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_file_inodes": 0, 2024-02-17T13:07:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_inode_file_caps": 0, 2024-02-17T13:07:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_revoke": 0, 2024-02-17T13:07:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_grant": 0, 2024-02-17T13:07:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T13:07:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-17T13:07:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T13:07:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_caps": 0, 2024-02-17T13:07:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_caps_dirty": 0, 2024-02-17T13:07:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_cap_release": 0, 2024-02-17T13:07:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "process_request_cap_release": 0 2024-02-17T13:07:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cache": { 2024-02-17T13:07:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_update": 0, 2024-02-17T13:07:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_update_receipt": 0, 2024-02-17T13:07:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_try_discover": 0, 2024-02-17T13:07:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_send_discover": 0, 2024-02-17T13:07:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_handle_discover": 0, 2024-02-17T13:07:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays": 0, 2024-02-17T13:07:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays_delayed": 0, 2024-02-17T13:07:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays_enqueuing": 0, 2024-02-17T13:07:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_created": 0, 2024-02-17T13:07:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_enqueued": 0, 2024-02-17T13:07:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_reintegrated": 0, 2024-02-17T13:07:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_migrated": 0, 2024-02-17T13:07:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_processing": 0, 2024-02-17T13:07:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_enqueued": 0, 2024-02-17T13:07:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_prioritized": 0, 2024-02-17T13:07:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_started": 0, 2024-02-17T13:07:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_completed": 0, 2024-02-17T13:07:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T13:07:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_exportdir": 0, 2024-02-17T13:07:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_flush": 0, 2024-02-17T13:07:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_fragmentdir": 0, 2024-02-17T13:07:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_fragstats": 0, 2024-02-17T13:07:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_inodestats": 0 2024-02-17T13:07:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_log": { 2024-02-17T13:07:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "evlrg": 0, 2024-02-17T13:07:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "evadd": 0, 2024-02-17T13:07:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "evex": 0, 2024-02-17T13:07:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "evtrm": 0, 2024-02-17T13:07:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "ev": 58588, 2024-02-17T13:07:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "evexg": 0, 2024-02-17T13:07:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "evexd": 0, 2024-02-17T13:07:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "segadd": 0, 2024-02-17T13:07:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "segex": 0, 2024-02-17T13:07:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "segtrm": 0, 2024-02-17T13:07:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "seg": 80, 2024-02-17T13:07:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "segmjr": 6, 2024-02-17T13:07:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "segexg": 0, 2024-02-17T13:07:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "segexd": 0, 2024-02-17T13:07:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "expos": 4194304, 2024-02-17T13:07:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "wrpos": 215356383, 2024-02-17T13:07:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "rdpos": 215350363, 2024-02-17T13:07:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "jlat": { 2024-02-17T13:07:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "replayed": 58588 2024-02-17T13:07:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_mem": { 2024-02-17T13:07:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 1161, 2024-02-17T13:07:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "ino+": 11789, 2024-02-17T13:07:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "ino-": 10628, 2024-02-17T13:07:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "dir": 76, 2024-02-17T13:07:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "dir+": 132, 2024-02-17T13:07:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "dir-": 56, 2024-02-17T13:07:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "dn": 11671, 2024-02-17T13:07:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "dn+": 11940, 2024-02-17T13:07:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "dn-": 269, 2024-02-17T13:07:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "cap": 0, 2024-02-17T13:07:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "cap+": 0, 2024-02-17T13:07:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "cap-": 0, 2024-02-17T13:07:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "rss": 66900, 2024-02-17T13:07:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "heap": 215420 2024-02-17T13:07:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_server": { 2024-02-17T13:07:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "dispatch_client_request": 0, 2024-02-17T13:07:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "dispatch_server_request": 0, 2024-02-17T13:07:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_request": 0, 2024-02-17T13:07:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_session": 0, 2024-02-17T13:07:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_peer_request": 0, 2024-02-17T13:07:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "req_create_latency": { 2024-02-17T13:07:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getattr_latency": { 2024-02-17T13:07:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getfilelock_latency": { 2024-02-17T13:07:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "req_link_latency": { 2024-02-17T13:07:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookup_latency": { 2024-02-17T13:07:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookuphash_latency": { 2024-02-17T13:07:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupino_latency": { 2024-02-17T13:07:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupname_latency": { 2024-02-17T13:07:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupparent_latency": { 2024-02-17T13:07:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupsnap_latency": { 2024-02-17T13:07:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lssnap_latency": { 2024-02-17T13:07:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mkdir_latency": { 2024-02-17T13:07:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mknod_latency": { 2024-02-17T13:07:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mksnap_latency": { 2024-02-17T13:07:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "req_open_latency": { 2024-02-17T13:07:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "req_readdir_latency": { 2024-02-17T13:07:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rename_latency": { 2024-02-17T13:07:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "req_renamesnap_latency": { 2024-02-17T13:07:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "req_snapdiff_latency": { 2024-02-17T13:07:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmdir_latency": { 2024-02-17T13:07:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmsnap_latency": { 2024-02-17T13:07:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmxattr_latency": { 2024-02-17T13:07:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setattr_latency": { 2024-02-17T13:07:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setdirlayout_latency": { 2024-02-17T13:07:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setfilelock_latency": { 2024-02-17T13:07:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setlayout_latency": { 2024-02-17T13:07:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setxattr_latency": { 2024-02-17T13:07:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "req_symlink_latency": { 2024-02-17T13:07:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "req_unlink_latency": { 2024-02-17T13:07:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "cap_revoke_eviction": 0, 2024-02-17T13:07:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "cap_acquisition_throttle": 0, 2024-02-17T13:07:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getvxattr_latency": { 2024-02-17T13:07:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_sessions": { 2024-02-17T13:07:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "session_count": 1, 2024-02-17T13:07:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "session_add": 3, 2024-02-17T13:07:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "session_remove": 2, 2024-02-17T13:07:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "sessions_open": 3, 2024-02-17T13:07:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "sessions_stale": 0, 2024-02-17T13:07:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "total_load": 0, 2024-02-17T13:07:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "average_load": 0, 2024-02-17T13:07:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "avg_session_uptime": 4415, 2024-02-17T13:07:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "mdthresh_evicted": 0 2024-02-17T13:07:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool": { 2024-02-17T13:07:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_bytes": 0, 2024-02-17T13:07:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_items": 0, 2024-02-17T13:07:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T13:07:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_items": 0, 2024-02-17T13:07:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T13:07:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_items": 0, 2024-02-17T13:07:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T13:07:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T13:07:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T13:07:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T13:07:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T13:07:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_items": 0, 2024-02-17T13:07:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T13:07:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T13:07:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_bytes": 0, 2024-02-17T13:07:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_items": 0, 2024-02-17T13:07:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_bytes": 0, 2024-02-17T13:07:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_items": 0, 2024-02-17T13:07:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T13:07:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T13:07:14.533 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T13:07:14.533 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T13:07:14.533 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T13:07:14.533 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_items": 0, 2024-02-17T13:07:14.533 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_bytes": 0, 2024-02-17T13:07:14.533 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_items": 0, 2024-02-17T13:07:14.533 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T13:07:14.533 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T13:07:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_bytes": 0, 2024-02-17T13:07:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_items": 0, 2024-02-17T13:07:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_bytes": 0, 2024-02-17T13:07:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_items": 0, 2024-02-17T13:07:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T13:07:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_items": 0, 2024-02-17T13:07:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T13:07:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_items": 0, 2024-02-17T13:07:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_bytes": 285790, 2024-02-17T13:07:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_items": 1324, 2024-02-17T13:07:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_bytes": 0, 2024-02-17T13:07:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_items": 0, 2024-02-17T13:07:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bytes": 0, 2024-02-17T13:07:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_items": 0, 2024-02-17T13:07:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_bytes": 0, 2024-02-17T13:07:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_items": 0, 2024-02-17T13:07:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_bytes": 0, 2024-02-17T13:07:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_items": 0, 2024-02-17T13:07:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_bytes": 8056, 2024-02-17T13:07:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_items": 168, 2024-02-17T13:07:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T13:07:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_items": 0, 2024-02-17T13:07:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_bytes": 0, 2024-02-17T13:07:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_items": 0, 2024-02-17T13:07:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_bytes": 11176484, 2024-02-17T13:07:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_items": 133174, 2024-02-17T13:07:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_bytes": 0, 2024-02-17T13:07:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_items": 0, 2024-02-17T13:07:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_bytes": 0, 2024-02-17T13:07:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_items": 0 2024-02-17T13:07:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter": { 2024-02-17T13:07:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "op_active": 0, 2024-02-17T13:07:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "op_laggy": 0, 2024-02-17T13:07:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "op_send": 4165, 2024-02-17T13:07:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "op_send_bytes": 21, 2024-02-17T13:07:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "op_resend": 3, 2024-02-17T13:07:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "op_reply": 4165, 2024-02-17T13:07:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "op_latency": { 2024-02-17T13:07:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 4165, 2024-02-17T13:07:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 3.754955631, 2024-02-17T13:07:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000901549 2024-02-17T13:07:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "op_inflight": 0, 2024-02-17T13:07:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "oplen_avg": { 2024-02-17T13:07:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 4165, 2024-02-17T13:07:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 4166 2024-02-17T13:07:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "op": 4165, 2024-02-17T13:07:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "op_r": 4165, 2024-02-17T13:07:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "op_w": 0, 2024-02-17T13:07:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "op_rmw": 0, 2024-02-17T13:07:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "op_pg": 0, 2024-02-17T13:07:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_stat": 2340, 2024-02-17T13:07:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_create": 0, 2024-02-17T13:07:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_read": 1823, 2024-02-17T13:07:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_write": 0, 2024-02-17T13:07:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_writefull": 0, 2024-02-17T13:07:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_writesame": 0, 2024-02-17T13:07:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_append": 0, 2024-02-17T13:07:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_zero": 0, 2024-02-17T13:07:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_truncate": 0, 2024-02-17T13:07:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_delete": 0, 2024-02-17T13:07:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_mapext": 0, 2024-02-17T13:07:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_sparse_read": 0, 2024-02-17T13:07:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_clonerange": 0, 2024-02-17T13:07:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_getxattr": 1, 2024-02-17T13:07:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_setxattr": 0, 2024-02-17T13:07:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_cmpxattr": 0, 2024-02-17T13:07:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_rmxattr": 0, 2024-02-17T13:07:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_resetxattrs": 0, 2024-02-17T13:07:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_call": 0, 2024-02-17T13:07:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_watch": 0, 2024-02-17T13:07:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_notify": 0, 2024-02-17T13:07:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T13:07:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_pgls": 0, 2024-02-17T13:07:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_pgls_filter": 0, 2024-02-17T13:07:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_other": 0, 2024-02-17T13:07:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_active": 0, 2024-02-17T13:07:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_send": 0, 2024-02-17T13:07:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_resend": 0, 2024-02-17T13:07:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_ping": 0, 2024-02-17T13:07:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_active": 0, 2024-02-17T13:07:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_send": 0, 2024-02-17T13:07:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_resend": 0, 2024-02-17T13:07:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_active": 0, 2024-02-17T13:07:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_send": 0, 2024-02-17T13:07:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_resend": 0, 2024-02-17T13:07:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_active": 0, 2024-02-17T13:07:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_send": 0, 2024-02-17T13:07:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_resend": 0, 2024-02-17T13:07:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "command_active": 0, 2024-02-17T13:07:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "command_send": 0, 2024-02-17T13:07:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "command_resend": 0, 2024-02-17T13:07:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "map_epoch": 113, 2024-02-17T13:07:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "map_full": 0, 2024-02-17T13:07:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "map_inc": 27, 2024-02-17T13:07:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_sessions": 11, 2024-02-17T13:07:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_session_open": 12, 2024-02-17T13:07:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_session_close": 1, 2024-02-17T13:07:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_laggy": 0, 2024-02-17T13:07:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_wr": 0, 2024-02-17T13:07:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_rd": 2, 2024-02-17T13:07:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_del": 0 2024-02-17T13:07:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "oft": { 2024-02-17T13:07:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_objs": 0, 2024-02-17T13:07:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_kv_pairs": 0, 2024-02-17T13:07:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_updates": 0, 2024-02-17T13:07:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_removes": 0 2024-02-17T13:07:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "osd-slow-ops": { 2024-02-17T13:07:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "slow_ops_count": 0 2024-02-17T13:07:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "purge_queue": { 2024-02-17T13:07:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_ops": 0, 2024-02-17T13:07:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_ops_high_water": 0, 2024-02-17T13:07:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing": 0, 2024-02-17T13:07:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_high_water": 0, 2024-02-17T13:07:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executed_ops": 0, 2024-02-17T13:07:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executed": 0, 2024-02-17T13:07:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_item_in_journal": 0 2024-02-17T13:07:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T13:07:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-02-17T13:07:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-02-17T13:07:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-02-17T13:07:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 4579, 2024-02-17T13:07:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 213863466, 2024-02-17T13:07:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 4579, 2024-02-17T13:07:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-02-17T13:07:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-02-17T13:07:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 4579, 2024-02-17T13:07:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 213863466, 2024-02-17T13:07:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-02-17T13:07:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-objecter_bytes": { 2024-02-17T13:07:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-02-17T13:07:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-02-17T13:07:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-02-17T13:07:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-02-17T13:07:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-02-17T13:07:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 4165, 2024-02-17T13:07:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 211162407, 2024-02-17T13:07:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 4165, 2024-02-17T13:07:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 211162407, 2024-02-17T13:07:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-02-17T13:07:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-objecter_ops": { 2024-02-17T13:07:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-02-17T13:07:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 1024, 2024-02-17T13:07:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-02-17T13:07:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-02-17T13:07:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-02-17T13:07:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 4165, 2024-02-17T13:07:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 4165, 2024-02-17T13:07:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 4165, 2024-02-17T13:07:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 4165, 2024-02-17T13:07:14.554 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-02-17T13:07:14.554 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.554 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.554 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.554 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:14.554 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.554 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-write_buf_throttle": { 2024-02-17T13:07:14.554 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-02-17T13:07:14.554 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 3758096384, 2024-02-17T13:07:14.555 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-02-17T13:07:14.555 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-02-17T13:07:14.555 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-02-17T13:07:14.555 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:14.555 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:14.555 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-02-17T13:07:14.555 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-02-17T13:07:14.555 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 0, 2024-02-17T13:07:14.555 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 0, 2024-02-17T13:07:14.556 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-02-17T13:07:14.556 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.556 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.556 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.556 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:14.556 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:14.556 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-write_buf_throttle-0x5630b4e90960": { 2024-02-17T13:07:14.556 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-02-17T13:07:14.556 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 3758096384, 2024-02-17T13:07:14.557 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-02-17T13:07:14.557 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-02-17T13:07:14.557 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-02-17T13:07:14.557 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:14.557 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:14.557 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-02-17T13:07:14.557 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-02-17T13:07:14.557 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 0, 2024-02-17T13:07:14.558 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 0, 2024-02-17T13:07:14.558 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-02-17T13:07:14.558 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:14.558 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:14.558 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:14.558 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:14.558 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:14.558 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:07:14.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:14 smithi119 ceph-mon[36188]: pgmap v546: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 309 op/s 2024-02-17T13:07:14.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:14 smithi119 ceph-mon[36188]: osdmap e113: 12 total, 12 up, 12 in 2024-02-17T13:07:15.207 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-02-17T13:07:15.207 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-02-17T13:07:15.207 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-02-17T13:07:15.207 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-02-17T13:07:15.207 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T13:07:15.207 DEBUG:tasks.check_counter:Getting stats from j 2024-02-17T13:07:15.208 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph daemon mds.j perf dump 2024-02-17T13:07:15.450 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T13:07:16.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:16 smithi146 ceph-mon[36454]: pgmap v548: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 4.9 MiB/s wr, 198 op/s 2024-02-17T13:07:16.487 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:16 smithi018 ceph-mon[32625]: pgmap v548: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 4.9 MiB/s wr, 198 op/s 2024-02-17T13:07:16.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:16 smithi119 ceph-mon[36188]: pgmap v548: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 4.9 MiB/s wr, 198 op/s 2024-02-17T13:07:17.025 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-02-17T13:07:17.026 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T13:07:17.026 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 0, 2024-02-17T13:07:17.026 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 2, 2024-02-17T13:07:17.026 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 0, 2024-02-17T13:07:17.026 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 0, 2024-02-17T13:07:17.026 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 4, 2024-02-17T13:07:17.026 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 0, 2024-02-17T13:07:17.026 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.001091968, 2024-02-17T13:07:17.027 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.000000000, 2024-02-17T13:07:17.027 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.000262857, 2024-02-17T13:07:17.027 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T13:07:17.027 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:17.027 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:17.027 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:17.027 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:17.027 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:17.028 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:17.028 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:17.028 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:17.028 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:17.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:17.028 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T13:07:17.028 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T13:07:17.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:17.029 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T13:07:17.029 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 335, 2024-02-17T13:07:17.029 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 322, 2024-02-17T13:07:17.029 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 1603120, 2024-02-17T13:07:17.029 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 193296, 2024-02-17T13:07:17.029 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 2, 2024-02-17T13:07:17.029 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 1, 2024-02-17T13:07:17.029 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.056066964, 2024-02-17T13:07:17.030 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.018017161, 2024-02-17T13:07:17.030 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.040735582, 2024-02-17T13:07:17.030 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.006236233, 2024-02-17T13:07:17.030 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:17.030 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 322, 2024-02-17T13:07:17.030 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.012488191, 2024-02-17T13:07:17.030 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000038783 2024-02-17T13:07:17.030 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:17.030 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:17.031 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:17.031 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:17.031 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:17.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:17.031 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 1603120, 2024-02-17T13:07:17.031 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 193296 2024-02-17T13:07:17.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:17.031 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T13:07:17.032 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 3, 2024-02-17T13:07:17.032 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 245, 2024-02-17T13:07:17.032 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 480, 2024-02-17T13:07:17.032 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 275440, 2024-02-17T13:07:17.032 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 6, 2024-02-17T13:07:17.032 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 3, 2024-02-17T13:07:17.032 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.017223621, 2024-02-17T13:07:17.032 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.012121590, 2024-02-17T13:07:17.033 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.002222118, 2024-02-17T13:07:17.033 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T13:07:17.033 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:17.033 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 242, 2024-02-17T13:07:17.033 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.020599423, 2024-02-17T13:07:17.033 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000085121 2024-02-17T13:07:17.033 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:17.033 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:17.034 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:17.034 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:17.034 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:17.034 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:17.034 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-17T13:07:17.035 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 275440 2024-02-17T13:07:17.035 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:17.035 INFO:teuthology.orchestra.run.smithi018.stdout: "cct": { 2024-02-17T13:07:17.035 INFO:teuthology.orchestra.run.smithi018.stdout: "total_workers": 0, 2024-02-17T13:07:17.035 INFO:teuthology.orchestra.run.smithi018.stdout: "unhealthy_workers": 0 2024-02-17T13:07:17.035 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:17.035 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool": { 2024-02-17T13:07:17.035 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_bytes": 0, 2024-02-17T13:07:17.036 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_items": 0, 2024-02-17T13:07:17.036 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T13:07:17.036 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_items": 0, 2024-02-17T13:07:17.036 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T13:07:17.036 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_items": 0, 2024-02-17T13:07:17.036 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T13:07:17.036 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T13:07:17.036 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T13:07:17.037 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T13:07:17.037 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T13:07:17.037 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_items": 0, 2024-02-17T13:07:17.037 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T13:07:17.037 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T13:07:17.037 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_bytes": 0, 2024-02-17T13:07:17.037 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_items": 0, 2024-02-17T13:07:17.037 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_bytes": 0, 2024-02-17T13:07:17.038 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_items": 0, 2024-02-17T13:07:17.038 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T13:07:17.038 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T13:07:17.038 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T13:07:17.038 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T13:07:17.038 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T13:07:17.038 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_items": 0, 2024-02-17T13:07:17.038 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_bytes": 0, 2024-02-17T13:07:17.038 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_items": 0, 2024-02-17T13:07:17.039 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T13:07:17.039 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T13:07:17.039 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_bytes": 0, 2024-02-17T13:07:17.039 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_items": 0, 2024-02-17T13:07:17.039 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_bytes": 0, 2024-02-17T13:07:17.039 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_items": 0, 2024-02-17T13:07:17.039 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T13:07:17.039 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_items": 0, 2024-02-17T13:07:17.040 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T13:07:17.040 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_items": 0, 2024-02-17T13:07:17.040 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_bytes": 148731, 2024-02-17T13:07:17.040 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_items": 31, 2024-02-17T13:07:17.040 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_bytes": 0, 2024-02-17T13:07:17.040 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_items": 0, 2024-02-17T13:07:17.040 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bytes": 0, 2024-02-17T13:07:17.040 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_items": 0, 2024-02-17T13:07:17.041 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_bytes": 0, 2024-02-17T13:07:17.041 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_items": 0, 2024-02-17T13:07:17.041 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_bytes": 0, 2024-02-17T13:07:17.041 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_items": 0, 2024-02-17T13:07:17.041 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_bytes": 0, 2024-02-17T13:07:17.042 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_items": 0, 2024-02-17T13:07:17.042 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T13:07:17.042 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_items": 0, 2024-02-17T13:07:17.042 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_bytes": 0, 2024-02-17T13:07:17.042 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_items": 0, 2024-02-17T13:07:17.042 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_bytes": 928, 2024-02-17T13:07:17.042 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_items": 2, 2024-02-17T13:07:17.042 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_bytes": 0, 2024-02-17T13:07:17.042 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_items": 0, 2024-02-17T13:07:17.043 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_bytes": 0, 2024-02-17T13:07:17.043 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_items": 0 2024-02-17T13:07:17.043 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-02-17T13:07:17.043 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T13:07:17.043 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-02-17T13:07:17.043 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-02-17T13:07:17.043 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-02-17T13:07:17.043 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 334, 2024-02-17T13:07:17.044 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 1553389, 2024-02-17T13:07:17.044 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:17.044 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 334, 2024-02-17T13:07:17.044 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-02-17T13:07:17.044 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-02-17T13:07:17.044 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 334, 2024-02-17T13:07:17.044 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 1553389, 2024-02-17T13:07:17.044 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-02-17T13:07:17.045 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-02-17T13:07:17.045 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-02-17T13:07:17.045 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-02-17T13:07:17.045 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:17.045 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-02-17T13:07:17.045 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-02-17T13:07:17.739 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-02-17T13:07:17.740 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-02-17T13:07:17.740 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-02-17T13:07:17.740 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-02-17T13:07:17.740 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-02-17T13:07:17.740 DEBUG:tasks.check_counter:Getting stats from b 2024-02-17T13:07:17.740 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph daemon mds.b perf dump 2024-02-17T13:07:18.153 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T13:07:18.465 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:18 smithi119 ceph-mon[36188]: pgmap v549: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 5.2 MiB/s wr, 204 op/s 2024-02-17T13:07:18.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:18 smithi018 ceph-mon[32625]: pgmap v549: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 5.2 MiB/s wr, 204 op/s 2024-02-17T13:07:18.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:18 smithi146 ceph-mon[36454]: pgmap v549: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 5.2 MiB/s wr, 204 op/s 2024-02-17T13:07:19.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:19 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:07:19.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:19 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:07:19.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:19 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T13:07:20.371 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:20 smithi119 ceph-mon[36188]: pgmap v550: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 5.1 MiB/s wr, 198 op/s 2024-02-17T13:07:20.371 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:20 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:07:20.371 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:20 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:07:20.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:20 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:07:20.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:20 smithi018 ceph-mon[32625]: pgmap v550: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 5.1 MiB/s wr, 198 op/s 2024-02-17T13:07:20.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:20 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:07:20.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:20 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:07:20.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:20 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:07:20.546 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-02-17T13:07:20.546 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T13:07:20.546 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 138222, 2024-02-17T13:07:20.546 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 169410, 2024-02-17T13:07:20.546 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 35923120, 2024-02-17T13:07:20.547 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 263652955, 2024-02-17T13:07:20.547 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 14, 2024-02-17T13:07:20.547 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-02-17T13:07:20.547 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 12.000895673, 2024-02-17T13:07:20.547 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 5.017327207, 2024-02-17T13:07:20.547 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 5.435791455, 2024-02-17T13:07:20.547 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 2.130115860, 2024-02-17T13:07:20.547 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:20.548 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 169408, 2024-02-17T13:07:20.548 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 19.468515579, 2024-02-17T13:07:20.548 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000114920 2024-02-17T13:07:20.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.548 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:20.548 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 230633, 2024-02-17T13:07:20.548 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.043358418, 2024-02-17T13:07:20.548 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000187 2024-02-17T13:07:20.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.549 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 2588656, 2024-02-17T13:07:20.549 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 417904 2024-02-17T13:07:20.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.549 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T13:07:20.549 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 174579, 2024-02-17T13:07:20.549 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 207645, 2024-02-17T13:07:20.549 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 49610926, 2024-02-17T13:07:20.550 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 206631158, 2024-02-17T13:07:20.550 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 368, 2024-02-17T13:07:20.550 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 7, 2024-02-17T13:07:20.550 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 12.367362232, 2024-02-17T13:07:20.550 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 5.357061401, 2024-02-17T13:07:20.550 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 7.065121158, 2024-02-17T13:07:20.550 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.979987508, 2024-02-17T13:07:20.550 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:20.551 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 207644, 2024-02-17T13:07:20.551 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 22.392395083, 2024-02-17T13:07:20.551 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000107840 2024-02-17T13:07:20.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.551 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:20.551 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 323239, 2024-02-17T13:07:20.551 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.056292927, 2024-02-17T13:07:20.551 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000174 2024-02-17T13:07:20.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.552 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 42384, 2024-02-17T13:07:20.552 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 129680 2024-02-17T13:07:20.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.552 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T13:07:20.552 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 396303, 2024-02-17T13:07:20.552 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 514377, 2024-02-17T13:07:20.552 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 114615964, 2024-02-17T13:07:20.553 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 511619979, 2024-02-17T13:07:20.553 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 17, 2024-02-17T13:07:20.553 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-02-17T13:07:20.553 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 26.611423576, 2024-02-17T13:07:20.553 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 11.563827854, 2024-02-17T13:07:20.553 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 16.551475777, 2024-02-17T13:07:20.553 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 1.680305650, 2024-02-17T13:07:20.553 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:20.554 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 514375, 2024-02-17T13:07:20.554 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 46.085157462, 2024-02-17T13:07:20.554 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000089594 2024-02-17T13:07:20.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.554 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:20.554 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 760440, 2024-02-17T13:07:20.554 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.149869990, 2024-02-17T13:07:20.554 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000197 2024-02-17T13:07:20.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.555 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 58320, 2024-02-17T13:07:20.555 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 495168 2024-02-17T13:07:20.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.555 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-02-17T13:07:20.555 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-02-17T13:07:20.555 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-02-17T13:07:20.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.556 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-02-17T13:07:20.556 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-02-17T13:07:20.556 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-02-17T13:07:20.556 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 106779, 2024-02-17T13:07:20.556 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 99.525917708, 2024-02-17T13:07:20.556 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000932073 2024-02-17T13:07:20.556 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:20.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.557 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-02-17T13:07:20.557 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-02-17T13:07:20.557 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-02-17T13:07:20.557 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 32499, 2024-02-17T13:07:20.557 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 8.400302792, 2024-02-17T13:07:20.557 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000258478 2024-02-17T13:07:20.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:20.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.558 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-02-17T13:07:20.558 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 177129, 2024-02-17T13:07:20.558 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 156168, 2024-02-17T13:07:20.558 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-02-17T13:07:20.558 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 156168, 2024-02-17T13:07:20.558 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1266.905531048, 2024-02-17T13:07:20.558 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.008112452 2024-02-17T13:07:20.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.559 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-02-17T13:07:20.559 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 20961, 2024-02-17T13:07:20.559 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 7, 2024-02-17T13:07:20.559 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-02-17T13:07:20.559 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 2947, 2024-02-17T13:07:20.559 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 16, 2024-02-17T13:07:20.559 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 15, 2024-02-17T13:07:20.559 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 1969, 2024-02-17T13:07:20.559 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 669, 2024-02-17T13:07:20.560 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 1160, 2024-02-17T13:07:20.560 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 140, 2024-02-17T13:07:20.560 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 1013, 2024-02-17T13:07:20.560 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 3069, 2024-02-17T13:07:20.560 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 486, 2024-02-17T13:07:20.560 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 486, 2024-02-17T13:07:20.560 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 27, 2024-02-17T13:07:20.561 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 267164, 2024-02-17T13:07:20.561 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 173344, 2024-02-17T13:07:20.561 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 20944, 2024-02-17T13:07:20.561 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 160, 2024-02-17T13:07:20.561 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-02-17T13:07:20.561 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-02-17T13:07:20.561 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 1408, 2024-02-17T13:07:20.561 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 96167, 2024-02-17T13:07:20.561 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 13, 2024-02-17T13:07:20.562 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 33, 2024-02-17T13:07:20.562 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 2093, 2024-02-17T13:07:20.562 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 14, 2024-02-17T13:07:20.562 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 556, 2024-02-17T13:07:20.562 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-02-17T13:07:20.562 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 1636, 2024-02-17T13:07:20.562 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-02-17T13:07:20.562 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 682, 2024-02-17T13:07:20.563 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 211764133, 2024-02-17T13:07:20.563 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 7, 2024-02-17T13:07:20.563 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 1636, 2024-02-17T13:07:20.563 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-02-17T13:07:20.563 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T13:07:20.563 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-02-17T13:07:20.563 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 602, 2024-02-17T13:07:20.563 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 26, 2024-02-17T13:07:20.563 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 576, 2024-02-17T13:07:20.564 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 1034, 2024-02-17T13:07:20.564 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 19, 2024-02-17T13:07:20.564 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 51307, 2024-02-17T13:07:20.564 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 81995, 2024-02-17T13:07:20.564 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T13:07:20.564 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 2592, 2024-02-17T13:07:20.564 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T13:07:20.564 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 78632, 2024-02-17T13:07:20.565 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 19560, 2024-02-17T13:07:20.565 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 7038, 2024-02-17T13:07:20.565 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 92482 2024-02-17T13:07:20.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.565 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-02-17T13:07:20.565 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 50, 2024-02-17T13:07:20.565 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 47, 2024-02-17T13:07:20.565 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 28, 2024-02-17T13:07:20.566 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 190, 2024-02-17T13:07:20.566 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 362, 2024-02-17T13:07:20.566 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 261, 2024-02-17T13:07:20.566 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-02-17T13:07:20.566 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-02-17T13:07:20.566 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 18809, 2024-02-17T13:07:20.566 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 18548, 2024-02-17T13:07:20.566 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-02-17T13:07:20.567 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-02-17T13:07:20.567 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-02-17T13:07:20.567 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-02-17T13:07:20.567 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-02-17T13:07:20.567 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-02-17T13:07:20.567 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-02-17T13:07:20.567 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 26, 2024-02-17T13:07:20.567 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 34, 2024-02-17T13:07:20.567 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-02-17T13:07:20.568 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 31, 2024-02-17T13:07:20.568 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-02-17T13:07:20.568 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0 2024-02-17T13:07:20.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.568 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-02-17T13:07:20.568 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-02-17T13:07:20.568 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 118646, 2024-02-17T13:07:20.568 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 117419, 2024-02-17T13:07:20.569 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 117419, 2024-02-17T13:07:20.569 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 1227, 2024-02-17T13:07:20.569 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-02-17T13:07:20.569 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 3004, 2024-02-17T13:07:20.569 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 239, 2024-02-17T13:07:20.569 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 237, 2024-02-17T13:07:20.569 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 237, 2024-02-17T13:07:20.569 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 2, 2024-02-17T13:07:20.570 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 1, 2024-02-17T13:07:20.570 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-02-17T13:07:20.570 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 4, 2024-02-17T13:07:20.570 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 509740127, 2024-02-17T13:07:20.570 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 513457860, 2024-02-17T13:07:20.570 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 0, 2024-02-17T13:07:20.570 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-02-17T13:07:20.570 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 56484, 2024-02-17T13:07:20.570 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 287.971877628, 2024-02-17T13:07:20.571 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.005098291 2024-02-17T13:07:20.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.571 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 0 2024-02-17T13:07:20.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.571 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-02-17T13:07:20.571 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 1229, 2024-02-17T13:07:20.571 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 21426, 2024-02-17T13:07:20.571 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 20197, 2024-02-17T13:07:20.572 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 182, 2024-02-17T13:07:20.572 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 385, 2024-02-17T13:07:20.572 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 203, 2024-02-17T13:07:20.572 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 1989, 2024-02-17T13:07:20.572 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 43938, 2024-02-17T13:07:20.572 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 41949, 2024-02-17T13:07:20.572 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 428, 2024-02-17T13:07:20.572 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 21825, 2024-02-17T13:07:20.573 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 21397, 2024-02-17T13:07:20.573 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 94576, 2024-02-17T13:07:20.573 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 215420 2024-02-17T13:07:20.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.573 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-02-17T13:07:20.573 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 250724, 2024-02-17T13:07:20.573 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 26327, 2024-02-17T13:07:20.573 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 177129, 2024-02-17T13:07:20.573 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 2569, 2024-02-17T13:07:20.574 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 54631, 2024-02-17T13:07:20.574 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-02-17T13:07:20.574 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 19488, 2024-02-17T13:07:20.574 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 118.772498027, 2024-02-17T13:07:20.574 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006094647 2024-02-17T13:07:20.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.574 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-02-17T13:07:20.574 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 32401, 2024-02-17T13:07:20.574 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 222.680030373, 2024-02-17T13:07:20.575 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006872628 2024-02-17T13:07:20.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.575 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-02-17T13:07:20.575 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.575 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.575 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.575 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-02-17T13:07:20.576 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.576 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.576 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.576 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-02-17T13:07:20.576 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 35955, 2024-02-17T13:07:20.576 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 232.253814414, 2024-02-17T13:07:20.576 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006459569 2024-02-17T13:07:20.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.577 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-02-17T13:07:20.577 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.577 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.577 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.577 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-02-17T13:07:20.577 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1, 2024-02-17T13:07:20.578 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000105643, 2024-02-17T13:07:20.578 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000105643 2024-02-17T13:07:20.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.578 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-02-17T13:07:20.578 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.578 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.578 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.578 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-02-17T13:07:20.579 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.579 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.579 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.579 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-02-17T13:07:20.579 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.580 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.580 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.580 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-02-17T13:07:20.580 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 18, 2024-02-17T13:07:20.581 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.007685734, 2024-02-17T13:07:20.581 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000426985 2024-02-17T13:07:20.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.581 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-02-17T13:07:20.581 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 119, 2024-02-17T13:07:20.581 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.668492697, 2024-02-17T13:07:20.581 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.005617585 2024-02-17T13:07:20.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.582 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-02-17T13:07:20.582 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.582 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.582 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.582 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-02-17T13:07:20.582 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 18, 2024-02-17T13:07:20.583 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 18.650941259, 2024-02-17T13:07:20.583 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 1.036163403 2024-02-17T13:07:20.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.583 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-02-17T13:07:20.583 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 9, 2024-02-17T13:07:20.583 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.013863723, 2024-02-17T13:07:20.583 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001540413 2024-02-17T13:07:20.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.583 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-02-17T13:07:20.584 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 45130, 2024-02-17T13:07:20.584 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 265.056598439, 2024-02-17T13:07:20.584 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.005873179 2024-02-17T13:07:20.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.584 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-02-17T13:07:20.584 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3594, 2024-02-17T13:07:20.584 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 281.374799270, 2024-02-17T13:07:20.584 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.078290150 2024-02-17T13:07:20.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.585 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-02-17T13:07:20.585 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.585 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.585 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.585 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-02-17T13:07:20.585 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.586 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.586 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.586 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-02-17T13:07:20.586 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1, 2024-02-17T13:07:20.586 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000915672, 2024-02-17T13:07:20.586 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000915672 2024-02-17T13:07:20.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.586 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-02-17T13:07:20.587 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 11, 2024-02-17T13:07:20.587 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1.976422028, 2024-02-17T13:07:20.587 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.179674729 2024-02-17T13:07:20.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.587 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-02-17T13:07:20.587 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.587 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.587 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.588 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-02-17T13:07:20.588 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 20, 2024-02-17T13:07:20.588 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.283877013, 2024-02-17T13:07:20.588 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.014193850 2024-02-17T13:07:20.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.588 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-02-17T13:07:20.588 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.589 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.589 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.589 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-02-17T13:07:20.589 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 587, 2024-02-17T13:07:20.589 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 2.244224157, 2024-02-17T13:07:20.589 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.003823209 2024-02-17T13:07:20.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.589 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-02-17T13:07:20.590 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.590 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.590 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.590 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-02-17T13:07:20.590 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 7, 2024-02-17T13:07:20.590 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.019491934, 2024-02-17T13:07:20.590 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.002784562 2024-02-17T13:07:20.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.591 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-02-17T13:07:20.591 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.591 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.591 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.591 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-02-17T13:07:20.591 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 18809, 2024-02-17T13:07:20.592 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 122.901770665, 2024-02-17T13:07:20.592 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006534200 2024-02-17T13:07:20.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.592 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-02-17T13:07:20.592 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-02-17T13:07:20.592 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-02-17T13:07:20.592 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.592 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.593 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.593 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:20.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.593 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-02-17T13:07:20.593 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 2, 2024-02-17T13:07:20.593 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 4, 2024-02-17T13:07:20.593 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 2, 2024-02-17T13:07:20.593 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 2, 2024-02-17T13:07:20.593 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-02-17T13:07:20.594 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 27453, 2024-02-17T13:07:20.594 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 13726, 2024-02-17T13:07:20.594 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 1051, 2024-02-17T13:07:20.594 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-02-17T13:07:20.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.594 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-02-17T13:07:20.594 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 360, 2024-02-17T13:07:20.594 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 360, 2024-02-17T13:07:20.595 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T13:07:20.595 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-02-17T13:07:20.595 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T13:07:20.595 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-02-17T13:07:20.595 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T13:07:20.595 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T13:07:20.596 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T13:07:20.596 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T13:07:20.596 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T13:07:20.596 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-02-17T13:07:20.596 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T13:07:20.596 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T13:07:20.596 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-02-17T13:07:20.596 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-02-17T13:07:20.597 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-02-17T13:07:20.597 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-02-17T13:07:20.597 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T13:07:20.597 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T13:07:20.597 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T13:07:20.597 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T13:07:20.597 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T13:07:20.597 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-02-17T13:07:20.598 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-02-17T13:07:20.598 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-02-17T13:07:20.598 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T13:07:20.598 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T13:07:20.598 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-02-17T13:07:20.598 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-02-17T13:07:20.598 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-02-17T13:07:20.598 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-02-17T13:07:20.599 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T13:07:20.599 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-02-17T13:07:20.599 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T13:07:20.599 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-02-17T13:07:20.599 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 1238348, 2024-02-17T13:07:20.599 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 1283, 2024-02-17T13:07:20.599 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-02-17T13:07:20.599 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-02-17T13:07:20.599 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-02-17T13:07:20.600 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-02-17T13:07:20.600 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-02-17T13:07:20.600 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-02-17T13:07:20.600 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-02-17T13:07:20.600 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-02-17T13:07:20.600 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 7936, 2024-02-17T13:07:20.600 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 165, 2024-02-17T13:07:20.600 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T13:07:20.601 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-02-17T13:07:20.601 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-02-17T13:07:20.601 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-02-17T13:07:20.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 5159239, 2024-02-17T13:07:20.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 91549, 2024-02-17T13:07:20.601 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-02-17T13:07:20.601 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-02-17T13:07:20.601 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-02-17T13:07:20.602 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-02-17T13:07:20.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.602 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-02-17T13:07:20.602 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-02-17T13:07:20.602 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-02-17T13:07:20.602 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 95398, 2024-02-17T13:07:20.602 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 534471989, 2024-02-17T13:07:20.602 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 37, 2024-02-17T13:07:20.602 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 95398, 2024-02-17T13:07:20.603 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-02-17T13:07:20.603 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 95398, 2024-02-17T13:07:20.603 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1349.100230364, 2024-02-17T13:07:20.603 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.014141808 2024-02-17T13:07:20.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.603 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-02-17T13:07:20.603 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-02-17T13:07:20.603 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 95398, 2024-02-17T13:07:20.603 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 126456 2024-02-17T13:07:20.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.604 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 95398, 2024-02-17T13:07:20.604 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 2467, 2024-02-17T13:07:20.604 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 92931, 2024-02-17T13:07:20.604 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-02-17T13:07:20.604 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-02-17T13:07:20.604 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 2673, 2024-02-17T13:07:20.604 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 11713, 2024-02-17T13:07:20.605 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 798, 2024-02-17T13:07:20.605 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 57256, 2024-02-17T13:07:20.605 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 712, 2024-02-17T13:07:20.605 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-02-17T13:07:20.605 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-02-17T13:07:20.605 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-02-17T13:07:20.605 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 16, 2024-02-17T13:07:20.605 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 19382, 2024-02-17T13:07:20.606 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-02-17T13:07:20.606 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-02-17T13:07:20.606 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-02-17T13:07:20.606 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 1664, 2024-02-17T13:07:20.606 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 23426, 2024-02-17T13:07:20.606 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-02-17T13:07:20.606 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-02-17T13:07:20.606 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-02-17T13:07:20.606 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-02-17T13:07:20.607 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-02-17T13:07:20.607 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-02-17T13:07:20.607 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T13:07:20.607 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-02-17T13:07:20.607 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-02-17T13:07:20.607 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-02-17T13:07:20.607 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-02-17T13:07:20.607 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-02-17T13:07:20.608 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-02-17T13:07:20.608 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-02-17T13:07:20.608 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-02-17T13:07:20.608 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-02-17T13:07:20.608 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-02-17T13:07:20.608 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-02-17T13:07:20.608 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-02-17T13:07:20.608 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-02-17T13:07:20.609 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-02-17T13:07:20.609 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-02-17T13:07:20.609 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-02-17T13:07:20.609 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-02-17T13:07:20.609 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-02-17T13:07:20.609 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-02-17T13:07:20.609 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 113, 2024-02-17T13:07:20.609 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-02-17T13:07:20.610 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 28, 2024-02-17T13:07:20.610 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 12, 2024-02-17T13:07:20.610 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 12, 2024-02-17T13:07:20.610 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2024-02-17T13:07:20.610 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-02-17T13:07:20.610 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 6954, 2024-02-17T13:07:20.610 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 14, 2024-02-17T13:07:20.610 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 1848 2024-02-17T13:07:20.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.611 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-02-17T13:07:20.611 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 1, 2024-02-17T13:07:20.611 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-02-17T13:07:20.611 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-02-17T13:07:20.611 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-02-17T13:07:20.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.611 INFO:teuthology.orchestra.run.smithi119.stdout: "osd-slow-ops": { 2024-02-17T13:07:20.611 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 2 2024-02-17T13:07:20.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.612 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-02-17T13:07:20.612 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-02-17T13:07:20.612 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 6, 2024-02-17T13:07:20.612 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-02-17T13:07:20.612 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 6, 2024-02-17T13:07:20.612 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 18548, 2024-02-17T13:07:20.612 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 18548, 2024-02-17T13:07:20.613 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-02-17T13:07:20.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.613 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T13:07:20.613 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:20.613 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-02-17T13:07:20.613 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:20.613 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 709100, 2024-02-17T13:07:20.613 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 136233494, 2024-02-17T13:07:20.613 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:20.614 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 709100, 2024-02-17T13:07:20.614 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-02-17T13:07:20.614 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-02-17T13:07:20.614 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 709100, 2024-02-17T13:07:20.614 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 136233494, 2024-02-17T13:07:20.614 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:20.614 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.614 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.615 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.615 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:20.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.615 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-02-17T13:07:20.615 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:20.615 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-02-17T13:07:20.615 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:20.615 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-02-17T13:07:20.616 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-02-17T13:07:20.616 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:20.616 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:20.616 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 95398, 2024-02-17T13:07:20.616 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 536347905, 2024-02-17T13:07:20.616 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 95398, 2024-02-17T13:07:20.616 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 536347905, 2024-02-17T13:07:20.616 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:20.616 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.617 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.617 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.617 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:20.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.617 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-02-17T13:07:20.617 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:20.617 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-02-17T13:07:20.617 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:20.618 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-02-17T13:07:20.618 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-02-17T13:07:20.618 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:20.618 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:20.618 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 95398, 2024-02-17T13:07:20.618 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 95398, 2024-02-17T13:07:20.618 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 95398, 2024-02-17T13:07:20.618 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 95398, 2024-02-17T13:07:20.619 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:20.619 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.619 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.619 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.619 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:20.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.619 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-02-17T13:07:20.619 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:20.620 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-02-17T13:07:20.620 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:20.620 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 18548, 2024-02-17T13:07:20.620 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 1876028, 2024-02-17T13:07:20.620 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:20.620 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 18548, 2024-02-17T13:07:20.620 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-02-17T13:07:20.620 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-02-17T13:07:20.621 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 772, 2024-02-17T13:07:20.621 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1876028, 2024-02-17T13:07:20.621 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:20.621 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.621 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.621 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:20.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:20.623 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x556ad04f46e0": { 2024-02-17T13:07:20.623 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 34742, 2024-02-17T13:07:20.623 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-02-17T13:07:20.624 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:20.624 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 118646, 2024-02-17T13:07:20.624 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 509263556, 2024-02-17T13:07:20.624 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:20.624 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 118646, 2024-02-17T13:07:20.624 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-02-17T13:07:20.624 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-02-17T13:07:20.624 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 56484, 2024-02-17T13:07:20.625 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 509228814, 2024-02-17T13:07:20.625 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:20.625 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:20.625 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:20.625 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:20.625 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:20.625 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:20.625 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-02-17T13:07:20.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:20 smithi146 ceph-mon[36454]: pgmap v550: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 5.1 MiB/s wr, 198 op/s 2024-02-17T13:07:20.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:20 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T13:07:20.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:20 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T13:07:20.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:20 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' 2024-02-17T13:07:22.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:22 smithi018 ceph-mon[32625]: pgmap v551: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 5.7 MiB/s wr, 201 op/s 2024-02-17T13:07:22.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:22 smithi119 ceph-mon[36188]: pgmap v551: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 5.7 MiB/s wr, 201 op/s 2024-02-17T13:07:22.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:22 smithi146 ceph-mon[36454]: pgmap v551: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 5.7 MiB/s wr, 201 op/s 2024-02-17T13:07:23.213 INFO:tasks.check_counter:Daemon mds.b mds.exported=33 2024-02-17T13:07:23.213 INFO:tasks.check_counter:Daemon mds.b mds.imported=14 2024-02-17T13:07:23.213 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-02-17T13:07:23.213 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=18 2024-02-17T13:07:23.213 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=11 2024-02-17T13:07:23.213 DEBUG:tasks.check_counter:Getting stats from e 2024-02-17T13:07:23.214 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph daemon mds.e perf dump 2024-02-17T13:07:23.480 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T13:07:24.267 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:24 smithi119 ceph-mon[36188]: pgmap v552: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 5.7 MiB/s wr, 201 op/s 2024-02-17T13:07:24.268 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:24 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:07:24.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:24 smithi018 ceph-mon[32625]: pgmap v552: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 5.7 MiB/s wr, 201 op/s 2024-02-17T13:07:24.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:24 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:07:24.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:24 smithi146 ceph-mon[36454]: pgmap v552: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 5.7 MiB/s wr, 201 op/s 2024-02-17T13:07:24.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:24 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:07:25.952 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-02-17T13:07:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T13:07:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 175316, 2024-02-17T13:07:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 138916, 2024-02-17T13:07:25.952 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 57815703, 2024-02-17T13:07:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 120622494, 2024-02-17T13:07:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 9, 2024-02-17T13:07:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 3, 2024-02-17T13:07:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 11.004854391, 2024-02-17T13:07:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 4.672561138, 2024-02-17T13:07:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 7.023668486, 2024-02-17T13:07:25.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.660121393, 2024-02-17T13:07:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 138919, 2024-02-17T13:07:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 8.352388348, 2024-02-17T13:07:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000060124 2024-02-17T13:07:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 287378, 2024-02-17T13:07:25.954 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.049177207, 2024-02-17T13:07:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000171 2024-02-17T13:07:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 2025456, 2024-02-17T13:07:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 232704 2024-02-17T13:07:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T13:07:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 173714, 2024-02-17T13:07:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 257172, 2024-02-17T13:07:25.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 52799906, 2024-02-17T13:07:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 245074853, 2024-02-17T13:07:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 66, 2024-02-17T13:07:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-02-17T13:07:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 13.313031926, 2024-02-17T13:07:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 5.509350109, 2024-02-17T13:07:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 7.873515496, 2024-02-17T13:07:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.480586703, 2024-02-17T13:07:25.956 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 257170, 2024-02-17T13:07:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 27.316358452, 2024-02-17T13:07:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000106219 2024-02-17T13:07:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 363564, 2024-02-17T13:07:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.085279931, 2024-02-17T13:07:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000234 2024-02-17T13:07:25.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 4992, 2024-02-17T13:07:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 132608 2024-02-17T13:07:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T13:07:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 38527, 2024-02-17T13:07:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 36082, 2024-02-17T13:07:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 12959108, 2024-02-17T13:07:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 87243785, 2024-02-17T13:07:25.958 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 15, 2024-02-17T13:07:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 9, 2024-02-17T13:07:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 3.919054270, 2024-02-17T13:07:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 1.681698757, 2024-02-17T13:07:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 1.379767227, 2024-02-17T13:07:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.933732490, 2024-02-17T13:07:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 36076, 2024-02-17T13:07:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 4.178239453, 2024-02-17T13:07:25.959 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000115817 2024-02-17T13:07:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 37755, 2024-02-17T13:07:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.008273354, 2024-02-17T13:07:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000219 2024-02-17T13:07:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.960 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 2096, 2024-02-17T13:07:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 464560 2024-02-17T13:07:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-02-17T13:07:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-02-17T13:07:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-02-17T13:07:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-02-17T13:07:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-02-17T13:07:25.961 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-02-17T13:07:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 50263, 2024-02-17T13:07:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 52.099175523, 2024-02-17T13:07:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001036531 2024-02-17T13:07:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-02-17T13:07:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-02-17T13:07:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-02-17T13:07:25.962 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 26873, 2024-02-17T13:07:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 3.874360811, 2024-02-17T13:07:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000144172 2024-02-17T13:07:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-02-17T13:07:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 91921, 2024-02-17T13:07:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 86763, 2024-02-17T13:07:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-02-17T13:07:25.963 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 86763, 2024-02-17T13:07:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 317.169547535, 2024-02-17T13:07:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.003655585 2024-02-17T13:07:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-02-17T13:07:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 5158, 2024-02-17T13:07:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2024-02-17T13:07:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-02-17T13:07:25.964 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 28, 2024-02-17T13:07:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 2, 2024-02-17T13:07:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 1, 2024-02-17T13:07:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 1468, 2024-02-17T13:07:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 249, 2024-02-17T13:07:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 1093, 2024-02-17T13:07:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 126, 2024-02-17T13:07:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 1107, 2024-02-17T13:07:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 66, 2024-02-17T13:07:25.965 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 338, 2024-02-17T13:07:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 338, 2024-02-17T13:07:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 40, 2024-02-17T13:07:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 194896, 2024-02-17T13:07:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 101715, 2024-02-17T13:07:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 5148, 2024-02-17T13:07:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 56, 2024-02-17T13:07:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-02-17T13:07:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-02-17T13:07:25.966 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 3876, 2024-02-17T13:07:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 84000, 2024-02-17T13:07:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-02-17T13:07:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 1, 2024-02-17T13:07:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 48, 2024-02-17T13:07:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 17, 2024-02-17T13:07:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 632, 2024-02-17T13:07:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-02-17T13:07:25.967 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 346, 2024-02-17T13:07:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-02-17T13:07:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 735, 2024-02-17T13:07:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 211953355, 2024-02-17T13:07:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 7, 2024-02-17T13:07:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 346, 2024-02-17T13:07:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-02-17T13:07:25.968 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T13:07:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-02-17T13:07:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 0, 2024-02-17T13:07:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T13:07:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T13:07:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 346, 2024-02-17T13:07:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 2, 2024-02-17T13:07:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 6570, 2024-02-17T13:07:25.969 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 58956, 2024-02-17T13:07:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T13:07:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 1541, 2024-02-17T13:07:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T13:07:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 27392, 2024-02-17T13:07:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 14822, 2024-02-17T13:07:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 4191, 2024-02-17T13:07:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 80096 2024-02-17T13:07:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.970 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-02-17T13:07:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 33, 2024-02-17T13:07:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 126, 2024-02-17T13:07:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 79, 2024-02-17T13:07:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 151, 2024-02-17T13:07:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 88, 2024-02-17T13:07:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 236, 2024-02-17T13:07:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 93, 2024-02-17T13:07:25.971 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-02-17T13:07:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 15121, 2024-02-17T13:07:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 14885, 2024-02-17T13:07:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-02-17T13:07:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-02-17T13:07:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-02-17T13:07:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-02-17T13:07:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-02-17T13:07:25.972 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-02-17T13:07:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-02-17T13:07:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T13:07:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 1, 2024-02-17T13:07:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-02-17T13:07:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 3, 2024-02-17T13:07:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-02-17T13:07:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0 2024-02-17T13:07:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.973 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-02-17T13:07:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-02-17T13:07:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 88053, 2024-02-17T13:07:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2024-02-17T13:07:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2024-02-17T13:07:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 88053, 2024-02-17T13:07:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-02-17T13:07:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2024-02-17T13:07:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 128, 2024-02-17T13:07:25.974 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2024-02-17T13:07:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2024-02-17T13:07:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 128, 2024-02-17T13:07:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 9, 2024-02-17T13:07:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-02-17T13:07:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2024-02-17T13:07:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 4194304, 2024-02-17T13:07:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 242073129, 2024-02-17T13:07:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 0, 2024-02-17T13:07:25.975 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-02-17T13:07:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 26377, 2024-02-17T13:07:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 129.657215901, 2024-02-17T13:07:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004915540 2024-02-17T13:07:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 0 2024-02-17T13:07:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.976 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-02-17T13:07:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 1037, 2024-02-17T13:07:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 15973, 2024-02-17T13:07:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 14936, 2024-02-17T13:07:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 147, 2024-02-17T13:07:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 229, 2024-02-17T13:07:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 82, 2024-02-17T13:07:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 1468, 2024-02-17T13:07:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 22645, 2024-02-17T13:07:25.977 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 21177, 2024-02-17T13:07:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 275, 2024-02-17T13:07:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 16072, 2024-02-17T13:07:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 15797, 2024-02-17T13:07:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 70404, 2024-02-17T13:07:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 207228 2024-02-17T13:07:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-02-17T13:07:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 163183, 2024-02-17T13:07:25.978 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 1097, 2024-02-17T13:07:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 91921, 2024-02-17T13:07:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 2871, 2024-02-17T13:07:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 19453, 2024-02-17T13:07:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-02-17T13:07:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 15103, 2024-02-17T13:07:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 91.428196127, 2024-02-17T13:07:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006053644 2024-02-17T13:07:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.979 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-02-17T13:07:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1441, 2024-02-17T13:07:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 25.834694507, 2024-02-17T13:07:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.017928309 2024-02-17T13:07:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-02-17T13:07:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.980 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-02-17T13:07:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-02-17T13:07:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 23856, 2024-02-17T13:07:25.981 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 43.973888776, 2024-02-17T13:07:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001843305 2024-02-17T13:07:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-02-17T13:07:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.982 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-02-17T13:07:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-02-17T13:07:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-02-17T13:07:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-02-17T13:07:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.984 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-02-17T13:07:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-02-17T13:07:25.985 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-02-17T13:07:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-02-17T13:07:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-02-17T13:07:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.987 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-02-17T13:07:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 25900, 2024-02-17T13:07:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 73.402607907, 2024-02-17T13:07:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.002834077 2024-02-17T13:07:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-02-17T13:07:25.988 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4385, 2024-02-17T13:07:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 21.907247672, 2024-02-17T13:07:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004995951 2024-02-17T13:07:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-02-17T13:07:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-02-17T13:07:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-02-17T13:07:25.990 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-02-17T13:07:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-02-17T13:07:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-02-17T13:07:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 17, 2024-02-17T13:07:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.165287142, 2024-02-17T13:07:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.009722773 2024-02-17T13:07:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-02-17T13:07:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-02-17T13:07:25.993 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 940, 2024-02-17T13:07:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1.380590313, 2024-02-17T13:07:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001468713 2024-02-17T13:07:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-02-17T13:07:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-02-17T13:07:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-02-17T13:07:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.995 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-02-17T13:07:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 15121, 2024-02-17T13:07:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 59.077035091, 2024-02-17T13:07:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.003906952 2024-02-17T13:07:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-02-17T13:07:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-02-17T13:07:25.996 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-02-17T13:07:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-02-17T13:07:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 1, 2024-02-17T13:07:25.997 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 3, 2024-02-17T13:07:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 2, 2024-02-17T13:07:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 1, 2024-02-17T13:07:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-02-17T13:07:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 22656, 2024-02-17T13:07:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 22656, 2024-02-17T13:07:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 995, 2024-02-17T13:07:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-02-17T13:07:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:25.998 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-02-17T13:07:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-02-17T13:07:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-02-17T13:07:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T13:07:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-02-17T13:07:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T13:07:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-02-17T13:07:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T13:07:25.999 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T13:07:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T13:07:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T13:07:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T13:07:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-02-17T13:07:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T13:07:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T13:07:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-02-17T13:07:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-02-17T13:07:26.000 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-02-17T13:07:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-02-17T13:07:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T13:07:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T13:07:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T13:07:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T13:07:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T13:07:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-02-17T13:07:26.001 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-02-17T13:07:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-02-17T13:07:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T13:07:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T13:07:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-02-17T13:07:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-02-17T13:07:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-02-17T13:07:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-02-17T13:07:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T13:07:26.002 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-02-17T13:07:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T13:07:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-02-17T13:07:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 2331554, 2024-02-17T13:07:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 1360, 2024-02-17T13:07:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 440, 2024-02-17T13:07:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 5, 2024-02-17T13:07:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-02-17T13:07:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-02-17T13:07:26.003 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-02-17T13:07:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-02-17T13:07:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-02-17T13:07:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-02-17T13:07:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 7936, 2024-02-17T13:07:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 165, 2024-02-17T13:07:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T13:07:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-02-17T13:07:26.004 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-02-17T13:07:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-02-17T13:07:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 4194229, 2024-02-17T13:07:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 73123, 2024-02-17T13:07:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-02-17T13:07:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-02-17T13:07:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-02-17T13:07:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-02-17T13:07:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:26.005 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-02-17T13:07:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-02-17T13:07:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-02-17T13:07:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 44627, 2024-02-17T13:07:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 241115971, 2024-02-17T13:07:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 33, 2024-02-17T13:07:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 44627, 2024-02-17T13:07:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-02-17T13:07:26.006 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 44627, 2024-02-17T13:07:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 221.003490994, 2024-02-17T13:07:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004952237 2024-02-17T13:07:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-02-17T13:07:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-02-17T13:07:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 44627, 2024-02-17T13:07:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 45838 2024-02-17T13:07:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:26.007 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 44627, 2024-02-17T13:07:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 1186, 2024-02-17T13:07:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 43441, 2024-02-17T13:07:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-02-17T13:07:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-02-17T13:07:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 0, 2024-02-17T13:07:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 10, 2024-02-17T13:07:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 840, 2024-02-17T13:07:26.008 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 27217, 2024-02-17T13:07:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 485, 2024-02-17T13:07:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-02-17T13:07:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-02-17T13:07:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-02-17T13:07:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 2, 2024-02-17T13:07:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 14961, 2024-02-17T13:07:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-02-17T13:07:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-02-17T13:07:26.009 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-02-17T13:07:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 346, 2024-02-17T13:07:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 20, 2024-02-17T13:07:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-02-17T13:07:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-02-17T13:07:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-02-17T13:07:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-02-17T13:07:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-02-17T13:07:26.010 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-02-17T13:07:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T13:07:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-02-17T13:07:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-02-17T13:07:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-02-17T13:07:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-02-17T13:07:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-02-17T13:07:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-02-17T13:07:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-02-17T13:07:26.011 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-02-17T13:07:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-02-17T13:07:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-02-17T13:07:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-02-17T13:07:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-02-17T13:07:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-02-17T13:07:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-02-17T13:07:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-02-17T13:07:26.012 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-02-17T13:07:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-02-17T13:07:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-02-17T13:07:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-02-17T13:07:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 113, 2024-02-17T13:07:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-02-17T13:07:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 27, 2024-02-17T13:07:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 12, 2024-02-17T13:07:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 12, 2024-02-17T13:07:26.013 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2024-02-17T13:07:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-02-17T13:07:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 1423, 2024-02-17T13:07:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 0, 2024-02-17T13:07:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 534 2024-02-17T13:07:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-02-17T13:07:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 1, 2024-02-17T13:07:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-02-17T13:07:26.014 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-02-17T13:07:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-02-17T13:07:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "osd-slow-ops": { 2024-02-17T13:07:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 0 2024-02-17T13:07:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-02-17T13:07:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-02-17T13:07:26.015 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 6, 2024-02-17T13:07:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-02-17T13:07:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 6, 2024-02-17T13:07:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 14885, 2024-02-17T13:07:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 14885, 2024-02-17T13:07:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-02-17T13:07:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T13:07:26.016 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-02-17T13:07:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 387553, 2024-02-17T13:07:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 88666618, 2024-02-17T13:07:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 387553, 2024-02-17T13:07:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-02-17T13:07:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-02-17T13:07:26.017 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 387553, 2024-02-17T13:07:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 88666618, 2024-02-17T13:07:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-02-17T13:07:26.018 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-02-17T13:07:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-02-17T13:07:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-02-17T13:07:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 44627, 2024-02-17T13:07:26.019 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 242621108, 2024-02-17T13:07:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 44627, 2024-02-17T13:07:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 242621108, 2024-02-17T13:07:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:26.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-02-17T13:07:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-02-17T13:07:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-02-17T13:07:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-02-17T13:07:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:26.021 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 44627, 2024-02-17T13:07:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 44627, 2024-02-17T13:07:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 44627, 2024-02-17T13:07:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 44627, 2024-02-17T13:07:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:26.022 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-02-17T13:07:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-02-17T13:07:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 14885, 2024-02-17T13:07:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 1505137, 2024-02-17T13:07:26.023 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 14885, 2024-02-17T13:07:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-02-17T13:07:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-02-17T13:07:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 840, 2024-02-17T13:07:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1505137, 2024-02-17T13:07:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:26.024 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x557cd48a46e0": { 2024-02-17T13:07:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 574, 2024-02-17T13:07:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-02-17T13:07:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 88053, 2024-02-17T13:07:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 237878825, 2024-02-17T13:07:26.025 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 88053, 2024-02-17T13:07:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-02-17T13:07:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-02-17T13:07:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 26377, 2024-02-17T13:07:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 237878251, 2024-02-17T13:07:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:26.026 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:26.027 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-02-17T13:07:26.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:26 smithi018 ceph-mon[32625]: pgmap v553: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 6.3 MiB/s wr, 214 op/s 2024-02-17T13:07:26.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:26 smithi119 ceph-mon[36188]: pgmap v553: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 6.3 MiB/s wr, 214 op/s 2024-02-17T13:07:26.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:26 smithi146 ceph-mon[36454]: pgmap v553: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 6.3 MiB/s wr, 214 op/s 2024-02-17T13:07:26.909 INFO:tasks.check_counter:Daemon mds.e mds.exported=1 2024-02-17T13:07:26.909 INFO:tasks.check_counter:Daemon mds.e mds.imported=17 2024-02-17T13:07:26.909 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=7 2024-02-17T13:07:26.910 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-02-17T13:07:26.910 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T13:07:26.910 DEBUG:tasks.check_counter:Getting stats from h 2024-02-17T13:07:26.910 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph daemon mds.h perf dump 2024-02-17T13:07:27.149 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T13:07:28.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:28 smithi119 ceph-mon[36188]: pgmap v554: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 5.3 MiB/s wr, 173 op/s 2024-02-17T13:07:28.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:28 smithi018 ceph-mon[32625]: pgmap v554: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 5.3 MiB/s wr, 173 op/s 2024-02-17T13:07:28.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:28 smithi146 ceph-mon[36454]: pgmap v554: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 5.3 MiB/s wr, 173 op/s 2024-02-17T13:07:29.636 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-02-17T13:07:29.636 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T13:07:29.637 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 44627, 2024-02-17T13:07:29.637 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 42110, 2024-02-17T13:07:29.637 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 13685721, 2024-02-17T13:07:29.637 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 109372244, 2024-02-17T13:07:29.637 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 64, 2024-02-17T13:07:29.637 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 8, 2024-02-17T13:07:29.637 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 5.193431313, 2024-02-17T13:07:29.638 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 1.865335407, 2024-02-17T13:07:29.638 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 1.926800483, 2024-02-17T13:07:29.638 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 1.597871421, 2024-02-17T13:07:29.638 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:29.638 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 42106, 2024-02-17T13:07:29.638 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 5.935242448, 2024-02-17T13:07:29.638 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000140959 2024-02-17T13:07:29.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.639 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:29.639 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 52996, 2024-02-17T13:07:29.639 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.011577188, 2024-02-17T13:07:29.639 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000218 2024-02-17T13:07:29.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.639 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 2030480, 2024-02-17T13:07:29.639 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 741216 2024-02-17T13:07:29.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.640 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T13:07:29.640 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 136944, 2024-02-17T13:07:29.640 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 125742, 2024-02-17T13:07:29.640 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 44187470, 2024-02-17T13:07:29.640 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 97796740, 2024-02-17T13:07:29.640 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 10, 2024-02-17T13:07:29.641 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 5, 2024-02-17T13:07:29.641 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 10.215227849, 2024-02-17T13:07:29.641 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 4.831937401, 2024-02-17T13:07:29.641 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 5.684117392, 2024-02-17T13:07:29.641 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.670356477, 2024-02-17T13:07:29.641 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:29.641 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 125745, 2024-02-17T13:07:29.642 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 9.425479086, 2024-02-17T13:07:29.642 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000074957 2024-02-17T13:07:29.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.642 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:29.642 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 232108, 2024-02-17T13:07:29.643 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.043141771, 2024-02-17T13:07:29.643 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000185 2024-02-17T13:07:29.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.643 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 5520, 2024-02-17T13:07:29.644 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 140864 2024-02-17T13:07:29.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.644 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T13:07:29.644 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 238995, 2024-02-17T13:07:29.644 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 352966, 2024-02-17T13:07:29.644 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 75045415, 2024-02-17T13:07:29.644 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 420840816, 2024-02-17T13:07:29.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 16, 2024-02-17T13:07:29.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 8, 2024-02-17T13:07:29.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 17.859371714, 2024-02-17T13:07:29.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 7.230662684, 2024-02-17T13:07:29.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 10.596893390, 2024-02-17T13:07:29.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 1.667530512, 2024-02-17T13:07:29.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:29.646 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 352962, 2024-02-17T13:07:29.646 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 42.630429722, 2024-02-17T13:07:29.646 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000120779 2024-02-17T13:07:29.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.646 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:29.646 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 475656, 2024-02-17T13:07:29.646 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.113901586, 2024-02-17T13:07:29.647 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000239 2024-02-17T13:07:29.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.647 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T13:07:29.647 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T13:07:29.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.647 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-02-17T13:07:29.647 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-02-17T13:07:29.647 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-02-17T13:07:29.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.648 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-02-17T13:07:29.648 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-02-17T13:07:29.648 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-02-17T13:07:29.648 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 51827, 2024-02-17T13:07:29.648 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 72.227206592, 2024-02-17T13:07:29.648 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001393621 2024-02-17T13:07:29.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:29.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.649 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-02-17T13:07:29.649 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-02-17T13:07:29.649 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-02-17T13:07:29.649 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 35162, 2024-02-17T13:07:29.649 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 7.202193860, 2024-02-17T13:07:29.650 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000204828 2024-02-17T13:07:29.650 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:29.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.650 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-02-17T13:07:29.650 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 132440, 2024-02-17T13:07:29.650 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 122837, 2024-02-17T13:07:29.650 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-02-17T13:07:29.651 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 122837, 2024-02-17T13:07:29.651 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 579.388329902, 2024-02-17T13:07:29.651 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004716724 2024-02-17T13:07:29.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.651 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-02-17T13:07:29.651 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 9603, 2024-02-17T13:07:29.651 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 16, 2024-02-17T13:07:29.651 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-02-17T13:07:29.652 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 92, 2024-02-17T13:07:29.652 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 9, 2024-02-17T13:07:29.652 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 9, 2024-02-17T13:07:29.652 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 2280, 2024-02-17T13:07:29.652 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 443, 2024-02-17T13:07:29.653 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 1064, 2024-02-17T13:07:29.653 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 773, 2024-02-17T13:07:29.653 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 1647, 2024-02-17T13:07:29.653 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 1062, 2024-02-17T13:07:29.653 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 351, 2024-02-17T13:07:29.653 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 351, 2024-02-17T13:07:29.653 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 33, 2024-02-17T13:07:29.653 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 213362, 2024-02-17T13:07:29.653 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 112063, 2024-02-17T13:07:29.654 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 9591, 2024-02-17T13:07:29.654 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 157, 2024-02-17T13:07:29.654 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 9, 2024-02-17T13:07:29.654 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-02-17T13:07:29.654 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 1669, 2024-02-17T13:07:29.654 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 88312, 2024-02-17T13:07:29.654 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-02-17T13:07:29.654 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 12, 2024-02-17T13:07:29.654 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 546, 2024-02-17T13:07:29.655 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 9, 2024-02-17T13:07:29.655 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 874, 2024-02-17T13:07:29.655 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-02-17T13:07:29.655 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 503, 2024-02-17T13:07:29.655 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-02-17T13:07:29.655 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 735, 2024-02-17T13:07:29.655 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 211953355, 2024-02-17T13:07:29.655 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 7, 2024-02-17T13:07:29.655 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 503, 2024-02-17T13:07:29.656 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-02-17T13:07:29.656 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T13:07:29.656 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-02-17T13:07:29.656 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 40, 2024-02-17T13:07:29.656 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T13:07:29.656 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 40, 2024-02-17T13:07:29.656 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 463, 2024-02-17T13:07:29.656 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 13, 2024-02-17T13:07:29.657 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 25612, 2024-02-17T13:07:29.657 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 85346, 2024-02-17T13:07:29.657 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T13:07:29.657 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 2651, 2024-02-17T13:07:29.657 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T13:07:29.657 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 45754, 2024-02-17T13:07:29.657 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 14529, 2024-02-17T13:07:29.657 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 4974, 2024-02-17T13:07:29.658 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 93617 2024-02-17T13:07:29.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.658 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-02-17T13:07:29.658 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 112, 2024-02-17T13:07:29.658 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 88, 2024-02-17T13:07:29.658 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 63, 2024-02-17T13:07:29.658 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 193, 2024-02-17T13:07:29.658 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 324, 2024-02-17T13:07:29.659 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 222, 2024-02-17T13:07:29.659 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 16, 2024-02-17T13:07:29.659 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-02-17T13:07:29.659 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 20011, 2024-02-17T13:07:29.659 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 19789, 2024-02-17T13:07:29.659 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-02-17T13:07:29.659 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-02-17T13:07:29.659 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-02-17T13:07:29.659 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-02-17T13:07:29.660 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-02-17T13:07:29.660 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-02-17T13:07:29.660 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-02-17T13:07:29.660 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T13:07:29.660 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 17, 2024-02-17T13:07:29.660 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-02-17T13:07:29.660 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 18, 2024-02-17T13:07:29.660 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-02-17T13:07:29.661 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0 2024-02-17T13:07:29.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-02-17T13:07:29.661 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-02-17T13:07:29.661 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 95703, 2024-02-17T13:07:29.661 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2024-02-17T13:07:29.661 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2024-02-17T13:07:29.661 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 95703, 2024-02-17T13:07:29.661 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-02-17T13:07:29.662 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2024-02-17T13:07:29.662 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 127, 2024-02-17T13:07:29.662 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2024-02-17T13:07:29.662 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2024-02-17T13:07:29.662 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 127, 2024-02-17T13:07:29.662 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 9, 2024-02-17T13:07:29.662 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-02-17T13:07:29.662 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2024-02-17T13:07:29.662 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 4194304, 2024-02-17T13:07:29.663 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 346182977, 2024-02-17T13:07:29.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 0, 2024-02-17T13:07:29.663 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-02-17T13:07:29.663 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 28497, 2024-02-17T13:07:29.663 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 185.757778124, 2024-02-17T13:07:29.663 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006518502 2024-02-17T13:07:29.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.663 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 0 2024-02-17T13:07:29.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.664 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-02-17T13:07:29.664 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 1175, 2024-02-17T13:07:29.664 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 21597, 2024-02-17T13:07:29.664 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 20422, 2024-02-17T13:07:29.664 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 192, 2024-02-17T13:07:29.664 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 343, 2024-02-17T13:07:29.664 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 151, 2024-02-17T13:07:29.664 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 2265, 2024-02-17T13:07:29.665 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 37760, 2024-02-17T13:07:29.665 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 35495, 2024-02-17T13:07:29.665 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 330, 2024-02-17T13:07:29.665 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 22038, 2024-02-17T13:07:29.665 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 21708, 2024-02-17T13:07:29.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 78524, 2024-02-17T13:07:29.665 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 207228 2024-02-17T13:07:29.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-02-17T13:07:29.666 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 191209, 2024-02-17T13:07:29.666 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 3825, 2024-02-17T13:07:29.666 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 132440, 2024-02-17T13:07:29.666 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 1963, 2024-02-17T13:07:29.666 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 13299, 2024-02-17T13:07:29.666 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-02-17T13:07:29.666 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 20108, 2024-02-17T13:07:29.666 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 80.909448130, 2024-02-17T13:07:29.667 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004023744 2024-02-17T13:07:29.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.667 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-02-17T13:07:29.667 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4863, 2024-02-17T13:07:29.667 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 81.293011606, 2024-02-17T13:07:29.667 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.016716638 2024-02-17T13:07:29.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.667 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-02-17T13:07:29.668 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.668 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.668 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.668 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-02-17T13:07:29.668 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.668 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.668 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.669 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-02-17T13:07:29.669 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 33903, 2024-02-17T13:07:29.669 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 139.579133806, 2024-02-17T13:07:29.669 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004117014 2024-02-17T13:07:29.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.669 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-02-17T13:07:29.669 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.669 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.669 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.670 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-02-17T13:07:29.670 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.670 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.670 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.670 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-02-17T13:07:29.670 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.670 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.671 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.671 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-02-17T13:07:29.671 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.671 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.671 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.671 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-02-17T13:07:29.671 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.671 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.672 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.672 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-02-17T13:07:29.673 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.673 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.673 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.673 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-02-17T13:07:29.673 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.673 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.673 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.674 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-02-17T13:07:29.674 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.674 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.674 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.674 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-02-17T13:07:29.674 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.674 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-02-17T13:07:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-02-17T13:07:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 40469, 2024-02-17T13:07:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 188.659971414, 2024-02-17T13:07:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004661839 2024-02-17T13:07:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-02-17T13:07:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2996, 2024-02-17T13:07:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 16.963651743, 2024-02-17T13:07:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.005662100 2024-02-17T13:07:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-02-17T13:07:29.677 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.677 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.677 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.677 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-02-17T13:07:29.677 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.677 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.677 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.678 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-02-17T13:07:29.678 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.678 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.678 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.678 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-02-17T13:07:29.678 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.678 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.678 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.679 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-02-17T13:07:29.679 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.679 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.679 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.679 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-02-17T13:07:29.679 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 83, 2024-02-17T13:07:29.679 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.905331180, 2024-02-17T13:07:29.679 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.010907604 2024-02-17T13:07:29.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.680 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-02-17T13:07:29.680 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.680 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.680 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.680 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-02-17T13:07:29.680 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 404, 2024-02-17T13:07:29.680 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1.390352596, 2024-02-17T13:07:29.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.003441466 2024-02-17T13:07:29.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.681 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-02-17T13:07:29.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.681 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.681 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-02-17T13:07:29.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.682 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.682 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.682 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-02-17T13:07:29.682 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.682 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.682 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.682 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-02-17T13:07:29.683 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 20011, 2024-02-17T13:07:29.683 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 69.687429427, 2024-02-17T13:07:29.683 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.003482456 2024-02-17T13:07:29.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.683 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-02-17T13:07:29.683 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-02-17T13:07:29.683 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-02-17T13:07:29.683 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.683 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.684 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.684 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:29.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.684 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-02-17T13:07:29.684 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 2, 2024-02-17T13:07:29.684 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 3, 2024-02-17T13:07:29.684 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 1, 2024-02-17T13:07:29.684 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 2, 2024-02-17T13:07:29.684 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-02-17T13:07:29.685 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 25867, 2024-02-17T13:07:29.685 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 12933, 2024-02-17T13:07:29.685 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 1023, 2024-02-17T13:07:29.685 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-02-17T13:07:29.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.685 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-02-17T13:07:29.685 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-02-17T13:07:29.685 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-02-17T13:07:29.685 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T13:07:29.686 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-02-17T13:07:29.686 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T13:07:29.686 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-02-17T13:07:29.686 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T13:07:29.686 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T13:07:29.686 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T13:07:29.686 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T13:07:29.686 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T13:07:29.686 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-02-17T13:07:29.687 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T13:07:29.687 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T13:07:29.687 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-02-17T13:07:29.687 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-02-17T13:07:29.687 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-02-17T13:07:29.687 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-02-17T13:07:29.687 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T13:07:29.687 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T13:07:29.688 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T13:07:29.688 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T13:07:29.688 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T13:07:29.688 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-02-17T13:07:29.688 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-02-17T13:07:29.688 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-02-17T13:07:29.688 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T13:07:29.688 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T13:07:29.689 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-02-17T13:07:29.689 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-02-17T13:07:29.689 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-02-17T13:07:29.689 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-02-17T13:07:29.689 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T13:07:29.689 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-02-17T13:07:29.689 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T13:07:29.689 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-02-17T13:07:29.690 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 665549, 2024-02-17T13:07:29.690 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 1024, 2024-02-17T13:07:29.690 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-02-17T13:07:29.690 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-02-17T13:07:29.690 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-02-17T13:07:29.690 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-02-17T13:07:29.690 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-02-17T13:07:29.690 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-02-17T13:07:29.690 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-02-17T13:07:29.691 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-02-17T13:07:29.691 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 7936, 2024-02-17T13:07:29.691 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 165, 2024-02-17T13:07:29.691 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T13:07:29.691 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-02-17T13:07:29.691 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-02-17T13:07:29.691 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-02-17T13:07:29.691 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 5100810, 2024-02-17T13:07:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 77412, 2024-02-17T13:07:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-02-17T13:07:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-02-17T13:07:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-02-17T13:07:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-02-17T13:07:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-02-17T13:07:29.692 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-02-17T13:07:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-02-17T13:07:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 52297, 2024-02-17T13:07:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 346284682, 2024-02-17T13:07:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 27, 2024-02-17T13:07:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 52297, 2024-02-17T13:07:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-02-17T13:07:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 52297, 2024-02-17T13:07:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 274.794144928, 2024-02-17T13:07:29.693 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.005254491 2024-02-17T13:07:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-02-17T13:07:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-02-17T13:07:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 52297, 2024-02-17T13:07:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 53642 2024-02-17T13:07:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 52297, 2024-02-17T13:07:29.694 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 1516, 2024-02-17T13:07:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 50781, 2024-02-17T13:07:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-02-17T13:07:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-02-17T13:07:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 0, 2024-02-17T13:07:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 10, 2024-02-17T13:07:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 997, 2024-02-17T13:07:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 29494, 2024-02-17T13:07:29.695 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 523, 2024-02-17T13:07:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-02-17T13:07:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-02-17T13:07:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-02-17T13:07:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 9, 2024-02-17T13:07:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 19910, 2024-02-17T13:07:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-02-17T13:07:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-02-17T13:07:29.696 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-02-17T13:07:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 517, 2024-02-17T13:07:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 20, 2024-02-17T13:07:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-02-17T13:07:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-02-17T13:07:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-02-17T13:07:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-02-17T13:07:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-02-17T13:07:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-02-17T13:07:29.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T13:07:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-02-17T13:07:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-02-17T13:07:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-02-17T13:07:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-02-17T13:07:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-02-17T13:07:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-02-17T13:07:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-02-17T13:07:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-02-17T13:07:29.698 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-02-17T13:07:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-02-17T13:07:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-02-17T13:07:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-02-17T13:07:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-02-17T13:07:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-02-17T13:07:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-02-17T13:07:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-02-17T13:07:29.699 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-02-17T13:07:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-02-17T13:07:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-02-17T13:07:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 113, 2024-02-17T13:07:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-02-17T13:07:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 27, 2024-02-17T13:07:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 12, 2024-02-17T13:07:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 12, 2024-02-17T13:07:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2024-02-17T13:07:29.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-02-17T13:07:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 1585, 2024-02-17T13:07:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 32, 2024-02-17T13:07:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 545 2024-02-17T13:07:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-02-17T13:07:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 1, 2024-02-17T13:07:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-02-17T13:07:29.701 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-02-17T13:07:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-02-17T13:07:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "osd-slow-ops": { 2024-02-17T13:07:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 1 2024-02-17T13:07:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-02-17T13:07:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-02-17T13:07:29.702 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 6, 2024-02-17T13:07:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-02-17T13:07:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 6, 2024-02-17T13:07:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 19789, 2024-02-17T13:07:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 19789, 2024-02-17T13:07:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-02-17T13:07:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T13:07:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:29.703 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-02-17T13:07:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 420562, 2024-02-17T13:07:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 95039724, 2024-02-17T13:07:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 420562, 2024-02-17T13:07:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-02-17T13:07:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-02-17T13:07:29.704 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 420562, 2024-02-17T13:07:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 95039724, 2024-02-17T13:07:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.705 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-02-17T13:07:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-02-17T13:07:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-02-17T13:07:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-02-17T13:07:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 52297, 2024-02-17T13:07:29.706 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 348284771, 2024-02-17T13:07:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 52297, 2024-02-17T13:07:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 348284771, 2024-02-17T13:07:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.707 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-02-17T13:07:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-02-17T13:07:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-02-17T13:07:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-02-17T13:07:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:29.708 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 52297, 2024-02-17T13:07:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 52297, 2024-02-17T13:07:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 52297, 2024-02-17T13:07:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 52297, 2024-02-17T13:07:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:29.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-02-17T13:07:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-02-17T13:07:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 19789, 2024-02-17T13:07:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 2000345, 2024-02-17T13:07:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:29.710 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 19789, 2024-02-17T13:07:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-02-17T13:07:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-02-17T13:07:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 997, 2024-02-17T13:07:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 2000345, 2024-02-17T13:07:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x560e3d69a6e0": { 2024-02-17T13:07:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 643367, 2024-02-17T13:07:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-02-17T13:07:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 95703, 2024-02-17T13:07:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 341988673, 2024-02-17T13:07:29.712 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 95703, 2024-02-17T13:07:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-02-17T13:07:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-02-17T13:07:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 28497, 2024-02-17T13:07:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 341345306, 2024-02-17T13:07:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:29.713 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:29.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:29.714 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-02-17T13:07:30.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:30 smithi018 ceph-mon[32625]: pgmap v555: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 5.1 MiB/s wr, 169 op/s 2024-02-17T13:07:30.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:30 smithi119 ceph-mon[36188]: pgmap v555: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 5.1 MiB/s wr, 169 op/s 2024-02-17T13:07:30.584 INFO:tasks.check_counter:Daemon mds.h mds.exported=12 2024-02-17T13:07:30.584 INFO:tasks.check_counter:Daemon mds.h mds.imported=9 2024-02-17T13:07:30.584 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-02-17T13:07:30.584 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-02-17T13:07:30.584 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T13:07:30.585 DEBUG:tasks.check_counter:Getting stats from k 2024-02-17T13:07:30.585 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph daemon mds.k perf dump 2024-02-17T13:07:30.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:30 smithi146 ceph-mon[36454]: pgmap v555: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 5.1 MiB/s wr, 169 op/s 2024-02-17T13:07:30.822 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T13:07:32.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:32 smithi119 ceph-mon[36188]: pgmap v556: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 6.2 MiB/s wr, 229 op/s 2024-02-17T13:07:32.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:32 smithi018 ceph-mon[32625]: pgmap v556: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 6.2 MiB/s wr, 229 op/s 2024-02-17T13:07:32.641 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-02-17T13:07:32.642 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T13:07:32.642 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 117555, 2024-02-17T13:07:32.642 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 93462, 2024-02-17T13:07:32.642 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 41117939, 2024-02-17T13:07:32.642 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 107770381, 2024-02-17T13:07:32.642 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 11, 2024-02-17T13:07:32.642 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 4, 2024-02-17T13:07:32.643 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 8.106289366, 2024-02-17T13:07:32.643 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 3.537528535, 2024-02-17T13:07:32.643 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 4.849756365, 2024-02-17T13:07:32.643 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.588174611, 2024-02-17T13:07:32.643 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:32.643 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 93460, 2024-02-17T13:07:32.643 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 9.032104782, 2024-02-17T13:07:32.643 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000096641 2024-02-17T13:07:32.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.644 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:32.644 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 182470, 2024-02-17T13:07:32.644 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.032255216, 2024-02-17T13:07:32.644 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000176 2024-02-17T13:07:32.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.644 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T13:07:32.644 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T13:07:32.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.645 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T13:07:32.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 143468, 2024-02-17T13:07:32.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 204846, 2024-02-17T13:07:32.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 44004410, 2024-02-17T13:07:32.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 208593714, 2024-02-17T13:07:32.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 27, 2024-02-17T13:07:32.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 5, 2024-02-17T13:07:32.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 11.145374869, 2024-02-17T13:07:32.645 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 4.706628809, 2024-02-17T13:07:32.646 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 7.440554549, 2024-02-17T13:07:32.646 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.357788542, 2024-02-17T13:07:32.646 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:32.646 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 204843, 2024-02-17T13:07:32.646 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 28.968926230, 2024-02-17T13:07:32.646 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000141420 2024-02-17T13:07:32.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.646 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:32.647 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 289484, 2024-02-17T13:07:32.647 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.069135470, 2024-02-17T13:07:32.647 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000238 2024-02-17T13:07:32.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.647 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-02-17T13:07:32.647 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 133728 2024-02-17T13:07:32.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.647 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T13:07:32.647 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 28322, 2024-02-17T13:07:32.648 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 31130, 2024-02-17T13:07:32.648 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 9176991, 2024-02-17T13:07:32.648 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 94735639, 2024-02-17T13:07:32.648 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 52, 2024-02-17T13:07:32.648 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 5, 2024-02-17T13:07:32.648 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 3.295673610, 2024-02-17T13:07:32.648 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 1.290216970, 2024-02-17T13:07:32.648 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 1.210885348, 2024-02-17T13:07:32.649 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.792169754, 2024-02-17T13:07:32.649 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:32.649 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 31130, 2024-02-17T13:07:32.649 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 4.365006802, 2024-02-17T13:07:32.649 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000140218 2024-02-17T13:07:32.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.649 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:32.649 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 37425, 2024-02-17T13:07:32.649 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.012052182, 2024-02-17T13:07:32.650 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000322 2024-02-17T13:07:32.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.650 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 2014016, 2024-02-17T13:07:32.650 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 694912 2024-02-17T13:07:32.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.650 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-02-17T13:07:32.650 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-02-17T13:07:32.650 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-02-17T13:07:32.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.651 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-02-17T13:07:32.651 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-02-17T13:07:32.651 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-02-17T13:07:32.651 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 34441, 2024-02-17T13:07:32.651 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 42.990025238, 2024-02-17T13:07:32.651 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001248222 2024-02-17T13:07:32.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:32.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.652 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-02-17T13:07:32.652 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-02-17T13:07:32.652 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-02-17T13:07:32.652 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 19460, 2024-02-17T13:07:32.652 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 2.952043172, 2024-02-17T13:07:32.652 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000151698 2024-02-17T13:07:32.652 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:32.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.653 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-02-17T13:07:32.653 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 75683, 2024-02-17T13:07:32.653 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 74819, 2024-02-17T13:07:32.653 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-02-17T13:07:32.653 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 74819, 2024-02-17T13:07:32.653 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 315.467337502, 2024-02-17T13:07:32.653 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004216406 2024-02-17T13:07:32.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.654 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-02-17T13:07:32.654 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 864, 2024-02-17T13:07:32.654 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 8, 2024-02-17T13:07:32.654 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-02-17T13:07:32.654 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 96, 2024-02-17T13:07:32.654 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 10, 2024-02-17T13:07:32.654 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 5, 2024-02-17T13:07:32.654 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 1944, 2024-02-17T13:07:32.655 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 567, 2024-02-17T13:07:32.655 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 1108, 2024-02-17T13:07:32.655 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 269, 2024-02-17T13:07:32.655 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 1124, 2024-02-17T13:07:32.655 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 311, 2024-02-17T13:07:32.655 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 383, 2024-02-17T13:07:32.655 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 383, 2024-02-17T13:07:32.655 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 15, 2024-02-17T13:07:32.656 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 120293, 2024-02-17T13:07:32.656 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 72355, 2024-02-17T13:07:32.656 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 859, 2024-02-17T13:07:32.656 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 42, 2024-02-17T13:07:32.656 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 5, 2024-02-17T13:07:32.656 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-02-17T13:07:32.657 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 1164, 2024-02-17T13:07:32.657 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 16754, 2024-02-17T13:07:32.657 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-02-17T13:07:32.657 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 6, 2024-02-17T13:07:32.657 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 241, 2024-02-17T13:07:32.657 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 6, 2024-02-17T13:07:32.657 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 517, 2024-02-17T13:07:32.657 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-02-17T13:07:32.657 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 340, 2024-02-17T13:07:32.658 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 1, 2024-02-17T13:07:32.658 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 735, 2024-02-17T13:07:32.658 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 211953355, 2024-02-17T13:07:32.658 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 7, 2024-02-17T13:07:32.658 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 340, 2024-02-17T13:07:32.658 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-02-17T13:07:32.658 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T13:07:32.658 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-02-17T13:07:32.659 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 0, 2024-02-17T13:07:32.659 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T13:07:32.659 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T13:07:32.659 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 340, 2024-02-17T13:07:32.659 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 4, 2024-02-17T13:07:32.659 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 15498, 2024-02-17T13:07:32.659 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 48002, 2024-02-17T13:07:32.659 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T13:07:32.659 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 2106, 2024-02-17T13:07:32.660 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T13:07:32.660 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 32138, 2024-02-17T13:07:32.660 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 11635, 2024-02-17T13:07:32.660 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 4064, 2024-02-17T13:07:32.660 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 54109 2024-02-17T13:07:32.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.660 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-02-17T13:07:32.660 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 18, 2024-02-17T13:07:32.661 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 181, 2024-02-17T13:07:32.661 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 109, 2024-02-17T13:07:32.661 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 176, 2024-02-17T13:07:32.661 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 71, 2024-02-17T13:07:32.661 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 138, 2024-02-17T13:07:32.661 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-02-17T13:07:32.661 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-02-17T13:07:32.661 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 10936, 2024-02-17T13:07:32.662 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 10798, 2024-02-17T13:07:32.662 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-02-17T13:07:32.662 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-02-17T13:07:32.662 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-02-17T13:07:32.662 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-02-17T13:07:32.662 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-02-17T13:07:32.662 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-02-17T13:07:32.662 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-02-17T13:07:32.662 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T13:07:32.663 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 11, 2024-02-17T13:07:32.663 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-02-17T13:07:32.663 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 15, 2024-02-17T13:07:32.663 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-02-17T13:07:32.663 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0 2024-02-17T13:07:32.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.663 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-02-17T13:07:32.663 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-02-17T13:07:32.664 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 59450, 2024-02-17T13:07:32.664 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2024-02-17T13:07:32.664 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2024-02-17T13:07:32.664 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 59450, 2024-02-17T13:07:32.664 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-02-17T13:07:32.664 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2024-02-17T13:07:32.664 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 82, 2024-02-17T13:07:32.664 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2024-02-17T13:07:32.664 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2024-02-17T13:07:32.665 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 82, 2024-02-17T13:07:32.665 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 6, 2024-02-17T13:07:32.665 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-02-17T13:07:32.665 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2024-02-17T13:07:32.665 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 4194304, 2024-02-17T13:07:32.665 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 218559565, 2024-02-17T13:07:32.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 0, 2024-02-17T13:07:32.665 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-02-17T13:07:32.666 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 18551, 2024-02-17T13:07:32.666 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 121.244295488, 2024-02-17T13:07:32.666 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006535728 2024-02-17T13:07:32.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.666 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 0 2024-02-17T13:07:32.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-02-17T13:07:32.666 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 889, 2024-02-17T13:07:32.666 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 11920, 2024-02-17T13:07:32.667 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 11031, 2024-02-17T13:07:32.667 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 150, 2024-02-17T13:07:32.667 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 327, 2024-02-17T13:07:32.667 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 177, 2024-02-17T13:07:32.667 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 1869, 2024-02-17T13:07:32.667 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 21390, 2024-02-17T13:07:32.667 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 19521, 2024-02-17T13:07:32.667 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 270, 2024-02-17T13:07:32.668 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 12550, 2024-02-17T13:07:32.668 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 12280, 2024-02-17T13:07:32.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 73000, 2024-02-17T13:07:32.668 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 207228 2024-02-17T13:07:32.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.668 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-02-17T13:07:32.668 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 107478, 2024-02-17T13:07:32.668 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 1377, 2024-02-17T13:07:32.669 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 75683, 2024-02-17T13:07:32.669 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 1693, 2024-02-17T13:07:32.669 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 5591, 2024-02-17T13:07:32.669 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-02-17T13:07:32.669 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 10910, 2024-02-17T13:07:32.669 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 41.208046117, 2024-02-17T13:07:32.670 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.003777089 2024-02-17T13:07:32.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.670 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-02-17T13:07:32.670 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2633, 2024-02-17T13:07:32.670 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 43.973060895, 2024-02-17T13:07:32.670 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.016700744 2024-02-17T13:07:32.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.670 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-02-17T13:07:32.670 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.671 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.671 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.671 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-02-17T13:07:32.671 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.671 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.671 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.672 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-02-17T13:07:32.672 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 21148, 2024-02-17T13:07:32.672 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 45.500074254, 2024-02-17T13:07:32.672 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.002151507 2024-02-17T13:07:32.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.672 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-02-17T13:07:32.672 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.672 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.673 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.673 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-02-17T13:07:32.673 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.673 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.673 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.673 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-02-17T13:07:32.673 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.674 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.674 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.674 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-02-17T13:07:32.674 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.674 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.674 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.674 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-02-17T13:07:32.675 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.675 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.675 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.676 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-02-17T13:07:32.676 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.676 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.676 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.676 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-02-17T13:07:32.676 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.677 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.677 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.677 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-02-17T13:07:32.677 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.677 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.677 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.678 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-02-17T13:07:32.678 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.678 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.678 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.678 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-02-17T13:07:32.678 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.678 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.678 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.679 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-02-17T13:07:32.679 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 26472, 2024-02-17T13:07:32.679 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 122.898948854, 2024-02-17T13:07:32.679 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004642601 2024-02-17T13:07:32.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.679 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-02-17T13:07:32.679 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2390, 2024-02-17T13:07:32.679 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 10.012063148, 2024-02-17T13:07:32.680 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004189147 2024-02-17T13:07:32.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.680 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-02-17T13:07:32.680 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.680 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.680 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.680 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-02-17T13:07:32.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.681 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.681 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-02-17T13:07:32.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.681 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.682 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-02-17T13:07:32.682 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.682 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.682 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.682 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-02-17T13:07:32.682 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.682 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.683 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.683 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-02-17T13:07:32.683 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 6, 2024-02-17T13:07:32.683 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.057014147, 2024-02-17T13:07:32.683 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.009502357 2024-02-17T13:07:32.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.683 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-02-17T13:07:32.683 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.684 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.684 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.684 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-02-17T13:07:32.684 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 324, 2024-02-17T13:07:32.684 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.672696166, 2024-02-17T13:07:32.684 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.002076222 2024-02-17T13:07:32.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.685 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-02-17T13:07:32.685 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.685 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.685 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.685 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-02-17T13:07:32.685 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.685 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.685 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.686 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-02-17T13:07:32.686 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.686 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.686 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.686 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-02-17T13:07:32.686 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 10936, 2024-02-17T13:07:32.686 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 51.145433921, 2024-02-17T13:07:32.687 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004676795 2024-02-17T13:07:32.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.687 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-02-17T13:07:32.687 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-02-17T13:07:32.687 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-02-17T13:07:32.687 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.687 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.687 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.688 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:32.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.688 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-02-17T13:07:32.688 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 1, 2024-02-17T13:07:32.688 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 3, 2024-02-17T13:07:32.688 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 2, 2024-02-17T13:07:32.688 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 1, 2024-02-17T13:07:32.688 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-02-17T13:07:32.688 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 21764, 2024-02-17T13:07:32.689 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 21764, 2024-02-17T13:07:32.689 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 1052, 2024-02-17T13:07:32.689 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-02-17T13:07:32.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.689 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-02-17T13:07:32.689 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-02-17T13:07:32.689 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-02-17T13:07:32.689 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T13:07:32.689 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-02-17T13:07:32.690 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T13:07:32.690 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-02-17T13:07:32.690 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T13:07:32.690 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T13:07:32.690 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T13:07:32.690 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T13:07:32.690 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T13:07:32.690 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-02-17T13:07:32.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T13:07:32.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T13:07:32.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-02-17T13:07:32.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-02-17T13:07:32.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-02-17T13:07:32.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-02-17T13:07:32.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T13:07:32.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T13:07:32.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T13:07:32.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T13:07:32.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T13:07:32.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-02-17T13:07:32.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-02-17T13:07:32.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-02-17T13:07:32.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T13:07:32.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T13:07:32.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-02-17T13:07:32.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-02-17T13:07:32.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-02-17T13:07:32.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-02-17T13:07:32.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T13:07:32.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-02-17T13:07:32.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T13:07:32.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-02-17T13:07:32.693 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 2752426, 2024-02-17T13:07:32.693 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 1348, 2024-02-17T13:07:32.694 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-02-17T13:07:32.694 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-02-17T13:07:32.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-02-17T13:07:32.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-02-17T13:07:32.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-02-17T13:07:32.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-02-17T13:07:32.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-02-17T13:07:32.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-02-17T13:07:32.695 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 7936, 2024-02-17T13:07:32.695 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 165, 2024-02-17T13:07:32.695 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T13:07:32.695 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-02-17T13:07:32.695 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-02-17T13:07:32.695 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-02-17T13:07:32.695 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 4487964, 2024-02-17T13:07:32.695 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 84633, 2024-02-17T13:07:32.696 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-02-17T13:07:32.696 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-02-17T13:07:32.696 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-02-17T13:07:32.696 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-02-17T13:07:32.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.696 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-02-17T13:07:32.696 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-02-17T13:07:32.696 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-02-17T13:07:32.697 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 32272, 2024-02-17T13:07:32.697 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 217058047, 2024-02-17T13:07:32.697 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 33, 2024-02-17T13:07:32.697 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 32272, 2024-02-17T13:07:32.697 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-02-17T13:07:32.697 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 32272, 2024-02-17T13:07:32.697 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 194.488155532, 2024-02-17T13:07:32.697 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006026529 2024-02-17T13:07:32.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.698 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-02-17T13:07:32.698 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-02-17T13:07:32.698 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 32272, 2024-02-17T13:07:32.698 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 33329 2024-02-17T13:07:32.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.698 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 32272, 2024-02-17T13:07:32.698 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 1016, 2024-02-17T13:07:32.698 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 31256, 2024-02-17T13:07:32.699 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-02-17T13:07:32.699 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-02-17T13:07:32.699 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 0, 2024-02-17T13:07:32.699 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 10, 2024-02-17T13:07:32.699 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 668, 2024-02-17T13:07:32.699 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 19219, 2024-02-17T13:07:32.699 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 406, 2024-02-17T13:07:32.699 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-02-17T13:07:32.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-02-17T13:07:32.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-02-17T13:07:32.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 10, 2024-02-17T13:07:32.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 10901, 2024-02-17T13:07:32.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-02-17T13:07:32.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-02-17T13:07:32.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-02-17T13:07:32.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 345, 2024-02-17T13:07:32.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 20, 2024-02-17T13:07:32.701 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-02-17T13:07:32.701 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-02-17T13:07:32.701 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-02-17T13:07:32.701 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-02-17T13:07:32.701 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-02-17T13:07:32.701 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-02-17T13:07:32.701 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T13:07:32.701 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-02-17T13:07:32.702 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-02-17T13:07:32.702 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-02-17T13:07:32.702 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-02-17T13:07:32.702 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-02-17T13:07:32.702 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-02-17T13:07:32.702 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-02-17T13:07:32.702 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-02-17T13:07:32.702 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-02-17T13:07:32.702 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-02-17T13:07:32.703 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-02-17T13:07:32.703 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-02-17T13:07:32.703 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-02-17T13:07:32.703 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-02-17T13:07:32.703 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-02-17T13:07:32.703 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-02-17T13:07:32.703 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-02-17T13:07:32.703 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-02-17T13:07:32.704 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-02-17T13:07:32.704 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 113, 2024-02-17T13:07:32.704 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-02-17T13:07:32.704 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 27, 2024-02-17T13:07:32.704 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 12, 2024-02-17T13:07:32.704 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 12, 2024-02-17T13:07:32.704 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2024-02-17T13:07:32.704 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-02-17T13:07:32.704 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 1335, 2024-02-17T13:07:32.705 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 16, 2024-02-17T13:07:32.705 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 399 2024-02-17T13:07:32.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.705 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-02-17T13:07:32.705 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 1, 2024-02-17T13:07:32.705 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-02-17T13:07:32.705 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-02-17T13:07:32.705 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-02-17T13:07:32.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.706 INFO:teuthology.orchestra.run.smithi119.stdout: "osd-slow-ops": { 2024-02-17T13:07:32.706 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 1 2024-02-17T13:07:32.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.706 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-02-17T13:07:32.706 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-02-17T13:07:32.706 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 6, 2024-02-17T13:07:32.706 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-02-17T13:07:32.706 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 6, 2024-02-17T13:07:32.707 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 10798, 2024-02-17T13:07:32.707 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 10798, 2024-02-17T13:07:32.707 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-02-17T13:07:32.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.707 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T13:07:32.707 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:32.707 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-02-17T13:07:32.707 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:32.708 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 289341, 2024-02-17T13:07:32.708 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 68230518, 2024-02-17T13:07:32.708 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:32.708 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 289341, 2024-02-17T13:07:32.708 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-02-17T13:07:32.708 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-02-17T13:07:32.708 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 289341, 2024-02-17T13:07:32.708 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 68230518, 2024-02-17T13:07:32.709 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:32.709 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.709 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.709 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:32.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.709 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-02-17T13:07:32.709 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:32.710 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-02-17T13:07:32.710 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:32.710 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-02-17T13:07:32.710 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-02-17T13:07:32.710 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:32.710 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:32.710 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 32272, 2024-02-17T13:07:32.710 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 218150773, 2024-02-17T13:07:32.711 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 32272, 2024-02-17T13:07:32.711 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 218150773, 2024-02-17T13:07:32.711 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:32.711 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.711 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.711 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:32.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.711 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-02-17T13:07:32.712 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:32.712 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-02-17T13:07:32.712 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:32.712 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-02-17T13:07:32.712 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-02-17T13:07:32.712 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:32.712 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:32.712 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 32272, 2024-02-17T13:07:32.713 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 32272, 2024-02-17T13:07:32.713 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 32272, 2024-02-17T13:07:32.713 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 32272, 2024-02-17T13:07:32.713 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:32.713 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.713 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.713 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.713 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:32.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.714 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-02-17T13:07:32.714 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-02-17T13:07:32.714 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-02-17T13:07:32.714 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:32.714 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 10798, 2024-02-17T13:07:32.714 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 1092854, 2024-02-17T13:07:32.714 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:32.714 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 10798, 2024-02-17T13:07:32.715 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-02-17T13:07:32.715 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-02-17T13:07:32.715 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 668, 2024-02-17T13:07:32.715 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1092854, 2024-02-17T13:07:32.715 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:32.715 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.715 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.715 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:32.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-02-17T13:07:32.716 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x5617bc1026e0": { 2024-02-17T13:07:32.716 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 455757, 2024-02-17T13:07:32.716 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-02-17T13:07:32.716 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-02-17T13:07:32.716 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 59450, 2024-02-17T13:07:32.716 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 214365261, 2024-02-17T13:07:32.716 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:32.717 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 59450, 2024-02-17T13:07:32.717 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-02-17T13:07:32.717 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-02-17T13:07:32.717 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 18551, 2024-02-17T13:07:32.717 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 213909504, 2024-02-17T13:07:32.717 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-02-17T13:07:32.717 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-02-17T13:07:32.717 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-02-17T13:07:32.718 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-02-17T13:07:32.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:32.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-02-17T13:07:32.718 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-02-17T13:07:32.718 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:32 smithi146 ceph-mon[36454]: pgmap v556: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 6.2 MiB/s wr, 229 op/s 2024-02-17T13:07:33.424 INFO:tasks.check_counter:Daemon mds.k mds.exported=6 2024-02-17T13:07:33.424 INFO:tasks.check_counter:Daemon mds.k mds.imported=6 2024-02-17T13:07:33.424 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=7 2024-02-17T13:07:33.424 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-02-17T13:07:33.424 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T13:07:33.424 DEBUG:tasks.check_counter:Getting stats from c 2024-02-17T13:07:33.425 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph daemon mds.c perf dump 2024-02-17T13:07:33.695 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T13:07:34.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:34 smithi119 ceph-mon[36188]: pgmap v557: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 4.4 MiB/s wr, 168 op/s 2024-02-17T13:07:34.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:34 smithi146 ceph-mon[36454]: pgmap v557: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 4.4 MiB/s wr, 168 op/s 2024-02-17T13:07:34.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:34 smithi018 ceph-mon[32625]: pgmap v557: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 4.4 MiB/s wr, 168 op/s 2024-02-17T13:07:35.411 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-02-17T13:07:35.412 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T13:07:35.412 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 2828, 2024-02-17T13:07:35.412 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 2829, 2024-02-17T13:07:35.412 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 133117970, 2024-02-17T13:07:35.412 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 879204, 2024-02-17T13:07:35.412 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 8, 2024-02-17T13:07:35.413 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 4, 2024-02-17T13:07:35.413 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.290044445, 2024-02-17T13:07:35.413 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.070744115, 2024-02-17T13:07:35.413 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.123079753, 2024-02-17T13:07:35.413 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.076514861, 2024-02-17T13:07:35.413 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:35.413 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2827, 2024-02-17T13:07:35.413 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.089324323, 2024-02-17T13:07:35.413 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000031596 2024-02-17T13:07:35.414 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.414 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:35.414 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.414 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.414 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.414 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.414 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T13:07:35.414 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T13:07:35.415 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.415 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T13:07:35.415 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 1358, 2024-02-17T13:07:35.415 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1282, 2024-02-17T13:07:35.415 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 102917684, 2024-02-17T13:07:35.415 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 488176, 2024-02-17T13:07:35.415 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 6, 2024-02-17T13:07:35.415 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 5, 2024-02-17T13:07:35.415 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.177834909, 2024-02-17T13:07:35.416 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.035204107, 2024-02-17T13:07:35.416 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.116664131, 2024-02-17T13:07:35.416 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.030370588, 2024-02-17T13:07:35.416 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:35.416 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1282, 2024-02-17T13:07:35.416 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.049247915, 2024-02-17T13:07:35.416 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000038414 2024-02-17T13:07:35.416 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.417 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:35.417 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.417 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.417 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.417 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.417 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 2019536, 2024-02-17T13:07:35.417 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 200128 2024-02-17T13:07:35.417 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.417 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T13:07:35.418 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 703, 2024-02-17T13:07:35.418 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 947, 2024-02-17T13:07:35.418 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 109230852, 2024-02-17T13:07:35.418 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 811819, 2024-02-17T13:07:35.418 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 9, 2024-02-17T13:07:35.418 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 6, 2024-02-17T13:07:35.418 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.124388388, 2024-02-17T13:07:35.418 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.022175663, 2024-02-17T13:07:35.419 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.077998510, 2024-02-17T13:07:35.419 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.017089574, 2024-02-17T13:07:35.419 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:35.419 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 944, 2024-02-17T13:07:35.419 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.038394691, 2024-02-17T13:07:35.419 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000040672 2024-02-17T13:07:35.419 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.419 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:35.419 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.420 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.420 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.420 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.420 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-17T13:07:35.420 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 593808 2024-02-17T13:07:35.420 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.420 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-02-17T13:07:35.420 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-02-17T13:07:35.420 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-02-17T13:07:35.421 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.421 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-02-17T13:07:35.421 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-02-17T13:07:35.421 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-02-17T13:07:35.421 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 7391, 2024-02-17T13:07:35.421 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.172108728, 2024-02-17T13:07:35.421 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000023286 2024-02-17T13:07:35.421 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:35.421 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.422 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-02-17T13:07:35.422 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-02-17T13:07:35.422 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-02-17T13:07:35.422 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.422 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.422 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.422 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:35.422 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.422 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-02-17T13:07:35.423 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 0, 2024-02-17T13:07:35.423 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 0, 2024-02-17T13:07:35.423 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-02-17T13:07:35.423 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.423 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.423 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.423 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.423 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-02-17T13:07:35.423 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 0, 2024-02-17T13:07:35.424 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-02-17T13:07:35.424 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-02-17T13:07:35.424 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 0, 2024-02-17T13:07:35.424 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-02-17T13:07:35.424 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-02-17T13:07:35.424 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 21095, 2024-02-17T13:07:35.424 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 28, 2024-02-17T13:07:35.424 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 21067, 2024-02-17T13:07:35.424 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-02-17T13:07:35.425 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 21055, 2024-02-17T13:07:35.425 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2024-02-17T13:07:35.425 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 0, 2024-02-17T13:07:35.425 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 0, 2024-02-17T13:07:35.425 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 16, 2024-02-17T13:07:35.425 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 0, 2024-02-17T13:07:35.425 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 0, 2024-02-17T13:07:35.425 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2024-02-17T13:07:35.425 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 0, 2024-02-17T13:07:35.425 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-02-17T13:07:35.426 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-02-17T13:07:35.426 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-02-17T13:07:35.426 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-02-17T13:07:35.426 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-02-17T13:07:35.426 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-02-17T13:07:35.426 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-02-17T13:07:35.426 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2024-02-17T13:07:35.426 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2024-02-17T13:07:35.426 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-02-17T13:07:35.427 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2024-02-17T13:07:35.427 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-02-17T13:07:35.427 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 0, 2024-02-17T13:07:35.427 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 0, 2024-02-17T13:07:35.427 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-02-17T13:07:35.427 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2024-02-17T13:07:35.427 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-02-17T13:07:35.427 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T13:07:35.428 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-02-17T13:07:35.428 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-02-17T13:07:35.428 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T13:07:35.428 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T13:07:35.428 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2024-02-17T13:07:35.428 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-02-17T13:07:35.428 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 0, 2024-02-17T13:07:35.428 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2024-02-17T13:07:35.429 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T13:07:35.429 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-17T13:07:35.429 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T13:07:35.429 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 0, 2024-02-17T13:07:35.429 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2024-02-17T13:07:35.429 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2024-02-17T13:07:35.429 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 0 2024-02-17T13:07:35.429 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.429 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-02-17T13:07:35.430 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-02-17T13:07:35.430 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-02-17T13:07:35.430 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-02-17T13:07:35.430 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 0, 2024-02-17T13:07:35.430 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2024-02-17T13:07:35.430 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-02-17T13:07:35.430 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-02-17T13:07:35.430 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-02-17T13:07:35.431 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2024-02-17T13:07:35.431 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2024-02-17T13:07:35.431 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-02-17T13:07:35.431 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-02-17T13:07:35.431 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-02-17T13:07:35.431 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-02-17T13:07:35.431 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-02-17T13:07:35.431 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-02-17T13:07:35.431 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-02-17T13:07:35.432 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T13:07:35.432 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-02-17T13:07:35.432 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-02-17T13:07:35.432 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-02-17T13:07:35.432 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-02-17T13:07:35.432 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-02-17T13:07:35.432 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.432 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-02-17T13:07:35.433 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-02-17T13:07:35.433 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 0, 2024-02-17T13:07:35.433 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-02-17T13:07:35.433 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-02-17T13:07:35.433 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 95722, 2024-02-17T13:07:35.433 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-02-17T13:07:35.433 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-02-17T13:07:35.433 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 0, 2024-02-17T13:07:35.433 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-02-17T13:07:35.434 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-02-17T13:07:35.434 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 127, 2024-02-17T13:07:35.434 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 9, 2024-02-17T13:07:35.434 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-02-17T13:07:35.434 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-02-17T13:07:35.434 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-02-17T13:07:35.434 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 346207642, 2024-02-17T13:07:35.434 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 346202437, 2024-02-17T13:07:35.434 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-02-17T13:07:35.435 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.435 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.435 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.435 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.435 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 95722 2024-02-17T13:07:35.435 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.435 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-02-17T13:07:35.435 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 1111, 2024-02-17T13:07:35.436 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 21373, 2024-02-17T13:07:35.436 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 20262, 2024-02-17T13:07:35.436 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 103, 2024-02-17T13:07:35.436 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 159, 2024-02-17T13:07:35.436 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 56, 2024-02-17T13:07:35.436 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 21111, 2024-02-17T13:07:35.436 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 21668, 2024-02-17T13:07:35.436 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 557, 2024-02-17T13:07:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 0, 2024-02-17T13:07:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 0, 2024-02-17T13:07:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2024-02-17T13:07:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 76476, 2024-02-17T13:07:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 215420 2024-02-17T13:07:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-02-17T13:07:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 0, 2024-02-17T13:07:35.437 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2024-02-17T13:07:35.438 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 0, 2024-02-17T13:07:35.438 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 0, 2024-02-17T13:07:35.438 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 0, 2024-02-17T13:07:35.438 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-02-17T13:07:35.438 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.438 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.438 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.438 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.438 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-02-17T13:07:35.439 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.439 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.439 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.439 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.439 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-02-17T13:07:35.439 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.439 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.439 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.440 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.440 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-02-17T13:07:35.440 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.440 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.440 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.440 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.440 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-02-17T13:07:35.440 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.440 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.441 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.441 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.441 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-02-17T13:07:35.441 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.441 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.441 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.441 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.441 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-02-17T13:07:35.442 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.442 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.442 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.442 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.442 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-02-17T13:07:35.442 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.442 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.442 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.442 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.443 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-02-17T13:07:35.443 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.443 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.443 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.443 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.443 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-02-17T13:07:35.443 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.443 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.443 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.444 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.444 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-02-17T13:07:35.444 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.444 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.444 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.444 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.444 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-02-17T13:07:35.444 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.445 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.445 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.445 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.446 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-02-17T13:07:35.446 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.446 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.446 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.446 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.446 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-02-17T13:07:35.446 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.446 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.447 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.447 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.447 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-02-17T13:07:35.447 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.447 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.447 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.447 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.447 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-02-17T13:07:35.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.448 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.448 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.448 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-02-17T13:07:35.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.448 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.449 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.449 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-02-17T13:07:35.449 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.449 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.449 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.449 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.449 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-02-17T13:07:35.449 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.449 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.450 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.450 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-02-17T13:07:35.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.450 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.450 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.450 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-02-17T13:07:35.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.451 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.451 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.451 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.451 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-02-17T13:07:35.451 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.451 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.451 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.451 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.451 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-02-17T13:07:35.452 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.452 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.452 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.452 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.452 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-02-17T13:07:35.452 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.452 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.452 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.453 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.453 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-02-17T13:07:35.453 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.453 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.453 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.453 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.453 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-02-17T13:07:35.453 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.454 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.454 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.454 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.454 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-02-17T13:07:35.454 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.454 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.454 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.454 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.454 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-02-17T13:07:35.455 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.455 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.455 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.455 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.455 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-02-17T13:07:35.455 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.455 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.455 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.456 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.456 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-02-17T13:07:35.456 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-02-17T13:07:35.456 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-02-17T13:07:35.456 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.456 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.456 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.456 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:35.456 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.457 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-02-17T13:07:35.457 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 2, 2024-02-17T13:07:35.457 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 3, 2024-02-17T13:07:35.457 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 1, 2024-02-17T13:07:35.457 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 2, 2024-02-17T13:07:35.457 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-02-17T13:07:35.457 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 0, 2024-02-17T13:07:35.457 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 0, 2024-02-17T13:07:35.458 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 2397, 2024-02-17T13:07:35.458 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-02-17T13:07:35.458 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.458 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-02-17T13:07:35.458 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-02-17T13:07:35.458 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-02-17T13:07:35.458 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T13:07:35.458 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-02-17T13:07:35.459 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T13:07:35.459 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-02-17T13:07:35.459 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T13:07:35.459 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T13:07:35.459 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T13:07:35.459 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T13:07:35.459 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T13:07:35.459 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-02-17T13:07:35.459 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T13:07:35.460 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T13:07:35.460 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-02-17T13:07:35.460 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-02-17T13:07:35.460 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-02-17T13:07:35.460 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-02-17T13:07:35.460 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T13:07:35.460 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T13:07:35.460 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T13:07:35.461 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T13:07:35.461 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T13:07:35.461 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-02-17T13:07:35.461 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-02-17T13:07:35.461 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-02-17T13:07:35.461 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T13:07:35.461 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T13:07:35.461 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-02-17T13:07:35.462 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-02-17T13:07:35.462 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-02-17T13:07:35.462 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-02-17T13:07:35.462 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T13:07:35.462 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-02-17T13:07:35.462 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T13:07:35.462 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-02-17T13:07:35.462 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 288653, 2024-02-17T13:07:35.462 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 1352, 2024-02-17T13:07:35.463 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-02-17T13:07:35.463 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-02-17T13:07:35.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-02-17T13:07:35.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-02-17T13:07:35.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-02-17T13:07:35.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-02-17T13:07:35.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-02-17T13:07:35.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-02-17T13:07:35.463 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 7936, 2024-02-17T13:07:35.464 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 165, 2024-02-17T13:07:35.464 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T13:07:35.464 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-02-17T13:07:35.464 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-02-17T13:07:35.464 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-02-17T13:07:35.464 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 17785485, 2024-02-17T13:07:35.464 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 137697, 2024-02-17T13:07:35.464 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-02-17T13:07:35.465 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-02-17T13:07:35.465 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-02-17T13:07:35.465 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-02-17T13:07:35.465 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.465 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-02-17T13:07:35.465 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-02-17T13:07:35.465 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-02-17T13:07:35.465 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 4454, 2024-02-17T13:07:35.465 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 21, 2024-02-17T13:07:35.465 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 3, 2024-02-17T13:07:35.466 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 4454, 2024-02-17T13:07:35.466 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-02-17T13:07:35.466 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 4454, 2024-02-17T13:07:35.466 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 5.129915917, 2024-02-17T13:07:35.466 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001151754 2024-02-17T13:07:35.466 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.466 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-02-17T13:07:35.466 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-02-17T13:07:35.466 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 4454, 2024-02-17T13:07:35.467 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 4455 2024-02-17T13:07:35.467 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.467 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 4454, 2024-02-17T13:07:35.467 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 4454, 2024-02-17T13:07:35.467 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 0, 2024-02-17T13:07:35.467 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-02-17T13:07:35.467 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-02-17T13:07:35.467 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 2520, 2024-02-17T13:07:35.468 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 0, 2024-02-17T13:07:35.468 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 1932, 2024-02-17T13:07:35.468 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 0, 2024-02-17T13:07:35.468 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 0, 2024-02-17T13:07:35.468 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-02-17T13:07:35.468 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-02-17T13:07:35.468 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-02-17T13:07:35.468 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-02-17T13:07:35.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 0, 2024-02-17T13:07:35.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-02-17T13:07:35.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-02-17T13:07:35.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-02-17T13:07:35.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 1, 2024-02-17T13:07:35.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 0, 2024-02-17T13:07:35.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-02-17T13:07:35.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-02-17T13:07:35.469 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-02-17T13:07:35.470 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-02-17T13:07:35.470 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-02-17T13:07:35.470 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-02-17T13:07:35.470 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T13:07:35.470 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-02-17T13:07:35.470 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-02-17T13:07:35.470 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-02-17T13:07:35.470 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-02-17T13:07:35.470 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-02-17T13:07:35.471 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-02-17T13:07:35.471 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-02-17T13:07:35.471 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-02-17T13:07:35.471 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-02-17T13:07:35.471 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-02-17T13:07:35.471 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-02-17T13:07:35.471 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-02-17T13:07:35.471 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-02-17T13:07:35.471 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-02-17T13:07:35.472 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-02-17T13:07:35.472 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-02-17T13:07:35.472 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-02-17T13:07:35.472 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-02-17T13:07:35.472 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-02-17T13:07:35.472 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 113, 2024-02-17T13:07:35.472 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-02-17T13:07:35.472 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 27, 2024-02-17T13:07:35.473 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 11, 2024-02-17T13:07:35.473 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 12, 2024-02-17T13:07:35.473 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 1, 2024-02-17T13:07:35.473 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-02-17T13:07:35.473 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 0, 2024-02-17T13:07:35.473 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 2, 2024-02-17T13:07:35.473 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 0 2024-02-17T13:07:35.473 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.473 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-02-17T13:07:35.474 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 0, 2024-02-17T13:07:35.474 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-02-17T13:07:35.474 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-02-17T13:07:35.474 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-02-17T13:07:35.474 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.474 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-02-17T13:07:35.474 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-02-17T13:07:35.474 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.474 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-02-17T13:07:35.475 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-02-17T13:07:35.475 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2024-02-17T13:07:35.475 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-02-17T13:07:35.475 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2024-02-17T13:07:35.475 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 0, 2024-02-17T13:07:35.475 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2024-02-17T13:07:35.475 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-02-17T13:07:35.475 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.475 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T13:07:35.476 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:35.476 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-02-17T13:07:35.476 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:35.476 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 4885, 2024-02-17T13:07:35.476 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 344802339, 2024-02-17T13:07:35.476 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:35.476 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 4885, 2024-02-17T13:07:35.476 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-02-17T13:07:35.477 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-02-17T13:07:35.477 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 4885, 2024-02-17T13:07:35.477 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 344802339, 2024-02-17T13:07:35.477 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:35.477 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.477 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.477 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.477 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:35.478 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.478 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-02-17T13:07:35.478 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:35.478 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-02-17T13:07:35.478 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:35.478 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:35.478 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:35.478 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:35.478 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 4454, 2024-02-17T13:07:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 342016300, 2024-02-17T13:07:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 4454, 2024-02-17T13:07:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 342016300, 2024-02-17T13:07:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-02-17T13:07:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-02-17T13:07:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 4454, 2024-02-17T13:07:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 4454, 2024-02-17T13:07:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 4454, 2024-02-17T13:07:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 4454, 2024-02-17T13:07:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.482 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:35.482 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.482 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-02-17T13:07:35.482 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:35.482 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-02-17T13:07:35.482 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:35.482 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:35.482 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:35.482 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:35.483 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:35.483 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-02-17T13:07:35.483 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-02-17T13:07:35.483 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-02-17T13:07:35.483 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-02-17T13:07:35.483 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:35.483 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.483 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.483 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.484 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:35.484 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:35.484 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x55b9fccfe960": { 2024-02-17T13:07:35.484 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:35.484 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-02-17T13:07:35.484 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:35.484 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:35.484 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:35.484 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:35.485 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:35.485 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-02-17T13:07:35.485 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-02-17T13:07:35.485 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-02-17T13:07:35.485 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-02-17T13:07:35.485 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:35.485 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:35.485 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:35.485 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:35.486 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:35.486 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:35.486 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-02-17T13:07:36.216 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-02-17T13:07:36.216 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-02-17T13:07:36.216 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-02-17T13:07:36.216 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-02-17T13:07:36.216 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T13:07:36.217 DEBUG:tasks.check_counter:Getting stats from f 2024-02-17T13:07:36.217 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph daemon mds.f perf dump 2024-02-17T13:07:36.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:36 smithi146 ceph-mon[36454]: pgmap v558: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 5.6 MiB/s wr, 221 op/s 2024-02-17T13:07:36.463 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T13:07:36.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:36 smithi018 ceph-mon[32625]: pgmap v558: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 5.6 MiB/s wr, 221 op/s 2024-02-17T13:07:36.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:36 smithi119 ceph-mon[36188]: pgmap v558: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 5.6 MiB/s wr, 221 op/s 2024-02-17T13:07:38.162 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-02-17T13:07:38.162 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T13:07:38.162 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 269700, 2024-02-17T13:07:38.162 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 334506, 2024-02-17T13:07:38.162 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 86766498, 2024-02-17T13:07:38.162 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 293660412, 2024-02-17T13:07:38.163 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 56, 2024-02-17T13:07:38.163 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 6, 2024-02-17T13:07:38.163 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 14.632915189, 2024-02-17T13:07:38.163 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 6.369302095, 2024-02-17T13:07:38.163 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 10.132776257, 2024-02-17T13:07:38.163 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.421692326, 2024-02-17T13:07:38.163 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:38.163 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 334507, 2024-02-17T13:07:38.164 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 9.544621111, 2024-02-17T13:07:38.164 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000028533 2024-02-17T13:07:38.164 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.164 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:38.164 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 532561, 2024-02-17T13:07:38.164 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.085187147, 2024-02-17T13:07:38.164 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000159 2024-02-17T13:07:38.164 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.165 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 2019168, 2024-02-17T13:07:38.165 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 233632 2024-02-17T13:07:38.165 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.165 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T13:07:38.165 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 41485, 2024-02-17T13:07:38.165 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 45848, 2024-02-17T13:07:38.165 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 10391760, 2024-02-17T13:07:38.165 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 120980524, 2024-02-17T13:07:38.166 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 12, 2024-02-17T13:07:38.166 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 8, 2024-02-17T13:07:38.166 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 2.769817727, 2024-02-17T13:07:38.166 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 1.134100504, 2024-02-17T13:07:38.166 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 1.381581128, 2024-02-17T13:07:38.166 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.388272202, 2024-02-17T13:07:38.166 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:38.166 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 45844, 2024-02-17T13:07:38.166 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1.434950807, 2024-02-17T13:07:38.167 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000031300 2024-02-17T13:07:38.167 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.167 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:38.167 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 57540, 2024-02-17T13:07:38.167 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.009407473, 2024-02-17T13:07:38.167 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000163 2024-02-17T13:07:38.167 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.167 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 7600, 2024-02-17T13:07:38.168 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 616112 2024-02-17T13:07:38.168 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.168 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T13:07:38.168 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 55747, 2024-02-17T13:07:38.168 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 61344, 2024-02-17T13:07:38.168 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 16557388, 2024-02-17T13:07:38.168 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 115212689, 2024-02-17T13:07:38.168 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 21, 2024-02-17T13:07:38.168 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 7, 2024-02-17T13:07:38.169 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 3.356519752, 2024-02-17T13:07:38.169 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 1.367200623, 2024-02-17T13:07:38.169 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 1.798270134, 2024-02-17T13:07:38.169 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.409164091, 2024-02-17T13:07:38.169 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:38.169 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 61342, 2024-02-17T13:07:38.169 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 2.108514387, 2024-02-17T13:07:38.169 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000034373 2024-02-17T13:07:38.170 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.170 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:38.170 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 83788, 2024-02-17T13:07:38.170 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.013008576, 2024-02-17T13:07:38.170 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000155 2024-02-17T13:07:38.170 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.170 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T13:07:38.170 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T13:07:38.171 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.171 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-02-17T13:07:38.171 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-02-17T13:07:38.171 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-02-17T13:07:38.171 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.171 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-02-17T13:07:38.171 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-02-17T13:07:38.171 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-02-17T13:07:38.171 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 47500, 2024-02-17T13:07:38.172 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 40.512270586, 2024-02-17T13:07:38.172 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000852889 2024-02-17T13:07:38.172 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:38.172 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.172 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-02-17T13:07:38.172 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-02-17T13:07:38.172 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-02-17T13:07:38.172 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 25932, 2024-02-17T13:07:38.173 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 2.470780665, 2024-02-17T13:07:38.173 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000095279 2024-02-17T13:07:38.173 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:38.173 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.173 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-02-17T13:07:38.173 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 101544, 2024-02-17T13:07:38.173 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 95281, 2024-02-17T13:07:38.173 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-02-17T13:07:38.173 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 95281, 2024-02-17T13:07:38.174 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 297.332356296, 2024-02-17T13:07:38.174 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.003120583 2024-02-17T13:07:38.174 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.174 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-02-17T13:07:38.174 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 6263, 2024-02-17T13:07:38.174 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 15, 2024-02-17T13:07:38.174 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-02-17T13:07:38.174 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 80, 2024-02-17T13:07:38.175 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 8, 2024-02-17T13:07:38.175 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 5, 2024-02-17T13:07:38.175 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 1829, 2024-02-17T13:07:38.175 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 427, 2024-02-17T13:07:38.175 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 1104, 2024-02-17T13:07:38.175 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 298, 2024-02-17T13:07:38.175 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 1219, 2024-02-17T13:07:38.175 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 413, 2024-02-17T13:07:38.175 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 313, 2024-02-17T13:07:38.176 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 314, 2024-02-17T13:07:38.176 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 25, 2024-02-17T13:07:38.176 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 166144, 2024-02-17T13:07:38.176 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 91493, 2024-02-17T13:07:38.176 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 6255, 2024-02-17T13:07:38.176 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 56, 2024-02-17T13:07:38.176 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 11, 2024-02-17T13:07:38.176 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-02-17T13:07:38.177 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 1662, 2024-02-17T13:07:38.177 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 12996, 2024-02-17T13:07:38.177 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-02-17T13:07:38.177 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 8, 2024-02-17T13:07:38.177 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 290, 2024-02-17T13:07:38.177 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 14, 2024-02-17T13:07:38.177 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 639, 2024-02-17T13:07:38.177 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-02-17T13:07:38.178 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 274, 2024-02-17T13:07:38.178 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-02-17T13:07:38.178 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 735, 2024-02-17T13:07:38.178 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 211953355, 2024-02-17T13:07:38.178 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 7, 2024-02-17T13:07:38.178 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 274, 2024-02-17T13:07:38.178 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-02-17T13:07:38.178 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T13:07:38.179 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-02-17T13:07:38.179 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-02-17T13:07:38.179 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T13:07:38.179 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T13:07:38.179 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 274, 2024-02-17T13:07:38.179 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 5, 2024-02-17T13:07:38.179 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 24066, 2024-02-17T13:07:38.179 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 62589, 2024-02-17T13:07:38.179 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T13:07:38.180 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 2758, 2024-02-17T13:07:38.180 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T13:07:38.180 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 47470, 2024-02-17T13:07:38.180 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 15760, 2024-02-17T13:07:38.180 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 6102, 2024-02-17T13:07:38.180 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 67834 2024-02-17T13:07:38.180 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.180 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-02-17T13:07:38.181 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 22, 2024-02-17T13:07:38.181 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 178, 2024-02-17T13:07:38.181 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 106, 2024-02-17T13:07:38.181 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 196, 2024-02-17T13:07:38.181 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 61, 2024-02-17T13:07:38.181 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 107, 2024-02-17T13:07:38.181 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-02-17T13:07:38.181 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-02-17T13:07:38.181 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 13469, 2024-02-17T13:07:38.182 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 13362, 2024-02-17T13:07:38.182 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-02-17T13:07:38.182 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-02-17T13:07:38.182 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-02-17T13:07:38.182 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-02-17T13:07:38.182 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-02-17T13:07:38.182 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-02-17T13:07:38.182 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-02-17T13:07:38.183 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T13:07:38.183 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 13, 2024-02-17T13:07:38.183 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-02-17T13:07:38.183 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 13, 2024-02-17T13:07:38.183 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-02-17T13:07:38.183 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-02-17T13:07:38.183 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.183 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-02-17T13:07:38.183 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-02-17T13:07:38.184 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 78835, 2024-02-17T13:07:38.184 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-02-17T13:07:38.184 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-02-17T13:07:38.184 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 78835, 2024-02-17T13:07:38.184 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-02-17T13:07:38.184 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-02-17T13:07:38.184 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 105, 2024-02-17T13:07:38.184 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-02-17T13:07:38.185 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-02-17T13:07:38.185 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 105, 2024-02-17T13:07:38.185 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 8, 2024-02-17T13:07:38.185 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-02-17T13:07:38.185 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-02-17T13:07:38.185 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-02-17T13:07:38.185 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 286738522, 2024-02-17T13:07:38.185 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 0, 2024-02-17T13:07:38.185 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-02-17T13:07:38.186 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 25415, 2024-02-17T13:07:38.186 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 151.551903338, 2024-02-17T13:07:38.186 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.005963088 2024-02-17T13:07:38.186 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.186 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 0 2024-02-17T13:07:38.186 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.186 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-02-17T13:07:38.186 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 872, 2024-02-17T13:07:38.187 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 14540, 2024-02-17T13:07:38.187 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 13668, 2024-02-17T13:07:38.187 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 162, 2024-02-17T13:07:38.187 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 311, 2024-02-17T13:07:38.187 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 149, 2024-02-17T13:07:38.187 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 1709, 2024-02-17T13:07:38.187 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 28301, 2024-02-17T13:07:38.187 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 26592, 2024-02-17T13:07:38.187 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 230, 2024-02-17T13:07:38.188 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 15488, 2024-02-17T13:07:38.188 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 15258, 2024-02-17T13:07:38.188 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 73288, 2024-02-17T13:07:38.188 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 207228 2024-02-17T13:07:38.188 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.188 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-02-17T13:07:38.188 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 145518, 2024-02-17T13:07:38.188 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 3580, 2024-02-17T13:07:38.189 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 101544, 2024-02-17T13:07:38.189 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 2305, 2024-02-17T13:07:38.189 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 12160, 2024-02-17T13:07:38.189 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-02-17T13:07:38.189 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 13419, 2024-02-17T13:07:38.189 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 36.411149365, 2024-02-17T13:07:38.189 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002713402 2024-02-17T13:07:38.189 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.190 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-02-17T13:07:38.190 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3547, 2024-02-17T13:07:38.190 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 61.016055143, 2024-02-17T13:07:38.190 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.017202158 2024-02-17T13:07:38.190 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.190 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-02-17T13:07:38.190 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.190 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.190 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.191 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.191 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-02-17T13:07:38.191 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.191 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.191 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.191 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.191 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-02-17T13:07:38.191 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 30606, 2024-02-17T13:07:38.192 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 33.166494003, 2024-02-17T13:07:38.192 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001083659 2024-02-17T13:07:38.192 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.192 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-02-17T13:07:38.192 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.192 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.192 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.192 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.193 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-02-17T13:07:38.193 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1, 2024-02-17T13:07:38.193 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000090347, 2024-02-17T13:07:38.193 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000090347 2024-02-17T13:07:38.193 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.193 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-02-17T13:07:38.193 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.193 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.193 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.194 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.194 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-02-17T13:07:38.194 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.194 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.194 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.194 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.194 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-02-17T13:07:38.194 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.195 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.195 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.195 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.196 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-02-17T13:07:38.196 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.196 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.196 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.196 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.196 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-02-17T13:07:38.196 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.196 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.197 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.197 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.197 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-02-17T13:07:38.197 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.197 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.197 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.197 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.197 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-02-17T13:07:38.198 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.198 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.198 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.198 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.198 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-02-17T13:07:38.198 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.198 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.198 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.198 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.199 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-02-17T13:07:38.199 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 30855, 2024-02-17T13:07:38.199 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 124.371239757, 2024-02-17T13:07:38.199 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.004030829 2024-02-17T13:07:38.199 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.199 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-02-17T13:07:38.199 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3105, 2024-02-17T13:07:38.199 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 7.851570513, 2024-02-17T13:07:38.200 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002528686 2024-02-17T13:07:38.200 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.200 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-02-17T13:07:38.200 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.200 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.200 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.200 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.200 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-02-17T13:07:38.201 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.201 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.201 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.201 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.201 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-02-17T13:07:38.201 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.201 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.201 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.202 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.202 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-02-17T13:07:38.202 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.202 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.202 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.202 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.202 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-02-17T13:07:38.202 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.202 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.203 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.203 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.203 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-02-17T13:07:38.203 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 10, 2024-02-17T13:07:38.203 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.076576542, 2024-02-17T13:07:38.203 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.007657654 2024-02-17T13:07:38.203 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.203 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-02-17T13:07:38.204 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.204 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.204 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.204 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.204 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-02-17T13:07:38.204 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 269, 2024-02-17T13:07:38.204 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.272921250, 2024-02-17T13:07:38.204 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001014577 2024-02-17T13:07:38.205 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.205 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-02-17T13:07:38.205 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.205 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.205 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.205 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.205 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-02-17T13:07:38.205 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.205 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.206 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.206 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.206 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-02-17T13:07:38.206 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.206 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.206 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.206 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.206 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-02-17T13:07:38.207 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 13469, 2024-02-17T13:07:38.207 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 34.166259376, 2024-02-17T13:07:38.207 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002536658 2024-02-17T13:07:38.207 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.207 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-02-17T13:07:38.207 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-02-17T13:07:38.207 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-02-17T13:07:38.207 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.207 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.208 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.208 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:38.208 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.208 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-02-17T13:07:38.208 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 2, 2024-02-17T13:07:38.208 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 2, 2024-02-17T13:07:38.208 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 0, 2024-02-17T13:07:38.208 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 2, 2024-02-17T13:07:38.209 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-02-17T13:07:38.209 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 24103, 2024-02-17T13:07:38.209 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 12051, 2024-02-17T13:07:38.209 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 1055, 2024-02-17T13:07:38.209 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-02-17T13:07:38.209 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.209 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-02-17T13:07:38.209 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-02-17T13:07:38.210 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-02-17T13:07:38.210 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T13:07:38.210 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-02-17T13:07:38.210 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T13:07:38.210 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-02-17T13:07:38.210 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T13:07:38.210 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T13:07:38.210 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T13:07:38.210 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T13:07:38.211 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T13:07:38.211 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-02-17T13:07:38.211 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T13:07:38.211 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T13:07:38.211 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-02-17T13:07:38.211 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-02-17T13:07:38.211 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-02-17T13:07:38.211 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-02-17T13:07:38.212 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T13:07:38.212 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T13:07:38.212 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T13:07:38.212 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T13:07:38.212 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T13:07:38.212 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-02-17T13:07:38.212 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-02-17T13:07:38.212 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-02-17T13:07:38.212 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T13:07:38.213 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T13:07:38.213 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-02-17T13:07:38.213 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-02-17T13:07:38.213 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-02-17T13:07:38.213 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-02-17T13:07:38.213 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T13:07:38.214 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-02-17T13:07:38.214 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T13:07:38.214 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-02-17T13:07:38.214 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 2721529, 2024-02-17T13:07:38.214 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 1413, 2024-02-17T13:07:38.214 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-02-17T13:07:38.214 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-02-17T13:07:38.214 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-02-17T13:07:38.215 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-02-17T13:07:38.215 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-02-17T13:07:38.215 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-02-17T13:07:38.215 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-02-17T13:07:38.215 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-02-17T13:07:38.215 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 7936, 2024-02-17T13:07:38.215 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 165, 2024-02-17T13:07:38.215 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T13:07:38.215 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-02-17T13:07:38.216 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-02-17T13:07:38.216 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-02-17T13:07:38.216 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 4449274, 2024-02-17T13:07:38.216 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 80791, 2024-02-17T13:07:38.216 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-02-17T13:07:38.216 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-02-17T13:07:38.216 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-02-17T13:07:38.216 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-02-17T13:07:38.217 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.217 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-02-17T13:07:38.217 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-02-17T13:07:38.217 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-02-17T13:07:38.217 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 41993, 2024-02-17T13:07:38.217 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 286059857, 2024-02-17T13:07:38.217 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 33, 2024-02-17T13:07:38.217 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 41993, 2024-02-17T13:07:38.218 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-02-17T13:07:38.218 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 41993, 2024-02-17T13:07:38.218 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 232.225245944, 2024-02-17T13:07:38.218 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.005530094 2024-02-17T13:07:38.218 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.218 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-02-17T13:07:38.218 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-02-17T13:07:38.218 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 41993, 2024-02-17T13:07:38.218 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 43188 2024-02-17T13:07:38.219 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.219 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 41993, 2024-02-17T13:07:38.219 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 1078, 2024-02-17T13:07:38.219 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 40915, 2024-02-17T13:07:38.219 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-02-17T13:07:38.219 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-02-17T13:07:38.219 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 0, 2024-02-17T13:07:38.219 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 10, 2024-02-17T13:07:38.220 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 789, 2024-02-17T13:07:38.220 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 26204, 2024-02-17T13:07:38.220 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 454, 2024-02-17T13:07:38.220 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-02-17T13:07:38.220 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-02-17T13:07:38.220 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-02-17T13:07:38.220 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 8, 2024-02-17T13:07:38.220 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 13460, 2024-02-17T13:07:38.221 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-02-17T13:07:38.221 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-02-17T13:07:38.221 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-02-17T13:07:38.221 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 286, 2024-02-17T13:07:38.221 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 20, 2024-02-17T13:07:38.221 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-02-17T13:07:38.221 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-02-17T13:07:38.221 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-02-17T13:07:38.222 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-02-17T13:07:38.222 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-02-17T13:07:38.222 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-02-17T13:07:38.222 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T13:07:38.222 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-02-17T13:07:38.222 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-02-17T13:07:38.222 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-02-17T13:07:38.222 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-02-17T13:07:38.222 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-02-17T13:07:38.223 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-02-17T13:07:38.223 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-02-17T13:07:38.223 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-02-17T13:07:38.223 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-02-17T13:07:38.223 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-02-17T13:07:38.223 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-02-17T13:07:38.224 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-02-17T13:07:38.224 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-02-17T13:07:38.224 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-02-17T13:07:38.224 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-02-17T13:07:38.224 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-02-17T13:07:38.224 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-02-17T13:07:38.224 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-02-17T13:07:38.224 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-02-17T13:07:38.225 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 113, 2024-02-17T13:07:38.225 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-02-17T13:07:38.225 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 27, 2024-02-17T13:07:38.225 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 12, 2024-02-17T13:07:38.225 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 12, 2024-02-17T13:07:38.225 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-02-17T13:07:38.225 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-02-17T13:07:38.225 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 1469, 2024-02-17T13:07:38.225 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 30, 2024-02-17T13:07:38.226 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 458 2024-02-17T13:07:38.226 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.226 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-02-17T13:07:38.226 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 1, 2024-02-17T13:07:38.226 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-02-17T13:07:38.226 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-02-17T13:07:38.226 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-02-17T13:07:38.226 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.227 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-02-17T13:07:38.227 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-02-17T13:07:38.227 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.227 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-02-17T13:07:38.227 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-02-17T13:07:38.227 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 6, 2024-02-17T13:07:38.227 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-02-17T13:07:38.227 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 6, 2024-02-17T13:07:38.227 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 13362, 2024-02-17T13:07:38.228 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 13362, 2024-02-17T13:07:38.228 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-02-17T13:07:38.228 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.228 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T13:07:38.228 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:38.228 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-02-17T13:07:38.228 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:38.228 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 366928, 2024-02-17T13:07:38.229 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 80663508, 2024-02-17T13:07:38.229 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:38.229 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 366928, 2024-02-17T13:07:38.229 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-02-17T13:07:38.229 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-02-17T13:07:38.229 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 366928, 2024-02-17T13:07:38.229 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 80663508, 2024-02-17T13:07:38.229 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:38.230 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.230 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.230 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.230 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:38.230 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.230 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-02-17T13:07:38.230 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:38.230 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-02-17T13:07:38.231 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:38.231 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:38.231 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:38.231 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:38.231 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:38.231 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 41993, 2024-02-17T13:07:38.231 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 287410979, 2024-02-17T13:07:38.231 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 41993, 2024-02-17T13:07:38.232 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 287410979, 2024-02-17T13:07:38.232 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:38.232 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.232 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.232 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.232 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:38.232 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.232 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-02-17T13:07:38.232 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:38.233 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-02-17T13:07:38.233 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:38.233 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:38.233 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:38.233 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:38.233 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:38.233 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 41993, 2024-02-17T13:07:38.233 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 41993, 2024-02-17T13:07:38.234 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 41993, 2024-02-17T13:07:38.234 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 41993, 2024-02-17T13:07:38.234 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:38.234 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.234 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.234 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.234 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:38.234 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.234 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-02-17T13:07:38.235 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:38.235 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-02-17T13:07:38.235 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:38.235 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 13362, 2024-02-17T13:07:38.235 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 1351362, 2024-02-17T13:07:38.235 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:38.235 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 13362, 2024-02-17T13:07:38.235 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-02-17T13:07:38.236 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-02-17T13:07:38.236 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 789, 2024-02-17T13:07:38.236 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1351362, 2024-02-17T13:07:38.236 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:38.236 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.236 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.236 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.236 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:38.236 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:38.237 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x55a75a9f46e0": { 2024-02-17T13:07:38.237 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 10782, 2024-02-17T13:07:38.237 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-02-17T13:07:38.237 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:38.237 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 78835, 2024-02-17T13:07:38.237 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 282544218, 2024-02-17T13:07:38.237 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:38.237 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 78835, 2024-02-17T13:07:38.238 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-02-17T13:07:38.238 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-02-17T13:07:38.238 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 25415, 2024-02-17T13:07:38.238 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 282533436, 2024-02-17T13:07:38.238 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:38.238 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:38.238 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:38.238 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:38.238 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:38.239 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:38.239 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-02-17T13:07:38.447 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:38 smithi146 ceph-mon[36454]: pgmap v559: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 3.6 MiB/s wr, 155 op/s 2024-02-17T13:07:38.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:38 smithi018 ceph-mon[32625]: pgmap v559: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 3.6 MiB/s wr, 155 op/s 2024-02-17T13:07:38.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:38 smithi119 ceph-mon[36188]: pgmap v559: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 3.6 MiB/s wr, 155 op/s 2024-02-17T13:07:39.006 INFO:tasks.check_counter:Daemon mds.f mds.exported=8 2024-02-17T13:07:39.007 INFO:tasks.check_counter:Daemon mds.f mds.imported=14 2024-02-17T13:07:39.007 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-02-17T13:07:39.007 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-02-17T13:07:39.007 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T13:07:39.007 DEBUG:tasks.check_counter:Getting stats from i 2024-02-17T13:07:39.007 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph daemon mds.i perf dump 2024-02-17T13:07:39.250 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T13:07:40.507 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:40 smithi146 ceph-mon[36454]: pgmap v560: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 3.3 MiB/s wr, 149 op/s 2024-02-17T13:07:40.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:40 smithi018 ceph-mon[32625]: pgmap v560: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 3.3 MiB/s wr, 149 op/s 2024-02-17T13:07:40.589 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:40 smithi119 ceph-mon[36188]: pgmap v560: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 3.3 MiB/s wr, 149 op/s 2024-02-17T13:07:40.915 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-02-17T13:07:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T13:07:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 290, 2024-02-17T13:07:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 274, 2024-02-17T13:07:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 42255623, 2024-02-17T13:07:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 96820, 2024-02-17T13:07:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 7, 2024-02-17T13:07:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 3, 2024-02-17T13:07:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.045019884, 2024-02-17T13:07:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.006116275, 2024-02-17T13:07:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.031852739, 2024-02-17T13:07:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.005611132, 2024-02-17T13:07:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 274, 2024-02-17T13:07:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.012475396, 2024-02-17T13:07:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000045530 2024-02-17T13:07:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 257376, 2024-02-17T13:07:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 30576 2024-02-17T13:07:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T13:07:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 318, 2024-02-17T13:07:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 354, 2024-02-17T13:07:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 33574382, 2024-02-17T13:07:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 245928, 2024-02-17T13:07:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 7, 2024-02-17T13:07:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 6, 2024-02-17T13:07:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.045788313, 2024-02-17T13:07:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.007776301, 2024-02-17T13:07:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.027292073, 2024-02-17T13:07:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.008250447, 2024-02-17T13:07:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 352, 2024-02-17T13:07:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.012322920, 2024-02-17T13:07:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000035008 2024-02-17T13:07:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-17T13:07:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 147280 2024-02-17T13:07:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T13:07:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 211, 2024-02-17T13:07:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 213, 2024-02-17T13:07:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 37239569, 2024-02-17T13:07:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 65636, 2024-02-17T13:07:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 6, 2024-02-17T13:07:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 5, 2024-02-17T13:07:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.036123077, 2024-02-17T13:07:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.004431034, 2024-02-17T13:07:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.024873141, 2024-02-17T13:07:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.005458192, 2024-02-17T13:07:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 211, 2024-02-17T13:07:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.012744903, 2024-02-17T13:07:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000060402 2024-02-17T13:07:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T13:07:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T13:07:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-02-17T13:07:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-02-17T13:07:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-02-17T13:07:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-02-17T13:07:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-02-17T13:07:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-02-17T13:07:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1214, 2024-02-17T13:07:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.081204644, 2024-02-17T13:07:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000066890 2024-02-17T13:07:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-02-17T13:07:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-02-17T13:07:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-02-17T13:07:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-02-17T13:07:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 0, 2024-02-17T13:07:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 0, 2024-02-17T13:07:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-02-17T13:07:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-02-17T13:07:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 0, 2024-02-17T13:07:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-02-17T13:07:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-02-17T13:07:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 0, 2024-02-17T13:07:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-02-17T13:07:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-02-17T13:07:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 6896, 2024-02-17T13:07:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 4159, 2024-02-17T13:07:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 2737, 2024-02-17T13:07:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-02-17T13:07:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 955, 2024-02-17T13:07:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2024-02-17T13:07:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 0, 2024-02-17T13:07:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 0, 2024-02-17T13:07:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 14, 2024-02-17T13:07:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 0, 2024-02-17T13:07:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 0, 2024-02-17T13:07:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2024-02-17T13:07:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 0, 2024-02-17T13:07:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-02-17T13:07:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-02-17T13:07:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-02-17T13:07:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-02-17T13:07:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-02-17T13:07:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-02-17T13:07:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-02-17T13:07:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2024-02-17T13:07:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2024-02-17T13:07:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-02-17T13:07:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2024-02-17T13:07:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-02-17T13:07:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 682, 2024-02-17T13:07:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 212486312, 2024-02-17T13:07:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 7, 2024-02-17T13:07:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2024-02-17T13:07:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-02-17T13:07:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T13:07:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-02-17T13:07:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-02-17T13:07:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T13:07:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T13:07:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2024-02-17T13:07:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-02-17T13:07:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 0, 2024-02-17T13:07:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2024-02-17T13:07:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T13:07:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-17T13:07:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T13:07:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 0, 2024-02-17T13:07:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2024-02-17T13:07:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2024-02-17T13:07:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 0 2024-02-17T13:07:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-02-17T13:07:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-02-17T13:07:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-02-17T13:07:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-02-17T13:07:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 0, 2024-02-17T13:07:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2024-02-17T13:07:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-02-17T13:07:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-02-17T13:07:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-02-17T13:07:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2024-02-17T13:07:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2024-02-17T13:07:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-02-17T13:07:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-02-17T13:07:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-02-17T13:07:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-02-17T13:07:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-02-17T13:07:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-02-17T13:07:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-02-17T13:07:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T13:07:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-02-17T13:07:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-02-17T13:07:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-02-17T13:07:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-02-17T13:07:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-02-17T13:07:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-02-17T13:07:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-02-17T13:07:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 0, 2024-02-17T13:07:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-02-17T13:07:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-02-17T13:07:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 29402, 2024-02-17T13:07:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-02-17T13:07:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-02-17T13:07:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 0, 2024-02-17T13:07:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-02-17T13:07:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-02-17T13:07:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 3, 2024-02-17T13:07:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 10, 2024-02-17T13:07:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-02-17T13:07:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-02-17T13:07:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 509740127, 2024-02-17T13:07:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 515336092, 2024-02-17T13:07:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 515334477, 2024-02-17T13:07:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-02-17T13:07:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 29402 2024-02-17T13:07:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-02-17T13:07:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 453, 2024-02-17T13:07:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 6859, 2024-02-17T13:07:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 6406, 2024-02-17T13:07:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 87, 2024-02-17T13:07:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 132, 2024-02-17T13:07:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 45, 2024-02-17T13:07:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 6896, 2024-02-17T13:07:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 6980, 2024-02-17T13:07:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 84, 2024-02-17T13:07:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 0, 2024-02-17T13:07:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 0, 2024-02-17T13:07:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2024-02-17T13:07:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 58144, 2024-02-17T13:07:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 215396 2024-02-17T13:07:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-02-17T13:07:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 0, 2024-02-17T13:07:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2024-02-17T13:07:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 0, 2024-02-17T13:07:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 0, 2024-02-17T13:07:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 0, 2024-02-17T13:07:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-02-17T13:07:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-02-17T13:07:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-02-17T13:07:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-02-17T13:07:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-02-17T13:07:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-02-17T13:07:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-02-17T13:07:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-02-17T13:07:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-02-17T13:07:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-02-17T13:07:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-02-17T13:07:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-02-17T13:07:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-02-17T13:07:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-02-17T13:07:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-02-17T13:07:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.953 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.953 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.953 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.953 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-02-17T13:07:40.953 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.953 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.953 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.953 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.954 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-02-17T13:07:40.954 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.954 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.954 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.954 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.954 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-02-17T13:07:40.954 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.954 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.954 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.955 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.955 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-02-17T13:07:40.955 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.955 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.955 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.955 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.955 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-02-17T13:07:40.955 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.956 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.956 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.956 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.956 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-02-17T13:07:40.956 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.956 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.956 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.956 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.957 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-02-17T13:07:40.957 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.957 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.957 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.957 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.957 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-02-17T13:07:40.957 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.957 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.958 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.958 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.958 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-02-17T13:07:40.958 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.958 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.958 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.958 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.958 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-02-17T13:07:40.959 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.959 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.959 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.959 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.959 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-02-17T13:07:40.959 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.959 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.959 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.960 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.960 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-02-17T13:07:40.960 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.960 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.960 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.960 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.960 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-02-17T13:07:40.960 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.961 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.961 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.961 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.961 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-02-17T13:07:40.961 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.961 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.961 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.961 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.962 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-02-17T13:07:40.962 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-02-17T13:07:40.962 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-02-17T13:07:40.962 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.962 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.962 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.962 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:40.962 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.963 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-02-17T13:07:40.963 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 2, 2024-02-17T13:07:40.963 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 2, 2024-02-17T13:07:40.963 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 0, 2024-02-17T13:07:40.963 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 0, 2024-02-17T13:07:40.963 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-02-17T13:07:40.963 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 0, 2024-02-17T13:07:40.963 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 0, 2024-02-17T13:07:40.963 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 2402, 2024-02-17T13:07:40.964 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-02-17T13:07:40.964 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.964 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-02-17T13:07:40.964 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-02-17T13:07:40.964 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-02-17T13:07:40.964 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T13:07:40.964 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-02-17T13:07:40.965 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T13:07:40.965 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-02-17T13:07:40.965 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T13:07:40.965 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T13:07:40.965 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T13:07:40.965 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T13:07:40.965 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T13:07:40.965 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-02-17T13:07:40.966 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T13:07:40.966 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T13:07:40.966 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-02-17T13:07:40.966 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-02-17T13:07:40.966 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-02-17T13:07:40.966 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-02-17T13:07:40.966 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T13:07:40.966 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T13:07:40.966 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T13:07:40.967 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T13:07:40.967 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T13:07:40.967 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-02-17T13:07:40.967 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-02-17T13:07:40.967 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-02-17T13:07:40.967 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T13:07:40.967 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T13:07:40.967 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-02-17T13:07:40.968 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-02-17T13:07:40.968 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-02-17T13:07:40.968 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-02-17T13:07:40.968 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T13:07:40.968 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-02-17T13:07:40.968 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T13:07:40.968 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-02-17T13:07:40.968 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 280292, 2024-02-17T13:07:40.969 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 716, 2024-02-17T13:07:40.969 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 88, 2024-02-17T13:07:40.969 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 1, 2024-02-17T13:07:40.969 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-02-17T13:07:40.969 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-02-17T13:07:40.969 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-02-17T13:07:40.969 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-02-17T13:07:40.969 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-02-17T13:07:40.970 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-02-17T13:07:40.970 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 7936, 2024-02-17T13:07:40.970 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 165, 2024-02-17T13:07:40.970 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T13:07:40.970 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-02-17T13:07:40.970 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-02-17T13:07:40.970 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-02-17T13:07:40.970 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 6212790, 2024-02-17T13:07:40.971 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 45880, 2024-02-17T13:07:40.971 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-02-17T13:07:40.971 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-02-17T13:07:40.971 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-02-17T13:07:40.971 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-02-17T13:07:40.971 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.971 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-02-17T13:07:40.971 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-02-17T13:07:40.972 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-02-17T13:07:40.972 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 741, 2024-02-17T13:07:40.972 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 26, 2024-02-17T13:07:40.972 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 4, 2024-02-17T13:07:40.972 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 741, 2024-02-17T13:07:40.972 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-02-17T13:07:40.972 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 741, 2024-02-17T13:07:40.972 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1.972968266, 2024-02-17T13:07:40.973 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002662575 2024-02-17T13:07:40.973 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.973 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-02-17T13:07:40.973 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-02-17T13:07:40.973 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 741, 2024-02-17T13:07:40.973 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 742 2024-02-17T13:07:40.973 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.973 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 741, 2024-02-17T13:07:40.974 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 741, 2024-02-17T13:07:40.974 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 0, 2024-02-17T13:07:40.974 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-02-17T13:07:40.974 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-02-17T13:07:40.974 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 390, 2024-02-17T13:07:40.974 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 0, 2024-02-17T13:07:40.974 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 348, 2024-02-17T13:07:40.974 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 0, 2024-02-17T13:07:40.975 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 0, 2024-02-17T13:07:40.975 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-02-17T13:07:40.975 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-02-17T13:07:40.975 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-02-17T13:07:40.975 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-02-17T13:07:40.975 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 0, 2024-02-17T13:07:40.975 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-02-17T13:07:40.975 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-02-17T13:07:40.975 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-02-17T13:07:40.976 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 2, 2024-02-17T13:07:40.976 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 0, 2024-02-17T13:07:40.976 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-02-17T13:07:40.976 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-02-17T13:07:40.976 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-02-17T13:07:40.976 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-02-17T13:07:40.976 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-02-17T13:07:40.977 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-02-17T13:07:40.977 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T13:07:40.977 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-02-17T13:07:40.977 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-02-17T13:07:40.977 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-02-17T13:07:40.977 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-02-17T13:07:40.977 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-02-17T13:07:40.977 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-02-17T13:07:40.978 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-02-17T13:07:40.978 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-02-17T13:07:40.978 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-02-17T13:07:40.978 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-02-17T13:07:40.978 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-02-17T13:07:40.978 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-02-17T13:07:40.978 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-02-17T13:07:40.978 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-02-17T13:07:40.979 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-02-17T13:07:40.979 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-02-17T13:07:40.979 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-02-17T13:07:40.979 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-02-17T13:07:40.979 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-02-17T13:07:40.979 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 113, 2024-02-17T13:07:40.979 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-02-17T13:07:40.979 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 4, 2024-02-17T13:07:40.979 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 11, 2024-02-17T13:07:40.980 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 11, 2024-02-17T13:07:40.980 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-02-17T13:07:40.980 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-02-17T13:07:40.980 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 0, 2024-02-17T13:07:40.980 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 2, 2024-02-17T13:07:40.980 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 0 2024-02-17T13:07:40.980 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.981 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-02-17T13:07:40.981 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 0, 2024-02-17T13:07:40.981 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-02-17T13:07:40.981 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-02-17T13:07:40.981 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-02-17T13:07:40.981 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.981 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-02-17T13:07:40.981 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-02-17T13:07:40.982 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.982 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-02-17T13:07:40.982 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-02-17T13:07:40.982 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2024-02-17T13:07:40.982 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-02-17T13:07:40.982 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2024-02-17T13:07:40.982 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 0, 2024-02-17T13:07:40.982 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2024-02-17T13:07:40.982 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-02-17T13:07:40.983 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.983 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T13:07:40.983 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:40.983 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-02-17T13:07:40.983 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:40.983 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 815, 2024-02-17T13:07:40.983 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 112989582, 2024-02-17T13:07:40.983 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:40.984 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 815, 2024-02-17T13:07:40.984 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-02-17T13:07:40.984 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-02-17T13:07:40.984 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 815, 2024-02-17T13:07:40.984 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 112989582, 2024-02-17T13:07:40.984 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:40.984 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.984 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.985 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.985 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:40.985 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.985 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-02-17T13:07:40.985 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:40.985 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-02-17T13:07:40.985 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:40.985 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:40.986 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:40.986 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:40.986 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:40.986 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 741, 2024-02-17T13:07:40.986 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 112603693, 2024-02-17T13:07:40.986 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 741, 2024-02-17T13:07:40.986 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 112603693, 2024-02-17T13:07:40.986 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:40.986 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.987 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.987 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.987 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:40.987 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.987 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-02-17T13:07:40.987 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:40.987 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-02-17T13:07:40.987 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:40.988 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:40.988 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:40.988 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:40.988 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:40.988 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 741, 2024-02-17T13:07:40.988 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 741, 2024-02-17T13:07:40.988 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 741, 2024-02-17T13:07:40.988 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 741, 2024-02-17T13:07:40.989 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:40.989 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.989 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.989 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.989 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:40.989 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.989 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-02-17T13:07:40.989 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:40.990 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-02-17T13:07:40.990 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:40.990 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:40.990 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:40.990 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:40.990 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:40.990 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-02-17T13:07:40.990 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-02-17T13:07:40.991 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-02-17T13:07:40.991 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-02-17T13:07:40.991 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:40.991 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.991 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.991 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.991 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:40.991 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:40.991 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x555bc25de960": { 2024-02-17T13:07:40.992 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:40.992 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-02-17T13:07:40.992 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:40.992 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:40.992 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:40.992 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:40.992 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:40.992 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-02-17T13:07:40.993 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-02-17T13:07:40.993 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-02-17T13:07:40.993 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-02-17T13:07:40.993 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:40.993 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:40.993 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:40.993 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:40.993 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:40.994 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:40.994 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-02-17T13:07:41.640 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-02-17T13:07:41.640 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-02-17T13:07:41.640 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-02-17T13:07:41.641 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-02-17T13:07:41.641 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T13:07:41.641 DEBUG:tasks.check_counter:Getting stats from l 2024-02-17T13:07:41.641 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 -- ceph daemon mds.l perf dump 2024-02-17T13:07:41.880 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T13:07:42.263 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:42 smithi146 ceph-mon[36454]: pgmap v561: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.5 MiB/s wr, 168 op/s 2024-02-17T13:07:42.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:42 smithi018 ceph-mon[32625]: pgmap v561: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.5 MiB/s wr, 168 op/s 2024-02-17T13:07:42.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:42 smithi119 ceph-mon[36188]: pgmap v561: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.5 MiB/s wr, 168 op/s 2024-02-17T13:07:43.592 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-02-17T13:07:43.593 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T13:07:43.593 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 1169, 2024-02-17T13:07:43.593 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1171, 2024-02-17T13:07:43.593 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 92597321, 2024-02-17T13:07:43.594 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 363623, 2024-02-17T13:07:43.594 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 8, 2024-02-17T13:07:43.594 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 4, 2024-02-17T13:07:43.594 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.144197228, 2024-02-17T13:07:43.594 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.026925580, 2024-02-17T13:07:43.594 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.078312348, 2024-02-17T13:07:43.594 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.029928242, 2024-02-17T13:07:43.595 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:43.595 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1169, 2024-02-17T13:07:43.595 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.037112385, 2024-02-17T13:07:43.595 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000031747 2024-02-17T13:07:43.595 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.595 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:43.595 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.595 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.596 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.596 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.596 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T13:07:43.596 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T13:07:43.596 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.596 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T13:07:43.596 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 954, 2024-02-17T13:07:43.596 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1203, 2024-02-17T13:07:43.597 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 58970279, 2024-02-17T13:07:43.597 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 892365, 2024-02-17T13:07:43.597 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 8, 2024-02-17T13:07:43.597 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 6, 2024-02-17T13:07:43.597 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.112418351, 2024-02-17T13:07:43.597 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.031531533, 2024-02-17T13:07:43.597 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.047634945, 2024-02-17T13:07:43.598 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.024533947, 2024-02-17T13:07:43.598 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:43.598 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1200, 2024-02-17T13:07:43.598 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.041105685, 2024-02-17T13:07:43.598 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000034254 2024-02-17T13:07:43.598 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.598 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:43.598 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.598 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.599 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.599 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.599 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-17T13:07:43.599 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 596288 2024-02-17T13:07:43.599 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.599 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T13:07:43.600 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 2593, 2024-02-17T13:07:43.600 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 2524, 2024-02-17T13:07:43.600 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 90635940, 2024-02-17T13:07:43.600 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 876254, 2024-02-17T13:07:43.600 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 5, 2024-02-17T13:07:43.600 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 4, 2024-02-17T13:07:43.600 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.283480411, 2024-02-17T13:07:43.600 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.074487809, 2024-02-17T13:07:43.601 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.135274991, 2024-02-17T13:07:43.601 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.067195010, 2024-02-17T13:07:43.601 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T13:07:43.601 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2524, 2024-02-17T13:07:43.601 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.089432899, 2024-02-17T13:07:43.601 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000035433 2024-02-17T13:07:43.601 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.602 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-02-17T13:07:43.602 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.602 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.602 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.603 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.603 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 1993424, 2024-02-17T13:07:43.603 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 203872 2024-02-17T13:07:43.603 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.603 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-02-17T13:07:43.603 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-02-17T13:07:43.603 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-02-17T13:07:43.603 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.604 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-02-17T13:07:43.604 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-02-17T13:07:43.604 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-02-17T13:07:43.604 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 7224, 2024-02-17T13:07:43.604 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.164780745, 2024-02-17T13:07:43.604 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000022810 2024-02-17T13:07:43.604 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:43.605 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.605 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-02-17T13:07:43.605 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-02-17T13:07:43.605 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-02-17T13:07:43.605 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.605 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.606 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.606 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:43.606 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.606 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-02-17T13:07:43.606 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 0, 2024-02-17T13:07:43.606 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 0, 2024-02-17T13:07:43.606 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-02-17T13:07:43.606 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.606 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.607 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.607 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.607 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-02-17T13:07:43.607 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 0, 2024-02-17T13:07:43.608 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-02-17T13:07:43.608 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-02-17T13:07:43.608 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 0, 2024-02-17T13:07:43.608 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-02-17T13:07:43.608 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-02-17T13:07:43.608 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 16176, 2024-02-17T13:07:43.609 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 93, 2024-02-17T13:07:43.609 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 16083, 2024-02-17T13:07:43.609 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-02-17T13:07:43.609 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 16044, 2024-02-17T13:07:43.609 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2024-02-17T13:07:43.609 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 0, 2024-02-17T13:07:43.609 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 0, 2024-02-17T13:07:43.609 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 32, 2024-02-17T13:07:43.610 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 0, 2024-02-17T13:07:43.610 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 0, 2024-02-17T13:07:43.610 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2024-02-17T13:07:43.610 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 0, 2024-02-17T13:07:43.610 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-02-17T13:07:43.610 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-02-17T13:07:43.610 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-02-17T13:07:43.610 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-02-17T13:07:43.610 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-02-17T13:07:43.611 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-02-17T13:07:43.611 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-02-17T13:07:43.611 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2024-02-17T13:07:43.611 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2024-02-17T13:07:43.611 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-02-17T13:07:43.611 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2024-02-17T13:07:43.611 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-02-17T13:07:43.611 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 0, 2024-02-17T13:07:43.612 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 0, 2024-02-17T13:07:43.612 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-02-17T13:07:43.612 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2024-02-17T13:07:43.612 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-02-17T13:07:43.612 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T13:07:43.612 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-02-17T13:07:43.612 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-02-17T13:07:43.612 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T13:07:43.613 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T13:07:43.613 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2024-02-17T13:07:43.613 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-02-17T13:07:43.613 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 0, 2024-02-17T13:07:43.613 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2024-02-17T13:07:43.613 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T13:07:43.613 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-17T13:07:43.613 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T13:07:43.614 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 0, 2024-02-17T13:07:43.614 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2024-02-17T13:07:43.614 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2024-02-17T13:07:43.614 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 0 2024-02-17T13:07:43.614 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.614 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-02-17T13:07:43.614 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-02-17T13:07:43.614 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-02-17T13:07:43.615 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-02-17T13:07:43.615 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 0, 2024-02-17T13:07:43.615 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2024-02-17T13:07:43.615 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-02-17T13:07:43.615 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-02-17T13:07:43.615 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-02-17T13:07:43.615 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2024-02-17T13:07:43.615 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2024-02-17T13:07:43.615 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-02-17T13:07:43.616 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-02-17T13:07:43.616 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-02-17T13:07:43.616 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-02-17T13:07:43.616 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-02-17T13:07:43.616 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-02-17T13:07:43.616 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-02-17T13:07:43.616 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T13:07:43.617 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-02-17T13:07:43.617 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-02-17T13:07:43.617 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-02-17T13:07:43.617 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-02-17T13:07:43.617 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-02-17T13:07:43.617 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.617 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-02-17T13:07:43.618 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-02-17T13:07:43.618 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 0, 2024-02-17T13:07:43.618 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-02-17T13:07:43.618 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-02-17T13:07:43.618 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 88408, 2024-02-17T13:07:43.618 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-02-17T13:07:43.618 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-02-17T13:07:43.618 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 0, 2024-02-17T13:07:43.619 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-02-17T13:07:43.619 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-02-17T13:07:43.619 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 128, 2024-02-17T13:07:43.619 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 9, 2024-02-17T13:07:43.619 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-02-17T13:07:43.619 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-02-17T13:07:43.619 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-02-17T13:07:43.619 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 243184800, 2024-02-17T13:07:43.620 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 243179595, 2024-02-17T13:07:43.620 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-02-17T13:07:43.620 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.620 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.620 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.620 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.620 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 88408 2024-02-17T13:07:43.621 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.621 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-02-17T13:07:43.621 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 903, 2024-02-17T13:07:43.621 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 16001, 2024-02-17T13:07:43.621 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 15098, 2024-02-17T13:07:43.622 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 74, 2024-02-17T13:07:43.622 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 86, 2024-02-17T13:07:43.622 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 12, 2024-02-17T13:07:43.622 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 16218, 2024-02-17T13:07:43.622 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 16266, 2024-02-17T13:07:43.622 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 48, 2024-02-17T13:07:43.622 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 0, 2024-02-17T13:07:43.622 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 0, 2024-02-17T13:07:43.623 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2024-02-17T13:07:43.623 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 69188, 2024-02-17T13:07:43.623 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 215420 2024-02-17T13:07:43.623 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.623 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-02-17T13:07:43.623 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 0, 2024-02-17T13:07:43.623 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2024-02-17T13:07:43.623 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 0, 2024-02-17T13:07:43.624 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 0, 2024-02-17T13:07:43.624 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 0, 2024-02-17T13:07:43.624 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-02-17T13:07:43.624 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.624 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.624 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.624 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.625 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-02-17T13:07:43.625 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.625 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.625 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.625 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.625 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-02-17T13:07:43.625 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.625 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.626 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.626 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.626 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-02-17T13:07:43.626 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.626 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.626 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.626 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.626 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-02-17T13:07:43.627 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.627 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.627 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.627 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.627 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-02-17T13:07:43.627 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.627 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.627 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.628 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.628 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-02-17T13:07:43.628 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.628 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.628 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.628 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.628 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-02-17T13:07:43.628 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.629 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.629 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.629 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.629 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-02-17T13:07:43.629 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.629 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.629 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.629 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.630 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-02-17T13:07:43.630 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.630 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.630 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.630 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.630 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-02-17T13:07:43.630 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.630 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.631 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.631 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.631 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-02-17T13:07:43.631 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.632 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.632 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.632 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.632 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-02-17T13:07:43.632 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.633 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.633 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.633 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.633 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-02-17T13:07:43.633 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.633 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.633 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.633 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.634 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-02-17T13:07:43.634 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.634 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.634 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.634 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.634 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-02-17T13:07:43.634 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.634 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.635 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.635 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.635 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-02-17T13:07:43.635 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.635 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.635 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.635 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.635 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-02-17T13:07:43.635 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.636 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.636 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.636 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.636 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-02-17T13:07:43.636 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.636 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.636 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.636 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.637 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-02-17T13:07:43.637 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.637 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.637 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.637 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.637 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-02-17T13:07:43.637 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.637 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.638 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.638 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.638 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-02-17T13:07:43.638 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.638 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.638 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.638 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.638 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-02-17T13:07:43.638 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.639 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.639 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.639 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.639 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-02-17T13:07:43.639 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.639 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.639 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.639 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.640 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-02-17T13:07:43.640 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.640 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.640 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.640 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.640 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-02-17T13:07:43.640 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.640 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.641 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.641 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.641 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-02-17T13:07:43.641 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.641 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.641 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.641 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.641 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-02-17T13:07:43.642 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.642 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.642 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.642 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.642 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-02-17T13:07:43.642 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.642 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.642 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.643 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.643 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-02-17T13:07:43.643 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-02-17T13:07:43.643 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-02-17T13:07:43.643 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.643 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.643 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.643 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:43.644 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.644 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-02-17T13:07:43.644 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-02-17T13:07:43.644 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 3, 2024-02-17T13:07:43.644 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 2, 2024-02-17T13:07:43.644 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 2, 2024-02-17T13:07:43.644 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-02-17T13:07:43.644 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 0, 2024-02-17T13:07:43.645 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 0, 2024-02-17T13:07:43.645 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 3745, 2024-02-17T13:07:43.645 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-02-17T13:07:43.645 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.645 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-02-17T13:07:43.645 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-02-17T13:07:43.645 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-02-17T13:07:43.645 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T13:07:43.646 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-02-17T13:07:43.646 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T13:07:43.646 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-02-17T13:07:43.646 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T13:07:43.646 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T13:07:43.646 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T13:07:43.646 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T13:07:43.647 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T13:07:43.647 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-02-17T13:07:43.647 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T13:07:43.647 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T13:07:43.647 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-02-17T13:07:43.647 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-02-17T13:07:43.647 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-02-17T13:07:43.647 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-02-17T13:07:43.648 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T13:07:43.648 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T13:07:43.648 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T13:07:43.648 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T13:07:43.648 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T13:07:43.648 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-02-17T13:07:43.648 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-02-17T13:07:43.648 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-02-17T13:07:43.649 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T13:07:43.649 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T13:07:43.649 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-02-17T13:07:43.649 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-02-17T13:07:43.649 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-02-17T13:07:43.650 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-02-17T13:07:43.650 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T13:07:43.650 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-02-17T13:07:43.650 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T13:07:43.650 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-02-17T13:07:43.650 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 267443, 2024-02-17T13:07:43.651 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 1031, 2024-02-17T13:07:43.651 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-02-17T13:07:43.651 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-02-17T13:07:43.651 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-02-17T13:07:43.651 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-02-17T13:07:43.652 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-02-17T13:07:43.652 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-02-17T13:07:43.652 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-02-17T13:07:43.652 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-02-17T13:07:43.653 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 7936, 2024-02-17T13:07:43.653 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 165, 2024-02-17T13:07:43.653 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T13:07:43.653 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-02-17T13:07:43.653 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-02-17T13:07:43.653 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-02-17T13:07:43.653 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 13650707, 2024-02-17T13:07:43.653 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 93383, 2024-02-17T13:07:43.654 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-02-17T13:07:43.654 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-02-17T13:07:43.654 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-02-17T13:07:43.654 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-02-17T13:07:43.654 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.654 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-02-17T13:07:43.654 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-02-17T13:07:43.654 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-02-17T13:07:43.655 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 4283, 2024-02-17T13:07:43.655 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 21, 2024-02-17T13:07:43.655 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 3, 2024-02-17T13:07:43.655 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 4283, 2024-02-17T13:07:43.655 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-02-17T13:07:43.655 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 4283, 2024-02-17T13:07:43.655 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 3.518942133, 2024-02-17T13:07:43.655 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000821606 2024-02-17T13:07:43.655 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.656 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-02-17T13:07:43.656 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-02-17T13:07:43.656 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 4283, 2024-02-17T13:07:43.656 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 4284 2024-02-17T13:07:43.656 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.656 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 4283, 2024-02-17T13:07:43.656 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 4283, 2024-02-17T13:07:43.656 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 0, 2024-02-17T13:07:43.657 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-02-17T13:07:43.657 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-02-17T13:07:43.657 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 2384, 2024-02-17T13:07:43.657 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 0, 2024-02-17T13:07:43.657 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 1897, 2024-02-17T13:07:43.657 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 0, 2024-02-17T13:07:43.657 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 0, 2024-02-17T13:07:43.657 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-02-17T13:07:43.658 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-02-17T13:07:43.658 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-02-17T13:07:43.658 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-02-17T13:07:43.658 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 0, 2024-02-17T13:07:43.658 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-02-17T13:07:43.658 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-02-17T13:07:43.658 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-02-17T13:07:43.658 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 1, 2024-02-17T13:07:43.658 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 0, 2024-02-17T13:07:43.659 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-02-17T13:07:43.659 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-02-17T13:07:43.659 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-02-17T13:07:43.659 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-02-17T13:07:43.659 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-02-17T13:07:43.659 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-02-17T13:07:43.659 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T13:07:43.659 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-02-17T13:07:43.660 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-02-17T13:07:43.660 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-02-17T13:07:43.660 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-02-17T13:07:43.660 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-02-17T13:07:43.660 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-02-17T13:07:43.660 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-02-17T13:07:43.660 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-02-17T13:07:43.660 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-02-17T13:07:43.661 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-02-17T13:07:43.661 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-02-17T13:07:43.661 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-02-17T13:07:43.661 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-02-17T13:07:43.661 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-02-17T13:07:43.662 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-02-17T13:07:43.662 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-02-17T13:07:43.662 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-02-17T13:07:43.662 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-02-17T13:07:43.662 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-02-17T13:07:43.662 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 113, 2024-02-17T13:07:43.662 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-02-17T13:07:43.663 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 27, 2024-02-17T13:07:43.663 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 10, 2024-02-17T13:07:43.663 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 10, 2024-02-17T13:07:43.663 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-02-17T13:07:43.663 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-02-17T13:07:43.663 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 0, 2024-02-17T13:07:43.663 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 2, 2024-02-17T13:07:43.663 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 0 2024-02-17T13:07:43.663 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.664 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-02-17T13:07:43.664 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 0, 2024-02-17T13:07:43.664 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-02-17T13:07:43.664 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-02-17T13:07:43.664 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-02-17T13:07:43.664 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.665 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-02-17T13:07:43.665 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-02-17T13:07:43.665 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.665 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-02-17T13:07:43.665 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-02-17T13:07:43.665 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2024-02-17T13:07:43.665 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-02-17T13:07:43.665 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2024-02-17T13:07:43.666 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 0, 2024-02-17T13:07:43.666 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2024-02-17T13:07:43.666 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-02-17T13:07:43.666 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.666 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T13:07:43.666 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:43.666 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-02-17T13:07:43.666 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:43.667 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 4712, 2024-02-17T13:07:43.667 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 241754986, 2024-02-17T13:07:43.667 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:43.667 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 4712, 2024-02-17T13:07:43.667 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-02-17T13:07:43.667 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-02-17T13:07:43.667 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 4712, 2024-02-17T13:07:43.667 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 241754986, 2024-02-17T13:07:43.667 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:43.668 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.668 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.668 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.668 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:43.668 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.668 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-02-17T13:07:43.668 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:43.668 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-02-17T13:07:43.669 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:43.669 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:43.669 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:43.669 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:43.669 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:43.670 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 4283, 2024-02-17T13:07:43.670 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 239023472, 2024-02-17T13:07:43.670 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 4283, 2024-02-17T13:07:43.670 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 239023472, 2024-02-17T13:07:43.670 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:43.670 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.670 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.671 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.671 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:43.671 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.671 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-02-17T13:07:43.671 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:43.671 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-02-17T13:07:43.671 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:43.671 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:43.672 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:43.672 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:43.672 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:43.672 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 4283, 2024-02-17T13:07:43.672 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 4283, 2024-02-17T13:07:43.672 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 4283, 2024-02-17T13:07:43.672 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 4283, 2024-02-17T13:07:43.672 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:43.673 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.673 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.673 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.673 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:43.673 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.673 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-02-17T13:07:43.673 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:43.673 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-02-17T13:07:43.673 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:43.674 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:43.674 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:43.674 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:43.674 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:43.674 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-02-17T13:07:43.674 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-02-17T13:07:43.674 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-02-17T13:07:43.674 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-02-17T13:07:43.675 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:43.675 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.675 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.675 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.675 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:43.675 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-02-17T13:07:43.675 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x559eab48a960": { 2024-02-17T13:07:43.675 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-02-17T13:07:43.675 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-02-17T13:07:43.676 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-02-17T13:07:43.676 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-02-17T13:07:43.676 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-02-17T13:07:43.676 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-02-17T13:07:43.676 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-02-17T13:07:43.676 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-02-17T13:07:43.676 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-02-17T13:07:43.676 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-02-17T13:07:43.677 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-02-17T13:07:43.677 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-02-17T13:07:43.677 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-02-17T13:07:43.677 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-02-17T13:07:43.677 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-02-17T13:07:43.677 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:43.677 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-02-17T13:07:43.677 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-02-17T13:07:44.387 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-02-17T13:07:44.387 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-02-17T13:07:44.387 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-02-17T13:07:44.388 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-02-17T13:07:44.388 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T13:07:44.388 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-17T13:07:44.400 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-17T13:07:44.400 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T13:07:44.400 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-17T13:07:44.420 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T13:07:44.420 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-17T13:07:44.479 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T13:07:44.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:44 smithi018 ceph-mon[32625]: pgmap v562: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-02-17T13:07:44.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:44 smithi119 ceph-mon[36188]: pgmap v562: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-02-17T13:07:44.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.645+0000 7f0e198f1640 1 -- 172.21.15.18:0/985717000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e14105380 msgr2=0x7f0e1410cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:44.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.645+0000 7f0e198f1640 1 --2- 172.21.15.18:0/985717000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e14105380 0x7f0e1410cf50 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009cd0 tx=0x7f0e0c02c0f0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 -- 172.21.15.18:0/985717000 shutdown_connections 2024-02-17T13:07:44.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 --2- 172.21.15.18:0/985717000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e14105380 0x7f0e1410cf50 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 --2- 172.21.15.18:0/985717000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e14100c60 0x7f0e14104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 --2- 172.21.15.18:0/985717000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e141002b0 0x7f0e14100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 -- 172.21.15.18:0/985717000 >> 172.21.15.18:0/985717000 conn(0x7f0e14077120 msgr2=0x7f0e14077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 -- 172.21.15.18:0/985717000 shutdown_connections 2024-02-17T13:07:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 -- 172.21.15.18:0/985717000 wait complete. 2024-02-17T13:07:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 Processor -- start 2024-02-17T13:07:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 -- start start 2024-02-17T13:07:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e141002b0 0x7f0e14196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:44.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e14105380 0x7f0e14196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e1419af80 0x7f0e1419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1419b970 con 0x7f0e1419af80 2024-02-17T13:07:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1419bae0 con 0x7f0e141002b0 2024-02-17T13:07:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.646+0000 7f0e198f1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1419bc50 con 0x7f0e14105380 2024-02-17T13:07:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e137fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e1419af80 0x7f0e1419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e137fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e1419af80 0x7f0e1419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54850/0 (socket says 172.21.15.18:54850) 2024-02-17T13:07:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e137fe640 1 -- 172.21.15.18:0/3436177761 learned_addr learned my addr 172.21.15.18:0/3436177761 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e12ffd640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e141002b0 0x7f0e14196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e137fe640 1 -- 172.21.15.18:0/3436177761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e14105380 msgr2=0x7f0e14196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e137fe640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e14105380 0x7f0e14196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e137fe640 1 -- 172.21.15.18:0/3436177761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e141002b0 msgr2=0x7f0e14196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e137fe640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e141002b0 0x7f0e14196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e137fe640 1 -- 172.21.15.18:0/3436177761 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c009900 con 0x7f0e1419af80 2024-02-17T13:07:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e137fe640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e1419af80 0x7f0e1419b430 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c004af0 tx=0x7f0e0c032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e188ef640 1 -- 172.21.15.18:0/3436177761 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0328b0 con 0x7f0e1419af80 2024-02-17T13:07:44.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e188ef640 1 -- 172.21.15.18:0/3436177761 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e0c003e50 con 0x7f0e1419af80 2024-02-17T13:07:44.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e198f1640 1 -- 172.21.15.18:0/3436177761 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e141a3230 con 0x7f0e1419af80 2024-02-17T13:07:44.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e188ef640 1 -- 172.21.15.18:0/3436177761 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c004740 con 0x7f0e1419af80 2024-02-17T13:07:44.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.647+0000 7f0e198f1640 1 -- 172.21.15.18:0/3436177761 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e141a3650 con 0x7f0e1419af80 2024-02-17T13:07:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.648+0000 7f0e198f1640 1 -- 172.21.15.18:0/3436177761 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e140683c0 con 0x7f0e1419af80 2024-02-17T13:07:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.649+0000 7f0e188ef640 1 -- 172.21.15.18:0/3436177761 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0e0c00a040 con 0x7f0e1419af80 2024-02-17T13:07:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.650+0000 7f0e188ef640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0e000a2230 0x7f0e000a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.650+0000 7f0e12ffd640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0e000a2230 0x7f0e000a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.650+0000 7f0e188ef640 1 -- 172.21.15.18:0/3436177761 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0e0c0f0c80 con 0x7f0e1419af80 2024-02-17T13:07:44.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.651+0000 7f0e12ffd640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0e000a2230 0x7f0e000a46f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009b60 tx=0x7f0e04005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:44.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.655+0000 7f0e188ef640 1 -- 172.21.15.18:0/3436177761 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0e0c0b9a40 con 0x7f0e1419af80 2024-02-17T13:07:44.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:44 smithi146 ceph-mon[36454]: pgmap v562: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-02-17T13:07:44.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.838+0000 7f0e198f1640 1 -- 172.21.15.18:0/3436177761 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e14100c60 con 0x7f0e1419af80 2024-02-17T13:07:44.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.840+0000 7f0e188ef640 1 -- 172.21.15.18:0/3436177761 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+23079 (secure 0 0 0) 0x7f0e0c0bd100 con 0x7f0e1419af80 2024-02-17T13:07:44.829 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:07:44.829 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":113,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T13:07:13.933088+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":112,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/1074404031":"2024-02-18T12:49:52.647628+0000","172.21.15.18:6801/3026633658":"2024-02-18T12:49:52.647628+0000","172.21.15.18:6800/3026633658":"2024-02-18T12:49:52.647628+0000","172.21.15.18:0/2513093443":"2024-02-18T12:49:52.647628+0000","172.21.15.18:0/1103049228":"2024-02-18T12:49:52.647628+0000","172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/3776356008":"2024-02-18T12:49:52.647628+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":30,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T13:07:44.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.842+0000 7f0e198f1640 1 -- 172.21.15.18:0/3436177761 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0e000a2230 msgr2=0x7f0e000a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:44.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.842+0000 7f0e198f1640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0e000a2230 0x7f0e000a46f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009b60 tx=0x7f0e04005f90 comp rx=0 tx=0).stop 2024-02-17T13:07:44.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.842+0000 7f0e198f1640 1 -- 172.21.15.18:0/3436177761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e1419af80 msgr2=0x7f0e1419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:44.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.842+0000 7f0e198f1640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e1419af80 0x7f0e1419b430 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c004af0 tx=0x7f0e0c032570 comp rx=0 tx=0).stop 2024-02-17T13:07:44.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.842+0000 7f0e198f1640 1 -- 172.21.15.18:0/3436177761 shutdown_connections 2024-02-17T13:07:44.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.842+0000 7f0e198f1640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f0e000a2230 0x7f0e000a46f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.842+0000 7f0e198f1640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e1419af80 0x7f0e1419b430 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.842+0000 7f0e198f1640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e14105380 0x7f0e14196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.842+0000 7f0e198f1640 1 --2- 172.21.15.18:0/3436177761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e141002b0 0x7f0e14196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.842+0000 7f0e198f1640 1 -- 172.21.15.18:0/3436177761 >> 172.21.15.18:0/3436177761 conn(0x7f0e14077120 msgr2=0x7f0e140fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:44.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.842+0000 7f0e198f1640 1 -- 172.21.15.18:0/3436177761 shutdown_connections 2024-02-17T13:07:44.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.842+0000 7f0e198f1640 1 -- 172.21.15.18:0/3436177761 wait complete. 2024-02-17T13:07:44.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T12:43:12.450140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T12:47:43.690543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T12:47:44.653229+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 30, 'snap_epoch': 112, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T13:07:44.843 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T13:07:44.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.960+0000 7f3ee89f4640 1 -- 172.21.15.18:0/297747289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ee41002a0 msgr2=0x7f3ee4105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:44.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.960+0000 7f3ee89f4640 1 --2- 172.21.15.18:0/297747289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ee41002a0 0x7f3ee4105dd0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009930 tx=0x7f3ecc02bc30 comp rx=0 tx=0).stop 2024-02-17T13:07:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.960+0000 7f3ee89f4640 1 -- 172.21.15.18:0/297747289 shutdown_connections 2024-02-17T13:07:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.960+0000 7f3ee89f4640 1 --2- 172.21.15.18:0/297747289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee4106500 0x7f3ee410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.960+0000 7f3ee89f4640 1 --2- 172.21.15.18:0/297747289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ee41002a0 0x7f3ee4105dd0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.960+0000 7f3ee89f4640 1 --2- 172.21.15.18:0/297747289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ee40ff8f0 0x7f3ee40ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.960+0000 7f3ee89f4640 1 -- 172.21.15.18:0/297747289 >> 172.21.15.18:0/297747289 conn(0x7f3ee4076640 msgr2=0x7f3ee40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.960+0000 7f3ee89f4640 1 -- 172.21.15.18:0/297747289 shutdown_connections 2024-02-17T13:07:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.960+0000 7f3ee89f4640 1 -- 172.21.15.18:0/297747289 wait complete. 2024-02-17T13:07:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee89f4640 1 Processor -- start 2024-02-17T13:07:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee89f4640 1 -- start start 2024-02-17T13:07:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee89f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ee40ff8f0 0x7f3ee419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee89f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee4106500 0x7f3ee419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee89f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ee4197670 0x7f3ee4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee89f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee4198140 con 0x7f3ee4197670 2024-02-17T13:07:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee89f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee41982b0 con 0x7f3ee4106500 2024-02-17T13:07:44.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee89f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee4198470 con 0x7f3ee40ff8f0 2024-02-17T13:07:44.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ee4197670 0x7f3ee4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:44.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee2d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ee4197670 0x7f3ee4197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54874/0 (socket says 172.21.15.18:54874) 2024-02-17T13:07:44.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee2d9d640 1 -- 172.21.15.18:0/4173970711 learned_addr learned my addr 172.21.15.18:0/4173970711 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:44.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee259c640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ee40ff8f0 0x7f3ee419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:44.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3edbfff640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee4106500 0x7f3ee419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee2d9d640 1 -- 172.21.15.18:0/4173970711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ee40ff8f0 msgr2=0x7f3ee419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee2d9d640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ee40ff8f0 0x7f3ee419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee2d9d640 1 -- 172.21.15.18:0/4173970711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee4106500 msgr2=0x7f3ee419daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.961+0000 7f3ee2d9d640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee4106500 0x7f3ee419daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.962+0000 7f3ee2d9d640 1 -- 172.21.15.18:0/4173970711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc009560 con 0x7f3ee4197670 2024-02-17T13:07:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.962+0000 7f3edbfff640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee4106500 0x7f3ee419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:07:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.962+0000 7f3ee259c640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ee40ff8f0 0x7f3ee419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:07:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.962+0000 7f3ee2d9d640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ee4197670 0x7f3ee4197af0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00cce0 tx=0x7f3edc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.962+0000 7f3edb7fe640 1 -- 172.21.15.18:0/4173970711 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00f450 con 0x7f3ee4197670 2024-02-17T13:07:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.962+0000 7f3edb7fe640 1 -- 172.21.15.18:0/4173970711 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3edc00fa90 con 0x7f3ee4197670 2024-02-17T13:07:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.962+0000 7f3ee89f4640 1 -- 172.21.15.18:0/4173970711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee41a4410 con 0x7f3ee4197670 2024-02-17T13:07:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.962+0000 7f3edb7fe640 1 -- 172.21.15.18:0/4173970711 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc018b90 con 0x7f3ee4197670 2024-02-17T13:07:44.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.962+0000 7f3ee89f4640 1 -- 172.21.15.18:0/4173970711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee41a48a0 con 0x7f3ee4197670 2024-02-17T13:07:44.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.963+0000 7f3ee89f4640 1 -- 172.21.15.18:0/4173970711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee41029a0 con 0x7f3ee4197670 2024-02-17T13:07:44.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.964+0000 7f3edb7fe640 1 -- 172.21.15.18:0/4173970711 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3edc00fc00 con 0x7f3ee4197670 2024-02-17T13:07:44.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.965+0000 7f3edb7fe640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ed00a2230 0x7f3ed00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:44.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.965+0000 7f3ee259c640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ed00a2230 0x7f3ed00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:44.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.965+0000 7f3edb7fe640 1 -- 172.21.15.18:0/4173970711 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3edc0ce8f0 con 0x7f3ee4197670 2024-02-17T13:07:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.965+0000 7f3ee259c640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ed00a2230 0x7f3ed00a46f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40058c0 tx=0x7f3ed4005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:44.969+0000 7f3edb7fe640 1 -- 172.21.15.18:0/4173970711 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3edc097540 con 0x7f3ee4197670 2024-02-17T13:07:45.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.155+0000 7f3ee89f4640 1 -- 172.21.15.18:0/4173970711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ee41096b0 con 0x7f3ee4197670 2024-02-17T13:07:45.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.155+0000 7f3edb7fe640 1 -- 172.21.15.18:0/4173970711 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3edc09ac00 con 0x7f3ee4197670 2024-02-17T13:07:45.144 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-02-17T13:07:45.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.157+0000 7f3ee89f4640 1 -- 172.21.15.18:0/4173970711 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ed00a2230 msgr2=0x7f3ed00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.157+0000 7f3ee89f4640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ed00a2230 0x7f3ed00a46f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40058c0 tx=0x7f3ed4005850 comp rx=0 tx=0).stop 2024-02-17T13:07:45.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.157+0000 7f3ee89f4640 1 -- 172.21.15.18:0/4173970711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ee4197670 msgr2=0x7f3ee4197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.157+0000 7f3ee89f4640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ee4197670 0x7f3ee4197af0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00cce0 tx=0x7f3edc00a620 comp rx=0 tx=0).stop 2024-02-17T13:07:45.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.157+0000 7f3ee89f4640 1 -- 172.21.15.18:0/4173970711 shutdown_connections 2024-02-17T13:07:45.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.157+0000 7f3ee89f4640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f3ed00a2230 0x7f3ed00a46f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.157+0000 7f3ee89f4640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ee4197670 0x7f3ee4197af0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.157+0000 7f3ee89f4640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee4106500 0x7f3ee419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.157+0000 7f3ee89f4640 1 --2- 172.21.15.18:0/4173970711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ee40ff8f0 0x7f3ee419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.157+0000 7f3ee89f4640 1 -- 172.21.15.18:0/4173970711 >> 172.21.15.18:0/4173970711 conn(0x7f3ee4076640 msgr2=0x7f3ee4104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:45.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.157+0000 7f3ee89f4640 1 -- 172.21.15.18:0/4173970711 shutdown_connections 2024-02-17T13:07:45.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.157+0000 7f3ee89f4640 1 -- 172.21.15.18:0/4173970711 wait complete. 2024-02-17T13:07:45.158 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T13:07:45.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.276+0000 7f6a854df640 1 -- 172.21.15.18:0/2048566703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800685f0 msgr2=0x7f6a800689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.276+0000 7f6a854df640 1 --2- 172.21.15.18:0/2048566703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800685f0 0x7f6a800689d0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009930 tx=0x7f6a7802bc50 comp rx=0 tx=0).stop 2024-02-17T13:07:45.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.276+0000 7f6a854df640 1 -- 172.21.15.18:0/2048566703 shutdown_connections 2024-02-17T13:07:45.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.276+0000 7f6a854df640 1 --2- 172.21.15.18:0/2048566703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a801064c0 0x7f6a8010e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.276+0000 7f6a854df640 1 --2- 172.21.15.18:0/2048566703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a80068fa0 0x7f6a80105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.276+0000 7f6a854df640 1 --2- 172.21.15.18:0/2048566703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800685f0 0x7f6a800689d0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.276+0000 7f6a854df640 1 -- 172.21.15.18:0/2048566703 >> 172.21.15.18:0/2048566703 conn(0x7f6a80077180 msgr2=0x7f6a80077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:45.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.276+0000 7f6a854df640 1 -- 172.21.15.18:0/2048566703 shutdown_connections 2024-02-17T13:07:45.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.276+0000 7f6a854df640 1 -- 172.21.15.18:0/2048566703 wait complete. 2024-02-17T13:07:45.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.277+0000 7f6a854df640 1 Processor -- start 2024-02-17T13:07:45.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.277+0000 7f6a854df640 1 -- start start 2024-02-17T13:07:45.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.277+0000 7f6a854df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800685f0 0x7f6a80100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:45.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.277+0000 7f6a854df640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a801064c0 0x7f6a80100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:45.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.277+0000 7f6a854df640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a80100ac0 0x7f6a801a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:45.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.277+0000 7f6a854df640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a801a5fc0 con 0x7f6a800685f0 2024-02-17T13:07:45.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.277+0000 7f6a854df640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a801a6130 con 0x7f6a80100ac0 2024-02-17T13:07:45.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.277+0000 7f6a854df640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a801a62f0 con 0x7f6a801064c0 2024-02-17T13:07:45.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.277+0000 7f6a7effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800685f0 0x7f6a80100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:45.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.277+0000 7f6a7effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800685f0 0x7f6a80100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54884/0 (socket says 172.21.15.18:54884) 2024-02-17T13:07:45.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.277+0000 7f6a7effd640 1 -- 172.21.15.18:0/315940589 learned_addr learned my addr 172.21.15.18:0/315940589 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:45.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a7f7fe640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a80100ac0 0x7f6a801a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:45.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a7e7fc640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a801064c0 0x7f6a80100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:45.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a7effd640 1 -- 172.21.15.18:0/315940589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a801064c0 msgr2=0x7f6a80100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a7effd640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a801064c0 0x7f6a80100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a7effd640 1 -- 172.21.15.18:0/315940589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a80100ac0 msgr2=0x7f6a801a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a7effd640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a80100ac0 0x7f6a801a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a7effd640 1 -- 172.21.15.18:0/315940589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a78009560 con 0x7f6a800685f0 2024-02-17T13:07:45.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a7e7fc640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a801064c0 0x7f6a80100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:07:45.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a7effd640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800685f0 0x7f6a80100040 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009230 tx=0x7f6a78004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:45.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a77fff640 1 -- 172.21.15.18:0/315940589 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78004440 con 0x7f6a800685f0 2024-02-17T13:07:45.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a77fff640 1 -- 172.21.15.18:0/315940589 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a780052d0 con 0x7f6a800685f0 2024-02-17T13:07:45.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a854df640 1 -- 172.21.15.18:0/315940589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a801a6570 con 0x7f6a800685f0 2024-02-17T13:07:45.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a77fff640 1 -- 172.21.15.18:0/315940589 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78003d40 con 0x7f6a800685f0 2024-02-17T13:07:45.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.278+0000 7f6a854df640 1 -- 172.21.15.18:0/315940589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a801a6a80 con 0x7f6a800685f0 2024-02-17T13:07:45.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.279+0000 7f6a854df640 1 -- 172.21.15.18:0/315940589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a80067f70 con 0x7f6a800685f0 2024-02-17T13:07:45.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.280+0000 7f6a77fff640 1 -- 172.21.15.18:0/315940589 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6a78005440 con 0x7f6a800685f0 2024-02-17T13:07:45.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.281+0000 7f6a77fff640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6a6c0a2260 0x7f6a6c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:45.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.281+0000 7f6a7e7fc640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6a6c0a2260 0x7f6a6c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:45.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.281+0000 7f6a77fff640 1 -- 172.21.15.18:0/315940589 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6a780f0160 con 0x7f6a800685f0 2024-02-17T13:07:45.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.281+0000 7f6a7e7fc640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6a6c0a2260 0x7f6a6c0a4720 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6a80101ad0 tx=0x7f6a68008040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:45.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.285+0000 7f6a77fff640 1 -- 172.21.15.18:0/315940589 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6a780b8ec0 con 0x7f6a800685f0 2024-02-17T13:07:45.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.466+0000 7f6a854df640 1 -- 172.21.15.18:0/315940589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a80109680 con 0x7f6a800685f0 2024-02-17T13:07:45.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.467+0000 7f6a77fff640 1 -- 172.21.15.18:0/315940589 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6a780bc580 con 0x7f6a800685f0 2024-02-17T13:07:45.456 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-02-17T13:07:45.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.469+0000 7f6a854df640 1 -- 172.21.15.18:0/315940589 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6a6c0a2260 msgr2=0x7f6a6c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.469+0000 7f6a854df640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6a6c0a2260 0x7f6a6c0a4720 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6a80101ad0 tx=0x7f6a68008040 comp rx=0 tx=0).stop 2024-02-17T13:07:45.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.469+0000 7f6a854df640 1 -- 172.21.15.18:0/315940589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800685f0 msgr2=0x7f6a80100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.469+0000 7f6a854df640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800685f0 0x7f6a80100040 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009230 tx=0x7f6a78004040 comp rx=0 tx=0).stop 2024-02-17T13:07:45.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.469+0000 7f6a854df640 1 -- 172.21.15.18:0/315940589 shutdown_connections 2024-02-17T13:07:45.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.469+0000 7f6a854df640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6a6c0a2260 0x7f6a6c0a4720 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.469+0000 7f6a854df640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a80100ac0 0x7f6a801a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.469+0000 7f6a854df640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a801064c0 0x7f6a80100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.469+0000 7f6a854df640 1 --2- 172.21.15.18:0/315940589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800685f0 0x7f6a80100040 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.469+0000 7f6a854df640 1 -- 172.21.15.18:0/315940589 >> 172.21.15.18:0/315940589 conn(0x7f6a80077180 msgr2=0x7f6a800fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:45.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.469+0000 7f6a854df640 1 -- 172.21.15.18:0/315940589 shutdown_connections 2024-02-17T13:07:45.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.469+0000 7f6a854df640 1 -- 172.21.15.18:0/315940589 wait complete. 2024-02-17T13:07:45.468 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:45 smithi018 ceph-mon[32625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:07:45.468 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:45 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3436177761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T13:07:45.468 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:45 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/4173970711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T13:07:45.470 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T13:07:45.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.586+0000 7fde548df640 1 -- 172.21.15.18:0/1189312547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde501053e0 msgr2=0x7fde5010cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.586+0000 7fde548df640 1 --2- 172.21.15.18:0/1189312547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde501053e0 0x7fde5010cfb0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fde48009cd0 tx=0x7fde4802c0f0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.586+0000 7fde548df640 1 -- 172.21.15.18:0/1189312547 shutdown_connections 2024-02-17T13:07:45.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.586+0000 7fde548df640 1 --2- 172.21.15.18:0/1189312547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde501053e0 0x7fde5010cfb0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.586+0000 7fde548df640 1 --2- 172.21.15.18:0/1189312547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde50100cc0 0x7fde50104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.586+0000 7fde548df640 1 --2- 172.21.15.18:0/1189312547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde50100310 0x7fde501006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.586+0000 7fde548df640 1 -- 172.21.15.18:0/1189312547 >> 172.21.15.18:0/1189312547 conn(0x7fde500771e0 msgr2=0x7fde500775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:45.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.586+0000 7fde548df640 1 -- 172.21.15.18:0/1189312547 shutdown_connections 2024-02-17T13:07:45.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.586+0000 7fde548df640 1 -- 172.21.15.18:0/1189312547 wait complete. 2024-02-17T13:07:45.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde548df640 1 Processor -- start 2024-02-17T13:07:45.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde548df640 1 -- start start 2024-02-17T13:07:45.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde548df640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde50100310 0x7fde5019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:45.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde548df640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde501053e0 0x7fde5019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:45.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde548df640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde50197600 0x7fde50197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:45.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde548df640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde50197ff0 con 0x7fde501053e0 2024-02-17T13:07:45.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde548df640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde50198160 con 0x7fde50100310 2024-02-17T13:07:45.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde548df640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde50198320 con 0x7fde50197600 2024-02-17T13:07:45.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde45d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde501053e0 0x7fde5019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:45.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde45d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde501053e0 0x7fde5019d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54892/0 (socket says 172.21.15.18:54892) 2024-02-17T13:07:45.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde45d9b640 1 -- 172.21.15.18:0/1525738557 learned_addr learned my addr 172.21.15.18:0/1525738557 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:45.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde4ed9d640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde50197600 0x7fde50197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:45.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde4e59c640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde50100310 0x7fde5019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:45.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde45d9b640 1 -- 172.21.15.18:0/1525738557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde50197600 msgr2=0x7fde50197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde45d9b640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde50197600 0x7fde50197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde45d9b640 1 -- 172.21.15.18:0/1525738557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde50100310 msgr2=0x7fde5019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde45d9b640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde50100310 0x7fde5019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.587+0000 7fde45d9b640 1 -- 172.21.15.18:0/1525738557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde48009900 con 0x7fde501053e0 2024-02-17T13:07:45.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.588+0000 7fde45d9b640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde501053e0 0x7fde5019d990 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fde4000d770 tx=0x7fde4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:45.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.588+0000 7fde47fff640 1 -- 172.21.15.18:0/1525738557 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40009c30 con 0x7fde501053e0 2024-02-17T13:07:45.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.588+0000 7fde548df640 1 -- 172.21.15.18:0/1525738557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde501a4310 con 0x7fde501053e0 2024-02-17T13:07:45.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.588+0000 7fde47fff640 1 -- 172.21.15.18:0/1525738557 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde4000de80 con 0x7fde501053e0 2024-02-17T13:07:45.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.588+0000 7fde47fff640 1 -- 172.21.15.18:0/1525738557 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4000f6b0 con 0x7fde501053e0 2024-02-17T13:07:45.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.588+0000 7fde548df640 1 -- 172.21.15.18:0/1525738557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde501a4870 con 0x7fde501053e0 2024-02-17T13:07:45.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.589+0000 7fde548df640 1 -- 172.21.15.18:0/1525738557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde500683c0 con 0x7fde501053e0 2024-02-17T13:07:45.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.590+0000 7fde47fff640 1 -- 172.21.15.18:0/1525738557 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fde40010460 con 0x7fde501053e0 2024-02-17T13:07:45.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.591+0000 7fde47fff640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fde3c0a2230 0x7fde3c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:45.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.591+0000 7fde4e59c640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fde3c0a2230 0x7fde3c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:45.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.591+0000 7fde47fff640 1 -- 172.21.15.18:0/1525738557 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fde400cdd50 con 0x7fde501053e0 2024-02-17T13:07:45.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.591+0000 7fde4e59c640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fde3c0a2230 0x7fde3c0a46f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fde38005fd0 tx=0x7fde3800b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:45.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.595+0000 7fde47fff640 1 -- 172.21.15.18:0/1525738557 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fde40096a50 con 0x7fde501053e0 2024-02-17T13:07:45.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:45 smithi119 ceph-mon[36188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:07:45.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:45 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3436177761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T13:07:45.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:45 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/4173970711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T13:07:45.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:45 smithi146 ceph-mon[36454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-17T13:07:45.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:45 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3436177761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T13:07:45.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:45 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/4173970711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T13:07:45.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.780+0000 7fde548df640 1 -- 172.21.15.18:0/1525738557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde50100cc0 con 0x7fde501053e0 2024-02-17T13:07:45.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.780+0000 7fde47fff640 1 -- 172.21.15.18:0/1525738557 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde4009a110 con 0x7fde501053e0 2024-02-17T13:07:45.770 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-02-17T13:07:45.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.782+0000 7fde548df640 1 -- 172.21.15.18:0/1525738557 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fde3c0a2230 msgr2=0x7fde3c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.782+0000 7fde548df640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fde3c0a2230 0x7fde3c0a46f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fde38005fd0 tx=0x7fde3800b390 comp rx=0 tx=0).stop 2024-02-17T13:07:45.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.782+0000 7fde548df640 1 -- 172.21.15.18:0/1525738557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde501053e0 msgr2=0x7fde5019d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.782+0000 7fde548df640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde501053e0 0x7fde5019d990 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fde4000d770 tx=0x7fde4000dc30 comp rx=0 tx=0).stop 2024-02-17T13:07:45.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.782+0000 7fde548df640 1 -- 172.21.15.18:0/1525738557 shutdown_connections 2024-02-17T13:07:45.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.782+0000 7fde548df640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fde3c0a2230 0x7fde3c0a46f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.782+0000 7fde548df640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde50197600 0x7fde50197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.782+0000 7fde548df640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde501053e0 0x7fde5019d990 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.782+0000 7fde548df640 1 --2- 172.21.15.18:0/1525738557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde50100310 0x7fde5019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.782+0000 7fde548df640 1 -- 172.21.15.18:0/1525738557 >> 172.21.15.18:0/1525738557 conn(0x7fde500771e0 msgr2=0x7fde50103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:45.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.782+0000 7fde548df640 1 -- 172.21.15.18:0/1525738557 shutdown_connections 2024-02-17T13:07:45.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.782+0000 7fde548df640 1 -- 172.21.15.18:0/1525738557 wait complete. 2024-02-17T13:07:45.782 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-17T13:07:45.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.900+0000 7fad09040640 1 -- 172.21.15.18:0/1717179117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad04100c10 msgr2=0x7fad04104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.900+0000 7fad09040640 1 --2- 172.21.15.18:0/1717179117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad04100c10 0x7fad04104c50 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7facf8009930 tx=0x7facf802bc30 comp rx=0 tx=0).stop 2024-02-17T13:07:45.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.900+0000 7fad09040640 1 -- 172.21.15.18:0/1717179117 shutdown_connections 2024-02-17T13:07:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.900+0000 7fad09040640 1 --2- 172.21.15.18:0/1717179117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad04105330 0x7fad0410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.900+0000 7fad09040640 1 --2- 172.21.15.18:0/1717179117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad04100c10 0x7fad04104c50 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.900+0000 7fad09040640 1 --2- 172.21.15.18:0/1717179117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad04100260 0x7fad04100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.900+0000 7fad09040640 1 -- 172.21.15.18:0/1717179117 >> 172.21.15.18:0/1717179117 conn(0x7fad04077180 msgr2=0x7fad04077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.900+0000 7fad09040640 1 -- 172.21.15.18:0/1717179117 shutdown_connections 2024-02-17T13:07:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.900+0000 7fad09040640 1 -- 172.21.15.18:0/1717179117 wait complete. 2024-02-17T13:07:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.900+0000 7fad09040640 1 Processor -- start 2024-02-17T13:07:45.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad09040640 1 -- start start 2024-02-17T13:07:45.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad09040640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad04100260 0x7fad04198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:45.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad09040640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad04105330 0x7fad04198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:45.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad09040640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad04199100 0x7fad041a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:45.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad09040640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad041a3d50 con 0x7fad04199100 2024-02-17T13:07:45.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad09040640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad041a3ec0 con 0x7fad04105330 2024-02-17T13:07:45.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad09040640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad041a4080 con 0x7fad04100260 2024-02-17T13:07:45.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad0359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad04199100 0x7fad041a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:45.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad0359e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad04199100 0x7fad041a36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54906/0 (socket says 172.21.15.18:54906) 2024-02-17T13:07:45.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad0359e640 1 -- 172.21.15.18:0/1359975264 learned_addr learned my addr 172.21.15.18:0/1359975264 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:45.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad0259c640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad04105330 0x7fad04198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:45.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad02d9d640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad04100260 0x7fad04198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:45.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad0359e640 1 -- 172.21.15.18:0/1359975264 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad04100260 msgr2=0x7fad04198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad0359e640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad04100260 0x7fad04198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad0359e640 1 -- 172.21.15.18:0/1359975264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad04105330 msgr2=0x7fad04198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:45.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad0359e640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad04105330 0x7fad04198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:45.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad0359e640 1 -- 172.21.15.18:0/1359975264 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8009560 con 0x7fad04199100 2024-02-17T13:07:45.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad0359e640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad04199100 0x7fad041a36b0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7facfc00e970 tx=0x7facfc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:45.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7facebfff640 1 -- 172.21.15.18:0/1359975264 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc009720 con 0x7fad04199100 2024-02-17T13:07:45.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.901+0000 7fad09040640 1 -- 172.21.15.18:0/1359975264 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad041a4360 con 0x7fad04199100 2024-02-17T13:07:45.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.902+0000 7facebfff640 1 -- 172.21.15.18:0/1359975264 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facfc005500 con 0x7fad04199100 2024-02-17T13:07:45.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.902+0000 7facebfff640 1 -- 172.21.15.18:0/1359975264 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc010430 con 0x7fad04199100 2024-02-17T13:07:45.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.902+0000 7fad09040640 1 -- 172.21.15.18:0/1359975264 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad041a4840 con 0x7fad04199100 2024-02-17T13:07:45.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.903+0000 7fad09040640 1 -- 172.21.15.18:0/1359975264 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad04067f70 con 0x7fad04199100 2024-02-17T13:07:45.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.904+0000 7facebfff640 1 -- 172.21.15.18:0/1359975264 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7facfc005000 con 0x7fad04199100 2024-02-17T13:07:45.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.904+0000 7facebfff640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7facec0a2230 0x7facec0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:45.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.904+0000 7facebfff640 1 -- 172.21.15.18:0/1359975264 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8484+0+0 (secure 0 0 0) 0x7facfc014070 con 0x7fad04199100 2024-02-17T13:07:45.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.904+0000 7fad02d9d640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7facec0a2230 0x7facec0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:45.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.905+0000 7fad02d9d640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7facec0a2230 0x7facec0a46f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7facf4005fd0 tx=0x7facf4009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:45.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:45.908+0000 7facebfff640 1 -- 172.21.15.18:0/1359975264 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7facfc097320 con 0x7fad04199100 2024-02-17T13:07:46.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.104+0000 7fad09040640 1 -- 172.21.15.18:0/1359975264 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fad04108580 con 0x7fad04199100 2024-02-17T13:07:46.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.105+0000 7facebfff640 1 -- 172.21.15.18:0/1359975264 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v113) v1 ==== 81+0+946 (secure 0 0 0) 0x7facfc09a9e0 con 0x7fad04199100 2024-02-17T13:07:46.094 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1074404031 2024-02-18T12:49:52.647628+0000 2024-02-17T13:07:46.094 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/3026633658 2024-02-18T12:49:52.647628+0000 2024-02-17T13:07:46.095 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/3026633658 2024-02-18T12:49:52.647628+0000 2024-02-17T13:07:46.095 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2513093443 2024-02-18T12:49:52.647628+0000 2024-02-17T13:07:46.095 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1103049228 2024-02-18T12:49:52.647628+0000 2024-02-17T13:07:46.095 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2724676594 2024-02-18T12:40:50.114081+0000 2024-02-17T13:07:46.095 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1333332933 2024-02-18T12:40:50.114081+0000 2024-02-17T13:07:46.095 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3776356008 2024-02-18T12:49:52.647628+0000 2024-02-17T13:07:46.096 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1092699235 2024-02-18T12:40:50.114081+0000 2024-02-17T13:07:46.096 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1745633107 2024-02-18T12:40:22.805777+0000 2024-02-17T13:07:46.096 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1543388552 2024-02-18T12:40:22.805777+0000 2024-02-17T13:07:46.096 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3602994927 2024-02-18T12:40:22.805777+0000 2024-02-17T13:07:46.096 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2150265194 2024-02-18T12:40:22.805777+0000 2024-02-17T13:07:46.096 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1086757713 2024-02-18T12:40:50.114081+0000 2024-02-17T13:07:46.096 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1086757713 2024-02-18T12:40:50.114081+0000 2024-02-17T13:07:46.097 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1543388552 2024-02-18T12:40:22.805777+0000 2024-02-17T13:07:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:listed 16 entries 2024-02-17T13:07:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.107+0000 7fad09040640 1 -- 172.21.15.18:0/1359975264 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7facec0a2230 msgr2=0x7facec0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.107+0000 7fad09040640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7facec0a2230 0x7facec0a46f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7facf4005fd0 tx=0x7facf4009500 comp rx=0 tx=0).stop 2024-02-17T13:07:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.107+0000 7fad09040640 1 -- 172.21.15.18:0/1359975264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad04199100 msgr2=0x7fad041a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.107+0000 7fad09040640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad04199100 0x7fad041a36b0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7facfc00e970 tx=0x7facfc00ed70 comp rx=0 tx=0).stop 2024-02-17T13:07:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.107+0000 7fad09040640 1 -- 172.21.15.18:0/1359975264 shutdown_connections 2024-02-17T13:07:46.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.107+0000 7fad09040640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7facec0a2230 0x7facec0a46f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:46.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.107+0000 7fad09040640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad04199100 0x7fad041a36b0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:46.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.107+0000 7fad09040640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad04105330 0x7fad04198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:46.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.107+0000 7fad09040640 1 --2- 172.21.15.18:0/1359975264 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad04100260 0x7fad04198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:46.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.107+0000 7fad09040640 1 -- 172.21.15.18:0/1359975264 >> 172.21.15.18:0/1359975264 conn(0x7fad04077180 msgr2=0x7fad040fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:46.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.107+0000 7fad09040640 1 -- 172.21.15.18:0/1359975264 shutdown_connections 2024-02-17T13:07:46.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:46.107+0000 7fad09040640 1 -- 172.21.15.18:0/1359975264 wait complete. 2024-02-17T13:07:46.107 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-17T13:07:46.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:07:46.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-17T13:07:46.459 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:46 smithi018 ceph-mon[32625]: pgmap v563: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.7 MiB/s wr, 95 op/s 2024-02-17T13:07:46.459 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:46 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/315940589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T13:07:46.459 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:46 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1525738557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T13:07:46.459 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:46 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1359975264' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-17T13:07:46.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:46 smithi119 ceph-mon[36188]: pgmap v563: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.7 MiB/s wr, 95 op/s 2024-02-17T13:07:46.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:46 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/315940589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T13:07:46.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:46 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1525738557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T13:07:46.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:46 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1359975264' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-17T13:07:46.660 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi018.front.sepia.ceph.com 2024-02-17T13:07:46.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:07:46.660 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-17T13:07:46.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:46 smithi146 ceph-mon[36454]: pgmap v563: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.7 MiB/s wr, 95 op/s 2024-02-17T13:07:46.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:46 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/315940589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T13:07:46.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:46 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1525738557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T13:07:46.697 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:46 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1359975264' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-17T13:07:46.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:07:46.705 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-02-17T13:07:46.784 INFO:teuthology.orchestra.run.smithi018.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-17T13:07:46.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:07:46.785 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-17T13:07:46.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:07:46.880 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link delete ceph-brx 2024-02-17T13:07:46.979 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-17T13:07:47.046 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-02-17T13:07:47.047 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-17T13:07:47.086 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20261 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/c4a6d263ef6e1b77f54c7f47e9c0777436c371420f77f52ee59b880196c7087b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PELTLG55I3GY2RCWNKRNEUJ54Z:/var/lib/containers/storage/overlay/l/VS5ZK5VME22ZSRMDRO6PS5KIY5,upperdir=/var/lib/containers/storage/overlay/c4a6d263ef6e1b77f54c7f47e9c0777436c371420f77f52ee59b880196c7087b/diff,workdir=/var/lib/containers/storage/overlay/c4a6d263ef6e1b77f54c7f47e9c0777436c371420f77f52ee59b880196c7087b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f38619cd11128ac8462ed74b4ca43492df2d1cc1c9e21ca15133998b72fa290a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PELTLG55I3GY2RCWNKRNEUJ54Z:/var/lib/containers/storage/overlay/l/VS5ZK5VME22ZSRMDRO6PS5KIY5,upperdir=/var/lib/containers/storage/overlay/f38619cd11128ac8462ed74b4ca43492df2d1cc1c9e21ca15133998b72fa290a/diff,workdir=/var/lib/containers/storage/overlay/f38619cd11128ac8462ed74b4ca43492df2d1cc1c9e21ca15133998b72fa290a/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/bb19a3065ddb79fc97e99b2a19b8c1f9b6b45b1424c1ba2a53cb05fbed4fef63/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PELTLG55I3GY2RCWNKRNEUJ54Z:/var/lib/containers/storage/overlay/l/VS5ZK5VME22ZSRMDRO6PS5KIY5,upperdir=/var/lib/containers/storage/overlay/bb19a3065ddb79fc97e99b2a19b8c1f9b6b45b1424c1ba2a53cb05fbed4fef63/diff,workdir=/var/lib/containers/storage/overlay/bb19a3065ddb79fc97e99b2a19b8c1f9b6b45b1424c1ba2a53cb05fbed4fef63/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/db17a5e3ca4c54a5a42dc3a2666a79eb2ea402d478c1df54a8e98d8484802dba/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PELTLG55I3GY2RCWNKRNEUJ54Z:/var/lib/containers/storage/overlay/l/VS5ZK5VME22ZSRMDRO6PS5KIY5,upperdir=/var/lib/containers/storage/overlay/db17a5e3ca4c54a5a42dc3a2666a79eb2ea402d478c1df54a8e98d8484802dba/diff,workdir=/var/lib/containers/storage/overlay/db17a5e3ca4c54a5a42dc3a2666a79eb2ea402d478c1df54a8e98d8484802dba/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b558dd80af95aac5220bf59f3d34b21aea02c2936cabc1af416057a494fdbd07/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PELTLG55I3GY2RCWNKRNEUJ54Z:/var/lib/containers/storage/overlay/l/VS5ZK5VME22ZSRMDRO6PS5KIY5,upperdir=/var/lib/containers/storage/overlay/b558dd80af95aac5220bf59f3d34b21aea02c2936cabc1af416057a494fdbd07/diff,workdir=/var/lib/containers/storage/overlay/b558dd80af95aac5220bf59f3d34b21aea02c2936cabc1af416057a494fdbd07/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0e0ace8071c5d6b499fcf91c72225bd77666d004798b188af23e82cbf622152d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/PELTLG55I3GY2RCWNKRNEUJ54Z:/var/lib/containers/storage/overlay/l/VS5ZK5VME22ZSRMDRO6PS5KIY5,upperdir=/var/lib/containers/storage/overlay/0e0ace8071c5d6b499fcf91c72225bd77666d004798b188af23e82cbf622152d/diff,workdir=/var/lib/containers/storage/overlay/0e0ace8071c5d6b499fcf91c72225bd77666d004798b188af23e82cbf622152d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/285dd869f6889c49b6849362a8dd719d66ffaf9915cae144156f9e9bdb255b2e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PELTLG55I3GY2RCWNKRNEUJ54Z:/var/lib/containers/storage/overlay/l/VS5ZK5VME22ZSRMDRO6PS5KIY5,upperdir=/var/lib/containers/storage/overlay/285dd869f6889c49b6849362a8dd719d66ffaf9915cae144156f9e9bdb255b2e/diff,workdir=/var/lib/containers/storage/overlay/285dd869f6889c49b6849362a8dd719d66ffaf9915cae144156f9e9bdb255b2e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7ac2cf82d3fa32cdc333a2a5ed5e72de1f7359d228792a560f5a871571492bc0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PELTLG55I3GY2RCWNKRNEUJ54Z:/var/lib/containers/storage/overlay/l/VS5ZK5VME22ZSRMDRO6PS5KIY5,upperdir=/var/lib/containers/storage/overlay/7ac2cf82d3fa32cdc333a2a5ed5e72de1f7359d228792a560f5a871571492bc0/diff,workdir=/var/lib/containers/storage/overlay/7ac2cf82d3fa32cdc333a2a5ed5e72de1f7359d228792a560f5a871571492bc0/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/39cdac1995f3cf5ee2e812528b4e64177993b662473c9009b8707c9c78566424/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PELTLG55I3GY2RCWNKRNEUJ54Z:/var/lib/containers/storage/overlay/l/VS5ZK5VME22ZSRMDRO6PS5KIY5,upperdir=/var/lib/containers/storage/overlay/39cdac1995f3cf5ee2e812528b4e64177993b662473c9009b8707c9c78566424/diff,workdir=/var/lib/containers/storage/overlay/39cdac1995f3cf5ee2e812528b4e64177993b662473c9009b8707c9c78566424/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/71106f7df6f55627fadf869791e82a8ce6df166091bbe342cd4335b68180515e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/PELTLG55I3GY2RCWNKRNEUJ54Z:/var/lib/containers/storage/overlay/l/VS5ZK5VME22ZSRMDRO6PS5KIY5,upperdir=/var/lib/containers/storage/overlay/71106f7df6f55627fadf869791e82a8ce6df166091bbe342cd4335b68180515e/diff,workdir=/var/lib/containers/storage/overlay/71106f7df6f55627fadf869791e82a8ce6df166091bbe342cd4335b68180515e/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 2024-02-17T13:07:47.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:07:47.087 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-02-17T13:07:47.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T13:07:47.150 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link delete ceph-brx 2024-02-17T13:07:47.239 INFO:teuthology.orchestra.run.smithi018.stderr:Cannot find device "ceph-brx" 2024-02-17T13:07:47.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T13:07:47.240 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-17T13:07:47.253 INFO:tasks.cephadm:Teardown begin 2024-02-17T13:07:47.253 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-17T13:07:47.253 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T13:07:47.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.386+0000 7f6f1556c640 1 -- 172.21.15.18:0/1171228751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f100733b0 msgr2=0x7f6f10073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.386+0000 7f6f1556c640 1 --2- 172.21.15.18:0/1171228751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f100733b0 0x7f6f10073790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009860 tx=0x7f6f0002bc00 comp rx=0 tx=0).stop 2024-02-17T13:07:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.386+0000 7f6f1556c640 1 -- 172.21.15.18:0/1171228751 shutdown_connections 2024-02-17T13:07:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.386+0000 7f6f1556c640 1 --2- 172.21.15.18:0/1171228751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f1010ddc0 0x7f6f101101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.386+0000 7f6f1556c640 1 --2- 172.21.15.18:0/1171228751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f10073cd0 0x7f6f1010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.386+0000 7f6f1556c640 1 --2- 172.21.15.18:0/1171228751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f100733b0 0x7f6f10073790 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.386+0000 7f6f1556c640 1 -- 172.21.15.18:0/1171228751 >> 172.21.15.18:0/1171228751 conn(0x7f6f10079170 msgr2=0x7f6f10079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:47.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.389+0000 7f6f1556c640 1 -- 172.21.15.18:0/1171228751 shutdown_connections 2024-02-17T13:07:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.389+0000 7f6f1556c640 1 -- 172.21.15.18:0/1171228751 wait complete. 2024-02-17T13:07:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.389+0000 7f6f1556c640 1 Processor -- start 2024-02-17T13:07:47.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f1556c640 1 -- start start 2024-02-17T13:07:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f1556c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f100733b0 0x7f6f1019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f1556c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f10073cd0 0x7f6f1019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f1556c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f1010ddc0 0x7f6f10199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f1556c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f10199fa0 con 0x7f6f10073cd0 2024-02-17T13:07:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f1556c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1019a110 con 0x7f6f100733b0 2024-02-17T13:07:47.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f1556c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1019a2d0 con 0x7f6f1010ddc0 2024-02-17T13:07:47.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f07fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f10073cd0 0x7f6f1019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f100733b0 0x7f6f1019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f07fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f10073cd0 0x7f6f1019fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54930/0 (socket says 172.21.15.18:54930) 2024-02-17T13:07:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f07fff640 1 -- 172.21.15.18:0/3975791114 learned_addr learned my addr 172.21.15.18:0/3975791114 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f0f7fe640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f1010ddc0 0x7f6f10199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f07fff640 1 -- 172.21.15.18:0/3975791114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f1010ddc0 msgr2=0x7f6f10199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.390+0000 7f6f07fff640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f1010ddc0 0x7f6f10199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.391+0000 7f6f07fff640 1 -- 172.21.15.18:0/3975791114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f100733b0 msgr2=0x7f6f1019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:47.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.391+0000 7f6f07fff640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f100733b0 0x7f6f1019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.391+0000 7f6f07fff640 1 -- 172.21.15.18:0/3975791114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f00009490 con 0x7f6f10073cd0 2024-02-17T13:07:47.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.391+0000 7f6f07fff640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f10073cd0 0x7f6f1019fc90 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800e9c0 tx=0x7f6ef800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:47.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.391+0000 7f6f0cff9640 1 -- 172.21.15.18:0/3975791114 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800ccc0 con 0x7f6f10073cd0 2024-02-17T13:07:47.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.391+0000 7f6f1556c640 1 -- 172.21.15.18:0/3975791114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1019a5b0 con 0x7f6f10073cd0 2024-02-17T13:07:47.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.391+0000 7f6f0cff9640 1 -- 172.21.15.18:0/3975791114 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ef800ce20 con 0x7f6f10073cd0 2024-02-17T13:07:47.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.391+0000 7f6f0cff9640 1 -- 172.21.15.18:0/3975791114 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8010430 con 0x7f6f10073cd0 2024-02-17T13:07:47.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.391+0000 7f6f1556c640 1 -- 172.21.15.18:0/3975791114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f101a67f0 con 0x7f6f10073cd0 2024-02-17T13:07:47.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.393+0000 7f6f1556c640 1 -- 172.21.15.18:0/3975791114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f10067f70 con 0x7f6f10073cd0 2024-02-17T13:07:47.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.394+0000 7f6f0cff9640 1 -- 172.21.15.18:0/3975791114 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6ef8004f80 con 0x7f6f10073cd0 2024-02-17T13:07:47.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.395+0000 7f6f0cff9640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6efc0a2230 0x7f6efc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:47.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.395+0000 7f6f0cff9640 1 -- 172.21.15.18:0/3975791114 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6ef8014070 con 0x7f6f10073cd0 2024-02-17T13:07:47.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.396+0000 7f6f0effd640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6efc0a2230 0x7f6efc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:47.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.396+0000 7f6f0effd640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6efc0a2230 0x7f6efc0a46f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009830 tx=0x7f6f000056d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:47.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.403+0000 7f6f0cff9640 1 -- 172.21.15.18:0/3975791114 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6ef80970d0 con 0x7f6f10073cd0 2024-02-17T13:07:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.636+0000 7f6f1556c640 1 -- 172.21.15.18:0/3975791114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f10108f40 con 0x7f6f10073cd0 2024-02-17T13:07:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.637+0000 7f6f0cff9640 1 -- 172.21.15.18:0/3975791114 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+11400 (secure 0 0 0) 0x7f6ef809a790 con 0x7f6f10073cd0 2024-02-17T13:07:47.626 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:07:47.626 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":82,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T13:07:43.984288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":240,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_29495":{"gid":29495,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3761238301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3761238301},{"type":"v1","addr":"172.21.15.146:6839","nonce":3761238301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T13:07:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 82 2024-02-17T13:07:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.639+0000 7f6f1556c640 1 -- 172.21.15.18:0/3975791114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6efc0a2230 msgr2=0x7f6efc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.639+0000 7f6f1556c640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6efc0a2230 0x7f6efc0a46f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009830 tx=0x7f6f000056d0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.639+0000 7f6f1556c640 1 -- 172.21.15.18:0/3975791114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f10073cd0 msgr2=0x7f6f1019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.639+0000 7f6f1556c640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f10073cd0 0x7f6f1019fc90 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800e9c0 tx=0x7f6ef800ee80 comp rx=0 tx=0).stop 2024-02-17T13:07:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.639+0000 7f6f1556c640 1 -- 172.21.15.18:0/3975791114 shutdown_connections 2024-02-17T13:07:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.639+0000 7f6f1556c640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f6efc0a2230 0x7f6efc0a46f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.639+0000 7f6f1556c640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f1010ddc0 0x7f6f10199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.639+0000 7f6f1556c640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f10073cd0 0x7f6f1019fc90 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.639+0000 7f6f1556c640 1 --2- 172.21.15.18:0/3975791114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f100733b0 0x7f6f1019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.639+0000 7f6f1556c640 1 -- 172.21.15.18:0/3975791114 >> 172.21.15.18:0/3975791114 conn(0x7f6f10079170 msgr2=0x7f6f100ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.639+0000 7f6f1556c640 1 -- 172.21.15.18:0/3975791114 shutdown_connections 2024-02-17T13:07:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.639+0000 7f6f1556c640 1 -- 172.21.15.18:0/3975791114 wait complete. 2024-02-17T13:07:47.639 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T13:07:47.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.753+0000 7f44f72f4640 1 -- 172.21.15.18:0/3255678987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0100080 msgr2=0x7f44f01004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:47.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.753+0000 7f44f72f4640 1 --2- 172.21.15.18:0/3255678987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0100080 0x7f44f01004e0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009900 tx=0x7f44e402bcd0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 -- 172.21.15.18:0/3255678987 shutdown_connections 2024-02-17T13:07:47.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 --2- 172.21.15.18:0/3255678987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0100a20 0x7f44f010cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 --2- 172.21.15.18:0/3255678987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0100080 0x7f44f01004e0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 --2- 172.21.15.18:0/3255678987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f0106140 0x7f44f0106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 -- 172.21.15.18:0/3255678987 >> 172.21.15.18:0/3255678987 conn(0x7f44f00771b0 msgr2=0x7f44f00775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:47.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 -- 172.21.15.18:0/3255678987 shutdown_connections 2024-02-17T13:07:47.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 -- 172.21.15.18:0/3255678987 wait complete. 2024-02-17T13:07:47.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 Processor -- start 2024-02-17T13:07:47.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 -- start start 2024-02-17T13:07:47.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f0100080 0x7f44f0101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:47.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0100a20 0x7f44f0102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:47.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0106140 0x7f44f01027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:47.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f01a5b40 con 0x7f44f0100a20 2024-02-17T13:07:47.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f01a5cb0 con 0x7f44f0106140 2024-02-17T13:07:47.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f72f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f01a5e70 con 0x7f44f0100080 2024-02-17T13:07:47.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.754+0000 7f44f488f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0100a20 0x7f44f0102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:47.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f488f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0100a20 0x7f44f0102270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54962/0 (socket says 172.21.15.18:54962) 2024-02-17T13:07:47.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f488f640 1 -- 172.21.15.18:0/1436491385 learned_addr learned my addr 172.21.15.18:0/1436491385 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:47.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f5891640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0106140 0x7f44f01027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:47.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f5090640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f0100080 0x7f44f0101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:47.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f488f640 1 -- 172.21.15.18:0/1436491385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f0100080 msgr2=0x7f44f0101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:47.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f488f640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f0100080 0x7f44f0101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f488f640 1 -- 172.21.15.18:0/1436491385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0106140 msgr2=0x7f44f01027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:47.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f488f640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0106140 0x7f44f01027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f488f640 1 -- 172.21.15.18:0/1436491385 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4009560 con 0x7f44f0100a20 2024-02-17T13:07:47.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f5891640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0106140 0x7f44f01027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:07:47.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f488f640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0100a20 0x7f44f0102270 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f44e4005940 tx=0x7f44e4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:47.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44e27fc640 1 -- 172.21.15.18:0/1436491385 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4004440 con 0x7f44f0100a20 2024-02-17T13:07:47.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44e27fc640 1 -- 172.21.15.18:0/1436491385 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44e40052d0 con 0x7f44f0100a20 2024-02-17T13:07:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f72f4640 1 -- 172.21.15.18:0/1436491385 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f01a60f0 con 0x7f44f0100a20 2024-02-17T13:07:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44e27fc640 1 -- 172.21.15.18:0/1436491385 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4003da0 con 0x7f44f0100a20 2024-02-17T13:07:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.755+0000 7f44f72f4640 1 -- 172.21.15.18:0/1436491385 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f01a6600 con 0x7f44f0100a20 2024-02-17T13:07:47.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.756+0000 7f44f72f4640 1 -- 172.21.15.18:0/1436491385 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f00676e0 con 0x7f44f0100a20 2024-02-17T13:07:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.757+0000 7f44e27fc640 1 -- 172.21.15.18:0/1436491385 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f44e4005440 con 0x7f44f0100a20 2024-02-17T13:07:47.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.758+0000 7f44e27fc640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44dc0a2230 0x7f44dc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.758+0000 7f44f5090640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44dc0a2230 0x7f44dc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.758+0000 7f44e27fc640 1 -- 172.21.15.18:0/1436491385 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f44e40f0f40 con 0x7f44f0100a20 2024-02-17T13:07:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.758+0000 7f44f5090640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44dc0a2230 0x7f44dc0a46f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f44ec005fd0 tx=0x7f44ec00b510 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.762+0000 7f44e27fc640 1 -- 172.21.15.18:0/1436491385 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f44e40b9d00 con 0x7f44f0100a20 2024-02-17T13:07:47.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.982+0000 7f44f72f4640 1 -- 172.21.15.18:0/1436491385 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44f01101a0 con 0x7f44f0100a20 2024-02-17T13:07:47.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.983+0000 7f44e27fc640 1 -- 172.21.15.18:0/1436491385 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+11400 (secure 0 0 0) 0x7f44e40bd3c0 con 0x7f44f0100a20 2024-02-17T13:07:47.972 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:07:47.973 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3401600921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3401600921},{"type":"v1","addr":"172.21.15.18:6835","nonce":3401600921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/3018278996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":3018278996},{"type":"v1","addr":"172.21.15.18:6837","nonce":3018278996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":82,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T12:47:45.407920+0000","modified":"2024-02-17T13:07:43.984288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24455,"mds_3":24448,"mds_4":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6839/1294470347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":1294470347},{"type":"v1","addr":"172.21.15.18:6839","nonce":1294470347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6841/2569825427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2569825427},{"type":"v1","addr":"172.21.15.18:6841","nonce":2569825427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4241790141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4241790141},{"type":"v1","addr":"172.21.15.146:6833","nonce":4241790141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.119:6833/21392902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":21392902},{"type":"v1","addr":"172.21.15.119:6833","nonce":21392902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6835/405923223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":405923223},{"type":"v1","addr":"172.21.15.146:6835","nonce":405923223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.119:6835/2304139172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2304139172},{"type":"v1","addr":"172.21.15.119:6835","nonce":2304139172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6837/1732085011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1732085011},{"type":"v1","addr":"172.21.15.146:6837","nonce":1732085011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":240,"addr":"172.21.15.119:6837/248613392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":248613392},{"type":"v1","addr":"172.21.15.119:6837","nonce":248613392}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6839/2236251663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":2236251663},{"type":"v1","addr":"172.21.15.119:6839","nonce":2236251663}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_29495":{"gid":29495,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6839/3761238301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3761238301},{"type":"v1","addr":"172.21.15.146:6839","nonce":3761238301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T13:07:47.973 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 82 2024-02-17T13:07:47.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.985+0000 7f44f72f4640 1 -- 172.21.15.18:0/1436491385 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44dc0a2230 msgr2=0x7f44dc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:47.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.985+0000 7f44f72f4640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44dc0a2230 0x7f44dc0a46f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f44ec005fd0 tx=0x7f44ec00b510 comp rx=0 tx=0).stop 2024-02-17T13:07:47.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.985+0000 7f44f72f4640 1 -- 172.21.15.18:0/1436491385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0100a20 msgr2=0x7f44f0102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:47.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.985+0000 7f44f72f4640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0100a20 0x7f44f0102270 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f44e4005940 tx=0x7f44e4004040 comp rx=0 tx=0).stop 2024-02-17T13:07:47.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.985+0000 7f44f72f4640 1 -- 172.21.15.18:0/1436491385 shutdown_connections 2024-02-17T13:07:47.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.985+0000 7f44f72f4640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f44dc0a2230 0x7f44dc0a46f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.985+0000 7f44f72f4640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f0106140 0x7f44f01027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.985+0000 7f44f72f4640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0100a20 0x7f44f0102270 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.985+0000 7f44f72f4640 1 --2- 172.21.15.18:0/1436491385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44f0100080 0x7f44f0101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:47.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.985+0000 7f44f72f4640 1 -- 172.21.15.18:0/1436491385 >> 172.21.15.18:0/1436491385 conn(0x7f44f00771b0 msgr2=0x7f44f00ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:47.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.985+0000 7f44f72f4640 1 -- 172.21.15.18:0/1436491385 shutdown_connections 2024-02-17T13:07:47.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:47.985+0000 7f44f72f4640 1 -- 172.21.15.18:0/1436491385 wait complete. 2024-02-17T13:07:47.985 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T13:07:48.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 -- 172.21.15.18:0/3981269201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02101002f0 msgr2=0x7f02101006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 --2- 172.21.15.18:0/3981269201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02101002f0 0x7f02101006d0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f0204009930 tx=0x7f020402bc30 comp rx=0 tx=0).stop 2024-02-17T13:07:48.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 -- 172.21.15.18:0/3981269201 shutdown_connections 2024-02-17T13:07:48.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 --2- 172.21.15.18:0/3981269201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02101053c0 0x7f021010cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 --2- 172.21.15.18:0/3981269201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0210100ca0 0x7f0210104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 --2- 172.21.15.18:0/3981269201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02101002f0 0x7f02101006d0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 -- 172.21.15.18:0/3981269201 >> 172.21.15.18:0/3981269201 conn(0x7f0210077180 msgr2=0x7f0210077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:48.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 -- 172.21.15.18:0/3981269201 shutdown_connections 2024-02-17T13:07:48.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 -- 172.21.15.18:0/3981269201 wait complete. 2024-02-17T13:07:48.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 Processor -- start 2024-02-17T13:07:48.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 -- start start 2024-02-17T13:07:48.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02101002f0 0x7f021019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:48.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.099+0000 7f021701c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02101053c0 0x7f021019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:48.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f021701c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0210197730 0x7f0210197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:48.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f021701c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0210198200 con 0x7f0210197730 2024-02-17T13:07:48.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f021701c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0210198370 con 0x7f02101002f0 2024-02-17T13:07:48.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f021701c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0210198530 con 0x7f02101053c0 2024-02-17T13:07:48.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f02155b9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0210197730 0x7f0210197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:48.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f02155b9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0210197730 0x7f0210197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54978/0 (socket says 172.21.15.18:54978) 2024-02-17T13:07:48.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f02155b9640 1 -- 172.21.15.18:0/2313978591 learned_addr learned my addr 172.21.15.18:0/2313978591 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:48.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f0214db8640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02101002f0 0x7f021019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:48.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f020ffff640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02101053c0 0x7f021019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:48.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f02155b9640 1 -- 172.21.15.18:0/2313978591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02101053c0 msgr2=0x7f021019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f02155b9640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02101053c0 0x7f021019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f02155b9640 1 -- 172.21.15.18:0/2313978591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02101002f0 msgr2=0x7f021019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f02155b9640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02101002f0 0x7f021019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f02155b9640 1 -- 172.21.15.18:0/2313978591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0204009560 con 0x7f0210197730 2024-02-17T13:07:48.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f0214db8640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02101002f0 0x7f021019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:07:48.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f02155b9640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0210197730 0x7f0210197bb0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f020800eaa0 tx=0x7f020800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:48.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f020dffb640 1 -- 172.21.15.18:0/2313978591 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020800cdb0 con 0x7f0210197730 2024-02-17T13:07:48.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f020dffb640 1 -- 172.21.15.18:0/2313978591 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0208005020 con 0x7f0210197730 2024-02-17T13:07:48.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f021701c640 1 -- 172.21.15.18:0/2313978591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02101a44d0 con 0x7f0210197730 2024-02-17T13:07:48.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.100+0000 7f020dffb640 1 -- 172.21.15.18:0/2313978591 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02080116b0 con 0x7f0210197730 2024-02-17T13:07:48.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.101+0000 7f021701c640 1 -- 172.21.15.18:0/2313978591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02101a49b0 con 0x7f0210197730 2024-02-17T13:07:48.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.101+0000 7f021701c640 1 -- 172.21.15.18:0/2313978591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02101017c0 con 0x7f0210197730 2024-02-17T13:07:48.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.102+0000 7f020dffb640 1 -- 172.21.15.18:0/2313978591 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0208005520 con 0x7f0210197730 2024-02-17T13:07:48.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.103+0000 7f020dffb640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f02000a22b0 0x7f02000a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:48.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.103+0000 7f0214db8640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f02000a22b0 0x7f02000a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:48.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.103+0000 7f020dffb640 1 -- 172.21.15.18:0/2313978591 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f02080ce2e0 con 0x7f0210197730 2024-02-17T13:07:48.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.104+0000 7f0214db8640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f02000a22b0 0x7f02000a4770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0204006010 tx=0x7f02040047a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:48.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.108+0000 7f020dffb640 1 -- 172.21.15.18:0/2313978591 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0208096f30 con 0x7f0210197730 2024-02-17T13:07:48.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.293+0000 7f021701c640 1 -- 172.21.15.18:0/2313978591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0210100ca0 con 0x7f0210197730 2024-02-17T13:07:48.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.295+0000 7f020dffb640 1 -- 172.21.15.18:0/2313978591 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+23079 (secure 0 0 0) 0x7f020801f090 con 0x7f0210197730 2024-02-17T13:07:48.284 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-02-17T13:07:48.284 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":113,"fsid":"7fca71fe-cd91-11ee-95ba-87774f69a715","created":"2024-02-17T12:39:48.882857+0000","modified":"2024-02-17T13:07:13.933088+0000","last_up_change":"2024-02-17T12:46:55.107166+0000","last_in_change":"2024-02-17T12:46:35.031934+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T12:43:12.450140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T12:47:43.690543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T12:47:44.653229+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":112,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31ca3f0e-c84e-4779-984f-daae32ce96f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6803","nonce":417036941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6805","nonce":417036941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6809","nonce":417036941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":417036941},{"type":"v1","addr":"172.21.15.18:6807","nonce":417036941}]},"public_addr":"172.21.15.18:6803/417036941","cluster_addr":"172.21.15.18:6805/417036941","heartbeat_back_addr":"172.21.15.18:6809/417036941","heartbeat_front_addr":"172.21.15.18:6807/417036941","state":["exists","up"]},{"osd":1,"uuid":"8c69905b-1e0f-4b63-90d6-bb6d7a5f752f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6801","nonce":3915030131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6803","nonce":3915030131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6807","nonce":3915030131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3915030131},{"type":"v1","addr":"172.21.15.119:6805","nonce":3915030131}]},"public_addr":"172.21.15.119:6801/3915030131","cluster_addr":"172.21.15.119:6803/3915030131","heartbeat_back_addr":"172.21.15.119:6807/3915030131","heartbeat_front_addr":"172.21.15.119:6805/3915030131","state":["exists","up"]},{"osd":2,"uuid":"5964f903-92b4-4f81-b9d7-861f14ef1d38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6801","nonce":3369250578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6803","nonce":3369250578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6807","nonce":3369250578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3369250578},{"type":"v1","addr":"172.21.15.146:6805","nonce":3369250578}]},"public_addr":"172.21.15.146:6801/3369250578","cluster_addr":"172.21.15.146:6803/3369250578","heartbeat_back_addr":"172.21.15.146:6807/3369250578","heartbeat_front_addr":"172.21.15.146:6805/3369250578","state":["exists","up"]},{"osd":3,"uuid":"c7ef3a4f-6073-4998-812e-dc57d4760ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6811","nonce":3584006256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6813","nonce":3584006256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6817","nonce":3584006256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":3584006256},{"type":"v1","addr":"172.21.15.18:6815","nonce":3584006256}]},"public_addr":"172.21.15.18:6811/3584006256","cluster_addr":"172.21.15.18:6813/3584006256","heartbeat_back_addr":"172.21.15.18:6817/3584006256","heartbeat_front_addr":"172.21.15.18:6815/3584006256","state":["exists","up"]},{"osd":4,"uuid":"c1c67fe4-8ba9-4767-89a7-edab8c686496","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6809","nonce":1444866818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6811","nonce":1444866818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6815","nonce":1444866818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1444866818},{"type":"v1","addr":"172.21.15.119:6813","nonce":1444866818}]},"public_addr":"172.21.15.119:6809/1444866818","cluster_addr":"172.21.15.119:6811/1444866818","heartbeat_back_addr":"172.21.15.119:6815/1444866818","heartbeat_front_addr":"172.21.15.119:6813/1444866818","state":["exists","up"]},{"osd":5,"uuid":"33c58b7e-7d68-40e7-b8bb-b441343fb6de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6809","nonce":4227453651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6811","nonce":4227453651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6815","nonce":4227453651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":4227453651},{"type":"v1","addr":"172.21.15.146:6813","nonce":4227453651}]},"public_addr":"172.21.15.146:6809/4227453651","cluster_addr":"172.21.15.146:6811/4227453651","heartbeat_back_addr":"172.21.15.146:6815/4227453651","heartbeat_front_addr":"172.21.15.146:6813/4227453651","state":["exists","up"]},{"osd":6,"uuid":"c3ef630d-efa7-4b1f-98f5-6235d1081fc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6819","nonce":171901734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6821","nonce":171901734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6825","nonce":171901734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":171901734},{"type":"v1","addr":"172.21.15.18:6823","nonce":171901734}]},"public_addr":"172.21.15.18:6819/171901734","cluster_addr":"172.21.15.18:6821/171901734","heartbeat_back_addr":"172.21.15.18:6825/171901734","heartbeat_front_addr":"172.21.15.18:6823/171901734","state":["exists","up"]},{"osd":7,"uuid":"4dfa2e7c-2f62-4cf8-aa8b-74dd1640770a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6817","nonce":833114737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6819","nonce":833114737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6823","nonce":833114737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":833114737},{"type":"v1","addr":"172.21.15.119:6821","nonce":833114737}]},"public_addr":"172.21.15.119:6817/833114737","cluster_addr":"172.21.15.119:6819/833114737","heartbeat_back_addr":"172.21.15.119:6823/833114737","heartbeat_front_addr":"172.21.15.119:6821/833114737","state":["exists","up"]},{"osd":8,"uuid":"e49e3406-56e4-422e-b157-e5c12ee087a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6817","nonce":1027539241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6819","nonce":1027539241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6823","nonce":1027539241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":1027539241},{"type":"v1","addr":"172.21.15.146:6821","nonce":1027539241}]},"public_addr":"172.21.15.146:6817/1027539241","cluster_addr":"172.21.15.146:6819/1027539241","heartbeat_back_addr":"172.21.15.146:6823/1027539241","heartbeat_front_addr":"172.21.15.146:6821/1027539241","state":["exists","up"]},{"osd":9,"uuid":"a41cd245-5402-474b-9e7e-f15557523fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6827","nonce":3939472833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6829","nonce":3939472833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6833","nonce":3939472833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3939472833},{"type":"v1","addr":"172.21.15.18:6831","nonce":3939472833}]},"public_addr":"172.21.15.18:6827/3939472833","cluster_addr":"172.21.15.18:6829/3939472833","heartbeat_back_addr":"172.21.15.18:6833/3939472833","heartbeat_front_addr":"172.21.15.18:6831/3939472833","state":["exists","up"]},{"osd":10,"uuid":"5693c9d1-dd15-44e1-a2f6-c9a9a4811b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6825","nonce":3116452609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6827","nonce":3116452609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6831","nonce":3116452609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3116452609},{"type":"v1","addr":"172.21.15.119:6829","nonce":3116452609}]},"public_addr":"172.21.15.119:6825/3116452609","cluster_addr":"172.21.15.119:6827/3116452609","heartbeat_back_addr":"172.21.15.119:6831/3116452609","heartbeat_front_addr":"172.21.15.119:6829/3116452609","state":["exists","up"]},{"osd":11,"uuid":"e507f2ad-80c0-44f5-be10-38d3da841477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6825","nonce":2169300109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6827","nonce":2169300109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6831","nonce":2169300109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2169300109},{"type":"v1","addr":"172.21.15.146:6829","nonce":2169300109}]},"public_addr":"172.21.15.146:6825/2169300109","cluster_addr":"172.21.15.146:6827/2169300109","heartbeat_back_addr":"172.21.15.146:6831/2169300109","heartbeat_front_addr":"172.21.15.146:6829/2169300109","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:22.356419+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:42:46.270569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:09.946726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:33.980963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:43:58.024583+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:22.588011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:44:47.467250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:12.161570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:45:37.077729+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:02.797814+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:27.660711+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T12:46:53.600443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/1074404031":"2024-02-18T12:49:52.647628+0000","172.21.15.18:6801/3026633658":"2024-02-18T12:49:52.647628+0000","172.21.15.18:6800/3026633658":"2024-02-18T12:49:52.647628+0000","172.21.15.18:0/2513093443":"2024-02-18T12:49:52.647628+0000","172.21.15.18:0/1103049228":"2024-02-18T12:49:52.647628+0000","172.21.15.18:0/2724676594":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1333332933":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/3776356008":"2024-02-18T12:49:52.647628+0000","172.21.15.18:0/1092699235":"2024-02-18T12:40:50.114081+0000","172.21.15.18:0/1745633107":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6801/1543388552":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/3602994927":"2024-02-18T12:40:22.805777+0000","172.21.15.18:0/2150265194":"2024-02-18T12:40:22.805777+0000","172.21.15.18:6800/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6801/1086757713":"2024-02-18T12:40:50.114081+0000","172.21.15.18:6800/1543388552":"2024-02-18T12:40:22.805777+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":30,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T13:07:48.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.297+0000 7f021701c640 1 -- 172.21.15.18:0/2313978591 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f02000a22b0 msgr2=0x7f02000a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.297+0000 7f021701c640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f02000a22b0 0x7f02000a4770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0204006010 tx=0x7f02040047a0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.297+0000 7f021701c640 1 -- 172.21.15.18:0/2313978591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0210197730 msgr2=0x7f0210197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.297+0000 7f021701c640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0210197730 0x7f0210197bb0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f020800eaa0 tx=0x7f020800ef60 comp rx=0 tx=0).stop 2024-02-17T13:07:48.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.297+0000 7f021701c640 1 -- 172.21.15.18:0/2313978591 shutdown_connections 2024-02-17T13:07:48.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.297+0000 7f021701c640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f02000a22b0 0x7f02000a4770 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.297+0000 7f021701c640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0210197730 0x7f0210197bb0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.297+0000 7f021701c640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02101053c0 0x7f021019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.297+0000 7f021701c640 1 --2- 172.21.15.18:0/2313978591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02101002f0 0x7f021019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.297+0000 7f021701c640 1 -- 172.21.15.18:0/2313978591 >> 172.21.15.18:0/2313978591 conn(0x7f0210077180 msgr2=0x7f02101040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:48.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.297+0000 7f021701c640 1 -- 172.21.15.18:0/2313978591 shutdown_connections 2024-02-17T13:07:48.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.297+0000 7f021701c640 1 -- 172.21.15.18:0/2313978591 wait complete. 2024-02-17T13:07:48.297 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-17T13:07:48.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.412+0000 7f4ac8525640 1 -- 172.21.15.18:0/987176095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ac01023f0 msgr2=0x7f4ac01027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.412+0000 7f4ac8525640 1 --2- 172.21.15.18:0/987176095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ac01023f0 0x7f4ac01027d0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009900 tx=0x7f4abc02bcd0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.412+0000 7f4ac8525640 1 -- 172.21.15.18:0/987176095 shutdown_connections 2024-02-17T13:07:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.412+0000 7f4ac8525640 1 --2- 172.21.15.18:0/987176095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ac010b7c0 0x7f4ac010dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.412+0000 7f4ac8525640 1 --2- 172.21.15.18:0/987176095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac0102d10 0x7f4ac010b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.412+0000 7f4ac8525640 1 --2- 172.21.15.18:0/987176095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ac01023f0 0x7f4ac01027d0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.412+0000 7f4ac8525640 1 -- 172.21.15.18:0/987176095 >> 172.21.15.18:0/987176095 conn(0x7f4ac00771e0 msgr2=0x7f4ac00775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:48.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.412+0000 7f4ac8525640 1 -- 172.21.15.18:0/987176095 shutdown_connections 2024-02-17T13:07:48.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.412+0000 7f4ac8525640 1 -- 172.21.15.18:0/987176095 wait complete. 2024-02-17T13:07:48.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac8525640 1 Processor -- start 2024-02-17T13:07:48.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac8525640 1 -- start start 2024-02-17T13:07:48.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac8525640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ac01023f0 0x7f4ac0100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:48.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac8525640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac0102d10 0x7f4ac0100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:48.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac8525640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ac010b7c0 0x7f4ac0100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:48.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac8525640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac01a5b00 con 0x7f4ac01023f0 2024-02-17T13:07:48.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac8525640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac01a5c70 con 0x7f4ac010b7c0 2024-02-17T13:07:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac8525640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac01a5e30 con 0x7f4ac0102d10 2024-02-17T13:07:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac62c1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ac01023f0 0x7f4ac0100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac62c1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ac01023f0 0x7f4ac0100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54992/0 (socket says 172.21.15.18:54992) 2024-02-17T13:07:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac62c1640 1 -- 172.21.15.18:0/2225807696 learned_addr learned my addr 172.21.15.18:0/2225807696 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac5ac0640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac0102d10 0x7f4ac0100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac6ac2640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ac010b7c0 0x7f4ac0100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac62c1640 1 -- 172.21.15.18:0/2225807696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac0102d10 msgr2=0x7f4ac0100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac62c1640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac0102d10 0x7f4ac0100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac62c1640 1 -- 172.21.15.18:0/2225807696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ac010b7c0 msgr2=0x7f4ac0100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac62c1640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ac010b7c0 0x7f4ac0100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac62c1640 1 -- 172.21.15.18:0/2225807696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc009560 con 0x7f4ac01023f0 2024-02-17T13:07:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.413+0000 7f4ac6ac2640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ac010b7c0 0x7f4ac0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T13:07:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.414+0000 7f4ac62c1640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ac01023f0 0x7f4ac0100040 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005940 tx=0x7f4abc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.414+0000 7f4ab77fe640 1 -- 172.21.15.18:0/2225807696 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc004440 con 0x7f4ac01023f0 2024-02-17T13:07:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.414+0000 7f4ac8525640 1 -- 172.21.15.18:0/2225807696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac01a60b0 con 0x7f4ac01023f0 2024-02-17T13:07:48.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.414+0000 7f4ab77fe640 1 -- 172.21.15.18:0/2225807696 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4abc0052d0 con 0x7f4ac01023f0 2024-02-17T13:07:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.414+0000 7f4ab77fe640 1 -- 172.21.15.18:0/2225807696 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc003da0 con 0x7f4ac01023f0 2024-02-17T13:07:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.414+0000 7f4ac8525640 1 -- 172.21.15.18:0/2225807696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac01a65c0 con 0x7f4ac01023f0 2024-02-17T13:07:48.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.415+0000 7f4ac8525640 1 -- 172.21.15.18:0/2225807696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac00683c0 con 0x7f4ac01023f0 2024-02-17T13:07:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.416+0000 7f4ab77fe640 1 -- 172.21.15.18:0/2225807696 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4abc005440 con 0x7f4ac01023f0 2024-02-17T13:07:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.416+0000 7f4ab77fe640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f4aac0a2230 0x7f4aac0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.417+0000 7f4ac5ac0640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f4aac0a2230 0x7f4aac0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.417+0000 7f4ab77fe640 1 -- 172.21.15.18:0/2225807696 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4abc0f0940 con 0x7f4ac01023f0 2024-02-17T13:07:48.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.417+0000 7f4ac5ac0640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f4aac0a2230 0x7f4aac0a46f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0005e10 tx=0x7f4ab0005da0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:48.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.421+0000 7f4ab77fe640 1 -- 172.21.15.18:0/2225807696 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4abc0b9700 con 0x7f4ac01023f0 2024-02-17T13:07:48.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:48 smithi018 ceph-mon[32625]: pgmap v564: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 481 KiB/s wr, 42 op/s 2024-02-17T13:07:48.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:48 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3975791114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:07:48.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:48 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1436491385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:07:48.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:48 smithi119 ceph-mon[36188]: pgmap v564: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 481 KiB/s wr, 42 op/s 2024-02-17T13:07:48.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:48 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3975791114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:07:48.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:48 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1436491385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:07:48.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.647+0000 7f4ac8525640 1 -- 172.21.15.18:0/2225807696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4ac01112d0 con 0x7f4ac01023f0 2024-02-17T13:07:48.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.650+0000 7f4ab77fe640 1 -- 172.21.15.18:0/2225807696 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4abc0bcdc0 con 0x7f4ac01023f0 2024-02-17T13:07:48.639 INFO:teuthology.orchestra.run.smithi018.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-17T13:07:48.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.653+0000 7f4ac8525640 1 -- 172.21.15.18:0/2225807696 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f4aac0a2230 msgr2=0x7f4aac0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.653+0000 7f4ac8525640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f4aac0a2230 0x7f4aac0a46f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0005e10 tx=0x7f4ab0005da0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.653+0000 7f4ac8525640 1 -- 172.21.15.18:0/2225807696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ac01023f0 msgr2=0x7f4ac0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.653+0000 7f4ac8525640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ac01023f0 0x7f4ac0100040 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005940 tx=0x7f4abc004040 comp rx=0 tx=0).stop 2024-02-17T13:07:48.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.653+0000 7f4ac8525640 1 -- 172.21.15.18:0/2225807696 shutdown_connections 2024-02-17T13:07:48.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.653+0000 7f4ac8525640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f4aac0a2230 0x7f4aac0a46f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.653+0000 7f4ac8525640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ac010b7c0 0x7f4ac0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.653+0000 7f4ac8525640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ac0102d10 0x7f4ac0100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.653+0000 7f4ac8525640 1 --2- 172.21.15.18:0/2225807696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ac01023f0 0x7f4ac0100040 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.653+0000 7f4ac8525640 1 -- 172.21.15.18:0/2225807696 >> 172.21.15.18:0/2225807696 conn(0x7f4ac00771e0 msgr2=0x7f4ac00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:48.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.653+0000 7f4ac8525640 1 -- 172.21.15.18:0/2225807696 shutdown_connections 2024-02-17T13:07:48.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.653+0000 7f4ac8525640 1 -- 172.21.15.18:0/2225807696 wait complete. 2024-02-17T13:07:48.654 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-17T13:07:48.687 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-mon[36454]: pgmap v564: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 481 KiB/s wr, 42 op/s 2024-02-17T13:07:48.688 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3975791114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:07:48.688 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1436491385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T13:07:48.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.776+0000 7fd1b73b5640 1 -- 172.21.15.18:0/46589618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1b0100260 msgr2=0x7fd1b0100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.776+0000 7fd1b73b5640 1 --2- 172.21.15.18:0/46589618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1b0100260 0x7fd1b0100640 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009930 tx=0x7fd1ac02bc30 comp rx=0 tx=0).stop 2024-02-17T13:07:48.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.776+0000 7fd1b73b5640 1 -- 172.21.15.18:0/46589618 shutdown_connections 2024-02-17T13:07:48.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.776+0000 7fd1b73b5640 1 --2- 172.21.15.18:0/46589618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b0105330 0x7fd1b010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.776+0000 7fd1b73b5640 1 --2- 172.21.15.18:0/46589618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1b0100c10 0x7fd1b0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.776+0000 7fd1b73b5640 1 --2- 172.21.15.18:0/46589618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1b0100260 0x7fd1b0100640 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.776+0000 7fd1b73b5640 1 -- 172.21.15.18:0/46589618 >> 172.21.15.18:0/46589618 conn(0x7fd1b0077180 msgr2=0x7fd1b0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:48.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.776+0000 7fd1b73b5640 1 -- 172.21.15.18:0/46589618 shutdown_connections 2024-02-17T13:07:48.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.776+0000 7fd1b73b5640 1 -- 172.21.15.18:0/46589618 wait complete. 2024-02-17T13:07:48.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b73b5640 1 Processor -- start 2024-02-17T13:07:48.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b73b5640 1 -- start start 2024-02-17T13:07:48.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b73b5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1b0100260 0x7fd1b019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:48.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b73b5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1b0105330 0x7fd1b019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:48.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b73b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b0197610 0x7fd1b0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:48.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b73b5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0198000 con 0x7fd1b0100260 2024-02-17T13:07:48.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b73b5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0198170 con 0x7fd1b0197610 2024-02-17T13:07:48.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b73b5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0198330 con 0x7fd1b0105330 2024-02-17T13:07:48.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b5151640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1b0100260 0x7fd1b019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:48.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b5151640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1b0100260 0x7fd1b019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55050/0 (socket says 172.21.15.18:55050) 2024-02-17T13:07:48.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b5151640 1 -- 172.21.15.18:0/1602615294 learned_addr learned my addr 172.21.15.18:0/1602615294 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:48.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b5952640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b0197610 0x7fd1b0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:48.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b4950640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1b0105330 0x7fd1b019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:48.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b5151640 1 -- 172.21.15.18:0/1602615294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1b0105330 msgr2=0x7fd1b019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b5151640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1b0105330 0x7fd1b019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b5151640 1 -- 172.21.15.18:0/1602615294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b0197610 msgr2=0x7fd1b0197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b5151640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b0197610 0x7fd1b0197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b5151640 1 -- 172.21.15.18:0/1602615294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a8009840 con 0x7fd1b0100260 2024-02-17T13:07:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b5952640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b0197610 0x7fd1b0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T13:07:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.777+0000 7fd1b5151640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1b0100260 0x7fd1b019d460 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009230 tx=0x7fd1ac02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.778+0000 7fd1a67fc640 1 -- 172.21.15.18:0/1602615294 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac004350 con 0x7fd1b0100260 2024-02-17T13:07:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.778+0000 7fd1a67fc640 1 -- 172.21.15.18:0/1602615294 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1ac0044b0 con 0x7fd1b0100260 2024-02-17T13:07:48.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.778+0000 7fd1b73b5640 1 -- 172.21.15.18:0/1602615294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ac009560 con 0x7fd1b0100260 2024-02-17T13:07:48.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.778+0000 7fd1a67fc640 1 -- 172.21.15.18:0/1602615294 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac003d00 con 0x7fd1b0100260 2024-02-17T13:07:48.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.778+0000 7fd1b73b5640 1 -- 172.21.15.18:0/1602615294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b01a4690 con 0x7fd1b0100260 2024-02-17T13:07:48.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.779+0000 7fd1b73b5640 1 -- 172.21.15.18:0/1602615294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b0067f70 con 0x7fd1b0100260 2024-02-17T13:07:48.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.780+0000 7fd1a67fc640 1 -- 172.21.15.18:0/1602615294 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd1ac004df0 con 0x7fd1b0100260 2024-02-17T13:07:48.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.780+0000 7fd1a67fc640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd19c0a22b0 0x7fd19c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:48.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.780+0000 7fd1b4950640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd19c0a22b0 0x7fd19c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:48.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.781+0000 7fd1a67fc640 1 -- 172.21.15.18:0/1602615294 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8870+0+0 (secure 0 0 0) 0x7fd1ac0f08d0 con 0x7fd1b0100260 2024-02-17T13:07:48.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.781+0000 7fd1b4950640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd19c0a22b0 0x7fd19c0a4770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00fdb80 tx=0x7fd1a0005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:48.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:48.784+0000 7fd1a67fc640 1 -- 172.21.15.18:0/1602615294 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd1ac0b94b0 con 0x7fd1b0100260 2024-02-17T13:07:48.948 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l[66721]: ignoring --setuser ceph since I am not root 2024-02-17T13:07:48.948 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l[66721]: ignoring --setgroup ceph since I am not root 2024-02-17T13:07:48.948 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-mds[66725]: -- 172.21.15.146:0/2392907272 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b977aa1a0 con 0x559b97781000 2024-02-17T13:07:48.948 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l[66721]: starting mds.l at 2024-02-17T13:07:48.949 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i[67657]: ignoring --setuser ceph since I am not root 2024-02-17T13:07:48.949 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i[67657]: ignoring --setgroup ceph since I am not root 2024-02-17T13:07:48.949 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-mds[67661]: -- 172.21.15.146:0/4198703757 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5615ceb141a0 con 0x5615ceaea800 2024-02-17T13:07:48.949 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i[67657]: starting mds.i at 2024-02-17T13:07:48.950 INFO:journalctl@ceph.mds.c.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-c[67349]: ignoring --setuser ceph since I am not root 2024-02-17T13:07:48.950 INFO:journalctl@ceph.mds.c.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-c[67349]: ignoring --setgroup ceph since I am not root 2024-02-17T13:07:48.950 INFO:journalctl@ceph.mds.c.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-mds[67353]: -- 172.21.15.146:0/825510134 <== mon.1 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b9ba3141a0 con 0x55b9ba2eac00 2024-02-17T13:07:48.950 INFO:journalctl@ceph.mds.c.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-c[67349]: starting mds.c at 2024-02-17T13:07:48.951 INFO:journalctl@ceph.mds.f.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-f[67032]: ignoring --setuser ceph since I am not root 2024-02-17T13:07:48.951 INFO:journalctl@ceph.mds.f.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-f[67032]: ignoring --setgroup ceph since I am not root 2024-02-17T13:07:48.951 INFO:journalctl@ceph.mds.f.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-mds[67036]: -- 172.21.15.146:0/81342811 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5593672da1a0 con 0x5593672b0c00 2024-02-17T13:07:48.951 INFO:journalctl@ceph.mds.f.smithi146.stdout:Feb 17 13:07:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-f[67032]: starting mds.f at 2024-02-17T13:07:48.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.004+0000 7fd1b73b5640 1 -- 172.21.15.18:0/1602615294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd1b0109600 con 0x7fd1b0100260 2024-02-17T13:07:49.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a[32621]: 2024-02-17T13:07:48.648+0000 7f9b650db700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-17T13:07:49.022 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 13:07:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-a[74545]: ignoring --setuser ceph since I am not root 2024-02-17T13:07:49.022 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 13:07:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-a[74545]: ignoring --setgroup ceph since I am not root 2024-02-17T13:07:49.022 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 13:07:48 smithi018 ceph-mds[74572]: -- 172.21.15.18:0/3820230489 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559544d921a0 con 0x559544d68c00 2024-02-17T13:07:49.022 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 13:07:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-a[74545]: starting mds.a at 2024-02-17T13:07:49.023 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 13:07:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g[74164]: ignoring --setuser ceph since I am not root 2024-02-17T13:07:49.023 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 13:07:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g[74164]: ignoring --setgroup ceph since I am not root 2024-02-17T13:07:49.023 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 13:07:48 smithi018 ceph-mds[74168]: -- 172.21.15.18:0/4087070985 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563b6dbfa1a0 con 0x563b6dbd0c00 2024-02-17T13:07:49.023 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 13:07:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g[74164]: starting mds.g at 2024-02-17T13:07:49.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:49 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2313978591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T13:07:49.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:49 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2225807696' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-17T13:07:49.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:49 smithi018 ceph-mon[32625]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-17T13:07:49.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:49 smithi018 ceph-mon[32625]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-17T13:07:49.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:49 smithi018 ceph-mon[32625]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-02-17T13:07:49.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:49 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/2225807696' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-17T13:07:49.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:49 smithi018 ceph-mon[32625]: osdmap e114: 12 total, 12 up, 12 in 2024-02-17T13:07:49.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:49 smithi018 ceph-mon[32625]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-02-17T13:07:49.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:49 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1602615294' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-17T13:07:49.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2313978591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T13:07:49.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2225807696' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-17T13:07:49.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mon[36188]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-17T13:07:49.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mon[36188]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-17T13:07:49.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mon[36188]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-02-17T13:07:49.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/2225807696' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-17T13:07:49.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mon[36188]: osdmap e114: 12 total, 12 up, 12 in 2024-02-17T13:07:49.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mon[36188]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-02-17T13:07:49.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1602615294' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-17T13:07:49.594 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mds[67314]: -- 172.21.15.119:0/2513793107 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c8abea21a0 con 0x55c8abe78c00 2024-02-17T13:07:49.594 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b[67310]: ignoring --setuser ceph since I am not root 2024-02-17T13:07:49.594 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b[67310]: ignoring --setgroup ceph since I am not root 2024-02-17T13:07:49.594 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b[67310]: starting mds.b at 2024-02-17T13:07:49.595 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mds[66992]: -- 172.21.15.119:0/3997800754 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5595410ca1a0 con 0x5595410a0800 2024-02-17T13:07:49.595 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: ignoring --setuser ceph since I am not root 2024-02-17T13:07:49.595 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: ignoring --setgroup ceph since I am not root 2024-02-17T13:07:49.595 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: starting mds.h at 2024-02-17T13:07:49.595 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:07:49.392+0000 7f38cce8a700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-02-17T13:07:49.596 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mds[66365]: -- 172.21.15.119:0/3136731511 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560feb8fa1a0 con 0x560feb8d0800 2024-02-17T13:07:49.596 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k[66361]: ignoring --setuser ceph since I am not root 2024-02-17T13:07:49.596 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k[66361]: ignoring --setgroup ceph since I am not root 2024-02-17T13:07:49.596 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k[66361]: starting mds.k at 2024-02-17T13:07:49.597 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-mds[66688]: -- 172.21.15.119:0/4078775918 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56411898c1a0 con 0x564118962c00 2024-02-17T13:07:49.597 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-e[66684]: ignoring --setuser ceph since I am not root 2024-02-17T13:07:49.597 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-e[66684]: ignoring --setgroup ceph since I am not root 2024-02-17T13:07:49.597 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 13:07:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-e[66684]: starting mds.e at 2024-02-17T13:07:49.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.656+0000 7fd1a67fc640 1 -- 172.21.15.18:0/1602615294 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd1b0109600 con 0x7fd1b0100260 2024-02-17T13:07:49.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.659+0000 7fd1b73b5640 1 -- 172.21.15.18:0/1602615294 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd19c0a22b0 msgr2=0x7fd19c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:49.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.659+0000 7fd1b73b5640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd19c0a22b0 0x7fd19c0a4770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00fdb80 tx=0x7fd1a0005f90 comp rx=0 tx=0).stop 2024-02-17T13:07:49.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.659+0000 7fd1b73b5640 1 -- 172.21.15.18:0/1602615294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1b0100260 msgr2=0x7fd1b019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:49.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.659+0000 7fd1b73b5640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1b0100260 0x7fd1b019d460 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009230 tx=0x7fd1ac02c650 comp rx=0 tx=0).stop 2024-02-17T13:07:49.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.659+0000 7fd1b73b5640 1 -- 172.21.15.18:0/1602615294 shutdown_connections 2024-02-17T13:07:49.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.659+0000 7fd1b73b5640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7fd19c0a22b0 0x7fd19c0a4770 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:49.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.659+0000 7fd1b73b5640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b0197610 0x7fd1b0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:49.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.659+0000 7fd1b73b5640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1b0105330 0x7fd1b019d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:49.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.659+0000 7fd1b73b5640 1 --2- 172.21.15.18:0/1602615294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1b0100260 0x7fd1b019d460 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:49.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.659+0000 7fd1b73b5640 1 -- 172.21.15.18:0/1602615294 >> 172.21.15.18:0/1602615294 conn(0x7fd1b0077180 msgr2=0x7fd1b0068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:49.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.659+0000 7fd1b73b5640 1 -- 172.21.15.18:0/1602615294 shutdown_connections 2024-02-17T13:07:49.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.659+0000 7fd1b73b5640 1 -- 172.21.15.18:0/1602615294 wait complete. 2024-02-17T13:07:49.665 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-17T13:07:49.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:49 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2313978591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T13:07:49.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:49 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2225807696' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-17T13:07:49.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:49 smithi146 ceph-mon[36454]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-17T13:07:49.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:49 smithi146 ceph-mon[36454]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-17T13:07:49.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:49 smithi146 ceph-mon[36454]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-02-17T13:07:49.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:49 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/2225807696' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-17T13:07:49.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:49 smithi146 ceph-mon[36454]: osdmap e114: 12 total, 12 up, 12 in 2024-02-17T13:07:49.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:49 smithi146 ceph-mon[36454]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-02-17T13:07:49.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:49 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1602615294' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-17T13:07:49.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.790+0000 7f50a1c55640 1 -- 172.21.15.18:0/1009046230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f509c068fa0 msgr2=0x7f509c105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:49.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.790+0000 7f50a1c55640 1 --2- 172.21.15.18:0/1009046230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f509c068fa0 0x7f509c105de0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f5084009930 tx=0x7f508402bc30 comp rx=0 tx=0).stop 2024-02-17T13:07:49.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.790+0000 7f50a1c55640 1 -- 172.21.15.18:0/1009046230 shutdown_connections 2024-02-17T13:07:49.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.790+0000 7f50a1c55640 1 --2- 172.21.15.18:0/1009046230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f509c1064c0 0x7f509c10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:49.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.790+0000 7f50a1c55640 1 --2- 172.21.15.18:0/1009046230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f509c068fa0 0x7f509c105de0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:49.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.790+0000 7f50a1c55640 1 --2- 172.21.15.18:0/1009046230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c0685f0 0x7f509c0689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:49.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.790+0000 7f50a1c55640 1 -- 172.21.15.18:0/1009046230 >> 172.21.15.18:0/1009046230 conn(0x7f509c077180 msgr2=0x7f509c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:49.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.790+0000 7f50a1c55640 1 -- 172.21.15.18:0/1009046230 shutdown_connections 2024-02-17T13:07:49.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.790+0000 7f50a1c55640 1 -- 172.21.15.18:0/1009046230 wait complete. 2024-02-17T13:07:49.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f50a1c55640 1 Processor -- start 2024-02-17T13:07:49.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f50a1c55640 1 -- start start 2024-02-17T13:07:49.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f50a1c55640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c0685f0 0x7f509c100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:49.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f50a1c55640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f509c1064c0 0x7f509c100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:49.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f50a1c55640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f509c100ac0 0x7f509c1a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:49.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f50a1c55640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c1a5f80 con 0x7f509c100ac0 2024-02-17T13:07:49.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f50a1c55640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c1a60f0 con 0x7f509c0685f0 2024-02-17T13:07:49.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f50a1c55640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c1a62b0 con 0x7f509c1064c0 2024-02-17T13:07:49.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f509bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f509c100ac0 0x7f509c1a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:49.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f509bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f509c100ac0 0x7f509c1a59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60412/0 (socket says 172.21.15.18:60412) 2024-02-17T13:07:49.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f509bfff640 1 -- 172.21.15.18:0/3043360267 learned_addr learned my addr 172.21.15.18:0/3043360267 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:49.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f509b7fe640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c0685f0 0x7f509c100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:49.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f5092ffd640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f509c1064c0 0x7f509c100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:49.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f509bfff640 1 -- 172.21.15.18:0/3043360267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f509c1064c0 msgr2=0x7f509c100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:49.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f509bfff640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f509c1064c0 0x7f509c100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:49.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f509bfff640 1 -- 172.21.15.18:0/3043360267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c0685f0 msgr2=0x7f509c100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:49.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f509bfff640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c0685f0 0x7f509c100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:49.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.791+0000 7f509bfff640 1 -- 172.21.15.18:0/3043360267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5084009560 con 0x7f509c100ac0 2024-02-17T13:07:49.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.792+0000 7f509bfff640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f509c100ac0 0x7f509c1a59b0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f509400eaa0 tx=0x7f509400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:49.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.792+0000 7f50997fa640 1 -- 172.21.15.18:0/3043360267 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509400cdb0 con 0x7f509c100ac0 2024-02-17T13:07:49.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.792+0000 7f50997fa640 1 -- 172.21.15.18:0/3043360267 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5094005020 con 0x7f509c100ac0 2024-02-17T13:07:49.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.792+0000 7f50a1c55640 1 -- 172.21.15.18:0/3043360267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509c1a6590 con 0x7f509c100ac0 2024-02-17T13:07:49.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.792+0000 7f50997fa640 1 -- 172.21.15.18:0/3043360267 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50940116b0 con 0x7f509c100ac0 2024-02-17T13:07:49.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.792+0000 7f50a1c55640 1 -- 172.21.15.18:0/3043360267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c1a6a70 con 0x7f509c100ac0 2024-02-17T13:07:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.793+0000 7f50a1c55640 1 -- 172.21.15.18:0/3043360267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509c067f70 con 0x7f509c100ac0 2024-02-17T13:07:49.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.794+0000 7f50997fa640 1 -- 172.21.15.18:0/3043360267 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5094005520 con 0x7f509c100ac0 2024-02-17T13:07:49.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.794+0000 7f50997fa640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f50880a2260 0x7f50880a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:49.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.795+0000 7f509b7fe640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f50880a2260 0x7f50880a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:49.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.795+0000 7f50997fa640 1 -- 172.21.15.18:0/3043360267 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8870+0+0 (secure 0 0 0) 0x7f50940ce810 con 0x7f509c100ac0 2024-02-17T13:07:49.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.795+0000 7f509b7fe640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f50880a2260 0x7f50880a4720 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f508c00bbe0 tx=0x7f508c00b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:49.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.798+0000 7f50997fa640 1 -- 172.21.15.18:0/3043360267 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f50940972e0 con 0x7f509c100ac0 2024-02-17T13:07:49.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:49.979+0000 7f50a1c55640 1 -- 172.21.15.18:0/3043360267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f509c101570 con 0x7f509c100ac0 2024-02-17T13:07:50.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: pgmap v566: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 449 KiB/s wr, 43 op/s 2024-02-17T13:07:50.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-17T13:07:50.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-17T13:07:50.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: Cluster is now healthy 2024-02-17T13:07:50.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/1602615294' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-17T13:07:50.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.146:6837/195497270,v1:172.21.15.146:6839/195497270] up:boot 2024-02-17T13:07:50.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.146:6833/4157053786,v1:172.21.15.146:6835/4157053786] up:boot 2024-02-17T13:07:50.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.146:6832/521200987,v1:172.21.15.146:6834/521200987] up:boot 2024-02-17T13:07:50.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.18:6838/929582263,v1:172.21.15.18:6840/929582263] up:boot 2024-02-17T13:07:50.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.18:6839/928734128,v1:172.21.15.18:6841/928734128] up:boot 2024-02-17T13:07:50.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.146:6836/4146231449,v1:172.21.15.146:6838/4146231449] up:boot 2024-02-17T13:07:50.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6833/1568635681,v1:172.21.15.119:6835/1568635681] up:boot 2024-02-17T13:07:50.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6832/1910828003,v1:172.21.15.119:6834/1910828003] up:boot 2024-02-17T13:07:50.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6836/272244662,v1:172.21.15.119:6837/272244662] up:boot 2024-02-17T13:07:50.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: mds.? [v2:172.21.15.119:6838/3224545609,v1:172.21.15.119:6839/3224545609] up:boot 2024-02-17T13:07:50.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: fsmap 12 up:standby 2024-02-17T13:07:50.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T13:07:50.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T13:07:50.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T13:07:50.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T13:07:50.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T13:07:50.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T13:07:50.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T13:07:50.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T13:07:50.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T13:07:50.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T13:07:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:50 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3043360267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-17T13:07:50.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: pgmap v566: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 449 KiB/s wr, 43 op/s 2024-02-17T13:07:50.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-17T13:07:50.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-17T13:07:50.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: Cluster is now healthy 2024-02-17T13:07:50.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/1602615294' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-17T13:07:50.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.146:6837/195497270,v1:172.21.15.146:6839/195497270] up:boot 2024-02-17T13:07:50.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.146:6833/4157053786,v1:172.21.15.146:6835/4157053786] up:boot 2024-02-17T13:07:50.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.146:6832/521200987,v1:172.21.15.146:6834/521200987] up:boot 2024-02-17T13:07:50.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.18:6838/929582263,v1:172.21.15.18:6840/929582263] up:boot 2024-02-17T13:07:50.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.18:6839/928734128,v1:172.21.15.18:6841/928734128] up:boot 2024-02-17T13:07:50.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.146:6836/4146231449,v1:172.21.15.146:6838/4146231449] up:boot 2024-02-17T13:07:50.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6833/1568635681,v1:172.21.15.119:6835/1568635681] up:boot 2024-02-17T13:07:50.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6832/1910828003,v1:172.21.15.119:6834/1910828003] up:boot 2024-02-17T13:07:50.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6836/272244662,v1:172.21.15.119:6837/272244662] up:boot 2024-02-17T13:07:50.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: mds.? [v2:172.21.15.119:6838/3224545609,v1:172.21.15.119:6839/3224545609] up:boot 2024-02-17T13:07:50.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: fsmap 12 up:standby 2024-02-17T13:07:50.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T13:07:50.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T13:07:50.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T13:07:50.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T13:07:50.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T13:07:50.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T13:07:50.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T13:07:50.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T13:07:50.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T13:07:50.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T13:07:50.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:50 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3043360267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-17T13:07:50.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.658+0000 7f50997fa640 1 -- 172.21.15.18:0/3043360267 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) v1 ==== 181+0+0 (secure 0 0 0) 0x7f509401f020 con 0x7f509c100ac0 2024-02-17T13:07:50.647 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'cephfs_metadata' removed 2024-02-17T13:07:50.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.661+0000 7f50a1c55640 1 -- 172.21.15.18:0/3043360267 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f50880a2260 msgr2=0x7f50880a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:50.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.661+0000 7f50a1c55640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f50880a2260 0x7f50880a4720 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f508c00bbe0 tx=0x7f508c00b390 comp rx=0 tx=0).stop 2024-02-17T13:07:50.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.661+0000 7f50a1c55640 1 -- 172.21.15.18:0/3043360267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f509c100ac0 msgr2=0x7f509c1a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:50.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.661+0000 7f50a1c55640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f509c100ac0 0x7f509c1a59b0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f509400eaa0 tx=0x7f509400ef60 comp rx=0 tx=0).stop 2024-02-17T13:07:50.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.662+0000 7f50a1c55640 1 -- 172.21.15.18:0/3043360267 shutdown_connections 2024-02-17T13:07:50.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.662+0000 7f50a1c55640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f50880a2260 0x7f50880a4720 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f508c00bbe0 tx=0x7f508c00b390 comp rx=0 tx=0).stop 2024-02-17T13:07:50.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.662+0000 7f50a1c55640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f509c100ac0 0x7f509c1a59b0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:50.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.662+0000 7f50a1c55640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f509c1064c0 0x7f509c100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:50.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.662+0000 7f50a1c55640 1 --2- 172.21.15.18:0/3043360267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c0685f0 0x7f509c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:50.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.662+0000 7f50a1c55640 1 -- 172.21.15.18:0/3043360267 >> 172.21.15.18:0/3043360267 conn(0x7f509c077180 msgr2=0x7f509c0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:50.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.665+0000 7f50a1c55640 1 -- 172.21.15.18:0/3043360267 shutdown_connections 2024-02-17T13:07:50.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.665+0000 7f50a1c55640 1 -- 172.21.15.18:0/3043360267 wait complete. 2024-02-17T13:07:50.673 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-17T13:07:50.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: pgmap v566: 97 pgs: 97 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 449 KiB/s wr, 43 op/s 2024-02-17T13:07:50.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-17T13:07:50.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-17T13:07:50.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: Cluster is now healthy 2024-02-17T13:07:50.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/1602615294' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-17T13:07:50.698 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.146:6837/195497270,v1:172.21.15.146:6839/195497270] up:boot 2024-02-17T13:07:50.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.146:6833/4157053786,v1:172.21.15.146:6835/4157053786] up:boot 2024-02-17T13:07:50.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.146:6832/521200987,v1:172.21.15.146:6834/521200987] up:boot 2024-02-17T13:07:50.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.18:6838/929582263,v1:172.21.15.18:6840/929582263] up:boot 2024-02-17T13:07:50.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.18:6839/928734128,v1:172.21.15.18:6841/928734128] up:boot 2024-02-17T13:07:50.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.146:6836/4146231449,v1:172.21.15.146:6838/4146231449] up:boot 2024-02-17T13:07:50.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6833/1568635681,v1:172.21.15.119:6835/1568635681] up:boot 2024-02-17T13:07:50.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6832/1910828003,v1:172.21.15.119:6834/1910828003] up:boot 2024-02-17T13:07:50.699 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6836/272244662,v1:172.21.15.119:6837/272244662] up:boot 2024-02-17T13:07:50.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: mds.? [v2:172.21.15.119:6838/3224545609,v1:172.21.15.119:6839/3224545609] up:boot 2024-02-17T13:07:50.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: fsmap 12 up:standby 2024-02-17T13:07:50.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T13:07:50.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T13:07:50.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T13:07:50.700 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T13:07:50.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T13:07:50.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T13:07:50.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T13:07:50.701 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T13:07:50.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T13:07:50.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: from='mgr.15528 172.21.15.18:0/1964764167' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T13:07:50.702 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:50 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3043360267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-17T13:07:50.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.840+0000 7f25eda81640 1 -- 172.21.15.18:0/1534639166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25e8100260 msgr2=0x7f25e8100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:50.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.840+0000 7f25eda81640 1 --2- 172.21.15.18:0/1534639166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25e8100260 0x7f25e8100640 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009930 tx=0x7f25d802bc30 comp rx=0 tx=0).stop 2024-02-17T13:07:50.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.840+0000 7f25eda81640 1 -- 172.21.15.18:0/1534639166 shutdown_connections 2024-02-17T13:07:50.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.840+0000 7f25eda81640 1 --2- 172.21.15.18:0/1534639166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e8105330 0x7f25e810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:50.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.840+0000 7f25eda81640 1 --2- 172.21.15.18:0/1534639166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8100c10 0x7f25e8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:50.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.840+0000 7f25eda81640 1 --2- 172.21.15.18:0/1534639166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25e8100260 0x7f25e8100640 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:50.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.840+0000 7f25eda81640 1 -- 172.21.15.18:0/1534639166 >> 172.21.15.18:0/1534639166 conn(0x7f25e8077180 msgr2=0x7f25e8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:50.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.840+0000 7f25eda81640 1 -- 172.21.15.18:0/1534639166 shutdown_connections 2024-02-17T13:07:50.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.840+0000 7f25eda81640 1 -- 172.21.15.18:0/1534639166 wait complete. 2024-02-17T13:07:50.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.840+0000 7f25eda81640 1 Processor -- start 2024-02-17T13:07:50.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25eda81640 1 -- start start 2024-02-17T13:07:50.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25eda81640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e8100260 0x7f25e819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:50.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25eda81640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8105330 0x7f25e819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:50.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25eda81640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25e81975f0 0x7f25e8197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:50.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25eda81640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8197fe0 con 0x7f25e81975f0 2024-02-17T13:07:50.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25eda81640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8198150 con 0x7f25e8100260 2024-02-17T13:07:50.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25eda81640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8198310 con 0x7f25e8105330 2024-02-17T13:07:50.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25e7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25e81975f0 0x7f25e8197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:50.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25e7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25e81975f0 0x7f25e8197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60428/0 (socket says 172.21.15.18:60428) 2024-02-17T13:07:50.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25e7fff640 1 -- 172.21.15.18:0/4224822842 learned_addr learned my addr 172.21.15.18:0/4224822842 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-02-17T13:07:50.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25e6ffd640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8105330 0x7f25e819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:50.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25e77fe640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e8100260 0x7f25e819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:50.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25e7fff640 1 -- 172.21.15.18:0/4224822842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8105330 msgr2=0x7f25e819d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:50.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25e7fff640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8105330 0x7f25e819d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:50.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25e7fff640 1 -- 172.21.15.18:0/4224822842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e8100260 msgr2=0x7f25e819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:50.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25e7fff640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e8100260 0x7f25e819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:50.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25e7fff640 1 -- 172.21.15.18:0/4224822842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d8009560 con 0x7f25e81975f0 2024-02-17T13:07:50.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25e6ffd640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8105330 0x7f25e819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:07:50.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.841+0000 7f25e77fe640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e8100260 0x7f25e819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T13:07:50.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.842+0000 7f25e7fff640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25e81975f0 0x7f25e8197aa0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f25e000eb10 tx=0x7f25e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:50.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.842+0000 7f25e4ff9640 1 -- 172.21.15.18:0/4224822842 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0004b20 con 0x7f25e81975f0 2024-02-17T13:07:50.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.842+0000 7f25e4ff9640 1 -- 172.21.15.18:0/4224822842 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25e0004c80 con 0x7f25e81975f0 2024-02-17T13:07:50.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.842+0000 7f25eda81640 1 -- 172.21.15.18:0/4224822842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e81a4300 con 0x7f25e81975f0 2024-02-17T13:07:50.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.842+0000 7f25e4ff9640 1 -- 172.21.15.18:0/4224822842 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0010650 con 0x7f25e81975f0 2024-02-17T13:07:50.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.842+0000 7f25eda81640 1 -- 172.21.15.18:0/4224822842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e81a47c0 con 0x7f25e81975f0 2024-02-17T13:07:50.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.843+0000 7f25eda81640 1 -- 172.21.15.18:0/4224822842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e8067f70 con 0x7f25e81975f0 2024-02-17T13:07:50.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.845+0000 7f25e4ff9640 1 -- 172.21.15.18:0/4224822842 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f25e00107f0 con 0x7f25e81975f0 2024-02-17T13:07:50.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.845+0000 7f25e4ff9640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f25d00a2230 0x7f25d00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T13:07:50.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.845+0000 7f25e4ff9640 1 -- 172.21.15.18:0/4224822842 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8316+0+0 (secure 0 0 0) 0x7f25e0014070 con 0x7f25e81975f0 2024-02-17T13:07:50.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.845+0000 7f25e77fe640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f25d00a2230 0x7f25d00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T13:07:50.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.846+0000 7f25e77fe640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f25d00a2230 0x7f25d00a46f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f25d8005ab0 tx=0x7f25d8005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T13:07:50.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:50.848+0000 7f25e4ff9640 1 -- 172.21.15.18:0/4224822842 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f25e0097440 con 0x7f25e81975f0 2024-02-17T13:07:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.132+0000 7f25eda81640 1 -- 172.21.15.18:0/4224822842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f25e81085c0 con 0x7f25e81975f0 2024-02-17T13:07:51.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.664+0000 7f25e4ff9640 1 -- 172.21.15.18:0/4224822842 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) v1 ==== 169+0+0 (secure 0 0 0) 0x7f25e009ab00 con 0x7f25e81975f0 2024-02-17T13:07:51.653 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'cephfs_data' removed 2024-02-17T13:07:51.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.668+0000 7f25eda81640 1 -- 172.21.15.18:0/4224822842 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f25d00a2230 msgr2=0x7f25d00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:51.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.668+0000 7f25eda81640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f25d00a2230 0x7f25d00a46f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f25d8005ab0 tx=0x7f25d8005fd0 comp rx=0 tx=0).stop 2024-02-17T13:07:51.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.668+0000 7f25eda81640 1 -- 172.21.15.18:0/4224822842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25e81975f0 msgr2=0x7f25e8197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T13:07:51.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.668+0000 7f25eda81640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25e81975f0 0x7f25e8197aa0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f25e000eb10 tx=0x7f25e000c2d0 comp rx=0 tx=0).stop 2024-02-17T13:07:51.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.668+0000 7f25eda81640 1 -- 172.21.15.18:0/4224822842 shutdown_connections 2024-02-17T13:07:51.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.668+0000 7f25eda81640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.18:6800/2186095810,v1:172.21.15.18:6801/2186095810] conn(0x7f25d00a2230 0x7f25d00a46f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:51.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.668+0000 7f25eda81640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25e81975f0 0x7f25e8197aa0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:51.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.668+0000 7f25eda81640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8105330 0x7f25e819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:51.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.668+0000 7f25eda81640 1 --2- 172.21.15.18:0/4224822842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25e8100260 0x7f25e819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T13:07:51.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.668+0000 7f25eda81640 1 -- 172.21.15.18:0/4224822842 >> 172.21.15.18:0/4224822842 conn(0x7f25e8077180 msgr2=0x7f25e80fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T13:07:51.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.669+0000 7f25eda81640 1 -- 172.21.15.18:0/4224822842 shutdown_connections 2024-02-17T13:07:51.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-02-17T13:07:51.669+0000 7f25eda81640 1 -- 172.21.15.18:0/4224822842 wait complete. 2024-02-17T13:07:51.676 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-17T13:07:51.717 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-17T13:07:51.768 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-17T13:07:51.801 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-17T13:07:51.802 DEBUG:teuthology.orchestra.run.smithi018:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-17T13:07:51.817 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-17T13:07:51.832 DEBUG:teuthology.orchestra.run.smithi146:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-17T13:07:51.856 INFO:tasks.cephadm:Stopping all daemons... 2024-02-17T13:07:51.857 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-17T13:07:51.858 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.a 2024-02-17T13:07:51.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:51 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/3043360267' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-17T13:07:51.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:51 smithi146 ceph-mon[36454]: osdmap e115: 12 total, 12 up, 12 in 2024-02-17T13:07:51.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:51 smithi146 ceph-mon[36454]: from='client.? 172.21.15.18:0/4224822842' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-17T13:07:52.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:51 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/3043360267' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-17T13:07:52.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:51 smithi018 ceph-mon[32625]: osdmap e115: 12 total, 12 up, 12 in 2024-02-17T13:07:52.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:51 smithi018 ceph-mon[32625]: from='client.? 172.21.15.18:0/4224822842' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-17T13:07:52.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:51 smithi018 systemd[1]: Stopping Ceph mon.a for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:07:52.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:51 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/3043360267' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-17T13:07:52.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:51 smithi119 ceph-mon[36188]: osdmap e115: 12 total, 12 up, 12 in 2024-02-17T13:07:52.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:51 smithi119 ceph-mon[36188]: from='client.? 172.21.15.18:0/4224822842' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-17T13:07:52.491 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a[32621]: 2024-02-17T13:07:52.185+0000 7f9b6a8e6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:07:52.492 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a[32621]: 2024-02-17T13:07:52.185+0000 7f9b6a8e6700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-17T13:07:52.492 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:52 smithi018 podman[112002]: 2024-02-17 13:07:52.216394761 +0000 UTC m=+0.148280913 container died 0786d0b495a8fef71e03a9f2f818b75f54ace37ef0feafe2838aae4f0f7e9261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-02-17T13:07:52.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:52 smithi018 podman[112002]: 2024-02-17 13:07:52.585961752 +0000 UTC m=+0.517847903 container cleanup 0786d0b495a8fef71e03a9f2f818b75f54ace37ef0feafe2838aae4f0f7e9261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-02-17T13:07:52.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Feb 17 13:07:52 smithi018 bash[112002]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-a 2024-02-17T13:07:53.454 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.a.service' 2024-02-17T13:07:53.500 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:07:53.501 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-17T13:07:53.502 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-17T13:07:53.502 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.b 2024-02-17T13:07:53.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:53 smithi119 systemd[1]: Stopping Ceph mon.b for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:07:54.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:53 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-b[36183]: 2024-02-17T13:07:53.953+0000 7f40e7dcd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:07:54.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:53 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-b[36183]: 2024-02-17T13:07:53.953+0000 7f40e7dcd700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-17T13:07:54.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:54 smithi119 podman[73676]: 2024-02-17 13:07:54.157363612 +0000 UTC m=+0.349514066 container died 42ee1f6ae41230c3c762c51bd55e827a0753e767db96901b5fb37977b9174d60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-02-17T13:07:54.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:54 smithi119 podman[73676]: 2024-02-17 13:07:54.507086963 +0000 UTC m=+0.699237416 container cleanup 42ee1f6ae41230c3c762c51bd55e827a0753e767db96901b5fb37977b9174d60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T13:07:54.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Feb 17 13:07:54 smithi119 bash[73676]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-b 2024-02-17T13:07:55.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.b.service' 2024-02-17T13:07:55.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:07:55.216 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-17T13:07:55.216 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-17T13:07:55.216 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.c 2024-02-17T13:07:55.505 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:55 smithi146 systemd[1]: Stopping Ceph mon.c for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:07:55.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:55 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-c[36450]: 2024-02-17T13:07:55.516+0000 7fb1ef54b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:07:55.947 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:55 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-c[36450]: 2024-02-17T13:07:55.516+0000 7fb1ef54b700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-17T13:07:55.948 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:55 smithi146 podman[76414]: 2024-02-17 13:07:55.670030728 +0000 UTC m=+0.255687144 container died a924da2e36e4e94fd58fb612193f6b35d6c3fb2a54118be7a9d531088f8bccae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T13:07:56.323 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:56 smithi146 podman[76414]: 2024-02-17 13:07:56.058523466 +0000 UTC m=+0.644179873 container cleanup a924da2e36e4e94fd58fb612193f6b35d6c3fb2a54118be7a9d531088f8bccae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-c, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-17T13:07:56.323 INFO:journalctl@ceph.mon.c.smithi146.stdout:Feb 17 13:07:56 smithi146 bash[76414]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mon-c 2024-02-17T13:07:56.484 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mon.c.service' 2024-02-17T13:07:56.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:07:56.535 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-17T13:07:56.536 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-17T13:07:56.536 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.x 2024-02-17T13:07:56.838 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 13:07:56 smithi018 systemd[1]: Stopping Ceph mgr.x for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:07:57.191 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 13:07:56 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T13:07:56.848+0000 7fe720294700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:07:57.192 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 13:07:56 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x[32852]: 2024-02-17T13:07:56.848+0000 7fe720294700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-17T13:07:57.192 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 13:07:56 smithi018 podman[112124]: 2024-02-17 13:07:56.880165301 +0000 UTC m=+0.134195744 container died 186b5dfab411674446b6d0f42f4bd36de0d597c7491e6336e502c65bfadbf521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-17T13:07:57.521 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 13:07:57 smithi018 podman[112124]: 2024-02-17 13:07:57.416890657 +0000 UTC m=+0.670921099 container cleanup 186b5dfab411674446b6d0f42f4bd36de0d597c7491e6336e502c65bfadbf521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x, RELEASE=main-6822c7b, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-17T13:07:57.521 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 13:07:57 smithi018 bash[112124]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x 2024-02-17T13:07:57.882 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 13:07:57 smithi018 podman[112138]: 2024-02-17 13:07:57.643562477 +0000 UTC m=+0.755597392 container remove 186b5dfab411674446b6d0f42f4bd36de0d597c7491e6336e502c65bfadbf521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:07:57.882 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 13:07:57 smithi018 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.x.service: Deactivated successfully. 2024-02-17T13:07:57.882 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 13:07:57 smithi018 systemd[1]: Stopped Ceph mgr.x for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T13:07:57.882 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Feb 17 13:07:57 smithi018 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.x.service: Consumed 1min 34.409s CPU time. 2024-02-17T13:07:57.883 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.x.service' 2024-02-17T13:07:57.920 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:07:57.921 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-17T13:07:57.921 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-17T13:07:57.921 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.y 2024-02-17T13:07:58.237 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 13:07:57 smithi119 systemd[1]: Stopping Ceph mgr.y for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:07:58.542 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 13:07:58 smithi119 podman[73798]: 2024-02-17 13:07:58.249216932 +0000 UTC m=+0.125153582 container died 68953ec3dd37828f08a33aab1711b2a0c38f33a38388b14ce7abb84bb373bad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD) 2024-02-17T13:07:58.840 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 13:07:58 smithi119 podman[73798]: 2024-02-17 13:07:58.769794715 +0000 UTC m=+0.645731365 container cleanup 68953ec3dd37828f08a33aab1711b2a0c38f33a38388b14ce7abb84bb373bad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-17T13:07:58.840 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Feb 17 13:07:58 smithi119 bash[73798]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-y 2024-02-17T13:07:59.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.y.service' 2024-02-17T13:07:59.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:07:59.216 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-17T13:07:59.216 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-17T13:07:59.216 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.z 2024-02-17T13:07:59.546 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 13:07:59 smithi146 systemd[1]: Stopping Ceph mgr.z for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:07:59.860 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 13:07:59 smithi146 podman[76537]: 2024-02-17 13:07:59.558789546 +0000 UTC m=+0.146520016 container died f16d38e893343102c0ca121ac5ae56c169c47dac856a3ae350757f9a40f859b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=main-6822c7b, GIT_BRANCH=HEAD) 2024-02-17T13:08:00.197 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 13:07:59 smithi146 podman[76537]: 2024-02-17 13:07:59.979691466 +0000 UTC m=+0.567421942 container cleanup f16d38e893343102c0ca121ac5ae56c169c47dac856a3ae350757f9a40f859b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-17T13:08:00.198 INFO:journalctl@ceph.mgr.z.smithi146.stdout:Feb 17 13:07:59 smithi146 bash[76537]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mgr-z 2024-02-17T13:08:00.429 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mgr.z.service' 2024-02-17T13:08:00.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:08:00.467 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-17T13:08:00.467 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-17T13:08:00.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.0 2024-02-17T13:08:00.743 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:00 smithi018 systemd[1]: Stopping Ceph osd.0 for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:08:01.021 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0[47251]: 2024-02-17T13:08:00.754+0000 7f770743e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:08:01.021 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0[47251]: 2024-02-17T13:08:00.754+0000 7f770743e700 -1 osd.0 116 *** Got signal Terminated *** 2024-02-17T13:08:01.022 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0[47251]: 2024-02-17T13:08:00.754+0000 7f770743e700 -1 osd.0 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T13:08:06.593 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:06 smithi018 podman[112247]: 2024-02-17 13:08:06.345547814 +0000 UTC m=+5.687269497 container died 45f66f351be22ae43d4fde6159f80593d1cda6d5b2268b8c522a33fd09b511c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-17T13:08:06.860 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:06 smithi018 podman[112247]: 2024-02-17 13:08:06.679275475 +0000 UTC m=+6.020997173 container cleanup 45f66f351be22ae43d4fde6159f80593d1cda6d5b2268b8c522a33fd09b511c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-17T13:08:06.860 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:06 smithi018 bash[112247]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0 2024-02-17T13:08:07.271 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:06 smithi018 podman[112264]: 2024-02-17 13:08:06.872098552 +0000 UTC m=+0.530374687 container remove 45f66f351be22ae43d4fde6159f80593d1cda6d5b2268b8c522a33fd09b511c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-17T13:08:07.571 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:07 smithi018 podman[112357]: 2024-02-17 13:08:07.194697798 +0000 UTC m=+0.018099634 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T13:08:07.572 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:07 smithi018 podman[112357]: 2024-02-17 13:08:07.322920618 +0000 UTC m=+0.146322435 container create d0363ef0cb4a067307cf8720ef58c047b62287315d08397836246362b4cd93d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=main-6822c7b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-17T13:08:07.572 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:07 smithi018 podman[112357]: 2024-02-17 13:08:07.508514277 +0000 UTC m=+0.331916098 container init d0363ef0cb4a067307cf8720ef58c047b62287315d08397836246362b4cd93d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0-deactivate, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-17T13:08:07.572 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:07 smithi018 podman[112357]: 2024-02-17 13:08:07.511659011 +0000 UTC m=+0.335060839 container start d0363ef0cb4a067307cf8720ef58c047b62287315d08397836246362b4cd93d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:08:07.572 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:07 smithi018 podman[112357]: 2024-02-17 13:08:07.583735129 +0000 UTC m=+0.407136948 container attach d0363ef0cb4a067307cf8720ef58c047b62287315d08397836246362b4cd93d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-17T13:08:08.021 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:07 smithi018 podman[112357]: 2024-02-17 13:08:07.809377719 +0000 UTC m=+0.632779528 container died d0363ef0cb4a067307cf8720ef58c047b62287315d08397836246362b4cd93d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-17T13:08:08.368 INFO:journalctl@ceph.osd.0.smithi018.stdout:Feb 17 13:08:08 smithi018 podman[112382]: 2024-02-17 13:08:08.362116975 +0000 UTC m=+0.543841608 container remove d0363ef0cb4a067307cf8720ef58c047b62287315d08397836246362b4cd93d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-0-deactivate, ceph=True, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-17T13:08:08.374 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.0.service' 2024-02-17T13:08:08.419 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:08:08.419 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-17T13:08:08.419 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-17T13:08:08.420 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.1 2024-02-17T13:08:08.711 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:08 smithi119 systemd[1]: Stopping Ceph osd.1 for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:08:09.090 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:08 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1[40780]: 2024-02-17T13:08:08.722+0000 7f40b3793700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:08:09.090 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:08 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1[40780]: 2024-02-17T13:08:08.722+0000 7f40b3793700 -1 osd.1 116 *** Got signal Terminated *** 2024-02-17T13:08:09.090 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:08 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1[40780]: 2024-02-17T13:08:08.722+0000 7f40b3793700 -1 osd.1 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T13:08:14.545 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:14 smithi119 podman[73923]: 2024-02-17 13:08:14.302748709 +0000 UTC m=+5.678820692 container died 870e46b5497d2eb06dab2f4b5bfc3374aed9e5fd1b00d6c7497bcadff4b643f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-17T13:08:14.840 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:14 smithi119 podman[73923]: 2024-02-17 13:08:14.648212143 +0000 UTC m=+6.024284171 container cleanup 870e46b5497d2eb06dab2f4b5bfc3374aed9e5fd1b00d6c7497bcadff4b643f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-17T13:08:14.840 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:14 smithi119 bash[73923]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1 2024-02-17T13:08:15.340 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:14 smithi119 podman[73940]: 2024-02-17 13:08:14.999667392 +0000 UTC m=+0.696878454 container remove 870e46b5497d2eb06dab2f4b5bfc3374aed9e5fd1b00d6c7497bcadff4b643f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:08:15.732 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:15 smithi119 podman[74033]: 2024-02-17 13:08:15.306531651 +0000 UTC m=+0.017412151 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T13:08:15.733 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:15 smithi119 podman[74033]: 2024-02-17 13:08:15.408651913 +0000 UTC m=+0.119532398 container create b7430119f56f0a24f4b2690281afa26ad45bdc812c614bbc1868af18255cc7d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1-deactivate, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-02-17T13:08:15.733 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:15 smithi119 podman[74033]: 2024-02-17 13:08:15.628218349 +0000 UTC m=+0.339098848 container init b7430119f56f0a24f4b2690281afa26ad45bdc812c614bbc1868af18255cc7d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T13:08:15.733 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:15 smithi119 podman[74033]: 2024-02-17 13:08:15.631559073 +0000 UTC m=+0.342439567 container start b7430119f56f0a24f4b2690281afa26ad45bdc812c614bbc1868af18255cc7d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-17T13:08:15.733 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:15 smithi119 podman[74033]: 2024-02-17 13:08:15.744582093 +0000 UTC m=+0.455462590 container attach b7430119f56f0a24f4b2690281afa26ad45bdc812c614bbc1868af18255cc7d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:08:16.590 INFO:journalctl@ceph.osd.1.smithi119.stdout:Feb 17 13:08:16 smithi119 podman[74033]: 2024-02-17 13:08:16.292759189 +0000 UTC m=+1.003639682 container died b7430119f56f0a24f4b2690281afa26ad45bdc812c614bbc1868af18255cc7d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux ) 2024-02-17T13:08:17.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.1.service' 2024-02-17T13:08:17.165 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:08:17.165 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-17T13:08:17.165 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-17T13:08:17.165 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.2 2024-02-17T13:08:17.448 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:17 smithi146 systemd[1]: Stopping Ceph osd.2 for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:08:17.947 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:17 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2[41608]: 2024-02-17T13:08:17.481+0000 7fa5a1cd6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:08:17.948 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:17 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2[41608]: 2024-02-17T13:08:17.481+0000 7fa5a1cd6700 -1 osd.2 116 *** Got signal Terminated *** 2024-02-17T13:08:17.948 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:17 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2[41608]: 2024-02-17T13:08:17.481+0000 7fa5a1cd6700 -1 osd.2 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T13:08:23.197 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:22 smithi146 podman[76672]: 2024-02-17 13:08:22.923099833 +0000 UTC m=+5.542854097 container died d8efbf996202cb6404617fa51bed8fa0808a6bafae339f43c9317614a9c9263d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-17T13:08:23.698 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:23 smithi146 podman[76672]: 2024-02-17 13:08:23.215935707 +0000 UTC m=+5.835690005 container cleanup d8efbf996202cb6404617fa51bed8fa0808a6bafae339f43c9317614a9c9263d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-17T13:08:23.698 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:23 smithi146 bash[76672]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2 2024-02-17T13:08:23.698 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:23 smithi146 podman[76689]: 2024-02-17 13:08:23.458556139 +0000 UTC m=+0.538198261 container remove d8efbf996202cb6404617fa51bed8fa0808a6bafae339f43c9317614a9c9263d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:08:24.083 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:23 smithi146 podman[76779]: 2024-02-17 13:08:23.673261002 +0000 UTC m=+0.017934709 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T13:08:24.083 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:23 smithi146 podman[76779]: 2024-02-17 13:08:23.826117413 +0000 UTC m=+0.170791110 container create 6ca9b7b952c613dcae7fe7bbff861c53ea439834c62e562f68b30c14024987bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-17T13:08:24.084 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:24 smithi146 podman[76779]: 2024-02-17 13:08:24.095147448 +0000 UTC m=+0.439821166 container init 6ca9b7b952c613dcae7fe7bbff861c53ea439834c62e562f68b30c14024987bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b) 2024-02-17T13:08:24.373 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:24 smithi146 podman[76779]: 2024-02-17 13:08:24.098336505 +0000 UTC m=+0.443010209 container start 6ca9b7b952c613dcae7fe7bbff861c53ea439834c62e562f68b30c14024987bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-17T13:08:24.374 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:24 smithi146 podman[76779]: 2024-02-17 13:08:24.178151493 +0000 UTC m=+0.522825205 container attach 6ca9b7b952c613dcae7fe7bbff861c53ea439834c62e562f68b30c14024987bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-17T13:08:24.636 INFO:journalctl@ceph.osd.2.smithi146.stdout:Feb 17 13:08:24 smithi146 podman[76779]: 2024-02-17 13:08:24.385034114 +0000 UTC m=+0.729707829 container died 6ca9b7b952c613dcae7fe7bbff861c53ea439834c62e562f68b30c14024987bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-2-deactivate, RELEASE=main-6822c7b, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-17T13:08:24.947 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.2.service' 2024-02-17T13:08:24.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:08:24.991 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-17T13:08:24.991 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-17T13:08:24.992 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.3 2024-02-17T13:08:25.271 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:25 smithi018 systemd[1]: Stopping Ceph osd.3 for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:08:25.538 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:25 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3[53931]: 2024-02-17T13:08:25.288+0000 7f999ca82700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:08:25.538 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:25 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3[53931]: 2024-02-17T13:08:25.288+0000 7f999ca82700 -1 osd.3 116 *** Got signal Terminated *** 2024-02-17T13:08:25.539 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:25 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3[53931]: 2024-02-17T13:08:25.288+0000 7f999ca82700 -1 osd.3 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T13:08:26.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:26 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:26.577+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:27.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:27 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:27.176+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:27.521 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:27 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:27.133+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:27.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:27 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:27.581+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:28.447 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:28 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:28.138+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:28.521 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:28 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:28.160+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:28.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:28 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:28.559+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:29.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:28 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:28.623+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:29.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:29 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:29.168+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:29.521 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:29 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:29.171+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:29.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:29 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:29.595+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:30.021 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:29 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:29.646+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:30.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:30 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:30.188+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:30.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:30 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:30.207+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:30.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:30 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:30.644+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:31.021 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:30 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:30.691+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:31.022 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:30 smithi018 podman[112460]: 2024-02-17 13:08:30.829913556 +0000 UTC m=+5.635420086 container died ab91728db216f8fd42aff3371ddb766006e1680ee2739d49d29e1eb1708c52db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-17T13:08:31.337 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:31 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:31.235+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:31.337 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:31 smithi018 podman[112460]: 2024-02-17 13:08:31.155585931 +0000 UTC m=+5.961092458 container cleanup ab91728db216f8fd42aff3371ddb766006e1680ee2739d49d29e1eb1708c52db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-17T13:08:31.337 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:31 smithi018 bash[112460]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3 2024-02-17T13:08:31.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:31 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:31.189+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:31.656 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:31 smithi018 podman[112490]: 2024-02-17 13:08:31.348470154 +0000 UTC m=+0.512182093 container remove ab91728db216f8fd42aff3371ddb766006e1680ee2739d49d29e1eb1708c52db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T13:08:31.927 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:31 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:31.667+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:31.928 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:31 smithi018 podman[112579]: 2024-02-17 13:08:31.685093923 +0000 UTC m=+0.018172778 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T13:08:31.928 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:31 smithi018 podman[112579]: 2024-02-17 13:08:31.799532176 +0000 UTC m=+0.132611064 container create 43a32648935973703b0d45b913da26aae329c0ce7cf125cc4cfc1730cf809bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T13:08:31.947 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:31 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:31.656+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:32.090 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:31 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:31.660+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:32.203 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:32 smithi018 podman[112579]: 2024-02-17 13:08:32.060727833 +0000 UTC m=+0.393806692 container init 43a32648935973703b0d45b913da26aae329c0ce7cf125cc4cfc1730cf809bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-02-17T13:08:32.203 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:32 smithi018 podman[112579]: 2024-02-17 13:08:32.064320978 +0000 UTC m=+0.397399826 container start 43a32648935973703b0d45b913da26aae329c0ce7cf125cc4cfc1730cf809bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-02-17T13:08:32.203 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:32 smithi018 podman[112579]: 2024-02-17 13:08:32.139534131 +0000 UTC m=+0.472612980 container attach 43a32648935973703b0d45b913da26aae329c0ce7cf125cc4cfc1730cf809bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-17T13:08:32.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:32 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:32.193+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:32.522 INFO:journalctl@ceph.osd.3.smithi018.stdout:Feb 17 13:08:32 smithi018 podman[112579]: 2024-02-17 13:08:32.36592374 +0000 UTC m=+0.699002596 container died 43a32648935973703b0d45b913da26aae329c0ce7cf125cc4cfc1730cf809bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-17T13:08:32.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:32 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:32.213+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:32.923 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:32 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:32.661+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:32.943 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.3.service' 2024-02-17T13:08:32.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:32 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:32.631+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:32.987 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:08:32.987 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-17T13:08:32.987 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-17T13:08:32.988 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.4 2024-02-17T13:08:33.016 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:32 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:32.700+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:33.297 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:33 smithi119 systemd[1]: Stopping Ceph osd.4 for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:08:33.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:33 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:33.173+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:33.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:33 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:33.188+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:33.591 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:33 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4[47651]: 2024-02-17T13:08:33.308+0000 7fc90e9ef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:08:33.591 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:33 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4[47651]: 2024-02-17T13:08:33.308+0000 7fc90e9ef700 -1 osd.4 116 *** Got signal Terminated *** 2024-02-17T13:08:33.591 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:33 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4[47651]: 2024-02-17T13:08:33.308+0000 7fc90e9ef700 -1 osd.4 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T13:08:33.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:33 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:33.597+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:34.021 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:33 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:33.620+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:34.090 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:33 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:33.714+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:34.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:34 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:34.148+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:34.521 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:34 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:34.225+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:34.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:34 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:34.619+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:34.996 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:34 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:34.672+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:35.021 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:34 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:34.589+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:35.340 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:35 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4[47651]: 2024-02-17T13:08:35.007+0000 7fc9077dc700 -1 osd.4 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:11.901168+0000 front 2024-02-17T13:08:11.901122+0000 (oldest deadline 2024-02-17T13:08:34.801305+0000) 2024-02-17T13:08:35.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:35 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:35.117+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:35.521 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:35 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:35.195+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:35.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:35 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:35.664+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:35.985 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:35 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:35.677+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:35.985 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:35 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:35.677+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:36.021 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:35 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:35.639+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:36.340 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:35 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4[47651]: 2024-02-17T13:08:35.996+0000 7fc9077dc700 -1 osd.4 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:11.901168+0000 front 2024-02-17T13:08:11.901122+0000 (oldest deadline 2024-02-17T13:08:34.801305+0000) 2024-02-17T13:08:36.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:36 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:36.150+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:36.521 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:36 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:36.206+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:36.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:36 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:36.626+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:36.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:36 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:36.626+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:36.989 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:36 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:36.670+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:36.989 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:36 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:36.670+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:37.021 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:36 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:36.623+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:37.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:36 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:36.623+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:37.316 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:37 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4[47651]: 2024-02-17T13:08:37.000+0000 7fc9077dc700 -1 osd.4 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:11.901168+0000 front 2024-02-17T13:08:11.901122+0000 (oldest deadline 2024-02-17T13:08:34.801305+0000) 2024-02-17T13:08:37.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:37 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:37.155+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:37.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:37 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:37.189+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:37.590 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:37 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:37.327+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:37.726 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:37 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:37.578+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:37.726 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:37 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:37.578+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:37.986 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:37 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:37.686+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:37.986 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:37 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:37.686+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:38.021 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:37 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:37.645+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:38.021 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:37 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:37.645+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:38.331 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:37 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4[47651]: 2024-02-17T13:08:37.998+0000 7fc9077dc700 -1 osd.4 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:11.901168+0000 front 2024-02-17T13:08:11.901122+0000 (oldest deadline 2024-02-17T13:08:34.801305+0000) 2024-02-17T13:08:38.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:38 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:38.177+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:38.521 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:38 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:38.146+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:38.590 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:38 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:38.341+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:38.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:38 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:38.610+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:38.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:38 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:38.610+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:39.021 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:38 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:38.659+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:39.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:38 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:38.659+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:39.084 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:38 smithi119 podman[74151]: 2024-02-17 13:08:38.828969576 +0000 UTC m=+5.639721132 container died 670d395af67807ad30c3920acc7845b6b138ab3150b81a981400908463a571ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-17T13:08:39.084 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:38 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:38.666+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:39.084 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:38 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:38.666+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:39.340 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:39 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:39.294+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:39.341 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:39 smithi119 podman[74151]: 2024-02-17 13:08:39.183386606 +0000 UTC m=+5.994138160 container cleanup 670d395af67807ad30c3920acc7845b6b138ab3150b81a981400908463a571ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-17T13:08:39.341 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:39 smithi119 bash[74151]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4 2024-02-17T13:08:39.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:39 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:39.104+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:39.639 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:39 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:39.213+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:39.639 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:39 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:39.213+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.993401+0000 front 2024-02-17T13:08:12.993360+0000 (oldest deadline 2024-02-17T13:08:38.893350+0000) 2024-02-17T13:08:39.834 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:39 smithi119 podman[74169]: 2024-02-17 13:08:39.501307016 +0000 UTC m=+0.666610560 container remove 670d395af67807ad30c3920acc7845b6b138ab3150b81a981400908463a571ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, RELEASE=main-6822c7b, ceph=True) 2024-02-17T13:08:39.834 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:39 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:39.667+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:39.835 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:39 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:39.667+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:39.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:39 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:39.799+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:39.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:39 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:39.649+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:39.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:39 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:39.649+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:40.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:39 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:39.680+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:40.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:39 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:39.680+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:40.090 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:39 smithi119 podman[74260]: 2024-02-17 13:08:39.748692878 +0000 UTC m=+0.017509253 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T13:08:40.091 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:39 smithi119 podman[74260]: 2024-02-17 13:08:39.868710691 +0000 UTC m=+0.137527030 container create 31c45006405197eba44a80a6bb76d0511b5936b42206aea703a79c4cc5e4f422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:08:40.380 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:40 smithi119 podman[74260]: 2024-02-17 13:08:40.104329297 +0000 UTC m=+0.373145643 container init 31c45006405197eba44a80a6bb76d0511b5936b42206aea703a79c4cc5e4f422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-02-17T13:08:40.381 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:40 smithi119 podman[74260]: 2024-02-17 13:08:40.107322624 +0000 UTC m=+0.376138972 container start 31c45006405197eba44a80a6bb76d0511b5936b42206aea703a79c4cc5e4f422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-02-17T13:08:40.381 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:40 smithi119 podman[74260]: 2024-02-17 13:08:40.18768862 +0000 UTC m=+0.456504980 container attach 31c45006405197eba44a80a6bb76d0511b5936b42206aea703a79c4cc5e4f422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:08:40.381 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:40 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:40.298+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:40.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:40 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:40.196+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:40.448 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:40 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:40.196+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.993401+0000 front 2024-02-17T13:08:12.993360+0000 (oldest deadline 2024-02-17T13:08:38.893350+0000) 2024-02-17T13:08:40.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:40 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:40.110+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:40.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:40 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:40.110+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:40.670 INFO:journalctl@ceph.osd.4.smithi119.stdout:Feb 17 13:08:40 smithi119 podman[74260]: 2024-02-17 13:08:40.392104906 +0000 UTC m=+0.660921271 container died 31c45006405197eba44a80a6bb76d0511b5936b42206aea703a79c4cc5e4f422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-4-deactivate, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T13:08:40.946 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:40 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:40.700+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:40.946 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:40 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:40.700+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:40.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:40 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:40.841+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:40.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:40 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:40.647+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:40.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:40 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:40.647+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:40.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.4.service' 2024-02-17T13:08:41.021 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:40 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:40.726+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:41.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:40 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:40.726+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:41.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:08:41.055 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-17T13:08:41.055 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-17T13:08:41.055 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.5 2024-02-17T13:08:41.348 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:41 smithi146 systemd[1]: Stopping Ceph osd.5 for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:08:41.348 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:41 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:41.233+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:41.348 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:41 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:41.233+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.993401+0000 front 2024-02-17T13:08:12.993360+0000 (oldest deadline 2024-02-17T13:08:38.893350+0000) 2024-02-17T13:08:41.349 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:41 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:41.358+0000 7fad2dfce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:08:41.349 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:41 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:41.358+0000 7fad2dfce700 -1 osd.5 116 *** Got signal Terminated *** 2024-02-17T13:08:41.349 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:41 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:41.358+0000 7fad2dfce700 -1 osd.5 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T13:08:41.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:41 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:41.127+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:41.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:41 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:41.127+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:41.590 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:41 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:41.286+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:41.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:41 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:41.861+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:41.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:41 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:41.676+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:41.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:41 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:41.676+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:42.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:41 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:41.749+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:42.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:41 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:41.749+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:42.090 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:41 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:41.701+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:42.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:41 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:41.701+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:42.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:42 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:42.155+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:42.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:42 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:42.155+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:42.590 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:42 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:42.277+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:42.662 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:42 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:42.233+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:42.662 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:42 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:42.233+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.993401+0000 front 2024-02-17T13:08:12.993360+0000 (oldest deadline 2024-02-17T13:08:38.893350+0000) 2024-02-17T13:08:42.949 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:42 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:42.894+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:42.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:42 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:42.673+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:42.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:42 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:42.673+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:43.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:42 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:42.762+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:43.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:42 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:42.762+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:43.090 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:42 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:42.738+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:43.090 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:42 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:42.738+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:43.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:43 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:43.159+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:43.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:43 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:43.159+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:43.590 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:43 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:43.292+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:43.698 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:43 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:43.281+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:43.698 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:43 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:43.281+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.993401+0000 front 2024-02-17T13:08:12.993360+0000 (oldest deadline 2024-02-17T13:08:38.893350+0000) 2024-02-17T13:08:44.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:43 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:43.771+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:44.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:43 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:43.771+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:44.184 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:43 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:43.803+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:44.185 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:43 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:43.803+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:44.198 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:43 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:43.858+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:44.198 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:43 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:43.711+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:44.198 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:43 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:43.711+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:44.372 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:44 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:44.295+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:44.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:44 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:44.195+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:44.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:44 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:44.195+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:44.698 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:44 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:44.293+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:44.698 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:44 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:44.293+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.993401+0000 front 2024-02-17T13:08:12.993360+0000 (oldest deadline 2024-02-17T13:08:38.893350+0000) 2024-02-17T13:08:45.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:44 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:44.762+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:45.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:44 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:44.762+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:45.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:44 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:44.762+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.749719+0000 front 2024-02-17T13:08:21.749822+0000 (oldest deadline 2024-02-17T13:08:44.649861+0000) 2024-02-17T13:08:45.090 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:44 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:44.762+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:45.090 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:44 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:44.762+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:45.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:44 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:44.762+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:45.198 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:44 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:44.810+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:45.198 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:44 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:44.810+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:45.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:44 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:44.743+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:45.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:44 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:44.743+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:45.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:44 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:44.743+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:45.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:45 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:45.229+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:45.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:45 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:45.229+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:45.590 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:45 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:45.289+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:45.686 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:45 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:45.273+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:45.687 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:45 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:45.273+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.993401+0000 front 2024-02-17T13:08:12.993360+0000 (oldest deadline 2024-02-17T13:08:38.893350+0000) 2024-02-17T13:08:45.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:45 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:45.783+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:45.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:45 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:45.783+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:45.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:45 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:45.697+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:45.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:45 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:45.697+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:45.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:45 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:45.697+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:46.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:45 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:45.812+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:46.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:45 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:45.812+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:46.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:45 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:45.812+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:46.242 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:45 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:45.783+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:46.243 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:45 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:45.783+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:46.243 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:45 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:45.783+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.749719+0000 front 2024-02-17T13:08:21.749822+0000 (oldest deadline 2024-02-17T13:08:44.649861+0000) 2024-02-17T13:08:46.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:46 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:46.253+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:46.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:46 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:46.253+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:46.590 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:46 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:46.323+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:46.662 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:46 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:46.243+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.891900+0000 front 2024-02-17T13:08:04.891825+0000 (oldest deadline 2024-02-17T13:08:26.591834+0000) 2024-02-17T13:08:46.662 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:46 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:46.243+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.993401+0000 front 2024-02-17T13:08:12.993360+0000 (oldest deadline 2024-02-17T13:08:38.893350+0000) 2024-02-17T13:08:46.663 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:46 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5[48596]: 2024-02-17T13:08:46.243+0000 7fad2adc3700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.594463+0000 front 2024-02-17T13:08:20.594494+0000 (oldest deadline 2024-02-17T13:08:45.294739+0000) 2024-02-17T13:08:46.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:46 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:46.828+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:46.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:46 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:46.828+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:46.949 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:46 smithi146 podman[76897]: 2024-02-17 13:08:46.786636116 +0000 UTC m=+5.529010880 container died 24e0914f9ffd4537521e661524242cc8d124f4352e06c930f654053a65ee4594 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T13:08:46.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:46 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:46.673+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:46.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:46 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:46.673+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:46.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:46 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:46.673+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:47.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:46 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:46.756+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:47.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:46 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:46.756+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:47.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:46 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:46.756+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.749719+0000 front 2024-02-17T13:08:21.749822+0000 (oldest deadline 2024-02-17T13:08:44.649861+0000) 2024-02-17T13:08:47.266 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:46 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:46.856+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:47.266 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:46 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:46.856+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:47.266 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:46 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:46.856+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:47.398 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:47 smithi146 podman[76897]: 2024-02-17 13:08:47.225364844 +0000 UTC m=+5.967739596 container cleanup 24e0914f9ffd4537521e661524242cc8d124f4352e06c930f654053a65ee4594 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T13:08:47.399 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:47 smithi146 bash[76897]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5 2024-02-17T13:08:47.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:47.245+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:47.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:47.245+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:47.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:47.245+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:47.590 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:47.276+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:47.698 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:47 smithi146 podman[76915]: 2024-02-17 13:08:47.409722902 +0000 UTC m=+0.626284339 container remove 24e0914f9ffd4537521e661524242cc8d124f4352e06c930f654053a65ee4594 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5, org.label-schema.license=GPLv2, ceph=True, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-17T13:08:47.698 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:47.647+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:47.699 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:47.647+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:47.699 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:47.647+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:47.708 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:47.717+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:47.708 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:47.717+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:47.708 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:47.717+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.749719+0000 front 2024-02-17T13:08:21.749822+0000 (oldest deadline 2024-02-17T13:08:44.649861+0000) 2024-02-17T13:08:48.077 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:47 smithi146 podman[77007]: 2024-02-17 13:08:47.742015865 +0000 UTC m=+0.017784502 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T13:08:48.077 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:47 smithi146 podman[77007]: 2024-02-17 13:08:47.868922842 +0000 UTC m=+0.144691463 container create 0da6d67fab8881abd2da15f720a90c9ee4ca27156ddf1ecff2f17c2788234d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:08:48.077 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:48 smithi146 podman[77007]: 2024-02-17 13:08:48.087815791 +0000 UTC m=+0.363584405 container init 0da6d67fab8881abd2da15f720a90c9ee4ca27156ddf1ecff2f17c2788234d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-17T13:08:48.077 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:47.832+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:48.078 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:47 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:47.832+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:48.090 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:47.826+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:48.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:47.826+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:48.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:47 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:47.826+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:48.380 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:48 smithi146 podman[77007]: 2024-02-17 13:08:48.091106174 +0000 UTC m=+0.366874797 container start 0da6d67fab8881abd2da15f720a90c9ee4ca27156ddf1ecff2f17c2788234d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-17T13:08:48.381 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:48 smithi146 podman[77007]: 2024-02-17 13:08:48.18777319 +0000 UTC m=+0.463541826 container attach 0da6d67fab8881abd2da15f720a90c9ee4ca27156ddf1ecff2f17c2788234d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-17T13:08:48.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:48.212+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:48.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:48.212+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:48.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:48.212+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:48.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:48 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:48.269+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:48.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:48 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:48.269+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:08:48.637 INFO:journalctl@ceph.osd.5.smithi146.stdout:Feb 17 13:08:48 smithi146 podman[77032]: 2024-02-17 13:08:48.416538055 +0000 UTC m=+0.018835702 container died 0da6d67fab8881abd2da15f720a90c9ee4ca27156ddf1ecff2f17c2788234d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-5-deactivate, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T13:08:48.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:48.873+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:48.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:48.873+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:48.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:48.648+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:48.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:48.648+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:48.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:48 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:48.648+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:49.000 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.5.service' 2024-02-17T13:08:49.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:48.762+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:49.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:48.762+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:49.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:48 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:48.762+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.749719+0000 front 2024-02-17T13:08:21.749822+0000 (oldest deadline 2024-02-17T13:08:44.649861+0000) 2024-02-17T13:08:49.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:08:49.041 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-17T13:08:49.041 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-17T13:08:49.041 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.6 2024-02-17T13:08:49.092 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:48 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:48.803+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:49.092 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:48 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:48.803+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:49.092 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:48 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:48.803+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:49.326 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:49.234+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:49.326 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:49.234+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:49.327 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:49.234+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:49.327 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:49 smithi018 systemd[1]: Stopping Ceph osd.6 for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:08:49.327 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:49.336+0000 7f202ff75700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:08:49.327 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:49.336+0000 7f202ff75700 -1 osd.6 116 *** Got signal Terminated *** 2024-02-17T13:08:49.327 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:49.336+0000 7f202ff75700 -1 osd.6 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T13:08:49.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:49.242+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:49.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:49.242+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:08:49.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:49 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:49.914+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:49.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:49 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:49.914+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:49.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:49 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:49.676+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:49.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:49 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:49.676+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:49.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:49 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:49.676+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:50.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:49.741+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:50.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:49.741+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:50.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:49.741+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.749719+0000 front 2024-02-17T13:08:21.749822+0000 (oldest deadline 2024-02-17T13:08:44.649861+0000) 2024-02-17T13:08:50.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:49.816+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:50.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:49.816+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:50.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:49 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:49.816+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:50.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:50.231+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:50.524 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:50.231+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:50.524 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:50.231+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:50.590 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:50 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:50.289+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:50.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:50 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:50.289+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:08:50.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:50 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:50.885+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:50.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:50 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:50.885+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:50.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:50 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:50.657+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:50.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:50 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:50.657+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:50.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:50 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:50.657+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:51.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:50.767+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:51.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:50.767+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:51.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:50 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:50.767+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.749719+0000 front 2024-02-17T13:08:21.749822+0000 (oldest deadline 2024-02-17T13:08:44.649861+0000) 2024-02-17T13:08:51.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:50 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:50.798+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:51.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:50 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:50.798+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:51.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:50 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:50.798+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:51.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:51 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:51.233+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:51.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:51 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:51.233+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:51.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:51 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:51.233+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:51.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:51 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:51.288+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:51.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:51 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:51.288+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:08:51.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:51 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:51.914+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:51.948 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:51 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:51.914+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:51.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:51 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:51.675+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:51.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:51 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:51.675+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:51.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:51 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:51.675+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:52.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:51 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:51.760+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:52.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:51 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:51.760+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:52.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:51 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:51.760+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.749719+0000 front 2024-02-17T13:08:21.749822+0000 (oldest deadline 2024-02-17T13:08:44.649861+0000) 2024-02-17T13:08:52.023 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:51 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:51.760+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.851178+0000 front 2024-02-17T13:08:29.851204+0000 (oldest deadline 2024-02-17T13:08:51.451891+0000) 2024-02-17T13:08:52.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:51 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:51.841+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:52.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:51 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:51.841+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:52.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:51 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:51.841+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:52.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:52.204+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:52.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:52.204+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:52.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:52.204+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:52.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:52 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:52.248+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:52.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:52 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:52.248+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:08:52.935 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:52 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:52.691+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:52.935 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:52 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:52.691+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:52.935 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:52 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:52.691+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:53.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:52.713+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:53.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:52.713+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:53.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:52.713+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.749719+0000 front 2024-02-17T13:08:21.749822+0000 (oldest deadline 2024-02-17T13:08:44.649861+0000) 2024-02-17T13:08:53.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:52 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:52.713+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.851178+0000 front 2024-02-17T13:08:29.851204+0000 (oldest deadline 2024-02-17T13:08:51.451891+0000) 2024-02-17T13:08:53.198 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:52 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:52.945+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:53.198 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:52 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:52.945+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:53.247 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:52 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:52.878+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:53.247 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:52 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:52.878+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:53.247 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:52 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:52.878+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:53.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:53 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:53.158+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:53.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:53 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:53.158+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:53.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:53 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:53.158+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:53.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:53 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:53.158+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:08:53.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:53 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:53.257+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:53.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:53 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:53.257+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:08:53.944 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:53 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:53.653+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:53.944 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:53 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:53.653+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:53.945 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:53 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:53.653+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:54.022 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:53 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:53.719+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.345091+0000 front 2024-02-17T13:08:04.345072+0000 (oldest deadline 2024-02-17T13:08:28.445352+0000) 2024-02-17T13:08:54.058 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:53 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:53.719+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.447443+0000 front 2024-02-17T13:08:12.447357+0000 (oldest deadline 2024-02-17T13:08:35.947552+0000) 2024-02-17T13:08:54.058 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:53 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:53.719+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.749719+0000 front 2024-02-17T13:08:21.749822+0000 (oldest deadline 2024-02-17T13:08:44.649861+0000) 2024-02-17T13:08:54.059 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:53 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6[60929]: 2024-02-17T13:08:53.719+0000 7f202cd6a700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.851178+0000 front 2024-02-17T13:08:29.851204+0000 (oldest deadline 2024-02-17T13:08:51.451891+0000) 2024-02-17T13:08:54.198 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:53 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:53.955+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:54.198 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:53 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:53.955+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:54.198 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:53 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:53.955+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:08:54.208 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:53 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:53.868+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:54.208 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:53 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:53.868+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:54.208 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:53 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:53.868+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:54.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:54 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:54.192+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:54.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:54 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:54.192+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:54.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:54 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:54.192+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:54.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:54 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:54.192+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:08:54.590 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:54 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:54.218+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:54.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:54 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:54.218+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:08:54.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:54 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:54.613+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:54.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:54 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:54.613+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:54.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:54 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:54.613+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:55.068 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:54 smithi018 podman[112688]: 2024-02-17 13:08:54.823729517 +0000 UTC m=+5.577750849 container died 07846cd7f3a59d87241a1c1e730598e9cde84cc8f60254833bbbd552a05fcf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-17T13:08:55.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:54 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:54.882+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:55.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:54 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:54.882+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:55.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:54 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:54.882+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:55.427 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:55 smithi018 podman[112688]: 2024-02-17 13:08:55.178159437 +0000 UTC m=+5.932180772 container cleanup 07846cd7f3a59d87241a1c1e730598e9cde84cc8f60254833bbbd552a05fcf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-02-17T13:08:55.427 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:55 smithi018 bash[112688]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6 2024-02-17T13:08:55.427 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:55 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:55.228+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:55.427 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:55 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:55.228+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:55.427 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:55 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:55.228+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:55.428 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:55 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:55.228+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:08:55.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:54 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:54.986+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:55.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:54 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:54.986+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:55.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:54 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:54.986+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:08:55.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:55 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:55.244+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:55.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:55 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:55.244+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:08:55.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:55 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:55.244+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:28.983207+0000 front 2024-02-17T13:08:28.983100+0000 (oldest deadline 2024-02-17T13:08:54.883073+0000) 2024-02-17T13:08:55.772 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:55 smithi018 podman[112705]: 2024-02-17 13:08:55.437735881 +0000 UTC m=+0.608361502 container remove 07846cd7f3a59d87241a1c1e730598e9cde84cc8f60254833bbbd552a05fcf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-02-17T13:08:55.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:55 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:55.622+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:55.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:55 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:55.622+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:55.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:55 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:55.622+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:56.080 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:55 smithi018 podman[112794]: 2024-02-17 13:08:55.796872781 +0000 UTC m=+0.112243181 container create 322f61cce5444ed962d7a0a84ae6e2579a7affb196c5c18354f4906b87121d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-17T13:08:56.081 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:55 smithi018 podman[112794]: 2024-02-17 13:08:55.702374728 +0000 UTC m=+0.017745128 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T13:08:56.081 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:56 smithi018 podman[112794]: 2024-02-17 13:08:56.008035062 +0000 UTC m=+0.323405462 container init 322f61cce5444ed962d7a0a84ae6e2579a7affb196c5c18354f4906b87121d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b) 2024-02-17T13:08:56.081 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:56 smithi018 podman[112794]: 2024-02-17 13:08:56.011000863 +0000 UTC m=+0.326371264 container start 322f61cce5444ed962d7a0a84ae6e2579a7affb196c5c18354f4906b87121d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T13:08:56.081 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:56 smithi018 podman[112794]: 2024-02-17 13:08:56.09111132 +0000 UTC m=+0.406481731 container attach 322f61cce5444ed962d7a0a84ae6e2579a7affb196c5c18354f4906b87121d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-02-17T13:08:56.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:55 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:55.835+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:56.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:55 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:55.835+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:56.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:55 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:55.835+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:56.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:55 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:55.835+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:08:56.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:56 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:56.025+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:56.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:56 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:56.025+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:56.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:56 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:56.025+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:08:56.522 INFO:journalctl@ceph.osd.6.smithi018.stdout:Feb 17 13:08:56 smithi018 podman[112794]: 2024-02-17 13:08:56.311221124 +0000 UTC m=+0.626591525 container died 322f61cce5444ed962d7a0a84ae6e2579a7affb196c5c18354f4906b87121d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-17T13:08:56.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:56 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:56.186+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:56.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:56 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:56.186+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:56.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:56 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:56.186+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:56.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:56 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:56.186+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:08:56.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:56 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:56.200+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:56.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:56 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:56.200+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:08:56.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:56 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:56.200+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:28.983207+0000 front 2024-02-17T13:08:28.983100+0000 (oldest deadline 2024-02-17T13:08:54.883073+0000) 2024-02-17T13:08:56.935 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.6.service' 2024-02-17T13:08:56.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:56 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:56.662+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:56.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:56 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:56.662+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:56.948 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:56 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:56.662+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:56.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:08:56.978 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-17T13:08:56.978 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-17T13:08:56.978 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.7 2024-02-17T13:08:57.143 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:57 smithi119 systemd[1]: Stopping Ceph osd.7 for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:08:57.143 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:56 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:56.865+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:57.144 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:56 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:56.865+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:57.144 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:56 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:56.865+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:57.144 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:56 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:56.865+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:08:57.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:57 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:57.020+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:57.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:57 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:57.020+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:57.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:57 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:57.020+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:08:57.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:57 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:57.191+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:57.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:57 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:57.191+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:57.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:57 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:57.191+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:57.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:57 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:57.191+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:08:57.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:57 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:57.153+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:57.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:57 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:57.153+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:08:57.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:57 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:57.153+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:28.983207+0000 front 2024-02-17T13:08:28.983100+0000 (oldest deadline 2024-02-17T13:08:54.883073+0000) 2024-02-17T13:08:57.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:57 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:57.292+0000 7f2e60832700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:08:57.592 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:57 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:57.292+0000 7f2e60832700 -1 osd.7 116 *** Got signal Terminated *** 2024-02-17T13:08:57.592 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:57 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:57.292+0000 7f2e60832700 -1 osd.7 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T13:08:58.042 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:57 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:57.712+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:58.042 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:57 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:57.712+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:58.042 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:57 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:57.712+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:58.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:57 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:57.830+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:58.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:57 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:57.830+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:58.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:57 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:57.830+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:58.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:57 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:57.830+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:08:58.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:58 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:58.052+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:58.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:58 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:58.052+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:58.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:58 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:58.052+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:08:58.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:58 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:58.185+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:58.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:58 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:58.185+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:58.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:58 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:58.185+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:58.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:58 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:58.185+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:08:58.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:58 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:58.193+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:58.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:58 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:58.193+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:08:58.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:58 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:58.193+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:28.983207+0000 front 2024-02-17T13:08:28.983100+0000 (oldest deadline 2024-02-17T13:08:54.883073+0000) 2024-02-17T13:08:59.070 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:58 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:58.718+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:08:59.070 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:58 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:58.718+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:08:59.070 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:58 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:58.718+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:08:59.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:58 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:58.824+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:08:59.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:58 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:58.824+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:08:59.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:58 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:58.824+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:08:59.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:58 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:58.824+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:08:59.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:59 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:59.080+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:08:59.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:59 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:59.080+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:08:59.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:08:59 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:08:59.080+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:08:59.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:59 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:59.220+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:08:59.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:59 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:59.220+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:08:59.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:59 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:59.220+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:08:59.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:08:59 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:08:59.220+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:08:59.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:59 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:59.237+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:08:59.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:59 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:59.237+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:08:59.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:59 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:59.237+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:28.983207+0000 front 2024-02-17T13:08:28.983100+0000 (oldest deadline 2024-02-17T13:08:54.883073+0000) 2024-02-17T13:08:59.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:08:59 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:08:59.237+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.883524+0000 front 2024-02-17T13:08:34.883548+0000 (oldest deadline 2024-02-17T13:08:58.983730+0000) 2024-02-17T13:09:00.041 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:59 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:59.726+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:00.041 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:59 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:59.726+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:00.041 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:08:59 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:08:59.726+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:00.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:59 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:59.848+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:00.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:59 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:59.848+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:00.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:59 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:59.848+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:00.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:59 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:59.848+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:00.092 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:08:59 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:08:59.848+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:00.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:00 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:00.052+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:09:00.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:00 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:00.052+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:09:00.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:00 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:00.052+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:09:00.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:00 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:00.052+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.580200+0000 front 2024-02-17T13:08:34.580105+0000 (oldest deadline 2024-02-17T13:08:59.880020+0000) 2024-02-17T13:09:00.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:00.215+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:00.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:00.215+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:00.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:00.215+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:00.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:00 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:00.215+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:00.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:00 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:09:00.239+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:09:00.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:00 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:09:00.239+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:09:00.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:00 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:09:00.239+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:28.983207+0000 front 2024-02-17T13:08:28.983100+0000 (oldest deadline 2024-02-17T13:08:54.883073+0000) 2024-02-17T13:09:00.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:00 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:09:00.239+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.883524+0000 front 2024-02-17T13:08:34.883548+0000 (oldest deadline 2024-02-17T13:08:58.983730+0000) 2024-02-17T13:09:00.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:00 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:00.696+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:00.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:00 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:00.696+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:00.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:00 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:00.696+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:01.275 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:00 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:00.879+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:01.276 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:00 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:00.880+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:01.276 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:00 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:00.880+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:01.276 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:00 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:00.880+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:01.276 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:00 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:00.880+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:01.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:01.090+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:09:01.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:01.090+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:09:01.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:01.090+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:09:01.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:01.090+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.580200+0000 front 2024-02-17T13:08:34.580105+0000 (oldest deadline 2024-02-17T13:08:59.880020+0000) 2024-02-17T13:09:01.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:01 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:01.237+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:01.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:01 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:01.237+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:01.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:01 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:01.237+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:01.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:01 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:01.237+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:01.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:01 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:09:01.285+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:09:01.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:01 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:09:01.285+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:09:01.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:01 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:09:01.285+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:28.983207+0000 front 2024-02-17T13:08:28.983100+0000 (oldest deadline 2024-02-17T13:08:54.883073+0000) 2024-02-17T13:09:01.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:01 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:09:01.285+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.883524+0000 front 2024-02-17T13:08:34.883548+0000 (oldest deadline 2024-02-17T13:08:58.983730+0000) 2024-02-17T13:09:01.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:01.678+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:01.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:01.678+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:01.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:01.678+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:01.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:01 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:01.678+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:02.267 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:01 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:01.895+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:02.268 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:01 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:01.895+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:02.268 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:01 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:01.895+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:02.268 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:01 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:01.895+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:02.268 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:01 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:01.895+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:02.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:02 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:02.104+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:09:02.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:02 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:02.104+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:09:02.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:02 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:02.104+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:09:02.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:02 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:02.104+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.580200+0000 front 2024-02-17T13:08:34.580105+0000 (oldest deadline 2024-02-17T13:08:59.880020+0000) 2024-02-17T13:09:02.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:02 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:02.207+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:02.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:02 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:02.207+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:02.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:02 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:02.207+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:02.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:02 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:02.207+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:02.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:09:02.278+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:12.080092+0000 front 2024-02-17T13:08:12.080141+0000 (oldest deadline 2024-02-17T13:08:36.780204+0000) 2024-02-17T13:09:02.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:09:02.278+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:17.881079+0000 front 2024-02-17T13:08:17.881053+0000 (oldest deadline 2024-02-17T13:08:47.282044+0000) 2024-02-17T13:09:02.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:09:02.278+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:28.983207+0000 front 2024-02-17T13:08:28.983100+0000 (oldest deadline 2024-02-17T13:08:54.883073+0000) 2024-02-17T13:09:02.591 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7[54546]: 2024-02-17T13:09:02.278+0000 7f2e5961f700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.883524+0000 front 2024-02-17T13:08:34.883548+0000 (oldest deadline 2024-02-17T13:08:58.983730+0000) 2024-02-17T13:09:03.045 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:02 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:02.719+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:03.045 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:02 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:02.719+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:03.046 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:02 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:02.719+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:03.046 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:02 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:02.719+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:03.091 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:02 smithi119 podman[74379]: 2024-02-17 13:09:02.836832804 +0000 UTC m=+5.649026112 container died dde9188ecbfbc57de369a6c795e95e1817beeeae6eb725e2e172dc9282174425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-17T13:09:03.091 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:02.923+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:03.092 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:02.923+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:03.092 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:02.923+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:03.092 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:02.923+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:03.092 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:02.923+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:03.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:03.055+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:09:03.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:03.055+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:09:03.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:03.055+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:09:03.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:03.055+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.580200+0000 front 2024-02-17T13:08:34.580105+0000 (oldest deadline 2024-02-17T13:08:59.880020+0000) 2024-02-17T13:09:03.449 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:03 smithi119 podman[74379]: 2024-02-17 13:09:03.209645611 +0000 UTC m=+6.021838900 container cleanup dde9188ecbfbc57de369a6c795e95e1817beeeae6eb725e2e172dc9282174425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-17T13:09:03.450 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:03 smithi119 bash[74379]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7 2024-02-17T13:09:03.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:03 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:03.255+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:03.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:03 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:03.255+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:03.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:03 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:03.255+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:03.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:03 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:03.255+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:03.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:03 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:03.255+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:03.767 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:03 smithi119 podman[74397]: 2024-02-17 13:09:03.46050574 +0000 UTC m=+0.612838657 container remove dde9188ecbfbc57de369a6c795e95e1817beeeae6eb725e2e172dc9282174425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-17T13:09:04.053 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:03 smithi119 podman[74489]: 2024-02-17 13:09:03.681563475 +0000 UTC m=+0.017752296 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T13:09:04.053 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:03 smithi119 podman[74489]: 2024-02-17 13:09:03.794633411 +0000 UTC m=+0.130822235 container create 0a55c4dadc3a2cc5efafec7d3e8c3a86bec1adb7174516d5bcbd3debc3b6b789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-17T13:09:04.053 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:04 smithi119 podman[74489]: 2024-02-17 13:09:04.064152383 +0000 UTC m=+0.400341208 container init 0a55c4dadc3a2cc5efafec7d3e8c3a86bec1adb7174516d5bcbd3debc3b6b789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7-deactivate, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-02-17T13:09:04.054 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:03.938+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:04.054 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:03.938+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:04.054 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:03.938+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:04.054 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:03.938+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:04.054 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:03 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:03.938+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:04.078 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:03.723+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:04.078 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:03.723+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:04.078 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:03.723+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:04.078 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:03 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:03.723+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:04.341 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:04 smithi119 podman[74489]: 2024-02-17 13:09:04.067334939 +0000 UTC m=+0.403523762 container start 0a55c4dadc3a2cc5efafec7d3e8c3a86bec1adb7174516d5bcbd3debc3b6b789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-02-17T13:09:04.341 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:04 smithi119 podman[74489]: 2024-02-17 13:09:04.155617899 +0000 UTC m=+0.491806745 container attach 0a55c4dadc3a2cc5efafec7d3e8c3a86bec1adb7174516d5bcbd3debc3b6b789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7-deactivate, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-02-17T13:09:04.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:04 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:04.088+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:09:04.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:04 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:04.088+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:09:04.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:04 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:04.088+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:09:04.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:04 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:04.088+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.580200+0000 front 2024-02-17T13:08:34.580105+0000 (oldest deadline 2024-02-17T13:08:59.880020+0000) 2024-02-17T13:09:04.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:04 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:04.271+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:04.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:04 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:04.271+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:04.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:04 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:04.271+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:04.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:04 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:04.271+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:04.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:04 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:04.271+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:04.714 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:04 smithi119 podman[74489]: 2024-02-17 13:09:04.355503113 +0000 UTC m=+0.691691938 container died 0a55c4dadc3a2cc5efafec7d3e8c3a86bec1adb7174516d5bcbd3debc3b6b789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-17T13:09:04.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.7.service' 2024-02-17T13:09:04.979 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:04 smithi119 podman[74515]: 2024-02-17 13:09:04.933758144 +0000 UTC m=+0.569775592 container remove 0a55c4dadc3a2cc5efafec7d3e8c3a86bec1adb7174516d5bcbd3debc3b6b789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-02-17T13:09:04.980 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:04 smithi119 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.7.service: Deactivated successfully. 2024-02-17T13:09:04.980 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:04 smithi119 systemd[1]: Stopped Ceph osd.7 for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T13:09:04.980 INFO:journalctl@ceph.osd.7.smithi119.stdout:Feb 17 13:09:04 smithi119 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.7.service: Consumed 4min 10.079s CPU time. 2024-02-17T13:09:04.980 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:04 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:04.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:04.980 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:04 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:04.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:04.981 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:04 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:04.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:04.981 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:04 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:04.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:04.981 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:04 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:04.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:04.997 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:09:04.997 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-17T13:09:04.997 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-02-17T13:09:04.997 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.8 2024-02-17T13:09:05.027 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:04 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:04.765+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:05.027 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:04 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:04.765+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:05.027 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:04 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:04.765+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:05.028 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:04 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:04.765+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:05.308 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:05.050+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:09:05.308 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:05.050+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:09:05.308 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:05.050+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:09:05.309 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:05.050+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.580200+0000 front 2024-02-17T13:08:34.580105+0000 (oldest deadline 2024-02-17T13:08:59.880020+0000) 2024-02-17T13:09:05.309 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:05 smithi146 systemd[1]: Stopping Ceph osd.8 for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:09:05.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:05 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:05.230+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:05.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:05 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:05.230+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:05.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:05 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:05.230+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:05.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:05 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:05.230+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:05.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:05 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:05.230+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:05.698 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:05.317+0000 7ff986979700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:09:05.699 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:05.317+0000 7ff986979700 -1 osd.8 116 *** Got signal Terminated *** 2024-02-17T13:09:05.699 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:05.317+0000 7ff986979700 -1 osd.8 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T13:09:06.001 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:05.719+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:06.001 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:05.719+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:06.001 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:05.719+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:06.002 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:05 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:05.719+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:06.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:05 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:05.941+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:06.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:05 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:05.941+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:06.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:05 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:05.941+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:06.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:05 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:05.941+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:06.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:05 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:05.941+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:06.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:06 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:06.011+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:09:06.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:06 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:06.011+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:09:06.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:06 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:06.011+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:09:06.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:06 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:06.011+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.580200+0000 front 2024-02-17T13:08:34.580105+0000 (oldest deadline 2024-02-17T13:08:59.880020+0000) 2024-02-17T13:09:06.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:06 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:06.197+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:06.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:06 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:06.197+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:06.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:06 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:06.197+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:06.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:06 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:06.197+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:06.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:06 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:06.197+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:07.045 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:06 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:06.711+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:07.045 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:06 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:06.711+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:07.045 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:06 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:06.711+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:07.045 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:06 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:06.711+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:07.319 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:07 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:07.054+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:09:07.319 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:07 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:07.054+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:09:07.320 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:07 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:07.054+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:09:07.320 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:07 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:07.054+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.580200+0000 front 2024-02-17T13:08:34.580105+0000 (oldest deadline 2024-02-17T13:08:59.880020+0000) 2024-02-17T13:09:07.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:06 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:06.902+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:07.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:06 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:06.902+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:07.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:06 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:06.902+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:07.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:06 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:06.902+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:07.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:06 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:06.902+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:07.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:07 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:07.164+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:07.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:07 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:07.164+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:07.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:07 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:07.164+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:07.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:07 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:07.164+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:07.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:07 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:07.164+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:08.059 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:07 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:07.709+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:08.060 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:07 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:07.709+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:08.060 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:07 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:07.709+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:08.060 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:07 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:07.709+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:08.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:07 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:07.933+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:08.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:07 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:07.933+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:08.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:07 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:07.933+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:08.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:07 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:07.933+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:08.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:07 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:07.933+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:08.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:08 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:08.068+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:09:08.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:08 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:08.068+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:09:08.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:08 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:08.068+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:09:08.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:08 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:08.068+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.580200+0000 front 2024-02-17T13:08:34.580105+0000 (oldest deadline 2024-02-17T13:08:59.880020+0000) 2024-02-17T13:09:08.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:08 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:08.130+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:08.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:08 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:08.130+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:08.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:08 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:08.130+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:08.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:08 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:08.130+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:08.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:08 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:08.130+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:08.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:08 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:08.692+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:08.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:08 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:08.692+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:08.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:08 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:08.692+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:08.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:08 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:08.692+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:08.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:08 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:08.692+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:09.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:08 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:08.917+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:09.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:08 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:08.917+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:09.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:08 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:08.917+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:09.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:08 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:08.917+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:09.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:08 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:08.917+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:09.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:08 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:08.917+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:09.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:09 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:09.028+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:09:09.450 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:09 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:09.028+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:09:09.450 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:09 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:09.028+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:09:09.450 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:09 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:09.028+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.580200+0000 front 2024-02-17T13:08:34.580105+0000 (oldest deadline 2024-02-17T13:08:59.880020+0000) 2024-02-17T13:09:09.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:09 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:09.085+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:09.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:09 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:09.085+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:09.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:09 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:09.085+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:09.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:09 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:09.085+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:09.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:09 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:09.085+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:09.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:09 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:09.707+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:09.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:09 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:09.707+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:09.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:09 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:09.707+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:09.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:09 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:09.707+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:09.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:09 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:09.707+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:10.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:09 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:09.935+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:10.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:09 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:09.935+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:10.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:09 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:09.935+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:10.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:09 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:09.935+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:10.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:09 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:09.935+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:10.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:09 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:09.935+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:10.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:10 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:10.015+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:13.175603+0000 front 2024-02-17T13:08:13.175758+0000 (oldest deadline 2024-02-17T13:08:38.976310+0000) 2024-02-17T13:09:10.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:10 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:10.015+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:18.976768+0000 front 2024-02-17T13:08:18.976790+0000 (oldest deadline 2024-02-17T13:08:44.276907+0000) 2024-02-17T13:09:10.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:10 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:10.015+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:27.678791+0000 front 2024-02-17T13:08:27.678858+0000 (oldest deadline 2024-02-17T13:08:53.578637+0000) 2024-02-17T13:09:10.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:10 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8[55627]: 2024-02-17T13:09:10.015+0000 7ff98376e700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:34.580200+0000 front 2024-02-17T13:08:34.580105+0000 (oldest deadline 2024-02-17T13:08:59.880020+0000) 2024-02-17T13:09:10.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:10 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:10.072+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:10.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:10 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:10.072+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:10.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:10 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:10.072+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:10.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:10 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:10.072+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:10.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:10 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:10.072+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:10.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:10 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:10.072+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.358527+0000 front 2024-02-17T13:08:45.358564+0000 (oldest deadline 2024-02-17T13:09:09.959326+0000) 2024-02-17T13:09:11.057 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:10 smithi146 podman[77113]: 2024-02-17 13:09:10.744758147 +0000 UTC m=+5.545412695 container died 628e781e5fa162b82a1bff3ee76098ec324ae12892caa795a0c4c86277fc0460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-17T13:09:11.057 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:11 smithi146 podman[77113]: 2024-02-17 13:09:11.06772953 +0000 UTC m=+5.868384084 container cleanup 628e781e5fa162b82a1bff3ee76098ec324ae12892caa795a0c4c86277fc0460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-17T13:09:11.058 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:10 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:10.755+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:11.058 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:10 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:10.755+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:11.058 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:10 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:10.755+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:11.058 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:10 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:10.755+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:11.058 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:10 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:10.755+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:11.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:10 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:10.937+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:11.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:10 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:10.937+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:11.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:10 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:10.937+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:11.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:10 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:10.937+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:11.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:10 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:10.937+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:11.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:10 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:10.937+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:11.448 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:11 smithi146 bash[77113]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8 2024-02-17T13:09:11.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:11 smithi146 podman[77142]: 2024-02-17 13:09:11.285255772 +0000 UTC m=+0.535554639 container remove 628e781e5fa162b82a1bff3ee76098ec324ae12892caa795a0c4c86277fc0460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-17T13:09:11.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:11 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:11.037+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:11.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:11 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:11.037+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:11.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:11 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:11.037+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:11.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:11 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:11.037+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:11.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:11 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:11.037+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:11.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:11 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:11.037+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.358527+0000 front 2024-02-17T13:08:45.358564+0000 (oldest deadline 2024-02-17T13:09:09.959326+0000) 2024-02-17T13:09:12.027 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:11 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:11.797+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:12.028 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:11 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:11.797+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:12.028 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:11 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:11.797+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:12.028 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:11 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:11.797+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:12.028 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:11 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:11.797+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:12.028 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:11 smithi146 podman[77233]: 2024-02-17 13:09:11.621760218 +0000 UTC m=+0.017452822 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T13:09:12.029 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:11 smithi146 podman[77233]: 2024-02-17 13:09:11.735851694 +0000 UTC m=+0.131544285 container create 4c0566fb9dd3b1ad6b9ebe96b70a2eb43dd42bd80cca45eb3f3e56fec6f78d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-17T13:09:12.029 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:11 smithi146 podman[77233]: 2024-02-17 13:09:11.954708442 +0000 UTC m=+0.350401045 container init 4c0566fb9dd3b1ad6b9ebe96b70a2eb43dd42bd80cca45eb3f3e56fec6f78d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8-deactivate, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-17T13:09:12.029 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:11 smithi146 podman[77233]: 2024-02-17 13:09:11.95774453 +0000 UTC m=+0.353437113 container start 4c0566fb9dd3b1ad6b9ebe96b70a2eb43dd42bd80cca45eb3f3e56fec6f78d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8-deactivate, RELEASE=main-6822c7b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-17T13:09:12.029 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:12 smithi146 podman[77233]: 2024-02-17 13:09:12.037981816 +0000 UTC m=+0.433674421 container attach 4c0566fb9dd3b1ad6b9ebe96b70a2eb43dd42bd80cca45eb3f3e56fec6f78d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-17T13:09:12.272 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:12 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:12.025+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:12.273 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:12 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:12.025+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:12.273 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:12 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:12.025+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:12.273 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:12 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:12.025+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:12.273 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:12 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:12.025+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:12.273 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:12 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:12.025+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.358527+0000 front 2024-02-17T13:08:45.358564+0000 (oldest deadline 2024-02-17T13:09:09.959326+0000) 2024-02-17T13:09:12.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:11 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:11.980+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:12.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:11 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:11.980+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:12.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:11 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:11.980+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:12.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:11 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:11.980+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:12.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:11 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:11.980+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:12.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:11 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:11.980+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:12.449 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:12 smithi146 podman[77233]: 2024-02-17 13:09:12.252150369 +0000 UTC m=+0.647842967 container died 4c0566fb9dd3b1ad6b9ebe96b70a2eb43dd42bd80cca45eb3f3e56fec6f78d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-17T13:09:12.737 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.8.service' 2024-02-17T13:09:12.762 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:12 smithi146 podman[77258]: 2024-02-17 13:09:12.724837357 +0000 UTC m=+0.467444085 container remove 4c0566fb9dd3b1ad6b9ebe96b70a2eb43dd42bd80cca45eb3f3e56fec6f78d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-8-deactivate, RELEASE=main-6822c7b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T13:09:12.762 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:12 smithi146 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.8.service: Deactivated successfully. 2024-02-17T13:09:12.762 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:12 smithi146 systemd[1]: Stopped Ceph osd.8 for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T13:09:12.762 INFO:journalctl@ceph.osd.8.smithi146.stdout:Feb 17 13:09:12 smithi146 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.8.service: Consumed 3min 23.371s CPU time. 2024-02-17T13:09:12.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:09:12.803 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-02-17T13:09:12.803 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-02-17T13:09:12.804 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.9 2024-02-17T13:09:13.127 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:12 smithi018 systemd[1]: Stopping Ceph osd.9 for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:09:13.128 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:13 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:13.059+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:13.128 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:13 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:13.059+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:13.128 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:13 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:13.059+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:13.128 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:13 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:13.059+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:13.128 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:13 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:13.059+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:13.128 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:13 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:13.059+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.358527+0000 front 2024-02-17T13:08:45.358564+0000 (oldest deadline 2024-02-17T13:09:09.959326+0000) 2024-02-17T13:09:13.128 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:13 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:13.137+0000 7f2e75959700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:09:13.128 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:13 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:13.137+0000 7f2e75959700 -1 osd.9 116 *** Got signal Terminated *** 2024-02-17T13:09:13.129 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:13 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:13.137+0000 7f2e75959700 -1 osd.9 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T13:09:13.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:12 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:12.770+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:13.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:12 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:12.770+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:13.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:12 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:12.770+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:13.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:12 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:12.770+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:13.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:12 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:12.770+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:13.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:12 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:12.985+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:13.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:12 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:12.985+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:13.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:12 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:12.985+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:13.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:12 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:12.985+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:13.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:12 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:12.985+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:13.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:12 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:12.985+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:14.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:13 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:13.791+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:14.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:13 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:13.791+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:14.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:13 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:13.791+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:14.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:13 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:13.791+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:14.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:13 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:13.791+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:14.243 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:13 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:13.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:14.243 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:13 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:13.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:14.243 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:13 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:13.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:14.243 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:13 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:13.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:14.243 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:13 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:13.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:14.243 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:13 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:13.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:14.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:14 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:14.076+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:14.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:14 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:14.076+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:14.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:14 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:14.076+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:14.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:14 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:14.076+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:14.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:14 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:14.076+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:14.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:14 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:14.076+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.358527+0000 front 2024-02-17T13:08:45.358564+0000 (oldest deadline 2024-02-17T13:09:09.959326+0000) 2024-02-17T13:09:15.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:14 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:14.789+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:15.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:14 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:14.789+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:15.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:14 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:14.789+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:15.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:14 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:14.789+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:15.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:14 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:14.789+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:15.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:14 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:14.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:15.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:14 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:14.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:15.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:14 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:14.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:15.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:14 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:14.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:15.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:14 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:14.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:15.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:14 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:14.963+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:15.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:15 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:15.098+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:15.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:15 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:15.098+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:15.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:15 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:15.098+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:15.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:15 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:15.098+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:15.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:15 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:15.098+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:15.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:15 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:15.098+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.358527+0000 front 2024-02-17T13:08:45.358564+0000 (oldest deadline 2024-02-17T13:09:09.959326+0000) 2024-02-17T13:09:16.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:15 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:15.768+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:16.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:15 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:15.768+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:16.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:15 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:15.768+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:16.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:15 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:15.768+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:16.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:15 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:15.768+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:16.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:15 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:15.768+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:16.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.004+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:16.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.004+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:16.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.004+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:16.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.004+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:16.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.004+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:16.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.004+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:16.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.004+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.556698+0000 front 2024-02-17T13:08:53.556863+0000 (oldest deadline 2024-02-17T13:09:15.256546+0000) 2024-02-17T13:09:16.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:16 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:16.063+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:16.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:16 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:16.063+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:16.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:16 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:16.063+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:16.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:16 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:16.063+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:16.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:16 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:16.063+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:16.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:16 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:16.063+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.358527+0000 front 2024-02-17T13:08:45.358564+0000 (oldest deadline 2024-02-17T13:09:09.959326+0000) 2024-02-17T13:09:17.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:16 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:16.743+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:17.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:16 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:16.743+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:17.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:16 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:16.743+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:17.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:16 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:16.743+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:17.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:16 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:16.743+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:17.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:16 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:16.743+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:17.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.984+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:17.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.984+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:17.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.984+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:17.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.984+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:17.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.984+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:17.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.984+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:17.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:16 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:16.984+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.556698+0000 front 2024-02-17T13:08:53.556863+0000 (oldest deadline 2024-02-17T13:09:15.256546+0000) 2024-02-17T13:09:17.522 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:17.102+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:17.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:17.102+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:17.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:17.102+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:17.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:17.102+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:17.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:17.102+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:17.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:17 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:17.102+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.358527+0000 front 2024-02-17T13:08:45.358564+0000 (oldest deadline 2024-02-17T13:09:09.959326+0000) 2024-02-17T13:09:18.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:17 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:17.761+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:18.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:17 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:17.761+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:18.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:17 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:17.761+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:18.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:17 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:17.761+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:18.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:17 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:17.761+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:18.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:17 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:17.761+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:18.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:17 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:17.991+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:18.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:17 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:17.991+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:18.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:17 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:17.991+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:18.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:17 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:17.991+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:18.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:17 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:17.991+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:18.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:17 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:17.991+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:18.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:17 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:17.991+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.556698+0000 front 2024-02-17T13:08:53.556863+0000 (oldest deadline 2024-02-17T13:09:15.256546+0000) 2024-02-17T13:09:18.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:18 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:18.092+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:02.250092+0000 front 2024-02-17T13:08:02.250183+0000 (oldest deadline 2024-02-17T13:08:26.950210+0000) 2024-02-17T13:09:18.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:18 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:18.092+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.952266+0000 front 2024-02-17T13:08:10.952228+0000 (oldest deadline 2024-02-17T13:08:39.752936+0000) 2024-02-17T13:09:18.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:18 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:18.092+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:20.854016+0000 front 2024-02-17T13:08:20.853985+0000 (oldest deadline 2024-02-17T13:08:46.754061+0000) 2024-02-17T13:09:18.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:18 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:18.092+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.055253+0000 front 2024-02-17T13:08:29.055228+0000 (oldest deadline 2024-02-17T13:08:52.555508+0000) 2024-02-17T13:09:18.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:18 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:18.092+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:37.757111+0000 front 2024-02-17T13:08:37.757175+0000 (oldest deadline 2024-02-17T13:09:03.057200+0000) 2024-02-17T13:09:18.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:18 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9[67720]: 2024-02-17T13:09:18.092+0000 7f2e6e746700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.358527+0000 front 2024-02-17T13:08:45.358564+0000 (oldest deadline 2024-02-17T13:09:09.959326+0000) 2024-02-17T13:09:18.936 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:18 smithi018 podman[112899]: 2024-02-17 13:09:18.632374718 +0000 UTC m=+5.615087331 container died dad64f14ca7d9f77a11d6f5d058da0fb4d9b2aa7a58e3f22957b570e0d091c26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:09:19.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:18 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:18.779+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:19.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:18 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:18.779+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:19.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:18 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:18.779+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:19.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:18 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:18.779+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:19.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:18 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:18.779+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:19.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:18 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:18.779+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:19.236 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:19 smithi018 podman[112899]: 2024-02-17 13:09:19.029426917 +0000 UTC m=+6.012139523 container cleanup dad64f14ca7d9f77a11d6f5d058da0fb4d9b2aa7a58e3f22957b570e0d091c26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-17T13:09:19.236 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:19 smithi018 bash[112899]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9 2024-02-17T13:09:19.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:19 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:19.039+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:19.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:19 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:19.039+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:19.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:19 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:19.039+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:19.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:19 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:19.039+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:19.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:19 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:19.039+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:19.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:19 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:19.039+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:19.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:19 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:19.039+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.556698+0000 front 2024-02-17T13:08:53.556863+0000 (oldest deadline 2024-02-17T13:09:15.256546+0000) 2024-02-17T13:09:19.523 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:19 smithi018 podman[112914]: 2024-02-17 13:09:19.247093053 +0000 UTC m=+0.614762875 container remove dad64f14ca7d9f77a11d6f5d058da0fb4d9b2aa7a58e3f22957b570e0d091c26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, ceph=True) 2024-02-17T13:09:19.848 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:19 smithi018 podman[113004]: 2024-02-17 13:09:19.450330534 +0000 UTC m=+0.018745974 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T13:09:19.848 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:19 smithi018 podman[113004]: 2024-02-17 13:09:19.556307607 +0000 UTC m=+0.124723076 container create d3c60d5c67fd830b28cbc7a54533977d5f15139f52505fb0bc6a6d16bb63da2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-17T13:09:19.848 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:19 smithi018 podman[113004]: 2024-02-17 13:09:19.775632486 +0000 UTC m=+0.344047922 container init d3c60d5c67fd830b28cbc7a54533977d5f15139f52505fb0bc6a6d16bb63da2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-17T13:09:19.848 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:19 smithi018 podman[113004]: 2024-02-17 13:09:19.778961037 +0000 UTC m=+0.347376466 container start d3c60d5c67fd830b28cbc7a54533977d5f15139f52505fb0bc6a6d16bb63da2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, io.buildah.version=1.33.5) 2024-02-17T13:09:19.849 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:19 smithi018 podman[113004]: 2024-02-17 13:09:19.858444906 +0000 UTC m=+0.426860333 container attach d3c60d5c67fd830b28cbc7a54533977d5f15139f52505fb0bc6a6d16bb63da2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:09:20.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:19 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:19.758+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:20.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:19 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:19.758+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:20.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:19 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:19.758+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:20.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:19 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:19.758+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:20.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:19 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:19.758+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:20.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:19 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:19.758+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:20.273 INFO:journalctl@ceph.osd.9.smithi018.stdout:Feb 17 13:09:20 smithi018 podman[113004]: 2024-02-17 13:09:20.065347076 +0000 UTC m=+0.633762503 container died d3c60d5c67fd830b28cbc7a54533977d5f15139f52505fb0bc6a6d16bb63da2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:09:20.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:20 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:20.011+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:20.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:20 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:20.011+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:20.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:20 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:20.011+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:20.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:20 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:20.011+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:20.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:20 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:20.011+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:20.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:20 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:20.011+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:20.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:20 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:20.011+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.556698+0000 front 2024-02-17T13:08:53.556863+0000 (oldest deadline 2024-02-17T13:09:15.256546+0000) 2024-02-17T13:09:20.736 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.9.service' 2024-02-17T13:09:20.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:09:20.778 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-02-17T13:09:20.778 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-02-17T13:09:20.778 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.10 2024-02-17T13:09:21.075 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:20 smithi119 systemd[1]: Stopping Ceph osd.10 for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:09:21.075 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:21.055+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:21.076 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:21.055+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:21.076 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:21.055+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:21.076 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:21.055+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:21.076 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:21.055+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:21.076 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:21.055+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:21.076 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:21.055+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.556698+0000 front 2024-02-17T13:08:53.556863+0000 (oldest deadline 2024-02-17T13:09:15.256546+0000) 2024-02-17T13:09:21.077 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:21.085+0000 7f2d81cd7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:09:21.077 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:21.085+0000 7f2d81cd7700 -1 osd.10 116 *** Got signal Terminated *** 2024-02-17T13:09:21.077 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:21 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:21.085+0000 7f2d81cd7700 -1 osd.10 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T13:09:21.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:20 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:20.807+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:21.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:20 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:20.807+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:21.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:20 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:20.807+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:21.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:20 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:20.807+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:21.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:20 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:20.807+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:21.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:20 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:20.807+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:22.080 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:21 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:21.808+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:22.080 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:21 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:21.808+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:22.081 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:21 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:21.808+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:22.081 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:21 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:21.808+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:22.081 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:21 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:21.808+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:22.081 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:21 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:21.808+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:22.591 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:22 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:22.101+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:22.592 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:22 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:22.101+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:22.592 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:22 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:22.101+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:22.592 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:22 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:22.101+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:22.592 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:22 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:22.101+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:22.592 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:22 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:22.101+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:22.592 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:22 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:22.101+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.556698+0000 front 2024-02-17T13:08:53.556863+0000 (oldest deadline 2024-02-17T13:09:15.256546+0000) 2024-02-17T13:09:23.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:22 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:22.817+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:23.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:22 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:22.817+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:23.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:22 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:22.817+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:23.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:22 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:22.817+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:23.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:22 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:22.817+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:23.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:22 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:22.817+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:23.591 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:23 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:23.116+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:23.592 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:23 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:23.116+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:23.592 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:23 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:23.116+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:23.592 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:23 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:23.116+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:23.592 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:23 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:23.116+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:23.592 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:23 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:23.116+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:23.592 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:23 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:23.116+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.556698+0000 front 2024-02-17T13:08:53.556863+0000 (oldest deadline 2024-02-17T13:09:15.256546+0000) 2024-02-17T13:09:24.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:23 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:23.774+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:24.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:23 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:23.774+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:24.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:23 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:23.774+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:24.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:23 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:23.774+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:24.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:23 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:23.774+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:24.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:23 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:23.774+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:24.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:24 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:24.067+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:24.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:24 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:24.067+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:24.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:24 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:24.067+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:24.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:24 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:24.067+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:24.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:24 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:24.067+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:24.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:24 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:24.067+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:24.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:24 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:24.067+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.556698+0000 front 2024-02-17T13:08:53.556863+0000 (oldest deadline 2024-02-17T13:09:15.256546+0000) 2024-02-17T13:09:25.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:24 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:24.823+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:25.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:24 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:24.823+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:25.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:24 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:24.823+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:25.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:24 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:24.823+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:25.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:24 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:24.823+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:25.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:24 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:24.823+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:25.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.027+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:25.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.027+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:25.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.027+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:25.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.027+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:25.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.027+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:25.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.027+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:25.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.027+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.556698+0000 front 2024-02-17T13:08:53.556863+0000 (oldest deadline 2024-02-17T13:09:15.256546+0000) 2024-02-17T13:09:26.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:25 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:25.865+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:26.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:25 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:25.865+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:26.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:25 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:25.865+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:26.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:25 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:25.865+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:26.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:25 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:25.865+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:26.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:25 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:25.865+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:26.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:25 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:25.865+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-02-17T13:09:02.235707+0000 front 2024-02-17T13:09:02.235805+0000 (oldest deadline 2024-02-17T13:09:25.135681+0000) 2024-02-17T13:09:26.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.978+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:05.451102+0000 front 2024-02-17T13:08:05.451219+0000 (oldest deadline 2024-02-17T13:08:30.751119+0000) 2024-02-17T13:09:26.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.978+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:10.751553+0000 front 2024-02-17T13:08:10.751580+0000 (oldest deadline 2024-02-17T13:08:34.851735+0000) 2024-02-17T13:09:26.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.978+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.853295+0000 front 2024-02-17T13:08:21.853370+0000 (oldest deadline 2024-02-17T13:08:44.753116+0000) 2024-02-17T13:09:26.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.978+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-02-17T13:08:29.453893+0000 front 2024-02-17T13:08:29.454117+0000 (oldest deadline 2024-02-17T13:08:55.353952+0000) 2024-02-17T13:09:26.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.978+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:35.354264+0000 front 2024-02-17T13:08:35.354289+0000 (oldest deadline 2024-02-17T13:08:58.854420+0000) 2024-02-17T13:09:26.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.978+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:43.555638+0000 front 2024-02-17T13:08:43.555735+0000 (oldest deadline 2024-02-17T13:09:08.855673+0000) 2024-02-17T13:09:26.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:25 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10[61623]: 2024-02-17T13:09:25.978+0000 7f2d7eacc700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.556698+0000 front 2024-02-17T13:08:53.556863+0000 (oldest deadline 2024-02-17T13:09:15.256546+0000) 2024-02-17T13:09:26.841 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:26 smithi119 podman[74595]: 2024-02-17 13:09:26.578202119 +0000 UTC m=+5.599885338 container died 1a624659c666519e1bc5f1fe1ea439455f935f266dcba97b7092c6f7d77822ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-17T13:09:27.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:26 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:26.856+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:27.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:26 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:26.856+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:27.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:26 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:26.856+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:27.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:26 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:26.856+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:27.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:26 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:26.856+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:27.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:26 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:26.856+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:27.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:26 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:26.856+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-02-17T13:09:02.235707+0000 front 2024-02-17T13:09:02.235805+0000 (oldest deadline 2024-02-17T13:09:25.135681+0000) 2024-02-17T13:09:27.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:26 smithi119 podman[74595]: 2024-02-17 13:09:26.910241047 +0000 UTC m=+5.931924289 container cleanup 1a624659c666519e1bc5f1fe1ea439455f935f266dcba97b7092c6f7d77822ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-17T13:09:27.341 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:26 smithi119 bash[74595]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10 2024-02-17T13:09:27.342 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:27 smithi119 podman[74613]: 2024-02-17 13:09:27.11962502 +0000 UTC m=+0.535810458 container remove 1a624659c666519e1bc5f1fe1ea439455f935f266dcba97b7092c6f7d77822ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, GIT_BRANCH=HEAD) 2024-02-17T13:09:27.762 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:27 smithi119 podman[74708]: 2024-02-17 13:09:27.419790864 +0000 UTC m=+0.017754674 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T13:09:27.763 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:27 smithi119 podman[74708]: 2024-02-17 13:09:27.545216914 +0000 UTC m=+0.143180730 container create 319882069c05e3ce68f9d2f05ecc4976d535f19bbccb73b646cdee2319a3abd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T13:09:27.763 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:27 smithi119 podman[74708]: 2024-02-17 13:09:27.772861356 +0000 UTC m=+0.370825168 container init 319882069c05e3ce68f9d2f05ecc4976d535f19bbccb73b646cdee2319a3abd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-17T13:09:28.051 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:27 smithi119 podman[74708]: 2024-02-17 13:09:27.776431181 +0000 UTC m=+0.374394982 container start 319882069c05e3ce68f9d2f05ecc4976d535f19bbccb73b646cdee2319a3abd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-17T13:09:28.051 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:27 smithi119 podman[74708]: 2024-02-17 13:09:27.864417578 +0000 UTC m=+0.462381387 container attach 319882069c05e3ce68f9d2f05ecc4976d535f19bbccb73b646cdee2319a3abd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-17T13:09:28.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:27 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:27.836+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:28.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:27 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:27.836+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:28.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:27 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:27.836+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:28.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:27 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:27.836+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:28.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:27 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:27.836+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:28.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:27 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:27.836+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:28.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:27 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:27.836+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-02-17T13:09:02.235707+0000 front 2024-02-17T13:09:02.235805+0000 (oldest deadline 2024-02-17T13:09:25.135681+0000) 2024-02-17T13:09:28.323 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:28 smithi119 podman[74708]: 2024-02-17 13:09:28.062321319 +0000 UTC m=+0.660285138 container died 319882069c05e3ce68f9d2f05ecc4976d535f19bbccb73b646cdee2319a3abd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10-deactivate, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True) 2024-02-17T13:09:28.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.10.service' 2024-02-17T13:09:28.590 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:28 smithi119 podman[74734]: 2024-02-17 13:09:28.5592444 +0000 UTC m=+0.492291906 container remove 319882069c05e3ce68f9d2f05ecc4976d535f19bbccb73b646cdee2319a3abd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-10-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-17T13:09:28.590 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:28 smithi119 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.10.service: Deactivated successfully. 2024-02-17T13:09:28.590 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:28 smithi119 systemd[1]: Stopped Ceph osd.10 for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T13:09:28.590 INFO:journalctl@ceph.osd.10.smithi119.stdout:Feb 17 13:09:28 smithi119 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.10.service: Consumed 3min 18.000s CPU time. 2024-02-17T13:09:28.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:09:28.605 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-02-17T13:09:28.606 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-02-17T13:09:28.606 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.11 2024-02-17T13:09:28.933 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:28 smithi146 systemd[1]: Stopping Ceph osd.11 for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:09:28.934 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:28 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:28.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:28.934 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:28 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:28.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:28.934 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:28 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:28.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:28.934 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:28 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:28.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:28.934 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:28 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:28.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:28.934 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:28 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:28.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:28.934 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:28 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:28.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-02-17T13:09:02.235707+0000 front 2024-02-17T13:09:02.235805+0000 (oldest deadline 2024-02-17T13:09:25.135681+0000) 2024-02-17T13:09:28.935 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:28 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:28.942+0000 7fdec582c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:09:28.935 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:28 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:28.942+0000 7fdec582c700 -1 osd.11 116 *** Got signal Terminated *** 2024-02-17T13:09:28.935 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:28 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:28.942+0000 7fdec582c700 -1 osd.11 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T13:09:30.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:29 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:29.854+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:30.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:29 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:29.854+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:30.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:29 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:29.854+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:30.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:29 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:29.854+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:30.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:29 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:29.854+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:30.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:29 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:29.854+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:30.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:29 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:29.854+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-02-17T13:09:02.235707+0000 front 2024-02-17T13:09:02.235805+0000 (oldest deadline 2024-02-17T13:09:25.135681+0000) 2024-02-17T13:09:31.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:30 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:30.878+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:31.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:30 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:30.878+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:31.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:30 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:30.878+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:31.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:30 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:30.878+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:31.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:30 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:30.878+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:31.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:30 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:30.878+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:31.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:30 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:30.878+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-02-17T13:09:02.235707+0000 front 2024-02-17T13:09:02.235805+0000 (oldest deadline 2024-02-17T13:09:25.135681+0000) 2024-02-17T13:09:32.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:31 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:31.916+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:32.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:31 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:31.916+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:32.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:31 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:31.916+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:32.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:31 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:31.916+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:32.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:31 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:31.916+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:32.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:31 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:31.916+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:32.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:31 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:31.916+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-02-17T13:09:02.235707+0000 front 2024-02-17T13:09:02.235805+0000 (oldest deadline 2024-02-17T13:09:25.135681+0000) 2024-02-17T13:09:33.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:32 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:32.903+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:33.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:32 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:32.903+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:33.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:32 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:32.903+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:33.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:32 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:32.903+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:33.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:32 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:32.903+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:33.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:32 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:32.903+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:33.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:32 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:32.903+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-02-17T13:09:02.235707+0000 front 2024-02-17T13:09:02.235805+0000 (oldest deadline 2024-02-17T13:09:25.135681+0000) 2024-02-17T13:09:34.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:33 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:33.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-02-17T13:08:04.724811+0000 front 2024-02-17T13:08:04.724787+0000 (oldest deadline 2024-02-17T13:08:26.424976+0000) 2024-02-17T13:09:34.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:33 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:33.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-02-17T13:08:09.926098+0000 front 2024-02-17T13:08:09.926166+0000 (oldest deadline 2024-02-17T13:08:35.826284+0000) 2024-02-17T13:09:34.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:33 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:33.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6804 osd.2 since back 2024-02-17T13:08:21.528349+0000 front 2024-02-17T13:08:21.528420+0000 (oldest deadline 2024-02-17T13:08:43.828415+0000) 2024-02-17T13:09:34.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:33 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:33.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-02-17T13:08:36.031381+0000 front 2024-02-17T13:08:36.031340+0000 (oldest deadline 2024-02-17T13:09:01.331427+0000) 2024-02-17T13:09:34.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:33 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:33.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.146:6812 osd.5 since back 2024-02-17T13:08:45.933035+0000 front 2024-02-17T13:08:45.932999+0000 (oldest deadline 2024-02-17T13:09:08.233174+0000) 2024-02-17T13:09:34.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:33 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:33.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-02-17T13:08:53.534206+0000 front 2024-02-17T13:08:53.534185+0000 (oldest deadline 2024-02-17T13:09:15.234192+0000) 2024-02-17T13:09:34.200 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:33 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11[62270]: 2024-02-17T13:09:33.861+0000 7fdec2621700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-02-17T13:09:02.235707+0000 front 2024-02-17T13:09:02.235805+0000 (oldest deadline 2024-02-17T13:09:25.135681+0000) 2024-02-17T13:09:34.592 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:34 smithi146 podman[77337]: 2024-02-17 13:09:34.316366218 +0000 UTC m=+5.498840940 container died ad3f44772022292ef81f5e87858048593dc07cd554d0ed98f71ede3d0e987cf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-17T13:09:34.911 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:34 smithi146 podman[77337]: 2024-02-17 13:09:34.694209785 +0000 UTC m=+5.876684500 container cleanup ad3f44772022292ef81f5e87858048593dc07cd554d0ed98f71ede3d0e987cf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b) 2024-02-17T13:09:34.911 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:34 smithi146 bash[77337]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11 2024-02-17T13:09:35.199 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:34 smithi146 podman[77357]: 2024-02-17 13:09:34.921161096 +0000 UTC m=+0.604699207 container remove ad3f44772022292ef81f5e87858048593dc07cd554d0ed98f71ede3d0e987cf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-17T13:09:35.523 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:35 smithi146 podman[77445]: 2024-02-17 13:09:35.239145467 +0000 UTC m=+0.106070087 container create 46e4f5e7ec04a7fcbfb5fdc69b402fb265f9bcb8e46b0f002493492d60c05386 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux ) 2024-02-17T13:09:35.523 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:35 smithi146 podman[77445]: 2024-02-17 13:09:35.150925101 +0000 UTC m=+0.017849697 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T13:09:35.523 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:35 smithi146 podman[77445]: 2024-02-17 13:09:35.449771307 +0000 UTC m=+0.316695893 container init 46e4f5e7ec04a7fcbfb5fdc69b402fb265f9bcb8e46b0f002493492d60c05386 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-02-17T13:09:35.523 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:35 smithi146 podman[77445]: 2024-02-17 13:09:35.452974342 +0000 UTC m=+0.319898924 container start 46e4f5e7ec04a7fcbfb5fdc69b402fb265f9bcb8e46b0f002493492d60c05386 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:09:35.523 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:35 smithi146 podman[77445]: 2024-02-17 13:09:35.532819933 +0000 UTC m=+0.399744533 container attach 46e4f5e7ec04a7fcbfb5fdc69b402fb265f9bcb8e46b0f002493492d60c05386 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T13:09:35.949 INFO:journalctl@ceph.osd.11.smithi146.stdout:Feb 17 13:09:35 smithi146 podman[77445]: 2024-02-17 13:09:35.743692869 +0000 UTC m=+0.610617455 container died 46e4f5e7ec04a7fcbfb5fdc69b402fb265f9bcb8e46b0f002493492d60c05386 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-17T13:09:36.315 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@osd.11.service' 2024-02-17T13:09:36.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:09:36.355 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-02-17T13:09:36.356 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-02-17T13:09:36.356 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.a 2024-02-17T13:09:36.646 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 13:09:36 smithi018 systemd[1]: Stopping Ceph mds.a for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:09:37.023 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 13:09:36 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-a[74545]: 2024-02-17T13:09:36.654+0000 7f6fdd6e6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:09:37.023 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 13:09:36 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-a[74545]: 2024-02-17T13:09:36.654+0000 7f6fdd6e6700 -1 mds.a *** got signal Terminated *** 2024-02-17T13:09:40.921 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 13:09:40 smithi018 podman[113121]: 2024-02-17 13:09:40.666365972 +0000 UTC m=+4.100864910 container died 49bad5597965c24f8e1c58ce7d64aa19fb61a5aeab0f59225f146f84c3b1de02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-17T13:09:41.273 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 13:09:41 smithi018 podman[113121]: 2024-02-17 13:09:41.021705476 +0000 UTC m=+4.456204450 container cleanup 49bad5597965c24f8e1c58ce7d64aa19fb61a5aeab0f59225f146f84c3b1de02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-17T13:09:41.273 INFO:journalctl@ceph.mds.a.smithi018.stdout:Feb 17 13:09:41 smithi018 bash[113121]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-a 2024-02-17T13:09:41.591 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.a.service' 2024-02-17T13:09:41.627 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:09:41.628 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-02-17T13:09:41.628 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-02-17T13:09:41.628 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.d 2024-02-17T13:09:42.023 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 13:09:41 smithi018 systemd[1]: Stopping Ceph mds.d for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:09:42.023 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 13:09:41 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-d[73568]: 2024-02-17T13:09:41.948+0000 7faa45306700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:09:42.024 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 13:09:41 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-d[73568]: 2024-02-17T13:09:41.948+0000 7faa45306700 -1 mds.d *** got signal Terminated *** 2024-02-17T13:09:46.273 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 13:09:45 smithi018 podman[113242]: 2024-02-17 13:09:45.960494738 +0000 UTC m=+4.103874579 container died f1a3c63002a3f298b2f92426a5815d1ebfff4979cc5860491778bd0317bb4401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-d, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=main-6822c7b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:09:46.666 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 13:09:46 smithi018 podman[113242]: 2024-02-17 13:09:46.500162528 +0000 UTC m=+4.643542387 container cleanup f1a3c63002a3f298b2f92426a5815d1ebfff4979cc5860491778bd0317bb4401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T13:09:46.666 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 13:09:46 smithi018 bash[113242]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-d 2024-02-17T13:09:46.898 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.d.service' 2024-02-17T13:09:46.921 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 13:09:46 smithi018 podman[113256]: 2024-02-17 13:09:46.676180842 +0000 UTC m=+0.711127872 container remove f1a3c63002a3f298b2f92426a5815d1ebfff4979cc5860491778bd0317bb4401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:09:46.921 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 13:09:46 smithi018 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.d.service: Deactivated successfully. 2024-02-17T13:09:46.921 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 13:09:46 smithi018 systemd[1]: Stopped Ceph mds.d for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T13:09:46.922 INFO:journalctl@ceph.mds.d.smithi018.stdout:Feb 17 13:09:46 smithi018 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.d.service: Consumed 1.137s CPU time. 2024-02-17T13:09:46.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:09:46.932 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-02-17T13:09:46.933 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-02-17T13:09:46.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.g 2024-02-17T13:09:47.273 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 13:09:47 smithi018 systemd[1]: Stopping Ceph mds.g for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:09:47.773 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 13:09:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g[74164]: 2024-02-17T13:09:47.403+0000 7ff7742d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:09:47.773 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 13:09:47 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g[74164]: 2024-02-17T13:09:47.403+0000 7ff7742d4700 -1 mds.g *** got signal Terminated *** 2024-02-17T13:09:49.023 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 13:09:48 smithi018 podman[113363]: 2024-02-17 13:09:48.726263639 +0000 UTC m=+1.563096440 container died b924061d027f877776c491a7ec820dea1687e8a8bb6dfaa9fe38ff1399472678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T13:09:49.353 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 13:09:49 smithi018 podman[113363]: 2024-02-17 13:09:49.184413295 +0000 UTC m=+2.021246095 container cleanup b924061d027f877776c491a7ec820dea1687e8a8bb6dfaa9fe38ff1399472678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-17T13:09:49.353 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 13:09:49 smithi018 bash[113363]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g 2024-02-17T13:09:49.610 INFO:journalctl@ceph.mds.g.smithi018.stdout:Feb 17 13:09:49 smithi018 podman[113377]: 2024-02-17 13:09:49.363225731 +0000 UTC m=+0.635286026 container remove b924061d027f877776c491a7ec820dea1687e8a8bb6dfaa9fe38ff1399472678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-g, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-17T13:09:49.613 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.g.service' 2024-02-17T13:09:49.647 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:09:49.647 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-02-17T13:09:49.648 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-02-17T13:09:49.648 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.j 2024-02-17T13:09:49.981 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 13:09:49 smithi018 systemd[1]: Stopping Ceph mds.j for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:09:50.273 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 13:09:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-j[73217]: 2024-02-17T13:09:49.990+0000 7fc1672db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:09:50.273 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 13:09:49 smithi018 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-j[73217]: 2024-02-17T13:09:49.990+0000 7fc1672db700 -1 mds.j *** got signal Terminated *** 2024-02-17T13:09:51.973 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 13:09:51 smithi018 podman[113484]: 2024-02-17 13:09:51.68222705 +0000 UTC m=+1.802444289 container died 0514dd8cfd4ad6128f9131d8caf12ce024aece22e585152ac28fec3a2b0e0100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-j, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=main-6822c7b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-17T13:09:52.273 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 13:09:52 smithi018 podman[113484]: 2024-02-17 13:09:52.082356396 +0000 UTC m=+2.202573609 container cleanup 0514dd8cfd4ad6128f9131d8caf12ce024aece22e585152ac28fec3a2b0e0100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-j, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-17T13:09:52.273 INFO:journalctl@ceph.mds.j.smithi018.stdout:Feb 17 13:09:52 smithi018 bash[113484]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-j 2024-02-17T13:09:52.418 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.j.service' 2024-02-17T13:09:52.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:09:52.452 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-02-17T13:09:52.452 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-02-17T13:09:52.453 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.b 2024-02-17T13:09:52.758 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 13:09:52 smithi119 systemd[1]: Stopping Ceph mds.b for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:09:53.092 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 13:09:52 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b[67310]: 2024-02-17T13:09:52.767+0000 7f5dd2ebf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:09:53.092 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 13:09:52 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b[67310]: 2024-02-17T13:09:52.767+0000 7f5dd2ebf700 -1 mds.b *** got signal Terminated *** 2024-02-17T13:09:57.042 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 13:09:56 smithi119 podman[74814]: 2024-02-17 13:09:56.775665458 +0000 UTC m=+4.115204982 container died 5365242334d82ead07e8d572b8a18710b6fc389070628189fb083265d39c5435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b) 2024-02-17T13:09:57.342 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 13:09:57 smithi119 podman[74814]: 2024-02-17 13:09:57.15042363 +0000 UTC m=+4.489963150 container cleanup 5365242334d82ead07e8d572b8a18710b6fc389070628189fb083265d39c5435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-17T13:09:57.343 INFO:journalctl@ceph.mds.b.smithi119.stdout:Feb 17 13:09:57 smithi119 bash[74814]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-b 2024-02-17T13:09:57.545 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.b.service' 2024-02-17T13:09:57.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:09:57.584 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-02-17T13:09:57.584 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-02-17T13:09:57.584 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.e 2024-02-17T13:09:57.842 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 13:09:57 smithi119 systemd[1]: Stopping Ceph mds.e for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:09:58.342 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 13:09:57 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-e[66684]: 2024-02-17T13:09:57.945+0000 7fd974fc7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:09:58.342 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 13:09:57 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-e[66684]: 2024-02-17T13:09:57.945+0000 7fd974fc7700 -1 mds.e *** got signal Terminated *** 2024-02-17T13:10:01.699 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 13:10:01 smithi119 podman[74947]: 2024-02-17 13:10:01.439195511 +0000 UTC m=+3.629289899 container died 2e488f45841efc458b33320995459fd6163fda94e443733212abadb9399fe098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-e, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-17T13:10:02.092 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 13:10:01 smithi119 podman[74947]: 2024-02-17 13:10:01.804871935 +0000 UTC m=+3.994966296 container cleanup 2e488f45841efc458b33320995459fd6163fda94e443733212abadb9399fe098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-17T13:10:02.093 INFO:journalctl@ceph.mds.e.smithi119.stdout:Feb 17 13:10:01 smithi119 bash[74947]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-e 2024-02-17T13:10:02.327 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.e.service' 2024-02-17T13:10:02.364 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:10:02.364 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-02-17T13:10:02.364 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-02-17T13:10:02.365 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.h 2024-02-17T13:10:02.592 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:10:02 smithi119 systemd[1]: Stopping Ceph mds.h for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:10:03.092 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:10:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:10:02.699+0000 7f38cae86700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:10:03.092 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:10:02 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h[66988]: 2024-02-17T13:10:02.699+0000 7f38cae86700 -1 mds.h *** got signal Terminated *** 2024-02-17T13:10:05.674 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:10:05 smithi119 podman[75070]: 2024-02-17 13:10:05.412553302 +0000 UTC m=+2.824585521 container died a6a555ffabe7369da98155cd093947b26b39a5be8621ec8e281091dece695312 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T13:10:06.041 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:10:05 smithi119 podman[75070]: 2024-02-17 13:10:05.783309564 +0000 UTC m=+3.195341792 container cleanup a6a555ffabe7369da98155cd093947b26b39a5be8621ec8e281091dece695312 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:10:06.041 INFO:journalctl@ceph.mds.h.smithi119.stdout:Feb 17 13:10:05 smithi119 bash[75070]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-h 2024-02-17T13:10:06.195 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.h.service' 2024-02-17T13:10:06.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:10:06.230 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-02-17T13:10:06.231 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-02-17T13:10:06.231 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.k 2024-02-17T13:10:06.592 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 13:10:06 smithi119 systemd[1]: Stopping Ceph mds.k for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:10:06.593 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 13:10:06 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k[66361]: 2024-02-17T13:10:06.535+0000 7fb13fc57700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:10:06.593 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 13:10:06 smithi119 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k[66361]: 2024-02-17T13:10:06.535+0000 7fb13fc57700 -1 mds.k *** got signal Terminated *** 2024-02-17T13:10:09.727 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 13:10:09 smithi119 podman[75192]: 2024-02-17 13:10:09.41521993 +0000 UTC m=+2.959847320 container died 16c69fa7b6ed4011ace77e8c630d6cc6d051610a60a6ff51921ffc8991f93d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-17T13:10:09.727 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 13:10:09 smithi119 podman[75192]: 2024-02-17 13:10:09.736413099 +0000 UTC m=+3.281040489 container cleanup 16c69fa7b6ed4011ace77e8c630d6cc6d051610a60a6ff51921ffc8991f93d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-17T13:10:10.092 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 13:10:09 smithi119 bash[75192]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k 2024-02-17T13:10:10.092 INFO:journalctl@ceph.mds.k.smithi119.stdout:Feb 17 13:10:09 smithi119 podman[75206]: 2024-02-17 13:10:09.904079904 +0000 UTC m=+0.485013112 container remove 16c69fa7b6ed4011ace77e8c630d6cc6d051610a60a6ff51921ffc8991f93d33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-k, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-17T13:10:10.115 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.k.service' 2024-02-17T13:10:10.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:10:10.149 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-02-17T13:10:10.149 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-02-17T13:10:10.149 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.c 2024-02-17T13:10:10.450 INFO:journalctl@ceph.mds.c.smithi146.stdout:Feb 17 13:10:10 smithi146 systemd[1]: Stopping Ceph mds.c for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:10:10.711 INFO:journalctl@ceph.mds.c.smithi146.stdout:Feb 17 13:10:10 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-c[67349]: 2024-02-17T13:10:10.550+0000 7f3dca0a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:10:10.711 INFO:journalctl@ceph.mds.c.smithi146.stdout:Feb 17 13:10:10 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-c[67349]: 2024-02-17T13:10:10.550+0000 7f3dca0a2700 -1 mds.c *** got signal Terminated *** 2024-02-17T13:10:13.001 INFO:journalctl@ceph.mds.c.smithi146.stdout:Feb 17 13:10:12 smithi146 podman[77550]: 2024-02-17 13:10:12.723428916 +0000 UTC m=+2.371951791 container died bcf9fbbd17ff9bc23257588dc2c007dab41271004e16674ae583e1e90ef4538e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-02-17T13:10:13.342 INFO:journalctl@ceph.mds.c.smithi146.stdout:Feb 17 13:10:13 smithi146 podman[77550]: 2024-02-17 13:10:13.092787752 +0000 UTC m=+2.741310638 container cleanup bcf9fbbd17ff9bc23257588dc2c007dab41271004e16674ae583e1e90ef4538e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-17T13:10:13.343 INFO:journalctl@ceph.mds.c.smithi146.stdout:Feb 17 13:10:13 smithi146 bash[77550]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-c 2024-02-17T13:10:13.501 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.c.service' 2024-02-17T13:10:13.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:10:13.540 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-02-17T13:10:13.541 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-02-17T13:10:13.541 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.f 2024-02-17T13:10:13.888 INFO:journalctl@ceph.mds.f.smithi146.stdout:Feb 17 13:10:13 smithi146 systemd[1]: Stopping Ceph mds.f for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:10:14.200 INFO:journalctl@ceph.mds.f.smithi146.stdout:Feb 17 13:10:13 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-f[67032]: 2024-02-17T13:10:13.895+0000 7f1ecffcb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:10:14.200 INFO:journalctl@ceph.mds.f.smithi146.stdout:Feb 17 13:10:13 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-f[67032]: 2024-02-17T13:10:13.895+0000 7f1ecffcb700 -1 mds.f *** got signal Terminated *** 2024-02-17T13:10:16.980 INFO:journalctl@ceph.mds.f.smithi146.stdout:Feb 17 13:10:16 smithi146 podman[77682]: 2024-02-17 13:10:16.734759423 +0000 UTC m=+2.967222862 container died da48079880bd953f14d04cc6033dc9d3663598ff6dff2e6585f561aca9a89511 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-17T13:10:17.248 INFO:journalctl@ceph.mds.f.smithi146.stdout:Feb 17 13:10:17 smithi146 podman[77682]: 2024-02-17 13:10:17.047859456 +0000 UTC m=+3.280322859 container cleanup da48079880bd953f14d04cc6033dc9d3663598ff6dff2e6585f561aca9a89511 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-f, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-17T13:10:17.248 INFO:journalctl@ceph.mds.f.smithi146.stdout:Feb 17 13:10:17 smithi146 bash[77682]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-f 2024-02-17T13:10:17.466 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.f.service' 2024-02-17T13:10:17.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:10:17.503 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-02-17T13:10:17.503 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-02-17T13:10:17.503 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.i 2024-02-17T13:10:17.817 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:10:17 smithi146 systemd[1]: Stopping Ceph mds.i for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:10:18.200 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:10:17 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i[67657]: 2024-02-17T13:10:17.825+0000 7f1e6cb3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:10:18.200 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:10:17 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i[67657]: 2024-02-17T13:10:17.825+0000 7f1e6cb3f700 -1 mds.i *** got signal Terminated *** 2024-02-17T13:10:21.059 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:10:20 smithi146 podman[77804]: 2024-02-17 13:10:20.723554604 +0000 UTC m=+2.991212847 container died db4ba8766bbe2406f15b55236bade6fa9f44868916caee9a08bacc9e811897f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T13:10:21.334 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:10:21 smithi146 podman[77804]: 2024-02-17 13:10:21.150724888 +0000 UTC m=+3.418383126 container cleanup db4ba8766bbe2406f15b55236bade6fa9f44868916caee9a08bacc9e811897f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:10:21.334 INFO:journalctl@ceph.mds.i.smithi146.stdout:Feb 17 13:10:21 smithi146 bash[77804]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-i 2024-02-17T13:10:21.554 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.i.service' 2024-02-17T13:10:21.589 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:10:21.590 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-02-17T13:10:21.590 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-02-17T13:10:21.590 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.l 2024-02-17T13:10:21.912 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:10:21 smithi146 systemd[1]: Stopping Ceph mds.l for 7fca71fe-cd91-11ee-95ba-87774f69a715... 2024-02-17T13:10:22.200 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:10:21 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l[66721]: 2024-02-17T13:10:21.919+0000 7fbbf90a9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T13:10:22.200 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:10:21 smithi146 ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l[66721]: 2024-02-17T13:10:21.919+0000 7fbbf90a9700 -1 mds.l *** got signal Terminated *** 2024-02-17T13:10:25.070 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:10:24 smithi146 podman[77923]: 2024-02-17 13:10:24.729059334 +0000 UTC m=+2.906735160 container died 79ede5d44042f490af8a075e46db7a0b0287851a86669ae79b72a16eda966e58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-17T13:10:25.330 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:10:25 smithi146 podman[77923]: 2024-02-17 13:10:25.153867241 +0000 UTC m=+3.331543066 container cleanup 79ede5d44042f490af8a075e46db7a0b0287851a86669ae79b72a16eda966e58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-17T13:10:25.330 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:10:25 smithi146 bash[77923]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l 2024-02-17T13:10:25.569 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.l.service' 2024-02-17T13:10:25.592 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:10:25 smithi146 podman[77938]: 2024-02-17 13:10:25.339065156 +0000 UTC m=+0.602196277 container remove 79ede5d44042f490af8a075e46db7a0b0287851a86669ae79b72a16eda966e58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-7fca71fe-cd91-11ee-95ba-87774f69a715-mds-l, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T13:10:25.592 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:10:25 smithi146 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.l.service: Deactivated successfully. 2024-02-17T13:10:25.592 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:10:25 smithi146 systemd[1]: Stopped Ceph mds.l for 7fca71fe-cd91-11ee-95ba-87774f69a715. 2024-02-17T13:10:25.593 INFO:journalctl@ceph.mds.l.smithi146.stdout:Feb 17 13:10:25 smithi146 systemd[1]: ceph-7fca71fe-cd91-11ee-95ba-87774f69a715@mds.l.service: Consumed 16.828s CPU time. 2024-02-17T13:10:25.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T13:10:25.603 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-02-17T13:10:25.603 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-17T13:10:25.803 INFO:teuthology.orchestra.run.smithi018.stdout:Deleting cluster with fsid: 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T13:10:28.225 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-17T13:10:28.417 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T13:10:31.002 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-17T13:10:31.192 INFO:teuthology.orchestra.run.smithi146.stdout:Deleting cluster with fsid: 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T13:10:33.661 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-17T13:10:33.697 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-17T13:10:33.732 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-17T13:10:33.761 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-17T13:10:33.763 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187/remote/smithi018/crash 2024-02-17T13:10:33.764 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/crash -- . 2024-02-17T13:10:33.806 INFO:teuthology.orchestra.run.smithi018.stderr:tar: /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-17T13:10:33.807 INFO:teuthology.orchestra.run.smithi018.stderr:tar: Error is not recoverable: exiting now 2024-02-17T13:10:33.808 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187/remote/smithi119/crash 2024-02-17T13:10:33.809 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/crash -- . 2024-02-17T13:10:33.854 INFO:teuthology.orchestra.run.smithi119.stderr:tar: /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-17T13:10:33.854 INFO:teuthology.orchestra.run.smithi119.stderr:tar: Error is not recoverable: exiting now 2024-02-17T13:10:33.856 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187/remote/smithi146/crash 2024-02-17T13:10:33.857 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/crash -- . 2024-02-17T13:10:33.897 INFO:teuthology.orchestra.run.smithi146.stderr:tar: /var/lib/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-17T13:10:33.897 INFO:teuthology.orchestra.run.smithi146.stderr:tar: Error is not recoverable: exiting now 2024-02-17T13:10:33.898 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-17T13:10:33.899 DEBUG:teuthology.orchestra.run.smithi018:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-02-17T13:10:33.933 INFO:tasks.cephadm:Compressing logs... 2024-02-17T13:10:33.934 DEBUG:teuthology.orchestra.run.smithi018:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-17T13:10:33.976 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-17T13:10:33.979 DEBUG:teuthology.orchestra.run.smithi146:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-17T13:10:34.002 INFO:teuthology.orchestra.run.smithi018.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-17T13:10:34.003 INFO:teuthology.orchestra.run.smithi018.stderr:: No such file or directory 2024-02-17T13:10:34.003 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.6.log 2024-02-17T13:10:34.003 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mon.a.log 2024-02-17T13:10:34.004 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-17T13:10:34.005 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.6.log: /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.g.log 2024-02-17T13:10:34.005 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.j.log 2024-02-17T13:10:34.005 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.audit.log: /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.3.log 2024-02-17T13:10:34.008 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mgr.x.log 2024-02-17T13:10:34.008 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.0.log 2024-02-17T13:10:34.008 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mgr.x.log: 90.1% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-17T13:10:34.011 INFO:teuthology.orchestra.run.smithi018.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-17T13:10:34.012 INFO:teuthology.orchestra.run.smithi018.stderr: 94.6% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.j.log.gz 2024-02-17T13:10:34.013 INFO:teuthology.orchestra.run.smithi146.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-17T13:10:34.013 INFO:teuthology.orchestra.run.smithi146.stderr:: No such file or directory 2024-02-17T13:10:34.013 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.l.log 2024-02-17T13:10:34.013 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-17T13:10:34.014 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.i.log 2024-02-17T13:10:34.014 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.l.log: /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.11.log 2024-02-17T13:10:34.014 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mgr.z.log 2024-02-17T13:10:34.014 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.8.log 2024-02-17T13:10:34.014 INFO:teuthology.orchestra.run.smithi146.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-17T13:10:34.014 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.5.log 2024-02-17T13:10:34.014 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.f.log 2024-02-17T13:10:34.014 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-17T13:10:34.015 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-17T13:10:34.015 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.log 2024-02-17T13:10:34.015 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-17T13:10:34.015 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.2.log 2024-02-17T13:10:34.019 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.log: 87.8% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-17T13:10:34.020 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.c.log 2024-02-17T13:10:34.020 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.2.log: 92.3% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mgr.z.log.gz 2024-02-17T13:10:34.020 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-17T13:10:34.023 INFO:teuthology.orchestra.run.smithi119.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-17T13:10:34.023 INFO:teuthology.orchestra.run.smithi119.stderr:: No such file or directory 2024-02-17T13:10:34.023 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.10.log 2024-02-17T13:10:34.023 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-17T13:10:34.023 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.b.log 2024-02-17T13:10:34.023 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.4.log 2024-02-17T13:10:34.024 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.1.log 2024-02-17T13:10:34.024 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.k.log 2024-02-17T13:10:34.024 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.h.log 2024-02-17T13:10:34.024 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.7.log 2024-02-17T13:10:34.024 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.e.log 2024-02-17T13:10:34.024 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mgr.y.log 2024-02-17T13:10:34.024 INFO:teuthology.orchestra.run.smithi119.stderr:gzip/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.e.log: -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-17T13:10:34.025 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.log 2024-02-17T13:10:34.025 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-17T13:10:34.025 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mon.b.log 2024-02-17T13:10:34.027 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-17T13:10:34.028 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.log 2024-02-17T13:10:34.028 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-17T13:10:34.028 INFO:teuthology.orchestra.run.smithi018.stderr: 84.0% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-17T13:10:34.028 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.log: 87.8% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-17T13:10:34.028 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.9.log 2024-02-17T13:10:34.028 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.d.log 2024-02-17T13:10:34.029 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-volume.log: /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mgr.y.log.gz 2024-02-17T13:10:34.031 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mon.c.log 2024-02-17T13:10:34.040 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.9.log: 91.0% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-17T13:10:34.040 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.a.log 2024-02-17T13:10:34.047 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.d.log.gz 2024-02-17T13:10:34.055 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-volume.log: /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-17T13:10:34.132 INFO:teuthology.orchestra.run.smithi119.stderr: 83.7% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-17T13:10:34.145 INFO:teuthology.orchestra.run.smithi119.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-17T13:10:34.153 INFO:teuthology.orchestra.run.smithi119.stderr: 90.3% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-17T13:10:34.164 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-17T13:10:34.448 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.a.log: 89.9% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mgr.x.log.gz 2024-02-17T13:10:34.505 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-17T13:10:34.721 INFO:teuthology.orchestra.run.smithi146.stderr: 91.9% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mon.c.log.gz 2024-02-17T13:10:36.127 INFO:teuthology.orchestra.run.smithi018.stderr: 92.2% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mon.a.log.gz 2024-02-17T13:10:37.458 INFO:teuthology.orchestra.run.smithi119.stderr: 91.9% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mon.b.log.gz 2024-02-17T13:10:38.006 INFO:teuthology.orchestra.run.smithi018.stderr: 95.1% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.g.log.gz 2024-02-17T13:10:38.543 INFO:teuthology.orchestra.run.smithi018.stderr: 95.2% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.a.log.gz 2024-02-17T13:10:39.304 INFO:teuthology.orchestra.run.smithi146.stderr: 95.2% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.l.log.gz 2024-02-17T13:10:40.228 INFO:teuthology.orchestra.run.smithi146.stderr: 95.1% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.c.log.gz 2024-02-17T13:10:42.068 INFO:teuthology.orchestra.run.smithi146.stderr: 95.1% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.i.log.gz 2024-02-17T13:10:45.192 INFO:teuthology.orchestra.run.smithi018.stderr: 94.0% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.9.log.gz 2024-02-17T13:10:46.128 INFO:teuthology.orchestra.run.smithi018.stderr: 93.9% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.6.log.gz 2024-02-17T13:10:46.335 INFO:teuthology.orchestra.run.smithi018.stderr: 93.8% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.0.log.gz 2024-02-17T13:10:48.408 INFO:teuthology.orchestra.run.smithi146.stderr: 93.9% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.11.log.gz 2024-02-17T13:10:48.931 INFO:teuthology.orchestra.run.smithi146.stderr: 94.0% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.8.log.gz 2024-02-17T13:10:49.735 INFO:teuthology.orchestra.run.smithi018.stderr: 93.8% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.3.log.gz 2024-02-17T13:10:49.737 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-02-17T13:10:49.737 INFO:teuthology.orchestra.run.smithi018.stderr:real 0m15.747s 2024-02-17T13:10:49.737 INFO:teuthology.orchestra.run.smithi018.stderr:user 1m0.405s 2024-02-17T13:10:49.737 INFO:teuthology.orchestra.run.smithi018.stderr:sys 0m1.947s 2024-02-17T13:10:51.044 INFO:teuthology.orchestra.run.smithi146.stderr: 93.9% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.5.log.gz 2024-02-17T13:10:52.202 INFO:teuthology.orchestra.run.smithi146.stderr: 93.9% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.2.log.gz 2024-02-17T13:11:50.454 INFO:teuthology.orchestra.run.smithi146.stderr: 93.2% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.f.log.gz 2024-02-17T13:11:50.456 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-02-17T13:11:50.456 INFO:teuthology.orchestra.run.smithi146.stderr:real 1m16.461s 2024-02-17T13:11:50.456 INFO:teuthology.orchestra.run.smithi146.stderr:user 2m35.271s 2024-02-17T13:11:50.456 INFO:teuthology.orchestra.run.smithi146.stderr:sys 0m4.989s 2024-02-17T13:12:13.706 INFO:teuthology.orchestra.run.smithi119.stderr: 94.0% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.4.log.gz 2024-02-17T13:12:20.205 INFO:teuthology.orchestra.run.smithi119.stderr: 93.8% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.1.log.gz 2024-02-17T13:12:51.697 INFO:teuthology.orchestra.run.smithi119.stderr: 93.9% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.10.log.gz 2024-02-17T13:13:11.866 INFO:teuthology.orchestra.run.smithi119.stderr: 93.9% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-osd.7.log.gz 2024-02-17T13:17:11.170 INFO:teuthology.orchestra.run.smithi119.stderr: 93.3% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.e.log.gz 2024-02-17T13:18:38.447 INFO:teuthology.orchestra.run.smithi119.stderr: 93.1% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.k.log.gz 2024-02-17T13:19:56.202 INFO:teuthology.orchestra.run.smithi119.stderr: 93.1% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.h.log.gz 2024-02-17T13:21:06.307 INFO:teuthology.orchestra.run.smithi119.stderr: 93.3% -- replaced with /var/log/ceph/7fca71fe-cd91-11ee-95ba-87774f69a715/ceph-mds.b.log.gz 2024-02-17T13:21:06.309 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-02-17T13:21:06.309 INFO:teuthology.orchestra.run.smithi119.stderr:real 10m32.308s 2024-02-17T13:21:06.309 INFO:teuthology.orchestra.run.smithi119.stderr:user 6m33.382s 2024-02-17T13:21:06.309 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m17.008s 2024-02-17T13:21:06.310 INFO:tasks.cephadm:Archiving logs... 2024-02-17T13:21:06.311 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187/remote/smithi018/log 2024-02-17T13:21:06.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-17T13:21:14.110 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187/remote/smithi119/log 2024-02-17T13:21:14.112 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-17T13:22:17.635 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187/remote/smithi146/log 2024-02-17T13:22:17.637 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-17T13:22:38.219 INFO:tasks.cephadm:Removing cluster... 2024-02-17T13:22:38.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 --force 2024-02-17T13:22:38.418 INFO:teuthology.orchestra.run.smithi018.stdout:Deleting cluster with fsid: 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T13:22:38.743 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 --force 2024-02-17T13:22:39.011 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T13:22:39.715 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7fca71fe-cd91-11ee-95ba-87774f69a715 --force 2024-02-17T13:22:39.909 INFO:teuthology.orchestra.run.smithi146.stdout:Deleting cluster with fsid: 7fca71fe-cd91-11ee-95ba-87774f69a715 2024-02-17T13:22:40.386 INFO:tasks.cephadm:Removing cephadm ... 2024-02-17T13:22:40.387 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-17T13:22:40.406 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-17T13:22:40.422 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-17T13:22:40.437 INFO:tasks.cephadm:Teardown complete 2024-02-17T13:22:40.438 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-17T13:22:40.450 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-17T13:22:40.451 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-17T13:22:40.454 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-17T13:22:40.466 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-17T13:22:40.579 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-17T13:22:40.592 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-17T13:22:40.592 DEBUG:teuthology.orchestra.run.smithi018:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-17T13:22:40.595 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-17T13:22:40.597 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-17T13:22:40.608 INFO:teuthology.orchestra.run.smithi018.stderr:bash: line 1: ntpq: command not found 2024-02-17T13:22:40.609 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-02-17T13:22:40.611 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-02-17T13:22:40.614 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-17T13:22:40.614 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2024-02-17T13:22:40.614 INFO:teuthology.orchestra.run.smithi018.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 +4609us[+4633us] +/- 121ms 2024-02-17T13:22:40.614 INFO:teuthology.orchestra.run.smithi018.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 -1607us[-1583us] +/- 74ms 2024-02-17T13:22:40.614 INFO:teuthology.orchestra.run.smithi018.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -1382us[-1358us] +/- 91ms 2024-02-17T13:22:40.614 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T13:22:40.615 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-17T13:22:40.615 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-02-17T13:22:40.615 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 51 +4595us[+4607us] +/- 121ms 2024-02-17T13:22:40.615 INFO:teuthology.orchestra.run.smithi146.stdout:^* hv02.front.sepia.ceph.com 3 6 377 51 -1606us[-1594us] +/- 74ms 2024-02-17T13:22:40.615 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 -1384us[-1384us] +/- 91ms 2024-02-17T13:22:40.616 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T13:22:40.706 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-17T13:22:40.707 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-02-17T13:22:40.707 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 52 +4596us[+4596us] +/- 121ms 2024-02-17T13:22:40.707 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -1601us[-1600us] +/- 74ms 2024-02-17T13:22:40.707 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 -1358us[-1358us] +/- 91ms 2024-02-17T13:22:40.707 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T13:22:40.708 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-17T13:22:40.720 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-17T13:22:40.720 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-17T13:22:40.740 DEBUG:teuthology.orchestra.run.smithi018:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-17T13:22:40.832 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-17T13:22:40.927 DEBUG:teuthology.orchestra.run.smithi146:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-17T13:22:41.001 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-17T13:22:41.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T13:22:41.035 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-17T13:22:41.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T13:22:41.067 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-17T13:22:41.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T13:22:41.098 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-17T13:22:41.110 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-17T13:22:41.148 INFO:teuthology.task.internal:Duration was 3069.742994 seconds 2024-02-17T13:22:41.148 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-17T13:22:41.183 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-17T13:22:41.183 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-17T13:22:41.186 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-17T13:22:41.189 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-17T13:22:41.230 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-17T13:22:41.232 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-17T13:22:41.251 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-17T13:22:41.738 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-17T13:22:41.738 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi018.front.sepia.ceph.com 2024-02-17T13:22:41.739 DEBUG:teuthology.orchestra.run.smithi018:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-17T13:22:41.761 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-02-17T13:22:41.761 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-17T13:22:41.795 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2024-02-17T13:22:41.796 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-17T13:22:41.827 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-17T13:22:41.827 DEBUG:teuthology.orchestra.run.smithi018:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-17T13:22:41.830 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-17T13:22:41.838 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-17T13:22:41.896 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-17T13:22:41.896 DEBUG:teuthology.orchestra.run.smithi018:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-17T13:22:41.902 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-17T13:22:41.916 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-17T13:22:42.350 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-17T13:22:42.363 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-17T13:22:42.363 DEBUG:teuthology.orchestra.run.smithi018:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-17T13:22:42.391 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-17T13:22:42.443 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-17T13:22:42.471 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-17T13:22:42.483 DEBUG:teuthology.orchestra.run.smithi018:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-17T13:22:42.486 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-17T13:22:42.488 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-17T13:22:42.508 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = core 2024-02-17T13:22:42.530 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-02-17T13:22:42.536 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2024-02-17T13:22:42.560 DEBUG:teuthology.orchestra.run.smithi018:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-17T13:22:42.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T13:22:42.578 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-17T13:22:42.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T13:22:42.610 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-17T13:22:42.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T13:22:42.624 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-17T13:22:42.636 INFO:teuthology.task.internal:Transferring archived files... 2024-02-17T13:22:42.637 DEBUG:teuthology.misc:Transferring archived files from smithi018:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187/remote/smithi018 2024-02-17T13:22:42.637 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-17T13:22:42.682 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187/remote/smithi119 2024-02-17T13:22:42.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-17T13:22:42.722 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187/remote/smithi146 2024-02-17T13:22:42.722 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-17T13:22:42.764 INFO:teuthology.task.internal:Removing archive directory... 2024-02-17T13:22:42.764 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-17T13:22:42.767 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-17T13:22:42.769 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-17T13:22:42.813 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-17T13:22:42.828 INFO:teuthology.task.internal:Not uploading archives. 2024-02-17T13:22:42.828 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-17T13:22:42.848 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-17T13:22:42.849 DEBUG:teuthology.orchestra.run.smithi018:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-17T13:22:42.851 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-17T13:22:42.853 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-17T13:22:42.863 INFO:teuthology.orchestra.run.smithi018.stdout: 265584 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 13:22 /home/ubuntu/cephtest 2024-02-17T13:22:42.865 INFO:teuthology.orchestra.run.smithi119.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 13:22 /home/ubuntu/cephtest 2024-02-17T13:22:42.867 INFO:teuthology.orchestra.run.smithi146.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 13:22 /home/ubuntu/cephtest 2024-02-17T13:22:42.868 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-17T13:22:42.882 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-17T13:22:42.948 INFO:teuthology.nuke:Checking targets against current locks 2024-02-17T13:22:42.975 DEBUG:teuthology.nuke:shortname: smithi018 2024-02-17T13:22:42.976 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-17T13:22:43.001 DEBUG:teuthology.nuke:shortname: smithi119 2024-02-17T13:22:43.001 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-17T13:22:43.007 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi018.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 12:23:24.230014', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLObfZd/saF9BH2sXCdZad6yo1Qc46W/b4Kp+fhPkbWbn7NvGrcngqhUDdBXoehoNgFLsAotJRGuzcsZvsROwTo='} 2024-02-17T13:22:43.024 DEBUG:teuthology.nuke:shortname: smithi146 2024-02-17T13:22:43.024 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-17T13:22:43.028 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 12:23:24.228797', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE4Pik3gsxWbyQqQD788uEvHLv4KWovWW+R9FZ+VPWNTAqiu1BIcBFv6sAJeLMEEw9RNDh8rxoWubGCgVrPjfHE='} 2024-02-17T13:22:43.048 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563187', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 12:23:24.231077', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYHYO1lPAwXnyp9sfjwl3DRRzg3MVYz/LkR3WLLC0DQtZKNq+tc8V8bJI97n4Xsjj4miJ9KNYZwEAtf8PnfP7I='} 2024-02-17T13:22:43.058 INFO:teuthology.orchestra.console.smithi018:Power off 2024-02-17T13:22:43.059 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-17T13:22:43.086 DEBUG:teuthology.orchestra.console.smithi018:power off output: Chassis Power Control: Down/Off 2024-02-17T13:22:43.087 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T13:22:43.107 INFO:teuthology.orchestra.console.smithi119:Power off 2024-02-17T13:22:43.108 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-17T13:22:43.124 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is on 2024-02-17T13:22:43.132 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2024-02-17T13:22:43.132 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T13:22:43.157 INFO:teuthology.orchestra.console.smithi146:Power off 2024-02-17T13:22:43.157 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-17T13:22:43.171 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-02-17T13:22:43.185 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off 2024-02-17T13:22:43.185 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T13:22:43.210 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2024-02-17T13:22:47.125 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T13:22:47.173 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T13:22:47.211 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T13:22:47.237 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is on 2024-02-17T13:22:47.287 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-02-17T13:22:47.323 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2024-02-17T13:22:51.240 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T13:22:51.288 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T13:22:51.325 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T13:22:51.354 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is off 2024-02-17T13:22:51.399 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-02-17T13:22:51.437 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2024-02-17T13:22:51.455 INFO:teuthology.orchestra.console.smithi018:Power off completed 2024-02-17T13:22:55.404 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T13:22:55.438 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T13:22:55.514 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2024-02-17T13:22:55.549 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2024-02-17T13:22:55.616 INFO:teuthology.orchestra.console.smithi119:Power off completed 2024-02-17T13:22:55.650 INFO:teuthology.orchestra.console.smithi146:Power off completed 2024-02-17T13:22:55.756 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 3069.742993593216 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi018 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6822c7b54e949006be7a0802650d01723d834d1d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=3c0730b0578a473eb855034826bbffc6 status: fail success: false 2024-02-17T13:22:55.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-17T13:22:55.846 INFO:teuthology.run:FAIL